Skip to content

evals: load eval prompts from disk#1458

Draft
wgqqqqq wants to merge 2 commits into
GCWing:evals-on-releasefrom
wgqqqqq:codex/evals-prompt-runtime
Draft

evals: load eval prompts from disk#1458
wgqqqqq wants to merge 2 commits into
GCWing:evals-on-releasefrom
wgqqqqq:codex/evals-prompt-runtime

Conversation

@wgqqqqq

@wgqqqqq wgqqqqq commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add runtime prompt loading for CLI/eval prompts via --prompts-dir and BITFUN_PROMPTS_DIR
  • include runtime prompt file fingerprints in prompt cache identity
  • add full model exchange tracing override for request-body validation

Verification

  • cargo test -p bitfun-core runtime_prompt_override_reaches_anthropic_request_body_system_field -- --nocapture
  • cargo check -p bitfun-cli
  • cargo test -p bitfun-cli prompts -- --nocapture
  • git diff --check

Manual validation

  • Ran bitfun-cli with /tmp/bitfun-prompt-test/agentic_mode.md and verified the Anthropic request body top-level system field was overridden in the request trace.

@wgqqqqq wgqqqqq changed the title feat(cli): load eval prompts from disk evals: load eval prompts from disk Jul 8, 2026
@wgqqqqq wgqqqqq marked this pull request as draft July 8, 2026 03:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant