JVNAUTOSCI-2596: freeze disposable read-only A1 candidate#307
Draft
witbrock wants to merge 2 commits into
Draft
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a disposable, non-production “thin read-only turn” A1 candidate for JVNAUTOSCI-2596, intended to exercise a frozen OpenAI Responses-based loop over an existing read-only MCP tool palette and emit a local JSON transcript for comparison runs.
Changes:
- Introduces
scripts/try_thin_readonly_turn.py, a standalone stateless Responses loop with a hard read-only tool boundary enforced via Internal MCP method definitions. - Adds
tests/test_thin_readonly_turn.pycovering tool-continuation mechanics, malformed provider tool-call correlation handling, and read/write boundary enforcement.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| scripts/try_thin_readonly_turn.py | Implements the disposable stateless Responses-based candidate loop and read-only tool boundary checks. |
| tests/test_thin_readonly_turn.py | Adds focused tests validating tool continuation, provider protocol errors, and boundary rejection behaviour. |
Comments suppressed due to low confidence (1)
scripts/try_thin_readonly_turn.py:348
- After moving the
sys.pathmanipulation behind a helper, call it early inmain()(before importing fromsrc...) so the script still works when executed from outside the repo root, while keeping imports side-effect free for test imports.
args = parser.parse_args(argv)
try:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
User outcome
Provide one genuinely thin, permissive comparison path for the authorised six-run A1 experiment, so evidence rather than another prewritten controller theory can determine whether A2 or any deletion slice should exist.
Material changes
gpt-5.6-luna, stateless/store-false Responses calls, two tool batches, Michael's actor scope, and a broad 52-tool read-only palette.This PR is the branch-visible freeze artefact for JVNAUTOSCI-2596. Candidate base:
95c6a4caad1f50b0845f2e654b97522cc586426b. Frozen candidate commit:207178f0d4a7133a70aa320cd267e311797f7407.The earlier pre-freeze SHA was superseded before any case was supplied or seen solely to remove import-time path mutation and avoid rebuilding a constant set on each tool invocation. Prompt, model, tool palette, actor, limits, and experiment protocol did not change.
Evidence
python -m pytest -q tests/test_thin_readonly_turn.py— 4 passed.read.This evidence supports candidate mechanics and the read-only boundary only. It says nothing yet about usefulness or relative performance.
Decision boundary
Do not merge this PR. It is a disposable A1 candidate, not a proposed production controller. Source, prompt, model, tools, actor, and limits are frozen before the cases are revealed.
Michael supplies three standalone cases after the current-Von comparison profile is also frozen. Each case is attempted once here and once on current Von: exactly six submissions, no retry, rephrase, tuning, replacement, or seventh run. H1/JVNAUTOSCI-2597 belongs to Michael and alone decides CONTINUE, REVISE, REORDER, or STOP. A2 and the first deletion slice have no authority until that decision.