Add testing + fixes for anthropic => responses SDK streaming path - #400
Conversation
…reams Providers with supports_streaming == false have their full response wrapped as a single chunk (single_bytes_stream) and fake-streamed. complete_stream_internal gated allow_full_response_fallback on spec.supports_streaming, which is false for exactly those providers, so StreamTransformSession could not detect the full response as a stream and passed the raw provider JSON through instead of emitting SSE events (BT-6217; Azure Foundry anthropic models returned an empty streamText result). Always enable the fallback -- the native-streaming Vertex full-response edge relies on it too. Adds transform_stream_synthesizes_responses_stream_from_full_anthropic_response, which asserts a full Anthropic response is synthesized into a well-formed Responses stream (output_item.added -> output_text.delta w/ item_id -> output_item.done) rather than passed through raw. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 23f0c9a99f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| output_index: Option<u32>, | ||
| text: String, | ||
| ) { | ||
| let item = responses_message_item(item_id, "completed", Some(&text)); |
There was a problem hiding this comment.
When a source ends because of a token limit, the terminal event is response.incomplete, but this newly synthesized message is always marked completed. Native captured Responses streams mark both the output item and the corresponding entry in response.output as incomplete in this scenario (for example, payloads/snapshots/multimodalRequest/responses/response-streaming.json), so consumers can otherwise accept truncated text as a complete message. Derive the item status from the finish reason or terminal event.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
fixed
| let Some(obj) = event.as_object_mut() else { | ||
| continue; | ||
| }; | ||
| let output_index = obj.get("output_index").and_then(Value::as_u64).unwrap_or(0); |
There was a problem hiding this comment.
When an event has a missing, non-numeric, or out-of-range output_index, this raw map access silently substitutes zero and correlates the delta with msg_0 instead of returning an explicit conversion error. Deserialize the generated Responses events into a typed event enum/view before inspecting or modifying their semantic fields, as required for provider-boundary conversion code.
AGENTS.md reference: AGENTS.md:L21-L24
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
fixed
Drop the verbose explanatory comment blocks added with the BT-6217 fix, keeping only single-sentence comments. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The synthesized Responses message item was hardcoded to "completed", so a truncated turn emitted response.incomplete alongside an item (and a response.output entry) marked completed. Native captured Responses streams mark both incomplete in this case. Also stamp text deltas with the item id already known by the caller instead of re-reading output_index out of the generated JSON, which removed a silent unwrap_or(0) fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 22993740b5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
Replaces the hardcoded `allow_full_response_fallback = true` with `transform_provider_stream`, which builds the session via `StreamTransformSession::new` (the library default). The router no longer invents a fallback policy at the call site. `transform_stream` and `transform_stream_with_capture` were private to the crate and had no remaining callers, so they are removed; the explicit-flag API stays available on `StreamTransformSession::with_full_response_fallback`, which lingua covers directly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 97950a2c19
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
Tool calls emitted output_item.added plus argument deltas but never function_call_arguments.done or output_item.done, and the function_call item carried no id. The AI SDK emits its tool-call on output_item.done, so tool calls never surfaced to consumers -- the same defect as the text envelope, which the earlier fix only covered for text. Open tool items are now tracked per output index and closed at finish, ordered alongside the message item, and included in response.completed.output. Adds streaming coverage to transforms-ai-sdk.test.ts, which drives lingua's transformed stream through the real AI SDK. A malformed Responses stream does not throw -- content is silently dropped -- so the assertions check that text or tool calls are actually surfaced. Reverting the stream synthesis fixes turns all 9 streaming cases red. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 91e8140758
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
Erin McNulty (erin2722)
changed the title
| requires_json_response: _, | ||
| strategy: _, | ||
| } = request.inner; | ||
| let allow_full_response_fallback = spec.supports_streaming; |
There was a problem hiding this comment.
This seems to have been erroneously introduced by me when fixing the "vertex will report streaming but respond with a full repsonse" bug-- this was overly perscriptive for other streaming paths. vertex behavior still works here.
| } | ||
| }); | ||
| } | ||
|
|
There was a problem hiding this comment.
good call on adding this, it caught one additional bug and I confirmed it would've caught the other issues solved in this PR
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c796088d08
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| #[derive(Debug, Deserialize)] | ||
| #[serde(tag = "type")] | ||
| enum ResponsesToolStreamEvent { |
There was a problem hiding this comment.
These enums duplicate provider-wire event shapes already modeled by ResponsesOutputItemAddedEvent, ResponsesFunctionCallArgumentsDeltaEvent, and ResponsesCustomToolCallInputDeltaEvent in responses_adapter.rs. If that adapter's emitted schema changes, this second definition can drift and make valid synthesized calls fail inside record_responses_tool_items; expose or extend the canonical views—or keep this state recording in the adapter—instead of defining another wire enum here.
AGENTS.md reference: AGENTS.md:L20-L20
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
sure, fixed
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cdc601b302
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 54fba30cc0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| let state = responses_output_index_states | ||
| .entry(choice_index) | ||
| .or_default(); |
There was a problem hiding this comment.
When Anthropic emits parallel tool_use blocks, its adapter assigns each UniversalStreamChoice the content-block index (anthropic/adapter.rs lines 1120-1126), while the terminal message_delta always uses choice index 0 (lines 1024-1027). Keying the newly added tool_items storage by that choice index means calls from blocks 1 and above are never taken when the terminal event arrives, so they receive no response.output_item.done and disappear from response.completed.output. Fresh evidence beyond the earlier terminal-output comment is that tool accumulation is now explicitly nested inside these per-choice states; tool items should instead be finalized response-wide.
Useful? React with 👍 / 👎.
| let finish_item_status = match finish_reason.as_deref() { | ||
| Some("length") => "incomplete", | ||
| _ => "completed", |
There was a problem hiding this comment.
When a Bedrock guardrail, Google safety stop, or Anthropic refusal is normalized to content_filter, this match labels every newly synthesized message or tool item as completed. The repository's canonical FinishReason::is_incomplete explicitly includes ContentFilter, so consumers can otherwise accept filtered or partial content as successfully complete; derive the status from that canonical predicate and apply the same correction to the terminal Responses event.
Useful? React with 👍 / 👎.
…bt-6217-anthropic-responses-streaming
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6d4e2ce0fa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| output_format, | ||
| allow_full_response_fallback, | ||
| ), | ||
| session: lingua::StreamTransformSession::new(output_format), |
There was a problem hiding this comment.
When a model with supports_streaming == false returns a complete Responses payload, enabling full-response fallback here converts its output into one UniversalStreamChoice per item, but Responses stream synthesis consumes only chunk.choices.first(). Typical Responses payloads contain a reasoning item followed by the assistant message, so the synthesized stream drops the actual text and tool items—often emitting only an empty terminal response. Expand every choice or merge the full response's output items before stream synthesis.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
fixed
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 339d4d2666
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| let mut entries: Vec<(u32, Value, Option<Value>)> = Vec::new(); | ||
|
|
||
| if let Some((output_index, item_id, text)) = message { |
There was a problem hiding this comment.
When synthesizing a Responses stream for a turn containing reasoning, the adapter emits response.reasoning_summary_text.delta, but this completion list is populated only with the message and tool items. Consequently no reasoning response.output_item.done is emitted and the terminal response.output omits the reasoning item; this is especially visible when fake-streaming a full Responses payload containing an rs_* item followed by text. Track and finalize reasoning output items alongside messages and tools.
AGENTS.md reference: AGENTS.md:L11-L11
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c9f0840e7c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".
| if let Some(content) = delta.content { | ||
| text.push_str(&content); | ||
| } | ||
| merged.reasoning.extend(delta.reasoning); |
There was a problem hiding this comment.
When fake-streaming a full Responses payload containing multiple reasoning items, this folds every item into one delta, after which Responses synthesis assigns them all the single identity rs_0; empty-summary items disappear entirely. Fresh evidence beyond the earlier full-output fix is payloads/snapshots/toolCallRequest/responses/followup-response.json lines 19-44, which contains five distinct reasoning items that this path cannot reproduce, so the terminal response.output loses their boundaries and IDs. Preserve each full-response output item rather than merging all reasoning vectors.
AGENTS.md reference: AGENTS.md:L11-L11
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
that is fine for this path
2 participants
see https://github.com/braintrustdata/braintrust/pull/18176