Skip to content

[e2e] Resolve the served model id from /v1/models - #54

Closed
weaverbot wants to merge 1 commit into
mainfrom
weaver/issue-40
Closed

[e2e] Resolve the served model id from /v1/models#54
weaverbot wants to merge 1 commit into
mainfrom
weaver/issue-40

Conversation

@weaverbot

Copy link
Copy Markdown

The nightly e2e 404'd on every eval request after marin-core floated to
0.2.58.dev202607250745: marin-serve began launching vLLM with
--served-model-name set to the resolved weights cache path, so /v1/models no
longer advertised the requested id Qwen/Qwen3-0.6B and lm-eval's
model=Qwen/Qwen3-0.6B requests returned NotFoundError (run 30152121453). The TPU
served and cleanup stopped the Iris job; the failure was purely the model-name
mismatch, and the runner had assumed the served name equals the HF id it asked to
serve.

serve_eval now reads /v1/models once the endpoint reports ready and uses the
advertised id as the request model, keeping the requested HF id only as the
lm-eval tokenizer for length bookkeeping. It prefers the requested id when the
endpoint advertises it, adopts the sole advertised id otherwise, and refuses an
ambiguous mismatch. This holds for both providers and survives future marin-serve
renames.

The served-name change is a marin-core regression tracked in
marin-community/marin#7636; this change keeps the eval lane green regardless.

Part of #40

The nightly e2e 404'd on every eval request after marin-core floated to
0.2.58.dev202607250745: marin-serve began launching vLLM with
--served-model-name set to the resolved weights cache path, so /v1/models no
longer advertised the requested id Qwen/Qwen3-0.6B and lm-eval's
model=Qwen/Qwen3-0.6B requests returned NotFoundError. The runner had assumed
the served name equals the HF id it asked to serve.

serve_eval now reads /v1/models once the endpoint is ready and uses the
advertised id as the request model, keeping the requested HF id only as the
lm-eval tokenizer for length bookkeeping. It prefers the requested id when the
endpoint advertises it, adopts the sole advertised id otherwise, and refuses an
ambiguous mismatch, so the eval lane holds across both providers and future
marin-serve renames.

The served-name change is a marin-core regression tracked in
marin-community/marin#7636.

Part of #40
@weaverbot

Copy link
Copy Markdown
Author

🤖 Closing — this will be fixed on the marin side. Root cause is marin-serve advertising the resolved weights cache path as the served model name (marin-community/marin#7636); with that fixed upstream, the /v1/models discovery workaround here isn't needed. Triage details in #40.

@weaverbot weaverbot closed this Jul 25, 2026
@weaverbot

Copy link
Copy Markdown
Author

🤖 The marin-side fix (marin-community/marin#7637) is verified end-to-end on a TPU v5litepod-4 serving Qwen/Qwen3-0.6B: /v1/models again advertises the requested id, a completion addressed by that id returns 200, and the resolved cache path returns 404 — stable across a cold mirror and a warm cache hit. This harness workaround stays closed as unnecessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-generated Automated failure tracking

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants