Sprint-39 ceremony: extract 0.6.0 — temporal validity role + resolution anchor#32
Merged
Conversation
…420) + extract_batch skeleton Unify the extract Python package under the synapt.* namespace (PEP 420), matching synapt.recall + synapt.premium. extract was the odd one out in underscore-top-level form (Layne 2026-07-13). - Package dir: src/synapt_extract/ -> src/synapt/extract/ (namespace package; NO src/synapt/__init__.py, so it coexists with synapt.recall/synapt.premium). - Import path: synapt_extract -> synapt.extract / synapt.extract.batch. PyPI DIST name UNCHANGED (synapt-extract). pyproject: packages.find namespaces=true + package-data key synapt.extract. - prompt.py repo-root resolution parents[4] -> parents[5] (module moved one level deeper). The response-format string literal "synapt_extraction_stage1" is left intact (renamed only package references, keyed on the trailing dot). - Existing 21 test files + examples/dogfood.py imports updated. 307 existing tests pass against the renamed package (verified, fresh venv). - extract_batch SKELETON folded in at synapt/extract/batch.py: API conformed to the pinned contract + Sentinel's spec (async, BatchFailureReason Literal, BatchUnit, injected infer seam, BatchUnitResult, _coerce_shape/_strip_output_hygiene). Bodies raise NotImplementedError -- impl lands in the follow-up PR. Sentinel's rebased extract#28 will COLLECT and run RED (not ImportError). NO PUBLISH (Layne-gated; breaking version bump deferred). npm/TS (@synapt-dev/extract) HELD -- Python only for now. Cross-repo follow-on (Opus's recall-wiring lane, post-publish): recall imports the old path -- synapt/pyproject.toml `synapt-extract>=0.5.0` (dist dep, unaffected) and consolidate.py:46 `from synapt_extract import ...` (module import -> synapt.extract). Contained now by NO-PUBLISH: recall runs against published v0.5.0 (old path); the consolidate.py:46 update happens when the renamed version is published + recall rewires. Premium boundary: OSS (IL primitive, no identity/org).
…name (Sentinel review) Reviewer-2 (Sentinel) caught refs the .py-only sweep missed — CI build-python copied prompts to the removed src/synapt_extract/prompts (0 prompt JSON) and schema-drift diffed the removed src/synapt_extract/schemas. PATH refs src/synapt_extract/ -> src/synapt/extract/: - .github/workflows/ci.yml (prompt copy 40-41,208-209; schema-drift diff 157) - .github/workflows/publish-pypi.yml (prompt copy 28-29) - .gitignore (generated prompts path) - SECURITY.md (prompt + schema copy commands) IMPORT refs from synapt_extract import -> from synapt.extract import + `synapt_extract` prose -> `synapt.extract`: - README.md + packages/python/README.md Python examples KEPT unchanged (correct): response-format name strings "synapt_extract_stage1" / "synapt_extraction_stage1"; synapt_extract.egg-info (dist stays synapt-extract, untracked/generated anyway); @synapt-dev/extract npm (held). Verified by fruit: simulated the CI copy with fixed paths -> 4 prompt JSON (incl. capabilities.json) bundle under the package; _INSTALLED_PROMPTS resolves; registry loads. Whole-repo grep now clean except the intended KEEPs.
…p420 chore(extract): rename synapt_extract → synapt.extract (PEP 420) + extract_batch skeleton
test: specify extract_batch contract
…all#868)
Implements the batch.py skeleton bodies against Sentinel's extract#28 contract
(16 tests / 49 cases) + Atlas's SHA-pinned real-failure fixtures. All 49 pass;
full suite 356 pass (307 baseline + 49), no regressions.
- _strip_output_hygiene (Class-A, NET-NEW): strip ``` fences (+ trailing epilogue)
then `//` line comments STRING-LITERAL-AWARE (a `//` inside a JSON string / URL
survives; tracks in-string + escape state).
- _coerce_shape (Class-B, harvest): derives the per-item field whitelist from
build_extraction_schema(capabilities) (general, Q3), coerces scalar→array,
omits null/non-string optionals, keeps invalid REQUIRED fields so finalize
rejects them (→ schema_invalid). entity_refs gated on the entities capability
(Q2 in-scope-coerce / out-of-scope-drop); temporal_refs → raw/resolved only.
- extract_batch (async): unique-id ValueError before inference; empty→noop; per-call
capabilities with per-unit overrides; standard-profile default; out-of-band
per-unit prompt (unit text only, no id/boundary in model-visible text); injected
infer seam; one deterministic retry per failed unit (2 attempts) then a terminal
{source_unit_id,status,reason} marker; count-invariant len(out)==len(in);
dropped = valid-but-empty (the 10/45 mode) caught + retried.
Premium boundary: OSS (IL primitive; no identity/org). No publish.
…ntinel re-gate)
Sentinel's fidelity gate on extract#30 (Opus confirmed) found 3 runtime edge cases
the 49 fixtures + reviews missed. All fixed + regression-locked; full suite 362 green.
HIGH-1 — infer() exception voided the whole batch (escaped the per-attempt guard).
Now contained inside the per-unit retry loop: an infer exception → this unit's
failure (retry once, then terminal "dropped" — no output produced, closest Q5
class), while neighbours still produce their slots. Count-invariant preserved.
HIGH-2 — non-object leaves silently deleted before strict validation (facts:[null]
→ dropped; facts:[null,valid] → silently ok). Now non-dict leaves are preserved
verbatim into finalize, which rejects them → schema_invalid (a null sibling fails
its whole unit rather than vanishing).
HIGH-3 — _is_empty_extraction false-dropped valid general-capability packets
(entities-only / summary-only) via a recall-shaped _NON_CONTENT_CAPABILITIES leak.
Removed it; emptiness is now type-aware over ALL requested schema payloads (array
non-empty OR scalar/object present) — Q3 general primitive, not recall-shaped.
Regression locks (Sentinel's requested cases): infer-exception-mid-batch,
non-dict-leaf {facts null / decisions 42 / temporal string / null-beside-valid},
metadata-only {entities-only / summary-only scalar control}.
Also: stale SKELETON/NotImplementedError module header + contradictory
category→default docstring corrected. Boundary: OSS. No publish.
feat(extract): implement extract_batch — 49/49 spec green (recall#868)
config/design/extract-temporal-role-2026-07-14.md. Replaces recall's B2-LLM-re-judges-
temporal duct tape (recall#875/876) with a real extract IL fix — the direction and
resolution belong at extraction, where the source sentence and source date are both
directly available, not in a second recall-side LLM pass re-deriving them.
DIRECTION (role): each temporal ref now carries a validity ROLE — effective (valid FROM
resolved), expiry (valid UNTIL resolved), range (valid FROM resolved TO resolved_end),
superseded (was valid UNTIL resolved, a prior-state fact), or point (no clear direction,
fallback). role + resolved_end are BASE-tier on the "temporal_refs" capability (no longer
gated behind the separate "temporal_classes" capability recall never requested) — this is
the root cause Apollo diagnosed: resolved_end was unreachable at recall's capability tier,
and even reachable, a bare "point" ref has no field encoding start-vs-end. type/context stay
temporal_classes-gated (non-load-bearing extras once role carries direction).
RESOLUTION (source-date anchor): BatchUnit gained an optional `date` field — the unit's
SOURCE date, threaded into Stage-1 as the resolution anchor for partial/relative dates
("April 30" only resolves correctly against the fact's actual source year). Sentinel's
real-path finding: a 2025-sourced "API key expires April 30" resolved to 2026-04-30 with
the pre-fix decomposed path — wrong year, no anchor at all was threaded through. Also fixed
a latent prompt-rendering bug found while building this: an absent `date` rendered the
literal string "Resolve relative dates using: None." instead of omitting the instruction.
Changes:
- builder.py::_temporal_ref_schema — role + resolved_end moved to base properties.
- validate.py — role added to allowed keys + enum validation + role=="range" requires
resolved_end (mirrors the existing type=="range" rule).
- batch.py::BatchUnit — new optional `date` field, threaded into build_extraction_prompt.
_coerce_shape/_coerce_item needed ZERO changes — the whitelist is schema-driven, so
preserving role+resolved_end through the batch path is a verified CONSEQUENCE of the
schema fix, not separate coercion code.
- finalize.py::_detect_capabilities — temporal_classes heuristic now keys on `type`
presence only (resolved_end no longer implies the gated capability was exercised).
- prompts/v1/temporal_refs.txt — role classification instructions (trigger examples per
role) + the date instruction now wrapped in {{#if date}} (fixes the "None" bug).
- schemas/temporal-ref/v1.json — published JSON-schema document synced (role property +
the role=="range" conditional requirement).
- Version bump 0.5.0 -> 0.6.0 (additive schema = minor). npm/@synapt-dev/extract parity
is a tracked follow-up, not this release — no TS extract_batch equivalent exists yet,
so the specific bug this fixes doesn't reproduce there.
TDD, fixtures pinned from REAL extract calls throughout (the discipline this bug's own
history demands — the original temporal fix passed both author-tests and reviewer-fruit
because both used a hand-built envelope; only Sentinel's pinned-real-output caught it).
36 new tests across validate/builder/batch/finalize/prompt, including a capstone test
replicating Sentinel's exact wrong-year scenario end-to-end through a real extract_batch
call. 382 total extract tests green, no regression.
Recall-side consumption (deterministic role->bound map, replacing the null placeholder)
is a follow-up PR, sequenced after this lands per the design note.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ZMaT1FQJD6rqfN77piMHm
…chema sync
Sentinel's REQUEST CHANGES on the Python-only cut: the shared prompt/schema surface was
left in a half-state (the shared Stage-1 prompt asks the model for `role`, but the TS
validate/builder surfaces reject it), which the repo's own schema-drift-check + ts/py
parity contract won't merge. Decision (Opus, per Layne's ts/py-parity directive): carry the
small additive-role parity here, not isolate assets. This completes the public-contract
surface around the (already Opus+Sentinel-confirmed) Python runtime.
Canonical schema + Python public type:
- schemas/temporal-ref/v1.json at the REPO ROOT synced to the package copy (role + the
role=="range"->resolved_end conditional) — `diff -r` clean, schema-drift-check green. The
Python-only cut updated only the package copy.
- schema.py SynaptTemporalRef TypedDict gains `role` — the v0.6 runtime emits role, so the
public type no longer rejects its own output. Guard test asserts the TypedDict declares
role AND its Literal members exactly equal the validator's VALID_TEMPORAL_ROLES.
- batch.py module/coercion docstrings updated for BatchUnit.date + base-tier role/resolved_end
(Sentinel's non-blocking doc note).
TypeScript parity (the additive-role slice only — mirrors the Python decisions exactly):
- schema.ts: SynaptTemporalRef interface gains `role?`.
- builder.ts: temporalRefSchema emits role + resolved_end BASE-tier; type/context stay
temporal_classes-gated.
- validate.ts: VALID_TEMPORAL_ROLES + role added to TEMPORAL_REF_KEYS + role enum validation
(+ role=="range" requires resolved_end).
- finalize.ts: temporal_classes detection keys on `type` only (drops the resolved_end
disjunct, now that resolved_end is base-tier).
- prompt-data.ts: embedded temporal_refs fragment updated to byte-match the shared
prompts/v1/temporal_refs.txt (the fragment-parity test enforces this; TS's renderer
supports the {{#if date}} conditional identically to Python).
- Version: @synapt-dev/extract 0.5.0 -> 0.6.0, in lockstep with synapt-extract (both were
synced at 0.5.0; a Python-only bump would introduce the exact version drift parity exists
to prevent). Publish is release-gated, unaffected.
The FULL TS parity (a TS extract_batch/batch.ts port + parity-guard hardening) remains the
post-validation follow-up per config/design/extract-ts-py-parity-plan-2026-07-15.md; this is
only the additive-role coherence needed to keep the shared surface consistent.
Verification — cross-language parity proven empirically, not asserted:
- buildExtractionSchema temporal_refs item: TS and Python produce BYTE-IDENTICAL JSON.
- validate + finalize on 5 shared role fixtures (valid expiry/effective/range, invalid enum,
range-without-resolved_end): LINE-FOR-LINE identical verdicts, error paths, and
temporal_classes detection across both languages.
- The embedded prompt fragment == the shared file (byte-exact, enforced by the existing
fragment-parity test).
Gates: schema-drift-check green (diff -r clean), 383 Python tests (+ TypedDict guard),
265 TS tests (+10 mirroring the Python role tests), tsc --noEmit clean. Schema file count
unchanged (13, wheel gate safe).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ZMaT1FQJD6rqfN77piMHm
…ot this slice (Sentinel) Sentinel's re-review caught a real half-state: my TS package.json bump to 0.6.0 left the tracked package-lock.json still at 0.5.0 (I bumped only package.json). Rather than complete the bump (sync the lockfile to 0.6.0), REVERT it — the version-sync belongs to the deferred full-parity effort per the dispatch's own scoping, and the additive-role coherence slice needs no npm version change. This restores packages/ts/package.json + package-lock.json to their consistent origin 0.5.0 state (the lockfile was never touched). synapt-extract (PyPI) stays at 0.6.0 — it ships the feature runtime + the extract_batch this all traces to. @synapt-dev/extract (npm) carries the same additive-role parity at 0.5.0; its version bump is a tracked follow-up, not this PR. CHANGELOG corrected accordingly. Behavior is IDENTICAL to fd8dff3 — a version-string revert only; all schema/validate/finalize/ prompt output is unchanged (265 TS tests + tsc still green). Reviewer replays on fd8dff3 remain valid. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ZMaT1FQJD6rqfN77piMHm
…anchor feat(extract): temporal validity role + source-date resolution anchor
…ssifier Sentinel's wheel QA (2026-07-15): content + runtime GREEN, HOLD on 2 immutable-metadata items — once published, these can't be corrected without a new version. 1. Documentation URL: https://synapt.dev/docs/extract 404s (verified). Changed to https://synapt.dev/schemas/ (verified 200) — the real, live docs surface for this package's schemas. 2. Removed the "Typing :: Typed" classifier: the wheel ships no py.typed marker, so the classifier was FALSE (Sentinel's mypy consumer probe proved it — a direct mypy pass against the package surfaces real annotation errors). Publishing py.typed hastily would ship a typing contract the source can't currently honor. py.typed + a type-check gate is now tracked in the deferred ts/py full-parity plan (config/design/extract-ts-py-parity-plan-2026-07-15.md) instead of rushed here. Verified by fruit, not just source read: rebuilt the wheel + sdist, twine check PASSED on both, METADATA read directly from the built wheel confirms both fixes (Documentation URL + no Typing::Typed classifier), no py.typed marker present (consistent), schema count unchanged at 13 (wheel-packaging gate). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ZMaT1FQJD6rqfN77piMHm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sprint-39 → main ceremony
Ships
synapt-extract0.6.0: temporal validity role + source-date resolution anchor, plus release-metadata fixes.What's in this ceremony
6abcf44): temporal validityrole(effective/expiry/range/superseded/point) at the BASEtemporal_refscapability +BatchUnit.datesource-date resolution anchor. Full ts/py parity for the additive-role slice. Fixes the wrong-year + directionless-bound gap Sentinel's real-path finding opened in recall's decomposed consolidation path.79acca7, direct commit — Sentinel's wheel-QA ratified requirements, mechanical, not a judgment call): Documentation URL corrected (/docs/extract404s →/schemas/200, verified both by fruit), removed the falseTyping :: Typedclassifier (nopy.typedships — verified consistent), rebuilt wheel+sdist withtwine checkPASSED on both, METADATA confirmed directly from the built artifact.Verification
tsc --noEmitclean.twine checkPASSED (wheel + sdist).Sequencing
Sentinel does a final quick re-twine-check on this ceremony's merge SHA (the actual artifact about to publish) before
gh release create v0.6.0fires the PyPI publish workflow.Premium boundary
OSS —
synapt-dev/extractis a public package; no identity/org/entitlement logic.Merge strategy: regular merge commit (never squash — preserves shared ancestry for future ceremonies, per this team's established discipline).