test: add operation-aware artifact agreement coverage - #55
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add comprehensive agreement tests for the operation-aware kernel artifacts produced by the merged response and audit assembly functions:
EvaluationTraceOperationAwareDecisionResponseAuditEvidenceThis PR proves that one evaluation produces one coherent story across all three artifacts.
It adds no production behavior.
Scope
The PR adds:
and narrowly updates the PR 32 roadmap entry.
No file under
src/changes.Agreement Matrix
The tests verify agreement across every shared field:
Agreement is checked across:
Optional-field presence is part of agreement. A value on one artifact and
Noneon another is detected as a mismatch.Independent Vocabulary Mapping
The tests define independent, test-owned mappings from trace enums to the decisions-owned response and audit enums:
The mappings are not imported from the production assembler.
Exhaustiveness tests prove complete source and target coverage.
Response Forms
Both supported response forms are covered.
Reference-only response:
Embedded response:
All top-level agreement fields remain consistent in either form.
Matched-Rule Agreement
AuditEvidence.matched_rule_idsis independently projected from ordered trace rule evidence.Only
MATCHEDresults are included.The tests prove:
A reversed matched-rule order is detected as disagreement.
Evidence Provenance
The tests verify that request-owned evidence references are preserved in
AuditEvidence:Coverage includes:
Response and trace models remain free of these request-only fields.
Evaluation States
Complete agreement is proven for:
The failed semantic bundle-validation case correctly remains:
It is never converted to deny and never uses the superseded
invalid_policy_bundleclassification for the duplicate-rule scenario.All six governed failure reasons are covered.
Negative Mutation Matrix
The suite includes exactly 24 test-only disagreement mutations covering:
Every mutation is detected.
Assertions identify the mismatched field without requiring a production agreement validator.
Serialization
Agreement is preserved through:
model_dump(mode="json");model_dump(mode="json", exclude_none=True);model_dump_json();Contract-required nullable fields remain present.
Determinism and Immutability
Tests prove:
Boundaries
The new test module does not:
OperationAwareEvaluationEngine;Complete canonical fixture equality remains PR 37.
Gateway-owned enforcement facts remain outside kernel artifact agreement.
Validation
109 passed636 passed2,998 passed, 86 skipped263 passed3,715 passed, 86 skippedgit diff --check: passed