feat(tenancy): capture tenant_id at ingest — Phase 1, capture only

[DarrenZal]

Records which tenant a document belongs to at write time, so per-tenant isolation stays possible later.

⚠️ Read this first

This is capture, not enforcement. Nothing filters on tenant_id. It does not make data safe to expose to an external user — koi-query-api.ts buildPrivacyFilter() still returns '' for any authenticated caller ("Authenticated users see all data"), and access is gated solely by a hardcoded @regen.network email check. A tenant-stamped document is exactly as exposed as an unstamped one.

Deploy order is load-bearing. Run migration 108 before deploying the code. The patched writers name tenant_id in their INSERT column list, so against a pre-108 database every write raises UndefinedColumnError and the row is not written (reproduced: rows written = 0). It fails loudly, but it fails closed. Same in reverse — roll back the code before the migration.

Why now, before any tenancy decision

Provenance not recorded at write time cannot be reconstructed. On prod today:

SELECT count(*) FROM koi_entity_chunk_links l
  LEFT JOIN koi_memories m ON l.document_rid = m.rid WHERE m.rid IS NULL;
-> 200,751 of 3,841,436 (5.22%)

Origin permanently undecidable. Every document ingested without a tenant key joins that set. The column is inert if we never build tenancy, and a prerequisite for any version of it we do build — so it's a cheap hedge rather than a bet on one design.

Scope — three changes

Change	Why
migrations/108_tenant_id.sql	tenant_id on koi_memories mirroring 015's shape, a CHECK rejecting blanks, two partial indexes
koi_event_bridge_semantic.py	The document-level writer (is_chunk FALSE)
koi_event_bridge_v2.py	Extends the existing is_private/access_source promotion block (chunk rows)
scripts/doc_scanner.py	--tenant, threaded through the full chain including watch mode

Sticky by COALESCE(existing, new): a re-ingest must never silently re-attribute a document that already has an owner.

Deliberately excluded

entity_registry (41 insert sites) and entity_relationships (30) — the two largest write surfaces. entity_registry is a global namespace (UNIQUE (normalized_text, entity_type), resolved with no scope), so adding a tenant key forks every entity per tenant and destroys cross-tenant aggregation. That is a product decision, not a migration. It also means document-level stamping alone does not stop entity-level leakage — worth deciding before anyone relies on this.

Verified against real Postgres, not by inspection

• migration applies and re-applies cleanly (idempotent, including the CHECK)
• first ingest sets tenant · re-ingest with a different tenant does not steal the document · NULL re-ingest does not wipe · untenanted doc can be claimed later
• '' and whitespace-only rejected by the CHECK
• all touched Python compiles

The blank CHECK exists because COALESCE would treat '' as a real owner and permanently block the correct tenant. It enforces blankness only — stickiness remains a per-writer convention, and an unpatched writer using plain EXCLUDED.tenant_id would silently re-attribute. Stated in the migration.

Second commit fixes five defects an adversarial review found in the first

1. The down migration sat inside the runner's glob. Both run_migrations.sh and run_migrations_with_backup.sh do ls -1 "$MIGRATION_DIR"/*.sql | sort; the down file matched and sorts after the up, so a runner-driven apply would create the column, immediately drop it, and report Failed: 0. Moved to migrations/down/.
2. I had patched the wrong bridge site. koi_event_bridge_v2.create_new_version is only ever called with chunk_event — it writes chunk rows. The document writer is in koi_event_bridge_semantic.py and was untouched, so the first commit attributed zero documents.
3. --tenant was silently dropped for unchanged documents — scan_repo short-circuits before the writer, and --force is rejected in watch mode.
4. Deploy order was documented nowhere.
5. The locking note was inverted — ADD CONSTRAINT ... CHECK takes AccessExclusive and blocks readers; CREATE INDEX takes ShareLock and blocks writers only.

What I'd like reviewed

• Is tenant_id the right name/shape, given koi_memories already carries is_private + access_source?
• Should stickiness be a BEFORE UPDATE trigger rather than a per-writer convention?
• Is excluding entity_registry the right call, or does it make document-level capture not worth having?

🤖 Generated with Claude Code