python-math #11: fix(delphi): Clojure-parity math writers + committing write path in PostgresClient - #2624
Draft
jucor wants to merge 1 commit into
Draft
python-math #11: fix(delphi): Clojure-parity math writers + committing write path in PostgresClient#2624jucor wants to merge 1 commit into
jucor wants to merge 1 commit into
Conversation
This was referenced Jul 18, 2026
Draft
jucor
marked this pull request as draft
July 18, 2026 02:43
jucor
force-pushed
the
spr/edge/a861cd87
branch
2 times, most recently
from
July 18, 2026 13:35
7c607b1 to
e5344f9
Compare
This was referenced Jul 18, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates Delphi’s Postgres integration to better match the production Clojure SQL semantics and to ensure math writes actually commit under SQLAlchemy 2.x, while also hardening JSON serialization of math blobs that may contain NumPy scalar types.
Changes:
- Add a committed write helper (
_write_returningviaengine.begin()) and route math upsert writers through it;execute()now commits as well. - Rewrite/extend math table writers for Clojure parity (including atomic
increment_math_tickand a newmath_bidtopidmodel/writer). - Add global poll helpers (
poll_votes_since,poll_moderation_since) and enforce deterministic row ordering for vote polling; add shared NumPy-to-JSON serialization helper + regression test coverage.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| delphi/polismath/database/postgres.py | Adds committed write path, Clojure-parity upsert SQL for math writers, atomic tick increment, deterministic vote ordering, and global watermark poll helpers. |
| delphi/polismath/utils/serialization.py | Introduces shared convert_numpy_types helper for JSON serialization of NumPy scalars/arrays. |
| delphi/polismath/regression/utils.py | Hoists and re-exports convert_numpy_types from the new shared serialization module for backward compatibility. |
| delphi/tests/test_math_writer_numpy_serialization.py | Adds regression tests ensuring math writers can serialize real computed blobs containing NumPy integer scalars. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
jucor
force-pushed
the
spr/edge/a861cd87
branch
2 times, most recently
from
July 21, 2026 19:49
81f5d9d to
c9ec6ad
Compare
This was referenced Jul 25, 2026
Draft
Draft
Draft
python-math #43: docs(delphi): s7 goal state + journal — mode collapse executed, battery 20/20
#2672
Draft
Draft
Draft
This was referenced Jul 28, 2026
Draft
jucor
changed the title
3 tasks
…g write path in PostgresClient ## What The four math writers in `PostgresClient` (the functions that write math results to the Postgres tables the server reads) had zero live callers (verified: only commented-out references in `test_postgres_real_data.py`) and did not match the Clojure SQL. This rewrites them to the verified Clojure statements and fixes the write path so INSERTs actually commit. ## Writer fixes - `write_math_main`: `caching_tick = COALESCE((select max(caching_tick)+1 from math_main where math_env=?), 1)` upsert (`postgres.clj:323-338`). `math_env` is the column that keys each math result row — the server only reads rows matching its own setting. The production TypeScript prefetch (`pca.ts:84-151`) polls for `caching_tick > last`, so this is fidelity-critical. - `increment_math_tick`: atomic `INSERT ... ON CONFLICT ... math_tick+1 RETURNING` (`postgres.clj:292-295`), replacing the previous read-modify-write. - Add the `MathBidToPid` model + `write_math_bidtopid` (net-new; the server's `participants.ts` depends on it). - `write_participant_stats`: add the shared `math_tick` (Clojure writes all three data tables with one tick, `conv_man.clj:158-169`). ## Committing write path Add `_write_returning` (using `engine.begin()`) and route the four writers through it: `query()` uses `engine.connect()` (SQLAlchemy 2.0 commit-as-you-go), which rolls back INSERTs on close. Mocked unit tests could not catch this; the Postgres integration test did. `execute()` now commits too. ## Watermark reads for the poll loops Add `poll_votes_since` (`WHERE created > wm ORDER BY zid,tid,pid,created`, sign-flipped, `postgres.clj:132-145`) and `poll_moderation_since` (`postgres.clj:148-161`). Also add `ORDER BY zid,tid,pid,created` to `poll_votes` so the full-history rebuild reproduces Clojure's row-insertion order (base-cluster IDs seed k-means -> parity). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> commit-id:6324b652
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The four math writers in
PostgresClient(the functions that write math results to the Postgres tables the server reads) had zero live callers (verified: only commented-out references intest_postgres_real_data.py) and did not match the Clojure SQL. This rewrites them to the verified Clojure statements and fixes the write path so INSERTs actually commit.Writer fixes
write_math_main:caching_tick = COALESCE((select max(caching_tick)+1 from math_main where math_env=?), 1)upsert (postgres.clj:323-338).math_envis the column that keys each math result row — the server only reads rows matching its own setting. The production TypeScript prefetch (pca.ts:84-151) polls forcaching_tick > last, so this is fidelity-critical.increment_math_tick: atomicINSERT ... ON CONFLICT ... math_tick+1 RETURNING(postgres.clj:292-295), replacing the previous read-modify-write.MathBidToPidmodel +write_math_bidtopid(net-new; the server'sparticipants.tsdepends on it).write_participant_stats: add the sharedmath_tick(Clojure writes all three data tables with one tick,conv_man.clj:158-169).Committing write path
Add
_write_returning(usingengine.begin()) and route the four writers through it:query()usesengine.connect()(SQLAlchemy 2.0 commit-as-you-go), which rolls back INSERTs on close. Mocked unit tests could not catch this; the Postgres integration test did.execute()now commits too.Watermark reads for the poll loops
Add
poll_votes_since(WHERE created > wm ORDER BY zid,tid,pid,created, sign-flipped,postgres.clj:132-145) andpoll_moderation_since(postgres.clj:148-161). Also addORDER BY zid,tid,pid,createdtopoll_votesso the full-history rebuild reproduces Clojure's row-insertion order (base-cluster IDs seed k-means -> parity).Co-Authored-By: Claude Fable 5 noreply@anthropic.com
commit-id:6324b652
Stack: