Skip to content

python-math #7: feat(math): group-k-smoother in clojure-legacy engine mode (PR-D smoother) - #2620

Draft
jucor wants to merge 1 commit into
spr/edge/be96f1dafrom
spr/edge/d4dbbfa9
Draft

python-math #7: feat(math): group-k-smoother in clojure-legacy engine mode (PR-D smoother)#2620
jucor wants to merge 1 commit into
spr/edge/be96f1dafrom
spr/edge/d4dbbfa9

Conversation

@jucor

@jucor jucor commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

What

Ports Clojure's :group-k-smoother (conversation.clj:454-478), which damps K (the number of opinion groups) so it only switches to a new best value after :group-k-buffer (= 4, conversation.clj:154) consecutive ticks agree. This applies only in clojure-legacy engine mode (the setting that reproduces the old Clojure engine's behavior exactly); default improved mode keeps the existing best_k logic bit-for-bit.

Changes

  • New polismath/pca_kmeans_rep/group_k_smoother.py: a pure function group_k_smoother_update(prev_state, silhouettes_by_k, buffer=4) -> (new_state, smoothed_k). Implements the exact Clojure rule, INCLUDING:
    • the HIGHER-k-wins tie-break: Clojure's apply max-key ... (keys ...) returns the LAST maximal argument, and array-map keys iterate ascending — so ties go to the higher k. conversation.py's improved best_k uses strict > (LOWER k wins) and is left untouched.
    • the Fix stale group-k-smoother crash on large conversations #2536 clamp (conversation.clj:469-478): a carried smoothed_k absent from the current clusterings falls back to this_k, so smoothed_k is always a valid key and downstream never KeyErrors.
    • first-tick acceptance of this_k (when smoothed_k is None) — the cold-start invariant that keeps the two modes identical on tick 1.
  • Only the top-level smoother is ported; Python has no subgroups (subgroup_clusters is hardcoded {}), so the parallel subgroup smoother (conversation.clj:520-560) is intentionally omitted.
  • Conversation._compute_clusters: in legacy mode computes silhouettes_by_k from group_clusterings, runs the smoother against prev_group_k_smoother (threaded via recompute(), PR-A), picks group_clusters = group_clusterings[smoothed_k], and stores the new smoother state + per-k clusterings on the conversation object (NOT persisted — conv_man.clj:52-74). Improved mode: selected_k = best_k, and the new fields stay {} (inert).

Semantic judgment call (integrator, please verify)

The degenerate early-return paths in _compute_clusters (fewer than 2 in-conversation participants / fewer than 2 base clusters) do NOT touch the smoother state, so a degenerate tick preserves the prior in-memory smoother memory rather than resetting it; the clamp protects the next real tick.

Testing

TDD RED evidence (before implementing group_k_smoother.py): tests/test_group_k_smoother.py collection error — "ModuleNotFoundError: No module named 'polismath.pca_kmeans_rep.group_k_smoother'".

Tests: 12 new — 10 pure-function (buffer counting, reset-on-change, brief alternation, clamp present/absent, first-tick, higher-k tie-break) + 2 pipeline integration (legacy no-flicker-then-switch-after-4 via a silhouette stub over 9 chained update_votes ticks: smoothed_k == [2,2,2,2,2,2,2,3,3]; improved mode leaves the smoother inert). Full suite 434 passed, 17 skipped, 47 xfailed.

Co-Authored-By: Claude Fable 5 noreply@anthropic.com

commit-id:d4dbbfa9


Stack:


⚠️ Part of a stack created by spr. Do not merge manually using the UI - doing so may have unexpected results.

@jucor
jucor marked this pull request as draft July 18, 2026 01:48
@jucor
jucor force-pushed the spr/edge/d4dbbfa9 branch from e289f05 to 9128aa8 Compare July 18, 2026 01:59
@jucor jucor changed the title feat(math): group-k-smoother in clojure-legacy engine mode (PR-D smoother) feat(math): group-k-smoother in clojure-legacy engine mode (PR-D smoo… Jul 18, 2026
@jucor
jucor changed the base branch from spr/edge/b39add7e to edge July 18, 2026 13:27
@jucor
jucor force-pushed the spr/edge/d4dbbfa9 branch from 9128aa8 to d22ebc2 Compare July 18, 2026 13:28
@jucor jucor changed the title feat(math): group-k-smoother in clojure-legacy engine mode (PR-D smoo… feat(math): group-k-smoother in clojure-legacy engine mode (PR-D smoother) Jul 18, 2026
@jucor
jucor changed the base branch from edge to spr/edge/81a957ec July 18, 2026 13:28
This was referenced Jul 25, 2026
@jucor
jucor force-pushed the spr/edge/d4dbbfa9 branch from 19ceef8 to 35c4edb Compare July 28, 2026 00:11
@jucor
jucor force-pushed the spr/edge/be96f1da branch from efefe9a to 30e7d37 Compare July 28, 2026 00:12
@jucor jucor changed the title feat(math): group-k-smoother in clojure-legacy engine mode (PR-D smoother) python-math #7: feat(math): group-k-smoother in clojure-legacy engine mode (PR-D smoother) Jul 28, 2026
… mode (PR-D smoother)

## What

Ports Clojure's `:group-k-smoother` (`conversation.clj:454-478`), which damps K (the number of opinion groups) so it only switches to a new best value after `:group-k-buffer` (= 4, `conversation.clj:154`) consecutive ticks agree. This applies only in `clojure-legacy` engine mode (the setting that reproduces the old Clojure engine's behavior exactly); default `improved` mode keeps the existing `best_k` logic bit-for-bit.

## Changes

- New `polismath/pca_kmeans_rep/group_k_smoother.py`: a pure function `group_k_smoother_update(prev_state, silhouettes_by_k, buffer=4) -> (new_state, smoothed_k)`. Implements the exact Clojure rule, INCLUDING:
  - the HIGHER-k-wins tie-break: Clojure's `apply max-key ... (keys ...)` returns the LAST maximal argument, and array-map keys iterate ascending — so ties go to the higher k. `conversation.py`'s improved `best_k` uses strict `>` (LOWER k wins) and is left untouched.
  - the #2536 clamp (`conversation.clj:469-478`): a carried `smoothed_k` absent from the current clusterings falls back to `this_k`, so `smoothed_k` is always a valid key and downstream never KeyErrors.
  - first-tick acceptance of `this_k` (when `smoothed_k` is `None`) — the cold-start invariant that keeps the two modes identical on tick 1.
- Only the top-level smoother is ported; Python has no subgroups (`subgroup_clusters` is hardcoded `{}`), so the parallel subgroup smoother (`conversation.clj:520-560`) is intentionally omitted.
- `Conversation._compute_clusters`: in legacy mode computes `silhouettes_by_k` from `group_clusterings`, runs the smoother against `prev_group_k_smoother` (threaded via `recompute()`, PR-A), picks `group_clusters = group_clusterings[smoothed_k]`, and stores the new smoother state + per-k clusterings on the conversation object (NOT persisted — `conv_man.clj:52-74`). Improved mode: `selected_k = best_k`, and the new fields stay `{}` (inert).

## Semantic judgment call (integrator, please verify)

The degenerate early-return paths in `_compute_clusters` (fewer than 2 in-conversation participants / fewer than 2 base clusters) do NOT touch the smoother state, so a degenerate tick preserves the prior in-memory smoother memory rather than resetting it; the clamp protects the next real tick.

## Testing

TDD RED evidence (before implementing `group_k_smoother.py`): `tests/test_group_k_smoother.py` collection error — "ModuleNotFoundError: No module named 'polismath.pca_kmeans_rep.group_k_smoother'".

Tests: 12 new — 10 pure-function (buffer counting, reset-on-change, brief alternation, clamp present/absent, first-tick, higher-k tie-break) + 2 pipeline integration (legacy no-flicker-then-switch-after-4 via a silhouette stub over 9 chained `update_votes` ticks: `smoothed_k == [2,2,2,2,2,2,2,3,3]`; improved mode leaves the smoother inert). Full suite 434 passed, 17 skipped, 47 xfailed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

commit-id:d4dbbfa9
@jucor
jucor force-pushed the spr/edge/d4dbbfa9 branch from 35c4edb to ba62851 Compare July 28, 2026 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants