Skip to content

feat: add raw data table under each dashboard chart - #1502

Merged
renemadsen merged 6 commits into
stablefrom
feat/insight-dashboard-raw-data-table
Jul 29, 2026
Merged

feat: add raw data table under each dashboard chart#1502
renemadsen merged 6 commits into
stablefrom
feat/insight-dashboard-raw-data-table

Conversation

@renemadsen

Copy link
Copy Markdown
Member

Each chart already renders an aggregated table of percentages and counts. This adds a second, collapsible table showing one row per Answer — every field the Answer holds in the database, plus its AnswerValues pivoted into one column per question.

Design and plan are committed alongside the code under docs/superpowers/.

Shape

  • Rows — the answers that fed that chart: dashboard date range + location/tag, the item's filter question/answer, ignored options removed.
  • Answer columns — Id, Microting UID, Finished at, Duration, Site, Tags, Unit, Language, Survey config shown by default; Time zone, UTC adjusted, Created/Updated at, Version, Workflow state and raw FK ids behind the column picker, and always present in the export.
  • Question columns — every non-removed question, ordered by QuestionIndex, labelled "N – text". multi questions expand to one column per option, each cell carrying the chosen option's own name.

Notable implementation points

  • AnswerFilterHelper reproduces the answer-selection half of ChartDataHelpers.CalculateDashboardItem predicate for predicate, each annotated with its source line, so the table's row count reconciles with the chart above it. Two documented deviations: it also filters Answer.WorkflowState, and it replaces the materialised filter-question id list with an equivalent correlated subquery.
  • Cell values resolve by question kind, not option name. The SDK auto-generates options whose translation is the type keyword (smiley1, number, text, next), so preferring the option name would render text instead of the answer and smiley1 instead of Meget glad. Smileys read from the weight ladder; free-text and number read from AnswerValue.Value.
  • Question and option text resolve by the logged-in user's language. Answer.LanguageId is hardcoded to Danish on import and is not trustworthy.
  • Paging always breaks ties on Id; without it, answers sharing a FinishedAt could be duplicated and skipped across pages.
  • Text-type items are rejected — ChartDataHelpers filters those down a different branch this helper deliberately does not mirror.

Scope

Plugin only. No -base change, no EF migration, no SDK change — read-only over existing SDK tables.

Verification

  • dotnet build and ng build clean
  • 6 jest tests passing, confirmed non-vacuous by mutation testing
  • Table visually confirmed correct in the running app

Known gaps, called out rather than buried:

  • The Playwright specs added here have not been executed. The row-count reconciliation invariant has no automated proof behind it yet.
  • The Excel export is capped at 25 000 rows and fails with a localized message rather than exhausting memory. Batching the id lookup would lift the cap.
  • ChartDataHelpers is not refactored onto the shared helper — it is 4 274 lines driving every chart in the product, so the two can drift. Tracked as a follow-up in the spec.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EnP42zmHAgo2NZQsa6zdhG

renemadsen and others added 6 commits July 29, 2026 09:41
One row per Answer beneath each chart, with every Answer field and the
AnswerValues pivoted into one column per question (multi-select exploding
into one column per option).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnP42zmHAgo2NZQsa6zdhG
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnP42zmHAgo2NZQsa6zdhG
CLAUDE.md forbids committing plugin code under eform-angular-frontend;
the plan wrongly instructed a commit per backend task there. Plugin
commits happen only in this repo after devgetchanges.sh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnP42zmHAgo2NZQsa6zdhG
Each chart already renders an aggregated table of percentages and counts.
This adds a second, collapsible table showing one row per Answer, with
every field the Answer holds in the database plus its AnswerValues
pivoted into one column per question. Multi-select questions expand to
one column per option, each cell carrying the chosen option's own name.

Backend:
- RawDataController exposes a paged endpoint and an xlsx export.
- AnswerFilterHelper reproduces the answer-selection rules of
  ChartDataHelpers.CalculateDashboardItem predicate for predicate, with
  each one annotated with its source line, so the table's row count
  reconciles with the chart above it. Two documented deviations: it also
  filters Answer.WorkflowState, and it replaces the materialised
  filter-question id list with an equivalent correlated subquery.
- RawDataColumnBuilder derives the column set from the survey's questions
  and options; RawDataTranslations resolves question and option text by
  the logged-in user's language, since Answer.LanguageId is hardcoded to
  Danish on import and is not trustworthy.
- Cell values are resolved by question kind rather than by option name:
  the SDK auto-generates options whose translation is the type keyword
  ("smiley1", "number", "text", "next"), so smileys read from the weight
  ladder and free-text/number read from AnswerValue.Value.
- Paging always breaks ties on Id; without it, answers sharing a
  FinishedAt could be duplicated and skipped across pages.
- Text-type items are rejected: ChartDataHelpers filters them down a
  different branch that this helper deliberately does not mirror.

Frontend:
- dashboard-raw-data-view renders an mtx-grid whose columns are built
  from the server's column descriptors, with server-side paging and
  sorting, a column picker for the audit fields, and an Excel export.
  Data loads on first expand so the dashboard response does not grow.

Excel export is capped at 25000 rows and fails with a localized message
rather than exhausting memory; batching the id lookup would lift it.

Verified: dotnet build and ng build clean, 6 jest tests passing (checked
non-vacuous by mutation). The Playwright specs added here have not yet
been executed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnP42zmHAgo2NZQsa6zdhG
The PR previously relied on Playwright specs that had never been run, and
on nothing at all for the backend. This adds NUnit coverage to
InsightDashboard.Pn.Test, which the insight-dashboard-test-dotnet check
already runs against a 420_SDK database seeded from 420_SDK.sql.

RawData_RowSet_MatchesChartAnswers asserts the table's row set equals the
answers the chart plotted, for every dashboard fixture already used by
ChartDataUTests. The expected set is restated independently from the spec
rather than copied from AnswerFilterHelper, so the two cannot drift
together, and every returned answer is checked to actually carry a value
for the measured question.

RawData_Columns_CoverEveryQuestion asserts one column per question in
QuestionIndex order, multi questions expanded to one column per option
with the question label as prefix, and the audit columns present but
hidden by default.

RawData_Cells_ShowAnswersNotOptionKeywords guards the defect found in
review: the SDK names auto-generated options after the question type
("smiley1", "number", "text", "next"), so a naive implementation renders
the keyword instead of the answer.

To make cell resolution testable, the private ResolveSingleValue moves
out of RawDataService into a public RawDataValueResolver, which also owns
the not-answered sentinel and the skipped-question check. Behaviour is
unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnP42zmHAgo2NZQsa6zdhG
CI ran these for the first time and three of four failed. All three were
defects in the test code, not the feature - the run showed the table
rendering 137 answers correctly.

- Every test shares one page from beforeAll, so each unconditional
  toggle click alternated the disclosure open and shut. The third test
  therefore collapsed what the second had opened and could not find the
  grid. Expanding now goes through ensureExpanded(), which only clicks
  when the grid is absent.
- Row count was asserted the instant the grid became visible, before the
  response had rendered any rows. Now waits on the first row with an
  auto-retrying assertion.
- chartAmountTotal() read the last cell of the wrong row, yielding 5
  against 137 actual answers. Rather than repair a fragile scrape, the
  comparison is dropped: RawDataUTests.RawData_RowSet_MatchesChartAnswers
  already reconciles answer sets against the database, which is both
  stricter and stable. The Playwright spec now checks only that the UI is
  self-consistent, and the unused helper is removed.

The multi-select column check is skipped when the seed survey has no
multi question, since that is a property of the fixture rather than of
the feature, and the expansion itself is asserted in RawDataUTests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnP42zmHAgo2NZQsa6zdhG
@renemadsen
renemadsen merged commit 06bcf86 into stable Jul 29, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant