Skip to content

Phase 22 — Cluster diff over time

Last updated: 2026-05-08

The Patterns view (Phase 10) shows the latest cluster snapshot — recurring themes across the firm's audit portfolio. Phase 22 makes the time dimension visible: for each filter set, the current snapshot is automatically compared against the one captured immediately before it. The Patterns tab now has a "Show diff vs previous snapshot" toggle that classifies every cluster as added / grew / shrank / unchanged / removed and surfaces member-overlap percentages.

This is the compound-knowledge moat made tangible: a firm that has run AuditForge for six months can show how its portfolio of recurring patterns evolves as remediations land and new themes emerge. The investor-brief framing ("the 100th audit sees the prior 99") becomes a screenshot rather than a claim.

Storage model

Per filter set (make_filter_key(firm_id, primitive, severity, canonical_only)), the platform now keeps two snapshot files:

auditforge/portfolio_clusters/<filter_key>.json           # current
auditforge/portfolio_clusters/<filter_key>.previous.json  # one snapshot back

Each call to POST /findings/portfolio-clusters/recompute does a copy-then-overwrite: the existing current file is moved to .previous.json, and the new result replaces .json. Only one prior snapshot is retained — diff is single-hop, not full-history. Full history would multiply storage and require careful retention-policy decisions; single-hop covers the dominant ROI case ("what changed since last time we computed").

Endpoint

GET /auditforge/findings/portfolio-clusters/diff
    ?firm_id=...&primitive=...&severity=...&canonical_only=true

Same auth and firm-scoping as the existing portfolio-clusters endpoints (admin-token sees all; per-user session auto-scopes to firm). Returns:

{
  "filter_key": "abc123...",
  "has_current": true,
  "has_previous": true,
  "current_generated_at": "2026-05-08T12:00:00Z",
  "previous_generated_at": "2026-05-01T12:00:00Z",
  "summary": {
    "added": 1, "removed": 1, "grew": 2, "shrank": 0, "unchanged": 4
  },
  "entries": [
    {
      "change": "grew",
      "current_id": "c-abc",
      "previous_id": "c-prev",
      "current_theme": "Flow-down gaps in 'Subcontractor Cybersecurity'…",
      "previous_theme": "Flow-down gaps to subs",
      "current_size": 7,
      "previous_size": 4,
      "current_engagement_count": 5,
      "previous_engagement_count": 3,
      "member_overlap": 0.667
    },
    ...
  ]
}

has_current=false means no recompute has ever run for these filters. has_previous=false (with has_current=true) means this is the first snapshot — recompute again later to capture a second snapshot for diff.

Matching algorithm

The hard part of cluster-diff: an Opus call may rephrase a theme between recomputes, so we can't match purely by theme string. Phase 22 uses finding-id Jaccard overlap:

For each pair of (current_cluster, previous_cluster): 

jaccard = |current.member_finding_ids ∩ previous.member_finding_ids|
        / |current.member_finding_ids ∪ previous.member_finding_ids|

Pairs with jaccard ≥ 0.30 are candidates. Greedy assignment: sort candidates by jaccard descending, accept matches until each cluster is matched at most once. Unmatched current clusters become added; unmatched previous clusters become removed. Matched pairs classify as grew / shrank / unchanged based on member-count delta.

The 0.30 threshold is conservative — two clusters need to share roughly a third of their member findings to count as "the same pattern over time." Lowering it would over-match (any two clusters that happen to share one finding would link); raising it would under-match (real cluster evolutions where the LLM swapped 2-3 members would split into added/removed pairs).

UX

A new "Show diff vs previous snapshot" button sits below the result summary in the Patterns tab. When enabled:

  • The cluster grid is replaced by a diff panel
  • Summary chips: 1 NEW · 2 GREW · 0 SHRANK · 4 STEADY · 1 RESOLVED
  • Each entry has a colored badge (green/blue/amber/gray/red), the current theme, the prior theme (if it changed), member-count delta, engagement-count delta, and the overlap percentage

Toggling off restores the cluster grid.

Files

  • app/auditforge/portfolio_clusters.py — compute_cluster_diff, load_previous_result, _persist_previous helpers; save_cached_result now captures the prior snapshot before overwriting
  • app/auditforge_endpoints.py — GET /findings/portfolio-clusters/diff
  • frontend/src/api/auditforge.ts — getPortfolioClustersDiff, ClusterDiffResponse types
  • frontend/src/components/PortfolioClusters.tsx — toggle + ClusterDiffPanel component
  • tests/test_auditforge_cluster_diff.py — 11 unit tests covering matching invariants, threshold, summary counts, edge cases