Insights

# Insights

The Insights page (`/insights`) is a product-specific analytics dashboard built on top of the platform's charting primitives. It surfaces economics across your AI transformation pipeline: where value sits, how quickly it pays back, and which items deserve immediate attention.

## Overview

Insights is powered by `getInsightsSummary()` from `features/custom/server/dashboard-data.ts`, which aggregates data from your opportunity records. The page is part of `features/custom/` -- the product-specific layer -- so it reflects your organization's configured opportunity data type.

### What You'll See

- **Portfolio snapshot** -- Net annual value headline, with three KPI tiles (annual value, estimated investment, average payback months)
- **Quick wins panel** -- Top 5 items ranked by fast payback, strong execution fit, and high score, with direct links
- **Pipeline funnel** -- Record count by stage (idea, assessed, approved, deployed)
- **Category chart** -- Distribution of opportunities by category
- **Effort vs. value chart** -- Items grouped by implementation effort level (low, medium, high)
- **Payback band chart** -- Items grouped by payback horizon (under 3 months, 3-6 months, 6-12 months, over 12 months)
- **Opportunity map** -- Bubble chart with payback (x), annual value in $K (y), and weighted score (bubble size)
- **Top opportunities table** -- Top 10 ranked by annual value with direct record links
- **Action paths** -- Quick links to the Priority Matrix tool, ROI Calculator tool, and AI chat for rollout planning

## How It Works

### Data Aggregation Pipeline

The `getInsightsSummary()` server action performs a single-pass aggregation over all opportunity entities:

1. **Fetch** -- Queries all entities with `entity_type_slug: "opportunity"` for the active tenant (up to 5000)
2. **Enrich** -- For each opportunity, extracts economics via `getOpportunityEconomics()`:
   - `annualValue` -- parsed from `estimated_annual_value` content field
   - `implementationCost` -- parsed from `implementation_cost` content field
   - `paybackMonths` -- computed as `(implementationCost / annualValue) * 12`
   - `implementationEffort` -- string from `implementation_effort` field
   - `score` -- from `metadata.weighted_score`
3. **Aggregate** -- Single iteration builds multiple breakdowns simultaneously:
   - Pipeline stages (using `STAGE_ORDER` from opportunity constants)
   - Categories (from `content.category`)
   - Effort levels (low, medium, high, unknown)
   - Payback bands (under 3mo, 3-6mo, 6-12mo, over 12mo, TBD)
4. **Derive** -- Computes summary metrics:
   - `realizedValue` -- sum of annual value for deployed/realized stages
   - `realizationRate` -- percentage of total value that has been realized
   - `avgScore`, `avgPaybackMonths` -- means across all opportunities with data
5. **Rank** -- Builds ranked lists:
   - `topByValue` -- top 10 by annual value (tiebreak by score)
   - `quickWins` -- top 5 by composite score: `score * 20 + annualValue / 5000 - paybackMonths * 8 - effortRank * 10`
   - `opportunityMap` -- top 24 for bubble chart (sorted by score, then value)

### Custom Server Actions

The data pipeline lives entirely in `features/custom/server/`:

- **`dashboard-data.ts`** -- `getInsightsSummary()` is the main aggregation function. Returns the full `InsightsSummary` type with all breakdowns. This is a server action called from the Insights page component.

- **`dashboard-metrics.ts`** -- Pure utility functions:
  - `parseNumericValue(raw)` -- parses dollar amounts, handles K/M/B suffixes, strips formatting
  - `getOpportunityEconomics(content)` -- extracts and computes economics from entity content
  - `getOpportunityPaybackBand(months)` -- classifies payback into time bands
  - `getOpportunityEffortRank(effort)` -- converts effort string to numeric rank (low=1, high=3)

### Relationship to Extraction Results

The Insights page derives its data from entity content fields that are typically populated by the extraction system. When agents extract `estimated_annual_value`, `implementation_cost`, `implementation_effort`, and other fields, those values flow into the Insights aggregation on the next page load.

This creates a feedback loop: agents extract structured data from unstructured sources, the data populates entity fields, and Insights visualizes the aggregate picture. Improving extraction quality directly improves Insights accuracy.

### Category and Stage Breakdowns

Category values come from the `content.category` field on opportunities. They are not hardcoded -- the chart adapts to whatever categories exist in the data.

Pipeline stages use `STAGE_ORDER` from `features/custom/lib/opportunity-constants.ts`, which defines the canonical progression. All standard stages appear in the pipeline chart even if they have zero items, ensuring a consistent funnel shape.

## For Agents

Agents interact with the underlying opportunity records, not the Insights page directly:

- **`searchEntities("opportunity")`** -- List opportunities with field values
- **`getEntityStats("opportunity")`** -- Aggregate counts and field summaries
- **`updateEntity(id, { status: "approved" })`** -- Advance an item through the pipeline, which updates the funnel chart on next page load

The `insights` page derives its data entirely from the `opportunity` entity type -- keeping agents and the UI in sync automatically.

## Manual Test Script

### Prerequisites

- Logged in as member or above
- At least 5 opportunity records exist, preferably with `estimated_annual_value`, `implementation_cost`, and `implementation_effort` populated

### Happy Path

1. **Navigate to Insights**
   - Go to `/insights`
   - Expected: Portfolio snapshot card visible; KPI tiles show numeric values (not `---`)

2. **Verify quick wins panel**
   - Expected: Up to 5 items listed; each shows category, effort, and payback badges
   - Click one -- expected: navigates to the opportunity detail page

3. **Check pipeline funnel**
   - Expected: Funnel shows stages with item counts; widest bar is the earliest stage
   - Click a stage -- expected: navigates to the opportunity list filtered by that stage

4. **Check opportunity map**
   - Expected: Bubble chart renders with labeled axes; hovering a bubble shows tooltip with name, value, payback
   - Click a bubble -- expected: navigates to that opportunity's detail page

5. **Verify action paths**
   - Click **Run the Priority Matrix** -- expected: navigates to `/tools/priority-matrix`
   - Click **Ask for a recommended rollout plan** -- expected: navigates to `/chat`

### Edge Cases

- **No opportunities:** Empty state card with a "Identify opportunities with AI" button pointing to `/chat`
- **All opportunities missing payback data:** Average payback KPI shows `---` (not NaN or an error)
- **Single opportunity:** Charts render with one data point rather than blank

### Regression Checks

- [ ] `formatCurrencyCompact()` renders values like `$1.2M` not raw numbers
- [ ] Quick wins panel does not exceed 5 items
- [ ] Category chart handles custom category values (not just hardcoded labels)
- [ ] Bubble chart `z` (size) uses `Math.max(score, 1)` -- never zero-size bubbles

## Troubleshooting

| Symptom | Likely Cause | Fix |
|---|---|---|
| All KPIs show `---` | Opportunities exist but fields are unpopulated | Run extraction to populate `estimated_annual_value`, `implementation_cost` |
| Pipeline funnel is empty | No opportunities have a `status` field set | Check schema; ensure `status` is a defined field with stage values |
| Charts render but wrong data | Opportunity records use different field names | Check `dashboard-data.ts` for the field keys it reads |
| Page crashes with no data | `getInsightsSummary` threw an error | Check server logs; ensure the `opportunity` entity type exists |
| Quick wins empty despite data | No items have both annual value and payback months | Ensure `implementation_cost` is populated so payback can be computed |

## Related Modules

- **Entity System** (`features/entities/`) -- opportunity records are the data source
- **Extraction** (`features/entities/extraction/`) -- populates the fields that Insights aggregates
- **Charts** (`features/charts/`) -- BubbleChart used for the opportunity map
- **Custom Tools** (`features/custom/tools/`) -- Priority Matrix and ROI Calculator linked from action paths
- **Dashboard** -- general-purpose dashboard; Insights is the product-specific counterpart