Knowledge Loops

## Overview

A **knowledge loop** is a reusable, typed description of a knowledge-production
cycle a tenant runs over and over:

```
sources → synthesis → evidence-backed records → review / human gate → surfaces → eval
```

It is **not a new primitive**. A `KnowledgeLoopDefinition` is a legible authoring
lens over the existing [Loop OS](/docs/features/loops): it compiles down to a
`GoalLoopTemplate` / `GoalSystemSpec`, so it rides the existing goal-loop runner,
rubric eval (`runRubricJudge`), human-gate, and closure-scoring machinery with no
new table, engine, or scorer (see [ADR-0057](/) and the
[`no-parallel-systems`](/) rule).

The contract adds the two things a raw `GoalLoopTemplate` lacks:

1. a **six-stage vocabulary** with an explicit evidence convention, and
2. a **`safetyClass` / `domain`** taxonomy with a validator that forces a human
   review gate on clinical/regulated knowledge — so clinical claims can never ride
   a generic-docs loop.

Tenants declare loops as **data** on `TenantModule.knowledgeLoops`. DOC'S/Praxium
ships `docs-clinical-evidence` (clinical) + `docs-operating-knowledge`
(non-clinical SOPs); Sprinter ships `sprinter-product-knowledge`. None of it puts
tenant slugs in platform code.

## Key concepts

### The six stages

| Stage          | Field        | Maps onto (`GoalSystemSpec`)        | Loop-OS closure fact |
| -------------- | ------------ | ----------------------------------- | -------------------- |
| **Sources**    | `sources`    | `workSources` (+ `inputs`)          | `hasSensor`          |
| **Synthesis**  | `synthesis`  | `actions` / `skills`                | `hasOutputContract`  |
| **Records**    | `records`    | `entityTypes` + `outputs`           | `hasOutputContract`  |
| **Review gate**| `reviewGate` | `humanGates`                        | `hasCritic`          |
| **Surfaces**   | `surfaces`   | `outputs[].publishDestinations`     | —                    |
| **Eval**       | `eval`       | `scorecard` (+ `impactMetrics`, `learningPath`) | `hasCritic` / `hasImproveHook` |

`expectedClosure(def)` projects a definition onto the six
[CLOSURE_CRITERIA](/docs/features/loops#closure_criteria-6-facts) so you can see
the closure score a well-formed loop is _designed_ to reach (a persistent
knowledge loop reaches 6/6).

### Safety class & domain

`safetyClass` is the loop's safety posture and drives the validator:

- `clinical` — health/medical claims. **Mandatory** human (clinician) gate.
- `regulated` — legal/financial/compliance claims. **Mandatory** human gate.
- `standard` — internal/product/operating knowledge. Human gate optional.

`domain` (`clinical | research | operations | product | ai | general`) tags the
subject. The validator's **`clinical-domain-fence`** rejects `domain: "clinical"`
on a `standard` loop — the guarantee that clinical claims stay out of generic or
product docs.

### Evidence-backed records

`records.evidenceFields` names the fields that carry citations / provenance
(e.g. `source_count`, `source_quality_avg`, `evidence`, `source_urls`). Evidence
fields are **required** for `clinical`/`regulated` loops and recommended for the
rest. At runtime these ride the platform's field-level evidence primitives
(`entity_responses.field_meta` → `FieldMetaSource`; see
[Response System](/docs/features/response-system)).

### Research iteration (optional Stage 7)

A plain knowledge loop is a single production pass: `sources → records`. Setting
the optional `researchIteration` block turns it into a **Karpathy-style
auto-research loop** — each pass reads the prior pass's eval/coverage, files the
NEXT pass's work as tagged gaps, dedupes against what the graph already knows, and
halts on a bounded budget. It is what makes a knowledge loop _self-feeding_.

```
… → eval → file the next gaps → (gaps re-enter as sources) → repeat, bounded
```

| Part            | Field                          | Maps onto (`GoalSystemSpec`)                                    |
| --------------- | ------------------------------ | -------------------------------------------------------------- |
| Gap generation  | `gapGeneration`                | a 2nd `outputs[]` (gap batch) + `entityTypes` + `learningPath` + `inputs` directive |
| Novelty / dedupe| `novelty`                      | `impactMetrics` (`novelty_ratio`) + a dedupe `inputs` directive |
| Stop criteria   | `stop`                         | `stopConditions` strings                                        |

- **`gapGeneration`** — `{ entityTypeSlug, label, feedbackTag, instructions, maxPerIteration? }`.
  Each iteration files its next-search work as `feedbackTag`-tagged records
  (usually `task`). The `feedbackTag` **must** be referenced by a `sources[].sourceRef`
  so the gaps re-enter — this is the loop-closer the validator enforces.
- **`novelty`** — `{ dedupeKey, minNoveltyRatio? }`. `dedupeKey` names the
  dimensions (flat fields OR relation coordinates like `["modality", "condition"]`)
  whose values identify a duplicate; the agent refreshes the existing record
  instead of creating a new one. `minNoveltyRatio` is the saturation floor.
- **`stop`** — `{ maxIterations?, coverageTarget?, saturationRounds? }`. The
  bounded auto-research budget. `coverageTarget.metric` must reference an
  `eval.impactMetrics[].name`.

> **Stop semantics.** The compiler emits the budget into BOTH `spec.stopConditions`
> (a hard halt — but the goal-loop runner only consults these for `terminal`-mode
> loops) AND `spec.inputs` (an advisory "Stop budget: …" directive). Since
> knowledge loops are `persistent`, the inputs path is what reaches the agent:
> `buildBriefMarkdown` renders `## Inputs`, so the agent self-regulates against the
> budget every iteration. A future enhancement could make the runner consult
> `stopConditions` for persistent loops too (a goal-loop-runner change, separate PR).

This is not a new engine — it is the typed expression of the loop DOC'S already
runs in prose (the `research-coverage-sweep` → `research-gap` task →
`research-gap-dispatch` cycle). A loop without `researchIteration` is unchanged.

## How it works

### Validation

`validateKnowledgeLoop(def)` parses against the schema and the cross-field safety
invariants, returning structured issues:

| Code                     | Severity | Fires when                                                            |
| ------------------------ | -------- | -------------------------------------------------------------------- |
| `safety-human-gate`      | error    | clinical/regulated loop has no human gate                            |
| `clinical-domain-fence`  | error    | `domain: clinical` on a non-clinical safety class                   |
| `safety-status-lifecycle`| error    | clinical/regulated records lack `statusField` + `approvedValue`     |
| `evidence-required`      | error    | clinical/regulated records have no `evidenceFields`                 |
| `eval-unscored`          | error    | no agent-scored criterion AND no human gate — nothing can grade     |
| `evidence-recommended`   | warn     | standard loop has no evidence fields                                 |
| `eval-human-only`        | warn     | no numeric criterion — `runRubricJudge` will skip                   |
| `eval-prompt-missing`    | warn     | no `evaluationPrompt` — the judge is under-constrained              |
| `eval-weights`           | warn     | criterion weights don't sum to 1.0                                  |
| `queue-record-mismatch`  | warn     | review queue targets a different entity type than the records       |
| `research-gap-feedback-unwired`     | error | `gapGeneration.feedbackTag` is not a `tag=` filter on any `sources[].sourceRef` (gaps never re-enter) |
| `research-gap-feedback-type-mismatch` | error | a source carries the tag but reads a different entity type than `gapGeneration.entityTypeSlug` (gaps aren't re-claimed) |
| `research-coverage-metric-unknown`  | warn  | `stop.coverageTarget.metric` matches no `eval.impactMetrics[].name` (and isn't the compiler-synthesized `novelty_ratio`) |
| `research-novelty-metric-conflict`  | warn  | a pre-declared `novelty_ratio` impact metric disagrees with `novelty.minNoveltyRatio` |
| `research-unbounded`                | warn  | `researchIteration` declares no stop criteria — auto-research is unbounded |
| `research-saturation-needs-floor`   | warn  | `stop.saturationRounds` set without `novelty.minNoveltyRatio` to measure against |

`validateAllTenantKnowledgeLoops()` runs this across every registered tenant — a
conformance gate (a broken declaration fails a unit test).

### Compile to the Loop OS

`knowledgeLoopToGoalLoopTemplate(def)` produces a `GoalLoopTemplate` whose `spec`
round-trips through `goalSystemSpecSchema`. From there the loop is indistinguishable
from any other goal loop: it is materialized by the wizard / template path, run by
the `goal-checkin` action-tick spine, graded by `runRubricJudge`, and closure-scored
by `scoreLoopClosure`.

#### Reachable through the canonical template registry (ADR-0057)

Compiled tenant knowledge loops are registered into the platform goal-loop
template registry through a provider seam, so they resolve through the same
`getGoalLoopTemplate` / `listGoalLoopTemplates` / `createGoalFromTemplate` tools
as the built-in templates — no separate lookup path:

- `registerGoalLoopTemplateProvider(provider)` in
  `features/entities/lib/goal-loop-templates.ts` is the platform seam (it never
  imports `features/custom`). `getGoalLoopTemplate(slug)` consults the static
  `GOAL_LOOP_TEMPLATES` first, then the registered provider.
- `features/custom/server/register-knowledge-loop-templates.ts` (wired into
  `features/tools/bootstrap.ts`) compiles each tenant loop with
  `knowledgeLoopToGoalLoopTemplate` and feeds the seam. It reads the
  `*_KNOWLEDGE_LOOPS` data files directly (pure data, no tenant UI graph) to keep
  the tool cold-start path lean; a drift test pins that list to
  `getAllTenantKnowledgeLoops()`.
- **Static templates win on slug collision** — a tenant loop can never shadow a
  built-in template. (This is why the deprecated hardcoded `docs-evidence-research`
  keeps precedence over its `docs-clinical-evidence` successor until it is
  deleted.)

### Reading & summarizing

```ts
import {
  getTenantKnowledgeLoops,
  getTenantKnowledgeLoopCatalog,
} from "@/features/custom/lib/knowledge-loops";

getTenantKnowledgeLoops("docs"); // KnowledgeLoopDefinition[]
getTenantKnowledgeLoopCatalog("docs"); // KnowledgeLoopSummary[] (view-ready)
```

`summarizeKnowledgeLoop(def)` returns a `KnowledgeLoopSummary` with per-stage
completeness, the designed closure score, the real review queue
(`{ entityTypeSlug, statusField, pendingValue }`), and a primary CTA — view-ready
data for a catalog surface. `formatKnowledgeLoopBrief(def)` emits a versioned
markdown brief for handing the loop to an agent.

## How a tenant adds a knowledge loop

1. **Pick the primitives it composes.** Identify the existing slugs for each
   stage: source entity types, the synthesis action keys / skills, the record
   entity type (and its status + evidence fields), the review criteria set, and
   the surfaces (workspace / view / entity-list). Declare a new entity type only
   if no record type fits (Sprinter added `pf_knowledge`).

2. **Author the definition as data** in a client-safe sibling file, importing the
   type only:

   ```ts
   // features/custom/tenants/<slug>/knowledge-loops.ts
   import type { KnowledgeLoopDefinition } from "@/features/loops/lib/knowledge-loop";

   const myLoop: KnowledgeLoopDefinition = {
     slug: "<slug>-<kind>",
     name: "...",
     description: "...",
     goal: "...",
     domain: "product",
     knowledgeKind: "product-decision",
     safetyClass: "standard",
     category: "venture",
     businessObjective: "growth",
     sources: [ /* ... */ ],
     synthesis: { instructions: "...", actionKeys: [], skills: [], autonomyTier: "draft" },
     records: { entityTypeSlug: "...", label: "...", evidenceFields: ["..."], statusField: "status", draftValue: "draft", approvedValue: "published" },
     reviewGate: { humanApprovalRequired: true, gateLabel: "...", requiredBefore: "...", reviewerRole: "editor" },
     surfaces: [ /* ... */ ],
     eval: { criteria: [ /* ≥1 agent-scored */ ], passThreshold: 75, evaluationPrompt: "..." },
   };

   export const MY_KNOWLEDGE_LOOPS: readonly KnowledgeLoopDefinition[] = [myLoop];
   ```

3. **Wire it onto the tenant module:**

   ```ts
   // features/custom/tenants/<slug>/index.ts
   import { MY_KNOWLEDGE_LOOPS } from "./knowledge-loops";
   // ...
   knowledgeLoops: MY_KNOWLEDGE_LOOPS,
   ```

4. **Add a declaration test** asserting `validateKnowledgeLoop` returns zero
   errors and that the loop compiles
   (`knowledgeLoopToGoalLoopTemplate` → `goalSystemSpecSchema.parse`). The
   cross-tenant `validateAllTenantKnowledgeLoops` test then covers it too.

5. **Push any new entity type** to the DB with `pnpm tenant:push` (a deploy step).

### Making it a self-feeding research loop

Add the optional `researchIteration` block when the loop should iterate — file
its own next searches, dedupe, and halt on a budget:

```ts
researchIteration: {
  gapGeneration: {
    entityTypeSlug: "task", // gaps land as tasks…
    label: "Research-gap task",
    feedbackTag: "research-gap", // …tagged so a source re-claims them
    instructions: "From the coverage scorecard, file a gap task for each thin/stale cell.",
    maxPerIteration: 8,
  },
  novelty: { dedupeKey: ["modality", "condition"], minNoveltyRatio: 0.5 },
  stop: {
    maxIterations: 12,
    coverageTarget: { metric: "modality_condition_cells_with_min_claims", target: 90 },
    saturationRounds: 2,
  },
},
```

**The one hard rule:** a source must re-claim the gaps. Add (or reuse) a source
whose `sourceRef` references the `feedbackTag` — e.g.
`{ key: "open-gaps", kind: "custom", sourceRef: "entities:task?tag=research-gap&status!=completed", entityTypeSlug: "task" }`.
The validator's `research-gap-feedback-unwired` error fires if you forget, because
a loop that files gaps nothing re-claims is an open pipeline, not a closed loop.

## How artifacts flow to views & pages

- **Records** are entities of `records.entityTypeSlug`. The platform renders them
  generically (schema-driven cards / list / detail) at `/entities`, and any
  tenant [view](/docs/features/view-system) or [custom page](/docs/features/custom-pages)
  can bind a `DataSource` to them — no per-loop UI required.
- **The review queue** is `records.entityTypeSlug` filtered to
  `reviewGate.queueRef.pendingValue` on `statusField` (e.g. evidence-claims where
  `status = under_review`). DOC'S surfaces this through the
  clinician-review-queue / the Operator workspace's Knowledge Hub.
- **Surfaces** name where approved records live — a workspace, a view, a page, or
  the auto entity-list. `summarizeKnowledgeLoop().primaryCta` picks the right link
  (the review queue when gated, else the primary surface).

## Sprinter: the product-knowledge operating surface

The `sprinter-product-knowledge` loop ships a purpose-built operating surface —
two code-resident custom pages in the Sprinter tenant module — so the loop is
something Tyler/Tai actually run, not just a definition. Both are pure-render
Server Components fed by a single `records` DataSource (the full `pf_knowledge`
corpus); the pure view-model
(`features/custom/tenants/sprinter/lib/product-knowledge-model.ts`) partitions
the records client-side. Empty states are honest — the scorecard and map render
accurate zeros until records land.

- **Product Knowledge cockpit** (`/t/sprinter/p/product-knowledge`) — the
  operating view:
  - **Loop readiness scorecard** — the primitive's designed closure (0–6) plus
    LIVE progress against the loop's own declared `eval.impactMetrics` targets
    (published count, % evidence-backed) and taxonomy coverage. This is the
    _knowledge-loop's_ readiness, distinct from the org-wide AI-readiness
    scorecard in [`features/ai-native`](/docs/features/ai-native) — they don't
    share data.
  - **Knowledge map** — a kind × status coverage matrix flagging which
    `knowledge_kind`s have zero published records (the documentation gaps).
  - **Review inbox** — the human gate: records at `in_review`, longest-waiting
    first, each deep-linking into Admin where the editor publishes or archives.
- **Evidence board** (`/t/sprinter/p/product-knowledge-evidence`) — the library
  view: the published, evidence-backed **product narrative** (published records
  grouped by kind in reading order, each annotated with an evidence-strength
  signal) plus the **reusable-demo queue** (`demo-asset` records, ready demos
  first).

Records (`pf_knowledge`) carry a `knowledge_kind` taxonomy — capability,
customer-proof, case-study, product-decision, objection, experiment,
agent-playbook, demo-asset — and a `draft → in_review → published → archived`
lifecycle. The page routes are seeded by the idempotent migration
`20260615000000_seed_sprinter_product_knowledge_pages.sql`; worked example
records ship as a tested data module
(`data/product-knowledge-seed-records.ts`) written by the dry-run-safe
`scripts/seed-sprinter-product-knowledge.ts --apply`.

This mirrors the DOC'S/Praxium operating model (typed loop → real surfaces →
human gate) without conflating with it: the `standard` safety class + `product`
domain keep it cleanly on the non-clinical side of the validator's fence.

## Examples

| Loop                          | Tenant   | Records         | Safety     | Gate                          |
| ----------------------------- | -------- | --------------- | ---------- | ----------------------------- |
| `docs-clinical-evidence`      | docs     | `evidence-claim`| clinical   | clinician approves promotion  |
| `docs-operating-knowledge`    | docs     | `sop`           | standard   | operator reviews before active|
| `sprinter-product-knowledge`  | sprinter | `pf_knowledge`  | standard   | editor approves before publish|

## API reference

```ts
// features/loops/lib/knowledge-loop.ts
KnowledgeLoopDefinitionSchema; // zod schema (.strict())
KnowledgeResearchIterationSchema; // optional Stage 7 (.strict())
validateKnowledgeLoop(input): { ok, issues } // incl. research-* invariants
summarizeKnowledgeLoop(def): KnowledgeLoopSummary // .research is non-null for research loops
expectedClosure(def): ExpectedClosure
formatKnowledgeLoopBrief(def): string // adds "## 7. Research iteration" when present

// features/loops/lib/knowledge-loop-template.ts
knowledgeLoopToGoalLoopTemplate(def): GoalLoopTemplate

// features/custom/lib/knowledge-loops.ts
getTenantKnowledgeLoops(tenantSlug)
getTenantKnowledgeLoopCatalog(tenantSlug)
getAllTenantKnowledgeLoops()
validateAllTenantKnowledgeLoops()
computeKnowledgeLoopProgress(def, records) // generic, shape-agnostic corpus counts

// features/custom/tenants/sprinter/lib/product-knowledge-model.ts (Sprinter surface)
readKnowledgeRecord(entity)        // EntityRecord → normalized ProductKnowledgeRecord
buildReadinessScorecard(def, recs) // closure + live impact-metric progress + coverage
buildKnowledgeMap(recs)            // kind × status coverage matrix + gaps
buildReviewInbox(recs)             // in_review queue (FIFO)
buildEvidenceBoard(recs)           // published, narrative-ordered, strength-annotated
buildDemoQueue(recs)               // demo-asset records, ready-first
```

## Design decisions

- **Lens, not parallel system.** A knowledge loop compiles to `GoalLoopTemplate`.
  There is no `knowledge_loops` table and no second runner (ADR-0057,
  `no-parallel-systems`).
- **Safety is typed and enforced.** The clinical fence + mandatory-gate
  invariants are unit-tested, not convention.
- **Metadata field, not a `DeclarationKind`.** `knowledgeLoops` rides
  `TenantModule` like `offers`/`quizzes`; adding a declaration kind would touch
  the cascade, CLI, and admin sync for no gain.
- **Pure-data declarations.** Tenant files `import type` only, so no zod/runtime
  leaks into the client bundle.

## Related modules

- [Loops](/docs/features/loops) — the Loop OS this compiles onto
- [Research Library](/docs/features/research-library) — the worked clinical pattern
- [Evals](/docs/features/evals) — `runRubricJudge` grades the output
- [Response System](/docs/features/response-system) — field-level evidence primitives
- [Tenant modules](/) (`.claude/rules/tenant-modules.md`) — where loops are declared