Documentation source
Documentation source
Documentation source
Living SOPs as entities — imported from documents, run on the action spine, scored for delegation, and bridged into the AI opportunity portfolio.
# SOPs & Process OS
The Process OS turns an organization's standard operating procedures into living records: imported from real documents, executed as runs with per-step live status, scored for human→agent delegation readiness, and mined for AI-automation opportunities. It is built entirely on the platform's six primitives — no SOP-specific engine, tables, or tools exist (ADR-0055).
## Key concepts
- **SOP** — a canonical `sop` entity. Status lifecycle `draft → active → archived` lives in content; publishing a draft is a standard entity status update.
- **Step** — a child `task` entity (`parent_id` = the SOP, 1-based `content.sequence_order`). Each step carries a `delegation_state` (`human_only → trialing → agent_supervised → agent_autonomous`) — the same field the Delegate flow promotes — plus `due_rule` (a reusable due-date rule anchored to an occurrence or business period), `reviewer_id`, `evidence_requirements`, and `tools`. A step's embedded `content.steps[]` sub-steps (`{ref, title, instructions, procedure_refs[], delegation_state?, images?}`) are the click-level breakdown shown in the process flow.
- **Procedure** — a first-class `procedure` system entity (2026-07-12), `parent_id` = its owning SOP: a reusable, screenshot-rich, click-by-click how-to (e.g. "§4.8.5 — Post journal entry") with a stable `ref`, `title`, `group`, `systems`, and ordered `steps[]` (`{n, text, group?, images: [{url, caption?}]}`). A step's sub-steps drill into a procedure by exact `ref` match via `procedure_refs`, materialized as the typed `task —uses_procedure→ procedure` relation. Screenshots stay image refs on a procedure step (`{url, caption}`) — never their own entity records (Tyler's ruling); procedures are reference, not work — no status machine, no owner, no run state.
- **Occurrence** — the stable identity of one SOP run: a keyed parent session (`sessions.metadata.sop_occurrence_key`, format `sop-occurrence:v1:<sha256>`), one per tenant per scheduled period or per ad-hoc request (`buildSopOccurrenceKey`, `features/sops/occurrence.ts`). A partial unique index enforces one canonical parent occurrence per tenant/key while leaving child step sessions and retries free to share it (`supabase/migrations/20260712000040_sop_occurrence_key.sql`). Re-running the same period never creates a duplicate; reopening a run always finds the same one.
- **Run** — `ensureSopRunActions` compiles the SOP into a `sop-run-<sopId>` parent action plus `sop-step-<taskEntityId>` child actions chained with `depends_on`, each carrying a compiled `SopStepSnapshotV1` (instructions, input contract, output plan, verification, dependencies, evidence requirements, approval policy, and — when a sub-step drills into a procedure — its **procedure pins**, see below). `runSop` triggers the tree through `trigger_task_atomic`; human steps wait in `waiting_human` and surface an evidence checklist against their `evidenceRequirements`; agent steps execute via the session executor. `refreshScheduledSopRunActions` recompiles the tree immediately before a scheduled (cron) fire so template edits land on the _next_ occurrence, not the in-flight one.
- **Procedure pin** — captured at compile time (`ensureSopRunActions` → `resolveStepProcedurePins`): each step's sub-step `procedure_refs` are resolved against the SOP's child procedures and pinned onto the step snapshot as `{procedureEntityId, ref, contentHash, entityUpdatedAt}` (`SopProcedurePinV1`, `features/sops/occurrence.ts`). A pin permanently records _which version_ of a procedure a run executed against. An unresolved ref is skipped and logged — never thrown; a missing procedure must not break a compile.
- **Procedure drift** — a per-step signal (`SopStep.procedureDrift` / `SopRunAction.procedureDrift`) computed by the read-model: true when any of a step's pinned `contentHash` values no longer match the _live_ procedure's hash (same `buildSopTemplateHash` helper used at pin time, so the comparison is byte-comparable — no extra query beyond the already-loaded procedure rows). Surfaces as an outline "Procedure updated since this run" badge in `run-step-action-details.tsx`.
- **Routine** — the parent run action flipped to a cron trigger. No separate scheduler.
- **Opportunity** — a `use_case` entity captured from a delegatable step by the deterministic `sop-to-opportunity` ActionDefinition, linked back via a `sop —supports→ use_case` relation and ranked by the existing AI portfolio.
- **Presentation fields (2026-07-11)** — optional content on the `sop` and `task` system entity types for governed, document-anchored SOPs: SOP `version`, `revision_date`, `focus_areas` (object array `{title, detail}`), `related_documents` (`{name, description, location}`), `related_accounts` (`{entity, account, responsible}`), `raci` (`{activity, responsible, reviewer, other}`), `roles` (`{role, person, responsibility}`), `source_document_id`; task `timing`, `owner_role`. All optional and additive — legacy SOPs are unaffected. `defineSystemEntityType` supports these via a new `items` JSON-Schema-per-array-field option (`features/entities/system-types/define-system-entity-type.ts`), which the writer gate uses to declare every key a seed writes.
## How it works
```
document ──import skill──▶ sop (draft) + ordered task steps + procedure entities
│ publish (status update)
▼
sop (active) ──ensureSopRunActions──▶ sop-run action tree
│ (resolves procedure_refs → procedure pins) │ runSop / cron
▼ ▼
read-model projection ◀────────────────────────── sessions (per-step state,
(+ procedureDrift vs live procedure) occurrence-keyed)
│
▼
/api/sops ──▶ SOP library UI (steps · procedures · runs · opportunities)
```
- **Read model** (`features/sops/server/read-model.ts`): `getSopDetail` projects entities + actions + sessions into `SopSummary` / `SopDetail`, including per-step live state (`todo / now / done / blocked`), the presentation fields above, the SOP's child `procedure` entities, and the run handle (`SopDetail.run`). It also computes live procedure content hashes once (`computeLiveProcedureHashes`) and reuses them for both the checklist projection and the run-action projection (`projectOccurrenceRunActions` in `features/sops/lib/project-run-actions.ts`) to derive `procedureDrift` per step. Served by `GET /api/sops` and `GET /api/sops/[id]`.
- **Execution spine** (`features/sops/server/run-sop.ts`, `features/sops/occurrence.ts`, `features/sops/lib/compile-step-contract.ts`): `ensureSopRunActions` loads the SOP's steps _and_ its child procedures, resolves each step's sub-step `procedure_refs` into pins (`resolveStepProcedurePins`), and compiles the full step contract (instructions, source references, target scope, output plan, verification, dependency DAG, tool permissions, resolved executor, approval policy, due rule, evidence requirements) onto the action's metadata. `runSop` triggers the compiled tree under a stable occurrence key; `features/actions/lib/trigger-session-plan.ts` is the actual snapshot-assembly seam that threads the compiled contract (including procedure pins) into the persisted `sopStepSnapshot` on each child session.
- **SOP detail surface** (`features/sops/surfaces/`): composed from gated sections that render only when their field is present — `sop-hero.tsx` (identity, version/revision, systems, "View source document" provenance link), `sop-focus-areas.tsx` (scannable "Before you start" callout band), `sop-process-flow.tsx` (numbered task rail, collapsed by default, expanding to each task's embedded subtask `steps` with procedure-ref chips — gated on `hasRichProcessFlow`), `sop-roles-raci.tsx` (collapsible roles + RACI matrix), `sop-resources.tsx` (related documents + a filterable, collapsed related-accounts table). A SOP with none of these fields renders exactly as before.
- **Procedure drill-down** (`features/sops/surfaces/sop-procedure-sheet.tsx`, `sop-screenshot-gallery.tsx`, `sop-procedure-lib.ts`): the deepest level of the SOP spine — overview → task → sub-step → click-by-click procedure — rendered as a side sheet so the operator keeps the process flow behind it. Screenshots are lazy (nothing mounts until a procedure is opened); a step's images render as a capped thumbnail strip (`splitScreenshotStrip`, default 4 visible + overflow count) that opens a **procedure-wide filmstrip lightbox** — `flattenProcedureImages` flattens every screenshot across the whole procedure into one ordered sequence so prev/next crosses step boundaries, captioned by owning step (mirrors reading the source document linearly instead of one image at a time).
- **Run UX**: the SOP detail header carries a Run button (active SOPs only; disabled while a run is in flight). The Runs tab reuses the actions module's `RoutineRecentRuns` and offers the routine toggle (cron presets or custom). Step states poll every 5s while a run is active. Each step's evidence requirements render as a checklist during execution (`run-step-action-details.tsx`, `run-step-action-card.tsx`); a step with `procedureDrift: true` additionally shows an outline "Procedure updated since this run" badge.
- **Legacy compatibility**: `features/sops/server/legacy-period-run.ts` (`resolveLegacyPeriodRun`) resolves pre-occurrence `period_run` execution records into the current occurrence shape so historical SOP runs continue to render without a backfill migration.
- **Import**: the `import-sops-from-document` tenant skill (manifest-bound action, drafts-only posture) reads a document and batch-creates the SOP + parented, ordered steps + child procedures. Library entry point: "Import SOPs" → document picker → drafts rail for review/publish.
- **Opportunities**: the Opportunities tab (injected through the generic `extraTabs` prop from custom code) lists captured `use_case` rows with expected value and offers idempotent extraction; rows flow into `/ai-portfolio`.
## API reference
| Surface | Shape |
| -------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET /api/sops?status=&workstreamId=` | `{ sops: SopSummary[] }` |
| `GET /api/sops/[id]` | `{ sop: SopDetail }` (includes `steps[].state`, `steps[].procedureDrift`, `procedures[]`, `run`) |
| `POST /api/sops/import` | `{ documentId }` → 202; runs the import skill |
| `POST /api/sops/[id]/run` | 202 `{ parentSessionId }`; 409 unless `status === "active"` |
| `POST /api/sops/[id]/routine` | `{ enabled, schedule? }` → ensures the run action + sets/clears the cron trigger |
| `ensureSopRunActions(tenantId, sopId)` | compiles/recompiles the `sop-run-<sopId>` action tree, resolving procedure pins; returns `{parentActionId, stepActionIds}` |
| `refreshScheduledSopRunActions(tenantId, sopId)` | recompile-before-cron-fire variant; returns `{dispatchable: false, reason: "sop-inactive"}` instead of throwing for an archived SOP |
| `runSop(tenantId, sopId, userId)` | server seam behind the run route; triggers the compiled tree under the occurrence key |
| `buildSopOccurrenceKey(input)` / `buildSopTemplateHash(input)` | occurrence identity + content-hash primitives (`features/sops/occurrence.ts`) — the same hash function pins a procedure at compile time and re-derives its live hash for drift comparison |
| `resolveLegacyPeriodRun(...)` | adapts a pre-occurrence `period_run` into the current run shape |
| `extractSopOpportunities(sopId)` | custom server action (gates `entities.team.update`) running the `sop-to-opportunity` ActionDefinition |
## For agents
Zero dedicated SOP tools exist (D9 — skills over tools). Agents work SOPs through generic entity tools guided by three skills:
- **`manage-sops`** (platform skill) — the canonical work-model: how to read SOPs, steps, and their child `procedure` entities (`getDescendants`), the status vocabulary, delegation states, procedure traversal via `uses_procedure`, and runs-as-occurrence-keyed-sessions.
- **`import-sops-from-document`** (tenant skill + action row) — drafts SOPs from an uploaded document. Steps MUST set `parentId` in the same `batchCreateEntities` call and a 1-based `content.sequence_order`; never "Step N" title prefixes. Reusable click-by-click procedures are captured as `procedure` records (`parent_id` = the SOP), materialized to their referencing steps via the typed `uses_procedure` relation.
- **`sop-opportunity-analyst`** (platform skill) — when and how to run the `sop-to-opportunity` extraction and refine its recorded assumptions.
Per-tenant action rows are seeded by `scripts/seed-{oci,marbella}-skills.ts` (idempotent; the import action requires a LOCAL agent with `document` + `entity` tool groups).
## Design decisions
- **No parallel run system** — runs are ordinary action/session trees; routines are cron triggers on the same action (ADR-0055).
- **Procedures are a first-class entity type, not embedded JSONB** (2026-07-12, reversing an earlier `sop.content.desktop_procedures[]` design) — a procedure needs independent versioning (pinning, drift), independent identity for the `uses_procedure` relation, and reuse across steps/SOPs. It stays reference-only: no status machine, no owner, no run state — those live on the referencing task's sessions.
- **Occurrence identity is a keyed parent session, not a new run table** — a dated or ad-hoc SOP run's stable identity lives in `sessions.metadata.sop_occurrence_key`, enforced by a partial unique index (`tenant_id`, key, `parent_id IS NULL`). Extends the existing action/session primitive per ADR-0055 rather than introducing a parallel "SOP run" table.
- **Procedure pins live in the step's session-metadata snapshot (JSONB), not a run-history table** — a pin is a property of the one `SopStepSnapshotV1` already captured at compile time; it needs no independent lifecycle, query pattern, or table. `procedurePins` is optional and only ever appended, never synthesized, so existing persisted snapshots without pins still parse.
- **Screenshots are image refs, never entities** — a procedure step's `images: [{url, caption?}]` point at uploaded files; screenshots don't need independent identity, versioning, or relations, so making them entities would be an unjustified primitive.
- **Drift is computed, not stored** — `procedureDrift` is derived on read by comparing a pin's `contentHash` against the live procedure's hash using the identical `buildSopTemplateHash` function; there's no drift table or background job, and it self-corrects the moment the procedure (or the pin) changes.
- **Drafts gate publication** — the import skill writes drafts only; human review + publish is the safety boundary, not write-time approval.
- **EV is deterministic and auditable** — extraction records every default in `content.assumptions`; humans refine by editing the `use_case`.
- **Platform stays slug-free** — `use_case`/`sop` product logic lives in `features/custom/**`; the platform's only concession is the generic `extraTabs` injection point.
- **Tenants adapt, don't fork, the platform SOP surface** — Marbella's bespoke `sop-library.tsx` / `sop-detail*.tsx` were deleted (2026-07-11); `operating-partner/surfaces/sop-workbench.tsx` now adapts the platform `SopLibraryView`/`SopDetailPane`, injecting only the tenant's add-step affordance via the pane's `headerActions` prop. Prefer this pattern over a parallel tenant SOP UI.
- **Structured presentation fields are additive, never evaluated** — the object-array SOP/task fields (RACI, roles, focus areas, embedded subtask steps) are read-model projections for display; they carry no execution semantics. Runs remain ordinary action/session trees regardless of how rich the SOP's content is.
## Related modules
- [Agent system](/docs/features/agent-system) — executor, delegation, permissions
- [Tool system](/docs/features/tool-system) — generic entity tools the skills rely on
- [Skills](/docs/features/skills) — manifest bindings and `runSkill`
- [Sessions](/docs/features/sessions) — the run substrate; occurrence identity and step snapshots are session metadata
- [Actions](/docs/features/actions) — the compiled run tree (`sop-run-<sopId>` parent + `sop-step-<taskEntityId>` children)