Obsidian Interop

## Overview

The Obsidian interop layer lets knowledge workers maintain entity data as plain markdown files in an Obsidian vault and exchange them with the platform. It also gives every entity a first-class prose body field — a place for narrative, notes, and linked thinking that does not fit in structured schema fields.

The interop layer has four components:

1. **Body field** — `description` TEXT column promoted to a rich editor with FTS indexing.
2. **TipTap editor with wikilinks** — `[[wikilink]]` autocomplete that creates entity relations.
3. **Entity markdown format** — Obsidian-compatible YAML frontmatter + H1 title + markdown body.
4. **Vault import/export** — Bidirectional zip exchange with wikilink relation resolution.

## Key Concepts

### Body Field

The `description` column on the `entities` table is the entity's primary prose body. It is:

- A free-form markdown text field, not part of `json_schema` — available on every entity type without any schema changes.
- FTS-indexed at **weight B** (same weight as description fields, below title at weight A, above content at weight C).
- Rendered in the entity detail page **above structured fields**, in a collapsible section.
- Included in `searchEntities`, `getEntity`, `createEntity`, and `updateEntity` AI tool inputs/outputs as the `body` field.

### Wikilinks

A wikilink is an inline reference to another entity written as `[[Entity Title]]`. When the body is saved:

1. The editor serializes the body to markdown, preserving `[[wikilinks]]` as plain text tokens.
2. `syncWikilinkRelations()` reconciles the `mention` entity relations for this entity: stale mentions (wikilinks removed from the body) are deleted; new mentions are created.
3. In view mode, wikilink tokens are resolved back to entity links and rendered as clickable nodes via the wikilink TipTap extension's view renderer (see `features/entities/components/editor/wikilink-extension.ts` + `wikilink-hover-card.tsx`).

The `mention` relation type is auto-managed — it is separate from intentionally created structural relations like `belongs_to` or `manages`.

### Mention Relations

`entity_relations` rows with `relationship_type: "mention"` track which entities are referenced in a given entity's body. They are:

- Created and deleted automatically by `syncWikilinkRelations()` on every body save.
- Queryable like any other relation via `getRelatedEntities()`.
- Excluded from the "Connections" section of the entity detail page by default (they are rendered inline in the body editor instead).

### TypeSpec Markdown Format (Type Definitions)

`features/entities/type-spec/` provides a markdown representation for **entity type definitions** (schemas, fields, connections, scoring criteria). See [Entity System — TypeSpec Markdown Format](/docs/features/entity-system#type-spec-markdown-format) for the full format reference.

At runtime, two API routes expose the TypeSpec compiler:

- `POST /api/entity-types/from-typespec` — create or update a type from markdown.
- `GET /api/entity-types/[slug]/typespec` — export an existing type as markdown.

### Entity Instance Markdown Format (Records)

`features/entities/type-spec/entity-markdown.ts` handles individual entity records — distinct from the type definition format above.

**Format:**

```markdown
---
type: opportunity
tags: [ai, automation]
status: researching
estimated_annual_value: 250000
---

# Opportunity Title

Body prose goes here. Can reference [[Related Company]] or [[Pain Point]] using wikilinks.
```

- **Frontmatter** — `type` (entity type slug), `tags`, and all structured `content` fields by their JSON schema key names.
- **H1 title** — The entity title.
- **Body** — Everything after the H1 becomes the `description` field. Wikilinks are preserved as `[[title]]` tokens.

## How It Works

### Rich Editor Architecture

`EntityBodyEditor` (`features/entities/components/entity-body-editor.tsx`) is a TipTap editor with two custom extensions:

**`wikilink-extension.ts`**

- Defines a custom `wikilink` inline node that stores `data-title` and `data-entity-id` attributes.
- Registers an input rule: typing `[[` switches to wikilink-suggest mode.
- A ProseMirror plugin intercepts keystrokes in suggest mode and queries `/api/search/global` to render an autocomplete dropdown.
- Selecting a result inserts a `wikilink` node. The node renders as a styled chip in edit mode and as a clickable link wrapped in `EntityHoverCard` in view mode (rendered by `wikilink-extension.ts`'s `addNodeView` and the standalone `wikilink-hover-card.tsx`).
- On serialization to markdown, wikilink nodes are output as `[[Entity Title]]` (the display title, not the UUID) for Obsidian compatibility.

**`slash-command-extension.ts`**

- A ProseMirror plugin that intercepts `/` at the start of a line or after whitespace.
- Renders a floating command palette with fuzzy-search filtering.
- Commands are registered as a static list with an icon, label, and TipTap `chain()` action.

### Wikilink Relation Sync

```typescript
// features/entities/server/wikilink-relations.ts
export async function syncWikilinkRelations(
  entityId: string,
  wikilinkTitles: string[]
): Promise<void>
```

Called on every body save (debounced 500 ms in the editor). Steps:

1. Look up entity IDs for all `wikilinkTitles` via `resolveWikilinkTitles()`.
2. Fetch all existing `mention` relations from this entity.
3. Delete relations whose target entity title is no longer in the wikilink list.
4. Insert relations for wikilinks that do not yet have a relation row.

Titles that do not match any entity are silently skipped (the wikilink renders as plain text in view mode).

### Vault Export

`POST /api/entities/export/vault` accepts a filter (type slug and/or specific IDs), fetches the matching entities with their types, runs `generateEntityMarkdown()` on each, and returns a ZIP archive where each file is `{entity-slug}.md`.

```typescript
// Generate a single entity's markdown file
generateEntityMarkdown(entity: EntityRecord, entityType: EntityTypeRecord): string
```

The function serializes:
- Frontmatter: `type`, `tags`, all non-empty `content` fields
- H1: entity title
- Body: `entity.description` (preserved verbatim, including any `[[wikilinks]]`)

### Vault Import

`POST /api/entities/import/vault` accepts a `multipart/form-data` request with a `vault` ZIP file field.

**Two-pass import:**

| Pass | What happens |
|---|---|
| Pass 1 | Extract each markdown file. Parse with `parseEntityMarkdown()`. Create entities (type resolution, slug generation, activity log). Build a `{ title → entityId }` map. |
| Pass 2 | For each imported entity, extract `wikilinks[]` from the parsed markdown. Resolve titles against the pass-1 map. Call `syncWikilinkRelations()` to create `mention` relations. |

The two-pass strategy handles forward references — file A can wikilink to file B even if B appears later in the archive. Wikilinks referencing titles not found in the import batch are resolved against existing platform entities.

```typescript
// Parse a markdown file back to entity input
parseEntityMarkdown(markdown: string): ParsedEntityMarkdown

interface ParsedEntityMarkdown {
  title: string;
  description?: string;
  content: Record<string, unknown>;
  tags?: string[];
  typeSlug?: string;
  wikilinks: string[];
}
```

## API Reference

### Body Field in AI Tools

The `body` field is a first-class input/output on all core entity tools:

| Tool | `body` behavior |
|---|---|
| `searchEntities` | Result items include `body: string \| null` (truncated at 500 chars in search results) |
| `getEntity` | Full `body` text included in response |
| `createEntity` | Optional `body: string` input (markdown text) |
| `updateEntity` | Optional `body: string` input — replaces the existing body entirely (unlike `content`, which is merged) |

Agent prompt context includes entity body text so agents have access to prose notes and narrative alongside structured fields.

### TypeSpec Runtime API

| Endpoint | Method | Auth | Description |
|---|---|---|---|
| `/api/entity-types/from-typespec` | `POST` | Session (admin) | Create or update an entity type from a TypeSpec markdown string. |
| `/api/entity-types/[slug]/typespec` | `GET` | Session | Export an entity type as a TypeSpec markdown string. |

**`POST /api/entity-types/from-typespec` request body:**

```typescript
{
  markdown: string;
  tenantScoped?: boolean;  // default: false (global type)
}
```

Returns the created or updated `EntityTypeRecord`.

### Vault API

| Endpoint | Method | Auth | Description |
|---|---|---|---|
| `/api/entities/export/vault` | `POST` | Session (`entities.team.read`) | Export entities as a zip of markdown files. |
| `/api/entities/import/vault` | `POST` | Session (`entities.team.create`) | Import a zip of markdown files as entities with wikilink resolution. |

**Export request:**

```typescript
{
  typeSlug?: string;   // Filter by entity type
  ids?: string[];      // Specific entity IDs (omit for all)
}
```

Returns `application/zip`.

**Import request:** `multipart/form-data` with a `vault` file field (`.zip`).

Returns:

```typescript
{
  imported: number;
  failed: number;
  errors: string[];
}
```

### Wikilink Relations

| Function | Signature | Description |
|---|---|---|
| `syncWikilinkRelations(entityId, wikilinkTitles)` | `(entityId: string, wikilinkTitles: string[]) => Promise<void>` | Full-reconcile `mention` relations for an entity. |
| `resolveWikilinkTitles(titles, tenantId)` | `(titles: string[], tenantId: string) => Promise<Record<string, string>>` | Look up entity IDs by title within a tenant. |

### Entity Markdown (`features/entities/type-spec/entity-markdown.ts`)

| Function | Signature | Description |
|---|---|---|
| `generateEntityMarkdown(entity, entityType)` | `(entity: EntityRecord, entityType: EntityTypeRecord) => string` | Serialize an entity to Obsidian-compatible markdown. |
| `parseEntityMarkdown(markdown)` | `(markdown: string) => ParsedEntityMarkdown` | Parse a markdown file into entity create/update input. |

## For Agents

Agents can use the Obsidian interop layer in several ways:

**Reading and writing body text:**

```
getEntity(id: "abc-123")
// Response includes: { ..., body: "This entity tracks the migration of..." }

updateEntity(id: "abc-123", body: "Updated notes with [[Related Entity]] linked here.")
// Sets description field; creates/updates mention relation automatically
```

**Designing new entity types via TypeSpec:**

1. Use `GET /api/entity-types/[slug]/typespec` to read an existing type's definition.
2. Modify the markdown (add fields, update extraction instructions, add scoring).
3. Submit via `POST /api/entity-types/from-typespec` to apply the changes.

**Vault exchange:**

Agents with file system access (e.g., via an MCP filesystem tool) can:
- Export selected entities to markdown files for editing in Obsidian.
- Import a folder of markdown files as platform entities.

## Design Decisions

**`description` column over a new `body` column.** Entities already had a `description` column that was sparsely used. Promoting it avoids a rename migration and keeps the column name neutral — "description" reads naturally for all entity types, not just note-heavy ones.

**Wikilinks serialize as `[[title]]`, not `[[id]]`.** Obsidian uses titles as the primary link target, not UUIDs. Using titles preserves compatibility with Obsidian's own wikilink resolution and makes exported markdown files human-readable without a lookup table.

**Full-reconcile on save, not incremental diff.** `syncWikilinkRelations()` unconditionally deletes and re-inserts mention relations on every save. This keeps the sync logic O(n) and self-healing. The tradeoff is a few extra DB writes per body edit — acceptable because body edits are infrequent and mentions are cheap to recreate.

**Two-pass vault import for forward references.** A single-pass import would need a topological sort of files to resolve `[[wikilinks]]` correctly. Two passes eliminate this requirement and handle cycles. The cost is two DB round-trips per import batch, which is negligible compared to file parsing.

**TypeSpec runtime API requires admin session.** Entity type schema changes are high-impact — they affect rendering, extraction, and import behavior across all records of that type. Requiring an admin session adds a human checkpoint and prevents unprivileged automation from reshaping the data model.

**`body` replaces on `updateEntity`, `content` merges.** Structured content fields are merged to prevent agents from overwriting fields they did not intend to change. Body text is narrative prose — replacing it on update is the expected behavior (like updating a document). Merging prose text has no clear semantic meaning.

## Related Modules

- [Entity System](/docs/features/entity-system) -- Core entity CRUD, TypeSpec format, search, and FTS
- [Tool System](/docs/features/tool-system) -- AI tools that expose body field read/write to agents
- [Agent System](/docs/features/agent-system) -- Agents that use body text for context and produce notes
- [Graph](/docs/features/graph) -- `mention` relations appear in the entity relationship graph