Entity Table Migration to Generic DataTable

Problem

Two table systems coexist in the codebase:

1. Old entity data table (features/entities/components/data-table/) — 1,089-line monolith + 30 supporting files (~5,700 lines total). Tightly coupled to entity types. Provides all production features: view toggle (table/grid/kanban), per-column filters, URL-synced state, bulk operations (edit/tag/extract/export/delete), column/filter presets, split view, density toggle, summary footer, connected-to filter.

2. New generic DataTable (features/data-table/) — ~5,700 lines across 50 files. Decoupled from entities. Provides: type-aware rendering, 6 inline editors, CSS-variable column resize, FocusSink keyboard navigation, clipboard copy/paste, row virtualization, row selection, generic bulk actions, pagination (client + server), column visibility. Currently used by TaskDataTable and available for DataTableBlock.

The new module is architecturally better but lacks entity-specific orchestration (view toggle, filters, bulk ops). The old module has all the features but is a monolith. Both must converge — maintaining two table systems creates double the bug surface and blocks feature parity.

Solution

Incremental migration: compose old entity orchestration features around the new generic DataTable<T> renderer. Replace the monolith from the inside out.

Architecture

EntityListOrchestrator (new wrapper, replaces old data-table.tsx)
├── EntityToolbar (adapted from old data-table-toolbar.tsx)
│   ├── Search (debounced)
│   ├── FilterPresets (from old filter-presets.tsx)
│   ├── ConnectedToFilter (from old connected-filter-button.tsx)
│   ├── ColumnPresets (from old column-presets.tsx)
│   ├── ColumnVisibility → delegates to DataTable API
│   ├── DensityToggle → delegates to DataTable config
│   ├── SplitViewToggle
│   ├── SummaryFooterToggle
│   └── ViewModeToggle (table / grid / kanban)
├── BulkActionsBar (old bulk-actions-bar.tsx — already decoupled)
│   ├── BulkEditDialog
│   ├── BulkTag / BulkUntag
│   ├── BulkExtract
│   ├── BulkExport
│   └── BulkDelete
├── FilterBadges (active filter indicators with clear)
├── OverflowMenu (import CSV, export CSV, save view)
│
├── [table mode] DataTable<EntityRecord> ← generic renderer
│   ├── EntityColumnSchema adapter (json_schema → ColumnSchema[])
│   ├── EntityCellEdit adapter (onCellEdit → PATCH /api/entities/[id])
│   └── Server-side sort/filter delegation (onSort, onFilter → URL params)
│
├── [grid mode] EntityGridView (existing, no changes)
└── [kanban mode] KanbanBoard (existing, no changes)

Key insight

The old monolith does two jobs: orchestration (toolbar, filters, view toggle, bulk ops) and table rendering (cells, rows, headers, keyboard nav). The migration replaces only the rendering layer with DataTable<T> while preserving the orchestration layer entirely.

Phases

Phase 1: Column Schema Adapter

Build buildEntityColumnSchemas() that converts EntityTypeRecord.json_schema into ColumnSchema<EntityRecord>[]. Must handle:

• Built-in columns: title (sticky, avatar+link), score, tags, created
• Schema fields: infer ColumnType from JSON Schema type/format/enum
• x-display-type overrides (currency, percentage, status)
• Relation columns (read-only)
• statusMap config for colored badges
• Default visibility (first 8 fields)

Phase 2: Entity Edit Adapter

Wire onCellEdit to PATCH /api/entities/[id] with router.refresh(). Handle:

• Type coercion for each field type
• Optimistic updates via React Query
• Error rollback to original value
• Permission gating (canEdit check)

Phase 3: EntityListOrchestrator Wrapper

Replace the 1,089-line monolith with a composition wrapper:

• Mount the old toolbar (with minimal adaptation to pass config to DataTable)
• Wire old filter system: useDataTableFilters + useDataTableUrlFilters → DataTable's onFilter callback
• Wire old bulk actions: DataTable's useRowSelection → BulkActionsBar
• Wire server-side sort/pagination: DataTable's onSort / onPageChange → URL params
• Preserve view mode toggle switching between DataTable / GridView / KanbanBoard

Phase 4: DataTableBlock Migration

Swap DataTableBlock from old DataTable to new EntityListOrchestrator (compact mode).

Phase 5: Cleanup

Delete old features/entities/components/data-table/ directory. The monolith is fully replaced.

Acceptance Criteria

Each phase must pass before proceeding to the next:

• Phase 1: buildEntityColumnSchemas() produces identical column rendering to old columns.tsx for all field types. Unit tests with snapshot comparison.
• Phase 2: Inline editing persists to API, handles errors, respects permissions. Matches old behavior.
• Phase 3: Entity list page renders identically in table/grid/kanban modes. All toolbar features work. URL state syncs correctly. Bulk actions operate on selected rows. Filter presets save/load. Visual verification at desktop + mobile viewports.
• Phase 4: DataTableBlock renders entities correctly in compact and full modes.
• Phase 5: Old directory deleted. pnpm typecheck && pnpm test && pnpm build pass. Zero references to deleted files.
• Overall: No feature from the 23-item regression list is missing after migration.

Regression Inventory

These features MUST survive the migration (tracked from the PR 678 regression analysis):

Trade-offs

Incremental vs. big-bang migration: Incremental is slower but safer. Each phase is independently testable and shippable. The old code stays as fallback until the new composition is proven.

Dual system maintenance window: During migration, both systems exist. This is acceptable because they serve different consumers (entities vs. tasks) and don't interact. The window should be kept short — target completion within 2-3 focused sessions.

Toolbar adaptation complexity: The old toolbar is tightly coupled to the old table's TanStack instance (calls table.getAllColumns(), col.toggleVisibility()). Phase 3 will need to either expose these from the new DataTable via ref/context, or adapt the toolbar to use the new DataTable's config-driven API. The latter is cleaner but more work.