Documentation source
Entity Table Migration to Generic DataTable
Migrate entity list pages from the monolithic data table to the generic DataTable<T> platform component
## Problem Two table systems coexist in the codebase: 1. **Old entity data table** (`features/entities/components/data-table/`) — 1,089-line monolith + 30 supporting files (~5,700 lines total). Tightly coupled to entity types. Provides all production features: view toggle (table/grid/kanban), per-column filters, URL-synced state, bulk operations (edit/tag/extract/export/delete), column/filter presets, split view, density toggle, summary footer, connected-to filter. 2. **New generic DataTable** (`features/data-table/`) — ~5,700 lines across 50 files. Decoupled from entities. Provides: type-aware rendering, 6 inline editors, CSS-variable column resize, FocusSink keyboard navigation, clipboard copy/paste, row virtualization, row selection, generic bulk actions, pagination (client + server), column visibility. Currently used by `TaskDataTable` and available for `DataTableBlock`. The new module is architecturally better but lacks entity-specific orchestration (view toggle, filters, bulk ops). The old module has all the features but is a monolith. Both must converge — maintaining two table systems creates double the bug surface and blocks feature parity. ## Solution Incremental migration: compose old entity orchestration features around the new generic `DataTable<T>` renderer. Replace the monolith from the inside out. ### Architecture ``` EntityListOrchestrator (new wrapper, replaces old data-table.tsx) ├── EntityToolbar (adapted from old data-table-toolbar.tsx) │ ├── Search (debounced) │ ├── FilterPresets (from old filter-presets.tsx) │ ├── ConnectedToFilter (from old connected-filter-button.tsx) │ ├── ColumnPresets (from old column-presets.tsx) │ ├── ColumnVisibility → delegates to DataTable API │ ├── DensityToggle → delegates to DataTable config │ ├── SplitViewToggle │ ├── SummaryFooterToggle │ └── ViewModeToggle (table / grid / kanban) ├── BulkActionsBar (old bulk-actions-bar.tsx — already decoupled) │ ├── BulkEditDialog │ ├── BulkTag / BulkUntag │ ├── BulkExtract │ ├── BulkExport │ └── BulkDelete ├── FilterBadges (active filter indicators with clear) ├── OverflowMenu (import CSV, export CSV, save view) │ ├── [table mode] DataTable<EntityRecord> ← generic renderer │ ├── EntityColumnSchema adapter (json_schema → ColumnSchema[]) │ ├── EntityCellEdit adapter (onCellEdit → PATCH /api/entities/[id]) │ └── Server-side sort/filter delegation (onSort, onFilter → URL params) │ ├── [grid mode] EntityGridView (existing, no changes) └── [kanban mode] KanbanBoard (existing, no changes) ``` ### Key insight The old monolith does two jobs: **orchestration** (toolbar, filters, view toggle, bulk ops) and **table rendering** (cells, rows, headers, keyboard nav). The migration replaces only the rendering layer with `DataTable<T>` while preserving the orchestration layer entirely. ## Phases ### Phase 1: Column Schema Adapter Build `buildEntityColumnSchemas()` that converts `EntityTypeRecord.json_schema` into `ColumnSchema<EntityRecord>[]`. Must handle: - Built-in columns: title (sticky, avatar+link), score, tags, created - Schema fields: infer ColumnType from JSON Schema type/format/enum - `x-display-type` overrides (currency, percentage, status) - Relation columns (read-only) - `statusMap` config for colored badges - Default visibility (first 8 fields) ### Phase 2: Entity Edit Adapter Wire `onCellEdit` to `PATCH /api/entities/[id]` with `router.refresh()`. Handle: - Type coercion for each field type - Optimistic updates via React Query - Error rollback to original value - Permission gating (canEdit check) ### Phase 3: EntityListOrchestrator Wrapper Replace the 1,089-line monolith with a composition wrapper: - Mount the old toolbar (with minimal adaptation to pass config to DataTable) - Wire old filter system: `useDataTableFilters` + `useDataTableUrlFilters` → DataTable's `onFilter` callback - Wire old bulk actions: DataTable's `useRowSelection` → `BulkActionsBar` - Wire server-side sort/pagination: DataTable's `onSort` / `onPageChange` → URL params - Preserve view mode toggle switching between DataTable / GridView / KanbanBoard ### Phase 4: DataTableBlock Migration Swap `DataTableBlock` from old `DataTable` to new `EntityListOrchestrator` (compact mode). ### Phase 5: Cleanup Delete old `features/entities/components/data-table/` directory. The monolith is fully replaced. ## Acceptance Criteria Each phase must pass before proceeding to the next: - [ ] **Phase 1**: `buildEntityColumnSchemas()` produces identical column rendering to old `columns.tsx` for all field types. Unit tests with snapshot comparison. - [ ] **Phase 2**: Inline editing persists to API, handles errors, respects permissions. Matches old behavior. - [ ] **Phase 3**: Entity list page renders identically in table/grid/kanban modes. All toolbar features work. URL state syncs correctly. Bulk actions operate on selected rows. Filter presets save/load. Visual verification at desktop + mobile viewports. - [ ] **Phase 4**: DataTableBlock renders entities correctly in compact and full modes. - [ ] **Phase 5**: Old directory deleted. `pnpm typecheck && pnpm test && pnpm build` pass. Zero references to deleted files. - [ ] **Overall**: No feature from the [23-item regression list](#regression-inventory) is missing after migration. ## Regression Inventory These features MUST survive the migration (tracked from the PR 678 regression analysis): | # | Feature | Phase | |---|---------|-------| | 1 | View mode toggle (table/grid/kanban) | 3 | | 2 | Kanban view with drag-and-drop, group-by, hide-empty | 3 | | 3 | Grid/card view | 3 | | 4 | Per-column filter popovers (type-aware) | 3 | | 5 | Filter presets (save/load/delete) | 3 | | 6 | URL-synced filters (f.* params, sort, search, tag) | 3 | | 7 | Connected-to filter (cross-entity relation) | 3 | | 8 | Column presets (save/load layouts) | 3 | | 9 | Bulk edit field dialog | 3 | | 10 | Bulk tag / untag | 3 | | 11 | Bulk extract (AI extraction) | 3 | | 12 | Import CSV/Excel | 3 | | 13 | Save View dialog | 3 | | 14 | Summary footer (count + sums) | 3 | | 15 | Density toggle (wired to UI) | 3 | | 16 | Split view toggle | 3 | | 17 | Active filter badges with clear | 3 | | 18 | Server-side sort | 3 | | 19 | Server-side filtering | 3 | | 20 | Column pinning (visual effect) | 1 | | 21 | Loading/skeleton state | 3 | | 22 | Search debounce (300ms) | 3 | | 23 | Mobile card fallback (below lg) | 3 | ## Trade-offs **Incremental vs. big-bang migration**: Incremental is slower but safer. Each phase is independently testable and shippable. The old code stays as fallback until the new composition is proven. **Dual system maintenance window**: During migration, both systems exist. This is acceptable because they serve different consumers (entities vs. tasks) and don't interact. The window should be kept short — target completion within 2-3 focused sessions. **Toolbar adaptation complexity**: The old toolbar is tightly coupled to the old table's TanStack instance (calls `table.getAllColumns()`, `col.toggleVisibility()`). Phase 3 will need to either expose these from the new DataTable via ref/context, or adapt the toolbar to use the new DataTable's config-driven API. The latter is cleaner but more work.