Agent Context and Caching

Problem

The current agent stack works, but it is too prompt-string heavy and too expensive for long-lived use:

• Chat injects every entity type schema into the prompt on every turn.
• Entity-scoped chat trusts client-supplied entity context instead of reloading the latest server truth.
• Chat, workflow, extraction, heartbeat, and inbox each assemble context differently.
• Stable instructions and tool definitions are not provider-cache aware.
• Long conversations have no provider-level context management strategy.
• Shared context and memories can grow without meaningful prompt budgeting.
• Agents are not given a structured tenant-authored business profile, so OCI-specific knowledge depends on whatever happens to exist in records or prior conversations.

This leaves quality, latency, and cost on the table.

Goals

1. Create a shared context pipeline that all agent entry points can reuse.
2. Keep stable context rich, but compact enough to avoid wasting prompt budget.
3. Add tenant-level business profile support so OCI and future clients can author durable business context once and inject it everywhere.
4. Add provider-aware caching and context management for Anthropic-backed agents.
5. Improve prompts and tool instructions around retrieval-first behavior, freshness, citations, and value framing.
6. Keep the implementation platform-safe: no product-specific slugs in platform code.

Design

Shared Context Layers

Prompts should be assembled from distinct layers with different stability and cost profiles:

1. Stable cached context
   • Base system prompt
   • Tenant-authored workspace business profile
   • Compact data-type summary
   • Stable shared-context guidance (corrections, lessons, routing)
2. Dynamic session context
   • Current date/time
   • Current user and role
   • Active entity context
   • Recent bounded memories
3. Execution-local context
   • Workflow node instructions
   • Extraction field instructions
   • Heartbeat attention context

Workspace Business Profile

Add a tenant setting key, agent_context, with a structured schema:

{
  summary: string;
  industry?: string;
  businessModel?: string;
  customers?: string[];
  valueDrivers?: string[];
  coreProcesses?: string[];
  keySystems?: string[];
  successMetrics?: string[];
  terminology?: string[];
  constraints?: string[];
  currentPriorities?: string[];
  differentiators?: string[];
}

This becomes the durable place to encode what OCI actually does, how it makes money, which processes matter, which systems it relies on, and how success should be judged.

Prompt Compaction

Replace raw full-schema injection with a compact type summary:

• show type name, slug, description, active fields, and relation labels
• cap fields/relations per type
• highlight the focused type when a chat is scoped to a record
• instruct agents to use listEntityTypes({ detailed: true }) when they need the full schema

Entity context should also be compacted:

• summarize field values as named bullets
• cap field count and value length
• avoid dumping large JSON blobs into the system prompt

Runtime Improvements

Enhance the shared runtime to support:

• deterministic tool ordering
• Anthropic cache control on stable system/tool definitions
• Anthropic context management for long chats
• pass-through toolChoice and prepareStep so callers can tighten agent loops when needed

Safety Hardening

Chat should no longer trust client-supplied entity title/type/content as system-prompt input. If an entity ID is provided, the route should reload the latest tenant-scoped entity context from the server and only inject that verified version.

Acceptance Criteria

• Chat, workflow, heartbeat, inbox, and extraction all can consume the same workspace business profile prompt
• Chat uses server-resolved entity context instead of trusting client payload fields
• Entity type prompt section is compact and bounded
• Shared context and memories are bounded for prompt size
• Anthropic runs receive cache-control breakpoints on stable system/tool definitions
• Anthropic chat runs receive context-management settings for old tool-use cleanup
• Runtime exposes toolChoice / prepareStep
• System-agent prompts are updated for retrieval-first behavior, citations, and value framing
• OCI tenant seed populates agent_context
• Tests cover the new workspace-context and provider-options behavior

Rollout Notes

• agent_context is additive and backward-compatible. Workspaces without it continue to run with existing prompts.
• Anthropic caching/context-management should only be applied when the resolved model ID is Claude-family.
• Tool ordering must remain deterministic so prompt caching gets reuse from identical tool definitions.