fleet-memory/hindsight-docs/docs/sdks/integrations/codex.md
Ben 0b17a67c70
feat: add Hindsight memory integration for OpenAI Codex CLI (#730)
* feat(codex): add Hindsight memory integration for OpenAI Codex CLI

Hooks-based integration that gives Codex CLI long-term memory via Hindsight.
Three hooks keep memory in sync: SessionStart (daemon pre-warm), UserPromptSubmit
(recall + context injection), Stop (retain conversation to memory).

Key differences from the Claude Code integration:
- Codex transcript format: JSONL with {msg: {type, message}} (user_message/agent_message)
- No CODEX_PLUGIN_ROOT env var — install.sh writes hooks.json with absolute paths
- State stored in ~/.hindsight/codex/state/ (not CLAUDE_PLUGIN_DATA)
- No async: true in hooks (not supported by Codex)
- No SessionEnd event
- hooks.json written to ~/.codex/hooks.json with codex_hooks = true in config.toml

* fix(codex): fix transcript parser for actual Codex disk format

Codex stores sessions as rollout-*.jsonl with response_item entries:
  User:      {type:response_item, payload:{type:message, role:user, content:[{type:input_text, text:...}]}}
  Assistant: {type:response_item, payload:{type:message, role:assistant, phase:final_answer, content:[{type:output_text, text:...}]}}

Previous parser expected an undocumented {msg:{type:user_message}} format from the Rust protocol spec
that does not match the actual on-disk storage format.

* feat(codex): add reflect mode to UserPromptSubmit hook

Add recallMode config option (default: 'recall') that switches the
UserPromptSubmit hook between:
- 'recall': existing behavior, fast raw facts list
- 'reflect': agentic synthesis loop, returns coherent prose answer

Also adds reflect() method to HindsightClient and HINDSIGHT_RECALL_MODE
env var override. Reflect uses a 25s timeout (vs 10s for recall).

* feat(codex): auto mode for recall/reflect selection

Add recallMode: 'auto' (new default) that picks the operation per-query:
- Synthesis patterns (what do you know, what's my, summarize, etc.) → reflect
- All other prompts → recall (fast, raw facts, better for code tasks)

* feat(codex): add automated test suite and finalize recall-only mode

* docs(codex): add docs page and sidebar entry for Codex CLI integration
2026-03-30 10:51:53 +02:00

8.5 KiB

sidebar_position
6

OpenAI Codex CLI

Persistent memory for OpenAI Codex CLI using Hindsight. Three Python hook scripts automatically recall relevant context before each prompt and retain conversations after each turn — no changes to your Codex workflow required.

Quick Start

# 1. Clone the Hindsight repo and install the plugin
git clone https://github.com/vectorize-io/hindsight.git
cd hindsight/hindsight-integrations/codex
./install.sh

# 2. Configure your Hindsight connection
cat > ~/.hindsight/codex.json << 'EOF'
{
  "hindsightApiUrl": "https://api.hindsight.vectorize.io",
  "hindsightApiToken": "hsk_your_token_here",
  "bankId": "codex"
}
EOF

# 3. Start Codex — memory is live
codex

For a local Hindsight instance, set hindsightApiUrl to http://localhost:9077 and omit hindsightApiToken.

Features

  • Auto-recall — on every user prompt, queries Hindsight for relevant memories and injects them as additionalContext (invisible to the transcript, visible to Codex)
  • Auto-retain — after each Codex response, stores the conversation transcript to Hindsight for future recall
  • Dynamic bank IDs — supports per-project memory isolation based on the working directory
  • Session-level upsert — uses the session ID as the document ID so re-running the same session updates rather than duplicates stored content
  • Zero dependencies — pure Python stdlib, no pip install required

Architecture

The plugin uses three Codex hook events:

Hook Event Purpose
session_start.py SessionStart Warm up — verify Hindsight is reachable
recall.py UserPromptSubmit Auto-recall — query memories, inject as additionalContext
retain.py Stop Auto-retain — extract transcript, POST to Hindsight (async)

On UserPromptSubmit, the hook reads the prompt, queries Hindsight for the most relevant memories, and outputs a hookSpecificOutput.additionalContext block. Codex prepends this to the conversation before sending it to the model:

<hindsight_memories>
Relevant memories from past conversations...
Current time - 2026-03-27 09:14

- Project uses FastAPI with asyncpg — not SQLAlchemy [world] (2026-03-26)
- Preferred testing framework: pytest with pytest-asyncio [experience] (2026-03-26)
</hindsight_memories>

On Stop, the hook reads the session transcript, strips previously injected memory tags (to prevent feedback loops), and POSTs the conversation to Hindsight asynchronously.

Connection Modes

Connect to a running Hindsight server (cloud or self-hosted):

{
  "hindsightApiUrl": "https://api.hindsight.vectorize.io",
  "hindsightApiToken": "hsk_your_token"
}

2. Local Daemon

Run hindsight-embed locally. The session_start.py hook will detect it on apiPort (default 9077). The daemon is not auto-started by the Codex plugin — start it separately:

uvx hindsight-embed

Then leave hindsightApiUrl empty in your config and the plugin will connect to http://localhost:9077.

Configuration

Settings are loaded from ~/.hindsight/codex.json. Every setting can also be overridden via environment variable.

Loading order (later entries win):

  1. Built-in defaults
  2. Plugin settings.json (at ~/.hindsight/codex/settings.json)
  3. User config (~/.hindsight/codex.json)
  4. Environment variables

Connection

Setting Env Var Default Description
hindsightApiUrl HINDSIGHT_API_URL "" URL of the Hindsight API server. Required.
hindsightApiToken HINDSIGHT_API_TOKEN null API token for authentication. Required for Hindsight Cloud.
apiPort HINDSIGHT_API_PORT 9077 Port for the local hindsight-embed daemon.

Memory Bank

Setting Env Var Default Description
bankId HINDSIGHT_BANK_ID "codex" The bank to read from and write to. All sessions share this bank unless dynamicBankId is enabled.
bankMission HINDSIGHT_BANK_MISSION coding assistant prompt Describes the agent's purpose. Sent when creating or updating the bank.
retainMission extraction prompt Instructions for Hindsight's fact extraction — what to extract from coding conversations.
dynamicBankId HINDSIGHT_DYNAMIC_BANK_ID false When true, derives a unique bank ID from dynamicBankGranularity fields — useful for per-project isolation.
dynamicBankGranularity ["agent", "project"] Which fields to combine for dynamic bank IDs. "project" = working directory, "agent" = agent name.
bankIdPrefix "" Prefix prepended to all bank IDs.
agentName HINDSIGHT_AGENT_NAME "codex" Agent name used in dynamic bank ID derivation.

Auto-Recall

Setting Env Var Default Description
autoRecall HINDSIGHT_AUTO_RECALL true Master switch for auto-recall.
recallBudget HINDSIGHT_RECALL_BUDGET "mid" Search depth: "low" (fast), "mid" (balanced), "high" (thorough).
recallMaxTokens HINDSIGHT_RECALL_MAX_TOKENS 1024 Max tokens in the recalled memory block.
recallTypes ["world", "experience"] Memory types to retrieve.
recallContextTurns HINDSIGHT_RECALL_CONTEXT_TURNS 1 Prior turns to include when building the recall query. 1 = latest prompt only.
recallMaxQueryChars HINDSIGHT_RECALL_MAX_QUERY_CHARS 800 Max characters in the query sent to Hindsight.
recallRoles ["user", "assistant"] Which roles to include when building a multi-turn query.
recallPromptPreamble built-in Text placed above the recalled memories in the injected context block.

Auto-Retain

Setting Env Var Default Description
autoRetain HINDSIGHT_AUTO_RETAIN true Master switch for auto-retain.
retainMode HINDSIGHT_RETAIN_MODE "full-session" "full-session" sends the full transcript per session (upserted by session ID). "chunked" sends sliding windows every N turns.
retainEveryNTurns 10 Retain fires every N turns. 1 = every turn. Higher values reduce API calls.
retainOverlapTurns 2 Extra turns included from the previous chunk (chunked mode only).
retainRoles ["user", "assistant"] Which roles to include in the retained transcript.
retainTags ["{session_id}"] Tags attached to the stored document. {session_id} is replaced at runtime.
retainMetadata {} Arbitrary key-value metadata attached to the stored document.
retainContext "codex" Label identifying the source integration. Useful when multiple integrations write to the same bank.

Debug

Setting Env Var Default Description
debug HINDSIGHT_DEBUG false Enable verbose logging to stderr. All log lines are prefixed with [Hindsight].

Per-Project Memory

To give each project its own isolated memory bank, enable dynamic bank IDs:

{
  "dynamicBankId": true,
  "dynamicBankGranularity": ["agent", "project"]
}

With this config, running Codex in ~/projects/api and ~/projects/frontend stores and recalls memories separately. Bank IDs are derived from the working directory path.

Troubleshooting

Hooks not firing: Check that ~/.codex/config.toml contains codex_hooks = true under [features]. Re-run install.sh to write this automatically.

No memories recalled: Recall returns results only after something has been retained. Either complete one Codex session first, or seed your bank manually using the cookbook example.

Memory not being stored: retainEveryNTurns defaults to 10 — retain only fires every 10 turns. While testing, add "retainEveryNTurns": 1 to ~/.hindsight/codex.json.

Debug mode: Add "debug": true to ~/.hindsight/codex.json to see what Hindsight is doing on each turn:

[Hindsight] Recalling from bank 'codex', query length: 42
[Hindsight] Injecting 3 memories
[Hindsight] Retaining to bank 'codex', doc 'sess-abc123', 2 messages, 847 chars

High latency on recall: Use "recallBudget": "low" or reduce recallMaxTokens to speed up recall queries.