From 9aaa78b9a6b74313a784bd5534f826a5dc2ad901 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nicol=C3=B2=20Boschi?= Date: Thu, 26 Feb 2026 18:01:56 +0100 Subject: [PATCH] fix: doc build --- .../docs/developer/api/memory-banks.mdx | 8 +- .../docs/developer/configuration.md | 2 +- .../developer/api/memory-banks.mdx | 161 ++++++++++-- .../developer/api/mental-models.mdx | 28 +++ .../version-0.4/developer/api/recall.mdx | 69 ++++-- .../version-0.4/developer/api/retain.mdx | 2 +- .../version-0.4/developer/configuration.md | 147 ++++++++--- .../version-0.4/developer/mcp-server.md | 210 +++++++++++++++- .../version-0.4/developer/observations.mdx | 66 +++-- .../version-0.4/developer/reflect.mdx | 14 +- .../version-0.4/developer/retain.md | 23 ++ .../version-0.4/sdks/integrations/chat.md | 163 ++++++++++++ .../sdks/integrations/local-mcp.md | 66 ++++- .../version-0.4-sidebars.json | 8 + .../references/cookbook/index.md | 4 +- .../references/developer/api/main-methods.md | 10 +- .../references/developer/api/memory-banks.md | 232 ++++++++++++++---- .../references/developer/api/mental-models.md | 28 +++ .../references/developer/api/recall.md | 94 +++++-- .../references/developer/api/retain.md | 15 +- .../references/developer/configuration.md | 137 +++++++++-- .../references/developer/installation.md | 2 +- .../references/developer/mcp-server.md | 210 +++++++++++++++- .../references/developer/observations.md | 72 ++++-- .../references/developer/reflect.md | 27 +- .../references/developer/retain.md | 23 ++ .../references/sdks/integrations/chat.md | 163 ++++++++++++ .../references/sdks/integrations/local-mcp.md | 66 ++++- 28 files changed, 1790 insertions(+), 260 deletions(-) create mode 100644 hindsight-docs/versioned_docs/version-0.4/sdks/integrations/chat.md create mode 100644 skills/hindsight-docs/references/sdks/integrations/chat.md diff --git a/hindsight-docs/docs/developer/api/memory-banks.mdx b/hindsight-docs/docs/developer/api/memory-banks.mdx index 23431100..3e53a2a3 100644 --- a/hindsight-docs/docs/developer/api/memory-banks.mdx +++ b/hindsight-docs/docs/developer/api/memory-banks.mdx @@ -54,9 +54,9 @@ hindsight bank create my-bank ## Bank Configuration -Each memory bank can be configured independently per operation. Configuration can be set via the [bank config API](#updating-configuration), the [Control Plane UI](/developer/index), or [server-wide environment variables](/developer/configuration). +Each memory bank can be configured independently per operation. Configuration can be set via the [bank config API](#updating-configuration), the [Control Plane UI](/), or [server-wide environment variables](/developer/configuration). -### retain_mission +### retain_mission {#retain-configuration} A plain-language description of what this bank should pay attention to during extraction. The mission is injected into the extraction prompt alongside the built-in rules — it steers focus without replacing the extraction logic. @@ -83,7 +83,7 @@ Only active when `retain_extraction_mode` is `custom`. Replaces the built-in ext See [Retain configuration](/developer/configuration#retain) for environment variable names and defaults. -### enable_observations +### enable_observations {#observations-configuration} Toggles automatic observation consolidation on or off. Defaults to `true` when the observations feature is enabled on the server. @@ -199,7 +199,7 @@ The response distinguishes: This removes all bank-level overrides. The bank reverts to server-wide defaults (set via environment variables). -You can also update configuration directly from the [Control Plane UI](/developer/index) — navigate to a bank and open the **Configuration** tab. +You can also update configuration directly from the [Control Plane UI](/) — navigate to a bank and open the **Configuration** tab. --- diff --git a/hindsight-docs/docs/developer/configuration.md b/hindsight-docs/docs/developer/configuration.md index 17a2c5de..cb5aea1b 100644 --- a/hindsight-docs/docs/developer/configuration.md +++ b/hindsight-docs/docs/developer/configuration.md @@ -724,7 +724,7 @@ export HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_KEY=base64encodedkey== For production deployments, use `s3`, `gcs`, or `azure` to avoid storing large binary files in your PostgreSQL database. Set `HINDSIGHT_API_FILE_DELETE_AFTER_RETAIN=true` (the default) to delete files after memory extraction, which minimizes storage costs. ::: -### Observations (Experimental) +### Observations (Experimental) {#observations} Observations are consolidated knowledge synthesized from facts. diff --git a/hindsight-docs/versioned_docs/version-0.4/developer/api/memory-banks.mdx b/hindsight-docs/versioned_docs/version-0.4/developer/api/memory-banks.mdx index 96c0d110..3e53a2a3 100644 --- a/hindsight-docs/versioned_docs/version-0.4/developer/api/memory-banks.mdx +++ b/hindsight-docs/versioned_docs/version-0.4/developer/api/memory-banks.mdx @@ -46,49 +46,162 @@ Make sure you've completed the [Quick Start](./quickstart) to install the client ```bash -# Set mission -hindsight bank mission my-bank "I am a research assistant specializing in ML" - -# Set disposition -hindsight bank disposition my-bank \ - --skepticism 4 \ - --literalism 3 \ - --empathy 3 +hindsight bank create my-bank ``` -## Mission and Disposition +## Bank Configuration -Mission and disposition are optional settings that influence how the bank reasons during [reflect](./reflect) operations. +Each memory bank can be configured independently per operation. Configuration can be set via the [bank config API](#updating-configuration), the [Control Plane UI](/), or [server-wide environment variables](/developer/configuration). -:::info -Mission and disposition only affect the `reflect` operation. They do not impact `retain`, `recall`, or other memory operations. -::: +### retain_mission {#retain-configuration} -### Mission +A plain-language description of what this bank should pay attention to during extraction. The mission is injected into the extraction prompt alongside the built-in rules — it steers focus without replacing the extraction logic. -The mission is a first-person narrative providing context for reasoning: +``` +e.g. Always include technical decisions, API design choices, and architectural trade-offs. + Ignore meeting logistics, greetings, and social exchanges. +``` + +Works alongside any extraction mode. Leave blank for general-purpose extraction. + +### retain_extraction_mode + +Controls how aggressively facts are extracted: + +| Mode | Description | +|------|-------------| +| `concise` *(default)* | Selective — only facts worth remembering long-term | +| `verbose` | Captures more detail per fact; slower and uses more tokens | +| `custom` | Write your own extraction rules via `retain_custom_instructions` | + +### retain_custom_instructions + +Only active when `retain_extraction_mode` is `custom`. Replaces the built-in extraction rules entirely with your own instructions. + +See [Retain configuration](/developer/configuration#retain) for environment variable names and defaults. + +### enable_observations {#observations-configuration} + +Toggles automatic observation consolidation on or off. Defaults to `true` when the observations feature is enabled on the server. + +### observations_mission + +Defines what this bank should synthesise into durable observations. Replaces the built-in consolidation rules entirely — leave blank to use the server default. + +``` +e.g. Observations are stable facts about people and projects. + Always include preferences, skills, and recurring patterns. + Ignore one-off events and ephemeral state. +``` + +See [Observations configuration](/developer/configuration#observations) for environment variable names and defaults. + +### mission + +A first-person narrative that provides identity and framing context for `reflect`. The agent uses this to ground its reasoning and apply a consistent perspective. + +``` +e.g. You are a senior engineering assistant. + Always ground answers in documented decisions and rationale. + Ignore speculation. Be direct and precise. +``` + +### disposition_skepticism + +How skeptical vs trusting the bank is when evaluating claims during `reflect`. Scale 1–5. - + - + -### Disposition Traits +| Value | Behaviour | +|-------|-----------| +| `1` | Trusting — accepts information at face value | +| `3` *(default)* | Balanced | +| `5` | Skeptical — questions and doubts claims | -Disposition traits influence how reasoning is performed during reflection. Each trait is scored 1 to 5: +### disposition_literalism -| Trait | Low (1) | High (5) | -|-------|---------|----------| -| **Skepticism** | Trusting, accepts information at face value | Skeptical, questions and doubts claims | -| **Literalism** | Flexible interpretation, reads between the lines | Literal interpretation, takes things exactly as stated | -| **Empathy** | Detached, focuses on facts and logic | Empathetic, considers emotional context | +How literally to interpret information during `reflect`. Scale 1–5. + +| Value | Behaviour | +|-------|-----------| +| `1` | Flexible — reads between the lines, considers context | +| `3` *(default)* | Balanced | +| `5` | Literal — takes things exactly as stated | + +### disposition_empathy + +How much to weight emotional context when reasoning during `reflect`. Scale 1–5. + +| Value | Behaviour | +|-------|-----------| +| `1` | Detached — focuses on facts and logic | +| `3` *(default)* | Balanced | +| `5` | Empathetic — considers emotional context | + +:::info +Disposition traits and `mission` only affect the `reflect` operation. `retain_mission` and `observations_mission` are separate per-operation settings. +::: + +--- + +## Updating Configuration + +Bank configuration fields (retain mission, extraction mode, observations mission, etc.) are managed via a **separate config API**, not the `create_bank` call. This lets you change operational settings independently from the bank's identity and disposition. + +### Setting Configuration Overrides + + + + + + + + + + +You can update any subset of fields — only the keys you provide are changed. + +### Reading the Current Configuration + + + + + + + + + + +The response distinguishes: +- **`config`** — the fully resolved configuration (server defaults merged with bank overrides) +- **`overrides`** — only the fields explicitly overridden for this bank + +### Resetting to Defaults + + + + + + + + + + +This removes all bank-level overrides. The bank reverts to server-wide defaults (set via environment variables). + +You can also update configuration directly from the [Control Plane UI](/) — navigate to a bank and open the **Configuration** tab. + +--- ## Directives diff --git a/hindsight-docs/versioned_docs/version-0.4/developer/api/mental-models.mdx b/hindsight-docs/versioned_docs/version-0.4/developer/api/mental-models.mdx index 1bdcc5ab..8873ac54 100644 --- a/hindsight-docs/versioned_docs/version-0.4/developer/api/mental-models.mdx +++ b/hindsight-docs/versioned_docs/version-0.4/developer/api/mental-models.mdx @@ -244,6 +244,34 @@ curl -X DELETE "http://localhost:8888/v1/default/banks/my-bank/mental-models/{me --- +## Tags and Visibility + +Mental models support the same tag system as memories. When you assign tags to a mental model, those tags control both **which memories it reads** during refresh and **when it is surfaced** during reflect. + +### How tags affect mental model refresh + +When a mental model is refreshed (manually or automatically), it runs an internal reflect call to regenerate its content. If the mental model has tags, that reflect call uses `all_strict` tag matching — meaning it will only read memories that carry **all** of the mental model's tags. Untagged memories are excluded. + +``` +Mental model tags: ["user:alice"] + +During refresh, it reads: + ✅ "Alice prefers async communication" — has "user:alice" + ✅ "Team uses Slack for announcements" — has "user:alice" (plus other tags) + ❌ "Company policy: no meetings on Fridays" — untagged, excluded + ❌ "Bob dislikes long meetings" — no "user:alice" tag +``` + +This means a mental model tagged `["user:alice"]` will also pick up memories tagged `["user:alice", "team"]` — extra tags on a memory don't disqualify it. Only the mental model's own tags are required to be present. + +### How tags affect mental model lookup during reflect + +When you call `reflect` with tags, those same tags are used to filter which mental models the agent can see. A mental model is visible only if its tags overlap with the tags on the reflect request. + +For more details on tag matching modes (`any`, `any_strict`, `all`, `all_strict`) and worked examples, see the [Recall tags reference](./recall#tags). + +--- + ## Use Cases | Use Case | Example | diff --git a/hindsight-docs/versioned_docs/version-0.4/developer/api/recall.mdx b/hindsight-docs/versioned_docs/version-0.4/developer/api/recall.mdx index 3433c43a..a6ef4f4b 100644 --- a/hindsight-docs/versioned_docs/version-0.4/developer/api/recall.mdx +++ b/hindsight-docs/versioned_docs/version-0.4/developer/api/recall.mdx @@ -130,28 +130,59 @@ Filters recall to only memories that match the specified tags. When omitted, all The `tags_match` parameter controls the filtering logic: -- `any` (default) — memory matches if it has at least one of the specified tags, or has no tags at all. Use this for "user-specific + shared global" patterns. -- `any_strict` — memory matches if it has at least one of the specified tags, and untagged memories are excluded. Use this when you want only explicitly scoped memories. -- `all` — memory matches if it has every specified tag, or has no tags at all. -- `all_strict` — memory matches if it has every specified tag, and untagged memories are excluded. +| Mode | Untagged memories | Match condition | +|------|-------------------|-----------------| +| `any` (default) | Included | Memory has **at least one** of the specified tags | +| `any_strict` | Excluded | Memory has **at least one** of the specified tags | +| `all` | Included | Memory has **all** of the specified tags | +| `all_strict` | Excluded | Memory has **all** of the specified tags | - - - - - +#### Scenario setup - - - - - +Consider a bank with these four memories: - - - - - +| Memory | Tags | +|--------|------| +| "Alice prefers async communication" | `["user:alice"]` | +| "Bob dislikes long meetings" | `["user:bob"]` | +| "Team uses Slack for announcements" | `["user:alice", "team"]` | +| "Company policy: no meetings on Fridays" | *(untagged)* | + +#### `any` — OR matching, includes untagged (default) + +Returns memories that have **at least one** matching tag, plus untagged memories. + + + +Use this for **shared global knowledge + user-specific** patterns, where untagged memories represent information everyone should see. + +#### `any_strict` — OR matching, excludes untagged + +Same as `any` but untagged memories are excluded. + + + +Use this when memories are **fully partitioned by tags** and untagged memories should never be visible. + +#### `all` — AND matching, includes untagged + +Returns memories that have **every** specified tag, plus untagged memories. + + + +Use this when memories must belong to a **specific intersection** of scopes (e.g., only memories relevant to both a user and a project), while still surfacing shared global knowledge. + +#### `all_strict` — AND matching, excludes untagged + +Returns memories that have **every** specified tag, and excludes untagged memories. + + + +Use this for strict scope enforcement where a memory must explicitly belong to **all** specified contexts. + +:::tip Extra tags are fine +A memory with tags `["user:alice", "team", "project:x"]` will still match a filter of `["user:alice", "team"]` under `all_strict` — extra tags on the memory are not a problem. The filter only requires the memory to contain **at least** the specified tags. +::: ### trace diff --git a/hindsight-docs/versioned_docs/version-0.4/developer/api/retain.mdx b/hindsight-docs/versioned_docs/version-0.4/developer/api/retain.mdx index f95d5b52..7149b59c 100644 --- a/hindsight-docs/versioned_docs/version-0.4/developer/api/retain.mdx +++ b/hindsight-docs/versioned_docs/version-0.4/developer/api/retain.mdx @@ -90,7 +90,7 @@ Providing context consistently is one of the highest-leverage things you can do ### metadata -Arbitrary key-value string pairs attached to every fact extracted from this item. For example: `{"source": "slack", "channel": "engineering", "thread_id": "T123"}`. The LLM never sees this field — it is passed through as-is and stored on each memory unit. During recall, every returned memory includes its metadata, which lets you do client-side filtering or static enrichment without extra lookups — for example, linking a memory back to its source URL, thread ID, or any application-specific identifier. +Arbitrary key-value string pairs that provide context about this item. For example: `{"source": "slack", "channel": "engineering", "thread_id": "T123"}`. Metadata is included in the fact extraction prompt, so the LLM can use it as additional context when extracting facts — for instance, knowing the document title or source can improve accuracy. It is also stored on each memory unit and returned with every recalled memory, letting you do client-side filtering or static enrichment without extra lookups — for example, linking a memory back to its source URL, thread ID, or any application-specific identifier. ### document_id diff --git a/hindsight-docs/versioned_docs/version-0.4/developer/configuration.md b/hindsight-docs/versioned_docs/version-0.4/developer/configuration.md index 35e8d3cf..cb5aea1b 100644 --- a/hindsight-docs/versioned_docs/version-0.4/developer/configuration.md +++ b/hindsight-docs/versioned_docs/version-0.4/developer/configuration.md @@ -408,7 +408,7 @@ Supported OpenAI embedding dimensions: | Variable | Description | Default | |----------|-------------|---------| -| `HINDSIGHT_API_RERANKER_PROVIDER` | Provider: `local`, `tei`, `cohere`, `flashrank`, `litellm`, `litellm-sdk`, or `rrf` | `local` | +| `HINDSIGHT_API_RERANKER_PROVIDER` | Provider: `local`, `tei`, `cohere`, `zeroentropy`, `flashrank`, `litellm`, `litellm-sdk`, or `rrf` | `local` | | `HINDSIGHT_API_RERANKER_LOCAL_MODEL` | Model for local provider | `cross-encoder/ms-marco-MiniLM-L-6-v2` | | `HINDSIGHT_API_RERANKER_LOCAL_MAX_CONCURRENT` | Max concurrent local reranking (prevents CPU thrashing under load) | `4` | | `HINDSIGHT_API_RERANKER_LOCAL_TRUST_REMOTE_CODE` | Allow loading models with custom code (security risk, disabled by default) | `false` | @@ -424,6 +424,8 @@ Supported OpenAI embedding dimensions: | `HINDSIGHT_API_RERANKER_LITELLM_SDK_API_KEY` | LiteLLM **SDK** API key for direct reranking (no proxy needed) | - | | `HINDSIGHT_API_RERANKER_LITELLM_SDK_MODEL` | LiteLLM SDK rerank model (e.g., `deepinfra/Qwen3-reranker-8B`) | `cohere/rerank-english-v3.0` | | `HINDSIGHT_API_RERANKER_LITELLM_SDK_API_BASE` | Custom API base URL for LiteLLM SDK (optional) | - | +| `HINDSIGHT_API_RERANKER_ZEROENTROPY_API_KEY` | ZeroEntropy API key for reranking | - | +| `HINDSIGHT_API_RERANKER_ZEROENTROPY_MODEL` | ZeroEntropy rerank model (`zerank-2`, `zerank-2-small`) | `zerank-2` | | `HINDSIGHT_API_RERANKER_FLASHRANK_MODEL` | FlashRank model for fast CPU-based reranking | `ms-marco-MiniLM-L-12-v2` | | `HINDSIGHT_API_RERANKER_FLASHRANK_CACHE_DIR` | Cache directory for FlashRank models | System default | @@ -453,6 +455,11 @@ export HINDSIGHT_API_RERANKER_COHERE_API_KEY=your-azure-api-key export HINDSIGHT_API_RERANKER_COHERE_MODEL=rerank-english-v3.0 export HINDSIGHT_API_RERANKER_COHERE_BASE_URL=https://your-azure-cohere-endpoint.com +# ZeroEntropy - cloud-based reranking (state-of-the-art accuracy) +export HINDSIGHT_API_RERANKER_PROVIDER=zeroentropy +export HINDSIGHT_API_RERANKER_ZEROENTROPY_API_KEY=your-api-key +export HINDSIGHT_API_RERANKER_ZEROENTROPY_MODEL=zerank-2 # or zerank-2-small + # LiteLLM proxy - unified gateway for multiple reranking providers (requires running LiteLLM proxy server) export HINDSIGHT_API_RERANKER_PROVIDER=litellm export HINDSIGHT_API_RERANKER_LITELLM_API_BASE=http://localhost:4000 @@ -540,42 +547,51 @@ Controls the retain (memory ingestion) pipeline. | `HINDSIGHT_API_RETAIN_MAX_COMPLETION_TOKENS` | Max completion tokens for fact extraction LLM calls | `64000` | | `HINDSIGHT_API_RETAIN_CHUNK_SIZE` | Max characters per chunk for fact extraction. Larger chunks extract fewer LLM calls but may lose context. | `3000` | | `HINDSIGHT_API_RETAIN_EXTRACTION_MODE` | Fact extraction mode: `concise`, `verbose`, or `custom` | `concise` | -| `HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS` | Custom extraction guidelines (only used when mode is `custom`) | - | +| `HINDSIGHT_API_RETAIN_MISSION` | What this bank should pay attention to during extraction. Steers the LLM without replacing the extraction rules — works alongside any extraction mode. | - | +| `HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS` | Full prompt override for fact extraction (only used when mode is `custom`). Replaces built-in extraction rules entirely. | - | | `HINDSIGHT_API_RETAIN_EXTRACT_CAUSAL_LINKS` | Extract causal relationships between facts | `true` | | `HINDSIGHT_API_RETAIN_BATCH_ENABLED` | Use LLM Batch API for fact extraction (50% cost savings, only with async operations) | `false` | | `HINDSIGHT_API_RETAIN_BATCH_POLL_INTERVAL_SECONDS` | Batch API polling interval in seconds | `60` | -#### Extraction Modes +#### Customizing retain: when to use what -The extraction mode controls how aggressively facts are extracted from content: +There are three levels of customization for the retain pipeline. Start with the simplest that covers your needs: -- **`concise`** (default): Selective extraction that focuses on significant, long-term valuable facts. Filters out greetings, filler, and trivial information. Produces fewer but higher-quality facts with better performance. +| Goal | Use | +|------|-----| +| Steer what topics to focus on or deprioritize | `HINDSIGHT_API_RETAIN_MISSION` | +| Extract more detail per fact | `HINDSIGHT_API_RETAIN_EXTRACTION_MODE=verbose` | +| Completely replace the extraction rules | `HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom` + `HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS` | -- **`verbose`**: Detailed extraction that captures every piece of information with maximum verbosity. Produces more facts with extensive detail but slower performance and higher token usage. +**`HINDSIGHT_API_RETAIN_MISSION` — steer extraction without replacing it (recommended starting point)** -- **`custom`**: Inject your own extraction guidelines while keeping the structural parts of the prompt (output format, coreference resolution, temporal handling, etc.) intact. Useful for A/B testing different extraction strategies or domain-specific customization. - -**Example: Custom Extraction Mode** +Tell the bank what to pay attention to during extraction, in plain language. The mission is injected into the extraction prompt alongside the built-in rules — it narrows focus without replacing the underlying logic. Works with any extraction mode (`concise`, `verbose`, `custom`). ```bash -# Set mode to custom -export HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom +export HINDSIGHT_API_RETAIN_MISSION="Focus on technical decisions, architecture choices, and team member expertise. Deprioritize social or personal information." +``` -# Define custom guidelines (multi-line is fine) +**`HINDSIGHT_API_RETAIN_EXTRACTION_MODE=verbose` — more detail per fact** + +Use when you need richer facts with full context, relationships, and verbosity. Slower and uses more tokens than `concise`. + +**`HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom` + `HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS` — full control** + +Replaces the built-in selectivity rules entirely. The structural parts of the prompt (output format, temporal handling, coreference resolution) remain intact — only the extraction guidelines are replaced. + +Use this when `retain_mission` isn't sufficient and you need strict inclusion/exclusion logic. + +```bash +export HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom export HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS="ONLY extract facts that are: ✅ Technical decisions and their rationale ✅ Architecture patterns and design choices ✅ Performance metrics and benchmarks -✅ Code reviews and feedback DO NOT extract: -❌ Generic greetings or pleasantries +❌ Greetings or social conversation ❌ Process chatter (\"let me check\", \"one moment\") -❌ Repeated information already captured - -CONSOLIDATE related technical discussions into ONE fact when possible. - -Ask yourself: 'Would this technical context be useful in 6 months?' If no, skip it." +❌ Anything that would not be useful in 6 months" ``` ### File Processing @@ -708,7 +724,7 @@ export HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_KEY=base64encodedkey== For production deployments, use `s3`, `gcs`, or `azure` to avoid storing large binary files in your PostgreSQL database. Set `HINDSIGHT_API_FILE_DELETE_AFTER_RETAIN=true` (the default) to delete files after memory extraction, which minimizes storage costs. ::: -### Observations (Experimental) +### Observations (Experimental) {#observations} Observations are consolidated knowledge synthesized from facts. @@ -717,12 +733,64 @@ Observations are consolidated knowledge synthesized from facts. | `HINDSIGHT_API_ENABLE_OBSERVATIONS` | Enable observation consolidation | `true` | | `HINDSIGHT_API_CONSOLIDATION_BATCH_SIZE` | Memories to load per batch (internal optimization) | `50` | | `HINDSIGHT_API_CONSOLIDATION_MAX_TOKENS` | Max tokens for recall when finding related observations during consolidation | `1024` | +| `HINDSIGHT_API_CONSOLIDATION_LLM_BATCH_SIZE` | Number of facts sent to the LLM in a single consolidation call. Higher values reduce LLM calls and improve throughput at the cost of larger prompts. Set to `1` to disable batching. | `8` | +| `HINDSIGHT_API_OBSERVATIONS_MISSION` | What this bank should synthesise into durable observations. Replaces the built-in consolidation rules — leave unset to use the server default. | - | + +#### Customizing observations: when to use what + +| Goal | Use | +|------|-----| +| Default behavior: durable specific facts, no ephemeral state | Leave unset | +| Change what observations *are* for this bank (different shape, different purpose) | `HINDSIGHT_API_OBSERVATIONS_MISSION` | + +**`HINDSIGHT_API_OBSERVATIONS_MISSION` — redefine what this bank synthesises** + +By default, observations are durable, specific facts synthesized from memories — the kind of knowledge that stays true over time (preferences, skills, relationships, recurring patterns). Ephemeral state is filtered out. Contradictions are tracked with temporal markers. + +Set `HINDSIGHT_API_OBSERVATIONS_MISSION` to replace this definition entirely. Write a plain-language description of what observations should be for your use case. The LLM will use this instead of the default rules when deciding what to create or update. Leave it unset to keep the server default. + +:::tip When to use observations_mission +Use it when the default durable-knowledge behavior doesn't match your use case. Common scenarios: +- You want **broader event summaries** rather than isolated facts +- You want observations **grouped by time period** (weekly, monthly) +- You want a **different granularity** (one observation per project rather than per fact) +- You have a **domain-specific** notion of what's worth remembering +::: + +**Example: Weekly event summaries** + +```bash +export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are broad summaries of project events grouped by week. Each observation should capture what happened, what was decided, and what was blocked — not individual facts. Merge related events into cohesive weekly narratives." +``` + +**Example: Person-centric knowledge** + +```bash +export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are durable facts about specific named people: their preferences, skills, relationships, and behavioral patterns. Only create observations for facts that are stable over time and tied to a named individual." +``` + +**Example: Support ticket patterns** + +```bash +export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are recurring patterns in customer support interactions: common failure modes, frequently requested features, and pain points that appear across multiple tickets." +``` ### Reflect | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_REFLECT_MAX_ITERATIONS` | Max tool call iterations before forcing a response | `10` | +| `HINDSIGHT_API_REFLECT_MISSION` | Global reflect mission (identity and reasoning framing). Overridden per bank via config API. | - | + +#### Disposition + +Disposition traits control how the bank reasons during reflect operations. Each trait is on a scale of 1–5. These are hierarchical — they can be overridden per bank via the [config API](./configuration.md#hierarchical-configuration). + +| Variable | Description | Default | +|----------|-------------|---------| +| `HINDSIGHT_API_DISPOSITION_SKEPTICISM` | How skeptical vs trusting (1=trusting, 5=skeptical) | `3` | +| `HINDSIGHT_API_DISPOSITION_LITERALISM` | How literally to interpret information (1=flexible, 5=literal) | `3` | +| `HINDSIGHT_API_DISPOSITION_EMPATHY` | How much to consider emotional context (1=detached, 5=empathetic) | `3` | ### MCP Server @@ -731,10 +799,36 @@ Configuration for MCP server endpoints. | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_MCP_ENABLED` | Enable MCP server at `/mcp/{bank_id}/` | `true` | +| `HINDSIGHT_API_MCP_ENABLED_TOOLS` | Comma-separated allowlist of MCP tools to expose globally (empty = all tools) | - | | `HINDSIGHT_API_MCP_AUTH_TOKEN` | Bearer token for MCP authentication (optional) | - | | `HINDSIGHT_API_MCP_LOCAL_BANK_ID` | Memory bank ID for local MCP | `mcp` | | `HINDSIGHT_API_MCP_INSTRUCTIONS` | Additional instructions appended to retain/recall tool descriptions | - | +**Tool Access Control:** + +`HINDSIGHT_API_MCP_ENABLED_TOOLS` restricts which MCP tools are registered at the server level. This is useful for read-only deployments or limiting surface area: + +```bash +# Expose only recall (read-only deployment) +export HINDSIGHT_API_MCP_ENABLED_TOOLS=recall + +# Expose recall and reflect only +export HINDSIGHT_API_MCP_ENABLED_TOOLS=recall,reflect +``` + +Available tool names: `retain`, `recall`, `reflect`, `list_banks`, `create_bank`, `list_mental_models`, `get_mental_model`, `create_mental_model`, `update_mental_model`, `delete_mental_model`, `refresh_mental_model`, `list_directives`, `create_directive`, `delete_directive`, `list_memories`, `get_memory`, `delete_memory`, `list_documents`, `get_document`, `delete_document`, `list_operations`, `get_operation`, `cancel_operation`, `list_tags`, `get_bank`, `get_bank_stats`, `update_bank`, `delete_bank`, `clear_memories`. + +This can also be overridden per bank via the [config API](#hierarchical-configuration): + +```bash +# Restrict a specific bank to read-only MCP access +curl -X PATCH http://localhost:8888/v1/default/banks/my-bank/config \ + -H "Content-Type: application/json" \ + -d '{"updates": {"mcp_enabled_tools": ["recall"]}}' +``` + +When a bank-level `mcp_enabled_tools` is set, tools not in the list return a clear error when invoked (they still appear in the tools list for MCP protocol compatibility). + **MCP Authentication:** By default, the MCP endpoint is open. For production deployments, set `HINDSIGHT_API_MCP_AUTH_TOKEN` to require Bearer token authentication: @@ -913,8 +1007,9 @@ This design prevents bugs where global defaults are used instead of bank overrid Configuration fields are categorized for security: 1. **Configurable Fields** - Safe behavioral settings that can be customized per-bank: - - Retention: `retain_chunk_size`, `retain_extraction_mode`, `retain_custom_instructions` - - Consolidation: `enable_observations` + - Retention: `retain_chunk_size`, `retain_extraction_mode`, `retain_mission`, `retain_custom_instructions` + - Observations: `enable_observations`, `observations_mission` + - MCP access control: `mcp_enabled_tools` 2. **Credential Fields** - NEVER exposed or configurable via API: - API keys: `*_api_key` (all LLM API keys) @@ -929,13 +1024,7 @@ Configuration fields are categorized for security: | Variable | Description | Default | |----------|-------------|---------| -| `HINDSIGHT_API_ENABLE_BANK_CONFIG_API` | Enable per-bank config API | `false` | - -**Important:** The bank config API is **disabled by default** for security. Enable it explicitly: - -```bash -export HINDSIGHT_API_ENABLE_BANK_CONFIG_API=true -``` +| `HINDSIGHT_API_ENABLE_BANK_CONFIG_API` | Enable per-bank config API | `true` | #### API Endpoints diff --git a/hindsight-docs/versioned_docs/version-0.4/developer/mcp-server.md b/hindsight-docs/versioned_docs/version-0.4/developer/mcp-server.md index 5048bd53..3a8b61cf 100644 --- a/hindsight-docs/versioned_docs/version-0.4/developer/mcp-server.md +++ b/hindsight-docs/versioned_docs/version-0.4/developer/mcp-server.md @@ -100,12 +100,12 @@ The MCP server operates in two modes depending on the URL: | Mode | URL | Tools | bank_id | |------|-----|-------|---------| -| **Single-bank** | `/mcp/{bank_id}/` | Memory + mental model tools | Implicit from URL | -| **Multi-bank** | `/mcp/` | All tools including bank management | Explicit `bank_id` parameter on each tool | +| **Single-bank** | `/mcp/{bank_id}/` | 26 tools (memory, mental models, directives, documents, operations, tags, bank management) | Implicit from URL | +| **Multi-bank** | `/mcp/` | All 29 tools including `list_banks`, `create_bank`, `get_bank_stats` | Explicit `bank_id` parameter on each tool | **Single-bank mode** (recommended) scopes all operations to the bank in the URL. Tools don't expose a `bank_id` parameter. -**Multi-bank mode** exposes all tools with an optional `bank_id` parameter, plus bank management tools (`list_banks`, `create_bank`). +**Multi-bank mode** exposes all tools with an optional `bank_id` parameter, plus bank management tools (`list_banks`, `create_bank`, `get_bank_stats`). --- @@ -120,6 +120,9 @@ Store information to long-term memory. | `content` | string | Yes | The fact or memory to store | | `context` | string | No | Category for the memory (default: `general`) | | `timestamp` | string | No | ISO 8601 timestamp for when the event occurred | +| `tags` | list[string] | No | Tags for organizing and filtering this memory | +| `metadata` | object | No | Key-value metadata to attach (e.g., `{"source": "slack"}`) | +| `document_id` | string | No | Associate this memory with an existing document | **Example:** ```json @@ -127,7 +130,8 @@ Store information to long-term memory. "name": "retain", "arguments": { "content": "User prefers Python over JavaScript for backend development", - "context": "programming_preferences" + "context": "programming_preferences", + "tags": ["user:alice", "preferences"] } } ``` @@ -148,13 +152,20 @@ Search memories to provide personalized responses. |-----------|------|----------|-------------| | `query` | string | Yes | Natural language search query | | `max_tokens` | integer | No | Maximum tokens to return (default: 4096) | +| `budget` | string | No | Search thoroughness: `low`, `mid`, or `high` (default: `high`) | +| `types` | list[string] | No | Filter by fact type: `world`, `experience`, `opinion`. Defaults to all | +| `tags` | list[string] | No | Filter memories by tags | +| `tags_match` | string | No | Tag matching mode: `any` (default) or `all` | +| `query_timestamp` | string | No | ISO 8601 timestamp — recall as if asking at this point in time | **Example:** ```json { "name": "recall", "arguments": { - "query": "What are the user's programming language preferences?" + "query": "What are the user's programming language preferences?", + "tags": ["preferences"], + "budget": "high" } } ``` @@ -176,6 +187,10 @@ Generate thoughtful analysis by synthesizing stored memories with the bank's per | `query` | string | Yes | The question or topic to reflect on | | `context` | string | No | Optional context about why this reflection is needed | | `budget` | string | No | Search budget: `low`, `mid`, or `high` (default: `low`) | +| `max_tokens` | integer | No | Maximum tokens in the response (default: 4096) | +| `response_schema` | object | No | JSON Schema for structured output. When provided, the response includes a `structured_output` field | +| `tags` | list[string] | No | Filter memories by tags before reflecting | +| `tags_match` | string | No | Tag matching mode: `any` (default) or `all` | **Example:** ```json @@ -183,7 +198,8 @@ Generate thoughtful analysis by synthesizing stored memories with the bank's per "name": "reflect", "arguments": { "query": "Based on my past decisions, what architectural style do I prefer?", - "budget": "mid" + "budget": "mid", + "tags": ["architecture"] } } ``` @@ -206,6 +222,7 @@ Create a mental model — a living document that stays current with your memorie | `mental_model_id` | string | No | Custom ID (alphanumeric lowercase with hyphens). Auto-generated if not provided | | `tags` | list[string] | No | Tags for organizing and filtering models | | `max_tokens` | integer | No | Maximum tokens for model content (default: 2048) | +| `trigger_refresh_after_consolidation` | boolean | No | Auto-refresh this model after memory consolidation (default: `false`) | **Example:** ```json @@ -254,6 +271,7 @@ Update a mental model's metadata or settings. | `source_query` | string | No | New source query | | `tags` | list[string] | No | New tags | | `max_tokens` | integer | No | New max tokens | +| `trigger_refresh_after_consolidation` | boolean | No | Auto-refresh after consolidation. Only set when you want to change this setting | --- @@ -295,6 +313,186 @@ Create a new memory bank or retrieve an existing one. --- +### list_directives + +List all directives in a bank. Directives are instructions that guide how the memory system processes and responds to queries. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `tags` | list[string] | No | Filter directives by tags | +| `active_only` | boolean | No | Only return active directives (default: `true`) | + +--- + +### create_directive + +Create a new directive in a bank. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `name` | string | Yes | Human-readable name for the directive | +| `content` | string | Yes | The directive content/instruction | +| `priority` | integer | No | Priority level (higher = more important) | +| `is_active` | boolean | No | Whether the directive is active (default: `true`) | +| `tags` | list[string] | No | Tags for organizing directives | + +--- + +### delete_directive + +Delete a directive by ID. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `directive_id` | string | Yes | The ID of the directive to delete | + +--- + +### list_memories + +Browse stored memories with optional filtering and pagination. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `type` | string | No | Filter by fact type: `world`, `experience`, or `opinion` | +| `q` | string | No | Search query to filter memories | +| `limit` | integer | No | Maximum number of results (default: 100) | +| `offset` | integer | No | Number of results to skip for pagination (default: 0) | + +--- + +### get_memory + +Retrieve a specific memory by ID. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `memory_id` | string | Yes | The ID of the memory to retrieve | + +--- + +### delete_memory + +Permanently delete a specific memory. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `memory_id` | string | Yes | The ID of the memory to delete | + +--- + +### list_documents + +List documents that have been ingested into the memory bank. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `q` | string | No | Search query to filter documents | +| `limit` | integer | No | Maximum number of results (default: 100) | + +--- + +### get_document + +Retrieve a specific document by ID, including its metadata. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `document_id` | string | Yes | The ID of the document to retrieve | + +--- + +### delete_document + +Delete a document and all memories linked to it. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `document_id` | string | Yes | The ID of the document to delete | + +--- + +### list_operations + +List async operations (retain processing, mental model refresh, etc.) with optional status filtering. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `status` | string | No | Filter by status: `pending`, `running`, `completed`, `failed`, `cancelled` | +| `limit` | integer | No | Maximum number of results (default: 100) | + +--- + +### get_operation + +Get the status and details of an async operation. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `operation_id` | string | Yes | The ID of the operation to check | + +--- + +### cancel_operation + +Cancel a pending or running async operation. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `operation_id` | string | Yes | The ID of the operation to cancel | + +--- + +### list_tags + +List all unique tags used in a bank, optionally filtered by pattern. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `q` | string | No | Glob pattern to filter tags (e.g., `project:*`) | +| `limit` | integer | No | Maximum number of results (default: 100) | + +--- + +### get_bank + +Get information about a memory bank, including its name, mission, and disposition. + +--- + +### get_bank_stats (multi-bank mode only) + +Get statistics for a memory bank (node/link counts). + +--- + +### update_bank + +Update a memory bank's metadata. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `name` | string | No | New human-friendly name for the bank | +| `mission` | string | No | New mission describing who the agent is and what they're trying to accomplish | + +--- + +### delete_bank + +Permanently delete a memory bank and all its data (memories, documents, entities, mental models). + +--- + +### clear_memories + +Clear all memories from a bank without deleting the bank itself. Optionally filter by fact type to only clear specific kinds of memories. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `type` | string | No | Fact type to clear: `world`, `experience`, or `opinion`. If not specified, clears all | + +--- + ## Integration with AI Assistants The MCP server can be used with any MCP-compatible AI assistant. See the [Authentication](#authentication) section above for Claude Code and Claude Desktop configuration examples. diff --git a/hindsight-docs/versioned_docs/version-0.4/developer/observations.mdx b/hindsight-docs/versioned_docs/version-0.4/developer/observations.mdx index f1066459..b59de72e 100644 --- a/hindsight-docs/versioned_docs/version-0.4/developer/observations.mdx +++ b/hindsight-docs/versioned_docs/version-0.4/developer/observations.mdx @@ -131,26 +131,64 @@ This ensures responses stay accurate even as the underlying data changes. --- -## Mission-Oriented Consolidation +## Observations Mission -The bank's **mission** directly influences what knowledge gets consolidated into observations. When you set a mission on your memory bank, the consolidation engine focuses on extracting knowledge that serves that mission. +You can define exactly what this bank should synthesise by setting an **observations mission** (`observations_mission`). This replaces the built-in durable-knowledge rules with your own instructions, letting you control what shape observations take. -**Example:** +``` +e.g. Observations are stable facts about people and projects. + Always include preferences, skills, and recurring patterns. + Ignore one-off events and ephemeral state. +``` - +Leave it blank to use the server default — durable, specific facts that stay true over time (preferences, skills, relationships, recurring patterns), with ephemeral state filtered out. -With this mission, the consolidation engine will: -- **Prioritize** customer preferences, issue patterns, and communication styles -- **Skip** ephemeral details that don't serve support goals -- **Synthesize** observations focused on helping customers +**Examples:** -Without a mission, the engine performs general-purpose consolidation. With a mission, it becomes focused and efficient — extracting only knowledge that matters for your use case. +| `observations_mission` | What gets synthesised | +|------------------------|----------------------| +| *(unset — default)* | Durable facts: preferences, skills, relationships, recurring patterns | +| *"Observations are weekly summaries of sprint outcomes and blockers"* | Broad event summaries grouped by time period | +| *"Observations are stable facts about named individuals only"* | Person-centric knowledge, tied to specific people | +| *"Observations are recurring patterns in customer support interactions"* | Failure modes, common requests, pain points | -| Mission | Observations Focus | -|---------|-------------------| -| *Customer support agent* | Customer preferences, issue patterns, resolution history | -| *Code review assistant* | Coding patterns, team conventions, common mistakes | -| *Research assistant* | Topic expertise, source reliability, methodology preferences | +Set `observations_mission` via the [bank config API](/developer/api/memory-banks#observations-configuration) or the [`HINDSIGHT_API_OBSERVATIONS_MISSION`](/developer/configuration#observations) environment variable. + +--- + +## Observation Lifecycle & Invalidation + +### When Memories Are Deleted + +Observations are derived from source memories. When source memories are removed, Hindsight automatically keeps observations consistent: + +| Action | Effect on observations | +|--------|----------------------| +| Delete a document | All observations derived from the document's memories are deleted | +| Delete individual memories (by type) | Observations sourced from those memories are deleted | +| Delete an entire bank | All observations are deleted along with everything else | + +After deletion, the **remaining source memories** that fed the affected observations have their consolidation state reset, so they will be re-consolidated on the next consolidation run and produce fresh observations. + +### Clearing Observations for a Specific Memory + +You can clear all observations derived from a single memory without deleting the memory itself. This is useful when you want to force re-synthesis of a memory's contribution to consolidated knowledge. + +Use the `DELETE /v1/default/banks/{bank_id}/memories/{memory_id}/observations` endpoint. This will: +1. Delete all observations that list the memory as a source +2. Reset `consolidated_at` on the memory itself and any other source memories that contributed to those observations +3. Trigger a consolidation job so fresh observations are produced automatically + +### Resetting All Observations + +To wipe all consolidated knowledge and start over: + +```python +# Clear all observations for a bank +client.clear_observations(bank_id="my-bank") +``` + +This resets the consolidation state for all source memories in the bank, so the next consolidation run will re-derive all observations from scratch. --- diff --git a/hindsight-docs/versioned_docs/version-0.4/developer/reflect.mdx b/hindsight-docs/versioned_docs/version-0.4/developer/reflect.mdx index 2c93620a..7acb5888 100644 --- a/hindsight-docs/versioned_docs/version-0.4/developer/reflect.mdx +++ b/hindsight-docs/versioned_docs/version-0.4/developer/reflect.mdx @@ -117,14 +117,18 @@ When you create a memory bank, you can configure its disposition using three tra ### Mission: Natural Language Identity -Beyond numeric traits, you can provide a natural language **mission** that describes the bank's identity: +Beyond numeric traits, you can provide a natural language **mission** that describes the bank's identity and reasoning context: -The mission tells Hindsight what knowledge to prioritize and shapes how disposition traits are applied: -- "keep track of system designs" → focuses consolidation on architectural decisions -- "prefer simplicity over cutting-edge" + high skepticism → questions complex solutions -- Explicit guidance → consistent memory focus across conversations +The reflect mission frames how the agent reasons and responds: +- Provides identity context: who the agent is and what it cares about +- Shapes how disposition traits are applied in practice +- Keeps reasoning consistent across conversations + +:::info Per-operation missions +The reflect mission only affects `reflect()`. To steer what gets extracted during `retain()`, use [`retain_mission`](/developer/api/memory-banks#retain-configuration). To control what gets synthesised into observations, use [`observations_mission`](/developer/api/memory-banks#observations-configuration). +::: --- diff --git a/hindsight-docs/versioned_docs/version-0.4/developer/retain.md b/hindsight-docs/versioned_docs/version-0.4/developer/retain.md index 87b14b85..fc85d5e0 100644 --- a/hindsight-docs/versioned_docs/version-0.4/developer/retain.md +++ b/hindsight-docs/versioned_docs/version-0.4/developer/retain.md @@ -176,6 +176,29 @@ All stored in your isolated **memory bank**, ready for `recall()` and `reflect() --- +## Steering Extraction with a Mission + +By default, `retain()` extracts all significant facts from the content. You can narrow this focus with a **retain mission** (`retain_mission`) — a plain-language description of what this bank should pay attention to. + +``` +e.g. Always include technical decisions, API design choices, and architectural trade-offs. + Ignore meeting logistics, greetings, and social exchanges. +``` + +The mission is injected into the extraction prompt alongside the built-in rules — it steers the LLM without replacing the extraction logic. It works with any extraction mode (`concise`, `verbose`, `custom`). + +For finer control, you can also change the **extraction mode**: + +| Mode | When to use | +|------|-------------| +| `concise` *(default)* | General-purpose — selective, fast | +| `verbose` | When you need richer facts with full context and relationships | +| `custom` | When you want to write your own extraction rules entirely | + +Set `retain_mission` and `retain_extraction_mode` via the [bank config API](/developer/api/memory-banks#retain-configuration) or the [`HINDSIGHT_API_RETAIN_MISSION`](/developer/configuration#retain) environment variable. + +--- + ## Observation Consolidation After `retain()` completes, Hindsight automatically triggers **observation consolidation** in the background. This process: diff --git a/hindsight-docs/versioned_docs/version-0.4/sdks/integrations/chat.md b/hindsight-docs/versioned_docs/version-0.4/sdks/integrations/chat.md new file mode 100644 index 00000000..8b024f52 --- /dev/null +++ b/hindsight-docs/versioned_docs/version-0.4/sdks/integrations/chat.md @@ -0,0 +1,163 @@ +--- +sidebar_position: 5 +--- + +# Vercel Chat SDK + +The `@vectorize-io/hindsight-chat` package gives your [Vercel Chat SDK](https://github.com/vercel/chat) bots persistent, per-user memory with a single handler wrapper. Works with Slack, Discord, Teams, Google Chat, GitHub, and Linear. + +## Installation + +```bash +npm install @vectorize-io/hindsight-chat +``` + +## Quick Start + +```typescript +import { Chat } from 'chat'; +import { HindsightClient } from '@vectorize-io/hindsight-client'; +import { withHindsightChat } from '@vectorize-io/hindsight-chat'; +import { streamText } from 'ai'; +import { openai } from '@ai-sdk/openai'; + +const chat = new Chat({ connectors: [/* your connectors */] }); +const hindsight = new HindsightClient({ apiKey: process.env.HINDSIGHT_API_KEY }); + +chat.onNewMention( + withHindsightChat( + { + client: hindsight, + bankId: (msg) => msg.author.userId, // per-user memory + }, + async (thread, message, ctx) => { + await thread.subscribe(); + + const result = await streamText({ + model: openai('gpt-4o'), + system: ctx.memoriesAsSystemPrompt(), + messages: [{ role: 'user', content: message.text }], + }); + + // Stream the response + const chunks: string[] = []; + for await (const chunk of result.textStream) { + chunks.push(chunk); + } + const fullResponse = chunks.join(''); + await thread.post(fullResponse); + + // Store the conversation in memory + await ctx.retain( + `User: ${message.text}\nAssistant: ${fullResponse}` + ); + } + ) +); +``` + +## Configuration + +### `withHindsightChat(options, handler)` + +Returns a standard Chat SDK handler `(thread, message) => Promise`. + +#### Options + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `client` | `HindsightClient` | *required* | Hindsight client instance | +| `bankId` | `string \| (msg) => string` | *required* | Memory bank ID or resolver function | +| `recall.enabled` | `boolean` | `true` | Auto-recall memories before handler | +| `recall.budget` | `'low' \| 'mid' \| 'high'` | `'mid'` | Processing budget for recall | +| `recall.maxTokens` | `number` | API default | Max tokens for recall results | +| `recall.types` | `FactType[]` | all | Filter to specific fact types | +| `recall.includeEntities` | `boolean` | `true` | Include entity observations | +| `retain.enabled` | `boolean` | `false` | Auto-retain inbound messages | +| `retain.async` | `boolean` | `true` | Fire-and-forget retain | +| `retain.tags` | `string[]` | – | Tags for retained memories | +| `retain.metadata` | `Record` | – | Metadata for retained memories | + +### Context (`ctx`) + +The third argument passed to your handler: + +| Property/Method | Description | +|----------------|-------------| +| `ctx.bankId` | Resolved bank ID | +| `ctx.memories` | Array of recalled memories | +| `ctx.entities` | Entity observations (or null) | +| `ctx.memoriesAsSystemPrompt(options?)` | Format memories for LLM system prompt | +| `ctx.retain(content, options?)` | Store content in memory | +| `ctx.recall(query, options?)` | Search memories | +| `ctx.reflect(query, options?)` | Reason over memories | + +## Examples + +### Subscribed Message Handler + +```typescript +chat.onSubscribedMessage( + withHindsightChat( + { + client: hindsight, + bankId: (msg) => msg.author.userId, + recall: { budget: 'high', maxTokens: 1000 }, + }, + async (thread, message, ctx) => { + const result = await generateText({ + model: openai('gpt-4o'), + system: ctx.memoriesAsSystemPrompt(), + messages: [{ role: 'user', content: message.text }], + }); + await thread.post(result.text); + } + ) +); +``` + +### Auto-Retain Inbound Messages + +```typescript +chat.onNewMention( + withHindsightChat( + { + client: hindsight, + bankId: (msg) => msg.author.userId, + retain: { enabled: true, tags: ['slack', 'inbound'] }, + }, + async (thread, message, ctx) => { + // Inbound message is already being retained automatically + const result = await generateText({ + model: openai('gpt-4o'), + system: ctx.memoriesAsSystemPrompt(), + messages: [{ role: 'user', content: message.text }], + }); + await thread.post(result.text); + + // Retain the assistant response separately + await ctx.retain(`Assistant: ${result.text}`, { + tags: ['slack', 'outbound'], + }); + } + ) +); +``` + +### Static Bank ID (Shared Memory) + +```typescript +// All users share the same memory bank +chat.onNewMention( + withHindsightChat( + { client: hindsight, bankId: 'shared-team-memory' }, + async (thread, message, ctx) => { + // ... + } + ) +); +``` + +## Error Handling + +Memory failures never break your bot. Auto-recall and auto-retain errors are logged as warnings and the handler continues with empty memories. Manual `ctx.retain()`, `ctx.recall()`, and `ctx.reflect()` calls propagate errors normally so you can handle them as needed. diff --git a/hindsight-docs/versioned_docs/version-0.4/sdks/integrations/local-mcp.md b/hindsight-docs/versioned_docs/version-0.4/sdks/integrations/local-mcp.md index 8d3740a3..524b2b2f 100644 --- a/hindsight-docs/versioned_docs/version-0.4/sdks/integrations/local-mcp.md +++ b/hindsight-docs/versioned_docs/version-0.4/sdks/integrations/local-mcp.md @@ -66,22 +66,74 @@ claude mcp add --transport http hindsight http://localhost:8888/mcp/my-bank/ ## Available Tools -The local server exposes the full tool set: +The local server exposes the full tool set (29 tools in multi-bank mode, 26 in single-bank mode): + +**Core Memory** + +| Tool | Description | +|------|-------------| +| `retain` | Store information to long-term memory with optional tags, metadata, and document association | +| `recall` | Search memories with natural language, configurable budget, type filters, and tag filters | +| `reflect` | Synthesize memories into a reasoned answer with optional structured output | + +**Mental Models** | Tool | Description | |------|-------------| -| `retain` | Store information to long-term memory (fire-and-forget) | -| `recall` | Search memories with natural language | -| `reflect` | Synthesize memories into a reasoned answer | -| `list_banks` | List all memory banks | -| `create_bank` | Create or configure a memory bank | | `list_mental_models` | List pinned reflections for a bank | | `get_mental_model` | Get a specific mental model | -| `create_mental_model` | Create a new mental model | +| `create_mental_model` | Create a new mental model with optional auto-refresh trigger | | `update_mental_model` | Update a mental model's metadata | | `delete_mental_model` | Delete a mental model | | `refresh_mental_model` | Regenerate a mental model's content | +**Directives** + +| Tool | Description | +|------|-------------| +| `list_directives` | List directives that guide memory processing | +| `create_directive` | Create a new directive | +| `delete_directive` | Delete a directive | + +**Memory Browsing** + +| Tool | Description | +|------|-------------| +| `list_memories` | Browse memories with filtering and pagination | +| `get_memory` | Get a specific memory by ID | +| `delete_memory` | Delete a specific memory | + +**Documents** + +| Tool | Description | +|------|-------------| +| `list_documents` | List ingested documents | +| `get_document` | Get a specific document | +| `delete_document` | Delete a document and its linked memories | + +**Operations** + +| Tool | Description | +|------|-------------| +| `list_operations` | List async operations with status filtering | +| `get_operation` | Check operation status and progress | +| `cancel_operation` | Cancel a pending or running operation | + +**Tags & Bank Management** + +| Tool | Description | +|------|-------------| +| `list_tags` | List unique tags used in a bank | +| `get_bank` | Get bank profile (name, mission, disposition) | +| `get_bank_stats` | Get bank statistics (multi-bank only) | +| `update_bank` | Update bank name or mission | +| `delete_bank` | Delete an entire bank and all its data | +| `clear_memories` | Clear memories without deleting the bank | +| `list_banks` | List all memory banks (multi-bank only) | +| `create_bank` | Create or configure a memory bank (multi-bank only) | + +For detailed parameter documentation, see the [MCP Server reference](/developer/mcp-server#available-tools). + ## Environment Variables All standard [Hindsight configuration variables](/developer/configuration) are supported. Key ones for local use: diff --git a/hindsight-docs/versioned_sidebars/version-0.4-sidebars.json b/hindsight-docs/versioned_sidebars/version-0.4-sidebars.json index afb8be35..32e3e619 100644 --- a/hindsight-docs/versioned_sidebars/version-0.4-sidebars.json +++ b/hindsight-docs/versioned_sidebars/version-0.4-sidebars.json @@ -208,6 +208,14 @@ "id": "sdks/integrations/ai-sdk", "label": "Vercel AI SDK" }, + { + "type": "doc", + "id": "sdks/integrations/chat", + "label": "Vercel Chat SDK", + "customProps": { + "icon": "/img/icons/vercel.png" + } + }, { "type": "doc", "id": "sdks/integrations/crewai", diff --git a/skills/hindsight-docs/references/cookbook/index.md b/skills/hindsight-docs/references/cookbook/index.md index 3d583ac5..ba631b27 100644 --- a/skills/hindsight-docs/references/cookbook/index.md +++ b/skills/hindsight-docs/references/cookbook/index.md @@ -6,8 +6,8 @@ Learn how to build with Hindsight through practical examples: -- **[Recipes](#recipes)** - Step-by-step guides and patterns for common use cases -- **[Applications](#applications)** - Complete, runnable applications demonstrating Hindsight integration +- **Recipes** - Step-by-step guides and patterns for common use cases +- **Applications** - Complete, runnable applications demonstrating Hindsight integration **💡 Extra tags are fine** +> +A memory with tags `["user:alice", "team", "project:x"]` will still match a filter of `["user:alice", "team"]` under `all_strict` — extra tags on the memory are not a problem. The filter only requires the memory to contain **at least** the specified tags. ### trace When set to `true`, the response includes a detailed debug trace covering the query embedding, entry points, per-strategy retrieval results, RRF fusion candidates, reranked results, temporal constraints detected, and per-phase timings. Has no effect on the retrieval logic itself. Useful for understanding why specific memories were or were not returned. diff --git a/skills/hindsight-docs/references/developer/api/retain.md b/skills/hindsight-docs/references/developer/api/retain.md index 80fdbb4c..d27fc738 100644 --- a/skills/hindsight-docs/references/developer/api/retain.md +++ b/skills/hindsight-docs/references/developer/api/retain.md @@ -136,7 +136,7 @@ hindsight memory retain my-bank "Alice got promoted" \ ### metadata -Arbitrary key-value string pairs attached to every fact extracted from this item. For example: `{"source": "slack", "channel": "engineering", "thread_id": "T123"}`. The LLM never sees this field — it is passed through as-is and stored on each memory unit. During recall, every returned memory includes its metadata, which lets you do client-side filtering or static enrichment without extra lookups — for example, linking a memory back to its source URL, thread ID, or any application-specific identifier. +Arbitrary key-value string pairs that provide context about this item. For example: `{"source": "slack", "channel": "engineering", "thread_id": "T123"}`. Metadata is included in the fact extraction prompt, so the LLM can use it as additional context when extracting facts — for instance, knowing the document title or source can improve accuracy. It is also stored on each memory unit and returned with every recalled memory, letting you do client-side filtering or static enrichment without extra lookups — for example, linking a memory back to its source URL, thread ID, or any application-specific identifier. ### document_id @@ -158,7 +158,7 @@ Tags control **visibility scoping** — which memories are visible during recall Use consistent naming patterns to keep tag filtering predictable. Common conventions: `user:` for per-user scoping, `session:` for session isolation, `room:` for chat rooms, `topic:` for category filtering. The bank also exposes a list-tags endpoint that returns all tags with their memory counts, useful for UI autocomplete or wildcard expansion. -See [Recall API](./recall#filter-by-tags) for filtering by tags during retrieval. +See [Recall API](./recall#tags) for filtering by tags during retrieval. ### Response @@ -211,16 +211,13 @@ Upload files directly — Hindsight converts them to text and extracts memories ```bash # Upload a single file (PDF, DOCX, PPTX, XLSX, images, audio, and more) -hindsight memory retain-files my-bank report.pdf +hindsight memory retain-files my-bank "$SAMPLE_FILE" # Upload a directory of files -hindsight memory retain-files my-bank ./documents/ - -# Upload and wait for processing to complete (polls until done) -hindsight memory retain-files my-bank report.pdf +hindsight memory retain-files my-bank "$SCRIPT_DIR/" # Queue files for background processing (returns immediately) -hindsight memory retain-files my-bank ./documents/ --async +hindsight memory retain-files my-bank "$SCRIPT_DIR/" --async ``` ### HTTP @@ -228,7 +225,7 @@ hindsight memory retain-files my-bank ./documents/ --async ```bash # Via HTTP API (multipart/form-data) curl -X POST "${HINDSIGHT_URL}/v1/default/banks/my-bank/files/retain" \ - -F "files=@report.pdf;type=application/octet-stream" \ + -F "files=@${SAMPLE_FILE};type=application/octet-stream" \ -F "request={\"files_metadata\": [{\"context\": \"quarterly report\"}]}" ``` diff --git a/skills/hindsight-docs/references/developer/configuration.md b/skills/hindsight-docs/references/developer/configuration.md index 5d00844d..17a2c5de 100644 --- a/skills/hindsight-docs/references/developer/configuration.md +++ b/skills/hindsight-docs/references/developer/configuration.md @@ -408,7 +408,7 @@ Supported OpenAI embedding dimensions: | Variable | Description | Default | |----------|-------------|---------| -| `HINDSIGHT_API_RERANKER_PROVIDER` | Provider: `local`, `tei`, `cohere`, `flashrank`, `litellm`, `litellm-sdk`, or `rrf` | `local` | +| `HINDSIGHT_API_RERANKER_PROVIDER` | Provider: `local`, `tei`, `cohere`, `zeroentropy`, `flashrank`, `litellm`, `litellm-sdk`, or `rrf` | `local` | | `HINDSIGHT_API_RERANKER_LOCAL_MODEL` | Model for local provider | `cross-encoder/ms-marco-MiniLM-L-6-v2` | | `HINDSIGHT_API_RERANKER_LOCAL_MAX_CONCURRENT` | Max concurrent local reranking (prevents CPU thrashing under load) | `4` | | `HINDSIGHT_API_RERANKER_LOCAL_TRUST_REMOTE_CODE` | Allow loading models with custom code (security risk, disabled by default) | `false` | @@ -424,6 +424,8 @@ Supported OpenAI embedding dimensions: | `HINDSIGHT_API_RERANKER_LITELLM_SDK_API_KEY` | LiteLLM **SDK** API key for direct reranking (no proxy needed) | - | | `HINDSIGHT_API_RERANKER_LITELLM_SDK_MODEL` | LiteLLM SDK rerank model (e.g., `deepinfra/Qwen3-reranker-8B`) | `cohere/rerank-english-v3.0` | | `HINDSIGHT_API_RERANKER_LITELLM_SDK_API_BASE` | Custom API base URL for LiteLLM SDK (optional) | - | +| `HINDSIGHT_API_RERANKER_ZEROENTROPY_API_KEY` | ZeroEntropy API key for reranking | - | +| `HINDSIGHT_API_RERANKER_ZEROENTROPY_MODEL` | ZeroEntropy rerank model (`zerank-2`, `zerank-2-small`) | `zerank-2` | | `HINDSIGHT_API_RERANKER_FLASHRANK_MODEL` | FlashRank model for fast CPU-based reranking | `ms-marco-MiniLM-L-12-v2` | | `HINDSIGHT_API_RERANKER_FLASHRANK_CACHE_DIR` | Cache directory for FlashRank models | System default | @@ -453,6 +455,11 @@ export HINDSIGHT_API_RERANKER_COHERE_API_KEY=your-azure-api-key export HINDSIGHT_API_RERANKER_COHERE_MODEL=rerank-english-v3.0 export HINDSIGHT_API_RERANKER_COHERE_BASE_URL=https://your-azure-cohere-endpoint.com +# ZeroEntropy - cloud-based reranking (state-of-the-art accuracy) +export HINDSIGHT_API_RERANKER_PROVIDER=zeroentropy +export HINDSIGHT_API_RERANKER_ZEROENTROPY_API_KEY=your-api-key +export HINDSIGHT_API_RERANKER_ZEROENTROPY_MODEL=zerank-2 # or zerank-2-small + # LiteLLM proxy - unified gateway for multiple reranking providers (requires running LiteLLM proxy server) export HINDSIGHT_API_RERANKER_PROVIDER=litellm export HINDSIGHT_API_RERANKER_LITELLM_API_BASE=http://localhost:4000 @@ -540,42 +547,51 @@ Controls the retain (memory ingestion) pipeline. | `HINDSIGHT_API_RETAIN_MAX_COMPLETION_TOKENS` | Max completion tokens for fact extraction LLM calls | `64000` | | `HINDSIGHT_API_RETAIN_CHUNK_SIZE` | Max characters per chunk for fact extraction. Larger chunks extract fewer LLM calls but may lose context. | `3000` | | `HINDSIGHT_API_RETAIN_EXTRACTION_MODE` | Fact extraction mode: `concise`, `verbose`, or `custom` | `concise` | -| `HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS` | Custom extraction guidelines (only used when mode is `custom`) | - | +| `HINDSIGHT_API_RETAIN_MISSION` | What this bank should pay attention to during extraction. Steers the LLM without replacing the extraction rules — works alongside any extraction mode. | - | +| `HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS` | Full prompt override for fact extraction (only used when mode is `custom`). Replaces built-in extraction rules entirely. | - | | `HINDSIGHT_API_RETAIN_EXTRACT_CAUSAL_LINKS` | Extract causal relationships between facts | `true` | | `HINDSIGHT_API_RETAIN_BATCH_ENABLED` | Use LLM Batch API for fact extraction (50% cost savings, only with async operations) | `false` | | `HINDSIGHT_API_RETAIN_BATCH_POLL_INTERVAL_SECONDS` | Batch API polling interval in seconds | `60` | -#### Extraction Modes +#### Customizing retain: when to use what -The extraction mode controls how aggressively facts are extracted from content: +There are three levels of customization for the retain pipeline. Start with the simplest that covers your needs: -- **`concise`** (default): Selective extraction that focuses on significant, long-term valuable facts. Filters out greetings, filler, and trivial information. Produces fewer but higher-quality facts with better performance. +| Goal | Use | +|------|-----| +| Steer what topics to focus on or deprioritize | `HINDSIGHT_API_RETAIN_MISSION` | +| Extract more detail per fact | `HINDSIGHT_API_RETAIN_EXTRACTION_MODE=verbose` | +| Completely replace the extraction rules | `HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom` + `HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS` | -- **`verbose`**: Detailed extraction that captures every piece of information with maximum verbosity. Produces more facts with extensive detail but slower performance and higher token usage. +**`HINDSIGHT_API_RETAIN_MISSION` — steer extraction without replacing it (recommended starting point)** -- **`custom`**: Inject your own extraction guidelines while keeping the structural parts of the prompt (output format, coreference resolution, temporal handling, etc.) intact. Useful for A/B testing different extraction strategies or domain-specific customization. - -**Example: Custom Extraction Mode** +Tell the bank what to pay attention to during extraction, in plain language. The mission is injected into the extraction prompt alongside the built-in rules — it narrows focus without replacing the underlying logic. Works with any extraction mode (`concise`, `verbose`, `custom`). ```bash -# Set mode to custom -export HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom +export HINDSIGHT_API_RETAIN_MISSION="Focus on technical decisions, architecture choices, and team member expertise. Deprioritize social or personal information." +``` -# Define custom guidelines (multi-line is fine) +**`HINDSIGHT_API_RETAIN_EXTRACTION_MODE=verbose` — more detail per fact** + +Use when you need richer facts with full context, relationships, and verbosity. Slower and uses more tokens than `concise`. + +**`HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom` + `HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS` — full control** + +Replaces the built-in selectivity rules entirely. The structural parts of the prompt (output format, temporal handling, coreference resolution) remain intact — only the extraction guidelines are replaced. + +Use this when `retain_mission` isn't sufficient and you need strict inclusion/exclusion logic. + +```bash +export HINDSIGHT_API_RETAIN_EXTRACTION_MODE=custom export HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS="ONLY extract facts that are: ✅ Technical decisions and their rationale ✅ Architecture patterns and design choices ✅ Performance metrics and benchmarks -✅ Code reviews and feedback DO NOT extract: -❌ Generic greetings or pleasantries +❌ Greetings or social conversation ❌ Process chatter (\"let me check\", \"one moment\") -❌ Repeated information already captured - -CONSOLIDATE related technical discussions into ONE fact when possible. - -Ask yourself: 'Would this technical context be useful in 6 months?' If no, skip it." +❌ Anything that would not be useful in 6 months" ``` ### File Processing @@ -717,12 +733,64 @@ Observations are consolidated knowledge synthesized from facts. | `HINDSIGHT_API_ENABLE_OBSERVATIONS` | Enable observation consolidation | `true` | | `HINDSIGHT_API_CONSOLIDATION_BATCH_SIZE` | Memories to load per batch (internal optimization) | `50` | | `HINDSIGHT_API_CONSOLIDATION_MAX_TOKENS` | Max tokens for recall when finding related observations during consolidation | `1024` | +| `HINDSIGHT_API_CONSOLIDATION_LLM_BATCH_SIZE` | Number of facts sent to the LLM in a single consolidation call. Higher values reduce LLM calls and improve throughput at the cost of larger prompts. Set to `1` to disable batching. | `8` | +| `HINDSIGHT_API_OBSERVATIONS_MISSION` | What this bank should synthesise into durable observations. Replaces the built-in consolidation rules — leave unset to use the server default. | - | + +#### Customizing observations: when to use what + +| Goal | Use | +|------|-----| +| Default behavior: durable specific facts, no ephemeral state | Leave unset | +| Change what observations *are* for this bank (different shape, different purpose) | `HINDSIGHT_API_OBSERVATIONS_MISSION` | + +**`HINDSIGHT_API_OBSERVATIONS_MISSION` — redefine what this bank synthesises** + +By default, observations are durable, specific facts synthesized from memories — the kind of knowledge that stays true over time (preferences, skills, relationships, recurring patterns). Ephemeral state is filtered out. Contradictions are tracked with temporal markers. + +Set `HINDSIGHT_API_OBSERVATIONS_MISSION` to replace this definition entirely. Write a plain-language description of what observations should be for your use case. The LLM will use this instead of the default rules when deciding what to create or update. Leave it unset to keep the server default. + +:::tip When to use observations_mission +Use it when the default durable-knowledge behavior doesn't match your use case. Common scenarios: +- You want **broader event summaries** rather than isolated facts +- You want observations **grouped by time period** (weekly, monthly) +- You want a **different granularity** (one observation per project rather than per fact) +- You have a **domain-specific** notion of what's worth remembering +::: + +**Example: Weekly event summaries** + +```bash +export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are broad summaries of project events grouped by week. Each observation should capture what happened, what was decided, and what was blocked — not individual facts. Merge related events into cohesive weekly narratives." +``` + +**Example: Person-centric knowledge** + +```bash +export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are durable facts about specific named people: their preferences, skills, relationships, and behavioral patterns. Only create observations for facts that are stable over time and tied to a named individual." +``` + +**Example: Support ticket patterns** + +```bash +export HINDSIGHT_API_OBSERVATIONS_MISSION="Observations are recurring patterns in customer support interactions: common failure modes, frequently requested features, and pain points that appear across multiple tickets." +``` ### Reflect | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_REFLECT_MAX_ITERATIONS` | Max tool call iterations before forcing a response | `10` | +| `HINDSIGHT_API_REFLECT_MISSION` | Global reflect mission (identity and reasoning framing). Overridden per bank via config API. | - | + +#### Disposition + +Disposition traits control how the bank reasons during reflect operations. Each trait is on a scale of 1–5. These are hierarchical — they can be overridden per bank via the [config API](./configuration.md#hierarchical-configuration). + +| Variable | Description | Default | +|----------|-------------|---------| +| `HINDSIGHT_API_DISPOSITION_SKEPTICISM` | How skeptical vs trusting (1=trusting, 5=skeptical) | `3` | +| `HINDSIGHT_API_DISPOSITION_LITERALISM` | How literally to interpret information (1=flexible, 5=literal) | `3` | +| `HINDSIGHT_API_DISPOSITION_EMPATHY` | How much to consider emotional context (1=detached, 5=empathetic) | `3` | ### MCP Server @@ -731,10 +799,36 @@ Configuration for MCP server endpoints. | Variable | Description | Default | |----------|-------------|---------| | `HINDSIGHT_API_MCP_ENABLED` | Enable MCP server at `/mcp/{bank_id}/` | `true` | +| `HINDSIGHT_API_MCP_ENABLED_TOOLS` | Comma-separated allowlist of MCP tools to expose globally (empty = all tools) | - | | `HINDSIGHT_API_MCP_AUTH_TOKEN` | Bearer token for MCP authentication (optional) | - | | `HINDSIGHT_API_MCP_LOCAL_BANK_ID` | Memory bank ID for local MCP | `mcp` | | `HINDSIGHT_API_MCP_INSTRUCTIONS` | Additional instructions appended to retain/recall tool descriptions | - | +**Tool Access Control:** + +`HINDSIGHT_API_MCP_ENABLED_TOOLS` restricts which MCP tools are registered at the server level. This is useful for read-only deployments or limiting surface area: + +```bash +# Expose only recall (read-only deployment) +export HINDSIGHT_API_MCP_ENABLED_TOOLS=recall + +# Expose recall and reflect only +export HINDSIGHT_API_MCP_ENABLED_TOOLS=recall,reflect +``` + +Available tool names: `retain`, `recall`, `reflect`, `list_banks`, `create_bank`, `list_mental_models`, `get_mental_model`, `create_mental_model`, `update_mental_model`, `delete_mental_model`, `refresh_mental_model`, `list_directives`, `create_directive`, `delete_directive`, `list_memories`, `get_memory`, `delete_memory`, `list_documents`, `get_document`, `delete_document`, `list_operations`, `get_operation`, `cancel_operation`, `list_tags`, `get_bank`, `get_bank_stats`, `update_bank`, `delete_bank`, `clear_memories`. + +This can also be overridden per bank via the [config API](#hierarchical-configuration): + +```bash +# Restrict a specific bank to read-only MCP access +curl -X PATCH http://localhost:8888/v1/default/banks/my-bank/config \ + -H "Content-Type: application/json" \ + -d '{"updates": {"mcp_enabled_tools": ["recall"]}}' +``` + +When a bank-level `mcp_enabled_tools` is set, tools not in the list return a clear error when invoked (they still appear in the tools list for MCP protocol compatibility). + **MCP Authentication:** By default, the MCP endpoint is open. For production deployments, set `HINDSIGHT_API_MCP_AUTH_TOKEN` to require Bearer token authentication: @@ -913,8 +1007,9 @@ This design prevents bugs where global defaults are used instead of bank overrid Configuration fields are categorized for security: 1. **Configurable Fields** - Safe behavioral settings that can be customized per-bank: - - Retention: `retain_chunk_size`, `retain_extraction_mode`, `retain_custom_instructions` - - Consolidation: `enable_observations` + - Retention: `retain_chunk_size`, `retain_extraction_mode`, `retain_mission`, `retain_custom_instructions` + - Observations: `enable_observations`, `observations_mission` + - MCP access control: `mcp_enabled_tools` 2. **Credential Fields** - NEVER exposed or configurable via API: - API keys: `*_api_key` (all LLM API keys) diff --git a/skills/hindsight-docs/references/developer/installation.md b/skills/hindsight-docs/references/developer/installation.md index 7e870023..e30ba8be 100644 --- a/skills/hindsight-docs/references/developer/installation.md +++ b/skills/hindsight-docs/references/developer/installation.md @@ -107,7 +107,7 @@ Install it with: pip install sentence-transformers ``` ::: -See [Configuration](./configuration#embeddings-and-reranking) for all embedding provider options. +See [Configuration](./configuration#embeddings) for all embedding provider options. ### Available Tags diff --git a/skills/hindsight-docs/references/developer/mcp-server.md b/skills/hindsight-docs/references/developer/mcp-server.md index 5048bd53..3a8b61cf 100644 --- a/skills/hindsight-docs/references/developer/mcp-server.md +++ b/skills/hindsight-docs/references/developer/mcp-server.md @@ -100,12 +100,12 @@ The MCP server operates in two modes depending on the URL: | Mode | URL | Tools | bank_id | |------|-----|-------|---------| -| **Single-bank** | `/mcp/{bank_id}/` | Memory + mental model tools | Implicit from URL | -| **Multi-bank** | `/mcp/` | All tools including bank management | Explicit `bank_id` parameter on each tool | +| **Single-bank** | `/mcp/{bank_id}/` | 26 tools (memory, mental models, directives, documents, operations, tags, bank management) | Implicit from URL | +| **Multi-bank** | `/mcp/` | All 29 tools including `list_banks`, `create_bank`, `get_bank_stats` | Explicit `bank_id` parameter on each tool | **Single-bank mode** (recommended) scopes all operations to the bank in the URL. Tools don't expose a `bank_id` parameter. -**Multi-bank mode** exposes all tools with an optional `bank_id` parameter, plus bank management tools (`list_banks`, `create_bank`). +**Multi-bank mode** exposes all tools with an optional `bank_id` parameter, plus bank management tools (`list_banks`, `create_bank`, `get_bank_stats`). --- @@ -120,6 +120,9 @@ Store information to long-term memory. | `content` | string | Yes | The fact or memory to store | | `context` | string | No | Category for the memory (default: `general`) | | `timestamp` | string | No | ISO 8601 timestamp for when the event occurred | +| `tags` | list[string] | No | Tags for organizing and filtering this memory | +| `metadata` | object | No | Key-value metadata to attach (e.g., `{"source": "slack"}`) | +| `document_id` | string | No | Associate this memory with an existing document | **Example:** ```json @@ -127,7 +130,8 @@ Store information to long-term memory. "name": "retain", "arguments": { "content": "User prefers Python over JavaScript for backend development", - "context": "programming_preferences" + "context": "programming_preferences", + "tags": ["user:alice", "preferences"] } } ``` @@ -148,13 +152,20 @@ Search memories to provide personalized responses. |-----------|------|----------|-------------| | `query` | string | Yes | Natural language search query | | `max_tokens` | integer | No | Maximum tokens to return (default: 4096) | +| `budget` | string | No | Search thoroughness: `low`, `mid`, or `high` (default: `high`) | +| `types` | list[string] | No | Filter by fact type: `world`, `experience`, `opinion`. Defaults to all | +| `tags` | list[string] | No | Filter memories by tags | +| `tags_match` | string | No | Tag matching mode: `any` (default) or `all` | +| `query_timestamp` | string | No | ISO 8601 timestamp — recall as if asking at this point in time | **Example:** ```json { "name": "recall", "arguments": { - "query": "What are the user's programming language preferences?" + "query": "What are the user's programming language preferences?", + "tags": ["preferences"], + "budget": "high" } } ``` @@ -176,6 +187,10 @@ Generate thoughtful analysis by synthesizing stored memories with the bank's per | `query` | string | Yes | The question or topic to reflect on | | `context` | string | No | Optional context about why this reflection is needed | | `budget` | string | No | Search budget: `low`, `mid`, or `high` (default: `low`) | +| `max_tokens` | integer | No | Maximum tokens in the response (default: 4096) | +| `response_schema` | object | No | JSON Schema for structured output. When provided, the response includes a `structured_output` field | +| `tags` | list[string] | No | Filter memories by tags before reflecting | +| `tags_match` | string | No | Tag matching mode: `any` (default) or `all` | **Example:** ```json @@ -183,7 +198,8 @@ Generate thoughtful analysis by synthesizing stored memories with the bank's per "name": "reflect", "arguments": { "query": "Based on my past decisions, what architectural style do I prefer?", - "budget": "mid" + "budget": "mid", + "tags": ["architecture"] } } ``` @@ -206,6 +222,7 @@ Create a mental model — a living document that stays current with your memorie | `mental_model_id` | string | No | Custom ID (alphanumeric lowercase with hyphens). Auto-generated if not provided | | `tags` | list[string] | No | Tags for organizing and filtering models | | `max_tokens` | integer | No | Maximum tokens for model content (default: 2048) | +| `trigger_refresh_after_consolidation` | boolean | No | Auto-refresh this model after memory consolidation (default: `false`) | **Example:** ```json @@ -254,6 +271,7 @@ Update a mental model's metadata or settings. | `source_query` | string | No | New source query | | `tags` | list[string] | No | New tags | | `max_tokens` | integer | No | New max tokens | +| `trigger_refresh_after_consolidation` | boolean | No | Auto-refresh after consolidation. Only set when you want to change this setting | --- @@ -295,6 +313,186 @@ Create a new memory bank or retrieve an existing one. --- +### list_directives + +List all directives in a bank. Directives are instructions that guide how the memory system processes and responds to queries. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `tags` | list[string] | No | Filter directives by tags | +| `active_only` | boolean | No | Only return active directives (default: `true`) | + +--- + +### create_directive + +Create a new directive in a bank. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `name` | string | Yes | Human-readable name for the directive | +| `content` | string | Yes | The directive content/instruction | +| `priority` | integer | No | Priority level (higher = more important) | +| `is_active` | boolean | No | Whether the directive is active (default: `true`) | +| `tags` | list[string] | No | Tags for organizing directives | + +--- + +### delete_directive + +Delete a directive by ID. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `directive_id` | string | Yes | The ID of the directive to delete | + +--- + +### list_memories + +Browse stored memories with optional filtering and pagination. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `type` | string | No | Filter by fact type: `world`, `experience`, or `opinion` | +| `q` | string | No | Search query to filter memories | +| `limit` | integer | No | Maximum number of results (default: 100) | +| `offset` | integer | No | Number of results to skip for pagination (default: 0) | + +--- + +### get_memory + +Retrieve a specific memory by ID. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `memory_id` | string | Yes | The ID of the memory to retrieve | + +--- + +### delete_memory + +Permanently delete a specific memory. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `memory_id` | string | Yes | The ID of the memory to delete | + +--- + +### list_documents + +List documents that have been ingested into the memory bank. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `q` | string | No | Search query to filter documents | +| `limit` | integer | No | Maximum number of results (default: 100) | + +--- + +### get_document + +Retrieve a specific document by ID, including its metadata. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `document_id` | string | Yes | The ID of the document to retrieve | + +--- + +### delete_document + +Delete a document and all memories linked to it. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `document_id` | string | Yes | The ID of the document to delete | + +--- + +### list_operations + +List async operations (retain processing, mental model refresh, etc.) with optional status filtering. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `status` | string | No | Filter by status: `pending`, `running`, `completed`, `failed`, `cancelled` | +| `limit` | integer | No | Maximum number of results (default: 100) | + +--- + +### get_operation + +Get the status and details of an async operation. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `operation_id` | string | Yes | The ID of the operation to check | + +--- + +### cancel_operation + +Cancel a pending or running async operation. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `operation_id` | string | Yes | The ID of the operation to cancel | + +--- + +### list_tags + +List all unique tags used in a bank, optionally filtered by pattern. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `q` | string | No | Glob pattern to filter tags (e.g., `project:*`) | +| `limit` | integer | No | Maximum number of results (default: 100) | + +--- + +### get_bank + +Get information about a memory bank, including its name, mission, and disposition. + +--- + +### get_bank_stats (multi-bank mode only) + +Get statistics for a memory bank (node/link counts). + +--- + +### update_bank + +Update a memory bank's metadata. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `name` | string | No | New human-friendly name for the bank | +| `mission` | string | No | New mission describing who the agent is and what they're trying to accomplish | + +--- + +### delete_bank + +Permanently delete a memory bank and all its data (memories, documents, entities, mental models). + +--- + +### clear_memories + +Clear all memories from a bank without deleting the bank itself. Optionally filter by fact type to only clear specific kinds of memories. + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| `type` | string | No | Fact type to clear: `world`, `experience`, or `opinion`. If not specified, clears all | + +--- + ## Integration with AI Assistants The MCP server can be used with any MCP-compatible AI assistant. See the [Authentication](#authentication) section above for Claude Code and Claude Desktop configuration examples. diff --git a/skills/hindsight-docs/references/developer/observations.md b/skills/hindsight-docs/references/developer/observations.md index 9f218b16..281a0d85 100644 --- a/skills/hindsight-docs/references/developer/observations.md +++ b/skills/hindsight-docs/references/developer/observations.md @@ -139,32 +139,64 @@ This ensures responses stay accurate even as the underlying data changes. --- -## Mission-Oriented Consolidation +## Observations Mission -The bank's **mission** directly influences what knowledge gets consolidated into observations. When you set a mission on your memory bank, the consolidation engine focuses on extracting knowledge that serves that mission. +You can define exactly what this bank should synthesise by setting an **observations mission** (`observations_mission`). This replaces the built-in durable-knowledge rules with your own instructions, letting you control what shape observations take. -**Example:** - -```python -client.create_bank( - bank_id="support-agent", - mission="You're a customer support agent - keep track of " - "customer preferences, past issues, and communication styles." -) +``` +e.g. Observations are stable facts about people and projects. + Always include preferences, skills, and recurring patterns. + Ignore one-off events and ephemeral state. ``` -With this mission, the consolidation engine will: -- **Prioritize** customer preferences, issue patterns, and communication styles -- **Skip** ephemeral details that don't serve support goals -- **Synthesize** observations focused on helping customers +Leave it blank to use the server default — durable, specific facts that stay true over time (preferences, skills, relationships, recurring patterns), with ephemeral state filtered out. -Without a mission, the engine performs general-purpose consolidation. With a mission, it becomes focused and efficient — extracting only knowledge that matters for your use case. +**Examples:** -| Mission | Observations Focus | -|---------|-------------------| -| *Customer support agent* | Customer preferences, issue patterns, resolution history | -| *Code review assistant* | Coding patterns, team conventions, common mistakes | -| *Research assistant* | Topic expertise, source reliability, methodology preferences | +| `observations_mission` | What gets synthesised | +|------------------------|----------------------| +| *(unset — default)* | Durable facts: preferences, skills, relationships, recurring patterns | +| *"Observations are weekly summaries of sprint outcomes and blockers"* | Broad event summaries grouped by time period | +| *"Observations are stable facts about named individuals only"* | Person-centric knowledge, tied to specific people | +| *"Observations are recurring patterns in customer support interactions"* | Failure modes, common requests, pain points | + +Set `observations_mission` via the [bank config API](/developer/api/memory-banks#observations-configuration) or the [`HINDSIGHT_API_OBSERVATIONS_MISSION`](/developer/configuration#observations) environment variable. + +--- + +## Observation Lifecycle & Invalidation + +### When Memories Are Deleted + +Observations are derived from source memories. When source memories are removed, Hindsight automatically keeps observations consistent: + +| Action | Effect on observations | +|--------|----------------------| +| Delete a document | All observations derived from the document's memories are deleted | +| Delete individual memories (by type) | Observations sourced from those memories are deleted | +| Delete an entire bank | All observations are deleted along with everything else | + +After deletion, the **remaining source memories** that fed the affected observations have their consolidation state reset, so they will be re-consolidated on the next consolidation run and produce fresh observations. + +### Clearing Observations for a Specific Memory + +You can clear all observations derived from a single memory without deleting the memory itself. This is useful when you want to force re-synthesis of a memory's contribution to consolidated knowledge. + +Use the `DELETE /v1/default/banks/{bank_id}/memories/{memory_id}/observations` endpoint. This will: +1. Delete all observations that list the memory as a source +2. Reset `consolidated_at` on the memory itself and any other source memories that contributed to those observations +3. Trigger a consolidation job so fresh observations are produced automatically + +### Resetting All Observations + +To wipe all consolidated knowledge and start over: + +```python +# Clear all observations for a bank +client.clear_observations(bank_id="my-bank") +``` + +This resets the consolidation state for all source memories in the bank, so the next consolidation run will re-derive all observations from scratch. --- diff --git a/skills/hindsight-docs/references/developer/reflect.md b/skills/hindsight-docs/references/developer/reflect.md index 9fcb8f6c..1acbe04c 100644 --- a/skills/hindsight-docs/references/developer/reflect.md +++ b/skills/hindsight-docs/references/developer/reflect.md @@ -112,26 +112,27 @@ When you create a memory bank, you can configure its disposition using three tra ### Mission: Natural Language Identity -Beyond numeric traits, you can provide a natural language **mission** that describes the bank's identity: +Beyond numeric traits, you can provide a natural language **mission** that describes the bank's identity and reasoning context: ```python -client.create_bank( - bank_id="architect-bank", - mission="You're a senior software architect - keep track of system designs, " +client.create_bank(bank_id="architect-bank") +client.update_bank_config( + "architect-bank", + reflect_mission="You're a senior software architect - keep track of system designs, " "technology decisions, and architectural patterns. Prefer simplicity over cutting-edge.", - disposition={ - "skepticism": 4, # Questions new technologies - "literalism": 4, # Focuses on concrete specs - "empathy": 2 # Prioritizes technical facts - } + disposition_skepticism=4, # Questions new technologies + disposition_literalism=4, # Focuses on concrete specs + disposition_empathy=2, # Prioritizes technical facts ) ``` -The mission tells Hindsight what knowledge to prioritize and shapes how disposition traits are applied: -- "keep track of system designs" → focuses consolidation on architectural decisions -- "prefer simplicity over cutting-edge" + high skepticism → questions complex solutions -- Explicit guidance → consistent memory focus across conversations +The reflect mission frames how the agent reasons and responds: +- Provides identity context: who the agent is and what it cares about +- Shapes how disposition traits are applied in practice +- Keeps reasoning consistent across conversations +:::info Per-operation missions +The reflect mission only affects `reflect()`. To steer what gets extracted during `retain()`, use [`retain_mission`](/developer/api/memory-banks#retain-configuration). To control what gets synthesised into observations, use [`observations_mission`](/developer/api/memory-banks#observations-configuration). --- ## Disposition Shapes Reasoning diff --git a/skills/hindsight-docs/references/developer/retain.md b/skills/hindsight-docs/references/developer/retain.md index 87b14b85..fc85d5e0 100644 --- a/skills/hindsight-docs/references/developer/retain.md +++ b/skills/hindsight-docs/references/developer/retain.md @@ -176,6 +176,29 @@ All stored in your isolated **memory bank**, ready for `recall()` and `reflect() --- +## Steering Extraction with a Mission + +By default, `retain()` extracts all significant facts from the content. You can narrow this focus with a **retain mission** (`retain_mission`) — a plain-language description of what this bank should pay attention to. + +``` +e.g. Always include technical decisions, API design choices, and architectural trade-offs. + Ignore meeting logistics, greetings, and social exchanges. +``` + +The mission is injected into the extraction prompt alongside the built-in rules — it steers the LLM without replacing the extraction logic. It works with any extraction mode (`concise`, `verbose`, `custom`). + +For finer control, you can also change the **extraction mode**: + +| Mode | When to use | +|------|-------------| +| `concise` *(default)* | General-purpose — selective, fast | +| `verbose` | When you need richer facts with full context and relationships | +| `custom` | When you want to write your own extraction rules entirely | + +Set `retain_mission` and `retain_extraction_mode` via the [bank config API](/developer/api/memory-banks#retain-configuration) or the [`HINDSIGHT_API_RETAIN_MISSION`](/developer/configuration#retain) environment variable. + +--- + ## Observation Consolidation After `retain()` completes, Hindsight automatically triggers **observation consolidation** in the background. This process: diff --git a/skills/hindsight-docs/references/sdks/integrations/chat.md b/skills/hindsight-docs/references/sdks/integrations/chat.md new file mode 100644 index 00000000..8b024f52 --- /dev/null +++ b/skills/hindsight-docs/references/sdks/integrations/chat.md @@ -0,0 +1,163 @@ +--- +sidebar_position: 5 +--- + +# Vercel Chat SDK + +The `@vectorize-io/hindsight-chat` package gives your [Vercel Chat SDK](https://github.com/vercel/chat) bots persistent, per-user memory with a single handler wrapper. Works with Slack, Discord, Teams, Google Chat, GitHub, and Linear. + +## Installation + +```bash +npm install @vectorize-io/hindsight-chat +``` + +## Quick Start + +```typescript +import { Chat } from 'chat'; +import { HindsightClient } from '@vectorize-io/hindsight-client'; +import { withHindsightChat } from '@vectorize-io/hindsight-chat'; +import { streamText } from 'ai'; +import { openai } from '@ai-sdk/openai'; + +const chat = new Chat({ connectors: [/* your connectors */] }); +const hindsight = new HindsightClient({ apiKey: process.env.HINDSIGHT_API_KEY }); + +chat.onNewMention( + withHindsightChat( + { + client: hindsight, + bankId: (msg) => msg.author.userId, // per-user memory + }, + async (thread, message, ctx) => { + await thread.subscribe(); + + const result = await streamText({ + model: openai('gpt-4o'), + system: ctx.memoriesAsSystemPrompt(), + messages: [{ role: 'user', content: message.text }], + }); + + // Stream the response + const chunks: string[] = []; + for await (const chunk of result.textStream) { + chunks.push(chunk); + } + const fullResponse = chunks.join(''); + await thread.post(fullResponse); + + // Store the conversation in memory + await ctx.retain( + `User: ${message.text}\nAssistant: ${fullResponse}` + ); + } + ) +); +``` + +## Configuration + +### `withHindsightChat(options, handler)` + +Returns a standard Chat SDK handler `(thread, message) => Promise`. + +#### Options + +| Option | Type | Default | Description | +|--------|------|---------|-------------| +| `client` | `HindsightClient` | *required* | Hindsight client instance | +| `bankId` | `string \| (msg) => string` | *required* | Memory bank ID or resolver function | +| `recall.enabled` | `boolean` | `true` | Auto-recall memories before handler | +| `recall.budget` | `'low' \| 'mid' \| 'high'` | `'mid'` | Processing budget for recall | +| `recall.maxTokens` | `number` | API default | Max tokens for recall results | +| `recall.types` | `FactType[]` | all | Filter to specific fact types | +| `recall.includeEntities` | `boolean` | `true` | Include entity observations | +| `retain.enabled` | `boolean` | `false` | Auto-retain inbound messages | +| `retain.async` | `boolean` | `true` | Fire-and-forget retain | +| `retain.tags` | `string[]` | – | Tags for retained memories | +| `retain.metadata` | `Record` | – | Metadata for retained memories | + +### Context (`ctx`) + +The third argument passed to your handler: + +| Property/Method | Description | +|----------------|-------------| +| `ctx.bankId` | Resolved bank ID | +| `ctx.memories` | Array of recalled memories | +| `ctx.entities` | Entity observations (or null) | +| `ctx.memoriesAsSystemPrompt(options?)` | Format memories for LLM system prompt | +| `ctx.retain(content, options?)` | Store content in memory | +| `ctx.recall(query, options?)` | Search memories | +| `ctx.reflect(query, options?)` | Reason over memories | + +## Examples + +### Subscribed Message Handler + +```typescript +chat.onSubscribedMessage( + withHindsightChat( + { + client: hindsight, + bankId: (msg) => msg.author.userId, + recall: { budget: 'high', maxTokens: 1000 }, + }, + async (thread, message, ctx) => { + const result = await generateText({ + model: openai('gpt-4o'), + system: ctx.memoriesAsSystemPrompt(), + messages: [{ role: 'user', content: message.text }], + }); + await thread.post(result.text); + } + ) +); +``` + +### Auto-Retain Inbound Messages + +```typescript +chat.onNewMention( + withHindsightChat( + { + client: hindsight, + bankId: (msg) => msg.author.userId, + retain: { enabled: true, tags: ['slack', 'inbound'] }, + }, + async (thread, message, ctx) => { + // Inbound message is already being retained automatically + const result = await generateText({ + model: openai('gpt-4o'), + system: ctx.memoriesAsSystemPrompt(), + messages: [{ role: 'user', content: message.text }], + }); + await thread.post(result.text); + + // Retain the assistant response separately + await ctx.retain(`Assistant: ${result.text}`, { + tags: ['slack', 'outbound'], + }); + } + ) +); +``` + +### Static Bank ID (Shared Memory) + +```typescript +// All users share the same memory bank +chat.onNewMention( + withHindsightChat( + { client: hindsight, bankId: 'shared-team-memory' }, + async (thread, message, ctx) => { + // ... + } + ) +); +``` + +## Error Handling + +Memory failures never break your bot. Auto-recall and auto-retain errors are logged as warnings and the handler continues with empty memories. Manual `ctx.retain()`, `ctx.recall()`, and `ctx.reflect()` calls propagate errors normally so you can handle them as needed. diff --git a/skills/hindsight-docs/references/sdks/integrations/local-mcp.md b/skills/hindsight-docs/references/sdks/integrations/local-mcp.md index 8d3740a3..524b2b2f 100644 --- a/skills/hindsight-docs/references/sdks/integrations/local-mcp.md +++ b/skills/hindsight-docs/references/sdks/integrations/local-mcp.md @@ -66,22 +66,74 @@ claude mcp add --transport http hindsight http://localhost:8888/mcp/my-bank/ ## Available Tools -The local server exposes the full tool set: +The local server exposes the full tool set (29 tools in multi-bank mode, 26 in single-bank mode): + +**Core Memory** + +| Tool | Description | +|------|-------------| +| `retain` | Store information to long-term memory with optional tags, metadata, and document association | +| `recall` | Search memories with natural language, configurable budget, type filters, and tag filters | +| `reflect` | Synthesize memories into a reasoned answer with optional structured output | + +**Mental Models** | Tool | Description | |------|-------------| -| `retain` | Store information to long-term memory (fire-and-forget) | -| `recall` | Search memories with natural language | -| `reflect` | Synthesize memories into a reasoned answer | -| `list_banks` | List all memory banks | -| `create_bank` | Create or configure a memory bank | | `list_mental_models` | List pinned reflections for a bank | | `get_mental_model` | Get a specific mental model | -| `create_mental_model` | Create a new mental model | +| `create_mental_model` | Create a new mental model with optional auto-refresh trigger | | `update_mental_model` | Update a mental model's metadata | | `delete_mental_model` | Delete a mental model | | `refresh_mental_model` | Regenerate a mental model's content | +**Directives** + +| Tool | Description | +|------|-------------| +| `list_directives` | List directives that guide memory processing | +| `create_directive` | Create a new directive | +| `delete_directive` | Delete a directive | + +**Memory Browsing** + +| Tool | Description | +|------|-------------| +| `list_memories` | Browse memories with filtering and pagination | +| `get_memory` | Get a specific memory by ID | +| `delete_memory` | Delete a specific memory | + +**Documents** + +| Tool | Description | +|------|-------------| +| `list_documents` | List ingested documents | +| `get_document` | Get a specific document | +| `delete_document` | Delete a document and its linked memories | + +**Operations** + +| Tool | Description | +|------|-------------| +| `list_operations` | List async operations with status filtering | +| `get_operation` | Check operation status and progress | +| `cancel_operation` | Cancel a pending or running operation | + +**Tags & Bank Management** + +| Tool | Description | +|------|-------------| +| `list_tags` | List unique tags used in a bank | +| `get_bank` | Get bank profile (name, mission, disposition) | +| `get_bank_stats` | Get bank statistics (multi-bank only) | +| `update_bank` | Update bank name or mission | +| `delete_bank` | Delete an entire bank and all its data | +| `clear_memories` | Clear memories without deleting the bank | +| `list_banks` | List all memory banks (multi-bank only) | +| `create_bank` | Create or configure a memory bank (multi-bank only) | + +For detailed parameter documentation, see the [MCP Server reference](/developer/mcp-server#available-tools). + ## Environment Variables All standard [Hindsight configuration variables](/developer/configuration) are supported. Key ones for local use: