Commit graph

906 commits

Author SHA1 Message Date
Nicolò Boschi
fa0e63b088 fix(docker): copy uv.lock into build context to pin greenlet version
Without the lock file, uv sync resolves fresh and picks up greenlet
3.4.0 which lacks arm64 wheels for manylinux_2_41, breaking the
multi-arch Docker build.
2026-04-08 18:21:28 +02:00
Nicolò Boschi
27cb7e43e0 Release v0.5.0
- Update version to 0.5.0 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- Helm chart
- Create documentation version-0.5
2026-04-08 17:56:47 +02:00
Ben
9e23e83abf
Add Codex persistent memory blog post (#812)
* Add Codex persistent memory blog post
2026-04-08 10:44:27 -04:00
Nicolò Boschi
bdf93f0660
fix: exclude local-llm from [all] extra, add as opt-in to hindsight-all (#936)
* fix: exclude local-llm from [all] extra to avoid heavy llama-cpp-python dep

local-llm (llama-cpp-python) requires C++ compilation and is only needed
for the built-in llamacpp provider. Keep it as a separate opt-in:
pip install 'hindsight-api-slim[local-llm]'

* feat: add local-llm optional extra to hindsight-all

Allows: pip install 'hindsight-all[local-llm]' to get built-in llamacpp support.

* chore: regenerate uv.lock from workspace root
2026-04-08 16:06:26 +02:00
Aldous
b0e8ac0f4d
feat(openclaw): add configurable retain tags (#937)
Co-authored-by: Aldous the Orchestrator <Aldoustheorchestrator@users.noreply.github.com>
2026-04-08 15:52:32 +02:00
Nicolò Boschi
f74b577e02
feat: add built-in llama.cpp LLM provider for local inference (#933)
* feat: add built-in llama.cpp LLM provider for fully local inference

Add `llamacpp` as a new LLM provider that manages a llama-cpp-python server
subprocess. Auto-downloads Gemma 4 E2B Q4_K_M (~3.5 GB) on first use and
runs inference locally via Metal/CUDA with no external services needed.

- New provider: `HINDSIGHT_API_LLM_PROVIDER=llamacpp`
- Singleton server shared across retain/reflect/consolidation
- Configurable: model path, GPU layers, context size, grammar enforcement
- User-extensible via `HINDSIGHT_API_LLAMACPP_EXTRA_ARGS`
- Flash attention + prompt caching enabled by default
- LLM provider cleanup on shutdown (stops subprocess)
- hindsight-embed: `--ui` flag on `daemon start`, removed FORCE_CPU on macOS
- Docs: configuration.md, models.mdx, providers grid updated

* chore: regenerate docs skill and update lockfile for local-llm dep
2026-04-08 15:22:10 +02:00
Nicolò Boschi
3c633e5e16
feat: add retain update_mode='append' for document content concatenation (#932)
* feat: add update_mode='append' for retain to concatenate content to existing documents

When retaining with update_mode='append' and a document_id that already exists,
the new content is appended to the existing document text and the full document
is reprocessed. Delta retain automatically skips unchanged chunks, so only the
new content triggers LLM extraction.

- Add update_mode field to MemoryItem (API), RetainContentDict (internal), MCP tools
- Validate that update_mode='append' requires a document_id
- Fetch existing document content and prepend before processing in orchestrator
- Update Python, TypeScript, Go generated clients and top-level client wrappers
- Add tests for append, multiple appends, no-existing-doc, validation, and default replace

* fix: add update_mode field to Rust CLI and client MemoryItem initializers

* chore: regenerate docs skill references for update_mode
2026-04-08 14:39:16 +02:00
Nicolò Boschi
cf0537ba7e
chore: drop hindsight-hermes integration (#931)
* chore: drop hindsight-hermes integration in favor of native Hermes memory provider

Hermes Agent now ships with a native Hindsight memory provider (NousResearch/hermes-agent#5094),
making our pip-installable hindsight-hermes package redundant.

Removes:
- hindsight-integrations/hermes/ (source, tests, config)
- CI job, release script entry, changelog generator references
- Cookbook page and pip package changelog (referenced deleted code)

Keeps:
- Integration docs (updated by #881 for native provider)
- Blog posts (historical, already have deprecation notices)
- Sidebar/banner entries (still valid for native integration)

* fix(docs): remove broken cookbook link to deleted hermes-memory page
2026-04-08 11:59:35 +02:00
Nicolò Boschi
e5944b63e7
feat: add OpenRouter support for LLM, embeddings, and reranking (#930)
* docs: add best practice for filtering recall by memory shape (#856)

Add guidance on using entity labels with `tag: true` to deterministically
filter recall results when a bank contains different memory shapes
(e.g., concise rules vs. detailed procedures).

* feat: add OpenRouter support for LLM, embeddings, and reranking

OpenRouter is OpenAI-compatible for chat/embeddings and Cohere-compatible
for reranking, so no new provider classes are needed.

- LLM: added as OpenAICompatibleLLM provider (default model: qwen/qwen3.5-9b)
- Embeddings: reuses OpenAIEmbeddings with OpenRouter base URL (default: perplexity/pplx-embed-v1-0.6b)
- Reranker: reuses CohereCrossEncoder with OpenRouter rerank endpoint (default: cohere/rerank-v3.5)
- API key fallback chain: dedicated key → shared OPENROUTER_API_KEY → LLM_API_KEY

* chore: regenerate docs skill references and fix formatting
2026-04-08 11:24:21 +02:00
Nicolò Boschi
37348c859e
feat: include occurred_end and mentioned_at in think-prompt fact serialization (#929)
Extend format_facts_for_prompt() to include occurred_end and mentioned_at
temporal fields (when non-null), matching the MemoryFact model. Also add
RecallResponse.to_prompt_string() to Python and TypeScript client SDKs so
users can serialize recall results (with chunks and entity summaries) into
LLM-ready prompt strings.

Closes #924
2026-04-08 10:33:14 +02:00
Nicolò Boschi
cece2c903c
fix: make LiteLLM SDK embeddings encoding_format configurable (#928)
* fix: make LiteLLM SDK embeddings encoding_format configurable (#925)

The hardcoded encoding_format='float' breaks providers like Voyage AI
(only accepts 'base64') and Gemini (doesn't support the parameter at all).

Add HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_ENCODING_FORMAT config option
that defaults to 'float' for backwards compatibility. Set to empty string
to omit the parameter for incompatible providers.

* chore: regenerate docs skill after configuration change
2026-04-08 09:41:11 +02:00
Derek Bouius
d7c73f4342
security: bump lodash, lodash-es, defu in root lockfile (#915)
* security: bump lodash, lodash-es, and defu in root lockfile

Fixes Dependabot alerts in the root npm workspace lockfile:
- GHSA-r5fr-rjxr-66jc (high) lodash <4.18.1     (alert #338)
- GHSA-r5fr-rjxr-66jc (high) lodash-es <4.18.1  (alert #335)
- GHSA-737v-mqg7-c878 (high) defu <6.1.7        (alert #343)

defu (6.1.4 -> 6.1.7) and lodash (4.17.23 -> 4.18.1) were bumped via
targeted `npm update`. lodash-es was pinned exactly to 4.17.23 by
@chevrotain packages (transitive dep of mermaid in hindsight-docs),
so a `lodash-es` override (>=4.18.1) is added to the root package.json
to force resolution to the patched 4.18.1.

Verified: `npm ci` succeeds with 0 vulnerabilities. Mermaid/chevrotain
consumers all dedupe to lodash-es 4.18.1. lodash-es 4.x is semver-
compatible.

* chore: regenerate hindsight-docs skill

Picks up FAQ and best-practice sections added in #905 that were not
regenerated at merge time, so that `verify-generated-files` passes
for this branch.
2026-04-08 09:11:29 +02:00
Derek Bouius
3b9d2db091
security: bump vite across integrations (high CVE fix) (#913)
* security: bump vite across integrations to patched versions

Fixes Dependabot alerts for vite transitive dev dependency:
- GHSA-v2wj-q39q-566r (high): server.fs.deny bypass with queries
- GHSA-p9ff-h696-f583 (high): related vite server vulnerability

Adds a `vite` entry to the npm `overrides` in each integration's
package.json to force the patched version (>=8.0.5). To make this
possible in ai-sdk, chat, and openclaw — which pinned vitest ^4.0.18
whose vite peer is `^6.0.0 || ^7.0.0` — the minor-compatible bump
vitest ^4.0.18 -> ^4.1.2 is also included. vitest 4.1.x supports
vite 8.x (peer: ^6 || ^7 || ^8), so all six integrations converge on
vite 8.x consistently.

paperclip had no overrides block; one was added.

Verified locally: `npm ci && npx vitest run` passes in all six
integrations (ai-sdk 23, chat 28, openclaw 66, opencode 89, paperclip 27,
nemoclaw 36 tests).

* chore: regenerate hindsight-docs skill

Picks up FAQ and best-practice sections added in #905 that were not
regenerated at merge time, so that `verify-generated-files` passes
for this branch.
2026-04-08 09:11:21 +02:00
eason
9790d904e0
fix: clamp out-of-range content_index in _map_results_to_contents (#908)
Some LLM providers (e.g. Anthropic Haiku) return 1-indexed
content_index values. When only one content item is provided,
this causes KeyError: 1 since the dict only has key 0.

Clamp content_index to the valid range instead of crashing.

Fixes #873

Co-authored-by: easonysliu <easonysliu@tencent.com>
2026-04-08 09:10:59 +02:00
Ben
2463efd0f2
Update author name from Mike to Michael (#917) 2026-04-07 13:42:29 -04:00
Ben
6674ee4706
Remove hindsight-cloud tag from guest post (#916) 2026-04-07 13:22:48 -04:00
Nicolò Boschi
57f154454d
fix(recall): cap entity fanout in graph expansion (#911)
* fix(recall): cap entity fanout in graph expansion to prevent slow queries

On large banks, the entity co-occurrence self-join in _expand_combined()
produces massive intermediate row counts when seeds reference high-fanout
entities (e.g. an entity with 25K+ mentions). This causes recall latency
to degrade significantly.

Changes:
- Replace unbounded entity self-join with LATERAL per-entity cap
  (graph_per_entity_limit, default 200), reducing intermediate rows
  from potentially millions to at most num_entities * 200
- Add ORDER BY unit_id DESC in LATERAL subquery for deterministic
  recency-biased sampling (rides the PK index, no extra sort)
- Add timeout fallback (graph_expansion_timeout, default 10s) that
  drops entity expansion and falls back to semantic+causal only
- Add composite index (entity_id, unit_id) on unit_entities for
  index-only scans in the LATERAL subquery
- Merge 3 unmerged migration heads into one
- Fix recall_perf.py dotenv override issue

Unlike the approach in #895, this does NOT filter out hub entities
entirely — all entities are kept but capped equally, preserving
retrieval quality for queries about frequently-mentioned entities.

Benchmarked on a 67K-unit bank (top entity = 25K mentions):
- retrieval_graph: 0.337s → 0.055s (84% faster)
- end-to-end recall: 0.912s → 0.519s (43% faster)

* fix(tests): fix broken test_combined_scoring and test_reranking_proof_count

- test_combined_scoring: replace MagicMock(spec=RetrievalResult) with real
  dataclass instances — MagicMock attributes returned nested mocks that
  failed on >= comparisons with int
- test_reranking_proof_count: remove deleted `embedding` param from
  RetrievalResult constructor, use None for occurred_start/end to get
  neutral recency (datetime.now gave recency=1.0 which boosted scores)

* refactor: rename config to link_expansion_ prefix, fix observation fanout

- Rename GRAPH_PER_ENTITY_LIMIT → LINK_EXPANSION_PER_ENTITY_LIMIT and
  GRAPH_EXPANSION_TIMEOUT → LINK_EXPANSION_TIMEOUT to follow the
  convention that these are specific to the link_expansion graph retriever
- Apply the same LATERAL per-entity cap to _expand_observations(), which
  had the same unbounded self-join through unit_entities

* style: fix formatting in config.py
2026-04-07 18:59:50 +02:00
Ben
4028dd91f8
blog: One Memory for Every AI Tool I Use (#914)
* blog: One Memory for Every AI Tool I Use (guest post)
2026-04-07 12:57:48 -04:00
Aldous
0e81d1a25e
feat(openclaw): support bankId for static banks (#910)
* feat(openclaw): support exact static bank ids

* test(openclaw): use generic static bank id example

* feat(openclaw): support bankId static bank configuration

---------

Co-authored-by: Aldous the Orchestrator <Aldoustheorchestrator@users.noreply.github.com>
2026-04-07 17:13:20 +02:00
Derek Bouius
8a2388a48f
security: bump litellm to >=1.83.0 (#912)
Fixes Dependabot alerts:
- GHSA-jjhc-v7c2-5hh6 (critical): Authentication bypass via OIDC userinfo
  cache key collision (CVE-2026-35030)
- GHSA-53mr-6c8q-9789 (high): related litellm vulnerability

Updates both hindsight-api-slim and hindsight-integrations/litellm to
require litellm >=1.83.0. The previous upper cap (<=1.82.6) was set due
to the 1.82.7/1.82.8 supply chain compromise, which has since been yanked
from PyPI; 1.83.0 was published from the new secure CI/CD v2 pipeline
and is safe.

The uv.lock diffs are large because the current uv version (0.9.11)
upgrades the lockfile format (adds revision=3 and upload-time fields);
only litellm itself changes version (1.81.10/1.80.10 -> 1.83.0).

All 68 tests in hindsight-integrations/litellm pass against 1.83.0.
2026-04-07 16:54:24 +02:00
Nicolò Boschi
48185a4bee
fix(mcp): validate UUID inputs and add sync_retain tool (#906)
* fix(mcp): validate UUID inputs at engine level and add sync_retain tool (#888)

- Add UUID validation in memory_engine for get_memory_unit, delete_memory_unit,
  get_mental_model, delete_mental_model, get_mental_model_history (raises ValueError)
- Catch ValueError → 400 in HTTP route handlers
- Add sync_retain MCP tool that calls retain_batch_async directly for immediate
  availability (no polling needed)
- Register sync_retain in _ALL_TOOLS, _SINGLE_BANK_TOOLS, UI MCP_TOOL_GROUPS
- Add code-review check for MCP tool registration completeness

* fix: remove UUID validation for mental model IDs (column is TEXT, not UUID)

Mental model IDs are TEXT columns that accept arbitrary string IDs
(e.g., 'team-communication-preferences'). UUID validation was incorrectly
added to get_mental_model, delete_mental_model, and get_mental_model_history.
2026-04-07 11:59:59 +02:00
Nicolò Boschi
7e23f8e149
fix(config): validate entity_labels structure on PATCH (#902)
* test: add regression tests for #874 and #894

Add tests for None event_date in fact extraction (AttributeError fix)
and for _register_profile skipping .env overwrite with short config keys.

* fix(config): validate entity_labels structure on PATCH (#891)

Config PATCH accepted bare strings in entity_labels values without
validation, causing silent failures at retain time. Now validates
via parse_entity_labels() before writing to DB, and fixes the
BankTemplateConfig type from list[str] to list[dict[str, Any]].

* fix(scripts): handle Python client generator README crash gracefully

The openapi-generator sometimes crashes writing README_onlypackage.mustache.
Allow the failure with || true since all API/model files are generated
before that step, and add a verification check for api_client.py.

* chore: regenerate docs skill openapi.json
2026-04-07 11:58:02 +02:00
Nicolò Boschi
f659bb17c4
docs: add best practice for filtering recall by memory shape (#856) (#905)
Add guidance on using entity labels with `tag: true` to deterministically
filter recall results when a bank contains different memory shapes
(e.g., concise rules vs. detailed procedures).
2026-04-07 10:41:32 +02:00
Nicolò Boschi
f31f82627c
fix: add paperclip and opencode to changelog generator (#903)
* fix: add paperclip and opencode to changelog valid integrations

* fix: add paperclip and opencode package names to changelog generator

* release(paperclip): v0.1.1
2026-04-07 10:25:53 +02:00
DK09876
e1c6220f0e
feat: add OpenCode persistent memory plugin (#853)
* feat: add OpenCode persistent memory plugin

Add hindsight-opencode integration with:
- Three custom tools: hindsight_retain, hindsight_recall, hindsight_reflect
- Auto-retain on session.idle with document_id deduplication
- Memory injection on session start via system transform hook
- Memory preservation during context window compaction
- Sliding window retain with retainOverlapTurns support
- 4-level config hierarchy (defaults, user file, plugin options, env vars)
- Dynamic bank ID derivation (agent, project, channel, user dimensions)
- CI job, release script entry, docs page

79 tests across 6 test files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review findings for opencode integration

1. Pre-compaction retain now uses shared retainSession() helper,
   respecting retainMode, documentId, and session_id metadata
   consistently with idle-retain (was bypassing retention policy).

2. System transform recall is only consumed after successful injection.
   If Hindsight is briefly unavailable, the plugin retries on the next
   LLM call instead of permanently skipping recall for the session.

3. Config validation for retainMode and recallBudget — typos like
   "full_session" or "maximum" now log a warning and fall back to
   the default instead of silently changing retention semantics.

85 tests (6 new covering compaction documentId, recall retry, and
config validation).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: docs/tools findings from second review round

1. Remove "session" from supported dynamic bank fields in docs —
   the implementation can't vary bank ID per session since it's
   derived once at plugin startup.

2. Explicit tools (retain, reflect) now call ensureBankMission()
   before API calls, so bankMission/retainMission are applied even
   when the agent uses tools exclusively without triggering hooks.

3. Added tests for mission setup via tools path.

88 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: recall retry semantics and README bank scoping clarity

1. recallForContext now returns { context, ok } to distinguish
   "no results" (ok=true) from "API error" (ok=false). System
   transform consumes the session on ok=true even with 0 results,
   so empty banks don't cause repeated queries. Only transient API
   failures preserve retry.

2. README clarifies that channel/user bank dimensions are process-
   scoped (set via env vars before launch), not per-session dynamic
   within a running OpenCode process.

89 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: review fixes for opencode integration

- Rename CI job from build-opencode-integration to test-opencode-integration
  to match naming convention for integrations that run tests
- Fix tsconfig module resolution to Node16 (consistent with other integrations)
- Extract shared makeConfig test helper to avoid duplication across 3 test files

* fix: remove unused PluginState import from tools.ts

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>
2026-04-07 10:11:57 +02:00
Nicolò Boschi
66cbdda3cb
test: add regression tests for #874 and #894 (#901)
Add tests for None event_date in fact extraction (AttributeError fix)
and for _register_profile skipping .env overwrite with short config keys.
2026-04-07 09:43:25 +02:00
Nicolò Boschi
cf4bd598b4
fix: make bank_id metric label opt-in to prevent OTel memory leak (#898)
* fix: make bank_id metric label opt-in to prevent OTel memory leak

bank_id as an OTel metric attribute creates unbounded histogram growth
since each unique bank_id produces never-evicted time series. Default
to excluding it; opt in with HINDSIGHT_API_METRICS_INCLUDE_BANK_ID=true
for deployments with few banks.

Closes #850

* refactor: use config.py for metrics_include_bank_id setting

Move HINDSIGHT_API_METRICS_INCLUDE_BANK_ID from direct os.getenv in
metrics.py to the standard HindsightConfig path. Add configuration
documentation.
2026-04-07 09:42:59 +02:00
Nicolò Boschi
443c94c827
fix(mcp): auto-coerce string-encoded JSON in tool arguments (#849) (#899)
LLM agents frequently serialize list/dict tool arguments as JSON strings
instead of native types (e.g., tags='["a","b"]' instead of tags=["a","b"]),
causing Pydantic validation failures. This extends _make_tools_tolerant to
detect array/object parameters from the JSON Schema and auto-coerce string
values via json.loads before validation.

Also fixes _make_tools_tolerant compatibility with FastMCP 3.x by adding
a _get_mcp_tools helper that supports both 2.x and 3.x internal APIs.
2026-04-07 09:33:12 +02:00
Abdulkadirklc
26794aab09
feat(recall): add proof_count boost to combined scoring (#821)
* feat(recall): add proof_count boost to combined scoring

Observations with more supporting evidence now rank slightly higher
in recall results. proof_count is threaded through the retrieval
pipeline and applied as a multiplicative boost in reranking:

- types.py: add proof_count field to RetrievalResult
- retrieval.py: include proof_count in SELECT columns
- reranking.py: add log1p-normalized proof_count boost (alpha=0.1)

The boost uses the same multiplicative pattern as recency and temporal
signals. proof_count=1 is neutral, proof_count=50 gives ~+5% boost.
Non-observation fact types are unaffected (neutral 0.5).

* fix(retrieval): Apply proof_count boost to graph and temporal retrieval, normalize scaling

* fix(retrieval): correct proof_norm math to zero-center at count 1

* fix(retrieval): Apply proof_count boost to link_expansion retrieval

* fix: remove BFS zombie, clamp proof_norm to [0,1], fix test comment (log1p->math.log)
2026-04-07 09:32:44 +02:00
Nicolò Boschi
7863ffeb49
fix(paperclip): address review fixes for paperclip integration (#900)
- Add CI job for paperclip integration tests with change detection
- Add paperclip to valid release integrations
- Validate hindsightApiUrl is set in loadConfig()
- Log warnings on recall/retain failures instead of silently swallowing
- Remove hardcoded timeout from reflect call
- Fix tsconfig module resolution to Node16
- Update tests to pass required hindsightApiUrl
2026-04-07 09:32:24 +02:00
Octopus
9e2890ba81
fix(embed): skip profile .env overwrite when config has no HINDSIGHT_API_* keys (#896)
When the daemon is already running, ensure_running() calls _register_profile()
with a config dict using short keys (llm_api_key, llm_provider, etc.) that do
not match the HINDSIGHT_API_* prefix filter. This caused api_config to always
be empty, and create_profile() would overwrite the existing .env with an empty
file on every CLI command.

Add an early return guard so _register_profile() skips the create_profile()
call when api_config is empty, preserving any existing profile configuration.

Fixes #894
2026-04-07 09:28:53 +02:00
Chris Bartholomew
e0e65c44f6
fix(query_analyzer): handle dateparser internal crashes gracefully (#893)
DateparserQueryAnalyzer.analyze() called dateparser.search.search_dates()
without any error handling, so internal bugs in the third-party library
propagated all the way up the search/consolidation pipeline and failed
the calling task.

Observed traceback:

  File ".../engine/query_analyzer.py", line 140, in analyze
    results = self._search_dates(query, settings=settings)
  File ".../dateparser/search/search.py", line 294, in search_dates
    "Dates": self.search.search_parse(...)
  File ".../dateparser/search/search.py", line 168, in search_parse
    translated, original = self.search(shortname, text, settings)
  File ".../dateparser/languages/locale.py", line 224, in translate_search
    [original_tokens[i], original_tokens[i + 1]],
  IndexError: list index out of range

Wrap the call in a try/except so any parser failure is treated as
"no temporal constraint found" — the caller can then fall back to
non-temporal retrieval instead of erroring out the whole task. The
failure is logged at WARNING level so we still notice it.

Add a regression test that monkey-patches _search_dates to raise an
IndexError and asserts the analyzer returns an empty constraint and
emits a warning log.
2026-04-07 09:26:27 +02:00
dependabot[bot]
6881f63781
chore(deps): bump actions/github-script from 7 to 8 (#879)
Bumps [actions/github-script](https://github.com/actions/github-script) from 7 to 8.
- [Release notes](https://github.com/actions/github-script/releases)
- [Commits](https://github.com/actions/github-script/compare/v7...v8)

---
updated-dependencies:
- dependency-name: actions/github-script
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-07 09:25:55 +02:00
Daniyar
6cb309f72b
Fix AttributeError when event_date is None in fact_extraction (#875)
* Fix AttributeError when event_date is None in fact_extraction

`_extract_facts_from_chunk` crashes with `'NoneType' object has no
attribute 'isoformat'` when retaining documents without a timestamp.

Two locations fixed:
- Line 1058: debug log called `event_date.isoformat()` without a None
  check
- Line 921: `parse_datetime_flexible()` can return None, so re-check
  before calling `.strftime()` / `.isoformat()`

Fixes #874

* Revert unnecessary None guard on line 921

The original `if event_date is not None:` already guards that block.
Only line 1058 needed the fix.
2026-04-07 09:22:07 +02:00
shun yi
f9fe6953a3
fix: Windows compatibility for hindsight-embed (#867)
- Add cross-platform file locking support
- Use fcntl on Unix-like systems, msvcrt on Windows
- Add detailed documentation explaining why we don't use external libraries
- Fixes issue where module couldn't be imported on Windows due to missing fcntl

Co-authored-by: yishun.eason <yishun.eason@bytedance.com>
2026-04-07 09:15:59 +02:00
Volodymyr Prypeshniuk
07de798c3b
feat(google): add support for google embeddings and reranker (#863)
* Add support for google embeddings gemini/vertex and google reranker via vertex search api

* Add reference docs
2026-04-07 09:15:31 +02:00
Byeonghoon Yoo
cefa75545a
feat(helm): add persistent volume for local model cache (#861)
* feat(helm): add persistent volume for local model cache

When using local reranker (e.g., BAAI/bge-reranker-v2-m3) or local
embedding models, the models are downloaded to /home/hindsight/.cache
on every pod restart, causing slow startup and unnecessary bandwidth.

Add optional persistent volume support:
- api: PVC mounted at /home/hindsight/.cache
- worker: volumeClaimTemplate (StatefulSet) at same path

Disabled by default. Enable via:
  api.persistence.modelCache.enabled: true
  worker.persistence.modelCache.enabled: true

Closes #860

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(helm): add extraVolumes and extraVolumeMounts for api and worker

Allow users to mount arbitrary volumes (configMaps, secrets, emptyDir,
etc.) into api and worker pods via values, following common helm chart
library conventions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 09:14:09 +02:00
Octopus
cd99eef4c5
fix: use max_tokens for OpenAI-compatible endpoints with custom base URL (#858)
Mistral (and several other providers) reject 'max_completion_tokens' with a 422
because they haven't adopted the newer OpenAI parameter name. When the openai
provider is configured with a custom base_url (e.g. Mistral, Together AI),
fall back to the widely-supported 'max_tokens' parameter.

Native OpenAI (no custom base_url) and Groq still use 'max_completion_tokens'.

Fixes #852
2026-04-07 09:13:08 +02:00
Ben
cd4b3e96e2
blog: Persistent Memory for AutoGen Agents with Hindsight (#883)
* Add AutoGen persistent memory blog post
2026-04-06 14:59:43 -04:00
Ben
e02e7ad3d4
blog: Hindsight is now a native memory provider in Hermes Agent (#882)
* Add Hermes native memory provider blog post
2026-04-06 10:55:48 -04:00
Ben
98fee1e380
docs(hermes): update integration docs for plugin overhaul (hermes-agent#5094) (#881)
* docs(hermes): update integration docs for hermes-agent plugin overhaul
2026-04-06 10:54:59 -04:00
Nicolò Boschi
906b740dd7 fix(docs): add missing SEO frontmatter to paperclip integration 2026-04-02 17:37:02 +02:00
Nicolò Boschi
7990381f6a
fix(ci): resolve all CI failures (#847)
* fix(ci): resolve all CI failures — unversioned integrations, test retries

- Move integration docs to separate unversioned docs plugin (docs-integrations/)
  so new integrations don't need to be duplicated across versioned_docs
- Remove integration pages from versioned_docs (v0.3, v0.4) — sidebar
  entries now use links instead of doc refs
- Add missing title/description SEO frontmatter to autogen.md
- Add retry logic (2 attempts) to test-doc-examples.sh for transient
  LLM timeouts
- Add pytest-rerunfailures to test-api with --reruns 2 for flaky
  Gemini-dependent integration tests

* ci: retrigger

* fix: graph entity inheritance, SyncTaskBackend error propagation, fact_type test regressions

- Fix observation entity inheritance in get_graph_data: the unit_entities
  query only fetched entities for visible observation IDs, not their source
  memory IDs, so the inheritance loop always found an empty entity_map
- Remove error swallowing in SyncTaskBackend._execute_task so test failures
  surface instead of being silently logged
- Wrap remaining consolidation submission call sites with try/except since
  consolidation is non-critical for those operations
- Fix test_sync_backend test to expect errors to propagate
- Remove fact_type=["world"] filter from test_document_upsert_behavior and
  test_mentioned_at_from_context_string (same PR #848 regression)
- Remove flaky marker from consolidation test (now deterministic)
2026-04-02 17:17:42 +02:00
Ben
045e8910d1
Blog: Hindsight Is #1 on BEAM — the Benchmark That Tests Memory at 10M Tokens (#851)
* Add BEAM SOTA blog post
2026-04-02 11:02:16 -04:00
Ben
81441ee9af
feat(paperclip): add hindsight-paperclip TypeScript integration (#773)
* feat(paperclip): add hindsight-paperclip TypeScript integration

Adds long-term memory for Paperclip AI agents via a lightweight
TypeScript/Node.js npm package with no runtime dependencies.

- recall() / retain() functions for heartbeat lifecycle hooks
- createMemoryMiddleware() for Express HTTP adapter agents
- Bank ID strategy: paperclip::{companyId}::{agentId} (configurable)
- Skill file for agents to call Hindsight REST API directly
- 27 unit tests covering bank derivation, recall, and retain
- Docs page at sdks/integrations/paperclip

* Remove skills file from paperclip integration

* Rename package to @vectorize-io/hindsight-paperclip
2026-04-02 14:26:45 +02:00
Nicolò Boschi
30a319a6ab
feat: bank template import/export with Template Hub (#819)
* feat(api): add bank template import/export endpoints

Add POST /banks/{bank_id}/import and GET /banks/{bank_id}/export
endpoints for declarative bank setup via JSON manifests.

A template manifest (version 1) can include bank config overrides
and mental model definitions. Import creates or updates mental
models matched by id, applies config as per-bank overrides, and
returns async operation IDs for content generation.

Export dumps a bank's explicit overrides and mental models as a
manifest that can be re-imported into another bank.

Includes control plane UI: bank creation dialog now accepts an
optional template JSON to pre-configure the bank on creation.

* docs: add Template Gallery page and bank templates reference

- Template Gallery (/templates) with search, category filter, manifest
  preview modal with copy-to-clipboard
- 5 starter templates: Customer Support, Research Assistant, Personal
  Journal, Code Review Buddy, Meeting Notes
- Bank Templates API reference doc (developer/api/bank-templates)
- Sidebar entry under API section

* docs: add Template Gallery links to navbar and sidebar

- Top navbar: "Templates" link between Integrations and Changelog
- Sidebar: "Template Gallery" in Resources section

* fix(docs): remove emoji icons, autofocus search, fix placeholder in template gallery

* docs: rename to Bank Templates, move to Resources sidebar only

* docs: add Bank Templates to Resources navbar dropdown

* feat(api): add directives to bank template import/export

- Add BankTemplateDirective model with name, content, priority, is_active, tags
- Import creates/updates directives matched by name
- Export includes all directives (active and inactive)
- Validation: duplicate names rejected, empty name/content caught
- Tests: 24 tests covering directives create/update, existing vs new
  bank import, validation, export with directives, full round-trip

* docs: add directives to bank templates docs and sample templates

* feat(api): add JSON Schema endpoint for bank template validation

- GET /v1/default/bank-template-schema returns the JSON Schema
  auto-generated from the Pydantic BankTemplateManifest model
- Static schema file at docs/static/bank-template-schema.json
- Docs updated with schema endpoint, static file link, and
  validation examples (Python jsonschema, Node ajv-cli)

* feat(api): live schema validation on import, fix schema endpoint path

- Move schema endpoint to /v1/bank-template-schema (system-level, not per-bank)
- Import endpoint now accepts raw JSON and validates with Pydantic manually,
  returning clean 400 errors instead of raw 422s for all validation failures
- All validation (schema + semantic) returns consistent 400 with detailed messages

* docs: add interactive JSON Schema viewer to Bank Templates page

Renders the Pydantic-generated schema as a collapsible property tree
with types, required badges, defaults, and descriptions. The schema
is imported from the static bank-template-schema.json file.

* ui: add template toggle switch and browse link to bank creation dialog

- Replace always-visible textarea with a switch toggle ("Import from template")
- Textarea only shows when switch is on, keeping the dialog clean by default
- Add "Browse templates" link pointing to hindsight.vectorize.io/templates
- Reset template state when switch is toggled off or dialog is cancelled

* ui: add empty state with Add Document CTA to data view

When a bank has 0 memories, the data view (all tabs: constellation,
graph, table, timeline) shows a centered empty state with a CTA
button that opens the Add Document dialog.

* docs: replace templates with Conversation and Coding Agent

Remove generic placeholder templates. Add two practical templates
based on actual integration patterns:

- Conversation: for chat agents (LiteLLM, LangGraph, Pydantic AI,
  Vercel AI SDK). Tracks user preferences, open threads.
- Coding Agent: for Claude Code/Codex. Tracks technical decisions,
  project context, developer preferences. High literalism.

* docs: rename gallery to Bank Templates Hub, keep API doc as Bank Templates

* docs: register layout-template and file-json icons in navbar and sidebar

* docs: register layout-template icon in DefaultNavbarItem for dropdown items

* docs: show integration icons on template cards

Templates now have an optional `integrations` field referencing
integration IDs from integrations.json. Icons are resolved at render
time and shown in the card header next to the category badge.

* docs: add Personal Assistant template for OpenClaw, Hermes, NemoClaw

* feat: add Export Template to bank actions + map all integrations to templates

- Add "Export Template" to the bank Actions dropdown — exports config,
  mental models, and directives as JSON, copies to clipboard
- Add export API route and client method
- Map remaining integrations to templates: CrewAI, AG2, Agno, Strands,
  LlamaIndex, local-mcp, skills → Conversation; hindclaw → Personal Assistant

* feat: add --template flag to LoCoMo benchmark + remove schema from Hub

- LoCoMo benchmark accepts --template <path> to apply a bank template
  manifest (config, mental models, directives) before ingestion
- Template is applied per-bank in both single-phase and two-phase modes
- BenchmarkRunner.apply_template() reuses the same engine methods as
  the /import API endpoint
- Remove Manifest Schema section from Bank Templates Hub page
  (schema stays in the API reference doc)

* refactor: remove description field from bank template manifest

* docs: remove tags, fact_types, and directives from starter templates

* docs: remove reflect_mission and disposition fields from starter templates

* build: validate template manifests against JSON Schema during docs build

* cleanup: remove unused JsonSchemaViewer component

* docs: remove retain_extraction_mode from starter templates

* ui: enable word wrap in template manifest preview

* docs: add link to Bank Templates reference doc from Hub page

* docs: convert bank templates doc to mdx with multi-language code snippets

- Convert bank-templates.md to .mdx with Tabs/CodeSnippet components
- Add example files: bank-templates.py, .mjs, .sh, .go with doc markers
- Examples cover import, dry-run, export, round-trip, and schema
- Regenerate OpenAPI spec and all client SDKs (Python, TS, Rust, Go)

* fix: migration revision collision + use typed models in benchmark template

- Rename merge migration d6e7f8a9b0c1 -> d6e7f8a9b0c2 to resolve
  revision ID collision with case_insensitive_entities_trgm_index
- Update a4b5c6d7e8f9 down_revision to point to the renamed migration
- Fix f-string lint in case_insensitive migration
- BenchmarkRunner.apply_template() now validates manifest through
  BankTemplateManifest Pydantic model instead of raw dict access
- Remove redundant inline imports (json, Path already at module top)

* fix(docs): add missing Go tab to dry-run code snippet

* ci: retrigger

* fix: sync skills openapi.json + fix bankId null type error in export

- Copy updated openapi.json to skills/hindsight-docs/references/
- Add null guard for bankId in Export Template onClick handler

* fix: sync generated files (memory_engine formatting, docs skill references)

* cleanup: remove obsolete migration collision workaround
2026-04-02 12:21:53 +02:00
Nicolò Boschi
9cfdd464a9
fix(retain): preserve normalized experience fact types (#848)
* fix(retain): preserve normalized experience fact types and remove deprecated opinion type

The ExtractedFactType conversion was re-checking for raw "assistant" fact_type
after the parsing layer had already normalized it to "experience". Since
fact_from_llm.fact_type was always "experience" (never "assistant"), the ternary
always fell through to "world", silently losing experience classification.

Also removes the deprecated "opinion" fact type from internal extraction models,
database constraints/indexes (via migration), and dead code paths. The public API
surface (descriptions, response models, backwards-compat filter) is unchanged.

* refactor(retain): drop unused confidence_score column

The confidence_score column was only ever non-null for opinion facts
(which are now removed). It was always written as NULL and never read
back from the database. Remove it from:
- DB model and migration (DROP COLUMN)
- INSERT queries in fact_storage.py
- retain_async/retain_batch_async parameters
- RetainContext/RetainResult extension models
- RetainBatch dataclass
2026-04-02 12:20:37 +02:00
Nicolò Boschi
8d1bfbbd2b
feat: add detail parameter to list/get mental models (#846)
* feat: add detail parameter to list/get mental models (#825)

Add a `detail` query parameter (metadata|content|full) to both list and get
mental model endpoints (HTTP + MCP) to control response size. This reduces
payload for agent boot flows and MCP clients where context budget is limited.

Closes #825

* fix: update Rust CLI for optional mental model fields

The generated Rust client now has content/source_query as Option<String>
after the detail parameter was added. Update CLI code to handle optionals.
2026-04-02 11:52:45 +02:00
Nicolò Boschi
7d6c570a3a
fix(embed): clear stale daemon on port before starting (#843)
* fix(embed): clear stale daemon on port before starting new one (#843)

When `uvx hindsight-embed@latest` resolves to a new version, the old
daemon may still be bound to the port, causing EADDRINUSE. Before
starting a daemon, check if the port is occupied, verify it's a
hindsight process via /health, and SIGTERM it if so.

* chore: remove unused signal import from test

* refactor: use cross-platform port check instead of lsof-only

Use socket for port check (works on all platforms), extract PID lookup
into a helper with Windows (netstat) and Unix (lsof) paths, and
extract kill logic into a testable static method.

* refactor: reuse cross-platform helpers in stop() and stop_ui()
2026-04-02 10:57:28 +02:00
Nicolò Boschi
26a64cc00e
fix(api): clear memories endpoint no longer deletes the bank profile (#837)
DELETE /v1/default/banks/{id}/memories and the MCP clear_memories tool
were calling delete_bank() without distinguishing from the actual delete-bank
endpoint. When no fact_type filter was provided, the bank row itself was
deleted along with its memories.

Add a delete_bank_profile parameter to delete_bank() (default True) and
pass False from all clear-memories callers so the bank profile, disposition,
and background are preserved.
2026-04-01 18:34:39 +02:00