Commit graph

365 commits

Author SHA1 Message Date
Derek Bouius
ee4510a762
fix(deps): address critical and high severity security vulnerabilities (#827)
* fix(deps): address critical and high severity security vulnerabilities

Bump vulnerable dependencies to patched versions across the monorepo:

Python (critical/high):
- fastmcp >=2.14.0 → >=3.2.0 (SSRF, path traversal, OAuth confused deputy, command injection)
- langchain-core >=1.2.11 → >=1.2.22 (path traversal in legacy load_prompt)

Python (low):
- cryptography >=46.0.5 → >=46.0.6 (incomplete DNS name constraint enforcement)
- pygments: add >=2.20.0 pin (ReDoS via GUID regex)

Node.js:
- serialize-javascript ^7.0.3 → ^7.0.5 (CPU exhaustion DoS)
- handlebars: add >=4.7.9 override (JS injection via AST type confusion)
- path-to-regexp: add >=0.1.13 override (ReDoS via route params)
- brace-expansion: add version range override (process hang/memory exhaustion)

Also adds type: ignore comments for FastMCP 2.x private attribute access that
ty now flags since FastMCP 3.x removed _tool_manager (guarded by try/except
and hasattr at runtime).

Regenerated all lock files across API, integrations, and tests.

* fix(deps): add ajv v8 scoped overrides for schema-utils and ajv-keywords

The global ajv ^6.14.0 override caused schema-utils and ajv-keywords to
receive ajv v6, but they require ajv v8 (for dist/compile/codegen). Add
scoped overrides to ensure these packages get ajv v8 while the global
override remains for packages that need v6.

* fix(tests): remove stateless_http from FastMCP() constructor calls

FastMCP 3.x no longer accepts stateless_http in the constructor. The
tests call tools directly without HTTP transport, so the parameter is
not needed.

* fix: update MCP tests for FastMCP 3.x _tool_manager removal

FastMCP 3.x removed _tool_manager. Tests now use
_local_provider._components for sync tool dict access and
mcp.list_tools() for async filtered tool listing.

* fix: resolve docusaurus build failures (ajv overrides + missing blog date)

- Remove global ajv ^6.14.0 override and scoped ajv-keywords/schema-utils
  overrides that caused webpack compilation errors manifesting as
  "Cannot read properties of undefined (reading 'date')" during SSR
  and "these parameters are deprecated" warnings. Natural version
  resolution (v6.12.6+ for v6 consumers, v8+ for v8 consumers) already
  satisfies the security fix (>= 6.12.3).
- Add missing date frontmatter to learning-capabilities blog post.

* chore: regenerate openapi spec and docs skill
2026-04-01 09:20:34 +02:00
Nicolò Boschi
6f7437be21
blog: What's New in Hindsight 0.4.22 release notes and changelog (#818) 2026-03-31 18:47:30 +02:00
Nicolò Boschi
d7f6723546 Release v0.4.22
- Update version to 0.4.22 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- Helm chart
- Sync documentation to version-0.4
2026-03-31 18:14:59 +02:00
Nicolò Boschi
2c32ffadc9
fix(mental-models): add tags_match and tag_groups to trigger config (#786) (#804)
When a mental model has tags, refresh_mental_model hardcoded
tags_match="all_strict", causing empty results when most memories
are untagged. Add configurable tags_match and tag_groups fields
to MentalModelTrigger so users can control refresh filtering.

- Add tags_match (any/all/any_strict/all_strict) to override default
- Add tag_groups for compound boolean tag expressions during refresh
- Default behavior unchanged (all_strict when tags present)
- Update both refresh paths (task-based and direct)
- Add UI controls in Create/Update mental model dialogs
- Regenerate OpenAPI spec and client SDKs
2026-03-31 18:09:01 +02:00
Nicolò Boschi
baf5447de2
refactor: replace LLMProvider classmethods with from_env() and document missing config fields (#816)
CI failures are unrelated to this PR:
- test_mental_models_dimension_change_empty_table: database OID error (infrastructure flake)
- test_reflect_searches_mental_models_when_available: LLM-dependent assertion (flaky)
2026-03-31 18:00:41 +02:00
Ben
410c208746
What's New: multi-org support and credit transfers (March 29) (#815)
* Add What's New post: multi-org support and credit transfers
2026-03-31 10:19:08 -04:00
Amin Bolakhrif
f841bcb92d
feat: add optional LiteLLM SDK embedding output dimensions (#809)
* feat: add optional LiteLLM SDK embedding output dimensions

Allow configuring an optional output dimension for litellm-sdk embeddings and pass it through only when set, while preserving default behavior.

Made-with: Cursor

* test: assert wrapped init error for invalid dimensions

Add a LiteLLM SDK embeddings test that verifies invalid OpenAI dimensions fail during initialize() and preserve provider error details in the wrapped RuntimeError.

Made-with: Cursor
2026-03-31 14:58:02 +02:00
Nicolò Boschi
627ec5d524
feat: expose document_metadata in API and control plane (#798)
* feat: expose document_metadata in API and control plane

Add document_metadata (sourced from retain_params.metadata) to both
list and get document endpoints. Display it in the control plane
documents table and detail panel. Drop the unused metadata column
from the documents table (was always stored as empty {}).

* fix: code review fixes for document_metadata feature

- Remove unnecessary `import json as _json` (json already imported at module level)
- Simplify redundant truthiness checks in retain_params parsing
- Regenerate OpenAPI spec and client SDKs (Python, TypeScript, Go)
- Add tests for document_metadata in get_document and list_documents

* feat(ui): improve documents table and detail panel

- Relative timestamps with full date on hover
- Remove context column from table
- Metadata shown as k=v badges (blue, like tags)
- Size in bytes instead of chars
- Document IDs wrap instead of truncating
- Detail panel wider (560px)
- Retain params: context, event_date, metadata badges
2026-03-31 11:42:02 +02:00
Nicolò Boschi
b104bad02c
fix(codex): merge new settings on upgrade instead of skipping (#780)
The installer skipped settings.json entirely if it already existed,
leaving version and new config keys stale. Now merges: updates version,
adds new upstream keys, preserves user customizations.

Also fixes pre-existing typo: RERANK_URL → rerank_url in ZeroEntropy
cross-encoder.
2026-03-31 09:05:49 +02:00
Chris Bartholomew
45ffc7fe90
SEO: add title and description to all integration pages (#787)
* SEO: add title and description to all integration pages

All 17 integration docs pages were missing title and description
frontmatter, causing Docusaurus to generate unhelpful titles like
"OpenClaw | Hindsight" and pull body text as meta descriptions.

- Add keyword-rich title and description frontmatter to all integration
  pages in both docs/ (current) and versioned_docs/version-0.4/
- Add scripts/check-integration-seo.mjs to enforce title + description
  on all future integration pages
- Wire the check into the build script so it runs locally and in CI

* Fix missing frontmatter on docs/sdks/integrations/openclaw.md

* Regenerate docs skill after integration page SEO updates
2026-03-30 17:53:43 -04:00
Chris Bartholomew
99122055f0 Improve OpenClaw post title, tags, and meta description
- Retitle to match search intent: "How to Add Persistent Memory to
  OpenClaw with Hindsight" targets openclaw memory/persistent memory queries
- Add intro paragraph before <!-- truncate --> so Docusaurus generates a
  proper meta description instead of "TL;DR"
- Expand tags from [openclaw] to include memory, agents, persistent-memory,
  knowledge-graph
2026-03-30 15:41:55 -04:00
Nicolò Boschi
75e2679cf1 release(llamaindex): v0.1.3 2026-03-30 18:34:25 +02:00
DK09876
d93dfea8ce
fix(llamaindex): document_id, memory API, and ReAct trace fixes (#777)
* fix(llamaindex): use uuid for document_id and sync version metadata

- Replace timestamp-based document_id with uuid4 hex to prevent
  collisions on rapid retains (timestamp_ms can duplicate in tight loops)
- Sync __version__ in __init__.py to match pyproject.toml (0.1.2)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(docs): pass memory to run() instead of ReActAgent constructor

LlamaIndex 0.14.x ReActAgent does not accept a memory parameter in
its constructor — it's silently dropped via **kwargs. Memory must be
passed to agent.run(memory=...) where AgentWorkflow picks it up.

Also fixes the undefined `tools` variable (now `tools=[]`).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(llamaindex): strip ReAct reasoning traces from retained assistant messages

HindsightMemory.put/aput now extracts only the final Answer: text from
assistant messages containing ReAct reasoning (Thought:/Action:/Observation:
prefixes), preventing internal reasoning traces from polluting long-term memory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(llamaindex): fix docstring example to pass memory to run()

The HindsightMemory class docstring showed the broken pattern of passing
memory= to the ReActAgent constructor, which silently drops it. Updated
to show the correct pattern: pass memory to agent.run().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 18:33:03 +02:00
Nicolò Boschi
e5209b18b3
docs: add RERANKER_ZEROENTROPY_BASE_URL to configuration page (#779)
Document the new configurable base URL for the ZeroEntropy reranker
provider added in #766. Also fix a type error where RERANK_URL was
renamed to rerank_url but one usage was missed.
2026-03-30 17:51:04 +02:00
Nicolò Boschi
a7adfbb0df release(codex): v0.2.0 2026-03-30 17:50:02 +02:00
Nicolò Boschi
2c72af5525 release(openclaw): v0.5.1 2026-03-30 16:58:18 +02:00
DK09876
7af01e35e9
fix(docs): use tools=[] in BaseMemory example (#772)
The automatic memory example referenced an undefined `tools` variable.
Since HindsightMemory handles retain/recall transparently, no tools
are needed — use an empty list.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 16:56:22 +02:00
Ben
f30ca3deda
Fix blog homepage: Hindsight Cloud section always shows top 3 posts (#770)
* Fix blog homepage: show all posts so Cloud section always gets top 3
2026-03-30 10:40:57 -04:00
Ben
d61517d502
Update MCP OAuth blog post date to 2026-03-30 (#769) 2026-03-30 13:53:41 +00:00
Ben
df17570d8a
blog: What's New in Hindsight Cloud — Native OAuth for MCP Clients (#731)
* Add MCP OAuth blog post
2026-03-30 09:33:58 -04:00
Nicolò Boschi
7a9e99998a
docs: 0.4.21 release blog post and changelog (#765)
* docs: 0.4.21 release blog post and changelog

* fix(blog): align 0.4.21 code snippets with docs, add release image

* chore: regenerate docs skill references
2026-03-30 15:28:27 +02:00
Nicolò Boschi
cc3cdc2f83 Release v0.4.21
- Update version to 0.4.21 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- Helm chart
- Sync documentation to version-0.4
2026-03-30 14:52:50 +02:00
Nicolò Boschi
88630f93d7 release(hermes): v0.5.0 2026-03-30 14:50:04 +02:00
Nicolò Boschi
6a28373ecf release(openclaw): v0.5.0 2026-03-30 14:47:01 +02:00
Nicolò Boschi
b8b40e458b release(codex): v0.1.1 2026-03-30 14:40:58 +02:00
Nicolò Boschi
e65bd361cc release(llamaindex): v0.1.2 2026-03-30 14:37:15 +02:00
Nicolò Boschi
56489a58d1
refactor(llamaindex): merge into single hindsight-llamaindex package (#760)
* refactor(llamaindex): merge two packages into single hindsight-llamaindex

Merge `llama-index-tools-hindsight` and `llama-index-memory-hindsight` into
a single `hindsight-llamaindex` package following our naming convention.

- Rename package to `hindsight-llamaindex` (Python module: `hindsight_llamaindex`)
- Move HindsightToolSpec and HindsightMemory into the same package
- Delete `llamaindex-memory/` directory
- Add CI test job for llamaindex integration
- Update docs, blog post, and integrations.json

* fix(blog): update llamaindex blog post for merged package

- Move date to 2026-03-30
- Add HindsightMemory (automatic BaseMemory) pattern
- Fix "bank must exist first" pitfall — mission auto-creates
- Align all code examples with docs page
- Update architecture diagram to show both patterns

* fix(docs): add llamaindex/openai icons, rename Codex

- Add llamaindex.png and openai.png icons
- Rename "OpenAI Codex CLI" to "Codex" in integrations.json and docs
- Use openai.png icon for Codex integration
2026-03-30 14:30:48 +02:00
Nicolò Boschi
a8a63818c7
feat(api): add duration_ms to audit log entries (#758)
* feat(api): add duration_ms to audit log entries

Server-computed duration in milliseconds (started_at → ended_at) on
the list audit logs endpoint. Null when ended_at is not set.

Closes #749

* feat(api): add duration_ms to audit log entries and type audit endpoints

- Add server-computed duration_ms (started_at → ended_at) to audit log
  list response. Null when ended_at is not set.
- Add typed Pydantic response models for both audit log endpoints
  (list and stats) so they appear in the OpenAPI spec.
- Regenerate OpenAPI spec and all client SDKs.

Closes #749

* chore: regenerate docs skill after audit log response models
2026-03-30 12:32:02 +02:00
Nicolò Boschi
d8050387e4
fix(mcp): handle Claude Code GET probe and make stateless_http configurable (#757)
* fix(mcp): handle Claude Code GET probe and make stateless_http configurable (#751)

Claude Code v2.1.84+ sends a GET to /mcp/ before POST initialize,
which fails with 405 (stateless) or 400 (stateful). Intercept
sessionless GET requests in MCPMiddleware and return 200 OK so the
client proceeds to POST initialize.

Also make stateless_http configurable via HINDSIGHT_API_MCP_STATELESS
(default: false/stateful) instead of hardcoding true.

Closes #751

* docs: add HINDSIGHT_API_MCP_STATELESS to configuration reference
2026-03-30 12:15:06 +02:00
Nicolò Boschi
38e03e419d
Convert codex tool_choice test to pytest style (#752)
* Convert codex tool_choice test to pytest style

Follow-up to #734: replace unittest.TestCase + manual sys.path
manipulation with idiomatic pytest + @pytest.mark.asyncio,
matching the rest of the test suite.

* Fix test_hierarchical_fields_categorization for new configurable fields

Update expected count from 20 to 21 and add assertions for fields
added by recent PRs: retain_default_strategy, retain_strategies,
max_observations_per_scope, reflect_source_facts_max_tokens,
llm_gemini_safety_settings, mcp_enabled_tools.

* Add LlamaIndex doc to v0.4 versioned docs and sidebars

The LlamaIndex integration doc was added to docs/ (next version) in
#672 but not to versioned_docs/version-0.4/, causing a broken link
on the /integrations page which resolves to the latest version.

* Regenerate docs skill references

Run generate-docs-skill.sh to pick up new integration pages
(codex, llamaindex) and updated configuration docs.

* Add Codex integration doc to v0.4 versioned docs and sidebar

Same issue as LlamaIndex: doc was added to docs/ (next) but not
versioned_docs/version-0.4/, causing broken link on /integrations.
2026-03-30 11:58:59 +02:00
Nicolò Boschi
ecf16ea1e6
fix(codex): cleanup dead code, add release lifecycle and docs (#753)
* fix(codex): cleanup dead code and add to release lifecycle

- Remove orphaned reflect() method from client.py (leftover from dropped auto-mode)
- Remove dead retainToolCalls config default (never wired through)
- Add codex to release-integration.sh valid integrations
- Add settings.json version fallback to release script
- Add codex CI test job in test.yml
- Add codex to integrations.json registry

* docs(codex): add changelog page and link from integration docs

* feat(codex): add hosted installer script (get-codex)

Add self-contained installer at hindsight.vectorize.io/get-codex that
downloads scripts from GitHub, configures hooks, and supports local/cloud
mode selection — no git clone required.

Update docs and README to use the one-liner install:
  curl -fsSL https://hindsight.vectorize.io/get-codex | bash

* chore(codex): remove install.sh in favor of hosted get-codex

* fix(docs): use /next/ prefix for codex changelog link

* fix(docs): use GitHub link for codex changelog back-link
2026-03-30 11:34:43 +02:00
Ben
0b17a67c70
feat: add Hindsight memory integration for OpenAI Codex CLI (#730)
* feat(codex): add Hindsight memory integration for OpenAI Codex CLI

Hooks-based integration that gives Codex CLI long-term memory via Hindsight.
Three hooks keep memory in sync: SessionStart (daemon pre-warm), UserPromptSubmit
(recall + context injection), Stop (retain conversation to memory).

Key differences from the Claude Code integration:
- Codex transcript format: JSONL with {msg: {type, message}} (user_message/agent_message)
- No CODEX_PLUGIN_ROOT env var — install.sh writes hooks.json with absolute paths
- State stored in ~/.hindsight/codex/state/ (not CLAUDE_PLUGIN_DATA)
- No async: true in hooks (not supported by Codex)
- No SessionEnd event
- hooks.json written to ~/.codex/hooks.json with codex_hooks = true in config.toml

* fix(codex): fix transcript parser for actual Codex disk format

Codex stores sessions as rollout-*.jsonl with response_item entries:
  User:      {type:response_item, payload:{type:message, role:user, content:[{type:input_text, text:...}]}}
  Assistant: {type:response_item, payload:{type:message, role:assistant, phase:final_answer, content:[{type:output_text, text:...}]}}

Previous parser expected an undocumented {msg:{type:user_message}} format from the Rust protocol spec
that does not match the actual on-disk storage format.

* feat(codex): add reflect mode to UserPromptSubmit hook

Add recallMode config option (default: 'recall') that switches the
UserPromptSubmit hook between:
- 'recall': existing behavior, fast raw facts list
- 'reflect': agentic synthesis loop, returns coherent prose answer

Also adds reflect() method to HindsightClient and HINDSIGHT_RECALL_MODE
env var override. Reflect uses a 25s timeout (vs 10s for recall).

* feat(codex): auto mode for recall/reflect selection

Add recallMode: 'auto' (new default) that picks the operation per-query:
- Synthesis patterns (what do you know, what's my, summarize, etc.) → reflect
- All other prompts → recall (fast, raw facts, better for code tasks)

* feat(codex): add automated test suite and finalize recall-only mode

* docs(codex): add docs page and sidebar entry for Codex CLI integration
2026-03-30 10:51:53 +02:00
DK09876
2d787c4ffd
feat: add LlamaIndex integration (#672)
* feat: add LlamaIndex integration for Hindsight

Add hindsight-llamaindex package providing persistent memory tools for
LlamaIndex agents via the native BaseToolSpec pattern. Includes retain,
recall, and reflect tools, a convenience factory, global config, full
test suite, docs page, blog post, and integrations.json entry.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback for llamaindex integration

- Fix ReActAgent API: from_tools() → constructor, chat() → await run()
- Add create_bank step to all quickstart examples
- Add production patterns section to docs (tags, error handling, bank lifecycle)
- Add memory scoping recommendation to README
- Add when-not-to-use section to blog post
- Add LlamaIndex compatibility tests (agent acceptance, FunctionTool.call)
- Fix self-hosted auth wording in cookbook notebook

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use async client methods and asyncio.run() for runnable examples

- Use await client.acreate_bank() instead of sync create_bank() to
  avoid "event loop already running" errors in notebooks and async contexts
- Wrap plain Python examples in async def main() + asyncio.run(main())
  so they are copy-paste runnable as scripts
- Add Jupyter notebook tip to docs showing top-level await pattern
- Bank lifecycle example in docs now uses async acreate_bank

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add async tool methods to avoid event loop conflicts

HindsightToolSpec now provides both sync and async tool implementations
using LlamaIndex's (sync_fn, async_fn) tuple pattern in spec_functions.
Async agents (ReActAgent, etc.) use aretain/arecall/areflect natively,
avoiding the "Timeout context manager should be used inside a task"
error that occurred when sync _run_async() was called from within an
active event loop.

- Add aretain_memory, arecall_memory, areflect_on_memory async methods
- Extract shared kwargs builders (_retain_kwargs, _recall_kwargs, etc.)
- spec_functions now uses tuples: [("retain_memory", "aretain_memory"), ...]
- Tests verify tools have both sync fn and async fn set
- Notebook verified end-to-end with nbclient against local Hindsight

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove blog post from integration PR

The blog post will be pulled in separately from its own PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Address PR review: add context label, document_id auto-gen, bank mission, graceful errors

- Add `retain_context` param (default: "llamaindex") as source label on retain ops
- Auto-generate `document_id` as `{session_id}-{timestamp_ms}` when not provided
- Add `retain_async` param (default: True) for non-blocking retain processing
- Add `mission` param for automatic bank creation/management on first use
- Change error handling from raising HindsightError to graceful log + return message
- Add per-operation timeout constants in _client.py
- Add `context` and `mission` fields to config.py and configure()
- Update docs: document as standalone package (not LlamaHub), new params, patterns
- Tests: 51 passing (up from 34), covering all new features

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restructure to LlamaIndex namespace packages + add BaseMemory implementation

Tools package (llama-index-tools-hindsight):
- Restructured from hindsight_llamaindex/ to llama_index/tools/hindsight/
- Import: from llama_index.tools.hindsight import HindsightToolSpec
- Follows PEP 420 implicit namespace package convention
- Removed retain_async param (client.retain() doesn't support async_processing)

Memory package (llama-index-memory-hindsight):
- New package: llama_index/memory/hindsight/
- HindsightMemory(BaseMemory) for automatic memory
- put() auto-retains user/assistant messages to Hindsight
- get(input) auto-recalls relevant memories, prepends as system message
- Graceful error handling, bank mission management, document_id generation
- 28 unit tests passing

Both packages follow LlamaIndex community conventions for future LlamaHub submission.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 10:43:35 +02:00
Nicolò Boschi
b32767caa8
feat: add max_observations_per_scope bank config (#729)
* feat: add max_observations_per_scope bank config

Adds a configurable limit on the number of observations per tag scope.
When the limit is reached, consolidation only updates/deletes existing
observations — no new ones are created. Enforcement is done via a
constrained Pydantic response model (max_length on creates list) so the
LLM structurally cannot exceed the limit, plus prompt guidance.

- Config: HINDSIGHT_API_MAX_OBSERVATIONS_PER_SCOPE (-1 = unlimited)
- Reorder action execution: deletes → updates → creates
- Dynamic _ConsolidationBatchResponse with max_length constraint
- Prompt CAPACITY CONSTRAINT section when near/at limit
- Observations with no tags skip the limit entirely
- Control plane UI field + docs

* fix: strengthen max_observations tests with mock LLM + defensive truncation

- Rewrite integration tests to use MockLLM with deterministic responses
  (one observation per fact) instead of relying on real LLM behavior
- Add defensive truncation in _consolidate_batch_with_llm as belt-and-
  suspenders — catches LLM providers that ignore JSON schema max_length
- Tests now assert exact counts, not just upper bounds
2026-03-30 10:29:56 +02:00
Ben
a69bdbb55f
How We Built a 4-Way Hybrid Search System That Actually Runs in Parallel (#708)
* Add blog: How We Built a 4-Way Parallel Hybrid Search System

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Add cover image for parallel hybrid search post

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update parallel hybrid search post date to 2026-03-27

* Set author to chrislatimer

* Update recall docs link

* review: align blog post to actual retrieval code

- Reframe as evolutionary narrative (V1 asyncio.gather → connection sharing)
- Add missing reranker section (cross-encoder + multiplicative boost scoring)
- Replace MPFP references with LinkExpansion (3-signal CTE)
- Fix SQL to match actual UNION ALL approach, explain CTE planner issue
- Fix acquire_with_retry, index types (ivfflat→HNSW), fusion code
- Remove fabricated perf numbers
- Add alpha calibration rationale and connection contention insight

* add nicoloboschi and benfrank241 as co-authors

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>
2026-03-27 11:21:01 -04:00
Chris Bartholomew
dffb87080f
fix(migrations): bypass PgBouncer for advisory locks via MIGRATION_DATABASE_URL (#726)
* fix(migrations): use HINDSIGHT_API_MIGRATION_DATABASE_URL when set

Session-level advisory locks are broken when the database URL goes
through PgBouncer in transaction mode: the backend connection is
returned to the pool on COMMIT, orphaning the lock, so multiple pods
can simultaneously run migrations for the same schema.

When HINDSIGHT_API_MIGRATION_DATABASE_URL is set, use it for both
the advisory lock connection and the Alembic run.  Callers should
point this at the direct PostgreSQL endpoint (bypassing the pooler)
so the session-level lock is held for the full migration duration.

* refactor(migrations): move MIGRATION_DATABASE_URL to standard config

Wire HINDSIGHT_API_MIGRATION_DATABASE_URL through HindsightConfig
instead of reading os.getenv() directly in migrations.py. Add the
field to the dataclass, from_env(), log_config(), all call sites,
.env.example, and the configuration docs page.

* fix: update test mocks for migration_database_url kwarg and regenerate docs skill

---------

Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>
2026-03-27 16:01:38 +01:00
Nicolò Boschi
3c78b717b0
docs: add AG2 integration page (#723)
- Add AG2 integration doc with quick start, configuration, GroupChat example, and API reference
- Add to sidebar, versioned sidebar, and integrations hub
- Add AG2 icon
2026-03-27 10:58:42 +01:00
Nicolò Boschi
9321c59bf1 release(ag2): v0.1.1 2026-03-27 10:12:30 +01:00
Nicolò Boschi
083295dc6f
feat: add audit log for feature usage tracking (#717)
* feat: add audit log for feature usage tracking

Add full auditability for all mutating and core API operations across
HTTP, MCP, and system (worker) transports. Audit entries record raw
request/response as JSONB, timing (started_at/ended_at), action, and
transport type.

Backend:
- New audit_log table with JSONB columns for expandability without
  future migrations (merge migration of 3 existing heads)
- AuditLogger with fire-and-forget writes via asyncio.create_task
- @audited decorator on 28 HTTP route handlers
- MCP tool audit wrapping for 16 auditable tools
- Worker task execution wrapped with audit_context
- List endpoint with action, transport, date range filters + pagination
- Stats endpoint with per-day counts for charting
- Configurable retention sweep (concurrent-safe DELETE)

Config (env-only, static):
- HINDSIGHT_API_AUDIT_LOG_ENABLED (default: false)
- HINDSIGHT_API_AUDIT_LOG_ACTIONS (comma-separated allowlist, empty=all)
- HINDSIGHT_API_AUDIT_LOG_RETENTION_DAYS (default: -1, keep forever)

Control Plane:
- New "Audit Logs" tab on bank configuration page
- Line chart showing request volume (today/7d/30d) with action filter
- Filterable table with action, transport, date range filters
- Paginated list with detail dialog showing raw request/response JSON

Tests:
- 13 tests covering list, filters, pagination, stats, disabled mode,
  action allowlist, and ordering

* fix: split 3-way merge migration into two 2-way merges

Alembic doesn't support 3-parent merge migrations. Split into a no-op
merge of 2 heads (b1c2d3e4f5g6) followed by the audit_log table
migration merging the third head.

* fix: correct merge migration to merge actual 2 heads

The original analysis incorrectly identified 3 heads. There were only 2
(a3b4c5d6e7f8 and c8e5f2a3b4d1). Remove the unnecessary intermediate
merge migration and fix the audit_log migration to merge these 2 heads.

* fix: use 'heads' instead of 'head' in migration runner

Alembic's upgrade('head') fails when multiple heads exist (e.g. from
namespace package overlaps between hindsight-api and hindsight-api-slim).
Using 'heads' (plural) handles this gracefully by upgrading all branches.

* chore: regenerate OpenAPI spec with audit log endpoints

* chore: regenerate TypeScript client and docs skill OpenAPI spec

Python and Go clients still need regeneration (requires Docker).

* chore: regenerate all client SDKs (Python, Go, TypeScript)

Adds generated audit log API clients for Python (audit_api.py),
Go (api_audit.go), and TypeScript client type updates.
2026-03-27 09:52:03 +01:00
Ben
62c0992075
Teaching the Llama to Remember (#707)
Llama index blog
2026-03-26 15:44:18 -04:00
Nicolò Boschi
02b0f7799d
docs: add Volcano Engine as supported LLM provider (#715)
* docs: add Volcano Engine as supported LLM provider

Follow-up to #714. Add Volcano Engine (ByteDance) to the documentation:
- LLM providers grid component
- Provider list in configuration docs
- Provider example with base URL and default model
- Default model table in models page

* chore: regenerate docs skill references
2026-03-26 18:21:34 +01:00
Nicolò Boschi
200bab233e
feat(openclaw): add recallInjectionPosition config to preserve prompt cache (#710)
* feat(openclaw): add recallInjectionPosition config to preserve prompt cache

Add configurable injection position for recalled memories to avoid
breaking prefix-based prompt caching (Anthropic/Google) when agents
have large static system prompts.

Options: 'prepend' (default, current behavior), 'append' (end of
system prompt, preserves cache), 'user' (before user message).

Closes #703

* docs(openclaw): document all plugin config flags

Add missing config options to the OpenClaw docs: recallTopK,
recallTypes, recallContextTurns, recallMaxQueryChars,
recallPromptPreamble, recallInjectionPosition, recallRoles,
retainEveryNTurns, retainOverlapTurns, and debug.
2026-03-26 16:09:25 +01:00
Nicolò Boschi
91397190c0
docs(claude-code): tidy configuration reference and sync README (#706)
* docs(claude-code): tidy configuration reference and sync README

Add missing settings (retainMode, retainToolCalls, retainTags,
retainMetadata, embedPackagePath, llmApiKeyEnv, agentName, and
several recall options) that existed in code but not in docs.
Restructure config tables with prose introductions, clearer
descriptions, and consistent layout across both files.

* refactor(claude-code): remove recallTopK setting

Unused client-side cap — Hindsight server already controls result
count via recallBudget and recallMaxTokens.
2026-03-26 14:07:35 +01:00
Nicolò Boschi
ffc96bec97 release(claude-code): v0.3.0 2026-03-26 12:58:25 +01:00
Nicolò Boschi
64d96a9c53 release(claude-code): v0.2.0 2026-03-26 12:11:28 +01:00
Nicolò Boschi
349c112c61
docs: add supported platforms and Windows installation guide (#700)
* docs: add supported platforms section and Windows installation guide

Adds a platform compatibility table (Linux, macOS, Windows) and a
dedicated Windows setup section with step-by-step instructions for
installing PostgreSQL + pgvector and running Hindsight natively.
Follows up on #699 which added Windows native support.

Also fixes a ty type-check error in metrics.py for the conditional
resource module import.

* chore: sync generated clients and lock file after #699

Regenerate client SDKs to pick up ValidationError model changes
and update uv.lock with platform-specific uvloop/winloop deps.

* docs: update Windows section — pg0 now supports Windows

pg0 v0.12.0 added Windows support, so embedded DB works everywhere.
Restructure Windows section to show simple install-and-run first,
with external PostgreSQL as an optional alternative.

* chore: sync generated docs skill and openapi references
2026-03-26 12:01:30 +01:00
Mr. Khachaturov
939cb40a73
fix: include Pydantic v2 fields in ValidationError OpenAPI schema (#697)
FastAPI generates the ValidationError schema with only loc, msg, and
type, but Pydantic v2 actually returns input, ctx, and url as well.
Generated clients with strict JSON decoding (Go's DisallowUnknownFields)
cannot parse real 422 responses — the actual validation message gets
replaced by a confusing JSON decoding error.

- Patch the OpenAPI schema in create_app() to add input, ctx, url
- Regenerate spec and Go client
2026-03-26 11:25:46 +01:00
Nicolò Boschi
9e5a066d26
feat: add 'none' LLM provider for chunk-only storage mode (#691)
Adds a proper 'none' provider option so users can run Hindsight as a
chunk store with semantic search but without any LLM dependency, replacing
the hacky workaround of setting provider to 'mock'.

When HINDSIGHT_API_LLM_PROVIDER=none:
- Retain automatically uses chunks mode (no fact extraction)
- Recall works normally (semantic search, BM25, graph retrieval)
- Reflect returns HTTP 400 with clear error message
- Consolidation/observations are disabled
- Mental model refresh returns HTTP 400
- No API key required
2026-03-25 18:01:20 +01:00
Nicolò Boschi
5095d5e36f
feat(reflect): make source facts in search_observations configurable (#688)
* feat(reflect): make source facts in search_observations configurable

The recent fix (#669) hardcoded include_source_facts=False in
search_observations to prevent context overflow. This makes it
configurable via HINDSIGHT_API_REFLECT_SOURCE_FACTS_MAX_TOKENS
(env/tenant/bank), defaulting to -1 (disabled).

- -1: source facts disabled (current behavior, default)
- 0: source facts enabled with no token limit
- >0: source facts enabled with a token budget

* docs: add reflect_source_facts_max_tokens to configuration reference

* fix: update configurable fields count in tests and regenerate docs skill
2026-03-25 17:54:49 +01:00
Nicolò Boschi
0ff36548e0
feat(hermes): file-based config + updated docs (#686)
* feat(hermes): file-based config + updated docs

Replace the old dataclass/configure() singleton with a plain dict
config loaded from ~/.hindsight/hermes.json — same field names and
conventions as the openclaw and claude-code integrations.

Loading order: defaults → config file → env var overrides.

- config.py: rewritten with load_config() returning a plain dict,
  DEFAULTS matching openclaw/claude-code fields, ENV_OVERRIDES with
  typed casting
- tools.py: register() uses load_config() instead of raw env vars
- __init__.py: clean exports (removed configure/get_config/reset_config)
- README.md: full rewrite with config file examples, tables by category
- docs/hermes.md: full rewrite with quick start, architecture, all
  config tables, gateway section, troubleshooting
- tests: updated for new config pattern, 46 tests pass

* ci: add test job for hermes integration

* chore: regenerate docs skill for hermes integration
2026-03-25 16:11:38 +01:00