Commit graph

499 commits

Author SHA1 Message Date
Nicolò Boschi
0758827d39
fix: npx hindsight-control-plane fails (#408) 2026-02-19 18:23:13 +01:00
Nicolò Boschi
ea8163c56d
fix(mcp): unify hindsight-mcp-local and server mcp (#407)
* fix(mcp): stateless param not supported anymore

* fixes

* fix: npx hindsight-control-plane fails
2026-02-19 17:57:17 +01:00
Nicolò Boschi
ac73948706
fix: docker startup fails with named docker volumes (#405) 2026-02-19 16:42:27 +01:00
Nicolò Boschi
5569d4adba
feat: include source facts in observation recall (#404)
* feat: include source facts in observation recall

* feat: include source facts in observation recall

* feat: include source facts in observation recall

* feat: include source facts in observation recall

* fix(cli): add missing source_facts field to IncludeOptions initializer
2026-02-19 14:54:19 +01:00
Nicolò Boschi
e785b05831
fix(mcp): stateless param not supported anymore (#406) 2026-02-19 13:42:54 +01:00
Nicolò Boschi
58c4d65778
fix: reranker crashes on provider error (#403)
* fix: reranker crashes on provider error

* fix: reranker crashes on provider error
2026-02-19 11:38:37 +01:00
Derek Bouius
c3ef1555bf
fix: reduce temporal ordering offset from 10s to 10ms per fact (#402)
The 10-second offset per fact caused significant timestamp drift when
ingesting many items — e.g. 600 facts would shift the last fact by
~100 minutes from its actual event time. This broke timeline views
and made occurred_start/mentioned_at unreliable for temporal queries.

Reducing to 10ms preserves fact ordering while keeping timestamps
within ~8 seconds of the original values even for large batches.
2026-02-19 10:48:18 +01:00
Nicolò Boschi
dcaa9f14ab
fix: clients don't respect timeout setting (#400) 2026-02-19 10:47:47 +01:00
Ben
41db2960c5
feat: add CrewAI integration for persistent crew memory (#319)
* feat: add CrewAI integration for persistent crew memory

Implements a CrewAI ExternalMemory storage backend that maps CrewAI's
Storage interface (save/search/reset) to Hindsight's retain/recall/delete
APIs, giving crews long-term memory with fact extraction, entity tracking,
and temporal awareness across runs.

Key features:
- HindsightStorage: drop-in Storage backend for CrewAI ExternalMemory
- HindsightReflectTool: BaseTool exposing Hindsight's reflect API
- Per-agent memory banks with customizable bank resolver
- Async compatibility layer for CrewAI's threading model
- 35 unit tests, docs site page, example script

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: move CrewAI example to hindsight-cookbook

Move research_crew.py example from hindsight-integrations/crewai/examples/
to the cookbook repo and update the integration README to link there instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: add GitHub Actions test job for CrewAI integration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: add uv.lock for frozen installs in CI

The test-crewai-integration CI job uses `uv sync --frozen` which
requires a committed lock file.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 17:05:52 +01:00
Nicolò Boschi
f78278ea89
fix: document not tracked if has 0 extracted facts (#399)
* fix: document not tracked if has 0 extracted facts

* fix: document not tracked if has 0 extracted facts
2026-02-18 17:01:57 +01:00
Nicolò Boschi
117dd6988d
doc: changelog for 0.4.12 (#397)
* changelog for 0.4.12

* changelog for 0.4.12
2026-02-18 14:55:19 +01:00
Nicolò Boschi
7c78ae2371 Release v0.4.12
- Update version to 0.4.12 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-litellm, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- OpenClaw integration: hindsight-integrations/openclaw
- AI SDK integration: hindsight-integrations/ai-sdk
- Helm chart
- Sync documentation to version-0.4
2026-02-18 14:13:33 +01:00
Nicolò Boschi
6c695eb9f8
fix: improve openclaw test coverage (#396)
* fix: improve openclaw test coverage

* test(openclaw): export stripMemoryTags/extractRecallQuery and add hook integration tests

- Extract stripMemoryTags and extractRecallQuery as exported pure functions
  from index.ts so hooks share one implementation and tests cover the real code
- Update before_agent_start to call extractRecallQuery; update agent_end to
  call stripMemoryTags instead of duplicating the regex inline
- Rewrite index.test.ts to import the real functions (no more local duplicate)
  and add 11 tests for extractRecallQuery covering all envelope-stripping cases
- Add tests/hooks.integration.test.ts: loads the plugin via mock MoltbotPluginAPI
  in HTTP mode, spies on client.recall/retain, and exercises all hook behaviours:
  excluded providers, short messages, memory injection format, tag stripping,
  transcript formatting, array content blocks, metadata, document_id derivation
2026-02-18 14:10:33 +01:00
Nicolò Boschi
7eafba661e
feat: add iris as file parser (#395)
* feat: add iris as file parser

* fix
2026-02-18 14:09:56 +01:00
Anton Evseev
c461013047
fix(openclaw): shell safety, HTTP dual-mode, lazy reinit, per-user banks (#388)
- exec→execFile: bypass shell entirely, preventing injection via
  special characters in chat history
- HTTP dual-mode: client can now talk directly to the Hindsight API
  via HTTP (setBankMission, retain, recall) when apiUrl is configured,
  bypassing the subprocess/CLI entirely for production deployments
- HindsightClientOptions: replace 5 positional constructor args with
  a typed options object for clarity and extensibility
- sanitize(): strip null bytes from strings — Node 22 rejects them
  in execFile() args
- recall timeout: accept optional timeoutMs parameter for both HTTP
  and subprocess modes; subprocess gets a longer 30s default
- In-flight recall dedup: concurrent recalls for the same bank reuse
  one promise instead of firing duplicate requests
- Timeout/abort handling: graceful warn-level logging instead of
  error spam when recall times out
- Error cause chaining: wrap errors with { cause } for better
  debugging stack traces
- lazyReinit: recover from startup health check failure with 30s
  cooldown and concurrency guard
- Per-user banks: derive bank ID from senderId (not channelId) for
  proper memory isolation per user across channels
- buildClientOptions(): centralized helper replaces 7 duplicated
  constructor call sites

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 13:09:02 +01:00
Nicolò Boschi
7c99feb018
fix(go-client): add go build to CI (#393)
* feat(go-client): add NewAPIClientWithToken helper and expand recall vs reflect FAQ

- Add NewAPIClientWithToken convenience function to Go client for easy authenticated client creation
- Expand FAQ with detailed "When should I use recall vs reflect?" guidance including practical examples

* fix(go-client): add go build to CI and preserve hindsight_client.go in generator

- Add explicit 'go build ./...' step before integration tests for faster compile feedback
- Preserve hindsight_client.go as a maintained file in generate-clients.sh
2026-02-18 13:06:17 +01:00
Nicolò Boschi
d06a0259cc
feat: improve ai sdk tools (#394) 2026-02-18 13:06:03 +01:00
Eliah Rusin
be8728b313
fix(go-client): use monorepo-compatible module path (#392)
The Go SDK declared its module as github.com/vectorize-io/hindsight-client-go,
but that repository doesn't exist. Update to
github.com/vectorize-io/hindsight/hindsight-clients/go to match the actual
monorepo path, enabling standard `go get` imports with directory-prefixed tags.

Also enables isGoSubmodule in the OpenAPI generator config and updates all
import references across tests, docs, and the client generation script.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 10:35:13 +01:00
Derek Bouius
917893aac7
fix: restore entity retrieval in recall (#391)
Entity retrieval was removed in ab5e31f2 ("chore: remove dead code")
but the code was not dead — it populated the entities dict and
per-fact entity names returned by the recall endpoint.

This restores:
- fact_entity_map query joining unit_entities and entities tables
- entity_names on each MemoryFact result
- entities_dict with EntityState objects ordered by fact relevance
- entity count in recall log line
2026-02-18 10:33:58 +01:00
Nicolò Boschi
224b7b74c1
feat: accept pdf, images and office files (#390)
* feat: accept pdf, images and office files

* refactor: rename FileConverter to FileParser, simplify file retain API

- Rename engine/converters/ → engine/parsers/, FileConverter → FileParser,
  ConverterRegistry → FileParserRegistry, MarkitdownConverter → MarkitdownParser
- Rename env var HINDSIGHT_API_FILE_CONVERTER → HINDSIGHT_API_FILE_PARSER
- Remove async/document_tags params from FileRetainRequest (always async now)
- Add retain_files() to Python Hindsight client and retainFiles() to TypeScript client
- Add sample.pdf to doc examples for working file upload demonstrations
- Update test_file_retain.py to use new parser names and always-async behavior
- Fix Go client missing os import in api_files.go
- Simplify postgresql.py storage to minimal schema

* fix: update rust CLI tests to use is_supported_file instead of is_text_file

* fix: patch Go api_files.go to add missing 'os' import after generation

* fix: insert 'os' import after 'net/url' in api_files.go patch for correct position

* chore: regenerate OpenAPI spec and clients (converter→parser description update)
2026-02-17 18:15:03 +01:00
Chris Bartholomew
8114ef440e
fix(python-client): async method parity and server keepalive timeout (#387)
* Fix async method parity and server keepalive timeout

The Python client's async methods were missing parameters available in
their sync counterparts, and the server's default keepalive timeout was
shorter than the client's, causing ServerDisconnectedError on reused
connections.

Server:
- Set uvicorn timeout_keep_alive to 30s (default was 5s). The Python
  client (aiohttp) has a 15s client-side keepalive, so the server must
  hold connections longer to prevent the client from writing to a
  closed socket.

Python client - async method parity:
- arecall(): add trace, query_timestamp, include_entities,
  include_chunks, max_entity_tokens, max_chunk_tokens. Return
  RecallResponse instead of list[RecallResult].
- areflect(): add max_tokens and response_schema.
- acreate_bank(): new async method.
- aset_mission(): new async method.
- adelete_bank(): new async method.

Tests:
- Add test verifying uvicorn keepalive timeout exceeds client default.
- Add async tests for arecall (include_chunks, include_entities, trace,
  full params), areflect (max_tokens, structured output), and
  adelete_bank.

* Fix flaky tag tests by using entity-rich content and asserting on tags

The tag tests were unreliable because:
- Generic content ("Project X meeting notes") was frequently collapsed
  during fact extraction, leaving no memories to recall
- Assertions checked LLM-rewritten text for literal substrings instead
  of checking tags, which is what the tests are actually verifying

Fix: use distinctive, entity-rich content (named people with specific
actions) that reliably survives fact extraction, and assert on tag
membership rather than text content.
2026-02-17 16:00:14 +01:00
Nicolò Boschi
6bad667344
fix(openclaw): error E2BIG on large content ingested (#389) 2026-02-17 14:11:23 +01:00
Nicolò Boschi
b3f0205ead
doc: add faq page (#383) 2026-02-17 11:26:27 +01:00
Nicolò Boschi
476726c2a2
feat: support azure pg_diskann (#381) 2026-02-16 16:08:54 +01:00
Nicolò Boschi
970f1b3534
ci: ensure docs get created with no extracted facts (#379) 2026-02-16 14:43:32 +01:00
Nicolò Boschi
5883e5af2d
doc: add go client examples (#380)
* doc: add go client examples

* doc: add go client examples
2026-02-16 14:43:03 +01:00
Nicolò Boschi
95c4220477
feat: support for pgvectorscale (DiskANN) (#378)
* feat: support for pgvectorscale (DiskANN)

* feat: support for pgvectorscale (DiskANN)
2026-02-16 14:19:56 +01:00
Nicolò Boschi
6e30980add
feat: use official go generator for Go client (#377)
* ci: add Go client integration tests

Add test-go-client job to CI workflow following the same pattern as
Python, TypeScript, and Rust client tests. The job:
- Sets up Go 1.23 with dependency caching
- Starts the Hindsight API server
- Runs integration tests using the 'integration' build tag
- Displays server logs on failure

The integration tests (hindsight-clients/go/integration_test.go) cover
all core operations: retain, recall, reflect, bank management, and
end-to-end workflows.

* Move Go cookbook content to hindsight-cookbook repo

Removes Go-specific cookbook content that was added in PR #375:
- applications/go-memory-service.md
- recipes/go-quickstart.md
- recipes/go-concurrent-pipeline.md

These have been moved to the hindsight-cookbook repository where
cookbook content should live per project conventions.

* feat(go): add CI test for Go client and patch for ogen null handling

- Add test-go-client job to GitHub Actions CI workflow
- Create post-generation patch script (patch-ogen.sh) to fix ogen's
  handling of null values in optional string fields
- Patch OptString.Decode() to check jx.Next() type before decoding,
  properly handling explicit null in JSON responses

The patch ensures generated code persists across regenerations and
handles the Hindsight API's nullable optional fields correctly.

Fixes: Go client integration tests for retain and bank operations
Note: Some tests still fail for nullable arrays/objects - those
require additional patches for other Opt* types.

* feat: use official go generator for Go client

* feat: use official go generator for Go client

* ci fixes

* chore: sync Go client with latest OpenAPI spec

- Add model_child_operation_status.go (new model)
- Update model_operation_status_response.go with child operations
- Update go.mod/go.sum dependencies
- Update api/openapi.yaml
2026-02-16 14:04:12 +01:00
Nicolò Boschi
40d42c58aa
feat: support Batch API for retain (openai/groq) (#365)
* feat: support Batch API for retain (openai/groq)

* api

* stop batch api if sync

* fix(ui): improve toast notifications with brand colors and proper styling

- Replace all window.alert() calls with toast notifications
- Add interceptor-based error handling in API client
- Use different toast styles based on HTTP status codes (4xx = warning, 5xx = error)
- Apply Hindsight brand colors to toasts (primary blue for info, destructive red for errors, etc.)
- Remove obsolete error handling files (hindsight-client-with-toast.ts, api-error-handler.ts)
- Fix toast background conflicts by removing base bg-background class

* fix: restore retain_batch_tokens config that was accidentally removed during rebase
2026-02-16 13:31:50 +01:00
Nicolò Boschi
aefb3fcf4d
fix: improve async batch retain with large payloads (#366)
* fix: improve async batch retain with large payloads

* fix: improve async batch retain with large payloads

* api

* api

* api

* api

* api

* Clean up perf benchmark: keep only Python files

- Remove README.md and PERFORMANCE_FINDINGS.md
- Remove results/ JSON files (gitignored)
- Remove test_data/ directory
- Keep only __init__.py and retain_perf.py

* docs: explain automatic batch optimization for async retain

- Add section explaining Hindsight automatically handles batch sizing
- Users don't need to manually tune batch sizes with async mode
- Hindsight splits large batches (>10k tokens) into optimized sub-batches
- Include example showing best practices

* docs: remove emojis and code example from performance page

* fix: correct OperationDetails type to match API response

- Change optional fields to use | null instead of ?
- Fixes TypeScript compilation error in control plane build

* fix: use discriminated union for OperationDetails type

- Support both success and error states properly
- Fixes TypeScript error when setting error state

* fix: use unique document_ids in batch retain examples

- Each item in a batch must have unique document_id
- Update both Python and JavaScript examples
- Fixes test-doc-examples CI failure

* chore: trigger CI

* fix: test mocking and duplicate document_ids in examples

- Mock _get_pool() in test_async_retain_tags.py to avoid _initialized error
- Set _initialized = True on mocked MemoryEngine instances
- Fix duplicate document_ids in retain.py and retain.mjs examples

* fix: properly mock async pool/connection and fix more duplicate document_ids

- Use AsyncMock for pool.acquire() to fix 'can't be used in await' error
- Fix duplicate document_ids in retain-async examples (retain.py and retain.mjs)
- Remove batch-level document_id parameter that caused duplicates

* ci: collect all doc example failures and show summary

- Run all Python/Node.js/CLI examples regardless of individual failures
- Collect failure list and display summary at the end
- Show pass/fail count and list of failed files
- Exit with failure only after running all examples

* refactor: extract doc example testing to standalone script

- Create scripts/test-doc-examples.sh to run all examples
- Collects logs of failed examples separately
- Shows full error logs only for failures at the end
- Clean summary with pass/fail counts
- Proper exit codes
- Replaces inline bash in CI workflow

* fix: doc examples - duplicate document_ids and error handling

- retain.py: move document_id to item level to avoid duplicates
- documents.mjs: add error handling for getDocument to show clear error message

* fix: update tests for duplicate document_id validation

- test_async_retain_tags: verify operation structure instead of exact UUID
- test_delete_bank: use unique document_ids (team-doc-1, team-doc-2)
2026-02-16 12:51:42 +01:00
Eliah Rusin
2a47389f2c
feat: add Go client SDK with ogen code generation (#375)
Add a Go client for the Hindsight API using ogen for strongly-typed code
generation from the OpenAPI 3.1 spec. The client provides a high-level
wrapper with functional options around the generated code, covering all
core operations (retain, recall, reflect, bank management).

Includes:
- ogen-based code generation with OpenAPI 3.1 spec preprocessing
- High-level Client wrapper with idiomatic Go API
- Functional options for all operations (WithBudget, WithTags, etc.)
- OgenClient() escape hatch for advanced operations
- Integration tests and godoc examples
- Go SDK reference docs and cookbook entries (quickstart, concurrent
  pipeline, memory-augmented API service)
- Updated generate-clients.sh with Go generation step

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 11:35:40 +01:00
abix5
b4b5c44a87
fix: propagate document tags in async retain path (#374) 2026-02-16 10:04:48 +01:00
Anton Evseev
d5e62162e8
fix(openclaw): remove unused imports, retry health check, suppress unhandled rejection (#373)
- Remove unused `fs` and `execSync` imports from `embed-manager.ts`
- Remove unused `join` import from `index.ts`
- Add retry logic to external API health check (3 attempts, 2s delay) —
  container DNS may not be ready on first boot
- Use ES2022 `{ cause: error }` for better error chain preservation
- Add `.catch(() => {})` to `initPromise` to suppress Node.js unhandled
  rejection warnings (error is properly handled later in `service.start()`)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 10:02:19 +01:00
Nicolò Boschi
7dad9da02d
feat: allow chunks-only in recall (max_tokens=0) (#364)
* feat: allow chunks only in recall

* feat: fetch chunks independently of max_tokens filtering

Changes:
- Chunks now fetched BEFORE max_tokens filtering (Step 5.5)
- Implements batching: (max_chunk_tokens / retain_chunk_size) * 2
- Loop-based fetching until budget exhausted or no more chunks
- Handles varying chunk sizes across documents
- When max_tokens=0: returns 0 facts but still returns chunks
- When max_tokens>0: backward compatible (chunks match filtered facts)

Tests:
- Added test_recall_chunks_independence.py with 5 comprehensive tests
- Tests chunk independence, batching, ordering, and backward compat

Docs:
- Updated recall.mdx to explain new chunk behavior
- Updated memory_engine.py docstrings

Fixes chunk-related test failures by reordering chunks to match
filtered facts when max_tokens > 0 (backward compatibility).

* fix: fetch chunks after token filtering when max_tokens>0

Changes:
- When max_tokens=0: fetch chunks BEFORE token filtering (new behavior)
- When max_tokens>0: fetch chunks AFTER token filtering (backward compat)
- This ensures chunk ordering matches filtered facts for max_tokens>0
- Fixes test failures in test_chunks_and_entities_follow_fact_order,
  test_chunk_fact_mapping, test_chunk_ordering_preservation, etc.

The previous approach tried to reorder prefetched chunks, but that
caused issues when the chunk budget was exhausted before all facts
were processed. The new approach fetches chunks based on the correct
fact set for each scenario.

* fix: use ConfigResolver for bank-specific retain_chunk_size

Fixes error: Field 'retain_chunk_size' is bank-configurable and cannot
be accessed from global config.

Changed from:
- config.retain_chunk_size (global config, not allowed)

To:
- bank_config.retain_chunk_size (resolved from ConfigResolver)

This ensures the correct chunk size is used for each bank, respecting
any bank-specific overrides.

* fix: correct Budget import in test_recall_chunks_independence

Changed from:
- from hindsight_api.engine.interface import Budget (incorrect)

To:
- from hindsight_api.engine.memory_engine import Budget (correct)

This fixes the ImportError that was preventing the tests from running.

* fix: prevent infinite loop in chunk fetching and improve test content

- Add max(1, ...) to estimated_batch_size to prevent division resulting in 0
- Update test content to use more substantial examples that generate facts
- Add request_context parameter to all retain_async and recall_async test calls

* refactor: simplify chunk fetching to always use pre-filtering approach

Remove backward compatibility code that fetched chunks after token
filtering. Now chunks are always fetched from top-scored results
before max_tokens filtering, regardless of max_tokens value.

This simplifies the code by:
- Removing duplicate chunk fetching logic
- Eliminating conditional behavior based on max_tokens
- Making chunk fetching behavior consistent and predictable

Chunks are still fetched in batches and respect max_chunk_tokens limit.
2026-02-13 16:56:35 +01:00
Nicolò Boschi
ff55283018
doc: changelog and blog post for 0.4.11 (#363)
* doc: changelog and blog post for 0.4.11

* doc: changelog and blog post for 0.4.11
2026-02-13 11:45:41 +01:00
Nicolò Boschi
b3b541fc53 Release v0.4.11
- Update version to 0.4.11 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-litellm, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- OpenClaw integration: hindsight-integrations/openclaw
- AI SDK integration: hindsight-integrations/ai-sdk
- Helm chart
- Sync documentation to version-0.4
2026-02-13 10:52:14 +01:00
Nicolò Boschi
4f112101ac
fix(openclaw): avoid memory retain recursion (#362) 2026-02-13 10:47:39 +01:00
Nicolò Boschi
e408b7e072
feat: support litellm-sdk as reranker and embeddings (#357)
* feat: support litellm-sdk for reranker endpoint

* feat: support litellm-sdk for reranker endpoint

* fix: make litellm SDK cohere test fixture async function-scoped

* fix: store litellm module reference during initialization to avoid import issues

* feat: add LiteLLM SDK embeddings support

- Add LiteLLMSDKEmbeddings class for direct API access without proxy
- Support multiple providers: Cohere, OpenAI, Together AI, HuggingFace, Voyage AI
- Automatic dimension detection via test embedding
- Provider-specific API key mapping
- Batch processing support (configurable batch size)
- Comprehensive test coverage (17 unit tests)
- Update documentation with configuration examples

Implements embeddings in same PR as reranker per user request

* fix: correct config mocking in embeddings factory tests

- Mock get_config() from its source module (hindsight_api.config)
- Fixes factory tests that were returning LocalSTEmbeddings instead of LiteLLMSDKEmbeddings
- All 17 unit tests now passing

* fix: skip Cohere integration tests when API key is invalid

- Catch initialization errors and skip tests instead of failing
- Prevents CI failures when COHERE_API_KEY is set but invalid
- Integration tests now properly skip when authentication fails

* fix: skip Cohere reranker integration tests when API key is invalid

- Add same error handling as embeddings tests
- Prevents CI failures when COHERE_API_KEY is set but invalid
- Tests now properly skip when authentication fails

* Revert "fix: skip Cohere reranker integration tests when API key is invalid"

This reverts commit 655dacaffb25851ff48e202b4379fc8332a66df7.

* Revert "fix: skip Cohere integration tests when API key is invalid"

This reverts commit 5d00548e39faa3da6b427ace89589a216816f9e7.

* fix: pass API key directly to litellm SDK functions

- Add api_key parameter to arerank(), rerank(), aembedding(), and embedding() calls
- Prevents authentication issues in multi-process environments (pytest-xdist)
- More reliable than relying solely on environment variables
- Update test assertions to expect api_key parameter

* feat: pass api_base parameter to litellm SDK calls and remove hasattr check

* fix: raise errors instead of silently returning 0.0 scores

* refactor: pass API keys directly in kwargs instead of setting env vars
2026-02-12 23:53:12 +01:00
Nicolò Boschi
d871c3009d
feat: support timescale pg_textsearch as text search extension (#359)
* feat: support timescale pg_textsearch as text search extension

* refactor: deduplicate text search query in retrieve_semantic_bm25_combined

Instead of maintaining 3 complete query copies (native, vchord, pg_textsearch),
now we:
- Build backend-specific parts (score_expr, order_by, where_filter)
- Use a single query template with injected backend-specific parts

This makes maintenance easier - changes to the semantic CTE or overall structure
only need to be made once.
2026-02-12 17:38:13 +01:00
DK09876
d8376ecf6b
Fix incorrect MCP tool parameters in docs (#358)
- Remove phantom `max_results` param from recall (only has query + max_tokens)
- Remove `budget` param from local-mcp recall (only reflect has budget)
- Add missing `name` and `mission` optional params to create_bank
- Add missing `mental_model_id` optional param to create_mental_model

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 08:52:05 -07:00
Nicolò Boschi
71e408c27b
chore: remove dead code (#356)
* chore: remove dead code

* chore: remove dead code

* feat: support litellm-sdk for reranker endpoint
2026-02-12 14:44:00 +01:00
Nicolò Boschi
c029807add
feat: support for other text and vector search pg extensions (#355)
* feat: support for other text and vector search pg extensions

* test: increase timeout for test_batch_chunking_behavior to account for VectorChord BM25 tokenization overhead

* feat: support for other text and vector search pg extensions
2026-02-12 14:13:04 +01:00
Nicolò Boschi
8d731f2e5f
feat: implement hierarchical configuration (system, tenant, bank) (#329)
* feat: implement hierarchical configuration (system, tenant, bank)

* feat: implement hierarchical configuration (system, tenant, bank)

* docs: add instructions for hierarchical config in CLAUDE.md

* feat: add ENABLE_BANK_CONFIG_API flag (disabled by default)

- Add HINDSIGHT_API_ENABLE_BANK_CONFIG_API env var (default: false)
- Return 403 Forbidden from bank config endpoints when disabled
- Update tests to enable the flag
- Update CLAUDE.md documentation

This provides security control over the bank configuration API,
ensuring it's only accessible when explicitly enabled.

* docs: add hierarchical configuration section

* feat(cli): add bank config commands (config, set-config, reset-config)

- Add 'hindsight bank config' to view bank configuration
- Add 'hindsight bank set-config' to update LLM settings per bank
- Add 'hindsight bank reset-config' to reset to defaults
- Implements client API calls to new bank config endpoints

* fix(cli): fix compilation errors in bank config commands

- Fix type signature: use ApiClient instead of api::Client
- Fix confirmation: use ui::prompt_confirmation instead of ui::confirm
- Fix error handling: use anyhow! macro instead of errors::Error
- Fix type conversion: convert HashMap to serde_json::Map for API call

* feat: implement type-safe hierarchical config with bank overrides

Implements a production-ready hierarchical configuration system that prevents
accidentally using global defaults when bank-specific overrides exist.

- Created StaticConfigProxy that wraps HindsightConfig
- get_config() now returns proxy that blocks access to bank-configurable fields
- Raises ConfigFieldAccessError with clear message when accessing configurable fields
- Added _get_raw_config() for internal use only
- Forces developers to use resolve_full_config(bank_id, context) for bank settings

- Added resolve_full_config() method that returns complete HindsightConfig
- Resolves hierarchy: Global (env) → Tenant → Bank
- No caching to support multi-server deployments (always fresh from DB)
- LLM provider pooling handles expensive operations separately

- Updated entire retain pipeline to pass resolved config through call chain
- memory_engine.py: Resolves config at top level where bank_id/context available
- orchestrator.py: Accepts and passes config to fact_extraction
- fact_extraction.py: Uses passed config instead of get_config()
- utils.py: Added optional config param for backward compatibility

- consolidator.py: Uses resolve_full_config() for enable_observations check
- memory_engine.py: Resolves config before triggering consolidation

- Renamed "Memory Bank" to "Bank Configuration" with tabs
- Combined Stats and Operations into "General" tab
- Consolidated Profile and Configuration into "Configuration" tab
- Moved Actions dropdown to page level (outside tabs)

- Created new component for managing bank-specific config
- Displays configurable fields: retain_chunk_size, retain_extraction_mode, etc.
- Edit via dialog with form validation
- Reset to defaults via AlertDialog confirmation
- Shows field IDs in monospace for clarity
- Visual separation with borders and hover effects

- Removed inline edit mode, switched to dialog-based editing
- Separate dialogs for Disposition and Mission editing
- Read-only display with clear edit buttons
- Removed duplicate stats cards and operations

- bank-stats-view.tsx: Overview statistics (memories, links, documents, pending ops)
- bank-operations-view.tsx: Background operations table with filtering

**Problem**: Consolidation always used global enable_observations, ignoring bank overrides
**Root Cause**: consolidator.py called get_config() instead of resolving bank-specific config
**Solution**: Pass resolved config through the entire pipeline

**Problem**: asyncpg returning JSONB as JSON string instead of parsed dict
**Solution**: Explicit JSON parsing in config_resolver.py with type checking

- All 19 API integration tests pass
- All 10 hierarchical config tests pass
- Retain operations work correctly with bank-specific config
- Consolidation respects bank-specific enable_observations setting

- Updated developer/configuration.md with type-safe config access pattern
- Added examples showing correct usage patterns
- Documented ConfigFieldAccessError and resolution methods

- get_config() now returns StaticConfigProxy (blocks configurable field access)
- Code accessing bank-configurable fields must use resolve_full_config()
- Clear migration path with helpful error messages

Fixes hierarchical configuration to be production-ready with proper type safety.

* refactor: remove LLM client pool and simplify config resolver

Since LLM config (provider, model, api_key) is now static and not
bank-configurable, the LLMClientPool is no longer needed.

Changes:
- Remove hindsight_api/llm_client_pool.py (no longer needed)
- Remove memory_engine._get_bank_llm_config() (dead code, never called)
- Simplify config_resolver.py by eliminating duplication between
  resolve_full_config() and get_bank_config()
- get_bank_config() now calls resolve_full_config() and filters results
- Remove outdated "LLM provider pooling" comments from docstrings

All tests pass (10 hierarchical config tests, 19 API integration tests)

* fix: update tests to use _get_raw_config() for configurable fields

Fixed test fixtures that were accessing configurable fields (like
enable_observations) from get_config(), which now raises
ConfigFieldAccessError due to type-safe config access.

Changes:
- test_consolidation.py: Changed enable_observations fixture to use
  _get_raw_config() instead of get_config()
- test_consolidation.py: Updated test_consolidation_returns_disabled_status
  to set bank config instead of mocking get_config()
- test_link_expansion_retrieval.py: Changed fixture to use _get_raw_config()
- test_observations.py: Changed disable_observations fixture to use
  _get_raw_config()
- Regenerated OpenAPI spec and clients

All 39 previously failing tests now pass.

* fix: add missing config parameter to test calls of extract_facts_from_text()

Fixed 45 test failures where tests were calling extract_facts_from_text()
without the new required config parameter.

Changes:
- Added config=_get_raw_config() to all extract_facts_from_text() calls
- Fixed test_main_module.py to patch _get_raw_config instead of get_config
- Updated 6 test files with 37 function call sites

All tests should now pass.

* fix: add missing config parameter to test_skip_podcast_meta_commentary

One more test was missing the config parameter for extract_facts_from_text().
2026-02-12 13:14:57 +01:00
Nicolò Boschi
f9a8a8e01e
fix: resolve based_on schema/serialization issues in reflect API (#348)
* fix: add default values to OpenAPI schema for default_factory fields

This commit fixes the OpenAPI schema to include default values for fields
using default_factory, which improves schema accuracy and client generation.

Changes:
1. Added FieldWithDefault() helper to inject default values into OpenAPI schema
2. Updated 14 fields using default_factory to include defaults in schema:
   - ReflectBasedOn.{memories, mental_models, directives}
   - ReflectTrace.{tool_calls, llm_calls}
   - All tags fields
   - All trigger fields
   - All include fields

3. Regenerated OpenAPI spec with proper defaults

4. Added tests to verify API returns correct format with empty banks

Note: This fixes the schema but doesn't change the v0.3.0 -> v0.4.0 breaking
change where based_on went from list to object. Clients should handle both
formats for backward compatibility.

* fix: remove client imports from API test

The test was failing in CI because it imported the client library
which isn't installed in the API test environment.

Changed to test only API JSON response format, not client parsing.
This is more appropriate for an API test anyway.

* test: add client tests for ReflectResponse parsing

Added comprehensive tests in hindsight-clients/python/tests to verify:
- v0.4.0+ format with empty based_on object
- v0.4.0+ format with null based_on
- v0.4.0+ format with populated facts
- v0.3.0 format (list) correctly fails validation
- Missing based_on field handling

These tests document the v0.3.0 -> v0.4.0 breaking change where
based_on changed from list to object.
2026-02-12 11:26:12 +01:00
Damien
a713b68b1f
Implement the vchord / pgvector support (#350)
* Implement the vchord / pgvector support

* feat(alembic): detect vector extension and create appropriate index
2026-02-12 10:47:07 +01:00
Nicolò Boschi
93ddd41621
feat: add reverse proxy support (#346)
* feat: add reverse proxy support

* improve

* improve

* improve

* improve

* improve

* fix: update integration test to use modern 'docker compose' command

- Replace 'docker-compose' with 'docker compose' (Docker Compose v2+)
- Add fallback to legacy docker-compose command for compatibility
- Fixes test failures on systems using Docker Compose plugin

* ci: trigger test rerun

* fix: make docker-compose detection more robust for CI

- Add get_docker_compose_command() to detect available command
- Use shutil.which() to check command availability
- Dynamically use correct command (docker compose vs docker-compose)
- Should work in both modern and legacy Docker environments

* fix: docker-compose networking in base path integration test

Fix connection refused error in test_reverse_proxy_simple_config by
handling host vs bridge networking modes correctly:

- Linux (host mode): nginx listens on 18080 directly, no port mapping
- Mac/Windows (bridge mode): nginx listens on 80, mapped to 18080

With host networking, port mappings in docker-compose don't work since
the container binds directly to the host's network namespace.
2026-02-12 10:09:53 +01:00
DK09876
7ee229ba23
Fix MCP extra args rejection and bank ID resolution priority (#351)
* Fix MCP extra args rejection and bank ID resolution priority

Two fixes to the MCP middleware:

1. Strip unknown tool arguments: LLMs frequently add extra fields
   like "explanation" to tool calls. FastMCP's Pydantic TypeAdapter
   rejects these with "Unexpected keyword argument". The middleware
   now intercepts tools/call requests and removes unknown fields
   before they reach validation.

2. Bank ID resolution priority: Path now takes priority over header.
   Previously X-Bank-Id header was checked first, meaning /mcp/my-bank/
   with X-Bank-Id: other-bank would silently use other-bank in multi-bank
   mode. Now the URL path is authoritative — single-bank mode connections
   cannot be overridden by headers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update MCP server docs with mental model tools and fixes

- Add all mental model tools (create, list, get, update, delete, refresh)
- Add list_banks and create_bank tool docs
- Document single-bank vs multi-bank modes
- Fix bank selection priority: path > header > default
- Add Accept header to curl example
- Add timestamp param to retain, max_tokens to recall

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 10:09:28 +01:00
Chris Latimer
29c0890f22 Copy page button on docs 2026-02-11 19:21:24 -07:00
Chris Bartholomew
a1f22dabd2
Replace waitlist links with direct Hindsight Cloud signup URL (#349)
The waitlist is no longer needed. Update all references from
vectorize.io/hindsight/cloud to ui.hindsight.vectorize.io/signup
and change "request early access" language to "sign up".
2026-02-11 20:51:32 +01:00
Nicolò Boschi
60574ee08f
fix: add trust_code env config (#347)
* fix: add trust_code env config

* doc
2026-02-11 17:06:59 +01:00