* feat: add built-in llama.cpp LLM provider for fully local inference
Add `llamacpp` as a new LLM provider that manages a llama-cpp-python server
subprocess. Auto-downloads Gemma 4 E2B Q4_K_M (~3.5 GB) on first use and
runs inference locally via Metal/CUDA with no external services needed.
- New provider: `HINDSIGHT_API_LLM_PROVIDER=llamacpp`
- Singleton server shared across retain/reflect/consolidation
- Configurable: model path, GPU layers, context size, grammar enforcement
- User-extensible via `HINDSIGHT_API_LLAMACPP_EXTRA_ARGS`
- Flash attention + prompt caching enabled by default
- LLM provider cleanup on shutdown (stops subprocess)
- hindsight-embed: `--ui` flag on `daemon start`, removed FORCE_CPU on macOS
- Docs: configuration.md, models.mdx, providers grid updated
* chore: regenerate docs skill and update lockfile for local-llm dep
* docs: add best practice for filtering recall by memory shape (#856)
Add guidance on using entity labels with `tag: true` to deterministically
filter recall results when a bank contains different memory shapes
(e.g., concise rules vs. detailed procedures).
* feat: add OpenRouter support for LLM, embeddings, and reranking
OpenRouter is OpenAI-compatible for chat/embeddings and Cohere-compatible
for reranking, so no new provider classes are needed.
- LLM: added as OpenAICompatibleLLM provider (default model: qwen/qwen3.5-9b)
- Embeddings: reuses OpenAIEmbeddings with OpenRouter base URL (default: perplexity/pplx-embed-v1-0.6b)
- Reranker: reuses CohereCrossEncoder with OpenRouter rerank endpoint (default: cohere/rerank-v3.5)
- API key fallback chain: dedicated key → shared OPENROUTER_API_KEY → LLM_API_KEY
* chore: regenerate docs skill references and fix formatting
* fix: make LiteLLM SDK embeddings encoding_format configurable (#925)
The hardcoded encoding_format='float' breaks providers like Voyage AI
(only accepts 'base64') and Gemini (doesn't support the parameter at all).
Add HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_ENCODING_FORMAT config option
that defaults to 'float' for backwards compatibility. Set to empty string
to omit the parameter for incompatible providers.
* chore: regenerate docs skill after configuration change
* fix(recall): cap entity fanout in graph expansion to prevent slow queries
On large banks, the entity co-occurrence self-join in _expand_combined()
produces massive intermediate row counts when seeds reference high-fanout
entities (e.g. an entity with 25K+ mentions). This causes recall latency
to degrade significantly.
Changes:
- Replace unbounded entity self-join with LATERAL per-entity cap
(graph_per_entity_limit, default 200), reducing intermediate rows
from potentially millions to at most num_entities * 200
- Add ORDER BY unit_id DESC in LATERAL subquery for deterministic
recency-biased sampling (rides the PK index, no extra sort)
- Add timeout fallback (graph_expansion_timeout, default 10s) that
drops entity expansion and falls back to semantic+causal only
- Add composite index (entity_id, unit_id) on unit_entities for
index-only scans in the LATERAL subquery
- Merge 3 unmerged migration heads into one
- Fix recall_perf.py dotenv override issue
Unlike the approach in #895, this does NOT filter out hub entities
entirely — all entities are kept but capped equally, preserving
retrieval quality for queries about frequently-mentioned entities.
Benchmarked on a 67K-unit bank (top entity = 25K mentions):
- retrieval_graph: 0.337s → 0.055s (84% faster)
- end-to-end recall: 0.912s → 0.519s (43% faster)
* fix(tests): fix broken test_combined_scoring and test_reranking_proof_count
- test_combined_scoring: replace MagicMock(spec=RetrievalResult) with real
dataclass instances — MagicMock attributes returned nested mocks that
failed on >= comparisons with int
- test_reranking_proof_count: remove deleted `embedding` param from
RetrievalResult constructor, use None for occurred_start/end to get
neutral recency (datetime.now gave recency=1.0 which boosted scores)
* refactor: rename config to link_expansion_ prefix, fix observation fanout
- Rename GRAPH_PER_ENTITY_LIMIT → LINK_EXPANSION_PER_ENTITY_LIMIT and
GRAPH_EXPANSION_TIMEOUT → LINK_EXPANSION_TIMEOUT to follow the
convention that these are specific to the link_expansion graph retriever
- Apply the same LATERAL per-entity cap to _expand_observations(), which
had the same unbounded self-join through unit_entities
* style: fix formatting in config.py
* fix: make bank_id metric label opt-in to prevent OTel memory leak
bank_id as an OTel metric attribute creates unbounded histogram growth
since each unique bank_id produces never-evicted time series. Default
to excluding it; opt in with HINDSIGHT_API_METRICS_INCLUDE_BANK_ID=true
for deployments with few banks.
Closes#850
* refactor: use config.py for metrics_include_bank_id setting
Move HINDSIGHT_API_METRICS_INCLUDE_BANK_ID from direct os.getenv in
metrics.py to the standard HindsightConfig path. Add configuration
documentation.
* perf: 3-phase retain pipeline — fix deadlocks, cap temporal links, query-time entity expansion
Major retain pipeline overhaul addressing deadlocks, write amplification,
and TimeoutErrors. Restructures retain into three phases:
Phase 1: Entity resolution on separate connection (read-heavy)
Phase 2: Core write transaction (atomic) — facts, unit_entities, links
Phase 3: Best-effort display data (error-isolated) — entity viz links, stats
Key changes:
- Sorted bulk INSERT FROM unnest() prevents deadlocks
- Temporal links capped to top-20 per unit (95% reduction)
- Batched semantic ANN via temp table + LATERAL
- Query-time entity expansion via unit_entities self-join
- Entity viz links moved to Phase 3 (post-transaction)
- HINDSIGHT_API_RETAIN_MAX_CONCURRENT config (default: 32)
* fix: increase semantic link top_k from 5 to 20
The hardcoded top_k=5 was artificially limiting semantic link creation.
Link expansion retrieval can consume up to budget (50-200) semantic
neighbors per seed set, but each fact only had 5 outgoing edges — making
the bidirectional graph very sparse.
Increasing to 20 gives retrieval 4x more edges to work with. The ANN
probe cost is unchanged (same HNSW traversal per fact, just returning
more rows). INSERT cost is negligible (~14k rows via bulk INSERT).
Also: all 18 TimeoutErrors in the latest benchmark (beam-1m-u20) were
from Gemini LLM calls, zero from the database — confirming the entity
resolution split eliminated DB timeouts entirely.
* perf: move semantic ANN search to Phase 1 to avoid transaction timeouts
The batched LATERAL ANN query (700 HNSW probes) was the last remaining
source of DB TimeoutErrors — all 29 in the latest benchmark were from
create_semantic_links_batch inside the Phase 2 write transaction.
Split semantic link creation into three phases:
- Phase 1 (separate conn, autocommit): ANN search via temp table + LATERAL.
No transaction locks, no contention with concurrent writers.
- Phase 2 (write transaction): within-batch numpy similarities (instant) +
INSERT of both within-batch and Phase 1 ANN results. No DB reads.
- Phase 3 (flush_pending_stats): future hook point for re-checking ANN
results after commit to catch links missed by concurrent batches.
Also adds 7 unit tests for compute_semantic_links_within_batch covering
empty input, identical/orthogonal embeddings, threshold filtering, top_k
cap, and tuple structure validation.
* fix: handle placeholder unit_ids in Phase 1 ANN search (not valid UUIDs)
* test: add Phase 1 ANN cross-batch test + configurable test PG port
- New test_semantic_links_phase1_ann_cross_batch verifies that the Phase 1
ANN search with placeholder unit IDs correctly creates cross-batch
semantic links after remapping to real IDs.
- Test PG port now configurable via HINDSIGHT_TEST_PG_PORT env var
(default: 5556) to avoid conflicts with running benchmark daemons.
* perf: remove retry_with_backoff from retain, set semaphore default to 4
Remove retry_with_backoff from _run_db_work and _run_delta_db_work:
- Deadlocks are prevented by sorted bulk INSERT (no need for retry)
- Transient timeouts are handled by the worker poller's task-level retry
(3 attempts, 60s spacing) which is better than rapid internal retries
that amplify I/O pressure during contention storms
Set HINDSIGHT_API_RETAIN_MAX_CONCURRENT default from 32 to 4:
- The semaphore gates Phase 1 (ANN + entity resolution) + Phase 2 (writes)
- At 4 concurrent, HNSW index I/O is manageable; at 10+ concurrent the
probes saturate disk and cause cascading timeouts
- LLM extraction still runs at full parallelism (semaphore acquired after)
* fix: add fact_type filter to Phase 1 ANN query to use per-bank HNSW indexes
The LATERAL ANN query was falling back to sequential scan + sort (90ms/probe)
because the per-bank HNSW indexes are partial indexes filtered on fact_type.
Without fact_type in the WHERE clause, PostgreSQL couldn't use them.
Fix: iterate over ('world', 'experience') and run one HNSW-indexed ANN per
type. EXPLAIN shows 8ms/probe (was 90ms) — 11x faster.
700 probes × 8ms × 2 types = ~11s total (was ~63s via seq scan).
* fix: scope temporal links by fact_type + add integration tests
Temporal links now filter by fact_type in the LATERAL query — world facts
only link to world facts, experience to experience. This matches how
retrieval filters results and avoids wasted cross-type link rows.
New integration tests:
- test_semantic_ann_uses_hnsw_index: verifies Phase 1 ANN creates
cross-batch semantic links (tests fact_type filter + placeholder remap)
- test_temporal_links_scoped_by_fact_type: verifies world facts get
temporal links to other world facts but NOT to experience facts
* fix: tolerate individual chunk LLM failures instead of failing entire batch
Changed asyncio.gather(*tasks) to asyncio.gather(*tasks, return_exceptions=True)
in both chunk-level and content-level fact extraction. A single chunk timeout
(e.g., Gemini >90s) no longer discards all other successfully extracted facts.
For a 50MB document with 17k chunks, even a 2% chunk failure rate previously
caused 0 completions (entire batch discarded). Now 16,700 facts are extracted
and only the 300 failed chunks are skipped with a warning log.
* fix: batch temporal LATERAL query for large documents (16k+ chunks)
The LATERAL query for temporal links passed all unit_ids at once into
unnest(), causing PostgreSQL timeouts on documents with 16k+ chunks.
Split into batches of 500 units per query to keep each under the
command_timeout.
Also identified: HNSW index creation on shared pg0 instances with
50k+ existing units exceeds the 60s command_timeout. This is a
test infrastructure issue (shared pg0 accumulates data) but also
affects production when creating new banks on large instances.
* feat: streaming chunk batching for large documents (RETAIN_CHUNK_BATCH_SIZE)
Process chunks in mini-batches of N (default 500), committing each batch
to the DB before starting the next. This prevents OOM kills on large
documents (50MB / 17k+ chunks) by keeping only ~500 facts + embeddings
in memory at a time instead of 50k+.
Each mini-batch goes through the full Phase 1 → 2 → 3 pipeline
independently, sharing the same document_id. On recovery (process dies
mid-way), delta retain detects already-committed chunks via content_hash
and skips them — only remaining chunks get re-extracted.
Config: HINDSIGHT_API_RETAIN_CHUNK_BATCH_SIZE (default: 500, 0 to disable)
Per-bank configurable via the hierarchical config system.
Tests:
- test_streaming_chunk_batching_produces_same_facts
- test_streaming_chunk_batching_recovery (delta retain skips committed chunks)
- test_streaming_disabled_for_small_docs
* perf(retain): producer-consumer pipeline + deferred semantic ANN
Replace the sequential streaming loop with a producer-consumer pipeline:
- LLM producer fires concurrent chunk extractions (semaphore-bounded)
- DB consumer drains queue in batches, runs Phase 1+2+3 per batch
- LLM and DB work overlap instead of running sequentially
Defer semantic links to a single final ANN pass after all batches commit:
- Remove within-batch semantic links from Phase 2 (was 2.6s/batch)
- Run parallel ANN (4 connections) after all facts committed
- top_k reduced from 50 to 20 (recall uses at most 20 neighbors)
- Recovery via operation result_metadata checkpoint
Additional optimizations:
- skip_exists_check on temporal/causal link INSERT (saves ~0.5s/batch)
- WHERE EXISTS guard on semantic link INSERT (handles document upsert)
- timeout=300s on ANN queries and bulk INSERT for large banks
- Demote [ANN] debug logs to logger.debug()
- Fix docstring typos (agent_id → bank_id)
- Fix content_index remapping in producer-consumer batches
- Fix delta retain passing contents vs delta_contents
50MB benchmark (mock LLM): 9.2 min (was 23 min) — 2.5x faster.
BEAM 10m benchmark: zero deadlocks, zero DB errors.
* refactor(retain): remove legacy fallback code paths
- Remove process_entities_batch (legacy single-connection entity processing)
- Remove extract_entities_batch_optimized (only caller was the above)
- Remove fallback entity processing inside Phase 2 transaction
- Remove legacy ANN inline fallback in create_semantic_links_batch
- Remove fallback entity_links direct-insert path in Phase 3
- Make resolved_entity_ids/entity_to_unit/unit_to_entity_ids required params
* refactor(retain): replace tuple returns with dataclasses, remove dead code
- Add EntityResolutionResult and Phase1Result dataclasses in types.py
- Replace 4-tuple return from _pre_resolve_phase1 with Phase1Result
- Remove dead `entity_links = []` variables in retain_batch and _try_delta_retain
- Remove unused `confidence_score` parameter from orchestrator.retain_batch
and _retain_batch_async_internal (was accepted but never used)
* fix(entity-resolver): remove LIKE full-scan fallbacks, use index-only trigram matching
The entity resolution query had LIKE '%...' substring conditions that bypassed
the GIN trigram index, causing full sequential scans of the entities table.
On banks with 10k+ entities, this caused TimeoutErrors (observed in BEAM 10m).
Changes:
- Remove LIKE fallbacks, use trigram % operator only (GIN index-based)
- Lower similarity threshold from 0.3 to 0.15 to catch substring relationships
- Use LOWER() on both sides for case-insensitive matching
- Migration: recreate GIN trigram index on LOWER(canonical_name)
* fix: remove schema prefix from index names in trigram migration
* fix(delta-retain): use same chunk_size as streaming path (3000 vs 120000)
_chunk_contents_for_delta defaulted to chunk_size=120000 while the streaming
path used 3000. On retry, delta re-chunked the document with different
boundaries, found 0 matching chunks, and fell through to full re-extraction.
This wasted all LLM calls on already-committed chunks.
Fix: use the same default (3000) so chunk hashes match on recovery.
* fix(retain): persist generated document_id in operation metadata for retry recovery
When no document_id is provided, retain generates a UUID. On retry, a new UUID
was generated, making delta retain and streaming chunk-hash recovery unable to
find previously committed chunks. All LLM extraction was wasted on retry.
Fix: resolve document_id early in retain_batch (before delta), persist it to
operation result_metadata, and recover it on retry. Both delta and streaming
paths now see the same document_id across attempts.
* refactor(retain): unify into single streaming pipeline, remove non-streaming path
All retains now go through the producer-consumer streaming pipeline,
regardless of document size. Small documents are processed as a single batch.
This eliminates the maintenance burden of two separate code paths.
Also fix document upsert: compare content hash to distinguish recovery
(same content, partially committed) from update (different content, needs
cascade-delete). Previously, existing chunks always triggered recovery mode.
* refactor(retain): remove dead code, replace raw dicts with Phase3Context dataclass
- Remove dead _handle_zero_facts_documents (no callers after path unification)
- Remove unused imports: defaultdict, EntityLink
- Replace raw dict phase3_context with typed Phase3Context dataclass
- Update _build_and_insert_entity_links_phase3 to use typed parameter
Remove the BFS spreading activation and MPFP (Multi-Path Fact Propagation)
graph retrieval strategies, leaving link_expansion as the sole graph
retrieval algorithm. Rename MPFPTimings to GraphRetrievalTimings and
mpfp_timings field to graph_timings since the timing struct is used by
LinkExpansionRetriever.
Deleted:
- hindsight-api-slim/hindsight_api/engine/search/mpfp_retrieval.py
- hindsight-api-slim/tests/test_mpfp_retrieval.py
Removed config: HINDSIGHT_API_MPFP_TOP_K_NEIGHBORS
Enable passing arbitrary extra_body parameters to OpenAI-compatible API
calls via a JSON-encoded env var. This supports custom model servers
(e.g. vLLM) that need parameters like chat_template_kwargs to control
thinking mode.
Co-authored-by: EMIRHAN GAZI <emirhan.gazi+tcell@turkcell.com.tr>
* feat: add optional LiteLLM SDK embedding output dimensions
Allow configuring an optional output dimension for litellm-sdk embeddings and pass it through only when set, while preserving default behavior.
Made-with: Cursor
* test: assert wrapped init error for invalid dimensions
Add a LiteLLM SDK embeddings test that verifies invalid OpenAI dimensions fail during initialize() and preserve provider error details in the wrapped RuntimeError.
Made-with: Cursor
* fix(mcp): handle Claude Code GET probe and make stateless_http configurable (#751)
Claude Code v2.1.84+ sends a GET to /mcp/ before POST initialize,
which fails with 405 (stateless) or 400 (stateful). Intercept
sessionless GET requests in MCPMiddleware and return 200 OK so the
client proceeds to POST initialize.
Also make stateless_http configurable via HINDSIGHT_API_MCP_STATELESS
(default: false/stateful) instead of hardcoding true.
Closes#751
* docs: add HINDSIGHT_API_MCP_STATELESS to configuration reference
* feat: add max_observations_per_scope bank config
Adds a configurable limit on the number of observations per tag scope.
When the limit is reached, consolidation only updates/deletes existing
observations — no new ones are created. Enforcement is done via a
constrained Pydantic response model (max_length on creates list) so the
LLM structurally cannot exceed the limit, plus prompt guidance.
- Config: HINDSIGHT_API_MAX_OBSERVATIONS_PER_SCOPE (-1 = unlimited)
- Reorder action execution: deletes → updates → creates
- Dynamic _ConsolidationBatchResponse with max_length constraint
- Prompt CAPACITY CONSTRAINT section when near/at limit
- Observations with no tags skip the limit entirely
- Control plane UI field + docs
* fix: strengthen max_observations tests with mock LLM + defensive truncation
- Rewrite integration tests to use MockLLM with deterministic responses
(one observation per fact) instead of relying on real LLM behavior
- Add defensive truncation in _consolidate_batch_with_llm as belt-and-
suspenders — catches LLM providers that ignore JSON schema max_length
- Tests now assert exact counts, not just upper bounds
* fix(migrations): use HINDSIGHT_API_MIGRATION_DATABASE_URL when set
Session-level advisory locks are broken when the database URL goes
through PgBouncer in transaction mode: the backend connection is
returned to the pool on COMMIT, orphaning the lock, so multiple pods
can simultaneously run migrations for the same schema.
When HINDSIGHT_API_MIGRATION_DATABASE_URL is set, use it for both
the advisory lock connection and the Alembic run. Callers should
point this at the direct PostgreSQL endpoint (bypassing the pooler)
so the session-level lock is held for the full migration duration.
* refactor(migrations): move MIGRATION_DATABASE_URL to standard config
Wire HINDSIGHT_API_MIGRATION_DATABASE_URL through HindsightConfig
instead of reading os.getenv() directly in migrations.py. Add the
field to the dataclass, from_env(), log_config(), all call sites,
.env.example, and the configuration docs page.
* fix: update test mocks for migration_database_url kwarg and regenerate docs skill
---------
Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>
* fix: silence noisy google_genai.models INFO logging
The google-genai SDK logs "AFC is enabled with max remote calls: 10"
at INFO level on every initialization. Set its logger to WARNING.
* fix: regenerate docs skill in release-integration script
The release script generates changelog/SDK pages but never re-ran
generate-docs-skill.sh, causing CI to fail with out-of-sync skill
files after every integration release. Now it regenerates the skill
and includes the output in the release commit.
Also adds the missing ag2 skill files from the latest release.
* feat: add audit log for feature usage tracking
Add full auditability for all mutating and core API operations across
HTTP, MCP, and system (worker) transports. Audit entries record raw
request/response as JSONB, timing (started_at/ended_at), action, and
transport type.
Backend:
- New audit_log table with JSONB columns for expandability without
future migrations (merge migration of 3 existing heads)
- AuditLogger with fire-and-forget writes via asyncio.create_task
- @audited decorator on 28 HTTP route handlers
- MCP tool audit wrapping for 16 auditable tools
- Worker task execution wrapped with audit_context
- List endpoint with action, transport, date range filters + pagination
- Stats endpoint with per-day counts for charting
- Configurable retention sweep (concurrent-safe DELETE)
Config (env-only, static):
- HINDSIGHT_API_AUDIT_LOG_ENABLED (default: false)
- HINDSIGHT_API_AUDIT_LOG_ACTIONS (comma-separated allowlist, empty=all)
- HINDSIGHT_API_AUDIT_LOG_RETENTION_DAYS (default: -1, keep forever)
Control Plane:
- New "Audit Logs" tab on bank configuration page
- Line chart showing request volume (today/7d/30d) with action filter
- Filterable table with action, transport, date range filters
- Paginated list with detail dialog showing raw request/response JSON
Tests:
- 13 tests covering list, filters, pagination, stats, disabled mode,
action allowlist, and ordering
* fix: split 3-way merge migration into two 2-way merges
Alembic doesn't support 3-parent merge migrations. Split into a no-op
merge of 2 heads (b1c2d3e4f5g6) followed by the audit_log table
migration merging the third head.
* fix: correct merge migration to merge actual 2 heads
The original analysis incorrectly identified 3 heads. There were only 2
(a3b4c5d6e7f8 and c8e5f2a3b4d1). Remove the unnecessary intermediate
merge migration and fix the audit_log migration to merge these 2 heads.
* fix: use 'heads' instead of 'head' in migration runner
Alembic's upgrade('head') fails when multiple heads exist (e.g. from
namespace package overlaps between hindsight-api and hindsight-api-slim).
Using 'heads' (plural) handles this gracefully by upgrading all branches.
* chore: regenerate OpenAPI spec with audit log endpoints
* chore: regenerate TypeScript client and docs skill OpenAPI spec
Python and Go clients still need regeneration (requires Docker).
* chore: regenerate all client SDKs (Python, Go, TypeScript)
Adds generated audit log API clients for Python (audit_api.py),
Go (api_audit.go), and TypeScript client type updates.
- Add 'ark' and 'volcano' as valid LLM providers (both are aliases for Volcano Engine)
- Set default model to 'doubao-pro-32k' for both providers
- Add them to OpenAICompatibleLLM provider list
- Exclude from json_object response format support
Co-authored-by: yishun.eason <yishun.eason@bytedance.com>
Adds a proper 'none' provider option so users can run Hindsight as a
chunk store with semantic search but without any LLM dependency, replacing
the hacky workaround of setting provider to 'mock'.
When HINDSIGHT_API_LLM_PROVIDER=none:
- Retain automatically uses chunks mode (no fact extraction)
- Recall works normally (semantic search, BM25, graph retrieval)
- Reflect returns HTTP 400 with clear error message
- Consolidation/observations are disabled
- Mental model refresh returns HTTP 400
- No API key required
* feat(reflect): make source facts in search_observations configurable
The recent fix (#669) hardcoded include_source_facts=False in
search_observations to prevent context overflow. This makes it
configurable via HINDSIGHT_API_REFLECT_SOURCE_FACTS_MAX_TOKENS
(env/tenant/bank), defaulting to -1 (disabled).
- -1: source facts disabled (current behavior, default)
- 0: source facts enabled with no token limit
- >0: source facts enabled with a token budget
* docs: add reflect_source_facts_max_tokens to configuration reference
* fix: update configurable fields count in tests and regenerate docs skill
* feat: add LiteLLM LLM provider for Bedrock and 100+ providers
Add a new `litellm` LLM provider that uses the LiteLLM SDK for chat
completions and tool calling, enabling AWS Bedrock and 100+ other
providers for Hindsight's core engine (retain, recall, reflect).
- New LiteLLMLLM provider in engine/providers/litellm_llm.py
- Registered in factory, valid providers list, and no-api-key set
- Refactored API key validation to use requires_api_key() helper
- Added boto3 dependency for Bedrock auth
- Updated docs: configuration, models, monitoring, providers grid
* feat: add bedrock as first-class LLM provider alias
Add `bedrock` as a dedicated provider name that auto-prepends the
`bedrock/` prefix to model names and delegates to LiteLLMLLM under
the hood. This makes Bedrock support more discoverable — users set
`HINDSIGHT_API_LLM_PROVIDER=bedrock` with plain Bedrock model IDs.
* test: add Bedrock to CI provider tests
- Add bedrock/us.amazon.nova-lite-v1:0 to MODEL_MATRIX in test_llm_provider.py
- Add AWS credential check in should_skip_provider()
- Pass AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION_NAME secrets to test-api job
- Update default bedrock model to amazon.nova-2-lite-v1:0
* fix: regenerate docs skill files and bump memory test timeout
- Regenerate skills/hindsight-docs references after docs changes
- Bump test_llm_provider_memory_operations timeout to 600s for slower
providers like Bedrock via LiteLLM
* test: skip bedrock lite models in memory operations test
Nova Lite has a 10K output token limit which is too low for fact
extraction (requires 64K). The api_methods test (completion, tools,
structured output) already validates the provider works correctly.
* test: use Nova Pro for bedrock CI tests to cover full memory pipeline
Nova Lite only supports 10K output tokens, too low for fact extraction.
Switch to Nova Pro which supports the full 64K output needed for
retain/reflect operations. This ensures bedrock is tested on all
Hindsight functionalities, not just basic API methods.
* test: switch bedrock CI to Nova 2 Lite (supports 64K output tokens)
Nova v1 models (Pro, Lite) have a 10K output token limit which is
too low for fact extraction. Nova 2 Lite supports 64K+ output tokens,
enabling full memory pipeline testing (retain + reflect).
* feat(openclaw): remove hardcoded default models, rely on Hindsight API defaults
* feat(claude-code): remove hardcoded default models, rely on Hindsight API defaults
* feat(claude-code,docs): remove hardcoded default models from claude-code integration and docs
* feat: use claude-haiku-4-5 as default Anthropic model
* feat: upgrade MiniMax default model from M2.5 to M2.7
MiniMax has released MiniMax-M2.7, their latest model with a 1M context
window (up from 204K). This updates the default model across config,
docs, and examples. M2.5 remains fully compatible for users who prefer it.
- Update PROVIDER_DEFAULT_MODELS to MiniMax-M2.7
- Update .env.example and documentation references
- Add test_minimax_provider.py with M2.7 and backward compat tests
* chore: remove test file per review feedback
---------
Co-authored-by: PR Bot <pr-bot@minimaxi.com>
* feat(retain): add verbatim extraction mode
Adds retain_extraction_mode="verbatim" that stores each chunk as-is
without LLM summarization. The LLM still runs to extract entities,
temporal info, and location for full indexability — only the fact text
is replaced with the original chunk content (one memory per chunk).
Useful for RAG-style indexing and benchmarks where original text
must be preserved in memory.
- Add "verbatim" to RETAIN_EXTRACTION_MODES in config.py
- Add VERBATIM_FACT_EXTRACTION_PROMPT with instructions to preserve text
- Add _collapse_to_verbatim() post-processing to enforce 1 fact/chunk
- Expose in bank config UI dropdown with updated description
- Update configuration.md docs with verbatim mode description
- Add unit test for _collapse_to_verbatim and integration test via LLM
- Fix pre-existing main.py CLI override missing new reranker fields
- Fix pre-existing cross_encoder.py ty type error via setattr
* refactor(retain): verbatim mode skips 'what' field entirely
Instead of asking the LLM to echo the chunk text back into 'what' and
then discarding it, verbatim mode now uses a dedicated schema
(VerbatimExtractedFact) that omits the 'what' field altogether.
The LLM only returns metadata (entities, temporal info, location, who),
saving output tokens and avoiding any risk of paraphrasing before the
backfill.
- Add VerbatimExtractedFact / VerbatimFactExtractionResponse models
- Verbatim mode skips causal-relations section (nothing to relate causally)
- _extract_facts_from_chunk: allow missing 'what' in verbatim mode,
set combined_text="" (backfilled by _collapse_to_verbatim)
- Update verbatim prompt to say DO NOT include 'what'
* feat(retain): add index_only extraction mode
Zero-LLM retain mode: chunks are stored as-is with no LLM call, no
entity extraction, and no temporal indexing. Embeddings still run for
semantic search. User-provided entities via RetainContent.entities
are the sole source of entity data.
Early return placed before the batch-API check so no LLM queue or
concurrency locks are acquired.
- Add "index_only" to RETAIN_EXTRACTION_MODES
- Add _extract_facts_index_only() with pure Python chunking path
- Add to UI dropdown and update description
- Update configuration.md with index_only docs and table entry
- Add unit test asserting zero token usage and exact text preservation
* feat(retain): add named retain strategies
Allows mixing extraction modes in a single bank via named strategies.
Each strategy is a set of hierarchical config overrides (extraction_mode,
chunk_size, entity_labels, entities_allow_free_form, etc.) applied on
top of the resolved bank config at retain time.
- retain_strategies: dict of strategy_name → config overrides (bank config)
- retain_default_strategy: default strategy when none specified (bank config)
- strategy field on /retain request: per-call override
- apply_strategy() in config_resolver applies overrides via dataclasses.replace()
- strategy propagates through retain_batch_async → _retain_batch_async_internal
and through the async worker task payload
- Any hierarchical field is overridable per strategy, including entity_labels
and entities_allow_free_form
- Docs updated with strategy configuration example and RRF fairness note
- Unit test for apply_strategy covering overrides, unknown strategy, and
non-hierarchical field filtering
* feat(retain): add per-item strategy and strategy tests
- Add `strategy` field to `MemoryItem` so individual items in a retain
request can override the request-level strategy
- Add `strategy` field to `FileRetainMetadata` for per-file strategy
override in file retain requests
- Group memory items by effective strategy in `api_retain`; each group
is processed as a separate batch, results are aggregated
- Thread strategy through `submit_async_file_retain` →
`_handle_file_convert_retain` → retain task payload
- Add `operation_ids` to `RetainResponse` for async requests with
mixed per-item strategies
- Add `test_strategy_overrides_extraction_mode_for_index_only`: unit
test verifying a named strategy with index_only bypasses the LLM
- Add `test_retain_request_per_item_strategy_field`: unit test for
per-item strategy grouping logic
* feat(ui): add retain strategies and default strategy to bank config UI
- Add StrategiesEditor component: per-strategy cards with name input and
JSON overrides textarea; supports add/remove; validates JSON inline
- Add Default Strategy text input (retain_default_strategy)
- Update RetainEdits type and retainSlice() to include both new fields
- Regenerate OpenAPI spec (retain_strategies, retain_default_strategy,
per-item strategy on MemoryItem/FileRetainMetadata, operation_ids on
RetainResponse)
* refactor(ui): move retain strategies into its own dedicated config section
* feat(ui): improve retain strategies UX and add strategy to document dialog
- Strategy form now includes entity section (free form toggle + entity labels editor)
- Default strategy selector moved outside tab panel, above strategy chips
- Strategy tabs redesigned with underline indicator style for clarity
- Remove strategy confirms with AlertDialog
- Fix tab re-render bug when typing strategy name (skipSyncRef)
- Add strategy field to Add New Document dialog (text + per-file for uploads)
- File upload collapsible uses same Document/Tags/Source tabbed layout
- API: validate empty strategy names in config_resolver
- api.ts: add strategy field to retain and uploadFiles types
* fix: forward strategy through HTTP layer and SDK; add integration test
- route.ts: extract and forward `strategy` from request body to retainBatch
- TypeScript SDK: accept and forward `strategy` in retainBatch options and per-item
- config_resolver.py: validate empty strategy name keys on update
- bank-config-view.tsx: merge entity fields into RetainStrategyForm, redesign strategy tabs with underline style, add confirmation dialog for removal, fix tab-reset-on-typing with skipSyncRef, move default strategy selector outside panel
- bank-selector.tsx: add strategy field to Add Document dialog (per-file in tabbed collapsible)
- test_retain.py: add end-to-end integration test verifying named strategy application (index_only = 0 LLM tokens)
* fix: regenerate TypeScript client with strategy field in RetainRequest/MemoryItem
- Regenerate OpenAPI spec to include strategy field in RetainRequest and MemoryItem
- Regenerate TypeScript client from updated spec
- Add strategy to MemoryItemInput interface
- Remove (item as any) cast now that strategy is properly typed
* rename: index_only extraction mode → chunks
* remove top-level strategy from RetainRequest; strategy is per-item only
* fix(clients): update Go and Python generated clients with strategy/operation_ids fields
* fix(ci): update hierarchical field count, add strategy to Rust MemoryItem initializers
* fix(go-client): minimal targeted YAML updates for strategy/operation_ids fields
* feat: introduce hindsight-api-slim and hindsight-all-slim packages
Closes#552
- Move all source code from hindsight-api/ to new hindsight-api-slim/
- hindsight-api-slim has heavy ML deps (torch, sentence-transformers,
transformers, einops, flashrank, mlx, mlx-lm, safetensors) and
pg0-embedded as optional extras: [local-ml], [embedded-db], [all]
- hindsight-api becomes a zero-code meta-package depending on
hindsight-api-slim[all] for full backward compatibility
- Add hindsight-all-slim meta-package: hindsight-api-slim + client + embed
- hindsight-all updated to depend on hindsight-api-slim[all]
- pg0.py: lazy-import pg0 with clear ImportError pointing to [embedded-db]
- Dockerfile: replace sed hack with proper uv sync --extra flags
- Update release.yml, test.yml, lint.sh, release.sh, CLAUDE.md and
all path references throughout the repo
* refactor: rename hindsight/ directory to hindsight-all/
* docs: document hindsight-api-slim and hindsight-all-slim package variants
Add package variants table and extras explanation to installation.md
* docs: remove emojis from installation.md, use professional tone
* docs: link Docker slim variant to pip package variants section
* docs: consolidate Docker image variants into single table
* ci: fix working-directory paths after package restructure
- Replace all hindsight-api → hindsight-api-slim in test.yml
- Replace hindsight → hindsight-all in test.yml
- Add --extra embedded-db to test-embed API install step
* ci: add local-ml and embedded-db extras to API sync steps
These extras were previously implicit in the old hindsight-api package
(which bundled everything). Now that hindsight-api-slim uses optional
extras, we must explicitly request local-ml and embedded-db in CI.
* ci: add API install step with embedded-db to test-embed smoke test
The smoke test starts hindsight-api as a daemon, which requires pg0-embedded.
Add a dedicated install step for hindsight-api-slim with embedded-db extra
so the daemon can start successfully.
* ci: remove --no-install-project when using optional extras
When --no-install-project is combined with --extra, the optional deps
are not installed because extras require the project to be active.
Remove --no-install-project from steps that need local-ml or embedded-db.
* ci: fix ordering of uv sync steps to preserve optional extras
When uv sync runs for a different workspace member, it removes optional
extras installed for other members. Fix by always running extra-requiring
API sync last, after other workspace member syncs.
Also remove --no-install-project from embedded-db sync in test-embed,
as --no-install-project prevents optional extras from being active.
* ci: add local-ml extra to test-embed API install for smoke test
The smoke test starts the full API server which needs sentence-transformers
for local embeddings (default provider). Add local-ml extra to the install.
* ci: simplify extras with --all-extras and add slim pip smoke test
- Replace explicit --extra local-ml --extra embedded-db with --all-extras
for cleaner, more maintainable sync steps
- Add test-pip-slim job: tests hindsight-api-slim[embedded-db] without
local ML models, using Cohere for embeddings/reranking (mirrors Docker
slim smoke test approach)
* ci: simplify slim smoke test to health check only (mirrors Docker test)
2026-03-13 13:50:03 +01:00
Renamed from hindsight-api/hindsight_api/config.py (Browse further)
