* feat(skill): validate links, strip images, include openapi.json and changelog
- Add post-processing step to rewrite Docusaurus site-root paths (e.g.
/developer/foo) to proper relative .md paths within the skill
- Strip markdown and HTML images from all generated files since assets
are not bundled with the skill
- Copy hindsight-docs/static/openapi.json into references/openapi.json
and map /api-reference links to it
- Include changelog.md from src/pages/ alongside faq and best-practices
- Add final validation step that fails the build if any link still
points outside the skill directory
* ci: run generate-docs-skill in verify-generated-files job
* fix(skill): strip unresolvable site-root links instead of leaving them broken
* fix(skill): write file when images stripped but no links rewritten
* chore(skill): regenerate with fixed links, stripped images, changelog and openapi
* fix(skill): handle changelog as directory, add agno/hermes integrations, rebase on main
- Add IntegrationsBanner component with infinite left-to-right CSS scroll animation showing all clients, integrations, and LLM providers
- Place banner below the navbar on every page via Navbar theme wrapper
- Add Agno and Hermes to both the IntegrationsGrid and the banner
- Remove right border from doc sidebar via custom.css
* feat: independent versioning for integrations
- Add per-integration changelog pages at /changelog/integrations/<name>
- Move main changelog to changelog/index.md (URL unchanged)
- Add --integration flag to generate-changelog for LLM-based per-integration changelog generation
- Add scripts/release-integration.sh <name> <version> for cutting integration releases
- Add .github/workflows/release-integration.yml to publish on integrations/** tags
- Remove integrations from main release.sh and release.yml cycle
* fix: add agno and hermes integration docs to version-0.4 for production build
* chore: apply ruff formatting to generate_changelog.py
* feat: add 4-tab code parity across all documentation examples
Every code snippet Tabs block now has Python, Node.js, CLI, and Go variants.
Raw HTTP/curl tabs replaced with proper SDK calls.
New example files:
- Go: retain.go, recall.go, reflect.go, memory-banks.go, directives.go,
mental-models.go, documents.go, main-methods.go
- Shell: memory-banks.sh, directives.sh, mental-models.sh
- Node.js: mental-models.mjs
Extended example files with missing sections:
- recall.mjs/sh: world/experience/observation types, token-budget, all tag modes
- reflect.sh: reflect-with-params, reflect-disposition, reflect-sources, reflect-with-tags
- reflect.mjs: reflect-with-tags, fixed reflect-sources API usage
- retain.mjs/sh: retain-conversation, retain-batch, retain-files-batch
SDK/CLI additions:
- TypeScript: getMentalModelHistory method
- CLI recall: --tags, --tags-match flags
- CLI reflect: --tags, --tags-match, --include-facts flags
- CLI directive update: --is-active flag
- CLI bank set-config: --retain-mission, --retain-extraction-mode,
--observations-mission, --reflect-mission, --disposition-* flags
Build validation:
- scripts/check-code-parity.mjs validates 4-tab parity across all MDX files
- Integrated into npm run build — fails if any Tabs block is missing a variant
* fix: fix doc examples for Go, Node.js, CLI + add mental model with-id examples
- Fix Go Budget constants: BUDGET_HIGH/LOW/MID → HIGH/LOW/MID
- Fix Go documents.go: ListDocuments returns []map[string]interface{}, use map access
- Fix Go retain.go: use correct relative path for sample.pdf
- Fix Node.js createMentalModel: use positional args (name, sourceQuery) not object
- Add CLI 'history' subcommand for mental models (api.rs, main.rs, mental_model.rs)
- Rebuild TypeScript/Python clients to support id param in createMentalModel
- Add create-mental-model-with-id examples across all 4 languages and docs
* fix: move id param to end of create_mental_model signature for backwards compat
* Fix entity_id null constraint for non-ASCII entity names (Turkish İ etc.)
Python's str.lower() and PostgreSQL's LOWER() produce different results for
some Unicode characters. The most common case is Turkish İ (U+0130):
Python: 'İstanbul'.lower() == 'i\u0307stanbul' (i + combining dot, 2 chars)
PostgreSQL: LOWER('İstanbul') == 'istanbul' (plain i, 1 char)
In _resolve_from_candidates, the fallback SELECT for conflicted entity names
passed Python-lowercased strings to LOWER(canonical_name) = ANY($names), so
PostgreSQL couldn't match them. entity_ids[idx] stayed None, which then
caused a NOT NULL violation on unit_entities.entity_id, failing the entire
retain.
Fix: pass original mixed-case names to the fallback SELECT and use
LOWER(canonical_name) = ANY(SELECT LOWER(n) FROM unnest($2) AS n) so
PostgreSQL lowercases both sides identically. The query also returns the
original input_name so we can add a Python-lowercased key to id_by_name
for the assignment loop that uses Python-lowercased keys.
* Add regression test for Unicode entity conflict
The Pydantic model extraction paths (batch API and parallel extraction) used
fact_from_llm.fact_type directly, bypassing the \"assistant\" → \"experience\"
conversion and causing DB CHECK constraint violations.
Unified the conversion logic across all paths:
- \"assistant\" → \"experience\"
- \"world\" → \"world\"
- anything else: fall back to fact_kind (\"assistant\" → \"experience\"), else \"world\"
* feat: independent versioning for integrations
- Add per-integration changelog pages at /changelog/integrations/<name>
- Move main changelog to changelog/index.md (URL unchanged)
- Add --integration flag to generate-changelog for LLM-based per-integration changelog generation
- Add scripts/release-integration.sh <name> <version> for cutting integration releases
- Add .github/workflows/release-integration.yml to publish on integrations/** tags
- Remove integrations from main release.sh and release.yml cycle
* fix: add agno and hermes integration docs to version-0.4 for production build
* chore: apply ruff formatting to generate_changelog.py
* feat: upgrade MiniMax default model from M2.5 to M2.7
MiniMax has released MiniMax-M2.7, their latest model with a 1M context
window (up from 204K). This updates the default model across config,
docs, and examples. M2.5 remains fully compatible for users who prefer it.
- Update PROVIDER_DEFAULT_MODELS to MiniMax-M2.7
- Update .env.example and documentation references
- Add test_minimax_provider.py with M2.7 and backward compat tests
* chore: remove test file per review feedback
---------
Co-authored-by: PR Bot <pr-bot@minimaxi.com>
* feat(typescript-client): add Deno compatibility
- Switch build from tsc to tsup for dual CJS + ESM output with proper exports field
- Add deno_setup.ts preload that injects Jest-compatible globals (describe/test/expect) via @std/testing/bdd and @std/expect
- Fix generated client.gen.ts: exclude hey-api internal `client` field from RequestInit spread to avoid conflict with Deno.HttpClient
- Add test:deno npm script using --unstable-sloppy-imports and --preload
- Add test-typescript-client-deno CI job using denoland/setup-deno@v2 (v2.x)
- Update docs: rename page to TypeScript / JavaScript Client, add Deno installation section
* feat: add Deno compatibility to ai-sdk and chat integrations
- Switch ai-sdk and chat builds from tsc to tsup (ESM bundle, eliminates
extension-less import issues in Deno)
- Add deno.json import map to ai-sdk redirecting 'vitest' to a custom
vitest-compat.ts shim and bare npm specifiers to npm: URLs
- Add vitest-compat.ts shim implementing vi.fn()/vi.spyOn()/vi.mocked()
using @std/expect's Symbol.for("@MOCK") interface so toHaveBeenCalledWith
and other mock matchers work under Deno
- Add test:deno script to ai-sdk (all 30 tests pass under Deno)
* ci: add Deno test job for ai-sdk integration
Adds a new test-ai-sdk-integration-deno CI job that runs the ai-sdk
unit tests under Deno LTS, verifying Deno compatibility of the package.
* fix: remove broken link to non-existent n8n blog post in streamlit post
* fix: patch client.gen.ts for Deno compatibility during generation
Add a post-generation patch step to generate-clients.sh that removes
the hey-api internal 'client' field from the RequestInit spread in
client.gen.ts. Deno's Request constructor rejects 'client' because it
conflicts with the Deno.HttpClient option name.
* feat: add Agno integration with Hindsight memory toolkit
Add hindsight-agno package providing Hindsight memory tools (retain,
recall, reflect) as an Agno Toolkit, following the same pattern as
Agno's Mem0Tools. Includes per-user bank isolation, global config,
bank auto-creation, and memory_instructions() for system prompt
injection.
Also adds cookbook documentation page with architecture diagrams,
quick start examples, and configuration reference.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: remove n8n blog post, add Agno icon, bind to release process
- Remove n8n blog post from the agno integration branch
- Add Agno logo icon and map hindsight-agno SDK tag in CookbookGrid
- Add hindsight-agno to release.sh PYTHON_PACKAGES array
- Add build, publish, artifact upload, and release asset steps in release.yml
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: remove cookbook page (moved to hindsight-cookbook repo)
The Agno cookbook application now lives in
vectorize-io/hindsight-cookbook/applications/agno-memory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent silent memory loss on consolidation LLM failure
When all LLM retries are exhausted during consolidation, memories were
being marked consolidated_at unconditionally, permanently excluding them
from future consolidation runs without producing any observations.
Fix with two complementary mechanisms:
- Adaptive batch splitting: on LLM failure, the batch is halved and
retried recursively down to batch_size=1, recovering most transient
failures (rate limits, Pydantic validation on long prompts) without
operator intervention
- consolidation_failed_at column: only single-memory batches that still
fail after all retries are marked here instead of consolidated_at, so
they remain visible and retryable
- New API endpoint POST /v1/default/banks/{bank_id}/consolidation/retry-failed
resets these memories for the next consolidation run
* chore: regenerate OpenAPI spec
* fix: rename consolidation endpoint from /retry-failed to /recover
* fix: add consolidation_failed_at column, adaptive batch splitting, and recovery API
- Migration a3b4c5d6e7f8: add consolidation_failed_at TIMESTAMPTZ column to
memory_units with an index for efficient failure queries; properly chains off
g7h8i9j0k1l2 (backsweep_orphan_observations)
- Consolidator: filter pending memories with consolidation_failed_at IS NULL
so failed memories are not re-fetched in an infinite loop
- Consolidator: adaptive batch splitting — when a batch exhausts all 3 LLM
retries, halve it and retry sub-batches recursively; only single-memory
batches that also exhaust all retries get consolidation_failed_at set
- New tests (9 total) covering: adaptive splitting recovers all memories,
larger batch splitting, single-memory permanent failure, exclusion from
next run, partial batch failure, recover resets columns, recover returns
0 when none failed, recover-then-consolidate succeeds, HTTP endpoint
* chore: regenerate Go, Python, TypeScript clients with recover consolidation endpoint
* feat: add Recover Consolidation action to bank Actions dropdown
* style: apply ruff formatting to http.py and config.py
* fix: handle consolidation scope in large batch test mock LLM
The mock LLM was returning {"facts": ...} for ALL calls including consolidation.
Consolidation doesn't use skip_validation=True so it expects a _ConsolidationBatchResponse
instance, not a raw dict. Before this PR consolidation silently swallowed the AttributeError
(failed=False was returned); now failed=True triggers adaptive splitting and timeouts.
Fix: return _ConsolidationBatchResponse() when scope=="consolidation".
* fix: restrict claude-agent-sdk to macOS platform only (no Linux wheel available)
Also fix pre-existing type errors: use setattr for XLM-RoBERTa monkey-patch
and add missing reranker_local_fp16/bucket_batching/batch_size fields to main.py config constructor.
* fix: add UV_INDEX_STRATEGY=unsafe-best-match to fix markupsafe cp314 wheel conflict
PyTorch CPU index serves markupsafe==3.0.3 with only cp314 wheels.
uv's default first-index strategy stops at the first index with any version
even if no compatible wheel exists. unsafe-best-match searches all indices
for the best compatible wheel, falling back to PyPI for markupsafe.
* fix: use explicit pytorch index to prevent markupsafe wheel conflict
Configure the pytorch CPU index as explicit=true in pyproject.toml so it is
ONLY used for torch (via [tool.uv.sources]). All other packages (including
markupsafe) are resolved exclusively from PyPI, preventing the pytorch index
from serving incompatible cp314-only wheels for non-pytorch packages.
Remove UV_INDEX and UV_INDEX_STRATEGY from CI workflow (no longer needed
since the index is now configured in pyproject.toml).
* ci: trigger CI run
* ci: retry trigger
* ci: trigger after remote URL fix
* ci: add workflow_dispatch to unblock manual trigger
* fix: remove empty env blocks left after UV_INDEX removal
* fix: add type: ignore for optional claude_agent_sdk imports (macOS-only)
* fix: correct type: ignore rules for claude_agent_sdk and fix utcnow deprecation
* feat(retain): add verbatim extraction mode
Adds retain_extraction_mode="verbatim" that stores each chunk as-is
without LLM summarization. The LLM still runs to extract entities,
temporal info, and location for full indexability — only the fact text
is replaced with the original chunk content (one memory per chunk).
Useful for RAG-style indexing and benchmarks where original text
must be preserved in memory.
- Add "verbatim" to RETAIN_EXTRACTION_MODES in config.py
- Add VERBATIM_FACT_EXTRACTION_PROMPT with instructions to preserve text
- Add _collapse_to_verbatim() post-processing to enforce 1 fact/chunk
- Expose in bank config UI dropdown with updated description
- Update configuration.md docs with verbatim mode description
- Add unit test for _collapse_to_verbatim and integration test via LLM
- Fix pre-existing main.py CLI override missing new reranker fields
- Fix pre-existing cross_encoder.py ty type error via setattr
* refactor(retain): verbatim mode skips 'what' field entirely
Instead of asking the LLM to echo the chunk text back into 'what' and
then discarding it, verbatim mode now uses a dedicated schema
(VerbatimExtractedFact) that omits the 'what' field altogether.
The LLM only returns metadata (entities, temporal info, location, who),
saving output tokens and avoiding any risk of paraphrasing before the
backfill.
- Add VerbatimExtractedFact / VerbatimFactExtractionResponse models
- Verbatim mode skips causal-relations section (nothing to relate causally)
- _extract_facts_from_chunk: allow missing 'what' in verbatim mode,
set combined_text="" (backfilled by _collapse_to_verbatim)
- Update verbatim prompt to say DO NOT include 'what'
* feat(retain): add index_only extraction mode
Zero-LLM retain mode: chunks are stored as-is with no LLM call, no
entity extraction, and no temporal indexing. Embeddings still run for
semantic search. User-provided entities via RetainContent.entities
are the sole source of entity data.
Early return placed before the batch-API check so no LLM queue or
concurrency locks are acquired.
- Add "index_only" to RETAIN_EXTRACTION_MODES
- Add _extract_facts_index_only() with pure Python chunking path
- Add to UI dropdown and update description
- Update configuration.md with index_only docs and table entry
- Add unit test asserting zero token usage and exact text preservation
* feat(retain): add named retain strategies
Allows mixing extraction modes in a single bank via named strategies.
Each strategy is a set of hierarchical config overrides (extraction_mode,
chunk_size, entity_labels, entities_allow_free_form, etc.) applied on
top of the resolved bank config at retain time.
- retain_strategies: dict of strategy_name → config overrides (bank config)
- retain_default_strategy: default strategy when none specified (bank config)
- strategy field on /retain request: per-call override
- apply_strategy() in config_resolver applies overrides via dataclasses.replace()
- strategy propagates through retain_batch_async → _retain_batch_async_internal
and through the async worker task payload
- Any hierarchical field is overridable per strategy, including entity_labels
and entities_allow_free_form
- Docs updated with strategy configuration example and RRF fairness note
- Unit test for apply_strategy covering overrides, unknown strategy, and
non-hierarchical field filtering
* feat(retain): add per-item strategy and strategy tests
- Add `strategy` field to `MemoryItem` so individual items in a retain
request can override the request-level strategy
- Add `strategy` field to `FileRetainMetadata` for per-file strategy
override in file retain requests
- Group memory items by effective strategy in `api_retain`; each group
is processed as a separate batch, results are aggregated
- Thread strategy through `submit_async_file_retain` →
`_handle_file_convert_retain` → retain task payload
- Add `operation_ids` to `RetainResponse` for async requests with
mixed per-item strategies
- Add `test_strategy_overrides_extraction_mode_for_index_only`: unit
test verifying a named strategy with index_only bypasses the LLM
- Add `test_retain_request_per_item_strategy_field`: unit test for
per-item strategy grouping logic
* feat(ui): add retain strategies and default strategy to bank config UI
- Add StrategiesEditor component: per-strategy cards with name input and
JSON overrides textarea; supports add/remove; validates JSON inline
- Add Default Strategy text input (retain_default_strategy)
- Update RetainEdits type and retainSlice() to include both new fields
- Regenerate OpenAPI spec (retain_strategies, retain_default_strategy,
per-item strategy on MemoryItem/FileRetainMetadata, operation_ids on
RetainResponse)
* refactor(ui): move retain strategies into its own dedicated config section
* feat(ui): improve retain strategies UX and add strategy to document dialog
- Strategy form now includes entity section (free form toggle + entity labels editor)
- Default strategy selector moved outside tab panel, above strategy chips
- Strategy tabs redesigned with underline indicator style for clarity
- Remove strategy confirms with AlertDialog
- Fix tab re-render bug when typing strategy name (skipSyncRef)
- Add strategy field to Add New Document dialog (text + per-file for uploads)
- File upload collapsible uses same Document/Tags/Source tabbed layout
- API: validate empty strategy names in config_resolver
- api.ts: add strategy field to retain and uploadFiles types
* fix: forward strategy through HTTP layer and SDK; add integration test
- route.ts: extract and forward `strategy` from request body to retainBatch
- TypeScript SDK: accept and forward `strategy` in retainBatch options and per-item
- config_resolver.py: validate empty strategy name keys on update
- bank-config-view.tsx: merge entity fields into RetainStrategyForm, redesign strategy tabs with underline style, add confirmation dialog for removal, fix tab-reset-on-typing with skipSyncRef, move default strategy selector outside panel
- bank-selector.tsx: add strategy field to Add Document dialog (per-file in tabbed collapsible)
- test_retain.py: add end-to-end integration test verifying named strategy application (index_only = 0 LLM tokens)
* fix: regenerate TypeScript client with strategy field in RetainRequest/MemoryItem
- Regenerate OpenAPI spec to include strategy field in RetainRequest and MemoryItem
- Regenerate TypeScript client from updated spec
- Add strategy to MemoryItemInput interface
- Remove (item as any) cast now that strategy is properly typed
* rename: index_only extraction mode → chunks
* remove top-level strategy from RetainRequest; strategy is per-item only
* fix(clients): update Go and Python generated clients with strategy/operation_ids fields
* fix(ci): update hierarchical field count, add strategy to Rust MemoryItem initializers
* fix(go-client): minimal targeted YAML updates for strategy/operation_ids fields
* feat: add hindsight-hermes integration for Hermes Agent
* chore: add Hermes docs page, icon, and release process bindings
- Add cookbook page for Hermes integration (synced with README)
- Add Hermes icon and map hindsight-hermes SDK tag in CookbookGrid
- Add cookbook entry to index.mdx
- Add hindsight-hermes to release.sh PYTHON_PACKAGES array
- Add build, publish, artifact upload, and release asset steps in release.yml
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* docs: revamp sidebar with icon grid components and language support
- Merge Clients and Integrations sections into the developer sidebar
(removed top-level SDKs navbar item)
- Reorder sidebar: Architecture → API → Clients → Integrations → Hosting
- Unify icon system using react-icons (LuXxx/SiXxx) via customProps.icon
- Add uppercase section titles with increased spacing and reduced indentation
- Rename Node.js → "JavaScript / TypeScript" with TypeScript icon
- Add reusable IconGrid and SupportedGrids components (ClientsGrid,
IntegrationsGrid, LLMProvidersGrid)
- Use grids in FAQ, Models, Overview, and Quick Start pages
- Convert developer/index.md, models.md, faq.md to MDX for JSX support
* docs: add Best Practices page as unversioned standalone page
- Add src/pages/best-practices.mdx covering core concepts (memory banks,
taxonomy, memory types), bank configuration (missions, dispositions,
entity labels), retain (formats, context, document_id, tags, observation
scopes), recall (budget, tag filtering, include options), reflect
(recall vs reflect decision, response_schema, auditing), mental models,
and anti-patterns
- Add Resources section to sidebar with Best Practices and FAQ links
- Update generate-docs-skill.sh to include standalone pages (best-practices,
faq) from src/pages/ into the agent skill references
- SKILL.md now surfaces best-practices.md as the recommended starting point
* fix: remove leftover merge conflict markers in DocSidebarItem Link
* fix: add missing lu-star, lu-circle-help, lu-file-text icons to sidebar map
* fix: remove duplicate LuFileText import
* fix: add Best Practices and FAQ to Resources navbar dropdown
* docs: hide right TOC and add manual TOC to best practices page
* docs: hide right TOC and add manual TOC to FAQ page
* fix: add lu-star icon to navbar item icon map
* fix: correct broken anchor in best practices TOC
* blog: add n8n persistent memory workflows post
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: add cover image for n8n memory workflows post
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: update n8n cover image
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: remove broken screenshot references from n8n post
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: add Hindsight Cloud option and n8n Cloud guidance
- Add Cloud vs self-hosted setup paths in Step 1
- Show both Cloud and self-hosted URLs for retain/recall/reflect nodes
- Note that Cloud eliminates the localhost IP gotcha
- Mention n8n Cloud compatibility (requires Hindsight Cloud or public endpoint)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: update n8n post date to 2026-03-16
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: update n8n post with optimized content and fix accuracy
- Use optimized version of the blog post
- Fix blog cross-links to use date-prefixed URLs
- Fix retain response to match actual API (success, bank_id, items_count, async)
- Fix recall response to match actual API (text, type, entities — not confidence/source)
- Update title to "How to Add Persistent Memory to n8n Workflows"
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: update n8n post title
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add config vars for local reranker FP16 and bucket batching (#588)
* fix: add missing reranker local fields to CLI config override and fix ty type error
- Add reranker_local_fp16, reranker_local_bucket_batching, reranker_local_batch_size
to the manual HindsightConfig() constructor call in main.py (CLI override block)
- Replace direct module attribute assignment with setattr() in the transformers 5.x
monkey-patch so ty can resolve it without raising unresolved-attribute
* docs(skills): encourage rich context over pre-summarized strings in retain
The previous guidance told agents to distill content before calling
retain (e.g. "Be specific: store X not Y"). This misrepresents the
actual architecture: the server runs a full extraction pipeline (fact
extraction, entity linking, embeddings) on whatever is passed in.
- Add "How Hindsight Works" section explaining the server-side pipeline
- Update retain examples to pass full-context observations
- Replace "Be specific" with "Pass rich context"
- Clarify that --context is metadata labeling, not a content filter
Closes#592
* docs(skills): add raw conversation transcript example for retain
* docs: add config vars for local reranker FP16 and bucket batching (#588)
* fix: add missing reranker local fields to CLI config override and fix ty type error
- Add reranker_local_fp16, reranker_local_bucket_batching, reranker_local_batch_size
to the manual HindsightConfig() constructor call in main.py (CLI override block)
- Replace direct module attribute assignment with setattr() in the transformers 5.x
monkey-patch so ty can resolve it without raising unresolved-attribute
* fix(migration): backsweep orphaned observation memory units
Delete observation rows whose every source_memory_id points to a
deleted memory unit, left behind before PR #580 fixed the chunk FK
cascade and before delete_document() called
_delete_stale_observations_for_memories.
Closes#572 (data cleanup for pre-existing installs).
* fix(migration): broaden backsweep to cover all fact types and bank-level orphans
- Pass 1: delete any memory_units row (all fact_types) whose bank_id no
longer exists in banks — catches orphans from bank deletions that
predate a FK cascade between the two tables.
- Pass 2: delete observation rows whose every source_memory_id points to
a deleted memory unit, regardless of document_id/chunk_id anchors.
* test(migration): verify backsweep removes orphans and preserves legit rows
Adds a focused migration test that:
- Starts a fresh pg0 instance at revision f6g7h8i9j0k1
- Seeds orphaned rows for both backsweep passes (ghost-bank + all-dead-sources)
- Seeds legitimate rows that must survive
- Applies the backsweep migration to head
- Asserts the expected rows are deleted/preserved
The foreign key from memory_units.chunk_id to chunks.chunk_id used
ON DELETE SET NULL, which left ghost memory_units rows (chunk_id nulled
out, no parent document) after a document was deleted. Switching to
ON DELETE CASCADE lets the existing document -> chunks -> memory_units
cascade clean up everything in one pass.
Closes#572
Signed-off-by: JiangNan <1394485448@qq.com>
Some MCP clients (e.g., Claude Code) don't send an Accept header,
causing the MCP SDK to reject requests with 406 Not Acceptable. The
middleware now ensures Accept includes application/json and
text/event-stream when missing.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Add comprehensive docstrings to all API namespace classes
- Add return type annotations (Any) to all methods
- Add detailed Args and Returns sections to method docstrings
- Improve HindsightClient class docstring with Attributes section
- Add type annotations to __init__ parameters
Co-authored-by: 陈家名 <chenjiaming@icloud.com>
Gemini 3.1+ thinking models include a thought_signature field in functionCall
parts. When reconstructing conversation history for subsequent turns, this
signature must be preserved or the API returns 400 INVALID_ARGUMENT.
- Add optional thought_signature field to LLMToolCall
- Capture thought_signature from Gemini response parts
- Pass thought_signature back when reconstructing multi-turn history
- Add gemini-3.1-flash-lite-preview to the LLM provider test matrix
* feat: add compound tag filtering via tag_groups
Adds tag_groups to RecallRequest and ReflectRequest to express arbitrary
boolean tag predicates: leaf {tags, match}, and/or/not compounds.
Top-level groups are AND-ed. Existing tags/tags_match unchanged.
Examples:
Step filter AND user scope:
tag_groups: [{tags: ["step:5","step:8"], match: "any_strict"},
{tags: ["user:alice"], match: "all_strict"}]
Exclusion:
tag_groups: [{tags: ["user:alice"], match: "all_strict"},
{not: {tags: ["archived"], match: "any_strict"}}]
- Recursive SQL builder (build_tag_groups_where_clause) threads through
all 4 retrieval strategies (semantic/BM25, temporal, graph, MPFP)
- Python-side filter (filter_results_by_tag_groups) for post-traversal
- 22 new unit tests
- OpenAPI spec + all clients regenerated (Rust, Python, TypeScript, Go)
* fix: add tag_groups: None to Rust CLI struct initializers
* fix: add tag_groups: None to Rust client test RecallRequest initializer
* feat: reject tags+tag_groups together, add tag_groups integration tests
- Add model_validator to RecallRequest and ReflectRequest that returns 422
when both `tags` and `tag_groups` are set (mutually exclusive)
- Add 5 integration tests for tag_groups compound filtering:
* validation: 422 when both fields are set
* AND filter: two leaf groups (step scope AND user scope)
* OR compound: user:alice OR user:bob
* NOT compound: user:alice AND NOT archived
* Nested: user:alice AND (step:5 OR step:8)
* ci: trigger CI run
* docs: revamp sidebar with icon grid components and language support
- Merge Clients and Integrations sections into the developer sidebar
(removed top-level SDKs navbar item)
- Reorder sidebar: Architecture → API → Clients → Integrations → Hosting
- Unify icon system using react-icons (LuXxx/SiXxx) via customProps.icon
- Add uppercase section titles with increased spacing and reduced indentation
- Rename Node.js → "JavaScript / TypeScript" with TypeScript icon
- Add reusable IconGrid and SupportedGrids components (ClientsGrid,
IntegrationsGrid, LLMProvidersGrid)
- Use grids in FAQ, Models, Overview, and Quick Start pages
- Convert developer/index.md, models.md, faq.md to MDX for JSX support
* fix: use inline style for label color to prevent link color inheritance
* fix: label visibility and rename JavaScript/TypeScript to TypeScript
* feat: add HTTP client to grid and OpenAI Compatible to LLM providers grid
- Delete test_minimax_provider.py which imports non-existent `create_llm`
function (should be `create_llm_provider`), causing pytest collection errors
- Add scripts/smoke-test-slim.sh: shared retain + recall validation script
used by both Docker slim and pip slim CI jobs
- Update docker/test-image.sh to run retain/recall after health check for
all API targets
- Update test-pip-slim CI job to run the shared smoke test script
* feat: introduce hindsight-api-slim and hindsight-all-slim packages
Closes#552
- Move all source code from hindsight-api/ to new hindsight-api-slim/
- hindsight-api-slim has heavy ML deps (torch, sentence-transformers,
transformers, einops, flashrank, mlx, mlx-lm, safetensors) and
pg0-embedded as optional extras: [local-ml], [embedded-db], [all]
- hindsight-api becomes a zero-code meta-package depending on
hindsight-api-slim[all] for full backward compatibility
- Add hindsight-all-slim meta-package: hindsight-api-slim + client + embed
- hindsight-all updated to depend on hindsight-api-slim[all]
- pg0.py: lazy-import pg0 with clear ImportError pointing to [embedded-db]
- Dockerfile: replace sed hack with proper uv sync --extra flags
- Update release.yml, test.yml, lint.sh, release.sh, CLAUDE.md and
all path references throughout the repo
* refactor: rename hindsight/ directory to hindsight-all/
* docs: document hindsight-api-slim and hindsight-all-slim package variants
Add package variants table and extras explanation to installation.md
* docs: remove emojis from installation.md, use professional tone
* docs: link Docker slim variant to pip package variants section
* docs: consolidate Docker image variants into single table
* ci: fix working-directory paths after package restructure
- Replace all hindsight-api → hindsight-api-slim in test.yml
- Replace hindsight → hindsight-all in test.yml
- Add --extra embedded-db to test-embed API install step
* ci: add local-ml and embedded-db extras to API sync steps
These extras were previously implicit in the old hindsight-api package
(which bundled everything). Now that hindsight-api-slim uses optional
extras, we must explicitly request local-ml and embedded-db in CI.
* ci: add API install step with embedded-db to test-embed smoke test
The smoke test starts hindsight-api as a daemon, which requires pg0-embedded.
Add a dedicated install step for hindsight-api-slim with embedded-db extra
so the daemon can start successfully.
* ci: remove --no-install-project when using optional extras
When --no-install-project is combined with --extra, the optional deps
are not installed because extras require the project to be active.
Remove --no-install-project from steps that need local-ml or embedded-db.
* ci: fix ordering of uv sync steps to preserve optional extras
When uv sync runs for a different workspace member, it removes optional
extras installed for other members. Fix by always running extra-requiring
API sync last, after other workspace member syncs.
Also remove --no-install-project from embedded-db sync in test-embed,
as --no-install-project prevents optional extras from being active.
* ci: add local-ml extra to test-embed API install for smoke test
The smoke test starts the full API server which needs sentence-transformers
for local embeddings (default provider). Add local-ml extra to the install.
* ci: simplify extras with --all-extras and add slim pip smoke test
- Replace explicit --extra local-ml --extra embedded-db with --all-extras
for cleaner, more maintainable sync steps
- Add test-pip-slim job: tests hindsight-api-slim[embedded-db] without
local ML models, using Cohere for embeddings/reranking (mirrors Docker
slim smoke test approach)
* ci: simplify slim smoke test to health check only (mirrors Docker test)
* fix: register embedded profiles in CLI metadata on daemon start
When HindsightEmbedded(profile="myapp") starts a daemon, the profile
was never written to metadata.json or given a .env file, making it
invisible to `hindsight-embed profile list` and other CLI commands.
Add _register_profile() to DaemonEmbedManager which saves HINDSIGHT_API_*
config to ~/.hindsight/profiles/{name}.env and registers the port in
metadata.json. Called after a successful new daemon start and when the
daemon is already running, so orphaned profiles also get registered on
next use.
* fix: truncate documents exceeding LiteLLM reranker context limit
Add HINDSIGHT_API_RERANKER_LITELLM_MAX_TOKENS_PER_DOC env var for both
litellm and litellm-sdk reranker providers. When set, documents are
truncated to the configured token limit using tiktoken (cl100k_base)
before being sent to the reranker, preventing BadRequestError for
models with small context windows (e.g. 1024-token limit).
* refactor: use shared _tiktoken_encoder for doc truncation in LiteLLM reranker
* refactor: use _get_tiktoken_encoding() consistently, remove eager module-level encoder instance
* doc: add HINDSIGHT_API_RERANKER_LITELLM_MAX_TOKENS_PER_DOC to configuration reference
