* feat: add Agno integration with Hindsight memory toolkit
Add hindsight-agno package providing Hindsight memory tools (retain,
recall, reflect) as an Agno Toolkit, following the same pattern as
Agno's Mem0Tools. Includes per-user bank isolation, global config,
bank auto-creation, and memory_instructions() for system prompt
injection.
Also adds cookbook documentation page with architecture diagrams,
quick start examples, and configuration reference.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: remove n8n blog post, add Agno icon, bind to release process
- Remove n8n blog post from the agno integration branch
- Add Agno logo icon and map hindsight-agno SDK tag in CookbookGrid
- Add hindsight-agno to release.sh PYTHON_PACKAGES array
- Add build, publish, artifact upload, and release asset steps in release.yml
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: remove cookbook page (moved to hindsight-cookbook repo)
The Agno cookbook application now lives in
vectorize-io/hindsight-cookbook/applications/agno-memory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent silent memory loss on consolidation LLM failure
When all LLM retries are exhausted during consolidation, memories were
being marked consolidated_at unconditionally, permanently excluding them
from future consolidation runs without producing any observations.
Fix with two complementary mechanisms:
- Adaptive batch splitting: on LLM failure, the batch is halved and
retried recursively down to batch_size=1, recovering most transient
failures (rate limits, Pydantic validation on long prompts) without
operator intervention
- consolidation_failed_at column: only single-memory batches that still
fail after all retries are marked here instead of consolidated_at, so
they remain visible and retryable
- New API endpoint POST /v1/default/banks/{bank_id}/consolidation/retry-failed
resets these memories for the next consolidation run
* chore: regenerate OpenAPI spec
* fix: rename consolidation endpoint from /retry-failed to /recover
* fix: add consolidation_failed_at column, adaptive batch splitting, and recovery API
- Migration a3b4c5d6e7f8: add consolidation_failed_at TIMESTAMPTZ column to
memory_units with an index for efficient failure queries; properly chains off
g7h8i9j0k1l2 (backsweep_orphan_observations)
- Consolidator: filter pending memories with consolidation_failed_at IS NULL
so failed memories are not re-fetched in an infinite loop
- Consolidator: adaptive batch splitting — when a batch exhausts all 3 LLM
retries, halve it and retry sub-batches recursively; only single-memory
batches that also exhaust all retries get consolidation_failed_at set
- New tests (9 total) covering: adaptive splitting recovers all memories,
larger batch splitting, single-memory permanent failure, exclusion from
next run, partial batch failure, recover resets columns, recover returns
0 when none failed, recover-then-consolidate succeeds, HTTP endpoint
* chore: regenerate Go, Python, TypeScript clients with recover consolidation endpoint
* feat: add Recover Consolidation action to bank Actions dropdown
* style: apply ruff formatting to http.py and config.py
* fix: handle consolidation scope in large batch test mock LLM
The mock LLM was returning {"facts": ...} for ALL calls including consolidation.
Consolidation doesn't use skip_validation=True so it expects a _ConsolidationBatchResponse
instance, not a raw dict. Before this PR consolidation silently swallowed the AttributeError
(failed=False was returned); now failed=True triggers adaptive splitting and timeouts.
Fix: return _ConsolidationBatchResponse() when scope=="consolidation".
* fix: restrict claude-agent-sdk to macOS platform only (no Linux wheel available)
Also fix pre-existing type errors: use setattr for XLM-RoBERTa monkey-patch
and add missing reranker_local_fp16/bucket_batching/batch_size fields to main.py config constructor.
* fix: add UV_INDEX_STRATEGY=unsafe-best-match to fix markupsafe cp314 wheel conflict
PyTorch CPU index serves markupsafe==3.0.3 with only cp314 wheels.
uv's default first-index strategy stops at the first index with any version
even if no compatible wheel exists. unsafe-best-match searches all indices
for the best compatible wheel, falling back to PyPI for markupsafe.
* fix: use explicit pytorch index to prevent markupsafe wheel conflict
Configure the pytorch CPU index as explicit=true in pyproject.toml so it is
ONLY used for torch (via [tool.uv.sources]). All other packages (including
markupsafe) are resolved exclusively from PyPI, preventing the pytorch index
from serving incompatible cp314-only wheels for non-pytorch packages.
Remove UV_INDEX and UV_INDEX_STRATEGY from CI workflow (no longer needed
since the index is now configured in pyproject.toml).
* ci: trigger CI run
* ci: retry trigger
* ci: trigger after remote URL fix
* ci: add workflow_dispatch to unblock manual trigger
* fix: remove empty env blocks left after UV_INDEX removal
* fix: add type: ignore for optional claude_agent_sdk imports (macOS-only)
* fix: correct type: ignore rules for claude_agent_sdk and fix utcnow deprecation
* feat(retain): add verbatim extraction mode
Adds retain_extraction_mode="verbatim" that stores each chunk as-is
without LLM summarization. The LLM still runs to extract entities,
temporal info, and location for full indexability — only the fact text
is replaced with the original chunk content (one memory per chunk).
Useful for RAG-style indexing and benchmarks where original text
must be preserved in memory.
- Add "verbatim" to RETAIN_EXTRACTION_MODES in config.py
- Add VERBATIM_FACT_EXTRACTION_PROMPT with instructions to preserve text
- Add _collapse_to_verbatim() post-processing to enforce 1 fact/chunk
- Expose in bank config UI dropdown with updated description
- Update configuration.md docs with verbatim mode description
- Add unit test for _collapse_to_verbatim and integration test via LLM
- Fix pre-existing main.py CLI override missing new reranker fields
- Fix pre-existing cross_encoder.py ty type error via setattr
* refactor(retain): verbatim mode skips 'what' field entirely
Instead of asking the LLM to echo the chunk text back into 'what' and
then discarding it, verbatim mode now uses a dedicated schema
(VerbatimExtractedFact) that omits the 'what' field altogether.
The LLM only returns metadata (entities, temporal info, location, who),
saving output tokens and avoiding any risk of paraphrasing before the
backfill.
- Add VerbatimExtractedFact / VerbatimFactExtractionResponse models
- Verbatim mode skips causal-relations section (nothing to relate causally)
- _extract_facts_from_chunk: allow missing 'what' in verbatim mode,
set combined_text="" (backfilled by _collapse_to_verbatim)
- Update verbatim prompt to say DO NOT include 'what'
* feat(retain): add index_only extraction mode
Zero-LLM retain mode: chunks are stored as-is with no LLM call, no
entity extraction, and no temporal indexing. Embeddings still run for
semantic search. User-provided entities via RetainContent.entities
are the sole source of entity data.
Early return placed before the batch-API check so no LLM queue or
concurrency locks are acquired.
- Add "index_only" to RETAIN_EXTRACTION_MODES
- Add _extract_facts_index_only() with pure Python chunking path
- Add to UI dropdown and update description
- Update configuration.md with index_only docs and table entry
- Add unit test asserting zero token usage and exact text preservation
* feat(retain): add named retain strategies
Allows mixing extraction modes in a single bank via named strategies.
Each strategy is a set of hierarchical config overrides (extraction_mode,
chunk_size, entity_labels, entities_allow_free_form, etc.) applied on
top of the resolved bank config at retain time.
- retain_strategies: dict of strategy_name → config overrides (bank config)
- retain_default_strategy: default strategy when none specified (bank config)
- strategy field on /retain request: per-call override
- apply_strategy() in config_resolver applies overrides via dataclasses.replace()
- strategy propagates through retain_batch_async → _retain_batch_async_internal
and through the async worker task payload
- Any hierarchical field is overridable per strategy, including entity_labels
and entities_allow_free_form
- Docs updated with strategy configuration example and RRF fairness note
- Unit test for apply_strategy covering overrides, unknown strategy, and
non-hierarchical field filtering
* feat(retain): add per-item strategy and strategy tests
- Add `strategy` field to `MemoryItem` so individual items in a retain
request can override the request-level strategy
- Add `strategy` field to `FileRetainMetadata` for per-file strategy
override in file retain requests
- Group memory items by effective strategy in `api_retain`; each group
is processed as a separate batch, results are aggregated
- Thread strategy through `submit_async_file_retain` →
`_handle_file_convert_retain` → retain task payload
- Add `operation_ids` to `RetainResponse` for async requests with
mixed per-item strategies
- Add `test_strategy_overrides_extraction_mode_for_index_only`: unit
test verifying a named strategy with index_only bypasses the LLM
- Add `test_retain_request_per_item_strategy_field`: unit test for
per-item strategy grouping logic
* feat(ui): add retain strategies and default strategy to bank config UI
- Add StrategiesEditor component: per-strategy cards with name input and
JSON overrides textarea; supports add/remove; validates JSON inline
- Add Default Strategy text input (retain_default_strategy)
- Update RetainEdits type and retainSlice() to include both new fields
- Regenerate OpenAPI spec (retain_strategies, retain_default_strategy,
per-item strategy on MemoryItem/FileRetainMetadata, operation_ids on
RetainResponse)
* refactor(ui): move retain strategies into its own dedicated config section
* feat(ui): improve retain strategies UX and add strategy to document dialog
- Strategy form now includes entity section (free form toggle + entity labels editor)
- Default strategy selector moved outside tab panel, above strategy chips
- Strategy tabs redesigned with underline indicator style for clarity
- Remove strategy confirms with AlertDialog
- Fix tab re-render bug when typing strategy name (skipSyncRef)
- Add strategy field to Add New Document dialog (text + per-file for uploads)
- File upload collapsible uses same Document/Tags/Source tabbed layout
- API: validate empty strategy names in config_resolver
- api.ts: add strategy field to retain and uploadFiles types
* fix: forward strategy through HTTP layer and SDK; add integration test
- route.ts: extract and forward `strategy` from request body to retainBatch
- TypeScript SDK: accept and forward `strategy` in retainBatch options and per-item
- config_resolver.py: validate empty strategy name keys on update
- bank-config-view.tsx: merge entity fields into RetainStrategyForm, redesign strategy tabs with underline style, add confirmation dialog for removal, fix tab-reset-on-typing with skipSyncRef, move default strategy selector outside panel
- bank-selector.tsx: add strategy field to Add Document dialog (per-file in tabbed collapsible)
- test_retain.py: add end-to-end integration test verifying named strategy application (index_only = 0 LLM tokens)
* fix: regenerate TypeScript client with strategy field in RetainRequest/MemoryItem
- Regenerate OpenAPI spec to include strategy field in RetainRequest and MemoryItem
- Regenerate TypeScript client from updated spec
- Add strategy to MemoryItemInput interface
- Remove (item as any) cast now that strategy is properly typed
* rename: index_only extraction mode → chunks
* remove top-level strategy from RetainRequest; strategy is per-item only
* fix(clients): update Go and Python generated clients with strategy/operation_ids fields
* fix(ci): update hierarchical field count, add strategy to Rust MemoryItem initializers
* fix(go-client): minimal targeted YAML updates for strategy/operation_ids fields
* feat: add hindsight-hermes integration for Hermes Agent
* chore: add Hermes docs page, icon, and release process bindings
- Add cookbook page for Hermes integration (synced with README)
- Add Hermes icon and map hindsight-hermes SDK tag in CookbookGrid
- Add cookbook entry to index.mdx
- Add hindsight-hermes to release.sh PYTHON_PACKAGES array
- Add build, publish, artifact upload, and release asset steps in release.yml
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* blog: add n8n persistent memory workflows post
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: add cover image for n8n memory workflows post
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: update n8n cover image
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: remove broken screenshot references from n8n post
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: add Hindsight Cloud option and n8n Cloud guidance
- Add Cloud vs self-hosted setup paths in Step 1
- Show both Cloud and self-hosted URLs for retain/recall/reflect nodes
- Note that Cloud eliminates the localhost IP gotcha
- Mention n8n Cloud compatibility (requires Hindsight Cloud or public endpoint)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: update n8n post date to 2026-03-16
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: update n8n post with optimized content and fix accuracy
- Use optimized version of the blog post
- Fix blog cross-links to use date-prefixed URLs
- Fix retain response to match actual API (success, bank_id, items_count, async)
- Fix recall response to match actual API (text, type, entities — not confidence/source)
- Update title to "How to Add Persistent Memory to n8n Workflows"
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* blog: update n8n post title
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add compound tag filtering via tag_groups
Adds tag_groups to RecallRequest and ReflectRequest to express arbitrary
boolean tag predicates: leaf {tags, match}, and/or/not compounds.
Top-level groups are AND-ed. Existing tags/tags_match unchanged.
Examples:
Step filter AND user scope:
tag_groups: [{tags: ["step:5","step:8"], match: "any_strict"},
{tags: ["user:alice"], match: "all_strict"}]
Exclusion:
tag_groups: [{tags: ["user:alice"], match: "all_strict"},
{not: {tags: ["archived"], match: "any_strict"}}]
- Recursive SQL builder (build_tag_groups_where_clause) threads through
all 4 retrieval strategies (semantic/BM25, temporal, graph, MPFP)
- Python-side filter (filter_results_by_tag_groups) for post-traversal
- 22 new unit tests
- OpenAPI spec + all clients regenerated (Rust, Python, TypeScript, Go)
* fix: add tag_groups: None to Rust CLI struct initializers
* fix: add tag_groups: None to Rust client test RecallRequest initializer
* feat: reject tags+tag_groups together, add tag_groups integration tests
- Add model_validator to RecallRequest and ReflectRequest that returns 422
when both `tags` and `tag_groups` are set (mutually exclusive)
- Add 5 integration tests for tag_groups compound filtering:
* validation: 422 when both fields are set
* AND filter: two leaf groups (step scope AND user scope)
* OR compound: user:alice OR user:bob
* NOT compound: user:alice AND NOT archived
* Nested: user:alice AND (step:5 OR step:8)
* ci: trigger CI run
* docs: revamp sidebar with icon grid components and language support
- Merge Clients and Integrations sections into the developer sidebar
(removed top-level SDKs navbar item)
- Reorder sidebar: Architecture → API → Clients → Integrations → Hosting
- Unify icon system using react-icons (LuXxx/SiXxx) via customProps.icon
- Add uppercase section titles with increased spacing and reduced indentation
- Rename Node.js → "JavaScript / TypeScript" with TypeScript icon
- Add reusable IconGrid and SupportedGrids components (ClientsGrid,
IntegrationsGrid, LLMProvidersGrid)
- Use grids in FAQ, Models, Overview, and Quick Start pages
- Convert developer/index.md, models.md, faq.md to MDX for JSX support
* fix: use inline style for label color to prevent link color inheritance
* fix: label visibility and rename JavaScript/TypeScript to TypeScript
* feat: add HTTP client to grid and OpenAI Compatible to LLM providers grid
- API: POST /v1/default/banks/{bank_id}/operations/{operation_id}/retry
resets status to pending so the worker re-executes the task
- UI: Retry button on failed operations in the operations view
- Control plane proxy route + ControlPlaneClient.retryOperation()
- Updated OpenAPI spec, all generated clients, and operations docs
* doc: add pydantic-ai-persistent-memory blog post
* doc: update Pydantic AI blog cover image
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* doc: SEO-optimized rewrite of Pydantic AI blog post
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: filter operations by type + fix stale closure in auto-refresh
- Add `type` query param to GET /operations endpoint and engine layer
- Add operation type dropdown filter in Background Operations UI
- Fix auto-refresh interval using stale statusFilter/offset closure by
adding filter state to useEffect deps and wrapping loadOperations in
useCallback (fixes#522)
- Regenerate OpenAPI spec and all SDK clients
* fix: update Rust CLI list_operations call with new type parameter
* feat: add update document tags endpoint with observation invalidation
Adds PATCH /v1/default/banks/{bank_id}/documents/{document_id} to change
tags on a document without re-processing content.
- Updates tags on the document and all associated memory units atomically
- Invalidates observations derived from the document's memory units
- Resets consolidated_at on the document's own units for re-consolidation
- Also resets consolidated_at on co-source memories from other documents
that shared those observations (matching delete_document behavior)
- Triggers async consolidation when observations are invalidated
- 9 new tests covering all invalidation scenarios
UI: adds inline tag editor to the document detail panel in the control plane
Docs: new "Update Document Tags" section in documents.mdx with Python/JS examples
* refactor: simplify UpdateDocumentTagsResponse to {success: true}
* refactor: make PATCH /documents generic update_document endpoint
Renames update_document_tags → update_document (engine + HTTP + clients + UI).
Currently only tags are supported; the structure is open for future fields.
Tags are the only field with side effects (observation invalidation + re-consolidation).
* doc: add adding-memory-to-openclaw-with-hindsight blog post
* doc: update OpenClaw blog cover image
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* doc: update OpenClaw blog title
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* doc: add Hindsight Cloud note to external API section
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: mental model refresh history tracking and UI diff view
- DB migration: add history JSONB column to mental_models table
- Track previous content on each refresh in update_mental_model
- Add get_mental_model_history() engine method
- New GET /mental-models/{id}/history endpoint
- Control plane proxy route and getMentalModelHistory() in api.ts
- MentalModelDetailModal: add History tab with lazy loading, carousel
navigation (left=older, right=newer), word-level content diff view
* fix: resolve alembic migration head conflict for mental model history
* feat: mental model history tracking, side-by-side diff UI, and config flag
- Track content changes on every mental model update/refresh (persisted in JSONB history column)
- New GET /mental-models/{id}/history endpoint returning changes most-recent-first
- Side-by-side diff view in History tab (Before/After columns, line-level highlights)
- Actions dropdown in detail panel (Edit, Refresh, View History, Delete)
- HINDSIGHT_API_ENABLE_MENTAL_MODEL_HISTORY config flag (default: true)
- Also adds missing HINDSIGHT_API_ENABLE_OBSERVATION_HISTORY to configuration docs
- Python client wrapper method get_mental_model_history()
- Tests for history persistence (recorded, ordered, name-only skipped, missing returns None)
- Fix NameError: timezone not imported in update_mental_model
* fix: call get_mental_model_history before delete in doc example
* feat: add source facts token limits to consolidation and recall
- Add two new configurable (per-bank) parameters:
- consolidation_source_facts_max_tokens: total token budget for source
facts across all observations in the consolidation prompt (-1 = unlimited)
- consolidation_source_facts_max_tokens_per_observation: per-observation
cap so each observation gets a fair share of source facts (-1 = unlimited,
default 256)
- Both are also exposed as recall API parameters via SourceFactsIncludeOptions
(max_tokens and max_tokens_per_observation)
- Consolidation now uses resolve_full_config to respect bank-level overrides
- Improve consolidation prompt: temporal metadata (occurred_start=, | Involving:)
is now clearly separated from observation text, with a concrete example showing
the expected synthesis style and explicit rules not to copy raw fact lines
- Add tests for recall source facts capping and consolidation config forwarding
- Expose all three new fields in the control plane bank config UI
- Document new env vars in configuration.md
- Regenerate OpenAPI spec and all SDK clients
* fix: reorder observations UI fields and rename Label Groups to Entity Labels
* fix: revert Entities section title (only rename inner label)
* doc: add consolidation source facts and batch size fields to memory-banks docs
* feat: add observation history tracking and UI diff view
- Track observation changes over time in a JSONB history column,
appending each update's previous state (text, tags, dates, sources)
instead of overwriting
- Add HINDSIGHT_API_ENABLE_OBSERVATION_HISTORY config flag (default: true)
to toggle history recording
- Expose history field in get_memory_unit for observations
- Fix observations/[modelId] route that was proxying to wrong endpoint
- Add History tab in observation modal and History section in panel,
showing word-level and tag diffs between each change (newest first)
- Extract shared ObservationHistoryView component used by both modal and panel
- Add --random-port flag to start.sh to run multiple dev instances
- Scope Next.js distDir by port to prevent lock file collisions between instances
- Restyle consolidation pending badge (rounded-md with border) and add
inline refresh button; fix loading flicker on data refresh
* feat: dedicated observation history endpoint with source facts diff
- Add GET /memories/{id}/history endpoint returning enriched history with
resolved source fact texts and is_new flags per change
- Deprecate history field in GET /memories/{id} (always returns empty list)
- Reconstruct cumulative source facts per history entry by working backwards
from current state, marking newly added facts with is_new
- Replace inline history panel with "View History" button opening modal
- History modal fetches from dedicated endpoint lazily on tab switch
- Timeline view now opens MemoryDetailModal instead of side panel
- History view uses prev/next navigation (left = older, right = newer)
- Fix --random-port: pass dynamic API_PORT as HINDSIGHT_CP_DATAPLANE_API_URL
to control plane, preserving caller values over .env
* feat: allow per-request file parser selection with fallback chains
Clients can now specify which parser(s) to use when calling the file
retain endpoint, instead of being locked to the server-side default.
Changes:
- `parser` field added to `FileRetainRequest` (request-level default)
and `FileRetainMetadata` (per-file override); accepts a single name
or an ordered fallback chain (list)
- Resolution priority: per-file > request-level > server default
- `HINDSIGHT_API_FILE_PARSER` now accepts a comma-separated fallback
chain (e.g. `iris,markitdown`); fully backward-compatible
- New `HINDSIGHT_API_FILE_PARSER_ALLOWLIST` env var restricts which
parsers clients may request (defaults to all registered parsers)
- Invalid/disallowed parser names are rejected with HTTP 400
- `FileParserRegistry.convert_with_fallback()` tries each parser in
order, falling back on UnsupportedFileTypeError, empty content, or
any other error
- Worker updated to use the fallback chain stored per-task
- OpenAPI spec and all generated clients regenerated
* fix: handle on_file_convert_complete hook and rebase onto main
- Return ConvertResult dataclass from convert_with_fallback() instead
of a plain str, carrying both the content and the winning parser name
- Use winning_parser_name in the on_file_convert_complete hook so
parser_name reflects the parser that actually succeeded, not the chain
- Update all test calls to submit_async_file_retain() to use the new
per-item parser field instead of the removed top-level parser= kwarg
* docs: document HINDSIGHT_API_FILE_PARSER fallback chain and ALLOWLIST
* feat: add source facts token limits to consolidation and recall
- Add two new configurable (per-bank) parameters:
- consolidation_source_facts_max_tokens: total token budget for source
facts across all observations in the consolidation prompt (-1 = unlimited)
- consolidation_source_facts_max_tokens_per_observation: per-observation
cap so each observation gets a fair share of source facts (-1 = unlimited,
default 256)
- Both are also exposed as recall API parameters via SourceFactsIncludeOptions
(max_tokens and max_tokens_per_observation)
- Consolidation now uses resolve_full_config to respect bank-level overrides
- Improve consolidation prompt: temporal metadata (occurred_start=, | Involving:)
is now clearly separated from observation text, with a concrete example showing
the expected synthesis style and explicit rules not to copy raw fact lines
- Add tests for recall source facts capping and consolidation config forwarding
- Expose all three new fields in the control plane bank config UI
- Document new env vars in configuration.md
- Regenerate OpenAPI spec and all SDK clients
* fix: reorder observations UI fields and rename Label Groups to Entity Labels
* fix: revert Entities section title (only rename inner label)
* doc: add consolidation source facts and batch size fields to memory-banks docs
* doc: update cookbook
* fix(cookbook): preserve tag keys during sync, strip local .md links
- Fix extract_tags_from_readme/notebook to return dict[str,str] preserving
sdk/topic keys instead of bare values, preventing topics like
"Customer Service" from being misclassified as SDK
- Add strip_local_md_links() to remove relative .md references that
would cause broken link errors in Docusaurus build
* ci: run test-doc-examples independently without waiting for test-rust-cli
Build the CLI directly in the job instead of downloading the artifact,
so test-doc-examples can start at the beginning in parallel with all other jobs.
* feat: webhook system with task-owned retry, retain.completed event, and UI
- New webhook system: register per-bank webhooks with HMAC signing, configurable
HTTP method/timeout/headers/params (http_config JSONB), and PATCH support
- Webhook deliveries run as async_operations (webhook_delivery type) with
task-owned retry via RetryTaskAt exception and exponential backoff
(60s / 5m / 30m / 2h / 8h, max 6 attempts)
- New retain.completed event fires per-document for both sync and async retain
- Delivery debug info (status code, response body) stored in result_metadata
- Control plane UI: webhooks tab per bank with create/edit/delete and a
deliveries table with cursor pagination and expandable response details
- 28 webhook tests covering HMAC signing, delivery retries, CRUD endpoints,
PATCH update, and retain.completed queuing
- Docs page at developer/api/webhooks documenting event payloads and delivery
- OpenAPI spec and all client SDKs (Python, TypeScript, Rust, Go) regenerated
* fix: update tests for task-owned retry model and guard _webhook_manager attribute
- test_worker.py: test_executor_exception_triggers_retry now raises RetryTaskAt
(plain exceptions are immediate failures in the new system); rename
test_executor_exception_marks_failed_after_max_retries to
test_executor_exception_marks_failed_immediately to reflect new semantics
- test_batch_api.py: remove max_retries kwarg from WorkerPoller constructor
- memory_engine.py: use getattr for _webhook_manager in _fire_retain_webhook
to avoid AttributeError when engine is created without __init__ (tests)
* fix: remove max_retries from benchmark WorkerPoller call
* fix(webhooks): transactional outbox, observations_deleted tracking, sidebar
- Queue webhook delivery rows atomically with the primary operation using the
transactional outbox pattern — prevents lost events on process crash:
- Retain (sync + async): outbox_callback passed into orchestrator.retain_batch
and called inside the DB transaction, replacing the post-commit fire call
- Consolidation: new _mark_operation_completed_and_fire_webhook combines the
status UPDATE and webhook INSERT in one transaction
- Added fire_event_with_conn() to WebhookManager for in-connection delivery
- Track observations_deleted count in consolidation stats and expose it in the
consolidation.completed webhook payload (was always None)
- Add Webhooks page to docs sidebar
- Document at-least-once delivery guarantee with operation_id dedup guidance
* fix(ui): add retain.completed to available webhook event types
* feat(ui): add delete confirmation dialog for webhooks
* fix(webhooks): include operation_id in task_payload so delivery is marked completed
The task_payload JSON was missing the operation_id field, causing execute_task
to see operation_id=None and skip _mark_operation_completed — leaving every
delivery row stuck in 'pending' forever.
Added a test that inserts a real async_operations row and verifies the status
transitions to 'completed' after a successful execute_task call.
* style: fix prettier formatting in webhooks-view
* Add LiteLLM persistent memory blog post
* doc: add blog image for LiteLLM post
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* doc: update cookbook
* fix(cookbook): preserve tag keys during sync, strip local .md links
- Fix extract_tags_from_readme/notebook to return dict[str,str] preserving
sdk/topic keys instead of bare values, preventing topics like
"Customer Service" from being misclassified as SDK
- Add strip_local_md_links() to remove relative .md references that
would cause broken link errors in Docusaurus build
* ci: run test-doc-examples independently without waiting for test-rust-cli
Build the CLI directly in the job instead of downloading the artifact,
so test-doc-examples can start at the beginning in parallel with all other jobs.
* feat: add Pydantic AI integration to CI, release pipeline, and docs
- Add test-pydantic-ai-integration job to CI (test.yml)
- Add build, publish, and artifact steps to release workflow (release.yml)
- Add hindsight-integrations/pydantic-ai to release.sh version bumping
- Add Pydantic AI documentation page (sdks/integrations/pydantic-ai.md)
- Add Pydantic AI entry to sidebar with icon
* docs: remove Requirements section from pydantic-ai integration page
* feat: add tags filtering and fix offset pagination docs for list documents API
- Add `tags` and `tags_match` query params to GET /banks/{bank_id}/documents
- Supports any, all, any_strict, all_strict matching modes (default: any_strict)
- Fix `q` param description — it's a case-insensitive substring match on document ID only
- Add tests for offset pagination and all tags_match modes
- Regenerate OpenAPI spec and Python/TypeScript/Go clients
- Document the new filtering options in docs/developer/api/documents.mdx
* fix(cli): pass new tags/tags_match args to list_documents
* feat: add Pydantic AI integration to CI, release pipeline, and docs
- Add test-pydantic-ai-integration job to CI (test.yml)
- Add build, publish, and artifact steps to release workflow (release.yml)
- Add hindsight-integrations/pydantic-ai to release.sh version bumping
- Add Pydantic AI documentation page (sdks/integrations/pydantic-ai.md)
- Add Pydantic AI entry to sidebar with icon
* docs: remove Requirements section from pydantic-ai integration page
* docs: move entity labels detail to memory-banks, simplify retain overview
* docs: move entity labels blurb under entity-recognition section in retain
* docs: update metadata filtering FAQ to cover entity graph retrieval and entity labels tag option
* docs: enable TOC and fix missing separators in FAQ
* docs: add benchmarks leaderboard screenshot and link to models page
* docs: add 'Which model should I use?' FAQ entry with leaderboard screenshot
* docs: fix leaderboard description to cover retain, reflect, and observations
* feat: support timestamp="unset" to retain content without a date
When callers retain timeless content (e.g. fictional documents, static
reference material), passing timestamp="unset" now skips the utcnow()
default so mentioned_at is stored as NULL instead of an artificial date.
- HTTP: validate_timestamp recognises "unset" sentinel and threads it
through api_retain as event_date=None (key present, value None), which
the orchestrator distinguishes from key-absent (still defaults to now)
- Orchestrator: new branching logic separates "key absent" → utcnow()
from "key present but None" → no date
- types.py: RetainContent.event_date and ProcessedFact.mentioned_at are
now datetime | None; removed the unused _now_utc factory
- fact_extraction.py: all event_date params accept datetime | None;
_build_user_message emits "Event Date: Unknown" when None; removed
mentioned_at from the Fact LLM response model (LLM never sets it)
- embedding_processing: skip date suffix when fact_date is None
- entity_resolver: COALESCE(event_date, now()) for first_seen/last_seen
so entities table NOT NULL constraint is preserved
- link_utils: skip temporal linking for units without event_date
- Migration aa2b3c4d5e6f: DROP NOT NULL on memory_units.event_date
- Tests: test_retain_no_timestamp and test_retain_omit_timestamp_defaults_to_now
- Docs + OpenAPI + TypeScript client updated
* refactor: replace _TIMESTAMP_UNKNOWN sentinel with plain string comparison
The sentinel object() was only needed to distinguish "unset" from None
at the boundary — but since the field type is datetime | str | None,
"unset" can pass through the validator unchanged and be compared directly.
* chore: regenerate OpenAPI spec and clients after timestamp type change
timestamp field is now datetime | str | None to accept the "unset" sentinel value.
* feat: observation_scopes field to drive observations granularity
* fix(migration): make a2b3c4d5e6f7 a no-op to fix CI on fresh DB
The z1u2v3w4x5y6 migration already creates observation_scopes directly,
so the rename migration fails on fresh installs where observation_tags
never existed.
* chore: remove no-op migration a2b3c4d5e6f7
* feat: regenerate clients with observation_scopes field
- Add observation_scopes to OpenAPI spec and all generated clients
- Fix Rust build.rs to handle anyOf with >2 variants containing null
(previously only handled 2-item anyOf, causing progenitor to panic
on the observation_scopes union type)
* fix(rust): add observation_scopes: None to MemoryItem struct literals
* fix(api): add title to observation_scopes Field for deterministic client generation
Adding title="ObservationScopes" makes the inline anyOf schema use
the explicit name instead of deriving it from the field name, which
was non-deterministic between arm64 (macOS) and amd64 (CI) Docker.
Also fixes description: "each entity" -> "each tag".
* fix(scripts): use linux/amd64 Docker for client generation to ensure reproducibility
Both Python and Go client generation now use --platform linux/amd64
Docker, ensuring identical output on macOS arm64 (local) and Linux
amd64 (CI). Also switches Go from JAR+Java to Docker to eliminate
Java version variability.
* chore: update generated clients to API v0.4.14
* fix(test): add retry logic to test_retain_chinese_content to handle non-deterministic LLM output
* fix(test): mark test_retain_chinese_content as xfail due to non-deterministic LLM translation
* feat: filter graph memories with tags
* fix(cli): pass new q/tags/tags_match args to get_graph
* docs: use CodeSnippet for tags_match examples in recall.mdx
* fix: handle observations regeneration when memories get deleted
* feat: add clear_memory_observations endpoint and regenerate clients
- Add DELETE /banks/{id}/memories/{memory_id}/observations endpoint
- Add observations lifecycle/invalidation section to docs
- Regenerate OpenAPI spec and all clients (Python, TypeScript, Go, Rust)
* refactor: use dedicated response model for clear_memory_observations, remove code example from docs
