Commit graph

112 commits

Author SHA1 Message Date
Ben
a01bb18bc4
blog: Time-Aware Spreading Activation for Memory Graphs (#547)
doc: add blog post on time-aware spreading activation for memory graphs
2026-03-12 12:55:14 -04:00
Ben
5b360c83d2
doc: Run Hindsight with Ollama: Local AI Memory, No API Keys Needed (#536)
* doc: add run-hindsight-with-ollama blog post
2026-03-11 11:39:59 -04:00
Nicolò Boschi
00ac3d8834
doc: add 0.4.17 release blog post (#538) 2026-03-10 17:40:10 +01:00
Nicolò Boschi
2191654b1f Release v0.4.17
- Update version to 0.4.17 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-litellm, hindsight-crewai, hindsight-pydantic-ai, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- OpenClaw integration: hindsight-integrations/openclaw
- AI SDK integration: hindsight-integrations/ai-sdk
- Chat SDK integration: hindsight-integrations/chat
- Helm chart
- Sync documentation to version-0.4
2026-03-10 17:18:35 +01:00
Nicolò Boschi
dcaacbe407
feat: add manual retry for failed async operations (#537)
- API: POST /v1/default/banks/{bank_id}/operations/{operation_id}/retry
  resets status to pending so the worker re-executes the task
- UI: Retry button on failed operations in the operations view
- Control plane proxy route + ControlPlaneClient.retryOperation()
- Updated OpenAPI spec, all generated clients, and operations docs
2026-03-10 17:16:11 +01:00
Ben
7bcf26097c
doc: Your Pydantic AI Agent Forgets You After Every Run. Fix It in 5 Lines. (#531)
* doc: add pydantic-ai-persistent-memory blog post

* doc: update Pydantic AI blog cover image

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* doc: SEO-optimized rewrite of Pydantic AI blog post

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 16:39:20 -04:00
Nicolò Boschi
f7a60f898d
feat: filter operations by type + fix stale auto-refresh closure (#522) (#527)
* feat: filter operations by type + fix stale closure in auto-refresh

- Add `type` query param to GET /operations endpoint and engine layer
- Add operation type dropdown filter in Background Operations UI
- Fix auto-refresh interval using stale statusFilter/offset closure by
  adding filter state to useEffect deps and wrapping loadOperations in
  useCallback (fixes #522)
- Regenerate OpenAPI spec and all SDK clients

* fix: update Rust CLI list_operations call with new type parameter
2026-03-09 13:17:00 +01:00
Nicolò Boschi
1b4ad7f435
feat: change tags for a document (#517)
* feat: add update document tags endpoint with observation invalidation

Adds PATCH /v1/default/banks/{bank_id}/documents/{document_id} to change
tags on a document without re-processing content.

- Updates tags on the document and all associated memory units atomically
- Invalidates observations derived from the document's memory units
- Resets consolidated_at on the document's own units for re-consolidation
- Also resets consolidated_at on co-source memories from other documents
  that shared those observations (matching delete_document behavior)
- Triggers async consolidation when observations are invalidated
- 9 new tests covering all invalidation scenarios

UI: adds inline tag editor to the document detail panel in the control plane
Docs: new "Update Document Tags" section in documents.mdx with Python/JS examples

* refactor: simplify UpdateDocumentTagsResponse to {success: true}

* refactor: make PATCH /documents generic update_document endpoint

Renames update_document_tags → update_document (engine + HTTP + clients + UI).
Currently only tags are supported; the structure is open for future fields.
Tags are the only field with side effects (observation invalidation + re-consolidation).
2026-03-07 09:00:13 +01:00
Ben
d9d7021a49
doc: Upgrading OpenClaw's Memory with Hindsight (#515)
* doc: add adding-memory-to-openclaw-with-hindsight blog post

* doc: update OpenClaw blog cover image

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* doc: update OpenClaw blog title

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* doc: add Hindsight Cloud note to external API section

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 12:35:12 -05:00
Nicolò Boschi
e2baca8bfe
feat: mental model history tracking and UI diff view (#516)
* feat: mental model refresh history tracking and UI diff view

- DB migration: add history JSONB column to mental_models table
- Track previous content on each refresh in update_mental_model
- Add get_mental_model_history() engine method
- New GET /mental-models/{id}/history endpoint
- Control plane proxy route and getMentalModelHistory() in api.ts
- MentalModelDetailModal: add History tab with lazy loading, carousel
  navigation (left=older, right=newer), word-level content diff view

* fix: resolve alembic migration head conflict for mental model history

* feat: mental model history tracking, side-by-side diff UI, and config flag

- Track content changes on every mental model update/refresh (persisted in JSONB history column)
- New GET /mental-models/{id}/history endpoint returning changes most-recent-first
- Side-by-side diff view in History tab (Before/After columns, line-level highlights)
- Actions dropdown in detail panel (Edit, Refresh, View History, Delete)
- HINDSIGHT_API_ENABLE_MENTAL_MODEL_HISTORY config flag (default: true)
- Also adds missing HINDSIGHT_API_ENABLE_OBSERVATION_HISTORY to configuration docs
- Python client wrapper method get_mental_model_history()
- Tests for history persistence (recorded, ordered, name-only skipped, missing returns None)
- Fix NameError: timezone not imported in update_mental_model

* fix: call get_mental_model_history before delete in doc example
2026-03-06 17:50:48 +01:00
Nicolò Boschi
576473b6aa
feat: observation history tracking and diff UI (#513)
* feat: add source facts token limits to consolidation and recall

- Add two new configurable (per-bank) parameters:
  - consolidation_source_facts_max_tokens: total token budget for source
    facts across all observations in the consolidation prompt (-1 = unlimited)
  - consolidation_source_facts_max_tokens_per_observation: per-observation
    cap so each observation gets a fair share of source facts (-1 = unlimited,
    default 256)
- Both are also exposed as recall API parameters via SourceFactsIncludeOptions
  (max_tokens and max_tokens_per_observation)
- Consolidation now uses resolve_full_config to respect bank-level overrides
- Improve consolidation prompt: temporal metadata (occurred_start=, | Involving:)
  is now clearly separated from observation text, with a concrete example showing
  the expected synthesis style and explicit rules not to copy raw fact lines
- Add tests for recall source facts capping and consolidation config forwarding
- Expose all three new fields in the control plane bank config UI
- Document new env vars in configuration.md
- Regenerate OpenAPI spec and all SDK clients

* fix: reorder observations UI fields and rename Label Groups to Entity Labels

* fix: revert Entities section title (only rename inner label)

* doc: add consolidation source facts and batch size fields to memory-banks docs

* feat: add observation history tracking and UI diff view

- Track observation changes over time in a JSONB history column,
  appending each update's previous state (text, tags, dates, sources)
  instead of overwriting
- Add HINDSIGHT_API_ENABLE_OBSERVATION_HISTORY config flag (default: true)
  to toggle history recording
- Expose history field in get_memory_unit for observations
- Fix observations/[modelId] route that was proxying to wrong endpoint
- Add History tab in observation modal and History section in panel,
  showing word-level and tag diffs between each change (newest first)
- Extract shared ObservationHistoryView component used by both modal and panel
- Add --random-port flag to start.sh to run multiple dev instances
- Scope Next.js distDir by port to prevent lock file collisions between instances
- Restyle consolidation pending badge (rounded-md with border) and add
  inline refresh button; fix loading flicker on data refresh

* feat: dedicated observation history endpoint with source facts diff

- Add GET /memories/{id}/history endpoint returning enriched history with
  resolved source fact texts and is_new flags per change
- Deprecate history field in GET /memories/{id} (always returns empty list)
- Reconstruct cumulative source facts per history entry by working backwards
  from current state, marking newly added facts with is_new
- Replace inline history panel with "View History" button opening modal
- History modal fetches from dedicated endpoint lazily on tab switch
- Timeline view now opens MemoryDetailModal instead of side panel
- History view uses prev/next navigation (left = older, right = newer)
- Fix --random-port: pass dynamic API_PORT as HINDSIGHT_CP_DATAPLANE_API_URL
  to control plane, preserving caller values over .env
2026-03-06 16:16:05 +01:00
Nicolò Boschi
99220d0527
feat: per-request file parser selection with fallback chains (#514)
* feat: allow per-request file parser selection with fallback chains

Clients can now specify which parser(s) to use when calling the file
retain endpoint, instead of being locked to the server-side default.

Changes:
- `parser` field added to `FileRetainRequest` (request-level default)
  and `FileRetainMetadata` (per-file override); accepts a single name
  or an ordered fallback chain (list)
- Resolution priority: per-file > request-level > server default
- `HINDSIGHT_API_FILE_PARSER` now accepts a comma-separated fallback
  chain (e.g. `iris,markitdown`); fully backward-compatible
- New `HINDSIGHT_API_FILE_PARSER_ALLOWLIST` env var restricts which
  parsers clients may request (defaults to all registered parsers)
- Invalid/disallowed parser names are rejected with HTTP 400
- `FileParserRegistry.convert_with_fallback()` tries each parser in
  order, falling back on UnsupportedFileTypeError, empty content, or
  any other error
- Worker updated to use the fallback chain stored per-task
- OpenAPI spec and all generated clients regenerated

* fix: handle on_file_convert_complete hook and rebase onto main

- Return ConvertResult dataclass from convert_with_fallback() instead
  of a plain str, carrying both the content and the winning parser name
- Use winning_parser_name in the on_file_convert_complete hook so
  parser_name reflects the parser that actually succeeded, not the chain
- Update all test calls to submit_async_file_retain() to use the new
  per-item parser field instead of the removed top-level parser= kwarg

* docs: document HINDSIGHT_API_FILE_PARSER fallback chain and ALLOWLIST
2026-03-06 16:15:43 +01:00
Nicolò Boschi
5d05962db0
feat: add source facts token limits to consolidation and recall (#509)
* feat: add source facts token limits to consolidation and recall

- Add two new configurable (per-bank) parameters:
  - consolidation_source_facts_max_tokens: total token budget for source
    facts across all observations in the consolidation prompt (-1 = unlimited)
  - consolidation_source_facts_max_tokens_per_observation: per-observation
    cap so each observation gets a fair share of source facts (-1 = unlimited,
    default 256)
- Both are also exposed as recall API parameters via SourceFactsIncludeOptions
  (max_tokens and max_tokens_per_observation)
- Consolidation now uses resolve_full_config to respect bank-level overrides
- Improve consolidation prompt: temporal metadata (occurred_start=, | Involving:)
  is now clearly separated from observation text, with a concrete example showing
  the expected synthesis style and explicit rules not to copy raw fact lines
- Add tests for recall source facts capping and consolidation config forwarding
- Expose all three new fields in the control plane bank config UI
- Document new env vars in configuration.md
- Regenerate OpenAPI spec and all SDK clients

* fix: reorder observations UI fields and rename Label Groups to Entity Labels

* fix: revert Entities section title (only rename inner label)

* doc: add consolidation source facts and batch size fields to memory-banks docs
2026-03-06 13:01:33 +01:00
Nicolò Boschi
1e5aa7de4d
doc: add 0.4.16 release blog post and changelog (#502) 2026-03-05 18:29:55 +01:00
Nicolò Boschi
58fdac44f7 Release v0.4.16
- Update version to 0.4.16 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-litellm, hindsight-crewai, hindsight-pydantic-ai, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- OpenClaw integration: hindsight-integrations/openclaw
- AI SDK integration: hindsight-integrations/ai-sdk
- Chat SDK integration: hindsight-integrations/chat
- Helm chart
- Sync documentation to version-0.4
2026-03-05 17:54:59 +01:00
Ben
7ed57fdd85
doc: Give Your OpenAI App a Memory in 5 Minutes (#498)
* doc: add add-memory-to-openai-application blog post


---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:34:04 -05:00
Ben
3f2a6ec9ce
doc: add MCP blog post (#492)
* doc: add MCP agent memory blog post
2026-03-04 15:02:58 -05:00
Nicolò Boschi
abbf874d84
feat: webhook system with retain.completed event, UI, and docs (#487)
* doc: update cookbook

* fix(cookbook): preserve tag keys during sync, strip local .md links

- Fix extract_tags_from_readme/notebook to return dict[str,str] preserving
  sdk/topic keys instead of bare values, preventing topics like
  "Customer Service" from being misclassified as SDK
- Add strip_local_md_links() to remove relative .md references that
  would cause broken link errors in Docusaurus build

* ci: run test-doc-examples independently without waiting for test-rust-cli

Build the CLI directly in the job instead of downloading the artifact,
so test-doc-examples can start at the beginning in parallel with all other jobs.

* feat: webhook system with task-owned retry, retain.completed event, and UI

- New webhook system: register per-bank webhooks with HMAC signing, configurable
  HTTP method/timeout/headers/params (http_config JSONB), and PATCH support
- Webhook deliveries run as async_operations (webhook_delivery type) with
  task-owned retry via RetryTaskAt exception and exponential backoff
  (60s / 5m / 30m / 2h / 8h, max 6 attempts)
- New retain.completed event fires per-document for both sync and async retain
- Delivery debug info (status code, response body) stored in result_metadata
- Control plane UI: webhooks tab per bank with create/edit/delete and a
  deliveries table with cursor pagination and expandable response details
- 28 webhook tests covering HMAC signing, delivery retries, CRUD endpoints,
  PATCH update, and retain.completed queuing
- Docs page at developer/api/webhooks documenting event payloads and delivery
- OpenAPI spec and all client SDKs (Python, TypeScript, Rust, Go) regenerated

* fix: update tests for task-owned retry model and guard _webhook_manager attribute

- test_worker.py: test_executor_exception_triggers_retry now raises RetryTaskAt
  (plain exceptions are immediate failures in the new system); rename
  test_executor_exception_marks_failed_after_max_retries to
  test_executor_exception_marks_failed_immediately to reflect new semantics
- test_batch_api.py: remove max_retries kwarg from WorkerPoller constructor
- memory_engine.py: use getattr for _webhook_manager in _fire_retain_webhook
  to avoid AttributeError when engine is created without __init__ (tests)

* fix: remove max_retries from benchmark WorkerPoller call

* fix(webhooks): transactional outbox, observations_deleted tracking, sidebar

- Queue webhook delivery rows atomically with the primary operation using the
  transactional outbox pattern — prevents lost events on process crash:
  - Retain (sync + async): outbox_callback passed into orchestrator.retain_batch
    and called inside the DB transaction, replacing the post-commit fire call
  - Consolidation: new _mark_operation_completed_and_fire_webhook combines the
    status UPDATE and webhook INSERT in one transaction
  - Added fire_event_with_conn() to WebhookManager for in-connection delivery

- Track observations_deleted count in consolidation stats and expose it in the
  consolidation.completed webhook payload (was always None)

- Add Webhooks page to docs sidebar

- Document at-least-once delivery guarantee with operation_id dedup guidance

* fix(ui): add retain.completed to available webhook event types

* feat(ui): add delete confirmation dialog for webhooks

* fix(webhooks): include operation_id in task_payload so delivery is marked completed

The task_payload JSON was missing the operation_id field, causing execute_task
to see operation_id=None and skip _mark_operation_completed — leaving every
delivery row stuck in 'pending' forever.

Added a test that inserts a real async_operations row and verifies the status
transitions to 'completed' after a successful execute_task call.

* style: fix prettier formatting in webhooks-view
2026-03-04 14:17:01 +01:00
Ben
3857a30491
blog: add LiteLLM persistent memory post (#481)
* Add LiteLLM persistent memory blog post

* doc: add blog image for LiteLLM post

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 14:22:42 -05:00
Nicolò Boschi
3d87ef5cee
doc: update cookbook (#479)
* doc: update cookbook

* fix(cookbook): preserve tag keys during sync, strip local .md links

- Fix extract_tags_from_readme/notebook to return dict[str,str] preserving
  sdk/topic keys instead of bare values, preventing topics like
  "Customer Service" from being misclassified as SDK
- Add strip_local_md_links() to remove relative .md references that
  would cause broken link errors in Docusaurus build

* ci: run test-doc-examples independently without waiting for test-rust-cli

Build the CLI directly in the job instead of downloading the artifact,
so test-doc-examples can start at the beginning in parallel with all other jobs.
2026-03-03 18:46:59 +01:00
Nicolò Boschi
5c3d3274d7
docs: 0.4.15 release blog post and changelog (#477)
* docs: add 0.4.15 release blog post and changelog

* docs: update 0.4.15 blog cover image
2026-03-03 15:52:56 +01:00
Nicolò Boschi
144e4c49d1 Release v0.4.15
- Update version to 0.4.15 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-litellm, hindsight-crewai, hindsight-pydantic-ai, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- OpenClaw integration: hindsight-integrations/openclaw
- AI SDK integration: hindsight-integrations/ai-sdk
- Chat SDK integration: hindsight-integrations/chat
- Helm chart
- Sync documentation to version-0.4
2026-03-03 15:03:42 +01:00
Ben
8138fa9002
blog: add CrewAI persistent memory post (#471)
* Add CrewAI persistent memory blog post

* Update blog: add image, remove full example and alternatives sections

* Add CrewAI blog hero image
2026-03-02 16:00:27 -05:00
Nicolò Boschi
1d70abfe85
feat: add tags filtering and q description fix for list documents API (#468)
* feat: add Pydantic AI integration to CI, release pipeline, and docs

- Add test-pydantic-ai-integration job to CI (test.yml)
- Add build, publish, and artifact steps to release workflow (release.yml)
- Add hindsight-integrations/pydantic-ai to release.sh version bumping
- Add Pydantic AI documentation page (sdks/integrations/pydantic-ai.md)
- Add Pydantic AI entry to sidebar with icon

* docs: remove Requirements section from pydantic-ai integration page

* feat: add tags filtering and fix offset pagination docs for list documents API

- Add `tags` and `tags_match` query params to GET /banks/{bank_id}/documents
- Supports any, all, any_strict, all_strict matching modes (default: any_strict)
- Fix `q` param description — it's a case-insensitive substring match on document ID only
- Add tests for offset pagination and all tags_match modes
- Regenerate OpenAPI spec and Python/TypeScript/Go clients
- Document the new filtering options in docs/developer/api/documents.mdx

* fix(cli): pass new tags/tags_match args to list_documents
2026-03-02 17:03:16 +01:00
Nicolò Boschi
ecf609c8aa
feat: add Pydantic AI integration to CI, release pipeline, and docs (#467)
* feat: add Pydantic AI integration to CI, release pipeline, and docs

- Add test-pydantic-ai-integration job to CI (test.yml)
- Add build, publish, and artifact steps to release workflow (release.yml)
- Add hindsight-integrations/pydantic-ai to release.sh version bumping
- Add Pydantic AI documentation page (sdks/integrations/pydantic-ai.md)
- Add Pydantic AI entry to sidebar with icon

* docs: remove Requirements section from pydantic-ai integration page
2026-03-02 15:40:37 +01:00
Nicolò Boschi
ab70da1ead
docs: entities vs tags vs metadata (#466)
* docs: move entity labels detail to memory-banks, simplify retain overview

* docs: move entity labels blurb under entity-recognition section in retain

* docs: update metadata filtering FAQ to cover entity graph retrieval and entity labels tag option

* docs: enable TOC and fix missing separators in FAQ

* docs: add benchmarks leaderboard screenshot and link to models page

* docs: add 'Which model should I use?' FAQ entry with leaderboard screenshot

* docs: fix leaderboard description to cover retain, reflect, and observations
2026-03-02 14:47:40 +01:00
Nicolò Boschi
f903948a26
feat: support timestamp="unset" to retain content without a date (#465)
* feat: support timestamp="unset" to retain content without a date

When callers retain timeless content (e.g. fictional documents, static
reference material), passing timestamp="unset" now skips the utcnow()
default so mentioned_at is stored as NULL instead of an artificial date.

- HTTP: validate_timestamp recognises "unset" sentinel and threads it
  through api_retain as event_date=None (key present, value None), which
  the orchestrator distinguishes from key-absent (still defaults to now)
- Orchestrator: new branching logic separates "key absent" → utcnow()
  from "key present but None" → no date
- types.py: RetainContent.event_date and ProcessedFact.mentioned_at are
  now datetime | None; removed the unused _now_utc factory
- fact_extraction.py: all event_date params accept datetime | None;
  _build_user_message emits "Event Date: Unknown" when None; removed
  mentioned_at from the Fact LLM response model (LLM never sets it)
- embedding_processing: skip date suffix when fact_date is None
- entity_resolver: COALESCE(event_date, now()) for first_seen/last_seen
  so entities table NOT NULL constraint is preserved
- link_utils: skip temporal linking for units without event_date
- Migration aa2b3c4d5e6f: DROP NOT NULL on memory_units.event_date
- Tests: test_retain_no_timestamp and test_retain_omit_timestamp_defaults_to_now
- Docs + OpenAPI + TypeScript client updated

* refactor: replace _TIMESTAMP_UNKNOWN sentinel with plain string comparison

The sentinel object() was only needed to distinguish "unset" from None
at the boundary — but since the field type is datetime | str | None,
"unset" can pass through the validator unchanged and be compared directly.

* chore: regenerate OpenAPI spec and clients after timestamp type change

timestamp field is now datetime | str | None to accept the "unset" sentinel value.
2026-03-02 12:03:23 +01:00
Nicolò Boschi
55af468187
feat: observation_scopes field to drive observations granularity (#447)
* feat: observation_scopes field to drive observations granularity

* fix(migration): make a2b3c4d5e6f7 a no-op to fix CI on fresh DB

The z1u2v3w4x5y6 migration already creates observation_scopes directly,
so the rename migration fails on fresh installs where observation_tags
never existed.

* chore: remove no-op migration a2b3c4d5e6f7

* feat: regenerate clients with observation_scopes field

- Add observation_scopes to OpenAPI spec and all generated clients
- Fix Rust build.rs to handle anyOf with >2 variants containing null
  (previously only handled 2-item anyOf, causing progenitor to panic
  on the observation_scopes union type)

* fix(rust): add observation_scopes: None to MemoryItem struct literals

* fix(api): add title to observation_scopes Field for deterministic client generation

Adding title="ObservationScopes" makes the inline anyOf schema use
the explicit name instead of deriving it from the field name, which
was non-deterministic between arm64 (macOS) and amd64 (CI) Docker.

Also fixes description: "each entity" -> "each tag".

* fix(scripts): use linux/amd64 Docker for client generation to ensure reproducibility

Both Python and Go client generation now use --platform linux/amd64
Docker, ensuring identical output on macOS arm64 (local) and Linux
amd64 (CI). Also switches Go from JAR+Java to Docker to eliminate
Java version variability.

* chore: update generated clients to API v0.4.14

* fix(test): add retry logic to test_retain_chinese_content to handle non-deterministic LLM output

* fix(test): mark test_retain_chinese_content as xfail due to non-deterministic LLM translation
2026-02-28 10:46:21 +01:00
Nicolò Boschi
2b5fb10dab
doc: 0.4.14 (#449) 2026-02-27 15:41:08 +01:00
Nicolò Boschi
5443c18bfc
doc: improvements (#448) 2026-02-27 14:59:11 +01:00
Nicolò Boschi
145454533c Release v0.4.14
- Update version to 0.4.14 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-litellm, hindsight-crewai, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- OpenClaw integration: hindsight-integrations/openclaw
- AI SDK integration: hindsight-integrations/ai-sdk
- Chat SDK integration: hindsight-integrations/chat
- Helm chart
- Sync documentation to version-0.4
2026-02-26 18:03:14 +01:00
Nicolò Boschi
0bb5ca4caf
feat: filter graph memories with tags (#431)
* feat: filter graph memories with tags

* fix(cli): pass new q/tags/tags_match args to get_graph

* docs: use CodeSnippet for tags_match examples in recall.mdx
2026-02-25 10:31:40 +01:00
Nicolò Boschi
ac9a94ade3
fix: handle observations regeneration when memories get deleted (#429)
* fix: handle observations regeneration when memories get deleted

* feat: add clear_memory_observations endpoint and regenerate clients

- Add DELETE /banks/{id}/memories/{memory_id}/observations endpoint
- Add observations lifecycle/invalidation section to docs
- Regenerate OpenAPI spec and all clients (Python, TypeScript, Go, Rust)

* refactor: use dedicated response model for clear_memory_observations, remove code example from docs
2026-02-24 13:26:33 +01:00
Nicolò Boschi
2a32273226
feat: increase customization for reflect, retain and consolidation (#419) 2026-02-23 20:35:32 +01:00
Nicolò Boschi
278344b3b3
doc: improve api explanation (#415)
* doc: improve api explanation

* doc: improve api explanation

* doc: improve api explanation

* fix: add include_facts to reflect client, fix retain.sh temp files, fix main-methods based_on access

* fix: create report.pdf in working directory for retain.sh file upload examples
2026-02-20 17:02:01 +01:00
Nicolò Boschi
4f431b4ace
doc: 0.4.13 changelog (#411) 2026-02-19 20:48:50 +01:00
Nicolò Boschi
2993fdd2f9 Release v0.4.13
- Update version to 0.4.13 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-litellm, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- OpenClaw integration: hindsight-integrations/openclaw
- AI SDK integration: hindsight-integrations/ai-sdk
- Helm chart
- Sync documentation to version-0.4
2026-02-19 18:46:07 +01:00
Nicolò Boschi
5569d4adba
feat: include source facts in observation recall (#404)
* feat: include source facts in observation recall

* feat: include source facts in observation recall

* feat: include source facts in observation recall

* feat: include source facts in observation recall

* fix(cli): add missing source_facts field to IncludeOptions initializer
2026-02-19 14:54:19 +01:00
Nicolò Boschi
7c78ae2371 Release v0.4.12
- Update version to 0.4.12 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-litellm, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- OpenClaw integration: hindsight-integrations/openclaw
- AI SDK integration: hindsight-integrations/ai-sdk
- Helm chart
- Sync documentation to version-0.4
2026-02-18 14:13:33 +01:00
Nicolò Boschi
224b7b74c1
feat: accept pdf, images and office files (#390)
* feat: accept pdf, images and office files

* refactor: rename FileConverter to FileParser, simplify file retain API

- Rename engine/converters/ → engine/parsers/, FileConverter → FileParser,
  ConverterRegistry → FileParserRegistry, MarkitdownConverter → MarkitdownParser
- Rename env var HINDSIGHT_API_FILE_CONVERTER → HINDSIGHT_API_FILE_PARSER
- Remove async/document_tags params from FileRetainRequest (always async now)
- Add retain_files() to Python Hindsight client and retainFiles() to TypeScript client
- Add sample.pdf to doc examples for working file upload demonstrations
- Update test_file_retain.py to use new parser names and always-async behavior
- Fix Go client missing os import in api_files.go
- Simplify postgresql.py storage to minimal schema

* fix: update rust CLI tests to use is_supported_file instead of is_text_file

* fix: patch Go api_files.go to add missing 'os' import after generation

* fix: insert 'os' import after 'net/url' in api_files.go patch for correct position

* chore: regenerate OpenAPI spec and clients (converter→parser description update)
2026-02-17 18:15:03 +01:00
Nicolò Boschi
5883e5af2d
doc: add go client examples (#380)
* doc: add go client examples

* doc: add go client examples
2026-02-16 14:43:03 +01:00
Nicolò Boschi
aefb3fcf4d
fix: improve async batch retain with large payloads (#366)
* fix: improve async batch retain with large payloads

* fix: improve async batch retain with large payloads

* api

* api

* api

* api

* api

* Clean up perf benchmark: keep only Python files

- Remove README.md and PERFORMANCE_FINDINGS.md
- Remove results/ JSON files (gitignored)
- Remove test_data/ directory
- Keep only __init__.py and retain_perf.py

* docs: explain automatic batch optimization for async retain

- Add section explaining Hindsight automatically handles batch sizing
- Users don't need to manually tune batch sizes with async mode
- Hindsight splits large batches (>10k tokens) into optimized sub-batches
- Include example showing best practices

* docs: remove emojis and code example from performance page

* fix: correct OperationDetails type to match API response

- Change optional fields to use | null instead of ?
- Fixes TypeScript compilation error in control plane build

* fix: use discriminated union for OperationDetails type

- Support both success and error states properly
- Fixes TypeScript error when setting error state

* fix: use unique document_ids in batch retain examples

- Each item in a batch must have unique document_id
- Update both Python and JavaScript examples
- Fixes test-doc-examples CI failure

* chore: trigger CI

* fix: test mocking and duplicate document_ids in examples

- Mock _get_pool() in test_async_retain_tags.py to avoid _initialized error
- Set _initialized = True on mocked MemoryEngine instances
- Fix duplicate document_ids in retain.py and retain.mjs examples

* fix: properly mock async pool/connection and fix more duplicate document_ids

- Use AsyncMock for pool.acquire() to fix 'can't be used in await' error
- Fix duplicate document_ids in retain-async examples (retain.py and retain.mjs)
- Remove batch-level document_id parameter that caused duplicates

* ci: collect all doc example failures and show summary

- Run all Python/Node.js/CLI examples regardless of individual failures
- Collect failure list and display summary at the end
- Show pass/fail count and list of failed files
- Exit with failure only after running all examples

* refactor: extract doc example testing to standalone script

- Create scripts/test-doc-examples.sh to run all examples
- Collects logs of failed examples separately
- Shows full error logs only for failures at the end
- Clean summary with pass/fail counts
- Proper exit codes
- Replaces inline bash in CI workflow

* fix: doc examples - duplicate document_ids and error handling

- retain.py: move document_id to item level to avoid duplicates
- documents.mjs: add error handling for getDocument to show clear error message

* fix: update tests for duplicate document_id validation

- test_async_retain_tags: verify operation structure instead of exact UUID
- test_delete_bank: use unique document_ids (team-doc-1, team-doc-2)
2026-02-16 12:51:42 +01:00
Nicolò Boschi
b3b541fc53 Release v0.4.11
- Update version to 0.4.11 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-litellm, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- OpenClaw integration: hindsight-integrations/openclaw
- AI SDK integration: hindsight-integrations/ai-sdk
- Helm chart
- Sync documentation to version-0.4
2026-02-13 10:52:14 +01:00
Nicolò Boschi
8d731f2e5f
feat: implement hierarchical configuration (system, tenant, bank) (#329)
* feat: implement hierarchical configuration (system, tenant, bank)

* feat: implement hierarchical configuration (system, tenant, bank)

* docs: add instructions for hierarchical config in CLAUDE.md

* feat: add ENABLE_BANK_CONFIG_API flag (disabled by default)

- Add HINDSIGHT_API_ENABLE_BANK_CONFIG_API env var (default: false)
- Return 403 Forbidden from bank config endpoints when disabled
- Update tests to enable the flag
- Update CLAUDE.md documentation

This provides security control over the bank configuration API,
ensuring it's only accessible when explicitly enabled.

* docs: add hierarchical configuration section

* feat(cli): add bank config commands (config, set-config, reset-config)

- Add 'hindsight bank config' to view bank configuration
- Add 'hindsight bank set-config' to update LLM settings per bank
- Add 'hindsight bank reset-config' to reset to defaults
- Implements client API calls to new bank config endpoints

* fix(cli): fix compilation errors in bank config commands

- Fix type signature: use ApiClient instead of api::Client
- Fix confirmation: use ui::prompt_confirmation instead of ui::confirm
- Fix error handling: use anyhow! macro instead of errors::Error
- Fix type conversion: convert HashMap to serde_json::Map for API call

* feat: implement type-safe hierarchical config with bank overrides

Implements a production-ready hierarchical configuration system that prevents
accidentally using global defaults when bank-specific overrides exist.

- Created StaticConfigProxy that wraps HindsightConfig
- get_config() now returns proxy that blocks access to bank-configurable fields
- Raises ConfigFieldAccessError with clear message when accessing configurable fields
- Added _get_raw_config() for internal use only
- Forces developers to use resolve_full_config(bank_id, context) for bank settings

- Added resolve_full_config() method that returns complete HindsightConfig
- Resolves hierarchy: Global (env) → Tenant → Bank
- No caching to support multi-server deployments (always fresh from DB)
- LLM provider pooling handles expensive operations separately

- Updated entire retain pipeline to pass resolved config through call chain
- memory_engine.py: Resolves config at top level where bank_id/context available
- orchestrator.py: Accepts and passes config to fact_extraction
- fact_extraction.py: Uses passed config instead of get_config()
- utils.py: Added optional config param for backward compatibility

- consolidator.py: Uses resolve_full_config() for enable_observations check
- memory_engine.py: Resolves config before triggering consolidation

- Renamed "Memory Bank" to "Bank Configuration" with tabs
- Combined Stats and Operations into "General" tab
- Consolidated Profile and Configuration into "Configuration" tab
- Moved Actions dropdown to page level (outside tabs)

- Created new component for managing bank-specific config
- Displays configurable fields: retain_chunk_size, retain_extraction_mode, etc.
- Edit via dialog with form validation
- Reset to defaults via AlertDialog confirmation
- Shows field IDs in monospace for clarity
- Visual separation with borders and hover effects

- Removed inline edit mode, switched to dialog-based editing
- Separate dialogs for Disposition and Mission editing
- Read-only display with clear edit buttons
- Removed duplicate stats cards and operations

- bank-stats-view.tsx: Overview statistics (memories, links, documents, pending ops)
- bank-operations-view.tsx: Background operations table with filtering

**Problem**: Consolidation always used global enable_observations, ignoring bank overrides
**Root Cause**: consolidator.py called get_config() instead of resolving bank-specific config
**Solution**: Pass resolved config through the entire pipeline

**Problem**: asyncpg returning JSONB as JSON string instead of parsed dict
**Solution**: Explicit JSON parsing in config_resolver.py with type checking

- All 19 API integration tests pass
- All 10 hierarchical config tests pass
- Retain operations work correctly with bank-specific config
- Consolidation respects bank-specific enable_observations setting

- Updated developer/configuration.md with type-safe config access pattern
- Added examples showing correct usage patterns
- Documented ConfigFieldAccessError and resolution methods

- get_config() now returns StaticConfigProxy (blocks configurable field access)
- Code accessing bank-configurable fields must use resolve_full_config()
- Clear migration path with helpful error messages

Fixes hierarchical configuration to be production-ready with proper type safety.

* refactor: remove LLM client pool and simplify config resolver

Since LLM config (provider, model, api_key) is now static and not
bank-configurable, the LLMClientPool is no longer needed.

Changes:
- Remove hindsight_api/llm_client_pool.py (no longer needed)
- Remove memory_engine._get_bank_llm_config() (dead code, never called)
- Simplify config_resolver.py by eliminating duplication between
  resolve_full_config() and get_bank_config()
- get_bank_config() now calls resolve_full_config() and filters results
- Remove outdated "LLM provider pooling" comments from docstrings

All tests pass (10 hierarchical config tests, 19 API integration tests)

* fix: update tests to use _get_raw_config() for configurable fields

Fixed test fixtures that were accessing configurable fields (like
enable_observations) from get_config(), which now raises
ConfigFieldAccessError due to type-safe config access.

Changes:
- test_consolidation.py: Changed enable_observations fixture to use
  _get_raw_config() instead of get_config()
- test_consolidation.py: Updated test_consolidation_returns_disabled_status
  to set bank config instead of mocking get_config()
- test_link_expansion_retrieval.py: Changed fixture to use _get_raw_config()
- test_observations.py: Changed disable_observations fixture to use
  _get_raw_config()
- Regenerated OpenAPI spec and clients

All 39 previously failing tests now pass.

* fix: add missing config parameter to test calls of extract_facts_from_text()

Fixed 45 test failures where tests were calling extract_facts_from_text()
without the new required config parameter.

Changes:
- Added config=_get_raw_config() to all extract_facts_from_text() calls
- Fixed test_main_module.py to patch _get_raw_config instead of get_config
- Updated 6 test files with 37 function call sites

All tests should now pass.

* fix: add missing config parameter to test_skip_podcast_meta_commentary

One more test was missing the config parameter for extract_facts_from_text().
2026-02-12 13:14:57 +01:00
Nicolò Boschi
f9a8a8e01e
fix: resolve based_on schema/serialization issues in reflect API (#348)
* fix: add default values to OpenAPI schema for default_factory fields

This commit fixes the OpenAPI schema to include default values for fields
using default_factory, which improves schema accuracy and client generation.

Changes:
1. Added FieldWithDefault() helper to inject default values into OpenAPI schema
2. Updated 14 fields using default_factory to include defaults in schema:
   - ReflectBasedOn.{memories, mental_models, directives}
   - ReflectTrace.{tool_calls, llm_calls}
   - All tags fields
   - All trigger fields
   - All include fields

3. Regenerated OpenAPI spec with proper defaults

4. Added tests to verify API returns correct format with empty banks

Note: This fixes the schema but doesn't change the v0.3.0 -> v0.4.0 breaking
change where based_on went from list to object. Clients should handle both
formats for backward compatibility.

* fix: remove client imports from API test

The test was failing in CI because it imported the client library
which isn't installed in the API test environment.

Changed to test only API JSON response format, not client parsing.
This is more appropriate for an API test anyway.

* test: add client tests for ReflectResponse parsing

Added comprehensive tests in hindsight-clients/python/tests to verify:
- v0.4.0+ format with empty based_on object
- v0.4.0+ format with null based_on
- v0.4.0+ format with populated facts
- v0.3.0 format (list) correctly fails validation
- Missing based_on field handling

These tests document the v0.3.0 -> v0.4.0 breaking change where
based_on changed from list to object.
2026-02-12 11:26:12 +01:00
Chris Latimer
4499254f6d Memory conflict blog post 2026-02-09 11:18:30 -07:00
Nicolò Boschi
08f1cda3bf Release v0.4.10
- Update version to 0.4.10 in all components
- Regenerate OpenAPI spec and client SDKs
- Python packages: hindsight-api, hindsight-dev, hindsight-all, hindsight-litellm, hindsight-embed
- Python client: hindsight-clients/python
- TypeScript client: hindsight-clients/typescript
- Rust CLI: hindsight-cli
- Control Plane: hindsight-control-plane
- OpenClaw integration: hindsight-integrations/openclaw
- AI SDK integration: hindsight-integrations/ai-sdk
- Helm chart
- Sync documentation to version-0.4
2026-02-09 11:44:20 +01:00
Nicolò Boschi
c4ef090a20
feat: support markdown in reflect and mental models (#307)
* feat: support markdown in reflect and mental models

* chore: regenerate clients and OpenAPI spec with markdown field descriptions
2026-02-06 10:49:13 +01:00
Nicolò Boschi
0d8d805832
ci: ensure backwards/forward compatibility of the API (#306) 2026-02-05 18:43:05 +01:00
Nicolò Boschi
90ad003c46
docs: add AI SDK integration documentation (#304)
* docs: add AI SDK integration documentation

- Add comprehensive AI SDK documentation in docs/sdks/integrations/ai-sdk.md
  - Detailed description of all three memory tools (retain, recall, reflect)
  - Complete parameter documentation and return types
  - Advanced usage patterns (streaming, multi-user, ToolLoopAgent)
  - HTTP client example for zero-dependency usage
  - TypeScript types and API reference
  - Best practices and system prompt examples

- Update AI SDK README to brief quickstart with link to docs
  - Single source of truth: comprehensive docs in documentation site
  - README now focuses on quick setup and points to full docs
  - Maintains features list and basic example for npm page

* fix
2026-02-05 17:05:18 +01:00