fleet-memory/hindsight-api-slim/tests
Nicolò Boschi 30a319a6ab
feat: bank template import/export with Template Hub (#819)
* feat(api): add bank template import/export endpoints

Add POST /banks/{bank_id}/import and GET /banks/{bank_id}/export
endpoints for declarative bank setup via JSON manifests.

A template manifest (version 1) can include bank config overrides
and mental model definitions. Import creates or updates mental
models matched by id, applies config as per-bank overrides, and
returns async operation IDs for content generation.

Export dumps a bank's explicit overrides and mental models as a
manifest that can be re-imported into another bank.

Includes control plane UI: bank creation dialog now accepts an
optional template JSON to pre-configure the bank on creation.

* docs: add Template Gallery page and bank templates reference

- Template Gallery (/templates) with search, category filter, manifest
  preview modal with copy-to-clipboard
- 5 starter templates: Customer Support, Research Assistant, Personal
  Journal, Code Review Buddy, Meeting Notes
- Bank Templates API reference doc (developer/api/bank-templates)
- Sidebar entry under API section

* docs: add Template Gallery links to navbar and sidebar

- Top navbar: "Templates" link between Integrations and Changelog
- Sidebar: "Template Gallery" in Resources section

* fix(docs): remove emoji icons, autofocus search, fix placeholder in template gallery

* docs: rename to Bank Templates, move to Resources sidebar only

* docs: add Bank Templates to Resources navbar dropdown

* feat(api): add directives to bank template import/export

- Add BankTemplateDirective model with name, content, priority, is_active, tags
- Import creates/updates directives matched by name
- Export includes all directives (active and inactive)
- Validation: duplicate names rejected, empty name/content caught
- Tests: 24 tests covering directives create/update, existing vs new
  bank import, validation, export with directives, full round-trip

* docs: add directives to bank templates docs and sample templates

* feat(api): add JSON Schema endpoint for bank template validation

- GET /v1/default/bank-template-schema returns the JSON Schema
  auto-generated from the Pydantic BankTemplateManifest model
- Static schema file at docs/static/bank-template-schema.json
- Docs updated with schema endpoint, static file link, and
  validation examples (Python jsonschema, Node ajv-cli)

* feat(api): live schema validation on import, fix schema endpoint path

- Move schema endpoint to /v1/bank-template-schema (system-level, not per-bank)
- Import endpoint now accepts raw JSON and validates with Pydantic manually,
  returning clean 400 errors instead of raw 422s for all validation failures
- All validation (schema + semantic) returns consistent 400 with detailed messages

* docs: add interactive JSON Schema viewer to Bank Templates page

Renders the Pydantic-generated schema as a collapsible property tree
with types, required badges, defaults, and descriptions. The schema
is imported from the static bank-template-schema.json file.

* ui: add template toggle switch and browse link to bank creation dialog

- Replace always-visible textarea with a switch toggle ("Import from template")
- Textarea only shows when switch is on, keeping the dialog clean by default
- Add "Browse templates" link pointing to hindsight.vectorize.io/templates
- Reset template state when switch is toggled off or dialog is cancelled

* ui: add empty state with Add Document CTA to data view

When a bank has 0 memories, the data view (all tabs: constellation,
graph, table, timeline) shows a centered empty state with a CTA
button that opens the Add Document dialog.

* docs: replace templates with Conversation and Coding Agent

Remove generic placeholder templates. Add two practical templates
based on actual integration patterns:

- Conversation: for chat agents (LiteLLM, LangGraph, Pydantic AI,
  Vercel AI SDK). Tracks user preferences, open threads.
- Coding Agent: for Claude Code/Codex. Tracks technical decisions,
  project context, developer preferences. High literalism.

* docs: rename gallery to Bank Templates Hub, keep API doc as Bank Templates

* docs: register layout-template and file-json icons in navbar and sidebar

* docs: register layout-template icon in DefaultNavbarItem for dropdown items

* docs: show integration icons on template cards

Templates now have an optional `integrations` field referencing
integration IDs from integrations.json. Icons are resolved at render
time and shown in the card header next to the category badge.

* docs: add Personal Assistant template for OpenClaw, Hermes, NemoClaw

* feat: add Export Template to bank actions + map all integrations to templates

- Add "Export Template" to the bank Actions dropdown — exports config,
  mental models, and directives as JSON, copies to clipboard
- Add export API route and client method
- Map remaining integrations to templates: CrewAI, AG2, Agno, Strands,
  LlamaIndex, local-mcp, skills → Conversation; hindclaw → Personal Assistant

* feat: add --template flag to LoCoMo benchmark + remove schema from Hub

- LoCoMo benchmark accepts --template <path> to apply a bank template
  manifest (config, mental models, directives) before ingestion
- Template is applied per-bank in both single-phase and two-phase modes
- BenchmarkRunner.apply_template() reuses the same engine methods as
  the /import API endpoint
- Remove Manifest Schema section from Bank Templates Hub page
  (schema stays in the API reference doc)

* refactor: remove description field from bank template manifest

* docs: remove tags, fact_types, and directives from starter templates

* docs: remove reflect_mission and disposition fields from starter templates

* build: validate template manifests against JSON Schema during docs build

* cleanup: remove unused JsonSchemaViewer component

* docs: remove retain_extraction_mode from starter templates

* ui: enable word wrap in template manifest preview

* docs: add link to Bank Templates reference doc from Hub page

* docs: convert bank templates doc to mdx with multi-language code snippets

- Convert bank-templates.md to .mdx with Tabs/CodeSnippet components
- Add example files: bank-templates.py, .mjs, .sh, .go with doc markers
- Examples cover import, dry-run, export, round-trip, and schema
- Regenerate OpenAPI spec and all client SDKs (Python, TS, Rust, Go)

* fix: migration revision collision + use typed models in benchmark template

- Rename merge migration d6e7f8a9b0c1 -> d6e7f8a9b0c2 to resolve
  revision ID collision with case_insensitive_entities_trgm_index
- Update a4b5c6d7e8f9 down_revision to point to the renamed migration
- Fix f-string lint in case_insensitive migration
- BenchmarkRunner.apply_template() now validates manifest through
  BankTemplateManifest Pydantic model instead of raw dict access
- Remove redundant inline imports (json, Path already at module top)

* fix(docs): add missing Go tab to dry-run code snippet

* ci: retrigger

* fix: sync skills openapi.json + fix bankId null type error in export

- Copy updated openapi.json to skills/hindsight-docs/references/
- Add null guard for bankId in Export Template onClick handler

* fix: sync generated files (memory_engine formatting, docs skill references)

* cleanup: remove obsolete migration collision workaround
2026-04-02 12:21:53 +02:00
..
fixtures feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
__init__.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
conftest.py perf: 3-phase retain pipeline — fix deadlocks, cap temporal links, query-time entity expansion (#722) 2026-04-01 12:52:49 +02:00
test_admin_backup_restore.py fix(migrations): bypass PgBouncer for advisory locks via MIGRATION_DATABASE_URL (#726) 2026-03-27 16:01:38 +01:00
test_agents_api.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_async_batch_retain.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_async_retain_tags.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_audit_log.py feat: add audit log for feature usage tracking (#717) 2026-03-27 09:52:03 +01:00
test_bank_templates.py feat: bank template import/export with Template Hub (#819) 2026-04-02 12:21:53 +02:00
test_base_path.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_batch_api.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_batch_api_integration.py fix(retain): preserve normalized experience fact types (#848) 2026-04-02 12:20:37 +02:00
test_batch_api_validation.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_batch_chunking.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_causal_relations.py refactor: replace LLMProvider classmethods with from_env() and document missing config fields (#816) 2026-03-31 18:00:41 +02:00
test_causal_relationships.py refactor: replace LLMProvider classmethods with from_env() and document missing config fields (#816) 2026-03-31 18:00:41 +02:00
test_chunking.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_codex_tool_choice.py Convert codex tool_choice test to pytest style (#752) 2026-03-30 11:58:59 +02:00
test_cohere_cross_encoder.py fix(reranker): use httpx for Cohere Azure endpoints to avoid 404 errors (#790) 2026-03-31 17:44:42 +02:00
test_combined_scoring.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_config_validation.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_consolidation.py fix(consolidation): improve observation quality with structured processing rules (#814) 2026-04-01 12:44:52 +02:00
test_consolidation_failure_recovery.py fix: prevent silent memory loss on consolidation LLM failure (#601) 2026-03-17 20:15:33 +01:00
test_custom_embedding_dimension.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_delta_retain.py feat(retain): delta retain — skip LLM for unchanged chunks on upsert (#701) 2026-03-26 13:50:55 +01:00
test_document_tracking.py feat: expose document_metadata in API and control plane (#798) 2026-03-31 11:42:02 +02:00
test_entity_labels.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_entity_resolver.py fix(entity_resolver): prevent _pending_stats/_pending_cooccurrences memory leak (#662) 2026-03-23 16:06:04 +01:00
test_entity_resolver_pg_trgm.py test: add unit tests for pg_trgm auto-detection and ValidationResult.accept_with() enrichment (#650) 2026-03-23 10:33:09 +01:00
test_experience_fact_type_passthrough.py fix(retain): preserve normalized experience fact types (#848) 2026-04-02 12:20:37 +02:00
test_extensions.py fix(retain): preserve normalized experience fact types (#848) 2026-04-02 12:20:37 +02:00
test_fact_extraction_agent_experience.py refactor: replace LLMProvider classmethods with from_env() and document missing config fields (#816) 2026-03-31 18:00:41 +02:00
test_fact_extraction_analysis.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_fact_extraction_metadata.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_fact_extraction_output_ratio.py refactor: replace LLMProvider classmethods with from_env() and document missing config fields (#816) 2026-03-31 18:00:41 +02:00
test_fact_extraction_quality.py refactor: replace LLMProvider classmethods with from_env() and document missing config fields (#816) 2026-03-31 18:00:41 +02:00
test_fact_extraction_retry.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_fact_ordering.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_file_retain.py Fix non-atomic async operation creation (#619) 2026-03-19 16:38:04 +01:00
test_file_storage_s3.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_gemini_safety_settings.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_graph_filtering.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_hierarchical_config.py fix: resolve 25 test regressions from streaming retain pipeline (#722) (#836) 2026-04-01 17:59:10 +02:00
test_hnsw_indexes.py fix: per-bank index creation respects HINDSIGHT_API_VECTOR_EXTENSION config (#755) 2026-03-30 11:37:26 +02:00
test_horse_observations.py fix(consolidation): improve observation quality with structured processing rules (#814) 2026-04-01 12:44:52 +02:00
test_http_api_integration.py fix(api): clear memories endpoint no longer deletes the bank profile (#837) 2026-04-01 18:34:39 +02:00
test_iris_parser.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_link_expansion_retrieval.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_link_utils.py perf: 3-phase retain pipeline — fix deadlocks, cap temporal links, query-time entity expansion (#722) 2026-04-01 12:52:49 +02:00
test_list_documents.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_litellm_sdk_cross_encoder.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_litellm_sdk_embeddings.py feat: add optional LiteLLM SDK embedding output dimensions (#809) 2026-03-31 14:58:02 +02:00
test_llm_provider.py fix(retain): preserve normalized experience fact types (#848) 2026-04-02 12:20:37 +02:00
test_llm_token_metrics.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_llm_tools.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_llm_wrapper.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_lmstudio_tool_choice.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_load_large_batch.py fix: prevent silent memory loss on consolidation LLM failure (#601) 2026-03-17 20:15:33 +01:00
test_main_module.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_mcp_endpoint_routing.py fix: MCP tool calls fail when MCP_AUTH_TOKEN and TENANT_API_KEY differ (#635) 2026-03-20 18:36:05 +01:00
test_mcp_extension.py fix(deps): address critical and high severity security vulnerabilities (#827) 2026-04-01 09:20:34 +02:00
test_mcp_routing.py fix(deps): address critical and high severity security vulnerabilities (#827) 2026-04-01 09:20:34 +02:00
test_mcp_tool_filtering.py feat(mcp): add filter_mcp_tools hook for per-user tool visibility (#737) 2026-03-30 10:33:09 +02:00
test_mcp_tools.py feat: add detail parameter to list/get mental models (#846) 2026-04-02 11:52:45 +02:00
test_mental_model_hooks.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_mental_models.py fix(mental-models): add tags_match and tag_groups to trigger config (#786) (#804) 2026-03-31 18:09:01 +02:00
test_metrics.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_migration_backsweep.py fix(migration): backsweep orphaned observation memory units (#584) 2026-03-16 14:06:33 +01:00
test_migrations_thread_safety.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_multilingual.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_none_llm_provider.py feat: add 'none' LLM provider for chunk-only storage mode (#691) 2026-03-25 18:01:20 +01:00
test_observation_invalidation.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_observations.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_op_cancellation.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_per_operation_llm_config.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_provider_default_models.py security: exclude litellm 1.82.8 (supply chain compromise) (#673) 2026-03-25 10:21:02 +01:00
test_query_analyzer.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_recall_chunks_independence.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_reflect_agent.py Add wall-clock timeout to reflect operations (#643) 2026-03-23 09:19:26 +01:00
test_reflect_empty_based_on.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_reflect_source_facts_config.py feat(reflect): make source facts in search_observations configurable (#688) 2026-03-25 17:54:49 +01:00
test_reflect_tracing.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_reflections.py feat: fact_types and mental model exclusion filters for reflect (#615) 2026-03-19 17:03:41 +01:00
test_reranker_error_handling.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_retain.py perf: 3-phase retain pipeline — fix deadlocks, cap temporal links, query-time entity expansion (#722) 2026-04-01 12:52:49 +02:00
test_schema_isolation.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_search_trace.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_server_module.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_source_facts_tokens.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_sql_schema_safety.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_strip_code_fences.py fix: strip markdown code fences from all LLM providers, not just local (#646) 2026-03-22 21:29:16 +01:00
test_supabase_tenant.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_tags_visibility.py feat: compound tag filtering via tag_groups (#562) 2026-03-13 14:30:11 +01:00
test_tei_cross_encoder.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_temporal_ranges.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_think.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_tracing.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_tracing_integration.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_tracing_spans_verification.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_unknown_params.py feat(api): warn on unknown request parameters via X-Ignored-Params header (#802) 2026-03-31 10:34:04 +02:00
test_validation_result_enrichment.py test: add unit tests for pg_trgm auto-detection and ValidationResult.accept_with() enrichment (#650) 2026-03-23 10:33:09 +01:00
test_vertexai_provider.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_webhooks.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
test_worker.py Fix orphaned batch_retain parents when child fails via unhandled exception (#618) 2026-03-19 14:55:26 +01:00