fleet-memory/hindsight-api/tests
Nicolò Boschi 8d731f2e5f
feat: implement hierarchical configuration (system, tenant, bank) (#329)
* feat: implement hierarchical configuration (system, tenant, bank)

* feat: implement hierarchical configuration (system, tenant, bank)

* docs: add instructions for hierarchical config in CLAUDE.md

* feat: add ENABLE_BANK_CONFIG_API flag (disabled by default)

- Add HINDSIGHT_API_ENABLE_BANK_CONFIG_API env var (default: false)
- Return 403 Forbidden from bank config endpoints when disabled
- Update tests to enable the flag
- Update CLAUDE.md documentation

This provides security control over the bank configuration API,
ensuring it's only accessible when explicitly enabled.

* docs: add hierarchical configuration section

* feat(cli): add bank config commands (config, set-config, reset-config)

- Add 'hindsight bank config' to view bank configuration
- Add 'hindsight bank set-config' to update LLM settings per bank
- Add 'hindsight bank reset-config' to reset to defaults
- Implements client API calls to new bank config endpoints

* fix(cli): fix compilation errors in bank config commands

- Fix type signature: use ApiClient instead of api::Client
- Fix confirmation: use ui::prompt_confirmation instead of ui::confirm
- Fix error handling: use anyhow! macro instead of errors::Error
- Fix type conversion: convert HashMap to serde_json::Map for API call

* feat: implement type-safe hierarchical config with bank overrides

Implements a production-ready hierarchical configuration system that prevents
accidentally using global defaults when bank-specific overrides exist.

- Created StaticConfigProxy that wraps HindsightConfig
- get_config() now returns proxy that blocks access to bank-configurable fields
- Raises ConfigFieldAccessError with clear message when accessing configurable fields
- Added _get_raw_config() for internal use only
- Forces developers to use resolve_full_config(bank_id, context) for bank settings

- Added resolve_full_config() method that returns complete HindsightConfig
- Resolves hierarchy: Global (env) → Tenant → Bank
- No caching to support multi-server deployments (always fresh from DB)
- LLM provider pooling handles expensive operations separately

- Updated entire retain pipeline to pass resolved config through call chain
- memory_engine.py: Resolves config at top level where bank_id/context available
- orchestrator.py: Accepts and passes config to fact_extraction
- fact_extraction.py: Uses passed config instead of get_config()
- utils.py: Added optional config param for backward compatibility

- consolidator.py: Uses resolve_full_config() for enable_observations check
- memory_engine.py: Resolves config before triggering consolidation

- Renamed "Memory Bank" to "Bank Configuration" with tabs
- Combined Stats and Operations into "General" tab
- Consolidated Profile and Configuration into "Configuration" tab
- Moved Actions dropdown to page level (outside tabs)

- Created new component for managing bank-specific config
- Displays configurable fields: retain_chunk_size, retain_extraction_mode, etc.
- Edit via dialog with form validation
- Reset to defaults via AlertDialog confirmation
- Shows field IDs in monospace for clarity
- Visual separation with borders and hover effects

- Removed inline edit mode, switched to dialog-based editing
- Separate dialogs for Disposition and Mission editing
- Read-only display with clear edit buttons
- Removed duplicate stats cards and operations

- bank-stats-view.tsx: Overview statistics (memories, links, documents, pending ops)
- bank-operations-view.tsx: Background operations table with filtering

**Problem**: Consolidation always used global enable_observations, ignoring bank overrides
**Root Cause**: consolidator.py called get_config() instead of resolving bank-specific config
**Solution**: Pass resolved config through the entire pipeline

**Problem**: asyncpg returning JSONB as JSON string instead of parsed dict
**Solution**: Explicit JSON parsing in config_resolver.py with type checking

- All 19 API integration tests pass
- All 10 hierarchical config tests pass
- Retain operations work correctly with bank-specific config
- Consolidation respects bank-specific enable_observations setting

- Updated developer/configuration.md with type-safe config access pattern
- Added examples showing correct usage patterns
- Documented ConfigFieldAccessError and resolution methods

- get_config() now returns StaticConfigProxy (blocks configurable field access)
- Code accessing bank-configurable fields must use resolve_full_config()
- Clear migration path with helpful error messages

Fixes hierarchical configuration to be production-ready with proper type safety.

* refactor: remove LLM client pool and simplify config resolver

Since LLM config (provider, model, api_key) is now static and not
bank-configurable, the LLMClientPool is no longer needed.

Changes:
- Remove hindsight_api/llm_client_pool.py (no longer needed)
- Remove memory_engine._get_bank_llm_config() (dead code, never called)
- Simplify config_resolver.py by eliminating duplication between
  resolve_full_config() and get_bank_config()
- get_bank_config() now calls resolve_full_config() and filters results
- Remove outdated "LLM provider pooling" comments from docstrings

All tests pass (10 hierarchical config tests, 19 API integration tests)

* fix: update tests to use _get_raw_config() for configurable fields

Fixed test fixtures that were accessing configurable fields (like
enable_observations) from get_config(), which now raises
ConfigFieldAccessError due to type-safe config access.

Changes:
- test_consolidation.py: Changed enable_observations fixture to use
  _get_raw_config() instead of get_config()
- test_consolidation.py: Updated test_consolidation_returns_disabled_status
  to set bank config instead of mocking get_config()
- test_link_expansion_retrieval.py: Changed fixture to use _get_raw_config()
- test_observations.py: Changed disable_observations fixture to use
  _get_raw_config()
- Regenerated OpenAPI spec and clients

All 39 previously failing tests now pass.

* fix: add missing config parameter to test calls of extract_facts_from_text()

Fixed 45 test failures where tests were calling extract_facts_from_text()
without the new required config parameter.

Changes:
- Added config=_get_raw_config() to all extract_facts_from_text() calls
- Fixed test_main_module.py to patch _get_raw_config instead of get_config
- Updated 6 test files with 37 function call sites

All tests should now pass.

* fix: add missing config parameter to test_skip_podcast_meta_commentary

One more test was missing the config parameter for extract_facts_from_text().
2026-02-12 13:14:57 +01:00
..
fixtures rename to hindsight (#2) 2025-11-25 19:28:26 +01:00
__init__.py rename to hindsight (#2) 2025-11-25 19:28:26 +01:00
conftest.py fix: improve claude code and codex for /reflect (#285) 2026-02-04 13:34:45 +01:00
test_admin_backup_restore.py feat: run db migrations offline (optionally) (#114) 2026-01-07 15:49:51 +01:00
test_agents_api.py feat: introduce mental models (#132) 2026-01-16 11:16:41 +01:00
test_base_path.py feat: add reverse proxy support (#346) 2026-02-12 10:09:53 +01:00
test_batch_chunking.py feat: extensions (#54) 2025-12-22 11:05:23 +01:00
test_causal_relations.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00
test_causal_relationships.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00
test_chunking.py improve retain performances, caching and tests 2025-12-08 18:21:56 +01:00
test_combined_scoring.py feat: extensions (#54) 2025-12-22 11:05:23 +01:00
test_config_validation.py fix: hindsight-embed profiles are not loaded correctly (#316) 2026-02-06 17:13:54 +01:00
test_consolidation.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00
test_custom_embedding_dimension.py fix: improve mental model consolidation (#197) 2026-01-26 09:54:25 +01:00
test_document_tracking.py feat: extensions (#54) 2025-12-22 11:05:23 +01:00
test_extensions.py Add actual LLM token usage fields to RetainResult (#342) 2026-02-11 10:41:41 +01:00
test_fact_extraction_analysis.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00
test_fact_extraction_output_ratio.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00
test_fact_extraction_quality.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00
test_fact_ordering.py feat: introduce mental models (#132) 2026-01-16 11:16:41 +01:00
test_hierarchical_config.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00
test_http_api_integration.py fix: retain async with timestamp might fails (#253) 2026-01-30 14:54:32 +01:00
test_link_expansion_retrieval.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00
test_link_utils.py prepare for release 2025-12-03 11:52:25 +01:00
test_llm_provider.py fix(ci): resolve flaky test failures in api tests (#311) 2026-02-06 13:56:59 +01:00
test_llm_token_metrics.py feat: support for codex and claude-code as llm (#276) 2026-02-02 12:54:44 +01:00
test_llm_tools.py chore: internal renames (#204) 2026-01-27 09:53:28 +01:00
test_load_large_batch.py feat: new 'worker' service (#176) 2026-01-20 10:17:56 +01:00
test_main_module.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00
test_mcp_endpoint_routing.py Harden MCP server: fix routing, validation, and usage metering (#341) 2026-02-11 10:41:20 +01:00
test_mcp_extension.py Harden MCP server: fix routing, validation, and usage metering (#341) 2026-02-11 10:41:20 +01:00
test_mcp_local.py feat(mcp): add timestamp to retain (#190) 2026-01-23 16:00:43 +01:00
test_mcp_routing.py Harden MCP server: fix routing, validation, and usage metering (#341) 2026-02-11 10:41:20 +01:00
test_mcp_tools.py Harden MCP server: fix routing, validation, and usage metering (#341) 2026-02-11 10:41:20 +01:00
test_mental_model_hooks.py Add extension hooks for mental model operations (#260) 2026-01-31 09:30:53 -05:00
test_mental_models.py doc: prepare doc for 0.4.10 (#325) 2026-02-09 11:42:37 +01:00
test_metrics.py chore: remove dead code (#245) 2026-01-30 09:16:32 +01:00
test_mpfp_retrieval.py fix: improve mpfp retrieval (#146) 2026-01-12 18:58:05 +01:00
test_multilingual.py chore: remove dead code (#245) 2026-01-30 09:16:32 +01:00
test_observations.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00
test_per_operation_llm_config.py feat: add more config options for llm retries (#234) 2026-01-29 17:50:43 +01:00
test_provider_default_models.py feat: support for codex and claude-code as llm (#276) 2026-02-02 12:54:44 +01:00
test_query_analyzer.py fix: ci and ui build (#9) 2025-12-03 15:47:53 +01:00
test_reflect_agent.py fix: graph endpoint not showing links for observations (#214) 2026-01-28 14:51:25 +01:00
test_reflect_empty_based_on.py fix: resolve based_on schema/serialization issues in reflect API (#348) 2026-02-12 11:26:12 +01:00
test_reflect_tracing.py feat: add otel traceability (#330) 2026-02-10 12:20:48 +01:00
test_reflections.py feat: improve mental models ux on control plane (#297) 2026-02-04 15:49:03 +01:00
test_retain.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00
test_schema_isolation.py fix: improve mental model consolidation (#197) 2026-01-26 09:54:25 +01:00
test_search_trace.py feat: extensions (#54) 2025-12-22 11:05:23 +01:00
test_server_module.py fix: improve mental model consolidation (#197) 2026-01-26 09:54:25 +01:00
test_sql_schema_safety.py chore: internal renames (#204) 2026-01-27 09:53:28 +01:00
test_supabase_tenant.py Add Supabase tenant extension as built-in (#267) 2026-02-09 10:16:47 +01:00
test_tags_visibility.py chore: cleanup benchmarks runner with old flags (#212) 2026-01-28 13:22:48 +01:00
test_tei_cross_encoder.py fix: add defensive error handling to PyTorch device detection (#221) 2026-01-28 18:14:54 +01:00
test_temporal_ranges.py feat(mcp): add timestamp to retain (#190) 2026-01-23 16:00:43 +01:00
test_think.py chore: remove dead code (#245) 2026-01-30 09:16:32 +01:00
test_tracing.py feat: add otel traceability (#330) 2026-02-10 12:20:48 +01:00
test_tracing_integration.py feat: add otel traceability (#330) 2026-02-10 12:20:48 +01:00
test_tracing_spans_verification.py feat: add otel traceability (#330) 2026-02-10 12:20:48 +01:00
test_vertexai_provider.py Switch Vertex AI provider to native genai SDK (#242) 2026-01-30 08:35:59 +01:00
test_worker.py fix: worker doesn't pick up correct default schema (#259) 2026-01-31 09:15:59 +01:00