Commit graph

25 commits

Author SHA1 Message Date
Nicolò Boschi
a56cd044e5
feat: 4-tab code parity across all documentation examples (#613)
* feat: independent versioning for integrations

- Add per-integration changelog pages at /changelog/integrations/<name>
- Move main changelog to changelog/index.md (URL unchanged)
- Add --integration flag to generate-changelog for LLM-based per-integration changelog generation
- Add scripts/release-integration.sh <name> <version> for cutting integration releases
- Add .github/workflows/release-integration.yml to publish on integrations/** tags
- Remove integrations from main release.sh and release.yml cycle

* fix: add agno and hermes integration docs to version-0.4 for production build

* chore: apply ruff formatting to generate_changelog.py

* feat: add 4-tab code parity across all documentation examples

Every code snippet Tabs block now has Python, Node.js, CLI, and Go variants.
Raw HTTP/curl tabs replaced with proper SDK calls.

New example files:
- Go: retain.go, recall.go, reflect.go, memory-banks.go, directives.go,
  mental-models.go, documents.go, main-methods.go
- Shell: memory-banks.sh, directives.sh, mental-models.sh
- Node.js: mental-models.mjs

Extended example files with missing sections:
- recall.mjs/sh: world/experience/observation types, token-budget, all tag modes
- reflect.sh: reflect-with-params, reflect-disposition, reflect-sources, reflect-with-tags
- reflect.mjs: reflect-with-tags, fixed reflect-sources API usage
- retain.mjs/sh: retain-conversation, retain-batch, retain-files-batch

SDK/CLI additions:
- TypeScript: getMentalModelHistory method
- CLI recall: --tags, --tags-match flags
- CLI reflect: --tags, --tags-match, --include-facts flags
- CLI directive update: --is-active flag
- CLI bank set-config: --retain-mission, --retain-extraction-mode,
  --observations-mission, --reflect-mission, --disposition-* flags

Build validation:
- scripts/check-code-parity.mjs validates 4-tab parity across all MDX files
- Integrated into npm run build — fails if any Tabs block is missing a variant

* fix: fix doc examples for Go, Node.js, CLI + add mental model with-id examples

- Fix Go Budget constants: BUDGET_HIGH/LOW/MID → HIGH/LOW/MID
- Fix Go documents.go: ListDocuments returns []map[string]interface{}, use map access
- Fix Go retain.go: use correct relative path for sample.pdf
- Fix Node.js createMentalModel: use positional args (name, sourceQuery) not object
- Add CLI 'history' subcommand for mental models (api.rs, main.rs, mental_model.rs)
- Rebuild TypeScript/Python clients to support id param in createMentalModel
- Add create-mental-model-with-id examples across all 4 languages and docs

* fix: move id param to end of create_mental_model signature for backwards compat
2026-03-19 11:31:51 +01:00
Nicolò Boschi
f7a60f898d
feat: filter operations by type + fix stale auto-refresh closure (#522) (#527)
* feat: filter operations by type + fix stale closure in auto-refresh

- Add `type` query param to GET /operations endpoint and engine layer
- Add operation type dropdown filter in Background Operations UI
- Fix auto-refresh interval using stale statusFilter/offset closure by
  adding filter state to useEffect deps and wrapping loadOperations in
  useCallback (fixes #522)
- Regenerate OpenAPI spec and all SDK clients

* fix: update Rust CLI list_operations call with new type parameter
2026-03-09 13:17:00 +01:00
Nicolò Boschi
1d70abfe85
feat: add tags filtering and q description fix for list documents API (#468)
* feat: add Pydantic AI integration to CI, release pipeline, and docs

- Add test-pydantic-ai-integration job to CI (test.yml)
- Add build, publish, and artifact steps to release workflow (release.yml)
- Add hindsight-integrations/pydantic-ai to release.sh version bumping
- Add Pydantic AI documentation page (sdks/integrations/pydantic-ai.md)
- Add Pydantic AI entry to sidebar with icon

* docs: remove Requirements section from pydantic-ai integration page

* feat: add tags filtering and fix offset pagination docs for list documents API

- Add `tags` and `tags_match` query params to GET /banks/{bank_id}/documents
- Supports any, all, any_strict, all_strict matching modes (default: any_strict)
- Fix `q` param description — it's a case-insensitive substring match on document ID only
- Add tests for offset pagination and all tags_match modes
- Regenerate OpenAPI spec and Python/TypeScript/Go clients
- Document the new filtering options in docs/developer/api/documents.mdx

* fix(cli): pass new tags/tags_match args to list_documents
2026-03-02 17:03:16 +01:00
Nicolò Boschi
0bb5ca4caf
feat: filter graph memories with tags (#431)
* feat: filter graph memories with tags

* fix(cli): pass new q/tags/tags_match args to get_graph

* docs: use CodeSnippet for tags_match examples in recall.mdx
2026-02-25 10:31:40 +01:00
Nicolò Boschi
2a32273226
feat: increase customization for reflect, retain and consolidation (#419) 2026-02-23 20:35:32 +01:00
Nicolò Boschi
224b7b74c1
feat: accept pdf, images and office files (#390)
* feat: accept pdf, images and office files

* refactor: rename FileConverter to FileParser, simplify file retain API

- Rename engine/converters/ → engine/parsers/, FileConverter → FileParser,
  ConverterRegistry → FileParserRegistry, MarkitdownConverter → MarkitdownParser
- Rename env var HINDSIGHT_API_FILE_CONVERTER → HINDSIGHT_API_FILE_PARSER
- Remove async/document_tags params from FileRetainRequest (always async now)
- Add retain_files() to Python Hindsight client and retainFiles() to TypeScript client
- Add sample.pdf to doc examples for working file upload demonstrations
- Update test_file_retain.py to use new parser names and always-async behavior
- Fix Go client missing os import in api_files.go
- Simplify postgresql.py storage to minimal schema

* fix: update rust CLI tests to use is_supported_file instead of is_text_file

* fix: patch Go api_files.go to add missing 'os' import after generation

* fix: insert 'os' import after 'net/url' in api_files.go patch for correct position

* chore: regenerate OpenAPI spec and clients (converter→parser description update)
2026-02-17 18:15:03 +01:00
Nicolò Boschi
8d731f2e5f
feat: implement hierarchical configuration (system, tenant, bank) (#329)
* feat: implement hierarchical configuration (system, tenant, bank)

* feat: implement hierarchical configuration (system, tenant, bank)

* docs: add instructions for hierarchical config in CLAUDE.md

* feat: add ENABLE_BANK_CONFIG_API flag (disabled by default)

- Add HINDSIGHT_API_ENABLE_BANK_CONFIG_API env var (default: false)
- Return 403 Forbidden from bank config endpoints when disabled
- Update tests to enable the flag
- Update CLAUDE.md documentation

This provides security control over the bank configuration API,
ensuring it's only accessible when explicitly enabled.

* docs: add hierarchical configuration section

* feat(cli): add bank config commands (config, set-config, reset-config)

- Add 'hindsight bank config' to view bank configuration
- Add 'hindsight bank set-config' to update LLM settings per bank
- Add 'hindsight bank reset-config' to reset to defaults
- Implements client API calls to new bank config endpoints

* fix(cli): fix compilation errors in bank config commands

- Fix type signature: use ApiClient instead of api::Client
- Fix confirmation: use ui::prompt_confirmation instead of ui::confirm
- Fix error handling: use anyhow! macro instead of errors::Error
- Fix type conversion: convert HashMap to serde_json::Map for API call

* feat: implement type-safe hierarchical config with bank overrides

Implements a production-ready hierarchical configuration system that prevents
accidentally using global defaults when bank-specific overrides exist.

- Created StaticConfigProxy that wraps HindsightConfig
- get_config() now returns proxy that blocks access to bank-configurable fields
- Raises ConfigFieldAccessError with clear message when accessing configurable fields
- Added _get_raw_config() for internal use only
- Forces developers to use resolve_full_config(bank_id, context) for bank settings

- Added resolve_full_config() method that returns complete HindsightConfig
- Resolves hierarchy: Global (env) → Tenant → Bank
- No caching to support multi-server deployments (always fresh from DB)
- LLM provider pooling handles expensive operations separately

- Updated entire retain pipeline to pass resolved config through call chain
- memory_engine.py: Resolves config at top level where bank_id/context available
- orchestrator.py: Accepts and passes config to fact_extraction
- fact_extraction.py: Uses passed config instead of get_config()
- utils.py: Added optional config param for backward compatibility

- consolidator.py: Uses resolve_full_config() for enable_observations check
- memory_engine.py: Resolves config before triggering consolidation

- Renamed "Memory Bank" to "Bank Configuration" with tabs
- Combined Stats and Operations into "General" tab
- Consolidated Profile and Configuration into "Configuration" tab
- Moved Actions dropdown to page level (outside tabs)

- Created new component for managing bank-specific config
- Displays configurable fields: retain_chunk_size, retain_extraction_mode, etc.
- Edit via dialog with form validation
- Reset to defaults via AlertDialog confirmation
- Shows field IDs in monospace for clarity
- Visual separation with borders and hover effects

- Removed inline edit mode, switched to dialog-based editing
- Separate dialogs for Disposition and Mission editing
- Read-only display with clear edit buttons
- Removed duplicate stats cards and operations

- bank-stats-view.tsx: Overview statistics (memories, links, documents, pending ops)
- bank-operations-view.tsx: Background operations table with filtering

**Problem**: Consolidation always used global enable_observations, ignoring bank overrides
**Root Cause**: consolidator.py called get_config() instead of resolving bank-specific config
**Solution**: Pass resolved config through the entire pipeline

**Problem**: asyncpg returning JSONB as JSON string instead of parsed dict
**Solution**: Explicit JSON parsing in config_resolver.py with type checking

- All 19 API integration tests pass
- All 10 hierarchical config tests pass
- Retain operations work correctly with bank-specific config
- Consolidation respects bank-specific enable_observations setting

- Updated developer/configuration.md with type-safe config access pattern
- Added examples showing correct usage patterns
- Documented ConfigFieldAccessError and resolution methods

- get_config() now returns StaticConfigProxy (blocks configurable field access)
- Code accessing bank-configurable fields must use resolve_full_config()
- Clear migration path with helpful error messages

Fixes hierarchical configuration to be production-ready with proper type safety.

* refactor: remove LLM client pool and simplify config resolver

Since LLM config (provider, model, api_key) is now static and not
bank-configurable, the LLMClientPool is no longer needed.

Changes:
- Remove hindsight_api/llm_client_pool.py (no longer needed)
- Remove memory_engine._get_bank_llm_config() (dead code, never called)
- Simplify config_resolver.py by eliminating duplication between
  resolve_full_config() and get_bank_config()
- get_bank_config() now calls resolve_full_config() and filters results
- Remove outdated "LLM provider pooling" comments from docstrings

All tests pass (10 hierarchical config tests, 19 API integration tests)

* fix: update tests to use _get_raw_config() for configurable fields

Fixed test fixtures that were accessing configurable fields (like
enable_observations) from get_config(), which now raises
ConfigFieldAccessError due to type-safe config access.

Changes:
- test_consolidation.py: Changed enable_observations fixture to use
  _get_raw_config() instead of get_config()
- test_consolidation.py: Updated test_consolidation_returns_disabled_status
  to set bank config instead of mocking get_config()
- test_link_expansion_retrieval.py: Changed fixture to use _get_raw_config()
- test_observations.py: Changed disable_observations fixture to use
  _get_raw_config()
- Regenerated OpenAPI spec and clients

All 39 previously failing tests now pass.

* fix: add missing config parameter to test calls of extract_facts_from_text()

Fixed 45 test failures where tests were calling extract_facts_from_text()
without the new required config parameter.

Changes:
- Added config=_get_raw_config() to all extract_facts_from_text() calls
- Fixed test_main_module.py to patch _get_raw_config instead of get_config
- Updated 6 test files with 37 function call sites

All tests should now pass.

* fix: add missing config parameter to test_skip_podcast_meta_commentary

One more test was missing the config parameter for extract_facts_from_text().
2026-02-12 13:14:57 +01:00
Nicolò Boschi
90e370ef35
fix: misc fixes for observations and mental models (#209)
* fix: misc fixes for observations and mental models

* feat: improve graph retrieval for observations

- Update LinkExpansionRetriever to traverse through source_memory_ids
  for observation entity connections (avoiding data duplication)
- Remove entity link copy from world facts to observations in consolidator
- Add tests for link expansion graph retrieval
- Add directives_applied field to ReflectResult
- Include user's other changes (CLI, docs, client updates)

* fix: CI test failures

- Add mental_model_id parameter to create_mental_model function
- Fix ToolCallTrace not including reason field from ToolCall
- Improve test_link_expansion_observation_graph_retrieval to wait for consolidation with retry

* chore: reduce link expansion log verbosity

* Revert "chore: reduce link expansion log verbosity"

This reverts commit 3ce759391cead1012157785fa78fef16ef9bfe3b.

* feat: add semantic/temporal/entity links as fallback in graph retrieval

- Add fallback query for semantic, temporal, and entity links from memory_links
- Check both directions (outgoing and incoming links)
- Weight fallback results at 0.5x to prioritize entity links via unit_entities
- Fixes graph retrieval returning 0 when data has cross-cluster temporal connections

* fix: enable observations fixture for link expansion test

- Add enable_observations fixture to ensure observations are created
- Increase wait time from 10 to 30 seconds for CI reliability
2026-01-27 15:37:57 +01:00
Nicolò Boschi
5b52a84fff
chore: internal renames (#204)
This commit renames the terminology across the entire codebase:
- "mental models" (fact_type='mental_model' in memory_units) → "observations"
- "reflections" table (stored reflect responses) → "mental_models"

Changes include:
- Database migration to rename tables, indexes, and constraints
- API endpoints: /reflections → /mental-models, /mental-models → /observations
- Config: ENABLE_MENTAL_MODELS → ENABLE_OBSERVATIONS
- Response models and Pydantic classes
- Reflect agent tools and prompts
- Control plane UI and routes
- Documentation and examples
- Regenerated OpenAPI spec and client SDKs (Python, TypeScript)
- Rust CLI: reflection commands → mental-model commands
- LiteLLM: updated fact_types documentation
2026-01-27 09:53:28 +01:00
Nicolò Boschi
8e39cb7bc8
fix: improve mental model consolidation (#197)
* fix: improve mental model consolidation

* fix skill names

* fixes

* fix: add missing list_tenants to test mocks and update CLI for async refresh
2026-01-26 09:54:25 +01:00
Nicolò Boschi
9db64ecda3
feat: revisit mental models, directives and reflections (#179)
* chore: run benchmarks with reflect mode

* chore: run benchmarks with reflect mode

* fixes

* new mm

* bunch of fixes

* initial commit

* fixes

* fixes

* fixes

* fix: sometimes memories gets extracted in the wrong language
2026-01-22 17:13:16 +01:00
Nicolò Boschi
fe4ed1db73
feat(clients): mental models api (#172)
* feat(clients): mental models api

* fixes

* more tests

* fixes
2026-01-19 14:49:49 +01:00
Nicolò Boschi
4f2833873c
feat: introduce mental models (#132)
* mental models

* DRAFT: refactor entity observations

* fix db patch

* agentic

* agentic

* reflect agent

* new style

* more

* fix ci

* fix

* fix
2026-01-16 11:16:41 +01:00
Nicolò Boschi
1eeced3116
feat(cli): accept more file types on retain-files (#163)
* feat(cli): accept more file types on retain-files

* feat(cli): accept more file types on retain-files
2026-01-15 18:34:44 +01:00
Nicolò Boschi
26bf5714cd
fix: entities list only show 100 entities (#142)
* fix: entities list only show 100 entities

* fix: update Rust CLI for entities pagination API changes
2026-01-12 18:50:53 +01:00
Nicolò Boschi
4135a6cee5
ci: frozen uv sync (#138)
* ci: frozen uv sync

* fix: add missing authorization parameter to get_agent_stats in CLI

The generated Rust client was updated with an authorization header
parameter for get_agent_stats, but the CLI code wasn't updated.
2026-01-09 16:43:00 +01:00
Nicolò Boschi
e06a6120a3
feat(misc): update clients types, test coverage, improve /health endpoint and add changelog (#70)
* doc: changelog and delete doc info

* others

* others

* fixes

* fixes
2025-12-23 12:49:31 +01:00
Nicolò Boschi
2a0c490c9e
feat: extensions (#54) 2025-12-22 11:05:23 +01:00
Nicolò Boschi
fa554b8980 brandind and misc fixes 2025-12-11 12:46:48 +01:00
Nicolò Boschi
040eb33ea6
improve ci and tests (#22)
* improve ci and tests

* add more tests

* fixes

* fix tests

* fix more

* fix

* fix

* fix

* fix

* fix for real

* tests and doc

* fix cp

* fix link pg0

* fix pg0

* fix pg0

* fix pg0

* even better

* more

* fix
2025-12-09 19:16:00 +01:00
Nicolò Boschi
c41490085a fix node build 2025-12-05 01:21:30 +01:00
Nicolò Boschi
06b956a553 new names 2025-12-04 21:34:05 +01:00
Nicolò Boschi
6073ac4ffd docs, packages and quick start 2025-12-04 12:49:01 +01:00
Nicolò Boschi
d099ef870d rename to new names 2025-11-27 16:22:16 +01:00
Nicolò Boschi
b1ff2e8823
rename to hindsight (#2) 2025-11-25 19:28:26 +01:00
Renamed from memora-cli/src/api.rs (Browse further)