Commit graph

27 commits

Author SHA1 Message Date
Nicolò Boschi
90e370ef35
fix: misc fixes for observations and mental models (#209)
* fix: misc fixes for observations and mental models

* feat: improve graph retrieval for observations

- Update LinkExpansionRetriever to traverse through source_memory_ids
  for observation entity connections (avoiding data duplication)
- Remove entity link copy from world facts to observations in consolidator
- Add tests for link expansion graph retrieval
- Add directives_applied field to ReflectResult
- Include user's other changes (CLI, docs, client updates)

* fix: CI test failures

- Add mental_model_id parameter to create_mental_model function
- Fix ToolCallTrace not including reason field from ToolCall
- Improve test_link_expansion_observation_graph_retrieval to wait for consolidation with retry

* chore: reduce link expansion log verbosity

* Revert "chore: reduce link expansion log verbosity"

This reverts commit 3ce759391cead1012157785fa78fef16ef9bfe3b.

* feat: add semantic/temporal/entity links as fallback in graph retrieval

- Add fallback query for semantic, temporal, and entity links from memory_links
- Check both directions (outgoing and incoming links)
- Weight fallback results at 0.5x to prioritize entity links via unit_entities
- Fixes graph retrieval returning 0 when data has cross-cluster temporal connections

* fix: enable observations fixture for link expansion test

- Add enable_observations fixture to ensure observations are created
- Increase wait time from 10 to 30 seconds for CI reliability
2026-01-27 15:37:57 +01:00
Nicolò Boschi
5b52a84fff
chore: internal renames (#204)
This commit renames the terminology across the entire codebase:
- "mental models" (fact_type='mental_model' in memory_units) → "observations"
- "reflections" table (stored reflect responses) → "mental_models"

Changes include:
- Database migration to rename tables, indexes, and constraints
- API endpoints: /reflections → /mental-models, /mental-models → /observations
- Config: ENABLE_MENTAL_MODELS → ENABLE_OBSERVATIONS
- Response models and Pydantic classes
- Reflect agent tools and prompts
- Control plane UI and routes
- Documentation and examples
- Regenerated OpenAPI spec and client SDKs (Python, TypeScript)
- Rust CLI: reflection commands → mental-model commands
- LiteLLM: updated fact_types documentation
2026-01-27 09:53:28 +01:00
Nicolò Boschi
8e39cb7bc8
fix: improve mental model consolidation (#197)
* fix: improve mental model consolidation

* fix skill names

* fixes

* fix: add missing list_tenants to test mocks and update CLI for async refresh
2026-01-26 09:54:25 +01:00
Nicolò Boschi
9db64ecda3
feat: revisit mental models, directives and reflections (#179)
* chore: run benchmarks with reflect mode

* chore: run benchmarks with reflect mode

* fixes

* new mm

* bunch of fixes

* initial commit

* fixes

* fixes

* fixes

* fix: sometimes memories gets extracted in the wrong language
2026-01-22 17:13:16 +01:00
Nicolò Boschi
4c792400c1
feat: new 'worker' service (#176)
* feat: new 'worker' service

* doc

* docs

* tests
2026-01-20 10:17:56 +01:00
Chris Bartholomew
870cfccabb
Add structured JSON logging support (#170)
* Add structured JSON logging support

Add HINDSIGHT_API_LOG_FORMAT environment variable to configure log output
format. Options are "text" (default, human-readable) and "json" (structured).

JSON format outputs logs with a "severity" field that cloud logging systems
can parse for proper log level categorization. Also writes to stdout instead
of stderr so log levels are correctly interpreted.

* Rename GCPJsonFormatter to JsonFormatter
2026-01-19 09:01:24 +01:00
Nicolò Boschi
4f2833873c
feat: introduce mental models (#132)
* mental models

* DRAFT: refactor entity observations

* fix db patch

* agentic

* agentic

* reflect agent

* new style

* more

* fix ci

* fix

* fix
2026-01-16 11:16:41 +01:00
Nicolò Boschi
fa53917c63
feat: support custom url for openai embeddings & cohere (#150)
* feat: support custom url for openai embeddings & cohere

* feat: support custom url for openai embeddings & cohere
2026-01-13 14:01:44 +01:00
Nicolò Boschi
59913086be
fix: batch queries on recall (#149)
* fix: batch queries on recall

* fix: batch queries on recall
2026-01-13 13:20:22 +01:00
Nicolò Boschi
7935b0accd
fix: improve mpfp retrieval (#146)
* fix: improve mpfp retrieval

* fix: improve mpfp retrieval

* fix: improve embeddings service performances

* fix: improve embeddings service performances

* fix: improve embeddings service performances

* fix: improve embeddings service performances
2026-01-12 18:58:05 +01:00
Nicolò Boschi
eb2702bcba
misc: performance improvements (#140)
* misc: performance improvements

* misc: performance improvements

* misc: performance improvements
2026-01-09 14:47:20 +01:00
Nicolò Boschi
a6798f7e2a
fix: improve tei client parameters (#137)
* fix: improve tei client parameters

* fix: improve tei client parameters

* fix: improve tei client parameters
2026-01-09 11:31:22 +01:00
Nicolò Boschi
fb31a35a86
feat: retain modes (#136)
* feat: retain modes

* fix db patch
2026-01-09 11:30:36 +01:00
Nicolò Boschi
ba99b4422a
fix: misc perf improvements (#133)
* fix: misc perf improvements

* more tests

* fix test

* fix: update test files for new extract_facts_from_text signature

- Replace test_fact_extraction_token_analysis with test_fact_extraction_basic_analysis
  using inline sample content instead of external file
- Update test_fact_extraction_output_ratio.py to unpack 3 return values
  (facts, chunks, usage) instead of 2

* fix: make temporal tests more flexible for LLM variation

- test_temporal_absolute_conversion: check occurred_start field instead of
  requiring specific text in facts
- test_date_field_calculation_yesterday: make assertions conditional on
  having temporal data, add more content for better extraction
- test_temporal_ordering: reduce minimum required facts from 3 to 2
2026-01-08 22:49:04 +01:00
Nicolò Boschi
33fac2c5e2
feat: add configs for database connection (#128) 2026-01-08 16:37:08 +01:00
Nicolò Boschi
49e233cdb7
fix: duplicated causal relationships and token optimization (#126)
* fix: duplicated causal relationships and token optimization

* doc

* doc
2026-01-08 14:43:48 +01:00
Nicolò Boschi
e6709d541f
feat: support different provider/models per operation (#125)
* feat: support different provider/models per operation

* fix tests
2026-01-08 14:02:57 +01:00
Nicolò Boschi
233bd2e5d4
feat: run db migrations offline (optionally) (#114)
* feat: run db migrations offline (optionally)

* fix
2026-01-07 15:49:51 +01:00
Nicolò Boschi
7715a5110e
fix: make retain max completion tokens configurable (#109)
* fix: make retain max completion tokens configurable

* fix: make retain max completion tokens configurable
2026-01-07 10:26:42 +01:00
Chris Bartholomew
ce45d301ce
Add operation validator extension support with proper HTTP error handling (#86)
* Load operation validator extension in main entry point

Enable the operation validator extension to be loaded from environment
configuration and passed to MemoryEngine, allowing pre/post operation
hooks for usage metering, rate limiting, and audit logging.

* Fix reflect background task authentication and add internal flag

- Pass API key to background opinion storage task for proper auth
- Add internal flag to RequestContext for tracking internal operations
- Background opinion storage now authenticates correctly with tenant

* Add api_key_id to RequestContext for usage tracking

- Add api_key_id field to RequestContext to track which API key was used
- Enables per-API-key usage analytics in the metering system

* Fix HTTP error handling for authentication and validation errors

- Add status_code parameter to ValidationResult and OperationValidationError
- Convert OperationValidationError to HTTPException with proper status codes
- Fix authentication errors to return 401 instead of raising internal errors
- Re-raise HTTPException in exception handlers to prevent swallowing errors

* Fix AuthenticationError handling in memory engine

- Raise AuthenticationError from memory_engine._authenticate_tenant instead
  of HTTPException so unit tests pass
- Add AuthenticationError handling in HTTP layer to convert to 401 responses
- Fixes failing TestMemoryEngineTenantAuth tests

* Add global exception handler for AuthenticationError

Returns proper 401 status code for all authentication failures
across all endpoints, not just the ones with explicit handlers.

* Simplify exception handling: use global AuthenticationError handler

- Remove redundant individual exception handlers
- Add 'except AuthenticationError: raise' before generic Exception handlers
  to let global handler process auth errors uniformly

* Refactor background tasks to use tenant_id instead of api_key

This makes the core more generic - it passes tenant_id (which is
extension-agnostic) rather than api_key (which is cloud-specific).

- Add tenant_id field to RequestContext
- Pass tenant_id instead of api_key to background tasks
- Extensions can check internal=True with tenant_id to bypass normal auth

* Fix exception propagation: include HTTPException in re-raise

After cleanup of redundant exception handlers, 404 errors were
returning 500 because HTTPException was caught by the generic
except Exception handler. Fixed by combining AuthenticationError
and HTTPException in the re-raise pattern.
2026-01-01 20:19:52 -05:00
Chris Bartholomew
dfa7cec05b
Load operation validator extension in main entry point (#72)
Enable the operation validator extension to be loaded from environment
configuration and passed to MemoryEngine, allowing pre/post operation
hooks for usage metering, rate limiting, and audit logging.
2025-12-23 15:47:26 +01:00
Nicolò Boschi
e6511e7d77
feat: refactor hindsight-embed architecture (#66)
* feat: refactor hindsight-embed architecture

* feat: refactor hindsight-embed architecture

* refactor deamin

* refactor deamin

* refactor deamin

* refactor deamin
2025-12-22 22:02:40 +01:00
Nicolò Boschi
2a0c490c9e
feat: extensions (#54) 2025-12-22 11:05:23 +01:00
Nicolò Boschi
1c6acc3ba0
feat: simplify mcp installation + ui standalone (#41) 2025-12-18 10:24:56 +01:00
Nicolò Boschi
9394cf92f2
fix: doc build and lint files (#34)
* fix doc build

* fix doc build
2025-12-16 13:49:09 +01:00
Nicolò Boschi
fa554b8980 brandind and misc fixes 2025-12-11 12:46:48 +01:00
Nicolò Boschi
f42476bf94
fix: make sure openai provider works + docs updates (#23)
* fix: make sure openai provider works

* fix: make sure openai provider works

* fix
2025-12-10 16:10:10 +01:00