Move the Supabase tenant extension into the hindsight-api package so users
can enable it with just an environment variable — no file copying or Docker
image modifications needed.
Key improvements over the original submission:
- JWKS-based local JWT verification (no network call per request) with
automatic fallback to /auth/v1/user for legacy HS256 projects
- Service key is now optional (only needed for HS256 or health checks)
- UUID validation on user IDs before schema name construction
- Schema prefix validation against Postgres identifier rules
- Key rotation handling with automatic JWKS cache refresh
- Proper logging via Python logging module
- Tenant extension lifecycle hooks (on_startup/on_shutdown) wired into
the server lifespan
- Public tenant_extension property on MemoryEngine
- 54 unit tests covering both verification modes, cache behavior, error
paths, and the extension loader
- README updated to reflect JWKS-first architecture
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: tagged directives should be applied to tagged mental models
* test: add unit test for based_on structure
Verify that reflect returns the correct based_on structure with:
- directives as dicts (id, name, content) in based_on.directives
- mental models as MemoryFact objects in based_on.mental-models
- memories separated properly
This ensures directives and mental models are not mixed together
in the API response.
* feat: ai sdk integration
* more fixes
* fix(security): mental model refresh tag-based security
- Mental model refresh now passes tags with all_strict matching
- Consolidation only triggers refresh for mental models with matching tags
- Consolidation filters related observations by tags (all_strict)
- Added tests to verify tag-based security boundaries
- Updated OpenAPI spec to include tags and text_preview in list_documents
- Added tags column to documents UI table
* chore: regenerate OpenAPI spec after rebase
* fix: improve consolidation prompt for contradiction handling and mental model refresh security
- Enhanced consolidation prompt to be more explicit about capturing temporal changes in contradictions
- Fixed mental model refresh security: tagged memories now only trigger refresh of mental models with matching tags
- Added stricter tag filtering to prevent cross-scope mental model refreshes
Fixes test_consolidation_merges_contradictions by improving LLM instructions to use temporal markers like "used to X, now Y" when merging contradictory facts.
Note: test_refresh_with_tags_only_accesses_same_tagged_models still needs investigation - REFLECT operation may need additional tag filtering.
* fix: mental model refresh security - proper tag filtering in search
Fixed tool_search_mental_models to properly handle all_strict tag matching mode by using the centralized build_tags_where_clause function. Previously, the function only handled "all" vs "any" modes and always included untagged mental models when using non-"all" modes.
This ensures that when a tagged mental model is refreshed with all_strict matching, it cannot access untagged mental models, preventing cross-scope information leakage.
Fixes test_refresh_with_tags_only_accesses_same_tagged_models.
Note: test_sensory_dimension_preservation is failing but this is a pre-existing issue on main branch - the LLM model (gpt-oss-20b) is not extracting facts from sensory text. Not related to security changes.
* chore: apply formatting from pre-commit hook
* fix: allow untagged mental models to be refreshed by any consolidation
Untagged mental models are considered "global" and should be refreshed
by any consolidation, regardless of whether tagged or untagged memories
were consolidated. This maintains security boundaries while allowing
global mental models to stay fresh.
When tagged memories are consolidated:
- Refresh mental models with matching tags (security boundary)
- Also refresh untagged mental models (they're global)
- DO NOT refresh mental models with different tags
When untagged memories are consolidated:
- Only refresh untagged mental models
- DO NOT refresh tagged mental models (security boundary)
Fixes test_consolidation_only_refreshes_matching_tagged_models.
* fix: custom pg schema is not reliable
* fix
* fix
* fix: WorkerPoller now always has tenant extension
Ensures WorkerPoller follows same pattern as MemoryEngine - always
creates a DefaultTenantExtension if none is provided, preventing
NoneType errors when calling list_tenants().
Fixes test failures in test_worker.py
* fix: DefaultTenantExtension honors explicit schema parameter
Allows WorkerPoller's schema parameter to be passed through to
DefaultTenantExtension via config dict, maintaining backward
compatibility for tests that use schema parameter without
providing a tenant extension.
Fixes test_poller_with_custom_schema test failure.
* feat: support for codex and claude-code as llm
* Remove refactoring plan file
* Consolidate Anthropic tests into main LLM provider test suite
- Add Anthropic models (Sonnet, Opus, Haiku) to MODEL_MATRIX
- Remove separate test_anthropic_provider.py file
- All Anthropic models now tested with standard memory operations
* Add provider-specific default models
Each LLM provider now has a sensible default model that's used when
HINDSIGHT_API_LLM_MODEL is not explicitly set. This simplifies
configuration - users can specify just the provider and API key.
Changes:
- Add PROVIDER_DEFAULT_MODELS mapping in config.py
- Update config logic to use provider defaults for both global and
per-operation LLM configs
- Add comprehensive tests for provider default model selection
- Document provider defaults in models.md
Example usage:
export HINDSIGHT_API_LLM_PROVIDER=anthropic
export HINDSIGHT_API_LLM_API_KEY=sk-ant-xxx
# Automatically uses claude-sonnet-4-20250514
Provider defaults:
- openai: gpt-5-mini
- anthropic: claude-sonnet-4-20250514
- gemini: gemini-2.5-flash
- groq: openai/gpt-oss-120b
- ollama: gemma3:12b
- lmstudio: local-model
- vertexai: gemini-2.0-flash-001
- openai-codex: o3-mini
- claude-code: claude-sonnet-4-20250514
- mock: mock-model
* Update provider default models
- openai: gpt-5-mini -> o3-mini
- anthropic: claude-sonnet-4-20250514 -> claude-haiku-4-5-20251001
- openai-codex: o3-mini -> gpt-5.2-codex
- claude-code: claude-sonnet-4-20250514 -> claude-sonnet-4-5-20250929
Updated tests and documentation to reflect new defaults.
* Move OpenAI Codex and Claude Code setup to models.md
Moved detailed setup instructions for OpenAI Codex and Claude Code from
configuration.md to models.md where they better fit with model-specific
documentation.
Changes:
- Move "OpenAI Codex Setup" section from configuration.md to models.md
- Move "Claude Code Setup" section from configuration.md to models.md
- Add cross-reference tip in configuration.md pointing to models.md
- Update default model in Claude Code example to claude-sonnet-4-5-20250929
- Keep basic provider examples in configuration.md for quick reference
This makes the configuration.md page more focused on environment
variables while models.md contains provider-specific setup details.
The batch_retain and consolidation task handlers created internal
RequestContext objects without tenant_id or api_key_id. This meant
downstream operations (consolidation, mental model refreshes) triggered
by async workers lost the original caller's request context.
Fix by passing tenant_id and api_key_id through the task payload dict
in submit_async_retain and submit_async_consolidation, then restoring
them in the corresponding handlers (_handle_batch_retain,
_handle_consolidation).
Add dataclasses and hook methods to OperationValidatorExtension for
tracking mental model operations:
- MentalModelGetContext/Result: context and result for GET operations
- MentalModelRefreshResult: result for refresh operations with token counts
- validate_mental_model_get: pre-operation validation hook
- on_mental_model_get_complete: post-GET completion hook
- on_mental_model_refresh_complete: post-refresh completion hook
Invoke hooks in http.py (GET endpoint) and memory_engine.py (refresh).
Add tests verifying hooks are called with correct parameters.
* chore: remove dead code
* chore: remove extract_opinions from test and regenerate openapi
- Remove extract_opinions parameter from test_fact_extraction_analysis
- Regenerate OpenAPI spec after removing entity observations code
* chore: update generated files and apply formatting
- Regenerate Python and TypeScript client SDKs after main merge
- Apply ruff formatting to llm_wrapper.py
* fix: accept and filter deprecated 'opinion' fact type in recall
The dead code removal eliminated support for the 'opinion' fact type,
but existing clients may still pass it. Instead of rejecting it with
a ValueError, silently filter it out before validation to maintain
backward compatibility.
Previously, _authenticate_tenant only skipped extension auth for
internal requests when _current_schema was set to a non-public schema.
This caused async HTTP retain (document upload with async_processing=True)
to fail with AuthenticationError because the worker had no API key and
the schema was "public".
Remove the public-schema guard since internal tasks were already
authenticated at submission time. The worker sets _current_schema from
the task's _schema field for tenant schemas, and it defaults to "public"
for public schema tasks — both are valid.
* feat: support vertex as llm provider
* fix
* fix: add uv index-strategy to resolve dependency conflicts with pytorch index
When using pytorch index for faster torch downloads in CI,
filelock dependency resolution was failing because pytorch index
only has older versions. Adding unsafe-best-match strategy allows
uv to search all configured indexes.
Also fix type checking warnings from ty.
* fix: add index-strategy to root pyproject.toml for workspace-level uv resolution
* chore: regenerate client SDKs after Vertex AI support
Tenant schemas were never migrated when new migrations were deployed.
Only the public schema was migrated at startup, and tenant schemas only
got migrations when first provisioned. This meant existing tenants
missed any new columns (e.g. task_payload, worker_id, claimed_at on
async_operations), causing the worker poller to crash silently.
Changes:
- Run migrations on all existing tenant schemas at startup when a
tenant_extension is configured. Each schema migration is wrapped in
try/except so one failure doesn't block others.
- Add try/except in WorkerPoller.recover_own_tasks() so a broken
schema doesn't prevent the polling loop from starting.
- Add try/except in WorkerPoller._claim_batch_for_schema() so a
broken schema doesn't prevent claiming tasks from other schemas.
* chore: cleanup benchmarks runner with old flags
* fix tests
* fix: observations rely on source_memory_ids, no link copying
Observations no longer copy any memory_links from their source facts.
Instead, retrieval uses source_memory_ids to traverse:
- Entity connections: observation → source_memory_ids → unit_entities
- Semantic similarity: observations have their own embeddings
- Temporal proximity: observations have their own temporal fields
This avoids data duplication and fixes bidirectionality issues with
entity links being copied to observations.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* test: update consolidation test for source_memory_ids behavior
Updated test_consolidation_creates_memory_links to test_consolidation_uses_source_memory_ids
to reflect the new behavior where observations use source_memory_ids instead of memory_links
for traversal.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: misc fixes for observations and mental models
* feat: improve graph retrieval for observations
- Update LinkExpansionRetriever to traverse through source_memory_ids
for observation entity connections (avoiding data duplication)
- Remove entity link copy from world facts to observations in consolidator
- Add tests for link expansion graph retrieval
- Add directives_applied field to ReflectResult
- Include user's other changes (CLI, docs, client updates)
* fix: CI test failures
- Add mental_model_id parameter to create_mental_model function
- Fix ToolCallTrace not including reason field from ToolCall
- Improve test_link_expansion_observation_graph_retrieval to wait for consolidation with retry
* chore: reduce link expansion log verbosity
* Revert "chore: reduce link expansion log verbosity"
This reverts commit 3ce759391cead1012157785fa78fef16ef9bfe3b.
* feat: add semantic/temporal/entity links as fallback in graph retrieval
- Add fallback query for semantic, temporal, and entity links from memory_links
- Check both directions (outgoing and incoming links)
- Weight fallback results at 0.5x to prioritize entity links via unit_entities
- Fixes graph retrieval returning 0 when data has cross-cluster temporal connections
* fix: enable observations fixture for link expansion test
- Add enable_observations fixture to ensure observations are created
- Increase wait time from 10 to 30 seconds for CI reliability
Background tasks (async retain, consolidation, reflections) fail in
multi-tenant deployments because the worker executes tasks without
setting the tenant schema context. This causes two failures:
1. The cancellation check in execute_task queries public.async_operations
instead of the tenant's schema, finds no row, and skips the task as
"cancelled" — even though it wasn't.
2. Even if that were fixed, _authenticate_tenant would throw
AuthenticationError because background tasks have no API key.
Changes:
- Poller passes task.schema into task_dict so execute_task can set it
- execute_task sets _current_schema before the cancellation check
- Task handlers use RequestContext(internal=True) to signal background ops
- _authenticate_tenant skips extension auth for internal requests when
schema is already set
- BrokerTaskBackend uses schema_getter for dynamic schema resolution
when submitting tasks and waiting for results
- Pass tenant_extension to WorkerPoller in create_app
The graph endpoint's table_rows response was missing three fields that
the control plane UI expects:
- tags: memory unit tags (shown in Tags column)
- created_at: creation timestamp (shown in Created column for mental models)
- proof_count: source memory count (shown in Sources column for mental models)
All three columns exist on the memory_units table but were not being
selected or included in the response.
* chore: run benchmarks with reflect mode
* chore: run benchmarks with reflect mode
* fixes
* new mm
* bunch of fixes
* initial commit
* fixes
* fixes
* fixes
* fix: sometimes memories gets extracted in the wrong language
* feat: Record LLM token metrics via Prometheus
Wire up the existing token metrics infrastructure to actually record
token usage from LLM calls. The MetricsCollector already had
record_tokens() method and Prometheus counters (hindsight.tokens.input,
hindsight.tokens.output), but they were never being populated.
Changes:
- Import get_metrics_collector in llm_wrapper.py
- Call record_tokens() after successful LLM calls for:
- OpenAI/Groq (using response.usage.prompt_tokens, completion_tokens)
- Anthropic (using response.usage.input_tokens, output_tokens)
- Gemini (using response.usage_metadata.prompt_token_count, candidates_token_count)
- Add test file to verify token metrics are recorded
Note: Ollama's native API doesn't return token usage, so metrics
are not recorded for that provider.
The token metrics will now be available via /metrics endpoint:
- hindsight_tokens_input_total
- hindsight_tokens_output_total
* feat: add per-request token usage tracking to retain and reflect endpoints
- Add TokenUsage model with input_tokens, output_tokens, total_tokens
- Return usage metrics in retain response (sync operations only)
- Return usage metrics in reflect response
- Update Python, TypeScript, and Rust clients
- Add API documentation for usage fields
- Add changelog entry
* Load operation validator extension in main entry point
Enable the operation validator extension to be loaded from environment
configuration and passed to MemoryEngine, allowing pre/post operation
hooks for usage metering, rate limiting, and audit logging.
* Fix reflect background task authentication and add internal flag
- Pass API key to background opinion storage task for proper auth
- Add internal flag to RequestContext for tracking internal operations
- Background opinion storage now authenticates correctly with tenant
* Add api_key_id to RequestContext for usage tracking
- Add api_key_id field to RequestContext to track which API key was used
- Enables per-API-key usage analytics in the metering system
* Fix HTTP error handling for authentication and validation errors
- Add status_code parameter to ValidationResult and OperationValidationError
- Convert OperationValidationError to HTTPException with proper status codes
- Fix authentication errors to return 401 instead of raising internal errors
- Re-raise HTTPException in exception handlers to prevent swallowing errors
* Fix AuthenticationError handling in memory engine
- Raise AuthenticationError from memory_engine._authenticate_tenant instead
of HTTPException so unit tests pass
- Add AuthenticationError handling in HTTP layer to convert to 401 responses
- Fixes failing TestMemoryEngineTenantAuth tests
* Add global exception handler for AuthenticationError
Returns proper 401 status code for all authentication failures
across all endpoints, not just the ones with explicit handlers.
* Simplify exception handling: use global AuthenticationError handler
- Remove redundant individual exception handlers
- Add 'except AuthenticationError: raise' before generic Exception handlers
to let global handler process auth errors uniformly
* Refactor background tasks to use tenant_id instead of api_key
This makes the core more generic - it passes tenant_id (which is
extension-agnostic) rather than api_key (which is cloud-specific).
- Add tenant_id field to RequestContext
- Pass tenant_id instead of api_key to background tasks
- Extensions can check internal=True with tenant_id to bypass normal auth
* Fix exception propagation: include HTTPException in re-raise
After cleanup of redundant exception handlers, 404 errors were
returning 500 because HTTPException was caught by the generic
except Exception handler. Fixed by combining AuthenticationError
and HTTPException in the re-raise pattern.
Allows tuning of entity observation generation via environment variables.
## New Environment Variables
- `HINDSIGHT_API_OBSERVATION_MIN_FACTS` - Minimum facts required to
generate entity observations (default: 5)
- `HINDSIGHT_API_OBSERVATION_TOP_ENTITIES` - Maximum entities to process
per retain batch (default: 5)
## Changes
- Added threshold configuration to HindsightConfig
- Updated memory_engine.py to use config values
- Updated observation_regeneration.py to use config values
## Use Case
Lower thresholds generate more observations (better recall, higher cost).
Higher thresholds are more selective (lower cost, may miss patterns).
Example:
```bash
# Generate more observations
docker run -e HINDSIGHT_API_OBSERVATION_MIN_FACTS=3 \
-e HINDSIGHT_API_OBSERVATION_TOP_ENTITIES=10 ...
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Task handlers were swallowing exceptions, causing operations to be
marked as completed even when they failed. This prevented the retry
logic in execute_task() from working and led to accumulation of
pending operations that never completed.
Fixed handlers:
- _handle_batch_retain: remove try/except wrapper
- _handle_access_count_update: remove try/except wrapper
- _handle_regenerate_observations: remove outer try/except, keep
inner one for individual entity failures
