Background tasks (async retain, consolidation, reflections) fail in
multi-tenant deployments because the worker executes tasks without
setting the tenant schema context. This causes two failures:
1. The cancellation check in execute_task queries public.async_operations
instead of the tenant's schema, finds no row, and skips the task as
"cancelled" — even though it wasn't.
2. Even if that were fixed, _authenticate_tenant would throw
AuthenticationError because background tasks have no API key.
Changes:
- Poller passes task.schema into task_dict so execute_task can set it
- execute_task sets _current_schema before the cancellation check
- Task handlers use RequestContext(internal=True) to signal background ops
- _authenticate_tenant skips extension auth for internal requests when
schema is already set
- BrokerTaskBackend uses schema_getter for dynamic schema resolution
when submitting tasks and waiting for results
- Pass tenant_extension to WorkerPoller in create_app
The graph endpoint's table_rows response was missing three fields that
the control plane UI expects:
- tags: memory unit tags (shown in Tags column)
- created_at: creation timestamp (shown in Created column for mental models)
- proof_count: source memory count (shown in Sources column for mental models)
All three columns exist on the memory_units table but were not being
selected or included in the response.
Gemini requires the 'name' field in tool/function response messages,
while OpenAI infers it from tool_call_id. Without it, Gemini returns:
'function_response.name: Name cannot be empty'
Added 'name' field to both tool result messages in the reflect agent.
* chore: run benchmarks with reflect mode
* chore: run benchmarks with reflect mode
* fixes
* new mm
* bunch of fixes
* initial commit
* fixes
* fixes
* fixes
* fix: sometimes memories gets extracted in the wrong language
Remove device_map from model_kwargs as it conflicts with CrossEncoder's
internal .to(device) call. The low_cpu_mem_usage=False setting alone is
sufficient to prevent lazy loading (meta tensors).
* fix: prevent meta tensor issues when accelerate is installed without GPU
When accelerate is installed but no GPU is available, transformers can
incorrectly use lazy loading (meta tensors) which fails when
sentence-transformers tries to move the model to a device.
The fix checks hardware and installed packages to determine the right
loading strategy:
- GPU available: device=None, device_map=None (auto-detect GPU)
- No GPU + accelerate: device='cpu', device_map='cpu' (force CPU loading)
- No GPU + no accelerate: device='cpu', device_map=None (normal CPU)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
* fix: add filelock for model initialization in parallel tests
When pytest-xdist runs multiple workers in parallel, they all try to
load models from the HuggingFace cache simultaneously, causing race
conditions and intermittent meta tensor errors.
Added filelock around embeddings and cross_encoder initialization in
conftest.py, similar to how pg0 database setup is serialized. Models
are now pre-initialized in the fixture before being passed to tests.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
* fix: add MPS support for macOS Apple Silicon
Extend GPU detection to include Apple MPS backend in addition to CUDA.
This ensures macOS users with Apple Silicon use MPS acceleration
instead of being incorrectly routed to the CPU fallback path.
* Add structured JSON logging support
Add HINDSIGHT_API_LOG_FORMAT environment variable to configure log output
format. Options are "text" (default, human-readable) and "json" (structured).
JSON format outputs logs with a "severity" field that cloud logging systems
can parse for proper log level categorization. Also writes to stdout instead
of stderr so log levels are correctly interpreted.
* Rename GCPJsonFormatter to JsonFormatter
* Fix: Load extensions in server.py for multi-worker deployments
When running with multiple workers (--workers 2), uvicorn uses
`hindsight_api.server:app` import string instead of passing an app
object. The server.py module was not loading tenant/operation validator
extensions, causing authentication bypass in production.
This fix:
- Adds extension loading to server.py matching main.py behavior
- Sets extension context on tenant extension for schema provisioning
- Adds comprehensive unit tests for server.py extension loading
The tests specifically verify:
- TENANT extension is loaded when HINDSIGHT_API_TENANT_EXTENSION is set
- OPERATION_VALIDATOR is loaded when configured
- Extensions are passed to MemoryEngine constructor
- Extension context is set on tenant extension
- Server works correctly without extensions configured
* Add unit tests for main.py extension loading (single-worker path)
* fix: misc perf improvements
* more tests
* fix test
* fix: update test files for new extract_facts_from_text signature
- Replace test_fact_extraction_token_analysis with test_fact_extraction_basic_analysis
using inline sample content instead of external file
- Update test_fact_extraction_output_ratio.py to unpack 3 return values
(facts, chunks, usage) instead of 2
* fix: make temporal tests more flexible for LLM variation
- test_temporal_absolute_conversion: check occurred_start field instead of
requiring specific text in facts
- test_date_field_calculation_yesterday: make assertions conditional on
having temporal data, add more content for better extraction
- test_temporal_ordering: reduce minimum required facts from 3 to 2
Call ensure_embedding_dimension after running migrations for tenant
schemas. This ensures the embedding column dimension matches the
model's dimension, which may differ from the default 384 dimensions
used in the initial migration.
Without this fix, using embedding providers with different dimensions
(e.g., Cohere's embed-english-v3.0 with 1024 dims) would fail with
"expected 384 dimensions, not 1024" errors on tenant schemas.
The /v1/default/banks/{bank_id}/stats endpoint was missing the
request_context parameter and tenant authentication call, causing
it to query the public schema instead of the tenant's schema.
This resulted in stats always returning zeros for multi-tenant
deployments since the data lives in tenant-specific schemas.
Added request_context dependency and _authenticate_tenant() call
to properly set the tenant schema before querying stats.
* expose the delete API
* add deleteBank
* Add a button and confirmation dialog to delete a memory bank
* commit lint changes
* add CI test for delete bank
* revert alembic lint changes due to version differences
* revert alembic lint changes
* fix the delete bank test
* account for ruff lint third party alembic
* feat(mcp): add async_processing parameter to retain tool
Add async_processing parameter (default: True) to the MCP retain tool
to allow non-blocking memory storage. When True, memories are queued
for background processing and the tool returns immediately. When False,
the tool waits for completion before returning.
This matches the async behavior available in the HTTP API.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(mcp): add list_memories and reflect tools
Add two missing MCP tools to achieve feature parity with HTTP API:
- list_memories: browse memories with pagination and full-text search
(equivalent to GET /memories/list)
- reflect: LLM-based reasoning over memories with disposition awareness
(equivalent to POST /reflect)
Both tools follow the existing pattern with JSON string responses
and proper error handling.
* docs: improve CLAUDE.md with detailed architecture info
- Add memory types explanation (world, experience, opinion, observation)
- Document retain/ and search/ submodule structure
- Add commands for single test run, ruff format, ty type checking
- Note MCP server implementation in API layer
- Add optional environment variables section
- Clarify conventions (no Python files at root, npm workspaces)
* chore: add .mcp.json and .osgrep to gitignore
These are user-specific development tool configs that should not be committed.
* changes
* refactor(mcp): remove list_memories tool
The list_memories endpoint is for debugging/exploration, not agent use.
Agents should use recall for semantic search instead.
Feedback from maintainer: "this tool is misleading for the agent,
it should use recall, the list method is mostly for debugging and
exploration, not for real usage"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor(mcp): remove list_banks and create_bank tools
These admin/orchestration tools are not needed for typical agent usage.
Agents work with a single configured bank via X-Bank-Id header.
MCP now exposes only core memory operations:
- retain: store memories
- recall: semantic search
- reflect: LLM reasoning over memories
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Anton Evseev <a.evseev@xsolla.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: Record LLM token metrics via Prometheus
Wire up the existing token metrics infrastructure to actually record
token usage from LLM calls. The MetricsCollector already had
record_tokens() method and Prometheus counters (hindsight.tokens.input,
hindsight.tokens.output), but they were never being populated.
Changes:
- Import get_metrics_collector in llm_wrapper.py
- Call record_tokens() after successful LLM calls for:
- OpenAI/Groq (using response.usage.prompt_tokens, completion_tokens)
- Anthropic (using response.usage.input_tokens, output_tokens)
- Gemini (using response.usage_metadata.prompt_token_count, candidates_token_count)
- Add test file to verify token metrics are recorded
Note: Ollama's native API doesn't return token usage, so metrics
are not recorded for that provider.
The token metrics will now be available via /metrics endpoint:
- hindsight_tokens_input_total
- hindsight_tokens_output_total
* feat: add per-request token usage tracking to retain and reflect endpoints
- Add TokenUsage model with input_tokens, output_tokens, total_tokens
- Return usage metrics in retain response (sync operations only)
- Return usage metrics in reflect response
- Update Python, TypeScript, and Rust clients
- Add API documentation for usage fields
- Add changelog entry
