The daemon_client unconditionally overwrites HINDSIGHT_API_DATABASE_URL
with pg0://hindsight-embed, preventing users from using an external
PostgreSQL instance.
This is a problem for VPS deployments running as root, where pg0's
embedded PostgreSQL fails with 'initdb: cannot be run as root'.
This change checks if the env var is already set before defaulting
to pg0, allowing users to point to an external PostgreSQL while
preserving the default embedded behavior.
Fixes#261
Pre-download cl100k_base tiktoken encoding (used by OpenAI models) during
Docker build to avoid runtime download delays.
Applied to both api-only and standalone stages.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: sanitize null bytes from text fields before PostgreSQL insertion
Fixes 'invalid byte sequence for encoding UTF8: 0x00' error during batch retain
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor: consolidate _sanitize_text into fact_extraction module
Address review feedback: reuse existing _sanitize_text from fact_extraction
instead of duplicating in fact_storage.
The consolidated function now handles both:
- Null bytes (\x00) for PostgreSQL compatibility
- Unicode surrogates (U+D800-U+DFFF) for UTF-8/LLM API compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: rename openclawd to openclaw
* fix: rename openclawd to openclaw
* Revise OpenClaw documentation and remove dev section
Updated the description of local memory for OpenClaw agents and removed the development section along with requirements and links.
- Fix XML tag: <hindsight-context> → <hindsight_memories>
- Remove embedPort config option (not implemented in code)
- Add default bankMission text to config docs
- Add 'Why Auto-Recall?' section explaining conceptual advantage over tools
- Add JSON format example showing metadata structure
- Add 'Local-First Design' section emphasizing privacy/cost/ownership benefits
- Update intro to highlight local-first and zero-cost aspects
These changes better align the docs with the blog post's narrative about why
auto-recall is better than tool-based memory and why local-first matters.
* fix: rename moltbot to openclawd
* fix
* fix
* fix: use single shared pg0 database for all banks + add default mission
This commit fixes a critical database isolation issue and adds the default
mission feature for the openclawd plugin.
## Changes:
**hindsight-embed:**
- Fixed daemon_client.py to use single shared database: pg0://hindsight-embed
- Previously, each bank_id would create a separate pg0 instance (wrong!)
- Now all banks share the same database with isolation via bank_id parameter
- Updated README to clarify database architecture
**openclawd plugin (v0.0.5):**
- Added default bank mission describing OpenClawd's multi-channel assistant role
- Added setBankMission() method to client
- Integrated mission setting during plugin initialization
- Added bankMission to plugin config schema with sensible default
- Updated docs to explain shared database architecture
## Why this matters:
Bank isolation should happen WITHIN the database (via separate tables/schemas),
not via separate database instances. Using HINDSIGHT_EMBED_BANK_ID to create
separate pg0 databases was architecturally wrong and caused confusion.
* ci: rename moltbot to openclawd in workflows and release script
- Updated build-moltbot-integration → build-openclawd-integration in test.yml
- Updated release-moltbot-integration → release-openclawd-integration in release.yml
- Updated all working directories from moltbot to openclawd
- Updated artifact names from moltbot-integration to openclawd-integration
- Added openclawd package.json to release.sh version bump script
- Add 3 retries with exponential backoff (10s -> 20s -> 40s)
- Set HF_HUB_DOWNLOAD_TIMEOUT=600 for longer timeout
- Fixes transient network failures during HuggingFace downloads
- Applied to both api-only and standalone stages
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* chore: remove dead code
* chore: remove extract_opinions from test and regenerate openapi
- Remove extract_opinions parameter from test_fact_extraction_analysis
- Regenerate OpenAPI spec after removing entity observations code
* chore: update generated files and apply formatting
- Regenerate Python and TypeScript client SDKs after main merge
- Apply ruff formatting to llm_wrapper.py
* fix: accept and filter deprecated 'opinion' fact type in recall
The dead code removal eliminated support for the 'opinion' fact type,
but existing clients may still pass it. Instead of rejecting it with
a ValueError, silently filter it out before validation to maintain
backward compatibility.
* feat(mcp): add Bearer token authentication support
Add HINDSIGHT_API_MCP_AUTH_TOKEN environment variable to enable
authentication for MCP endpoint. When set, all requests must include
a valid Authorization header (Bearer token or direct token).
If not set, MCP endpoint remains open for backwards compatibility
with local development environments.
* fix: propagate Bearer token from MCP middleware to tools for tenant auth
MCP tools were creating RequestContext() without api_key, causing
"Invalid API key" errors when tenant extension validates requests.
Now the Bearer token is extracted in middleware, stored in a context
variable, and passed through to all MCP tool RequestContext instances.
Previously, _authenticate_tenant only skipped extension auth for
internal requests when _current_schema was set to a non-public schema.
This caused async HTTP retain (document upload with async_processing=True)
to fail with AuthenticationError because the worker had no API key and
the schema was "public".
Remove the public-schema guard since internal tasks were already
authenticated at submission time. The worker sets _current_schema from
the task's _schema field for tenant schemas, and it defaults to "public"
for public schema tasks — both are valid.
The control plane proxy routes never sent an Authorization header to
the dataplane API. With the tenant extension active, all GUI requests
failed with "Invalid API key".
Add HINDSIGHT_CP_DATAPLANE_API_KEY env var support to hindsight-client.ts
and propagate auth headers to both SDK clients and all direct fetch routes.
- bank consolidate: add --wait flag to poll for completion status
- bank consolidate: add --poll-interval option (default 10s)
- document list: add --date filter (yesterday, today, YYYY-MM-DD, or all)
[skip ci]
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
When the backend graph API returns an error, the SDK sets response.data
to undefined. NextResponse.json(undefined) throws "Value is not JSON
serializable". Check for error/missing data before serializing.
Replace the OpenAI-compatible endpoint approach with the native
google-genai SDK for Vertex AI. This eliminates the custom token
refresher, TokenInjectingTransport, and async lifecycle complexity
while also removing the 8192 output token cap that the OpenAI
endpoint enforced.
Changes:
- vertexai provider now uses genai.Client(vertexai=True) instead of
AsyncOpenAI with token-injecting transport
- Routes through existing _call_gemini/_call_with_tools_gemini paths
- Strips google/ prefix from model names (native SDK uses bare names)
- Preserves service account key auth via credentials parameter
- Delete vertexai_token_refresher.py (no longer needed)
- Strip markdown code fences in consolidator JSON parsing
- Rewrite vertexai tests for native SDK integration
* feat: support vertex as llm provider
* fix
* fix: add uv index-strategy to resolve dependency conflicts with pytorch index
When using pytorch index for faster torch downloads in CI,
filelock dependency resolution was failing because pytorch index
only has older versions. Adding unsafe-best-match strategy allows
uv to search all configured indexes.
Also fix type checking warnings from ty.
* fix: add index-strategy to root pyproject.toml for workspace-level uv resolution
* chore: regenerate client SDKs after Vertex AI support
Tenant schemas were never migrated when new migrations were deployed.
Only the public schema was migrated at startup, and tenant schemas only
got migrations when first provisioned. This meant existing tenants
missed any new columns (e.g. task_payload, worker_id, claimed_at on
async_operations), causing the worker poller to crash silently.
Changes:
- Run migrations on all existing tenant schemas at startup when a
tenant_extension is configured. Each schema migration is wrapped in
try/except so one failure doesn't block others.
- Add try/except in WorkerPoller.recover_own_tasks() so a broken
schema doesn't prevent the polling loop from starting.
- Add try/except in WorkerPoller._claim_batch_for_schema() so a
broken schema doesn't prevent claiming tasks from other schemas.
The worker loaded the tenant extension for the poller (schema discovery)
but did not pass it to MemoryEngine. When execute_task set _current_schema
via the _schema field, _authenticate_tenant would immediately reset it to
"public" because self._tenant_extension was None, causing all worker writes
to land in the public schema instead of the tenant schema.
Move load_extension() before MemoryEngine creation and pass
tenant_extension to the constructor.
The mental_models.id column was changed from UUID to TEXT in migration
u6p7q8r9s0t1, but the exclude_ids filter in search_mental_models still
cast the parameter as ::uuid[]. This caused every search_mental_models
call during reflect to fail with "operator does not exist: text <> uuid",
forcing the reflect agent to waste all 5 iterations on retries and
producing degraded mental model content.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: include correct __version__ in python packages
* fix(embed): force CPU mode for local models in daemon to prevent XPC crashes
Adds HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU and HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU
environment variables to force CPU-only operation for local sentence-transformer models.
This prevents XPC_ERROR_CONNECTION_INVALID crashes on macOS when running in daemon mode.
The issue occurs because PyTorch's MPS (Metal Performance Shaders) backend has unstable
XPC connections in background processes, leading to C++ assertion failures that Python
exception handlers cannot catch.
Changes:
- config.py: Add ENV_*_FORCE_CPU constants and config dataclass fields
- embeddings.py: Add force_cpu parameter to LocalSTEmbeddings constructor
- cross_encoder.py: Add force_cpu parameter to LocalSTCrossEncoder constructor
- main.py: Set force CPU env vars in daemon mode, add fields to config constructor
The daemon mode automatically enables force CPU for both embeddings and reranker,
while normal mode allows hardware acceleration (GPU/MPS) as before.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: add defensive error handling to PyTorch device detection
Wraps all PyTorch device detection code (torch.cuda.is_available()
and torch.backends.mps.is_available()) in try-except blocks that
gracefully fall back to CPU if any errors occur.
This complements PR #218's force_cpu configuration by ensuring the
code works reliably in all environments without configuration:
- CI environments with CPU-only PyTorch builds
- Systems without proper GPU/MPS support
- Partial or misconfigured PyTorch installations
The defensive approach prevents startup failures while still taking
advantage of GPU/MPS acceleration when available and force_cpu is
not explicitly set.
Changes:
- embeddings.py: Added try-except in initialize() and _reinitialize_model_sync()
- cross_encoder.py: Added try-except in initialize() and _reinitialize_model_sync()
* refactor: use get_config() for embeddings and reranker force_cpu
Changes create_embeddings_from_env() and create_cross_encoder_from_env()
to read configuration via get_config() instead of directly accessing
os.environ. This ensures consistency across the codebase and properly
respects the force_cpu configuration set by daemon mode.
Changes:
- embeddings.py: Use config.embeddings_local_model and config.embeddings_local_force_cpu
- cross_encoder.py: Use config.reranker_local_model and config.reranker_local_force_cpu
- Both: Use get_config() for provider, tei_url, and other config fields
- Note: Some fields not in config (like max_concurrent for local reranker) still read from os.environ
This fixes the issue where force_cpu was read inconsistently from environment
variables instead of using the centralized config system.
* test: clear config cache in test_create_from_env
Fixes test failure caused by cached config not picking up
environment variable changes in test. The test now calls
clear_config_cache() before and after patching os.environ
to ensure the factory function reads the test's env vars.
* refactor: add reranker_local_max_concurrent to config system
Adds reranker_local_max_concurrent to HindsightConfig dataclass
and removes the workaround in create_cross_encoder_from_env() that
was reading it directly from os.environ.
Changes:
- config.py: Add reranker_local_max_concurrent field to dataclass and from_env()
- main.py: Add reranker_local_max_concurrent to manual config constructor
- cross_encoder.py: Use config.reranker_local_max_concurrent instead of os.environ
This completes the refactoring to use the centralized config system
for all reranker configuration.
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
