576016f5dc
* feat: add @vectorize-io/hindsight-embed daemon lifecycle package
Create a new top-level `hindsight-embed-npm/` package that owns the daemon
lifecycle for the Python `hindsight-embed` CLI: spawning via `uvx`, writing
the profile, waiting for `/health`, and shutting down. Nothing more.
Deliberately does not ship an HTTP client — `@vectorize-io/hindsight-client`
already covers retain / recall / reflect / createBank against the Hindsight
API, and the two packages compose: once `manager.start()` returns, consumers
talk to the daemon via `new HindsightClient({ baseUrl: manager.getBaseUrl() })`.
`HindsightEmbedManagerOptions.env` forwards an arbitrary `Record<string,
string>` to both the daemon process and the profile config via `--env K=V`,
and `extraProfileCreateArgs` / `extraDaemonStartArgs` escape hatches cover
any new CLI flag without waiting for a wrapper release.
Refactor `hindsight-integrations/openclaw` to consume both packages:
`HindsightEmbedManager` for daemon lifecycle in local mode, `HindsightClient`
for all HTTP memory operations. Drop the bespoke subprocess/HTTP client that
used to live in openclaw. The retain queue stays local to openclaw (it's a
client-side reliability workaround with a single consumer today — will move
to the client package or server-side when a second consumer needs it).
Wire the new package into the main release pipeline (versioned alongside
the other core packages, published from `v*` tags) and add a CI build job.
* docs: add Embedded Node.js SDK page for @vectorize-io/hindsight-embed
* refactor: rename hindsight-embed-npm to hindsight-all, restructure docs sidebar
The Node package previously named @vectorize-io/hindsight-embed was
semantically misnamed: hindsight-embed (Python) is a CLI tool, while what
this Node package actually provides is the Node equivalent of hindsight-all
— a programmatic lifecycle manager for a local Hindsight daemon. Rename to
match.
Package rename
- hindsight-embed-npm/ → hindsight-all-npm/ (git mv, history preserved)
- @vectorize-io/hindsight-embed → @vectorize-io/hindsight-all
- class HindsightEmbedManager → HindsightServer (matches Python hindsight-all)
- HindsightEmbedManagerOptions → HindsightServerOptions
- src/manager.ts → src/server.ts, src/manager.test.ts → src/server.test.ts
- openclaw (index.ts, backfill.ts, tests) and the claude-code Python port
updated to reference the new names
Docs restructure
- Split sdks/python.md: now client-only content. New sdks/hindsight-all.md
covers the programmatic hindsight-all Python package (HindsightServer and
HindsightEmbedded).
- Rename sdks/embed-npm.md → sdks/hindsight-all-npm.md with HindsightServer
examples.
- New "Installation" sidebar section, placed after Hosting, containing
Docker / Kubernetes / Bare Metal (anchor links into developer/installation)
plus Programmatic API (Python), Programmatic API (Node.js), and Daemon CLI.
- Add si-docker, si-kubernetes, si-nodedotjs, lu-hard-drive to the sidebar
ICON_MAP.
Docs dev-server fix
- docusaurus.config.ts: drop the flaky NODE_ENV sniff for including the
"Next" version. Use INCLUDE_CURRENT_VERSION exclusively. NODE_ENV was
unreliable across hot-reload paths and caused the Next version to
disappear intermittently when editing files.
- scripts/dev/start-docs.sh: export INCLUDE_CURRENT_VERSION=true so local
dev always shows Next; production builds leave it unset.
Lockfile cleanup
- package-lock.json and hindsight-integrations/openclaw/package-lock.json
had extraneous hindsight-embed-npm blocks left over from the rename.
Removed manually and verified with npm install.
* ci: fix openclaw jobs by pre-building workspace deps; regenerate docs-skill
The build-openclaw-integration and test-openclaw-integration jobs failed
with "Failed to resolve entry for package @vectorize-io/hindsight-all"
because openclaw depends on two monorepo workspaces via `file:` deps
(@vectorize-io/hindsight-client and @vectorize-io/hindsight-all) whose
`dist/` directories are gitignored and never built before openclaw's npm ci.
Both jobs now install the root workspace and build the two deps first,
mirroring the release-control-plane pattern.
Also regenerate skills/hindsight-docs/references/* via
./scripts/generate-docs-skill.sh:
- new skill pages for sdks/hindsight-all{.md,-npm.md}
- updated skill pages for sdks/embed.md and sdks/python.md to match
the new H1s and split content
- incidental refreshes to changelog/index.md, developer/models.md,
openapi.json, and uv.lock that verify-generated-files picked up
* ci: build openclaw before running tests so symlink test can realpath dist
247 lines
6.5 KiB
Markdown
247 lines
6.5 KiB
Markdown
---
|
|
sidebar_position: 5
|
|
---
|
|
|
|
# Daemon CLI (hindsight-embed)
|
|
|
|
Zero-configuration local memory system with automatic daemon management. Perfect for development, prototyping, and single-user applications.
|
|
|
|
## Overview
|
|
|
|
`hindsight-embed` is a zero-configuration SDK that wraps the Hindsight API and PostgreSQL database into a single auto-managed local daemon. It's designed for development, prototyping, and single-user applications where you want memory capabilities without infrastructure overhead.
|
|
|
|
**How it works:**
|
|
|
|
1. **First command triggers startup**: When you run any `hindsight-embed` command, it checks if a local daemon is running
|
|
2. **Auto-daemon management**: If no daemon exists, it automatically spawns `hindsight-api --daemon` in the background
|
|
3. **Embedded database**: The daemon uses `pg0` (embedded PostgreSQL) — no separate database installation required
|
|
4. **Command forwarding**: Your command is forwarded to the local daemon via HTTP (localhost:8888)
|
|
5. **Auto-shutdown**: After 5 minutes of inactivity (configurable), the daemon gracefully shuts down to free resources
|
|
|
|
**Key features:**
|
|
|
|
- **Zero setup** — One `configure` command and you're ready
|
|
- **Automatic lifecycle** — Daemon starts on-demand, stops when idle
|
|
- **Isolated storage** — Each bank gets its own embedded PostgreSQL database
|
|
- **Local-only** — Binds to `127.0.0.1:8888`, not accessible from network
|
|
- **Production-grade engine** — Uses the same memory engine as the full API service
|
|
|
|
Think of it as SQLite for long-term memory — all the power of Hindsight without managing servers.
|
|
|
|
## Installation
|
|
|
|
Install via `uvx` (recommended - always latest version):
|
|
|
|
```bash
|
|
# Run directly without installation
|
|
uvx hindsight-embed@latest configure
|
|
|
|
# Or use pipx for persistent installation
|
|
pipx install hindsight-embed
|
|
```
|
|
|
|
## Quick Start
|
|
|
|
### 1. Configure
|
|
|
|
```bash
|
|
# Interactive configuration
|
|
hindsight-embed configure
|
|
|
|
# Or non-interactive via environment variables
|
|
export HINDSIGHT_EMBED_LLM_PROVIDER=openai
|
|
export HINDSIGHT_EMBED_LLM_API_KEY=sk-xxxxxxxxxxxx
|
|
export HINDSIGHT_EMBED_LLM_MODEL=gpt-4o-mini
|
|
hindsight-embed configure
|
|
```
|
|
|
|
Configuration is saved to `~/.hindsight/embed`:
|
|
|
|
```bash
|
|
HINDSIGHT_EMBED_LLM_PROVIDER=openai
|
|
HINDSIGHT_EMBED_LLM_MODEL=gpt-4o-mini
|
|
HINDSIGHT_EMBED_BANK_ID=default
|
|
HINDSIGHT_EMBED_LLM_API_KEY=sk-xxxxxxxxxxxx
|
|
|
|
# Daemon settings (macOS: force CPU to avoid MPS/XPC issues)
|
|
HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU=1
|
|
HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU=1
|
|
```
|
|
|
|
### 2. Use Memory Operations
|
|
|
|
```bash
|
|
# Store a memory
|
|
hindsight-embed memory retain default "User prefers dark mode"
|
|
|
|
# Query memories
|
|
hindsight-embed memory recall default "user preferences"
|
|
|
|
# Reasoning with memory
|
|
hindsight-embed memory reflect default "What color scheme should I use?"
|
|
```
|
|
|
|
The daemon starts automatically on first use!
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Description | Default |
|
|
|----------|-------------|---------|
|
|
| `HINDSIGHT_EMBED_LLM_API_KEY` | **Required**. API key for LLM provider | - |
|
|
| `HINDSIGHT_EMBED_LLM_PROVIDER` | LLM provider: `openai`, `anthropic`, `gemini`, `groq`, `minimax`, `ollama` | `openai` |
|
|
| `HINDSIGHT_EMBED_LLM_MODEL` | Model name | `gpt-4o-mini` |
|
|
| `HINDSIGHT_EMBED_BANK_ID` | Default memory bank ID | `default` |
|
|
| `HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT` | Seconds before daemon auto-exits when idle (0 = never) | `300` |
|
|
|
|
**Provider Examples:**
|
|
|
|
```bash
|
|
# OpenAI
|
|
export HINDSIGHT_EMBED_LLM_PROVIDER=openai
|
|
export HINDSIGHT_EMBED_LLM_API_KEY=sk-xxxxxxxxxxxx
|
|
export HINDSIGHT_EMBED_LLM_MODEL=gpt-4o
|
|
|
|
# Groq (fast inference)
|
|
export HINDSIGHT_EMBED_LLM_PROVIDER=groq
|
|
export HINDSIGHT_EMBED_LLM_API_KEY=gsk_xxxxxxxxxxxx
|
|
export HINDSIGHT_EMBED_LLM_MODEL=llama-3.3-70b-versatile
|
|
|
|
# Anthropic
|
|
export HINDSIGHT_EMBED_LLM_PROVIDER=anthropic
|
|
export HINDSIGHT_EMBED_LLM_API_KEY=sk-ant-xxxxxxxxxxxx
|
|
export HINDSIGHT_EMBED_LLM_MODEL=claude-sonnet-4-20250514
|
|
```
|
|
|
|
## Daemon Management
|
|
|
|
### Idle Timeout
|
|
|
|
Customize how long the daemon stays alive when idle:
|
|
|
|
```bash
|
|
# Never timeout (daemon runs until manually stopped)
|
|
export HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT=0
|
|
|
|
# Shorter timeout: 1 minute
|
|
export HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT=60
|
|
|
|
# Longer timeout: 30 minutes
|
|
export HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT=1800
|
|
```
|
|
|
|
### Daemon Commands
|
|
|
|
```bash
|
|
# Check daemon status
|
|
hindsight-embed daemon status
|
|
|
|
# View daemon logs in real-time
|
|
hindsight-embed daemon logs -f
|
|
|
|
# Stop daemon manually
|
|
hindsight-embed daemon stop
|
|
```
|
|
|
|
## Commands
|
|
|
|
All memory operations follow the same interface as the CLI:
|
|
|
|
### Retain (Store Memory)
|
|
|
|
```bash
|
|
hindsight-embed memory retain <bank_id> "content"
|
|
|
|
# With context
|
|
hindsight-embed memory retain <bank_id> "content" --context "source information"
|
|
|
|
# Background processing
|
|
hindsight-embed memory retain <bank_id> "content" --async
|
|
```
|
|
|
|
### Recall (Search)
|
|
|
|
```bash
|
|
hindsight-embed memory recall <bank_id> "query"
|
|
|
|
# With budget control
|
|
hindsight-embed memory recall <bank_id> "query" --budget high
|
|
|
|
# Show trace
|
|
hindsight-embed memory recall <bank_id> "query" --trace
|
|
```
|
|
|
|
### Reflect (Generate Response)
|
|
|
|
```bash
|
|
hindsight-embed memory reflect <bank_id> "prompt"
|
|
|
|
# With additional context
|
|
hindsight-embed memory reflect <bank_id> "prompt" --context "additional info"
|
|
```
|
|
|
|
### Bank Management
|
|
|
|
```bash
|
|
# List all banks
|
|
hindsight-embed bank list
|
|
|
|
# View bank stats
|
|
hindsight-embed bank stats <bank_id>
|
|
|
|
# Set bank name
|
|
hindsight-embed bank name <bank_id> "My Assistant"
|
|
|
|
# Set bank mission
|
|
hindsight-embed bank mission <bank_id> "I am a helpful AI assistant"
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Daemon Won't Start
|
|
|
|
Check the daemon logs:
|
|
|
|
```bash
|
|
hindsight-embed daemon logs
|
|
# Or watch in real-time
|
|
hindsight-embed daemon logs -f
|
|
```
|
|
|
|
Common issues:
|
|
- **Missing API key**: Set `HINDSIGHT_EMBED_LLM_API_KEY`
|
|
- **Port conflict**: Another service using port 8888
|
|
- **Permissions**: Check `~/.hindsight/` directory permissions
|
|
|
|
### Daemon Exits Immediately
|
|
|
|
Check if you have the idle timeout set too low:
|
|
|
|
```bash
|
|
# Disable idle timeout for debugging
|
|
export HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT=0
|
|
hindsight-embed daemon status
|
|
```
|
|
|
|
### Reset Configuration
|
|
|
|
```bash
|
|
# Remove config file and reconfigure
|
|
rm ~/.hindsight/embed
|
|
hindsight-embed configure
|
|
```
|
|
|
|
## When to Use
|
|
|
|
**Perfect for:**
|
|
- Development and prototyping
|
|
- Single-user applications
|
|
- Local-first tools
|
|
- Quick experiments with Hindsight
|
|
|
|
**Not suitable for:**
|
|
- Production multi-user deployments
|
|
- Network-accessible services
|
|
- High-availability requirements
|
|
- Multi-tenant applications
|
|
|
|
For production deployments, use the [API Service](/developer/services) with external PostgreSQL instead.
|