* feat: add @vectorize-io/hindsight-embed daemon lifecycle package
Create a new top-level `hindsight-embed-npm/` package that owns the daemon
lifecycle for the Python `hindsight-embed` CLI: spawning via `uvx`, writing
the profile, waiting for `/health`, and shutting down. Nothing more.
Deliberately does not ship an HTTP client — `@vectorize-io/hindsight-client`
already covers retain / recall / reflect / createBank against the Hindsight
API, and the two packages compose: once `manager.start()` returns, consumers
talk to the daemon via `new HindsightClient({ baseUrl: manager.getBaseUrl() })`.
`HindsightEmbedManagerOptions.env` forwards an arbitrary `Record<string,
string>` to both the daemon process and the profile config via `--env K=V`,
and `extraProfileCreateArgs` / `extraDaemonStartArgs` escape hatches cover
any new CLI flag without waiting for a wrapper release.
Refactor `hindsight-integrations/openclaw` to consume both packages:
`HindsightEmbedManager` for daemon lifecycle in local mode, `HindsightClient`
for all HTTP memory operations. Drop the bespoke subprocess/HTTP client that
used to live in openclaw. The retain queue stays local to openclaw (it's a
client-side reliability workaround with a single consumer today — will move
to the client package or server-side when a second consumer needs it).
Wire the new package into the main release pipeline (versioned alongside
the other core packages, published from `v*` tags) and add a CI build job.
* docs: add Embedded Node.js SDK page for @vectorize-io/hindsight-embed
* refactor: rename hindsight-embed-npm to hindsight-all, restructure docs sidebar
The Node package previously named @vectorize-io/hindsight-embed was
semantically misnamed: hindsight-embed (Python) is a CLI tool, while what
this Node package actually provides is the Node equivalent of hindsight-all
— a programmatic lifecycle manager for a local Hindsight daemon. Rename to
match.
Package rename
- hindsight-embed-npm/ → hindsight-all-npm/ (git mv, history preserved)
- @vectorize-io/hindsight-embed → @vectorize-io/hindsight-all
- class HindsightEmbedManager → HindsightServer (matches Python hindsight-all)
- HindsightEmbedManagerOptions → HindsightServerOptions
- src/manager.ts → src/server.ts, src/manager.test.ts → src/server.test.ts
- openclaw (index.ts, backfill.ts, tests) and the claude-code Python port
updated to reference the new names
Docs restructure
- Split sdks/python.md: now client-only content. New sdks/hindsight-all.md
covers the programmatic hindsight-all Python package (HindsightServer and
HindsightEmbedded).
- Rename sdks/embed-npm.md → sdks/hindsight-all-npm.md with HindsightServer
examples.
- New "Installation" sidebar section, placed after Hosting, containing
Docker / Kubernetes / Bare Metal (anchor links into developer/installation)
plus Programmatic API (Python), Programmatic API (Node.js), and Daemon CLI.
- Add si-docker, si-kubernetes, si-nodedotjs, lu-hard-drive to the sidebar
ICON_MAP.
Docs dev-server fix
- docusaurus.config.ts: drop the flaky NODE_ENV sniff for including the
"Next" version. Use INCLUDE_CURRENT_VERSION exclusively. NODE_ENV was
unreliable across hot-reload paths and caused the Next version to
disappear intermittently when editing files.
- scripts/dev/start-docs.sh: export INCLUDE_CURRENT_VERSION=true so local
dev always shows Next; production builds leave it unset.
Lockfile cleanup
- package-lock.json and hindsight-integrations/openclaw/package-lock.json
had extraneous hindsight-embed-npm blocks left over from the rename.
Removed manually and verified with npm install.
* ci: fix openclaw jobs by pre-building workspace deps; regenerate docs-skill
The build-openclaw-integration and test-openclaw-integration jobs failed
with "Failed to resolve entry for package @vectorize-io/hindsight-all"
because openclaw depends on two monorepo workspaces via `file:` deps
(@vectorize-io/hindsight-client and @vectorize-io/hindsight-all) whose
`dist/` directories are gitignored and never built before openclaw's npm ci.
Both jobs now install the root workspace and build the two deps first,
mirroring the release-control-plane pattern.
Also regenerate skills/hindsight-docs/references/* via
./scripts/generate-docs-skill.sh:
- new skill pages for sdks/hindsight-all{.md,-npm.md}
- updated skill pages for sdks/embed.md and sdks/python.md to match
the new H1s and split content
- incidental refreshes to changelog/index.md, developer/models.md,
openapi.json, and uv.lock that verify-generated-files picked up
* ci: build openclaw before running tests so symlink test can realpath dist
146 lines
5.4 KiB
Markdown
146 lines
5.4 KiB
Markdown
---
|
|
sidebar_position: 2
|
|
---
|
|
|
|
# Programmatic API (Python)
|
|
|
|
The `hindsight-all` Python package lets your code spawn and manage a local Hindsight daemon without deploying any server infrastructure. It bundles the Hindsight API server, embedded PostgreSQL, and the Python client into one install — `pip install hindsight-all` and you can start a fully-functional Hindsight instance from a few lines of Python.
|
|
|
|
The daemon runs as a **separate OS process** on `127.0.0.1` (not in your Python process memory). Your code talks to it over HTTP via the bundled `HindsightClient`.
|
|
|
|
If you already have a Hindsight server running and just need a client, use [Python Client (hindsight-client)](./python.md) instead.
|
|
|
|
## How it works
|
|
|
|
`hindsight-all` exposes two primary APIs:
|
|
|
|
- **`HindsightServer`** — explicit lifecycle. Use it as a context manager when you want deterministic startup/shutdown (e.g. in tests).
|
|
- **`HindsightEmbedded`** — auto-managed. Starts a daemon on first use, reuses it across calls, shuts it down after an idle timeout. Easiest for application code that doesn't want to think about lifecycle.
|
|
|
|
Both end up talking to the same underlying daemon via the same `HindsightClient` HTTP interface — the difference is only how the server process is managed.
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
pip install hindsight-all
|
|
```
|
|
|
|
The `hindsight-all` wheel bundles `hindsight-api-slim`, `hindsight-client`, and `hindsight-embed` as dependencies, so one `pip install` gets you everything.
|
|
|
|
## `HindsightServer` — explicit lifecycle
|
|
|
|
Use `HindsightServer` as a context manager when you want the server to start immediately, run for the duration of a block, and shut down cleanly afterwards. Ideal for tests and short-lived scripts.
|
|
|
|
```python
|
|
import os
|
|
from hindsight import HindsightServer, HindsightClient
|
|
|
|
with HindsightServer(
|
|
llm_provider="openai",
|
|
llm_model="gpt-4o-mini",
|
|
llm_api_key=os.environ["OPENAI_API_KEY"],
|
|
) as server:
|
|
client = HindsightClient(base_url=server.url)
|
|
|
|
client.retain(bank_id="my-bank", content="Alice works at Google")
|
|
results = client.recall(bank_id="my-bank", query="What does Alice do?")
|
|
for r in results:
|
|
print(r.text)
|
|
|
|
answer = client.reflect(bank_id="my-bank", query="Tell me about Alice")
|
|
print(answer.text)
|
|
# Server is stopped here
|
|
```
|
|
|
|
## `HindsightEmbedded` — auto-managed
|
|
|
|
`HindsightEmbedded` is the simplest way to use Hindsight in Python. It automatically manages a background daemon for you — starts on first use, stays alive across calls, shuts down after an idle timeout.
|
|
|
|
```python
|
|
from hindsight import HindsightEmbedded
|
|
import os
|
|
|
|
# Server starts automatically on first call
|
|
client = HindsightEmbedded(
|
|
profile="myapp", # Profile for data isolation
|
|
llm_provider="openai",
|
|
llm_model="gpt-4o-mini",
|
|
llm_api_key=os.environ["OPENAI_API_KEY"],
|
|
)
|
|
|
|
# Use immediately - no manual server management needed
|
|
client.retain(bank_id="my-bank", content="Alice works at Google")
|
|
results = client.recall(bank_id="my-bank", query="What does Alice do?")
|
|
|
|
# Server continues running (auto-stops after idle timeout)
|
|
# Or explicitly stop it:
|
|
client.close(stop_daemon=True)
|
|
```
|
|
|
|
### What's a Profile?
|
|
|
|
A profile is an isolated Hindsight environment. Each profile gets its own embedded PostgreSQL database (stored in `~/.pg0/instances/hindsight-embed-{profile}/`) and its own API server. Use different profiles to separate environments (dev/prod), applications, or users.
|
|
|
|
### When to use which
|
|
|
|
| Use case | Pick |
|
|
|---|---|
|
|
| Tests, short-lived scripts, deterministic startup/shutdown | `HindsightServer` (context manager) |
|
|
| Long-running application, auto-start on first use, don't want to manage lifecycle | `HindsightEmbedded` |
|
|
| Existing Hindsight server running elsewhere | [`hindsight-client`](./python.md) directly |
|
|
|
|
## API namespaces
|
|
|
|
Both `HindsightEmbedded` and `HindsightClient` expose organized API namespaces for bank management, mental models, directives, and memories:
|
|
|
|
```python
|
|
from hindsight import HindsightEmbedded
|
|
import os
|
|
|
|
embedded = HindsightEmbedded(
|
|
profile="myapp",
|
|
llm_provider="openai",
|
|
llm_api_key=os.environ["OPENAI_API_KEY"],
|
|
)
|
|
|
|
# Core operations
|
|
embedded.retain(bank_id="test", content="Hello")
|
|
results = embedded.recall(bank_id="test", query="Hello")
|
|
|
|
# Bank management
|
|
embedded.banks.create(bank_id="test", name="Test Bank", mission="Help users")
|
|
embedded.banks.set_mission(bank_id="test", mission="Updated mission")
|
|
embedded.banks.delete(bank_id="test")
|
|
|
|
# Mental models
|
|
embedded.mental_models.create(
|
|
bank_id="test",
|
|
name="User Preferences",
|
|
content="User prefers dark mode"
|
|
)
|
|
models = embedded.mental_models.list(bank_id="test")
|
|
|
|
# Directives
|
|
embedded.directives.create(
|
|
bank_id="test",
|
|
name="Response Style",
|
|
content="Be concise and friendly"
|
|
)
|
|
directives = embedded.directives.list(bank_id="test")
|
|
|
|
# List memories
|
|
memories = embedded.memories.list(bank_id="test", type="world", limit=50)
|
|
```
|
|
|
|
API namespaces ensure the daemon is running before each call, so daemon crashes are handled gracefully:
|
|
|
|
```python
|
|
# ✅ GOOD - Uses API namespace (daemon restarts handled)
|
|
embedded.banks.create(bank_id="test", name="Test")
|
|
|
|
# ❌ BAD - Direct client access (daemon crashes NOT handled)
|
|
client = embedded.client
|
|
client.create_bank(bank_id="test", name="Test") # Fails if daemon crashed
|
|
```
|
|
|
|
For the full reference of retain/recall/reflect methods and their options (which work the same regardless of how you obtain the client) see the [Python Client page](./python.md).
|