fleet-memory/skills/hindsight-docs/references/sdks/hindsight-all-npm.md
Nicolò Boschi 576016f5dc
feat: add @vectorize-io/hindsight-all daemon lifecycle package (#949)
* feat: add @vectorize-io/hindsight-embed daemon lifecycle package

Create a new top-level `hindsight-embed-npm/` package that owns the daemon
lifecycle for the Python `hindsight-embed` CLI: spawning via `uvx`, writing
the profile, waiting for `/health`, and shutting down. Nothing more.

Deliberately does not ship an HTTP client — `@vectorize-io/hindsight-client`
already covers retain / recall / reflect / createBank against the Hindsight
API, and the two packages compose: once `manager.start()` returns, consumers
talk to the daemon via `new HindsightClient({ baseUrl: manager.getBaseUrl() })`.

`HindsightEmbedManagerOptions.env` forwards an arbitrary `Record<string,
string>` to both the daemon process and the profile config via `--env K=V`,
and `extraProfileCreateArgs` / `extraDaemonStartArgs` escape hatches cover
any new CLI flag without waiting for a wrapper release.

Refactor `hindsight-integrations/openclaw` to consume both packages:
`HindsightEmbedManager` for daemon lifecycle in local mode, `HindsightClient`
for all HTTP memory operations. Drop the bespoke subprocess/HTTP client that
used to live in openclaw. The retain queue stays local to openclaw (it's a
client-side reliability workaround with a single consumer today — will move
to the client package or server-side when a second consumer needs it).

Wire the new package into the main release pipeline (versioned alongside
the other core packages, published from `v*` tags) and add a CI build job.

* docs: add Embedded Node.js SDK page for @vectorize-io/hindsight-embed

* refactor: rename hindsight-embed-npm to hindsight-all, restructure docs sidebar

The Node package previously named @vectorize-io/hindsight-embed was
semantically misnamed: hindsight-embed (Python) is a CLI tool, while what
this Node package actually provides is the Node equivalent of hindsight-all
— a programmatic lifecycle manager for a local Hindsight daemon. Rename to
match.

Package rename
  - hindsight-embed-npm/ → hindsight-all-npm/ (git mv, history preserved)
  - @vectorize-io/hindsight-embed → @vectorize-io/hindsight-all
  - class HindsightEmbedManager → HindsightServer (matches Python hindsight-all)
  - HindsightEmbedManagerOptions → HindsightServerOptions
  - src/manager.ts → src/server.ts, src/manager.test.ts → src/server.test.ts
  - openclaw (index.ts, backfill.ts, tests) and the claude-code Python port
    updated to reference the new names

Docs restructure
  - Split sdks/python.md: now client-only content. New sdks/hindsight-all.md
    covers the programmatic hindsight-all Python package (HindsightServer and
    HindsightEmbedded).
  - Rename sdks/embed-npm.md → sdks/hindsight-all-npm.md with HindsightServer
    examples.
  - New "Installation" sidebar section, placed after Hosting, containing
    Docker / Kubernetes / Bare Metal (anchor links into developer/installation)
    plus Programmatic API (Python), Programmatic API (Node.js), and Daemon CLI.
  - Add si-docker, si-kubernetes, si-nodedotjs, lu-hard-drive to the sidebar
    ICON_MAP.

Docs dev-server fix
  - docusaurus.config.ts: drop the flaky NODE_ENV sniff for including the
    "Next" version. Use INCLUDE_CURRENT_VERSION exclusively. NODE_ENV was
    unreliable across hot-reload paths and caused the Next version to
    disappear intermittently when editing files.
  - scripts/dev/start-docs.sh: export INCLUDE_CURRENT_VERSION=true so local
    dev always shows Next; production builds leave it unset.

Lockfile cleanup
  - package-lock.json and hindsight-integrations/openclaw/package-lock.json
    had extraneous hindsight-embed-npm blocks left over from the rename.
    Removed manually and verified with npm install.

* ci: fix openclaw jobs by pre-building workspace deps; regenerate docs-skill

The build-openclaw-integration and test-openclaw-integration jobs failed
with "Failed to resolve entry for package @vectorize-io/hindsight-all"
because openclaw depends on two monorepo workspaces via `file:` deps
(@vectorize-io/hindsight-client and @vectorize-io/hindsight-all) whose
`dist/` directories are gitignored and never built before openclaw's npm ci.
Both jobs now install the root workspace and build the two deps first,
mirroring the release-control-plane pattern.

Also regenerate skills/hindsight-docs/references/* via
./scripts/generate-docs-skill.sh:
  - new skill pages for sdks/hindsight-all{.md,-npm.md}
  - updated skill pages for sdks/embed.md and sdks/python.md to match
    the new H1s and split content
  - incidental refreshes to changelog/index.md, developer/models.md,
    openapi.json, and uv.lock that verify-generated-files picked up

* ci: build openclaw before running tests so symlink test can realpath dist
2026-04-10 15:51:44 +02:00

4.9 KiB

sidebar_position
6

Programmatic API (Node.js)

The @vectorize-io/hindsight-all npm package is the Node.js equivalent of the Python hindsight-all package. It lets your Node code spawn and supervise a local Hindsight daemon without deploying any server infrastructure — pair it with @vectorize-io/hindsight-client for memory operations.

The daemon runs as a separate OS process on 127.0.0.1 (not in your Node process). Your code talks to it over HTTP via HindsightClient.

This package does not ship an HTTP client — it only owns the server process. Once the daemon is running, talk to it with @vectorize-io/hindsight-client against server.getBaseUrl(). The two packages compose: one owns the process, the other owns the API surface.

How it works

  1. server.start() resolves the underlying hindsight-embed command (via uvx from PyPI, or uv run --directory <path> for a local checkout).
  2. Runs profile create <name> --merge --port <port> [--env KEY=VALUE ...] with every entry from options.env forwarded as --env.
  3. Runs daemon --profile <name> start.
  4. Polls http://host:port/health until it returns 200 or the readyTimeoutMs budget is exhausted.
  5. server.stop() runs daemon --profile <name> stop.

The server is intentionally transparent: new daemon env vars or CLI flags never require a wrapper release — pass them through env, extraProfileCreateArgs, or extraDaemonStartArgs.

Requirements

  • Node.js ≥ 22 — uses global fetch and AbortSignal.timeout.
  • uv / uvx on PATH — used to download and run the Hindsight daemon. Install via docs.astral.sh/uv.

Install

npm install @vectorize-io/hindsight-all @vectorize-io/hindsight-client

Example

import { HindsightServer, consoleLogger } from '@vectorize-io/hindsight-all';
import { HindsightClient } from '@vectorize-io/hindsight-client';

const server = new HindsightServer({
  profile: 'my-app',
  port: 9077,
  env: {
    HINDSIGHT_API_LLM_PROVIDER: 'anthropic',
    HINDSIGHT_API_LLM_API_KEY: process.env.ANTHROPIC_API_KEY,
    HINDSIGHT_API_LLM_MODEL: 'claude-sonnet-4-20250514',
    HINDSIGHT_EMBED_DAEMON_IDLE_TIMEOUT: '0',
  },
  logger: consoleLogger,
});

await server.start();

const client = new HindsightClient({ baseUrl: server.getBaseUrl() });
await client.retain('user-123', 'User prefers dark mode.');
const recall = await client.recall('user-123', 'what are the user preferences?');

await server.stop();

For a remote Hindsight API, skip the server entirely and point HindsightClient directly at the remote URL.

HindsightServerOptions

Option Type Default Description
profile string "default" Profile name passed to --profile on every sub-command.
port number 8888 TCP port the daemon listens on.
host string "127.0.0.1" Hostname the daemon binds to (used for health checks).
embedVersion string "latest" Version of the underlying hindsight-embed package to run via uvx.
embedPackagePath string Local checkout path — takes precedence over embedVersion. Uses uv run --directory instead of uvx.
env Record<string, string | undefined> {} Environment variables passed to the daemon process and written into the profile config via --env KEY=VALUE. The preferred way to surface any HINDSIGHT_API_* / HINDSIGHT_EMBED_* setting.
extraProfileCreateArgs string[] [] Extra args appended verbatim to profile create.
extraDaemonStartArgs string[] [] Extra args appended verbatim to daemon start.
platformCpuWorkaround boolean true on macOS Auto-set HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU=1 and HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU=1 to avoid Metal/MPS crashes. Caller-supplied env values win over the auto-applied ones.
readyTimeoutMs number 30000 Max time to wait for /health to return 200.
readyPollIntervalMs number 1000 Polling interval while waiting for /health.
logger Logger silent Pluggable logger (debug/info/warn/error). consoleLogger and silentLogger helpers are exported.

Server methods

Method Returns Description
start() Promise<void> Configure profile, spawn the daemon, wait for /health. Idempotent — safe to re-run.
stop() Promise<void> Stop the daemon. Never throws; logs and resolves even on failure.
checkHealth() Promise<boolean> One-shot /health probe with a 2 s timeout.
getBaseUrl() string http://host:port — pass this straight to HindsightClient.
getProfile() string The profile name this server operates on.

For memory operations (retain, recall, reflect, bank management) use @vectorize-io/hindsight-client.