* feat: add @vectorize-io/hindsight-embed daemon lifecycle package
Create a new top-level `hindsight-embed-npm/` package that owns the daemon
lifecycle for the Python `hindsight-embed` CLI: spawning via `uvx`, writing
the profile, waiting for `/health`, and shutting down. Nothing more.
Deliberately does not ship an HTTP client — `@vectorize-io/hindsight-client`
already covers retain / recall / reflect / createBank against the Hindsight
API, and the two packages compose: once `manager.start()` returns, consumers
talk to the daemon via `new HindsightClient({ baseUrl: manager.getBaseUrl() })`.
`HindsightEmbedManagerOptions.env` forwards an arbitrary `Record<string,
string>` to both the daemon process and the profile config via `--env K=V`,
and `extraProfileCreateArgs` / `extraDaemonStartArgs` escape hatches cover
any new CLI flag without waiting for a wrapper release.
Refactor `hindsight-integrations/openclaw` to consume both packages:
`HindsightEmbedManager` for daemon lifecycle in local mode, `HindsightClient`
for all HTTP memory operations. Drop the bespoke subprocess/HTTP client that
used to live in openclaw. The retain queue stays local to openclaw (it's a
client-side reliability workaround with a single consumer today — will move
to the client package or server-side when a second consumer needs it).
Wire the new package into the main release pipeline (versioned alongside
the other core packages, published from `v*` tags) and add a CI build job.
* docs: add Embedded Node.js SDK page for @vectorize-io/hindsight-embed
* refactor: rename hindsight-embed-npm to hindsight-all, restructure docs sidebar
The Node package previously named @vectorize-io/hindsight-embed was
semantically misnamed: hindsight-embed (Python) is a CLI tool, while what
this Node package actually provides is the Node equivalent of hindsight-all
— a programmatic lifecycle manager for a local Hindsight daemon. Rename to
match.
Package rename
- hindsight-embed-npm/ → hindsight-all-npm/ (git mv, history preserved)
- @vectorize-io/hindsight-embed → @vectorize-io/hindsight-all
- class HindsightEmbedManager → HindsightServer (matches Python hindsight-all)
- HindsightEmbedManagerOptions → HindsightServerOptions
- src/manager.ts → src/server.ts, src/manager.test.ts → src/server.test.ts
- openclaw (index.ts, backfill.ts, tests) and the claude-code Python port
updated to reference the new names
Docs restructure
- Split sdks/python.md: now client-only content. New sdks/hindsight-all.md
covers the programmatic hindsight-all Python package (HindsightServer and
HindsightEmbedded).
- Rename sdks/embed-npm.md → sdks/hindsight-all-npm.md with HindsightServer
examples.
- New "Installation" sidebar section, placed after Hosting, containing
Docker / Kubernetes / Bare Metal (anchor links into developer/installation)
plus Programmatic API (Python), Programmatic API (Node.js), and Daemon CLI.
- Add si-docker, si-kubernetes, si-nodedotjs, lu-hard-drive to the sidebar
ICON_MAP.
Docs dev-server fix
- docusaurus.config.ts: drop the flaky NODE_ENV sniff for including the
"Next" version. Use INCLUDE_CURRENT_VERSION exclusively. NODE_ENV was
unreliable across hot-reload paths and caused the Next version to
disappear intermittently when editing files.
- scripts/dev/start-docs.sh: export INCLUDE_CURRENT_VERSION=true so local
dev always shows Next; production builds leave it unset.
Lockfile cleanup
- package-lock.json and hindsight-integrations/openclaw/package-lock.json
had extraneous hindsight-embed-npm blocks left over from the rename.
Removed manually and verified with npm install.
* ci: fix openclaw jobs by pre-building workspace deps; regenerate docs-skill
The build-openclaw-integration and test-openclaw-integration jobs failed
with "Failed to resolve entry for package @vectorize-io/hindsight-all"
because openclaw depends on two monorepo workspaces via `file:` deps
(@vectorize-io/hindsight-client and @vectorize-io/hindsight-all) whose
`dist/` directories are gitignored and never built before openclaw's npm ci.
Both jobs now install the root workspace and build the two deps first,
mirroring the release-control-plane pattern.
Also regenerate skills/hindsight-docs/references/* via
./scripts/generate-docs-skill.sh:
- new skill pages for sdks/hindsight-all{.md,-npm.md}
- updated skill pages for sdks/embed.md and sdks/python.md to match
the new H1s and split content
- incidental refreshes to changelog/index.md, developer/models.md,
openapi.json, and uv.lock that verify-generated-files picked up
* ci: build openclaw before running tests so symlink test can realpath dist
* feat(openclaw): add session pattern filtering for ignore and stateless sessions
Adds three new config options to the OpenClaw plugin that allow filtering
sessions by key pattern before recall and retain operations fire:
- `ignoreSessionPatterns`: glob patterns for sessions to skip entirely
(no recall, no retain). Useful for cron/scheduled agent sessions.
- `statelessSessionPatterns`: glob patterns for read-only sessions —
retain is always skipped; recall is also skipped when
`skipStatelessSessions` is true (default).
- `skipStatelessSessions`: boolean (default: true). When false, sessions
matching statelessSessionPatterns can still recall but never retain.
Pattern syntax mirrors lossless-claw: `*` matches non-colon characters,
`**` matches anything including colons. Session keys follow the OpenClaw
format `agent:<agentId>:<type>:<uuid>`.
Example config:
ignoreSessionPatterns: ["agent:*:cron:**"]
statelessSessionPatterns: ["agent:*:subagent:**", "agent:*💓**"]
skipStatelessSessions: true
Implementation:
- New `session-patterns.ts` module with compile/match utilities
- Session filter applied in `before_prompt_build` and `agent_end` hooks
immediately after the existing `excludeProviders` check
- New fields wired through `getPluginConfig`
- Schema added to `openclaw.plugin.json` (additionalProperties: false
was already set, causing config validation errors without this)
- 11 unit tests in `session-patterns.test.ts`
- 5 integration tests added to `hooks.integration.test.ts`
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(openclaw): support HINDSIGHT_API_TOKEN in integration tests
Pass HINDSIGHT_API_TOKEN env var through to HindsightClient and plugin
config in integration tests so tests work against authenticated APIs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(openclaw): document session pattern filtering options
Add ignoreSessionPatterns, statelessSessionPatterns, and skipStatelessSessions
to the README config table with glob syntax reference and usage examples.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Marco Rutsch <marco@rutimka.de>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: update openclaw tests to use before_prompt_build hook and split doc-examples CI per language
- Update hooks.integration.test.ts: rename describe block and all
triggerHook calls from 'before_agent_start' to 'before_prompt_build'
to match the hook registered in index.ts (changed in PR #480)
- Fix 'includes the user message' test: prependContext contains memories
(bullet list), not the raw user query; update assertion accordingly
- Split test-doc-examples CI job into a matrix over [python, node, cli, go]
so each language runs in parallel; language-specific setup steps
(Rust/CLI build, Node.js, Python client, TypeScript client) are
conditional on matrix.language to avoid unnecessary work
* fix: spy on HindsightClient prototype to intercept all per-bank client instances
getClientForContext creates new HindsightClient instances per bank when
dynamicBankId is true, so vi.spyOn(c, 'recall') on the default client
never captured calls. Spy on HindsightClient.prototype instead so all
dynamically created bank clients are intercepted.
* feat(openclaw): squash branch updates for fork PR
* revert(api): drop memory_engine query normalization from this PR
* fix(openclaw): harden hook isolation and sanitize recall logging
* chore(openclaw): gate missing-senderId notice behind debug logger
* fix(openclaw): address remaining PR review follow-ups
* fix(openclaw): address upstream review comments on isolation and tests
* feat(openclaw): prepend current timestamp to recalled memory context
* chore(openclaw): sync package-lock version to 0.4.14
* chore(openclaw): format recall timestamp as yyyy-mm-dd HH:MM
* feat(openclaw): add configurable recall context composition
- Add recallRoles config to filter which message roles are included in recall query context
- Add recallContextTurns to control how many user turns of prior context to include
- Add recallMaxQueryChars to cap composed query length
- Reduce default max_tokens from 2048 to 1024 for recall responses
- Update documentation and plugin schema with new configuration options
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): put latest user message at end of recall query, add debug to schema
- Reorder composed recall query so latest user message is at the bottom,
giving embedding models the most weight where it matters most
- Update truncateRecallQuery to trim oldest context lines first,
always preserving the suffix (priority instruction + latest message)
- Add debug flag to openclaw.plugin.json schema
- Update tests to reflect new query order
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): add verbose debug logging for recall/retain
- Log full recall query (not just first 50 chars)
- Log all raw recall results with scores and content before topK trimming
- Log retain transcript preview and document ID
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): strip sender metadata envelope from prior context in recall query
Prior context messages passed to composeRecallQuery contained raw OpenClaw
envelope blocks (Sender/untrusted metadata JSON) which were diluting the
semantic signal of the recall query. Strip them the same way extractRecallQuery
already does for the latest message.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): add debug log for event.messages at recall time
Helps diagnose why recallContextTurns > 1 may not show extra context
by logging message count and roles available in event.messages.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): strip sender metadata envelope from rawMessage before recall query extraction
The rawMessage from Telegram group chats arrives wrapped in a:
---
Sender (untrusted metadata):
```json {...}```
<actual message>
---
envelope. This wasn't being stripped before extractRecallQuery used it,
so the full envelope including JSON metadata was being sent as the recall
query, severely diluting semantic relevance.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): warn when recallContextTurns > 1 but event.messages is empty
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): read messages from event.context.sessionEntry.messages for recall and retain
event.messages was always empty — the actual conversation history is at
event.context.sessionEntry.messages. Fall back to event.messages for
backwards compatibility. This fixes recallContextTurns and retain both
being unable to see the conversation history.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): extract stripMetadataEnvelopes helper and apply to retain path
- Add shared stripMetadataEnvelopes() to strip OpenClaw sender/conversation
metadata blocks from message content in all paths (recall query extraction,
prior context composition, and retain transcript)
- This prevents metadata-polluted memories (name/sender ID facts) from being
stored and ensures recall queries contain clean user text only
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): strip metadata envelopes after channel envelope extraction too
The prompt format is: [ChannelName ...]\n<metadata envelope>\n<message>
After extracting content after [ChannelName], the metadata envelope was
still present. Now stripMetadataEnvelopes runs again after the channel
envelope extraction step.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): switch recall hook from before_agent_start to before_prompt_build
before_prompt_build runs after session load and has messages available,
enabling recallContextTurns to work correctly. before_agent_start runs
pre-session with no messages.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): move current time inside memory tag, simplify recall query format
- Move "Current time" line inside <hindsight_memories> so it's not exposed
to the recall search as part of the query context
- Remove RECALL_QUERY_PRIORITY_INSTRUCTION and "Latest user message:" label
from composed recall query — the raw message is more effective for
semantic search without the extra prompt noise
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): address PR review comments on bank ID fallback and memory leaks
- Add early return in deriveBankId when ctx is undefined, falling back
to static default bank instead of generating a placeholder-filled ID
- Remove unused RECALL_QUERY_PRIORITY_INSTRUCTION dead constant
- Evict from banksWithMissionSet when evicting from clientsByBankId
to prevent unbounded memory growth in long-running instances
- Fix integration test hook name: before_agent_start → before_prompt_build
- Fix integration test assertions to match actual composeRecallQuery output
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): extract sender ID from inbound metadata blocks for bank ID derivation
Agent-phase hooks (before_prompt_build, agent_end) don't carry senderId in ctx
by design. Parse it from the "Conversation info / Sender (untrusted metadata)"
JSON blocks that OpenClaw injects into the prompt/messages instead.
- Add extractSenderIdFromText() helper that scans all metadata blocks and
returns the first sender_id / id field found
- before_prompt_build: extract from event.prompt/rawMessage, spread into ctx
before calling deriveBankId and getClientForContext
- agent_end: scan user messages for the metadata block, spread into effectiveCtx
before calling deriveBankId and getClientForContext
- Gracefully skipped when senderId is already present in ctx
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): scan messages from end for sender ID to handle group chats
When multiple users have spoken in a session, scanning from the front
returns the first sender in history rather than the one who triggered
the current agent run. Reverse the slice before finding so we always
pick the most recent user message's sender ID.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): use event.messages for sender ID in agent_end, not sessionEntry
sessionEntry.messages is the cleaned-up history without OpenClaw's injected
metadata prefix blocks. event.messages is the raw payload that still contains
the "Conversation info (untrusted metadata)" JSON — so parse sender_id from
there instead.
Also removes the unnecessary senderIdBySession cache added in the previous
attempt, since event.messages has everything needed directly.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): cache sender ID from before_prompt_build for use in agent_end
event.prompt in before_prompt_build contains OpenClaw's injected metadata
blocks with sender_id. event.messages in agent_end is clean history without
them — so parsing messages in agent_end never finds a sender ID.
Fix: cache the resolved sender ID (keyed by sessionKey) when it's extracted
in before_prompt_build, then look it up by sessionKey in agent_end.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* docs(openclaw): revert Auto-Recall token count to 1024 as unchanged from main
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix(openclaw): revert recallMaxTokens default from 2048 to 1024 to match main
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* fix: zeroentropy rerank URL missing /v1 prefix and MCP routing tests
- Fix ZeroEntropy reranker URL: /models/rerank -> /v1/models/rerank (#453)
- Fix test_mcp_routing tests: update assertions to use submit_async_retain
instead of the non-existent async_processing=False/retain_batch_async pattern
* fix(openclaw): pass retainEveryNTurns through getPluginConfig and set it to 1 in tests
getPluginConfig was not forwarding retainEveryNTurns from the raw config,
so pluginConfig.retainEveryNTurns was always undefined (defaulting to 10).
The integration tests use retainEveryNTurns: 1 so retain fires every turn.
* fix: improve openclaw test coverage
* test(openclaw): export stripMemoryTags/extractRecallQuery and add hook integration tests
- Extract stripMemoryTags and extractRecallQuery as exported pure functions
from index.ts so hooks share one implementation and tests cover the real code
- Update before_agent_start to call extractRecallQuery; update agent_end to
call stripMemoryTags instead of duplicating the regex inline
- Rewrite index.test.ts to import the real functions (no more local duplicate)
and add 11 tests for extractRecallQuery covering all envelope-stripping cases
- Add tests/hooks.integration.test.ts: loads the plugin via mock MoltbotPluginAPI
in HTTP mode, spies on client.recall/retain, and exercises all hook behaviours:
excluded providers, short messages, memory injection format, tag stripping,
transcript formatting, array content blocks, metadata, document_id derivation
Nicolò Boschi