holetron-lab/fleet-memory
1
0
Fork 0
Code Releases Packages Activity Actions

feat: add LlamaIndex integration (#672)

Browse source 
* feat: add LlamaIndex integration for Hindsight

Add hindsight-llamaindex package providing persistent memory tools for
LlamaIndex agents via the native BaseToolSpec pattern. Includes retain,
recall, and reflect tools, a convenience factory, global config, full
test suite, docs page, blog post, and integrations.json entry.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback for llamaindex integration

- Fix ReActAgent API: from_tools() → constructor, chat() → await run()
- Add create_bank step to all quickstart examples
- Add production patterns section to docs (tags, error handling, bank lifecycle)
- Add memory scoping recommendation to README
- Add when-not-to-use section to blog post
- Add LlamaIndex compatibility tests (agent acceptance, FunctionTool.call)
- Fix self-hosted auth wording in cookbook notebook

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use async client methods and asyncio.run() for runnable examples

- Use await client.acreate_bank() instead of sync create_bank() to
  avoid "event loop already running" errors in notebooks and async contexts
- Wrap plain Python examples in async def main() + asyncio.run(main())
  so they are copy-paste runnable as scripts
- Add Jupyter notebook tip to docs showing top-level await pattern
- Bank lifecycle example in docs now uses async acreate_bank

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add async tool methods to avoid event loop conflicts

HindsightToolSpec now provides both sync and async tool implementations
using LlamaIndex's (sync_fn, async_fn) tuple pattern in spec_functions.
Async agents (ReActAgent, etc.) use aretain/arecall/areflect natively,
avoiding the "Timeout context manager should be used inside a task"
error that occurred when sync _run_async() was called from within an
active event loop.

- Add aretain_memory, arecall_memory, areflect_on_memory async methods
- Extract shared kwargs builders (_retain_kwargs, _recall_kwargs, etc.)
- spec_functions now uses tuples: [("retain_memory", "aretain_memory"), ...]
- Tests verify tools have both sync fn and async fn set
- Notebook verified end-to-end with nbclient against local Hindsight

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove blog post from integration PR

The blog post will be pulled in separately from its own PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Address PR review: add context label, document_id auto-gen, bank mission, graceful errors

- Add `retain_context` param (default: "llamaindex") as source label on retain ops
- Auto-generate `document_id` as `{session_id}-{timestamp_ms}` when not provided
- Add `retain_async` param (default: True) for non-blocking retain processing
- Add `mission` param for automatic bank creation/management on first use
- Change error handling from raising HindsightError to graceful log + return message
- Add per-operation timeout constants in _client.py
- Add `context` and `mission` fields to config.py and configure()
- Update docs: document as standalone package (not LlamaHub), new params, patterns
- Tests: 51 passing (up from 34), covering all new features

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restructure to LlamaIndex namespace packages + add BaseMemory implementation

Tools package (llama-index-tools-hindsight):
- Restructured from hindsight_llamaindex/ to llama_index/tools/hindsight/
- Import: from llama_index.tools.hindsight import HindsightToolSpec
- Follows PEP 420 implicit namespace package convention
- Removed retain_async param (client.retain() doesn't support async_processing)

Memory package (llama-index-memory-hindsight):
- New package: llama_index/memory/hindsight/
- HindsightMemory(BaseMemory) for automatic memory
- put() auto-retains user/assistant messages to Hindsight
- get(input) auto-recalls relevant memories, prepends as system message
- Graceful error handling, bank mission management, document_id generation
- 28 unit tests passing

Both packages follow LlamaIndex community conventions for future LlamaHub submission.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
DK09876 2026-03-30 01:43:35 -07:00 committed by GitHub
parent 111e8c70a2
commit 2d787c4ffd
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
20 changed files with 7335 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

256
hindsight-docs/docs/sdks/integrations/llamaindex.md Normal file
View file

@ -0,0 +1,256 @@
---
sidebar_position: 8
---
# LlamaIndex
Persistent long-term memory for [LlamaIndex](https://docs.llamaindex.ai/) agents via Hindsight. Two packages are available:
- **`llama-index-tools-hindsight`** — Agent-driven memory tools (retain/recall/reflect)
- **`llama-index-memory-hindsight`** — Automatic memory via LlamaIndex's `BaseMemory` interface
Both follow the LlamaIndex namespace package convention (`from llama_index.tools.hindsight import ...`).
## Installation
```bash
# Tools only (agent-driven)
pip install llama-index-tools-hindsight
# Memory only (automatic)
pip install llama-index-memory-hindsight
# Both
pip install llama-index-tools-hindsight llama-index-memory-hindsight
```
---
## Automatic Memory (BaseMemory)
The simplest way to add Hindsight memory to a LlamaIndex agent. Messages are automatically stored on each turn, and relevant memories are recalled and injected as context.
```python
import asyncio
from hindsight_client import Hindsight
from llama_index.memory.hindsight import HindsightMemory
from llama_index.core.agent import ReActAgent
from llama_index.llms.openai import OpenAI
async def main():
client = Hindsight(base_url="http://localhost:8888")
memory = HindsightMemory.from_client(
client=client,
bank_id="user-123",
mission="Track user preferences and project context",
)
agent = ReActAgent(tools=tools, llm=OpenAI(model="gpt-4o"), memory=memory)
response = await agent.run("Remember that I prefer dark mode")
print(response)
asyncio.run(main())
```
### How It Works
| Event | What Happens |
|-------|-------------|
| Agent receives input | `get(input)` recalls relevant memories from Hindsight, prepends as system message |
| Agent produces output | `put(message)` retains the message to Hindsight for future recall |
| New session starts | Previous memories are available via recall; local chat buffer starts empty |
### `HindsightMemory.from_client()`
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `client` | `Hindsight` | *required* | Hindsight client instance |
| `bank_id` | `str` | *required* | Memory bank ID |
| `mission` | `str` | `None` | Bank mission — auto-creates bank on first use |
| `context` | `str` | `"llamaindex"` | Source label for retain operations |
| `budget` | `str` | `"mid"` | Recall budget level |
| `max_tokens` | `int` | `4096` | Max recall tokens |
| `tags` | `list[str]` | `None` | Tags for retain operations |
| `recall_tags` | `list[str]` | `None` | Tags to filter recall |
| `recall_tags_match` | `str` | `"any"` | Tag matching mode |
| `system_prompt` | `str` | *(built-in)* | Template for memory system message. Must contain `{memories}` |
| `chat_history_limit` | `int` | `100` | Max messages in local buffer |
Also available: `HindsightMemory.from_url(hindsight_api_url, bank_id, ...)` for creating without a pre-built client.
---
## Agent-Driven Tools (BaseToolSpec)
For explicit control, expose retain/recall/reflect as tools the agent can choose to call.
### Quick Start: Tool Spec
```python
import asyncio
from hindsight_client import Hindsight
from llama_index.tools.hindsight import HindsightToolSpec
from llama_index.llms.openai import OpenAI
from llama_index.core.agent import ReActAgent
async def main():
client = Hindsight(base_url="http://localhost:8888")
spec = HindsightToolSpec(
client=client,
bank_id="user-123",
mission="Track user preferences",
)
tools = spec.to_tool_list()
agent = ReActAgent(tools=tools, llm=OpenAI(model="gpt-4o"))
response = await agent.run("Remember that I prefer dark mode")
print(response)
asyncio.run(main())
```
### Quick Start: Factory Function
```python
from llama_index.tools.hindsight import create_hindsight_tools
tools = create_hindsight_tools(
client=client,
bank_id="user-123",
mission="Track user preferences",
)
```
### Selecting Tools
```python
# Via to_tool_list()
tools = spec.to_tool_list(spec_functions=["recall_memory", "reflect_on_memory"])
# Via factory flags
tools = create_hindsight_tools(
client=client,
bank_id="user-123",
include_retain=True,
include_recall=True,
include_reflect=False,
)
```
### Configuration
Set defaults via `configure()`, override per-call:
```python
from llama_index.tools.hindsight import configure
configure(
hindsight_api_url="http://localhost:8888",
api_key="your-api-key", # or set HINDSIGHT_API_KEY env var
budget="mid",
tags=["source:llamaindex"],
context="my-app",
mission="Track user preferences",
)
# Now create tools without passing client/url
tools = create_hindsight_tools(bank_id="user-123")
```
### `HindsightToolSpec()`
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `bank_id` | `str` | *required* | Hindsight memory bank to operate on |
| `client` | `Hindsight` | `None` | Pre-configured Hindsight client |
| `hindsight_api_url` | `str` | `None` | API URL (used if no client provided) |
| `api_key` | `str` | `None` | API key (used if no client provided) |
| `budget` | `str` | `None` → `"mid"` | Recall/reflect budget: `low`, `mid`, `high` |
| `max_tokens` | `int` | `None` → `4096` | Max tokens for recall results |
| `tags` | `list[str]` | `None` | Tags applied when storing memories |
| `recall_tags` | `list[str]` | `None` | Tags to filter recall results |
| `recall_tags_match` | `str` | `None` → `"any"` | Tag matching: `any`, `all`, `any_strict`, `all_strict` |
| `retain_metadata` | `dict[str, str]` | `None` | Default metadata for retain operations |
| `retain_document_id` | `str` | `None` | Document ID for retain. Auto-generates `{session}-{timestamp}` if not set |
| `retain_context` | `str` | `"llamaindex"` | Source label for retain operations |
| `recall_types` | `list[str]` | `None` | Fact types: `world`, `experience`, `opinion`, `observation` |
| `recall_include_entities` | `bool` | `False` | Include entity info in recall results |
| `reflect_context` | `str` | `None` | Additional context for reflect |
| `reflect_max_tokens` | `int` | `None` | Max tokens for reflect (defaults to `max_tokens`) |
| `reflect_response_schema` | `dict` | `None` | JSON schema to constrain reflect output |
| `reflect_tags` | `list[str]` | `None` | Tags for reflect (defaults to `recall_tags`) |
| `reflect_tags_match` | `str` | `None` | Tag matching for reflect (defaults to `recall_tags_match`) |
| `mission` | `str` | `None` | Bank mission — auto-creates bank on first use |
---
## Production Patterns
### Bank Mission
Set a mission to give the memory engine context for fact extraction:
```python
# Tools
spec = HindsightToolSpec(
client=client,
bank_id="user-123",
mission="Track user coding preferences, project context, and technical decisions",
)
# Memory
memory = HindsightMemory.from_client(
client=client,
bank_id="user-123",
mission="Track user coding preferences, project context, and technical decisions",
)
```
The bank is created automatically on first use. If it already exists, creation is silently skipped.
### Memory Scoping with Tags
```python
spec = HindsightToolSpec(
client=client,
bank_id="user-123",
tags=["source:chat", "session:abc"], # applied to all retains
recall_tags=["source:chat"], # filter recalls to chat memories
recall_tags_match="any",
)
```
### Error Handling
Both packages handle errors gracefully — operations are logged and return friendly messages instead of raising exceptions. Agents continue functioning even if memory is unavailable.
### Combining Tools + Memory
Use both packages together for maximum flexibility:
```python
from llama_index.tools.hindsight import create_hindsight_tools
from llama_index.memory.hindsight import HindsightMemory
# Automatic memory for context enrichment
memory = HindsightMemory.from_client(client=client, bank_id="user-123")
# Explicit tools for agent-driven reflect
tools = create_hindsight_tools(
client=client,
bank_id="user-123",
include_retain=False, # memory handles retain automatically
include_recall=False, # memory handles recall automatically
include_reflect=True, # agent can still explicitly reflect
)
agent = ReActAgent(tools=tools, llm=llm, memory=memory)
```
## Requirements
- Python 3.10+
- `llama-index-core >= 0.11.0`
- `hindsight-client >= 0.4.0`

10
hindsight-docs/src/data/integrations.json
View file

@ -120,6 +120,16 @@
"link": "/sdks/integrations/langgraph",
"icon": "/img/icons/langgraph.png"
},
{
"id": "llamaindex",
"name": "LlamaIndex",
"description": "Add persistent memory to LlamaIndex agents via the native BaseToolSpec pattern. Retain, recall, and reflect as standard tools.",
"type": "official",
"by": "hindsight",
"category": "framework",
"link": "/sdks/integrations/llamaindex",
"icon": "/img/icons/llamaindex.png"
},
{
"id": "nemoclaw",
"name": "NemoClaw",

44
hindsight-integrations/llamaindex-memory/README.md Normal file
View file

@ -0,0 +1,44 @@
# llama-index-memory-hindsight
Automatic long-term memory for LlamaIndex agents via [Hindsight](https://github.com/vectorize-io/hindsight).
Implements LlamaIndex's `BaseMemory` interface:
- **`put()`** — automatically retains user/assistant messages to Hindsight
- **`get()`** — recalls relevant memories and injects them as context
- **`reset()`** — clears the local chat buffer
## Installation
```bash
pip install llama-index-memory-hindsight
```
## Quick Start
```python
from hindsight_client import Hindsight
from llama_index.memory.hindsight import HindsightMemory
from llama_index.core.agent import ReActAgent
from llama_index.llms.openai import OpenAI
client = Hindsight(base_url="http://localhost:8888")
memory = HindsightMemory.from_client(
client=client,
bank_id="user-123",
mission="Track user preferences and project context",
)
agent = ReActAgent(tools=tools, llm=OpenAI(model="gpt-4o"), memory=memory)
```
## How It Works
- When the agent receives a message, `get(input)` recalls relevant memories from Hindsight and prepends them as a system message
- When the agent produces output, `put(message)` stores the conversation turn in Hindsight for future recall
- Chat history is kept in a local buffer for the current session; Hindsight provides cross-session persistence
## Requirements
- Python 3.10+
- `llama-index-core >= 0.11.0`
- `hindsight-client >= 0.4.0`

22
hindsight-integrations/llamaindex-memory/llama_index/memory/hindsight/__init__.py Normal file
View file

@ -0,0 +1,22 @@
"""Hindsight memory for LlamaIndex agents.
Provides automatic long-term memory via Hindsight's retain/recall APIs.
Messages are automatically stored on ``put()`` and relevant memories
are recalled on ``get()`` to enrich agent prompts.
Usage::
from llama_index.memory.hindsight import HindsightMemory
memory = HindsightMemory.from_client(
client=client,
bank_id="user-123",
)
agent = ReActAgent(tools=tools, llm=llm, memory=memory)
"""
from .base import HindsightMemory
__version__ = "0.1.0"
__all__ = ["HindsightMemory"]

369
hindsight-integrations/llamaindex-memory/llama_index/memory/hindsight/base.py Normal file
View file

@ -0,0 +1,369 @@
"""Hindsight BaseMemory implementation for LlamaIndex.
Provides automatic memory for LlamaIndex agents:
- ``put()`` retains messages to Hindsight for long-term storage
- ``get()`` recalls relevant memories and prepends them as context
- Chat history is kept in-memory for the current session
"""
import logging
import time
import uuid
from typing import Any, Optional
from hindsight_client import Hindsight
from llama_index.core.bridge.pydantic import Field, PrivateAttr
from llama_index.core.llms import ChatMessage, MessageRole
from llama_index.core.memory.types import BaseMemory
logger = logging.getLogger(__name__)
DEFAULT_SYSTEM_PROMPT = (
"Below are relevant memories from previous conversations:\n{memories}\n"
"Use these memories to provide more personalized and contextual responses."
)
class HindsightMemory(BaseMemory):
"""Automatic long-term memory for LlamaIndex agents via Hindsight.
On ``put()``, user and assistant messages are automatically retained
to Hindsight. On ``get()``, relevant memories are recalled and
prepended as a system message to enrich the agent's context.
This follows the same pattern as Mem0's LlamaIndex integration:
a local chat buffer for the current session, with Hindsight
providing cross-session long-term memory.
Args:
bank_id: Hindsight memory bank to operate on.
context: Source label for retain operations.
budget: Recall budget level (low/mid/high).
max_tokens: Maximum tokens for recall results.
tags: Tags applied when storing memories.
recall_tags: Tags to filter when recalling.
recall_tags_match: Tag matching mode.
system_prompt: Template for the memory system message.
Must contain ``{memories}`` placeholder.
chat_history_limit: Max messages to keep in local buffer.
Oldest messages are dropped when exceeded.
Example::
from hindsight_client import Hindsight
from llama_index.memory.hindsight import HindsightMemory
client = Hindsight(base_url="http://localhost:8888")
memory = HindsightMemory.from_client(
client=client,
bank_id="user-123",
mission="Track user preferences",
)
# Use with any LlamaIndex agent
agent = ReActAgent(tools=tools, llm=llm, memory=memory)
"""
bank_id: str = Field(description="Hindsight memory bank ID")
context: str = Field(default="llamaindex", description="Source label for retain")
budget: str = Field(default="mid", description="Recall budget level")
max_tokens: int = Field(default=4096, description="Max tokens for recall")
tags: Optional[list[str]] = Field(default=None, description="Tags for retain")
recall_tags: Optional[list[str]] = Field(
default=None, description="Tags to filter recall"
)
recall_tags_match: str = Field(default="any", description="Tag matching mode")
system_prompt: str = Field(
default=DEFAULT_SYSTEM_PROMPT, description="Memory system message template"
)
chat_history_limit: int = Field(
default=100, description="Max messages in local buffer"
)
_client: Hindsight = PrivateAttr()
_chat_history: list[ChatMessage] = PrivateAttr(default_factory=list)
_session_id: str = PrivateAttr()
_bank_initialized: bool = PrivateAttr(default=False)
_mission: Optional[str] = PrivateAttr(default=None)
def __init__(self, client: Hindsight, mission: Optional[str] = None, **kwargs: Any):
super().__init__(**kwargs)
self._client = client
self._session_id = str(uuid.uuid4())[:8]
self._mission = mission
self._chat_history = []
self._bank_initialized = False
@classmethod
def class_name(cls) -> str:
return "HindsightMemory"
@classmethod
def from_defaults(cls, **kwargs: Any) -> "HindsightMemory":
"""Create from defaults. Prefer ``from_client()`` instead."""
raise NotImplementedError(
"Use HindsightMemory.from_client() or HindsightMemory.from_url() instead."
)
@classmethod
def from_client(
cls,
client: Hindsight,
bank_id: str,
*,
mission: Optional[str] = None,
context: str = "llamaindex",
budget: str = "mid",
max_tokens: int = 4096,
tags: Optional[list[str]] = None,
recall_tags: Optional[list[str]] = None,
recall_tags_match: str = "any",
system_prompt: str = DEFAULT_SYSTEM_PROMPT,
chat_history_limit: int = 100,
) -> "HindsightMemory":
"""Create a HindsightMemory with a pre-configured client.
Args:
client: Hindsight client instance.
bank_id: Memory bank ID.
mission: Bank mission (creates bank on first use if set).
context: Source label for retain operations.
budget: Recall budget level.
max_tokens: Max recall tokens.
tags: Tags for retain operations.
recall_tags: Tags to filter recall.
recall_tags_match: Tag matching mode.
system_prompt: Memory system message template.
chat_history_limit: Max local buffer size.
"""
return cls(
client=client,
bank_id=bank_id,
mission=mission,
context=context,
budget=budget,
max_tokens=max_tokens,
tags=tags,
recall_tags=recall_tags,
recall_tags_match=recall_tags_match,
system_prompt=system_prompt,
chat_history_limit=chat_history_limit,
)
@classmethod
def from_url(
cls,
hindsight_api_url: str,
bank_id: str,
*,
api_key: Optional[str] = None,
**kwargs: Any,
) -> "HindsightMemory":
"""Create a HindsightMemory from an API URL.
Args:
hindsight_api_url: Hindsight API URL.
bank_id: Memory bank ID.
api_key: Optional API key.
**kwargs: Additional arguments passed to ``from_client()``.
"""
client_kwargs: dict[str, Any] = {"base_url": hindsight_api_url, "timeout": 30.0}
if api_key:
client_kwargs["api_key"] = api_key
client = Hindsight(**client_kwargs)
return cls.from_client(client=client, bank_id=bank_id, **kwargs)
def _ensure_bank(self) -> None:
if self._bank_initialized or not self._mission:
return
try:
self._client.create_bank(
bank_id=self.bank_id,
name=self.bank_id,
mission=self._mission,
)
except Exception as e:
logger.debug(f"Bank creation for {self.bank_id}: {e}")
self._bank_initialized = True
async def _aensure_bank(self) -> None:
if self._bank_initialized or not self._mission:
return
try:
await self._client.acreate_bank(
bank_id=self.bank_id,
name=self.bank_id,
mission=self._mission,
)
except Exception as e:
logger.debug(f"Bank creation for {self.bank_id}: {e}")
self._bank_initialized = True
def _generate_document_id(self) -> str:
return f"{self._session_id}-{int(time.time() * 1000)}"
def _retain_message(self, message: ChatMessage) -> None:
"""Retain a message to Hindsight (sync)."""
if message.role not in (MessageRole.USER, MessageRole.ASSISTANT):
return
content = str(message.content) if message.content else ""
if not content.strip():
return
try:
self._ensure_bank()
kwargs: dict[str, Any] = {
"bank_id": self.bank_id,
"content": content,
"context": self.context,
"document_id": self._generate_document_id(),
"metadata": {"role": message.role.value, "source": "llamaindex"},
}
if self.tags:
kwargs["tags"] = self.tags
self._client.retain(**kwargs)
except Exception as e:
logger.error(f"Failed to retain message: {e}")
async def _aretain_message(self, message: ChatMessage) -> None:
"""Retain a message to Hindsight (async)."""
if message.role not in (MessageRole.USER, MessageRole.ASSISTANT):
return
content = str(message.content) if message.content else ""
if not content.strip():
return
try:
await self._aensure_bank()
kwargs: dict[str, Any] = {
"bank_id": self.bank_id,
"content": content,
"context": self.context,
"document_id": self._generate_document_id(),
"metadata": {"role": message.role.value, "source": "llamaindex"},
}
if self.tags:
kwargs["tags"] = self.tags
await self._client.aretain(**kwargs)
except Exception as e:
logger.error(f"Failed to retain message: {e}")
def _recall_memories(self, query: str) -> str:
"""Recall relevant memories (sync)."""
try:
self._ensure_bank()
kwargs: dict[str, Any] = {
"bank_id": self.bank_id,
"query": query,
"budget": self.budget,
"max_tokens": self.max_tokens,
}
if self.recall_tags:
kwargs["tags"] = self.recall_tags
kwargs["tags_match"] = self.recall_tags_match
response = self._client.recall(**kwargs)
if not response.results:
return ""
lines = [r.text for r in response.results]
return "\n".join(f"- {line}" for line in lines)
except Exception as e:
logger.error(f"Failed to recall memories: {e}")
return ""
async def _arecall_memories(self, query: str) -> str:
"""Recall relevant memories (async)."""
try:
await self._aensure_bank()
kwargs: dict[str, Any] = {
"bank_id": self.bank_id,
"query": query,
"budget": self.budget,
"max_tokens": self.max_tokens,
}
if self.recall_tags:
kwargs["tags"] = self.recall_tags
kwargs["tags_match"] = self.recall_tags_match
response = await self._client.arecall(**kwargs)
if not response.results:
return ""
lines = [r.text for r in response.results]
return "\n".join(f"- {line}" for line in lines)
except Exception as e:
logger.error(f"Failed to recall memories: {e}")
return ""
# -- BaseMemory interface --
def get(self, input: Optional[str] = None, **kwargs: Any) -> list[ChatMessage]:
"""Get chat history, enriched with recalled Hindsight memories.
If ``input`` is provided, relevant memories are recalled and
prepended as a system message.
"""
messages: list[ChatMessage] = []
if input:
memories_text = self._recall_memories(input)
if memories_text:
system_content = self.system_prompt.format(memories=memories_text)
messages.append(
ChatMessage(role=MessageRole.SYSTEM, content=system_content)
)
messages.extend(self._chat_history)
return messages
async def aget(
self, input: Optional[str] = None, **kwargs: Any
) -> list[ChatMessage]:
"""Async version of get()."""
messages: list[ChatMessage] = []
if input:
memories_text = await self._arecall_memories(input)
if memories_text:
system_content = self.system_prompt.format(memories=memories_text)
messages.append(
ChatMessage(role=MessageRole.SYSTEM, content=system_content)
)
messages.extend(self._chat_history)
return messages
def get_all(self) -> list[ChatMessage]:
"""Get all messages in the local chat buffer."""
return list(self._chat_history)
def put(self, message: ChatMessage) -> None:
"""Store a message in local buffer and retain to Hindsight."""
self._chat_history.append(message)
# Trim to limit
if len(self._chat_history) > self.chat_history_limit:
self._chat_history = self._chat_history[-self.chat_history_limit :]
self._retain_message(message)
async def aput(self, message: ChatMessage) -> None:
"""Async version of put()."""
self._chat_history.append(message)
if len(self._chat_history) > self.chat_history_limit:
self._chat_history = self._chat_history[-self.chat_history_limit :]
await self._aretain_message(message)
def set(self, messages: list[ChatMessage]) -> None:
"""Set the chat history, retaining new messages to Hindsight."""
existing_len = len(self._chat_history)
self._chat_history = list(messages)
# Retain only new messages (beyond previous length)
for msg in messages[existing_len:]:
self._retain_message(msg)
async def aset(self, messages: list[ChatMessage]) -> None:
"""Async version of set()."""
existing_len = len(self._chat_history)
self._chat_history = list(messages)
for msg in messages[existing_len:]:
await self._aretain_message(msg)
def reset(self) -> None:
"""Reset the local chat buffer. Does not clear Hindsight memories."""
self._chat_history = []

54
hindsight-integrations/llamaindex-memory/pyproject.toml Normal file
View file

@ -0,0 +1,54 @@
[project]
name = "llama-index-memory-hindsight"
version = "0.1.0"
description = "LlamaIndex memory integration for Hindsight - automatic long-term memory for AI agents"
readme = "README.md"
requires-python = ">=3.10"
license = { text = "MIT" }
authors = [
{ name = "Vectorize", email = "support@vectorize.io" }
]
keywords = [
"ai",
"memory",
"llamaindex",
"llama-index",
"agents",
"hindsight",
]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
dependencies = [
"llama-index-core>=0.11.0",
"hindsight-client>=0.4.0",
]
[project.urls]
Homepage = "https://github.com/vectorize-io/hindsight"
Documentation = "https://github.com/vectorize-io/hindsight/tree/main/hindsight-integrations/llamaindex-memory"
Repository = "https://github.com/vectorize-io/hindsight"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["llama_index/"]
[tool.pytest.ini_options]
testpaths = ["tests"]
[dependency-groups]
dev = [
"pytest>=9.0.2",
"ruff>=0.8.0",
]

0
hindsight-integrations/llamaindex-memory/tests/__init__.py Normal file
View file

348
hindsight-integrations/llamaindex-memory/tests/test_memory.py Normal file
View file

@ -0,0 +1,348 @@
"""Unit tests for Hindsight LlamaIndex memory."""
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from llama_index.core.llms import ChatMessage, MessageRole
from llama_index.memory.hindsight import HindsightMemory
def _mock_client():
"""Create a mock Hindsight client."""
client = MagicMock()
client.retain = MagicMock()
client.recall = MagicMock()
client.create_bank = MagicMock()
client.aretain = AsyncMock()
client.arecall = AsyncMock()
client.acreate_bank = AsyncMock()
return client
def _mock_recall_response(texts: list[str]):
response = MagicMock()
results = []
for t in texts:
r = MagicMock()
r.text = t
results.append(r)
response.results = results
return response
class TestHindsightMemoryCreation:
def test_from_client(self):
client = _mock_client()
memory = HindsightMemory.from_client(
client=client,
bank_id="test-bank",
)
assert memory.bank_id == "test-bank"
assert memory.context == "llamaindex"
assert memory.budget == "mid"
def test_from_client_with_options(self):
client = _mock_client()
memory = HindsightMemory.from_client(
client=client,
bank_id="test-bank",
mission="Track preferences",
context="my-app",
budget="high",
tags=["source:chat"],
)
assert memory.bank_id == "test-bank"
assert memory.context == "my-app"
assert memory.budget == "high"
assert memory.tags == ["source:chat"]
def test_from_url(self):
with patch("llama_index.memory.hindsight.base.Hindsight") as mock_cls:
mock_cls.return_value = _mock_client()
memory = HindsightMemory.from_url(
hindsight_api_url="http://localhost:8888",
bank_id="test-bank",
)
assert memory.bank_id == "test-bank"
mock_cls.assert_called_once_with(
base_url="http://localhost:8888", timeout=30.0
)
def test_from_defaults_raises(self):
with pytest.raises(NotImplementedError):
HindsightMemory.from_defaults()
def test_class_name(self):
assert HindsightMemory.class_name() == "HindsightMemory"
class TestPut:
def test_put_user_message_retains(self):
client = _mock_client()
memory = HindsightMemory.from_client(client=client, bank_id="test")
msg = ChatMessage(role=MessageRole.USER, content="I like Python")
memory.put(msg)
client.retain.assert_called_once()
kwargs = client.retain.call_args[1]
assert kwargs["bank_id"] == "test"
assert kwargs["content"] == "I like Python"
assert kwargs["context"] == "llamaindex"
assert kwargs["metadata"]["role"] == "user"
def test_put_assistant_message_retains(self):
client = _mock_client()
memory = HindsightMemory.from_client(client=client, bank_id="test")
msg = ChatMessage(role=MessageRole.ASSISTANT, content="Noted!")
memory.put(msg)
client.retain.assert_called_once()
kwargs = client.retain.call_args[1]
assert kwargs["metadata"]["role"] == "assistant"
def test_put_system_message_skipped(self):
client = _mock_client()
memory = HindsightMemory.from_client(client=client, bank_id="test")
msg = ChatMessage(role=MessageRole.SYSTEM, content="You are helpful")
memory.put(msg)
client.retain.assert_not_called()
def test_put_empty_content_skipped(self):
client = _mock_client()
memory = HindsightMemory.from_client(client=client, bank_id="test")
msg = ChatMessage(role=MessageRole.USER, content=" ")
memory.put(msg)
client.retain.assert_not_called()
def test_put_adds_to_local_history(self):
client = _mock_client()
memory = HindsightMemory.from_client(client=client, bank_id="test")
msg = ChatMessage(role=MessageRole.USER, content="hello")
memory.put(msg)
assert len(memory.get_all()) == 1
assert memory.get_all()[0].content == "hello"
def test_put_trims_to_limit(self):
client = _mock_client()
memory = HindsightMemory.from_client(
client=client, bank_id="test", chat_history_limit=3
)
for i in range(5):
memory.put(ChatMessage(role=MessageRole.USER, content=f"msg-{i}"))
history = memory.get_all()
assert len(history) == 3
assert history[0].content == "msg-2"
assert history[2].content == "msg-4"
def test_put_tags_passed(self):
client = _mock_client()
memory = HindsightMemory.from_client(
client=client, bank_id="test", tags=["source:chat"]
)
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
kwargs = client.retain.call_args[1]
assert kwargs["tags"] == ["source:chat"]
def test_put_retain_failure_is_graceful(self):
client = _mock_client()
client.retain.side_effect = RuntimeError("connection refused")
memory = HindsightMemory.from_client(client=client, bank_id="test")
# Should not raise
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
# Message still in local history
assert len(memory.get_all()) == 1
def test_put_generates_document_id(self):
client = _mock_client()
memory = HindsightMemory.from_client(client=client, bank_id="test")
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
kwargs = client.retain.call_args[1]
doc_id = kwargs["document_id"]
parts = doc_id.rsplit("-", 1)
assert len(parts) == 2
assert parts[1].isdigit()
class TestGet:
def test_get_without_input_returns_history(self):
client = _mock_client()
memory = HindsightMemory.from_client(client=client, bank_id="test")
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
memory.put(ChatMessage(role=MessageRole.ASSISTANT, content="hi there"))
messages = memory.get()
assert len(messages) == 2
client.recall.assert_not_called()
def test_get_with_input_recalls_memories(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response(
["User likes Python", "User prefers dark mode"]
)
memory = HindsightMemory.from_client(client=client, bank_id="test")
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
messages = memory.get(input="What are my preferences?")
# Should have system message + chat history
assert len(messages) == 2
assert messages[0].role == MessageRole.SYSTEM
assert "User likes Python" in str(messages[0].content)
assert "User prefers dark mode" in str(messages[0].content)
assert messages[1].content == "hello"
def test_get_with_input_no_memories(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response([])
memory = HindsightMemory.from_client(client=client, bank_id="test")
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
messages = memory.get(input="anything")
# No system message when no memories found
assert len(messages) == 1
assert messages[0].content == "hello"
def test_get_recall_passes_params(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response(["fact"])
memory = HindsightMemory.from_client(
client=client,
bank_id="test",
budget="high",
max_tokens=2048,
recall_tags=["scope:user"],
recall_tags_match="all",
)
memory.get(input="query")
kwargs = client.recall.call_args[1]
assert kwargs["budget"] == "high"
assert kwargs["max_tokens"] == 2048
assert kwargs["tags"] == ["scope:user"]
assert kwargs["tags_match"] == "all"
def test_get_recall_failure_is_graceful(self):
client = _mock_client()
client.recall.side_effect = RuntimeError("timeout")
memory = HindsightMemory.from_client(client=client, bank_id="test")
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
# Should not raise, returns history without memories
messages = memory.get(input="query")
assert len(messages) == 1
def test_get_custom_system_prompt(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response(["fact1"])
memory = HindsightMemory.from_client(
client=client,
bank_id="test",
system_prompt="MEMORIES: {memories}",
)
messages = memory.get(input="query")
assert str(messages[0].content) == "MEMORIES: - fact1"
class TestSet:
def test_set_replaces_history(self):
client = _mock_client()
memory = HindsightMemory.from_client(client=client, bank_id="test")
memory.put(ChatMessage(role=MessageRole.USER, content="old"))
new_messages = [
ChatMessage(role=MessageRole.USER, content="new1"),
ChatMessage(role=MessageRole.ASSISTANT, content="new2"),
]
memory.set(new_messages)
history = memory.get_all()
assert len(history) == 2
assert history[0].content == "new1"
def test_set_retains_only_new_messages(self):
client = _mock_client()
memory = HindsightMemory.from_client(client=client, bank_id="test")
memory.put(ChatMessage(role=MessageRole.USER, content="existing"))
client.retain.reset_mock()
# Set with 3 messages (1 existing + 2 new)
messages = [
ChatMessage(role=MessageRole.USER, content="existing"),
ChatMessage(role=MessageRole.USER, content="new1"),
ChatMessage(role=MessageRole.ASSISTANT, content="new2"),
]
memory.set(messages)
# Should retain only the 2 new messages
assert client.retain.call_count == 2
class TestReset:
def test_reset_clears_local_history(self):
client = _mock_client()
memory = HindsightMemory.from_client(client=client, bank_id="test")
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
assert len(memory.get_all()) == 1
memory.reset()
assert len(memory.get_all()) == 0
class TestBankMission:
def test_creates_bank_with_mission_on_put(self):
client = _mock_client()
memory = HindsightMemory.from_client(
client=client, bank_id="test", mission="Track preferences"
)
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
client.create_bank.assert_called_once_with(
bank_id="test",
name="test",
mission="Track preferences",
)
def test_creates_bank_with_mission_on_get(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response([])
memory = HindsightMemory.from_client(
client=client, bank_id="test", mission="Track preferences"
)
memory.get(input="query")
client.create_bank.assert_called_once()
def test_bank_creation_idempotent(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response([])
memory = HindsightMemory.from_client(
client=client, bank_id="test", mission="mission"
)
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
memory.get(input="query")
assert client.create_bank.call_count == 1
def test_no_bank_creation_without_mission(self):
client = _mock_client()
memory = HindsightMemory.from_client(client=client, bank_id="test")
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
client.create_bank.assert_not_called()
def test_bank_creation_failure_is_graceful(self):
client = _mock_client()
client.create_bank.side_effect = RuntimeError("already exists")
memory = HindsightMemory.from_client(
client=client, bank_id="test", mission="mission"
)
# Should not raise
memory.put(ChatMessage(role=MessageRole.USER, content="hello"))
client.retain.assert_called_once()

2403
hindsight-integrations/llamaindex-memory/uv.lock Normal file
View file

File diff suppressed because it is too large Load diff

77
hindsight-integrations/llamaindex/README.md Normal file
View file

@ -0,0 +1,77 @@
# llama-index-tools-hindsight
LlamaIndex tools integration for [Hindsight](https://github.com/vectorize-io/hindsight) — persistent long-term memory for AI agents.
Provides Hindsight memory as a native LlamaIndex `BaseToolSpec`, giving agents retain/recall/reflect capabilities through LlamaIndex's standard tool interface.
For automatic memory (auto-recall on input, auto-retain on output), see [`llama-index-memory-hindsight`](../llamaindex-memory/).
## Installation
```bash
pip install llama-index-tools-hindsight
```
## Quick Start
```python
import asyncio
from hindsight_client import Hindsight
from llama_index.tools.hindsight import HindsightToolSpec
from llama_index.llms.openai import OpenAI
from llama_index.core.agent import ReActAgent
async def main():
client = Hindsight(base_url="http://localhost:8888")
spec = HindsightToolSpec(
client=client,
bank_id="user-123",
mission="Track user preferences",
)
tools = spec.to_tool_list()
agent = ReActAgent(tools=tools, llm=OpenAI(model="gpt-4o"))
response = await agent.run("Remember that I prefer dark mode")
print(response)
asyncio.run(main())
```
### Factory Function
```python
from llama_index.tools.hindsight import create_hindsight_tools
tools = create_hindsight_tools(
client=client,
bank_id="user-123",
include_reflect=False, # only retain + recall
)
```
## Configuration
```python
from llama_index.tools.hindsight import configure
configure(
hindsight_api_url="http://localhost:8888",
api_key="your-api-key",
budget="mid",
tags=["source:llamaindex"],
context="my-app",
mission="Track user preferences",
)
```
## Requirements
- Python 3.10+
- `llama-index-core >= 0.11.0`
- `hindsight-client >= 0.4.0`
## Documentation
- [Integration docs](https://docs.hindsight.vectorize.io/docs/sdks/integrations/llamaindex)
- [Hindsight API docs](https://docs.hindsight.vectorize.io)

30
hindsight-integrations/llamaindex/llama_index/tools/hindsight/__init__.py Normal file
View file

@ -0,0 +1,30 @@
"""Hindsight memory tools for LlamaIndex agents.
Provides a ``BaseToolSpec`` subclass and a convenience factory that give
LlamaIndex agents long-term memory via Hindsight's retain/recall/reflect APIs.
Usage::
from llama_index.tools.hindsight import HindsightToolSpec, create_hindsight_tools
"""
from .base import HindsightToolSpec, create_hindsight_tools
from .config import (
HindsightLlamaIndexConfig,
configure,
get_config,
reset_config,
)
from .errors import HindsightError
__version__ = "0.1.0"
__all__ = [
"configure",
"get_config",
"reset_config",
"HindsightLlamaIndexConfig",
"HindsightError",
"HindsightToolSpec",
"create_hindsight_tools",
]

39
hindsight-integrations/llamaindex/llama_index/tools/hindsight/_client.py Normal file
View file

@ -0,0 +1,39 @@
"""Shared Hindsight client resolution logic."""
from typing import Any, Optional
from hindsight_client import Hindsight
from .config import get_config
from .errors import HindsightError
# Per-operation timeouts (seconds)
TIMEOUT_RETAIN = 15.0
TIMEOUT_RECALL = 10.0
TIMEOUT_REFLECT = 30.0
TIMEOUT_BANK = 15.0
TIMEOUT_DEFAULT = 30.0
def resolve_client(
client: Optional[Hindsight],
hindsight_api_url: Optional[str],
api_key: Optional[str],
) -> Hindsight:
"""Resolve a Hindsight client from explicit args or global config."""
if client is not None:
return client
config = get_config()
url = hindsight_api_url or (config.hindsight_api_url if config else None)
key = api_key or (config.api_key if config else None)
if url is None:
raise HindsightError(
"No Hindsight API URL configured. Pass client= or hindsight_api_url=, or call configure() first."
)
kwargs: dict[str, Any] = {"base_url": url, "timeout": TIMEOUT_DEFAULT}
if key:
kwargs["api_key"] = key
return Hindsight(**kwargs)

462
hindsight-integrations/llamaindex/llama_index/tools/hindsight/base.py Normal file
View file

@ -0,0 +1,462 @@
"""LlamaIndex tool spec for Hindsight memory operations.
Provides a ``BaseToolSpec`` subclass and a convenience factory that create
LlamaIndex-compatible tools backed by Hindsight's retain/recall/reflect APIs.
"""
import logging
import time
import uuid
from typing import Any, Optional
from hindsight_client import Hindsight
from llama_index.core.tools.tool_spec.base import BaseToolSpec
from ._client import resolve_client
from .config import get_config
logger = logging.getLogger(__name__)
class HindsightToolSpec(BaseToolSpec):
"""LlamaIndex tool spec providing Hindsight memory tools.
Exposes retain, recall, and reflect as tools that LlamaIndex agents
can call natively via ``to_tool_list()``.
Args:
bank_id: The Hindsight memory bank to operate on.
client: Pre-configured Hindsight client (preferred).
hindsight_api_url: API URL (used if no client provided).
api_key: API key (used if no client provided).
budget: Recall/reflect budget level (low/mid/high).
max_tokens: Maximum tokens for recall results.
tags: Tags applied when storing memories via retain.
recall_tags: Tags to filter when searching memories.
recall_tags_match: Tag matching mode (any/all/any_strict/all_strict).
retain_metadata: Default metadata dict for retain operations.
retain_document_id: Default document_id for retain. If None,
auto-generates ``{session_id}-{timestamp_ms}`` per call.
retain_context: Source label for retain operations (default: "llamaindex").
recall_types: Fact types to filter (world, experience, opinion, observation).
recall_include_entities: Include entity information in recall results.
reflect_context: Additional context for reflect operations.
reflect_max_tokens: Max tokens for reflect results (defaults to max_tokens).
reflect_response_schema: JSON schema to constrain reflect output format.
reflect_tags: Tags to filter memories used in reflect (defaults to recall_tags).
reflect_tags_match: Tag matching for reflect (defaults to recall_tags_match).
mission: Bank mission for fact extraction. If provided, the bank
is created/updated with this mission on first use.
Example::
from hindsight_client import Hindsight
from llama_index.tools.hindsight import HindsightToolSpec
client = Hindsight(base_url="http://localhost:8888")
spec = HindsightToolSpec(client=client, bank_id="user-123")
tools = spec.to_tool_list()
# Use with a LlamaIndex agent
agent = ReActAgent(tools=tools, llm=llm)
"""
# Tuples provide both sync and async implementations to LlamaIndex.
# Async is used by async agents (ReActAgent, etc.); sync is a fallback.
spec_functions = [
("retain_memory", "aretain_memory"),
("recall_memory", "arecall_memory"),
("reflect_on_memory", "areflect_on_memory"),
]
def __init__(
self,
*,
bank_id: str,
client: Optional[Hindsight] = None,
hindsight_api_url: Optional[str] = None,
api_key: Optional[str] = None,
budget: Optional[str] = None,
max_tokens: Optional[int] = None,
tags: Optional[list[str]] = None,
recall_tags: Optional[list[str]] = None,
recall_tags_match: Optional[str] = None,
# Retain options
retain_metadata: Optional[dict[str, str]] = None,
retain_document_id: Optional[str] = None,
retain_context: Optional[str] = None,
# Recall options
recall_types: Optional[list[str]] = None,
recall_include_entities: bool = False,
# Reflect options
reflect_context: Optional[str] = None,
reflect_max_tokens: Optional[int] = None,
reflect_response_schema: Optional[dict[str, Any]] = None,
reflect_tags: Optional[list[str]] = None,
reflect_tags_match: Optional[str] = None,
# Bank management
mission: Optional[str] = None,
):
super().__init__()
self._client = resolve_client(client, hindsight_api_url, api_key)
self._bank_id = bank_id
self._session_id = str(uuid.uuid4())[:8]
self._bank_initialized = False
# Resolve effective values using None-sentinel config fallback
config = get_config()
self._tags = tags if tags is not None else (config.tags if config else None)
self._recall_tags = (
recall_tags
if recall_tags is not None
else (config.recall_tags if config else None)
)
self._recall_tags_match = (
recall_tags_match
if recall_tags_match is not None
else (config.recall_tags_match if config else "any")
)
self._budget = (
budget if budget is not None else (config.budget if config else "mid")
)
self._max_tokens = (
max_tokens
if max_tokens is not None
else (config.max_tokens if config else 4096)
)
# Retain-specific
self._retain_metadata = retain_metadata
self._retain_document_id = retain_document_id
self._retain_context = (
retain_context
if retain_context is not None
else (config.context if config else "llamaindex")
)
# Recall-specific
self._recall_types = recall_types
self._recall_include_entities = recall_include_entities
# Reflect-specific
self._reflect_context = reflect_context
self._reflect_max_tokens = reflect_max_tokens
self._reflect_response_schema = reflect_response_schema
self._reflect_tags = reflect_tags
self._reflect_tags_match = reflect_tags_match
# Bank management
self._mission = (
mission if mission is not None else (config.mission if config else None)
)
def _ensure_bank(self) -> None:
"""Create/update the bank with mission if not already done."""
if self._bank_initialized or not self._mission:
return
try:
self._client.create_bank(
bank_id=self._bank_id,
name=self._bank_id,
mission=self._mission,
)
self._bank_initialized = True
logger.debug(f"Created/updated bank: {self._bank_id}")
except Exception as e:
# Bank may already exist — that's fine
self._bank_initialized = True
logger.debug(f"Bank creation for {self._bank_id}: {e}")
async def _aensure_bank(self) -> None:
"""Async version of _ensure_bank."""
if self._bank_initialized or not self._mission:
return
try:
await self._client.acreate_bank(
bank_id=self._bank_id,
name=self._bank_id,
mission=self._mission,
)
self._bank_initialized = True
logger.debug(f"Created/updated bank: {self._bank_id}")
except Exception as e:
self._bank_initialized = True
logger.debug(f"Bank creation for {self._bank_id}: {e}")
def _generate_document_id(self) -> str:
"""Generate a unique document_id for retain operations."""
return f"{self._session_id}-{int(time.time() * 1000)}"
def _retain_kwargs(self, content: str) -> dict[str, Any]:
kwargs: dict[str, Any] = {
"bank_id": self._bank_id,
"content": content,
"context": self._retain_context,
}
if self._tags:
kwargs["tags"] = self._tags
if self._retain_metadata:
kwargs["metadata"] = self._retain_metadata
# Use explicit document_id if set, otherwise auto-generate
kwargs["document_id"] = self._retain_document_id or self._generate_document_id()
return kwargs
def _recall_kwargs(self, query: str) -> dict[str, Any]:
kwargs: dict[str, Any] = {
"bank_id": self._bank_id,
"query": query,
"budget": self._budget,
"max_tokens": self._max_tokens,
}
if self._recall_tags:
kwargs["tags"] = self._recall_tags
kwargs["tags_match"] = self._recall_tags_match
if self._recall_types:
kwargs["types"] = self._recall_types
if self._recall_include_entities:
kwargs["include_entities"] = True
return kwargs
def _reflect_kwargs(self, query: str) -> dict[str, Any]:
kwargs: dict[str, Any] = {
"bank_id": self._bank_id,
"query": query,
"budget": self._budget,
}
if self._reflect_context:
kwargs["context"] = self._reflect_context
effective_reflect_max = self._reflect_max_tokens or self._max_tokens
if effective_reflect_max:
kwargs["max_tokens"] = effective_reflect_max
if self._reflect_response_schema:
kwargs["response_schema"] = self._reflect_response_schema
effective_reflect_tags = (
self._reflect_tags if self._reflect_tags is not None else self._recall_tags
)
effective_reflect_tags_match = (
self._reflect_tags_match or self._recall_tags_match
)
if effective_reflect_tags:
kwargs["tags"] = effective_reflect_tags
kwargs["tags_match"] = effective_reflect_tags_match
return kwargs
@staticmethod
def _format_recall(response: Any) -> str:
if not response.results:
return "No relevant memories found."
lines = []
for i, result in enumerate(response.results, 1):
lines.append(f"{i}. {result.text}")
return "\n".join(lines)
# -- Sync methods (used outside async contexts) --
def retain_memory(self, content: str) -> str:
"""Store information to long-term memory for later retrieval.
Use this to save important facts, user preferences, decisions,
or any information that should be remembered across conversations.
Args:
content: The information to store in memory.
"""
try:
self._ensure_bank()
self._client.retain(**self._retain_kwargs(content))
return "Memory stored successfully."
except Exception as e:
logger.error(f"Retain failed: {e}")
return f"Failed to store memory: {e}"
def recall_memory(self, query: str) -> str:
"""Search long-term memory for relevant information.
Use this to find previously stored facts, preferences, or context.
Returns a numbered list of matching memories.
Args:
query: What to search for in memory.
"""
try:
self._ensure_bank()
response = self._client.recall(**self._recall_kwargs(query))
return self._format_recall(response)
except Exception as e:
logger.error(f"Recall failed: {e}")
return f"Failed to search memory: {e}"
def reflect_on_memory(self, query: str) -> str:
"""Synthesize a thoughtful answer from long-term memories.
Use this when you need a coherent summary or reasoned response
about what you know, rather than raw memory facts.
Args:
query: The question to reflect on using stored memories.
"""
try:
self._ensure_bank()
response = self._client.reflect(**self._reflect_kwargs(query))
return response.text or "No relevant memories found."
except Exception as e:
logger.error(f"Reflect failed: {e}")
return f"Failed to reflect on memory: {e}"
# -- Async methods (used by async agents like ReActAgent) --
async def aretain_memory(self, content: str) -> str:
"""Store information to long-term memory for later retrieval.
Use this to save important facts, user preferences, decisions,
or any information that should be remembered across conversations.
Args:
content: The information to store in memory.
"""
try:
await self._aensure_bank()
await self._client.aretain(**self._retain_kwargs(content))
return "Memory stored successfully."
except Exception as e:
logger.error(f"Retain failed: {e}")
return f"Failed to store memory: {e}"
async def arecall_memory(self, query: str) -> str:
"""Search long-term memory for relevant information.
Use this to find previously stored facts, preferences, or context.
Returns a numbered list of matching memories.
Args:
query: What to search for in memory.
"""
try:
await self._aensure_bank()
response = await self._client.arecall(**self._recall_kwargs(query))
return self._format_recall(response)
except Exception as e:
logger.error(f"Recall failed: {e}")
return f"Failed to search memory: {e}"
async def areflect_on_memory(self, query: str) -> str:
"""Synthesize a thoughtful answer from long-term memories.
Use this when you need a coherent summary or reasoned response
about what you know, rather than raw memory facts.
Args:
query: The question to reflect on using stored memories.
"""
try:
await self._aensure_bank()
response = await self._client.areflect(**self._reflect_kwargs(query))
return response.text or "No relevant memories found."
except Exception as e:
logger.error(f"Reflect failed: {e}")
return f"Failed to reflect on memory: {e}"
def create_hindsight_tools(
*,
bank_id: str,
client: Optional[Hindsight] = None,
hindsight_api_url: Optional[str] = None,
api_key: Optional[str] = None,
budget: Optional[str] = None,
max_tokens: Optional[int] = None,
tags: Optional[list[str]] = None,
recall_tags: Optional[list[str]] = None,
recall_tags_match: Optional[str] = None,
# Retain options
retain_metadata: Optional[dict[str, str]] = None,
retain_document_id: Optional[str] = None,
retain_context: Optional[str] = None,
# Recall options
recall_types: Optional[list[str]] = None,
recall_include_entities: bool = False,
# Reflect options
reflect_context: Optional[str] = None,
reflect_max_tokens: Optional[int] = None,
reflect_response_schema: Optional[dict[str, Any]] = None,
reflect_tags: Optional[list[str]] = None,
reflect_tags_match: Optional[str] = None,
# Bank management
mission: Optional[str] = None,
include_retain: bool = True,
include_recall: bool = True,
include_reflect: bool = True,
) -> list:
"""Create Hindsight memory tools for a LlamaIndex agent.
Convenience factory that creates a ``HindsightToolSpec`` and returns
a filtered list of ``FunctionTool`` instances ready for use with any
LlamaIndex agent.
Args:
bank_id: The Hindsight memory bank to operate on.
client: Pre-configured Hindsight client (preferred).
hindsight_api_url: API URL (used if no client provided).
api_key: API key (used if no client provided).
budget: Recall/reflect budget level (low/mid/high).
max_tokens: Maximum tokens for recall results.
tags: Tags applied when storing memories via retain.
recall_tags: Tags to filter when searching memories.
recall_tags_match: Tag matching mode (any/all/any_strict/all_strict).
retain_metadata: Default metadata dict for retain operations.
retain_document_id: Default document_id for retain. If None,
auto-generates per call.
retain_context: Source label for retain operations.
recall_types: Fact types to filter (world, experience, opinion, observation).
recall_include_entities: Include entity information in recall results.
reflect_context: Additional context for reflect operations.
reflect_max_tokens: Max tokens for reflect results (defaults to max_tokens).
reflect_response_schema: JSON schema to constrain reflect output format.
reflect_tags: Tags to filter memories used in reflect (defaults to recall_tags).
reflect_tags_match: Tag matching for reflect (defaults to recall_tags_match).
mission: Bank mission for fact extraction context.
include_retain: Include the retain (store) tool.
include_recall: Include the recall (search) tool.
include_reflect: Include the reflect (synthesize) tool.
Returns:
List of LlamaIndex FunctionTool instances.
Raises:
HindsightError: If no client or API URL can be resolved.
"""
spec = HindsightToolSpec(
bank_id=bank_id,
client=client,
hindsight_api_url=hindsight_api_url,
api_key=api_key,
budget=budget,
max_tokens=max_tokens,
tags=tags,
recall_tags=recall_tags,
recall_tags_match=recall_tags_match,
retain_metadata=retain_metadata,
retain_document_id=retain_document_id,
retain_context=retain_context,
recall_types=recall_types,
recall_include_entities=recall_include_entities,
reflect_context=reflect_context,
reflect_max_tokens=reflect_max_tokens,
reflect_response_schema=reflect_response_schema,
reflect_tags=reflect_tags,
reflect_tags_match=reflect_tags_match,
mission=mission,
)
spec_functions: list[tuple[str, str]] = []
if include_retain:
spec_functions.append(("retain_memory", "aretain_memory"))
if include_recall:
spec_functions.append(("recall_memory", "arecall_memory"))
if include_reflect:
spec_functions.append(("reflect_on_memory", "areflect_on_memory"))
if not spec_functions:
return []
return spec.to_tool_list(spec_functions=spec_functions)

101
hindsight-integrations/llamaindex/llama_index/tools/hindsight/config.py Normal file
View file

@ -0,0 +1,101 @@
"""Global configuration for Hindsight-LlamaIndex integration."""
import os
from dataclasses import dataclass
from typing import Optional
DEFAULT_HINDSIGHT_API_URL = "https://api.hindsight.vectorize.io"
HINDSIGHT_API_KEY_ENV = "HINDSIGHT_API_KEY"
@dataclass
class HindsightLlamaIndexConfig:
"""Connection and default settings for the LlamaIndex integration.
Attributes:
hindsight_api_url: URL of the Hindsight API server.
api_key: API key for Hindsight authentication.
budget: Default recall budget level (low/mid/high).
max_tokens: Default maximum tokens for recall results.
tags: Default tags applied when storing memories.
recall_tags: Default tags to filter when searching memories.
recall_tags_match: Tag matching mode (any/all/any_strict/all_strict).
context: Source label for retain operations (default: "llamaindex").
mission: Bank mission for fact extraction context.
verbose: Enable verbose logging.
"""
hindsight_api_url: str = DEFAULT_HINDSIGHT_API_URL
api_key: Optional[str] = None
budget: str = "mid"
max_tokens: int = 4096
tags: Optional[list[str]] = None
recall_tags: Optional[list[str]] = None
recall_tags_match: str = "any"
context: str = "llamaindex"
mission: Optional[str] = None
verbose: bool = False
_global_config: Optional[HindsightLlamaIndexConfig] = None
def configure(
hindsight_api_url: Optional[str] = None,
api_key: Optional[str] = None,
budget: str = "mid",
max_tokens: int = 4096,
tags: Optional[list[str]] = None,
recall_tags: Optional[list[str]] = None,
recall_tags_match: str = "any",
context: str = "llamaindex",
mission: Optional[str] = None,
verbose: bool = False,
) -> HindsightLlamaIndexConfig:
"""Configure Hindsight connection and default settings.
Args:
hindsight_api_url: Hindsight API URL (default: production).
api_key: API key. Falls back to HINDSIGHT_API_KEY env var.
budget: Default recall budget (low/mid/high).
max_tokens: Default max tokens for recall.
tags: Default tags for retain operations.
recall_tags: Default tags to filter recall/search.
recall_tags_match: Tag matching mode.
context: Source label for retain operations.
mission: Bank mission for fact extraction context.
verbose: Enable verbose logging.
Returns:
The configured HindsightLlamaIndexConfig.
"""
global _global_config
resolved_url = hindsight_api_url or DEFAULT_HINDSIGHT_API_URL
resolved_key = api_key or os.environ.get(HINDSIGHT_API_KEY_ENV)
_global_config = HindsightLlamaIndexConfig(
hindsight_api_url=resolved_url,
api_key=resolved_key,
budget=budget,
max_tokens=max_tokens,
tags=tags,
recall_tags=recall_tags,
recall_tags_match=recall_tags_match,
context=context,
mission=mission,
verbose=verbose,
)
return _global_config
def get_config() -> Optional[HindsightLlamaIndexConfig]:
"""Get the current global configuration."""
return _global_config
def reset_config() -> None:
"""Reset global configuration to None."""
global _global_config
_global_config = None

7
hindsight-integrations/llamaindex/llama_index/tools/hindsight/errors.py Normal file
View file

@ -0,0 +1,7 @@
"""Hindsight-LlamaIndex error types."""
class HindsightError(Exception):
"""Exception raised when a Hindsight memory operation fails."""
pass

54
hindsight-integrations/llamaindex/pyproject.toml Normal file
View file

@ -0,0 +1,54 @@
[project]
name = "llama-index-tools-hindsight"
version = "0.1.0"
description = "LlamaIndex tools integration for Hindsight - persistent memory for AI agents"
readme = "README.md"
requires-python = ">=3.10"
license = { text = "MIT" }
authors = [
{ name = "Vectorize", email = "support@vectorize.io" }
]
keywords = [
"ai",
"memory",
"llamaindex",
"llama-index",
"agents",
"hindsight",
]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
dependencies = [
"llama-index-core>=0.11.0",
"hindsight-client>=0.4.0",
]
[project.urls]
Homepage = "https://github.com/vectorize-io/hindsight"
Documentation = "https://github.com/vectorize-io/hindsight/tree/main/hindsight-integrations/llamaindex"
Repository = "https://github.com/vectorize-io/hindsight"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["llama_index/"]
[tool.pytest.ini_options]
testpaths = ["tests"]
[dependency-groups]
dev = [
"pytest>=9.0.2",
"ruff>=0.8.0",
]

0
hindsight-integrations/llamaindex/tests/__init__.py Normal file
View file

59
hindsight-integrations/llamaindex/tests/test_manual.py Normal file
View file

@ -0,0 +1,59 @@
"""Manual integration test for Hindsight LlamaIndex tools.
Requires a running Hindsight server at http://localhost:8888.
Run with: uv run pytest tests/test_manual.py -v -s --no-header
"""
import uuid
import pytest
from hindsight_client import Hindsight
from llama_index.tools.hindsight import HindsightToolSpec, create_hindsight_tools
HINDSIGHT_URL = "http://localhost:8888"
@pytest.fixture
def client():
return Hindsight(base_url=HINDSIGHT_URL, timeout=30.0)
@pytest.fixture
def bank_id(client):
bid = f"test-llamaindex-{uuid.uuid4().hex[:8]}"
client.create_bank(bank_id=bid, name=bid)
return bid
@pytest.mark.skip(reason="Requires running Hindsight server")
class TestManualToolSpec:
def test_retain_and_recall_round_trip(self, client, bank_id):
spec = HindsightToolSpec(client=client, bank_id=bank_id)
# Retain a memory
result = spec.retain_memory("The user prefers dark mode in all applications.")
assert result == "Memory stored successfully."
# Recall it
result = spec.recall_memory("What are the user's UI preferences?")
assert "dark mode" in result.lower()
def test_create_hindsight_tools_factory(self, client, bank_id):
tools = create_hindsight_tools(client=client, bank_id=bank_id)
assert len(tools) == 3
# Find retain tool by name
retain_tool = next(t for t in tools if t.metadata.name == "retain_memory")
result = retain_tool("The user's favorite language is Python.")
assert "stored" in result.lower()
def test_reflect(self, client, bank_id):
spec = HindsightToolSpec(client=client, bank_id=bank_id)
spec.retain_memory("The user is a backend developer.")
spec.retain_memory("The user uses Python and Go daily.")
spec.retain_memory("The user prefers vim keybindings.")
result = spec.reflect_on_memory("What kind of developer is this user?")
assert len(result) > 0
assert result != "No relevant memories found."

597
hindsight-integrations/llamaindex/tests/test_tools.py Normal file
View file

@ -0,0 +1,597 @@
"""Unit tests for Hindsight LlamaIndex tools."""
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from llama_index.tools.hindsight import (
HindsightToolSpec,
configure,
create_hindsight_tools,
reset_config,
)
from llama_index.tools.hindsight.errors import HindsightError
def _mock_client():
"""Create a mock Hindsight client with sync and async methods."""
client = MagicMock()
client.retain = MagicMock()
client.recall = MagicMock()
client.reflect = MagicMock()
client.create_bank = MagicMock()
client.aretain = AsyncMock()
client.arecall = AsyncMock()
client.areflect = AsyncMock()
client.acreate_bank = AsyncMock()
return client
def _mock_recall_response(texts: list[str]):
response = MagicMock()
results = []
for t in texts:
r = MagicMock()
r.text = t
results.append(r)
response.results = results
return response
def _mock_reflect_response(text: str):
response = MagicMock()
response.text = text
return response
def _mock_retain_response():
response = MagicMock()
response.success = True
return response
class TestHindsightToolSpec:
def test_spec_functions_list(self):
assert HindsightToolSpec.spec_functions == [
("retain_memory", "aretain_memory"),
("recall_memory", "arecall_memory"),
("reflect_on_memory", "areflect_on_memory"),
]
def test_to_tool_list_returns_three_tools(self):
client = _mock_client()
spec = HindsightToolSpec(bank_id="test", client=client)
tools = spec.to_tool_list()
assert len(tools) == 3
def test_to_tool_list_selective(self):
client = _mock_client()
spec = HindsightToolSpec(bank_id="test", client=client)
tools = spec.to_tool_list(spec_functions=[("recall_memory", "arecall_memory")])
assert len(tools) == 1
assert tools[0].metadata.name == "recall_memory"
class TestCreateHindsightTools:
def setup_method(self):
reset_config()
def teardown_method(self):
reset_config()
def test_returns_three_tools_by_default(self):
client = _mock_client()
tools = create_hindsight_tools(bank_id="test", client=client)
assert len(tools) == 3
def test_include_retain_only(self):
client = _mock_client()
tools = create_hindsight_tools(
bank_id="test",
client=client,
include_retain=True,
include_recall=False,
include_reflect=False,
)
assert len(tools) == 1
assert tools[0].metadata.name == "retain_memory"
def test_include_recall_only(self):
client = _mock_client()
tools = create_hindsight_tools(
bank_id="test",
client=client,
include_retain=False,
include_recall=True,
include_reflect=False,
)
assert len(tools) == 1
assert tools[0].metadata.name == "recall_memory"
def test_include_reflect_only(self):
client = _mock_client()
tools = create_hindsight_tools(
bank_id="test",
client=client,
include_retain=False,
include_recall=False,
include_reflect=True,
)
assert len(tools) == 1
assert tools[0].metadata.name == "reflect_on_memory"
def test_no_tools_when_all_excluded(self):
client = _mock_client()
tools = create_hindsight_tools(
bank_id="test",
client=client,
include_retain=False,
include_recall=False,
include_reflect=False,
)
assert len(tools) == 0
def test_raises_without_client_or_config(self):
with pytest.raises(HindsightError, match="No Hindsight API URL"):
create_hindsight_tools(bank_id="test")
def test_falls_back_to_global_config(self):
configure(hindsight_api_url="http://localhost:8888")
with patch("llama_index.tools.hindsight._client.Hindsight") as mock_cls:
mock_cls.return_value = _mock_client()
tools = create_hindsight_tools(bank_id="test")
assert len(tools) == 3
mock_cls.assert_called_once_with(
base_url="http://localhost:8888", timeout=30.0
)
def test_explicit_url_overrides_config(self):
configure(hindsight_api_url="http://config:8888")
with patch("llama_index.tools.hindsight._client.Hindsight") as mock_cls:
mock_cls.return_value = _mock_client()
create_hindsight_tools(
bank_id="test", hindsight_api_url="http://explicit:9999"
)
mock_cls.assert_called_once_with(
base_url="http://explicit:9999", timeout=30.0
)
class TestRetainTool:
def test_retain_stores_memory(self):
client = _mock_client()
client.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(bank_id="test-bank", client=client)
result = spec.retain_memory("The user likes Python")
assert result == "Memory stored successfully."
call_kwargs = client.retain.call_args[1]
assert call_kwargs["bank_id"] == "test-bank"
assert call_kwargs["content"] == "The user likes Python"
assert call_kwargs["context"] == "llamaindex"
def test_retain_passes_tags(self):
client = _mock_client()
client.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(
bank_id="test-bank", client=client, tags=["source:chat"]
)
spec.retain_memory("some content")
call_kwargs = client.retain.call_args[1]
assert call_kwargs["tags"] == ["source:chat"]
def test_retain_passes_metadata(self):
client = _mock_client()
client.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(
bank_id="test",
client=client,
retain_metadata={"source": "chat", "session": "abc"},
)
spec.retain_memory("content")
call_kwargs = client.retain.call_args[1]
assert call_kwargs["metadata"] == {"source": "chat", "session": "abc"}
def test_retain_passes_explicit_document_id(self):
client = _mock_client()
client.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(
bank_id="test", client=client, retain_document_id="session-123"
)
spec.retain_memory("content")
call_kwargs = client.retain.call_args[1]
assert call_kwargs["document_id"] == "session-123"
def test_retain_auto_generates_document_id(self):
client = _mock_client()
client.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(bank_id="test", client=client)
spec.retain_memory("content")
call_kwargs = client.retain.call_args[1]
doc_id = call_kwargs["document_id"]
# Auto-generated format: {session_id}-{timestamp_ms}
parts = doc_id.rsplit("-", 1)
assert len(parts) == 2
assert parts[1].isdigit()
def test_retain_passes_context_label(self):
client = _mock_client()
client.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(bank_id="test", client=client, retain_context="my-app")
spec.retain_memory("content")
call_kwargs = client.retain.call_args[1]
assert call_kwargs["context"] == "my-app"
def test_retain_defaults_to_llamaindex_context(self):
client = _mock_client()
client.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(bank_id="test", client=client)
spec.retain_memory("content")
call_kwargs = client.retain.call_args[1]
assert call_kwargs["context"] == "llamaindex"
def test_retain_returns_error_message_on_failure(self):
"""Errors are returned gracefully, not raised."""
client = _mock_client()
client.retain.side_effect = RuntimeError("connection refused")
spec = HindsightToolSpec(bank_id="test", client=client)
result = spec.retain_memory("content")
assert "Failed to store memory" in result
assert "connection refused" in result
class TestRecallTool:
def test_recall_returns_numbered_results(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response(
["User likes Python", "User is in NYC"]
)
spec = HindsightToolSpec(bank_id="test-bank", client=client)
result = spec.recall_memory("user preferences")
assert "1. User likes Python" in result
assert "2. User is in NYC" in result
def test_recall_empty_results(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response([])
spec = HindsightToolSpec(bank_id="test", client=client)
result = spec.recall_memory("anything")
assert result == "No relevant memories found."
def test_recall_passes_budget_and_max_tokens(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response(["fact"])
spec = HindsightToolSpec(
bank_id="test", client=client, budget="high", max_tokens=2048
)
spec.recall_memory("query")
call_kwargs = client.recall.call_args[1]
assert call_kwargs["budget"] == "high"
assert call_kwargs["max_tokens"] == 2048
def test_recall_passes_tags(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response(["fact"])
spec = HindsightToolSpec(
bank_id="test",
client=client,
recall_tags=["scope:user"],
recall_tags_match="all",
)
spec.recall_memory("query")
call_kwargs = client.recall.call_args[1]
assert call_kwargs["tags"] == ["scope:user"]
assert call_kwargs["tags_match"] == "all"
def test_recall_passes_types(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response(["fact"])
spec = HindsightToolSpec(
bank_id="test",
client=client,
recall_types=["world", "experience"],
)
spec.recall_memory("query")
call_kwargs = client.recall.call_args[1]
assert call_kwargs["types"] == ["world", "experience"]
def test_recall_passes_include_entities(self):
client = _mock_client()
client.recall.return_value = _mock_recall_response(["fact"])
spec = HindsightToolSpec(
bank_id="test", client=client, recall_include_entities=True
)
spec.recall_memory("query")
call_kwargs = client.recall.call_args[1]
assert call_kwargs["include_entities"] is True
def test_recall_returns_error_message_on_failure(self):
"""Errors are returned gracefully, not raised."""
client = _mock_client()
client.recall.side_effect = RuntimeError("timeout")
spec = HindsightToolSpec(bank_id="test", client=client)
result = spec.recall_memory("query")
assert "Failed to search memory" in result
class TestReflectTool:
def test_reflect_returns_text(self):
client = _mock_client()
client.reflect.return_value = _mock_reflect_response(
"The user is a Python developer who prefers functional patterns."
)
spec = HindsightToolSpec(bank_id="test-bank", client=client)
result = spec.reflect_on_memory("What do you know about the user?")
assert (
result == "The user is a Python developer who prefers functional patterns."
)
def test_reflect_empty_returns_fallback(self):
client = _mock_client()
client.reflect.return_value = _mock_reflect_response("")
spec = HindsightToolSpec(bank_id="test", client=client)
result = spec.reflect_on_memory("anything")
assert result == "No relevant memories found."
def test_reflect_passes_budget(self):
client = _mock_client()
client.reflect.return_value = _mock_reflect_response("answer")
spec = HindsightToolSpec(bank_id="test", client=client, budget="high")
spec.reflect_on_memory("query")
call_kwargs = client.reflect.call_args[1]
assert call_kwargs["budget"] == "high"
def test_reflect_passes_context(self):
client = _mock_client()
client.reflect.return_value = _mock_reflect_response("answer")
spec = HindsightToolSpec(
bank_id="test",
client=client,
reflect_context="The user is asking about project setup",
)
spec.reflect_on_memory("query")
call_kwargs = client.reflect.call_args[1]
assert call_kwargs["context"] == "The user is asking about project setup"
def test_reflect_passes_max_tokens_and_response_schema(self):
client = _mock_client()
client.reflect.return_value = _mock_reflect_response("answer")
schema = {"type": "object", "properties": {"summary": {"type": "string"}}}
spec = HindsightToolSpec(
bank_id="test",
client=client,
reflect_max_tokens=2048,
reflect_response_schema=schema,
)
spec.reflect_on_memory("query")
call_kwargs = client.reflect.call_args[1]
assert call_kwargs["max_tokens"] == 2048
assert call_kwargs["response_schema"] == schema
def test_reflect_passes_tags(self):
client = _mock_client()
client.reflect.return_value = _mock_reflect_response("answer")
spec = HindsightToolSpec(
bank_id="test",
client=client,
reflect_tags=["scope:global"],
reflect_tags_match="all",
)
spec.reflect_on_memory("query")
call_kwargs = client.reflect.call_args[1]
assert call_kwargs["tags"] == ["scope:global"]
assert call_kwargs["tags_match"] == "all"
def test_reflect_falls_back_to_recall_tags(self):
client = _mock_client()
client.reflect.return_value = _mock_reflect_response("answer")
spec = HindsightToolSpec(
bank_id="test",
client=client,
recall_tags=["scope:user"],
recall_tags_match="any",
)
spec.reflect_on_memory("query")
call_kwargs = client.reflect.call_args[1]
assert call_kwargs["tags"] == ["scope:user"]
assert call_kwargs["tags_match"] == "any"
def test_reflect_returns_error_message_on_failure(self):
"""Errors are returned gracefully, not raised."""
client = _mock_client()
client.reflect.side_effect = RuntimeError("timeout")
spec = HindsightToolSpec(bank_id="test", client=client)
result = spec.reflect_on_memory("query")
assert "Failed to reflect on memory" in result
class TestBankMission:
def test_creates_bank_with_mission_on_first_use(self):
client = _mock_client()
client.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(
bank_id="test-bank", client=client, mission="Track user preferences"
)
spec.retain_memory("content")
client.create_bank.assert_called_once_with(
bank_id="test-bank",
name="test-bank",
mission="Track user preferences",
)
def test_bank_creation_is_idempotent(self):
client = _mock_client()
client.retain.return_value = _mock_retain_response()
client.recall.return_value = _mock_recall_response(["fact"])
spec = HindsightToolSpec(
bank_id="test-bank", client=client, mission="my mission"
)
spec.retain_memory("content")
spec.recall_memory("query")
# create_bank should only be called once
assert client.create_bank.call_count == 1
def test_bank_creation_failure_is_graceful(self):
client = _mock_client()
client.create_bank.side_effect = RuntimeError("already exists")
client.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(
bank_id="test-bank", client=client, mission="my mission"
)
# Should not raise
result = spec.retain_memory("content")
assert result == "Memory stored successfully."
def test_no_bank_creation_without_mission(self):
client = _mock_client()
client.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(bank_id="test-bank", client=client)
spec.retain_memory("content")
client.create_bank.assert_not_called()
def test_mission_from_config(self):
reset_config()
configure(
hindsight_api_url="http://localhost:8888",
mission="config mission",
)
with patch("llama_index.tools.hindsight._client.Hindsight") as mock_cls:
mock_instance = _mock_client()
mock_cls.return_value = mock_instance
mock_instance.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(bank_id="test")
spec.retain_memory("content")
mock_instance.create_bank.assert_called_once_with(
bank_id="test",
name="test",
mission="config mission",
)
reset_config()
class TestLlamaIndexCompatibility:
"""Verify tools integrate correctly with LlamaIndex agent classes."""
def test_tools_have_correct_metadata(self):
"""Each tool should have name, description, and fn_schema."""
client = _mock_client()
spec = HindsightToolSpec(bank_id="test", client=client)
tools = spec.to_tool_list()
for tool in tools:
assert tool.metadata.name is not None
assert tool.metadata.description is not None
assert tool.metadata.fn_schema is not None
def test_tool_names_match_spec_functions(self):
client = _mock_client()
spec = HindsightToolSpec(bank_id="test", client=client)
tools = spec.to_tool_list()
tool_names = {t.metadata.name for t in tools}
assert tool_names == {"retain_memory", "recall_memory", "reflect_on_memory"}
def test_tools_accepted_by_react_agent(self):
"""ReActAgent should accept our tools without error."""
from llama_index.core.agent import ReActAgent
from llama_index.core.llms import MockLLM
client = _mock_client()
spec = HindsightToolSpec(bank_id="test", client=client)
tools = spec.to_tool_list()
# Should not raise — verifies tool format is compatible
agent = ReActAgent(tools=tools, llm=MockLLM())
assert agent is not None
def test_tools_have_both_sync_and_async(self):
"""Each tool should have both sync fn and async fn."""
client = _mock_client()
spec = HindsightToolSpec(bank_id="test", client=client)
tools = spec.to_tool_list()
for tool in tools:
assert tool._fn is not None, f"{tool.metadata.name} missing sync fn"
assert tool._async_fn is not None, f"{tool.metadata.name} missing async fn"
def test_retain_tool_callable_via_function_tool(self):
"""FunctionTool.call() should invoke retain_memory correctly."""
client = _mock_client()
client.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(bank_id="test", client=client)
tools = spec.to_tool_list(spec_functions=[("retain_memory", "aretain_memory")])
tool = tools[0]
result = tool.call(content="test memory")
assert "stored successfully" in str(result)
client.retain.assert_called_once()
def test_recall_tool_callable_via_function_tool(self):
"""FunctionTool.call() should invoke recall_memory correctly."""
client = _mock_client()
client.recall.return_value = _mock_recall_response(["some fact"])
spec = HindsightToolSpec(bank_id="test", client=client)
tools = spec.to_tool_list(spec_functions=[("recall_memory", "arecall_memory")])
tool = tools[0]
result = tool.call(query="test query")
assert "some fact" in str(result)
client.recall.assert_called_once()
class TestConfigFallback:
def setup_method(self):
reset_config()
def teardown_method(self):
reset_config()
def test_budget_falls_back_to_config(self):
configure(hindsight_api_url="http://localhost:8888")
with patch("llama_index.tools.hindsight._client.Hindsight") as mock_cls:
mock_instance = _mock_client()
mock_cls.return_value = mock_instance
mock_instance.recall.return_value = _mock_recall_response(["fact"])
# Configure with custom budget
reset_config()
configure(hindsight_api_url="http://localhost:8888", budget="high")
spec = HindsightToolSpec(bank_id="test")
spec.recall_memory("query")
call_kwargs = mock_instance.recall.call_args[1]
assert call_kwargs["budget"] == "high"
def test_explicit_budget_overrides_config(self):
configure(hindsight_api_url="http://localhost:8888", budget="high")
with patch("llama_index.tools.hindsight._client.Hindsight") as mock_cls:
mock_instance = _mock_client()
mock_cls.return_value = mock_instance
mock_instance.recall.return_value = _mock_recall_response(["fact"])
spec = HindsightToolSpec(bank_id="test", budget="low")
spec.recall_memory("query")
call_kwargs = mock_instance.recall.call_args[1]
assert call_kwargs["budget"] == "low"
def test_tags_fall_back_to_config(self):
configure(hindsight_api_url="http://localhost:8888", tags=["config:tag"])
with patch("llama_index.tools.hindsight._client.Hindsight") as mock_cls:
mock_instance = _mock_client()
mock_cls.return_value = mock_instance
mock_instance.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(bank_id="test")
spec.retain_memory("content")
call_kwargs = mock_instance.retain.call_args[1]
assert call_kwargs["tags"] == ["config:tag"]
def test_context_falls_back_to_config(self):
configure(hindsight_api_url="http://localhost:8888", context="my-app")
with patch("llama_index.tools.hindsight._client.Hindsight") as mock_cls:
mock_instance = _mock_client()
mock_cls.return_value = mock_instance
mock_instance.retain.return_value = _mock_retain_response()
spec = HindsightToolSpec(bank_id="test")
spec.retain_memory("content")
call_kwargs = mock_instance.retain.call_args[1]
assert call_kwargs["context"] == "my-app"

2403
hindsight-integrations/llamaindex/uv.lock Normal file
View file

File diff suppressed because it is too large Load diff