fleet-memory/hindsight-integrations/llamaindex/tests/test_manual.py
DK09876 2d787c4ffd
feat: add LlamaIndex integration (#672)
* feat: add LlamaIndex integration for Hindsight

Add hindsight-llamaindex package providing persistent memory tools for
LlamaIndex agents via the native BaseToolSpec pattern. Includes retain,
recall, and reflect tools, a convenience factory, global config, full
test suite, docs page, blog post, and integrations.json entry.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback for llamaindex integration

- Fix ReActAgent API: from_tools() → constructor, chat() → await run()
- Add create_bank step to all quickstart examples
- Add production patterns section to docs (tags, error handling, bank lifecycle)
- Add memory scoping recommendation to README
- Add when-not-to-use section to blog post
- Add LlamaIndex compatibility tests (agent acceptance, FunctionTool.call)
- Fix self-hosted auth wording in cookbook notebook

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use async client methods and asyncio.run() for runnable examples

- Use await client.acreate_bank() instead of sync create_bank() to
  avoid "event loop already running" errors in notebooks and async contexts
- Wrap plain Python examples in async def main() + asyncio.run(main())
  so they are copy-paste runnable as scripts
- Add Jupyter notebook tip to docs showing top-level await pattern
- Bank lifecycle example in docs now uses async acreate_bank

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add async tool methods to avoid event loop conflicts

HindsightToolSpec now provides both sync and async tool implementations
using LlamaIndex's (sync_fn, async_fn) tuple pattern in spec_functions.
Async agents (ReActAgent, etc.) use aretain/arecall/areflect natively,
avoiding the "Timeout context manager should be used inside a task"
error that occurred when sync _run_async() was called from within an
active event loop.

- Add aretain_memory, arecall_memory, areflect_on_memory async methods
- Extract shared kwargs builders (_retain_kwargs, _recall_kwargs, etc.)
- spec_functions now uses tuples: [("retain_memory", "aretain_memory"), ...]
- Tests verify tools have both sync fn and async fn set
- Notebook verified end-to-end with nbclient against local Hindsight

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove blog post from integration PR

The blog post will be pulled in separately from its own PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Address PR review: add context label, document_id auto-gen, bank mission, graceful errors

- Add `retain_context` param (default: "llamaindex") as source label on retain ops
- Auto-generate `document_id` as `{session_id}-{timestamp_ms}` when not provided
- Add `retain_async` param (default: True) for non-blocking retain processing
- Add `mission` param for automatic bank creation/management on first use
- Change error handling from raising HindsightError to graceful log + return message
- Add per-operation timeout constants in _client.py
- Add `context` and `mission` fields to config.py and configure()
- Update docs: document as standalone package (not LlamaHub), new params, patterns
- Tests: 51 passing (up from 34), covering all new features

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Restructure to LlamaIndex namespace packages + add BaseMemory implementation

Tools package (llama-index-tools-hindsight):
- Restructured from hindsight_llamaindex/ to llama_index/tools/hindsight/
- Import: from llama_index.tools.hindsight import HindsightToolSpec
- Follows PEP 420 implicit namespace package convention
- Removed retain_async param (client.retain() doesn't support async_processing)

Memory package (llama-index-memory-hindsight):
- New package: llama_index/memory/hindsight/
- HindsightMemory(BaseMemory) for automatic memory
- put() auto-retains user/assistant messages to Hindsight
- get(input) auto-recalls relevant memories, prepends as system message
- Graceful error handling, bank mission management, document_id generation
- 28 unit tests passing

Both packages follow LlamaIndex community conventions for future LlamaHub submission.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 10:43:35 +02:00

59 lines
2 KiB
Python

"""Manual integration test for Hindsight LlamaIndex tools.
Requires a running Hindsight server at http://localhost:8888.
Run with: uv run pytest tests/test_manual.py -v -s --no-header
"""
import uuid
import pytest
from hindsight_client import Hindsight
from llama_index.tools.hindsight import HindsightToolSpec, create_hindsight_tools
HINDSIGHT_URL = "http://localhost:8888"
@pytest.fixture
def client():
return Hindsight(base_url=HINDSIGHT_URL, timeout=30.0)
@pytest.fixture
def bank_id(client):
bid = f"test-llamaindex-{uuid.uuid4().hex[:8]}"
client.create_bank(bank_id=bid, name=bid)
return bid
@pytest.mark.skip(reason="Requires running Hindsight server")
class TestManualToolSpec:
def test_retain_and_recall_round_trip(self, client, bank_id):
spec = HindsightToolSpec(client=client, bank_id=bank_id)
# Retain a memory
result = spec.retain_memory("The user prefers dark mode in all applications.")
assert result == "Memory stored successfully."
# Recall it
result = spec.recall_memory("What are the user's UI preferences?")
assert "dark mode" in result.lower()
def test_create_hindsight_tools_factory(self, client, bank_id):
tools = create_hindsight_tools(client=client, bank_id=bank_id)
assert len(tools) == 3
# Find retain tool by name
retain_tool = next(t for t in tools if t.metadata.name == "retain_memory")
result = retain_tool("The user's favorite language is Python.")
assert "stored" in result.lower()
def test_reflect(self, client, bank_id):
spec = HindsightToolSpec(client=client, bank_id=bank_id)
spec.retain_memory("The user is a backend developer.")
spec.retain_memory("The user uses Python and Go daily.")
spec.retain_memory("The user prefers vim keybindings.")
result = spec.reflect_on_memory("What kind of developer is this user?")
assert len(result) > 0
assert result != "No relevant memories found."