fleet-memory/hindsight-integrations/langgraph/tests/test_tools.py
DK09876 b4320254b2
feat: add LangGraph integration (#610)
* feat: add LangGraph integration with tools, nodes, and store patterns

Add hindsight-langgraph SDK providing three integration patterns:
- Tools: retain/recall/reflect as LangChain tools for ReAct agents
- Nodes: automatic memory injection and storage as graph steps
- Store: LangGraph BaseStore implementation for checkpoint-based memory

Fix: remove `from __future__ import annotations` in nodes.py which
prevented LangGraph from passing RunnableConfig to node functions
(runtime type inspection saw string annotations instead of actual types).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: register langgraph with independent versioning system

- Set version to 0.1.0 (integrations are versioned independently)
- Add langgraph to VALID_INTEGRATIONS in release-integration.sh
- Add changelog page for langgraph integration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove manual cookbook recipe page

The sync-cookbook script will auto-generate this from the notebook
in hindsight-cookbook once PR #17 is merged.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: comprehensive improvements to langgraph integration

Code fixes:
- Retain node only stores latest messages instead of all history (prevents duplicates)
- Handle multimodal msg.content (list type) in nodes
- Fix store docstring separator "/" → "."
- Apply search filters before pagination in store
- Add ttl parameter to store.aput for LangGraph BaseStore compat
- Fix _ensure_bank to not cache failed bank creations
- Fix falsy value bugs (or → is not None) in tools
- Remove from __future__ import annotations from all files
- Consistent default budget="mid" across tools/nodes/store
- Bump langgraph floor to >=0.3.0, remove duplicate dev deps

Docs fixes:
- Fix broken Cloud client example (base_url is required)
- Complete API reference tables with all parameters
- Add Limitations and Notes section (async-only store, etc.)
- Add Requirements section
- Fix broken cookbook link and Cloud claim in blog post

All 61 unit tests pass. E2E tested against Hindsight Cloud:
tools, nodes, store, configure(), multimodal content.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove blog post (lives in hindsight-marketing-content)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove Hindsight Cloud section from langgraph docs

Keep OSS docs self-hosted-first, consistent with other integration
docs (crewai, pydantic-ai, agno). Cloud setup details live in the
cookbook notebooks.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: explicitly mention LangChain compatibility in langgraph integration

The tools pattern (create_hindsight_tools) only depends on
langchain-core and works with plain LangChain via bind_tools() —
no LangGraph required. Update docs to make this clear with both
LangGraph and LangChain quick start examples.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review findings

1. Guard manual test files with if __name__ == "__main__" so pytest
   doesn't collect and execute them during test runs
2. Remove semantic fallback in HindsightStore.aget() — only return
   exact document_id matches, not unrelated semantic search hits
3. Make langgraph an optional dependency — tools pattern only needs
   langchain-core. Install with pip install hindsight-langgraph[langgraph]
   for nodes and store patterns. Lazy imports with clear error messages.
4. Clean up README to be self-hosted-first, consistent with other
   integration docs
5. Update docs requirements section to reflect optional langgraph dep

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback for langgraph integration

- Fix #2: Add per-bank asyncio.Lock to _ensure_bank for concurrency safety
- Fix #3: Clamp search score to max(0.0, ...) to prevent negative values
- Fix #4: Implement suffix matching in _handle_list_namespaces
- Fix #5: Truncate namespaces to max_depth instead of filtering (per BaseStore contract)
- Fix #6: Remove list_namespaces/alist_namespaces overrides — let base class handle prefix=/suffix= kwargs
- Fix #7: Document ephemeral namespace tracking and get() limitations in class docstring
- Fix #8: Add stable ID to recall node SystemMessage, document ordering behavior
- Fix #9: Change budget/max_tokens/recall_tags_match defaults to None so global config fallback works
- Fix #10: Conditionally populate __all__ so import * works without langgraph installed
- Fix #11: Bump langgraph lower bound from >=0.3.0 to >=0.5.0
- Fix #12: Extract _resolve_client to shared _client.py module

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address remaining review gaps for langgraph integration

- Add output_key parameter to create_recall_node for prompt ordering control
- Add prefix/suffix/combined filter tests for list_namespaces
- Add output_key unit tests (memory text, none on empty, none on error)
- Remove unused imports and backward-compat alias in tools.py
- Update docs with output_key usage example and API reference

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: relax langgraph version constraint to >=0.3.0

Research confirmed all required APIs (BaseStore, SearchItem, Result,
GetOp, PutOp, SearchOp, ListNamespacesOp) are available since
langgraph-checkpoint 2.0.7, which maps to langgraph >=0.2.63.
Using >=0.3.0 as a clean semver boundary — >=0.5.0 was unnecessarily
conservative and excluded many compatible versions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 13:36:57 +01:00

400 lines
13 KiB
Python

"""Unit tests for Hindsight LangGraph tools."""
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from hindsight_langgraph import (
configure,
create_hindsight_tools,
reset_config,
)
from hindsight_langgraph.errors import HindsightError
def _mock_client():
"""Create a mock Hindsight client with async methods."""
client = MagicMock()
client.aretain = AsyncMock()
client.arecall = AsyncMock()
client.areflect = AsyncMock()
return client
def _mock_recall_response(texts: list[str]):
response = MagicMock()
results = []
for t in texts:
r = MagicMock()
r.text = t
results.append(r)
response.results = results
return response
def _mock_reflect_response(text: str):
response = MagicMock()
response.text = text
return response
def _mock_retain_response():
response = MagicMock()
response.success = True
return response
class TestCreateHindsightTools:
def setup_method(self):
reset_config()
def teardown_method(self):
reset_config()
def test_returns_three_tools_by_default(self):
client = _mock_client()
tools = create_hindsight_tools(bank_id="test", client=client)
assert len(tools) == 3
def test_include_retain_only(self):
client = _mock_client()
tools = create_hindsight_tools(
bank_id="test",
client=client,
include_retain=True,
include_recall=False,
include_reflect=False,
)
assert len(tools) == 1
assert tools[0].name == "hindsight_retain"
def test_include_recall_only(self):
client = _mock_client()
tools = create_hindsight_tools(
bank_id="test",
client=client,
include_retain=False,
include_recall=True,
include_reflect=False,
)
assert len(tools) == 1
assert tools[0].name == "hindsight_recall"
def test_include_reflect_only(self):
client = _mock_client()
tools = create_hindsight_tools(
bank_id="test",
client=client,
include_retain=False,
include_recall=False,
include_reflect=True,
)
assert len(tools) == 1
assert tools[0].name == "hindsight_reflect"
def test_no_tools_when_all_excluded(self):
client = _mock_client()
tools = create_hindsight_tools(
bank_id="test",
client=client,
include_retain=False,
include_recall=False,
include_reflect=False,
)
assert len(tools) == 0
def test_raises_without_client_or_config(self):
with pytest.raises(HindsightError, match="No Hindsight API URL"):
create_hindsight_tools(bank_id="test")
def test_falls_back_to_global_config(self):
configure(hindsight_api_url="http://localhost:8888")
with patch("hindsight_langgraph._client.Hindsight") as mock_cls:
mock_cls.return_value = _mock_client()
tools = create_hindsight_tools(bank_id="test")
assert len(tools) == 3
mock_cls.assert_called_once_with(
base_url="http://localhost:8888", timeout=30.0
)
def test_explicit_url_overrides_config(self):
configure(hindsight_api_url="http://config:8888")
with patch("hindsight_langgraph._client.Hindsight") as mock_cls:
mock_cls.return_value = _mock_client()
create_hindsight_tools(
bank_id="test", hindsight_api_url="http://explicit:9999"
)
mock_cls.assert_called_once_with(
base_url="http://explicit:9999", timeout=30.0
)
class TestRetainTool:
@pytest.mark.asyncio
async def test_retain_stores_memory(self):
client = _mock_client()
client.aretain.return_value = _mock_retain_response()
tools = create_hindsight_tools(
bank_id="test-bank",
client=client,
include_recall=False,
include_reflect=False,
)
result = await tools[0].ainvoke("The user likes Python")
assert result == "Memory stored successfully."
client.aretain.assert_called_once_with(
bank_id="test-bank", content="The user likes Python"
)
@pytest.mark.asyncio
async def test_retain_passes_tags(self):
client = _mock_client()
client.aretain.return_value = _mock_retain_response()
tools = create_hindsight_tools(
bank_id="test-bank",
client=client,
tags=["source:chat"],
include_recall=False,
include_reflect=False,
)
await tools[0].ainvoke("some content")
call_kwargs = client.aretain.call_args[1]
assert call_kwargs["tags"] == ["source:chat"]
@pytest.mark.asyncio
async def test_retain_raises_hindsight_error(self):
client = _mock_client()
client.aretain.side_effect = RuntimeError("connection refused")
tools = create_hindsight_tools(
bank_id="test",
client=client,
include_recall=False,
include_reflect=False,
)
with pytest.raises(HindsightError, match="Retain failed"):
await tools[0].ainvoke("content")
class TestRecallTool:
@pytest.mark.asyncio
async def test_recall_returns_numbered_results(self):
client = _mock_client()
client.arecall.return_value = _mock_recall_response(
["User likes Python", "User is in NYC"]
)
tools = create_hindsight_tools(
bank_id="test-bank",
client=client,
include_retain=False,
include_reflect=False,
)
result = await tools[0].ainvoke("user preferences")
assert "1. User likes Python" in result
assert "2. User is in NYC" in result
@pytest.mark.asyncio
async def test_recall_empty_results(self):
client = _mock_client()
client.arecall.return_value = _mock_recall_response([])
tools = create_hindsight_tools(
bank_id="test",
client=client,
include_retain=False,
include_reflect=False,
)
result = await tools[0].ainvoke("anything")
assert result == "No relevant memories found."
@pytest.mark.asyncio
async def test_recall_passes_budget_and_max_tokens(self):
client = _mock_client()
client.arecall.return_value = _mock_recall_response(["fact"])
tools = create_hindsight_tools(
bank_id="test",
client=client,
budget="high",
max_tokens=2048,
include_retain=False,
include_reflect=False,
)
await tools[0].ainvoke("query")
call_kwargs = client.arecall.call_args[1]
assert call_kwargs["budget"] == "high"
assert call_kwargs["max_tokens"] == 2048
@pytest.mark.asyncio
async def test_recall_passes_tags(self):
client = _mock_client()
client.arecall.return_value = _mock_recall_response(["fact"])
tools = create_hindsight_tools(
bank_id="test",
client=client,
recall_tags=["scope:user"],
recall_tags_match="all",
include_retain=False,
include_reflect=False,
)
await tools[0].ainvoke("query")
call_kwargs = client.arecall.call_args[1]
assert call_kwargs["tags"] == ["scope:user"]
assert call_kwargs["tags_match"] == "all"
class TestReflectTool:
@pytest.mark.asyncio
async def test_reflect_returns_text(self):
client = _mock_client()
client.areflect.return_value = _mock_reflect_response(
"The user is a Python developer who prefers functional patterns."
)
tools = create_hindsight_tools(
bank_id="test-bank",
client=client,
include_retain=False,
include_recall=False,
)
result = await tools[0].ainvoke("What do you know about the user?")
assert (
result == "The user is a Python developer who prefers functional patterns."
)
@pytest.mark.asyncio
async def test_reflect_empty_returns_fallback(self):
client = _mock_client()
client.areflect.return_value = _mock_reflect_response("")
tools = create_hindsight_tools(
bank_id="test",
client=client,
include_retain=False,
include_recall=False,
)
result = await tools[0].ainvoke("anything")
assert result == "No relevant memories found."
@pytest.mark.asyncio
async def test_reflect_passes_budget(self):
client = _mock_client()
client.areflect.return_value = _mock_reflect_response("answer")
tools = create_hindsight_tools(
bank_id="test",
client=client,
budget="high",
include_retain=False,
include_recall=False,
)
await tools[0].ainvoke("query")
call_kwargs = client.areflect.call_args[1]
assert call_kwargs["budget"] == "high"
@pytest.mark.asyncio
async def test_reflect_passes_context(self):
client = _mock_client()
client.areflect.return_value = _mock_reflect_response("answer")
tools = create_hindsight_tools(
bank_id="test",
client=client,
reflect_context="The user is asking about project setup",
include_retain=False,
include_recall=False,
)
await tools[0].ainvoke("query")
call_kwargs = client.areflect.call_args[1]
assert call_kwargs["context"] == "The user is asking about project setup"
@pytest.mark.asyncio
async def test_reflect_passes_max_tokens_and_response_schema(self):
client = _mock_client()
client.areflect.return_value = _mock_reflect_response("answer")
schema = {"type": "object", "properties": {"summary": {"type": "string"}}}
tools = create_hindsight_tools(
bank_id="test",
client=client,
reflect_max_tokens=2048,
reflect_response_schema=schema,
include_retain=False,
include_recall=False,
)
await tools[0].ainvoke("query")
call_kwargs = client.areflect.call_args[1]
assert call_kwargs["max_tokens"] == 2048
assert call_kwargs["response_schema"] == schema
@pytest.mark.asyncio
async def test_reflect_passes_tags(self):
client = _mock_client()
client.areflect.return_value = _mock_reflect_response("answer")
tools = create_hindsight_tools(
bank_id="test",
client=client,
reflect_tags=["scope:global"],
reflect_tags_match="all",
include_retain=False,
include_recall=False,
)
await tools[0].ainvoke("query")
call_kwargs = client.areflect.call_args[1]
assert call_kwargs["tags"] == ["scope:global"]
assert call_kwargs["tags_match"] == "all"
class TestRetainExtendedParams:
@pytest.mark.asyncio
async def test_retain_passes_metadata(self):
client = _mock_client()
client.aretain.return_value = _mock_retain_response()
tools = create_hindsight_tools(
bank_id="test",
client=client,
retain_metadata={"source": "chat", "session": "abc"},
include_recall=False,
include_reflect=False,
)
await tools[0].ainvoke("content")
call_kwargs = client.aretain.call_args[1]
assert call_kwargs["metadata"] == {"source": "chat", "session": "abc"}
@pytest.mark.asyncio
async def test_retain_passes_document_id(self):
client = _mock_client()
client.aretain.return_value = _mock_retain_response()
tools = create_hindsight_tools(
bank_id="test",
client=client,
retain_document_id="session-123",
include_recall=False,
include_reflect=False,
)
await tools[0].ainvoke("content")
call_kwargs = client.aretain.call_args[1]
assert call_kwargs["document_id"] == "session-123"
class TestRecallExtendedParams:
@pytest.mark.asyncio
async def test_recall_passes_types(self):
client = _mock_client()
client.arecall.return_value = _mock_recall_response(["fact"])
tools = create_hindsight_tools(
bank_id="test",
client=client,
recall_types=["world", "experience"],
include_retain=False,
include_reflect=False,
)
await tools[0].ainvoke("query")
call_kwargs = client.arecall.call_args[1]
assert call_kwargs["types"] == ["world", "experience"]
@pytest.mark.asyncio
async def test_recall_passes_include_entities(self):
client = _mock_client()
client.arecall.return_value = _mock_recall_response(["fact"])
tools = create_hindsight_tools(
bank_id="test",
client=client,
recall_include_entities=True,
include_retain=False,
include_reflect=False,
)
await tools[0].ainvoke("query")
call_kwargs = client.arecall.call_args[1]
assert call_kwargs["include_entities"] is True