* Fix async method parity and server keepalive timeout
The Python client's async methods were missing parameters available in
their sync counterparts, and the server's default keepalive timeout was
shorter than the client's, causing ServerDisconnectedError on reused
connections.
Server:
- Set uvicorn timeout_keep_alive to 30s (default was 5s). The Python
client (aiohttp) has a 15s client-side keepalive, so the server must
hold connections longer to prevent the client from writing to a
closed socket.
Python client - async method parity:
- arecall(): add trace, query_timestamp, include_entities,
include_chunks, max_entity_tokens, max_chunk_tokens. Return
RecallResponse instead of list[RecallResult].
- areflect(): add max_tokens and response_schema.
- acreate_bank(): new async method.
- aset_mission(): new async method.
- adelete_bank(): new async method.
Tests:
- Add test verifying uvicorn keepalive timeout exceeds client default.
- Add async tests for arecall (include_chunks, include_entities, trace,
full params), areflect (max_tokens, structured output), and
adelete_bank.
* Fix flaky tag tests by using entity-rich content and asserting on tags
The tag tests were unreliable because:
- Generic content ("Project X meeting notes") was frequently collapsed
during fact extraction, leaving no memories to recall
- Assertions checked LLM-rewritten text for literal substrings instead
of checking tags, which is what the tests are actually verifying
Fix: use distinctive, entity-rich content (named people with specific
actions) that reliably survives fact extraction, and assert on tag
membership rather than text content.
* feat(litellm): async retain with sync option, fix client session cleanup
- Add sync parameter to retain() for blocking vs background operation
- Default to async retain (sync=False) for better performance
- Add get_pending_retain_errors() to check async failures
- Fix "Unclosed client session" warnings by properly closing clients
- Fix "Timeout context manager" asyncio errors by creating fresh clients
- Each API call now creates and closes its own client (aiohttp limitation)
- Add _get_client() and _close_client() helpers for consistent handling
- Update recall(), reflect(), _retain_sync() and _inject_memories()
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(litellm): add reflect support and require explicit hindsight_query
- Make hindsight_query required when inject_memories=True to enforce
intentional memory queries (no automatic last-user-message fallback)
- Add reflect_context parameter for shaping LLM reasoning in reflect
- Add reflect_response_schema for structured JSON output from reflect
- Add _reflect_sync() and _reflect_async() methods in callbacks
- Update wrappers.py to support response_schema in reflect/areflect
This improves the developer experience by making memory injection
explicit and adds full reflect API support through the integration.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(litellm): rename recall_budget to budget, add per-call reflect context
- Rename `recall_budget` parameter to `budget` for consistency with API
- Add `hindsight_reflect_context` kwarg for per-call reflect context override
- Fix reflect() to not pass None values for optional parameters
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* docs(litellm): update README for new API structure and features
- Document configure() vs set_defaults() separation
- Add hindsight_query requirement when inject_memories=True
- Document async retain (sync=False default) and get_pending_retain_errors()
- Add hindsight_reflect_context per-call override documentation
- Document budget parameter (renamed from recall_budget)
- Add reflect_context and reflect_response_schema options
- Update all code examples to use new API structure
- Add new functions to API Reference table
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test(litellm): update tests for new configure/set_defaults API
- Update tests to use separate configure() and set_defaults() calls
- Fix test assertions to check config vs defaults appropriately
- Add tests for legacy parameter backwards compatibility
- Add new TestSetDefaults test class
- Fix _format_memories test call signature (settings, config order)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add set_bank_mission(), deprecate set_bank_background()
- Add mission parameter to hindsight_client.create_bank()
- Add set_bank_mission() function to hindsight_litellm
- Deprecate set_bank_background() with DeprecationWarning
- Update _create_or_update_bank() to support mission parameter
- Update README and docstrings to document the new API
The 'background' field has been deprecated in the Hindsight API in favor
of 'mission' which is used for mental model generation.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Remove deprecated background parameter and legacy configure() parameters
- Remove set_bank_background() in favor of set_bank_mission()
- Remove background parameter from _create_or_update_bank()
- Remove background parameter from hindsight_client.create_bank()
- Remove legacy parameters from configure() (bank_id, document_id, budget, etc.)
- These have been replaced by the set_defaults() API
- Remove legacy test cases for deprecated parameters
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: update tests and docs to use mission instead of background
The create_bank() parameter was renamed from background to mission.
Update all tests and doc examples to use the new parameter name.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>
* chore: run benchmarks with reflect mode
* chore: run benchmarks with reflect mode
* fixes
* new mm
* bunch of fixes
* initial commit
* fixes
* fixes
* fixes
* fix: sometimes memories gets extracted in the wrong language
* Add the LLM_PROVIDER in example
* fix the assert in testing recall
* trial to fix failing client tests
NotImplementedError: Cannot copy out of meta tensor; no data! Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to() when moving module from meta to a different device.
* lock the sentence transformer packages to align with the breaking changes around lazy tensor loading
* Add the LLM_PROVIDER in example
* fix the assert in testing recall
* trial to fix failing client tests
* pre-cache the model so CI doesn't need workarounds
* remove assert that is a race condition
The test was checking that the bank count increased, but with parallel tests (-n 8), other tests can delete their banks while this test is running, causing a race condition. The important assertion is assert test_bank_id in final_banks - which verifies the bank was actually created.
* add debug to figure out why docker build fails sometimes
* use the CPU only version of pytorch to avoid pulling cuda libraries
* add best match strategy to uv
* change the example openai model
