fleet-memory/hindsight-api-slim/hindsight_api/engine/retain
Nicolò Boschi f74b577e02
feat: add built-in llama.cpp LLM provider for local inference (#933)
* feat: add built-in llama.cpp LLM provider for fully local inference

Add `llamacpp` as a new LLM provider that manages a llama-cpp-python server
subprocess. Auto-downloads Gemma 4 E2B Q4_K_M (~3.5 GB) on first use and
runs inference locally via Metal/CUDA with no external services needed.

- New provider: `HINDSIGHT_API_LLM_PROVIDER=llamacpp`
- Singleton server shared across retain/reflect/consolidation
- Configurable: model path, GPU layers, context size, grammar enforcement
- User-extensible via `HINDSIGHT_API_LLAMACPP_EXTRA_ARGS`
- Flash attention + prompt caching enabled by default
- LLM provider cleanup on shutdown (stops subprocess)
- hindsight-embed: `--ui` flag on `daemon start`, removed FORCE_CPU on macOS
- Docs: configuration.md, models.mdx, providers grid updated

* chore: regenerate docs skill and update lockfile for local-llm dep
2026-04-08 15:22:10 +02:00
..
__init__.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
bank_utils.py fix(db): respect vector extension config in per-bank index migration (#832) 2026-04-01 12:22:08 +02:00
chunk_storage.py feat(retain): delta retain — skip LLM for unchanged chunks on upsert (#701) 2026-03-26 13:50:55 +01:00
embedding_processing.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
embedding_utils.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
entity_labels.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
entity_processing.py perf: 3-phase retain pipeline — fix deadlocks, cap temporal links, query-time entity expansion (#722) 2026-04-01 12:52:49 +02:00
fact_extraction.py feat: add built-in llama.cpp LLM provider for local inference (#933) 2026-04-08 15:22:10 +02:00
fact_storage.py feat: add retain update_mode='append' for document content concatenation (#932) 2026-04-08 14:39:16 +02:00
link_creation.py perf: 3-phase retain pipeline — fix deadlocks, cap temporal links, query-time entity expansion (#722) 2026-04-01 12:52:49 +02:00
link_utils.py fix: resolve 25 test regressions from streaming retain pipeline (#722) (#836) 2026-04-01 17:59:10 +02:00
orchestrator.py feat: add retain update_mode='append' for document content concatenation (#932) 2026-04-08 14:39:16 +02:00
types.py feat: add retain update_mode='append' for document content concatenation (#932) 2026-04-08 14:39:16 +02:00