fleet-memory/hindsight-api-slim/hindsight_api
Nicolò Boschi f74b577e02
feat: add built-in llama.cpp LLM provider for local inference (#933)
* feat: add built-in llama.cpp LLM provider for fully local inference

Add `llamacpp` as a new LLM provider that manages a llama-cpp-python server
subprocess. Auto-downloads Gemma 4 E2B Q4_K_M (~3.5 GB) on first use and
runs inference locally via Metal/CUDA with no external services needed.

- New provider: `HINDSIGHT_API_LLM_PROVIDER=llamacpp`
- Singleton server shared across retain/reflect/consolidation
- Configurable: model path, GPU layers, context size, grammar enforcement
- User-extensible via `HINDSIGHT_API_LLAMACPP_EXTRA_ARGS`
- Flash attention + prompt caching enabled by default
- LLM provider cleanup on shutdown (stops subprocess)
- hindsight-embed: `--ui` flag on `daemon start`, removed FORCE_CPU on macOS
- Docs: configuration.md, models.mdx, providers grid updated

* chore: regenerate docs skill and update lockfile for local-llm dep
2026-04-08 15:22:10 +02:00
..
admin fix(migrations): bypass PgBouncer for advisory locks via MIGRATION_DATABASE_URL (#726) 2026-03-27 16:01:38 +01:00
alembic fix(recall): cap entity fanout in graph expansion (#911) 2026-04-07 18:59:50 +02:00
api feat: add retain update_mode='append' for document content concatenation (#932) 2026-04-08 14:39:16 +02:00
engine feat: add built-in llama.cpp LLM provider for local inference (#933) 2026-04-08 15:22:10 +02:00
extensions fix(retain): preserve normalized experience fact types (#848) 2026-04-02 12:20:37 +02:00
webhooks feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
worker Fix orphaned batch_retain parents when child fails via unhandled exception (#618) 2026-03-19 14:55:26 +01:00
__init__.py Release v0.4.22 2026-03-31 18:14:59 +02:00
banner.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
config.py feat: add built-in llama.cpp LLM provider for local inference (#933) 2026-04-08 15:22:10 +02:00
config_resolver.py fix(config): validate entity_labels structure on PATCH (#902) 2026-04-07 11:58:02 +02:00
daemon.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
main.py docs: add supported platforms and Windows installation guide (#700) 2026-03-26 12:01:30 +01:00
mcp_local.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
mcp_tools.py feat: add retain update_mode='append' for document content concatenation (#932) 2026-04-08 14:39:16 +02:00
metrics.py fix: make bank_id metric label opt-in to prevent OTel memory leak (#898) 2026-04-07 09:42:59 +02:00
migrations.py fix(migrations): bypass PgBouncer for advisory locks via MIGRATION_DATABASE_URL (#726) 2026-03-27 16:01:38 +01:00
models.py fix(retain): preserve normalized experience fact types (#848) 2026-04-02 12:20:37 +02:00
pg0.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
server.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
tracing.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
utils.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00