fleet-memory/hindsight-docs/docs
Nicolò Boschi eeb938fc65
fix: truncate documents exceeding LiteLLM reranker context limit (#549)
* fix: register embedded profiles in CLI metadata on daemon start

When HindsightEmbedded(profile="myapp") starts a daemon, the profile
was never written to metadata.json or given a .env file, making it
invisible to `hindsight-embed profile list` and other CLI commands.

Add _register_profile() to DaemonEmbedManager which saves HINDSIGHT_API_*
config to ~/.hindsight/profiles/{name}.env and registers the port in
metadata.json. Called after a successful new daemon start and when the
daemon is already running, so orphaned profiles also get registered on
next use.

* fix: truncate documents exceeding LiteLLM reranker context limit

Add HINDSIGHT_API_RERANKER_LITELLM_MAX_TOKENS_PER_DOC env var for both
litellm and litellm-sdk reranker providers. When set, documents are
truncated to the configured token limit using tiktoken (cl100k_base)
before being sent to the reranker, preventing BadRequestError for
models with small context windows (e.g. 1024-token limit).

* refactor: use shared _tiktoken_encoder for doc truncation in LiteLLM reranker

* refactor: use _get_tiktoken_encoding() consistently, remove eager module-level encoder instance

* doc: add HINDSIGHT_API_RERANKER_LITELLM_MAX_TOKENS_PER_DOC to configuration reference
2026-03-13 10:18:18 +01:00
..
developer fix: truncate documents exceeding LiteLLM reranker context limit (#549) 2026-03-13 10:18:18 +01:00
sdks feat: add MiniMax LLM provider support (#550) 2026-03-13 10:17:55 +01:00