914ba7962c
* perf: 3-phase retain pipeline — fix deadlocks, cap temporal links, query-time entity expansion
Major retain pipeline overhaul addressing deadlocks, write amplification,
and TimeoutErrors. Restructures retain into three phases:
Phase 1: Entity resolution on separate connection (read-heavy)
Phase 2: Core write transaction (atomic) — facts, unit_entities, links
Phase 3: Best-effort display data (error-isolated) — entity viz links, stats
Key changes:
- Sorted bulk INSERT FROM unnest() prevents deadlocks
- Temporal links capped to top-20 per unit (95% reduction)
- Batched semantic ANN via temp table + LATERAL
- Query-time entity expansion via unit_entities self-join
- Entity viz links moved to Phase 3 (post-transaction)
- HINDSIGHT_API_RETAIN_MAX_CONCURRENT config (default: 32)
* fix: increase semantic link top_k from 5 to 20
The hardcoded top_k=5 was artificially limiting semantic link creation.
Link expansion retrieval can consume up to budget (50-200) semantic
neighbors per seed set, but each fact only had 5 outgoing edges — making
the bidirectional graph very sparse.
Increasing to 20 gives retrieval 4x more edges to work with. The ANN
probe cost is unchanged (same HNSW traversal per fact, just returning
more rows). INSERT cost is negligible (~14k rows via bulk INSERT).
Also: all 18 TimeoutErrors in the latest benchmark (beam-1m-u20) were
from Gemini LLM calls, zero from the database — confirming the entity
resolution split eliminated DB timeouts entirely.
* perf: move semantic ANN search to Phase 1 to avoid transaction timeouts
The batched LATERAL ANN query (700 HNSW probes) was the last remaining
source of DB TimeoutErrors — all 29 in the latest benchmark were from
create_semantic_links_batch inside the Phase 2 write transaction.
Split semantic link creation into three phases:
- Phase 1 (separate conn, autocommit): ANN search via temp table + LATERAL.
No transaction locks, no contention with concurrent writers.
- Phase 2 (write transaction): within-batch numpy similarities (instant) +
INSERT of both within-batch and Phase 1 ANN results. No DB reads.
- Phase 3 (flush_pending_stats): future hook point for re-checking ANN
results after commit to catch links missed by concurrent batches.
Also adds 7 unit tests for compute_semantic_links_within_batch covering
empty input, identical/orthogonal embeddings, threshold filtering, top_k
cap, and tuple structure validation.
* fix: handle placeholder unit_ids in Phase 1 ANN search (not valid UUIDs)
* test: add Phase 1 ANN cross-batch test + configurable test PG port
- New test_semantic_links_phase1_ann_cross_batch verifies that the Phase 1
ANN search with placeholder unit IDs correctly creates cross-batch
semantic links after remapping to real IDs.
- Test PG port now configurable via HINDSIGHT_TEST_PG_PORT env var
(default: 5556) to avoid conflicts with running benchmark daemons.
* perf: remove retry_with_backoff from retain, set semaphore default to 4
Remove retry_with_backoff from _run_db_work and _run_delta_db_work:
- Deadlocks are prevented by sorted bulk INSERT (no need for retry)
- Transient timeouts are handled by the worker poller's task-level retry
(3 attempts, 60s spacing) which is better than rapid internal retries
that amplify I/O pressure during contention storms
Set HINDSIGHT_API_RETAIN_MAX_CONCURRENT default from 32 to 4:
- The semaphore gates Phase 1 (ANN + entity resolution) + Phase 2 (writes)
- At 4 concurrent, HNSW index I/O is manageable; at 10+ concurrent the
probes saturate disk and cause cascading timeouts
- LLM extraction still runs at full parallelism (semaphore acquired after)
* fix: add fact_type filter to Phase 1 ANN query to use per-bank HNSW indexes
The LATERAL ANN query was falling back to sequential scan + sort (90ms/probe)
because the per-bank HNSW indexes are partial indexes filtered on fact_type.
Without fact_type in the WHERE clause, PostgreSQL couldn't use them.
Fix: iterate over ('world', 'experience') and run one HNSW-indexed ANN per
type. EXPLAIN shows 8ms/probe (was 90ms) — 11x faster.
700 probes × 8ms × 2 types = ~11s total (was ~63s via seq scan).
* fix: scope temporal links by fact_type + add integration tests
Temporal links now filter by fact_type in the LATERAL query — world facts
only link to world facts, experience to experience. This matches how
retrieval filters results and avoids wasted cross-type link rows.
New integration tests:
- test_semantic_ann_uses_hnsw_index: verifies Phase 1 ANN creates
cross-batch semantic links (tests fact_type filter + placeholder remap)
- test_temporal_links_scoped_by_fact_type: verifies world facts get
temporal links to other world facts but NOT to experience facts
* fix: tolerate individual chunk LLM failures instead of failing entire batch
Changed asyncio.gather(*tasks) to asyncio.gather(*tasks, return_exceptions=True)
in both chunk-level and content-level fact extraction. A single chunk timeout
(e.g., Gemini >90s) no longer discards all other successfully extracted facts.
For a 50MB document with 17k chunks, even a 2% chunk failure rate previously
caused 0 completions (entire batch discarded). Now 16,700 facts are extracted
and only the 300 failed chunks are skipped with a warning log.
* fix: batch temporal LATERAL query for large documents (16k+ chunks)
The LATERAL query for temporal links passed all unit_ids at once into
unnest(), causing PostgreSQL timeouts on documents with 16k+ chunks.
Split into batches of 500 units per query to keep each under the
command_timeout.
Also identified: HNSW index creation on shared pg0 instances with
50k+ existing units exceeds the 60s command_timeout. This is a
test infrastructure issue (shared pg0 accumulates data) but also
affects production when creating new banks on large instances.
* feat: streaming chunk batching for large documents (RETAIN_CHUNK_BATCH_SIZE)
Process chunks in mini-batches of N (default 500), committing each batch
to the DB before starting the next. This prevents OOM kills on large
documents (50MB / 17k+ chunks) by keeping only ~500 facts + embeddings
in memory at a time instead of 50k+.
Each mini-batch goes through the full Phase 1 → 2 → 3 pipeline
independently, sharing the same document_id. On recovery (process dies
mid-way), delta retain detects already-committed chunks via content_hash
and skips them — only remaining chunks get re-extracted.
Config: HINDSIGHT_API_RETAIN_CHUNK_BATCH_SIZE (default: 500, 0 to disable)
Per-bank configurable via the hierarchical config system.
Tests:
- test_streaming_chunk_batching_produces_same_facts
- test_streaming_chunk_batching_recovery (delta retain skips committed chunks)
- test_streaming_disabled_for_small_docs
* perf(retain): producer-consumer pipeline + deferred semantic ANN
Replace the sequential streaming loop with a producer-consumer pipeline:
- LLM producer fires concurrent chunk extractions (semaphore-bounded)
- DB consumer drains queue in batches, runs Phase 1+2+3 per batch
- LLM and DB work overlap instead of running sequentially
Defer semantic links to a single final ANN pass after all batches commit:
- Remove within-batch semantic links from Phase 2 (was 2.6s/batch)
- Run parallel ANN (4 connections) after all facts committed
- top_k reduced from 50 to 20 (recall uses at most 20 neighbors)
- Recovery via operation result_metadata checkpoint
Additional optimizations:
- skip_exists_check on temporal/causal link INSERT (saves ~0.5s/batch)
- WHERE EXISTS guard on semantic link INSERT (handles document upsert)
- timeout=300s on ANN queries and bulk INSERT for large banks
- Demote [ANN] debug logs to logger.debug()
- Fix docstring typos (agent_id → bank_id)
- Fix content_index remapping in producer-consumer batches
- Fix delta retain passing contents vs delta_contents
50MB benchmark (mock LLM): 9.2 min (was 23 min) — 2.5x faster.
BEAM 10m benchmark: zero deadlocks, zero DB errors.
* refactor(retain): remove legacy fallback code paths
- Remove process_entities_batch (legacy single-connection entity processing)
- Remove extract_entities_batch_optimized (only caller was the above)
- Remove fallback entity processing inside Phase 2 transaction
- Remove legacy ANN inline fallback in create_semantic_links_batch
- Remove fallback entity_links direct-insert path in Phase 3
- Make resolved_entity_ids/entity_to_unit/unit_to_entity_ids required params
* refactor(retain): replace tuple returns with dataclasses, remove dead code
- Add EntityResolutionResult and Phase1Result dataclasses in types.py
- Replace 4-tuple return from _pre_resolve_phase1 with Phase1Result
- Remove dead `entity_links = []` variables in retain_batch and _try_delta_retain
- Remove unused `confidence_score` parameter from orchestrator.retain_batch
and _retain_batch_async_internal (was accepted but never used)
* fix(entity-resolver): remove LIKE full-scan fallbacks, use index-only trigram matching
The entity resolution query had LIKE '%...' substring conditions that bypassed
the GIN trigram index, causing full sequential scans of the entities table.
On banks with 10k+ entities, this caused TimeoutErrors (observed in BEAM 10m).
Changes:
- Remove LIKE fallbacks, use trigram % operator only (GIN index-based)
- Lower similarity threshold from 0.3 to 0.15 to catch substring relationships
- Use LOWER() on both sides for case-insensitive matching
- Migration: recreate GIN trigram index on LOWER(canonical_name)
* fix: remove schema prefix from index names in trigram migration
* fix(delta-retain): use same chunk_size as streaming path (3000 vs 120000)
_chunk_contents_for_delta defaulted to chunk_size=120000 while the streaming
path used 3000. On retry, delta re-chunked the document with different
boundaries, found 0 matching chunks, and fell through to full re-extraction.
This wasted all LLM calls on already-committed chunks.
Fix: use the same default (3000) so chunk hashes match on recovery.
* fix(retain): persist generated document_id in operation metadata for retry recovery
When no document_id is provided, retain generates a UUID. On retry, a new UUID
was generated, making delta retain and streaming chunk-hash recovery unable to
find previously committed chunks. All LLM extraction was wasted on retry.
Fix: resolve document_id early in retain_batch (before delta), persist it to
operation result_metadata, and recover it on retry. Both delta and streaming
paths now see the same document_id across attempts.
* refactor(retain): unify into single streaming pipeline, remove non-streaming path
All retains now go through the producer-consumer streaming pipeline,
regardless of document size. Small documents are processed as a single batch.
This eliminates the maintenance burden of two separate code paths.
Also fix document upsert: compare content hash to distinguish recovery
(same content, partially committed) from update (different content, needs
cascade-delete). Previously, existing chunks always triggered recovery mode.
* refactor(retain): remove dead code, replace raw dicts with Phase3Context dataclass
- Remove dead _handle_zero_facts_documents (no callers after path unification)
- Remove unused imports: defaultdict, EntityLink
- Replace raw dict phase3_context with typed Phase3Context dataclass
- Update _build_and_insert_entity_links_phase3 to use typed parameter
1516 lines
74 KiB
Python
1516 lines
74 KiB
Python
"""
|
|
Centralized configuration for Hindsight API.
|
|
|
|
All environment variables and their defaults are defined here.
|
|
"""
|
|
|
|
import json
|
|
import logging
|
|
import os
|
|
import sys
|
|
from dataclasses import dataclass, field, fields
|
|
from datetime import datetime, timezone
|
|
from typing import Any
|
|
|
|
from dotenv import find_dotenv, load_dotenv
|
|
|
|
# Load .env file, searching current and parent directories (overrides existing env vars)
|
|
load_dotenv(find_dotenv(usecwd=True), override=True)
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
|
|
class ConfigFieldAccessError(AttributeError):
|
|
"""Raised when trying to access a bank-configurable field from global config."""
|
|
|
|
pass
|
|
|
|
|
|
class StaticConfigProxy:
|
|
"""
|
|
Proxy that wraps HindsightConfig and only allows access to static (non-configurable) fields.
|
|
|
|
Raises ConfigFieldAccessError when trying to access configurable fields that vary per-bank.
|
|
Forces developers to use get_resolved_config(bank_id, context) for bank-specific settings.
|
|
"""
|
|
|
|
def __init__(self, config: "HindsightConfig"):
|
|
object.__setattr__(self, "_config", config)
|
|
object.__setattr__(self, "_configurable_fields", HindsightConfig.get_configurable_fields())
|
|
|
|
def __getattribute__(self, name: str):
|
|
if name.startswith("_"):
|
|
return object.__getattribute__(self, name)
|
|
|
|
configurable_fields = object.__getattribute__(self, "_configurable_fields")
|
|
if name in configurable_fields:
|
|
raise ConfigFieldAccessError(
|
|
f"Field '{name}' is bank-configurable and cannot be accessed from global config. "
|
|
f"Use ConfigResolver.resolve_full_config(bank_id, context) to get bank-specific config. "
|
|
f"This prevents accidentally using global defaults when bank-specific overrides exist."
|
|
)
|
|
|
|
config = object.__getattribute__(self, "_config")
|
|
return getattr(config, name)
|
|
|
|
def __setattr__(self, name: str, value):
|
|
raise AttributeError("Config is read-only. Modifications must go through ConfigResolver.")
|
|
|
|
|
|
# Configuration field markers for hierarchical configuration
|
|
def hierarchical(default_value):
|
|
"""
|
|
Mark a config field as hierarchical (can be overridden per-tenant/bank).
|
|
|
|
Hierarchical fields can be customized at the tenant or bank level via database
|
|
configuration. Examples: LLM settings, retention parameters, retrieval settings.
|
|
"""
|
|
return field(default=default_value, metadata={"hierarchical": True})
|
|
|
|
|
|
def static(default_value):
|
|
"""
|
|
Mark a config field as static (server-level only, cannot be overridden).
|
|
|
|
Static fields are infrastructure-level settings that affect the entire server
|
|
and cannot vary per tenant or bank. Examples: database URL, API port, worker settings.
|
|
"""
|
|
return field(default=default_value, metadata={"hierarchical": False})
|
|
|
|
|
|
# Configuration key normalization utilities
|
|
def normalize_config_key(key: str) -> str:
|
|
"""
|
|
Convert environment variable format to Python field name format.
|
|
|
|
Examples:
|
|
HINDSIGHT_API_LLM_PROVIDER -> llm_provider
|
|
LLM_MODEL -> llm_model
|
|
llm_model -> llm_model (already normalized)
|
|
|
|
Args:
|
|
key: Environment variable name or Python field name
|
|
|
|
Returns:
|
|
Normalized Python field name (lowercase snake_case)
|
|
"""
|
|
if key.startswith("HINDSIGHT_API_"):
|
|
key = key[len("HINDSIGHT_API_") :]
|
|
return key.lower()
|
|
|
|
|
|
def normalize_config_dict(config: dict[str, Any]) -> dict[str, Any]:
|
|
"""
|
|
Normalize all keys in a config dict to Python field names.
|
|
|
|
Allows users to provide config overrides in either format:
|
|
- Python field format: {"llm_provider": "openai"}
|
|
- Env var format: {"HINDSIGHT_API_LLM_PROVIDER": "openai"}
|
|
|
|
Args:
|
|
config: Dict with env var or Python field names as keys
|
|
|
|
Returns:
|
|
Dict with all keys normalized to Python field names
|
|
"""
|
|
return {normalize_config_key(k): v for k, v in config.items()}
|
|
|
|
|
|
# Environment variable names
|
|
ENV_DATABASE_URL = "HINDSIGHT_API_DATABASE_URL"
|
|
ENV_MIGRATION_DATABASE_URL = "HINDSIGHT_API_MIGRATION_DATABASE_URL"
|
|
ENV_DATABASE_SCHEMA = "HINDSIGHT_API_DATABASE_SCHEMA"
|
|
ENV_LLM_PROVIDER = "HINDSIGHT_API_LLM_PROVIDER"
|
|
ENV_LLM_API_KEY = "HINDSIGHT_API_LLM_API_KEY"
|
|
ENV_LLM_MODEL = "HINDSIGHT_API_LLM_MODEL"
|
|
ENV_LLM_BASE_URL = "HINDSIGHT_API_LLM_BASE_URL"
|
|
ENV_LLM_MAX_CONCURRENT = "HINDSIGHT_API_LLM_MAX_CONCURRENT"
|
|
ENV_LLM_MAX_RETRIES = "HINDSIGHT_API_LLM_MAX_RETRIES"
|
|
ENV_LLM_INITIAL_BACKOFF = "HINDSIGHT_API_LLM_INITIAL_BACKOFF"
|
|
ENV_LLM_MAX_BACKOFF = "HINDSIGHT_API_LLM_MAX_BACKOFF"
|
|
ENV_LLM_TIMEOUT = "HINDSIGHT_API_LLM_TIMEOUT"
|
|
ENV_LLM_GROQ_SERVICE_TIER = "HINDSIGHT_API_LLM_GROQ_SERVICE_TIER"
|
|
ENV_LLM_OPENAI_SERVICE_TIER = "HINDSIGHT_API_LLM_OPENAI_SERVICE_TIER"
|
|
ENV_LLM_EXTRA_BODY = "HINDSIGHT_API_LLM_EXTRA_BODY"
|
|
|
|
# Defaults for service tiers
|
|
DEFAULT_LLM_GROQ_SERVICE_TIER = "auto" # "on_demand", "flex", or "auto"
|
|
DEFAULT_LLM_OPENAI_SERVICE_TIER = None # None (default) or "flex" (50% cheaper)
|
|
DEFAULT_LLM_EXTRA_BODY = None # None = no extra body params; JSON dict merged into OpenAI extra_body
|
|
|
|
# Per-operation LLM configuration (optional, falls back to global LLM config)
|
|
ENV_RETAIN_LLM_PROVIDER = "HINDSIGHT_API_RETAIN_LLM_PROVIDER"
|
|
ENV_RETAIN_LLM_API_KEY = "HINDSIGHT_API_RETAIN_LLM_API_KEY"
|
|
ENV_RETAIN_LLM_MODEL = "HINDSIGHT_API_RETAIN_LLM_MODEL"
|
|
ENV_RETAIN_LLM_BASE_URL = "HINDSIGHT_API_RETAIN_LLM_BASE_URL"
|
|
ENV_RETAIN_LLM_MAX_CONCURRENT = "HINDSIGHT_API_RETAIN_LLM_MAX_CONCURRENT"
|
|
ENV_RETAIN_LLM_MAX_RETRIES = "HINDSIGHT_API_RETAIN_LLM_MAX_RETRIES"
|
|
ENV_RETAIN_LLM_INITIAL_BACKOFF = "HINDSIGHT_API_RETAIN_LLM_INITIAL_BACKOFF"
|
|
ENV_RETAIN_LLM_MAX_BACKOFF = "HINDSIGHT_API_RETAIN_LLM_MAX_BACKOFF"
|
|
ENV_RETAIN_LLM_TIMEOUT = "HINDSIGHT_API_RETAIN_LLM_TIMEOUT"
|
|
|
|
ENV_REFLECT_LLM_PROVIDER = "HINDSIGHT_API_REFLECT_LLM_PROVIDER"
|
|
ENV_REFLECT_LLM_API_KEY = "HINDSIGHT_API_REFLECT_LLM_API_KEY"
|
|
ENV_REFLECT_LLM_MODEL = "HINDSIGHT_API_REFLECT_LLM_MODEL"
|
|
ENV_REFLECT_LLM_BASE_URL = "HINDSIGHT_API_REFLECT_LLM_BASE_URL"
|
|
ENV_REFLECT_LLM_MAX_CONCURRENT = "HINDSIGHT_API_REFLECT_LLM_MAX_CONCURRENT"
|
|
ENV_REFLECT_LLM_MAX_RETRIES = "HINDSIGHT_API_REFLECT_LLM_MAX_RETRIES"
|
|
ENV_REFLECT_LLM_INITIAL_BACKOFF = "HINDSIGHT_API_REFLECT_LLM_INITIAL_BACKOFF"
|
|
ENV_REFLECT_LLM_MAX_BACKOFF = "HINDSIGHT_API_REFLECT_LLM_MAX_BACKOFF"
|
|
ENV_REFLECT_LLM_TIMEOUT = "HINDSIGHT_API_REFLECT_LLM_TIMEOUT"
|
|
|
|
ENV_CONSOLIDATION_LLM_PROVIDER = "HINDSIGHT_API_CONSOLIDATION_LLM_PROVIDER"
|
|
ENV_CONSOLIDATION_LLM_API_KEY = "HINDSIGHT_API_CONSOLIDATION_LLM_API_KEY"
|
|
ENV_CONSOLIDATION_LLM_MODEL = "HINDSIGHT_API_CONSOLIDATION_LLM_MODEL"
|
|
ENV_CONSOLIDATION_LLM_BASE_URL = "HINDSIGHT_API_CONSOLIDATION_LLM_BASE_URL"
|
|
ENV_CONSOLIDATION_LLM_MAX_CONCURRENT = "HINDSIGHT_API_CONSOLIDATION_LLM_MAX_CONCURRENT"
|
|
ENV_CONSOLIDATION_LLM_MAX_RETRIES = "HINDSIGHT_API_CONSOLIDATION_LLM_MAX_RETRIES"
|
|
ENV_CONSOLIDATION_LLM_INITIAL_BACKOFF = "HINDSIGHT_API_CONSOLIDATION_LLM_INITIAL_BACKOFF"
|
|
ENV_CONSOLIDATION_LLM_MAX_BACKOFF = "HINDSIGHT_API_CONSOLIDATION_LLM_MAX_BACKOFF"
|
|
ENV_CONSOLIDATION_LLM_TIMEOUT = "HINDSIGHT_API_CONSOLIDATION_LLM_TIMEOUT"
|
|
|
|
ENV_EMBEDDINGS_PROVIDER = "HINDSIGHT_API_EMBEDDINGS_PROVIDER"
|
|
ENV_EMBEDDINGS_LOCAL_MODEL = "HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL"
|
|
ENV_EMBEDDINGS_LOCAL_FORCE_CPU = "HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU"
|
|
ENV_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE = "HINDSIGHT_API_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE"
|
|
ENV_EMBEDDINGS_TEI_URL = "HINDSIGHT_API_EMBEDDINGS_TEI_URL"
|
|
ENV_EMBEDDINGS_OPENAI_API_KEY = "HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY"
|
|
ENV_EMBEDDINGS_OPENAI_MODEL = "HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL"
|
|
ENV_EMBEDDINGS_OPENAI_BASE_URL = "HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL"
|
|
|
|
# Cohere configuration (separate for embeddings and reranker)
|
|
ENV_EMBEDDINGS_COHERE_API_KEY = "HINDSIGHT_API_EMBEDDINGS_COHERE_API_KEY"
|
|
ENV_EMBEDDINGS_COHERE_MODEL = "HINDSIGHT_API_EMBEDDINGS_COHERE_MODEL"
|
|
ENV_EMBEDDINGS_COHERE_BASE_URL = "HINDSIGHT_API_EMBEDDINGS_COHERE_BASE_URL"
|
|
ENV_RERANKER_COHERE_API_KEY = "HINDSIGHT_API_RERANKER_COHERE_API_KEY"
|
|
ENV_RERANKER_COHERE_MODEL = "HINDSIGHT_API_RERANKER_COHERE_MODEL"
|
|
ENV_RERANKER_COHERE_BASE_URL = "HINDSIGHT_API_RERANKER_COHERE_BASE_URL"
|
|
|
|
# Deprecated: Legacy shared Cohere API key (for backward compatibility)
|
|
ENV_COHERE_API_KEY = "HINDSIGHT_API_COHERE_API_KEY"
|
|
|
|
# LiteLLM configuration (separate for embeddings and reranker)
|
|
ENV_EMBEDDINGS_LITELLM_API_BASE = "HINDSIGHT_API_EMBEDDINGS_LITELLM_API_BASE"
|
|
ENV_EMBEDDINGS_LITELLM_API_KEY = "HINDSIGHT_API_EMBEDDINGS_LITELLM_API_KEY"
|
|
ENV_EMBEDDINGS_LITELLM_MODEL = "HINDSIGHT_API_EMBEDDINGS_LITELLM_MODEL"
|
|
ENV_RERANKER_LITELLM_API_BASE = "HINDSIGHT_API_RERANKER_LITELLM_API_BASE"
|
|
ENV_RERANKER_LITELLM_API_KEY = "HINDSIGHT_API_RERANKER_LITELLM_API_KEY"
|
|
ENV_RERANKER_LITELLM_MODEL = "HINDSIGHT_API_RERANKER_LITELLM_MODEL"
|
|
ENV_RERANKER_LITELLM_MAX_TOKENS_PER_DOC = "HINDSIGHT_API_RERANKER_LITELLM_MAX_TOKENS_PER_DOC"
|
|
|
|
# LiteLLM SDK configuration (direct API access, no proxy needed)
|
|
ENV_EMBEDDINGS_LITELLM_SDK_API_KEY = "HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_API_KEY"
|
|
ENV_EMBEDDINGS_LITELLM_SDK_MODEL = "HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_MODEL"
|
|
ENV_EMBEDDINGS_LITELLM_SDK_API_BASE = "HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_API_BASE"
|
|
ENV_EMBEDDINGS_LITELLM_SDK_OUTPUT_DIMENSIONS = "HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_OUTPUT_DIMENSIONS"
|
|
ENV_RERANKER_LITELLM_SDK_API_KEY = "HINDSIGHT_API_RERANKER_LITELLM_SDK_API_KEY"
|
|
ENV_RERANKER_LITELLM_SDK_MODEL = "HINDSIGHT_API_RERANKER_LITELLM_SDK_MODEL"
|
|
ENV_RERANKER_LITELLM_SDK_API_BASE = "HINDSIGHT_API_RERANKER_LITELLM_SDK_API_BASE"
|
|
|
|
# Deprecated: Legacy shared LiteLLM config (for backward compatibility)
|
|
ENV_LITELLM_API_BASE = "HINDSIGHT_API_LITELLM_API_BASE"
|
|
ENV_LITELLM_API_KEY = "HINDSIGHT_API_LITELLM_API_KEY"
|
|
|
|
ENV_RERANKER_PROVIDER = "HINDSIGHT_API_RERANKER_PROVIDER"
|
|
ENV_RERANKER_LOCAL_MODEL = "HINDSIGHT_API_RERANKER_LOCAL_MODEL"
|
|
ENV_RERANKER_LOCAL_FORCE_CPU = "HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU"
|
|
ENV_RERANKER_LOCAL_MAX_CONCURRENT = "HINDSIGHT_API_RERANKER_LOCAL_MAX_CONCURRENT"
|
|
ENV_RERANKER_LOCAL_TRUST_REMOTE_CODE = "HINDSIGHT_API_RERANKER_LOCAL_TRUST_REMOTE_CODE"
|
|
ENV_RERANKER_LOCAL_FP16 = "HINDSIGHT_API_RERANKER_LOCAL_FP16"
|
|
ENV_RERANKER_LOCAL_BUCKET_BATCHING = "HINDSIGHT_API_RERANKER_LOCAL_BUCKET_BATCHING"
|
|
ENV_RERANKER_LOCAL_BATCH_SIZE = "HINDSIGHT_API_RERANKER_LOCAL_BATCH_SIZE"
|
|
ENV_RERANKER_TEI_URL = "HINDSIGHT_API_RERANKER_TEI_URL"
|
|
ENV_RERANKER_TEI_BATCH_SIZE = "HINDSIGHT_API_RERANKER_TEI_BATCH_SIZE"
|
|
ENV_RERANKER_TEI_MAX_CONCURRENT = "HINDSIGHT_API_RERANKER_TEI_MAX_CONCURRENT"
|
|
ENV_RERANKER_MAX_CANDIDATES = "HINDSIGHT_API_RERANKER_MAX_CANDIDATES"
|
|
ENV_RERANKER_FLASHRANK_MODEL = "HINDSIGHT_API_RERANKER_FLASHRANK_MODEL"
|
|
ENV_RERANKER_FLASHRANK_CACHE_DIR = "HINDSIGHT_API_RERANKER_FLASHRANK_CACHE_DIR"
|
|
|
|
# ZeroEntropy configuration (reranker only)
|
|
ENV_RERANKER_ZEROENTROPY_API_KEY = "HINDSIGHT_API_RERANKER_ZEROENTROPY_API_KEY"
|
|
ENV_RERANKER_ZEROENTROPY_MODEL = "HINDSIGHT_API_RERANKER_ZEROENTROPY_MODEL"
|
|
ENV_RERANKER_ZEROENTROPY_BASE_URL = "HINDSIGHT_API_RERANKER_ZEROENTROPY_BASE_URL"
|
|
|
|
ENV_VECTOR_EXTENSION = "HINDSIGHT_API_VECTOR_EXTENSION"
|
|
ENV_TEXT_SEARCH_EXTENSION = "HINDSIGHT_API_TEXT_SEARCH_EXTENSION"
|
|
|
|
ENV_HOST = "HINDSIGHT_API_HOST"
|
|
ENV_PORT = "HINDSIGHT_API_PORT"
|
|
ENV_BASE_PATH = "HINDSIGHT_API_BASE_PATH"
|
|
ENV_LOG_LEVEL = "HINDSIGHT_API_LOG_LEVEL"
|
|
ENV_LOG_FORMAT = "HINDSIGHT_API_LOG_FORMAT"
|
|
ENV_WORKERS = "HINDSIGHT_API_WORKERS"
|
|
ENV_MCP_ENABLED = "HINDSIGHT_API_MCP_ENABLED"
|
|
ENV_MCP_ENABLED_TOOLS = "HINDSIGHT_API_MCP_ENABLED_TOOLS"
|
|
ENV_MCP_STATELESS = "HINDSIGHT_API_MCP_STATELESS"
|
|
ENV_ENABLE_BANK_CONFIG_API = "HINDSIGHT_API_ENABLE_BANK_CONFIG_API"
|
|
ENV_GRAPH_RETRIEVER = "HINDSIGHT_API_GRAPH_RETRIEVER"
|
|
ENV_RECALL_MAX_CONCURRENT = "HINDSIGHT_API_RECALL_MAX_CONCURRENT"
|
|
ENV_RECALL_CONNECTION_BUDGET = "HINDSIGHT_API_RECALL_CONNECTION_BUDGET"
|
|
ENV_RECALL_MAX_QUERY_TOKENS = "HINDSIGHT_API_RECALL_MAX_QUERY_TOKENS"
|
|
ENV_MENTAL_MODEL_REFRESH_CONCURRENCY = "HINDSIGHT_API_MENTAL_MODEL_REFRESH_CONCURRENCY"
|
|
|
|
# OpenTelemetry tracing configuration
|
|
ENV_OTEL_TRACES_ENABLED = "HINDSIGHT_API_OTEL_TRACES_ENABLED"
|
|
ENV_OTEL_EXPORTER_OTLP_ENDPOINT = "HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT"
|
|
ENV_OTEL_EXPORTER_OTLP_HEADERS = "HINDSIGHT_API_OTEL_EXPORTER_OTLP_HEADERS"
|
|
ENV_OTEL_SERVICE_NAME = "HINDSIGHT_API_OTEL_SERVICE_NAME"
|
|
ENV_OTEL_DEPLOYMENT_ENVIRONMENT = "HINDSIGHT_API_OTEL_DEPLOYMENT_ENVIRONMENT"
|
|
|
|
# Vertex AI configuration
|
|
ENV_LLM_VERTEXAI_PROJECT_ID = "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID"
|
|
ENV_LLM_VERTEXAI_REGION = "HINDSIGHT_API_LLM_VERTEXAI_REGION"
|
|
ENV_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY = "HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY"
|
|
|
|
# Gemini safety settings
|
|
ENV_LLM_GEMINI_SAFETY_SETTINGS = "HINDSIGHT_API_LLM_GEMINI_SAFETY_SETTINGS"
|
|
|
|
# Retain settings
|
|
ENV_RETAIN_MAX_COMPLETION_TOKENS = "HINDSIGHT_API_RETAIN_MAX_COMPLETION_TOKENS"
|
|
ENV_RETAIN_CHUNK_SIZE = "HINDSIGHT_API_RETAIN_CHUNK_SIZE"
|
|
ENV_RETAIN_EXTRACT_CAUSAL_LINKS = "HINDSIGHT_API_RETAIN_EXTRACT_CAUSAL_LINKS"
|
|
ENV_RETAIN_EXTRACTION_MODE = "HINDSIGHT_API_RETAIN_EXTRACTION_MODE"
|
|
ENV_RETAIN_MISSION = "HINDSIGHT_API_RETAIN_MISSION"
|
|
ENV_RETAIN_CUSTOM_INSTRUCTIONS = "HINDSIGHT_API_RETAIN_CUSTOM_INSTRUCTIONS"
|
|
ENV_RETAIN_DEFAULT_STRATEGY = "HINDSIGHT_API_RETAIN_DEFAULT_STRATEGY"
|
|
ENV_RETAIN_BATCH_TOKENS = "HINDSIGHT_API_RETAIN_BATCH_TOKENS"
|
|
ENV_RETAIN_ENTITY_LOOKUP = "HINDSIGHT_API_RETAIN_ENTITY_LOOKUP"
|
|
ENV_RETAIN_BATCH_ENABLED = "HINDSIGHT_API_RETAIN_BATCH_ENABLED"
|
|
ENV_RETAIN_BATCH_POLL_INTERVAL_SECONDS = "HINDSIGHT_API_RETAIN_BATCH_POLL_INTERVAL_SECONDS"
|
|
ENV_RETAIN_CHUNK_BATCH_SIZE = "HINDSIGHT_API_RETAIN_CHUNK_BATCH_SIZE"
|
|
|
|
# File storage configuration
|
|
ENV_FILE_STORAGE_TYPE = "HINDSIGHT_API_FILE_STORAGE_TYPE"
|
|
ENV_FILE_STORAGE_S3_BUCKET = "HINDSIGHT_API_FILE_STORAGE_S3_BUCKET"
|
|
ENV_FILE_STORAGE_S3_REGION = "HINDSIGHT_API_FILE_STORAGE_S3_REGION"
|
|
ENV_FILE_STORAGE_S3_ENDPOINT = "HINDSIGHT_API_FILE_STORAGE_S3_ENDPOINT"
|
|
ENV_FILE_STORAGE_S3_ACCESS_KEY_ID = "HINDSIGHT_API_FILE_STORAGE_S3_ACCESS_KEY_ID"
|
|
ENV_FILE_STORAGE_S3_SECRET_ACCESS_KEY = "HINDSIGHT_API_FILE_STORAGE_S3_SECRET_ACCESS_KEY"
|
|
ENV_FILE_STORAGE_GCS_BUCKET = "HINDSIGHT_API_FILE_STORAGE_GCS_BUCKET"
|
|
ENV_FILE_STORAGE_GCS_SERVICE_ACCOUNT_KEY = "HINDSIGHT_API_FILE_STORAGE_GCS_SERVICE_ACCOUNT_KEY"
|
|
ENV_FILE_STORAGE_AZURE_CONTAINER = "HINDSIGHT_API_FILE_STORAGE_AZURE_CONTAINER"
|
|
ENV_FILE_STORAGE_AZURE_ACCOUNT_NAME = "HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_NAME"
|
|
ENV_FILE_STORAGE_AZURE_ACCOUNT_KEY = "HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_KEY"
|
|
ENV_FILE_PARSER = "HINDSIGHT_API_FILE_PARSER"
|
|
ENV_FILE_PARSER_ALLOWLIST = "HINDSIGHT_API_FILE_PARSER_ALLOWLIST"
|
|
ENV_FILE_PARSER_IRIS_TOKEN = "HINDSIGHT_API_FILE_PARSER_IRIS_TOKEN"
|
|
ENV_FILE_PARSER_IRIS_ORG_ID = "HINDSIGHT_API_FILE_PARSER_IRIS_ORG_ID"
|
|
ENV_FILE_CONVERSION_MAX_BATCH_SIZE_MB = "HINDSIGHT_API_FILE_CONVERSION_MAX_BATCH_SIZE_MB"
|
|
ENV_FILE_CONVERSION_MAX_BATCH_SIZE = "HINDSIGHT_API_FILE_CONVERSION_MAX_BATCH_SIZE"
|
|
ENV_ENABLE_FILE_UPLOAD_API = "HINDSIGHT_API_ENABLE_FILE_UPLOAD_API"
|
|
ENV_FILE_DELETE_AFTER_RETAIN = "HINDSIGHT_API_FILE_DELETE_AFTER_RETAIN"
|
|
|
|
# Observations settings (consolidated knowledge from facts)
|
|
ENV_ENABLE_OBSERVATIONS = "HINDSIGHT_API_ENABLE_OBSERVATIONS"
|
|
ENV_CONSOLIDATION_BATCH_SIZE = "HINDSIGHT_API_CONSOLIDATION_BATCH_SIZE"
|
|
ENV_CONSOLIDATION_LLM_BATCH_SIZE = "HINDSIGHT_API_CONSOLIDATION_LLM_BATCH_SIZE"
|
|
ENV_CONSOLIDATION_MAX_TOKENS = "HINDSIGHT_API_CONSOLIDATION_MAX_TOKENS"
|
|
ENV_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS = "HINDSIGHT_API_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS"
|
|
ENV_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS_PER_OBSERVATION = (
|
|
"HINDSIGHT_API_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS_PER_OBSERVATION"
|
|
)
|
|
ENV_OBSERVATIONS_MISSION = "HINDSIGHT_API_OBSERVATIONS_MISSION"
|
|
ENV_MAX_OBSERVATIONS_PER_SCOPE = "HINDSIGHT_API_MAX_OBSERVATIONS_PER_SCOPE"
|
|
ENV_ENABLE_OBSERVATION_HISTORY = "HINDSIGHT_API_ENABLE_OBSERVATION_HISTORY"
|
|
ENV_ENABLE_MENTAL_MODEL_HISTORY = "HINDSIGHT_API_ENABLE_MENTAL_MODEL_HISTORY"
|
|
|
|
# Webhook configuration (global, static - server-level only)
|
|
ENV_WEBHOOK_URL = "HINDSIGHT_API_WEBHOOK_URL"
|
|
ENV_WEBHOOK_SECRET = "HINDSIGHT_API_WEBHOOK_SECRET"
|
|
ENV_WEBHOOK_EVENT_TYPES = "HINDSIGHT_API_WEBHOOK_EVENT_TYPES"
|
|
ENV_WEBHOOK_DELIVERY_POLL_INTERVAL_SECONDS = "HINDSIGHT_API_WEBHOOK_DELIVERY_POLL_INTERVAL_SECONDS"
|
|
|
|
# Optimization flags
|
|
ENV_SKIP_LLM_VERIFICATION = "HINDSIGHT_API_SKIP_LLM_VERIFICATION"
|
|
ENV_LAZY_RERANKER = "HINDSIGHT_API_LAZY_RERANKER"
|
|
|
|
# Database migrations
|
|
ENV_RUN_MIGRATIONS_ON_STARTUP = "HINDSIGHT_API_RUN_MIGRATIONS_ON_STARTUP"
|
|
|
|
# Database connection pool
|
|
ENV_DB_POOL_MIN_SIZE = "HINDSIGHT_API_DB_POOL_MIN_SIZE"
|
|
ENV_DB_POOL_MAX_SIZE = "HINDSIGHT_API_DB_POOL_MAX_SIZE"
|
|
ENV_DB_COMMAND_TIMEOUT = "HINDSIGHT_API_DB_COMMAND_TIMEOUT"
|
|
ENV_DB_ACQUIRE_TIMEOUT = "HINDSIGHT_API_DB_ACQUIRE_TIMEOUT"
|
|
|
|
# Worker configuration (distributed task processing)
|
|
ENV_WORKER_ENABLED = "HINDSIGHT_API_WORKER_ENABLED"
|
|
ENV_WORKER_ID = "HINDSIGHT_API_WORKER_ID"
|
|
ENV_WORKER_POLL_INTERVAL_MS = "HINDSIGHT_API_WORKER_POLL_INTERVAL_MS"
|
|
ENV_WORKER_MAX_RETRIES = "HINDSIGHT_API_WORKER_MAX_RETRIES"
|
|
ENV_WORKER_HTTP_PORT = "HINDSIGHT_API_WORKER_HTTP_PORT"
|
|
ENV_WORKER_MAX_SLOTS = "HINDSIGHT_API_WORKER_MAX_SLOTS"
|
|
ENV_WORKER_CONSOLIDATION_MAX_SLOTS = "HINDSIGHT_API_WORKER_CONSOLIDATION_MAX_SLOTS"
|
|
ENV_RETAIN_MAX_CONCURRENT = "HINDSIGHT_API_RETAIN_MAX_CONCURRENT"
|
|
|
|
# Reflect agent settings
|
|
ENV_REFLECT_MAX_ITERATIONS = "HINDSIGHT_API_REFLECT_MAX_ITERATIONS"
|
|
ENV_REFLECT_MAX_CONTEXT_TOKENS = "HINDSIGHT_API_REFLECT_MAX_CONTEXT_TOKENS"
|
|
ENV_REFLECT_WALL_TIMEOUT = "HINDSIGHT_API_REFLECT_WALL_TIMEOUT"
|
|
ENV_REFLECT_MISSION = "HINDSIGHT_API_REFLECT_MISSION"
|
|
ENV_REFLECT_SOURCE_FACTS_MAX_TOKENS = "HINDSIGHT_API_REFLECT_SOURCE_FACTS_MAX_TOKENS"
|
|
|
|
# Audit log settings
|
|
ENV_AUDIT_LOG_ENABLED = "HINDSIGHT_API_AUDIT_LOG_ENABLED"
|
|
ENV_AUDIT_LOG_ACTIONS = "HINDSIGHT_API_AUDIT_LOG_ACTIONS"
|
|
ENV_AUDIT_LOG_RETENTION_DAYS = "HINDSIGHT_API_AUDIT_LOG_RETENTION_DAYS"
|
|
|
|
# Disposition settings
|
|
ENV_DISPOSITION_SKEPTICISM = "HINDSIGHT_API_DISPOSITION_SKEPTICISM"
|
|
ENV_DISPOSITION_LITERALISM = "HINDSIGHT_API_DISPOSITION_LITERALISM"
|
|
ENV_DISPOSITION_EMPATHY = "HINDSIGHT_API_DISPOSITION_EMPATHY"
|
|
|
|
# Default values
|
|
DEFAULT_DATABASE_URL = "pg0"
|
|
DEFAULT_DATABASE_SCHEMA = "public"
|
|
DEFAULT_LLM_PROVIDER = "openai"
|
|
|
|
# Provider-specific default models
|
|
PROVIDER_DEFAULT_MODELS = {
|
|
"openai": "gpt-4o-mini",
|
|
"anthropic": "claude-haiku-4-5",
|
|
"gemini": "gemini-2.5-flash",
|
|
"groq": "openai/gpt-oss-120b",
|
|
"minimax": "MiniMax-M2.7",
|
|
"ollama": "gemma3:12b",
|
|
"lmstudio": "local-model",
|
|
"vertexai": "google/gemini-2.5-flash-lite",
|
|
"openai-codex": "gpt-5.2-codex",
|
|
"claude-code": "claude-sonnet-4-5-20250929",
|
|
"mock": "mock-model",
|
|
"none": "none",
|
|
"litellm": "gpt-4o-mini",
|
|
"bedrock": "us.amazon.nova-2-lite-v1:0",
|
|
"volcano": "doubao-pro-32k",
|
|
}
|
|
DEFAULT_LLM_MODEL = "gpt-4o-mini" # Fallback if provider not in table
|
|
DEFAULT_LLM_MAX_CONCURRENT = 32
|
|
DEFAULT_LLM_MAX_RETRIES = 10 # Max retry attempts for LLM API calls
|
|
DEFAULT_LLM_INITIAL_BACKOFF = 1.0 # Initial backoff in seconds for retry exponential backoff
|
|
DEFAULT_LLM_MAX_BACKOFF = 60.0 # Max backoff cap in seconds for retry exponential backoff
|
|
DEFAULT_LLM_TIMEOUT = 120.0 # seconds
|
|
|
|
# Vertex AI defaults
|
|
DEFAULT_LLM_VERTEXAI_PROJECT_ID = None # Required for Vertex AI
|
|
DEFAULT_LLM_VERTEXAI_REGION = "us-central1"
|
|
DEFAULT_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY = None # Optional, uses ADC if not set
|
|
|
|
# Gemini safety settings defaults
|
|
DEFAULT_LLM_GEMINI_SAFETY_SETTINGS = None # None = use Gemini default safety settings
|
|
|
|
DEFAULT_EMBEDDINGS_PROVIDER = "local"
|
|
DEFAULT_EMBEDDINGS_LOCAL_MODEL = "BAAI/bge-small-en-v1.5"
|
|
DEFAULT_EMBEDDINGS_LOCAL_FORCE_CPU = False # Force CPU mode for local embeddings (avoids MPS/XPC issues on macOS)
|
|
DEFAULT_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE = False # Security: disabled by default, required for some models
|
|
DEFAULT_EMBEDDINGS_OPENAI_MODEL = "text-embedding-3-small"
|
|
DEFAULT_EMBEDDING_DIMENSION = 384
|
|
|
|
DEFAULT_RERANKER_PROVIDER = "local"
|
|
DEFAULT_RERANKER_LOCAL_MODEL = "cross-encoder/ms-marco-MiniLM-L-6-v2"
|
|
DEFAULT_RERANKER_LOCAL_FORCE_CPU = False # Force CPU mode for local reranker (avoids MPS/XPC issues on macOS)
|
|
DEFAULT_RERANKER_LOCAL_MAX_CONCURRENT = 4 # Limit concurrent CPU-bound reranking to prevent thrashing
|
|
DEFAULT_RERANKER_LOCAL_TRUST_REMOTE_CODE = (
|
|
False # Security: disabled by default, required for some models like jina-reranker-v2
|
|
)
|
|
DEFAULT_RERANKER_LOCAL_FP16 = False # FP16 inference: opt-in, faster on MPS/CUDA (not CPU)
|
|
DEFAULT_RERANKER_LOCAL_BUCKET_BATCHING = False # Length-sorted bucket batching: opt-in, 36-54% speedup
|
|
DEFAULT_RERANKER_LOCAL_BATCH_SIZE = 32 # Batch size for local reranker predict() calls
|
|
DEFAULT_RERANKER_TEI_BATCH_SIZE = 128
|
|
DEFAULT_RERANKER_TEI_MAX_CONCURRENT = 8
|
|
DEFAULT_RERANKER_MAX_CANDIDATES = 300
|
|
DEFAULT_RERANKER_FLASHRANK_MODEL = "ms-marco-MiniLM-L-12-v2" # Best balance of speed and quality
|
|
DEFAULT_RERANKER_FLASHRANK_CACHE_DIR = None # Use default cache directory
|
|
|
|
DEFAULT_EMBEDDINGS_COHERE_MODEL = "embed-english-v3.0"
|
|
DEFAULT_RERANKER_COHERE_MODEL = "rerank-english-v3.0"
|
|
|
|
DEFAULT_RERANKER_ZEROENTROPY_MODEL = "zerank-2"
|
|
|
|
# Vector extension (pgvector, vchord, or pgvectorscale)
|
|
DEFAULT_VECTOR_EXTENSION = "pgvector" # Options: "pgvector", "vchord", "pgvectorscale"
|
|
|
|
# Text search extension (native PostgreSQL, vchord BM25, or Timescale pg_textsearch)
|
|
DEFAULT_TEXT_SEARCH_EXTENSION = "native" # Options: "native", "vchord", "pg_textsearch"
|
|
|
|
# LiteLLM defaults
|
|
DEFAULT_LITELLM_API_BASE = "http://localhost:4000"
|
|
DEFAULT_EMBEDDINGS_LITELLM_MODEL = "text-embedding-3-small"
|
|
DEFAULT_RERANKER_LITELLM_MODEL = "cohere/rerank-english-v3.0"
|
|
DEFAULT_RERANKER_LITELLM_MAX_TOKENS_PER_DOC: int | None = None
|
|
|
|
# LiteLLM SDK defaults
|
|
DEFAULT_EMBEDDINGS_LITELLM_SDK_MODEL = "cohere/embed-english-v3.0"
|
|
DEFAULT_RERANKER_LITELLM_SDK_MODEL = "cohere/rerank-english-v3.0"
|
|
|
|
DEFAULT_HOST = "0.0.0.0"
|
|
DEFAULT_PORT = 8888
|
|
DEFAULT_BASE_PATH = "" # Empty string = root path
|
|
DEFAULT_LOG_LEVEL = "info"
|
|
DEFAULT_LOG_FORMAT = "text" # Options: "text", "json"
|
|
DEFAULT_WORKERS = 1
|
|
DEFAULT_MCP_ENABLED = True
|
|
DEFAULT_MCP_ENABLED_TOOLS: list[str] | None = None # None = all tools enabled
|
|
DEFAULT_MCP_STATELESS = False # False = stateful (supports SSE/GET); True = stateless (POST-only)
|
|
DEFAULT_ENABLE_BANK_CONFIG_API = True
|
|
DEFAULT_GRAPH_RETRIEVER = "link_expansion"
|
|
DEFAULT_RECALL_MAX_CONCURRENT = 32 # Max concurrent recall operations per worker
|
|
DEFAULT_RECALL_CONNECTION_BUDGET = 4 # Max concurrent DB connections per recall operation
|
|
DEFAULT_RECALL_MAX_QUERY_TOKENS = 500 # Maximum tokens allowed in recall query
|
|
DEFAULT_MENTAL_MODEL_REFRESH_CONCURRENCY = 8 # Max concurrent mental model refreshes
|
|
|
|
# Retain settings
|
|
DEFAULT_RETAIN_MAX_COMPLETION_TOKENS = 64000 # Max tokens for fact extraction LLM call
|
|
DEFAULT_RETAIN_CHUNK_SIZE = 3000 # Max chars per chunk for fact extraction
|
|
DEFAULT_RETAIN_EXTRACT_CAUSAL_LINKS = True # Extract causal links between facts
|
|
DEFAULT_RETAIN_EXTRACTION_MODE = "concise" # Extraction mode: "concise", "verbose", or "custom"
|
|
RETAIN_EXTRACTION_MODES = ("concise", "verbose", "custom", "verbatim", "chunks") # Allowed extraction modes
|
|
DEFAULT_RETAIN_MISSION = None # Declarative spec of what to retain (injected into any extraction mode)
|
|
DEFAULT_RETAIN_CUSTOM_INSTRUCTIONS = None # Custom extraction guidelines (only used when mode="custom")
|
|
DEFAULT_RETAIN_DEFAULT_STRATEGY = None # Default strategy name (None = no strategy override)
|
|
DEFAULT_RETAIN_STRATEGIES: dict | None = None # Named retain strategies (dict of name → config overrides)
|
|
DEFAULT_RETAIN_CHUNK_BATCH_SIZE = (
|
|
100 # Max chunks per streaming batch. Each chunk produces ~17 facts, so 100 chunks = ~1700 facts/batch.
|
|
)
|
|
DEFAULT_RETAIN_BATCH_TOKENS = 10_000 # ~40KB of text # Max chars per sub-batch for async retain auto-splitting
|
|
DEFAULT_RETAIN_ENTITY_LOOKUP = "trigram" # "full" or "trigram"
|
|
DEFAULT_RETAIN_BATCH_ENABLED = False # Use LLM Batch API for fact extraction (only when async=True)
|
|
DEFAULT_RETAIN_BATCH_POLL_INTERVAL_SECONDS = 60 # Batch API polling interval in seconds
|
|
|
|
# File storage defaults
|
|
DEFAULT_FILE_STORAGE_TYPE = "native" # PostgreSQL BYTEA storage
|
|
DEFAULT_FILE_PARSER = "markitdown" # Default parser fallback chain (comma-separated, e.g. "iris,markitdown")
|
|
DEFAULT_FILE_PARSER_ALLOWLIST = None # Allowlist of parsers clients may request (None = all registered parsers)
|
|
DEFAULT_FILE_CONVERSION_MAX_BATCH_SIZE_MB = 100 # Max total batch size in MB (all files combined)
|
|
DEFAULT_FILE_CONVERSION_MAX_BATCH_SIZE = 10 # Max files per batch upload
|
|
DEFAULT_ENABLE_FILE_UPLOAD_API = True # Enable file upload endpoint
|
|
DEFAULT_FILE_DELETE_AFTER_RETAIN = True # Delete file bytes after retain (saves storage)
|
|
|
|
# Observations defaults (consolidated knowledge from facts)
|
|
DEFAULT_ENABLE_OBSERVATIONS = True # Observations enabled by default
|
|
DEFAULT_ENABLE_OBSERVATION_HISTORY = True # Observation history tracking enabled by default
|
|
DEFAULT_ENABLE_MENTAL_MODEL_HISTORY = True # Mental model history tracking enabled by default
|
|
DEFAULT_CONSOLIDATION_BATCH_SIZE = 50 # Memories to load per batch (internal memory optimization)
|
|
DEFAULT_CONSOLIDATION_LLM_BATCH_SIZE = 8 # Facts per LLM call (1 = no batching; >1 = batch mode)
|
|
DEFAULT_CONSOLIDATION_MAX_TOKENS = 512 # Max tokens for recall when finding related observations
|
|
DEFAULT_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS = (
|
|
-1
|
|
) # Total token budget for source facts in consolidation recall (-1 = unlimited)
|
|
DEFAULT_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS_PER_OBSERVATION = (
|
|
256 # Max tokens of source facts per observation in consolidation prompt (-1 = unlimited)
|
|
)
|
|
DEFAULT_OBSERVATIONS_MISSION = None # Declarative spec of what observations are for this bank
|
|
DEFAULT_MAX_OBSERVATIONS_PER_SCOPE = -1 # Max observations per tag scope (-1 = unlimited)
|
|
|
|
# Database migrations
|
|
DEFAULT_RUN_MIGRATIONS_ON_STARTUP = True
|
|
|
|
# Database connection pool
|
|
DEFAULT_DB_POOL_MIN_SIZE = 5
|
|
DEFAULT_DB_POOL_MAX_SIZE = 100
|
|
DEFAULT_DB_COMMAND_TIMEOUT = 60 # seconds
|
|
DEFAULT_DB_ACQUIRE_TIMEOUT = 30 # seconds
|
|
|
|
# Worker configuration (distributed task processing)
|
|
DEFAULT_WORKER_ENABLED = True # API runs worker by default (standalone mode)
|
|
DEFAULT_WORKER_ID = None # Will use hostname if not specified
|
|
DEFAULT_WORKER_POLL_INTERVAL_MS = 500 # Poll database every 500ms
|
|
DEFAULT_WORKER_MAX_RETRIES = 3 # Max retries before marking task failed
|
|
DEFAULT_WORKER_HTTP_PORT = 8889 # HTTP port for worker metrics/health
|
|
DEFAULT_WORKER_MAX_SLOTS = 10 # Total concurrent tasks per worker
|
|
DEFAULT_WORKER_CONSOLIDATION_MAX_SLOTS = 2 # Max concurrent consolidation tasks per worker
|
|
DEFAULT_RETAIN_MAX_CONCURRENT = 4 # Max concurrent retain DB phases (HNSW reads + writes). Limits I/O contention.
|
|
|
|
# Reflect agent settings
|
|
DEFAULT_REFLECT_MAX_ITERATIONS = 10 # Max tool call iterations before forcing response
|
|
DEFAULT_REFLECT_MAX_CONTEXT_TOKENS = 100_000 # Max accumulated context tokens before forcing final prompt
|
|
DEFAULT_REFLECT_WALL_TIMEOUT = 300 # Wall-clock timeout in seconds for the entire reflect operation (5 minutes)
|
|
DEFAULT_REFLECT_SOURCE_FACTS_MAX_TOKENS = -1 # Token budget for source facts in search_observations (-1 = disabled)
|
|
|
|
# Disposition defaults (None = not set, fall back to bank DB value or 3)
|
|
DEFAULT_DISPOSITION_SKEPTICISM = None
|
|
DEFAULT_DISPOSITION_LITERALISM = None
|
|
DEFAULT_DISPOSITION_EMPATHY = None
|
|
|
|
# OpenTelemetry tracing configuration
|
|
DEFAULT_OTEL_TRACES_ENABLED = False # Disabled by default for backward compatibility
|
|
DEFAULT_OTEL_SERVICE_NAME = "hindsight-api"
|
|
DEFAULT_OTEL_DEPLOYMENT_ENVIRONMENT = "development"
|
|
|
|
# Audit log defaults
|
|
DEFAULT_AUDIT_LOG_ENABLED = False # Disabled by default
|
|
DEFAULT_AUDIT_LOG_ACTIONS = "" # Empty = audit all eligible actions
|
|
DEFAULT_AUDIT_LOG_RETENTION_DAYS = -1 # -1 = keep forever
|
|
|
|
# Default MCP tool descriptions (can be customized via env vars)
|
|
DEFAULT_MCP_RETAIN_DESCRIPTION = """Store important information to long-term memory.
|
|
|
|
Use this tool PROACTIVELY whenever the user shares:
|
|
- Personal facts, preferences, or interests
|
|
- Important events or milestones
|
|
- User history, experiences, or background
|
|
- Decisions, opinions, or stated preferences
|
|
- Goals, plans, or future intentions
|
|
- Relationships or people mentioned
|
|
- Work context, projects, or responsibilities"""
|
|
|
|
DEFAULT_MCP_RECALL_DESCRIPTION = """Search memories to provide personalized, context-aware responses.
|
|
|
|
Use this tool PROACTIVELY to:
|
|
- Check user's preferences before making suggestions
|
|
- Recall user's history to provide continuity
|
|
- Remember user's goals and context
|
|
- Personalize responses based on past interactions"""
|
|
|
|
# Default embedding dimension (used by initial migration, adjusted at runtime)
|
|
EMBEDDING_DIMENSION = DEFAULT_EMBEDDING_DIMENSION
|
|
|
|
# Webhook configuration defaults
|
|
DEFAULT_WEBHOOK_URL = None # None = no global webhook configured
|
|
DEFAULT_WEBHOOK_SECRET = None # None = no signing
|
|
DEFAULT_WEBHOOK_EVENT_TYPES = "consolidation.completed" # Comma-separated; default = all supported events
|
|
DEFAULT_WEBHOOK_DELIVERY_POLL_INTERVAL_SECONDS = 30 # How often to poll for pending deliveries
|
|
|
|
|
|
class JsonFormatter(logging.Formatter):
|
|
"""JSON formatter for structured logging.
|
|
|
|
Outputs logs in JSON format with a 'severity' field that cloud logging
|
|
systems (GCP, AWS CloudWatch, etc.) can parse to correctly categorize log levels.
|
|
"""
|
|
|
|
SEVERITY_MAP = {
|
|
logging.DEBUG: "DEBUG",
|
|
logging.INFO: "INFO",
|
|
logging.WARNING: "WARNING",
|
|
logging.ERROR: "ERROR",
|
|
logging.CRITICAL: "CRITICAL",
|
|
}
|
|
|
|
def format(self, record: logging.LogRecord) -> str:
|
|
log_entry = {
|
|
"severity": self.SEVERITY_MAP.get(record.levelno, "DEFAULT"),
|
|
"message": record.getMessage(),
|
|
"timestamp": datetime.now(timezone.utc).isoformat(),
|
|
"logger": record.name,
|
|
}
|
|
|
|
# Add exception info if present
|
|
if record.exc_info:
|
|
log_entry["exception"] = self.formatException(record.exc_info)
|
|
|
|
return json.dumps(log_entry)
|
|
|
|
|
|
def _parse_str_list(value: str) -> list[str]:
|
|
"""Parse a comma-separated string into a non-empty list of stripped tokens."""
|
|
return [v.strip() for v in value.split(",") if v.strip()]
|
|
|
|
|
|
def _validate_extraction_mode(mode: str) -> str:
|
|
"""Validate and normalize extraction mode."""
|
|
mode_lower = mode.lower()
|
|
if mode_lower not in RETAIN_EXTRACTION_MODES:
|
|
logger.warning(
|
|
f"Invalid extraction mode '{mode}', must be one of {RETAIN_EXTRACTION_MODES}. "
|
|
f"Defaulting to '{DEFAULT_RETAIN_EXTRACTION_MODE}'."
|
|
)
|
|
return DEFAULT_RETAIN_EXTRACTION_MODE
|
|
return mode_lower
|
|
|
|
|
|
def _get_default_model_for_provider(provider: str) -> str:
|
|
"""Get the default model for a given provider."""
|
|
return PROVIDER_DEFAULT_MODELS.get(provider.lower(), DEFAULT_LLM_MODEL)
|
|
|
|
|
|
@dataclass
|
|
class HindsightConfig:
|
|
"""Configuration container for Hindsight API."""
|
|
|
|
# Database
|
|
database_url: str
|
|
migration_database_url: str | None
|
|
database_schema: str
|
|
vector_extension: str # "pgvector" or "vchord"
|
|
text_search_extension: str # "native" or "vchord"
|
|
|
|
# LLM (default, used as fallback for per-operation config)
|
|
llm_provider: str
|
|
llm_api_key: str | None
|
|
llm_model: str
|
|
llm_base_url: str | None
|
|
llm_max_concurrent: int
|
|
llm_max_retries: int
|
|
llm_initial_backoff: float
|
|
llm_max_backoff: float
|
|
llm_timeout: float
|
|
llm_groq_service_tier: str # Groq: "on_demand", "flex", or "auto"
|
|
llm_openai_service_tier: str | None # OpenAI: None (default) or "flex" (50% cheaper)
|
|
llm_extra_body: (
|
|
dict | None
|
|
) # Extra body params merged into OpenAI-compatible API calls (e.g. {"chat_template_kwargs": {"enable_thinking": true}})
|
|
|
|
# Vertex AI configuration
|
|
llm_vertexai_project_id: str | None
|
|
llm_vertexai_region: str
|
|
llm_vertexai_service_account_key: str | None
|
|
|
|
# Gemini safety settings (None = use Gemini defaults; list of dicts with category/threshold)
|
|
llm_gemini_safety_settings: list | None
|
|
|
|
# Per-operation LLM configuration (None = use default LLM config)
|
|
retain_llm_provider: str | None
|
|
retain_llm_api_key: str | None
|
|
retain_llm_model: str | None
|
|
retain_llm_base_url: str | None
|
|
retain_llm_max_concurrent: int | None
|
|
retain_llm_max_retries: int | None
|
|
retain_llm_initial_backoff: float | None
|
|
retain_llm_max_backoff: float | None
|
|
retain_llm_timeout: float | None
|
|
|
|
reflect_llm_provider: str | None
|
|
reflect_llm_api_key: str | None
|
|
reflect_llm_model: str | None
|
|
reflect_llm_base_url: str | None
|
|
reflect_llm_max_concurrent: int | None
|
|
reflect_llm_max_retries: int | None
|
|
reflect_llm_initial_backoff: float | None
|
|
reflect_llm_max_backoff: float | None
|
|
reflect_llm_timeout: float | None
|
|
|
|
consolidation_llm_provider: str | None
|
|
consolidation_llm_api_key: str | None
|
|
consolidation_llm_model: str | None
|
|
consolidation_llm_base_url: str | None
|
|
consolidation_llm_max_concurrent: int | None
|
|
consolidation_llm_max_retries: int | None
|
|
consolidation_llm_initial_backoff: float | None
|
|
consolidation_llm_max_backoff: float | None
|
|
consolidation_llm_timeout: float | None
|
|
|
|
# Embeddings
|
|
embeddings_provider: str
|
|
embeddings_local_model: str
|
|
embeddings_local_force_cpu: bool
|
|
embeddings_local_trust_remote_code: bool
|
|
embeddings_tei_url: str | None
|
|
embeddings_openai_base_url: str | None
|
|
embeddings_cohere_api_key: str | None
|
|
embeddings_cohere_model: str
|
|
embeddings_cohere_base_url: str | None
|
|
embeddings_litellm_api_base: str
|
|
embeddings_litellm_api_key: str | None
|
|
embeddings_litellm_model: str
|
|
embeddings_litellm_sdk_api_key: str | None
|
|
embeddings_litellm_sdk_model: str
|
|
embeddings_litellm_sdk_api_base: str | None
|
|
embeddings_litellm_sdk_output_dimensions: int | None
|
|
|
|
# Reranker
|
|
reranker_provider: str
|
|
reranker_local_model: str
|
|
reranker_local_force_cpu: bool
|
|
reranker_local_max_concurrent: int
|
|
reranker_local_trust_remote_code: bool
|
|
reranker_local_fp16: bool
|
|
reranker_local_bucket_batching: bool
|
|
reranker_local_batch_size: int
|
|
reranker_tei_url: str | None
|
|
reranker_tei_batch_size: int
|
|
reranker_tei_max_concurrent: int
|
|
reranker_max_candidates: int
|
|
reranker_cohere_api_key: str | None
|
|
reranker_cohere_model: str
|
|
reranker_cohere_base_url: str | None
|
|
reranker_litellm_api_base: str
|
|
reranker_litellm_api_key: str | None
|
|
reranker_litellm_model: str
|
|
reranker_litellm_max_tokens_per_doc: int | None
|
|
reranker_litellm_sdk_api_key: str | None
|
|
reranker_litellm_sdk_model: str
|
|
reranker_litellm_sdk_api_base: str | None
|
|
reranker_zeroentropy_api_key: str | None
|
|
reranker_zeroentropy_model: str
|
|
reranker_zeroentropy_base_url: str | None
|
|
|
|
# Server
|
|
host: str
|
|
port: int
|
|
base_path: str
|
|
log_level: str
|
|
log_format: str
|
|
mcp_enabled: bool
|
|
mcp_enabled_tools: list[str] | None # None = all tools; explicit list = allowlist
|
|
mcp_stateless: bool # True = stateless HTTP (POST-only); False = stateful (supports GET/SSE)
|
|
enable_bank_config_api: bool
|
|
|
|
# Recall
|
|
graph_retriever: str
|
|
recall_max_concurrent: int
|
|
recall_connection_budget: int
|
|
recall_max_query_tokens: int
|
|
mental_model_refresh_concurrency: int
|
|
|
|
# Retain settings
|
|
retain_max_completion_tokens: int
|
|
retain_chunk_size: int
|
|
retain_extract_causal_links: bool
|
|
retain_extraction_mode: str
|
|
retain_mission: str | None
|
|
retain_custom_instructions: str | None
|
|
retain_default_strategy: str | None
|
|
retain_strategies: dict | None
|
|
retain_batch_tokens: int
|
|
retain_batch_enabled: bool
|
|
retain_batch_poll_interval_seconds: int
|
|
retain_entity_lookup: str # "full" or "trigram"
|
|
retain_chunk_batch_size: int # Max chunks per streaming batch (0 = disabled)
|
|
|
|
# File storage (static - server-level only)
|
|
file_storage_type: str # "native" (PostgreSQL) or "s3" (S3-compatible)
|
|
file_storage_s3_bucket: str | None # S3 bucket name (required for s3 storage)
|
|
file_storage_s3_region: str | None # S3 region (optional, uses SDK default)
|
|
file_storage_s3_endpoint: str | None # S3 endpoint URL (for MinIO, R2, etc.)
|
|
file_storage_s3_access_key_id: str | None # S3 access key (optional, uses env/IAM)
|
|
file_storage_s3_secret_access_key: str | None # S3 secret key (optional, uses env/IAM)
|
|
file_storage_gcs_bucket: str | None # GCS bucket name (required for gcs storage)
|
|
file_storage_gcs_service_account_key: str | None # GCS service account key JSON (optional, uses ADC)
|
|
file_storage_azure_container: str | None # Azure container name (required for azure storage)
|
|
file_storage_azure_account_name: str | None # Azure storage account name
|
|
file_storage_azure_account_key: str | None # Azure storage account key
|
|
file_parser: list[str] # Ordered fallback chain of parsers (e.g. ["iris", "markitdown"])
|
|
file_parser_allowlist: list[str] | None # Parsers clients may request (None = all registered)
|
|
file_parser_iris_token: str | None # Vectorize API token for iris parser (VECTORIZE_TOKEN)
|
|
file_parser_iris_org_id: str | None # Vectorize org ID for iris parser (VECTORIZE_ORG_ID)
|
|
file_conversion_max_batch_size_mb: int # Max total batch size in MB (all files combined)
|
|
file_conversion_max_batch_size: int # Max files per request
|
|
enable_file_upload_api: bool
|
|
file_delete_after_retain: bool
|
|
|
|
# Observations settings (consolidated knowledge from facts)
|
|
enable_observations: bool
|
|
enable_observation_history: bool
|
|
enable_mental_model_history: bool
|
|
consolidation_batch_size: int
|
|
consolidation_llm_batch_size: int
|
|
consolidation_max_tokens: int
|
|
consolidation_source_facts_max_tokens: int
|
|
consolidation_source_facts_max_tokens_per_observation: int
|
|
observations_mission: str | None
|
|
max_observations_per_scope: int
|
|
|
|
# Entity labels (controlled vocabulary of key:value classification labels extracted at retain time)
|
|
# List of label group dicts: [{key, description, type, optional, values: [{value, description}]}]
|
|
entity_labels: list | None
|
|
# Whether to extract regular named entities alongside entity labels (default: True)
|
|
# When False: only label entities are extracted (or no entities at all if no labels configured)
|
|
entities_allow_free_form: bool
|
|
|
|
# Reflect agent settings
|
|
reflect_mission: str | None
|
|
reflect_source_facts_max_tokens: int
|
|
|
|
# Disposition settings (hierarchical - can be overridden per bank; None = fall back to DB)
|
|
disposition_skepticism: int | None
|
|
disposition_literalism: int | None
|
|
disposition_empathy: int | None
|
|
|
|
# Optimization flags
|
|
skip_llm_verification: bool
|
|
lazy_reranker: bool
|
|
|
|
# Database migrations
|
|
run_migrations_on_startup: bool
|
|
|
|
# Database connection pool
|
|
db_pool_min_size: int
|
|
db_pool_max_size: int
|
|
db_command_timeout: int
|
|
db_acquire_timeout: int
|
|
|
|
# Worker configuration (distributed task processing)
|
|
worker_enabled: bool
|
|
worker_id: str | None
|
|
worker_poll_interval_ms: int
|
|
worker_max_retries: int
|
|
worker_http_port: int
|
|
worker_max_slots: int
|
|
worker_consolidation_max_slots: int
|
|
retain_max_concurrent: int
|
|
|
|
# Reflect agent settings
|
|
reflect_max_iterations: int
|
|
reflect_max_context_tokens: int
|
|
reflect_wall_timeout: int
|
|
|
|
# OpenTelemetry tracing configuration
|
|
otel_traces_enabled: bool
|
|
otel_exporter_otlp_endpoint: str | None
|
|
otel_exporter_otlp_headers: str | None
|
|
otel_service_name: str
|
|
otel_deployment_environment: str
|
|
|
|
# Audit log configuration (static - server-level only)
|
|
audit_log_enabled: bool # Master switch for audit logging
|
|
audit_log_actions: list[str] # Allowlist of action types (empty = all)
|
|
audit_log_retention_days: int # -1 = keep forever, >0 = delete after N days
|
|
|
|
# Webhook configuration (static - server-level only, not per-bank)
|
|
webhook_url: str | None # Global webhook URL (None = disabled)
|
|
webhook_secret: str | None # HMAC signing secret (None = unsigned)
|
|
webhook_event_types: list[str] # Event types to deliver globally
|
|
webhook_delivery_poll_interval_seconds: int # How often the delivery worker polls
|
|
|
|
# Class-level sets for configuration categorization
|
|
|
|
# CREDENTIAL_FIELDS: Never exposed via API, never configurable per-tenant/bank
|
|
_CREDENTIAL_FIELDS = {
|
|
# API Keys
|
|
"llm_api_key",
|
|
"retain_llm_api_key",
|
|
"reflect_llm_api_key",
|
|
"consolidation_llm_api_key",
|
|
# Base URLs (could expose infrastructure)
|
|
"llm_base_url",
|
|
"retain_llm_base_url",
|
|
"reflect_llm_base_url",
|
|
"consolidation_llm_base_url",
|
|
"embeddings_tei_base_url",
|
|
"reranker_tei_base_url",
|
|
"reranker_cohere_base_url",
|
|
"reranker_zeroentropy_base_url",
|
|
# Service Account Keys
|
|
"llm_vertexai_service_account_key",
|
|
# File storage credentials
|
|
"file_storage_s3_access_key_id",
|
|
"file_storage_s3_secret_access_key",
|
|
"file_storage_gcs_service_account_key",
|
|
"file_storage_azure_account_key",
|
|
# File parser credentials
|
|
"file_parser_iris_token",
|
|
}
|
|
|
|
# CONFIGURABLE_FIELDS: Safe behavioral settings that can be customized per-tenant/bank
|
|
# These fields are manually tagged as safe to expose and modify.
|
|
# Excludes credentials, infrastructure config, provider/model selection, and performance tuning.
|
|
_CONFIGURABLE_FIELDS = {
|
|
# MCP tool access control
|
|
"mcp_enabled_tools",
|
|
# Retention settings (behavioral)
|
|
"retain_chunk_size",
|
|
"retain_extraction_mode",
|
|
"retain_mission",
|
|
"retain_custom_instructions",
|
|
"retain_default_strategy",
|
|
"retain_strategies",
|
|
"retain_chunk_batch_size",
|
|
# Entity labels (controlled vocabulary for entity classification)
|
|
"entity_labels",
|
|
"entities_allow_free_form",
|
|
# Consolidation settings
|
|
"enable_observations",
|
|
"consolidation_llm_batch_size",
|
|
"consolidation_source_facts_max_tokens",
|
|
"consolidation_source_facts_max_tokens_per_observation",
|
|
"observations_mission",
|
|
"max_observations_per_scope",
|
|
# Reflect settings
|
|
"reflect_mission",
|
|
"reflect_source_facts_max_tokens",
|
|
# Disposition settings
|
|
"disposition_skepticism",
|
|
"disposition_literalism",
|
|
"disposition_empathy",
|
|
# Gemini safety settings (controls content filtering for Gemini/VertexAI providers)
|
|
"llm_gemini_safety_settings",
|
|
}
|
|
|
|
@property
|
|
def file_conversion_max_batch_size_bytes(self) -> int:
|
|
"""Get maximum total batch size in bytes."""
|
|
return self.file_conversion_max_batch_size_mb * 1024 * 1024
|
|
|
|
@classmethod
|
|
def get_configurable_fields(cls) -> set[str]:
|
|
"""
|
|
Get set of field names that are configurable per-tenant/bank via API.
|
|
|
|
Configurable fields are manually tagged behavioral settings that are safe
|
|
to expose and modify (e.g., retain_chunk_size, custom_instructions).
|
|
Excludes credentials, infrastructure config, and provider/model selection.
|
|
|
|
Returns:
|
|
Set of configurable field names
|
|
"""
|
|
return cls._CONFIGURABLE_FIELDS.copy()
|
|
|
|
@classmethod
|
|
def get_credential_fields(cls) -> set[str]:
|
|
"""
|
|
Get set of field names that are credentials (NEVER exposed via API).
|
|
|
|
Credential fields include API keys, base URLs, and service account keys.
|
|
These must never be returned in API responses or accepted in updates.
|
|
|
|
Returns:
|
|
Set of credential field names
|
|
"""
|
|
return cls._CREDENTIAL_FIELDS.copy()
|
|
|
|
@classmethod
|
|
def get_hierarchical_fields(cls) -> set[str]:
|
|
"""
|
|
DEPRECATED: Use get_configurable_fields() instead.
|
|
|
|
Kept for backward compatibility during migration.
|
|
"""
|
|
return cls.get_configurable_fields()
|
|
|
|
@classmethod
|
|
def get_static_fields(cls) -> set[str]:
|
|
"""
|
|
Get set of field names that are static (server-level only).
|
|
|
|
Static fields are infrastructure-level settings that cannot vary
|
|
per tenant or bank. These include database config, API port, worker settings, etc.
|
|
Also includes credential fields which are never configurable.
|
|
|
|
Returns:
|
|
Set of static field names
|
|
"""
|
|
# Get all field names from dataclass
|
|
all_fields = {f.name for f in fields(cls)}
|
|
# Static fields = all fields - configurable fields
|
|
return all_fields - cls._CONFIGURABLE_FIELDS
|
|
|
|
def validate(self) -> None:
|
|
"""Validate configuration values and raise errors for invalid combinations."""
|
|
# Validate vector_extension
|
|
valid_extensions = ("pgvector", "vchord", "pgvectorscale")
|
|
if self.vector_extension not in valid_extensions:
|
|
raise ValueError(
|
|
f"Invalid vector_extension: {self.vector_extension}. Must be one of: {', '.join(valid_extensions)}"
|
|
)
|
|
|
|
# Validate text_search_extension
|
|
valid_text_search = ("native", "vchord", "pg_textsearch")
|
|
if self.text_search_extension not in valid_text_search:
|
|
raise ValueError(
|
|
f"Invalid text_search_extension: {self.text_search_extension}. Must be one of: {', '.join(valid_text_search)}"
|
|
)
|
|
|
|
# When LLM provider is "none", force chunks-only mode and disable LLM-dependent features
|
|
if self.llm_provider == "none":
|
|
self.retain_extraction_mode = "chunks"
|
|
self.enable_observations = False
|
|
logger.info(
|
|
"LLM provider set to 'none': forcing retain_extraction_mode='chunks', "
|
|
"disabling observations/consolidation. Reflect will return HTTP 400."
|
|
)
|
|
|
|
# RETAIN_MAX_COMPLETION_TOKENS must be greater than RETAIN_CHUNK_SIZE
|
|
# to ensure the LLM has enough output capacity to extract facts from chunks
|
|
# (not applicable when provider is "none" since no LLM calls are made)
|
|
if self.llm_provider != "none" and self.retain_max_completion_tokens <= self.retain_chunk_size:
|
|
raise ValueError(
|
|
f"Invalid configuration: HINDSIGHT_API_RETAIN_MAX_COMPLETION_TOKENS "
|
|
f"({self.retain_max_completion_tokens}) must be greater than "
|
|
f"HINDSIGHT_API_RETAIN_CHUNK_SIZE ({self.retain_chunk_size}). "
|
|
f"\n\nYou have two options to fix this:"
|
|
f"\n 1. Increase HINDSIGHT_API_RETAIN_MAX_COMPLETION_TOKENS to a value > {self.retain_chunk_size}"
|
|
f"\n 2. Use a model that supports at least {self.retain_max_completion_tokens} output tokens"
|
|
f"\n (current model: {self.retain_llm_model or self.llm_model}, "
|
|
f"provider: {self.retain_llm_provider or self.llm_provider})"
|
|
)
|
|
|
|
@classmethod
|
|
def from_env(cls) -> "HindsightConfig":
|
|
"""Create configuration from environment variables."""
|
|
# Get provider first to determine default model
|
|
llm_provider = os.getenv(ENV_LLM_PROVIDER, DEFAULT_LLM_PROVIDER)
|
|
llm_model = os.getenv(ENV_LLM_MODEL) or _get_default_model_for_provider(llm_provider)
|
|
|
|
config = cls(
|
|
# Database
|
|
database_url=os.getenv(ENV_DATABASE_URL, DEFAULT_DATABASE_URL),
|
|
migration_database_url=os.getenv(ENV_MIGRATION_DATABASE_URL) or None,
|
|
database_schema=os.getenv(ENV_DATABASE_SCHEMA, DEFAULT_DATABASE_SCHEMA),
|
|
vector_extension=os.getenv(ENV_VECTOR_EXTENSION, DEFAULT_VECTOR_EXTENSION).lower(),
|
|
text_search_extension=os.getenv(ENV_TEXT_SEARCH_EXTENSION, DEFAULT_TEXT_SEARCH_EXTENSION).lower(),
|
|
# LLM
|
|
llm_provider=llm_provider,
|
|
llm_api_key=os.getenv(ENV_LLM_API_KEY),
|
|
llm_model=llm_model,
|
|
llm_base_url=os.getenv(ENV_LLM_BASE_URL) or None,
|
|
llm_max_concurrent=int(os.getenv(ENV_LLM_MAX_CONCURRENT, str(DEFAULT_LLM_MAX_CONCURRENT))),
|
|
llm_max_retries=int(os.getenv(ENV_LLM_MAX_RETRIES, str(DEFAULT_LLM_MAX_RETRIES))),
|
|
llm_initial_backoff=float(os.getenv(ENV_LLM_INITIAL_BACKOFF, str(DEFAULT_LLM_INITIAL_BACKOFF))),
|
|
llm_max_backoff=float(os.getenv(ENV_LLM_MAX_BACKOFF, str(DEFAULT_LLM_MAX_BACKOFF))),
|
|
llm_timeout=float(os.getenv(ENV_LLM_TIMEOUT, str(DEFAULT_LLM_TIMEOUT))),
|
|
llm_groq_service_tier=os.getenv(ENV_LLM_GROQ_SERVICE_TIER, DEFAULT_LLM_GROQ_SERVICE_TIER),
|
|
llm_openai_service_tier=os.getenv(ENV_LLM_OPENAI_SERVICE_TIER, DEFAULT_LLM_OPENAI_SERVICE_TIER),
|
|
llm_extra_body=json.loads(os.getenv(ENV_LLM_EXTRA_BODY, "null")),
|
|
# Vertex AI
|
|
llm_vertexai_project_id=os.getenv(ENV_LLM_VERTEXAI_PROJECT_ID) or DEFAULT_LLM_VERTEXAI_PROJECT_ID,
|
|
llm_vertexai_region=os.getenv(ENV_LLM_VERTEXAI_REGION, DEFAULT_LLM_VERTEXAI_REGION),
|
|
llm_vertexai_service_account_key=os.getenv(ENV_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY)
|
|
or DEFAULT_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY,
|
|
# Gemini safety settings (JSON-encoded list of {category, threshold} dicts)
|
|
llm_gemini_safety_settings=json.loads(os.getenv(ENV_LLM_GEMINI_SAFETY_SETTINGS, "null")),
|
|
# Per-operation LLM config (None = use default)
|
|
retain_llm_provider=os.getenv(ENV_RETAIN_LLM_PROVIDER) or None,
|
|
retain_llm_api_key=os.getenv(ENV_RETAIN_LLM_API_KEY) or None,
|
|
retain_llm_model=os.getenv(ENV_RETAIN_LLM_MODEL)
|
|
or (
|
|
_get_default_model_for_provider(os.getenv(ENV_RETAIN_LLM_PROVIDER))
|
|
if os.getenv(ENV_RETAIN_LLM_PROVIDER)
|
|
else None
|
|
),
|
|
retain_llm_base_url=os.getenv(ENV_RETAIN_LLM_BASE_URL) or None,
|
|
retain_llm_max_concurrent=int(os.getenv(ENV_RETAIN_LLM_MAX_CONCURRENT))
|
|
if os.getenv(ENV_RETAIN_LLM_MAX_CONCURRENT)
|
|
else None,
|
|
retain_llm_max_retries=int(os.getenv(ENV_RETAIN_LLM_MAX_RETRIES))
|
|
if os.getenv(ENV_RETAIN_LLM_MAX_RETRIES)
|
|
else None,
|
|
retain_llm_initial_backoff=float(os.getenv(ENV_RETAIN_LLM_INITIAL_BACKOFF))
|
|
if os.getenv(ENV_RETAIN_LLM_INITIAL_BACKOFF)
|
|
else None,
|
|
retain_llm_max_backoff=float(os.getenv(ENV_RETAIN_LLM_MAX_BACKOFF))
|
|
if os.getenv(ENV_RETAIN_LLM_MAX_BACKOFF)
|
|
else None,
|
|
retain_llm_timeout=float(os.getenv(ENV_RETAIN_LLM_TIMEOUT)) if os.getenv(ENV_RETAIN_LLM_TIMEOUT) else None,
|
|
reflect_llm_provider=os.getenv(ENV_REFLECT_LLM_PROVIDER) or None,
|
|
reflect_llm_api_key=os.getenv(ENV_REFLECT_LLM_API_KEY) or None,
|
|
reflect_llm_model=os.getenv(ENV_REFLECT_LLM_MODEL)
|
|
or (
|
|
_get_default_model_for_provider(os.getenv(ENV_REFLECT_LLM_PROVIDER))
|
|
if os.getenv(ENV_REFLECT_LLM_PROVIDER)
|
|
else None
|
|
),
|
|
reflect_llm_base_url=os.getenv(ENV_REFLECT_LLM_BASE_URL) or None,
|
|
reflect_llm_max_concurrent=int(os.getenv(ENV_REFLECT_LLM_MAX_CONCURRENT))
|
|
if os.getenv(ENV_REFLECT_LLM_MAX_CONCURRENT)
|
|
else None,
|
|
reflect_llm_max_retries=int(os.getenv(ENV_REFLECT_LLM_MAX_RETRIES))
|
|
if os.getenv(ENV_REFLECT_LLM_MAX_RETRIES)
|
|
else None,
|
|
reflect_llm_initial_backoff=float(os.getenv(ENV_REFLECT_LLM_INITIAL_BACKOFF))
|
|
if os.getenv(ENV_REFLECT_LLM_INITIAL_BACKOFF)
|
|
else None,
|
|
reflect_llm_max_backoff=float(os.getenv(ENV_REFLECT_LLM_MAX_BACKOFF))
|
|
if os.getenv(ENV_REFLECT_LLM_MAX_BACKOFF)
|
|
else None,
|
|
reflect_llm_timeout=float(os.getenv(ENV_REFLECT_LLM_TIMEOUT))
|
|
if os.getenv(ENV_REFLECT_LLM_TIMEOUT)
|
|
else None,
|
|
consolidation_llm_provider=os.getenv(ENV_CONSOLIDATION_LLM_PROVIDER) or None,
|
|
consolidation_llm_api_key=os.getenv(ENV_CONSOLIDATION_LLM_API_KEY) or None,
|
|
consolidation_llm_model=os.getenv(ENV_CONSOLIDATION_LLM_MODEL)
|
|
or (
|
|
_get_default_model_for_provider(os.getenv(ENV_CONSOLIDATION_LLM_PROVIDER))
|
|
if os.getenv(ENV_CONSOLIDATION_LLM_PROVIDER)
|
|
else None
|
|
),
|
|
consolidation_llm_base_url=os.getenv(ENV_CONSOLIDATION_LLM_BASE_URL) or None,
|
|
consolidation_llm_max_concurrent=int(os.getenv(ENV_CONSOLIDATION_LLM_MAX_CONCURRENT))
|
|
if os.getenv(ENV_CONSOLIDATION_LLM_MAX_CONCURRENT)
|
|
else None,
|
|
consolidation_llm_max_retries=int(os.getenv(ENV_CONSOLIDATION_LLM_MAX_RETRIES))
|
|
if os.getenv(ENV_CONSOLIDATION_LLM_MAX_RETRIES)
|
|
else None,
|
|
consolidation_llm_initial_backoff=float(os.getenv(ENV_CONSOLIDATION_LLM_INITIAL_BACKOFF))
|
|
if os.getenv(ENV_CONSOLIDATION_LLM_INITIAL_BACKOFF)
|
|
else None,
|
|
consolidation_llm_max_backoff=float(os.getenv(ENV_CONSOLIDATION_LLM_MAX_BACKOFF))
|
|
if os.getenv(ENV_CONSOLIDATION_LLM_MAX_BACKOFF)
|
|
else None,
|
|
consolidation_llm_timeout=float(os.getenv(ENV_CONSOLIDATION_LLM_TIMEOUT))
|
|
if os.getenv(ENV_CONSOLIDATION_LLM_TIMEOUT)
|
|
else None,
|
|
# Embeddings
|
|
embeddings_provider=os.getenv(ENV_EMBEDDINGS_PROVIDER, DEFAULT_EMBEDDINGS_PROVIDER),
|
|
embeddings_local_model=os.getenv(ENV_EMBEDDINGS_LOCAL_MODEL, DEFAULT_EMBEDDINGS_LOCAL_MODEL),
|
|
embeddings_local_force_cpu=os.getenv(
|
|
ENV_EMBEDDINGS_LOCAL_FORCE_CPU, str(DEFAULT_EMBEDDINGS_LOCAL_FORCE_CPU)
|
|
).lower()
|
|
in ("true", "1"),
|
|
embeddings_local_trust_remote_code=os.getenv(
|
|
ENV_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE, str(DEFAULT_EMBEDDINGS_LOCAL_TRUST_REMOTE_CODE)
|
|
).lower()
|
|
in ("true", "1"),
|
|
embeddings_tei_url=os.getenv(ENV_EMBEDDINGS_TEI_URL),
|
|
embeddings_openai_base_url=os.getenv(ENV_EMBEDDINGS_OPENAI_BASE_URL) or None,
|
|
# Cohere embeddings (with backward-compatible fallback to shared API key)
|
|
embeddings_cohere_api_key=os.getenv(ENV_EMBEDDINGS_COHERE_API_KEY) or os.getenv(ENV_COHERE_API_KEY),
|
|
embeddings_cohere_model=os.getenv(ENV_EMBEDDINGS_COHERE_MODEL, DEFAULT_EMBEDDINGS_COHERE_MODEL),
|
|
embeddings_cohere_base_url=os.getenv(ENV_EMBEDDINGS_COHERE_BASE_URL) or None,
|
|
# LiteLLM embeddings (with backward-compatible fallback to shared config)
|
|
embeddings_litellm_api_base=os.getenv(ENV_EMBEDDINGS_LITELLM_API_BASE)
|
|
or os.getenv(ENV_LITELLM_API_BASE, DEFAULT_LITELLM_API_BASE),
|
|
embeddings_litellm_api_key=os.getenv(ENV_EMBEDDINGS_LITELLM_API_KEY) or os.getenv(ENV_LITELLM_API_KEY),
|
|
embeddings_litellm_model=os.getenv(ENV_EMBEDDINGS_LITELLM_MODEL, DEFAULT_EMBEDDINGS_LITELLM_MODEL),
|
|
# LiteLLM SDK embeddings (direct API access)
|
|
embeddings_litellm_sdk_api_key=os.getenv(ENV_EMBEDDINGS_LITELLM_SDK_API_KEY),
|
|
embeddings_litellm_sdk_model=os.getenv(
|
|
ENV_EMBEDDINGS_LITELLM_SDK_MODEL, DEFAULT_EMBEDDINGS_LITELLM_SDK_MODEL
|
|
),
|
|
embeddings_litellm_sdk_api_base=os.getenv(ENV_EMBEDDINGS_LITELLM_SDK_API_BASE) or None,
|
|
embeddings_litellm_sdk_output_dimensions=int(v)
|
|
if (v := os.getenv(ENV_EMBEDDINGS_LITELLM_SDK_OUTPUT_DIMENSIONS))
|
|
else None,
|
|
# Reranker
|
|
reranker_provider=os.getenv(ENV_RERANKER_PROVIDER, DEFAULT_RERANKER_PROVIDER),
|
|
reranker_local_model=os.getenv(ENV_RERANKER_LOCAL_MODEL, DEFAULT_RERANKER_LOCAL_MODEL),
|
|
reranker_local_force_cpu=os.getenv(
|
|
ENV_RERANKER_LOCAL_FORCE_CPU, str(DEFAULT_RERANKER_LOCAL_FORCE_CPU)
|
|
).lower()
|
|
in ("true", "1"),
|
|
reranker_local_max_concurrent=int(
|
|
os.getenv(ENV_RERANKER_LOCAL_MAX_CONCURRENT, str(DEFAULT_RERANKER_LOCAL_MAX_CONCURRENT))
|
|
),
|
|
reranker_local_trust_remote_code=os.getenv(
|
|
ENV_RERANKER_LOCAL_TRUST_REMOTE_CODE, str(DEFAULT_RERANKER_LOCAL_TRUST_REMOTE_CODE)
|
|
).lower()
|
|
in ("true", "1"),
|
|
reranker_local_fp16=os.getenv(ENV_RERANKER_LOCAL_FP16, str(DEFAULT_RERANKER_LOCAL_FP16)).lower()
|
|
in ("true", "1"),
|
|
reranker_local_bucket_batching=os.getenv(
|
|
ENV_RERANKER_LOCAL_BUCKET_BATCHING, str(DEFAULT_RERANKER_LOCAL_BUCKET_BATCHING)
|
|
).lower()
|
|
in ("true", "1"),
|
|
reranker_local_batch_size=int(
|
|
os.getenv(ENV_RERANKER_LOCAL_BATCH_SIZE, str(DEFAULT_RERANKER_LOCAL_BATCH_SIZE))
|
|
),
|
|
reranker_tei_url=os.getenv(ENV_RERANKER_TEI_URL),
|
|
reranker_tei_batch_size=int(os.getenv(ENV_RERANKER_TEI_BATCH_SIZE, str(DEFAULT_RERANKER_TEI_BATCH_SIZE))),
|
|
reranker_tei_max_concurrent=int(
|
|
os.getenv(ENV_RERANKER_TEI_MAX_CONCURRENT, str(DEFAULT_RERANKER_TEI_MAX_CONCURRENT))
|
|
),
|
|
reranker_max_candidates=int(os.getenv(ENV_RERANKER_MAX_CANDIDATES, str(DEFAULT_RERANKER_MAX_CANDIDATES))),
|
|
# Cohere reranker (with backward-compatible fallback to shared API key)
|
|
reranker_cohere_api_key=os.getenv(ENV_RERANKER_COHERE_API_KEY) or os.getenv(ENV_COHERE_API_KEY),
|
|
reranker_cohere_model=os.getenv(ENV_RERANKER_COHERE_MODEL, DEFAULT_RERANKER_COHERE_MODEL),
|
|
reranker_cohere_base_url=os.getenv(ENV_RERANKER_COHERE_BASE_URL) or None,
|
|
# LiteLLM reranker (with backward-compatible fallback to shared config)
|
|
reranker_litellm_api_base=os.getenv(ENV_RERANKER_LITELLM_API_BASE)
|
|
or os.getenv(ENV_LITELLM_API_BASE, DEFAULT_LITELLM_API_BASE),
|
|
reranker_litellm_api_key=os.getenv(ENV_RERANKER_LITELLM_API_KEY) or os.getenv(ENV_LITELLM_API_KEY),
|
|
reranker_litellm_model=os.getenv(ENV_RERANKER_LITELLM_MODEL, DEFAULT_RERANKER_LITELLM_MODEL),
|
|
reranker_litellm_max_tokens_per_doc=int(v)
|
|
if (v := os.getenv(ENV_RERANKER_LITELLM_MAX_TOKENS_PER_DOC))
|
|
else DEFAULT_RERANKER_LITELLM_MAX_TOKENS_PER_DOC,
|
|
# LiteLLM SDK reranker (direct API access)
|
|
reranker_litellm_sdk_api_key=os.getenv(ENV_RERANKER_LITELLM_SDK_API_KEY),
|
|
reranker_litellm_sdk_model=os.getenv(ENV_RERANKER_LITELLM_SDK_MODEL, DEFAULT_RERANKER_LITELLM_SDK_MODEL),
|
|
reranker_litellm_sdk_api_base=os.getenv(ENV_RERANKER_LITELLM_SDK_API_BASE) or None,
|
|
# ZeroEntropy reranker
|
|
reranker_zeroentropy_api_key=os.getenv(ENV_RERANKER_ZEROENTROPY_API_KEY),
|
|
reranker_zeroentropy_model=os.getenv(ENV_RERANKER_ZEROENTROPY_MODEL, DEFAULT_RERANKER_ZEROENTROPY_MODEL),
|
|
reranker_zeroentropy_base_url=os.getenv(ENV_RERANKER_ZEROENTROPY_BASE_URL) or None,
|
|
# Server
|
|
host=os.getenv(ENV_HOST, DEFAULT_HOST),
|
|
port=int(os.getenv(ENV_PORT, DEFAULT_PORT)),
|
|
base_path=os.getenv(ENV_BASE_PATH, DEFAULT_BASE_PATH),
|
|
log_level=os.getenv(ENV_LOG_LEVEL, DEFAULT_LOG_LEVEL),
|
|
log_format=os.getenv(ENV_LOG_FORMAT, DEFAULT_LOG_FORMAT).lower(),
|
|
mcp_enabled=os.getenv(ENV_MCP_ENABLED, str(DEFAULT_MCP_ENABLED)).lower() == "true",
|
|
mcp_enabled_tools=[t.strip() for t in os.getenv(ENV_MCP_ENABLED_TOOLS).split(",") if t.strip()]
|
|
if os.getenv(ENV_MCP_ENABLED_TOOLS)
|
|
else DEFAULT_MCP_ENABLED_TOOLS,
|
|
mcp_stateless=os.getenv(ENV_MCP_STATELESS, str(DEFAULT_MCP_STATELESS)).lower() == "true",
|
|
enable_bank_config_api=os.getenv(ENV_ENABLE_BANK_CONFIG_API, str(DEFAULT_ENABLE_BANK_CONFIG_API)).lower()
|
|
== "true",
|
|
# Recall
|
|
graph_retriever=os.getenv(ENV_GRAPH_RETRIEVER, DEFAULT_GRAPH_RETRIEVER),
|
|
recall_max_concurrent=int(os.getenv(ENV_RECALL_MAX_CONCURRENT, str(DEFAULT_RECALL_MAX_CONCURRENT))),
|
|
recall_connection_budget=int(
|
|
os.getenv(ENV_RECALL_CONNECTION_BUDGET, str(DEFAULT_RECALL_CONNECTION_BUDGET))
|
|
),
|
|
recall_max_query_tokens=int(os.getenv(ENV_RECALL_MAX_QUERY_TOKENS, str(DEFAULT_RECALL_MAX_QUERY_TOKENS))),
|
|
mental_model_refresh_concurrency=int(
|
|
os.getenv(ENV_MENTAL_MODEL_REFRESH_CONCURRENCY, str(DEFAULT_MENTAL_MODEL_REFRESH_CONCURRENCY))
|
|
),
|
|
# Optimization flags
|
|
skip_llm_verification=os.getenv(ENV_SKIP_LLM_VERIFICATION, "false").lower() == "true",
|
|
lazy_reranker=os.getenv(ENV_LAZY_RERANKER, "false").lower() == "true",
|
|
# Retain settings
|
|
retain_max_completion_tokens=int(
|
|
os.getenv(ENV_RETAIN_MAX_COMPLETION_TOKENS, str(DEFAULT_RETAIN_MAX_COMPLETION_TOKENS))
|
|
),
|
|
retain_chunk_size=int(os.getenv(ENV_RETAIN_CHUNK_SIZE, str(DEFAULT_RETAIN_CHUNK_SIZE))),
|
|
retain_extract_causal_links=os.getenv(
|
|
ENV_RETAIN_EXTRACT_CAUSAL_LINKS, str(DEFAULT_RETAIN_EXTRACT_CAUSAL_LINKS)
|
|
).lower()
|
|
== "true",
|
|
retain_extraction_mode=_validate_extraction_mode(
|
|
os.getenv(ENV_RETAIN_EXTRACTION_MODE, DEFAULT_RETAIN_EXTRACTION_MODE)
|
|
),
|
|
retain_mission=os.getenv(ENV_RETAIN_MISSION) or DEFAULT_RETAIN_MISSION,
|
|
retain_custom_instructions=os.getenv(ENV_RETAIN_CUSTOM_INSTRUCTIONS) or DEFAULT_RETAIN_CUSTOM_INSTRUCTIONS,
|
|
retain_default_strategy=os.getenv(ENV_RETAIN_DEFAULT_STRATEGY) or DEFAULT_RETAIN_DEFAULT_STRATEGY,
|
|
retain_strategies=DEFAULT_RETAIN_STRATEGIES,
|
|
retain_batch_tokens=int(os.getenv(ENV_RETAIN_BATCH_TOKENS, str(DEFAULT_RETAIN_BATCH_TOKENS))),
|
|
retain_entity_lookup=os.getenv(ENV_RETAIN_ENTITY_LOOKUP, DEFAULT_RETAIN_ENTITY_LOOKUP),
|
|
retain_batch_enabled=os.getenv(ENV_RETAIN_BATCH_ENABLED, str(DEFAULT_RETAIN_BATCH_ENABLED)).lower()
|
|
== "true",
|
|
retain_batch_poll_interval_seconds=int(
|
|
os.getenv(ENV_RETAIN_BATCH_POLL_INTERVAL_SECONDS, str(DEFAULT_RETAIN_BATCH_POLL_INTERVAL_SECONDS))
|
|
),
|
|
retain_chunk_batch_size=int(os.getenv(ENV_RETAIN_CHUNK_BATCH_SIZE, str(DEFAULT_RETAIN_CHUNK_BATCH_SIZE))),
|
|
# File storage
|
|
file_storage_type=os.getenv(ENV_FILE_STORAGE_TYPE, DEFAULT_FILE_STORAGE_TYPE),
|
|
file_storage_s3_bucket=os.getenv(ENV_FILE_STORAGE_S3_BUCKET) or None,
|
|
file_storage_s3_region=os.getenv(ENV_FILE_STORAGE_S3_REGION) or None,
|
|
file_storage_s3_endpoint=os.getenv(ENV_FILE_STORAGE_S3_ENDPOINT) or None,
|
|
file_storage_s3_access_key_id=os.getenv(ENV_FILE_STORAGE_S3_ACCESS_KEY_ID) or None,
|
|
file_storage_s3_secret_access_key=os.getenv(ENV_FILE_STORAGE_S3_SECRET_ACCESS_KEY) or None,
|
|
file_storage_gcs_bucket=os.getenv(ENV_FILE_STORAGE_GCS_BUCKET) or None,
|
|
file_storage_gcs_service_account_key=os.getenv(ENV_FILE_STORAGE_GCS_SERVICE_ACCOUNT_KEY) or None,
|
|
file_storage_azure_container=os.getenv(ENV_FILE_STORAGE_AZURE_CONTAINER) or None,
|
|
file_storage_azure_account_name=os.getenv(ENV_FILE_STORAGE_AZURE_ACCOUNT_NAME) or None,
|
|
file_storage_azure_account_key=os.getenv(ENV_FILE_STORAGE_AZURE_ACCOUNT_KEY) or None,
|
|
file_parser=_parse_str_list(os.getenv(ENV_FILE_PARSER, DEFAULT_FILE_PARSER)),
|
|
file_parser_allowlist=_parse_str_list(os.getenv(ENV_FILE_PARSER_ALLOWLIST))
|
|
if os.getenv(ENV_FILE_PARSER_ALLOWLIST)
|
|
else None,
|
|
file_parser_iris_token=os.getenv(ENV_FILE_PARSER_IRIS_TOKEN) or None,
|
|
file_parser_iris_org_id=os.getenv(ENV_FILE_PARSER_IRIS_ORG_ID) or None,
|
|
file_conversion_max_batch_size_mb=int(
|
|
os.getenv(ENV_FILE_CONVERSION_MAX_BATCH_SIZE_MB, str(DEFAULT_FILE_CONVERSION_MAX_BATCH_SIZE_MB))
|
|
),
|
|
file_conversion_max_batch_size=int(
|
|
os.getenv(ENV_FILE_CONVERSION_MAX_BATCH_SIZE, str(DEFAULT_FILE_CONVERSION_MAX_BATCH_SIZE))
|
|
),
|
|
enable_file_upload_api=os.getenv(ENV_ENABLE_FILE_UPLOAD_API, str(DEFAULT_ENABLE_FILE_UPLOAD_API)).lower()
|
|
== "true",
|
|
file_delete_after_retain=os.getenv(
|
|
ENV_FILE_DELETE_AFTER_RETAIN, str(DEFAULT_FILE_DELETE_AFTER_RETAIN)
|
|
).lower()
|
|
== "true",
|
|
# Observations settings (consolidated knowledge from facts)
|
|
enable_observations=os.getenv(ENV_ENABLE_OBSERVATIONS, str(DEFAULT_ENABLE_OBSERVATIONS)).lower() == "true",
|
|
enable_observation_history=os.getenv(
|
|
ENV_ENABLE_OBSERVATION_HISTORY, str(DEFAULT_ENABLE_OBSERVATION_HISTORY)
|
|
).lower()
|
|
== "true",
|
|
enable_mental_model_history=os.getenv(
|
|
ENV_ENABLE_MENTAL_MODEL_HISTORY, str(DEFAULT_ENABLE_MENTAL_MODEL_HISTORY)
|
|
).lower()
|
|
== "true",
|
|
consolidation_batch_size=int(
|
|
os.getenv(ENV_CONSOLIDATION_BATCH_SIZE, str(DEFAULT_CONSOLIDATION_BATCH_SIZE))
|
|
),
|
|
consolidation_llm_batch_size=int(
|
|
os.getenv(ENV_CONSOLIDATION_LLM_BATCH_SIZE, str(DEFAULT_CONSOLIDATION_LLM_BATCH_SIZE))
|
|
),
|
|
consolidation_max_tokens=int(
|
|
os.getenv(ENV_CONSOLIDATION_MAX_TOKENS, str(DEFAULT_CONSOLIDATION_MAX_TOKENS))
|
|
),
|
|
consolidation_source_facts_max_tokens=int(
|
|
os.getenv(ENV_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS, str(DEFAULT_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS))
|
|
),
|
|
consolidation_source_facts_max_tokens_per_observation=int(
|
|
os.getenv(
|
|
ENV_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS_PER_OBSERVATION,
|
|
str(DEFAULT_CONSOLIDATION_SOURCE_FACTS_MAX_TOKENS_PER_OBSERVATION),
|
|
)
|
|
),
|
|
observations_mission=os.getenv(ENV_OBSERVATIONS_MISSION) or DEFAULT_OBSERVATIONS_MISSION,
|
|
max_observations_per_scope=int(
|
|
os.getenv(ENV_MAX_OBSERVATIONS_PER_SCOPE, str(DEFAULT_MAX_OBSERVATIONS_PER_SCOPE))
|
|
),
|
|
entity_labels=None,
|
|
entities_allow_free_form=True,
|
|
# Database migrations
|
|
run_migrations_on_startup=os.getenv(ENV_RUN_MIGRATIONS_ON_STARTUP, "true").lower() == "true",
|
|
# Database connection pool
|
|
db_pool_min_size=int(os.getenv(ENV_DB_POOL_MIN_SIZE, str(DEFAULT_DB_POOL_MIN_SIZE))),
|
|
db_pool_max_size=int(os.getenv(ENV_DB_POOL_MAX_SIZE, str(DEFAULT_DB_POOL_MAX_SIZE))),
|
|
db_command_timeout=int(os.getenv(ENV_DB_COMMAND_TIMEOUT, str(DEFAULT_DB_COMMAND_TIMEOUT))),
|
|
db_acquire_timeout=int(os.getenv(ENV_DB_ACQUIRE_TIMEOUT, str(DEFAULT_DB_ACQUIRE_TIMEOUT))),
|
|
# Worker configuration
|
|
worker_enabled=os.getenv(ENV_WORKER_ENABLED, str(DEFAULT_WORKER_ENABLED)).lower() == "true",
|
|
worker_id=os.getenv(ENV_WORKER_ID) or DEFAULT_WORKER_ID,
|
|
worker_poll_interval_ms=int(os.getenv(ENV_WORKER_POLL_INTERVAL_MS, str(DEFAULT_WORKER_POLL_INTERVAL_MS))),
|
|
worker_max_retries=int(os.getenv(ENV_WORKER_MAX_RETRIES, str(DEFAULT_WORKER_MAX_RETRIES))),
|
|
worker_http_port=int(os.getenv(ENV_WORKER_HTTP_PORT, str(DEFAULT_WORKER_HTTP_PORT))),
|
|
worker_max_slots=int(os.getenv(ENV_WORKER_MAX_SLOTS, str(DEFAULT_WORKER_MAX_SLOTS))),
|
|
worker_consolidation_max_slots=int(
|
|
os.getenv(ENV_WORKER_CONSOLIDATION_MAX_SLOTS, str(DEFAULT_WORKER_CONSOLIDATION_MAX_SLOTS))
|
|
),
|
|
retain_max_concurrent=int(os.getenv(ENV_RETAIN_MAX_CONCURRENT, str(DEFAULT_RETAIN_MAX_CONCURRENT))),
|
|
# Reflect agent settings
|
|
reflect_max_iterations=int(os.getenv(ENV_REFLECT_MAX_ITERATIONS, str(DEFAULT_REFLECT_MAX_ITERATIONS))),
|
|
reflect_max_context_tokens=int(
|
|
os.getenv(ENV_REFLECT_MAX_CONTEXT_TOKENS, str(DEFAULT_REFLECT_MAX_CONTEXT_TOKENS))
|
|
),
|
|
reflect_wall_timeout=int(os.getenv(ENV_REFLECT_WALL_TIMEOUT, str(DEFAULT_REFLECT_WALL_TIMEOUT))),
|
|
reflect_mission=os.getenv(ENV_REFLECT_MISSION) or None,
|
|
reflect_source_facts_max_tokens=int(
|
|
os.getenv(ENV_REFLECT_SOURCE_FACTS_MAX_TOKENS, str(DEFAULT_REFLECT_SOURCE_FACTS_MAX_TOKENS))
|
|
),
|
|
# Disposition settings (None = fall back to DB value)
|
|
disposition_skepticism=int(os.getenv(ENV_DISPOSITION_SKEPTICISM))
|
|
if os.getenv(ENV_DISPOSITION_SKEPTICISM)
|
|
else DEFAULT_DISPOSITION_SKEPTICISM,
|
|
disposition_literalism=int(os.getenv(ENV_DISPOSITION_LITERALISM))
|
|
if os.getenv(ENV_DISPOSITION_LITERALISM)
|
|
else DEFAULT_DISPOSITION_LITERALISM,
|
|
disposition_empathy=int(os.getenv(ENV_DISPOSITION_EMPATHY))
|
|
if os.getenv(ENV_DISPOSITION_EMPATHY)
|
|
else DEFAULT_DISPOSITION_EMPATHY,
|
|
# OpenTelemetry tracing configuration
|
|
otel_traces_enabled=os.getenv(ENV_OTEL_TRACES_ENABLED, str(DEFAULT_OTEL_TRACES_ENABLED)).lower()
|
|
in ("true", "1", "yes"),
|
|
otel_exporter_otlp_endpoint=os.getenv(ENV_OTEL_EXPORTER_OTLP_ENDPOINT) or None,
|
|
otel_exporter_otlp_headers=os.getenv(ENV_OTEL_EXPORTER_OTLP_HEADERS) or None,
|
|
otel_service_name=os.getenv(ENV_OTEL_SERVICE_NAME, DEFAULT_OTEL_SERVICE_NAME),
|
|
otel_deployment_environment=os.getenv(ENV_OTEL_DEPLOYMENT_ENVIRONMENT, DEFAULT_OTEL_DEPLOYMENT_ENVIRONMENT),
|
|
# Audit log configuration (static, server-level only)
|
|
audit_log_enabled=os.getenv(ENV_AUDIT_LOG_ENABLED, str(DEFAULT_AUDIT_LOG_ENABLED)).lower() == "true",
|
|
audit_log_actions=[
|
|
a.strip() for a in os.getenv(ENV_AUDIT_LOG_ACTIONS, DEFAULT_AUDIT_LOG_ACTIONS).split(",") if a.strip()
|
|
],
|
|
audit_log_retention_days=int(
|
|
os.getenv(ENV_AUDIT_LOG_RETENTION_DAYS, str(DEFAULT_AUDIT_LOG_RETENTION_DAYS))
|
|
),
|
|
# Webhook configuration (static, server-level only)
|
|
webhook_url=os.getenv(ENV_WEBHOOK_URL) or DEFAULT_WEBHOOK_URL,
|
|
webhook_secret=os.getenv(ENV_WEBHOOK_SECRET) or DEFAULT_WEBHOOK_SECRET,
|
|
webhook_event_types=[
|
|
t.strip()
|
|
for t in os.getenv(ENV_WEBHOOK_EVENT_TYPES, DEFAULT_WEBHOOK_EVENT_TYPES).split(",")
|
|
if t.strip()
|
|
],
|
|
webhook_delivery_poll_interval_seconds=int(
|
|
os.getenv(
|
|
ENV_WEBHOOK_DELIVERY_POLL_INTERVAL_SECONDS,
|
|
str(DEFAULT_WEBHOOK_DELIVERY_POLL_INTERVAL_SECONDS),
|
|
)
|
|
),
|
|
)
|
|
config.validate()
|
|
return config
|
|
|
|
def get_llm_base_url(self) -> str:
|
|
"""Get the LLM base URL, with provider-specific defaults."""
|
|
if self.llm_base_url:
|
|
return self.llm_base_url
|
|
|
|
provider = self.llm_provider.lower()
|
|
if provider == "groq":
|
|
return "https://api.groq.com/openai/v1"
|
|
elif provider == "ollama":
|
|
return "http://localhost:11434/v1"
|
|
elif provider == "lmstudio":
|
|
return "http://localhost:1234/v1"
|
|
else:
|
|
return ""
|
|
|
|
def get_python_log_level(self) -> int:
|
|
"""Get the Python logging level from the configured log level string."""
|
|
log_level_map = {
|
|
"critical": logging.CRITICAL,
|
|
"error": logging.ERROR,
|
|
"warning": logging.WARNING,
|
|
"info": logging.INFO,
|
|
"debug": logging.DEBUG,
|
|
"trace": logging.DEBUG, # Python doesn't have TRACE, use DEBUG
|
|
}
|
|
return log_level_map.get(self.log_level.lower(), logging.INFO)
|
|
|
|
def configure_logging(self) -> None:
|
|
"""Configure Python logging based on the log level and format.
|
|
|
|
When log_format is "json", outputs structured JSON logs with a severity
|
|
field that GCP Cloud Logging can parse for proper log level categorization.
|
|
"""
|
|
root_logger = logging.getLogger()
|
|
root_logger.setLevel(self.get_python_log_level())
|
|
|
|
# Remove existing handlers
|
|
for handler in root_logger.handlers[:]:
|
|
root_logger.removeHandler(handler)
|
|
|
|
# Create handler writing to stdout (GCP treats stderr as ERROR)
|
|
handler = logging.StreamHandler(sys.stdout)
|
|
handler.setLevel(self.get_python_log_level())
|
|
|
|
if self.log_format == "json":
|
|
handler.setFormatter(JsonFormatter())
|
|
else:
|
|
handler.setFormatter(logging.Formatter("%(asctime)s - %(levelname)s - %(name)s - %(message)s"))
|
|
|
|
root_logger.addHandler(handler)
|
|
|
|
# Silence noisy third-party loggers
|
|
logging.getLogger("google_genai.models").setLevel(logging.WARNING)
|
|
|
|
def log_config(self) -> None:
|
|
"""Log the current configuration (without sensitive values)."""
|
|
logger.info(f"Database: {self.database_url} (schema: {self.database_schema})")
|
|
if self.migration_database_url:
|
|
logger.info(f"Migration database: {self.migration_database_url}")
|
|
logger.info(f"LLM: provider={self.llm_provider}, model={self.llm_model}")
|
|
if self.retain_llm_provider or self.retain_llm_model:
|
|
retain_provider = self.retain_llm_provider or self.llm_provider
|
|
retain_model = self.retain_llm_model or self.llm_model
|
|
logger.info(f"LLM (retain): provider={retain_provider}, model={retain_model}")
|
|
if self.reflect_llm_provider or self.reflect_llm_model:
|
|
reflect_provider = self.reflect_llm_provider or self.llm_provider
|
|
reflect_model = self.reflect_llm_model or self.llm_model
|
|
logger.info(f"LLM (reflect): provider={reflect_provider}, model={reflect_model}")
|
|
if self.consolidation_llm_provider or self.consolidation_llm_model:
|
|
consolidation_provider = self.consolidation_llm_provider or self.llm_provider
|
|
consolidation_model = self.consolidation_llm_model or self.llm_model
|
|
logger.info(f"LLM (consolidation): provider={consolidation_provider}, model={consolidation_model}")
|
|
logger.info(f"Embeddings: provider={self.embeddings_provider}")
|
|
logger.info(f"Reranker: provider={self.reranker_provider}")
|
|
logger.info(f"Graph retriever: {self.graph_retriever}")
|
|
|
|
|
|
# Cached config instance
|
|
_config_cache: HindsightConfig | None = None
|
|
|
|
|
|
def get_config() -> StaticConfigProxy:
|
|
"""
|
|
Get global configuration with ONLY static (non-configurable) fields accessible.
|
|
|
|
This returns a proxy that prevents access to bank-configurable fields
|
|
(like enable_observations, retain_chunk_size, etc.).
|
|
|
|
For bank-specific configuration, use:
|
|
config_resolver.resolve_full_config(bank_id, context)
|
|
|
|
This design prevents accidentally using global defaults when bank-specific
|
|
overrides exist.
|
|
|
|
Returns:
|
|
StaticConfigProxy that only exposes static infrastructure fields
|
|
|
|
Raises:
|
|
ConfigFieldAccessError: If you try to access a bank-configurable field
|
|
"""
|
|
return StaticConfigProxy(_get_raw_config())
|
|
|
|
|
|
def _get_raw_config() -> HindsightConfig:
|
|
"""
|
|
Get raw config (internal use only).
|
|
|
|
INTERNAL USE ONLY. Do not use this directly in application code.
|
|
Use get_config() for static fields or ConfigResolver.resolve_full_config() for bank-specific config.
|
|
"""
|
|
global _config_cache
|
|
if _config_cache is None:
|
|
_config_cache = HindsightConfig.from_env()
|
|
return _config_cache
|
|
|
|
|
|
def clear_config_cache() -> None:
|
|
"""Clear the config cache. Useful for testing or reloading config."""
|
|
global _config_cache
|
|
_config_cache = None
|