69dec8ec34
* feat: add comprehensive OpenTelemetry tracing - Add tool execution spans for reflect operations - Add tool call information (names, params) to spans - Change verification scope from 'test' to 'verification' - Add hindsight.reflect_generation span for done() processing - Implement no-op tracer for improved code readability - Update documentation for OTEL configuration - Resolve merge conflicts from rebase * fix: properly serialize Pydantic models in span recording - Add _serialize_for_span() helper to handle Pydantic models - Update all providers to use the helper function - Fixes test failures with 'Object of type X is not JSON serializable' * feat: add Grafana LGTM stack for unified local observability Add Grafana LGTM (Loki, Grafana, Tempo, Mimir) as the recommended local development observability stack. This provides traces, metrics, and logs in a single Docker container instead of separate tools. Changes: - Add scripts/dev/grafana/ with docker-compose and README - Add scripts/dev/start-grafana.sh startup script - Update .env.example to reference Grafana LGTM - Update configuration docs to emphasize Grafana LGTM as primary option - Reorder OTLP backend list to show Grafana LGTM first Benefits: - Single container vs multiple separate tools (Jaeger, SigNoz, etc.) - ~515MB image with full observability stack - Compatible with existing OTLP configuration - Simpler local development setup * chore: remove SigNoz scripts and references Remove SigNoz observability stack in favor of Grafana LGTM as the sole recommended local development tracing solution. Changes: - Delete scripts/dev/signoz/ directory and all SigNoz configurations - Delete scripts/dev/start-signoz.sh startup script - Remove SigNoz references from .env.example - Remove SigNoz from OTLP backends list in configuration docs Grafana LGTM provides the same capabilities (traces, metrics, logs) in a simpler single-container setup. * feat: add consolidation span hierarchy for tracing Add parent-child span structure for consolidation operations: - hindsight.consolidation: Parent span for each memory being processed - hindsight.consolidation_recall: Child span for finding related observations - LLM call span: Automatically created by LLM provider (scope="consolidation") This enables detailed timing breakdown in Grafana Tempo: - Total consolidation time per memory - Time spent in recall - Time spent in LLM call - Time spent executing actions (create/update) All consolidation tests pass (31/31). * feat: add Prometheus metrics and GenAI dashboard to Grafana stack Add comprehensive metrics and dashboarding to the Grafana LGTM stack: Metrics Collection: - Configure Prometheus to scrape Hindsight API /metrics endpoint - Scrape interval: 10 seconds - Targets hindsight-api on host.docker.internal:8888 GenAI Dashboard: - Pre-configured dashboard with 6 panels: - LLM call rate (by provider/model) - LLM call duration (p50/p95 by scope) - Token usage - input tokens/sec by scope - Token usage - output tokens/sec by scope - Operations rate (retain/recall/reflect/consolidation) - Operation duration p95 by operation type Configuration: - Mount prometheus.yml for metrics scraping - Mount dashboards directory for auto-provisioning - Add host.docker.internal mapping for container->host access - Dashboard provisioning with auto-reload every 10s Documentation: - Updated README with metrics viewing instructions - Added PromQL query examples - Documented dashboard access and navigation This provides full observability: traces (Tempo) + metrics (Prometheus/Mimir) + dashboards (Grafana) * refactor: merge Grafana setup into existing monitoring stack Consolidate the separate scripts/dev/grafana/ setup into the existing scripts/dev/monitoring/ stack, using Grafana LGTM (Loki, Grafana, Tempo, Mimir). Changes: - Remove separate scripts/dev/grafana/ directory and start-grafana.sh - Rewrite scripts/dev/monitoring/start.sh to use Docker + Grafana LGTM (was: download native Prometheus/Grafana binaries) - Add docker-compose.yaml for Grafana LGTM container - Add prometheus.yml for scraping Hindsight API metrics - Mount existing dashboards from monitoring/grafana/dashboards/ - Add comprehensive README.md Benefits: - Single unified monitoring command: ./scripts/dev/start-monitoring.sh - Uses existing dashboard files (hindsight-operations, hindsight-llm, hindsight-api-service) - Simpler setup: Docker-based vs downloading/running native binaries - Full observability: traces + metrics + logs + dashboards in one container - Standard ports: Grafana on 3000, OTLP on 4317/4318 Architecture: - Grafana LGTM container (~515MB) provides all components - Dashboards auto-provisioned from monitoring/grafana/dashboards/ - Prometheus scrapes host.docker.internal:8888/metrics - Shared hindsight-network for future service-to-service tracing * fix: run monitoring stack in foreground for easy Ctrl+C stop Change docker-compose from detached (-d) to foreground mode. Users can now stop the stack with Ctrl+C instead of needing to run docker-compose down separately. * fix: remove invalid home dashboard path and obsolete version field - Remove GF_DASHBOARDS_DEFAULT_HOME_DASHBOARD_PATH environment variable (was pointing to wrong path causing 'Failed to load home dashboard' error) - Remove obsolete 'version' field from docker-compose.yaml (docker-compose v2+ doesn't require version field) * fix: load Hindsight dashboards in Grafana LGTM Mount Hindsight dashboard JSON files and custom provisioning config to make dashboards visible in Grafana. Changes: - Mount hindsight-operations.json, hindsight-llm.json, hindsight-api-service.json to /otel-lgtm/ - Create grafana-dashboards.yaml with all dashboard providers (default + Hindsight) - Mount custom provisioning config to override LGTM default All 3 Hindsight dashboards now appear in Grafana UI with metrics from Prometheus scraping the Hindsight API /metrics endpoint. * fix: configure Prometheus to scrape Hindsight API metrics Update prometheus.yml to include both OTLP receiver config (from LGTM) and scrape_configs for pulling metrics from Hindsight API. Changes: - Mount prometheus.yml to /otel-lgtm/prometheus.yaml (where LGTM reads it) - Add scrape_configs section to pull from host.docker.internal:8888/metrics - Keep OTLP receiver configuration for trace metrics - Set scrape_interval to 5s Verified: Prometheus now successfully scrapes hindsight_llm_calls_total and other Hindsight metrics. Dashboards now show live data! * feat: add comprehensive tracing for recall and improve reflect/mental_model_refresh spans - Add recall operation tracing with parent-child span hierarchy - Parent: hindsight.recall with attributes (bank_id, query, fact_types, etc.) - Children: recall_embedding, recall_retrieval, recall_fusion, recall_rerank - Fixed context propagation using start_as_current_span() - Improve reflect tracing spans - Remove reflect_generation spans, use reflect instead - Change done() tool processing to hindsight.reflect_tool_call - Fix mental_model_refresh span nesting - Add _skip_span parameter to reflect_async to avoid duplicate hindsight.reflect spans - Mental model refresh now has clean span hierarchy without nested reflect parent - Add comprehensive tracing verification tests - Test span hierarchy and attributes for all operations - Verify parent-child relationships - 5 passing tests covering recall, reflect, consolidation, and mental_model_refresh * refactor: remove redundant is_tracing_enabled() checks - Remove all is_tracing_enabled() conditional checks before tracing calls - NoOpTracer/NoOpSpan handle disabled tracing automatically - Simplify code by always calling tracer methods directly - Fix NoOpTracer.start_as_current_span() to yield NoOpSpan instead of None Changes: - memory_engine.py: Remove 5 is_tracing_enabled checks in recall spans - agent.py: Remove 2 is_tracing_enabled checks in reflect tool spans - tracing.py: Fix NoOpTracer context manager to yield proper NoOpSpan This eliminates ~50 lines of redundant conditional code while maintaining identical behavior. * docs: simplify distributed tracing section in monitoring.md - Make tracing documentation more concise - Focus on span hierarchy and attributes - Remove verbose troubleshooting and performance sections - Keep configuration.md for env vars only
177 lines
5.2 KiB
TOML
177 lines
5.2 KiB
TOML
[build-system]
|
|
requires = ["hatchling"]
|
|
build-backend = "hatchling.build"
|
|
|
|
[project]
|
|
name = "hindsight-api"
|
|
version = "0.4.10"
|
|
description = "Hindsight: Agent Memory That Works Like Human Memory"
|
|
readme = "README.md"
|
|
requires-python = ">=3.11"
|
|
dependencies = [
|
|
"asyncpg>=0.29.0",
|
|
"python-dotenv>=1.0.0",
|
|
"openai>=1.0.0",
|
|
"pydantic>=2.0.0",
|
|
"rich>=13.0.0",
|
|
"langchain-text-splitters>=0.3.0",
|
|
"fastapi[standard]>=0.120.3",
|
|
"uvicorn>=0.38.0",
|
|
"wsproto>=1.0.0",
|
|
"sqlalchemy>=2.0.44",
|
|
"alembic>=1.17.1",
|
|
"pgvector>=0.4.1",
|
|
"greenlet>=3.2.4",
|
|
"psycopg2-binary>=2.9.11",
|
|
"tiktoken>=0.12.0",
|
|
"httpx>=0.27.0",
|
|
"PyJWT[crypto]>=2.8.0",
|
|
"fastmcp>=2.14.0", # CVE-2025-66416
|
|
"pg0-embedded>=0.11.0",
|
|
"python-dateutil>=2.8.0",
|
|
"opentelemetry-api>=1.20.0",
|
|
"opentelemetry-sdk>=1.20.0",
|
|
"opentelemetry-instrumentation-fastapi>=0.41b0",
|
|
"opentelemetry-exporter-prometheus>=0.41b0",
|
|
"opentelemetry-exporter-otlp-proto-http>=1.20.0",
|
|
"opentelemetry-semantic-conventions>=0.41b0",
|
|
"dateparser>=1.2.2",
|
|
"google-genai>=1.0.0",
|
|
"google-auth>=2.0.0",
|
|
"anthropic>=0.40.0",
|
|
"typer>=0.9.0",
|
|
"cohere>=5.0.0",
|
|
"flashrank>=0.2.0",
|
|
# Local ML models for embeddings/reranking - can be excluded in Docker with INCLUDE_LOCAL_MODELS=false
|
|
"sentence-transformers>=3.3.0",
|
|
"transformers>=4.53.0", # Security fixes for ReDoS vulnerabilities
|
|
"torch>=2.6.0", # CVE fix for remote code execution
|
|
"uvloop>=0.22.1",
|
|
# Transitive dependency security fixes
|
|
"pyasn1>=0.6.2", # DoS vulnerability fix
|
|
"urllib3>=2.6.3", # Decompression-bomb safeguards bypass fix
|
|
"langchain-core>=1.2.5", # Serialization injection vulnerability fix
|
|
"filelock>=3.20.1", # TOCTOU race condition fix
|
|
"authlib>=1.6.6", # Account takeover vulnerability fix
|
|
"aiohttp>=3.13.3", # Multiple DoS vulnerabilities
|
|
"claude-agent-sdk>=0.1.27",
|
|
]
|
|
|
|
[project.optional-dependencies]
|
|
test = [
|
|
"pytest>=7.0.0",
|
|
"pytest-asyncio>=0.21.0",
|
|
"pytest-timeout>=2.4.0",
|
|
"pytest-xdist>=3.0.0",
|
|
"filelock>=3.20.1", # TOCTOU race condition fix
|
|
]
|
|
|
|
[project.scripts]
|
|
hindsight-api = "hindsight_api.main:main"
|
|
hindsight-worker = "hindsight_api.worker.main:main"
|
|
hindsight-local-mcp = "hindsight_api.mcp_local:main"
|
|
hindsight-admin = "hindsight_api.admin.cli:main"
|
|
|
|
[tool.hatch.build.targets.wheel]
|
|
packages = ["hindsight_api"]
|
|
|
|
[tool.hatch.build.targets.wheel.sources]
|
|
"hindsight_api" = "hindsight_api"
|
|
|
|
[tool.hatch.build.targets.sdist]
|
|
include = [
|
|
"hindsight_api/**/*",
|
|
]
|
|
|
|
[tool.hatch.build]
|
|
include = [
|
|
"hindsight_api/**/*.py",
|
|
"hindsight_api/alembic/**/*",
|
|
]
|
|
|
|
[tool.pytest.ini_options]
|
|
log_cli = true
|
|
log_cli_level = "INFO"
|
|
log_cli_format = "%(asctime)s - %(levelname)s - %(name)s - %(message)s"
|
|
log_cli_date_format = "%Y-%m-%d %H:%M:%S"
|
|
addopts = "--timeout 120 -n 8 --dist loadgroup --durations=10 -v"
|
|
asyncio_mode = "auto"
|
|
asyncio_default_fixture_loop_scope = "function"
|
|
log_auto_indent = true
|
|
filterwarnings = [
|
|
"ignore:The @wait_container_is_ready decorator is deprecated:DeprecationWarning",
|
|
"ignore::RuntimeWarning:asyncio",
|
|
]
|
|
|
|
[dependency-groups]
|
|
dev = [
|
|
"pytest>=9.0.0",
|
|
"pytest-asyncio>=1.3.0",
|
|
"pytest-timeout>=2.4.0",
|
|
"pytest-xdist>=3.8.0",
|
|
"python-dotenv>=1.2.1",
|
|
"filelock>=3.20.1", # TOCTOU race condition fix
|
|
"ruff>=0.8.0",
|
|
"ty>=0.0.1",
|
|
]
|
|
|
|
[tool.ruff]
|
|
line-length = 120
|
|
target-version = "py311"
|
|
exclude = [
|
|
"tests/",
|
|
"**/tests/",
|
|
]
|
|
|
|
[tool.ruff.lint]
|
|
select = [
|
|
"E", # pycodestyle errors
|
|
"W", # pycodestyle warnings
|
|
"F", # Pyflakes
|
|
"I", # isort
|
|
]
|
|
ignore = [
|
|
"E501", # line too long (handled by formatter)
|
|
"E402", # module import not at top of file
|
|
"F401", # unused import (too noisy during development)
|
|
"F841", # unused variable (too noisy during development)
|
|
"F811", # redefined while unused
|
|
"F821", # undefined name (forward references in type hints)
|
|
]
|
|
|
|
[tool.ruff.lint.isort]
|
|
known-third-party = ["alembic"]
|
|
|
|
[tool.ruff.format]
|
|
quote-style = "double"
|
|
indent-style = "space"
|
|
|
|
[tool.uv]
|
|
# Allow uv to search all configured indexes for packages, not just the first one
|
|
# This prevents dependency resolution failures when using pytorch index + PyPI
|
|
index-strategy = "unsafe-best-match"
|
|
|
|
[tool.ty]
|
|
# Type checking configuration
|
|
# ty is an extremely fast Python type checker from Astral (same team as ruff/uv)
|
|
|
|
[tool.ty.environment]
|
|
python-version = "3.11"
|
|
|
|
[tool.ty.src]
|
|
exclude = [
|
|
"tests/",
|
|
"hindsight_api/alembic/",
|
|
]
|
|
|
|
[tool.ty.rules]
|
|
# Disable noisy rules while keeping important ones
|
|
invalid-argument-type = "ignore" # False positives with **kwargs patterns
|
|
invalid-return-type = "ignore" # Often intentional in async code
|
|
invalid-parameter-default = "ignore" # Optional params with None default
|
|
possibly-missing-attribute = "ignore" # Common with Optional types
|
|
invalid-raise = "ignore" # False positives with exception tracking
|
|
call-non-callable = "ignore" # False positives with Optional types
|
|
invalid-key = "ignore" # Pydantic ConfigDict not understood
|
|
invalid-method-override = "ignore" # Intentional signature differences
|
|
unresolved-reference = "ignore" # Forward references not always resolved
|