fleet-memory/hindsight-api-slim/hindsight_api/engine
Nicolò Boschi 083295dc6f
feat: add audit log for feature usage tracking (#717)
* feat: add audit log for feature usage tracking

Add full auditability for all mutating and core API operations across
HTTP, MCP, and system (worker) transports. Audit entries record raw
request/response as JSONB, timing (started_at/ended_at), action, and
transport type.

Backend:
- New audit_log table with JSONB columns for expandability without
  future migrations (merge migration of 3 existing heads)
- AuditLogger with fire-and-forget writes via asyncio.create_task
- @audited decorator on 28 HTTP route handlers
- MCP tool audit wrapping for 16 auditable tools
- Worker task execution wrapped with audit_context
- List endpoint with action, transport, date range filters + pagination
- Stats endpoint with per-day counts for charting
- Configurable retention sweep (concurrent-safe DELETE)

Config (env-only, static):
- HINDSIGHT_API_AUDIT_LOG_ENABLED (default: false)
- HINDSIGHT_API_AUDIT_LOG_ACTIONS (comma-separated allowlist, empty=all)
- HINDSIGHT_API_AUDIT_LOG_RETENTION_DAYS (default: -1, keep forever)

Control Plane:
- New "Audit Logs" tab on bank configuration page
- Line chart showing request volume (today/7d/30d) with action filter
- Filterable table with action, transport, date range filters
- Paginated list with detail dialog showing raw request/response JSON

Tests:
- 13 tests covering list, filters, pagination, stats, disabled mode,
  action allowlist, and ordering

* fix: split 3-way merge migration into two 2-way merges

Alembic doesn't support 3-parent merge migrations. Split into a no-op
merge of 2 heads (b1c2d3e4f5g6) followed by the audit_log table
migration merging the third head.

* fix: correct merge migration to merge actual 2 heads

The original analysis incorrectly identified 3 heads. There were only 2
(a3b4c5d6e7f8 and c8e5f2a3b4d1). Remove the unnecessary intermediate
merge migration and fix the audit_log migration to merge these 2 heads.

* fix: use 'heads' instead of 'head' in migration runner

Alembic's upgrade('head') fails when multiple heads exist (e.g. from
namespace package overlaps between hindsight-api and hindsight-api-slim).
Using 'heads' (plural) handles this gracefully by upgrading all branches.

* chore: regenerate OpenAPI spec with audit log endpoints

* chore: regenerate TypeScript client and docs skill OpenAPI spec

Python and Go clients still need regeneration (requires Docker).

* chore: regenerate all client SDKs (Python, Go, TypeScript)

Adds generated audit log API clients for Python (audit_api.py),
Go (api_audit.go), and TypeScript client type updates.
2026-03-27 09:52:03 +01:00
..
consolidation fix: prevent silent memory loss on consolidation LLM failure (#601) 2026-03-17 20:15:33 +01:00
directives feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
parsers feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
providers feat: add support for ark and volcano LLM providers (#714) 2026-03-26 17:14:13 +01:00
reflect feat(reflect): make source facts in search_observations configurable (#688) 2026-03-25 17:54:49 +01:00
retain docs(python-client): async-first pydoc + low-level API access + missing params (#711) 2026-03-26 16:09:58 +01:00
search fix: control plane UI fixes for recall and data view (#693) 2026-03-25 18:42:57 +01:00
storage feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
__init__.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
audit.py feat: add audit log for feature usage tracking (#717) 2026-03-27 09:52:03 +01:00
cross_encoder.py docs: add config vars for local reranker FP16 and bucket batching (#589) 2026-03-16 17:35:09 +01:00
db_budget.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
db_utils.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
embeddings.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
entity_resolver.py fix(entity_resolver): prevent _pending_stats/_pending_cooccurrences memory leak (#662) 2026-03-23 16:06:04 +01:00
interface.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
jina_mlx_reranker.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
llm_interface.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
llm_wrapper.py feat: add support for ark and volcano LLM providers (#714) 2026-03-26 17:14:13 +01:00
memory_engine.py feat: add audit log for feature usage tracking (#717) 2026-03-27 09:52:03 +01:00
operation_metadata.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
query_analyzer.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
response_models.py fix: return metadata in recall responses (#680) 2026-03-25 11:24:18 +01:00
task_backend.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
utils.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00