fleet-memory/hindsight-api/hindsight_api/engine
Nicolò Boschi 0560f6260d
fix: cancel in-flight async ops when bank is deleted (#545)
* fix: cancel async ops on bank delete via CASCADE FK + heartbeat checkpoints

- Add migration e5f6g7h8i9j0: FK ON DELETE CASCADE from async_operations
  and webhooks to banks, so deleting a bank auto-removes all its ops/webhooks
- Add _check_op_alive() helper: returns False if op row was deleted (cascade)
- Add consolidation checkpoint: after each LLM batch commit, abort early if
  op was deleted mid-run (returns status='cancelled')
- Add retain checkpoint: between sub-batches, abort early if op was deleted
- _mark_operation_completed/failed/completed_and_fire_webhook: gracefully
  handle missing row (UPDATE 0) with log instead of silent error
- Thread operation_id into run_consolidation_job() for checkpoint access
- Fix y0t1u2v3w4x5 and a1b2c3d4e5f6 migrations: add IF NOT EXISTS to prevent
  failure on idempotent re-runs
- Add 10 tests covering cascade delete, _check_op_alive, graceful mark methods,
  consolidation checkpoint, and retain checkpoint

* refactor: use RETURNING + fetchrow instead of execute + string comparison

* fix: add bank upsert before async_operations FK inserts and update tests

- memory_engine.py: upsert bank in submit_async_retain before async_operations INSERT
- http.py: upsert bank in api_create_webhook before webhooks INSERT
- test_worker.py, test_async_batch_retain.py, test_webhooks.py: add _ensure_bank
  helper calls before direct async_operations/webhooks inserts to satisfy FK constraint

* fix: mock bank_utils.get_bank_profile in unit test with mocked pool
2026-03-12 09:39:31 +01:00
..
consolidation fix: cancel in-flight async ops when bank is deleted (#545) 2026-03-12 09:39:31 +01:00
directives feat: revisit mental models, directives and reflections (#179) 2026-01-22 17:13:16 +01:00
parsers Fix Iris parser httpx read timeout for file uploads (#524) 2026-03-09 11:22:44 +01:00
providers fix: normalize named tool_choice to required + filtered tools for OpenAI-compatible providers (#528) 2026-03-09 15:47:51 +01:00
reflect feat: entity labels — optional, free_values, multi_value, UI polish (#450) 2026-03-02 13:05:25 +01:00
retain perf: replace window-function retrieval with UNION ALL + per-bank HNSW indexes (#541) 2026-03-11 12:09:50 +01:00
search perf: replace window-function retrieval with UNION ALL + per-bank HNSW indexes (#541) 2026-03-11 12:09:50 +01:00
storage Fix GCS auth for Workload Identity Federation credentials (#518) 2026-03-07 08:59:51 +01:00
__init__.py feat: extensions (#54) 2025-12-22 11:05:23 +01:00
cross_encoder.py feat: add jina-mlx reranker provider for Apple Silicon (#542) 2026-03-11 15:15:58 +01:00
db_budget.py fix: batch queries on recall (#149) 2026-01-13 13:20:22 +01:00
db_utils.py fix: strip null bytes from parsed file content before retain (#535) 2026-03-10 16:24:11 +01:00
embeddings.py fix: pass encoding_format="float" in LiteLLM embedding calls (#434) 2026-02-25 10:32:05 +01:00
entity_resolver.py fix(consolidation): respect bank mission over ephemeral-state heuristic (#525) 2026-03-09 15:04:36 +01:00
interface.py feat: add tags filtering and q description fix for list documents API (#468) 2026-03-02 17:03:16 +01:00
jina_mlx_reranker.py feat: add jina-mlx reranker provider for Apple Silicon (#542) 2026-03-11 15:15:58 +01:00
llm_interface.py feat: support Batch API for retain (openai/groq) (#365) 2026-02-16 13:31:50 +01:00
llm_wrapper.py fix(consolidation): respect bank mission over ephemeral-state heuristic (#525) 2026-03-09 15:04:36 +01:00
memory_engine.py fix: cancel in-flight async ops when bank is deleted (#545) 2026-03-12 09:39:31 +01:00
operation_metadata.py fix: improve async batch retain with large payloads (#366) 2026-02-16 12:51:42 +01:00
query_analyzer.py fix(performance): improve recall and retain performance on large banks (#469) 2026-03-03 13:35:22 +01:00
response_models.py feat: include source facts in observation recall (#404) 2026-02-19 14:54:19 +01:00
task_backend.py fix: retain async fails if timestamp is set (#251) 2026-01-30 13:04:33 +01:00
utils.py feat: implement hierarchical configuration (system, tenant, bank) (#329) 2026-02-12 13:14:57 +01:00