fleet-memory/hindsight-api-slim/hindsight_api/alembic/versions
Nicolò Boschi f09ad9deac
fix(migration): backsweep orphaned observation memory units (#584)
* fix(migration): backsweep orphaned observation memory units

Delete observation rows whose every source_memory_id points to a
deleted memory unit, left behind before PR #580 fixed the chunk FK
cascade and before delete_document() called
_delete_stale_observations_for_memories.

Closes #572 (data cleanup for pre-existing installs).

* fix(migration): broaden backsweep to cover all fact types and bank-level orphans

- Pass 1: delete any memory_units row (all fact_types) whose bank_id no
  longer exists in banks — catches orphans from bank deletions that
  predate a FK cascade between the two tables.
- Pass 2: delete observation rows whose every source_memory_id points to
  a deleted memory unit, regardless of document_id/chunk_id anchors.

* test(migration): verify backsweep removes orphans and preserves legit rows

Adds a focused migration test that:
- Starts a fresh pg0 instance at revision f6g7h8i9j0k1
- Seeds orphaned rows for both backsweep passes (ghost-bank + all-dead-sources)
- Seeds legitimate rows that must survive
- Applies the backsweep migration to head
- Asserts the expected rows are deleted/preserved
2026-03-16 14:06:33 +01:00
..
5a366d414dce_initial_schema.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
a1b2c3d4e5f6_add_file_storage_table.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
a2b3c4d5e6f7_add_text_signals_column.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
a2b3c4d5e6f8_add_gin_index_source_memory_ids.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
aa2b3c4d5e6f_nullable_event_date.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
b3c4d5e6f7g8_add_temporal_date_indexes.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
b4c5d6e7f8a9_backfill_observation_scopes.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
b7c4d8e9f1a2_add_chunks_table.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
c1a2b3d4e5f6_enable_pg_trgm_and_entities_trgm_index.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
c3d4e5f6g7h8_add_history_to_mental_models.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
c8e5f2a3b4d1_add_retain_params_to_documents.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
d2e3f4a5b6c7_add_memory_links_expansion_indexes.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
d4e5f6g7h8i9_gin_source_memory_ids_fastupdate_off.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
d5e6f7a8b9c0_add_bank_internal_id_and_per_bank_hnsw.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
d9f6a3b4c5e2_rename_bank_to_interactions.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
e0a1b2c3d4e5_disposition_to_3_traits.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
e4f5a6b7c8d9_add_webhooks_tables.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
e5f6g7h8i9j0_cascade_delete_ops_on_bank_delete.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
f1a2b3c4d5e6_add_memory_links_composite_index.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
f6g7h8i9j0k1_chunk_fk_cascade_delete.py fix: change chunk FK to CASCADE so doc deletion removes linked memory units (#580) 2026-03-16 12:24:22 +01:00
f7g8h9i0j1k2_add_webhook_http_config.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
g2a3b4c5d6e7_add_tags_column.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
g7h8i9j0k1l2_backsweep_orphan_observations.py fix(migration): backsweep orphaned observation memory units (#584) 2026-03-16 14:06:33 +01:00
h3c4d5e6f7g8_mental_models_v4.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
i4d5e6f7g8h9_delete_opinions.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
j5e6f7g8h9i0_mental_model_versions.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
k6f7g8h9i0j1_add_directive_subtype.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
l7g8h9i0j1k2_add_worker_columns.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
m8h9i0j1k2l3_mental_model_id_to_text.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
n9i0j1k2l3m4_learnings_and_pinned_reflections.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
o0j1k2l3m4n5_migrate_mental_models_data.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
p1k2l3m4n5o6_new_knowledge_architecture.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
q2l3m4n5o6p7_fix_mental_model_fact_type.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
r3m4n5o6p7q8_add_reflect_response_to_reflections.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
rename_personality_to_disposition.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
s4n5o6p7q8r9_add_consolidated_at_to_memory_units.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
t5o6p7q8r9s0_rename_mental_models_to_observations.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
u6p7q8r9s0t1_mental_models_text_id.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
v7q8r9s0t1u2_add_max_tokens_to_mental_models.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
w8r9s0t1u2v3_fix_mental_models_pk_isolation.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
x9s0t1u2v3w4_add_bank_config_column.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
y0t1u2v3w4x5_add_result_metadata_gin_index.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00
z1u2v3w4x5y6_add_observation_tags_to_memory_units.py feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) 2026-03-13 13:50:03 +01:00