ad2cf72aab
* perf: add GIN index on source_memory_ids for observation lookup Addresses a 927x performance regression (45ms → 0.049ms) reported by a user with ~77k observations. The array overlap operator (&&) on source_memory_ids was doing a full sequential scan over all observations, causing recall timeouts (57-64s) and slow user recall (18-27s avg). The partial GIN index reduces consolidation recall from timeout to ~15s and user recall to ~6s. * fix: use pre-bounded memory_links for observation graph expansion Replace raw unit_entities join in _expand_observations() with the same memory_links entity graph used by non-observation fact types. The previous approach joined unit_entities twice (seeds→entities→connected_sources), which explodes at scale (30-70s at 100k observations). The LIMIT 500 workaround was non-deterministic and dropped valid results. Using memory_links (pre-bounded to MAX_LINKS_PER_ENTITY=50 at retain time) is algorithmically identical to the non-observation entity expansion and keeps graph retrieval at ~2s p50 even at 100k observations. Also fix migration down_revision (z1u2v3w4x5y6 → d2e3f4a5b6c7) and add observation generation + fact-type filtering to the recall perf benchmark. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| fusion.py | ||
| graph_retrieval.py | ||
| link_expansion_retrieval.py | ||
| mpfp_retrieval.py | ||
| reranking.py | ||
| retrieval.py | ||
| tags.py | ||
| temporal_extraction.py | ||
| think_utils.py | ||
| trace.py | ||
| tracer.py | ||
| types.py | ||