fleet-memory/hindsight-api-slim/tests/test_reranker_error_handling.py
Nicolò Boschi 15ea23d5d6
feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560)
* feat: introduce hindsight-api-slim and hindsight-all-slim packages

Closes #552

- Move all source code from hindsight-api/ to new hindsight-api-slim/
- hindsight-api-slim has heavy ML deps (torch, sentence-transformers,
  transformers, einops, flashrank, mlx, mlx-lm, safetensors) and
  pg0-embedded as optional extras: [local-ml], [embedded-db], [all]
- hindsight-api becomes a zero-code meta-package depending on
  hindsight-api-slim[all] for full backward compatibility
- Add hindsight-all-slim meta-package: hindsight-api-slim + client + embed
- hindsight-all updated to depend on hindsight-api-slim[all]
- pg0.py: lazy-import pg0 with clear ImportError pointing to [embedded-db]
- Dockerfile: replace sed hack with proper uv sync --extra flags
- Update release.yml, test.yml, lint.sh, release.sh, CLAUDE.md and
  all path references throughout the repo

* refactor: rename hindsight/ directory to hindsight-all/

* docs: document hindsight-api-slim and hindsight-all-slim package variants

Add package variants table and extras explanation to installation.md

* docs: remove emojis from installation.md, use professional tone

* docs: link Docker slim variant to pip package variants section

* docs: consolidate Docker image variants into single table

* ci: fix working-directory paths after package restructure

- Replace all hindsight-api → hindsight-api-slim in test.yml
- Replace hindsight → hindsight-all in test.yml
- Add --extra embedded-db to test-embed API install step

* ci: add local-ml and embedded-db extras to API sync steps

These extras were previously implicit in the old hindsight-api package
(which bundled everything). Now that hindsight-api-slim uses optional
extras, we must explicitly request local-ml and embedded-db in CI.

* ci: add API install step with embedded-db to test-embed smoke test

The smoke test starts hindsight-api as a daemon, which requires pg0-embedded.
Add a dedicated install step for hindsight-api-slim with embedded-db extra
so the daemon can start successfully.

* ci: remove --no-install-project when using optional extras

When --no-install-project is combined with --extra, the optional deps
are not installed because extras require the project to be active.
Remove --no-install-project from steps that need local-ml or embedded-db.

* ci: fix ordering of uv sync steps to preserve optional extras

When uv sync runs for a different workspace member, it removes optional
extras installed for other members. Fix by always running extra-requiring
API sync last, after other workspace member syncs.

Also remove --no-install-project from embedded-db sync in test-embed,
as --no-install-project prevents optional extras from being active.

* ci: add local-ml extra to test-embed API install for smoke test

The smoke test starts the full API server which needs sentence-transformers
for local embeddings (default provider). Add local-ml extra to the install.

* ci: simplify extras with --all-extras and add slim pip smoke test

- Replace explicit --extra local-ml --extra embedded-db with --all-extras
  for cleaner, more maintainable sync steps
- Add test-pip-slim job: tests hindsight-api-slim[embedded-db] without
  local ML models, using Cohere for embeddings/reranking (mirrors Docker
  slim smoke test approach)

* ci: simplify slim smoke test to health check only (mirrors Docker test)
2026-03-13 13:50:03 +01:00

71 lines
2.7 KiB
Python

"""
Regression test for UnboundLocalError in recall when the reranker raises.
Before the fix, `scored_results` and `pre_filtered_count` were only assigned
inside the `try` block, but referenced in the `finally` block. If
`reranker_instance.rerank()` (or `ensure_initialized()`) raised, the `finally`
block crashed with `UnboundLocalError` instead of propagating the original
exception.
Fix: initialise both variables to safe defaults before the try/finally block.
"""
from datetime import datetime, timezone
from unittest.mock import AsyncMock, patch
import pytest
@pytest.mark.asyncio
async def test_recall_reranker_error_does_not_raise_unbound_local(memory, request_context):
"""Recall must propagate the reranker's exception, not an UnboundLocalError."""
bank_id = f"test_reranker_err_{datetime.now(timezone.utc).timestamp()}"
try:
await memory.retain_async(
bank_id=bank_id,
content="Paris is the capital of France",
request_context=request_context,
)
# Simulate a reranker failure (e.g. Cohere API error on empty/small candidate set)
rerank_mock = AsyncMock(side_effect=RuntimeError("reranker API error"))
memory._cross_encoder_reranker._initialized = True # skip ensure_initialized
with patch.object(memory._cross_encoder_reranker, "rerank", rerank_mock):
with pytest.raises(Exception, match="reranker API error"):
await memory.recall_async(
bank_id=bank_id,
query="capital of France",
request_context=request_context,
)
finally:
await memory.delete_bank(bank_id, request_context=request_context)
@pytest.mark.asyncio
async def test_recall_reranker_init_error_does_not_raise_unbound_local(memory, request_context):
"""Same regression when ensure_initialized() raises (before pre_filtered_count is set)."""
bank_id = f"test_reranker_init_err_{datetime.now(timezone.utc).timestamp()}"
try:
await memory.retain_async(
bank_id=bank_id,
content="Paris is the capital of France",
request_context=request_context,
)
init_mock = AsyncMock(side_effect=RuntimeError("reranker init failed"))
memory._cross_encoder_reranker._initialized = False
with patch.object(memory._cross_encoder_reranker, "ensure_initialized", init_mock):
with pytest.raises(Exception, match="reranker init failed"):
await memory.recall_async(
bank_id=bank_id,
query="capital of France",
request_context=request_context,
)
finally:
await memory.delete_bank(bank_id, request_context=request_context)