fleet-memory/hindsight-api/hindsight_api/engine
Anton Evseev ef9d3a15cb
fix: sanitize null bytes from text fields before PostgreSQL insertion (#238)
* fix: sanitize null bytes from text fields before PostgreSQL insertion

Fixes 'invalid byte sequence for encoding UTF8: 0x00' error during batch retain

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: consolidate _sanitize_text into fact_extraction module

Address review feedback: reuse existing _sanitize_text from fact_extraction
instead of duplicating in fact_storage.

The consolidated function now handles both:
- Null bytes (\x00) for PostgreSQL compatibility
- Unicode surrogates (U+D800-U+DFFF) for UTF-8/LLM API compatibility

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-31 09:16:23 +01:00
..
consolidation Switch Vertex AI provider to native genai SDK (#242) 2026-01-30 08:35:59 +01:00
directives feat: revisit mental models, directives and reflections (#179) 2026-01-22 17:13:16 +01:00
mental_models feat: revisit mental models, directives and reflections (#179) 2026-01-22 17:13:16 +01:00
reflect fix: search_mental_models uuid type mismatch after text id migration (#225) 2026-01-28 20:04:19 +01:00
retain fix: sanitize null bytes from text fields before PostgreSQL insertion (#238) 2026-01-31 09:16:23 +01:00
search fix: graph endpoint not showing links for observations (#214) 2026-01-28 14:51:25 +01:00
__init__.py feat: extensions (#54) 2025-12-22 11:05:23 +01:00
cross_encoder.py feat: support vertex as llm provider (#233) 2026-01-29 16:13:57 -05:00
db_budget.py fix: batch queries on recall (#149) 2026-01-13 13:20:22 +01:00
db_utils.py misc: performance improvements (#140) 2026-01-09 14:47:20 +01:00
embeddings.py feat: support vertex as llm provider (#233) 2026-01-29 16:13:57 -05:00
entity_resolver.py fix: misc perf improvements (#133) 2026-01-08 22:49:04 +01:00
interface.py chore: remove dead code (#245) 2026-01-30 09:16:32 +01:00
llm_wrapper.py chore: remove dead code (#245) 2026-01-30 09:16:32 +01:00
memory_engine.py chore: remove dead code (#245) 2026-01-30 09:16:32 +01:00
query_analyzer.py fix: improve mpfp retrieval (#146) 2026-01-12 18:58:05 +01:00
response_models.py chore: remove dead code (#245) 2026-01-30 09:16:32 +01:00
task_backend.py fix: retain async fails if timestamp is set (#251) 2026-01-30 13:04:33 +01:00
utils.py chore: remove dead code (#245) 2026-01-30 09:16:32 +01:00