From 15ea23d5d6229838b3ee97dc07e86552762aa29e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nicol=C3=B2=20Boschi?= Date: Fri, 13 Mar 2026 13:50:03 +0100 Subject: [PATCH] feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat: introduce hindsight-api-slim and hindsight-all-slim packages Closes #552 - Move all source code from hindsight-api/ to new hindsight-api-slim/ - hindsight-api-slim has heavy ML deps (torch, sentence-transformers, transformers, einops, flashrank, mlx, mlx-lm, safetensors) and pg0-embedded as optional extras: [local-ml], [embedded-db], [all] - hindsight-api becomes a zero-code meta-package depending on hindsight-api-slim[all] for full backward compatibility - Add hindsight-all-slim meta-package: hindsight-api-slim + client + embed - hindsight-all updated to depend on hindsight-api-slim[all] - pg0.py: lazy-import pg0 with clear ImportError pointing to [embedded-db] - Dockerfile: replace sed hack with proper uv sync --extra flags - Update release.yml, test.yml, lint.sh, release.sh, CLAUDE.md and all path references throughout the repo * refactor: rename hindsight/ directory to hindsight-all/ * docs: document hindsight-api-slim and hindsight-all-slim package variants Add package variants table and extras explanation to installation.md * docs: remove emojis from installation.md, use professional tone * docs: link Docker slim variant to pip package variants section * docs: consolidate Docker image variants into single table * ci: fix working-directory paths after package restructure - Replace all hindsight-api → hindsight-api-slim in test.yml - Replace hindsight → hindsight-all in test.yml - Add --extra embedded-db to test-embed API install step * ci: add local-ml and embedded-db extras to API sync steps These extras were previously implicit in the old hindsight-api package (which bundled everything). Now that hindsight-api-slim uses optional extras, we must explicitly request local-ml and embedded-db in CI. * ci: add API install step with embedded-db to test-embed smoke test The smoke test starts hindsight-api as a daemon, which requires pg0-embedded. Add a dedicated install step for hindsight-api-slim with embedded-db extra so the daemon can start successfully. * ci: remove --no-install-project when using optional extras When --no-install-project is combined with --extra, the optional deps are not installed because extras require the project to be active. Remove --no-install-project from steps that need local-ml or embedded-db. * ci: fix ordering of uv sync steps to preserve optional extras When uv sync runs for a different workspace member, it removes optional extras installed for other members. Fix by always running extra-requiring API sync last, after other workspace member syncs. Also remove --no-install-project from embedded-db sync in test-embed, as --no-install-project prevents optional extras from being active. * ci: add local-ml extra to test-embed API install for smoke test The smoke test starts the full API server which needs sentence-transformers for local embeddings (default provider). Add local-ml extra to the install. * ci: simplify extras with --all-extras and add slim pip smoke test - Replace explicit --extra local-ml --extra embedded-db with --all-extras for cleaner, more maintainable sync steps - Add test-pip-slim job: tests hindsight-api-slim[embedded-db] without local ML models, using Cohere for embeddings/reranking (mirrors Docker slim smoke test approach) * ci: simplify slim smoke test to health check only (mirrors Docker test) --- .github/workflows/release.yml | 34 ++- .github/workflows/test.yml | 186 ++++++++++------ CLAUDE.md | 28 +-- docker/standalone/Dockerfile | 23 +- {hindsight => hindsight-all-slim}/README.md | 0 hindsight-all-slim/pyproject.toml | 33 +++ hindsight-all/README.md | 48 +++++ .../hindsight/__init__.py | 0 .../hindsight/api_namespaces.py | 0 .../hindsight/client_wrapper.py | 0 .../hindsight/embedded.py | 0 .../hindsight/server.py | 0 {hindsight => hindsight-all}/pyproject.toml | 4 +- {hindsight => hindsight-all}/tests/README.md | 0 .../tests/__init__.py | 0 .../tests/test_embedded.py | 0 .../tests/test_embedded_namespaces.py | 0 .../tests/test_server_integration.py | 0 hindsight-api-slim/README.md | 137 ++++++++++++ .../hindsight_api/__init__.py | 0 .../hindsight_api/admin/__init__.py | 0 .../hindsight_api/admin/cli.py | 0 .../hindsight_api/alembic/README | 0 .../hindsight_api/alembic/env.py | 0 .../hindsight_api/alembic/script.py.mako | 0 .../versions/5a366d414dce_initial_schema.py | 0 .../a1b2c3d4e5f6_add_file_storage_table.py | 0 .../a2b3c4d5e6f7_add_text_signals_column.py | 0 ...4d5e6f8_add_gin_index_source_memory_ids.py | 0 .../aa2b3c4d5e6f_nullable_event_date.py | 0 .../b3c4d5e6f7g8_add_temporal_date_indexes.py | 0 ...4c5d6e7f8a9_backfill_observation_scopes.py | 0 .../versions/b7c4d8e9f1a2_add_chunks_table.py | 0 ..._enable_pg_trgm_and_entities_trgm_index.py | 0 ...d4e5f6g7h8_add_history_to_mental_models.py | 0 ...f2a3b4d1_add_retain_params_to_documents.py | 0 ...b6c7_add_memory_links_expansion_indexes.py | 0 ...i9_gin_source_memory_ids_fastupdate_off.py | 0 ..._add_bank_internal_id_and_per_bank_hnsw.py | 0 ...9f6a3b4c5e2_rename_bank_to_interactions.py | 0 .../e0a1b2c3d4e5_disposition_to_3_traits.py | 0 .../e4f5a6b7c8d9_add_webhooks_tables.py | 0 ...8i9j0_cascade_delete_ops_on_bank_delete.py | 0 ...c4d5e6_add_memory_links_composite_index.py | 0 .../f7g8h9i0j1k2_add_webhook_http_config.py | 0 .../versions/g2a3b4c5d6e7_add_tags_column.py | 0 .../versions/h3c4d5e6f7g8_mental_models_v4.py | 0 .../versions/i4d5e6f7g8h9_delete_opinions.py | 0 .../j5e6f7g8h9i0_mental_model_versions.py | 0 .../k6f7g8h9i0j1_add_directive_subtype.py | 0 .../l7g8h9i0j1k2_add_worker_columns.py | 0 .../m8h9i0j1k2l3_mental_model_id_to_text.py | 0 ...k2l3m4_learnings_and_pinned_reflections.py | 0 ...o0j1k2l3m4n5_migrate_mental_models_data.py | 0 ...p1k2l3m4n5o6_new_knowledge_architecture.py | 0 ...q2l3m4n5o6p7_fix_mental_model_fact_type.py | 0 ...7q8_add_reflect_response_to_reflections.py | 0 .../rename_personality_to_disposition.py | 0 ...8r9_add_consolidated_at_to_memory_units.py | 0 ...s0_rename_mental_models_to_observations.py | 0 .../u6p7q8r9s0t1_mental_models_text_id.py | 0 ...9s0t1u2_add_max_tokens_to_mental_models.py | 0 ...s0t1u2v3_fix_mental_models_pk_isolation.py | 0 .../x9s0t1u2v3w4_add_bank_config_column.py | 0 ...1u2v3w4x5_add_result_metadata_gin_index.py | 0 ...y6_add_observation_tags_to_memory_units.py | 0 .../hindsight_api/api/__init__.py | 0 .../hindsight_api/api/http.py | 0 .../hindsight_api/api/mcp.py | 0 .../hindsight_api/banner.py | 0 .../hindsight_api/config.py | 0 .../hindsight_api/config_resolver.py | 0 .../hindsight_api/daemon.py | 0 .../hindsight_api/engine/__init__.py | 0 .../engine/consolidation/__init__.py | 0 .../engine/consolidation/consolidator.py | 0 .../engine/consolidation/prompts.py | 0 .../hindsight_api/engine/cross_encoder.py | 0 .../hindsight_api/engine/db_budget.py | 0 .../hindsight_api/engine/db_utils.py | 0 .../engine/directives/__init__.py | 0 .../hindsight_api/engine/directives/models.py | 0 .../hindsight_api/engine/embeddings.py | 0 .../hindsight_api/engine/entity_resolver.py | 0 .../hindsight_api/engine/interface.py | 0 .../hindsight_api/engine/jina_mlx_reranker.py | 0 .../hindsight_api/engine/llm_interface.py | 0 .../hindsight_api/engine/llm_wrapper.py | 0 .../hindsight_api/engine/memory_engine.py | 0 .../engine/operation_metadata.py | 0 .../hindsight_api/engine/parsers/__init__.py | 0 .../hindsight_api/engine/parsers/base.py | 0 .../hindsight_api/engine/parsers/iris.py | 0 .../engine/parsers/markitdown.py | 0 .../engine/providers/__init__.py | 0 .../engine/providers/anthropic_llm.py | 0 .../engine/providers/claude_code_llm.py | 0 .../engine/providers/codex_llm.py | 0 .../engine/providers/gemini_llm.py | 0 .../engine/providers/mock_llm.py | 0 .../engine/providers/openai_compatible_llm.py | 0 .../hindsight_api/engine/query_analyzer.py | 0 .../hindsight_api/engine/reflect/__init__.py | 0 .../hindsight_api/engine/reflect/agent.py | 0 .../hindsight_api/engine/reflect/models.py | 0 .../engine/reflect/observations.py | 0 .../hindsight_api/engine/reflect/prompts.py | 0 .../hindsight_api/engine/reflect/tools.py | 0 .../engine/reflect/tools_schema.py | 0 .../hindsight_api/engine/response_models.py | 0 .../hindsight_api/engine/retain/__init__.py | 0 .../hindsight_api/engine/retain/bank_utils.py | 0 .../engine/retain/chunk_storage.py | 0 .../engine/retain/embedding_processing.py | 0 .../engine/retain/embedding_utils.py | 0 .../engine/retain/entity_labels.py | 0 .../engine/retain/entity_processing.py | 0 .../engine/retain/fact_extraction.py | 0 .../engine/retain/fact_storage.py | 0 .../engine/retain/link_creation.py | 0 .../hindsight_api/engine/retain/link_utils.py | 0 .../engine/retain/orchestrator.py | 0 .../hindsight_api/engine/retain/types.py | 0 .../hindsight_api/engine/search/__init__.py | 0 .../hindsight_api/engine/search/fusion.py | 0 .../engine/search/graph_retrieval.py | 0 .../engine/search/link_expansion_retrieval.py | 0 .../engine/search/mpfp_retrieval.py | 0 .../hindsight_api/engine/search/reranking.py | 0 .../hindsight_api/engine/search/retrieval.py | 0 .../hindsight_api/engine/search/tags.py | 0 .../engine/search/temporal_extraction.py | 0 .../engine/search/think_utils.py | 0 .../hindsight_api/engine/search/trace.py | 0 .../hindsight_api/engine/search/tracer.py | 0 .../hindsight_api/engine/search/types.py | 0 .../hindsight_api/engine/storage/__init__.py | 0 .../hindsight_api/engine/storage/azure.py | 0 .../hindsight_api/engine/storage/base.py | 0 .../hindsight_api/engine/storage/gcs.py | 0 .../engine/storage/postgresql.py | 0 .../hindsight_api/engine/storage/s3.py | 0 .../hindsight_api/engine/task_backend.py | 0 .../hindsight_api/engine/utils.py | 0 .../hindsight_api/extensions/__init__.py | 0 .../hindsight_api/extensions/base.py | 0 .../extensions/builtin/__init__.py | 0 .../extensions/builtin/supabase_tenant.py | 0 .../extensions/builtin/tenant.py | 0 .../hindsight_api/extensions/context.py | 0 .../hindsight_api/extensions/http.py | 0 .../hindsight_api/extensions/loader.py | 0 .../hindsight_api/extensions/mcp.py | 0 .../extensions/operation_validator.py | 0 .../hindsight_api/extensions/tenant.py | 0 .../hindsight_api/main.py | 0 .../hindsight_api/mcp_local.py | 0 .../hindsight_api/mcp_tools.py | 0 .../hindsight_api/metrics.py | 0 .../hindsight_api/migrations.py | 0 .../hindsight_api/models.py | 0 .../hindsight_api/pg0.py | 13 +- .../hindsight_api/server.py | 0 .../hindsight_api/tracing.py | 0 .../hindsight_api/utils.py | 0 .../hindsight_api/webhooks/__init__.py | 0 .../hindsight_api/webhooks/manager.py | 0 .../hindsight_api/webhooks/models.py | 0 .../hindsight_api/worker/__init__.py | 0 .../hindsight_api/worker/exceptions.py | 0 .../hindsight_api/worker/main.py | 0 .../hindsight_api/worker/poller.py | 0 hindsight-api-slim/pyproject.toml | 198 ++++++++++++++++++ .../tests/__init__.py | 0 .../tests/conftest.py | 0 .../tests/fixtures/README.md | 0 .../fixtures/locomo_conversation_sample.json | 0 .../tests/test_admin_backup_restore.py | 0 .../tests/test_agents_api.py | 0 .../tests/test_async_batch_retain.py | 0 .../tests/test_async_retain_tags.py | 0 .../tests/test_base_path.py | 0 .../tests/test_batch_api.py | 0 .../tests/test_batch_api_integration.py | 0 .../tests/test_batch_api_validation.py | 0 .../tests/test_batch_chunking.py | 0 .../tests/test_causal_relations.py | 0 .../tests/test_causal_relationships.py | 0 .../tests/test_chunking.py | 0 .../tests/test_combined_scoring.py | 0 .../tests/test_config_validation.py | 0 .../tests/test_consolidation.py | 0 .../tests/test_custom_embedding_dimension.py | 0 .../tests/test_document_tracking.py | 0 .../tests/test_entity_labels.py | 0 .../tests/test_extensions.py | 0 .../tests/test_fact_extraction_analysis.py | 0 .../tests/test_fact_extraction_metadata.py | 0 .../test_fact_extraction_output_ratio.py | 0 .../tests/test_fact_extraction_quality.py | 0 .../tests/test_fact_extraction_retry.py | 0 .../tests/test_fact_ordering.py | 0 .../tests/test_file_retain.py | 0 .../tests/test_file_storage_s3.py | 0 .../tests/test_gemini_safety_settings.py | 0 .../tests/test_graph_filtering.py | 0 .../tests/test_hierarchical_config.py | 0 .../tests/test_hnsw_indexes.py | 0 .../tests/test_http_api_integration.py | 0 .../tests/test_iris_parser.py | 0 .../tests/test_link_expansion_retrieval.py | 0 .../tests/test_link_utils.py | 0 .../tests/test_list_documents.py | 0 .../tests/test_litellm_sdk_cross_encoder.py | 0 .../tests/test_litellm_sdk_embeddings.py | 0 .../tests/test_llm_provider.py | 0 .../tests/test_llm_token_metrics.py | 0 .../tests/test_llm_tools.py | 0 .../tests/test_llm_wrapper.py | 0 .../tests/test_lmstudio_tool_choice.py | 0 .../tests/test_load_large_batch.py | 0 .../tests/test_main_module.py | 0 .../tests/test_mcp_endpoint_routing.py | 0 .../tests/test_mcp_extension.py | 0 .../tests/test_mcp_routing.py | 0 .../tests/test_mcp_tools.py | 0 .../tests/test_mental_model_hooks.py | 0 .../tests/test_mental_models.py | 0 .../tests/test_metrics.py | 0 .../tests/test_migrations_thread_safety.py | 0 .../tests/test_minimax_provider.py | 0 .../tests/test_mpfp_retrieval.py | 0 .../tests/test_multilingual.py | 0 .../tests/test_observation_invalidation.py | 0 .../tests/test_observations.py | 0 .../tests/test_op_cancellation.py | 0 .../tests/test_per_operation_llm_config.py | 0 .../tests/test_provider_default_models.py | 0 .../tests/test_query_analyzer.py | 0 .../tests/test_recall_chunks_independence.py | 0 .../tests/test_reflect_agent.py | 0 .../tests/test_reflect_empty_based_on.py | 0 .../tests/test_reflect_tracing.py | 0 .../tests/test_reflections.py | 0 .../tests/test_reranker_error_handling.py | 0 .../tests/test_retain.py | 0 .../tests/test_schema_isolation.py | 0 .../tests/test_search_trace.py | 0 .../tests/test_server_module.py | 0 .../tests/test_source_facts_tokens.py | 0 .../tests/test_sql_schema_safety.py | 0 .../tests/test_supabase_tenant.py | 0 .../tests/test_tags_visibility.py | 0 .../tests/test_tei_cross_encoder.py | 0 .../tests/test_temporal_ranges.py | 0 .../tests/test_think.py | 0 .../tests/test_tracing.py | 0 .../tests/test_tracing_integration.py | 0 .../tests/test_tracing_spans_verification.py | 0 .../tests/test_vertexai_provider.py | 0 .../tests/test_webhooks.py | 0 .../tests/test_worker.py | 0 hindsight-api/pyproject.toml | 185 +--------------- hindsight-docs/docs/developer/extensions.md | 2 +- hindsight-docs/docs/developer/installation.md | 128 ++++++----- .../version-0.4/developer/extensions.md | 2 +- .../hindsight_embed/daemon_embed_manager.py | 2 +- .../tests/test_base_path_deployment.py | 2 +- pyproject.toml | 2 +- scripts/hooks/lint.sh | 8 +- scripts/release.sh | 4 +- .../references/developer/extensions.md | 2 +- uv.lock | 99 +++++++-- 273 files changed, 762 insertions(+), 378 deletions(-) rename {hindsight => hindsight-all-slim}/README.md (100%) create mode 100644 hindsight-all-slim/pyproject.toml create mode 100644 hindsight-all/README.md rename {hindsight => hindsight-all}/hindsight/__init__.py (100%) rename {hindsight => hindsight-all}/hindsight/api_namespaces.py (100%) rename {hindsight => hindsight-all}/hindsight/client_wrapper.py (100%) rename {hindsight => hindsight-all}/hindsight/embedded.py (100%) rename {hindsight => hindsight-all}/hindsight/server.py (100%) rename {hindsight => hindsight-all}/pyproject.toml (89%) rename {hindsight => hindsight-all}/tests/README.md (100%) rename {hindsight => hindsight-all}/tests/__init__.py (100%) rename {hindsight => hindsight-all}/tests/test_embedded.py (100%) rename {hindsight => hindsight-all}/tests/test_embedded_namespaces.py (100%) rename {hindsight => hindsight-all}/tests/test_server_integration.py (100%) create mode 100644 hindsight-api-slim/README.md rename {hindsight-api => hindsight-api-slim}/hindsight_api/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/admin/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/admin/cli.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/README (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/env.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/script.py.mako (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/5a366d414dce_initial_schema.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/a1b2c3d4e5f6_add_file_storage_table.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/a2b3c4d5e6f7_add_text_signals_column.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/a2b3c4d5e6f8_add_gin_index_source_memory_ids.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/aa2b3c4d5e6f_nullable_event_date.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/b3c4d5e6f7g8_add_temporal_date_indexes.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/b4c5d6e7f8a9_backfill_observation_scopes.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/b7c4d8e9f1a2_add_chunks_table.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/c1a2b3d4e5f6_enable_pg_trgm_and_entities_trgm_index.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/c3d4e5f6g7h8_add_history_to_mental_models.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/c8e5f2a3b4d1_add_retain_params_to_documents.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/d2e3f4a5b6c7_add_memory_links_expansion_indexes.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/d4e5f6g7h8i9_gin_source_memory_ids_fastupdate_off.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/d5e6f7a8b9c0_add_bank_internal_id_and_per_bank_hnsw.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/d9f6a3b4c5e2_rename_bank_to_interactions.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/e0a1b2c3d4e5_disposition_to_3_traits.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/e4f5a6b7c8d9_add_webhooks_tables.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/e5f6g7h8i9j0_cascade_delete_ops_on_bank_delete.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/f1a2b3c4d5e6_add_memory_links_composite_index.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/f7g8h9i0j1k2_add_webhook_http_config.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/g2a3b4c5d6e7_add_tags_column.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/h3c4d5e6f7g8_mental_models_v4.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/i4d5e6f7g8h9_delete_opinions.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/j5e6f7g8h9i0_mental_model_versions.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/k6f7g8h9i0j1_add_directive_subtype.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/l7g8h9i0j1k2_add_worker_columns.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/m8h9i0j1k2l3_mental_model_id_to_text.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/n9i0j1k2l3m4_learnings_and_pinned_reflections.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/o0j1k2l3m4n5_migrate_mental_models_data.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/p1k2l3m4n5o6_new_knowledge_architecture.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/q2l3m4n5o6p7_fix_mental_model_fact_type.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/r3m4n5o6p7q8_add_reflect_response_to_reflections.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/rename_personality_to_disposition.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/s4n5o6p7q8r9_add_consolidated_at_to_memory_units.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/t5o6p7q8r9s0_rename_mental_models_to_observations.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/u6p7q8r9s0t1_mental_models_text_id.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/v7q8r9s0t1u2_add_max_tokens_to_mental_models.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/w8r9s0t1u2v3_fix_mental_models_pk_isolation.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/x9s0t1u2v3w4_add_bank_config_column.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/y0t1u2v3w4x5_add_result_metadata_gin_index.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/alembic/versions/z1u2v3w4x5y6_add_observation_tags_to_memory_units.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/api/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/api/http.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/api/mcp.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/banner.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/config.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/config_resolver.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/daemon.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/consolidation/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/consolidation/consolidator.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/consolidation/prompts.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/cross_encoder.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/db_budget.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/db_utils.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/directives/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/directives/models.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/embeddings.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/entity_resolver.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/interface.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/jina_mlx_reranker.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/llm_interface.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/llm_wrapper.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/memory_engine.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/operation_metadata.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/parsers/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/parsers/base.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/parsers/iris.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/parsers/markitdown.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/providers/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/providers/anthropic_llm.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/providers/claude_code_llm.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/providers/codex_llm.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/providers/gemini_llm.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/providers/mock_llm.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/providers/openai_compatible_llm.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/query_analyzer.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/reflect/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/reflect/agent.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/reflect/models.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/reflect/observations.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/reflect/prompts.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/reflect/tools.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/reflect/tools_schema.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/response_models.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/bank_utils.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/chunk_storage.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/embedding_processing.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/embedding_utils.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/entity_labels.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/entity_processing.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/fact_extraction.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/fact_storage.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/link_creation.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/link_utils.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/orchestrator.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/retain/types.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/fusion.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/graph_retrieval.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/link_expansion_retrieval.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/mpfp_retrieval.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/reranking.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/retrieval.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/tags.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/temporal_extraction.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/think_utils.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/trace.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/tracer.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/search/types.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/storage/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/storage/azure.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/storage/base.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/storage/gcs.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/storage/postgresql.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/storage/s3.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/task_backend.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/engine/utils.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/extensions/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/extensions/base.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/extensions/builtin/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/extensions/builtin/supabase_tenant.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/extensions/builtin/tenant.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/extensions/context.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/extensions/http.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/extensions/loader.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/extensions/mcp.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/extensions/operation_validator.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/extensions/tenant.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/main.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/mcp_local.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/mcp_tools.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/metrics.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/migrations.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/models.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/pg0.py (93%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/server.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/tracing.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/utils.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/webhooks/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/webhooks/manager.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/webhooks/models.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/worker/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/worker/exceptions.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/worker/main.py (100%) rename {hindsight-api => hindsight-api-slim}/hindsight_api/worker/poller.py (100%) create mode 100644 hindsight-api-slim/pyproject.toml rename {hindsight-api => hindsight-api-slim}/tests/__init__.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/conftest.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/fixtures/README.md (100%) rename {hindsight-api => hindsight-api-slim}/tests/fixtures/locomo_conversation_sample.json (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_admin_backup_restore.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_agents_api.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_async_batch_retain.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_async_retain_tags.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_base_path.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_batch_api.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_batch_api_integration.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_batch_api_validation.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_batch_chunking.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_causal_relations.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_causal_relationships.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_chunking.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_combined_scoring.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_config_validation.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_consolidation.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_custom_embedding_dimension.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_document_tracking.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_entity_labels.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_extensions.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_fact_extraction_analysis.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_fact_extraction_metadata.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_fact_extraction_output_ratio.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_fact_extraction_quality.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_fact_extraction_retry.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_fact_ordering.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_file_retain.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_file_storage_s3.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_gemini_safety_settings.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_graph_filtering.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_hierarchical_config.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_hnsw_indexes.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_http_api_integration.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_iris_parser.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_link_expansion_retrieval.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_link_utils.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_list_documents.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_litellm_sdk_cross_encoder.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_litellm_sdk_embeddings.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_llm_provider.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_llm_token_metrics.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_llm_tools.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_llm_wrapper.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_lmstudio_tool_choice.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_load_large_batch.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_main_module.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_mcp_endpoint_routing.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_mcp_extension.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_mcp_routing.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_mcp_tools.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_mental_model_hooks.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_mental_models.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_metrics.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_migrations_thread_safety.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_minimax_provider.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_mpfp_retrieval.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_multilingual.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_observation_invalidation.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_observations.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_op_cancellation.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_per_operation_llm_config.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_provider_default_models.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_query_analyzer.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_recall_chunks_independence.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_reflect_agent.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_reflect_empty_based_on.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_reflect_tracing.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_reflections.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_reranker_error_handling.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_retain.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_schema_isolation.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_search_trace.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_server_module.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_source_facts_tokens.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_sql_schema_safety.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_supabase_tenant.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_tags_visibility.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_tei_cross_encoder.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_temporal_ranges.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_think.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_tracing.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_tracing_integration.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_tracing_spans_verification.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_vertexai_provider.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_webhooks.py (100%) rename {hindsight-api => hindsight-api-slim}/tests/test_worker.py (100%) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f86d1c5e..538d0813 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -30,12 +30,20 @@ jobs: working-directory: ./hindsight-clients/python run: uv build --out-dir dist + - name: Build hindsight-api-slim + working-directory: ./hindsight-api-slim + run: uv build --out-dir dist + - name: Build hindsight-api working-directory: ./hindsight-api run: uv build --out-dir dist - name: Build hindsight-all - working-directory: ./hindsight + working-directory: ./hindsight-all + run: uv build --out-dir dist + + - name: Build hindsight-all-slim + working-directory: ./hindsight-all-slim run: uv build --out-dir dist - name: Build hindsight-litellm @@ -54,13 +62,19 @@ jobs: working-directory: ./hindsight-integrations/pydantic-ai run: uv build --out-dir dist - # Publish in order (client and api first, then hindsight-all which depends on them) + # Publish in order (client and api-slim first, then api/all wrappers which depend on them) - name: Publish hindsight-client to PyPI uses: pypa/gh-action-pypi-publish@release/v1 with: packages-dir: ./hindsight-clients/python/dist skip-existing: true + - name: Publish hindsight-api-slim to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 + with: + packages-dir: ./hindsight-api-slim/dist + skip-existing: true + - name: Publish hindsight-api to PyPI uses: pypa/gh-action-pypi-publish@release/v1 with: @@ -70,7 +84,13 @@ jobs: - name: Publish hindsight-all to PyPI uses: pypa/gh-action-pypi-publish@release/v1 with: - packages-dir: ./hindsight/dist + packages-dir: ./hindsight-all/dist + skip-existing: true + + - name: Publish hindsight-all-slim to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 + with: + packages-dir: ./hindsight-all-slim/dist skip-existing: true - name: Publish hindsight-litellm to PyPI @@ -104,8 +124,10 @@ jobs: name: python-packages path: | hindsight-clients/python/dist/* + hindsight-api-slim/dist/* hindsight-api/dist/* - hindsight/dist/* + hindsight-all/dist/* + hindsight-all-slim/dist/* hindsight-integrations/litellm/dist/* hindsight-embed/dist/* hindsight-integrations/crewai/dist/* @@ -641,8 +663,10 @@ jobs: mkdir -p release-assets # Python packages cp artifacts/python-packages/hindsight-clients/python/dist/* release-assets/ || true + cp artifacts/python-packages/hindsight-api-slim/dist/* release-assets/ || true cp artifacts/python-packages/hindsight-api/dist/* release-assets/ || true - cp artifacts/python-packages/hindsight/dist/* release-assets/ || true + cp artifacts/python-packages/hindsight-all/dist/* release-assets/ || true + cp artifacts/python-packages/hindsight-all-slim/dist/* release-assets/ || true cp artifacts/python-packages/hindsight-integrations/litellm/dist/* release-assets/ || true cp artifacts/python-packages/hindsight-integrations/pydantic-ai/dist/* release-assets/ || true cp artifacts/python-packages/hindsight-embed/dist/* release-assets/ || true diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 058675f2..bcfb0d07 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -29,7 +29,7 @@ jobs: python-version: ${{ matrix.python-version }} - name: Build hindsight-api - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: uv build build-typescript-client: @@ -249,23 +249,23 @@ jobs: python-version-file: ".python-version" - name: Build API - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: uv build - name: Install API dependencies - working-directory: ./hindsight-api - run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match + working-directory: ./hindsight-api-slim + run: uv sync --frozen --all-extras --index-strategy unsafe-best-match - name: Cache HuggingFace models uses: actions/cache@v5 with: path: ~/.cache/huggingface - key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }} + key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }} restore-keys: | ${{ runner.os }}-huggingface- - name: Pre-download models - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: | uv run python -c " from sentence_transformers import SentenceTransformer, CrossEncoder @@ -444,23 +444,23 @@ jobs: python-version-file: ".python-version" - name: Build API - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: uv build - name: Install dependencies - working-directory: ./hindsight-api - run: uv sync --frozen --extra test --no-install-project --index-strategy unsafe-best-match + working-directory: ./hindsight-api-slim + run: uv sync --frozen --all-extras --index-strategy unsafe-best-match - name: Cache HuggingFace models uses: actions/cache@v5 with: path: ~/.cache/huggingface - key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }} + key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }} restore-keys: | ${{ runner.os }}-huggingface- - name: Pre-download models - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: | uv run python -c " from sentence_transformers import SentenceTransformer, CrossEncoder @@ -472,7 +472,7 @@ jobs: " - name: Run tests - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: uv run pytest tests -v test-python-client: @@ -507,7 +507,7 @@ jobs: python-version-file: ".python-version" - name: Build API - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: uv build - name: Build Python client @@ -519,19 +519,19 @@ jobs: run: uv sync --frozen --extra test --index-strategy unsafe-best-match - name: Install API dependencies - working-directory: ./hindsight-api - run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match + working-directory: ./hindsight-api-slim + run: uv sync --frozen --all-extras --index-strategy unsafe-best-match - name: Cache HuggingFace models uses: actions/cache@v5 with: path: ~/.cache/huggingface - key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }} + key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }} restore-keys: | ${{ runner.os }}-huggingface- - name: Pre-download models - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: | uv run python -c " from sentence_transformers import SentenceTransformer, CrossEncoder @@ -615,12 +615,12 @@ jobs: node-version: '20' - name: Build API - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: uv build - name: Install API dependencies - working-directory: ./hindsight-api - run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match + working-directory: ./hindsight-api-slim + run: uv sync --frozen --all-extras --index-strategy unsafe-best-match - name: Install TypeScript client dependencies working-directory: ./hindsight-clients/typescript @@ -634,12 +634,12 @@ jobs: uses: actions/cache@v5 with: path: ~/.cache/huggingface - key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }} + key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }} restore-keys: | ${{ runner.os }}-huggingface- - name: Pre-download models - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: | uv run python -c " from sentence_transformers import SentenceTransformer, CrossEncoder @@ -754,23 +754,23 @@ jobs: key: ${{ runner.os }}-cargo-client-${{ hashFiles('hindsight-clients/rust/Cargo.lock') }} - name: Build API - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: uv build - name: Install API dependencies - working-directory: ./hindsight-api - run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match + working-directory: ./hindsight-api-slim + run: uv sync --frozen --all-extras --index-strategy unsafe-best-match - name: Cache HuggingFace models uses: actions/cache@v5 with: path: ~/.cache/huggingface - key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }} + key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }} restore-keys: | ${{ runner.os }}-huggingface- - name: Pre-download models - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: | uv run python -c " from sentence_transformers import SentenceTransformer, CrossEncoder @@ -855,23 +855,23 @@ jobs: cache-dependency-path: hindsight-clients/go/go.sum - name: Build API - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: uv build - name: Install API dependencies - working-directory: ./hindsight-api - run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match + working-directory: ./hindsight-api-slim + run: uv sync --frozen --all-extras --index-strategy unsafe-best-match - name: Cache HuggingFace models uses: actions/cache@v5 with: path: ~/.cache/huggingface - key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }} + key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }} restore-keys: | ${{ runner.os }}-huggingface- - name: Pre-download models - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: | uv run python -c " from sentence_transformers import SentenceTransformer, CrossEncoder @@ -959,27 +959,27 @@ jobs: node-version: '22' - name: Build API - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: uv build - - name: Install API dependencies - working-directory: ./hindsight-api - run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match - - name: Install embed dependencies working-directory: ./hindsight-embed run: uv sync --frozen --index-strategy unsafe-best-match + - name: Install API dependencies + working-directory: ./hindsight-api-slim + run: uv sync --frozen --all-extras --index-strategy unsafe-best-match + - name: Cache HuggingFace models uses: actions/cache@v5 with: path: ~/.cache/huggingface - key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }} + key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }} restore-keys: | ${{ runner.os }}-huggingface- - name: Pre-download models - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: | uv run python -c " from sentence_transformers import SentenceTransformer, CrossEncoder @@ -1061,27 +1061,27 @@ jobs: python-version-file: ".python-version" - name: Build API - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: uv build - - name: Install API dependencies - working-directory: ./hindsight-api - run: uv sync --frozen --no-install-project --index-strategy unsafe-best-match - - name: Install integration test dependencies working-directory: ./hindsight-integration-tests run: uv sync --frozen + - name: Install API dependencies + working-directory: ./hindsight-api-slim + run: uv sync --frozen --all-extras --index-strategy unsafe-best-match + - name: Cache HuggingFace models uses: actions/cache@v5 with: path: ~/.cache/huggingface - key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }} + key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }} restore-keys: | ${{ runner.os }}-huggingface- - name: Pre-download models - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: | uv run python -c " from sentence_transformers import SentenceTransformer, CrossEncoder @@ -1215,6 +1215,64 @@ jobs: working-directory: ./hindsight-integrations/pydantic-ai run: uv run pytest tests -v + test-pip-slim: + runs-on: ubuntu-latest + env: + HINDSIGHT_API_LLM_PROVIDER: vertexai + HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY: /tmp/gcp-credentials.json + HINDSIGHT_API_LLM_MODEL: google/gemini-2.5-flash-lite + HINDSIGHT_API_EMBEDDINGS_PROVIDER: cohere + HINDSIGHT_API_RERANKER_PROVIDER: cohere + HINDSIGHT_API_COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }} + + steps: + - uses: actions/checkout@v4 + + - name: Setup GCP credentials + run: | + printf '%s' '${{ secrets.GCP_VERTEXAI_CREDENTIALS }}' > /tmp/gcp-credentials.json + PROJECT_ID=$(jq -r '.project_id' /tmp/gcp-credentials.json) + echo "HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=$PROJECT_ID" >> $GITHUB_ENV + + - name: Install uv + uses: astral-sh/setup-uv@v5 + with: + enable-cache: true + prune-cache: false + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version-file: ".python-version" + + - name: Install hindsight-api-slim (embedded-db only, no local ML) + working-directory: ./hindsight-api-slim + run: uv sync --frozen --extra embedded-db --index-strategy unsafe-best-match + + - name: Start API server and smoke test + working-directory: ./hindsight-api-slim + run: | + uv run hindsight-api --port 8888 > /tmp/slim-api-server.log 2>&1 & + for i in $(seq 1 60); do + if curl -s http://localhost:8888/health | grep -q "healthy"; then + echo "API server is ready after ${i}s" + break + fi + if [ $i -eq 60 ]; then + echo "API server failed to start after 60s" + cat /tmp/slim-api-server.log + exit 1 + fi + sleep 1 + done + echo "PASS: slim pip package smoke test (server started and healthy)" + + - name: Show API server logs + if: always() + run: | + echo "=== API Server Logs ===" + cat /tmp/slim-api-server.log 2>/dev/null || true + test-embed: runs-on: ubuntu-latest env: @@ -1248,6 +1306,10 @@ jobs: working-directory: ./hindsight-embed run: uv sync --frozen --index-strategy unsafe-best-match + - name: Install API dependencies (with local-ml and embedded-db for smoke test) + working-directory: ./hindsight-api-slim + run: uv sync --frozen --all-extras --index-strategy unsafe-best-match + - name: Cache HuggingFace models uses: actions/cache@v5 with: @@ -1299,24 +1361,24 @@ jobs: python-version-file: ".python-version" - name: Build hindsight-all - working-directory: ./hindsight + working-directory: ./hindsight-all run: uv build - name: Install dependencies - working-directory: ./hindsight + working-directory: ./hindsight-all run: uv sync --frozen --extra test --index-strategy unsafe-best-match - name: Cache HuggingFace models uses: actions/cache@v5 with: path: ~/.cache/huggingface - key: ${{ runner.os }}-huggingface-all-${{ hashFiles('hindsight/pyproject.toml') }} + key: ${{ runner.os }}-huggingface-all-${{ hashFiles('hindsight-all/pyproject.toml') }} restore-keys: | ${{ runner.os }}-huggingface-all- ${{ runner.os }}-huggingface- - name: Run unit tests - working-directory: ./hindsight + working-directory: ./hindsight-all run: uv run pytest tests/ -v test-doc-examples: @@ -1383,17 +1445,17 @@ jobs: cache: 'npm' cache-dependency-path: package-lock.json - - name: Build and install API - working-directory: ./hindsight-api - run: | - uv build - uv sync --frozen --no-install-project --index-strategy unsafe-best-match - - name: Install Python client dependencies if: matrix.language == 'python' working-directory: ./hindsight-clients/python run: uv sync --frozen --extra test --index-strategy unsafe-best-match + - name: Build and install API + working-directory: ./hindsight-api-slim + run: | + uv build + uv sync --frozen --all-extras --index-strategy unsafe-best-match + - name: Install TypeScript client if: matrix.language == 'node' run: | @@ -1404,12 +1466,12 @@ jobs: uses: actions/cache@v5 with: path: ~/.cache/huggingface - key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }} + key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }} restore-keys: | ${{ runner.os }}-huggingface- - name: Pre-download models - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: | uv run python -c " from sentence_transformers import SentenceTransformer, CrossEncoder @@ -1497,7 +1559,7 @@ jobs: uses: actions/cache@v5 with: path: ~/.cache/huggingface - key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api/pyproject.toml') }} + key: ${{ runner.os }}-huggingface-${{ hashFiles('hindsight-api-slim/pyproject.toml') }} restore-keys: | ${{ runner.os }}-huggingface- @@ -1506,11 +1568,11 @@ jobs: run: uv sync --frozen --extra test --index-strategy unsafe-best-match - name: Install current hindsight-api - working-directory: ./hindsight-api - run: uv sync --frozen --index-strategy unsafe-best-match + working-directory: ./hindsight-api-slim + run: uv sync --frozen --all-extras --index-strategy unsafe-best-match - name: Pre-download models - working-directory: ./hindsight-api + working-directory: ./hindsight-api-slim run: | uv run python -c " from sentence_transformers import SentenceTransformer, CrossEncoder diff --git a/CLAUDE.md b/CLAUDE.md index 4a6fb7ff..a8560c8a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -17,20 +17,20 @@ Hindsight is an agent memory system that provides long-term memory for AI agents ./scripts/dev/start-api.sh # Run all tests (parallelized with pytest-xdist) -cd hindsight-api && uv run pytest tests/ +cd hindsight-api-slim && uv run pytest tests/ # Run specific test file -cd hindsight-api && uv run pytest tests/test_http_api_integration.py -v +cd hindsight-api-slim && uv run pytest tests/test_http_api_integration.py -v # Run single test function -cd hindsight-api && uv run pytest tests/test_retain.py::test_retain_simple -v +cd hindsight-api-slim && uv run pytest tests/test_retain.py::test_retain_simple -v # Lint and format -cd hindsight-api && uv run ruff check . -cd hindsight-api && uv run ruff format . +cd hindsight-api-slim && uv run ruff check . +cd hindsight-api-slim && uv run ruff format . # Type checking (uses ty - extremely fast type checker from Astral) -cd hindsight-api && uv run ty check hindsight_api/ +cd hindsight-api-slim && uv run ty check hindsight_api/ ``` ### Control Plane (Next.js) @@ -72,7 +72,7 @@ cd hindsight-control-plane && npm run dev ## Architecture ### Monorepo Structure -- **hindsight-api/**: Core FastAPI server with memory engine (Python, uv) +- **hindsight-api-slim/**: Core FastAPI server with memory engine (Python, uv) - **hindsight/**: Embedded Python bundle (hindsight-all package) - **hindsight-control-plane/**: Admin UI (Next.js, npm) - **hindsight-cli/**: CLI tool (Rust, cargo, uses progenitor for API client) @@ -81,7 +81,7 @@ cd hindsight-control-plane && npm run dev - **hindsight-integrations/**: Framework integrations (LiteLLM, OpenAI) - **hindsight-dev/**: Development tools and benchmarks -### Core Engine (hindsight-api/hindsight_api/engine/) +### Core Engine (hindsight-api-slim/hindsight_api/engine/) - `memory_engine.py`: Main orchestrator (~170KB) for retain/recall/reflect operations - `llm_wrapper.py`: LLM abstraction supporting OpenAI, Anthropic, Gemini, Groq, MiniMax, Ollama, LM Studio - `embeddings.py`: Embedding generation (local sentence-transformers or TEI) @@ -101,7 +101,7 @@ cd hindsight-control-plane && npm run dev - `fusion.py`: Reciprocal rank fusion for combining results - `reranking.py`: Cross-encoder reranking -### API Layer (hindsight-api/hindsight_api/api/) +### API Layer (hindsight-api-slim/hindsight_api/api/) - `http.py`: FastAPI HTTP routers (~80KB) for all REST endpoints - `mcp.py`: Model Context Protocol server implementation @@ -111,13 +111,13 @@ Main operations: - **Reflect**: Disposition-aware reasoning using memories and mental models. ### Database -PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api/hindsight_api/alembic/`. Migrations run automatically on API startup. +PostgreSQL with pgvector. Schema managed via Alembic migrations in `hindsight-api-slim/hindsight_api/alembic/`. Migrations run automatically on API startup. Key tables: `banks`, `memory_units`, `documents`, `entities`, `entity_links` ### Adding Database Migrations -1. **Create a new migration file** in `hindsight-api/hindsight_api/alembic/versions/`: +1. **Create a new migration file** in `hindsight-api-slim/hindsight_api/alembic/versions/`: - File name format: `_.py` (e.g., `f1a2b3c4d5e6_add_new_index.py`) - Use a unique hex revision ID (12 chars) - Set `down_revision` to the previous migration's revision ID @@ -251,7 +251,7 @@ Fields must be categorized as either **hierarchical** (can be overridden per-ten #### Adding a New Configuration Field -1. **config.py** (`hindsight-api/hindsight_api/config.py`): +1. **config.py** (`hindsight-api-slim/hindsight_api/config.py`): - Add `ENV_*` constant for the environment variable name (e.g., `ENV_MY_SETTING = "HINDSIGHT_API_MY_SETTING"`) - Add `DEFAULT_*` constant for the default value - Add field to `HindsightConfig` dataclass with type annotation @@ -268,7 +268,7 @@ Fields must be categorized as either **hierarchical** (can be overridden per-ten # Static field - just don't add to _HIERARCHICAL_FIELDS ``` -2. **main.py** (`hindsight-api/hindsight_api/main.py`): +2. **main.py** (`hindsight-api-slim/hindsight_api/main.py`): - Add field to the manual `HindsightConfig()` constructor call (search for "CLI override") 3. **Use hierarchical config in MemoryEngine**: @@ -308,7 +308,7 @@ cp .env.example .env # Edit .env with LLM API key # Python deps -uv sync --directory hindsight-api/ +uv sync --directory hindsight-api-slim/ # Node deps (uses npm workspaces) npm install diff --git a/docker/standalone/Dockerfile b/docker/standalone/Dockerfile index f0fb3888..71a27a4a 100644 --- a/docker/standalone/Dockerfile +++ b/docker/standalone/Dockerfile @@ -42,25 +42,22 @@ RUN apt-get update && apt-get install -y \ && pip install --no-cache-dir uv # Copy dependency files and README (required by pyproject.toml) -COPY hindsight-api/pyproject.toml ./api/ -COPY hindsight-api/README.md ./api/ +COPY hindsight-api-slim/pyproject.toml ./api/ +COPY hindsight-api-slim/README.md ./api/ WORKDIR /app/api -# Remove local ML model dependencies if INCLUDE_LOCAL_MODELS=false -# This creates a smaller image when using external providers (TEI, OpenAI, Cohere) -RUN if [ "$INCLUDE_LOCAL_MODELS" != "true" ]; then \ - echo "Removing local-models dependencies (sentence-transformers, torch, transformers)..." && \ - sed -i '/"sentence-transformers/d' pyproject.toml && \ - sed -i '/"transformers/d' pyproject.toml && \ - sed -i '/"torch/d' pyproject.toml; \ +# Sync dependencies using appropriate extras based on INCLUDE_LOCAL_MODELS +# local-ml: torch, sentence-transformers, transformers, einops, flashrank, mlx (optional) +# embedded-db: pg0-embedded (always included for embedded PostgreSQL support) +RUN if [ "$INCLUDE_LOCAL_MODELS" = "true" ]; then \ + uv sync --extra local-ml --extra embedded-db; \ + else \ + uv sync --extra embedded-db; \ fi -# Sync dependencies (will create lock file if needed) -RUN uv sync - # Copy source code (alembic migrations are inside hindsight_api/) -COPY hindsight-api/hindsight_api ./hindsight_api +COPY hindsight-api-slim/hindsight_api ./hindsight_api # Install the local package (uv sync only installed dependencies, not the package itself) RUN uv pip install -e . diff --git a/hindsight/README.md b/hindsight-all-slim/README.md similarity index 100% rename from hindsight/README.md rename to hindsight-all-slim/README.md diff --git a/hindsight-all-slim/pyproject.toml b/hindsight-all-slim/pyproject.toml new file mode 100644 index 00000000..432c62e5 --- /dev/null +++ b/hindsight-all-slim/pyproject.toml @@ -0,0 +1,33 @@ +[build-system] +requires = ["setuptools>=61"] +build-backend = "setuptools.build_meta" + +[project] +name = "hindsight-all-slim" +version = "0.4.17" +description = "Hindsight: Agent Memory That Works Like Human Memory - Slim All-in-One Bundle" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + "hindsight-api-slim>=0.4.17", + "hindsight-client>=0.0.7", + "hindsight-embed>=0.1.0", +] + +[tool.uv.sources] +hindsight-api-slim = { workspace = true } +hindsight-client = { workspace = true } +hindsight-embed = { workspace = true } + +[project.optional-dependencies] +test = [ + "pytest>=7.0.0", + "pytest-asyncio>=0.21.0", +] + +[tool.setuptools] +packages = [] + +[tool.pytest.ini_options] +asyncio_mode = "auto" +asyncio_default_fixture_loop_scope = "function" diff --git a/hindsight-all/README.md b/hindsight-all/README.md new file mode 100644 index 00000000..6668dace --- /dev/null +++ b/hindsight-all/README.md @@ -0,0 +1,48 @@ +# hindsight-all + +All-in-one package for Hindsight - Agent Memory That Works Like Human Memory + +## Quick Start + +```python +from hindsight import start_server, HindsightClient + +# Start server with embedded PostgreSQL +server = start_server( + llm_provider="groq", + llm_api_key="your-api-key", + llm_model="openai/gpt-oss-120b" +) + +# Create client +client = HindsightClient(base_url=server.url) + +# Store memories +client.put(agent_id="assistant", content="User prefers Python for data analysis") + +# Search memories +results = client.search(agent_id="assistant", query="programming preferences") + +# Generate contextual response +response = client.think(agent_id="assistant", query="What languages should I recommend?") + +# Stop server when done +server.stop() +``` + +## Using Context Manager + +```python +from hindsight import HindsightServer, HindsightClient + +with HindsightServer(llm_provider="groq", llm_api_key="...") as server: + client = HindsightClient(base_url=server.url) + # ... use client ... +# Server automatically stops +``` + +## Installation + +```bash +pip install hindsight-all +``` diff --git a/hindsight/hindsight/__init__.py b/hindsight-all/hindsight/__init__.py similarity index 100% rename from hindsight/hindsight/__init__.py rename to hindsight-all/hindsight/__init__.py diff --git a/hindsight/hindsight/api_namespaces.py b/hindsight-all/hindsight/api_namespaces.py similarity index 100% rename from hindsight/hindsight/api_namespaces.py rename to hindsight-all/hindsight/api_namespaces.py diff --git a/hindsight/hindsight/client_wrapper.py b/hindsight-all/hindsight/client_wrapper.py similarity index 100% rename from hindsight/hindsight/client_wrapper.py rename to hindsight-all/hindsight/client_wrapper.py diff --git a/hindsight/hindsight/embedded.py b/hindsight-all/hindsight/embedded.py similarity index 100% rename from hindsight/hindsight/embedded.py rename to hindsight-all/hindsight/embedded.py diff --git a/hindsight/hindsight/server.py b/hindsight-all/hindsight/server.py similarity index 100% rename from hindsight/hindsight/server.py rename to hindsight-all/hindsight/server.py diff --git a/hindsight/pyproject.toml b/hindsight-all/pyproject.toml similarity index 89% rename from hindsight/pyproject.toml rename to hindsight-all/pyproject.toml index 41ff1287..b6fbc173 100644 --- a/hindsight/pyproject.toml +++ b/hindsight-all/pyproject.toml @@ -9,13 +9,13 @@ description = "Hindsight: Agent Memory That Works Like Human Memory - All-in-One readme = "README.md" requires-python = ">=3.11" dependencies = [ - "hindsight-api>=0.0.7", + "hindsight-api-slim[all]>=0.4.17", "hindsight-client>=0.0.7", "hindsight-embed>=0.1.0", ] [tool.uv.sources] -hindsight-api = { workspace = true } +hindsight-api-slim = { workspace = true } hindsight-client = { workspace = true } hindsight-embed = { workspace = true } diff --git a/hindsight/tests/README.md b/hindsight-all/tests/README.md similarity index 100% rename from hindsight/tests/README.md rename to hindsight-all/tests/README.md diff --git a/hindsight/tests/__init__.py b/hindsight-all/tests/__init__.py similarity index 100% rename from hindsight/tests/__init__.py rename to hindsight-all/tests/__init__.py diff --git a/hindsight/tests/test_embedded.py b/hindsight-all/tests/test_embedded.py similarity index 100% rename from hindsight/tests/test_embedded.py rename to hindsight-all/tests/test_embedded.py diff --git a/hindsight/tests/test_embedded_namespaces.py b/hindsight-all/tests/test_embedded_namespaces.py similarity index 100% rename from hindsight/tests/test_embedded_namespaces.py rename to hindsight-all/tests/test_embedded_namespaces.py diff --git a/hindsight/tests/test_server_integration.py b/hindsight-all/tests/test_server_integration.py similarity index 100% rename from hindsight/tests/test_server_integration.py rename to hindsight-all/tests/test_server_integration.py diff --git a/hindsight-api-slim/README.md b/hindsight-api-slim/README.md new file mode 100644 index 00000000..09ab4898 --- /dev/null +++ b/hindsight-api-slim/README.md @@ -0,0 +1,137 @@ +# Hindsight API + +**Memory System for AI Agents** — Temporal + Semantic + Entity Memory Architecture using PostgreSQL with pgvector. + +Hindsight gives AI agents persistent memory that works like human memory: it stores facts, tracks entities and relationships, handles temporal reasoning ("what happened last spring?"), and forms opinions based on configurable disposition traits. + +## Installation + +```bash +pip install hindsight-api +``` + +## Quick Start + +### Run the Server + +```bash +# Set your LLM provider +export HINDSIGHT_API_LLM_PROVIDER=openai +export HINDSIGHT_API_LLM_API_KEY=sk-xxxxxxxxxxxx + +# Start the server (uses embedded PostgreSQL by default) +hindsight-api +``` + +The server starts at http://localhost:8888 with: +- REST API for memory operations +- MCP server at `/mcp` for tool-use integration + +### Use the Python API + +```python +from hindsight_api import MemoryEngine + +# Create and initialize the memory engine +memory = MemoryEngine() +await memory.initialize() + +# Create a memory bank for your agent +bank = await memory.create_memory_bank( + name="my-assistant", + background="A helpful coding assistant" +) + +# Store a memory +await memory.retain( + memory_bank_id=bank.id, + content="The user prefers Python for data science projects" +) + +# Recall memories +results = await memory.recall( + memory_bank_id=bank.id, + query="What programming language does the user prefer?" +) + +# Reflect with reasoning +response = await memory.reflect( + memory_bank_id=bank.id, + query="Should I recommend Python or R for this ML project?" +) +``` + +## CLI Options + +```bash +hindsight-api --help + +# Common options +hindsight-api --port 9000 # Custom port (default: 8888) +hindsight-api --host 127.0.0.1 # Bind to localhost only +hindsight-api --workers 4 # Multiple worker processes +hindsight-api --log-level debug # Verbose logging +``` + +## Configuration + +Configure via environment variables: + +| Variable | Description | Default | +|----------|-------------|---------| +| `HINDSIGHT_API_DATABASE_URL` | PostgreSQL connection string | `pg0` (embedded) | +| `HINDSIGHT_API_LLM_PROVIDER` | `openai`, `anthropic`, `gemini`, `groq`, `ollama`, `lmstudio` | `openai` | +| `HINDSIGHT_API_LLM_API_KEY` | API key for LLM provider | - | +| `HINDSIGHT_API_LLM_MODEL` | Model name | `gpt-4o-mini` | +| `HINDSIGHT_API_HOST` | Server bind address | `0.0.0.0` | +| `HINDSIGHT_API_PORT` | Server port | `8888` | + +### Example with External PostgreSQL + +```bash +export HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@localhost:5432/hindsight +export HINDSIGHT_API_LLM_PROVIDER=groq +export HINDSIGHT_API_LLM_API_KEY=gsk_xxxxxxxxxxxx + +hindsight-api +``` + +## Docker + +```bash +docker run --rm -it -p 8888:8888 \ + -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \ + -v $HOME/.hindsight-docker:/home/hindsight/.pg0 \ + ghcr.io/vectorize-io/hindsight:latest +``` + +## MCP Server + +For local MCP integration without running the full API server: + +```bash +hindsight-local-mcp +``` + +This runs a stdio-based MCP server that can be used directly with MCP-compatible clients. + +## Key Features + +- **Multi-Strategy Retrieval (TEMPR)** — Semantic, keyword, graph, and temporal search combined with RRF fusion +- **Entity Graph** — Automatic entity extraction and relationship tracking +- **Temporal Reasoning** — Native support for time-based queries +- **Disposition Traits** — Configurable skepticism, literalism, and empathy influence opinion formation +- **Three Memory Types** — World facts, bank actions, and formed opinions with confidence scores + +## Documentation + +Full documentation: [https://hindsight.vectorize.io](https://hindsight.vectorize.io) + +- [Installation Guide](https://hindsight.vectorize.io/developer/installation) +- [Configuration Reference](https://hindsight.vectorize.io/developer/configuration) +- [API Reference](https://hindsight.vectorize.io/api-reference) +- [Python SDK](https://hindsight.vectorize.io/sdks/python) + +## License + +Apache 2.0 diff --git a/hindsight-api/hindsight_api/__init__.py b/hindsight-api-slim/hindsight_api/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/__init__.py rename to hindsight-api-slim/hindsight_api/__init__.py diff --git a/hindsight-api/hindsight_api/admin/__init__.py b/hindsight-api-slim/hindsight_api/admin/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/admin/__init__.py rename to hindsight-api-slim/hindsight_api/admin/__init__.py diff --git a/hindsight-api/hindsight_api/admin/cli.py b/hindsight-api-slim/hindsight_api/admin/cli.py similarity index 100% rename from hindsight-api/hindsight_api/admin/cli.py rename to hindsight-api-slim/hindsight_api/admin/cli.py diff --git a/hindsight-api/hindsight_api/alembic/README b/hindsight-api-slim/hindsight_api/alembic/README similarity index 100% rename from hindsight-api/hindsight_api/alembic/README rename to hindsight-api-slim/hindsight_api/alembic/README diff --git a/hindsight-api/hindsight_api/alembic/env.py b/hindsight-api-slim/hindsight_api/alembic/env.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/env.py rename to hindsight-api-slim/hindsight_api/alembic/env.py diff --git a/hindsight-api/hindsight_api/alembic/script.py.mako b/hindsight-api-slim/hindsight_api/alembic/script.py.mako similarity index 100% rename from hindsight-api/hindsight_api/alembic/script.py.mako rename to hindsight-api-slim/hindsight_api/alembic/script.py.mako diff --git a/hindsight-api/hindsight_api/alembic/versions/5a366d414dce_initial_schema.py b/hindsight-api-slim/hindsight_api/alembic/versions/5a366d414dce_initial_schema.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/5a366d414dce_initial_schema.py rename to hindsight-api-slim/hindsight_api/alembic/versions/5a366d414dce_initial_schema.py diff --git a/hindsight-api/hindsight_api/alembic/versions/a1b2c3d4e5f6_add_file_storage_table.py b/hindsight-api-slim/hindsight_api/alembic/versions/a1b2c3d4e5f6_add_file_storage_table.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/a1b2c3d4e5f6_add_file_storage_table.py rename to hindsight-api-slim/hindsight_api/alembic/versions/a1b2c3d4e5f6_add_file_storage_table.py diff --git a/hindsight-api/hindsight_api/alembic/versions/a2b3c4d5e6f7_add_text_signals_column.py b/hindsight-api-slim/hindsight_api/alembic/versions/a2b3c4d5e6f7_add_text_signals_column.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/a2b3c4d5e6f7_add_text_signals_column.py rename to hindsight-api-slim/hindsight_api/alembic/versions/a2b3c4d5e6f7_add_text_signals_column.py diff --git a/hindsight-api/hindsight_api/alembic/versions/a2b3c4d5e6f8_add_gin_index_source_memory_ids.py b/hindsight-api-slim/hindsight_api/alembic/versions/a2b3c4d5e6f8_add_gin_index_source_memory_ids.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/a2b3c4d5e6f8_add_gin_index_source_memory_ids.py rename to hindsight-api-slim/hindsight_api/alembic/versions/a2b3c4d5e6f8_add_gin_index_source_memory_ids.py diff --git a/hindsight-api/hindsight_api/alembic/versions/aa2b3c4d5e6f_nullable_event_date.py b/hindsight-api-slim/hindsight_api/alembic/versions/aa2b3c4d5e6f_nullable_event_date.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/aa2b3c4d5e6f_nullable_event_date.py rename to hindsight-api-slim/hindsight_api/alembic/versions/aa2b3c4d5e6f_nullable_event_date.py diff --git a/hindsight-api/hindsight_api/alembic/versions/b3c4d5e6f7g8_add_temporal_date_indexes.py b/hindsight-api-slim/hindsight_api/alembic/versions/b3c4d5e6f7g8_add_temporal_date_indexes.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/b3c4d5e6f7g8_add_temporal_date_indexes.py rename to hindsight-api-slim/hindsight_api/alembic/versions/b3c4d5e6f7g8_add_temporal_date_indexes.py diff --git a/hindsight-api/hindsight_api/alembic/versions/b4c5d6e7f8a9_backfill_observation_scopes.py b/hindsight-api-slim/hindsight_api/alembic/versions/b4c5d6e7f8a9_backfill_observation_scopes.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/b4c5d6e7f8a9_backfill_observation_scopes.py rename to hindsight-api-slim/hindsight_api/alembic/versions/b4c5d6e7f8a9_backfill_observation_scopes.py diff --git a/hindsight-api/hindsight_api/alembic/versions/b7c4d8e9f1a2_add_chunks_table.py b/hindsight-api-slim/hindsight_api/alembic/versions/b7c4d8e9f1a2_add_chunks_table.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/b7c4d8e9f1a2_add_chunks_table.py rename to hindsight-api-slim/hindsight_api/alembic/versions/b7c4d8e9f1a2_add_chunks_table.py diff --git a/hindsight-api/hindsight_api/alembic/versions/c1a2b3d4e5f6_enable_pg_trgm_and_entities_trgm_index.py b/hindsight-api-slim/hindsight_api/alembic/versions/c1a2b3d4e5f6_enable_pg_trgm_and_entities_trgm_index.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/c1a2b3d4e5f6_enable_pg_trgm_and_entities_trgm_index.py rename to hindsight-api-slim/hindsight_api/alembic/versions/c1a2b3d4e5f6_enable_pg_trgm_and_entities_trgm_index.py diff --git a/hindsight-api/hindsight_api/alembic/versions/c3d4e5f6g7h8_add_history_to_mental_models.py b/hindsight-api-slim/hindsight_api/alembic/versions/c3d4e5f6g7h8_add_history_to_mental_models.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/c3d4e5f6g7h8_add_history_to_mental_models.py rename to hindsight-api-slim/hindsight_api/alembic/versions/c3d4e5f6g7h8_add_history_to_mental_models.py diff --git a/hindsight-api/hindsight_api/alembic/versions/c8e5f2a3b4d1_add_retain_params_to_documents.py b/hindsight-api-slim/hindsight_api/alembic/versions/c8e5f2a3b4d1_add_retain_params_to_documents.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/c8e5f2a3b4d1_add_retain_params_to_documents.py rename to hindsight-api-slim/hindsight_api/alembic/versions/c8e5f2a3b4d1_add_retain_params_to_documents.py diff --git a/hindsight-api/hindsight_api/alembic/versions/d2e3f4a5b6c7_add_memory_links_expansion_indexes.py b/hindsight-api-slim/hindsight_api/alembic/versions/d2e3f4a5b6c7_add_memory_links_expansion_indexes.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/d2e3f4a5b6c7_add_memory_links_expansion_indexes.py rename to hindsight-api-slim/hindsight_api/alembic/versions/d2e3f4a5b6c7_add_memory_links_expansion_indexes.py diff --git a/hindsight-api/hindsight_api/alembic/versions/d4e5f6g7h8i9_gin_source_memory_ids_fastupdate_off.py b/hindsight-api-slim/hindsight_api/alembic/versions/d4e5f6g7h8i9_gin_source_memory_ids_fastupdate_off.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/d4e5f6g7h8i9_gin_source_memory_ids_fastupdate_off.py rename to hindsight-api-slim/hindsight_api/alembic/versions/d4e5f6g7h8i9_gin_source_memory_ids_fastupdate_off.py diff --git a/hindsight-api/hindsight_api/alembic/versions/d5e6f7a8b9c0_add_bank_internal_id_and_per_bank_hnsw.py b/hindsight-api-slim/hindsight_api/alembic/versions/d5e6f7a8b9c0_add_bank_internal_id_and_per_bank_hnsw.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/d5e6f7a8b9c0_add_bank_internal_id_and_per_bank_hnsw.py rename to hindsight-api-slim/hindsight_api/alembic/versions/d5e6f7a8b9c0_add_bank_internal_id_and_per_bank_hnsw.py diff --git a/hindsight-api/hindsight_api/alembic/versions/d9f6a3b4c5e2_rename_bank_to_interactions.py b/hindsight-api-slim/hindsight_api/alembic/versions/d9f6a3b4c5e2_rename_bank_to_interactions.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/d9f6a3b4c5e2_rename_bank_to_interactions.py rename to hindsight-api-slim/hindsight_api/alembic/versions/d9f6a3b4c5e2_rename_bank_to_interactions.py diff --git a/hindsight-api/hindsight_api/alembic/versions/e0a1b2c3d4e5_disposition_to_3_traits.py b/hindsight-api-slim/hindsight_api/alembic/versions/e0a1b2c3d4e5_disposition_to_3_traits.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/e0a1b2c3d4e5_disposition_to_3_traits.py rename to hindsight-api-slim/hindsight_api/alembic/versions/e0a1b2c3d4e5_disposition_to_3_traits.py diff --git a/hindsight-api/hindsight_api/alembic/versions/e4f5a6b7c8d9_add_webhooks_tables.py b/hindsight-api-slim/hindsight_api/alembic/versions/e4f5a6b7c8d9_add_webhooks_tables.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/e4f5a6b7c8d9_add_webhooks_tables.py rename to hindsight-api-slim/hindsight_api/alembic/versions/e4f5a6b7c8d9_add_webhooks_tables.py diff --git a/hindsight-api/hindsight_api/alembic/versions/e5f6g7h8i9j0_cascade_delete_ops_on_bank_delete.py b/hindsight-api-slim/hindsight_api/alembic/versions/e5f6g7h8i9j0_cascade_delete_ops_on_bank_delete.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/e5f6g7h8i9j0_cascade_delete_ops_on_bank_delete.py rename to hindsight-api-slim/hindsight_api/alembic/versions/e5f6g7h8i9j0_cascade_delete_ops_on_bank_delete.py diff --git a/hindsight-api/hindsight_api/alembic/versions/f1a2b3c4d5e6_add_memory_links_composite_index.py b/hindsight-api-slim/hindsight_api/alembic/versions/f1a2b3c4d5e6_add_memory_links_composite_index.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/f1a2b3c4d5e6_add_memory_links_composite_index.py rename to hindsight-api-slim/hindsight_api/alembic/versions/f1a2b3c4d5e6_add_memory_links_composite_index.py diff --git a/hindsight-api/hindsight_api/alembic/versions/f7g8h9i0j1k2_add_webhook_http_config.py b/hindsight-api-slim/hindsight_api/alembic/versions/f7g8h9i0j1k2_add_webhook_http_config.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/f7g8h9i0j1k2_add_webhook_http_config.py rename to hindsight-api-slim/hindsight_api/alembic/versions/f7g8h9i0j1k2_add_webhook_http_config.py diff --git a/hindsight-api/hindsight_api/alembic/versions/g2a3b4c5d6e7_add_tags_column.py b/hindsight-api-slim/hindsight_api/alembic/versions/g2a3b4c5d6e7_add_tags_column.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/g2a3b4c5d6e7_add_tags_column.py rename to hindsight-api-slim/hindsight_api/alembic/versions/g2a3b4c5d6e7_add_tags_column.py diff --git a/hindsight-api/hindsight_api/alembic/versions/h3c4d5e6f7g8_mental_models_v4.py b/hindsight-api-slim/hindsight_api/alembic/versions/h3c4d5e6f7g8_mental_models_v4.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/h3c4d5e6f7g8_mental_models_v4.py rename to hindsight-api-slim/hindsight_api/alembic/versions/h3c4d5e6f7g8_mental_models_v4.py diff --git a/hindsight-api/hindsight_api/alembic/versions/i4d5e6f7g8h9_delete_opinions.py b/hindsight-api-slim/hindsight_api/alembic/versions/i4d5e6f7g8h9_delete_opinions.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/i4d5e6f7g8h9_delete_opinions.py rename to hindsight-api-slim/hindsight_api/alembic/versions/i4d5e6f7g8h9_delete_opinions.py diff --git a/hindsight-api/hindsight_api/alembic/versions/j5e6f7g8h9i0_mental_model_versions.py b/hindsight-api-slim/hindsight_api/alembic/versions/j5e6f7g8h9i0_mental_model_versions.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/j5e6f7g8h9i0_mental_model_versions.py rename to hindsight-api-slim/hindsight_api/alembic/versions/j5e6f7g8h9i0_mental_model_versions.py diff --git a/hindsight-api/hindsight_api/alembic/versions/k6f7g8h9i0j1_add_directive_subtype.py b/hindsight-api-slim/hindsight_api/alembic/versions/k6f7g8h9i0j1_add_directive_subtype.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/k6f7g8h9i0j1_add_directive_subtype.py rename to hindsight-api-slim/hindsight_api/alembic/versions/k6f7g8h9i0j1_add_directive_subtype.py diff --git a/hindsight-api/hindsight_api/alembic/versions/l7g8h9i0j1k2_add_worker_columns.py b/hindsight-api-slim/hindsight_api/alembic/versions/l7g8h9i0j1k2_add_worker_columns.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/l7g8h9i0j1k2_add_worker_columns.py rename to hindsight-api-slim/hindsight_api/alembic/versions/l7g8h9i0j1k2_add_worker_columns.py diff --git a/hindsight-api/hindsight_api/alembic/versions/m8h9i0j1k2l3_mental_model_id_to_text.py b/hindsight-api-slim/hindsight_api/alembic/versions/m8h9i0j1k2l3_mental_model_id_to_text.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/m8h9i0j1k2l3_mental_model_id_to_text.py rename to hindsight-api-slim/hindsight_api/alembic/versions/m8h9i0j1k2l3_mental_model_id_to_text.py diff --git a/hindsight-api/hindsight_api/alembic/versions/n9i0j1k2l3m4_learnings_and_pinned_reflections.py b/hindsight-api-slim/hindsight_api/alembic/versions/n9i0j1k2l3m4_learnings_and_pinned_reflections.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/n9i0j1k2l3m4_learnings_and_pinned_reflections.py rename to hindsight-api-slim/hindsight_api/alembic/versions/n9i0j1k2l3m4_learnings_and_pinned_reflections.py diff --git a/hindsight-api/hindsight_api/alembic/versions/o0j1k2l3m4n5_migrate_mental_models_data.py b/hindsight-api-slim/hindsight_api/alembic/versions/o0j1k2l3m4n5_migrate_mental_models_data.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/o0j1k2l3m4n5_migrate_mental_models_data.py rename to hindsight-api-slim/hindsight_api/alembic/versions/o0j1k2l3m4n5_migrate_mental_models_data.py diff --git a/hindsight-api/hindsight_api/alembic/versions/p1k2l3m4n5o6_new_knowledge_architecture.py b/hindsight-api-slim/hindsight_api/alembic/versions/p1k2l3m4n5o6_new_knowledge_architecture.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/p1k2l3m4n5o6_new_knowledge_architecture.py rename to hindsight-api-slim/hindsight_api/alembic/versions/p1k2l3m4n5o6_new_knowledge_architecture.py diff --git a/hindsight-api/hindsight_api/alembic/versions/q2l3m4n5o6p7_fix_mental_model_fact_type.py b/hindsight-api-slim/hindsight_api/alembic/versions/q2l3m4n5o6p7_fix_mental_model_fact_type.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/q2l3m4n5o6p7_fix_mental_model_fact_type.py rename to hindsight-api-slim/hindsight_api/alembic/versions/q2l3m4n5o6p7_fix_mental_model_fact_type.py diff --git a/hindsight-api/hindsight_api/alembic/versions/r3m4n5o6p7q8_add_reflect_response_to_reflections.py b/hindsight-api-slim/hindsight_api/alembic/versions/r3m4n5o6p7q8_add_reflect_response_to_reflections.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/r3m4n5o6p7q8_add_reflect_response_to_reflections.py rename to hindsight-api-slim/hindsight_api/alembic/versions/r3m4n5o6p7q8_add_reflect_response_to_reflections.py diff --git a/hindsight-api/hindsight_api/alembic/versions/rename_personality_to_disposition.py b/hindsight-api-slim/hindsight_api/alembic/versions/rename_personality_to_disposition.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/rename_personality_to_disposition.py rename to hindsight-api-slim/hindsight_api/alembic/versions/rename_personality_to_disposition.py diff --git a/hindsight-api/hindsight_api/alembic/versions/s4n5o6p7q8r9_add_consolidated_at_to_memory_units.py b/hindsight-api-slim/hindsight_api/alembic/versions/s4n5o6p7q8r9_add_consolidated_at_to_memory_units.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/s4n5o6p7q8r9_add_consolidated_at_to_memory_units.py rename to hindsight-api-slim/hindsight_api/alembic/versions/s4n5o6p7q8r9_add_consolidated_at_to_memory_units.py diff --git a/hindsight-api/hindsight_api/alembic/versions/t5o6p7q8r9s0_rename_mental_models_to_observations.py b/hindsight-api-slim/hindsight_api/alembic/versions/t5o6p7q8r9s0_rename_mental_models_to_observations.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/t5o6p7q8r9s0_rename_mental_models_to_observations.py rename to hindsight-api-slim/hindsight_api/alembic/versions/t5o6p7q8r9s0_rename_mental_models_to_observations.py diff --git a/hindsight-api/hindsight_api/alembic/versions/u6p7q8r9s0t1_mental_models_text_id.py b/hindsight-api-slim/hindsight_api/alembic/versions/u6p7q8r9s0t1_mental_models_text_id.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/u6p7q8r9s0t1_mental_models_text_id.py rename to hindsight-api-slim/hindsight_api/alembic/versions/u6p7q8r9s0t1_mental_models_text_id.py diff --git a/hindsight-api/hindsight_api/alembic/versions/v7q8r9s0t1u2_add_max_tokens_to_mental_models.py b/hindsight-api-slim/hindsight_api/alembic/versions/v7q8r9s0t1u2_add_max_tokens_to_mental_models.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/v7q8r9s0t1u2_add_max_tokens_to_mental_models.py rename to hindsight-api-slim/hindsight_api/alembic/versions/v7q8r9s0t1u2_add_max_tokens_to_mental_models.py diff --git a/hindsight-api/hindsight_api/alembic/versions/w8r9s0t1u2v3_fix_mental_models_pk_isolation.py b/hindsight-api-slim/hindsight_api/alembic/versions/w8r9s0t1u2v3_fix_mental_models_pk_isolation.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/w8r9s0t1u2v3_fix_mental_models_pk_isolation.py rename to hindsight-api-slim/hindsight_api/alembic/versions/w8r9s0t1u2v3_fix_mental_models_pk_isolation.py diff --git a/hindsight-api/hindsight_api/alembic/versions/x9s0t1u2v3w4_add_bank_config_column.py b/hindsight-api-slim/hindsight_api/alembic/versions/x9s0t1u2v3w4_add_bank_config_column.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/x9s0t1u2v3w4_add_bank_config_column.py rename to hindsight-api-slim/hindsight_api/alembic/versions/x9s0t1u2v3w4_add_bank_config_column.py diff --git a/hindsight-api/hindsight_api/alembic/versions/y0t1u2v3w4x5_add_result_metadata_gin_index.py b/hindsight-api-slim/hindsight_api/alembic/versions/y0t1u2v3w4x5_add_result_metadata_gin_index.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/y0t1u2v3w4x5_add_result_metadata_gin_index.py rename to hindsight-api-slim/hindsight_api/alembic/versions/y0t1u2v3w4x5_add_result_metadata_gin_index.py diff --git a/hindsight-api/hindsight_api/alembic/versions/z1u2v3w4x5y6_add_observation_tags_to_memory_units.py b/hindsight-api-slim/hindsight_api/alembic/versions/z1u2v3w4x5y6_add_observation_tags_to_memory_units.py similarity index 100% rename from hindsight-api/hindsight_api/alembic/versions/z1u2v3w4x5y6_add_observation_tags_to_memory_units.py rename to hindsight-api-slim/hindsight_api/alembic/versions/z1u2v3w4x5y6_add_observation_tags_to_memory_units.py diff --git a/hindsight-api/hindsight_api/api/__init__.py b/hindsight-api-slim/hindsight_api/api/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/api/__init__.py rename to hindsight-api-slim/hindsight_api/api/__init__.py diff --git a/hindsight-api/hindsight_api/api/http.py b/hindsight-api-slim/hindsight_api/api/http.py similarity index 100% rename from hindsight-api/hindsight_api/api/http.py rename to hindsight-api-slim/hindsight_api/api/http.py diff --git a/hindsight-api/hindsight_api/api/mcp.py b/hindsight-api-slim/hindsight_api/api/mcp.py similarity index 100% rename from hindsight-api/hindsight_api/api/mcp.py rename to hindsight-api-slim/hindsight_api/api/mcp.py diff --git a/hindsight-api/hindsight_api/banner.py b/hindsight-api-slim/hindsight_api/banner.py similarity index 100% rename from hindsight-api/hindsight_api/banner.py rename to hindsight-api-slim/hindsight_api/banner.py diff --git a/hindsight-api/hindsight_api/config.py b/hindsight-api-slim/hindsight_api/config.py similarity index 100% rename from hindsight-api/hindsight_api/config.py rename to hindsight-api-slim/hindsight_api/config.py diff --git a/hindsight-api/hindsight_api/config_resolver.py b/hindsight-api-slim/hindsight_api/config_resolver.py similarity index 100% rename from hindsight-api/hindsight_api/config_resolver.py rename to hindsight-api-slim/hindsight_api/config_resolver.py diff --git a/hindsight-api/hindsight_api/daemon.py b/hindsight-api-slim/hindsight_api/daemon.py similarity index 100% rename from hindsight-api/hindsight_api/daemon.py rename to hindsight-api-slim/hindsight_api/daemon.py diff --git a/hindsight-api/hindsight_api/engine/__init__.py b/hindsight-api-slim/hindsight_api/engine/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/engine/__init__.py rename to hindsight-api-slim/hindsight_api/engine/__init__.py diff --git a/hindsight-api/hindsight_api/engine/consolidation/__init__.py b/hindsight-api-slim/hindsight_api/engine/consolidation/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/engine/consolidation/__init__.py rename to hindsight-api-slim/hindsight_api/engine/consolidation/__init__.py diff --git a/hindsight-api/hindsight_api/engine/consolidation/consolidator.py b/hindsight-api-slim/hindsight_api/engine/consolidation/consolidator.py similarity index 100% rename from hindsight-api/hindsight_api/engine/consolidation/consolidator.py rename to hindsight-api-slim/hindsight_api/engine/consolidation/consolidator.py diff --git a/hindsight-api/hindsight_api/engine/consolidation/prompts.py b/hindsight-api-slim/hindsight_api/engine/consolidation/prompts.py similarity index 100% rename from hindsight-api/hindsight_api/engine/consolidation/prompts.py rename to hindsight-api-slim/hindsight_api/engine/consolidation/prompts.py diff --git a/hindsight-api/hindsight_api/engine/cross_encoder.py b/hindsight-api-slim/hindsight_api/engine/cross_encoder.py similarity index 100% rename from hindsight-api/hindsight_api/engine/cross_encoder.py rename to hindsight-api-slim/hindsight_api/engine/cross_encoder.py diff --git a/hindsight-api/hindsight_api/engine/db_budget.py b/hindsight-api-slim/hindsight_api/engine/db_budget.py similarity index 100% rename from hindsight-api/hindsight_api/engine/db_budget.py rename to hindsight-api-slim/hindsight_api/engine/db_budget.py diff --git a/hindsight-api/hindsight_api/engine/db_utils.py b/hindsight-api-slim/hindsight_api/engine/db_utils.py similarity index 100% rename from hindsight-api/hindsight_api/engine/db_utils.py rename to hindsight-api-slim/hindsight_api/engine/db_utils.py diff --git a/hindsight-api/hindsight_api/engine/directives/__init__.py b/hindsight-api-slim/hindsight_api/engine/directives/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/engine/directives/__init__.py rename to hindsight-api-slim/hindsight_api/engine/directives/__init__.py diff --git a/hindsight-api/hindsight_api/engine/directives/models.py b/hindsight-api-slim/hindsight_api/engine/directives/models.py similarity index 100% rename from hindsight-api/hindsight_api/engine/directives/models.py rename to hindsight-api-slim/hindsight_api/engine/directives/models.py diff --git a/hindsight-api/hindsight_api/engine/embeddings.py b/hindsight-api-slim/hindsight_api/engine/embeddings.py similarity index 100% rename from hindsight-api/hindsight_api/engine/embeddings.py rename to hindsight-api-slim/hindsight_api/engine/embeddings.py diff --git a/hindsight-api/hindsight_api/engine/entity_resolver.py b/hindsight-api-slim/hindsight_api/engine/entity_resolver.py similarity index 100% rename from hindsight-api/hindsight_api/engine/entity_resolver.py rename to hindsight-api-slim/hindsight_api/engine/entity_resolver.py diff --git a/hindsight-api/hindsight_api/engine/interface.py b/hindsight-api-slim/hindsight_api/engine/interface.py similarity index 100% rename from hindsight-api/hindsight_api/engine/interface.py rename to hindsight-api-slim/hindsight_api/engine/interface.py diff --git a/hindsight-api/hindsight_api/engine/jina_mlx_reranker.py b/hindsight-api-slim/hindsight_api/engine/jina_mlx_reranker.py similarity index 100% rename from hindsight-api/hindsight_api/engine/jina_mlx_reranker.py rename to hindsight-api-slim/hindsight_api/engine/jina_mlx_reranker.py diff --git a/hindsight-api/hindsight_api/engine/llm_interface.py b/hindsight-api-slim/hindsight_api/engine/llm_interface.py similarity index 100% rename from hindsight-api/hindsight_api/engine/llm_interface.py rename to hindsight-api-slim/hindsight_api/engine/llm_interface.py diff --git a/hindsight-api/hindsight_api/engine/llm_wrapper.py b/hindsight-api-slim/hindsight_api/engine/llm_wrapper.py similarity index 100% rename from hindsight-api/hindsight_api/engine/llm_wrapper.py rename to hindsight-api-slim/hindsight_api/engine/llm_wrapper.py diff --git a/hindsight-api/hindsight_api/engine/memory_engine.py b/hindsight-api-slim/hindsight_api/engine/memory_engine.py similarity index 100% rename from hindsight-api/hindsight_api/engine/memory_engine.py rename to hindsight-api-slim/hindsight_api/engine/memory_engine.py diff --git a/hindsight-api/hindsight_api/engine/operation_metadata.py b/hindsight-api-slim/hindsight_api/engine/operation_metadata.py similarity index 100% rename from hindsight-api/hindsight_api/engine/operation_metadata.py rename to hindsight-api-slim/hindsight_api/engine/operation_metadata.py diff --git a/hindsight-api/hindsight_api/engine/parsers/__init__.py b/hindsight-api-slim/hindsight_api/engine/parsers/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/engine/parsers/__init__.py rename to hindsight-api-slim/hindsight_api/engine/parsers/__init__.py diff --git a/hindsight-api/hindsight_api/engine/parsers/base.py b/hindsight-api-slim/hindsight_api/engine/parsers/base.py similarity index 100% rename from hindsight-api/hindsight_api/engine/parsers/base.py rename to hindsight-api-slim/hindsight_api/engine/parsers/base.py diff --git a/hindsight-api/hindsight_api/engine/parsers/iris.py b/hindsight-api-slim/hindsight_api/engine/parsers/iris.py similarity index 100% rename from hindsight-api/hindsight_api/engine/parsers/iris.py rename to hindsight-api-slim/hindsight_api/engine/parsers/iris.py diff --git a/hindsight-api/hindsight_api/engine/parsers/markitdown.py b/hindsight-api-slim/hindsight_api/engine/parsers/markitdown.py similarity index 100% rename from hindsight-api/hindsight_api/engine/parsers/markitdown.py rename to hindsight-api-slim/hindsight_api/engine/parsers/markitdown.py diff --git a/hindsight-api/hindsight_api/engine/providers/__init__.py b/hindsight-api-slim/hindsight_api/engine/providers/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/engine/providers/__init__.py rename to hindsight-api-slim/hindsight_api/engine/providers/__init__.py diff --git a/hindsight-api/hindsight_api/engine/providers/anthropic_llm.py b/hindsight-api-slim/hindsight_api/engine/providers/anthropic_llm.py similarity index 100% rename from hindsight-api/hindsight_api/engine/providers/anthropic_llm.py rename to hindsight-api-slim/hindsight_api/engine/providers/anthropic_llm.py diff --git a/hindsight-api/hindsight_api/engine/providers/claude_code_llm.py b/hindsight-api-slim/hindsight_api/engine/providers/claude_code_llm.py similarity index 100% rename from hindsight-api/hindsight_api/engine/providers/claude_code_llm.py rename to hindsight-api-slim/hindsight_api/engine/providers/claude_code_llm.py diff --git a/hindsight-api/hindsight_api/engine/providers/codex_llm.py b/hindsight-api-slim/hindsight_api/engine/providers/codex_llm.py similarity index 100% rename from hindsight-api/hindsight_api/engine/providers/codex_llm.py rename to hindsight-api-slim/hindsight_api/engine/providers/codex_llm.py diff --git a/hindsight-api/hindsight_api/engine/providers/gemini_llm.py b/hindsight-api-slim/hindsight_api/engine/providers/gemini_llm.py similarity index 100% rename from hindsight-api/hindsight_api/engine/providers/gemini_llm.py rename to hindsight-api-slim/hindsight_api/engine/providers/gemini_llm.py diff --git a/hindsight-api/hindsight_api/engine/providers/mock_llm.py b/hindsight-api-slim/hindsight_api/engine/providers/mock_llm.py similarity index 100% rename from hindsight-api/hindsight_api/engine/providers/mock_llm.py rename to hindsight-api-slim/hindsight_api/engine/providers/mock_llm.py diff --git a/hindsight-api/hindsight_api/engine/providers/openai_compatible_llm.py b/hindsight-api-slim/hindsight_api/engine/providers/openai_compatible_llm.py similarity index 100% rename from hindsight-api/hindsight_api/engine/providers/openai_compatible_llm.py rename to hindsight-api-slim/hindsight_api/engine/providers/openai_compatible_llm.py diff --git a/hindsight-api/hindsight_api/engine/query_analyzer.py b/hindsight-api-slim/hindsight_api/engine/query_analyzer.py similarity index 100% rename from hindsight-api/hindsight_api/engine/query_analyzer.py rename to hindsight-api-slim/hindsight_api/engine/query_analyzer.py diff --git a/hindsight-api/hindsight_api/engine/reflect/__init__.py b/hindsight-api-slim/hindsight_api/engine/reflect/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/engine/reflect/__init__.py rename to hindsight-api-slim/hindsight_api/engine/reflect/__init__.py diff --git a/hindsight-api/hindsight_api/engine/reflect/agent.py b/hindsight-api-slim/hindsight_api/engine/reflect/agent.py similarity index 100% rename from hindsight-api/hindsight_api/engine/reflect/agent.py rename to hindsight-api-slim/hindsight_api/engine/reflect/agent.py diff --git a/hindsight-api/hindsight_api/engine/reflect/models.py b/hindsight-api-slim/hindsight_api/engine/reflect/models.py similarity index 100% rename from hindsight-api/hindsight_api/engine/reflect/models.py rename to hindsight-api-slim/hindsight_api/engine/reflect/models.py diff --git a/hindsight-api/hindsight_api/engine/reflect/observations.py b/hindsight-api-slim/hindsight_api/engine/reflect/observations.py similarity index 100% rename from hindsight-api/hindsight_api/engine/reflect/observations.py rename to hindsight-api-slim/hindsight_api/engine/reflect/observations.py diff --git a/hindsight-api/hindsight_api/engine/reflect/prompts.py b/hindsight-api-slim/hindsight_api/engine/reflect/prompts.py similarity index 100% rename from hindsight-api/hindsight_api/engine/reflect/prompts.py rename to hindsight-api-slim/hindsight_api/engine/reflect/prompts.py diff --git a/hindsight-api/hindsight_api/engine/reflect/tools.py b/hindsight-api-slim/hindsight_api/engine/reflect/tools.py similarity index 100% rename from hindsight-api/hindsight_api/engine/reflect/tools.py rename to hindsight-api-slim/hindsight_api/engine/reflect/tools.py diff --git a/hindsight-api/hindsight_api/engine/reflect/tools_schema.py b/hindsight-api-slim/hindsight_api/engine/reflect/tools_schema.py similarity index 100% rename from hindsight-api/hindsight_api/engine/reflect/tools_schema.py rename to hindsight-api-slim/hindsight_api/engine/reflect/tools_schema.py diff --git a/hindsight-api/hindsight_api/engine/response_models.py b/hindsight-api-slim/hindsight_api/engine/response_models.py similarity index 100% rename from hindsight-api/hindsight_api/engine/response_models.py rename to hindsight-api-slim/hindsight_api/engine/response_models.py diff --git a/hindsight-api/hindsight_api/engine/retain/__init__.py b/hindsight-api-slim/hindsight_api/engine/retain/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/__init__.py rename to hindsight-api-slim/hindsight_api/engine/retain/__init__.py diff --git a/hindsight-api/hindsight_api/engine/retain/bank_utils.py b/hindsight-api-slim/hindsight_api/engine/retain/bank_utils.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/bank_utils.py rename to hindsight-api-slim/hindsight_api/engine/retain/bank_utils.py diff --git a/hindsight-api/hindsight_api/engine/retain/chunk_storage.py b/hindsight-api-slim/hindsight_api/engine/retain/chunk_storage.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/chunk_storage.py rename to hindsight-api-slim/hindsight_api/engine/retain/chunk_storage.py diff --git a/hindsight-api/hindsight_api/engine/retain/embedding_processing.py b/hindsight-api-slim/hindsight_api/engine/retain/embedding_processing.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/embedding_processing.py rename to hindsight-api-slim/hindsight_api/engine/retain/embedding_processing.py diff --git a/hindsight-api/hindsight_api/engine/retain/embedding_utils.py b/hindsight-api-slim/hindsight_api/engine/retain/embedding_utils.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/embedding_utils.py rename to hindsight-api-slim/hindsight_api/engine/retain/embedding_utils.py diff --git a/hindsight-api/hindsight_api/engine/retain/entity_labels.py b/hindsight-api-slim/hindsight_api/engine/retain/entity_labels.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/entity_labels.py rename to hindsight-api-slim/hindsight_api/engine/retain/entity_labels.py diff --git a/hindsight-api/hindsight_api/engine/retain/entity_processing.py b/hindsight-api-slim/hindsight_api/engine/retain/entity_processing.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/entity_processing.py rename to hindsight-api-slim/hindsight_api/engine/retain/entity_processing.py diff --git a/hindsight-api/hindsight_api/engine/retain/fact_extraction.py b/hindsight-api-slim/hindsight_api/engine/retain/fact_extraction.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/fact_extraction.py rename to hindsight-api-slim/hindsight_api/engine/retain/fact_extraction.py diff --git a/hindsight-api/hindsight_api/engine/retain/fact_storage.py b/hindsight-api-slim/hindsight_api/engine/retain/fact_storage.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/fact_storage.py rename to hindsight-api-slim/hindsight_api/engine/retain/fact_storage.py diff --git a/hindsight-api/hindsight_api/engine/retain/link_creation.py b/hindsight-api-slim/hindsight_api/engine/retain/link_creation.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/link_creation.py rename to hindsight-api-slim/hindsight_api/engine/retain/link_creation.py diff --git a/hindsight-api/hindsight_api/engine/retain/link_utils.py b/hindsight-api-slim/hindsight_api/engine/retain/link_utils.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/link_utils.py rename to hindsight-api-slim/hindsight_api/engine/retain/link_utils.py diff --git a/hindsight-api/hindsight_api/engine/retain/orchestrator.py b/hindsight-api-slim/hindsight_api/engine/retain/orchestrator.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/orchestrator.py rename to hindsight-api-slim/hindsight_api/engine/retain/orchestrator.py diff --git a/hindsight-api/hindsight_api/engine/retain/types.py b/hindsight-api-slim/hindsight_api/engine/retain/types.py similarity index 100% rename from hindsight-api/hindsight_api/engine/retain/types.py rename to hindsight-api-slim/hindsight_api/engine/retain/types.py diff --git a/hindsight-api/hindsight_api/engine/search/__init__.py b/hindsight-api-slim/hindsight_api/engine/search/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/__init__.py rename to hindsight-api-slim/hindsight_api/engine/search/__init__.py diff --git a/hindsight-api/hindsight_api/engine/search/fusion.py b/hindsight-api-slim/hindsight_api/engine/search/fusion.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/fusion.py rename to hindsight-api-slim/hindsight_api/engine/search/fusion.py diff --git a/hindsight-api/hindsight_api/engine/search/graph_retrieval.py b/hindsight-api-slim/hindsight_api/engine/search/graph_retrieval.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/graph_retrieval.py rename to hindsight-api-slim/hindsight_api/engine/search/graph_retrieval.py diff --git a/hindsight-api/hindsight_api/engine/search/link_expansion_retrieval.py b/hindsight-api-slim/hindsight_api/engine/search/link_expansion_retrieval.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/link_expansion_retrieval.py rename to hindsight-api-slim/hindsight_api/engine/search/link_expansion_retrieval.py diff --git a/hindsight-api/hindsight_api/engine/search/mpfp_retrieval.py b/hindsight-api-slim/hindsight_api/engine/search/mpfp_retrieval.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/mpfp_retrieval.py rename to hindsight-api-slim/hindsight_api/engine/search/mpfp_retrieval.py diff --git a/hindsight-api/hindsight_api/engine/search/reranking.py b/hindsight-api-slim/hindsight_api/engine/search/reranking.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/reranking.py rename to hindsight-api-slim/hindsight_api/engine/search/reranking.py diff --git a/hindsight-api/hindsight_api/engine/search/retrieval.py b/hindsight-api-slim/hindsight_api/engine/search/retrieval.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/retrieval.py rename to hindsight-api-slim/hindsight_api/engine/search/retrieval.py diff --git a/hindsight-api/hindsight_api/engine/search/tags.py b/hindsight-api-slim/hindsight_api/engine/search/tags.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/tags.py rename to hindsight-api-slim/hindsight_api/engine/search/tags.py diff --git a/hindsight-api/hindsight_api/engine/search/temporal_extraction.py b/hindsight-api-slim/hindsight_api/engine/search/temporal_extraction.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/temporal_extraction.py rename to hindsight-api-slim/hindsight_api/engine/search/temporal_extraction.py diff --git a/hindsight-api/hindsight_api/engine/search/think_utils.py b/hindsight-api-slim/hindsight_api/engine/search/think_utils.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/think_utils.py rename to hindsight-api-slim/hindsight_api/engine/search/think_utils.py diff --git a/hindsight-api/hindsight_api/engine/search/trace.py b/hindsight-api-slim/hindsight_api/engine/search/trace.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/trace.py rename to hindsight-api-slim/hindsight_api/engine/search/trace.py diff --git a/hindsight-api/hindsight_api/engine/search/tracer.py b/hindsight-api-slim/hindsight_api/engine/search/tracer.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/tracer.py rename to hindsight-api-slim/hindsight_api/engine/search/tracer.py diff --git a/hindsight-api/hindsight_api/engine/search/types.py b/hindsight-api-slim/hindsight_api/engine/search/types.py similarity index 100% rename from hindsight-api/hindsight_api/engine/search/types.py rename to hindsight-api-slim/hindsight_api/engine/search/types.py diff --git a/hindsight-api/hindsight_api/engine/storage/__init__.py b/hindsight-api-slim/hindsight_api/engine/storage/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/engine/storage/__init__.py rename to hindsight-api-slim/hindsight_api/engine/storage/__init__.py diff --git a/hindsight-api/hindsight_api/engine/storage/azure.py b/hindsight-api-slim/hindsight_api/engine/storage/azure.py similarity index 100% rename from hindsight-api/hindsight_api/engine/storage/azure.py rename to hindsight-api-slim/hindsight_api/engine/storage/azure.py diff --git a/hindsight-api/hindsight_api/engine/storage/base.py b/hindsight-api-slim/hindsight_api/engine/storage/base.py similarity index 100% rename from hindsight-api/hindsight_api/engine/storage/base.py rename to hindsight-api-slim/hindsight_api/engine/storage/base.py diff --git a/hindsight-api/hindsight_api/engine/storage/gcs.py b/hindsight-api-slim/hindsight_api/engine/storage/gcs.py similarity index 100% rename from hindsight-api/hindsight_api/engine/storage/gcs.py rename to hindsight-api-slim/hindsight_api/engine/storage/gcs.py diff --git a/hindsight-api/hindsight_api/engine/storage/postgresql.py b/hindsight-api-slim/hindsight_api/engine/storage/postgresql.py similarity index 100% rename from hindsight-api/hindsight_api/engine/storage/postgresql.py rename to hindsight-api-slim/hindsight_api/engine/storage/postgresql.py diff --git a/hindsight-api/hindsight_api/engine/storage/s3.py b/hindsight-api-slim/hindsight_api/engine/storage/s3.py similarity index 100% rename from hindsight-api/hindsight_api/engine/storage/s3.py rename to hindsight-api-slim/hindsight_api/engine/storage/s3.py diff --git a/hindsight-api/hindsight_api/engine/task_backend.py b/hindsight-api-slim/hindsight_api/engine/task_backend.py similarity index 100% rename from hindsight-api/hindsight_api/engine/task_backend.py rename to hindsight-api-slim/hindsight_api/engine/task_backend.py diff --git a/hindsight-api/hindsight_api/engine/utils.py b/hindsight-api-slim/hindsight_api/engine/utils.py similarity index 100% rename from hindsight-api/hindsight_api/engine/utils.py rename to hindsight-api-slim/hindsight_api/engine/utils.py diff --git a/hindsight-api/hindsight_api/extensions/__init__.py b/hindsight-api-slim/hindsight_api/extensions/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/extensions/__init__.py rename to hindsight-api-slim/hindsight_api/extensions/__init__.py diff --git a/hindsight-api/hindsight_api/extensions/base.py b/hindsight-api-slim/hindsight_api/extensions/base.py similarity index 100% rename from hindsight-api/hindsight_api/extensions/base.py rename to hindsight-api-slim/hindsight_api/extensions/base.py diff --git a/hindsight-api/hindsight_api/extensions/builtin/__init__.py b/hindsight-api-slim/hindsight_api/extensions/builtin/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/extensions/builtin/__init__.py rename to hindsight-api-slim/hindsight_api/extensions/builtin/__init__.py diff --git a/hindsight-api/hindsight_api/extensions/builtin/supabase_tenant.py b/hindsight-api-slim/hindsight_api/extensions/builtin/supabase_tenant.py similarity index 100% rename from hindsight-api/hindsight_api/extensions/builtin/supabase_tenant.py rename to hindsight-api-slim/hindsight_api/extensions/builtin/supabase_tenant.py diff --git a/hindsight-api/hindsight_api/extensions/builtin/tenant.py b/hindsight-api-slim/hindsight_api/extensions/builtin/tenant.py similarity index 100% rename from hindsight-api/hindsight_api/extensions/builtin/tenant.py rename to hindsight-api-slim/hindsight_api/extensions/builtin/tenant.py diff --git a/hindsight-api/hindsight_api/extensions/context.py b/hindsight-api-slim/hindsight_api/extensions/context.py similarity index 100% rename from hindsight-api/hindsight_api/extensions/context.py rename to hindsight-api-slim/hindsight_api/extensions/context.py diff --git a/hindsight-api/hindsight_api/extensions/http.py b/hindsight-api-slim/hindsight_api/extensions/http.py similarity index 100% rename from hindsight-api/hindsight_api/extensions/http.py rename to hindsight-api-slim/hindsight_api/extensions/http.py diff --git a/hindsight-api/hindsight_api/extensions/loader.py b/hindsight-api-slim/hindsight_api/extensions/loader.py similarity index 100% rename from hindsight-api/hindsight_api/extensions/loader.py rename to hindsight-api-slim/hindsight_api/extensions/loader.py diff --git a/hindsight-api/hindsight_api/extensions/mcp.py b/hindsight-api-slim/hindsight_api/extensions/mcp.py similarity index 100% rename from hindsight-api/hindsight_api/extensions/mcp.py rename to hindsight-api-slim/hindsight_api/extensions/mcp.py diff --git a/hindsight-api/hindsight_api/extensions/operation_validator.py b/hindsight-api-slim/hindsight_api/extensions/operation_validator.py similarity index 100% rename from hindsight-api/hindsight_api/extensions/operation_validator.py rename to hindsight-api-slim/hindsight_api/extensions/operation_validator.py diff --git a/hindsight-api/hindsight_api/extensions/tenant.py b/hindsight-api-slim/hindsight_api/extensions/tenant.py similarity index 100% rename from hindsight-api/hindsight_api/extensions/tenant.py rename to hindsight-api-slim/hindsight_api/extensions/tenant.py diff --git a/hindsight-api/hindsight_api/main.py b/hindsight-api-slim/hindsight_api/main.py similarity index 100% rename from hindsight-api/hindsight_api/main.py rename to hindsight-api-slim/hindsight_api/main.py diff --git a/hindsight-api/hindsight_api/mcp_local.py b/hindsight-api-slim/hindsight_api/mcp_local.py similarity index 100% rename from hindsight-api/hindsight_api/mcp_local.py rename to hindsight-api-slim/hindsight_api/mcp_local.py diff --git a/hindsight-api/hindsight_api/mcp_tools.py b/hindsight-api-slim/hindsight_api/mcp_tools.py similarity index 100% rename from hindsight-api/hindsight_api/mcp_tools.py rename to hindsight-api-slim/hindsight_api/mcp_tools.py diff --git a/hindsight-api/hindsight_api/metrics.py b/hindsight-api-slim/hindsight_api/metrics.py similarity index 100% rename from hindsight-api/hindsight_api/metrics.py rename to hindsight-api-slim/hindsight_api/metrics.py diff --git a/hindsight-api/hindsight_api/migrations.py b/hindsight-api-slim/hindsight_api/migrations.py similarity index 100% rename from hindsight-api/hindsight_api/migrations.py rename to hindsight-api-slim/hindsight_api/migrations.py diff --git a/hindsight-api/hindsight_api/models.py b/hindsight-api-slim/hindsight_api/models.py similarity index 100% rename from hindsight-api/hindsight_api/models.py rename to hindsight-api-slim/hindsight_api/models.py diff --git a/hindsight-api/hindsight_api/pg0.py b/hindsight-api-slim/hindsight_api/pg0.py similarity index 93% rename from hindsight-api/hindsight_api/pg0.py rename to hindsight-api-slim/hindsight_api/pg0.py index 1be997c1..3cc3d3f8 100644 --- a/hindsight-api/hindsight_api/pg0.py +++ b/hindsight-api-slim/hindsight_api/pg0.py @@ -1,7 +1,11 @@ +from __future__ import annotations + import asyncio import logging +from typing import TYPE_CHECKING -from pg0 import Pg0 +if TYPE_CHECKING: + from pg0 import Pg0 logger = logging.getLogger(__name__) @@ -31,6 +35,13 @@ class EmbeddedPostgres: def _get_pg0(self) -> Pg0: if self._pg0 is None: + try: + from pg0 import Pg0 + except ImportError: + raise ImportError( + "pg0-embedded is required for embedded PostgreSQL. " + "Install it with: pip install 'hindsight-api-slim[embedded-db]'" + ) kwargs = { "name": self.name, "username": self.username, diff --git a/hindsight-api/hindsight_api/server.py b/hindsight-api-slim/hindsight_api/server.py similarity index 100% rename from hindsight-api/hindsight_api/server.py rename to hindsight-api-slim/hindsight_api/server.py diff --git a/hindsight-api/hindsight_api/tracing.py b/hindsight-api-slim/hindsight_api/tracing.py similarity index 100% rename from hindsight-api/hindsight_api/tracing.py rename to hindsight-api-slim/hindsight_api/tracing.py diff --git a/hindsight-api/hindsight_api/utils.py b/hindsight-api-slim/hindsight_api/utils.py similarity index 100% rename from hindsight-api/hindsight_api/utils.py rename to hindsight-api-slim/hindsight_api/utils.py diff --git a/hindsight-api/hindsight_api/webhooks/__init__.py b/hindsight-api-slim/hindsight_api/webhooks/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/webhooks/__init__.py rename to hindsight-api-slim/hindsight_api/webhooks/__init__.py diff --git a/hindsight-api/hindsight_api/webhooks/manager.py b/hindsight-api-slim/hindsight_api/webhooks/manager.py similarity index 100% rename from hindsight-api/hindsight_api/webhooks/manager.py rename to hindsight-api-slim/hindsight_api/webhooks/manager.py diff --git a/hindsight-api/hindsight_api/webhooks/models.py b/hindsight-api-slim/hindsight_api/webhooks/models.py similarity index 100% rename from hindsight-api/hindsight_api/webhooks/models.py rename to hindsight-api-slim/hindsight_api/webhooks/models.py diff --git a/hindsight-api/hindsight_api/worker/__init__.py b/hindsight-api-slim/hindsight_api/worker/__init__.py similarity index 100% rename from hindsight-api/hindsight_api/worker/__init__.py rename to hindsight-api-slim/hindsight_api/worker/__init__.py diff --git a/hindsight-api/hindsight_api/worker/exceptions.py b/hindsight-api-slim/hindsight_api/worker/exceptions.py similarity index 100% rename from hindsight-api/hindsight_api/worker/exceptions.py rename to hindsight-api-slim/hindsight_api/worker/exceptions.py diff --git a/hindsight-api/hindsight_api/worker/main.py b/hindsight-api-slim/hindsight_api/worker/main.py similarity index 100% rename from hindsight-api/hindsight_api/worker/main.py rename to hindsight-api-slim/hindsight_api/worker/main.py diff --git a/hindsight-api/hindsight_api/worker/poller.py b/hindsight-api-slim/hindsight_api/worker/poller.py similarity index 100% rename from hindsight-api/hindsight_api/worker/poller.py rename to hindsight-api-slim/hindsight_api/worker/poller.py diff --git a/hindsight-api-slim/pyproject.toml b/hindsight-api-slim/pyproject.toml new file mode 100644 index 00000000..aa651ed3 --- /dev/null +++ b/hindsight-api-slim/pyproject.toml @@ -0,0 +1,198 @@ +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "hindsight-api-slim" +version = "0.4.17" +description = "Hindsight: Agent Memory That Works Like Human Memory" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + "asyncpg>=0.29.0", + "python-dotenv>=1.0.0", + "openai>=1.0.0", + "pydantic>=2.0.0", + "rich>=13.0.0", + "langchain-text-splitters>=0.3.0", + "fastapi[standard]>=0.120.3", + "uvicorn>=0.38.0", + "wsproto>=1.0.0", + "sqlalchemy>=2.0.44", + "alembic>=1.17.1", + "pgvector>=0.4.1", + "greenlet>=3.2.4", + "psycopg2-binary>=2.9.11", + "tiktoken>=0.12.0", + "httpx>=0.27.0", + "PyJWT[crypto]>=2.8.0", + "fastmcp>=2.14.0", # CVE-2025-66416 + "python-dateutil>=2.8.0", + "opentelemetry-api>=1.20.0", + "opentelemetry-sdk>=1.20.0", + "opentelemetry-instrumentation-fastapi>=0.41b0", + "opentelemetry-exporter-prometheus>=0.41b0", + "opentelemetry-exporter-otlp-proto-http>=1.20.0", + "opentelemetry-semantic-conventions>=0.41b0", + "dateparser>=1.2.2", + "google-genai>=1.0.0", + "google-auth>=2.0.0", + "anthropic>=0.40.0", + "typer>=0.9.0", + "cohere>=5.0.0", + "litellm>=1.0.0", + "markitdown[pdf,docx,pptx,xlsx,xls]>=0.1.4", # File to markdown conversion + "obstore>=0.4.0", # S3/GCS/Azure object storage client (Rust-backed) + "uvloop>=0.22.1", + # Transitive dependency security fixes + "pyasn1>=0.6.2", # DoS vulnerability fix + "urllib3>=2.6.3", # Decompression-bomb safeguards bypass fix + "langchain-core>=1.2.11", # Serialization injection + SSRF vulnerability fix + "langsmith>=0.6.3", # SSRF via tracing header injection fix + "protobuf>=6.33.5", # JSON recursion depth bypass fix + "pillow>=12.1.1", # Out-of-bounds write in PSD image loading fix + "cryptography>=46.0.5", # Subgroup attack vulnerability fix + "filelock>=3.20.1", # TOCTOU race condition fix + "authlib>=1.6.6", # Account takeover vulnerability fix + "aiohttp>=3.13.3", # Multiple DoS vulnerabilities + "claude-agent-sdk>=0.1.27", +] + +[project.optional-dependencies] +local-ml = [ + # Local ML models for embeddings/reranking + "sentence-transformers>=3.3.0", + "transformers>=4.53.0", # Security fixes for ReDoS vulnerabilities + "torch>=2.6.0", # CVE fix for remote code execution + "einops>=0.8.2", + "flashrank>=0.2.0", + # Apple Silicon local inference + "mlx>=0.31.0", + "mlx-lm>=0.31.1", + "safetensors>=0.6.2", +] +embedded-db = [ + "pg0-embedded>=0.11.0", +] +all = [ + "hindsight-api-slim[local-ml,embedded-db]", +] +test = [ + "pytest>=7.0.0", + "pytest-asyncio>=0.21.0", + "pytest-timeout>=2.4.0", + "pytest-xdist>=3.0.0", + "filelock>=3.20.1", # TOCTOU race condition fix + "testcontainers>=4.0.0", +] + +[project.scripts] +hindsight-api = "hindsight_api.main:main" +hindsight-worker = "hindsight_api.worker.main:main" +hindsight-local-mcp = "hindsight_api.mcp_local:main" +hindsight-admin = "hindsight_api.admin.cli:main" + +[tool.hatch.build.targets.wheel] +packages = ["hindsight_api"] + +[tool.hatch.build.targets.wheel.sources] +"hindsight_api" = "hindsight_api" + +[tool.hatch.build.targets.sdist] +include = [ + "hindsight_api/**/*", +] + +[tool.hatch.build] +include = [ + "hindsight_api/**/*.py", + "hindsight_api/alembic/**/*", +] + +[tool.pytest.ini_options] +log_cli = true +log_cli_level = "INFO" +log_cli_format = "%(asctime)s - %(levelname)s - %(name)s - %(message)s" +log_cli_date_format = "%Y-%m-%d %H:%M:%S" +addopts = "--timeout 300 -n 8 --dist loadgroup --durations=10 -v" +asyncio_mode = "auto" +asyncio_default_fixture_loop_scope = "function" +log_auto_indent = true +filterwarnings = [ + "ignore:The @wait_container_is_ready decorator is deprecated:DeprecationWarning", + "ignore::RuntimeWarning:asyncio", +] + +[dependency-groups] +dev = [ + "pytest>=9.0.0", + "pytest-asyncio>=1.3.0", + "pytest-timeout>=2.4.0", + "pytest-xdist>=3.8.0", + "python-dotenv>=1.2.1", + "filelock>=3.20.1", # TOCTOU race condition fix + "ruff>=0.8.0", + "ty>=0.0.1", + "testcontainers>=4.0.0", +] + +[tool.ruff] +line-length = 120 +target-version = "py311" +exclude = [ + "tests/", + "**/tests/", +] + +[tool.ruff.lint] +select = [ + "E", # pycodestyle errors + "W", # pycodestyle warnings + "F", # Pyflakes + "I", # isort +] +ignore = [ + "E501", # line too long (handled by formatter) + "E402", # module import not at top of file + "F401", # unused import (too noisy during development) + "F841", # unused variable (too noisy during development) + "F811", # redefined while unused + "F821", # undefined name (forward references in type hints) +] + +[tool.ruff.lint.isort] +known-third-party = ["alembic"] + +[tool.ruff.format] +quote-style = "double" +indent-style = "space" + +[tool.uv] +# Allow uv to search all configured indexes for packages, not just the first one +# This prevents dependency resolution failures when using pytorch index + PyPI +index-strategy = "unsafe-best-match" + +[tool.ty] +# Type checking configuration +# ty is an extremely fast Python type checker from Astral (same team as ruff/uv) + +[tool.ty.environment] +python-version = "3.11" + +[tool.ty.src] +exclude = [ + "tests/", + "hindsight_api/alembic/", +] + +[tool.ty.rules] +# Disable noisy rules while keeping important ones +invalid-argument-type = "ignore" # False positives with **kwargs patterns +invalid-return-type = "ignore" # Often intentional in async code +invalid-parameter-default = "ignore" # Optional params with None default +possibly-missing-attribute = "ignore" # Common with Optional types +invalid-raise = "ignore" # False positives with exception tracking +call-non-callable = "ignore" # False positives with Optional types +invalid-key = "ignore" # Pydantic ConfigDict not understood +invalid-method-override = "ignore" # Intentional signature differences +unresolved-reference = "ignore" # Forward references not always resolved diff --git a/hindsight-api/tests/__init__.py b/hindsight-api-slim/tests/__init__.py similarity index 100% rename from hindsight-api/tests/__init__.py rename to hindsight-api-slim/tests/__init__.py diff --git a/hindsight-api/tests/conftest.py b/hindsight-api-slim/tests/conftest.py similarity index 100% rename from hindsight-api/tests/conftest.py rename to hindsight-api-slim/tests/conftest.py diff --git a/hindsight-api/tests/fixtures/README.md b/hindsight-api-slim/tests/fixtures/README.md similarity index 100% rename from hindsight-api/tests/fixtures/README.md rename to hindsight-api-slim/tests/fixtures/README.md diff --git a/hindsight-api/tests/fixtures/locomo_conversation_sample.json b/hindsight-api-slim/tests/fixtures/locomo_conversation_sample.json similarity index 100% rename from hindsight-api/tests/fixtures/locomo_conversation_sample.json rename to hindsight-api-slim/tests/fixtures/locomo_conversation_sample.json diff --git a/hindsight-api/tests/test_admin_backup_restore.py b/hindsight-api-slim/tests/test_admin_backup_restore.py similarity index 100% rename from hindsight-api/tests/test_admin_backup_restore.py rename to hindsight-api-slim/tests/test_admin_backup_restore.py diff --git a/hindsight-api/tests/test_agents_api.py b/hindsight-api-slim/tests/test_agents_api.py similarity index 100% rename from hindsight-api/tests/test_agents_api.py rename to hindsight-api-slim/tests/test_agents_api.py diff --git a/hindsight-api/tests/test_async_batch_retain.py b/hindsight-api-slim/tests/test_async_batch_retain.py similarity index 100% rename from hindsight-api/tests/test_async_batch_retain.py rename to hindsight-api-slim/tests/test_async_batch_retain.py diff --git a/hindsight-api/tests/test_async_retain_tags.py b/hindsight-api-slim/tests/test_async_retain_tags.py similarity index 100% rename from hindsight-api/tests/test_async_retain_tags.py rename to hindsight-api-slim/tests/test_async_retain_tags.py diff --git a/hindsight-api/tests/test_base_path.py b/hindsight-api-slim/tests/test_base_path.py similarity index 100% rename from hindsight-api/tests/test_base_path.py rename to hindsight-api-slim/tests/test_base_path.py diff --git a/hindsight-api/tests/test_batch_api.py b/hindsight-api-slim/tests/test_batch_api.py similarity index 100% rename from hindsight-api/tests/test_batch_api.py rename to hindsight-api-slim/tests/test_batch_api.py diff --git a/hindsight-api/tests/test_batch_api_integration.py b/hindsight-api-slim/tests/test_batch_api_integration.py similarity index 100% rename from hindsight-api/tests/test_batch_api_integration.py rename to hindsight-api-slim/tests/test_batch_api_integration.py diff --git a/hindsight-api/tests/test_batch_api_validation.py b/hindsight-api-slim/tests/test_batch_api_validation.py similarity index 100% rename from hindsight-api/tests/test_batch_api_validation.py rename to hindsight-api-slim/tests/test_batch_api_validation.py diff --git a/hindsight-api/tests/test_batch_chunking.py b/hindsight-api-slim/tests/test_batch_chunking.py similarity index 100% rename from hindsight-api/tests/test_batch_chunking.py rename to hindsight-api-slim/tests/test_batch_chunking.py diff --git a/hindsight-api/tests/test_causal_relations.py b/hindsight-api-slim/tests/test_causal_relations.py similarity index 100% rename from hindsight-api/tests/test_causal_relations.py rename to hindsight-api-slim/tests/test_causal_relations.py diff --git a/hindsight-api/tests/test_causal_relationships.py b/hindsight-api-slim/tests/test_causal_relationships.py similarity index 100% rename from hindsight-api/tests/test_causal_relationships.py rename to hindsight-api-slim/tests/test_causal_relationships.py diff --git a/hindsight-api/tests/test_chunking.py b/hindsight-api-slim/tests/test_chunking.py similarity index 100% rename from hindsight-api/tests/test_chunking.py rename to hindsight-api-slim/tests/test_chunking.py diff --git a/hindsight-api/tests/test_combined_scoring.py b/hindsight-api-slim/tests/test_combined_scoring.py similarity index 100% rename from hindsight-api/tests/test_combined_scoring.py rename to hindsight-api-slim/tests/test_combined_scoring.py diff --git a/hindsight-api/tests/test_config_validation.py b/hindsight-api-slim/tests/test_config_validation.py similarity index 100% rename from hindsight-api/tests/test_config_validation.py rename to hindsight-api-slim/tests/test_config_validation.py diff --git a/hindsight-api/tests/test_consolidation.py b/hindsight-api-slim/tests/test_consolidation.py similarity index 100% rename from hindsight-api/tests/test_consolidation.py rename to hindsight-api-slim/tests/test_consolidation.py diff --git a/hindsight-api/tests/test_custom_embedding_dimension.py b/hindsight-api-slim/tests/test_custom_embedding_dimension.py similarity index 100% rename from hindsight-api/tests/test_custom_embedding_dimension.py rename to hindsight-api-slim/tests/test_custom_embedding_dimension.py diff --git a/hindsight-api/tests/test_document_tracking.py b/hindsight-api-slim/tests/test_document_tracking.py similarity index 100% rename from hindsight-api/tests/test_document_tracking.py rename to hindsight-api-slim/tests/test_document_tracking.py diff --git a/hindsight-api/tests/test_entity_labels.py b/hindsight-api-slim/tests/test_entity_labels.py similarity index 100% rename from hindsight-api/tests/test_entity_labels.py rename to hindsight-api-slim/tests/test_entity_labels.py diff --git a/hindsight-api/tests/test_extensions.py b/hindsight-api-slim/tests/test_extensions.py similarity index 100% rename from hindsight-api/tests/test_extensions.py rename to hindsight-api-slim/tests/test_extensions.py diff --git a/hindsight-api/tests/test_fact_extraction_analysis.py b/hindsight-api-slim/tests/test_fact_extraction_analysis.py similarity index 100% rename from hindsight-api/tests/test_fact_extraction_analysis.py rename to hindsight-api-slim/tests/test_fact_extraction_analysis.py diff --git a/hindsight-api/tests/test_fact_extraction_metadata.py b/hindsight-api-slim/tests/test_fact_extraction_metadata.py similarity index 100% rename from hindsight-api/tests/test_fact_extraction_metadata.py rename to hindsight-api-slim/tests/test_fact_extraction_metadata.py diff --git a/hindsight-api/tests/test_fact_extraction_output_ratio.py b/hindsight-api-slim/tests/test_fact_extraction_output_ratio.py similarity index 100% rename from hindsight-api/tests/test_fact_extraction_output_ratio.py rename to hindsight-api-slim/tests/test_fact_extraction_output_ratio.py diff --git a/hindsight-api/tests/test_fact_extraction_quality.py b/hindsight-api-slim/tests/test_fact_extraction_quality.py similarity index 100% rename from hindsight-api/tests/test_fact_extraction_quality.py rename to hindsight-api-slim/tests/test_fact_extraction_quality.py diff --git a/hindsight-api/tests/test_fact_extraction_retry.py b/hindsight-api-slim/tests/test_fact_extraction_retry.py similarity index 100% rename from hindsight-api/tests/test_fact_extraction_retry.py rename to hindsight-api-slim/tests/test_fact_extraction_retry.py diff --git a/hindsight-api/tests/test_fact_ordering.py b/hindsight-api-slim/tests/test_fact_ordering.py similarity index 100% rename from hindsight-api/tests/test_fact_ordering.py rename to hindsight-api-slim/tests/test_fact_ordering.py diff --git a/hindsight-api/tests/test_file_retain.py b/hindsight-api-slim/tests/test_file_retain.py similarity index 100% rename from hindsight-api/tests/test_file_retain.py rename to hindsight-api-slim/tests/test_file_retain.py diff --git a/hindsight-api/tests/test_file_storage_s3.py b/hindsight-api-slim/tests/test_file_storage_s3.py similarity index 100% rename from hindsight-api/tests/test_file_storage_s3.py rename to hindsight-api-slim/tests/test_file_storage_s3.py diff --git a/hindsight-api/tests/test_gemini_safety_settings.py b/hindsight-api-slim/tests/test_gemini_safety_settings.py similarity index 100% rename from hindsight-api/tests/test_gemini_safety_settings.py rename to hindsight-api-slim/tests/test_gemini_safety_settings.py diff --git a/hindsight-api/tests/test_graph_filtering.py b/hindsight-api-slim/tests/test_graph_filtering.py similarity index 100% rename from hindsight-api/tests/test_graph_filtering.py rename to hindsight-api-slim/tests/test_graph_filtering.py diff --git a/hindsight-api/tests/test_hierarchical_config.py b/hindsight-api-slim/tests/test_hierarchical_config.py similarity index 100% rename from hindsight-api/tests/test_hierarchical_config.py rename to hindsight-api-slim/tests/test_hierarchical_config.py diff --git a/hindsight-api/tests/test_hnsw_indexes.py b/hindsight-api-slim/tests/test_hnsw_indexes.py similarity index 100% rename from hindsight-api/tests/test_hnsw_indexes.py rename to hindsight-api-slim/tests/test_hnsw_indexes.py diff --git a/hindsight-api/tests/test_http_api_integration.py b/hindsight-api-slim/tests/test_http_api_integration.py similarity index 100% rename from hindsight-api/tests/test_http_api_integration.py rename to hindsight-api-slim/tests/test_http_api_integration.py diff --git a/hindsight-api/tests/test_iris_parser.py b/hindsight-api-slim/tests/test_iris_parser.py similarity index 100% rename from hindsight-api/tests/test_iris_parser.py rename to hindsight-api-slim/tests/test_iris_parser.py diff --git a/hindsight-api/tests/test_link_expansion_retrieval.py b/hindsight-api-slim/tests/test_link_expansion_retrieval.py similarity index 100% rename from hindsight-api/tests/test_link_expansion_retrieval.py rename to hindsight-api-slim/tests/test_link_expansion_retrieval.py diff --git a/hindsight-api/tests/test_link_utils.py b/hindsight-api-slim/tests/test_link_utils.py similarity index 100% rename from hindsight-api/tests/test_link_utils.py rename to hindsight-api-slim/tests/test_link_utils.py diff --git a/hindsight-api/tests/test_list_documents.py b/hindsight-api-slim/tests/test_list_documents.py similarity index 100% rename from hindsight-api/tests/test_list_documents.py rename to hindsight-api-slim/tests/test_list_documents.py diff --git a/hindsight-api/tests/test_litellm_sdk_cross_encoder.py b/hindsight-api-slim/tests/test_litellm_sdk_cross_encoder.py similarity index 100% rename from hindsight-api/tests/test_litellm_sdk_cross_encoder.py rename to hindsight-api-slim/tests/test_litellm_sdk_cross_encoder.py diff --git a/hindsight-api/tests/test_litellm_sdk_embeddings.py b/hindsight-api-slim/tests/test_litellm_sdk_embeddings.py similarity index 100% rename from hindsight-api/tests/test_litellm_sdk_embeddings.py rename to hindsight-api-slim/tests/test_litellm_sdk_embeddings.py diff --git a/hindsight-api/tests/test_llm_provider.py b/hindsight-api-slim/tests/test_llm_provider.py similarity index 100% rename from hindsight-api/tests/test_llm_provider.py rename to hindsight-api-slim/tests/test_llm_provider.py diff --git a/hindsight-api/tests/test_llm_token_metrics.py b/hindsight-api-slim/tests/test_llm_token_metrics.py similarity index 100% rename from hindsight-api/tests/test_llm_token_metrics.py rename to hindsight-api-slim/tests/test_llm_token_metrics.py diff --git a/hindsight-api/tests/test_llm_tools.py b/hindsight-api-slim/tests/test_llm_tools.py similarity index 100% rename from hindsight-api/tests/test_llm_tools.py rename to hindsight-api-slim/tests/test_llm_tools.py diff --git a/hindsight-api/tests/test_llm_wrapper.py b/hindsight-api-slim/tests/test_llm_wrapper.py similarity index 100% rename from hindsight-api/tests/test_llm_wrapper.py rename to hindsight-api-slim/tests/test_llm_wrapper.py diff --git a/hindsight-api/tests/test_lmstudio_tool_choice.py b/hindsight-api-slim/tests/test_lmstudio_tool_choice.py similarity index 100% rename from hindsight-api/tests/test_lmstudio_tool_choice.py rename to hindsight-api-slim/tests/test_lmstudio_tool_choice.py diff --git a/hindsight-api/tests/test_load_large_batch.py b/hindsight-api-slim/tests/test_load_large_batch.py similarity index 100% rename from hindsight-api/tests/test_load_large_batch.py rename to hindsight-api-slim/tests/test_load_large_batch.py diff --git a/hindsight-api/tests/test_main_module.py b/hindsight-api-slim/tests/test_main_module.py similarity index 100% rename from hindsight-api/tests/test_main_module.py rename to hindsight-api-slim/tests/test_main_module.py diff --git a/hindsight-api/tests/test_mcp_endpoint_routing.py b/hindsight-api-slim/tests/test_mcp_endpoint_routing.py similarity index 100% rename from hindsight-api/tests/test_mcp_endpoint_routing.py rename to hindsight-api-slim/tests/test_mcp_endpoint_routing.py diff --git a/hindsight-api/tests/test_mcp_extension.py b/hindsight-api-slim/tests/test_mcp_extension.py similarity index 100% rename from hindsight-api/tests/test_mcp_extension.py rename to hindsight-api-slim/tests/test_mcp_extension.py diff --git a/hindsight-api/tests/test_mcp_routing.py b/hindsight-api-slim/tests/test_mcp_routing.py similarity index 100% rename from hindsight-api/tests/test_mcp_routing.py rename to hindsight-api-slim/tests/test_mcp_routing.py diff --git a/hindsight-api/tests/test_mcp_tools.py b/hindsight-api-slim/tests/test_mcp_tools.py similarity index 100% rename from hindsight-api/tests/test_mcp_tools.py rename to hindsight-api-slim/tests/test_mcp_tools.py diff --git a/hindsight-api/tests/test_mental_model_hooks.py b/hindsight-api-slim/tests/test_mental_model_hooks.py similarity index 100% rename from hindsight-api/tests/test_mental_model_hooks.py rename to hindsight-api-slim/tests/test_mental_model_hooks.py diff --git a/hindsight-api/tests/test_mental_models.py b/hindsight-api-slim/tests/test_mental_models.py similarity index 100% rename from hindsight-api/tests/test_mental_models.py rename to hindsight-api-slim/tests/test_mental_models.py diff --git a/hindsight-api/tests/test_metrics.py b/hindsight-api-slim/tests/test_metrics.py similarity index 100% rename from hindsight-api/tests/test_metrics.py rename to hindsight-api-slim/tests/test_metrics.py diff --git a/hindsight-api/tests/test_migrations_thread_safety.py b/hindsight-api-slim/tests/test_migrations_thread_safety.py similarity index 100% rename from hindsight-api/tests/test_migrations_thread_safety.py rename to hindsight-api-slim/tests/test_migrations_thread_safety.py diff --git a/hindsight-api/tests/test_minimax_provider.py b/hindsight-api-slim/tests/test_minimax_provider.py similarity index 100% rename from hindsight-api/tests/test_minimax_provider.py rename to hindsight-api-slim/tests/test_minimax_provider.py diff --git a/hindsight-api/tests/test_mpfp_retrieval.py b/hindsight-api-slim/tests/test_mpfp_retrieval.py similarity index 100% rename from hindsight-api/tests/test_mpfp_retrieval.py rename to hindsight-api-slim/tests/test_mpfp_retrieval.py diff --git a/hindsight-api/tests/test_multilingual.py b/hindsight-api-slim/tests/test_multilingual.py similarity index 100% rename from hindsight-api/tests/test_multilingual.py rename to hindsight-api-slim/tests/test_multilingual.py diff --git a/hindsight-api/tests/test_observation_invalidation.py b/hindsight-api-slim/tests/test_observation_invalidation.py similarity index 100% rename from hindsight-api/tests/test_observation_invalidation.py rename to hindsight-api-slim/tests/test_observation_invalidation.py diff --git a/hindsight-api/tests/test_observations.py b/hindsight-api-slim/tests/test_observations.py similarity index 100% rename from hindsight-api/tests/test_observations.py rename to hindsight-api-slim/tests/test_observations.py diff --git a/hindsight-api/tests/test_op_cancellation.py b/hindsight-api-slim/tests/test_op_cancellation.py similarity index 100% rename from hindsight-api/tests/test_op_cancellation.py rename to hindsight-api-slim/tests/test_op_cancellation.py diff --git a/hindsight-api/tests/test_per_operation_llm_config.py b/hindsight-api-slim/tests/test_per_operation_llm_config.py similarity index 100% rename from hindsight-api/tests/test_per_operation_llm_config.py rename to hindsight-api-slim/tests/test_per_operation_llm_config.py diff --git a/hindsight-api/tests/test_provider_default_models.py b/hindsight-api-slim/tests/test_provider_default_models.py similarity index 100% rename from hindsight-api/tests/test_provider_default_models.py rename to hindsight-api-slim/tests/test_provider_default_models.py diff --git a/hindsight-api/tests/test_query_analyzer.py b/hindsight-api-slim/tests/test_query_analyzer.py similarity index 100% rename from hindsight-api/tests/test_query_analyzer.py rename to hindsight-api-slim/tests/test_query_analyzer.py diff --git a/hindsight-api/tests/test_recall_chunks_independence.py b/hindsight-api-slim/tests/test_recall_chunks_independence.py similarity index 100% rename from hindsight-api/tests/test_recall_chunks_independence.py rename to hindsight-api-slim/tests/test_recall_chunks_independence.py diff --git a/hindsight-api/tests/test_reflect_agent.py b/hindsight-api-slim/tests/test_reflect_agent.py similarity index 100% rename from hindsight-api/tests/test_reflect_agent.py rename to hindsight-api-slim/tests/test_reflect_agent.py diff --git a/hindsight-api/tests/test_reflect_empty_based_on.py b/hindsight-api-slim/tests/test_reflect_empty_based_on.py similarity index 100% rename from hindsight-api/tests/test_reflect_empty_based_on.py rename to hindsight-api-slim/tests/test_reflect_empty_based_on.py diff --git a/hindsight-api/tests/test_reflect_tracing.py b/hindsight-api-slim/tests/test_reflect_tracing.py similarity index 100% rename from hindsight-api/tests/test_reflect_tracing.py rename to hindsight-api-slim/tests/test_reflect_tracing.py diff --git a/hindsight-api/tests/test_reflections.py b/hindsight-api-slim/tests/test_reflections.py similarity index 100% rename from hindsight-api/tests/test_reflections.py rename to hindsight-api-slim/tests/test_reflections.py diff --git a/hindsight-api/tests/test_reranker_error_handling.py b/hindsight-api-slim/tests/test_reranker_error_handling.py similarity index 100% rename from hindsight-api/tests/test_reranker_error_handling.py rename to hindsight-api-slim/tests/test_reranker_error_handling.py diff --git a/hindsight-api/tests/test_retain.py b/hindsight-api-slim/tests/test_retain.py similarity index 100% rename from hindsight-api/tests/test_retain.py rename to hindsight-api-slim/tests/test_retain.py diff --git a/hindsight-api/tests/test_schema_isolation.py b/hindsight-api-slim/tests/test_schema_isolation.py similarity index 100% rename from hindsight-api/tests/test_schema_isolation.py rename to hindsight-api-slim/tests/test_schema_isolation.py diff --git a/hindsight-api/tests/test_search_trace.py b/hindsight-api-slim/tests/test_search_trace.py similarity index 100% rename from hindsight-api/tests/test_search_trace.py rename to hindsight-api-slim/tests/test_search_trace.py diff --git a/hindsight-api/tests/test_server_module.py b/hindsight-api-slim/tests/test_server_module.py similarity index 100% rename from hindsight-api/tests/test_server_module.py rename to hindsight-api-slim/tests/test_server_module.py diff --git a/hindsight-api/tests/test_source_facts_tokens.py b/hindsight-api-slim/tests/test_source_facts_tokens.py similarity index 100% rename from hindsight-api/tests/test_source_facts_tokens.py rename to hindsight-api-slim/tests/test_source_facts_tokens.py diff --git a/hindsight-api/tests/test_sql_schema_safety.py b/hindsight-api-slim/tests/test_sql_schema_safety.py similarity index 100% rename from hindsight-api/tests/test_sql_schema_safety.py rename to hindsight-api-slim/tests/test_sql_schema_safety.py diff --git a/hindsight-api/tests/test_supabase_tenant.py b/hindsight-api-slim/tests/test_supabase_tenant.py similarity index 100% rename from hindsight-api/tests/test_supabase_tenant.py rename to hindsight-api-slim/tests/test_supabase_tenant.py diff --git a/hindsight-api/tests/test_tags_visibility.py b/hindsight-api-slim/tests/test_tags_visibility.py similarity index 100% rename from hindsight-api/tests/test_tags_visibility.py rename to hindsight-api-slim/tests/test_tags_visibility.py diff --git a/hindsight-api/tests/test_tei_cross_encoder.py b/hindsight-api-slim/tests/test_tei_cross_encoder.py similarity index 100% rename from hindsight-api/tests/test_tei_cross_encoder.py rename to hindsight-api-slim/tests/test_tei_cross_encoder.py diff --git a/hindsight-api/tests/test_temporal_ranges.py b/hindsight-api-slim/tests/test_temporal_ranges.py similarity index 100% rename from hindsight-api/tests/test_temporal_ranges.py rename to hindsight-api-slim/tests/test_temporal_ranges.py diff --git a/hindsight-api/tests/test_think.py b/hindsight-api-slim/tests/test_think.py similarity index 100% rename from hindsight-api/tests/test_think.py rename to hindsight-api-slim/tests/test_think.py diff --git a/hindsight-api/tests/test_tracing.py b/hindsight-api-slim/tests/test_tracing.py similarity index 100% rename from hindsight-api/tests/test_tracing.py rename to hindsight-api-slim/tests/test_tracing.py diff --git a/hindsight-api/tests/test_tracing_integration.py b/hindsight-api-slim/tests/test_tracing_integration.py similarity index 100% rename from hindsight-api/tests/test_tracing_integration.py rename to hindsight-api-slim/tests/test_tracing_integration.py diff --git a/hindsight-api/tests/test_tracing_spans_verification.py b/hindsight-api-slim/tests/test_tracing_spans_verification.py similarity index 100% rename from hindsight-api/tests/test_tracing_spans_verification.py rename to hindsight-api-slim/tests/test_tracing_spans_verification.py diff --git a/hindsight-api/tests/test_vertexai_provider.py b/hindsight-api-slim/tests/test_vertexai_provider.py similarity index 100% rename from hindsight-api/tests/test_vertexai_provider.py rename to hindsight-api-slim/tests/test_vertexai_provider.py diff --git a/hindsight-api/tests/test_webhooks.py b/hindsight-api-slim/tests/test_webhooks.py similarity index 100% rename from hindsight-api/tests/test_webhooks.py rename to hindsight-api-slim/tests/test_webhooks.py diff --git a/hindsight-api/tests/test_worker.py b/hindsight-api-slim/tests/test_worker.py similarity index 100% rename from hindsight-api/tests/test_worker.py rename to hindsight-api-slim/tests/test_worker.py diff --git a/hindsight-api/pyproject.toml b/hindsight-api/pyproject.toml index 51ef1dfe..ecb5946a 100644 --- a/hindsight-api/pyproject.toml +++ b/hindsight-api/pyproject.toml @@ -1,6 +1,6 @@ [build-system] -requires = ["hatchling"] -build-backend = "hatchling.build" +requires = ["setuptools>=61"] +build-backend = "setuptools.build_meta" [project] name = "hindsight-api" @@ -9,182 +9,11 @@ description = "Hindsight: Agent Memory That Works Like Human Memory" readme = "README.md" requires-python = ">=3.11" dependencies = [ - "asyncpg>=0.29.0", - "python-dotenv>=1.0.0", - "openai>=1.0.0", - "pydantic>=2.0.0", - "rich>=13.0.0", - "langchain-text-splitters>=0.3.0", - "fastapi[standard]>=0.120.3", - "uvicorn>=0.38.0", - "wsproto>=1.0.0", - "sqlalchemy>=2.0.44", - "alembic>=1.17.1", - "pgvector>=0.4.1", - "greenlet>=3.2.4", - "psycopg2-binary>=2.9.11", - "tiktoken>=0.12.0", - "httpx>=0.27.0", - "PyJWT[crypto]>=2.8.0", - "fastmcp>=2.14.0", # CVE-2025-66416 - "pg0-embedded>=0.11.0", - "python-dateutil>=2.8.0", - "opentelemetry-api>=1.20.0", - "opentelemetry-sdk>=1.20.0", - "opentelemetry-instrumentation-fastapi>=0.41b0", - "opentelemetry-exporter-prometheus>=0.41b0", - "opentelemetry-exporter-otlp-proto-http>=1.20.0", - "opentelemetry-semantic-conventions>=0.41b0", - "dateparser>=1.2.2", - "google-genai>=1.0.0", - "google-auth>=2.0.0", - "anthropic>=0.40.0", - "typer>=0.9.0", - "cohere>=5.0.0", - "flashrank>=0.2.0", - "litellm>=1.0.0", - "markitdown[pdf,docx,pptx,xlsx,xls]>=0.1.4", # File to markdown conversion - "obstore>=0.4.0", # S3/GCS/Azure object storage client (Rust-backed) - # Local ML models for embeddings/reranking - can be excluded in Docker with INCLUDE_LOCAL_MODELS=false - "sentence-transformers>=3.3.0", - "transformers>=4.53.0", # Security fixes for ReDoS vulnerabilities - "torch>=2.6.0", # CVE fix for remote code execution - "uvloop>=0.22.1", - # Transitive dependency security fixes - "pyasn1>=0.6.2", # DoS vulnerability fix - "urllib3>=2.6.3", # Decompression-bomb safeguards bypass fix - "langchain-core>=1.2.11", # Serialization injection + SSRF vulnerability fix - "langsmith>=0.6.3", # SSRF via tracing header injection fix - "protobuf>=6.33.5", # JSON recursion depth bypass fix - "pillow>=12.1.1", # Out-of-bounds write in PSD image loading fix - "cryptography>=46.0.5", # Subgroup attack vulnerability fix - "filelock>=3.20.1", # TOCTOU race condition fix - "authlib>=1.6.6", # Account takeover vulnerability fix - "aiohttp>=3.13.3", # Multiple DoS vulnerabilities - "claude-agent-sdk>=0.1.27", - "einops>=0.8.2", - "mlx>=0.31.0", - "mlx-lm>=0.31.1", - "safetensors>=0.6.2", + "hindsight-api-slim[all]>=0.4.17", ] -[project.optional-dependencies] -test = [ - "pytest>=7.0.0", - "pytest-asyncio>=0.21.0", - "pytest-timeout>=2.4.0", - "pytest-xdist>=3.0.0", - "filelock>=3.20.1", # TOCTOU race condition fix - "testcontainers>=4.0.0", -] +[tool.uv.sources] +hindsight-api-slim = { workspace = true } -[project.scripts] -hindsight-api = "hindsight_api.main:main" -hindsight-worker = "hindsight_api.worker.main:main" -hindsight-local-mcp = "hindsight_api.mcp_local:main" -hindsight-admin = "hindsight_api.admin.cli:main" - -[tool.hatch.build.targets.wheel] -packages = ["hindsight_api"] - -[tool.hatch.build.targets.wheel.sources] -"hindsight_api" = "hindsight_api" - -[tool.hatch.build.targets.sdist] -include = [ - "hindsight_api/**/*", -] - -[tool.hatch.build] -include = [ - "hindsight_api/**/*.py", - "hindsight_api/alembic/**/*", -] - -[tool.pytest.ini_options] -log_cli = true -log_cli_level = "INFO" -log_cli_format = "%(asctime)s - %(levelname)s - %(name)s - %(message)s" -log_cli_date_format = "%Y-%m-%d %H:%M:%S" -addopts = "--timeout 300 -n 8 --dist loadgroup --durations=10 -v" -asyncio_mode = "auto" -asyncio_default_fixture_loop_scope = "function" -log_auto_indent = true -filterwarnings = [ - "ignore:The @wait_container_is_ready decorator is deprecated:DeprecationWarning", - "ignore::RuntimeWarning:asyncio", -] - -[dependency-groups] -dev = [ - "pytest>=9.0.0", - "pytest-asyncio>=1.3.0", - "pytest-timeout>=2.4.0", - "pytest-xdist>=3.8.0", - "python-dotenv>=1.2.1", - "filelock>=3.20.1", # TOCTOU race condition fix - "ruff>=0.8.0", - "ty>=0.0.1", - "testcontainers>=4.0.0", -] - -[tool.ruff] -line-length = 120 -target-version = "py311" -exclude = [ - "tests/", - "**/tests/", -] - -[tool.ruff.lint] -select = [ - "E", # pycodestyle errors - "W", # pycodestyle warnings - "F", # Pyflakes - "I", # isort -] -ignore = [ - "E501", # line too long (handled by formatter) - "E402", # module import not at top of file - "F401", # unused import (too noisy during development) - "F841", # unused variable (too noisy during development) - "F811", # redefined while unused - "F821", # undefined name (forward references in type hints) -] - -[tool.ruff.lint.isort] -known-third-party = ["alembic"] - -[tool.ruff.format] -quote-style = "double" -indent-style = "space" - -[tool.uv] -# Allow uv to search all configured indexes for packages, not just the first one -# This prevents dependency resolution failures when using pytorch index + PyPI -index-strategy = "unsafe-best-match" - -[tool.ty] -# Type checking configuration -# ty is an extremely fast Python type checker from Astral (same team as ruff/uv) - -[tool.ty.environment] -python-version = "3.11" - -[tool.ty.src] -exclude = [ - "tests/", - "hindsight_api/alembic/", -] - -[tool.ty.rules] -# Disable noisy rules while keeping important ones -invalid-argument-type = "ignore" # False positives with **kwargs patterns -invalid-return-type = "ignore" # Often intentional in async code -invalid-parameter-default = "ignore" # Optional params with None default -possibly-missing-attribute = "ignore" # Common with Optional types -invalid-raise = "ignore" # False positives with exception tracking -call-non-callable = "ignore" # False positives with Optional types -invalid-key = "ignore" # Pydantic ConfigDict not understood -invalid-method-override = "ignore" # Intentional signature differences -unresolved-reference = "ignore" # Forward references not always resolved +[tool.setuptools] +packages = [] diff --git a/hindsight-docs/docs/developer/extensions.md b/hindsight-docs/docs/developer/extensions.md index ba8ef011..0078d729 100644 --- a/hindsight-docs/docs/developer/extensions.md +++ b/hindsight-docs/docs/developer/extensions.md @@ -30,7 +30,7 @@ HINDSIGHT_API_TENANT_SUPABASE_URL=https://your-project.supabase.co HINDSIGHT_API_TENANT_SUPABASE_SERVICE_KEY=your-service-role-key ``` -See the [source code](https://github.com/vectorize-io/hindsight/blob/main/hindsight-api/hindsight_api/extensions/builtin/supabase_tenant.py) for complete configuration options and implementation details. +See the [source code](https://github.com/vectorize-io/hindsight/blob/main/hindsight-api-slim/hindsight_api/extensions/builtin/supabase_tenant.py) for complete configuration options and implementation details. For other multi-tenant setups with separate schemas per tenant (e.g., custom JWT-based auth), implement a custom `TenantExtension`. diff --git a/hindsight-docs/docs/developer/installation.md b/hindsight-docs/docs/developer/installation.md index e30ba8be..d5ce0c3c 100644 --- a/hindsight-docs/docs/developer/installation.md +++ b/hindsight-docs/docs/developer/installation.md @@ -8,28 +8,28 @@ Hindsight can be deployed in several ways depending on your infrastructure and r ## Prerequisites -### PostgreSQL with pgvector +### PostgreSQL -Hindsight requires PostgreSQL with the **pgvector** extension for vector similarity search. +Hindsight requires PostgreSQL 14+ with a vector extension for similarity search. The supported extensions are: + +- **pgvector** (default) +- **pgvectorscale** +- **vchord** + +Configure which one to use with `HINDSIGHT_API_VECTOR_EXTENSION`. See [Configuration](./configuration) for details. **By default**, Hindsight uses **pg0** — an embedded PostgreSQL that runs locally on your machine. This is convenient for development but **not recommended for production**. -**For production**, use an external PostgreSQL with pgvector: +**For production**, use an external PostgreSQL with one of the supported vector extensions: - **Supabase** — Managed PostgreSQL with pgvector built-in - **Neon** — Serverless PostgreSQL with pgvector -- **Azure Database for PostgreSQL** — With pgvector and pg_diskann (DiskANN) support +- **Azure Database for PostgreSQL** — With pgvector and pgvectorscale support - **AWS RDS** / **Cloud SQL** — With pgvector extension enabled -- **Self-hosted** — PostgreSQL 14+ with pgvector installed +- **Self-hosted** — PostgreSQL 14+ with your preferred vector extension ### LLM Provider -You need an LLM API key for fact extraction, entity resolution, and answer generation: - -- **Groq** (recommended): Fast inference with `gpt-oss-20b` -- **OpenAI**: GPT-4o, GPT-4o-mini -- **Ollama**: Run models locally - -See [Models](./models) for detailed comparison and configuration. +You need an LLM API key for fact extraction, entity resolution, and answer generation. See [Models](./models) for supported providers, model recommendations, and configuration. --- @@ -53,61 +53,12 @@ docker run --rm -it --pull always -p 8888:8888 -p 9999:9999 \ ### Docker Image Variants -Hindsight provides two image variants with different size/capability tradeoffs: +| Variant | Size (AMD64) | Size (ARM64) | When to use | +|---------|--------------|--------------|-------------| +| **Full** (`latest`) | ~9 GB | ~3.7 GB | Default. Works out of the box with no external services except the LLM. | +| **Slim** (`slim`) | ~500 MB | ~500 MB | Use when you already rely on external services for embeddings and reranking (OpenAI, Cohere, TEI). Significantly smaller image, faster deploys. Requires [external providers](./configuration#embeddings). | -| Variant | Size (AMD64) | Size (ARM64) | Use Case | -|---------|--------------|--------------|----------| -| **Full** (`latest`) | ~9 GB | ~3.7 GB | Includes local ML models (embeddings, reranking) | -| **Slim** (`slim`) | ~500 MB | ~500 MB | Requires external embedding/reranking providers | - -**Full image** (default): -```bash -docker run --rm -it -p 8888:8888 \ - -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \ - ghcr.io/vectorize-io/hindsight:latest -``` -- ✅ Works out of the box with local ML models -- ✅ No additional services needed -- ❌ Larger image size (AMD64 includes CUDA libraries for GPU support) - -**Slim image**: -```bash -docker run --rm -it -p 8888:8888 \ - -e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \ - -e HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai \ - -e HINDSIGHT_API_RERANKER_PROVIDER=cohere \ - -e HINDSIGHT_API_COHERE_API_KEY=$COHERE_API_KEY \ - ghcr.io/vectorize-io/hindsight:latest-slim -``` -- ✅ Dramatically smaller image (~95% reduction on AMD64) -- ✅ Faster pull/deploy times -- ✅ Lower memory footprint -- ❌ Requires external embedding/reranking services (OpenAI, Cohere, TEI) - -**When to use slim:** -- Cloud deployments where image size matters -- Using managed embedding services (OpenAI, Cohere) -- Running on Text Embeddings Inference (TEI) infrastructure -- Kubernetes environments with fast pull requirements - -:::warning Slim Image Requires External Providers -If you run the slim image **without** setting external embedding providers, you'll see this error: - -``` -ImportError: sentence-transformers is required for LocalSTEmbeddings. -Install it with: pip install sentence-transformers -``` - -**Fix:** Always set embedding and reranking providers when using slim images: -```bash --e HINDSIGHT_API_EMBEDDINGS_PROVIDER=openai --e HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=sk-xxx --e HINDSIGHT_API_RERANKER_PROVIDER=cohere --e HINDSIGHT_API_COHERE_API_KEY=xxx -``` -::: - -See [Configuration](./configuration#embeddings) for all embedding provider options. +The slim image corresponds to the [`hindsight-api-slim`](#package-variants) pip package. See [Configuration](./configuration#embeddings) for external provider options. ### Available Tags @@ -120,7 +71,7 @@ ghcr.io/vectorize-io/hindsight:0.4.9-slim # Slim, specific version # API only ghcr.io/vectorize-io/hindsight-api:latest -ghcr.io/vectorize-io/hindsight-api:slim +ghcr.io/vectorize-io/hindsight-api:latest-slim # Control Plane only ghcr.io/vectorize-io/hindsight-control-plane:latest @@ -175,14 +126,17 @@ See the [Helm chart values.yaml](https://github.com/vectorize-io/hindsight/tree/ ## Bare Metal (pip) -**Best for**: Custom deployments, integration into existing Python applications +**Best for**: Running Hindsight as a standalone service on a host machine. ### Install ```bash -pip install hindsight-all +pip install hindsight-api # Full — works out of the box +pip install hindsight-api-slim # Slim — requires external services for embeddings, reranking, and the database ``` +When using `hindsight-api-slim`, you must configure external providers for all model operations. See [Configuration](./configuration#embeddings) for details. + ### Run with Embedded Database For development and testing, Hindsight can run with an embedded PostgreSQL (pg0): @@ -253,6 +207,42 @@ PORT=80 HINDSIGHT_CP_DATAPLANE_API_URL=https://api.hindsight.io npx @vectorize-i --- +## Embedded in a Python Application + +**Best for**: Using Hindsight programmatically from Python without running a separate server process. + +```bash +pip install hindsight-all # Full — works out of the box +pip install hindsight-all-slim # Slim — requires external services for embeddings, reranking, and the database +``` + +`hindsight-all` supports two modes of embedding: + +**In-process** (`HindsightServer`): the server runs in a background thread inside your application. Best when you want the tightest integration and are already managing your own process lifecycle. + +```python +from hindsight import HindsightServer, HindsightClient + +with HindsightServer(llm_provider="openai", llm_api_key="sk-xxx") as server: + client = HindsightClient(base_url=server.url) + client.retain(bank_id="alice", content="Alice prefers concise answers.") + results = client.recall(bank_id="alice", query="How should I respond to Alice?") +``` + +**Managed subprocess** (`HindsightEmbedded`): the server runs as a background daemon process, shared across multiple Python processes or sessions. The daemon starts on first use and shuts down automatically after an idle timeout. + +```python +from hindsight import HindsightEmbedded + +client = HindsightEmbedded(llm_provider="openai", llm_api_key="sk-xxx") +client.retain(bank_id="alice", content="Alice prefers concise answers.") +results = client.recall(bank_id="alice", query="How should I respond to Alice?") +``` + +See the [Python SDK](../sdks/python.md) for the full API reference. + +--- + ## Next Steps - [Configuration](./configuration.md) — Environment variables and settings diff --git a/hindsight-docs/versioned_docs/version-0.4/developer/extensions.md b/hindsight-docs/versioned_docs/version-0.4/developer/extensions.md index ba8ef011..0078d729 100644 --- a/hindsight-docs/versioned_docs/version-0.4/developer/extensions.md +++ b/hindsight-docs/versioned_docs/version-0.4/developer/extensions.md @@ -30,7 +30,7 @@ HINDSIGHT_API_TENANT_SUPABASE_URL=https://your-project.supabase.co HINDSIGHT_API_TENANT_SUPABASE_SERVICE_KEY=your-service-role-key ``` -See the [source code](https://github.com/vectorize-io/hindsight/blob/main/hindsight-api/hindsight_api/extensions/builtin/supabase_tenant.py) for complete configuration options and implementation details. +See the [source code](https://github.com/vectorize-io/hindsight/blob/main/hindsight-api-slim/hindsight_api/extensions/builtin/supabase_tenant.py) for complete configuration options and implementation details. For other multi-tenant setups with separate schemas per tenant (e.g., custom JWT-based auth), implement a custom `TenantExtension`. diff --git a/hindsight-embed/hindsight_embed/daemon_embed_manager.py b/hindsight-embed/hindsight_embed/daemon_embed_manager.py index 06cf7be0..6b634f37 100644 --- a/hindsight-embed/hindsight_embed/daemon_embed_manager.py +++ b/hindsight-embed/hindsight_embed/daemon_embed_manager.py @@ -91,7 +91,7 @@ class DaemonEmbedManager(EmbedManager): def _find_api_command(self) -> list[str]: """Find the command to run hindsight-api.""" # Check if we're in development mode - dev_api_path = Path(__file__).parent.parent.parent / "hindsight-api" + dev_api_path = Path(__file__).parent.parent.parent / "hindsight-api-slim" if dev_api_path.exists() and (dev_api_path / "pyproject.toml").exists(): return ["uv", "run", "--project", str(dev_api_path), "hindsight-api"] diff --git a/hindsight-integration-tests/tests/test_base_path_deployment.py b/hindsight-integration-tests/tests/test_base_path_deployment.py index 66824b10..42a576ba 100644 --- a/hindsight-integration-tests/tests/test_base_path_deployment.py +++ b/hindsight-integration-tests/tests/test_base_path_deployment.py @@ -26,7 +26,7 @@ import pytest # Paths REPO_ROOT = Path(__file__).parent.parent.parent -API_PATH = REPO_ROOT / "hindsight-api" +API_PATH = REPO_ROOT / "hindsight-api-slim" COMPOSE_EXAMPLES_PATH = REPO_ROOT / "docker" / "docker-compose" / "nginx" # Add hindsight-api to path for direct API testing diff --git a/pyproject.toml b/pyproject.toml index f6a2dc60..cca18a47 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,5 +1,5 @@ [tool.uv.workspace] -members = ["hindsight", "hindsight-api", "hindsight-dev", "hindsight-mcp-server", "hindsight-clients/python", "hindsight-embed"] +members = ["hindsight-all", "hindsight-api", "hindsight-api-slim", "hindsight-all-slim", "hindsight-dev", "hindsight-mcp-server", "hindsight-clients/python", "hindsight-embed"] [tool.uv] # Allow uv to search all configured indexes for packages, not just the first one diff --git a/scripts/hooks/lint.sh b/scripts/hooks/lint.sh index 34bd2a5e..f794e315 100755 --- a/scripts/hooks/lint.sh +++ b/scripts/hooks/lint.sh @@ -42,10 +42,10 @@ echo " Running lints in parallel..." run_task "eslint" "$REPO_ROOT/hindsight-control-plane" "npx eslint --fix src/**/*.{ts,tsx}" run_task "prettier" "$REPO_ROOT/hindsight-control-plane" "npx prettier --write src/**/*.{ts,tsx}" -# Python hindsight-api tasks -run_task "ruff-api-check" "$REPO_ROOT/hindsight-api" "uv run ruff check --fix ." -run_task "ruff-api-format" "$REPO_ROOT/hindsight-api" "uv run ruff format ." -run_task "ty-api" "$REPO_ROOT/hindsight-api" "uv run ty check hindsight_api" +# Python hindsight-api-slim tasks +run_task "ruff-api-check" "$REPO_ROOT/hindsight-api-slim" "uv run ruff check --fix ." +run_task "ruff-api-format" "$REPO_ROOT/hindsight-api-slim" "uv run ruff format ." +run_task "ty-api" "$REPO_ROOT/hindsight-api-slim" "uv run ty check hindsight_api" # Python hindsight-dev tasks run_task "ruff-dev-check" "$REPO_ROOT/hindsight-dev" "uv run ruff check --fix ." diff --git a/scripts/release.sh b/scripts/release.sh index 017aaf83..77190302 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -65,7 +65,7 @@ fi print_info "Updating version in all components..." # Update Python packages -PYTHON_PACKAGES=("hindsight-api" "hindsight-dev" "hindsight" "hindsight-integrations/litellm" "hindsight-integrations/crewai" "hindsight-integrations/pydantic-ai" "hindsight-embed") +PYTHON_PACKAGES=("hindsight-api" "hindsight-api-slim" "hindsight-all-slim" "hindsight-dev" "hindsight-all" "hindsight-integrations/litellm" "hindsight-integrations/crewai" "hindsight-integrations/pydantic-ai" "hindsight-embed") for package in "${PYTHON_PACKAGES[@]}"; do PYPROJECT_FILE="$package/pyproject.toml" if [ -f "$PYPROJECT_FILE" ]; then @@ -79,7 +79,7 @@ done # Update __version__ in Python __init__.py files PYTHON_INIT_FILES=( - "hindsight-api/hindsight_api/__init__.py" + "hindsight-api-slim/hindsight_api/__init__.py" "hindsight-embed/hindsight_embed/__init__.py" "hindsight-clients/python/hindsight_client_api/__init__.py" ) diff --git a/skills/hindsight-docs/references/developer/extensions.md b/skills/hindsight-docs/references/developer/extensions.md index ba8ef011..0078d729 100644 --- a/skills/hindsight-docs/references/developer/extensions.md +++ b/skills/hindsight-docs/references/developer/extensions.md @@ -30,7 +30,7 @@ HINDSIGHT_API_TENANT_SUPABASE_URL=https://your-project.supabase.co HINDSIGHT_API_TENANT_SUPABASE_SERVICE_KEY=your-service-role-key ``` -See the [source code](https://github.com/vectorize-io/hindsight/blob/main/hindsight-api/hindsight_api/extensions/builtin/supabase_tenant.py) for complete configuration options and implementation details. +See the [source code](https://github.com/vectorize-io/hindsight/blob/main/hindsight-api-slim/hindsight_api/extensions/builtin/supabase_tenant.py) for complete configuration options and implementation details. For other multi-tenant setups with separate schemas per tenant (e.g., custom JWT-based auth), implement a custom `TenantExtension`. diff --git a/uv.lock b/uv.lock index c30408cd..92660eee 100644 --- a/uv.lock +++ b/uv.lock @@ -15,7 +15,9 @@ resolution-markers = [ [manifest] members = [ "hindsight-all", + "hindsight-all-slim", "hindsight-api", + "hindsight-api-slim", "hindsight-client", "hindsight-dev", "hindsight-embed", @@ -1432,9 +1434,9 @@ wheels = [ [[package]] name = "hindsight-all" version = "0.4.17" -source = { editable = "hindsight" } +source = { editable = "hindsight-all" } dependencies = [ - { name = "hindsight-api" }, + { name = "hindsight-api-slim", extra = ["all"] }, { name = "hindsight-client" }, { name = "hindsight-embed" }, ] @@ -1447,7 +1449,33 @@ test = [ [package.metadata] requires-dist = [ - { name = "hindsight-api", editable = "hindsight-api" }, + { name = "hindsight-api-slim", extras = ["all"], editable = "hindsight-api-slim" }, + { name = "hindsight-client", editable = "hindsight-clients/python" }, + { name = "hindsight-embed", editable = "hindsight-embed" }, + { name = "pytest", marker = "extra == 'test'", specifier = ">=7.0.0" }, + { name = "pytest-asyncio", marker = "extra == 'test'", specifier = ">=0.21.0" }, +] +provides-extras = ["test"] + +[[package]] +name = "hindsight-all-slim" +version = "0.4.17" +source = { editable = "hindsight-all-slim" } +dependencies = [ + { name = "hindsight-api-slim" }, + { name = "hindsight-client" }, + { name = "hindsight-embed" }, +] + +[package.optional-dependencies] +test = [ + { name = "pytest" }, + { name = "pytest-asyncio" }, +] + +[package.metadata] +requires-dist = [ + { name = "hindsight-api-slim", editable = "hindsight-api-slim" }, { name = "hindsight-client", editable = "hindsight-clients/python" }, { name = "hindsight-embed", editable = "hindsight-embed" }, { name = "pytest", marker = "extra == 'test'", specifier = ">=7.0.0" }, @@ -1459,6 +1487,17 @@ provides-extras = ["test"] name = "hindsight-api" version = "0.4.17" source = { editable = "hindsight-api" } +dependencies = [ + { name = "hindsight-api-slim", extra = ["all"] }, +] + +[package.metadata] +requires-dist = [{ name = "hindsight-api-slim", extras = ["all"], editable = "hindsight-api-slim" }] + +[[package]] +name = "hindsight-api-slim" +version = "0.4.17" +source = { editable = "hindsight-api-slim" } dependencies = [ { name = "aiohttp" }, { name = "alembic" }, @@ -1469,11 +1508,9 @@ dependencies = [ { name = "cohere" }, { name = "cryptography" }, { name = "dateparser" }, - { name = "einops" }, { name = "fastapi", extra = ["standard"] }, { name = "fastmcp" }, { name = "filelock" }, - { name = "flashrank" }, { name = "google-auth" }, { name = "google-genai" }, { name = "greenlet" }, @@ -1483,8 +1520,6 @@ dependencies = [ { name = "langsmith" }, { name = "litellm" }, { name = "markitdown", extra = ["docx", "pdf", "pptx", "xls", "xlsx"] }, - { name = "mlx" }, - { name = "mlx-lm" }, { name = "obstore" }, { name = "openai" }, { name = "opentelemetry-api" }, @@ -1493,7 +1528,6 @@ dependencies = [ { name = "opentelemetry-instrumentation-fastapi" }, { name = "opentelemetry-sdk" }, { name = "opentelemetry-semantic-conventions" }, - { name = "pg0-embedded" }, { name = "pgvector" }, { name = "pillow" }, { name = "protobuf" }, @@ -1504,12 +1538,8 @@ dependencies = [ { name = "python-dateutil" }, { name = "python-dotenv" }, { name = "rich" }, - { name = "safetensors" }, - { name = "sentence-transformers" }, { name = "sqlalchemy" }, { name = "tiktoken" }, - { name = "torch" }, - { name = "transformers" }, { name = "typer" }, { name = "urllib3" }, { name = "uvicorn" }, @@ -1518,6 +1548,30 @@ dependencies = [ ] [package.optional-dependencies] +all = [ + { name = "einops" }, + { name = "flashrank" }, + { name = "mlx" }, + { name = "mlx-lm" }, + { name = "pg0-embedded" }, + { name = "safetensors" }, + { name = "sentence-transformers" }, + { name = "torch" }, + { name = "transformers" }, +] +embedded-db = [ + { name = "pg0-embedded" }, +] +local-ml = [ + { name = "einops" }, + { name = "flashrank" }, + { name = "mlx" }, + { name = "mlx-lm" }, + { name = "safetensors" }, + { name = "sentence-transformers" }, + { name = "torch" }, + { name = "transformers" }, +] test = [ { name = "filelock" }, { name = "pytest" }, @@ -1551,23 +1605,24 @@ requires-dist = [ { name = "cohere", specifier = ">=5.0.0" }, { name = "cryptography", specifier = ">=46.0.5" }, { name = "dateparser", specifier = ">=1.2.2" }, - { name = "einops", specifier = ">=0.8.2" }, + { name = "einops", marker = "extra == 'local-ml'", specifier = ">=0.8.2" }, { name = "fastapi", extras = ["standard"], specifier = ">=0.120.3" }, { name = "fastmcp", specifier = ">=2.14.0" }, { name = "filelock", specifier = ">=3.20.1" }, { name = "filelock", marker = "extra == 'test'", specifier = ">=3.20.1" }, - { name = "flashrank", specifier = ">=0.2.0" }, + { name = "flashrank", marker = "extra == 'local-ml'", specifier = ">=0.2.0" }, { name = "google-auth", specifier = ">=2.0.0" }, { name = "google-genai", specifier = ">=1.0.0" }, { name = "greenlet", specifier = ">=3.2.4" }, + { name = "hindsight-api-slim", extras = ["local-ml", "embedded-db"], marker = "extra == 'all'" }, { name = "httpx", specifier = ">=0.27.0" }, { name = "langchain-core", specifier = ">=1.2.11" }, { name = "langchain-text-splitters", specifier = ">=0.3.0" }, { name = "langsmith", specifier = ">=0.6.3" }, { name = "litellm", specifier = ">=1.0.0" }, { name = "markitdown", extras = ["pdf", "docx", "pptx", "xlsx", "xls"], specifier = ">=0.1.4" }, - { name = "mlx", specifier = ">=0.31.0" }, - { name = "mlx-lm", specifier = ">=0.31.1" }, + { name = "mlx", marker = "extra == 'local-ml'", specifier = ">=0.31.0" }, + { name = "mlx-lm", marker = "extra == 'local-ml'", specifier = ">=0.31.1" }, { name = "obstore", specifier = ">=0.4.0" }, { name = "openai", specifier = ">=1.0.0" }, { name = "opentelemetry-api", specifier = ">=1.20.0" }, @@ -1576,7 +1631,7 @@ requires-dist = [ { name = "opentelemetry-instrumentation-fastapi", specifier = ">=0.41b0" }, { name = "opentelemetry-sdk", specifier = ">=1.20.0" }, { name = "opentelemetry-semantic-conventions", specifier = ">=0.41b0" }, - { name = "pg0-embedded", specifier = ">=0.11.0" }, + { name = "pg0-embedded", marker = "extra == 'embedded-db'", specifier = ">=0.11.0" }, { name = "pgvector", specifier = ">=0.4.1" }, { name = "pillow", specifier = ">=12.1.1" }, { name = "protobuf", specifier = ">=6.33.5" }, @@ -1591,20 +1646,20 @@ requires-dist = [ { name = "python-dateutil", specifier = ">=2.8.0" }, { name = "python-dotenv", specifier = ">=1.0.0" }, { name = "rich", specifier = ">=13.0.0" }, - { name = "safetensors", specifier = ">=0.6.2" }, - { name = "sentence-transformers", specifier = ">=3.3.0" }, + { name = "safetensors", marker = "extra == 'local-ml'", specifier = ">=0.6.2" }, + { name = "sentence-transformers", marker = "extra == 'local-ml'", specifier = ">=3.3.0" }, { name = "sqlalchemy", specifier = ">=2.0.44" }, { name = "testcontainers", marker = "extra == 'test'", specifier = ">=4.0.0" }, { name = "tiktoken", specifier = ">=0.12.0" }, - { name = "torch", specifier = ">=2.6.0" }, - { name = "transformers", specifier = ">=4.53.0" }, + { name = "torch", marker = "extra == 'local-ml'", specifier = ">=2.6.0" }, + { name = "transformers", marker = "extra == 'local-ml'", specifier = ">=4.53.0" }, { name = "typer", specifier = ">=0.9.0" }, { name = "urllib3", specifier = ">=2.6.3" }, { name = "uvicorn", specifier = ">=0.38.0" }, { name = "uvloop", specifier = ">=0.22.1" }, { name = "wsproto", specifier = ">=1.0.0" }, ] -provides-extras = ["test"] +provides-extras = ["local-ml", "embedded-db", "all", "test"] [package.metadata.requires-dev] dev = [