fleet-memory/hindsight-api-slim/hindsight_api/extensions/tenant.py
Nicolò Boschi 15ea23d5d6
feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560)
* feat: introduce hindsight-api-slim and hindsight-all-slim packages

Closes #552

- Move all source code from hindsight-api/ to new hindsight-api-slim/
- hindsight-api-slim has heavy ML deps (torch, sentence-transformers,
  transformers, einops, flashrank, mlx, mlx-lm, safetensors) and
  pg0-embedded as optional extras: [local-ml], [embedded-db], [all]
- hindsight-api becomes a zero-code meta-package depending on
  hindsight-api-slim[all] for full backward compatibility
- Add hindsight-all-slim meta-package: hindsight-api-slim + client + embed
- hindsight-all updated to depend on hindsight-api-slim[all]
- pg0.py: lazy-import pg0 with clear ImportError pointing to [embedded-db]
- Dockerfile: replace sed hack with proper uv sync --extra flags
- Update release.yml, test.yml, lint.sh, release.sh, CLAUDE.md and
  all path references throughout the repo

* refactor: rename hindsight/ directory to hindsight-all/

* docs: document hindsight-api-slim and hindsight-all-slim package variants

Add package variants table and extras explanation to installation.md

* docs: remove emojis from installation.md, use professional tone

* docs: link Docker slim variant to pip package variants section

* docs: consolidate Docker image variants into single table

* ci: fix working-directory paths after package restructure

- Replace all hindsight-api → hindsight-api-slim in test.yml
- Replace hindsight → hindsight-all in test.yml
- Add --extra embedded-db to test-embed API install step

* ci: add local-ml and embedded-db extras to API sync steps

These extras were previously implicit in the old hindsight-api package
(which bundled everything). Now that hindsight-api-slim uses optional
extras, we must explicitly request local-ml and embedded-db in CI.

* ci: add API install step with embedded-db to test-embed smoke test

The smoke test starts hindsight-api as a daemon, which requires pg0-embedded.
Add a dedicated install step for hindsight-api-slim with embedded-db extra
so the daemon can start successfully.

* ci: remove --no-install-project when using optional extras

When --no-install-project is combined with --extra, the optional deps
are not installed because extras require the project to be active.
Remove --no-install-project from steps that need local-ml or embedded-db.

* ci: fix ordering of uv sync steps to preserve optional extras

When uv sync runs for a different workspace member, it removes optional
extras installed for other members. Fix by always running extra-requiring
API sync last, after other workspace member syncs.

Also remove --no-install-project from embedded-db sync in test-embed,
as --no-install-project prevents optional extras from being active.

* ci: add local-ml extra to test-embed API install for smoke test

The smoke test starts the full API server which needs sentence-transformers
for local embeddings (default provider). Add local-ml extra to the install.

* ci: simplify extras with --all-extras and add slim pip smoke test

- Replace explicit --extra local-ml --extra embedded-db with --all-extras
  for cleaner, more maintainable sync steps
- Add test-pip-slim job: tests hindsight-api-slim[embedded-db] without
  local ML models, using Cohere for embeddings/reranking (mirrors Docker
  slim smoke test approach)

* ci: simplify slim smoke test to health check only (mirrors Docker test)
2026-03-13 13:50:03 +01:00

158 lines
5.4 KiB
Python

"""Tenant Extension for multi-tenancy and API key authentication."""
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import Any
from hindsight_api.extensions.base import Extension
from hindsight_api.models import RequestContext
class AuthenticationError(Exception):
"""Raised when authentication fails."""
def __init__(self, reason: str, headers: dict[str, str] | None = None):
self.reason = reason
self.headers = headers or {}
super().__init__(f"Authentication failed: {reason}")
@dataclass
class TenantContext:
"""
Tenant context returned by authentication.
Contains the PostgreSQL schema name for tenant isolation.
All database queries will use fully-qualified table names
with this schema (e.g., schema_name.memory_units).
"""
schema_name: str
@dataclass
class Tenant:
"""
Represents a tenant for worker discovery.
Used by list_tenants() to return tenant information including
the PostgreSQL schema name for database operations.
"""
schema: str
class TenantExtension(Extension, ABC):
"""
Extension for multi-tenancy and API key authentication.
This extension validates incoming requests and returns the tenant context
including the PostgreSQL schema to use for database operations.
Built-in implementation:
hindsight_api.extensions.builtin.tenant.ApiKeyTenantExtension
Enable via environment variable:
HINDSIGHT_API_TENANT_EXTENSION=hindsight_api.extensions.builtin.tenant:ApiKeyTenantExtension
HINDSIGHT_API_TENANT_API_KEY=your-secret-key
The returned schema_name is used for fully-qualified table names in queries,
enabling tenant isolation at the database level.
"""
@abstractmethod
async def authenticate(self, context: RequestContext) -> TenantContext:
"""
Authenticate the action context and return tenant context.
Args:
context: The action context containing API key and other auth data.
Returns:
TenantContext with the schema_name for database operations.
Raises:
AuthenticationError: If authentication fails.
"""
...
@abstractmethod
async def list_tenants(self) -> list[Tenant]:
"""
List all tenants that should be processed by workers.
This method is used by the worker to discover all tenants that need
task polling. Workers will poll for pending tasks in each tenant's schema.
Returns:
List of Tenant objects containing schema information.
For single-tenant setups, return [Tenant(schema="public")].
"""
...
async def get_tenant_config(self, context: RequestContext) -> dict[str, Any]:
"""
Get tenant-specific configuration overrides.
This method is called during hierarchical configuration resolution to get
tenant-level config overrides. The returned dict should contain Python field
names (lowercase snake_case) as keys, not environment variable names.
Example:
{"llm_model": "gpt-4", "retain_extraction_mode": "verbose"}
The default implementation returns an empty dict (no tenant-specific config).
Override this method in custom extensions to provide tenant-specific configuration.
Args:
context: The request context containing tenant information.
Returns:
Dict of config field names to values (only configurable fields).
Empty dict if no tenant-specific config.
"""
return {}
async def get_allowed_config_fields(self, context: RequestContext, bank_id: str) -> set[str] | None:
"""
Get set of config fields that this tenant/bank is allowed to modify.
This method controls which configurable fields can be modified via the bank config API.
It enables fine-grained permission control per tenant or per bank.
Examples:
- Return None: Allow all configurable fields (default)
- Return {"retain_chunk_size", "retain_custom_instructions"}: Allow only these fields
- Return set(): Allow no modifications (read-only)
The default implementation returns None (all configurable fields allowed).
Override this method in custom extensions to implement custom permission logic.
Args:
context: The request context containing tenant information.
bank_id: The bank identifier for per-bank permissions.
Returns:
Set of allowed field names, or None to allow all configurable fields.
Returned fields must be a subset of HindsightConfig.get_configurable_fields().
"""
return None
async def authenticate_mcp(self, context: RequestContext) -> TenantContext:
"""
Authenticate MCP requests.
By default, this calls authenticate(). Override this method to provide
different authentication behavior for MCP endpoints (e.g., to disable
auth for backwards compatibility with existing MCP servers).
Args:
context: The action context containing API key and other auth data.
Returns:
TenantContext with the schema_name for database operations.
Raises:
AuthenticationError: If authentication fails.
"""
return await self.authenticate(context)