feat: per-request file parser selection with fallback chains (#514)

* feat: allow per-request file parser selection with fallback chains

Clients can now specify which parser(s) to use when calling the file
retain endpoint, instead of being locked to the server-side default.

Changes:
- `parser` field added to `FileRetainRequest` (request-level default)
  and `FileRetainMetadata` (per-file override); accepts a single name
  or an ordered fallback chain (list)
- Resolution priority: per-file > request-level > server default
- `HINDSIGHT_API_FILE_PARSER` now accepts a comma-separated fallback
  chain (e.g. `iris,markitdown`); fully backward-compatible
- New `HINDSIGHT_API_FILE_PARSER_ALLOWLIST` env var restricts which
  parsers clients may request (defaults to all registered parsers)
- Invalid/disallowed parser names are rejected with HTTP 400
- `FileParserRegistry.convert_with_fallback()` tries each parser in
  order, falling back on UnsupportedFileTypeError, empty content, or
  any other error
- Worker updated to use the fallback chain stored per-task
- OpenAPI spec and all generated clients regenerated

* fix: handle on_file_convert_complete hook and rebase onto main

- Return ConvertResult dataclass from convert_with_fallback() instead
  of a plain str, carrying both the content and the winning parser name
- Use winning_parser_name in the on_file_convert_complete hook so
  parser_name reflects the parser that actually succeeded, not the chain
- Update all test calls to submit_async_file_retain() to use the new
  per-item parser field instead of the removed top-level parser= kwarg

* docs: document HINDSIGHT_API_FILE_PARSER fallback chain and ALLOWLIST

hindsight-api/hindsight_api/api/http.py

@@ -477,6 +477,11 @@ class FileRetainMetadata(BaseModel):
     metadata: dict[str, Any] | None = Field(default=None, description="Additional metadata")
     tags: list[str] | None = Field(default=None, description="Tags for this file")
     timestamp: str | None = Field(default=None, description="ISO timestamp")
+    parser: str | list[str] | None = Field(
+        default=None,
+        description="Parser or ordered fallback chain for this file (overrides request-level parser). "
+        "E.g. 'iris' or ['iris', 'markitdown'].",
+    )
 
 
 class FileRetainRequest(BaseModel):
@@ -485,14 +490,21 @@ class FileRetainRequest(BaseModel):
     model_config = ConfigDict(
         json_schema_extra={
             "example": {
+                "parser": "iris",
                 "files_metadata": [
                     {"document_id": "report_2024", "tags": ["quarterly"]},
-                    {"context": "meeting notes"},
+                    {"context": "meeting notes", "parser": ["iris", "markitdown"]},
                 ],
             }
         }
     )
 
+    parser: str | list[str] | None = Field(
+        default=None,
+        description="Default parser or ordered fallback chain for all files in this request. "
+        "E.g. 'markitdown' or ['iris', 'markitdown']. Falls back to server default if not set. "
+        "Per-file 'parser' in files_metadata takes precedence over this value.",
+    )
     files_metadata: list[FileRetainMetadata] | None = Field(
         default=None,
         description="Metadata for each file (optional, must match number of files if provided)",
@@ -4321,8 +4333,14 @@ def _register_routes(app: FastAPI):
         "Use the operations endpoint to monitor progress.\n\n"
         "**Request format:** multipart/form-data with:\n"
         "- `files`: One or more files to upload\n"
-        "- `request`: JSON string with FileRetainRequest model (files_metadata)\n\n"
-        "**Note:** File parser is configured server-side via `HINDSIGHT_API_FILE_PARSER` (default: markitdown).",
+        "- `request`: JSON string with FileRetainRequest model\n\n"
+        "**Parser selection:**\n"
+        "- Set `parser` in the request body to override the server default for all files.\n"
+        "- Set `parser` inside a `files_metadata` entry for per-file control.\n"
+        "- Pass a list (e.g. `['iris', 'markitdown']`) to define an ordered fallback chain — "
+        "each parser is tried in sequence until one succeeds.\n"
+        "- Falls back to the server default (`HINDSIGHT_API_FILE_PARSER`) if not specified.\n"
+        "- Only parsers enabled on the server may be requested; others return HTTP 400.",
         operation_id="file_retain",
         tags=["Files"],
     )
@@ -4368,20 +4386,39 @@ def _register_routes(app: FastAPI):
                     detail=f"files_metadata count ({len(request_data.files_metadata)}) must match files count ({len(files)})",
                 )
 
+            # Resolve the registered parser names for allowlist validation
+            registered_parsers = app.state.memory._parser_registry.list_parsers()
+            allowlist = config.file_parser_allowlist if config.file_parser_allowlist is not None else registered_parsers
+
+            def _resolve_parser(raw: str | list[str] | None) -> list[str]:
+                """Normalize parser value to a non-empty list of names."""
+                if raw is None:
+                    return config.file_parser
+                return [raw] if isinstance(raw, str) else list(raw)
+
+            def _validate_parsers(parsers: list[str], context: str) -> None:
+                """Raise HTTP 400 if any parser name is not in the allowlist."""
+                disallowed = [p for p in parsers if p not in allowlist]
+                if disallowed:
+                    raise HTTPException(
+                        status_code=400,
+                        detail=f"Parser(s) not available ({context}): {disallowed}. Available: {allowlist}",
+                    )
+
+            # Validate request-level parser early (before reading files)
+            if request_data.parser is not None:
+                _validate_parsers(_resolve_parser(request_data.parser), "request-level parser")
+
             # Prepare file items and calculate total batch size
+            import io
+
             file_items = []
             total_batch_size = 0
 
             for i, file in enumerate(files):
                 # Read file content to check size
                 file_content = await file.read()
-                size = len(file_content)
-                total_batch_size += size
-
-                # Create a temporary file-like object from the bytes
-                import io
-
-                file_obj = io.BytesIO(file_content)
+                total_batch_size += len(file_content)
 
                 # Create a mock UploadFile with the necessary attributes
                 class FileWrapper:
@@ -4389,7 +4426,6 @@ def _register_routes(app: FastAPI):
                         self._content = content
                         self.filename = filename
                         self.content_type = content_type
-                        self._buffer = io.BytesIO(content)
 
                     async def read(self):
                         return self._content
@@ -4400,6 +4436,12 @@ def _register_routes(app: FastAPI):
                 file_meta = request_data.files_metadata[i] if request_data.files_metadata else FileRetainMetadata()
                 doc_id = file_meta.document_id or f"file_{uuid.uuid4()}"
 
+                # Resolve and validate per-file parser chain
+                # Priority: per-file > request-level > server default
+                raw_parser = file_meta.parser if file_meta.parser is not None else request_data.parser
+                parser_chain = _resolve_parser(raw_parser)
+                _validate_parsers(parser_chain, f"file '{file.filename}'")
+
                 item = {
                     "file": wrapped_file,
                     "document_id": doc_id,
@@ -4407,6 +4449,7 @@ def _register_routes(app: FastAPI):
                     "metadata": file_meta.metadata or {},
                     "tags": file_meta.tags or [],
                     "timestamp": file_meta.timestamp,
+                    "parser": parser_chain,
                 }
                 file_items.append(item)
 
@@ -4421,7 +4464,6 @@ def _register_routes(app: FastAPI):
             result = await app.state.memory.submit_async_file_retain(
                 bank_id=bank_id,
                 file_items=file_items,
-                parser=config.file_parser,
                 document_tags=None,
                 request_context=request_context,
             )

hindsight-api/hindsight_api/config.py

@@ -280,6 +280,7 @@ ENV_FILE_STORAGE_AZURE_CONTAINER = "HINDSIGHT_API_FILE_STORAGE_AZURE_CONTAINER"
 ENV_FILE_STORAGE_AZURE_ACCOUNT_NAME = "HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_NAME"
 ENV_FILE_STORAGE_AZURE_ACCOUNT_KEY = "HINDSIGHT_API_FILE_STORAGE_AZURE_ACCOUNT_KEY"
 ENV_FILE_PARSER = "HINDSIGHT_API_FILE_PARSER"
+ENV_FILE_PARSER_ALLOWLIST = "HINDSIGHT_API_FILE_PARSER_ALLOWLIST"
 ENV_FILE_PARSER_IRIS_TOKEN = "HINDSIGHT_API_FILE_PARSER_IRIS_TOKEN"
 ENV_FILE_PARSER_IRIS_ORG_ID = "HINDSIGHT_API_FILE_PARSER_IRIS_ORG_ID"
 ENV_FILE_CONVERSION_MAX_BATCH_SIZE_MB = "HINDSIGHT_API_FILE_CONVERSION_MAX_BATCH_SIZE_MB"
@@ -439,7 +440,8 @@ DEFAULT_RETAIN_BATCH_POLL_INTERVAL_SECONDS = 60  # Batch API polling interval in
 
 # File storage defaults
 DEFAULT_FILE_STORAGE_TYPE = "native"  # PostgreSQL BYTEA storage
-DEFAULT_FILE_PARSER = "markitdown"  # File parser to use (markitdown is the only supported parser)
+DEFAULT_FILE_PARSER = "markitdown"  # Default parser fallback chain (comma-separated, e.g. "iris,markitdown")
+DEFAULT_FILE_PARSER_ALLOWLIST = None  # Allowlist of parsers clients may request (None = all registered parsers)
 DEFAULT_FILE_CONVERSION_MAX_BATCH_SIZE_MB = 100  # Max total batch size in MB (all files combined)
 DEFAULT_FILE_CONVERSION_MAX_BATCH_SIZE = 10  # Max files per batch upload
 DEFAULT_ENABLE_FILE_UPLOAD_API = True  # Enable file upload endpoint
@@ -550,6 +552,11 @@ class JsonFormatter(logging.Formatter):
         return json.dumps(log_entry)
 
 
+def _parse_str_list(value: str) -> list[str]:
+    """Parse a comma-separated string into a non-empty list of stripped tokens."""
+    return [v.strip() for v in value.split(",") if v.strip()]
+
+
 def _validate_extraction_mode(mode: str) -> str:
     """Validate and normalize extraction mode."""
     mode_lower = mode.lower()
@@ -709,7 +716,8 @@ class HindsightConfig:
     file_storage_azure_container: str | None  # Azure container name (required for azure storage)
     file_storage_azure_account_name: str | None  # Azure storage account name
     file_storage_azure_account_key: str | None  # Azure storage account key
-    file_parser: str  # File parser to use (e.g., "markitdown", "iris")
+    file_parser: list[str]  # Ordered fallback chain of parsers (e.g. ["iris", "markitdown"])
+    file_parser_allowlist: list[str] | None  # Parsers clients may request (None = all registered)
     file_parser_iris_token: str | None  # Vectorize API token for iris parser (VECTORIZE_TOKEN)
     file_parser_iris_org_id: str | None  # Vectorize org ID for iris parser (VECTORIZE_ORG_ID)
     file_conversion_max_batch_size_mb: int  # Max total batch size in MB (all files combined)
@@ -1151,7 +1159,10 @@ class HindsightConfig:
             file_storage_azure_container=os.getenv(ENV_FILE_STORAGE_AZURE_CONTAINER) or None,
             file_storage_azure_account_name=os.getenv(ENV_FILE_STORAGE_AZURE_ACCOUNT_NAME) or None,
             file_storage_azure_account_key=os.getenv(ENV_FILE_STORAGE_AZURE_ACCOUNT_KEY) or None,
-            file_parser=os.getenv(ENV_FILE_PARSER, DEFAULT_FILE_PARSER),
+            file_parser=_parse_str_list(os.getenv(ENV_FILE_PARSER, DEFAULT_FILE_PARSER)),
+            file_parser_allowlist=_parse_str_list(os.getenv(ENV_FILE_PARSER_ALLOWLIST))
+            if os.getenv(ENV_FILE_PARSER_ALLOWLIST)
+            else None,
             file_parser_iris_token=os.getenv(ENV_FILE_PARSER_IRIS_TOKEN) or None,
             file_parser_iris_org_id=os.getenv(ENV_FILE_PARSER_IRIS_ORG_ID) or None,
             file_conversion_max_batch_size_mb=int(

hindsight-api/hindsight_api/engine/memory_engine.py

@@ -653,13 +653,19 @@ class MemoryEngine(MemoryEngineInterface):
             # Retrieve file from storage
             file_data = await self._file_storage.retrieve(storage_key)
 
-            # Convert to markdown
-            parser = self._parser_registry.get_parser(
-                name=task_dict.get("parser"),
+            # Convert to markdown using the ordered fallback chain stored in the task payload.
+            # task_dict["parser"] is always a list[str] set at submission time.
+            parser_chain: list[str] = task_dict.get("parser") or []
+            if not parser_chain:
+                raise ValueError("No parser chain defined for file_convert_retain task")
+            convert_result = await self._parser_registry.convert_with_fallback(
+                parsers=parser_chain,
+                file_data=file_data,
                 filename=filename,
                 content_type=task_dict.get("content_type"),
             )
-            markdown_content = await parser.convert(file_data, filename)
+            markdown_content = convert_result.content
+            winning_parser = convert_result.parser_name
         except Exception as e:
             # Re-raise with filename context for better error reporting
             error_msg = f"Failed to parse file '{filename}': {str(e)}"
@@ -686,7 +692,7 @@ class MemoryEngine(MemoryEngineInterface):
                 await self._operation_validator.on_file_convert_complete(
                     FileConvertResult(
                         bank_id=bank_id,
-                        parser_name=task_dict.get("parser", "unknown"),
+                        parser_name=winning_parser,
                         filename=filename,
                         output_chars=len(markdown_content),
                         output_text=markdown_content,
@@ -7093,7 +7099,6 @@ class MemoryEngine(MemoryEngineInterface):
         self,
         bank_id: str,
         file_items: list[dict[str, Any]],
-        parser: str,
         document_tags: list[str] | None,
         request_context: "RequestContext",
     ) -> dict[str, Any]:
@@ -7112,7 +7117,7 @@ class MemoryEngine(MemoryEngineInterface):
                 - metadata: Optional metadata dict
                 - tags: Optional tags list
                 - timestamp: Optional timestamp
-            parser: Parser name (e.g., "markitdown")
+                - parser: Ordered list of parser names to try (fallback chain)
             document_tags: Tags applied to all documents
             request_context: Request context for authentication
 
@@ -7168,7 +7173,7 @@ class MemoryEngine(MemoryEngineInterface):
                 "storage_key": storage_key,
                 "original_filename": file.filename,
                 "content_type": file.content_type or "application/octet-stream",
-                "parser": parser,
+                "parser": item["parser"],
                 "context": item.get("context"),
                 "metadata": item.get("metadata", {}),
                 "tags": item.get("tags", []),

hindsight-api/hindsight_api/engine/parsers/__init__.py

@@ -1,10 +1,31 @@
 """File parser implementations."""
 
+import logging
+from dataclasses import dataclass
+
 from .base import FileParser, UnsupportedFileTypeError
 from .iris import IrisParser
 from .markitdown import MarkitdownParser
 
-__all__ = ["FileParser", "UnsupportedFileTypeError", "IrisParser", "MarkitdownParser", "FileParserRegistry"]
+__all__ = [
+    "FileParser",
+    "UnsupportedFileTypeError",
+    "IrisParser",
+    "MarkitdownParser",
+    "FileParserRegistry",
+    "ConvertResult",
+]
+
+
+@dataclass
+class ConvertResult:
+    """Result of a successful file conversion."""
+
+    content: str
+    parser_name: str
+
+
+logger = logging.getLogger(__name__)
 
 
 class FileParserRegistry:
@@ -57,6 +78,51 @@ class FileParserRegistry:
 
         raise ValueError(f"No parser found for {filename}. Available parsers: {list(self._parsers.keys())}")
 
+    async def convert_with_fallback(
+        self,
+        parsers: list[str],
+        file_data: bytes,
+        filename: str,
+        content_type: str | None = None,
+    ) -> ConvertResult:
+        """
+        Try each parser in order, falling back on failure or empty content.
+
+        Moves to the next parser if the current one raises UnsupportedFileTypeError
+        or returns empty content. Any other exception (RuntimeError, network error,
+        etc.) also triggers a fallback so the chain is exhausted before failing.
+
+        Args:
+            parsers: Ordered list of parser names to try
+            file_data: Raw file bytes
+            filename: Original filename
+            content_type: MIME type (optional)
+
+        Returns:
+            ConvertResult with the parsed content and the name of the parser that succeeded
+
+        Raises:
+            ValueError: If a parser name is not registered
+            RuntimeError: If all parsers fail or return empty content
+        """
+        last_error: Exception | None = None
+        for name in parsers:
+            parser = self.get_parser(name, filename, content_type)
+            try:
+                content = await parser.convert(file_data, filename)
+                if content and content.strip():
+                    return ConvertResult(content=content, parser_name=name)
+                logger.warning(f"Parser '{name}' returned empty content for '{filename}', trying next")
+                last_error = RuntimeError(f"Parser '{name}' returned no content for '{filename}'")
+            except UnsupportedFileTypeError as e:
+                logger.warning(f"Parser '{name}' does not support '{filename}', trying next: {e}")
+                last_error = e
+            except Exception as e:
+                logger.warning(f"Parser '{name}' failed for '{filename}', trying next: {e}")
+                last_error = e
+
+        raise last_error or RuntimeError(f"No parsers available for '{filename}'")
+
     def list_parsers(self) -> list[str]:
         """Get list of registered parser names."""
         return list(self._parsers.keys())

hindsight-api/hindsight_api/main.py

@@ -268,6 +268,7 @@ def main():
             file_storage_azure_account_name=config.file_storage_azure_account_name,
             file_storage_azure_account_key=config.file_storage_azure_account_key,
             file_parser=config.file_parser,
+            file_parser_allowlist=config.file_parser_allowlist,
             file_parser_iris_token=config.file_parser_iris_token,
             file_parser_iris_org_id=config.file_parser_iris_org_id,
             file_conversion_max_batch_size_mb=config.file_conversion_max_batch_size_mb,

hindsight-api/tests/test_file_retain.py

@@ -403,13 +403,13 @@ async def test_file_conversion_creates_separate_retain_operation(memory_no_llm_v
             "metadata": {"source": "test"},
             "tags": ["test_tag"],
             "timestamp": None,
+            "parser": ["markitdown"],
         }
     ]
 
     result = await memory_no_llm_verify.submit_async_file_retain(
         bank_id=bank_id,
         file_items=file_items,
-        parser="markitdown",
         document_tags=["two_phase_test"],
         request_context=context,
     )
@@ -521,6 +521,7 @@ async def test_file_conversion_failure_sets_status_to_failed(memory_no_llm_verif
             "metadata": {},
             "tags": [],
             "timestamp": None,
+            "parser": ["failing_converter"],
         }
     ]
 
@@ -528,7 +529,6 @@ async def test_file_conversion_failure_sets_status_to_failed(memory_no_llm_verif
     result = await memory_no_llm_verify.submit_async_file_retain(
         bank_id=bank_id,
         file_items=file_items,
-        parser="failing_converter",
         document_tags=None,
         request_context=context,
     )
@@ -620,13 +620,13 @@ async def test_on_file_convert_complete_hook_called(memory_no_llm_verify, sample
             "metadata": {},
             "tags": [],
             "timestamp": None,
+            "parser": ["markitdown"],
         }
     ]
 
     await memory_no_llm_verify.submit_async_file_retain(
         bank_id=bank_id,
         file_items=file_items,
-        parser="markitdown",
         document_tags=None,
         request_context=context,
     )
@@ -677,6 +677,7 @@ async def test_on_file_convert_complete_hook_called_for_each_file(memory_no_llm_
             "metadata": {},
             "tags": [],
             "timestamp": None,
+            "parser": ["markitdown"],
         },
         {
             "file": MockFile(b"Second document content", "second.txt", "text/plain"),
@@ -685,13 +686,13 @@ async def test_on_file_convert_complete_hook_called_for_each_file(memory_no_llm_
             "metadata": {},
             "tags": [],
             "timestamp": None,
+            "parser": ["markitdown"],
         },
     ]
 
     await memory_no_llm_verify.submit_async_file_retain(
         bank_id=bank_id,
         file_items=file_items,
-        parser="markitdown",
         document_tags=None,
         request_context=context,
     )
@@ -750,13 +751,13 @@ async def test_on_file_convert_complete_hook_not_called_on_conversion_failure(me
             "metadata": {},
             "tags": [],
             "timestamp": None,
+            "parser": ["hookfail_parser"],
         }
     ]
 
     await memory_no_llm_verify.submit_async_file_retain(
         bank_id=bank_id,
         file_items=file_items,
-        parser="hookfail_parser",
         document_tags=None,
         request_context=context,
     )

hindsight-clients/go/api/openapi.yaml

@@ -2492,9 +2492,14 @@ paths:
 
         **Request format:** multipart/form-data with:
         - `files`: One or more files to upload
-        - `request`: JSON string with FileRetainRequest model (files_metadata)
+        - `request`: JSON string with FileRetainRequest model
 
-        **Note:** File parser is configured server-side via `HINDSIGHT_API_FILE_PARSER` (default: markitdown).
+        **Parser selection:**
+        - Set `parser` in the request body to override the server default for all files.
+        - Set `parser` inside a `files_metadata` entry for per-file control.
+        - Pass a list (e.g. `['iris', 'markitdown']`) to define an ordered fallback chain — each parser is tried in sequence until one succeeds.
+        - Falls back to the server default (`HINDSIGHT_API_FILE_PARSER`) if not specified.
+        - Only parsers enabled on the server may be requested; others return HTTP 400.
       operationId: file_retain
       parameters:
       - explode: false

hindsight-clients/go/api_files.go

@@ -78,9 +78,14 @@ Use the operations endpoint to monitor progress.
 
 **Request format:** multipart/form-data with:
 - `files`: One or more files to upload
-- `request`: JSON string with FileRetainRequest model (files_metadata)
+- `request`: JSON string with FileRetainRequest model
 
-**Note:** File parser is configured server-side via `HINDSIGHT_API_FILE_PARSER` (default: markitdown).
+**Parser selection:**
+- Set `parser` in the request body to override the server default for all files.
+- Set `parser` inside a `files_metadata` entry for per-file control.
+- Pass a list (e.g. `['iris', 'markitdown']`) to define an ordered fallback chain — each parser is tried in sequence until one succeeds.
+- Falls back to the server default (`HINDSIGHT_API_FILE_PARSER`) if not specified.
+- Only parsers enabled on the server may be requested; others return HTTP 400.
 
  @param ctx context.Context - for authentication, logging, cancellation, deadlines, tracing, etc. Passed from http.Request or context.Background().
  @param bankId

hindsight-clients/python/hindsight_client_api/api/files_api.py

@@ -61,7 +61,7 @@ class FilesApi:
     ) -> FileRetainResponse:
         """Convert files to memories
 
-        Upload files (PDF, DOCX, etc.), convert them to markdown, and retain as memories.  This endpoint handles file upload, conversion, and memory creation in a single operation.  **Features:** - Supports PDF, DOCX, PPTX, XLSX, images (with OCR), audio (with transcription) - Automatic file-to-markdown conversion using pluggable parsers - Files stored in object storage (PostgreSQL by default, S3 for production) - Each file becomes a separate document with optional metadata/tags - Always processes asynchronously — returns operation IDs immediately  **The system automatically:** 1. Stores uploaded files in object storage 2. Converts files to markdown 3. Creates document records with file metadata 4. Extracts facts and creates memory units (same as regular retain)  Use the operations endpoint to monitor progress.  **Request format:** multipart/form-data with: - `files`: One or more files to upload - `request`: JSON string with FileRetainRequest model (files_metadata)  **Note:** File parser is configured server-side via `HINDSIGHT_API_FILE_PARSER` (default: markitdown).
+        Upload files (PDF, DOCX, etc.), convert them to markdown, and retain as memories.  This endpoint handles file upload, conversion, and memory creation in a single operation.  **Features:** - Supports PDF, DOCX, PPTX, XLSX, images (with OCR), audio (with transcription) - Automatic file-to-markdown conversion using pluggable parsers - Files stored in object storage (PostgreSQL by default, S3 for production) - Each file becomes a separate document with optional metadata/tags - Always processes asynchronously — returns operation IDs immediately  **The system automatically:** 1. Stores uploaded files in object storage 2. Converts files to markdown 3. Creates document records with file metadata 4. Extracts facts and creates memory units (same as regular retain)  Use the operations endpoint to monitor progress.  **Request format:** multipart/form-data with: - `files`: One or more files to upload - `request`: JSON string with FileRetainRequest model  **Parser selection:** - Set `parser` in the request body to override the server default for all files. - Set `parser` inside a `files_metadata` entry for per-file control. - Pass a list (e.g. `['iris', 'markitdown']`) to define an ordered fallback chain — each parser is tried in sequence until one succeeds. - Falls back to the server default (`HINDSIGHT_API_FILE_PARSER`) if not specified. - Only parsers enabled on the server may be requested; others return HTTP 400.
 
         :param bank_id: (required)
         :type bank_id: str
@@ -141,7 +141,7 @@ class FilesApi:
     ) -> ApiResponse[FileRetainResponse]:
         """Convert files to memories
 
-        Upload files (PDF, DOCX, etc.), convert them to markdown, and retain as memories.  This endpoint handles file upload, conversion, and memory creation in a single operation.  **Features:** - Supports PDF, DOCX, PPTX, XLSX, images (with OCR), audio (with transcription) - Automatic file-to-markdown conversion using pluggable parsers - Files stored in object storage (PostgreSQL by default, S3 for production) - Each file becomes a separate document with optional metadata/tags - Always processes asynchronously — returns operation IDs immediately  **The system automatically:** 1. Stores uploaded files in object storage 2. Converts files to markdown 3. Creates document records with file metadata 4. Extracts facts and creates memory units (same as regular retain)  Use the operations endpoint to monitor progress.  **Request format:** multipart/form-data with: - `files`: One or more files to upload - `request`: JSON string with FileRetainRequest model (files_metadata)  **Note:** File parser is configured server-side via `HINDSIGHT_API_FILE_PARSER` (default: markitdown).
+        Upload files (PDF, DOCX, etc.), convert them to markdown, and retain as memories.  This endpoint handles file upload, conversion, and memory creation in a single operation.  **Features:** - Supports PDF, DOCX, PPTX, XLSX, images (with OCR), audio (with transcription) - Automatic file-to-markdown conversion using pluggable parsers - Files stored in object storage (PostgreSQL by default, S3 for production) - Each file becomes a separate document with optional metadata/tags - Always processes asynchronously — returns operation IDs immediately  **The system automatically:** 1. Stores uploaded files in object storage 2. Converts files to markdown 3. Creates document records with file metadata 4. Extracts facts and creates memory units (same as regular retain)  Use the operations endpoint to monitor progress.  **Request format:** multipart/form-data with: - `files`: One or more files to upload - `request`: JSON string with FileRetainRequest model  **Parser selection:** - Set `parser` in the request body to override the server default for all files. - Set `parser` inside a `files_metadata` entry for per-file control. - Pass a list (e.g. `['iris', 'markitdown']`) to define an ordered fallback chain — each parser is tried in sequence until one succeeds. - Falls back to the server default (`HINDSIGHT_API_FILE_PARSER`) if not specified. - Only parsers enabled on the server may be requested; others return HTTP 400.
 
         :param bank_id: (required)
         :type bank_id: str
@@ -221,7 +221,7 @@ class FilesApi:
     ) -> RESTResponseType:
         """Convert files to memories
 
-        Upload files (PDF, DOCX, etc.), convert them to markdown, and retain as memories.  This endpoint handles file upload, conversion, and memory creation in a single operation.  **Features:** - Supports PDF, DOCX, PPTX, XLSX, images (with OCR), audio (with transcription) - Automatic file-to-markdown conversion using pluggable parsers - Files stored in object storage (PostgreSQL by default, S3 for production) - Each file becomes a separate document with optional metadata/tags - Always processes asynchronously — returns operation IDs immediately  **The system automatically:** 1. Stores uploaded files in object storage 2. Converts files to markdown 3. Creates document records with file metadata 4. Extracts facts and creates memory units (same as regular retain)  Use the operations endpoint to monitor progress.  **Request format:** multipart/form-data with: - `files`: One or more files to upload - `request`: JSON string with FileRetainRequest model (files_metadata)  **Note:** File parser is configured server-side via `HINDSIGHT_API_FILE_PARSER` (default: markitdown).
+        Upload files (PDF, DOCX, etc.), convert them to markdown, and retain as memories.  This endpoint handles file upload, conversion, and memory creation in a single operation.  **Features:** - Supports PDF, DOCX, PPTX, XLSX, images (with OCR), audio (with transcription) - Automatic file-to-markdown conversion using pluggable parsers - Files stored in object storage (PostgreSQL by default, S3 for production) - Each file becomes a separate document with optional metadata/tags - Always processes asynchronously — returns operation IDs immediately  **The system automatically:** 1. Stores uploaded files in object storage 2. Converts files to markdown 3. Creates document records with file metadata 4. Extracts facts and creates memory units (same as regular retain)  Use the operations endpoint to monitor progress.  **Request format:** multipart/form-data with: - `files`: One or more files to upload - `request`: JSON string with FileRetainRequest model  **Parser selection:** - Set `parser` in the request body to override the server default for all files. - Set `parser` inside a `files_metadata` entry for per-file control. - Pass a list (e.g. `['iris', 'markitdown']`) to define an ordered fallback chain — each parser is tried in sequence until one succeeds. - Falls back to the server default (`HINDSIGHT_API_FILE_PARSER`) if not specified. - Only parsers enabled on the server may be requested; others return HTTP 400.
 
         :param bank_id: (required)
         :type bank_id: str

hindsight-clients/typescript/generated/sdk.gen.ts

@@ -1096,9 +1096,14 @@ export const retainMemories = <ThrowOnError extends boolean = false>(
  *
  * **Request format:** multipart/form-data with:
  * - `files`: One or more files to upload
- * - `request`: JSON string with FileRetainRequest model (files_metadata)
+ * - `request`: JSON string with FileRetainRequest model
  *
- * **Note:** File parser is configured server-side via `HINDSIGHT_API_FILE_PARSER` (default: markitdown).
+ * **Parser selection:**
+ * - Set `parser` in the request body to override the server default for all files.
+ * - Set `parser` inside a `files_metadata` entry for per-file control.
+ * - Pass a list (e.g. `['iris', 'markitdown']`) to define an ordered fallback chain — each parser is tried in sequence until one succeeds.
+ * - Falls back to the server default (`HINDSIGHT_API_FILE_PARSER`) if not specified.
+ * - Only parsers enabled on the server may be requested; others return HTTP 400.
  */
 export const fileRetain = <ThrowOnError extends boolean = false>(
   options: Options<FileRetainData, ThrowOnError>,

hindsight-docs/docs/developer/configuration.md

@@ -603,11 +603,37 @@ Configuration for the file upload and conversion pipeline (used by `POST /v1/def
 | Variable | Description | Default |
 |----------|-------------|---------|
 | `HINDSIGHT_API_ENABLE_FILE_UPLOAD_API` | Enable the file upload API endpoint | `true` |
-| `HINDSIGHT_API_FILE_PARSER` | File parser to use (`markitdown`, `iris`) | `markitdown` |
+| `HINDSIGHT_API_FILE_PARSER` | Server-side default parser or fallback chain (comma-separated, e.g. `iris,markitdown`) | `markitdown` |
+| `HINDSIGHT_API_FILE_PARSER_ALLOWLIST` | Comma-separated list of parsers clients are allowed to request. If not set, all registered parsers are allowed. | — |
 | `HINDSIGHT_API_FILE_CONVERSION_MAX_BATCH_SIZE` | Max files per upload request | `10` |
 | `HINDSIGHT_API_FILE_CONVERSION_MAX_BATCH_SIZE_MB` | Max total upload size per request (MB) | `100` |
 | `HINDSIGHT_API_FILE_DELETE_AFTER_RETAIN` | Delete stored files after memory extraction completes | `true` |
 
+#### Parser selection
+
+Clients can override the server default by passing `parser` in the request body of `POST /v1/default/banks/{bank_id}/files/retain`. Both the server default and the per-request field accept a single parser name or an ordered **fallback chain** — each parser is tried in sequence until one succeeds.
+
+```bash
+# Server default: try iris first, fall back to markitdown if iris fails
+export HINDSIGHT_API_FILE_PARSER=iris,markitdown
+
+# Restrict what clients may request (optional — defaults to all registered parsers)
+export HINDSIGHT_API_FILE_PARSER_ALLOWLIST=markitdown,iris
+```
+
+```json
+// Per-request override (in the JSON body of the file retain endpoint)
+{
+  "parser": "iris",
+  "files_metadata": [
+    { "document_id": "report" },
+    { "document_id": "fallback_doc", "parser": ["iris", "markitdown"] }
+  ]
+}
+```
+
+Clients that request a parser not in the allowlist receive HTTP 400.
+
 #### Parser: markitdown (default)
 
 Local file-to-markdown conversion using [Microsoft's markitdown](https://github.com/microsoft/markitdown). No external service required.
@@ -626,10 +652,13 @@ Cloud-based extraction via [Vectorize Iris](https://docs.vectorize.io/build-depl
 **Supported formats:** PDF, DOCX, DOC, PPTX, PPT, XLSX, XLS, images (JPG, JPEG, PNG, GIF, BMP, TIFF, WEBP), HTML, TXT, MD, CSV.
 
 ```bash
-# Use iris parser (requires Vectorize account)
+# Use iris as the only parser
 export HINDSIGHT_API_FILE_PARSER=iris
 export HINDSIGHT_API_FILE_PARSER_IRIS_TOKEN=your-vectorize-token
 export HINDSIGHT_API_FILE_PARSER_IRIS_ORG_ID=your-org-id
+
+# Or: try iris first, fall back to markitdown if iris fails or rejects the file type
+export HINDSIGHT_API_FILE_PARSER=iris,markitdown
 ```
 
 ```bash

hindsight-docs/static/openapi.json

@@ -3660,7 +3660,7 @@
           "Files"
         ],
         "summary": "Convert files to memories",
-        "description": "Upload files (PDF, DOCX, etc.), convert them to markdown, and retain as memories.\n\nThis endpoint handles file upload, conversion, and memory creation in a single operation.\n\n**Features:**\n- Supports PDF, DOCX, PPTX, XLSX, images (with OCR), audio (with transcription)\n- Automatic file-to-markdown conversion using pluggable parsers\n- Files stored in object storage (PostgreSQL by default, S3 for production)\n- Each file becomes a separate document with optional metadata/tags\n- Always processes asynchronously \u2014 returns operation IDs immediately\n\n**The system automatically:**\n1. Stores uploaded files in object storage\n2. Converts files to markdown\n3. Creates document records with file metadata\n4. Extracts facts and creates memory units (same as regular retain)\n\nUse the operations endpoint to monitor progress.\n\n**Request format:** multipart/form-data with:\n- `files`: One or more files to upload\n- `request`: JSON string with FileRetainRequest model (files_metadata)\n\n**Note:** File parser is configured server-side via `HINDSIGHT_API_FILE_PARSER` (default: markitdown).",
+        "description": "Upload files (PDF, DOCX, etc.), convert them to markdown, and retain as memories.\n\nThis endpoint handles file upload, conversion, and memory creation in a single operation.\n\n**Features:**\n- Supports PDF, DOCX, PPTX, XLSX, images (with OCR), audio (with transcription)\n- Automatic file-to-markdown conversion using pluggable parsers\n- Files stored in object storage (PostgreSQL by default, S3 for production)\n- Each file becomes a separate document with optional metadata/tags\n- Always processes asynchronously \u2014 returns operation IDs immediately\n\n**The system automatically:**\n1. Stores uploaded files in object storage\n2. Converts files to markdown\n3. Creates document records with file metadata\n4. Extracts facts and creates memory units (same as regular retain)\n\nUse the operations endpoint to monitor progress.\n\n**Request format:** multipart/form-data with:\n- `files`: One or more files to upload\n- `request`: JSON string with FileRetainRequest model\n\n**Parser selection:**\n- Set `parser` in the request body to override the server default for all files.\n- Set `parser` inside a `files_metadata` entry for per-file control.\n- Pass a list (e.g. `['iris', 'markitdown']`) to define an ordered fallback chain \u2014 each parser is tried in sequence until one succeeds.\n- Falls back to the server default (`HINDSIGHT_API_FILE_PARSER`) if not specified.\n- Only parsers enabled on the server may be requested; others return HTTP 400.",
         "operationId": "file_retain",
         "parameters": [
           {