Commit graph

2 commits

Author SHA1 Message Date
Chris Bartholomew
d2504ac5ed
Fix GCS auth for Workload Identity Federation credentials (#518)
* Fix GCS auth for external_account credentials (Workload Identity)

obstore's built-in credential parsing only supports service_account and
authorized_user JSON types. Use google.auth as a credential_provider
callback to support all credential types including external_account
(Workload Identity Federation), impersonated credentials, and metadata
server credentials.

* Hide GOOGLE_APPLICATION_CREDENTIALS during GCSStore construction

GCSStore eagerly parses the credential file from env vars even when a
custom credential_provider is passed. Temporarily unset the env var
during construction so obstore doesn't choke on external_account
credential files (Workload Identity Federation).

* Support HINDSIGHT_GOOGLE_CREDENTIALS_FILE for GCS auth

When GOOGLE_APPLICATION_CREDENTIALS must be unset to prevent obstore
from parsing unsupported credential types (e.g. external_account),
google.auth can load credentials from HINDSIGHT_GOOGLE_CREDENTIALS_FILE
instead. This avoids mutating env vars at runtime.

* Simplify GCS credential workaround: hide env var during construction

Remove HINDSIGHT_GOOGLE_CREDENTIALS_FILE indirection. Instead, let
google.auth.default() load credentials normally via GOOGLE_APPLICATION_CREDENTIALS,
then temporarily hide the env var during GCSStore() construction so obstore
doesn't try to parse credential types it doesn't support.

* Work around obstore bug: hide env var during GCSStore construction

obstore always parses credential files from GOOGLE_APPLICATION_CREDENTIALS
and the well-known ADC path, even when credential_provider is supplied
(contrary to docs). This crashes on external_account credentials from
Workload Identity Federation.

Temporarily hide the env var during GCSStore() construction. google.auth
has already loaded credentials by this point via credential_provider.
2026-03-07 08:59:51 +01:00
Nicolò Boschi
224b7b74c1
feat: accept pdf, images and office files (#390)
* feat: accept pdf, images and office files

* refactor: rename FileConverter to FileParser, simplify file retain API

- Rename engine/converters/ → engine/parsers/, FileConverter → FileParser,
  ConverterRegistry → FileParserRegistry, MarkitdownConverter → MarkitdownParser
- Rename env var HINDSIGHT_API_FILE_CONVERTER → HINDSIGHT_API_FILE_PARSER
- Remove async/document_tags params from FileRetainRequest (always async now)
- Add retain_files() to Python Hindsight client and retainFiles() to TypeScript client
- Add sample.pdf to doc examples for working file upload demonstrations
- Update test_file_retain.py to use new parser names and always-async behavior
- Fix Go client missing os import in api_files.go
- Simplify postgresql.py storage to minimal schema

* fix: update rust CLI tests to use is_supported_file instead of is_text_file

* fix: patch Go api_files.go to add missing 'os' import after generation

* fix: insert 'os' import after 'net/url' in api_files.go patch for correct position

* chore: regenerate OpenAPI spec and clients (converter→parser description update)
2026-02-17 18:15:03 +01:00