aefb3fcf4d
* fix: improve async batch retain with large payloads * fix: improve async batch retain with large payloads * api * api * api * api * api * Clean up perf benchmark: keep only Python files - Remove README.md and PERFORMANCE_FINDINGS.md - Remove results/ JSON files (gitignored) - Remove test_data/ directory - Keep only __init__.py and retain_perf.py * docs: explain automatic batch optimization for async retain - Add section explaining Hindsight automatically handles batch sizing - Users don't need to manually tune batch sizes with async mode - Hindsight splits large batches (>10k tokens) into optimized sub-batches - Include example showing best practices * docs: remove emojis and code example from performance page * fix: correct OperationDetails type to match API response - Change optional fields to use | null instead of ? - Fixes TypeScript compilation error in control plane build * fix: use discriminated union for OperationDetails type - Support both success and error states properly - Fixes TypeScript error when setting error state * fix: use unique document_ids in batch retain examples - Each item in a batch must have unique document_id - Update both Python and JavaScript examples - Fixes test-doc-examples CI failure * chore: trigger CI * fix: test mocking and duplicate document_ids in examples - Mock _get_pool() in test_async_retain_tags.py to avoid _initialized error - Set _initialized = True on mocked MemoryEngine instances - Fix duplicate document_ids in retain.py and retain.mjs examples * fix: properly mock async pool/connection and fix more duplicate document_ids - Use AsyncMock for pool.acquire() to fix 'can't be used in await' error - Fix duplicate document_ids in retain-async examples (retain.py and retain.mjs) - Remove batch-level document_id parameter that caused duplicates * ci: collect all doc example failures and show summary - Run all Python/Node.js/CLI examples regardless of individual failures - Collect failure list and display summary at the end - Show pass/fail count and list of failed files - Exit with failure only after running all examples * refactor: extract doc example testing to standalone script - Create scripts/test-doc-examples.sh to run all examples - Collects logs of failed examples separately - Shows full error logs only for failures at the end - Clean summary with pass/fail counts - Proper exit codes - Replaces inline bash in CI workflow * fix: doc examples - duplicate document_ids and error handling - retain.py: move document_id to item level to avoid duplicates - documents.mjs: add error handling for getDocument to show clear error message * fix: update tests for duplicate document_id validation - test_async_retain_tags: verify operation structure instead of exact UUID - test_delete_bank: use unique document_ids (team-doc-1, team-doc-2)
69 lines
1.7 KiB
Python
69 lines
1.7 KiB
Python
"""
|
|
Typed metadata models for async operations.
|
|
|
|
These dataclasses define the structure of result_metadata for different operation types.
|
|
The metadata is exposed in the API for debugging purposes and may change without notice.
|
|
"""
|
|
|
|
from dataclasses import asdict, dataclass
|
|
from typing import Any
|
|
|
|
|
|
@dataclass
|
|
class BatchRetainParentMetadata:
|
|
"""Metadata for parent batch_retain operations (when split into sub-batches)."""
|
|
|
|
items_count: int
|
|
total_tokens: int
|
|
num_sub_batches: int
|
|
is_parent: bool = True
|
|
|
|
def to_dict(self) -> dict[str, Any]:
|
|
"""Convert to dict for JSON serialization."""
|
|
return asdict(self)
|
|
|
|
|
|
@dataclass
|
|
class BatchRetainChildMetadata:
|
|
"""Metadata for child batch_retain operations (individual sub-batches)."""
|
|
|
|
items_count: int
|
|
parent_operation_id: str
|
|
sub_batch_index: int
|
|
total_sub_batches: int
|
|
|
|
def to_dict(self) -> dict[str, Any]:
|
|
"""Convert to dict for JSON serialization."""
|
|
return asdict(self)
|
|
|
|
|
|
@dataclass
|
|
class RetainMetadata:
|
|
"""Metadata for regular retain operations (non-batched, deprecated async path)."""
|
|
|
|
items_count: int
|
|
|
|
def to_dict(self) -> dict[str, Any]:
|
|
"""Convert to dict for JSON serialization."""
|
|
return asdict(self)
|
|
|
|
|
|
@dataclass
|
|
class ConsolidationMetadata:
|
|
"""Metadata for consolidation operations."""
|
|
|
|
# Currently empty, but structure for future fields
|
|
def to_dict(self) -> dict[str, Any]:
|
|
"""Convert to dict for JSON serialization."""
|
|
return asdict(self)
|
|
|
|
|
|
@dataclass
|
|
class RefreshMentalModelMetadata:
|
|
"""Metadata for mental model refresh operations."""
|
|
|
|
mental_model_id: str
|
|
|
|
def to_dict(self) -> dict[str, Any]:
|
|
"""Convert to dict for JSON serialization."""
|
|
return asdict(self)
|