aefb3fcf4d
* fix: improve async batch retain with large payloads * fix: improve async batch retain with large payloads * api * api * api * api * api * Clean up perf benchmark: keep only Python files - Remove README.md and PERFORMANCE_FINDINGS.md - Remove results/ JSON files (gitignored) - Remove test_data/ directory - Keep only __init__.py and retain_perf.py * docs: explain automatic batch optimization for async retain - Add section explaining Hindsight automatically handles batch sizing - Users don't need to manually tune batch sizes with async mode - Hindsight splits large batches (>10k tokens) into optimized sub-batches - Include example showing best practices * docs: remove emojis and code example from performance page * fix: correct OperationDetails type to match API response - Change optional fields to use | null instead of ? - Fixes TypeScript compilation error in control plane build * fix: use discriminated union for OperationDetails type - Support both success and error states properly - Fixes TypeScript error when setting error state * fix: use unique document_ids in batch retain examples - Each item in a batch must have unique document_id - Update both Python and JavaScript examples - Fixes test-doc-examples CI failure * chore: trigger CI * fix: test mocking and duplicate document_ids in examples - Mock _get_pool() in test_async_retain_tags.py to avoid _initialized error - Set _initialized = True on mocked MemoryEngine instances - Fix duplicate document_ids in retain.py and retain.mjs examples * fix: properly mock async pool/connection and fix more duplicate document_ids - Use AsyncMock for pool.acquire() to fix 'can't be used in await' error - Fix duplicate document_ids in retain-async examples (retain.py and retain.mjs) - Remove batch-level document_id parameter that caused duplicates * ci: collect all doc example failures and show summary - Run all Python/Node.js/CLI examples regardless of individual failures - Collect failure list and display summary at the end - Show pass/fail count and list of failed files - Exit with failure only after running all examples * refactor: extract doc example testing to standalone script - Create scripts/test-doc-examples.sh to run all examples - Collects logs of failed examples separately - Shows full error logs only for failures at the end - Clean summary with pass/fail counts - Proper exit codes - Replaces inline bash in CI workflow * fix: doc examples - duplicate document_ids and error handling - retain.py: move document_id to item level to avoid duplicates - documents.mjs: add error handling for getDocument to show clear error message * fix: update tests for duplicate document_id validation - test_async_retain_tags: verify operation structure instead of exact UUID - test_delete_bank: use unique document_ids (team-doc-1, team-doc-2)
122 lines
3.6 KiB
Python
122 lines
3.6 KiB
Python
#!/usr/bin/env python3
|
|
"""
|
|
Retain API examples for Hindsight.
|
|
Run: python examples/api/retain.py
|
|
"""
|
|
import os
|
|
import requests
|
|
|
|
HINDSIGHT_URL = os.getenv("HINDSIGHT_API_URL", "http://localhost:8888")
|
|
|
|
# =============================================================================
|
|
# Setup (not shown in docs)
|
|
# =============================================================================
|
|
from hindsight_client import Hindsight
|
|
|
|
client = Hindsight(base_url=HINDSIGHT_URL)
|
|
|
|
# =============================================================================
|
|
# Doc Examples
|
|
# =============================================================================
|
|
|
|
# [docs:retain-basic]
|
|
client.retain(
|
|
bank_id="my-bank",
|
|
content="Alice works at Google as a software engineer"
|
|
)
|
|
# [/docs:retain-basic]
|
|
|
|
|
|
# [docs:retain-with-context]
|
|
client.retain(
|
|
bank_id="my-bank",
|
|
content="Alice got promoted to senior engineer",
|
|
context="career update",
|
|
timestamp="2024-03-15T10:00:00Z"
|
|
)
|
|
# [/docs:retain-with-context]
|
|
|
|
|
|
# [docs:retain-batch]
|
|
client.retain_batch(
|
|
bank_id="my-bank",
|
|
items=[
|
|
{"content": "Alice works at Google", "context": "career", "document_id": "conversation_001_msg_1"},
|
|
{"content": "Bob is a data scientist at Meta", "context": "career", "document_id": "conversation_001_msg_2"},
|
|
{"content": "Alice and Bob are friends", "context": "relationship", "document_id": "conversation_001_msg_3"}
|
|
]
|
|
)
|
|
# [/docs:retain-batch]
|
|
|
|
|
|
# [docs:retain-async]
|
|
# Start async ingestion (returns immediately)
|
|
result = client.retain_batch(
|
|
bank_id="my-bank",
|
|
items=[
|
|
{"content": "Large batch item 1", "document_id": "large-doc-1"},
|
|
{"content": "Large batch item 2", "document_id": "large-doc-2"},
|
|
],
|
|
retain_async=True
|
|
)
|
|
|
|
# Check if it was processed asynchronously
|
|
print(result.var_async) # True
|
|
# [/docs:retain-async]
|
|
|
|
|
|
# [docs:retain-with-tags]
|
|
# Tag individual items for visibility scoping
|
|
client.retain_batch(
|
|
bank_id="my-bank",
|
|
items=[
|
|
{
|
|
"content": "User Alice said she loves the new dashboard",
|
|
"tags": ["user:alice", "feedback"],
|
|
"document_id": "user_feedback_001"
|
|
},
|
|
{
|
|
"content": "User Bob reported a bug in the search feature",
|
|
"tags": ["user:bob", "bug-report"],
|
|
"document_id": "user_feedback_002"
|
|
}
|
|
]
|
|
)
|
|
# [/docs:retain-with-tags]
|
|
|
|
|
|
# [docs:retain-with-document-tags]
|
|
# Apply tags to all items in a batch
|
|
client.retain_batch(
|
|
bank_id="my-bank",
|
|
items=[
|
|
{"content": "Alice mentioned she prefers dark mode"},
|
|
{"content": "Bob asked about keyboard shortcuts"}
|
|
],
|
|
document_id="support_session_123",
|
|
document_tags=["session:123", "support"] # Applied to all items
|
|
)
|
|
# [/docs:retain-with-document-tags]
|
|
|
|
|
|
# [docs:retain-list-tags]
|
|
# List all tags in a bank
|
|
response = requests.get(f"{HINDSIGHT_URL}/v1/default/banks/my-bank/tags")
|
|
tags = response.json()
|
|
for tag in tags["items"]:
|
|
print(f"{tag['tag']}: {tag['count']} memories")
|
|
|
|
# Search with wildcards (* matches any characters)
|
|
response = requests.get(f"{HINDSIGHT_URL}/v1/default/banks/my-bank/tags", params={"q": "user:*"})
|
|
user_tags = response.json()
|
|
response = requests.get(f"{HINDSIGHT_URL}/v1/default/banks/my-bank/tags", params={"q": "*-admin"})
|
|
admin_tags = response.json()
|
|
# [/docs:retain-list-tags]
|
|
|
|
|
|
# =============================================================================
|
|
# Cleanup (not shown in docs)
|
|
# =============================================================================
|
|
requests.delete(f"{HINDSIGHT_URL}/v1/default/banks/my-bank")
|
|
|
|
print("retain.py: All examples passed")
|