abbf874d84
* doc: update cookbook
* fix(cookbook): preserve tag keys during sync, strip local .md links
- Fix extract_tags_from_readme/notebook to return dict[str,str] preserving
sdk/topic keys instead of bare values, preventing topics like
"Customer Service" from being misclassified as SDK
- Add strip_local_md_links() to remove relative .md references that
would cause broken link errors in Docusaurus build
* ci: run test-doc-examples independently without waiting for test-rust-cli
Build the CLI directly in the job instead of downloading the artifact,
so test-doc-examples can start at the beginning in parallel with all other jobs.
* feat: webhook system with task-owned retry, retain.completed event, and UI
- New webhook system: register per-bank webhooks with HMAC signing, configurable
HTTP method/timeout/headers/params (http_config JSONB), and PATCH support
- Webhook deliveries run as async_operations (webhook_delivery type) with
task-owned retry via RetryTaskAt exception and exponential backoff
(60s / 5m / 30m / 2h / 8h, max 6 attempts)
- New retain.completed event fires per-document for both sync and async retain
- Delivery debug info (status code, response body) stored in result_metadata
- Control plane UI: webhooks tab per bank with create/edit/delete and a
deliveries table with cursor pagination and expandable response details
- 28 webhook tests covering HMAC signing, delivery retries, CRUD endpoints,
PATCH update, and retain.completed queuing
- Docs page at developer/api/webhooks documenting event payloads and delivery
- OpenAPI spec and all client SDKs (Python, TypeScript, Rust, Go) regenerated
* fix: update tests for task-owned retry model and guard _webhook_manager attribute
- test_worker.py: test_executor_exception_triggers_retry now raises RetryTaskAt
(plain exceptions are immediate failures in the new system); rename
test_executor_exception_marks_failed_after_max_retries to
test_executor_exception_marks_failed_immediately to reflect new semantics
- test_batch_api.py: remove max_retries kwarg from WorkerPoller constructor
- memory_engine.py: use getattr for _webhook_manager in _fire_retain_webhook
to avoid AttributeError when engine is created without __init__ (tests)
* fix: remove max_retries from benchmark WorkerPoller call
* fix(webhooks): transactional outbox, observations_deleted tracking, sidebar
- Queue webhook delivery rows atomically with the primary operation using the
transactional outbox pattern — prevents lost events on process crash:
- Retain (sync + async): outbox_callback passed into orchestrator.retain_batch
and called inside the DB transaction, replacing the post-commit fire call
- Consolidation: new _mark_operation_completed_and_fire_webhook combines the
status UPDATE and webhook INSERT in one transaction
- Added fire_event_with_conn() to WebhookManager for in-connection delivery
- Track observations_deleted count in consolidation stats and expose it in the
consolidation.completed webhook payload (was always None)
- Add Webhooks page to docs sidebar
- Document at-least-once delivery guarantee with operation_id dedup guidance
* fix(ui): add retain.completed to available webhook event types
* feat(ui): add delete confirmation dialog for webhooks
* fix(webhooks): include operation_id in task_payload so delivery is marked completed
The task_payload JSON was missing the operation_id field, causing execute_task
to see operation_id=None and skip _mark_operation_completed — leaving every
delivery row stuck in 'pending' forever.
Added a test that inserts a real async_operations row and verifies the status
transitions to 'completed' after a successful execute_task call.
* style: fix prettier formatting in webhooks-view
51 lines
1.5 KiB
Python
51 lines
1.5 KiB
Python
"""Pydantic models for the webhook system."""
|
|
|
|
from datetime import datetime
|
|
from enum import StrEnum
|
|
|
|
from pydantic import BaseModel, Field
|
|
|
|
|
|
class WebhookEventType(StrEnum):
|
|
CONSOLIDATION_COMPLETED = "consolidation.completed"
|
|
RETAIN_COMPLETED = "retain.completed"
|
|
|
|
|
|
class ConsolidationEventData(BaseModel):
|
|
observations_created: int | None = None
|
|
observations_updated: int | None = None
|
|
observations_deleted: int | None = None
|
|
error_message: str | None = None
|
|
|
|
|
|
class RetainEventData(BaseModel):
|
|
document_id: str | None = None
|
|
tags: list[str] | None = None
|
|
|
|
|
|
class WebhookEvent(BaseModel):
|
|
event: WebhookEventType
|
|
bank_id: str
|
|
operation_id: str
|
|
status: str # "completed" or "failed"
|
|
timestamp: datetime
|
|
data: ConsolidationEventData | RetainEventData
|
|
|
|
|
|
class WebhookHttpConfig(BaseModel):
|
|
"""HTTP delivery configuration for a webhook."""
|
|
|
|
method: str = Field(default="POST", description="HTTP method: GET or POST")
|
|
timeout_seconds: int = Field(default=30, description="HTTP request timeout in seconds")
|
|
headers: dict[str, str] = Field(default_factory=dict, description="Custom HTTP headers")
|
|
params: dict[str, str] = Field(default_factory=dict, description="Custom HTTP query parameters")
|
|
|
|
|
|
class WebhookConfig(BaseModel):
|
|
id: str
|
|
bank_id: str | None
|
|
url: str
|
|
secret: str | None
|
|
event_types: list[str]
|
|
enabled: bool
|
|
http_config: WebhookHttpConfig = Field(default_factory=WebhookHttpConfig)
|