abbf874d84
* doc: update cookbook
* fix(cookbook): preserve tag keys during sync, strip local .md links
- Fix extract_tags_from_readme/notebook to return dict[str,str] preserving
sdk/topic keys instead of bare values, preventing topics like
"Customer Service" from being misclassified as SDK
- Add strip_local_md_links() to remove relative .md references that
would cause broken link errors in Docusaurus build
* ci: run test-doc-examples independently without waiting for test-rust-cli
Build the CLI directly in the job instead of downloading the artifact,
so test-doc-examples can start at the beginning in parallel with all other jobs.
* feat: webhook system with task-owned retry, retain.completed event, and UI
- New webhook system: register per-bank webhooks with HMAC signing, configurable
HTTP method/timeout/headers/params (http_config JSONB), and PATCH support
- Webhook deliveries run as async_operations (webhook_delivery type) with
task-owned retry via RetryTaskAt exception and exponential backoff
(60s / 5m / 30m / 2h / 8h, max 6 attempts)
- New retain.completed event fires per-document for both sync and async retain
- Delivery debug info (status code, response body) stored in result_metadata
- Control plane UI: webhooks tab per bank with create/edit/delete and a
deliveries table with cursor pagination and expandable response details
- 28 webhook tests covering HMAC signing, delivery retries, CRUD endpoints,
PATCH update, and retain.completed queuing
- Docs page at developer/api/webhooks documenting event payloads and delivery
- OpenAPI spec and all client SDKs (Python, TypeScript, Rust, Go) regenerated
* fix: update tests for task-owned retry model and guard _webhook_manager attribute
- test_worker.py: test_executor_exception_triggers_retry now raises RetryTaskAt
(plain exceptions are immediate failures in the new system); rename
test_executor_exception_marks_failed_after_max_retries to
test_executor_exception_marks_failed_immediately to reflect new semantics
- test_batch_api.py: remove max_retries kwarg from WorkerPoller constructor
- memory_engine.py: use getattr for _webhook_manager in _fire_retain_webhook
to avoid AttributeError when engine is created without __init__ (tests)
* fix: remove max_retries from benchmark WorkerPoller call
* fix(webhooks): transactional outbox, observations_deleted tracking, sidebar
- Queue webhook delivery rows atomically with the primary operation using the
transactional outbox pattern — prevents lost events on process crash:
- Retain (sync + async): outbox_callback passed into orchestrator.retain_batch
and called inside the DB transaction, replacing the post-commit fire call
- Consolidation: new _mark_operation_completed_and_fire_webhook combines the
status UPDATE and webhook INSERT in one transaction
- Added fire_event_with_conn() to WebhookManager for in-connection delivery
- Track observations_deleted count in consolidation stats and expose it in the
consolidation.completed webhook payload (was always None)
- Add Webhooks page to docs sidebar
- Document at-least-once delivery guarantee with operation_id dedup guidance
* fix(ui): add retain.completed to available webhook event types
* feat(ui): add delete confirmation dialog for webhooks
* fix(webhooks): include operation_id in task_payload so delivery is marked completed
The task_payload JSON was missing the operation_id field, causing execute_task
to see operation_id=None and skip _mark_operation_completed — leaving every
delivery row stuck in 'pending' forever.
Added a test that inserts a real async_operations row and verifies the status
transitions to 'completed' after a successful execute_task call.
* style: fix prettier formatting in webhooks-view
96 lines
3.1 KiB
Text
96 lines
3.1 KiB
Text
---
|
|
sidebar_position: 10
|
|
---
|
|
|
|
# Webhooks
|
|
|
|
Hindsight can notify your application in real-time when memory events occur by sending HTTP POST requests to a URL you configure.
|
|
|
|
## Delivery and Retries
|
|
|
|
Webhooks are registered per memory bank and fire automatically when matching events occur. Each delivery attempt is tracked, and failed deliveries are retried with exponential backoff:
|
|
|
|
| Attempt | Delay after failure |
|
|
|---------|---------------------|
|
|
| 1 | 5 seconds |
|
|
| 2 | 5 minutes |
|
|
| 3 | 30 minutes |
|
|
| 4 | 2 hours |
|
|
| 5 | 5 hours |
|
|
| 6 | Permanent failure |
|
|
|
|
A delivery is considered failed if your endpoint returns a non-2xx status code or does not respond within the configured timeout (default 30 seconds). After 6 failed attempts, the delivery is marked as permanently failed and no further retries are made.
|
|
|
|
:::info At-least-once delivery
|
|
Webhook delivery tasks are queued in the same database transaction as the primary operation (e.g. the retain or consolidation write). This means if the server crashes after committing but before sending, the delivery task survives and will be retried. As a result, **your endpoint may receive the same event more than once** — use the `operation_id` field to deduplicate if needed.
|
|
:::
|
|
|
|
## Event Types
|
|
|
|
### `consolidation.completed`
|
|
|
|
Fired after Hindsight finishes consolidating new memories into observations for a bank.
|
|
|
|
**Payload:**
|
|
|
|
```json
|
|
{
|
|
"event": "consolidation.completed",
|
|
"bank_id": "my-bank",
|
|
"operation_id": "a1b2c3d4e5f6",
|
|
"status": "completed",
|
|
"timestamp": "2026-03-04T12:00:00Z",
|
|
"data": {
|
|
"observations_created": 3,
|
|
"observations_updated": 1,
|
|
"observations_deleted": null,
|
|
"error_message": null
|
|
}
|
|
}
|
|
```
|
|
|
|
**`data` fields:**
|
|
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `observations_created` | `integer \| null` | Number of new observations created |
|
|
| `observations_updated` | `integer \| null` | Number of existing observations updated |
|
|
| `observations_deleted` | `integer \| null` | Number of observations deleted |
|
|
| `error_message` | `string \| null` | Set when `status` is `"failed"` |
|
|
|
|
**`status` values:** `"completed"` or `"failed"`
|
|
|
|
---
|
|
|
|
### `retain.completed`
|
|
|
|
Fired once per document after a retain operation completes (both synchronous and asynchronous). When retaining a batch of N documents, N separate events are fired.
|
|
|
|
**Payload:**
|
|
|
|
```json
|
|
{
|
|
"event": "retain.completed",
|
|
"bank_id": "my-bank",
|
|
"operation_id": "a1b2c3d4e5f6",
|
|
"status": "completed",
|
|
"timestamp": "2026-03-04T12:00:01Z",
|
|
"data": {
|
|
"document_id": "doc-abc123",
|
|
"tags": ["meeting", "q1-2026"]
|
|
}
|
|
}
|
|
```
|
|
|
|
**`data` fields:**
|
|
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `document_id` | `string \| null` | The document ID if one was provided in the retain request |
|
|
| `tags` | `string[] \| null` | Document-level tags applied during retain |
|
|
|
|
**Notes:**
|
|
- For async retain (`async: true`), `operation_id` matches the `operation_id` returned by the retain API.
|
|
- For sync retain, `operation_id` is a generated identifier for tracing purposes.
|
|
- One event is fired per content item in the retain request.
|
|
|