29a542dc23
* feat: Record LLM token metrics via Prometheus Wire up the existing token metrics infrastructure to actually record token usage from LLM calls. The MetricsCollector already had record_tokens() method and Prometheus counters (hindsight.tokens.input, hindsight.tokens.output), but they were never being populated. Changes: - Import get_metrics_collector in llm_wrapper.py - Call record_tokens() after successful LLM calls for: - OpenAI/Groq (using response.usage.prompt_tokens, completion_tokens) - Anthropic (using response.usage.input_tokens, output_tokens) - Gemini (using response.usage_metadata.prompt_token_count, candidates_token_count) - Add test file to verify token metrics are recorded Note: Ollama's native API doesn't return token usage, so metrics are not recorded for that provider. The token metrics will now be available via /metrics endpoint: - hindsight_tokens_input_total - hindsight_tokens_output_total * feat: add per-request token usage tracking to retain and reflect endpoints - Add TokenUsage model with input_tokens, output_tokens, total_tokens - Return usage metrics in retain response (sync operations only) - Return usage metrics in reflect response - Update Python, TypeScript, and Rust clients - Add API documentation for usage fields - Add changelog entry
131 lines
4.7 KiB
Text
131 lines
4.7 KiB
Text
---
|
|
sidebar_position: 2
|
|
---
|
|
|
|
# Ingest Data
|
|
|
|
Store documents, conversations, and raw content into Hindsight to automatically extract and create memories.
|
|
|
|
When you **retain** content, Hindsight doesn't just store the raw text—it intelligently analyzes the content to extract meaningful facts, identify entities, and build a connected knowledge graph. This process transforms unstructured information into structured, queryable memories.
|
|
|
|
import Tabs from '@theme/Tabs';
|
|
import TabItem from '@theme/TabItem';
|
|
import CodeSnippet from '@site/src/components/CodeSnippet';
|
|
|
|
{/* Import raw source files */}
|
|
import retainPy from '!!raw-loader!@site/examples/api/retain.py';
|
|
import retainMjs from '!!raw-loader!@site/examples/api/retain.mjs';
|
|
import retainSh from '!!raw-loader!@site/examples/api/retain.sh';
|
|
|
|
:::info How Retain Works
|
|
Learn about fact extraction, entity resolution, and graph construction in the [Retain Architecture](/developer/retain) guide.
|
|
:::
|
|
|
|
:::tip Prerequisites
|
|
Make sure you've completed the [Quick Start](./quickstart) to install the client and start the server.
|
|
:::
|
|
|
|
## Store a Single Memory
|
|
|
|
<Tabs>
|
|
<TabItem value="python" label="Python">
|
|
<CodeSnippet code={retainPy} section="retain-basic" language="python" />
|
|
</TabItem>
|
|
<TabItem value="node" label="Node.js">
|
|
<CodeSnippet code={retainMjs} section="retain-basic" language="javascript" />
|
|
</TabItem>
|
|
<TabItem value="cli" label="CLI">
|
|
<CodeSnippet code={retainSh} section="retain-basic" language="bash" />
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
## The Importance of Context
|
|
|
|
The `context` parameter is crucial for guiding how Hindsight extracts memories from your content. Think of it as providing a lens through which the system interprets the information.
|
|
|
|
**Why context matters:**
|
|
- **Steers memory extraction**: Context tells the memory bank what type of information to focus on and how to interpret ambiguous content
|
|
- **Improves relevance**: Memories extracted with proper context are more accurately categorized and easier to retrieve
|
|
- **Disambiguates meaning**: The same sentence can have different implications depending on context (e.g., "the project was terminated" means different things in a career vs. product context)
|
|
|
|
## Store with Context and Date
|
|
|
|
Always provide context and event dates for optimal memory extraction:
|
|
|
|
<Tabs>
|
|
<TabItem value="python" label="Python">
|
|
<CodeSnippet code={retainPy} section="retain-with-context" language="python" />
|
|
</TabItem>
|
|
<TabItem value="node" label="Node.js">
|
|
<CodeSnippet code={retainMjs} section="retain-with-context" language="javascript" />
|
|
</TabItem>
|
|
<TabItem value="cli" label="CLI">
|
|
<CodeSnippet code={retainSh} section="retain-with-context" language="bash" />
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
The `timestamp` defaults to the current time if not specified. Providing explicit timestamps enables temporal queries like "What happened last spring?"
|
|
|
|
### Response Fields
|
|
|
|
The retain response includes:
|
|
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `success` | bool | Whether the operation succeeded |
|
|
| `bank_id` | string | The memory bank ID |
|
|
| `items_count` | int | Number of items processed |
|
|
| `async` | bool | Whether processed asynchronously |
|
|
| `usage` | TokenUsage | Token usage metrics for LLM calls (synchronous only) |
|
|
|
|
The `usage` field contains token metrics for cost tracking:
|
|
- `input_tokens`: Tokens consumed by prompts
|
|
- `output_tokens`: Tokens generated by the LLM
|
|
- `total_tokens`: Sum of input and output tokens
|
|
|
|
Note: `usage` is only present for synchronous operations. Async operations (`async: true`) do not return usage metrics.
|
|
|
|
## Batch Ingestion
|
|
|
|
Store multiple items in a single request. **Batch ingestion is the recommended approach** as it significantly improves performance by reducing network overhead and allowing Hindsight to optimize the memory extraction process across related content.
|
|
|
|
<Tabs>
|
|
<TabItem value="python" label="Python">
|
|
<CodeSnippet code={retainPy} section="retain-batch" language="python" />
|
|
</TabItem>
|
|
<TabItem value="node" label="Node.js">
|
|
<CodeSnippet code={retainMjs} section="retain-batch" language="javascript" />
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
The `document_id` groups related memories for later management.
|
|
|
|
## Store from Files
|
|
|
|
<Tabs>
|
|
<TabItem value="cli" label="CLI">
|
|
|
|
```bash
|
|
# Single file
|
|
hindsight memory retain-files my-bank document.txt
|
|
|
|
# Directory (recursive by default)
|
|
hindsight memory retain-files my-bank ./documents/
|
|
```
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
|
|
## Async Ingestion
|
|
|
|
For large batches, use async ingestion to avoid blocking:
|
|
|
|
<Tabs>
|
|
<TabItem value="python" label="Python">
|
|
<CodeSnippet code={retainPy} section="retain-async" language="python" />
|
|
</TabItem>
|
|
<TabItem value="node" label="Node.js">
|
|
<CodeSnippet code={retainMjs} section="retain-async" language="javascript" />
|
|
</TabItem>
|
|
</Tabs>
|