---
sidebar_position: 2
---

# Ingest Data

Store documents, conversations, and raw content into Hindsight to automatically extract and create memories.

When you **retain** content, Hindsight doesn't just store the raw text—it intelligently analyzes the content to extract meaningful facts, identify entities, and build a connected knowledge graph. This process transforms unstructured information into structured, queryable memories.

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CodeSnippet from '@site/src/components/CodeSnippet';

{/* Import raw source files */}
import retainPy from '!!raw-loader!@site/examples/api/retain.py';
import retainMjs from '!!raw-loader!@site/examples/api/retain.mjs';
import retainSh from '!!raw-loader!@site/examples/api/retain.sh';

:::info How Retain Works
Learn about fact extraction, entity resolution, and graph construction in the [Retain Architecture](/developer/retain) guide.
:::

:::tip Prerequisites
Make sure you've completed the [Quick Start](./quickstart) to install the client and start the server.
:::

## Store a Single Memory

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={retainPy} section="retain-basic" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={retainMjs} section="retain-basic" language="javascript" />
</TabItem>
<TabItem value="cli" label="CLI">
<CodeSnippet code={retainSh} section="retain-basic" language="bash" />
</TabItem>
</Tabs>

## The Importance of Context

The `context` parameter is crucial for guiding how Hindsight extracts memories from your content. Think of it as providing a lens through which the system interprets the information.

**Why context matters:**
- **Steers memory extraction**: Context tells the memory bank what type of information to focus on and how to interpret ambiguous content
- **Improves relevance**: Memories extracted with proper context are more accurately categorized and easier to retrieve
- **Disambiguates meaning**: The same sentence can have different implications depending on context (e.g., "the project was terminated" means different things in a career vs. product context)

## Store with Context and Date

Always provide context and event dates for optimal memory extraction:

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={retainPy} section="retain-with-context" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={retainMjs} section="retain-with-context" language="javascript" />
</TabItem>
<TabItem value="cli" label="CLI">
<CodeSnippet code={retainSh} section="retain-with-context" language="bash" />
</TabItem>
</Tabs>

The `timestamp` defaults to the current time if not specified. Providing explicit timestamps enables temporal queries like "What happened last spring?"

### Response Fields

The retain response includes:

| Field | Type | Description |
|-------|------|-------------|
| `success` | bool | Whether the operation succeeded |
| `bank_id` | string | The memory bank ID |
| `items_count` | int | Number of items processed |
| `async` | bool | Whether processed asynchronously |
| `usage` | TokenUsage | Token usage metrics for LLM calls (synchronous only) |

The `usage` field contains token metrics for cost tracking:
- `input_tokens`: Tokens consumed by prompts
- `output_tokens`: Tokens generated by the LLM
- `total_tokens`: Sum of input and output tokens

Note: `usage` is only present for synchronous operations. Async operations (`async: true`) do not return usage metrics.

## Batch Ingestion

Store multiple items in a single request. **Batch ingestion is the recommended approach** as it significantly improves performance by reducing network overhead and allowing Hindsight to optimize the memory extraction process across related content.

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={retainPy} section="retain-batch" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={retainMjs} section="retain-batch" language="javascript" />
</TabItem>
</Tabs>

The `document_id` groups related memories for later management.

## Store from Files

<Tabs>
<TabItem value="cli" label="CLI">

```bash
# Single file
hindsight memory retain-files my-bank document.txt

# Directory (recursive by default)
hindsight memory retain-files my-bank ./documents/
```

</TabItem>
</Tabs>


## Async Ingestion

For large batches, use async ingestion to avoid blocking:

<Tabs>
<TabItem value="python" label="Python">
<CodeSnippet code={retainPy} section="retain-async" language="python" />
</TabItem>
<TabItem value="node" label="Node.js">
<CodeSnippet code={retainMjs} section="retain-async" language="javascript" />
</TabItem>
</Tabs>