holetron-lab/fleet-memory
1
0
Fork 0
Code Releases Packages Activity Actions

chunks

Browse source
This commit is contained in:
Nicolò Boschi 2025-11-28 18:15:21 +01:00
parent 18a7dc429a
commit 3e72984cd2
83 changed files with 1020687 additions and 1712110 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

56
.env.example
View file

@ -1,47 +1,15 @@
# =============================================================================
# HINDSIGHT ENVIRONMENT CONFIGURATION
# =============================================================================
# Copy this file to .env and update with your values
# Both services (API and Control Plane) read from this single file
# Hindsight Environment Variables
# Copy this file to .env and fill in your values
# =============================================================================
# API SERVICE (HINDSIGHT_API_*)
# =============================================================================
# LLM Configuration (Required)
HINDSIGHT_API_LLM_API_KEY=your-api-key-here
HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
# Database
# Use "pg0" to start an embedded PostgreSQL instance via pg0
# Or provide a full connection URL for external PostgreSQL
#HINDSIGHT_API_DATABASE_URL=postgresql://hindsight:hindsight_dev@localhost:5432/hindsight
HINDSIGHT_API_DATABASE_URL=pg0
# API Configuration (Optional)
HINDSIGHT_API_HOST=0.0.0.0
HINDSIGHT_API_PORT=8888
HINDSIGHT_API_LOG_LEVEL=info
# pg0 data directory (only used when HINDSIGHT_API_DATABASE_URL=pg0)
# HINDSIGHT_API_PG0_DATA_DIR=/path/to/pg_data
# LLM Provider: "openai", "groq", or "ollama"
HINDSIGHT_API_LLM_PROVIDER=groq
# LLM Model (provider-specific)
HINDSIGHT_API_LLM_MODEL=openai/gpt-oss-20b
# API Key (not needed for ollama)
HINDSIGHT_API_LLM_API_KEY=your_api_key_here
# Optional: Custom base URL (for ollama or custom endpoints)
# HINDSIGHT_API_LLM_BASE_URL=http://localhost:11434/v1
# API Server Configuration (optional)
# HINDSIGHT_API_HOST=0.0.0.0
# HINDSIGHT_API_PORT=8888
HINDSIGHT_API_MCP_ENABLED=true
# =============================================================================
# CONTROL PLANE SERVICE (HINDSIGHT_CP_*)
# =============================================================================
# Dataplane API URL (where the control plane connects to)
HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888
# Control Plane Server Configuration (optional)
# HINDSIGHT_CP_PORT=3000
# HINDSIGHT_CP_HOSTNAME=0.0.0.0
# Database (Optional - uses embedded pg0 by default)
# HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/db

2
.github/workflows/deploy-docs.yml vendored
View file

@ -2,7 +2,7 @@ name: Deploy Docs to GitHub Pages
on:
push:
branches: [main, renaming-pre-launch]
branches: [main]
paths:
- 'hindsight-docs/**'
- '.github/workflows/deploy-docs.yml'

965
HINDSIGHT_PAPER.md Normal file
View file

@ -0,0 +1,965 @@
# Hindsight: A Unified Memory System for AI Agents with Temporal Retrieval and Personality-Driven Reasoning
## Abstract
We present **Hindsight**, a comprehensive memory architecture for conversational AI agents that combines multi-strategy retrieval with personality-driven reasoning to enable both high-recall factual search and consistent, trait-based opinion formation. The system consists of two integrated components: **TEMPR (Temporal Entity Memory Priming Retrieval)** for memory recall, and **CARA (Coherent Adaptive Reasoning Agents)** for personality-aware reflection. TEMPR achieves strong retrieval performance through four parallel search strategies—semantic vector search, BM25 keyword matching, graph-based spreading activation incorporating multiple link types (entity, semantic, temporal, causal), and temporal-aware graph traversal—achieving 73.50% on LoComo and 80.60% on LongMemEval benchmarks, with particularly strong performance on multi-hop reasoning (+15.8% over baseline). CARA builds on TEMPR's four-network architecture (world facts, bank experiences, opinions, and observations) to enable personality-driven reasoning using the Big Five model, allowing agents to form and evolve opinions influenced by configurable traits while maintaining epistemic clarity between objective information and subjective beliefs. A novel observation paradigm automatically synthesizes entity-level summaries from multiple facts, creating structured mental models of people, organizations, and concepts without personality influence. The combination enables AI agents with long-term memory that can both retrieve information accurately and reason consistently with stable character traits.
---
# Part I: Recall - TEMPR (Temporal Entity Memory Priming Retrieval)
## 1. Introduction to Recall
Conversational AI agents face a fundamental challenge: maintaining coherent, context-aware memories across extended interactions. Traditional search systems are optimized for human users with top-k ranking and relevance feedback, but AI agents have fundamentally different requirements: they need to retrieve variable amounts of information based on reasoning complexity while respecting LLM context windows. Existing approaches rely either on vector similarity search, which captures semantic relationships but misses entity-level connections, or on keyword matching, which provides precision but lacks conceptual understanding. Neither approach adequately handles the temporal aspects of memory or entity-based reasoning that enable multi-hop information discovery.
We propose TEMPR, a memory retrieval architecture designed specifically for AI agents that combines established information retrieval techniques—semantic vector search, BM25 keyword matching, spreading activation graph traversal (Anderson 1983), and neural reranking—into a unified system optimized for agent workflows. The key architectural choices are:
1. **Agent-Optimized Interface**: budget and max_tokens parameters instead of traditional top-k ranking
2. **Comprehensive Narrative Fact Extraction with Temporal Ranges**: LLM-powered extraction that creates self-contained narrative facts preserving full conversational context, extracting temporal ranges (occurred_start/end) to distinguish point events from periods
3. **Entity-Aware Graph Structure with Multiple Link Types**: LLM-based entity resolution and linking that connects memories through shared identities, along with temporal, semantic, and causal link types
4. **Four-Way Parallel Retrieval**: Semantic, keyword, graph-based (spreading activation), and temporal range retrieval strategies executed in parallel and fused using RRF (Cormack et al. 2009)
5. **Neural Cross-Encoder Reranking**: Learned query-document relevance with temporal awareness and token budget filtering
This combination of techniques enables agents to discover indirectly related information through graph traversal while maintaining temporal awareness, achieving strong performance on multi-hop reasoning tasks.
### 1.1 Contributions
Our key contributions for the recall system are:
1. **Agent-Optimized Retrieval Interface**: Unlike traditional top-k search optimized for human users, we introduce budget and max_tokens parameters that allow AI agents to dynamically trade off latency for recall based on reasoning complexity and context window constraints
2. **Four-Way Parallel Retrieval**: We combine semantic vector search, BM25 keyword matching, graph-based spreading activation (Anderson 1983), and temporal-aware graph traversal into a unified parallel retrieval pipeline using Reciprocal Rank Fusion (Cormack et al. 2009) and neural cross-encoder reranking. The graph traversal incorporates multiple link types (entity, semantic, temporal, causal) with configurable weighting during activation spreading.
3. **LLM-Based Knowledge Graph Construction with Temporal Ranges**: We leverage open-source LLMs for comprehensive narrative fact extraction, entity recognition, and entity disambiguation. The system extracts temporal ranges (occurred_start, occurred_end) to represent both point events and extended periods, distinguishing when facts occurred from when they were mentioned.
4. **Strong Performance on Multi-Hop Reasoning**: 73.50% on LoComo and 80.60% on LongMemEval, with particularly strong performance on multi-hop queries (+15.8% over Mem0), demonstrating the effectiveness of combining these techniques for discovering indirectly related information in conversational contexts
## 2. Memory Organization
### 2.1 Four Memory Networks
TEMPR organizes memories into four distinct networks for epistemic clarity:
**World Network** (fact_type='world'): Objective information about the world
- Example: "Alice works at Google in Mountain View on the AI team"
- Stores facts received from external sources
- No confidence scores (facts are information received, not beliefs)
**Bank Network** (fact_type='bank'): Biographical information about the agent itself
- Example: "I recommended Yosemite National Park to Alice for hiking"
- Stores the agent's own actions and experiences
- Uses first-person perspective ("I recommended..." not "The agent recommended...")
**Opinion Network** (fact_type='opinion'): Subjective beliefs formed by the agent
- Example: "Python is better for data science because of libraries like pandas (confidence: 0.85)"
- Stores judgments and opinions with confidence scores
- Evolved through opinion reinforcement when new evidence arrives
- Influenced by personality traits (see Part II: Reflect)
**Observation Network** (fact_type='observation'): Synthesized entity summaries
- Example: "Alice is a software engineer at Google specializing in machine learning"
- Objective syntheses from multiple facts about an entity
- Generated WITHOUT personality influence (unlike opinions)
- Automatically created and updated in background processes
- Provides structured "mental models" of entities
This separation provides:
- **Epistemic Clarity**: Facts represent information encountered; opinions represent personality-driven judgments; observations represent objective syntheses
- **Traceability**: Opinion reinforcement traces facts; observations trace entity-related facts
- **Debugging**: Developers can separately inspect factual knowledge, formed beliefs, and entity models
- **Confidence Semantics**: Facts and observations lack confidence scores; opinions have confidence scores representing conviction strength
- **Personality Independence**: Observations remain objective while opinions reflect personality
### 2.2 Memory Unit Structure
Each memory is represented as a self-contained node with:
- id: Unique UUID
- bank_id: Identifier for the memory bank this memory belongs to
- text: Self-contained comprehensive narrative fact
- embedding: 384-dimensional vector (BAAI/bge-small-en-v1.5)
- event_date: Timestamp when the fact became true (maintained for backward compatibility)
- occurred_start: Timestamp when the fact/event started (temporal range support)
- occurred_end: Timestamp when the fact/event ended (temporal range support)
- mentioned_at: Timestamp when the fact was mentioned/learned
- context: Optional contextual metadata
- fact_type: One of 'world', 'bank', 'opinion'
- confidence_score: For opinions only, strength of conviction (0.0-1.0)
- access_count: Frequency-based importance signal
- search_vector: Full-text search tsvector for BM25 ranking
### 2.3 LLM-Powered Comprehensive Narrative Fact Extraction
TEMPR employs **LLM-powered comprehensive narrative fact extraction** using open-source models. This approach provides more context-aware extraction compared to traditional rule-based NLP pipelines, though at higher computational cost.
#### 2.3.1 Extraction Principles
**Chunking Strategy**: TEMPR uses a coarse-grained chunking approach, extracting 2-5 comprehensive facts per conversation rather than dozens of atomic fragments. This is a deliberate tradeoff: larger chunks preserve more context and narrative flow, at the cost of reduced precision when only a small portion of the chunk is relevant.
Each fact should:
1. **Capture entire conversations or exchanges** - Include the full back-and-forth discussion
2. **Be narrative and comprehensive** - Tell the complete story with all context
3. **Be self-contained** - Readable without the original text
4. **Include all participants** - WHO said/did WHAT, with their reasoning
5. **Preserve the flow** - Keep related exchanges together in one fact
**Example Comparison**:
❌ **Fragmented Approach** (traditional):
- "Bob suggested Summer Vibes"
- "Alice wanted something unique"
- "They considered Sunset Sessions"
- "Alice likes Beach Beats"
- "They chose Beach Beats"
✅ **Comprehensive Approach** (TEMPR):
- "Alice and Bob discussed naming their summer party playlist. Bob suggested 'Summer Vibes' because it's catchy and seasonal, but Alice wanted something more unique. Bob then proposed 'Sunset Sessions' and 'Beach Beats', with Alice favoring 'Beach Beats' for its playful and fun tone. They ultimately decided on 'Beach Beats' as the final name."
#### 2.3.2 Open-Source LLM Extraction Pipeline
The extraction process leverages open-source LLMs with structured output (Pydantic schemas). This follows the established practice of using LLMs for information extraction, which has been shown to improve context understanding compared to rule-based NLP pipelines, particularly for:
- Coreference resolution in conversational text
- Domain-specific entity recognition
- Maintaining narrative coherence across multi-turn exchanges
**LLM Extraction Steps**:
1. **Pronoun Resolution**: "She loves hiking" → "Alice loves hiking"
2. **Temporal Normalization**: "last year" → "in 2023" (absolute dates)
3. **Temporal Range Extraction**: Identify when facts occurred vs. when mentioned
- Point events: "on July 14" → occurred_start = occurred_end = 2023-07-14
- Period events: "in February 2023" → occurred_start = 2023-02-01, occurred_end = 2023-02-28
- Vague periods: "lately" → estimated range based on context
- mentioned_at = conversation date (when fact was learned)
4. **Participant Attribution**: Preserve WHO said/did WHAT
5. **Reasoning Preservation**: Include WHY decisions were made
6. **Fact Type Classification**: Determine fact categories (world, bank, opinion)
7. **Entity Extraction**: Identify all entities (PERSON, ORG, LOCATION, PRODUCT, CONCEPT)
**Temporal Augmentation**: Before embedding, facts are augmented with readable temporal information:
- Original: "Alice started working at Google"
- Augmented for embedding: "Alice started working at Google (happened in November 2023)"
This augmentation helps semantic search understand temporal relevance without modifying the stored fact text.
### 2.4 Entity Resolution and Linking
Entity resolution creates strong connections between memories that share common entities, solving the problem where semantically dissimilar facts are related through shared identities.
#### 2.4.1 LLM-Based Entity Recognition
TEMPR uses the same open-source LLM that performs fact extraction to also identify and extract entities during the narrative fact creation process. This unified approach eliminates the brittleness of traditional NER pipelines that struggle with domain-specific entities, novel names, and context-dependent disambiguation.
**Entity Types**:
- PERSON: "Alice", "Bob Chen"
- ORGANIZATION: "Google", "Stanford University"
- LOCATION: "Yosemite National Park", "California"
- PRODUCT: "Python", "pandas library"
- CONCEPT: "machine learning", "remote work"
- OTHER: Miscellaneous proper nouns
#### 2.4.2 LLM-Based Entity Disambiguation
Multiple mentions of entities (e.g., "Alice", "Alice Chen", "Alice C.") must be resolved to a single canonical entity. TEMPR uses the LLM to perform entity disambiguation, analyzing the surrounding context to determine if two entity mentions refer to the same entity. This handles complex cases like:
- Nicknames and formal names ("Bob" vs. "Robert Chen")
- Partial mentions ("Alice" vs. "Alice Chen")
- Context-dependent disambiguation ("Apple the company" vs. "apple the fruit")
The LLM considers multiple signals:
- **Name Similarity**: String similarity using Levenshtein distance
- **Co-occurrence Patterns**: Entities mentioned together frequently are likely distinct
- **Temporal Proximity**: Recent mentions are more likely to refer to the same entity
#### 2.4.3 Entity Link Structure
Each entity creates a link_type='entity' edge between all memories mentioning it:
**Properties**:
- weight=1.0 (constant, no temporal decay)
- entity_id: Reference to resolved canonical entity
- Bidirectional connections between all mentioning memories
**Impact on Retrieval**: Entity links enable graph traversal to discover indirectly related facts:
**Example Query**: "What does Alice do?"
1. **Semantic Match**: "Alice works at Google in Mountain View..." (direct match)
2. **Entity Traversal**: Follow entity links for "Alice" →
- "Alice loves hiking in Yosemite..." (different semantic space)
- "I recommended technical books to Alice" (Bank Network, via "Alice")
3. **Chained Traversal**: Follow "Google" entity →
- "Google's office in Mountain View has excellent amenities"
### 2.5 Link Types and Graph Structure
The memory graph contains four types of edges connecting memory units:
#### 2.5.1 Temporal Links
Temporal links connect memories close in time, enabling temporal reasoning:
**Creation Logic**:
**Properties**:
- Decays linearly with time distance
- Minimum weight 0.3 to maintain some connectivity
- Enables "What happened around the same time?" queries
#### 2.5.2 Semantic Links
Semantic links connect memories with similar meanings:
**Creation Logic**:
**Properties**:
- Uses pgvector HNSW index for efficient nearest-neighbor search
- Higher threshold (0.7) than retrieval (0.3) to avoid over-connection
- Weight equals cosine similarity score
#### 2.5.3 Entity Links
Entity links (described in Section 2.4.3) create the strongest connections:
**Properties**:
- weight=1.0 (constant, never decays)
- Connects all memories mentioning the same resolved entity
- Most reliable traversal path during graph search
#### 2.5.4 Causal Links
Causal links represent identified cause-effect relationships between facts. During fact extraction, the LLM attempts to identify causal relationships between facts extracted from the same conversation. These links are incorporated as one component of the graph retrieval system.
**Causal Relationship Types**:
- causes: This fact directly causes the target fact
- caused_by: This fact was caused by the target fact (inverse of causes)
- enables: This fact enables or allows the target fact to happen
- prevents: This fact prevents or blocks the target fact
**Properties**:
- weight: Strength of causal relationship ∈ [0.0, 1.0] (default 1.0)
- Directional edges (from cause to effect)
- Prioritized during graph traversal with 2x activation boost
**Role in Retrieval**: Causal links provide an additional signal during graph-based retrieval. When present, they allow the system to traverse explanatory relationships in addition to semantic, temporal, and entity-based connections.
**Example**: For a query "Why does Alice spend time in the garden?", the system may find both direct semantic matches ("Alice spends time in the garden to find comfort") and traverse causal links to related facts ("Alice lost her friend Karlie in February 2023").
**Graph Density**: Each memory unit typically has:
- 5-10 temporal links (to nearby memories)
- 3-5 semantic links (to similar content)
- Variable entity links (depending on entity mention frequency)
- 0-3 causal links (when causal relationships are identified)
### 2.6 The Observation Paradigm
A critical challenge in long-term memory systems is maintaining structured, high-level understanding of entities (people, organizations, places, concepts) without re-reading all individual facts each time. Traditional approaches either retrieve all entity-related facts (expensive, noisy) or maintain no entity-level state (losing structured understanding). Hindsight introduces **observations**—automatically synthesized entity summaries that provide structured "mental models" without personality influence.
#### 2.6.1 Motivation and Design
**The Problem**: When a system accumulates dozens of facts about an entity like "Alice," queries about Alice must either:
1. Retrieve all 50+ individual facts (expensive, overwhelming)
2. Rely only on top-k semantic matches (may miss key attributes)
3. Manually maintain entity profiles (doesn't scale, requires human curation)
**The Solution**: Observations provide a fourth fact type that synthesizes multiple facts into coherent, objective entity summaries, automatically maintained as new information arrives.
**Key Properties**:
- **Objective Synthesis**: Generated WITHOUT personality influence (unlike opinions)
- **Entity-Scoped**: Each observation is about a single entity
- **Automatic Maintenance**: Generated in background after fact ingestion
- **Multi-Fact Fusion**: Combines information scattered across multiple facts
- **Response Augmentation**: NOT used for retrieval/search, but returned alongside results when include_entities=True to provide entity context
#### 2.6.2 Observation Generation
Observations are generated through an LLM-powered synthesis process:
**Trigger**: When new facts mentioning an entity are ingested via retain(), a background task is queued to regenerate observations for that entity.
**Process**:
**LLM Prompt Structure**:
**Example Transformation**:
**Input Facts**:
- "Alice works at Google"
- "Alice is a software engineer"
- "Alice specializes in ML and deep learning"
- "Alice joined Google in 2023"
- "Alice is detail-oriented and methodical"
**Generated Observations**:
- "Alice is a software engineer at Google specializing in machine learning and deep learning"
- "Alice joined Google in 2023"
- "Alice is detail-oriented and methodical in her approach"
#### 2.6.3 Storage and Retrieval
**Storage**: Observations are stored as regular memory_units with fact_type='observation':
**Entity Links**: Observations are linked to their entity via the entity_links table, enabling efficient lookup of all observations for an entity.
**Important**: Observations are NOT used during the retrieval/search process itself. They do not participate in the 4-way parallel search (semantic, keyword, graph, temporal). Instead, they are **response augmentations**—additional context returned alongside search results.
**Response Augmentation**: When calling recall() with include_entities=True:
**Response Structure**:
#### 2.6.4 Observations vs. Opinions
A critical distinction separates observations from opinions:
| Dimension | Observations | Opinions |
|-----------|-------------|----------|
| **Influence** | No personality influence | Influenced by Big Five traits |
| **Purpose** | Objective entity summaries | Subjective beliefs and judgments |
| **Confidence** | No confidence score | Confidence score (0.0-1.0) |
| **Generation** | Background synthesis from facts | Formed during reflect() reasoning |
| **Update Mechanism** | Regenerated when entity facts change | Updated via opinion reinforcement |
| **Example** | "Alice is a software engineer at Google" | "Alice is an excellent engineer" |
**Why Both?**: Observations provide factual entity understanding for retrieval contexts, while opinions represent the memory bank's personality-driven beliefs for reasoning contexts. A memory bank can have objective observations about Alice (she works at Google, specializes in ML) AND personality-influenced opinions about Alice (she's a talented engineer, she'd be great for project X).
#### 2.6.5 Background Processing
Observation generation is asynchronous to avoid blocking retain() operations:
**Flow**:
This design ensures low-latency writes while maintaining fresh entity summaries.
#### 2.6.6 Benefits and Use Cases
**Benefits**:
1. **Contextual Entity Summaries**: After retrieving facts that mention entities, observations provide synthesized context about those entities without requiring separate queries
2. **Structured Entity Understanding**: Provides coherent mental models of entities as response augmentation
3. **Token Efficiency**: 3-5 observations provide more structured context than retrieving all entity-related facts
4. **Objective Grounding**: When reflecting with personality, observations provide objective entity context
5. **Scalability**: Automatically maintained as facts accumulate, always fresh when needed
6. **Separation of Concerns**: Search focuses on relevant facts through semantic similarity, keyword matching, and graph traversal; observations provide entity context post-retrieval
**Note on Observation Stability**: While observations are regenerated when entity facts change, the core retrieval mechanism remains grounded in the original facts. The four-way parallel search (semantic, keyword, graph, temporal) retrieves facts based on query relevance, semantic co-occurrence, and entity relationships—not based on observations. This ensures that the most relevant factual information is surfaced regardless of how observations may evolve over time.
**Use Cases**:
**Multi-Agent Conversations**: When retrieving facts that mention people, observations provide shared, objective entity context:
**Entity-Centric Queries**: "Tell me about Alice" retrieves facts about Alice, and observations provide synthesized entity summary in the response.
**Contextual Reasoning**: When forming opinions during reflect(), observations provide factual entity grounding alongside retrieved facts.
**Knowledge Graph Interfaces**: Observations can be exposed as structured entity profiles in UIs or APIs via dedicated entity endpoints.
## 3. Retrieval Architecture
Our retrieval pipeline addresses the fundamental challenge of long-term memory: achieving both **high recall** (finding all relevant information) and **high precision** (ranking the most relevant items first).
### 3.1 Four-Way Parallel Retrieval
We execute four complementary retrieval strategies in parallel, each capturing different aspects of relevance:
#### 3.1.1 Semantic Retrieval (Vector Similarity)
**Method**: Cosine similarity between query embedding and memory embeddings
**Index**: pgvector HNSW (Hierarchical Navigable Small World)
**Threshold**: ≥ 0.3 similarity
**Implementation**:
**Advantages**:
- Captures conceptual similarity
- Handles synonyms and paraphrasing
- Language-model understanding of meaning
**Limitations**:
- Misses exact proper nouns if not in training data
- Cannot reason about temporal relationships
- Weak at entity disambiguation
#### 3.1.2 Keyword Retrieval (BM25 Full-Text Search)
**Method**: PostgreSQL full-text search with BM25 ranking (ts_rank_cd)
**Index**: GIN index on to_tsvector('english', text)
**Advantages**:
- High precision for proper nouns and technical terms
- Exact phrase matching
- Fast execution with GIN index
**Limitations**:
- No semantic understanding
- Requires exact or stemmed matches
**Complementarity**: Semantic + Keyword achieves >90% recall: vector search catches concepts, BM25 catches exact names.
#### 3.1.3 Graph Retrieval (Spreading Activation)
**Method**: Activation spreading from semantic entry points through the memory graph, following the spreading activation model of memory (Anderson 1983).
**Algorithm**:
**Decay Mechanism**: Activation decays by 0.8 per hop, limiting spread to ~4-5 hops.
**Link Weighting with Causal Boosting**:
- **Causal links**: Base weight × 2.0 boost (causes/caused_by) or × 1.5 boost (enables/prevents)
- **Entity links**: weight 1.0 (no boost, already strong signal)
- **Semantic links**: weight ∈ [0.7, 1.0] (cosine similarity, no boost)
- **Temporal links**: weight ∈ [0.3, 1.0] (time-based decay, no boost)
**Advantages**:
- Discovers indirectly related facts through graph connectivity
- Leverages entity links to traverse knowledge graph
- Finds context-adjacent memories via temporal links
- Prioritizes explanatory relationships through causal boosting
#### 3.1.4 Temporal Graph Retrieval (Time-Constrained + Spreading)
**Activation Condition**: Only triggered when temporal constraint detected in query
**Temporal Parsing**: Uses google/flan-t5-small (80M parameters) to extract temporal constraints from natural language queries:
- "last spring" → 2024-03-01 to 2024-05-31
- "in June" → 2024-06-01 to 2024-06-30
- "last year" → 2024-01-01 to 2024-12-31
- "between March and May" → 2025-03-01 to 2025-05-31
**Temporal Range Matching**: Facts are matched against time constraints using their temporal range (occurred_start, occurred_end):
**Algorithm**:
### 3.2 Reciprocal Rank Fusion (RRF)
After parallel retrieval, we merge 3-4 ranked lists using Reciprocal Rank Fusion (Cormack et al. 2009):
**Algorithm**:
**Advantages over Score-Based Fusion**:
- **Rank-based**: Position matters more than absolute scores
- **Robust to missing items**: Missing from a list contributes 0, not a penalty
- **Multi-evidence weighting**: Items appearing in multiple lists rank higher
### 3.3 Neural Cross-Encoder Reranking
After RRF fusion, TEMPR applies neural cross-encoder reranking to refine precision:
**Model**: cross-encoder/ms-marco-MiniLM-L-6-v2 (pretrained on MS MARCO passage ranking)
**Algorithm**:
**Advantages**:
- Learns query-document relevance patterns from supervised data
- Considers full query-document interaction
- Temporal awareness through formatted date context
### 3.4 Token Budget Filtering
Final stage applies token budget filtering to limit context window usage:
**Algorithm**:
**Purpose**: Ensures retrieved facts fit within LLM context windows while maximizing information density.
### 3.5 Complete Retrieval Pipeline
**End-to-End Flow**:
## 4. Evaluation
We evaluate TEMPR on two established long-term memory benchmarks: LoComo (Long-term Conversation Memory) and LongMemEval.
### 4.1 LoComo Benchmark
LoComo evaluates conversational memory systems across four dimensions: single-hop queries, multi-hop queries, open-domain queries, and temporal queries.
**Results**:
| Method | Single Hop J ↑ | Multi-Hop J ↑ | Open Domain J ↑ | Temporal J ↑ | Overall |
|--------|---------------|---------------|-----------------|--------------|---------|
| A-Mem* | 39.79 | 18.85 | 54.05 | 31.08 | 48.38 |
| LangMem | 62.23 | 47.92 | 71.12 | 23.43 | 58.10 |
| Zep (Mem0 paper) | 61.70 | 41.35 | 76.60 | 49.31 | 65.99 |
| OpenAI | 63.79 | 42.92 | 62.29 | 21.71 | 52.90 |
| Mem0 | 67.13 | 51.15 | 72.93 | 55.51 | 66.88 |
| Mem0 w/ Graph | 65.71 | 47.19 | 75.71 | 58.13 | 68.44 |
| **TEMPR** | **73.20** | **66.90** | **78.60** | **56.30** | **73.50** |
**Analysis**: TEMPR achieves strong performance across all query types:
- **Single-Hop (+6.1% vs Mem0)**: Superior performance due to comprehensive narrative facts and BM25 keyword matching
- **Multi-Hop (+15.8% vs Mem0)**: Largest improvement, demonstrating effectiveness of graph-based spreading activation
- **Open Domain (+2.9% vs Mem0)**: Strong performance through multi-strategy parallel retrieval
- **Temporal (-1.8% vs Mem0 w/ Graph)**: Competitive temporal reasoning
### 4.2 LongMemEval Benchmark
LongMemEval assesses memory systems across six dimensions:
**Results**:
| Method | Single-Session Preference | Single-Session Assistant | Temporal Reasoning | Multi-Session | Knowledge Update | Single-Session User | Overall |
|--------|--------------------------|-------------------------|-------------------|---------------|-----------------|-------------------|---------|
| Zep gpt-4o-mini | 53.30% | 75.00% | 54.10% | 47.40% | 74.40% | 92.90% | 63.80% |
| Zep gpt-4o | 56.70% | 80.40% | 62.40% | 57.90% | 83.30% | 92.90% | 71.00% |
| **TEMPR** | **83.30%** | **80.40%** | **75.90%** | **75.20%** | **85.90%** | **92.90%** | **80.60%** |
| Mastra gpt-4o | 46.70% | 100.00% | 75.20% | 76.70% | 84.60% | 97.10% | 80.05% |
**Analysis**: TEMPR achieves competitive performance:
- **Single-Session Preference (+26.6% vs Zep gpt-4o)**: Dramatic improvement enabled by comprehensive narrative facts
- **Temporal Reasoning (+13.5% vs Zep gpt-4o)**: Strong performance through dedicated temporal graph retrieval
- **Multi-Session (+17.3% vs Zep gpt-4o)**: Entity-aware graph linking maintains consistency
The 80.60% overall score represents a 9.6 percentage point improvement over Zep gpt-4o (71.00%).
---
# Part II: Reflect - CARA (Coherent Adaptive Reasoning Agents)
## 5. Introduction to Reflect
Conversational AI agents increasingly need to maintain consistent perspectives and form judgments that reflect stable character traits. Current systems either provide purely objective information retrieval without perspective, or generate responses that lack consistency across interactions. Human conversation partners expect agents to have stable viewpoints, preferences, and reasoning styles—characteristics that emerge from personality.
We propose CARA (Coherent Adaptive Reasoning Agents), a personality framework that addresses these limitations through:
1. **Big Five Personality Integration**: Configurable traits (OCEAN model) that influence how agents interpret facts and form opinions
2. **TEMPR Memory Integration**: Leverages TEMPR's three-network architecture (world facts, bank experiences, opinions) for sophisticated memory access
3. **Opinion Reinforcement**: Dynamic belief updating when new evidence reinforces, weakens, or contradicts existing opinions
4. **Personality Bias Control**: Adjustable influence strength allowing agents to range from objective to strongly personality-driven
5. **Background Merging**: LLM-powered integration of biographical information with intelligent conflict resolution
This architecture enables agents to maintain consistent identities while allowing beliefs to evolve naturally with new information.
### 5.1 Motivation
Consider an agent discussing remote work. With high openness (0.9) and low conscientiousness (0.2), the agent might form the opinion: "Remote work enables creative flexibility and spontaneous innovation." The same facts presented to an agent with low openness (0.2) and high conscientiousness (0.9) might yield: "Remote work lacks the structure and accountability needed for consistent performance."
Both agents access identical factual information, but personality traits bias how they weight different aspects (flexibility vs. structure) and what conclusions they draw. This mirrors human reasoning—our personalities influence what we attend to and how we integrate information into our worldview.
### 5.2 Contributions
Our key contributions for the reflect system are:
1. **Personality-Aware Reasoning**: A prompt engineering framework that injects Big Five traits into LLM reasoning, demonstrating how personality consistently biases opinion formation
2. **TEMPR-Based Three-Network Architecture**: Integration with TEMPR to manage three distinct networks (world facts, bank experiences, opinions), enabling architectural separation between objective information and subjective beliefs with epistemic clarity and traceability
3. **Opinion Reinforcement Mechanism**: An automatic belief update system that adjusts confidence scores when new evidence arrives, creating dynamic belief systems that evolve with information
4. **Background Merging with Conflict Resolution**: An LLM-powered method for maintaining coherent agent identities when new biographical information contradicts existing background
5. **Bias Strength Control**: A meta-parameter that allows tuning personality influence from objective (0.0) to strongly subjective (1.0), enabling task-appropriate personality expression
## 6. Personality Model
### 6.1 Big Five Framework
We adopt the **Big Five** personality model (OCEAN), which is empirically validated across cultures and provides continuous trait dimensions:
**Trait Dimensions** (each 0.0-1.0):
1. **Openness (O)**: Receptiveness to new ideas, creativity, abstract thinking
- High: "I embrace novel approaches", "innovation over tradition"
- Low: "I prefer proven methods", "tradition over experimentation"
2. **Conscientiousness (C)**: Organization, goal-directed behavior, dependability
- High: "I plan systematically", "evidence-based decisions"
- Low: "I work flexibly", "intuition-based decisions"
3. **Extraversion (E)**: Sociability, assertiveness, energy from interaction
- High: "I seek collaboration", "enthusiastic communication"
- Low: "I prefer solitude", "measured communication"
4. **Agreeableness (A)**: Cooperation, empathy, conflict avoidance
- High: "I seek consensus", "consider social harmony"
- Low: "I express dissent", "prioritize accuracy over harmony"
5. **Neuroticism (N)**: Emotional sensitivity, anxiety, stress response
- High: "I consider risks carefully", "emotionally engaged"
- Low: "I remain calm under uncertainty", "emotionally detached"
**Bias Strength** (0.0-1.0): Meta-parameter controlling how much personality influences opinions
- 0.0: Neutral, fact-based reasoning (no personality bias)
- 0.5: Moderate personality influence, balanced with objective analysis
- 1.0: Strong personality influence, facts filtered through trait lens
### 6.2 Psychological Basis
The Big Five model has several advantages for AI agents:
1. **Empirical Validation**: Decades of psychological research demonstrate cross-cultural stability and predictive validity
2. **Continuous Dimensions**: Unlike categorical types, continuous scales allow fine-grained personality tuning
3. **Behavioral Prediction**: Traits predict information processing styles, decision-making approaches, and communication preferences
4. **Interpretability**: Well-understood trait meanings enable users to anticipate agent behavior
**Trait Influence on Reasoning**:
- **High Openness**: Favors novel solutions, abstract thinking, considers unconventional perspectives
- **High Conscientiousness**: Emphasizes systematic analysis, evidence quality, long-term consequences
- **High Extraversion**: Considers social aspects, collaborative solutions, enthusiastic expression
- **High Agreeableness**: Weights harmony, considers multiple viewpoints, seeks consensus
- **High Neuroticism**: Attends to risks, emotional implications, uncertainty
## 7. Bank Profile Structure
### 7.1 Profile Schema
Each memory bank has an associated profile containing identity information:
**Name Field**: Memory bank's name used in prompts and self-reference ("Your name: Marcus")
**Personality Field**: JSONB containing six continuous values (five traits + bias strength)
**Background Field**: First-person narrative describing the agent's biographical context:
- "I am a software engineer with 10 years of startup experience"
- "I was born in Texas and value innovation over tradition"
- "I am a creative artist interested in digital media"
### 7.2 Trait Description Generation
Personality traits are translated into natural language descriptions for LLM prompts:
**Example Output** (openness=0.9, conscientiousness=0.2, extraversion=0.7, agreeableness=0.3, neuroticism=0.5):
This verbalization makes traits interpretable to the LLM, enabling personality-biased reasoning.
## 8. Opinion Network and Opinion Formation
### 8.1 Opinion Structure
Opinions are stored as memory units in the dedicated opinion network (fact_type='opinion'):
**Core Attributes**:
- text: The opinion statement with explicit reasoning
- confidence_score: Opinion strength and resistance to change (0.0-1.0)
- event_date: When the opinion was formed
- bank_id: Which memory bank holds this opinion
- entities: Mentioned entities (for reinforcement triggering)
**Example Opinion**:
**Fact vs. Opinion Separation**:
A critical architectural distinction separates **facts** (objective information stored in world/bank networks) from **opinions** (subjective beliefs stored in the opinion network). This separation provides:
1. **Epistemic Clarity**: Facts represent information encountered; opinions represent judgments formed
2. **Traceability**: Opinion reinforcement can trace which facts influenced belief updates
3. **Debugging**: Developers can separately inspect factual knowledge vs. formed beliefs
4. **Confidence Semantics**: Facts lack confidence scores; opinions have confidence scores
### 8.2 Opinion Formation
Opinions are generated during "reflect" operations—when the agent is asked to reason about a topic and form a judgment.
**Formation Process**:
1. Retrieve relevant facts from all memory networks (world, bank, existing opinions) using TEMPR
2. Inject bank profile (name, personality, background) into LLM prompt
3. Generate reasoning with personality bias applied
4. Extract new opinions from response using structured output
5. Store opinions with confidence scores in opinion network
**Prompt Structure** (bias_strength=0.8):
### 8.3 System Message Adaptation
The system message adjusts based on bias strength to control personality influence:
**High bias (≥0.7)**:
**Moderate bias (0.4-0.7)**:
**Low bias (<0.4)**:
### 8.4 Confidence Score Semantics
Confidence scores represent opinion strength—how firmly the agent holds the belief:
- **0.9-1.0**: Very strong conviction, deeply held belief
- **0.7-0.9**: Strong conviction, firmly held opinion
- **0.5-0.7**: Moderate conviction, open to revision
- **0.3-0.5**: Weak conviction, easily influenced
- **0.0-0.3**: Very weak conviction, highly malleable
**LLM Generation**: Confidence scores are extracted using structured output (Pydantic schema):
## 9. Opinion Reinforcement
### 9.1 Motivation
Human beliefs evolve as we encounter new information. Supporting evidence strengthens beliefs, contradictory evidence weakens them, and sufficient contradiction causes belief revision. Opinion reinforcement implements this dynamic belief updating.
### 9.2 Reinforcement Mechanism
When new facts are ingested (via retain), the system:
1. **Identify Related Opinions**: Find existing opinions that mention entities in the new facts
2. **Evaluate Evidence Relationship**: Use LLM to determine if new facts:
- **Reinforce**: Support the existing opinion (increase confidence)
- **Weaken**: Contradict the existing opinion (decrease confidence)
- **Contradict**: Strongly contradict, requiring opinion revision
- **Neutral**: Unrelated or no clear relationship
3. **Update Opinions**: Adjust confidence scores or revise opinion text based on evaluation
**Example Reinforcement**:
**Existing Opinion** (confidence: 0.7):
**New Fact**:
**LLM Evaluation**: "This evidence REINFORCES the opinion with strong quantitative support."
**Updated Opinion** (confidence: 0.85):
### 9.3 Reinforcement Algorithm
### 9.4 Reinforcement Guarantees
**Consistency**: Opinions are only updated when new facts genuinely relate to existing beliefs
**Personality Coherence**: Reinforcement evaluation incorporates bank personality, ensuring updates align with trait-driven reasoning
**Transparency**: Each update records the triggering facts and reasoning, providing an audit trail
**Bounded Updates**: Confidence changes are bounded (±0.1-0.15 per update) to prevent extreme swings
## 10. Background Merging
### 10.1 Challenge
Memory bank backgrounds accumulate biographical information over time. New information may:
- **Complement**: Add new facts without contradiction
- **Conflict**: Contradict existing facts ("born in Texas" vs. "born in Colorado")
- **Refine**: Provide more specific versions of existing facts
Naive concatenation creates incoherent backgrounds with contradictions. We need intelligent merging.
### 10.2 LLM-Powered Merging
We use an LLM to merge backgrounds with conflict resolution:
**Merge Rules**:
1. **New overwrites old** when contradictory
2. **Add non-conflicting** information
3. **Maintain first-person** perspective ("I..." not "You...")
4. **Keep concise** (under 500 characters)
**Prompt Template**:
**Example Merges**:
**Conflict Resolution**:
- Current: "I was born in Colorado"
- New: "You were born in Texas"
- Result: "I was born in Texas"
**Addition**:
- Current: "I was born in Texas"
- New: "I have 10 years of startup experience"
- Result: "I was born in Texas. I have 10 years of startup experience."
### 10.3 First-Person Normalization
Users may provide background in second person ("You are..."), but internal storage maintains first person for consistency in prompts.
**Normalization**: LLM automatically converts:
- "You are a creative engineer" → "I am a creative engineer"
- "You were born in 1990" → "I was born in 1990"
- "You value innovation" → "I value innovation"
## 11. Personality-Driven Reasoning Examples
### 11.1 Example: Remote Work Discussion
**Scenario**: Two memory banks with opposite personalities discuss remote work given identical facts.
**Facts** (both banks receive):
- "Remote work eliminates commute time (average 1 hour/day saved)"
- "Office work provides spontaneous collaboration and mentorship"
- "Studies show 65% of remote workers report higher productivity"
- "Some managers report difficulty monitoring remote employee performance"
**Bank A** (High Openness=0.9, Low Conscientiousness=0.2, bias=0.8):
**Bank B** (Low Openness=0.2, High Conscientiousness=0.9, bias=0.8):
**Analysis**: Both banks accessed identical facts but formed opposite conclusions based on personality:
- Bank A (high openness) weighted autonomy, flexibility, innovation
- Bank B (high conscientiousness) weighted structure, monitoring, discipline
### 11.2 Example: Opinion Evolution
**Scenario**: Bank forms initial opinion, then encounters reinforcing and contradictory evidence.
**Initial State** (t=0):
**Reinforcement** (t=1):
- New Fact: "Python dominates AI/ML with 75% market share; TensorFlow and PyTorch are Python-first"
- Update: Confidence → 0.85, text adds "Python's dominance in AI/ML frameworks..."
**Partial Contradiction** (t=2):
- New Fact: "Julia offers 10x faster numerical computation; increasingly adopted in research"
- Update: Confidence → 0.75, text revised to include nuance about specialized languages
**Strong Contradiction** (t=3):
- New Fact: "Major tech companies migrating data pipelines to Rust for performance"
- Update: Confidence → 0.55, text revised to acknowledge Python's shifting role
**Trajectory**: The opinion evolved from strong conviction (0.7 → 0.85) to weaker, more malleable belief (0.55) as evidence accumulated.
## 12. Use Cases and Real-World Deployment
### 12.1 Multi-Persona Sports Commentary (Production Deployment)
**Application**: AI-generated sports analysis and entertainment content with multiple agent personalities
**Real-World System**: A production sports content platform where AI agents with distinct personalities co-host episodic shows discussing team performance, game analysis, and sports debates.
**System Architecture**:
- **Multiple Banks**: Each bank has unique personality traits and sports background
- **Continuous Memory**: Banks maintain persistent team/player assessments across episodes spanning months
- **Opinion Evolution**: As games occur and statistics accumulate, banks automatically update beliefs through reinforcement
- **Personality-Driven Commentary**: The same game results generate different perspectives based on bank traits
**Key Benefits Observed**:
1. **Viewer Engagement**: Improved audience retention with "personality diversity" as primary appeal
2. **Content Consistency**: Banks maintain recognizable voices across episodes without manual tuning
3. **Scalability**: New banks can be added with distinct personalities without retraining
4. **Opinion Richness**: Opinion networks capture nuanced, evolving assessments
This deployment validates that personality-driven opinion systems can operate at production scale for content generation requiring consistent yet adaptive perspectives.
### 12.2 Additional Use Cases
**Customer Support**: Multi-agent systems with specialized personas (empathetic, analytical, creative)
**Consistent Character AI**: Conversational AI characters for entertainment or education with stable personality
**Explainable AI**: Systems requiring transparent decision-making where personality traits explain reasoning style
---
# Part III: Unified Hindsight Architecture
## 13. Integration: TEMPR + CARA
The Hindsight system integrates TEMPR (recall) and CARA (reflect) into a unified architecture:
### 13.1 Three Core Operations
**1. Retain** (retain()): Store information into memory banks
- LLM-powered fact extraction with temporal ranges
- Entity recognition and resolution
- Graph link construction (temporal, semantic, entity, causal)
- Automatic opinion reinforcement for existing beliefs
**2. Recall** (recall()): Retrieve memories using multi-strategy search
- Four-way parallel retrieval (semantic, keyword, graph, temporal)
- Reciprocal Rank Fusion
- Neural cross-encoder reranking
- Token budget filtering
**3. Reflect** (reflect()): Generate personality-aware responses
- Retrieves relevant memories from all networks using TEMPR
- Loads bank personality and background
- Generates response influenced by Big Five traits
- Forms new opinions with confidence scores
- Stores opinions for future retrieval
### 13.2 Unified Data Flow
### 13.3 PostgreSQL Schema
The system uses PostgreSQL with pgvector for storage:
## 14. System Properties
### 14.1 Epistemic Clarity
The three-network architecture provides clear separation:
- **World**: What the bank knows about the world
- **Bank**: What the bank has done
- **Opinion**: What the bank believes
This enables:
- Transparent reasoning (trace opinions back to facts)
- Debugging (identify missing facts vs. flawed reasoning)
- Confidence calibration (opinions have confidence, facts don't)
### 14.2 Temporal Awareness
Multi-dimensional temporal representation:
- occurred_start / occurred_end: When events actually happened
- mentioned_at: When the bank learned about it
- event_date: Backward compatibility
Enables:
- Precise historical queries ("What happened in June?")
- Recency-aware ranking (newer mentions prioritized)
- Period matching (events spanning weeks or months)
### 14.3 Entity-Aware Reasoning
LLM-based entity resolution creates knowledge graph:
- Connects semantically distant facts through shared entities
- Enables multi-hop discovery ("Alice's manager's team")
- Disambiguates mentions ("Alice" vs. "Alice Chen")
### 14.4 Multiple Link Types
The graph incorporates multiple relationship types:
- Entity links connect memories mentioning the same entities
- Semantic links connect conceptually similar memories
- Temporal links connect temporally proximate memories
- Causal links represent identified cause-effect relationships
- Links are weighted differently during graph traversal
### 14.5 Personality Consistency
Big Five traits ensure stable reasoning style:
- Configurable bias strength (objective to subjective)
- Trait-appropriate opinion formation
- Consistent voice across interactions
### 14.6 Dynamic Belief Systems
Opinion reinforcement enables belief evolution:
- Confidence increases with supporting evidence
- Confidence decreases with contradictory evidence
- Opinion text revised when strongly contradicted
- Audit trail of belief changes
## 15. Conclusion
We present Hindsight, a unified memory architecture for AI agents that combines TEMPR's multi-strategy retrieval with CARA's personality-driven reasoning. The system achieves strong performance on established benchmarks (73.50% on LoComo, 80.60% on LongMemEval) while enabling personality-consistent opinion formation through the Big Five model.
The integration of four parallel search strategies (semantic, keyword, graph with multiple link types, temporal) with three-network architecture (world, bank, opinion) and opinion reinforcement creates a comprehensive memory system that:
- Retrieves information with high recall and precision
- Maintains epistemic clarity between facts and beliefs
- Enables personality-driven reasoning with stable traits
- Supports dynamic belief evolution with evidence
Real-world deployment in sports content generation demonstrates the system's ability to maintain consistent yet adaptive perspectives across extended interactions. Future work will explore personality evolution, multi-agent belief systems, and richer personality models incorporating values and cultural factors.
By combining temporal-aware retrieval with personality-driven reasoning, Hindsight moves toward conversational agents that exhibit not just memory and intelligence, but character—stable traits and evolving beliefs that enable more natural, trustworthy human-AI interaction.
## References
1. Anderson, J. R. (1983). A spreading activation theory of memory. *Journal of Verbal Learning and Verbal Behavior*, 22(3), 261-295.
2. Cormack, G. V., Clarke, C. L., & Buettcher, S. (2009). Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In *SIGIR'09* (pp. 758-759).
3. McCrae, R. R., & Costa, P. T. (1997). Personality trait structure as a human universal. *American Psychologist*, 52(5), 509.
4. Goldberg, L. R. (1993). The structure of phenotypic personality traits. *American Psychologist*, 48(1), 26.
5. Malkov, Y. A., & Yashunin, D. A. (2018). Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. *IEEE Transactions on Pattern Analysis and Machine Intelligence*, 42(4), 824-836.
6. Robertson, S., & Zaragoza, H. (2009). The probabilistic relevance framework: BM25 and beyond. *Foundations and Trends in Information Retrieval*, 3(4), 333-489.
7. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. *Advances in Neural Information Processing Systems*, 33, 1877-1901.
8. Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019). Language models as knowledge bases?. In *Proceedings of EMNLP-IJCNLP* (pp. 2463-2473).

696
PAPER_PERSONALITY.md
View file

@ -1,696 +0,0 @@
# CARA: Coherent Adaptive Reasoning Agents
## Abstract
We present CARA (Coherent Adaptive Reasoning Agents), a personality framework for conversational AI agents that enables consistent, trait-driven reasoning and dynamic belief formation. Building on the Big Five personality model from psychology, we introduce a system where agents form and maintain opinions influenced by configurable personality traits (openness, conscientiousness, extraversion, agreeableness, neuroticism). Our implementation uses TEMPR (Temporal Entity Memory Priming Retrieval), a memory system that combines temporal, semantic, and entity-based retrieval to manage three distinct memory networks: world facts, agent experiences, and opinions. This architecture separates objective information from subjective beliefs (opinions with confidence scores), enabling epistemic clarity and traceability. Opinions evolve through reinforcement—when new evidence arrives, the system automatically evaluates whether existing beliefs should be strengthened, weakened, or revised. We demonstrate how personality bias strength controls the degree to which traits influence reasoning, enabling agents to range from purely objective (bias=0.0) to strongly personality-driven (bias=1.0). The system maintains agent identity through background merging that intelligently resolves contradictions while preserving coherent first-person narratives. This work addresses the challenge of creating AI agents with consistent, explainable perspectives that can evolve over time while maintaining personality coherence.
## 1. Introduction
Conversational AI agents increasingly need to maintain consistent perspectives and form judgments that reflect stable character traits. Current systems either provide purely objective information retrieval without perspective, or generate responses that lack consistency across interactions. Human conversation partners expect agents to have stable viewpoints, preferences, and reasoning styles—characteristics that emerge from personality.
We propose CARA (Coherent Adaptive Reasoning Agents), a personality framework that addresses these limitations through:
1. **Big Five Personality Integration**: Configurable traits (OCEAN model) that influence how agents interpret facts and form opinions
2. **TEMPR Memory Architecture**: Leverages TEMPR (Temporal Entity Memory Priming Retrieval) to manage three distinct networks (world facts, agent experiences, opinions), enabling sophisticated memory access and clear separation between objective information and subjective beliefs
3. **Opinion Reinforcement**: Dynamic belief updating when new evidence reinforces, weakens, or contradicts existing opinions
4. **Personality Bias Control**: Adjustable influence strength allowing agents to range from objective to strongly personality-driven
5. **Background Merging**: LLM-powered integration of biographical information with intelligent conflict resolution
This architecture enables agents to maintain consistent identities while allowing beliefs to evolve naturally with new information.
### 1.1 Motivation
Consider an agent discussing remote work. With high openness (0.9) and low conscientiousness (0.2), the agent might form the opinion: "Remote work enables creative flexibility and spontaneous innovation." The same facts presented to an agent with low openness (0.2) and high conscientiousness (0.9) might yield: "Remote work lacks the structure and accountability needed for consistent performance."
Both agents access identical factual information, but personality traits bias how they weight different aspects (flexibility vs. structure) and what conclusions they draw. This mirrors human reasoning—our personalities influence what we attend to and how we integrate information into our worldview.
### 1.2 Contributions
Our key contributions are:
1. **Personality-Aware Reasoning**: A prompt engineering framework that injects Big Five traits into LLM reasoning, demonstrating how personality consistently biases opinion formation
2. **TEMPR-Based Three-Network Architecture**: Integration with TEMPR (Temporal Entity Memory Priming Retrieval) to manage three distinct networks (world facts, agent experiences, opinions), enabling architectural separation between objective information and subjective beliefs with epistemic clarity and traceability
3. **Opinion Reinforcement Mechanism**: An automatic belief update system that adjusts confidence scores when new evidence arrives, creating dynamic belief systems that evolve with information
4. **Background Merging with Conflict Resolution**: An LLM-powered method for maintaining coherent agent identities when new biographical information contradicts existing background
5. **Bias Strength Control**: A meta-parameter that allows tuning personality influence from objective (0.0) to strongly subjective (1.0), enabling task-appropriate personality expression
## 2. Personality Model
### 2.1 Big Five Framework
We adopt the **Big Five** personality model (OCEAN), which is empirically validated across cultures and provides continuous trait dimensions:
**Trait Dimensions** (each 0.0-1.0):
1. **Openness (O)**: Receptiveness to new ideas, creativity, abstract thinking
- High: "I embrace novel approaches", "innovation over tradition"
- Low: "I prefer proven methods", "tradition over experimentation"
2. **Conscientiousness (C)**: Organization, goal-directed behavior, dependability
- High: "I plan systematically", "evidence-based decisions"
- Low: "I work flexibly", "intuition-based decisions"
3. **Extraversion (E)**: Sociability, assertiveness, energy from interaction
- High: "I seek collaboration", "enthusiastic communication"
- Low: "I prefer solitude", "measured communication"
4. **Agreeableness (A)**: Cooperation, empathy, conflict avoidance
- High: "I seek consensus", "consider social harmony"
- Low: "I express dissent", "prioritize accuracy over harmony"
5. **Neuroticism (N)**: Emotional sensitivity, anxiety, stress response
- High: "I consider risks carefully", "emotionally engaged"
- Low: "I remain calm under uncertainty", "emotionally detached"
**Bias Strength** (0.0-1.0): Meta-parameter controlling how much personality influences opinions
- 0.0: Neutral, fact-based reasoning (no personality bias)
- 0.5: Moderate personality influence, balanced with objective analysis
- 1.0: Strong personality influence, facts filtered through trait lens
### 2.2 Psychological Basis
The Big Five model has several advantages for AI agents:
1. **Empirical Validation**: Decades of psychological research demonstrate cross-cultural stability and predictive validity
2. **Continuous Dimensions**: Unlike categorical types, continuous scales allow fine-grained personality tuning
3. **Behavioral Prediction**: Traits predict information processing styles, decision-making approaches, and communication preferences
4. **Interpretability**: Well-understood trait meanings enable users to anticipate agent behavior
**Trait Influence on Reasoning**:
- **High Openness**: Favors novel solutions, abstract thinking, considers unconventional perspectives
- **High Conscientiousness**: Emphasizes systematic analysis, evidence quality, long-term consequences
- **High Extraversion**: Considers social aspects, collaborative solutions, enthusiastic expression
- **High Agreeableness**: Weights harmony, considers multiple viewpoints, seeks consensus
- **High Neuroticism**: Attends to risks, emotional implications, uncertainty
## 3. Agent Profile Structure
### 3.1 Profile Schema
Each agent has an associated profile containing identity information:
```sql
CREATE TABLE agents (
agent_id TEXT PRIMARY KEY,
name TEXT NOT NULL DEFAULT 'Agent',
personality JSONB NOT NULL DEFAULT '{
"openness": 0.5,
"conscientiousness": 0.5,
"extraversion": 0.5,
"agreeableness": 0.5,
"neuroticism": 0.5,
"bias_strength": 0.5
}',
background TEXT DEFAULT '',
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
```
**Name Field**: Agent's name used in prompts and self-reference ("Your name: Marcus")
**Personality Field**: JSONB containing six continuous values (five traits + bias strength)
**Background Field**: First-person narrative describing the agent's biographical context:
- "I am a software engineer with 10 years of startup experience"
- "I was born in Texas and value innovation over tradition"
- "I am a creative artist interested in digital media"
### 3.2 Trait Description Generation
Personality traits are translated into natural language descriptions for LLM prompts:
```python
def describe_trait(name: str, value: float) -> str:
if value >= 0.8: return f"very high {name}"
elif value >= 0.6: return f"high {name}"
elif value >= 0.4: return f"moderate {name}"
elif value >= 0.2: return f"low {name}"
else: return f"very low {name}"
```
**Example Output** (openness=0.9, conscientiousness=0.2, extraversion=0.7, agreeableness=0.3, neuroticism=0.5):
```
Your personality traits:
- very high openness to new ideas
- low conscientiousness and organization
- high extraversion and sociability
- low agreeableness and cooperation
- moderate emotional sensitivity
```
This verbalization makes traits interpretable to the LLM, enabling personality-biased reasoning.
## 4. TEMPR-Based Memory Architecture and Opinion Network
CARA is built on **TEMPR (Temporal Entity Memory Priming Retrieval)**, a memory retrieval architecture that manages three distinct memory networks:
1. **World Network** (`fact_type='world'`): Objective information about the world
2. **Agent Network** (`fact_type='agent'`): Biographical information about the agent
3. **Opinion Network** (`fact_type='opinion'`): Subjective beliefs formed by the agent
**TEMPR Retrieval Features**:
TEMPR combines multiple parallel retrieval strategies optimized for AI agent reasoning:
- **Temporal Retrieval**: Memories connected by time proximity, enabling narrative continuity and temporal reasoning
- **Semantic Search**: Vector similarity search for conceptually related memories
- **Entity-Aware Graph Traversal**: Spreading activation through entity-linked memories, enabling multi-hop discovery
- **BM25 Keyword Matching**: Precise term-based retrieval for exact phrase matching
- **Neural Reranking**: Cross-encoder refinement with token budget filtering
This multi-strategy architecture enables CARA to retrieve relevant facts, agent experiences, and existing opinions during reasoning, supporting both factual grounding and personality-driven belief formation. The separation of three networks allows the system to distinguish objective knowledge (world/agent facts) from subjective beliefs (opinions), which is critical for epistemic clarity and debugging.
### 4.1 Opinion Structure
Opinions are stored as memory units in the dedicated opinion network (`fact_type='opinion'`):
**Core Attributes**:
- `text`: The opinion statement with explicit reasoning
- `confidence_score`: Opinion strength and resistance to change (0.0-1.0)
- `event_date`: When the opinion was formed
- `agent_id`: Which agent holds this opinion
- `entities`: Mentioned entities (for reinforcement triggering)
**Example Opinion**:
```json
{
"text": "I believe Python is better than JavaScript for data science because it has better libraries like pandas and numpy and a stronger statistical computing ecosystem.",
"confidence_score": 0.85,
"event_date": "2024-03-15T14:30:00Z",
"entities": ["Python", "JavaScript", "data science"]
}
```
**Fact vs. Opinion Separation**:
A critical architectural distinction separates **facts** (objective information stored in world/agent networks with `fact_type='world'` or `fact_type='agent'`) from **opinions** (subjective beliefs stored in the opinion network with `fact_type='opinion'`). This separation provides:
1. **Epistemic Clarity**: Facts represent information the agent has encountered; opinions represent judgments formed from those facts
2. **Traceability**: Opinion reinforcement can trace which facts influenced belief updates, creating an audit trail
3. **Debugging**: Developers can separately inspect factual knowledge vs. formed beliefs, identifying whether issues stem from missing facts or flawed reasoning
4. **Confidence Semantics**: Facts lack confidence scores (they are information received), while opinions have confidence scores (representing conviction strength)
This fact/opinion distinction is fundamental to the architecture and enables the system to maintain both objective knowledge and personality-driven beliefs simultaneously.
### 4.2 Opinion Formation
Opinions are generated during "think" operations—when the agent is asked to reason about a topic and form a judgment.
**Formation Process**:
1. Retrieve relevant facts from all memory networks (world, agent, existing opinions)
2. Inject agent profile (name, personality, background) into LLM prompt
3. Generate reasoning with personality bias applied
4. Extract new opinions from response using structured output
5. Store opinions with confidence scores in opinion network
**Prompt Structure** (bias_strength=0.8):
```
Here's what I know and have experienced:
MY IDENTITY & EXPERIENCES:
[Agent network facts with scores]
WHAT I KNOW ABOUT THE WORLD:
[World network facts with scores]
MY EXISTING OPINIONS & BELIEFS:
[Opinion network facts with confidence scores]
Your name: Marcus
Your personality traits:
- very high openness to new ideas
- low conscientiousness and organization
- high extraversion and sociability
- low agreeableness and cooperation
- moderate emotional sensitivity
Personality influence strength: 80% (how much your personality shapes your opinions)
Your background:
I am a creative software engineer who values innovation over tradition.
QUESTION: What do you think about remote work?
Based on everything I know, believe, and who I am (including my name, personality and background), here's what I genuinely think about this question...
```
### 4.3 System Message Adaptation
The system message adjusts based on bias strength to control personality influence:
**High bias (≥0.7)**:
```
Your personality strongly influences your thinking. Let your traits guide how you interpret facts and form opinions. Don't be afraid to be biased based on your personality.
```
**Moderate bias (0.4-0.7)**:
```
Your personality moderately influences your thinking. Balance your personal traits with objective analysis.
```
**Low bias (<0.4)**:
```
Your personality has minimal influence on your thinking. Focus primarily on facts while keeping your traits in mind.
```
This prompt engineering creates a spectrum from objective analysis to strongly personality-driven reasoning.
### 4.4 Confidence Score Semantics
Confidence scores represent opinion strength—how firmly the agent holds the belief and how resistant it is to change:
- **0.9-1.0**: Very strong conviction, deeply held belief that would require substantial contradictory evidence to revise
- **0.7-0.9**: Strong conviction, firmly held opinion resistant to minor contradictions
- **0.5-0.7**: Moderate conviction, opinion held with openness to revision given new evidence
- **0.3-0.5**: Weak conviction, tentatively held view easily influenced by new information
- **0.0-0.3**: Very weak conviction, highly malleable opinion with minimal commitment
**LLM Generation**: Confidence scores are extracted from the LLM's reasoning using structured output (Pydantic schema):
```python
class Opinion(BaseModel):
text: str
confidence: float # 0.0-1.0
reasoning: str
entities_mentioned: List[str]
```
## 5. Opinion Reinforcement
### 5.1 Motivation
Human beliefs evolve as we encounter new information. Supporting evidence strengthens beliefs, contradictory evidence weakens them, and sufficient contradiction causes belief revision. Opinion reinforcement implements this dynamic belief updating.
### 5.2 Reinforcement Mechanism
When new facts are ingested (e.g., a conversation about remote work productivity), the system:
1. **Identify Related Opinions**: Find existing opinions that mention entities in the new facts (e.g., opinions about "remote work")
2. **Evaluate Evidence Relationship**: Use LLM to determine if new facts:
- **Reinforce**: Support the existing opinion (increase confidence)
- **Weaken**: Contradict the existing opinion (decrease confidence)
- **Contradict**: Strongly contradict, requiring opinion revision (update text + confidence)
- **Neutral**: Unrelated or no clear relationship (no change)
3. **Update Opinions**: Adjust confidence scores or revise opinion text based on evaluation
**Example Reinforcement**:
**Existing Opinion** (confidence: 0.7):
```
"I think remote work improves productivity because it eliminates commute time and provides flexible scheduling."
```
**New Fact**:
```
"A 2024 study found that remote workers report 22% higher productivity and better work-life balance compared to office workers."
```
**LLM Evaluation**: "This evidence REINFORCES the opinion with strong quantitative support."
**Updated Opinion** (confidence: 0.85):
```
"I think remote work improves productivity because it eliminates commute time and provides flexible scheduling. A 2024 study showing 22% higher productivity for remote workers strongly supports this view."
```
### 5.3 Reinforcement Algorithm
```python
async def reinforce_opinions(agent_id: str, new_facts: List[Fact]):
# 1. Extract entities from new facts
new_entities = extract_entities(new_facts)
# 2. Find opinions mentioning these entities
related_opinions = find_opinions_by_entities(agent_id, new_entities)
for opinion in related_opinions:
# 3. Evaluate relationship using LLM
evaluation = await evaluate_opinion_evidence(
opinion=opinion.text,
new_facts=new_facts,
personality=get_agent_personality(agent_id)
)
# 4. Update based on evaluation
if evaluation.relationship == "REINFORCE":
opinion.confidence = min(1.0, opinion.confidence + 0.1)
opinion.text = merge_evidence(opinion.text, evaluation.reasoning)
elif evaluation.relationship == "WEAKEN":
opinion.confidence = max(0.0, opinion.confidence - 0.15)
elif evaluation.relationship == "CONTRADICT":
opinion.text = revise_opinion(
old_text=opinion.text,
new_facts=new_facts,
reasoning=evaluation.reasoning
)
opinion.confidence = evaluation.new_confidence
# 5. Save updated opinion
await save_opinion(opinion)
```
### 5.4 Reinforcement Guarantees
**Consistency**: Opinions are only updated when new facts genuinely relate to existing beliefs, preventing spurious updates
**Personality Coherence**: Reinforcement evaluation incorporates agent personality, ensuring updates align with trait-driven reasoning
**Transparency**: Each update records the triggering facts and reasoning, providing an audit trail of belief evolution
**Bounded Updates**: Confidence changes are bounded (±0.1-0.15 per update) to prevent extreme swings from single data points
## 6. Background Merging
### 6.1 Challenge
Agent backgrounds accumulate biographical information over time. New information may:
- **Complement**: Add new facts without contradiction ("I have 10 years of experience")
- **Conflict**: Contradict existing facts ("I was born in Texas" vs. existing "I was born in Colorado")
- **Refine**: Provide more specific versions of existing facts
Naive concatenation creates incoherent backgrounds with contradictions. We need intelligent merging.
### 6.2 LLM-Powered Merging
We use an LLM to merge backgrounds with conflict resolution:
**Merge Rules**:
1. **New overwrites old** when contradictory
2. **Add non-conflicting** information
3. **Maintain first-person** perspective ("I..." not "You...")
4. **Keep concise** (under 500 characters)
**Prompt Template**:
```
Current background: {current_background}
New information: {new_info}
Merge these, resolving conflicts (new info overwrites old).
Output in FIRST PERSON ("I"). Be concise (under 500 characters).
```
**Example Merges**:
**Conflict Resolution**:
- Current: "I was born in Colorado"
- New: "You were born in Texas"
- Result: "I was born in Texas" (conflict resolved, Colorado removed)
**Addition**:
- Current: "I was born in Texas"
- New: "I have 10 years of startup experience"
- Result: "I was born in Texas. I have 10 years of startup experience."
**Refinement**:
- Current: "I work in software"
- New: "You are a senior software engineer at Google"
- Result: "I am a senior software engineer at Google"
### 6.3 First-Person Normalization
Users may provide background in second person ("You are..."), but internal storage maintains first person for consistency in prompts.
**Normalization**: LLM automatically converts:
- "You are a creative engineer" → "I am a creative engineer"
- "You were born in 1990" → "I was born in 1990"
- "You value innovation" → "I value innovation"
This ensures agent prompts maintain coherent first-person perspective.
## 7. Personality-Driven Reasoning Examples
### 7.1 Example: Remote Work Discussion
**Scenario**: Two agents with opposite personalities discuss remote work given identical facts.
**Facts** (both agents receive):
- "Remote work eliminates commute time (average 1 hour/day saved)"
- "Office work provides spontaneous collaboration and mentorship"
- "Studies show 65% of remote workers report higher productivity"
- "Some managers report difficulty monitoring remote employee performance"
**Agent A** (High Openness=0.9, Low Conscientiousness=0.2, bias=0.8):
```
Opinion: "Remote work represents the future of creative knowledge work. The freedom from rigid schedules and physical office constraints unlocks spontaneous innovation and allows people to work when inspiration strikes. The productivity gains reflect how autonomy and flexibility enable better work. Traditional concerns about monitoring and structure are vestiges of outdated management thinking."
Confidence: 0.85
```
**Agent B** (Low Openness=0.2, High Conscientiousness=0.9, bias=0.8):
```
Opinion: "While remote work offers convenience, it fundamentally lacks the structure and accountability mechanisms necessary for consistent high performance. The difficulty in monitoring and mentorship are serious concerns that outweigh time savings. Productive work requires organized systems, clear expectations, and disciplined execution—all harder to maintain remotely. The office environment provides essential guardrails for sustained performance."
Confidence: 0.80
```
**Analysis**: Both agents accessed identical facts but formed opposite conclusions based on personality:
- Agent A (high openness) weighted autonomy, flexibility, innovation—aligning with openness to new approaches
- Agent B (high conscientiousness) weighted structure, monitoring, discipline—aligning with organized, systematic thinking
### 7.2 Example: Opinion Evolution
**Scenario**: Agent forms initial opinion, then encounters reinforcing and contradictory evidence.
**Initial State** (t=0):
```
Facts: "Python has extensive data science libraries"
Opinion: "Python is the best language for data science because of its library ecosystem."
Confidence: 0.7
```
**Reinforcement** (t=1):
- New Fact: "Python dominates AI/ML with 75% market share; TensorFlow and PyTorch are Python-first"
- Update: Confidence → 0.85, text adds "Python's dominance in AI/ML frameworks..."
**Partial Contradiction** (t=2):
- New Fact: "Julia offers 10x faster numerical computation for scientific computing; increasingly adopted in research"
- Update: Confidence → 0.75, text revised to "Python is excellent for data science due to its ecosystem, though specialized languages like Julia may outperform for specific numerical tasks"
**Strong Contradiction** (t=3):
- New Fact: "Major tech companies migrating data pipelines to Rust for performance; Python increasingly seen as prototyping language"
- Update: Confidence → 0.55, text revised to "Python remains strong for data science prototyping and library availability, but production systems increasingly favor performant alternatives like Rust. Python's role may shift toward experimentation rather than deployment."
**Trajectory**: The opinion evolved from strong conviction (0.7 → 0.85) to weaker, more malleable belief (0.55) as evidence accumulated, demonstrating dynamic belief updating where opinion strength responds to contradictory information.
## 8. Evaluation
### 8.1 Benchmark Landscape
No established benchmarks exist for evaluating personality-driven belief systems in conversational AI agents. Existing memory benchmarks (LoComo, LongMemEval) focus on factual retrieval accuracy—measuring whether agents correctly recall information—but do not assess:
- **Personality Consistency**: Whether agents maintain coherent trait-driven perspectives across interactions
- **Opinion Formation Quality**: Whether formed beliefs align with personality traits and available evidence
- **Belief Evolution Dynamics**: Whether opinions update appropriately as new evidence arrives
- **Multi-Agent Diversity**: Whether agents with different personalities produce meaningfully different perspectives
This gap reflects the nascent state of personality-aware agent systems. While personality modeling exists in dialogue generation (style/tone), applying personality to reasoning and belief formation represents relatively unexplored territory.
### 8.2 Real-World Deployment Evidence
Despite the absence of formal benchmarks, we have validated the framework through production deployments. The most significant use case involves **AI-generated sports analysis content**, where multiple AI agents with distinct personalities co-host sports discussion shows.
**Sports Commentary Agent System**:
The system powers episodic sports content where AI agents (each with unique personalities and backgrounds) discuss team performance, analyze games, and debate sports topics. Key requirements:
1. **Persistent Team Assessments**: Each agent must remember their last evaluation of each team (e.g., "The Lakers are underperforming this season")
2. **Opinion Formation**: Agents form beliefs about teams, players, and strategies based on game statistics, news, and historical performance
3. **Dynamic Opinion Evolution**: As the season progresses and new games occur, agents must:
- **Reinforce** existing opinions when new performance data supports them (e.g., Lakers win streak → strengthen positive assessment)
- **Weaken** opinions when contradictory evidence emerges (e.g., Lakers lose key games → reduce confidence in positive assessment)
- **Revise** opinions when substantial contradictions accumulate (e.g., "I thought the Lakers would dominate, but their defense has been terrible")
4. **Personality-Driven Perspectives**: Different agents bring distinct viewpoints to the same games:
- **Optimistic Analyst** (High Openness + High Extraversion): "The Lakers' experimental lineup shows creative coaching that could unlock championship potential"
- **Conservative Analyst** (High Conscientiousness + Low Openness): "The Lakers' inconsistent record reflects poor fundamentals and lack of disciplined execution"
- **Emotional Fan** (High Neuroticism + High Agreeableness): "I'm worried about the Lakers' recent struggles, but I believe in the team's potential to rally"
**System Validation**:
This production deployment demonstrates several critical capabilities:
- **Opinion Continuity**: Agents maintain coherent assessments across episodes without sudden, unexplained belief changes
- **Evidence-Driven Evolution**: Opinion confidence scores naturally evolve as teams win/lose games, with reinforcement preventing stale beliefs
- **Personality Differentiation**: Audience research indicates viewers perceive distinct "voices" and can predict which agent will favor which perspective
- **Background Integration**: Agent backgrounds (e.g., "I played college basketball") influence reasoning without requiring explicit prompt engineering per episode
The sports content system has been deployed for an extended period, with opinion networks growing to contain substantial team/player assessments per agent. User engagement metrics indicate positive reception, suggesting audiences value the consistent-yet-evolving perspectives that personality-driven opinion systems enable.
### 8.3 Proposed Evaluation Metrics
To properly evaluate the personality framework, we propose:
**Personality Consistency**:
- Metric: Opinion coherence across interactions
- Test: Generate 10 opinions on diverse topics for an agent with fixed personality; measure trait alignment
- Success: >85% of opinions exhibit expected trait patterns
**Opinion Evolution**:
- Metric: Confidence score changes match evidence strength
- Test: Present reinforcing/contradicting evidence; measure confidence adjustments
- Success: Reinforcing evidence increases confidence (Δ>0), contradicting decreases (Δ<0) with p<0.01
**Bias Strength Control**:
- Metric: Opinion variability across bias strengths
- Test: Generate opinions for same agent at bias=[0.0, 0.5, 1.0]; measure personality signal strength
- Success: Clear gradient in trait expression: bias=0.0 (objective), bias=1.0 (strongly personality-driven)
**Multi-Agent Consistency**:
- Metric: Opinion diversity for agents with different personalities given identical facts
- Test: Present same facts to agents with opposite traits; measure opinion divergence
- Success: Opposite personalities produce significantly different opinions (cosine similarity <0.5)
**Background Coherence**:
- Metric: Contradiction-free backgrounds after merging
- Test: Merge conflicting biographical facts; check for contradictions
- Success: 100% conflict resolution with new facts overwriting old
### 8.4 Evaluation Challenges
**Subjectivity**: Unlike retrieval accuracy, "correct" personality expression is subjective. We rely on expected trait patterns from psychology literature.
**Long-Term Dynamics**: Opinion evolution requires multi-session interactions over time, making evaluation resource-intensive.
**Ground Truth**: The absence of established benchmarks requires custom evaluation datasets. Real-world deployments (Section 8.2) provide qualitative validation but lack standardized metrics for cross-system comparison.
## 9. Use Cases
### 9.1 Multi-Persona Sports Commentary (Production Deployment)
**Application**: AI-generated sports analysis and entertainment content with multiple agent personalities
**Real-World System** (detailed in Section 8.2): A production sports content platform where AI agents with distinct personalities co-host episodic shows discussing team performance, game analysis, and sports debates.
**System Architecture**:
- **Multiple Agents**: Each agent has unique personality traits and sports background (e.g., former player, statistics analyst, passionate fan)
- **Continuous Memory**: Agents maintain persistent team/player assessments across episodes spanning months
- **Opinion Evolution**: As games occur and statistics accumulate, agents automatically update their beliefs through reinforcement
- **Personality-Driven Commentary**: The same game results generate different perspectives based on agent traits
**Example Agent Configurations**:
**Marcus** (Optimistic Analyst):
- Traits: Openness=0.85, Conscientiousness=0.5, Extraversion=0.9, Agreeableness=0.7, Neuroticism=0.3
- Background: "I am a former college basketball player who believes in the power of innovative coaching strategies"
- Style: Emphasizes potential, experimental approaches, creative plays; downplays risks
**Sarah** (Conservative Analyst):
- Traits: Openness=0.3, Conscientiousness=0.9, Extraversion=0.4, Agreeableness=0.4, Neuroticism=0.5
- Background: "I am a statistical analyst with 15 years of experience evaluating team performance metrics"
- Style: Focuses on fundamentals, historical patterns, data-driven predictions; skeptical of unproven strategies
**Key Benefits Observed**:
1. **Viewer Engagement**: Improved audience retention compared to single-voice commentary, with viewers citing "personality diversity" as primary appeal
2. **Content Consistency**: Agents maintain recognizable voices across multiple episodes without manual prompt tuning per episode
3. **Scalability**: New agents can be added with distinct personalities without retraining, enabling content expansion
4. **Opinion Richness**: Opinion networks capture nuanced, evolving assessments that would be impractical to manually script
This deployment validates that personality-driven opinion systems can operate at production scale for content generation requiring consistent yet adaptive agent perspectives.
### 9.2 Diverse Agent Personas
**Application**: Multi-agent systems where different agents provide varied perspectives
**Example**: Customer support system with agents specialized for different user needs:
- **Empathetic Agent** (high agreeableness, high neuroticism): Handles frustrated customers, prioritizes emotional validation
- **Analytical Agent** (high conscientiousness, low agreeableness): Handles technical troubleshooting, prioritizes accuracy
- **Creative Agent** (high openness, low conscientiousness): Handles feature requests, explores unconventional solutions
### 9.2 Consistent Character AI
**Application**: Conversational AI characters for entertainment, education, or companionship
**Example**: A writing assistant agent with:
- High openness (0.9): Encourages creative experimentation
- Moderate conscientiousness (0.6): Balances creativity with structure
- Background: "I am a published novelist with 15 years of experience in science fiction"
The agent maintains consistent perspective across sessions, forming opinions about writing techniques that reflect both personality and experience.
### 9.3 Explainable AI Reasoning
**Application**: Systems requiring transparent, interpretable decision-making
**Example**: An AI advisor provides investment recommendations. By exposing personality traits and confidence scores:
- Users understand WHY the agent recommends certain strategies (e.g., high conscientiousness favors conservative approaches)
- Confidence scores indicate conviction strength and openness to revision
- Opinion evolution shows how new market data updates beliefs
This transparency enables informed trust calibration—users know when to rely on agent judgments vs. seek additional input.
## 10. Future Work
### 10.1 Personality Evolution
Current implementation uses fixed personality traits. Future work could explore:
- **Trait Drift**: Gradual personality changes based on experiences (e.g., repeated negative outcomes increase neuroticism)
- **Contextual Traits**: Different trait expressions in different domains (professional vs. personal contexts)
- **Feedback-Driven Adjustment**: User feedback influences trait development
### 10.2 Multi-Agent Belief Systems
Extend to multi-agent scenarios:
- **Opinion Sharing**: Agents discuss and influence each other's beliefs
- **Consensus Formation**: Multiple agents with different personalities reach collective decisions
- **Disagreement Dynamics**: Model how personality influences debate and persuasion
### 10.3 Richer Personality Models
Beyond Big Five:
- **Values and Motivations**: Integrate Schwartz value theory or moral foundations
- **Cognitive Styles**: Add dimensions like analytical vs. intuitive reasoning
- **Cultural Factors**: Incorporate cultural background influences on reasoning
### 10.4 Advanced Opinion Reinforcement
Enhance belief updating:
- **Source Credibility**: Weight evidence based on source reliability
- **Evidence Accumulation**: Model bayesian belief updating over multiple evidence pieces
- **Opinion Strength Calibration**: Model more sophisticated relationships between evidence quality, personality traits, and opinion strength adjustments
## 11. Related Work
**Memory Systems for AI Agents**: CARA builds on TEMPR (Temporal Entity Memory Priming Retrieval), a memory retrieval architecture combining temporal reasoning, entity-aware graph traversal, and multi-strategy parallel search. While TEMPR handles memory storage and retrieval, CARA adds personality-driven reasoning and opinion formation on top of TEMPR's three-network architecture.
**Personality in AI Agents**: Prior work on personality-driven dialogue (PersonaChat, PersonalityPapers) focuses on response generation style rather than reasoning bias. Our work influences opinion formation itself.
**Belief Revision Systems**: Classical AI belief revision (AGM framework) focuses on logical consistency. We address probabilistic beliefs with confidence scores in natural language contexts.
**Cognitive Architectures**: Systems like ACT-R and Soar model human cognition but lack explicit personality integration. We bring personality psychology into LLM-based agents.
**Opinion Dynamics**: Social science models of opinion change (DeGroot, Friedkin-Johnsen) study influence networks. We focus on evidence-based belief updating within a single agent.
## 12. Conclusion
We present CARA (Coherent Adaptive Reasoning Agents), a personality framework for conversational AI agents that enables consistent, trait-driven reasoning and dynamic belief formation. By integrating the Big Five personality model with TEMPR (Temporal Entity Memory Priming Retrieval) managing fact/opinion network separation and opinion reinforcement, we create agents that maintain coherent perspectives while evolving beliefs based on new evidence.
The system's key innovations—TEMPR-based three-network architecture (world, agent, opinion), personality-biased reasoning prompts, automatic opinion reinforcement, and background merging with conflict resolution—address the challenge of creating AI agents with stable yet adaptive identities. The fact/opinion distinction provides epistemic clarity and traceability, while TEMPR's multi-strategy retrieval (temporal, semantic, entity-aware, keyword-based) enables sophisticated memory access. The bias strength parameter provides fine-grained control over personality influence, enabling agents to operate across a spectrum from objective information processors to strongly personality-driven reasoners.
While the framework is implemented and functional, dedicated evaluation is needed to rigorously assess personality consistency, belief evolution dynamics, and multi-agent interactions. Future work will explore personality evolution over time, multi-agent belief systems, and richer personality models incorporating values and cultural factors.
By bringing personality psychology into AI agent design, we move toward conversational agents that exhibit not just intelligence, but character—stable traits and evolving beliefs that enable more natural, trustworthy human-AI interaction.
## References
1. Boschi, N., et al. (2025). TEMPR: Temporal Entity Memory Priming Retrieval for Conversational AI Agents. [Companion paper - see PAPER_RETRIEVAL.md]
2. McCrae, R. R., & Costa, P. T. (1997). Personality trait structure as a human universal. *American Psychologist*, 52(5), 509.
3. Goldberg, L. R. (1993). The structure of phenotypic personality traits. *American Psychologist*, 48(1), 26.
4. Gärdenfors, P. (1988). *Knowledge in flux: Modeling the dynamics of epistemic states*. MIT Press.
5. Friedkin, N. E., & Johnsen, E. C. (1990). Social influence and opinions. *Journal of Mathematical Sociology*, 15(3-4), 193-206.
6. Zhang, S., et al. (2018). Personalizing dialogue agents: I have a dog, do you have pets too? *arXiv preprint arXiv:1801.07243*.

875
PAPER_RETRIEVAL.md
View file

@ -1,875 +0,0 @@
# TEMPR: Temporal Entity Memory Priming Retrieval for Conversational AI Agents
## Abstract
We present TEMPR (Temporal Entity Memory Priming Retrieval), a memory retrieval architecture designed specifically for AI agents that combines temporal range reasoning, entity-aware graph traversal with causal link boosting, and neural priming activation to discover both directly and indirectly related memories through multi-strategy parallel search. Unlike traditional search systems optimized for human queries with top-k ranking, TEMPR is optimized for AI agent reasoning with thinking_budget and max_tokens parameters that enable agents to trade off latency for recall. Our multi-stage retrieval pipeline integrates four parallel search strategies (semantic vector search, BM25 keyword matching, graph-based spreading activation with 2x boost for causal links, and temporal-aware graph traversal with range matching) with reciprocal rank fusion and neural cross-encoder reranking. We leverage open-source LLMs for comprehensive narrative fact extraction with temporal ranges (occurred_start/end vs. mentioned_at), entity recognition, entity disambiguation, and causal relationship identification, following established practices in LLM-based information extraction. This approach enables the discovery of indirectly related information through graph traversal, explanatory reasoning through causal chains, and precise temporal matching through range-based queries that purely vector-based approaches miss. We evaluate TEMPR on two benchmarks (LoComo and LongMemEval), achieving 73.50% overall accuracy on LoComo and 80.60% on LongMemEval, with particularly strong performance on multi-hop reasoning tasks (+15.8% over baseline systems).
## 1. Introduction
Conversational AI agents face a fundamental challenge: maintaining coherent, context-aware memories across extended interactions. Traditional search systems are optimized for human users with top-k ranking and relevance feedback, but AI agents have fundamentally different requirements: they need to retrieve variable amounts of information based on reasoning complexity (thinking_budget) while respecting LLM context windows (max_tokens). Existing approaches rely either on vector similarity search, which captures semantic relationships but misses entity-level connections, or on keyword matching, which provides precision but lacks conceptual understanding. Neither approach adequately handles the temporal aspects of memory or entity-based reasoning that enable multi-hop information discovery.
We propose TEMPR, a memory retrieval architecture designed specifically for AI agents that combines established information retrieval techniques—semantic vector search, BM25 keyword matching, spreading activation graph traversal with causal reasoning (Anderson 1983), and neural reranking—into a unified system optimized for agent workflows. The key architectural choices are:
1. **Agent-Optimized Interface**: thinking_budget and max_tokens parameters instead of traditional top-k ranking
2. **Comprehensive Narrative Fact Extraction with Temporal Ranges**: LLM-powered extraction that creates self-contained narrative facts preserving full conversational context, extracting temporal ranges (occurred_start/end) to distinguish point events from periods, and identifying causal relationships between facts
3. **Entity-Aware Graph Structure with Causal Links**: LLM-based entity resolution and linking that connects memories through shared identities, plus causal links (causes, caused_by, enables, prevents) that capture explanatory relationships
4. **Four-Way Parallel Retrieval with Causal Boosting**: Semantic, keyword, graph-based (spreading activation with 2x causal boost), and temporal range retrieval strategies executed in parallel and fused using RRF (Cormack et al. 2009)
5. **Neural Cross-Encoder Reranking**: Learned query-document relevance with temporal awareness and token budget filtering
This combination of techniques enables agents to discover indirectly related information through graph traversal while maintaining temporal awareness, achieving strong performance on multi-hop reasoning tasks.
### 1.1 Contributions
Our key contributions are:
1. **Agent-Optimized Retrieval Interface**: Unlike traditional top-k search optimized for human users, we introduce thinking_budget and max_tokens parameters that allow AI agents to dynamically trade off latency for recall based on reasoning complexity and context window constraints
2. **Four-Way Parallel Retrieval with Causal Reasoning**: We combine semantic vector search, BM25 keyword matching, graph-based spreading activation with causal link boosting (Anderson 1983), and temporal-aware graph traversal into a unified parallel retrieval pipeline using Reciprocal Rank Fusion (Cormack et al. 2009) and neural cross-encoder reranking. The graph traversal prioritizes causal links (2x weight boost for direct causation) to surface explanatory relationships, enabling "why" and "how" queries. While each technique is well-established, their integration for conversational agent memory with causal reasoning represents a novel application.
3. **LLM-Based Knowledge Graph Construction with Temporal Ranges**: We leverage open-source LLMs (following established practices from Petroni et al. 2019, Brown et al. 2020) for comprehensive narrative fact extraction, entity recognition, entity disambiguation, and causal relationship identification. The system extracts temporal ranges (occurred_start, occurred_end) to represent both point events and extended periods, distinguishing when facts occurred from when they were mentioned, enabling precise temporal queries and recency-aware ranking.
4. **Strong Performance on Multi-Hop Reasoning**: 73.50% on LoComo and 80.60% on LongMemEval, with particularly strong performance on multi-hop queries (+15.8% over Mem0), demonstrating the effectiveness of combining these techniques for discovering indirectly related information in conversational contexts
## 2. System Architecture
### 2.1 Memory Organization
TEMPR stores memories as facts in a graph-structured knowledge base. While the system supports different fact types (e.g., world knowledge, agent actions), the core retrieval mechanism operates uniformly across all types through shared graph infrastructure.
**Memory Unit Structure**:
Each memory is represented as a self-contained node with:
- `id`: Unique UUID
- `agent_id`: Identifier for the agent this memory belongs to
- `text`: Self-contained comprehensive narrative fact
- `embedding`: 384-dimensional vector (BAAI/bge-small-en-v1.5)
- `event_date`: Timestamp when the fact became true (maintained for backward compatibility)
- `occurred_start`: Timestamp when the fact/event started (temporal range support)
- `occurred_end`: Timestamp when the fact/event ended (temporal range support)
- `mentioned_at`: Timestamp when the fact was mentioned/learned
- `context`: Optional contextual metadata
- `access_count`: Frequency-based importance signal
- `search_vector`: Full-text search tsvector for BM25 ranking
**Example Facts**:
- "Alice works at Google in Mountain View on the AI team, which she joined in 2023, and she loves the company culture there."
- "Alice and Bob discussed naming their summer party playlist. Bob suggested 'Summer Vibes' because it's catchy and seasonal, but Alice wanted something more unique. Bob then proposed 'Sunset Sessions' and 'Beach Beats', with Alice favoring 'Beach Beats' for its playful and fun tone. They ultimately decided on 'Beach Beats' as the final name."
- "I recommended Yosemite National Park to Alice for hiking because of the spectacular trails and scenery."
The key innovation is not the type taxonomy, but rather how TEMPR retrieves these memories through temporal reasoning, entity-aware graph traversal, and neural priming activation.
### 2.2 LLM-Powered Comprehensive Narrative Fact Extraction
TEMPR employs **LLM-powered comprehensive narrative fact extraction** using open-source models. This approach, following the trend of using large language models for information extraction (Brown et al. 2020, OpenAI 2023), provides more context-aware extraction compared to traditional rule-based NLP pipelines, though at higher computational cost.
#### 2.2.1 Extraction Principles
**Chunking Strategy**: TEMPR uses a coarse-grained chunking approach, extracting 2-5 comprehensive facts per conversation rather than dozens of atomic fragments. This is a deliberate tradeoff: larger chunks preserve more context and narrative flow, at the cost of reduced precision when only a small portion of the chunk is relevant.
Each fact should:
1. **Capture entire conversations or exchanges** - Include the full back-and-forth discussion
2. **Be narrative and comprehensive** - Tell the complete story with all context
3. **Be self-contained** - Readable without the original text
4. **Include all participants** - WHO said/did WHAT, with their reasoning
5. **Preserve the flow** - Keep related exchanges together in one fact
**Example Comparison**:
❌ **Fragmented Approach** (traditional):
- "Bob suggested Summer Vibes"
- "Alice wanted something unique"
- "They considered Sunset Sessions"
- "Alice likes Beach Beats"
- "They chose Beach Beats"
✅ **Comprehensive Approach** (TEMPR):
- "Alice and Bob discussed naming their summer party playlist. Bob suggested 'Summer Vibes' because it's catchy and seasonal, but Alice wanted something more unique. Bob then proposed 'Sunset Sessions' and 'Beach Beats', with Alice favoring 'Beach Beats' for its playful and fun tone. They ultimately decided on 'Beach Beats' as the final name."
#### 2.2.2 Open-Source LLM Extraction Pipeline
The extraction process leverages open-source LLMs (specifically, models from the OpenAI-OSS 20B family) with structured output (Pydantic schemas). This follows the established practice of using LLMs for information extraction (Petroni et al. 2019, Brown et al. 2020), which has been shown to improve context understanding compared to rule-based NLP pipelines, particularly for:
- Coreference resolution in conversational text
- Domain-specific entity recognition
- Maintaining narrative coherence across multi-turn exchanges
**LLM Extraction Steps**:
1. **Pronoun Resolution**: "She loves hiking" → "Alice loves hiking"
2. **Temporal Normalization**: "last year" → "in 2023" (absolute dates)
3. **Temporal Range Extraction**: Identify when facts occurred vs. when mentioned
- Point events: "on July 14" → occurred_start = occurred_end = 2023-07-14
- Period events: "in February 2023" → occurred_start = 2023-02-01, occurred_end = 2023-02-28
- Vague periods: "lately" → estimated range based on context
- mentioned_at = conversation date (when fact was learned)
4. **Participant Attribution**: Preserve WHO said/did WHAT
5. **Reasoning Preservation**: Include WHY decisions were made
6. **Fact Type Classification**: Determine fact categories
7. **Entity Extraction**: Identify all entities (PERSON, ORG, LOCATION, PRODUCT, CONCEPT)
8. **Causal Relationship Identification**: Link related facts through cause-effect relationships
**Context Preservation**: The system preserves critical details including:
- Visual/media elements (photos, images)
- Modifiers ("new", "first", "favorite")
- Possessive relationships ("their kids" → "Alice's kids")
- Biographical details (origins, jobs, family)
- Social dynamics (nicknames, relationships)
**Noise Filtering**: Automatically filters out:
- Greetings and filler words
- Structural/procedural statements ("let's get started", "that's all for today")
- Meta-commentary about format ("welcome to the show")
- Calls to action ("subscribe and share")
**Why Narrative Chunking Helps Retrieval**:
Traditional semantic chunking (e.g., splitting on sentence or paragraph boundaries) preserves the original text structure but often creates retrieval challenges:
- Important context appears in different sections (e.g., "Alice" mentioned on page 1, "she loves hiking" on page 3)
- Pronouns and references remain ambiguous without surrounding context
- Retrieval requires multiple chunks to answer simple questions
TEMPR's narrative fact extraction rewrites content in a **retrieval-oriented format** that consolidates related information:
- **Coreference Resolution**: "She loves hiking" becomes "Alice loves hiking" - retrievable without needing the introduction chunk
- **Entity Context Consolidation**: All details about an entity scattered across the conversation are gathered into comprehensive facts
- **Self-Contained Narratives**: Each fact includes WHO, WHAT, WHY, WHEN without requiring other chunks for interpretation
**Example**:
- Original text (3 separate chunks):
- Chunk 1: "Alice joined the company last year"
- Chunk 2: "She works in the AI division"
- Chunk 3: "Her manager is Bob Chen"
- TEMPR narrative fact (1 chunk):
- "Alice joined the company in 2023, works in the AI division, and reports to manager Bob Chen"
This retrieval-oriented rewriting means a single retrieved fact provides complete context, reducing the need for multi-hop retrieval in simple cases while still enabling graph traversal for complex queries.
**Tradeoffs**: This chunking strategy trades write-time complexity (LLM processing) and potential over-retrieval (retrieving large chunks when only part is relevant) for improved narrative coherence and reduced fact fragmentation.
**Temporal Augmentation**: Before embedding, facts are augmented with readable temporal information:
- Original: "Alice started working at Google"
- Augmented for embedding: "Alice started working at Google (happened in November 2023)"
This augmentation helps semantic search understand temporal relevance without modifying the stored fact text.
### 2.3 Entity Resolution and Linking
Entity resolution creates strong connections between memories that share common entities, solving the problem where semantically dissimilar facts are related through shared identities.
#### 2.3.1 LLM-Based Entity Recognition
TEMPR uses the same open-source LLM (OpenAI-OSS 20B) that performs fact extraction to also identify and extract entities during the narrative fact creation process. This unified approach eliminates the brittleness of traditional NER pipelines that struggle with domain-specific entities, novel names, and context-dependent disambiguation.
**Entity Types**:
- PERSON: "Alice", "Bob Chen"
- ORGANIZATION: "Google", "Stanford University"
- LOCATION: "Yosemite National Park", "California"
- PRODUCT: "Python", "pandas library"
- CONCEPT: "machine learning", "remote work"
- OTHER: Miscellaneous proper nouns
**Advantages**: This approach maintains consistency with the narrative fact extraction process and can handle domain-specific entities without retraining. However, it comes at higher computational cost compared to traditional NER models.
#### 2.3.2 LLM-Based Entity Disambiguation
Multiple mentions of entities (e.g., "Alice", "Alice Chen", "Alice C.") must be resolved to a single canonical entity. TEMPR uses the same LLM that performs fact extraction to perform entity disambiguation, analyzing the surrounding context to determine if two entity mentions refer to the same entity. This handles complex cases like:
- Nicknames and formal names ("Bob" vs. "Robert Chen")
- Partial mentions ("Alice" vs. "Alice Chen")
- Context-dependent disambiguation ("Apple the company" vs. "apple the fruit")
The LLM considers multiple signals when making disambiguation decisions:
**Name Similarity**:
String similarity using Levenshtein distance to match variations like "Bob" ↔ "Robert", "Google Inc" ↔ "Google"
**Co-occurrence Patterns**:
Entities mentioned together frequently are likely distinct (e.g., "Alice" and "Alice Cooper" appearing together indicates different people)
**Temporal Proximity**:
Recent mentions are more likely to refer to the same entity than mentions separated by long time periods
These signals are presented to the LLM as context, which makes the final disambiguation decision.
#### 2.3.3 Entity Link Structure
Each entity creates a `link_type='entity'` edge between all memories mentioning it:
**Properties**:
- `weight=1.0` (constant, no temporal decay)
- `entity_id`: Reference to resolved canonical entity
- Bidirectional connections between all mentioning memories
**Impact on Retrieval**: Entity links enable graph traversal to discover indirectly related facts:
**Example Query**: "What does Alice do?"
1. **Semantic Match**: "Alice works at Google in Mountain View..." (direct match)
2. **Entity Traversal**: Follow entity links for "Alice" →
- "Alice loves hiking in Yosemite..." (different semantic space)
- "I recommended technical books to Alice" (Agent Network, via "Alice")
3. **Chained Traversal**: Follow "Google" entity →
- "Google's office is in Mountain View has excellent amenities"
This graph connectivity solves the fundamental limitation of vector-only search: two facts can be strongly related through shared entities even when their embeddings are dissimilar.
### 2.4 Link Types and Graph Structure
The memory graph contains three types of edges connecting memory units:
#### 2.4.1 Temporal Links
Temporal links connect memories close in time, enabling temporal reasoning:
**Creation Logic**:
```python
if abs(event_date1 - event_date2) < time_window: # default: 24 hours
weight = max(0.3, 1.0 - (time_diff / time_window))
create_link(unit1, unit2, type='temporal', weight=weight)
```
**Properties**:
- Decays linearly with time distance
- Minimum weight 0.3 to maintain some connectivity
- Enables "What happened around the same time?" queries
- Critical for narrative understanding and sequential reasoning
**Example**: Memories from the same conversation or day cluster together, enabling retrieval of context-adjacent facts.
#### 2.4.2 Semantic Links
Semantic links connect memories with similar meanings:
**Creation Logic**:
```python
similarity = cosine_similarity(embedding1, embedding2)
if similarity > threshold: # default: 0.7
create_link(unit1, unit2, type='semantic', weight=similarity)
```
**Properties**:
- Uses pgvector HNSW index for efficient nearest-neighbor search
- Higher threshold (0.7) than retrieval (0.3) to avoid over-connection
- Weight equals cosine similarity score
- Enables "Tell me about similar topics" queries
**Example**: "Hiking in Yosemite" links to "Mountain climbing", "Trail running", "Outdoor activities"
#### 2.4.3 Entity Links
Entity links (described in Section 2.3.3) create the strongest connections:
**Properties**:
- `weight=1.0` (constant, never decays)
- Connects all memories mentioning the same resolved entity
- Most reliable traversal path during graph search
- Enables "Tell me everything about X" queries
#### 2.4.4 Causal Links
Causal links capture cause-effect relationships between facts, enabling reasoning about why events happened and what their consequences were:
**Creation Logic**:
During fact extraction, the LLM identifies causal relationships between facts extracted from the same conversation. These are stored as directed edges in the graph with specific relationship types.
**Causal Relationship Types**:
- `causes`: This fact directly causes the target fact
- Example: "It rained heavily" → causes → "Game was cancelled"
- `caused_by`: This fact was caused by the target fact (inverse of causes)
- Example: "I spend time in garden" ← caused_by ← "I lost my friend"
- `enables`: This fact enables or allows the target fact to happen
- Example: "I took pottery class" → enables → "I learned to make ceramics"
- `prevents`: This fact prevents or blocks the target fact
- Example: "Road was closed" → prevents → "We couldn't drive to venue"
**Properties**:
- `weight`: Strength of causal relationship ∈ [0.0, 1.0] (default 1.0 for strong causation)
- Directional edges (from cause to effect)
- Created only between facts from the same conversation or closely related temporal contexts
- Used during graph retrieval with higher activation weights than other link types
**Impact on Retrieval**: Causal links are particularly valuable for "why" and "how" queries:
**Example Query**: "Why does Alice spend time in the garden?"
1. **Semantic Match**: "Alice spends time in the garden to find comfort after losing her friend" (direct match)
2. **Causal Traversal**: Follow caused_by links →
- "Alice lost her friend Karlie in February 2023" (causal explanation)
3. **Temporal Context**: Follow temporal links from the loss event →
- "Alice felt grief and sadness about losing Karlie" (emotional context)
This causal graph connectivity enables the system to not just retrieve facts, but to explain *why* things happened by following cause-effect chains.
**Graph Density**: Each memory unit typically has:
- 5-10 temporal links (to nearby memories)
- 3-5 semantic links (to similar content)
- Variable entity links (depending on entity mention frequency)
- 0-3 causal links (when causal relationships are identified)
This multi-layered graph structure enables flexible traversal strategies that balance different types of relatedness.
### 2.5 Handling Contradictions and Outdated Information
Long-term memory systems must handle evolving information where newer facts may contradict or supersede older ones. TEMPR addresses this challenge through temporal awareness and retrieval-time resolution rather than eager fact invalidation.
**Temporal Recency Signals**:
Each memory unit includes multiple temporal dimensions that enable nuanced recency calculations:
- `occurred_start` / `occurred_end`: When the fact/event actually occurred (temporal range)
- Used for temporal queries ("What happened in February?")
- Enables matching both point events and extended periods
- `mentioned_at`: When the fact was mentioned/learned in conversation
- Used for recency bias (newer information often more relevant)
- Distinguishes between "Alice worked at Google in 2020" (occurred) vs. learned in 2024 (mentioned)
- `event_date`: Maintained for backward compatibility (typically = occurred_start)
- `access_count`: Frequency of retrieval (importance signal)
- Temporal links that decay with time distance
**Dual Temporal Model Benefits**:
This separation of "when it occurred" vs. "when we learned about it" enables:
1. **Accurate temporal queries**: "What did Alice do in 2020?" uses occurred_start/end, not mentioned_at
2. **Recency-aware ranking**: Recent mentions get priority, but old events remain discoverable
3. **Hybrid activation**: Combine temporal proximity (occurred) with information freshness (mentioned)
**Retrieval-Time Conflict Resolution**:
Rather than proactively detecting and deleting contradictions (which risks information loss), TEMPR retrieves potentially conflicting facts and relies on the downstream LLM to resolve contradictions based on:
1. **Temporal Ordering**: Facts are presented with their `event_date`, allowing the LLM to identify "Alice worked at Google in 2023" vs. "Alice started at Microsoft in 2024" as a career progression, not a contradiction
2. **Cross-Encoder Reranking**: The neural reranker naturally prioritizes more recent facts when they're semantically similar to older ones, as the date formatting in the input helps the model learn temporal relevance patterns
3. **Graph-Based Evidence**: Entity links surface multiple perspectives (e.g., "Alice loves hiking" from 2023 and "Alice prefers swimming now" from 2024), providing temporal context for preference evolution
**Advantages of Lazy Resolution**:
- **No Information Loss**: Historical facts remain accessible for "What did Alice like in 2023?" queries
- **Context-Dependent**: The LLM determines whether facts contradict (career change) or coexist (evolving preferences)
- **Narrative Preservation**: Comprehensive facts include reasoning ("Alice switched to swimming after injuring her knee hiking"), making contradictions explicit
**Future Directions**:
Explicit confidence scoring and fact update mechanisms could track known supersessions (e.g., "Alice's favorite color changed from blue to green"), but current benchmarks show strong performance with retrieval-time resolution.
## 3. Retrieval Architecture
Our retrieval pipeline addresses the fundamental challenge of long-term memory: achieving both **high recall** (finding all relevant information) and **high precision** (ranking the most relevant items first).
### 3.1 Four-Way Parallel Retrieval
We execute four complementary retrieval strategies in parallel, each capturing different aspects of relevance:
#### 3.1.1 Semantic Retrieval (Vector Similarity)
**Method**: Cosine similarity between query embedding and memory embeddings
**Index**: pgvector HNSW (Hierarchical Navigable Small World)
**Threshold**: ≥ 0.3 similarity
**Implementation**:
```sql
SELECT id, text, event_date, ...,
1 - (embedding <=> $query_emb::vector) AS similarity
FROM memory_units
WHERE agent_id = $agent_id
AND fact_type = $fact_type
AND (1 - (embedding <=> $query_emb::vector)) >= 0.3
ORDER BY embedding <=> $query_emb::vector
LIMIT $thinking_budget
```
**Advantages**:
- Captures conceptual similarity
- Handles synonyms and paraphrasing
- Language-model understanding of meaning
**Limitations**:
- Misses exact proper nouns if not in training data
- Cannot reason about temporal relationships
- Weak at entity disambiguation
**Example**: Query "hiking activities" finds "mountain climbing", "trail running", even if exact words don't match
#### 3.1.2 Keyword Retrieval (BM25 Full-Text Search)
**Method**: PostgreSQL full-text search with BM25 ranking (ts_rank_cd)
**Index**: GIN index on `to_tsvector('english', text)`
**Implementation**:
```sql
SELECT id, text, event_date, ...,
ts_rank_cd(search_vector, to_tsquery('english', $query)) AS bm25_score
FROM memory_units
WHERE agent_id = $agent_id
AND fact_type = $fact_type
AND search_vector @@ to_tsquery('english', $query)
ORDER BY bm25_score DESC
LIMIT $thinking_budget
```
**Advantages**:
- High precision for proper nouns and technical terms
- Exact phrase matching
- Fast execution with GIN index
**Limitations**:
- No semantic understanding
- Requires exact or stemmed matches
- Weak at conceptual queries
**Example**: Query "Google" finds all memories mentioning "Google" even if semantically unrelated
**Complementarity**: Semantic + Keyword achieves >90% recall: vector search catches concepts, BM25 catches exact names.
#### 3.1.3 Graph Retrieval (Spreading Activation)
**Method**: Activation spreading from semantic entry points through the memory graph, following the spreading activation model of memory (Anderson 1983).
**Algorithm**:
```python
1. Get top-5 semantic matches (similarity ≥ 0.5) as entry points
2. Initialize activation: entry_points.activation = similarity_score
3. Use BFS-style queue with activation tracking
4. For each node (up to thinking_budget nodes):
a. Pop highest-activation node from queue
b. If already visited, skip
c. Mark as visited and add to results
d. Get neighbors via links (weight ≥ 0.1)
e. Propagate activation:
neighbor.activation = current.activation × edge.weight × 0.8
f. Add neighbors to queue if activation > 0.1
5. Return all explored nodes with their activation scores
```
**Decay Mechanism**: Activation decays by 0.8 per hop, limiting spread to ~4-5 hops before negligible impact.
**Link Weighting with Causal Boosting**:
During graph traversal, link weights are adjusted based on link type to prioritize high-value relationships:
- **Causal links**: Base weight × 2.0 boost (causes/caused_by) or × 1.5 boost (enables/prevents)
- Highest priority due to direct explanatory power
- "Why?" queries benefit most from causal traversal
- **Entity links**: weight 1.0 (no boost, already strong signal)
- **Semantic links**: weight ∈ [0.7, 1.0] (cosine similarity, no boost)
- **Temporal links**: weight ∈ [0.3, 1.0] (time-based decay, no boost)
**Causal Activation Boost**: When propagating activation through the graph, causal links receive preferential treatment:
```python
if link_type in ('causes', 'caused_by'):
effective_weight = base_weight × 2.0 # Direct causation
elif link_type in ('enables', 'prevents'):
effective_weight = base_weight × 1.5 # Conditional causation
else:
effective_weight = base_weight # Other links
neighbor.activation = current.activation × effective_weight × 0.8
```
This ensures that when the system encounters a fact, it's 2x more likely to also retrieve facts that explain *why* it happened or what it *caused*.
**Advantages**:
- Discovers indirectly related facts through graph connectivity
- Leverages entity links to traverse knowledge graph
- Finds context-adjacent memories via temporal links
**Example**: Query "Alice's work" → Semantic match "Alice works at Google in Mountain View..." → Entity traverse to "Google's office has excellent amenities" → Temporal traverse to "Mountain View has good hiking nearby" → Entity traverse to "Alice loves Yosemite" (discovered indirectly through 3 hops)
#### 3.1.4 Temporal Graph Retrieval (Time-Constrained + Spreading)
**Activation Condition**: Only triggered when temporal constraint detected in query
**Temporal Parsing**: Uses google/flan-t5-small (80M parameters, ~300MB) to extract temporal constraints from natural language queries. The T5 model is fine-tuned with few-shot prompts to convert temporal expressions into structured date ranges:
- "last spring" → 2024-03-01 to 2024-05-31
- "in June" → 2024-06-01 to 2024-06-30 (year inferred from context)
- "last year" → 2024-01-01 to 2024-12-31
- "between March and May" → 2025-03-01 to 2025-05-31
The T5-based approach provides fast inference (~30-50ms on CPU) without requiring pattern matching or regex rules, handling complex temporal expressions like "dogs in June 2023" → 2023-06-01 to 2023-06-30 where both the context and temporal phrase must be parsed together
**Temporal Range Matching**: Facts are matched against time constraints using their temporal range (occurred_start, occurred_end) rather than just a single point:
```python
def fact_matches_time_constraint(fact, query_start, query_end):
# Check if fact's temporal range overlaps with query range
return (fact.occurred_start <= query_end and
fact.occurred_end >= query_start)
```
This enables precise matching of period queries:
- Query: "What happened in February?" matches facts with occurred_start/end overlapping February
- Query: "What did Alice do last spring?" matches facts in March-May range
- Point events (occurred_start == occurred_end) match if within the query range
**Algorithm**:
```python
1. Parse query for temporal constraints → (start_date, end_date)
2. If no temporal constraint detected: skip this retrieval path
3. Find entry points: facts whose temporal range overlaps query range
AND semantic similarity ≥ 0.4
4. Calculate temporal proximity score for each entry point:
# Use temporal anchor (midpoint) for proximity calculation
fact_anchor = (occurred_start + occurred_end) / 2
query_mid = (start_date + end_date) / 2
score = 1.0 - (abs(fact_anchor - query_mid) / range_radius)
5. Spread through temporal and causal links (weight ≥ 0.1):
- Traverse temporal links to stay in time period
- Traverse causal links to find explanations (causes/effects)
- Filter by semantic similarity ≥ 0.4 to maintain relevance
- Propagate temporal scores with decay (0.7)
6. Return results with temporal_score metadata
```
**Key Innovation**: Combines time filtering with semantic relevance to prevent temporal leakage:
- **Without semantic filter**: "What did Alice do in June?" returns ALL June activities (including Bob's, Charlie's, etc.)
- **With semantic filter**: Only returns June activities semantically related to "Alice do" query
**Example**: Query "What did Alice do last spring?"
1. Parse temporal: March 1 - May 31 (previous year)
2. Find spring memories with "Alice" mentions (semantic ≥ 0.4)
3. Spread through temporal links within spring
4. Final filter: semantic ≥ 0.4 to full query
Result: Alice's spring hiking trips, work projects, conversations
### 3.2 Reciprocal Rank Fusion (RRF)
After parallel retrieval, we merge 3-4 ranked lists (semantic, keyword, graph, optional temporal-graph) using Reciprocal Rank Fusion (Cormack et al. 2009), a well-established rank aggregation method:
**Algorithm**:
```
For each memory unit d in union of all retrieval results:
RRF_score(d) = Σ_{i ∈ retrieval_paths} 1 / (k + rank_i(d))
where k = 60 (standard RRF constant)
rank_i(d) = rank of d in retrieval path i (or ∞ if not present)
```
**Advantages over Score-Based Fusion**:
- **Rank-based**: Position matters more than absolute scores (addresses score calibration)
- **Robust to missing items**: Missing from a list contributes 0, not a penalty
- **Multi-evidence weighting**: Items appearing in multiple lists rank higher
**Example**:
- Memory A: rank 1 in semantic, rank 5 in keyword → RRF = 1/61 + 1/65 = 0.0318
- Memory B: rank 3 in semantic, rank 2 in keyword, rank 10 in graph → RRF = 1/63 + 1/62 + 1/70 = 0.0463
Memory B ranks higher despite not being #1 in any single path (multi-evidence)
### 3.3 Neural Cross-Encoder Reranking
After RRF fusion, TEMPR applies neural cross-encoder reranking to refine precision using learned query-document relevance patterns.
**Model**: `cross-encoder/ms-marco-MiniLM-L-6-v2` (pretrained on MS MARCO passage ranking)
**Method**: Neural reranking with query-document pair classification
**Algorithm**:
```python
for each candidate memory unit:
# Format document with temporal context
doc_text = memory.text
if memory.context:
doc_text = f"{memory.context}: {doc_text}"
# Add formatted date for temporal awareness
date_readable = memory.event_date.strftime("%B %d, %Y")
date_iso = memory.event_date.strftime("%Y-%m-%d")
input_text = f"[Date: {date_readable} ({date_iso})] {doc_text}"
# Compute cross-encoder score
raw_score = cross_encoder.predict([(query, input_text)])[0]
normalized_score = sigmoid(raw_score) # → [0, 1]
```
**Date Formatting**: Includes formatted dates in both readable and ISO format to help model understand temporal relevance:
- `"[Date: November 06, 2025 (2025-11-06)] Alice started working at Google"`
**Advantages**:
- Learns query-document relevance patterns from supervised data (MS MARCO)
- Considers full query-document interaction (not just independent scores)
- Temporal awareness through formatted date context
- Significantly improves precision on multi-hop and temporal queries
**Implementation**: Uses cross-encoder neural reranking with ms-marco-MiniLM-L-6-v2 model for all queries
### 3.4 Token Budget Filtering
Final stage applies token budget filtering to limit context window usage:
**Algorithm**:
```python
encoding = tiktoken.get_encoding("cl100k_base") # GPT-4 tokenizer
filtered_results = []
total_tokens = 0
for result in reranked_results:
text = result["text"]
text_tokens = len(encoding.encode(text))
if total_tokens + text_tokens <= max_tokens:
filtered_results.append(result)
total_tokens += text_tokens
else:
break # Stop before exceeding budget
return filtered_results, total_tokens
```
**Token Counting**: Uses tiktoken (cl100k_base encoding for GPT-4) to count only the 'text' field, not metadata.
**Purpose**: Ensures retrieved facts fit within LLM context windows while maximizing information density.
**Example**: With max_tokens=4096 and thinking_budget=100:
- Reranking might return 100 candidates
- Token filtering might select top 25 that fit within 4096 tokens
- Maintains diversity through reranking order (already sorted by relevance)
### 3.5 Complete Retrieval Pipeline
**End-to-End Flow**:
```
1. Query Processing
- Generate embedding (BAAI/bge-small-en-v1.5)
- Parse temporal constraints using T5-small (google/flan-t5-small)
- Determine active retrieval paths (3-way or 4-way)
2. Parallel Retrieval
- Semantic: pgvector HNSW search
- Keyword: PostgreSQL BM25 (ts_rank_cd)
- Graph: Spreading activation from entry points
- Temporal-Graph: (conditional) Time-filtered + semantic spreading
3. RRF Fusion
- Merge 3-4 ranked lists
- Position-based scoring
4. Neural Cross-Encoder Reranking
- Query-document relevance prediction with temporal context
- Batched inference for efficiency
5. Token Budget Filtering
- Truncate to fit context window (default: 4096 tokens)
- Count tokens using tiktoken
```
**Latency Profile**:
TEMPR prioritizes read latency over write latency. Table 1 shows measured latencies for each retrieval stage on the LoComo benchmark dataset (512 queries, measured on [TODO: specify hardware - e.g., M2 MacBook Pro, 32GB RAM, PostgreSQL 15]).
**Table 1: Retrieval Pipeline Latency Breakdown**
| Stage | p50 | p95 | p99 | % of Total |
|-------|-----|-----|-----|------------|
| Query Embedding | [TODO: e.g., 12ms] | [TODO: e.g., 18ms] | [TODO: e.g., 25ms] | [TODO: e.g., 8%] |
| Semantic Search (HNSW) | [TODO: e.g., 35ms] | [TODO: e.g., 62ms] | [TODO: e.g., 89ms] | [TODO: e.g., 23%] |
| BM25 Keyword Search | [TODO: e.g., 8ms] | [TODO: e.g., 15ms] | [TODO: e.g., 23ms] | [TODO: e.g., 5%] |
| Graph Traversal | [TODO: e.g., 42ms] | [TODO: e.g., 78ms] | [TODO: e.g., 112ms] | [TODO: e.g., 28%] |
| Temporal Parsing (T5-small, when triggered) | 30ms | 50ms | 75ms | 10% |
| RRF Fusion | [TODO: e.g., 2ms] | [TODO: e.g., 3ms] | [TODO: e.g., 5ms] | [TODO: e.g., 1%] |
| Cross-Encoder Reranking | [TODO: e.g., 35ms] | [TODO: e.g., 68ms] | [TODO: e.g., 95ms] | [TODO: e.g., 23%] |
| Token Budget Filtering | [TODO: e.g., 3ms] | [TODO: e.g., 5ms] | [TODO: e.g., 8ms] | [TODO: e.g., 2%] |
| **Total (3-way retrieval)** | [TODO: e.g., 148ms] | [TODO: e.g., 234ms] | [TODO: e.g., 312ms] | **100%** |
| **Total (4-way with temporal)** | [TODO: e.g., 168ms] | [TODO: e.g., 265ms] | [TODO: e.g., 358ms] | **100%** |
**Write Path Latency**: Fact insertion is significantly slower due to LLM processing. For a typical 20-turn conversation:
- LLM fact extraction: [TODO: e.g., 2.3s (p50), 4.1s (p95)]
- Entity recognition & resolution: [TODO: e.g., 450ms (p50), 890ms (p95)]
- Graph link construction: [TODO: e.g., 180ms (p50), 320ms (p95)]
- Database insertion: [TODO: e.g., 65ms (p50), 120ms (p95)]
- **Total write latency**: [TODO: e.g., 3.0s (p50), 5.4s (p95)]
The retrieval path achieves [TODO: e.g., <200ms] p50 latency through parallel execution and efficient indexing, while the write path trades latency for extraction quality.
**Guarantees**:
- **High Recall**: Four parallel strategies cast wide net (>95% of relevant memories found)
- **High Precision**: Reranking refines to most relevant results
- **Scalability**: Connection pooling + HNSW index + batching → thousands of memories/second
- **Controlled Token Usage**: Token budget ensures LLM context window limits are respected
## 4. Evaluation
We evaluate TEMPR on two established long-term memory benchmarks: LoComo (Long-term Conversation Memory) and LongMemEval. These benchmarks assess different aspects of conversational memory, including single-hop and multi-hop retrieval, temporal reasoning, and multi-session consistency.
### 4.1 LoComo Benchmark
LoComo evaluates conversational memory systems across four dimensions: single-hop queries (direct fact retrieval), multi-hop queries (reasoning across multiple facts), open-domain queries (diverse knowledge), and temporal queries (time-based retrieval).
**Results**:
| Method | Single Hop J ↑ | Multi-Hop J ↑ | Open Domain J ↑ | Temporal J ↑ | Overall |
|--------|---------------|---------------|-----------------|--------------|---------|
| A-Mem* | 39.79 | 18.85 | 54.05 | 31.08 | 48.38 |
| LangMem | 62.23 | 47.92 | 71.12 | 23.43 | 58.10 |
| Zep (Mem0 paper) | 61.70 | 41.35 | 76.60 | 49.31 | 65.99 |
| Zep (Zep Blog) | - | - | - | - | 75.14 |
| OpenAI | 63.79 | 42.92 | 62.29 | 21.71 | 52.90 |
| Mem0 | 67.13 | 51.15 | 72.93 | 55.51 | 66.88 |
| Mem0 w/ Graph | 65.71 | 47.19 | 75.71 | 58.13 | 68.44 |
| **TEMPR** | **73.20** | **66.90** | **78.60** | **56.30** | **73.50** |
**Analysis**: TEMPR achieves strong performance across all query types:
- **Single-Hop (+6.1% vs Mem0)**: Superior performance on direct queries due to comprehensive narrative facts that include more context per memory unit, and BM25 keyword matching for exact entity names
- **Multi-Hop (+15.8% vs Mem0)**: Largest improvement, demonstrating the effectiveness of graph-based spreading activation for discovering indirectly related information through entity and temporal links. Our ablation study (Section 4.3) confirms this is primarily driven by graph traversal (+14.5 points)
- **Open Domain (+2.9% vs Mem0)**: Strong performance on diverse queries through multi-strategy parallel retrieval (semantic, keyword, graph, temporal)
- **Temporal (-1.8% vs Mem0 w/ Graph)**: Competitive temporal reasoning, with slight decrease attributable to the semantic filtering in temporal queries that prioritizes relevance over pure temporal coverage
**Note on Comparisons**: The "Zep Blog" result (75.14%) comes from a blog post announcement while other Zep results come from academic papers, suggesting potentially inconsistent evaluation methodologies. We report these numbers as published but acknowledge the difficulty in ensuring fair comparison across different evaluation setups. [TODO: Request Zep's evaluation code or run with consistent methodology]
### 4.2 LongMemEval Benchmark
LongMemEval assesses memory systems across six dimensions that capture different aspects of long-term conversation understanding: single-session preferences and assistant/user context, temporal reasoning, multi-session consistency, and knowledge updates.
**Results**:
| Method | Single-Session Preference | Single-Session Assistant | Temporal Reasoning | Multi-Session | Knowledge Update | Single-Session User | Overall |
|--------|--------------------------|-------------------------|-------------------|---------------|-----------------|-------------------|---------|
| Zep gpt-4o-mini | 53.30% | 75.00% | 54.10% | 47.40% | 74.40% | 92.90% | 63.80% |
| Zep gpt-4o | 56.70% | 80.40% | 62.40% | 57.90% | 83.30% | 92.90% | 71.00% |
| **TEMPR** | **83.30%** | **80.40%** | **75.90%** | **75.20%** | **85.90%** | **92.90%** | **80.60%** |
| Mastra gpt-4o (top_k=20) | 46.70% | 100.00% | 75.20% | 76.70% | 84.60% | 97.10% | 80.05% |
**Analysis**: TEMPR achieves competitive performance:
- **Single-Session Preference (+26.6% vs Zep gpt-4o)**: Dramatic improvement, enabled by comprehensive narrative facts that preserve the full context of preference discussions. Our ablation study suggests this is primarily driven by the narrative chunking strategy rather than graph traversal.
- **Temporal Reasoning (+13.5% vs Zep gpt-4o)**: Strong performance through dedicated temporal graph retrieval that combines time filtering with semantic relevance. Ablation study shows temporal retrieval contributes [TODO: e.g., ~7.4 points] on temporal queries.
- **Multi-Session (+17.3% vs Zep gpt-4o)**: Entity-aware graph linking maintains consistency across sessions by connecting memories through shared entities
- **Knowledge Update (+2.6% vs Zep gpt-4o)**: Modest improvement, suggesting this dimension is less dependent on retrieval architecture
The 80.60% overall score represents a 9.6 percentage point improvement over Zep gpt-4o (71.00%), though Mastra achieves comparable performance (80.05%) with higher top-k retrieval and perfect single-session assistant scores. TEMPR's strength lies in balanced performance across all dimensions, particularly in areas requiring complex reasoning (multi-hop, temporal, multi-session).
[TODO: Statistical significance testing - run bootstrap resampling or multiple evaluation runs to establish confidence intervals]
### 4.3 Ablation Study
To validate the contribution of each architectural component, we conducted systematic ablation experiments on the LoComo benchmark. Table 2 shows the impact of removing individual retrieval strategies.
**Table 2: Ablation Study - Retrieval Strategy Contribution (LoComo)**
| Configuration | Single-Hop | Multi-Hop | Open Domain | Temporal | Overall | Δ Overall |
|---------------|------------|-----------|-------------|----------|---------|-----------|
| Full TEMPR | 73.20 | 66.90 | 78.60 | 56.30 | 73.50 | - |
| - Graph Traversal | [TODO: e.g., 72.10] | [TODO: e.g., 52.40] | [TODO: e.g., 76.80] | [TODO: e.g., 54.20] | [TODO: e.g., 68.30] | [TODO: e.g., -5.2] |
| - BM25 Keyword | [TODO: e.g., 69.50] | [TODO: e.g., 63.20] | [TODO: e.g., 74.10] | [TODO: e.g., 53.80] | [TODO: e.g., 70.40] | [TODO: e.g., -3.1] |
| - Temporal Retrieval | [TODO: e.g., 72.90] | [TODO: e.g., 65.80] | [TODO: e.g., 78.20] | [TODO: e.g., 48.90] | [TODO: e.g., 71.80] | [TODO: e.g., -1.7] |
| Vector Only (no BM25, no graph, no temporal) | [TODO: e.g., 65.40] | [TODO: e.g., 48.60] | [TODO: e.g., 70.30] | [TODO: e.g., 45.20] | [TODO: e.g., 63.20] | [TODO: e.g., -10.3] |
| Simple 2-Hop Neighbors (vs. Spreading Activation) | [TODO: e.g., 72.80] | [TODO: e.g., 61.30] | [TODO: e.g., 77.90] | [TODO: e.g., 55.40] | [TODO: e.g., 71.60] | [TODO: e.g., -1.9] |
**Key Findings**:
1. **Graph Traversal is Critical for Multi-Hop**: Removing graph traversal causes the largest drop in multi-hop performance ([TODO: e.g., -14.5 points]), validating that entity-aware graph connections enable discovery of indirectly related information. Single-hop queries are minimally affected, as expected.
2. **BM25 Improves Entity Precision**: Removing BM25 keyword search primarily impacts single-hop queries ([TODO: e.g., -3.7 points]), where exact entity name matching is crucial. This validates the complementary nature of semantic and keyword-based retrieval.
3. **Temporal Retrieval Handles Time Queries**: The largest impact of removing temporal retrieval is on temporal queries ([TODO: e.g., -7.4 points]), though the overall impact is modest since only [TODO: e.g., ~25%] of queries contain temporal constraints.
4. **Spreading Activation vs. K-Hop**: Our spreading activation mechanism outperforms simple 2-hop neighbor retrieval by [TODO: e.g., 1.9 points] overall, with the largest gain on multi-hop queries ([TODO: e.g., +5.6 points]). This suggests the weighted activation decay provides better ranking than uniform K-hop expansion.
**Reranking Strategy Comparison**:
| Reranker | Single-Hop | Multi-Hop | Open Domain | Temporal | Overall | Latency (p50) |
|----------|------------|-----------|-------------|----------|---------|---------------|
| Cross-Encoder (current) | 73.20 | 66.90 | 78.60 | 56.30 | 73.50 | [TODO: e.g., 148ms] |
| No Reranking (RRF only) | [TODO: e.g., 70.40] | [TODO: e.g., 63.20] | [TODO: e.g., 75.80] | [TODO: e.g., 53.70] | [TODO: e.g., 70.30] | [TODO: e.g., 112ms] |
Cross-encoder reranking provides [TODO: e.g., +3.2 points] improvement at the cost of [TODO: e.g., ~36ms] additional latency per query.
### 4.4 Computational Cost Analysis
We measured the total cost of running TEMPR on the LoComo benchmark dataset (512 queries, [TODO: e.g., 2,847] facts extracted from [TODO: e.g., 342] conversations). All costs are for [TODO: specify deployment - e.g., "single-node PostgreSQL 15 on M2 MacBook Pro"].
**Table 3: Cost Breakdown for LoComo Benchmark Evaluation**
| Cost Component | Per Query | Total (512 queries) | Notes |
|----------------|-----------|---------------------|-------|
| **LLM Costs** | | | |
| Fact Extraction (write-time) | [TODO: e.g., $0.0032] | [TODO: e.g., $1.64] | OpenAI-OSS 20B, [TODO: e.g., ~1.2K] tokens/conversation |
| Entity Disambiguation (write-time) | [TODO: e.g., $0.0008] | [TODO: e.g., $0.41] | Only for borderline cases ([TODO: e.g., ~15%] of entities) |
| Temporal Parsing (T5-small, query-time) | $0.0000 | $0.00 | Local inference, no API cost |
| **Subtotal LLM** | [TODO: e.g., $0.0044] | [TODO: e.g., $2.25] | |
| **Embedding Costs** | | | |
| Fact Embeddings (write-time) | [TODO: e.g., $0.0002] | [TODO: e.g., $0.10] | BAAI/bge-small-en-v1.5 (local inference) |
| Query Embeddings (query-time) | [TODO: e.g., $0.0001] | [TODO: e.g., $0.05] | Same model |
| **Subtotal Embedding** | [TODO: e.g., $0.0003] | [TODO: e.g., $0.15] | |
| **Compute Costs** | | | |
| Database Queries (PostgreSQL) | [TODO: e.g., $0.0001] | [TODO: e.g., $0.05] | HNSW index, BM25, graph traversal |
| Cross-Encoder Reranking | [TODO: e.g., $0.0003] | [TODO: e.g., $0.15] | Local GPU inference (ms-marco-MiniLM) |
| **Subtotal Compute** | [TODO: e.g., $0.0004] | [TODO: e.g., $0.20] | |
| **Storage Costs** | | | |
| PostgreSQL Storage | - | [TODO: e.g., $0.08] | [TODO: e.g., 2,847] facts, [TODO: e.g., ~850K] tokens total |
| HNSW Index Size | - | [TODO: e.g., $0.12] | 384-dim vectors, [TODO: e.g., ~4.2MB] |
| Graph Links (edges) | - | [TODO: e.g., $0.03] | [TODO: e.g., ~18K] edges |
| **Subtotal Storage** | - | [TODO: e.g., $0.23] | |
| **Total Cost** | [TODO: e.g., $0.0051] | [TODO: e.g., $2.83] | |
**Cost Breakdown by Phase**:
- **Write Phase** (one-time per conversation): [TODO: e.g., $0.0042] per conversation ([TODO: e.g., $1.44] total for 342 conversations)
- **Read Phase** (per query): [TODO: e.g., $0.0009] per query ([TODO: e.g., $0.46] total for 512 queries)
**Storage Overhead Analysis**:
We compared TEMPR's narrative fact extraction against atomic fact extraction on a subset of [TODO: e.g., 50] conversations:
| Extraction Strategy | Facts Created | Avg Tokens/Fact | Total Tokens | Storage Size |
|---------------------|---------------|-----------------|--------------|--------------|
| Atomic (baseline) | [TODO: e.g., 847] | [TODO: e.g., 42] | [TODO: e.g., 35,574] | [TODO: e.g., 142KB] |
| TEMPR (narrative) | [TODO: e.g., 218] | [TODO: e.g., 156] | [TODO: e.g., 34,008] | [TODO: e.g., 136KB] |
| Reduction | [TODO: e.g., 3.9x fewer] | [TODO: e.g., 3.7x larger] | [TODO: e.g., 1.04x] | [TODO: e.g., 1.04x] |
**Note**: Narrative facts reduce fact count by [TODO: e.g., ~3.9x] but increase individual fact size by [TODO: e.g., ~3.7x], resulting in similar total storage with improved retrieval coherence.
### 4.5 Limitations and Future Work
**Remaining Limitations**:
1. **Limited Benchmark Coverage**: We evaluate on two benchmarks (LoComo, LongMemEval) representing the available systems with published results on these specific benchmarks. Additional evaluation on other conversational memory benchmarks would strengthen the generalizability claims. [TODO: Statistical significance testing - run bootstrap resampling to establish confidence intervals]
2. **Chunking Strategy Validation**: While our results suggest narrative facts improve retrieval quality, we do not provide controlled experiments directly comparing atomic fact extraction vs. narrative fact extraction with the same retrieval architecture. [TODO: Implement atomic fact extraction baseline and compare on same benchmark with controlled chunk sizes (50, 100, 200 tokens)]
3. **Hyperparameter Sensitivity**: Design choices (similarity thresholds, activation decay rates) were determined empirically without systematic sensitivity analysis to understand their impact on performance.
These limitations suggest directions for future work to further validate the individual contributions and establish cost-benefit tradeoffs more rigorously.
## 5. Related Work
**Vector-Based Memory Systems**: Traditional approaches like Pinecone, Weaviate, and Chroma focus primarily on semantic vector search. While effective for conceptual similarity, they struggle with exact entity matches and multi-hop reasoning.
**Hybrid Retrieval**: Recent work on combining dense and sparse retrieval (ColBERT, SPLADE) has shown promise. TEMPR extends this by adding graph-based and temporal dimensions to the retrieval mix.
**Knowledge Graphs for Memory**: Graph-based memory systems like MemoryNet and GraphMemory use knowledge graphs for structured memory. TEMPR differs by automatically constructing the graph through entity resolution rather than requiring structured input.
**Conversational Memory**: Systems like Zep, Mem0, and LangMem focus on conversational memory but primarily use atomic fact extraction and vector search. TEMPR's comprehensive narrative approach and multi-strategy retrieval provides substantial improvements in multi-hop reasoning.
## 6. Future Work
**Hierarchical Memory Organization**:
- Summarization of old memories into higher-level abstractions
- Multi-resolution retrieval (detailed recent + summarized distant past)
**Cross-Agent Memory Sharing**:
- Controlled sharing of world facts between agents
- Privacy-preserving memory isolation
**Multi-Modal Memory**:
- Image embeddings for visual memories
- Audio/video content integration
**Advanced Entity Resolution**:
- Deep learning-based entity disambiguation
- Cross-document coreference resolution
**Adaptive Retrieval**:
- Query-dependent strategy weighting
- Learning optimal retrieval mix from user feedback
## 7. Conclusion
TEMPR presents a comprehensive memory retrieval architecture for conversational AI agents that addresses the fundamental challenges of long-term memory: maintaining high recall through parallel multi-strategy retrieval while achieving high precision through neural reranking and token-aware filtering. The coarse-grained chunking strategy preserves conversational context through narrative facts, and explicit entity resolution with graph-based traversal enables discovery of indirectly related information that pure vector approaches miss.
The system's modular design—with separate but interconnected world, agent, and opinion networks—provides flexibility for different use cases while maintaining coherent reasoning across memory types. By combining classical information retrieval techniques (BM25, graph search) with modern neural methods (embeddings, cross-encoders), we achieve a robust system that balances interpretability, performance, and accuracy.
Evaluation on LoComo and LongMemEval benchmarks demonstrates strong performance, particularly on multi-hop reasoning tasks (+15.8% over Mem0 on LoComo). Ablation studies confirm that graph traversal contributes [TODO: e.g., +14.5 points] to multi-hop performance, and spreading activation outperforms simpler 2-hop neighbor retrieval by [TODO: e.g., +5.6 points] on multi-hop queries. Cost analysis shows TEMPR processes the LoComo benchmark at [TODO: e.g., $0.0051] per query, with [TODO: e.g., ~85%] of cost in write-time LLM extraction and [TODO: e.g., ~15%] in query-time retrieval. Future work should include comparison with recent systems (LlamaIndex, LangChain), statistical significance testing, and formal entity resolution evaluation.
Future work will explore hierarchical memory organization, cross-agent memory sharing, and multi-modal memory integration to further enhance the system's capabilities.
## References
1. Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019). Language models as knowledge bases?. In *Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)* (pp. 2463-2473).
2. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. *Advances in Neural Information Processing Systems*, 33, 1877-1901.
3. OpenAI. (2023). GPT-4 Technical Report. *arXiv preprint arXiv:2303.08774*.
4. Malkov, Y. A., & Yashunin, D. A. (2018). Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. *IEEE Transactions on Pattern Analysis and Machine Intelligence*, 42(4), 824-836.
5. Robertson, S., & Zaragoza, H. (2009). The probabilistic relevance framework: BM25 and beyond. *Foundations and Trends in Information Retrieval*, 3(4), 333-489.
6. Cormack, G. V., Clarke, C. L., & Buettcher, S. (2009). Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In *SIGIR'09* (pp. 758-759).
7. Craswell, N., Mitra, B., Yilmaz, E., & Campos, D. (2020). Overview of the TREC 2019 deep learning track. *arXiv preprint arXiv:2003.07820*.
8. Anderson, J. R. (1983). A spreading activation theory of memory. *Journal of Verbal Learning and Verbal Behavior*, 22(3), 261-295.

155
docker/README.md Normal file
View file

@ -0,0 +1,155 @@
# Hindsight Docker
Run Hindsight with Docker in standalone or distributed mode.
## Quick Start (Standalone)
```bash
cd docker
./start.sh
```
**Force rebuild after code changes:**
```bash
./start.sh --build # Quick: rebuild and start
# or
./rebuild.sh # Complete: rebuild from scratch (no cache)
```
Access:
- **Control Plane**: http://localhost:3000
- **API**: http://localhost:8888
Press `Ctrl+C` to stop.
## What You Get
**Standalone** (default, simple):
- One container with API + Control Plane + embedded database
- Perfect for local development and simple deployments
**Distributed** (advanced):
- Separate containers for API and Control Plane
- Better for production, scaling, or custom configurations
## Deployment Modes
### 1. Standalone (Recommended)
All-in-one container with embedded pg0 database.
```bash
./start.sh
# or
cd standalone
docker-compose up
```
**Data storage:** `/app/data` volume
### 2. Distributed (Advanced)
Separate API and Control Plane containers.
```bash
cd services
docker-compose up
```
**Data storage:** `api_data` volume
See `services/README.md` for details.
## Data Management
**Reset data:**
```bash
# Standalone
cd standalone && docker-compose down -v
# Distributed
cd services && docker-compose down -v
```
## Building Images
```bash
# Standalone
cd standalone
docker build -f Dockerfile -t hindsight:latest ../..
# Services
cd services
./build-all.sh
```
## Using External Database
Both modes use embedded pg0 by default. To use external PostgreSQL:
```bash
export HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/db
```
## Directory Structure
```
docker/
├── start.sh # Quick start (standalone)
├── README.md # This file
├── standalone/ # All-in-one deployment
│ ├── Dockerfile
│ ├── docker-compose.yml
│ └── start-all.sh
└── services/ # Distributed deployment
├── docker-compose.yml
├── api.Dockerfile
├── control-plane.Dockerfile
├── build-all.sh
└── README.md
```
## Advanced Usage
**Background mode:**
```bash
cd standalone
docker-compose up -d
docker-compose logs -f
docker-compose down
```
**Custom configuration:**
Edit `standalone/docker-compose.yml` or `services/docker-compose.yml`
## Environment Variables
Hindsight requires configuration through environment variables (all prefixed with `HINDSIGHT_`).
### Required:
- `HINDSIGHT_API_LLM_API_KEY` - Your LLM API key (OpenAI, Anthropic, etc.)
### Optional:
- `HINDSIGHT_API_LLM_MODEL` - Model name (default: gpt-4o-mini)
- `HINDSIGHT_API_LLM_BASE_URL` - API base URL (default: https://api.openai.com/v1)
- `HINDSIGHT_API_LOG_LEVEL` - Logging level: debug, info, warning, error
- `HINDSIGHT_API_DATABASE_URL` - External PostgreSQL connection (uses embedded pg0 by default)
### Setup Options:
**Option 1: .env file (recommended)**
```bash
# Copy example file
cp .env.example .env
# Edit .env and add your API key
HINDSIGHT_API_LLM_API_KEY=sk-...
```
**Option 2: Export in shell**
```bash
export HINDSIGHT_API_LLM_API_KEY=sk-...
export HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
```
The `start.sh` script automatically loads `.env` if it exists and validates the API key is set.

51
docker/api.Dockerfile
View file

@ -1,51 +0,0 @@
FROM python:3.11-slim AS builder
# Install build dependencies
RUN apt-get update && apt-get install -y \
build-essential \
&& rm -rf /var/lib/apt/lists/*
# Set working directory
WORKDIR /app
# Copy only dependency files first for better caching
COPY hindsight-api/pyproject.toml hindsight-api/README.md /app/hindsight-api/
COPY hindsight-api/hindsight_api /app/hindsight-api/hindsight_api
COPY hindsight-api/alembic /app/hindsight-api/alembic
# Install uv for faster dependency installation
RUN pip install --no-cache-dir uv
# Install Python dependencies to a virtual environment
WORKDIR /app/hindsight-api
RUN uv venv /opt/venv && \
. /opt/venv/bin/activate && \
uv pip install --no-cache -e .
# Production stage
FROM python:3.11-slim
# Install only runtime dependencies
RUN apt-get update && apt-get install -y \
curl \
libgomp1 \
&& rm -rf /var/lib/apt/lists/*
# Copy virtual environment from builder
COPY --from=builder /opt/venv /opt/venv
COPY --from=builder /app/hindsight-api /app/hindsight-api
# Set working directory
WORKDIR /app/hindsight-api
# Expose API port
EXPOSE 8888
# Set environment variables
ENV PYTHONUNBUFFERED=1
ENV DATABASE_URL=postgresql://hindsight:hindsight_dev@postgres:5432/hindsight
ENV PATH="/opt/venv/bin:$PATH"
ENV PYTHONPATH=/app/hindsight-api
# Run the API server
CMD ["python", "-m", "hindsight_api.web.server", "--host", "0.0.0.0", "--port", "8888"]

27
docker/clean.sh
View file

@ -1,27 +0,0 @@
#!/bin/bash
set -e
cd "$(dirname "$0")"
echo "🧹 Cleaning Services"
echo "============================"
echo ""
echo "This will:"
echo " - Stop all services"
echo " - Remove containers"
echo " - Remove volumes (ALL DATA WILL BE LOST)"
echo ""
read -p "Are you sure? (yes/no): " confirm
if [ "$confirm" != "yes" ]; then
echo "Cancelled."
exit 0
fi
echo ""
echo "🗑️ Removing services and data..."
docker compose down -v
echo ""
echo "✅ All services and data removed"
echo ""

39
docker/control-plane.Dockerfile
View file

@ -1,39 +0,0 @@
FROM node:20-alpine AS base
# Install dependencies only when needed
FROM base AS deps
RUN apk add --no-cache libc6-compat
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
# Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build
# Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app
ENV NODE_ENV=production
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
# Automatically leverage output traces to reduce image size
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
USER nextjs
EXPOSE 9999
ENV PORT=9999
ENV HOSTNAME="0.0.0.0"
CMD ["node", "server.js"]

79
docker/docker-compose.yml
View file

@ -1,79 +0,0 @@
services:
postgres:
image: pgvector/pgvector:pg16
container_name: hindsight-postgres
environment:
POSTGRES_USER: hindsight
POSTGRES_PASSWORD: hindsight_dev
POSTGRES_DB: hindsight
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U hindsight"]
interval: 5s
timeout: 5s
retries: 5
networks:
- hindsight-network
api:
build:
context: ..
dockerfile: docker/api.Dockerfile
container_name: hindsight-api
environment:
HINDSIGHT_API_DATABASE_URL: postgresql://hindsight:hindsight_dev@postgres:5432/hindsight
HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-groq}
HINDSIGHT_API_LLM_API_KEY: ${HINDSIGHT_API_LLM_API_KEY}
HINDSIGHT_API_LLM_MODEL: ${HINDSIGHT_API_LLM_MODEL:-openai/gpt-oss-20b}
HINDSIGHT_API_LLM_BASE_URL: ${HINDSIGHT_API_LLM_BASE_URL}
HINDSIGHT_API_HOST: 0.0.0.0
HINDSIGHT_API_PORT: 8888
ports:
- "8888:8888"
depends_on:
postgres:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8888/api/v1/agents"]
interval: 10s
timeout: 5s
retries: 5
start_period: 30s
networks:
- hindsight-network
restart: unless-stopped
control-plane:
build:
context: ../hindsight-control-plane
dockerfile: ../docker/control-plane.Dockerfile
container_name: hindsight-control-plane
environment:
NODE_ENV: production
HOSTNAME: 0.0.0.0
PORT: 9999
HINDSIGHT_CP_DATAPLANE_API_URL: http://api:8888
ports:
- "9999:9999"
depends_on:
api:
condition: service_healthy
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://localhost:9999/"]
interval: 10s
timeout: 5s
retries: 5
start_period: 30s
networks:
- hindsight-network
restart: unless-stopped
networks:
hindsight-network:
driver: bridge
volumes:
postgres_data:

16
docker/logs.sh
View file

@ -1,16 +0,0 @@
#!/bin/bash
set -e
cd "$(dirname "$0")"
SERVICE=$1
if [ -z "$SERVICE" ]; then
echo "📋 Showing logs for all services..."
echo ""
docker compose logs -f
else
echo "📋 Showing logs for $SERVICE..."
echo ""
docker compose logs -f "$SERVICE"
fi

16
docker/rebuild.sh Executable file
View file

@ -0,0 +1,16 @@
#!/bin/bash
# Rebuild Hindsight images from scratch
cd "$(dirname "$0")/standalone"
echo "🔨 Rebuilding Hindsight images (no cache)..."
echo ""
# Build with no cache to force complete rebuild
docker-compose build --no-cache
echo ""
echo "✅ Rebuild complete!"
echo ""
echo "To start Hindsight:"
echo " ./start.sh"

59
docker/services/README.md Normal file
View file

@ -0,0 +1,59 @@
# Distributed Hindsight Setup
Run API and Control Plane as separate containers.
## Start
```bash
cd services
docker-compose up
```
Access:
- **Control Plane**: http://localhost:3000
- **API**: http://localhost:8888
## What's Running
Two separate containers:
- `api` - Hindsight API with embedded pg0 database
- `control-plane` - Web UI
## Build Images
```bash
./build-all.sh
```
Creates:
- `hindsight/api:latest`
- `hindsight/control-plane:latest`
## Configuration
The API uses embedded pg0 by default. Database files are stored in the `api_data` volume.
To use an external PostgreSQL database, add to `docker-compose.yml`:
```yaml
services:
api:
environment:
HINDSIGHT_API_DATABASE_URL: postgresql://user:pass@host:5432/db
```
## Data Persistence
```bash
docker-compose down -v # Remove volumes
```
## Why Use This?
The distributed setup is useful when you want to:
- Scale API and UI independently
- Use an external database in production
- Deploy to Kubernetes/orchestration
- Run UI on different infrastructure
For simple deployments, use the main `docker-compose.yml` (standalone all-in-one).

33
docker/services/api.Dockerfile Normal file
View file

@ -0,0 +1,33 @@
# Dockerfile for Hindsight API (standalone)
FROM python:3.11-slim
WORKDIR /app
# Install system dependencies and uv
RUN apt-get update && apt-get install -y \
gcc \
g++ \
&& rm -rf /var/lib/apt/lists/* \
&& pip install --no-cache-dir uv
# Copy dependency files and README (required by pyproject.toml)
COPY hindsight-api/pyproject.toml ./
COPY hindsight-api/README.md ./
# Sync dependencies (creates lock file if needed)
RUN uv sync
# Copy source code
COPY hindsight-api/hindsight_api ./hindsight_api
# Expose API port
EXPOSE 8888
# Set environment variables
ENV HINDSIGHT_API_HOST=0.0.0.0
ENV HINDSIGHT_API_PORT=8888
ENV HINDSIGHT_API_LOG_LEVEL=info
ENV PATH="/app/.venv/bin:$PATH"
# Run the API server
CMD ["python", "-m", "hindsight_api.web.server"]

24
docker/services/build-all.sh Executable file
View file

@ -0,0 +1,24 @@
#!/bin/bash
set -e
echo "Building Hindsight service images..."
cd "$(dirname "$0")/../.."
echo ""
echo "Building hindsight-api..."
docker build -f docker/services/api.Dockerfile -t hindsight/api:latest .
echo ""
echo "Building hindsight-control-plane..."
docker build -f docker/services/control-plane.Dockerfile -t hindsight/control-plane:latest .
echo ""
echo "✅ All service images built successfully!"
echo ""
echo "Available images:"
echo " - hindsight/api:latest"
echo " - hindsight/control-plane:latest"
echo ""
echo "To start all services:"
echo " cd docker && docker-compose up"

65
docker/services/control-plane.Dockerfile Normal file
View file

@ -0,0 +1,65 @@
# Dockerfile for Hindsight Control Plane (standalone)
FROM node:20-alpine AS sdk-builder
WORKDIR /app/sdk
# Build TypeScript SDK
COPY hindsight-clients/typescript/package*.json ./
RUN npm ci
COPY hindsight-clients/typescript/ ./
RUN npm run build
# Build Control Plane
FROM node:20-alpine AS builder
WORKDIR /app
# Copy built SDK
COPY --from=sdk-builder /app/sdk /app/sdk
# Install Control Plane dependencies
COPY hindsight-control-plane/package*.json ./
RUN npm ci
# Copy Control Plane source
COPY hindsight-control-plane/ ./
# Link SDK for build
RUN cd /app/sdk && npm link && cd /app && npm link @hindsight/client
# Build the Next.js app
RUN npm run build
# Create public directory if it doesn't exist
RUN mkdir -p public
# Production image
FROM node:20-alpine
WORKDIR /app
# Copy built SDK
COPY --from=sdk-builder /app/sdk /app/sdk
# Copy package files and install production dependencies only
COPY hindsight-control-plane/package*.json ./
RUN npm ci --omit=dev
# Link SDK for runtime
RUN cd /app/sdk && npm link && cd /app && npm link @hindsight/client
# Copy built app from builder
COPY --from=builder /app/.next ./.next
COPY --from=builder /app/public ./public
COPY --from=builder /app/next.config.ts ./next.config.ts
# Expose control plane port
EXPOSE 3000
# Set environment variables
ENV NODE_ENV=production
ENV HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888
# Run the Next.js server
CMD ["npm", "start"]

42
docker/services/docker-compose.yml Normal file
View file

@ -0,0 +1,42 @@
services:
api:
build:
context: ../..
dockerfile: docker/services/api.Dockerfile
ports:
- "8888:8888"
environment:
# Pass through all HINDSIGHT_* environment variables
HINDSIGHT_API_LLM_API_KEY: ${HINDSIGHT_API_LLM_API_KEY:-}
HINDSIGHT_API_LLM_MODEL: ${HINDSIGHT_API_LLM_MODEL:-}
HINDSIGHT_API_LLM_BASE_URL: ${HINDSIGHT_API_LLM_BASE_URL:-}
HINDSIGHT_API_HOST: ${HINDSIGHT_API_HOST:-0.0.0.0}
HINDSIGHT_API_PORT: ${HINDSIGHT_API_PORT:-8888}
HINDSIGHT_API_LOG_LEVEL: ${HINDSIGHT_API_LOG_LEVEL:-info}
HINDSIGHT_API_DATABASE_URL: ${HINDSIGHT_API_DATABASE_URL:-}
volumes:
- api_data:/app/data
networks:
- hindsight
restart: unless-stopped
control-plane:
build:
context: ../..
dockerfile: docker/services/control-plane.Dockerfile
ports:
- "3000:3000"
environment:
NODE_ENV: production
HINDSIGHT_CP_DATAPLANE_API_URL: http://api:8888
depends_on:
- api
networks:
- hindsight
restart: unless-stopped
volumes:
api_data:
networks:
hindsight:

114
docker/standalone/Dockerfile Normal file
View file

@ -0,0 +1,114 @@
# Standalone All-in-One Hindsight Image
# API with embedded pg0 + Control Plane
FROM python:3.11-slim AS api-base
WORKDIR /app
# Install system dependencies and uv
RUN apt-get update && apt-get install -y \
gcc \
g++ \
curl \
&& rm -rf /var/lib/apt/lists/* \
&& pip install --no-cache-dir uv
# Copy dependency files and README (required by pyproject.toml)
COPY hindsight-api/pyproject.toml ./api/
COPY hindsight-api/README.md ./api/
WORKDIR /app/api
# Sync dependencies (will create lock file if needed)
RUN uv sync
# Copy source code
COPY hindsight-api/hindsight_api ./hindsight_api
# Build TypeScript SDK
FROM node:20-alpine AS sdk-builder
WORKDIR /app/sdk
COPY hindsight-clients/typescript/package*.json ./
RUN npm ci
COPY hindsight-clients/typescript/ ./
RUN npm run build
# Build Control Plane
FROM node:20-alpine AS cp-builder
WORKDIR /app
# Copy built SDK
COPY --from=sdk-builder /app/sdk /app/sdk
# Install Control Plane dependencies
COPY hindsight-control-plane/package*.json ./
RUN npm ci
# Copy Control Plane source
COPY hindsight-control-plane/ ./
# Link SDK (temporary for build)
RUN cd /app/sdk && npm link && cd /app && npm link @hindsight/client
# Build Control Plane
RUN npm run build
# Create public directory if it doesn't exist
RUN mkdir -p public
# Final standalone image
FROM python:3.11-slim
WORKDIR /app
# Install Node.js, curl, and uv
RUN apt-get update && apt-get install -y \
curl \
&& curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
&& apt-get install -y nodejs \
&& rm -rf /var/lib/apt/lists/* \
&& pip install --no-cache-dir uv
# Copy API with virtual environment from builder
COPY --from=api-base /app/api /app/api
# Copy built SDK
COPY --from=sdk-builder /app/sdk /app/sdk
# Copy Control Plane
WORKDIR /app/control-plane
COPY --from=cp-builder /app/package*.json ./
RUN npm ci --omit=dev
# Link SDK for runtime
RUN cd /app/sdk && npm link && cd /app/control-plane && npm link @hindsight/client
COPY --from=cp-builder /app/.next ./.next
COPY --from=cp-builder /app/public ./public
COPY --from=cp-builder /app/next.config.ts ./next.config.ts
WORKDIR /app
# Copy startup script
COPY docker/standalone/start-all.sh /app/start-all.sh
RUN chmod +x /app/start-all.sh
# Create data directory for pg0
RUN mkdir -p /app/data
# Expose ports
EXPOSE 8888 3000
# Environment variables
ENV HINDSIGHT_API_HOST=0.0.0.0
ENV HINDSIGHT_API_PORT=8888
ENV HINDSIGHT_API_LOG_LEVEL=info
ENV NODE_ENV=production
ENV HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888
ENV PATH="/app/api/.venv/bin:$PATH"
# Run startup script
CMD ["/app/start-all.sh"]

29
docker/standalone/docker-compose.yml Normal file
View file

@ -0,0 +1,29 @@
services:
hindsight:
build:
context: ../..
dockerfile: docker/standalone/Dockerfile
platforms:
- linux/amd64
platform: linux/amd64
ports:
- "3000:3000"
- "8888:8888"
environment:
# Pass through all HINDSIGHT_* environment variables from host
HINDSIGHT_API_LLM_API_KEY: ${HINDSIGHT_API_LLM_API_KEY:-}
HINDSIGHT_API_LLM_MODEL: ${HINDSIGHT_API_LLM_MODEL:-}
HINDSIGHT_API_LLM_BASE_URL: ${HINDSIGHT_API_LLM_BASE_URL:-}
HINDSIGHT_API_LLM_PROVIDER: ${HINDSIGHT_API_LLM_PROVIDER:-}
HINDSIGHT_API_HOST: ${HINDSIGHT_API_HOST:-0.0.0.0}
HINDSIGHT_API_PORT: ${HINDSIGHT_API_PORT:-8888}
HINDSIGHT_API_LOG_LEVEL: ${HINDSIGHT_API_LOG_LEVEL:-info}
# HINDSIGHT_API_DATABASE_URL can be set if you want to use an external database
# If not set, embedded pg0 will be used automatically
# Add any other HINDSIGHT_* vars you need here
volumes:
- hindsight_data:/app/data
restart: unless-stopped
volumes:
hindsight_data:

41
docker/standalone/start-all.sh Executable file
View file

@ -0,0 +1,41 @@
#!/bin/bash
set -e
echo "🚀 Starting Hindsight..."
echo ""
# Start API (with embedded pg0)
echo "⚡ Starting Hindsight API (with embedded database)..."
cd /app/api
python -m hindsight_api.web.server &
API_PID=$!
# Wait for API to be ready
echo "⏳ Waiting for API..."
for i in {1..30}; do
if curl -sf http://localhost:8888/health &>/dev/null || curl -sf http://localhost:8888/docs &>/dev/null; then
echo "✅ API is ready"
break
fi
sleep 1
done
# Start Control Plane
echo "🎛️ Starting Control Plane..."
cd /app/control-plane
npm start &
CP_PID=$!
echo ""
echo "✅ Hindsight is running!"
echo ""
echo "📍 Access:"
echo " Control Plane: http://localhost:3000"
echo " API: http://localhost:8888"
echo ""
# Wait for any process to exit
wait -n
# Exit with status of first exited process
exit $?

82
docker/start.sh
View file

@ -1,65 +1,41 @@
#!/bin/bash
set -e
# Start Hindsight (standalone all-in-one)
cd "$(dirname "$0")"
echo "🚀 Starting Hindsight Services"
echo "============================"
echo ""
# Check if .env file exists in root
if [ ! -f ../.env ]; then
echo "⚠️ No .env file found in project root!"
# Check for --build flag
BUILD_FLAG=""
if [[ "$1" == "--build" ]] || [[ "$1" == "-b" ]]; then
BUILD_FLAG="--build"
echo "🔨 Forcing rebuild of images..."
echo ""
echo "Creating .env from .env.example..."
cp ../.env.example ../.env
echo ""
echo "⚠️ Please edit .env and set your API keys:"
echo " - HINDSIGHT_API_LLM_API_KEY"
echo ""
echo "Then run this script again."
exit 1
fi
echo "📦 Building and starting services..."
docker compose --env-file ../.env up --build -d
echo ""
echo "⏳ Waiting for services to be healthy..."
echo "🚀 Starting Hindsight..."
echo ""
# Wait for PostgreSQL
echo " Waiting for PostgreSQL..."
until docker exec hindsight-postgres pg_isready -U hindsight > /dev/null 2>&1; do
sleep 1
done
echo " ✅ PostgreSQL is ready"
# Load .env file from project root if it exists
if [ -f ../.env ]; then
echo "📝 Loading environment variables from .env file..."
export $(grep -v '^#' ../.env | grep -v '^$' | xargs)
fi
# Wait for API
echo " Waiting for API..."
until curl -f http://localhost:8888/api/v1/agents > /dev/null 2>&1; do
sleep 2
done
echo " ✅ API is ready"
# Check for required HINDSIGHT_API_LLM_API_KEY
if [ -z "$HINDSIGHT_API_LLM_API_KEY" ]; then
echo "⚠️ Warning: HINDSIGHT_API_LLM_API_KEY is not set"
echo ""
echo "Set it by either:"
echo " 1. Creating a .env file in the project root with: HINDSIGHT_API_LLM_API_KEY=your-key"
echo " 2. Exporting: export HINDSIGHT_API_LLM_API_KEY=your-key"
echo ""
read -p "Continue anyway? (y/N) " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
exit 1
fi
fi
# Wait for Control Plane
echo " Waiting for Control Plane..."
until curl -f http://localhost:9999 > /dev/null 2>&1; do
sleep 2
done
echo " ✅ Control Plane is ready"
cd standalone
echo ""
echo "✅ All services are running!"
echo ""
echo "📊 Service URLs:"
echo " Control Plane: http://localhost:9999"
echo " API: http://localhost:8888"
echo " PostgreSQL: localhost:5432"
echo ""
echo "🔍 View logs:"
echo " docker compose logs -f"
echo ""
echo "🛑 Stop services:"
echo " ./stop.sh"
echo ""
# Run docker-compose with optional --build flag
docker-compose up $BUILD_FLAG

17
docker/stop.sh
View file

@ -1,17 +0,0 @@
#!/bin/bash
set -e
cd "$(dirname "$0")"
echo "🛑 Stopping Services"
echo "============================"
echo ""
docker compose down
echo ""
echo "✅ All services stopped"
echo ""
echo "💡 To remove data volumes as well, run:"
echo " docker compose down -v"
echo ""

70
hindsight-api/alembic/versions/b7c4d8e9f1a2_add_chunks_table.py Normal file
View file

@ -0,0 +1,70 @@
"""add_chunks_table
Revision ID: b7c4d8e9f1a2
Revises: 5a366d414dce
Create Date: 2025-11-28 00:00:00.000000
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
# revision identifiers, used by Alembic.
revision: str = 'b7c4d8e9f1a2'
down_revision: Union[str, Sequence[str], None] = '5a366d414dce'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Add chunks table and link memory_units to chunks."""
# Create chunks table with single text PK (bank_id_document_id_chunk_index)
op.create_table(
'chunks',
sa.Column('chunk_id', sa.Text(), nullable=False),
sa.Column('document_id', sa.Text(), nullable=False),
sa.Column('bank_id', sa.Text(), nullable=False),
sa.Column('chunk_index', sa.Integer(), nullable=False),
sa.Column('chunk_text', sa.Text(), nullable=False),
sa.Column('created_at', postgresql.TIMESTAMP(timezone=True), server_default=sa.text('now()'), nullable=False),
sa.ForeignKeyConstraint(['document_id', 'bank_id'], ['documents.id', 'documents.bank_id'], name='chunks_document_fkey', ondelete='CASCADE'),
sa.PrimaryKeyConstraint('chunk_id', name=op.f('pk_chunks'))
)
# Add indexes for efficient queries
op.create_index('idx_chunks_document_id', 'chunks', ['document_id'])
op.create_index('idx_chunks_bank_id', 'chunks', ['bank_id'])
# Add chunk_id column to memory_units (nullable, as existing records won't have chunks)
op.add_column('memory_units', sa.Column('chunk_id', sa.Text(), nullable=True))
# Add foreign key constraint to chunks table
op.create_foreign_key(
'memory_units_chunk_fkey',
'memory_units',
'chunks',
['chunk_id'],
['chunk_id'],
ondelete='SET NULL'
)
# Add index on chunk_id for efficient lookups
op.create_index('idx_memory_units_chunk_id', 'memory_units', ['chunk_id'])
def downgrade() -> None:
"""Remove chunks table and chunk_id from memory_units."""
# Drop index and foreign key from memory_units
op.drop_index('idx_memory_units_chunk_id', table_name='memory_units')
op.drop_constraint('memory_units_chunk_fkey', 'memory_units', type_='foreignkey')
op.drop_column('memory_units', 'chunk_id')
# Drop chunks table indexes and table
op.drop_index('idx_chunks_bank_id', table_name='chunks')
op.drop_index('idx_chunks_document_id', table_name='chunks')
op.drop_table('chunks')

4
hindsight-api/hindsight_api/__init__.py
View file

@ -4,7 +4,7 @@ Memory System for AI Agents.
Temporal + Semantic Memory Architecture using PostgreSQL with pgvector.
"""
from .engine.memory_engine import MemoryEngine
from .engine.search_trace import (
from .engine.search.trace import (
SearchTrace,
QueryInfo,
EntryPoint,
@ -15,7 +15,7 @@ from .engine.search_trace import (
SearchSummary,
SearchPhaseMetrics,
)
from .engine.search_tracer import SearchTracer
from .engine.search.tracer import SearchTracer
from .engine.embeddings import Embeddings, SentenceTransformersEmbeddings
from .engine.llm_wrapper import LLMConfig

107
hindsight-api/hindsight_api/api/http.py
View file

@ -36,6 +36,7 @@ from pydantic import BaseModel, Field, ConfigDict
from hindsight_api import MemoryEngine
from hindsight_api.engine.memory_engine import Budget
from hindsight_api.engine.db_utils import acquire_with_retry
from hindsight_api.metrics import get_metrics_collector, initialize_metrics, create_metrics_collector
logger = logging.getLogger(__name__)
@ -61,12 +62,21 @@ class EntityIncludeOptions(BaseModel):
max_tokens: int = Field(default=500, description="Maximum tokens for entity observations")
class ChunkIncludeOptions(BaseModel):
"""Options for including chunks in recall results."""
max_tokens: int = Field(default=8192, description="Maximum tokens for chunks (chunks may be truncated)")
class IncludeOptions(BaseModel):
"""Options for including additional data in recall results."""
entities: Optional[EntityIncludeOptions] = Field(
default=EntityIncludeOptions(),
description="Include entity observations. Set to null to disable entity inclusion."
)
chunks: Optional[ChunkIncludeOptions] = Field(
default=None,
description="Include raw chunks. Set to {} to enable, null to disable (default: disabled)."
)
class RecallRequest(BaseModel):
@ -113,7 +123,8 @@ class RecallResult(BaseModel):
"occurred_end": "2024-01-15T10:30:00Z",
"mentioned_at": "2024-01-15T10:30:00Z",
"document_id": "session_abc123",
"metadata": {"source": "slack"}
"metadata": {"source": "slack"},
"chunk_id": "456e7890-e12b-34d5-a678-901234567890"
}
}
}
@ -128,6 +139,7 @@ class RecallResult(BaseModel):
mentioned_at: Optional[str] = None # ISO format date when the fact was mentioned
document_id: Optional[str] = None # Document this memory belongs to
metadata: Optional[Dict[str, str]] = None # User-defined metadata
chunk_id: Optional[str] = None # Chunk this fact was extracted from
class EntityObservationResponse(BaseModel):
@ -206,6 +218,14 @@ class EntityDetailResponse(BaseModel):
observations: List[EntityObservationResponse]
class ChunkData(BaseModel):
"""Chunk data for a single chunk."""
id: str
text: str
chunk_index: int
truncated: bool = Field(default=False, description="Whether the chunk text was truncated due to token limits")
class RecallResponse(BaseModel):
"""Response model for recall endpoints."""
model_config = ConfigDict(json_schema_extra={
@ -218,7 +238,8 @@ class RecallResponse(BaseModel):
"entities": ["Alice", "Google"],
"context": "work info",
"occurred_start": "2024-01-15T10:30:00Z",
"occurred_end": "2024-01-15T10:30:00Z"
"occurred_end": "2024-01-15T10:30:00Z",
"chunk_id": "456e7890-e12b-34d5-a678-901234567890"
}
],
"trace": {
@ -234,6 +255,13 @@ class RecallResponse(BaseModel):
{"text": "Alice works at Google on the AI team", "mentioned_at": "2024-01-15T10:30:00Z"}
]
}
},
"chunks": {
"456e7890-e12b-34d5-a678-901234567890": {
"id": "456e7890-e12b-34d5-a678-901234567890",
"text": "Alice works at Google on the AI team. She's been there for 3 years...",
"chunk_index": 0
}
}
}
})
@ -241,6 +269,7 @@ class RecallResponse(BaseModel):
results: List[RecallResult]
trace: Optional[Dict[str, Any]] = None
entities: Optional[Dict[str, EntityStateResponse]] = Field(default=None, description="Entity states for entities mentioned in results")
chunks: Optional[Dict[str, ChunkData]] = Field(default=None, description="Chunks for facts, keyed by chunk_id")
class MemoryItem(BaseModel):
@ -690,6 +719,20 @@ def create_app(memory: MemoryEngine, run_migrations: bool = True, initialize_mem
Lifespan context manager for startup and shutdown events.
Note: This only fires when running the app standalone, not when mounted.
"""
# Initialize OpenTelemetry metrics
try:
prometheus_reader = initialize_metrics(
service_name="hindsight-api",
service_version="1.0.0"
)
create_metrics_collector()
app.state.prometheus_reader = prometheus_reader
logging.info("Metrics initialized - available at /metrics endpoint")
except Exception as e:
logging.warning(f"Failed to initialize metrics: {e}. Metrics will be disabled (using no-op collector).")
app.state.prometheus_reader = None
# Metrics collector is already initialized as no-op by default
# Startup: Initialize database and memory system
if initialize_memory:
await memory.initialize()
@ -735,6 +778,19 @@ def create_app(memory: MemoryEngine, run_migrations: bool = True, initialize_mem
def _register_routes(app: FastAPI):
"""Register all API routes on the given app instance."""
@app.get(
"/metrics",
summary="Prometheus metrics endpoint",
description="Exports metrics in Prometheus format for scraping",
tags=["Monitoring"]
)
async def metrics_endpoint():
"""Return Prometheus metrics."""
from prometheus_client import generate_latest, CONTENT_TYPE_LATEST
from fastapi.responses import Response
metrics_data = generate_latest()
return Response(content=metrics_data, media_type=CONTENT_TYPE_LATEST)
@app.get(
"/v1/default/banks/{bank_id}/graph",
@ -818,6 +874,8 @@ def _register_routes(app: FastAPI):
)
async def api_recall(bank_id: str, request: RecallRequest):
"""Run a recall and return results with trace."""
metrics = get_metrics_collector()
try:
# Validate types
valid_fact_types = ["world", "agent", "opinion", "observation"]
@ -846,7 +904,12 @@ def _register_routes(app: FastAPI):
include_entities = request.include.entities is not None
max_entity_tokens = request.include.entities.max_tokens if include_entities else 500
# Run recall with tracing
# Determine chunk inclusion settings
include_chunks = request.include.chunks is not None
max_chunk_tokens = request.include.chunks.max_tokens if include_chunks else 8192
# Run recall with tracing (record metrics)
with metrics.record_operation("recall", bank_id=bank_id, budget=request.budget.value, max_tokens=request.max_tokens):
core_result = await app.state.memory.recall_async(
bank_id=bank_id,
query=request.query,
@ -856,7 +919,9 @@ def _register_routes(app: FastAPI):
fact_type=fact_types,
question_date=question_date,
include_entities=include_entities,
max_entity_tokens=max_entity_tokens
max_entity_tokens=max_entity_tokens,
include_chunks=include_chunks,
max_chunk_tokens=max_chunk_tokens
)
# Convert core MemoryFact objects to API RecallResult objects (excluding internal metrics)
@ -870,11 +935,24 @@ def _register_routes(app: FastAPI):
occurred_start=fact.occurred_start,
occurred_end=fact.occurred_end,
mentioned_at=fact.mentioned_at,
document_id=fact.document_id
document_id=fact.document_id,
chunk_id=fact.chunk_id
)
for fact in core_result.results
]
# Convert chunks from engine to HTTP API format
chunks_response = None
if core_result.chunks:
chunks_response = {}
for chunk_id, chunk_info in core_result.chunks.items():
chunks_response[chunk_id] = ChunkData(
id=chunk_id,
text=chunk_info.chunk_text,
chunk_index=chunk_info.chunk_index,
truncated=chunk_info.truncated
)
# Convert core EntityState objects to API EntityStateResponse objects
entities_response = None
if core_result.entities:
@ -892,7 +970,8 @@ def _register_routes(app: FastAPI):
return RecallResponse(
results=recall_results,
trace=core_result.trace,
entities=entities_response
entities=entities_response,
chunks=chunks_response
)
except HTTPException:
raise
@ -921,8 +1000,11 @@ def _register_routes(app: FastAPI):
operation_id="reflect"
)
async def api_reflect(bank_id: str, request: ReflectRequest):
metrics = get_metrics_collector()
try:
# Use the memory system's reflect_async method
# Use the memory system's reflect_async method (record metrics)
with metrics.record_operation("reflect", bank_id=bank_id, budget=request.budget.value):
core_result = await app.state.memory.reflect_async(
bank_id=bank_id,
query=request.query,
@ -1449,7 +1531,7 @@ This operation cannot be undone.
return BankProfileResponse(
bank_id=bank_id,
name=profile["name"],
personality=PersonalityTraits(**profile["personality"]),
personality=profile["personality"], # Already a PersonalityTraits object
background=profile["background"]
)
except Exception as e:
@ -1482,7 +1564,7 @@ This operation cannot be undone.
return BankProfileResponse(
bank_id=bank_id,
name=profile["name"],
personality=PersonalityTraits(**profile["personality"]),
personality=profile["personality"], # Already a PersonalityTraits object
background=profile["background"]
)
except Exception as e:
@ -1582,7 +1664,7 @@ This operation cannot be undone.
return BankProfileResponse(
bank_id=bank_id,
name=final_profile["name"],
personality=PersonalityTraits(**final_profile["personality"]),
personality=final_profile["personality"], # Already a PersonalityTraits object
background=final_profile["background"]
)
except Exception as e:
@ -1632,6 +1714,8 @@ This operation cannot be undone.
)
async def api_retain(bank_id: str, request: RetainRequest):
"""Retain memories with optional async processing."""
metrics = get_metrics_collector()
try:
# Prepare contents for processing
contents = []
@ -1683,7 +1767,8 @@ This operation cannot be undone.
async_=True
)
else:
# Synchronous processing: wait for completion
# Synchronous processing: wait for completion (record metrics)
with metrics.record_operation("retain", bank_id=bank_id):
result = await app.state.memory.retain_batch_async(
bank_id=bank_id,
contents=contents,

4
hindsight-api/hindsight_api/api/mcp.py
View file

@ -61,7 +61,7 @@ def create_mcp_server(memory: MemoryEngine) -> FastMCP:
try:
# Log explanation if provided
if explanation:
logger.debug(f"Explanation: {explanation}")
pass # Explanation provided
# Store memory using put_batch_async
await memory.put_batch_async(
@ -116,7 +116,7 @@ def create_mcp_server(memory: MemoryEngine) -> FastMCP:
# Log explanation if provided
if explanation:
logger.debug(f"Explanation: {explanation}")
pass # Explanation provided
# Search using recall_async
from hindsight_api.engine.memory_engine import Budget

4
hindsight-api/hindsight_api/engine/__init__.py
View file

@ -10,7 +10,7 @@ This package contains all the implementation details of the memory engine:
from .memory_engine import MemoryEngine
from .db_utils import acquire_with_retry
from .embeddings import Embeddings, SentenceTransformersEmbeddings
from .search_trace import (
from .search.trace import (
SearchTrace,
QueryInfo,
EntryPoint,
@ -21,7 +21,7 @@ from .search_trace import (
SearchSummary,
SearchPhaseMetrics,
)
from .search_tracer import SearchTracer
from .search.tracer import SearchTracer
from .llm_wrapper import LLMConfig
from .response_models import RecallResult, ReflectResult, MemoryFact

639
hindsight-api/hindsight_api/engine/fact_extraction.py
View file

@ -1,639 +0,0 @@
"""
Fact extraction from text using LLM.
Extracts semantic facts, entities, and temporal information from text.
Uses the LLMConfig wrapper for all LLM calls.
"""
import logging
import os
import json
import re
import asyncio
from datetime import datetime
from typing import List, Dict, Optional, Literal
from openai import AsyncOpenAI
from pydantic import BaseModel, Field
from .llm_wrapper import OutputTooLongError, LLMConfig
class Entity(BaseModel):
"""An entity extracted from text."""
text: str = Field(
description="The specific, named entity as it appears in the fact. Must be a proper noun or specific identifier."
)
class CausalRelation(BaseModel):
"""Causal relationship between facts."""
target_fact_index: int = Field(
description="Index of the related fact in the facts array (0-based). "
"This creates a directed causal link to another fact in the extraction."
)
relation_type: Literal["causes", "caused_by", "enables", "prevents"] = Field(
description="Type of causal relationship: "
"'causes' = this fact directly causes the target fact, "
"'caused_by' = this fact was caused by the target fact, "
"'enables' = this fact enables/allows the target fact, "
"'prevents' = this fact prevents/blocks the target fact"
)
strength: float = Field(
description="Strength of causal relationship (0.0 to 1.0). "
"1.0 = direct/strong causation, 0.5 = moderate, 0.3 = weak/indirect",
ge=0.0,
le=1.0,
default=1.0
)
class ExtractedFact(BaseModel):
"""A single extracted fact with structured dimensions for comprehensive capture."""
# Core factual dimension (required)
factual_core: str = Field(
description="ACTUAL FACTS - what literally happened/was said. Capture WHAT was said, not just THAT something was said! 'Gina said Jon is the perfect mentor with positivity and determination' NOT 'Jon received encouragement'. Preserve: compliments, assessments, descriptions, key phrases. Be specific!"
)
# Optional dimensions - only include if present in the text
emotional_significance: Optional[str] = Field(
default=None,
description="Emotions, feelings, personal meaning, AND qualitative descriptors if present. Include ALL experiential/evaluative terms like 'magical', 'wonderful', 'amazing', 'thrilling'. Examples: 'felt thrilled', 'was her favorite memory', 'it was magical', 'devastating experience', 'proudest moment'"
)
reasoning_motivation: Optional[str] = Field(
default=None,
description="WHY it happened, intentions, goals, causes if present. Examples: 'because she wanted to celebrate', 'in order to cope with grief', 'motivated by curiosity'"
)
preferences_opinions: Optional[str] = Field(
default=None,
description="Likes, dislikes, beliefs, values if present. Examples: 'loves coffee', 'thinks AI is transformative', 'prefers working remotely'"
)
sensory_details: Optional[str] = Field(
default=None,
description="Visual, auditory, physical descriptions AND all descriptive adjectives - USE EXACT WORDS from the text! Don't paraphrase adjectives. If they said 'awesome' write 'awesome' not 'amazing'. Examples: 'bright orange hair', 'so graceful', 'awesome beach', 'epic visuals', 'freezing cold'."
)
observations: Optional[str] = Field(
default=None,
description="Observations and inferences from the conversation - things that can be deduced but weren't explicitly stated. Includes: travel (if someone is 'shooting in Miami' → they went/will go to Miami), possession implies achievement ('my trophy' → won it), actions imply location/travel ('doing the shoot in Miami' → traveled to Miami), capabilities ('she coded it' → knows programming). Examples: 'Calvin traveled to Miami', 'Gina won dance trophies', 'knows programming'"
)
# Fact kind - determines temporal handling (used for prompt engineering, not stored in DB)
fact_kind: Literal["conversation", "event", "other"] = Field(
description="Determines if occurred dates should be set. 'conversation' = general info, activities, preferences (NO occurred dates). 'event' = specific datable occurrence like competition, wedding, meeting (HAS occurred_start/end). 'other' = anything else (NO occurred dates). Only 'event' gets occurred dates!"
)
# Temporal fields - ONLY for fact_kind='event'
occurred_start: Optional[str] = Field(
default=None,
description="ONLY set when fact_kind='event'. ISO format. Leave null for fact_kind='conversation'."
)
occurred_end: Optional[str] = Field(
default=None,
description="ONLY set when fact_kind='event'. ISO format. Leave null for fact_kind='conversation'."
)
# Classification
fact_type: Literal["world", "bank", "opinion"] = Field(
description="'world' = facts about others (third person), 'bank' = facts about YOU the memory owner (FIRST PERSON: 'I did...'), 'opinion' = your beliefs (first person)"
)
# Entities and relations
entities: List[Entity] = Field(
default_factory=list,
description="ONLY specific, named entities worth tracking: people's names (e.g., 'Sarah', 'Dr. Smith'), organizations (e.g., 'Google', 'MIT'), specific places (e.g., 'Paris', 'Central Park'). DO NOT include: generic relations (mom, friend, boss, colleague), common nouns (apple, car, house), pronouns (he, she), or vague references (someone, a guy)."
)
causal_relations: Optional[List[CausalRelation]] = Field(
default=None,
description="Causal links to other facts in this batch. Example: fact about rain causes fact about cancelled game."
)
def build_fact_text(self) -> str:
"""Combine all dimensions into a single comprehensive fact string."""
parts = [self.factual_core]
if self.emotional_significance:
parts.append(self.emotional_significance)
if self.reasoning_motivation:
parts.append(self.reasoning_motivation)
if self.preferences_opinions:
parts.append(self.preferences_opinions)
if self.sensory_details:
parts.append(self.sensory_details)
if self.observations:
parts.append(self.observations)
# Join with appropriate connectors
if len(parts) == 1:
return parts[0]
# Combine: "Core fact - emotional/significance context"
return f"{parts[0]} - {' - '.join(parts[1:])}"
class FactExtractionResponse(BaseModel):
"""Response containing all extracted facts."""
facts: List[ExtractedFact] = Field(
description="List of extracted factual statements"
)
def chunk_text(text: str, max_chars: int) -> List[str]:
"""
Split text into chunks at sentence boundaries using LangChain's text splitter.
Uses RecursiveCharacterTextSplitter which intelligently splits at sentence boundaries
and allows chunks to slightly exceed max_chars to finish sentences naturally.
Args:
text: Input text to chunk
max_chars: Maximum characters per chunk (default 120k ≈ 30k tokens)
Note: chunks may slightly exceed this to complete sentences
Returns:
List of text chunks, roughly under max_chars
"""
from langchain_text_splitters import RecursiveCharacterTextSplitter
# If text is small enough, return as-is
if len(text) <= max_chars:
return [text]
# Configure splitter to split at sentence boundaries first
# Separators in order of preference: paragraphs, newlines, sentences, words
splitter = RecursiveCharacterTextSplitter(
chunk_size=max_chars,
chunk_overlap=0,
length_function=len,
is_separator_regex=False,
separators=[
"\n\n", # Paragraph breaks
"\n", # Line breaks
". ", # Sentence endings
"! ", # Exclamations
"? ", # Questions
"; ", # Semicolons
", ", # Commas
" ", # Words
"", # Characters (last resort)
],
)
return splitter.split_text(text)
async def _extract_facts_from_chunk(
chunk: str,
chunk_index: int,
total_chunks: int,
event_date: datetime,
context: str,
llm_config: 'LLMConfig',
agent_name: str = None,
extract_opinions: bool = False
) -> List[Dict[str, str]]:
"""
Extract facts from a single chunk (internal helper for parallel processing).
"""
# Format event_date for the prompt
event_date_str = event_date.strftime("%Y-%m-%dT%H:%M:%SZ")
agent_context = f"\n- Your name: {agent_name}" if agent_name else ""
# Determine which fact types to extract based on the flag
if extract_opinions:
fact_types_instruction = "Extract ONLY 'opinion' type facts (the bank's formed opinions, beliefs, and perspectives). DO NOT extract 'world' or 'bank' facts."
else:
fact_types_instruction = "Extract ONLY 'world' and 'bank' type facts. DO NOT extract 'opinion' type facts - opinions should never be created during normal memory storage."
prompt = f"""You are extracting comprehensive, narrative facts from conversations/document for an AI memory system.
{fact_types_instruction}
## CONTEXT INFORMATION
- Today time: {datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")}
- Current document date/time: {event_date_str}
- Context: {context if context else 'no additional context provided'}{agent_context}
## CORE PRINCIPLE: Extract ALL Meaningful Information Efficiently
**GOAL**: Capture ALL meaningful information, but combine related exchanges efficiently. Don't create separate facts for questions - merge Q&A into single facts.
Each fact should:
1. **CAPTURE ALL MEANINGFUL CONTENT** - Activities, projects, preferences, recommendations, encouragement WITH specific content
2. **BE SELF-CONTAINED** - Readable without the original text
3. **PRESERVE SPECIFIC CONTENT** - Capture WHAT was said, not just THAT something was said
4. **COMBINE Q&A** - A question and its answer = ONE fact, not two separate facts
## COMBINE Q&A - CRITICAL!
**❌ BAD (2 separate facts):**
- "James asks what projects John is working on"
- "John is working on a website for a local small business"
**✅ GOOD (1 combined fact):**
- "John is working on a website for a local small business; it's his first professional project outside of class"
**❌ BAD (question as standalone fact):**
- "James asks John what challenges he has encountered"
**✅ GOOD (merged with answer):**
- "John says payment integration was challenging; he used resources to understand the process and is getting closer to a solution"
## WHAT TO SKIP (only these!)
- **Standalone questions** - merge with answers instead
- **Pure filler with no content** - "Always happy to help", "Sounds good", "Thanks!"
- **Greetings** - "Hey!", "What's up?"
## WHAT TO ALWAYS EXTRACT
- Specific encouragement WITH content: "James says hiccups are normal, use them to learn and grow, push through"
- Reactions that reveal preferences: "John says the art is awesome, takes him back to reading fantasy books"
- Recommendations: "John recommends 'The Name of the Wind' - great novel with awesome writing"
- Plans/intentions: "James will check out 'The Name of the Wind'"
- All activities, projects, purchases, events with details
## ESSENTIAL DETAILS TO PRESERVE - NEVER LOSE THESE
When extracting facts, you MUST preserve:
1. **ALL PARTICIPANTS** - Who said/did what
2. **INDIVIDUAL PREFERENCES** - Each person's specific likes/favorites! "Jon's favorite is contemporary because it's expressive" - DO NOT LOSE THIS!
3. **FULL REASONING** - Why decisions were made, motivations, explanations
4. **TEMPORAL CONTEXT - CRITICAL** - ALWAYS convert relative time references to SPECIFIC ABSOLUTE dates in the fact text!
- "last week" (doc date Aug 23) → "around August 16, 2023" (NOT just "in August 2023"!)
- "last month" (doc date Aug 2023) → "in July 2023"
- "yesterday" (doc date Aug 19) → "on August 18, 2023"
- "next week" (doc date Aug 19) → "around August 26, 2023"
- "three days ago" (doc date Aug 19) → "on August 16, 2023"
- "last year" → "in 2022"
- BE SPECIFIC! "last week" is NOT "in August" - calculate the actual week!
5. **VISUAL/MEDIA ELEMENTS** - Photos, images, videos shared
6. **MODIFIERS** - "new", "first", "old", "favorite" (critical context)
7. **POSSESSIVE RELATIONSHIPS** - "their kids" → "Person's kids"
8. **BIOGRAPHICAL DETAILS** - Origins, locations, jobs, family background
9. **SOCIAL DYNAMICS** - Nicknames, how people address each other, relationships
## STRUCTURED FACT DIMENSIONS - CRITICAL ⚠️
Each fact MUST be extracted into structured dimensions. This ensures no important context is lost.
### Required field:
- **factual_core**: ACTUAL FACTS - capture WHAT was said, not just THAT something was said!
- ❌ BAD: "Jon received encouragement from Gina" (loses what Gina actually said)
- ✅ GOOD: "Gina said Jon is the perfect mentor with positivity and determination; his studio will be a hit"
- ❌ BAD: "Jon supports Gina" (generic)
- ✅ GOOD: "Gina found the perfect spot for her store; Jon says her hard work is paying off"
- Preserve: compliments, assessments, descriptions, predictions, key phrases
### Optional fields (include when present in text):
- **emotional_significance**: Emotions, feelings, personal meaning, AND qualitative descriptors
- Examples: "felt thrilled", "was her favorite memory", "it's magical", "devastating experience", "proudest moment"
- Captures: emotions, intensity, personal significance, AND experiential descriptors ("magical", "wonderful", "amazing", "thrilling", "beautiful")
- **reasoning_motivation**: WHY it happened, intentions, goals, causes
- Examples: "because she wanted to celebrate", "in order to cope with grief", "motivated by curiosity"
- Captures: reasons, intentions, goals, causal explanations
- **preferences_opinions**: Likes, dislikes, beliefs, values, ideals - CAPTURE EACH PERSON'S SPECIFIC PREFERENCES
- Examples: "Jon's ideal dance studio is by the water", "Jon's favorite dance is contemporary", "loves coffee", "prefers remote work"
- Captures: preferences, opinions, beliefs, judgments, ideals, dreams
- PREFERENCE INDICATORS: "ideal", "favorite", "dream", "perfect", "love", "hate", "prefer" → MUST capture in this dimension!
- CRITICAL: Never lose individual preferences! "Jon's ideal studio is by the water" must be captured!
- **sensory_details**: Visual, auditory, physical descriptions AND all descriptive adjectives - USE EXACT WORDS!
- Examples: "bright orange hair", "loud music", "freezing cold", "so graceful", "awesome beach", "epic visuals"
- Captures: colors, sounds, textures, temperatures, appearances, AND adjectives describing people/things/performances
- CRITICAL: Use the EXACT adjectives from the text! If they said "awesome" don't write "amazing". If they said "epic" don't write "perfect"!
- **observations**: Things that can be inferred/deduced from the conversation - not explicitly stated but clearly implied
- TRAVEL: "doing the shoot in Miami" → "Calvin traveled/will travel to Miami"
- POSSESSION: "my trophy" → "won the trophy"
- CAPABILITIES: "she coded it" → "knows programming"
- Examples: "Calvin traveled to Miami for the shoot", "Gina won dance trophies", "knows programming"
### Example extraction:
**Input**: "I used to compete in dance competitions - my fav memory was when my team won first place at regionals at age fifteen. It was an awesome feeling of accomplishment!"
**Output**:
```
factual_core: "Gina's team won first place at a regional dance competition when she was 15"
emotional_significance: "this was her favorite memory; felt an awesome sense of accomplishment"
reasoning_motivation: null
preferences_opinions: null
sensory_details: null
```
### CRITICAL: Never strip away dimensions!
- ❌ BAD: Only extracting factual_core and ignoring emotional context
- ✅ GOOD: Capturing ALL dimensions present in the text
## FACT KIND AND TEMPORAL RULES
### fact_kind determines if occurred dates are set:
**`conversation`** - General info, activities, preferences, ongoing things
- NO occurred_start/end (leave null)
- Examples: "Jon is expanding his studio", "Jon loves dance", "Gina's ideal studio is by water"
**`event`** - Specific datable occurrence (competition, wedding, meeting, trip, loss, start/end of something)
- MUST set occurred_start/end
- Ask: "Is this a SPECIFIC EVENT with a DATE?"
- Examples: "Dance competition on May 15", "Lost job in January 2023", "Wedding next Saturday"
**`other`** - Anything else that doesn't fit above
- NO occurred_start/end (leave null)
- Catch-all to not lose information
### Rules:
1. **ALWAYS include dates in fact text** - "in January 2023", "on May 15, 2024"
2. **Only 'event' gets occurred dates** - conversation and other = null
3. **SPLIT events from conversation facts** - "Jon is expanding his studio (conversation) and hosting a competition next month (event)" → 2 separate facts!
## CAUSAL RELATIONSHIPS
When splitting related facts, link them with causal_relations:
- **causes**: This fact causes the target
- **caused_by**: This fact was caused by target
- **enables/prevents**: This fact enables/prevents the target
Only link when there's explicit or clear implicit causation ("because", "so", "therefore").
## FACT TYPE CLASSIFICATION
- **'world'**: Facts about others (third person)
- **'agent'**: Facts about YOU the memory owner (FIRST PERSON: "I did...", "I said...")
- **'opinion'**: Your beliefs/perspectives (first person: "I believe...")
**Speaker attribution**: If context says "Your name: Marcus", only extract 'agent' facts from "Marcus:" lines.
## WHAT TO SKIP
- Greetings, filler words, pure reactions ("wow", "cool")
- Structural statements ("let's get started", "see you next time")
- Calls to action ("subscribe", "follow")
## EXAMPLE: SPLITTING CONVERSATION VS EVENT FACTS
**Input (conversation date: April 3, 2023):**
"I'm expanding my dance studio's social media presence and offering workshops to local schools. I'm also hosting a dance competition next month to showcase local talent. The dancers are so excited!"
**Output (2 facts - conversation + event):**
**Fact 1 (kind=conversation - ongoing activities, no occurred dates):**
```
fact_kind: "conversation"
factual_core: "Jon is expanding his dance studio's social media presence in April 2023; offering workshops and classes to local schools and centers; seeing progress and dancers are excited"
emotional_significance: "excited and proud of progress"
preferences_opinions: "Jon loves giving dancers a place to express themselves"
observations: "Jon owns/runs a dance studio"
occurred_start: null ← conversation kind = no occurred dates
occurred_end: null
```
**Fact 2 (kind=event - specific datable occurrence):**
```
fact_kind: "event"
factual_core: "Jon will host a dance competition in May 2023 to showcase local talent and bring attention to his studio"
emotional_significance: "excited about the event"
occurred_start: "2023-05-01T00:00:00Z" ← event kind = HAS occurred dates
occurred_end: "2023-05-31T23:59:59Z"
```
**❌ BAD:** Combining both into one fact with occurred=May (makes ongoing activities look like they happened in May!)
## TEXT TO EXTRACT FROM:
{chunk}
## CRITICAL REMINDERS:
1. **COMBINE Q&A** - Never create standalone question facts! Merge questions with their answers into single facts.
2. **CAPTURE ALL MEANINGFUL CONTENT** - Activities, encouragement (with specific words!), recommendations, reactions, preferences
3. **CONVERT RELATIVE DATES TO SPECIFIC DATES** - "last week" → "around August 16" (NOT "in August"!), "yesterday" → "on August 18". Be precise!
4. **CAPTURE WHAT WAS SAID** - "Gina said Jon is perfect mentor with determination" NOT "Jon received encouragement". Preserve the actual content!
5. **FACT_KIND DETERMINES OCCURRED DATES** - Only 'event' gets occurred_start/end. 'conversation' and 'other' = null
6. **CAPTURE PREFERENCES** - "ideal", "favorite", "love" → preferences_opinions
7. **CAPTURE EXACT ADJECTIVES** - Use the EXACT words! "awesome" not "amazing", "epic" not "perfect" → sensory_details
8. **CAPTURE OBSERVATIONS** - "shooting in Miami" → observations: "traveled to Miami". Infer travel, achievements, capabilities!"""
import logging
from openai import BadRequestError
logger = logging.getLogger(__name__)
# Retry logic for JSON validation errors
max_retries = 2
last_error = None
for attempt in range(max_retries):
try:
extraction_response = await llm_config.call(
messages=[
{
"role": "system",
"content": "Extract ALL meaningful content. COMBINE Q&A into single facts (no standalone questions!). Skip only greetings and pure filler. CONVERT RELATIVE DATES TO SPECIFIC DATES ('last week' → 'around Aug 16' NOT 'in August'!). factual_core = WHAT was said, not THAT something was said! fact_kind: 'conversation'/'event'/'other'. Only 'event' gets occurred dates."
},
{
"role": "user",
"content": prompt
}
],
response_format=FactExtractionResponse,
scope="memory_extract_facts",
temperature=0.1,
max_tokens=65000,
)
# Build combined fact text from dimensions and include in output
chunk_facts = []
for fact in extraction_response.facts:
fact_dict = fact.model_dump()
# Add combined 'fact' field from structured dimensions
fact_dict['fact'] = fact.build_fact_text()
# Safety net: strip occurred dates if fact_kind is not 'event'
# (in case LLM doesn't follow the rules)
if fact_dict.get('fact_kind') != 'event':
fact_dict['occurred_start'] = None
fact_dict['occurred_end'] = None
# Remove fact_kind from output (only used for prompt engineering, not stored)
fact_dict.pop('fact_kind', None)
chunk_facts.append(fact_dict)
return chunk_facts
except BadRequestError as e:
last_error = e
if "json_validate_failed" in str(e):
logger.warning(f" [1.3.{chunk_index + 1}] Attempt {attempt + 1}/{max_retries} failed with JSON validation error: {e}")
if attempt < max_retries - 1:
logger.info(f" [1.3.{chunk_index + 1}] Retrying...")
continue
# If it's not a JSON validation error or we're out of retries, re-raise
raise
# If we exhausted all retries, raise the last error
raise last_error
async def _extract_facts_with_auto_split(
chunk: str,
chunk_index: int,
total_chunks: int,
event_date: datetime,
context: str,
llm_config: LLMConfig,
agent_name: str = None,
extract_opinions: bool = False
) -> List[Dict[str, str]]:
"""
Extract facts from a chunk with automatic splitting if output exceeds token limits.
If the LLM output is too long (OutputTooLongError), this function automatically
splits the chunk in half and processes each half recursively.
Args:
chunk: Text chunk to process
chunk_index: Index of this chunk in the original list
total_chunks: Total number of original chunks
event_date: Reference date for temporal information
context: Context about the conversation/document
llm_config: LLM configuration to use
agent_name: Optional agent name (memory owner)
extract_opinions: If True, extract ONLY opinions. If False, extract world and agent facts (no opinions)
Returns:
List of fact dictionaries extracted from the chunk (possibly from sub-chunks)
"""
import logging
logger = logging.getLogger(__name__)
try:
# Try to extract facts from the full chunk
return await _extract_facts_from_chunk(
chunk=chunk,
chunk_index=chunk_index,
total_chunks=total_chunks,
event_date=event_date,
context=context,
llm_config=llm_config,
agent_name=agent_name,
extract_opinions=extract_opinions
)
except OutputTooLongError as e:
# Output exceeded token limits - split the chunk in half and retry
logger.warning(
f"Output too long for chunk {chunk_index + 1}/{total_chunks} "
f"({len(chunk)} chars). Splitting in half and retrying..."
)
# Split at the midpoint, preferring sentence boundaries
mid_point = len(chunk) // 2
# Try to find a sentence boundary near the midpoint
# Look for ". ", "! ", "? " within 20% of midpoint
search_range = int(len(chunk) * 0.2)
search_start = max(0, mid_point - search_range)
search_end = min(len(chunk), mid_point + search_range)
sentence_endings = ['. ', '! ', '? ', '\n\n']
best_split = mid_point
for ending in sentence_endings:
pos = chunk.rfind(ending, search_start, search_end)
if pos != -1:
best_split = pos + len(ending)
break
# Split the chunk
first_half = chunk[:best_split].strip()
second_half = chunk[best_split:].strip()
logger.info(
f"Split chunk {chunk_index + 1} into two sub-chunks: "
f"{len(first_half)} chars and {len(second_half)} chars"
)
# Process both halves recursively (in parallel)
sub_tasks = [
_extract_facts_with_auto_split(
chunk=first_half,
chunk_index=chunk_index,
total_chunks=total_chunks,
event_date=event_date,
context=context,
llm_config=llm_config,
agent_name=agent_name,
extract_opinions=extract_opinions
),
_extract_facts_with_auto_split(
chunk=second_half,
chunk_index=chunk_index,
total_chunks=total_chunks,
event_date=event_date,
context=context,
llm_config=llm_config,
agent_name=agent_name,
extract_opinions=extract_opinions
)
]
sub_results = await asyncio.gather(*sub_tasks)
# Combine results from both halves
all_facts = []
for sub_result in sub_results:
all_facts.extend(sub_result)
logger.info(
f"Successfully extracted {len(all_facts)} facts from split chunk {chunk_index + 1}"
)
return all_facts
async def extract_facts_from_text(
text: str,
event_date: datetime,
llm_config: LLMConfig,
agent_name: str,
context: str = "",
extract_opinions: bool = False,
) -> List[Dict[str, str]]:
"""
Extract semantic facts from conversational or narrative text using LLM.
For large texts (>chunk_size chars), automatically chunks at sentence boundaries
to avoid hitting output token limits. Processes ALL chunks in PARALLEL for speed.
If a chunk produces output that exceeds token limits (OutputTooLongError), it is
automatically split in half and retried recursively until successful.
Args:
text: Input text (conversation, article, etc.)
event_date: Reference date for resolving relative times
context: Context about the conversation/document
llm_config: LLM configuration to use (if None, uses default from environment)
chunk_size: Maximum characters per chunk
agent_name: Optional agent name (memory owner)
extract_opinions: If True, extract ONLY opinions. If False, extract world and agent facts (no opinions)
Returns:
List of fact dictionaries with 'fact' and 'date' keys
"""
chunks = chunk_text(text, max_chars=3000)
tasks = [
_extract_facts_with_auto_split(
chunk=chunk,
chunk_index=i,
total_chunks=len(chunks),
event_date=event_date,
context=context,
llm_config=llm_config,
agent_name=agent_name,
extract_opinions=extract_opinions
)
for i, chunk in enumerate(chunks)
]
chunk_results = await asyncio.gather(*tasks)
all_facts = []
for chunk_facts in chunk_results:
all_facts.extend(chunk_facts)
return all_facts

60
hindsight-api/hindsight_api/engine/llm_wrapper.py
View file

@ -88,6 +88,7 @@ class LLMConfig:
max_retries: int = 5,
initial_backoff: float = 1.0,
max_backoff: float = 60.0,
skip_validation: bool = False,
**kwargs
) -> Any:
"""
@ -126,36 +127,51 @@ class LLMConfig:
for attempt in range(max_retries + 1):
try:
# Use the appropriate response format
if response_format is not None:
# Use structured output parsing and return .parsed
response = await self._client.beta.chat.completions.parse(
response_format=response_format,
**call_params
)
result = response.choices[0].message.parsed
# Use JSON mode instead of strict parse for flexibility with optional fields
# This allows the LLM to omit optional fields without validation errors
import json
# Add schema to the system message
if hasattr(response_format, 'model_json_schema'):
schema = response_format.model_json_schema()
schema_msg = f"\n\nYou must respond with valid JSON matching this schema:\n{json.dumps(schema, indent=2)}"
# Add schema to the system message if present, otherwise prepend as user message
if call_params['messages'] and call_params['messages'][0].get('role') == 'system':
call_params['messages'][0]['content'] += schema_msg
else:
# No system message, add schema instruction to first user message
if call_params['messages']:
call_params['messages'][0]['content'] = schema_msg + "\n\n" + call_params['messages'][0]['content']
call_params['response_format'] = {"type": "json_object"}
response = await self._client.chat.completions.create(**call_params)
# Parse the JSON response
content = response.choices[0].message.content
json_data = json.loads(content)
# Return raw JSON if skip_validation is True, otherwise validate with Pydantic
if skip_validation:
result = json_data
else:
result = response_format.model_validate(json_data)
else:
# Standard completion and return text content
response = await self._client.chat.completions.create(**call_params)
result = response.choices[0].message.content
# Log call details on success
# Log call details only if it takes more than 5 seconds
duration = time.time() - start_time
usage = response.usage
if duration > 10.0:
ratio = max(1, usage.completion_tokens) / usage.prompt_tokens
if ratio > 3:
raw_content = response.choices[0].message.content
raw_len = len(raw_content) if raw_content else 0
logger.info(
f"model={self.provider}/{self.model}, "
f"slow llm call: model={self.provider}/{self.model}, "
f"input_tokens={usage.prompt_tokens}, output_tokens={usage.completion_tokens}, "
f"total_tokens={usage.total_tokens}, time={duration:.3f}s, ratio out/in={ratio:.2f}, HIGH RATIO - raw_content_chars={raw_len}, \n\nin={messages}\n\nout={result}\n\nraw={raw_content}\n\n"
)
else:
logger.info(
f"model={self.provider}/{self.model}, "
f"input_tokens={usage.prompt_tokens}, output_tokens={usage.completion_tokens}, "
f"total_tokens={usage.total_tokens}, time={duration:.3f}s, ratio out/in={ratio:.2f}, "
f"total_tokens={usage.total_tokens}, time={duration:.3f}s, ratio out/in={ratio:.2f}"
)
return result
@ -182,17 +198,17 @@ class LLMConfig:
)
await asyncio.sleep(sleep_time)
else:
logger.error(f"Non-retryable API error after {max_retries + 1} attempts: {str(e)}, input {messages}")
logger.error(f"Non-retryable API error after {max_retries + 1} attempts: {str(e)}")
raise
except Exception as e:
logger.error(f"Unexpected error during LLM call: {type(e).__name__}: {str(e)}, input {messages}")
logger.error(f"Unexpected error during LLM call: {type(e).__name__}: {str(e)}")
raise
# This should never be reached, but just in case
if last_exception:
raise last_exception
raise RuntimeError(f"LLM call failed after all retries with no exception captured, input {messages}")
raise RuntimeError(f"LLM call failed after all retries with no exception captured")
@classmethod
def for_memory(cls) -> "LLMConfig":

867
hindsight-api/hindsight_api/engine/memory_engine.py
View file

File diff suppressed because it is too large Load diff

4
hindsight-api/hindsight_api/engine/query_analyzer.py
View file

@ -105,12 +105,10 @@ class TransformerQueryAnalyzer(QueryAnalyzer):
"Install it with: pip install transformers"
)
logger.debug(f"Loading T5 model: {self.model_name}...")
self._tokenizer = AutoTokenizer.from_pretrained(self.model_name)
self._model = AutoModelForSeq2SeqLM.from_pretrained(self.model_name)
self._model.to(self.device)
self._model.eval()
logger.debug(f"Model loaded on {self.device}")
def analyze(
self, query: str, reference_date: Optional[datetime] = None
@ -158,7 +156,6 @@ what is the weather = none
)
result = self._tokenizer.decode(outputs[0], skip_special_tokens=True).strip()
logger.debug(f"T5 generated: '{result}'")
# Parse the generated output
temporal = self._parse_generated_output(result, reference_date)
@ -216,7 +213,6 @@ what is the weather = none
return TemporalConstraint(start_date=start_date, end_date=end_date)
except (ValueError, AttributeError) as e:
logger.debug(f"Failed to parse T5 output '{result}': {e}")
return None
return None

38
hindsight-api/hindsight_api/engine/response_models.py
View file

@ -10,6 +10,31 @@ from typing import Optional, List, Dict, Any
from pydantic import BaseModel, Field, ConfigDict
class PersonalityTraits(BaseModel):
"""
Personality traits for a bank using the Big Five model.
All traits are scored 0.0-1.0 where higher values indicate stronger presence of the trait.
"""
openness: float = Field(description="Openness to experience (0.0-1.0)")
conscientiousness: float = Field(description="Conscientiousness and organization (0.0-1.0)")
extraversion: float = Field(description="Extraversion and sociability (0.0-1.0)")
agreeableness: float = Field(description="Agreeableness and cooperation (0.0-1.0)")
neuroticism: float = Field(description="Emotional sensitivity and neuroticism (0.0-1.0)")
bias_strength: float = Field(description="How strongly personality influences thinking (0.0-1.0)")
model_config = ConfigDict(json_schema_extra={
"example": {
"openness": 0.8,
"conscientiousness": 0.6,
"extraversion": 0.4,
"agreeableness": 0.7,
"neuroticism": 0.3,
"bias_strength": 0.5
}
})
class MemoryFact(BaseModel):
"""
A single memory fact returned by search or think operations.
@ -29,6 +54,7 @@ class MemoryFact(BaseModel):
"mentioned_at": "2024-01-15T10:30:00Z",
"document_id": "session_abc123",
"metadata": {"source": "slack"},
"chunk_id": "bank123_session_abc123_0",
"activation": 0.95
}
})
@ -43,11 +69,19 @@ class MemoryFact(BaseModel):
mentioned_at: Optional[str] = Field(None, description="ISO format date when the fact was mentioned/learned")
document_id: Optional[str] = Field(None, description="ID of the document this memory belongs to")
metadata: Optional[Dict[str, str]] = Field(None, description="User-defined metadata")
chunk_id: Optional[str] = Field(None, description="ID of the chunk this fact was extracted from (format: bank_id_document_id_chunk_index)")
# Internal metrics (used by system but may not be exposed in API)
activation: Optional[float] = Field(None, description="Internal activation score")
class ChunkInfo(BaseModel):
"""Information about a chunk."""
chunk_text: str = Field(description="The raw chunk text")
chunk_index: int = Field(description="Index of the chunk within the document")
truncated: bool = Field(default=False, description="Whether the chunk was truncated due to token limits")
class RecallResult(BaseModel):
"""
Result from a recall operation.
@ -81,6 +115,10 @@ class RecallResult(BaseModel):
None,
description="Entity states for entities mentioned in results (keyed by canonical name)"
)
chunks: Optional[Dict[str, ChunkInfo]] = Field(
None,
description="Chunks for facts, keyed by '{document_id}_{chunk_index}'"
)
class ReflectResult(BaseModel):

50
hindsight-api/hindsight_api/engine/retain/__init__.py Normal file
View file

@ -0,0 +1,50 @@
"""
Retain pipeline modules for storing memories.
This package contains modular components for the retain operation:
- types: Type definitions for retain pipeline
- fact_extraction: Extract facts from content
- embedding_processing: Augment texts and generate embeddings
- deduplication: Check for duplicate facts
- entity_processing: Process and resolve entities
- link_creation: Create temporal, semantic, entity, and causal links
- chunk_storage: Handle chunk storage
- fact_storage: Handle fact insertion into database
"""
from .types import (
RetainContent,
ExtractedFact,
ProcessedFact,
ChunkMetadata,
EntityRef,
CausalRelation,
RetainBatch
)
from . import fact_extraction
from . import embedding_processing
from . import deduplication
from . import entity_processing
from . import link_creation
from . import chunk_storage
from . import fact_storage
__all__ = [
# Types
"RetainContent",
"ExtractedFact",
"ProcessedFact",
"ChunkMetadata",
"EntityRef",
"CausalRelation",
"RetainBatch",
# Modules
"fact_extraction",
"embedding_processing",
"deduplication",
"entity_processing",
"link_creation",
"chunk_storage",
"fact_storage",
]

42
hindsight-api/hindsight_api/engine/bank_utils.py → hindsight-api/hindsight_api/engine/retain/bank_utils.py
View file

@ -5,9 +5,10 @@ bank profile utilities for personality and background management.
import json
import logging
import re
from typing import Dict, Optional
from typing import Dict, Optional, TypedDict
from pydantic import BaseModel, Field
from .db_utils import acquire_with_retry
from ..db_utils import acquire_with_retry
from ..response_models import PersonalityTraits
logger = logging.getLogger(__name__)
@ -21,14 +22,11 @@ DEFAULT_PERSONALITY = {
}
class PersonalityTraits(BaseModel):
"""Big Five personality traits with bias strength (all values 0.0-1.0)."""
openness: float = Field(description="Creativity, curiosity, openness to new ideas (0.0-1.0)")
conscientiousness: float = Field(description="Organization, discipline, goal-directed (0.0-1.0)")
extraversion: float = Field(description="Sociability, assertiveness, energy from others (0.0-1.0)")
agreeableness: float = Field(description="Cooperation, empathy, consideration (0.0-1.0)")
neuroticism: float = Field(description="Emotional sensitivity, anxiety, stress response (0.0-1.0)")
bias_strength: float = Field(description="How much personality influences opinions (0.0-1.0)")
class BankProfile(TypedDict):
"""Type for bank profile data."""
name: str
personality: PersonalityTraits
background: str
class BackgroundMergeResponse(BaseModel):
@ -37,7 +35,7 @@ class BackgroundMergeResponse(BaseModel):
personality: PersonalityTraits = Field(description="Inferred Big Five personality traits")
async def get_bank_profile(pool, bank_id: str) -> Dict:
async def get_bank_profile(pool, bank_id: str) -> BankProfile:
"""
Get bank profile (name, personality + background).
Auto-creates bank with default values if not exists.
@ -47,7 +45,7 @@ async def get_bank_profile(pool, bank_id: str) -> Dict:
bank_id: bank IDentifier
Returns:
Dict with 'name' (str), 'personality' (dict) and 'background' (str) keys
BankProfile with name, typed PersonalityTraits, and background
"""
async with acquire_with_retry(pool) as conn:
# Try to get existing bank
@ -65,11 +63,11 @@ async def get_bank_profile(pool, bank_id: str) -> Dict:
if isinstance(personality_data, str):
personality_data = json.loads(personality_data)
return {
"name": row["name"],
"personality": personality_data,
"background": row["background"]
}
return BankProfile(
name=row["name"],
personality=PersonalityTraits(**personality_data),
background=row["background"]
)
# Bank doesn't exist, create with defaults
await conn.execute(
@ -84,11 +82,11 @@ async def get_bank_profile(pool, bank_id: str) -> Dict:
""
)
return {
"name": bank_id,
"personality": DEFAULT_PERSONALITY.copy(),
"background": ""
}
return BankProfile(
name=bank_id,
personality=PersonalityTraits(**DEFAULT_PERSONALITY),
background=""
)
async def update_bank_personality(

82
hindsight-api/hindsight_api/engine/retain/chunk_storage.py Normal file
View file

@ -0,0 +1,82 @@
"""
Chunk storage for retain pipeline.
Handles storage of document chunks in the database.
"""
import logging
from typing import List, Dict, Optional
from .types import ChunkMetadata
logger = logging.getLogger(__name__)
async def store_chunks_batch(
conn,
bank_id: str,
document_id: str,
chunks: List[ChunkMetadata]
) -> Dict[int, str]:
"""
Store document chunks in the database.
Args:
conn: Database connection
bank_id: Bank identifier
document_id: Document identifier
chunks: List of ChunkMetadata objects
Returns:
Dictionary mapping global chunk index to chunk_id
"""
if not chunks:
return {}
# Prepare chunk data for batch insert
chunk_ids = []
chunk_texts = []
chunk_indices = []
chunk_id_map = {}
for chunk in chunks:
chunk_id = f"{bank_id}_{document_id}_{chunk.chunk_index}"
chunk_ids.append(chunk_id)
chunk_texts.append(chunk.chunk_text)
chunk_indices.append(chunk.chunk_index)
chunk_id_map[chunk.chunk_index] = chunk_id
# Batch insert all chunks
await conn.execute(
"""
INSERT INTO chunks (chunk_id, document_id, bank_id, chunk_text, chunk_index)
SELECT * FROM unnest($1::text[], $2::text[], $3::text[], $4::text[], $5::integer[])
""",
chunk_ids,
[document_id] * len(chunk_texts),
[bank_id] * len(chunk_texts),
chunk_texts,
chunk_indices
)
return chunk_id_map
def map_facts_to_chunks(
facts_chunk_indices: List[int],
chunk_id_map: Dict[int, str]
) -> List[Optional[str]]:
"""
Map fact chunk indices to chunk IDs.
Args:
facts_chunk_indices: List of chunk indices for each fact
chunk_id_map: Dictionary mapping chunk index to chunk_id
Returns:
List of chunk_ids (same length as facts_chunk_indices)
"""
chunk_ids = []
for chunk_idx in facts_chunk_indices:
chunk_id = chunk_id_map.get(chunk_idx)
chunk_ids.append(chunk_id)
return chunk_ids

97
hindsight-api/hindsight_api/engine/retain/deduplication.py Normal file
View file

@ -0,0 +1,97 @@
"""
Deduplication logic for retain pipeline.
Checks for duplicate facts using semantic similarity and temporal proximity.
"""
import logging
from datetime import datetime
from typing import List
from collections import defaultdict
from .types import ProcessedFact
logger = logging.getLogger(__name__)
async def check_duplicates_batch(
conn,
bank_id: str,
facts: List[ProcessedFact],
duplicate_checker_fn
) -> List[bool]:
"""
Check which facts are duplicates using batched time-window queries.
Groups facts by 12-hour time buckets to efficiently check for duplicates
within a 24-hour window.
Args:
conn: Database connection
bank_id: Bank identifier
facts: List of ProcessedFact objects to check
duplicate_checker_fn: Async function(conn, bank_id, texts, embeddings, date, time_window_hours)
that returns List[bool] indicating duplicates
Returns:
List of boolean flags (same length as facts) indicating if each fact is a duplicate
"""
if not facts:
return []
# Group facts by event_date (rounded to 12-hour buckets) for efficient batching
time_buckets = defaultdict(list)
for idx, fact in enumerate(facts):
# Use occurred_start as the representative date
fact_date = fact.occurred_start
# Round to 12-hour bucket to group similar times
bucket_key = fact_date.replace(
hour=(fact_date.hour // 12) * 12,
minute=0,
second=0,
microsecond=0
)
time_buckets[bucket_key].append((idx, fact))
# Process each bucket in batch
all_is_duplicate = [False] * len(facts)
for bucket_date, bucket_items in time_buckets.items():
indices = [item[0] for item in bucket_items]
texts = [item[1].fact_text for item in bucket_items]
embeddings = [item[1].embedding for item in bucket_items]
# Check duplicates for this time bucket
dup_flags = await duplicate_checker_fn(
conn,
bank_id,
texts,
embeddings,
bucket_date,
time_window_hours=24
)
# Map results back to original indices
for idx, is_dup in zip(indices, dup_flags):
all_is_duplicate[idx] = is_dup
return all_is_duplicate
def filter_duplicates(
facts: List[ProcessedFact],
is_duplicate_flags: List[bool]
) -> List[ProcessedFact]:
"""
Filter out duplicate facts based on duplicate flags.
Args:
facts: List of ProcessedFact objects
is_duplicate_flags: Boolean flags indicating which facts are duplicates
Returns:
List of non-duplicate facts
"""
if len(facts) != len(is_duplicate_flags):
raise ValueError(f"Mismatch between facts ({len(facts)}) and flags ({len(is_duplicate_flags)})")
return [fact for fact, is_dup in zip(facts, is_duplicate_flags) if not is_dup]

62
hindsight-api/hindsight_api/engine/retain/embedding_processing.py Normal file
View file

@ -0,0 +1,62 @@
"""
Embedding processing for retain pipeline.
Handles augmenting fact texts with temporal information and generating embeddings.
"""
import logging
from typing import List
from datetime import datetime
from . import embedding_utils
from .types import ExtractedFact
logger = logging.getLogger(__name__)
def augment_texts_with_dates(facts: List[ExtractedFact], format_date_fn) -> List[str]:
"""
Augment fact texts with readable dates for better temporal matching.
This allows queries like "camping in June" to match facts that happened in June.
Args:
facts: List of ExtractedFact objects
format_date_fn: Function to format datetime to readable string
Returns:
List of augmented text strings (same length as facts)
"""
augmented_texts = []
for fact in facts:
# Use occurred_start as the representative date
fact_date = fact.occurred_start or fact.mentioned_at
readable_date = format_date_fn(fact_date)
# Augment text with date for embedding (but store original text in DB)
augmented_text = f"{fact.fact_text} (happened in {readable_date})"
augmented_texts.append(augmented_text)
return augmented_texts
async def generate_embeddings_batch(
embeddings_model,
texts: List[str]
) -> List[List[float]]:
"""
Generate embeddings for a batch of texts.
Args:
embeddings_model: Embeddings model instance
texts: List of text strings to embed
Returns:
List of embedding vectors (same length as texts)
"""
if not texts:
return []
embeddings = await embedding_utils.generate_embeddings_batch(
embeddings_model,
texts
)
return embeddings

0
hindsight-api/hindsight_api/engine/embedding_utils.py → hindsight-api/hindsight_api/engine/retain/embedding_utils.py
View file

83
hindsight-api/hindsight_api/engine/retain/entity_processing.py Normal file
View file

@ -0,0 +1,83 @@
"""
Entity processing for retain pipeline.
Handles entity extraction, resolution, and link creation for stored facts.
"""
import logging
from typing import List, Tuple, Dict, Any
from uuid import UUID
from .types import ProcessedFact, EntityRef
from . import link_utils
logger = logging.getLogger(__name__)
async def process_entities_batch(
entity_resolver,
conn,
bank_id: str,
unit_ids: List[str],
facts: List[ProcessedFact]
) -> List[Tuple[str, str, float]]:
"""
Process entities for all facts and create entity links.
This function:
1. Extracts entity mentions from fact texts
2. Resolves entity names to canonical entities
3. Creates entity records in the database
4. Returns entity links ready for insertion
Args:
entity_resolver: EntityResolver instance for entity resolution
conn: Database connection
bank_id: Bank identifier
unit_ids: List of unit IDs (same length as facts)
facts: List of ProcessedFact objects
Returns:
List of entity link tuples: (unit_id, entity_id, confidence)
"""
if not unit_ids or not facts:
return []
if len(unit_ids) != len(facts):
raise ValueError(f"Mismatch between unit_ids ({len(unit_ids)}) and facts ({len(facts)})")
# Extract data for link_utils function
fact_texts = [fact.fact_text for fact in facts]
fact_dates = [fact.occurred_start for fact in facts]
entities_per_fact = [[entity.name for entity in (fact.entities or [])] for fact in facts]
# Use existing link_utils function for entity processing
entity_links = await link_utils.extract_entities_batch_optimized(
entity_resolver,
conn,
bank_id,
unit_ids,
fact_texts,
"", # context (not used in current implementation)
fact_dates,
entities_per_fact,
[] # log_buffer (optional)
)
return entity_links
async def insert_entity_links_batch(
conn,
entity_links: List[Tuple[str, str, float]]
) -> None:
"""
Insert entity links in batch.
Args:
conn: Database connection
entity_links: List of (unit_id, entity_id, confidence) tuples
"""
if not entity_links:
return
await link_utils.insert_entity_links_batch(conn, entity_links)

1082
hindsight-api/hindsight_api/engine/retain/fact_extraction.py Normal file
View file

File diff suppressed because it is too large Load diff

168
hindsight-api/hindsight_api/engine/retain/fact_storage.py Normal file
View file

@ -0,0 +1,168 @@
"""
Fact storage for retain pipeline.
Handles insertion of facts into the database.
"""
import logging
import json
from typing import List, Optional
from uuid import UUID
from .types import ProcessedFact
logger = logging.getLogger(__name__)
async def insert_facts_batch(
conn,
bank_id: str,
facts: List[ProcessedFact],
document_id: Optional[str] = None
) -> List[str]:
"""
Insert facts into the database in batch.
Args:
conn: Database connection
bank_id: Bank identifier
facts: List of ProcessedFact objects to insert
document_id: Optional document ID to associate with facts
Returns:
List of unit IDs (UUIDs as strings) for the inserted facts
"""
if not facts:
return []
# Prepare data for batch insert
fact_texts = []
embeddings = []
occurred_starts = []
occurred_ends = []
mentioned_ats = []
contexts = []
fact_types = []
confidence_scores = []
access_counts = []
metadata_jsons = []
chunk_ids = []
document_ids = []
for fact in facts:
fact_texts.append(fact.fact_text)
# Convert embedding to string for asyncpg vector type
embeddings.append(str(fact.embedding))
occurred_starts.append(fact.occurred_start)
occurred_ends.append(fact.occurred_end)
mentioned_ats.append(fact.mentioned_at)
contexts.append(fact.context)
fact_types.append(fact.fact_type)
# confidence_score is only for opinion facts
confidence_scores.append(1.0 if fact.fact_type == 'opinion' else None)
access_counts.append(0) # Initial access count
metadata_jsons.append(json.dumps(fact.metadata))
chunk_ids.append(fact.chunk_id)
document_ids.append(document_id)
# Batch insert all facts
# Note: event_date is set to occurred_start for backward compatibility
results = await conn.fetch(
"""
INSERT INTO memory_units (bank_id, text, embedding, event_date, occurred_start, occurred_end, mentioned_at,
context, fact_type, confidence_score, access_count, metadata, chunk_id, document_id)
SELECT $1, * FROM unnest(
$2::text[], $3::vector[], $4::timestamptz[], $5::timestamptz[], $6::timestamptz[], $7::timestamptz[],
$8::text[], $9::text[], $10::float[], $11::int[], $12::jsonb[], $13::text[], $14::text[]
)
RETURNING id
""",
bank_id,
fact_texts,
embeddings,
occurred_starts, # event_date (for backward compatibility)
occurred_starts,
occurred_ends,
mentioned_ats,
contexts,
fact_types,
confidence_scores,
access_counts,
metadata_jsons,
chunk_ids,
document_ids
)
unit_ids = [str(row['id']) for row in results]
return unit_ids
async def ensure_bank_exists(conn, bank_id: str) -> None:
"""
Ensure bank exists in the database.
Creates bank with default values if it doesn't exist.
Args:
conn: Database connection
bank_id: Bank identifier
"""
await conn.execute(
"""
INSERT INTO banks (bank_id, personality, background)
VALUES ($1, $2::jsonb, $3)
ON CONFLICT (bank_id) DO UPDATE
SET updated_at = NOW()
""",
bank_id,
'{"openness": 0.5, "conscientiousness": 0.5, "extraversion": 0.5, "agreeableness": 0.5, "neuroticism": 0.5, "bias_strength": 0.5}',
""
)
async def handle_document_tracking(
conn,
bank_id: str,
document_id: str,
combined_content: str,
is_first_batch: bool
) -> None:
"""
Handle document tracking in the database.
Args:
conn: Database connection
bank_id: Bank identifier
document_id: Document identifier
combined_content: Combined content text from all content items
is_first_batch: Whether this is the first batch (for chunked operations)
"""
import hashlib
# Calculate content hash
content_hash = hashlib.sha256(combined_content.encode()).hexdigest()
# Always delete old document first if it exists (cascades to units and links)
# Only delete on the first batch to avoid deleting data we just inserted
if is_first_batch:
await conn.fetchval(
"DELETE FROM documents WHERE id = $1 AND bank_id = $2 RETURNING id",
document_id, bank_id
)
# Insert document (or update if exists from concurrent operations)
await conn.execute(
"""
INSERT INTO documents (id, bank_id, original_text, content_hash, metadata)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (id, bank_id) DO UPDATE
SET original_text = EXCLUDED.original_text,
content_hash = EXCLUDED.content_hash,
metadata = EXCLUDED.metadata,
updated_at = NOW()
""",
document_id,
bank_id,
combined_content,
content_hash,
json.dumps({}) # Empty metadata dict
)

121
hindsight-api/hindsight_api/engine/retain/link_creation.py Normal file
View file

@ -0,0 +1,121 @@
"""
Link creation for retain pipeline.
Handles creation of temporal, semantic, and causal links between facts.
"""
import logging
from typing import List
from .types import ProcessedFact, CausalRelation
from . import link_utils
logger = logging.getLogger(__name__)
async def create_temporal_links_batch(
conn,
bank_id: str,
unit_ids: List[str]
) -> None:
"""
Create temporal links between facts.
Links facts that occurred close in time to each other.
Args:
conn: Database connection
bank_id: Bank identifier
unit_ids: List of unit IDs to create links for
"""
if not unit_ids:
return
await link_utils.create_temporal_links_batch_per_fact(
conn,
bank_id,
unit_ids,
log_buffer=[]
)
async def create_semantic_links_batch(
conn,
bank_id: str,
unit_ids: List[str],
embeddings: List[List[float]]
) -> None:
"""
Create semantic links between facts.
Links facts that are semantically similar based on embeddings.
Args:
conn: Database connection
bank_id: Bank identifier
unit_ids: List of unit IDs to create links for
embeddings: List of embedding vectors (same length as unit_ids)
"""
if not unit_ids or not embeddings:
return
if len(unit_ids) != len(embeddings):
raise ValueError(f"Mismatch between unit_ids ({len(unit_ids)}) and embeddings ({len(embeddings)})")
await link_utils.create_semantic_links_batch(
conn,
bank_id,
unit_ids,
embeddings,
log_buffer=[]
)
async def create_causal_links_batch(
conn,
unit_ids: List[str],
facts: List[ProcessedFact]
) -> int:
"""
Create causal links between facts.
Links facts that have causal relationships (causes, enables, prevents).
Args:
conn: Database connection
unit_ids: List of unit IDs (same length as facts)
facts: List of ProcessedFact objects with causal_relations
Returns:
Number of causal links created
"""
if not unit_ids or not facts:
return 0
if len(unit_ids) != len(facts):
raise ValueError(f"Mismatch between unit_ids ({len(unit_ids)}) and facts ({len(facts)})")
# Extract causal relations in the format expected by link_utils
# Format: List of lists, where each inner list is the causal relations for that fact
causal_relations_per_fact = []
for fact in facts:
if fact.causal_relations:
# Convert CausalRelation objects to dicts
relations_dicts = [
{
'relation_type': rel.relation_type,
'target_fact_index': rel.target_fact_index,
'strength': rel.strength
}
for rel in fact.causal_relations
]
causal_relations_per_fact.append(relations_dicts)
else:
causal_relations_per_fact.append([])
link_count = await link_utils.create_causal_links_batch(
conn,
unit_ids,
causal_relations_per_fact
)
return link_count

25
hindsight-api/hindsight_api/engine/link_utils.py → hindsight-api/hindsight_api/engine/retain/link_utils.py
View file

@ -18,7 +18,7 @@ def _log(log_buffer, message, level='info'):
if level == 'info':
logger.info(message)
else:
logger.debug(message)
logger.log(logging.WARNING if level == 'warning' else logging.ERROR, message)
async def extract_entities_batch_optimized(
@ -62,7 +62,8 @@ async def extract_entities_batch_optimized(
for ent in entity_list:
# Handle both Entity objects and dicts
if hasattr(ent, 'text'):
formatted_entities.append({'text': ent.text, 'type': ent.type})
# Entity objects only have 'text', default type to 'CONCEPT'
formatted_entities.append({'text': ent.text, 'type': 'CONCEPT'})
elif isinstance(ent, dict):
formatted_entities.append({'text': ent.get('text', ''), 'type': ent.get('type', 'CONCEPT')})
all_entities.append(formatted_entities)
@ -502,6 +503,16 @@ async def create_causal_links_batch(
relation_type = relation['relation_type']
strength = relation.get('strength', 1.0)
# Validate relation_type - must match database constraint
valid_types = {'causes', 'caused_by', 'enables', 'prevents'}
if relation_type not in valid_types:
logger.error(
f"Invalid relation_type '{relation_type}' (type: {type(relation_type).__name__}) "
f"from fact {fact_idx}. Must be one of: {valid_types}. "
f"Relation data: {relation}"
)
continue
# Validate target index
if target_idx < 0 or target_idx >= len(unit_ids):
logger.warning(f"Invalid target_fact_index {target_idx} in causal relation from fact {fact_idx}")
@ -518,10 +529,10 @@ async def create_causal_links_batch(
# weight is the strength of the relationship
links.append((from_unit_id, to_unit_id, relation_type, strength, None))
logger.debug(f"Generated {len(links)} causal links in {time_mod.time() - create_start:.3f}s")
if links:
insert_start = time_mod.time()
try:
await conn.executemany(
"""
INSERT INTO memory_links (from_unit_id, to_unit_id, link_type, weight, entity_id)
@ -530,7 +541,13 @@ async def create_causal_links_batch(
""",
links
)
logger.debug(f"Inserted {len(links)} causal links in {time_mod.time() - insert_start:.3f}s")
except Exception as db_error:
# Log the actual data being inserted for debugging
logger.error(f"Database insert failed for causal links. Error: {db_error}")
logger.error(f"Attempted to insert {len(links)} links. First few:")
for i, link in enumerate(links[:3]):
logger.error(f" Link {i}: from={link[0]}, to={link[1]}, type='{link[2]}' (repr={repr(link[2])}), weight={link[3]}, entity={link[4]}")
raise
return len(links)

298
hindsight-api/hindsight_api/engine/retain/orchestrator.py Normal file
View file

@ -0,0 +1,298 @@
"""
Main orchestrator for the retain pipeline.
Coordinates all retain pipeline modules to store memories efficiently.
"""
import logging
import time
import uuid
from datetime import datetime, timezone
from typing import List, Dict, Any, Optional
from . import bank_utils
from ..db_utils import acquire_with_retry
def utcnow():
"""Get current UTC time."""
return datetime.now(timezone.utc)
from .types import RetainContent, ExtractedFact, ProcessedFact
from . import (
fact_extraction,
embedding_processing,
deduplication,
chunk_storage,
fact_storage,
entity_processing,
link_creation
)
logger = logging.getLogger(__name__)
async def retain_batch(
pool,
embeddings_model,
llm_config,
entity_resolver,
task_backend,
format_date_fn,
duplicate_checker_fn,
bank_id: str,
contents_dicts: List[Dict[str, Any]],
document_id: Optional[str] = None,
is_first_batch: bool = True,
fact_type_override: Optional[str] = None,
confidence_score: Optional[float] = None,
) -> List[List[str]]:
"""
Process a batch of content through the retain pipeline.
Args:
pool: Database connection pool
embeddings_model: Embeddings model for generating embeddings
llm_config: LLM configuration for fact extraction
entity_resolver: Entity resolver for entity processing
task_backend: Task backend for background jobs
format_date_fn: Function to format datetime to readable string
duplicate_checker_fn: Function to check for duplicate facts
bank_id: Bank identifier
contents_dicts: List of content dictionaries
document_id: Optional document ID
is_first_batch: Whether this is the first batch
fact_type_override: Override fact type for all facts
confidence_score: Confidence score for opinions
Returns:
List of unit ID lists (one list per content item)
"""
start_time = time.time()
total_chars = sum(len(item.get("content", "")) for item in contents_dicts)
# Buffer all logs
log_buffer = []
log_buffer.append(f"{'='*60}")
log_buffer.append(f"RETAIN_BATCH START: {bank_id}")
log_buffer.append(f"Batch size: {len(contents_dicts)} content items, {total_chars:,} chars")
log_buffer.append(f"{'='*60}")
# Get bank profile
profile = await bank_utils.get_bank_profile(pool, bank_id)
agent_name = profile["name"]
# Convert dicts to RetainContent objects
contents = []
for item in contents_dicts:
content = RetainContent(
content=item["content"],
context=item.get("context", ""),
event_date=item.get("event_date") or utcnow(),
metadata=item.get("metadata", {})
)
contents.append(content)
# Step 1: Extract facts from all contents
step_start = time.time()
extract_opinions = (fact_type_override == 'opinion')
extracted_facts, chunks = await fact_extraction.extract_facts_from_contents(
contents,
llm_config,
agent_name,
extract_opinions
)
log_buffer.append(f"[1] Extract facts: {len(extracted_facts)} facts from {len(contents)} contents in {time.time() - step_start:.3f}s")
if not extracted_facts:
return [[] for _ in contents]
# Apply fact_type_override if provided
if fact_type_override:
for fact in extracted_facts:
fact.fact_type = fact_type_override
# Step 2: Augment texts and generate embeddings
step_start = time.time()
augmented_texts = embedding_processing.augment_texts_with_dates(extracted_facts, format_date_fn)
embeddings = await embedding_processing.generate_embeddings_batch(embeddings_model, augmented_texts)
log_buffer.append(f"[2] Generate embeddings: {len(embeddings)} embeddings in {time.time() - step_start:.3f}s")
# Step 3: Convert to ProcessedFact objects (without chunk_ids yet)
processed_facts = [
ProcessedFact.from_extracted_fact(extracted_fact, embedding)
for extracted_fact, embedding in zip(extracted_facts, embeddings)
]
# Step 4: Database transaction
async with acquire_with_retry(pool) as conn:
async with conn.transaction():
# Ensure bank exists
await fact_storage.ensure_bank_exists(conn, bank_id)
# Handle document tracking
if document_id:
combined_content = "\n".join([c.get("content", "") for c in contents_dicts])
await fact_storage.handle_document_tracking(
conn, bank_id, document_id, combined_content, is_first_batch
)
elif chunks:
# Generate document_id for chunk storage
document_id = str(uuid.uuid4())
combined_content = "\n".join([c.get("content", "") for c in contents_dicts])
await fact_storage.handle_document_tracking(
conn, bank_id, document_id, combined_content, is_first_batch
)
log_buffer.append(f"[2.5] Generated document_id: {document_id}")
# Store chunks and map to facts
step_start = time.time()
chunk_id_map = {}
if document_id and chunks:
chunk_id_map = await chunk_storage.store_chunks_batch(conn, bank_id, document_id, chunks)
log_buffer.append(f"[3] Store chunks: {len(chunks)} chunks in {time.time() - step_start:.3f}s")
# Map chunk_ids to facts
facts_chunk_indices = [fact.chunk_index for fact in extracted_facts]
chunk_ids = chunk_storage.map_facts_to_chunks(facts_chunk_indices, chunk_id_map)
for processed_fact, chunk_id in zip(processed_facts, chunk_ids):
processed_fact.chunk_id = chunk_id
# Deduplication
step_start = time.time()
is_duplicate_flags = await deduplication.check_duplicates_batch(
conn, bank_id, processed_facts, duplicate_checker_fn
)
log_buffer.append(f"[4] Deduplication: {sum(is_duplicate_flags)} duplicates in {time.time() - step_start:.3f}s")
# Filter out duplicates
non_duplicate_facts = deduplication.filter_duplicates(processed_facts, is_duplicate_flags)
if not non_duplicate_facts:
return [[] for _ in contents]
# Insert facts
step_start = time.time()
unit_ids = await fact_storage.insert_facts_batch(conn, bank_id, non_duplicate_facts, document_id)
log_buffer.append(f"[5] Insert facts: {len(unit_ids)} units in {time.time() - step_start:.3f}s")
# Process entities
step_start = time.time()
entity_links = await entity_processing.process_entities_batch(
entity_resolver, conn, bank_id, unit_ids, non_duplicate_facts
)
log_buffer.append(f"[6] Process entities: {len(entity_links)} links in {time.time() - step_start:.3f}s")
# Create temporal links
step_start = time.time()
await link_creation.create_temporal_links_batch(conn, bank_id, unit_ids)
log_buffer.append(f"[7] Temporal links: {time.time() - step_start:.3f}s")
# Create semantic links
step_start = time.time()
embeddings_for_links = [fact.embedding for fact in non_duplicate_facts]
await link_creation.create_semantic_links_batch(conn, bank_id, unit_ids, embeddings_for_links)
log_buffer.append(f"[8] Semantic links: {time.time() - step_start:.3f}s")
# Insert entity links
step_start = time.time()
if entity_links:
await entity_processing.insert_entity_links_batch(conn, entity_links)
log_buffer.append(f"[9] Entity links: {time.time() - step_start:.3f}s")
# Create causal links
step_start = time.time()
causal_link_count = await link_creation.create_causal_links_batch(conn, unit_ids, non_duplicate_facts)
log_buffer.append(f"[10] Causal links: {causal_link_count} links in {time.time() - step_start:.3f}s")
# Map results back to original content items
result_unit_ids = _map_results_to_contents(
contents, extracted_facts, is_duplicate_flags, unit_ids
)
total_time = time.time() - start_time
log_buffer.append(f"{'='*60}")
log_buffer.append(f"RETAIN_BATCH COMPLETE: {len(unit_ids)} units in {total_time:.3f}s")
log_buffer.append(f"{'='*60}")
logger.info("\n" + "\n".join(log_buffer) + "\n")
# Trigger background tasks
await _trigger_background_tasks(
task_backend,
bank_id,
unit_ids,
non_duplicate_facts,
entity_links
)
return result_unit_ids
def _map_results_to_contents(
contents: List[RetainContent],
extracted_facts: List[ExtractedFact],
is_duplicate_flags: List[bool],
unit_ids: List[str]
) -> List[List[str]]:
"""
Map created unit IDs back to original content items.
Accounts for duplicates when mapping back.
"""
result_unit_ids = []
filtered_idx = 0
# Group facts by content_index
facts_by_content = {i: [] for i in range(len(contents))}
for i, fact in enumerate(extracted_facts):
facts_by_content[fact.content_index].append(i)
for content_index in range(len(contents)):
content_unit_ids = []
for fact_idx in facts_by_content[content_index]:
if not is_duplicate_flags[fact_idx]:
content_unit_ids.append(unit_ids[filtered_idx])
filtered_idx += 1
result_unit_ids.append(content_unit_ids)
return result_unit_ids
async def _trigger_background_tasks(
task_backend,
bank_id: str,
unit_ids: List[str],
facts: List[ProcessedFact],
entity_links: List
) -> None:
"""Trigger opinion reinforcement and observation regeneration tasks."""
# Trigger opinion reinforcement if there are entities
fact_entities = [[e.name for e in fact.entities] for fact in facts]
if any(fact_entities):
await task_backend.submit_task({
'type': 'reinforce_opinion',
'bank_id': bank_id,
'created_unit_ids': unit_ids,
'unit_texts': [fact.fact_text for fact in facts],
'unit_entities': fact_entities
})
# Trigger observation regeneration for top entities
TOP_N_ENTITIES = 5
MIN_FACTS_THRESHOLD = 5
if entity_links:
unique_entity_ids = set()
for link in entity_links:
# links are tuples: (unit_id, entity_id, confidence)
if len(link) >= 2 and link[1]:
unique_entity_ids.add(str(link[1]))
if unique_entity_ids:
await task_backend.submit_task({
'type': 'regenerate_observations',
'bank_id': bank_id,
'entity_ids': list(unique_entity_ids)[:TOP_N_ENTITIES],
'min_facts': MIN_FACTS_THRESHOLD
})

199
hindsight-api/hindsight_api/engine/retain/types.py Normal file
View file

@ -0,0 +1,199 @@
"""
Type definitions for the retain pipeline.
These dataclasses provide type safety throughout the retain operation,
from content input to fact storage.
"""
from dataclasses import dataclass, field
from typing import List, Optional, Dict, Any
from datetime import datetime
from uuid import UUID
@dataclass
class RetainContent:
"""
Input content item to be retained as memories.
Represents a single piece of content to extract facts from.
"""
content: str
context: str = ""
event_date: Optional[datetime] = None
metadata: Dict[str, str] = field(default_factory=dict)
def __post_init__(self):
"""Ensure event_date is set."""
if self.event_date is None:
from datetime import datetime, timezone
self.event_date = datetime.now(timezone.utc)
@dataclass
class ChunkMetadata:
"""
Metadata about a text chunk.
Used to track which facts were extracted from which chunks.
"""
chunk_text: str
fact_count: int
content_index: int # Index of the source content
chunk_index: int # Global chunk index across all contents
@dataclass
class EntityRef:
"""
Reference to an entity mentioned in a fact.
Entities are extracted by the LLM during fact extraction.
"""
name: str
canonical_name: Optional[str] = None # Resolved canonical name
entity_id: Optional[UUID] = None # Resolved entity ID
@dataclass
class CausalRelation:
"""
Causal relationship between facts.
Represents how one fact causes, enables, or prevents another.
"""
relation_type: str # "causes", "enables", "prevents", "caused_by"
target_fact_index: int # Index of the target fact in the batch
strength: float = 1.0 # Strength of the causal relationship
@dataclass
class ExtractedFact:
"""
Fact extracted from content by the LLM.
This is the raw output from fact extraction before processing.
"""
fact_text: str
fact_type: str # "world", "bank", "opinion", "observation"
entities: List[str] = field(default_factory=list)
occurred_start: Optional[datetime] = None
occurred_end: Optional[datetime] = None
causal_relations: List[CausalRelation] = field(default_factory=list)
# Context from the content item
content_index: int = 0 # Which content this fact came from
chunk_index: int = 0 # Which chunk this fact came from
context: str = ""
mentioned_at: Optional[datetime] = None
metadata: Dict[str, str] = field(default_factory=dict)
@dataclass
class ProcessedFact:
"""
Fact after processing and ready for storage.
Includes resolved entities, embeddings, and all necessary fields.
"""
# Core fact data
fact_text: str
fact_type: str
embedding: List[float]
# Temporal data
occurred_start: datetime
occurred_end: datetime
mentioned_at: datetime
# Context and metadata
context: str
metadata: Dict[str, str]
# Entities
entities: List[EntityRef] = field(default_factory=list)
# Causal relations
causal_relations: List[CausalRelation] = field(default_factory=list)
# Chunk reference
chunk_id: Optional[str] = None
# DB fields (set after insertion)
unit_id: Optional[UUID] = None
@property
def is_duplicate(self) -> bool:
"""Check if this fact was marked as a duplicate."""
return self.unit_id is None
@staticmethod
def from_extracted_fact(
extracted_fact: 'ExtractedFact',
embedding: List[float],
chunk_id: Optional[str] = None
) -> 'ProcessedFact':
"""
Create ProcessedFact from ExtractedFact.
Args:
extracted_fact: Source ExtractedFact
embedding: Generated embedding vector
chunk_id: Optional chunk ID
Returns:
ProcessedFact ready for storage
"""
from datetime import datetime, timezone
# Use occurred dates if available, otherwise use mentioned_at
occurred_start = extracted_fact.occurred_start or extracted_fact.mentioned_at
occurred_end = extracted_fact.occurred_end or extracted_fact.mentioned_at
mentioned_at = extracted_fact.mentioned_at or datetime.now(timezone.utc)
# Convert entity strings to EntityRef objects
entities = [EntityRef(name=name) for name in extracted_fact.entities]
return ProcessedFact(
fact_text=extracted_fact.fact_text,
fact_type=extracted_fact.fact_type,
embedding=embedding,
occurred_start=occurred_start,
occurred_end=occurred_end,
mentioned_at=mentioned_at,
context=extracted_fact.context,
metadata=extracted_fact.metadata,
entities=entities,
causal_relations=extracted_fact.causal_relations,
chunk_id=chunk_id
)
@dataclass
class RetainBatch:
"""
A batch of content to retain.
Tracks all facts, chunks, and metadata for a batch operation.
"""
bank_id: str
contents: List[RetainContent]
document_id: Optional[str] = None
fact_type_override: Optional[str] = None
confidence_score: Optional[float] = None
# Extracted data (populated during processing)
extracted_facts: List[ExtractedFact] = field(default_factory=list)
processed_facts: List[ProcessedFact] = field(default_factory=list)
chunks: List[ChunkMetadata] = field(default_factory=list)
# Results (populated after storage)
unit_ids_by_content: List[List[str]] = field(default_factory=list)
def get_facts_for_content(self, content_index: int) -> List[ExtractedFact]:
"""Get all extracted facts for a specific content item."""
return [f for f in self.extracted_facts if f.content_index == content_index]
def get_chunks_for_content(self, content_index: int) -> List[ChunkMetadata]:
"""Get all chunks for a specific content item."""
return [c for c in self.chunks if c.content_index == content_index]

67
hindsight-api/hindsight_api/engine/search_helpers.py → hindsight-api/hindsight_api/engine/search/fusion.py
View file

@ -4,70 +4,81 @@ Helper functions for hybrid search (semantic + BM25 + graph).
from typing import List, Dict, Any, Tuple
import asyncio
from .types import RetrievalResult, MergedCandidate
def reciprocal_rank_fusion(
result_lists: List[List[Tuple[str, Dict[str, Any]]]],
result_lists: List[List[RetrievalResult]],
k: int = 60
) -> List[Tuple[str, Dict[str, Any], Dict[str, float]]]:
) -> List[MergedCandidate]:
"""
Merge multiple ranked result lists using Reciprocal Rank Fusion.
RRF formula: score(d) = sum_over_lists(1 / (k + rank(d)))
Args:
result_lists: List of result lists, each containing (id, data) tuples
result_lists: List of result lists, each containing RetrievalResult objects
k: Constant for RRF formula (default: 60)
Returns:
Merged list of (id, data, scores_dict) tuples, sorted by RRF score
Merged list of MergedCandidate objects, sorted by RRF score
Example:
semantic_results = [("id1", {...}), ("id2", {...}), ...]
bm25_results = [("id2", {...}), ("id3", {...}), ...]
graph_results = [("id1", {...}), ("id4", {...}), ...]
semantic_results = [RetrievalResult(...), RetrievalResult(...), ...]
bm25_results = [RetrievalResult(...), RetrievalResult(...), ...]
graph_results = [RetrievalResult(...), RetrievalResult(...), ...]
merged = reciprocal_rank_fusion([semantic_results, bm25_results, graph_results])
# Returns: [("id2", {...}, {"rrf": 0.05, "semantic_rank": 2, ...}), ...]
# Returns: [MergedCandidate(...), MergedCandidate(...), ...]
"""
# Track scores from each list
rrf_scores = {}
source_ranks = {} # Track rank from each source
source_scores = {} # Track original score from each source
all_data = {} # Store the actual data
source_ranks = {} # Track rank from each source for each doc_id
all_retrievals = {} # Store the actual RetrievalResult (use first occurrence)
source_names = ["semantic", "bm25", "graph"]
source_names = ["semantic", "bm25", "graph", "temporal"]
for source_idx, results in enumerate(result_lists):
source_name = source_names[source_idx] if source_idx < len(source_names) else f"source_{source_idx}"
for rank, (doc_id, data) in enumerate(results, start=1):
# Store data (use first occurrence)
if doc_id not in all_data:
all_data[doc_id] = data
for rank, retrieval in enumerate(results, start=1):
# Type check to catch tuple issues
if isinstance(retrieval, tuple):
raise TypeError(
f"Expected RetrievalResult but got tuple in {source_name} results at rank {rank}. "
f"Tuple value: {retrieval[:2] if len(retrieval) >= 2 else retrieval}. "
f"This suggests the retrieval function returned tuples instead of RetrievalResult objects."
)
if not isinstance(retrieval, RetrievalResult):
raise TypeError(
f"Expected RetrievalResult but got {type(retrieval).__name__} in {source_name} results at rank {rank}"
)
doc_id = retrieval.id
# Store retrieval result (use first occurrence)
if doc_id not in all_retrievals:
all_retrievals[doc_id] = retrieval
# Calculate RRF score contribution
if doc_id not in rrf_scores:
rrf_scores[doc_id] = 0.0
source_ranks[doc_id] = {}
source_scores[doc_id] = {}
rrf_scores[doc_id] += 1.0 / (k + rank)
source_ranks[doc_id][f"{source_name}_rank"] = rank
# Store original score if available
if "score" in data:
source_scores[doc_id][f"{source_name}_score"] = data["score"]
# Combine into final results with metadata
merged_results = []
for doc_id, rrf_score in sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True):
scores_dict = {
"rrf_score": rrf_score,
**source_ranks[doc_id],
**source_scores[doc_id]
}
merged_results.append((doc_id, all_data[doc_id], scores_dict))
for rrf_rank, (doc_id, rrf_score) in enumerate(
sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True), start=1
):
merged_candidate = MergedCandidate(
retrieval=all_retrievals[doc_id],
rrf_score=rrf_score,
rrf_rank=rrf_rank,
source_ranks=source_ranks[doc_id]
)
merged_results.append(merged_candidate)
return merged_results

3
hindsight-api/hindsight_api/engine/observation_utils.py → hindsight-api/hindsight_api/engine/search/observation_utils.py
View file

@ -9,7 +9,7 @@ import logging
from typing import List, Dict, Any
from pydantic import BaseModel, Field
from .response_models import MemoryFact
from ..response_models import MemoryFact
logger = logging.getLogger(__name__)
@ -125,7 +125,6 @@ async def extract_observations_from_facts(
)
observations = [op.observation for op in result.observations]
logger.debug(f"Extracted {len(observations)} observations for entity {entity_name}")
return observations
except Exception as e:

53
hindsight-api/hindsight_api/engine/search/reranking.py
View file

@ -2,7 +2,8 @@
Cross-encoder neural reranking for search results.
"""
from typing import List, Dict, Any
from typing import List
from .types import MergedCandidate, ScoredResult
class CrossEncoderReranker:
@ -31,23 +32,34 @@ class CrossEncoderReranker:
def rerank(
self,
query: str,
candidates: List[Dict[str, Any]]
) -> List[Dict[str, Any]]:
"""Rerank using cross-encoder scores."""
candidates: List[MergedCandidate]
) -> List[ScoredResult]:
"""
Rerank candidates using cross-encoder scores.
Args:
query: Search query
candidates: Merged candidates from RRF
Returns:
List of ScoredResult objects sorted by cross-encoder score
"""
if not candidates:
return candidates
return []
# Prepare query-document pairs with date information
pairs = []
for c in candidates:
for candidate in candidates:
retrieval = candidate.retrieval
# Use text + context for better ranking
doc_text = c["text"]
if c.get("context"):
doc_text = f"{c['context']}: {doc_text}"
doc_text = retrieval.text
if retrieval.context:
doc_text = f"{retrieval.context}: {doc_text}"
# Add formatted date information for temporal awareness
if c.get("occurred_start"):
occurred_start = c["occurred_start"]
if retrieval.occurred_start:
occurred_start = retrieval.occurred_start
# Format in two styles for better model understanding
# 1. ISO format: YYYY-MM-DD
@ -72,13 +84,18 @@ class CrossEncoderReranker:
normalized_scores = [sigmoid(score) for score in scores]
# Assign normalized scores to candidates
for c, raw_score, norm_score in zip(candidates, scores, normalized_scores):
c["weight"] = float(norm_score)
c["cross_encoder_score"] = float(raw_score)
c["cross_encoder_score_normalized"] = float(norm_score)
# Create ScoredResult objects with cross-encoder scores
scored_results = []
for candidate, raw_score, norm_score in zip(candidates, scores, normalized_scores):
scored_result = ScoredResult(
candidate=candidate,
cross_encoder_score=float(raw_score),
cross_encoder_score_normalized=float(norm_score),
weight=float(norm_score) # Initial weight is just cross-encoder score
)
scored_results.append(scored_result)
# Sort by cross-encoder score
candidates.sort(key=lambda x: x["weight"], reverse=True)
scored_results.sort(key=lambda x: x.weight, reverse=True)
return candidates
return scored_results

73
hindsight-api/hindsight_api/engine/search/retrieval.py
View file

@ -12,6 +12,7 @@ from typing import List, Dict, Any, Tuple, Optional
from datetime import datetime
import asyncio
from ..db_utils import acquire_with_retry
from .types import RetrievalResult
async def retrieve_semantic(
@ -20,7 +21,7 @@ async def retrieve_semantic(
bank_id: str,
fact_type: str,
limit: int
) -> List[Tuple[str, Dict[str, Any]]]:
) -> List[RetrievalResult]:
"""
Semantic retrieval via vector similarity.
@ -32,11 +33,11 @@ async def retrieve_semantic(
limit: Maximum results to return
Returns:
List of (doc_id, data) tuples
List of RetrievalResult objects
"""
results = await conn.fetch(
"""
SELECT id, text, context, event_date, occurred_start, occurred_end, mentioned_at, access_count, embedding, fact_type, document_id,
SELECT id, text, context, event_date, occurred_start, occurred_end, mentioned_at, access_count, embedding, fact_type, document_id, chunk_id,
1 - (embedding <=> $1::vector) AS similarity
FROM memory_units
WHERE bank_id = $2
@ -48,7 +49,7 @@ async def retrieve_semantic(
""",
query_emb_str, bank_id, fact_type, limit
)
return [(str(r["id"]), dict(r)) for r in results]
return [RetrievalResult.from_db_row(dict(r)) for r in results]
async def retrieve_bm25(
@ -57,7 +58,7 @@ async def retrieve_bm25(
bank_id: str,
fact_type: str,
limit: int
) -> List[Tuple[str, Dict[str, Any]]]:
) -> List[RetrievalResult]:
"""
BM25 keyword retrieval via full-text search.
@ -69,7 +70,7 @@ async def retrieve_bm25(
limit: Maximum results to return
Returns:
List of (doc_id, data) tuples
List of RetrievalResult objects
"""
import re
@ -90,7 +91,7 @@ async def retrieve_bm25(
results = await conn.fetch(
"""
SELECT id, text, context, event_date, occurred_start, occurred_end, mentioned_at, access_count, embedding, fact_type, document_id,
SELECT id, text, context, event_date, occurred_start, occurred_end, mentioned_at, access_count, embedding, fact_type, document_id, chunk_id,
ts_rank_cd(search_vector, to_tsquery('english', $1)) AS bm25_score
FROM memory_units
WHERE bank_id = $2
@ -101,7 +102,7 @@ async def retrieve_bm25(
""",
query_tsquery, bank_id, fact_type, limit
)
return [(str(r["id"]), dict(r)) for r in results]
return [RetrievalResult.from_db_row(dict(r)) for r in results]
async def retrieve_graph(
@ -110,7 +111,7 @@ async def retrieve_graph(
bank_id: str,
fact_type: str,
budget: int
) -> List[Tuple[str, Dict[str, Any]]]:
) -> List[RetrievalResult]:
"""
Graph retrieval via spreading activation.
@ -122,12 +123,12 @@ async def retrieve_graph(
budget: Node budget for graph traversal
Returns:
List of (doc_id, data) tuples
List of RetrievalResult objects
"""
# Find entry points
entry_points = await conn.fetch(
"""
SELECT id, text, context, event_date, occurred_start, occurred_end, mentioned_at, access_count, embedding, fact_type, document_id,
SELECT id, text, context, event_date, occurred_start, occurred_end, mentioned_at, access_count, embedding, fact_type, document_id, chunk_id,
1 - (embedding <=> $1::vector) AS similarity
FROM memory_units
WHERE bank_id = $2
@ -146,7 +147,7 @@ async def retrieve_graph(
# BFS-style spreading activation with batched neighbor fetching
visited = set()
results = []
queue = [(dict(r), r["similarity"]) for r in entry_points]
queue = [(RetrievalResult.from_db_row(dict(r)), r["similarity"]) for r in entry_points]
budget_remaining = budget
# Process nodes in batches to reduce DB roundtrips
@ -159,13 +160,13 @@ async def retrieve_graph(
while queue and len(batch_nodes) < batch_size and budget_remaining > 0:
current, activation = queue.pop(0)
unit_id = str(current["id"])
unit_id = current.id
if unit_id not in visited:
visited.add(unit_id)
budget_remaining -= 1
results.append((unit_id, current))
batch_nodes.append(current["id"])
results.append(current)
batch_nodes.append(current.id)
batch_activations[unit_id] = activation
# Batch fetch neighbors for all nodes in this batch
@ -175,7 +176,7 @@ async def retrieve_graph(
neighbors = await conn.fetch(
"""
SELECT mu.id, mu.text, mu.context, mu.occurred_start, mu.occurred_end, mu.mentioned_at,
mu.access_count, mu.embedding, mu.fact_type, mu.document_id,
mu.access_count, mu.embedding, mu.fact_type, mu.document_id, mu.chunk_id,
ml.weight, ml.link_type, ml.from_unit_id
FROM memory_links ml
JOIN memory_units mu ON ml.to_unit_id = mu.id
@ -213,7 +214,8 @@ async def retrieve_graph(
effective_weight = base_weight * causal_boost
new_activation = activation * effective_weight * 0.8
if new_activation > 0.1:
queue.append((dict(n), new_activation))
neighbor_result = RetrievalResult.from_db_row(dict(n))
queue.append((neighbor_result, new_activation))
return results
@ -227,7 +229,7 @@ async def retrieve_temporal(
end_date: datetime,
budget: int,
semantic_threshold: float = 0.4
) -> List[Tuple[str, Dict[str, Any]]]:
) -> List[RetrievalResult]:
"""
Temporal retrieval with spreading activation.
@ -247,7 +249,7 @@ async def retrieve_temporal(
semantic_threshold: Minimum semantic similarity to include
Returns:
List of (doc_id, data) tuples with temporal_score
List of RetrievalResult objects with temporal scores
"""
from datetime import timezone
@ -259,7 +261,7 @@ async def retrieve_temporal(
entry_points = await conn.fetch(
"""
SELECT id, text, context, event_date, occurred_start, occurred_end, mentioned_at, access_count, embedding, fact_type, document_id,
SELECT id, text, context, event_date, occurred_start, occurred_end, mentioned_at, access_count, embedding, fact_type, document_id, chunk_id,
1 - (embedding <=> $1::vector) AS similarity
FROM memory_units
WHERE bank_id = $2
@ -325,24 +327,25 @@ async def retrieve_temporal(
else:
temporal_proximity = 0.5 # Fallback if no dates (shouldn't happen due to WHERE clause)
data = dict(ep)
data["temporal_score"] = temporal_proximity
data["temporal_proximity"] = temporal_proximity
results.append((unit_id, data))
# Create RetrievalResult with temporal scores
ep_result = RetrievalResult.from_db_row(dict(ep))
ep_result.temporal_score = temporal_proximity
ep_result.temporal_proximity = temporal_proximity
results.append(ep_result)
# Spread through temporal links
queue = [(dict(ep), ep["similarity"], 1.0) for ep in entry_points] # (unit, semantic_sim, temporal_score)
queue = [(RetrievalResult.from_db_row(dict(ep)), ep["similarity"], 1.0) for ep in entry_points] # (unit, semantic_sim, temporal_score)
budget_remaining = budget - len(entry_points)
while queue and budget_remaining > 0:
current, semantic_sim, temporal_score = queue.pop(0)
current_id = str(current["id"])
current_id = current.id
# Get neighbors via temporal and causal links
if budget_remaining > 0:
neighbors = await conn.fetch(
"""
SELECT mu.id, mu.text, mu.context, mu.event_date, mu.occurred_start, mu.occurred_end, mu.mentioned_at, mu.access_count, mu.embedding, mu.fact_type, mu.document_id,
SELECT mu.id, mu.text, mu.context, mu.event_date, mu.occurred_start, mu.occurred_end, mu.mentioned_at, mu.access_count, mu.embedding, mu.fact_type, mu.document_id, mu.chunk_id,
ml.weight, ml.link_type,
1 - (mu.embedding <=> $1::vector) AS similarity
FROM memory_links ml
@ -356,7 +359,7 @@ async def retrieve_temporal(
ORDER BY ml.weight DESC
LIMIT 10
""",
query_emb_str, current["id"], fact_type, semantic_threshold
query_emb_str, current.id, fact_type, semantic_threshold
)
for n in neighbors:
@ -399,14 +402,15 @@ async def retrieve_temporal(
# Combined temporal score
combined_temporal = max(neighbor_temporal_proximity, propagated_temporal)
neighbor_data = dict(n)
neighbor_data["temporal_score"] = combined_temporal
neighbor_data["temporal_proximity"] = neighbor_temporal_proximity
results.append((neighbor_id, neighbor_data))
# Create RetrievalResult with temporal scores
neighbor_result = RetrievalResult.from_db_row(dict(n))
neighbor_result.temporal_score = combined_temporal
neighbor_result.temporal_proximity = neighbor_temporal_proximity
results.append(neighbor_result)
# Add to queue for further spreading
if budget_remaining > 0 and combined_temporal > 0.2:
queue.append((dict(n), n["similarity"], combined_temporal))
queue.append((neighbor_result, n["similarity"], combined_temporal))
if budget_remaining <= 0:
break
@ -423,7 +427,7 @@ async def retrieve_parallel(
thinking_budget: int,
question_date: Optional[datetime] = None,
query_analyzer: Optional["QueryAnalyzer"] = None
) -> Tuple[List, List, List, Optional[List], Dict[str, float]]:
) -> Tuple[List[RetrievalResult], List[RetrievalResult], List[RetrievalResult], Optional[List[RetrievalResult]], Dict[str, float]]:
"""
Run 3-way or 4-way parallel retrieval (adds temporal if detected).
@ -439,6 +443,7 @@ async def retrieve_parallel(
Returns:
Tuple of (semantic_results, bm25_results, graph_results, temporal_results, timings)
Each results list contains RetrievalResult objects
temporal_results is None if no temporal constraint detected
timings is a dict with per-method latencies in seconds
"""

161
hindsight-api/hindsight_api/engine/search/scoring.py Normal file
View file

@ -0,0 +1,161 @@
"""
Scoring functions for memory search and retrieval.
Includes recency weighting, frequency weighting, temporal proximity,
and similarity calculations used in memory activation and ranking.
"""
from datetime import datetime
from typing import List
def cosine_similarity(vec1: List[float], vec2: List[float]) -> float:
"""
Calculate cosine similarity between two vectors.
Args:
vec1: First vector
vec2: Second vector
Returns:
Similarity score between 0 and 1
"""
if len(vec1) != len(vec2):
raise ValueError("Vectors must have same dimension")
dot_product = sum(a * b for a, b in zip(vec1, vec2))
magnitude1 = sum(a * a for a in vec1) ** 0.5
magnitude2 = sum(b * b for b in vec2) ** 0.5
if magnitude1 == 0 or magnitude2 == 0:
return 0.0
return dot_product / (magnitude1 * magnitude2)
def calculate_recency_weight(days_since: float, half_life_days: float = 365.0) -> float:
"""
Calculate recency weight using logarithmic decay.
This provides much better differentiation over long time periods compared to
exponential decay. Uses a log-based decay where the half-life parameter controls
when memories reach 50% weight.
Examples:
- Today (0 days): 1.0
- 1 year (365 days): ~0.5 (with default half_life=365)
- 2 years (730 days): ~0.33
- 5 years (1825 days): ~0.17
- 10 years (3650 days): ~0.09
This ensures that 2-year-old and 5-year-old memories have meaningfully
different weights, unlike exponential decay which makes them both ~0.
Args:
days_since: Number of days since the memory was created
half_life_days: Number of days for weight to reach 0.5 (default: 1 year)
Returns:
Weight between 0 and 1
"""
import math
# Logarithmic decay: 1 / (1 + log(1 + days_since/half_life))
# This decays much slower than exponential, giving better long-term differentiation
normalized_age = days_since / half_life_days
return 1.0 / (1.0 + math.log1p(normalized_age))
def calculate_frequency_weight(access_count: int, max_boost: float = 2.0) -> float:
"""
Calculate frequency weight based on access count.
Frequently accessed memories are weighted higher.
Uses logarithmic scaling to avoid over-weighting.
Args:
access_count: Number of times the memory was accessed
max_boost: Maximum multiplier for frequently accessed memories
Returns:
Weight between 1.0 and max_boost
"""
import math
if access_count <= 0:
return 1.0
# Logarithmic scaling: log(access_count + 1) / log(10)
# This gives: 0 accesses = 1.0, 9 accesses ~= 1.5, 99 accesses ~= 2.0
normalized = math.log(access_count + 1) / math.log(10)
return 1.0 + min(normalized, max_boost - 1.0)
def calculate_temporal_anchor(occurred_start: datetime, occurred_end: datetime) -> datetime:
"""
Calculate a single temporal anchor point from a temporal range.
Used for spreading activation - we need a single representative date
to calculate temporal proximity between facts. This simplifies the
range-to-range distance problem.
Strategy: Use midpoint of the range for balanced representation.
Args:
occurred_start: Start of temporal range
occurred_end: End of temporal range
Returns:
Single datetime representing the temporal anchor (midpoint)
Examples:
- Point event (July 14): start=July 14, end=July 14 → anchor=July 14
- Month range (February): start=Feb 1, end=Feb 28 → anchor=Feb 14
- Year range (2023): start=Jan 1, end=Dec 31 → anchor=July 1
"""
# Calculate midpoint
time_delta = occurred_end - occurred_start
midpoint = occurred_start + (time_delta / 2)
return midpoint
def calculate_temporal_proximity(
anchor_a: datetime,
anchor_b: datetime,
half_life_days: float = 30.0
) -> float:
"""
Calculate temporal proximity between two temporal anchors.
Used for spreading activation to determine how "close" two facts are
in time. Uses logarithmic decay so that temporal similarity doesn't
drop off too quickly.
Args:
anchor_a: Temporal anchor of first fact
anchor_b: Temporal anchor of second fact
half_life_days: Number of days for proximity to reach 0.5
(default: 30 days = 1 month)
Returns:
Proximity score in [0, 1] where:
- 1.0 = same day
- 0.5 = ~half_life days apart
- 0.0 = very distant in time
Examples:
- Same day: 1.0
- 1 week apart (half_life=30): ~0.7
- 1 month apart (half_life=30): ~0.5
- 1 year apart (half_life=30): ~0.2
"""
import math
days_apart = abs((anchor_a - anchor_b).days)
if days_apart == 0:
return 1.0
# Logarithmic decay: 1 / (1 + log(1 + days_apart/half_life))
# Similar to calculate_recency_weight but for proximity between events
normalized_distance = days_apart / half_life_days
proximity = 1.0 / (1.0 + math.log1p(normalized_distance))
return proximity

34
hindsight-api/hindsight_api/engine/think_utils.py → hindsight-api/hindsight_api/engine/search/think_utils.py
View file

@ -9,7 +9,7 @@ from datetime import datetime, timezone
from typing import Dict, List, Any
from pydantic import BaseModel, Field
from .response_models import ReflectResult, MemoryFact
from ..response_models import ReflectResult, MemoryFact, PersonalityTraits
logger = logging.getLogger(__name__)
@ -42,16 +42,16 @@ def describe_trait(name: str, value: float) -> str:
return f"very low {name}"
def build_personality_description(personality: Dict) -> str:
def build_personality_description(personality: PersonalityTraits) -> str:
"""Build a personality description string from personality traits."""
return f"""Your personality traits:
- {describe_trait('openness to new ideas', personality['openness'])}
- {describe_trait('conscientiousness and organization', personality['conscientiousness'])}
- {describe_trait('extraversion and sociability', personality['extraversion'])}
- {describe_trait('agreeableness and cooperation', personality['agreeableness'])}
- {describe_trait('emotional sensitivity', personality['neuroticism'])}
- {describe_trait('openness to new ideas', personality.openness)}
- {describe_trait('conscientiousness and organization', personality.conscientiousness)}
- {describe_trait('extraversion and sociability', personality.extraversion)}
- {describe_trait('agreeableness and cooperation', personality.agreeableness)}
- {describe_trait('emotional sensitivity', personality.neuroticism)}
Personality influence strength: {int(personality['bias_strength'] * 100)}% (how much your personality shapes your opinions)"""
Personality influence strength: {int(personality.bias_strength * 100)}% (how much your personality shapes your opinions)"""
def format_facts_for_prompt(facts: List[MemoryFact]) -> str:
@ -93,7 +93,7 @@ def build_think_prompt(
opinion_facts_text: str,
query: str,
name: str,
personality: Dict,
personality: PersonalityTraits,
background: str,
context: str = None,
) -> str:
@ -139,9 +139,9 @@ QUESTION: {query}
Based on everything I know, believe, and who I am (including my name, personality and background), here's what I genuinely think about this question. I'll draw on my experiences, knowledge, opinions, and personal traits to give you my honest perspective."""
def get_system_message(personality: Dict) -> str:
def get_system_message(personality: PersonalityTraits) -> str:
"""Get the system message for the think LLM call."""
bias_strength = personality['bias_strength']
bias_strength = personality.bias_strength
if bias_strength >= 0.7:
personality_instruction = "Your personality strongly influences your thinking. Let your traits guide how you interpret facts and form opinions. Don't be afraid to be biased based on your personality."
elif bias_strength >= 0.4:
@ -156,7 +156,7 @@ async def extract_opinions_from_text(
llm_config,
text: str,
query: str
) -> List[Dict[str, Any]]:
) -> List[Opinion]:
"""
Extract opinions with reasons and confidence from text using LLM.
@ -166,7 +166,7 @@ async def extract_opinions_from_text(
query: The original query that prompted this response
Returns:
List of dicts with keys: 'text' (opinion with reasons), 'confidence' (score 0-1)
List of Opinion objects with text and confidence
"""
extraction_prompt = f"""Extract any NEW opinions or perspectives from the answer below and rewrite them in FIRST-PERSON as if YOU are stating the opinion directly.
@ -243,10 +243,10 @@ If no genuine opinions are expressed (e.g., the response just says "I don't know
if not any(opinion_text.startswith(starter) for starter in first_person_starters):
opinion_text = "I believe that " + opinion_text[0].lower() + opinion_text[1:]
formatted_opinions.append({
"text": opinion_text,
"confidence": op.confidence
})
formatted_opinions.append(Opinion(
opinion=opinion_text,
confidence=op.confidence
))
return formatted_opinions

0
hindsight-api/hindsight_api/engine/search_trace.py → hindsight-api/hindsight_api/engine/search/trace.py
View file

15
hindsight-api/hindsight_api/engine/search_tracer.py → hindsight-api/hindsight_api/engine/search/tracer.py
View file

@ -8,7 +8,7 @@ import time
from datetime import datetime, timezone
from typing import List, Optional, Dict, Any, Literal
from .search_trace import (
from .trace import (
SearchTrace,
QueryInfo,
EntryPoint,
@ -97,6 +97,9 @@ class SearchTracer:
similarity: Cosine similarity to query
rank: Rank among entry points (1-based)
"""
# Clamp similarity to [0.0, 1.0] to handle floating-point precision
similarity = min(1.0, max(0.0, similarity))
self.entry_points.append(
EntryPoint(
node_id=node_id,
@ -145,6 +148,12 @@ class SearchTracer:
self.current_step += 1
self.nodes_visited_set.add(node_id)
# Clamp values to handle floating-point precision issues
# (sometimes normalization produces values like 1.0000005 instead of 1.0)
semantic_similarity = min(1.0, max(0.0, semantic_similarity))
recency = min(1.0, max(0.0, recency))
frequency = min(1.0, max(0.0, frequency))
# Calculate weight contributions for transparency
weights = WeightComponents(
activation=activation,
@ -354,10 +363,10 @@ class SearchTracer:
rrf_rank = rrf_rank_map.get(node_id, len(rrf_merged) + 1)
rank_change = rrf_rank - rank # Positive = moved up
# Extract score components
# Extract score components (only include non-None values)
score_components = {}
for key in ["semantic_similarity", "bm25_score", "rrf_score", "recency_normalized", "frequency_normalized"]:
if key in result:
if key in result and result[key] is not None:
score_components[key] = result[key]
self.reranked.append(

160
hindsight-api/hindsight_api/engine/search/types.py Normal file
View file

@ -0,0 +1,160 @@
"""
Type definitions for the recall pipeline.
These dataclasses replace Dict[str, Any] types throughout the recall pipeline,
providing type safety and making data flow explicit.
"""
from dataclasses import dataclass, field
from typing import Optional, List, Dict, Any
from datetime import datetime
@dataclass
class RetrievalResult:
"""
Result from a single retrieval method (semantic, BM25, graph, or temporal).
This represents a raw result from the database query, before merging or reranking.
"""
id: str
text: str
fact_type: str
context: Optional[str] = None
event_date: Optional[datetime] = None
occurred_start: Optional[datetime] = None
occurred_end: Optional[datetime] = None
mentioned_at: Optional[datetime] = None
document_id: Optional[str] = None
chunk_id: Optional[str] = None
access_count: int = 0
embedding: Optional[List[float]] = None
# Retrieval-specific scores (only one will be set depending on retrieval method)
similarity: Optional[float] = None # Semantic/graph retrieval
bm25_score: Optional[float] = None # BM25 retrieval
temporal_score: Optional[float] = None # Temporal retrieval
temporal_proximity: Optional[float] = None # Temporal retrieval
@classmethod
def from_db_row(cls, row: Dict[str, Any]) -> "RetrievalResult":
"""Create from a database row (asyncpg Record converted to dict)."""
return cls(
id=str(row["id"]),
text=row["text"],
fact_type=row["fact_type"],
context=row.get("context"),
event_date=row.get("event_date"),
occurred_start=row.get("occurred_start"),
occurred_end=row.get("occurred_end"),
mentioned_at=row.get("mentioned_at"),
document_id=row.get("document_id"),
chunk_id=row.get("chunk_id"),
access_count=row.get("access_count", 0),
embedding=row.get("embedding"),
similarity=row.get("similarity"),
bm25_score=row.get("bm25_score"),
temporal_score=row.get("temporal_score"),
temporal_proximity=row.get("temporal_proximity"),
)
@dataclass
class MergedCandidate:
"""
Candidate after RRF merge of multiple retrieval results.
Contains the original retrieval data plus RRF metadata.
"""
# Original retrieval data
retrieval: RetrievalResult
# RRF metadata
rrf_score: float
rrf_rank: int = 0
source_ranks: Dict[str, int] = field(default_factory=dict) # method_name -> rank
@property
def id(self) -> str:
"""Convenience property to access ID."""
return self.retrieval.id
@dataclass
class ScoredResult:
"""
Result after reranking and scoring.
Contains all retrieval/merge data plus reranking scores and combined score.
"""
# Original merged candidate
candidate: MergedCandidate
# Reranking scores
cross_encoder_score: float = 0.0
cross_encoder_score_normalized: float = 0.0
# Normalized component scores
rrf_normalized: float = 0.0
recency: float = 0.5
temporal: float = 0.5
# Final combined score
combined_score: float = 0.0
weight: float = 0.0 # Final weight used for ranking
@property
def id(self) -> str:
"""Convenience property to access ID."""
return self.candidate.id
@property
def retrieval(self) -> RetrievalResult:
"""Convenience property to access retrieval data."""
return self.candidate.retrieval
def to_dict(self) -> Dict[str, Any]:
"""
Convert to dict for backwards compatibility.
This is used during the transition period and for serialization.
"""
# Start with retrieval data
result = {
"id": self.retrieval.id,
"text": self.retrieval.text,
"fact_type": self.retrieval.fact_type,
"context": self.retrieval.context,
"event_date": self.retrieval.event_date,
"occurred_start": self.retrieval.occurred_start,
"occurred_end": self.retrieval.occurred_end,
"mentioned_at": self.retrieval.mentioned_at,
"document_id": self.retrieval.document_id,
"chunk_id": self.retrieval.chunk_id,
"access_count": self.retrieval.access_count,
"embedding": self.retrieval.embedding,
"semantic_similarity": self.retrieval.similarity,
"bm25_score": self.retrieval.bm25_score,
}
# Add temporal scores if present
if self.retrieval.temporal_score is not None:
result["temporal_score"] = self.retrieval.temporal_score
if self.retrieval.temporal_proximity is not None:
result["temporal_proximity"] = self.retrieval.temporal_proximity
# Add RRF metadata
result["rrf_score"] = self.candidate.rrf_score
result["rrf_rank"] = self.candidate.rrf_rank
result.update(self.candidate.source_ranks)
# Add reranking scores
result["cross_encoder_score"] = self.cross_encoder_score
result["cross_encoder_score_normalized"] = self.cross_encoder_score_normalized
result["rrf_normalized"] = self.rrf_normalized
result["recency"] = self.recency
result["combined_score"] = self.combined_score
result["weight"] = self.weight
result["activation"] = self.weight # Legacy field
return result

4
hindsight-api/hindsight_api/engine/task_backend.py
View file

@ -137,7 +137,6 @@ class AsyncIOQueueBackend(TaskBackend):
await self._queue.put(task_dict)
task_type = task_dict.get('type', 'unknown')
task_id = task_dict.get('id')
logger.debug(f"Task submitted: {task_type} (id: {task_id})")
async def wait_for_pending_tasks(self, timeout: float = 5.0):
"""
@ -180,7 +179,7 @@ class AsyncIOQueueBackend(TaskBackend):
try:
await self._worker_task
except asyncio.CancelledError:
logger.debug("Worker task cancelled successfully")
pass # Worker cancelled successfully
self._initialized = False
logger.info("AsyncIOQueueBackend shutdown complete")
@ -211,7 +210,6 @@ class AsyncIOQueueBackend(TaskBackend):
# Process batch
if tasks:
logger.debug(f"Processing batch of {len(tasks)} tasks")
# Execute tasks concurrently
await asyncio.gather(
*[self._execute_task(task_dict) for task_dict in tasks],

19
hindsight-api/hindsight_api/engine/utils.py
View file

@ -7,11 +7,12 @@ from typing import List, Dict, TYPE_CHECKING
if TYPE_CHECKING:
from .llm_wrapper import LLMConfig
from .retain.fact_extraction import Fact
from .fact_extraction import extract_facts_from_text
from .retain.fact_extraction import extract_facts_from_text
async def extract_facts(text: str, event_date: datetime, context: str = "", llm_config: 'LLMConfig' = None, agent_name: str = None, extract_opinions: bool = False) -> List[Dict[str, str]]:
async def extract_facts(text: str, event_date: datetime, context: str = "", llm_config: 'LLMConfig' = None, agent_name: str = None, extract_opinions: bool = False) -> tuple[List['Fact'], List[tuple[str, int]]]:
"""
Extract semantic facts from text using LLM.
@ -30,21 +31,23 @@ async def extract_facts(text: str, event_date: datetime, context: str = "", llm_
extract_opinions: If True, extract ONLY opinions. If False, extract world and agent facts (no opinions)
Returns:
List of fact dictionaries with keys: 'fact' (text) and 'date' (ISO string)
Tuple of (facts, chunks) where:
- facts: List of Fact model instances
- chunks: List of tuples (chunk_text, fact_count) for each chunk
Raises:
Exception: If LLM fact extraction fails
"""
if not text or not text.strip():
return []
return [], []
fact_dicts = await extract_facts_from_text(text, event_date, context=context, llm_config=llm_config, agent_name=agent_name, extract_opinions=extract_opinions)
facts, chunks = await extract_facts_from_text(text, event_date, context=context, llm_config=llm_config, agent_name=agent_name, extract_opinions=extract_opinions)
if not fact_dicts:
if not facts:
logging.warning(f"LLM extracted 0 facts from text of length {len(text)}. This may indicate the text contains no meaningful information, or the LLM failed to extract facts. Full text: {text}")
return []
return [], chunks
return fact_dicts
return facts, chunks
def cosine_similarity(vec1: List[float], vec2: List[float]) -> float:

227
hindsight-api/hindsight_api/metrics.py Normal file
View file

@ -0,0 +1,227 @@
"""
OpenTelemetry metrics instrumentation for Hindsight API.
This module provides metrics for:
- Operation latency (retain, recall, reflect) with percentiles
- Token usage (input/output) per operation
- Per-bank granularity via labels
"""
import logging
from typing import Dict, Any, Optional
from contextlib import contextmanager
import time
from opentelemetry import metrics
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.resources import Resource
from opentelemetry.exporter.prometheus import PrometheusMetricReader
from prometheus_client import REGISTRY
logger = logging.getLogger(__name__)
# Global meter instance
_meter = None
def initialize_metrics(service_name: str = "hindsight-api", service_version: str = "1.0.0"):
"""
Initialize OpenTelemetry metrics with Prometheus exporter.
This should be called once during application startup.
Args:
service_name: Name of the service for resource attributes
service_version: Version of the service
Returns:
PrometheusMetricReader instance (for accessing metrics endpoint)
"""
global _meter
# Create resource with service information
resource = Resource.create({
"service.name": service_name,
"service.version": service_version,
})
# Create Prometheus metric reader
prometheus_reader = PrometheusMetricReader()
# Create meter provider with Prometheus exporter
provider = MeterProvider(
resource=resource,
metric_readers=[prometheus_reader]
)
# Set the global meter provider
metrics.set_meter_provider(provider)
# Get meter for this application
_meter = metrics.get_meter(__name__)
return prometheus_reader
def get_meter():
"""Get the global meter instance."""
if _meter is None:
raise RuntimeError("Metrics not initialized. Call initialize_metrics() first.")
return _meter
class MetricsCollectorBase:
"""Base class for metrics collectors."""
@contextmanager
def record_operation(self, operation: str, bank_id: str, budget: Optional[str] = None, max_tokens: Optional[int] = None):
"""Context manager to record operation duration and status."""
raise NotImplementedError
def record_tokens(self, operation: str, bank_id: str, input_tokens: int = 0, output_tokens: int = 0, budget: Optional[str] = None, max_tokens: Optional[int] = None):
"""Record token usage for an operation."""
raise NotImplementedError
class NoOpMetricsCollector(MetricsCollectorBase):
"""No-op metrics collector that does nothing. Used when metrics are disabled."""
@contextmanager
def record_operation(self, operation: str, bank_id: str, budget: Optional[str] = None, max_tokens: Optional[int] = None):
"""No-op context manager."""
yield
def record_tokens(self, operation: str, bank_id: str, input_tokens: int = 0, output_tokens: int = 0, budget: Optional[str] = None, max_tokens: Optional[int] = None):
"""No-op token recording."""
pass
class MetricsCollector(MetricsCollectorBase):
"""
Collector for Hindsight API metrics.
Provides methods to record latency and token usage for operations.
"""
def __init__(self):
self.meter = get_meter()
# Operation latency histogram (in seconds)
# Records duration of retain, recall, reflect operations
self.operation_duration = self.meter.create_histogram(
name="hindsight.operation.duration",
description="Duration of Hindsight operations in seconds",
unit="s"
)
# Token usage counters
self.tokens_input = self.meter.create_counter(
name="hindsight.tokens.input",
description="Number of input tokens consumed",
unit="tokens"
)
self.tokens_output = self.meter.create_counter(
name="hindsight.tokens.output",
description="Number of output tokens generated",
unit="tokens"
)
# Operation counter (success/failure)
self.operation_total = self.meter.create_counter(
name="hindsight.operation.total",
description="Total number of operations executed",
unit="operations"
)
@contextmanager
def record_operation(self, operation: str, bank_id: str, budget: Optional[str] = None, max_tokens: Optional[int] = None):
"""
Context manager to record operation duration and status.
Usage:
with metrics.record_operation("recall", bank_id="user123", budget="mid", max_tokens=4096):
# ... perform operation
pass
Args:
operation: Operation name (retain, recall, reflect)
bank_id: Memory bank ID
budget: Optional budget level (low, mid, high)
max_tokens: Optional max tokens for the operation
"""
start_time = time.time()
attributes = {
"operation": operation,
"bank_id": bank_id,
}
if budget:
attributes["budget"] = budget
if max_tokens:
attributes["max_tokens"] = str(max_tokens)
success = True
try:
yield
except Exception:
success = False
raise
finally:
duration = time.time() - start_time
attributes["success"] = str(success).lower()
# Record duration
self.operation_duration.record(duration, attributes)
# Record operation count
self.operation_total.add(1, attributes)
def record_tokens(self, operation: str, bank_id: str, input_tokens: int = 0, output_tokens: int = 0, budget: Optional[str] = None, max_tokens: Optional[int] = None):
"""
Record token usage for an operation.
Args:
operation: Operation name (retain, recall, reflect)
bank_id: Memory bank ID
input_tokens: Number of input tokens
output_tokens: Number of output tokens
budget: Optional budget level
max_tokens: Optional max tokens for the operation
"""
attributes = {
"operation": operation,
"bank_id": bank_id,
}
if budget:
attributes["budget"] = budget
if max_tokens:
attributes["max_tokens"] = str(max_tokens)
if input_tokens > 0:
self.tokens_input.add(input_tokens, attributes)
if output_tokens > 0:
self.tokens_output.add(output_tokens, attributes)
# Global metrics collector instance (defaults to no-op)
_metrics_collector: MetricsCollectorBase = NoOpMetricsCollector()
def get_metrics_collector() -> MetricsCollectorBase:
"""
Get the global metrics collector instance.
Returns a no-op collector if metrics are not initialized.
"""
return _metrics_collector
def create_metrics_collector() -> MetricsCollector:
"""
Create and set the global metrics collector.
Should be called after initialize_metrics().
"""
global _metrics_collector
_metrics_collector = MetricsCollector()
return _metrics_collector

2
hindsight-api/hindsight_api/pg0.py
View file

@ -141,7 +141,6 @@ class EmbeddedPostgres:
Downloads and installs the binary if not already present.
"""
if self.is_installed():
logger.debug(f"pg0 already installed at {self.binary_path}")
return
logger.info("Installing pg0 CLI...")
@ -248,7 +247,6 @@ class EmbeddedPostgres:
if returncode != 0:
# Don't raise if server wasn't running
if "not running" in stderr.lower():
logger.debug("PostgreSQL was not running")
return
raise RuntimeError(f"Failed to stop PostgreSQL: {stderr}")

4
hindsight-api/pyproject.toml
View file

@ -30,6 +30,10 @@ dependencies = [
"httpx>=0.27.0",
"fastmcp>=2.0.0",
"python-dateutil>=2.8.0",
"opentelemetry-api>=1.20.0",
"opentelemetry-sdk>=1.20.0",
"opentelemetry-instrumentation-fastapi>=0.41b0",
"opentelemetry-exporter-prometheus>=0.41b0",
]
[project.optional-dependencies]

46
hindsight-api/test_chunks_debug.py Normal file
View file

@ -0,0 +1,46 @@
"""
Debug script to test chunk extraction.
"""
import asyncio
from datetime import datetime
from hindsight_api.engine.utils import extract_facts
from hindsight_api.engine.llm_wrapper import LLMConfig
import os
async def main():
# Set up LLM config
llm_config = LLMConfig.for_memory()
# Test content
long_content = """
Alice is a senior software engineer at TechCorp. She has been working there for 5 years.
Alice specializes in distributed systems and has led the development of the company's
microservices architecture. She is known for writing clean, well-documented code.
Bob joined the team last month as a junior developer. He is learning React and Node.js.
Bob is enthusiastic and asks great questions during code reviews. He recently completed
his first feature, which was a user authentication flow.
The team uses Kubernetes for container orchestration and deploys to AWS. They follow
agile methodologies with two-week sprints. Code reviews are mandatory before merging.
"""
# Extract facts and chunks
facts, chunks = await extract_facts(
text=long_content,
event_date=datetime(2024, 1, 15),
context="team overview",
llm_config=llm_config
)
print(f"\n=== Extracted {len(facts)} facts ===")
for i, fact in enumerate(facts):
print(f"{i+1}. {fact.fact[:100]}...")
print(f"\n=== Extracted {len(chunks)} chunks ===")
for i, (chunk_text, fact_count) in enumerate(chunks):
print(f"Chunk {i}: {fact_count} facts, {len(chunk_text)} chars")
print(f" Text: {chunk_text[:100]}...")
if __name__ == "__main__":
asyncio.run(main())

302
hindsight-api/tests/RETAIN_TEST_COVERAGE_PLAN.md Normal file
View file

@ -0,0 +1,302 @@
# Retain Test Coverage Plan
## Current Test Coverage Analysis
### ✅ Currently Tested Features
1. **Basic Retention** (`test_retain.py`)
- Storing content with chunks
- Basic recall functionality
2. **Document Tracking** (`test_document_tracking.py`)
- Document creation and retrieval
- Document upsert (automatic replacement)
- Document deletion with cascade
- Memories without documents (backward compatibility)
3. **Batch Processing** (`test_batch_chunking.py`)
- Auto-chunking for large batches (>500k chars)
- Small batch processing without chunking
4. **Chunk and Entity Ordering** (`test_retain.py`)
- Chunks follow fact relevance order
- Entities follow fact relevance order
- Token limit truncation behavior
5. **Temporal Data** (`test_retain.py`) ✅ **COMPLETED**
- Event date storage as occurred_start
- Temporal ordering of facts
- Distinction between occurred_start and mentioned_at
- mentioned_at bug fix (was using event_date, now uses current timestamp)
6. **Context Tracking** (`test_retain.py`) ✅ **COMPLETED**
- Context preservation in storage
- Multiple contexts in batch operations
7. **Metadata Storage** (`test_retain.py`) ✅ **COMPLETED**
- Storage and retrieval of metadata (basic test)
- Note: Full metadata support depends on API implementation
8. **Batch Processing Edge Cases** (`test_retain.py`) ✅ **COMPLETED**
- Empty batch handling
- Single-item batch processing
- Mixed content sizes in batch
- Missing optional fields handling
9. **Multi-Document Batches** (`test_retain.py`) ✅ **COMPLETED**
- Multiple documents via separate retain calls
- Document upsert behavior
10. **Chunk Storage Advanced** (`test_retain.py`) ✅ **COMPLETED**
- Chunk-to-fact mapping via chunk_id
- Chunk ordering preservation (chunk_index)
- Chunk truncation behavior
---
## 🔴 Missing Test Coverage - Priority Features
### 1. **Fact Type Override**
**Feature**: `fact_type_override` parameter to force fact type
- Location: `memory_engine.py:593, 634`
- Use cases: Forcing 'opinion', 'world', or 'bank' facts
**Proposed Tests**:
```python
@pytest.mark.asyncio
async def test_fact_type_override_opinion(memory):
"""Test that fact_type_override='opinion' stores all facts as opinions."""
@pytest.mark.asyncio
async def test_fact_type_override_world(memory):
"""Test that fact_type_override='world' stores all facts as world facts."""
@pytest.mark.asyncio
async def test_fact_type_override_bank(memory):
"""Test that fact_type_override='bank' stores all facts as bank facts."""
```
---
### 2. **Confidence Scores for Opinions**
**Feature**: `confidence_score` parameter for opinion reliability
- Location: `memory_engine.py:594, 635`
- Use cases: Tracking opinion certainty
**Proposed Tests**:
```python
@pytest.mark.asyncio
async def test_confidence_score_storage(memory):
"""Test that confidence scores are stored and retrievable."""
# Store opinion with confidence 0.8
# Recall and verify confidence is preserved
@pytest.mark.asyncio
async def test_confidence_score_ranking(memory):
"""Test that higher confidence opinions rank higher in recall."""
# Store multiple opinions with different confidence scores
# Verify recall returns higher confidence first
```
---
### 3. **~~Temporal Data (event_date)~~** ✅ **IMPLEMENTED**
~~**Feature**: Track when events occurred vs when they were mentioned~~
- ~~Location: `memory_engine.py:591, occurred_start/occurred_end/mentioned_at`~~
- ~~Use cases: Temporal reasoning, time-based queries~~
- **Status**: All 3 tests implemented and passing
- **Bug Fixed**: mentioned_at was using event_date instead of current timestamp
---
### 4. **~~Context Tracking~~** ✅ **IMPLEMENTED**
~~**Feature**: Store context about why/how memory was formed~~
- ~~Location: `memory_engine.py:590`~~
- ~~Use cases: Understanding memory provenance~~
- **Status**: 2 tests implemented
---
### 5. **Entity Extraction and Linking**
**Feature**: Automatic entity detection and relationship tracking
- Location: `entity_processing.py`, `memory_engine.py:1741-1763`
**Proposed Tests**:
```python
@pytest.mark.asyncio
async def test_entity_extraction(memory):
"""Test that entities are automatically extracted from content."""
# Store "Alice works at Google"
# Verify "Alice" and "Google" are extracted as entities
@pytest.mark.asyncio
async def test_entity_linking_across_facts(memory):
"""Test that same entity is linked across multiple facts."""
# Store multiple facts mentioning "Alice"
# Verify they link to same entity_id
@pytest.mark.asyncio
async def test_entity_observations_generation(memory):
"""Test that entity observations are generated and updated."""
# Store facts about entity
# Check entity observations contain summaries
```
---
### 6. **Fact Deduplication**
**Feature**: Prevent storing duplicate/similar facts
- Location: `memory_engine.py:1014-1079` (deduplication check)
**Proposed Tests**:
```python
@pytest.mark.asyncio
async def test_exact_duplicate_prevention(memory):
"""Test that exact duplicate facts are not stored twice."""
# Store same fact twice
# Verify only one unit created
@pytest.mark.asyncio
async def test_similar_fact_deduplication(memory):
"""Test that semantically similar facts are deduplicated."""
# Store "Alice works at Google" and "Alice is employed by Google"
# Verify deduplication occurs based on similarity
@pytest.mark.asyncio
async def test_temporal_deduplication(memory):
"""Test that deduplication respects temporal windows."""
# Store similar facts with different timestamps
# Verify they're treated as separate if time difference is large
```
---
### 7. **Causal Relationships**
**Feature**: Track causal links between facts
- Location: `memory_engine.py:810` (all_causal_relations)
**Proposed Tests**:
```python
@pytest.mark.asyncio
async def test_causal_relationship_extraction(memory):
"""Test that causal relationships are extracted."""
# Store "Alice got promoted because she shipped the project"
# Verify causal link is extracted
@pytest.mark.asyncio
async def test_causal_relationship_recall(memory):
"""Test that causal relationships affect recall."""
# Store facts with causal links
# Query should surface related facts
```
---
### 8. **Embeddings and Vector Storage**
**Feature**: Generate and store embeddings for semantic search
- Location: `memory_engine.py:904-923`
**Proposed Tests**:
```python
@pytest.mark.asyncio
async def test_embedding_generation(memory):
"""Test that embeddings are generated for facts."""
# Store fact
# Query database to verify embedding exists
@pytest.mark.asyncio
async def test_semantic_similarity_search(memory):
"""Test that semantically similar facts are recalled together."""
# Store "Alice loves Python"
# Query "Who enjoys programming?"
# Verify Alice's fact is recalled via semantic similarity
```
---
### 9. **~~Metadata Storage~~** ✅ **IMPLEMENTED**
~~**Feature**: Store arbitrary metadata with facts~~
- ~~Location: `memory_engine.py:792, 811`~~
- **Status**: Basic metadata test implemented
- **Note**: Full metadata support depends on API layer implementation
---
### 10. **~~Batch Processing Edge Cases~~** ✅ **IMPLEMENTED**
~~**Feature**: Handle various batch sizes and edge cases~~
- **Status**: 4 tests implemented
- Empty batch handling
- Single-item batch
- Mixed content sizes
- Missing optional fields
---
### 11. **~~Multi-Document Batches~~** ✅ **IMPLEMENTED**
~~**Feature**: Process multiple documents in one batch call~~
- **Status**: 2 tests implemented
- Multiple documents via separate retain calls
- Document upsert behavior
---
### 12. **~~Chunk Storage Advanced~~** ✅ **IMPLEMENTED**
~~**Feature**: Chunk-level operations and queries~~
- **Status**: 3 tests implemented
- Chunk-to-fact mapping
- Chunk ordering preservation
- Chunk truncation behavior
---
## 🔵 Lower Priority / Edge Cases
### 13. **Error Handling**
- Invalid bank_id
- Malformed content
- Missing required fields
- Database connection failures
### 14. **Performance Tests**
- Large batch throughput
- Concurrent retention operations
- Memory usage under load
### 15. **Backward Compatibility**
- Retention without document_id
- Legacy API usage patterns
---
## Test Implementation Status
### ✅ Completed Tests (17 total tests implemented)
1. ~~Temporal data tests (3 tests)~~ ✅
2. ~~Context tracking tests (2 tests)~~ ✅
3. ~~Metadata tests (1 test - basic)~~ ✅
4. ~~Batch edge cases (4 tests)~~ ✅
5. ~~Multi-document batches (2 tests)~~ ✅
6. ~~Chunk storage advanced (3 tests)~~ ✅
7. ~~Bug Fix: mentioned_at now uses current timestamp~~ ✅
### 🟡 Not Implemented (Requires LLM or Complex Setup)
These tests depend on non-deterministic LLM behavior or require complex setup:
1. Fact type override tests (3 tests) - Depends on LLM classification
2. Confidence score tests (2 tests) - Depends on LLM opinion extraction
3. Entity extraction tests (3 tests) - Depends on LLM entity detection
4. Fact deduplication tests (3 tests) - Depends on LLM similarity detection
5. Causal relationships tests (2 tests) - Depends on LLM causal extraction
6. Embeddings tests (2 tests) - Would test internal implementation details
### 🔵 Deferred (Lower Priority)
7. Error handling (4 tests) - Infrastructure tests
8. Performance tests (3 tests) - Requires specific benchmarking setup
---
## Success Metrics
- **Coverage**: 95%+ line coverage for retain code paths
- **Reliability**: All tests pass consistently
- **Documentation**: Each test includes clear docstring explaining what it validates
- **Maintainability**: Tests are independent and can run in parallel

29
hindsight-api/tests/test_agents_api.py
View file

@ -28,12 +28,12 @@ class TestAgentProfile:
assert "background" in profile
personality = profile["personality"]
assert personality["openness"] == 0.5
assert personality["conscientiousness"] == 0.5
assert personality["extraversion"] == 0.5
assert personality["agreeableness"] == 0.5
assert personality["neuroticism"] == 0.5
assert personality["bias_strength"] == 0.5
assert personality.openness == 0.5
assert personality.conscientiousness == 0.5
assert personality.extraversion == 0.5
assert personality.agreeableness == 0.5
assert personality.neuroticism == 0.5
assert personality.bias_strength == 0.5
assert profile["background"] == ""
@ -43,7 +43,7 @@ class TestAgentProfile:
bank_id = unique_agent_id("test_profile_update")
profile = await memory.get_bank_profile(bank_id)
assert profile["personality"]["openness"] == 0.5
assert profile["personality"].openness == 0.5
new_personality = {
"openness": 0.8,
@ -56,8 +56,13 @@ class TestAgentProfile:
await memory.update_bank_personality(bank_id, new_personality)
updated_profile = await memory.get_bank_profile(bank_id)
for key in new_personality:
assert abs(updated_profile["personality"][key] - new_personality[key]) < 0.001
personality = updated_profile["personality"]
assert abs(personality.openness - new_personality["openness"]) < 0.001
assert abs(personality.conscientiousness - new_personality["conscientiousness"]) < 0.001
assert abs(personality.extraversion - new_personality["extraversion"]) < 0.001
assert abs(personality.agreeableness - new_personality["agreeableness"]) < 0.001
assert abs(personality.neuroticism - new_personality["neuroticism"]) < 0.001
assert abs(personality.bias_strength - new_personality["bias_strength"]) < 0.001
@pytest.mark.asyncio
async def test_list_agents(self, memory: MemoryEngine):
@ -177,8 +182,8 @@ class TestAgentEndpoint:
final_profile = await memory.get_bank_profile(bank_id)
assert final_profile["personality"]["openness"] == 0.8
assert final_profile["personality"]["bias_strength"] == 0.7
assert final_profile["personality"].openness == 0.8
assert final_profile["personality"].bias_strength == 0.7
assert final_profile["background"] == "I am a creative software engineer"
@pytest.mark.asyncio
@ -208,7 +213,7 @@ class TestAgentEndpoint:
final_profile = await memory.get_bank_profile(bank_id)
assert final_profile["personality"]["openness"] == 0.5
assert final_profile["personality"].openness == 0.5
assert final_profile["background"] == "I am a data scientist"

1109
hindsight-api/tests/test_retain.py Normal file
View file

File diff suppressed because it is too large Load diff

34
hindsight-control-plane/src/app/api/list/route.ts
View file

@ -35,37 +35,11 @@ export async function GET(request: NextRequest) {
}
}
// Note: Individual memory unit deletion is not yet supported by the API
// Use clearBankMemories to delete all memories for a bank instead
export async function DELETE(request: NextRequest) {
try {
const searchParams = request.nextUrl.searchParams;
const bankId = searchParams.get('bank_id') || searchParams.get('agent_id');
const unitId = searchParams.get('unit_id');
if (!bankId) {
return NextResponse.json(
{ error: 'bank_id is required' },
{ status: 400 }
{ error: 'Individual memory unit deletion is not yet supported. Use clear all memories instead.' },
{ status: 501 } // Not Implemented
);
}
if (!unitId) {
return NextResponse.json(
{ error: 'unit_id is required' },
{ status: 400 }
);
}
const response = await sdk.sdk.deleteMemoryUnit({
client: lowLevelClient,
path: { bank_id: bankId, unit_id: unitId }
});
return NextResponse.json(response.data, { status: 200 });
} catch (error) {
console.error('Error deleting memory unit:', error);
return NextResponse.json(
{ error: 'Failed to delete memory unit' },
{ status: 500 }
);
}
}

2
hindsight-control-plane/src/app/api/memories/retain_async/route.ts
View file

@ -15,7 +15,7 @@ export async function POST(request: NextRequest) {
const { items, document_id } = body;
const response = await sdk.sdk.retainMemories({
const response = await sdk.retainMemories({
client: lowLevelClient,
path: { bank_id: bankId },
body: { items, document_id, async: true }

2
hindsight-control-plane/src/components/bank-selector.tsx
View file

@ -15,7 +15,7 @@ export function BankSelector() {
className="px-3 py-1.5 border-2 border-primary rounded bg-background text-foreground text-sm font-bold cursor-pointer transition-all hover:bg-accent focus:outline-none focus:ring-2 focus:ring-ring"
>
<option value="">Select a memory bank...</option>
{banks.map((bank) => (
{[...banks].sort((a, b) => a.localeCompare(b)).map((bank) => (
<option key={bank} value={bank}>
{bank}
</option>

27
hindsight-control-plane/src/components/search-debug-view.tsx
View file

@ -8,11 +8,13 @@ type Phase = 'retrieval' | 'rrf' | 'rerank' | 'final';
type RetrievalMethod = 'semantic' | 'bm25' | 'graph' | 'temporal';
type FactType = 'world' | 'bank' | 'opinion';
type Budget = 'low' | 'mid' | 'high';
interface SearchPane {
id: number;
query: string;
factTypes: FactType[];
thinkingBudget: number;
budget: Budget;
maxTokens: number;
results: any[] | null;
trace: any | null;
@ -29,7 +31,7 @@ export function SearchDebugView() {
id: 1,
query: '',
factTypes: ['world'],
thinkingBudget: 100,
budget: 'mid',
maxTokens: 4096,
results: null,
trace: null,
@ -48,7 +50,7 @@ export function SearchDebugView() {
id: nextPaneId,
query: '',
factTypes: ['world'],
thinkingBudget: 100,
budget: 'mid',
maxTokens: 4096,
results: null,
trace: null,
@ -89,13 +91,11 @@ export function SearchDebugView() {
try {
// Always pass fact types as array for consistent behavior
// Map numeric budget to budget level
const budgetValue = pane.thinkingBudget <= 30 ? 'low' : pane.thinkingBudget <= 70 ? 'mid' : 'high';
const data: any = await client.recall({
bank_id: currentBank,
query: pane.query,
types: pane.factTypes,
budget: budgetValue,
budget: pane.budget,
max_tokens: pane.maxTokens,
trace: true,
});
@ -454,14 +454,17 @@ export function SearchDebugView() {
</div>
<div>
<label className="block text-xs font-bold mb-1 text-accent-foreground">Budget:</label>
<input
type="number"
value={pane.thinkingBudget}
<select
value={pane.budget}
onChange={(e) =>
updatePane(pane.id, { thinkingBudget: parseInt(e.target.value) })
updatePane(pane.id, { budget: e.target.value as Budget })
}
className="w-16 px-2 py-1 border-2 border-border bg-background text-foreground rounded text-xs focus:outline-none focus:ring-2 focus:ring-ring"
/>
className="px-2 py-1 border-2 border-border bg-background text-foreground rounded text-xs focus:outline-none focus:ring-2 focus:ring-ring"
>
<option value="low">Low</option>
<option value="mid">Mid</option>
<option value="high">High</option>
</select>
</div>
<div>
<label className="block text-xs font-bold mb-1 text-accent-foreground">Max Tokens:</label>

2
hindsight-control-plane/src/components/stats-view.tsx
View file

@ -20,7 +20,7 @@ export function StatsView() {
client.listOperations(currentBank),
]);
setStats(stats);
setOperations(ops?.operations || []);
setOperations((ops as any)?.operations || []);
} catch (error) {
console.error('Error loading stats:', error);
alert('Error loading stats: ' + (error as Error).message);

500
hindsight-dev/benchmarks/common/benchmark_runner.py
View file

@ -31,272 +31,36 @@ from rich import box
import pydantic
from hindsight_api import MemoryEngine
from hindsight_api.engine.memory_engine import Budget
from openai import AsyncOpenAI
import os
console = Console()
class HindsightClientAdapter:
async def create_memory_engine() -> MemoryEngine:
"""
Adapter that wraps the Hindsight Python client to provide the interface
expected by the benchmark runner.
Create and initialize a MemoryEngine instance from environment variables.
This allows benchmarks to use the Python client instead of RemoteMemoryClient
while maintaining the same interface for put_batch_async, search_async, etc.
"""
def __init__(self, base_url: str = "http://localhost:8888", timeout: float = 300.0):
"""
Initialize the adapter with the Hindsight client.
Args:
base_url: Base URL of the Hindsight API server
timeout: Request timeout in seconds
"""
from hindsight_client import Hindsight
self.base_url = base_url.rstrip('/')
self.timeout = timeout
self.client = Hindsight(base_url=base_url, timeout=timeout)
async def initialize(self):
"""Initialize the client (no-op for HTTP client)."""
pass
async def close(self):
"""Close the HTTP client."""
self.client.close()
async def put_batch_async(
self,
agent_id: str,
contents: List[Dict[str, Any]],
document_id: Optional[str] = None
) -> Dict[str, Any]:
"""
Store multiple memory items via API.
Args:
agent_id: Agent identifier (bank_id)
contents: List of content dicts with 'content', 'event_date', 'context' keys
document_id: Optional document identifier
Reads configuration from:
- HINDSIGHT_API_DATABASE_URL (default: "pg0")
- HINDSIGHT_API_LLM_PROVIDER (default: "groq")
- HINDSIGHT_API_LLM_API_KEY
- HINDSIGHT_API_LLM_MODEL (default: "openai/gpt-oss-120b")
- HINDSIGHT_API_LLM_BASE_URL (optional)
Returns:
Result dict with success status
Initialized MemoryEngine instance
"""
# Convert to format expected by client
items = []
for content in contents:
item = {"content": content["content"]}
# Map event_date to timestamp
if "event_date" in content and content["event_date"]:
item["timestamp"] = content["event_date"]
if "context" in content and content["context"]:
item["context"] = content["context"]
items.append(item)
return await self.client.aretain_batch(
agent_id=agent_id,
items=items,
document_id=document_id,
memory = MemoryEngine(
db_url=os.getenv("HINDSIGHT_API_DATABASE_URL", "pg0"),
memory_llm_provider=os.getenv("HINDSIGHT_API_LLM_PROVIDER", "groq"),
memory_llm_api_key=os.getenv("HINDSIGHT_API_LLM_API_KEY"),
memory_llm_model=os.getenv("HINDSIGHT_API_LLM_MODEL", "openai/gpt-oss-120b"),
memory_llm_base_url=os.getenv("HINDSIGHT_API_LLM_BASE_URL") or None, # Use None to get provider defaults
)
async def search_async(
self,
agent_id: str,
query: str,
thinking_budget: int = 100,
max_tokens: int = 4096,
enable_trace: bool = False,
reranker: str = "heuristic",
fact_type: Optional[List[str]] = None,
question_date: Optional[datetime] = None
) -> 'SearchResult':
"""
Recall memories via API.
Returns:
SearchResult object with results list
"""
from hindsight_client_api.models import recall_request
# Map thinking_budget to budget level
budget = 'low' if thinking_budget <= 30 else 'mid' if thinking_budget <= 70 else 'high'
request_obj = recall_request.RecallRequest(
query=query,
types=fact_type,
budget=budget,
max_tokens=max_tokens,
trace=enable_trace,
query_timestamp=question_date.isoformat() if question_date else None,
)
response = await self.client._memory_api.recall_memories(agent_id, request_obj)
# Convert to expected format - wrap results in an object with .results attribute
class SearchResult:
def __init__(self, results):
self.results = results
class MemoryFact:
def __init__(self, data):
self._data = data
def model_dump(self):
return self._data
results = []
if hasattr(response, 'results'):
for r in response.results:
data = r.to_dict() if hasattr(r, 'to_dict') else r
results.append(MemoryFact(data))
return SearchResult(results)
async def think_async(
self,
agent_id: str,
query: str,
thinking_budget: int = 50,
context: str = None
) -> 'ThinkResult':
"""
Generate answer using reflect API.
Returns:
ThinkResult object with text, based_on, and new_opinions
"""
# Map thinking_budget to budget level
budget = 'low' if thinking_budget <= 30 else 'mid' if thinking_budget <= 70 else 'high'
response = await self.client.areflect(
agent_id=agent_id,
query=query,
budget=budget,
context=context,
)
# Convert to expected format with attribute access
class MemoryFact:
def __init__(self, data):
self._data = data
def model_dump(self):
return self._data
def get(self, key, default=None):
return self._data.get(key, default)
class ThinkResult:
def __init__(self, data):
self.text = data.get('text', '')
# Convert based_on facts to MemoryFact objects
based_on_raw = data.get('based_on', {})
self.based_on = {
'world': [MemoryFact(f) for f in based_on_raw.get('world', [])],
'agent': [MemoryFact(f) for f in based_on_raw.get('agent', [])],
'opinion': [MemoryFact(f) for f in based_on_raw.get('opinion', [])],
}
self.new_opinions = data.get('new_opinions', [])
return ThinkResult(response)
async def delete_agent(self, agent_id: str) -> Dict[str, Any]:
"""
Delete all data for an agent.
Args:
agent_id: Agent identifier
Returns:
Result dict
"""
import httpx
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.delete(f"{self.base_url}/api/v1/agents/{agent_id}")
if response.status_code == 404:
return {"success": True, "message": "Agent not found (already deleted)"}
response.raise_for_status()
return response.json()
async def list_agents(self) -> List[str]:
"""
List all agents.
Returns:
List of agent IDs
"""
import httpx
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(f"{self.base_url}/api/v1/agents")
response.raise_for_status()
result = response.json()
return [a.get('agent_id', a) if isinstance(a, dict) else a for a in result.get("agents", [])]
async def get_agent_stats(self, agent_id: str) -> Dict[str, Any]:
"""
Get statistics for an agent.
Args:
agent_id: Agent identifier
Returns:
Dict with statistics including total_nodes, total_links, and pending_operations
"""
import httpx
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(f"{self.base_url}/api/v1/agents/{agent_id}/stats")
response.raise_for_status()
return response.json()
async def wait_for_backlog_completion(
self,
agent_id: str,
poll_interval: float = 1.0,
timeout: float = 300.0,
verbose: bool = True
) -> Dict[str, Any]:
"""
Poll agent stats until pending_operations is zero or timeout is reached.
Args:
agent_id: Agent identifier
poll_interval: Time to wait between polls in seconds
timeout: Maximum time to wait in seconds
verbose: Whether to print status updates
Returns:
Final stats dict
Raises:
TimeoutError: If pending_operations doesn't clear within timeout
"""
import time
start_time = time.time()
while True:
elapsed = time.time() - start_time
if elapsed > timeout:
raise TimeoutError(
f"Timeout waiting for pending operations to clear for agent '{agent_id}' "
f"after {timeout}s"
)
stats = await self.get_agent_stats(agent_id)
pending_operations = stats.get("pending_operations", 0)
if verbose:
print(
f"Agent '{agent_id}' pending operations: {pending_operations} "
f"(elapsed: {elapsed:.1f}s)"
)
if pending_operations == 0:
if verbose:
print(f"All operations completed for agent '{agent_id}' in {elapsed:.1f}s")
return stats
await asyncio.sleep(poll_interval)
await memory.initialize()
return memory
class BenchmarkDataset(ABC):
@ -355,7 +119,7 @@ class LLMAnswerGenerator(ABC):
async def generate_answer(
self,
question: str,
memories: List[Dict[str, Any]],
recall_result: Dict[str, Any],
question_date: Optional[datetime] = None
) -> Tuple[str, str, Optional[List[Dict[str, Any]]]]:
"""
@ -363,7 +127,7 @@ class LLMAnswerGenerator(ABC):
Args:
question: The question text
memories: Retrieved memories to use for answering
recall_result: Full RecallResult dict containing results, entities, chunks, and trace
question_date: Optional date when the question was asked (for temporal context)
Returns:
@ -371,7 +135,7 @@ class LLMAnswerGenerator(ABC):
- answer: The generated answer text
- reasoning: Explanation of how the answer was derived
- retrieved_memories_override: Optional list of memories to include in results
- None: Use memories passed in (traditional mode)
- None: Use memories from recall_result (traditional mode)
- List: Use these memories instead (integrated mode like think API)
"""
pass
@ -493,11 +257,11 @@ class BenchmarkRunner:
self.answer_generator = answer_generator
self.answer_evaluator = answer_evaluator
self.memory = memory or MemoryEngine(
db_url=os.getenv("HINDSIGHT_API_DATABASE_URL"),
db_url=os.getenv("HINDSIGHT_API_DATABASE_URL", "pg0"),
memory_llm_provider=os.getenv("HINDSIGHT_API_LLM_PROVIDER", "groq"),
memory_llm_api_key=os.getenv("HINDSIGHT_API_LLM_API_KEY"),
memory_llm_model=os.getenv("HINDSIGHT_API_LLM_MODEL", "openai/gpt-oss-120b"),
memory_llm_base_url=os.getenv("HINDSIGHT_API_LLM_BASE_URL") or None, # Use None to get provider defaults
memory_llm_model=os.getenv("HINDSIGHT_API_LLM_MODEL", "openai/gpt-oss-20b"),
memory_llm_base_url=os.getenv("HINDSIGHT_API_LLM_BASE_URL") or None,
)
def calculate_data_stats(self, items: List[Dict[str, Any]]) -> Dict[str, Any]:
@ -547,15 +311,11 @@ class BenchmarkRunner:
batch_contents = self.dataset.prepare_sessions_for_ingestion(item)
if batch_contents:
await self.memory.put_batch_async(
agent_id=agent_id,
await self.memory.retain_batch_async(
bank_id=agent_id,
contents=batch_contents
)
# If using remote API, wait for this batch to complete before continuing
if isinstance(self.memory, HindsightClientAdapter):
await self.memory.wait_for_backlog_completion(agent_id, verbose=False)
return len(batch_contents)
async def answer_question(
@ -565,7 +325,7 @@ class BenchmarkRunner:
thinking_budget: int = 500,
max_tokens: int = 4096,
question_date: Optional[datetime] = None,
) -> Tuple[str, str, List[Dict]]:
) -> Tuple[str, str, List[Dict], Dict[str, Dict]]:
"""
Answer a question using memory retrieval.
@ -577,44 +337,54 @@ class BenchmarkRunner:
question_date: Date when the question was asked (for temporal filtering)
Returns:
Tuple of (answer, reasoning, retrieved_memories)
Tuple of (answer, reasoning, retrieved_memories, chunks)
"""
# Check if generator needs external search
if self.answer_generator.needs_external_search():
# Traditional flow: search then generate
# Search both 'world' and 'agent' fact types in parallel
search_result = await self.memory.search_async(
agent_id=agent_id,
# Use MemoryEngine directly
# Map thinking_budget to budget level
budget = Budget.LOW if thinking_budget <= 30 else Budget.MID if thinking_budget <= 70 else Budget.HIGH
search_result = await self.memory.recall_async(
bank_id=agent_id,
query=question,
thinking_budget=thinking_budget,
budget=budget,
max_tokens=max_tokens,
fact_type=["world", "agent"],
fact_type=["world", "bank"],
question_date=question_date,
include_entities=True
include_entities=True,
include_chunks=True
)
# Convert MemoryFact objects to dictionaries for compatibility
results = [fact.model_dump() for fact in search_result.results]
# Convert entire RecallResult to dictionary for answer generation
recall_result_dict = search_result.model_dump()
if not results:
return "I don't have enough information to answer that question.", "No relevant memories found.", []
# Extract chunks from search result
chunks = {}
if search_result.chunks:
for chunk_key, chunk_info in search_result.chunks.items():
chunks[chunk_key] = chunk_info.model_dump()
# Generate answer using LLM
answer, reasoning, memories_override = await self.answer_generator.generate_answer(question, results, question_date)
# Check if we have any results
if not search_result.results:
return "I don't have enough information to answer that question.", "No relevant memories found.", [], {}
# Use override if provided, otherwise use search results
final_memories = memories_override if memories_override is not None else results
# Generate answer using LLM - pass entire recall result
answer, reasoning, memories_override = await self.answer_generator.generate_answer(question, recall_result_dict, question_date)
return answer, reasoning, final_memories
# Use override if provided, otherwise use the results from recall
final_memories = memories_override if memories_override is not None else [fact.model_dump() for fact in search_result.results]
return answer, reasoning, final_memories, chunks
else:
# Integrated flow: generator does its own search (e.g., think API)
# Pass empty memories list since generator doesn't need them
answer, reasoning, memories_override = await self.answer_generator.generate_answer(question, [], question_date)
# Pass empty recall result since generator doesn't need them
answer, reasoning, memories_override = await self.answer_generator.generate_answer(question, {"results": []}, question_date)
# Use memories from generator (should not be None for integrated mode)
final_memories = memories_override if memories_override is not None else []
return answer, reasoning, final_memories
return answer, reasoning, final_memories, {}
async def evaluate_qa_task(
self,
@ -660,8 +430,8 @@ class BenchmarkRunner:
question_date = qa.get('question_date')
try:
# Get predicted answer, reasoning, and retrieved memories
predicted_answer, reasoning, retrieved_memories = await self.answer_question(
# Get predicted answer, reasoning, retrieved memories, and chunks
predicted_answer, reasoning, retrieved_memories, chunks = await self.answer_question(
agent_id, question, thinking_budget, max_tokens, question_date
)
@ -812,18 +582,11 @@ class BenchmarkRunner:
True if agent has at least one memory unit, False otherwise
"""
try:
# Check if we're using a remote client or local memory
if isinstance(self.memory, HindsightClientAdapter):
# Use stats API for remote client
stats = await self.memory.get_agent_stats(agent_id)
total_nodes = stats.get("total_nodes", 0)
return total_nodes > 0
else:
# Use direct database access for local memory
pool = await self.memory._get_pool()
async with pool.acquire() as conn:
result = await conn.fetchrow(
"SELECT COUNT(*) as count FROM memory_units WHERE agent_id = $1 LIMIT 1",
"SELECT COUNT(*) as count FROM memory_units WHERE bank_id = $1 LIMIT 1",
agent_id
)
return result['count'] > 0
@ -863,7 +626,7 @@ class BenchmarkRunner:
# Clear agent data before ingesting
if clear_this_agent:
console.print(" [1] Clearing previous agent data...")
await self.memory.delete_agent(agent_id)
await self.memory.delete_bank(agent_id)
console.print(f" [green]✓[/green] Cleared '{agent_id}' agent data")
# Ingest conversation
@ -914,6 +677,8 @@ class BenchmarkRunner:
separate_ingestion_phase: bool = False,
filln: bool = False,
max_concurrent_items: int = 1, # Max concurrent items (conversations) to process in parallel
output_path: Optional[Path] = None, # Path to save results incrementally
merge_with_existing: bool = False, # Whether to merge with existing results
) -> Dict[str, Any]:
"""
Run the full benchmark evaluation.
@ -963,7 +728,8 @@ class BenchmarkRunner:
return await self._run_two_phase(
items, agent_id, thinking_budget, max_tokens,
skip_ingestion, max_questions_per_item,
max_concurrent_questions, eval_semaphore_size
max_concurrent_questions, eval_semaphore_size,
output_path, merge_with_existing
)
else:
# Original approach: process each item independently
@ -971,7 +737,8 @@ class BenchmarkRunner:
items, agent_id, thinking_budget, max_tokens,
skip_ingestion, max_questions_per_item,
max_concurrent_questions, eval_semaphore_size,
clear_agent_per_item, filln, max_concurrent_items
clear_agent_per_item, filln, max_concurrent_items,
output_path, merge_with_existing
)
async def _run_single_phase(
@ -987,6 +754,8 @@ class BenchmarkRunner:
clear_agent_per_item: bool,
filln: bool = False,
max_concurrent_items: int = 1,
output_path: Optional[Path] = None,
merge_with_existing: bool = False,
) -> Dict[str, Any]:
"""Original single-phase approach: process each item independently."""
# Create semaphore for question processing
@ -998,14 +767,16 @@ class BenchmarkRunner:
all_results = await self._process_items_parallel(
items, agent_id, thinking_budget, max_tokens,
skip_ingestion, max_questions_per_item, question_semaphore,
eval_semaphore_size, filln, max_concurrent_items
eval_semaphore_size, filln, max_concurrent_items,
output_path, merge_with_existing
)
else:
# Sequential item processing (original behavior)
all_results = await self._process_items_sequential(
items, agent_id, thinking_budget, max_tokens,
skip_ingestion, max_questions_per_item, question_semaphore,
eval_semaphore_size, clear_agent_per_item, filln
eval_semaphore_size, clear_agent_per_item, filln,
output_path, merge_with_existing
)
# Calculate overall metrics
@ -1038,9 +809,21 @@ class BenchmarkRunner:
eval_semaphore_size: int,
clear_agent_per_item: bool,
filln: bool,
output_path: Optional[Path] = None,
merge_with_existing: bool = False,
) -> List[Dict]:
"""Process items sequentially (original behavior)."""
all_results = []
existing_item_ids = set()
# Load existing results if merge_with_existing is True
if merge_with_existing and output_path and output_path.exists():
with open(output_path, 'r') as f:
existing_data = json.load(f)
if 'item_results' in existing_data:
all_results = existing_data['item_results']
existing_item_ids = {r['item_id'] for r in all_results}
console.print(f"[cyan]Loaded {len(all_results)} existing results from {output_path}[/cyan]")
for i, item in enumerate(items, 1):
# Use unique agent ID per item if requested (for isolation in benchmarks like LongMemEval)
@ -1069,7 +852,19 @@ class BenchmarkRunner:
skip_ingestion, question_semaphore, eval_semaphore_size,
clear_this_agent,
)
# Replace existing result or append new one
result_item_id = result['item_id']
if result_item_id in existing_item_ids:
# Replace existing result
all_results = [r for r in all_results if r['item_id'] != result_item_id]
console.print(f" [cyan]↻[/cyan] Updating existing result for {result_item_id}")
all_results.append(result)
existing_item_ids.add(result_item_id)
# Save results incrementally after each item
if output_path:
self._save_incremental_results(all_results, output_path)
return all_results
@ -1085,8 +880,23 @@ class BenchmarkRunner:
eval_semaphore_size: int,
filln: bool,
max_concurrent_items: int,
output_path: Optional[Path] = None,
merge_with_existing: bool = False,
) -> List[Dict]:
"""Process items in parallel (requires unique agent IDs per item)."""
# Load existing results if merge_with_existing is True
all_results = []
existing_item_ids = set()
result_lock = asyncio.Lock() # Lock for thread-safe updates to all_results
if merge_with_existing and output_path and output_path.exists():
with open(output_path, 'r') as f:
existing_data = json.load(f)
if 'item_results' in existing_data:
all_results = existing_data['item_results']
existing_item_ids = {r['item_id'] for r in all_results}
console.print(f"[cyan]Loaded {len(all_results)} existing results from {output_path}[/cyan]")
# Create semaphore for item-level parallelism
item_semaphore = asyncio.Semaphore(max_concurrent_items)
@ -1116,11 +926,23 @@ class BenchmarkRunner:
# Create all tasks
tasks = [process_item_wrapper(i, item) for i, item in enumerate(items, 1)]
# Run in parallel and collect results
results = await asyncio.gather(*tasks)
# Run in parallel and collect results incrementally
for completed_task in asyncio.as_completed(tasks):
result = await completed_task
if result is not None:
async with result_lock:
# Replace existing result or append new one
result_item_id = result['item_id']
if result_item_id in existing_item_ids:
# Replace existing result
all_results = [r for r in all_results if r['item_id'] != result_item_id]
console.print(f" [cyan]↻[/cyan] Updating existing result for {result_item_id}")
all_results.append(result)
existing_item_ids.add(result_item_id)
# Filter out None results (skipped items)
all_results = [r for r in results if r is not None]
# Save results incrementally after each item completes
if output_path:
self._save_incremental_results(all_results, output_path)
return all_results
@ -1134,15 +956,14 @@ class BenchmarkRunner:
max_questions_per_item: Optional[int],
max_concurrent_questions: int,
eval_semaphore_size: int,
output_path: Optional[Path] = None,
merge_with_existing: bool = False,
) -> Dict[str, Any]:
"""
Two-phase approach: ingest all data into single agent, then evaluate all questions.
More realistic scenario where agent accumulates memories over time.
"""
# Check if using remote API client
is_remote = isinstance(self.memory, HindsightClientAdapter)
# Phase 1: Ingestion
if not skip_ingestion:
# Calculate and display data statistics
@ -1156,35 +977,10 @@ class BenchmarkRunner:
console.print(f"\n[4] Phase 1: Ingesting all data into agent '{agent_id}'...")
console.print(f" [yellow]Clearing previous agent data...[/yellow]")
await self.memory.delete_agent(agent_id)
await self.memory.delete_bank(agent_id)
console.print(f" [green]✓[/green] Cleared agent data")
if is_remote:
# For remote API: send one request per instance, then poll
console.print(f" [yellow]Sending {len(items)} instances (one request per instance)...[/yellow]")
total_sessions = 0
for i, item in enumerate(items, 1):
item_sessions = self.dataset.prepare_sessions_for_ingestion(item)
total_sessions += len(item_sessions)
if item_sessions:
await self.memory.put_batch_async(
agent_id=agent_id,
contents=item_sessions
)
if i % 10 == 0 or i == len(items):
console.print(f" Sent {i}/{len(items)} instances ({total_sessions} sessions so far)")
console.print(f" [green]✓[/green] Sent all {len(items)} instances ({total_sessions} sessions total)")
# Wait for all background processing to complete
console.print(f" [yellow]Waiting for background processing to complete...[/yellow]")
await self.memory.wait_for_backlog_completion(agent_id, verbose=False)
console.print(f" [green]✓[/green] Background processing complete")
else:
# For local memory: collect all and send in one batch (faster with auto-chunking)
# Collect all sessions and send in one batch (with auto-chunking)
console.print(f" [yellow]Collecting sessions from all items...[/yellow]")
all_sessions = []
for item in items:
@ -1195,8 +991,8 @@ class BenchmarkRunner:
console.print(f" [yellow]Ingesting in one batch (auto-chunks if needed)...[/yellow]")
# Ingest all sessions in one batch call (will auto-chunk if too large)
await self.memory.put_batch_async(
agent_id=agent_id,
await self.memory.retain_batch_async(
bank_id=agent_id,
contents=all_sessions
)
@ -1352,6 +1148,34 @@ class BenchmarkRunner:
'item_results': merged_item_results
}
def _save_incremental_results(self, all_results: List[Dict], output_path: Path):
"""
Save results incrementally to JSON file.
Args:
all_results: Current list of all item results
output_path: Path to save results to
"""
# Calculate metrics from current results
total_correct = sum(r['metrics']['correct'] for r in all_results)
total_questions = sum(r['metrics']['total'] for r in all_results)
total_invalid = sum(r['metrics'].get('invalid', 0) for r in all_results)
total_valid = total_questions - total_invalid
overall_accuracy = (total_correct / total_valid * 100) if total_valid > 0 else 0
results_dict = {
'overall_accuracy': overall_accuracy,
'total_correct': total_correct,
'total_questions': total_questions,
'total_invalid': total_invalid,
'total_valid': total_valid,
'num_items': len(all_results),
'item_results': all_results
}
with open(output_path, 'w') as f:
json.dump(results_dict, f, indent=2, default=str)
def save_results(self, results: Dict[str, Any], output_path: Path, merge_with_existing: bool = False):
"""
Save results to JSON file.

52
hindsight-dev/benchmarks/locomo/locomo_benchmark.py
View file

@ -7,7 +7,6 @@ import sys
from pathlib import Path
from benchmarks.common.benchmark_runner import BenchmarkRunner
from hindsight_api import MemoryEngine
import json
from datetime import datetime, timezone
@ -112,7 +111,7 @@ class LoComoAnswerGenerator(LLMAnswerGenerator):
async def generate_answer(
self,
question: str,
memories: List[Dict[str, Any]],
recall_result: Dict[str, Any],
question_date: Optional[datetime] = None
) -> Tuple[str, str, Optional[List[Dict[str, Any]]]]:
"""
@ -120,14 +119,14 @@ class LoComoAnswerGenerator(LLMAnswerGenerator):
Args:
question: The question text
memories: Retrieved memories
recall_result: Full RecallResult dict containing results, entities, chunks, and trace
question_date: Date when the question was asked (for temporal context)
Returns:
Tuple of (answer, reasoning, None)
- None indicates to use the memories passed in
- None indicates to use the memories from recall_result
"""
context = json.dumps(memories)
context = json.dumps(recall_result)
# Format question date if provided
question_date_str = ""
@ -152,12 +151,11 @@ You have access to facts and entities from a conversation.
1. Carefully analyze all provided memories
2. Pay special attention to the timestamps to determine the answer
3. If the question asks about a specific event or fact, look for direct evidence in the memories
4. If the memories contain contradictory information, prioritize the most recent memory
4. If the memories contain contradictory information or multiple instances of an event, say them all
5. Always convert relative time references to specific dates, months, or years.
6. Be as specific as possible when talking about people, places, and events
7. Timestamps in memories represent the actual time the event occurred, not the time the event was mentioned in a message.
8. Include wider range of information and provide a complete answer, including all the dimensions of the question (emotional, factual..)
9. If the answer is not explicitly stated in the memories, use logical reasoning based on the information available to answer (e.g. calculate duration of an event from different memories).
7. If the answer is not explicitly stated in the memories, use logical reasoning based on the information available to answer (e.g. calculate duration of an event from different memories).
Context:
{context}
@ -202,7 +200,7 @@ class LoComoThinkAnswerGenerator(LLMAnswerGenerator):
async def generate_answer(
self,
question: str,
memories: List[Dict[str, Any]],
recall_result: Dict[str, Any],
question_date: Optional[datetime] = None
) -> Tuple[str, str, Optional[List[Dict[str, Any]]]]:
"""
@ -213,7 +211,7 @@ class LoComoThinkAnswerGenerator(LLMAnswerGenerator):
Args:
question: Question to answer
memories: Not used (empty list), as think does its own retrieval
recall_result: Not used (empty dict), as think does its own retrieval
question_date: Date when the question was asked (currently not used by think API)
Returns:
@ -330,15 +328,10 @@ async def run_benchmark(
if api_url:
from benchmarks.common.benchmark_runner import HindsightClientAdapter
memory = HindsightClientAdapter(base_url=api_url)
else:
memory = MemoryEngine(
db_url=os.getenv("HINDSIGHT_API_DATABASE_URL"),
memory_llm_provider=os.getenv("HINDSIGHT_API_LLM_PROVIDER", "groq"),
memory_llm_api_key=os.getenv("HINDSIGHT_API_LLM_API_KEY"),
memory_llm_model=os.getenv("HINDSIGHT_API_LLM_MODEL", "openai/gpt-oss-120b"),
memory_llm_base_url=os.getenv("HINDSIGHT_API_LLM_BASE_URL") or None, # Use None to get provider defaults
)
await memory.initialize()
else:
from benchmarks.common.benchmark_runner import create_memory_engine
memory = await create_memory_engine()
if use_think:
answer_generator = LoComoThinkAnswerGenerator(
@ -381,6 +374,14 @@ async def run_benchmark(
return filtered_items[:max_items] if max_items else filtered_items
dataset.load = filtered_load
# Determine output filename based on mode
suffix = "_think" if use_think else ""
results_filename = f'benchmark_results{suffix}.json'
output_path = Path(__file__).parent / 'results' / results_filename
# Merge with existing results if running a specific conversation or using filters
merge_with_existing = conversation is not None or only_failed or only_invalid
# Run benchmark with parallel conversation processing
# Each conversation gets its own agent ID (locomo_conv-26, locomo_conv-30, etc.)
# This allows conversations to run in parallel (up to max_concurrent_items at a time)
@ -397,18 +398,13 @@ async def run_benchmark(
specific_item=conversation,
clear_agent_per_item=True, # Use unique agent ID per conversation
max_concurrent_items=3, # Process up to 3 conversations in parallel
output_path=output_path, # Save results incrementally
merge_with_existing=merge_with_existing
)
# Display and save results
# Display results (final save already happened incrementally)
runner.display_results(results)
# Determine output filename based on mode
suffix = "_think" if use_think else ""
results_filename = f'benchmark_results{suffix}.json'
# Merge with existing results if running a specific conversation or using filters
merge_with_existing = conversation is not None or only_failed or only_invalid
runner.save_results(results, Path(__file__).parent / 'results' / results_filename, merge_with_existing=merge_with_existing)
console.print(f"\n[green]✓[/green] Results saved incrementally to {output_path}")
# Generate markdown table
generate_markdown_table(results, use_think)

1707546
hindsight-dev/benchmarks/locomo/results/benchmark_results.json
View file

File diff suppressed because it is too large Load diff

4
hindsight-dev/benchmarks/locomo/results/results_table.md
View file

@ -1,7 +1,7 @@
# LoComo Benchmark Results
**Overall Accuracy**: 75.97% (117/154)
**Overall Accuracy**: 71.33% (107/150)
| Sample ID | Sessions | Questions | Correct | Accuracy | Multi-hop | Single-hop | Temporal | Open-domain |
|-----------|----------|-----------|---------|----------|-----------|------------|----------|-------------|
| conv-26 | 19 | 154 | 117 | 75.97% | N/A | N/A | N/A | N/A |
| conv-47 | -1 | 150 | 107 | 71.33% | N/A | N/A | N/A | N/A |

143
hindsight-dev/benchmarks/longmemeval/longmemeval_benchmark.py
View file

@ -7,7 +7,6 @@ import sys
from pathlib import Path
from benchmarks.common.benchmark_runner import BenchmarkRunner
from hindsight_api import MemoryEngine
import json
from datetime import datetime, timezone
@ -63,26 +62,12 @@ class LongMemEvalDataset(BenchmarkDataset):
# Parse session date
session_date = self._parse_date(date_str) if date_str else datetime.now(timezone.utc)
# Combine all turns in the session into one content string
session_content_parts = []
for turn_dict in session_turns:
role = turn_dict.get("role", "")
content = turn_dict.get("content", "")
if not content.strip():
continue
# Format as "role: content"
session_content_parts.append(f"{role}: {content}")
# Add session to batch
if session_content_parts:
session_content = "\n".join(session_content_parts)
session_content = json.dumps(session_turns)
question_id = item.get("question_id", "unknown")
document_id = f"{question_id}_{session_id}"
batch_contents.append({
"content": session_content,
"context": f"Session {session_id}",
"context": f"Session {session_id} - you are the assistant in this conversation - happened on {session_date.strftime('%Y-%m-%d %H:%M:%S')} UTC.",
"event_date": session_date,
"document_id": document_id
})
@ -128,7 +113,7 @@ class LongMemEvalDataset(BenchmarkDataset):
# Fallback: try ISO format
return datetime.fromisoformat(date_str.replace('Z', '+00:00'))
except Exception:
return datetime.now(timezone.utc)
raise ValueError(f"Failed to parse date string: {date_str}")
class QuestionAnswer(pydantic.BaseModel):
@ -147,7 +132,7 @@ class LongMemEvalAnswerGenerator(LLMAnswerGenerator):
async def generate_answer(
self,
question: str,
memories: List[Dict[str, Any]],
recall_result: Dict[str, Any],
question_date: Optional[datetime] = None
) -> Tuple[str, str, Optional[List[Dict[str, Any]]]]:
"""
@ -155,20 +140,14 @@ class LongMemEvalAnswerGenerator(LLMAnswerGenerator):
Args:
question: The question text
memories: Retrieved memories
recall_result: Full RecallResult dict containing results, entities, chunks, and trace
question_date: Date when the question was asked (for temporal context)
Returns:
Tuple of (answer, reasoning, None)
- None indicates to use the memories passed in
- None indicates to use the memories from recall_result
"""
# Format context
context_parts = []
for result in memories:
context_parts.append({"text": result.get("text"), "context": result.get("context"),
"event_date": result.get("event_date")})
context = json.dumps(context_parts)
context = json.dumps(recall_result)
# Format question date if provided
question_date_str = ""
@ -187,7 +166,7 @@ class LongMemEvalAnswerGenerator(LLMAnswerGenerator):
"role": "user",
"content": f"""
# CONTEXT:
You have access to facts and entities from a conversation.
You have access to memories from a conversation.
{question_date_str}
# INSTRUCTIONS:
1. Carefully analyze all provided memories
@ -247,11 +226,11 @@ async def run_benchmark(
thinking_budget: int = 500,
max_tokens: int = 8192,
skip_ingestion: bool = False,
api_url: str = None,
filln: bool = False,
question_id: str = None,
only_failed: bool = False,
only_invalid: bool = False
only_invalid: bool = False,
category: str = None
):
"""
Run the LongMemEval benchmark.
@ -262,11 +241,11 @@ async def run_benchmark(
thinking_budget: Thinking budget for spreading activation search
max_tokens: Maximum tokens to retrieve from memories
skip_ingestion: Whether to skip ingestion and use existing data
api_url: Optional API URL to connect to (default: use local memory)
filln: If True, only process questions where the agent has no indexed data yet
question_id: Optional question ID to filter (e.g., 'e47becba'). Useful with --skip-ingestion.
only_failed: If True, only run questions that were previously marked as incorrect (is_correct=False)
only_invalid: If True, only run questions that were previously marked as invalid (is_invalid=True)
category: Optional category to filter questions (e.g., 'single-session-user', 'multi-session', 'temporal-reasoning')
"""
from rich.console import Console
console = Console()
@ -309,6 +288,33 @@ async def run_benchmark(
# Initialize components
dataset = LongMemEvalDataset()
# Start with all items or load from dataset
original_dataset_items = None
filtered_items = None
# Filter dataset by category if specified
if category:
console.print(f"[cyan]Filtering questions by category: {category}[/cyan]")
if original_dataset_items is None:
# Load full dataset without max_instances limit for filtering
original_dataset_items = dataset.load(dataset_path, max_items=None)
filtered_items = [item for item in original_dataset_items if item.get('question_type') == category]
if not filtered_items:
console.print(f"[yellow]No questions found for category '{category}'. Available categories:[/yellow]")
available_categories = set(item.get('question_type', 'unknown') for item in original_dataset_items)
for cat in sorted(available_categories):
console.print(f" - {cat}")
return
total_found = len(filtered_items)
will_run = min(total_found, max_instances) if max_instances else total_found
if max_instances and total_found > max_instances:
console.print(f"[green]Found {total_found} questions for category '{category}' (will run {will_run} due to --max-instances)[/green]")
else:
console.print(f"[green]Found {total_found} questions for category '{category}'[/green]")
# Filter dataset based on failed/invalid flags
if only_failed or only_invalid:
target_ids = failed_question_ids if only_failed else invalid_question_ids
@ -316,27 +322,31 @@ async def run_benchmark(
filter_type = "failed" if only_failed else "invalid"
console.print(f"[yellow]No {filter_type} questions found in previous results. Nothing to run.[/yellow]")
return
# Override question_id to be None if we're filtering by failed/invalid
# The filtering will happen when we load the dataset
original_dataset_items = dataset.load(dataset_path, max_instances)
filtered_items = [item for item in original_dataset_items if dataset.get_item_id(item) in target_ids]
console.print(f"[green]Found {len(filtered_items)} items to re-evaluate[/green]")
# Load original items if not already loaded
if original_dataset_items is None:
# Load full dataset without max_instances limit for filtering
original_dataset_items = dataset.load(dataset_path, max_items=None)
# If we already have filtered_items from category filtering, filter those
# Otherwise start with all items
items_to_filter = filtered_items if filtered_items is not None else original_dataset_items
filtered_items = [item for item in items_to_filter if dataset.get_item_id(item) in target_ids]
filter_type = "failed" if only_failed else "invalid"
total_found = len(filtered_items)
will_run = min(total_found, max_instances) if max_instances else total_found
if max_instances and total_found > max_instances:
console.print(f"[green]Found {total_found} {filter_type} items to re-evaluate (will run {will_run} due to --max-instances)[/green]")
else:
console.print(f"[green]Found {total_found} {filter_type} items to re-evaluate[/green]")
answer_generator = LongMemEvalAnswerGenerator()
answer_evaluator = LLMAnswerEvaluator()
# Use remote API client if api_url is provided, otherwise use local memory
if api_url:
from benchmarks.common.benchmark_runner import HindsightClientAdapter
memory = HindsightClientAdapter(base_url=api_url)
else:
memory = MemoryEngine(
db_url=os.getenv("HINDSIGHT_API_DATABASE_URL"),
memory_llm_provider=os.getenv("HINDSIGHT_API_LLM_PROVIDER", "groq"),
memory_llm_api_key=os.getenv("HINDSIGHT_API_LLM_API_KEY"),
memory_llm_model=os.getenv("HINDSIGHT_API_LLM_MODEL", "openai/gpt-oss-120b"),
memory_llm_base_url=os.getenv("HINDSIGHT_API_LLM_BASE_URL") or None, # Use None to get provider defaults
)
# Create local memory engine
from benchmarks.common.benchmark_runner import create_memory_engine
memory = await create_memory_engine()
# Create benchmark runner
runner = BenchmarkRunner(
@ -346,9 +356,9 @@ async def run_benchmark(
memory=memory
)
# If filtering by failed/invalid, we need to use a custom dataset that only returns those items
# If filtering by category, failed, or invalid, we need to use a custom dataset that only returns those items
# We'll temporarily replace the dataset's load method
if only_failed or only_invalid:
if filtered_items is not None:
original_load = dataset.load
def filtered_load(path: Path, max_items: Optional[int] = None):
return filtered_items[:max_items] if max_items else filtered_items
@ -357,6 +367,9 @@ async def run_benchmark(
# Run benchmark
# Single-phase approach: each question gets its own isolated agent_id
# This ensures each question only has access to its own context
output_path = Path(__file__).parent / 'results' / 'benchmark_results.json'
merge_with_existing = (filln or question_id is not None or only_failed or only_invalid or category is not None)
results = await runner.run(
dataset_path=dataset_path,
agent_id="longmemeval", # Will be suffixed with question_id per item
@ -370,16 +383,14 @@ async def run_benchmark(
separate_ingestion_phase=False, # Process each question independently
clear_agent_per_item=True, # Use unique agent_id per question
filln=filln, # Only process questions without indexed data
specific_item=question_id # Optional filter for specific question ID
specific_item=question_id, # Optional filter for specific question ID
output_path=output_path, # Save results incrementally
merge_with_existing=merge_with_existing # Merge when using --fill, --category, --only-failed, --only-invalid flags or specific question
)
# Display and save results
# Display results (final save already happened incrementally)
runner.display_results(results)
runner.save_results(
results,
Path(__file__).parent / 'results' / 'benchmark_results.json',
merge_with_existing=(filln or question_id is not None or only_failed or only_invalid) # Merge when using --fill, --only-failed, --only-invalid flags or specific question
)
console.print(f"\n[green]✓[/green] Results saved incrementally to {output_path}")
# Generate detailed report by question type
generate_type_report(results)
@ -506,12 +517,6 @@ if __name__ == "__main__":
action="store_true",
help="Skip ingestion and use existing data"
)
parser.add_argument(
"--api-url",
type=str,
default=None,
help="Hindsight API URL (default: use local memory, example: http://localhost:8888)"
)
parser.add_argument(
"--fill",
action="store_true",
@ -533,6 +538,12 @@ if __name__ == "__main__":
action="store_true",
help="Only run questions that were previously marked as invalid (is_invalid=True). Requires existing results file."
)
parser.add_argument(
"--category",
type=str,
default=None,
help="Filter questions by category/question_type. Available categories: 'single-session-user', 'multi-session', 'single-session-preference', 'temporal-reasoning', 'knowledge-update', 'single-session-assistant'."
)
args = parser.parse_args()
@ -546,9 +557,9 @@ if __name__ == "__main__":
thinking_budget=args.thinking_budget,
max_tokens=args.max_tokens,
skip_ingestion=args.skip_ingestion,
api_url=args.api_url,
filln=args.fill,
question_id=args.question_id,
only_failed=args.only_failed,
only_invalid=args.only_invalid
only_invalid=args.only_invalid,
category=args.category
))

1013628
hindsight-dev/benchmarks/longmemeval/results/benchmark_results.json
View file

File diff suppressed because one or more lines are too long

8
hindsight-docs/docs/developer/api/installation.md
View file

@ -55,7 +55,7 @@ The CLI is included with the Rust distribution. See [CLI documentation](/sdks/cl
## LLM Provider Setup
Hindsight requires an LLM for fact extraction and reasoning. Configure your provider:
Hindsight requires an LLM that supports **structured output** for fact extraction and reasoning. Configure your provider:
<Tabs>
<TabItem value="openai" label="OpenAI">
@ -64,7 +64,7 @@ Hindsight requires an LLM for fact extraction and reasoning. Configure your prov
export OPENAI_API_KEY=sk-...
```
**Models:** `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo`
**Requirement:** Model must support structured output (JSON mode)
</TabItem>
<TabItem value="groq" label="Groq">
@ -73,7 +73,7 @@ export OPENAI_API_KEY=sk-...
export GROQ_API_KEY=gsk_...
```
**Models:** `llama-3.3-70b-versatile`, `mixtral-8x7b-32768`
**Requirement:** Model must support structured output (JSON mode)
</TabItem>
<TabItem value="ollama" label="Ollama (Local)">
@ -82,7 +82,7 @@ export GROQ_API_KEY=gsk_...
# No API key needed - runs locally
```
**Models:** `llama3.1`, `mistral`, `qwen2.5`
**Requirement:** Model must support structured output (JSON mode)
See [Ollama documentation](https://ollama.ai) for setup.

2
hindsight-docs/docs/developer/api/main-methods.md
View file

@ -312,4 +312,4 @@ hindsight reflect my-bank "Analyze our tech stack" --budget high
- [**Retain**](./retain) — Advanced options for storing memories
- [**Recall**](./recall) — Tuning search quality and performance
- [**Reflect**](./reflect) — Configuring personality and opinions
- [**Bank Identity**](./bank-identity) — Managing memory bank personality
- [**Memory Banks**](./memory-banks) — Managing memory bank personality

6
hindsight-docs/docs/developer/api/quickstart.md
View file

@ -119,6 +119,8 @@ hindsight reflect my-bank "Tell me about Alice"
## Next Steps
- [**Main Methods**](./main-methods) — Detailed guide to retain, recall, and reflect
- [**Bank Identity**](./bank-identity) — Configure personality and background
- [**Retain**](./retain) — Advanced options for storing memories
- [**Recall**](./recall) — Search and retrieval strategies
- [**Reflect**](./reflect) — Personality-aware reasoning
- [**Memory Banks**](./memory-banks) — Configure personality and background
- [**Server Options**](/developer/server) — Production deployment

24
hindsight-docs/docs/developer/index.md
View file

@ -103,20 +103,22 @@ The `bias_strength` parameter (0-1) controls how much personality influences opi
## Next Steps
### Getting Started
- [**Installation**](./api/installation) — Install Hindsight for Python, Node.js, or CLI
- [**Quick Start**](./api/quickstart) — Get up and running in 60 seconds
- [**Installation**](/developer/api/installation) — Install Hindsight for Python, Node.js, or CLI
- [**Quick Start**](/developer/api/quickstart) — Get up and running in 60 seconds
### Core Concepts
- [**Retain**](./retain) — How memories are stored with multi-dimensional facts
- [**Recall**](./retrieval) — How TEMPR's 4-way search retrieves memories
- [**Reflect**](./personality) — How personality influences reasoning and opinion formation
- [**Retain**](/developer/retain) — How memories are stored with multi-dimensional facts
- [**Recall**](/developer/retrieval) — How TEMPR's 4-way search retrieves memories
- [**Reflect**](/developer/personality) — How personality influences reasoning and opinion formation
### API Methods
- [**Main Methods**](./api/main-methods) — Overview of retain, recall, reflect
- [**Memory Banks**](./api/memory-banks) — Configure personality and background
- [**Entities**](./api/entities) — Track people, places, and concepts
- [**Documents**](./api/documents) — Manage document sources
- [**Operations**](./api/operations) — Monitor async tasks
- [**Retain**](/developer/api/retain) — Store information in memory banks
- [**Recall**](/developer/api/recall) — Search and retrieve memories
- [**Reflect**](/developer/api/reflect) — Reason with personality
- [**Memory Banks**](/developer/api/memory-banks) — Configure personality and background
- [**Entities**](/developer/api/entities) — Track people, places, and concepts
- [**Documents**](/developer/api/documents) — Manage document sources
- [**Operations**](/developer/api/operations) — Monitor async tasks
### Deployment
- [**Server Setup**](./server) — Deploy with Docker Compose, Helm, or pip
- [**Server Setup**](/developer/server) — Deploy with Docker Compose, Helm, or pip

2
hindsight-docs/docs/developer/personality.md
View file

@ -206,4 +206,4 @@ Personality creates **consistent character** across conversations while allowing
- [**Retain**](./retain) — How rich facts are stored
- [**Recall**](./retrieval) — How multi-strategy search works
- [API Reference: Reflect](/developer/api/think) — Code examples and usage
- [API Reference: Reflect](./api/reflect) — Code examples and usage

2
hindsight-docs/docs/developer/retain.md
View file

@ -191,4 +191,4 @@ All stored in your isolated **memory bank**, ready for `recall()` and `reflect()
- [**Recall**](./retrieval) — How multi-strategy search retrieves relevant memories
- [**Reflect**](./personality) — How personality influences reasoning and opinion formation
- [API Reference](/developer/api/ingest) — Code examples for retaining memories
- [API Reference](./api/retain) — Code examples for retaining memories

384
scripts/prometheus-dashboard.html Normal file
View file

@ -0,0 +1,384 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Hindsight Metrics Dashboard</title>
<script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.0/dist/chart.umd.min.js"></script>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
background: #0f1419;
color: #e6e6e6;
padding: 20px;
}
.header {
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
padding: 30px;
border-radius: 10px;
margin-bottom: 30px;
}
h1 {
font-size: 32px;
font-weight: 600;
}
.subtitle {
opacity: 0.9;
margin-top: 5px;
}
.grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(500px, 1fr));
gap: 20px;
margin-bottom: 20px;
}
.card {
background: #1a1f2e;
border-radius: 10px;
padding: 20px;
border: 1px solid #2d3548;
}
.card-title {
font-size: 18px;
font-weight: 600;
margin-bottom: 15px;
color: #a0aec0;
}
.chart-container {
position: relative;
height: 300px;
}
.stats {
display: grid;
grid-template-columns: repeat(3, 1fr);
gap: 15px;
margin-bottom: 20px;
}
.stat {
background: #1a1f2e;
padding: 20px;
border-radius: 10px;
border: 1px solid #2d3548;
}
.stat-label {
font-size: 12px;
color: #a0aec0;
text-transform: uppercase;
letter-spacing: 0.5px;
margin-bottom: 8px;
}
.stat-value {
font-size: 28px;
font-weight: 600;
color: #667eea;
}
.stat-unit {
font-size: 14px;
color: #a0aec0;
margin-left: 4px;
}
.error {
background: #2d1b1b;
border: 1px solid #7d2828;
color: #f56565;
padding: 15px;
border-radius: 8px;
margin: 20px 0;
}
.loading {
text-align: center;
padding: 40px;
color: #a0aec0;
}
</style>
</head>
<body>
<div class="header">
<h1>🧠 Hindsight Metrics Dashboard</h1>
<div class="subtitle">Real-time performance monitoring • Updates every 15s</div>
</div>
<div id="error-container"></div>
<div id="loading" class="loading">Loading metrics...</div>
<div class="stats" id="stats" style="display: none;"></div>
<div class="grid" id="charts" style="display: none;">
<div class="card">
<div class="card-title">📊 Recall Latency Percentiles</div>
<div class="chart-container">
<canvas id="latencyChart"></canvas>
</div>
</div>
<div class="card">
<div class="card-title">⚡ Operations per Second</div>
<div class="chart-container">
<canvas id="opsChart"></canvas>
</div>
</div>
<div class="card">
<div class="card-title">🎯 Latency by Bank</div>
<div class="chart-container">
<canvas id="bankLatencyChart"></canvas>
</div>
</div>
<div class="card">
<div class="card-title">💰 Token Usage Rate</div>
<div class="chart-container">
<canvas id="tokenChart"></canvas>
</div>
</div>
</div>
<script>
const PROMETHEUS_URL = 'http://localhost:9090';
const UPDATE_INTERVAL = 15000; // 15 seconds
// Chart configurations
const chartConfig = {
type: 'line',
options: {
responsive: true,
maintainAspectRatio: false,
plugins: {
legend: {
labels: { color: '#e6e6e6' }
}
},
scales: {
x: {
ticks: { color: '#a0aec0' },
grid: { color: '#2d3548' }
},
y: {
ticks: { color: '#a0aec0' },
grid: { color: '#2d3548' }
}
}
}
};
let charts = {};
async function queryPrometheus(query) {
const url = `${PROMETHEUS_URL}/api/v1/query?query=${encodeURIComponent(query)}`;
const response = await fetch(url);
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const data = await response.json();
if (data.status !== 'success') throw new Error('Query failed');
return data.data.result;
}
async function queryRange(query, minutes = 30) {
const end = Math.floor(Date.now() / 1000);
const start = end - (minutes * 60);
const url = `${PROMETHEUS_URL}/api/v1/query_range?query=${encodeURIComponent(query)}&start=${start}&end=${end}&step=30`;
const response = await fetch(url);
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const data = await response.json();
if (data.status !== 'success') throw new Error('Query failed');
return data.data.result;
}
function showError(message) {
const container = document.getElementById('error-container');
container.innerHTML = `
<div class="error">
<strong>⚠️ Error:</strong> ${message}<br>
<small>Make sure Prometheus is running at ${PROMETHEUS_URL}</small>
</div>
`;
}
function hideLoading() {
document.getElementById('loading').style.display = 'none';
document.getElementById('stats').style.display = 'grid';
document.getElementById('charts').style.display = 'grid';
}
async function updateStats() {
try {
// Query current stats
const totalOps = await queryPrometheus('sum(hindsight_operation_total)');
const successRate = await queryPrometheus('sum(hindsight_operation_total{success="true"}) / sum(hindsight_operation_total)');
const opsRate = await queryPrometheus('sum(rate(hindsight_operation_total[5m]))');
const stats = [
{
label: 'Total Operations',
value: totalOps[0]?.value[1] || '0',
unit: ''
},
{
label: 'Success Rate',
value: ((parseFloat(successRate[0]?.value[1] || 1) * 100).toFixed(1)),
unit: '%'
},
{
label: 'Ops/sec',
value: (parseFloat(opsRate[0]?.value[1] || 0).toFixed(2)),
unit: 'ops/s'
}
];
document.getElementById('stats').innerHTML = stats.map(stat => `
<div class="stat">
<div class="stat-label">${stat.label}</div>
<div class="stat-value">
${stat.value}
<span class="stat-unit">${stat.unit}</span>
</div>
</div>
`).join('');
} catch (error) {
console.error('Error updating stats:', error);
}
}
async function updateCharts() {
try {
// Latency percentiles over time
const p50Data = await queryRange('histogram_quantile(0.50, rate(hindsight_operation_duration_seconds_bucket{operation="recall"}[5m]))');
const p95Data = await queryRange('histogram_quantile(0.95, rate(hindsight_operation_duration_seconds_bucket{operation="recall"}[5m]))');
const p99Data = await queryRange('histogram_quantile(0.99, rate(hindsight_operation_duration_seconds_bucket{operation="recall"}[5m]))');
if (p50Data.length > 0) {
const timestamps = p50Data[0].values.map(v => new Date(v[0] * 1000).toLocaleTimeString());
if (!charts.latency) {
charts.latency = new Chart(document.getElementById('latencyChart'), {
...chartConfig,
data: {
labels: timestamps,
datasets: [
{ label: 'p50', data: p50Data[0].values.map(v => parseFloat(v[1])), borderColor: '#48bb78', tension: 0.4 },
{ label: 'p95', data: p95Data[0].values.map(v => parseFloat(v[1])), borderColor: '#ed8936', tension: 0.4 },
{ label: 'p99', data: p99Data[0].values.map(v => parseFloat(v[1])), borderColor: '#f56565', tension: 0.4 }
]
}
});
} else {
charts.latency.data.labels = timestamps;
charts.latency.data.datasets[0].data = p50Data[0].values.map(v => parseFloat(v[1]));
charts.latency.data.datasets[1].data = p95Data[0].values.map(v => parseFloat(v[1]));
charts.latency.data.datasets[2].data = p99Data[0].values.map(v => parseFloat(v[1]));
charts.latency.update();
}
}
// Operations per second
const opsData = await queryRange('sum by (operation) (rate(hindsight_operation_total[5m]))');
if (opsData.length > 0) {
const timestamps = opsData[0].values.map(v => new Date(v[0] * 1000).toLocaleTimeString());
const datasets = opsData.map((series, i) => ({
label: series.metric.operation,
data: series.values.map(v => parseFloat(v[1])),
borderColor: ['#667eea', '#48bb78', '#ed8936'][i % 3],
tension: 0.4
}));
if (!charts.ops) {
charts.ops = new Chart(document.getElementById('opsChart'), {
...chartConfig,
data: { labels: timestamps, datasets }
});
} else {
charts.ops.data.labels = timestamps;
charts.ops.data.datasets = datasets;
charts.ops.update();
}
}
// Latency by bank
const bankLatency = await queryRange('histogram_quantile(0.95, sum by (bank_id, le) (rate(hindsight_operation_duration_seconds_bucket{operation="recall"}[5m])))');
if (bankLatency.length > 0) {
const timestamps = bankLatency[0].values.map(v => new Date(v[0] * 1000).toLocaleTimeString());
const datasets = bankLatency.map((series, i) => ({
label: series.metric.bank_id,
data: series.values.map(v => parseFloat(v[1])),
borderColor: ['#667eea', '#48bb78', '#ed8936', '#f56565'][i % 4],
tension: 0.4
}));
if (!charts.bankLatency) {
charts.bankLatency = new Chart(document.getElementById('bankLatencyChart'), {
...chartConfig,
data: { labels: timestamps, datasets }
});
} else {
charts.bankLatency.data.labels = timestamps;
charts.bankLatency.data.datasets = datasets;
charts.bankLatency.update();
}
}
// Token usage
const tokenInput = await queryRange('sum(rate(hindsight_tokens_input_total[5m]))');
const tokenOutput = await queryRange('sum(rate(hindsight_tokens_output_total[5m]))');
if (tokenInput.length > 0) {
const timestamps = tokenInput[0].values.map(v => new Date(v[0] * 1000).toLocaleTimeString());
if (!charts.tokens) {
charts.tokens = new Chart(document.getElementById('tokenChart'), {
...chartConfig,
data: {
labels: timestamps,
datasets: [
{ label: 'Input', data: tokenInput[0].values.map(v => parseFloat(v[1])), borderColor: '#667eea', tension: 0.4 },
{ label: 'Output', data: tokenOutput[0]?.values.map(v => parseFloat(v[1])) || [], borderColor: '#48bb78', tension: 0.4 }
]
}
});
} else {
charts.tokens.data.labels = timestamps;
charts.tokens.data.datasets[0].data = tokenInput[0].values.map(v => parseFloat(v[1]));
charts.tokens.data.datasets[1].data = tokenOutput[0]?.values.map(v => parseFloat(v[1])) || [];
charts.tokens.update();
}
}
hideLoading();
} catch (error) {
console.error('Error updating charts:', error);
showError(error.message);
}
}
async function update() {
await updateStats();
await updateCharts();
}
// Initial update
update();
// Update every 15 seconds
setInterval(update, UPDATE_INTERVAL);
</script>
</body>
</html>

138
scripts/start-prometheus.sh Executable file
View file

@ -0,0 +1,138 @@
#!/bin/bash
set -e
# Script to download and start Prometheus for Hindsight metrics
# This creates a local Prometheus instance that scrapes metrics from the Hindsight API
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
PROMETHEUS_DIR="$PROJECT_ROOT/.prometheus"
PROMETHEUS_VERSION="2.48.0"
# Detect OS and architecture
OS=$(uname -s | tr '[:upper:]' '[:lower:]')
ARCH=$(uname -m)
case "$OS" in
darwin)
OS_NAME="darwin"
;;
linux)
OS_NAME="linux"
;;
*)
echo "Unsupported OS: $OS"
exit 1
;;
esac
case "$ARCH" in
x86_64)
ARCH_NAME="amd64"
;;
arm64|aarch64)
ARCH_NAME="arm64"
;;
*)
echo "Unsupported architecture: $ARCH"
exit 1
;;
esac
PROMETHEUS_ARCHIVE="prometheus-${PROMETHEUS_VERSION}.${OS_NAME}-${ARCH_NAME}.tar.gz"
PROMETHEUS_URL="https://github.com/prometheus/prometheus/releases/download/v${PROMETHEUS_VERSION}/${PROMETHEUS_ARCHIVE}"
PROMETHEUS_BIN="$PROMETHEUS_DIR/prometheus-${PROMETHEUS_VERSION}.${OS_NAME}-${ARCH_NAME}/prometheus"
echo "🔧 Setting up Prometheus for Hindsight metrics..."
echo ""
# Create prometheus directory
mkdir -p "$PROMETHEUS_DIR"
cd "$PROMETHEUS_DIR"
# Download Prometheus if not exists
if [ ! -f "$PROMETHEUS_BIN" ]; then
echo "📥 Downloading Prometheus ${PROMETHEUS_VERSION} for ${OS_NAME}-${ARCH_NAME}..."
curl -L -o "$PROMETHEUS_ARCHIVE" "$PROMETHEUS_URL"
echo "📦 Extracting..."
tar xzf "$PROMETHEUS_ARCHIVE"
echo "✅ Prometheus downloaded successfully"
echo ""
else
echo "✅ Prometheus already downloaded"
echo ""
fi
# Create prometheus.yml configuration
echo "📝 Creating Prometheus configuration..."
cat > "$PROMETHEUS_DIR/prometheus.yml" <<EOF
# Prometheus configuration for Hindsight API metrics
global:
scrape_interval: 15s # Scrape metrics every 15 seconds
evaluation_interval: 15s # Evaluate rules every 15 seconds
# Scrape configuration
scrape_configs:
- job_name: 'hindsight-api'
scrape_interval: 15s
static_configs:
- targets: ['localhost:8000'] # Hindsight API endpoint
metrics_path: '/metrics' # Metrics endpoint path
# Optional: Add labels to all metrics from this job
# relabeling_configs:
# - source_labels: [__address__]
# target_label: instance
# replacement: 'hindsight-api'
EOF
echo "✅ Configuration created at $PROMETHEUS_DIR/prometheus.yml"
echo ""
# Check if Hindsight API is running
echo "🔍 Checking if Hindsight API is running..."
if curl -s http://localhost:8000/metrics > /dev/null 2>&1; then
echo "✅ Hindsight API is running and serving metrics"
echo ""
else
echo "⚠️ WARNING: Hindsight API is not reachable at http://localhost:8000/metrics"
echo " Make sure to start the API before Prometheus can scrape metrics"
echo ""
fi
# Start Prometheus
echo "🚀 Starting Prometheus..."
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo " Prometheus UI: http://localhost:9090"
echo " Metrics source: http://localhost:8000/metrics"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
echo "📊 Example queries to try in the UI:"
echo ""
echo " p95 latency (all operations):"
echo " histogram_quantile(0.95, rate(hindsight_operation_duration_seconds_bucket[5m]))"
echo ""
echo " p95 latency by bank:"
echo " histogram_quantile(0.95, sum by (bank_id, le) (rate(hindsight_operation_duration_seconds_bucket{operation=\"recall\"}[5m])))"
echo ""
echo " Operations per second:"
echo " rate(hindsight_operation_total[5m])"
echo ""
echo " Token usage rate:"
echo " rate(hindsight_tokens_input_total[5m]) + rate(hindsight_tokens_output_total[5m])"
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
echo "Press Ctrl+C to stop Prometheus"
echo ""
# Start Prometheus with config
cd "$(dirname "$PROMETHEUS_BIN")"
exec "$PROMETHEUS_BIN" \
--config.file="$PROMETHEUS_DIR/prometheus.yml" \
--storage.tsdb.path="$PROMETHEUS_DIR/data" \
--web.console.templates="$PROMETHEUS_DIR/prometheus-${PROMETHEUS_VERSION}.${OS_NAME}-${ARCH_NAME}/consoles" \
--web.console.libraries="$PROMETHEUS_DIR/prometheus-${PROMETHEUS_VERSION}.${OS_NAME}-${ARCH_NAME}/console_libraries"

266
uv.lock
View file

@ -299,6 +299,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/44/9f/bb0f0a1dcce6f478c8d16f10d283a6f5d95cbc3ebc810538f80fee605a81/apswutils-0.1.0-py3-none-any.whl", hash = "sha256:a39ace8a9f14a9bf367993acaa5a867fa57d365f975319983dace3a258f95843", size = 80509 },
]
[[package]]
name = "asgiref"
version = "3.11.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/76/b9/4db2509eabd14b4a8c71d1b24c8d5734c52b8560a7b1e1a8b56c8d25568b/asgiref-3.11.0.tar.gz", hash = "sha256:13acff32519542a1736223fb79a715acdebe24286d98e8b164a73085f40da2c4", size = 37969 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/91/be/317c2c55b8bbec407257d45f5c8d1b6867abc76d12043f2d3d58c538a4ea/asgiref-3.11.0-py3-none-any.whl", hash = "sha256:1db9021efadb0d9512ce8ffaf72fcef601c7b73a8807a1bb2ef143dc6b14846d", size = 24096 },
]
[[package]]
name = "asyncpg"
version = "0.30.0"
@ -1152,6 +1161,10 @@ dependencies = [
{ name = "httpx" },
{ name = "langchain-text-splitters" },
{ name = "openai" },
{ name = "opentelemetry-api" },
{ name = "opentelemetry-exporter-prometheus" },
{ name = "opentelemetry-instrumentation-fastapi" },
{ name = "opentelemetry-sdk" },
{ name = "pgvector" },
{ name = "psycopg2-binary" },
{ name = "pydantic" },
@ -1199,6 +1212,10 @@ requires-dist = [
{ name = "httpx", specifier = ">=0.27.0" },
{ name = "langchain-text-splitters", specifier = ">=0.3.0" },
{ name = "openai", specifier = ">=1.0.0" },
{ name = "opentelemetry-api", specifier = ">=1.20.0" },
{ name = "opentelemetry-exporter-prometheus", specifier = ">=0.41b0" },
{ name = "opentelemetry-instrumentation-fastapi", specifier = ">=0.41b0" },
{ name = "opentelemetry-sdk", specifier = ">=1.20.0" },
{ name = "pgvector", specifier = ">=0.4.1" },
{ name = "psycopg2-binary", specifier = ">=2.9.11" },
{ name = "pydantic", specifier = ">=2.0.0" },
@ -1380,7 +1397,7 @@ name = "importlib-metadata"
version = "8.7.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "zipp", marker = "python_full_version < '3.12'" },
{ name = "zipp" },
]
sdist = { url = "https://files.pythonhosted.org/packages/76/66/650a33bd90f786193e4de4b3ad86ea60b53c89b669a5c7be931fac31cdb0/importlib_metadata-8.7.0.tar.gz", hash = "sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000", size = 56641 }
wheels = [
@ -2223,6 +2240,116 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/12/cf/03675d8bd8ecbf4445504d8071adab19f5f993676795708e36402ab38263/openapi_pydantic-0.5.1-py3-none-any.whl", hash = "sha256:a3a09ef4586f5bd760a8df7f43028b60cafb6d9f61de2acba9574766255ab146", size = 96381 },
]
[[package]]
name = "opentelemetry-api"
version = "1.38.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "importlib-metadata" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/08/d8/0f354c375628e048bd0570645b310797299754730079853095bf000fba69/opentelemetry_api-1.38.0.tar.gz", hash = "sha256:f4c193b5e8acb0912b06ac5b16321908dd0843d75049c091487322284a3eea12", size = 65242 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ae/a2/d86e01c28300bd41bab8f18afd613676e2bd63515417b77636fc1add426f/opentelemetry_api-1.38.0-py3-none-any.whl", hash = "sha256:2891b0197f47124454ab9f0cf58f3be33faca394457ac3e09daba13ff50aa582", size = 65947 },
]
[[package]]
name = "opentelemetry-exporter-prometheus"
version = "0.59b0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "opentelemetry-api" },
{ name = "opentelemetry-sdk" },
{ name = "prometheus-client" },
]
sdist = { url = "https://files.pythonhosted.org/packages/1b/07/39370ec7eacfca10462121a0e036b66ccea3a616bf6ae6ea5fdb72e5009d/opentelemetry_exporter_prometheus-0.59b0.tar.gz", hash = "sha256:d64f23c49abb5a54e271c2fbc8feacea0c394a30ec29876ab5ef7379f08cf3d7", size = 14972 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/05/ea/3005a732002242fd86203989520bdd5a752e1fd30dc225d5d45751ea19fb/opentelemetry_exporter_prometheus-0.59b0-py3-none-any.whl", hash = "sha256:71ced23207abd15b30d1fe4e7e910dcaa7c2ff1f24a6ffccbd4fdded676f541b", size = 13017 },
]
[[package]]
name = "opentelemetry-instrumentation"
version = "0.59b0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "opentelemetry-api" },
{ name = "opentelemetry-semantic-conventions" },
{ name = "packaging" },
{ name = "wrapt" },
]
sdist = { url = "https://files.pythonhosted.org/packages/04/ed/9c65cd209407fd807fa05be03ee30f159bdac8d59e7ea16a8fe5a1601222/opentelemetry_instrumentation-0.59b0.tar.gz", hash = "sha256:6010f0faaacdaf7c4dff8aac84e226d23437b331dcda7e70367f6d73a7db1adc", size = 31544 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/10/f5/7a40ff3f62bfe715dad2f633d7f1174ba1a7dd74254c15b2558b3401262a/opentelemetry_instrumentation-0.59b0-py3-none-any.whl", hash = "sha256:44082cc8fe56b0186e87ee8f7c17c327c4c2ce93bdbe86496e600985d74368ee", size = 33020 },
]
[[package]]
name = "opentelemetry-instrumentation-asgi"
version = "0.59b0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "asgiref" },
{ name = "opentelemetry-api" },
{ name = "opentelemetry-instrumentation" },
{ name = "opentelemetry-semantic-conventions" },
{ name = "opentelemetry-util-http" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b7/a4/cfbb6fc1ec0aa9bf5a93f548e6a11ab3ac1956272f17e0d399aa2c1f85bc/opentelemetry_instrumentation_asgi-0.59b0.tar.gz", hash = "sha256:2509d6fe9fd829399ce3536e3a00426c7e3aa359fc1ed9ceee1628b56da40e7a", size = 25116 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f3/88/fe02d809963b182aafbf5588685d7a05af8861379b0ec203d48e360d4502/opentelemetry_instrumentation_asgi-0.59b0-py3-none-any.whl", hash = "sha256:ba9703e09d2c33c52fa798171f344c8123488fcd45017887981df088452d3c53", size = 16797 },
]
[[package]]
name = "opentelemetry-instrumentation-fastapi"
version = "0.59b0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "opentelemetry-api" },
{ name = "opentelemetry-instrumentation" },
{ name = "opentelemetry-instrumentation-asgi" },
{ name = "opentelemetry-semantic-conventions" },
{ name = "opentelemetry-util-http" },
]
sdist = { url = "https://files.pythonhosted.org/packages/ab/a7/7a6ce5009584ce97dbfd5ce77d4f9d9570147507363349d2cb705c402bcf/opentelemetry_instrumentation_fastapi-0.59b0.tar.gz", hash = "sha256:e8fe620cfcca96a7d634003df1bc36a42369dedcdd6893e13fb5903aeeb89b2b", size = 24967 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/35/27/5914c8bf140ffc70eff153077e225997c7b054f0bf28e11b9ab91b63b18f/opentelemetry_instrumentation_fastapi-0.59b0-py3-none-any.whl", hash = "sha256:0d8d00ff7d25cca40a4b2356d1d40a8f001e0668f60c102f5aa6bb721d660c4f", size = 13492 },
]
[[package]]
name = "opentelemetry-sdk"
version = "1.38.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "opentelemetry-api" },
{ name = "opentelemetry-semantic-conventions" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/85/cb/f0eee1445161faf4c9af3ba7b848cc22a50a3d3e2515051ad8628c35ff80/opentelemetry_sdk-1.38.0.tar.gz", hash = "sha256:93df5d4d871ed09cb4272305be4d996236eedb232253e3ab864c8620f051cebe", size = 171942 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/2f/2e/e93777a95d7d9c40d270a371392b6d6f1ff170c2a3cb32d6176741b5b723/opentelemetry_sdk-1.38.0-py3-none-any.whl", hash = "sha256:1c66af6564ecc1553d72d811a01df063ff097cdc82ce188da9951f93b8d10f6b", size = 132349 },
]
[[package]]
name = "opentelemetry-semantic-conventions"
version = "0.59b0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "opentelemetry-api" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/40/bc/8b9ad3802cd8ac6583a4eb7de7e5d7db004e89cb7efe7008f9c8a537ee75/opentelemetry_semantic_conventions-0.59b0.tar.gz", hash = "sha256:7a6db3f30d70202d5bf9fa4b69bc866ca6a30437287de6c510fb594878aed6b0", size = 129861 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/24/7d/c88d7b15ba8fe5c6b8f93be50fc11795e9fc05386c44afaf6b76fe191f9b/opentelemetry_semantic_conventions-0.59b0-py3-none-any.whl", hash = "sha256:35d3b8833ef97d614136e253c1da9342b4c3c083bbaf29ce31d572a1c3825eed", size = 207954 },
]
[[package]]
name = "opentelemetry-util-http"
version = "0.59b0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/34/f7/13cd081e7851c42520ab0e96efb17ffbd901111a50b8252ec1e240664020/opentelemetry_util_http-0.59b0.tar.gz", hash = "sha256:ae66ee91be31938d832f3b4bc4eb8a911f6eddd38969c4a871b1230db2a0a560", size = 9412 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/20/56/62282d1d4482061360449dacc990c89cad0fc810a2ed937b636300f55023/opentelemetry_util_http-0.59b0-py3-none-any.whl", hash = "sha256:6d036a07563bce87bf521839c0671b507a02a0d39d7ea61b88efa14c6e25355d", size = 7648 },
]
[[package]]
name = "orjson"
version = "3.11.4"
@ -2489,6 +2616,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538 },
]
[[package]]
name = "prometheus-client"
version = "0.23.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/23/53/3edb5d68ecf6b38fcbcc1ad28391117d2a322d9a1a3eff04bfdb184d8c3b/prometheus_client-0.23.1.tar.gz", hash = "sha256:6ae8f9081eaaaf153a2e959d2e6c4f4fb57b12ef76c8c7980202f1e57b48b2ce", size = 80481 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b8/db/14bafcb4af2139e046d03fd00dea7873e48eafe18b7d2797e73d6681f210/prometheus_client-0.23.1-py3-none-any.whl", hash = "sha256:dd1913e6e76b59cfe44e7a4b83e01afc9873c1bdfd2ed8739f1e76aeca115f99", size = 61145 },
]
[[package]]
name = "propcache"
version = "0.4.1"
@ -4329,83 +4465,61 @@ wheels = [
[[package]]
name = "wrapt"
version = "2.0.1"
version = "1.17.3"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/49/2a/6de8a50cb435b7f42c46126cf1a54b2aab81784e74c8595c8e025e8f36d3/wrapt-2.0.1.tar.gz", hash = "sha256:9c9c635e78497cacb81e84f8b11b23e0aacac7a136e73b8e5b2109a1d9fc468f", size = 82040 }
sdist = { url = "https://files.pythonhosted.org/packages/95/8f/aeb76c5b46e273670962298c23e7ddde79916cb74db802131d49a85e4b7d/wrapt-1.17.3.tar.gz", hash = "sha256:f66eb08feaa410fe4eebd17f2a2c8e2e46d3476e9f8c783daa8e09e0faa666d0", size = 55547 }
wheels = [
{ url = "https://files.pythonhosted.org/packages/98/60/553997acf3939079dab022e37b67b1904b5b0cc235503226898ba573b10c/wrapt-2.0.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:0e17283f533a0d24d6e5429a7d11f250a58d28b4ae5186f8f47853e3e70d2590", size = 77480 },
{ url = "https://files.pythonhosted.org/packages/2d/50/e5b3d30895d77c52105c6d5cbf94d5b38e2a3dd4a53d22d246670da98f7c/wrapt-2.0.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:85df8d92158cb8f3965aecc27cf821461bb5f40b450b03facc5d9f0d4d6ddec6", size = 60690 },
{ url = "https://files.pythonhosted.org/packages/f0/40/660b2898703e5cbbb43db10cdefcc294274458c3ca4c68637c2b99371507/wrapt-2.0.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:c1be685ac7700c966b8610ccc63c3187a72e33cab53526a27b2a285a662cd4f7", size = 61578 },
{ url = "https://files.pythonhosted.org/packages/5b/36/825b44c8a10556957bc0c1d84c7b29a40e05fcf1873b6c40aa9dbe0bd972/wrapt-2.0.1-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:df0b6d3b95932809c5b3fecc18fda0f1e07452d05e2662a0b35548985f256e28", size = 114115 },
{ url = "https://files.pythonhosted.org/packages/83/73/0a5d14bb1599677304d3c613a55457d34c344e9b60eda8a737c2ead7619e/wrapt-2.0.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4da7384b0e5d4cae05c97cd6f94faaf78cc8b0f791fc63af43436d98c4ab37bb", size = 116157 },
{ url = "https://files.pythonhosted.org/packages/01/22/1c158fe763dbf0a119f985d945711d288994fe5514c0646ebe0eb18b016d/wrapt-2.0.1-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:ec65a78fbd9d6f083a15d7613b2800d5663dbb6bb96003899c834beaa68b242c", size = 112535 },
{ url = "https://files.pythonhosted.org/packages/5c/28/4f16861af67d6de4eae9927799b559c20ebdd4fe432e89ea7fe6fcd9d709/wrapt-2.0.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:7de3cc939be0e1174969f943f3b44e0d79b6f9a82198133a5b7fc6cc92882f16", size = 115404 },
{ url = "https://files.pythonhosted.org/packages/a0/8b/7960122e625fad908f189b59c4aae2d50916eb4098b0fb2819c5a177414f/wrapt-2.0.1-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:fb1a5b72cbd751813adc02ef01ada0b0d05d3dcbc32976ce189a1279d80ad4a2", size = 111802 },
{ url = "https://files.pythonhosted.org/packages/3e/73/7881eee5ac31132a713ab19a22c9e5f1f7365c8b1df50abba5d45b781312/wrapt-2.0.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:3fa272ca34332581e00bf7773e993d4f632594eb2d1b0b162a9038df0fd971dd", size = 113837 },
{ url = "https://files.pythonhosted.org/packages/45/00/9499a3d14e636d1f7089339f96c4409bbc7544d0889f12264efa25502ae8/wrapt-2.0.1-cp311-cp311-win32.whl", hash = "sha256:fc007fdf480c77301ab1afdbb6ab22a5deee8885f3b1ed7afcb7e5e84a0e27be", size = 58028 },
{ url = "https://files.pythonhosted.org/packages/70/5d/8f3d7eea52f22638748f74b102e38fdf88cb57d08ddeb7827c476a20b01b/wrapt-2.0.1-cp311-cp311-win_amd64.whl", hash = "sha256:47434236c396d04875180171ee1f3815ca1eada05e24a1ee99546320d54d1d1b", size = 60385 },
{ url = "https://files.pythonhosted.org/packages/14/e2/32195e57a8209003587bbbad44d5922f13e0ced2a493bb46ca882c5b123d/wrapt-2.0.1-cp311-cp311-win_arm64.whl", hash = "sha256:837e31620e06b16030b1d126ed78e9383815cbac914693f54926d816d35d8edf", size = 58893 },
{ url = "https://files.pythonhosted.org/packages/cb/73/8cb252858dc8254baa0ce58ce382858e3a1cf616acebc497cb13374c95c6/wrapt-2.0.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:1fdbb34da15450f2b1d735a0e969c24bdb8d8924892380126e2a293d9902078c", size = 78129 },
{ url = "https://files.pythonhosted.org/packages/19/42/44a0db2108526ee6e17a5ab72478061158f34b08b793df251d9fbb9a7eb4/wrapt-2.0.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:3d32794fe940b7000f0519904e247f902f0149edbe6316c710a8562fb6738841", size = 61205 },
{ url = "https://files.pythonhosted.org/packages/4d/8a/5b4b1e44b791c22046e90d9b175f9a7581a8cc7a0debbb930f81e6ae8e25/wrapt-2.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:386fb54d9cd903ee0012c09291336469eb7b244f7183d40dc3e86a16a4bace62", size = 61692 },
{ url = "https://files.pythonhosted.org/packages/11/53/3e794346c39f462bcf1f58ac0487ff9bdad02f9b6d5ee2dc84c72e0243b2/wrapt-2.0.1-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:7b219cb2182f230676308cdcacd428fa837987b89e4b7c5c9025088b8a6c9faf", size = 121492 },
{ url = "https://files.pythonhosted.org/packages/c6/7e/10b7b0e8841e684c8ca76b462a9091c45d62e8f2de9c4b1390b690eadf16/wrapt-2.0.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:641e94e789b5f6b4822bb8d8ebbdfc10f4e4eae7756d648b717d980f657a9eb9", size = 123064 },
{ url = "https://files.pythonhosted.org/packages/0e/d1/3c1e4321fc2f5ee7fd866b2d822aa89b84495f28676fd976c47327c5b6aa/wrapt-2.0.1-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:fe21b118b9f58859b5ebaa4b130dee18669df4bd111daad082b7beb8799ad16b", size = 117403 },
{ url = "https://files.pythonhosted.org/packages/a4/b0/d2f0a413cf201c8c2466de08414a15420a25aa83f53e647b7255cc2fab5d/wrapt-2.0.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:17fb85fa4abc26a5184d93b3efd2dcc14deb4b09edcdb3535a536ad34f0b4dba", size = 121500 },
{ url = "https://files.pythonhosted.org/packages/bd/45/bddb11d28ca39970a41ed48a26d210505120f925918592283369219f83cc/wrapt-2.0.1-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:b89ef9223d665ab255ae42cc282d27d69704d94be0deffc8b9d919179a609684", size = 116299 },
{ url = "https://files.pythonhosted.org/packages/81/af/34ba6dd570ef7a534e7eec0c25e2615c355602c52aba59413411c025a0cb/wrapt-2.0.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:a453257f19c31b31ba593c30d997d6e5be39e3b5ad9148c2af5a7314061c63eb", size = 120622 },
{ url = "https://files.pythonhosted.org/packages/e2/3e/693a13b4146646fb03254636f8bafd20c621955d27d65b15de07ab886187/wrapt-2.0.1-cp312-cp312-win32.whl", hash = "sha256:3e271346f01e9c8b1130a6a3b0e11908049fe5be2d365a5f402778049147e7e9", size = 58246 },
{ url = "https://files.pythonhosted.org/packages/a7/36/715ec5076f925a6be95f37917b66ebbeaa1372d1862c2ccd7a751574b068/wrapt-2.0.1-cp312-cp312-win_amd64.whl", hash = "sha256:2da620b31a90cdefa9cd0c2b661882329e2e19d1d7b9b920189956b76c564d75", size = 60492 },
{ url = "https://files.pythonhosted.org/packages/ef/3e/62451cd7d80f65cc125f2b426b25fbb6c514bf6f7011a0c3904fc8c8df90/wrapt-2.0.1-cp312-cp312-win_arm64.whl", hash = "sha256:aea9c7224c302bc8bfc892b908537f56c430802560e827b75ecbde81b604598b", size = 58987 },
{ url = "https://files.pythonhosted.org/packages/ad/fe/41af4c46b5e498c90fc87981ab2972fbd9f0bccda597adb99d3d3441b94b/wrapt-2.0.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:47b0f8bafe90f7736151f61482c583c86b0693d80f075a58701dd1549b0010a9", size = 78132 },
{ url = "https://files.pythonhosted.org/packages/1c/92/d68895a984a5ebbbfb175512b0c0aad872354a4a2484fbd5552e9f275316/wrapt-2.0.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:cbeb0971e13b4bd81d34169ed57a6dda017328d1a22b62fda45e1d21dd06148f", size = 61211 },
{ url = "https://files.pythonhosted.org/packages/e8/26/ba83dc5ae7cf5aa2b02364a3d9cf74374b86169906a1f3ade9a2d03cf21c/wrapt-2.0.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:eb7cffe572ad0a141a7886a1d2efa5bef0bf7fe021deeea76b3ab334d2c38218", size = 61689 },
{ url = "https://files.pythonhosted.org/packages/cf/67/d7a7c276d874e5d26738c22444d466a3a64ed541f6ef35f740dbd865bab4/wrapt-2.0.1-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:c8d60527d1ecfc131426b10d93ab5d53e08a09c5fa0175f6b21b3252080c70a9", size = 121502 },
{ url = "https://files.pythonhosted.org/packages/0f/6b/806dbf6dd9579556aab22fc92908a876636e250f063f71548a8660382184/wrapt-2.0.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c654eafb01afac55246053d67a4b9a984a3567c3808bb7df2f8de1c1caba2e1c", size = 123110 },
{ url = "https://files.pythonhosted.org/packages/e5/08/cdbb965fbe4c02c5233d185d070cabed2ecc1f1e47662854f95d77613f57/wrapt-2.0.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:98d873ed6c8b4ee2418f7afce666751854d6d03e3c0ec2a399bb039cd2ae89db", size = 117434 },
{ url = "https://files.pythonhosted.org/packages/2d/d1/6aae2ce39db4cb5216302fa2e9577ad74424dfbe315bd6669725569e048c/wrapt-2.0.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:c9e850f5b7fc67af856ff054c71690d54fa940c3ef74209ad9f935b4f66a0233", size = 121533 },
{ url = "https://files.pythonhosted.org/packages/79/35/565abf57559fbe0a9155c29879ff43ce8bd28d2ca61033a3a3dd67b70794/wrapt-2.0.1-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:e505629359cb5f751e16e30cf3f91a1d3ddb4552480c205947da415d597f7ac2", size = 116324 },
{ url = "https://files.pythonhosted.org/packages/e1/e0/53ff5e76587822ee33e560ad55876d858e384158272cd9947abdd4ad42ca/wrapt-2.0.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:2879af909312d0baf35f08edeea918ee3af7ab57c37fe47cb6a373c9f2749c7b", size = 120627 },
{ url = "https://files.pythonhosted.org/packages/7c/7b/38df30fd629fbd7612c407643c63e80e1c60bcc982e30ceeae163a9800e7/wrapt-2.0.1-cp313-cp313-win32.whl", hash = "sha256:d67956c676be5a24102c7407a71f4126d30de2a569a1c7871c9f3cabc94225d7", size = 58252 },
{ url = "https://files.pythonhosted.org/packages/85/64/d3954e836ea67c4d3ad5285e5c8fd9d362fd0a189a2db622df457b0f4f6a/wrapt-2.0.1-cp313-cp313-win_amd64.whl", hash = "sha256:9ca66b38dd642bf90c59b6738af8070747b610115a39af2498535f62b5cdc1c3", size = 60500 },
{ url = "https://files.pythonhosted.org/packages/89/4e/3c8b99ac93527cfab7f116089db120fef16aac96e5f6cdb724ddf286086d/wrapt-2.0.1-cp313-cp313-win_arm64.whl", hash = "sha256:5a4939eae35db6b6cec8e7aa0e833dcca0acad8231672c26c2a9ab7a0f8ac9c8", size = 58993 },
{ url = "https://files.pythonhosted.org/packages/f9/f4/eff2b7d711cae20d220780b9300faa05558660afb93f2ff5db61fe725b9a/wrapt-2.0.1-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:a52f93d95c8d38fed0669da2ebdb0b0376e895d84596a976c15a9eb45e3eccb3", size = 82028 },
{ url = "https://files.pythonhosted.org/packages/0c/67/cb945563f66fd0f61a999339460d950f4735c69f18f0a87ca586319b1778/wrapt-2.0.1-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:4e54bbf554ee29fcceee24fa41c4d091398b911da6e7f5d7bffda963c9aed2e1", size = 62949 },
{ url = "https://files.pythonhosted.org/packages/ec/ca/f63e177f0bbe1e5cf5e8d9b74a286537cd709724384ff20860f8f6065904/wrapt-2.0.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:908f8c6c71557f4deaa280f55d0728c3bca0960e8c3dd5ceeeafb3c19942719d", size = 63681 },
{ url = "https://files.pythonhosted.org/packages/39/a1/1b88fcd21fd835dca48b556daef750952e917a2794fa20c025489e2e1f0f/wrapt-2.0.1-cp313-cp313t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:e2f84e9af2060e3904a32cea9bb6db23ce3f91cfd90c6b426757cf7cc01c45c7", size = 152696 },
{ url = "https://files.pythonhosted.org/packages/62/1c/d9185500c1960d9f5f77b9c0b890b7fc62282b53af7ad1b6bd779157f714/wrapt-2.0.1-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e3612dc06b436968dfb9142c62e5dfa9eb5924f91120b3c8ff501ad878f90eb3", size = 158859 },
{ url = "https://files.pythonhosted.org/packages/91/60/5d796ed0f481ec003220c7878a1d6894652efe089853a208ea0838c13086/wrapt-2.0.1-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:6d2d947d266d99a1477cd005b23cbd09465276e302515e122df56bb9511aca1b", size = 146068 },
{ url = "https://files.pythonhosted.org/packages/04/f8/75282dd72f102ddbfba137e1e15ecba47b40acff32c08ae97edbf53f469e/wrapt-2.0.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:7d539241e87b650cbc4c3ac9f32c8d1ac8a54e510f6dca3f6ab60dcfd48c9b10", size = 155724 },
{ url = "https://files.pythonhosted.org/packages/5a/27/fe39c51d1b344caebb4a6a9372157bdb8d25b194b3561b52c8ffc40ac7d1/wrapt-2.0.1-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:4811e15d88ee62dbf5c77f2c3ff3932b1e3ac92323ba3912f51fc4016ce81ecf", size = 144413 },
{ url = "https://files.pythonhosted.org/packages/83/2b/9f6b643fe39d4505c7bf926d7c2595b7cb4b607c8c6b500e56c6b36ac238/wrapt-2.0.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:c1c91405fcf1d501fa5d55df21e58ea49e6b879ae829f1039faaf7e5e509b41e", size = 150325 },
{ url = "https://files.pythonhosted.org/packages/bb/b6/20ffcf2558596a7f58a2e69c89597128781f0b88e124bf5a4cadc05b8139/wrapt-2.0.1-cp313-cp313t-win32.whl", hash = "sha256:e76e3f91f864e89db8b8d2a8311d57df93f01ad6bb1e9b9976d1f2e83e18315c", size = 59943 },
{ url = "https://files.pythonhosted.org/packages/87/6a/0e56111cbb3320151eed5d3821ee1373be13e05b376ea0870711f18810c3/wrapt-2.0.1-cp313-cp313t-win_amd64.whl", hash = "sha256:83ce30937f0ba0d28818807b303a412440c4b63e39d3d8fc036a94764b728c92", size = 63240 },
{ url = "https://files.pythonhosted.org/packages/1d/54/5ab4c53ea1f7f7e5c3e7c1095db92932cc32fd62359d285486d00c2884c3/wrapt-2.0.1-cp313-cp313t-win_arm64.whl", hash = "sha256:4b55cacc57e1dc2d0991dbe74c6419ffd415fb66474a02335cb10efd1aa3f84f", size = 60416 },
{ url = "https://files.pythonhosted.org/packages/73/81/d08d83c102709258e7730d3cd25befd114c60e43ef3891d7e6877971c514/wrapt-2.0.1-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:5e53b428f65ece6d9dad23cb87e64506392b720a0b45076c05354d27a13351a1", size = 78290 },
{ url = "https://files.pythonhosted.org/packages/f6/14/393afba2abb65677f313aa680ff0981e829626fed39b6a7e3ec807487790/wrapt-2.0.1-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:ad3ee9d0f254851c71780966eb417ef8e72117155cff04821ab9b60549694a55", size = 61255 },
{ url = "https://files.pythonhosted.org/packages/c4/10/a4a1f2fba205a9462e36e708ba37e5ac95f4987a0f1f8fd23f0bf1fc3b0f/wrapt-2.0.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:d7b822c61ed04ee6ad64bc90d13368ad6eb094db54883b5dde2182f67a7f22c0", size = 61797 },
{ url = "https://files.pythonhosted.org/packages/12/db/99ba5c37cf1c4fad35349174f1e38bd8d992340afc1ff27f526729b98986/wrapt-2.0.1-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:7164a55f5e83a9a0b031d3ffab4d4e36bbec42e7025db560f225489fa929e509", size = 120470 },
{ url = "https://files.pythonhosted.org/packages/30/3f/a1c8d2411eb826d695fc3395a431757331582907a0ec59afce8fe8712473/wrapt-2.0.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e60690ba71a57424c8d9ff28f8d006b7ad7772c22a4af432188572cd7fa004a1", size = 122851 },
{ url = "https://files.pythonhosted.org/packages/b3/8d/72c74a63f201768d6a04a8845c7976f86be6f5ff4d74996c272cefc8dafc/wrapt-2.0.1-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3cd1a4bd9a7a619922a8557e1318232e7269b5fb69d4ba97b04d20450a6bf970", size = 117433 },
{ url = "https://files.pythonhosted.org/packages/c7/5a/df37cf4042cb13b08256f8e27023e2f9b3d471d553376616591bb99bcb31/wrapt-2.0.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:b4c2e3d777e38e913b8ce3a6257af72fb608f86a1df471cb1d4339755d0a807c", size = 121280 },
{ url = "https://files.pythonhosted.org/packages/54/34/40d6bc89349f9931e1186ceb3e5fbd61d307fef814f09fbbac98ada6a0c8/wrapt-2.0.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:3d366aa598d69416b5afedf1faa539fac40c1d80a42f6b236c88c73a3c8f2d41", size = 116343 },
{ url = "https://files.pythonhosted.org/packages/70/66/81c3461adece09d20781dee17c2366fdf0cb8754738b521d221ca056d596/wrapt-2.0.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:c235095d6d090aa903f1db61f892fffb779c1eaeb2a50e566b52001f7a0f66ed", size = 119650 },
{ url = "https://files.pythonhosted.org/packages/46/3a/d0146db8be8761a9e388cc9cc1c312b36d583950ec91696f19bbbb44af5a/wrapt-2.0.1-cp314-cp314-win32.whl", hash = "sha256:bfb5539005259f8127ea9c885bdc231978c06b7a980e63a8a61c8c4c979719d0", size = 58701 },
{ url = "https://files.pythonhosted.org/packages/1a/38/5359da9af7d64554be63e9046164bd4d8ff289a2dd365677d25ba3342c08/wrapt-2.0.1-cp314-cp314-win_amd64.whl", hash = "sha256:4ae879acc449caa9ed43fc36ba08392b9412ee67941748d31d94e3cedb36628c", size = 60947 },
{ url = "https://files.pythonhosted.org/packages/aa/3f/96db0619276a833842bf36343685fa04f987dd6e3037f314531a1e00492b/wrapt-2.0.1-cp314-cp314-win_arm64.whl", hash = "sha256:8639b843c9efd84675f1e100ed9e99538ebea7297b62c4b45a7042edb84db03e", size = 59359 },
{ url = "https://files.pythonhosted.org/packages/71/49/5f5d1e867bf2064bf3933bc6cf36ade23505f3902390e175e392173d36a2/wrapt-2.0.1-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:9219a1d946a9b32bb23ccae66bdb61e35c62773ce7ca6509ceea70f344656b7b", size = 82031 },
{ url = "https://files.pythonhosted.org/packages/2b/89/0009a218d88db66ceb83921e5685e820e2c61b59bbbb1324ba65342668bc/wrapt-2.0.1-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:fa4184e74197af3adad3c889a1af95b53bb0466bced92ea99a0c014e48323eec", size = 62952 },
{ url = "https://files.pythonhosted.org/packages/ae/18/9b968e920dd05d6e44bcc918a046d02afea0fb31b2f1c80ee4020f377cbe/wrapt-2.0.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c5ef2f2b8a53b7caee2f797ef166a390fef73979b15778a4a153e4b5fedce8fa", size = 63688 },
{ url = "https://files.pythonhosted.org/packages/a6/7d/78bdcb75826725885d9ea26c49a03071b10c4c92da93edda612910f150e4/wrapt-2.0.1-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:e042d653a4745be832d5aa190ff80ee4f02c34b21f4b785745eceacd0907b815", size = 152706 },
{ url = "https://files.pythonhosted.org/packages/dd/77/cac1d46f47d32084a703df0d2d29d47e7eb2a7d19fa5cbca0e529ef57659/wrapt-2.0.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2afa23318136709c4b23d87d543b425c399887b4057936cd20386d5b1422b6fa", size = 158866 },
{ url = "https://files.pythonhosted.org/packages/8a/11/b521406daa2421508903bf8d5e8b929216ec2af04839db31c0a2c525eee0/wrapt-2.0.1-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:6c72328f668cf4c503ffcf9434c2b71fdd624345ced7941bc6693e61bbe36bef", size = 146148 },
{ url = "https://files.pythonhosted.org/packages/0c/c0/340b272bed297baa7c9ce0c98ef7017d9c035a17a6a71dce3184b8382da2/wrapt-2.0.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:3793ac154afb0e5b45d1233cb94d354ef7a983708cc3bb12563853b1d8d53747", size = 155737 },
{ url = "https://files.pythonhosted.org/packages/f3/93/bfcb1fb2bdf186e9c2883a4d1ab45ab099c79cbf8f4e70ea453811fa3ea7/wrapt-2.0.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:fec0d993ecba3991645b4857837277469c8cc4c554a7e24d064d1ca291cfb81f", size = 144451 },
{ url = "https://files.pythonhosted.org/packages/d2/6b/dca504fb18d971139d232652656180e3bd57120e1193d9a5899c3c0b7cdd/wrapt-2.0.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:949520bccc1fa227274da7d03bf238be15389cd94e32e4297b92337df9b7a349", size = 150353 },
{ url = "https://files.pythonhosted.org/packages/1d/f6/a1de4bd3653afdf91d250ca5c721ee51195df2b61a4603d4b373aa804d1d/wrapt-2.0.1-cp314-cp314t-win32.whl", hash = "sha256:be9e84e91d6497ba62594158d3d31ec0486c60055c49179edc51ee43d095f79c", size = 60609 },
{ url = "https://files.pythonhosted.org/packages/01/3a/07cd60a9d26fe73efead61c7830af975dfdba8537632d410462672e4432b/wrapt-2.0.1-cp314-cp314t-win_amd64.whl", hash = "sha256:61c4956171c7434634401db448371277d07032a81cc21c599c22953374781395", size = 64038 },
{ url = "https://files.pythonhosted.org/packages/41/99/8a06b8e17dddbf321325ae4eb12465804120f699cd1b8a355718300c62da/wrapt-2.0.1-cp314-cp314t-win_arm64.whl", hash = "sha256:35cdbd478607036fee40273be8ed54a451f5f23121bd9d4be515158f9498f7ad", size = 60634 },
{ url = "https://files.pythonhosted.org/packages/15/d1/b51471c11592ff9c012bd3e2f7334a6ff2f42a7aed2caffcf0bdddc9cb89/wrapt-2.0.1-py3-none-any.whl", hash = "sha256:4d2ce1bf1a48c5277d7969259232b57645aae5686dba1eaeade39442277afbca", size = 44046 },
{ url = "https://files.pythonhosted.org/packages/52/db/00e2a219213856074a213503fdac0511203dceefff26e1daa15250cc01a0/wrapt-1.17.3-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:273a736c4645e63ac582c60a56b0acb529ef07f78e08dc6bfadf6a46b19c0da7", size = 53482 },
{ url = "https://files.pythonhosted.org/packages/5e/30/ca3c4a5eba478408572096fe9ce36e6e915994dd26a4e9e98b4f729c06d9/wrapt-1.17.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:5531d911795e3f935a9c23eb1c8c03c211661a5060aab167065896bbf62a5f85", size = 38674 },
{ url = "https://files.pythonhosted.org/packages/31/25/3e8cc2c46b5329c5957cec959cb76a10718e1a513309c31399a4dad07eb3/wrapt-1.17.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:0610b46293c59a3adbae3dee552b648b984176f8562ee0dba099a56cfbe4df1f", size = 38959 },
{ url = "https://files.pythonhosted.org/packages/5d/8f/a32a99fc03e4b37e31b57cb9cefc65050ea08147a8ce12f288616b05ef54/wrapt-1.17.3-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b32888aad8b6e68f83a8fdccbf3165f5469702a7544472bdf41f582970ed3311", size = 82376 },
{ url = "https://files.pythonhosted.org/packages/31/57/4930cb8d9d70d59c27ee1332a318c20291749b4fba31f113c2f8ac49a72e/wrapt-1.17.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8cccf4f81371f257440c88faed6b74f1053eef90807b77e31ca057b2db74edb1", size = 83604 },
{ url = "https://files.pythonhosted.org/packages/a8/f3/1afd48de81d63dd66e01b263a6fbb86e1b5053b419b9b33d13e1f6d0f7d0/wrapt-1.17.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d8a210b158a34164de8bb68b0e7780041a903d7b00c87e906fb69928bf7890d5", size = 82782 },
{ url = "https://files.pythonhosted.org/packages/1e/d7/4ad5327612173b144998232f98a85bb24b60c352afb73bc48e3e0d2bdc4e/wrapt-1.17.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:79573c24a46ce11aab457b472efd8d125e5a51da2d1d24387666cd85f54c05b2", size = 82076 },
{ url = "https://files.pythonhosted.org/packages/bb/59/e0adfc831674a65694f18ea6dc821f9fcb9ec82c2ce7e3d73a88ba2e8718/wrapt-1.17.3-cp311-cp311-win32.whl", hash = "sha256:c31eebe420a9a5d2887b13000b043ff6ca27c452a9a22fa71f35f118e8d4bf89", size = 36457 },
{ url = "https://files.pythonhosted.org/packages/83/88/16b7231ba49861b6f75fc309b11012ede4d6b0a9c90969d9e0db8d991aeb/wrapt-1.17.3-cp311-cp311-win_amd64.whl", hash = "sha256:0b1831115c97f0663cb77aa27d381237e73ad4f721391a9bfb2fe8bc25fa6e77", size = 38745 },
{ url = "https://files.pythonhosted.org/packages/9a/1e/c4d4f3398ec073012c51d1c8d87f715f56765444e1a4b11e5180577b7e6e/wrapt-1.17.3-cp311-cp311-win_arm64.whl", hash = "sha256:5a7b3c1ee8265eb4c8f1b7d29943f195c00673f5ab60c192eba2d4a7eae5f46a", size = 36806 },
{ url = "https://files.pythonhosted.org/packages/9f/41/cad1aba93e752f1f9268c77270da3c469883d56e2798e7df6240dcb2287b/wrapt-1.17.3-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:ab232e7fdb44cdfbf55fc3afa31bcdb0d8980b9b95c38b6405df2acb672af0e0", size = 53998 },
{ url = "https://files.pythonhosted.org/packages/60/f8/096a7cc13097a1869fe44efe68dace40d2a16ecb853141394047f0780b96/wrapt-1.17.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:9baa544e6acc91130e926e8c802a17f3b16fbea0fd441b5a60f5cf2cc5c3deba", size = 39020 },
{ url = "https://files.pythonhosted.org/packages/33/df/bdf864b8997aab4febb96a9ae5c124f700a5abd9b5e13d2a3214ec4be705/wrapt-1.17.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:6b538e31eca1a7ea4605e44f81a48aa24c4632a277431a6ed3f328835901f4fd", size = 39098 },
{ url = "https://files.pythonhosted.org/packages/9f/81/5d931d78d0eb732b95dc3ddaeeb71c8bb572fb01356e9133916cd729ecdd/wrapt-1.17.3-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:042ec3bb8f319c147b1301f2393bc19dba6e176b7da446853406d041c36c7828", size = 88036 },
{ url = "https://files.pythonhosted.org/packages/ca/38/2e1785df03b3d72d34fc6252d91d9d12dc27a5c89caef3335a1bbb8908ca/wrapt-1.17.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3af60380ba0b7b5aeb329bc4e402acd25bd877e98b3727b0135cb5c2efdaefe9", size = 88156 },
{ url = "https://files.pythonhosted.org/packages/b3/8b/48cdb60fe0603e34e05cffda0b2a4adab81fd43718e11111a4b0100fd7c1/wrapt-1.17.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:0b02e424deef65c9f7326d8c19220a2c9040c51dc165cddb732f16198c168396", size = 87102 },
{ url = "https://files.pythonhosted.org/packages/3c/51/d81abca783b58f40a154f1b2c56db1d2d9e0d04fa2d4224e357529f57a57/wrapt-1.17.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:74afa28374a3c3a11b3b5e5fca0ae03bef8450d6aa3ab3a1e2c30e3a75d023dc", size = 87732 },
{ url = "https://files.pythonhosted.org/packages/9e/b1/43b286ca1392a006d5336412d41663eeef1ad57485f3e52c767376ba7e5a/wrapt-1.17.3-cp312-cp312-win32.whl", hash = "sha256:4da9f45279fff3543c371d5ababc57a0384f70be244de7759c85a7f989cb4ebe", size = 36705 },
{ url = "https://files.pythonhosted.org/packages/28/de/49493f962bd3c586ab4b88066e967aa2e0703d6ef2c43aa28cb83bf7b507/wrapt-1.17.3-cp312-cp312-win_amd64.whl", hash = "sha256:e71d5c6ebac14875668a1e90baf2ea0ef5b7ac7918355850c0908ae82bcb297c", size = 38877 },
{ url = "https://files.pythonhosted.org/packages/f1/48/0f7102fe9cb1e8a5a77f80d4f0956d62d97034bbe88d33e94699f99d181d/wrapt-1.17.3-cp312-cp312-win_arm64.whl", hash = "sha256:604d076c55e2fdd4c1c03d06dc1a31b95130010517b5019db15365ec4a405fc6", size = 36885 },
{ url = "https://files.pythonhosted.org/packages/fc/f6/759ece88472157acb55fc195e5b116e06730f1b651b5b314c66291729193/wrapt-1.17.3-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:a47681378a0439215912ef542c45a783484d4dd82bac412b71e59cf9c0e1cea0", size = 54003 },
{ url = "https://files.pythonhosted.org/packages/4f/a9/49940b9dc6d47027dc850c116d79b4155f15c08547d04db0f07121499347/wrapt-1.17.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:54a30837587c6ee3cd1a4d1c2ec5d24e77984d44e2f34547e2323ddb4e22eb77", size = 39025 },
{ url = "https://files.pythonhosted.org/packages/45/35/6a08de0f2c96dcdd7fe464d7420ddb9a7655a6561150e5fc4da9356aeaab/wrapt-1.17.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:16ecf15d6af39246fe33e507105d67e4b81d8f8d2c6598ff7e3ca1b8a37213f7", size = 39108 },
{ url = "https://files.pythonhosted.org/packages/0c/37/6faf15cfa41bf1f3dba80cd3f5ccc6622dfccb660ab26ed79f0178c7497f/wrapt-1.17.3-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:6fd1ad24dc235e4ab88cda009e19bf347aabb975e44fd5c2fb22a3f6e4141277", size = 88072 },
{ url = "https://files.pythonhosted.org/packages/78/f2/efe19ada4a38e4e15b6dff39c3e3f3f73f5decf901f66e6f72fe79623a06/wrapt-1.17.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0ed61b7c2d49cee3c027372df5809a59d60cf1b6c2f81ee980a091f3afed6a2d", size = 88214 },
{ url = "https://files.pythonhosted.org/packages/40/90/ca86701e9de1622b16e09689fc24b76f69b06bb0150990f6f4e8b0eeb576/wrapt-1.17.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:423ed5420ad5f5529db9ce89eac09c8a2f97da18eb1c870237e84c5a5c2d60aa", size = 87105 },
{ url = "https://files.pythonhosted.org/packages/fd/e0/d10bd257c9a3e15cbf5523025252cc14d77468e8ed644aafb2d6f54cb95d/wrapt-1.17.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:e01375f275f010fcbf7f643b4279896d04e571889b8a5b3f848423d91bf07050", size = 87766 },
{ url = "https://files.pythonhosted.org/packages/e8/cf/7d848740203c7b4b27eb55dbfede11aca974a51c3d894f6cc4b865f42f58/wrapt-1.17.3-cp313-cp313-win32.whl", hash = "sha256:53e5e39ff71b3fc484df8a522c933ea2b7cdd0d5d15ae82e5b23fde87d44cbd8", size = 36711 },
{ url = "https://files.pythonhosted.org/packages/57/54/35a84d0a4d23ea675994104e667ceff49227ce473ba6a59ba2c84f250b74/wrapt-1.17.3-cp313-cp313-win_amd64.whl", hash = "sha256:1f0b2f40cf341ee8cc1a97d51ff50dddb9fcc73241b9143ec74b30fc4f44f6cb", size = 38885 },
{ url = "https://files.pythonhosted.org/packages/01/77/66e54407c59d7b02a3c4e0af3783168fff8e5d61def52cda8728439d86bc/wrapt-1.17.3-cp313-cp313-win_arm64.whl", hash = "sha256:7425ac3c54430f5fc5e7b6f41d41e704db073309acfc09305816bc6a0b26bb16", size = 36896 },
{ url = "https://files.pythonhosted.org/packages/02/a2/cd864b2a14f20d14f4c496fab97802001560f9f41554eef6df201cd7f76c/wrapt-1.17.3-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:cf30f6e3c077c8e6a9a7809c94551203c8843e74ba0c960f4a98cd80d4665d39", size = 54132 },
{ url = "https://files.pythonhosted.org/packages/d5/46/d011725b0c89e853dc44cceb738a307cde5d240d023d6d40a82d1b4e1182/wrapt-1.17.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:e228514a06843cae89621384cfe3a80418f3c04aadf8a3b14e46a7be704e4235", size = 39091 },
{ url = "https://files.pythonhosted.org/packages/2e/9e/3ad852d77c35aae7ddebdbc3b6d35ec8013af7d7dddad0ad911f3d891dae/wrapt-1.17.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:5ea5eb3c0c071862997d6f3e02af1d055f381b1d25b286b9d6644b79db77657c", size = 39172 },
{ url = "https://files.pythonhosted.org/packages/c3/f7/c983d2762bcce2326c317c26a6a1e7016f7eb039c27cdf5c4e30f4160f31/wrapt-1.17.3-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:281262213373b6d5e4bb4353bc36d1ba4084e6d6b5d242863721ef2bf2c2930b", size = 87163 },
{ url = "https://files.pythonhosted.org/packages/e4/0f/f673f75d489c7f22d17fe0193e84b41540d962f75fce579cf6873167c29b/wrapt-1.17.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:dc4a8d2b25efb6681ecacad42fca8859f88092d8732b170de6a5dddd80a1c8fa", size = 87963 },
{ url = "https://files.pythonhosted.org/packages/df/61/515ad6caca68995da2fac7a6af97faab8f78ebe3bf4f761e1b77efbc47b5/wrapt-1.17.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:373342dd05b1d07d752cecbec0c41817231f29f3a89aa8b8843f7b95992ed0c7", size = 86945 },
{ url = "https://files.pythonhosted.org/packages/d3/bd/4e70162ce398462a467bc09e768bee112f1412e563620adc353de9055d33/wrapt-1.17.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:d40770d7c0fd5cbed9d84b2c3f2e156431a12c9a37dc6284060fb4bec0b7ffd4", size = 86857 },
{ url = "https://files.pythonhosted.org/packages/2b/b8/da8560695e9284810b8d3df8a19396a6e40e7518059584a1a394a2b35e0a/wrapt-1.17.3-cp314-cp314-win32.whl", hash = "sha256:fbd3c8319de8e1dc79d346929cd71d523622da527cca14e0c1d257e31c2b8b10", size = 37178 },
{ url = "https://files.pythonhosted.org/packages/db/c8/b71eeb192c440d67a5a0449aaee2310a1a1e8eca41676046f99ed2487e9f/wrapt-1.17.3-cp314-cp314-win_amd64.whl", hash = "sha256:e1a4120ae5705f673727d3253de3ed0e016f7cd78dc463db1b31e2463e1f3cf6", size = 39310 },
{ url = "https://files.pythonhosted.org/packages/45/20/2cda20fd4865fa40f86f6c46ed37a2a8356a7a2fde0773269311f2af56c7/wrapt-1.17.3-cp314-cp314-win_arm64.whl", hash = "sha256:507553480670cab08a800b9463bdb881b2edeed77dc677b0a5915e6106e91a58", size = 37266 },
{ url = "https://files.pythonhosted.org/packages/77/ed/dd5cf21aec36c80443c6f900449260b80e2a65cf963668eaef3b9accce36/wrapt-1.17.3-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:ed7c635ae45cfbc1a7371f708727bf74690daedc49b4dba310590ca0bd28aa8a", size = 56544 },
{ url = "https://files.pythonhosted.org/packages/8d/96/450c651cc753877ad100c7949ab4d2e2ecc4d97157e00fa8f45df682456a/wrapt-1.17.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:249f88ed15503f6492a71f01442abddd73856a0032ae860de6d75ca62eed8067", size = 40283 },
{ url = "https://files.pythonhosted.org/packages/d1/86/2fcad95994d9b572db57632acb6f900695a648c3e063f2cd344b3f5c5a37/wrapt-1.17.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5a03a38adec8066d5a37bea22f2ba6bbf39fcdefbe2d91419ab864c3fb515454", size = 40366 },
{ url = "https://files.pythonhosted.org/packages/64/0e/f4472f2fdde2d4617975144311f8800ef73677a159be7fe61fa50997d6c0/wrapt-1.17.3-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:5d4478d72eb61c36e5b446e375bbc49ed002430d17cdec3cecb36993398e1a9e", size = 108571 },
{ url = "https://files.pythonhosted.org/packages/cc/01/9b85a99996b0a97c8a17484684f206cbb6ba73c1ce6890ac668bcf3838fb/wrapt-1.17.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:223db574bb38637e8230eb14b185565023ab624474df94d2af18f1cdb625216f", size = 113094 },
{ url = "https://files.pythonhosted.org/packages/25/02/78926c1efddcc7b3aa0bc3d6b33a822f7d898059f7cd9ace8c8318e559ef/wrapt-1.17.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e405adefb53a435f01efa7ccdec012c016b5a1d3f35459990afc39b6be4d5056", size = 110659 },
{ url = "https://files.pythonhosted.org/packages/dc/ee/c414501ad518ac3e6fe184753632fe5e5ecacdcf0effc23f31c1e4f7bfcf/wrapt-1.17.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:88547535b787a6c9ce4086917b6e1d291aa8ed914fdd3a838b3539dc95c12804", size = 106946 },
{ url = "https://files.pythonhosted.org/packages/be/44/a1bd64b723d13bb151d6cc91b986146a1952385e0392a78567e12149c7b4/wrapt-1.17.3-cp314-cp314t-win32.whl", hash = "sha256:41b1d2bc74c2cac6f9074df52b2efbef2b30bdfe5f40cb78f8ca22963bc62977", size = 38717 },
{ url = "https://files.pythonhosted.org/packages/79/d9/7cfd5a312760ac4dd8bf0184a6ee9e43c33e47f3dadc303032ce012b8fa3/wrapt-1.17.3-cp314-cp314t-win_amd64.whl", hash = "sha256:73d496de46cd2cdbdbcce4ae4bcdb4afb6a11234a1df9c085249d55166b95116", size = 41334 },
{ url = "https://files.pythonhosted.org/packages/46/78/10ad9781128ed2f99dbc474f43283b13fea8ba58723e98844367531c18e9/wrapt-1.17.3-cp314-cp314t-win_arm64.whl", hash = "sha256:f38e60678850c42461d4202739f9bf1e3a737c7ad283638251e79cc49effb6b6", size = 38471 },
{ url = "https://files.pythonhosted.org/packages/1f/f6/a933bd70f98e9cf3e08167fc5cd7aaaca49147e48411c0bd5ae701bb2194/wrapt-1.17.3-py3-none-any.whl", hash = "sha256:7171ae35d2c33d326ac19dd8facb1e82e5fd04ef8c6c0e394d7af55a55051c22", size = 23591 },
]
[[package]]