Add RAG vs Hindsight examples (#15)
This commit is contained in:
Derek Bouius
GitHub
parent
e96cb9694a
commit
bbfdcd36e4
2 changed files with 234 additions and 3 deletions
|
|
@ -22,14 +22,14 @@ Hindsight solves these problems with a memory system designed specifically for A
|
|||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph app[Your Application]
|
||||
subgraph app["<b>Your Application</b>"]
|
||||
Agent[AI Agent]
|
||||
end
|
||||
|
||||
subgraph hindsight[Hindsight]
|
||||
subgraph hindsight["<b>Hindsight</b>"]
|
||||
API[API Server]
|
||||
|
||||
subgraph bank[Memory Bank]
|
||||
subgraph bank["<b>Memory Bank</b>"]
|
||||
Documents[Documents]
|
||||
Memories[Memories]
|
||||
Entities[Entities]
|
||||
|
|
@ -104,6 +104,7 @@ The `bias_strength` parameter (0-1) controls how much personality influences opi
|
|||
|
||||
### Getting Started
|
||||
- [**Quick Start**](/developer/api/quickstart) — Install and get up and running in 60 seconds
|
||||
- [**RAG vs Hindsight**](/developer/rag-vs-hindsight) — See how Hindsight differs from traditional RAG with real examples
|
||||
|
||||
### Core Concepts
|
||||
- [**Retain**](/developer/retain) — How memories are stored with multi-dimensional facts
|
||||
|
|
|
|||
230
hindsight-docs/docs/developer/rag-vs-hindsight.md
Normal file
230
hindsight-docs/docs/developer/rag-vs-hindsight.md
Normal file
|
|
@ -0,0 +1,230 @@
|
|||
---
|
||||
sidebar_position: 2
|
||||
---
|
||||
|
||||
# RAG vs Hindsight: Why Your AI Needs Real Memory
|
||||
|
||||
Traditional RAG (Retrieval-Augmented Generation) finds documents similar to your query. Hindsight gives your AI actual memory—with temporal reasoning, entity understanding, and evolving beliefs.
|
||||
|
||||
Here's what that difference looks like in practice.
|
||||
|
||||
---
|
||||
|
||||
## Example 1: Multi-Hop Reasoning
|
||||
|
||||
**Scenario:** Your AI assistant has learned these facts over time:
|
||||
- "Alice is the tech lead on Project Atlas"
|
||||
- "Project Atlas uses Kubernetes for deployment"
|
||||
- "The Kubernetes cluster had an outage last Tuesday"
|
||||
|
||||
**User asks:** "Was Alice affected by any recent issues?"
|
||||
|
||||
### RAG Response
|
||||
> I found information about Alice being a tech lead, but I don't have specific information about recent issues affecting her.
|
||||
|
||||
**Why it fails:** RAG only finds facts semantically similar to "Alice" + "recent issues". It can't connect Alice → Project Atlas → Kubernetes → outage because those facts exist in different semantic spaces.
|
||||
|
||||
### Hindsight Response
|
||||
> Yes, Alice was likely affected. She's the tech lead on Project Atlas, which uses Kubernetes for deployment. The Kubernetes cluster had an outage last Tuesday, which would have impacted her team's deployments.
|
||||
|
||||
**Why it works:** Hindsight's knowledge graph links Alice → Project Atlas → Kubernetes → outage through entity relationships. Graph traversal discovers the connection even though no single fact mentions both Alice and the outage.
|
||||
|
||||
---
|
||||
|
||||
## Example 2: Temporal Reasoning
|
||||
|
||||
**Scenario:** An AI assistant has accumulated months of conversation history:
|
||||
- March: "Alice started the migration to microservices"
|
||||
- April: "Alice completed the authentication service"
|
||||
- May: "Alice presented the architecture review"
|
||||
- October: "Alice is now focusing on performance optimization"
|
||||
|
||||
**User asks:** "What did Alice accomplish last spring?"
|
||||
|
||||
### RAG Response
|
||||
> Alice has worked on microservices migration, authentication service, architecture review, and performance optimization.
|
||||
|
||||
**Why it fails:** RAG treats "last spring" as keywords, returning ALL mentions of Alice regardless of when they occurred. No temporal filtering.
|
||||
|
||||
### Hindsight Response
|
||||
> Last spring (March-May), Alice started the migration to microservices, completed the authentication service, and presented the architecture review. She's since moved on to performance optimization.
|
||||
|
||||
**Why it works:** Hindsight parses "last spring" into a date range (March-May), then retrieves only facts with timestamps in that window. The temporal-graph strategy ensures historical queries return chronologically accurate results.
|
||||
|
||||
---
|
||||
|
||||
## Example 3: Consistent Personality
|
||||
|
||||
**Scenario:** Two AI assistants have the same facts about remote work:
|
||||
- "Remote work saves 1 hour of commute daily"
|
||||
- "65% of employees report higher productivity"
|
||||
- "Some teams report communication challenges"
|
||||
- "Office presence improves spontaneous collaboration"
|
||||
|
||||
**User asks:** "Should our company adopt remote work?"
|
||||
|
||||
### RAG Response (varies each time)
|
||||
> Remote work has benefits like saved commute time and higher productivity, but also challenges like communication issues. Consider your team's needs.
|
||||
|
||||
**Why it fails:** Generic, fence-sitting response. No consistent perspective. Ask again tomorrow, get a slightly different answer.
|
||||
|
||||
### Hindsight Response (Conservative assistant, high conscientiousness)
|
||||
> While the productivity data is encouraging, I'd recommend a cautious approach. The communication challenges and reduced spontaneous collaboration could impact team cohesion long-term. Consider a hybrid model that maintains some office presence for important collaborative work.
|
||||
|
||||
### Hindsight Response (Innovative assistant, high openness)
|
||||
> The data strongly supports remote work—65% productivity gains and an hour saved daily per employee. Communication challenges are solvable with the right tools. I'd embrace this as an opportunity to modernize how the team works.
|
||||
|
||||
**Why it works:** Each Hindsight bank has a personality (Big Five traits) that shapes how it interprets facts. Same data, consistent but different perspectives. The assistant develops opinions that persist and evolve over time.
|
||||
|
||||
---
|
||||
|
||||
## Example 4: Evolving Beliefs
|
||||
|
||||
**Scenario:** An AI coding assistant forms opinions based on experience.
|
||||
|
||||
**Week 1:**
|
||||
```python
|
||||
retain("User struggled with Python async/await syntax")
|
||||
retain("User successfully implemented the feature using threads")
|
||||
```
|
||||
*Assistant forms opinion:* "User prefers synchronous Python patterns" (confidence: 0.7)
|
||||
|
||||
**Week 3:**
|
||||
```python
|
||||
retain("User asked about asyncio best practices")
|
||||
retain("User implemented async database calls successfully")
|
||||
```
|
||||
*Opinion updated:* "User is growing comfortable with async patterns" (confidence: 0.6)
|
||||
|
||||
**User asks:** "Should I use async for this new API endpoint?"
|
||||
|
||||
### RAG Response
|
||||
> Async can improve performance for I/O-bound operations. Here's how to implement it...
|
||||
|
||||
**Why it fails:** No memory of user's journey. Same generic advice given to everyone.
|
||||
|
||||
### Hindsight Response
|
||||
> Given your recent success with async database calls, I'd recommend async for this endpoint too. You've gotten comfortable with the patterns—this is a good opportunity to apply them. Want me to show an approach similar to what worked in your database implementation?
|
||||
|
||||
**Why it works:** Hindsight tracked the user's evolving skill with async Python. Its opinion (with confidence score) updated as new evidence arrived. The response reflects accumulated understanding of this specific user's journey.
|
||||
|
||||
---
|
||||
|
||||
## Example 5: Entity Understanding
|
||||
|
||||
**Scenario:** A customer support AI learns about a user over multiple sessions:
|
||||
- Session 1: "I'm having trouble with my Pro subscription"
|
||||
- Session 2: "The mobile app crashes when I open settings"
|
||||
- Session 3: "I switched to annual billing last month"
|
||||
- Session 4: "The desktop app is working fine"
|
||||
|
||||
**User asks:** "What do you know about my account?"
|
||||
|
||||
### RAG Response
|
||||
> You've mentioned issues with Pro subscription, mobile app crashes in settings, switching to annual billing, and the desktop app working fine.
|
||||
|
||||
**Why it fails:** Returns disconnected facts in no particular structure. No synthesized understanding.
|
||||
|
||||
### Hindsight Response
|
||||
> You're on a Pro subscription with annual billing (switched last month). You've experienced some mobile app issues—specifically crashes in the settings menu—though your desktop app has been working fine. Would you like help troubleshooting the mobile app crashes?
|
||||
|
||||
**Why it works:** Hindsight automatically generates "observations"—synthesized summaries of entities (in this case, the user). Instead of listing disconnected facts, it presents a coherent picture and proactively offers relevant help.
|
||||
|
||||
---
|
||||
|
||||
## Example 6: Connecting Conversations Across Time
|
||||
|
||||
**Scenario:** A personal AI assistant has had conversations over several months:
|
||||
- January: "I'm thinking about learning guitar"
|
||||
- February: "I bought a used acoustic guitar"
|
||||
- March: "Practice is going well, learning basic chords"
|
||||
- June: "Work has been crazy, no time for hobbies"
|
||||
- September: "Feeling burned out, need something relaxing"
|
||||
|
||||
**User asks:** "What should I do to unwind this weekend?"
|
||||
|
||||
### RAG Response
|
||||
> Here are some popular relaxation activities: meditation, reading, nature walks, or picking up a creative hobby.
|
||||
|
||||
**Why it fails:** Generic suggestions. No awareness of the user's existing interests or history.
|
||||
|
||||
### Hindsight Response
|
||||
> You mentioned feeling burned out—have you picked up the guitar lately? You were making good progress with chords back in March before work got busy. Music could be a good way to unwind, and you already have the guitar.
|
||||
|
||||
**Why it works:** Hindsight connects the burnout complaint to the dormant hobby through entity and temporal reasoning. It remembers the user's journey: interest → purchase → progress → life interruption, and surfaces relevant context at the right moment.
|
||||
|
||||
---
|
||||
|
||||
## The Technical Difference
|
||||
|
||||
### How RAG Works
|
||||
```
|
||||
Query → Embed → Find similar vectors → Return top-k chunks
|
||||
```
|
||||
- One search strategy (semantic similarity)
|
||||
- No entity linking
|
||||
- No temporal understanding
|
||||
- No persistent beliefs
|
||||
- Stateless between queries
|
||||
|
||||
### How Hindsight Works
|
||||
```
|
||||
Query → Parse temporal expressions → Execute 4 parallel strategies:
|
||||
• Semantic search (like RAG)
|
||||
• Keyword search (catches exact names)
|
||||
• Graph traversal (entity/causal connections)
|
||||
• Temporal-graph (time-filtered relationships)
|
||||
→ Fuse results → Rerank → Apply personality → Generate response
|
||||
```
|
||||
- Four complementary search strategies
|
||||
- Knowledge graph with entity/temporal/causal links
|
||||
- Temporal parsing and filtering
|
||||
- Opinion formation and evolution
|
||||
- Consistent personality across sessions
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Capability | RAG | Hindsight |
|
||||
|------------|-----|-----------|
|
||||
| **Multi-hop reasoning** | Misses indirect connections | Graph traversal finds relationships |
|
||||
| **Temporal queries** | "Last spring" = keyword match | Parses dates, filters to time range |
|
||||
| **Personality** | Generic responses | Consistent character with Big Five traits |
|
||||
| **Learning** | Stateless | Opinions evolve with evidence |
|
||||
| **Entity understanding** | Disconnected facts | Synthesized mental models |
|
||||
| **Context** | Top-k similar chunks | Rich connections across all memory |
|
||||
|
||||
---
|
||||
|
||||
## Try It Yourself
|
||||
|
||||
```bash
|
||||
# Start Hindsight
|
||||
docker run -p 8888:8888 -p 9999:9999 \
|
||||
-e HINDSIGHT_API_LLM_PROVIDER=openai \
|
||||
-e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
|
||||
-e HINDSIGHT_API_LLM_MODEL=gpt-4o-mini \
|
||||
vectorize/hindsight
|
||||
```
|
||||
|
||||
```python
|
||||
from hindsight import HindsightClient
|
||||
|
||||
client = HindsightClient(base_url="http://localhost:8888")
|
||||
|
||||
# Store memories (not just chunks—rich facts with entities and timestamps)
|
||||
client.retain(bank_id="my-agent", content="Alice is the tech lead on Project Atlas")
|
||||
client.retain(bank_id="my-agent", content="Project Atlas uses Kubernetes for deployment")
|
||||
client.retain(bank_id="my-agent", content="The Kubernetes cluster had an outage last Tuesday")
|
||||
|
||||
# Query with understanding (not just similarity)
|
||||
response = client.reflect(
|
||||
bank_id="my-agent",
|
||||
query="Was Alice affected by any recent issues?"
|
||||
)
|
||||
print(response.text)
|
||||
# "Yes, Alice was likely affected. She's the tech lead on Project Atlas..."
|
||||
```
|
||||
|
||||
Your AI deserves real memory, not just search.
|
||||
Loading…
Reference in a new issue