b32767caa8
* feat: add max_observations_per_scope bank config Adds a configurable limit on the number of observations per tag scope. When the limit is reached, consolidation only updates/deletes existing observations — no new ones are created. Enforcement is done via a constrained Pydantic response model (max_length on creates list) so the LLM structurally cannot exceed the limit, plus prompt guidance. - Config: HINDSIGHT_API_MAX_OBSERVATIONS_PER_SCOPE (-1 = unlimited) - Reorder action execution: deletes → updates → creates - Dynamic _ConsolidationBatchResponse with max_length constraint - Prompt CAPACITY CONSTRAINT section when near/at limit - Observations with no tags skip the limit entirely - Control plane UI field + docs * fix: strengthen max_observations tests with mock LLM + defensive truncation - Rewrite integration tests to use MockLLM with deterministic responses (one observation per fact) instead of relying on real LLM behavior - Add defensive truncation in _consolidate_batch_with_llm as belt-and- suspenders — catches LLM providers that ignore JSON schema max_length - Tests now assert exact counts, not just upper bounds
90 lines
5.2 KiB
Python
90 lines
5.2 KiB
Python
"""Prompts for the consolidation engine."""
|
|
|
|
# Default mission when no bank-specific mission is set
|
|
_DEFAULT_MISSION = "Track every detail: names, numbers, dates, places, and relationships. Prefer specifics over abstractions, never generalise."
|
|
|
|
# Processing rules — always present regardless of mission
|
|
_PROCESSING_RULES = """Processing rules (always apply):
|
|
- REDUNDANT: same info worded differently → UPDATE the existing observation.
|
|
- CONTRADICTION/UPDATE: capture both states with temporal markers ("used to X, now Y").
|
|
- RESOLVE REFERENCES: when a new fact provides a concrete value resolving a vague placeholder in an existing observation (e.g. "home country", "hometown", "birthplace", "native language", "her ex", "that city"), UPDATE the observation to embed the resolved value explicitly. Example: new fact says "grandma in Sweden" + existing observation says "moved from her home country" → update to "home country is Sweden".
|
|
- NEVER merge observations about different people or unrelated topics."""
|
|
|
|
# Data section — format placeholders {facts_text} and {observations_text} are substituted at call time
|
|
_BATCH_DATA_SECTION = """
|
|
NEW FACTS:
|
|
{facts_text}
|
|
|
|
EXISTING OBSERVATIONS (JSON array, pooled from recalls across all facts above):
|
|
{observations_text}
|
|
|
|
Each observation includes:
|
|
- id: unique identifier for updating
|
|
- text: the observation content
|
|
- proof_count: number of supporting memories
|
|
- occurred_start/occurred_end: temporal range of source facts
|
|
- source_memories: array of supporting facts with their text and dates
|
|
|
|
Compare the facts against existing observations:
|
|
- Same topic as an existing observation → UPDATE it (observation_id + source_fact_ids)
|
|
- New topic with durable knowledge → CREATE a new observation (source_fact_ids)
|
|
- Cross-reference facts within the batch: a later fact may resolve a vague reference in an earlier one
|
|
- Purely ephemeral facts → omit them unless the MISSION above explicitly targets such data (e.g. timestamped events, session state, screen content)"""
|
|
|
|
# Output format — JSON braces escaped as {{ }} so .format() leaves them literal
|
|
_BATCH_OUTPUT_FORMAT = """
|
|
Output a JSON object with three arrays.
|
|
|
|
## EXAMPLE
|
|
|
|
Input facts:
|
|
[a1b2c3d4-e5f6-7890-abcd-ef1234567890] Alice mentioned she works long hours, often past midnight | Involving: Alice (occurred_start=2024-01-15, mentioned_at=2024-01-15)
|
|
[b2c3d4e5-f6a7-8901-bcde-f12345678901] Alice said she's exhausted from the project deadlines | Involving: Alice (occurred_start=2024-01-20, mentioned_at=2024-01-20)
|
|
|
|
Good observation text — clean prose, no metadata, each fact tracked distinctly:
|
|
"Alice works long hours, often past midnight."
|
|
"Alice feels exhausted from project deadlines."
|
|
|
|
Bad observation text — NEVER do this (verbatim copy of fact text with metadata):
|
|
"Alice mentioned she works long hours, often past midnight | Involving: Alice (occurred_start=2024-01-15, mentioned_at=2024-01-15)"
|
|
|
|
Observation text rules:
|
|
- Write clean prose — NEVER copy raw fact lines or their metadata (temporal fields, "Involving:", "When:" labels, UUIDs).
|
|
- Parenthesized metadata like (occurred_start=...) and pipe-separated labels like "| Involving: ..." are fact formatting — strip them entirely from observation text.
|
|
- How many observations to create and how much to aggregate is driven by the MISSION above.
|
|
|
|
{{"creates": [{{"text": "Alice works long hours, often past midnight.", "source_fact_ids": ["a1b2c3d4-e5f6-7890-abcd-ef1234567890"]}}, {{"text": "Alice feels exhausted from project deadlines.", "source_fact_ids": ["b2c3d4e5-f6a7-8901-bcde-f12345678901"]}}],
|
|
"updates": [{{"text": "Alice works at Acme Corp as a senior engineer", "observation_id": "c3d4e5f6-a7b8-9012-cdef-123456789012", "source_fact_ids": ["d4e5f6a7-b8c9-0123-defa-234567890123"]}}],
|
|
"deletes": [{{"observation_id": "e5f6a7b8-c9d0-1234-efab-345678901234"}}]}}
|
|
|
|
Rules:
|
|
- "source_fact_ids": copy the EXACT UUID strings shown in brackets [uuid] from NEW FACTS — never use integers or positions.
|
|
- "observation_id": copy the EXACT "id" UUID string from EXISTING OBSERVATIONS.
|
|
- One create/update may reference multiple facts when they jointly support the observation.
|
|
- "deletes": only when an observation is directly superseded or contradicted by new facts.
|
|
- Do NOT include "tags" — handled automatically.
|
|
- Return {{"creates": [], "updates": [], "deletes": []}} if nothing durable is found."""
|
|
|
|
|
|
def build_batch_consolidation_prompt(
|
|
observations_mission: str | None = None,
|
|
observation_capacity_note: str | None = None,
|
|
) -> str:
|
|
"""
|
|
Build the consolidation prompt for batch mode (multiple facts per LLM call).
|
|
|
|
The mission defines *what* to track (customisable per bank).
|
|
Processing rules and output format are always present regardless of mission.
|
|
"""
|
|
mission = observations_mission or _DEFAULT_MISSION
|
|
|
|
capacity_section = ""
|
|
if observation_capacity_note:
|
|
capacity_section = f"\n\n## CAPACITY CONSTRAINT\n{observation_capacity_note}"
|
|
|
|
return (
|
|
"You are a memory consolidation system. Synthesize facts into observations "
|
|
"and merge with existing observations when appropriate.\n\n"
|
|
f"## MISSION\n{mission}{capacity_section}\n\n"
|
|
f"{_PROCESSING_RULES}" + _BATCH_DATA_SECTION + _BATCH_OUTPUT_FORMAT
|
|
)
|