fleet-memory/scripts/dev/start.sh
Nicolò Boschi 576473b6aa
feat: observation history tracking and diff UI (#513)
* feat: add source facts token limits to consolidation and recall

- Add two new configurable (per-bank) parameters:
  - consolidation_source_facts_max_tokens: total token budget for source
    facts across all observations in the consolidation prompt (-1 = unlimited)
  - consolidation_source_facts_max_tokens_per_observation: per-observation
    cap so each observation gets a fair share of source facts (-1 = unlimited,
    default 256)
- Both are also exposed as recall API parameters via SourceFactsIncludeOptions
  (max_tokens and max_tokens_per_observation)
- Consolidation now uses resolve_full_config to respect bank-level overrides
- Improve consolidation prompt: temporal metadata (occurred_start=, | Involving:)
  is now clearly separated from observation text, with a concrete example showing
  the expected synthesis style and explicit rules not to copy raw fact lines
- Add tests for recall source facts capping and consolidation config forwarding
- Expose all three new fields in the control plane bank config UI
- Document new env vars in configuration.md
- Regenerate OpenAPI spec and all SDK clients

* fix: reorder observations UI fields and rename Label Groups to Entity Labels

* fix: revert Entities section title (only rename inner label)

* doc: add consolidation source facts and batch size fields to memory-banks docs

* feat: add observation history tracking and UI diff view

- Track observation changes over time in a JSONB history column,
  appending each update's previous state (text, tags, dates, sources)
  instead of overwriting
- Add HINDSIGHT_API_ENABLE_OBSERVATION_HISTORY config flag (default: true)
  to toggle history recording
- Expose history field in get_memory_unit for observations
- Fix observations/[modelId] route that was proxying to wrong endpoint
- Add History tab in observation modal and History section in panel,
  showing word-level and tag diffs between each change (newest first)
- Extract shared ObservationHistoryView component used by both modal and panel
- Add --random-port flag to start.sh to run multiple dev instances
- Scope Next.js distDir by port to prevent lock file collisions between instances
- Restyle consolidation pending badge (rounded-md with border) and add
  inline refresh button; fix loading flicker on data refresh

* feat: dedicated observation history endpoint with source facts diff

- Add GET /memories/{id}/history endpoint returning enriched history with
  resolved source fact texts and is_new flags per change
- Deprecate history field in GET /memories/{id} (always returns empty list)
- Reconstruct cumulative source facts per history entry by working backwards
  from current state, marking newly added facts with is_new
- Replace inline history panel with "View History" button opening modal
- History modal fetches from dedicated endpoint lazily on tab switch
- Timeline view now opens MemoryDetailModal instead of side panel
- History view uses prev/next navigation (left = older, right = newer)
- Fix --random-port: pass dynamic API_PORT as HINDSIGHT_CP_DATAPLANE_API_URL
  to control plane, preserving caller values over .env
2026-03-06 16:16:05 +01:00

107 lines
2.4 KiB
Bash
Executable file

#!/bin/bash
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
# Parse --random-port flag
RANDOM_PORT=false
for arg in "$@"; do
if [ "$arg" = "--random-port" ]; then
RANDOM_PORT=true
fi
done
# Load .env to pick up HINDSIGHT_API_PORT if set
ROOT_DIR="$(git rev-parse --show-toplevel)"
if [ -f "$ROOT_DIR/.env" ]; then
set -a
source "$ROOT_DIR/.env"
set +a
fi
get_free_port() {
python3 -c "import socket; s=socket.socket(); s.bind(('', 0)); print(s.getsockname()[1]); s.close()"
}
if [ "$RANDOM_PORT" = true ]; then
API_PORT="$(get_free_port)"
CP_PORT="$(get_free_port)"
echo "Using random ports — API: $API_PORT, Control Plane: $CP_PORT"
else
API_PORT="${HINDSIGHT_API_PORT:-8888}"
CP_PORT="${HINDSIGHT_CP_PORT:-9999}"
fi
PIDS=()
kill_tree() {
local pid=$1
local children
children=$(pgrep -P "$pid" 2>/dev/null) || true
for child in $children; do
kill_tree "$child"
done
kill "$pid" 2>/dev/null || true
}
cleanup() {
echo ""
echo "Shutting down..."
for pid in "${PIDS[@]}"; do
kill_tree "$pid"
done
wait 2>/dev/null || true
}
trap cleanup EXIT INT TERM
# Start API
echo "Starting API server..."
"$SCRIPT_DIR/start-api.sh" --port "$API_PORT" &
API_PID=$!
PIDS+=($API_PID)
# Wait for API to be ready
echo "Waiting for API to be ready..."
API_READY=false
for i in {1..60}; do
if curl -sf "http://localhost:${API_PORT}/health" &>/dev/null; then
echo "API is ready"
API_READY=true
break
fi
if ! kill -0 "$API_PID" 2>/dev/null; then
echo "API process exited unexpectedly"
exit 1
fi
sleep 1
done
if [ "$API_READY" = false ]; then
echo "API did not become ready in time"
exit 1
fi
# Start Control Plane
echo ""
PORT="$CP_PORT" HINDSIGHT_CP_DATAPLANE_API_URL="http://localhost:${API_PORT}" "$SCRIPT_DIR/start-control-plane.sh" &
CP_PID=$!
PIDS+=($CP_PID)
echo ""
echo "Hindsight is running!"
echo ""
echo " API: http://localhost:${API_PORT}"
echo " Control Plane: http://localhost:${CP_PORT}"
echo ""
echo "Press Ctrl+C to stop both services."
echo ""
# Poll until any service exits (wait -n requires bash 4.3+, not available on macOS)
while true; do
for pid in "${PIDS[@]}"; do
if ! kill -0 "$pid" 2>/dev/null; then
echo "A service exited unexpectedly (PID $pid)"
exit 1
fi
done
sleep 2
done