fleet-memory/hindsight-docs
Nicolò Boschi 9b96becc5c
feat: entity labels — optional, free_values, multi_value, UI polish (#450)
* feat: entity labels

* feat: entity labels — optional, free_values, multi_value, UI polish

Completes the entity labels system:

**Schema & extraction**
- Dynamic Pydantic Labels model per fact: each group becomes a typed
  field (Literal | None, list[Literal], str | None, or list[str])
- `optional: bool` flag per group — non-optional enum fields appear in
  JSON schema required array so structured-output providers enforce them
- `free_values: bool` flag per group — accepts any LLM-generated string
  instead of a predefined enum; example values shown as hints in prompt
- New `is_label_entity()` helper for labels-only mode filtering that
  handles both enum lookup and free_values key-prefix matching
- Sentinel rejection: "None"/"null"/"n/a" strings dropped in post-processing

**BM25 / dense retrieval**
- `text_signals` column on memory_units: entity names + date tokens for
  enriched BM25 indexing without polluting stored fact text
- Dense embedding includes occurred_end when it differs from occurred_start
- Alembic migration z1u2v3w4x5y6 (merge revision fixing two heads)

**UI (bank-config-view)**
- Shadcn Switch replaces custom Toggle for both entity-labels and observations
- Shadcn Checkbox for multi/optional/free_values per group
- Input heights bumped to h-8 throughout the editor
- "Label Groups" → "Entity Labels", "Free-form entities" → "Entities"
- Free-text groups show "Example hints" banner in values section

**Tests (45 unit + 3 LLM integration)**
- build_labels_model: single, multi, mixed, free_values optional/required/multi
- is_label_entity: enum match, free_values prefix match, no false positives
- Post-processing: null/absent/string-None/free_values/sentinels/multi-value
- Schema: labels in required, structured object, no labels when unconfigured
- LLM integration: single-value enum, multi-value enum, free_values retain

**Docs**
- retain.md: new Entity Labels section covering groups, flags, examples
- configuration.md: retain_free_form_entities env var + entity_labels note

* fix(tests): update hierarchical fields count for entity_labels additions

entity_labels and retain_free_form_entities are hierarchical fields,
bumping the expected count from 11 to 13.

* fix(migration): rename text_signals revision to avoid collision with main

Main branch claimed z1u2v3w4x5y6 for observation_scopes. Rename our
text_signals migration to a2b3c4d5e6f7, chaining after z1u2v3w4x5y6.

* refactor(entity-labels): simplify free_values — always str|None, no multi

- free_values groups always produce str | None (multi_value and optional
  flags are ignored for free text groups — always optional, never multi)
- Prompt section for free_values groups shows only key + description,
  no values list (users put examples in the description instead)
- UI: section title "Entities", toggle "Free Form Entities", replace
  per-group checkboxes with a type dropdown (Enum / Free text); only
  show multi checkbox and values list when type is Enum
- Update tests to reflect new behaviour

* refactor(entity-labels): replace free_values/multi_value booleans with type field

- LabelGroup now uses type: "value" | "multi-values" | "text" instead of
  free_values/multi_value boolean pair
- Backward-compat migration converts legacy dicts automatically
- Rename retain_free_form_entities → entities_allow_free_form throughout
- Update UI dropdown to show Single value / Multi-values / Free text
- Remove separate multi checkbox (captured by type selection)
- Update docs examples and configuration.md
- Update all tests to use new field names

* fix(migration): backfill observation_scopes column for DBs with swapped z1u2v3w4x5y6

Local DBs that had z1u2v3w4x5y6 applied when it referred to the old
text_signals migration (before it was renamed to a2b3c4d5e6f7) won't have
observation_scopes in their memory_units table. This migration adds the
column with IF NOT EXISTS so it's a no-op on clean installs.

* feat(entity-labels): add tag field to auto-populate memory unit tags from labels

When a LabelGroup has tag=True, extracted key:value entities for that group
are automatically written to the memory unit's tags array. This lets entity
labels double as tags, enabling immediate filtering via the existing
tags/tags_match API params with no extra infrastructure.

- Add tag: bool = False to LabelGroup
- _inject_label_tags() helper called in both sync and batch extraction paths
- UI: add Tag checkbox per label group row
- Docs: document the new tag field
- Tests: 4 new unit tests covering all tag injection paths

* style: ruff format migration file

* fix(migration): fix multiple alembic heads after rebase — point text_signals after nullable_event_date

* fix(clients): update timestamp field to use Timestamp wrapper type after timestamp=unset feature

* style: ruff format agent.py

* fix(docs): update Go quickstart example to use NullableTimestamp for timestamp field
2026-03-02 13:05:25 +01:00
..
blog doc: 0.4.14 (#449) 2026-02-27 15:41:08 +01:00
docs feat: entity labels — optional, free_values, multi_value, UI polish (#450) 2026-03-02 13:05:25 +01:00
examples feat: entity labels — optional, free_values, multi_value, UI polish (#450) 2026-03-02 13:05:25 +01:00
src doc: improvements (#448) 2026-02-27 14:59:11 +01:00
static feat: support timestamp="unset" to retain content without a date (#465) 2026-03-02 12:03:23 +01:00
versioned_docs doc: improvements (#448) 2026-02-27 14:59:11 +01:00
versioned_sidebars doc: improvements (#448) 2026-02-27 14:59:11 +01:00
.gitignore rename to hindsight (#2) 2025-11-25 19:28:26 +01:00
docusaurus.config.ts doc: improvements (#448) 2026-02-27 14:59:11 +01:00
package.json doc: hide next version (#226) 2026-01-29 08:46:21 +01:00
README.md rename to hindsight (#2) 2025-11-25 19:28:26 +01:00
sidebars.ts doc: improvements (#448) 2026-02-27 14:59:11 +01:00
tsconfig.json rename to hindsight (#2) 2025-11-25 19:28:26 +01:00
versions.json doc: release notes for 0.4.0 (#217) 2026-01-28 16:54:05 +01:00

Website

This website is built using Docusaurus, a modern static website generator.

Installation

npm install

Local Development

npm start

This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.

Build

npm run build

This command generates static content into the build directory and can be served using any static contents hosting service.

Deployment

Using SSH:

USE_SSH=true npm run deploy

Not using SSH:

GIT_USER=<Your GitHub username> npm run deploy

If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the gh-pages branch.