Commit graph

9 commits

Author SHA1 Message Date
Derek Bouius
300d089b6a
fix: resolve remaining Dependabot security alerts (#833)
* fix: resolve remaining Dependabot security alerts

- Regenerate package-lock.json so npm overrides take effect
  (serialize-javascript, handlebars, path-to-regexp, brace-expansion)
- Upgrade Pygments 2.19.2 -> 2.20.0 in crewai and integration-tests
  lockfiles (fixes ReDoS via GUID matching)

* fix: resolve duplicate alembic revision ID d6e7f8a9b0c1

Two migrations shared the same revision ID: the merge migration
(drop_documents_metadata_column) and the trigram index migration
(case_insensitive_entities_trgm_index). Assign a new unique ID
to the trigram migration and update the downstream dependency.

* chore: fix lint formatting for generated and existing files
2026-04-01 17:22:38 +02:00
Derek Bouius
ee4510a762
fix(deps): address critical and high severity security vulnerabilities (#827)
* fix(deps): address critical and high severity security vulnerabilities

Bump vulnerable dependencies to patched versions across the monorepo:

Python (critical/high):
- fastmcp >=2.14.0 → >=3.2.0 (SSRF, path traversal, OAuth confused deputy, command injection)
- langchain-core >=1.2.11 → >=1.2.22 (path traversal in legacy load_prompt)

Python (low):
- cryptography >=46.0.5 → >=46.0.6 (incomplete DNS name constraint enforcement)
- pygments: add >=2.20.0 pin (ReDoS via GUID regex)

Node.js:
- serialize-javascript ^7.0.3 → ^7.0.5 (CPU exhaustion DoS)
- handlebars: add >=4.7.9 override (JS injection via AST type confusion)
- path-to-regexp: add >=0.1.13 override (ReDoS via route params)
- brace-expansion: add version range override (process hang/memory exhaustion)

Also adds type: ignore comments for FastMCP 2.x private attribute access that
ty now flags since FastMCP 3.x removed _tool_manager (guarded by try/except
and hasattr at runtime).

Regenerated all lock files across API, integrations, and tests.

* fix(deps): add ajv v8 scoped overrides for schema-utils and ajv-keywords

The global ajv ^6.14.0 override caused schema-utils and ajv-keywords to
receive ajv v6, but they require ajv v8 (for dist/compile/codegen). Add
scoped overrides to ensure these packages get ajv v8 while the global
override remains for packages that need v6.

* fix(tests): remove stateless_http from FastMCP() constructor calls

FastMCP 3.x no longer accepts stateless_http in the constructor. The
tests call tools directly without HTTP transport, so the parameter is
not needed.

* fix: update MCP tests for FastMCP 3.x _tool_manager removal

FastMCP 3.x removed _tool_manager. Tests now use
_local_provider._components for sync tool dict access and
mcp.list_tools() for async filtered tool listing.

* fix: resolve docusaurus build failures (ajv overrides + missing blog date)

- Remove global ajv ^6.14.0 override and scoped ajv-keywords/schema-utils
  overrides that caused webpack compilation errors manifesting as
  "Cannot read properties of undefined (reading 'date')" during SSR
  and "these parameters are deprecated" warnings. Natural version
  resolution (v6.12.6+ for v6 consumers, v8+ for v8 consumers) already
  satisfies the security fix (>= 6.12.3).
- Add missing date frontmatter to learning-capabilities blog post.

* chore: regenerate openapi spec and docs skill
2026-04-01 09:20:34 +02:00
Nicolò Boschi
4c4b3568db
fix(security): address all Dependabot vulnerability alerts (#617)
Python (uv.lock, pyproject.toml):
- authlib 1.6.6 → 1.6.9 (JWS header injection, OIDC hash binding, Bleichenbacher padding oracle)
- pyasn1 0.6.2 → 0.6.3 (unbounded recursion DoS)
- pyjwt 2.10.1 → 2.12.1 (unknown crit header extensions - also in integration-tests and crewai)
- orjson 3.11.4 → 3.11.7 (deeply nested JSON recursion DoS)
- tornado 6.5.2 → 6.5.5 (multipart DoS, incomplete cookie validation)

npm (package.json, package-lock.json):
- next ^16.1.6 → ^16.1.7 (HTTP smuggling, CSRF bypass, cache DoS, null origin bypass)
- fast-xml-parser override updated to >=5.5.6 (numeric entity expansion bypass)
- undici override added >=7.24.0 (WebSocket overflow, smuggling, CRLF injection, DoS)
- flatted override added >=3.4.0 (unbounded recursion DoS)
- svgo override added >=3.3.3 (DOCTYPE entity expansion DoS)
- dompurify override added >=3.3.2 (XSS vulnerability)
2026-03-19 14:27:52 +01:00
Nicolò Boschi
15ea23d5d6
feat: introduce hindsight-api-slim and hindsight-all-slim packages (#560)
* feat: introduce hindsight-api-slim and hindsight-all-slim packages

Closes #552

- Move all source code from hindsight-api/ to new hindsight-api-slim/
- hindsight-api-slim has heavy ML deps (torch, sentence-transformers,
  transformers, einops, flashrank, mlx, mlx-lm, safetensors) and
  pg0-embedded as optional extras: [local-ml], [embedded-db], [all]
- hindsight-api becomes a zero-code meta-package depending on
  hindsight-api-slim[all] for full backward compatibility
- Add hindsight-all-slim meta-package: hindsight-api-slim + client + embed
- hindsight-all updated to depend on hindsight-api-slim[all]
- pg0.py: lazy-import pg0 with clear ImportError pointing to [embedded-db]
- Dockerfile: replace sed hack with proper uv sync --extra flags
- Update release.yml, test.yml, lint.sh, release.sh, CLAUDE.md and
  all path references throughout the repo

* refactor: rename hindsight/ directory to hindsight-all/

* docs: document hindsight-api-slim and hindsight-all-slim package variants

Add package variants table and extras explanation to installation.md

* docs: remove emojis from installation.md, use professional tone

* docs: link Docker slim variant to pip package variants section

* docs: consolidate Docker image variants into single table

* ci: fix working-directory paths after package restructure

- Replace all hindsight-api → hindsight-api-slim in test.yml
- Replace hindsight → hindsight-all in test.yml
- Add --extra embedded-db to test-embed API install step

* ci: add local-ml and embedded-db extras to API sync steps

These extras were previously implicit in the old hindsight-api package
(which bundled everything). Now that hindsight-api-slim uses optional
extras, we must explicitly request local-ml and embedded-db in CI.

* ci: add API install step with embedded-db to test-embed smoke test

The smoke test starts hindsight-api as a daemon, which requires pg0-embedded.
Add a dedicated install step for hindsight-api-slim with embedded-db extra
so the daemon can start successfully.

* ci: remove --no-install-project when using optional extras

When --no-install-project is combined with --extra, the optional deps
are not installed because extras require the project to be active.
Remove --no-install-project from steps that need local-ml or embedded-db.

* ci: fix ordering of uv sync steps to preserve optional extras

When uv sync runs for a different workspace member, it removes optional
extras installed for other members. Fix by always running extra-requiring
API sync last, after other workspace member syncs.

Also remove --no-install-project from embedded-db sync in test-embed,
as --no-install-project prevents optional extras from being active.

* ci: add local-ml extra to test-embed API install for smoke test

The smoke test starts the full API server which needs sentence-transformers
for local embeddings (default provider). Add local-ml extra to the install.

* ci: simplify extras with --all-extras and add slim pip smoke test

- Replace explicit --extra local-ml --extra embedded-db with --all-extras
  for cleaner, more maintainable sync steps
- Add test-pip-slim job: tests hindsight-api-slim[embedded-db] without
  local ML models, using Cohere for embeddings/reranking (mirrors Docker
  slim smoke test approach)

* ci: simplify slim smoke test to health check only (mirrors Docker test)
2026-03-13 13:50:03 +01:00
Nicolò Boschi
ea27ef95ec
fix: resolve all Dependabot security vulnerabilities (#486)
* fix: resolve all Dependabot security vulnerabilities

npm (package-lock.json):
- fast-xml-parser: 4.5.3 → 4.5.4 (critical entity encoding bypass + DoS)
- serialize-javascript: 6.0.2 → 7.0.4 (high RCE via RegExp/Date)
- minimatch: 3.1.2 → 3.1.5, 5.1.6 → 5.1.9, 9.0.5 → 9.0.9 (high ReDoS)
- ajv: 6.12.6 → 6.14.0, 8.17.1 → 8.18.0 (medium ReDoS with $data option)
- qs: 6.14.1 → 6.15.0 (low arrayLimit bypass DoS)
- rollup: 4.57.x → 4.59.0 in ai-sdk and openclaw integrations (high path traversal)

Python (uv.lock / pyproject.toml):
- cryptography: 46.0.3 → 46.0.5 (high subgroup attack on SECT curves)
- pillow: 12.0.0 → 12.1.1 (high out-of-bounds write in PSD loading)
- langchain-core: 1.2.7 → 1.2.17 (low SSRF in ChatOpenAI token counting)
- langsmith: 0.4.42 → 0.7.11 (medium SSRF via tracing header injection)
- protobuf: 6.33.1 → 6.33.5 (high JSON recursion depth bypass)

Rust (Cargo.lock):
- bytes: 1.11.0 → 1.11.1 in hindsight-clients/rust (medium integer overflow)

Remaining unfixable: diskcache <= 5.6.3 (no patched version available)

* fix: remove over-broad schema-utils ajv override that broke docs build

The 'schema-utils': {'ajv': '^8.18.0'} override was forcing schema-utils@3.x
(used by url-loader/file-loader with ajv-keywords@3.x) to use ajv@8.18.0.
In 8.18.0, internal property _formats was renamed to formats, breaking
ajv-keywords@3.x's _formatLimit.js which accesses ajv._formats.date.

Removing the broad override: schema-utils@4.3.3 (root level) already has
ajv@8.18.0 in its nested install from the prior npm update, while
schema-utils@3.x correctly falls back to the hoisted root ajv@6.14.0.
2026-03-04 13:14:50 +01:00
Nicolò Boschi
93ddd41621
feat: add reverse proxy support (#346)
* feat: add reverse proxy support

* improve

* improve

* improve

* improve

* improve

* fix: update integration test to use modern 'docker compose' command

- Replace 'docker-compose' with 'docker compose' (Docker Compose v2+)
- Add fallback to legacy docker-compose command for compatibility
- Fixes test failures on systems using Docker Compose plugin

* ci: trigger test rerun

* fix: make docker-compose detection more robust for CI

- Add get_docker_compose_command() to detect available command
- Use shutil.which() to check command availability
- Dynamically use correct command (docker compose vs docker-compose)
- Should work in both modern and legacy Docker environments

* fix: docker-compose networking in base path integration test

Fix connection refused error in test_reverse_proxy_simple_config by
handling host vs bridge networking modes correctly:

- Linux (host mode): nginx listens on 18080 directly, no port mapping
- Mac/Windows (bridge mode): nginx listens on 80, mapped to 18080

With host networking, port mappings in docker-compose don't work since
the container binds directly to the host's network namespace.
2026-02-12 10:09:53 +01:00
Nicolò Boschi
038a9c2313
fix(sec): upgrade vulnerable deps (#254)
* fix(sec): upgrade vulnerable deps

* feat: add comprehensive logging to upgrade tests

- Modify VersionRunner to write server logs to /tmp/upgrade-test-*.log files
- Add pytest hook to automatically dump server logs on test failure
- Add CI workflow step to show upgrade test logs (always runs)
- Improves debuggability when upgrade tests fail in CI

This addresses the issue where upgrade test failures in CI were
impossible to debug because API server logs were not visible.
2026-02-03 10:37:36 +01:00
Nicolò Boschi
4f2833873c
feat: introduce mental models (#132)
* mental models

* DRAFT: refactor entity observations

* fix db patch

* agentic

* agentic

* reflect agent

* new style

* more

* fix ci

* fix

* fix
2026-01-16 11:16:41 +01:00
Nicolò Boschi
76fd052b3a
misc: add mcp integration tests and increase test coverage (#98)
* misc: add mcp integration tests and increase test coverage

* misc: add mcp integration tests and increase test coverage

* misc: add mcp integration tests and increase test coverage
2026-01-05 11:16:55 +01:00