Upgrade and config migration¶

Outcome: Move a daari fleet from version N to N+1 in place, know what survives, and roll back in one step if it does not.

daari has no vendor to call during an upgrade; this page is the upgrade support. Every statement below cites the code that enforces it, so you can verify it against the version you are about to install.

Before you start¶

Read CHANGELOG.md and docs/RELEASE-vX.Y.Z.md for the target version. Breaking changes are called out there; there is no in-app migration wizard.
Note the running version: daari --version or curl -fsS http://127.0.0.1:11435/health (JSON includes "version"). Image/Helm installs can still use pip show daari, docker image inspect, or helm get values when the process is not reachable.
Back up ~/.daari/config.yaml and the durable stores listed under What survives. Caches can be skipped.
Run daari doctor and keep the output — it is your rollback baseline (Doctor and health).

Upgrade steps¶

Configuration is read once at process start (Settings.load() in daari/config/settings.py), so every path ends with a restart of the daemon.

pip / venv¶

source .venv/bin/activate
pip install --upgrade daari            # or: pip install -e ".[dev]" from a checkout
daari doctor
daari service restart

daari service restart runs systemctl --user restart daari.service on Linux and launchctl kickstart -k gui/$(id -u)/com.daari.gateway on macOS (label from daari/setup/service.py; it exits non-zero if daari service install was never run). If you run daari serve by hand, stop and start it. scripts/install.sh is safe to re-run: it recreates the venv, reinstalls, and pulls only missing models; it never touches ~/.daari.

Homebrew¶

brew update && brew upgrade daari
daari doctor
daari service restart

The formula lives in the naveenreddyalka/daari tap (Homebrew install).

Docker / Compose¶

Images are published as ghcr.io/naveenreddyalka/daari:vX.Y.Z. State lives in the /home/daari/.daari volume (Dockerfile), so a new image over the same volume is an in-place upgrade:

docker compose pull daari
docker compose up -d daari
docker compose exec daari daari doctor

Pin an explicit tag in docker-compose.yml; latest makes rollback a guess.

Helm¶

The chart is deploy/helm/daari/; the image tag is image.tag in values.yaml and probes are /health (liveness) and /ready (readiness) (Capacity and Helm):

helm upgrade daari deploy/helm/daari \
  --reuse-values --set image.tag=1.4.0 \
  --atomic --timeout 5m

--atomic rolls the release back automatically if new pods never pass /ready. With two or more replicas behind Redis the default rolling update keeps serving throughout; the L0/L1 stores are shared, so the new pods see the same cache the old ones filled.

Connection drain¶

App-level drain (#1104) makes zero-dropped-request rollouts a property of the binary, not only of Helm:

SIGTERM / SIGINT (or Helm preStop sleep then SIGTERM) flips GET /ready to 503 {"status":"shutting_down"} immediately so the load balancer / kube endpoints stop new traffic. GET /health stays 200 for liveness.
Admission drain — new arrivals at the in-flight gate get 503 + Retry-After (same shape as queue-full). Already-admitted requests and queued waiters complete.
uvicorn waits up to server.graceful_timeout_seconds (default 30, CLI --graceful-timeout, Helm gracefulTimeoutSeconds) for connections (including SSE) to finish, then lifespan awaits outstanding budget-alert webhook tasks (bounded) before closing pools.
Kubernetes — keep terminationGracePeriodSeconds > gracefulTimeoutSeconds + preStopSleepSeconds (chart defaults: 60 > 30 + 5). Details: Capacity and Helm.

Non-K8s (systemd, docker-compose, bare daari serve behind an LB): point the LB health check at /ready and send SIGTERM on stop/reload — the same sequence applies without a preStop hook.

Config compatibility policy¶

~/.daari/config.yaml is merged over the packaged defaults.yaml, then the active project profile, then DAARI_<SECTION>__<KEY> environment variables, and validated once by Settings.model_validate() (Settings.load() in daari/config/settings.py). Consequences, all pinned by tests in tests/unit/test_settings.py:

Situation	Behaviour	Why
Unknown top-level section (`typo_section: {}`)	Startup fails with `ValidationError: Extra inputs are not permitted` naming the key	`Settings` is a pydantic-settings `BaseSettings`, whose default is `extra="forbid"` (`test_unknown_top_level_key_fails_load`)
Unknown nested key (`server.future_option`, `cache.l0.future_knob`)	Logged as a warning (`daari.config`); the rest of the section still applies. `DAARI_STRICT_CONFIG=1` or `daari serve --strict` turns the warning into a startup error	Nested models ignore extras so an upgrade stays downgrade-safe, but a typo is no longer silent (`test_unknown_nested_key_is_ignored`, `tests/unit/test_config_validate.py`)
Wrong type (`server.port: eleven`)	Startup fails with a `ValidationError`	Field types are enforced at load (`test_wrong_type_in_config_file_fails_load`)
Runtime setattr out of range	Rejected (`validate_assignment=True` on runtime models)	`test_setattr_rejects_*`
Legacy `org:` block	Merged into `enterprise:` and dropped	Shim in `Settings.load()`; `test_org_alias_block_maps_to_enterprise`

What this means for an upgrade:

Downgrade-safe by design for nested keys. A config written for N+1 that adds a new nested key still loads on N (the key is ignored). A new top-level section does not — remove it before rolling back.
Removed or renamed keys are announced in CHANGELOG.md. Renamed keys keep a shim for at least one minor release (today: org → enterprise, tenant_id alias in daari/enterprise/config.py).
Validate before you restart. daari config validate (optional path; default ~/.daari/config.yaml) reports every unknown key — top-level and nested, with its dotted path — plus type errors and out-of-range values. Exit 0 is clean; exit 1 prints one finding per line. daari doctor mentions unknown keys on the config_keys check.

daari config validate
daari config validate ~/.daari/config.yaml
DAARI_STRICT_CONFIG=1 daari serve    # unknown nested keys fail startup
daari serve --strict

A non-zero config validate with unknown key: or a type / range message is the same class of failure the daemon would hit on start — fix it, then restart.

Fleet-pushed config (daari enterprise bootstrap) merges the org block and any routing / cache / frontier / guardrails dicts into config.yaml with an atomic 0o600 write (apply_org_config in daari/enterprise/bootstrap.py, daari/config/persist.py). Runtime policy sync applies only the allow-listed keys in SAFE_ROUTING_KEYS, SAFE_FRONTIER_KEYS, SAFE_CACHE_KEYS plus guardrails.enabled and boundaries; unknown keys are ignored and a value of the wrong type is logged as policy_sync_value_rejected and skipped without aborting the sync (daari/enterprise/policy_sync.py). Policy bundles may include an integer schema field (POLICY_SCHEMA in bootstrap.py, currently 1). Laptops on this release reject unknown majors (schema > POLICY_SCHEMA) so a fleet upgrading independently fails closed instead of silently mis-applying policy. Older clients ignore the schema key (and any other unknown keys). Omit schema for legacy bundles.

What survives, and what is rebuilt¶

Everything lives under ~/.daari/ (or the Docker volume / PVC). There is no Alembic; every store creates its own tables with CREATE TABLE IF NOT EXISTS on first open and, where the schema has changed, migrates in place on that same open. Operators can also run the same opens explicitly:

daari migrate --dry-run   # inspect pending additive migrations (SQLite + Postgres);
                          # exit 0 if additive-safe
daari migrate             # open ledger / virtual-keys / audit / responses /
                          # mcp-tasks and configured Postgres stores; report changes
daari doctor              # warns on pending migrate / PG schema lag (optional)
daari doctor --strict     # pending migrate fails the exit code

Rolling upgrade order (fleets with Postgres): run daari migrate (or --dry-run then apply) on a single operator host before rolling replicas to the new image. Migrate is additive and idempotent; mixed N/N+1 replicas are safe once the shared PG schema is at the shipped version. An unreachable observability.postgres_url is reported as an error row (not omitted) so migrate --dry-run never silently skips fleet stores.

Store	Location	Across an upgrade
L0 exact cache	`~/.daari/cache/l0` (diskcache) or Redis `daari:l0:*`	Survives. Keys are a SHA-256 of messages, model, temperature, tools, tier and sampling (`daari/cache/exact.py`). Safe to delete; it refills.
L1 semantic cache	`~/.daari/cache/l1` (diskcache) or Redis `daari:l1:entries`	Survives, keyed by model/temperature/tools/tier and `cache.l1.embedding_model` (`daari/cache/semantic.py`). Changing the embedder yields clean misses (no cross-model cosine hits); stale-model rows are preferred for eviction when over `max_entries`. Unreadable entry lists are treated as empty, not fatal.
Org shared cache	`~/.daari/org/<id>/shared-cache/`	Survives; same rebuildable semantics (`daari/enterprise/service.py`).
Usage ledger	`~/.daari/usage/ledger.sqlite3`	Durable. Tables rebuilt in place when a pre-#156 schema is found (`UsageLedger._migrate`, `daari/observability/usage.py`). If init fails the ledger disables itself rather than crashing the gateway.
Virtual keys / teams	`~/.daari/auth/virtual-keys.sqlite3`	Durable. Additive `ALTER TABLE … ADD COLUMN` migrations (`_migrate` in `daari/auth/virtual_keys.py`). Back this up — keys cannot be regenerated. Prefer `daari keys export --json --out …` / `daari keys import` over copying the SQLite file when migrating hosts or backends.
Rate-limit counters	`rate-limit.sqlite3` / Redis `daari:rl:*`	Ephemeral; windows expire.
Traces, Responses API objects	`~/.daari/traces/`	Durable, create-if-missing only.
Audit log	`~/.daari/audit/audit.sqlite3`	Append-only, durable (`daari/enterprise/audit.py`). Back up for compliance. `observability.retention.audit_days` defaults to 0 (keep forever) so an upgrade never silently deletes audit history; set it only when you want a stated retention period.
Feedback / training examples	`~/.daari/feedback/`, `~/.daari/training/`	Durable, create-if-missing only.
Postgres (`observability.backend=postgres`)	`usage`, `client_usage`, `traces`	Create-if-missing on connect; on failure the store disables itself (`daari/observability/postgres_usage.py`). No destructive migrations are ever run against Postgres.
Postgres request-log / keys / audit	`observability.postgres_url`	Fleet-durable. `daari migrate --dry-run` lists current vs expected schema (`postgres.request-log`, `postgres.virtual-keys`, `postgres.audit`); `daari migrate` opens each store and applies additive `ADD COLUMN IF NOT EXISTS`. Mixed replicas are safe after migrate (#1655).
Request log (JSONL)	`~/.daari/cursor-requests.log`	Rotated per `observability.request_log_*`; disposable. `observability.retention.request_log_days` (default 0) drops lines and old rotations past that window on `daari prune`. Prefer `observability.request_log.backend=postgres` for fleets.

Rule of thumb: caches and counters are disposable, .sqlite3 files are not. A backup of ~/.daari/config.yaml, auth/, usage/, audit/ and feedback/ is a complete backup.

Virtual keys backup / restore¶

Export hashes only (never plaintext secrets). Restore is an idempotent upsert by team_id / key_id — safe to re-run after a partial restore:

daari keys export --json --out keys-backup.json
daari keys import keys-backup.json --dry-run
daari keys import keys-backup.json

Rotated keys keep previous_key_hash + grace deadline so both secrets resolve within the overlap window after import. Unknown export schema versions are refused.

Migrations are forward-only. A ledger rebuilt by N+1 still opens on N (the extra model/token columns are ignored by older INSERTs); a virtual-keys database with added columns likewise. Nothing in the tree drops columns.

Rollback¶

Stop the daemon (or scale the deployment to the previous tag).
Reinstall the previous version: pip install daari==X.Y.Z · Homebrew has no versioned formula, so brew uninstall daari then pip install daari==X.Y.Z in a venv (or reinstall the formula from the tap commit for that release) · docker compose up -d with the previous tag · helm rollback daari <REVISION> (helm history daari lists them).
Restore ~/.daari/config.yaml from your backup if the new version added a top-level section (older code forbids unknown top-level keys, see above). Nested additions can stay.
Leave the .sqlite3 stores in place — migrations are additive. Only if the release notes say otherwise, restore them from backup.
Start, then daari doctor and compare with the baseline you saved.

Caches never need rolling back; if in doubt rm -rf ~/.daari/cache and let them refill.

Fleet ordering and version skew¶

A typical fleet is an org gateway (Helm/Docker, Redis + Postgres) plus daari on developer laptops that pull policy from it.

Gateway first, then laptops. The policy-sync protocol is a signed JSON object (X-Daari-Signature, HMAC-SHA256 of the body — daari/enterprise/bootstrap.py) with an optional integer schema field. Current daari refuses unknown majors; older clients ignore the schema key (and any other unknown keys), so a newer gateway can publish new keys before laptops upgrade. Gateway-first still means new policy is live the moment laptops upgrade; a laptop that sees schema ahead of its POLICY_SCHEMA fails closed instead of applying a half-understood bundle. After a successful sync, daari records a content hash of the verified payload (plus applied_at and a redacted source host) under ~/.daari/policy-sync-state.json. Run daari enterprise policy-status on each replica to confirm the hash matches what you expect; exit code 1 means policy_sync_url is set but nothing has applied yet. daari doctor surfaces the same never-applied case on the optional policy_status check.
Shared Redis/Postgres tolerate mixed versions after migrate. L0/L1 entries are version-agnostic (see the table). Postgres tables are create-if-missing plus additive ALTER … ADD COLUMN; run daari migrate before the rollout so mixed N/N+1 replicas share one schema. Do not drain the cache.
Rotate the signing secret only after both sides are upgraded if a release changes the signature scheme; the release notes will say so.
Laptops upgrade independently. Each laptop is its own install; use the pip/brew steps above (or your MDM). A laptop that fails config validation on start is loud (ValidationError), not silently degraded.

Verify¶

daari migrate --dry-run
daari doctor
curl -fsS http://127.0.0.1:11435/ready
daari report          # ledger still has history → durable stores survived

Next¶

→ Doctor and health · Capacity and Helm · Releasing (how the versions you consume are cut)