Capacity and Helm¶

Outcome: Size a gateway-heavy fleet and install the chart.

Sizing (rough)¶

Resource	Guidance
Gateway replica	Measured on an M4 Pro: see benchmark-load.md. Older estimate (~50–100 rps cache-heavy, ~5–15 rps L3-heavy) is superseded by that page.
Redis	~200–400 MB / 100k cache entries
Postgres	~1 KB/row ledger/traces; retain 30–90 days
HPA	CPU 70%; optional KEDA request-rate scaler is off until you enable it
Request body	`server.max_body_bytes` (default 10 MiB) returns 413 before buffering; see SECURITY.md. Override with `DAARI_SERVER__MAX_BODY_BYTES`.

Redis is an accelerator, not a single point of failure: set cache.backend: redis for shared L0/L1 and fleet-wide RPM/TPM counters, but expect a Redis outage to degrade rate limiting to per-replica SQLite (or rate_limit.fail_open) and mark /ready as degraded with HTTP 200 while the gateway keeps serving. Tune cache.redis_timeout_seconds (default 2s). Details: Auth and keys.

Helm¶

Chart: deploy/helm/daari/. Point Redis/Postgres/org pool via values. Image: ghcr.io/naveenreddyalka/daari.

Defaults are a single replica (replicaCount: 1, autoscaling.minReplicas: 1) with postgres.enabled: false. Batches, files, responses, the usage ledger, and the audit log then use per-pod SQLite — fine for one pod. Raising replicas (or HPA min) above 1 without Postgres splits those stores across pods (404s / divergent counters / incomplete daari audit export). helm install NOTES and daari doctor (via DAARI_FLEET_REPLICAS) warn on that combination.

For a multi-replica fleet, enable the chart Postgres helper (sets observability / batches / files / responses / audit backends):

replicaCount: 2
autoscaling:
  minReplicas: 2
postgres:
  enabled: true
  url: postgresql://daari:daari@postgres:5432/daari

Moving an installed release to a new image tag (helm upgrade --atomic, rollback, what survives in Redis/Postgres): Upgrade and config migration.

Request-rate autoscaling (KEDA)¶

CPU HPA misses cache-heavy replicas that stay idle while daari_requests_total climbs. autoscaling.keda is off by default and renders nothing — no ScaledObject, no CRDs. The chart does not install KEDA. On a cluster that already has KEDA and Prometheus:

autoscaling:
  enabled: true          # CPU HPA still renders
  keda:
    enabled: true
    serverAddress: http://prometheus-operated.monitoring.svc:9090
    query: sum(rate(daari_requests_total[1m]))
    threshold: "50"      # desired replicas ≈ query / threshold
    minReplicaCount: 1   # refused above 1 until postgres.enabled
    maxReplicaCount: 10

minReplicaCount above 1 with postgres.enabled: false fails helm template (per-pod SQLite). NOTES still warn when the effective replica floor is above 1 without Postgres. KEDA also creates its own HPA; leave autoscaling.enabled on if you still want the chart's CPU HPA beside it.

Bumping `image.tag` / `appVersion`¶

After a tagged release (e.g. v1.4.0), update both in the same change — do not create git tags from routine chart PRs:

deploy/helm/daari/Chart.yaml → appVersion: "X.Y.Z"
deploy/helm/daari/values.yaml → image.tag: "X.Y.Z"

Keep them equal to pyproject.toml / daari.__version__. tests/unit/test_helm_chart.py fails if they drift.

Pinning `image.digest` (cosign-verified pulls)¶

Tags are mutable. Regulated / air-gapped fleets should pin the exact blob they verified:

image:
  repository: ghcr.io/naveenreddyalka/daari
  tag: "1.4.0"   # kept for Chart sync / docs; ignored when digest is set
  digest: "sha256:…"  # Deployment + backup CronJob render repository@sha256:…

Leave digest empty (default) to keep repository:tag. When set, digest wins over tag. Verify with cosign verify against the digest (or tag that resolves to it) as documented in RELEASING.md — no chart-side signing workflow.

Graceful rollouts / drains¶

Defaults leave interactive SSE streams (chat, Responses, MCP) and in-flight batch drains a window to finish before kubelet SIGKILLs the pod. On SIGTERM the app flips GET /ready to 503 {"status":"shutting_down"} immediately (liveness /health stays 200), stops admitting new in-flight work, and lets queued waiters plus already-admitted requests finish (#1104):

Value	Default	Role
`gracefulTimeoutSeconds`	`30`	Sets `DAARI_SERVER__GRACEFUL_TIMEOUT_SECONDS` → uvicorn `timeout_graceful_shutdown`. Size for the longest stream you expect (and `sse_keepalive_seconds`).
`terminationGracePeriodSeconds`	`60`	Total time after SIGTERM before SIGKILL. Must be greater than `gracefulTimeoutSeconds + lifecycle.preStopSleepSeconds`.
`lifecycle.preStopSleepSeconds`	`5`	`preStop` sleep so Service endpoint removal propagates before SIGTERM. Set `0` to omit the lifecycle block.
`strategy.rollingUpdate`	`maxUnavailable: 0` / `maxSurge: 1`	Never drop capacity during upgrades.

Non-Helm (systemd, docker-compose, bare daari serve): send SIGTERM/SIGINT; /ready flips the same way. Override with daari serve --graceful-timeout N or DAARI_SERVER__GRACEFUL_TIMEOUT_SECONDS. See Upgrade and migrate.

Pod security and disruption budget¶

Defaults harden the pod (runAsNonRoot uid 1000, drop all capabilities, readOnlyRootFilesystem) and mount emptyDirs at /home/daari/.daari and /tmp so the gateway can still write local state under a read-only root. Swap the daari-home emptyDir for a PVC when you need durable per-pod SQLite across restarts.

podDisruptionBudget defaults to disabled. For multi-replica fleets (alongside HPA / replicaCount ≥ 2 and Postgres), enable it so voluntary node drains keep interactive SSE alive:

replicaCount: 2
autoscaling:
  minReplicas: 2
podDisruptionBudget:
  enabled: true
  minAvailable: 1

Use maxUnavailable: 1 instead of minAvailable when you have a single replica and still want a PDB (minAvailable=1 blocks draining the only pod). PDB and HPA compose: HPA sets desired replicas; the PDB only constrains voluntary evictions during drains/upgrades.

NetworkPolicy and topology spread¶

networkPolicy defaults to disabled so existing installs keep open cluster networking. Enable it for platform baseline reviews (restricted ingress/egress). Ingress peers come from networkPolicy.ingress.from (namespaceSelector / podSelector / ipBlock). When serviceMonitor.enabled is also true, the chart adds scrape peers from networkPolicy.ingress.prometheusFrom (default: namespace kubernetes.io/metadata.name=monitoring). Egress always allows DNS (UDP/TCP 53); Redis (6379) / Postgres (5432) / frontier HTTPS (443) / local-inference ports (11434, 8000, 8080) turn on when the matching values knobs are set. Redis egress also opens when rateLimit.redisUrl points at managed Redis (even with redis.enabled=false); Postgres egress opens when postgres.url is set (even with postgres.enabled=false). Append full rules with ingress.extra / egress.extra / networkPolicy.extraEgress.

networkPolicy:
  enabled: true
  ingress:
    from:
      - namespaceSelector:
          matchLabels:
            kubernetes.io/metadata.name: ingress-nginx
  # CIDR peers for managed DBs when URL-aware ports are not enough:
  extraEgress:
    - to:
        - ipBlock:
            cidr: 10.9.0.0/16
      ports:
        - protocol: TCP
          port: 5432
serviceMonitor:
  enabled: true

affinity, topologySpreadConstraints, priorityClassName, nodeSelector, tolerations, and initContainers pass through to the Deployment pod template. extraEnv appends raw env entries (supports valueFrom / secretKeyRef) after the string-only env map. Empty defaults leave rendering unchanged. For multi-AZ fleets, prefer zone spread (and optional pod anti-affinity) so HPA replicas do not pack onto one node:

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: ScheduleAnyway
    labelSelector:
      matchLabels:
        app.kubernetes.io/name: daari
affinity:
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchLabels:
              app.kubernetes.io/name: daari
          topologyKey: kubernetes.io/hostname

Prometheus ServiceMonitor¶

serviceMonitor defaults to disabled. On clusters with the Prometheus Operator (monitoring.coreos.com CRDs), enable it so kube-prometheus scrapes GET /metrics. Chart defaults already set DAARI_OBSERVABILITY__PROMETHEUS=true. Add serviceMonitor.labels when your operator selects monitors by release label.

Scrape path (pick one):

Mode	Values	ServiceMonitor target	Auth
API port (default)	`observability.metricsPort: 0`	Service port `http`	Bearer via `serviceMonitor.bearerTokenSecret` when `server.api_key` protects `/metrics`
Private metrics port	`observability.metricsPort: 9090` (example)	Service port `metrics`	None — scrape-only listener; keep off public ingress

# Private scrape listener (no Bearer on /metrics):
observability:
  metricsPort: 9090
serviceMonitor:
  enabled: true
  labels:
    release: kube-prometheus-stack

# API-port scrape with Bearer (metricsPort left at 0):
# kubectl create secret generic daari-metrics-token \
#   --from-literal=token="$DAARI_SERVER__API_KEY"
serviceMonitor:
  enabled: true
  bearerTokenSecret:
    name: daari-metrics-token
    key: token

When metricsPort > 0, the chart sets DAARI_OBSERVABILITY__METRICS_PORT, adds a metrics containerPort / Service port, and points ServiceMonitor at metrics (bearer settings are ignored for that endpoint).

See Prometheus metrics.

Grafana overview dashboard¶

grafanaDashboard defaults to disabled. When enabled, the chart renders a ConfigMap embedding deploy/grafana/daari-dashboard.json (packaged under files/grafana/ in the chart). Default labels include grafana_dashboard: "1" so the common Grafana sidecar / kube-prometheus-stack dashboard importer picks it up; override grafanaDashboard.labels / annotations for your operator.

grafanaDashboard:
  enabled: true
  labels:
    grafana_dashboard: "1"
  # annotations: {}

See Prometheus metrics.

Org GPU pool¶

orgPool defaults to disabled. When enabled, the Deployment sets DAARI_ROUTING__ORG_POOL__ENABLED and DAARI_ROUTING__ORG_POOL__BASE_URL so local tiers can fall through to a shared Ollama/vLLM pool before frontier. helm install NOTES echo the same when orgPool.enabled=true (including baseUrl). When serviceMonitor.enabled=true, NOTES also remind operators that Prometheus scrapes Service /metrics and that bearerTokenSecret is needed if the API key protects metrics on the API port.

Master key, rate limits, and frontier¶

For multi-replica + auth fleets, wire first-class Settings knobs from values instead of hand-patching the Deployment:

server:
  apiKeySecret:
    name: daari-master
    key: api-key
rateLimit:
  enabled: true
  rpm: 600
  tpm: 200000
  maxInFlight: 8          # DAARI_RATE_LIMIT__MAX_IN_FLIGHT
  models:                 # DAARI_RATE_LIMIT__MODELS (JSON)
    llama3-70b:
      rpm: 10
      tpm: 20000
      max_in_flight: 2
  # Prefer redis.enabled so L0/L1 and RPM/TPM share one Redis:
redis:
  enabled: true
  url: redis://redis:6379/0
frontier:
  enabled: true
  apiKeySecret:
    name: daari-frontier
    key: api-key

server.apiKeySecret mounts DAARI_SERVER__API_KEY. rateLimit.enabled sets DAARI_RATE_LIMIT__* (and, when redis.enabled is false, can set cache.backend=redis from rateLimit.redisUrl). frontier.enabled sets DAARI_FRONTIER__ENABLED; frontier.apiKeySecret mounts DAARI_FRONTIER_API_KEY. NOTES remind operators when Redis is missing for multi-replica rate limits.

localPool.frontierFallback (default false) mounts DAARI_ROUTING__LOCAL_POOL__FRONTIER_FALLBACK=true when set — chat escalates to L6 when every local-pool host is down. Pair it with frontier.enabled and a frontier API key (doctor warns otherwise).

localPool:
  frontierFallback: true
frontier:
  enabled: true
  apiKeySecret:
    name: daari-frontier
    key: api-key

orgPool:
  enabled: true
  baseUrl: http://gpu-pool.internal:11434

Session affinity and policy sync¶

routing.sessionAffinity (default false) mounts DAARI_ROUTING__SESSION_AFFINITY and DAARI_ROUTING__SESSION_AFFINITY_TTL_SECONDS so sticky sessions survive tool continuations across replicas when Redis backs the cache. Pair with redis.enabled — doctor warns when affinity is on without Redis.

enterprise.policySync.url mounts DAARI_ENTERPRISE__POLICY_SYNC_URL for periodic signed org-config refresh. enterprise.policySync.signingSecret mounts DAARI_ENTERPRISE__CONFIG_SIGNING_SECRET (HMAC for X-Daari-Signature). Empty url / empty secret omit those env vars.

routing:
  sessionAffinity: true
  sessionAffinityTtlSeconds: 1800
redis:
  enabled: true
  url: redis://redis:6379/0
enterprise:
  policySync:
    url: https://policy.example.com/daari/org-config
    signingSecret:
      name: daari-policy
      key: signing-secret

Header policy (pre-auth)¶

server.headerPolicy (default enabled: false) mounts DAARI_SERVER__HEADER_POLICY__ENABLED plus JSON env for required / deny / allow when turned on — the Helm knobs for server.header_policy. Disabled by default (env vars omitted). Covers /mcp as well as chat routes; open probes stay exempt.

server:
  headerPolicy:
    enabled: true
    required: ["X-Client-Id"]
    deny:
      - header: user-agent
        exact: "BadBot/1.0"
    allow:
      x-daari-env: ["prod", "staging"]

Embedder base URL¶

ollama.baseUrl defaults to empty, so the image keeps its localhost Ollama default and the chart does not set DAARI_OLLAMA__BASE_URL. Set it when the embedder should leave the pod — POST /v1/embeddings and L1 both use settings.ollama.base_url. This is separate from orgPool, which only sets DAARI_ROUTING__ORG_POOL__BASE_URL for chat routing.

ollama:
  baseUrl: http://ollama.internal:11434

ASR and TTS base URLs¶

asr.baseUrl / tts.baseUrl default to empty (chart omits env; routes return 501). Set them to mount DAARI_ASR__BASE_URL / DAARI_TTS__BASE_URL for a local OpenAI-compatible speech stack (whisper/vLLM, Kokoro/openedai-speech). Optional asr.model mounts DAARI_ASR__MODEL so the pod always presents the on-box whisper id. asr.frontierFallback defaults to false (chart omits env). Set it to true to mount DAARI_ASR__FRONTIER_FALLBACK so an empty asr.baseUrl may forward one transcription to the configured frontier. Optional tts.model / tts.voice mount DAARI_TTS__MODEL / DAARI_TTS__VOICE so the pod always presents the on-box id to the TTS server.

asr:
  baseUrl: http://whisper.internal:8000/v1
  model: ggml-base
  frontierFallback: false
tts:
  baseUrl: http://kokoro.internal:8880/v1
  model: kokoro
  voice: af_bella

Request deadline and request-log retention¶

upstream.requestDeadlineSeconds and observability.retention.requestLogDays default to empty so the chart does not set env. Set them to emit DAARI_UPSTREAM__REQUEST_DEADLINE_SECONDS (wall-clock budget across escalation hops) and DAARI_OBSERVABILITY__RETENTION__REQUEST_LOG_DAYS (days to keep gateway request-log lines; 0 keeps size-only rotation).

upstream:
  requestDeadlineSeconds: 120
observability:
  retention:
    requestLogDays: 30

OTLP logs¶

observability.otlpLogs (default false) mounts DAARI_OBSERVABILITY__OTLP_LOGS=true. observability.otlpEndpoint mounts OTEL_EXPORTER_OTLP_ENDPOINT when non-empty so the same collector that scrapes traces/metrics also receives gateway events. See OpenTelemetry GenAI. Doctor warns when logs are on without an endpoint.

observability:
  otlpLogs: true
  otlpEndpoint: http://otel-collector:4318

Next¶

→ Org cache · Upgrade and config migration · Batches