sure: new chart for the we-promise Sure app (1.0.0+up0.7.4)
One StatefulSet with two containers - the Puma web process and a Sidekiq worker - sharing one ReadWriteOnce PVC at /rails/storage. That shared volume is the reason they live in one pod: splitting them would need an RWX volume. Only the web container migrates, because the image entrypoint runs db:prepare solely for the './bin/rails server' command; the worker is started through args: so the entrypoint (and its jemalloc preload) stays in place and the two can never race. Probes are chosen for what they prove: liveness and startup use Rails' /up, which is a boot check and touches no database, while readiness uses '/', which runs User.count through authenticate_user! in self-hosted mode and therefore fails while Postgres is unreachable. The worker speaks no HTTP, so all three of its probes ask sidekiqmon whether this pod still holds a Sidekiq heartbeat in Redis. Three OnePasswordItem CRs (app, database, redis), ingress.domain kept required outside the Ingress because APP_DOMAIN and the WebAuthn relying party read it, strict values.schema.json, CI fixtures with throwaway Postgres 16 and Redis, and a Renovate manager on semver so the -alpha/-rc/-hotfix channel and the moving tags are never selected.
This commit is contained in:
jklein authored and 
Justin Klein committed
1 parent
819d7b2cf7
commit
93fdb3eaea
21 files changed
+1565
-3
No files matched your search
@@ -0,0 +1,165 @@
|
||||
{{- $replicas := include "sure.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.sure.replicas) -}}
|
||||
{{- $webStartup := fromYaml (include "sure.webStartupProbe" .Values.sure.web.startupProbe) -}}
|
||||
{{- $workerStartup := fromYaml (include "sure.workerStartupProbe" .Values.sure.worker.startupProbe) -}}
|
||||
{{- $workerLiveness := fromYaml (include "sure.workerLivenessProbe" .Values.sure.worker.livenessProbe) -}}
|
||||
Sure {{ .Chart.AppVersion }} — release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
|
||||
|
||||
{{- if .Values.ingress.enabled }}
|
||||
|
||||
ACCESS
|
||||
https://{{ .Values.ingress.domain }}
|
||||
|
||||
The certificate is requested by cert-manager through the "{{ .Values.ingress.clusterIssuer }}"
|
||||
cluster issuer. Until it has been issued the browser shows a warning; follow it with
|
||||
|
||||
kubectl -n {{ .Release.Namespace }} get certificate tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
|
||||
|
||||
The first account created on a fresh instance becomes the super admin — sign
|
||||
in yourself before handing the URL to anyone.
|
||||
{{- else }}
|
||||
|
||||
ACCESS — NO INGRESS
|
||||
ingress.enabled is false, so this release has no Ingress and no certificate.
|
||||
ingress.domain is still required: it becomes APP_DOMAIN (the host in every
|
||||
mail link) and the WebAuthn relying party the app binds passkeys to. Reach
|
||||
the instance inside the cluster with
|
||||
|
||||
kubectl -n {{ .Release.Namespace }} port-forward svc/{{ .Release.Name }}-{{ .Chart.Name }}-service 8080:80
|
||||
|
||||
and open http://localhost:8080. With the Ingress off the chart also renders
|
||||
RAILS_ASSUME_SSL and RAILS_FORCE_SSL as "false", which is what makes that
|
||||
plain-HTTP session able to log in at all; mail links still point at
|
||||
https://{{ .Values.ingress.domain }}/.
|
||||
{{- end }}
|
||||
{{ if eq $replicas "0" }}
|
||||
SCALED TO ZERO
|
||||
{{ if .Values.scaleDown }}scaleDown: true{{ else }}sure.replicas: 0{{ end }} — the StatefulSet runs with replicas: 0, so neither the web nor
|
||||
the worker process is running. PVC, Secrets, Service{{ if .Values.ingress.enabled }}, Ingress and the
|
||||
certificate stay{{ else }} stay{{ end }} in place; set it back and upgrade to bring both containers up.
|
||||
{{ end }}
|
||||
WHAT THE THREE 1PASSWORD ITEMS MUST CONTAIN
|
||||
The chart creates OnePasswordItem CRs; the operator turns each into a
|
||||
Kubernetes Secret whose keys are the FIELD NAMES of the 1Password item. A
|
||||
missing field does not fail the install — the release goes through and the
|
||||
pod then hangs in CreateContainerConfigError naming the key it could not
|
||||
find (kubectl -n {{ .Release.Namespace }} describe pod).
|
||||
|
||||
secrets.appSecretRef → {{ .Values.secrets.appSecretRef }}
|
||||
Secret {{ .Release.Name }}-{{ .Chart.Name }}-secret, field:
|
||||
secret-key-base SECRET_KEY_BASE, 64 random bytes hex
|
||||
(openssl rand -hex 64)
|
||||
|
||||
secrets.dbSecretRef → {{ .Values.secrets.dbSecretRef }}
|
||||
Secret {{ .Release.Name }}-{{ .Chart.Name }}-db-secret, fields:
|
||||
host DB_HOST, the PostgreSQL hostname
|
||||
port DB_PORT, usually 5432
|
||||
database POSTGRES_DB
|
||||
username POSTGRES_USER
|
||||
password POSTGRES_PASSWORD
|
||||
|
||||
secrets.redisSecretRef → {{ .Values.secrets.redisSecretRef }}
|
||||
Secret {{ .Release.Name }}-{{ .Chart.Name }}-redis-secret, field:
|
||||
connection-url REDIS_URL, e.g. redis://:password@host:6379/0
|
||||
|
||||
ROTATING SECRET_KEY_BASE DESTROYS MORE THAN SESSIONS
|
||||
In self-hosted mode Sure derives its Active Record encryption keys from
|
||||
SECRET_KEY_BASE (config/initializers/active_record_encryption.rb: three
|
||||
SHA-256 digests over that one value) unless
|
||||
ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY and its two companions are set. Every
|
||||
encrypted column in the database — provider tokens, OAuth credentials, MFA
|
||||
secrets — is therefore readable only with the SECRET_KEY_BASE it was written
|
||||
under. Replacing that value does not just log everyone out: it makes the
|
||||
existing encrypted data undecryptable, and no rotation of the item brings it
|
||||
back. Treat this field as permanent for the life of the database, or pin the
|
||||
three encryption keys explicitly through sure.web.env / sure.worker.env
|
||||
before the first start.
|
||||
|
||||
CHANGING A SECRET'S CONTENT RESTARTS NOTHING
|
||||
Helm never sees the content of a OnePasswordItem, so it cannot hash it. The
|
||||
pod template carries checksum/secret-ref annotations over the rendered
|
||||
OnePasswordItems, so pointing a *SecretRef at a DIFFERENT item rolls the
|
||||
pod — but editing a value INSIDE the same item does not: the operator
|
||||
updates the Kubernetes Secret while both containers keep the old value in
|
||||
their environment. After such a change:
|
||||
|
||||
kubectl -n {{ .Release.Namespace }} rollout restart statefulset/{{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
||||
|
||||
The operator's auto-restart annotation would not help here and is
|
||||
deliberately not set: it restarts Deployments only, and this is a
|
||||
StatefulSet.
|
||||
|
||||
POSTGRES AND REDIS ARE NOT PART OF THIS CHART
|
||||
Both must exist and be reachable before the first start. Redis is not a
|
||||
cache the app can do without — it holds the Sidekiq queues, the Rails cache
|
||||
store and the rate limiter.
|
||||
|
||||
What the probes do with that, per container:
|
||||
web liveness and startup use /up, which is Rails' boot check and
|
||||
touches no database — a Postgres outage therefore does not restart
|
||||
the pod. Readiness uses "/", which runs a User.count query, so the
|
||||
pod leaves the Service while the database is gone and returns on
|
||||
its own afterwards.
|
||||
worker all three probes run "sidekiqmon processes" and look for this
|
||||
pod's own entry, i.e. a live Sidekiq heartbeat in Redis. There is
|
||||
no HTTP and no dependency-free alternative: with Redis gone the
|
||||
worker has nothing to do, and liveness only restarts after
|
||||
{{ $workerLiveness.failureThreshold }} x {{ $workerLiveness.periodSeconds }}s = {{ mul $workerLiveness.failureThreshold $workerLiveness.periodSeconds }}s of missing heartbeat.
|
||||
|
||||
A pod that never turns Ready is usually a connection problem, not a chart
|
||||
problem: read the container logs, not the pod events.
|
||||
|
||||
kubectl -n {{ .Release.Namespace }} logs statefulset/{{ .Release.Name }}-{{ .Chart.Name }}-sfs -c {{ .Release.Name }}-{{ .Chart.Name }}-web
|
||||
|
||||
THE FIRST START MIGRATES, AND THE STARTUP PROBE IS THE CLOCK
|
||||
The image entrypoint runs "./bin/rails db:prepare" — but only when the
|
||||
command is "./bin/rails server". That is the web container, and only the web
|
||||
container: the worker runs "bundle exec sidekiq" and never migrates, so the
|
||||
two can never race. Puma binds the port after the migration, which means the
|
||||
whole schema creation happens inside the startup probe's budget:
|
||||
|
||||
web {{ $webStartup.failureThreshold }} x {{ $webStartup.periodSeconds }}s = {{ mul $webStartup.failureThreshold $webStartup.periodSeconds }}s
|
||||
worker {{ $workerStartup.failureThreshold }} x {{ $workerStartup.periodSeconds }}s = {{ mul $workerStartup.failureThreshold $workerStartup.periodSeconds }}s (it waits for the schema the web container creates)
|
||||
|
||||
Raise sure.web.startupProbe.failureThreshold BEFORE a large app-version jump
|
||||
or a first import, not after the crash loop has started — a restart lands in
|
||||
the middle of a migration. Back up the database before any app-version bump:
|
||||
migrations are one-way, and a helm rollback returns the old image to a
|
||||
database the new one has already rewritten.
|
||||
|
||||
ONE POD, ONE VOLUME, TWO CONTAINERS
|
||||
Uploads, imports and exports live on the ReadWriteOnce PVC
|
||||
{{ .Release.Name }}-{{ .Chart.Name }}-pvc, mounted at /rails/storage in both containers. That is
|
||||
the reason web and worker share a pod at all: a background job writes the
|
||||
same files the web process serves, and one RWO volume cannot be attached to
|
||||
two pods.
|
||||
{{- if gt (int $replicas) 1 }}
|
||||
|
||||
sure.replicas is {{ $replicas }}. Expect the extra pods to stay Pending on
|
||||
"Multi-Attach error for volume" unless the storage class is genuinely RWX.
|
||||
{{- end }}
|
||||
|
||||
Backup is two things at the same time: the database holds the accounts and
|
||||
one row per attachment, the PVC holds the bytes. Restoring one without the
|
||||
other gives an app full of broken references. Because the PVC is rendered by
|
||||
the chart, helm uninstall deletes it — the external database survives that,
|
||||
which makes an accidental uninstall look recoverable while the files are
|
||||
already gone.
|
||||
|
||||
THE CHART CONFIGURES NO MAIL AND NO AI
|
||||
Sure sends invitations and notifications over SMTP and can run an assistant
|
||||
against an OpenAI-compatible endpoint. This chart sets neither: no SMTP_*
|
||||
variable, no OPENAI_* variable. Mails are therefore quietly never sent, and
|
||||
the AI features stay off — nothing in the UI says so.
|
||||
|
||||
Both are plain environment variables, so they go into sure.web.env and, for
|
||||
anything a background job needs (the assistant runs in the worker), into
|
||||
sure.worker.env as well. The secretKeyRef form is available for the API key
|
||||
and the SMTP password. Note that upstream sizes AI usage at 1 GB of memory
|
||||
per process; the chart's computed memory limit is 3x its request.
|
||||
{{- if or .Values.sure.web.env .Values.sure.worker.env }}
|
||||
|
||||
Currently set: {{ len .Values.sure.web.env }} extra variable(s) on the web container,
|
||||
{{ len .Values.sure.worker.env }} on the worker, rendered after the ones the chart sets itself.
|
||||
An entry repeating a name the chart already sets wins at install time and is
|
||||
silently lost on the next helm upgrade — read it back off the StatefulSet.
|
||||
{{- end }}
|
||||
Reference in new issue
Block a user