sure: new chart for the we-promise Sure app (1.0.0+up0.7.4)
One StatefulSet with two containers - the Puma web process and a Sidekiq worker - sharing one ReadWriteOnce PVC at /rails/storage. That shared volume is the reason they live in one pod: splitting them would need an RWX volume. Only the web container migrates, because the image entrypoint runs db:prepare solely for the './bin/rails server' command; the worker is started through args: so the entrypoint (and its jemalloc preload) stays in place and the two can never race. Probes are chosen for what they prove: liveness and startup use Rails' /up, which is a boot check and touches no database, while readiness uses '/', which runs User.count through authenticate_user! in self-hosted mode and therefore fails while Postgres is unreachable. The worker speaks no HTTP, so all three of its probes ask sidekiqmon whether this pod still holds a Sidekiq heartbeat in Redis. Three OnePasswordItem CRs (app, database, redis), ingress.domain kept required outside the Ingress because APP_DOMAIN and the WebAuthn relying party read it, strict values.schema.json, CI fixtures with throwaway Postgres 16 and Redis, and a Renovate manager on semver so the -alpha/-rc/-hotfix channel and the moving tags are never selected.
This commit is contained in:
1 parent
819d7b2cf7
commit
93fdb3eaea
21 files changed
+1565
-3
No files matched your search
@@ -0,0 +1,280 @@
|
||||
# Scale the whole chart down without uninstalling it: scaleDown: true renders
|
||||
# every Deployment/StatefulSet with replicas: 0 (and suspends CronJobs) and
|
||||
# takes precedence over any per-workload replicas value. PVCs, Secrets and
|
||||
# Services stay in place; scaleDown: false brings the workloads back up.
|
||||
scaleDown: false
|
||||
|
||||
# 1Password item paths (op://<vault>/<item>). The 1Password operator turns each
|
||||
# OnePasswordItem into a Kubernetes Secret whose keys are the FIELD NAMES of the
|
||||
# item; the chart references those field names, so a renamed field is a pod in
|
||||
# CreateContainerConfigError, not a render error. NOTES.txt lists every field.
|
||||
#
|
||||
# Helm never sees the CONTENT of these secrets, so no checksum/config-style
|
||||
# annotation can cover them (README, guideline 9). The pod template does carry a
|
||||
# checksum/secret-ref over the rendered OnePasswordItems, so REPOINTING one of
|
||||
# the *SecretRef values at a different item restarts the pod; editing a value
|
||||
# INSIDE the same item does not, and needs a manual
|
||||
# kubectl rollout restart statefulset/<release>-sure-sfs
|
||||
# The operator's operator.1password.io/auto-restart annotation is deliberately
|
||||
# NOT set: it only ever restarts Deployments, so on this StatefulSet it would
|
||||
# advertise a guarantee that does not exist (README, guideline 9).
|
||||
secrets:
|
||||
# SECRET_KEY_BASE (field: secret-key-base)
|
||||
appSecretRef: null
|
||||
# PostgreSQL connection (fields: host, port, database, username, password)
|
||||
dbSecretRef: null
|
||||
# Redis connection URL (field: connection-url)
|
||||
redisSecretRef: null
|
||||
|
||||
sure:
|
||||
# Replica count for the StatefulSet; 0 is allowed and scales it down. The
|
||||
# chart-global scaleDown flag takes precedence.
|
||||
#
|
||||
# Both containers live in the SAME pod, so this value scales web and worker
|
||||
# together. That is a consequence of the single ReadWriteOnce PVC they share
|
||||
# at /rails/storage (Active Storage): splitting them into two workloads would
|
||||
# need a ReadWriteMany volume. Above 1 the extra pods normally stay Pending on
|
||||
# "Multi-Attach error for volume" - see NOTES.txt.
|
||||
replicas: 1
|
||||
# Pod-level hardening, shared by both containers. The upstream image declares
|
||||
# USER 1000:1000 (group and user "rails", created in the Dockerfile), so the
|
||||
# chart needs no deviation from the repo hardening standard; fsGroup 1000
|
||||
# makes the storage PVC writable for that uid.
|
||||
securityContext:
|
||||
pod:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
runAsGroup: 1000
|
||||
fsGroup: 1000
|
||||
fsGroupChangePolicy: Always
|
||||
# RuntimeDefault applies the container runtime's default seccomp filter,
|
||||
# which blocks the syscalls a normal workload never makes. It sits in the
|
||||
# pod context on purpose: from here it covers every container in the pod
|
||||
# - web and worker alike - including init containers, while a
|
||||
# container-level seccompProfile would have to be repeated for each one
|
||||
# (and silently missed on the next one added). A container that genuinely
|
||||
# needs a different filter can still override it under
|
||||
# <container>.securityContext.container, which wins over this value.
|
||||
seccompProfile:
|
||||
type: RuntimeDefault
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# web - Puma, the container that also runs the database migrations
|
||||
# ---------------------------------------------------------------------------
|
||||
web:
|
||||
securityContext:
|
||||
container:
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
runAsNonRoot: true
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
resources:
|
||||
# Limits are optional: memory and ephemeral-storage limits default to
|
||||
# limitFactor x the request, an explicitly set limit always wins, and a
|
||||
# CPU limit is only set when configured explicitly under "limits".
|
||||
#
|
||||
# The request follows upstream's own sizing note (docs/hosting/docker.md
|
||||
# for 0.7.4): steady-state memory "around 352 MB", comfortable under a
|
||||
# 512 MB limit, while "if you enable AI, run at least 1 GB". 512Mi request
|
||||
# x limitFactor 3 = 1536Mi limit, which holds both the everyday figure and
|
||||
# the AI recommendation without reserving the larger number on the node.
|
||||
# Read from upstream's documentation, not measured here.
|
||||
limitFactor: 3
|
||||
requests:
|
||||
cpu: 0.1
|
||||
memory: 512Mi
|
||||
ephemeral-storage: 10Mi
|
||||
# PROBES - what each one actually proves (README, guideline 12.3)
|
||||
#
|
||||
# /up is Rails' own health route (config/routes.rb: get "up" =>
|
||||
# "rails/health#show"). READ FROM THE RAILS SOURCE, NOT MEASURED HERE: that
|
||||
# controller renders 200 as soon as the app has booted without raising and
|
||||
# touches NO database and NO Redis. It is a boot check, and the widespread
|
||||
# belief that it tests the database is exactly what guideline 12.3 exists to
|
||||
# catch - upstream's own Helm chart (charts/sure in the app repository) uses
|
||||
# /up for readiness, which is the mistake this chart does not copy.
|
||||
#
|
||||
# Liveness therefore uses /up: a Postgres outage must not restart the pod.
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /up
|
||||
port: http
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 5
|
||||
failureThreshold: 3
|
||||
# Readiness uses "/" instead, because that path provably reaches the
|
||||
# database. Read from the source of 0.7.4, not measured: every request runs
|
||||
# ApplicationController's authenticate_user! before_action
|
||||
# (app/controllers/concerns/authentication.rb); without a session cookie -
|
||||
# which a kubelet probe never has - it falls into self_hosted_first_login?,
|
||||
# which is "config.app_mode.self_hosted? && User.count.zero?". User.count is
|
||||
# an SQL query. With the database up the request ends in a 302 to the login
|
||||
# or registration page, which the kubelet counts as success (2xx-3xx); with
|
||||
# the database down the query raises and Rails answers 500.
|
||||
#
|
||||
# Two conditions this depends on, both of them chart-set:
|
||||
# - SELF_HOSTED=true. In managed mode the same code short-circuits before
|
||||
# User.count and "/" would redirect without touching Postgres.
|
||||
# - the probe must not be answered by a redirect BEFORE the controller.
|
||||
# Sure's production.rb defaults RAILS_FORCE_SSL to true, and a forced
|
||||
# redirect to https is a 301 - i.e. a "success" that proves nothing.
|
||||
# The chart sets RAILS_ASSUME_SSL alongside it (see the env block that
|
||||
# the templates always render), so requests are treated as SSL and no
|
||||
# redirect happens. Overriding RAILS_ASSUME_SSL to "false" while
|
||||
# RAILS_FORCE_SSL stays "true" turns this probe back into a lie.
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /
|
||||
port: http
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 5
|
||||
failureThreshold: 3
|
||||
# The startup probe gates liveness and readiness, so neither needs an
|
||||
# initialDelaySeconds. Its budget has to cover the first start, which is the
|
||||
# expensive one: the image entrypoint (/rails/bin/docker-entrypoint) runs
|
||||
# "./bin/rails db:prepare" whenever the command is "./bin/rails server", so
|
||||
# the web container creates and migrates the schema before Puma binds the
|
||||
# port. Budget 10s x 60 = 600s. Raise it before a large app-version jump;
|
||||
# a restart in the middle of a migration is the failure mode it prevents.
|
||||
startupProbe:
|
||||
httpGet:
|
||||
path: /up
|
||||
port: http
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 5
|
||||
failureThreshold: 60
|
||||
# Extra environment variables, appended to the ones the chart always sets.
|
||||
# Values are rendered through tpl; value, secretKeyRef and configMapKeyRef
|
||||
# forms are supported.
|
||||
#
|
||||
# This is also where the AI features go - the chart models none of them as
|
||||
# values on purpose. OPENAI_ACCESS_TOKEN (secretKeyRef form), OPENAI_MODEL,
|
||||
# OPENAI_URI_BASE, LLM_CONTEXT_WINDOW, OPENAI_REQUEST_TIMEOUT,
|
||||
# AI_RESPONSE_TIMEOUT, ASSISTANT_MAX_TOOL_CALL_ITERATIONS and the SMTP_*
|
||||
# block are plain environment variables to this image. Set them on BOTH
|
||||
# containers where a background job needs them as well (the assistant runs
|
||||
# in the worker).
|
||||
#
|
||||
# Additive means additive: an entry repeating a name the chart already sets
|
||||
# reaches the pod twice at install time (the later one, yours, wins) but is
|
||||
# LOST on a later helm upgrade, because Kubernetes merges env by name and
|
||||
# the pair collapses to the chart's own value - with a green upgrade either
|
||||
# way (README, guideline 10).
|
||||
env: []
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# worker - Sidekiq: the same image, started with "bundle exec sidekiq" as
|
||||
# its args (the image entrypoint stays in place; see the StatefulSet)
|
||||
# ---------------------------------------------------------------------------
|
||||
worker:
|
||||
securityContext:
|
||||
container:
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: true
|
||||
runAsNonRoot: true
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
resources:
|
||||
# Same sizing as the web container, for a documented reason: upstream
|
||||
# notes that the sample/demo-data generator runs in the WORKER and is the
|
||||
# one operation that deterministically exceeds 512 MB (observed flat at
|
||||
# ~680 MB on a 2 GB container). The computed limit of 3 x 512Mi = 1536Mi
|
||||
# covers that; the request stays at the everyday figure.
|
||||
limitFactor: 3
|
||||
requests:
|
||||
cpu: 0.1
|
||||
memory: 512Mi
|
||||
ephemeral-storage: 10Mi
|
||||
# PROBES - the worker speaks no HTTP at all, so all three are exec probes.
|
||||
#
|
||||
# The command asks Sidekiq's own monitor whether THIS process is in the
|
||||
# process set: sidekiqmon prints one line per registered process, each
|
||||
# starting with the identity "<hostname>:<pid>:<nonce>" (sidekiq 8.0.5,
|
||||
# lib/sidekiq/component.rb), and in a pod the hostname is the pod name that
|
||||
# bash exposes as $HOSTNAME. Sidekiq keeps that registration alive with a
|
||||
# heartbeat whose Redis key expires after about 60s, so a process that hangs
|
||||
# disappears from the set while its PID still exists - which is precisely
|
||||
# what a process check like "pgrep sidekiq" could not tell you (and would
|
||||
# not tell you anything at all: Sidekiq is PID 1 here, so if it dies the
|
||||
# container dies with it and the check can never fail - a check a broken
|
||||
# system passes, README).
|
||||
#
|
||||
# READ FROM THE SIDEKIQ 8.0.5 SOURCE, NOT MEASURED: sidekiqmon is a plain
|
||||
# Ruby entry point that requires sidekiq/api only - it does NOT boot Rails -
|
||||
# and it reads REDIS_URL from the environment, which the chart sets on this
|
||||
# container. Cost per check is therefore one Ruby start, not an application
|
||||
# boot; the periods below are chosen on the assumption that this takes a few
|
||||
# seconds, and the PR install-test is the first place it actually runs.
|
||||
#
|
||||
# Liveness hangs on Redis, which guideline 12.3 allows only where no
|
||||
# dependency-free path proves anything - that is the case here (see above) -
|
||||
# and it is given a deliberately generous budget of 60s x 5 = 300s of
|
||||
# missing heartbeat, so a short Redis blip does not restart the pod.
|
||||
livenessProbe:
|
||||
exec:
|
||||
command:
|
||||
- /bin/bash
|
||||
- -c
|
||||
- bundle exec sidekiqmon processes | grep -q "^${HOSTNAME}:"
|
||||
periodSeconds: 60
|
||||
timeoutSeconds: 15
|
||||
failureThreshold: 5
|
||||
# Readiness uses the same check with a tighter budget: a worker that is not
|
||||
# registered in Redis is not doing any work, whatever its process state.
|
||||
readinessProbe:
|
||||
exec:
|
||||
command:
|
||||
- /bin/bash
|
||||
- -c
|
||||
- bundle exec sidekiqmon processes | grep -q "^${HOSTNAME}:"
|
||||
periodSeconds: 30
|
||||
timeoutSeconds: 15
|
||||
failureThreshold: 3
|
||||
# The startup probe gates the other two. Sidekiq boots the full Rails app
|
||||
# before it registers, and on a first install it waits for the web container
|
||||
# to finish db:prepare, so the budget matches the web container's: 20s x 30
|
||||
# = 600s.
|
||||
startupProbe:
|
||||
exec:
|
||||
command:
|
||||
- /bin/bash
|
||||
- -c
|
||||
- bundle exec sidekiqmon processes | grep -q "^${HOSTNAME}:"
|
||||
periodSeconds: 20
|
||||
timeoutSeconds: 15
|
||||
failureThreshold: 30
|
||||
# Extra environment variables for the worker, appended to the ones the chart
|
||||
# always sets. Same rules as sure.web.env above.
|
||||
env: []
|
||||
|
||||
# The Active Storage volume, mounted at /rails/storage in BOTH containers.
|
||||
# Deliberately not null-safe: an erased block yields an invalid PVC, which fails
|
||||
# loudly (README, guideline 12).
|
||||
storage:
|
||||
amount: 10Gi
|
||||
storageClass: null
|
||||
|
||||
ingress:
|
||||
# Gates the whole Ingress template. Note that ingress.domain is read three
|
||||
# more times outside it - APP_DOMAIN (the host Action Mailer builds link URLs
|
||||
# from), WEBAUTHN_RP_ID and WEBAUTHN_ALLOWED_ORIGINS (the relying party a
|
||||
# passkey is bound to) - so it keeps its own required() there and stays
|
||||
# mandatory with the Ingress switched off; only clusterIssuer becomes
|
||||
# unnecessary (README, guideline 12.1). Without that guard the app would come
|
||||
# up with WEBAUTHN_RP_ID="" and register passkeys against the wrong relying
|
||||
# party - rendered cleanly, pod green.
|
||||
#
|
||||
# ingress.enabled also decides how the app is told about TLS: enabled renders
|
||||
# RAILS_ASSUME_SSL=true and RAILS_FORCE_SSL=true (the Ingress terminates TLS,
|
||||
# so Rails must trust that and may set secure cookies and HSTS), disabled
|
||||
# renders both as "false" so a plain-HTTP port-forward can log in at all.
|
||||
enabled: true
|
||||
clusterIssuer: null
|
||||
domain: null
|
||||
className: nginx
|
||||
# Extra annotations for the Ingress; merged with the cert-manager
|
||||
# cluster-issuer annotation the chart always sets.
|
||||
annotations: {}
|
||||
Reference in new issue
Block a user