sure: new chart for the we-promise Sure app (1.0.0+up0.7.4) #36

Merged
jklein merged 3 commits from feat/sure-chart into main 2026-09-26 23:08:56 +02:00
3 Commits
Author SHA1 Message Date
jklein ee62607e2e sure: fix the readiness probe, the ephemeral-storage budget and HOME
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 8s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Successful in 4s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 10s
Validate Helm Charts / Install-test ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }} (pull_request) Successful in 2m37s
Second install-test run, three findings, all measured:

- Readiness on "/" failed with "http: server gave HTTP response to HTTPS
  client" on a pod that was answering correctly. The kubelet's HTTP prober
  follows redirects to the same host, and with RAILS_ASSUME_SSL=true the
  app builds its redirect as https://<pod-ip>:3000/registration/new, which
  Puma does not speak. The probe now runs curl without -L and accepts
  2xx-3xx itself. The same log also proves what the probe is for: against
  an empty database the redirect target was /registration/new, the branch
  User.count decides, so the path really does reach Postgres.
- The pod was evicted within two minutes of every start: "ephemeral local
  storage usage exceeds the total limit of containers 60Mi". emptyDirs and
  the stdout log both count there, and the image rebuilds its Bootsnap
  cache under /rails/tmp on every start because the emptyDir hides the one
  baked into the image. The request is now 256Mi (limit 768Mi) per app
  container - headroom, not a measurement.
- /home/rails is read-only, so Bundler created a throwaway home under
  TMPDIR on every invocation, worker probe included. HOME now points at a
  directory the init container creates.

The CI postgres fixture gets -d sure on its readiness probe, so the server
stops logging a missing database that nothing asked for.
2026-09-22 20:43:53 +00:00
jklein e92286b420 sure: give Ruby a temporary directory it accepts
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 7s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Successful in 4s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 10s
Validate Helm Charts / Install-test ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }} (pull_request) Failing after 16m0s
The first install-test run failed with both containers crash-looping
during boot:

  system temporary path is world-writable: /tmp
  /tmp is world-writable: /tmp
  . is not writable: /rails
  ArgumentError: could not find a temporary directory

raised from MiniMagick, which Active Storage loads while Rails boots.
Ruby's Dir.tmpdir skips a candidate that is world-writable without a
sticky bit, and a Kubernetes emptyDir is created mode 0777 with no sticky
bit - so with a read-only root filesystem there was no candidate left.

A tiny init container, the app's own image running as the same non-root
uid and with no capability added, now creates one private subdirectory
(mode 0700) inside each tmp emptyDir, and both containers run with
TMPDIR pointing at it. Creating an entry in a 0777 directory and chmod'ing
what one just created needs no privilege, so the hardening standard holds
unchanged.
2026-09-22 20:19:16 +00:00
jklein 0b36d91275 sure: new chart for the we-promise Sure app (1.0.0+up0.7.4)
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 8s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Successful in 4s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 10s
Validate Helm Charts / Install-test ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }} (pull_request) Failing after 15m54s
One StatefulSet with two containers - the Puma web process and a Sidekiq
worker - sharing one ReadWriteOnce PVC at /rails/storage. That shared
volume is the reason they live in one pod: splitting them would need an
RWX volume. Only the web container migrates, because the image entrypoint
runs db:prepare solely for the './bin/rails server' command; the worker is
started through args: so the entrypoint (and its jemalloc preload) stays
in place and the two can never race.

Probes are chosen for what they prove: liveness and startup use Rails'
/up, which is a boot check and touches no database, while readiness uses
'/', which runs User.count through authenticate_user! in self-hosted mode
and therefore fails while Postgres is unreachable. The worker speaks no
HTTP, so all three of its probes ask sidekiqmon whether this pod still
holds a Sidekiq heartbeat in Redis.

Three OnePasswordItem CRs (app, database, redis), ingress.domain kept
required outside the Ingress because APP_DOMAIN and the WebAuthn relying
party read it, strict values.schema.json, CI fixtures with throwaway
Postgres 16 and Redis, and a Renovate manager on semver so the
-alpha/-rc/-hotfix channel and the moving tags are never selected.
2026-09-22 19:54:00 +00:00