sure: fix the readiness probe, the ephemeral-storage budget and HOME
Second install-test run, three findings, all measured: - Readiness on "/" failed with "http: server gave HTTP response to HTTPS client" on a pod that was answering correctly. The kubelet's HTTP prober follows redirects to the same host, and with RAILS_ASSUME_SSL=true the app builds its redirect as https://<pod-ip>:3000/registration/new, which Puma does not speak. The probe now runs curl without -L and accepts 2xx-3xx itself. The same log also proves what the probe is for: against an empty database the redirect target was /registration/new, the branch User.count decides, so the path really does reach Postgres. - The pod was evicted within two minutes of every start: "ephemeral local storage usage exceeds the total limit of containers 60Mi". emptyDirs and the stdout log both count there, and the image rebuilds its Bootsnap cache under /rails/tmp on every start because the emptyDir hides the one baked into the image. The request is now 256Mi (limit 768Mi) per app container - headroom, not a measurement. - /home/rails is read-only, so Bundler created a throwaway home under TMPDIR on every invocation, worker probe included. HOME now points at a directory the init container creates. The CI postgres fixture gets -d sure on its readiness probe, so the server stops logging a missing database that nothing asked for.
This commit was merged in pull request #36.
This commit is contained in:
1 parent
a414af089d
commit
d9267b2180
4 files changed
+83
-39
No files matched your search
@@ -237,11 +237,15 @@ while the database is unreachable. The long startup budget covers the
|
||||
{{- include "sure.defaultedDict" (dict "defaults" $defaults "given" . "path" "sure.web.livenessProbe") -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "sure.webReadinessCommand" -}}
|
||||
{{- list "/bin/bash" "-c" "code=\"$(curl -sS -o /dev/null -w '%{http_code}' --max-time 5 http://127.0.0.1:3000/)\"; test \"$code\" -ge 200 -a \"$code\" -lt 400" | toJson -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "sure.webReadinessProbe" -}}
|
||||
{{- $defaults := dict
|
||||
"httpGet" (dict "path" "/" "port" "http")
|
||||
"exec" (dict "command" (include "sure.webReadinessCommand" . | fromJsonArray))
|
||||
"periodSeconds" 10
|
||||
"timeoutSeconds" 5
|
||||
"timeoutSeconds" 10
|
||||
"failureThreshold" 3 -}}
|
||||
{{- include "sure.defaultedDict" (dict "defaults" $defaults "given" . "path" "sure.web.readinessProbe") -}}
|
||||
{{- end -}}
|
||||
@@ -301,7 +305,7 @@ comes from.
|
||||
{{- define "sure.webResources" -}}
|
||||
{{- $defaults := dict
|
||||
"limitFactor" 3
|
||||
"requests" (dict "cpu" 0.1 "memory" "512Mi" "ephemeral-storage" "10Mi") -}}
|
||||
"requests" (dict "cpu" 0.1 "memory" "512Mi" "ephemeral-storage" "256Mi") -}}
|
||||
{{- $merged := fromYaml (include "sure.defaultedDict" (dict "defaults" $defaults "given" . "path" "sure.web.resources")) -}}
|
||||
{{- include "sure.resources" $merged -}}
|
||||
{{- end -}}
|
||||
@@ -309,7 +313,7 @@ comes from.
|
||||
{{- define "sure.workerResources" -}}
|
||||
{{- $defaults := dict
|
||||
"limitFactor" 3
|
||||
"requests" (dict "cpu" 0.1 "memory" "512Mi" "ephemeral-storage" "10Mi") -}}
|
||||
"requests" (dict "cpu" 0.1 "memory" "512Mi" "ephemeral-storage" "256Mi") -}}
|
||||
{{- $merged := fromYaml (include "sure.defaultedDict" (dict "defaults" $defaults "given" . "path" "sure.worker.resources")) -}}
|
||||
{{- include "sure.resources" $merged -}}
|
||||
{{- end -}}
|
||||
@@ -343,6 +347,15 @@ probe on "/" reach the database at all (see values.yaml).
|
||||
# init container in the StatefulSet, which explains the whole case.
|
||||
- name: TMPDIR
|
||||
value: "/tmp/ruby"
|
||||
# The image's home directory (/home/rails) is part of the read-only root
|
||||
# filesystem, and Bundler reacts to that by creating a throwaway home under
|
||||
# TMPDIR on EVERY invocation ("`/home/rails` is not writable. Bundler will use
|
||||
# `/tmp/ruby/bundler2026...` as your home directory temporarily", measured in
|
||||
# the install-test). That is one directory per `bundle exec` - including every
|
||||
# worker probe, every 30 seconds. Pointing HOME at a writable directory turns
|
||||
# that into one stable path.
|
||||
- name: HOME
|
||||
value: "/tmp/home"
|
||||
- name: PORT
|
||||
value: "3000"
|
||||
# Sure's production.rb defaults BOTH of these to true. The chart renders them
|
||||
|
||||
@@ -53,10 +53,12 @@ spec:
|
||||
# file yet.
|
||||
#
|
||||
# The fix is a directory that is writable without being world-writable.
|
||||
# This init container creates one per app container inside the same
|
||||
# emptyDirs (mode 0700, owned by the pod's uid) and both containers get
|
||||
# TMPDIR pointed at it. It runs as the same non-root uid as the app and
|
||||
# needs no capability at all: the emptyDir root is 0777, so creating an
|
||||
# This init container creates two of them per app container inside the
|
||||
# same emptyDirs (mode 0700, owned by the pod's uid): "ruby" for TMPDIR,
|
||||
# and "home" for HOME - the image's /home/rails is read-only, and
|
||||
# Bundler answers that by creating a throwaway home under TMPDIR on
|
||||
# every single invocation. It runs as the same non-root uid as the app
|
||||
# and needs no capability at all: the emptyDir root is 0777, so creating an
|
||||
# entry in it is allowed, and chmod on a directory one has just created
|
||||
# needs no privilege. That is deliberately weaker than the chown-as-root
|
||||
# init container paperless-ngx needs.
|
||||
@@ -71,7 +73,7 @@ spec:
|
||||
- /bin/bash
|
||||
- -ec
|
||||
- |
|
||||
for dir in /web-tmp/ruby /worker-tmp/ruby; do
|
||||
for dir in /web-tmp/ruby /web-tmp/home /worker-tmp/ruby /worker-tmp/home; do
|
||||
mkdir -p "$dir"
|
||||
chmod 700 "$dir"
|
||||
done
|
||||
|
||||
Reference in new issue
Block a user