The chart told operators that adding a duplicate env name to an installed release makes the next upgrade fail in the API server, quoting "The order in patch list ... doesn't match $setElementOrder list". Measured against helm 3.19.0 and Kubernetes 1.35, it does not fail: the upgrade succeeds, helm prints "Release has been upgraded", and Kubernetes merges the env list by name, so the duplicate pair collapses to the chart's own entry and the operator's override is discarded without a word. At install time the same values behave differently - both entries reach the pod, the later one wins, and the API server says so with "hides previous definition of ..., which may be dropped when using apply". The advice that followed from the old claim survives - such a switch only takes at install time - but the stated reason was wrong in the dangerous direction. A reader who expects a loud failure treats a green upgrade as proof the override took effect, and green is exactly what a silently dropped override looks like. The text now names the real failure mode and tells the reader to read the value back off the Deployment. README carries the same correction, because every chart in this repo offers the same additive env block. Also records what the earlier measurement left open. With the database scaled back up the pod returns to Ready on its own after about five seconds, restart count still 0, so leaving the liveness group untouched costs no manual recovery. And the limit of that signal belongs next to it: the db indicator validates a connection, not the schema, so a database that came back empty still put the pod back to Ready with /actuator/health at 200.
364 lines
19 KiB
Plaintext
364 lines
19 KiB
Plaintext
{{- $replicas := include "urlaubsverwaltung.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.urlaubsverwaltung.replicas) -}}
|
|
{{- $res := fromYaml (include "urlaubsverwaltung.effectiveResources" .Values.urlaubsverwaltung.resources) -}}
|
|
{{/*
|
|
Both figures go through formatNumber for the reason issue #157 documents: they
|
|
come back from fromYaml as float64, and a bare byte count would print as
|
|
"1e+06" here just as it did in the rendered manifest.
|
|
*/}}
|
|
{{- $memLimitRaw := index ($res.limits | default dict) "memory" -}}
|
|
{{- $memLimit := "the computed memory limit" -}}
|
|
{{- if not (kindIs "invalid" $memLimitRaw) -}}
|
|
{{- $memLimit = include "urlaubsverwaltung.formatNumber" $memLimitRaw -}}
|
|
{{- end -}}
|
|
{{- $memRequestRaw := index ($res.requests | default dict) "memory" -}}
|
|
{{- $memRequest := "the memory request" -}}
|
|
{{- if not (kindIs "invalid" $memRequestRaw) -}}
|
|
{{- $memRequest = include "urlaubsverwaltung.formatNumber" $memRequestRaw -}}
|
|
{{- end -}}
|
|
{{/*
|
|
The effective limitFactor. It cannot be read back out of $res: that dict is the
|
|
RENDERED resources block (requests / limits only), so $res.limitFactor is always
|
|
empty and would silently print the default even where another factor is in
|
|
force. Mirror the defaulting of the resources helper on the raw value instead.
|
|
*/}}
|
|
{{- $resIn := .Values.urlaubsverwaltung.resources -}}
|
|
{{- if kindIs "invalid" $resIn -}}
|
|
{{- $resIn = dict -}}
|
|
{{- end -}}
|
|
{{- $factor := 3 -}}
|
|
{{- if not (kindIs "invalid" (index $resIn "limitFactor")) -}}
|
|
{{- $factor = index $resIn "limitFactor" -}}
|
|
{{- end -}}
|
|
{{/* An explicitly set memory limit wins over the computed one - say which. */}}
|
|
{{- $limitsIn := index $resIn "limits" -}}
|
|
{{- if kindIs "invalid" $limitsIn -}}
|
|
{{- $limitsIn = dict -}}
|
|
{{- end -}}
|
|
{{- $memLimitExplicit := not (kindIs "invalid" (index $limitsIn "memory")) -}}
|
|
{{- $startup := fromYaml (include "urlaubsverwaltung.startupProbe" .Values.urlaubsverwaltung.startupProbe) -}}
|
|
{{- $budget := "" -}}
|
|
{{- if and (kindIs "float64" $startup.periodSeconds) (kindIs "float64" $startup.failureThreshold) -}}
|
|
{{- $budget = printf "%.0f seconds" (mulf $startup.periodSeconds $startup.failureThreshold) -}}
|
|
{{- end -}}
|
|
{{- $envNames := list -}}
|
|
{{- range $entry := .Values.urlaubsverwaltung.env }}
|
|
{{- $envNames = append $envNames $entry.name }}
|
|
{{- end }}
|
|
Urlaubsverwaltung {{ .Chart.AppVersion }} — release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
|
|
|
|
{{- if .Values.ingress.enabled }}
|
|
|
|
ACCESS
|
|
https://{{ .Values.ingress.domain }}
|
|
|
|
The certificate is requested by cert-manager through the "{{ .Values.ingress.clusterIssuer }}"
|
|
cluster issuer. Until it has been issued the browser shows a warning; follow it with
|
|
|
|
kubectl -n {{ .Release.Namespace }} get certificate tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
|
|
|
|
The Ingress publishes the application at "/" with no path restriction, so
|
|
everything the app serves is served under this host — including
|
|
/swagger-ui.html and /v3/api-docs, which this app version enables by default
|
|
and this chart does not turn off. Both are behind the login (they answer 302
|
|
to the identity provider like every other path), so this is exposure of an
|
|
API description, not of data. Turning them off is a
|
|
springdoc.api-docs.enabled / springdoc.swagger-ui.enabled property; the chart
|
|
sets neither, so it can be added through urlaubsverwaltung.env.
|
|
{{- else }}
|
|
|
|
ACCESS — NO INGRESS
|
|
ingress.enabled is false, so this release has no Ingress and no certificate;
|
|
ingress.clusterIssuer is not needed. Unlike other charts in this repo,
|
|
ingress.domain is read by NOTHING else here — no environment variable of the
|
|
container derives from it. The address the application considers its own is
|
|
the OIDC redirect URI, and that comes out of the 1Password item below, not
|
|
out of this value. The two can drift apart without any complaint.
|
|
|
|
Reach the release inside the cluster with
|
|
|
|
kubectl -n {{ .Release.Namespace }} port-forward svc/{{ .Release.Name }}-{{ .Chart.Name }}-service 8080:80
|
|
|
|
and open http://localhost:8080. The login redirects to your identity
|
|
provider, which will refuse to come back to a localhost callback unless that
|
|
exact URL is registered there as well.
|
|
{{- end }}
|
|
{{ if eq $replicas "0" }}
|
|
SCALED TO ZERO
|
|
{{ if .Values.scaleDown }}scaleDown: true{{ else }}urlaubsverwaltung.replicas: 0{{ end }} — the Deployment runs with replicas: 0, so the endpoint
|
|
above does not answer until you set it back and upgrade again. Both
|
|
OnePasswordItem CRs and the Service{{ if .Values.ingress.enabled }} and Ingress stay in place, so the
|
|
certificate is not re-issued.{{ else }} stay in place.{{ end }} This chart owns no volume: all
|
|
state lives in the external database, so scaling to zero loses nothing.
|
|
{{ end }}
|
|
SECRETS — THE FIELDS EACH 1PASSWORD ITEM MUST CARRY
|
|
The operator copies an item's FIELDS into the Kubernetes Secret, one key per
|
|
field name, and this chart reads each key by name into an environment
|
|
variable. The field names are decided here and appear nowhere in 1Password,
|
|
so they are exactly what you would otherwise have to read out of the
|
|
deployment template. They must match to the letter.
|
|
|
|
secrets.databaseSecretRef → {{ .Values.secrets.databaseSecretRef }}
|
|
→ Secret {{ .Release.Name }}-{{ .Chart.Name }}-db-secret
|
|
jdbc-url a full JDBC URL, e.g.
|
|
jdbc:postgresql://postgres.example.com:5432/urlaubsverwaltung
|
|
username
|
|
password
|
|
|
|
secrets.oidcSecretRef → {{ .Values.secrets.oidcSecretRef }}
|
|
→ Secret {{ .Release.Name }}-{{ .Chart.Name }}-oidc-secret
|
|
client-id
|
|
client-secret
|
|
issuer-name the provider's display name on the login button
|
|
issuer-uri the OIDC issuer, WITHOUT /.well-known/openid-configuration
|
|
jwt-issuer-uri the same issuer for the resource-server side
|
|
redirect-uri the full callback URL. This chart registers the client
|
|
under the id "default", so the path is
|
|
/login/oauth2/code/default — for example
|
|
https://{{ if .Values.ingress.enabled }}{{ .Values.ingress.domain }}{{ else }}example.com{{ end }}/login/oauth2/code/default
|
|
and the same URL has to be registered at the provider.
|
|
|
|
This chart deploys NO database and NO identity provider. Both have to exist
|
|
already and be reachable from this namespace.
|
|
|
|
A field spelled differently yields a Secret without that key. The pod then
|
|
hangs in CreateContainerConfigError while the Secret itself exists, so it
|
|
looks as if the operator had done its job. What it actually delivered:
|
|
|
|
kubectl -n {{ .Release.Namespace }} get secret {{ .Release.Name }}-{{ .Chart.Name }}-db-secret -o jsonpath='{.data}'
|
|
|
|
Pointing a ref at a DIFFERENT item rolls the pod; changing a value INSIDE the
|
|
same item does not. Helm never sees that content, so the rendered Deployment
|
|
is unchanged and this chart stamps no checksum over the secrets — nothing
|
|
restarts, and the pod keeps running with the old values:
|
|
|
|
kubectl -n {{ .Release.Namespace }} rollout restart deployment/{{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
|
|
THE IDENTITY PROVIDER IS REQUIRED AT BOOT, NOT AT LOGIN
|
|
On start the application resolves the provider's configuration itself, with a
|
|
GET on
|
|
|
|
<issuer-uri>/.well-known/openid-configuration
|
|
|
|
and it does this while building its security beans — before the HTTP port
|
|
opens. An issuer-uri that does not resolve, is unreachable from this
|
|
namespace, or answers something other than a discovery document therefore
|
|
does not produce a broken login page: the process exits with
|
|
|
|
Application run failed ... Unable to resolve Configuration with the
|
|
provided Issuer of "<issuer-uri>"
|
|
|
|
and the pod goes into CrashLoopBackOff having never served a request. So a
|
|
provider that is down at rollout time keeps this release from starting at
|
|
all. The reason is only in the log:
|
|
|
|
kubectl -n {{ .Release.Namespace }} logs deployment/{{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
|
|
Note the order: the database migration runs BEFORE this step. A rollout that
|
|
dies on the identity provider has already migrated the schema.
|
|
|
|
STARTUP MIGRATES THE DATABASE, AND ROLLBACK DOES NOT UNMIGRATE
|
|
Liquibase brings the schema up to date on every start, before anything is
|
|
served; against an empty database it creates it from scratch and the app then
|
|
seeds its initial settings, vacation types and sick-note types. The startup
|
|
probe covers that whole window{{ if $budget }} ({{ $budget }}){{ end }} and holds liveness and readiness off
|
|
until it is through. A measured cold start against an empty database — JVM,
|
|
full migration, seeding — took about 30 seconds; an upgrade that carries a
|
|
long migration takes longer, and if it outruns the budget the kubelet kills
|
|
the container mid-migration and the next attempt starts over.
|
|
|
|
helm rollback puts the previous app version back; the schema does not come
|
|
back with it, because it was migrated in place. Take a database backup BEFORE
|
|
an app-version upgrade. There is no volume in this release to back up — the
|
|
database is the whole of the state.
|
|
|
|
A failed rollout does not take the release down: the previous pod keeps
|
|
serving until the new one becomes Ready. "helm upgrade" without --wait
|
|
therefore reports success while the new pod crashloops behind it. Check:
|
|
|
|
kubectl -n {{ .Release.Namespace }} rollout status deployment/{{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
|
|
THE MEMORY LIMIT SIZES THE JAVA HEAP — IT IS NOT JUST A CEILING
|
|
The image is built with Cloud Native Buildpacks, and its memory calculator
|
|
reads the container's memory LIMIT at every start and derives -Xmx and the
|
|
metaspace from it. In this release that limit is {{ $memLimit }}.
|
|
{{- if $memLimitExplicit }}
|
|
It is set explicitly under urlaubsverwaltung.resources.limits.memory, which
|
|
wins over the factor the chart would otherwise apply.
|
|
{{- else }}
|
|
The chart does not set it directly: it is computed as
|
|
urlaubsverwaltung.resources.limitFactor ({{ include "urlaubsverwaltung.formatNumber" $factor }})
|
|
x the memory request ({{ $memRequest }}).
|
|
{{- end }}
|
|
|
|
So the memory limit is not an operational safety margin here, it is a JVM
|
|
tuning knob. Lowering it shrinks the heap; raising it raises the heap.
|
|
A limit chosen too tightly does not merely risk an OOMKill under load —
|
|
the calculator refuses to start at all when what is left after metaspace,
|
|
code cache and the thread stacks is negative, and that failure appears before
|
|
any application log line. The sizing it chose is the first line of the log:
|
|
|
|
kubectl -n {{ .Release.Namespace }} logs deployment/{{ .Release.Name }}-{{ .Chart.Name }}-deployment | head -1
|
|
|
|
THE THREE HEALTH PATHS AND WHAT EACH ONE ANSWERS FOR
|
|
This chart sets two health settings in the deployment template, and they are
|
|
what makes the three paths mean different things:
|
|
|
|
MANAGEMENT_ENDPOINT_HEALTH_GROUP_READINESS_INCLUDE=readinessState,db
|
|
MANAGEMENT_HEALTH_MAIL_ENABLED=false
|
|
|
|
/actuator/health everything the release actually runs: db,
|
|
diskSpace, ping, ssl and the two lifecycle
|
|
states. 200 UP on a healthy release. The mail
|
|
indicator is NOT in it — see the mail section.
|
|
→ point external monitoring here. This is the only path that also fails on
|
|
diskSpace or ssl.
|
|
|
|
/actuator/health/readiness readinessState AND db. Goes DOWN (503) on a
|
|
database outage, which takes the pod out of Ready
|
|
and stops traffic reaching it.
|
|
→ the readinessProbe.
|
|
|
|
/actuator/health/liveness livenessState alone, deliberately WITHOUT db.
|
|
Stays 200 through a database outage.
|
|
→ the livenessProbe and the startupProbe.
|
|
|
|
The split is the point: a database outage has to keep traffic away, but
|
|
restarting the pod does not bring the database back. Adding db to the
|
|
liveness group would turn every outage into a crash loop on top of the
|
|
outage. Measured on this chart, with the database scaled to zero underneath a
|
|
running pod: readiness 503, liveness 200, pod 0/1 Ready, restart count 0 —
|
|
the pod left Ready about 20 seconds after the database went away.
|
|
|
|
The other half of that measurement matters just as much: when the database
|
|
came back, the pod returned to Ready by itself within about five seconds,
|
|
still with restart count 0. Nothing had to be rolled, deleted or scaled by
|
|
hand. That is what leaving the liveness group alone buys — the outage is
|
|
ridden out and then simply over.
|
|
|
|
How readiness actually fails is worth knowing before you tune it. The db
|
|
indicator does not answer 503 quickly — it waits out the JDBC connection
|
|
timeout first, measured at 30 seconds against a database whose Service has no
|
|
endpoints at all. readinessProbe.timeoutSeconds is 5, so the kubelet gives up
|
|
long before the answer arrives and logs
|
|
|
|
Readiness probe failed: ... context deadline exceeded
|
|
|
|
rather than a 503. The pod leaves Ready either way, and faster; the 503 is
|
|
what a client with patience sees. Raising timeoutSeconds past the JDBC
|
|
timeout would swap the timeout for the 503 and delay the reaction — there is
|
|
no reason to.
|
|
|
|
WHAT A READY POD DOES AND DOES NOT PROVE
|
|
Ready now means the pod can reach the database — that is checked on every
|
|
readiness period, not once at boot. It still does not mean the identity
|
|
provider is reachable: OIDC is resolved once at startup (see below) and no
|
|
health indicator watches it afterwards, so a provider that goes down after
|
|
the rollout leaves a Ready pod on which nobody can log in.
|
|
|
|
Ready also proves reachability, not correctness: the db indicator validates a
|
|
connection, it never looks at the schema. Measured here — a database that came
|
|
back EMPTY put the pod straight back to Ready with /actuator/health at 200,
|
|
while not one table existed in it. A restored-but-wrong database reads as
|
|
healthy, so a probe going green after a database incident is not on its own
|
|
evidence that the data came back with it.
|
|
|
|
A database outage now shows up in the pod's status: READY drops to 0/1 while
|
|
STATUS stays Running and RESTARTS stays put. That combination — running, not
|
|
restarting, not Ready — is the signature of a dependency outage rather than
|
|
of a broken release. The reason is in the endpoint, not in the log:
|
|
|
|
kubectl -n {{ .Release.Namespace }} get pod -l app={{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
kubectl -n {{ .Release.Namespace }} port-forward svc/{{ .Release.Name }}-{{ .Chart.Name }}-service 8080:80
|
|
curl -s localhost:8080/actuator/health
|
|
|
|
The body names components only when show-details is on; the chart does not
|
|
set it, so the aggregate answers {"status":"DOWN","groups":[...]} and the
|
|
status code carries the message. To see WHICH component is down, add
|
|
|
|
urlaubsverwaltung.env:
|
|
- name: MANAGEMENT_ENDPOINT_HEALTH_SHOW_DETAILS
|
|
value: always
|
|
|
|
and know what that publishes to anyone who can reach the endpoint: the
|
|
database product, the container's filesystem path and free bytes, the
|
|
certificate chains, and the stack-trace class of whatever failed.
|
|
|
|
Note also that there is no shell in this image: "kubectl exec" cannot open
|
|
one, and neither can a debug sidecar sharing the process namespace run the
|
|
app's own tooling. Everything you can learn from a running pod, you learn
|
|
from its log and its HTTP endpoints.
|
|
|
|
MAIL AND THE APPLICATION URL ARE PLACEHOLDERS UNTIL YOU SET THEM
|
|
The chart hardcodes the mail wiring to values that work nowhere:
|
|
|
|
SPRING_MAIL_HOST=localhost, SPRING_MAIL_PORT=1025
|
|
UV_MAIL_FROM / UV_MAIL_REPLYTO = urlaubsverwaltung@example.org
|
|
UV_CALENDAR_ORGANIZER = organizer@example.org
|
|
UV_MAIL_APPLICATIONURL = http://localhost:8080
|
|
|
|
Nothing in this pod listens on localhost:1025, so no notification mail leaves
|
|
the release. UV_MAIL_APPLICATIONURL is what the app puts into the links
|
|
inside those mails — it is NOT derived from ingress.domain. Left as it is,
|
|
every link points at the reader's own machine.
|
|
|
|
BECAUSE that wiring is a placeholder, the chart sets
|
|
|
|
MANAGEMENT_HEALTH_MAIL_ENABLED=false
|
|
|
|
and the mail indicator is gone from /actuator/health. Otherwise the aggregate
|
|
would answer 503 DOWN on a perfectly healthy release for its whole lifetime,
|
|
and any monitor pointed at it would page forever about a mail server nobody
|
|
configured.
|
|
|
|
THE PART THAT BITES LATER: the day you configure real mail delivery, switch
|
|
the indicator back on in the SAME change —
|
|
|
|
urlaubsverwaltung.env:
|
|
- name: MANAGEMENT_HEALTH_MAIL_ENABLED
|
|
value: "true"
|
|
|
|
Left at false it does not report a false alarm, it reports nothing: the
|
|
endpoint answers 200 while the mail server is unreachable and notifications
|
|
are silently not delivered. The setting is only honest while there is no mail
|
|
server to be honest about.
|
|
|
|
Note that this variable repeats a name the chart already sets, so per the rule
|
|
below it only takes effect when it goes in at INSTALL time. Put into an
|
|
already installed release it does nothing at all, and says so nowhere: helm
|
|
reports a successful upgrade while the container keeps "false".
|
|
|
|
urlaubsverwaltung.env IS ADDITIVE ONLY, AND A COLLISION IS LOST ON UPGRADE
|
|
The entries are appended AFTER the chart's own list{{ if $envNames }} (this release appends:
|
|
{{ join ", " $envNames }}){{ end }}. An entry that repeats a name the
|
|
chart already sets does not replace it — it renders the variable twice, and
|
|
what happens to that pair depends on WHEN it first appears:
|
|
|
|
AT INSTALL both entries reach the pod and the later one — yours — wins in the
|
|
container. The API server accepts it with a warning:
|
|
|
|
Warning: spec.template.spec.containers[0].env[N]: hides previous definition
|
|
of "NAME", which may be dropped when using apply
|
|
|
|
AT UPGRADE that warning comes true. Kubernetes merges env by name, so the pair
|
|
collapses back to ONE entry and the surviving value is the CHART's, not yours.
|
|
Nothing fails: helm prints "Release has been upgraded", the revision counts up,
|
|
and the override is simply gone. Measured on this chart (helm 3.19.0,
|
|
Kubernetes 1.35): "helm get manifest" shows the variable twice while the live
|
|
Deployment shows it once, with the chart's value.
|
|
|
|
So the trap is silence, not an error. Treat the variables above as fixed for
|
|
the life of a release: to really change one, reinstall. And never read a green
|
|
upgrade as proof that an override took effect — read it off the object:
|
|
|
|
kubectl -n {{ .Release.Namespace }} get deployment {{ .Release.Name }}-{{ .Chart.Name }}-deployment \
|
|
-o jsonpath='{.spec.template.spec.containers[0].env}'
|
|
|
|
Anything sensitive belongs in valueFrom.secretKeyRef rather than a plain
|
|
value: a value is stored in the pod spec and readable by anyone who can read
|
|
pods in {{ .Release.Namespace }}.
|
|
{{- if not $envNames }}
|
|
|
|
urlaubsverwaltung.env is empty in this release, so the placeholders above are
|
|
live exactly as printed.
|
|
{{- end }}
|