jklein
f8acdfdfe1
MANAGEMENT_ENDPOINT_HEALTH_GROUP_READINESS_INCLUDE=readinessState,db puts the datasource into Spring Boot's readiness group, which shipped with readinessState alone: a total database outage left the pod 1/1 Ready while every request failed. Naming the members replaces the default, so readinessState is repeated. The liveness group stays untouched on purpose, because an outage has to keep traffic away rather than add a crash loop to it. MANAGEMENT_HEALTH_MAIL_ENABLED=false removes the mail indicator, which pinned the aggregate /actuator/health at 503 for the whole life of a healthy release, because the chart's mail wiring points at localhost:1025 where nothing listens. values.yaml and NOTES.txt document what that costs and when it has to be switched back on.
338 lines
17 KiB
Plaintext
338 lines
17 KiB
Plaintext
{{- $replicas := include "urlaubsverwaltung.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.urlaubsverwaltung.replicas) -}}
|
|
{{- $res := fromYaml (include "urlaubsverwaltung.effectiveResources" .Values.urlaubsverwaltung.resources) -}}
|
|
{{/*
|
|
Both figures go through formatNumber for the reason issue #157 documents: they
|
|
come back from fromYaml as float64, and a bare byte count would print as
|
|
"1e+06" here just as it did in the rendered manifest.
|
|
*/}}
|
|
{{- $memLimitRaw := index ($res.limits | default dict) "memory" -}}
|
|
{{- $memLimit := "the computed memory limit" -}}
|
|
{{- if not (kindIs "invalid" $memLimitRaw) -}}
|
|
{{- $memLimit = include "urlaubsverwaltung.formatNumber" $memLimitRaw -}}
|
|
{{- end -}}
|
|
{{- $memRequestRaw := index ($res.requests | default dict) "memory" -}}
|
|
{{- $memRequest := "the memory request" -}}
|
|
{{- if not (kindIs "invalid" $memRequestRaw) -}}
|
|
{{- $memRequest = include "urlaubsverwaltung.formatNumber" $memRequestRaw -}}
|
|
{{- end -}}
|
|
{{/*
|
|
The effective limitFactor. It cannot be read back out of $res: that dict is the
|
|
RENDERED resources block (requests / limits only), so $res.limitFactor is always
|
|
empty and would silently print the default even where another factor is in
|
|
force. Mirror the defaulting of the resources helper on the raw value instead.
|
|
*/}}
|
|
{{- $resIn := .Values.urlaubsverwaltung.resources -}}
|
|
{{- if kindIs "invalid" $resIn -}}
|
|
{{- $resIn = dict -}}
|
|
{{- end -}}
|
|
{{- $factor := 3 -}}
|
|
{{- if not (kindIs "invalid" (index $resIn "limitFactor")) -}}
|
|
{{- $factor = index $resIn "limitFactor" -}}
|
|
{{- end -}}
|
|
{{/* An explicitly set memory limit wins over the computed one - say which. */}}
|
|
{{- $limitsIn := index $resIn "limits" -}}
|
|
{{- if kindIs "invalid" $limitsIn -}}
|
|
{{- $limitsIn = dict -}}
|
|
{{- end -}}
|
|
{{- $memLimitExplicit := not (kindIs "invalid" (index $limitsIn "memory")) -}}
|
|
{{- $startup := fromYaml (include "urlaubsverwaltung.startupProbe" .Values.urlaubsverwaltung.startupProbe) -}}
|
|
{{- $budget := "" -}}
|
|
{{- if and (kindIs "float64" $startup.periodSeconds) (kindIs "float64" $startup.failureThreshold) -}}
|
|
{{- $budget = printf "%.0f seconds" (mulf $startup.periodSeconds $startup.failureThreshold) -}}
|
|
{{- end -}}
|
|
{{- $envNames := list -}}
|
|
{{- range $entry := .Values.urlaubsverwaltung.env }}
|
|
{{- $envNames = append $envNames $entry.name }}
|
|
{{- end }}
|
|
Urlaubsverwaltung {{ .Chart.AppVersion }} — release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
|
|
|
|
{{- if .Values.ingress.enabled }}
|
|
|
|
ACCESS
|
|
https://{{ .Values.ingress.domain }}
|
|
|
|
The certificate is requested by cert-manager through the "{{ .Values.ingress.clusterIssuer }}"
|
|
cluster issuer. Until it has been issued the browser shows a warning; follow it with
|
|
|
|
kubectl -n {{ .Release.Namespace }} get certificate tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
|
|
|
|
The Ingress publishes the application at "/" with no path restriction, so
|
|
everything the app serves is served under this host — including
|
|
/swagger-ui.html and /v3/api-docs, which this app version enables by default
|
|
and this chart does not turn off. Both are behind the login (they answer 302
|
|
to the identity provider like every other path), so this is exposure of an
|
|
API description, not of data. Turning them off is a
|
|
springdoc.api-docs.enabled / springdoc.swagger-ui.enabled property; the chart
|
|
sets neither, so it can be added through urlaubsverwaltung.env.
|
|
{{- else }}
|
|
|
|
ACCESS — NO INGRESS
|
|
ingress.enabled is false, so this release has no Ingress and no certificate;
|
|
ingress.clusterIssuer is not needed. Unlike other charts in this repo,
|
|
ingress.domain is read by NOTHING else here — no environment variable of the
|
|
container derives from it. The address the application considers its own is
|
|
the OIDC redirect URI, and that comes out of the 1Password item below, not
|
|
out of this value. The two can drift apart without any complaint.
|
|
|
|
Reach the release inside the cluster with
|
|
|
|
kubectl -n {{ .Release.Namespace }} port-forward svc/{{ .Release.Name }}-{{ .Chart.Name }}-service 8080:80
|
|
|
|
and open http://localhost:8080. The login redirects to your identity
|
|
provider, which will refuse to come back to a localhost callback unless that
|
|
exact URL is registered there as well.
|
|
{{- end }}
|
|
{{ if eq $replicas "0" }}
|
|
SCALED TO ZERO
|
|
{{ if .Values.scaleDown }}scaleDown: true{{ else }}urlaubsverwaltung.replicas: 0{{ end }} — the Deployment runs with replicas: 0, so the endpoint
|
|
above does not answer until you set it back and upgrade again. Both
|
|
OnePasswordItem CRs and the Service{{ if .Values.ingress.enabled }} and Ingress stay in place, so the
|
|
certificate is not re-issued.{{ else }} stay in place.{{ end }} This chart owns no volume: all
|
|
state lives in the external database, so scaling to zero loses nothing.
|
|
{{ end }}
|
|
SECRETS — THE FIELDS EACH 1PASSWORD ITEM MUST CARRY
|
|
The operator copies an item's FIELDS into the Kubernetes Secret, one key per
|
|
field name, and this chart reads each key by name into an environment
|
|
variable. The field names are decided here and appear nowhere in 1Password,
|
|
so they are exactly what you would otherwise have to read out of the
|
|
deployment template. They must match to the letter.
|
|
|
|
secrets.databaseSecretRef → {{ .Values.secrets.databaseSecretRef }}
|
|
→ Secret {{ .Release.Name }}-{{ .Chart.Name }}-db-secret
|
|
jdbc-url a full JDBC URL, e.g.
|
|
jdbc:postgresql://postgres.example.com:5432/urlaubsverwaltung
|
|
username
|
|
password
|
|
|
|
secrets.oidcSecretRef → {{ .Values.secrets.oidcSecretRef }}
|
|
→ Secret {{ .Release.Name }}-{{ .Chart.Name }}-oidc-secret
|
|
client-id
|
|
client-secret
|
|
issuer-name the provider's display name on the login button
|
|
issuer-uri the OIDC issuer, WITHOUT /.well-known/openid-configuration
|
|
jwt-issuer-uri the same issuer for the resource-server side
|
|
redirect-uri the full callback URL. This chart registers the client
|
|
under the id "default", so the path is
|
|
/login/oauth2/code/default — for example
|
|
https://{{ if .Values.ingress.enabled }}{{ .Values.ingress.domain }}{{ else }}example.com{{ end }}/login/oauth2/code/default
|
|
and the same URL has to be registered at the provider.
|
|
|
|
This chart deploys NO database and NO identity provider. Both have to exist
|
|
already and be reachable from this namespace.
|
|
|
|
A field spelled differently yields a Secret without that key. The pod then
|
|
hangs in CreateContainerConfigError while the Secret itself exists, so it
|
|
looks as if the operator had done its job. What it actually delivered:
|
|
|
|
kubectl -n {{ .Release.Namespace }} get secret {{ .Release.Name }}-{{ .Chart.Name }}-db-secret -o jsonpath='{.data}'
|
|
|
|
Pointing a ref at a DIFFERENT item rolls the pod; changing a value INSIDE the
|
|
same item does not. Helm never sees that content, so the rendered Deployment
|
|
is unchanged and this chart stamps no checksum over the secrets — nothing
|
|
restarts, and the pod keeps running with the old values:
|
|
|
|
kubectl -n {{ .Release.Namespace }} rollout restart deployment/{{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
|
|
THE IDENTITY PROVIDER IS REQUIRED AT BOOT, NOT AT LOGIN
|
|
On start the application resolves the provider's configuration itself, with a
|
|
GET on
|
|
|
|
<issuer-uri>/.well-known/openid-configuration
|
|
|
|
and it does this while building its security beans — before the HTTP port
|
|
opens. An issuer-uri that does not resolve, is unreachable from this
|
|
namespace, or answers something other than a discovery document therefore
|
|
does not produce a broken login page: the process exits with
|
|
|
|
Application run failed ... Unable to resolve Configuration with the
|
|
provided Issuer of "<issuer-uri>"
|
|
|
|
and the pod goes into CrashLoopBackOff having never served a request. So a
|
|
provider that is down at rollout time keeps this release from starting at
|
|
all. The reason is only in the log:
|
|
|
|
kubectl -n {{ .Release.Namespace }} logs deployment/{{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
|
|
Note the order: the database migration runs BEFORE this step. A rollout that
|
|
dies on the identity provider has already migrated the schema.
|
|
|
|
STARTUP MIGRATES THE DATABASE, AND ROLLBACK DOES NOT UNMIGRATE
|
|
Liquibase brings the schema up to date on every start, before anything is
|
|
served; against an empty database it creates it from scratch and the app then
|
|
seeds its initial settings, vacation types and sick-note types. The startup
|
|
probe covers that whole window{{ if $budget }} ({{ $budget }}){{ end }} and holds liveness and readiness off
|
|
until it is through. A measured cold start against an empty database — JVM,
|
|
full migration, seeding — took about 30 seconds; an upgrade that carries a
|
|
long migration takes longer, and if it outruns the budget the kubelet kills
|
|
the container mid-migration and the next attempt starts over.
|
|
|
|
helm rollback puts the previous app version back; the schema does not come
|
|
back with it, because it was migrated in place. Take a database backup BEFORE
|
|
an app-version upgrade. There is no volume in this release to back up — the
|
|
database is the whole of the state.
|
|
|
|
A failed rollout does not take the release down: the previous pod keeps
|
|
serving until the new one becomes Ready. "helm upgrade" without --wait
|
|
therefore reports success while the new pod crashloops behind it. Check:
|
|
|
|
kubectl -n {{ .Release.Namespace }} rollout status deployment/{{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
|
|
THE MEMORY LIMIT SIZES THE JAVA HEAP — IT IS NOT JUST A CEILING
|
|
The image is built with Cloud Native Buildpacks, and its memory calculator
|
|
reads the container's memory LIMIT at every start and derives -Xmx and the
|
|
metaspace from it. In this release that limit is {{ $memLimit }}.
|
|
{{- if $memLimitExplicit }}
|
|
It is set explicitly under urlaubsverwaltung.resources.limits.memory, which
|
|
wins over the factor the chart would otherwise apply.
|
|
{{- else }}
|
|
The chart does not set it directly: it is computed as
|
|
urlaubsverwaltung.resources.limitFactor ({{ include "urlaubsverwaltung.formatNumber" $factor }})
|
|
x the memory request ({{ $memRequest }}).
|
|
{{- end }}
|
|
|
|
So the memory limit is not an operational safety margin here, it is a JVM
|
|
tuning knob. Lowering it shrinks the heap; raising it raises the heap.
|
|
A limit chosen too tightly does not merely risk an OOMKill under load —
|
|
the calculator refuses to start at all when what is left after metaspace,
|
|
code cache and the thread stacks is negative, and that failure appears before
|
|
any application log line. The sizing it chose is the first line of the log:
|
|
|
|
kubectl -n {{ .Release.Namespace }} logs deployment/{{ .Release.Name }}-{{ .Chart.Name }}-deployment | head -1
|
|
|
|
THE THREE HEALTH PATHS AND WHAT EACH ONE ANSWERS FOR
|
|
This chart sets two health settings in the deployment template, and they are
|
|
what makes the three paths mean different things:
|
|
|
|
MANAGEMENT_ENDPOINT_HEALTH_GROUP_READINESS_INCLUDE=readinessState,db
|
|
MANAGEMENT_HEALTH_MAIL_ENABLED=false
|
|
|
|
/actuator/health everything the release actually runs: db,
|
|
diskSpace, ping, ssl and the two lifecycle
|
|
states. 200 UP on a healthy release. The mail
|
|
indicator is NOT in it — see the mail section.
|
|
→ point external monitoring here. This is the only path that also fails on
|
|
diskSpace or ssl.
|
|
|
|
/actuator/health/readiness readinessState AND db. Goes DOWN (503) on a
|
|
database outage, which takes the pod out of Ready
|
|
and stops traffic reaching it.
|
|
→ the readinessProbe.
|
|
|
|
/actuator/health/liveness livenessState alone, deliberately WITHOUT db.
|
|
Stays 200 through a database outage.
|
|
→ the livenessProbe and the startupProbe.
|
|
|
|
The split is the point: a database outage has to keep traffic away, but
|
|
restarting the pod does not bring the database back. Adding db to the
|
|
liveness group would turn every outage into a crash loop on top of the
|
|
outage. Measured on this chart, with the database scaled to zero underneath a
|
|
running pod: readiness 503, liveness 200, pod 0/1 Ready, restart count 0.
|
|
|
|
How readiness actually fails is worth knowing before you tune it. The db
|
|
indicator does not answer 503 quickly — it waits out the JDBC connection
|
|
timeout first, measured at 30 seconds against a database whose Service has no
|
|
endpoints at all. readinessProbe.timeoutSeconds is 5, so the kubelet gives up
|
|
long before the answer arrives and logs
|
|
|
|
Readiness probe failed: ... context deadline exceeded
|
|
|
|
rather than a 503. The pod leaves Ready either way, and faster; the 503 is
|
|
what a client with patience sees. Raising timeoutSeconds past the JDBC
|
|
timeout would swap the timeout for the 503 and delay the reaction — there is
|
|
no reason to.
|
|
|
|
WHAT A READY POD DOES AND DOES NOT PROVE
|
|
Ready now means the pod can reach the database — that is checked on every
|
|
readiness period, not once at boot. It still does not mean the identity
|
|
provider is reachable: OIDC is resolved once at startup (see below) and no
|
|
health indicator watches it afterwards, so a provider that goes down after
|
|
the rollout leaves a Ready pod on which nobody can log in.
|
|
|
|
A database outage now shows up in the pod's status: READY drops to 0/1 while
|
|
STATUS stays Running and RESTARTS stays put. That combination — running, not
|
|
restarting, not Ready — is the signature of a dependency outage rather than
|
|
of a broken release. The reason is in the endpoint, not in the log:
|
|
|
|
kubectl -n {{ .Release.Namespace }} get pod -l app={{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
kubectl -n {{ .Release.Namespace }} port-forward svc/{{ .Release.Name }}-{{ .Chart.Name }}-service 8080:80
|
|
curl -s localhost:8080/actuator/health
|
|
|
|
The body names components only when show-details is on; the chart does not
|
|
set it, so the aggregate answers {"status":"DOWN","groups":[...]} and the
|
|
status code carries the message. To see WHICH component is down, add
|
|
|
|
urlaubsverwaltung.env:
|
|
- name: MANAGEMENT_ENDPOINT_HEALTH_SHOW_DETAILS
|
|
value: always
|
|
|
|
and know what that publishes to anyone who can reach the endpoint: the
|
|
database product, the container's filesystem path and free bytes, the
|
|
certificate chains, and the stack-trace class of whatever failed.
|
|
|
|
Note also that there is no shell in this image: "kubectl exec" cannot open
|
|
one, and neither can a debug sidecar sharing the process namespace run the
|
|
app's own tooling. Everything you can learn from a running pod, you learn
|
|
from its log and its HTTP endpoints.
|
|
|
|
MAIL AND THE APPLICATION URL ARE PLACEHOLDERS UNTIL YOU SET THEM
|
|
The chart hardcodes the mail wiring to values that work nowhere:
|
|
|
|
SPRING_MAIL_HOST=localhost, SPRING_MAIL_PORT=1025
|
|
UV_MAIL_FROM / UV_MAIL_REPLYTO = urlaubsverwaltung@example.org
|
|
UV_CALENDAR_ORGANIZER = organizer@example.org
|
|
UV_MAIL_APPLICATIONURL = http://localhost:8080
|
|
|
|
Nothing in this pod listens on localhost:1025, so no notification mail leaves
|
|
the release. UV_MAIL_APPLICATIONURL is what the app puts into the links
|
|
inside those mails — it is NOT derived from ingress.domain. Left as it is,
|
|
every link points at the reader's own machine.
|
|
|
|
BECAUSE that wiring is a placeholder, the chart sets
|
|
|
|
MANAGEMENT_HEALTH_MAIL_ENABLED=false
|
|
|
|
and the mail indicator is gone from /actuator/health. Otherwise the aggregate
|
|
would answer 503 DOWN on a perfectly healthy release for its whole lifetime,
|
|
and any monitor pointed at it would page forever about a mail server nobody
|
|
configured.
|
|
|
|
THE PART THAT BITES LATER: the day you configure real mail delivery, switch
|
|
the indicator back on in the SAME change —
|
|
|
|
urlaubsverwaltung.env:
|
|
- name: MANAGEMENT_HEALTH_MAIL_ENABLED
|
|
value: "true"
|
|
|
|
Left at false it does not report a false alarm, it reports nothing: the
|
|
endpoint answers 200 while the mail server is unreachable and notifications
|
|
are silently not delivered. The setting is only honest while there is no mail
|
|
server to be honest about.
|
|
|
|
Note that this variable repeats a name the chart already sets, so per the
|
|
rule below it can only be added at INSTALL time — putting it into an already
|
|
installed release makes the next upgrade fail in the API server.
|
|
|
|
urlaubsverwaltung.env IS ADDITIVE ONLY, AND A COLLISION BREAKS THE UPGRADE
|
|
The entries are appended AFTER the chart's own list{{ if $envNames }} (this release appends:
|
|
{{ join ", " $envNames }}){{ end }}. An entry that repeats a name the
|
|
chart already sets does not replace it — it renders the variable twice, and
|
|
that has a sharp edge: adding such a name to an ALREADY INSTALLED release
|
|
makes the next upgrade fail in the API server, before anything changes, with
|
|
|
|
UPGRADE FAILED: cannot patch "{{ .Release.Name }}-{{ .Chart.Name }}-deployment" ...
|
|
The order in patch list ... doesn't match $setElementOrder list
|
|
|
|
because env entries are merged by name and a duplicate name breaks that
|
|
ordering. A fresh install with the same values goes through, so the failure
|
|
looks arbitrary until you know it. Treat the variables above as fixed for the
|
|
life of a release: to really change one, reinstall.
|
|
|
|
Anything sensitive belongs in valueFrom.secretKeyRef rather than a plain
|
|
value: a value is stored in the pod spec and readable by anyone who can read
|
|
pods in {{ .Release.Namespace }}.
|
|
{{- if not $envNames }}
|
|
|
|
urlaubsverwaltung.env is empty in this release, so the placeholders above are
|
|
live exactly as printed.
|
|
{{- end }}
|