Two replicas with cache=ispn, clustered through Keycloak's default jdbc-ping stack: discovery runs over the PostgreSQL that is there anyway, so no headless Service and no JGroups wiring. Deployment rather than StatefulSet - Keycloak 26 keeps everything in the database, user sessions included. The chart's defining property is that it has TWO INIT CONTAINERS and needs both. db, health-enabled and metrics-enabled are Keycloak BUILD options (verified in the 26.8.0 option table), the published image is augmented with their defaults, and supplying different ones at runtime makes Keycloak rewrite /opt/keycloak/lib/quarkus at start - which readOnlyRootFilesystem: true forbids and which upstream describes as a start that "creates an image that is mutable". Not setting db is no way out: the probes alone force it, because health-enabled is a build option too. An empty emptyDir over that path fails as well, because QuarkusEntryPoint loads quarkus/quarkus-application.dat before it dispatches any command, `build` included. So one init container seeds the directory into a volume, the next runs `kc.sh build` into it, and the serving container runs `start --optimized` - every container keeping the full hardening baseline, with no documented deviation to carry. Build options therefore exist in exactly one place, and the asymmetry is enforced rather than trusted: the serving container passes no build option at all, and a build option's variable name in keycloak.env fails the render with a message naming it. Hostname handling is the security-relevant part. KC_HOSTNAME is derived from ingress.domain as a full URL (which upstream requires alongside hostname-admin), hostname-strict stays at its default true, and proxy-headers is set rather than optional. The admin hostname is NOT presented as an access control: upstream states that hostname-admin "does not prevent accessing the Administration REST API endpoints via the frontend URL specified by the hostname option", so the chart renders no /admin deny rule and values.yaml, NOTES.txt and the wiki all say in the same words that restricting that API is reverse-proxy work. The management port 9000 is a containerPort and a ClusterIP Service for in-cluster scraping, and is in no Ingress: it serves /health/* and optionally /metrics without authentication. Liveness uses /health/live and deliberately not /health or /health/ready - those include the database, and a liveness probe on them would restart the whole cluster for as long as the database is away. Also in this change: - First PodDisruptionBudget in this repository, which makes it a convention (README section 11): minAvailable as a value, and rendered only while the computed replica count is >= 2, because minAvailable: 1 with one replica blocks every node drain forever with a message that names the PDB rather than the replica count. - First quay.io image, so quay.io joins the trusted registries in ci/trivy/data/ksv0125.yaml - configured rather than suppressed, so AVD-KSV-0125 keeps being able to say something. - Renovate block with NO version constraint, which is the counter-case to garage and stalwart and is measured rather than assumed: docker versioning already excludes the -N rebuild tags, the floating minors and latest/nightly. Three dry-run fixtures are recorded at the block, and against the committed config 26.7.4 offers exactly 26.7.5 (patch) and 26.8.0 (minor). - No automerge exception, by decision (Gitea #83): the global automerge applies, and the minor-upgrade procedure plus `kc.sh update-compatibility` is documented in NOTES.txt instead. - Two upstream defaults deviated from, each with its reason at the value: db-pool-max-size 20 instead of 100 (two replicas at 100 ask a default PostgreSQL for twice its max_connections), and uid 1000 / gid 0 instead of this repo's 20000/1000, because the image declares keycloak:x:1000:0 and chowns its install tree to that pair. Part of #83
374 lines
21 KiB
YAML
374 lines
21 KiB
YAML
{{- $settings := fromYaml (include "keycloak.settings" .) }}
|
|
{{- $adminHostname := eq (include "keycloak.adminHostnameEnabled" .) "true" }}
|
|
{{- /*
|
|
WHY THIS WORKLOAD HAS TWO INIT CONTAINERS, read before changing either of them
|
|
(Gitea #83). Three measured facts force the shape:
|
|
|
|
1. db, health-enabled and metrics-enabled are BUILD options (verified in the
|
|
26.8.0 option table, where each carries the tool icon). The published
|
|
image is augmented with their DEFAULTS - db=dev-file, health-enabled=false.
|
|
2. Supplying different build options at runtime makes Keycloak re-augment
|
|
itself at start, rewriting four files in /opt/keycloak/lib/quarkus. kc.sh
|
|
says so itself: "only exit code 10 means that implicit reaugmentation
|
|
occurred and a relaunch is needed". Upstream's container guide calls that
|
|
start what it is - it "creates an image that is mutable", which is exactly
|
|
what readOnlyRootFilesystem: true forbids. And not setting db is no way
|
|
out: THE PROBES ALONE force it, because health-enabled is a build option.
|
|
3. So the build has to happen before the serving container starts, into a
|
|
volume. But an EMPTY emptyDir over /opt/keycloak/lib/quarkus does not
|
|
work either, and this is the part that is not guessable:
|
|
QuarkusEntryPoint - the main class kc.sh runs for EVERY command, `build`
|
|
included - loads "quarkus/quarkus-application.dat" (the literal string is
|
|
in QuarkusEntryPoint.class) before it dispatches anything. With that file
|
|
hidden by a fresh mount, `kc.sh build` cannot start at all.
|
|
|
|
Hence: SEED copies the image's own augmentation output into the volume (reading
|
|
the real directory, writing only into the volume - so a read-only root
|
|
filesystem is fine), BUILD then mounts the now-seeded volume at the real path
|
|
and re-augments INTO it (again writing only into the volume), and the serving
|
|
container mounts the finished volume and runs `start --optimized`.
|
|
|
|
What that buys, and the reason it is worth two containers rather than one with
|
|
readOnlyRootFilesystem: false: every container in this pod keeps the full
|
|
hardening baseline. There is no documented deviation to carry, and no Trivy
|
|
suppression that would also silence the serving container.
|
|
*/ -}}
|
|
apiVersion: apps/v1
|
|
kind: Deployment
|
|
metadata:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
labels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
spec:
|
|
replicas: {{ include "keycloak.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.keycloak.replicas) }}
|
|
selector:
|
|
matchLabels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
template:
|
|
metadata:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
labels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
annotations:
|
|
# NO checksum/config HERE, AND THAT IS NOT AN OVERSIGHT. This chart
|
|
# renders no ConfigMap at all - Keycloak is configured entirely through
|
|
# options and KC_* variables - so there is no file whose change could
|
|
# fail to restart the workload. The rule (#82) has nothing to attach
|
|
# to. values.yaml states this at the top, because an absent annotation
|
|
# looks identical to a forgotten one.
|
|
#
|
|
# Hash of the rendered OnePasswordItems. This covers the SOURCE of the
|
|
# secrets, not their content: repointing secrets.databaseSecretRef or
|
|
# secrets.bootstrapAdminSecretRef at a different vault item rolls the
|
|
# pods instead of leaving the old credentials in the running processes.
|
|
# A change INSIDE an item cannot be covered - Helm never sees it (see
|
|
# keycloak.secret.yaml) - and NOTES.txt names the manual restart.
|
|
checksum/secret-ref: {{ include (print $.Template.BasePath "/keycloak.secret.yaml") . | sha256sum }}
|
|
spec:
|
|
automountServiceAccountToken: false
|
|
securityContext:
|
|
{{- include "keycloak.podSecurityContext" .Values.keycloak.securityContext | nindent 8 }}
|
|
initContainers:
|
|
# --- 1. SEED -------------------------------------------------------
|
|
# Copy the image's own augmentation output into the shared volume, so
|
|
# that the BUILD container below can bootstrap at all (fact 3 in the
|
|
# comment at the top of this file).
|
|
#
|
|
# It reads /opt/keycloak/lib/quarkus from the image and writes only
|
|
# into the volume, which is why readOnlyRootFilesystem: true holds
|
|
# here without any special handling.
|
|
#
|
|
# `command` is overridden because the image's entrypoint is kc.sh and
|
|
# this step runs no Keycloak command - the only other chart in this
|
|
# repo that overrides an entrypoint is convertx, and the same
|
|
# obligation applies: this line has to be re-read whenever the app
|
|
# version moves, because it depends on the image shipping a shell and
|
|
# coreutils. Both are present (usr/bin/sh -> bash, usr/bin/cp, verified
|
|
# in the image layer at 26.8.0).
|
|
#
|
|
# `cp -R` and not `cp -a`: -a preserves ownership, which a non-root
|
|
# process cannot do - it would fail with a diagnostic and a non-zero
|
|
# exit. Only the content and the directory structure matter here.
|
|
- name: {{ .Release.Name }}-{{ .Chart.Name }}-seed
|
|
image: "quay.io/keycloak/keycloak:{{ .Chart.AppVersion }}"
|
|
imagePullPolicy: IfNotPresent
|
|
command:
|
|
- /bin/sh
|
|
- -c
|
|
args:
|
|
- |
|
|
set -eu
|
|
cp -R /opt/keycloak/lib/quarkus/. /quarkus-build/
|
|
ls -l /quarkus-build/
|
|
resources:
|
|
{{- include "keycloak.buildResources" .Values.keycloak.build | nindent 12 }}
|
|
securityContext:
|
|
{{- include "keycloak.containerSecurityContext" .Values.keycloak.securityContext | nindent 12 }}
|
|
volumeMounts:
|
|
- name: quarkus-build
|
|
mountPath: /quarkus-build
|
|
# --- 2. BUILD ------------------------------------------------------
|
|
# The Quarkus augmentation, i.e. `kc.sh build`. It writes into the
|
|
# seeded volume, which is mounted at the real path here - so the root
|
|
# filesystem stays read-only and the output survives into the serving
|
|
# container.
|
|
#
|
|
# THE BUILD OPTIONS ARRIVE AS ENVIRONMENT VARIABLES, NOT AS CLI
|
|
# ARGUMENTS, and that is a safety decision rather than a style one:
|
|
# `command` is a fixed shell string here, and interpolating chart
|
|
# values into a shell string is how a feature name with a space in it
|
|
# becomes two arguments. As variables they are passed by the kubelet
|
|
# and never parsed by a shell. `kc.sh build` reads KC_* exactly as it
|
|
# reads flags.
|
|
#
|
|
# This is the ONE place build options exist in this chart. The serving
|
|
# container passes `start --optimized` and no build option at all, so
|
|
# there is no second list to drift out of step - and keycloak.env
|
|
# refuses a build option's variable name outright (see _helpers.tpl).
|
|
- name: {{ .Release.Name }}-{{ .Chart.Name }}-build
|
|
image: "quay.io/keycloak/keycloak:{{ .Chart.AppVersion }}"
|
|
imagePullPolicy: IfNotPresent
|
|
args:
|
|
- build
|
|
env:
|
|
{{- include "keycloak.buildEnv" . | nindent 12 }}
|
|
resources:
|
|
{{- include "keycloak.buildResources" .Values.keycloak.build | nindent 12 }}
|
|
securityContext:
|
|
{{- include "keycloak.containerSecurityContext" .Values.keycloak.securityContext | nindent 12 }}
|
|
volumeMounts:
|
|
- name: quarkus-build
|
|
mountPath: {{ include "keycloak.quarkusBuildDir" . }}
|
|
# Quarkus puts its temporary directory under the Keycloak home
|
|
# rather than honouring java.io.tmpdir (upstream issue #37533), so
|
|
# the augmentation needs both of these writable.
|
|
- name: data
|
|
mountPath: {{ include "keycloak.dataDir" . }}
|
|
- name: tmp
|
|
mountPath: /tmp
|
|
containers:
|
|
- name: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
# Fully qualified: an unqualified reference would resolve to Docker
|
|
# Hub by convention rather than by rule. quay.io is Keycloak's own
|
|
# registry and the first quay.io image in this repository - which is
|
|
# why ci/trivy/data/ksv0125.yaml had to learn about it.
|
|
image: "quay.io/keycloak/keycloak:{{ .Chart.AppVersion }}"
|
|
imagePullPolicy: IfNotPresent
|
|
# `start`, not `start-dev` - the development mode has insecure
|
|
# defaults and a local cache. The image declares no CMD, only
|
|
# ENTRYPOINT ["/opt/keycloak/bin/kc.sh"], so the subcommand has to
|
|
# come from here.
|
|
#
|
|
# --optimized says "this installation is already built, do not check
|
|
# and do not augment". It is what makes the read-only root filesystem
|
|
# possible, and it is also why NO build option may be passed to this
|
|
# container: with --optimized a build option is an error, and without
|
|
# it the container would rebuild on a path it cannot write.
|
|
args:
|
|
- start
|
|
- --optimized
|
|
env:
|
|
# THE HOSTNAME, DERIVED AND NOT LEFT OPEN - this is the
|
|
# security-relevant pair of this chart. Upstream on proxy-headers:
|
|
# "Misconfiguration might leave the server exposed to security
|
|
# vulnerabilities." The failure mode is Host header injection: with
|
|
# a hostname resolved from request headers, an attacker controls the
|
|
# URLs the server generates, which is how tokens and password
|
|
# resets end up pointed at somebody else's host.
|
|
#
|
|
# Two things keep that shut. hostname-strict stays at its default
|
|
# true, so nothing is taken from the request. And hostname is a
|
|
# FULL URL rather than a bare host, which upstream requires as soon
|
|
# as hostname-admin is in play: "If hostname-admin is configured,
|
|
# hostname must be set to a URL (not just hostname). Otherwise
|
|
# Keycloak would not know what is the correct frontend URL".
|
|
- name: KC_HOSTNAME
|
|
value: {{ include "keycloak.frontendUrl" . | quote }}
|
|
{{- if $adminHostname }}
|
|
# THE ADMIN HOSTNAME - and it is NOT an access boundary. It decides
|
|
# which URLs the server generates for the administration console
|
|
# (redirects, assets, the API address the console itself calls).
|
|
# Upstream, verbatim: "Using the hostname-admin option does not
|
|
# prevent accessing the Administration REST API endpoints via the
|
|
# frontend URL specified by the hostname option. If you want to
|
|
# restrict access to the Administration REST API, you need to do it
|
|
# on the reverse proxy level."
|
|
#
|
|
# So /admin/realms/... stays reachable through ingress.domain. This
|
|
# chart renders no deny rule for it: a path rule on /admin that has
|
|
# not been tested against a running console breaks redirects and
|
|
# asset loading, and a measure that only looks like one is worse
|
|
# than none. The full argument is at ingress.admin in values.yaml.
|
|
- name: KC_HOSTNAME_ADMIN
|
|
value: {{ include "keycloak.adminUrl" . | quote }}
|
|
{{- end }}
|
|
# Behind the Ingress the TLS ends at the controller, so the pod
|
|
# itself serves plain HTTP - which Keycloak only does when told to.
|
|
- name: KC_HTTP_ENABLED
|
|
value: "true"
|
|
# xforwarded, because the ingress-nginx controller sets
|
|
# X-Forwarded-* and not RFC 7239 Forwarded. This is not optional
|
|
# decoration: without it Keycloak sees the controller's address and
|
|
# scheme instead of the client's.
|
|
- name: KC_PROXY_HEADERS
|
|
value: "xforwarded"
|
|
- name: KC_HTTP_PORT
|
|
value: {{ $settings.ports.http | int64 | quote }}
|
|
- name: KC_HTTP_MANAGEMENT_PORT
|
|
value: {{ $settings.ports.management | int64 | quote }}
|
|
# CLUSTERING. ispn is already the default for `start` in production
|
|
# mode; it is set explicitly because it is the decision that makes
|
|
# the second replica a cluster member rather than a second island,
|
|
# and an explicit value cannot be lost to a change of upstream
|
|
# defaults.
|
|
#
|
|
# cache-stack is deliberately NOT set: the default is jdbc-ping,
|
|
# which discovers peers through the database that is there anyway.
|
|
# The alternative, cache-stack=kubernetes with DNS_PING, is marked
|
|
# "(deprecated)" upstream and would need a headless Service and a
|
|
# jgroups.dns.query - none of which this chart renders.
|
|
- name: KC_CACHE
|
|
value: "ispn"
|
|
{{- with $settings.cache.clusterName }}
|
|
# Only when set. The trap it addresses: "clusters that share the
|
|
# same database but should remain separate will merge unless they
|
|
# have distinct cluster names". See values.yaml - the default is
|
|
# upstream's ISPN on purpose.
|
|
- name: KC_CACHE_EMBEDDED_CLUSTER_NAME
|
|
value: {{ . | quote }}
|
|
{{- end }}
|
|
# A stable node name, from the pod name. Upstream's default is "a
|
|
# random name ... generated on each start, which makes it difficult
|
|
# to correlate metrics and logs across restarts" - and with two
|
|
# replicas that correlation is the first thing anybody needs when
|
|
# the cluster misbehaves.
|
|
- name: KC_CACHE_EMBEDDED_NODE_NAME
|
|
valueFrom:
|
|
fieldRef:
|
|
fieldPath: metadata.name
|
|
# THE DATABASE. Note what is NOT here: KC_DB. The vendor is a build
|
|
# option and is baked in by the init container above; setting it
|
|
# here would make this container re-augment on a read-only path.
|
|
- name: KC_DB_URL_HOST
|
|
value: {{ required "A database host must be provided (keycloak.db.host)" $settings.db.host | quote }}
|
|
- name: KC_DB_URL_PORT
|
|
value: {{ $settings.db.port | int64 | quote }}
|
|
- name: KC_DB_URL_DATABASE
|
|
value: {{ required "A database name must be provided (keycloak.db.database)" $settings.db.database | quote }}
|
|
- name: KC_DB_USERNAME
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-db-secret
|
|
key: username
|
|
# AS A VARIABLE, NOT A FILE, and that is forced: Keycloak 26 has no
|
|
# _FILE indirection for an option's value - no KC_DB_PASSWORD_FILE
|
|
# and no generic KC_<OPTION>_FILE. Checked against the 26.8.0
|
|
# option table, where every KC_*_FILE variable is a path option
|
|
# (certificates, key stores, trust stores, log and export files).
|
|
# The price is named at secrets.databaseSecretRef in values.yaml.
|
|
- name: KC_DB_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-db-secret
|
|
key: password
|
|
# POOL SIZES, per replica. Upstream's db-pool-max-size default is
|
|
# 100, which with two replicas asks a default PostgreSQL
|
|
# (max_connections = 100) for twice what it has. See values.yaml.
|
|
- name: KC_DB_POOL_INITIAL_SIZE
|
|
value: {{ $settings.db.pool.initialSize | int64 | quote }}
|
|
- name: KC_DB_POOL_MIN_SIZE
|
|
value: {{ $settings.db.pool.minSize | int64 | quote }}
|
|
- name: KC_DB_POOL_MAX_SIZE
|
|
value: {{ $settings.db.pool.maxSize | int64 | quote }}
|
|
{{- if $settings.bootstrapAdmin.enabled }}
|
|
# THE TEMPORARY BOOTSTRAP ADMINISTRATOR. Needed for a first
|
|
# install, because Keycloak "only allows to create the initial
|
|
# admin user from a local network connection", which a container
|
|
# never has.
|
|
#
|
|
# Two things NOTES.txt repeats because they cost time otherwise:
|
|
# these variable names exist only since Keycloak 26 (before:
|
|
# KEYCLOAK_ADMIN / KEYCLOAK_ADMIN_PASSWORD), and the account they
|
|
# create is TEMPORARY - creating a permanent administrator
|
|
# afterwards is a mandatory step, not advice.
|
|
- name: KC_BOOTSTRAP_ADMIN_USERNAME
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-bootstrap-admin-secret
|
|
key: username
|
|
- name: KC_BOOTSTRAP_ADMIN_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-bootstrap-admin-secret
|
|
key: password
|
|
{{- end }}
|
|
{{- with .Values.keycloak.env }}
|
|
{{- include "keycloak.env" (dict "root" $ "env" .) | nindent 12 }}
|
|
{{- end }}
|
|
ports:
|
|
- containerPort: {{ $settings.ports.http | int64 }}
|
|
protocol: TCP
|
|
name: http
|
|
# THE MANAGEMENT PORT. All three probes address it BY NAME, so
|
|
# renaming it here breaks them. It serves /health/* and - with
|
|
# keycloak.metrics.enabled - /metrics, both UNAUTHENTICATED, which
|
|
# is why no Ingress in this chart points at it and none should be
|
|
# added.
|
|
- containerPort: {{ $settings.ports.management | int64 }}
|
|
protocol: TCP
|
|
name: management
|
|
# JGroups. Declared for the record rather than for routing: nothing
|
|
# serves it and no Service exposes it. jdbc-ping finds the peers
|
|
# through the database and then opens DIRECT pod-to-pod
|
|
# connections on this port, so the replicas need plain L3
|
|
# reachability between pod IPs - no Service, headless or otherwise.
|
|
- containerPort: {{ $settings.ports.jgroups | int64 }}
|
|
protocol: TCP
|
|
name: jgroups
|
|
# Probes, resources and both security contexts go through the
|
|
# null-safe helpers (#99): an erased block means "chart defaults
|
|
# apply" instead of a vanished probe or an unhardened container.
|
|
# Never wrap these in an "if" - a guard hides the erased case
|
|
# completely instead of rendering a visible null.
|
|
#
|
|
# Which probe points where, and why liveness must NOT use /health or
|
|
# /health/ready, is argued at each helper in _helpers.tpl.
|
|
livenessProbe:
|
|
{{- include "keycloak.livenessProbe" .Values.keycloak.livenessProbe | nindent 12 }}
|
|
readinessProbe:
|
|
{{- include "keycloak.readinessProbe" .Values.keycloak.readinessProbe | nindent 12 }}
|
|
startupProbe:
|
|
{{- include "keycloak.startupProbe" .Values.keycloak.startupProbe | nindent 12 }}
|
|
resources:
|
|
{{- include "keycloak.effectiveResources" .Values.keycloak.resources | nindent 12 }}
|
|
securityContext:
|
|
{{- include "keycloak.containerSecurityContext" .Values.keycloak.securityContext | nindent 12 }}
|
|
volumeMounts:
|
|
# The finished augmentation, produced by the two init containers.
|
|
# Writable, although --optimized means nothing should write here:
|
|
# it is an emptyDir either way, and a writable mount is what keeps
|
|
# a re-augmentation (should a future version decide it needs one) a
|
|
# slow start rather than a crash loop.
|
|
- name: quarkus-build
|
|
mountPath: {{ include "keycloak.quarkusBuildDir" . }}
|
|
# Empty in the image, so this hides nothing. Needed because Quarkus
|
|
# places its temporary directory under the Keycloak home instead of
|
|
# honouring java.io.tmpdir (upstream issue #37533).
|
|
- name: data
|
|
mountPath: {{ include "keycloak.dataDir" . }}
|
|
- name: tmp
|
|
mountPath: /tmp
|
|
# DELIBERATELY NOT MOUNTED: /opt/keycloak/conf. It is not empty in
|
|
# the image - it carries keycloak.conf and, more importantly,
|
|
# cache-ispn.xml, which is the Infinispan configuration cache=ispn
|
|
# reads. An emptyDir there would hide it and the cluster would come up
|
|
# with no cache configuration.
|
|
restartPolicy: Always
|
|
volumes:
|
|
- name: quarkus-build
|
|
emptyDir: {}
|
|
- name: data
|
|
emptyDir: {}
|
|
- name: tmp
|
|
emptyDir: {}
|