Files
jklein 96c23b7912
Install-test / Install-test apache-tika (pull_request) Successful in 1m14s
Install-test / Install-test rss-bridge (pull_request) Successful in 54s
Install-test / Install-test patchmon (pull_request) Successful in 1m35s
Install-test / Install-test bentopdf (pull_request) Successful in 1m2s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 13s
Validate Helm Charts / Selector match (pull_request) Successful in 10s
Validate Helm Charts / Bump level (pull_request) Successful in 4s
Validate Helm Charts / YAML scalars (pull_request) Successful in 10s
Validate Helm Charts / NetworkPolicy (pull_request) Failing after 15s
Install-test / Install-test bulwark (pull_request) Successful in 1m25s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 9s
Validate Helm Charts / Build apache-tika (pull_request) Successful in 6s
Validate Helm Charts / Build bentopdf (pull_request) Successful in 5s
Validate Helm Charts / Trivy config apache-tika (pull_request) Successful in 14s
Validate Helm Charts / Build bulwark (pull_request) Successful in 5s
Validate Helm Charts / Build calibre-web-automated (pull_request) Successful in 6s
Validate Helm Charts / Build gotenberg (pull_request) Successful in 7s
Validate Helm Charts / Build rallly (pull_request) Successful in 13s
Validate Helm Charts / Build searxng (pull_request) Successful in 17s
Validate Helm Charts / Build sure (pull_request) Successful in 18s
Validate Helm Charts / Build template-chart (pull_request) Successful in 6s
Validate Helm Charts / Build cyberchef (pull_request) Successful in 5s
Validate Helm Charts / Build homeassistant (pull_request) Successful in 7s
Validate Helm Charts / Build keycloak (pull_request) Successful in 11s
Validate Helm Charts / Build outline (pull_request) Successful in 8s
Validate Helm Charts / Build patchmon (pull_request) Successful in 6s
Validate Helm Charts / Build rss-bridge (pull_request) Successful in 5s
Install-test / Install-test rallly (pull_request) Successful in 1m39s
Validate Helm Charts / Build urlaubsverwaltung (pull_request) Successful in 5s
Validate Helm Charts / Build zipline (pull_request) Successful in 5s
Validate Helm Charts / Build voucher-vault (pull_request) Successful in 6s
Validate Helm Charts / Build ytdlp-web-player (pull_request) Successful in 6s
Validate Helm Charts / Trivy config homeassistant (pull_request) Successful in 11s
Validate Helm Charts / Trivy config bentopdf (pull_request) Successful in 15s
Validate Helm Charts / Trivy config bulwark (pull_request) Successful in 13s
Validate Helm Charts / Trivy config calibre-web-automated (pull_request) Successful in 16s
Validate Helm Charts / Trivy config cyberchef (pull_request) Successful in 26s
Validate Helm Charts / Trivy config gotenberg (pull_request) Successful in 19s
Validate Helm Charts / Trivy config patchmon (pull_request) Successful in 13s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 19s
Validate Helm Charts / Trivy config outline (pull_request) Successful in 12s
Validate Helm Charts / Trivy config rallly (pull_request) Successful in 16s
Validate Helm Charts / Trivy config sure (pull_request) Successful in 13s
Validate Helm Charts / Trivy config template-chart (pull_request) Successful in 14s
Validate Helm Charts / Trivy config urlaubsverwaltung (pull_request) Successful in 21s
Validate Helm Charts / Trivy config rss-bridge (pull_request) Successful in 17s
Validate Helm Charts / Trivy config searxng (pull_request) Successful in 11s
Validate Helm Charts / Trivy config voucher-vault (pull_request) Successful in 15s
Validate Helm Charts / Trivy config ytdlp-web-player (pull_request) Successful in 13s
Validate Helm Charts / Trivy config zipline (pull_request) Successful in 13s
Install-test / Install-test calibre-web-automated (pull_request) Successful in 2m8s
Install-test / Install-test cyberchef (pull_request) Successful in 1m9s
Install-test / Install-test gotenberg (pull_request) Successful in 1m40s
Install-test / Install-test homeassistant (pull_request) Successful in 2m30s
Install-test / Install-test keycloak (pull_request) Successful in 1m58s
Install-test / Install-test searxng (pull_request) Successful in 1m5s
Install-test / Install-test outline (pull_request) Successful in 1m57s
Install-test / Install-test sure (pull_request) Successful in 2m30s
Install-test / Install-test urlaubsverwaltung (pull_request) Successful in 1m38s
Install-test / Install-test voucher-vault (pull_request) Successful in 2m6s
Install-test / Install-test ytdlp-web-player (pull_request) Successful in 1m5s
Install-test / Install-test zipline (pull_request) Successful in 1m50s
Validate Helm Charts / Install-test (pull_request) Successful in 29m30s
feat(keycloak)!: NetworkPolicy, on by default - the full convention in one chart
Major. This is the richest policy of the sweep and the one with the largest
blast radius, and it is the only chart whose three ports fall into three
different classes - so it is where the convention is visible in full:

  http (8080)       NOT addressed by any probe, so it CAN be restricted by
                    source, and it is: own namespace, allowedNamespaces, and
                    the ingress controller. The only real inbound restriction
                    in this whole rollout, possible only because the probes use
                    a different port.
  management (9000) the port all three probes address, so it stays open inside
                    the cluster. That matters here specifically: this surface
                    carries health and metrics UNAUTHENTICATED, and "no
                    Ingress" was its only protection before this object. The
                    policy turns "anything that can route to the pod" into
                    "anything inside the cluster"; probeSources closes the rest.
  jgroups (7800)    the cluster bus, restricted to the pods this policy itself
                    selects - a podSelector with no namespaceSelector, the
                    narrowest rule the object can express.

All four port numbers come from keycloak.settings rather than being written
down twice, so moving keycloak.ports.http moves the policy with it. And the
egress rule for the database is the CONFIGURED port, keycloak.db.port - this is
the one chart in the sweep that does not have to guess, because its database
port is a value instead of a field inside an opaque secret.

The outbound cluster-bus rule is not a duplicate of the inbound one: policyTypes
lists Egress, so a replica dialling its peer is subject to this policy too, and
without it the bus would be open inbound and blocked outbound - a cluster that
never forms, with two nodes that each look healthy.

THE BLAST RADIUS IS WRITTEN INTO THE CHART, because nothing in CI can catch it:
this release is the cluster's identity provider, no ingress controller runs in
the install-test, and the probes use 9000 - so the pod becomes Ready whether or
not the http rule is right. NOTES.txt therefore asks for two checks after the
first upgrade: log in, and look at whether any consumer uses the in-cluster
Service address instead of the public issuer URL (those namespaces need
allowedNamespaces). Not allowed at all, and named as the likely first surprise:
SMTP, because the mail server is configured per realm where the chart cannot
see it.

The test's overlay guard is tightened in the same commit, and keycloak is why.
It counted any source-restricted rule, and this chart restricts two of them in
its own defaults - so the guard would have been satisfied while the probe
branch, the only branch an overlay exists to reach, was never rendered. It now
counts restricted PROBE rules. All twelve failure paths were re-run against the
change.

Part of #5
2026-10-14 14:30:30 +00:00

443 lines
26 KiB
Plaintext

{{- /*
NOTHING IN CI READS THIS FILE. `helm template` does not emit notes and no
workflow or script under ci/ reads it, so every claim below carries
review-only assurance - it is checked by being read, not by a stage. A
rendering ERROR here does fail `helm template`, which is all the pipeline can
see of it.
*/ -}}
{{- $settings := fromYaml (include "keycloak.settings" .) -}}
{{- $replicas := int (include "keycloak.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.keycloak.replicas)) -}}
{{- $adminHostname := eq (include "keycloak.adminHostnameEnabled" .) "true" -}}
{{- $httpRoute := .Values.httpRoute | default dict -}}
{{- $httpRouteAdmin := $httpRoute.admin | default dict -}}
{{- $hostnames := $httpRoute.hostnames | default list -}}
{{- if not $hostnames -}}
{{- $hostnames = list .Values.ingress.domain -}}
{{- end -}}
{{- $adminHostnames := $httpRouteAdmin.hostnames | default list -}}
{{- if not $adminHostnames -}}
{{- $adminHostnames = list (.Values.ingress.admin | default dict).domain -}}
{{- end -}}
{{- $realmHosts := (fromYaml (include "keycloak.settings" .)).realmHosts | default list -}}
{{- $deploy := printf "%s-%s-deployment" .Release.Name .Chart.Name -}}
{{- $svc := printf "%s-%s-service" .Release.Name .Chart.Name -}}
Keycloak {{ .Chart.AppVersion }} installed as release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
{{ $replicas }} replica(s), cache=ispn with jdbc-ping discovery over the database,
external PostgreSQL at {{ $settings.db.host | default "<keycloak.db.host NOT SET>" }}:{{ $settings.db.port | int64 }}/{{ $settings.db.database }}.
ENDPOINTS - {{ if .Values.ingress.enabled }}INGRESS MODE (ingress.enabled: true, httpRoute.enabled: false){{ else if $httpRoute.enabled }}GATEWAY MODE (httpRoute.enabled: true, ingress.enabled: false){{ else }}NEITHER INGRESS NOR HTTPROUTE{{ end }}
{{ if .Values.ingress.enabled }} Public: {{ include "keycloak.frontendUrl" . }}
{{ else if $httpRoute.enabled }}{{ range $hostnames }} Public: https://{{ . }}
{{ end }}{{ else }} Public: no Ingress and no HTTPRoute (ingress.enabled and httpRoute.enabled are
both false) - in-cluster only
{{ end }}{{ if not $adminHostname }} Admin: no admin hostname (ingress.admin.enabled: false) - the console is
served on the public host and Keycloak generates its URLs for it
{{ else if $httpRoute.enabled }}{{ range $adminHostnames }} Admin: https://{{ . }}/admin
{{ end }}{{ else if .Values.ingress.enabled }} Admin: {{ include "keycloak.adminUrl" . }}/admin
{{ else }} Admin: {{ include "keycloak.adminUrl" . }}/admin is what KC_HOSTNAME_ADMIN says, and
NOTHING PUBLISHES IT. ingress.admin.enabled is true, so the console
still generates its URLs for that host, but with both modes off
there is no Ingress and no HTTPRoute to carry it. Use the
port-forward below, or switch a mode on.
{{ end }} In-cluster: http://{{ $svc }}.{{ .Release.Namespace }}.svc.cluster.local:{{ $settings.ports.http | int64 }}
Management port {{ $settings.ports.management | int64 }} is NOT published and must never be: it serves
/health, /health/live, /health/ready, /health/started{{ if $settings.metrics.enabled }} and /metrics{{ end }}
WITHOUT authentication. It has a ClusterIP Service
({{ .Release.Name }}-{{ .Chart.Name }}-management-service) for in-cluster scraping only.
{{ if $realmHosts }}
REALM HOSTNAMES - {{ len $realmHosts }} further name(s), each landing in ONE realm
The object names carry the LIST INDEX and not the realm, so this is the only
place that says which is which. Reordering keycloak.realmHosts therefore
renames these objects and recreates them.
{{ range $i, $entry := $realmHosts }}
https://{{ $entry.host }}
realm {{ $entry.realm }} (bare / -> /realms/{{ $entry.realm }}/account/)
object {{ $.Release.Name }}-{{ $.Chart.Name }}-realm-{{ if $httpRoute.enabled }}httproute{{ else }}ingress{{ end }}{{ if $i }}-{{ $i }}{{ end }}
{{- if not $httpRoute.enabled }}
cert {{ $entry.tlsSecretName }}{{ if $entry.clusterIssuer }}, requested by THIS object ({{ $entry.clusterIssuer }}){{ else }}, NOT requested by this object - it must already exist or be issued elsewhere{{ end }}
{{- end }}
{{ end }}
{{- /*
No "and nothing publishes them" branch here, deliberately: a filled
keycloak.realmHosts with both modes off ABORTS the render in
keycloak.ingress.yaml, so this block can only ever be reached with a mode on.
A branch for an impossible state reads like a state somebody has seen.
*/ -}}
{{- end }}
{{ if $httpRoute.enabled }}
GATEWAY MODE - WHAT THIS CHART DOES NOT RENDER, AND WHAT YOU MUST SET YOURSELF
This release renders {{ add (ternary 2 1 $adminHostname) (len $realmHosts) }} HTTPRoute(s) and NOTHING else of the routing path -
no Gateway, no class, no Certificate, no policy object.
{{ .Release.Name }}-{{ .Chart.Name }}-httproute {{ join ", " $hostnames }}
{{- if $adminHostname }}
{{ .Release.Name }}-{{ .Chart.Name }}-admin-httproute {{ join ", " $adminHostnames }}
{{- end }}
{{- range $i, $entry := $realmHosts }}
{{ $.Release.Name }}-{{ $.Chart.Name }}-realm-httproute{{ if $i }}-{{ $i }}{{ end }} {{ $entry.host }} (realm {{ $entry.realm }})
{{- end }}
Three things therefore belong to the cluster, and if any is missing the
hostnames above simply do not answer:
1. The Gateway-API CRDs. Without them this install would have failed, so
they are present - nothing to check.
2. The Gateway(s) named in parentRefs
{{- range $httpRoute.parentRefs }}
{{ .name }}{{ if .namespace }} in namespace {{ .namespace }}{{ end }}{{ if .sectionName }}, listener {{ .sectionName }}{{ end }} (public)
{{- end }}
{{- if $adminHostname }}
{{- range ($httpRouteAdmin.parentRefs | default $httpRoute.parentRefs) }}
{{ .name }}{{ if .namespace }} in namespace {{ .namespace }}{{ end }}{{ if .sectionName }}, listener {{ .sectionName }}{{ end }} (admin)
{{- end }}
{{- end }}
{{- range $i, $entry := $realmHosts }}
{{- range ($entry.parentRefs | default $httpRoute.parentRefs) }}
{{ .name }}{{ if .namespace }} in namespace {{ .namespace }}{{ end }}{{ if .sectionName }}, listener {{ .sectionName }}{{ end }} (realm {{ $entry.realm }})
{{- end }}
{{- end }}
must exist AND their listeners must allow routes from this namespace
({{ .Release.Namespace }}) in allowedRoutes. A rejected attachment is visible only
in the route's status:
kubectl -n {{ .Release.Namespace }} get httproute {{ .Release.Name }}-{{ .Chart.Name }}-httproute -o yaml
Read status.parents[].conditions - "Accepted: True" and
"ResolvedRefs: True" is what a working attachment looks like. An empty
status means no controller has looked at the route at all.
3. The TLS certificates for those hostnames, on the Gateway's listeners.
ingress.clusterIssuer is NOT used in this mode and is not required: the
chart renders no Certificate here, so there is no
tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress secret and nothing for cert-manager to do
for this release.
{{- if $realmHosts }}
The realm hostnames above are part of that: their tlsSecretName values
are not read in this mode either. Where several of them share a DNS zone,
ONE wildcard listener for that zone replaces one certificate per name -
which is the one thing this mode makes cheaper here rather than dearer.
{{- end }}
AND THE ONE YOU WILL NOT NOTICE UNTIL A LOGIN FAILS: THE PROXY BUFFERS.
Keycloak's responses carry large headers - JWTs, SAML assertions, long
Set-Cookie chains during the login flow. In Ingress mode that is handled by
nginx.ingress.kubernetes.io/proxy-buffer-size: 16k
nginx.ingress.kubernetes.io/proxy-buffers-number: 4
which this chart has never set: they come from ingress.annotations and
ingress.admin.annotations, i.e. from whoever installs it. IN THIS MODE THEY
ARE GONE and there is NO counterpart in the Gateway API - buffer sizing is
implementation policy, not routing, and this chart renders no policy object
on purpose (it would bind the chart to one Gateway implementation).
The failure mode, so you recognise it: the PROXY answers 502 while Keycloak
logs the request as successful. Nothing says a chart value was ignored,
because no chart value was involved. Set the equivalent next to the Gateway,
in your implementation's own object, BEFORE you point DNS at it - and raise
it, do not just match 16k, if you use long-lived tokens or SAML.
ingress.annotations and ingress.admin.annotations are inert here in general:
the Gateway API has no annotation channel for controller behaviour. An IP
allow-list for the admin host is the other thing that has to move - and read
the first trap below before relying on it in either mode.
{{ if $realmHosts }}
AND THE SAME LOSS ONCE MORE, ON THE REALM HOSTNAMES - where it is likelier to
be met rather than merely possible.
Each entry of keycloak.realmHosts carries its own annotations block in Ingress
mode, and in this mode every one of them is gone: {{ len $realmHosts }} hostname(s) here, none
of which can configure a proxy buffer through this chart any more. What the
entries CAN still express is the routing - the redirect into the realm is a
rule of the route and survives the mode change intact.
Why these hostnames are the awkward ones: they exist to put somebody in an
ACCOUNT CONSOLE. That is a logged-in session from the first request after the
redirect, so the large Authorization and Set-Cookie headers are the normal
traffic on these names rather than an edge case the way they are on a login
host that mostly serves anonymous requests. A buffer sized for the default
therefore fails here FIRST, and it fails as a 502 from the Gateway on a
hostname whose Ingress never had a chart value to point at - so there is
nothing to grep for and nothing to un-set.
Set the buffer equivalent for these hostnames too, in the Gateway
implementation's own object, and set it before the names resolve there.
{{- end }}
THE ADMIN HOSTNAME'S "INTERNAL CLASS" MOVES TO THE PARENT. What
ingress.admin.className does in the other mode, httpRoute.admin.parentRefs
does here: point the admin route at an internal Gateway. It falls back to
httpRoute.parentRefs, so leaving it empty puts both hostnames on the same
Gateway.
{{ end }}
THE 1PASSWORD ITEMS, AND WHICH FIELDS THEY MUST CARRY
The operator materializes one Secret key per item FIELD, so a missing or
misnamed field leaves the pod in CreateContainerConfigError - with no message
naming the field.
{{ .Values.secrets.databaseSecretRef | default "<secrets.databaseSecretRef NOT SET>" }}
username the PostgreSQL role
password that role's password
The role must be able to CREATE its schema: Keycloak builds and migrates
the whole schema itself on first start.
{{ if $settings.bootstrapAdmin.enabled }}
{{ .Values.secrets.bootstrapAdminSecretRef | default "<secrets.bootstrapAdminSecretRef NOT SET>" }}
username the TEMPORARY bootstrap administrator (upstream default name:
temp-admin)
password its password
{{ else }}
(no bootstrap admin item - keycloak.bootstrapAdmin.enabled is false, which is
the correct state once a permanent administrator exists)
{{ end }}
Both arrive as ENVIRONMENT VARIABLES, not as files, and that is forced:
Keycloak 26 has no _FILE form for an option's value. The price: the password
is in the container's process environment.
FIRST INSTALL - AND STEP 4 IS MANDATORY, NOT ADVICE
1. kubectl -n {{ .Release.Namespace }} rollout status deployment/{{ $deploy }}
Ready means /health/ready answered 200, which means the database is
reachable and the schema is in place. The first start is the slow one: it
creates the entire schema through Liquibase, and with {{ $replicas }} replica(s)
the second waits on the first's migration lock.
2. Check that the cluster really formed. THIS STEP IS NOT OPTIONAL AND NO
PROBE REPLACES IT: a node that never found its peer is Ready, healthy and
alone, and two such nodes are two islands that each believe they are the
cluster. The install-test cannot catch it either.
kubectl -n {{ .Release.Namespace }} logs deployment/{{ $deploy }} | grep -i "ISPN000094\|view accepted\|members"
Look for a view with {{ $replicas }} members. One member per pod means
jdbc-ping did not find the peer - check that both pods can reach each
other on port {{ $settings.ports.jgroups | int64 }} and that they really share one database.
3. Log in at {{ if and $adminHostname (not $httpRoute.enabled) }}{{ include "keycloak.adminUrl" . }}/admin{{ else if $adminHostname }}https://{{ first $adminHostnames }}/admin{{ else if .Values.ingress.enabled }}{{ include "keycloak.frontendUrl" . }}/admin{{ else if $httpRoute.enabled }}https://{{ first $hostnames }}/admin{{ else }}the in-cluster address, port-forwarded{{ end }}
with the bootstrap credentials from the vault item above.
4. CREATE A PERMANENT ADMINISTRATOR, then remove the temporary one.
The bootstrap account is TEMPORARY by design - upstream: "the account
should exist only for the duration necessary to perform operations needed
to gain permanent and more secure admin access. After that, the account
needs to be removed manually." This is different from Keycloak before 26,
where KEYCLOAK_ADMIN created a regular administrator and there was nothing
to clean up.
a) In the master realm, create a user, set a password, assign the
realm-admin / admin role.
b) Verify you can log in as that user.
c) Delete the temp-admin user.
d) helm upgrade with keycloak.bootstrapAdmin.enabled: false
5. Until step 4d, EVERY START LOGS a line about the initial administrator not
being created, because the master realm already exists and the bootstrap
path is skipped. It is harmless. It is also the kind of message somebody
eventually investigates, which is why 4d is part of the install and not a
tidy-up.
MINOR UPGRADES NEED THE CLUSTER DOWN - AND THE AUTOMERGE DOES NOT KNOW THAT
Keycloak supports a rolling update only WITHIN a minor line. A minor jump
(26.8 -> 26.9) requires every node shut down before any new node starts;
mixing two minor versions in one Infinispan cluster is not a supported
configuration, and nothing in the cluster reports that it happened.
THIS REPOSITORY'S RENOVATE AUTOMERGES EVERY UPDATE TYPE, MAJORS INCLUDED.
That is a deliberate, accepted risk and not an oversight (Gitea #83), but it
means an app-version PR that moves the minor can reach main, publish the
chart and be rolled out as a rolling update without anybody deciding to.
Watch app-version PRs for a changed minor, and if one has landed, take this
path rather than letting the rollout happen:
1. Decide it with the tool instead of the version number. With the OLD
version and the OLD configuration:
kc.sh update-compatibility metadata --file=/tmp/kc-compat.json
then with the NEW version and the NEW configuration:
kc.sh update-compatibility check --file=/tmp/kc-compat.json
Pass EVERY configuration option on both calls - upstream: "Omitting any
configuration options results in incomplete metadata, and could lead to
a wrong reported result".
Exit codes, which is what to automate on: 0 rolling update possible,
1 unexpected error, 2 invalid CLI option, 3 and 4 not possible - the
deployment must be shut down.
2. If it says shut down:
helm upgrade ... --set scaleDown=true # every pod stops
# back up the database AND the configuration before anything else
helm upgrade ... --set scaleDown=false # with the new version
A NOTE ON WHAT IS NOT SETTLED: upstream's 26.6 release notes call patch-level
rolling updates supported and active by default, while the 26.8.0 page for
update-compatibility says a rolling update is possible "when the Keycloak
version is the same" and uses a PATCH pair (26.2.0 -> 26.2.1) as its example
of an incompatible one. Those two cannot both be current. Until it is clear,
run the check for patch updates too and trust its exit code rather than the
size of the version step.
TRAPS WORTH KNOWING BEFORE YOU NEED THEM
- THE ADMIN HOSTNAME IS NOT AN ACCESS CONTROL. hostname-admin decides which
URLs the server generates, nothing more. Upstream: "Using the
hostname-admin option does not prevent accessing the Administration REST
API endpoints via the frontend URL specified by the hostname option. If you
want to restrict access to the Administration REST API, you need to do it
on the reverse proxy level." So /admin/realms/... is reachable through the
public host whatever you put on the admin host, and an IP allow-list on the
admin Ingress restricts nothing on its own. This chart renders no deny rule
for /admin - see values.yaml for why not.{{ if $httpRoute.enabled }} In GATEWAY MODE there is not
even a place in this chart to put one: no annotation channel, and no
implementation-specific policy object by design. The restriction belongs
next to the Gateway, which is where it can also be tested.{{ end }}
- A SHARED DATABASE MERGES TWO CLUSTERS. Node discovery runs through the
database, so - upstream - "clusters that share the same database but should
remain separate will merge unless they have distinct cluster names. The
default cluster name is ISPN." Two releases of this chart on one database
therefore form ONE cluster, silently.
{{ with $settings.cache.clusterName }}This release sets cache-embedded-cluster-name={{ . }}.{{ else }}This release uses the default name (ISPN) - keycloak.cache.clusterName is
unset. Read its comment in values.yaml before setting it: upstream also
warns about cache invalidation between differently-named clusters.{{ end }}
- READINESS ONLY CHECKS THE DATABASE WHILE metrics-enabled IS TRUE. Upstream's
table of health checks has a "Requires Metrics" column and the DATABASE
check is the row that says yes. So with keycloak.metrics.enabled false,
/health/ready answers 200 with the database gone and the pod keeps taking
logins it cannot serve. This release has it {{ if $settings.metrics.enabled }}ON, which is the chart default
and the reason the default is what it is.{{ else }}OFF, so the readiness probe
cannot currently detect a database outage.{{ end }}
- THERE IS NO ConfigMap IN THIS CHART, so no checksum/config annotation. That
is not an omission: Keycloak is configured entirely through options and
KC_* variables, so there is no config file whose change could fail to
restart anything.
- A CHANGE INSIDE A VAULT ITEM RESTARTS NOTHING. Helm never sees it. After
rotating a field:
kubectl -n {{ .Release.Namespace }} rollout restart deployment/{{ $deploy }}
(Repointing a secrets.*SecretRef at a different item DOES roll the pods -
that is covered by checksum/secret-ref.)
- THE POD HAS TWO INIT CONTAINERS AND NEEDS BOTH. db, health-enabled and
metrics-enabled are Keycloak BUILD options, the published image is built
with their defaults, and a runtime change makes Keycloak rewrite
/opt/keycloak/lib/quarkus at start - which readOnlyRootFilesystem: true
forbids. So one init container seeds that directory into a volume and the
next runs `kc.sh build` into it; the serving container then runs
`start --optimized`. If a pod is stuck in Init, read those logs first:
kubectl -n {{ .Release.Namespace }} logs <pod> -c {{ .Release.Name }}-{{ .Chart.Name }}-build
Build options are set ONLY through keycloak.db.vendor,
keycloak.metrics.enabled and keycloak.features. Putting one into
keycloak.env fails the render on purpose.
- POOL SIZES ARE PER REPLICA. keycloak.db.pool.maxSize is {{ $settings.db.pool.maxSize | int64 }}, so with the
current {{ $replicas }} replica(s) this release can open up to {{ mul ($settings.db.pool.maxSize | int64) $replicas }} connections.
Upstream's default is 100 per node, which with two nodes exceeds a default
PostgreSQL's max_connections of 100. Raise it together with the server,
never alone.
{{ if and $settings.podDisruptionBudget.enabled (ge $replicas 2) }}
- A PodDisruptionBudget IS in place ({{ .Release.Name }}-{{ .Chart.Name }}-pdb, minAvailable
{{ $settings.podDisruptionBudget.minAvailable | int64 }}), so a node drain cannot take all replicas at once.
{{ else if $settings.podDisruptionBudget.enabled }}
- NO PodDisruptionBudget was rendered, because the effective replica count is
{{ $replicas }}. That is deliberate: minAvailable 1 with a single replica forbids every
voluntary eviction, so `kubectl drain` would block forever on a pod nothing
may move - and the error would name the PDB rather than the replica count.
Run two replicas to get the budget.
{{ else }}
- NO PodDisruptionBudget (keycloak.podDisruptionBudget.enabled is false). A
node drain may take every replica at once.
{{ end }}
WHAT THIS CHART DOES NOT RENDER
- No PVC and no StatefulSet: Keycloak 26 keeps everything in the database,
user sessions included (persistent sessions are the default; volatile ones
are deprecated as of 26.8). There is no disk state to keep.
- No NetworkPolicy. jdbc-ping opens direct pod-to-pod connections on port
{{ $settings.ports.jgroups | int64 }}, so any policy added later has to allow that between the
replicas, or the cluster silently becomes two islands.
- No deny rule for /admin on the public Ingress{{ if $httpRoute.enabled }} or route{{ end }}, and no restriction of
the administration REST API. That is reverse-proxy work - see the first
trap.
- No Gateway, no GatewayClass and no listener, in either mode. Those are
platform objects and live in cluster-config; this chart renders an
HTTPRoute at most, which is the part that looks the same on every Gateway
implementation.
{{- if .Values.networkPolicy.enabled }}
NETWORKPOLICY IS ACTIVE — WHAT IT CAPS, AND WHAT IT DOES NOT
{{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy selects this workload's pods.
Inbound, and all three ports differ:
- http ({{ $settings.ports.http | int64 }}): RESTRICTED BY SOURCE — this namespace, anything in
networkPolicy.allowedNamespaces, and the ingress controller. No probe uses
this port, which is what makes the restriction possible.
- management ({{ $settings.ports.management | int64 }}): open inside the cluster, because all three probes
address it. Its health and metrics surface is UNAUTHENTICATED, and before
this object the only thing protecting it was that no Ingress pointed at
it. Narrow it with networkPolicy.probeSources.
- jgroups ({{ $settings.ports.jgroups | int64 }}): this release's own pods only.
Outbound: DNS, TCP {{ $settings.db.port | int64 }} (the CONFIGURED database port — this chart derives it
rather than guessing), {{ $settings.ports.jgroups | int64 }} (the cluster bus, outbound as well as in) and 443{{ with .Values.networkPolicy.egress }}{{ with .extraPorts }},
plus TCP {{ join ", " . }}{{ end }}{{ end }}.
TWO THINGS TO CHECK AFTER THE FIRST UPGRADE, because nothing else will.
1. LOG IN. This release is the cluster's identity provider, and the http rule
is the only source-restricted inbound rule in the repository. If the
ingress-controller source is wrong for this cluster, every login
everywhere stops — and no part of CI can see it, because no ingress
controller runs there and the probes use the management port, so the pod
becomes Ready either way. The default was measured against the live
cluster (DaemonSet rke2-traefik in kube-system, which serves the Gateway
API here too), but measured labels are not proven traffic.
2. CHECK YOUR CONSUMERS' ISSUER URLS. An application that reaches Keycloak on
its in-cluster Service address rather than its public URL is a pod in
another namespace, and the http rule blocks it. Those namespaces go into
networkPolicy.allowedNamespaces. Consumers using the public issuer URL
arrive through Traefik and are already covered.
AND MAIL IS NOT ALLOWED. Keycloak's SMTP server is configured per realm,
which this chart never sees, so verification and password-reset mail needs its
port (587 or 465) in networkPolicy.egress.extraPorts. The symptom otherwise is
mail that is never delivered, with a healthy pod behind it.
Three limits belong in the same breath, because none of them is visible in
the object itself:
- EGRESS IS BY PORT, NEVER BY DESTINATION. A NetworkPolicy matches CIDRs and
selectors and cannot express a hostname, so every egress rule permits its
port to ANY address. That is a limit of the object, not a shortcut — this
cluster's CNI (Canal: Flannel plus Calico) has no FQDN rule either. An
outbound port the chart does not know about is BLOCKED: add it to
networkPolicy.egress.extraPorts, and expect to find out through an
application-level connection error rather than through this object.
{{- if .Values.networkPolicy.probeSources }}
- THE PROBE PATH IS RESTRICTED in this release, because
networkPolicy.probeSources is set to
{{ join ", " .Values.networkPolicy.probeSources }}
Probes come from the IP of the WORKER NODE running the pod. If none of those
CIDRs covers the node this pod lands on, the kubelet's probes are dropped
and THE POD NEVER BECOMES READY — the rollout then times out and the symptom
looks like a broken application. Compare
kubectl -n {{ .Release.Namespace }} describe pod -l app={{ .Release.Name }}-{{ .Chart.Name }}-deployment
against `kubectl get node -o wide`.
{{- else }}
- THE PROBED PORT IS OPEN INSIDE THE CLUSTER. networkPolicy.probeSources is
empty, so that rule carries no source selector at all and any pod the CNI
routes to can reach it. Not from the internet — but this is NOT pod-level
isolation. Narrowing it means listing the worker node IPs in
networkPolicy.probeSources, because the kubelet probes from the node's own
address and the chart cannot know it; a wrong guess there leaves the pod
permanently un-Ready.
{{- end }}
- IT IS ONLY A BOUNDARY WHERE THE CNI ENFORCES POLICIES. Where it does not,
this object is accepted and filters nothing, so everything reads as
configured while nothing is filtered. `kubectl -n {{ .Release.Namespace }} get networkpolicy`
shows the object exists; only a connection attempt from an unrelated
namespace shows that it bites.
Switch it off with networkPolicy.enabled: false.
{{- end }}