feat(keycloak): new chart for Keycloak (1.0.0+up26.8.0)
Two replicas with cache=ispn, clustered through Keycloak's default jdbc-ping stack: discovery runs over the PostgreSQL that is there anyway, so no headless Service and no JGroups wiring. Deployment rather than StatefulSet - Keycloak 26 keeps everything in the database, user sessions included. The chart's defining property is that it has TWO INIT CONTAINERS and needs both. db, health-enabled and metrics-enabled are Keycloak BUILD options (verified in the 26.8.0 option table), the published image is augmented with their defaults, and supplying different ones at runtime makes Keycloak rewrite /opt/keycloak/lib/quarkus at start - which readOnlyRootFilesystem: true forbids and which upstream describes as a start that "creates an image that is mutable". Not setting db is no way out: the probes alone force it, because health-enabled is a build option too. An empty emptyDir over that path fails as well, because QuarkusEntryPoint loads quarkus/quarkus-application.dat before it dispatches any command, `build` included. So one init container seeds the directory into a volume, the next runs `kc.sh build` into it, and the serving container runs `start --optimized` - every container keeping the full hardening baseline, with no documented deviation to carry. Build options therefore exist in exactly one place, and the asymmetry is enforced rather than trusted: the serving container passes no build option at all, and a build option's variable name in keycloak.env fails the render with a message naming it. Hostname handling is the security-relevant part. KC_HOSTNAME is derived from ingress.domain as a full URL (which upstream requires alongside hostname-admin), hostname-strict stays at its default true, and proxy-headers is set rather than optional. The admin hostname is NOT presented as an access control: upstream states that hostname-admin "does not prevent accessing the Administration REST API endpoints via the frontend URL specified by the hostname option", so the chart renders no /admin deny rule and values.yaml, NOTES.txt and the wiki all say in the same words that restricting that API is reverse-proxy work. The management port 9000 is a containerPort and a ClusterIP Service for in-cluster scraping, and is in no Ingress: it serves /health/* and optionally /metrics without authentication. Liveness uses /health/live and deliberately not /health or /health/ready - those include the database, and a liveness probe on them would restart the whole cluster for as long as the database is away. Also in this change: - First PodDisruptionBudget in this repository, which makes it a convention (README section 11): minAvailable as a value, and rendered only while the computed replica count is >= 2, because minAvailable: 1 with one replica blocks every node drain forever with a message that names the PDB rather than the replica count. - First quay.io image, so quay.io joins the trusted registries in ci/trivy/data/ksv0125.yaml - configured rather than suppressed, so AVD-KSV-0125 keeps being able to say something. - Renovate block with NO version constraint, which is the counter-case to garage and stalwart and is measured rather than assumed: docker versioning already excludes the -N rebuild tags, the floating minors and latest/nightly. Three dry-run fixtures are recorded at the block, and against the committed config 26.7.4 offers exactly 26.7.5 (patch) and 26.8.0 (minor). - No automerge exception, by decision (Gitea #83): the global automerge applies, and the minor-upgrade procedure plus `kc.sh update-compatibility` is documented in NOTES.txt instead. - Two upstream defaults deviated from, each with its reason at the value: db-pool-max-size 20 instead of 100 (two replicas at 100 ask a default PostgreSQL for twice its max_connections), and uid 1000 / gid 0 instead of this repo's 20000/1000, because the image declares keycloak:x:1000:0 and chowns its install tree to that pair. Part of #83
This commit is contained in:
jklein authored and 
Justin Klein committed
1 parent
a23d4977c5
commit
4c1cff1744
19 files changed
+2563
No files matched your search
@@ -0,0 +1,193 @@
|
||||
{{- $settings := fromYaml (include "keycloak.settings" .) -}}
|
||||
{{- $replicas := int (include "keycloak.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.keycloak.replicas)) -}}
|
||||
{{- $adminHostname := eq (include "keycloak.adminHostnameEnabled" .) "true" -}}
|
||||
{{- $deploy := printf "%s-%s-deployment" .Release.Name .Chart.Name -}}
|
||||
{{- $svc := printf "%s-%s-service" .Release.Name .Chart.Name -}}
|
||||
Keycloak {{ .Chart.AppVersion }} installed as release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
|
||||
|
||||
{{ $replicas }} replica(s), cache=ispn with jdbc-ping discovery over the database,
|
||||
external PostgreSQL at {{ $settings.db.host | default "<keycloak.db.host NOT SET>" }}:{{ $settings.db.port | int64 }}/{{ $settings.db.database }}.
|
||||
|
||||
ENDPOINTS
|
||||
{{ if .Values.ingress.enabled }} Public: {{ include "keycloak.frontendUrl" . }}
|
||||
{{ else }} Public: no Ingress (ingress.enabled: false) - in-cluster only
|
||||
{{ end }}{{ if $adminHostname }} Admin: {{ include "keycloak.adminUrl" . }}/admin
|
||||
{{ else }} Admin: no admin hostname (ingress.admin.enabled: false) - the console is
|
||||
served on the public host and Keycloak generates its URLs for it
|
||||
{{ end }} In-cluster: http://{{ $svc }}.{{ .Release.Namespace }}.svc.cluster.local:{{ $settings.ports.http | int64 }}
|
||||
|
||||
Management port {{ $settings.ports.management | int64 }} is NOT published and must never be: it serves
|
||||
/health, /health/live, /health/ready, /health/started{{ if $settings.metrics.enabled }} and /metrics{{ end }}
|
||||
WITHOUT authentication. It has a ClusterIP Service
|
||||
({{ .Release.Name }}-{{ .Chart.Name }}-management-service) for in-cluster scraping only.
|
||||
|
||||
THE 1PASSWORD ITEMS, AND WHICH FIELDS THEY MUST CARRY
|
||||
The operator materializes one Secret key per item FIELD, so a missing or
|
||||
misnamed field leaves the pod in CreateContainerConfigError - with no message
|
||||
naming the field.
|
||||
|
||||
{{ .Values.secrets.databaseSecretRef | default "<secrets.databaseSecretRef NOT SET>" }}
|
||||
username the PostgreSQL role
|
||||
password that role's password
|
||||
The role must be able to CREATE its schema: Keycloak builds and migrates
|
||||
the whole schema itself on first start.
|
||||
{{ if $settings.bootstrapAdmin.enabled }}
|
||||
{{ .Values.secrets.bootstrapAdminSecretRef | default "<secrets.bootstrapAdminSecretRef NOT SET>" }}
|
||||
username the TEMPORARY bootstrap administrator (upstream default name:
|
||||
temp-admin)
|
||||
password its password
|
||||
{{ else }}
|
||||
(no bootstrap admin item - keycloak.bootstrapAdmin.enabled is false, which is
|
||||
the correct state once a permanent administrator exists)
|
||||
{{ end }}
|
||||
Both arrive as ENVIRONMENT VARIABLES, not as files, and that is forced:
|
||||
Keycloak 26 has no _FILE form for an option's value. The price: the password
|
||||
is in the container's process environment.
|
||||
|
||||
FIRST INSTALL - AND STEP 4 IS MANDATORY, NOT ADVICE
|
||||
|
||||
1. kubectl -n {{ .Release.Namespace }} rollout status deployment/{{ $deploy }}
|
||||
Ready means /health/ready answered 200, which means the database is
|
||||
reachable and the schema is in place. The first start is the slow one: it
|
||||
creates the entire schema through Liquibase, and with {{ $replicas }} replica(s)
|
||||
the second waits on the first's migration lock.
|
||||
|
||||
2. Check that the cluster really formed - this is what the second replica is
|
||||
for, and a silent failure here leaves two independent islands:
|
||||
kubectl -n {{ .Release.Namespace }} logs deployment/{{ $deploy }} | grep -i "ISPN000094\|view accepted\|members"
|
||||
Look for a view with {{ $replicas }} members. One member per pod means
|
||||
jdbc-ping did not find the peer.
|
||||
|
||||
3. Log in at {{ if $adminHostname }}{{ include "keycloak.adminUrl" . }}/admin{{ else if .Values.ingress.enabled }}{{ include "keycloak.frontendUrl" . }}/admin{{ else }}the in-cluster address, port-forwarded{{ end }}
|
||||
with the bootstrap credentials from the vault item above.
|
||||
|
||||
4. CREATE A PERMANENT ADMINISTRATOR, then remove the temporary one.
|
||||
The bootstrap account is TEMPORARY by design - upstream: "the account
|
||||
should exist only for the duration necessary to perform operations needed
|
||||
to gain permanent and more secure admin access. After that, the account
|
||||
needs to be removed manually." This is different from Keycloak before 26,
|
||||
where KEYCLOAK_ADMIN created a regular administrator and there was nothing
|
||||
to clean up.
|
||||
a) In the master realm, create a user, set a password, assign the
|
||||
realm-admin / admin role.
|
||||
b) Verify you can log in as that user.
|
||||
c) Delete the temp-admin user.
|
||||
d) helm upgrade with keycloak.bootstrapAdmin.enabled: false
|
||||
|
||||
5. Until step 4d, EVERY START LOGS a line about the initial administrator not
|
||||
being created, because the master realm already exists and the bootstrap
|
||||
path is skipped. It is harmless. It is also the kind of message somebody
|
||||
eventually investigates, which is why 4d is part of the install and not a
|
||||
tidy-up.
|
||||
|
||||
MINOR UPGRADES NEED THE CLUSTER DOWN - AND THE AUTOMERGE DOES NOT KNOW THAT
|
||||
|
||||
Keycloak supports a rolling update only WITHIN a minor line. A minor jump
|
||||
(26.8 -> 26.9) requires every node shut down before any new node starts;
|
||||
mixing two minor versions in one Infinispan cluster is not a supported
|
||||
configuration, and nothing in the cluster reports that it happened.
|
||||
|
||||
THIS REPOSITORY'S RENOVATE AUTOMERGES EVERY UPDATE TYPE, MAJORS INCLUDED.
|
||||
That is a deliberate, accepted risk and not an oversight (Gitea #83), but it
|
||||
means an app-version PR that moves the minor can reach main, publish the
|
||||
chart and be rolled out as a rolling update without anybody deciding to.
|
||||
Watch app-version PRs for a changed minor, and if one has landed, take this
|
||||
path rather than letting the rollout happen:
|
||||
|
||||
1. Decide it with the tool instead of the version number. With the OLD
|
||||
version and the OLD configuration:
|
||||
kc.sh update-compatibility metadata --file=/tmp/kc-compat.json
|
||||
then with the NEW version and the NEW configuration:
|
||||
kc.sh update-compatibility check --file=/tmp/kc-compat.json
|
||||
Pass EVERY configuration option on both calls - upstream: "Omitting any
|
||||
configuration options results in incomplete metadata, and could lead to
|
||||
a wrong reported result".
|
||||
Exit codes, which is what to automate on: 0 rolling update possible,
|
||||
1 unexpected error, 2 invalid CLI option, 3 and 4 not possible - the
|
||||
deployment must be shut down.
|
||||
|
||||
2. If it says shut down:
|
||||
helm upgrade ... --set scaleDown=true # every pod stops
|
||||
# back up the database AND the configuration before anything else
|
||||
helm upgrade ... --set scaleDown=false # with the new version
|
||||
|
||||
A NOTE ON WHAT IS NOT SETTLED: upstream's 26.6 release notes call patch-level
|
||||
rolling updates supported and active by default, while the 26.8.0 page for
|
||||
update-compatibility says a rolling update is possible "when the Keycloak
|
||||
version is the same" and uses a PATCH pair (26.2.0 -> 26.2.1) as its example
|
||||
of an incompatible one. Those two cannot both be current. Until it is clear,
|
||||
run the check for patch updates too and trust its exit code rather than the
|
||||
size of the version step.
|
||||
|
||||
TRAPS WORTH KNOWING BEFORE YOU NEED THEM
|
||||
|
||||
- THE ADMIN HOSTNAME IS NOT AN ACCESS CONTROL. hostname-admin decides which
|
||||
URLs the server generates, nothing more. Upstream: "Using the
|
||||
hostname-admin option does not prevent accessing the Administration REST
|
||||
API endpoints via the frontend URL specified by the hostname option. If you
|
||||
want to restrict access to the Administration REST API, you need to do it
|
||||
on the reverse proxy level." So /admin/realms/... is reachable through the
|
||||
public host whatever you put on the admin host, and an IP allow-list on the
|
||||
admin Ingress restricts nothing on its own. This chart renders no deny rule
|
||||
for /admin - see values.yaml for why not.
|
||||
|
||||
- A SHARED DATABASE MERGES TWO CLUSTERS. Node discovery runs through the
|
||||
database, so - upstream - "clusters that share the same database but should
|
||||
remain separate will merge unless they have distinct cluster names. The
|
||||
default cluster name is ISPN." Two releases of this chart on one database
|
||||
therefore form ONE cluster, silently.
|
||||
{{ with $settings.cache.clusterName }}This release sets cache-embedded-cluster-name={{ . }}.{{ else }}This release uses the default name (ISPN) - keycloak.cache.clusterName is
|
||||
unset. Read its comment in values.yaml before setting it: upstream also
|
||||
warns about cache invalidation between differently-named clusters.{{ end }}
|
||||
|
||||
- THERE IS NO ConfigMap IN THIS CHART, so no checksum/config annotation. That
|
||||
is not an omission: Keycloak is configured entirely through options and
|
||||
KC_* variables, so there is no config file whose change could fail to
|
||||
restart anything.
|
||||
|
||||
- A CHANGE INSIDE A VAULT ITEM RESTARTS NOTHING. Helm never sees it. After
|
||||
rotating a field:
|
||||
kubectl -n {{ .Release.Namespace }} rollout restart deployment/{{ $deploy }}
|
||||
(Repointing a secrets.*SecretRef at a different item DOES roll the pods -
|
||||
that is covered by checksum/secret-ref.)
|
||||
|
||||
- THE POD HAS TWO INIT CONTAINERS AND NEEDS BOTH. db, health-enabled and
|
||||
metrics-enabled are Keycloak BUILD options, the published image is built
|
||||
with their defaults, and a runtime change makes Keycloak rewrite
|
||||
/opt/keycloak/lib/quarkus at start - which readOnlyRootFilesystem: true
|
||||
forbids. So one init container seeds that directory into a volume and the
|
||||
next runs `kc.sh build` into it; the serving container then runs
|
||||
`start --optimized`. If a pod is stuck in Init, read those logs first:
|
||||
kubectl -n {{ .Release.Namespace }} logs <pod> -c {{ .Release.Name }}-{{ .Chart.Name }}-build
|
||||
Build options are set ONLY through keycloak.db.vendor,
|
||||
keycloak.metrics.enabled and keycloak.features. Putting one into
|
||||
keycloak.env fails the render on purpose.
|
||||
|
||||
- POOL SIZES ARE PER REPLICA. keycloak.db.pool.maxSize is {{ $settings.db.pool.maxSize | int64 }}, so with the
|
||||
current {{ $replicas }} replica(s) this release can open up to {{ mul ($settings.db.pool.maxSize | int64) $replicas }} connections.
|
||||
Upstream's default is 100 per node, which with two nodes exceeds a default
|
||||
PostgreSQL's max_connections of 100. Raise it together with the server,
|
||||
never alone.
|
||||
{{ if and $settings.podDisruptionBudget.enabled (ge $replicas 2) }}
|
||||
- A PodDisruptionBudget IS in place ({{ .Release.Name }}-{{ .Chart.Name }}-pdb, minAvailable
|
||||
{{ $settings.podDisruptionBudget.minAvailable | int64 }}), so a node drain cannot take all replicas at once.
|
||||
{{ else if $settings.podDisruptionBudget.enabled }}
|
||||
- NO PodDisruptionBudget was rendered, because the effective replica count is
|
||||
{{ $replicas }}. That is deliberate: minAvailable 1 with a single replica forbids every
|
||||
voluntary eviction, so `kubectl drain` would block forever on a pod nothing
|
||||
may move - and the error would name the PDB rather than the replica count.
|
||||
Run two replicas to get the budget.
|
||||
{{ else }}
|
||||
- NO PodDisruptionBudget (keycloak.podDisruptionBudget.enabled is false). A
|
||||
node drain may take every replica at once.
|
||||
{{ end }}
|
||||
WHAT THIS CHART DOES NOT RENDER
|
||||
|
||||
- No PVC and no StatefulSet: Keycloak 26 keeps everything in the database,
|
||||
user sessions included (persistent sessions are the default; volatile ones
|
||||
are deprecated as of 26.8). There is no disk state to keep.
|
||||
- No NetworkPolicy. jdbc-ping opens direct pod-to-pod connections on port
|
||||
{{ $settings.ports.jgroups | int64 }}, so any policy added later has to allow that between the
|
||||
replicas, or the cluster silently becomes two islands.
|
||||
- No deny rule for /admin on the public Ingress, and no restriction of the
|
||||
administration REST API. That is reverse-proxy work - see the first trap.
|
||||
Reference in new issue
Block a user