jlab-cluster/helm-charts
3
0 Fork 0
Code Issues 15 Pull Requests 3 Actions Packages Wiki Activity
Page: keycloak
Pages
CI CD Chart Style Guide Charts Forge and Tooling Home Lessons Learned Local Development New Chart Renovate Versioning Workflow apache tika bentopdf bulwark calibre web automated convertx cyberchef freesailarr garage gotenberg homeassistant keycloak matrix synapse minecraft router minecraft server outline paperless ngx patchmon rallly rss bridge samba satisfactory searxng stalwart sure teamspeak server template chart urlaubsverwaltung valkey voucher vault ytdlp web player zipline
Clone
12
keycloak
Justin Klein edited this page 2026-10-14 21:21:24 +00:00
Unescape Escape
Table of Contents
This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

keycloak

Deploys Keycloak — an OpenID Connect and SAML identity provider — as a two-node cluster against an external PostgreSQL.

Two replicas is the point of this chart rather than a tuning option. A mail server may be away briefly because SMTP retries; a blob store has one consumer. An SSO server takes every dependent login down with it. And the second replica is cheap here, because node discovery runs over the database that is there anyway: cache=ispn plus Keycloak's default jdbc-ping stack means no headless Service, no JGroups DNS wiring and no extra option.

What it costs is the thing to know before the first upgrade: Keycloak supports a rolling update only within a minor line. A minor jump needs the cluster shut down, and this repository's Renovate automerges every update type. See Minor upgrades — that section is the reason this page exists as much as the values table is.
Image quay.io/keycloak/keycloak — the only registry upstream publishes to, and the first quay.io image in this repo (hence the entry in ci/trivy/data/ksv0125.yaml)
Versions not listed on purpose — read the current pair off keycloak/Chart.yaml, see Chart index
Workloads Deployment, 2 replicas, two init containers — see Why two init containers
Storage none. No PVC: Keycloak 26 keeps everything in the database, user sessions included (persistent sessions are the default; volatile ones are deprecated as of 26.8)
Routing two mutually exclusive modes, Ingress (default) or Gateway API (httpRoute.enabled) — and two objects in either mode: the public hostname and the admin hostname (switchable off), plus one object per keycloak.realmHosts entry — empty by default — which gives a realm a hostname of its own (see One hostname per realm). The admin one is not an access control — see the first trap. In Gateway mode the proxy-buffer annotations this installation depends on are gone and have no counterpart — see Routing: Ingress or Gateway API
External dependencies an external PostgreSQL. Nothing else
Secrets 2 × OnePasswordItem, four fields in total
Network ClusterIP Service on 8080 (http) + a second on 9000 (management), up to four Ingresses, plus a NetworkPolicy on by default — the only one in the repo with a source-restricted http rule
Disruption PodDisruptionBudget, minAvailable: 1 — the first in this repo, and now a convention (style guide §11)
Install-test ci/keycloak/ — fixtures + test values, with two replicas and a throwaway PostgreSQL

Prerequisites

  • An external PostgreSQL, reachable from the cluster, with a database and a role that can create its own schema: Keycloak builds and migrates the entire schema itself through Liquibase on first start. A read-write grant on existing tables is not enough.
  • 1Password operator in the cluster, plus two items — see What the 1Password items must contain.
  • Two DNS names and a cert-manager cluster issuer, unless the admin hostname is switched off — one for the public endpoint, one for the administration console. In Gateway mode the cluster issuer is not needed from this release at all: both certificates sit on the Gateway's listeners. What is needed instead is the Gateway-API CRDs plus a Gateway whose listener admits routes from this namespace — and, before DNS is pointed at it, the proxy-buffer equivalent configured next to that Gateway (why).
  • Nothing else. No volume, no storage class, no cache service.

Installation

The registry URL and its credentials live in 1Password (item jlab-registry-access: username, password, charts-url) and never in a repository or a shell history file.

helm repo add jlab "$CHARTS_URL" --username "$REG_USER" --password "$REG_PASS"
helm repo update

helm install keycloak jlab/keycloak \
  --namespace keycloak --create-namespace \
  -f my-values.yaml

Minimal my-values.yaml:

secrets:
  databaseSecretRef: op://vault/keycloak-db
  bootstrapAdminSecretRef: op://vault/keycloak-bootstrap-admin

keycloak:
  db:
    host: postgres.example.internal

ingress:
  domain: id.example.com
  clusterIssuer: letsencrypt
  admin:
    domain: id-admin.example.com

Five values, and four of them have no default because no default could be right. helm install prints the chart's NOTES.txt, which names the four vault fields, the verification sequence and the traps in short form. Read it — it is the per-release version of this page.

The same installation in Gateway mode instead — the Gateways, their classes and their certificates belong to the cluster, not to this release:

ingress:
  enabled: false            # mutually exclusive with httpRoute.enabled
  domain: id.example.com    # STILL REQUIRED: it is KC_HOSTNAME as well
  admin:
    domain: id-admin.example.com

httpRoute:
  enabled: true
  parentRefs:
    - name: public
      namespace: gateway
      sectionName: https
  admin:
    # Empty falls back to httpRoute.parentRefs. An INTERNAL Gateway here is
    # what an internal ingress class is in the other mode.
    parentRefs:
      - name: internal
        namespace: gateway
        sectionName: https

This renders two routes, <release>-keycloak-httproute and <release>-keycloak-admin-httproute — plus one <release>-keycloak-realm-httproute… per keycloak.realmHosts entry — and nothing else of the routing path. ingress.clusterIssuer is not used; ingress.domain is still required because the Deployment reads it as KC_HOSTNAME. Set the proxy-buffer equivalent at the Gateway before pointing DNS at it — see Routing: Ingress or Gateway API.

What the 1Password items must contain

The chart renders two OnePasswordItems; the operator turns each into a Kubernetes Secret whose keys are the field names of the 1Password item. A missing or misnamed field leaves the pod in CreateContainerConfigError — with no message naming the field.

Values key Kubernetes Secret Field Content
secrets.databaseSecretRef <release>-keycloak-db-secret username the PostgreSQL role. It must be able to create its schema
password that role's password
secrets.bootstrapAdminSecretRef <release>-keycloak-bootstrap-admin-secret username the temporary bootstrap administrator (upstream's own default name is temp-admin)
password its password

Host, port and database name are deliberately not in the vault — they are plain values (keycloak.db.host / .port / .database), because they are not secret and because moving a database should not mean editing a vault entry.

Both arrive as environment variables, not as files, and that is forced rather than preferred. Keycloak 26 has no _FILE indirection for an option's value: there is no KC_DB_PASSWORD_FILE and no generic KC_<OPTION>_FILE. Checked against the 26.8.0 option table — every KC_*_FILE variable there is a path option (certificate, key store, trust store, log file, export file), i.e. "point at this file", never "read this option's value from a file". So the file form valkey argues for is unavailable here, as it was for garage for a completely different reason. The residual price, stated rather than hidden: the password is in the container's process environment, readable by anyone who can exec into the pod. (Keycloak's own Vault SPI, KC_VAULT=file, is a different and larger mechanism and deliberately not used.)

First install: the checklist

Step 4 is mandatory, not advice.

  1. The rollout.

    kubectl -n keycloak rollout status deployment/keycloak-keycloak-deployment

    Ready means /health/ready answered 200, which means the database is reachable and the schema is in place. The first start is the slow one: it creates the whole schema, and with two replicas the second waits on the first's migration lock. If a pod sits in Init, read the init containers first — see Why two init containers.

  2. Check that the cluster really formed. No probe replaces this step and neither does the install-test. A node that never found its peer is Ready, healthy and alone — readiness reports this node's own health — and two such nodes are two islands that each believe they are the cluster.

    kubectl -n keycloak logs deployment/keycloak-keycloak-deployment \
  | grep -i "ISPN000094\|view accepted\|members"

    Look for a view with two members. One member per pod means jdbc-ping did not find the peer: check that both pods can reach each other on port 7800 and that they really share one database.

  3. Log in at https://<ingress.admin.domain>/admin with the bootstrap credentials from the vault item.

  4. Create a permanent administrator, then remove the temporary one. The bootstrap account is temporary by design — upstream: "the account should exist only for the duration necessary to perform operations needed to gain permanent and more secure admin access. After that, the account needs to be removed manually." This is a change from Keycloak before 26, where KEYCLOAK_ADMIN created a regular administrator and there was nothing to clean up.

    1. In the master realm, create a user, set a password, assign the admin role.
    2. Verify you can log in as that user — before deleting anything.
    3. Delete the temp-admin user.
    4. helm upgrade with keycloak.bootstrapAdmin.enabled: false. That drops the variables and the vault item from the release.

  5. Until step 4.4, every start logs a line about the initial administrator not being created, because the master realm already exists and the bootstrap path is skipped. It is harmless — and it is exactly the kind of message somebody eventually investigates, which is why 4.4 is part of the install rather than a tidy-up.

Minor upgrades need the cluster down

Keycloak supports a rolling update only within a minor line. A minor jump (26.8 → 26.9) requires every node shut down before any new node starts; mixing two minor versions in one Infinispan cluster is not a supported configuration, and nothing in the cluster reports that it happened.

This repository's Renovate automerges every update type, majors included. That was decided deliberately rather than overlooked, and it means an app-version PR that moves the minor can reach main, publish the chart and be rolled out as a rolling update without anybody choosing to. So the procedure is a human obligation here, not a guard rail:

  1. Decide it with the tool, not with the version number. With the old version and the old configuration:

    kc.sh update-compatibility metadata --file=/tmp/kc-compat.json

    then with the new version and the new configuration:

    kc.sh update-compatibility check --file=/tmp/kc-compat.json

    Pass every configuration option on both calls — upstream: "Omitting any configuration options results in incomplete metadata, and could lead to a wrong reported result." Automate on the exit code, which is what it is for:

    Exit code Meaning
    0 rolling update is possible
    1 unexpected error (metadata file missing or corrupted)
    2 invalid CLI option
    3 / 4 not possible — the deployment must be shut down first

  2. If it says shut down:

    helm upgrade ... --set scaleDown=true     # every pod stops
# back up the database AND the configuration before anything else
helm upgrade ... --set scaleDown=false    # with the new version

One thing is not settled, and it is recorded rather than guessed. Upstream's 26.6 release notes call patch-level rolling updates supported and enabled by default, while the 26.8.0 page for update-compatibility says a rolling update is possible "when the Keycloak version is the same" and uses a patch pair (26.2.0 → 26.2.1) as its own example of an incompatible one. Those two statements cannot both be current. Until it is clear which is: run the check for patch updates too, and trust its exit code rather than the size of the version step.

Why this chart has two init containers

This is the part of the chart that looks like over-engineering until the four facts behind it are on the table. All four were measured for Gitea #83.

  1. db, health-enabled and metrics-enabled are Keycloak build options, not runtime options — each carries the tool icon in the 26.8.0 option table. (cache does not, which is why cache=ispn costs nothing here.) Their defaults are dev-file, false and false.
  2. The published image is augmented with those defaults. /opt/keycloak/lib/quarkus/ in the image holds generated-bytecode.jar, transformed-bytecode.jar, quarkus-application.dat and build-system.properties; conf/keycloak.conf is entirely commented out.
  3. Supplying different build options at runtime makes Keycloak re-augment itself at start, rewriting exactly those four files — inside the root filesystem. kc.sh says so itself: "only exit code 10 means that implicit reaugmentation occurred and a relaunch is needed." Upstream's container guide names the consequence: that start "significantly increases startup time and creates an image that is mutable". A mutable image is precisely what readOnlyRootFilesystem: true forbids — and "then don't set db" is no escape, because the probes alone force it: health-enabled is a build option too.
  4. An empty emptyDir over that directory does not work either, and this is the fact that is not guessable from the outside: QuarkusEntryPoint — the main class kc.sh runs for every command, build included — loads quarkus/quarkus-application.dat before it dispatches anything. With that file hidden by a fresh mount, kc.sh build cannot start.

So the pod is:

container mounts the build volume at does root filesystem
<release>-keycloak-seed (init) /quarkus-build cp -R /opt/keycloak/lib/quarkus/. /quarkus-build/ read-only
<release>-keycloak-build (init) /opt/keycloak/lib/quarkus kc.sh build read-only
<release>-keycloak-deployment /opt/keycloak/lib/quarkus kc.sh start --optimized read-only

Which is why it is two init containers and not one with a writable root filesystem: every container in this pod keeps the full hardening baseline. There is no documented deviation to carry, and no Trivy suppression that would also have silenced the serving container.

Three practical consequences:

  • Build options exist in exactly one place (the keycloak.buildEnv helper, which only the build container uses). The serving container passes start --optimized and no build option at all, so there is no second list that could drift. A build option's variable name in keycloak.env fails the render, naming the variable — because the failure it prevents is a crash loop whose cause is nowhere near its message.

  • Changing keycloak.db.vendor, keycloak.metrics.enabled or keycloak.features re-runs the build on the next pod start. That is correct and costs one init-container run; it migrates no data.

  • A pod stuck in Init is a build problem, and its log is one flag away:

    kubectl -n keycloak logs <pod> -c <release>-keycloak-build

Getting a file into the pod: themes, providers, extra CAs

There is no PVC and no ConfigMap in this chart, so until Gitea #88 nothing could put a file into this pod at all. keycloak.extraVolumes and keycloak.extraVolumeMounts are the generic way: both lists go verbatim into the pod spec, so a volume may be a ConfigMap, a Secret, a PVC or an emptyDir filled by an init container of your own. The chart does not decide how a theme is delivered.

The mounts are rendered twice — on the build init container and on the serving container. One list, two places, because the two cases resolve at different times:

  • A theme is read at runtime, so the serving container is where it takes effect — even though that container serves a pre-built installation.
  • An SPI provider is a build-time concern. Upstream, on dropping a JAR into providers/: "After that if you are using --optimized, you must run the build command in order to update the server's provider registry with the implementations from the JAR file." That build is the init container above. A JAR mounted only on the serving container is absent while the registry is written — it is never registered, the file sits there at runtime, and nothing reports it.

Not on the seed init container: it has a different layout (/quarkus-build) and runs no Keycloak command.

The side effect of the second place, which belongs stated rather than discovered: an extra volume is present at build time too. For a theme that is without consequence; for a Secret it means an init container of this pod sees it as well.

It places files; it configures nothing

  • An extra root CA is not trusted by being mounted. A file in the pod is in no truststore. Keycloak reads one only from truststore-paths, so the mount needs KC_TRUSTSTORE_PATHS pointed at the mounted directory. That is a runtime option (26.8.0 option table, no build icon), so it goes into keycloak.env and the build-option guard does not refuse it.
  • A theme is not activated by being mounted, and this chart cannot activate it. The selection lives per realm in Keycloak's database — Realm settings → Themes → Login theme. Mounting makes a theme selectable, never selected.

A custom login theme

The reason the issue was opened: Keycloak has no logo field on an identity provider. The icon on its login button comes only from a login theme, through kcLogoIdP-<alias> in that theme's theme.properties.

keycloak:
  extraVolumes:
    - name: login-theme
      configMap:
        name: my-login-theme
        # A theme's layout is <theme>/<type>/theme.properties, and a ConfigMap
        # key may not contain a slash — so the subdirectory comes from
        # items[].path, which may.
        items:
          - key: theme.properties
            path: login/theme.properties
  extraVolumeMounts:
    - name: login-theme
      # NOT /opt/keycloak/themes — see the trap below.
      mountPath: /opt/keycloak/themes/my-theme
      readOnly: true

with a theme.properties of

parent=keycloak.v2
kcLogoIdP-my-provider = fa fa-lock

and then Realm settings → Themes → Login theme → my-theme plus a pod restart.

Four properties of this route that cost time if they are not known:

  1. A ConfigMap carries binary content only in binaryData (base64). An SVG is text and fits in data; a PNG does not. The 1 MiB limit per ConfigMap is ample for a theme with SVG icons and not enough for bundled web fonts — those belong in a PVC or an image of your own.
  2. Themes are cached, and in production that is wanted. A theme change therefore takes effect only after a pod restart. Two things add up: this chart hashes no user-supplied ConfigMap either (it renders none of its own, so there is no checksum/config), so editing the theme ConfigMap restarts nothing by itself. Use kubectl -n keycloak rollout restart deployment/<release>-keycloak-deployment. Do not configure the cache away to avoid this.
  3. The realm selection is not the chart's — see above.
  4. A custom login theme can make the identity-provider buttons disappear entirely when its template does not include the social fragment correctly (reported repeatedly upstream, keycloak/keycloak#42711). After switching a realm's theme, look at the login page — a healthy pod says nothing about whether the buttons are still there.

Configurable values

Global

Key Type Default Description
scaleDown bool false Renders the Deployment with replicas: 0 without uninstalling. Takes precedence over keycloak.replicas. Also half of the minor-upgrade procedure above.

secrets

Key Type Default Description
secrets.databaseSecretRef string null Required always. op:// path of the item carrying username and password.
secrets.bootstrapAdminSecretRef string null Required while keycloak.bootstrapAdmin.enabled is true (the default).

keycloak

Key Type Default Description
keycloak.replicas int 2 An explicit 0 is allowed and is the per-workload way to stop the release. Above 1 the rolling-update restriction applies; below 2 no PDB is rendered.
keycloak.db.vendor string postgres A build option — it reaches kc.sh build, never the serving container. dev-file and dev-mem are rejected by the schema: they are Keycloak's development defaults, and an identity provider on an embedded database works until the pod moves. Only postgres is exercised by CI.
keycloak.db.host string null Required. Enforced by required() in the Deployment — a null would render an empty KC_DB_URL_HOST and a pod waiting on a connection it can never make.
keycloak.db.port int 5432
keycloak.db.database string keycloak
keycloak.db.pool.initialSize / .minSize int 2 / 2 No upstream default. A few warm connections remove the first-request latency after an idle period.
keycloak.db.pool.maxSize int 20 Deliberately not upstream's 100, and the sizes are per replica: two replicas at 100 would ask a default PostgreSQL (max_connections = 100) for twice what it has, and the failure arrives under load as "too many clients already" rather than at install time. Raise it together with the server's max_connections, and multiply by keycloak.replicas when you do.
keycloak.bootstrapAdmin.enabled bool true The temporary bootstrap administrator. Switch it off once a permanent administrator exists — see step 4 above.
keycloak.metrics.enabled bool true A build option, and on for the readiness probe rather than for the metrics — see the readiness trap. The price: /metrics then answers on the management port, unauthenticated, carrying realm names, client ids and login counters. That port is ClusterIP-only and in no Ingress. Switching this off is a decision about whether readiness can fail.
keycloak.features list [] Extra feature flags (--features). A build option. A list and not a comma-separated string, so two entries cannot arrive as one unknown feature name — which Keycloak rejects at build time, inside an init container.
keycloak.cache.clusterName string null --cache-embedded-cluster-name; null means Keycloak's default ISPN. Read the trap before setting it.
keycloak.ports.http int 8080
keycloak.ports.management int 9000 /health/* and optionally /metrics, unauthenticated. All three probes address it by name, so renaming the port breaks them. In no Ingress, by design.
keycloak.ports.jgroups int 7800 Declared as a containerPort for the record; nothing serves it and no Service exposes it. jdbc-ping opens direct pod-to-pod connections on it.
keycloak.resources.limitFactor number 3 Memory and ephemeral-storage limits are computed as factor × request.
keycloak.resources.requests object cpu: 500m, memory: 700Mi, ephemeral-storage: 256Mi 700Mi × 3 = 2100Mi, and that number is load-bearing rather than tidy: the image sizes the JVM heap relative to the container limit (-XX:MaxRAMPercentage=70), so the limit is what the heap is computed from. Upstream: "you should always set the memory limit", "at least 750 MB", and "the recommended memory limit is 2 GB". Lowering the request shrinks the heap proportionally.
keycloak.build.resources object cpu: 500m, memory: 512Mi, ephemeral-storage: 256Mi, factor 3 The two init containers. Transient — they are gone before the serving container starts — but they count for scheduling: a pod's effective request is the maximum of its init containers' and the sum of its containers'. More CPU here shortens every pod start.
keycloak.livenessProbe object httpGet /health/live on management, period 10s, timeout 5s, 3 failures Deliberately not /health or /health/ready — see the probes.
keycloak.readinessProbe object httpGet /health/ready on management, period 10s, timeout 5s, 3 failures Includes the database, and must.
keycloak.startupProbe object httpGet /health/started on management, period 10s, 60 failures (= 600s) Gates liveness and readiness. The budget covers a first start, which creates the whole schema through Liquibase while the second replica waits on the migration lock. The Quarkus augmentation is not inside it — that runs in an init container, which has no probe and no deadline.
keycloak.securityContext.pod object non-root 1000, runAsGroup 1000, fsGroup 1000, fsGroupChangePolicy: Always, seccompProfile: RuntimeDefault uid 1000 is the image's requirement: it declares keycloak:x:1000:0 in /etc/passwd and copies the install tree with --chown=1000:0, so 1000 owns everything the process reads. The group is 1000 and not the image's own 0, which is a deliberate divergence: upstream puts the user in the root group and makes the tree group-writable, which is the OpenShift arbitrary-uid pattern — it lets any uid in group 0 write. Counted across the image layer, 521 of 522 entries are owned by 1000:0 with every mode granting the owner full access, so pinning the owning uid makes group 0 unnecessary. What that buys: no root GID in this pod, hence no AVD-KSV-0116 finding and no suppression for it — a suppression would have had to cover the serving container too.
keycloak.securityContext.container object readOnlyRootFilesystem: true, all caps dropped, no privilege escalation, non-root Applied to every container in the pod, both init containers included. No capability is added and none is needed: 8080, 9000 and 7800 are all above 1024 and the entrypoint is a shell script calling java.
keycloak.podDisruptionBudget.enabled bool true
keycloak.podDisruptionBudget.minAvailable int 1 Rendered only while the computed replica count is >= 2 — see the PDB trap.
keycloak.env list [] Extra environment variables, rendered through tpl; value, secretKeyRef and configMapKeyRef forms. Additive, not overriding — see §10. A build option's variable name is refused outright (KC_DB, KC_HEALTH_ENABLED, KC_METRICS_ENABLED, KC_FEATURES, …). Useful entries: JAVA_OPTS_APPEND, KC_LOG_LEVEL, KC_SPI_*, and KC_TRUSTSTORE_PATHS for a CA mounted through keycloak.extraVolumes.
keycloak.extraVolumes list [] Additional pod volumes, rendered verbatim — any Kubernetes volume source. The way a file gets into this pod at all; see Getting a file into the pod. It places files and configures nothing: a CA still needs KC_TRUSTSTORE_PATHS, a theme still needs to be selected per realm.
keycloak.extraVolumeMounts list [] Additional mounts, rendered verbatim and twice — on the build init container as well as on the serving container. Deliberate: a theme is read at runtime, but an SPI provider JAR must be present while kc.sh build writes the provider registry, or it is never registered and nothing reports it. Never mount on /opt/keycloak/themes itself — see the trap.
keycloak.realmHosts list [] One hostname per realm, rendered in both routing modes (Gitea #122) — one Ingress per entry with nginx.ingress.kubernetes.io/app-root, or one HTTPRoute per entry whose Exact match on / carries a RequestRedirect filter. Empty renders nothing at all. Each entry: host and realm required, tlsSecretName required in Ingress mode, clusterIssuer, annotations and className optional there, parentRefs optional in Gateway mode. Object names carry the list index, so reordering renames and recreates them. See One hostname per realm.

ingress

Key Type Default Description
ingress.enabled bool true Gates both Ingress objects. Mutually exclusive with httpRoute.enabled. Until the Gateway-API support landed it gated only the public one, so ingress.enabled: false left the admin Ingress rendering — see Routing.
ingress.domain string null Required in every mode, including with both of them off — §12.1 checked in both directions: it is read a second time as KC_HOSTNAME, and Keycloak refuses to start in production mode without a hostname (hostname-strict defaults to true). The chart renders it as the full URL https://<domain>, which upstream requires as soon as hostname-admin is in play. Also the public HTTPRoute's hostname while httpRoute.hostnames is empty.
ingress.clusterIssuer string null Required while the Ingress is enabled. Read only in the Ingress template, so Gateway mode frees it — the certificates sit on the Gateway's listeners there.
ingress.className string nginx No counterpart in Gateway mode: the class sits on the Gateway.
ingress.annotations object {} Merged with the cert-manager annotation the chart always sets. No counterpart in Gateway mode — and for this chart that is the consequential one: this installation's proxy-buffer annotations live here. See Routing.
ingress.admin.enabled bool true The administration console on a hostname of its own, and not an Ingress switch: it also decides whether KC_HOSTNAME_ADMIN is set on the Deployment, so it gates the admin object in both modes and switching to Gateway mode does not require turning it off. Not an access control — see the first trap.
ingress.admin.domain string null Required while enabled. Also rendered as https://<domain>. Also the admin HTTPRoute's hostname while httpRoute.admin.hostnames is empty.
ingress.admin.className string null Falls back to ingress.className. An internal ingress class here is the useful half of the admin hostname. Its Gateway-mode counterpart is httpRoute.admin.parentRefs, not a class.
ingress.admin.annotations object {} Where an IP allow-list would go — with the caveat below firmly in mind. No counterpart in Gateway mode.

TLS secrets: tls-<release>-keycloak-ingress and tls-<release>-keycloak-admin-ingress. Two separate certificates, because these are two hostnames that may well sit on two different ingress classes — the shared-secret form in the style guide is for several objects on one host. In Gateway mode the chart renders neither secret and neither certificate.

httpRoute

The Gateway-API alternative to the two Ingresses, off by default. Added with group 3 of the Gateway-API sweep (Gitea #114); the form is the pilot's and is binding for every chart (style guide §4).

Key Type Default Description
httpRoute.enabled bool false Renders <release>-keycloak-httproute and — while ingress.admin.enabled — <release>-keycloak-admin-httproute, instead of the two Ingresses. Mutually exclusive with ingress.enabled: with both true the render aborts naming both values.
httpRoute.parentRefs list [] The Gateway(s) the public route attaches to — Gateway API ParentReference entries (name, and optionally namespace, sectionName, port, group, kind), passed through unchanged. Required while httpRoute.enabled is true: the render aborts on an empty list. The schema requires name on every entry and is otherwise left open, because Kubernetes owns that schema and adds fields to it across releases.
httpRoute.hostnames list [] Empty means [ingress.domain], so the public hostname has a single source of truth. Setting it is for a second or additional name — and note ingress.domain stays mandatory regardless, because it is also KC_HOSTNAME.
httpRoute.admin.parentRefs list [] The admin route's own parent. Empty falls back to httpRoute.parentRefs — the exact counterpart of ingress.admin.className falling back to ingress.className, because "which data path" is a property of the Gateway in this API rather than of the route. An internal Gateway here is what an internal ingress class is in the other mode, and still not the access control the first trap is about.
httpRoute.admin.hostnames list [] Empty means [ingress.admin.domain].

networkPolicy

The richest policy in the repository, and the one with the largest blast radius. Keycloak is the only chart whose three ports fall into three different classes, so it is where the convention is visible in full:

  • http (keycloak.ports.http, 8080): restricted by source — this namespace, allowedNamespaces, and the ingress controller. The only genuinely source-restricted inbound rule in the whole sweep, and possible only because no probe uses this port.
  • management (9000): the port all three probes address, so it stays open inside the cluster. This is the surface that carries health and metrics unauthenticated.
  • jgroups (7800): this release's own pods only — a podSelector with no namespaceSelector, the narrowest rule the object can express.

Outbound: DNS, the configured database port (keycloak.db.port — this chart does not have to guess), the cluster bus, and 443 for identity brokering.

Value Type Default Meaning
networkPolicy.enabled bool true Renders the policy. Only a boundary where the CNI enforces one — unenforced, the API server accepts it and nothing is filtered, so everything reads as configured.
networkPolicy.allowedNamespaces list [] Namespaces whose pods may reach the restricted ports, additionally to the release's own. Matched on the kubernetes.io/metadata.name label Kubernetes sets itself — no labelling work, and no typo protection either: a name that does not exist matches nothing and is silently ineffective. Namespace-granular, so an entry opens the port to every pod in it.
networkPolicy.probeSources list [] CIDRs allowed to reach the probed port. Empty means open, not blocked — the rule is then rendered with no from at all. Probes come from the IP of the worker node running the pod, which the chart cannot know; a list that misses it drops every probe and the pod never becomes Ready.
networkPolicy.ingressController object kube-system / app.kubernetes.io/name: rke2-traefik The controller allowed to reach the routed port once probeSources restricts it. The two keys render as one from entry, i.e. an AND — which is why this is not a line in allowedNamespaces, where it would open the port to every pod in kube-system. Default measured on the cluster, not assumed.
networkPolicy.egress.extraPorts list [] Additional outbound TCP ports, to any destination. The escape hatch from the egress cap, and the only alternative to switching the policy off.

Operating notes

Routing: Ingress or Gateway API

Two mutually exclusive modes, and exactly one of them may be on. ingress.enabled ships true, httpRoute.enabled ships false. Two routing objects in either mode — the public hostname and the admin hostname.

Rule Why
ingress.enabled and httpRoute.enabled cannot both be true — the render aborts naming both value paths two data paths onto one Service under one hostname is a conflict at DNS level already. For Keycloak it is worse than for a stateless app: the server derives KC_HOSTNAME from ingress.domain and generates absolute redirect and asset URLs for it, so a login that starts on one path and finishes on the other fails on the state cookie, not with an error naming the cause. A migration runs over a second hostname (httpRoute.hostnames set explicitly while the Ingress keeps ingress.domain), never in parallel.
httpRoute.parentRefs must not be empty — the render aborts a route without a parent attaches to no Gateway. The object is created, it is schema-valid, no controller rejects it, and it routes nothing: no event, no condition, and the only symptom is a hostname that does not answer.
ingress.admin.enabled gates the admin object in both modes it is not an Ingress switch. It also decides whether KC_HOSTNAME_ADMIN is set on the Deployment, i.e. whether the admin hostname is in play at all — so switching to Gateway mode must not require turning it off, which would silently remove the feature.
httpRoute.admin.parentRefs falls back to httpRoute.parentRefs the counterpart of ingress.admin.className falling back to ingress.className. "Which data path" is a property of the Gateway in this API, not of the route, so an internal Gateway here is what an internal ingress class is there. Without the fallback the admin route could only ever hang off the same Gateway as the public one, and the one useful half of the second hostname would be gone in this mode.

The admin Ingress did not use to hang off ingress.enabled, and that was fixed with this support rather than worked around. ingress.admin.enabled gated it alone, so helm template --set ingress.enabled=false emitted <release>-keycloak-admin-ingress and nothing else of the routing path: "no Ingress" did not mean no Ingress, and ingress.clusterIssuer stayed required for an object nobody asked for. It also made Gateway mode impossible — switching httpRoute on leaves ingress.enabled false, and the admin Ingress would have gone on claiming the admin host next to the admin route. The default render is unchanged; what changed is a combination that could not be coherent.

What Gateway mode costs this chart, and it is not hypothetical

Measured on the live cluster on 2026-10-12, on both of this chart's Ingress objects:

nginx.ingress.kubernetes.io/proxy-buffer-size:    16k
nginx.ingress.kubernetes.io/proxy-buffers-number: 4

The chart sets neither. They are operator-supplied, through ingress.annotations and ingress.admin.annotations — which is why reading this chart, or grepping the repository, does not reveal them. And they are not cosmetic: Keycloak's responses carry large headers (JWTs, SAML assertions, long Set-Cookie chains during the login flow), and a proxy whose buffer is too small answers 502 itself while Keycloak logs the request as successful. The symptom points at the application; the cause is at the proxy.

In Gateway mode both are gone, and the Gateway API has no counterpart: buffer sizing is implementation policy, not routing, and this chart renders no implementation-specific policy object on purpose — that would bind it to one Gateway implementation and to a CRD whose presence it cannot check (style guide §4). So the chart names the loss instead of rebuilding it: the Gateway branch of NOTES.txt prints it, and values.yaml and the schema carry it.

Before moving this host: set the equivalent next to the Gateway, in that implementation's own object, and raise it rather than merely matching 16k if long-lived tokens or SAML are in play.

One hostname per realm

keycloak.realmHosts is empty by default and renders nothing. Filled, each entry publishes this release's Service under a further hostname and sends that hostname's bare / to one realm's account console at /realms/<realm>/account/. It exists because Keycloak serves every realm from one deployment, so a realm that wants a name of its own needs a routing object and nothing else.

It renders in both modes, which is the convention rather than a bonus: a routing value in this repository is mode-independent by design — ingress.domain is read in Gateway mode too, and ingress.hosts is read in both for the charts that have no ingress.domain — so a feature that rendered in one mode only would silently disappear when an installation switched (style guide §4):

Mode Object per entry How the bare / reaches the realm
Ingress <release>-keycloak-realm-ingress, …-realm-ingress-1, … nginx.ingress.kubernetes.io/app-root, which ingress-nginx renders as if ($uri = /) { return 302 $scheme://$http_host<app-root>; } (nginx.tmpl:1000, controller-v1.14.5)
Gateway <release>-keycloak-realm-httproute, …-realm-httproute-1, … a rule matching Exact: / with a RequestRedirect filter, ReplaceFullPath to /realms/<realm>/account/ and statusCode: 302 written out; a second rule PathPrefix: / carries everything else to the Service 
keycloak:
  realmHosts:
    - host: realm-one.id.example.com
      realm: realm-one
      tlsSecretName: tls-id-wildcard-ingress   # a certificate that already exists
      clusterIssuer: ""                        # issued elsewhere - no annotation
      annotations:
        nginx.ingress.kubernetes.io/proxy-buffer-size: 16k
        nginx.ingress.kubernetes.io/proxy-buffers-number: "4"
    - host: id.other.example.com
      realm: realm-two
      tlsSecretName: tls-id-other-ingress
      clusterIssuer: letsencrypt-issuer        # issued FOR this object

One object per entry, never one object with several hostnames. spec.hostnames applies to the whole HTTPRoute and a rule's matches are path-based, so inside a shared route every entry would match Exact: / and the realm could not be told apart. The same holds for the Ingress side, where app-root is per object.

The realm does not follow from the hostname, and the chart does not try to derive it. Both are given per entry. In the installation this was written for, one of the four hostnames serves a realm whose name shares nothing with it.

Object names carry the list index, not the realm — a realm name is not a dependable object-name fragment (the 63-character label limit plus the release prefix, and a sanitised name can collide), the same reasoning as homeassistant's per-host routes. So reordering keycloak.realmHosts renames these objects and recreates them, with a brief gap on that hostname; the TLS secret is not the chart's and survives it. NOTES.txt prints which index carries which hostname and realm, because the object name does not say.

Why the TLS secret is required and the issuer is not

This is the deviation from the repo's Ingress convention — one secret named tls-<release>-<chart>-ingress, cert-manager annotation always set — and it is required rather than tolerated:

Field Convention Here, and why
tlsSecretName derived from release and chart Per entry, required, never derived. These hostnames live in zones whose certificates already exist, and several entries typically share one — a wildcard for a zone covers every realm host under it. A derived name per entry would ask cert-manager for a fresh certificate for a name a wildcard already covers: one ACME order per hostname, for nothing. An entry without one aborts the render instead of emitting a TLS block that silently serves the controller's default certificate — a browser-only failure.
clusterIssuer always set Optional per entry, and empty is the normal case. cert-manager keys a Certificate by the secret it writes, so of several objects naming one secret exactly one may carry the annotation. Set it only where the certificate is to be issued for this entry's own secret. Same deliberate omission as the qBittorrent object in freesailarr.
annotations chart-global Per entry. The set is not uniform: of the four hostnames this was written for, three carry the proxy-buffer pair and one does not, which one global block could not express.

What the render refuses, and what each refusal prevents

Abort The failure it pre-empts
an entry without host ingress-nginx reads an empty host as a catch-all for everything the controller serves that nothing else claims; an HTTPRoute with no hostnames attaches to every hostname of its listener. Both publish a realm's console under names nobody chose.
an entry without realm the target reads /realms//account/ — a 404 from Keycloak behind a hostname, a certificate and a route that are all healthy.
a realm containing / it is interpolated into a URL path, so a slash retargets the redirect instead of being escaped.
an entry without tlsSecretName, Ingress mode a TLS block with no secret: the controller's default certificate, and a failure visible only in a browser.
a host repeated, or equal to a hostname this release already publishes in Ingress mode the admission webhook rejects the second claim on a host-and-path pair — under Fleet that leaves the bundle in ErrApplied until forceSyncGeneration is raised. In Gateway mode the webhook never fires and both objects are accepted and merged, so which redirect a client gets is decided by object age.
a filled list with both modes off no object at all: no Ingress, no route, no DNS answer, and nothing in the rendered release that reveals the list was ignored.

The entry-shape checks sit in one helper (keycloak.realmHosts) so they exist in both modes — a check written in one routing template would not exist in the other. The Ingress template skips them while httpRoute.enabled is also true, so that "this entry needs a tlsSecretName" cannot answer a values file whose actual defect is that both modes are on; template execution order is not something a chart may rely on. Measured while writing it: without that condition, exactly that happened.

In Gateway mode these hostnames lose the proxy buffers too, and they feel it first

The per-entry annotations block is inert in Gateway mode, like ingress.annotations — see What Gateway mode costs this chart. It bites harder on the realm hostnames than on the chart's own two, for a reason specific to what they are for: they exist to put somebody in an account console, so a logged-in session with large Authorization and Set-Cookie headers is the normal traffic on these names rather than the edge case it is on a login host that mostly serves anonymous requests. A buffer sized at the default fails here first, and it fails as a 502 from the Gateway on a hostname whose Ingress never had a chart value to point at — nothing to grep for and nothing to un-set. NOTES.txt prints this in Gateway mode.

What the mode makes cheaper here: the entries' tlsSecretName values are not read either, and where several realm hostnames share a DNS zone one wildcard listener for that zone replaces one certificate per name.

Taking over hostnames another release serves today — the order, and the one unchecked question

This feature does not perform a migration and the chart does not know whether one is pending. Four such objects exist in the live keycloak namespace today — keycloak-realm-bekemeyer-klein-ingress, keycloak-realm-jk-family-ingress, keycloak-realm-mail-ingress, keycloak-realm-muth-event-ingress — owned by the Helm release jk-cluster-config-fleet-cd-keycloak-realm-ingress, i.e. by the jk-cluster-config repository, each pointing at this chart's Service under a hostname of its own. With realmHosts: [] nothing about them changes, which is precisely why adopting them is a separate step.

The order is not symmetric:

  1. Remove the objects in the owning repository first, and let that removal reach the cluster.
  2. Then install this chart with keycloak.realmHosts filled.

The window between the two is a short unreachability of those hostnames. The other order, or both at once, is worse: two Ingresses on one host and path are rejected by ingress-nginx's admission webhook, and under Fleet the bundle sits in ErrApplied until forceSyncGeneration is raised — measured, and written down in ytdlp-web-player's collision fail. In Gateway mode the duplicate is accepted instead of rejected, which is quieter and no better.

And the question nobody has answered yet: whether anything in jk-cluster-config depends on those four objects by name — a Fleet bundle, a monitor, an objectset reference. It is unchecked; only the keycloak namespace was read, not the second repository. Clear it before step 1.

That a host can change routing mode at all without an outage is measured — freesailarr.jk-dev.de switched on 2026-10-13 with no downtime — but that case was a single helm upgrade within one release, which removed the Ingress and created the route so that both never existed at once. Here the objects live in two repositories and two releases, and that is where the window comes from: the precedent covers the mechanics, not the coordination.

The admin hostname is not an access control

hostname-admin decides which URLs the server generates — redirects, the CSS and JS the console loads, and the API address the console itself calls. It closes nothing. Upstream, verbatim:

Using the hostname-admin option does not prevent accessing the Administration REST API endpoints via the frontend URL specified by the hostname option. If you want to restrict access to the Administration REST API, you need to do it on the reverse proxy level. Administration Console implicitly accesses the API using the URL as specified by the hostname-admin option.

So /admin/realms/... stays reachable through ingress.domain, with or without the second hostname, and an IP allow-list on the admin Ingress buys nothing on its own. Whoever wants the administration API restricted does it on the reverse proxy in front of the public host.

The chart renders no deny rule for /admin, and that is a decision. Two reasons, the second of which settled it: the console loads assets and performs redirects across more paths than /admin, so a rule written from the path name breaks it in ways that only show up in a browser; and a rule nobody has tested against a running console is a measure that merely looks like one, which is worse than an absent one.

What the second hostname is good for: the console's URLs are generated for it, so it can live on an internal ingress class or in a different DNS zone, and operators reach it by a name that is not the public login name.

A shared database merges two clusters

Node discovery runs through the database, so — upstream — "clusters that share the same database but should remain separate will merge unless they have distinct cluster names. The default cluster name is ISPN." Two releases of this chart pointed at the same database therefore form one Infinispan cluster, silently, with no error on either side.

If that is your situation, the first question is whether they should share a database at all — separate databases need no cluster name.

keycloak.cache.clusterName exists for the case where they must, and it is not set per release by default even though that is the obvious fix: the same upstream page warns about the other direction — "As invalidation of caches between clusters with distinct names will not work, this will lead to stale caches and incorrect behavior. Use only together with the stateless feature." That warning is not unambiguous about which configuration it applies to, and a chart default that trades a documented merge for a possibly undocumented invalidation gap is not an improvement. So: the value is here, the trap is written down, and the default is upstream's.

The management port is unauthenticated

Port 9000 carries /health, /health/live, /health/ready, /health/started and — with keycloak.metrics.enabled — /metrics, and it carries them without authentication. It has a ClusterIP Service of its own (<release>-keycloak-management-service) for in-cluster scraping, and it is in no Ingress.

It must never get one, and no value should be added that would allow it. An unauthenticated readiness endpoint on the internet tells an attacker when the database is down; /metrics tells them the realm and client names. The Service is separate from the serving Service precisely so that naming it is a deliberate act — on one combined Service, publishing it would be one wrong port name in an Ingress backend away.

Upstream's own alternative, legacy-observability-interface=true, moves health back onto the serving port where the Ingress already is. It is explicitly "not recommended for security reasons" and this chart does not offer it.

Readiness only checks the database while metrics are enabled

keycloak.metrics.enabled reads like an observability switch and is not only one. Upstream's table of available health checks carries a "Requires Metrics" column, and the row that says yes is the one that matters here:

Check Description Requires metrics
Database "Returns the status of the database connection pool" yes
Cluster "Returns the status of the cluster (network partitions)" no
Graceful shutdown returns DOWN once pre-shutdown has started no
Keycloak Initialized status of server initialization no

So with metrics-enabled=false, /health/ready answers 200 with the database gone, and the pod keeps taking logins it cannot serve.

That is exactly the shape the readiness probe that could not fail cost this repo on urlaubsverwaltung: the probe path was right and the aggregate behind it contained nothing external. The difference is only where it hides — there in a Spring health group, here behind an option named for metrics. Which is why this chart defaults keycloak.metrics.enabled to true, and argues for it as a readiness setting rather than a scraping one.

The price is real and is why the value still exists: /metrics then answers on the management port, unauthenticated, carrying realm names, client ids and login counters. That port is ClusterIP-only and in no Ingress, so the exposure is in-cluster — see the trap below. Switching it off is a decision about whether the readiness probe can fail at all, and nothing else in this chart replaces it.

The three probes use three different paths on purpose

Probe Path Contains a database check?
liveness /health/live no
readiness /health/ready yes
startup /health/started —

This is §12.3 in the one shape where upstream has already done the work: pointing liveness at /health/ready — or at the aggregate /health, which also includes the database — restarts every pod of the cluster for as long as the database is away. A restart cannot fix a database, so that turns a degraded login service into none at all.

The wrong path would look entirely plausible in a diff, which is why the reason sits on the helper and not only here. And the other half of the rule holds too: readiness must hang on the database, because a node that cannot reach it answers every login with an error.

A PDB with one replica blocks every drain

minAvailable: 1 with exactly one replica forbids every voluntary eviction, so kubectl drain blocks forever on a pod nothing is allowed to move — and the message names the PodDisruptionBudget rather than the replica count, which sends the reader to the wrong file.

The chart therefore renders the PDB only while the computed replica count is >= 2, which also covers scaleDown: true, where there are no pods and the object would describe a promise about nothing. NOTES.txt prints which of the two happened, so it does not have to be inferred from kubectl get pdb.

A volume on /opt/keycloak/themes hides the bundled themes

/opt/keycloak/themes is not empty in the image — base, keycloak and keycloak.v2 live there. A volume mounted at that path hides all three, and the login page is then broken rather than merely unstyled: the pod stays Ready, because /health/* knows nothing about themes.

Mount on a subpath that does not exist in the image, /opt/keycloak/themes/<name>, which hides nothing.

This is the same mechanism as the other path this chart deliberately leaves alone: /opt/keycloak/conf is never mounted, because it carries keycloak.conf and — more importantly — the cache-ispn.xml that cache=ispn reads. An emptyDir there would bring the cluster up with no cache configuration. Same class of mistake, and worth learning once: before mounting into an upstream image, check whether the directory is empty in it.

There is no ConfigMap, so there is no checksum/config

That is not an omission. Keycloak is configured entirely through options and KC_* variables, so no file is rendered, mounted or hashed, and the §9 rule has nothing to attach to. values.yaml states it at the top, because an absent annotation looks identical to a forgotten one.

What is covered is the op:// path of the two vault items, through checksum/secret-ref.

A change inside a vault item restarts nothing

Helm never sees it. After rotating a field:

kubectl -n keycloak rollout restart deployment/keycloak-keycloak-deployment

Repointing a secrets.*SecretRef at a different item does roll the pods — that is what checksum/secret-ref covers.

The 1Password operator's auto-restart annotation is not set, and here that is a decision rather than a constraint: unlike the StatefulSet charts in this repo it would actually work, because the operator enumerates Deployments. But its trigger is the item's version counter and not its content, so a re-save with identical values rolls the whole cluster. For an identity provider, an unannounced rolling restart because somebody opened a vault entry is worse than a deliberate one. §9 has the precedence trap if it is ever set.

jdbc-ping needs pod-to-pod reachability, and no NetworkPolicy knows that yet

Discovery runs through the database, but once the peers are known JGroups opens direct connections between pod IPs on port 7800. No Service mediates that, headless or otherwise — the replicas simply need L3 reachability to each other.

This chart renders no NetworkPolicy. If one is ever added, it has to allow 7800 between the replicas, or the cluster quietly becomes two islands with two green pods.

Two things to check after the first upgrade, because nothing else will

1. Log in. This release is the cluster's identity provider, and the http rule is the only source-restricted inbound rule in the repository. If the ingressController source is wrong for this cluster, every login everywhere stops — and no part of CI can see it: no ingress controller runs in the install-test, and the probes use the management port, so the pod becomes Ready either way. The default was measured against the live cluster (DaemonSet rke2-traefik in kube-system, which serves the Gateway API here too), but measured labels are not proven traffic.

2. Check your consumers' issuer URLs. An application that reaches Keycloak on its in-cluster Service address rather than its public URL is a pod in another namespace, and the http rule blocks it. Those namespaces go into networkPolicy.allowedNamespaces. Consumers using the public issuer URL arrive through Traefik and are already covered.

Measured on 2026-10-14, and the news is good: every readable in-cluster consumer uses the public name.

Consumer Value
bulwark OAUTH_ISSUER_URL = https://id.mail.bekemeyer-klein.de/realms/mail
homarr AUTH_OIDC_ISSUER = https://bk.id.bekemeyer-klein.de/realms/bekemeyer-klein
paperless-{jk,cmk,ngx} through their PAPERLESS_SOCIALACCOUNT_PROVIDERS block

Their traffic therefore arrives from Traefik and is permitted by the rule as it ships. A search across pod environments and ConfigMaps for Keycloak's in-cluster Service name (keycloak-keycloak-service, *.keycloak.svc) returned only Keycloak's own pods, reaching their database at keycloak-pg-rw.jlab-pg.svc.cluster.local:5432 — which incidentally confirms that the egress rule derived from keycloak.db.port is right for the live release.

The residual risk, stated rather than dismissed: anything that addresses Keycloak by its Service name would be cut off, and among the readable consumers there is none. A consumer whose issuer URL lives somewhere unreadable (a secret, a realm import) is not covered by this check — which is why the login test above stays the acceptance criterion. What changed is that there is now a measured reason to expect it to pass, instead of a hope.

The management port is the probed one, which is unfortunate here specifically

9000 carries the health and readiness endpoints and the metrics, unauthenticated, and before this object existed the only thing protecting it was that no Ingress pointed at it. Because all three probes address it, it cannot be restricted by source without the worker node's IP — so the policy improves it from anything that can route to the pod to anything inside the cluster, and networkPolicy.probeSources is what closes the rest.

The cluster bus needs a rule in both directions

policyTypes lists Egress, so a replica dialling its peer is subject to this policy too. The chart therefore renders jgroups inbound and outbound — without the second one the bus would be permitted one way only, and the result is a cluster that never forms while both nodes look healthy. Infinispan discovery here is jdbc-ping, so a blocked database costs the cluster as well as the data.

Mail is not allowed

Keycloak's SMTP server is configured per realm, which this chart never sees, so verification and password-reset mail needs its port (587 or 465) in networkPolicy.egress.extraPorts. The symptom otherwise is mail that is never delivered, with a healthy pod behind it.

The policy is only a boundary where the CNI enforces one

The API server accepts a NetworkPolicy whatever the cluster does with it. Where the CNI does not enforce policies, everything looks configured and nothing is filtered — which is worse than no policy, because it invites the assumption that something is being filtered. Verify by connecting from an unrelated namespace rather than by reading it off the manifest.

Measured for this cluster: the CNI is Canal (Flannel for the network, Calico for policy), 6/6, and it does enforce. Measured for CI: k3s brings its own policy controller and enforces too — but it exempts the kubelet's probe traffic, so a blocked probe path does not turn the install-test red. What CI does catch is anything the application itself needs.

Upgrades and backup

  • Patch updates within a minor line are the normal case and arrive as Renovate app-version PRs. Run the compatibility check anyway — see the unresolved doc contradiction in Minor upgrades.
  • Minor and major jumps need the procedure in that section. The automerge does not know about it.
  • The backup is the database. There is no PVC and nothing on disk to snapshot: realms, clients, users, keys and sessions are all in PostgreSQL. A Keycloak backup is a database backup plus the values file, and restoring means restoring both — the realm keys live in the database, so a database restored without them signs nothing a client will accept.

CI

ci/keycloak/ carries test-values.yaml, fixtures.yaml (throwaway PostgreSQL, the two Secrets the operator would materialize, and a minimal login-theme ConfigMap) and settings.env (INSTALL_TIMEOUT=900).

The install-test runs with two replicas, and the rollout is complete only when both pods are Ready — so a second replica that cannot start, deadlocks on the schema migration lock or fights the first over the database fails the run. Measured in Gitea PR #87: 71.8 seconds from helm install to both pods Ready, 0 restarts — three runs, all three the same.

Read that as an observation and not as an invariant, and the number that matters here is not the time but the restart count. Five runs in PR #92 — identical pod spec — measured 101s/1, 82s/0, 191s/0, 101s/1, 111s/1: three of five restarted one pod once. That looked like the new theme mount destabilizing the start, so the control case was measured on the same runner, with the mount commented out of the fixture (pod template hash 5844db56b6, byte-identical to what PR #87 measured): 81s/0 and 171s/1 — one of two.

So the restart happens without the mount as well: it is pre-existing, intermittent behaviour of this chart's first install, always exactly one restart on one pod, and it does not track the runner's speed (the fastest and the slowest run of the whole series restarted nothing). What the restart is could not be established, because the harness dumps container logs only when a run fails — a green run cannot explain a restart it reports, and answering it means making the final-state step print the restart reasons whenever a counter is above zero. A first install that serializes two replicas on the Liquibase migration lock is a plausible place for one, but that is a hypothesis and nothing has measured it.

The method is the transferable part: a counter that moves in the same round as a change is not evidence against the change until the control case has run on the same machine. Three clean runs two days earlier were not that control.

What that does not prove is that jdbc-ping found the peer, and reading it that way is the mistake this paragraph exists to prevent. Readiness reports this node's health, so a node that never found its peer is Ready, healthy and alone — and two such nodes are two islands that each believe they are the cluster. The Infinispan view has to be read out of the log, which is what step 2 of the first-install checklist is for. So the test rules out the louder half and leaves the quiet half to a human.

It also closes three things a render cannot: that the two init containers work at all, that readOnlyRootFilesystem: true holds for the serving container with emptyDirs only at the three paths it needs, and that start --optimized accepts the pre-built installation instead of re-augmenting onto a read-only path. And because health-enabled and metrics-enabled are build options, the fact that all three probe paths answer on port 9000 is the proof that the build container really did its work — a build that silently did nothing would leave them 404.

The install-test mounts a theme (Gitea #88), on the subpath /opt/keycloak/themes/ci-theme, so the extra-volume path is walked rather than only rendered. A green run closes three things: a mount on a subpath of a directory that does exist in the image is accepted alongside readOnlyRootFilesystem: true (the runtime has to create the mount point in a filesystem that is about to be read-only); kc.sh build still succeeds with the extra mount present, which is the half that matters because the mounts are rendered on the build container too; and start --optimized comes up with a runtime theme in place — the plan of #88 called that "a reasoned expectation, not verified", and this is the verification.

What the theme fixture does not prove, and it would be easy to overclaim: that the login page renders and that base, keycloak and keycloak.v2 are still listed. The health endpoints answer regardless of the themes directory, and nothing in this harness makes an HTTP request to / or runs a command in the pod. That the subpath hides nothing is a property of the mount path, not a measurement — and kcLogoIdP-<alias> needs a browser plus a realm setting, neither of which exists in CI.

What CI cannot prove: that hostname-admin serves the console and its resources cleanly on the second hostname. No ingress controller answers in the test cluster and nothing drives a browser. What it does check is that both Ingress objects are accepted, that the Service port they name exists, and that Keycloak accepts both hostname values as full URLs at start.

The Gateway mode is held at render level, by ci/httproute/keycloak.yaml plus ci/run-httproute-match-test.sh — which runs over every chart on every PR (CI/CD). The overlay switches this chart into Gateway mode and covers both routing objects: the public route with its hostname from ingress.domain (the httpRoute.hostnames-empty fallback, which is the path an installation actually takes) and the admin route with its parents falling back to httpRoute.parentRefs. 4 assertions — a parentRefs check and a backendRefs check per route — and both references are held against the serving Service of the same render by port number, because backendRefs takes a PortNumber and cannot name the port http the way the Ingress does.

An overlay rather than a change to the install-test fixture, and that is a deliberate trade: the install-test runs exactly one configuration per chart, and this chart's fixture is the expensive and informative one (two replicas, two init containers, kc.sh build under a read-only root, the Infinispan cluster forming over jdbc-ping). Trading all of it for a CRD schema check that cyberchef's fixture already performs against the real Gateway-API CRDs would be a bad swap.

What that does not cover, named so it is not mistaken for coverage: that either hostname answers — no Gateway controller runs in CI, so no route ever reaches Accepted: True — and the proxy-buffer annotations, which have no counterpart in this mode at all. That loss is documented in values.yaml and printed by NOTES.txt, and documentation is not something a test can assert. Nothing in CI reads NOTES.txt either: helm template does not emit it and no script under ci/ reads it, so its text carries review-only assurance.

See also

  • Chart style guide — §11 for the PodDisruptionBudget convention this chart introduced
  • Renovate — why this chart needs no version constraint, and the automerge decision
  • Lessons learned — The upgrade procedure the automerge did not know about, The application that rewrites its own installation at start and The mount that would have arrived after the build it was for, plus The precise message that hid the structural one from the realm-hostname round
  • CI/CD

helm-charts wiki

How we work

How it is built

How charts are written

Charts