Implements the plan on #83 together with the two findings and decisions recorded in its comments.
Shape
Deployment with two replicas, cache=ispn, clustered through Keycloak's default jdbc-ping stack — discovery runs over the PostgreSQL that is there anyway, so no headless Service, no JGroups wiring, no extra option. No PVC: Keycloak 26 keeps everything in the database, user sessions included (volatile sessions are deprecated as of 26.8).
Two Services (8080 and the management port 9000, the latter in no Ingress), two Ingresses, two OnePasswordItems, a PodDisruptionBudget, probes on all three native endpoints, computed limits through the limitFactor helper, replicas plus scaleDown, and a strict values.schema.json.
1. hostname-admin is not an access control. It is set so the admin URLs are right, and it is not presented as a boundary. Upstream, in values.yaml, NOTES.txt and on the wiki page in the same words: it "does not prevent accessing the Administration REST API endpoints via the frontend URL specified by the hostname option. If you want to restrict access to the Administration REST API, you need to do it on the reverse proxy level." No /admin deny rule is rendered.
2. Two init containers, because one would not work.db, health-enabled and metrics-enabled are build options (verified: each carries the tool icon in the 26.8.0 option table; cache does not). The published image is augmented with their defaults, and a runtime change makes Keycloak rewrite /opt/keycloak/lib/quarkus at start — which readOnlyRootFilesystem: true forbids, and which upstream calls a start that "creates an image that is mutable". Not setting db is no escape: the probes alone force it, because health-enabled is a build option.
An empty emptyDir over that path fails too, and that is the part that is not guessable: QuarkusEntryPoint.class carries the literal quarkus/quarkus-application.dat, and kc.sh runs that class for every command, build included — so with the file hidden, kc.sh build cannot start either. Hence seed → build → start --optimized:
container
mounts the volume at
does
root fs
…-seed (init)
/seed
cp -R /opt/keycloak/lib/quarkus/. /quarkus-build/
read-only
…-build (init)
/opt/keycloak/lib/quarkus
kc.sh build
read-only
…-deployment
/opt/keycloak/lib/quarkus
start --optimized
read-only
So no hardening deviation anywhere in this pod — which is why it is two init containers and not one with a writable root filesystem, and why no Trivy suppression was needed that would also have silenced the serving container.
The condition named in the decision — build options from one source — is enforced structurally, not by care: they exist only in the keycloak.buildEnv helper, the serving container passes start --optimized and no build option, and a build option's variable name in keycloak.envfails the render naming the variable.
Renovate: no constraint, and no automerge exception
The constraint garage and stalwart need is not needed here, and that is measured rather than assumed (Renovate 44.132.5, --platform=local --dry-run=lookup, real registry, against the committed config):
appVersion
updates offered
26.8.0
none; warnings: []
26.7.4
exactly two: 26.7.5 (patch), 26.8.0 (minor)
No run offered a -N rebuild tag (26.8.0-0 exists and points at the same digest as 26.8.0), a floating minor, latest or nightly. Docker versioning splits at the first dash and only offers candidates with the same suffix; ours has none. renovate-config-validator passes on the whole file.
Per the decision on #83 there is no packageRules automerge exception. The consequence is documented instead of prevented, prominently in NOTES.txt and on the wiki page: the minor-upgrade procedure (scaleDown: true, back up, bump, scale up), the statement that the automerge does not know about it, and the kc.sh update-compatibility metadata|check calls with their exit codes.
Also in this PR
First PodDisruptionBudget in the repo, so it is now a convention — README §11 and the wiki style guide. It is rendered only while the computed replica count is >= 2: minAvailable: 1 with one replica blocks kubectl drain forever with a message that names the PDB rather than the replica count.
First quay.io image, so quay.io joins ci/trivy/data/ksv0125.yaml — configured rather than suppressed, so AVD-KSV-0125 keeps its meaning.
Two deviations from upstream defaults, each argued at the value: db-pool-max-size20 instead of 100 (two replicas at 100 ask a default PostgreSQL for twice its max_connections), and uid 1000 / gid 0 instead of this repo's 20000/1000, because the image declares keycloak:x:1000:0 and chowns its install tree to that pair.
publish-keycloak.yml, keycloak in the change-app-version.yml choice list, fixtures under ci/keycloak/.
runAsGroup: 1000, not the image's own 0 — checked rather than suppressed after Trivy reported AVD-KSV-0116 on the first run. Counted across the image layer: 521 of 522 entries are owned by 1000:0, and every mode present grants the owner full access, so uid 1000 needs no group membership to read or execute anything. Group 0 is upstream's OpenShift arbitrary-uid pattern and buys nothing once the owning uid is pinned. No trivyignore entry was added, which matters more than one LOW finding: that file scopes by chart, so an exception would also have silenced the check for the serving container.
Two corrections found after the first CI runs
The first green run was not the end of the verification, and reading on changed a default. Keycloak's database health check is the one row of upstream's "Available Checks" table whose "Requires Metrics" column says yes. With metrics-enabled=false — this chart's default in the first commit — /health/ready answers 200 while the database is gone, and the pod keeps taking logins it cannot serve. The probe path was right and the aggregate behind it contained nothing external: the urlaubsverwaltung defect (#218) in a new hiding place, gated behind an option named for observability, which is nowhere near where a probe gets reviewed.
So keycloak.metrics.enabled now defaults to true and its comment argues about readiness rather than scraping. The price is named at the value: /metrics then answers on the management port, unauthenticated — a port that is ClusterIP-only and in no Ingress.
And an overclaim of mine, removed in the same commit.ci/keycloak/test-values.yaml said a green run proves jdbc-ping found the second replica. It does not: readiness reports one node's own health, so a node that never found its peer is Ready, healthy and alone — and two such nodes are two islands that each believe they are the cluster. What the test does rule out is the louder half: a replica that cannot start, deadlocks on the migration lock or fights the first over the database. The Infinispan view has to be read out of the log, which is what step 2 of NOTES.txt now says no probe and no install-test replaces.
Verification
Local (no container runtime in this workspace — no k3d, no Docker, no Trivy):
helm lint --strict keycloak — passes, 0 failed.
helm template against chart defaults and against ci/keycloak/test-values.yaml; helm package — ok.
ci/run-quantity-render-test.sh — all 27 charts, failed: 0; keycloak's two resources blocks (keycloak.resources, keycloak.build.resources) both exercised.
32 assertions on the fail paths and the null-safety paths: every build-option name in keycloak.env refused (upper- and lowercase); each of 12 chart-owned blocks erased to null still renders hardening, the right probe paths and the computed limits; scaleDown and replicas: 1 render no PDB while replicas: 3 does; every required() fires, and ingress.domain stays required with ingress.enabled: false while clusterIssuer is freed when both Ingresses are off; the schema rejects an unknown key, dev-file, a feature name with a space and an out-of-range port.
NOTES.txt rendered in three variants through the define/ConfigMap trick, since helm template does not render it.
The install-test runs with two replicas and a throwaway PostgreSQL. The rollout completes only when both pods are Ready, so a replica that cannot start or cannot share the database fails the run.
Measured (run 487, job "Install-test keycloak", Gitea docker runner, k3s v1.35.5, image pulled during the run): 71.8 s from helm install to both pods Ready — the first at ~70.8 s, the second one second later — 1/1 Running, 0 restarts, ending in OK: chart 'keycloak' installed and all workloads are Ready. That figure is now written at the startupProbe value as the justification for its 600 s budget, together with why the headroom stays: everything in that run was the cheap case, and a first install against a real database serializes the second replica behind the first's Liquibase lock instead of letting it arrive a second later.
Because health-enabled and metrics-enabled are build options, the fact that all three probe paths answer on port 9000 is the proof that the build init container did its work — a build that silently did nothing would leave them 404.
CI
Three runs on this branch, all green, each on jlab-git-actions-docker-runner-0 (k3s v1.35.5, kubectl v1.35.9):
The 10 "excepted" in the last column are this repo's existing repo-wide exceptions (no CPU limits, the namespace artefact, and so on) — this chart contributes no finding and no new suppression.
What stays open
Two of the plan's five measurement points, and both need a real deployment — which is why this PR says Part of and not Fixes:
4. that jdbc-ping really found the second replica. Two Ready replicas are not that proof, for the reason above. The Infinispan view is read out of the log (NOTES.txt step 2).
5. that hostname-admin serves the console and its resources cleanly on the second hostname. No ingress controller answers in the test cluster and nothing drives a browser. CI does prove that both Ingress objects are accepted, that the Service port they name exists, and that Keycloak accepts both hostname values as full URLs at start.
Closed by this PR: 1 (startup time, measured above), 2 (readOnlyRootFilesystem: true holds, with emptyDirs only at the three paths) and 3 (there is no _FILE form, so secretKeyRef it is).
Not verified locally at all, because this workspace has no container runtime and no Trivy: anything needing a running pod, and the trivy config scan — which does run in CI, reporting-only.
Implements the plan on #83 together with the two findings and decisions recorded in its comments.
## Shape
Deployment with **two replicas**, `cache=ispn`, clustered through Keycloak's default `jdbc-ping` stack — discovery runs over the PostgreSQL that is there anyway, so no headless Service, no JGroups wiring, no extra option. No PVC: Keycloak 26 keeps everything in the database, user sessions included (volatile sessions are deprecated as of 26.8).
Two Services (8080 and the management port 9000, the latter in **no** Ingress), two Ingresses, two `OnePasswordItem`s, a PodDisruptionBudget, probes on all three native endpoints, computed limits through the `limitFactor` helper, `replicas` plus `scaleDown`, and a strict `values.schema.json`.
## The two findings from #83, as built
**1. `hostname-admin` is not an access control.** It is set so the admin URLs are right, and it is *not* presented as a boundary. Upstream, in `values.yaml`, `NOTES.txt` and on the wiki page in the same words: it "does not prevent accessing the Administration REST API endpoints via the frontend URL specified by the `hostname` option. If you want to restrict access to the Administration REST API, you need to do it on the reverse proxy level." No `/admin` deny rule is rendered.
**2. Two init containers, because one would not work.** `db`, `health-enabled` and `metrics-enabled` are **build** options (verified: each carries the tool icon in the 26.8.0 option table; `cache` does not). The published image is augmented with their defaults, and a runtime change makes Keycloak rewrite `/opt/keycloak/lib/quarkus` at start — which `readOnlyRootFilesystem: true` forbids, and which upstream calls a start that "creates an image that is mutable". Not setting `db` is no escape: **the probes alone force it**, because `health-enabled` is a build option.
An empty `emptyDir` over that path fails too, and that is the part that is not guessable: `QuarkusEntryPoint.class` carries the literal `quarkus/quarkus-application.dat`, and `kc.sh` runs that class for **every** command, `build` included — so with the file hidden, `kc.sh build` cannot start either. Hence **seed → build → `start --optimized`**:
| container | mounts the volume at | does | root fs |
|---|---|---|---|
| `…-seed` (init) | `/seed` | `cp -R /opt/keycloak/lib/quarkus/. /quarkus-build/` | read-only |
| `…-build` (init) | `/opt/keycloak/lib/quarkus` | `kc.sh build` | read-only |
| `…-deployment` | `/opt/keycloak/lib/quarkus` | `start --optimized` | read-only |
So **no hardening deviation anywhere in this pod** — which is why it is two init containers and not one with a writable root filesystem, and why no Trivy suppression was needed that would also have silenced the serving container.
The condition named in the decision — build options from one source — is enforced structurally, not by care: they exist only in the `keycloak.buildEnv` helper, the serving container passes `start --optimized` and **no** build option, and a build option's variable name in `keycloak.env` **fails the render** naming the variable.
## Renovate: no constraint, and no automerge exception
The constraint `garage` and `stalwart` need is **not** needed here, and that is measured rather than assumed (Renovate 44.132.5, `--platform=local --dry-run=lookup`, real registry, against the **committed** config):
| `appVersion` | updates offered |
|---|---|
| `26.8.0` | none; `warnings: []` |
| `26.7.4` | exactly two: `26.7.5` (patch), `26.8.0` (minor) |
No run offered a `-N` rebuild tag (`26.8.0-0` exists and points at the same digest as `26.8.0`), a floating minor, `latest` or `nightly`. Docker versioning splits at the first dash and only offers candidates with the same suffix; ours has none. `renovate-config-validator` passes on the whole file.
Per the decision on #83 there is **no `packageRules` automerge exception**. The consequence is documented instead of prevented, prominently in `NOTES.txt` and on the wiki page: the minor-upgrade procedure (`scaleDown: true`, back up, bump, scale up), the statement that the automerge does not know about it, and the `kc.sh update-compatibility metadata|check` calls with their exit codes.
## Also in this PR
- **First `PodDisruptionBudget` in the repo, so it is now a convention** — README §11 and the wiki style guide. It is rendered only while the *computed* replica count is `>= 2`: `minAvailable: 1` with one replica blocks `kubectl drain` forever with a message that names the PDB rather than the replica count.
- **First `quay.io` image**, so `quay.io` joins `ci/trivy/data/ksv0125.yaml` — configured rather than suppressed, so AVD-KSV-0125 keeps its meaning.
- Two deviations from upstream defaults, each argued at the value: `db-pool-max-size` **20** instead of 100 (two replicas at 100 ask a default PostgreSQL for twice its `max_connections`), and uid **1000 / gid 0** instead of this repo's 20000/1000, because the image declares `keycloak:x:1000:0` and chowns its install tree to that pair.
- `publish-keycloak.yml`, `keycloak` in the `change-app-version.yml` choice list, fixtures under `ci/keycloak/`.
- **`runAsGroup: 1000`, not the image's own `0`** — checked rather than suppressed after Trivy reported `AVD-KSV-0116` on the first run. Counted across the image layer: 521 of 522 entries are owned by `1000:0`, and every mode present grants the **owner** full access, so uid 1000 needs no group membership to read or execute anything. Group 0 is upstream's OpenShift arbitrary-uid pattern and buys nothing once the owning uid is pinned. No `trivyignore` entry was added, which matters more than one LOW finding: that file scopes by *chart*, so an exception would also have silenced the check for the serving container.
## Two corrections found after the first CI runs
The first green run was not the end of the verification, and reading on changed a default. Keycloak's **database** health check is the one row of upstream's "Available Checks" table whose **"Requires Metrics"** column says *yes*. With `metrics-enabled=false` — this chart's default in the first commit — `/health/ready` answers **200 while the database is gone**, and the pod keeps taking logins it cannot serve. The probe path was right and the aggregate behind it contained nothing external: the `urlaubsverwaltung` defect (#218) in a new hiding place, gated behind an option named for *observability*, which is nowhere near where a probe gets reviewed.
So `keycloak.metrics.enabled` now defaults to **`true`** and its comment argues about readiness rather than scraping. The price is named at the value: `/metrics` then answers on the management port, unauthenticated — a port that is ClusterIP-only and in no Ingress.
**And an overclaim of mine, removed in the same commit.** `ci/keycloak/test-values.yaml` said a green run proves `jdbc-ping` found the second replica. It does not: readiness reports *one node's own* health, so a node that never found its peer is Ready, healthy and alone — and two such nodes are two islands that each believe they are the cluster. What the test *does* rule out is the louder half: a replica that cannot start, deadlocks on the migration lock or fights the first over the database. The Infinispan view has to be read out of the log, which is what step 2 of `NOTES.txt` now says no probe and no install-test replaces.
## Verification
Local (no container runtime in this workspace — no k3d, no Docker, no Trivy):
- `helm lint --strict keycloak` — passes, 0 failed.
- `helm template` against chart defaults **and** against `ci/keycloak/test-values.yaml`; `helm package` — ok.
- `ci/run-quantity-render-test.sh` — all 27 charts, `failed: 0`; keycloak's **two** resources blocks (`keycloak.resources`, `keycloak.build.resources`) both exercised.
- **32 assertions** on the fail paths and the null-safety paths: every build-option name in `keycloak.env` refused (upper- and lowercase); each of 12 chart-owned blocks erased to `null` still renders hardening, the right probe paths and the computed limits; `scaleDown` and `replicas: 1` render no PDB while `replicas: 3` does; every `required()` fires, and `ingress.domain` stays required with `ingress.enabled: false` while `clusterIssuer` is freed when both Ingresses are off; the schema rejects an unknown key, `dev-file`, a feature name with a space and an out-of-range port.
- `NOTES.txt` rendered in three variants through the `define`/ConfigMap trick, since `helm template` does not render it.
**The install-test runs with two replicas and a throwaway PostgreSQL.** The rollout completes only when *both* pods are Ready, so a replica that cannot start or cannot share the database fails the run.
**Measured** (run 487, job "Install-test keycloak", Gitea docker runner, k3s v1.35.5, image pulled during the run): **71.8 s** from `helm install` to both pods Ready — the first at ~70.8 s, the second one second later — `1/1 Running`, **0 restarts**, ending in `OK: chart 'keycloak' installed and all workloads are Ready`. That figure is now written at the `startupProbe` value as the justification for its 600 s budget, together with why the headroom stays: everything in that run was the cheap case, and a first install against a real database serializes the second replica behind the first's Liquibase lock instead of letting it arrive a second later.
Because `health-enabled` and `metrics-enabled` are *build* options, the fact that all three probe paths answer on port 9000 **is** the proof that the build init container did its work — a build that silently did nothing would leave them 404.
## CI
Three runs on this branch, all **green**, each on `jlab-git-actions-docker-runner-0` (k3s v1.35.5, kubectl v1.35.9):
| run | head | install-test | Trivy |
|---|---|---|---|
| [487](https://git.jk-dev.de/jlab-cluster/helm-charts/actions/runs/487) | `f784f40` | both pods Ready in **71.8 s**, 0 restarts | 95/96, one `AVD-KSV-0116` |
| [489](https://git.jk-dev.de/jlab-cluster/helm-charts/actions/runs/489) | `63d0c7f` | both pods Ready in **71.5 s**, 0 restarts | 95/96, same finding |
| [490](https://git.jk-dev.de/jlab-cluster/helm-charts/actions/runs/490) | `1c14c60` | both pods Ready in **~72 s**, 0 restarts | **0 misconfigurations** (`0 to act on, 10 excepted`) |
The 10 "excepted" in the last column are this repo's existing repo-wide exceptions (no CPU limits, the namespace artefact, and so on) — **this chart contributes no finding and no new suppression.**
## What stays open
Two of the plan's five measurement points, and both need a real deployment — which is why this PR says `Part of` and not `Fixes`:
- **4. that `jdbc-ping` really found the second replica.** Two Ready replicas are not that proof, for the reason above. The Infinispan view is read out of the log (`NOTES.txt` step 2).
- **5. that `hostname-admin` serves the console and its resources cleanly on the second hostname.** No ingress controller answers in the test cluster and nothing drives a browser. CI does prove that both Ingress objects are accepted, that the Service port they name exists, and that Keycloak accepts both hostname values as full URLs at start.
Closed by this PR: **1** (startup time, measured above), **2** (`readOnlyRootFilesystem: true` holds, with emptyDirs only at the three paths) and **3** (there is no `_FILE` form, so `secretKeyRef` it is).
Not verified locally at all, because this workspace has no container runtime and no Trivy: anything needing a running pod, and the `trivy config` scan — which does run in CI, reporting-only.
Part of #83
Two replicas with cache=ispn, clustered through Keycloak's default jdbc-ping
stack: discovery runs over the PostgreSQL that is there anyway, so no headless
Service and no JGroups wiring. Deployment rather than StatefulSet - Keycloak 26
keeps everything in the database, user sessions included.
The chart's defining property is that it has TWO INIT CONTAINERS and needs
both. db, health-enabled and metrics-enabled are Keycloak BUILD options
(verified in the 26.8.0 option table), the published image is augmented with
their defaults, and supplying different ones at runtime makes Keycloak rewrite
/opt/keycloak/lib/quarkus at start - which readOnlyRootFilesystem: true forbids
and which upstream describes as a start that "creates an image that is
mutable". Not setting db is no way out: the probes alone force it, because
health-enabled is a build option too. An empty emptyDir over that path fails as
well, because QuarkusEntryPoint loads quarkus/quarkus-application.dat before it
dispatches any command, `build` included. So one init container seeds the
directory into a volume, the next runs `kc.sh build` into it, and the serving
container runs `start --optimized` - every container keeping the full hardening
baseline, with no documented deviation to carry.
Build options therefore exist in exactly one place, and the asymmetry is
enforced rather than trusted: the serving container passes no build option at
all, and a build option's variable name in keycloak.env fails the render with a
message naming it.
Hostname handling is the security-relevant part. KC_HOSTNAME is derived from
ingress.domain as a full URL (which upstream requires alongside hostname-admin),
hostname-strict stays at its default true, and proxy-headers is set rather than
optional. The admin hostname is NOT presented as an access control: upstream
states that hostname-admin "does not prevent accessing the Administration REST
API endpoints via the frontend URL specified by the hostname option", so the
chart renders no /admin deny rule and values.yaml, NOTES.txt and the wiki all
say in the same words that restricting that API is reverse-proxy work.
The management port 9000 is a containerPort and a ClusterIP Service for
in-cluster scraping, and is in no Ingress: it serves /health/* and optionally
/metrics without authentication. Liveness uses /health/live and deliberately
not /health or /health/ready - those include the database, and a liveness probe
on them would restart the whole cluster for as long as the database is away.
Also in this change:
- First PodDisruptionBudget in this repository, which makes it a convention
(README section 11): minAvailable as a value, and rendered only while the
computed replica count is >= 2, because minAvailable: 1 with one replica
blocks every node drain forever with a message that names the PDB rather
than the replica count.
- First quay.io image, so quay.io joins the trusted registries in
ci/trivy/data/ksv0125.yaml - configured rather than suppressed, so
AVD-KSV-0125 keeps being able to say something.
- Renovate block with NO version constraint, which is the counter-case to
garage and stalwart and is measured rather than assumed: docker versioning
already excludes the -N rebuild tags, the floating minors and latest/nightly.
Three dry-run fixtures are recorded at the block, and against the committed
config 26.7.4 offers exactly 26.7.5 (patch) and 26.8.0 (minor).
- No automerge exception, by decision (Gitea #83): the global automerge applies,
and the minor-upgrade procedure plus `kc.sh update-compatibility` is
documented in NOTES.txt instead.
- Two upstream defaults deviated from, each with its reason at the value:
db-pool-max-size 20 instead of 100 (two replicas at 100 ask a default
PostgreSQL for twice its max_connections), and uid 1000 / gid 0 instead of
this repo's 20000/1000, because the image declares keycloak:x:1000:0 and
chowns its install tree to that pair.
Part of #83
Two corrections found while verifying the first CI run, not from the plan.
THE DEFAULT THAT MADE THE READINESS PROBE TOOTHLESS. Keycloak's DATABASE health
check is the one row of upstream's "Available Checks" table whose "Requires
Metrics" column says yes; Cluster, Graceful shutdown and Keycloak Initialized
say no. So with metrics-enabled=false - this chart's default until now -
/health/ready answers 200 while the database is gone, and the pod keeps taking
logins it cannot serve. The probe path was right and the aggregate behind it
contained nothing external, which is the urlaubsverwaltung defect (#218) in a
new hiding place: not a health group this time, but an option named for
observability, where nobody looks when reviewing a probe.
keycloak.metrics.enabled therefore defaults to TRUE, and its comment argues
about readiness rather than about scraping. The price is stated at the value:
/metrics then answers on the management port, unauthenticated, carrying realm
names, client ids and login counters - on a port that is ClusterIP-only and in
no Ingress. Switching it off is now documented as a decision about whether
readiness can fail at all. The CI fixture keeps it on and says why, so a green
run really does touch the database.
THE STARTUP BUDGET NOW CARRIES ITS MEASUREMENT. From the install-test of PR #87
on the Gitea docker runner (k3s v1.35.5, two replicas, empty database, image
pulled during the run): 71.8 seconds from `helm install` to BOTH pods Ready -
first at ~70.8s, second one second later, 0 restarts. The 600s budget stays,
with the reason for the headroom written down: everything in that run was the
cheap case, and a first install against a real database serializes the second
replica behind the first's Liquibase migration lock instead of letting it
arrive a second later.
AND AN OVERCLAIM OF MY OWN, REMOVED. ci/keycloak/test-values.yaml said a green
run proves jdbc-ping found the second replica. It does not: readiness reports
one node's own health, so a node that never found its peer is Ready, healthy
and alone - and two such nodes are two islands that each believe they are the
cluster. What the test does rule out is the louder half, a replica that cannot
start or cannot share the database. The Infinispan view has to be read out of
the log, which is what step 2 of NOTES.txt tells the operator to do, now with
the note that no probe and no install-test replaces that step. Measurement
point 4 of the plan therefore stays open.
Part of #83
The first Trivy run on this chart reported one finding out of 96 tests:
AVD-KSV-0116, "Runs with a root primary or supplementary GID", because the pod
context carried runAsGroup: 0 - copied from what the image expects.
Checked instead of suppressed, and the check says the group is not needed.
Counted across the image layer of quay.io/keycloak/keycloak:26.8.0: 521 of 522
entries are owned by 1000:0, and every mode present (drwxrwxr-x, -rwxrwxr-x,
-rw-rw-r--) grants the OWNER full access. The one remaining entry is /opt
itself, which needs only traverse. So uid 1000 gets everything it reads and
executes through the owner bits, whatever group it is in.
Group 0 is upstream's OpenShift arbitrary-uid pattern - root group plus a
group-writable tree, so that ANY uid landing in group 0 can write. It buys
nothing once the owning uid is pinned, which this chart does. Writes all go
into the three emptyDirs, which fsGroup 1000 makes writable.
So runAsGroup is 1000, the finding is gone rather than excepted, and no entry
was added to ci/trivy/trivyignore.yaml - which matters more than one LOW
finding: that file scopes by chart, so an AVD-KSV-0116 exception for keycloak
would also have silenced it for the serving container, on a check that is
making a fair point about an identity provider.
The condition under which this goes back to 0 is written at the value: a path
uid 1000 does not own. There is none in this image today.
Part of #83
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Implements the plan on #83 together with the two findings and decisions recorded in its comments.
Shape
Deployment with two replicas,
cache=ispn, clustered through Keycloak's defaultjdbc-pingstack — discovery runs over the PostgreSQL that is there anyway, so no headless Service, no JGroups wiring, no extra option. No PVC: Keycloak 26 keeps everything in the database, user sessions included (volatile sessions are deprecated as of 26.8).Two Services (8080 and the management port 9000, the latter in no Ingress), two Ingresses, two
OnePasswordItems, a PodDisruptionBudget, probes on all three native endpoints, computed limits through thelimitFactorhelper,replicasplusscaleDown, and a strictvalues.schema.json.The two findings from #83, as built
1.
hostname-adminis not an access control. It is set so the admin URLs are right, and it is not presented as a boundary. Upstream, invalues.yaml,NOTES.txtand on the wiki page in the same words: it "does not prevent accessing the Administration REST API endpoints via the frontend URL specified by thehostnameoption. If you want to restrict access to the Administration REST API, you need to do it on the reverse proxy level." No/admindeny rule is rendered.2. Two init containers, because one would not work.
db,health-enabledandmetrics-enabledare build options (verified: each carries the tool icon in the 26.8.0 option table;cachedoes not). The published image is augmented with their defaults, and a runtime change makes Keycloak rewrite/opt/keycloak/lib/quarkusat start — whichreadOnlyRootFilesystem: trueforbids, and which upstream calls a start that "creates an image that is mutable". Not settingdbis no escape: the probes alone force it, becausehealth-enabledis a build option.An empty
emptyDirover that path fails too, and that is the part that is not guessable:QuarkusEntryPoint.classcarries the literalquarkus/quarkus-application.dat, andkc.shruns that class for every command,buildincluded — so with the file hidden,kc.sh buildcannot start either. Hence seed → build →start --optimized:…-seed(init)/seedcp -R /opt/keycloak/lib/quarkus/. /quarkus-build/…-build(init)/opt/keycloak/lib/quarkuskc.sh build…-deployment/opt/keycloak/lib/quarkusstart --optimizedSo no hardening deviation anywhere in this pod — which is why it is two init containers and not one with a writable root filesystem, and why no Trivy suppression was needed that would also have silenced the serving container.
The condition named in the decision — build options from one source — is enforced structurally, not by care: they exist only in the
keycloak.buildEnvhelper, the serving container passesstart --optimizedand no build option, and a build option's variable name inkeycloak.envfails the render naming the variable.Renovate: no constraint, and no automerge exception
The constraint
garageandstalwartneed is not needed here, and that is measured rather than assumed (Renovate 44.132.5,--platform=local --dry-run=lookup, real registry, against the committed config):appVersion26.8.0warnings: []26.7.426.7.5(patch),26.8.0(minor)No run offered a
-Nrebuild tag (26.8.0-0exists and points at the same digest as26.8.0), a floating minor,latestornightly. Docker versioning splits at the first dash and only offers candidates with the same suffix; ours has none.renovate-config-validatorpasses on the whole file.Per the decision on #83 there is no
packageRulesautomerge exception. The consequence is documented instead of prevented, prominently inNOTES.txtand on the wiki page: the minor-upgrade procedure (scaleDown: true, back up, bump, scale up), the statement that the automerge does not know about it, and thekc.sh update-compatibility metadata|checkcalls with their exit codes.Also in this PR
PodDisruptionBudgetin the repo, so it is now a convention — README §11 and the wiki style guide. It is rendered only while the computed replica count is>= 2:minAvailable: 1with one replica blockskubectl drainforever with a message that names the PDB rather than the replica count.quay.ioimage, soquay.iojoinsci/trivy/data/ksv0125.yaml— configured rather than suppressed, so AVD-KSV-0125 keeps its meaning.db-pool-max-size20 instead of 100 (two replicas at 100 ask a default PostgreSQL for twice itsmax_connections), and uid 1000 / gid 0 instead of this repo's 20000/1000, because the image declareskeycloak:x:1000:0and chowns its install tree to that pair.publish-keycloak.yml,keycloakin thechange-app-version.ymlchoice list, fixtures underci/keycloak/.runAsGroup: 1000, not the image's own0— checked rather than suppressed after Trivy reportedAVD-KSV-0116on the first run. Counted across the image layer: 521 of 522 entries are owned by1000:0, and every mode present grants the owner full access, so uid 1000 needs no group membership to read or execute anything. Group 0 is upstream's OpenShift arbitrary-uid pattern and buys nothing once the owning uid is pinned. Notrivyignoreentry was added, which matters more than one LOW finding: that file scopes by chart, so an exception would also have silenced the check for the serving container.Two corrections found after the first CI runs
The first green run was not the end of the verification, and reading on changed a default. Keycloak's database health check is the one row of upstream's "Available Checks" table whose "Requires Metrics" column says yes. With
metrics-enabled=false— this chart's default in the first commit —/health/readyanswers 200 while the database is gone, and the pod keeps taking logins it cannot serve. The probe path was right and the aggregate behind it contained nothing external: theurlaubsverwaltungdefect (#218) in a new hiding place, gated behind an option named for observability, which is nowhere near where a probe gets reviewed.So
keycloak.metrics.enablednow defaults totrueand its comment argues about readiness rather than scraping. The price is named at the value:/metricsthen answers on the management port, unauthenticated — a port that is ClusterIP-only and in no Ingress.And an overclaim of mine, removed in the same commit.
ci/keycloak/test-values.yamlsaid a green run provesjdbc-pingfound the second replica. It does not: readiness reports one node's own health, so a node that never found its peer is Ready, healthy and alone — and two such nodes are two islands that each believe they are the cluster. What the test does rule out is the louder half: a replica that cannot start, deadlocks on the migration lock or fights the first over the database. The Infinispan view has to be read out of the log, which is what step 2 ofNOTES.txtnow says no probe and no install-test replaces.Verification
Local (no container runtime in this workspace — no k3d, no Docker, no Trivy):
helm lint --strict keycloak— passes, 0 failed.helm templateagainst chart defaults and againstci/keycloak/test-values.yaml;helm package— ok.ci/run-quantity-render-test.sh— all 27 charts,failed: 0; keycloak's two resources blocks (keycloak.resources,keycloak.build.resources) both exercised.keycloak.envrefused (upper- and lowercase); each of 12 chart-owned blocks erased tonullstill renders hardening, the right probe paths and the computed limits;scaleDownandreplicas: 1render no PDB whilereplicas: 3does; everyrequired()fires, andingress.domainstays required withingress.enabled: falsewhileclusterIssueris freed when both Ingresses are off; the schema rejects an unknown key,dev-file, a feature name with a space and an out-of-range port.NOTES.txtrendered in three variants through thedefine/ConfigMap trick, sincehelm templatedoes not render it.The install-test runs with two replicas and a throwaway PostgreSQL. The rollout completes only when both pods are Ready, so a replica that cannot start or cannot share the database fails the run.
Measured (run 487, job "Install-test keycloak", Gitea docker runner, k3s v1.35.5, image pulled during the run): 71.8 s from
helm installto both pods Ready — the first at ~70.8 s, the second one second later —1/1 Running, 0 restarts, ending inOK: chart 'keycloak' installed and all workloads are Ready. That figure is now written at thestartupProbevalue as the justification for its 600 s budget, together with why the headroom stays: everything in that run was the cheap case, and a first install against a real database serializes the second replica behind the first's Liquibase lock instead of letting it arrive a second later.Because
health-enabledandmetrics-enabledare build options, the fact that all three probe paths answer on port 9000 is the proof that the build init container did its work — a build that silently did nothing would leave them 404.CI
Three runs on this branch, all green, each on
jlab-git-actions-docker-runner-0(k3s v1.35.5, kubectl v1.35.9):f784f40AVD-KSV-011663d0c7f1c14c600 to act on, 10 excepted)The 10 "excepted" in the last column are this repo's existing repo-wide exceptions (no CPU limits, the namespace artefact, and so on) — this chart contributes no finding and no new suppression.
What stays open
Two of the plan's five measurement points, and both need a real deployment — which is why this PR says
Part ofand notFixes:jdbc-pingreally found the second replica. Two Ready replicas are not that proof, for the reason above. The Infinispan view is read out of the log (NOTES.txtstep 2).hostname-adminserves the console and its resources cleanly on the second hostname. No ingress controller answers in the test cluster and nothing drives a browser. CI does prove that both Ingress objects are accepted, that the Service port they name exists, and that Keycloak accepts both hostname values as full URLs at start.Closed by this PR: 1 (startup time, measured above), 2 (
readOnlyRootFilesystem: trueholds, with emptyDirs only at the three paths) and 3 (there is no_FILEform, sosecretKeyRefit is).Not verified locally at all, because this workspace has no container runtime and no Trivy: anything needing a running pod, and the
trivy configscan — which does run in CI, reporting-only.Part of #83