Merge pull request 'feat(keycloak): one hostname per realm, in both routing modes' (#125) from feat/keycloak-realm-hosts into main

README.md

@@ -23,7 +23,7 @@ Every chart has a wiki page of its own — `https://git.jk-dev.de/jlab-cluster/h
 | `garage` | Garage (S3-compatible object store) | **One** node, `replication_factor = 1` — a deliberate trade with its price named (Gitea #81): the volume layer underneath already replicates and encrypts, so Garage replication on top would be nine times the raw data across two layers that know nothing about each other. What that costs is that the factor applies to the **metadata** too, where a corrupted database is upstream-documented as "generally not recoverable" and a replicated volume replicates the corruption. Hence three settings that are mandatory rather than optional: `metadata_fsync = true` (**off** in Garage's defaults), the automatic metadata snapshots, and those snapshots on a **second PVC** — a file-system snapshot of a live metadata database is not a restore point, so the volume backup covers the blocks and not the index. The first PVC carries `metadata_dir` **and** `data_dir` as two subdirectories, because volume snapshots are atomic per volume: that is what keeps index and blocks consistent with each other. Two further properties shape the whole chart: the image is **`FROM scratch`** and contains exactly one file, `/garage` — no shell, so **exec probes are impossible here** and an init container would need a second image — and the RPC secret has to arrive as an **environment variable**, because `rpc_secret_file` rejects any file with a group bit set while the kubelet sets one (the reasoning sits at the value). Started with `garage server --single-node`, which creates the cluster layout in-process before any listener opens and is idempotent on the layout version, and with `--default-access-key`/`--default-bucket`, which **import** key and bucket from the 1Password item — so the vault is the source of truth instead of a copy of what the server generated. Readiness is the admin API's `/health`, liveness and startup deliberately are **not**: `/health` answers 503 until a layout exists, so a node waiting for one would be restarted forever |
 | `gotenberg` | Gotenberg | Deployment + Service only (no ingress) |
 | `homeassistant` | Home Assistant | StatefulSet; the only chart carrying Helm tests (`templates/tests/`) |
-| `keycloak` | Keycloak (OpenID Connect / SAML identity provider) | **Two** replicas with `cache=ispn`, clustered through Keycloak's default `jdbc-ping` stack — discovery runs over the PostgreSQL that is there anyway, so no headless Service and no JGroups wiring. Deployment, **no PVC** (Keycloak 26 keeps everything in the database, user sessions included). **The chart has two init containers and needs both** (Gitea #83): `db`, `health-enabled` and `metrics-enabled` are Keycloak *build* options, the published image is built with their defaults, and a runtime change makes Keycloak rewrite `/opt/keycloak/lib/quarkus` at start — which `readOnlyRootFilesystem: true` forbids and which upstream describes as a start that "creates an image that is mutable". An empty `emptyDir` over that path does not work either, because `QuarkusEntryPoint` loads `quarkus/quarkus-application.dat` before it dispatches any command, `build` included. So one init container **seeds** the directory into a volume, the next runs `kc.sh build` into it, and the serving container runs `start --optimized` — every container keeping the full hardening baseline. Build options therefore exist in exactly **one** place and a build option in `keycloak.env` fails the render. First chart with a `PodDisruptionBudget` (§11) and first `quay.io` image (hence the entry in `ci/trivy/data/ksv0125.yaml`). Two Services (8080, and the management port 9000 **never** published — it serves `/health/*` and optionally `/metrics` unauthenticated) and two Ingresses. **The admin hostname is not an access control**: `hostname-admin` only decides which URLs the server generates, and upstream states that it "does not prevent accessing the Administration REST API endpoints via the frontend URL" — restricting that API is reverse-proxy work, and the chart renders no `/admin` deny rule. `ingress.domain` is read as `KC_HOSTNAME` (as a full URL, which upstream requires alongside `hostname-admin`), so it stays required with the Ingress off. Liveness is `/health/live` and deliberately not `/health` or `/health/ready` — those include the database, and a liveness probe on them restarts the whole cluster for as long as the database is away. **`keycloak.metrics.enabled` defaults to `true` for the readiness probe, not for the metrics**: upstream's table of health checks carries a "Requires Metrics" column and the *database* check is the row that says yes, so with it off `/health/ready` answers 200 while the database is gone — the `urlaubsverwaltung` shape (#218) with the dependency hidden behind an option whose name says observability. Two traps at the values: upstream's `db-pool-max-size` default of 100 would ask a default PostgreSQL for 200 connections across two replicas (the chart ships 20), and two releases sharing one database **merge into one Infinispan cluster** unless the cluster names differ |
+| `keycloak` | Keycloak (OpenID Connect / SAML identity provider) | **Two** replicas with `cache=ispn`, clustered through Keycloak's default `jdbc-ping` stack — discovery runs over the PostgreSQL that is there anyway, so no headless Service and no JGroups wiring. Deployment, **no PVC** (Keycloak 26 keeps everything in the database, user sessions included). **The chart has two init containers and needs both** (Gitea #83): `db`, `health-enabled` and `metrics-enabled` are Keycloak *build* options, the published image is built with their defaults, and a runtime change makes Keycloak rewrite `/opt/keycloak/lib/quarkus` at start — which `readOnlyRootFilesystem: true` forbids and which upstream describes as a start that "creates an image that is mutable". An empty `emptyDir` over that path does not work either, because `QuarkusEntryPoint` loads `quarkus/quarkus-application.dat` before it dispatches any command, `build` included. So one init container **seeds** the directory into a volume, the next runs `kc.sh build` into it, and the serving container runs `start --optimized` — every container keeping the full hardening baseline. Build options therefore exist in exactly **one** place and a build option in `keycloak.env` fails the render. First chart with a `PodDisruptionBudget` (§11) and first `quay.io` image (hence the entry in `ci/trivy/data/ksv0125.yaml`). Two Services (8080, and the management port 9000 **never** published — it serves `/health/*` and optionally `/metrics` unauthenticated) and two Ingresses — plus one routing object per `keycloak.realmHosts` entry (Gitea #122), empty by default, which gives a realm a hostname of its own by sending that hostname's bare `/` to `/realms/<realm>/account/`: `app-root` in Ingress mode, an `Exact` match with a `RequestRedirect` filter in Gateway mode. Those entries carry a **required** per-entry TLS secret name and an **optional** per-entry issuer, the documented deviation in §4, because the hostnames they publish already have certificates and often share one. **The admin hostname is not an access control**: `hostname-admin` only decides which URLs the server generates, and upstream states that it "does not prevent accessing the Administration REST API endpoints via the frontend URL" — restricting that API is reverse-proxy work, and the chart renders no `/admin` deny rule. `ingress.domain` is read as `KC_HOSTNAME` (as a full URL, which upstream requires alongside `hostname-admin`), so it stays required with the Ingress off. Liveness is `/health/live` and deliberately not `/health` or `/health/ready` — those include the database, and a liveness probe on them restarts the whole cluster for as long as the database is away. **`keycloak.metrics.enabled` defaults to `true` for the readiness probe, not for the metrics**: upstream's table of health checks carries a "Requires Metrics" column and the *database* check is the row that says yes, so with it off `/health/ready` answers 200 while the database is gone — the `urlaubsverwaltung` shape (#218) with the dependency hidden behind an option whose name says observability. Two traps at the values: upstream's `db-pool-max-size` default of 100 would ask a default PostgreSQL for 200 connections across two replicas (the chart ships 20), and two releases sharing one database **merge into one Infinispan cluster** unless the cluster names differ |
 | `matrix-synapse` | Matrix Synapse | StatefulSet plus an nginx delegation Deployment/Service |
 | `minecraft-router` | itzg/mc-router | Deployment + routing ConfigMap + NodePort Service (TCP only, no ingress); `mappings` is a required value |
 | `minecraft-server` | itzg/minecraft-server | StatefulSet + PVC + Service + init-script ConfigMap (no ingress); service type is a value (`ClusterIP` by default, routed via `minecraft-router`). **Only the newest Java line is maintained** (chart 5.0.0, #158): the directory carries one line, older ones stay published as frozen artifacts (java8/java17 at `4.0.0`, java21 at `4.1.0`) and receive no further chart changes — publishing once per line out of one directory is what made the lines drift. To run an older JVM with the current chart, set `image.tagOverride`; the chart then checks that `minecraft.version` fits the Java line of the **effective** tag and fails the render with both values named. Two cases pass unchecked on purpose: a tag without a `java<N>` part and a `minecraft.version` that is not a plain numeric `x[.y[.z]]`. `image.tagOverride` is deliberately outside Renovate's reach — `renovate.json5` tracks `appVersion` only. Since chart 2.0.0 the tag is an **immutable dated upstream release** (`appVersion: <release>-java8` / `-java17` / `-java21` / `-java25`, e.g. `2026.8.2-java8`) instead of the continuously rebuilt rolling line tag. Pilot for the strict `values.schema.json` (#68), which is now binding for every chart |
@@ -181,7 +181,7 @@ Three things to know before touching it:
 - **What is deliberately not checked**, named so the gaps are not mistaken for coverage. *That traffic flows* — no Gateway controller runs in CI and none will, so `Accepted: True` is not obtainable; that claim is simply not made anywhere in this pipeline. *That any redirect is ever served* — the pairing check proves the rendered **object** carries a `RequestRedirect` rule on the bare prefix and nothing more: whether an implementation honours the `statusCode` or declines it and fails the whole route is outside what a render can see. *Mutual exclusion with the Ingress* — each chart aborts its own render when `ingress.enabled` and `httpRoute.enabled` are both true, which is where it belongs: this test renders one configuration per chart, and a rule here would have to assume the switch is chart-global, which may not survive the multi-app charts of the sweep. *Cross-namespace and non-Service backends*, and a rule with no `backendRefs` at all (legal for a filter-only rule) — all three are **reported as "not checked", never counted as passing**. The redirect rule the pairing check demands is itself backend-less, so a chart that satisfies that check also earns one "not checked" line; both verdicts are correct at once.
 - **Two anti-vacuum guards**, because the convention ships `httpRoute.enabled: false` and this test would otherwise be green over nothing. `ci/httproute/<chart>.yaml` is a render-only overlay applied **last** whose one job is to put a chart into Gateway mode — and a chart that *has* one and still renders no route **fails**. On top of that the run fails if **no assertion at all** held anywhere. `cyberchef` is exercised through its ordinary install-test fixture (which installs in Gateway mode, see below), every other converted chart through its overlay. For a multi-app chart the overlay also has to enter every shape the template has: `ci/httproute/freesailarr.yaml` moves one app onto a host of its own so that the shared-host route *and* the per-app route are both rendered.
 
-Measured in CI (run 719, the pilot): **30 charts, 4 assertions held**, whole job 13 s including checkout and the Helm setup, no cluster. Group 1 took that to 20 assertions over 10 charts and group 2 to 36 over 15; the prefix/redirect pairing of Gitea #118 adds the first check since the pilot rather than a chart, which brought it to 37 over 15 — one assertion for `freesailarr`'s `/qbittorrent`, plus one more "not checked" line for the backend-less redirect rule it demands. Group 3 took it to `56` assertions over `21` charts, from six charts of which three contribute more than one route: `keycloak` 4 (public and admin routes), `matrix-synapse` 4 (homeserver and delegation routes), `homeassistant` 5 (two routes, three rules — its overlay was the first in the repo that has to *supply* hostnames, since the chart has no `ingress.domain`), `bulwark` 2, `stalwart` 2, `ytdlp-web-player` 2. **`rss-bridge` finishes the sweep at `58` assertions over `22` charts** — read on this branch, which does **not** contain the realm-hostname round of Gitea #122 (measured separately at `60` over `21`, on a branch that did not contain `rss-bridge`). **Neither total includes the other, so the figure to trust from either is the delta**; the running total is whatever the first run on `main` prints once both have landed, and it should be read there rather than added up by hand. The delta here is one route, so two assertions, and its overlay is the **second** that has to supply hostnames — for a different reason than `homeassistant`'s, which is worth keeping apart. `homeassistant` has no `httpRoute.hostnames` key at all and its overlay supplies the names through `ingress.hosts`; `rss-bridge` *has* the key and its overlay fills it, because this chart has no `ingress.domain` to fall back to either but its hostnames are unambiguous (one Service, one path).
+Measured in CI (run 719, the pilot): **30 charts, 4 assertions held**, whole job 13 s including checkout and the Helm setup, no cluster. Group 1 took that to 20 assertions over 10 charts and group 2 to 36 over 15; the prefix/redirect pairing of Gitea #118 adds the first check since the pilot rather than a chart, which brought it to 37 over 15 — one assertion for `freesailarr`'s `/qbittorrent`, plus one more "not checked" line for the backend-less redirect rule it demands. Group 3 took it to `56` assertions over `21` charts, from six charts of which three contribute more than one route: `keycloak` 4 (public and admin routes), `matrix-synapse` 4 (homeserver and delegation routes), `homeassistant` 5 (two routes, three rules — its overlay was the first in the repo that has to *supply* hostnames, since the chart has no `ingress.domain`), `bulwark` 2, `stalwart` 2, `ytdlp-web-player` 2. **`rss-bridge` finishes the sweep at `58` assertions over `22` charts** — read on this branch, which does **not** contain the realm-hostname round of Gitea #122 (measured separately at `60` over `21`, on a branch that did not contain `rss-bridge`). **Neither total includes the other, so the figure to trust from either is the delta**; the running total was therefore read off the first branch that contained both — the realm-hostname branch rebased onto this one — and it is **`62` assertions over `22` charts**, `0` failed. `keycloak` contributes `8` of them: one `parentRefs` and one `backendRefs` assertion for each of its four routes, the two realm routes adding one "not checked" line each for their backend-less redirect rule (`keycloak: ok (8 assertion(s) held, 2 not checked)`). The #118 prefix/redirect pairing deliberately does **not** fire on those two: it is scoped to rules that strip a prefix with a `ReplacePrefixMatch` rewrite, and a realm route strips nothing — it redirects the bare root of a hostname that serves one realm. The delta here is one route, so two assertions, and its overlay is the **second** that has to supply hostnames — for a different reason than `homeassistant`'s, which is worth keeping apart. `homeassistant` has no `httpRoute.hostnames` key at all and its overlay supplies the names through `ingress.hosts`; `rss-bridge` *has* the key and its overlay fills it, because this chart has no `ingress.domain` to fall back to either but its hostnames are unambiguous (one Service, one path).
 
 **And in the same PR that chart's install-test went green, which is the event Gitea #51 was waiting for.** It had failed on the Gitea runner for months (uid 33 unable to read `/app/index.php`, startup probe 403 through its whole budget), and the chart was deliberately kept *out* of `ci/excluded-charts.txt` precisely so that the day it passed would be noticed rather than silenced. It passed: k3d cluster up, `helm install` in Ingress mode, pod `1/1 Ready` in 22 s with 0 restarts, and no `Permission denied` anywhere in the job. **The chart cannot be the reason, and that is measured rather than argued** — the rendered output for the fixture values is byte-identical to `origin/main`, so the pod spec that installed is exactly the one `main` produces. The implicated layer is the runner, which is where #51 had already put the burden of proof. Two cautions belong with the result: **nobody has identified what changed on the runner**, and a green test does not explain itself — it has reproduced on every run of that branch since, across several commits, so it is not a fluke - but that is still a colour and not a cause; and the install covers **Ingress** mode, since that is what the fixture installs — so this chart's Gateway mode stays render-only, exactly like its 21 siblings rather than worse than them.
 
@@ -390,6 +390,7 @@ These guidelines are **binding for every current and future chart** in this repo
 - `ingress.className` selects the ingress class and defaults to `nginx`.
 - `ingress.annotations` takes optional extra annotations that are merged with the cert-manager `cluster-issuer` annotation the chart always sets.
 - The TLS secret is named `tls-<release>-<chart>-ingress`.
+- **A set of objects publishing ONE backend under hostnames the chart does not own is the third admissible shape, and it buys its secret name and its issuer per entry (Gitea #122).** Where a chart's app serves several tenants from one deployment and each wants a hostname of its own — `keycloak`'s realms are the case this was written for — the hostnames come as a values list and the chart renders one object per entry. Two convention rules do not survive that, and both deviations are **required**, not merely tolerated: the TLS secret name is a per-entry value with **no** default, because these hostnames already have certificates and several entries typically share one (a wildcard for a zone), so a derived `tls-<release>-<chart>-...` name per entry would ask cert-manager for a second certificate for a name that is already covered — one ACME order per hostname, for nothing; and the cert-manager annotation becomes **optional per entry**, because cert-manager keys a Certificate by the secret it writes, so of several objects naming one secret exactly one may carry it. That is the same deliberate omission the qBittorrent object in `freesailarr` makes, for the same reason, and it is why the secret name is required rather than defaulted: a chart that invents the name cannot also leave the issuing to somebody else. Annotations are per entry too, since the set is not uniform across the hostnames. Object names carry the **list index**, for the reason spelled out in the homeassistant bullet below — a hostname is not a dependable object-name fragment, so reordering the list renames and recreates the objects — and the entries render in **both** routing modes: a routing value in this repository is mode-independent by design, so a feature rendering in one mode only would silently disappear when an installation switched. That is also why the per-entry `parentRefs` exists, as the counterpart of the per-entry class.
 - **Documented exception — homeassistant, and it extends to the Gateway mode**: keeps its upstream-style flexible ingress values (host/tls lists, no required values); only its `ingress.className` default follows the standard (`nginx`, clearable to omit the field). Because it has **no `ingress.domain`**, it has **no `httpRoute.hostnames`** either — the second chart without that key after `freesailarr`, and the schema rejects it outright so the absence is enforced rather than merely intended. The hostnames and the paths come from `ingress.hosts`, which is therefore read in **both** modes and is not an Ingress-only value, and the chart renders **one `HTTPRoute` per entry** of that list with one rule per path of the entry. One object carrying several hostnames would be wrong rather than merely different: an `HTTPRoute`'s rules apply to *every* hostname of the object, and per-host paths are this chart's whole point. `ingress.className`, `ingress.tls` and `ingress.annotations` are not read in Gateway mode. Route names carry the entry's **index**, not its hostname (a hostname is not a dependable object-name fragment), so reordering `ingress.hosts` renames routes and therefore recreates rather than edits them — harmless, since a route holds no state and no certificate, but worth knowing before it looks like a bug. Its `pathType` default is the awkward one; see the `ImplementationSpecific` rule in the Gateway-API bullets below.
 - **A shared domain with sub-paths is the second admissible ingress shape.** Where a chart exposes several apps, they may either each take a host of their own or share one host and be told apart by path (`domain.tld/sonarr`, `domain.tld/radarr`, …) — one DNS name and one certificate for the whole bundle. The rules for the shared shape: the domain is a single value (`ingress.host`), each app's path is its own value defaulting to `/<app>`, the paths render in one Ingress object `<release>-<chart>-ingress` with the standard TLS secret `tls-<release>-<chart>-ingress`, and an app that carries a host of its own always wins and keeps its own object — so both shapes mix without extra values. An app that needs a per-path proxy annotation (`rewrite-target` is per **object**, not per path) gets its own object on the same host, repeating the same TLS secret name and **omitting** the cert-manager annotation, so that exactly one certificate is requested. Apps that cannot serve a base path must be rendered at `/` or refused with a `fail` — never rendered somewhere they provably cannot work.
 - **The Gateway API is the second routing mode, and it is implementation-neutral by construction (Gitea #114).** A chart offers it through one values block, off by default: