feat: Gateway-API as an alternative to the Ingress in five annotation charts
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 14s
Validate Helm Charts / Selector match (pull_request) Successful in 14s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 14s
Validate Helm Charts / Build freesailarr (pull_request) Successful in 7s
Validate Helm Charts / Build outline (pull_request) Successful in 7s
Validate Helm Charts / Build paperless-ngx (pull_request) Successful in 8s
Validate Helm Charts / Build zipline (pull_request) Successful in 8s
Validate Helm Charts / Build calibre-web-automated (pull_request) Successful in 7s
Validate Helm Charts / Trivy config calibre-web-automated (pull_request) Successful in 19s
Validate Helm Charts / Trivy config freesailarr (pull_request) Successful in 24s
Validate Helm Charts / Trivy config outline (pull_request) Successful in 17s
Validate Helm Charts / Trivy config paperless-ngx (pull_request) Successful in 19s
Validate Helm Charts / Trivy config zipline (pull_request) Successful in 19s
Install-test / Install-test calibre-web-automated (pull_request) Successful in 2m11s
Install-test / Install-test freesailarr (pull_request) Successful in 3m55s
Install-test / Install-test outline (pull_request) Successful in 2m44s
Install-test / Install-test paperless-ngx (pull_request) Successful in 3m52s
Install-test / Install-test zipline (pull_request) Successful in 2m6s
Validate Helm Charts / Install-test (pull_request) Successful in 14m50s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 14s
Validate Helm Charts / Selector match (pull_request) Successful in 14s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 14s
Validate Helm Charts / Build freesailarr (pull_request) Successful in 7s
Validate Helm Charts / Build outline (pull_request) Successful in 7s
Validate Helm Charts / Build paperless-ngx (pull_request) Successful in 8s
Validate Helm Charts / Build zipline (pull_request) Successful in 8s
Validate Helm Charts / Build calibre-web-automated (pull_request) Successful in 7s
Validate Helm Charts / Trivy config calibre-web-automated (pull_request) Successful in 19s
Validate Helm Charts / Trivy config freesailarr (pull_request) Successful in 24s
Validate Helm Charts / Trivy config outline (pull_request) Successful in 17s
Validate Helm Charts / Trivy config paperless-ngx (pull_request) Successful in 19s
Validate Helm Charts / Trivy config zipline (pull_request) Successful in 19s
Install-test / Install-test calibre-web-automated (pull_request) Successful in 2m11s
Install-test / Install-test freesailarr (pull_request) Successful in 3m55s
Install-test / Install-test outline (pull_request) Successful in 2m44s
Install-test / Install-test paperless-ngx (pull_request) Successful in 3m52s
Install-test / Install-test zipline (pull_request) Successful in 2m6s
Validate Helm Charts / Install-test (pull_request) Successful in 14m50s
Group 2 of the #114 sweep: the charts whose behaviour hangs off an nginx annotation - calibre-web-automated, outline, paperless-ngx, zipline (proxy-body-size) and freesailarr (rewrite-target). Each gets the form the pilot (cyberchef, PR #115) fixed and group 1 (PR #116) repeated: an httpRoute block off by default, a template rendering ONLY an HTTPRoute with parentRefs - no Gateway, no gatewayClassName, no Certificate, no implementation-specific policy object - render aborts on both modes at once (naming both value paths), on an empty parentRefs and on a missing hostname, empty hostnames meaning [ingress.domain], a schema entry closed against unknown keys with an open parentRef definition, and a NOTES.txt that names the active mode and what the mode costs. Minor, additive, defaults unchanged: `helm template` with each chart's CI values is byte-identical to origin/main for all five. The upload limit is DOCUMENTED, not rebuilt (decision 2 of the issue). The Gateway API knows no body size; a policy object would bind the chart to one implementation. So the Gateway branch of NOTES.txt says, per chart and in the chart's own terms, that the value is NOT ENFORCED and names where to set it: an e-book library (calibre), an upload service (zipline), the web-UI half of the document intake while the consume directory keeps no limit at all (paperless), and the open #12 pair (outline). Four findings that were not in the plan, each argued in the file it changes: freesailarr gated freesailarr.sharedIngressApps on ingress.enabled, and that list is read by the <APP>__SERVER__URLBASE variables as well as by the routing objects. In Gateway mode it would have come out EMPTY while the route served radarr, sonarr and prowlarr under their prefixes: UrlBase unset, assets and API calls linked at the host root, broken UI behind a Ready pod. The helper's two guards (seerr/mediathekarr only at "/", no duplicate paths) would have gone unenforced there too. Fixed with freesailarr.published (ingress.enabled OR httpRoute.enabled) - the same shape as sure.publishedOverTls in group 1, and identical to ingress.enabled while httpRoute.enabled is false. Measured: on origin/main with ingress.enabled: false the render emits ZERO URLBASE variables; with this change in Gateway mode it emits three. paperless-ngx was the last chart in the repository rendering its Ingress UNCONDITIONALLY. Without an ingress.enabled switch there is no second value path for the mutual-exclusion abort to name, and the Ingress plus its unconditional required() on clusterIssuer would have kept rendering in Gateway mode - a value demanded for nothing, which is issue #114's "Folge 1" and README guideline 12. The switch was added, defaulting to true, so the render is unchanged. paperless-ngx PAPERLESS_URL had no required() of its own: it relied on the Ingress template's, which stops firing the moment the Ingress is switchable. With httpRoute.hostnames set and ingress.domain empty it would have rendered "https://" and refused every login POST as a CSRF failure. It now carries its own required(), naming what the value is for. calibre-web-automated claimed, in values.yaml and in its schema, that ingress.domain is read nowhere but in the Ingress template. The HTTPRoute reads it, so the claim now holds only while both modes are off. Corrected rather than left standing - the same correction garage needed in group 1. freesailarr is where the Gateway API is LESS machinery than what it replaces, not a translation of it. rewrite-target applies per Ingress OBJECT, so qBittorrent (no base path, prefix must be stripped) needed a second Ingress on the shared host, which then had to repeat the first object's TLS secret name, omit the cert-manager annotation conditionally so only one Certificate is requested, and match an ImplementationSpecific regex path for the rewrite's capture group. A URLRewrite filter applies per RULE: one route, one rule per app, the filter on qBittorrent's rule, a plain PathPrefix. All four crutches gone. It is also the only chart without httpRoute.hostnames - it has no ingress.domain either, and a chart-global hostname list could not say which of the ten apps it belonged to, so the hostnames stay in ingress.host and <app>.ingress.host. CI: one render overlay per chart under ci/httproute/, so the assertion is not vacuum-green - ci/run-httproute-match-test.sh goes from 20 held assertions over 10 charts to 36 over 15. freesailarr's overlay moves bazarr onto a host of its own, so the shared-host route AND the per-app route are both rendered; without that the per-app branch would be exercised by nothing. The install-test fixtures stay in Ingress mode: the install runs one configuration per chart, cyberchef already pays that price against the real Gateway-API CRDs, and for paperless-ngx that fixture is the only install of the valkey-subchart path anywhere. README also carries a stale figure fixed here: the HTTPRoute test was still documented at the pilot's 4 assertions over 2 charts, although group 1 had taken it to 20 over 10. Not proven, by construction: that traffic flows. No Gateway controller runs in CI. Not run locally either: k3d, Docker, Trivy. The Gateway branch of NOTES.txt is EXECUTED by the render test (a template error in it fails the job, verified) but its text is printed by no pipeline stage - helm template does not emit notes, and the install-test installs in Ingress mode. Part of #114
This commit is contained in:
34 files changed
+1794
-74
No files matched your search
@@ -176,9 +176,9 @@ Three things to know before touching it:
|
||||
|
||||
- **Helm is the YAML parser**, through `ci/yaml2json`, for the same reason as the selector-match test above — the runners have neither `pyyaml` nor `yq`, and a parser written in `awk` fails by reading nothing and reporting nothing.
|
||||
- **What is deliberately not checked**, named so the gaps are not mistaken for coverage. *That traffic flows* — no Gateway controller runs in CI and none will, so `Accepted: True` is not obtainable; that claim is simply not made anywhere in this pipeline. *Mutual exclusion with the Ingress* — each chart aborts its own render when `ingress.enabled` and `httpRoute.enabled` are both true, which is where it belongs: this test renders one configuration per chart, and a rule here would have to assume the switch is chart-global, which may not survive the multi-app charts of the sweep. *Cross-namespace and non-Service backends*, and a rule with no `backendRefs` at all (legal for a filter-only rule) — all three are **reported as "not checked", never counted as passing**.
|
||||
- **Two anti-vacuum guards**, because the convention ships `httpRoute.enabled: false` and this test would otherwise be green over nothing. `ci/httproute/<chart>.yaml` is a render-only overlay applied **last** whose one job is to put a chart into Gateway mode — and a chart that *has* one and still renders no route **fails**. On top of that the run fails if **no assertion at all** held anywhere. Today `cyberchef` is exercised through its ordinary install-test fixture (which installs in Gateway mode, see below) and `template-chart` through its overlay.
|
||||
- **Two anti-vacuum guards**, because the convention ships `httpRoute.enabled: false` and this test would otherwise be green over nothing. `ci/httproute/<chart>.yaml` is a render-only overlay applied **last** whose one job is to put a chart into Gateway mode — and a chart that *has* one and still renders no route **fails**. On top of that the run fails if **no assertion at all** held anywhere. `cyberchef` is exercised through its ordinary install-test fixture (which installs in Gateway mode, see below), every other converted chart through its overlay. For a multi-app chart the overlay also has to enter every shape the template has: `ci/httproute/freesailarr.yaml` moves one app onto a host of its own so that the shared-host route *and* the per-app route are both rendered.
|
||||
|
||||
Measured in CI (run 719): **30 charts, 4 assertions held**, whole job 13 s including checkout and the Helm setup, no cluster. The fail path was exercised rather than assumed: a Service renamed in the route, a port the Service does not expose, a port left out entirely, an emptied `parentRefs` with the chart's own guard disabled, an overlay that renders no route, and a run with zero assertions — each produced a `FAIL` naming both sides and a non-zero exit.
|
||||
Measured in CI (run 719, the pilot): **30 charts, 4 assertions held**, whole job 13 s including checkout and the Helm setup, no cluster. Group 1 took that to 20 assertions over 10 charts and group 2 to **36 over 15** — the sweep adds charts, not checks, so the job stays a render. The fail path was exercised rather than assumed: a Service renamed in the route, a port the Service does not expose, a port left out entirely, an emptied `parentRefs` with the chart's own guard disabled, an overlay that renders no route, and a run with zero assertions — each produced a `FAIL` naming both sides and a non-zero exit.
|
||||
|
||||
### Install-test (PR pipeline stage 2)
|
||||
|
||||
@@ -383,12 +383,17 @@ These guidelines are **binding for every current and future chart** in this repo
|
||||
- **`parentRefs` is mandatory in Gateway mode**, enforced by a `fail` on the empty list. A route without a parent attaches to nothing and is simply inert — created, schema-valid, rejected by nobody, reported by nothing.
|
||||
- **One source of truth for the hostname.** Empty `hostnames` means `[ingress.domain]`; two values that have to agree drift apart. `ingress.domain` is therefore read in Gateway mode too — it names the release's public hostname and is not an Ingress switch.
|
||||
- **`NOTES.txt` says which mode is active**, as `keycloak` already does for its cases, and names what the mode costs.
|
||||
- **What has no counterpart, and must be said rather than rebuilt.** The Gateway API has no annotation channel for controller behaviour. `rewrite-target` has a real equivalent (the `URLRewrite` filter in a rule); **`proxy-body-size` has none** — an upload limit is implementation-specific policy, so a chart that depends on one must print in its Gateway-mode `NOTES.txt` that the limit is **not enforced** and has to be set at the Gateway. Five charts *set* one by default (`bulwark`, `calibre-web-automated`, `outline`, `paperless-ngx`, `zipline`), and `garage` is a sixth that *depends* on one without setting it: its `ingress.annotations` comment tells the operator to set `proxy-body-size: "0"` because ingress-nginx's 1 MB default truncates S3 uploads. Measured while sweeping group 1 — "sets no behaviour-bearing annotation" is not the same question as "needs none", and the sweep screens on the second one. The same applies to auth annotations in front of an app with no login of its own: in Gateway mode that escape hatch does not exist in the chart at all (`cyberchef`, `searxng`), and so does any annotation a chart documents as its only lever for a response header (`bentopdf`: `X-Robots-Tag`).
|
||||
- **A value derived from `ingress.enabled` is a second thing the sweep has to look for, and it is not visible in the Ingress template.** Gateway mode sets `ingress.enabled: false` *by construction*, so any value computed from it flips — and where it stood for "is there a TLS-terminating proxy in front", the flipped value is wrong while nothing fails. Found in `sure`, whose `RAILS_ASSUME_SSL`/`RAILS_FORCE_SSL` would have gone `false` with TLS terminated at the Gateway: insecure cookies, no HSTS, `http://` links, green pod. The fix is one helper reading `ingress.enabled OR httpRoute.enabled`, not a new value. Still open on this count: `freesailarr` and `ytdlp-web-player` both read `.Values.ingress.enabled` outside their Ingress templates.
|
||||
- **CI**: either the chart's install-test fixture installs in Gateway mode, or it gets a render-only overlay at `ci/httproute/<chart>.yaml` — otherwise `ci/run-httproute-match-test.sh` is green over nothing for it, which that test fails on.
|
||||
- **What has no counterpart, and must be said rather than rebuilt.** The Gateway API has no annotation channel for controller behaviour. `rewrite-target` has a real equivalent (the `URLRewrite` filter in a rule); **`proxy-body-size` has none** — an upload limit is implementation-specific policy, so a chart that depends on one must print in its Gateway-mode `NOTES.txt` that the limit is **not enforced** and has to be set at the Gateway. Five charts *set* one by default (`bulwark`, `calibre-web-automated`, `outline`, `paperless-ngx`, `zipline` — all but `bulwark` carry that disclosure since group 2), and `garage` is a sixth that *depends* on one without setting it: its `ingress.annotations` comment tells the operator to set `proxy-body-size: "0"` because ingress-nginx's 1 MB default truncates S3 uploads. Measured while sweeping group 1 — "sets no behaviour-bearing annotation" is not the same question as "needs none", and the sweep screens on the second one. `freesailarr` is the mirror image of `garage` and was measured in group 2: it sets no body-size annotation and needs none itself, but `ingress.annotations` is the only lever an operator has for one (qBittorrent takes torrent-file uploads), so its Gateway-mode `NOTES.txt` says that an annotation set there is gone. The same applies to auth annotations in front of an app with no login of its own: in Gateway mode that escape hatch does not exist in the chart at all (`cyberchef`, `searxng`), and so does any annotation a chart documents as its only lever for a response header (`bentopdf`: `X-Robots-Tag`).
|
||||
- **`rewrite-target` is the one place where the Gateway API is less machinery than what it replaces, not a translation of it.** `rewrite-target` applies per **Ingress object**; a `URLRewrite` filter applies per **rule**. A chart serving several apps under one host, one of which needs its prefix stripped, therefore needs a *second* Ingress object for that app — which then has to repeat the other object's TLS secret name, omit the cert-manager annotation so only one certificate is requested (conditionally, for the case where no other object renders), and match on an `ImplementationSpecific` regex path whose capture group the rewrite refers to. In Gateway mode all four disappear: one route, one rule per app, the filter on one rule, a plain `PathPrefix`. Measured on `freesailarr` in group 2, the only chart here with a `rewrite-target`; `bulwark` is the second and is in group 3.
|
||||
- **A value derived from `ingress.enabled` is a second thing the sweep has to look for, and it is not visible in the Ingress template.** Gateway mode sets `ingress.enabled: false` *by construction*, so any value computed from it flips — and where it stood for "is there a TLS-terminating proxy in front" or "is this app published at all", the flipped value is wrong while nothing fails. Two finds so far, both fixed by one helper reading `ingress.enabled OR httpRoute.enabled` rather than by a new value:
|
||||
- `sure` (group 1), whose `RAILS_ASSUME_SSL`/`RAILS_FORCE_SSL` would have gone `false` with TLS terminated at the Gateway: insecure cookies, no HSTS, `http://` links, green pod. Helper `sure.publishedOverTls`.
|
||||
- `freesailarr` (group 2), whose `freesailarr.sharedIngressApps` was gated on `ingress.enabled` and is read by the `<APP>__SERVER__URLBASE` environment variables as well as by the routing objects. In Gateway mode that list would have come out **empty**, so Radarr, Sonarr and Prowlarr would have been told no base path while their routes served them under `/radarr`, `/sonarr`, `/prowlarr` — a UI whose assets and API calls are linked at the host root, behind a Ready pod. The two guards living in that helper (seerr/mediathekarr only at `/`, no duplicate paths on the shared host) would have gone unenforced in that mode for the same reason. Helper `freesailarr.published`.
|
||||
|
||||
Reference implementation: `template-chart/templates/xxx.httproute.yaml`. Pilot: `cyberchef`. Group 1 — the charts rendering one Ingress with no behaviour-bearing annotation — followed: `bentopdf`, `garage`, `patchmon`, `rallly`, `searxng`, `sure`, `urlaubsverwaltung`, `voucher-vault`. That leaves **twelve** charts plus `rss-bridge`, which is frozen for changes (#51) and is why #114 cannot be closed while #51 is open: group 2 is the annotation cases (`calibre-web-automated`, `outline`, `paperless-ngx`, `zipline`, `freesailarr`) and group 3 the special shapes (`homeassistant` is the documented ingress exception, `keycloak` and `bulwark` have two Ingresses each, `matrix-synapse` renders in a loop, `stalwart` renders per node group, `ytdlp-web-player` hangs off the oauth2-proxy subchart). `garage` was one of the named special cases and is done: it ships `ingress.enabled: false`, so both of its modes are off by default and its route is exercised by a render overlay rather than by the install-test, which keeps its Ingress.
|
||||
- **Documented exception — freesailarr**: the chart bundles several independently exposed apps in one pod, so it has **no** `ingress.domain`. It offers both shapes. By default each app carries its own `<app>.ingress.enabled` plus a required `<app>.ingress.host` (required only while that app's ingress is enabled and it is not on the shared domain) and renders its own Ingress object `<release>-<chart>-<app>-ingress` with its own TLS secret `tls-<release>-<chart>-<app>-ingress`. Setting the global `ingress.host` switches every app without a host of its own to the shared-domain shape under `<app>.ingress.path`; qBittorrent (no base path, needs a prefix-stripping rewrite) gets the extra object described above, and seerr as well as mediathekarr are pinned to `/` — mediathekarr because its UI links assets with root-absolute paths, which a rewrite cannot fix; with both of them on one shared host the duplicate-path guard fires and one of them needs a host of its own (its ingress is therefore off by default). `clusterIssuer`, `className` and `annotations` stay global defaults under `ingress.` and are overridable per app.
|
||||
Still open on this count: `ytdlp-web-player`, which reads `.Values.ingress.enabled` outside its Ingress template.
|
||||
- **CI**: either the chart's install-test fixture installs in Gateway mode, or it gets a render-only overlay at `ci/httproute/<chart>.yaml` — otherwise `ci/run-httproute-match-test.sh` is green over nothing for it, which that test fails on. An overlay for a multi-app chart has a second job: it must enter **every** shape the template has, or the branch nobody exercised is the one that breaks. `ci/httproute/freesailarr.yaml` gives one app a host of its own so both the shared-host route and the per-app route render.
|
||||
|
||||
Reference implementation: `template-chart/templates/xxx.httproute.yaml`. Pilot: `cyberchef`. Group 1 — the charts rendering one Ingress with no behaviour-bearing annotation — followed: `bentopdf`, `garage`, `patchmon`, `rallly`, `searxng`, `sure`, `urlaubsverwaltung`, `voucher-vault`. Group 2 — the charts whose behaviour hangs off an nginx annotation — followed: `calibre-web-automated`, `outline`, `paperless-ngx`, `zipline` (all four `proxy-body-size`) and `freesailarr` (`rewrite-target`). That leaves **seven** charts plus `rss-bridge`, which is frozen for changes (#51) and is why #114 cannot be closed while #51 is open: group 3 is the special shapes (`homeassistant` is the documented ingress exception, `keycloak` and `bulwark` have two Ingresses each, `matrix-synapse` renders in a loop, `stalwart` renders per node group, `ytdlp-web-player` hangs off the oauth2-proxy subchart). `garage` was one of the named special cases and is done: it ships `ingress.enabled: false`, so both of its modes are off by default and its route is exercised by a render overlay rather than by the install-test, which keeps its Ingress. `paperless-ngx` was the one chart in the repository that still rendered its Ingress **unconditionally**; group 2 gave it an `ingress.enabled` switch defaulting to `true`, because without a second value path there is nothing for the mutual-exclusion abort to name and the `required()` on `clusterIssuer` would have kept demanding a value the Gateway mode does not use.
|
||||
- **Documented exception — freesailarr, and it extends to the Gateway mode**: the chart bundles several independently exposed apps in one pod, so it has **no** `ingress.domain` — and therefore **no `httpRoute.hostnames`** either, the only chart without that key. A chart-global list of hostnames could not say which of the ten apps it belonged to, so the hostnames stay where they already are, in `ingress.host` and `<app>.ingress.host`; both are read by both modes and neither is an Ingress switch. In Gateway mode the per-app shape renders one `HTTPRoute` per app (`<release>-<chart>-<app>-httproute`) and the shared shape exactly one (`<release>-<chart>-httproute`) with one rule per app — qBittorrent included, since its prefix-stripping rewrite is a per-rule filter there. It offers both shapes. By default each app carries its own `<app>.ingress.enabled` plus a required `<app>.ingress.host` (required only while that app's ingress is enabled and it is not on the shared domain) and renders its own Ingress object `<release>-<chart>-<app>-ingress` with its own TLS secret `tls-<release>-<chart>-<app>-ingress`. Setting the global `ingress.host` switches every app without a host of its own to the shared-domain shape under `<app>.ingress.path`; qBittorrent (no base path, needs a prefix-stripping rewrite) gets the extra object described above, and seerr as well as mediathekarr are pinned to `/` — mediathekarr because its UI links assets with root-absolute paths, which a rewrite cannot fix; with both of them on one shared host the duplicate-path guard fires and one of them needs a host of its own (its ingress is therefore off by default). `clusterIssuer`, `className` and `annotations` stay global defaults under `ingress.` and are overridable per app.
|
||||
|
||||
### 5. Security hardening
|
||||
|
||||
|
||||
Reference in new issue
Block a user