Commit Graph
66 Commits
Author SHA1 Message Date
jklein e54cfe5fdd feat(matrix-synapse): Gateway-API as an alternative to the Ingress
Group 3 of the #114 sweep. TWO HTTPRoutes - and the shape is not the one the
issue described.

#114 says this chart renders its Ingresses in a LOOP OVER SEVERAL APPS. It
does not; that is freesailarr. It renders ONE Ingress object with TWO HOST
rules: the homeserver on ingress.domain at "/", and the delegation nginx on
matrix.delegation.serverUrl at "/.well-known/matrix". An HTTPRoute's
spec.hostnames applies to the WHOLE object, so two hostnames that want
different paths and different backends have to be two objects. That is the
only faithful translation, not a stylistic choice.

httpRoute.delegation.hostnames deliberately does NOT fall back to
httpRoute.hostnames. The server name is what REMOTE homeservers resolve and is
a different thing from the homeserver's own hostname; the migration order -
homeserver host first, server name last - depends on telling them apart, and a
chart-global hostname list covering both would erase the distinction.
httpRoute.delegation.parentRefs falls back to httpRoute.parentRefs, which
matters because the server name is usually an apex domain behind a different
Gateway.

ONE THING THIS MODE IMPROVES, and it is the awkward part of Matrix delegation.
Under ingress-nginx two Ingress objects cannot both claim one hostname - the
admission webhook rejects the second - which is why the delegation rule has to
live inside this chart's single Ingress object even though the apex domain
usually belongs to a website. HTTPRoutes attached to the same listener for the
same hostname are merged by the implementation, longest match winning, so the
/.well-known/matrix route can sit beside whatever owns that apex.

WHAT THIS MODE DROPS, MEASURED ON THE LIVE OBJECT ON 2026-10-12 - and ONE OF
THE TWO IS NOT WHAT IT LOOKS LIKE. The live Ingress carried, all of it
operator-supplied through ingress.annotations (this chart sets none of it):

    nginx.ingress.kubernetes.io/proxy-body-size:    50m
    nginx.org/client-max-body-size:                 50m
    nginx.ingress.kubernetes.io/configuration-snippet: |
      more_set_headers "X-Forwarded-For $remote_addr";
      more_set_headers "X-Forwarded-Proto $scheme";
      more_set_headers "Host $host:$server_port";

1. THE BODY SIZE IS THE REAL LOSS. Synapse's media repository takes uploads;
   its own ceiling is max_upload_size in homeserver.yaml, which this chart
   cannot see because the whole config arrives from 1Password. The effective
   limit is the smaller of the two, and in this mode the proxy half becomes
   the GATEWAY'S default - neither number anyone can point at is necessarily
   the one that decides. No counterpart in the Gateway API, so per decision 2
   of #114 it is documented, not rebuilt. nginx.org/client-max-body-size is
   the same limit spelled for NGINX Inc's controller, a hedge across both
   implementations, and it goes for the same reason. (#114's own measured
   table did not list matrix-synapse under proxy-body-size at all; the live
   object has it.)

2. THE SNIPPET IS NOT LOAD-BEARING, which took checking rather than assuming.
   It looks like what Synapse's x_forwarded setting depends on - client IP for
   rate limiting, and federation. It is not: more_set_headers is the
   headers-more module's OUTPUT-header directive ("Replaces (if any) or adds
   (if not any) the specified output headers"), the request-side directive
   being more_set_input_headers. So those three headers are added to the
   RESPONSE TO THE CLIENT; Synapse never sees them, and what it reads with
   x_forwarded: true is the REQUEST's X-Forwarded-For, which ingress-nginx
   supplies itself. The Gateway API's ResponseHeaderModifier could not rebuild
   it anyway (it knows no nginx variables), and should not: NOTES.txt says in
   as many words not to re-create it and not to read its absence as the cause
   of a rate-limiting or federation problem.

   Stated as what it is - a reading of the directive's documented semantics.
   Nothing here observed a Synapse instance with and without the snippet.

NOTES.txt now asks $published (ingress.enabled OR httpRoute.enabled) wherever
it used to ask ingress.enabled - the server-name/serverUrl mismatch warning,
the "clients work fine" aside and the scaled-to-zero paragraph were all really
asking whether anything publishes the release.

Minor: additive, defaults unchanged. Empty helm template diff against
origin/main with ci/matrix-synapse/test-values.yaml.

Part of #114
2026-10-13 08:59:09 +00:00
jklein 9625d8b182 feat(bulwark): Gateway-API as an alternative to the Ingress
Group 3 of the #114 sweep. One route, one rule, at the root - and THREE
EXPECTATIONS ABOUT THIS CHART DID NOT SURVIVE CONTACT WITH IT:

  - #114 says bulwark has TWO Ingress objects. It has ONE. keycloak has two.
  - #114's measured table lists bulwark under rewrite-target. It has no
    rewrite-target annotation and CANNOT have one: NEXT_PUBLIC_BASE_PATH is
    compiled into the bundle, so the application cannot be served under a
    sub-path, and stripping a prefix at the proxy would yield a page that
    loads and then fetches everything from the wrong place. Every occurrence
    of the string "rewrite-target" in this chart - values.yaml, the schema,
    the Ingress template - is an explanation of exactly that. A grep for the
    string finds the refutations and counts them as occurrences.
  - ci/run-httproute-match-test.sh said in its header that bulwark was the
    second rewrite-target chart and would meet the #118 prefix/redirect
    assurance when group 3 landed. Group 3 landed and it does not: with no
    URLRewrite on a prefix other than "/", the assurance's precondition is
    never met here. The header is corrected rather than left standing, and it
    now says freesailarr is the only chart the pairing applies to.

So no redirect rule is added, and none is needed.

WHAT THIS MODE DROPS, MEASURED ON THE LIVE OBJECT rather than inferred. On
2026-10-12 the live Ingress on webmail.bekemeyer-klein.de carried

    nginx.ingress.kubernetes.io/proxy-body-size: 25m

supplied by the OPERATOR through ingress.annotations - values.yaml recommends
exactly that value and explains why the chart cannot pick it, so the chart
sets nothing and a reader of this repository alone would not find it. It is
the single annotation this installation depends on, and the Gateway API has no
body-size counterpart: implementation policy, not routing. Per decisions 1 and
2 of #114 it is NAMED, not rebuilt - in values.yaml, in the schema and in the
Gateway branch of NOTES.txt, in this application's own terms: an attachment
above the Gateway's default is rejected before Bulwark sees a byte, the 413
comes from the proxy, and it arrives after the user has written the mail and
attached the file.

CORRECTED CLAIM, the same one garage and calibre-web-automated needed: the
Ingress template, the bulwark.publicUrl helper and the schema all stated that
nothing outside the Ingress template FUNCTIONALLY reads ingress.domain. The
HTTPRoute reads it, so the claim held only while both modes were off. All
three now say so, and the route carries a required() of its own.

bulwark.publicUrl follows the ACTIVE mode instead of always printing
ingress.domain. The two differ exactly when httpRoute.hostnames is set, which
is how a migration off the Ingress runs - and printing the old host there
would hand the operator an OIDC redirect URI for the hostname being migrated
away from, a failure that surfaces as a rejected login at the identity
provider and says nothing about this chart.

Minor: additive, defaults unchanged. Empty helm template diff against
origin/main with ci/bulwark/test-values.yaml.

Part of #114
2026-10-13 08:59:09 +00:00
jklein d0f0d49a93 feat(keycloak): Gateway-API as an alternative to the two Ingresses
Group 3 of the #114 sweep, first chart. keycloak has TWO Ingress objects -
the public hostname and the admin hostname - so it gets TWO HTTPRoutes, with
httpRoute.admin.parentRefs falling back to httpRoute.parentRefs exactly as
ingress.admin.className falls back to ingress.className: "which data path" is
a property of the Gateway in this API, not of the route, so an internal
Gateway here is what an internal ingress class is in the other mode.

The admin route stays gated on ingress.admin.enabled, because that value is
not an Ingress switch - it also decides whether KC_HOSTNAME_ADMIN is set on
the Deployment, i.e. whether the admin hostname exists at all. Switching to
Gateway mode therefore must not require turning it off.

FIXED ALONG THE WAY: the admin Ingress did not hang off ingress.enabled.
Measured on origin/main, `helm template -f ci/keycloak/test-values.yaml --set
ingress.enabled=false` emitted kc-keycloak-admin-ingress - so
"ingress.enabled: false" did not mean "no Ingress", and ingress.clusterIssuer
stayed required for an object nobody asked for. It also made Gateway mode
impossible: the admin Ingress would have gone on claiming the admin host next
to the admin HTTPRoute, one host over two data paths, which is the exact
conflict the mutual exclusion exists to prevent. The default render is
unchanged; what changed is a combination that could not be coherent.

WHAT THIS MODE DROPS, MEASURED ON THE LIVE CLUSTER rather than inferred. On
2026-10-12 BOTH of this chart's Ingress objects in jk-cluster carried

    nginx.ingress.kubernetes.io/proxy-buffer-size:    16k
    nginx.ingress.kubernetes.io/proxy-buffers-number: 4

set by the OPERATOR through ingress.annotations and ingress.admin.annotations
- the chart sets neither, which is why reading the chart alone does not reveal
them. The Gateway API has no annotation channel and no counterpart for proxy
buffer sizing; it is implementation policy, not routing. Per decision 1 of
#114 the chart renders no policy object, so this is NAMED and not rebuilt, in
values.yaml, in the schema and in the Gateway branch of NOTES.txt, with the
consequence spelled out in Keycloak's own terms: large headers (JWTs, SAML
assertions, long Set-Cookie chains on the login flow) make a too-small buffer
answer 502 at the PROXY while Keycloak logs the request as successful.

Named and deliberately not touched: four further Ingress objects in the same
namespace (keycloak-realm-*-ingress, with app-root) belong to the Helm release
jk-cluster-config-fleet-cd-keycloak-realm-ingress, i.e. to another repository.
They point at this release's Service under four further hostnames. This chart
renders exactly the two routes its own two Ingresses cover.

Minor: additive, defaults unchanged. Empty helm template diff against
origin/main with ci/keycloak/test-values.yaml.

Part of #114
2026-10-13 08:59:09 +00:00
jklein f202fdc3fc fix(freesailarr): redirect the bare /qbittorrent prefix in Gateway mode
Validate Helm Charts / Detect changed charts (pull_request) Successful in 5s
Validate Helm Charts / Resource quantities (pull_request) Successful in 15s
Validate Helm Charts / Selector match (pull_request) Successful in 14s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 14s
Validate Helm Charts / Build freesailarr (pull_request) Successful in 8s
Validate Helm Charts / Trivy config freesailarr (pull_request) Successful in 22s
Install-test / Install-test freesailarr (pull_request) Successful in 4m19s
Validate Helm Charts / Install-test (pull_request) Successful in 4m19s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 12s
Publish Freesailarr Helm Chart / Publish Freesailarr Helm Chart (push) Successful in 12s
SonarQube Analysis / SonarQube Analysis (push) Successful in 48s
qBittorrent has no base-path setting, so the chart serves it under a prefix
and strips that prefix before the request reaches it. For the slashed form
that is correct. For the BARE form - /qbittorrent, no trailing slash - the
request is rewritten to "/" and qBittorrent answers 200 with its login page,
while the browser's document URL keeps the prefixless form. Every relative
link in that page therefore resolves against the host root, and on the shared
host the root is seerr: the login form comes up unstyled and does nothing
while a different application answers its assets and XHRs. Nothing reports an
error at any level.

The shared-domain HTTPRoute now carries a second rule for qBittorrent: an
Exact match on qbittorrent.ingress.path with a RequestRedirect filter that
replaces the full path with the slashed form, statusCode 301. The Gateway API
ranks an Exact path match above every prefix match, so it wins over both
qBittorrent's own PathPrefix rule and seerr's "/" wherever it is emitted - the
rules' length-descending order stays a rendering convention. 301 rather than
308 because only 301 and 302 are Core in that API; an Extended value an
implementation declines makes it set Accepted=False on the WHOLE route, which
would take the four neighbouring apps' routing down with it, and that would
also contradict decision 1 of Gitea #114. statusCode is written out because it
defaults to 302. The rule is guarded on the app being qbittorrent and its path
not being "/", and it reads the path from the value, never a literal.

The INGRESS BRANCH IS UNCHANGED, deliberately. The remedy there is an
nginx configuration-snippet annotation, and the chart cannot ship it: snippets
are ignored unless the controller runs with allow-snippet-annotations=true,
off by default since ingress-nginx 1.9 and only switched on by accepting
annotation risk level "Critical". A shipped annotation would be a silent no-op
on every cluster keeping the default. The working snippet, that price, and why
a third Ingress object with pathType: Exact is not a substitute - buildLocation
in ingress-nginx controller-v1.14.5 returns a regex location in its
enforceRegex branch before reaching the PathTypeExact branch, and enforceRegex
is computed once per nginx server over every location of the hostname, one of
which carries rewrite-target - are all written out in values.yaml instead.

ci/run-httproute-match-test.sh gains a third assertion, per HTTPRoute object:
a rule stripping a prefix with ReplacePrefixMatch requires a rule in the same
route matching that path Exactly and carrying a RequestRedirect filter. This
is what stops the gap reappearing; bulwark is the second rewrite-target chart
and meets the same check when group 3 of Gitea #114 lands. 37 assertions over
15 charts, up from 36. Both fail paths were exercised: the rule deleted, and
the redirect pointed at a path other than the one stripped.

Minor and not major: with the Ingress branch untouched and httpRoute.enabled
defaulting to false, no existing release's render changes by a byte and no
value becomes required. Shown by an empty helm template diff against
origin/main in both Ingress shapes, shared-domain and per-app hosts.

Fixes #118
2026-10-12 20:06:09 +00:00
jklein bd1fdc09a0 feat: Gateway-API as an alternative to the Ingress in five annotation charts
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 14s
Validate Helm Charts / Selector match (pull_request) Successful in 14s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 14s
Validate Helm Charts / Build freesailarr (pull_request) Successful in 7s
Validate Helm Charts / Build outline (pull_request) Successful in 7s
Validate Helm Charts / Build paperless-ngx (pull_request) Successful in 8s
Validate Helm Charts / Build zipline (pull_request) Successful in 8s
Validate Helm Charts / Build calibre-web-automated (pull_request) Successful in 7s
Validate Helm Charts / Trivy config calibre-web-automated (pull_request) Successful in 19s
Validate Helm Charts / Trivy config freesailarr (pull_request) Successful in 24s
Validate Helm Charts / Trivy config outline (pull_request) Successful in 17s
Validate Helm Charts / Trivy config paperless-ngx (pull_request) Successful in 19s
Validate Helm Charts / Trivy config zipline (pull_request) Successful in 19s
Install-test / Install-test calibre-web-automated (pull_request) Successful in 2m11s
Install-test / Install-test freesailarr (pull_request) Successful in 3m55s
Install-test / Install-test outline (pull_request) Successful in 2m44s
Install-test / Install-test paperless-ngx (pull_request) Successful in 3m52s
Install-test / Install-test zipline (pull_request) Successful in 2m6s
Validate Helm Charts / Install-test (pull_request) Successful in 14m50s
Group 2 of the #114 sweep: the charts whose behaviour hangs off an nginx
annotation - calibre-web-automated, outline, paperless-ngx, zipline
(proxy-body-size) and freesailarr (rewrite-target).

Each gets the form the pilot (cyberchef, PR #115) fixed and group 1 (PR #116)
repeated: an httpRoute block off by default, a template rendering ONLY an
HTTPRoute with parentRefs - no Gateway, no gatewayClassName, no Certificate, no
implementation-specific policy object - render aborts on both modes at once
(naming both value paths), on an empty parentRefs and on a missing hostname,
empty hostnames meaning [ingress.domain], a schema entry closed against unknown
keys with an open parentRef definition, and a NOTES.txt that names the active
mode and what the mode costs.

Minor, additive, defaults unchanged: `helm template` with each chart's CI values
is byte-identical to origin/main for all five.

The upload limit is DOCUMENTED, not rebuilt (decision 2 of the issue). The
Gateway API knows no body size; a policy object would bind the chart to one
implementation. So the Gateway branch of NOTES.txt says, per chart and in the
chart's own terms, that the value is NOT ENFORCED and names where to set it: an
e-book library (calibre), an upload service (zipline), the web-UI half of the
document intake while the consume directory keeps no limit at all (paperless),
and the open #12 pair (outline).

Four findings that were not in the plan, each argued in the file it changes:

  freesailarr gated freesailarr.sharedIngressApps on ingress.enabled, and that
  list is read by the <APP>__SERVER__URLBASE variables as well as by the routing
  objects. In Gateway mode it would have come out EMPTY while the route served
  radarr, sonarr and prowlarr under their prefixes: UrlBase unset, assets and API
  calls linked at the host root, broken UI behind a Ready pod. The helper's two
  guards (seerr/mediathekarr only at "/", no duplicate paths) would have gone
  unenforced there too. Fixed with freesailarr.published (ingress.enabled OR
  httpRoute.enabled) - the same shape as sure.publishedOverTls in group 1, and
  identical to ingress.enabled while httpRoute.enabled is false. Measured: on
  origin/main with ingress.enabled: false the render emits ZERO URLBASE
  variables; with this change in Gateway mode it emits three.

  paperless-ngx was the last chart in the repository rendering its Ingress
  UNCONDITIONALLY. Without an ingress.enabled switch there is no second value
  path for the mutual-exclusion abort to name, and the Ingress plus its
  unconditional required() on clusterIssuer would have kept rendering in Gateway
  mode - a value demanded for nothing, which is issue #114's "Folge 1" and
  README guideline 12. The switch was added, defaulting to true, so the render is
  unchanged.

  paperless-ngx PAPERLESS_URL had no required() of its own: it relied on the
  Ingress template's, which stops firing the moment the Ingress is switchable.
  With httpRoute.hostnames set and ingress.domain empty it would have rendered
  "https://" and refused every login POST as a CSRF failure. It now carries its
  own required(), naming what the value is for.

  calibre-web-automated claimed, in values.yaml and in its schema, that
  ingress.domain is read nowhere but in the Ingress template. The HTTPRoute reads
  it, so the claim now holds only while both modes are off. Corrected rather than
  left standing - the same correction garage needed in group 1.

freesailarr is where the Gateway API is LESS machinery than what it replaces,
not a translation of it. rewrite-target applies per Ingress OBJECT, so
qBittorrent (no base path, prefix must be stripped) needed a second Ingress on
the shared host, which then had to repeat the first object's TLS secret name,
omit the cert-manager annotation conditionally so only one Certificate is
requested, and match an ImplementationSpecific regex path for the rewrite's
capture group. A URLRewrite filter applies per RULE: one route, one rule per
app, the filter on qBittorrent's rule, a plain PathPrefix. All four crutches
gone. It is also the only chart without httpRoute.hostnames - it has no
ingress.domain either, and a chart-global hostname list could not say which of
the ten apps it belonged to, so the hostnames stay in ingress.host and
<app>.ingress.host.

CI: one render overlay per chart under ci/httproute/, so the assertion is not
vacuum-green - ci/run-httproute-match-test.sh goes from 20 held assertions over
10 charts to 36 over 15. freesailarr's overlay moves bazarr onto a host of its
own, so the shared-host route AND the per-app route are both rendered; without
that the per-app branch would be exercised by nothing. The install-test fixtures
stay in Ingress mode: the install runs one configuration per chart, cyberchef
already pays that price against the real Gateway-API CRDs, and for paperless-ngx
that fixture is the only install of the valkey-subchart path anywhere.

README also carries a stale figure fixed here: the HTTPRoute test was still
documented at the pilot's 4 assertions over 2 charts, although group 1 had taken
it to 20 over 10.

Not proven, by construction: that traffic flows. No Gateway controller runs in
CI. Not run locally either: k3d, Docker, Trivy. The Gateway branch of NOTES.txt
is EXECUTED by the render test (a template error in it fails the job, verified)
but its text is printed by no pipeline stage - helm template does not emit
notes, and the install-test installs in Ingress mode.

Part of #114
2026-10-12 18:15:29 +00:00
jklein 33d7cedbab feat: Gateway-API as an alternative to the Ingress in eight charts
Validate Helm Charts / Detect changed charts (pull_request) Successful in 5s
Validate Helm Charts / Resource quantities (pull_request) Successful in 15s
Validate Helm Charts / Selector match (pull_request) Successful in 14s
Validate Helm Charts / Build bentopdf (pull_request) Successful in 9s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 17s
Validate Helm Charts / Trivy config bentopdf (pull_request) Successful in 19s
Validate Helm Charts / Build garage (pull_request) Successful in 8s
Validate Helm Charts / Build patchmon (pull_request) Successful in 7s
Validate Helm Charts / Build rallly (pull_request) Successful in 7s
Validate Helm Charts / Build searxng (pull_request) Successful in 6s
Validate Helm Charts / Build sure (pull_request) Successful in 7s
Validate Helm Charts / Build urlaubsverwaltung (pull_request) Successful in 6s
Validate Helm Charts / Trivy config garage (pull_request) Successful in 18s
Validate Helm Charts / Build voucher-vault (pull_request) Successful in 7s
Validate Helm Charts / Trivy config patchmon (pull_request) Successful in 16s
Validate Helm Charts / Trivy config rallly (pull_request) Successful in 18s
Install-test / Install-test bentopdf (pull_request) Successful in 1m10s
Validate Helm Charts / Trivy config searxng (pull_request) Successful in 22s
Validate Helm Charts / Trivy config sure (pull_request) Successful in 18s
Validate Helm Charts / Trivy config urlaubsverwaltung (pull_request) Successful in 17s
Validate Helm Charts / Trivy config voucher-vault (pull_request) Successful in 19s
Install-test / Install-test garage (pull_request) Successful in 1m17s
Install-test / Install-test patchmon (pull_request) Successful in 1m24s
Install-test / Install-test rallly (pull_request) Successful in 1m39s
Install-test / Install-test searxng (pull_request) Successful in 1m7s
Install-test / Install-test sure (pull_request) Successful in 2m39s
Install-test / Install-test voucher-vault (pull_request) Successful in 2m9s
Validate Helm Charts / Install-test (pull_request) Successful in 13m15s
Install-test / Install-test urlaubsverwaltung (pull_request) Successful in 1m50s
Publish BentoPDF Helm Chart / Publish BentoPDF Helm Chart (push) Successful in 10s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 9s
Publish Garage Helm Chart / Publish Garage Helm Chart (push) Successful in 9s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 9s
Publish Rallly Helm Chart / Publish Rallly Helm Chart (push) Successful in 10s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
Publish SearXNG Helm Chart / Publish SearXNG Helm Chart (push) Successful in 10s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 9s
Publish Sure Helm Chart / Publish Sure Helm Chart (push) Successful in 10s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
Publish Urlaubsverwaltung Helm Chart / Publish Urlaubsverwaltung Helm Chart (push) Successful in 10s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
Publish Patchmon Helm Chart / Publish Patchmon Helm Chart (push) Successful in 9s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
Publish VoucherVault Helm Chart / Publish VoucherVault Helm Chart (push) Successful in 10s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
SonarQube Analysis / SonarQube Analysis (push) Successful in 43s
Group 1 of the #114 sweep: the charts that render exactly one Ingress object
and set no behaviour-bearing nginx annotation - bentopdf, garage, patchmon,
rallly, searxng, sure, urlaubsverwaltung, voucher-vault.

Each gets the form the pilot (cyberchef, PR #115) fixed and template-chart
carries: an httpRoute block (enabled/parentRefs/hostnames, off by default), a
template rendering ONLY an HTTPRoute with parentRefs - no Gateway, no
gatewayClassName, no Certificate, no implementation-specific policy object -
render aborts on both modes at once (naming both value paths), on an empty
parentRefs and on a missing hostname, empty hostnames meaning [ingress.domain],
a schema entry closed against unknown keys with an open parentRef definition,
and a NOTES.txt that names the active mode and what the mode costs.

Minor, additive, defaults unchanged: `helm template` with each chart's CI
values is byte-identical to origin/main for seven of the eight, and for sure
identical in every rendered value (the diff is two YAML comment lines).

Three findings that were not in the plan, each argued in the file it changes:

  sure derived RAILS_ASSUME_SSL/RAILS_FORCE_SSL from ingress.enabled alone.
  Gateway mode sets that false by construction while TLS is still terminated at
  the Gateway, so both flags would have gone false: insecure cookies, no HSTS,
  http:// links, with a Ready pod and a working UI as the only symptom. They now
  read sure.publishedOverTls (ingress.enabled OR httpRoute.enabled), which is
  ingress.enabled while httpRoute.enabled is false - hence still Minor.

  garage claimed, in its Ingress template and its schema, that nothing outside
  that template reads ingress.domain. The HTTPRoute reads it, so the claim now
  holds only while both modes are off. Corrected rather than left standing.

  garage also depends on an upload limit it does not set: its annotations
  comment tells the operator to set proxy-body-size: "0" because ingress-nginx
  truncates S3 uploads at 1 MB. proxy-body-size has no Gateway-API counterpart,
  so its Gateway-mode NOTES prints that the limit is the Gateway's. README's
  list of affected charts named five and now names garage as a sixth.

clusterIssuer needed no change in any of the eight: its only required() sits
inside the Ingress template behind ingress.enabled, the same no-op the pilot
found. Annotation guidance that stops applying in Gateway mode moved into the
Gateway branch of NOTES.txt for bentopdf (X-Robots-Tag) and searxng
(forward-auth).

CI: one render overlay per chart under ci/httproute/, so the assertion is not
vacuum-green - ci/run-httproute-match-test.sh goes from 4 held assertions over
2 charts to 20 over 10. The install-test fixtures stay in Ingress mode: the
install runs one configuration per chart, cyberchef already pays that price
against the real Gateway-API CRDs, and for garage the Ingress is the only place
its Service port is referenced by NAME.

Not proven, by construction: that traffic flows. No Gateway controller runs in
CI. Not run locally either: k3d, Docker, Trivy.

Part of #114
2026-10-12 16:06:18 +00:00
jklein 50f683776e feat(cyberchef): Gateway-API as an alternative to the Ingress (pilot)
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 14s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 13s
Validate Helm Charts / Build cyberchef (pull_request) Successful in 6s
Validate Helm Charts / Trivy config cyberchef (pull_request) Successful in 18s
Validate Helm Charts / Build template-chart (pull_request) Successful in 7s
Validate Helm Charts / Trivy config template-chart (pull_request) Successful in 17s
Install-test / Install-test cyberchef (pull_request) Successful in 1m0s
Validate Helm Charts / Install-test (pull_request) Successful in 1m0s
Validate Helm Charts / Selector match (pull_request) Successful in 12s
The Gateway-API pilot of Gitea #114: one chart, the convention, the CI that
holds it. The remaining 20 charts follow in their own PRs.

The form, additive and off by default:

    httpRoute:
      enabled: false     # counterpart to ingress.enabled
      parentRefs: []     # the Gateway - a platform object
      hostnames: []      # empty = [ingress.domain]

Only HTTPRoute with parentRefs is rendered: no Gateway, no gatewayClassName,
no Certificate, and no implementation-specific policy object. That subset is
identical on nginx-gateway-fabric, Cilium, Envoy Gateway and Istio, which is
what lets the implementation be chosen later without touching the charts
again. Class and certificate move to the platform.

Four guards, all against silent failures:

  - mutually exclusive with ingress.enabled, with a render abort naming BOTH
    value paths. A migration runs over a second hostname, not in parallel.
  - parentRefs mandatory in Gateway mode, render abort on the empty list: a
    route without a parent is created, is schema-valid, and routes nothing.
  - hostnames empty means [ingress.domain], so the hostname has one source.
  - the mode is a VALUE, not auto-detection: .Capabilities.APIVersions is
    unreliable at template time and a missing CRD fails at install time.

ingress.clusterIssuer was already only read inside the Ingress template, so
the "conditionally required" half of #114 needed no change in this chart -
checked rather than assumed, and it has to be re-checked per chart in the
sweep.

CI, both halves, because neither covers the other:

  - the install-test installs cyberchef in GATEWAY mode against the REAL
    Gateway-API CRDs (standard channel v1.6.3, pinned and Renovate-tracked,
    applied --server-side). A schema-less stub would accept a malformed route
    and prove only that something was created. The Ingress path of this one
    chart is no longer installed; it is a built-in type that never routed in
    k3d anyway, and helm lint renders this chart's defaults - Ingress mode -
    on every PR.
  - ci/run-httproute-match-test.sh, over every chart on every PR: every
    backendRefs entry must name a Service, on a port, that the SAME render
    creates, and parentRefs must not be empty. The CRDs accept both of those
    mistakes, so the schema cannot catch them, and the only report would be a
    route status no controller writes in CI.

What stays unprovable, and is written down instead of implied: that traffic
flows. No Gateway controller runs in k3d.

Minor: additive with unchanged defaults, proven by an empty helm template
diff against origin/main for the default values.

Part of #114
2026-10-12 15:16:26 +00:00
jklein 6ade74db80 feat(stalwart): spread each node group over nodes and give each one a PDB
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
Publish Stalwart Helm Chart / Publish Stalwart Helm Chart (push) Successful in 7s
SonarQube Analysis / SonarQube Analysis (push) Successful in 27s
A two-replica node group was redundancy on paper: nothing kept both pods off
one node, and no PodDisruptionBudget kept a drain from taking both at once.
Both objects are now rendered per node group, gated on that group's COMPUTED
replica count being >= 2.

The distribution is a promise, not an effort: maxSkew 1 on
kubernetes.io/hostname with whenUnsatisfiable DoNotSchedule, selecting only
this group's own pods, so an "mx" and a "full" pod may share a node. The
selector comes from the chart (stalwart.groupLabels), never from the operator -
a spread constraint whose selector matches no pod is trivially satisfied,
distributes nothing and reports nothing.

nodeAffinityPolicy: Ignore is set explicitly, and that is the load-bearing
detail. Its API default is Honor, under which a narrowing nodeSelector shrinks
the set of counted domains until both replicas fit into one: skew 0, both pods
co-located, no Pending, no event, and a selector that is correct - so no check
can see it. With Ignore the same narrowing gives skew 2 and the second pod is
Pending. A render abort was the alternative and would also have forbidden a
nodeSelector spanning five nodes, which is legal.

A user entry colliding with the chart's own on (topologyKey,
whenUnsatisfiable) aborts the render naming both values, because the API
rejects the duplicate with a message about a list index and an apply-based path
merges the two over patchMergeKey: topologyKey silently, dropping one selector.

Minor rather than major, proved with a diff rather than argued: with the
shipped defaults (full: 1, mx: 0) and with ci/stalwart/test-values.yaml,
helm template renders byte-for-byte what 1.0.1+upv0.16.24 rendered - both
diffs empty.

New in CI, over every chart on every PR: ci/run-selector-match-test.sh holds
every rendered PodDisruptionBudget selector and every topologySpreadConstraints
labelSelector against the labels of the pod templates in the same render. Helm
itself is the YAML parser (ci/yaml2json), because the runners have neither
pyyaml nor yq and a hand-written parser could read nothing and report nothing.
podAffinity/podAntiAffinity are out of scope; matchExpressions-only selectors
are reported as not checked rather than counted as passing. Two anti-vacuum
guards: a chart with an overlay under ci/selector-match/ that renders no
selector fails, and a run that checks no selector at all fails.

Fixes #96
2026-10-07 19:36:04 +00:00
jklein 6e196c26a5 test(ci): restore the theme mount - the A/B shows the restart is pre-existing
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 14s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 12s
Install-test / Install-test keycloak (pull_request) Successful in 3m10s
Validate Helm Charts / Install-test (pull_request) Successful in 3m10s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
Publish Keycloak Helm Chart / Publish Keycloak Helm Chart (push) Successful in 7s
SonarQube Analysis / SonarQube Analysis (push) Successful in 28s
2026-10-06 11:43:46 +00:00
jklein 8cd0c14e3f test(ci): control run 598 recorded (81s, 0 restarts)
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 13s
Validate Helm Charts / Build keycloak (pull_request) Successful in 9s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 16s
Install-test / Install-test keycloak (pull_request) Successful in 9m24s
Validate Helm Charts / Install-test (pull_request) Successful in 9m24s
2026-10-06 11:29:44 +00:00
jklein 71cc545277 test(ci): A/B control run - keycloak install-test without the theme mount (temporary)
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 8s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 10s
Install-test / Install-test keycloak (pull_request) Successful in 2m14s
Validate Helm Charts / Install-test (pull_request) Successful in 2m14s
2026-10-06 11:22:26 +00:00
jklein d7067abfab docs(ci): the whole restart series, intermittent on an unchanged spec
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 9s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 10s
Install-test / Install-test keycloak (pull_request) Successful in 2m45s
Validate Helm Charts / Install-test (pull_request) Successful in 2m45s
2026-10-06 11:13:46 +00:00
jklein 5e47191252 docs(ci): three runs, and the load explanation does not hold
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 9s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 13s
Install-test / Install-test keycloak (pull_request) Successful in 3m59s
Validate Helm Charts / Install-test (pull_request) Successful in 3m59s
2026-10-06 11:04:16 +00:00
jklein 34c600f7a8 docs(ci): the second run settles it - the keycloak restart was the runner, not the pod spec
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 10s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 10s
Install-test / Install-test keycloak (pull_request) Successful in 4m2s
Validate Helm Charts / Install-test (pull_request) Successful in 4m2s
2026-10-06 10:54:48 +00:00
jklein 1cf59f5448 docs(ci): record the keycloak install-test run that was slower and restarted once
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 11s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 9s
Install-test / Install-test keycloak (pull_request) Successful in 2m18s
Validate Helm Charts / Install-test (pull_request) Successful in 2m18s
2026-10-06 10:46:57 +00:00
jklein d8b392bf64 feat(keycloak): extraVolumes/extraVolumeMounts for custom themes and providers (1.1.0+up26.8.0)
Validate Helm Charts / Detect changed charts (pull_request) Successful in 11s
Validate Helm Charts / Resource quantities (pull_request) Successful in 15s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 17s
Install-test / Install-test keycloak (pull_request) Successful in 3m9s
Validate Helm Charts / Install-test (pull_request) Successful in 3m9s
2026-10-06 10:26:13 +00:00
jklein b39f36ff5c feat(bulwark): new chart for Bulwark Webmail (1.0.0+up1.12.0)
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
Publish Bulwark Helm Chart / Publish Bulwark Helm Chart (push) Successful in 7s
SonarQube Analysis / SonarQube Analysis (push) Successful in 27s
A JMAP webmail client for the stalwart chart (Gitea #65): StatefulSet, one PVC
at /app/data, Service plus headless Service, Ingress at a host root, two
OnePasswordItems, three probes, computed limits, replicas plus scaleDown, strict
values.schema.json.

Everything below was measured against the source at tag 1.12.0 rather than read
out of the documentation, which was wrong about three of them.

THE DEFINING PROPERTY: THE VOLUME BEATS THE CHART, FOR EVERY VALUE, SILENTLY.
configManager.get resolves "admin override > env var > default", the admin
dashboard may write any key of CONFIG_ENV_MAP, and nothing reports that it just
disconnected the chart - so a helm upgrade can succeed and change nothing.
Upstream documents this for the server URL only; it is true of everything.
Exactly three values escape it (SESSION_SECRET(_FILE), ADMIN_PASSWORD,
BULWARK_TELEMETRY). The value that closes it, bulwark.adminConfigReadOnly,
defaults to false because its price is threefold: it freezes the settings policy
(and with it the only way to move Web Push off the vendor relay), it freezes an
override that ALREADY exists instead of removing it, and on a fresh install it
stops admin.json from being written at all - so the admin dashboard never comes
up and its login answers HTTP 500 while the pod stays Ready.

OIDC-only sign-in, with the real way back out of a lockout: not helm upgrade
with OAUTH_ONLY=false (oauthOnly goes through the same precedence chain) but
removing the override in the admin dashboard. That is why ADMIN_PASSWORD is set
- the dashboard is the door. It generates no random password and logs nothing,
so the #65 analogy does not apply.

SESSION_SECRET_FILE and OAUTH_CLIENT_SECRET_FILE are both used - the better path
garage (#81) and keycloak (#83) did not have - in one projected volume mounted
WITHOUT subPath, because both are re-read on every use and a subPath mount is
never refreshed, and at mode 0440 because secret files are owned by root and
fsGroup only supplies the group.

Two backup statements, both at the value: without the session secret the
synchronized settings are unreadable out of an intact volume, and if the
dashboard's OAuth setup was ever used the client secret sits in config.json on
that volume in cleartext.

Probes: all three a plain GET on /api/health, with the limit written at the
readiness value - it carries process, HTTP acceptance, a free event loop and
Next.js routing, and nothing about the reported state or about Stalwart.

No ConfigMap, so the checksum/config rule (#82) has nothing to attach to, and
values.yaml says so explicitly; the op:// paths are covered by
checksum/secret-ref.

Renovate: no versioningTemplate, measured rather than assumed. 315 tags in eight
shapes (releases, a second v-prefixed family of the same images, an -always
flavour, floating minors and names, commit SHAs, betas, attestations) and plain
docker versioning excludes all six unwanted ones. Dry runs at 1.12.0, 1.11.0 and
1.10.0 are recorded in the block.

Part of #89 - deliberately not "Fixes", because one binding decision of the
issue was changed out of necessity and wants ratification rather than an
automatic close: the plan specifies HEAD /api/health for all three probes, and a
Kubernetes probe cannot send HEAD at all (HTTPGetAction has no method field; the
kubelet always issues GET). The alternative, an exec probe, would mean starting a
Node process every few seconds in an image that carries no curl and no wget. GET
is used instead and is marginally more capable than HEAD, so every limit the
issue argues is unchanged - but the decision is the maintainer's to confirm.
2026-10-06 10:15:27 +00:00
jklein 320257c742 fix(keycloak): readiness could not fail on the database, and the measured budget
Two corrections found while verifying the first CI run, not from the plan.

THE DEFAULT THAT MADE THE READINESS PROBE TOOTHLESS. Keycloak's DATABASE health
check is the one row of upstream's "Available Checks" table whose "Requires
Metrics" column says yes; Cluster, Graceful shutdown and Keycloak Initialized
say no. So with metrics-enabled=false - this chart's default until now -
/health/ready answers 200 while the database is gone, and the pod keeps taking
logins it cannot serve. The probe path was right and the aggregate behind it
contained nothing external, which is the urlaubsverwaltung defect (#218) in a
new hiding place: not a health group this time, but an option named for
observability, where nobody looks when reviewing a probe.

keycloak.metrics.enabled therefore defaults to TRUE, and its comment argues
about readiness rather than about scraping. The price is stated at the value:
/metrics then answers on the management port, unauthenticated, carrying realm
names, client ids and login counters - on a port that is ClusterIP-only and in
no Ingress. Switching it off is now documented as a decision about whether
readiness can fail at all. The CI fixture keeps it on and says why, so a green
run really does touch the database.

THE STARTUP BUDGET NOW CARRIES ITS MEASUREMENT. From the install-test of PR #87
on the Gitea docker runner (k3s v1.35.5, two replicas, empty database, image
pulled during the run): 71.8 seconds from `helm install` to BOTH pods Ready -
first at ~70.8s, second one second later, 0 restarts. The 600s budget stays,
with the reason for the headroom written down: everything in that run was the
cheap case, and a first install against a real database serializes the second
replica behind the first's Liquibase migration lock instead of letting it
arrive a second later.

AND AN OVERCLAIM OF MY OWN, REMOVED. ci/keycloak/test-values.yaml said a green
run proves jdbc-ping found the second replica. It does not: readiness reports
one node's own health, so a node that never found its peer is Ready, healthy
and alone - and two such nodes are two islands that each believe they are the
cluster. What the test does rule out is the louder half, a replica that cannot
start or cannot share the database. The Infinispan view has to be read out of
the log, which is what step 2 of NOTES.txt tells the operator to do, now with
the note that no probe and no install-test replaces that step. Measurement
point 4 of the plan therefore stays open.

Part of #83
2026-10-04 18:00:20 +00:00
jklein 4c1cff1744 feat(keycloak): new chart for Keycloak (1.0.0+up26.8.0)
Two replicas with cache=ispn, clustered through Keycloak's default jdbc-ping
stack: discovery runs over the PostgreSQL that is there anyway, so no headless
Service and no JGroups wiring. Deployment rather than StatefulSet - Keycloak 26
keeps everything in the database, user sessions included.

The chart's defining property is that it has TWO INIT CONTAINERS and needs
both. db, health-enabled and metrics-enabled are Keycloak BUILD options
(verified in the 26.8.0 option table), the published image is augmented with
their defaults, and supplying different ones at runtime makes Keycloak rewrite
/opt/keycloak/lib/quarkus at start - which readOnlyRootFilesystem: true forbids
and which upstream describes as a start that "creates an image that is
mutable". Not setting db is no way out: the probes alone force it, because
health-enabled is a build option too. An empty emptyDir over that path fails as
well, because QuarkusEntryPoint loads quarkus/quarkus-application.dat before it
dispatches any command, `build` included. So one init container seeds the
directory into a volume, the next runs `kc.sh build` into it, and the serving
container runs `start --optimized` - every container keeping the full hardening
baseline, with no documented deviation to carry.

Build options therefore exist in exactly one place, and the asymmetry is
enforced rather than trusted: the serving container passes no build option at
all, and a build option's variable name in keycloak.env fails the render with a
message naming it.

Hostname handling is the security-relevant part. KC_HOSTNAME is derived from
ingress.domain as a full URL (which upstream requires alongside hostname-admin),
hostname-strict stays at its default true, and proxy-headers is set rather than
optional. The admin hostname is NOT presented as an access control: upstream
states that hostname-admin "does not prevent accessing the Administration REST
API endpoints via the frontend URL specified by the hostname option", so the
chart renders no /admin deny rule and values.yaml, NOTES.txt and the wiki all
say in the same words that restricting that API is reverse-proxy work.

The management port 9000 is a containerPort and a ClusterIP Service for
in-cluster scraping, and is in no Ingress: it serves /health/* and optionally
/metrics without authentication. Liveness uses /health/live and deliberately
not /health or /health/ready - those include the database, and a liveness probe
on them would restart the whole cluster for as long as the database is away.

Also in this change:
- First PodDisruptionBudget in this repository, which makes it a convention
  (README section 11): minAvailable as a value, and rendered only while the
  computed replica count is >= 2, because minAvailable: 1 with one replica
  blocks every node drain forever with a message that names the PDB rather
  than the replica count.
- First quay.io image, so quay.io joins the trusted registries in
  ci/trivy/data/ksv0125.yaml - configured rather than suppressed, so
  AVD-KSV-0125 keeps being able to say something.
- Renovate block with NO version constraint, which is the counter-case to
  garage and stalwart and is measured rather than assumed: docker versioning
  already excludes the -N rebuild tags, the floating minors and latest/nightly.
  Three dry-run fixtures are recorded at the block, and against the committed
  config 26.7.4 offers exactly 26.7.5 (patch) and 26.8.0 (minor).
- No automerge exception, by decision (Gitea #83): the global automerge applies,
  and the minor-upgrade procedure plus `kc.sh update-compatibility` is
  documented in NOTES.txt instead.
- Two upstream defaults deviated from, each with its reason at the value:
  db-pool-max-size 20 instead of 100 (two replicas at 100 ask a default
  PostgreSQL for twice its max_connections), and uid 1000 / gid 0 instead of
  this repo's 20000/1000, because the image declares keycloak:x:1000:0 and
  chowns its install tree to that pair.

Part of #83
2026-10-04 18:00:20 +00:00
jklein 79b7e44fdc feat(garage): new chart for the Garage object store
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 11s
Validate Helm Charts / Build garage (pull_request) Successful in 6s
Validate Helm Charts / Trivy config garage (pull_request) Successful in 16s
Install-test / Install-test garage (pull_request) Successful in 1m15s
Validate Helm Charts / Install-test (pull_request) Successful in 1m15s
One Garage node with replication_factor = 1, built from template-chart to the
style guide rather than copied from upstream's in-tree chart (Gitea #81). The
single-node trade is deliberate and its price is named at the value: the factor
applies to the metadata as well, where upstream calls a corrupted database
"generally not recoverable" and a replicated volume replicates the corruption
faithfully. The three countermeasures that work at factor 1 are therefore
mandatory, not optional - metadata_fsync = true (off in Garage's defaults), the
automatic metadata snapshots, and those snapshots on a second PVC.

Storage: PVC 1 carries metadata_dir AND data_dir as two subdirectories, because
volume snapshots are atomic per volume and that is what keeps index and blocks
consistent with each other; PVC 2 carries metadata_snapshots_dir, because a
recovery point must not die with the volume whose corruption it should repair.
Garage creates all three directories itself, which matters because the image is
FROM scratch and has no mkdir.

Two corrections to the approved plan, both verified against the source at tag
v2.4.1 and decided on the issue before implementation:

- The RPC secret arrives as GARAGE_RPC_SECRET via secretKeyRef, not through
  rpc_secret_file. The file form cannot work in a non-root pod:
  read_secret_file rejects any mode with a group or other bit set
  (src/garage/secrets.rs:144, mask 0o077) while the kubelet OR-s the group-read
  bit onto every file of a secret volume once fsGroup is set
  (pkg/volume/volume_linux.go), and Kubernetes can set the group of such a file
  but never its owner. The mechanism is documented at the value so the next
  round does not "fix" it back.
- The first start is done by the server, not by hand: garage server
  --single-node creates the layout in-process before any listener opens, is
  idempotent on the layout version and has no --version argument that could be
  reused, and --default-access-key/--default-bucket import key and bucket FROM
  the vault item. Both premises of the original "manual first start" decision
  fell with that.

Probes follow from the same source reading: readiness is GET /health on the
admin port, which answers 503 while no layout exists; liveness and startup are
a TCP connect, because a probe that can only fail would restart a node waiting
for its layout forever (same lesson as zipline, README 12.3). The S3 listener
binds independently of the layout, so a TCP probe on 3900 would have been the
lie the plan warned about.

CI: fixtures under ci/garage/ rather than an exclusion - with --single-node the
pod reaches Ready on its own, which makes the install-test an assertion about
the layout, the one-PVC subdirectory layout and readOnlyRootFilesystem instead
of a formality. The Renovate block carries a regex versioning anchored to
^v<major>.<minor>.<patch>$, because the registry receives 40-character
commit-SHA tags almost daily (measured 2026-10-03: the newest 22 tags are all
of that kind) and the four v1.99.x-internal builds sort above the v1.3.x line.

Part of #81
2026-10-03 20:29:38 +00:00
jklein a9ca5302de feat(paperless-ngx): optional valkey subchart, password from the redis item
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 12s
Validate Helm Charts / Build paperless-ngx (pull_request) Successful in 6s
Validate Helm Charts / Trivy config paperless-ngx (pull_request) Successful in 22s
Install-test / Install-test paperless-ngx (pull_request) Successful in 3m31s
Validate Helm Charts / Install-test (pull_request) Successful in 3m31s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
Publish Paperless-NGX Helm Chart / Publish Paperless-NGX Helm Chart (push) Successful in 10s
SonarQube Analysis / SonarQube Analysis (push) Successful in 33s
paperless-ngx cannot run without a Redis - PAPERLESS_REDIS is the Celery
broker - so every installation needed a second Helm release whose address
lived in a 1Password item and in neither chart. This adds our own published
valkey chart as an optional dependency, off by default and gated on
valkeySubchart.enabled (not valkey.enabled, which the subchart's strict
schema rejects and which Helm would only have complained about for whoever
first switched it on). Additive, default behaviour unchanged: minor,
12.1.3+up3.2.1 -> 12.2.0+up3.2.1.

Configured as a QUEUE (PVC + noeviction, the subchart's defaults), unlike
stalwart's cache. Read off the 3.2.1 source rather than assumed: one
PAPERLESS_REDIS url serves five consumers - the Celery broker, the result
backend (Redis in 3.x, the database before), the channels pub/sub layer and
two Django caches. allkeys-lru would not be a trade here but actively wrong,
because LRU evicts typeless across the whole keyspace and a backlog of
waiting consume tasks is exactly what it reads as cold; a dropped task is a
document silently never processed. The price, named in values.yaml rather
than smoothed over: two of the five consumers are caches, so a full instance
refuses writes - new tasks included - and the answer is memory sizing, not a
policy change. All three cache-like consumers are TTL-bounded, which is what
keeps that price small.

The password comes from the SAME item as the Redis connection, in a third
field auth-conf holding the complete line "requirepass <password>". It cannot
DEFAULT to secrets.redisSecretRef, and that is a property of Helm, not an
omission: a values.yaml is never templated and a parent cannot compute a
subchart's value. Measured - a template expression there lands verbatim in
the subchart's itemPath and fails as a YAML parse error naming neither the
value nor the cause.

Forgetting it is therefore caught in values.schema.json, by a conditional
rule rather than a fail() in a template. Also measured, and the reason:
Helm renders a subchart's templates BEFORE the parent's, so a guard in this
chart loses the race to the subchart's own required(), which reports
"secrets.authSecretRef" and cannot know the path belongs to the redis item.
A check that cannot fire is the mistake found while testing #77, so the
first draft of that guard was removed rather than shipped. The schema runs
before any rendering and names /valkey/secrets/authSecretRef.

One trap found while testing and closed here: "valkeySubchart: null" DELETES
the key, Helm then cannot resolve the condition path and treats the
dependency as ENABLED - so erasing the switch turned the subchart on with no
password set. Unlike every other block in this chart (issue #99), that one
is rejected by the schema instead of defaulting.

The install-test now runs with the subchart ON and the throwaway redis
Deployment removed: the url in the fixtures names ci-valkey-service, so the
bundled Valkey is the broker the release actually uses. #65 covered this
path by a render only - deliberately, since an unused Valkey would have made
that test look like it covered clustering. This is the first real
installation of the valkey-subchart path.

Documented, not glossed: the password now sits in that item twice and a
rotation has to touch both fields; with the subchart on the vault url is a
mandatory step, not a hint; and a OnePasswordItem materializes every field
of its item, so both Secrets carry all three keys - untidy, harmless, and
written down so nobody "tidies it up".

Fixes #78
2026-10-02 21:24:22 +00:00
jklein 780cb8eafd feat(stalwart): new chart for the Stalwart mail server
Cluster-capable from the start, with roles rather than symmetry: one
StatefulSet per entry of the "nodes" map, the key passed in STALWART_ROLE.

Since v0.16 Stalwart keeps its whole configuration inside the data store and
manages it over JMAP, so the chart's scope is deliberately narrow: it renders
config.json (exactly the DataStore object), the workloads, the Services, the
NetworkPolicies, the Ingress and a cert-manager Certificate. Listeners, blob
store, in-memory store, TLS object and cluster roles are operator work in the
WebUI, which is why NOTES.txt carries an eight-step first install.

config.json stays secret-free: authSecret uses the EnvironmentVariable variant
(verified against the serde definition of PostgreSqlStore at v0.16.24), so the
password arrives through secretKeyRef and checksum/config stays meaningful.
The username cannot follow it - authUsername is a plain String with no secret
form - so it is a public value and says so.

Services are per PORT, not per group: a NodePort is cluster-unique, so "full"
and "mx" can only accept SMTP behind one Service, selected through a per-port
pod label. The http port gets a per-group ClusterIP Service instead, so the
Ingress reaches exactly ingress.nodeGroup.

Four traps, documented rather than explained away:

- The role lookup FAILS OPEN. A node whose ClusterRole is missing from the
  database enables every listener and every task while staying Running with a
  green probe, and the log line arrives after the listeners are bound. Hence
  mx at replicas 0, per-port Services, and a NetworkPolicy that is a ceiling
  independent of what was bound.
- NET_BIND_SERVICE is a documented deviation needed to EXEC, not to bind. The
  image sets setcap cap_net_bind_service=+ep, which makes the kernel fail
  execve when the bounding set is empty. The values comment says both why it
  is there and that it is never used, because the second half is what stops
  the next hardening pass from removing it.
- externalTrafficPolicy must be Cluster - no pod runs on a control-plane node
  while the HA proxy delivers to them, so Local would drop every packet - and
  the source address that costs is restored only by the HA proxy's PROXY v2
  header.
- /healthz is rate-limited to 100 requests/60s per source IP, which is a node
  IP for kubelet probes, so all pods on a node share one bucket. The probe
  intervals are calculated (6 req/min/pod, ~16 pods per node) and the sum is a
  comment in values.yaml.

A first install needs bootstrap.enabled: true: the setup wizard appears only
when config.json is ABSENT, and this chart always renders one, so recovery
mode is upstream's documented path for a templated config.json.

Second chart in the repo with a dependency - the Valkey project's own chart,
off by default behind condition:, because Stalwart's cluster coordinator rides
on the in-memory store connection. The render aborts when more than one pod
runs with no in-memory store address anywhere.

Also wired up: publish-stalwart.yml through the shared block in
workflow-util/pipelines, the change-app-version.yml choice list, a Renovate
customManagers block anchored on the v prefix (which excludes the moving
latest / v0.16 / -alpine tags), CI fixtures, and the README and CLAUDE.md
lines that claimed ytdlp-web-player was the only chart with a dependency.

Verified locally: ci/helm-dependencies.sh, helm lint --strict (0 failed),
helm package, helm template against chart defaults and against
ci/stalwart/test-values.yaml, the rendered config.json parsed as JSON with the
field types the serde definition requires, ci/run-quantity-render-test.sh
(0 failed, both nodes.*.resources paths exercised), NOTES.txt rendered through
a scratch ConfigMap, and the null-safety and fail paths (securityContext=null
keeps NET_BIND_SERVICE; a cluster without an in-memory store, an unknown port
name, a group without http and an unknown ingress.nodeGroup all abort with a
named message; the schema rejects timeout, a string timeoutMs and a list-shaped
nodes). NO runtime verification was possible - there is no k3d or Docker daemon
in this workspace.

Part of #65
2026-10-02 20:59:17 +00:00
jklein 6a3b05503e Make the all-charts install-test repeatable, as one shared job
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 10s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 21s
The install-test matrix follows the charts a pull request changes, so a
chart nobody has touched since the forge move carries a green history from
a different runner - and that green says nothing about today. Gitea PR #48
showed what the answer is worth: correcting a comment in 21 charts made
every one of them "changed" and turned the run into the first all-charts
install-test after the move, 19 of 20 matrix charts Ready in 12m41s and
exactly one not (rss-bridge, Gitea #51). By accident.

The install-test job moves into the reusable .github/workflows/
install-test.yml and is called twice: by validate-charts.yml with the
charts a pull request changed, and by the new install-test-all-charts.yml
with all of them. Not copied for the second caller - a second copy is a
second chance to test differently, unnoticed, which is the argument that
gives ci/excluded-charts.txt exactly one parser. The parts that would
drift are the ones that were paid for once already: the pinned kubectl
with its skew check, envsubst, the containerised-job detection, the
per-leg cluster name.

Nothing about how a chart is installed changed. The moved job differs from
its predecessor in the matrix source (inputs.charts instead of the detect
job's output), an id on the install step, and a step that records the
result; every other step is byte-identical.

The all-charts run is workflow_dispatch only - a schedule would spend
compute every day on a question that changes rarely - with an optional
subset input, which is what lets a single chart be tested against the
runner as Gitea #51 needs. One throwaway cluster per chart, no reuse:
reuse saves time and changes what the test proves. An unknown or excluded
chart name, and an empty matrix, are loud errors rather than a green run
over nothing.

Every leg writes its own outcome into the job summary next to the four
facts that decide what the outcome means - date, runner label, K3S_IMAGE,
kubectl version - and a final job records the verdict over the matrix. A
green run nobody notes is unknown again next time. The excluded charts
appear with their justification, from the one parser, as in the pull
request pipeline. 20 charts in 12m41s is written down as the expectation,
so a later deviation is visible as one.

Fixes #52
2026-09-27 14:30:39 +00:00
jklein 813c796a7b feat(valkey): one Valkey instance per application (1.0.0+up9.1.2.alpine)
Validate Helm Charts / Resource quantities (pull_request) Successful in 8s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Successful in 4s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 11s
Validate Helm Charts / Install-test ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }} (pull_request) Successful in 1m22s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
Publish Valkey Helm Chart / Publish Valkey Helm Chart (push) Successful in 7s
A new chart for a single Valkey instance per application, replacing the
shared cluster-redis whose separation rests on one ACL file in one
1Password field - a construction in which every change touches six
applications and which cost a month of Paperless standstill.

Shape: StatefulSet with one replica, ClusterIP Service, no cluster mode,
no Ingress and no value to add one.

The password arrives as a finished configuration line ("requirepass
<secret>") in the 1Password field auth-conf, is mounted as a file and
included by the main configuration - never in argv, never in the
environment. The include is rendered last, so nothing in extraConfig can
override it. Because a misspelled or empty field would leave an open
server that looks perfectly healthy, an init container greps the mounted
file for the line and refuses to start the pod without it; it never
prints the file.

maxmemory is derived from the computed memory limit (maxmemoryFactor,
default 0.75) and a configured value above that limit aborts the render:
otherwise the kubelet's OOM kill decides what the eviction policy was
configured to decide. Persistence and eviction are documented as pairs -
queue (PVC + noeviction, the default) and cache (emptyDir +
allkeys-lru) - and the combination "no persistence + noeviction" carries
a warning in values.yaml and in NOTES.txt.

Probes: valkey-cli ping as an exec probe, authenticated through
VALKEYCLI_AUTH read out of the mounted file, with "-e" and an assertion
on the answer PONG. Without -e valkey-cli exits 0 on an error reply
(valkey 9.1.2, src/valkey-cli.c: the config.set_errcode branch in
cliReadReply), which would make the probe unable to fail.

Hardening: uid 999 / gid 1000 - the image's own valkey user - with
fsGroup matching, runAsNonRoot, readOnlyRootFilesystem, all capabilities
dropped and seccompProfile RuntimeDefault in the pod context so the init
container is covered too. /data is the only writable path.

NetworkPolicy on by default (first chart in this repo with one), with a
switch and a list of additional namespaces; values.yaml names that it is
only a boundary where the CNI enforces policies.

Wiring: publish-valkey.yml, the chart-name choice list, ci/valkey
fixtures for the persistent variant, a plain docker-versioning Renovate
block, README chart row.
2026-09-27 11:24:55 +00:00
jklein d9267b2180 sure: fix the readiness probe, the ephemeral-storage budget and HOME
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 6s
Publish Sure Helm Chart / Publish Sure Helm Chart (push) Successful in 6s
Second install-test run, three findings, all measured:

- Readiness on "/" failed with "http: server gave HTTP response to HTTPS
  client" on a pod that was answering correctly. The kubelet's HTTP prober
  follows redirects to the same host, and with RAILS_ASSUME_SSL=true the
  app builds its redirect as https://<pod-ip>:3000/registration/new, which
  Puma does not speak. The probe now runs curl without -L and accepts
  2xx-3xx itself. The same log also proves what the probe is for: against
  an empty database the redirect target was /registration/new, the branch
  User.count decides, so the path really does reach Postgres.
- The pod was evicted within two minutes of every start: "ephemeral local
  storage usage exceeds the total limit of containers 60Mi". emptyDirs and
  the stdout log both count there, and the image rebuilds its Bootsnap
  cache under /rails/tmp on every start because the emptyDir hides the one
  baked into the image. The request is now 256Mi (limit 768Mi) per app
  container - headroom, not a measurement.
- /home/rails is read-only, so Bundler created a throwaway home under
  TMPDIR on every invocation, worker probe included. HOME now points at a
  directory the init container creates.

The CI postgres fixture gets -d sure on its readiness probe, so the server
stops logging a missing database that nothing asked for.
2026-09-26 21:08:55 +00:00
jklein 93fdb3eaea sure: new chart for the we-promise Sure app (1.0.0+up0.7.4)
One StatefulSet with two containers - the Puma web process and a Sidekiq
worker - sharing one ReadWriteOnce PVC at /rails/storage. That shared
volume is the reason they live in one pod: splitting them would need an
RWX volume. Only the web container migrates, because the image entrypoint
runs db:prepare solely for the './bin/rails server' command; the worker is
started through args: so the entrypoint (and its jemalloc preload) stays
in place and the two can never race.

Probes are chosen for what they prove: liveness and startup use Rails'
/up, which is a boot check and touches no database, while readiness uses
'/', which runs User.count through authenticate_user! in self-hosted mode
and therefore fails while Postgres is unreachable. The worker speaks no
HTTP, so all three of its probes ask sidekiqmon whether this pod still
holds a Sidekiq heartbeat in Redis.

Three OnePasswordItem CRs (app, database, redis), ingress.domain kept
required outside the Ingress because APP_DOMAIN and the WebAuthn relying
party read it, strict values.schema.json, CI fixtures with throwaway
Postgres 16 and Redis, and a Renovate manager on semver so the
-alpha/-rc/-hotfix channel and the moving tags are never selected.
2026-09-26 21:08:55 +00:00
jklein ae621746a1 Merge main into the MediathekArr branch
Validate Helm Charts / Resource quantities (pull_request) Successful in 7s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Successful in 4s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 11s
Validate Helm Charts / Install-test ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }} (pull_request) Successful in 3m45s
The branch predates the Gitea move (#253) by 37 commits and both sides
touched the same two files. Merged rather than rebased, so the signed
commit of the original work stays intact, and resolved by combining
rather than choosing - both statements are true at once: the chart has
ten images now, and the repo lives on Gitea.

README.md, the one real conflict: this branch rewrote the freesailarr
image bullet from nine upstreams to ten, while main had grown a
minecraft-server bullet next to it that this branch never had. Kept
both - our "ten" wording and main's new bullet.

renovate.json5 merged cleanly, but "cleanly" is not "correctly", so the
result was read rather than trusted: the freesailarr exception now says
ten and gained the pointer to the mediathekarr digest block, while the
two things main added right beside it survived unchanged - the note that
the post-upgrade hook is allow-listed on the shared Gitea instance, and
the forge-neutral wording of the automerge rule. Nothing from before the
move brought "GitHub" back into a sentence that had been made neutral.
2026-09-17 11:40:21 +02:00
jklein 78d7020dab freesailarr: MediathekArr as the tenth container, off by default (3.1.0+up1.0.0)
MediathekArr fakes a Usenet indexer towards Prowlarr and a SABnzbd downloader
towards Sonarr/Radarr and fetches from the ARD and ZDF media libraries. Both
consumers already share gluetun's network namespace, so they reach it over
127.0.0.1 - no service, no DNS, no ingress detour. Purely additive: the flag
defaults to false and a default render is byte-identical to 3.0.5 (verified
against the previous chart for both the default values and the CI values).

The four things that fail silently without them:

- A START GATE on gluetun. MediathekArr fetches its ruleset once at start and
  never retries; started before the tunnel it answers every Newznab search with
  0 results while the container is up, Ready and quiet. Kubernetes cannot order
  two regular containers and an initContainer would deadlock, so the wait sits
  in front of the image entrypoint: poll gluetun's health server on
  127.0.0.1:9999 and exec the entrypoint afterwards; on timeout the container
  exits rather than starting blind, because a crash loop is visible and an empty
  indexer is not. The startup probe budget (300s) deliberately outlasts the gate
  (180s).
- incomplete/ AND complete/ ON THE SHARED DOWNLOADS CLAIM, not in an emptyDir.
  An emptyDir counts against the container's ephemeral-storage limit, and one
  large download would have the kubelet evict the whole pod with all ten
  containers - the trap ytdlp-web-player documents.
- THE COMPLETE PATH SONARR SEES. The claim is mounted at /downloads exactly as
  sonarr/radarr mount it, and the chart sets DOWNLOAD_COMPLETE_PATH itself, so
  the SABnzbd complete_dir it reports is the same absolute path on both sides.
- TWO PORTS: 5007 (SABnzbd role and UI, primary) and 5008 (Newznab role for
  Prowlarr), both declared to gluetun's firewall and both on one Service. The
  SABnzbd API is at /download/api - /api on 5007 answers with a misleading
  Newznab validation error.

Further details:

- Image pinned by digest (tag + digest): "latest" is a 2024 build, "beta"
  moves. Single-arch linux/amd64. The built-in helm-values manager cannot
  follow that pair, so renovate.json5 gets the chart's first customManagers
  block; verified by dry-run with two fixtures, since a matching and a broken
  rule look identical while the pin is current.
- Hardened as measured: non-root, uid/gid from puid/pgid,
  readOnlyRootFilesystem: true, all capabilities dropped, no PUID/PGID.
- Probes on measured 200 endpoints (there is no health endpoint): readiness and
  startup on /download/, liveness as an exec covering BOTH ports, because one
  container runs two processes and a wedged Newznab half is exactly what makes
  searches come back empty.
- No ingress by default: the UI links its assets with root-absolute paths, so it
  can only be served at the root of a host; on the shared domain that root
  belongs to seerr. A sub-path fails the render instead of producing a broken
  UI.
- values.yaml and NOTES.txt document the geo MARKER PROCEDURE (not a table of
  series), the escape route of a German exit and its price for the whole stack,
  the fail_message: null trap, and the HLS question as an open question.
- CI: mediathekarr enabled in ci/freesailarr/test-values.yaml, install timeout
  raised to 1200s for the extra image and the gate.

Fixes #245
2026-09-08 23:20:16 +02:00
jklein 622e4c3ff8 ci: drop excluded charts from the install-test matrix
A chart in ci/excluded-charts.txt got an install-test job that ran the script, printed SKIP and exited 0. GitHub knows only the exit code, so the check list showed "Install-test satisfactory OK" for a chart whose image was never pulled - indistinguishable from one that really came up in the cluster (#231). detect-changed-charts now removes those charts before the matrix is built, so no job exists for them and nothing claims they passed.

An absence says nothing by itself, so the detect job writes every excluded chart of the PR with its documented justification into the job summary. A "Skipped" would only say "not run"; the reason is the question a review actually asks.

The list is not parsed in YAML. The workflow calls ci/run-install-test.sh --list-excluded, the one parser, and compares whole strings with jq, so a chart name is never read as a pattern. The script's own exclusion branch now goes through the same excluded_reason() helper instead of a grep that used the chart name as a regex; it stays for local runs, where the CI filter does not apply.

Not viable, and recorded at the matrix as a comment: deciding per chart in an "if" on the install-test job, which would give a visible Skipped entry instead of an absent one. GitHub rejects it - jobs.<id>.if is evaluated before the matrix is expanded, so the matrix context does not exist there ("context \"matrix\" is not allowed here. available contexts are \"github\", \"inputs\", \"needs\", \"vars\"."). Run 33650724109 on the closed PR #233 ended in failure with zero jobs created.
2026-09-04 16:58:14 +02:00
jklein 7e168da492 ci: parse the exclusion list in exactly one place
Two consumers read ci/excluded-charts.txt: this script and the install-test matrix in validate-charts.yml, which has to drop excluded charts before the matrix is built (#231). Two parsers would be two chances to select different charts, and the disagreement would be silent in either direction. So the script grows a --list-excluded mode and the workflow will call it instead of re-implementing the parsing.

A line carrying only a chart name and no justification is deliberately not an exclusion: the README requires the reason, and a bare name would drop a chart out of CI with nothing to review. The reason lookup matches the name exactly instead of as a regex, so a chart name is never read as a pattern.

The workflow side is not part of this commit.
2026-09-04 16:52:08 +02:00
jklein bfa72181eb minecraft-server: move to the java25 line (4.1.0+up2026.8.2.java25)
Minecraft 26.2 requires Java 25, so the chart's default minecraft.version can
only be raised on a java25 image. appVersion becomes 2026.8.2-java25 - the same
dated upstream release the java21 line ships, so the only variables that change
are the JVM and the Minecraft version.

The chart version stays at 4.1.0: the Java lines are siblings of one chart
generation, not successors, and are told apart by the +up suffix.

The Java boundaries in the values.yaml / NOTES.txt comments are read off
Mojang's version manifest (javaVersion.majorVersion per release), not
estimated: 1.21.11 is the last release asking for Java 21, every release from
26.1 on asks for Java 25.
2026-09-01 17:30:22 +02:00
jklein 6b0d8c9232 ytdlp-web-player: new chart with an optional oauth2-proxy subchart (1.0.0+upv1.2.0)
Deploys matszwe02/ytdlp_web_player, a Flask app that fetches videos with
yt-dlp, transcodes them with ffmpeg and serves them as HLS. It ships no
authentication of any kind, so the chart carries a preconfigured oauth2-proxy
as an optional subchart, in the data path in front of the app.

This is the first chart in the repo with a dependency, so the rule "charts
have no dependencies" falls here. It is replaced in CLAUDE.md and README.md by
a bar: a dependency has to be optional and off by default, upstream's chart is
consumed as it comes (documented where it falls short of our conventions,
never patched), and the price is written down - above all that a subchart is
invisible to everything downstream watching Helm release versions, so a new
proxy release only reaches a cluster when this chart is republished.

CI learns the dependency in five places, not the three the plan expected:
helm lint, package, template and install all exit 1 while charts/ is unfilled,
so ci/helm-dependencies.sh runs before lint and package in validate-charts.yml,
before package in publish-helm-chart.yml, and inside run-install-test.sh,
run-trivy-config.sh and run-quantity-render-test.sh - the last of which renders
every chart on every PR and would have gone red first.

Notable chart decisions, each measured rather than assumed:

- The image has no USER and runs as root by default, but not by necessity: it
  runs under an arbitrary uid with readOnlyRootFilesystem and the full
  hardening baseline. Only the pinned uid is a deviation, and it is documented.
- Two emptyDirs, each justified separately. /data holds the video cache
  (DATA_PATH). /tmp is NOT mounted - measured across metadata extraction, HLS
  transcoding, sprite generation and audio extraction, the image never writes
  there. What it does write is $HOME/.cache, so /cache is mounted with HOME and
  XDG_CACHE_HOME pointed at it; without it yt-dlp re-downloads its JavaScript
  challenge-solver from github.com on every extraction.
- The proxy is wired through config.configFile, the only value the subchart
  passes through tpl - config.upstreams is emitted with toJson and would carry
  a literal {{ .Release.Name }} into the manifest.
- Rendering aborts when this chart's Ingress and the proxy's claim the same
  host: ingress-nginx rejects the second one and a Fleet bundle then sits in
  ErrApplied until forceSyncGeneration is bumped.

Fixes #214
2026-08-30 17:34:49 +02:00
jklein 148e15a1f6 ci: guard resource quantities against exponential notation
Adds ci/run-quantity-render-test.sh and a validate-charts job that runs it
over every chart on every PR.

The bug behind #157 has a loud and a silent half. Loud: a bare integer request
of 1048576 is reformatted by printf "%v" to "1.048576e+06", the helper rejects
the suffix it produced itself and the render aborts. Silent: with a fractional
limitFactor the *product* is reformatted after the suffix check, the render
succeeds, and the manifest carries memory: "1.4999985e+06" - rejected by
nothing until the API server sees it at apply time.

So the test renders each chart twice and, for the silent case, searches the
rendered text rather than the exit code. It runs over all charts, not the
changed ones: the resources helper is copied into every chart's _helpers.tpl,
so the regression arrives by copy, in a PR whose diff shows a new chart and not
the bug inside it. The resources paths come from values.schema.json (JSON, so
python3 reads it without pyyaml), which also means a new chart is enrolled
automatically and there is no list to maintain.

Charts that pin every computable limit in their fallback defaults never reach
the helper (gotenberg, rss-bridge). They are reported as "not exercised"
instead of counted as passing.
2026-08-29 01:25:48 +02:00
jklein 8df1913c5c bentopdf: new chart for the PDF toolbox (1.0.0+up2.8.7)
BentoPDF computes entirely in the visitor's browser; the container only serves
static files through nginx. The chart is therefore close to cyberchef - same
nginx-unprivileged base, uid 101, port 8080, probes on "/" - with four
differences that shaped the design.

Runtime config.json. The one setting reachable at runtime is the tool list.
It is a static file in the middle of the web root, so it is mounted by
subPath: a directory mount would hide the whole application. subPath mounts
are never refreshed by the kubelet, which makes the checksum/config annotation
(#82) load-bearing rather than decorative. bentopdf.config is passed through
verbatim, like searxng's settings, so a key upstream adds tomorrow is a values
change and not a chart change.

Three env switches that would kill the pod. PORT, DISABLE_IPV6 and
ROBOTS_NOINDEX are implemented as "sed -i" against files in the image, run
under "set -e" in /docker-entrypoint.d/, and the base image's entrypoint
aborts when one of them fails. With readOnlyRootFilesystem: true the pod
therefore never starts instead of ignoring the switch - measured in k3d as
CrashLoopBackOff, exit 1, "sed: can't create temp file
'/etc/nginx/nginx.confXXXXXX': Read-only file system". None of the three is
offered as a first-class value; values.yaml names the failure picture and the
escape hatch (env block plus readOnlyRootFilesystem: false).

Writable paths were measured, not copied from the upstream chart.
/etc/nginx/tmp holds the pid file and /var/cache/nginx is where nginx creates
its temp directories at startup - without it the master aborts with
'mkdir() "/var/cache/nginx/client_temp" failed'. /var/run is deliberately not
mounted: the running pod proves it is unnecessary.

Everything else about BentoPDF is a build-time argument and out of reach for
any chart; values.yaml says so explicitly rather than leaving it to be
discovered. Two accepted consequences are named there too: some tools fetch
WASM modules from a public CDN at runtime, and the served pages carry a
canonical link to the vendor's site.

Fixes #174
2026-08-28 18:22:30 +02:00
jklein 46942db53a searxng: new chart (1.0.0+up2026.8.22.9fea41204)
SearXNG as a stateless Deployment plus a settings ConfigMap, with the four CI
steps a new chart needs: publish workflow, change-app-version choice list,
ci/searxng/ fixtures and a renovate.json5 block.

Three points where the chart is not a cyberchef with another name:

* The image entrypoint writes settings.yml into /etc/searxng only when the
  file is missing. The chart always mounts one by subPath, so the copy and
  the random-secret sed never run and the container needs no write access
  there - which is what allows readOnlyRootFilesystem: true.
* The signing key is a required OnePasswordItem, not a ConfigMap key. It
  signs the settings cookie and the image-proxy URLs, and the entrypoint's
  self-generating fallback cannot fire here, so without it the instance would
  run on the literal "ultrasecretkey" - which webapp.py refuses with exit 1.
* All three probes use /healthz, the only path the bot detection lets through
  unconditionally. Measured: with the limiter active, "/" answers 429 while
  /healthz still answers 200.

ingress.domain is read a second time outside the Ingress template, for
SEARXNG_BASE_URL, and carries its own required() there - so it stays
mandatory with the Ingress switched off.

The Renovate versioning regex deviates from the plan on the issue and matches
the commit hash WITHOUT capturing it. Renovate parses a "build" group with
Number.parseInt, so a hash becomes the digits leading it; measured on
renovate 41.173.1 and 44.48.0, the planned form reports an image pushed
earlier the same day as an update to a newer one. Reasoning in the block and
in README.
2026-08-27 21:09:59 +02:00
jklein e5f98435ed ci: redact declared secret values from install-test diagnostics
The failure path prints container logs, and an app may print a value it was
given - gluetun logs its Wireguard address on startup. diagnose() now runs
through a filter that replaces the values of the declared REQUIRED_ENV
variables with the variable name. GitHub masks them as well, but that is the
platform's promise rather than this harness's, and the script runs locally too,
where nothing masks anything.
2026-08-26 14:37:46 +02:00
jklein 69a280399c ci: run the freesailarr install-test for real instead of skipping it
freesailarr was excluded from the install-test because gluetun never becomes
healthy without a real Wireguard key and no app container behind its kill
switch can become Ready. The green "Install-test freesailarr pass" was a
25-second SKIP on the most complex chart in the repo - a signal that proved
nothing.

With a CI-only Mullvad key in 1Password the reason is gone:

- ci/run-install-test.sh learns a general rule: ci/<chart>/fixtures.env.yaml is
  rendered with envsubst before it is applied, restricted to the variable names
  a chart declares in REQUIRED_ENV. A missing or empty declared variable fails
  the run loudly and names the variable - it never skips the chart.
- The install-test job loads the key from 1Password Connect for freesailarr
  only, following the pattern publish-helm-chart.yml documents: values via
  steps.<id>.outputs.*, mapped into the step's own env, quoted, with ":?"
  guards that name the missing op:// field before a cluster is created.
- Fixtures for freesailarr: three media claims, the gluetun secret template
  with variable names only, firewallOutboundSubnets so the kill switch does not
  block cluster DNS, and INSTALL_TIMEOUT=900 for nine containers.

freesailarr/Chart.yaml goes 3.0.0+up1.0.0 -> 3.0.1+up1.0.0. That bump is
deliberately empty of content: the install-test matrix follows changed chart
directories, and without it this PR could not run the test it adds.
2026-08-26 14:30:42 +02:00
jklein 70c7106fe8 Give the Trivy scan render values for freesailarr
ci/run-trivy-config.sh renders every changed chart before scanning it and exits
non-zero when the render fails - which it does for freesailarr, because the
chart is excluded from the install-test and therefore has no
ci/freesailarr/test-values.yaml to fall back on. The scan job would go red
without ever scanning anything.

The file supplies only the values the chart declares required (secret ref,
cluster issuer, the three media claims and one ingress host per app, since the
chart has no global ingress.domain) and nothing that would alter the security
posture the scan is meant to look at.
2026-08-24 13:51:29 +02:00
jklein 914c2d0205 Add freesailarr chart: gluetun-tunneled media automation stack
Single-pod StatefulSet running gluetun (appVersion 3.41.1) as Wireguard
VPN gateway with qbittorrent, prowlarr, flaresolverr, radarr, recyclarr,
sonarr, seerr and bazarr sharing its network namespace. Config volumes
via volumeClaimTemplates, media libraries as existing claims, per-app
enable/ingress toggles, computed FIREWALL_INPUT_PORTS, hardened
security contexts with documented per-image deviations. Wires up
publish workflow, change-app-version choice, install-test exclusion
and README overview.
2026-08-24 13:41:46 +02:00
jklein 6b25d082a7 renovate: chart-aware config with an automatic version bump
Renovate only ever rewrites appVersion or an image tag, so every PR it
opened against a chart directory failed the build-chart version check
(#138, #139). ci/renovate-bump-chart.sh recomputes
version: <semver>+up<appVersion> as a postUpgradeTasks hook: chart major
when the app major moves, chart patch otherwise.

Each chart renders its main image as <repo>:{{ .Chart.AppVersion }}, so
repository and version live in different files and Renovate cannot join
them. The mapping is one customManager per chart in renovate.json5,
deliberately not a comment inside each Chart.yaml - that would touch all
20 chart directories and force 20 publishes for a comment change.

Patch and digest updates automerge after a 3-day minimumReleaseAge once
the pipeline is green; majors arrive one line at a time to match the
stepwise app-major procedure. ci/** stays excluded: the install-test
fixtures pin throwaway infrastructure to major lines on purpose.
2026-08-24 11:42:43 +02:00
jklein 7e6b3beed5 rss-bridge: pin the unprivileged-port sysctl, quote env values
Follow-up to the template-chart hardening fixes in #133, which landed
while this branch was open.

The env block now quotes names, values and keyRef fields. Without it a
value like "5" renders as a YAML number and the API server rejects the
whole manifest. RSS-Bridge takes numeric settings that way
(RSSBRIDGE_http_timeout), so this was reachable with the first env entry
anyone adds.

The port-80 question is settled by measurement rather than by the
"it works in k3d" reasoning #133 warns about. Same pod spec in k3d
(k3s v1.35.5), only ip_unprivileged_port_start changed:

  1024, capabilities.drop [ALL]      nginx: bind() to 0.0.0.0:80 failed
                                     (13: Permission denied)
  1024, plus NET_BIND_SERVICE        identical failure - for a non-root
                                     process the capability lands in the
                                     permitted but not the effective
                                     set, so it buys nothing
  0                                  binds, HTTP 200

So the chart cannot buy safety with a capability the way samba does, and
port 80 is not negotiable either: nginx is configured for it inside the
image and the entrypoint's only lever is a sed against a read-only
/etc/nginx. What it can do is stop depending on the node default - the
pod securityContext now pins the sysctl to 0, so the bind no longer
relies on cluster configuration. Verified: the chart-installed pod
reports 0 and answers 200 through the Service.

Worth recording how that failure looks without probes: the entrypoint
backgrounds nginx and then execs php-fpm, so a failed bind leaves a
container reporting Running with no web server at all. Both test pods
above were Running. The probes on /?action=health are what turn it into
a pod that never becomes Ready.

trivy config still reports 0 to act on; the KSV-0117 statement is
updated to the measured reasoning.
2026-08-23 23:39:49 +02:00
jklein e016650f91 rss-bridge: qualify the image registry, except the port-80 finding
Two findings from the Trivy config scan that landed on main in #129.

AVD-KSV-0125 is fixed rather than excepted: writing the image as
docker.io/rssbridge/rss-bridge puts a real registry in the registry
position instead of "rssbridge", which is what ci/trivy/data/ksv0125.yaml
asks for. Verified in k3d that the qualified reference resolves to the
same digest (sha256:0eca99b2...) and still runs.

AVD-KSV-0117 (containerPort below 1024) becomes a scoped, documented
exception. nginx listens on 80 inside the image, and the entrypoint's
only way to change that is a sed against /etc/nginx/sites-enabled/default
which a read-only root filesystem rejects - so the alternative to port 80
is giving up readOnlyRootFilesystem. The premise of the check does not
hold here anyway: the container binds 80 as uid 33 with all capabilities
dropped and no NET_BIND_SERVICE.

trivy config now reports 0 to act on for this chart.
2026-08-23 23:33:23 +02:00
jklein 3fa67d55ef rss-bridge: new chart
Takes over the four loose manifests in cluster-config/rss-bridge/ as a
proper chart, on the current template-chart standard: strict values
schema, null-safe helpers, computed limits, replicas/scaleDown,
checksum/config, NOTES.txt.

Two things deviate from the repo standard on purpose:

- The Ingress models host AND TLS secret name per entry instead of the
  single conventional tls-<release>-<chart>-ingress. The installation
  this replaces has grown three separate certificates, one per host.
  Renaming them would make cert-manager issue three new ones and leave
  each host unreachable until it did - a self-inflicted outage on a
  deployment that has been up for over a year. Entries sharing a secret
  name are grouped into one tls entry, and omitting the name falls back
  to the convention, so consolidating later is a values change.

- Requests and limits are decoupled. The old manifest requested 2Gi of
  ephemeral storage, identical to its limit, reserving two gigabytes for
  a workload measured at 2.2 MB after fetching all five whitelisted
  bridges. The request drops to 64Mi; the limit stays at 2Gi.

Hardening was verified rather than assumed. The image ships no USER and
runs as root by default, but runs fully hardened once its writable paths
are provided: uid/gid 33, readOnlyRootFilesystem, all capabilities
dropped, seccompProfile RuntimeDefault. The writable paths were found by
running the image with a writable rootfs, exercising every bridge and
reading its diff layer - /app/cache, /run, /run/php, /var/lib/nginx.
/tmp did not show up that way, because PHP's opcache creates its lock
file there and unlinks it immediately; it only surfaced when the
hardened container died at startup. Stopping at the first writable path
would have shipped a chart that cannot start.

Probes use the app's own /?action=health, which renders fixed JSON
without touching the cache or any remote site, so neither a scraped site
going down nor an unwritable cache can restart the pod.

Fixes #128
2026-08-23 23:33:23 +02:00
jklein 00ea301e33 template-chart: seccomp profile, unprivileged port, and two placeholder fixes
Five changes to the scaffold every future chart is copied from:

1. seccompProfile.type: RuntimeDefault in the pod security context, in
   line with the rest of the seccomp series (#84). Pod context, because
   from there it also covers init containers and any container added
   later without being repeated.

2. containerPort 80 -> 8080, clearing AVD-KSV-0117. Note that the usual
   justification for this does not hold: a non-root container CAN bind
   port 80 on current Kubernetes, because the kubelet sets
   net.ipv4.ip_unprivileged_port_start=0 in the pod network namespace.
   Measured, not assumed - see the values.yaml comment. The reason to
   stay above 1024 is that this is a runtime default rather than a
   guarantee, and that every real chart here already does.

3. The ConfigMap placeholder key is renamed from the literal "key" to
   "example-setting", clearing AVD-KSV-01010 (secret-scanner patterns
   match any key named "key").

4. The image placeholder is fully qualified as ghcr.io/ORG/IMAGE, so the
   template stops teaching the unqualified form that produced the
   repo's unqualified Docker Hub images.

5. env values are rendered through "quote". Without it a value like
   "8080" renders as a YAML number and the API server rejects the whole
   manifest ("cannot unmarshal number into Go struct field
   EnvVar.value of type string"). Every real chart in the repo already
   quotes here; only the template did not. Found by instantiating the
   template and installing it, not by rendering it.

Verified by instantiating the scaffold into a working chart per the
README procedure and installing it in k3d: the workload becomes Ready,
the Service routes to the app on 8080 and an ephemeral container added
to the running pod inherits Seccomp: 2 from the pod context.

Refs #84
2026-08-23 23:29:51 +02:00
jklein 705ee066a4 CI: record the paperless-ngx root init container as a documented exception
The pod runs rootless as uid/gid 1000; only init-run-ownership is root, to
chown the /run emptyDir that s6-overlay requires to be owned by the runtime
uid. Justified in paperless-ngx/values.yaml, so AVD-KSV-0105/-0022/-0106 are
excepted for that chart with the reason, instead of being reported as if the
workload itself ran as root.
2026-08-23 23:14:12 +02:00
jklein 1ae5e1a7c3 CI: report Trivy misconfiguration findings for changed charts
Adds a trivy-config job to the PR pipeline that renders each changed chart
with Helm and runs `trivy config` over the rendered manifests. Reporting
only - TRIVY_CONFIG_MODE=report always exits 0, so the job never blocks a PR.

Rendered manifests rather than the chart directory: `trivy config <chart-dir>`
renders with default values, which fails on every chart here because of their
required() values - and Trivy does not fail with it, it logs "Skipping chart"
and prints an empty, green report.

Values come from the install-test fixtures (ci/<chart>/test-values.yaml), the
only committed per-chart values in the repo; none of them touches
securityContext, images or capabilities, so the scan sees the chart's own
hardening. Charts without fixtures get ci/trivy/values/<chart>.yaml.

Exceptions live in ci/trivy/trivyignore.yaml in Trivy's native ignore-file
schema, each with a statement - the script refuses to run without one and
reports suppressed findings with their reason instead of hiding them.
AVD-KSV-0125 is configured instead of excepted: ci/trivy/data/ksv0125.yaml
lists the trusted registries, so the check stays able to flag a genuinely
foreign source.

Refs #84.
2026-08-23 23:14:12 +02:00
jklein 4f489b0854 paperless-ngx: an empty barcodes block falls back to the chart defaults
A "barcodes:" key left without children (e.g. after deleting the last
remaining sub-key, PAPERLESS_CONSUMER_BARCODE_SCANNER, which paperless-ngx
3.0 removed) is YAML null and replaced the whole default sub-tree on merge.
Rendering then failed on the nil pointer, so the release simply stopped
updating.

An empty or null block now means "the chart defaults apply". The merge runs
through a new paperless-ngx.defaultedDict helper: nulls never delete, nested
maps merge recursively, and explicitly set values win - including false and 0,
which "default" and "mergeOverwrite" would swallow. A scalar where the
defaults define a map is rejected with the values path in the message.

Major bump: the meaning of a values override changes.

Fixes #61
2026-08-23 19:31:20 +02:00
jklein 227bcebeb6 Bring minecraft-server up to the chart style guidelines (1.0.0+upjava8)
Stage 2 of the Fleet migration for minecraft-server. Major bump: runtime
behaviour changes (probes, hardening) and the values interface breaks
(resource renames, strict schema).

Two real bugs first:
- The container declared port 35565 while the Service targeted 25565. It
  only worked because targetPort was numeric. Both are now server.port
  (25565) and the Service targets the named port.
- The download-serverpack init container was guarded by the map
  .Values.curseforgeServerpack instead of .enabled, so it rendered for
  every release and silently no-opped on an empty wget URL.

Conventions:
- Resources renamed to <release>-<chart>-<kind>, templates to
  <app>.<kind>.yaml. The PVC deliberately keeps its historical name -
  renaming it would provision a new, empty volume and orphan the worlds.
- Probes (liveness/readiness/startup) via the image's own mc-health, so
  Ready means the server answers a server-list ping. Startup budget 20 min.
- Hardened, values-configurable security contexts: non-root uid 1000,
  read-only root filesystem with an emptyDir at /tmp, all caps dropped.
- Resources from values with the limitFactor helper; an unset memory
  request is derived from the JVM heap so the two cannot drift apart.
- Additive env block rendered through tpl, pinned init image, storage
  class as a value, RCON wiring only when actually enabled.
- curseforge/ftb/curseforgeServerpack are now mutually exclusive.
- minecraft.version is pinned to the Java line instead of LATEST.
- The mc-router auto-discovery annotation is gone; the service type it
  implicitly switched is now the explicit value server.service.type.

Schema pilot (#68): values.schema.json with additionalProperties false on
every chart-owned level, rejecting all three known dead keys.
2026-08-23 16:40:44 +02:00
jklein 198058b8bb Update minecraft-server to the java21 line (0.4.0+upjava21)
Last of the three Java-line publishes; main comes to rest on java21, the line
of the only running server, and the registry holds all three tags.

Only the two version lines in Chart.yaml change. Because the source chart in
cluster-config is itself on java21, this final state renders byte-identical to
it for mc-ftb-skies-2 with its real values file - the published end state is
exactly what cluster-config runs today.

The CI install values move along with the appVersion, as in the java17 PR:
the JVM is fixed per published chart version and no single Minecraft version
boots on all three lines. Since the chart has no probes, a version the JVM
cannot load would crash after the rollout returned and still leave a green
install-test behind. minecraft.version therefore moves 1.20.1 -> 1.21.1, which
is also what ftb-skies-2 sets in its real values file, so CI exercises the
production version. ci/ is outside the chart directory and is neither packaged
nor published.
2026-08-23 16:12:15 +02:00
jklein eed1a48fc0 Update minecraft-server to the java17 line (0.4.0+upjava17)
Second of the three Java-line publishes. Only the two version lines in
Chart.yaml change; the rendered output differs from the published java8 chart
in exactly one line, the image tag.

The CI install values move along with the appVersion: the image tag is the
Java line, so the JVM is fixed per published chart version and the Minecraft
version has to match it (Java 8 runs up to 1.16.5, 1.18-1.20.4 need Java 17).
No single Minecraft version boots on all three lines, and because the chart
has no probes a version the JVM cannot load would still leave a green
install-test behind. minecraft.version therefore moves 1.12.2 -> 1.20.1.
ci/ is outside the chart directory and is neither packaged nor published.
2026-08-23 16:05:53 +02:00