Author SHA1 Message Date
jlab-renovate-bot 27e291dc48 Update dependency kubernetes/kubernetes to v1.37.1
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Build ${{ matrix.chart }} (pull_request) Skipped
Validate Helm Charts / Trivy config ${{ matrix.chart }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
Validate Helm Charts / Resource quantities (pull_request) Successful in 19s
Validate Helm Charts / Selector match (pull_request) Successful in 8s
renovate/stability-days Updates have met minimum release age requirement
2026-10-08 16:04:57 +00:00
jlab-renovate-bot b5b5d9a310 Update docker.io/stalwartlabs/stalwart Docker tag to v0.16.25
renovate/stability-days Updates have met minimum release age requirement
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Selector match (pull_request) Successful in 9s
Validate Helm Charts / Build stalwart (pull_request) Successful in 5s
Validate Helm Charts / Trivy config stalwart (pull_request) Successful in 11s
Install-test / Install-test stalwart (pull_request) Successful in 1m9s
Validate Helm Charts / Install-test (pull_request) Successful in 1m9s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 15s
Publish Stalwart Helm Chart / Publish Stalwart Helm Chart (push) Successful in 15s
SonarQube Analysis / SonarQube Analysis (push) Successful in 26s
Validate Helm Charts / Resource quantities (pull_request) Successful in 12s
2026-10-08 07:35:13 +00:00
jklein 6ade74db80 feat(stalwart): spread each node group over nodes and give each one a PDB
Publish Stalwart Helm Chart / Publish Stalwart Helm Chart (push) Successful in 7s
SonarQube Analysis / SonarQube Analysis (push) Successful in 27s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
A two-replica node group was redundancy on paper: nothing kept both pods off
one node, and no PodDisruptionBudget kept a drain from taking both at once.
Both objects are now rendered per node group, gated on that group's COMPUTED
replica count being >= 2.

The distribution is a promise, not an effort: maxSkew 1 on
kubernetes.io/hostname with whenUnsatisfiable DoNotSchedule, selecting only
this group's own pods, so an "mx" and a "full" pod may share a node. The
selector comes from the chart (stalwart.groupLabels), never from the operator -
a spread constraint whose selector matches no pod is trivially satisfied,
distributes nothing and reports nothing.

nodeAffinityPolicy: Ignore is set explicitly, and that is the load-bearing
detail. Its API default is Honor, under which a narrowing nodeSelector shrinks
the set of counted domains until both replicas fit into one: skew 0, both pods
co-located, no Pending, no event, and a selector that is correct - so no check
can see it. With Ignore the same narrowing gives skew 2 and the second pod is
Pending. A render abort was the alternative and would also have forbidden a
nodeSelector spanning five nodes, which is legal.

A user entry colliding with the chart's own on (topologyKey,
whenUnsatisfiable) aborts the render naming both values, because the API
rejects the duplicate with a message about a list index and an apply-based path
merges the two over patchMergeKey: topologyKey silently, dropping one selector.

Minor rather than major, proved with a diff rather than argued: with the
shipped defaults (full: 1, mx: 0) and with ci/stalwart/test-values.yaml,
helm template renders byte-for-byte what 1.0.1+upv0.16.24 rendered - both
diffs empty.

New in CI, over every chart on every PR: ci/run-selector-match-test.sh holds
every rendered PodDisruptionBudget selector and every topologySpreadConstraints
labelSelector against the labels of the pod templates in the same render. Helm
itself is the YAML parser (ci/yaml2json), because the runners have neither
pyyaml nor yq and a hand-written parser could read nothing and report nothing.
podAffinity/podAntiAffinity are out of scope; matchExpressions-only selectors
are reported as not checked rather than counted as passing. Two anti-vacuum
guards: a chart with an overlay under ci/selector-match/ that renders no
selector fails, and a run that checks no selector at all fails.

Fixes #96
2026-10-07 19:36:04 +00:00
jlab-renovate-bot 1698fc0aeb Update itzg/mc-router Docker tag to v1.47.2
renovate/stability-days Updates have met minimum release age requirement
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 10s
Validate Helm Charts / Trivy config minecraft-router (pull_request) Successful in 13s
Install-test / Install-test minecraft-router (pull_request) Successful in 5m9s
Validate Helm Charts / Install-test (pull_request) Successful in 5m9s
SonarQube Analysis / SonarQube Analysis (push) Successful in 27s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
Publish Minecraft-Router Helm Chart / Publish Minecraft-Router Helm Chart (push) Successful in 7s
Validate Helm Charts / Build minecraft-router (pull_request) Successful in 4s
2026-10-07 14:35:14 +00:00
jklein 6e196c26a5 test(ci): restore the theme mount - the A/B shows the restart is pre-existing
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 14s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 12s
Install-test / Install-test keycloak (pull_request) Successful in 3m10s
Validate Helm Charts / Install-test (pull_request) Successful in 3m10s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
Publish Keycloak Helm Chart / Publish Keycloak Helm Chart (push) Successful in 7s
SonarQube Analysis / SonarQube Analysis (push) Successful in 28s
2026-10-06 11:43:46 +00:00
jklein 8cd0c14e3f test(ci): control run 598 recorded (81s, 0 restarts)
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Build keycloak (pull_request) Successful in 9s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 16s
Install-test / Install-test keycloak (pull_request) Successful in 9m24s
Validate Helm Charts / Install-test (pull_request) Successful in 9m24s
Validate Helm Charts / Resource quantities (pull_request) Successful in 13s
2026-10-06 11:29:44 +00:00
jklein 71cc545277 test(ci): A/B control run - keycloak install-test without the theme mount (temporary)
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 8s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 10s
Install-test / Install-test keycloak (pull_request) Successful in 2m14s
Validate Helm Charts / Install-test (pull_request) Successful in 2m14s
2026-10-06 11:22:26 +00:00
jklein d7067abfab docs(ci): the whole restart series, intermittent on an unchanged spec
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 9s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 10s
Install-test / Install-test keycloak (pull_request) Successful in 2m45s
Validate Helm Charts / Install-test (pull_request) Successful in 2m45s
2026-10-06 11:13:46 +00:00
jklein 5e47191252 docs(ci): three runs, and the load explanation does not hold
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 9s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 13s
Install-test / Install-test keycloak (pull_request) Successful in 3m59s
Validate Helm Charts / Install-test (pull_request) Successful in 3m59s
2026-10-06 11:04:16 +00:00
jklein 34c600f7a8 docs(ci): the second run settles it - the keycloak restart was the runner, not the pod spec
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 10s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 10s
Install-test / Install-test keycloak (pull_request) Successful in 4m2s
Validate Helm Charts / Install-test (pull_request) Successful in 4m2s
2026-10-06 10:54:48 +00:00
jklein 1cf59f5448 docs(ci): record the keycloak install-test run that was slower and restarted once
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 11s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 9s
Install-test / Install-test keycloak (pull_request) Successful in 2m18s
Validate Helm Charts / Install-test (pull_request) Successful in 2m18s
2026-10-06 10:46:57 +00:00
jklein d8b392bf64 feat(keycloak): extraVolumes/extraVolumeMounts for custom themes and providers (1.1.0+up26.8.0)
Validate Helm Charts / Detect changed charts (pull_request) Successful in 11s
Validate Helm Charts / Resource quantities (pull_request) Successful in 15s
Validate Helm Charts / Build keycloak (pull_request) Successful in 4s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 17s
Install-test / Install-test keycloak (pull_request) Successful in 3m9s
Validate Helm Charts / Install-test (pull_request) Successful in 3m9s
2026-10-06 10:26:13 +00:00
jklein b39f36ff5c feat(bulwark): new chart for Bulwark Webmail (1.0.0+up1.12.0)
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
SonarQube Analysis / SonarQube Analysis (push) Successful in 27s
Publish Bulwark Helm Chart / Publish Bulwark Helm Chart (push) Successful in 7s
A JMAP webmail client for the stalwart chart (Gitea #65): StatefulSet, one PVC
at /app/data, Service plus headless Service, Ingress at a host root, two
OnePasswordItems, three probes, computed limits, replicas plus scaleDown, strict
values.schema.json.

Everything below was measured against the source at tag 1.12.0 rather than read
out of the documentation, which was wrong about three of them.

THE DEFINING PROPERTY: THE VOLUME BEATS THE CHART, FOR EVERY VALUE, SILENTLY.
configManager.get resolves "admin override > env var > default", the admin
dashboard may write any key of CONFIG_ENV_MAP, and nothing reports that it just
disconnected the chart - so a helm upgrade can succeed and change nothing.
Upstream documents this for the server URL only; it is true of everything.
Exactly three values escape it (SESSION_SECRET(_FILE), ADMIN_PASSWORD,
BULWARK_TELEMETRY). The value that closes it, bulwark.adminConfigReadOnly,
defaults to false because its price is threefold: it freezes the settings policy
(and with it the only way to move Web Push off the vendor relay), it freezes an
override that ALREADY exists instead of removing it, and on a fresh install it
stops admin.json from being written at all - so the admin dashboard never comes
up and its login answers HTTP 500 while the pod stays Ready.

OIDC-only sign-in, with the real way back out of a lockout: not helm upgrade
with OAUTH_ONLY=false (oauthOnly goes through the same precedence chain) but
removing the override in the admin dashboard. That is why ADMIN_PASSWORD is set
- the dashboard is the door. It generates no random password and logs nothing,
so the #65 analogy does not apply.

SESSION_SECRET_FILE and OAUTH_CLIENT_SECRET_FILE are both used - the better path
garage (#81) and keycloak (#83) did not have - in one projected volume mounted
WITHOUT subPath, because both are re-read on every use and a subPath mount is
never refreshed, and at mode 0440 because secret files are owned by root and
fsGroup only supplies the group.

Two backup statements, both at the value: without the session secret the
synchronized settings are unreadable out of an intact volume, and if the
dashboard's OAuth setup was ever used the client secret sits in config.json on
that volume in cleartext.

Probes: all three a plain GET on /api/health, with the limit written at the
readiness value - it carries process, HTTP acceptance, a free event loop and
Next.js routing, and nothing about the reported state or about Stalwart.

No ConfigMap, so the checksum/config rule (#82) has nothing to attach to, and
values.yaml says so explicitly; the op:// paths are covered by
checksum/secret-ref.

Renovate: no versioningTemplate, measured rather than assumed. 315 tags in eight
shapes (releases, a second v-prefixed family of the same images, an -always
flavour, floating minors and names, commit SHAs, betas, attestations) and plain
docker versioning excludes all six unwanted ones. Dry runs at 1.12.0, 1.11.0 and
1.10.0 are recorded in the block.

Part of #89 - deliberately not "Fixes", because one binding decision of the
issue was changed out of necessity and wants ratification rather than an
automatic close: the plan specifies HEAD /api/health for all three probes, and a
Kubernetes probe cannot send HEAD at all (HTTPGetAction has no method field; the
kubelet always issues GET). The alternative, an exec probe, would mean starting a
Node process every few seconds in an image that carries no curl and no wget. GET
is used instead and is marginally more capable than HEAD, so every limit the
issue argues is unchanged - but the decision is the maintainer's to confirm.
2026-10-06 10:15:27 +00:00
jlab-renovate-bot 40e29f7f72 Update lukevella/rallly Docker tag to v4.15.4
renovate/stability-days Updates have not met minimum release age requirement
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 8s
Install-test / Install-test rallly (pull_request) Successful in 1m35s
Validate Helm Charts / Install-test (pull_request) Successful in 1m35s
Validate Helm Charts / Build rallly (pull_request) Successful in 4s
Validate Helm Charts / Trivy config rallly (pull_request) Successful in 9s
Publish Rallly Helm Chart / Publish Rallly Helm Chart (push) Successful in 7s
SonarQube Analysis / SonarQube Analysis (push) Successful in 26s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
2026-10-06 09:04:07 +00:00
jklein 28567842f3 fix(keycloak): group 1000 instead of the image's root group
Publish Keycloak Helm Chart / Publish Keycloak Helm Chart (push) Successful in 7s
SonarQube Analysis / SonarQube Analysis (push) Successful in 28s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
The first Trivy run on this chart reported one finding out of 96 tests:
AVD-KSV-0116, "Runs with a root primary or supplementary GID", because the pod
context carried runAsGroup: 0 - copied from what the image expects.

Checked instead of suppressed, and the check says the group is not needed.
Counted across the image layer of quay.io/keycloak/keycloak:26.8.0: 521 of 522
entries are owned by 1000:0, and every mode present (drwxrwxr-x, -rwxrwxr-x,
-rw-rw-r--) grants the OWNER full access. The one remaining entry is /opt
itself, which needs only traverse. So uid 1000 gets everything it reads and
executes through the owner bits, whatever group it is in.

Group 0 is upstream's OpenShift arbitrary-uid pattern - root group plus a
group-writable tree, so that ANY uid landing in group 0 can write. It buys
nothing once the owning uid is pinned, which this chart does. Writes all go
into the three emptyDirs, which fsGroup 1000 makes writable.

So runAsGroup is 1000, the finding is gone rather than excepted, and no entry
was added to ci/trivy/trivyignore.yaml - which matters more than one LOW
finding: that file scopes by chart, so an AVD-KSV-0116 exception for keycloak
would also have silenced it for the serving container, on a check that is
making a fair point about an identity provider.

The condition under which this goes back to 0 is written at the value: a path
uid 1000 does not own. There is none in this image today.

Part of #83
2026-10-04 18:00:20 +00:00
jklein 320257c742 fix(keycloak): readiness could not fail on the database, and the measured budget
Two corrections found while verifying the first CI run, not from the plan.

THE DEFAULT THAT MADE THE READINESS PROBE TOOTHLESS. Keycloak's DATABASE health
check is the one row of upstream's "Available Checks" table whose "Requires
Metrics" column says yes; Cluster, Graceful shutdown and Keycloak Initialized
say no. So with metrics-enabled=false - this chart's default until now -
/health/ready answers 200 while the database is gone, and the pod keeps taking
logins it cannot serve. The probe path was right and the aggregate behind it
contained nothing external, which is the urlaubsverwaltung defect (#218) in a
new hiding place: not a health group this time, but an option named for
observability, where nobody looks when reviewing a probe.

keycloak.metrics.enabled therefore defaults to TRUE, and its comment argues
about readiness rather than about scraping. The price is stated at the value:
/metrics then answers on the management port, unauthenticated, carrying realm
names, client ids and login counters - on a port that is ClusterIP-only and in
no Ingress. Switching it off is now documented as a decision about whether
readiness can fail at all. The CI fixture keeps it on and says why, so a green
run really does touch the database.

THE STARTUP BUDGET NOW CARRIES ITS MEASUREMENT. From the install-test of PR #87
on the Gitea docker runner (k3s v1.35.5, two replicas, empty database, image
pulled during the run): 71.8 seconds from `helm install` to BOTH pods Ready -
first at ~70.8s, second one second later, 0 restarts. The 600s budget stays,
with the reason for the headroom written down: everything in that run was the
cheap case, and a first install against a real database serializes the second
replica behind the first's Liquibase migration lock instead of letting it
arrive a second later.

AND AN OVERCLAIM OF MY OWN, REMOVED. ci/keycloak/test-values.yaml said a green
run proves jdbc-ping found the second replica. It does not: readiness reports
one node's own health, so a node that never found its peer is Ready, healthy
and alone - and two such nodes are two islands that each believe they are the
cluster. What the test does rule out is the louder half, a replica that cannot
start or cannot share the database. The Infinispan view has to be read out of
the log, which is what step 2 of NOTES.txt tells the operator to do, now with
the note that no probe and no install-test replaces that step. Measurement
point 4 of the plan therefore stays open.

Part of #83
2026-10-04 18:00:20 +00:00
jklein 4c1cff1744 feat(keycloak): new chart for Keycloak (1.0.0+up26.8.0)
Two replicas with cache=ispn, clustered through Keycloak's default jdbc-ping
stack: discovery runs over the PostgreSQL that is there anyway, so no headless
Service and no JGroups wiring. Deployment rather than StatefulSet - Keycloak 26
keeps everything in the database, user sessions included.

The chart's defining property is that it has TWO INIT CONTAINERS and needs
both. db, health-enabled and metrics-enabled are Keycloak BUILD options
(verified in the 26.8.0 option table), the published image is augmented with
their defaults, and supplying different ones at runtime makes Keycloak rewrite
/opt/keycloak/lib/quarkus at start - which readOnlyRootFilesystem: true forbids
and which upstream describes as a start that "creates an image that is
mutable". Not setting db is no way out: the probes alone force it, because
health-enabled is a build option too. An empty emptyDir over that path fails as
well, because QuarkusEntryPoint loads quarkus/quarkus-application.dat before it
dispatches any command, `build` included. So one init container seeds the
directory into a volume, the next runs `kc.sh build` into it, and the serving
container runs `start --optimized` - every container keeping the full hardening
baseline, with no documented deviation to carry.

Build options therefore exist in exactly one place, and the asymmetry is
enforced rather than trusted: the serving container passes no build option at
all, and a build option's variable name in keycloak.env fails the render with a
message naming it.

Hostname handling is the security-relevant part. KC_HOSTNAME is derived from
ingress.domain as a full URL (which upstream requires alongside hostname-admin),
hostname-strict stays at its default true, and proxy-headers is set rather than
optional. The admin hostname is NOT presented as an access control: upstream
states that hostname-admin "does not prevent accessing the Administration REST
API endpoints via the frontend URL specified by the hostname option", so the
chart renders no /admin deny rule and values.yaml, NOTES.txt and the wiki all
say in the same words that restricting that API is reverse-proxy work.

The management port 9000 is a containerPort and a ClusterIP Service for
in-cluster scraping, and is in no Ingress: it serves /health/* and optionally
/metrics without authentication. Liveness uses /health/live and deliberately
not /health or /health/ready - those include the database, and a liveness probe
on them would restart the whole cluster for as long as the database is away.

Also in this change:
- First PodDisruptionBudget in this repository, which makes it a convention
  (README section 11): minAvailable as a value, and rendered only while the
  computed replica count is >= 2, because minAvailable: 1 with one replica
  blocks every node drain forever with a message that names the PDB rather
  than the replica count.
- First quay.io image, so quay.io joins the trusted registries in
  ci/trivy/data/ksv0125.yaml - configured rather than suppressed, so
  AVD-KSV-0125 keeps being able to say something.
- Renovate block with NO version constraint, which is the counter-case to
  garage and stalwart and is measured rather than assumed: docker versioning
  already excludes the -N rebuild tags, the floating minors and latest/nightly.
  Three dry-run fixtures are recorded at the block, and against the committed
  config 26.7.4 offers exactly 26.7.5 (patch) and 26.8.0 (minor).
- No automerge exception, by decision (Gitea #83): the global automerge applies,
  and the minor-upgrade procedure plus `kc.sh update-compatibility` is
  documented in NOTES.txt instead.
- Two upstream defaults deviated from, each with its reason at the value:
  db-pool-max-size 20 instead of 100 (two replicas at 100 ask a default
  PostgreSQL for twice its max_connections), and uid 1000 / gid 0 instead of
  this repo's 20000/1000, because the image declares keycloak:x:1000:0 and
  chowns its install tree to that pair.

Part of #83
2026-10-04 18:00:20 +00:00
jlab-renovate-bot a23d4977c5 Update urlaubsverwaltung/urlaubsverwaltung Docker tag to v6.15.0
renovate/stability-days Updates have met minimum release age requirement
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 9s
Validate Helm Charts / Build urlaubsverwaltung (pull_request) Successful in 4s
Validate Helm Charts / Trivy config urlaubsverwaltung (pull_request) Successful in 10s
Install-test / Install-test urlaubsverwaltung (pull_request) Successful in 2m3s
Validate Helm Charts / Install-test (pull_request) Successful in 2m3s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 8s
Publish Urlaubsverwaltung Helm Chart / Publish Urlaubsverwaltung Helm Chart (push) Successful in 7s
SonarQube Analysis / SonarQube Analysis (push) Successful in 28s
2026-10-04 15:40:59 +00:00
jlab-renovate-bot 639e0b5f57 Update docker.io/rssbridge/rss-bridge Docker tag to v2026
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
Publish RSS-Bridge Helm Chart / Publish RSS-Bridge Helm Chart (push) Successful in 7s
SonarQube Analysis / SonarQube Analysis (push) Successful in 27s
2026-10-04 15:12:22 +00:00
jlab-renovate-bot 236767070d Update dependency aquasecurity/trivy to v0.75.0
SonarQube Analysis / SonarQube Analysis (push) Successful in 28s
2026-10-04 15:11:02 +00:00
jlab-renovate-bot caea6e7353 Update Helm release oauth2-proxy to v10.7.1
renovate/stability-days Updates have met minimum release age requirement
Validate Helm Charts / Detect changed charts (pull_request) Successful in 7s
Validate Helm Charts / Resource quantities (pull_request) Successful in 25s
Validate Helm Charts / Build ytdlp-web-player (pull_request) Successful in 9s
Validate Helm Charts / Trivy config ytdlp-web-player (pull_request) Successful in 40s
Install-test / Install-test ytdlp-web-player (pull_request) Successful in 2m6s
Validate Helm Charts / Install-test (pull_request) Successful in 2m6s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 37s
Publish YTDLP Web Player Helm Chart / Publish YTDLP Web Player Helm Chart (push) Successful in 36s
SonarQube Analysis / SonarQube Analysis (push) Successful in 31s
2026-10-04 13:15:31 +00:00
jklein b19da4e580 fix(garage): a comment in the ConfigMap tripped Trivy's KSV-0109
Validate Helm Charts / Detect changed charts (pull_request) Successful in 2s
Validate Helm Charts / Resource quantities (pull_request) Successful in 11s
Install-test / Install-test garage (pull_request) Successful in 1m20s
Validate Helm Charts / Install-test (pull_request) Successful in 1m20s
Validate Helm Charts / Trivy config garage (pull_request) Successful in 16s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
Publish Garage Helm Chart / Publish Garage Helm Chart (push) Successful in 10s
SonarQube Analysis / SonarQube Analysis (push) Successful in 36s
Validate Helm Charts / Build garage (pull_request) Successful in 5s
Run 446 reported one HIGH finding, and it was prose, not data: the line
"metrics_require_token = true plus a token to close it" sits inside the
ConfigMap's data block, and KSV-0109 reads a <key> = <value> assignment with a
credential-ish key name as a stored secret - comments included. The ConfigMap
holds no credential at all; the RPC secret arrives as an environment variable
precisely so that it cannot.

Reworded rather than excepted: an entry in ci/trivy/trivyignore.yaml would
carry a permanent exception for one sentence, and the next reader of the report
would have to re-establish that the ConfigMap really is secret-free. The
replacement names the mechanism at the line, so the shape does not come back.
2026-10-03 20:45:56 +00:00
jklein 01336af953 docs(garage): state the admin surface precisely, including open /metrics
Validate Helm Charts / Detect changed charts (pull_request) Successful in 5s
Validate Helm Charts / Trivy config garage (pull_request) Successful in 15s
Install-test / Install-test garage (pull_request) Successful in 1m22s
Validate Helm Charts / Install-test (pull_request) Successful in 1m23s
Validate Helm Charts / Resource quantities (pull_request) Successful in 11s
Validate Helm Charts / Build garage (pull_request) Successful in 7s
"No admin_token is set" read like "nothing is open", and two parts of that
were wrong. /metrics needs no authentication either, because
metrics_require_token defaults to false - internal metrics on a ClusterIP port.
And a config token is not the only way to reach the authenticated API: Garage
2.x keeps admin tokens in the cluster as well ("garage admin-token create"),
which work without any entry in garage.toml. Both now stated at the ConfigMap
and the Service, plus the way to close /metrics.
2026-10-03 20:42:20 +00:00
jklein 5d10b02458 fix(garage): compressionLevel disables with "none", never with 0
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Trivy config garage (pull_request) Successful in 16s
Validate Helm Charts / Install-test (pull_request) Successful in 1m17s
Validate Helm Charts / Resource quantities (pull_request) Successful in 12s
Validate Helm Charts / Build garage (pull_request) Successful in 7s
Install-test / Install-test garage (pull_request) Successful in 1m17s
My own values comment claimed 0 switches compression off. It does not:
Garage's deserializer takes an integer OR the literal string "none"
(src/util/config.rs, deserialize_compression), and 0 is a valid integer that
zstd reads as its default level - 3, i.e. MORE compression than the chart's 1,
not none. A comment that invites an operator to write 0 for "off" is the
values-that-must-not-lie shape with the lie in the documentation instead of in
the template.

The schema now accepts an integer (negative included, those are zstd's fast
modes) or exactly "none", and the ConfigMap quotes the string form so the
rendered TOML stays valid. Verified: 1, 7, -3 render unquoted, "none" renders
quoted and the result still parses as TOML, and any other string is rejected by
the schema.

No version bump: the chart is not published yet.
2026-10-03 20:40:47 +00:00
jklein 79b7e44fdc feat(garage): new chart for the Garage object store
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 11s
Validate Helm Charts / Build garage (pull_request) Successful in 6s
Validate Helm Charts / Trivy config garage (pull_request) Successful in 16s
Install-test / Install-test garage (pull_request) Successful in 1m15s
Validate Helm Charts / Install-test (pull_request) Successful in 1m15s
One Garage node with replication_factor = 1, built from template-chart to the
style guide rather than copied from upstream's in-tree chart (Gitea #81). The
single-node trade is deliberate and its price is named at the value: the factor
applies to the metadata as well, where upstream calls a corrupted database
"generally not recoverable" and a replicated volume replicates the corruption
faithfully. The three countermeasures that work at factor 1 are therefore
mandatory, not optional - metadata_fsync = true (off in Garage's defaults), the
automatic metadata snapshots, and those snapshots on a second PVC.

Storage: PVC 1 carries metadata_dir AND data_dir as two subdirectories, because
volume snapshots are atomic per volume and that is what keeps index and blocks
consistent with each other; PVC 2 carries metadata_snapshots_dir, because a
recovery point must not die with the volume whose corruption it should repair.
Garage creates all three directories itself, which matters because the image is
FROM scratch and has no mkdir.

Two corrections to the approved plan, both verified against the source at tag
v2.4.1 and decided on the issue before implementation:

- The RPC secret arrives as GARAGE_RPC_SECRET via secretKeyRef, not through
  rpc_secret_file. The file form cannot work in a non-root pod:
  read_secret_file rejects any mode with a group or other bit set
  (src/garage/secrets.rs:144, mask 0o077) while the kubelet OR-s the group-read
  bit onto every file of a secret volume once fsGroup is set
  (pkg/volume/volume_linux.go), and Kubernetes can set the group of such a file
  but never its owner. The mechanism is documented at the value so the next
  round does not "fix" it back.
- The first start is done by the server, not by hand: garage server
  --single-node creates the layout in-process before any listener opens, is
  idempotent on the layout version and has no --version argument that could be
  reused, and --default-access-key/--default-bucket import key and bucket FROM
  the vault item. Both premises of the original "manual first start" decision
  fell with that.

Probes follow from the same source reading: readiness is GET /health on the
admin port, which answers 503 while no layout exists; liveness and startup are
a TCP connect, because a probe that can only fail would restart a node waiting
for its layout forever (same lesson as zipline, README 12.3). The S3 listener
binds independently of the layout, so a TCP probe on 3900 would have been the
lie the plan warned about.

CI: fixtures under ci/garage/ rather than an exclusion - with --single-node the
pod reaches Ready on its own, which makes the install-test an assertion about
the layout, the one-PVC subdirectory layout and readOnlyRootFilesystem instead
of a formality. The Renovate block carries a regex versioning anchored to
^v<major>.<minor>.<patch>$, because the registry receives 40-character
commit-SHA tags almost daily (measured 2026-10-03: the newest 22 tags are all
of that kind) and the four v1.99.x-internal builds sort above the v1.3.x line.

Part of #81
2026-10-03 20:29:38 +00:00
jlab-renovate-bot bf2e1009e4 Update docker.io/library/busybox Docker tag to v1.38.0
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
Publish Stalwart Helm Chart / Publish Stalwart Helm Chart (push) Successful in 10s
SonarQube Analysis / SonarQube Analysis (push) Successful in 31s
2026-10-02 22:07:25 +00:00
jklein a9ca5302de feat(paperless-ngx): optional valkey subchart, password from the redis item
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 12s
Validate Helm Charts / Build paperless-ngx (pull_request) Successful in 6s
Validate Helm Charts / Trivy config paperless-ngx (pull_request) Successful in 22s
Install-test / Install-test paperless-ngx (pull_request) Successful in 3m31s
Validate Helm Charts / Install-test (pull_request) Successful in 3m31s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
Publish Paperless-NGX Helm Chart / Publish Paperless-NGX Helm Chart (push) Successful in 10s
SonarQube Analysis / SonarQube Analysis (push) Successful in 33s
paperless-ngx cannot run without a Redis - PAPERLESS_REDIS is the Celery
broker - so every installation needed a second Helm release whose address
lived in a 1Password item and in neither chart. This adds our own published
valkey chart as an optional dependency, off by default and gated on
valkeySubchart.enabled (not valkey.enabled, which the subchart's strict
schema rejects and which Helm would only have complained about for whoever
first switched it on). Additive, default behaviour unchanged: minor,
12.1.3+up3.2.1 -> 12.2.0+up3.2.1.

Configured as a QUEUE (PVC + noeviction, the subchart's defaults), unlike
stalwart's cache. Read off the 3.2.1 source rather than assumed: one
PAPERLESS_REDIS url serves five consumers - the Celery broker, the result
backend (Redis in 3.x, the database before), the channels pub/sub layer and
two Django caches. allkeys-lru would not be a trade here but actively wrong,
because LRU evicts typeless across the whole keyspace and a backlog of
waiting consume tasks is exactly what it reads as cold; a dropped task is a
document silently never processed. The price, named in values.yaml rather
than smoothed over: two of the five consumers are caches, so a full instance
refuses writes - new tasks included - and the answer is memory sizing, not a
policy change. All three cache-like consumers are TTL-bounded, which is what
keeps that price small.

The password comes from the SAME item as the Redis connection, in a third
field auth-conf holding the complete line "requirepass <password>". It cannot
DEFAULT to secrets.redisSecretRef, and that is a property of Helm, not an
omission: a values.yaml is never templated and a parent cannot compute a
subchart's value. Measured - a template expression there lands verbatim in
the subchart's itemPath and fails as a YAML parse error naming neither the
value nor the cause.

Forgetting it is therefore caught in values.schema.json, by a conditional
rule rather than a fail() in a template. Also measured, and the reason:
Helm renders a subchart's templates BEFORE the parent's, so a guard in this
chart loses the race to the subchart's own required(), which reports
"secrets.authSecretRef" and cannot know the path belongs to the redis item.
A check that cannot fire is the mistake found while testing #77, so the
first draft of that guard was removed rather than shipped. The schema runs
before any rendering and names /valkey/secrets/authSecretRef.

One trap found while testing and closed here: "valkeySubchart: null" DELETES
the key, Helm then cannot resolve the condition path and treats the
dependency as ENABLED - so erasing the switch turned the subchart on with no
password set. Unlike every other block in this chart (issue #99), that one
is rejected by the schema instead of defaulting.

The install-test now runs with the subchart ON and the throwaway redis
Deployment removed: the url in the fixtures names ci-valkey-service, so the
bundled Valkey is the broker the release actually uses. #65 covered this
path by a render only - deliberately, since an unused Valkey would have made
that test look like it covered clustering. This is the first real
installation of the valkey-subchart path.

Documented, not glossed: the password now sits in that item twice and a
rotation has to touch both fields; with the subchart on the vault url is a
mandatory step, not a hint; and a OnePasswordItem materializes every field
of its item, so both Secrets carry all three keys - untidy, harmless, and
written down so nobody "tidies it up".

Fixes #78
2026-10-02 21:24:22 +00:00
jlab-renovate-bot a288ee231a Pin dependencies
SonarQube Analysis / SonarQube Analysis (push) Failing after 20s
2026-10-02 21:02:13 +00:00
jklein 07c75e80c1 feat(stalwart): take our own published valkey chart as the subchart
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 14s
Publish Stalwart Helm Chart / Publish Stalwart Helm Chart (push) Successful in 14s
SonarQube Analysis / SonarQube Analysis (push) Failing after 13s
Replaces the dependency on upstream's valkey chart with this repo's own,
resolved from the published index:

  version: "1.0.0+up9.1.2.alpine"
  repository: "https://git.jk-dev.de/api/packages/jlab-cluster/helm"

The condition is "valkeySubchart.enabled", NOT "valkey.enabled", and that is
forced rather than stylistic. Everything under "valkey:" is handed to the
subchart as its values and validated against ITS schema, and our valkey chart
ships additionalProperties: false with no "enabled" property - so
"valkey.enabled" is rejected by the subchart:

  valkey: - at '': additional properties 'enabled' not allowed

Measured. And because Helm skips a DISABLED subchart's schema validation, that
error would have surfaced only for whoever first set it to true: every render,
lint and install-test in CI would have stayed green. Adding "enabled" to
valkey/values.schema.json was rejected as the repair - that chart does not read
the key, and it would have needed its own PR and a publish first.

The host helper was also wrong for our chart and is fixed: our Service is
"<release>-valkey-service", not upstream's bare "<release>-valkey".

The values block is rewritten for our chart's schema, with the pair chosen
rather than inherited: CACHE (persistence off + allkeys-lru), because
everything Stalwart keeps in the in-memory store - rate limiters, auth tokens,
sessions, auto-ban entries, coordination pub/sub - is reconstructible, and the
node-id lease lives in the data store. The cost is named where it is set: with
allkeys-lru the rate limiters and the auto-ban FAIL OPEN under memory pressure,
so the answer is to size memory, not to switch to noeviction (which would make
writes error and take out authentication instead). secrets.authSecretRef wants
an item whose single field "auth-conf" holds the complete "requirepass <pw>"
line, not a bare password.

Chart.yaml's comments are corrected where they became false: nothing is
"consumed as it comes" any more and there is no shortfall of someone else's to
document. The Renovate reasoning stays and is now backed by measurement.

New trap, in Chart.yaml and in three places in the wiki: NEVER change valkey/
and a chart depending on it in the same PR. The dependency resolves from the
published index, so a combined PR lints, packages, renders AND install-tests
against the OLD subchart - green on an artifact that is not the one in the
diff. Two consecutive PRs.

Two things measured, not assumed, because either could have broken the update
path with no symptom but "no updates ever":
- helm dependency update AND build resolve the exact build-metadata version
  from the URL alone with an EMPTY HELM_REPOSITORY_CONFIG - a fresh runner's
  state - so the anonymous index needs no credential and
  ci/helm-dependencies.sh is untouched;
- Renovate 44.132.2 dry run with the dependency pinned low: manager helmv3
  matched the file, took registryUrls from Chart.yaml and offered
  newVersion "1.0.0+up9.1.2.alpine", versioning "helm". The self-hosted index
  and the "+up..." suffix are both handled.

README: the claim "no registry address appears in this repository" is narrowed
to what still holds - the publish target and its credentials come from
1Password and are not in the repo. The read path is now in Chart.yaml by
necessity, carries no credential, and the host was always in CLAUDE.md. Done
here rather than later so the paperless PR finds it already corrected.

Re-verified after the change: helm lint --strict (0 failed), helm package (the
subchart is embedded in the .tgz), helm template with the subchart OFF and ON
(ON was previously impossible), against CI test-values, the rendered
config.json, ci/run-quantity-render-test.sh (0 failed), NOTES.txt through the
scratch-ConfigMap trick, and the fail paths including the reworded abort. The
rendered subchart Service name matches what the helper emits, the cache pairing
really renders (maxmemory-policy allkeys-lru, appendonly no, save "", emptyDir)
and no PVC appears.

Part of #65
2026-10-02 20:59:17 +00:00
jklein 780cb8eafd feat(stalwart): new chart for the Stalwart mail server
Cluster-capable from the start, with roles rather than symmetry: one
StatefulSet per entry of the "nodes" map, the key passed in STALWART_ROLE.

Since v0.16 Stalwart keeps its whole configuration inside the data store and
manages it over JMAP, so the chart's scope is deliberately narrow: it renders
config.json (exactly the DataStore object), the workloads, the Services, the
NetworkPolicies, the Ingress and a cert-manager Certificate. Listeners, blob
store, in-memory store, TLS object and cluster roles are operator work in the
WebUI, which is why NOTES.txt carries an eight-step first install.

config.json stays secret-free: authSecret uses the EnvironmentVariable variant
(verified against the serde definition of PostgreSqlStore at v0.16.24), so the
password arrives through secretKeyRef and checksum/config stays meaningful.
The username cannot follow it - authUsername is a plain String with no secret
form - so it is a public value and says so.

Services are per PORT, not per group: a NodePort is cluster-unique, so "full"
and "mx" can only accept SMTP behind one Service, selected through a per-port
pod label. The http port gets a per-group ClusterIP Service instead, so the
Ingress reaches exactly ingress.nodeGroup.

Four traps, documented rather than explained away:

- The role lookup FAILS OPEN. A node whose ClusterRole is missing from the
  database enables every listener and every task while staying Running with a
  green probe, and the log line arrives after the listeners are bound. Hence
  mx at replicas 0, per-port Services, and a NetworkPolicy that is a ceiling
  independent of what was bound.
- NET_BIND_SERVICE is a documented deviation needed to EXEC, not to bind. The
  image sets setcap cap_net_bind_service=+ep, which makes the kernel fail
  execve when the bounding set is empty. The values comment says both why it
  is there and that it is never used, because the second half is what stops
  the next hardening pass from removing it.
- externalTrafficPolicy must be Cluster - no pod runs on a control-plane node
  while the HA proxy delivers to them, so Local would drop every packet - and
  the source address that costs is restored only by the HA proxy's PROXY v2
  header.
- /healthz is rate-limited to 100 requests/60s per source IP, which is a node
  IP for kubelet probes, so all pods on a node share one bucket. The probe
  intervals are calculated (6 req/min/pod, ~16 pods per node) and the sum is a
  comment in values.yaml.

A first install needs bootstrap.enabled: true: the setup wizard appears only
when config.json is ABSENT, and this chart always renders one, so recovery
mode is upstream's documented path for a templated config.json.

Second chart in the repo with a dependency - the Valkey project's own chart,
off by default behind condition:, because Stalwart's cluster coordinator rides
on the in-memory store connection. The render aborts when more than one pod
runs with no in-memory store address anywhere.

Also wired up: publish-stalwart.yml through the shared block in
workflow-util/pipelines, the change-app-version.yml choice list, a Renovate
customManagers block anchored on the v prefix (which excludes the moving
latest / v0.16 / -alpine tags), CI fixtures, and the README and CLAUDE.md
lines that claimed ytdlp-web-player was the only chart with a dependency.

Verified locally: ci/helm-dependencies.sh, helm lint --strict (0 failed),
helm package, helm template against chart defaults and against
ci/stalwart/test-values.yaml, the rendered config.json parsed as JSON with the
field types the serde definition requires, ci/run-quantity-render-test.sh
(0 failed, both nodes.*.resources paths exercised), NOTES.txt rendered through
a scratch ConfigMap, and the null-safety and fail paths (securityContext=null
keeps NET_BIND_SERVICE; a cluster without an in-memory store, an unknown port
name, a group without http and an unknown ingress.nodeGroup all abort with a
named message; the schema rejects timeout, a string timeoutMs and a list-shaped
nodes). NO runtime verification was possible - there is no k3d or Docker daemon
in this workspace.

Part of #65
2026-10-02 20:59:17 +00:00
jlab-renovate-bot ee43c33ef4 Update dependency kubernetes/kubernetes to v1.35.9
renovate/stability-days Updates have met minimum release age requirement
validate-charts.yml / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Blocked by required conditions
validate-charts.yml / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Blocked by required conditions
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Skipped
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
Validate Helm Charts / Resource quantities (pull_request) Successful in 10s
SonarQube Analysis / SonarQube Analysis (push) Failing after 24s
2026-10-02 16:13:30 +00:00
jlab-renovate-bot 9ae7eef99f Update lukevella/rallly Docker tag to v4.15.3
renovate/stability-days Updates have met minimum release age requirement
Validate Helm Charts / Detect changed charts (pull_request) Successful in 49s
Validate Helm Charts / Resource quantities (pull_request) Successful in 11s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 16s
Install-test / Install-test ${{ fromJSON(inputs.charts) }} (pull_request) Successful in 2m26s
Validate Helm Charts / Install-test (pull_request) Successful in 2m26s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Successful in 6s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 58s
Publish Rallly Helm Chart / Publish Rallly Helm Chart (push) Successful in 58s
SonarQube Analysis / SonarQube Analysis (push) Failing after 15s
2026-10-02 15:44:23 +00:00
jklein f284d209f7 sonar: correct the project key, and record what the first run measured
SonarQube Analysis / SonarQube Analysis (push) Successful in 42s
Validate Helm Charts / Resource quantities (pull_request) Successful in 9s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Skipped
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
The project key was a guess. The project exists in SonarQube as
JLab-Helm-Charts; jlab-cluster_helm-charts was reasoned from the instance
serving two owners, and it produced the one error message that covers both a
non-existent project and a token without permission on it:

  ERROR You're not authorized to analyze this project or the project doesn't
        exist on SonarQube and you're not authorized to create it.

Because the server does not distinguish those two cases, the next occurrence of
that message now means the token lacks "Execute Analysis" - which is worth a
comment in the file rather than a second round of guessing.

sonar.projectName goes away entirely. Once the project exists on the server the
display name belongs to the server, and a name sent with every analysis would
overwrite a rename there on every run, silently. The reason is a comment in the
file so the line is not added back as an obvious omission.

The README analyser table was built from SonarQube's language list and promised
more than this instance delivers. Measured in run 375: two quality profiles
loaded, json and yaml; the IaC Kubernetes sensor (3.4 s) and the secrets
detection over all 391 indexed files did the work. Shell and GitHub Actions did
not run, and no configuration here can make them - analysers are server-side
plugins. The consequence is now stated: the five ci/ scripts and the thirty
workflow files, the files carrying the actual logic, are not analysed at all.
Whether the instance gains those plugins is a decision outside this repository.

Run 375 also answered the three questions this path carried, all positively:
node24 is in the runner image, the ~59 MiB scanner download and its signature
arrived in roughly three seconds (so the MTU failure that once killed a 17 MB
download does not repeat), and a repository variable does reach a workflow
called from another repository - which settles the identical question the
publish route carried for OP_CONNECT_HOST. The publish route's upload is still
unproven and stays listed as such.

Fixes #74
2026-10-02 00:15:06 +00:00
jklein 67d8c94fb3 ci: call the shared SonarQube block, and say what it does not do
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Skipped
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
Validate Helm Charts / Resource quantities (pull_request) Successful in 9s
SonarQube Analysis / SonarQube Analysis (push) Failing after 1m0s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Adds the caller side of the analysis: .github/workflows/sonar.yml as a hull of
the same kind as the publish ones, and sonar-project.properties as the
project-specific half. The workflow itself is the shared block in
workflow-util/pipelines, bound to @v4.

No paths filter, unlike the publish hulls: the analysis looks at the whole tree
- the five shell scripts under ci/, every workflow file, the YAML and Helm
templates, the values.schema.json files, and the secret detection over
everything. A filter would leave the previous result standing after a change it
did not cover while still presenting it as current.

workflow_dispatch is on the hull on purpose. Three things about this path are
unmeasured - node24 in the runner image, the ~59 MiB scanner download, and
whether a repository variable reaches a workflow called from another repository
- and any of them may need more than one attempt. Without a dispatch every
attempt would cost a dummy commit on main.

No exclusions, including the Helm templates, which are not valid YAML because
of {{ }}. What the Helm and YAML analysers make of them is only learnable by
letting them run over them once, and an exclusion written on a guess hides
whatever else matches it. If the first run produces noise worth suppressing,
the exclusion belongs in sonar-project.properties with the finding it answers
named next to it.

The project key carries the owner (jlab-cluster_helm-charts) because the
instance serves two, and a key cannot be changed afterwards without losing the
project's history.

The analysis is not a gate. The Community Build analyses the main branch only,
so the trigger is a push to main and a finding appears after the merge - at
which point there is nothing left to block. README and the CI-CD wiki page both
say so, with the two conditions under which a gate would become right, so that
its absence is not read as an oversight.
2026-10-01 23:49:48 +00:00
jklein 8878329f31 ci: publish charts through the shared block in workflow-util/pipelines
Validate Helm Charts / Detect changed charts (pull_request) Successful in 6s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Skipped
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Skipped
Validate Helm Charts / Resource quantities (pull_request) Successful in 13s
Validate Helm Charts / Install-test (pull_request) Skipped
The reusable publish workflow now lives in workflow-util/pipelines and is
called pinned to @v3. All 25 thin hulls switch over and our copy is deleted in
the same commit: a hull pointing at a deleted file, or an orphaned file nobody
calls, would both be worse than the state before.

Each hull gains one line, registry-owner: ${{ github.repository_owner }}. The
owner used to be derived inside the workflow (#63); as a required input without
a default it is evaluated here, in the calling repository, where the value is
measured. That is the house rule of pipelines - a shared block never guesses an
owner segment - and it keeps the block free of any context semantics. The
workflow_dispatch and push triggers of every hull are untouched; after the move
the dispatch is the only way to trigger a publish by hand.

The Connect host is no longer wired into the block. It reads the repository
variable OP_CONNECT_HOST, which the maintainer has set, and fails in its first
step naming the variable when it is empty.

Documentation follows the same split as the code. The README entry no longer
describes a workflow of this repository; it names where the block lives, what
each caller owes it, and points at the publish-helm-chart page of that
repository's wiki for the reasons behind the upload step, which are no longer
ours to keep current. Nothing of them was lost in the move - the upload step's
shell body went across unchanged apart from two error messages that name the
owner as an input.

Also corrected, because the move made them wrong rather than merely stale:

- the helm-dependencies.sh entry still counted the local publish workflow among
  the five places it runs; it is still five, one of them now the shared block,
  which is why that block names the script as a prerequisite of its callers;
- two comments in install-test.yml pointed at publish-helm-chart.yml as a
  sibling file, one of them for a trap it documented "at length" - that long
  form is now on the pipelines wiki, and the comments say so;
- the hull count read 24, and reusable workflows with secrets: inherit are now
  confirmed to work across repositories too;
- the chart-creation step in CLAUDE.md said the reusable workflow is ours.

install-test.yml stays here, with the reason written down (decision 4 on the
issue): it is reusable in form but welded to the ci/ fixtures, the exclusion
parser and this repository's chart matrix. A block only one repository can use
is not shared, only moved. The note is there so the question is not asked again.

Fixes #66
2026-10-01 23:01:47 +00:00
jlab-renovate-bot f3ad3a9844 Update urlaubsverwaltung/urlaubsverwaltung Docker tag to v6.14.0
Publish Urlaubsverwaltung Helm Chart / Publish Urlaubsverwaltung Helm Chart (push) Successful in 8s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 7s
2026-09-30 18:57:54 +00:00
jklein b614b53602 ci: remove the temporary repository_owner probe, record the measurement
Validate Helm Charts / Resource quantities (pull_request) Successful in 7s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 2s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Skipped
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
github.repository_owner is populated on this Gitea instance: measured as
"jlab-cluster" (12 characters) in run 282 of validate-charts.yml. The
value is now in the comment above the upload step, so the fact survives
without the step that produced it.
2026-09-30 18:51:55 +00:00
jklein a12983fbea ci(publish): compose the registry URL from a base and the repository owner
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
Validate Helm Charts / Resource quantities (pull_request) Successful in 8s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 15s
charts-url in op://GitHub-Actions/jlab-registry-access held the full
registry URL and publish-helm-chart.yml appended /api/charts to it. It
now holds only the base up to /api/packages, and the owner segment comes
from github.repository_owner, so the target is composed as
<base>/<owner>/helm/api/charts.

Nothing in this repository points at the registry any more, not even a
path segment, and the second owner on the same instance needs no second
setting. The 26 thin publish-<chart>.yml callers are unchanged.

Two guards keep the switch from the old shape loud rather than a 404
against a masked URL: charts-url is rejected unless it ends in
/api/packages, naming /helm, /api/charts or an included owner
explicitly; and the derived owner is rejected when empty or when it is
not a usable URL path segment. Everything the upload step already had
is untouched - the per-field :? guards, the whitespace check on the
credentials, the host masking, the built file name, --fail-with-body.

The probe step in validate-charts.yml is temporary and measures whether
this forge populates github.repository_owner at all; it is removed again
once the value has been read off the run.
2026-09-30 18:49:56 +00:00
128 changed files with 12770 additions and 1718 deletions

No files matched your search

+7 -4
View File
@@ -40,7 +40,10 @@ on:
- ytdlp-web-player
- sure
- valkey
- convertx
- stalwart
- garage
- keycloak
- bulwark
# freesailarr is deliberately absent: none of its ten images takes
# its tag from appVersion (they are all pinned in its values.yaml),
# so this workflow would rewrite appVersion and republish the chart
@@ -67,18 +70,18 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: "Configure OP-Connect"
uses: 1password/load-secrets-action/configure@v5
uses: 1password/load-secrets-action/configure@70062d7a876d3eb6334754fa26efd2fbd90c32f2 # v5
with:
connect-host: https://op-connect.jk-dev.de
- name: "Load GitHub Credentials"
id: load-jk-cluster-bot-creds
uses: 1password/load-secrets-action@v5
uses: 1password/load-secrets-action@70062d7a876d3eb6334754fa26efd2fbd90c32f2 # v5
env:
OP_CONNECT_TOKEN: ${{ secrets.OP_CONNECT_TOKEN }}
JK_CLUSTER_BOT_TOKEN: op://GitHub-Actions/github-helm-charts-push-token/credential
JK_CLUSTER_USERNAME: op://GitHub-Actions/github-helm-charts-push-token/username
JK_CLUSTER_EMAIL: op://GitHub-Actions/github-helm-charts-push-token/email
- uses: actions/checkout@v7
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
token: ${{ env.JK_CLUSTER_BOT_TOKEN }}
- name: "Get Chart Version"
@@ -36,7 +36,7 @@ jobs:
count: ${{ steps.select.outputs.count }}
steps:
- name: "Checkout Repository"
uses: actions/checkout@v7
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: "Select the charts to test"
id: select
env:
+14 -11
View File
@@ -30,9 +30,10 @@ on:
OP_CONNECT_TOKEN:
# Needed by exactly one leg - freesailarr, for the 1Password Connect
# lookup of its CI Wireguard key. Declared OPTIONAL on purpose, where
# publish-helm-chart.yml declares its token required: on this instance a
# missing secret does not fail the job, it fails job CREATION, so the
# run shows no steps at all (README, "Where this repo lives"). With
# the shared publish block in workflow-util/pipelines declares its token
# required: on this instance a missing secret does not fail the job, it
# fails job CREATION, so the run shows no steps at all (README, "Where
# this repo lives"; the long form is on that repository's wiki). With
# required: true a pull request from a fork - which carries no secrets -
# would lose the entire install-test that way, every chart at once, with
# nothing naming the cause. Optional instead: only the freesailarr leg
@@ -119,7 +120,7 @@ jobs:
# invisible there. The value mirrors K3S_IMAGE above (v1.35.5), which is
# the cluster this client talks to.
# renovate: datasource=github-releases depName=kubernetes/kubernetes
KUBECTL_VERSION: v1.35.5
KUBECTL_VERSION: v1.37.1
# The same expression as runs-on above, written a second time because the
# env context is not available in runs-on - so it cannot be the other way
# round, and reading the label back off the runner is not possible either
@@ -131,9 +132,9 @@ jobs:
RUNNER_LABEL: ${{ github.server_url == 'https://github.com' && 'ubuntu-latest' || 'ubuntu-docker' }}
steps:
- name: "Checkout Repository"
uses: actions/checkout@v7
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: "Install Helm"
uses: azure/setup-helm@v5
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
- name: "Install kubectl"
run: |
# kubectl and the cluster are a pair: the client talks to the server
@@ -196,8 +197,8 @@ jobs:
# freesailarr is the one chart whose install-test needs credentials:
# without a real Wireguard key gluetun never completes a handshake, and
# behind its kill switch not a single app container becomes Ready. The key
# comes from 1Password Connect like the registry credentials in
# publish-helm-chart.yml, and it is a CI-ONLY Mullvad key: Mullvad ties an
# comes from 1Password Connect like the registry credentials in the shared
# publish block, and it is a CI-ONLY Mullvad key: Mullvad ties an
# address to a key and sends the tunnel to the endpoint it saw last, so
# reusing the production key would let every PR run pull the tunnel out
# from under the running stack - with the symptom in the cluster and the
@@ -205,18 +206,20 @@ jobs:
# touches the VPN key.
- name: "Configure OP-Connect"
if: matrix.chart == 'freesailarr'
uses: 1password/load-secrets-action/configure@v5
uses: 1password/load-secrets-action/configure@70062d7a876d3eb6334754fa26efd2fbd90c32f2 # v5
with:
connect-host: https://op-connect.jk-dev.de
- name: "Load freesailarr VPN credentials"
id: load-freesailarr-vpn
if: matrix.chart == 'freesailarr'
uses: 1password/load-secrets-action@v5
uses: 1password/load-secrets-action@70062d7a876d3eb6334754fa26efd2fbd90c32f2 # v5
env:
OP_CONNECT_TOKEN: ${{ secrets.OP_CONNECT_TOKEN }}
FREESAILARR_WG_KEY: op://GitHub-Actions/freesailarr-mullvad-test-config/credential
FREESAILARR_WG_ADDRESSES: op://GitHub-Actions/freesailarr-mullvad-test-config/ip-address
# The same trap publish-helm-chart.yml documents at length: v5 of
# The same trap the shared publish block documents at length - since
# Gitea #66 that long form lives on the wiki of workflow-util/pipelines,
# page "publish-helm-chart", not in this repository: v5 of
# load-secrets-action does NOT export the loaded values as environment
# variables for later steps, only its step outputs carry them, and the
# loading step logs "Populating variable" and stays green either way. So
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-apache-tika:
name: Publish Apache Tika Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'apache-tika'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-bentopdf:
name: Publish BentoPDF Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'bentopdf'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+16
View File
@@ -0,0 +1,16 @@
name: Publish Bulwark Helm Chart
on:
workflow_dispatch:
push:
branches:
- main
paths:
- bulwark/**
jobs:
publish-bulwark:
name: Publish Bulwark Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'bulwark'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,7 +9,8 @@ on:
jobs:
publish-calibre-web-automated:
name: Publish Calibre Web Automated Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'calibre-web-automated'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
-15
View File
@@ -1,15 +0,0 @@
name: Publish ConvertX Helm Chart
on:
workflow_dispatch:
push:
branches:
- main
paths:
- convertx/**
jobs:
publish-convertx:
name: Publish ConvertX Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'convertx'
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-cyberchef:
name: Publish Cyberchef Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'cyberchef'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-freesailarr:
name: Publish Freesailarr Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'freesailarr'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+16
View File
@@ -0,0 +1,16 @@
name: Publish Garage Helm Chart
on:
workflow_dispatch:
push:
branches:
- main
paths:
- garage/**
jobs:
publish-garage:
name: Publish Garage Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'garage'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-gotenberg:
name: Publish Gotenberg Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'gotenberg'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
-149
View File
@@ -1,149 +0,0 @@
name: Publish Helm Chart with patched Application Version
on:
workflow_dispatch:
inputs:
chart-location:
required: false
type: string
default: "."
description: "location of the helm chart directory"
chart-name:
required: true
type: string
description: "name of the helm chart"
workflow_call:
inputs:
chart-location:
required: false
type: string
default: "."
description: "location of the helm chart directory"
chart-name:
required: true
type: string
description: "name of the helm chart"
outputs:
chart-version:
description: "the version of the published helm chart"
value: ${{ jobs.publish-helm-chart.outputs.chart-version }}
secrets:
OP_CONNECT_TOKEN:
required: true
jobs:
publish-helm-chart:
outputs:
chart-version: ${{ steps.get-chart-version.outputs.CHART_VERSION }}
runs-on: ubuntu-latest
steps:
- name: "Checkout Repository"
uses: actions/checkout@v7
- name: "Install Helm"
uses: azure/setup-helm@v5
- name: "Configure OP-Connect"
uses: 1password/load-secrets-action/configure@v5
with:
connect-host: https://op-connect.jk-dev.de
# helm package exits 1 with "found in Chart.yaml, but missing in charts/
# directory" while a declared dependency has not been fetched, so this
# comes first. It is a no-op for every chart without a dependencies:
# block; ci/helm-dependencies.sh explains why it resolves from Chart.yaml
# instead of a committed Chart.lock. Note the consequence for the
# published artifact: the subchart .tgz is packaged INTO ours, so the
# version pinned in Chart.yaml is the version operators get, and nothing
# downstream ever sees the dependency again.
- name: "Fetch chart dependencies"
run: ${{ github.workspace }}/ci/helm-dependencies.sh "${{ inputs.chart-name }}"
working-directory: ${{ inputs.chart-location }}
- name: "Package Helm Chart"
run: helm package ${{ inputs.chart-name }}
working-directory: ${{ inputs.chart-location }}
- name: "Get Helm Chart Version"
id: get-chart-version
run: |-
CHART_VERSION=$(grep '^version:' ${{ inputs.chart-name}}/Chart.yaml | awk '{print $2}')
echo $CHART_VERSION
echo "CHART_VERSION=$CHART_VERSION" >> $GITHUB_OUTPUT
working-directory: ${{ inputs.chart-location }}
- name: "Load Chart Registry Credentials"
id: load-registry-creds
uses: 1password/load-secrets-action@v5
env:
OP_CONNECT_TOKEN: ${{ secrets.OP_CONNECT_TOKEN }}
REGISTRY_PASSWORD: op://GitHub-Actions/jlab-registry-access/password
REGISTRY_USERNAME: op://GitHub-Actions/jlab-registry-access/username
REGISTRY_CHARTS_URL: op://GitHub-Actions/jlab-registry-access/charts-url
# The target is a classic, index-based Helm registry, not an OCI one, so
# there is no "helm registry login" session and no "helm push": the upload
# is a single HTTP POST of the packaged .tgz to <charts-url>/api/charts,
# authenticated per request. Three properties of this step are load-bearing
# and each of them was paid for once already.
#
# 1) The values are mapped into the step's own env block and read as quoted
# shell variables - never interpolated into the command line. load-secrets-
# action v5 (bumped in #140) no longer exports the loaded values as
# environment variables for later steps the way v4 did; only the step
# outputs carry them. Every "${{ env.* }}" reference therefore resolved to
# an empty string from that bump on, with nothing reporting an error - the
# loading step logs "Populating variable" and stays green either way. The
# ":?" guards fail loudly and name the op:// field that is missing, and
# quoting keeps an empty value from silently shifting arguments.
#
# 2) The credentials never reach the process list. "curl --user $U:$P"
# would put them in argv; "curl --config -" reads its options from stdin
# instead, so nothing sensitive appears in any argument. The price of that
# config file is its value syntax: an unquoted value ends at the first
# space (measured on curl 8.5.0 - "has space" arrived as "has"), so the
# step refuses whitespace in the credentials up front rather than letting
# a truncated password come back as a puzzling 401. A Gitea access token
# is 40 hex characters, so this only ever fires on a mis-entered item.
#
# 3) "--fail-with-body" is what makes a failed upload fail the job. Unlike
# "helm push", curl exits 0 on HTTP 401/403/409 unless asked not to - the
# publish would go green while nothing was uploaded. Measured against Gitea
# 1.24.7: a 401 without the flag is exit 0, with the flag exit 22, and the
# response body is still printed so the log names the cause.
#
# The file name is built from the chart name and the version read above
# instead of a "<chart>-*.tgz" glob: --upload-file takes exactly one file,
# and a glob matching two turns the second into curl's URL argument, while
# a glob matching none is passed through literally. Both fail obscurely.
# "helm package" keeps the "+" of a "1.4.0+up1.9.2" version verbatim in the
# file name, so "<name>-<version>.tgz" is an exact match (measured).
- name: "Upload Helm Chart to the Chart Registry"
env:
REGISTRY_PASSWORD: ${{ steps.load-registry-creds.outputs.REGISTRY_PASSWORD }}
REGISTRY_USERNAME: ${{ steps.load-registry-creds.outputs.REGISTRY_USERNAME }}
REGISTRY_CHARTS_URL: ${{ steps.load-registry-creds.outputs.REGISTRY_CHARTS_URL }}
CHART_NAME: ${{ inputs.chart-name }}
CHART_VERSION: ${{ steps.get-chart-version.outputs.CHART_VERSION }}
run: |-
: "${REGISTRY_CHARTS_URL:?empty - check op://GitHub-Actions/jlab-registry-access/charts-url}"
: "${REGISTRY_USERNAME:?empty - check op://GitHub-Actions/jlab-registry-access/username}"
: "${REGISTRY_PASSWORD:?empty - check op://GitHub-Actions/jlab-registry-access/password}"
: "${CHART_NAME:?empty - the chart-name input did not arrive}"
: "${CHART_VERSION:?empty - could not read version: from ${CHART_NAME}/Chart.yaml}"
# 1Password masks the full URL, but a curl connection error prints only
# "<host>:<port>", which that mask would not cover. Mask the host too.
CHARTS_HOST="${REGISTRY_CHARTS_URL#*://}"
echo "::add-mask::${CHARTS_HOST%%/*}"
case "${REGISTRY_USERNAME}${REGISTRY_PASSWORD}" in
*[[:space:]]*)
echo "::error::username/password in op://GitHub-Actions/jlab-registry-access must not contain whitespace - curl --config truncates an unquoted value at the first space"
exit 1
;;
esac
CHART_PACKAGE="${CHART_NAME}-${CHART_VERSION}.tgz"
test -f "$CHART_PACKAGE" || {
echo "::error::${CHART_PACKAGE} does not exist - helm package produced: $(ls -1 ./*.tgz 2>&1)"
exit 1
}
echo "uploading ${CHART_PACKAGE}"
printf 'user = %s:%s\nurl = %s\n' \
"$REGISTRY_USERNAME" "$REGISTRY_PASSWORD" "${REGISTRY_CHARTS_URL%/}/api/charts" |
curl --config - \
--fail-with-body \
--silent --show-error \
--request POST \
--upload-file "$CHART_PACKAGE" \
--write-out '\nHTTP %{http_code}\n'
working-directory: ${{ inputs.chart-location }}
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-homeassistant:
name: Publish Homeassistant Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'homeassistant'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+16
View File
@@ -0,0 +1,16 @@
name: Publish Keycloak Helm Chart
on:
workflow_dispatch:
push:
branches:
- main
paths:
- keycloak/**
jobs:
publish-keycloak:
name: Publish Keycloak Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'keycloak'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-matrix-synapse:
name: Publish Matrix Synapse Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'matrix-synapse'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,7 +9,8 @@ on:
jobs:
publish-minecraft-router:
name: Publish Minecraft-Router Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'minecraft-router'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,7 +9,8 @@ on:
jobs:
publish-minecraft-server:
name: Publish Minecraft-Server Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'minecraft-server'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-outline:
name: Publish Outline Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'outline'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-paperless-ngx:
name: Publish Paperless-NGX Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'paperless-ngx'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-patchmon:
name: Publish Patchmon Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'patchmon'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-rallly:
name: Publish Rallly Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'rallly'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-rss-bridge:
name: Publish RSS-Bridge Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'rss-bridge'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-samba:
name: Publish Samba Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'samba'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-satisfactory:
name: Publish Satisfactory Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'satisfactory'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-searxng:
name: Publish SearXNG Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'searxng'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+16
View File
@@ -0,0 +1,16 @@
name: Publish Stalwart Helm Chart
on:
workflow_dispatch:
push:
branches:
- main
paths:
- stalwart/**
jobs:
publish-stalwart:
name: Publish Stalwart Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'stalwart'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-sure:
name: Publish Sure Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'sure'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,7 +9,8 @@ on:
jobs:
publish-teamspeak-server:
name: Publish TeamSpeak Server Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'teamspeak-server'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,7 +9,8 @@ on:
jobs:
publish-urlaubsverwaltung:
name: Publish Urlaubsverwaltung Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'urlaubsverwaltung'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-valkey:
name: Publish Valkey Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'valkey'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-voucher-vault:
name: Publish VoucherVault Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'voucher-vault'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,7 +9,8 @@ on:
jobs:
publish-ytdlp-web-player:
name: Publish YTDLP Web Player Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'ytdlp-web-player'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+2 -1
View File
@@ -9,7 +9,8 @@ on:
jobs:
publish-zipline:
name: Publish Zipline Helm Chart
uses: ./.github/workflows/publish-helm-chart.yml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'zipline'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+37
View File
@@ -0,0 +1,37 @@
name: SonarQube Analysis
# The analysis itself lives in workflow-util/pipelines, bound to a tag like
# every other call to that repository. This hull only decides WHEN it runs.
#
# No "paths" filter, unlike the publish-<chart>.yml hulls: the analysis looks at
# the whole tree - the five shell scripts under ci/, every workflow file, the
# YAML and Helm templates, the values.schema.json files, and the secret
# detection that covers everything. A filter would mean a change outside the
# filter leaves the last result standing while claiming to be current.
on:
# push on main, and only main, because the SonarQube Community Build analyses
# the main branch only - there is no pull-request analysis to hook into. The
# consequence belongs said out loud rather than discovered: a finding appears
# AFTER the merge. This is an observation of the state of main, not a gate,
# and the shared block deliberately carries no quality-gate step.
push:
branches:
- main
# workflow_dispatch, and it is not decoration. Three things about this path
# are unmeasured - whether node24 is in the runner image, whether the ~59 MiB
# scanner download gets through (a 17 MB one once did not, MTU 1500 against a
# pod network of 1450), and whether a repository variable reaches a workflow
# called from another repository. Any of them may need more than one attempt,
# and without a dispatch every attempt would cost a dummy commit on main.
# Only the maintainer can trigger it; the bot account cannot dispatch.
workflow_dispatch:
jobs:
sonar:
name: SonarQube Analysis
uses: workflow-util/pipelines/.gitea/workflows/sonar-analysis.yml@v4
# secrets: inherit passes SONAR_TOKEN through. The block declares it as
# required, and a missing required secret fails job CREATION on this
# instance, not the job - the run would stand there with no steps at all.
# SONAR_HOST_URL is a repository variable and needs nothing here.
secrets: inherit
+38 -8
View File
@@ -15,7 +15,7 @@ jobs:
any-install: ${{ steps.detect.outputs.any-install }}
steps:
- name: "Checkout Repository"
uses: actions/checkout@v7
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
fetch-depth: 0
- name: "Detect changed charts"
@@ -116,11 +116,11 @@ jobs:
chart: ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }}
steps:
- name: "Checkout Repository"
uses: actions/checkout@v7
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
with:
fetch-depth: 0
- name: "Install Helm"
uses: azure/setup-helm@v5
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
# Both steps below refuse to run while a chart declares a dependency that
# is not in its charts/ directory - lint with "[WARNING] chart directory
# is missing these dependencies", package with "found in Chart.yaml, but
@@ -167,12 +167,42 @@ jobs:
# build-chart, so it adds no wall-clock time to the pipeline.
steps:
- name: "Checkout Repository"
uses: actions/checkout@v7
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: "Install Helm"
uses: azure/setup-helm@v5
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
- name: "Render every chart with bare integer quantities"
run: ci/run-quantity-render-test.sh
selector-match:
name: "Selector match"
runs-on: ubuntu-latest
# The second job that ignores the changed-chart matrix and runs over *every*
# chart (Gitea #96), and for the same reason as quantity-render above: the
# mistake arrives by copy into a chart whose own diff nobody reads for it.
#
# What it asserts: every PodDisruptionBudget selector and every
# topologySpreadConstraints labelSelector a chart renders really matches the
# labels of a pod template in the SAME render. Both objects fail SILENTLY
# when their selector matches nothing - the spread constraint becomes a
# condition over zero pods and is trivially satisfied, the budget does its
# arithmetic over zero pods and never blocks an eviction - so neither a
# render error nor the install-test can catch it, and with
# whenUnsatisfiable: DoNotSchedule the object reads like a guarantee while
# distributing nothing.
#
# It needs no cluster and no Trivy: ~14 seconds for the whole repository,
# depends on no other job, and finishes well inside build-chart. The YAML is
# parsed by Helm itself through ci/yaml2json, because the runners have
# neither pyyaml nor yq and a hand-written parser would be able to read
# nothing and report nothing - see the script's header.
steps:
- name: "Checkout Repository"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: "Install Helm"
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
- name: "Check every rendered selector against the rendered pod labels"
run: ci/run-selector-match-test.sh
trivy-config:
name: "Trivy config ${{ matrix.chart }}"
needs: detect-changed-charts
@@ -190,16 +220,16 @@ jobs:
chart: ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }}
env:
# renovate: datasource=github-releases depName=aquasecurity/trivy
TRIVY_VERSION: v0.74.0
TRIVY_VERSION: v0.75.0
# "report" always exits 0. Switch to "gate" to make a finding that no
# documented exception covers fail the job - the intended step once the
# seccomp gap (#84) is closed.
TRIVY_CONFIG_MODE: report
steps:
- name: "Checkout Repository"
uses: actions/checkout@v7
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: "Install Helm"
uses: azure/setup-helm@v5
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
# A plain release download rather than aquasecurity/setup-trivy, and the
# reason is measured on the Gitea probe PR of #225, not assumed. That
# action runs actions/checkout against aquasecurity/trivy on github.com
+2 -2
View File
@@ -85,7 +85,7 @@ helm template <chart-name> --set ingress.domain=example.com --set ingress.cluste
Copy `template-chart/` and replace every `xxx` placeholder — in file names (`templates/xxx.deployment.yaml` → `templates/<app>.deployment.yaml`), in `Chart.yaml`, in the `values.yaml` keys and their `.Values.xxx.*` references inside templates, and in `values.schema.json` (both the values paths and the `xxx.` helper prefixes). The schema and the null-safe fallback helpers come with the template but their content is chart-specific — adapt them along the six points of the README section "Values that must not lie", and validate the result against the chart's real bundle values, not only against its defaults. Then wire up CI:
1. Add `.github/workflows/publish-<chart-name>.yml` (copy an existing one, e.g. `publish-outline.yml`) — it triggers on pushes to `main` touching `<chart-name>/**` and calls the reusable `publish-helm-chart.yml`.
1. Add `.github/workflows/publish-<chart-name>.yml` (copy an existing one, e.g. `publish-outline.yml`) — it triggers on pushes to `main` touching `<chart-name>/**` and calls the reusable `publish-helm-chart.yml`, which lives in `workflow-util/pipelines` and is bound to a tag. Copying an existing hull gets the `uses:` path, `registry-owner: ${{ github.repository_owner }}` and `secrets: inherit` right; writing one from memory does not.
2. Add the chart name to the `chart-name` choice list in `.github/workflows/change-app-version.yml`.
3. Add install-test fixtures under `ci/<chart-name>/` (`test-values.yaml`, `fixtures.yaml`) or an entry with justification in `ci/excluded-charts.txt` — the PR install-test fails if both are missing.
4. Add a `customManagers` block for the chart's main image in `renovate.json5` (one per chart, since the image repository lives in the template and the version in `Chart.yaml`) — without it Renovate never updates that chart's app version. See README section "Dependency updates (Renovate)".
@@ -117,4 +117,4 @@ The binding style guide for all current and future charts lives in **README.md (
- Resources: requests as values defaults; limits auto-computed by the `_helpers.tpl` helper (`limitFactor` default 3; no CPU limits unless set explicitly).
- Replicas per workload (explicit `0` allowed) plus a chart-global `scaleDown` flag that overrides them.
- `icon:` mandatory in every `Chart.yaml`; `helm lint --strict` without findings; CI fixtures under `ci/<chart>/` or a justified exclusion.
- **Chart dependencies are admissible but expensive** (the former "no dependencies" rule fell with #214; `ytdlp-web-player` is the only chart using one). A dependency has to be optional and off by default via `condition:`, and upstream's chart is consumed as it comes — where its defaults fall short of these conventions, that is **documented in our `values.yaml`, never patched**. The price to know before adding the next one: a subchart is invisible to everything downstream that watches Helm release versions, so a new upstream version — a security fix included — reaches a cluster only when the outer chart is republished (Renovate's built-in `helmv3` manager plus `ci/renovate-bump-chart.sh` is the whole safety net, and it belongs in a comment at the pinned version); and every helm command exits 1 while `charts/` is unfilled, which is why `ci/helm-dependencies.sh` runs before lint, package, render and install — including the render test that covers **all** charts on **every** PR. Full rules and the rest of the cost: README, "Chart dependencies".
- **Chart dependencies are admissible but expensive** (the former "no dependencies" rule fell with #214; `ytdlp-web-player` was the first to use one, `stalwart` the second, `paperless-ngx` the third). A dependency has to be optional and off by default via `condition:`, and upstream's chart is consumed as it comes — where its defaults fall short of these conventions, that is **documented in our `values.yaml`, never patched**. The price to know before adding the next one: a subchart is invisible to everything downstream that watches Helm release versions, so a new upstream version — a security fix included — reaches a cluster only when the outer chart is republished (Renovate's built-in `helmv3` manager plus `ci/renovate-bump-chart.sh` is the whole safety net, and it belongs in a comment at the pinned version); and every helm command exits 1 while `charts/` is unfilled, which is why `ci/helm-dependencies.sh` runs before lint, package, render and install — including the render test that covers **all** charts on **every** PR. Full rules and the rest of the cost: README, "Chart dependencies".
+91 -14
View File
@@ -16,30 +16,33 @@ Every chart has a wiki page of its own — `https://git.jk-dev.de/jlab-cluster/h
|---|---|---|
| `apache-tika` | Apache Tika | Deployment + Service only (no ingress) |
| `bentopdf` | BentoPDF | Stateless Deployment + `config.json` ConfigMap, `subPath`-mounted into the web root and always rendered (empty renders as `{}`) |
| `bulwark` | Bulwark Webmail (JMAP client for Stalwart) | **The companion to `stalwart`, and a pure client**: no mail, no database, no cache, no broker — the messages belong to the Stalwart release named by `bulwark.jmapServerUrl`. StatefulSet with **one** PVC at `/app/data`, carrying all four of Bulwark's directories (admin config, admin state with the audit log, encrypted user settings, telemetry) as subdirectories — one volume because snapshots are atomic per volume and never across them (the `garage` lesson, #81). No ConfigMap, so no `checksum/config` (§9 has nothing to attach to); the `op://` paths are covered by `checksum/secret-ref`. **The defining property, and it governs every value: the VOLUME BEATS THE CHART.** `configManager.get` resolves `admin override > env var > default`, the admin dashboard may write *any* key, and nothing says it just disconnected the chart — so a `helm upgrade` with a changed value can succeed and change nothing. Upstream documents this only for the server URL; it is true of everything. Exactly three values escape it (`SESSION_SECRET(_FILE)`, `ADMIN_PASSWORD`, `BULWARK_TELEMETRY`), and `bulwark.adminConfigReadOnly` closes it at a price that is **threefold and measured**: it freezes the settings policy (and with it the only way to move Web Push off the vendor relay), it freezes an override that *already* exists rather than removing it, and on a **fresh** install it stops `admin.json` from being written at all — so the admin dashboard never comes up and answers HTTP 500 while the pod stays Ready. Hence the default `false`. Sign-in is **OIDC-only** (`oauth.only`), and the way back out of a lockout is **not** `helm upgrade` with it `false` — `oauthOnly` is read through the same precedence chain, so the repair is removing the override in the **admin dashboard**, which is why `ADMIN_PASSWORD` is set at all (it generates no random password and logs nothing — the `#65` analogy does not hold here). Its **redirect URI carries the UI language** (`/<locale>/auth/callback`, 27 locales, and the language is not in the visible URL), so a single registered URI is the likeliest cause of a failed first sign-in. Both `SESSION_SECRET_FILE` and `OAUTH_CLIENT_SECRET_FILE` exist and are used — the better path `garage` and `keycloak` did not have — mounted **without `subPath`** so that a rotation takes effect without a restart, since both are re-read on every use. Two backup statements, both sharp: without the session secret the synchronized settings are unreadable **out of an intact volume**, and if the dashboard's OAuth setup was ever used the client secret sits in `config.json` on that volume in **cleartext**. All three probes are a plain GET on `/api/health` — **not** HEAD, which a `HTTPGetAction` cannot send — and the limit is written at the value: it carries process, HTTP acceptance, a free event loop and Next.js routing, and **nothing** about Stalwart, so with the mail server gone the pod is green. Served only at a host **root** (`NEXT_PUBLIC_BASE_PATH` is compiled in), one replica (single RWO PVC), two emptyDirs for `readOnlyRootFilesystem` (`/tmp`, `/app/.next/cache` — image optimization is active because `images.unoptimized` is in the lite branch only) |
| `calibre-web-automated` | Calibre-Web Automated | StatefulSet, extra consume-PVC |
| `convertx` | ConvertX | StatefulSet + PVC at `/app/data` (SQLite, uploads and converted output in one volume), no external database. **The only chart that overrides the image's `command` to bypass its entrypoint**: the image carries no `USER` and its entrypoint exists solely to drop to uid 1000 with `gosu`, so the chart starts the app directly and lets `runAsUser`/`fsGroup` do that work instead — the hardening baseline stays intact, at the price that any logic upstream later moves into that entrypoint is skipped silently (re-read it at every app-version update; the reasoning sits at the `command:` line). Pins uid 1000 rather than the repo's 20000, because the image owns `/home/convertx` with it and the uid owns PV data. `readOnlyRootFilesystem: true` with two emptyDirs, `/tmp` and `/home/convertx` — the latter is `HOME`, which the chart also sets explicitly, because LibreOffice, Calibre and fontconfig build their profiles there on the first conversion. Probes use `/healthcheck` (static 200, unauthenticated); `/` is never a 200, it 302s to `/setup` or `/login`. `JWT_SECRET` is a required `OnePasswordItem` — unset, the app mints a fresh one per process and every restart logs everyone out. The ingress `proxy-body-size` annotation is the **only** upload ceiling, since the app enforces none. ~1.4 GiB image, hence a raised `INSTALL_TIMEOUT` |
| `cyberchef` | CyberChef | Stateless Deployment |
| `freesailarr` | gluetun VPN media stack (qbittorrent, prowlarr, flaresolverr, radarr, recyclarr, sonarr, seerr, bazarr, and mediathekarr off by default) | Single-pod StatefulSet: all apps share gluetun's network namespace; config volumes via volumeClaimTemplates, media PVCs as existingClaims (optional per-category `subPath`, so one claim can serve all three categories). One ClusterIP Service per app (mediathekarr's carries both of its ports); no global `ingress.domain`. Ingress in two shapes that mix: one host per app (default, each app's host required), or one shared `ingress.host` with the apps as sub-paths (documented ingress exception below) |
| `garage` | Garage (S3-compatible object store) | **One** node, `replication_factor = 1` — a deliberate trade with its price named (Gitea #81): the volume layer underneath already replicates and encrypts, so Garage replication on top would be nine times the raw data across two layers that know nothing about each other. What that costs is that the factor applies to the **metadata** too, where a corrupted database is upstream-documented as "generally not recoverable" and a replicated volume replicates the corruption. Hence three settings that are mandatory rather than optional: `metadata_fsync = true` (**off** in Garage's defaults), the automatic metadata snapshots, and those snapshots on a **second PVC** — a file-system snapshot of a live metadata database is not a restore point, so the volume backup covers the blocks and not the index. The first PVC carries `metadata_dir` **and** `data_dir` as two subdirectories, because volume snapshots are atomic per volume: that is what keeps index and blocks consistent with each other. Two further properties shape the whole chart: the image is **`FROM scratch`** and contains exactly one file, `/garage` — no shell, so **exec probes are impossible here** and an init container would need a second image — and the RPC secret has to arrive as an **environment variable**, because `rpc_secret_file` rejects any file with a group bit set while the kubelet sets one (the reasoning sits at the value). Started with `garage server --single-node`, which creates the cluster layout in-process before any listener opens and is idempotent on the layout version, and with `--default-access-key`/`--default-bucket`, which **import** key and bucket from the 1Password item — so the vault is the source of truth instead of a copy of what the server generated. Readiness is the admin API's `/health`, liveness and startup deliberately are **not**: `/health` answers 503 until a layout exists, so a node waiting for one would be restarted forever |
| `gotenberg` | Gotenberg | Deployment + Service only (no ingress) |
| `homeassistant` | Home Assistant | StatefulSet; the only chart carrying Helm tests (`templates/tests/`) |
| `keycloak` | Keycloak (OpenID Connect / SAML identity provider) | **Two** replicas with `cache=ispn`, clustered through Keycloak's default `jdbc-ping` stack — discovery runs over the PostgreSQL that is there anyway, so no headless Service and no JGroups wiring. Deployment, **no PVC** (Keycloak 26 keeps everything in the database, user sessions included). **The chart has two init containers and needs both** (Gitea #83): `db`, `health-enabled` and `metrics-enabled` are Keycloak *build* options, the published image is built with their defaults, and a runtime change makes Keycloak rewrite `/opt/keycloak/lib/quarkus` at start — which `readOnlyRootFilesystem: true` forbids and which upstream describes as a start that "creates an image that is mutable". An empty `emptyDir` over that path does not work either, because `QuarkusEntryPoint` loads `quarkus/quarkus-application.dat` before it dispatches any command, `build` included. So one init container **seeds** the directory into a volume, the next runs `kc.sh build` into it, and the serving container runs `start --optimized` — every container keeping the full hardening baseline. Build options therefore exist in exactly **one** place and a build option in `keycloak.env` fails the render. First chart with a `PodDisruptionBudget` (§11) and first `quay.io` image (hence the entry in `ci/trivy/data/ksv0125.yaml`). Two Services (8080, and the management port 9000 **never** published — it serves `/health/*` and optionally `/metrics` unauthenticated) and two Ingresses. **The admin hostname is not an access control**: `hostname-admin` only decides which URLs the server generates, and upstream states that it "does not prevent accessing the Administration REST API endpoints via the frontend URL" — restricting that API is reverse-proxy work, and the chart renders no `/admin` deny rule. `ingress.domain` is read as `KC_HOSTNAME` (as a full URL, which upstream requires alongside `hostname-admin`), so it stays required with the Ingress off. Liveness is `/health/live` and deliberately not `/health` or `/health/ready` — those include the database, and a liveness probe on them restarts the whole cluster for as long as the database is away. **`keycloak.metrics.enabled` defaults to `true` for the readiness probe, not for the metrics**: upstream's table of health checks carries a "Requires Metrics" column and the *database* check is the row that says yes, so with it off `/health/ready` answers 200 while the database is gone — the `urlaubsverwaltung` shape (#218) with the dependency hidden behind an option whose name says observability. Two traps at the values: upstream's `db-pool-max-size` default of 100 would ask a default PostgreSQL for 200 connections across two replicas (the chart ships 20), and two releases sharing one database **merge into one Infinispan cluster** unless the cluster names differ |
| `matrix-synapse` | Matrix Synapse | StatefulSet plus an nginx delegation Deployment/Service |
| `minecraft-router` | itzg/mc-router | Deployment + routing ConfigMap + NodePort Service (TCP only, no ingress); `mappings` is a required value |
| `minecraft-server` | itzg/minecraft-server | StatefulSet + PVC + Service + init-script ConfigMap (no ingress); service type is a value (`ClusterIP` by default, routed via `minecraft-router`). **Only the newest Java line is maintained** (chart 5.0.0, #158): the directory carries one line, older ones stay published as frozen artifacts (java8/java17 at `4.0.0`, java21 at `4.1.0`) and receive no further chart changes — publishing once per line out of one directory is what made the lines drift. To run an older JVM with the current chart, set `image.tagOverride`; the chart then checks that `minecraft.version` fits the Java line of the **effective** tag and fails the render with both values named. Two cases pass unchecked on purpose: a tag without a `java<N>` part and a `minecraft.version` that is not a plain numeric `x[.y[.z]]`. `image.tagOverride` is deliberately outside Renovate's reach — `renovate.json5` tracks `appVersion` only. Since chart 2.0.0 the tag is an **immutable dated upstream release** (`appVersion: <release>-java8` / `-java17` / `-java21` / `-java25`, e.g. `2026.8.2-java8`) instead of the continuously rebuilt rolling line tag. Pilot for the strict `values.schema.json` (#68), which is now binding for every chart |
| `outline` | Outline | StatefulSet, separate DB/Redis secrets |
| `paperless-ngx` | Paperless-ngx | Largest chart: consume CronJob, SMB-connector PV/PVC, pre/post-consumption scripts ConfigMap |
| `paperless-ngx` | Paperless-ngx | Largest chart: consume CronJob, SMB-connector PV/PVC, pre/post-consumption scripts ConfigMap. Optional `valkey` subchart — **this repo's own chart, from the published index**, off by default and gated on `valkeySubchart.enabled` (not `valkey.enabled`, which the subchart's strict schema rejects). Configured as a **queue** (PVC + `noeviction`), unlike `stalwart`'s cache: `PAPERLESS_REDIS` is the Celery broker, and `allkeys-lru` evicts across the whole keyspace, where a backlog of waiting consume tasks is exactly what LRU reads as cold. The password comes from the **same 1Password item** as the Redis connection, in a third field `auth-conf` holding the complete line `requirepass <password>` — so the password sits in that item **twice** and a rotation has to touch both fields. Two more traps: with the subchart on, `redis-connection-url` in the vault **must** be repointed at `<release>-valkey-service:6379` or the bundled Valkey runs unused, and `valkeySubchart:` must not be erased — a null override deletes the key, Helm reads an unresolvable `condition:` as *enabled*, so erasing the switch turns the subchart **on** (rejected by `values.schema.json`). Never change `valkey/` and this chart in the same PR, see "Chart dependencies" |
| `patchmon` | PatchMon | One Deployment (the 2.x server serves frontend and `/api/*` from one process) plus an optional guacd sidecar; OIDC-only sign-in |
| `rallly` | Rallly | Deployment, DB/OIDC/SMTP secrets |
| `rss-bridge` | RSS-Bridge | Deployment + whitelist ConfigMap; `ingress.hosts` is a required list of host + optional `tlsSecretName`, so several hosts can keep their own existing certificates (documented deviation from the one-TLS-secret rule, see its `values.yaml`) |
| `samba` | Samba | StatefulSet, users secret + config ConfigMap |
| `satisfactory` | Satisfactory game server | StatefulSet, no ingress |
| `searxng` | SearXNG | Stateless Deployment + settings ConfigMap (`use_default_settings: true` plus a free-form `settings.extra` passthrough); the signing key is a required `OnePasswordItem`, an external Valkey an optional second one. `ingress.domain` is read twice — Ingress host and `SEARXNG_BASE_URL` — so it stays required with the Ingress switched off |
| `stalwart` | Stalwart mail server (SMTP, IMAP, POP3, JMAP, ManageSieve) | A webmail client for it is the separate `bulwark` chart (above), pointed at this release's JMAP endpoint — the two are only useful together, and nothing in either reports the other being away. **The chart configures almost nothing, and that is the chart's defining property.** Since v0.16 Stalwart keeps its whole configuration — every listener, the blob store, the in-memory store, TLS objects, domains and the cluster roles — inside the data store and manages it over JMAP; the only file on disk is a `config.json` that names the data store and nothing else. So the chart renders that ConfigMap (secret-free: `authSecret` uses the `EnvironmentVariable` variant, so the password arrives by `secretKeyRef` and `checksum/config` stays meaningful), one StatefulSet per entry of the `nodes` **map** with `STALWART_ROLE` and `STALWART_HOSTNAME`, one NodePort Service **per port** (not per group — a NodePort is cluster-unique, so `full` and `mx` can only take SMTP behind the same Service, selected by a per-port pod label), a per-group ClusterIP `http` Service for the Ingress, per-group headless Services and NetworkPolicies, and a cert-manager `Certificate` for the mail ports. Everything else is operator work in the WebUI, which is why `NOTES.txt` carries an eight-step first install. Four traps worth knowing before reading the chart: **the role lookup fails open** — a node whose `ClusterRole` is missing from the database enables every listener *and* every task while staying Running with a green probe, so `mx` ships at `replicas: 0` and the NetworkPolicy is a ceiling independent of what was bound; **`NET_BIND_SERVICE` is a documented deviation needed to `exec`, not to bind** (the image sets `setcap cap_net_bind_service=+ep`, so `drop: [ALL]` alone makes the kernel fail `execve` — removing it as "unused" stops the chart starting); **`externalTrafficPolicy` must be `Cluster`** (no pod runs on a control-plane node while the HA proxy delivers to them, so `Local` would drop every packet) and the source address it costs is restored only by the HA proxy's PROXY v2 header; and **`/healthz` is rate-limited to 100 req/60s per source IP**, which is a node IP for kubelet probes, so the probe intervals are calculated rather than chosen. A first install needs `bootstrap.enabled: true` because the setup wizard appears only when `config.json` is *absent*. Optional `valkey` subchart — **this repo's own chart, from the published index**, off by default and gated on `valkeySubchart.enabled` (not `valkey.enabled`, which the subchart's strict schema rejects); never change `valkey/` and this chart in the same PR, see "Chart dependencies" |
| `sure` | Sure (we-promise, the maintained Maybe-finance fork) | One StatefulSet with **two containers**: the Puma web process and a Sidekiq worker, sharing one ReadWriteOnce PVC at `/rails/storage` — that shared volume is the reason they live in one pod. Only the web container migrates (the image entrypoint runs `db:prepare` solely for the `./bin/rails server` command, so the worker is started through `args:` and skips it). Three 1Password items (app/database/redis); Postgres and Redis stay external. Readiness deliberately probes `/` rather than Rails' `/up`, which is a boot check and touches no database; the worker's three probes ask `sidekiqmon` for this pod's own heartbeat. `ingress.domain` is read for `APP_DOMAIN` and the WebAuthn relying party, so it stays required with the Ingress switched off |
| `teamspeak-server` | TeamSpeak server | StatefulSet, no ingress |
| `template-chart` | — | Scaffold for new charts (placeholder `xxx`), never published |
| `urlaubsverwaltung` | Urlaubsverwaltung | Deployment, DB + OIDC secrets |
| `valkey` | Valkey | One instance per application (cache or queue backend), replacing the shared `cluster-redis` whose single ACL file coupled six applications together. StatefulSet with one replica + ClusterIP Service, no cluster mode, no Ingress and no value to add one. The password arrives as a **finished configuration line** (`requirepass <secret>`) in the 1Password field `auth-conf`, is mounted as a file and `include`d by the config — never in argv, never in the environment; an init container refuses to start the pod unless that line is really there, because the alternative failure is an open server that looks healthy. `maxmemory` is **derived from the computed memory limit** and a configured value above it aborts the render. Persistence and eviction are a pair, not two switches (queue = PVC + `noeviction`, the default; cache = emptyDir + `allkeys-lru`). First chart carrying a **NetworkPolicy** (on by default, Gitea #5), and the first whose probes had to be proven capable of failing: `valkey-cli` exits 0 on an error reply unless `-e` is given |
| `voucher-vault` | VoucherVault | StatefulSet, DB + OIDC secrets |
| `ytdlp-web-player` | ytdlp_web_player | Stateless Deployment, video cache and tool cache as emptyDirs (no PVC). **The only chart with a dependency**: an optional `oauth2-proxy` subchart, off by default, wired in front of the app because the application ships no authentication at all — see "Chart dependencies" below |
| `ytdlp-web-player` | ytdlp_web_player | Stateless Deployment, video cache and tool cache as emptyDirs (no PVC). **The first chart with a dependency** (`stalwart` and `paperless-ngx` followed): an optional `oauth2-proxy` subchart, off by default, wired in front of the app because the application ships no authentication at all — see "Chart dependencies" below |
| `zipline` | Zipline | StatefulSet, DB secret |
## Versioning
@@ -52,23 +55,32 @@ Every chart has a wiki page of its own — `https://git.jk-dev.de/jlab-cluster/h
## CI/CD
- `publish-<chart>.yml` — one thin workflow per chart; triggers on pushes to `main` touching `<chart>/**` (or manually) and calls the reusable workflow below. **A push to a chart directory on `main` publishes that chart**, so the chart version must be bumped in the same change.
- `publish-helm-chart.yml` — reusable workflow: `helm package`, then a single HTTP `POST` of the packaged `.tgz` to the private chart registry. The target is a **classic, index-based** Helm registry (Gitea's `/api/charts` endpoint), not an OCI one: `helm push` speaks only OCI and is not used, there is no `helm registry login` session, and every request carries its own Basic auth. Credentials **and the target URL** come from 1Password Connect (`OP_CONNECT_TOKEN` secret, item `op://GitHub-Actions/jlab-registry-access/` with `username`, `password`, `charts-url`); the workflow appends `/api/charts` to `charts-url`. No registry address ever appears in this repository. The loaded values are read via **`steps.<id>.outputs.*`**, mapped into the step's own `env:` block and used as quoted shell variables — never interpolated into the command line. `load-secrets-action` **v5 no longer exports them as environment variables** for later steps the way v4 did, so after the v5 bump every `${{ env.* }}` reference resolved to an empty string while the loading step still logged `Populating variable` and stayed green. Each value is additionally guarded with `:?`, because an empty one used to collapse into a missing argument rather than an error. Three properties of the upload step are load-bearing:
- **Credentials stay out of the process list.** `curl --user "$U:$P"` would put them in `argv`; the step uses `curl --config -` and pipes the options in on stdin instead.
- **A failed upload fails the job.** Unlike `helm push`, `curl` exits **0** on HTTP 401/403/409 — a publish would go green with nothing uploaded. `--fail-with-body` turns those into a non-zero exit and still prints the response body, so the log names the cause.
- **The file name is built, not globbed.** `<chart>-<version>.tgz` is assembled from the chart name and the version already read from `Chart.yaml`; `--upload-file` takes exactly one file, and a glob matching two silently turns the second into curl's URL argument. `helm package` keeps the `+` of a `1.4.0+up1.9.2` version verbatim in the file name, so the built name is an exact match.
- `publish-helm-chart.yml` — **not in this repository any more.** The reusable workflow lives in [`workflow-util/pipelines`](https://git.jk-dev.de/workflow-util/pipelines), the shared building-block repo, and the callers here bind it to a tag: `uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3` (Gitea #66). It still does the same thing — `helm package`, then a single HTTP `POST` of the packaged `.tgz` to the private chart registry, which is a **classic, index-based** Helm registry and not an OCI one, so `helm push` is not used, there is no `helm registry login` session, and every request carries its own Basic auth. What changed is who supplies what:
- **`registry-owner: ${{ github.repository_owner }}`** in each caller's `with:`. The owner used to be derived inside the workflow (#63); it is now a required input with no default, so the expression is evaluated **here**, in the calling repository, where the value is measured. That is `pipelines`' house rule — a shared block never guesses an owner segment — and it is why every caller carries that one extra line.
- **`secrets: inherit`**, unchanged, for `OP_CONNECT_TOKEN`.
- the repository **variable `OP_CONNECT_HOST`**, which replaced the Connect host that used to be hardcoded in our copy. A shared block must not carry one tenant's infrastructure address; the block's first step fails and names the variable when it is unset.
Note: the registry **accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact, so a republish of an existing version is still silent — see #160.
The registry credentials and the target URL still come from 1Password Connect (item `op://GitHub-Actions/jlab-registry-access/` with `username`, `password`, `charts-url`). `charts-url` holds only the **base up to `/api/packages`** and the block composes `<charts-url>/<owner>/helm/api/charts`, so **the publish target and its credentials appear nowhere in this repository.** `ci/helm-dependencies.sh` is still run before `helm package`, and it is still **our** script: the block's checkout clones the calling repository, so it runs from this tree.
**What this does *not* claim any more**: that no registry address is in the repo at all. That wording was true until a chart took one of our own published charts as a subchart — `stalwart`'s `Chart.yaml` names `https://git.jk-dev.de/api/packages/jlab-cluster/helm` as a dependency `repository:`, because a Helm dependency is resolved from a URL in the chart and there is nowhere else to put it. The narrower claim above is the one that matters and it is unchanged: that is the **read** path, the index is anonymously readable (measured), so it carries **no credential** and grants nothing a `git clone` of this repo does not already imply. The host itself has always been in `CLAUDE.md`; what is new is only the read-only path segment.
The reasons behind the upload step are **not repeated here**, because they are no longer ours to keep current: why both halves of the composed URL are guarded, why the credentials go in through `curl --config -` instead of `--user`, why `--fail-with-body` is what makes a failed upload fail the job, and why the file name is built rather than globbed, are all on the [`publish-helm-chart` wiki page](https://git.jk-dev.de/workflow-util/pipelines/wiki/publish-helm-chart) of that repository. Nothing was dropped in the move — the upload step's shell body went across unchanged except for two error messages that name the owner as an input.
Note: the registry **accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact, so a republish of an existing version is still silent — see #160. That is also what makes a manual `workflow_dispatch` of any `publish-<chart>.yml` a harmless way to prove the shared block works.
- `change-app-version.yml` — manually dispatched app-version bump; the chart must be listed in its `chart-name` choice list.
- `ci/helm-dependencies.sh <chart-dir>` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one). It runs in **five** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in `publish-helm-chart.yml`, and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs.
- `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (the same changed-charts matrix, minus the charts listed in `ci/excluded-charts.txt`) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix, or run the all-charts install-test after the merge. The **resource-quantity render test** is the exception to that rule: it runs over every chart on every PR — see below.
- `ci/helm-dependencies.sh <chart-dir>` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but three: `ytdlp-web-player`, `stalwart`, `paperless-ngx`). It runs in **six** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in the shared publish block (which is why that block names this script as a prerequisite of its callers — see above), and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh`, `ci/run-selector-match-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs.
- `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (the same changed-charts matrix, minus the charts listed in `ci/excluded-charts.txt`) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix, or run the all-charts install-test after the merge. The **resource-quantity render test** and the **selector-match render test** are the two exceptions to that rule: both run over every chart on every PR — see below.
- `install-test.yml` — reusable workflow (`workflow_call`) holding the install-test **job** itself: the runner-label expression, the pinned `kubectl` with its skew check, `envsubst`, the containerised-job detection, the per-leg k3d cluster and the `freesailarr` VPN credentials. Its one input is a JSON array of chart names; its callers decide *which* charts are tested and whether the job runs at all, never *how*. Two callers: `validate-charts.yml` and `install-test-all-charts.yml`. **It is not copied for the second caller** — a second copy is a second chance to test differently, unnoticed, the same argument that gives `ci/excluded-charts.txt` exactly one parser.
**It stays in this repository, and that is a decision, not an oversight** (Gitea #66, decision 4). It is reusable in form but not in substance: it is welded to the `ci/` fixtures, to the `ci/excluded-charts.txt` parser and to this repository's chart matrix. A building block that only one repository can use is not shared, it is merely moved somewhere else — and moving it would buy nothing while putting a tag between us and a file we change often. `publish-helm-chart.yml` was the opposite case: it takes a chart name and a registry owner and knows nothing about this repository, which is exactly why it could go. This note exists so the question is not asked again in half a year.
- `install-test-all-charts.yml` — manually dispatched **all-charts install-test**: the same job, with a matrix over every installable chart instead of the changed ones. Optional input for a subset of charts (empty = all). See below for when to run it.
- `sonar.yml` — **SonarQube analysis**, a hull like the publish ones: it triggers on pushes to `main` (no `paths` filter — the analysis looks at the whole tree) or manually, and calls `workflow-util/pipelines/.gitea/workflows/sonar-analysis.yml@v4` with `secrets: inherit`. The project-specific half is `sonar-project.properties` in the repo root. **It is not a gate** and it does not block anything — see below.
### Where this repo lives, and what a Gitea runner does differently
Everything below was measured on this instance, not taken from documentation, and it is the reason for every deviation in `validate-charts.yml`. How the move got here is not repeated — that is Gitea #14.
**Reusable workflows with `secrets: inherit` work**, so the 24 thin `publish-<chart>.yml` hulls run unchanged. One quirk worth knowing before debugging the wrong thing: a *missing* secret does not fail the job, it fails job **creation**, so the run shows no steps at all.
**Reusable workflows with `secrets: inherit` work**, so the 25 thin `publish-<chart>.yml` hulls run unchanged — including now that the workflow they call lives in another repository (Gitea #66). One quirk worth knowing before debugging the wrong thing: a *missing* secret does not fail the job, it fails job **creation**, so the run shows no steps at all.
**Runner labels mean different machines than on GitHub.** `ubuntu-latest` here has **no Docker socket**; `ubuntu-docker` has one. Exactly one job cares — the install-test, because k3d is a Docker client — and its `runs-on` is therefore an expression rather than a fixed label, so the same file still works on a GitHub runner where `ubuntu-docker` does not exist. Every other job (lint/package, the quantity render, the Trivy scan) needs no socket and stays on `ubuntu-latest`.
@@ -130,6 +142,25 @@ Two details worth knowing before touching it:
- **The resources paths come from `values.schema.json`, not `values.yaml`** — it is JSON, so `python3` reads it without pyyaml (same runner constraint as `ci/run-trivy-config.sh`), and every chart-owned key must appear there anyway. A block counts when it is `#/definitions/resources` or carries `limitFactor`/`requests` itself, which is what keeps `minecraft-server`'s JVM-sizing `minecraft.resources` out. A chart whose templates render a resources block while its schema declares none **fails** the test with exactly that message: the schema is then out of sync with the chart.
- **A chart that never reaches the helper is reported, not counted as passing.** `gotenberg` sets every computable limit explicitly in its fallback defaults, and since a `null` never deletes a default there, no override can free them — nothing is multiplied, so nothing is proven. Those charts appear under "helper not reached" in the job summary rather than as a green tick for a code path that was never entered.
### Selector-match render test (all charts, every PR)
`ci/run-selector-match-test.sh [chart …]` (no argument = all charts) renders every chart with the same fixtures the install-test uses and checks that **every label selector the render contains really matches the labels of a pod template in that same render**. Two kinds of object are covered, and both of them fail *silently* when their selector matches nothing:
| Object | What a non-matching selector does | Who notices today |
|---|---|---|
| `topologySpreadConstraints` | the constraint is a condition over **zero** pods and is therefore trivially satisfied: nothing is distributed, no event, no log line | nobody — and with `whenUnsatisfiable: DoNotSchedule` the object reads like a guarantee |
| `PodDisruptionBudget` | the budget does its arithmetic over zero pods and never blocks an eviction, so `kubectl drain` takes every replica at once | nobody — `kubectl get pdb` shows an object that looks like protection |
Neither is a render error, neither fails the install-test, and neither shows up in the diff of the PR that causes it: the chart that breaks it is usually a chart that **renamed a label somewhere else**. Charts answer the first half of that by keeping the selector and the pod labels in one helper (`stalwart.groupLabels`, `keycloak`'s `app` label) — but one definition only helps while nobody writes a second one, and that is the half a test can hold. Introduced with Gitea #96, where the chart-owned spread constraint made the trivially-satisfied case reachable by design rather than by accident.
Three things to know before touching it:
- **Helm is the YAML parser, through the one-template chart `ci/yaml2json`.** The runners have neither `pyyaml` nor `yq` (measured; the same constraint `ci/run-trivy-config.sh` and `ci/run-quantity-render-test.sh` work under), and a YAML reader written in `awk` was the obvious alternative and the wrong one: its failure mode is a parser that quietly reads nothing, finds no selector, reports no mismatch and leaves the test green for a chart it never looked at. So the rendered manifest stream is handed to that chart as a **values file** and comes back as JSON, which `python3` reads from the standard library. The one textual step is the document split on a line that is exactly `---` in column one, and getting *that* wrong does not produce a false pass — the fragment stops being valid YAML and `helm` exits 1.
- **What is deliberately not checked, named so the gap is not mistaken for coverage.** `podAffinity`/`podAntiAffinity` are out of scope: their semantics are relative to pods that exist, so a term matching nothing is a legitimate configuration rather than a mistake. And a selector expressed **only** through `matchExpressions` is **reported as "not checked", never counted as passing** — evaluating set-based requirements means reimplementing label-selector semantics, which is a second parser by another name.
- **Two anti-vacuum guards, because this test can be green over nothing.** Both objects are rendered only above a replica threshold, and most charts' CI fixtures sit below it. `ci/selector-match/<chart>.yaml` is a render-only overlay applied **last**, whose only job is to push a chart over that threshold — and a chart that *has* one and still renders no selector **fails**, since that overlay existing is the whole claim that something was checked. On top of that the run fails if **no selector at all** was checked anywhere. Today `stalwart` is exercised through its overlay (two replicas in both node groups plus an `externalServices.valkey.host`, without which the chart's own "cluster with no in-memory store" abort fires) and `keycloak` through its ordinary install-test fixture, which already runs two replicas.
Measured: 30 charts, **5 selectors verified in ~14 s**, no cluster. The fail path was exercised rather than assumed — renaming `node-group` to `nodegroup` in a hand-written copy of the PDB selector produced two `FAIL` lines naming both the selector and the labels actually present, and a non-zero exit; the same was done for the spread constraint's selector.
### Install-test (PR pipeline stage 2)
For every chart a PR changes, `ci/run-install-test.sh <chart>` runs against a throwaway k3d cluster: it applies the stub `OnePasswordItem` CRD plus the chart's fixtures, `helm install`s the chart with its CI values, and waits for all Deployments/StatefulSets to roll out (per-chart timeout). Since probes and security hardening are active on all workloads, "Ready" means the app actually answers. On failure the job dumps pod descriptions, events and container logs.
@@ -145,7 +176,7 @@ Fixtures live at the **repo root** under `ci/` (deliberately *not* inside the ch
- `ci/<chart>/settings.env` — optional overrides: `INSTALL_TIMEOUT` for slow starters, and `REQUIRED_ENV="VAR_A VAR_B"` to declare the variables `fixtures.env.yaml` needs. **A missing or empty declared variable fails the install-test loudly, naming the variable — it never skips the chart.** Skipping would restore exactly the green-but-meaningless signal this mechanism exists to remove; the one legitimate case of absent secrets, a PR from a fork, belongs in an `if` on the workflow job rather than in the script
- `ci/excluded-charts.txt` — charts that cannot run in CI, one per line as `<chart> <justification>`. Currently: `satisfactory` (multi-GB image + 10GB+ Steam download), `template-chart` (scaffold with placeholder image). The justification is mandatory: a line carrying only a chart name is deliberately **not** an exclusion, because a bare name would drop a chart out of CI with nothing to review. **An excluded chart gets no install-test job at all** — see below
`freesailarr` is the only chart using `fixtures.env.yaml` today: gluetun needs a real Wireguard key, or it never completes a handshake and no app container behind its kill switch becomes Ready. The install-test job loads the key from 1Password Connect for that chart only, using the same pattern as `publish-helm-chart.yml` — `steps.<id>.outputs.*` into the step's own `env:`, quoted, with `:?` guards that name the missing `op://` field before the cluster is even created. The key in that item is a **CI-only Mullvad key**, deliberately not the production one: Mullvad ties an address to a key and routes the tunnel to the endpoint it saw last, so a shared key would make every PR run pull the tunnel out from under the running stack — with the symptom in the cluster and the cause in a pull request.
`freesailarr` is the only chart using `fixtures.env.yaml` today: gluetun needs a real Wireguard key, or it never completes a handshake and no app container behind its kill switch becomes Ready. The install-test job loads the key from 1Password Connect for that chart only, using the same pattern as the shared publish block — `steps.<id>.outputs.*` into the step's own `env:`, quoted, with `:?` guards that name the missing `op://` field before the cluster is even created. The key in that item is a **CI-only Mullvad key**, deliberately not the production one: Mullvad ties an address to a key and routes the tunnel to the endpoint it saw last, so a shared key would make every PR run pull the tunnel out from under the running stack — with the symptom in the cluster and the cause in a pull request.
**An excluded chart is dropped from the matrix, not skipped inside its job (#231).** It used to get a job that ran the script, printed `SKIP` and exited 0 — and GitHub knows only the exit code, so the check list showed `Install-test satisfactory ✓` for a chart whose image was never pulled, indistinguishable from one that really came up in the cluster. `detect-changed-charts` now removes the excluded charts before the matrix is built, so no job exists for them and nothing claims they passed. Because an absence on its own says nothing, that job writes every excluded chart of the PR **with its documented justification** into the job summary — a `Skipped` would only say "not run", and why is the question a review actually asks. The list is parsed in exactly one place: `ci/run-install-test.sh --list-excluded` emits `<chart><TAB><reason>` and the workflow calls it instead of re-implementing the format in YAML, since a second parser would be a second chance to select different charts — silently, and in either direction. The script keeps its own `SKIP` branch for local runs. What cannot be done instead: deciding per chart in an `if` on the install-test job, to get a visible `Skipped` entry rather than an absent one. GitHub rejects it with `context "matrix" is not allowed here` — `jobs.<id>.if` is evaluated **before** the matrix is expanded, so the `matrix` context does not exist there, and the whole workflow ends in failure with zero jobs created (measured on the closed PR #233).
@@ -164,7 +195,32 @@ Why it exists at all: the answer arrived once by accident. Gitea PR #48 correcte
- **Exclusions come from the one parser**, `ci/run-install-test.sh --list-excluded`, and the excluded charts appear in the job summary **with their justification**, exactly as in the PR pipeline.
- **The result is written down.** Every matrix leg records its own outcome next to the four facts that decide what the outcome means — date, runner label, `K3S_IMAGE`, `kubectl` version — and a final job records the verdict over the whole matrix. A green run nobody notes is unknown again next time. Per leg and not as one table, because a matrix leg only ever sees its own job summary.
- **Expected runtime: 20 charts in 12m41s** (Gitea PR #48, one job, all legs concurrently on one Docker daemon). The figure is there so a later deviation is visible as one; compare it against the chart count the run prints in its own summary — 24 at the time of writing, and one more with every new chart. The job timeout is 30 minutes per leg and lives in `install-test.yml`, i.e. it is the same for both callers on purpose.
- **A big image does not move that figure as much as it looks.** `convertx` was expected to, at ~1.4 GiB compressed — by a wide margin the largest image in the repo, and the reason its `settings.env` raises `INSTALL_TIMEOUT` to 900s. Measured on its first green run (Gitea PR #61) the whole leg took **2m35s**, cluster creation and rollout wait included, so it sits inside the existing envelope rather than setting a new one. Worth knowing before the next raised timeout is read as a prediction about the total: the legs run concurrently, so what the all-charts figure tracks is the **slowest** leg plus scheduling, not the sum of the pulls.
### SonarQube analysis (after the merge, and not a gate)
`sonar.yml` sends the whole tree through a SonarQube analysis. The workflow itself is the shared block [`sonar-analysis.yml`](https://git.jk-dev.de/workflow-util/pipelines/wiki/sonar-analysis) in `workflow-util/pipelines`, bound to `@v4`; the only thing on this side is `sonar-project.properties` in the repo root, holding the project key `JLab-Helm-Charts`. **That key is read off the existing project in SonarQube, not derived** — it is the one the maintainer set up, and a guessed key (`jlab-cluster_helm-charts`, reasoned from the instance serving two owners) is what made the first run fail. The file carries **no `sonar.projectName`** on purpose: once the project exists, the display name belongs to the server, and a name sent with every analysis would overwrite a rename there silently.
**What is analysed — measured, not taken from a language list.** The first run (run 375) loaded exactly **two quality profiles, `json` and `yaml`**; the sensors that did work were **IaC Kubernetes** (3.4 s) and the **secrets detection** (`TextAndSecretsSensor`) over all 391 indexed files.
| What is in this repository | Analyser | Runs? |
|---|---|---|
| 238 YAML files and 26 Helm template directories | IaC Kubernetes, YAML | yes |
| 26 `values.schema.json` | JSON | yes |
| the whole tree (391 files) | Secrets | yes |
| the 5 shell scripts under `ci/` — the whole CI logic | Shell | **no** |
| the 30 workflow files | GitHub Actions | **no** |
**The two missing rows are not a configuration mistake, and nothing in this repository can fix them: analysers come from the *server*, not from the scanner.** The language list the earlier version of this table was built from describes the newest SonarQube release; this instance does not have the Shell and GitHub Actions plugins. The consequence is worth stating plainly rather than leaving it to be rediscovered — **the files carrying the actual logic, the five `ci/` scripts and the thirty workflows, are not analysed at all.** What runs is IaC over the charts plus secret detection over everything. Whether the instance gets upgraded or gains those plugins is a decision for whoever runs it, not something this repository settles; `shellcheck` and review remain the only check on the scripts until then.
The secrets row is the valuable one that *does* run. **The premise of this repository is that no credential ever lands in it** — every secret is a 1Password `OnePasswordItem` reference — and until now nothing checked that premise. What it actually reports is still unknown: run 375 built the report (603 kB) but never uploaded it, because the upload is where the wrong project key failed.
**It is not a gate, and that is a decision.** The SonarQube Community Build analyses the **main branch only**; there is no pull-request analysis to hook into, so the trigger is a push to `main` and **a finding appears after the merge**. At that point there is nothing left to block, which is why the shared block carries no quality-gate step: a red gate behind the merge would only colour `main` red without having prevented anything, and a permanent red is a thing people learn to ignore — taking the credibility of `validate-charts` with it. A gate becomes the right thing under one of two conditions: pull-request analysis is available, or a deployment hangs off the workflow. Until then, read the findings in SonarQube; nothing in this repository turns red because of them.
**No exclusions, for the first round.** Including the Helm templates, which are not valid YAML because of `{{ }}`. What the Helm and YAML analysers make of them is only learnable by letting them run over them once, and excluding them up front would discard that answer along with the bulk of the repository. If the first run produces noise genuinely not worth reading, the exclusions go into `sonar-project.properties` with the finding they answer named next to them — not before.
**Prerequisites**, both on the instance and neither in a file here: the repository **variable `SONAR_HOST_URL`** (a variable and not a secret on purpose — the server address is not a credential, and as a variable it stays readable in the log, which is what makes a wrong one diagnosable) and the **secret `SONAR_TOKEN`**, which reaches the block through `secrets: inherit`. The block guards both and names the setting and its place when one is empty. A *missing* `SONAR_TOKEN` never gets that far: the block declares it `required`, and a missing required secret fails job **creation** on this instance, so the run would show no steps at all — the same quirk as everywhere else here.
**Three things about this path were unmeasured, and run 375 answered all three — positively.** `node24` **is** in the runner image, so the JavaScript scan action runs. The scanner download **gets through**: ~59 MiB of archive plus its signature in roughly three seconds, so the MTU failure that once killed a 17 MB download on this runner (see "What a GitHub runner had been providing quietly") does not repeat at five times the size. And a repository **variable does reach a workflow called from another repository** — the block's guard passed and the scanner started, which it could not have done without `vars.SONAR_HOST_URL` arriving. That last one also settles the identical question the publish route carried for `OP_CONNECT_HOST`: same forge version, same mechanism, a repository variable read inside a workflow called from another repository. What the publish route still has unproven is the upload itself, which has never run. `workflow_dispatch` stays on the hull for the retries: without it every attempt costs a dummy commit on `main`.
## Dependency updates (Renovate)
@@ -350,6 +406,11 @@ These guidelines are **binding for every current and future chart** in this repo
- Every Deployment/StatefulSet has a `replicas` value (default `1`, the values path per workload matches its section) that also accepts an explicit `0` for per-workload fine control.
- Every chart has a global `scaleDown` flag (default `false`): `scaleDown: true` takes precedence over all `replicas` values, renders every Deployment/StatefulSet with `replicas: 0` and suspends CronJobs. It **stops** the app but does not uninstall it — PVCs, Secrets, Services and Ingress stay in place; setting it back to `false` brings the workloads back up.
- Implemented via a per-chart `<chart>.replicas` helper in `_helpers.tpl` (reference in `template-chart`); the helper uses an explicit nil check (`kindIs "invalid"`) because Helm's `default` would swallow an explicit `0`.
- **A chart whose default is more than one replica ships a `PodDisruptionBudget`** — `policy/v1`, named `<release>-<chart>-pdb`, `minAvailable` as a value (default `1`), selector matching the workload's `app` label. `keycloak` (Gitea #83) is the first and sets the convention. The reason it is a rule and not a per-chart taste: a node drain is the most common reason a pod moves — it is what a node upgrade does — and without a budget the drain is free to take every replica at once, so the second replica buys nothing against the failure it was added for.
- **And it is rendered only while the effective replica count is `>= 2`.** `minAvailable: 1` with exactly one replica forbids every voluntary eviction, so `kubectl drain` blocks forever on a pod nothing may move — and the message names the PDB, not the replica count, which sends the reader to the wrong file. The same applies to `scaleDown: true`, where there are no pods and the object would describe a promise about nothing. So the condition is on the **computed** replica count (the `<chart>.replicas` helper), not on the raw value, and `NOTES.txt` prints which of the two happened rather than leaving it to be inferred from `kubectl get pdb`.
- **A chart whose replicas are a property of a *group* gets one PDB per group, with that group's own count** (Gitea #96; `stalwart` is the first, one node group per cluster role). The condition above then applies per group — with `full: 2` and `mx: 1` exactly one object is rendered — and `NOTES.txt` lists the outcome per group rather than once for the release. The name carries the group (`<release>-<chart>-<group>-pdb`), and so does the selector.
- **A budget without a distribution rule is half an answer.** The PDB keeps a drain from taking every replica at once; it says nothing about where the replicas were put in the first place. Where a chart's replicas are meant to survive the loss of a node, it also offers a **per-group switch that renders the `topologySpreadConstraints` entry itself, with a selector the chart fills from its own pod labels** — never one the operator writes, because a spread constraint whose selector matches no pod is trivially satisfied and distributes nothing, with a green rollout and no message. `stalwart` sets the convention: `nodes.<group>.spreadAcrossNodes` (default `true`, gated on the same computed `>= 2`), `maxSkew: 1` on `kubernetes.io/hostname`, `whenUnsatisfiable: DoNotSchedule` so the distribution is a promise rather than an effort, and a generic per-group pass-through (`affinity`, `nodeSelector`, `tolerations`, `topologySpreadConstraints`) for everything the switch does not cover. Three traps come with it and all three belong in the chart's `values.yaml`: `nodeAffinityPolicy: Ignore` must be set **explicitly** (its API default is `Honor`, under which a narrowing `nodeSelector` shrinks the set of counted domains until both replicas fit in one, giving skew 0 — co-location with no `Pending`, no event, and a selector that is *correct*, so no CI check can see it); that field is **beta** and is pruned without a word below Kubernetes 1.26, which brings that exact gap back; and the replica ceiling is **a rule, not a number** — as many replicas per group as there are schedulable workers, because `nodeTaintsPolicy` defaults to `Ignore` and control-plane domains therefore hold the minimum at 0 permanently. That ceiling comes from `nodeTaintsPolicy` and is **not** a consequence of setting `nodeAffinityPolicy`; the two are documented separately for that reason. A user entry colliding with the chart's own on `(topologyKey, whenUnsatisfiable)` **aborts the render naming both values**, because the two alternatives are an API error pointing at a list index and a silent merge over `patchMergeKey: topologyKey` that drops one selector and goes green.
- **Both selectors are held against the rendered pod labels on every PR** by `ci/run-selector-match-test.sh` — see "Selector-match render test" above. Needed because the shared-helper rule above is a convention, and a convention is the thing a copy breaks.
### 12. Values that must not lie (#68, #93, #99, #215)
@@ -377,7 +438,9 @@ Reference implementation: `template-chart/values.schema.json` and `template-char
The other half of the same rule: readiness **must** hang on the dependency, and a separate path is not proof that it does. `urlaubsverwaltung` probed `/actuator/health/liveness` and `/actuator/health/readiness` for two years and still stayed 1/1 Ready through a total database outage (#218), because Spring Boot's `readiness` group contains `readinessState` **alone** — the split paths were right, the group content was empty of anything external. Whatever aggregates behind a probe path (Spring health groups, a `/healthz` a framework assembles for you) has to be read off the **running** endpoint, not off the path name; the chart names the members explicitly (`MANAGEMENT_ENDPOINT_HEALTH_GROUP_READINESS_INCLUDE=readinessState,db`) so the composition is in the template rather than in an upstream default.
**4. A fallback reproduces *this chart's* defaults, not the repo standard.** Five charts deviate from the hardening baseline on purpose. A "cleaned up" standard default is not tidying, it is an outage: `calibre-web-automated` must keep `readOnlyRootFilesystem: false`, because in read-only mode the image disables its PUID/PGID remapping and the runtime uid silently moves from 1000 to 911 — **making existing volume data inaccessible**. The same applies to non-standard values that look like mistakes: `satisfactory` keeps `limitFactor: 2` (that is what holds the memory limit at the published 8Gi) and `fsGroupChangePolicy: OnRootMismatch` (which avoids a recursive chown of multi-GB game files on every start).
**4. A fallback reproduces *this chart's* defaults, not the repo standard.** Several charts deviate from the hardening baseline on purpose (no count here — the previous wording named one and went stale the next time a chart landed). A "cleaned up" standard default is not tidying, it is an outage: `calibre-web-automated` must keep `readOnlyRootFilesystem: false`, because in read-only mode the image disables its PUID/PGID remapping and the runtime uid silently moves from 1000 to 911 — **making existing volume data inaccessible**. The same applies to non-standard values that look like mistakes: `satisfactory` keeps `limitFactor: 2` (that is what holds the memory limit at the published 8Gi) and `fsGroupChangePolicy: OnRootMismatch` (which avoids a recursive chown of multi-GB game files on every start).
**The sharpest case is a capability that is granted and never used.** `stalwart`'s fallback has to carry `capabilities.add: [NET_BIND_SERVICE]`, because the image sets `setcap cap_net_bind_service=+ep` on its binary — which makes it a capability-dumb binary, and `capabilities(7)` then has the kernel fail the **`execve` itself** when the bounding set does not carry the capability: *"If the process did not obtain the full set of file permitted capabilities, then `execve(2)` fails with the error `EPERM`."* So `drop: [ALL]` with the standard empty `add` does not produce an unhardened container, it produces `exec /usr/local/bin/stalwart: operation not permitted` and a chart that cannot start. The capability is then spent entirely on the exec check and is never used to bind anything, which is exactly what makes it a magnet for a hardening pass looking for dead grants. This is why a deviation is documented with its **mechanism** and not only its name: "needed for binding" would have been wrong and would have invited the removal.
**5. A probe default carries the probe's context, not just its timings.** Two different symptoms, both from real charts: `teamspeak-server` addresses its TCP port by the name `file-transfer` because the voice service is UDP and cannot be probed — a default without that name leaves the workload **with no working probe at all**. `patchmon` sends a `Host` header because the server answers **403** to any request whose Host is neither loopback nor a CORS_ORIGIN host — a default without it means the pod never becomes Ready. Port names, `Host` headers and `exec` commands belong in the fallback; where the context is dynamic, pass the root context in and rebuild it (patchmon derives the header from `ingress.domain`).
@@ -404,6 +467,20 @@ Reference implementation: `template-chart/values.schema.json` and `template-char
Until `ytdlp-web-player` (#214) this section read "charts have no dependencies (`charts/` directories are empty)". That rule is gone, replaced by a bar rather than a ban — a dependency is **admissible, not free**, and the reason it stopped being a ban is worth keeping: the chart needed an oauth2-proxy in the data path in front of an application that ships no authentication at all, and proxy and app are only useful wired to each other.
Three charts carry one, and comparing them is the quickest way to see where the bar sits. `ytdlp-web-player` takes **upstream's** `oauth2-proxy` because the app has no authentication and the two are useless apart; there, point 2 above applies in full. `stalwart` (Gitea #65) takes **one of our own published charts**, `valkey`, because Stalwart's **cluster coordinator rides on the in-memory store connection**, so a multi-node install does not work without one. `paperless-ngx` (Gitea #78) takes the same one, because `PAPERLESS_REDIS` is the Celery broker and the application therefore cannot run without a Redis at all — before the subchart, every installation needed a second Helm release whose address lived in a 1Password item and in neither chart. All three are `condition:`-gated and off by default: the subchart is the path for an installation that has no Valkey yet, and one that already has points its connection value at that instead, so the dependency renders nothing.
**The same subchart, opposite configuration — and that comparison is the useful part.** `stalwart` runs it as a **cache** (`persistence.enabled: false` + `allkeys-lru`): its in-memory store holds rate-limiter counters, tokens and pub/sub, all reconstructible, and the cost it accepts is that the limiters fail open under memory pressure. `paperless-ngx` runs it as a **queue** (PVC + `noeviction`, the subchart's defaults): the broker's contents are work, and a dropped consume task is a document that is silently never processed. `allkeys-lru` would not be a trade there but actively wrong, because LRU evicts typeless across the whole keyspace and a backlog of waiting tasks is exactly what it reads as cold. The price `paperless-ngx` accepts instead: the same instance also serves the Django cache and the Celery result backend (both Redis in 3.x), so a full instance **refuses writes** — new tasks included — rather than dropping cold keys, and the answer is memory sizing, not a policy change. The valkey chart treats the two values as a **pair** and warns against mixing them; these two charts are the worked examples of picking either half deliberately.
**A dependency on one of our own charts is cheaper in one way and more dangerous in another.** Point 2 stops applying: there are no third-party defaults to leave unpatched, because the subchart follows these guidelines by being one of ours. In exchange it adds a hazard an external dependency does not have, and it is sharp enough to be a rule:
> **Never change a chart and a chart that depends on it in the same pull request.**
The dependency resolves from the **published index**, not from the working tree, and a chart is published only by a push to `main` that touches it. So a combined PR runs every `helm` command — lint, package, render **and the install-test** — against the **old** subchart, goes green on an artifact that is not the one in the diff, and leaves the subchart's own change exercised by nothing. Two consecutive PRs: change the subchart, merge, let it publish, then raise the `version:` in the dependent chart.
Two mechanical details of a self-referencing dependency, both measured for #65: the index is **anonymously readable**, so `ci/helm-dependencies.sh` needs no credential and resolves the exact build-metadata version (`1.0.0+up9.1.2.alpine`) from the URL alone with an empty `HELM_REPOSITORY_CONFIG` — the state a fresh runner is in. And Renovate's built-in `helmv3` manager does follow a self-hosted Gitea index: a dry run (Renovate 44.132.2) with the dependency pinned low reported `newVersion: "1.0.0+up9.1.2.alpine"`, `versioning: "helm"`, timestamp read from our index. Without that second result the published-version form would have had no update path at all, which is why it was measured rather than assumed.
One more trap, and it is the reason `stalwart` gates its subchart on `valkeySubchart.enabled` rather than the conventional `valkey.enabled`: **everything under the subchart's key is handed to the subchart as its values**, so a `condition:` key placed there is validated against *its* schema. Our charts ship `additionalProperties: false`, so `valkey.enabled` is rejected by the subchart — and because Helm skips a **disabled** subchart's schema validation, the error surfaces only for whoever first switches it on. Put the condition on a key of the parent's own.
**When a dependency earns its place.** All three, not two of three:
1. The dependent piece would otherwise be a second Helm release that only exists to serve this one, with the wiring between them living in neither chart.
+23
View File
@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
+36
View File
@@ -0,0 +1,36 @@
apiVersion: v2
name: bulwark
description: A Helm chart to deploy Bulwark Webmail, a JMAP webmail client (mail, calendar, contacts, files) for a Stalwart mail server
icon: https://avatars.githubusercontent.com/u/266882946
type: application
# appVersion is the exact image tag of ghcr.io/bulwarkmail/webmail.
#
# THE SUFFIX-FREE FORM, AND THAT IS A CHOICE RATHER THAN THE ONLY OPTION. This
# registry publishes every release TWICE - 1.12.0 and v1.12.0 are the same image
# under two names (counted 2026-10-05 against the ghcr.io tags API: 315 tags, and
# both families run the whole way back to 1.3.9). So unlike stalwart, garage,
# matrix-synapse and calibre-web-automated, where the "v" IS the tag and
# therefore part of appVersion and of the "+up" suffix, here it is optional and
# this chart pins the form without it. Do not "normalize" it to v1.12.0 later:
# that would rewrite appVersion and the chart version for an identical image.
#
# The tag census, Renovate's handling of each shape and the three dry-run
# fixtures that prove it are in renovate.json5's bulwark block rather than
# duplicated here. The short version: eight tag shapes, six of them things
# nobody should pin, and plain `docker` versioning excludes all six - so that
# block deliberately carries NO versioningTemplate.
version: 1.0.0+up1.12.0
appVersion: "1.12.0"
maintainers:
- name: jklein
# NO DEPENDENCIES, and nothing here wants one. Bulwark is a client: it holds no
# mail, has no database, no cache and no broker. The mail server it talks to is
# the `stalwart` chart (Gitea #65), a SEPARATE RELEASE reached over the network
# by the URL in bulwark.jmapServerUrl - not a subchart, because the two are
# installed, scaled, upgraded and backed up independently and a JMAP endpoint is
# an address rather than a bundled component. README, "Chart dependencies", for
# the bar a dependency has to clear.
+252
View File
@@ -0,0 +1,252 @@
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
{{- $replicas := int (include "bulwark.replicas" (dict "scaleDown" .Values.scaleDown "replicas" $settings.replicas)) -}}
{{- $sfs := printf "%s-%s-sfs" .Release.Name .Chart.Name -}}
{{- $svc := printf "%s-%s-service" .Release.Name .Chart.Name -}}
{{- $url := include "bulwark.publicUrl" . -}}
Bulwark Webmail {{ .Chart.AppVersion }} installed as release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
{{ $replicas }} replica(s). This is a CLIENT: it holds no mail and has no database.
The messages and the accounts belong to the Stalwart server at
{{ $settings.jmapServerUrl }}
If that server is away, this release is still Ready and still green - see
"WHAT THE PROBES DO NOT TELL YOU" below.
ENDPOINTS
{{ if .Values.ingress.enabled }} Web: {{ $url }}
{{ else }} Web: no Ingress (ingress.enabled: false) - in-cluster only
{{ end }} In-cluster: http://{{ $svc }}.{{ .Release.Namespace }}.svc.cluster.local:{{ $settings.port | int64 }}
{{ if .Values.ingress.enabled }} Admin: {{ $url }}/admin/login
{{ else }} Admin: /admin/login on the in-cluster address above. With no Ingress, reach it with
kubectl -n {{ .Release.Namespace }} port-forward statefulset/{{ $sfs }} 3000:{{ $settings.port | int64 }}
and open http://localhost:3000/admin/login
{{ end }}{{ if .Values.ingress.enabled }}
Bulwark is served at the ROOT of that host and cannot be moved under a path:
NEXT_PUBLIC_BASE_PATH is compiled into the image.
{{ end }}
THE 1PASSWORD ITEMS, AND WHICH FIELDS THEY MUST CARRY
The operator materializes one Secret key per item FIELD. A missing or misnamed
field here does NOT stop the pod - the two file-form secrets simply read as
absent - so the symptom is a feature that is quietly off rather than an error.
{{ .Values.secrets.appSecretRef | default "<secrets.appSecretRef NOT SET>" }}
session-secret openssl rand -base64 32
MANDATORY. Persistent sessions and the encryption of the
synchronized settings both hang on it, and the second
fails SILENTLY without it: settings sync is computed as
"enabled AND a secret exists", so with no secret it is
simply off and nothing says so.
admin-password the admin dashboard's initial password. CONSUMED ONCE -
see step 5 below.
{{ if $settings.oauth.enabled }}
{{ .Values.secrets.oauthSecretRef | default "<secrets.oauthSecretRef NOT SET>" }}
oauth-client-secret the confidential OIDC client's secret, as issued by
the identity provider when the client was registered.
{{ else }}
(no OAuth item - bulwark.oauth.enabled is false, so sign-in is by password)
{{ end }}
The file-form secret{{ if $settings.oauth.enabled }}s are{{ else }} is{{ end }} mounted as a DIRECTORY, not with subPath, and
re-read on every use - so rotating {{ if $settings.oauth.enabled }}either field{{ else }}that field{{ end }} in the vault takes effect
WITHOUT a restart. admin-password is the exception and no restart helps it.
{{ if $settings.oauth.enabled }}
BEFORE YOU INSTALL - THE OIDC CLIENT HAS TO EXIST FIRST
{{ if $settings.oauth.only }}
THIS IS NOT ADVICE AND IT IS NOT A FOLLOW-UP STEP. bulwark.oauth.only is ON,
which means the password form is hidden, which means that WITHOUT A WORKING
OIDC CLIENT THE FIRST SIGN-IN IS IMPOSSIBLE. There is no fallback for users.
{{ end }}
Register a CONFIDENTIAL client with the provider at
{{ $settings.oauth.issuerUrl }}
client id: {{ $settings.oauth.clientId }}
secret: into the vault item's oauth-client-secret field
THE REDIRECT URI CARRIES THE UI LANGUAGE, AND THAT IS THE ONE THING THAT WILL
COST YOU THE FIRST SIGN-IN IF YOU REGISTER A SINGLE URI. Bulwark builds it as
{{ $url }}/<locale>/auth/callback
where <locale> is the language the VISITOR'S BROWSER resolved to - so a German
browser sends /de/auth/callback and an English one /en/auth/callback, from the
same deployment. The language is not in the visible URL (the locale prefix is
"never" in this image), which is exactly why this is surprising: the address
bar shows no language and the redirect_uri contains one.
Register ONE PER LANGUAGE YOU EXPECT, or a wildcard if the provider allows it.
On Keycloak the wildcard is the practical answer:
{{ $url }}/*
Otherwise the 27 locales Bulwark supports are:
ar ca cs da de en es fa fr he hu it ja ko lv mn nb nl pl pt ro ru sk tr uk
zh zh-TW
If a sign-in fails with a redirect_uri mismatch, read the URI out of the
failed request rather than guessing which language it was - that is the value
the provider compares byte for byte.
POST-LOGOUT, IF YOU USE IT: bulwark.oauth.endSession is {{ $settings.oauth.endSession }}, and a
post-logout redirect URI is only sent when OAUTH_POST_LOGOUT_REDIRECT_URI is
set (this chart does not render it). If you add it through bulwark.env, it has
to be registered with the provider too.
The issuer must be reachable BY THE POD, and the endpoints it advertises must
share the issuer's origin or be publicly routable - Bulwark rejects a
discovered endpoint on a private address from a different origin. A
split-horizon DNS setup needs OAUTH_ALLOW_PRIVATE_ENDPOINTS through
bulwark.env; this chart does not render it.
{{ end }}
FIRST INSTALL
1. kubectl -n {{ .Release.Namespace }} rollout status statefulset/{{ $sfs }}
Ready means the HTTP server is accepting. It does NOT mean the mail server
is reachable - nothing in this chart checks that.
2. Confirm the setup wizard really is OFF. It is, as long as
bulwark.jmapServerUrl is set, and the chart refuses to render without it -
but this is the one thing worth seeing rather than trusting, because a
wizard that ran once changes the rules for good (step 4):
kubectl -n {{ .Release.Namespace }} logs statefulset/{{ $sfs }} | grep -i "SETUP REQUIRED"
Nothing found is the correct result. A "SETUP REQUIRED" banner with a token
means Bulwark is in bootstrap mode and something is wrong with the value.
3. Sign in{{ if $settings.oauth.only }} through the identity provider{{ end }} at {{ $url }}
4. READ THIS BEFORE YOU OPEN THE ADMIN DASHBOARD. The dashboard writes
OVERRIDES into config.json on the volume, and an override BEATS this chart
for that key - for good, and silently.
Priority is: admin override > environment variable > default.
So if you save a server URL in the dashboard, changing bulwark.jmapServerUrl
and running helm upgrade afterwards does nothing and reports success. The
same is true of every other key the dashboard can write. What the chart
sets is a DEFAULT, not a setting.
To see which keys are currently overridden:
kubectl -n {{ .Release.Namespace }} exec {{ $sfs }}-0 -- cat /app/data/admin/config.json
An empty object, or a file that does not exist, means this chart is in
force.
5. THE ADMIN PASSWORD IS CONSUMED EXACTLY ONCE. On the first start it is
hashed into /app/data/admin/admin.json and the environment variable is
never read again. Rotating the vault field afterwards has NO EFFECT -
changing the password goes through the dashboard, or by deleting admin.json
from the volume and restarting. Know that before you treat the vault field
as the source of truth.
IF NOBODY CAN SIGN IN - THE WAY BACK, AND IT IS NOT A helm upgrade
Read this now, not when it happens, because the obvious repair does not work.
The situation: the identity provider is down, or its configuration is wrong,
and bulwark.oauth.only hides the password form. No user can get in.
WHAT DOES NOT WORK RELIABLY: helm upgrade with bulwark.oauth.only=false.
oauthOnly is an ordinary config key, and every part of the app reads it
through the priority chain above - not from the environment. If the admin
dashboard's OAuth setup has ever been used, "oauthOnly": true is sitting in
config.json on the volume, and from then on the chart's value is ignored. The
upgrade succeeds and changes nothing.
WHAT DOES WORK, and it does not depend on the identity provider:
a) Open {{ $url }}/admin/login and sign in with the admin-password field
from the vault item. This door has its own password and is not behind
OIDC.{{ if eq $settings.stalwartAdminAccess "auto" }} (bulwark.stalwartAdminAccess is "auto", so a Stalwart
admin also gets in here without that password.){{ end }}
b) In the dashboard, REVERT the override - the dashboard's own "remove
override" on the OAuth settings. That is what puts the chart's value back
in charge.
IF THE DASHBOARD ITSELF IS UNREACHABLE, the last resort is the volume:
kubectl -n {{ .Release.Namespace }} exec {{ $sfs }}-0 -- cat /app/data/admin/config.json
# remove the "oauthOnly" key (and any other override in the way), then
kubectl -n {{ .Release.Namespace }} rollout restart statefulset/{{ $sfs }}
Deleting admin.json in the same directory resets the admin password to
whatever the vault field currently holds, on the next start.
WHAT THE PROBES DO NOT TELL YOU
All three probes are a plain HTTP GET on /api/health, and that endpoint returns
200 whatever state the application is in - for an unauthenticated caller the
body even says "healthy" unconditionally. So a green pod here means: the
process is alive, HTTP is being accepted, the event loop is free, and Next.js
can route.
IT SAYS NOTHING ABOUT THE MAIL SERVER. With Stalwart completely gone, this
release is Ready, green, and serves a web app that cannot fetch a single
message. Monitor the stalwart release for that - there is no endpoint here
that could cover it, and the one that would (/api/setup/test-jmap) sits behind
the setup token this chart disables.
It also says nothing about the reported state: /api/health reports "degraded"
on high V8 heap usage, but only to an authenticated caller asking for details,
and always with HTTP 200. A memory limit set too low therefore produces a pod
that is permanently degraded and permanently Ready.
BACKUPS - THE VOLUME ALONE IS NOT ONE, AND IT MAY NEED PROTECTING ITSELF
Two statements, both of them about {{ .Release.Name }}-{{ .Chart.Name }}-pvc:
1. A BACKUP OF THE VOLUME ALONE IS NOT A BACKUP.{{ if $settings.settingsSync }} The synchronized user
settings on it are encrypted with session-secret.{{ else }} The user settings are not
synchronized in this release, but persistent sessions still are.{{ end }} Lose the vault
item and that data is unreadable out of a perfectly intact volume. Back up
the 1Password item together with the volume, or the restore is a directory
of ciphertext.
2. A BACKUP OF THE VOLUME MAY CONTAIN A SECRET IN CLEARTEXT. If anybody has
used the dashboard's OAuth setup, the OIDC client secret is written as
PLAINTEXT into config.json on this volume. So the backup may need the same
protection the vault gets. Check with:
kubectl -n {{ .Release.Namespace }} exec {{ $sfs }}-0 -- grep -l oauthClientSecret /app/data/admin/config.json
TRAPS WORTH KNOWING BEFORE YOU NEED THEM
- WEB PUSH GOES THROUGH THE VENDOR'S RELAY until somebody changes it, and the
change is NOT a chart value. The built-in default is
https://notifications.relay.bulwarkmail.org and it is compiled into the
image. A self-hosted relay CAN be made the effective default and even
mandatory for all users - through the settings policy in the admin
dashboard, because that setting has no environment variable at all. So this
is a choice you have, not a limitation you are stuck with; it just lives in
the dashboard.
- THERE IS NO ConfigMap IN THIS CHART, so no checksum/config annotation, and
that is not an omission: there is no rendered config file whose change could
fail to restart anything. Repointing secrets.* at a different vault item
DOES roll the pod (checksum/secret-ref). Changing a value INSIDE an item
does not - but for the two file-form secrets it does not need to, because
they are re-read on every use.
{{ if $settings.adminConfigReadOnly }}
- bulwark.adminConfigReadOnly IS ON. The chart is authoritative and no
override can appear - and three things are frozen with it: the settings
policy (so Web Push cannot be moved to your own relay any more), any
override that was ALREADY on the volume (it cannot be removed either), and -
if this release has never started with it off - the admin password, which
means the admin dashboard never comes up and its login answers HTTP 500
while this pod stays Ready.
If you have just enabled this on a fresh install, turn it off, let the pod
start once so admin.json is written, and only then turn it back on.
{{ else }}
- bulwark.adminConfigReadOnly is OFF, which is the default and keeps the
dashboard working. The cost is the precedence trap of step 4: the volume can
override this chart at any time. Turning it on makes the chart authoritative
but freezes the settings policy, freezes any override already present, and
must never be the state of a FIRST install - see values.yaml for the
sequence.
{{ end }}
- THE SETTINGS POLICY, THE THEMES AND THE BRANDING ARE NOT IN THIS CHART.
They live in policy.json on the volume and are set in the dashboard. This
chart neither renders nor backs them up beyond the volume itself.
WHAT THIS CHART DOES NOT RENDER
- No ConfigMap (see above) and no NetworkPolicy. The pod needs egress to the
JMAP server and to the OIDC issuer; a policy added later has to allow both,
or sign-in breaks in a way that looks like a provider outage.
- No PodDisruptionBudget: with one replica a minAvailable of 1 would forbid
every voluntary eviction and make `kubectl drain` hang forever on a pod
nothing may move (README, guideline 11).
- No sub-path Ingress, and no value that could add one - the base path is
compiled into the image.
- No proxy-body-size annotation. Attachment uploads go through the Ingress and
nginx's default of 1m rejects larger ones with a 413 that reads like an
application fault; the right number is the mail server's limit, which this
chart does not know. Set it in ingress.annotations.
+464
View File
@@ -0,0 +1,464 @@
{{/*
Render the body of a container "resources:" block from a values dict of the form
resources:
limitFactor: 3 # optional, default 3
requests: { cpu, memory, ephemeral-storage / ephemeralStorage, ... }
limits: { ... } # optional, every entry optional
Rules (variant B, see issue #19):
- requests are rendered as given (camelCase "ephemeralStorage" is normalized
to the Kubernetes key "ephemeral-storage").
- an explicitly set limit always wins.
- a missing memory / ephemeral-storage limit is computed as
limitFactor x the corresponding request.
- a missing cpu limit is NOT set at all (the kubelet then treats CPU as
unlimited; CPU is compressible, so this cannot kill the workload).
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart, and a dependency is admissible but expensive (README, guideline 14) -
this chart takes none. The reference implementation is
template-chart/templates/_helpers.tpl.
*/}}
{{- define "bulwark.resources" -}}
{{- $resources := . | default dict -}}
{{- $factor := 3.0 -}}
{{- if not (kindIs "invalid" $resources.limitFactor) -}}
{{- $factor = float64 $resources.limitFactor -}}
{{- end -}}
{{- $requests := dict -}}
{{- range $key, $value := ($resources.requests | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $requests (include "bulwark.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- $limits := dict -}}
{{- range $key, $value := ($resources.limits | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $limits (include "bulwark.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- range $key, $request := $requests -}}
{{- if and (ne $key "cpu") (not (hasKey $limits $key)) -}}
{{- $_ := set $limits $key (include "bulwark.multiplyQuantity" (dict "quantity" $request "factor" $factor)) -}}
{{- end -}}
{{- end -}}
{{- $out := dict -}}
{{- with $requests }}{{- $_ := set $out "requests" . }}{{- end -}}
{{- with $limits }}{{- $_ := set $out "limits" . }}{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Normalize a resource name from values to its Kubernetes key.
*/}}
{{- define "bulwark.resourceKey" -}}
{{- if eq . "ephemeralStorage" -}}ephemeral-storage{{- else -}}{{ . }}{{- end -}}
{{- end -}}
{{/*
Multiply a Kubernetes quantity by a factor, keeping its suffix.
Input: dict "quantity" <string|number> "factor" <float64>.
*/}}
{{- define "bulwark.multiplyQuantity" -}}
{{- $quantity := include "bulwark.formatNumber" .quantity -}}
{{- $number := regexFind "^[0-9]+(\\.[0-9]+)?" $quantity -}}
{{- if not $number -}}
{{- fail (printf "cannot parse resource quantity %q" $quantity) -}}
{{- end -}}
{{- $suffix := trimPrefix $number $quantity -}}
{{- if not (regexMatch "^(m|k|M|G|T|P|E|Ki|Mi|Gi|Ti|Pi|Ei)?$" $suffix) -}}
{{- fail (printf "unsupported resource quantity suffix %q in %q" $suffix $quantity) -}}
{{- end -}}
{{- $product := mulf (float64 $number) .factor -}}
{{- if eq $product (floor $product) -}}
{{- printf "%d%s" (int64 $product) $suffix -}}
{{- else -}}
{{- printf "%s%s" (include "bulwark.formatNumber" $product) $suffix -}}
{{- end -}}
{{- end -}}
{{/*
Render a number as a plain decimal string, never in exponential notation
(issue #157): the null-safe fallback helpers pass values through fromYaml,
which parses every number as float64, and Go's "%v" switches float64 to
exponential notation from about 1e6 upwards - "1.048576e+06" is not a valid
Kubernetes quantity. Strings ("512Mi") are passed through untouched.
*/}}
{{- define "bulwark.formatNumber" -}}
{{- if kindIs "float64" . -}}
{{- regexReplaceAll "\\.?0+$" (printf "%.10f" .) "" -}}
{{- else -}}
{{- printf "%v" . -}}
{{- end -}}
{{- end -}}
{{/*
Render the replica count for the StatefulSet.
Input: dict "scaleDown" <flag|nil> "replicas" <value|nil>.
- scaleDown: true takes precedence and always yields 0.
- otherwise an explicitly set replicas value is used as given, including an
explicit 0 - Helm's "default" would swallow 0, hence the kindIs "invalid"
nil check.
- a missing replicas value defaults to 1.
A NOTE SPECIFIC TO THIS CHART: the schema caps this at 1, and the reason is the
single ReadWriteOnce PVC rather than anything about the application. Two pods
cannot both mount it, so a second replica would sit in ContainerCreating
forever - and the PVC is not optional here, because /app/data carries the admin
configuration, the audit log and the synchronized user settings. An RWX volume
would not make it a cluster either: nothing in Bulwark coordinates two
processes over that directory.
*/}}
{{- define "bulwark.replicas" -}}
{{- if .scaleDown -}}
0
{{- else if not (kindIs "invalid" .replicas) -}}
{{- .replicas -}}
{{- else -}}
1
{{- end -}}
{{- end -}}
{{/*
Merge a user-supplied values sub-tree onto a dict of chart defaults so that a
null / empty override falls back to the defaults instead of erasing them
(issues #61, #99). Input: dict "defaults" <dict> "given" <dict|nil> "path"
<string>. Output: YAML of the merged dict (consume it with fromYaml).
An override key written without children is YAML null, and Helm treats that
null as a deletion of the chart's default. This helper defines the opposite
semantics - an empty block means "chart defaults apply" - while an explicitly
set value still wins, including false and 0.
*/}}
{{- define "bulwark.defaultedDict" -}}
{{- $defaults := .defaults | default dict -}}
{{- $given := .given -}}
{{- $path := .path | default "value" -}}
{{- if kindIs "invalid" $given -}}
{{- $given = dict -}}
{{- end -}}
{{- if not (kindIs "map" $given) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" $path (kindOf $given) $given) -}}
{{- end -}}
{{- $out := deepCopy $defaults -}}
{{- range $key, $value := $given -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $default := index $defaults $key -}}
{{- if and (kindIs "map" $value) (kindIs "map" $default) -}}
{{- $_ := set $out $key (fromYaml (include "bulwark.defaultedDict" (dict "defaults" $default "given" $value "path" (printf "%s.%s" $path $key)))) -}}
{{- else if kindIs "map" $default -}}
{{- fail (printf "%s.%s must be a mapping or null, got %s (%v)" $path $key (kindOf $value) $value) -}}
{{- else -}}
{{- $_ := set $out $key $value -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Pick one sub-block out of a values block that may itself be null.
Input: dict "block" <dict|nil> "key" <string> "path" <string>.
Returns YAML of dict "value" <the sub-block, may be nil>.
*/}}
{{- define "bulwark.subBlock" -}}
{{- $block := .block -}}
{{- if kindIs "invalid" $block -}}
{{- $block = dict -}}
{{- end -}}
{{- if not (kindIs "map" $block) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" .path (kindOf $block) $block) -}}
{{- end -}}
{{- toYaml (dict "value" (index $block .key)) -}}
{{- end -}}
{{/*
THE EFFECTIVE BULWARK SETTINGS - everything the StatefulSet, the Service and
NOTES.txt read out of .Values.bulwark, merged onto this chart's defaults.
Why the whole settings block goes through the null-safe helper and not only the
three usual suspects (guideline 12.2: "the rule is about the shape, not about
those three block kinds"): every value below is dereferenced by a template, and
an erased one would not abort the render. An erased bulwark.port renders an
empty containerPort and an empty Service targetPort - rejected by nothing until
the API server sees it; an erased bulwark.oauth.enabled would silently turn OIDC
OFF on a release whose whole login path is OIDC, which is the quiet half and the
worse one. Satisfactory shipped exactly this shape with half its blocks
null-safe and half not (Gitea #16).
The defaults here MUST stay in sync with values.yaml, which remains the
documented, user-facing place for them; this copy is only the fallback.
Usage: {{ $settings := fromYaml (include "bulwark.settings" .) }}
*/}}
{{- define "bulwark.settings" -}}
{{- $given := (fromYaml (include "bulwark.subBlock" (dict "block" .Values "key" "bulwark" "path" ".Values"))).value -}}
{{- $defaults := dict
"replicas" 1
"port" 3000
"jmapServerUrl" nil
"appName" nil
"stalwartFeatures" true
"stalwartAdminAccess" "auto"
"settingsSync" true
"adminConfigReadOnly" false
"telemetry" false
"updateCheck" false
"logLevel" "info"
"logFormat" "json"
"writableCache" (dict
"tmpSizeLimit" "256Mi"
"nextCacheSizeLimit" "512Mi")
"oauth" (dict
"enabled" true
"only" true
"clientId" nil
"issuerUrl" nil
"scopes" nil
"endSession" true) -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" $given "path" "bulwark") -}}
{{- end -}}
{{/*
THE DATA DIRECTORY, defined once because two templates have to agree on it: the
StatefulSet mounts the PVC here and NOTES.txt names it.
/app/data, and it is NOT a value. Bulwark resolves its four directories
relative to the process working directory - getConfigDir() falls back to
<cwd>/data/admin and getStateDir() to <cwd>/data/admin-state (lib/admin/paths.ts
at tag 1.12.0) - and cwd is /app (the image's WORKDIR). A path that disagreed
with the mount would put the admin configuration and the audit log into the
container's ephemeral layer, which looks like it works until the pod is
replaced. The image creates all four subdirectories and chowns them to
1001:1001 (Dockerfile: mkdir -p /app/data/{settings,admin,admin-state,telemetry}
&& chown -R nextjs:nodejs /app/data) - but that happens in the IMAGE LAYER,
which the PVC mount covers, so the subdirectories are created again at runtime
by Bulwark itself (ensureConfigDir / ensureStateDir, both mkdir -p). fsGroup is
what makes the empty volume writable for uid 1001.
*/}}
{{- define "bulwark.dataDir" -}}/app/data{{- end -}}
{{/*
WHERE THE TWO FILE-FORM SECRETS ARE MOUNTED, and the one property of this mount
that must not be "tidied up": it is a WHOLE-DIRECTORY mount with NO subPath.
The kubelet never refreshes a subPath mount. Both files are re-read on every
use - getSessionSecret() calls readFileEnv on every call
(lib/auth/session-secret.ts) and getGlobalClientSecret() does the same on every
token exchange (lib/oauth/token-exchange.ts, no cache) - so rotating a field
inside the vault item takes effect WITHOUT a pod restart. With a subPath that
property is silently lost: the file in the container would keep the old value
forever while the vault, the Secret and `helm upgrade` all look correct.
*/}}
{{- define "bulwark.secretMountPath" -}}/etc/bulwark/secrets{{- end -}}
{{/*
The effective security contexts, probes and resources (issue #99).
The defaults below MUST stay in sync with values.yaml. They reproduce THIS
chart's defaults, not the repo standard.
*/}}
{{- define "bulwark.podSecurityContext" -}}
{{- $given := (fromYaml (include "bulwark.subBlock" (dict "block" . "key" "pod" "path" "bulwark.securityContext"))).value -}}
{{- $defaults := dict
"runAsNonRoot" true
"runAsUser" 1001
"runAsGroup" 1001
"fsGroup" 1001
"fsGroupChangePolicy" "Always"
"seccompProfile" (dict "type" "RuntimeDefault") -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" $given "path" "bulwark.securityContext.pod") -}}
{{- end -}}
{{- define "bulwark.containerSecurityContext" -}}
{{- $given := (fromYaml (include "bulwark.subBlock" (dict "block" . "key" "container" "path" "bulwark.securityContext"))).value -}}
{{- $defaults := dict
"allowPrivilegeEscalation" false
"readOnlyRootFilesystem" true
"runAsNonRoot" true
"capabilities" (dict "drop" (list "ALL") "add" (list)) -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" $given "path" "bulwark.securityContext.container") -}}
{{- end -}}
{{/*
ALL THREE PROBES ARE AN HTTP GET ON /api/health, AND WHAT THAT DOES AND DOES NOT
CARRY IS MEASURED AGAINST THE SOURCE AT TAG 1.12.0 - not read off the path name,
which is the mistake #218 and Gitea #83 each cost this repo once.
FIRST, WHY GET AND NOT HEAD, because the plan for this chart (Gitea #89) said
HEAD and HEAD IS NOT AVAILABLE TO A KUBERNETES PROBE AT ALL. The API type is
HTTPGetAction and it has no method field; the kubelet's prober issues a GET and
there is no value anywhere in a pod spec that changes that. The only way to send
a HEAD would be an exec probe shelling out to a HTTP client - and this image
contains no curl and no wget (npm and npx are deliberately removed from the
runtime layer), so it would mean `node -e` on every probe tick: a new Node
process every few seconds against an endpoint whose GET handler is already
cheap. That is strictly worse, so the chart uses the GET that Kubernetes
actually offers.
WHAT CHANGES BECAUSE OF THAT: almost nothing, and in the direction that helps.
Upstream's HEAD handler returns `new Response(null, { status: 200 })` and
computes nothing. The GET handler does gather diagnostics, which makes it very
slightly more capable than HEAD rather than less: its catch block answers 503
when `process.memoryUsage()` or `v8.getHeapStatistics()` itself throws. Every
limitation argued below is unchanged.
WHAT THE ENDPOINT DOES. The GET success path returns a hard `status: 200`
whatever state it computed, and for an UNAUTHENTICATED caller the body is
hard-coded to `healthy` as well (`status: detailed ? status : 'healthy'`);
`degraded` appears only with `?detailed=true` AND a caller that passes
canSeeInstanceDetails. Upstream tests exactly that -
lib/__tests__/health-route.test.ts expects HTTP 200 while `payload.status` is
`degraded`. A probe is unauthenticated and asks for no details, so it sees 200
and "healthy" always.
SO THE PROBE CANNOT REPORT THE APPLICATION'S STATE - and this is deliberately
NOT filed as "a probe that cannot fail", because that would be wrong and would
invite somebody to replace it with something worse. What a 200 here really
proves, all of it load-bearing: the Node process is alive, the HTTP server is
accepting, the event loop is not blocked, and Next.js can route a request. An
OOM kill, a crash, a wedged event loop and a container that never finished
booting all fail this probe. That is more than #218's endpoint carried and it is
exactly what a liveness and a startup probe are for.
WHAT IT DOES NOT CARRY, and no probe in this chart can: anything about
STALWART. Bulwark holds no mail and this endpoint does not ask the JMAP server
anything, so with the mail server gone /api/health is green and the pod stays
Ready. There is no better endpoint - /api/setup/test-jmap would really test the
connection, but it lives behind the bootstrap setup token and is unreachable in
this chart (the wizard is disabled by jmapServerUrl). The limit is written at
the readiness value in values.yaml because readiness is the probe whose promise
it weakens.
*/}}
{{- define "bulwark.livenessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/api/health" "port" "http" "scheme" "HTTP")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.livenessProbe") -}}
{{- end -}}
{{- define "bulwark.readinessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/api/health" "port" "http" "scheme" "HTTP")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.readinessProbe") -}}
{{- end -}}
{{/*
STARTUP: the same endpoint, and it gates the other two - which is why neither of
them carries an initialDelaySeconds.
BUDGET = periodSeconds x failureThreshold = 5 x 24 = 120s. What it has to cover
is not Bulwark's own boot, which is a Next.js standalone server starting in
about a second: it is attaching and mounting the network volume, and the
instrumentation hook that runs BEFORE the first request is served -
instrumentation.node.ts migrates the legacy admin layout, loads config.json and
policy.json off the volume, and on a first install hashes ADMIN_PASSWORD with
scrypt and writes admin.json. The scrypt hash is the one deliberately slow step
in there.
*/}}
{{- define "bulwark.startupProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/api/health" "port" "http" "scheme" "HTTP")
"periodSeconds" 5
"timeoutSeconds" 5
"failureThreshold" 24 -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.startupProbe") -}}
{{- end -}}
{{/*
The effective resources block, fed into the resources helper above so that an
erased "resources:" still yields the chart's requests and computed limits.
*/}}
{{- define "bulwark.effectiveResources" -}}
{{- $defaults := dict
"limitFactor" 3
"requests" (dict "cpu" "100m" "memory" "256Mi" "ephemeral-storage" "64Mi") -}}
{{- $merged := fromYaml (include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.resources")) -}}
{{- include "bulwark.resources" $merged -}}
{{- end -}}
{{/*
The additive env block, rendered through tpl with the value, secretKeyRef and
configMapKeyRef forms (guideline 10).
ADDITIVE MEANS ADDITIVE: an entry repeating a name this chart already sets does
not replace it. At install time both reach the pod and the later one wins; on an
EXISTING release the pair collapses to the chart's own value, because Kubernetes
merges env by name - and helm upgrade still reports success. So do not use this
block to override JMAP_SERVER_URL, OAUTH_ONLY or SESSION_SECRET_FILE; change the
values that produce them.
AND THERE IS A SECOND, SHARPER REASON NOT TO USE IT FOR BULWARK'S OWN SETTINGS,
specific to this application: an environment variable is only the MIDDLE layer
of three. configManager.get resolves "admin override > env var > default"
(lib/admin/config-manager.ts), so any key the admin dashboard has written into
config.json on the PVC beats whatever arrives here - silently. An entry added
through this block is therefore not a guarantee, it is a default that the volume
may already be overriding. values.yaml, under bulwark.adminConfigReadOnly, has
the whole precedence story.
*/}}
{{- define "bulwark.env" -}}
{{- $root := .root -}}
{{- range $value := .env }}
{{- if $value.value }}
- name: {{ $value.name | quote }}
# quote is required, not cosmetic: a value like "3000" or "true" renders
# unquoted as a YAML number/bool, and the API server then rejects the whole
# manifest with "cannot unmarshal number into Go struct field EnvVar.value of
# type string".
value: {{ tpl $value.value $root | quote }}
{{- else if $value.valueFrom }}
{{- if $value.valueFrom.secretKeyRef }}
- name: {{ $value.name }}
valueFrom:
secretKeyRef:
name: {{ tpl $value.valueFrom.secretKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.secretKeyRef.key $root | quote }}
{{- else if $value.valueFrom.configMapKeyRef }}
- name: {{ $value.name }}
valueFrom:
configMapKeyRef:
name: {{ tpl $value.valueFrom.configMapKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.configMapKeyRef.key $root | quote }}
{{- end }}
{{- end }}
{{- end }}
{{- end -}}
{{/*
The public base URL, derived from ingress.domain. NOTES.txt prints it and builds
the OIDC redirect URI from it, which is why it is a helper rather than several
copies of the same printf.
DELIBERATELY NO required() HERE, and that is the opposite of what it first looks
like it should have. The only caller is NOTES.txt, and `helm template` does not
render NOTES.txt (README, guideline 12.2) - so a required() in this helper would
be a guard that cannot fire in lint, in the render test or in CI, and would fire
for the first time during a real `helm install`. A guard that only runs in
production is worse than no guard.
The honest position for THIS chart, checked in both directions as guideline 12.1
asks: nothing outside the Ingress template FUNCTIONALLY reads ingress.domain.
Bulwark derives its own public URL from the request origin and takes no base-URL
variable, so with ingress.enabled: false the release is complete without a
domain - the calibre-web-automated case, where the gate frees both values. The
enforcement therefore sits entirely in bulwark.ingress.yaml, where it is reached
by every render that has the Ingress on (the default), and this helper renders a
visible placeholder instead of failing.
*/}}
{{- define "bulwark.publicUrl" -}}
{{- if .Values.ingress.domain -}}
https://{{ .Values.ingress.domain }}
{{- else -}}
https://<ingress.domain NOT SET>
{{- end -}}
{{- end -}}
+67
View File
@@ -0,0 +1,67 @@
{{- /*
ON BY DEFAULT (ingress.enabled: true): a webmail client nobody can reach is not
a webmail client.
AT THE ROOT OF THE HOST, AND THERE IS NO VALUE TO PUT IT UNDER A PATH.
NEXT_PUBLIC_BASE_PATH is compiled into the bundle - upstream: "These are
compiled into the bundle and cannot be changed at runtime" - so a sub-path
deployment needs a rebuilt image, which is outside what this repository does
(same limit as garage #81 and keycloak #83). A rewrite-target annotation is not a
way around it either: the bundle's own asset and API URLs are built for the root,
so stripping a prefix at the proxy produces a page that loads and then fetches
everything from the wrong place. The rule below is therefore "/" with pathType
Prefix, deliberately and permanently.
GUIDELINE 1, CHECKED IN BOTH DIRECTIONS (README, "Values that must not lie"),
AND FOR THIS CHART THE GATE FREES BOTH VALUES. Checked rather than assumed:
nothing outside this template FUNCTIONALLY reads ingress.domain. Bulwark derives
its own public URL from the request origin and takes no base-URL environment
variable - there is no KC_HOSTNAME equivalent here - so a release with
ingress.enabled: false is complete without a domain. That is the
calibre-web-automated case rather than the matrix-synapse one.
The one other READER is NOTES.txt, which prints the access URL and the OIDC
redirect URI. That read is deliberately NOT guarded with a required(): `helm
template` does not render NOTES.txt, so a guard there would fire for the first
time during a real `helm install` and never in lint, in the render test or in CI.
The bulwark.publicUrl helper renders a visible placeholder instead, and the
enforcement lives here - in the template that is reached by every render with the
Ingress on, which is the default.
*/ -}}
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-ingress
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
annotations:
cert-manager.io/cluster-issuer: {{ required "A Cluster-Issuer must be provided (ingress.clusterIssuer)" .Values.ingress.clusterIssuer }}
{{- /* Merged, not replaced. The one annotation worth adding here for this
application is nginx.ingress.kubernetes.io/proxy-body-size: attachment
uploads and the Files view go through this Ingress, and nginx's
default of 1m rejects anything larger with a 413 that reads like an
application fault. The chart does not set it because the right number
is the mail server's limit, which this chart does not know. */}}
{{- with .Values.ingress.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
ingressClassName: {{ .Values.ingress.className }}
rules:
- host: {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
http:
paths:
- backend:
service:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
port:
# BY NAME, so bulwark.port and this file cannot drift apart.
name: http
path: /
pathType: Prefix
tls:
- hosts:
- {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
secretName: tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
{{- end }}
+50
View File
@@ -0,0 +1,50 @@
{{- /*
ONE claim, mounted at /app/data, holding all FOUR of Bulwark's directories as
subdirectories of it:
admin config.json, policy.json, admin.json (the password hash),
plugin-config/, plugins/, themes/, branding uploads
admin-state admin-state.json (login timestamps), audit.log, the bootstrap
setup token
settings the encrypted per-account user settings
telemetry consent and instance id
ONE VOLUME AND NOT FOUR, because a snapshot is atomic per volume and never
across volumes - the durability lesson garage (#81) was built on. Split up, a
backup can capture an audit log that records logins the configuration does not
know about yet, or settings encrypted against a secret the configuration has
since been repointed from. There is no argument on the other side here: this is
a few hundred kilobytes of JSON plus an append-only log, not an object store.
Bulwark creates all four subdirectories itself on start (ensureConfigDir and
ensureStateDir, both mkdir -p). It has to: the image's own `mkdir -p
/app/data/...` ran in a layer that this mount covers. fsGroup is what makes the
freshly provisioned volume writable for uid 1001.
DELIBERATELY NOT null-SAFE: an erased size yields an invalid PVC that is
rejected at apply time, which is loud. A chart-side fallback would silently size
somebody's volume - and on an existing release it could not help anyway, because
a PVC cannot shrink. Same reasoning as garage's storage block and satisfactory's
documented exception (Gitea #39).
NEVER RENAMED. A renamed PVC makes Kubernetes provision a new, empty volume and
orphans the data - here that means admin.json, the only copy of the admin
password hash, and audit.log, which by its nature cannot be reconstructed.
*/ -}}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
accessModes:
# ReadWriteOnce, and the single replica follows from it rather than the other
# way round - see the bulwark.replicas helper.
- ReadWriteOnce
resources:
requests:
storage: {{ .Values.storage.amount }}
{{- if .Values.storage.storageClass }}
storageClassName: {{ .Values.storage.storageClass }}
{{- end }}
+109
View File
@@ -0,0 +1,109 @@
{{- /*
The 1Password items this chart materializes. TWO of them, split BY AUDIENCE
rather than by count:
<release>-bulwark-app-secret this release's own secrets - the session
secret and the admin password. Nobody else
knows or needs them.
<release>-bulwark-oauth-secret the OIDC client secret. NOT ours: the
identity provider (keycloak, #83) creates it
when the client is registered and rotates it
there, so it is the one of the three that a
second system also holds. A separate item is
what lets it be repointed and rotated without
touching the other two.
Same reasoning as garage's split between its RPC secret and its S3 key.
WHAT THE FIELD NAMES ARE AND WHY THEY ARE NOT FREE-FORM: the operator
materializes one Secret key per FIELD of the item, and this chart references
those keys by name - two as file paths in the StatefulSet's volumeMounts, one by
secretKeyRef. An item without the expected field leaves the container in
CreateContainerConfigError, or - worse for the file form - produces a container
that starts with a MISSING file, which readFileEnv answers with null rather than
an error. NOTES.txt lists every field for the installer.
TWO SECRETS, TWO DIFFERENT ROTATION STORIES, and the difference is a property of
the application rather than of this chart:
session-secret, oauth-client-secret FILE form. Re-read on every use with no
cache, so a rotation is live WITHOUT a
restart - see the whole-directory mount
in bulwark.sfs.yaml, which is what makes
that true.
admin-password CONSUMED ONCE. initAdminPassword hashes
it on the first start, writes admin.json
to the PVC, and never looks at the
variable again. Rotating the field after
that has NO effect, restart or not.
AND THE LIMIT THAT CANNOT BE CLOSED FOR EITHER: a change to the CONTENT of an
item restarts nothing, because Helm never sees a secret the 1Password operator
materializes and there is nothing to hash. What this chart does cover is the
op:// PATH, through checksum/secret-ref on the pod template. The operator's
auto-restart annotation is NOT set here and must not be: verified against
operator 1.12.0 it enumerates Deployments only, so on a StatefulSet it is a
silent no-op advertising a guarantee it does not provide (README, guideline 9).
*/ -}}
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-app-secret
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
# Must carry the fields: session-secret, admin-password
#
# session-secret: 32 bytes of randomness, base64 - `openssl rand -base64 32`.
# MANDATORY. Persistent "remember me" sessions and the encryption of the
# synchronized user settings both hang on it, and the second one fails
# SILENTLY without it: app/api/config/route.ts computes
# `settingsSyncEnabled: configManager.get(...) && hasSessionSecret()`, so
# bulwark.settingsSync: true plus a missing secret means synchronization is
# simply off, with no error anywhere.
#
# LOSING IT LOSES DATA THAT AN INTACT VOLUME STILL HOLDS: the settings on the
# PVC are encrypted with it. This item belongs in the backup next to the
# volume - a backup of the volume alone is not a backup.
#
# admin-password: the initial password of the ADMIN DASHBOARD.
# It is set because the dashboard is the way back out of an OIDC lockout
# (see bulwark.oauth.only in values.yaml), not because of any secret leak -
# measured, there is none: without ADMIN_PASSWORD Bulwark generates no
# random password and logs nothing, it just disables the dashboard.
#
# CONSUMED ON THE FIRST START ONLY. After admin.json exists on the PVC this
# field is never read again, so rotating it here changes nothing; the reset
# goes through the dashboard or by deleting admin.json.
itemPath: {{ required "You must provide an app secret reference (secrets.appSecretRef) - the 1Password item holding the fields 'session-secret' (openssl rand -base64 32; mandatory, the settings sync is silently disabled without it) and 'admin-password' (the admin dashboard, which is the only way back out of an OIDC lockout)" .Values.secrets.appSecretRef }}
{{- if $settings.oauth.enabled }}
---
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-oauth-secret
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
# Must carry the field: oauth-client-secret
#
# The confidential OIDC client's secret, as issued by the identity provider
# when the client is registered. The registration is a PRE-INSTALL step, not a
# follow-up: with bulwark.oauth.only (the default) and no working client, the
# first sign-in is impossible. NOTES.txt prints the redirect URI to register.
#
# It arrives as a FILE, through OAUTH_CLIENT_SECRET_FILE, so it is not in the
# container's process environment - which garage (#81) and keycloak (#83) could
# not achieve because neither application offered a file form. #81's other half
# was checked here and does not apply: lib/read-file-env.ts is readFileSync
# plus try/catch with no mode check, so the group-read bit fsGroup adds is
# harmless.
#
# AND THE ONE WAY A ROTATION SILENTLY DOES NOTHING: getGlobalClientSecret
# checks the ADMIN OVERRIDE FIRST. If the dashboard's OAuth setup has ever
# run, config.json on the PVC carries oauthClientSecret in PLAINTEXT and that
# value beats this file forever. That is also why a backup of the volume can
# contain this secret in the clear.
itemPath: {{ required "You must provide an OAuth client secret reference (secrets.oauthSecretRef) - the 1Password item holding the field 'oauth-client-secret', as issued by the identity provider when the OIDC client was registered. Required while bulwark.oauth.enabled is true; set it to false for a password-login install" .Values.secrets.oauthSecretRef }}
{{- end }}
+48
View File
@@ -0,0 +1,48 @@
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
selector:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
ports:
# The one port this application serves. targetPort addresses the container
# port BY NAME, so bulwark.port can be changed freely while renaming the port
# would break this and all three probes at once.
- protocol: TCP
port: {{ $settings.port | int64 }}
targetPort: http
name: http
# ClusterIP, and no value to change it. The way out of the cluster is the
# Ingress with a certificate: this is a webmail client that handles session
# cookies and mail content, so a NodePort would put it on a routable address
# over plain HTTP by accident.
type: ClusterIP
---
{{/*
The headless Service a StatefulSet needs for its serviceName. It exists because
a StatefulSet requires one, not because anything addresses the pod by its stable
DNS name - with one replica and no peer coordination there is nothing to address.
Kept separate from the ClusterIP Service above so that the latter stays a plain
load-balanced endpoint for the Ingress; pointing serviceName at a Service with a
ClusterIP is accepted by the API server but gives the pod no per-pod DNS record,
which is the kind of half-working arrangement that only shows up later.
*/}}
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-headless
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
clusterIP: None
selector:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
ports:
- protocol: TCP
port: {{ $settings.port | int64 }}
targetPort: http
name: http
+329
View File
@@ -0,0 +1,329 @@
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
{{- /*
A StatefulSet and not a Deployment, and the reason is the single ReadWriteOnce
PVC rather than any cluster identity: a rolling Deployment update blocks itself,
because the new pod waits for a volume the old pod still holds. The repo's
convention requires a StatefulSet for a stateful application anyway, and this one
is stateful - /app/data carries the admin configuration, the admin password hash,
the audit log and the encrypted user settings.
NO volumeClaimTemplates: this repo uses standalone PVC templates throughout, so
the StatefulSet can be renamed in a major PR with a --cascade=orphan note without
the data following the name (README, guideline 1).
NO INIT CONTAINER, and that is checked rather than skipped. Bulwark creates its
four data subdirectories itself on start (ensureConfigDir / ensureStateDir, both
mkdir -p), which it has to because the image's own mkdir ran in a layer this
mount covers; fsGroup makes the fresh volume writable for uid 1001. There is
nothing to seed and nothing to chown, so an init container would only be a second
image to pin and update.
GUARD AT RENDER TIME, not at start: a missing OIDC client id or issuer with
oauth.enabled would otherwise produce a release that installs cleanly, becomes
Ready, and then answers every sign-in attempt with a broken redirect - and with
oauth.only there is no password form to fall back on. The render fails instead,
naming the value. It lives HERE rather than in NOTES.txt because `helm template`
does not render NOTES.txt (guideline 12.2), so a guard there would pass every
check in CI while being absent.
*/ -}}
{{- if $settings.oauth.enabled }}
{{- if not $settings.oauth.clientId }}
{{- fail "bulwark.oauth.clientId is required while bulwark.oauth.enabled is true - without it the OIDC redirect is built against an empty client and every sign-in fails. With bulwark.oauth.only (the default) there is no password form to fall back to, so this would be an install nobody can log into. Register the client with the identity provider first; NOTES.txt prints the redirect URI." }}
{{- end }}
{{- if not $settings.oauth.issuerUrl }}
{{- fail "bulwark.oauth.issuerUrl is required while bulwark.oauth.enabled is true - Bulwark fetches the OIDC discovery document from it, and without it there is nothing to discover. For keycloak this is https://<host>/realms/<realm>." }}
{{- end }}
{{- end }}
{{- if not $settings.jmapServerUrl }}
{{- /* Not merely "the mail server is unset": an empty JMAP_SERVER_URL puts
detectSetupState() into "bootstrap" (lib/setup/state.ts), which serves the
FIRST-RUN WIZARD at /setup and 302s everything else there. The wizard then
writes its result into config.json on the PVC - and from that moment the
volume beats every value in this chart, permanently and silently. So an
unset value here does not produce a non-working release, it produces a
SILENTLY DIFFERENT ONE, which is worse. */}}
{{- fail "bulwark.jmapServerUrl is required - it is the address of the Stalwart server AND the switch that disables the first-run setup wizard. Left empty, Bulwark starts in bootstrap mode, serves its wizard at /setup, and whatever is clicked through there lands in config.json on the PVC - which from then on OVERRIDES every value this chart sets, silently. Set it to the stalwart release's JMAP endpoint." }}
{{- end }}
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
replicas: {{ include "bulwark.replicas" (dict "scaleDown" .Values.scaleDown "replicas" $settings.replicas) }}
serviceName: {{ .Release.Name }}-{{ .Chart.Name }}-headless
selector:
matchLabels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
template:
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
annotations:
# NO checksum/config ANNOTATION, AND THAT IS NOT AN OMISSION (#82).
# This chart renders no ConfigMap: Bulwark is configured through
# environment variables plus the two JSON files it writes itself onto the
# PVC. There is no rendered config file whose change could fail to
# restart the workload, so the rule has nothing to attach to. A changed
# environment variable changes the pod template directly and rolls the
# pod, which is what the annotation exists to achieve elsewhere.
#
# Hash of the rendered OnePasswordItems - the op:// PATHS, not their
# contents. Repointing secrets.appSecretRef or secrets.oauthSecretRef at
# a different vault item therefore rolls the pod instead of leaving the
# old credentials in the running process. A change INSIDE an item is not
# covered and cannot be, because Helm never sees it - but for the two
# FILE-form secrets it does not need to be: both are re-read on every use
# (see the secrets volume below), so a rotation is live without a
# restart. The exception is admin-password, which is consumed exactly
# once on the first start and which no restart brings back.
checksum/secret-ref: {{ include (print $.Template.BasePath "/bulwark.secret.yaml") . | sha256sum }}
spec:
automountServiceAccountToken: false
securityContext:
{{- include "bulwark.podSecurityContext" $.Values.bulwark.securityContext | nindent 8 }}
containers:
- name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
# Fully qualified. The suffix-free tag family, see Chart.yaml - this
# registry publishes every release twice (1.12.0 and v1.12.0) and the
# chart pins the form without the "v".
image: "ghcr.io/bulwarkmail/webmail:{{ .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
# No command and no args: the image's CMD is ["node", "server.js"] with
# WORKDIR /app, and the working directory is load-bearing here rather
# than incidental - Bulwark resolves its data directories relative to
# process.cwd(), so overriding either would move /app/data out from
# under the volume mount.
env:
# THE MAIL SERVER, AND THE SWITCH THAT KILLS THE SETUP WIZARD. With
# this set, detectSetupState() returns "env-managed" and /setup is
# not served at all, which is what makes this chart declarative.
#
# IT IS STILL BEATEN BY A SERVER URL SAVED IN THE ADMIN DASHBOARD -
# the one precedence trap upstream documents itself. See values.yaml.
- name: JMAP_SERVER_URL
value: {{ $settings.jmapServerUrl | quote }}
{{- with $settings.appName }}
- name: APP_NAME
value: {{ . | quote }}
{{- end }}
- name: STALWART_FEATURES
value: {{ $settings.stalwartFeatures | quote }}
- name: STALWART_ADMIN_ACCESS
value: {{ $settings.stalwartAdminAccess | quote }}
# The settings sync. Note that this flag alone is not enough: without
# a session secret, app/api/config/route.ts turns it off SILENTLY
# (`configManager.get(...) && hasSessionSecret()`). The secret below
# is therefore mandatory rather than recommended.
- name: SETTINGS_SYNC_ENABLED
value: {{ $settings.settingsSync | quote }}
# THE SESSION SECRET, AS A FILE PATH. Everything about why it is the
# file form and why the mount has no subPath is at the volume below.
#
# This is also ONE OF ONLY TWO VALUES IN THIS CHART THAT THE VOLUME
# CANNOT OVERRIDE: getSessionSecret (lib/auth/session-secret.ts) reads
# SESSION_SECRET, then SESSION_SECRET_FILE, and only then the admin
# override - the reverse of configManager.get's normal order.
- name: SESSION_SECRET_FILE
value: {{ include "bulwark.secretMountPath" . }}/session-secret
# THE ADMIN PASSWORD, AS AN ENVIRONMENT VARIABLE, AND THAT IS FORCED
# RATHER THAN CHOSEN: ADMIN_PASSWORD has no _FILE form in
# CONFIG_ENV_MAP, unlike the other two secrets here. The residual
# price, stated rather than hidden: it is in the container's process
# environment, so anyone who can exec into the pod or read its /proc
# can see it until the first start has hashed it into admin.json.
#
# WHY IT IS SET AT ALL: the admin dashboard is the only way back out
# of an OIDC lockout (OAUTH_ONLY below), and without this variable
# the dashboard is simply disabled - there is no random password and
# nothing in the log, measured in lib/admin/password.ts.
- name: ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-app-secret
key: admin-password
{{- if $settings.adminConfigReadOnly }}
# THE CHART IS AUTHORITATIVE FROM HERE ON - and this is the one value
# whose price is bigger than its name. It freezes the settings policy
# (and with it the only way to move Web Push off the vendor relay),
# it freezes an override that ALREADY exists instead of removing it,
# and on a FRESH install it prevents admin.json from being written at
# all - which takes the admin dashboard, i.e. the rescue door, away
# entirely while the pod stays Ready. values.yaml has the measurement
# and the only safe sequence.
- name: ADMIN_CONFIG_READONLY
value: "true"
{{- end }}
{{- if $settings.oauth.enabled }}
- name: OAUTH_ENABLED
value: "true"
# HIDES THE PASSWORD FORM. The way back out is NOT a helm upgrade
# with this false - oauthOnly is read through configManager.get by
# every consumer, so an override in config.json on the PVC wins. The
# repair is removing that override in the admin dashboard. NOTES.txt
# carries it.
- name: OAUTH_ONLY
value: {{ $settings.oauth.only | quote }}
- name: OAUTH_CLIENT_ID
value: {{ $settings.oauth.clientId | quote }}
- name: OAUTH_ISSUER_URL
value: {{ $settings.oauth.issuerUrl | quote }}
# The client secret as a FILE, so it stays out of the process
# environment - the better path garage (#81) and keycloak (#83) did
# not have. Re-read on every token exchange, so a rotation needs no
# restart; see the volume below for the subPath condition that makes
# that true.
- name: OAUTH_CLIENT_SECRET_FILE
value: {{ include "bulwark.secretMountPath" . }}/oauth-client-secret
{{- with $settings.oauth.scopes }}
- name: OAUTH_SCOPES
value: {{ . | quote }}
{{- end }}
- name: OAUTH_END_SESSION
value: {{ $settings.oauth.endSession | quote }}
{{- end }}
# Both OFF against the image's defaults, and deliberately: a mail
# client should not phone home from a private cluster unless somebody
# chose it, and Renovate already watches the image tag, so the
# version check would answer outbound a question already answered
# inbound.
- name: BULWARK_TELEMETRY
value: {{ ternary "on" "off" $settings.telemetry | quote }}
- name: BULWARK_UPDATE_CHECK
value: {{ ternary "on" "off" $settings.updateCheck | quote }}
- name: LOG_LEVEL
value: {{ $settings.logLevel | quote }}
- name: LOG_FORMAT
value: {{ $settings.logFormat | quote }}
# PORT and HOSTNAME are already set in the image (3000 / 0.0.0.0);
# PORT is repeated here only because bulwark.port makes it
# configurable, and the containerPort below has to agree with it.
- name: PORT
value: {{ $settings.port | quote }}
{{- with $.Values.bulwark.env }}
{{- include "bulwark.env" (dict "root" $ "env" .) | nindent 12 }}
{{- end }}
ports:
# The name "http" is what the Service and all three probes reference,
# so renaming it here breaks four things at once.
- name: http
containerPort: {{ $settings.port | int64 }}
protocol: TCP
# Probes, resources and both security contexts go through the null-safe
# helpers (#99): an erased block means "chart defaults apply" instead of
# a vanished probe or an unhardened container. Never wrap these in an
# "if" - a guard hides the erased case completely instead of rendering a
# visible null.
#
# All three are an HTTP GET on /api/health - not HEAD, because a
# Kubernetes probe cannot send one (HTTPGetAction has no method field).
# What that endpoint can and cannot report is argued at the helpers and
# at bulwark.readinessProbe in values.yaml; the short version is that it
# carries the process, HTTP acceptance, a free event loop and Next.js
# routing - and NOTHING about the mail server.
livenessProbe:
{{- include "bulwark.livenessProbe" $.Values.bulwark.livenessProbe | nindent 12 }}
readinessProbe:
{{- include "bulwark.readinessProbe" $.Values.bulwark.readinessProbe | nindent 12 }}
startupProbe:
{{- include "bulwark.startupProbe" $.Values.bulwark.startupProbe | nindent 12 }}
resources:
{{- include "bulwark.effectiveResources" $.Values.bulwark.resources | nindent 12 }}
securityContext:
{{- include "bulwark.containerSecurityContext" $.Values.bulwark.securityContext | nindent 12 }}
volumeMounts:
# THE ONE PVC, carrying all four of Bulwark's directories as
# subdirectories so that a volume snapshot catches configuration,
# runtime state, audit log and user settings atomically.
- name: data
mountPath: {{ include "bulwark.dataDir" . }}
# THE TWO FILE-FORM SECRETS, AS A WHOLE DIRECTORY AND NOT WITH
# subPath. This is the line a later round will want to "tidy up" into
# two subPath mounts, and that would silently break secret rotation:
# the kubelet never refreshes a subPath mount, while both values are
# re-read from disk on every use (getSessionSecret and
# getGlobalClientSecret both call readFileEnv with no cache). With
# this mount a rotated vault field is live without a restart; with
# subPath the container would hold the first value forever and
# nothing would report it.
- name: secrets
mountPath: {{ include "bulwark.secretMountPath" . }}
readOnly: true
# The two writable paths outside the volume, needed because
# readOnlyRootFilesystem is true. Both are CACHES, not state - see
# bulwark.writableCache in values.yaml for how the set was
# established and why neither belongs on the PVC.
- name: tmp
mountPath: /tmp
- name: next-cache
mountPath: /app/.next/cache
restartPolicy: Always
volumes:
- name: data
persistentVolumeClaim:
claimName: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
- name: secrets
# One projected volume over the two Secrets the 1Password operator
# materializes, so both files sit in one directory and the container
# needs one mount instead of two.
#
# defaultMode 0440 - owner and GROUP read, and the group bit is the
# part that must not be "tightened" to 0400. Files in a secret volume
# are owned by ROOT, not by the container's user; what gives uid 1001
# access is fsGroup, which the kubelet applies by chowning the files to
# group 1001. Without a group read bit, uid 1001 could not read either
# file - and the failure would be silent, because readFileEnv answers
# an EACCES with null exactly as it answers a missing file: the
# settings sync would switch itself off and every token exchange would
# send an empty client secret.
#
# (The kubelet ORs 0440 onto the mode of a secret volume once fsGroup
# is set, so 0400 would in practice be corrected to 0440 anyway. It is
# written out because a mode that only works because something else
# silently fixes it is not a mode anyone should have to re-derive.)
#
# #81's permission trap does not apply in the other direction either:
# lib/read-file-env.ts is readFileSync plus try/catch with no mode
# check, so unlike garage, Bulwark does not REJECT a group-readable
# secret file. That is what makes the file form usable here at all.
projected:
defaultMode: 0440
sources:
- secret:
name: {{ .Release.Name }}-{{ .Chart.Name }}-app-secret
items:
# Only the session secret becomes a file. admin-password is
# in the same Secret but arrives as an environment variable
# (it has no _FILE form), so projecting it here as well would
# put it on disk for no reader.
- key: session-secret
path: session-secret
# optional: the 1Password operator fills this Secret in-cluster
# AFTER Helm has applied the manifests, so on a first install
# the Secret does not exist yet. Without this the pod would sit
# in ContainerCreating until the operator caught up, and with a
# misconfigured operator it would sit there forever with the
# reason only in the pod's events. It starts instead, and the
# missing secret surfaces where it is actionable: the settings
# sync stays off and the log says so.
optional: true
{{- if $settings.oauth.enabled }}
- secret:
name: {{ .Release.Name }}-{{ .Chart.Name }}-oauth-secret
items:
- key: oauth-client-secret
path: oauth-client-secret
optional: true
{{- end }}
- name: tmp
emptyDir:
sizeLimit: {{ $settings.writableCache.tmpSizeLimit }}
# Next.js's own cache directory in standalone mode - the optimized-image
# cache and the fetch cache. It is a SUBDIRECTORY of .next, which is what
# makes this possible at all: /app/.next/static holds the build output and
# has to stay readable, and mounting the cache path does not shadow it.
- name: next-cache
emptyDir:
sizeLimit: {{ $settings.writableCache.nextCacheSizeLimit }}
+293
View File
@@ -0,0 +1,293 @@
{
"$schema": "https://json-schema.org/draft-07/schema#",
"title": "bulwark values",
"description": "Strict values schema (issue #68). additionalProperties is false on every chart-owned object level, so a key this chart does not read is rejected at render time instead of being silently ignored; the objects that go verbatim into the Kubernetes pod spec (probes, security contexts, global, free-form annotation maps) stay open, because the chart does not interpret their contents. The blocks the chart dereferences additionally accept null, because an erased block means 'chart defaults apply' (issue #99) rather than a vanished setting. Note what this schema CANNOT enforce: secrets.appSecretRef, bulwark.jmapServerUrl and the two OIDC values all ship as null and JSON Schema's 'required' tests presence, not null-ness, so a 'required' array would never fire - the enforcement is required() in bulwark.secret.yaml and fail() in bulwark.sfs.yaml (issue #18). And note what no schema can express at all for this chart: an admin override in config.json on the PVC beats every value below, silently (values.yaml, point 1).",
"type": "object",
"additionalProperties": false,
"properties": {
"global": {
"description": "Helm's conventional cross-chart values. Unused by this chart, kept open so an umbrella install cannot break on it.",
"type": "object"
},
"scaleDown": {
"description": "true renders the StatefulSet with replicas: 0 and takes precedence over bulwark.replicas. The PVC, Secrets, Service and Ingress stay in place.",
"type": "boolean"
},
"secrets": {
"type": "object",
"additionalProperties": false,
"properties": {
"appSecretRef": {
"description": "op:// item path. The item must carry the fields 'session-secret' and 'admin-password'. Required always. The session secret is mandatory rather than recommended: without it the settings sync is switched off SILENTLY, and the encrypted settings already on the volume become unreadable. The admin password is consumed exactly once, on the first start.",
"type": ["string", "null"]
},
"oauthSecretRef": {
"description": "op:// item path. The item must carry the field 'oauth-client-secret', as issued by the identity provider when the OIDC client was registered. Required while bulwark.oauth.enabled is true.",
"type": ["string", "null"]
}
}
},
"bulwark": {
"type": "object",
"additionalProperties": false,
"properties": {
"replicas": {
"description": "0 or 1. Capped at 1 because of the single ReadWriteOnce PVC, not because of the application: two pods cannot both mount it, so a second replica would sit in ContainerCreating forever. An RWX volume would not help either - nothing coordinates two processes over that directory, and the admin config is read into memory once at start.",
"type": "integer",
"minimum": 0,
"maximum": 1
},
"port": {
"$ref": "#/definitions/port"
},
"jmapServerUrl": {
"description": "The Stalwart JMAP endpoint. Required - and it does two things: it is the mail server's address AND the switch that disables the first-run setup wizard. Left empty, Bulwark starts in bootstrap mode and whatever is clicked through its wizard lands in config.json on the PVC, which then overrides every value here. Enforced by fail() in the StatefulSet, not by this schema (see the title).",
"type": ["string", "null"]
},
"appName": {
"description": "APP_NAME - the product name in the UI and the PWA. null leaves upstream's 'Webmail'.",
"type": ["string", "null"]
},
"stalwartFeatures": {
"description": "STALWART_FEATURES - the Stalwart-specific UI parts (password change, Sieve rules). Does not gate the server-side credential passthrough, which has its own upstream switch this chart does not render.",
"type": "boolean"
},
"stalwartAdminAccess": {
"description": "STALWART_ADMIN_ACCESS - who may open the admin dashboard, which after bulwark.oauth.only is the only way back into a broken login path. 'auto' (upstream's default, kept here) signs a Stalwart admin in without the Bulwark admin password; 'password' requires it; 'off' ignores Stalwart admin status entirely. The consequence of 'auto' is at the value in values.yaml: a Stalwart admin can write the config overrides that disconnect this chart from its own values.",
"type": "string",
"enum": ["auto", "password", "off"]
},
"settingsSync": {
"description": "SETTINGS_SYNC_ENABLED - per-account preferences follow the user instead of living in one browser. true means /app/data/settings on the PVC carries USER DATA, encrypted with the session secret. Without that secret this switch is ignored, silently.",
"type": "boolean"
},
"adminConfigReadOnly": {
"description": "ADMIN_CONFIG_READONLY - refuse every write to the admin config directory at the application layer, which makes this file authoritative and closes the precedence trap for good. FALSE by default because the price is threefold and measured: it also freezes the settings policy (and with it the only way to move Web Push off the vendor relay); it freezes an override that ALREADY exists rather than removing it, because removeAdminOverride is guarded by the same check; and on a FRESH install it prevents admin.json from being written at all, so the admin dashboard - the rescue door - never comes up and its login endpoint answers HTTP 500 while the pod stays Ready. values.yaml has the only safe sequence.",
"type": "boolean"
},
"telemetry": {
"description": "BULWARK_TELEMETRY. false. Upstream's default is already off (telemetry is opt-in), so what this buys is the LOCK: the environment beats the state file and upstream blocks the admin UI from changing it while the override is active.",
"type": "boolean"
},
"updateCheck": {
"description": "BULWARK_UPDATE_CHECK - the hourly version check. Unlike telemetry this is ON unless switched off, so false does change behaviour. Off because Renovate already watches the image tag inbound.",
"type": "boolean"
},
"logLevel": {
"type": "string",
"enum": ["error", "warn", "info", "debug"]
},
"logFormat": {
"description": "text or json. json here, against upstream's default of text, because the logs go to a cluster log collector rather than to a terminal.",
"type": "string",
"enum": ["text", "json"]
},
"oauth": {
"description": "May be null / empty: the chart then falls back to its defaults (OIDC on, password form hidden). That fallback is deliberate and is the safe direction - an erased block must not silently turn the whole sign-in path off on a release whose login is OIDC.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"enabled": {
"description": "OAUTH_ENABLED. Gates the OIDC values, the OAuth OnePasswordItem and the secret file mount.",
"type": "boolean"
},
"only": {
"description": "OAUTH_ONLY - hide the password form, so the identity provider is the only way in for a user. This chart's riskiest default, taken deliberately. The way back out of a lockout is NOT a helm upgrade with this false: oauthOnly is read through configManager.get, so an override in config.json on the PVC wins. The repair is removing that override in the admin dashboard.",
"type": "boolean"
},
"clientId": {
"description": "OAUTH_CLIENT_ID. Required while enabled - enforced by fail() in the StatefulSet.",
"type": ["string", "null"]
},
"issuerUrl": {
"description": "OAUTH_ISSUER_URL, the issuer Bulwark fetches the discovery document from (keycloak: https://<host>/realms/<realm>). Required while enabled - enforced by fail() in the StatefulSet. The endpoints it advertises must share the issuer's origin or be public, or Bulwark's discovery validator rejects them.",
"type": ["string", "null"]
},
"scopes": {
"description": "OAUTH_SCOPES - REPLACES the scope list rather than adding to it. null leaves upstream's default, which is the right starting point: an incomplete list here is a login that succeeds and then has no identity to work with. Use OAUTH_EXTRA_SCOPES through bulwark.env to add.",
"type": ["string", "null"]
},
"endSession": {
"description": "OAUTH_END_SESSION - signing out also ends the provider's session. true (upstream's default): on a shared browser a sign-out that leaves the provider signed in means the next click is straight back in.",
"type": "boolean"
}
}
},
"writableCache": {
"description": "The sizeLimit of the two emptyDirs that readOnlyRootFilesystem: true makes necessary - /tmp and Next.js's own cache at /app/.next/cache. Both are CACHES, not state: losing them on a restart costs one round of image re-optimization. They belong here rather than on the PVC, because a cache on a ReadWriteOnce network volume is slower than regenerating it and would grow the only volume that has to be backed up. These limits and the pod's ephemeral-storage request under bulwark.resources are two halves of one accounting and should move together.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"tmpSizeLimit": {
"type": ["string", "null"]
},
"nextCacheSizeLimit": {
"type": ["string", "null"]
}
}
},
"resources": {
"$ref": "#/definitions/resources"
},
"livenessProbe": {
"$ref": "#/definitions/probe"
},
"readinessProbe": {
"$ref": "#/definitions/probe"
},
"startupProbe": {
"$ref": "#/definitions/probe"
},
"securityContext": {
"$ref": "#/definitions/securityContext"
},
"env": {
"$ref": "#/definitions/env"
}
}
},
"storage": {
"description": "The single PVC at /app/data, carrying all four of Bulwark's directories as subdirectories - one volume, because a snapshot is atomic per volume and never across volumes (the lesson of Gitea #81). Deliberately NOT null-safe: an erased size yields an invalid PVC, which fails loudly at apply time. Issue #99 covers only the silent class, and on an existing release a chart-side fallback could not help anyway, because a PVC cannot shrink.",
"type": "object",
"additionalProperties": false,
"properties": {
"amount": {
"description": "What actually grows here is audit.log, and it grows with admin activity rather than with mail volume - the application does not rotate it.",
"type": ["string", "null"]
},
"storageClass": {
"description": "null uses the cluster's default StorageClass. ReadWriteOnce is enough and is what the single replica needs.",
"type": ["string", "null"]
}
}
},
"ingress": {
"type": "object",
"additionalProperties": false,
"properties": {
"enabled": {
"description": "Gates the whole Ingress template (issue #93). Checked in both directions for this chart: nothing outside that template FUNCTIONALLY reads ingress.domain - Bulwark derives its own public URL from the request origin and takes no base-URL variable - so switching the Ingress off frees domain as well as clusterIssuer. The only other reader is NOTES.txt, which renders a visible placeholder rather than carrying a required() that helm template could never reach.",
"type": "boolean"
},
"clusterIssuer": {
"type": ["string", "null"]
},
"domain": {
"description": "The public hostname. Bulwark is served at the ROOT of it and cannot be served under a path: NEXT_PUBLIC_BASE_PATH is compiled into the bundle, and a rewrite-target annotation does not help because the bundle's own asset URLs are built for the root.",
"type": ["string", "null"]
},
"className": {
"type": ["string", "null"]
},
"annotations": {
"description": "Extra Ingress annotations, merged with the cert-manager annotation the chart always sets. Free-form keys by nature, so this object stays open. The one worth adding for this application is nginx.ingress.kubernetes.io/proxy-body-size: attachment uploads go through this Ingress and nginx's default of 1m rejects anything larger with a 413 that reads like an application fault.",
"type": "object"
}
}
}
},
"definitions": {
"port": {
"type": ["integer", "null"],
"minimum": 1,
"maximum": 65535
},
"resources": {
"description": "May be null / empty: the chart then falls back to its default requests and computed limits (issue #99) instead of rendering a container without resource accounting. One thing specific to this chart: the memory LIMIT is what V8's heap limit is derived from, and /api/health reports 'degraded' once heap usage crosses its threshold - a report the probes cannot see, so a limit set too low produces a pod that is permanently degraded and permanently Ready.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"limitFactor": {
"type": "number",
"exclusiveMinimum": 0
},
"requests": {
"$ref": "#/definitions/resourceList"
},
"limits": {
"$ref": "#/definitions/resourceList"
}
}
},
"resourceList": {
"description": "Both spellings of the ephemeral-storage key are accepted because the resources helper normalizes camelCase to the Kubernetes key.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"cpu": {
"type": ["string", "number", "null"]
},
"memory": {
"type": ["string", "number", "null"]
},
"ephemeral-storage": {
"type": ["string", "number", "null"]
},
"ephemeralStorage": {
"type": ["string", "number", "null"]
}
}
},
"env": {
"description": "Additive container env entries. An entry repeating a name the chart already sets does not replace it - on an existing release the pair collapses to the chart's own value while helm upgrade reports success. And a second caveat specific to this application: an environment variable is only the MIDDLE of three layers, so anything set here is a default that an admin override in config.json on the PVC may already be beating, silently.",
"type": ["array", "null"],
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name"],
"properties": {
"name": {
"type": "string"
},
"value": {
"type": "string"
},
"valueFrom": {
"type": "object",
"additionalProperties": false,
"properties": {
"secretKeyRef": {
"$ref": "#/definitions/keyRef"
},
"configMapKeyRef": {
"$ref": "#/definitions/keyRef"
}
}
}
}
}
},
"keyRef": {
"type": "object",
"additionalProperties": false,
"required": ["name", "key"],
"properties": {
"name": {
"type": "string"
},
"key": {
"type": "string"
}
}
},
"securityContext": {
"description": "May be null / empty at any level: the chart then falls back to its hardening defaults (issue #99). This chart's fallback reproduces ITS defaults, deviations included - uid/gid/fsGroup 1001 rather than this repo's 20000, because that is the user the image creates (addgroup --gid 1001 nodejs / adduser --uid 1001 nextjs) and the owner of /app/data in the image layer. fsGroupChangePolicy is the repo's 'Always' and is correct here rather than merely default: /app/data holds a few JSON files and an append-only log, so the recursive chown is cheap, and it is what makes a freshly provisioned volume writable on the first start.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"pod": {
"description": "Merged onto the chart's pod hardening defaults and rendered as the pod's securityContext; kept open because the chart does not interpret the individual fields.",
"type": ["object", "null"]
},
"container": {
"description": "Merged onto the chart's container hardening defaults and rendered as the container's securityContext; kept open because the chart does not interpret the individual fields.",
"type": ["object", "null"]
}
}
},
"probe": {
"description": "Merged onto the chart's probe defaults and rendered as a container probe; kept open because the chart does not interpret the individual fields. May be null / empty, which means the defaults apply (issue #99) rather than the probe disappearing. All three defaults carry the same CONTEXT: an HTTP GET on /api/health, addressed at the port NAME 'http'. Not HEAD, although Bulwark offers one and it would be cheaper - a Kubernetes HTTPGetAction has no method field and the kubelet always issues a GET, and the only alternative (an exec probe) would mean a Node process every few seconds in an image with no curl or wget. What that proves and what it does not is measured rather than read off the path - it carries the process, HTTP acceptance, a free event loop and Next.js routing, and it carries NOTHING about the reported application state (the handler returns a hard 200 and hard-codes the body to 'healthy' for an unauthenticated caller) and NOTHING about Stalwart (with the mail server gone the probe is green and the pod stays Ready). There is no better endpoint: /api/setup/test-jmap would really test the connection but sits behind the bootstrap setup token, which this chart disables.",
"type": ["object", "null"]
}
}
}
+737
View File
@@ -0,0 +1,737 @@
# Bulwark Webmail - a JMAP webmail client (mail, calendar, contacts, files) for
# a Stalwart mail server. It is a CLIENT: it holds no mail, has no database, no
# cache and no broker. The messages, the accounts and SMTP are stalwart's
# (Gitea #65), reached over the network by bulwark.jmapServerUrl.
#
# READ THIS FIRST. Four properties shape every value below, and not one of them
# is visible from the outside (Gitea #89). Everything here was measured against
# the source at tag 1.12.0, not taken from the documentation - the documentation
# was wrong about two of them.
#
# 1. THE VOLUME BEATS THE CHART, FOR EVERY VALUE, SILENTLY.
# This is the single most important sentence in this file.
# lib/admin/config-manager.ts resolves a config value in this order:
#
# admin override (config.json on the PVC) > environment variable > default
#
# Every setting this chart renders as an environment variable is therefore a
# DEFAULT that the admin dashboard may already be overriding. The override is
# a key in config.json on the PVC, the dashboard may write ANY key of
# CONFIG_ENV_MAP (app/api/admin/config/route.ts validates against exactly
# that key list), and nothing in the dashboard says that it has just
# disconnected the chart. A `helm upgrade` with a changed value then changes
# nothing, reports success, and leaves no trace of why.
#
# The documentation names this only for the server URL ("A server URL saved
# by the wizard takes priority over the environment variable"). It is not
# special to the server URL. It is the behaviour of every key.
#
# THERE ARE EXACTLY THREE EXCEPTIONS, all verified, and they are the only
# values in this chart that the volume cannot silently beat:
# - SESSION_SECRET / SESSION_SECRET_FILE: getSessionSecret()
# (lib/auth/session-secret.ts) reads the environment FIRST and the admin
# override last - the reverse of the normal order.
# - ADMIN_PASSWORD: initAdminPassword() reads the environment only. (It is
# still consumed exactly once - see secrets.appSecretRef.)
# - BULWARK_TELEMETRY: effectiveConsent() takes the environment over the
# state file, and upstream blocks the admin UI from changing it while the
# override is set.
# The OAuth client secret is NOT an exception, which is the one most likely to
# be assumed - see secrets.oauthSecretRef.
#
# The way out, if you want the chart to be authoritative, is
# bulwark.adminConfigReadOnly - and it has a price that is bigger than it
# looks. Read it there before setting it.
#
# 2. THERE IS NO ConfigMap IN THIS CHART, AND THAT IS NOT AN OMISSION.
# Bulwark is configured entirely through environment variables plus the two
# JSON files it writes itself onto the PVC (config.json, policy.json). No
# config file is rendered, mounted or hashed here, so the repo's
# checksum/config rule (#82) has nothing to attach to: there is no ConfigMap
# whose change could fail to restart the workload. What IS covered is the
# op:// PATHS of the vault items, through checksum/secret-ref on the pod
# template, so repointing a secrets.* value really does roll the pod. A
# change INSIDE an item is not covered and cannot be - Helm never sees it -
# and for the two FILE-form secrets it does not need to be: both are re-read
# on every use, so a rotation takes effect without a restart (point 3).
#
# 3. THE TWO FILE-FORM SECRETS ARE MOUNTED WITHOUT subPath, ON PURPOSE.
# SESSION_SECRET_FILE and OAUTH_CLIENT_SECRET_FILE both exist (upstream calls
# them the file form of SESSION_SECRET and OAUTH_CLIENT_SECRET), and taking
# them keeps both values out of the container's process environment, which is
# what garage (#81) and keycloak (#83) could not do because neither
# application offered a file form at all. Here the file form is also
# ROTATABLE: readFileEnv() is called on every use with no cache, so a changed
# field in the vault item takes effect without a pod restart - but only
# because the Secret is mounted as a whole directory. The kubelet never
# refreshes a subPath mount, so a subPath here would silently freeze both
# secrets at their first value while everything else looked correct.
# ADMIN_PASSWORD has NO file form and stays an environment variable.
#
# 4. THREE OPTIONS ARE BAKED IN AT BUILD TIME AND THIS CHART CANNOT MOVE THEM.
# Upstream: "These are compiled into the bundle and cannot be changed at
# runtime ... To change build-time variables with the published Docker image,
# rebuild with matching --build-arg." Building our own image is outside what
# this repository does - the same limit as garage (#81) and keycloak (#83).
# - NEXT_PUBLIC_BASE_PATH: so Bulwark is served at the ROOT of a hostname.
# There is no sub-path deployment and no value here that could make one.
# - NEXT_PUBLIC_DEFAULT_LOCALE: the fallback UI language stays "en". A
# visitor with a matching Accept-Language gets their own language;
# everybody else gets English.
# - NEXT_PUBLIC_PUSH_RELAY_URL: see bulwark.pushRelay below. This one is
# NOT the disclosure it looks like.
# Scale the whole chart down without uninstalling it: scaleDown: true renders
# the StatefulSet with replicas: 0 and takes precedence over bulwark.replicas.
# The PVC, the Secrets, the Service and the Ingress stay in place;
# scaleDown: false brings the workload back up.
scaleDown: false
secrets:
# op:// path of the 1Password item carrying THIS RELEASE'S OWN secrets.
# REQUIRED - the chart does not render without it.
#
# Must carry the fields:
# session-secret -> mounted as a file, read via SESSION_SECRET_FILE
# admin-password -> ADMIN_PASSWORD (environment variable, no file form)
#
# WHY session-secret IS MANDATORY AND NOT OPTIONAL. Two things hang on it, and
# one of them fails silently without it: persistent "remember me" sessions,
# and the encryption of the synchronized user settings. app/api/config/route.ts
# computes `settingsSyncEnabled: configManager.get(...) && hasSessionSecret()`
# - so with bulwark.settingsSync true and no secret, synchronization is
# switched OFF with no error anywhere. Generate it with
# openssl rand -base64 32
#
# AND THE BACKUP STATEMENT, WHICH IS TWO-PART AND IS THE SHARPEST THING ON
# THIS PAGE. It belongs wherever this release's backups are described:
#
# (a) A BACKUP OF THE VOLUME ALONE IS NOT A BACKUP. The synchronized
# settings on the PVC are encrypted with session-secret. Lose the vault
# item and they are unreadable out of a perfectly intact volume. The
# secret has to be in the backup just as much as the volume is.
# (b) A BACKUP OF THE VOLUME MAY CONTAIN A SECRET IN CLEARTEXT. If anybody
# has ever used the admin dashboard's OAuth setup,
# app/api/admin/oauth/setup/route.ts has written oauthClientSecret as
# PLAINTEXT into config.json on the PVC. So the volume backup is not
# merely useless without the vault - it may itself need the protection
# the vault gets.
#
# WHY admin-password IS SET AT ALL, and the reason is NOT the one the issue
# originally gave. Measured: without ADMIN_PASSWORD, Bulwark does NOT generate
# a random password and writes NOTHING to stdout - lib/admin/password.ts
# simply logs "Admin dashboard disabled (no ADMIN_PASSWORD set)" and the
# dashboard stays off. (The only random value in the project is the bootstrap
# setup token, which is written into the state directory and never logged.) So
# there is no leak into the pod log here and the analogy to stalwart's
# STALWART_RECOVERY_ADMIN (#65) does not hold.
#
# The reason it is set is point 1 at the top of this file combined with
# bulwark.oauth.only: THE ADMIN DASHBOARD IS THE ONLY WAY BACK OUT OF AN
# OIDC LOCKOUT. Switching it off means taking away the door.
#
# THE FIELD IS CONSUMED EXACTLY ONCE, which is the part that surprises people
# later: initAdminPassword() returns as soon as admin.json exists on the PVC
# and never looks at the variable again; on the first start it hashes it with
# scrypt and persists the hash. ROTATING THE VAULT FIELD AFTERWARDS HAS NO
# EFFECT. A reset goes through the dashboard itself, or by deleting admin.json
# from the PVC. Same cause as point 1: the volume beats the chart.
appSecretRef: null
# op:// path of the 1Password item carrying the OIDC CLIENT SECRET.
# Required while bulwark.oauth.enabled is true (the default).
#
# Must carry the field:
# oauth-client-secret -> mounted as a file, read via OAUTH_CLIENT_SECRET_FILE
#
# A SEPARATE ITEM FROM appSecretRef, BY AUDIENCE: this value is not ours, it
# is created by the identity provider when the client is registered (keycloak,
# #83), it is rotated there, and it is the one of the three that another
# system also knows. Two items mean it can be repointed and rotated without
# touching this release's own secrets. Same reasoning as garage's split
# between its RPC secret and its S3 key.
#
# ROTATION NEEDS NO RESTART - with one condition the chart satisfies.
# getGlobalClientSecret() (lib/oauth/token-exchange.ts) calls readFileEnv on
# EVERY token exchange with no cache, so a new value in the vault is live as
# soon as the operator has written it. The condition is the whole-directory
# mount of point 3; a subPath mount would never be refreshed.
#
# AND THE ONE WAY A ROTATION STILL DOES NOTHING, which is point 1 again and is
# why this is not filed under "rotation just works": getGlobalClientSecret
# checks the ADMIN OVERRIDE FIRST. If the admin dashboard's OAuth setup has
# ever run, config.json on the PVC carries oauthClientSecret in plaintext and
# that value wins over this file - forever, and with no error. The repair is
# the same as for every other override: remove it in the dashboard (see
# bulwark.oauth.only).
#
# #81 DOES NOT APPLY HERE, and it was checked rather than assumed, because it
# is exactly what forced garage onto an environment variable: lib/read-file-env.ts
# is readFileSync plus try/catch and nothing else - no statSync, no
# `mode & 0o077` permission check. The group-read bit the kubelet ORs onto
# every file of a secret volume once fsGroup is set is therefore harmless, and
# the file form is load-bearing rather than merely available.
oauthSecretRef: null
bulwark:
# ONE, and the schema caps it there. The reason is this chart's single
# ReadWriteOnce PVC, not anything about the application: two pods cannot both
# mount it, so a second replica would sit in ContainerCreating forever. An RWX
# volume would not help either - nothing in Bulwark coordinates two processes
# over that directory, and the admin config is read into memory once at start
# (configManager.load() in instrumentation.node.ts), so two pods would hold
# two divergent views of it.
#
# An explicit 0 is allowed and is the per-workload way to stop the release;
# the chart-global scaleDown flag does the same for everything at once.
replicas: 1
# THE JMAP SERVER - the stalwart release this client talks to. REQUIRED, and
# it does two things at once, which is why it has no default.
#
# (a) it is the address of the mail server. Without it there is no mail.
# (b) IT SWITCHES OFF THE FIRST-RUN SETUP WIZARD. detectSetupState()
# (lib/setup/state.ts) returns "env-managed" as soon as this variable is
# non-empty, and in that state /setup is not served at all. That is what
# makes this chart declarative instead of "install, then click through a
# wizard whose result lands on a volume".
#
# An in-cluster stalwart release is reached at its http Service, e.g.
# http://<stalwart-release>-stalwart-<group>-http.<namespace>.svc.cluster.local:8080
# A published one is reached at its public https URL. Either works - this is
# server-to-server, the browser never sees it.
#
# THE TRAP, AND IT IS THE ONE THE UPSTREAM DOCUMENTATION ACTUALLY WARNS ABOUT:
# a server URL saved in the admin dashboard WINS over this value (point 1 at
# the top). If the dashboard has ever been used to set it, changing this value
# and running `helm upgrade` moves nothing, succeeds, and says nothing. Read
# the effective value back off the running app rather than off the chart:
# kubectl -n <ns> exec <pod> -- cat /app/data/admin/config.json
# An absent jmapServerUrl key in that file means this value is in force.
jmapServerUrl: null
# The product name in the UI and the PWA. null leaves upstream's "Webmail".
appName: null
# STALWART_FEATURES - the Stalwart-specific parts of the UI (password change,
# Sieve rule management). True, which is upstream's default and is the point
# of pairing this chart with stalwart at all.
#
# Note what it does NOT gate: the server-side credential-bearing passthrough
# at /api/account/stalwart/jmap has its own switch upstream
# (STALWART_JMAP_PASSTHROUGH_ENABLED, default true). This chart does not render
# it; an installation that wants the UI features but not the passthrough sets
# it through bulwark.env and should read the precedence note on that value
# first.
stalwartFeatures: true
# WHO MAY OPEN THE ADMIN DASHBOARD, and this value matters more here than its
# name suggests, because after bulwark.oauth.only the dashboard is the ONLY
# door back into a broken login path.
#
# Upstream's three modes (STALWART_ADMIN_ACCESS):
# auto a Stalwart admin sees the shield and is signed into /admin
# WITHOUT the Bulwark admin password. Upstream's default and the
# legacy behaviour.
# password a Stalwart admin sees the shield but must enter the Bulwark
# admin password like everybody else.
# off Stalwart admin status is ignored; /admin is reachable only via
# /admin/login with the Bulwark admin password.
#
# "auto", i.e. upstream's default, is kept DELIBERATELY and the consequence is
# stated rather than left to be discovered: with it, anybody who is an admin
# on the stalwart side can write config overrides here - and an override is
# exactly the thing that can disconnect this chart from its own values
# (point 1) or lock the login path (bulwark.oauth.only). That is usually the
# same person, which is why it is not changed; if it is not, "password"
# narrows the door to whoever holds the vault item, and "off" also removes the
# shield from the UI. Both require admin-password to be configured - which
# this chart always does - or the dashboard would become unreachable.
stalwartAdminAccess: auto
# SETTINGS_SYNC_ENABLED - per-account user preferences follow the user between
# browsers instead of living in one browser's local storage.
#
# ON. The consequence is that /app/data/settings on the PVC carries USER DATA,
# encrypted with the session secret - which is where the two-part backup
# statement at secrets.appSecretRef comes from. It also means the secret is
# mandatory rather than recommended: without it this switch is ignored,
# silently.
settingsSync: true
# ADMIN_CONFIG_READONLY - refuse every write to the admin CONFIG directory at
# the application layer, which makes the values in this file authoritative and
# closes the precedence trap of point 1 for good: with no override able to
# appear, bulwark.oauth.only (and every other value here) always takes effect.
#
# OFF BY DEFAULT, AND THE PRICE IS WHY. It is not one cost but three, and the
# third was measured rather than inferred - it is bigger than the plan for
# this chart assumed:
#
# (a) POLICY, THEMES AND BRANDING FREEZE TOO. assertWritable
# (lib/admin/paths.ts) guards setPolicy, markSetupComplete, the plugin
# and theme registries and the branding upload - not just the config
# override. So the settings policy becomes unchangeable, and with it the
# ONLY way to point Web Push at your own relay (see bulwark.pushRelay).
# (b) AN OVERRIDE THAT ALREADY EXISTS IS FROZEN IN PLACE, not removed.
# removeAdminOverride is guarded by the same assertWritable. Turning this
# on while config.json already carries `oauthOnly: true` therefore makes
# that lockout permanent and takes away the documented repair at the same
# time. If you are turning this on, read config.json first.
# (c) ON A FRESH INSTALL IT TAKES THE ADMIN DASHBOARD AWAY ENTIRELY - i.e.
# exactly the door this chart keeps a password for. initAdminPassword
# persists the hashed ADMIN_PASSWORD by calling writeConfigData, which
# begins with assertWritable('save admin password'). With no admin.json
# on the volume yet, that throws: the startup hook logs "Admin dashboard
# init skipped", the retry inside /api/admin/auth throws again, and the
# login endpoint answers HTTP 500. The pod is Ready and serving mail the
# whole time, so nothing reports it.
#
# THE ONLY SAFE SEQUENCE, which is also upstream's own framing ("Operators set
# ADMIN_CONFIG_READONLY=true after running the setup wizard"):
# 1. install with this false, let the pod start once, so admin.json exists
# 2. set every policy you want (push relay included) - after step 3 you
# cannot
# 3. check that config.json carries no override you still need removed
# 4. then helm upgrade with this true
# The chart cannot verify step 1 for you: it has no way to know what is on the
# volume. NOTES.txt repeats the sequence.
adminConfigReadOnly: false
oauth:
# OIDC SIGN-IN against the identity provider (keycloak, #83). ON.
enabled: true
# OAUTH_ONLY - hide the password form entirely, so the identity provider is
# the only way for a USER to sign in. ON, and this is the chart's riskiest
# default, taken deliberately with the way back out written down.
#
# THE RISK: if the provider is down or the OIDC configuration is wrong,
# nobody reaches their mail, and there is no password form to fall back to.
#
# THE WAY BACK, AND IT IS NOT WHAT IT LOOKS LIKE. `helm upgrade` with this
# set to false is NOT reliably the repair, and believing it was would be the
# trap of point 1 sitting inside the rescue instruction itself: oauthOnly is
# an ordinary key of CONFIG_ENV_MAP, every consumer reads it through
# configManager.get (app/api/auth/session/route.ts,
# app/api/auth/verify/route.ts, app/api/config/route.ts - none of them reads
# process.env), and the admin dashboard's OAuth setup writes
# `oauthOnly: true` into config.json when it is used. Once that override is
# on the PVC, this value is ignored and the upgrade reports success.
#
# WHAT REALLY WORKS, and it does not depend on the identity provider:
# OPEN THE ADMIN DASHBOARD (its own password, from the vault item) AND
# REMOVE THE OVERRIDE - the dashboard's revert, i.e. DELETE on
# /api/admin/config, which calls configManager.removeAdminOverride.
# After that the chart's value applies again. If the dashboard itself is
# unreachable, the remaining repair is editing or deleting config.json on
# the PVC directly. Both are in NOTES.txt.
#
# AND THE PRE-INSTALL STEP THIS DEFAULT MAKES MANDATORY: the OIDC client
# must be registered with the provider BEFORE the first install, not after.
# With oauth.only true and no working client, the very first sign-in is
# impossible. NOTES.txt prints the redirect URI to register.
only: true
# OAUTH_CLIENT_ID. Required while oauth.enabled is true.
clientId: null
# OAUTH_ISSUER_URL - the issuer, from which Bulwark fetches the discovery
# document. Required while oauth.enabled is true. For keycloak (#83) this is
# https://<keycloak-host>/realms/<realm>
#
# It must be reachable BY THE POD and the endpoints it advertises must share
# the issuer's origin or be public: getDiscoveryValidator
# (lib/oauth/token-exchange.ts) rejects a discovered endpoint on a private
# address from a different origin unless OAUTH_ALLOW_PRIVATE_ENDPOINTS is
# set. The chart does not render that escape hatch - a split-horizon setup
# that needs it can add it through bulwark.env.
issuerUrl: null
# OAUTH_SCOPES - replaces the scope list Bulwark would otherwise request.
# null leaves upstream's default, which is the right starting point; an
# incomplete list here is a login that succeeds and then has no identity to
# work with.
scopes: null
# OAUTH_END_SESSION - signing out of Bulwark also ends the provider's
# session through its end_session_endpoint. True, upstream's default: on a
# shared browser, a sign-out that leaves the provider signed in means the
# next click is straight back in. Set it false where the provider is shared
# with applications that should stay signed in.
endSession: true
# WEB PUSH RELAY - read this before writing it down as a disclosure, because
# the obvious reading of the build-time variable is wrong and was corrected
# against the source.
#
# NEXT_PUBLIC_PUSH_RELAY_URL is baked into the bundle (point 4 at the top), so
# this chart cannot move the BUILT-IN entry, which is upstream's hosted relay
# at https://notifications.relay.bulwarkmail.org. It would be wrong to stop
# there and declare Web Push "not changeable with the published image": that
# variable only sets the FALLBACK. lib/push-relays.ts resolves the effective
# relay as `resolveDefaultRelayUrl(policy)`, which takes policy.pushRelayUrl
# whenever it is a valid URL and uses the built-in one only otherwise; with
# policy.pushRelayUrlLocked an operator can additionally pin every user to it.
#
# So the correct, two-part statement, and both halves matter:
# - WITHOUT further action, Web Push for this release goes through the
# VENDOR'S relay. For a mail client that is a fact worth knowing, and it
# stays in the users' relay list either way, because the list is built
# from the bundle.
# - A SELF-HOSTED RELAY CAN BE MADE THE EFFECTIVE DEFAULT AND MANDATORY -
# but through policy.json, i.e. the ADMIN DASHBOARD, not through a chart
# value. There is deliberately no value here: pushRelayUrl lives in
# SettingsPolicy and has no entry in CONFIG_ENV_MAP, so no environment
# variable reaches it and a chart value would be a switch wired to
# nothing.
#
# AND THE INTERACTION TO KEEP IN MIND: bulwark.adminConfigReadOnly freezes
# setPolicy, so turning that on before setting the relay makes this change
# impossible afterwards. Order matters - see the sequence at that value.
#
# THERE IS DELIBERATELY NO VALUE UNDER THIS COMMENT. A `pushRelayUrl:` key
# here would render nothing and change nothing, which is the exact defect
# guideline 12 exists to remove - a key that looks like configuration and does
# nothing. The knowledge belongs in the file an operator reads; the switch
# belongs where it actually is, in the dashboard.
# BULWARK_TELEMETRY - upstream's anonymous telemetry.
#
# FALSE, AND THAT IS NOT MERELY RESTATING THE IMAGE'S DEFAULT, which is the
# reason this value exists at all. Measured in lib/telemetry/state.ts: consent
# defaults to 'off' and telemetry really is opt-in (the comment in
# instrumentation.node.ts calling it "on by default" is wrong - DEFAULTS carries
# `consent: 'off'`). What rendering the variable buys is the LOCK:
# effectiveConsent() takes the environment over the state file, and upstream
# blocks the admin UI from changing it while the override is active ("UI changes
# are blocked when env override is active"). So this is one of the very few
# settings in this chart where the chart beats the volume rather than the other
# way round - see the exception list in point 1 at the top of this file.
#
# Setting it true opts in; /app/data/telemetry on the PVC then holds the
# consent record and the instance id, which is why that directory is inside the
# single volume rather than an emptyDir.
telemetry: false
# BULWARK_UPDATE_CHECK - an hourly version check against
# version.telemetry.bulwarkmail.org.
#
# FALSE, AND HERE THE DEFAULT REALLY IS THE OTHER WAY: unlike telemetry, this
# check is ON unless switched off (lib/version-check/state.ts, disabledByEnv()
# recognizes only off/false/0/no - an absent variable means enabled). So this
# value does change behaviour.
#
# Off for two reasons: a mail client should not phone home from a private
# cluster unless somebody chose it, and Renovate already watches the image tag
# (renovate.json5) - so the check would answer outbound a question that is
# already answered inbound, and answer it to a third party.
updateCheck: false
# LOG_LEVEL - error, warn, info or debug.
logLevel: info
# LOG_FORMAT - text or json. JSON here, against upstream's default of text,
# because the logs go to a cluster log collector rather than to a terminal.
logFormat: json
# The HTTP port. 3000, which is what the image binds (ENV PORT=3000,
# EXPOSE 3000) and what HOSTNAME=0.0.0.0 makes reachable. Both the Service and
# all three probes address this port BY NAME ("http"), so changing the number
# here is safe and renaming the port is not.
port: 3000
resources:
# Limits are optional: memory and ephemeral-storage limits default to
# limitFactor x the request, an explicitly set limit always wins, and a CPU
# limit is only set when configured explicitly - CPU is compressible, so
# leaving it unlimited throttles instead of OOM-killing.
#
# 256Mi x 3 = 768Mi. A Next.js standalone server serving a SPA and proxying
# JMAP holds no mail and caches no mailbox, so the footprint is the Node
# heap plus the image-optimization cache - not a function of how much mail
# the users have.
#
# THE MEMORY LIMIT IS LOAD-BEARING IN ONE DIRECTION, though: /api/health
# reports `degraded` once V8 heap usage crosses its warning threshold, and
# the heap limit is derived from the container limit. That report is not
# visible to the probes (see bulwark.readinessProbe), so a limit set too low
# produces a pod that is permanently degraded, permanently Ready, and says
# so only to an authenticated caller asking for details.
#
# The ephemeral-storage request covers the two emptyDirs (/tmp and the
# Next.js cache) plus container logs. It does NOT cover /app/data, which is
# the PVC - see storage.amount.
limitFactor: 3
requests:
cpu: 100m
memory: 256Mi
ephemeral-storage: 64Mi
# ALL THREE PROBES ARE AN HTTP GET ON /api/health, and the limit of that is
# written at the readiness probe below rather than left for somebody to assume.
#
# A NOTE ON GET VERSUS HEAD, so nobody "fixes" this back. Bulwark offers a HEAD
# handler on this path and it would be the cheaper call - but a KUBERNETES PROBE
# CANNOT SEND HEAD. The API type is HTTPGetAction, it has no method field, and
# the kubelet always issues a GET. The only way to send a HEAD is an exec probe
# calling an HTTP client, and this image has neither curl nor wget (npm and npx
# are removed from the runtime layer), so it would mean starting a Node process
# every few seconds instead of one cheap HTTP request. The GET handler does
# gather diagnostics, which makes it marginally MORE capable than HEAD, not
# less: it can answer 503 when the diagnostics themselves throw.
#
# LIVENESS: the process is alive, accepting HTTP, its event loop is free and
# Next.js can route. An OOM kill, a crash and a wedged event loop all fail it.
livenessProbe:
httpGet:
path: /api/health
port: http
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# READINESS: the same endpoint, and HERE IS THE LIMIT, stated at the value
# because a probe path is not a promise (README guideline 12.3; #218 and
# Gitea #83 are the two times this repo paid for assuming otherwise).
#
# WHAT THIS PROBE CARRIES: the Node process is alive, the HTTP server accepts,
# the event loop is not blocked, Next.js routes. That is real and it is not
# nothing.
#
# WHAT IT DOES NOT CARRY, measured in app/api/health/route.ts at tag 1.12.0:
# - THE REPORTED STATE. The success path returns a hard `status: 200`
# regardless of what it computed, and for an unauthenticated caller the
# body is hard-coded to "healthy" as well
# (`status: detailed ? status : 'healthy'`); "degraded" appears only with
# ?detailed=true AND a caller that passes canSeeInstanceDetails. Upstream
# tests precisely this - lib/__tests__/health-route.test.ts expects HTTP
# 200 while payload.status is "degraded". A probe is unauthenticated and
# asks for no details, so it sees 200 and "healthy" always. The only 503
# this endpoint can produce comes from its catch block, i.e. when
# process.memoryUsage() or v8.getHeapStatistics() itself throws.
# - ANYTHING ABOUT STALWART. Bulwark holds no mail and this endpoint asks the
# JMAP server nothing. WITH THE MAIL SERVER COMPLETELY GONE, THIS PROBE IS
# GREEN AND THE POD STAYS Ready. Monitor the stalwart release for that;
# nothing here can.
#
# AND WHY IT IS STILL THIS ENDPOINT: there is no better one.
# /api/setup/test-jmap really would test the connection, but it lives behind
# the bootstrap setup token and this chart disables the wizard, so it is
# unreachable. Pointing readiness at "/" instead would only add Next.js
# rendering to the same set of guarantees, and would start failing on an
# unauthenticated redirect rather than on anything about the mail path.
readinessProbe:
httpGet:
path: /api/health
port: http
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# STARTUP: gates the other two, which is why neither carries an
# initialDelaySeconds.
#
# BUDGET = periodSeconds x failureThreshold = 5 x 24 = 120s. What that has to
# cover is not the Next.js server, which starts in about a second: it is
# attaching and mounting the network volume, and the instrumentation hook that
# runs before the first request is served (instrumentation.node.ts migrates
# the legacy admin layout, loads config.json and policy.json off the volume,
# and on a FIRST install hashes ADMIN_PASSWORD with scrypt and writes
# admin.json - the scrypt hash being the one deliberately slow step).
#
# NOT MEASURED AGAINST A RUNNING POD AT THE TIME OF WRITING, and said so
# rather than presented as calibrated: the budget is derived from what the
# start does, with head room, and the cost of head room is nothing because a
# pod that IS healthy becomes Ready on the first probe either way.
startupProbe:
httpGet:
path: /api/health
port: http
periodSeconds: 5
timeoutSeconds: 5
failureThreshold: 24
securityContext:
pod:
runAsNonRoot: true
# uid/gid 1001, NOT this repo's 20000, and that is the image's own user
# rather than a free choice: the Dockerfile creates
# addgroup --system --gid 1001 nodejs
# adduser --system --uid 1001 nextjs
# and copies the application tree plus /app/data with --chown=nextjs:nodejs.
# 1001 therefore OWNS everything the process reads, and the four data
# subdirectories are chowned to it in the image. Read off the Dockerfile at
# tag 1.12.0, not guessed.
#
# fsGroupChangePolicy: Always, the repo standard, and here it is the right
# one rather than the default one: /app/data holds a handful of JSON files,
# an audit log and a small settings tree, so the recursive chown is
# cheap - unlike garage's block store or satisfactory's game files, where
# it is the reason both deviate to OnRootMismatch. It also matters on the
# FIRST start, because a freshly provisioned volume arrives owned by root
# and Bulwark has to create its four subdirectories in it.
runAsUser: 1001
runAsGroup: 1001
fsGroup: 1001
fsGroupChangePolicy: Always
seccompProfile:
type: RuntimeDefault
container:
allowPrivilegeEscalation: false
# TRUE, with two emptyDirs carrying the writes that are not state. Which
# paths those are, and how the set was established, is at
# bulwark.writableCache below - it is the one part of this chart that was
# derived from the source and then PROVEN by the install-test rather than
# the other way round.
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
# EMPTY, and it can stay empty: the process binds 3000, which is above
# 1024, and the entrypoint is `node server.js` with no file
# capabilities - so there is no execve-time capability check like
# stalwart's.
add: []
# THE WRITABLE PATHS OUTSIDE THE VOLUME, as emptyDirs, because
# readOnlyRootFilesystem is true and Next.js writes in two places that are not
# application state.
#
# HOW THIS SET WAS ESTABLISHED, and the honest order of events: it was DERIVED
# from next.config.ts at tag 1.12.0 - `output: "standalone"`, and
# `images: { unoptimized: true }` applies only in the isLite branch, so the
# published image has image optimization ACTIVE and writes its cache under
# /app/.next/cache - and it was then PROVEN by the install-test, which starts
# the real image with this exact hardening and requires the pod to become
# Ready. A missing writable path here is not a subtle defect: it is an EACCES
# or EROFS at the first write.
#
# /tmp the conventional scratch path. Node, Next.js and the
# image-optimization pipeline all use it.
# /app/.next/cache Next.js's own cache directory in standalone mode: the
# optimized-image cache and the fetch cache. It is a
# SUBDIRECTORY of .next, which is why it can be an
# emptyDir at all - /app/.next/static holds the build
# output and must stay readable, and mounting the cache
# path does not shadow it.
#
# Both are CACHES and neither is state: losing them on a restart costs one
# round of image re-optimization. Do not "simplify" either one onto the PVC -
# a cache on a ReadWriteOnce network volume is slower than regenerating it,
# and it would grow the only volume that has to be backed up.
writableCache:
# Size limits on both, so a runaway cache fills an emptyDir instead of the
# node's disk. These are the sizeLimit of each emptyDir; the pod's
# ephemeral-storage request/limit (bulwark.resources) is the other half of
# the same accounting and the two should move together.
tmpSizeLimit: 256Mi
nextCacheSizeLimit: 512Mi
# Additive environment entries for the Bulwark container, rendered through tpl
# and supporting the value, secretKeyRef and configMapKeyRef forms.
#
# ADDITIVE MEANS ADDITIVE: an entry repeating a name the chart already sets
# does not replace it. At install time both reach the pod and the later one
# wins; on an EXISTING release the pair collapses to the chart's own value,
# because Kubernetes merges env by name - and helm upgrade still reports
# success. So do not use this to override JMAP_SERVER_URL, OAUTH_ONLY or
# SESSION_SECRET_FILE; change the values that produce them.
#
# AND THE SECOND, SHARPER CAVEAT, SPECIFIC TO THIS APPLICATION: an environment
# variable is only the MIDDLE of three layers (point 1 at the top of this
# file). Anything set here is a default that an admin override in config.json
# on the PVC may already be beating, silently. An entry here is therefore not
# a guarantee - unless bulwark.adminConfigReadOnly is on, and that has its own
# price.
#
# Genuinely useful here, all of them CONFIG_ENV_MAP keys this chart does not
# render: APP_DESCRIPTION, LOGIN_COMPANY_NAME and the other LOGIN_* branding
# variables, COOKIE_SAME_SITE, STALWART_JMAP_PASSTHROUGH_ENABLED,
# OAUTH_EXTRA_SCOPES, WOPI_CLIENT_URL.
env: []
storage:
# THE ONE PVC, mounted at /app/data, and "one" is the decision rather than the
# obvious shape - Bulwark keeps FOUR directories under it and they could each
# have their own volume:
#
# /app/data/admin config.json, policy.json, admin.json (the password
# hash), plugins, themes, branding uploads
# /app/data/admin-state admin-state.json (login timestamps), audit.log, the
# bootstrap setup token
# /app/data/settings the encrypted per-account user settings
# /app/data/telemetry consent and instance id
#
# ONE VOLUME, BECAUSE A SNAPSHOT IS ATOMIC PER VOLUME AND NEVER ACROSS THEM -
# the lesson garage (#81) was built on. Split over four, a backup could
# capture an audit log that records logins the configuration does not know
# about yet, or settings encrypted against a secret the config has since been
# repointed from. One volume keeps configuration, runtime state, audit trail
# and user settings consistent with each other, and there is no performance
# argument on the other side: this is a few hundred kilobytes of JSON plus an
# append-only log.
#
# Bulwark creates all four subdirectories itself on start (ensureConfigDir /
# ensureStateDir, both mkdir -p), which it has to, because the image's own
# mkdir ran in a layer that this mount covers.
#
# THE PVC IS DELIBERATELY NOT null-SAFE: an erased size yields an invalid PVC
# that is rejected at apply time, which is loud. A chart-side fallback would
# silently size somebody's volume - and on an existing release it could not
# help anyway, because a PVC cannot shrink. Same reasoning as garage's storage
# block and satisfactory's documented exception (Gitea #39).
#
# IT IS ALSO NEVER RENAMED: a renamed PVC makes Kubernetes provision a new,
# empty volume and orphans the data - including admin.json, which is the only
# copy of the admin password hash, and the audit log, which by its nature
# cannot be reconstructed.
#
# 1Gi is generous for the JSON files and the settings tree; what actually
# grows is audit.log, and it grows with admin activity rather than with mail
# volume. There is no rotation for it in the application, so this is the value
# to watch on a long-lived release.
amount: 1Gi
# null uses the cluster's default StorageClass. ReadWriteOnce is enough and is
# what the single replica needs; see bulwark.replicas for why more than one is
# not on offer.
storageClass: null
ingress:
# ON by default: a webmail client nobody can reach is not a webmail client.
enabled: true
# The public hostname. REQUIRED, AND REQUIRED EVEN WITH ingress.enabled:
# false - this is guideline 12.1, checked in both directions for this chart.
# The value is read a SECOND time outside the Ingress template: NOTES.txt
# derives the OIDC redirect URI from it, and that URI is what has to be
# registered with the identity provider before the first install. Gating it
# away would produce a release that renders cleanly and whose operator is told
# to register `https:///auth/callback`.
#
# BULWARK IS SERVED AT THE ROOT OF THIS HOST AND CANNOT BE SERVED UNDER A
# PATH. NEXT_PUBLIC_BASE_PATH is compiled into the bundle (point 4 at the top
# of this file), so a sub-path deployment would need a rebuilt image. The
# Ingress rule is therefore "/" and there is no value to change it - a
# rewrite-target annotation would not help either, because the bundle's own
# asset URLs are built for the root.
domain: null
# cert-manager cluster issuer. Required while the Ingress is enabled.
clusterIssuer: null
className: nginx
# Extra annotations, merged with the cert-manager annotation the chart always
# sets.
#
# ONE WORTH KNOWING ABOUT FOR THIS APPLICATION: file attachments and the Files
# view upload through this Ingress, and nginx's default
# proxy-body-size (1m) will reject anything larger with a 413 that looks like
# an application fault. Set
# nginx.ingress.kubernetes.io/proxy-body-size: "25m"
# (or whatever the mail server accepts) to match stalwart's own limit. The
# chart does not set it, because the right number is the mail server's and
# this chart does not know it.
annotations: {}
+69
View File
@@ -0,0 +1,69 @@
# Fixtures for the bulwark install-test: the two Secrets the 1Password operator
# would otherwise materialize from the OnePasswordItems the chart renders
# (release name: ci, stub CRD in ci/common/), plus a name for the JMAP endpoint
# to resolve to.
#
# NO DATABASE, NO CACHE, NO BROKER AND NO MAIL SERVER - and that is a property of
# the chart rather than a shortcut taken here. Bulwark is a client: it holds no
# mail and contacts the JMAP server per request, not at boot, so it starts and
# becomes Ready with nothing behind that URL. The flip side is the limitation the
# chart documents at bulwark.readinessProbe, and this file is the clearest
# illustration of it: everything needed to make this chart green is in these few
# objects, and none of it is a mail server.
---
apiVersion: v1
kind: Secret
metadata:
name: ci-bulwark-app-secret
stringData:
# The field NAMES are what matter here, not the values: the operator
# materializes one Secret key per item field, and the chart consumes these two
# in DIFFERENT ways - session-secret as a projected FILE, admin-password
# through a secretKeyRef.
#
# Only the second of those is actually asserted by the run. The projected
# volume is `optional: true` by design (so a first install does not hang
# waiting for the 1Password operator), so a misspelled session-secret yields a
# missing file and a pod that starts fine. A misspelled admin-password leaves
# the container in CreateContainerConfigError and fails the run.
#
# A real base64 32-byte value rather than a placeholder word: Bulwark derives
# encryption keys from it, and a value of the wrong shape would be the kind of
# thing that works in CI and not in production.
session-secret: Y2ktb25seS1zZXNzaW9uLXNlY3JldC1ub3QtYS1yZWFsLW9uZQ==
# Visibly throwaway. It is hashed with scrypt into admin.json on the first
# start and the cluster is deleted with the job; the only thing it protects is
# a dashboard that exists for a few minutes.
admin-password: ci-bulwark-admin-password
---
apiVersion: v1
kind: Secret
metadata:
name: ci-bulwark-oauth-secret
stringData:
# The OIDC client secret. Nothing in CI performs a token exchange - no identity
# provider runs here - so this exists to exercise the SECOND source of the
# chart's projected volume and the OAUTH_CLIENT_SECRET_FILE path, not the OAuth
# flow.
oauth-client-secret: ci-bulwark-oauth-client-secret
---
# The JMAP endpoint's name. A Service with NO SELECTOR and therefore no
# endpoints: the in-cluster DNS name resolves, and a connection to it is refused.
#
# DELIBERATELY NOT A DEPLOYMENT. ci/run-install-test.sh waits for every fixture
# Deployment and StatefulSet to roll out BEFORE it installs the chart, so a stub
# web server here would be one more workload able to fail a run that is not about
# it - and it would buy nothing, because a refused connection and a served 404
# are indistinguishable to a chart that never contacts this address at boot.
#
# It exists at all so the URL in test-values.yaml is a real name rather than an
# invented hostname, which keeps the fixture honest about what the chart does
# with it.
apiVersion: v1
kind: Service
metadata:
name: jmap-stub
spec:
ports:
- port: 8080
targetPort: 8080
+162
View File
@@ -0,0 +1,162 @@
# CI values for the bulwark install-test.
#
# WHY THIS CHART IS TESTED RATHER THAN EXCLUDED, and what a green run is worth.
# Bulwark needs no database, no cache and no broker, and it does not need a
# reachable JMAP server to START - the connection is made per request, not at
# boot - so a dummy URL is enough to bring the pod up. That makes this one of
# the cheapest install-tests in the repo and the only place where the two
# riskiest assumptions of the chart are exercised at all. Neither is observable
# in a render.
#
# WHAT A GREEN RUN PROVES:
#
# 1. readOnlyRootFilesystem: true HOLDS, WITH EXACTLY THE TWO emptyDirs THE
# CHART RENDERS - /tmp and /app/.next/cache. This is the measurement point
# the plan for Gitea #89 left open, and it is the reason this fixture
# exists: the set of writable paths was DERIVED from next.config.ts
# (output: "standalone", and images.unoptimized applies only in the isLite
# branch, so image optimization is active and writes under
# /app/.next/cache) and could not be measured without running the image.
# A missing writable path is not subtle - it is an EROFS or EACCES on the
# first write - so a pod that reaches Ready and stays Ready is the proof.
# A SUPERFLUOUS emptyDir would NOT be caught here; that half stays open and
# is named in the chart's wiki page.
# 2. THE PROBES CAN REALLY BE SATISFIED. All three are an HTTP GET on
# /api/health, which has to exist in the built bundle and answer 200
# unauthenticated. If it did not, the startup probe would exhaust its budget
# and the rollout would fail - and no render could have told us, because a
# path only exists in the built bundle.
# 3. The uid/gid 1001 read off the Dockerfile is right, and fsGroup makes a
# freshly provisioned volume writable for it: Bulwark creates its four
# subdirectories under /app/data itself on start, and the image's own mkdir
# ran in a layer this mount covers. Wrong uid, or no fsGroup, and the
# instrumentation hook fails to write admin.json.
# 4. ADMIN_PASSWORD is really consumed on the first start - the hook hashes it
# with scrypt and writes /app/data/admin/admin.json. That write is inside
# the startup budget, so a budget that was too short would fail here.
# Plus the mechanical half: the projected volume assembles, the Service port
# name matches the containerPort, the Ingress is accepted with its extra
# annotation alongside the cert-manager one, the resource quantities are valid,
# and the schema accepts a realistic non-default values file.
#
# ONE CLAIM THIS FIXTURE DELIBERATELY DOES NOT MAKE, because it would be
# false: it does not prove that the two FILE-form secret keys are spelled
# right. The projected volume marks both sources `optional: true` on purpose -
# so that a first install does not hang in ContainerCreating while the
# 1Password operator catches up - and the consequence is that a misspelled
# session-secret or oauth-client-secret key yields a MISSING FILE and a pod
# that starts perfectly. Only `admin-password` is proven, because it arrives
# through a non-optional secretKeyRef and a wrong name there leaves the
# container in CreateContainerConfigError.
#
# WHAT IT DOES NOT PROVE, and the first two matter most:
# - ANYTHING ABOUT JMAP. The URL below points at a Service that answers
# nothing useful. Bulwark comes up regardless, which is precisely the
# property the readiness probe cannot see (chart values.yaml,
# bulwark.readinessProbe) - so this test demonstrates the limitation rather
# than closing it. A green run here says nothing about mail working.
# - ANYTHING ABOUT OIDC. No identity provider runs in CI, so the sign-in path
# is never walked. What IS checked is that the chart's render guards fire on
# a missing client id or issuer (unit-testable at render time) and that the
# secret file is mounted where OAUTH_CLIENT_SECRET_FILE points.
# - THE PRECEDENCE TRAP. config.json never exists in this run, so the
# "volume beats the chart" behaviour - the chart's single most important
# operating caveat - is not exercised. It cannot be: it needs an admin
# writing an override, which needs a browser.
# - anything about an upgrade, or about a volume that already carries data.
# This installs once, from nothing.
secrets:
# Materialized by ci/bulwark/fixtures.yaml as ci-bulwark-app-secret and
# ci-bulwark-oauth-secret (the OnePasswordItem stub CRD comes from ci/common/).
appSecretRef: op://ci/bulwark-app
oauthSecretRef: op://ci/bulwark-oauth
bulwark:
# A dummy JMAP endpoint. It does NOT have to answer - Bulwark connects per
# request - but it does have to be SET, because an empty value puts the app
# into bootstrap mode and serves the setup wizard instead. So this value also
# exercises the path that keeps the chart declarative.
#
# It points at the selector-less Service in fixtures.yaml: a name that RESOLVES
# and then refuses the connection. That is deliberately not a Deployment -
# fixtures with a workload have to roll out before the chart is installed, so a
# stub serving nothing would be one more thing able to fail a run that is not
# about it. A refused connection and a served 404 are indistinguishable to this
# chart, because nothing here contacts the JMAP server at boot.
jmapServerUrl: http://jmap-stub.default.svc.cluster.local:8080
appName: Bulwark CI
# OIDC LEFT ON, i.e. the chart's default, although no provider answers in CI.
# That is deliberate: it is what makes the run exercise the OAuth
# OnePasswordItem, the second source of the projected volume and the
# OAUTH_CLIENT_SECRET_FILE path. Bulwark does not contact the issuer at boot,
# so an unreachable one costs nothing here - it would only fail a sign-in, and
# nothing signs in.
oauth:
enabled: true
only: true
clientId: bulwark-ci
issuerUrl: https://keycloak.ci.example.com/realms/ci
# ON, i.e. the chart's default, and it is what makes /app/data/settings get
# created and the session secret actually read: with the sync off, a missing or
# misnamed session-secret field would pass unnoticed.
settingsSync: true
# OFF - the chart's default, spelled out because it is load-bearing for what
# this test means. With it ON, a fresh install cannot write admin.json and the
# admin dashboard never comes up; the pod would still be Ready, so the run
# would be green while the chart's documented rescue door was missing. Leaving
# it off is what makes step 4 of "what a green run proves" an assertion.
adminConfigReadOnly: false
# Both off, as in the chart - and here with a second reason: a CI runner should
# not send a telemetry heartbeat or an update check to a vendor endpoint on
# every pull request.
telemetry: false
updateCheck: false
# debug, so that a failed run has something to read. The install-test dumps
# container logs on failure, and Bulwark's info level says very little about
# why a start went wrong.
logLevel: debug
# text rather than the chart's json: a human reads these logs out of a failed
# job, not a collector.
logFormat: text
resources:
# Small, to stay inside the runner's budget, while still exercising the limit
# derivation: 192Mi x limitFactor 3 = 576Mi. Not smaller than this on
# purpose - the memory LIMIT is what V8's heap limit is derived from, and a
# Next.js server with too little heap fails in a way that looks like an
# application fault.
requests:
cpu: 100m
memory: 192Mi
ephemeral-storage: 64Mi
storage:
amount: 256Mi
# k3d's dynamic provisioner. The intended production value does not exist on
# the runner - and a pre-bound hostPath PV would be worse than this one,
# because the kubelet does not apply fsGroup to a hostPath volume at all, so
# the test would silently stop exercising the ownership path that matters most
# here: Bulwark has to CREATE four subdirectories in a volume it did not make.
storageClass: local-path
ingress:
# Left ON although no ingress controller answers in CI: the point is that the
# object is accepted and that the Service port it names BY NAME ("http")
# actually exists - a typo there is otherwise invisible until someone opens the
# URL.
enabled: true
domain: bulwark.ci.example.com
clusterIssuer: ci-selfsigned
annotations:
# Exercises the annotation merge: this has to arrive ALONGSIDE the
# cert-manager annotation the chart always sets, not instead of it. It is
# also the annotation this chart's NOTES.txt tells operators to add, so the
# fixture is the shape a real install has.
nginx.ingress.kubernetes.io/proxy-body-size: "25m"
-17
View File
@@ -1,17 +0,0 @@
# Fixtures for the convertx install-test: the Secret the 1Password operator
# would otherwise materialize from the OnePasswordItem (release name: ci).
#
# Without it the pod stays in CreateContainerConfigError - the StatefulSet
# reads JWT_SECRET from this Secret's "jwt-secret" key, and the stub
# OnePasswordItem CRD in ci/common/ has no controller behind it. The key name
# here is therefore load-bearing and has to match the chart's secretKeyRef.
#
# ConvertX needs no external database, cache or storage service: its SQLite
# file, uploads and output all live on the chart's own PVC, so this is the
# whole fixture set.
apiVersion: v1
kind: Secret
metadata:
name: ci-convertx-secret
stringData:
jwt-secret: convertx-ci-jwt-secret-0123456789abcdef0123456789abcdef
-23
View File
@@ -1,23 +0,0 @@
# The image is ~1.4 GiB compressed - by a wide margin the largest in this
# repo, because LibreOffice, Calibre, ImageMagick, ffmpeg, Inkscape and a TeX
# Live installation are all inside it. One layer alone is 1354 MiB.
#
# The rollout wait therefore has to cover the PULL, not just the start: the
# runner reference point is 96 MB in 17.5s, which puts this image in the
# region of five minutes of pulling before the container is even created. On
# top of that comes the startup probe budget (30 x 10s = 300s), which is
# generous because ConvertX shells out to ~20 converter binaries at import
# just to log their versions.
#
# MEASURED, and the estimate was wrong in the safe direction: the first green
# install-test of this chart (Gitea PR #61) took 2m35s for the WHOLE job -
# cluster creation, fixtures, pull, install and rollout wait together. The
# 96 MB/17.5s reference does not transfer to ghcr.io on this runner.
#
# 900s stays anyway, and deliberately. A rollout-wait timeout is a ceiling,
# not a cost: kubectl waits exactly as long as the rollout needs, so a
# generous number is free on a good day and only spends time on a chart that
# is already failing. What it buys is the bad day - a cold registry, or a node
# that has to extract the 1354 MiB layer onto slow disk. The default of 300s
# would have no headroom at all for either.
INSTALL_TIMEOUT=900
-15
View File
@@ -1,15 +0,0 @@
# The OnePasswordItem CR only needs a non-null item path; the actual
# Kubernetes Secret comes from ci/convertx/fixtures.yaml.
secrets:
jwtSecretRef: op://ci/convertx-jwt
ingress:
domain: convertx.ci.local
# clusterIssuer is strictly required (no default); the Ingress just has to
# apply - there is no cert-manager in the CI cluster.
clusterIssuer: ci
# The chart's default is 10Gi; the install-test converts nothing, so 1Gi is
# enough and provisions faster on k3d's local-path class.
storage:
amount: 1Gi
+45
View File
@@ -0,0 +1,45 @@
# Fixtures for the garage install-test: the two Secrets the 1Password operator
# would otherwise materialize from the OnePasswordItems the chart renders
# (release name: ci, stub CRD in ci/common/).
#
# NO THROWAWAY INFRASTRUCTURE: Garage depends on nothing - no database, no
# cache, no other service. The whole deployment is one process and two volumes,
# which is also why this chart is cheap enough to test on every PR.
#
# THE FIELD NAMES MATTER AS MUCH AS THE VALUES, and here the VALUES matter too,
# which is unusual for a fixture: Garage validates all four of them on start and
# refuses to run on a malformed one. So these are not placeholders - a value
# that merely looks right (a 31-byte RPC secret, a 6-character access key) fails
# the run, which is exactly the check that makes the fixture worth having.
apiVersion: v1
kind: Secret
metadata:
name: ci-garage-rpc-secret
stringData:
# EXACTLY 32 bytes, hex-encoded = 64 hex characters. Garage decodes this and
# rejects anything else with "Invalid RPC secret key: expected 32 bytes of
# random hex" - it is the network key of the RPC layer, required even for a
# single node.
#
# A throwaway value, and a visibly throwaway one: this is not a secret, it is
# a test vector. Nothing here survives the run, the cluster is deleted with
# the job, and the node it keys has no peers.
rpc-secret: "c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1"
---
apiVersion: v1
kind: Secret
metadata:
name: ci-garage-s3-secret
stringData:
# The three fields Garage IMPORTS on start with --default-access-key and
# --default-bucket. Each one is validated (src/model/key_table.rs,
# src/model/bucket_alias_table.rs):
# access-key at least 8 characters, ASCII alphanumeric plus - _ .
# secret-key at least 16 characters, printable ASCII
# bucket a valid S3 bucket name, 3-63 characters, lowercase
# Shortening any of them below its minimum is the quickest way to check that
# this test really exercises the import path: the container exits instead of
# starting.
access-key: "ciGarageTestKey01"
secret-key: "c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2"
bucket: "ci-garage-test"
+103
View File
@@ -0,0 +1,103 @@
# CI values for the garage install-test.
#
# WHY THIS CHART IS TESTED RATHER THAN EXCLUDED, and what the test is worth:
# with garage.singleNode (the chart default) the server creates its own cluster
# layout before it opens any listener, so the pod reaches Ready without an
# operator. That is what makes an install-test possible here at all - without
# the flag the readiness probe would honestly never go green (no layout, no
# quorum, /health answers 503) and a green run would have required a probe that
# lies.
#
# WHAT A GREEN RUN PROVES - the four measurement points the plan for Gitea #81
# left open, three of which this closes on every PR:
#
# 1. Garage accepts metadata_dir and data_dir as two SUBDIRECTORIES OF ONE
# PVC and creates them itself. The image is FROM scratch and has no mkdir,
# so if the assumption were wrong the pod could not start at all.
# 2. readOnlyRootFilesystem: true holds with no emptyDir anywhere - the
# container writes only into the two volumes.
# 3. The layout is really applied in-process and the admin API answers 200 on
# /health afterwards: Ready is exactly that HTTP 200, so the rollout
# status IS the assertion.
# Plus the mechanical half: the Secret key names match what the StatefulSet
# references, the rendered garage.toml PARSES AS GARAGE READS IT (it opens
# LMDB, validates replication_factor against --single-node and refuses
# malformed keys), the access key and the bucket are imported from the vault
# fields, the resource quantities are valid, and the schema accepts a
# realistic non-default values file.
#
# WHAT IT DOES NOT PROVE, and no install-test here could:
# - that the metadata auto-snapshot really lands in metadata_snapshots_dir.
# The first snapshot is taken after interval/2, and the shortest interval
# that is not absurd is still minutes - longer than the run. A snapshot
# interval of seconds would test the timer, not the thing.
# - anything about Longhorn: the snapshot semantics the chart's whole storage
# layout is built on are a property of the storage layer, and the runner
# has local-path.
# - durability. metadata_fsync is set, and whether an fsync survives a power
# cut is not observable in a container.
secrets:
# Materialized by ci/garage/fixtures.yaml as ci-garage-rpc-secret and
# ci-garage-s3-secret (the OnePasswordItem stub CRD comes from ci/common/).
rpcSecretRef: op://ci/garage-rpc
s3SecretRef: op://ci/garage-s3
garage:
# Both ON, i.e. the chart's defaults, and they are the reason the run means
# something: the key and the bucket are imported FROM the fixture Secret, so
# a wrong field name or a malformed value fails the start rather than passing
# unnoticed.
singleNode: true
defaultAccessKey: true
defaultBucket: true
# ON against the chart's default of off, together with a root domain. It
# renders a second listener, a second containerPort, a Service port and the
# [s3_web] section of garage.toml - all of which Garage has to accept, and
# none of which the default install exercises.
web:
enabled: true
rootDomain: ".web.garage.ci.example.com"
s3:
region: garage-ci
rootDomain: ".s3.garage.ci.example.com"
# Shortened from the chart's "6h" so the snapshot WORKER is created and
# scheduled during the run. It still does not take a snapshot - the first one
# is due after interval/2, i.e. 15 minutes here - and that gap is named above
# rather than papered over with a one-second interval that would only prove
# the timer fires.
metadataAutoSnapshotInterval: "30m"
resources:
# Small, to stay inside the runner's budget, while still exercising the
# limit derivation: 256Mi x limitFactor 3 = 768Mi.
requests:
cpu: 100m
memory: 256Mi
ephemeral-storage: 32Mi
storage:
data:
amount: 512Mi
# k3d's dynamic provisioner. The intended production value,
# longhorn-encrypted-ns, does not exist on the runner - and a pre-bound
# hostPath PV would be worse than this one, because the kubelet does not
# apply fsGroup to a hostPath volume at all, so the test would silently
# stop exercising the ownership path that matters for a volume the
# container did not create.
storageClass: local-path
snapshots:
amount: 256Mi
storageClass: local-path
ingress:
# Left ON although no ingress controller answers in CI: the point is that the
# object is accepted and that the Service port it names by NAME ("s3")
# actually exists - a typo there is otherwise invisible until someone opens
# the URL.
enabled: true
domain: garage.ci.example.com
clusterIssuer: ci-selfsigned
+118
View File
@@ -0,0 +1,118 @@
# Fixtures for the keycloak install-test: a throwaway PostgreSQL plus the two
# Secrets the 1Password operator would otherwise materialize from the
# OnePasswordItems the chart renders (release name: ci, stub CRD in ci/common/).
#
# WHY A REAL DATABASE AND NOT A STUB: Keycloak does nothing without one. It
# creates its entire schema through Liquibase on first start, and - the reason
# this fixture is the valuable half of the test - jdbc-ping discovers the second
# replica THROUGH this database. A stub would turn a cluster test into a
# templating test.
#
# The image is pinned to a major line like every other fixture here
# (postgres:16-alpine); ci/** is excluded from Renovate on purpose, because
# this is test scaffolding and churning it only risks breaking the
# install-test.
apiVersion: v1
kind: Secret
metadata:
name: ci-keycloak-db-secret
stringData:
# The field NAMES are what matters here, not the values: the operator
# materializes one Secret key per item field, and the Deployment references
# these two by name. A rename on either side leaves the pod in
# CreateContainerConfigError, which is a state no render can show.
#
# They must agree with the POSTGRES_USER / POSTGRES_PASSWORD below, because
# Keycloak really connects.
username: keycloak
password: ci-keycloak-db-password
---
apiVersion: v1
kind: Secret
metadata:
name: ci-keycloak-bootstrap-admin-secret
stringData:
# The temporary bootstrap administrator. Keycloak creates it while the master
# realm is created, so these values are exercised on the very first start -
# and the variables are named KC_BOOTSTRAP_ADMIN_* only since Keycloak 26.
#
# A visibly throwaway password: nothing here survives the run, the cluster is
# deleted with the job, and the only thing it protects is a realm that exists
# for a few minutes.
username: ci-temp-admin
password: ci-bootstrap-admin-password
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
labels:
app: postgres
spec:
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: public.ecr.aws/docker/library/postgres:16-alpine
env:
- name: POSTGRES_USER
value: keycloak
- name: POSTGRES_PASSWORD
value: ci-keycloak-db-password
- name: POSTGRES_DB
value: keycloak
ports:
- containerPort: 5432
readinessProbe:
exec:
command: ["pg_isready", "-U", "keycloak"]
periodSeconds: 5
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
volumes:
# emptyDir: the database exists for the length of one job. Nothing here
# is meant to survive, and a PVC would only add a provisioner to the
# list of things that can fail a chart's test.
- name: data
emptyDir: {}
---
apiVersion: v1
kind: ConfigMap
metadata:
# A MINIMAL BUT REAL LOGIN THEME, so that the install-test actually walks the
# path keycloak.extraVolumes exists for (Gitea #88) instead of only rendering
# it. ci/keycloak/test-values.yaml mounts it on the SUBPATH
# /opt/keycloak/themes/ci-theme - never on /opt/keycloak/themes itself, which
# would hide the image's base, keycloak and keycloak.v2 themes.
name: ci-keycloak-theme
data:
# A theme's layout is <theme>/<type>/theme.properties, and a ConfigMap key
# cannot contain a slash - so the "login/" directory is produced by
# items[].path in the volume, which can. That indirection is exactly the part
# of the theme route a render cannot verify and a running pod can.
theme.properties: |
parent=keycloak.v2
# The reason the issue was opened: Keycloak has no logo field on an identity
# provider, so the icon on its login button comes only from a login theme.
# Nothing in CI drives a browser, so this line is carried rather than
# checked - what the run proves is that the theme reaches the container.
kcLogoIdP-ci-provider = fa fa-lock
---
apiVersion: v1
kind: Service
metadata:
name: postgres
spec:
selector:
app: postgres
ports:
- port: 5432
targetPort: 5432
+17
View File
@@ -0,0 +1,17 @@
# The default 300s is not enough here, and the three parts that fill the budget
# are worth naming so the number can be lowered again if one of them changes:
#
# 1. TWO INIT CONTAINERS PER POD. The build one runs a full Quarkus
# augmentation - upstream's own issue reports measure "Quarkus
# augmentation completed in" at roughly 13-23 seconds, and a CI runner is
# not a fast machine.
# 2. THE FIRST START CREATES THE WHOLE SCHEMA through Liquibase, and with two
# replicas the second pod waits on the first's migration lock. So the two
# pods do not become Ready in parallel.
# 3. The rollout is only complete when BOTH pods are Ready, which is
# deliberate (it is what makes cluster formation part of the assertion)
# and which means the slow parts add up rather than overlap.
#
# The chart's own startupProbe budget is 600s (10s x 60); this timeout has to
# be at least that, or the test gives up before the probe would have.
INSTALL_TIMEOUT=900
+222
View File
@@ -0,0 +1,222 @@
# CI values for the keycloak install-test.
#
# WHY THIS CHART IS TESTED RATHER THAN EXCLUDED, and what a green run is worth.
# Keycloak has no volume, needs one throwaway PostgreSQL and reaches Ready on
# its own, so an exclusion would be hard to justify. More importantly, this is
# the only place where the four riskiest assumptions of the chart are exercised
# at all - none of them is observable in a render.
#
# WHAT A GREEN RUN PROVES, point by point (the measurement points of Gitea #83):
#
# 1. THE TWO INIT CONTAINERS WORK AT ALL. The seed container copies the
# image's own augmentation output into the volume, the build container
# mounts that volume at /opt/keycloak/lib/quarkus and runs `kc.sh build`
# into it - both with readOnlyRootFilesystem: true. If the seeding
# assumption were wrong (QuarkusEntryPoint loads
# quarkus/quarkus-application.dat before it dispatches any command,
# `build` included), the build container could not start, and the pod
# would sit in Init with the reason only in its log.
# 2. readOnlyRootFilesystem: true HOLDS FOR THE SERVING CONTAINER, with
# emptyDirs only at /opt/keycloak/lib/quarkus, /opt/keycloak/data and
# /tmp - and NOT at /opt/keycloak/conf, which would hide the
# cache-ispn.xml that cache=ispn reads.
# 3. `start --optimized` ACCEPTS THE PRE-BUILT INSTALLATION and does not
# re-augment. A disagreement between the build options and the serving
# container would show up here as a crash loop rather than as a slow
# start, because the path it would want to rewrite is read-only.
# 4. TWO REPLICAS BOTH COME UP AND STAY UP against one database - the
# rollout is complete only when BOTH pods are Ready, so a second replica
# that crashed, deadlocked on the schema migration lock or fought the
# first over the database would fail the run. Measured in Gitea PR #87:
# 71.8s from `helm install` to both Ready, 0 restarts - three runs, all
# three 72s and 0 restarts.
# THAT WAS LUCK, OR A FAST WEEK, AND NOT AN INVARIANT - which took an A/B
# inside Gitea PR #92 to establish, because the first five runs there made
# it look like the new theme mount had destabilized the start:
# WITH the theme mount (spec of #88)
# run 585: 101s, ONE restart on one pod
# run 590: 82s, 0
# run 593: 191s, 0
# run 595: 101s, ONE
# run 597: 111s, ONE -> 3 of 5
# WITHOUT it (the same branch, the pair commented out; pod template
# hash 5844db56b6, i.e. BYTE-IDENTICAL to the spec PR #87 measured)
# run 598: 81s, 0
# run 599: 171s, ONE -> 1 of 2
# So the restart happens WITHOUT the mount as well, and #88 is not its
# cause. It is pre-existing, intermittent behaviour of THIS chart's first
# install: always exactly one restart on one pod, 35-80s before that pod
# reports Ready. It does not track the runner's speed either - the fastest
# (81s) and the slowest (191s) run of the whole series restarted nothing.
# WHAT THE RESTART IS HAS NOT BEEN ESTABLISHED, and that is the finding
# worth keeping: the harness dumps container logs only when a run FAILS
# (diagnose() hangs off the EXIT trap behind "RC -ne 0") and all seven runs
# PASSED - A GREEN RUN CANNOT EXPLAIN A RESTART IT REPORTS. Answering it
# means making the final-state step print the restart reasons whenever a
# counter is above zero; that is a change to ci/run-install-test.sh with a
# CI-CD wiki obligation of its own, and it was deliberately NOT made inside
# #88. A first install that serializes two replicas on the Liquibase
# migration lock is a plausible place for it - stated as the hypothesis it
# is, because nothing here measured it.
# THE METHOD IS THE PART TO REUSE: a counter that moved in the same round
# as a change is not evidence against the change until the control case has
# run on the SAME machine. Three clean runs two days earlier were not that
# control, and treating them as one would have cost this PR its conclusion
# in both directions.
# 5. A MOUNTED LOGIN THEME SURVIVES ALL OF THAT (Gitea #88) - the full
# argument sits at keycloak.extraVolumes further down, including what it
# still does not prove.
# Plus the mechanical half: the Secret key names match what the Deployment
# references, /health/started, /health/live and /health/ready all answer on
# the management port (so health-enabled really was baked in by the init
# container), the resource quantities are valid, and the schema accepts a
# realistic non-default values file.
#
# WHAT IT DOES NOT PROVE, and the first entry is the one not to misread:
# - THAT jdbc-ping ACTUALLY FOUND THE PEER. Two replicas both reaching Ready
# is NOT that proof, and believing it was is the mistake this paragraph
# exists to prevent: readiness reports this node's own health, so a node
# that never found its peer is Ready, healthy and alone - and two such
# nodes are two islands that each believe they are the cluster. The
# Infinispan view has to be read out of the log, which is what step 2 of
# NOTES.txt tells the operator to do. So measurement point 4 of the plan
# stays open after this test; what the test does rule out is the louder
# half (a second replica that cannot start or cannot share the database).
# - that hostname-admin serves the console cleanly on a second hostname. No
# ingress controller answers in CI and nothing drives a browser, so
# measurement point 5 stays open too. What IS checked is that both Ingress
# objects are accepted, that the Service port they name exists, and that
# Keycloak accepts both hostname values as full URLs at start.
# - anything about an upgrade. The rolling-update restriction between minor
# versions is a property of two versions meeting, and this test installs
# one.
# - that the PostgreSQL here resembles production. It is one pod with an
# emptyDir, so nothing about durability, connection limits or latency
# transfers.
secrets:
# Materialized by ci/keycloak/fixtures.yaml as ci-keycloak-db-secret and
# ci-keycloak-bootstrap-admin-secret (the OnePasswordItem stub CRD comes from
# ci/common/).
databaseSecretRef: op://ci/keycloak-db
bootstrapAdminSecretRef: op://ci/keycloak-bootstrap-admin
keycloak:
# TWO, i.e. the chart's default, and the whole reason this fixture is not a
# one-replica shortcut. With one replica the install-test would be green
# without ever showing whether the second node is found - and "a path the
# test does not walk is a path we have not proven" is the lesson Gitea #80
# paid for. The install-test script waits for the Deployment's rollout, and a
# rollout is only complete when BOTH pods are Ready, so cluster formation is
# inside the assertion rather than next to it.
replicas: 2
db:
# The throwaway PostgreSQL from fixtures.yaml. Plain Service name: both are
# in the same namespace.
host: postgres
database: keycloak
# Small but not degenerate: 2 replicas x 5 = 10 connections against
# postgres:16-alpine's default max_connections of 100, which leaves the
# fixture comfortable while still exercising the pool options.
pool:
initialSize: 1
minSize: 1
maxSize: 5
# The chart default, spelled out because it is load-bearing for what this
# test means: metrics-enabled is what puts the DATABASE check into
# /health/ready (upstream's check table, "Requires Metrics: Yes"). With it
# off, readiness would pass without ever touching the database, and a green
# run would say nothing about the connection at all.
#
# It is also a BUILD option, so with it true the init container has to build
# an installation differing from the published image in three options rather
# than two - which is what makes a build that silently did nothing fail the
# run instead of looking identical.
metrics:
enabled: true
# Set on purpose, although one release cannot collide with itself: it
# exercises the KC_CACHE_EMBEDDED_CLUSTER_NAME path, and a name Keycloak
# rejected would fail the start rather than pass unnoticed.
cache:
clusterName: ci-keycloak
resources:
# Below the chart's defaults to stay inside the runner's budget, and the
# memory number is chosen rather than rounded: the image sizes the heap at
# 70% of the LIMIT, so 600Mi x limitFactor 3 = 1800Mi gives a ~1260Mi heap.
# Upstream's floor is "at least 750 MB" for the limit; going much below
# this makes the JVM the reason a run fails.
requests:
cpu: 250m
memory: 600Mi
ephemeral-storage: 256Mi
build:
resources:
requests:
cpu: 250m
memory: 512Mi
ephemeral-storage: 256Mi
# THE EXTRA VOLUME PAIR, WALKED RATHER THAN RENDERED (Gitea #88). The theme
# ConfigMap comes from fixtures.yaml. Three things this puts inside the
# assertion that no render reaches:
#
# 1. A MOUNT ON A SUBPATH OF A DIRECTORY THAT EXISTS IN THE IMAGE IS
# ACCEPTED ALONGSIDE readOnlyRootFilesystem: true.
# /opt/keycloak/themes/ci-theme does not exist in the image, so the
# runtime has to create the mount point inside a root filesystem that is
# about to be read-only. If that did not work, every container of this
# pod would fail to start.
# 2. `kc.sh build` STILL SUCCEEDS WITH THE EXTRA MOUNT PRESENT. The mounts
# are rendered on the build init container too (that is what makes SPI
# providers work at all), so this fixture is also the test that the extra
# mount does not disturb the augmentation - a failure would leave the pod
# in Init with the reason only in a log.
# 3. `start --optimized` COMES UP WITH A RUNTIME THEME IN PLACE. The plan of
# #88 called that "a reasoned expectation, not verified"; a Ready pod with
# this mounted is the verification, and it is the one that matters for the
# unusual two-init-container construction of this chart.
#
# WHAT IT STILL DOES NOT PROVE, and it would be easy to overclaim here: that
# the LOGIN PAGE renders and that base/keycloak/keycloak.v2 are still listed.
# The health endpoints answer regardless of the themes directory, and nothing
# in this harness makes an HTTP request to / or runs a command in the pod. That
# the subpath hides nothing is a property of the mount path rather than a
# measurement - and the kcLogoIdP line needs a browser plus a realm setting,
# neither of which exists in CI.
extraVolumes:
- name: login-theme
configMap:
name: ci-keycloak-theme
items:
- key: theme.properties
path: login/theme.properties
extraVolumeMounts:
- name: login-theme
# THE SUBPATH, AND THE WHOLE POINT OF THE EXAMPLE: a volume on
# /opt/keycloak/themes would hide the image's own three themes and break
# the login page while the pod stayed Ready.
mountPath: /opt/keycloak/themes/ci-theme
readOnly: true
ingress:
# Left ON although no ingress controller answers in CI: the point is that
# both objects are accepted and that the Service port they name BY NAME
# ("http") actually exists - a typo there is otherwise invisible until
# somebody opens the URL.
enabled: true
domain: keycloak.ci.example.com
clusterIssuer: ci-selfsigned
admin:
# The second hostname, on as well, so the chart renders KC_HOSTNAME_ADMIN
# and Keycloak has to accept it as a full URL. That is the half of
# measurement point 5 a cluster-less test can reach: Keycloak validates
# both hostname values at start ("verified that full URL is used, incl.
# scheme and hostname"), so a chart that built a bare host here would fail
# the start instead of serving a broken console later.
enabled: true
domain: keycloak-admin.ci.example.com
+43 -42
View File
@@ -1,11 +1,19 @@
# Fixtures for the paperless-ngx install-test: throwaway postgres + redis,
# the secrets the 1Password operator would otherwise materialize (release
# name: ci), and storage plumbing. The chart hardcodes storageClassName
# Fixtures for the paperless-ngx install-test: a throwaway postgres, the
# secrets the 1Password operator would otherwise materialize (release name:
# ci), and storage plumbing. The chart hardcodes storageClassName
# "longhorn" (RWX for the consume PVC), which does not exist in k3d: a
# no-provisioner StorageClass named "longhorn" plus pre-bound hostPath PVs
# stand in for it. Paperless runs as uid 1000 with a read-only rootfs and
# hostPath volumes do not support fsGroup, so a node-setup Job pre-creates
# the exact subPath directories world-writable.
#
# THERE IS NO THROWAWAY REDIS HERE ANY MORE, and that is the change this file
# carries: the release brings its own Valkey through the subchart
# (ci/paperless-ngx/test-values.yaml says what that proves), so the broker under
# test is the chart's own object rather than a Deployment next to it. Leaving a
# redis Deployment in place would have been the worse of the two: the url would
# have had to name one of them, and whichever it named, the other would have sat
# there making the run look like it covered something it did not.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
@@ -76,12 +84,43 @@ spec:
path: /var/lib/ci-pv/paperless-ngx/consume
type: DirectoryOrCreate
---
# The two Secrets below stand in for ONE 1Password item. Both the chart's
# redis OnePasswordItem and the subchart's auth OnePasswordItem point at
# op://ci/paperless-redis, and a OnePasswordItem materializes EVERY field of the
# item it names - so in a real cluster both Secrets carry all three fields. They
# are written out that way here rather than trimmed to what each consumer reads,
# because the trimmed version would have been a fixture that is tidier than
# reality and would have hidden the only thing worth knowing: the extra keys are
# unused, not a leak and not a bug.
apiVersion: v1
kind: Secret
metadata:
name: ci-paperless-ngx-redis-secret
stringData:
redis-connection-url: redis://:cipass@redis:6379
# Names the SUBCHART's Service, which is the mandatory vault step of this
# configuration (NOTES.txt prints it). A wrong name here is the failure this
# install-test exists to catch: paperless would come up healthy and process
# nothing.
redis-connection-url: redis://:cipass@ci-valkey-service:6379
key-prefix: ci
# Same password as in the url above - that duplication is the chart's
# documented trap 1, and the fixture is the cheapest place to keep it honest:
# let the two drift and the run goes red on authentication.
auth-conf: requirepass cipass
---
# What the subchart's own OnePasswordItem (ci-valkey-secret) would become. The
# key the Valkey actually mounts is "auth-conf", and its value has to be a
# COMPLETE configuration line, because the Valkey configuration includes the
# file: give it anything that is not "requirepass <password>" and the init
# container refuses to start the pod, which is the failure that chart exists to
# make loud.
apiVersion: v1
kind: Secret
metadata:
name: ci-valkey-secret
stringData:
auth-conf: requirepass cipass
redis-connection-url: redis://:cipass@ci-valkey-service:6379
key-prefix: ci
---
apiVersion: v1
@@ -153,41 +192,3 @@ spec:
ports:
- port: 5432
targetPort: 5432
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
labels:
app: redis
spec:
replicas: 1
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: public.ecr.aws/docker/library/redis:7-alpine
args: ["redis-server", "--requirepass", "cipass"]
ports:
- containerPort: 6379
readinessProbe:
tcpSocket:
port: 6379
periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
name: redis
spec:
selector:
app: redis
ports:
- port: 6379
targetPort: 6379
+69
View File
@@ -1,4 +1,36 @@
# CI values for the paperless-ngx install-test.
#
# THE SUBCHART IS ON, AND THAT IS THE POINT OF THIS FILE. Gitea #65 added the
# same valkey dependency to stalwart but covered it by a local RENDER only -
# deliberately, because Stalwart's in-memory store is an object in its database,
# so a Valkey in that namespace would have been reachable and unused and the
# test would have looked like it covered the clustering path while covering
# nothing of it. Here it is the opposite: paperless-ngx reads PAPERLESS_REDIS
# from the vault item, so pointing that url at the subchart's Service makes the
# bundled Valkey the broker this release actually uses. So this is the first
# install of the valkey-subchart path anywhere, and what it proves is:
#
# 1. The dependency resolves from the published index on a real runner with no
# credentials, and lint/package/render/install all survive it.
# 2. The subchart's objects are accepted inside a parent release: its
# StatefulSet becomes Ready, which means its init container found a
# "requirepass " line in the mounted auth file and its three exec probes
# authenticated and got PONG back.
# 3. The PVC path of the subchart works under a parent release name
# (ci-valkey-*), i.e. the names really are <release>-valkey-<kind>.
# 4. paperless-ngx talks to it: the url in the fixture Secret names
# ci-valkey-service, so a wrong Service name or a password mismatch between
# the two fields of the "item" shows up as a failing run instead of as a
# quietly idle Valkey.
#
# What it still CANNOT prove is that documents are processed - that needs a real
# consume run and is out of scope here, as before.
secrets:
# One item for both, which is the configuration the chart is designed around:
# redis-connection-url, key-prefix AND auth-conf live in the same 1Password
# item. In CI the operator does not exist, so ci/paperless-ngx/fixtures.yaml
# stands in for it - and it stands in TWICE, because a OnePasswordItem
# materializes every field of the item it names. See the fixture.
redisSecretRef: op://ci/paperless-redis
databaseSecretRef: op://ci/paperless-db
paperlessSecretRef: op://ci/paperless
@@ -16,5 +48,42 @@ paperless:
# neither erase them nor break rendering.
barcodes:
valkeySubchart:
enabled: true
valkey:
secrets:
# The SAME op:// path as secrets.redisSecretRef above, typed out again
# because Helm cannot default one value from another (Gitea #78). Typing it
# here is also what exercises the parent's guard in the "satisfied"
# direction; the failing direction is covered by a local render, since the
# harness installs one configuration per chart.
authSecretRef: op://ci/paperless-redis
persistence:
# QUEUE, the chart's and the subchart's default - and the risky half, which
# is why it is not switched off here: a provisioned PVC, the pod's fsGroup
# against a volume it did not create, and an append-only file written and
# replayed on it.
enabled: true
appendonly: true
# k3d's dynamic provisioner. The subchart's default, longhorn-encrypted-ns,
# does not exist on the runner, and the no-provisioner "longhorn" class the
# paperless fixtures define is pre-bound to hostPath PVs for paperless' own
# claims - a hostPath volume gets no fsGroup from the kubelet at all, so
# reusing it here would silently stop testing the ownership path.
storageClass: local-path
size: 128Mi
valkey:
resources:
# Small, to stay inside the runner's budget, while still exercising the
# subchart's derivation: 64Mi x limitFactor 3 = 192Mi limit, maxmemory =
# 0.75 x that = 144Mi.
requests:
cpu: 50m
memory: 64Mi
ephemeral-storage: 32Mi
# smbConnection and oidc stay disabled (defaults): the SMB CSI driver and a
# real OIDC login flow are out of scope for the CI install-test.
+461
View File
@@ -0,0 +1,461 @@
#!/usr/bin/env bash
# Selector-match render test: every label selector a chart renders is held
# against the labels of the pod templates in the SAME render.
#
# Usage: ci/run-selector-match-test.sh [chart ...] (no argument = all charts)
#
# WHAT IT GUARDS AGAINST, AND WHY A GREEN ROLLOUT DOES NOT (Gitea #96)
#
# Two kinds of object in this repository select pods by label, and BOTH of them
# fail silently when the selector matches nothing:
#
# topologySpreadConstraints - a constraint whose labelSelector matches no pod
# is TRIVIALLY SATISFIED. It is a condition over zero pods, so every
# placement meets it: nothing is distributed, the rollout is green, no event
# is emitted and no log line appears. With whenUnsatisfiable: DoNotSchedule
# this is worse rather than better, because the object then reads like a
# guarantee that two replicas sit on different nodes.
# PodDisruptionBudget - a budget whose selector matches no pod does its
# arithmetic over zero pods and never blocks an eviction. `kubectl drain`
# takes the whole group at once while the object sits in the namespace
# looking like protection.
#
# Neither is a render error, neither is an install-test failure, and neither is
# visible in a diff: the chart that breaks it is usually a chart that RENAMED a
# label somewhere else. Charts keep their selector and their pod labels in one
# helper for that reason (stalwart.groupLabels, keycloak's app label) - but one
# definition only helps while nobody writes a second one, and that is the part a
# test can hold.
#
# Over ALL charts and not the changed ones, for the same reason as
# ci/run-quantity-render-test.sh: the mistake arrives by copy into a new chart,
# whose own diff nobody reads for it, and running over everything means there is
# no list to maintain - adding a chart enrols it.
#
# HOW THE YAML IS PARSED, AND WHY NOT WITH A PARSER WRITTEN HERE
#
# The runners have neither pyyaml nor yq - measured, the same constraint
# ci/run-trivy-config.sh and ci/run-quantity-render-test.sh work under. Writing
# a YAML reader in awk or sed was the obvious alternative and is the wrong one:
# its failure mode is a parser that quietly reads nothing, finds no selector,
# reports no mismatch, and leaves this test green for a chart it never looked
# at - the exact class of signal this repository removes rather than adds.
#
# So HELM is the parser, which is a YAML implementation that is already
# installed, already trusted, and already the one that produced the manifests.
# ci/yaml2json is a one-template chart that prints its values back as JSON; the
# rendered manifest stream is handed to it as a VALUES FILE and comes back as
# JSON that python3 reads with its standard library. What IS done textually is
# only the document split, on a line that is exactly "---" in column one, and
# the consequence of getting that wrong is not a false pass: the fragment stops
# being valid YAML and helm exits 1.
#
# WHAT IS DELIBERATELY NOT CHECKED
#
# - podAffinity / podAntiAffinity. They carry labelSelectors too, but their
# semantics are "relative to pods that exist", and an anti-affinity term
# matching nothing is a legitimate configuration rather than a mistake.
# Out of scope, and named so that the gap is not mistaken for coverage.
# - selectors expressed ONLY through matchExpressions. Evaluating a set-based
# requirement means reimplementing label-selector semantics, which is a
# second parser by another name. Those are REPORTED as "not checked" and
# are never counted as passing, because a green tick for a code path nobody
# entered is worse than an honest gap (README, "Two rules for every check
# added here").
#
# AND THE ANTI-VACUUM GUARDS, because this test can be green over nothing. A
# chart renders a PDB or a spread constraint only above a replica threshold, and
# every chart's CI fixtures sit below it. Two rules therefore apply:
#
# 1. ci/selector-match/<chart>.yaml is an overlay applied last, whose only job
# is to push a chart over that threshold. A chart that HAS one and still
# renders no selector FAILS - the overlay exists for exactly that, so an
# overlay that stopped working is a finding and not a shrug.
# 2. If no selector at all was checked across the whole run, the run fails.
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
CI_DIR="$ROOT/ci"
YAML2JSON="$CI_DIR/yaml2json"
OVERLAY_DIR="$CI_DIR/selector-match"
RELEASE="selector"
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
# --- Which charts -----------------------------------------------------------
CHARTS=()
if [ "$#" -gt 0 ]; then
CHARTS=("$@")
else
for DIR in "$ROOT"/*/; do
[ -f "$DIR/Chart.yaml" ] || continue
CHARTS+=("$(basename "$DIR")")
done
fi
if [ "${#CHARTS[@]}" -eq 0 ]; then
echo "ERROR: no chart directories found under $ROOT" >&2
exit 1
fi
FAILED=""
CHECKED=""
NOT_EXERCISED=""
WITH_GAPS=""
TOTAL_SELECTORS=0
SUMMARY_ROWS=""
for CHART in "${CHARTS[@]}"; do
if [ ! -f "$ROOT/$CHART/Chart.yaml" ]; then
echo "ERROR: '$CHART' is not a chart directory" >&2
exit 1
fi
# Same reason as in ci/run-quantity-render-test.sh: a chart with a declared
# dependency cannot be rendered at all while charts/ is empty, and this job
# runs over EVERY chart on EVERY pull request - so without this line a PR
# touching nothing but a README would go red on the charts that have one.
"$ROOT/ci/helm-dependencies.sh" "$ROOT/$CHART"
# Values: the same precedence the other render tests use, so a chart's CI
# values are maintained in one place and satisfy its required() values.
VALUES_ARGS=()
VALUES_LABEL="chart defaults"
if [ -f "$CI_DIR/$CHART/test-values.yaml" ]; then
VALUES_ARGS=(-f "$CI_DIR/$CHART/test-values.yaml")
VALUES_LABEL="ci/$CHART/test-values.yaml"
elif [ -f "$CI_DIR/trivy/values/$CHART.yaml" ]; then
VALUES_ARGS=(-f "$CI_DIR/trivy/values/$CHART.yaml")
VALUES_LABEL="ci/trivy/values/$CHART.yaml"
fi
HAS_OVERLAY=""
if [ -f "$OVERLAY_DIR/$CHART.yaml" ]; then
# Last, so it wins over the install-test values.
VALUES_ARGS+=(-f "$OVERLAY_DIR/$CHART.yaml")
VALUES_LABEL="$VALUES_LABEL + ci/selector-match/$CHART.yaml"
HAS_OVERLAY=1
fi
echo "::group::$CHART ($VALUES_LABEL)"
RENDER="$WORK/$CHART.manifest.yaml"
if ! helm template "$RELEASE" "$ROOT/$CHART" \
${VALUES_ARGS[@]+"${VALUES_ARGS[@]}"} >"$RENDER" 2>"$WORK/err"; then
sed 's/^/ /' "$WORK/err"
echo "::endgroup::"
echo "::error title=selector-match $CHART::$CHART does not render with its CI values, so no selector could be checked. Reproduce: helm template $RELEASE $CHART $(printf '%s ' "${VALUES_ARGS[@]+${VALUES_ARGS[@]}}")"
FAILED="$FAILED $CHART"
continue
fi
# --- The document split, and the only textual step in this script ----------
# One YAML sequence item per rendered document, content indented by four and
# the item marker on a line of its own - so a document whose first line is a
# comment ("# Source: ...") needs no special case. A wrong split does not
# produce a false pass: the result stops being parsable and helm exits 1.
DOCS="$WORK/$CHART.docs.yaml"
awk '
BEGIN { print "docs:"; started = 0 }
/^---[ \t]*$/ { print " -"; started = 1; next }
{
if (!started) { print " -"; started = 1 }
if (length($0) == 0) print ""; else print " " $0
}
' "$RENDER" >"$DOCS"
JSON="$WORK/$CHART.json"
if ! helm template y2j "$YAML2JSON" -f "$DOCS" 2>"$WORK/err" \
| awk '/^\{/ { print; exit }' >"$JSON"; then
sed 's/^/ /' "$WORK/err"
echo "::endgroup::"
echo "::error title=selector-match $CHART::the rendered manifests of $CHART could not be re-read as YAML by ci/yaml2json. That is a parse failure, not a selector finding: either the render contains a document this script's '---' split cut in the wrong place, or the manifest is not valid YAML. Helm's own message is in the job log."
FAILED="$FAILED $CHART"
continue
fi
if [ ! -s "$JSON" ]; then
echo "::endgroup::"
echo "::error title=selector-match $CHART::ci/yaml2json produced no JSON for $CHART - the manifests were not re-read, so nothing was checked."
FAILED="$FAILED $CHART"
continue
fi
# --- The comparison --------------------------------------------------------
# Inline, as in ci/run-quantity-render-test.sh, and on the standard library
# only - json, because the YAML is already gone by this point. It writes one
# "VERDICT|detail" line per selector and never decides the exit code itself:
# the loop below does, so a verdict can never be lost to a swallowed status.
set +e
python3 - "$CHART" "$JSON" >"$WORK/$CHART.out" 2>"$WORK/$CHART.pyerr" <<'PY'
import json, sys
chart, path = sys.argv[1], sys.argv[2]
docs = json.load(open(path)).get("docs") or []
# A pod template is any value of a key named "template" that carries a spec with
# containers. Found by walking rather than by enumerating kinds, so a CronJob's
# spec.jobTemplate.spec.template is covered without a special case, and so is
# whatever workload kind a chart adds next.
def pod_templates(node, path="$"):
out = []
if isinstance(node, dict):
for key, value in node.items():
here = "%s.%s" % (path, key)
if (key == "template" and isinstance(value, dict)
and isinstance(value.get("spec"), dict)
and "containers" in value["spec"]):
out.append((here, value))
out.extend(pod_templates(value, here))
elif isinstance(node, list):
for i, value in enumerate(node):
out.extend(pod_templates(value, "%s[%d]" % (path, i)))
return out
def labels_of(template):
meta = template.get("metadata")
labels = meta.get("labels") if isinstance(meta, dict) else None
return labels if isinstance(labels, dict) else {}
def name_of(doc):
meta = doc.get("metadata")
if isinstance(meta, dict) and meta.get("name"):
return str(meta["name"])
return "<unnamed %s>" % doc.get("kind", "object")
def matches(selector, labels):
"""matchLabels only. Every key must be present with the same value; the
comparison is on the string form, because YAML 'true' and the string
"true" are the same label value to Kubernetes."""
for key, value in selector.items():
if key not in labels or str(labels[key]) != str(value):
return False
return True
def classify(selector, where):
"""(matchLabels dict, None) when it can be evaluated, (None, reason) when
it cannot. A selector this script cannot evaluate is REPORTED, never
counted as a pass."""
if selector is None:
return None, ("%s has NO selector at all. For a topology spread "
"constraint that means it matches no pod and is "
"therefore vacuously satisfied; for a PodDisruptionBudget "
"it is rejected by the API server" % where)
if not isinstance(selector, dict):
return None, "%s selector is not a mapping (%r)" % (where, selector)
match_labels = selector.get("matchLabels")
if isinstance(match_labels, dict) and match_labels:
if selector.get("matchExpressions"):
# matchLabels AND matchExpressions: the matchLabels half is still
# a necessary condition, so checking it is sound - it just is not
# the whole selector. Said out loud rather than passed off as full
# coverage.
return match_labels, None
return match_labels, None
if selector.get("matchExpressions"):
return None, ("%s selects only through matchExpressions, which this "
"test does not evaluate - doing so would mean "
"reimplementing label-selector semantics, i.e. a second "
"parser. Reported, not counted as passing" % where)
if match_labels == {} or selector == {}:
return {}, None # the empty selector matches every pod
return None, "%s selector has neither matchLabels nor matchExpressions" % where
# Rooted at "<Kind>/<name>" rather than at "$", because a chart with one
# workload per node group renders several documents whose internal path is
# identical - and then every finding would name the same spot.
all_templates = []
for doc in docs:
if isinstance(doc, dict):
root = "%s/%s" % (doc.get("kind", "?"), name_of(doc))
all_templates.extend(pod_templates(doc, root))
lines = []
# 1. Every topologySpreadConstraints entry, against ITS OWN pod template. That
# is the strict and the correct comparison: the failure being guarded
# against is a constraint that does not count the very pods it sits on.
for where, template in all_templates:
spread = template.get("spec", {}).get("topologySpreadConstraints")
if not isinstance(spread, list):
continue
labels = labels_of(template)
for i, constraint in enumerate(spread):
if not isinstance(constraint, dict):
lines.append("FAIL|%s: topologySpreadConstraints[%d] at %s is not a mapping"
% (chart, i, where))
continue
key = constraint.get("topologyKey", "<none>")
tag = "spread[%d] on %s at %s" % (i, key, where)
match_labels, reason = classify(constraint.get("labelSelector"), tag)
if match_labels is None:
if constraint.get("labelSelector") is None:
lines.append("FAIL|%s: %s" % (chart, reason))
else:
lines.append("SKIP|%s: %s" % (chart, reason))
continue
if matches(match_labels, labels):
lines.append("OK|%s: %s matches its own pod template's labels (%s)"
% (chart, tag, ", ".join("%s=%s" % kv for kv in sorted(match_labels.items())) or "empty selector, matches every pod"))
else:
lines.append("FAIL|%s: %s selects %s but the pod template it is "
"attached to carries %s - the constraint therefore "
"matches NONE of its own pods, is trivially satisfied, "
"and distributes nothing while the rollout stays green. "
"Render the chart and compare the selector with the pod "
"template's labels; the usual cause is a label renamed "
"in one of the two places"
% (chart, tag,
", ".join("%s=%s" % kv for kv in sorted(match_labels.items())),
", ".join("%s=%s" % kv for kv in sorted(labels.items())) or "no labels at all"))
# 2. Every PodDisruptionBudget, against ALL pod templates of the render: a
# budget has no owning template, it only has to select some pod that exists.
pdbs = [d for d in docs if isinstance(d, dict) and d.get("kind") == "PodDisruptionBudget"]
for doc in pdbs:
tag = "PodDisruptionBudget %s" % name_of(doc)
spec = doc.get("spec") if isinstance(doc.get("spec"), dict) else {}
match_labels, reason = classify(spec.get("selector"), tag)
if match_labels is None:
lines.append(("FAIL|%s: %s" if spec.get("selector") is None else "SKIP|%s: %s")
% (chart, reason))
continue
if not all_templates:
lines.append("FAIL|%s: %s is rendered but this render contains no pod "
"template at all, so its selector cannot match anything"
% (chart, tag))
continue
hits = [w for w, t in all_templates if matches(match_labels, labels_of(t))]
if hits:
lines.append("OK|%s: %s matches %d pod template(s): %s"
% (chart, tag, len(hits), ", ".join(hits)))
else:
known = sorted({"%s=%s" % kv for _, t in all_templates for kv in labels_of(t).items()})
lines.append("FAIL|%s: %s selects %s, which matches NONE of the %d pod "
"template(s) in the same render. A PDB that matches no pod "
"never blocks an eviction, so a node drain takes every "
"replica at once while the object looks like protection. "
"Labels actually present on the pod templates: %s"
% (chart, tag,
", ".join("%s=%s" % kv for kv in sorted(match_labels.items())),
len(all_templates), "; ".join(known) or "none"))
for line in lines:
print(line)
PY
RC=$?
set -e
if [ "$RC" -gt 1 ]; then
sed 's/^/ /' "$WORK/$CHART.pyerr"
echo "::endgroup::"
echo "::error title=selector-match $CHART::the selector comparison itself failed for $CHART (python exit $RC). Treated as a failure, never as a pass."
FAILED="$FAILED $CHART"
continue
fi
CHART_FAIL=""
CHART_OK=0
CHART_SKIP=0
while IFS='|' read -r VERDICT DETAIL; do
case "$VERDICT" in
OK)
CHART_OK=$((CHART_OK + 1))
echo " ok $DETAIL"
;;
SKIP)
CHART_SKIP=$((CHART_SKIP + 1))
echo " gap $DETAIL"
;;
FAIL)
CHART_FAIL=1
echo " FAIL $DETAIL"
echo "::error title=selector-match $CHART::$DETAIL"
;;
*)
[ -n "$VERDICT" ] && echo " $VERDICT $DETAIL"
;;
esac
done <"$WORK/$CHART.out"
echo "::endgroup::"
TOTAL_SELECTORS=$((TOTAL_SELECTORS + CHART_OK))
if [ -n "$CHART_FAIL" ]; then
FAILED="$FAILED $CHART"
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | **failed** | $CHART_OK | $CHART_SKIP |"$'\n'
echo "$CHART: FAILED"
elif [ "$CHART_OK" -gt 0 ]; then
CHECKED="$CHECKED $CHART"
[ "$CHART_SKIP" -gt 0 ] && WITH_GAPS="$WITH_GAPS $CHART"
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | ok | $CHART_OK | $CHART_SKIP |"$'\n'
echo "$CHART: ok ($CHART_OK selector(s) matched, $CHART_SKIP not checked)"
elif [ -n "$HAS_OVERLAY" ]; then
# The overlay's ONE job is to push this chart over the threshold at which
# it renders a selector. If it did not, this test is vacuum-green for the
# chart it was written for, which is the failure the overlay prevents.
FAILED="$FAILED $CHART"
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | **failed** | 0 | $CHART_SKIP |"$'\n'
echo "::error title=selector-match $CHART::$CHART has an overlay at ci/selector-match/$CHART.yaml but rendered no PodDisruptionBudget and no topologySpreadConstraints, so nothing was checked. That overlay exists only to make those objects render - either it no longer reaches the threshold (replica count, a feature switch) or the chart stopped rendering them. A green tick here would be a tick for a check that ran over nothing."
echo "$CHART: FAILED (overlay present, no selector rendered)"
else
NOT_EXERCISED="$NOT_EXERCISED $CHART"
[ "$CHART_SKIP" -gt 0 ] && WITH_GAPS="$WITH_GAPS $CHART"
echo "$CHART: no PodDisruptionBudget and no topologySpreadConstraints rendered - nothing to check."
fi
done
# --- Report ------------------------------------------------------------------
count() { set -- $1; echo "$#"; }
listing() { set -- $1; [ "$#" -eq 0 ] && { echo "–"; return; }; printf '`%s`' "$1"; shift; for C in "$@"; do printf ', `%s`' "$C"; done; echo; }
SUMMARY="${GITHUB_STEP_SUMMARY:-/dev/null}"
{
echo "### Selector-match render test (Gitea #96)"
echo
echo "Every \`PodDisruptionBudget\` selector and every \`topologySpreadConstraints\` label"
echo "selector in a chart's render, held against the labels of the pod templates in the"
echo "**same** render. A selector that matches no pod is not a render error and not an"
echo "install-test failure: the spread constraint is then trivially satisfied and the"
echo "budget never blocks an eviction, both silently."
echo
echo "| Result | Count | Charts |"
echo "|---|---|---|"
echo "| Selectors checked and matching | $(count "$CHECKED") | $(listing "$CHECKED") |"
echo "| No such object rendered | $(count "$NOT_EXERCISED") | $(listing "$NOT_EXERCISED") |"
echo "| Failed | $(count "$FAILED") | $(listing "$FAILED") |"
echo
echo "$TOTAL_SELECTORS selector(s) verified in total."
if [ -n "$WITH_GAPS" ]; then
echo
echo "Charts with selectors this test does **not** evaluate (\`matchExpressions\` only — reported, never counted as passing): $(listing "$WITH_GAPS")."
fi
if [ -n "$SUMMARY_ROWS" ]; then
echo
echo "| Chart | Result | Checked | Not checked |"
echo "| --- | --- | --- | --- |"
printf '%s' "$SUMMARY_ROWS"
fi
} >>"$SUMMARY"
echo "===================== selector match test ======================"
echo "checked: $(count "$CHECKED")${CHECKED:+ -$CHECKED}"
echo "not exercised: $(count "$NOT_EXERCISED")${NOT_EXERCISED:+ -$NOT_EXERCISED}"
echo "failed: $(count "$FAILED")${FAILED:+ -$FAILED}"
echo "selectors verified: $TOTAL_SELECTORS"
echo "================================================================"
if [ -n "$FAILED" ]; then
echo "ERROR: selectors do not match the pod labels they are meant to select in:$FAILED" >&2
exit 1
fi
# The second anti-vacuum guard. Reaching zero means every chart fell into "no
# such object rendered", which is indistinguishable from a broken harness - and
# it is the state this test would silently decay into if the overlays were ever
# dropped.
if [ "$TOTAL_SELECTORS" -eq 0 ]; then
echo "ERROR: not a single selector was checked in this run. Either no chart renders a PodDisruptionBudget or a topologySpreadConstraints any more, or the overlays under ci/selector-match/ no longer push one over its replica threshold. A pass over zero assertions is not a pass." >&2
exit 1
fi
+38
View File
@@ -0,0 +1,38 @@
# Overlay for ci/run-selector-match-test.sh, applied ON TOP of
# ci/stalwart/test-values.yaml and nowhere else - not by the install-test, not by
# the Trivy scan, not by the quantity render test.
#
# WHY IT EXISTS. Both objects this test looks at are rendered only while a node
# group's COMPUTED replica count is >= 2, and the install-test fixture runs
# full: 1 / mx: 0 deliberately (a second pod would need a real in-memory store,
# and a running "mx" pod would fail-open into a second full node against an
# empty database). So without this file stalwart renders no PodDisruptionBudget
# and no topologySpreadConstraints, and the test would be green for the chart it
# was written for, having looked at nothing. ci/run-selector-match-test.sh
# therefore FAILS a chart that has an overlay and still renders no selector.
#
# This is a RENDER-ONLY overlay. Nothing here is installed anywhere, so the two
# reasons the install-test keeps mx at 0 do not apply.
nodes:
full:
replicas: 2
exposedPorts: [smtp, submissions, imaps, http]
mx:
# Two as well, so BOTH groups' selectors are checked. The per-group split is
# the whole point of this chart's version of the rule: one object per group,
# each selecting only its own pods. A single group would leave the case where
# a group's selector accidentally matches the OTHER group's pods untested.
replicas: 2
exposedPorts: [smtp, http]
externalServices:
valkey:
# REQUIRED HERE, and not cosmetic: with more than one Stalwart pod and no
# in-memory store address the chart ABORTS THE RENDER ("this release runs N
# Stalwart pods, which is a cluster, but no in-memory store has an
# address"). Nothing connects to this name - the render never resolves it -
# but without it this overlay would turn the whole job red instead of
# exercising it. valkeySubchart.enabled: true would be the other way out and
# is deliberately not taken: it would add a network fetch of the published
# valkey chart to a test that otherwise needs none.
host: valkey-selector-match-fixture
+104
View File
@@ -0,0 +1,104 @@
# Fixtures for the stalwart install-test: a throwaway Postgres, the one Secret
# the 1Password operator would otherwise materialize, and a stub CRD for
# cert-manager's Certificate (release name: ci; the OnePasswordItem stub CRD
# comes from ci/common/).
#
# NO VALKEY HERE, DELIBERATELY. The in-memory store is an object in Stalwart's
# DATABASE, not a chart value, so a Valkey in this namespace would be reachable
# and unused - it would make the test look like it covered the clustering path
# while covering nothing of it. The test runs a single node (nodes.full
# replicas 1, nodes.mx replicas 0), which is the configuration that needs no
# in-memory store at all. The subchart path is covered by a local render only,
# named as a known gap in the PR that introduced this chart.
#
# A stub CRD for Certificate rather than certificate.enabled: false, because
# switching the Certificate off would also drop the volume and volumeMount that
# carry it - and the mount is on the risky side of readOnlyRootFilesystem. With
# the stub the object applies, the Secret it names never appears, and the
# optional: true on the volume is exercised for real: the pods must start
# without it. That is the behaviour an operator meets on every first install,
# before cert-manager has answered the challenge.
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: certificates.cert-manager.io
spec:
group: cert-manager.io
names:
kind: Certificate
listKind: CertificateList
plural: certificates
singular: certificate
scope: Namespaced
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
x-kubernetes-preserve-unknown-fields: true
---
apiVersion: v1
kind: Secret
metadata:
name: ci-stalwart-datastore-secret
stringData:
# The ONLY field this chart reads from the data store item. config.json
# references it by variable name through
# "authSecret": {"@type":"EnvironmentVariable","variableName":"STALWART_DB_PASSWORD"}
# so this value has to match the throwaway Postgres below. A real password
# rather than an empty string on purpose: upstream's resolver treats an empty
# environment variable as "not found" and fails the store open, so an empty
# value here would fail the run for the right reason in the wrong place.
password: ci-stalwart-throwaway-password
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
labels:
app: postgres
spec:
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: public.ecr.aws/docker/library/postgres:16-alpine
env:
- name: POSTGRES_USER
value: stalwart
- name: POSTGRES_PASSWORD
value: ci-stalwart-throwaway-password
- name: POSTGRES_DB
value: stalwart
ports:
- containerPort: 5432
readinessProbe:
exec:
command: ["pg_isready", "-U", "stalwart"]
periodSeconds: 5
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
volumes:
- name: data
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: postgres
spec:
selector:
app: postgres
ports:
- port: 5432
targetPort: 5432
+123
View File
@@ -0,0 +1,123 @@
# CI values for the stalwart install-test.
#
# WHAT THIS TEST CAN AND CANNOT PROVE — read this before trusting a green run.
#
# It CANNOT prove a working mail server, and no install-test here could. Since
# v0.16 every listener, the blob store, the in-memory store, the TLS object and
# the cluster roles live in Stalwart's DATABASE and are created by a human in
# the WebUI. Against an empty database there is no SMTP service to test, so a
# green run says nothing about mail.
#
# What it DOES prove is the part that is the chart's own responsibility, and the
# first item is the reason this chart is tested rather than excluded:
#
# 1. THE CONTAINER EXECS AT ALL. The image carries
# "setcap cap_net_bind_service=+ep" on its binary, which makes the kernel
# fail execve itself when the capability is not in the bounding set. The
# first draft of this chart had capabilities.drop: [ALL] with nothing
# added and would have failed with "operation not permitted" in every
# install - caught by reading the Dockerfile, but exactly the class of
# failure an install-test exists to catch rather than reason about.
# 2. readOnlyRootFilesystem: true holds with the two image volumes plus /tmp
# as emptyDirs, and the init container really can write config.json into
# the /etc/stalwart emptyDir.
# 3. The rendered config.json parses AS STALWART READS IT, not just as JSON:
# the server opens the data store, which means "@type": "PostgreSql", the
# integer-millisecond timeout and the authSecret EnvironmentVariable
# indirection are all accepted. This is the one check that the password
# really arrives without ever being in the ConfigMap.
# 4. The Secret key names match what the templates reference, the ConfigMap
# mount works, the NodePorts can be allocated, the resource quantities are
# valid, the schema accepts a realistic (non-default) values file, and the
# NetworkPolicy objects are accepted without cutting off the probes.
#
# RECOVERY MODE IS ON, and that is not a shortcut - it is the only mode that can
# work here. The bootstrap wizard appears only when config.json is ABSENT, and
# this chart always renders one; upstream's documented path for a templated
# config.json is to start once with STALWART_RECOVERY_MODE=1. Recovery mode
# brings up the management HTTP endpoint and keeps the mail ports closed, which
# is precisely the reachable surface the probes use.
#
# THE ONE THING THIS FILE COULD NOT BE VERIFIED AGAINST: whether /healthz/live
# and /healthz/ready are served on the RECOVERY listener. The source shows them
# handled in the general HTTP request path with no auth and no network
# restriction, but recovery mode binds a single listener and skips most config
# parsing, and there is no cluster in the authoring environment to try it on.
# If the install-test goes red with the pods never becoming Ready, that is the
# reason, and the fix is either an exec probe or an entry in
# ci/excluded-charts.txt - see the PR for #65.
bootstrap:
enabled: true
logLevel: debug
dataStore:
postgres:
host: postgres
database: stalwart
username: stalwart
tls:
# OFF, against the chart's own default of true. The throwaway Postgres in
# fixtures.yaml serves no TLS, and the alternative - allowInvalidCerts -
# would be worse: it would pass whether or not TLS worked, so a chart
# change that broke the TLS path would still go green. A deviation that
# fails loudly beats one that hides.
enable: false
allowInvalidCerts: false
secrets:
# Materialized by ci/stalwart/fixtures.yaml as ci-stalwart-datastore-secret.
dataStoreSecretRef: op://ci/stalwart-datastore
# Both deliberately unset: neither is needed to bring a pod up, and leaving
# them out exercises the conditional env and OnePasswordItem blocks in the
# "absent" direction, which is the shape a minimal install really has.
recoveryAdminSecretRef: null
s3SecretRef: null
ingress:
# Left ON although no ingress controller answers in CI: the point is that the
# object is accepted and that the per-group http Service it names actually
# exists - a selector typo there is otherwise invisible until someone opens
# the URL.
enabled: true
domain: stalwart.ci.example.com
clusterIssuer: ci-selfsigned
nodeGroup: full
certificate:
# On, against a STUB CRD (fixtures.yaml). Nothing issues the Secret it names,
# which is the interesting case: the volume is mounted optional, so the pods
# must start without it. That is what every real first install looks like
# before cert-manager has answered the challenge.
enabled: true
clusterIssuer: ci-selfsigned
networkPolicy:
# Left ON. probeSources stays empty, so the http rule renders without a
# "from" selector - the configuration this chart ships. Whether the policy
# actually FILTERS is a property of the cluster's CNI and is not claimed by
# this test; what is claimed is that the objects are accepted and that they
# do not keep the kubelet's probes from working.
enabled: true
nodes:
full:
replicas: 1
exposedPorts: [smtp, submissions, imaps, http]
resources:
# Small, to stay inside the runner's budget, while still exercising the
# limit derivation: 256Mi x limitFactor 3 = 768Mi.
requests:
cpu: 100m
memory: 256Mi
ephemeral-storage: 64Mi
mx:
# Stays at 0, which is also the chart default. Two reasons rather than one:
# a second node would need an in-memory store (the render aborts without
# one), and the "mx" ClusterRole does not exist in an empty database, so a
# running mx pod would fail-open into a second full node and the two would
# fight over the same NodePort-backed SMTP Service. The StatefulSet is
# still created and the harness still waits on it - a 0-replica rollout
# completes immediately, so this covers the explicit-zero path of the
# replicas helper.
replicas: 0
exposedPorts: [smtp, http]
+6
View File
@@ -20,3 +20,9 @@ ksv0125:
- cr.jk-dev.de
- ghcr.io
- docker.io
# quay.io, added with the keycloak chart (Gitea #83) - Keycloak publishes
# its official image only there (quay.io/keycloak/keycloak), and it is so
# far the only chart in this repo using that registry. Listed rather than
# suppressed, for the reason the comment above gives: the check keeps being
# able to say something about an image from a genuinely foreign source.
- quay.io
+24
View File
@@ -0,0 +1,24 @@
apiVersion: v2
# NOT A CHART THIS REPOSITORY PUBLISHES. It is a YAML parser, and Helm is the
# only one the runners have.
#
# ci/run-selector-match-test.sh has to read label selectors out of rendered
# manifests, and the runners carry neither pyyaml nor yq - measured, and the
# same constraint ci/run-trivy-config.sh and ci/run-quantity-render-test.sh
# work under. The alternative to this chart is a YAML parser written in awk or
# sed, and the failure mode of that is the one this repository removes rather
# than adds: a parser that quietly reads nothing finds no selector, reports no
# mismatch, and the test is green for a chart it never looked at.
#
# So the manifest fragment is handed to Helm as a VALUES FILE, and the one
# template below prints it back as JSON, which python3 reads with its standard
# library. A fragment Helm cannot parse makes helm exit 1, which is loud.
#
# It lives under ci/ and therefore has no publish workflow, no version-bump
# requirement and no entry in change-app-version.yml: the chart-discovery loops
# in this repository look for a Chart.yaml one level below the repo root, and
# "ci" has none. The version below is meaningless for the same reason.
name: yaml2json
description: Internal CI helper - reprints a values file as JSON so python3 can read YAML without pyyaml
type: application
version: 0.0.0
+1
View File
@@ -0,0 +1 @@
{{ .Values | toJson }}
+4
View File
@@ -0,0 +1,4 @@
# Deliberately empty. Every value this chart renders comes from the -f file the
# caller passes, which is one document out of a rendered manifest stream. A
# default here would be merged into that document and would show up as a key
# the chart under test never rendered.
-12
View File
@@ -1,12 +0,0 @@
apiVersion: v2
name: convertx
description: A Helm chart to deploy ConvertX on Kubernetes
icon: https://raw.githubusercontent.com/C4illin/ConvertX/main/public/android-chrome-512x512.png
type: application
version: 1.0.0+upv0.19.0
appVersion: "v0.19.0"
maintainers:
- name: jklein
-224
View File
@@ -1,224 +0,0 @@
{{- $replicas := include "convertx.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.convertx.replicas) -}}
{{- $startup := fromYaml (include "convertx.startupProbe" .Values.convertx.startupProbe) -}}
{{- $autoDelete := include "convertx.autoDeleteEveryNHours" .Values.convertx.autoDeleteEveryNHours -}}
{{- $registration := include "convertx.boolString" (dict "value" .Values.convertx.accountRegistration "default" false "path" "convertx.accountRegistration") -}}
{{- $unauthenticated := include "convertx.boolString" (dict "value" .Values.convertx.allowUnauthenticated "default" false "path" "convertx.allowUnauthenticated") -}}
{{- $httpAllowed := include "convertx.boolString" (dict "value" .Values.convertx.httpAllowed "default" false "path" "convertx.httpAllowed") -}}
ConvertX {{ .Chart.AppVersion }} — release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
{{- if .Values.ingress.enabled }}
ACCESS
https://{{ .Values.ingress.domain }}
The certificate is requested by cert-manager through the "{{ .Values.ingress.clusterIssuer }}"
cluster issuer. Until it has been issued the browser shows a warning; follow it with
kubectl -n {{ .Release.Namespace }} get certificate tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
{{- else }}
ACCESS — NO INGRESS
ingress.enabled is false, so this release has no Ingress and no certificate,
and neither ingress.clusterIssuer nor ingress.domain is needed. ConvertX has
no base-URL setting of its own — it builds no absolute links — so nothing
outside the Ingress reads the domain in this chart.
Reach the release inside the cluster with
kubectl -n {{ .Release.Namespace }} port-forward svc/{{ .Release.Name }}-{{ .Chart.Name }}-service 8080:80
and open http://localhost:8080. Note that the session cookie is marked
"secure"{{ if eq $httpAllowed "true" }} — except that convertx.httpAllowed is true in this release, which
clears that flag, so logging in over plain http works{{ else }}, so logging in over plain http will NOT work: the browser
refuses to store the cookie. convertx.httpAllowed: true clears that flag{{ end }}.
{{- end }}
{{ if eq $replicas "0" }}
SCALED TO ZERO
{{ if .Values.scaleDown }}scaleDown: true{{ else }}convertx.replicas: 0{{ end }} — the StatefulSet runs with replicas: 0, so the endpoint
above does not answer until you set it back and upgrade again. PVC, Secret{{ if .Values.ingress.enabled }},
Service and Ingress stay in place, so neither the conversions nor the
certificate are touched.{{ else }} and
Service stay in place, so the conversions on the PVC are untouched.{{ end }}
{{ end }}
WHOEVER OPENS THE PAGE FIRST BECOMES THE ADMINISTRATOR
A fresh ConvertX has no users, and with no users it serves a SETUP FORM to
every visitor — no password, no invitation, nothing to bypass. The first
account created that way owns the instance, and convertx.accountRegistration
does not gate it: that value decides whether accounts TWO and onwards can be
created, not the first one.
So on a fresh install, register yourself before the DNS name is reachable by
anyone else. There is no second chance and no "reset admin" command; the
repair is deleting the users row from the SQLite database on the PVC.
{{- if eq $unauthenticated "true" }}
This release has convertx.allowUnauthenticated: true, so the point above is
moot in the worst way — see the warning further down.
{{- end }}
THE 1PASSWORD ITEM MUST CARRY THE FIELD THE CHART READS
secrets.jwtSecretRef points at
{{ .Values.secrets.jwtSecretRef }}
The 1Password operator materializes that item as the Kubernetes Secret
{{ .Release.Name }}-{{ .Chart.Name }}-secret, with one Secret key per FIELD of the item. So the field
name is not free-form: this chart reads exactly one field, and it must be
spelled
jwt-secret a long random string — openssl rand -hex 32
An item without it does not fail the install. The release goes through and
the pod then hangs in CreateContainerConfigError naming the key it could not
find. Watch the materialization with
kubectl -n {{ .Release.Namespace }} get onepassworditem {{ .Release.Name }}-{{ .Chart.Name }}-secret
kubectl -n {{ .Release.Namespace }} get secret {{ .Release.Name }}-{{ .Chart.Name }}-secret
WHY THIS VALUE IS REQUIRED RATHER THAN OPTIONAL: ConvertX signs its session
cookies with JWT_SECRET, and when the variable is unset it quietly generates
a random one PER PROCESS. Every restart — a node drain, an image update, an
OOM kill — would then invalidate every session, with no log line and no
error. Rotating the field has the same effect on purpose; after a rotation:
kubectl -n {{ .Release.Namespace }} rollout restart statefulset/{{ .Release.Name }}-{{ .Chart.Name }}-sfs
Editing the value INSIDE the item does not roll the pod by itself (Helm
never sees a secret's content). Pointing secrets.jwtSecretRef at a DIFFERENT
item does — the pod template carries a checksum over the rendered
OnePasswordItem.
TWO SWITCHES THAT OPEN THE INSTANCE UP
convertx.accountRegistration {{ $registration }}
convertx.allowUnauthenticated {{ $unauthenticated }}
Upstream warns explicitly against enabling either on an instance reachable
from the internet, and this chart puts a public DNS name in front by
default. accountRegistration lets anyone with the URL create an account;
allowUnauthenticated removes the login entirely, so anyone who reaches the
Service can upload files and read every conversion on the volume.
{{- if or (eq $registration "true") (eq $unauthenticated "true") }}
*** {{ if and (eq $registration "true") (eq $unauthenticated "true") }}BOTH OF THEM ARE ON{{ else }}ONE OF THEM IS ON{{ end }} IN THIS RELEASE. ***
{{- if eq $unauthenticated "true" }}
convertx.allowUnauthenticated: true — there is no login. Anyone who can
reach {{ if .Values.ingress.enabled }}https://{{ .Values.ingress.domain }}{{ else }}the Service{{ end }} can use it and read other people's files.
{{- end }}
{{- if eq $registration "true" }}
convertx.accountRegistration: true — anyone with the URL can sign up.
{{- end }}
If this instance is reachable from the internet, that is almost certainly
not what you wanted.
{{- end }}
RETENTION IS A VALUE, AND IT IS THE ONLY THING THAT EMPTIES THE VOLUME
{{- if eq $autoDelete "0" }}
convertx.autoDeleteEveryNHours is 0 — THE CLEANUP IS OFF. Uploads and
converted output accumulate on the PVC for as long as the release lives and
nothing reclaims them. Watch the volume, or set a non-zero number of hours.
{{- else }}
convertx.autoDeleteEveryNHours is {{ $autoDelete }} — ConvertX deletes the uploads and the
output of any job older than {{ $autoDelete }} hours. That is a hard delete, not an
archive: a conversion nobody downloaded in time is gone. Set it to 0 to
switch the cleanup off, and then size storage.amount for unbounded growth.
{{- end }}
STORAGE
{{ .Release.Name }}-{{ .Chart.Name }}-pvc, {{ .Values.storage.amount }}, {{ if .Values.storage.storageClass }}storage class "{{ .Values.storage.storageClass }}"{{ else }}on the cluster's default storage class
(storage.storageClass is unset){{ end }}, mounted at /app/data.
Everything this release persists is on that one volume: the SQLite database
(users, jobs, file names), every uploaded file and every converted result.
There is no external database and no object storage, so one snapshot of the
PVC is a complete backup — and losing it loses the accounts along with the
files.
Because the PVC is rendered by the chart, helm uninstall deletes it and the
data with it, unless the storage class reclaim policy says otherwise.
{{- if gt (int $replicas) 1 }}
convertx.replicas is {{ $replicas }}, and that does not work. The PVC is ReadWriteOnce, so
the extra pods normally stay Pending on "Multi-Attach error for volume"; on
a genuinely RWX storage class they would start and then corrupt the SQLite
database two processes are now writing. Set it back to 1.
{{- end }}
{{- if .Values.ingress.enabled }}
THE INGRESS IS THE ONLY UPLOAD CEILING
ingress.maxUploadFileSize {{ .Values.ingress.maxUploadFileSize }}
ConvertX enforces no size limit of its own — none, at any setting — so this
annotation is the whole story. A file above it never reaches the
application: the user gets the ingress controller's 413 page and nothing
appears in the pod log, which makes it look like a broken upload button
rather than a limit.
Raising it has a second cost: the upload lands on the PVC above, so a
ceiling far beyond storage.amount just moves the failure from a 413 to a
full volume.
{{- else }}
UPLOAD CEILING — NONE FROM THIS CHART
ingress.enabled is false, so the chart sets no body-size limit anywhere, and
ConvertX enforces none of its own. Whatever proxy you put in front brings
its ceiling with it — ingress-nginx defaults to 1m, small enough to look
like a broken upload button.
{{- end }}
WHAT A GREEN POD PROVES
All three probes are HTTP GETs against /healthcheck, which ConvertX answers
with a static 200 and no authentication. It touches neither the database nor
the volume, so a green pod proves the process is serving — not that a
conversion works and not that the volume is writable.
It is deliberately NOT "/": with the defaults of this chart "/" answers 302,
to /setup while no account exists and to /login afterwards. A probe would
pass on that redirect, and pass on a different one from day two onwards.
The startup probe grants {{ $startup.failureThreshold }} × {{ $startup.periodSeconds }}s = {{ mul $startup.failureThreshold $startup.periodSeconds }}s. That budget is generous because
ConvertX shells out to about twenty converter binaries at startup just to
log their versions.
THE FIRST PULL IS SLOW, AND THAT IS NORMAL
The image is roughly 1.4 GiB compressed — LibreOffice, Calibre, ImageMagick,
ffmpeg, Inkscape and a TeX Live installation are all inside it. On a node
that has not seen it before, expect several minutes in ContainerCreating
before the startup probe budget above even begins. A pod that looks stuck
there is usually still pulling:
kubectl -n {{ .Release.Namespace }} describe pod {{ .Release.Name }}-{{ .Chart.Name }}-sfs-0
THIS CHART BYPASSES THE IMAGE ENTRYPOINT
The image runs as root and drops privileges itself; this chart starts the
application directly (command: bun run dist/src/index.js) as uid 1000, with
fsGroup doing what the entrypoint's chown would have done. PUID, PGID and
UMASK therefore configure NOTHING here — setting them through convertx.env
has no effect at all, and the uid belongs in convertx.securityContext.pod.
The cost to know about: if upstream moves real work into that entrypoint,
this chart skips it silently. Re-read the upstream entrypoint.sh at every
app-version update.
{{- if .Values.convertx.env }}
EXTRA ENVIRONMENT VARIABLES ARE ACTIVE
convertx.env adds {{ len .Values.convertx.env }} variable(s), rendered after the ones the chart sets
itself. Two warnings that go with that block:
- WEBROOT moves every route of the application, /healthcheck included, so
it breaks all three probes while the app is perfectly healthy — and this
chart's Ingress still maps "/". Do not set it here.
- An entry repeating a name the chart already sets is additive, not a
replacement. At install yours wins; on an EXISTING release Kubernetes
merges env by name and the chart's own value survives instead, while helm
upgrade still reports success. Read it back off the StatefulSet.
{{- end }}
{{- if .Values.ingress.enabled }}
{{- with .Values.ingress.annotations }}
EXTRA INGRESS ANNOTATIONS ARE ACTIVE
{{- range $key, $value := . }}
{{ $key }}: {{ $value }}
{{- end }}
These are merged with the cert-manager and proxy-body-size annotations the
chart always sets. Annotations are interpreted by the ingress controller, so
a misspelled key is silently ignored — it does not fail the render.
{{- end }}
{{- end }}
-327
View File
@@ -1,327 +0,0 @@
{{/*
Render the body of a container "resources:" block from a values dict of the form
resources:
limitFactor: 3 # optional, default 3
requests: { cpu, memory, ephemeral-storage / ephemeralStorage, ... }
limits: { ... } # optional, every entry optional
Rules (variant B, see issue #19):
- requests are rendered as given (camelCase "ephemeralStorage" is normalized
to the Kubernetes key "ephemeral-storage").
- an explicitly set limit always wins.
- a missing memory / ephemeral-storage limit is computed as
limitFactor x the corresponding request.
- a missing cpu limit is NOT set at all (the kubelet then treats CPU as
unlimited; CPU is compressible, so this cannot kill the workload).
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart. template-chart/templates/_helpers.tpl is the reference implementation.
*/}}
{{- define "convertx.resources" -}}
{{- $resources := . | default dict -}}
{{- $factor := 3.0 -}}
{{- if not (kindIs "invalid" $resources.limitFactor) -}}
{{- $factor = float64 $resources.limitFactor -}}
{{- end -}}
{{- $requests := dict -}}
{{- range $key, $value := ($resources.requests | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $requests (include "convertx.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- $limits := dict -}}
{{- range $key, $value := ($resources.limits | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $limits (include "convertx.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- range $key, $request := $requests -}}
{{- if and (ne $key "cpu") (not (hasKey $limits $key)) -}}
{{- $_ := set $limits $key (include "convertx.multiplyQuantity" (dict "quantity" $request "factor" $factor)) -}}
{{- end -}}
{{- end -}}
{{- $out := dict -}}
{{- with $requests }}{{- $_ := set $out "requests" . }}{{- end -}}
{{- with $limits }}{{- $_ := set $out "limits" . }}{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Normalize a resource name from values to its Kubernetes key.
*/}}
{{- define "convertx.resourceKey" -}}
{{- if eq . "ephemeralStorage" -}}ephemeral-storage{{- else -}}{{ . }}{{- end -}}
{{- end -}}
{{/*
Multiply a Kubernetes quantity by a factor, keeping its suffix.
Input: dict "quantity" <string|number> "factor" <float64>.
Handles plain numbers ("2", "0.5", 1073741824), decimal ("1.5Gi"), binary
("512Mi", "1Gi", "10Ki") and milli/SI suffixes ("100m", "500M"). Integer
results are printed without a decimal point.
*/}}
{{- define "convertx.multiplyQuantity" -}}
{{- $quantity := include "convertx.formatNumber" .quantity -}}
{{- $number := regexFind "^[0-9]+(\\.[0-9]+)?" $quantity -}}
{{- if not $number -}}
{{- fail (printf "cannot parse resource quantity %q" $quantity) -}}
{{- end -}}
{{- $suffix := trimPrefix $number $quantity -}}
{{- if not (regexMatch "^(m|k|M|G|T|P|E|Ki|Mi|Gi|Ti|Pi|Ei)?$" $suffix) -}}
{{- fail (printf "unsupported resource quantity suffix %q in %q" $suffix $quantity) -}}
{{- end -}}
{{- $product := mulf (float64 $number) .factor -}}
{{- if eq $product (floor $product) -}}
{{- printf "%d%s" (int64 $product) $suffix -}}
{{- else -}}
{{- printf "%s%s" (include "convertx.formatNumber" $product) $suffix -}}
{{- end -}}
{{- end -}}
{{/*
Render a number as a plain decimal string, never in exponential notation.
Why this exists (issue #157): the null-safe fallback helpers pass values
through fromYaml, which parses every number as float64. Go's "%v" switches
float64 to exponential notation from about 1e6 upwards, so a bare byte count
like 1048576 became "1.048576e+06" - not a valid Kubernetes quantity, and the
render aborted. Strings ("512Mi") are passed through untouched.
*/}}
{{- define "convertx.formatNumber" -}}
{{- if kindIs "float64" . -}}
{{- regexReplaceAll "\\.?0+$" (printf "%.10f" .) "" -}}
{{- else -}}
{{- printf "%v" . -}}
{{- end -}}
{{- end -}}
{{/*
Render the replica count for the StatefulSet.
Rules (issue #25):
- scaleDown: true takes precedence and always yields 0.
- otherwise an explicitly set replicas value is used as given, including an
explicit 0 - Helm's "default" would swallow 0, hence the explicit
kindIs "invalid" nil check.
- a missing replicas value defaults to 1.
*/}}
{{- define "convertx.replicas" -}}
{{- if .scaleDown -}}
0
{{- else if not (kindIs "invalid" .replicas) -}}
{{- .replicas -}}
{{- else -}}
1
{{- end -}}
{{- end -}}
{{/*
Merge a user-supplied values sub-tree onto a dict of chart defaults so that a
null / empty override falls back to the defaults instead of erasing them.
Input: dict "defaults" <dict> "given" <dict|nil> "path" <string, optional
values path used in error messages>. Output: YAML of the merged dict (consume
it with fromYaml).
Why this exists (issues #61 and #99): Helm merges values maps recursively, but
an override key written without children - e.g. a leftover
securityContext:
after deleting its last remaining sub-key - is YAML null. Where the chart
defines a default, Helm treats that null as a deletion of the default; the
template then renders "securityContext: null": no readOnlyRootFilesystem, no
dropped capabilities, no allowPrivilegeEscalation: false. The workload starts
and reports healthy while being unhardened, so nothing points at the mistake.
This helper defines the opposite semantics: an empty block means "chart
defaults apply".
*/}}
{{- define "convertx.defaultedDict" -}}
{{- $defaults := .defaults | default dict -}}
{{- $given := .given -}}
{{- $path := .path | default "value" -}}
{{- if kindIs "invalid" $given -}}
{{- $given = dict -}}
{{- end -}}
{{- if not (kindIs "map" $given) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" $path (kindOf $given) $given) -}}
{{- end -}}
{{- $out := deepCopy $defaults -}}
{{- range $key, $value := $given -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $default := index $defaults $key -}}
{{- if and (kindIs "map" $value) (kindIs "map" $default) -}}
{{- $_ := set $out $key (fromYaml (include "convertx.defaultedDict" (dict "defaults" $default "given" $value "path" (printf "%s.%s" $path $key)))) -}}
{{- else if kindIs "map" $default -}}
{{- fail (printf "%s.%s must be a mapping or null, got %s (%v)" $path $key (kindOf $value) $value) -}}
{{- else -}}
{{- $_ := set $out $key $value -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Pick one sub-block out of a values block that may itself be null.
Input: dict "block" <dict|nil> "key" <string> "path" <string>.
Returns YAML of dict "value" <the sub-block, may be nil>, after rejecting a
non-mapping block with a message that names the values path. Needed because
".Values.convertx.securityContext" can itself be erased, and indexing into nil
inside a template is not an error but silently yields nothing.
*/}}
{{- define "convertx.subBlock" -}}
{{- $block := .block -}}
{{- if kindIs "invalid" $block -}}
{{- $block = dict -}}
{{- end -}}
{{- if not (kindIs "map" $block) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" .path (kindOf $block) $block) -}}
{{- end -}}
{{- toYaml (dict "value" (index $block .key)) -}}
{{- end -}}
{{/*
The effective security contexts, probes and resources (issue #99).
These defaults MUST stay in sync with the corresponding blocks in values.yaml,
which stays the documented, user-facing place for them; this copy is only the
fallback for the case where an override has erased the block.
TWO OF THEM REPRODUCE A DELIBERATE DEVIATION AND MUST NOT BE "TIDIED":
- runAsUser / runAsGroup are 1000, not the repo's arbitrary 20000. The image
creates its convertx user at uid 1000 and owns /home/convertx with it, and
that uid owns the data on the PersistentVolume. Restoring the repo default
here would move the runtime uid off the volume's data.
- the probes point at /healthcheck, not at "/". "/" answers 302 in every
default configuration of this chart, and the redirect target changes once
the first account exists. The port name "http" is part of the context and
belongs here too: a default that loses it leaves the workload with a probe
that cannot succeed.
*/}}
{{- define "convertx.podSecurityContext" -}}
{{- $given := (fromYaml (include "convertx.subBlock" (dict "block" . "key" "pod" "path" "convertx.securityContext"))).value -}}
{{- $defaults := dict
"runAsNonRoot" true
"runAsUser" 1000
"runAsGroup" 1000
"fsGroup" 1000
"fsGroupChangePolicy" "Always"
"seccompProfile" (dict "type" "RuntimeDefault") -}}
{{- include "convertx.defaultedDict" (dict "defaults" $defaults "given" $given "path" "convertx.securityContext.pod") -}}
{{- end -}}
{{- define "convertx.containerSecurityContext" -}}
{{- $given := (fromYaml (include "convertx.subBlock" (dict "block" . "key" "container" "path" "convertx.securityContext"))).value -}}
{{- $defaults := dict
"allowPrivilegeEscalation" false
"readOnlyRootFilesystem" true
"runAsNonRoot" true
"capabilities" (dict "drop" (list "ALL") "add" (list)) -}}
{{- include "convertx.defaultedDict" (dict "defaults" $defaults "given" $given "path" "convertx.securityContext.container") -}}
{{- end -}}
{{- define "convertx.livenessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/healthcheck" "port" "http")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "convertx.defaultedDict" (dict "defaults" $defaults "given" . "path" "convertx.livenessProbe") -}}
{{- end -}}
{{- define "convertx.readinessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/healthcheck" "port" "http")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "convertx.defaultedDict" (dict "defaults" $defaults "given" . "path" "convertx.readinessProbe") -}}
{{- end -}}
{{- define "convertx.startupProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/healthcheck" "port" "http")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 30 -}}
{{- include "convertx.defaultedDict" (dict "defaults" $defaults "given" . "path" "convertx.startupProbe") -}}
{{- end -}}
{{/*
The effective resources block, fed into the resources helper above so that an
erased "resources:" still yields the chart's requests and computed limits
instead of a container without any resource accounting.
The numbers reproduce values.yaml, which carries the reasoning: they are sized
for converter subprocesses (LibreOffice, Calibre, ImageMagick, ffmpeg), not
for an idle web server, and the ephemeral-storage figure has to cover the two
emptyDirs this chart mounts.
*/}}
{{- define "convertx.effectiveResources" -}}
{{- $defaults := dict
"limitFactor" 3
"requests" (dict "cpu" 0.25 "memory" "1Gi" "ephemeralStorage" "1Gi") -}}
{{- $merged := fromYaml (include "convertx.defaultedDict" (dict "defaults" $defaults "given" . "path" "convertx.resources")) -}}
{{- include "convertx.resources" $merged -}}
{{- end -}}
{{/*
Render a values boolean as the literal string the application accepts.
Input: dict "value" <bool|string|nil> "path" <values path, for the error>
"default" <bool, used when the value is nil>
Why this is not just "{{ .Values.x }}" (README, "Values that must not lie",
point 6): ConvertX parses each of these variables as
process.env.X?.toLowerCase() === "true"
so ANYTHING that is not the literal "true" means false - "yes", "1", "on",
"TRUE " with a trailing space. Two of the three variables this helper renders
open the instance up (ACCOUNT_REGISTRATION, ALLOW_UNAUTHENTICATED), and a
value written to switch one of them ON that silently lands on OFF is the kind
of failure nobody goes looking for, because the symptom is the safe state.
So: a bool renders as "true"/"false", the strings "true"/"false" render
normalized to lower case, nil takes the given default, and everything else
ABORTS THE RENDER naming the values path. Go's "not" is deliberately unused
here - it treats every non-empty string as true, so "not \"false\"" is false
and the string form would flip the wrong way.
*/}}
{{- define "convertx.boolString" -}}
{{- $v := .value -}}
{{- if kindIs "invalid" $v -}}
{{- $v = .default -}}
{{- end -}}
{{- if kindIs "bool" $v -}}
{{- ternary "true" "false" $v -}}
{{- else -}}
{{- $s := lower (printf "%v" $v) -}}
{{- if or (eq $s "true") (eq $s "false") -}}
{{- $s -}}
{{- else -}}
{{- fail (printf "%s must be true or false, got %q - ConvertX compares this variable against the literal string \"true\", so any other spelling would silently mean false" .path $v) -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{/*
Render AUTO_DELETE_EVERY_N_HOURS.
An explicit 0 is the documented way to switch the cleanup off, so this cannot
go through Helm's "default": that swallows a 0 and would restore 24-hour
deletion on exactly the install that asked for none. Same nil-check shape as
the replicas helper, and for the same reason.
*/}}
{{- define "convertx.autoDeleteEveryNHours" -}}
{{- if kindIs "invalid" . -}}
24
{{- else -}}
{{- include "convertx.formatNumber" . -}}
{{- end -}}
{{- end -}}
-34
View File
@@ -1,34 +0,0 @@
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-ingress
annotations:
cert-manager.io/cluster-issuer: {{ required "A Cluster-Issuer must be provided" .Values.ingress.clusterIssuer }}
# ConvertX enforces no upload limit of its own (it starts its HTTP server
# with maxRequestBodySize: Number.MAX_SAFE_INTEGER), so these two
# annotations are the ONLY ceiling on an upload. Both spellings are set
# because the two nginx ingress controllers read different keys.
nginx.ingress.kubernetes.io/proxy-body-size: {{ .Values.ingress.maxUploadFileSize }}
nginx.org/client-max-body-size: {{ .Values.ingress.maxUploadFileSize }}
{{- with .Values.ingress.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
ingressClassName: {{ .Values.ingress.className }}
rules:
- host: {{ required "A Domain/Host must be provided" .Values.ingress.domain }}
http:
paths:
- backend:
service:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
port:
name: http
path: /
pathType: Prefix
tls:
- hosts:
- {{ required "A Domain/Host must be provided" .Values.ingress.domain }}
secretName: tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
{{- end }}
-13
View File
@@ -1,13 +0,0 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: {{ .Values.storage.amount }}
{{- if .Values.storage.storageClass }}
storageClassName: {{ .Values.storage.storageClass }}
{{- end }}
-6
View File
@@ -1,6 +0,0 @@
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-secret
spec:
itemPath: {{ required "A Secret-Reference for the JWT signing key must be provided (secrets.jwtSecretRef)" .Values.secrets.jwtSecretRef }}
-13
View File
@@ -1,13 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
spec:
selector:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
ports:
- protocol: TCP
port: 80
targetPort: http
name: http
type: ClusterIP
-201
View File
@@ -1,201 +0,0 @@
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
serviceName: {{ .Release.Name }}-{{ .Chart.Name }}-service
replicas: {{ include "convertx.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.convertx.replicas) }}
selector:
matchLabels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
template:
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
annotations:
# Hash of the rendered OnePasswordItem, so that repointing
# secrets.jwtSecretRef at a DIFFERENT vault item really rolls the pod
# instead of leaving the old signing key in the running environment.
# A change of the item's CONTENT inside the vault is not covered -
# Helm never sees it; see the comment on secrets in values.yaml.
# The name differs from checksum/config on purpose: that one promises
# "the config changed", this one only "the config source changed".
#
# operator.1password.io/auto-restart is deliberately NOT set: the
# 1Password operator only restarts Deployments, so on a StatefulSet it
# is a silent no-op that advertises a guarantee it does not give.
checksum/secret-ref: {{ include (print $.Template.BasePath "/convertx.secret.yaml") . | sha256sum }}
spec:
automountServiceAccountToken: false
securityContext:
{{- include "convertx.podSecurityContext" .Values.convertx.securityContext | nindent 8 }}
containers:
- name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
image: "ghcr.io/c4illin/convertx:{{ .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
# THE IMAGE ENTRYPOINT IS BYPASSED ON PURPOSE. READ THIS BEFORE THE
# NEXT APP-VERSION BUMP - IT IS THE ONE BET THIS CHART MAKES.
#
# ghcr.io/c4illin/convertx declares
# ENTRYPOINT ["/entrypoint.sh"]
# CMD ["bun", "run", "dist/src/index.js"]
# and carries no USER, so the image runs as root by default and
# /entrypoint.sh is what drops it to uid 1000 with gosu. Setting
# "command" here replaces the ENTRYPOINT (and discards the CMD), so
# the application is started directly and Kubernetes does the
# privilege work instead: runAsUser/runAsGroup 1000 in the pod
# security context, and fsGroup 1000 in place of the entrypoint's
# recursive chown of /app/data. The hardening baseline of this repo
# stays intact - no root, no capabilities, read-only root fs.
#
# What the entrypoint of v0.19.0 actually does, read rather than
# assumed: nothing but uid/gid plumbing. No migration, no config
# generation, no secret generation. ConvertX creates and migrates
# its SQLite schema in-process at import time, so starting it
# directly is migration-safe. Its very first branch is
# if [ -z "$PUID" ] || [ "$PUID" -eq 0 ]; then ... exec "$@"
# i.e. with PUID unset it is already a pure pass-through - which is
# why bypassing it costs nothing TODAY and is exactly why the risk
# below is about tomorrow. The one line we do lose is
# "export HOME=/home/convertx", and this chart sets HOME itself
# below rather than relying on the runtime's passwd lookup.
#
# THE RISK: if upstream ever moves real work into the entrypoint - a
# data migration is the obvious candidate - this chart skips it
# silently. There is no error, no log line and no failed probe; the
# application simply starts against an unmigrated volume. So
# RE-READ entrypoint.sh AT EVERY APP-VERSION UPDATE, not only on
# major jumps, and if it has grown anything beyond uid/gid handling,
# stop and reconsider this override.
command:
- bun
- run
- dist/src/index.js
# Not decoration. The application resolves its data paths relative
# to the working directory - uploadsDir is "./data/uploads/" and
# outputDir is "./data/output/", hardcoded and configurable by no
# environment variable. The image sets WORKDIR /app, so this only
# restates it; it is here because a different working directory
# would put every upload somewhere other than the mounted PVC, and
# the application would report no error at all.
workingDir: /app
env:
# The entrypoint would have exported this; the image declares no
# ENV HOME, so without this line HOME comes from the container
# runtime's /etc/passwd lookup for uid 1000. That happens to
# resolve to /home/convertx in this image - and "happens to" is
# not a basis for the directory LibreOffice, Calibre, Inkscape and
# fontconfig write their profiles into on the first conversion.
# Set explicitly, and mounted writable below.
- name: HOME
value: "/home/convertx"
# The port the application binds. containerPort, the Service's
# targetPort and the three probes all name the same 3000; moving
# it means moving all four, which is why it is pinned here rather
# than exposed as a value.
- name: PORT
value: "3000"
# Signs the session cookies. Without it ConvertX falls back to
# randomUUID() per process, so every restart logs everyone out -
# see secrets.jwtSecretRef in values.yaml.
- name: JWT_SECRET
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-secret
key: jwt-secret
# The three booleans go through convertx.boolString, which renders
# the literal "true"/"false" the application's strict comparison
# needs and aborts the render on anything else.
- name: ACCOUNT_REGISTRATION
value: "{{ include "convertx.boolString" (dict "value" .Values.convertx.accountRegistration "default" false "path" "convertx.accountRegistration") }}"
- name: ALLOW_UNAUTHENTICATED
value: "{{ include "convertx.boolString" (dict "value" .Values.convertx.allowUnauthenticated "default" false "path" "convertx.allowUnauthenticated") }}"
- name: HTTP_ALLOWED
value: "{{ include "convertx.boolString" (dict "value" .Values.convertx.httpAllowed "default" false "path" "convertx.httpAllowed") }}"
- name: AUTO_DELETE_EVERY_N_HOURS
value: "{{ include "convertx.autoDeleteEveryNHours" .Values.convertx.autoDeleteEveryNHours }}"
{{- range $value := .Values.convertx.env }}
{{- if $value.value }}
- name: {{ $value.name | quote }}
# quote is required, not cosmetic: a value like "8080" or "true"
# renders unquoted as a YAML number/bool, and the API server then
# rejects the whole manifest.
value: {{ tpl $value.value $ | quote }}
{{- else if $value.valueFrom }}
{{- if $value.valueFrom.secretKeyRef }}
- name: {{ $value.name }}
valueFrom:
secretKeyRef:
name: {{ tpl $value.valueFrom.secretKeyRef.name $ | quote }}
key: {{ tpl $value.valueFrom.secretKeyRef.key $ | quote }}
{{- else if $value.valueFrom.configMapKeyRef }}
- name: {{ $value.name }}
valueFrom:
configMapKeyRef:
name: {{ tpl $value.valueFrom.configMapKeyRef.name $ | quote }}
key: {{ tpl $value.valueFrom.configMapKeyRef.key $ | quote }}
{{- end }}
{{- end }}
{{- end }}
ports:
- containerPort: 3000
protocol: TCP
name: http
# Probes, resources and both security contexts go through the
# null-safe helpers (#99): an erased block means "chart defaults
# apply" instead of a vanished probe or an unhardened container.
# Never wrap these in an "if" - a guard hides the erased case
# completely instead of rendering a visible null.
livenessProbe:
{{- include "convertx.livenessProbe" .Values.convertx.livenessProbe | nindent 12 }}
readinessProbe:
{{- include "convertx.readinessProbe" .Values.convertx.readinessProbe | nindent 12 }}
startupProbe:
{{- include "convertx.startupProbe" .Values.convertx.startupProbe | nindent 12 }}
resources:
{{- include "convertx.effectiveResources" .Values.convertx.resources | nindent 12 }}
securityContext:
{{- include "convertx.containerSecurityContext" .Values.convertx.securityContext | nindent 12 }}
volumeMounts:
# SQLite database, uploads and converted output - everything the
# application persists lives under this one path.
- mountPath: /app/data
name: data
# Two writable emptyDirs so the root filesystem can stay
# read-only. Neither is a precaution: ConvertX itself writes only
# under /app/data, but it runs its conversions as subprocesses
# (soffice, ebook-convert, magick, gs, xelatex, ffmpeg...) that
# inherit its environment, and those need scratch space.
#
# /home/convertx is HOME. LibreOffice creates its user profile
# there on first use - it is invoked without
# -env:UserInstallation, so there is no way to point it elsewhere
# - and Calibre, Inkscape, ImageMagick and fontconfig do the same
# with their own config and cache directories. The directory
# exists in the image (useradd -m) and is owned by 1000:1000, but
# it sits on the read-only root filesystem, so it has to be
# overmounted to be usable.
- mountPath: /home/convertx
name: home
# /tmp is the scratch directory of the toolchain: Ghostscript,
# TeX Live and Calibre all use it during a conversion.
#
# Both of these count against the container's ephemeral-storage
# limit - see the resources comment in values.yaml.
- mountPath: /tmp
name: tmp
restartPolicy: Always
volumes:
- name: data
persistentVolumeClaim:
claimName: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
- name: home
emptyDir: {}
- name: tmp
emptyDir: {}
-212
View File
@@ -1,212 +0,0 @@
{
"$schema": "https://json-schema.org/draft-07/schema#",
"title": "convertx values",
"description": "Strict values schema (issue #68). additionalProperties is false on every chart-owned object level, so a key that the chart does not read is rejected at render time instead of being silently ignored. The only objects that stay open are the ones passed verbatim into the Kubernetes pod spec (probes, security contexts, global) and the free-form Ingress annotation map - the chart does not interpret their contents, so it cannot judge which keys are meaningful. The hardening, probe and resources blocks additionally accept null, because an erased block falls back to the chart defaults (issue #99).",
"type": "object",
"additionalProperties": false,
"properties": {
"global": {
"description": "Helm's conventional cross-chart values. Unused by this chart, kept open so an umbrella install cannot break on it.",
"type": "object"
},
"scaleDown": {
"description": "true renders the StatefulSet with replicas: 0 and overrides convertx.replicas. PVC, Secret, Service and Ingress stay in place.",
"type": "boolean"
},
"secrets": {
"type": "object",
"additionalProperties": false,
"properties": {
"jwtSecretRef": {
"description": "REQUIRED. op:// path of the 1Password item carrying the field 'jwt-secret', which becomes JWT_SECRET. Enforced by required() in the template rather than by a 'required' array here: the key ships as null, so JSON Schema's required - which tests presence, not null-ness - could never fire on it.",
"type": ["string", "null"]
}
}
},
"convertx": {
"type": "object",
"additionalProperties": false,
"properties": {
"replicas": {
"description": "0 is allowed and scales the workload down. Values above 1 are accepted by the schema but do not work: the SQLite database and the uploads share one ReadWriteOnce PVC.",
"type": "integer",
"minimum": 0
},
"accountRegistration": {
"description": "ACCOUNT_REGISTRATION - lets anyone with the URL create an account. Defaults to false; upstream warns against enabling it on an internet-facing instance. A string is accepted for the 'true'/'false' spellings only; anything else aborts the render with a message naming this path, because the application would silently read it as false.",
"type": ["boolean", "string", "null"]
},
"allowUnauthenticated": {
"description": "ALLOW_UNAUTHENTICATED - drops authentication entirely. Defaults to false; upstream describes this as local-only. Same accepted spellings as accountRegistration.",
"type": ["boolean", "string", "null"]
},
"httpAllowed": {
"description": "HTTP_ALLOWED - clears the 'secure' flag on the session cookies. Only needed when ConvertX is served without TLS in front; behind this chart's Ingress the browser speaks https, so the default false is correct. Same accepted spellings as accountRegistration.",
"type": ["boolean", "string", "null"]
},
"autoDeleteEveryNHours": {
"description": "AUTO_DELETE_EVERY_N_HOURS - how old a job may get before ConvertX deletes its uploads and output. 0 disables the cleanup and is honoured as such (the chart uses a nil check, not Helm's 'default', which would swallow the 0). This is the retention policy for the PVC.",
"type": ["integer", "number", "null"],
"minimum": 0
},
"securityContext": {
"$ref": "#/definitions/securityContext"
},
"livenessProbe": {
"$ref": "#/definitions/probe"
},
"readinessProbe": {
"$ref": "#/definitions/probe"
},
"startupProbe": {
"$ref": "#/definitions/probe"
},
"env": {
"$ref": "#/definitions/env"
},
"resources": {
"$ref": "#/definitions/resources"
}
}
},
"storage": {
"description": "The data PVC: SQLite database, uploads and converted output. Not null-safe on purpose - an erased block yields a PVC without a size, which the API server rejects at apply time rather than the chart sizing the volume silently.",
"type": "object",
"additionalProperties": false,
"properties": {
"amount": {
"type": ["string", "null"]
},
"storageClass": {
"type": ["string", "null"]
}
}
},
"ingress": {
"type": "object",
"additionalProperties": false,
"properties": {
"enabled": {
"description": "Gates the whole Ingress template. false frees BOTH clusterIssuer and domain: nothing outside the Ingress reads the domain in this chart, because ConvertX has no base-URL or redirect-URI variable.",
"type": "boolean"
},
"maxUploadFileSize": {
"description": "Rendered into the ingress-nginx proxy-body-size annotation, so it only takes effect while the Ingress is enabled. It is the only upload ceiling that exists - ConvertX enforces none of its own.",
"type": ["string", "null"]
},
"clusterIssuer": {
"description": "REQUIRED while ingress.enabled is true (enforced by required() in the template).",
"type": ["string", "null"]
},
"domain": {
"description": "REQUIRED while ingress.enabled is true (enforced by required() in the template).",
"type": ["string", "null"]
},
"className": {
"type": ["string", "null"]
},
"annotations": {
"description": "Extra Ingress annotations, merged with the cert-manager and proxy-body-size annotations the chart always sets. Free-form keys by nature.",
"type": "object"
}
}
}
},
"definitions": {
"resources": {
"description": "May be null / empty: the chart then falls back to its default requests and computed limits (issue #99) instead of rendering a container without resource accounting.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"limitFactor": {
"type": "number",
"exclusiveMinimum": 0
},
"requests": {
"$ref": "#/definitions/resourceList"
},
"limits": {
"$ref": "#/definitions/resourceList"
}
}
},
"resourceList": {
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"cpu": {
"type": ["string", "number", "null"]
},
"memory": {
"type": ["string", "number", "null"]
},
"ephemeral-storage": {
"type": ["string", "number", "null"]
},
"ephemeralStorage": {
"type": ["string", "number", "null"]
}
}
},
"env": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name"],
"properties": {
"name": {
"type": "string"
},
"value": {
"type": "string"
},
"valueFrom": {
"type": "object",
"additionalProperties": false,
"properties": {
"secretKeyRef": {
"$ref": "#/definitions/keyRef"
},
"configMapKeyRef": {
"$ref": "#/definitions/keyRef"
}
}
}
}
}
},
"keyRef": {
"type": "object",
"additionalProperties": false,
"required": ["name", "key"],
"properties": {
"name": {
"type": "string"
},
"key": {
"type": "string"
}
}
},
"securityContext": {
"description": "May be null / empty at any level: the chart then falls back to its hardening defaults (issue #99) instead of rendering an unhardened workload. Those defaults pin runAsUser/runAsGroup 1000 - the uid the image creates and the uid that owns the data on the PersistentVolume - rather than the repo's arbitrary 20000.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"pod": {
"description": "Merged onto the chart's pod hardening defaults and rendered as the pod's securityContext; kept open because the chart does not interpret the individual fields.",
"type": ["object", "null"]
},
"container": {
"description": "Merged onto the chart's container hardening defaults and rendered as the container's securityContext; kept open because the chart does not interpret the individual fields.",
"type": ["object", "null"]
}
}
},
"probe": {
"description": "Merged onto the chart's probe defaults and rendered as a container probe; kept open because the chart does not interpret the individual fields. May be null / empty, which means the defaults apply (issue #99) rather than the probe disappearing. The defaults target /healthcheck on the port named http - overriding the path to '/' would probe a 302 whose target changes once the first account exists.",
"type": ["object", "null"]
}
}
}
-307
View File
@@ -1,307 +0,0 @@
# Scale the whole chart down without uninstalling it: scaleDown: true renders
# every Deployment/StatefulSet with replicas: 0 (and suspends CronJobs) and
# takes precedence over any per-workload replicas value. PVC, Secret, Service
# and Ingress stay in place; scaleDown: false brings the workload back up.
scaleDown: false
secrets:
# op:// item path for the JWT signing key. REQUIRED - the chart aborts the
# render without it, and that is deliberate rather than convenient.
#
# ConvertX signs its session cookies with JWT_SECRET and falls back to
# randomUUID() when the variable is unset (src/services/user.ts:
# "secret: process.env.JWT_SECRET ?? randomUUID()"). That fallback is
# per PROCESS, not per install: every restart - a node drain, an image
# update, an OOM kill - mints a fresh secret and silently invalidates every
# session. Nothing logs it; users simply find themselves logged out. A value
# that has to survive a restart is a secret, so it is one here.
#
# The item must carry this field, spelled exactly like this, because the
# 1Password operator copies an item's FIELD NAMES into the Secret's keys:
#
# jwt-secret a long random string (openssl rand -hex 32)
#
# A field spelled differently does not fail the install: the operator writes
# a Secret without that key and the pod stays in CreateContainerConfigError,
# which looks like a broken chart rather than a typo in a vault item.
jwtSecretRef: null
convertx:
# Replica count for this workload; 0 is allowed and scales it down.
# The chart-global scaleDown flag takes precedence.
#
# Raising this above 1 does not work and the chart does not stop you, so the
# reason is here: the SQLite database, the uploads and the converted output
# all live on the single ReadWriteOnce PVC below, which is a plain chart
# object rather than a volumeClaimTemplate. A second pod normally never
# schedules ("Multi-Attach error for volume"); on a genuinely RWX storage
# class it would schedule and then corrupt the WAL-mode SQLite file that two
# processes are now writing. On top of that each pod would sign cookies with
# its own JWT_SECRET only if the secret were unset - which is exactly why
# secrets.jwtSecretRef above is required.
replicas: 1
# --- Two switches that open the instance up. Both default to false. ---
#
# UPSTREAM WARNS EXPLICITLY AGAINST ENABLING EITHER ON AN INTERNET-FACING
# INSTANCE, and this chart puts an Ingress with a public DNS name in front
# by default, so that warning applies to the standard install here.
#
# Both are parsed strictly by the application: it compares the lowercased
# string against "true" and treats everything else as false. The chart
# therefore accepts only a boolean or the strings "true"/"false" (any case)
# and ABORTS THE RENDER on anything else - "yes" or "1" would otherwise be
# filed under "off" without a word, which is the wrong direction to fail in
# for a value whose whole job is to open the service up.
# Lets anyone with the URL create an account. Note what it does NOT gate:
# the FIRST account is always creatable, because a ConvertX with no users
# serves a setup form to whoever opens the page first. Keeping this false
# protects accounts two and onwards, not account one.
accountRegistration: false
# Drops authentication entirely - anyone reaching the Service can upload
# files and read every conversion. Upstream describes this as local-only.
allowUnauthenticated: false
# Clears the "secure" flag on the session cookies (the app computes
# "secure: !HTTP_ALLOWED"). Only needed when ConvertX is served over plain
# http, and that is NOT our case: the Ingress terminates TLS, so the browser
# speaks https and sets the cookie happily while only the in-cluster hop
# between ingress controller and pod is http - which the browser never sees.
# Turning this on behind our Ingress weakens the cookies for no gain.
# (Same shape as outline's forceHttps, and the same reason to leave it off.)
httpAllowed: false
# How often ConvertX deletes jobs - uploads and converted output - older
# than this many hours. 0 disables the cleanup entirely.
#
# This is the retention policy for the PVC below, which is why it is a value
# and not left at the upstream default. With cleanup disabled the volume
# grows with every conversion and nothing reclaims it; size storage.amount
# for that case, or accept that the pod eventually fills its volume.
#
# An explicit 0 is honoured (the chart uses a nil check, not Helm's
# "default", which would swallow the 0 and silently restore 24-hour
# deletion on a value set precisely to switch it off).
autoDeleteEveryNHours: 24
# DOCUMENTED DEVIATION FROM THE HARDENING BASELINE - runAsUser/runAsGroup.
#
# The deviation is about the NUMBER only: nothing in the baseline is
# relaxed. The repo default is an arbitrary uid (20000); this chart pins
# 1000, and the reason is the image, not a preference.
#
# ghcr.io/c4illin/convertx carries NO USER instruction - verified in the
# published v0.19.0 image config, where "User" is empty - so without a
# securityContext the pod would run as root. Its Dockerfile creates
# "convertx" with uid 1000, gid 1000 and home /home/convertx (useradd -u
# 1000 -g convertx -d /home/convertx -m), and the image's entrypoint exists
# only to drop to that user with gosu.
#
# 1000 rather than 20000 for two reasons that both have to hold:
# - /home/convertx belongs to 1000:1000 in the image, and HOME is where
# LibreOffice, Calibre and fontconfig put their profiles. An arbitrary
# uid gets a HOME of "/" from the runtime's passwd lookup instead.
# - the uid owns data on a PersistentVolume, so changing it later makes
# existing conversions inaccessible. This number is not free to tidy.
securityContext:
pod:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
# fsGroup is what replaces the entrypoint's recursive chown of
# /app/data: the kubelet applies the group to the mounted volume, so
# the container can write it as uid 1000 without any privileged step.
fsGroup: 1000
fsGroupChangePolicy: Always
# RuntimeDefault applies the container runtime's default seccomp filter,
# which blocks the syscalls a normal workload never makes. It sits in the
# pod context on purpose: from here it covers every container in the pod
# including init containers, while a container-level seccompProfile would
# have to be repeated for each one (and silently missed on the next one
# added). A container that genuinely needs a different filter can still
# override it under securityContext.container, which wins over this value.
seccompProfile:
type: RuntimeDefault
container:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
add: []
# ConvertX serves GET /healthcheck: a static 200 with {"status":"ok"},
# explicitly exempted from authentication (src/pages/healthcheck.tsx,
# "auth: false"). All three probes use it.
#
# Why NOT "/", the obvious default: with the chart defaults "/" is never a
# 200. It answers 302 to /setup while no user exists and 302 to /login
# afterwards, so a probe against it would pass on a redirect whose target
# changes the moment somebody registers - a green signal that means
# something different on day two than on day one. httpGet does count 3xx as
# success, which is what makes that the quiet kind of wrong.
#
# What the endpoint proves, and what it does not: the handler touches
# neither the database nor the volume, so it cannot report a broken PVC.
# It is still the right readiness target here, because the SQLite database
# is opened at module import - before app.listen() is reached - so a process
# that answers /healthcheck at all has already opened its database. There is
# no external dependency in this chart to hang readiness on; the "database"
# is a file on the pod's own volume.
#
# WEBROOT SHIFTS THIS PATH. The application mounts every route under the
# WEBROOT prefix, /healthcheck included. This chart does not offer WEBROOT
# as a value (see the env block below), and setting it through env without
# also moving these three paths leaves the pod probing a 404 forever.
livenessProbe:
httpGet:
path: /healthcheck
port: http
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
readinessProbe:
httpGet:
path: /healthcheck
port: http
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# The startup probe gates liveness/readiness, so those need no
# initialDelaySeconds. Budget = periodSeconds * failureThreshold = 300s.
#
# Generous on purpose. At import time - under NODE_ENV=production, which
# the image sets - ConvertX shells out to roughly twenty converter binaries
# to log their versions (soffice, ebook-convert, xelatex, magick, ffmpeg,
# markitdown and the rest). Those are fire-and-forget and do not block
# app.listen(), but they all contend for the pod's CPU at exactly the moment
# the first probe arrives, and the first soffice/ebook-convert run also
# builds its profile under HOME. A tight budget turns a slow node into a
# crash loop.
startupProbe:
httpGet:
path: /healthcheck
port: http
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 30
# Extra environment variables, appended to the ones the chart always sets.
# Values are rendered through tpl; value, secretKeyRef and configMapKeyRef
# forms are supported.
#
# This is where the upstream variables this chart does not expose as values
# belong - BRANDING, LANGUAGE, HIDE_HISTORY, MAX_CONVERT_PROCESS,
# UNAUTHENTICATED_USER_SHARING, TZ, FFMPEG_ARGS, FFMPEG_OUTPUT_ARGS. They
# change the UI or a converter's behaviour and nothing this chart renders,
# so they need no value of their own.
#
# THREE VARIABLES DO NOT BELONG HERE, and they fail in three different ways:
#
# WEBROOT moves EVERY route, /healthcheck included, so the three
# probes above start failing against a 404 while the app is
# perfectly healthy - and this chart's Ingress still maps "/".
# Serving under a sub-path needs the probe paths and the
# Ingress path changed together; the chart does not do that
# for you.
# PUID/PGID are read by the image entrypoint, which this chart bypasses
# /UMASK (see the command: override in the StatefulSet). They reach
# nothing and configure nothing here. Set the uid through
# securityContext.pod above, which is what actually applies.
# PORT the app would listen on it, but containerPort, the Service
# and the probes all still name 3000. Moving the port means
# moving all four, so the chart pins it instead.
#
# Additive means additive: an entry repeating a name the chart already sets
# does not replace it. At install the later entry - yours - wins; on an
# EXISTING release Kubernetes merges env by name, the pair collapses to the
# chart's own value, and helm upgrade still reports success. Read the value
# back off the StatefulSet rather than trusting a green upgrade.
env: []
resources:
# Limits are optional: memory and ephemeral-storage limits default to
# limitFactor x the request, an explicitly set limit always wins, and a
# CPU limit is only set when configured explicitly under "limits".
#
# The numbers are sized for conversions, not for an idle web server, and
# each one has a reason:
#
# cpu 0.25 An idle Bun process uses almost nothing, but a
# conversion saturates whatever it can get and the startup
# fires ~20 version probes at once. Per repo convention no
# CPU LIMIT is computed, so this is the scheduling floor
# and the throttling share under contention, not a cap -
# a conversion may burst well past it on an idle node.
#
# memory 1Gi soffice --headless is a few hundred MB on its own,
# ebook-convert pulls in Qt WebEngine, and ImageMagick on
# a large raster can want more than either. 1Gi requested,
# 3Gi limit (limitFactor 3). Memory is NOT compressible:
# a conversion that crosses the limit is an OOM kill of
# the whole pod, so raise the request before feeding the
# instance large video or raw-image work.
#
# ephemeral- 1Gi, and this one is easy to get wrong: EMPTYDIR VOLUMES
# storage COUNT AGAINST IT. The chart mounts two (/tmp and
# /home/convertx) for the converter toolchain, and
# Ghostscript, TeX and Calibre all scratch to /tmp. The
# uploads and the converted output do NOT count - those
# live on the PVC - so the figure covers scratch space
# only. Crossing the 3Gi limit evicts the pod.
limitFactor: 3
requests:
cpu: 0.25
memory: 1Gi
ephemeralStorage: 1Gi
# The volume behind /app/data: the SQLite database, every uploaded file and
# every converted result. There is no second store and no object storage.
#
# Not null-safe on purpose, following the repo's documented exception: an
# erased block yields a PVC without a size, which the API server rejects at
# apply time. That is louder than a volume quietly sized by the chart.
storage:
amount: 10Gi
storageClass: null
ingress:
# Gates the whole Ingress template. Checked in both directions as the style
# guide requires: NOTHING outside the Ingress reads ingress.domain in this
# chart - ConvertX has no base-URL, public-URL or redirect-URI variable at
# all (verified against the complete env list in src/helpers/env.ts) - so
# switching the Ingress off frees BOTH domain and clusterIssuer here.
enabled: true
# THE ONLY UPLOAD CEILING THERE IS.
#
# ConvertX sets no size limit of its own: it starts Elysia with
# maxRequestBodySize: Number.MAX_SAFE_INTEGER, and v0.19.0 has no upload
# variable of any kind. So unlike outline - where an app limit and an
# ingress limit sat a factor of ten apart and the smaller one won
# confusingly - there is exactly one number here, and it is this one.
#
# Two consequences worth knowing before changing it:
# - A file above this limit never reaches ConvertX. The user gets the
# ingress controller's 413 page, not a message from the application,
# and nothing appears in the pod log.
# - With ingress.enabled: false there is NO ceiling from this chart at
# all, and whatever proxy you put in front brings its own - usually
# ingress-nginx's 1m default, which is small enough to look like a
# broken upload button.
#
# Defaulted generously because the point of the application is converting
# video and large documents. What it costs: the body transits the ingress
# controller and then lands on the PVC, so raising it means checking
# storage.amount as well.
maxUploadFileSize: 1024m
clusterIssuer: null
domain: null
className: nginx
# Extra annotations for the Ingress; merged with the cert-manager
# cluster-issuer and proxy-body-size annotations the chart always sets.
annotations: {}
+23
View File
@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
+26
View File
@@ -0,0 +1,26 @@
apiVersion: v2
name: garage
description: A Helm chart to deploy Garage, an S3-compatible object store, as a single node on Kubernetes
icon: https://garagehq.deuxfleurs.fr/images/garage-logo.svg
type: application
# appVersion is the exact image tag, and the "v" is part of it - so it is part
# of appVersion and of the "+up" suffix. Precedent: stalwart (+upv0.16.24),
# calibre-web-automated, matrix-synapse, ytdlp-web-player.
#
# The image is docker.io/dxflrs/garage, the MULTI-ARCH one. Upstream's own chart
# pins dxflrs/amd64_garage instead; that is a per-architecture image and pinning
# it would tie the chart to amd64 for no gain.
#
# THE TAG TRAP THIS VERSION HAS TO SURVIVE: next to the release tags the
# registry continuously receives COMMIT-SHA TAGS (40 hex characters, the newest
# of them dated 2026-10-03). A Renovate rule without a version constraint would
# offer those as "updates", which is why the garage block in renovate.json5
# carries a versioningTemplate anchored to ^v<major>.<minor>.<patch>$ - same
# shape as the stalwart block, for the same reason.
version: 1.0.0+upv2.4.1
appVersion: "v2.4.1"
maintainers:
- name: jklein
+144
View File
@@ -0,0 +1,144 @@
{{- $settings := fromYaml (include "garage.settings" .) -}}
{{- $pod := printf "%s-%s-sfs-0" .Release.Name .Chart.Name -}}
{{- $svc := printf "%s-%s-service" .Release.Name .Chart.Name -}}
Garage {{ .Chart.AppVersion }} installed as release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
ONE node, replication_factor = {{ $settings.replicationFactor | int64 }}, metadata engine {{ $settings.dbEngine }}.
S3 ENDPOINT (in-cluster, path-style addressing):
http://{{ $svc }}.{{ .Release.Namespace }}.svc.cluster.local:{{ $settings.ports.s3 | int64 }}
region: {{ $settings.s3.region }}
{{- if .Values.ingress.enabled }}
published at: https://{{ .Values.ingress.domain }}
{{- end }}
{{- if $settings.web.enabled }}
static website endpoint on port {{ $settings.ports.web | int64 }}{{ with $settings.web.rootDomain }} (root domain {{ . }}){{ else }} - WITHOUT a rootDomain it serves nothing; set garage.web.rootDomain{{ end }}
{{- end }}
THE 1PASSWORD ITEMS, AND WHICH FIELDS THEY MUST CARRY
The operator materializes one Secret key per item FIELD, so a missing or
misnamed field leaves the pod in CreateContainerConfigError.
{{ .Values.secrets.rpcSecretRef }}
rpc-secret 32 bytes of random hex: openssl rand -hex 32
{{ if .Values.secrets.s3SecretRef }} {{ .Values.secrets.s3SecretRef }}
access-key >= 8 chars, [A-Za-z0-9-_.]: openssl rand -hex 12
secret-key >= 16 printable ASCII chars: openssl rand -hex 32
bucket a valid S3 bucket name (3-63 chars, lowercase)
{{ else }} (no S3 key item configured - secrets.s3SecretRef is unset)
{{ end }}
{{- if and $settings.singleNode (or $settings.defaultAccessKey $settings.defaultBucket) }}
THE DIRECTION IS INWARDS: Garage IMPORTS{{ if $settings.defaultAccessKey }} the access key{{ end }}{{ if $settings.defaultBucket }}{{ if $settings.defaultAccessKey }} and{{ end }} the bucket{{ end }} from
that item on start. The vault is the source of truth and nothing has to be
copied out of the server.
{{- end }}
FIRST START
{{ if $settings.singleNode }}
Nothing to do by hand. The server created its own cluster layout before it
opened any listener{{ if $settings.defaultAccessKey }}, imported the access key{{ end }}{{ if $settings.defaultBucket }} and created the bucket{{ end }}.
Verify it, in this order - all commands need /garage as the command, because
the image has no shell:
1. kubectl -n {{ .Release.Namespace }} rollout status statefulset/{{ .Release.Name }}-{{ .Chart.Name }}-sfs
Ready means the admin API answered 200 on /health, which means a layout
exists and the node has quorum. Not-Ready with a running container means
the layout is missing - check the logs for "Created initial layout".
2. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage status
One node, with a role (zone dc1, a capacity taken from the detected disk
size). A node WITHOUT a role means no layout was applied.
3. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage layout show
{{- if $settings.defaultBucket }}
4. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage bucket list
kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage bucket info <bucket>
The bucket exists and the imported key has read/write/owner on it.
{{- end }}
5. One write through the S3 API from a client pod, which is the only step
that proves the endpoint rather than the process.
{{ else }}
garage.singleNode is false, so THIS RELEASE SERVES NOTHING YET and the pod
will stay not-Ready until a layout exists - that is correct, not a defect: the
readiness probe reports /health, and /health answers 503 while there is no
layout. Apply one by hand, in this order:
1. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage status
Note the node id and the current layout version (0 on a fresh install).
2. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- \
/garage layout assign -z dc1 -c <capacity> <node-id>
The capacity is this node's usable size, roughly storage.data.amount
({{ .Values.storage.data.amount }}). It takes BYTE suffixes B, KB, MB, GB, TB, PB - not
Kubernetes' Gi/Mi, so write e.g. 10GB, not 10Gi.
3. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage layout show
Read it before applying. The next step is NOT reversible by repeating it.
4. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage layout apply --version <N+1>
N is the version from step 1, so a fresh cluster takes --version 1.
NEVER call this twice with the same --version: the flag is a guard
against applying a staged change you have not seen, and reusing a number
is how a layout is applied twice.
5. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage bucket create <bucket>
kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- \
/garage key import --yes <access-key> <secret-key>
kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- \
/garage bucket allow --read --write --owner --key <access-key> <bucket>
"key import" rather than "key create", so the credential still comes FROM
the vault item. "key create" generates one and then it has to travel the
other way, through a terminal, with the vault as a copy.
6. The pod becomes Ready within one probe period after step 4.
{{ end }}
WHAT THIS CHART DOES NOT RENDER, AND WHAT IS MISSING WITHOUT IT
- THE LONGHORN BACKUP LABELS. recurring-job.longhorn.io/daily-snapshot and
.../daily-backup belong to the consuming bundle, not to this chart. They do
NOT apply automatically: without them the off-site half of the backup plan
does not exist, it only looks like it does.
- The namespace's volume-encryption secret, if the StorageClass is
longhorn-encrypted-ns: it resolves
${pvc.namespace}-pvc-encryption-secret, and without that entry the PVCs
stay Pending.
- No kubernetes_discovery CRD and no bootstrap peers: a single node has no
members to find.
TRAPS WORTH KNOWING BEFORE YOU NEED THEM
- THE METADATA IS NOT IN YOUR VOLUME BACKUP in any usable form. A
file-system snapshot of a live metadata database is not a restore point -
taken during a write it is corrupt. What IS a restore point are Garage's
own snapshots, on the second PVC, taken every {{ $settings.metadataAutoSnapshotInterval }}.
- THE WINDOW IS SHORT: exactly TWO snapshots are kept, the first after half
the interval and every later one after the interval times a random
1.0-1.2. With metadata_auto_snapshot_interval = {{ $settings.metadataAutoSnapshotInterval }} the oldest
recovery point is therefore between one and two and a half intervals old,
never more. A corruption found later than that is older than every
snapshot that still exists.
- metadata_fsync is ON here and OFF in Garage's defaults. If you ever see it
missing from garage.toml, that is a regression, not a cleanup.
- replication_factor 1 applies to the METADATA as well. Upstream calls a
corrupted metadata database "generally not recoverable", and the volume
layer replicates a corrupted file as faithfully as a good one.
- ROTATING THE S3 SECRET IS NOT A RE-SAVE. Garage refuses to start when the
access key already exists with a different secret ("is associated with a
secret key different than the one given in GARAGE_DEFAULT_SECRET_KEY"). To
rotate: either put a NEW access-key id into the item (the old key stays and
can be deleted with "/garage key delete"), or delete the old key first and
then update both fields.
- A CHANGE INSIDE A VAULT ITEM RESTARTS NOTHING. Helm never sees it. After
changing a field:
kubectl -n {{ .Release.Namespace }} rollout restart statefulset/{{ .Release.Name }}-{{ .Chart.Name }}-sfs
(Repointing a secrets.*SecretRef at a different item DOES roll the pod -
that is covered by checksum/secret-ref.)
- THE IMAGE HAS NO SHELL. "kubectl exec -it ... -- sh" does not work and
never will; every command above uses /garage directly. Exec probes are
impossible in this chart for the same reason.
- THE RPC SECRET IS IN THE PROCESS ENVIRONMENT, not in a file. That is
forced, not chosen: Garage rejects a secret file with any group bit set,
and the kubelet sets one. values.yaml has the mechanism.
+392
View File
@@ -0,0 +1,392 @@
{{/*
Render the body of a container "resources:" block from a values dict of the form
resources:
limitFactor: 3 # optional, default 3
requests: { cpu, memory, ephemeral-storage / ephemeralStorage, ... }
limits: { ... } # optional, every entry optional
Rules (variant B, see issue #19):
- requests are rendered as given (camelCase "ephemeralStorage" is normalized
to the Kubernetes key "ephemeral-storage").
- an explicitly set limit always wins.
- a missing memory / ephemeral-storage limit is computed as
limitFactor x the corresponding request.
- a missing cpu limit is NOT set at all (the kubelet then treats CPU as
unlimited; CPU is compressible, so this cannot kill the workload).
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart, and a dependency is admissible but expensive (README, guideline 14) -
this chart takes none. The reference implementation is
template-chart/templates/_helpers.tpl.
*/}}
{{- define "garage.resources" -}}
{{- $resources := . | default dict -}}
{{- $factor := 3.0 -}}
{{- if not (kindIs "invalid" $resources.limitFactor) -}}
{{- $factor = float64 $resources.limitFactor -}}
{{- end -}}
{{- $requests := dict -}}
{{- range $key, $value := ($resources.requests | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $requests (include "garage.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- $limits := dict -}}
{{- range $key, $value := ($resources.limits | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $limits (include "garage.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- range $key, $request := $requests -}}
{{- if and (ne $key "cpu") (not (hasKey $limits $key)) -}}
{{- $_ := set $limits $key (include "garage.multiplyQuantity" (dict "quantity" $request "factor" $factor)) -}}
{{- end -}}
{{- end -}}
{{- $out := dict -}}
{{- with $requests }}{{- $_ := set $out "requests" . }}{{- end -}}
{{- with $limits }}{{- $_ := set $out "limits" . }}{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Normalize a resource name from values to its Kubernetes key.
*/}}
{{- define "garage.resourceKey" -}}
{{- if eq . "ephemeralStorage" -}}ephemeral-storage{{- else -}}{{ . }}{{- end -}}
{{- end -}}
{{/*
Multiply a Kubernetes quantity by a factor, keeping its suffix.
Input: dict "quantity" <string|number> "factor" <float64>.
*/}}
{{- define "garage.multiplyQuantity" -}}
{{- $quantity := include "garage.formatNumber" .quantity -}}
{{- $number := regexFind "^[0-9]+(\\.[0-9]+)?" $quantity -}}
{{- if not $number -}}
{{- fail (printf "cannot parse resource quantity %q" $quantity) -}}
{{- end -}}
{{- $suffix := trimPrefix $number $quantity -}}
{{- if not (regexMatch "^(m|k|M|G|T|P|E|Ki|Mi|Gi|Ti|Pi|Ei)?$" $suffix) -}}
{{- fail (printf "unsupported resource quantity suffix %q in %q" $suffix $quantity) -}}
{{- end -}}
{{- $product := mulf (float64 $number) .factor -}}
{{- if eq $product (floor $product) -}}
{{- printf "%d%s" (int64 $product) $suffix -}}
{{- else -}}
{{- printf "%s%s" (include "garage.formatNumber" $product) $suffix -}}
{{- end -}}
{{- end -}}
{{/*
Render a number as a plain decimal string, never in exponential notation
(issue #157): the null-safe fallback helpers pass values through fromYaml,
which parses every number as float64, and Go's "%v" switches float64 to
exponential notation from about 1e6 upwards - "1.048576e+06" is not a valid
Kubernetes quantity. Strings ("512Mi") are passed through untouched.
*/}}
{{- define "garage.formatNumber" -}}
{{- if kindIs "float64" . -}}
{{- regexReplaceAll "\\.?0+$" (printf "%.10f" .) "" -}}
{{- else -}}
{{- printf "%v" . -}}
{{- end -}}
{{- end -}}
{{/*
Render the replica count for the StatefulSet.
Input: dict "scaleDown" <flag|nil> "replicas" <value|nil>.
- scaleDown: true takes precedence and always yields 0.
- otherwise an explicitly set replicas value is used as given, including an
explicit 0 - Helm's "default" would swallow 0, hence the kindIs "invalid"
nil check.
- a missing replicas value defaults to 1.
A NOTE SPECIFIC TO THIS CHART: more than one replica is not a cluster. Garage
nodes find each other through RPC and a cluster LAYOUT, and this chart renders
neither peer bootstrapping nor the kubernetes_discovery CRD (see values.yaml).
A second replica would come up as a lone node with its own empty metadata db on
its own volume, and with --single-node it refuses to start at all once it sees
the other node. The value exists because guideline 11 requires it and because
0 is the documented way to stop the release; the schema caps it at 1.
*/}}
{{- define "garage.replicas" -}}
{{- if .scaleDown -}}
0
{{- else if not (kindIs "invalid" .replicas) -}}
{{- .replicas -}}
{{- else -}}
1
{{- end -}}
{{- end -}}
{{/*
Merge a user-supplied values sub-tree onto a dict of chart defaults so that a
null / empty override falls back to the defaults instead of erasing them
(issues #61, #99). Input: dict "defaults" <dict> "given" <dict|nil> "path"
<string>. Output: YAML of the merged dict (consume it with fromYaml).
An override key written without children is YAML null, and Helm treats that
null as a deletion of the chart's default. This helper defines the opposite
semantics - an empty block means "chart defaults apply" - while an explicitly
set value still wins, including false and 0.
*/}}
{{- define "garage.defaultedDict" -}}
{{- $defaults := .defaults | default dict -}}
{{- $given := .given -}}
{{- $path := .path | default "value" -}}
{{- if kindIs "invalid" $given -}}
{{- $given = dict -}}
{{- end -}}
{{- if not (kindIs "map" $given) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" $path (kindOf $given) $given) -}}
{{- end -}}
{{- $out := deepCopy $defaults -}}
{{- range $key, $value := $given -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $default := index $defaults $key -}}
{{- if and (kindIs "map" $value) (kindIs "map" $default) -}}
{{- $_ := set $out $key (fromYaml (include "garage.defaultedDict" (dict "defaults" $default "given" $value "path" (printf "%s.%s" $path $key)))) -}}
{{- else if kindIs "map" $default -}}
{{- fail (printf "%s.%s must be a mapping or null, got %s (%v)" $path $key (kindOf $value) $value) -}}
{{- else -}}
{{- $_ := set $out $key $value -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Pick one sub-block out of a values block that may itself be null.
Input: dict "block" <dict|nil> "key" <string> "path" <string>.
Returns YAML of dict "value" <the sub-block, may be nil>.
*/}}
{{- define "garage.subBlock" -}}
{{- $block := .block -}}
{{- if kindIs "invalid" $block -}}
{{- $block = dict -}}
{{- end -}}
{{- if not (kindIs "map" $block) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" .path (kindOf $block) $block) -}}
{{- end -}}
{{- toYaml (dict "value" (index $block .key)) -}}
{{- end -}}
{{/*
THE EFFECTIVE GARAGE SETTINGS - everything the ConfigMap and the StatefulSet
read out of .Values.garage, merged onto this chart's defaults.
Why the whole settings block goes through the null-safe helper and not only the
three usual suspects (guideline 12.2: "the rule is about the shape, not about
those three block kinds"): every value below is dereferenced by a template, and
an erased one would not abort the render - it would render a garage.toml with
an empty right-hand side, or a containerPort with no number. The first produces
a pod that crash-loops on a config parse error, the second a manifest the API
server rejects at apply time. Both are failures that arrive after the render
looked fine, which is exactly the class #99 is about. Satisfactory shipped the
same shape with half its blocks null-safe and half not (Gitea #16), and the
half that was not took the Service's port with it.
The defaults here MUST stay in sync with values.yaml, which remains the
documented, user-facing place for them; this copy is only the fallback.
Usage: {{ $settings := fromYaml (include "garage.settings" .) }}
*/}}
{{- define "garage.settings" -}}
{{- $given := (fromYaml (include "garage.subBlock" (dict "block" .Values "key" "garage" "path" ".Values"))).value -}}
{{- $defaults := dict
"singleNode" true
"defaultAccessKey" true
"defaultBucket" true
"replicationFactor" 1
"dbEngine" "lmdb"
"metadataFsync" true
"dataFsync" false
"metadataAutoSnapshotInterval" "6h"
"compressionLevel" 1
"blockSize" "1M"
"bindAddress" "[::]"
"adminBindAddress" "0.0.0.0"
"ports" (dict "s3" 3900 "rpc" 3901 "web" 3902 "admin" 3903)
"s3" (dict "region" "garage" "rootDomain" nil)
"web" (dict "enabled" false "rootDomain" nil) -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" $given "path" "garage") -}}
{{- end -}}
{{/*
The three directories, defined once because TWO templates have to agree on
them: garage.toml names them and the StatefulSet mounts the volumes underneath
them. They are deliberately NOT values - a path that disagrees with its mount
is a pod that writes its metadata into the container's ephemeral layer, which
looks like it works until the pod is replaced.
The split is the point of this chart's storage layout (see values.yaml):
metadata and data are two SUBDIRECTORIES OF ONE VOLUME, the metadata snapshots
sit on a SECOND one. Garage creates all three itself on start
(src/model/garage.rs, create_dir_all for metadata_dir and data_dir;
src/model/snapshot.rs for the snapshots dir), which matters here because the
image is FROM scratch and carries no mkdir - nothing else in this pod could
create them.
*/}}
{{- define "garage.dataVolumePath" -}}/mnt/garage{{- end -}}
{{- define "garage.metadataDir" -}}{{ include "garage.dataVolumePath" . }}/meta{{- end -}}
{{- define "garage.dataDir" -}}{{ include "garage.dataVolumePath" . }}/data{{- end -}}
{{- define "garage.snapshotsDir" -}}/mnt/garage-snapshots{{- end -}}
{{/*
The effective security contexts, probes and resources (issue #99).
The defaults below MUST stay in sync with values.yaml. They reproduce THIS
chart's defaults, not the repo standard - see the fsGroupChangePolicy note.
*/}}
{{- define "garage.podSecurityContext" -}}
{{- $given := (fromYaml (include "garage.subBlock" (dict "block" . "key" "pod" "path" "garage.securityContext"))).value -}}
{{- $defaults := dict
"runAsNonRoot" true
"runAsUser" 20000
"runAsGroup" 1000
"fsGroup" 1000
"fsGroupChangePolicy" "OnRootMismatch"
"seccompProfile" (dict "type" "RuntimeDefault") -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" $given "path" "garage.securityContext.pod") -}}
{{- end -}}
{{- define "garage.containerSecurityContext" -}}
{{- $given := (fromYaml (include "garage.subBlock" (dict "block" . "key" "container" "path" "garage.securityContext"))).value -}}
{{- $defaults := dict
"allowPrivilegeEscalation" false
"readOnlyRootFilesystem" true
"runAsNonRoot" true
"capabilities" (dict "drop" (list "ALL") "add" (list)) -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" $given "path" "garage.securityContext.container") -}}
{{- end -}}
{{/*
LIVENESS: a TCP connect to the admin port, and NOT the /health endpoint the
readiness probe uses. The reason is the one zipline wrote down (README,
guideline 12.3) in its sharpest form: /health answers 503 until the cluster
LAYOUT exists (src/rpc/system.rs, "Layout not yet configured, special case" ->
ClusterHealthStatus::Unavailable -> 503 in src/api/admin/special.rs). A node
waiting for its layout - every install with garage.singleNode: false, and any
install whose layout was lost - would therefore be killed and restarted
forever, and a restart is precisely what cannot fix it.
What the TCP probe does prove is what a liveness probe is for: the process is
alive and its listener is open. A wedged process that stops accepting is
restarted; a healthy process that has nothing to serve yet is not.
*/}}
{{- define "garage.livenessProbe" -}}
{{- $defaults := dict
"tcpSocket" (dict "port" "admin")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.livenessProbe") -}}
{{- end -}}
{{/*
READINESS: GET /health on the admin port - the one endpoint that answers the
question a readiness probe asks, namely "can this node serve requests".
Measured against the source at tag v2.4.1 rather than chosen:
- the S3 API server is spawned independently of the layout state
(src/garage/server.rs, the S3/web/admin servers are all spawned after
initial_config), so the S3 socket binds and ANSWERS - with errors - before
any layout exists. A tcpSocket probe on port 3900 would therefore report a
service ready that accepts nothing, which is the lie this chart is not
allowed to tell.
- /health needs no token (@special in src/api/admin/router_v2.rs) and maps
ClusterHealthStatus straight onto the status code: Healthy and Degraded ->
200, Unavailable -> 503 with "Quorum is not available for some/all
partitions, reads and writes will fail". An unconfigured layout is a
special case in src/rpc/system.rs and yields Unavailable.
So with garage.singleNode: true the pod becomes Ready a moment after start
(initial_config applies the layout BEFORE the servers are spawned), and with
singleNode: false it stays not-Ready until an operator applies a layout by hand
- which is the truth, not a defect: until then the S3 endpoint accepts nothing
and no Service should route to it.
*/}}
{{- define "garage.readinessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/health" "port" "admin")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.readinessProbe") -}}
{{- end -}}
{{/*
STARTUP: TCP again, for the same reason as liveness - a startup probe that
fails forever kills the container, so it must not depend on a layout that only
an operator can create.
Budget = periodSeconds * failureThreshold = 5 * 24 = 120s. It covers the slow
part of a real start, which is not Garage: attaching and mounting two network
volumes, plus opening (and on an unclean shutdown recovering) the LMDB
metadata database. Garage's own startup work after that is sub-second.
*/}}
{{- define "garage.startupProbe" -}}
{{- $defaults := dict
"tcpSocket" (dict "port" "admin")
"periodSeconds" 5
"timeoutSeconds" 5
"failureThreshold" 24 -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.startupProbe") -}}
{{- end -}}
{{/*
The effective resources block, fed into the resources helper above so that an
erased "resources:" still yields the chart's requests and computed limits.
*/}}
{{- define "garage.effectiveResources" -}}
{{- $defaults := dict
"limitFactor" 3
"requests" (dict "cpu" "100m" "memory" "512Mi" "ephemeral-storage" "32Mi") -}}
{{- $merged := fromYaml (include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.resources")) -}}
{{- include "garage.resources" $merged -}}
{{- end -}}
{{/*
The additive env block, rendered through tpl with the value, secretKeyRef and
configMapKeyRef forms (guideline 10).
ADDITIVE MEANS ADDITIVE: an entry repeating a name this chart already sets does
not replace it. At install time both reach the pod and the later one wins; on
an EXISTING release the pair collapses to the chart's own value, because
Kubernetes merges env by name - and helm upgrade still reports success. So do
not use this block to override GARAGE_RPC_SECRET or the three
GARAGE_DEFAULT_* variables; change the values that produce them.
*/}}
{{- define "garage.env" -}}
{{- $root := .root -}}
{{- range $value := .env }}
{{- if $value.value }}
- name: {{ $value.name | quote }}
# quote is required, not cosmetic: a value like "3900" or "true" renders
# unquoted as a YAML number/bool, and the API server then rejects the whole
# manifest with "cannot unmarshal number into Go struct field EnvVar.value of
# type string".
value: {{ tpl $value.value $root | quote }}
{{- else if $value.valueFrom }}
{{- if $value.valueFrom.secretKeyRef }}
- name: {{ $value.name }}
valueFrom:
secretKeyRef:
name: {{ tpl $value.valueFrom.secretKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.secretKeyRef.key $root | quote }}
{{- else if $value.valueFrom.configMapKeyRef }}
- name: {{ $value.name }}
valueFrom:
configMapKeyRef:
name: {{ tpl $value.valueFrom.configMapKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.configMapKeyRef.key $root | quote }}
{{- end }}
{{- end }}
{{- end }}
{{- end -}}
+173
View File
@@ -0,0 +1,173 @@
{{- $settings := fromYaml (include "garage.settings" .) }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-configmap
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
data:
# The complete garage.toml. It carries NO SECRET: the RPC secret arrives as
# the environment variable GARAGE_RPC_SECRET (see the StatefulSet and
# values.yaml for why the file form cannot be used here), so this ConfigMap
# is safe to read and the checksum/config annotation on the pod template
# still means "the configuration changed".
#
# It is mounted with subPath at /etc/garage.toml, which is the path Garage
# looks at without being told (-c defaults to /etc/garage.toml). A subPath
# mount is never refreshed by the kubelet, which is exactly why the checksum
# annotation is mandatory rather than decorative.
garage.toml: |
# Managed by the garage Helm chart - changes here are overwritten on the
# next helm upgrade. Edit the chart's values instead.
# ONE node, and the replication factor is 1 BY DECISION, with the risk
# named (Gitea #81): longhorn-encrypted-ns already keeps three replicas of
# the encrypted volume, and Garage replication 3 on top of that would be
# nine times the raw data for two layers that know nothing about each
# other.
#
# WHAT THAT COSTS, because it is not free: upstream says of
# replication_factor = 1 "Do not use this for anything else than test
# deployments", and the reason is on its known-issues page - the same
# factor applies to the METADATA, so a single-replica setup is especially
# exposed to metadata corruption, and a corrupted database file is
# "generally not recoverable". Longhorn does not help against that: it
# replicates a corrupted file three times. Redundancy against device
# failure, yes; against file corruption, no. The three settings below are
# the countermeasures that do work at factor 1, and none of them is
# optional.
replication_factor = {{ $settings.replicationFactor | int64 }}
# LMDB is Garage's default since 0.9 and the faster engine. The price is
# named: the reported corruption after an unclean shutdown stays possible,
# the settings below only lower the probability, and at factor 1 there is
# no peer to recover from - only the last snapshot. Upstream ties its LMDB
# recommendation to replication factor >= 2; running it at 1 is this
# chart's deliberate deviation.
#
# The alternative is "sqlite", which is slower on metadata throughput. With
# a single mail server as the only consumer that ceiling is unlikely to be
# reached, so sqlite is a defensible choice for anyone who weighs the
# durability of a single-file database higher. "fjall" is experimental.
db_engine = "{{ $settings.dbEngine }}"
# metadata_dir and data_dir are two SUBDIRECTORIES OF ONE VOLUME; the
# snapshots live on a second one. Longhorn snapshots are atomic PER VOLUME
# and never across volumes, so keeping the index and the blocks together is
# what prevents the one failure mode that matters for a mail store: a
# backup whose metadata points at objects whose blocks were not written yet
# ("the mailbox lists a message whose body is missing").
#
# The snapshots are on the OTHER volume for the opposite reason: a recovery
# point must not die with the volume whose corruption it is there to heal.
metadata_dir = "{{ include "garage.metadataDir" . }}"
data_dir = "{{ include "garage.dataDir" . }}"
metadata_snapshots_dir = "{{ include "garage.snapshotsDir" . }}"
# metadata_fsync IS OFF IN GARAGE'S DEFAULTS, and this is the single
# setting most easily lost in a "tidy up the obvious defaults" pass. It is
# set here on purpose: Longhorn's block layer cannot help if the engine
# never calls fsync - a write that was never handed to the kernel is not in
# any replica and not in any snapshot. Garage's own example configuration
# pairs it with metadata_auto_snapshot_interval, as here.
metadata_fsync = {{ $settings.metadataFsync }}
# data_fsync stays off, which is Garage's default and the right trade for
# the blocks: a lost block is one object, is detectable by a scrub, and is
# replaceable from the sender's copy; a lost metadata write can take the
# whole index with it. Turning it on costs an fsync per written block.
data_fsync = {{ $settings.dataFsync }}
# THE ONLY CONSISTENT RECOVERY POINT THIS DEPLOYMENT HAS. File-system
# snapshots of metadata_dir guarantee nothing - Garage says so itself: a
# snapshot taken during a metadata write is corrupt and useless as a
# restore point, which is why Garage recommends ITS OWN snapshot mechanism
# in all cases. This is the correction that reversed a premise of the
# original plan: Longhorn backs up the blocks, not the metadata.
#
# KNOW THE WINDOW: Garage keeps exactly TWO snapshots (KEEP_SNAPSHOTS = 2),
# the first one after interval/2 and every later one after interval times a
# random 1.0-1.2. At "6h" that means the first after ~3h and a restore
# window of roughly 6-14.4h afterwards. A corruption discovered later than
# that is older than every snapshot that still exists.
metadata_auto_snapshot_interval = "{{ $settings.metadataAutoSnapshotInterval }}"
block_size = "{{ $settings.blockSize }}"
# An integer is the zstd level; the literal "none" disables compression and
# has to be QUOTED to stay valid TOML. Garage's deserializer takes exactly
# those two forms - and 0 is not the second one: it is a valid integer that
# zstd reads as its default level (3), so writing 0 to "switch compression
# off" quietly turns it up.
{{- if kindIs "string" $settings.compressionLevel }}
compression_level = {{ $settings.compressionLevel | quote }}
{{- else }}
compression_level = {{ $settings.compressionLevel | int64 }}
{{- end }}
# Inter-node RPC. There are no other nodes here (no bootstrap_peers, no
# kubernetes_discovery - see values.yaml), but Garage binds this listener
# regardless and refuses to start without an rpc_secret, which arrives from
# the environment.
rpc_bind_addr = "{{ $settings.bindAddress }}:{{ $settings.ports.rpc | int64 }}"
[s3_api]
api_bind_addr = "{{ $settings.bindAddress }}:{{ $settings.ports.s3 | int64 }}"
# The region name clients must use. It is not a cloud region, it is a label
# both sides have to agree on - a consumer configured for a different
# region gets a signature error, not a redirect.
s3_region = "{{ $settings.s3.region }}"
{{- with $settings.s3.rootDomain }}
# Vhost-style bucket access (<bucket>.<root_domain>). Only useful with an
# Ingress and a wildcard DNS entry in front of it; path-style access works
# without it and is what an in-cluster consumer uses.
root_domain = {{ . | quote }}
{{- end }}
{{- if $settings.web.enabled }}
[s3_web]
# Static website hosting, OFF by default (see values.yaml): it is a second
# HTTP listener and serves nothing at all without a root_domain, while the
# consumer this chart was built for speaks S3.
bind_addr = "{{ $settings.bindAddress }}:{{ $settings.ports.web | int64 }}"
{{- with $settings.web.rootDomain }}
root_domain = {{ . | quote }}
{{- end }}
{{- end }}
[admin]
# Bound on 0.0.0.0 rather than [::] BECAUSE ALL THREE PROBES USE THIS PORT.
# A dual-stack listener on [::] only accepts the kubelet's IPv4 connection
# while net.ipv6.bindv6only is 0 - true by default on Linux, but a node
# sysctl this chart cannot see, and the failure would be a pod that never
# becomes Ready for a reason nothing in the manifest points at. The S3 and
# RPC listeners keep the upstream [::] default; set garage.bindAddress to
# "0.0.0.0" on an IPv4-only cluster, or garage.adminBindAddress to "[::]"
# on an IPv6-only one.
#
# NO admin_token IS SET, and that is not an omission: /health, the one
# endpoint the probes use, is explicitly token-free (@special in the admin
# router), and nothing in this deployment calls the authenticated API - the
# CLI inside the pod talks RPC, not HTTP. A token here would be a fifth
# vault field guarding nothing that is in use.
#
# WHAT IS AND IS NOT REACHABLE AS A RESULT, stated precisely because "no
# token" reads like "nothing is open":
# /health open, by design, and the probes depend on it
# /metrics OPEN TOO - the metrics_require_token switch defaults to
# false, so without a metrics credential the endpoint needs no
# authentication. That is internal metrics on a ClusterIP port;
# to close it, set that switch true and add a credential.
#
# (And a note for whoever edits these lines: do NOT write them
# as "<key> = <value>" with a credential-ish key name. Trivy's
# KSV-0109 reads an assignment inside this ConfigMap's DATA -
# comments included - as a stored secret and reports HIGH. It
# cost one run here, 2026-10-03.)
# everything else 401/403 until a token exists. Note that a config
# token is not the only way to get one: Garage 2.x also keeps
# admin tokens in the cluster ("garage admin-token create"),
# and those work without any entry in this file.
# If you do want a config token, it is admin_token_file / the
# GARAGE_ADMIN_TOKEN environment variable - and the file form carries the
# same permission trap as rpc_secret_file (see values.yaml).
api_bind_addr = "{{ $settings.adminBindAddress }}:{{ $settings.ports.admin | int64 }}"
+50
View File
@@ -0,0 +1,50 @@
{{- /*
OFF BY DEFAULT (ingress.enabled: false), because the consumer this chart was
built for speaks S3 inside the cluster and needs no published endpoint. The
values exist anyway, built to this repo's ingress convention, so that
publishing the endpoint later is a values change rather than a chart change.
GUIDELINE 1, CHECKED IN BOTH DIRECTIONS (README, "Values that must not lie"):
nothing outside this template reads ingress.domain. The S3 API's vhost-style
suffix is its own value, garage.s3.rootDomain, because the two are not the same
thing - a wildcard vhost domain and the single host an Ingress answers on. So
with the Ingress switched off, BOTH domain and clusterIssuer are free, and
their required() lives here where it can actually fire.
WHAT IT PUBLISHES: the S3 API. There is no web UI to expose - Garage's admin
API is JSON and the authenticated half is closed in this chart (no token), and
the project's web interface is a separate program that is not part of this
image. Anyone publishing this should know what the object they are putting on
the internet is: an S3 endpoint that authenticates every request with a
signature, whose credentials are the two fields of the s3 vault item.
*/ -}}
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-ingress
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
annotations:
cert-manager.io/cluster-issuer: {{ required "A Cluster-Issuer must be provided (ingress.clusterIssuer)" .Values.ingress.clusterIssuer }}
{{- with .Values.ingress.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
ingressClassName: {{ .Values.ingress.className }}
rules:
- host: {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
http:
paths:
- backend:
service:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
port:
name: s3
path: /
pathType: Prefix
tls:
- hosts:
- {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
secretName: tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
{{- end }}
+63
View File
@@ -0,0 +1,63 @@
{{- /*
TWO claims, and the split is the durability design of this chart rather than a
sizing convenience (Gitea #81):
1. <release>-garage-pvc metadata_dir AND data_dir, as two
subdirectories of this one volume
2. <release>-garage-snapshots-pvc metadata_snapshots_dir
Claim 1 keeps index and blocks on ONE volume because Longhorn snapshots are
atomic per volume and never across volumes. Split over two volumes, a backup
can capture metadata that points at objects whose blocks were not written yet -
for a mail store exactly the failure "the mailbox lists a message whose body is
missing".
Claim 2 exists because Garage's own metadata snapshots are the only consistent
recovery point here (file-system snapshots of a live metadata database are not
one), and a recovery point stored on the volume whose corruption it should
repair is not a recovery point.
Both blocks are DELIBERATELY NOT null-safe: an erased size yields an invalid
PVC that is rejected at apply time, which is loud. A chart-side fallback would
silently size somebody's object store - and on an existing release it could not
help anyway, because a PVC cannot shrink. Same reasoning as satisfactory's
documented exception (Gitea #39) and valkey's persistence block.
Neither claim is ever renamed: a renamed PVC makes Kubernetes provision a new,
empty volume and orphans the data.
*/ -}}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: {{ .Values.storage.data.amount }}
{{- if .Values.storage.data.storageClass }}
storageClassName: {{ .Values.storage.data.storageClass }}
{{- end }}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-snapshots-pvc
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
# Sized for TWO snapshots of the metadata database plus head room, not
# for the objects - Garage keeps exactly two and deletes the older ones
# itself. A snapshot is a compacted copy of the LMDB file, so this needs
# roughly twice the size metadata_dir reaches.
storage: {{ .Values.storage.snapshots.amount }}
{{- if .Values.storage.snapshots.storageClass }}
storageClassName: {{ .Values.storage.snapshots.storageClass }}
{{- end }}
+71
View File
@@ -0,0 +1,71 @@
{{- /*
The 1Password items this chart materializes. Two of them, because they have
different audiences: the RPC secret belongs to this node alone, while the S3
access key is the credential the CONSUMER uses - the same item a consuming
chart (stalwart's secrets.s3SecretRef) points at, which is why it carries the
bucket name as a third field: endpoint, bucket, key and secret then have one
source of truth instead of being kept in step by hand.
WHAT THE FIELD NAMES ARE AND WHY THEY ARE NOT FREE-FORM: the operator
materializes one Secret key per FIELD of the item, and this chart references
those keys by name in the StatefulSet's env block. An item without the
expected field leaves the container in CreateContainerConfigError. NOTES.txt
lists all four fields for the installer.
AND THE LIMIT THAT CANNOT BE CLOSED: a change to the CONTENT of either item
restarts nothing. Helm never sees a secret the 1Password operator
materializes, so there is nothing to hash. What this chart does cover is the
op:// PATH, through checksum/secret-ref on the pod template. After rotating a
value INSIDE an item the manual step is
kubectl -n <ns> rollout restart statefulset/<release>-garage-sfs
and for the access key that is not enough on its own - see NOTES.txt, Garage
refuses to adopt a changed secret for an access key it already knows.
The 1Password operator's auto-restart annotation is NOT set here and must not
be: verified against operator 1.12.0 it enumerates Deployments only, so on a
StatefulSet it is a silent no-op advertising a guarantee it does not provide
(README, guideline 9).
*/ -}}
{{- $settings := fromYaml (include "garage.settings" .) -}}
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-rpc-secret
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
# Must carry the field: rpc-secret
#
# 32 bytes of random hex - Garage rejects anything else with "Invalid RPC
# secret key: expected 32 bytes of random hex". Generate it with
# openssl rand -hex 32
# It is required even for a single node: Garage uses it as the network key of
# its RPC layer and refuses to start without it ("rpc_secret value is
# missing, not present in config file or in environment").
itemPath: {{ required "You must provide an RPC secret reference (secrets.rpcSecretRef) - the 1Password item holding this cluster's 32-byte hex 'rpc-secret' field" .Values.secrets.rpcSecretRef }}
{{- $s3Needed := and $settings.singleNode (or $settings.defaultAccessKey $settings.defaultBucket) }}
{{- if or .Values.secrets.s3SecretRef $s3Needed }}
---
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-s3-secret
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
# Must carry the fields: access-key, secret-key, bucket
#
# THE DIRECTION IS INWARDS, and that is the part worth knowing: with
# garage.singleNode plus defaultAccessKey/defaultBucket, Garage IMPORTS these
# three values on start (--default-access-key / --default-bucket read them
# from the environment). The vault item is the source of truth, and nothing
# has to be copied out of the server afterwards.
#
# Garage validates all three on import and refuses to START on a malformed
# one, so they are not free-form labels (src/model/key_table.rs,
# src/model/bucket_alias_table.rs):
# access-key at least 8 characters, ASCII alphanumeric plus - _ .
# secret-key at least 16 characters, printable ASCII (U+0021-U+007E)
# bucket a valid S3 bucket name (3-63 characters, lowercase)
# NOTES.txt prints the generators.
itemPath: {{ required "You must provide an S3 key reference (secrets.s3SecretRef) - the 1Password item holding the fields access-key, secret-key and bucket. It is required while garage.singleNode is true together with defaultAccessKey or defaultBucket, because Garage imports those values on start" .Values.secrets.s3SecretRef }}
{{- end }}
+45
View File
@@ -0,0 +1,45 @@
{{- $settings := fromYaml (include "garage.settings" .) }}
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
selector:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
ports:
# The S3 endpoint. In-cluster consumers reach it at
# http://<release>-garage-service.<namespace>.svc.cluster.local:3900
# with path-style addressing (bucket in the path), which needs no
# root_domain and no DNS wildcard.
- protocol: TCP
port: {{ $settings.ports.s3 | int64 }}
targetPort: s3
name: s3
# The admin endpoint. With this chart's configuration it serves exactly two
# things without authentication - /health (token-free by design) and
# /metrics (because metrics_require_token defaults to false) - and answers
# everything else with 401/403 until an admin token exists, whether from
# the config or from "garage admin-token create". So it is a monitoring
# surface rather than a management one, and it is a ClusterIP port.
- protocol: TCP
port: {{ $settings.ports.admin | int64 }}
targetPort: admin
name: admin
{{- if $settings.web.enabled }}
- protocol: TCP
port: {{ $settings.ports.web | int64 }}
targetPort: web
name: web
{{- end }}
# DELIBERATELY NO RPC PORT. Port 3901 exists on the pod because Garage binds
# it either way, but nothing should reach it: a single node has no peers, and
# this chart renders neither bootstrap_peers nor the kubernetes_discovery CRD.
# A Service port would advertise a cluster interface that is not one.
#
# ClusterIP, and no value to change it. The S3 API authenticates every
# request, but the way out of the cluster is an Ingress with a certificate
# (ingress.enabled, off by default), not a NodePort that puts an object store
# on a routable address by accident.
type: ClusterIP
+208
View File
@@ -0,0 +1,208 @@
{{- $settings := fromYaml (include "garage.settings" .) }}
{{- /*
Catch the one flag combination Garage rejects at START, at render time instead:
--default-access-key and --default-bucket "can only be used in single-node
mode" (src/garage/server.rs). Without this check the chart would install
cleanly and the container would exit immediately, with the reason only in the
pod log.
*/ -}}
{{- if and (not $settings.singleNode) (or $settings.defaultAccessKey $settings.defaultBucket) }}
{{- fail "garage.defaultAccessKey and garage.defaultBucket require garage.singleNode: true - Garage refuses to start otherwise (\"Flags --default-access-key and --default-bucket can only be used in single-node mode\"). With singleNode: false, create the key and the bucket by hand after applying the cluster layout; NOTES.txt has the commands." }}
{{- end }}
{{- $importKey := and $settings.singleNode (or $settings.defaultAccessKey $settings.defaultBucket) }}
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
replicas: {{ include "garage.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.garage.replicas) }}
serviceName: {{ .Release.Name }}-{{ .Chart.Name }}-service
selector:
matchLabels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
template:
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
annotations:
# Hash of the rendered ConfigMap: a changed garage.toml changes the pod
# template and thus restarts the pod. Without it the change would reach
# nothing - the config is mounted with subPath (never refreshed by the
# kubelet) and Garage reads it once, at start.
checksum/config: {{ include (print $.Template.BasePath "/garage.configmap.yaml") . | sha256sum }}
# Hash of the rendered OnePasswordItems. This covers the SOURCE of the
# secrets, not their content: repointing secrets.rpcSecretRef or
# secrets.s3SecretRef at a different vault item rolls the pod instead of
# leaving the old credentials in the running process. A change INSIDE an
# item cannot be covered - Helm never sees it (see garage.secret.yaml).
checksum/secret-ref: {{ include (print $.Template.BasePath "/garage.secret.yaml") . | sha256sum }}
spec:
automountServiceAccountToken: false
securityContext:
{{- include "garage.podSecurityContext" .Values.garage.securityContext | nindent 8 }}
# NO INIT CONTAINER, and that is a property of the image rather than a
# choice: dxflrs/garage is built FROM scratch and contains exactly one
# file, /garage. There is no shell, no mkdir, no wget - so an init
# container would have to be a SECOND image to pin, track and update.
# Nothing needs one: Garage creates metadata_dir, data_dir and the
# snapshots directory itself on start (create_dir_all in
# src/model/garage.rs and src/model/snapshot.rs), and the volumes are made
# writable by fsGroup.
#
# The same property rules out exec probes in this chart for good - all
# three probes are HTTP or TCP, see the helpers.
containers:
- name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
# Fully qualified: an unqualified "dxflrs/garage" resolves to Docker
# Hub by convention rather than by rule. The multi-arch image, not
# upstream's own chart's dxflrs/amd64_garage.
image: "docker.io/dxflrs/garage:{{ .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
# command AND args, because the image's CMD is ["/garage", "server"]
# and flags have to go after the subcommand. Overriding only args
# would drop the "server" subcommand.
command:
- /garage
args:
- server
{{- if $settings.singleNode }}
# --single-node is what makes this chart's default install reach
# Ready without an operator (Gitea #81, second plan correction):
# initial_config() in src/garage/server.rs creates the cluster
# layout in-process BEFORE the S3, web and admin servers are
# spawned - zone "dc1", capacity taken from the detected disk size,
# then apply_staged_changes(1).
#
# IT IS IDEMPOTENT, and it cannot damage an existing cluster. The
# layout version decides: 0 creates the layout, 1 does nothing at
# all, anything above 1 refuses to start ("Refusing to run in
# single-node mode: layout version is already superior to 1"), and
# a node that sees other nodes refuses as well ("more nodes are
# already present in the cluster"). There is no --version argument
# that could be passed twice with the same value, which is the trap
# the CLI path (garage layout apply --version N+1) has.
#
# It requires replication_factor = 1 and says so loudly if it is
# not.
- --single-node
{{- if or $settings.defaultAccessKey $settings.defaultBucket }}
# Imports the access key from GARAGE_DEFAULT_ACCESS_KEY /
# GARAGE_DEFAULT_SECRET_KEY below, i.e. FROM the vault item INTO
# Garage. Re-running is guarded upstream: a key that already exists
# with a different secret aborts the start, and one that was deleted
# in the cluster cannot be added back.
- --default-access-key
{{- end }}
{{- if $settings.defaultBucket }}
# Creates the bucket from GARAGE_DEFAULT_BUCKET and grants the key
# above read/write/owner on it. Implies --default-access-key.
- --default-bucket
{{- end }}
{{- end }}
env:
# THE RPC SECRET AS AN ENVIRONMENT VARIABLE, NOT AS A FILE - and the
# full reason is in values.yaml under secrets.rpcSecretRef. The
# short version, because this is the line a later round will want to
# "fix": rpc_secret_file CANNOT WORK in a non-root pod.
# read_secret_file rejects any file whose mode has a group or other
# bit set (src/garage/secrets.rs: mode & 0o077), and the kubelet
# OR-s the group-read bit onto every file of a secret volume as soon
# as fsGroup is set. Without fsGroup the file stays root-owned and
# uid 20000 cannot read it. Kubernetes can set the group of such a
# file, never its owner, so no combination satisfies both.
- name: GARAGE_RPC_SECRET
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-rpc-secret
key: rpc-secret
{{- if $importKey }}
- name: GARAGE_DEFAULT_ACCESS_KEY
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-s3-secret
key: access-key
- name: GARAGE_DEFAULT_SECRET_KEY
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-s3-secret
key: secret-key
{{- end }}
{{- if and $settings.singleNode $settings.defaultBucket }}
- name: GARAGE_DEFAULT_BUCKET
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-s3-secret
key: bucket
{{- end }}
{{- with .Values.garage.env }}
{{- include "garage.env" (dict "root" $ "env" .) | nindent 12 }}
{{- end }}
ports:
- containerPort: {{ $settings.ports.s3 | int64 }}
protocol: TCP
name: s3
- containerPort: {{ $settings.ports.rpc | int64 }}
protocol: TCP
name: rpc
{{- if $settings.web.enabled }}
- containerPort: {{ $settings.ports.web | int64 }}
protocol: TCP
name: web
{{- end }}
# The port all three probes use. Its NAME is what the probe defaults
# reference, so renaming it here breaks them.
- containerPort: {{ $settings.ports.admin | int64 }}
protocol: TCP
name: admin
# Probes, resources and both security contexts go through the
# null-safe helpers (#99): an erased block means "chart defaults
# apply" instead of a vanished probe or an unhardened container.
# Never wrap these in an "if" - a guard hides the erased case
# completely instead of rendering a visible null.
#
# Which probe points at what, and why liveness and startup must NOT
# point at /health, is argued at each helper in _helpers.tpl.
livenessProbe:
{{- include "garage.livenessProbe" .Values.garage.livenessProbe | nindent 12 }}
readinessProbe:
{{- include "garage.readinessProbe" .Values.garage.readinessProbe | nindent 12 }}
startupProbe:
{{- include "garage.startupProbe" .Values.garage.startupProbe | nindent 12 }}
resources:
{{- include "garage.effectiveResources" .Values.garage.resources | nindent 12 }}
securityContext:
{{- include "garage.containerSecurityContext" .Values.garage.securityContext | nindent 12 }}
volumeMounts:
# ONE volume carrying metadata_dir and data_dir as subdirectories,
# so a Longhorn snapshot catches index and blocks in one atomic
# operation. Garage creates both subdirectories on start.
- name: data
mountPath: {{ include "garage.dataVolumePath" . }}
# The SECOND volume, holding the only consistent recovery point this
# deployment has. Separate on purpose: a snapshot that dies with the
# volume whose corruption it should repair is not a recovery point.
- name: snapshots
mountPath: {{ include "garage.snapshotsDir" . }}
- name: config
mountPath: /etc/garage.toml
subPath: garage.toml
readOnly: true
# No emptyDir for /tmp, and that is checked rather than assumed: with
# readOnlyRootFilesystem: true the root filesystem of this container
# holds exactly one file. Garage writes under metadata_dir, data_dir
# and the snapshots directory and uses no temporary directory
# (verified in src/model, src/db and src/block at v2.4.1).
restartPolicy: Always
volumes:
- name: data
persistentVolumeClaim:
claimName: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
- name: snapshots
persistentVolumeClaim:
claimName: {{ .Release.Name }}-{{ .Chart.Name }}-snapshots-pvc
- name: config
configMap:
name: {{ .Release.Name }}-{{ .Chart.Name }}-configmap
+308
View File
@@ -0,0 +1,308 @@
{
"$schema": "https://json-schema.org/draft-07/schema#",
"title": "garage values",
"description": "Strict values schema (issue #68). additionalProperties is false on every chart-owned object level, so a key this chart does not read is rejected at render time instead of being silently ignored; the objects that go verbatim into the Kubernetes pod spec (probes, security contexts, global, free-form annotation maps) stay open, because the chart does not interpret their contents. The blocks the chart dereferences additionally accept null, because an erased block means 'chart defaults apply' (issue #99) rather than a vanished setting. Note what this schema CANNOT enforce: secrets.rpcSecretRef ships as null and JSON Schema's 'required' tests presence, not null-ness, so it would never fire - the enforcement is required() in the template (issue #18).",
"type": "object",
"additionalProperties": false,
"properties": {
"global": {
"description": "Helm's conventional cross-chart values. Unused by this chart, kept open so an umbrella install cannot break on it.",
"type": "object"
},
"scaleDown": {
"description": "true renders the StatefulSet with replicas: 0 and takes precedence over garage.replicas.",
"type": "boolean"
},
"secrets": {
"type": "object",
"additionalProperties": false,
"properties": {
"rpcSecretRef": {
"description": "op:// item path. The item must carry the field 'rpc-secret' with 32 bytes of random hex. Required always - Garage refuses to start without an RPC secret even as a single node.",
"type": ["string", "null"]
},
"s3SecretRef": {
"description": "op:// item path. The item must carry the fields 'access-key', 'secret-key' and 'bucket'. Required while garage.singleNode is true together with defaultAccessKey or defaultBucket, because Garage imports those values on start.",
"type": ["string", "null"]
}
}
},
"garage": {
"type": "object",
"additionalProperties": false,
"properties": {
"replicas": {
"description": "0 or 1. Capped at 1 because more than one replica is not a cluster: this chart renders no peer bootstrapping and no discovery CRD, so a second pod would be a lone node with its own empty metadata database.",
"type": "integer",
"minimum": 0,
"maximum": 1
},
"singleNode": {
"description": "Run 'garage server --single-node', which creates the cluster layout in-process before the listeners open. Idempotent and guarded by the layout version; requires replicationFactor 1.",
"type": "boolean"
},
"defaultAccessKey": {
"description": "Import the S3 access key from the vault item on start (--default-access-key). Requires singleNode; the chart fails the render on the mixed combination, because Garage refuses it at start.",
"type": "boolean"
},
"defaultBucket": {
"description": "Create the bucket from the vault item's 'bucket' field on start (--default-bucket) and grant the imported key access to it. Requires singleNode.",
"type": "boolean"
},
"replicationFactor": {
"description": "1 for this deployment, and raising it is not an upgrade path: the factor is part of the cluster layout, so increasing it is a layout operation with data movement on a cluster that would first need more nodes.",
"type": "integer",
"minimum": 1
},
"dbEngine": {
"description": "Metadata engine. Changing it on an existing release needs 'garage convert-db', not a values change.",
"type": "string",
"enum": ["lmdb", "sqlite", "fjall"]
},
"metadataFsync": {
"description": "true here, against Garage's own default of false: the volume layer cannot help with a write the engine never handed to the kernel.",
"type": "boolean"
},
"dataFsync": {
"description": "false, Garage's default: a lost block is one object and is replaceable, a lost metadata write can take the index with it.",
"type": "boolean"
},
"metadataAutoSnapshotInterval": {
"description": "Go-style duration, e.g. '6h'. The only consistent recovery point this deployment has; an empty string is rejected because switching the snapshots off is not a configuration this chart offers at replication factor 1.",
"type": "string",
"minLength": 1
},
"blockSize": {
"type": "string",
"minLength": 1
},
"compressionLevel": {
"description": "zstd level for stored blocks, or the literal string \"none\" to disable compression. NOT 0 for disabling: Garage accepts 0 as an integer and zstd reads it as its default level (3), i.e. more compression than 1, not none. Negative integers are zstd's fast modes.",
"oneOf": [
{ "type": "integer" },
{ "type": "string", "enum": ["none"] }
]
},
"bindAddress": {
"description": "Host part of the S3, web and RPC listeners. '[::]' accepts IPv4 as well while net.ipv6.bindv6only is 0.",
"type": "string",
"minLength": 1
},
"adminBindAddress": {
"description": "Host part of the admin listener, separate from the others because all three probes use it.",
"type": "string",
"minLength": 1
},
"ports": {
"description": "May be null / empty: the chart then falls back to its defaults. The ConfigMap renders these into garage.toml's bind addresses and the StatefulSet renders them as containerPorts, so both files follow one source.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"s3": { "$ref": "#/definitions/port" },
"rpc": { "$ref": "#/definitions/port" },
"web": { "$ref": "#/definitions/port" },
"admin": { "$ref": "#/definitions/port" }
}
},
"s3": {
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"region": {
"description": "A label both sides must agree on; a consumer configured for a different one gets a signature error.",
"type": "string",
"minLength": 1
},
"rootDomain": {
"description": "Optional suffix for vhost-style bucket access. Deliberately not coupled to ingress.domain - the two are different things.",
"type": ["string", "null"]
}
}
},
"web": {
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"enabled": {
"description": "Static website hosting (the s3_web endpoint). Off: a second HTTP listener that serves nothing without a rootDomain.",
"type": "boolean"
},
"rootDomain": {
"type": ["string", "null"]
}
}
},
"resources": {
"$ref": "#/definitions/resources"
},
"livenessProbe": {
"$ref": "#/definitions/probe"
},
"readinessProbe": {
"$ref": "#/definitions/probe"
},
"startupProbe": {
"$ref": "#/definitions/probe"
},
"securityContext": {
"$ref": "#/definitions/securityContext"
},
"env": {
"$ref": "#/definitions/env"
}
}
},
"storage": {
"description": "The two PVCs. Deliberately NOT null-safe: an erased block yields an invalid PVC, which fails loudly at apply time. Issue #99 covers only the silent class, and on an existing release a chart-side fallback could not help anyway, because a PVC cannot shrink.",
"type": "object",
"additionalProperties": false,
"properties": {
"data": {
"description": "metadata_dir AND data_dir as two subdirectories of one volume, so a volume snapshot catches index and blocks atomically.",
"type": "object",
"additionalProperties": false,
"properties": {
"amount": { "type": ["string", "null"] },
"storageClass": { "type": ["string", "null"] }
}
},
"snapshots": {
"description": "metadata_snapshots_dir, on a second volume so the recovery point does not die with the volume whose corruption it should repair.",
"type": "object",
"additionalProperties": false,
"properties": {
"amount": { "type": ["string", "null"] },
"storageClass": { "type": ["string", "null"] }
}
}
}
},
"ingress": {
"type": "object",
"additionalProperties": false,
"properties": {
"enabled": {
"description": "Gates the whole Ingress template (issue #93). Checked in both directions for this chart: nothing outside that template reads ingress.domain, so switching the Ingress off frees domain as well as clusterIssuer.",
"type": "boolean"
},
"clusterIssuer": {
"type": ["string", "null"]
},
"domain": {
"type": ["string", "null"]
},
"className": {
"type": ["string", "null"]
},
"annotations": {
"description": "Extra Ingress annotations, merged with the cert-manager annotation the chart always sets. Free-form keys by nature, so this object stays open.",
"type": "object"
}
}
}
},
"definitions": {
"port": {
"type": ["integer", "null"],
"minimum": 1,
"maximum": 65535
},
"resources": {
"description": "May be null / empty: the chart then falls back to its default requests and computed limits (issue #99) instead of rendering a container without resource accounting.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"limitFactor": {
"type": "number",
"exclusiveMinimum": 0
},
"requests": {
"$ref": "#/definitions/resourceList"
},
"limits": {
"$ref": "#/definitions/resourceList"
}
}
},
"resourceList": {
"description": "Both spellings of the ephemeral-storage key are accepted because the resources helper normalizes camelCase to the Kubernetes key.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"cpu": {
"type": ["string", "number", "null"]
},
"memory": {
"type": ["string", "number", "null"]
},
"ephemeral-storage": {
"type": ["string", "number", "null"]
},
"ephemeralStorage": {
"type": ["string", "number", "null"]
}
}
},
"env": {
"description": "Additive container env entries. An entry repeating a name the chart already sets does not replace it - on an existing release the pair collapses to the chart's own value while helm upgrade reports success.",
"type": ["array", "null"],
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name"],
"properties": {
"name": {
"type": "string"
},
"value": {
"type": "string"
},
"valueFrom": {
"type": "object",
"additionalProperties": false,
"properties": {
"secretKeyRef": {
"$ref": "#/definitions/keyRef"
},
"configMapKeyRef": {
"$ref": "#/definitions/keyRef"
}
}
}
}
}
},
"keyRef": {
"type": "object",
"additionalProperties": false,
"required": ["name", "key"],
"properties": {
"name": {
"type": "string"
},
"key": {
"type": "string"
}
}
},
"securityContext": {
"description": "May be null / empty at any level: the chart then falls back to its hardening defaults (issue #99). This chart's fallback reproduces ITS defaults, deviations included - fsGroupChangePolicy: OnRootMismatch rather than the repo's Always, because an object store's data directory holds one file per block and a recursive chown on every pod start turns a restart into a multi-minute outage that grows with the store.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"pod": {
"description": "Merged onto the chart's pod hardening defaults and rendered as the pod's securityContext; kept open because the chart does not interpret the individual fields.",
"type": ["object", "null"]
},
"container": {
"description": "Merged onto the chart's container hardening defaults and rendered as the container's securityContext; kept open because the chart does not interpret the individual fields.",
"type": ["object", "null"]
}
}
},
"probe": {
"description": "Merged onto the chart's probe defaults and rendered as a container probe; kept open because the chart does not interpret the individual fields. May be null / empty, which means the defaults apply (issue #99) rather than the probe disappearing. The defaults carry the probe's CONTEXT: the port NAME 'admin' for all three, /health for readiness only, and a TCP connect for liveness and startup - because /health answers 503 until a cluster layout exists, and a probe that can only fail would restart a node forever.",
"type": ["object", "null"]
}
}
}
+437
View File
@@ -0,0 +1,437 @@
# Garage - an S3-compatible object store, deployed as a SINGLE NODE.
#
# READ THIS FIRST, because it is the premise every value below rests on
# (Gitea #81): this chart deploys ONE Garage node with replication_factor = 1,
# on the assumption that the volume layer underneath it is replicated and
# encrypted (longhorn-encrypted-ns in this cluster). Garage replication on top
# of that would be three times the raw data again, from two layers that know
# nothing about each other.
#
# WHAT THAT TRADE REALLY COSTS - named here because the chart cannot fix it:
# upstream says of replication_factor = 1 "Do not use this for anything else
# than test deployments", and the reason is that THE SAME FACTOR APPLIES TO THE
# METADATA. A corrupted metadata database is, in upstream's words, "generally
# not recoverable", and Longhorn does not help: it replicates the corrupted
# file three times. Redundancy against device failure, yes; against file
# corruption, no. What does help is the set of settings below -
# metadata_fsync, the automatic metadata snapshots, and those snapshots living
# on a second volume - and none of them is optional in this chart.
#
# THE IMAGE IS "FROM scratch" AND CONTAINS EXACTLY ONE FILE, /garage. No shell,
# no coreutils, no package manager. Three consequences that come up every time
# someone changes this chart:
# - exec probes are impossible here, for good. All three probes are HTTP or
# TCP (see the probe blocks below).
# - an init container would need a SECOND image to pin, track and update.
# Nothing needs one: Garage creates its own directories on start.
# - "kubectl exec" works, but only with /garage as the command and without a
# shell: kubectl exec <pod> -- /garage status
# Scale the whole chart down without uninstalling it: scaleDown: true renders
# the StatefulSet with replicas: 0 and takes precedence over garage.replicas.
# PVCs, Secrets, the Service and the Ingress stay in place; scaleDown: false
# brings the workload back up.
scaleDown: false
secrets:
# op:// path of the 1Password item carrying the field "rpc-secret": 32 bytes
# of random hex, i.e. "openssl rand -hex 32". REQUIRED - Garage uses it as
# the network key of its RPC layer and refuses to start without it, single
# node or not.
#
# IT ARRIVES AS AN ENVIRONMENT VARIABLE (GARAGE_RPC_SECRET), NOT AS A FILE,
# and that is the one place where this chart departs from how valkey does it.
# valkey argues explicitly for the file form - the password never appears in
# the process environment - and that argument is right in general. It cannot
# be followed here, and not because the file form would be inconvenient:
# IT IS STRUCTURALLY IMPOSSIBLE in a non-root pod.
#
# - Garage's rpc_secret_file rejects any file whose mode has a group OR
# other bit set: "if metadata.mode() & 0o077 != 0" in
# src/garage/secrets.rs (v2.4.1, line 144). The error message says
# "world-readable", but the mask tests the group bits as well.
# - The kubelet OR-s the group-read bit onto every file of a secret volume
# as soon as fsGroup is set: pkg/volume/volume_linux.go,
# changeFilePermission does Chmod(info.Mode() | 0440) for a read-only
# volume. So with fsGroup, Garage refuses to start.
# - Without fsGroup the file stays owned by root, and our uid 20000 cannot
# read it at all.
# - Kubernetes can set the GROUP of such a file, never its OWNER. There is
# no combination that satisfies both checks.
#
# The two ways around it were weighed and rejected (see Gitea #81):
# allow_world_readable_secrets = true would switch the permission check off
# for ALL of Garage's secret files, including ones nobody has set yet; and an
# init container copying the file with mode 0600 would need a second image
# for a scratch-based chart. What stays is the environment variable, which is
# also the dominant pattern in this repo (paperless-ngx, stalwart).
#
# The residual price, stated rather than hidden: the secret is in the
# container's process environment, so anyone who can exec into the pod or
# read its /proc can see it. Garage itself declares the variable with
# hide_env_values, so it does not print it in its own output.
#
# DO NOT "FIX" THIS BACK TO rpc_secret_file. The pod stops starting, and the
# error ("File /etc/garage-rpc/rpc-secret is world-readable!") points at a
# file mode nobody can change from inside Kubernetes.
rpcSecretRef: null
# op:// path of the 1Password item carrying the fields "access-key",
# "secret-key" and "bucket" - the S3 credential a CONSUMER uses, which is why
# it is a separate item from the RPC secret above: a consuming chart
# (stalwart's secrets.s3SecretRef) points at this same item, and keeping the
# bucket name in it means endpoint, bucket, key and secret have one source of
# truth instead of being kept in step by hand.
#
# Required while garage.singleNode is true together with defaultAccessKey or
# defaultBucket (the default), because Garage then IMPORTS these values on
# start. The direction is inwards: the vault item is written first, Garage
# adopts it, and nothing has to be copied out of the server afterwards.
#
# Garage validates all three and refuses to START on a malformed one:
# access-key at least 8 characters, ASCII alphanumeric plus - _ .
# secret-key at least 16 characters, printable ASCII
# bucket a valid S3 bucket name (3-63 characters, lowercase)
# NOTES.txt prints the generators for both keys.
#
# ROTATION IS NOT A RE-SAVE. Garage refuses to start when the access key
# already exists with a different secret ("is associated with a secret key
# different than the one given in GARAGE_DEFAULT_SECRET_KEY"), so rotating
# the secret means a NEW key id, or deleting the old key with the CLI first.
# NOTES.txt has the sequence.
s3SecretRef: null
garage:
# Replica count. 1 or 0 - and 0 is the documented way to stop the release
# without uninstalling it (the chart-global scaleDown flag does the same for
# everything at once).
#
# MORE THAN ONE REPLICA IS NOT A CLUSTER, which is why the schema caps this
# at 1: a second pod would come up as a lone node with its own empty metadata
# database on its own volume, because this chart renders neither
# bootstrap_peers nor the kubernetes_discovery CRD. With singleNode: true it
# would refuse to start at all once it saw the first node, which is the
# better of the two outcomes.
replicas: 1
# THE FIRST START, DONE BY GARAGE ITSELF INSTEAD OF BY HAND.
#
# --single-node makes the server create its own cluster layout on start
# (zone "dc1", capacity from the detected disk size) BEFORE it opens the S3,
# web and admin listeners. Without it, a fresh install has no layout, serves
# nothing, and stays not-Ready until an operator runs three CLI commands -
# which is a defensible way to run it, and NOTES.txt documents that path, but
# it is not a sane DEFAULT: a default install that never becomes Ready is not
# a default.
#
# IT CANNOT DAMAGE AN EXISTING CLUSTER, and that is worth knowing before
# trusting it with data. The layout version decides (src/garage/server.rs,
# initial_config): version 0 creates the layout, version 1 does nothing at
# all, anything above 1 aborts the start with "layout version is already
# superior to 1", and a node that sees other nodes aborts with "more nodes
# are already present in the cluster". It also requires replication_factor =
# 1 and says so if it is not. There is no --version argument here, so the
# trap of the CLI path - "never make multiple calls with the same value of
# the --version flag" - does not exist.
#
# Set to false to run the manual path (and then set defaultAccessKey and
# defaultBucket to false as well; the chart refuses the mixed combination,
# because Garage does).
singleNode: true
# Import the S3 access key from the vault item on start
# (--default-access-key, reading GARAGE_DEFAULT_ACCESS_KEY and
# GARAGE_DEFAULT_SECRET_KEY). Requires singleNode.
#
# This is what removes the one manual step that could not be automated in any
# other way: without it Garage GENERATES the key, and it would have to be
# copied out of the server into the vault by hand - a secret travelling in the
# wrong direction, through a terminal, with the vault as a copy rather than
# the original.
defaultAccessKey: true
# Create the bucket from the vault item's "bucket" field on start
# (--default-bucket) and grant the imported key read/write/owner on it.
# Implies defaultAccessKey upstream; requires singleNode.
defaultBucket: true
# ONE. Everything above is the reason, and raising it here is NOT an upgrade
# path: the factor is part of the cluster layout, so increasing it is a layout
# operation with data movement, on a cluster that would first need more nodes
# - which this chart does not deploy. The value exists so that a future
# multi-node setup is not blocked by the chart, not as a switch to flip.
replicationFactor: 1
# "lmdb" (Garage's default since 0.9 and the faster engine), "sqlite", or the
# experimental "fjall".
#
# The deviation to know about: upstream ties its LMDB recommendation to
# replication factor >= 2, and we run it at 1. The reported corruption after
# an unclean shutdown stays possible, metadata_fsync and the snapshots below
# only lower the probability, and at factor 1 there is no peer to recover
# from - only the last snapshot. sqlite is the defensible alternative for
# anyone who weighs a single-file database's durability higher than metadata
# throughput; with one mail server as the only consumer that ceiling is
# unlikely to be reached. Changing this on an EXISTING release needs
# "garage convert-db", not a values change.
dbEngine: lmdb
# ON, against Garage's own default of OFF. This is the single setting here
# most likely to be lost in a "clean up the obvious defaults" pass, so:
# Longhorn's block layer cannot help if the engine never calls fsync - a
# write that was never handed to the kernel is in no replica and in no
# snapshot. Garage's own example configuration pairs it with the snapshot
# interval below, as this chart does.
metadataFsync: true
# OFF, which is Garage's default and the right trade for the blocks: a lost
# block is one object, is detectable by a scrub, and is replaceable from the
# sender's copy; a lost metadata write can take the whole index with it.
# Turning it on costs an fsync per written block.
dataFsync: false
# THE ONLY CONSISTENT RECOVERY POINT THIS DEPLOYMENT HAS, and the reason the
# second PVC exists. File-system snapshots of a live metadata database
# guarantee nothing - Garage says so itself: a snapshot taken during a
# metadata write is corrupt and useless as a restore point, which is why
# Garage recommends its own mechanism in all cases. Longhorn backs up the
# blocks; it does not back up the metadata in any usable way.
#
# KNOW THE WINDOW BEFORE RELYING ON IT: Garage keeps exactly TWO snapshots,
# the first after interval/2 and every later one after interval times a
# random 1.0-1.2. At "6h" that is the first after ~3h and a restore window of
# roughly 6-14.4h afterwards. A corruption discovered later than that is
# older than every snapshot that still exists. Shorten the interval to narrow
# the window - at the cost of a compacted copy of the database every time.
#
# An empty string is rejected by the schema. A null falls back to this
# default, because switching the snapshots off is not a configuration this
# chart offers at replication factor 1.
metadataAutoSnapshotInterval: "6h"
# Garage's defaults, exposed because they are the two knobs that trade disk
# against CPU. blockSize is the chunk size objects are split into;
# compressionLevel is the zstd level for stored blocks (1 = fast, higher =
# smaller, negative levels are zstd's "fast" modes).
#
# TO DISABLE COMPRESSION, WRITE THE STRING "none" - and not 0, which is the
# trap this comment exists for: Garage's deserializer accepts an integer or
# the literal "none" (src/util/config.rs, deserialize_compression), and 0 is
# a perfectly valid integer that zstd reads as "use my default level", i.e.
# level 3 - MORE compression than the 1 below, not less. The schema therefore
# accepts an integer or exactly "none", and the ConfigMap quotes the string
# form so the rendered TOML stays valid.
blockSize: "1M"
compressionLevel: 1
# The host part of the S3, web and RPC listeners. "[::]" is upstream's
# default and accepts IPv4 as well, as long as net.ipv6.bindv6only is 0 -
# the Linux default, but a node sysctl this chart cannot see. Set "0.0.0.0"
# on an IPv4-only cluster that has changed it.
bindAddress: "[::]"
# The admin listener is bound on 0.0.0.0 SEPARATELY AND ON PURPOSE, because
# all three probes use it. If the kubelet's IPv4 probe could not reach a
# dual-stack listener, the symptom would be a pod that never becomes Ready
# for a reason nothing in the manifest points at. On an IPv6-only cluster set
# this to "[::]".
adminBindAddress: "0.0.0.0"
# The four ports, in ONE place: the ConfigMap renders them into garage.toml's
# bind addresses and the StatefulSet renders them as containerPorts, so a
# value here that disagreed with the other file would be a listener nothing
# routes to. 3900/3901/3902/3903 are upstream's defaults.
#
# The probes address the admin port by NAME ("admin"), not by number, so they
# follow a change here.
ports:
s3: 3900
rpc: 3901
web: 3902
admin: 3903
s3:
# The region name clients must use. Not a cloud region - a label both sides
# have to agree on; a consumer configured for a different one gets a
# signature error, not a redirect.
region: garage
# Optional suffix for vhost-style bucket access (<bucket>.<rootDomain>).
# Only useful together with an Ingress and a wildcard DNS entry;
# path-style access works without it and is what an in-cluster consumer
# uses. Deliberately NOT ingress.domain: the two are different things, and
# coupling them would make the Ingress host decide how buckets are
# addressed.
rootDomain: null
web:
# Static website hosting (the s3_web endpoint), OFF. It is a second HTTP
# listener, it serves nothing at all without a rootDomain, and the consumer
# this chart was built for speaks S3. Switching it on renders the
# containerPort, the Service port and the [s3_web] section.
enabled: false
rootDomain: null
resources:
# Limits are optional: memory and ephemeral-storage limits default to
# limitFactor x the request, an explicitly set limit always wins, and a CPU
# limit is only set when configured explicitly under "limits" - CPU is
# compressible, so leaving it unlimited throttles instead of OOM-killing.
#
# 512Mi x 3 = 1536Mi memory limit. Garage's own guidance is "at least 1GB"
# for a node, and LMDB's memory-mapped database makes the RESIDENT size
# follow the working set rather than the database size. The ephemeral
# request covers container logs only: everything Garage writes goes to one
# of the two PVCs.
limitFactor: 3
requests:
cpu: 100m
memory: 512Mi
ephemeral-storage: 32Mi
# LIVENESS: a TCP connect to the admin port, deliberately NOT the /health
# endpoint readiness uses. /health answers 503 while no cluster layout
# exists, so a liveness probe on it would kill and restart a node that is
# merely waiting for its layout - forever, and a restart is exactly what
# cannot fix that. Same lesson as zipline (README, guideline 12.3), in its
# sharpest form.
livenessProbe:
tcpSocket:
port: admin
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# READINESS: GET /health on the admin port - the one endpoint that answers
# the question a readiness probe asks.
#
# MEASURED AGAINST THE SOURCE AT v2.4.1, NOT CHOSEN: the S3 API server is
# spawned independently of the layout state, so the S3 socket binds and
# answers - with errors - before any layout exists. A tcpSocket probe on 3900
# would therefore report a service ready that accepts nothing. /health needs
# no token and maps the cluster health straight onto the status code: 200 for
# Healthy and Degraded, 503 for Unavailable, and an unconfigured layout is a
# special case that yields Unavailable.
#
# So with singleNode: true the pod becomes Ready a moment after start, and
# with singleNode: false it stays not-Ready until a layout is applied by
# hand. The second case is the truth and not a defect: until then the S3
# endpoint accepts nothing and no Service should route to it.
readinessProbe:
httpGet:
path: /health
port: admin
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# STARTUP: TCP again, for the same reason as liveness - a startup probe that
# fails forever kills the container, so it must not depend on a layout only
# an operator can create. The startup probe gates liveness and readiness,
# which is why those carry no initialDelaySeconds.
#
# Budget = periodSeconds * failureThreshold = 120s. It covers the slow part
# of a real start, which is not Garage: attaching and mounting two network
# volumes, plus opening (and after an unclean shutdown recovering) the LMDB
# database. Garage's own startup work after that is sub-second.
startupProbe:
tcpSocket:
port: admin
periodSeconds: 5
timeoutSeconds: 5
failureThreshold: 24
securityContext:
pod:
runAsNonRoot: true
# Any uid works: the image is FROM scratch, so there is no /etc/passwd to
# contradict and nothing in it is owned by a user. 20000 is this repo's
# default. Note that upstream's own chart uses 1000 - an install migrated
# from it would need chown'ed volumes or this value changed to 1000.
runAsUser: 20000
runAsGroup: 1000
fsGroup: 1000
# OnRootMismatch, DEVIATING from this repo's "Always" default, and the
# reason is the data: an object store's data directory holds one file per
# block, which is hundreds of thousands of files for a few hundred GB.
# "Always" chowns all of them recursively on EVERY pod start, which turns
# a restart into a multi-minute outage that gets worse as the store grows.
# OnRootMismatch checks the top directory's ownership and skips the walk
# when it already matches - which it does, from the second start onwards.
# Same reasoning as satisfactory's documented deviation.
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
container:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
# EMPTY, and it can stay empty: the ports are all above 1024, and the
# binary carries no file capabilities (it is a static Rust binary in a
# scratch image), so there is no execve-time capability check like
# stalwart's. Nothing here needs a capability.
add: []
# Additive environment entries for the Garage container, rendered through tpl
# and supporting the value, secretKeyRef and configMapKeyRef forms.
#
# ADDITIVE MEANS ADDITIVE: an entry repeating a name the chart already sets
# does not replace it. At install time both reach the pod and the later one
# wins; on an EXISTING release the pair collapses to the chart's own value,
# because Kubernetes merges env by name - and helm upgrade still reports
# success. So do not use this to override GARAGE_RPC_SECRET or the
# GARAGE_DEFAULT_* variables; change the values that produce them.
#
# Useful entries that are genuinely additive: RUST_LOG (the image sets
# garage=info) and GARAGE_ALLOW_PUNYCODE.
env: []
storage:
# Both blocks are deliberately NOT null-safe: an erased size yields an
# invalid PVC that is rejected at apply time, which is loud. A chart-side
# fallback would silently size somebody's object store - and on an existing
# release it could not help anyway, because a PVC cannot shrink.
data:
# metadata_dir AND data_dir, as two subdirectories of this ONE volume.
# Together on purpose: Longhorn snapshots are atomic per volume and never
# across volumes, so this is what keeps index and blocks consistent with
# each other in a snapshot. 10Gi is a starting size for a mail blob store,
# not a recommendation - it is the value most likely to need changing
# before the first install.
amount: 10Gi
# null uses the cluster's default StorageClass. In this cluster the
# intended value is longhorn-encrypted-ns, which is also where the backup
# story lives - and note what this chart does NOT render: the Longhorn
# recurring-job labels (daily-snapshot, daily-backup) belong to the
# consuming bundle, and without them the off-site half of the backup plan
# does not exist, it only looks like it does.
storageClass: null
snapshots:
# metadata_snapshots_dir. Sized for TWO compacted copies of the metadata
# database plus head room, not for the objects - Garage keeps exactly two
# and deletes older ones itself.
amount: 5Gi
storageClass: null
ingress:
# OFF. The consumer this chart was built for speaks S3 inside the cluster, so
# there is nothing to publish. The values exist, built to this repo's
# convention, so that publishing the endpoint later is a values change.
#
# Guideline 1, checked in both directions: nothing outside the Ingress
# template reads ingress.domain (the vhost suffix is its own value,
# garage.s3.rootDomain), so switching the Ingress off frees BOTH domain and
# clusterIssuer. With it on, both are required and the required() fires in
# the template.
enabled: false
clusterIssuer: null
domain: null
className: nginx
# Extra annotations for the Ingress; merged with the cert-manager
# cluster-issuer annotation the chart always sets. An S3 endpoint behind
# nginx usually wants nginx.ingress.kubernetes.io/proxy-body-size: "0" -
# without it the controller's 1m default truncates uploads.
annotations: {}
+50
View File
@@ -0,0 +1,50 @@
apiVersion: v2
name: keycloak
description: A Helm chart to deploy Keycloak, an OpenID Connect and SAML identity provider, as a two-node cluster on Kubernetes
icon: https://www.keycloak.org/resources/images/icon.svg
type: application
# appVersion is the exact image tag of quay.io/keycloak/keycloak. No "v"
# prefix here - upstream tags are plain three-part versions (26.8.0).
#
# THE TAG LANDSCAPE, AND WHY THIS CHART NEEDS NO VERSION CONSTRAINT. Measured
# on 2026-10-04 against quay.io's tag API, this registry carries four shapes
# side by side:
#
# 1. release tags 26.8.0, 26.7.5, ... <- what we track
# 2. REBUILD tags 26.8.0-0, 26.7.3-1, 26.7.2-2 - the same app
# version rebuilt (base image or CVE fix) without an app-version bump.
# 26.8.0-0 resolves to the SAME manifest digest as 26.8.0.
# 3. floating minor tags 26.8, 26.7, 26.6, ... - they move
# 4. floating names latest, nightly - they move daily
#
# This is the same GENUS of trap as garage's commit-SHA tags and stalwart's
# moving names, and it is the one case in this repo where the answer came out
# the other way: plain `docker` versioning already excludes all three unwanted
# shapes, so the renovate.json5 block carries NO versioningTemplate. Docker
# versioning splits a tag at the first dash and only ever offers candidates
# with the SAME suffix - our value has none, so the -N rebuilds are invisible
# to it - and 26.8 fails on the missing third part while latest/nightly parse
# as no version at all.
#
# THAT IS MEASURED, NOT ASSUMED, and the measurement is written down because
# the next round will otherwise add a constraint "to be safe" (renovate 44.132.5,
# --platform=local --dry-run=lookup against the real registry):
# appVersion 26.8.0 -> updates [], warnings []
# appVersion 26.7.0 -> exactly one: 26.8.0 (minor)
# appVersion 26.7.4 -> exactly two: 26.7.5 (patch), 26.8.0 (minor)
# No run offered a rebuild tag, a floating minor, latest or nightly.
#
# AND THE OTHER HALF, WHICH THE VERSION NUMBER CANNOT CARRY: Keycloak supports
# a rolling update only WITHIN a minor line. A minor jump needs the cluster
# shut down. The global `automerge: true` in renovate.json5 applies here
# anyway - a deliberate acceptance of that risk, not an oversight - so the
# procedure lives in NOTES.txt and on the wiki page instead of in a
# packageRules entry. Read the "MINOR UPGRADES" section of NOTES.txt before
# merging an app-version PR that changes the minor.
version: 1.1.0+up26.8.0
appVersion: "26.8.0"
maintainers:
- name: jklein
+204
View File
@@ -0,0 +1,204 @@
{{- $settings := fromYaml (include "keycloak.settings" .) -}}
{{- $replicas := int (include "keycloak.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.keycloak.replicas)) -}}
{{- $adminHostname := eq (include "keycloak.adminHostnameEnabled" .) "true" -}}
{{- $deploy := printf "%s-%s-deployment" .Release.Name .Chart.Name -}}
{{- $svc := printf "%s-%s-service" .Release.Name .Chart.Name -}}
Keycloak {{ .Chart.AppVersion }} installed as release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
{{ $replicas }} replica(s), cache=ispn with jdbc-ping discovery over the database,
external PostgreSQL at {{ $settings.db.host | default "<keycloak.db.host NOT SET>" }}:{{ $settings.db.port | int64 }}/{{ $settings.db.database }}.
ENDPOINTS
{{ if .Values.ingress.enabled }} Public: {{ include "keycloak.frontendUrl" . }}
{{ else }} Public: no Ingress (ingress.enabled: false) - in-cluster only
{{ end }}{{ if $adminHostname }} Admin: {{ include "keycloak.adminUrl" . }}/admin
{{ else }} Admin: no admin hostname (ingress.admin.enabled: false) - the console is
served on the public host and Keycloak generates its URLs for it
{{ end }} In-cluster: http://{{ $svc }}.{{ .Release.Namespace }}.svc.cluster.local:{{ $settings.ports.http | int64 }}
Management port {{ $settings.ports.management | int64 }} is NOT published and must never be: it serves
/health, /health/live, /health/ready, /health/started{{ if $settings.metrics.enabled }} and /metrics{{ end }}
WITHOUT authentication. It has a ClusterIP Service
({{ .Release.Name }}-{{ .Chart.Name }}-management-service) for in-cluster scraping only.
THE 1PASSWORD ITEMS, AND WHICH FIELDS THEY MUST CARRY
The operator materializes one Secret key per item FIELD, so a missing or
misnamed field leaves the pod in CreateContainerConfigError - with no message
naming the field.
{{ .Values.secrets.databaseSecretRef | default "<secrets.databaseSecretRef NOT SET>" }}
username the PostgreSQL role
password that role's password
The role must be able to CREATE its schema: Keycloak builds and migrates
the whole schema itself on first start.
{{ if $settings.bootstrapAdmin.enabled }}
{{ .Values.secrets.bootstrapAdminSecretRef | default "<secrets.bootstrapAdminSecretRef NOT SET>" }}
username the TEMPORARY bootstrap administrator (upstream default name:
temp-admin)
password its password
{{ else }}
(no bootstrap admin item - keycloak.bootstrapAdmin.enabled is false, which is
the correct state once a permanent administrator exists)
{{ end }}
Both arrive as ENVIRONMENT VARIABLES, not as files, and that is forced:
Keycloak 26 has no _FILE form for an option's value. The price: the password
is in the container's process environment.
FIRST INSTALL - AND STEP 4 IS MANDATORY, NOT ADVICE
1. kubectl -n {{ .Release.Namespace }} rollout status deployment/{{ $deploy }}
Ready means /health/ready answered 200, which means the database is
reachable and the schema is in place. The first start is the slow one: it
creates the entire schema through Liquibase, and with {{ $replicas }} replica(s)
the second waits on the first's migration lock.
2. Check that the cluster really formed. THIS STEP IS NOT OPTIONAL AND NO
PROBE REPLACES IT: a node that never found its peer is Ready, healthy and
alone, and two such nodes are two islands that each believe they are the
cluster. The install-test cannot catch it either.
kubectl -n {{ .Release.Namespace }} logs deployment/{{ $deploy }} | grep -i "ISPN000094\|view accepted\|members"
Look for a view with {{ $replicas }} members. One member per pod means
jdbc-ping did not find the peer - check that both pods can reach each
other on port {{ $settings.ports.jgroups | int64 }} and that they really share one database.
3. Log in at {{ if $adminHostname }}{{ include "keycloak.adminUrl" . }}/admin{{ else if .Values.ingress.enabled }}{{ include "keycloak.frontendUrl" . }}/admin{{ else }}the in-cluster address, port-forwarded{{ end }}
with the bootstrap credentials from the vault item above.
4. CREATE A PERMANENT ADMINISTRATOR, then remove the temporary one.
The bootstrap account is TEMPORARY by design - upstream: "the account
should exist only for the duration necessary to perform operations needed
to gain permanent and more secure admin access. After that, the account
needs to be removed manually." This is different from Keycloak before 26,
where KEYCLOAK_ADMIN created a regular administrator and there was nothing
to clean up.
a) In the master realm, create a user, set a password, assign the
realm-admin / admin role.
b) Verify you can log in as that user.
c) Delete the temp-admin user.
d) helm upgrade with keycloak.bootstrapAdmin.enabled: false
5. Until step 4d, EVERY START LOGS a line about the initial administrator not
being created, because the master realm already exists and the bootstrap
path is skipped. It is harmless. It is also the kind of message somebody
eventually investigates, which is why 4d is part of the install and not a
tidy-up.
MINOR UPGRADES NEED THE CLUSTER DOWN - AND THE AUTOMERGE DOES NOT KNOW THAT
Keycloak supports a rolling update only WITHIN a minor line. A minor jump
(26.8 -> 26.9) requires every node shut down before any new node starts;
mixing two minor versions in one Infinispan cluster is not a supported
configuration, and nothing in the cluster reports that it happened.
THIS REPOSITORY'S RENOVATE AUTOMERGES EVERY UPDATE TYPE, MAJORS INCLUDED.
That is a deliberate, accepted risk and not an oversight (Gitea #83), but it
means an app-version PR that moves the minor can reach main, publish the
chart and be rolled out as a rolling update without anybody deciding to.
Watch app-version PRs for a changed minor, and if one has landed, take this
path rather than letting the rollout happen:
1. Decide it with the tool instead of the version number. With the OLD
version and the OLD configuration:
kc.sh update-compatibility metadata --file=/tmp/kc-compat.json
then with the NEW version and the NEW configuration:
kc.sh update-compatibility check --file=/tmp/kc-compat.json
Pass EVERY configuration option on both calls - upstream: "Omitting any
configuration options results in incomplete metadata, and could lead to
a wrong reported result".
Exit codes, which is what to automate on: 0 rolling update possible,
1 unexpected error, 2 invalid CLI option, 3 and 4 not possible - the
deployment must be shut down.
2. If it says shut down:
helm upgrade ... --set scaleDown=true # every pod stops
# back up the database AND the configuration before anything else
helm upgrade ... --set scaleDown=false # with the new version
A NOTE ON WHAT IS NOT SETTLED: upstream's 26.6 release notes call patch-level
rolling updates supported and active by default, while the 26.8.0 page for
update-compatibility says a rolling update is possible "when the Keycloak
version is the same" and uses a PATCH pair (26.2.0 -> 26.2.1) as its example
of an incompatible one. Those two cannot both be current. Until it is clear,
run the check for patch updates too and trust its exit code rather than the
size of the version step.
TRAPS WORTH KNOWING BEFORE YOU NEED THEM
- THE ADMIN HOSTNAME IS NOT AN ACCESS CONTROL. hostname-admin decides which
URLs the server generates, nothing more. Upstream: "Using the
hostname-admin option does not prevent accessing the Administration REST
API endpoints via the frontend URL specified by the hostname option. If you
want to restrict access to the Administration REST API, you need to do it
on the reverse proxy level." So /admin/realms/... is reachable through the
public host whatever you put on the admin host, and an IP allow-list on the
admin Ingress restricts nothing on its own. This chart renders no deny rule
for /admin - see values.yaml for why not.
- A SHARED DATABASE MERGES TWO CLUSTERS. Node discovery runs through the
database, so - upstream - "clusters that share the same database but should
remain separate will merge unless they have distinct cluster names. The
default cluster name is ISPN." Two releases of this chart on one database
therefore form ONE cluster, silently.
{{ with $settings.cache.clusterName }}This release sets cache-embedded-cluster-name={{ . }}.{{ else }}This release uses the default name (ISPN) - keycloak.cache.clusterName is
unset. Read its comment in values.yaml before setting it: upstream also
warns about cache invalidation between differently-named clusters.{{ end }}
- READINESS ONLY CHECKS THE DATABASE WHILE metrics-enabled IS TRUE. Upstream's
table of health checks has a "Requires Metrics" column and the DATABASE
check is the row that says yes. So with keycloak.metrics.enabled false,
/health/ready answers 200 with the database gone and the pod keeps taking
logins it cannot serve. This release has it {{ if $settings.metrics.enabled }}ON, which is the chart default
and the reason the default is what it is.{{ else }}OFF, so the readiness probe
cannot currently detect a database outage.{{ end }}
- THERE IS NO ConfigMap IN THIS CHART, so no checksum/config annotation. That
is not an omission: Keycloak is configured entirely through options and
KC_* variables, so there is no config file whose change could fail to
restart anything.
- A CHANGE INSIDE A VAULT ITEM RESTARTS NOTHING. Helm never sees it. After
rotating a field:
kubectl -n {{ .Release.Namespace }} rollout restart deployment/{{ $deploy }}
(Repointing a secrets.*SecretRef at a different item DOES roll the pods -
that is covered by checksum/secret-ref.)
- THE POD HAS TWO INIT CONTAINERS AND NEEDS BOTH. db, health-enabled and
metrics-enabled are Keycloak BUILD options, the published image is built
with their defaults, and a runtime change makes Keycloak rewrite
/opt/keycloak/lib/quarkus at start - which readOnlyRootFilesystem: true
forbids. So one init container seeds that directory into a volume and the
next runs `kc.sh build` into it; the serving container then runs
`start --optimized`. If a pod is stuck in Init, read those logs first:
kubectl -n {{ .Release.Namespace }} logs <pod> -c {{ .Release.Name }}-{{ .Chart.Name }}-build
Build options are set ONLY through keycloak.db.vendor,
keycloak.metrics.enabled and keycloak.features. Putting one into
keycloak.env fails the render on purpose.
- POOL SIZES ARE PER REPLICA. keycloak.db.pool.maxSize is {{ $settings.db.pool.maxSize | int64 }}, so with the
current {{ $replicas }} replica(s) this release can open up to {{ mul ($settings.db.pool.maxSize | int64) $replicas }} connections.
Upstream's default is 100 per node, which with two nodes exceeds a default
PostgreSQL's max_connections of 100. Raise it together with the server,
never alone.
{{ if and $settings.podDisruptionBudget.enabled (ge $replicas 2) }}
- A PodDisruptionBudget IS in place ({{ .Release.Name }}-{{ .Chart.Name }}-pdb, minAvailable
{{ $settings.podDisruptionBudget.minAvailable | int64 }}), so a node drain cannot take all replicas at once.
{{ else if $settings.podDisruptionBudget.enabled }}
- NO PodDisruptionBudget was rendered, because the effective replica count is
{{ $replicas }}. That is deliberate: minAvailable 1 with a single replica forbids every
voluntary eviction, so `kubectl drain` would block forever on a pod nothing
may move - and the error would name the PDB rather than the replica count.
Run two replicas to get the budget.
{{ else }}
- NO PodDisruptionBudget (keycloak.podDisruptionBudget.enabled is false). A
node drain may take every replica at once.
{{ end }}
WHAT THIS CHART DOES NOT RENDER
- No PVC and no StatefulSet: Keycloak 26 keeps everything in the database,
user sessions included (persistent sessions are the default; volatile ones
are deprecated as of 26.8). There is no disk state to keep.
- No NetworkPolicy. jdbc-ping opens direct pod-to-pod connections on port
{{ $settings.ports.jgroups | int64 }}, so any policy added later has to allow that between the
replicas, or the cluster silently becomes two islands.
- No deny rule for /admin on the public Ingress, and no restriction of the
administration REST API. That is reverse-proxy work - see the first trap.
+545
View File
@@ -0,0 +1,545 @@
{{/*
Render the body of a container "resources:" block from a values dict of the form
resources:
limitFactor: 3 # optional, default 3
requests: { cpu, memory, ephemeral-storage / ephemeralStorage, ... }
limits: { ... } # optional, every entry optional
Rules (variant B, see issue #19):
- requests are rendered as given (camelCase "ephemeralStorage" is normalized
to the Kubernetes key "ephemeral-storage").
- an explicitly set limit always wins.
- a missing memory / ephemeral-storage limit is computed as
limitFactor x the corresponding request.
- a missing cpu limit is NOT set at all (the kubelet then treats CPU as
unlimited; CPU is compressible, so this cannot kill the workload).
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart, and a dependency is admissible but expensive (README, guideline 14) -
this chart takes none. The reference implementation is
template-chart/templates/_helpers.tpl.
*/}}
{{- define "keycloak.resources" -}}
{{- $resources := . | default dict -}}
{{- $factor := 3.0 -}}
{{- if not (kindIs "invalid" $resources.limitFactor) -}}
{{- $factor = float64 $resources.limitFactor -}}
{{- end -}}
{{- $requests := dict -}}
{{- range $key, $value := ($resources.requests | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $requests (include "keycloak.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- $limits := dict -}}
{{- range $key, $value := ($resources.limits | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $limits (include "keycloak.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- range $key, $request := $requests -}}
{{- if and (ne $key "cpu") (not (hasKey $limits $key)) -}}
{{- $_ := set $limits $key (include "keycloak.multiplyQuantity" (dict "quantity" $request "factor" $factor)) -}}
{{- end -}}
{{- end -}}
{{- $out := dict -}}
{{- with $requests }}{{- $_ := set $out "requests" . }}{{- end -}}
{{- with $limits }}{{- $_ := set $out "limits" . }}{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Normalize a resource name from values to its Kubernetes key.
*/}}
{{- define "keycloak.resourceKey" -}}
{{- if eq . "ephemeralStorage" -}}ephemeral-storage{{- else -}}{{ . }}{{- end -}}
{{- end -}}
{{/*
Multiply a Kubernetes quantity by a factor, keeping its suffix.
Input: dict "quantity" <string|number> "factor" <float64>.
*/}}
{{- define "keycloak.multiplyQuantity" -}}
{{- $quantity := include "keycloak.formatNumber" .quantity -}}
{{- $number := regexFind "^[0-9]+(\\.[0-9]+)?" $quantity -}}
{{- if not $number -}}
{{- fail (printf "cannot parse resource quantity %q" $quantity) -}}
{{- end -}}
{{- $suffix := trimPrefix $number $quantity -}}
{{- if not (regexMatch "^(m|k|M|G|T|P|E|Ki|Mi|Gi|Ti|Pi|Ei)?$" $suffix) -}}
{{- fail (printf "unsupported resource quantity suffix %q in %q" $suffix $quantity) -}}
{{- end -}}
{{- $product := mulf (float64 $number) .factor -}}
{{- if eq $product (floor $product) -}}
{{- printf "%d%s" (int64 $product) $suffix -}}
{{- else -}}
{{- printf "%s%s" (include "keycloak.formatNumber" $product) $suffix -}}
{{- end -}}
{{- end -}}
{{/*
Render a number as a plain decimal string, never in exponential notation
(issue #157): the null-safe fallback helpers pass values through fromYaml,
which parses every number as float64, and Go's "%v" switches float64 to
exponential notation from about 1e6 upwards - "1.048576e+06" is not a valid
Kubernetes quantity. Strings ("700Mi") are passed through untouched.
*/}}
{{- define "keycloak.formatNumber" -}}
{{- if kindIs "float64" . -}}
{{- regexReplaceAll "\\.?0+$" (printf "%.10f" .) "" -}}
{{- else -}}
{{- printf "%v" . -}}
{{- end -}}
{{- end -}}
{{/*
Merge a user-supplied values sub-tree onto a dict of chart defaults so that a
null / empty override falls back to the defaults instead of erasing them
(issues #61, #99). Input: dict "defaults" <dict> "given" <dict|nil> "path"
<string>. Output: YAML of the merged dict (consume it with fromYaml).
An override key written without children is YAML null, and Helm treats that
null as a deletion of the chart's default. This helper defines the opposite
semantics - an empty block means "chart defaults apply" - while an explicitly
set value still wins, including false and 0.
*/}}
{{- define "keycloak.defaultedDict" -}}
{{- $defaults := .defaults | default dict -}}
{{- $given := .given -}}
{{- $path := .path | default "value" -}}
{{- if kindIs "invalid" $given -}}
{{- $given = dict -}}
{{- end -}}
{{- if not (kindIs "map" $given) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" $path (kindOf $given) $given) -}}
{{- end -}}
{{- $out := deepCopy $defaults -}}
{{- range $key, $value := $given -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $default := index $defaults $key -}}
{{- if and (kindIs "map" $value) (kindIs "map" $default) -}}
{{- $_ := set $out $key (fromYaml (include "keycloak.defaultedDict" (dict "defaults" $default "given" $value "path" (printf "%s.%s" $path $key)))) -}}
{{- else if kindIs "map" $default -}}
{{- fail (printf "%s.%s must be a mapping or null, got %s (%v)" $path $key (kindOf $value) $value) -}}
{{- else -}}
{{- $_ := set $out $key $value -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Pick one sub-block out of a values block that may itself be null.
Input: dict "block" <dict|nil> "key" <string> "path" <string>.
Returns YAML of dict "value" <the sub-block, may be nil>.
*/}}
{{- define "keycloak.subBlock" -}}
{{- $block := .block -}}
{{- if kindIs "invalid" $block -}}
{{- $block = dict -}}
{{- end -}}
{{- if not (kindIs "map" $block) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" .path (kindOf $block) $block) -}}
{{- end -}}
{{- toYaml (dict "value" (index $block .key)) -}}
{{- end -}}
{{/*
THE EFFECTIVE KEYCLOAK SETTINGS - everything the Deployment, the Services, the
PodDisruptionBudget and NOTES.txt read out of .Values.keycloak, merged onto this
chart's defaults.
Why the whole settings block goes through the null-safe helper and not only the
three usual suspects (guideline 12.2: "the rule is about the shape, not about
those three block kinds"): every value below is dereferenced by a template, and
an erased one would not abort the render - it would render a containerPort with
no number, a `--db=` with an empty right-hand side, or a database host that is
the empty string. The first is rejected by the API server at apply time, the
second fails the BUILD inside the init container, and the third is a pod waiting
on a connection that cannot be made. All three arrive after the render looked
fine, which is exactly the class #99 is about. satisfactory shipped the same
shape with half its blocks null-safe and half not (Gitea #16), and the half that
was not took the Service's port with it.
The defaults here MUST stay in sync with values.yaml, which remains the
documented, user-facing place for them; this copy is only the fallback.
Usage: {{ $settings := fromYaml (include "keycloak.settings" .) }}
*/}}
{{- define "keycloak.settings" -}}
{{- $given := (fromYaml (include "keycloak.subBlock" (dict "block" .Values "key" "keycloak" "path" ".Values"))).value -}}
{{- $defaults := dict
"replicas" 2
"db" (dict
"vendor" "postgres"
"host" nil
"port" 5432
"database" "keycloak"
"pool" (dict "initialSize" 2 "minSize" 2 "maxSize" 20))
"bootstrapAdmin" (dict "enabled" true)
"metrics" (dict "enabled" true)
"features" (list)
"cache" (dict "clusterName" nil)
"ports" (dict "http" 8080 "management" 9000 "jgroups" 7800)
"podDisruptionBudget" (dict "enabled" true "minAvailable" 1) -}}
{{- include "keycloak.defaultedDict" (dict "defaults" $defaults "given" $given "path" "keycloak") -}}
{{- end -}}
{{/*
Render the replica count for the Deployment.
Input: dict "scaleDown" <flag|nil> "replicas" <value|nil>.
- scaleDown: true takes precedence and always yields 0.
- otherwise an explicitly set replicas value is used as given, including an
explicit 0 - Helm's "default" would swallow 0, hence the kindIs "invalid"
nil check.
- a missing replicas value defaults to 2.
A NOTE SPECIFIC TO THIS CHART: the default is 2, not 1, and that is the point
of the chart rather than a tuning choice - an SSO outage takes every dependent
login with it. Two replicas are cheap here because node discovery runs over the
database that is there anyway (cache=ispn plus Keycloak's default jdbc-ping
stack). What two replicas cost is the rolling-update restriction: upstream
supports a rolling update only inside a minor line, so a minor jump becomes a
procedure instead of a restart. See NOTES.txt.
*/}}
{{- define "keycloak.replicas" -}}
{{- if .scaleDown -}}
0
{{- else if not (kindIs "invalid" .replicas) -}}
{{- .replicas -}}
{{- else -}}
2
{{- end -}}
{{- end -}}
{{/*
THE BUILD OPTIONS, IN ONE PLACE - and this helper being the ONLY place is what
keeps the chart startable (Gitea #83).
Keycloak separates BUILD options from runtime options. Verified against the
26.8.0 "All configuration" table, where a build option carries the tool icon:
db, health-enabled, metrics-enabled, features and features-disabled are build
options; db-url-host, db-username, db-password, the db-pool-* sizes, hostname,
hostname-admin, hostname-strict, proxy-headers, http-enabled, http-port,
http-management-port, cache, cache-stack and the cache-embedded-* options are
not.
The published image is augmented with the build options' DEFAULTS (db=dev-file,
health-enabled=false, metrics-enabled=false). Supplying different ones at
runtime makes Keycloak re-augment itself at start, and the augmentation
rewrites /opt/keycloak/lib/quarkus - inside the root filesystem, which
readOnlyRootFilesystem: true makes read-only. Upstream's container guide calls
that start what it is: it "creates an image that is mutable".
So the BUILD init container receives exactly these options and the serving
container runs `start --optimized` and passes NO build option at all. That
asymmetry is deliberate: with `--optimized` Keycloak expects a pre-built
installation and a build option handed to it alongside is an error, and - more
importantly - there is no second list that could drift out of step with this
one. If a build option must be added, it is added HERE and nowhere else.
THEY ARE RENDERED AS ENVIRONMENT VARIABLES, NOT AS CLI ARGUMENTS, and that is a
safety decision rather than a style one. `kc.sh build` reads KC_* exactly as it
reads flags, and as variables the values are handed over by the kubelet instead
of being parsed by a shell - so a feature name containing a space cannot become
two arguments, and nothing here has to be shell-quoted.
health-enabled is hard-coded true rather than exposed as a value: all three
probes address /health/* on the management port, so a release with it switched
off would have no working probe at all - the same defect teamspeak-server's
port name exists to prevent (guideline 12.5).
*/}}
{{- define "keycloak.buildEnv" -}}
{{- $settings := fromYaml (include "keycloak.settings" .) -}}
- name: KC_DB
value: {{ required "A database vendor must be provided (keycloak.db.vendor)" $settings.db.vendor | quote }}
- name: KC_HEALTH_ENABLED
value: "true"
- name: KC_METRICS_ENABLED
value: {{ $settings.metrics.enabled | toString | quote }}
{{- with $settings.features }}
- name: KC_FEATURES
value: {{ join "," . | quote }}
{{- end -}}
{{- end -}}
{{/*
The variable names of Keycloak BUILD options, used to refuse them in the
additive env block (see keycloak.env below).
Read off the 26.8.0 option table. It is not claimed to be exhaustive - upstream
can add a build option tomorrow and this list would not know - which is why it
is the SECOND line of defence and not the first. The first is structural: the
serving container is rendered with `start --optimized` and no build option, so
there is no list of options on it to disagree with the init container's. This
guard only stops somebody re-introducing one through values.
*/}}
{{- define "keycloak.buildOptionEnvNames" -}}
- KC_DB
- KC_DB_KIND
- KC_DB_DRIVER
- KC_HEALTH_ENABLED
- KC_HTTP_MANAGEMENT_HEALTH_ENABLED
- KC_METRICS_ENABLED
- KC_FEATURES
- KC_FEATURES_DISABLED
- KC_HTTP_RELATIVE_PATH
- KC_HTTP_MANAGEMENT_RELATIVE_PATH
- KC_HTTPS_CLIENT_AUTH
- KC_HTTPS_MANAGEMENT_CLIENT_AUTH
- KC_TRANSACTION_XA_ENABLED
- KC_LEGACY_OBSERVABILITY_INTERFACE
{{- end -}}
{{/*
The frontend URL (KC_HOSTNAME) and the admin URL (KC_HOSTNAME_ADMIN).
BOTH ARE FULL URLs, NOT BARE HOSTS, and that is upstream's requirement: "If
hostname-admin is configured, hostname must be set to a URL (not just
hostname). Otherwise Keycloak would not know what is the correct frontend URL
(incl. port etc.)", and both values are "verified that full URL is used, incl.
scheme and hostname". https:// is the only scheme this chart renders - the
Ingress always terminates TLS, so the browser-facing scheme is https even
though the pod itself serves plain HTTP.
ingress.domain is required HERE and not only in the Ingress template, because
this is the read that makes it required even with ingress.enabled: false
(guideline 12.1): Keycloak refuses to start in production mode without a
hostname, hostname-strict being true by default.
*/}}
{{- define "keycloak.frontendUrl" -}}
https://{{ required "A Domain/Host must be provided (ingress.domain) - Keycloak derives KC_HOSTNAME from it and refuses to start without a hostname, so it stays required even with ingress.enabled: false" .Values.ingress.domain }}
{{- end -}}
{{- define "keycloak.adminUrl" -}}
https://{{ required "An admin Domain/Host must be provided (ingress.admin.domain) while ingress.admin.enabled is true" (((.Values.ingress | default dict).admin | default dict).domain) }}
{{- end -}}
{{/*
Is the admin hostname in play? Only then are KC_HOSTNAME_ADMIN and the second
Ingress rendered.
WHAT IT IS NOT: an access control. hostname-admin decides which URLs the server
GENERATES; upstream states plainly that it "does not prevent accessing the
Administration REST API endpoints via the frontend URL specified by the hostname
option" and that restricting that API is reverse-proxy work. The full reasoning
sits at ingress.admin.enabled in values.yaml, and the same sentence is in
NOTES.txt and on the wiki page - three places on purpose, because the value's
name suggests a boundary and the next reader will assume one.
*/}}
{{- define "keycloak.adminHostnameEnabled" -}}
{{- $ingress := .Values.ingress | default dict -}}
{{- $admin := $ingress.admin | default dict -}}
{{- if kindIs "invalid" $admin.enabled -}}
true
{{- else -}}
{{- $admin.enabled -}}
{{- end -}}
{{- end -}}
{{/*
The effective security contexts, probes and resources (issue #99).
The defaults below MUST stay in sync with values.yaml. They reproduce THIS
chart's defaults, not the repo standard - see the uid/gid note.
*/}}
{{- define "keycloak.podSecurityContext" -}}
{{- $given := (fromYaml (include "keycloak.subBlock" (dict "block" . "key" "pod" "path" "keycloak.securityContext"))).value -}}
{{- $defaults := dict
"runAsNonRoot" true
"runAsUser" 1000
"runAsGroup" 1000
"fsGroup" 1000
"fsGroupChangePolicy" "Always"
"seccompProfile" (dict "type" "RuntimeDefault") -}}
{{- include "keycloak.defaultedDict" (dict "defaults" $defaults "given" $given "path" "keycloak.securityContext.pod") -}}
{{- end -}}
{{- define "keycloak.containerSecurityContext" -}}
{{- $given := (fromYaml (include "keycloak.subBlock" (dict "block" . "key" "container" "path" "keycloak.securityContext"))).value -}}
{{- $defaults := dict
"allowPrivilegeEscalation" false
"readOnlyRootFilesystem" true
"runAsNonRoot" true
"capabilities" (dict "drop" (list "ALL") "add" (list)) -}}
{{- include "keycloak.defaultedDict" (dict "defaults" $defaults "given" $given "path" "keycloak.securityContext.container") -}}
{{- end -}}
{{/*
LIVENESS: /health/live on the management port, and deliberately NOT /health or
/health/ready.
Guideline 12.3 in the one shape where upstream has already separated the
checks: /health/live reports whether this process is alive and contains no
database check, while /health/ready and the aggregate /health do. A liveness
probe on either of those restarts every pod in the cluster for as long as the
database is away - which cannot fix a database, and which turns a degraded login
service into no login service. Same lesson as zipline; here it is one path
away, which is also why the wrong path would look entirely plausible in a diff.
*/}}
{{- define "keycloak.livenessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/health/live" "port" "management")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "keycloak.defaultedDict" (dict "defaults" $defaults "given" . "path" "keycloak.livenessProbe") -}}
{{- end -}}
{{/*
READINESS: /health/ready - which includes the database, and must. A node that
cannot reach its database answers every login with an error, so no Service
should route to it. This is the other half of guideline 12.3: a dependency-free
liveness path is only correct when readiness really does hang on the
dependency.
AND THE PATH ALONE DOES NOT GUARANTEE THAT, which is the part worth reading
before anybody "cleans up" keycloak.metrics.enabled. Upstream's table of
available checks carries a "Requires Metrics" column, and the DATABASE check -
"Returns the status of the database connection pool" - is the row that says
YES; Cluster, Graceful shutdown and Keycloak Initialized say no. So with
metrics-enabled=false this endpoint still answers 200 while the database is
gone, and the pod keeps taking logins it cannot serve.
That is exactly the urlaubsverwaltung shape (#218): the probe path was right
and the aggregate behind it contained nothing external. Here the dependency
hides behind an option whose name says observability, which is why
keycloak.metrics.enabled defaults to TRUE and why its comment in values.yaml
argues about readiness rather than about scraping.
*/}}
{{- define "keycloak.readinessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/health/ready" "port" "management")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "keycloak.defaultedDict" (dict "defaults" $defaults "given" . "path" "keycloak.readinessProbe") -}}
{{- end -}}
{{/*
STARTUP: /health/started, Keycloak's own "finished booting" endpoint. It gates
liveness and readiness, which is why neither carries an initialDelaySeconds.
Budget = periodSeconds * failureThreshold = 10 * 60 = 600s. What has to fit in
it is not only the JVM and Infinispan: on a FIRST install Keycloak creates its
whole schema through Liquibase, and with two replicas starting together the
second waits on the first's migration lock. Too short a budget there is a
CrashLoopBackOff that reads as an application fault and is really a stopwatch.
The Quarkus augmentation is NOT inside this budget - it runs in the init
container, which has no probe and no deadline. That is the second thing the
init container buys, after the read-only root filesystem.
*/}}
{{- define "keycloak.startupProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/health/started" "port" "management")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 60 -}}
{{- include "keycloak.defaultedDict" (dict "defaults" $defaults "given" . "path" "keycloak.startupProbe") -}}
{{- end -}}
{{/*
The effective resources blocks, fed into the resources helper above so that an
erased "resources:" still yields the chart's requests and computed limits.
Two blocks, because the init container does something different from the
serving container: a one-shot, CPU-bound Quarkus augmentation that is gone
before the serving container starts.
*/}}
{{- define "keycloak.effectiveResources" -}}
{{- $defaults := dict
"limitFactor" 3
"requests" (dict "cpu" "500m" "memory" "700Mi" "ephemeral-storage" "256Mi") -}}
{{- $merged := fromYaml (include "keycloak.defaultedDict" (dict "defaults" $defaults "given" . "path" "keycloak.resources")) -}}
{{- include "keycloak.resources" $merged -}}
{{- end -}}
{{- define "keycloak.buildResources" -}}
{{- $given := (fromYaml (include "keycloak.subBlock" (dict "block" . "key" "resources" "path" "keycloak.build"))).value -}}
{{- $defaults := dict
"limitFactor" 3
"requests" (dict "cpu" "500m" "memory" "512Mi" "ephemeral-storage" "256Mi") -}}
{{- $merged := fromYaml (include "keycloak.defaultedDict" (dict "defaults" $defaults "given" $given "path" "keycloak.build.resources")) -}}
{{- include "keycloak.resources" $merged -}}
{{- end -}}
{{/*
The additive env block, rendered through tpl with the value, secretKeyRef and
configMapKeyRef forms (guideline 10).
ADDITIVE MEANS ADDITIVE: an entry repeating a name this chart already sets does
not replace it. At install time both reach the pod and the later one wins; on an
EXISTING release the pair collapses to the chart's own value, because Kubernetes
merges env by name - and helm upgrade still reports success. So do not use this
block to override KC_HOSTNAME or the database variables; change the values that
produce them.
AND ONE CLASS OF ENTRY IS REFUSED OUTRIGHT: a BUILD option's variable name. A
build option in the serving container's environment disagrees with what the init
container built, and Keycloak answers a disagreement by re-augmenting - onto a
read-only path. The result is a crash loop whose cause is nowhere near the
error message, so the render fails here instead, naming the variable. This is a
`fail`, not a silent drop, for the reason guideline 12.6 gives: a value that
cannot be honoured belongs in an abort, not in an implicit branch.
*/}}
{{- define "keycloak.env" -}}
{{- $root := .root -}}
{{- $buildNames := fromYamlArray (include "keycloak.buildOptionEnvNames" .) -}}
{{- range $value := .env }}
{{- if has (upper ($value.name | toString)) $buildNames }}
{{- fail (printf "keycloak.env may not set %q: it is a Keycloak BUILD option, and a build option on the serving container disagrees with what the init container built - Keycloak then re-augments itself onto the read-only /opt/keycloak/lib/quarkus and crash-loops. Build options are set through the chart's own values (keycloak.db.vendor, keycloak.metrics.enabled, keycloak.features), which feed `kc.sh build` in the init container. See values.yaml, point 2." $value.name) }}
{{- end }}
{{- if $value.value }}
- name: {{ $value.name | quote }}
# quote is required, not cosmetic: a value like "8080" or "true" renders
# unquoted as a YAML number/bool, and the API server then rejects the whole
# manifest with "cannot unmarshal number into Go struct field EnvVar.value of
# type string".
value: {{ tpl $value.value $root | quote }}
{{- else if $value.valueFrom }}
{{- if $value.valueFrom.secretKeyRef }}
- name: {{ $value.name }}
valueFrom:
secretKeyRef:
name: {{ tpl $value.valueFrom.secretKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.secretKeyRef.key $root | quote }}
{{- else if $value.valueFrom.configMapKeyRef }}
- name: {{ $value.name }}
valueFrom:
configMapKeyRef:
name: {{ tpl $value.valueFrom.configMapKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.configMapKeyRef.key $root | quote }}
{{- end }}
{{- end }}
{{- end }}
{{- end -}}
{{/*
The three writable paths, defined once because two containers have to agree on
them.
/opt/keycloak/lib/quarkus is the one that matters and the one that is not
guessable: it is where the Quarkus augmentation writes, i.e. where `kc.sh
build` puts generated-bytecode.jar, transformed-bytecode.jar,
quarkus-application.dat and build-system.properties. Read out of the published
image's layer, not inferred. The init container writes it, the serving
container reads it, and because the directory is also the only place a
re-augmentation would write, mounting it writable is what lets the rest of the
root filesystem stay read-only.
/opt/keycloak/data is needed because Quarkus places its temporary directory
under the Keycloak home rather than honouring java.io.tmpdir (upstream issue
#37533). It is EMPTY in the image, so an emptyDir over it hides nothing.
/opt/keycloak/lib/quarkus is NOT empty in the image - which is exactly why it
is seeded by the init container rather than merely mounted; an unseeded
emptyDir there hides quarkus-application.dat, and QuarkusEntryPoint loads that
before any Keycloak code runs.
*/}}
{{- define "keycloak.quarkusBuildDir" -}}/opt/keycloak/lib/quarkus{{- end -}}
{{- define "keycloak.dataDir" -}}/opt/keycloak/data{{- end -}}
+405
View File
@@ -0,0 +1,405 @@
{{- $settings := fromYaml (include "keycloak.settings" .) }}
{{- $adminHostname := eq (include "keycloak.adminHostnameEnabled" .) "true" }}
{{- /*
WHY THIS WORKLOAD HAS TWO INIT CONTAINERS, read before changing either of them
(Gitea #83). Three measured facts force the shape:
1. db, health-enabled and metrics-enabled are BUILD options (verified in the
26.8.0 option table, where each carries the tool icon). The published
image is augmented with their DEFAULTS - db=dev-file, health-enabled=false.
2. Supplying different build options at runtime makes Keycloak re-augment
itself at start, rewriting four files in /opt/keycloak/lib/quarkus. kc.sh
says so itself: "only exit code 10 means that implicit reaugmentation
occurred and a relaunch is needed". Upstream's container guide calls that
start what it is - it "creates an image that is mutable", which is exactly
what readOnlyRootFilesystem: true forbids. And not setting db is no way
out: THE PROBES ALONE force it, because health-enabled is a build option.
3. So the build has to happen before the serving container starts, into a
volume. But an EMPTY emptyDir over /opt/keycloak/lib/quarkus does not
work either, and this is the part that is not guessable:
QuarkusEntryPoint - the main class kc.sh runs for EVERY command, `build`
included - loads "quarkus/quarkus-application.dat" (the literal string is
in QuarkusEntryPoint.class) before it dispatches anything. With that file
hidden by a fresh mount, `kc.sh build` cannot start at all.
Hence: SEED copies the image's own augmentation output into the volume (reading
the real directory, writing only into the volume - so a read-only root
filesystem is fine), BUILD then mounts the now-seeded volume at the real path
and re-augments INTO it (again writing only into the volume), and the serving
container mounts the finished volume and runs `start --optimized`.
What that buys, and the reason it is worth two containers rather than one with
readOnlyRootFilesystem: false: every container in this pod keeps the full
hardening baseline. There is no documented deviation to carry, and no Trivy
suppression that would also silence the serving container.
*/ -}}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
spec:
replicas: {{ include "keycloak.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.keycloak.replicas) }}
selector:
matchLabels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
template:
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
annotations:
# NO checksum/config HERE, AND THAT IS NOT AN OVERSIGHT. This chart
# renders no ConfigMap at all - Keycloak is configured entirely through
# options and KC_* variables - so there is no file whose change could
# fail to restart the workload. The rule (#82) has nothing to attach
# to. values.yaml states this at the top, because an absent annotation
# looks identical to a forgotten one.
#
# Hash of the rendered OnePasswordItems. This covers the SOURCE of the
# secrets, not their content: repointing secrets.databaseSecretRef or
# secrets.bootstrapAdminSecretRef at a different vault item rolls the
# pods instead of leaving the old credentials in the running processes.
# A change INSIDE an item cannot be covered - Helm never sees it (see
# keycloak.secret.yaml) - and NOTES.txt names the manual restart.
checksum/secret-ref: {{ include (print $.Template.BasePath "/keycloak.secret.yaml") . | sha256sum }}
spec:
automountServiceAccountToken: false
securityContext:
{{- include "keycloak.podSecurityContext" .Values.keycloak.securityContext | nindent 8 }}
initContainers:
# --- 1. SEED -------------------------------------------------------
# Copy the image's own augmentation output into the shared volume, so
# that the BUILD container below can bootstrap at all (fact 3 in the
# comment at the top of this file).
#
# It reads /opt/keycloak/lib/quarkus from the image and writes only
# into the volume, which is why readOnlyRootFilesystem: true holds
# here without any special handling.
#
# `command` is overridden because the image's entrypoint is kc.sh and
# this step runs no Keycloak command - the only other chart in this
# repo that overrides an entrypoint is convertx, and the same
# obligation applies: this line has to be re-read whenever the app
# version moves, because it depends on the image shipping a shell and
# coreutils. Both are present (usr/bin/sh -> bash, usr/bin/cp, verified
# in the image layer at 26.8.0).
#
# `cp -R` and not `cp -a`: -a preserves ownership, which a non-root
# process cannot do - it would fail with a diagnostic and a non-zero
# exit. Only the content and the directory structure matter here.
- name: {{ .Release.Name }}-{{ .Chart.Name }}-seed
image: "quay.io/keycloak/keycloak:{{ .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
command:
- /bin/sh
- -c
args:
- |
set -eu
cp -R /opt/keycloak/lib/quarkus/. /quarkus-build/
ls -l /quarkus-build/
resources:
{{- include "keycloak.buildResources" .Values.keycloak.build | nindent 12 }}
securityContext:
{{- include "keycloak.containerSecurityContext" .Values.keycloak.securityContext | nindent 12 }}
volumeMounts:
- name: quarkus-build
mountPath: /quarkus-build
# --- 2. BUILD ------------------------------------------------------
# The Quarkus augmentation, i.e. `kc.sh build`. It writes into the
# seeded volume, which is mounted at the real path here - so the root
# filesystem stays read-only and the output survives into the serving
# container.
#
# THE BUILD OPTIONS ARRIVE AS ENVIRONMENT VARIABLES, NOT AS CLI
# ARGUMENTS, and that is a safety decision rather than a style one:
# `command` is a fixed shell string here, and interpolating chart
# values into a shell string is how a feature name with a space in it
# becomes two arguments. As variables they are passed by the kubelet
# and never parsed by a shell. `kc.sh build` reads KC_* exactly as it
# reads flags.
#
# This is the ONE place build options exist in this chart. The serving
# container passes `start --optimized` and no build option at all, so
# there is no second list to drift out of step - and keycloak.env
# refuses a build option's variable name outright (see _helpers.tpl).
- name: {{ .Release.Name }}-{{ .Chart.Name }}-build
image: "quay.io/keycloak/keycloak:{{ .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
args:
- build
env:
{{- include "keycloak.buildEnv" . | nindent 12 }}
resources:
{{- include "keycloak.buildResources" .Values.keycloak.build | nindent 12 }}
securityContext:
{{- include "keycloak.containerSecurityContext" .Values.keycloak.securityContext | nindent 12 }}
volumeMounts:
- name: quarkus-build
mountPath: {{ include "keycloak.quarkusBuildDir" . }}
# Quarkus puts its temporary directory under the Keycloak home
# rather than honouring java.io.tmpdir (upstream issue #37533), so
# the augmentation needs both of these writable.
- name: data
mountPath: {{ include "keycloak.dataDir" . }}
- name: tmp
mountPath: /tmp
{{- with .Values.keycloak.extraVolumeMounts }}
# THE EXTRA MOUNTS, HERE AS WELL AS ON THE SERVING CONTAINER, AND
# THIS OCCURRENCE IS THE LOAD-BEARING ONE (Gitea #88). Do not
# "clean it up" to the serving container only: an SPI provider JAR
# is a BUILD-time concern. Upstream, on adding one: "After that if
# you are using `--optimized`, you must run the `build` command in
# order to update the server's provider registry with the
# implementations from the JAR file." That build is THIS container,
# so a JAR mounted only below is absent while the registry is
# written - it never gets registered and nothing reports it.
#
# One list for both places on purpose: two lists would be two
# chances to mount an extension in the wrong one, and the wrong one
# fails silently. The price is that every extra volume is present at
# build time too, which values.yaml names at the value.
{{- toYaml . | nindent 12 }}
{{- end }}
containers:
- name: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
# Fully qualified: an unqualified reference would resolve to Docker
# Hub by convention rather than by rule. quay.io is Keycloak's own
# registry and the first quay.io image in this repository - which is
# why ci/trivy/data/ksv0125.yaml had to learn about it.
image: "quay.io/keycloak/keycloak:{{ .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
# `start`, not `start-dev` - the development mode has insecure
# defaults and a local cache. The image declares no CMD, only
# ENTRYPOINT ["/opt/keycloak/bin/kc.sh"], so the subcommand has to
# come from here.
#
# --optimized says "this installation is already built, do not check
# and do not augment". It is what makes the read-only root filesystem
# possible, and it is also why NO build option may be passed to this
# container: with --optimized a build option is an error, and without
# it the container would rebuild on a path it cannot write.
args:
- start
- --optimized
env:
# THE HOSTNAME, DERIVED AND NOT LEFT OPEN - this is the
# security-relevant pair of this chart. Upstream on proxy-headers:
# "Misconfiguration might leave the server exposed to security
# vulnerabilities." The failure mode is Host header injection: with
# a hostname resolved from request headers, an attacker controls the
# URLs the server generates, which is how tokens and password
# resets end up pointed at somebody else's host.
#
# Two things keep that shut. hostname-strict stays at its default
# true, so nothing is taken from the request. And hostname is a
# FULL URL rather than a bare host, which upstream requires as soon
# as hostname-admin is in play: "If hostname-admin is configured,
# hostname must be set to a URL (not just hostname). Otherwise
# Keycloak would not know what is the correct frontend URL".
- name: KC_HOSTNAME
value: {{ include "keycloak.frontendUrl" . | quote }}
{{- if $adminHostname }}
# THE ADMIN HOSTNAME - and it is NOT an access boundary. It decides
# which URLs the server generates for the administration console
# (redirects, assets, the API address the console itself calls).
# Upstream, verbatim: "Using the hostname-admin option does not
# prevent accessing the Administration REST API endpoints via the
# frontend URL specified by the hostname option. If you want to
# restrict access to the Administration REST API, you need to do it
# on the reverse proxy level."
#
# So /admin/realms/... stays reachable through ingress.domain. This
# chart renders no deny rule for it: a path rule on /admin that has
# not been tested against a running console breaks redirects and
# asset loading, and a measure that only looks like one is worse
# than none. The full argument is at ingress.admin in values.yaml.
- name: KC_HOSTNAME_ADMIN
value: {{ include "keycloak.adminUrl" . | quote }}
{{- end }}
# Behind the Ingress the TLS ends at the controller, so the pod
# itself serves plain HTTP - which Keycloak only does when told to.
- name: KC_HTTP_ENABLED
value: "true"
# xforwarded, because the ingress-nginx controller sets
# X-Forwarded-* and not RFC 7239 Forwarded. This is not optional
# decoration: without it Keycloak sees the controller's address and
# scheme instead of the client's.
- name: KC_PROXY_HEADERS
value: "xforwarded"
- name: KC_HTTP_PORT
value: {{ $settings.ports.http | int64 | quote }}
- name: KC_HTTP_MANAGEMENT_PORT
value: {{ $settings.ports.management | int64 | quote }}
# CLUSTERING. ispn is already the default for `start` in production
# mode; it is set explicitly because it is the decision that makes
# the second replica a cluster member rather than a second island,
# and an explicit value cannot be lost to a change of upstream
# defaults.
#
# cache-stack is deliberately NOT set: the default is jdbc-ping,
# which discovers peers through the database that is there anyway.
# The alternative, cache-stack=kubernetes with DNS_PING, is marked
# "(deprecated)" upstream and would need a headless Service and a
# jgroups.dns.query - none of which this chart renders.
- name: KC_CACHE
value: "ispn"
{{- with $settings.cache.clusterName }}
# Only when set. The trap it addresses: "clusters that share the
# same database but should remain separate will merge unless they
# have distinct cluster names". See values.yaml - the default is
# upstream's ISPN on purpose.
- name: KC_CACHE_EMBEDDED_CLUSTER_NAME
value: {{ . | quote }}
{{- end }}
# A stable node name, from the pod name. Upstream's default is "a
# random name ... generated on each start, which makes it difficult
# to correlate metrics and logs across restarts" - and with two
# replicas that correlation is the first thing anybody needs when
# the cluster misbehaves.
- name: KC_CACHE_EMBEDDED_NODE_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
# THE DATABASE. Note what is NOT here: KC_DB. The vendor is a build
# option and is baked in by the init container above; setting it
# here would make this container re-augment on a read-only path.
- name: KC_DB_URL_HOST
value: {{ required "A database host must be provided (keycloak.db.host)" $settings.db.host | quote }}
- name: KC_DB_URL_PORT
value: {{ $settings.db.port | int64 | quote }}
- name: KC_DB_URL_DATABASE
value: {{ required "A database name must be provided (keycloak.db.database)" $settings.db.database | quote }}
- name: KC_DB_USERNAME
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-db-secret
key: username
# AS A VARIABLE, NOT A FILE, and that is forced: Keycloak 26 has no
# _FILE indirection for an option's value - no KC_DB_PASSWORD_FILE
# and no generic KC_<OPTION>_FILE. Checked against the 26.8.0
# option table, where every KC_*_FILE variable is a path option
# (certificates, key stores, trust stores, log and export files).
# The price is named at secrets.databaseSecretRef in values.yaml.
- name: KC_DB_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-db-secret
key: password
# POOL SIZES, per replica. Upstream's db-pool-max-size default is
# 100, which with two replicas asks a default PostgreSQL
# (max_connections = 100) for twice what it has. See values.yaml.
- name: KC_DB_POOL_INITIAL_SIZE
value: {{ $settings.db.pool.initialSize | int64 | quote }}
- name: KC_DB_POOL_MIN_SIZE
value: {{ $settings.db.pool.minSize | int64 | quote }}
- name: KC_DB_POOL_MAX_SIZE
value: {{ $settings.db.pool.maxSize | int64 | quote }}
{{- if $settings.bootstrapAdmin.enabled }}
# THE TEMPORARY BOOTSTRAP ADMINISTRATOR. Needed for a first
# install, because Keycloak "only allows to create the initial
# admin user from a local network connection", which a container
# never has.
#
# Two things NOTES.txt repeats because they cost time otherwise:
# these variable names exist only since Keycloak 26 (before:
# KEYCLOAK_ADMIN / KEYCLOAK_ADMIN_PASSWORD), and the account they
# create is TEMPORARY - creating a permanent administrator
# afterwards is a mandatory step, not advice.
- name: KC_BOOTSTRAP_ADMIN_USERNAME
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-bootstrap-admin-secret
key: username
- name: KC_BOOTSTRAP_ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-bootstrap-admin-secret
key: password
{{- end }}
{{- with .Values.keycloak.env }}
{{- include "keycloak.env" (dict "root" $ "env" .) | nindent 12 }}
{{- end }}
ports:
- containerPort: {{ $settings.ports.http | int64 }}
protocol: TCP
name: http
# THE MANAGEMENT PORT. All three probes address it BY NAME, so
# renaming it here breaks them. It serves /health/* and - with
# keycloak.metrics.enabled - /metrics, both UNAUTHENTICATED, which
# is why no Ingress in this chart points at it and none should be
# added.
- containerPort: {{ $settings.ports.management | int64 }}
protocol: TCP
name: management
# JGroups. Declared for the record rather than for routing: nothing
# serves it and no Service exposes it. jdbc-ping finds the peers
# through the database and then opens DIRECT pod-to-pod
# connections on this port, so the replicas need plain L3
# reachability between pod IPs - no Service, headless or otherwise.
- containerPort: {{ $settings.ports.jgroups | int64 }}
protocol: TCP
name: jgroups
# Probes, resources and both security contexts go through the
# null-safe helpers (#99): an erased block means "chart defaults
# apply" instead of a vanished probe or an unhardened container.
# Never wrap these in an "if" - a guard hides the erased case
# completely instead of rendering a visible null.
#
# Which probe points where, and why liveness must NOT use /health or
# /health/ready, is argued at each helper in _helpers.tpl.
livenessProbe:
{{- include "keycloak.livenessProbe" .Values.keycloak.livenessProbe | nindent 12 }}
readinessProbe:
{{- include "keycloak.readinessProbe" .Values.keycloak.readinessProbe | nindent 12 }}
startupProbe:
{{- include "keycloak.startupProbe" .Values.keycloak.startupProbe | nindent 12 }}
resources:
{{- include "keycloak.effectiveResources" .Values.keycloak.resources | nindent 12 }}
securityContext:
{{- include "keycloak.containerSecurityContext" .Values.keycloak.securityContext | nindent 12 }}
volumeMounts:
# The finished augmentation, produced by the two init containers.
# Writable, although --optimized means nothing should write here:
# it is an emptyDir either way, and a writable mount is what keeps
# a re-augmentation (should a future version decide it needs one) a
# slow start rather than a crash loop.
- name: quarkus-build
mountPath: {{ include "keycloak.quarkusBuildDir" . }}
# Empty in the image, so this hides nothing. Needed because Quarkus
# places its temporary directory under the Keycloak home instead of
# honouring java.io.tmpdir (upstream issue #37533).
- name: data
mountPath: {{ include "keycloak.dataDir" . }}
- name: tmp
mountPath: /tmp
{{- with .Values.keycloak.extraVolumeMounts }}
# The extra mounts, second of the two places (see the build init
# container above). This is where a login THEME is read - themes are
# a runtime concern, so one mounted here takes effect even though
# this container serves a pre-built installation.
{{- toYaml . | nindent 12 }}
{{- end }}
# DELIBERATELY NOT MOUNTED: /opt/keycloak/conf. It is not empty in
# the image - it carries keycloak.conf and, more importantly,
# cache-ispn.xml, which is the Infinispan configuration cache=ispn
# reads. An emptyDir there would hide it and the cluster would come up
# with no cache configuration.
restartPolicy: Always
volumes:
- name: quarkus-build
emptyDir: {}
- name: data
emptyDir: {}
- name: tmp
emptyDir: {}
{{- with .Values.keycloak.extraVolumes }}
# Verbatim, because a volume source can be any of the Kubernetes forms
# and the chart interprets none of them. The one path rule that matters
# is about the MOUNT and not the volume: never /opt/keycloak/themes
# itself, which would hide the image's base, keycloak and keycloak.v2
# themes - see values.yaml at keycloak.extraVolumes.
{{- toYaml . | nindent 8 }}
{{- end }}
+108
View File
@@ -0,0 +1,108 @@
{{- /*
TWO INGRESS OBJECTS, AND THE SECOND ONE IS NOT A SECURITY BOUNDARY.
The public one publishes the login endpoints, the token endpoints and the
account console. The admin one publishes the SAME backend under a second
hostname, so that the administration console - whose URLs Keycloak generates
from hostname-admin - can live on an internal ingress class or in a different
DNS zone.
WHAT IT DOES NOT DO, because the value's name invites the opposite assumption.
Upstream, verbatim: "Using the hostname-admin option does not prevent accessing
the Administration REST API endpoints via the frontend URL specified by the
hostname option. If you want to restrict access to the Administration REST API,
you need to do it on the reverse proxy level." So /admin/realms/... stays
reachable through ingress.domain, and an IP allow-list on the admin host
restricts nothing by itself.
This chart deliberately renders NO deny rule for /admin on the public Ingress.
Two reasons, and the second is the one that decided it: the administration
console loads assets and performs redirects across more paths than /admin, so a
rule written from the path name breaks the console in ways that only show up in
a browser; and a rule nobody has tested against a running console is a measure
that looks like one, which is worse than an absent one. Whoever needs the
restriction puts it on the proxy in front of the public host, where it can be
tested.
GUIDELINE 12.1, CHECKED IN BOTH DIRECTIONS: ingress.domain is read a second
time outside this template - as KC_HOSTNAME on the Deployment - so it stays
required even with ingress.enabled: false, and its required() lives in the
keycloak.frontendUrl helper rather than here. clusterIssuer is read only in
this file, so switching both Ingresses off frees it.
*/ -}}
{{- $adminEnabled := eq (include "keycloak.adminHostnameEnabled" .) "true" }}
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-ingress
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
annotations:
cert-manager.io/cluster-issuer: {{ required "A Cluster-Issuer must be provided (ingress.clusterIssuer)" .Values.ingress.clusterIssuer }}
{{- with .Values.ingress.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
ingressClassName: {{ .Values.ingress.className }}
rules:
- host: {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
http:
paths:
- backend:
service:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
port:
name: http
path: /
pathType: Prefix
tls:
- hosts:
- {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
secretName: tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
{{- end }}
{{- if $adminEnabled }}
{{- if .Values.ingress.enabled }}
---
{{- end }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-admin-ingress
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
annotations:
# Its own certificate, for its own host. A separate TLS secret
# (tls-<release>-keycloak-admin-ingress) rather than a shared one, because
# these are two different hostnames that may well sit on two different
# ingress classes - the shared-secret form in the style guide is for
# several objects on ONE host.
cert-manager.io/cluster-issuer: {{ required "A Cluster-Issuer must be provided (ingress.clusterIssuer)" .Values.ingress.clusterIssuer }}
{{- with ((.Values.ingress.admin | default dict).annotations) }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
# Falls back to ingress.className when not set. An internal ingress class
# here is the useful half of the admin hostname - and still not the access
# control the comment at the top of this file is about.
ingressClassName: {{ (.Values.ingress.admin | default dict).className | default .Values.ingress.className }}
rules:
- host: {{ required "An admin Domain/Host must be provided (ingress.admin.domain) while ingress.admin.enabled is true" ((.Values.ingress.admin | default dict).domain) }}
http:
paths:
# The SAME backend and the same path as the public Ingress. The
# administration console is not a separate application and not a
# separate path - only the hostname differs, and Keycloak generates
# its admin URLs for that hostname because of KC_HOSTNAME_ADMIN.
- backend:
service:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
port:
name: http
path: /
pathType: Prefix
tls:
- hosts:
- {{ required "An admin Domain/Host must be provided (ingress.admin.domain) while ingress.admin.enabled is true" ((.Values.ingress.admin | default dict).domain) }}
secretName: tls-{{ .Release.Name }}-{{ .Chart.Name }}-admin-ingress
{{- end }}
@@ -0,0 +1,40 @@
{{- $settings := fromYaml (include "keycloak.settings" .) }}
{{- /*
THE MANAGEMENT SERVICE, AND WHAT IT IS AND IS NOT FOR.
Keycloak serves /health, /health/live, /health/ready, /health/started and -
with keycloak.metrics.enabled - /metrics on the management port, and it serves
them WITHOUT AUTHENTICATION. The probes do not need this Service (the kubelet
addresses the container port directly); a metrics scraper or an uptime check
inside the cluster does, which is the only reason it exists.
IT IS A SEPARATE OBJECT FROM THE SERVING SERVICE ON PURPOSE. Putting port 9000
next to port 8080 on one Service would mean the Service an Ingress already
points at also carries the unauthenticated health and metrics surface - and
then publishing it is one wrong port name in an Ingress backend away, with
nothing in the diff that reads as a mistake. Separated, naming it is a
deliberate act.
AND IT MUST NEVER GET AN INGRESS. There is no value in this chart that would
publish it, and none should be added. An unauthenticated readiness endpoint on
the internet tells an attacker when the database is down; /metrics tells them
the realm and client names. Upstream's own alternative for this - setting
legacy-observability-interface=true, which moves health back onto the serving
port where the Ingress already is - is explicitly "not recommended for security
reasons", and this chart does not offer it.
*/ -}}
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-management-service
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
spec:
selector:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
ports:
- protocol: TCP
port: {{ $settings.ports.management | int64 }}
targetPort: management
name: management
type: ClusterIP
+38
View File
@@ -0,0 +1,38 @@
{{- $settings := fromYaml (include "keycloak.settings" .) }}
{{- $replicas := int (include "keycloak.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.keycloak.replicas)) }}
{{- /*
THE FIRST PodDisruptionBudget IN THIS REPOSITORY, and the convention it sets is
in README ("Pod disruption budgets") and in the wiki style guide.
WHY IT EXISTS HERE: with two replicas, a node drain is otherwise free to take
both pods at once - and a drain is the most common reason a pod moves, because
it is what a node upgrade does. Without this object the second replica buys
nothing against the failure it was added for. minAvailable rather than
maxUnavailable, because the guarantee wanted is about the service: at least one
node answers logins throughout the drain.
AND WHY IT IS NOT RENDERED BELOW TWO REPLICAS - the trap that makes a PDB worth
a condition rather than an unconditional object: minAvailable: 1 with exactly
ONE replica forbids every voluntary eviction, so `kubectl drain` blocks forever
on a pod nothing is allowed to move. The message then names the PDB and not the
replica count, which sends the reader to the wrong file. The same applies to
scaleDown: true, where there are no pods at all and the object would only
describe a promise about nothing.
So: rendered when the effective replica count is >= 2, skipped otherwise, and
NOTES.txt prints which of the two happened so that an operator does not have to
guess from `kubectl get pdb`.
*/ -}}
{{- if and $settings.podDisruptionBudget.enabled (ge $replicas 2) }}
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-pdb
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
spec:
minAvailable: {{ $settings.podDisruptionBudget.minAvailable | int64 }}
selector:
matchLabels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
{{- end }}
+84
View File
@@ -0,0 +1,84 @@
{{- /*
The 1Password items this chart materializes. Two of them, because they have
different lifetimes rather than different audiences: the database credential
lives as long as the release, while the bootstrap administrator is meant to be
switched off again after the first install.
WHAT THE FIELD NAMES ARE AND WHY THEY ARE NOT FREE-FORM: the operator
materializes one Secret key per FIELD of the item, and the Deployment
references those keys by name. An item without the expected field leaves the
container in CreateContainerConfigError - not in a crash loop, and not with a
message about the field. NOTES.txt lists all four fields for the installer.
THEY ARRIVE AS ENVIRONMENT VARIABLES AND THAT IS FORCED, NOT PREFERRED.
Keycloak 26 has no _FILE indirection for an option's value: there is no
KC_DB_PASSWORD_FILE and no generic KC_<OPTION>_FILE. Checked against the 26.8.0
option table - every KC_*_FILE variable there is a PATH option (certificate,
key store, trust store, log file, export file), i.e. "point at this file", not
"read this option's value from a file". So the file form valkey argues for is
simply unavailable here. DO NOT "FIX" THIS to a file mount: there is no option
that would read it. (Keycloak's Vault SPI, KC_VAULT=file, is a different and
larger mechanism with its own naming scheme, and adopting it would be its own
decision.)
AND THE LIMIT THAT CANNOT BE CLOSED: a change to the CONTENT of either item
restarts nothing. Helm never sees a secret the 1Password operator
materializes, so there is nothing to hash. What this chart does cover is the
op:// PATH, through checksum/secret-ref on the pod template. After rotating a
value INSIDE an item the manual step is
kubectl -n <ns> rollout restart deployment/<release>-keycloak-deployment
The 1Password operator's auto-restart annotation is NOT set here. Unlike the
StatefulSet charts in this repo it WOULD work - the operator enumerates
Deployments - so this is a decision rather than a constraint: its trigger is
the item's version COUNTER and not its content, so a re-save with identical
values rolls the whole cluster. For an identity provider, an unannounced
rolling restart because somebody opened a vault entry is worse than a
deliberate one. README guideline 9 has the precedence trap if it is ever set.
*/ -}}
{{- $settings := fromYaml (include "keycloak.settings" .) -}}
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-db-secret
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
spec:
# Must carry the fields: username, password
#
# The role needs to own its schema: Keycloak creates and migrates the whole
# schema itself through Liquibase on first start, so a read-write grant on
# existing tables is not enough.
#
# Host, port and database name are NOT in this item - they are plain values
# (keycloak.db.host / .port / .database), because they are not secret and
# because moving a database should not mean editing a vault entry.
itemPath: {{ required "You must provide a database secret reference (secrets.databaseSecretRef) - the 1Password item holding the fields 'username' and 'password' of the PostgreSQL role Keycloak uses" .Values.secrets.databaseSecretRef }}
{{- if $settings.bootstrapAdmin.enabled }}
---
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-bootstrap-admin-secret
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
spec:
# Must carry the fields: username, password
#
# THIS ACCOUNT IS TEMPORARY, and that is a change from Keycloak before 26 -
# the old KEYCLOAK_ADMIN / KEYCLOAK_ADMIN_PASSWORD pair created a regular
# administrator. Upstream on the current pair: "A user or service admin
# account created using one of the methods described below is temporary ...
# the account should exist only for the duration necessary to perform
# operations needed to gain permanent and more secure admin access. After
# that, the account needs to be removed manually."
#
# So creating a permanent administrator after the first install is a
# MANDATORY step, and switching keycloak.bootstrapAdmin.enabled to false
# afterwards removes both this item and the per-start log line about the
# initial administrator not being created. NOTES.txt has the sequence.
#
# Upstream's default for the username is "temp-admin"; this chart takes both
# fields from the vault so the pair has one source of truth.
itemPath: {{ required "You must provide a bootstrap admin secret reference (secrets.bootstrapAdminSecretRef) - the 1Password item holding the fields 'username' and 'password' of the TEMPORARY bootstrap administrator. Required while keycloak.bootstrapAdmin.enabled is true; set that to false once a permanent administrator exists" .Values.secrets.bootstrapAdminSecretRef }}
{{- end }}
+30
View File
@@ -0,0 +1,30 @@
{{- $settings := fromYaml (include "keycloak.settings" .) }}
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
spec:
selector:
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
ports:
# The serving port. Both Ingresses target it BY NAME ("http"), and
# in-cluster consumers reach it at
# http://<release>-keycloak-service.<namespace>.svc.cluster.local:8080
- protocol: TCP
port: {{ $settings.ports.http | int64 }}
targetPort: http
name: http
# DELIBERATELY NO MANAGEMENT PORT AND NO JGROUPS PORT HERE. The management
# port has a Service of its own (keycloak.management.service.yaml) so that
# whatever scrapes it has to name it explicitly rather than inherit it from
# the Service an Ingress points at - mixing the two is how an
# unauthenticated health endpoint ends up one annotation away from the
# internet. JGroups needs no Service at all: jdbc-ping discovers peers
# through the database and the nodes then talk pod IP to pod IP.
#
# ClusterIP, and no value to change it. The way out of the cluster is the
# Ingress with a certificate, not a NodePort that puts an identity provider
# on a routable address by accident.
type: ClusterIP
Loaded 100 of 128 files, more files were not shown because too many files have changed in this diff. Show more