One Garage node with replication_factor = 1, built from template-chart to the
style guide rather than copied from upstream's in-tree chart (Gitea #81). The
single-node trade is deliberate and its price is named at the value: the factor
applies to the metadata as well, where upstream calls a corrupted database
"generally not recoverable" and a replicated volume replicates the corruption
faithfully. The three countermeasures that work at factor 1 are therefore
mandatory, not optional - metadata_fsync = true (off in Garage's defaults), the
automatic metadata snapshots, and those snapshots on a second PVC.
Storage: PVC 1 carries metadata_dir AND data_dir as two subdirectories, because
volume snapshots are atomic per volume and that is what keeps index and blocks
consistent with each other; PVC 2 carries metadata_snapshots_dir, because a
recovery point must not die with the volume whose corruption it should repair.
Garage creates all three directories itself, which matters because the image is
FROM scratch and has no mkdir.
Two corrections to the approved plan, both verified against the source at tag
v2.4.1 and decided on the issue before implementation:
- The RPC secret arrives as GARAGE_RPC_SECRET via secretKeyRef, not through
rpc_secret_file. The file form cannot work in a non-root pod:
read_secret_file rejects any mode with a group or other bit set
(src/garage/secrets.rs:144, mask 0o077) while the kubelet OR-s the group-read
bit onto every file of a secret volume once fsGroup is set
(pkg/volume/volume_linux.go), and Kubernetes can set the group of such a file
but never its owner. The mechanism is documented at the value so the next
round does not "fix" it back.
- The first start is done by the server, not by hand: garage server
--single-node creates the layout in-process before any listener opens, is
idempotent on the layout version and has no --version argument that could be
reused, and --default-access-key/--default-bucket import key and bucket FROM
the vault item. Both premises of the original "manual first start" decision
fell with that.
Probes follow from the same source reading: readiness is GET /health on the
admin port, which answers 503 while no layout exists; liveness and startup are
a TCP connect, because a probe that can only fail would restart a node waiting
for its layout forever (same lesson as zipline, README 12.3). The S3 listener
binds independently of the layout, so a TCP probe on 3900 would have been the
lie the plan warned about.
CI: fixtures under ci/garage/ rather than an exclusion - with --single-node the
pod reaches Ready on its own, which makes the install-test an assertion about
the layout, the one-PVC subdirectory layout and readOnlyRootFilesystem instead
of a formality. The Renovate block carries a regex versioning anchored to
^v<major>.<minor>.<patch>$, because the registry receives 40-character
commit-SHA tags almost daily (measured 2026-10-03: the newest 22 tags are all
of that kind) and the four v1.99.x-internal builds sort above the v1.3.x line.
Part of #81
Cluster-capable from the start, with roles rather than symmetry: one
StatefulSet per entry of the "nodes" map, the key passed in STALWART_ROLE.
Since v0.16 Stalwart keeps its whole configuration inside the data store and
manages it over JMAP, so the chart's scope is deliberately narrow: it renders
config.json (exactly the DataStore object), the workloads, the Services, the
NetworkPolicies, the Ingress and a cert-manager Certificate. Listeners, blob
store, in-memory store, TLS object and cluster roles are operator work in the
WebUI, which is why NOTES.txt carries an eight-step first install.
config.json stays secret-free: authSecret uses the EnvironmentVariable variant
(verified against the serde definition of PostgreSqlStore at v0.16.24), so the
password arrives through secretKeyRef and checksum/config stays meaningful.
The username cannot follow it - authUsername is a plain String with no secret
form - so it is a public value and says so.
Services are per PORT, not per group: a NodePort is cluster-unique, so "full"
and "mx" can only accept SMTP behind one Service, selected through a per-port
pod label. The http port gets a per-group ClusterIP Service instead, so the
Ingress reaches exactly ingress.nodeGroup.
Four traps, documented rather than explained away:
- The role lookup FAILS OPEN. A node whose ClusterRole is missing from the
database enables every listener and every task while staying Running with a
green probe, and the log line arrives after the listeners are bound. Hence
mx at replicas 0, per-port Services, and a NetworkPolicy that is a ceiling
independent of what was bound.
- NET_BIND_SERVICE is a documented deviation needed to EXEC, not to bind. The
image sets setcap cap_net_bind_service=+ep, which makes the kernel fail
execve when the bounding set is empty. The values comment says both why it
is there and that it is never used, because the second half is what stops
the next hardening pass from removing it.
- externalTrafficPolicy must be Cluster - no pod runs on a control-plane node
while the HA proxy delivers to them, so Local would drop every packet - and
the source address that costs is restored only by the HA proxy's PROXY v2
header.
- /healthz is rate-limited to 100 requests/60s per source IP, which is a node
IP for kubelet probes, so all pods on a node share one bucket. The probe
intervals are calculated (6 req/min/pod, ~16 pods per node) and the sum is a
comment in values.yaml.
A first install needs bootstrap.enabled: true: the setup wizard appears only
when config.json is ABSENT, and this chart always renders one, so recovery
mode is upstream's documented path for a templated config.json.
Second chart in the repo with a dependency - the Valkey project's own chart,
off by default behind condition:, because Stalwart's cluster coordinator rides
on the in-memory store connection. The render aborts when more than one pod
runs with no in-memory store address anywhere.
Also wired up: publish-stalwart.yml through the shared block in
workflow-util/pipelines, the change-app-version.yml choice list, a Renovate
customManagers block anchored on the v prefix (which excludes the moving
latest / v0.16 / -alpine tags), CI fixtures, and the README and CLAUDE.md
lines that claimed ytdlp-web-player was the only chart with a dependency.
Verified locally: ci/helm-dependencies.sh, helm lint --strict (0 failed),
helm package, helm template against chart defaults and against
ci/stalwart/test-values.yaml, the rendered config.json parsed as JSON with the
field types the serde definition requires, ci/run-quantity-render-test.sh
(0 failed, both nodes.*.resources paths exercised), NOTES.txt rendered through
a scratch ConfigMap, and the null-safety and fail paths (securityContext=null
keeps NET_BIND_SERVICE; a cluster without an in-memory store, an unknown port
name, a group without http and an unknown ingress.nodeGroup all abort with a
named message; the schema rejects timeout, a string timeoutMs and a list-shaped
nodes). NO runtime verification was possible - there is no k3d or Docker daemon
in this workspace.
Part of #65
A new chart for a single Valkey instance per application, replacing the
shared cluster-redis whose separation rests on one ACL file in one
1Password field - a construction in which every change touches six
applications and which cost a month of Paperless standstill.
Shape: StatefulSet with one replica, ClusterIP Service, no cluster mode,
no Ingress and no value to add one.
The password arrives as a finished configuration line ("requirepass
<secret>") in the 1Password field auth-conf, is mounted as a file and
included by the main configuration - never in argv, never in the
environment. The include is rendered last, so nothing in extraConfig can
override it. Because a misspelled or empty field would leave an open
server that looks perfectly healthy, an init container greps the mounted
file for the line and refuses to start the pod without it; it never
prints the file.
maxmemory is derived from the computed memory limit (maxmemoryFactor,
default 0.75) and a configured value above that limit aborts the render:
otherwise the kubelet's OOM kill decides what the eviction policy was
configured to decide. Persistence and eviction are documented as pairs -
queue (PVC + noeviction, the default) and cache (emptyDir +
allkeys-lru) - and the combination "no persistence + noeviction" carries
a warning in values.yaml and in NOTES.txt.
Probes: valkey-cli ping as an exec probe, authenticated through
VALKEYCLI_AUTH read out of the mounted file, with "-e" and an assertion
on the answer PONG. Without -e valkey-cli exits 0 on an error reply
(valkey 9.1.2, src/valkey-cli.c: the config.set_errcode branch in
cliReadReply), which would make the probe unable to fail.
Hardening: uid 999 / gid 1000 - the image's own valkey user - with
fsGroup matching, runAsNonRoot, readOnlyRootFilesystem, all capabilities
dropped and seccompProfile RuntimeDefault in the pod context so the init
container is covered too. /data is the only writable path.
NetworkPolicy on by default (first chart in this repo with one), with a
switch and a list of additional namespaces; values.yaml names that it is
only a boundary where the CNI enforces policies.
Wiring: publish-valkey.yml, the chart-name choice list, ci/valkey
fixtures for the persistent variant, a plain docker-versioning Renovate
block, README chart row.
The doc comments on the resources and defaultedDict helpers justified the
copy-per-chart with "charts in this repo deliberately have no dependencies".
That rule fell with #214: a dependency is admissible now, just expensive, and
it sits in the README as guideline 14, not 13. The justification still holds,
but its reason is a different one - sharing the helper would mean a library
chart, a second artefact to publish, version and pull for something that
changes about once a year. searxng got that wording in PR #38; this carries it
to 21 further charts, 40 occurrences in all.
rss-bridge is deliberately left out. Its install-test fails on the Gitea
runner for a reason that predates this branch, and touching the chart would
keep it in the install-test matrix and the check red. Its copy of the sentence
is carried by #51, which has to open the chart anyway.
Comments only: every touched chart renders byte-identical against its
ci/<chart>/test-values.yaml. Each published chart takes a patch bump,
appVersion and +up suffix untouched; template-chart is never published and
takes none, but is corrected because every new chart inherits from it.
Two repo files carried the same claim at no version cost:
validate-charts.yml explained the all-charts quantity render test with it, and
change-app-version.yml still counted nine images for freesailarr - MediathekArr
made it ten.
One StatefulSet with two containers - the Puma web process and a Sidekiq
worker - sharing one ReadWriteOnce PVC at /rails/storage. That shared
volume is the reason they live in one pod: splitting them would need an
RWX volume. Only the web container migrates, because the image entrypoint
runs db:prepare solely for the './bin/rails server' command; the worker is
started through args: so the entrypoint (and its jemalloc preload) stays
in place and the two can never race.
Probes are chosen for what they prove: liveness and startup use Rails'
/up, which is a boot check and touches no database, while readiness uses
'/', which runs User.count through authenticate_user! in self-hosted mode
and therefore fails while Postgres is unreachable. The worker speaks no
HTTP, so all three of its probes ask sidekiqmon whether this pod still
holds a Sidekiq heartbeat in Redis.
Three OnePasswordItem CRs (app, database, redis), ingress.domain kept
required outside the Ingress because APP_DOMAIN and the WebAuthn relying
party read it, strict values.schema.json, CI fixtures with throwaway
Postgres 16 and Redis, and a Renovate manager on semver so the
-alpha/-rc/-hotfix channel and the moving tags are never selected.
Deploys matszwe02/ytdlp_web_player, a Flask app that fetches videos with
yt-dlp, transcodes them with ffmpeg and serves them as HLS. It ships no
authentication of any kind, so the chart carries a preconfigured oauth2-proxy
as an optional subchart, in the data path in front of the app.
This is the first chart in the repo with a dependency, so the rule "charts
have no dependencies" falls here. It is replaced in CLAUDE.md and README.md by
a bar: a dependency has to be optional and off by default, upstream's chart is
consumed as it comes (documented where it falls short of our conventions,
never patched), and the price is written down - above all that a subchart is
invisible to everything downstream watching Helm release versions, so a new
proxy release only reaches a cluster when this chart is republished.
CI learns the dependency in five places, not the three the plan expected:
helm lint, package, template and install all exit 1 while charts/ is unfilled,
so ci/helm-dependencies.sh runs before lint and package in validate-charts.yml,
before package in publish-helm-chart.yml, and inside run-install-test.sh,
run-trivy-config.sh and run-quantity-render-test.sh - the last of which renders
every chart on every PR and would have gone red first.
Notable chart decisions, each measured rather than assumed:
- The image has no USER and runs as root by default, but not by necessity: it
runs under an arbitrary uid with readOnlyRootFilesystem and the full
hardening baseline. Only the pinned uid is a deviation, and it is documented.
- Two emptyDirs, each justified separately. /data holds the video cache
(DATA_PATH). /tmp is NOT mounted - measured across metadata extraction, HLS
transcoding, sprite generation and audio extraction, the image never writes
there. What it does write is $HOME/.cache, so /cache is mounted with HOME and
XDG_CACHE_HOME pointed at it; without it yt-dlp re-downloads its JavaScript
challenge-solver from github.com on every extraction.
- The proxy is wired through config.configFile, the only value the subchart
passes through tpl - config.upstreams is emitted with toJson and would carry
a literal {{ .Release.Name }} into the manifest.
- Rendering aborts when this chart's Ingress and the proxy's claim the same
host: ingress-nginx rejects the second one and a Fleet bundle then sits in
ErrApplied until forceSyncGeneration is bumped.
Fixes#214
BentoPDF computes entirely in the visitor's browser; the container only serves
static files through nginx. The chart is therefore close to cyberchef - same
nginx-unprivileged base, uid 101, port 8080, probes on "/" - with four
differences that shaped the design.
Runtime config.json. The one setting reachable at runtime is the tool list.
It is a static file in the middle of the web root, so it is mounted by
subPath: a directory mount would hide the whole application. subPath mounts
are never refreshed by the kubelet, which makes the checksum/config annotation
(#82) load-bearing rather than decorative. bentopdf.config is passed through
verbatim, like searxng's settings, so a key upstream adds tomorrow is a values
change and not a chart change.
Three env switches that would kill the pod. PORT, DISABLE_IPV6 and
ROBOTS_NOINDEX are implemented as "sed -i" against files in the image, run
under "set -e" in /docker-entrypoint.d/, and the base image's entrypoint
aborts when one of them fails. With readOnlyRootFilesystem: true the pod
therefore never starts instead of ignoring the switch - measured in k3d as
CrashLoopBackOff, exit 1, "sed: can't create temp file
'/etc/nginx/nginx.confXXXXXX': Read-only file system". None of the three is
offered as a first-class value; values.yaml names the failure picture and the
escape hatch (env block plus readOnlyRootFilesystem: false).
Writable paths were measured, not copied from the upstream chart.
/etc/nginx/tmp holds the pid file and /var/cache/nginx is where nginx creates
its temp directories at startup - without it the master aborts with
'mkdir() "/var/cache/nginx/client_temp" failed'. /var/run is deliberately not
mounted: the running pod proves it is unnecessary.
Everything else about BentoPDF is a build-time argument and out of reach for
any chart; values.yaml says so explicitly rather than leaving it to be
discovered. Two accepted consequences are named there too: some tools fetch
WASM modules from a public CDN at runtime, and the served pages carry a
canonical link to the vendor's site.
Fixes#174
SearXNG as a stateless Deployment plus a settings ConfigMap, with the four CI
steps a new chart needs: publish workflow, change-app-version choice list,
ci/searxng/ fixtures and a renovate.json5 block.
Three points where the chart is not a cyberchef with another name:
* The image entrypoint writes settings.yml into /etc/searxng only when the
file is missing. The chart always mounts one by subPath, so the copy and
the random-secret sed never run and the container needs no write access
there - which is what allows readOnlyRootFilesystem: true.
* The signing key is a required OnePasswordItem, not a ConfigMap key. It
signs the settings cookie and the image-proxy URLs, and the entrypoint's
self-generating fallback cannot fire here, so without it the instance would
run on the literal "ultrasecretkey" - which webapp.py refuses with exit 1.
* All three probes use /healthz, the only path the bot detection lets through
unconditionally. Measured: with the limiter active, "/" answers 429 while
/healthz still answers 200.
ingress.domain is read a second time outside the Ingress template, for
SEARXNG_BASE_URL, and carries its own required() there - so it stays
mandatory with the Ingress switched off.
The Renovate versioning regex deviates from the plan on the issue and matches
the commit hash WITHOUT capturing it. Renovate parses a "build" group with
Number.parseInt, so a hash becomes the digits leading it; measured on
renovate 41.173.1 and 44.48.0, the planned form reports an image pushed
earlier the same day as an update to a newer one. Reasoning in the block and
in README.
The chart bundles nine independently versioned upstreams, and eight of them
were already pinned in values.yaml while only gluetun took its tag from
appVersion. That split had no upside: it made gluetun the one image Renovate
could not reach through the built-in helm-values manager, and it implied a
"main app" the stack does not have.
gluetun now carries gluetun.image.repository/tag like every other image, and
the StatefulSet renders it from there. No tag comes from appVersion any more.
Renovate picks all nine up through helm-values - the manager the repo already
relies on for sidecar images - and ci/renovate-bump-chart.sh bumps the chart
version with each update.
Consequences, all documented where they bite:
- renovate.json5 deliberately gets no app-version block for this chart. One
would either duplicate the helm-values updates or rewrite an appVersion that
selects no image. The absence is commented so it does not read as an
oversight.
- freesailarr leaves the change-app-version.yml choice list. That workflow
rewrites appVersion and republishes, which for this chart would change no
deployed image at all.
- appVersion becomes a hand-maintained marker for the gluetun line. Since
renovate-bump-chart.sh bumps the patch level on an unchanged appVersion, a
gluetun update has to be paired with a hand edit of appVersion, or the +up
suffix stops describing what the chart deploys. Noted in Chart.yaml,
values.yaml and README.
Takes over the four loose manifests in cluster-config/rss-bridge/ as a
proper chart, on the current template-chart standard: strict values
schema, null-safe helpers, computed limits, replicas/scaleDown,
checksum/config, NOTES.txt.
Two things deviate from the repo standard on purpose:
- The Ingress models host AND TLS secret name per entry instead of the
single conventional tls-<release>-<chart>-ingress. The installation
this replaces has grown three separate certificates, one per host.
Renaming them would make cert-manager issue three new ones and leave
each host unreachable until it did - a self-inflicted outage on a
deployment that has been up for over a year. Entries sharing a secret
name are grouped into one tls entry, and omitting the name falls back
to the convention, so consolidating later is a values change.
- Requests and limits are decoupled. The old manifest requested 2Gi of
ephemeral storage, identical to its limit, reserving two gigabytes for
a workload measured at 2.2 MB after fetching all five whitelisted
bridges. The request drops to 64Mi; the limit stays at 2Gi.
Hardening was verified rather than assumed. The image ships no USER and
runs as root by default, but runs fully hardened once its writable paths
are provided: uid/gid 33, readOnlyRootFilesystem, all capabilities
dropped, seccompProfile RuntimeDefault. The writable paths were found by
running the image with a writable rootfs, exercising every bridge and
reading its diff layer - /app/cache, /run, /run/php, /var/lib/nginx.
/tmp did not show up that way, because PHP's opcache creates its lock
file there and unlinks it immediately; it only surfaced when the
hardened container died at startup. Stopping at the first writable path
would have shipped a chart that cannot start.
Probes use the app's own /?action=health, which renders fixed JSON
without touching the cache or any remote site, so neither a scraped site
going down nor an unwritable cache can restart the pod.
Fixes#128
Imports the chart from cluster-config functionally unchanged, wires it into
the publish/validate/install-test pipelines and publishes it as the first of
three Java lines (0.4.0+upjava8).
The only deviation from the source chart is the chart-global scaleDown flag
plus a server.replicas value, following the satisfactory pattern: two of the
three servers stand at 0 replicas in the cluster, which the chart could not
express, and Fleet would otherwise start them on reconcile. The default
(scaleDown: false) still renders replicas: 1, so the rendering of the running
server is unchanged - verified byte-identical against the three real values
files for all three Java lines.
Known deficiencies (hardening, probes, naming scheme, hardcoded values,
missing icon, the misleading mc-router auto-discovery annotations) are left
untouched on purpose and are scheduled for stage 2.
Takes the existing mc-router chart over from cluster-config as-is so the
deployment can move to Fleet without any change to the rolled-out state.
Convention alignment follows in stage 2, the app update in stage 3.
The only deviation from "unchanged": Chart.yaml carried name:
minecraft-server (copy-paste from the sister chart). The publish job pushes
by the glob <directory>-*.tgz while helm package names the archive after the
Chart.yaml name, so without this correction the push finds no file.
Version 0.1.1+up1.25.1 - chart semver stays at 0.1.1, the +up suffix is repo
policy and does not affect rendering.
Also wires up CI: publish workflow, change-app-version choice list,
install-test values (CI-only node-port override, since 25565 is outside the
k3d default range) and the README chart table.
jklein
jlab-renovate-bot
jlab-renovate[bot]