feat(garage): new chart for the Garage object store
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 11s
Validate Helm Charts / Build garage (pull_request) Successful in 6s
Validate Helm Charts / Trivy config garage (pull_request) Successful in 16s
Install-test / Install-test garage (pull_request) Successful in 1m15s
Validate Helm Charts / Install-test (pull_request) Successful in 1m15s

One Garage node with replication_factor = 1, built from template-chart to the
style guide rather than copied from upstream's in-tree chart (Gitea #81). The
single-node trade is deliberate and its price is named at the value: the factor
applies to the metadata as well, where upstream calls a corrupted database
"generally not recoverable" and a replicated volume replicates the corruption
faithfully. The three countermeasures that work at factor 1 are therefore
mandatory, not optional - metadata_fsync = true (off in Garage's defaults), the
automatic metadata snapshots, and those snapshots on a second PVC.

Storage: PVC 1 carries metadata_dir AND data_dir as two subdirectories, because
volume snapshots are atomic per volume and that is what keeps index and blocks
consistent with each other; PVC 2 carries metadata_snapshots_dir, because a
recovery point must not die with the volume whose corruption it should repair.
Garage creates all three directories itself, which matters because the image is
FROM scratch and has no mkdir.

Two corrections to the approved plan, both verified against the source at tag
v2.4.1 and decided on the issue before implementation:

- The RPC secret arrives as GARAGE_RPC_SECRET via secretKeyRef, not through
  rpc_secret_file. The file form cannot work in a non-root pod:
  read_secret_file rejects any mode with a group or other bit set
  (src/garage/secrets.rs:144, mask 0o077) while the kubelet OR-s the group-read
  bit onto every file of a secret volume once fsGroup is set
  (pkg/volume/volume_linux.go), and Kubernetes can set the group of such a file
  but never its owner. The mechanism is documented at the value so the next
  round does not "fix" it back.
- The first start is done by the server, not by hand: garage server
  --single-node creates the layout in-process before any listener opens, is
  idempotent on the layout version and has no --version argument that could be
  reused, and --default-access-key/--default-bucket import key and bucket FROM
  the vault item. Both premises of the original "manual first start" decision
  fell with that.

Probes follow from the same source reading: readiness is GET /health on the
admin port, which answers 503 while no layout exists; liveness and startup are
a TCP connect, because a probe that can only fail would restart a node waiting
for its layout forever (same lesson as zipline, README 12.3). The S3 listener
binds independently of the layout, so a TCP probe on 3900 would have been the
lie the plan warned about.

CI: fixtures under ci/garage/ rather than an exclusion - with --single-node the
pod reaches Ready on its own, which makes the install-test an assertion about
the layout, the one-PVC subdirectory layout and readOnlyRootFilesystem instead
of a formality. The Renovate block carries a regex versioning anchored to
^v<major>.<minor>.<patch>$, because the registry receives 40-character
commit-SHA tags almost daily (measured 2026-10-03: the newest 22 tags are all
of that kind) and the four v1.99.x-internal builds sort above the v1.3.x line.

Part of #81
This commit is contained in:
jklein committed 2026-10-03 20:29:38 +00:00
1 parent bf2e1009e4
commit 79b7e44fdc
18 files changed
+2101

No files matched your search

+1
View File
@@ -19,6 +19,7 @@ Every chart has a wiki page of its own — `https://git.jk-dev.de/jlab-cluster/h
| `calibre-web-automated` | Calibre-Web Automated | StatefulSet, extra consume-PVC |
| `cyberchef` | CyberChef | Stateless Deployment |
| `freesailarr` | gluetun VPN media stack (qbittorrent, prowlarr, flaresolverr, radarr, recyclarr, sonarr, seerr, bazarr, and mediathekarr off by default) | Single-pod StatefulSet: all apps share gluetun's network namespace; config volumes via volumeClaimTemplates, media PVCs as existingClaims (optional per-category `subPath`, so one claim can serve all three categories). One ClusterIP Service per app (mediathekarr's carries both of its ports); no global `ingress.domain`. Ingress in two shapes that mix: one host per app (default, each app's host required), or one shared `ingress.host` with the apps as sub-paths (documented ingress exception below) |
| `garage` | Garage (S3-compatible object store) | **One** node, `replication_factor = 1` — a deliberate trade with its price named (Gitea #81): the volume layer underneath already replicates and encrypts, so Garage replication on top would be nine times the raw data across two layers that know nothing about each other. What that costs is that the factor applies to the **metadata** too, where a corrupted database is upstream-documented as "generally not recoverable" and a replicated volume replicates the corruption. Hence three settings that are mandatory rather than optional: `metadata_fsync = true` (**off** in Garage's defaults), the automatic metadata snapshots, and those snapshots on a **second PVC** — a file-system snapshot of a live metadata database is not a restore point, so the volume backup covers the blocks and not the index. The first PVC carries `metadata_dir` **and** `data_dir` as two subdirectories, because volume snapshots are atomic per volume: that is what keeps index and blocks consistent with each other. Two further properties shape the whole chart: the image is **`FROM scratch`** and contains exactly one file, `/garage` — no shell, so **exec probes are impossible here** and an init container would need a second image — and the RPC secret has to arrive as an **environment variable**, because `rpc_secret_file` rejects any file with a group bit set while the kubelet sets one (the reasoning sits at the value). Started with `garage server --single-node`, which creates the cluster layout in-process before any listener opens and is idempotent on the layout version, and with `--default-access-key`/`--default-bucket`, which **import** key and bucket from the 1Password item — so the vault is the source of truth instead of a copy of what the server generated. Readiness is the admin API's `/health`, liveness and startup deliberately are **not**: `/health` answers 503 until a layout exists, so a node waiting for one would be restarted forever |
| `gotenberg` | Gotenberg | Deployment + Service only (no ingress) |
| `homeassistant` | Home Assistant | StatefulSet; the only chart carrying Helm tests (`templates/tests/`) |
| `matrix-synapse` | Matrix Synapse | StatefulSet plus an nginx delegation Deployment/Service |