feat(garage): new chart for the Garage object store

One Garage node with replication_factor = 1, built from template-chart to the
style guide rather than copied from upstream's in-tree chart (Gitea #81). The
single-node trade is deliberate and its price is named at the value: the factor
applies to the metadata as well, where upstream calls a corrupted database
"generally not recoverable" and a replicated volume replicates the corruption
faithfully. The three countermeasures that work at factor 1 are therefore
mandatory, not optional - metadata_fsync = true (off in Garage's defaults), the
automatic metadata snapshots, and those snapshots on a second PVC.

Storage: PVC 1 carries metadata_dir AND data_dir as two subdirectories, because
volume snapshots are atomic per volume and that is what keeps index and blocks
consistent with each other; PVC 2 carries metadata_snapshots_dir, because a
recovery point must not die with the volume whose corruption it should repair.
Garage creates all three directories itself, which matters because the image is
FROM scratch and has no mkdir.

Two corrections to the approved plan, both verified against the source at tag
v2.4.1 and decided on the issue before implementation:

- The RPC secret arrives as GARAGE_RPC_SECRET via secretKeyRef, not through
  rpc_secret_file. The file form cannot work in a non-root pod:
  read_secret_file rejects any mode with a group or other bit set
  (src/garage/secrets.rs:144, mask 0o077) while the kubelet OR-s the group-read
  bit onto every file of a secret volume once fsGroup is set
  (pkg/volume/volume_linux.go), and Kubernetes can set the group of such a file
  but never its owner. The mechanism is documented at the value so the next
  round does not "fix" it back.
- The first start is done by the server, not by hand: garage server
  --single-node creates the layout in-process before any listener opens, is
  idempotent on the layout version and has no --version argument that could be
  reused, and --default-access-key/--default-bucket import key and bucket FROM
  the vault item. Both premises of the original "manual first start" decision
  fell with that.

Probes follow from the same source reading: readiness is GET /health on the
admin port, which answers 503 while no layout exists; liveness and startup are
a TCP connect, because a probe that can only fail would restart a node waiting
for its layout forever (same lesson as zipline, README 12.3). The S3 listener
binds independently of the layout, so a TCP probe on 3900 would have been the
lie the plan warned about.

CI: fixtures under ci/garage/ rather than an exclusion - with --single-node the
pod reaches Ready on its own, which makes the install-test an assertion about
the layout, the one-PVC subdirectory layout and readOnlyRootFilesystem instead
of a formality. The Renovate block carries a regex versioning anchored to
^v<major>.<minor>.<patch>$, because the registry receives 40-character
commit-SHA tags almost daily (measured 2026-10-03: the newest 22 tags are all
of that kind) and the four v1.99.x-internal builds sort above the v1.3.x line.

Part of #81

garage/templates/NOTES.txt

@@ -0,0 +1,144 @@
+{{- $settings := fromYaml (include "garage.settings" .) -}}
+{{- $pod := printf "%s-%s-sfs-0" .Release.Name .Chart.Name -}}
+{{- $svc := printf "%s-%s-service" .Release.Name .Chart.Name -}}
+Garage {{ .Chart.AppVersion }} installed as release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
+
+ONE node, replication_factor = {{ $settings.replicationFactor | int64 }}, metadata engine {{ $settings.dbEngine }}.
+
+S3 ENDPOINT (in-cluster, path-style addressing):
+  http://{{ $svc }}.{{ .Release.Namespace }}.svc.cluster.local:{{ $settings.ports.s3 | int64 }}
+  region: {{ $settings.s3.region }}
+{{- if .Values.ingress.enabled }}
+  published at: https://{{ .Values.ingress.domain }}
+{{- end }}
+{{- if $settings.web.enabled }}
+  static website endpoint on port {{ $settings.ports.web | int64 }}{{ with $settings.web.rootDomain }} (root domain {{ . }}){{ else }} - WITHOUT a rootDomain it serves nothing; set garage.web.rootDomain{{ end }}
+{{- end }}
+
+THE 1PASSWORD ITEMS, AND WHICH FIELDS THEY MUST CARRY
+The operator materializes one Secret key per item FIELD, so a missing or
+misnamed field leaves the pod in CreateContainerConfigError.
+
+  {{ .Values.secrets.rpcSecretRef }}
+    rpc-secret    32 bytes of random hex:  openssl rand -hex 32
+{{ if .Values.secrets.s3SecretRef }}  {{ .Values.secrets.s3SecretRef }}
+    access-key    >= 8 chars, [A-Za-z0-9-_.]:  openssl rand -hex 12
+    secret-key    >= 16 printable ASCII chars: openssl rand -hex 32
+    bucket        a valid S3 bucket name (3-63 chars, lowercase)
+{{ else }}  (no S3 key item configured - secrets.s3SecretRef is unset)
+{{ end }}
+{{- if and $settings.singleNode (or $settings.defaultAccessKey $settings.defaultBucket) }}
+THE DIRECTION IS INWARDS: Garage IMPORTS{{ if $settings.defaultAccessKey }} the access key{{ end }}{{ if $settings.defaultBucket }}{{ if $settings.defaultAccessKey }} and{{ end }} the bucket{{ end }} from
+that item on start. The vault is the source of truth and nothing has to be
+copied out of the server.
+{{- end }}
+
+FIRST START
+{{ if $settings.singleNode }}
+Nothing to do by hand. The server created its own cluster layout before it
+opened any listener{{ if $settings.defaultAccessKey }}, imported the access key{{ end }}{{ if $settings.defaultBucket }} and created the bucket{{ end }}.
+Verify it, in this order - all commands need /garage as the command, because
+the image has no shell:
+
+  1. kubectl -n {{ .Release.Namespace }} rollout status statefulset/{{ .Release.Name }}-{{ .Chart.Name }}-sfs
+     Ready means the admin API answered 200 on /health, which means a layout
+     exists and the node has quorum. Not-Ready with a running container means
+     the layout is missing - check the logs for "Created initial layout".
+
+  2. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage status
+     One node, with a role (zone dc1, a capacity taken from the detected disk
+     size). A node WITHOUT a role means no layout was applied.
+
+  3. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage layout show
+{{- if $settings.defaultBucket }}
+
+  4. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage bucket list
+     kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage bucket info <bucket>
+     The bucket exists and the imported key has read/write/owner on it.
+{{- end }}
+
+  5. One write through the S3 API from a client pod, which is the only step
+     that proves the endpoint rather than the process.
+{{ else }}
+garage.singleNode is false, so THIS RELEASE SERVES NOTHING YET and the pod
+will stay not-Ready until a layout exists - that is correct, not a defect: the
+readiness probe reports /health, and /health answers 503 while there is no
+layout. Apply one by hand, in this order:
+
+  1. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage status
+     Note the node id and the current layout version (0 on a fresh install).
+
+  2. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- \
+       /garage layout assign -z dc1 -c <capacity> <node-id>
+     The capacity is this node's usable size, roughly storage.data.amount
+     ({{ .Values.storage.data.amount }}). It takes BYTE suffixes B, KB, MB, GB, TB, PB - not
+     Kubernetes' Gi/Mi, so write e.g. 10GB, not 10Gi.
+
+  3. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage layout show
+     Read it before applying. The next step is NOT reversible by repeating it.
+
+  4. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage layout apply --version <N+1>
+     N is the version from step 1, so a fresh cluster takes --version 1.
+     NEVER call this twice with the same --version: the flag is a guard
+     against applying a staged change you have not seen, and reusing a number
+     is how a layout is applied twice.
+
+  5. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage bucket create <bucket>
+     kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- \
+       /garage key import --yes <access-key> <secret-key>
+     kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- \
+       /garage bucket allow --read --write --owner --key <access-key> <bucket>
+
+     "key import" rather than "key create", so the credential still comes FROM
+     the vault item. "key create" generates one and then it has to travel the
+     other way, through a terminal, with the vault as a copy.
+
+  6. The pod becomes Ready within one probe period after step 4.
+{{ end }}
+WHAT THIS CHART DOES NOT RENDER, AND WHAT IS MISSING WITHOUT IT
+
+  - THE LONGHORN BACKUP LABELS. recurring-job.longhorn.io/daily-snapshot and
+    .../daily-backup belong to the consuming bundle, not to this chart. They do
+    NOT apply automatically: without them the off-site half of the backup plan
+    does not exist, it only looks like it does.
+  - The namespace's volume-encryption secret, if the StorageClass is
+    longhorn-encrypted-ns: it resolves
+    ${pvc.namespace}-pvc-encryption-secret, and without that entry the PVCs
+    stay Pending.
+  - No kubernetes_discovery CRD and no bootstrap peers: a single node has no
+    members to find.
+
+TRAPS WORTH KNOWING BEFORE YOU NEED THEM
+
+  - THE METADATA IS NOT IN YOUR VOLUME BACKUP in any usable form. A
+    file-system snapshot of a live metadata database is not a restore point -
+    taken during a write it is corrupt. What IS a restore point are Garage's
+    own snapshots, on the second PVC, taken every {{ $settings.metadataAutoSnapshotInterval }}.
+  - THE WINDOW IS SHORT: exactly TWO snapshots are kept, the first after half
+    the interval and every later one after the interval times a random
+    1.0-1.2. With metadata_auto_snapshot_interval = {{ $settings.metadataAutoSnapshotInterval }} the oldest
+    recovery point is therefore between one and two and a half intervals old,
+    never more. A corruption found later than that is older than every
+    snapshot that still exists.
+  - metadata_fsync is ON here and OFF in Garage's defaults. If you ever see it
+    missing from garage.toml, that is a regression, not a cleanup.
+  - replication_factor 1 applies to the METADATA as well. Upstream calls a
+    corrupted metadata database "generally not recoverable", and the volume
+    layer replicates a corrupted file as faithfully as a good one.
+  - ROTATING THE S3 SECRET IS NOT A RE-SAVE. Garage refuses to start when the
+    access key already exists with a different secret ("is associated with a
+    secret key different than the one given in GARAGE_DEFAULT_SECRET_KEY"). To
+    rotate: either put a NEW access-key id into the item (the old key stays and
+    can be deleted with "/garage key delete"), or delete the old key first and
+    then update both fields.
+  - A CHANGE INSIDE A VAULT ITEM RESTARTS NOTHING. Helm never sees it. After
+    changing a field:
+      kubectl -n {{ .Release.Namespace }} rollout restart statefulset/{{ .Release.Name }}-{{ .Chart.Name }}-sfs
+    (Repointing a secrets.*SecretRef at a different item DOES roll the pod -
+    that is covered by checksum/secret-ref.)
+  - THE IMAGE HAS NO SHELL. "kubectl exec -it ... -- sh" does not work and
+    never will; every command above uses /garage directly. Exec probes are
+    impossible in this chart for the same reason.
+  - THE RPC SECRET IS IN THE PROCESS ENVIRONMENT, not in a file. That is
+    forced, not chosen: Garage rejects a secret file with any group bit set,
+    and the kubelet sets one. values.yaml has the mechanism.

garage/templates/_helpers.tpl

@@ -0,0 +1,392 @@
+{{/*
+Render the body of a container "resources:" block from a values dict of the form
+
+  resources:
+    limitFactor: 3          # optional, default 3
+    requests: { cpu, memory, ephemeral-storage / ephemeralStorage, ... }
+    limits:   { ... }       # optional, every entry optional
+
+Rules (variant B, see issue #19):
+  - requests are rendered as given (camelCase "ephemeralStorage" is normalized
+    to the Kubernetes key "ephemeral-storage").
+  - an explicitly set limit always wins.
+  - a missing memory / ephemeral-storage limit is computed as
+    limitFactor x the corresponding request.
+  - a missing cpu limit is NOT set at all (the kubelet then treats CPU as
+    unlimited; CPU is compressible, so this cannot kill the workload).
+
+NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
+the chart's name as define prefix, because sharing it would mean a library
+chart, and a dependency is admissible but expensive (README, guideline 14) -
+this chart takes none. The reference implementation is
+template-chart/templates/_helpers.tpl.
+*/}}
+{{- define "garage.resources" -}}
+{{- $resources := . | default dict -}}
+{{- $factor := 3.0 -}}
+{{- if not (kindIs "invalid" $resources.limitFactor) -}}
+{{- $factor = float64 $resources.limitFactor -}}
+{{- end -}}
+{{- $requests := dict -}}
+{{- range $key, $value := ($resources.requests | default dict) -}}
+{{- if not (kindIs "invalid" $value) -}}
+{{- $_ := set $requests (include "garage.resourceKey" $key) $value -}}
+{{- end -}}
+{{- end -}}
+{{- $limits := dict -}}
+{{- range $key, $value := ($resources.limits | default dict) -}}
+{{- if not (kindIs "invalid" $value) -}}
+{{- $_ := set $limits (include "garage.resourceKey" $key) $value -}}
+{{- end -}}
+{{- end -}}
+{{- range $key, $request := $requests -}}
+{{- if and (ne $key "cpu") (not (hasKey $limits $key)) -}}
+{{- $_ := set $limits $key (include "garage.multiplyQuantity" (dict "quantity" $request "factor" $factor)) -}}
+{{- end -}}
+{{- end -}}
+{{- $out := dict -}}
+{{- with $requests }}{{- $_ := set $out "requests" . }}{{- end -}}
+{{- with $limits }}{{- $_ := set $out "limits" . }}{{- end -}}
+{{- toYaml $out -}}
+{{- end -}}
+
+{{/*
+Normalize a resource name from values to its Kubernetes key.
+*/}}
+{{- define "garage.resourceKey" -}}
+{{- if eq . "ephemeralStorage" -}}ephemeral-storage{{- else -}}{{ . }}{{- end -}}
+{{- end -}}
+
+{{/*
+Multiply a Kubernetes quantity by a factor, keeping its suffix.
+Input: dict "quantity" <string|number> "factor" <float64>.
+*/}}
+{{- define "garage.multiplyQuantity" -}}
+{{- $quantity := include "garage.formatNumber" .quantity -}}
+{{- $number := regexFind "^[0-9]+(\\.[0-9]+)?" $quantity -}}
+{{- if not $number -}}
+{{- fail (printf "cannot parse resource quantity %q" $quantity) -}}
+{{- end -}}
+{{- $suffix := trimPrefix $number $quantity -}}
+{{- if not (regexMatch "^(m|k|M|G|T|P|E|Ki|Mi|Gi|Ti|Pi|Ei)?$" $suffix) -}}
+{{- fail (printf "unsupported resource quantity suffix %q in %q" $suffix $quantity) -}}
+{{- end -}}
+{{- $product := mulf (float64 $number) .factor -}}
+{{- if eq $product (floor $product) -}}
+{{- printf "%d%s" (int64 $product) $suffix -}}
+{{- else -}}
+{{- printf "%s%s" (include "garage.formatNumber" $product) $suffix -}}
+{{- end -}}
+{{- end -}}
+
+{{/*
+Render a number as a plain decimal string, never in exponential notation
+(issue #157): the null-safe fallback helpers pass values through fromYaml,
+which parses every number as float64, and Go's "%v" switches float64 to
+exponential notation from about 1e6 upwards - "1.048576e+06" is not a valid
+Kubernetes quantity. Strings ("512Mi") are passed through untouched.
+*/}}
+{{- define "garage.formatNumber" -}}
+{{- if kindIs "float64" . -}}
+{{- regexReplaceAll "\\.?0+$" (printf "%.10f" .) "" -}}
+{{- else -}}
+{{- printf "%v" . -}}
+{{- end -}}
+{{- end -}}
+
+{{/*
+Render the replica count for the StatefulSet.
+
+Input: dict "scaleDown" <flag|nil> "replicas" <value|nil>.
+
+  - scaleDown: true takes precedence and always yields 0.
+  - otherwise an explicitly set replicas value is used as given, including an
+    explicit 0 - Helm's "default" would swallow 0, hence the kindIs "invalid"
+    nil check.
+  - a missing replicas value defaults to 1.
+
+A NOTE SPECIFIC TO THIS CHART: more than one replica is not a cluster. Garage
+nodes find each other through RPC and a cluster LAYOUT, and this chart renders
+neither peer bootstrapping nor the kubernetes_discovery CRD (see values.yaml).
+A second replica would come up as a lone node with its own empty metadata db on
+its own volume, and with --single-node it refuses to start at all once it sees
+the other node. The value exists because guideline 11 requires it and because
+0 is the documented way to stop the release; the schema caps it at 1.
+*/}}
+{{- define "garage.replicas" -}}
+{{- if .scaleDown -}}
+0
+{{- else if not (kindIs "invalid" .replicas) -}}
+{{- .replicas -}}
+{{- else -}}
+1
+{{- end -}}
+{{- end -}}
+
+{{/*
+Merge a user-supplied values sub-tree onto a dict of chart defaults so that a
+null / empty override falls back to the defaults instead of erasing them
+(issues #61, #99). Input: dict "defaults" <dict> "given" <dict|nil> "path"
+<string>. Output: YAML of the merged dict (consume it with fromYaml).
+
+An override key written without children is YAML null, and Helm treats that
+null as a deletion of the chart's default. This helper defines the opposite
+semantics - an empty block means "chart defaults apply" - while an explicitly
+set value still wins, including false and 0.
+*/}}
+{{- define "garage.defaultedDict" -}}
+{{- $defaults := .defaults | default dict -}}
+{{- $given := .given -}}
+{{- $path := .path | default "value" -}}
+{{- if kindIs "invalid" $given -}}
+{{- $given = dict -}}
+{{- end -}}
+{{- if not (kindIs "map" $given) -}}
+{{- fail (printf "%s must be a mapping or null, got %s (%v)" $path (kindOf $given) $given) -}}
+{{- end -}}
+{{- $out := deepCopy $defaults -}}
+{{- range $key, $value := $given -}}
+{{- if not (kindIs "invalid" $value) -}}
+{{- $default := index $defaults $key -}}
+{{- if and (kindIs "map" $value) (kindIs "map" $default) -}}
+{{- $_ := set $out $key (fromYaml (include "garage.defaultedDict" (dict "defaults" $default "given" $value "path" (printf "%s.%s" $path $key)))) -}}
+{{- else if kindIs "map" $default -}}
+{{- fail (printf "%s.%s must be a mapping or null, got %s (%v)" $path $key (kindOf $value) $value) -}}
+{{- else -}}
+{{- $_ := set $out $key $value -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}
+{{- toYaml $out -}}
+{{- end -}}
+
+{{/*
+Pick one sub-block out of a values block that may itself be null.
+Input: dict "block" <dict|nil> "key" <string> "path" <string>.
+Returns YAML of dict "value" <the sub-block, may be nil>.
+*/}}
+{{- define "garage.subBlock" -}}
+{{- $block := .block -}}
+{{- if kindIs "invalid" $block -}}
+{{- $block = dict -}}
+{{- end -}}
+{{- if not (kindIs "map" $block) -}}
+{{- fail (printf "%s must be a mapping or null, got %s (%v)" .path (kindOf $block) $block) -}}
+{{- end -}}
+{{- toYaml (dict "value" (index $block .key)) -}}
+{{- end -}}
+
+{{/*
+THE EFFECTIVE GARAGE SETTINGS - everything the ConfigMap and the StatefulSet
+read out of .Values.garage, merged onto this chart's defaults.
+
+Why the whole settings block goes through the null-safe helper and not only the
+three usual suspects (guideline 12.2: "the rule is about the shape, not about
+those three block kinds"): every value below is dereferenced by a template, and
+an erased one would not abort the render - it would render a garage.toml with
+an empty right-hand side, or a containerPort with no number. The first produces
+a pod that crash-loops on a config parse error, the second a manifest the API
+server rejects at apply time. Both are failures that arrive after the render
+looked fine, which is exactly the class #99 is about. Satisfactory shipped the
+same shape with half its blocks null-safe and half not (Gitea #16), and the
+half that was not took the Service's port with it.
+
+The defaults here MUST stay in sync with values.yaml, which remains the
+documented, user-facing place for them; this copy is only the fallback.
+
+Usage: {{ $settings := fromYaml (include "garage.settings" .) }}
+*/}}
+{{- define "garage.settings" -}}
+{{- $given := (fromYaml (include "garage.subBlock" (dict "block" .Values "key" "garage" "path" ".Values"))).value -}}
+{{- $defaults := dict
+      "singleNode" true
+      "defaultAccessKey" true
+      "defaultBucket" true
+      "replicationFactor" 1
+      "dbEngine" "lmdb"
+      "metadataFsync" true
+      "dataFsync" false
+      "metadataAutoSnapshotInterval" "6h"
+      "compressionLevel" 1
+      "blockSize" "1M"
+      "bindAddress" "[::]"
+      "adminBindAddress" "0.0.0.0"
+      "ports" (dict "s3" 3900 "rpc" 3901 "web" 3902 "admin" 3903)
+      "s3" (dict "region" "garage" "rootDomain" nil)
+      "web" (dict "enabled" false "rootDomain" nil) -}}
+{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" $given "path" "garage") -}}
+{{- end -}}
+
+{{/*
+The three directories, defined once because TWO templates have to agree on
+them: garage.toml names them and the StatefulSet mounts the volumes underneath
+them. They are deliberately NOT values - a path that disagrees with its mount
+is a pod that writes its metadata into the container's ephemeral layer, which
+looks like it works until the pod is replaced.
+
+The split is the point of this chart's storage layout (see values.yaml):
+metadata and data are two SUBDIRECTORIES OF ONE VOLUME, the metadata snapshots
+sit on a SECOND one. Garage creates all three itself on start
+(src/model/garage.rs, create_dir_all for metadata_dir and data_dir;
+src/model/snapshot.rs for the snapshots dir), which matters here because the
+image is FROM scratch and carries no mkdir - nothing else in this pod could
+create them.
+*/}}
+{{- define "garage.dataVolumePath" -}}/mnt/garage{{- end -}}
+{{- define "garage.metadataDir" -}}{{ include "garage.dataVolumePath" . }}/meta{{- end -}}
+{{- define "garage.dataDir" -}}{{ include "garage.dataVolumePath" . }}/data{{- end -}}
+{{- define "garage.snapshotsDir" -}}/mnt/garage-snapshots{{- end -}}
+
+{{/*
+The effective security contexts, probes and resources (issue #99).
+
+The defaults below MUST stay in sync with values.yaml. They reproduce THIS
+chart's defaults, not the repo standard - see the fsGroupChangePolicy note.
+*/}}
+{{- define "garage.podSecurityContext" -}}
+{{- $given := (fromYaml (include "garage.subBlock" (dict "block" . "key" "pod" "path" "garage.securityContext"))).value -}}
+{{- $defaults := dict
+      "runAsNonRoot" true
+      "runAsUser" 20000
+      "runAsGroup" 1000
+      "fsGroup" 1000
+      "fsGroupChangePolicy" "OnRootMismatch"
+      "seccompProfile" (dict "type" "RuntimeDefault") -}}
+{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" $given "path" "garage.securityContext.pod") -}}
+{{- end -}}
+
+{{- define "garage.containerSecurityContext" -}}
+{{- $given := (fromYaml (include "garage.subBlock" (dict "block" . "key" "container" "path" "garage.securityContext"))).value -}}
+{{- $defaults := dict
+      "allowPrivilegeEscalation" false
+      "readOnlyRootFilesystem" true
+      "runAsNonRoot" true
+      "capabilities" (dict "drop" (list "ALL") "add" (list)) -}}
+{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" $given "path" "garage.securityContext.container") -}}
+{{- end -}}
+
+{{/*
+LIVENESS: a TCP connect to the admin port, and NOT the /health endpoint the
+readiness probe uses. The reason is the one zipline wrote down (README,
+guideline 12.3) in its sharpest form: /health answers 503 until the cluster
+LAYOUT exists (src/rpc/system.rs, "Layout not yet configured, special case" ->
+ClusterHealthStatus::Unavailable -> 503 in src/api/admin/special.rs). A node
+waiting for its layout - every install with garage.singleNode: false, and any
+install whose layout was lost - would therefore be killed and restarted
+forever, and a restart is precisely what cannot fix it.
+
+What the TCP probe does prove is what a liveness probe is for: the process is
+alive and its listener is open. A wedged process that stops accepting is
+restarted; a healthy process that has nothing to serve yet is not.
+*/}}
+{{- define "garage.livenessProbe" -}}
+{{- $defaults := dict
+      "tcpSocket" (dict "port" "admin")
+      "periodSeconds" 10
+      "timeoutSeconds" 5
+      "failureThreshold" 3 -}}
+{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.livenessProbe") -}}
+{{- end -}}
+
+{{/*
+READINESS: GET /health on the admin port - the one endpoint that answers the
+question a readiness probe asks, namely "can this node serve requests".
+
+Measured against the source at tag v2.4.1 rather than chosen:
+
+  - the S3 API server is spawned independently of the layout state
+    (src/garage/server.rs, the S3/web/admin servers are all spawned after
+    initial_config), so the S3 socket binds and ANSWERS - with errors - before
+    any layout exists. A tcpSocket probe on port 3900 would therefore report a
+    service ready that accepts nothing, which is the lie this chart is not
+    allowed to tell.
+  - /health needs no token (@special in src/api/admin/router_v2.rs) and maps
+    ClusterHealthStatus straight onto the status code: Healthy and Degraded ->
+    200, Unavailable -> 503 with "Quorum is not available for some/all
+    partitions, reads and writes will fail". An unconfigured layout is a
+    special case in src/rpc/system.rs and yields Unavailable.
+
+So with garage.singleNode: true the pod becomes Ready a moment after start
+(initial_config applies the layout BEFORE the servers are spawned), and with
+singleNode: false it stays not-Ready until an operator applies a layout by hand
+- which is the truth, not a defect: until then the S3 endpoint accepts nothing
+and no Service should route to it.
+*/}}
+{{- define "garage.readinessProbe" -}}
+{{- $defaults := dict
+      "httpGet" (dict "path" "/health" "port" "admin")
+      "periodSeconds" 10
+      "timeoutSeconds" 5
+      "failureThreshold" 3 -}}
+{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.readinessProbe") -}}
+{{- end -}}
+
+{{/*
+STARTUP: TCP again, for the same reason as liveness - a startup probe that
+fails forever kills the container, so it must not depend on a layout that only
+an operator can create.
+
+Budget = periodSeconds * failureThreshold = 5 * 24 = 120s. It covers the slow
+part of a real start, which is not Garage: attaching and mounting two network
+volumes, plus opening (and on an unclean shutdown recovering) the LMDB
+metadata database. Garage's own startup work after that is sub-second.
+*/}}
+{{- define "garage.startupProbe" -}}
+{{- $defaults := dict
+      "tcpSocket" (dict "port" "admin")
+      "periodSeconds" 5
+      "timeoutSeconds" 5
+      "failureThreshold" 24 -}}
+{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.startupProbe") -}}
+{{- end -}}
+
+{{/*
+The effective resources block, fed into the resources helper above so that an
+erased "resources:" still yields the chart's requests and computed limits.
+*/}}
+{{- define "garage.effectiveResources" -}}
+{{- $defaults := dict
+      "limitFactor" 3
+      "requests" (dict "cpu" "100m" "memory" "512Mi" "ephemeral-storage" "32Mi") -}}
+{{- $merged := fromYaml (include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.resources")) -}}
+{{- include "garage.resources" $merged -}}
+{{- end -}}
+
+{{/*
+The additive env block, rendered through tpl with the value, secretKeyRef and
+configMapKeyRef forms (guideline 10).
+
+ADDITIVE MEANS ADDITIVE: an entry repeating a name this chart already sets does
+not replace it. At install time both reach the pod and the later one wins; on
+an EXISTING release the pair collapses to the chart's own value, because
+Kubernetes merges env by name - and helm upgrade still reports success. So do
+not use this block to override GARAGE_RPC_SECRET or the three
+GARAGE_DEFAULT_* variables; change the values that produce them.
+*/}}
+{{- define "garage.env" -}}
+{{- $root := .root -}}
+{{- range $value := .env }}
+{{- if $value.value }}
+- name: {{ $value.name | quote }}
+  # quote is required, not cosmetic: a value like "3900" or "true" renders
+  # unquoted as a YAML number/bool, and the API server then rejects the whole
+  # manifest with "cannot unmarshal number into Go struct field EnvVar.value of
+  # type string".
+  value: {{ tpl $value.value $root | quote }}
+{{- else if $value.valueFrom }}
+{{- if $value.valueFrom.secretKeyRef }}
+- name: {{ $value.name }}
+  valueFrom:
+    secretKeyRef:
+      name: {{ tpl $value.valueFrom.secretKeyRef.name $root | quote }}
+      key: {{ tpl $value.valueFrom.secretKeyRef.key $root | quote }}
+{{- else if $value.valueFrom.configMapKeyRef }}
+- name: {{ $value.name }}
+  valueFrom:
+    configMapKeyRef:
+      name: {{ tpl $value.valueFrom.configMapKeyRef.name $root | quote }}
+      key: {{ tpl $value.valueFrom.configMapKeyRef.key $root | quote }}
+{{- end }}
+{{- end }}
+{{- end }}
+{{- end -}}

garage/templates/garage.configmap.yaml

@@ -0,0 +1,146 @@
+{{- $settings := fromYaml (include "garage.settings" .) }}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ .Release.Name }}-{{ .Chart.Name }}-configmap
+  labels:
+    app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+data:
+  # The complete garage.toml. It carries NO SECRET: the RPC secret arrives as
+  # the environment variable GARAGE_RPC_SECRET (see the StatefulSet and
+  # values.yaml for why the file form cannot be used here), so this ConfigMap
+  # is safe to read and the checksum/config annotation on the pod template
+  # still means "the configuration changed".
+  #
+  # It is mounted with subPath at /etc/garage.toml, which is the path Garage
+  # looks at without being told (-c defaults to /etc/garage.toml). A subPath
+  # mount is never refreshed by the kubelet, which is exactly why the checksum
+  # annotation is mandatory rather than decorative.
+  garage.toml: |
+    # Managed by the garage Helm chart - changes here are overwritten on the
+    # next helm upgrade. Edit the chart's values instead.
+
+    # ONE node, and the replication factor is 1 BY DECISION, with the risk
+    # named (Gitea #81): longhorn-encrypted-ns already keeps three replicas of
+    # the encrypted volume, and Garage replication 3 on top of that would be
+    # nine times the raw data for two layers that know nothing about each
+    # other.
+    #
+    # WHAT THAT COSTS, because it is not free: upstream says of
+    # replication_factor = 1 "Do not use this for anything else than test
+    # deployments", and the reason is on its known-issues page - the same
+    # factor applies to the METADATA, so a single-replica setup is especially
+    # exposed to metadata corruption, and a corrupted database file is
+    # "generally not recoverable". Longhorn does not help against that: it
+    # replicates a corrupted file three times. Redundancy against device
+    # failure, yes; against file corruption, no. The three settings below are
+    # the countermeasures that do work at factor 1, and none of them is
+    # optional.
+    replication_factor = {{ $settings.replicationFactor | int64 }}
+
+    # LMDB is Garage's default since 0.9 and the faster engine. The price is
+    # named: the reported corruption after an unclean shutdown stays possible,
+    # the settings below only lower the probability, and at factor 1 there is
+    # no peer to recover from - only the last snapshot. Upstream ties its LMDB
+    # recommendation to replication factor >= 2; running it at 1 is this
+    # chart's deliberate deviation.
+    #
+    # The alternative is "sqlite", which is slower on metadata throughput. With
+    # a single mail server as the only consumer that ceiling is unlikely to be
+    # reached, so sqlite is a defensible choice for anyone who weighs the
+    # durability of a single-file database higher. "fjall" is experimental.
+    db_engine = "{{ $settings.dbEngine }}"
+
+    # metadata_dir and data_dir are two SUBDIRECTORIES OF ONE VOLUME; the
+    # snapshots live on a second one. Longhorn snapshots are atomic PER VOLUME
+    # and never across volumes, so keeping the index and the blocks together is
+    # what prevents the one failure mode that matters for a mail store: a
+    # backup whose metadata points at objects whose blocks were not written yet
+    # ("the mailbox lists a message whose body is missing").
+    #
+    # The snapshots are on the OTHER volume for the opposite reason: a recovery
+    # point must not die with the volume whose corruption it is there to heal.
+    metadata_dir = "{{ include "garage.metadataDir" . }}"
+    data_dir = "{{ include "garage.dataDir" . }}"
+    metadata_snapshots_dir = "{{ include "garage.snapshotsDir" . }}"
+
+    # metadata_fsync IS OFF IN GARAGE'S DEFAULTS, and this is the single
+    # setting most easily lost in a "tidy up the obvious defaults" pass. It is
+    # set here on purpose: Longhorn's block layer cannot help if the engine
+    # never calls fsync - a write that was never handed to the kernel is not in
+    # any replica and not in any snapshot. Garage's own example configuration
+    # pairs it with metadata_auto_snapshot_interval, as here.
+    metadata_fsync = {{ $settings.metadataFsync }}
+    # data_fsync stays off, which is Garage's default and the right trade for
+    # the blocks: a lost block is one object, is detectable by a scrub, and is
+    # replaceable from the sender's copy; a lost metadata write can take the
+    # whole index with it. Turning it on costs an fsync per written block.
+    data_fsync = {{ $settings.dataFsync }}
+
+    # THE ONLY CONSISTENT RECOVERY POINT THIS DEPLOYMENT HAS. File-system
+    # snapshots of metadata_dir guarantee nothing - Garage says so itself: a
+    # snapshot taken during a metadata write is corrupt and useless as a
+    # restore point, which is why Garage recommends ITS OWN snapshot mechanism
+    # in all cases. This is the correction that reversed a premise of the
+    # original plan: Longhorn backs up the blocks, not the metadata.
+    #
+    # KNOW THE WINDOW: Garage keeps exactly TWO snapshots (KEEP_SNAPSHOTS = 2),
+    # the first one after interval/2 and every later one after interval times a
+    # random 1.0-1.2. At "6h" that means the first after ~3h and a restore
+    # window of roughly 6-14.4h afterwards. A corruption discovered later than
+    # that is older than every snapshot that still exists.
+    metadata_auto_snapshot_interval = "{{ $settings.metadataAutoSnapshotInterval }}"
+
+    block_size = "{{ $settings.blockSize }}"
+    compression_level = {{ $settings.compressionLevel | int64 }}
+
+    # Inter-node RPC. There are no other nodes here (no bootstrap_peers, no
+    # kubernetes_discovery - see values.yaml), but Garage binds this listener
+    # regardless and refuses to start without an rpc_secret, which arrives from
+    # the environment.
+    rpc_bind_addr = "{{ $settings.bindAddress }}:{{ $settings.ports.rpc | int64 }}"
+
+    [s3_api]
+    api_bind_addr = "{{ $settings.bindAddress }}:{{ $settings.ports.s3 | int64 }}"
+    # The region name clients must use. It is not a cloud region, it is a label
+    # both sides have to agree on - a consumer configured for a different
+    # region gets a signature error, not a redirect.
+    s3_region = "{{ $settings.s3.region }}"
+    {{- with $settings.s3.rootDomain }}
+    # Vhost-style bucket access (<bucket>.<root_domain>). Only useful with an
+    # Ingress and a wildcard DNS entry in front of it; path-style access works
+    # without it and is what an in-cluster consumer uses.
+    root_domain = {{ . | quote }}
+    {{- end }}
+
+    {{- if $settings.web.enabled }}
+
+    [s3_web]
+    # Static website hosting, OFF by default (see values.yaml): it is a second
+    # HTTP listener and serves nothing at all without a root_domain, while the
+    # consumer this chart was built for speaks S3.
+    bind_addr = "{{ $settings.bindAddress }}:{{ $settings.ports.web | int64 }}"
+    {{- with $settings.web.rootDomain }}
+    root_domain = {{ . | quote }}
+    {{- end }}
+    {{- end }}
+
+    [admin]
+    # Bound on 0.0.0.0 rather than [::] BECAUSE ALL THREE PROBES USE THIS PORT.
+    # A dual-stack listener on [::] only accepts the kubelet's IPv4 connection
+    # while net.ipv6.bindv6only is 0 - true by default on Linux, but a node
+    # sysctl this chart cannot see, and the failure would be a pod that never
+    # becomes Ready for a reason nothing in the manifest points at. The S3 and
+    # RPC listeners keep the upstream [::] default; set garage.bindAddress to
+    # "0.0.0.0" on an IPv4-only cluster, or garage.adminBindAddress to "[::]"
+    # on an IPv6-only one.
+    #
+    # NO admin_token IS SET, so every authenticated admin endpoint is closed.
+    # That is not an omission: /health, the one endpoint the probes use, is
+    # explicitly token-free (@special in the admin router), and nothing in this
+    # deployment calls the authenticated API - the CLI inside the pod talks RPC,
+    # not HTTP. Adding a token would mean a fifth vault field guarding nothing
+    # that is in use. If you need it, it is admin_token_file / the
+    # GARAGE_ADMIN_TOKEN environment variable - and the file form carries the
+    # same permission trap as rpc_secret_file (see values.yaml).
+    api_bind_addr = "{{ $settings.adminBindAddress }}:{{ $settings.ports.admin | int64 }}"

garage/templates/garage.ingress.yaml

@@ -0,0 +1,50 @@
+{{- /*
+OFF BY DEFAULT (ingress.enabled: false), because the consumer this chart was
+built for speaks S3 inside the cluster and needs no published endpoint. The
+values exist anyway, built to this repo's ingress convention, so that
+publishing the endpoint later is a values change rather than a chart change.
+
+GUIDELINE 1, CHECKED IN BOTH DIRECTIONS (README, "Values that must not lie"):
+nothing outside this template reads ingress.domain. The S3 API's vhost-style
+suffix is its own value, garage.s3.rootDomain, because the two are not the same
+thing - a wildcard vhost domain and the single host an Ingress answers on. So
+with the Ingress switched off, BOTH domain and clusterIssuer are free, and
+their required() lives here where it can actually fire.
+
+WHAT IT PUBLISHES: the S3 API. There is no web UI to expose - Garage's admin
+API is JSON and the authenticated half is closed in this chart (no token), and
+the project's web interface is a separate program that is not part of this
+image. Anyone publishing this should know what the object they are putting on
+the internet is: an S3 endpoint that authenticates every request with a
+signature, whose credentials are the two fields of the s3 vault item.
+*/ -}}
+{{- if .Values.ingress.enabled }}
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: {{ .Release.Name }}-{{ .Chart.Name }}-ingress
+  labels:
+    app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+  annotations:
+    cert-manager.io/cluster-issuer: {{ required "A Cluster-Issuer must be provided (ingress.clusterIssuer)" .Values.ingress.clusterIssuer }}
+    {{- with .Values.ingress.annotations }}
+    {{- toYaml . | nindent 4 }}
+    {{- end }}
+spec:
+  ingressClassName: {{ .Values.ingress.className }}
+  rules:
+    - host: {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
+      http:
+        paths:
+          - backend:
+              service:
+                name: {{ .Release.Name }}-{{ .Chart.Name }}-service
+                port:
+                  name: s3
+            path: /
+            pathType: Prefix
+  tls:
+    - hosts:
+        - {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
+      secretName: tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
+{{- end }}

garage/templates/garage.pvc.yaml

@@ -0,0 +1,63 @@
+{{- /*
+TWO claims, and the split is the durability design of this chart rather than a
+sizing convenience (Gitea #81):
+
+  1. <release>-garage-pvc            metadata_dir AND data_dir, as two
+                                     subdirectories of this one volume
+  2. <release>-garage-snapshots-pvc  metadata_snapshots_dir
+
+Claim 1 keeps index and blocks on ONE volume because Longhorn snapshots are
+atomic per volume and never across volumes. Split over two volumes, a backup
+can capture metadata that points at objects whose blocks were not written yet -
+for a mail store exactly the failure "the mailbox lists a message whose body is
+missing".
+
+Claim 2 exists because Garage's own metadata snapshots are the only consistent
+recovery point here (file-system snapshots of a live metadata database are not
+one), and a recovery point stored on the volume whose corruption it should
+repair is not a recovery point.
+
+Both blocks are DELIBERATELY NOT null-safe: an erased size yields an invalid
+PVC that is rejected at apply time, which is loud. A chart-side fallback would
+silently size somebody's object store - and on an existing release it could not
+help anyway, because a PVC cannot shrink. Same reasoning as satisfactory's
+documented exception (Gitea #39) and valkey's persistence block.
+
+Neither claim is ever renamed: a renamed PVC makes Kubernetes provision a new,
+empty volume and orphans the data.
+*/ -}}
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
+  labels:
+    app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+spec:
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      storage: {{ .Values.storage.data.amount }}
+  {{- if .Values.storage.data.storageClass }}
+  storageClassName: {{ .Values.storage.data.storageClass }}
+  {{- end }}
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: {{ .Release.Name }}-{{ .Chart.Name }}-snapshots-pvc
+  labels:
+    app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+spec:
+  accessModes:
+    - ReadWriteOnce
+  resources:
+    requests:
+      # Sized for TWO snapshots of the metadata database plus head room, not
+      # for the objects - Garage keeps exactly two and deletes the older ones
+      # itself. A snapshot is a compacted copy of the LMDB file, so this needs
+      # roughly twice the size metadata_dir reaches.
+      storage: {{ .Values.storage.snapshots.amount }}
+  {{- if .Values.storage.snapshots.storageClass }}
+  storageClassName: {{ .Values.storage.snapshots.storageClass }}
+  {{- end }}

garage/templates/garage.secret.yaml

@@ -0,0 +1,71 @@
+{{- /*
+The 1Password items this chart materializes. Two of them, because they have
+different audiences: the RPC secret belongs to this node alone, while the S3
+access key is the credential the CONSUMER uses - the same item a consuming
+chart (stalwart's secrets.s3SecretRef) points at, which is why it carries the
+bucket name as a third field: endpoint, bucket, key and secret then have one
+source of truth instead of being kept in step by hand.
+
+WHAT THE FIELD NAMES ARE AND WHY THEY ARE NOT FREE-FORM: the operator
+materializes one Secret key per FIELD of the item, and this chart references
+those keys by name in the StatefulSet's env block. An item without the
+expected field leaves the container in CreateContainerConfigError. NOTES.txt
+lists all four fields for the installer.
+
+AND THE LIMIT THAT CANNOT BE CLOSED: a change to the CONTENT of either item
+restarts nothing. Helm never sees a secret the 1Password operator
+materializes, so there is nothing to hash. What this chart does cover is the
+op:// PATH, through checksum/secret-ref on the pod template. After rotating a
+value INSIDE an item the manual step is
+  kubectl -n <ns> rollout restart statefulset/<release>-garage-sfs
+and for the access key that is not enough on its own - see NOTES.txt, Garage
+refuses to adopt a changed secret for an access key it already knows.
+The 1Password operator's auto-restart annotation is NOT set here and must not
+be: verified against operator 1.12.0 it enumerates Deployments only, so on a
+StatefulSet it is a silent no-op advertising a guarantee it does not provide
+(README, guideline 9).
+*/ -}}
+{{- $settings := fromYaml (include "garage.settings" .) -}}
+apiVersion: onepassword.com/v1
+kind: OnePasswordItem
+metadata:
+  name: {{ .Release.Name }}-{{ .Chart.Name }}-rpc-secret
+  labels:
+    app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+spec:
+  # Must carry the field: rpc-secret
+  #
+  # 32 bytes of random hex - Garage rejects anything else with "Invalid RPC
+  # secret key: expected 32 bytes of random hex". Generate it with
+  #   openssl rand -hex 32
+  # It is required even for a single node: Garage uses it as the network key of
+  # its RPC layer and refuses to start without it ("rpc_secret value is
+  # missing, not present in config file or in environment").
+  itemPath: {{ required "You must provide an RPC secret reference (secrets.rpcSecretRef) - the 1Password item holding this cluster's 32-byte hex 'rpc-secret' field" .Values.secrets.rpcSecretRef }}
+{{- $s3Needed := and $settings.singleNode (or $settings.defaultAccessKey $settings.defaultBucket) }}
+{{- if or .Values.secrets.s3SecretRef $s3Needed }}
+---
+apiVersion: onepassword.com/v1
+kind: OnePasswordItem
+metadata:
+  name: {{ .Release.Name }}-{{ .Chart.Name }}-s3-secret
+  labels:
+    app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+spec:
+  # Must carry the fields: access-key, secret-key, bucket
+  #
+  # THE DIRECTION IS INWARDS, and that is the part worth knowing: with
+  # garage.singleNode plus defaultAccessKey/defaultBucket, Garage IMPORTS these
+  # three values on start (--default-access-key / --default-bucket read them
+  # from the environment). The vault item is the source of truth, and nothing
+  # has to be copied out of the server afterwards.
+  #
+  # Garage validates all three on import and refuses to START on a malformed
+  # one, so they are not free-form labels (src/model/key_table.rs,
+  # src/model/bucket_alias_table.rs):
+  #   access-key  at least 8 characters, ASCII alphanumeric plus - _ .
+  #   secret-key  at least 16 characters, printable ASCII (U+0021-U+007E)
+  #   bucket      a valid S3 bucket name (3-63 characters, lowercase)
+  # NOTES.txt prints the generators.
+  itemPath: {{ required "You must provide an S3 key reference (secrets.s3SecretRef) - the 1Password item holding the fields access-key, secret-key and bucket. It is required while garage.singleNode is true together with defaultAccessKey or defaultBucket, because Garage imports those values on start" .Values.secrets.s3SecretRef }}
+{{- end }}

garage/templates/garage.service.yaml

@@ -0,0 +1,43 @@
+{{- $settings := fromYaml (include "garage.settings" .) }}
+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ .Release.Name }}-{{ .Chart.Name }}-service
+  labels:
+    app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+spec:
+  selector:
+    app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+  ports:
+    # The S3 endpoint. In-cluster consumers reach it at
+    #   http://<release>-garage-service.<namespace>.svc.cluster.local:3900
+    # with path-style addressing (bucket in the path), which needs no
+    # root_domain and no DNS wildcard.
+    - protocol: TCP
+      port: {{ $settings.ports.s3 | int64 }}
+      targetPort: s3
+      name: s3
+    # The admin endpoint. With no admin_token configured (see the ConfigMap)
+    # this serves exactly two things without authentication, /health and
+    # /metrics, and answers everything else with 401 - so it is a monitoring
+    # surface, not a management one.
+    - protocol: TCP
+      port: {{ $settings.ports.admin | int64 }}
+      targetPort: admin
+      name: admin
+    {{- if $settings.web.enabled }}
+    - protocol: TCP
+      port: {{ $settings.ports.web | int64 }}
+      targetPort: web
+      name: web
+    {{- end }}
+  # DELIBERATELY NO RPC PORT. Port 3901 exists on the pod because Garage binds
+  # it either way, but nothing should reach it: a single node has no peers, and
+  # this chart renders neither bootstrap_peers nor the kubernetes_discovery CRD.
+  # A Service port would advertise a cluster interface that is not one.
+  #
+  # ClusterIP, and no value to change it. The S3 API authenticates every
+  # request, but the way out of the cluster is an Ingress with a certificate
+  # (ingress.enabled, off by default), not a NodePort that puts an object store
+  # on a routable address by accident.
+  type: ClusterIP

garage/templates/garage.sfs.yaml

@@ -0,0 +1,208 @@
+{{- $settings := fromYaml (include "garage.settings" .) }}
+{{- /*
+Catch the one flag combination Garage rejects at START, at render time instead:
+--default-access-key and --default-bucket "can only be used in single-node
+mode" (src/garage/server.rs). Without this check the chart would install
+cleanly and the container would exit immediately, with the reason only in the
+pod log.
+*/ -}}
+{{- if and (not $settings.singleNode) (or $settings.defaultAccessKey $settings.defaultBucket) }}
+{{- fail "garage.defaultAccessKey and garage.defaultBucket require garage.singleNode: true - Garage refuses to start otherwise (\"Flags --default-access-key and --default-bucket can only be used in single-node mode\"). With singleNode: false, create the key and the bucket by hand after applying the cluster layout; NOTES.txt has the commands." }}
+{{- end }}
+{{- $importKey := and $settings.singleNode (or $settings.defaultAccessKey $settings.defaultBucket) }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+  labels:
+    app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+spec:
+  replicas: {{ include "garage.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.garage.replicas) }}
+  serviceName: {{ .Release.Name }}-{{ .Chart.Name }}-service
+  selector:
+    matchLabels:
+      app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+  template:
+    metadata:
+      name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+      labels:
+        app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+      annotations:
+        # Hash of the rendered ConfigMap: a changed garage.toml changes the pod
+        # template and thus restarts the pod. Without it the change would reach
+        # nothing - the config is mounted with subPath (never refreshed by the
+        # kubelet) and Garage reads it once, at start.
+        checksum/config: {{ include (print $.Template.BasePath "/garage.configmap.yaml") . | sha256sum }}
+        # Hash of the rendered OnePasswordItems. This covers the SOURCE of the
+        # secrets, not their content: repointing secrets.rpcSecretRef or
+        # secrets.s3SecretRef at a different vault item rolls the pod instead of
+        # leaving the old credentials in the running process. A change INSIDE an
+        # item cannot be covered - Helm never sees it (see garage.secret.yaml).
+        checksum/secret-ref: {{ include (print $.Template.BasePath "/garage.secret.yaml") . | sha256sum }}
+    spec:
+      automountServiceAccountToken: false
+      securityContext:
+        {{- include "garage.podSecurityContext" .Values.garage.securityContext | nindent 8 }}
+      # NO INIT CONTAINER, and that is a property of the image rather than a
+      # choice: dxflrs/garage is built FROM scratch and contains exactly one
+      # file, /garage. There is no shell, no mkdir, no wget - so an init
+      # container would have to be a SECOND image to pin, track and update.
+      # Nothing needs one: Garage creates metadata_dir, data_dir and the
+      # snapshots directory itself on start (create_dir_all in
+      # src/model/garage.rs and src/model/snapshot.rs), and the volumes are made
+      # writable by fsGroup.
+      #
+      # The same property rules out exec probes in this chart for good - all
+      # three probes are HTTP or TCP, see the helpers.
+      containers:
+        - name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
+          # Fully qualified: an unqualified "dxflrs/garage" resolves to Docker
+          # Hub by convention rather than by rule. The multi-arch image, not
+          # upstream's own chart's dxflrs/amd64_garage.
+          image: "docker.io/dxflrs/garage:{{ .Chart.AppVersion }}"
+          imagePullPolicy: IfNotPresent
+          # command AND args, because the image's CMD is ["/garage", "server"]
+          # and flags have to go after the subcommand. Overriding only args
+          # would drop the "server" subcommand.
+          command:
+            - /garage
+          args:
+            - server
+            {{- if $settings.singleNode }}
+            # --single-node is what makes this chart's default install reach
+            # Ready without an operator (Gitea #81, second plan correction):
+            # initial_config() in src/garage/server.rs creates the cluster
+            # layout in-process BEFORE the S3, web and admin servers are
+            # spawned - zone "dc1", capacity taken from the detected disk size,
+            # then apply_staged_changes(1).
+            #
+            # IT IS IDEMPOTENT, and it cannot damage an existing cluster. The
+            # layout version decides: 0 creates the layout, 1 does nothing at
+            # all, anything above 1 refuses to start ("Refusing to run in
+            # single-node mode: layout version is already superior to 1"), and
+            # a node that sees other nodes refuses as well ("more nodes are
+            # already present in the cluster"). There is no --version argument
+            # that could be passed twice with the same value, which is the trap
+            # the CLI path (garage layout apply --version N+1) has.
+            #
+            # It requires replication_factor = 1 and says so loudly if it is
+            # not.
+            - --single-node
+            {{- if or $settings.defaultAccessKey $settings.defaultBucket }}
+            # Imports the access key from GARAGE_DEFAULT_ACCESS_KEY /
+            # GARAGE_DEFAULT_SECRET_KEY below, i.e. FROM the vault item INTO
+            # Garage. Re-running is guarded upstream: a key that already exists
+            # with a different secret aborts the start, and one that was deleted
+            # in the cluster cannot be added back.
+            - --default-access-key
+            {{- end }}
+            {{- if $settings.defaultBucket }}
+            # Creates the bucket from GARAGE_DEFAULT_BUCKET and grants the key
+            # above read/write/owner on it. Implies --default-access-key.
+            - --default-bucket
+            {{- end }}
+            {{- end }}
+          env:
+            # THE RPC SECRET AS AN ENVIRONMENT VARIABLE, NOT AS A FILE - and the
+            # full reason is in values.yaml under secrets.rpcSecretRef. The
+            # short version, because this is the line a later round will want to
+            # "fix": rpc_secret_file CANNOT WORK in a non-root pod.
+            # read_secret_file rejects any file whose mode has a group or other
+            # bit set (src/garage/secrets.rs: mode & 0o077), and the kubelet
+            # OR-s the group-read bit onto every file of a secret volume as soon
+            # as fsGroup is set. Without fsGroup the file stays root-owned and
+            # uid 20000 cannot read it. Kubernetes can set the group of such a
+            # file, never its owner, so no combination satisfies both.
+            - name: GARAGE_RPC_SECRET
+              valueFrom:
+                secretKeyRef:
+                  name: {{ .Release.Name }}-{{ .Chart.Name }}-rpc-secret
+                  key: rpc-secret
+            {{- if $importKey }}
+            - name: GARAGE_DEFAULT_ACCESS_KEY
+              valueFrom:
+                secretKeyRef:
+                  name: {{ .Release.Name }}-{{ .Chart.Name }}-s3-secret
+                  key: access-key
+            - name: GARAGE_DEFAULT_SECRET_KEY
+              valueFrom:
+                secretKeyRef:
+                  name: {{ .Release.Name }}-{{ .Chart.Name }}-s3-secret
+                  key: secret-key
+            {{- end }}
+            {{- if and $settings.singleNode $settings.defaultBucket }}
+            - name: GARAGE_DEFAULT_BUCKET
+              valueFrom:
+                secretKeyRef:
+                  name: {{ .Release.Name }}-{{ .Chart.Name }}-s3-secret
+                  key: bucket
+            {{- end }}
+            {{- with .Values.garage.env }}
+            {{- include "garage.env" (dict "root" $ "env" .) | nindent 12 }}
+            {{- end }}
+          ports:
+            - containerPort: {{ $settings.ports.s3 | int64 }}
+              protocol: TCP
+              name: s3
+            - containerPort: {{ $settings.ports.rpc | int64 }}
+              protocol: TCP
+              name: rpc
+            {{- if $settings.web.enabled }}
+            - containerPort: {{ $settings.ports.web | int64 }}
+              protocol: TCP
+              name: web
+            {{- end }}
+            # The port all three probes use. Its NAME is what the probe defaults
+            # reference, so renaming it here breaks them.
+            - containerPort: {{ $settings.ports.admin | int64 }}
+              protocol: TCP
+              name: admin
+          # Probes, resources and both security contexts go through the
+          # null-safe helpers (#99): an erased block means "chart defaults
+          # apply" instead of a vanished probe or an unhardened container.
+          # Never wrap these in an "if" - a guard hides the erased case
+          # completely instead of rendering a visible null.
+          #
+          # Which probe points at what, and why liveness and startup must NOT
+          # point at /health, is argued at each helper in _helpers.tpl.
+          livenessProbe:
+            {{- include "garage.livenessProbe" .Values.garage.livenessProbe | nindent 12 }}
+          readinessProbe:
+            {{- include "garage.readinessProbe" .Values.garage.readinessProbe | nindent 12 }}
+          startupProbe:
+            {{- include "garage.startupProbe" .Values.garage.startupProbe | nindent 12 }}
+          resources:
+            {{- include "garage.effectiveResources" .Values.garage.resources | nindent 12 }}
+          securityContext:
+            {{- include "garage.containerSecurityContext" .Values.garage.securityContext | nindent 12 }}
+          volumeMounts:
+            # ONE volume carrying metadata_dir and data_dir as subdirectories,
+            # so a Longhorn snapshot catches index and blocks in one atomic
+            # operation. Garage creates both subdirectories on start.
+            - name: data
+              mountPath: {{ include "garage.dataVolumePath" . }}
+            # The SECOND volume, holding the only consistent recovery point this
+            # deployment has. Separate on purpose: a snapshot that dies with the
+            # volume whose corruption it should repair is not a recovery point.
+            - name: snapshots
+              mountPath: {{ include "garage.snapshotsDir" . }}
+            - name: config
+              mountPath: /etc/garage.toml
+              subPath: garage.toml
+              readOnly: true
+          # No emptyDir for /tmp, and that is checked rather than assumed: with
+          # readOnlyRootFilesystem: true the root filesystem of this container
+          # holds exactly one file. Garage writes under metadata_dir, data_dir
+          # and the snapshots directory and uses no temporary directory
+          # (verified in src/model, src/db and src/block at v2.4.1).
+      restartPolicy: Always
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
+        - name: snapshots
+          persistentVolumeClaim:
+            claimName: {{ .Release.Name }}-{{ .Chart.Name }}-snapshots-pvc
+        - name: config
+          configMap:
+            name: {{ .Release.Name }}-{{ .Chart.Name }}-configmap