jklein
79b7e44fdc
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 11s
Validate Helm Charts / Build garage (pull_request) Successful in 6s
Validate Helm Charts / Trivy config garage (pull_request) Successful in 16s
Install-test / Install-test garage (pull_request) Successful in 1m15s
Validate Helm Charts / Install-test (pull_request) Successful in 1m15s
One Garage node with replication_factor = 1, built from template-chart to the style guide rather than copied from upstream's in-tree chart (Gitea #81). The single-node trade is deliberate and its price is named at the value: the factor applies to the metadata as well, where upstream calls a corrupted database "generally not recoverable" and a replicated volume replicates the corruption faithfully. The three countermeasures that work at factor 1 are therefore mandatory, not optional - metadata_fsync = true (off in Garage's defaults), the automatic metadata snapshots, and those snapshots on a second PVC. Storage: PVC 1 carries metadata_dir AND data_dir as two subdirectories, because volume snapshots are atomic per volume and that is what keeps index and blocks consistent with each other; PVC 2 carries metadata_snapshots_dir, because a recovery point must not die with the volume whose corruption it should repair. Garage creates all three directories itself, which matters because the image is FROM scratch and has no mkdir. Two corrections to the approved plan, both verified against the source at tag v2.4.1 and decided on the issue before implementation: - The RPC secret arrives as GARAGE_RPC_SECRET via secretKeyRef, not through rpc_secret_file. The file form cannot work in a non-root pod: read_secret_file rejects any mode with a group or other bit set (src/garage/secrets.rs:144, mask 0o077) while the kubelet OR-s the group-read bit onto every file of a secret volume once fsGroup is set (pkg/volume/volume_linux.go), and Kubernetes can set the group of such a file but never its owner. The mechanism is documented at the value so the next round does not "fix" it back. - The first start is done by the server, not by hand: garage server --single-node creates the layout in-process before any listener opens, is idempotent on the layout version and has no --version argument that could be reused, and --default-access-key/--default-bucket import key and bucket FROM the vault item. Both premises of the original "manual first start" decision fell with that. Probes follow from the same source reading: readiness is GET /health on the admin port, which answers 503 while no layout exists; liveness and startup are a TCP connect, because a probe that can only fail would restart a node waiting for its layout forever (same lesson as zipline, README 12.3). The S3 listener binds independently of the layout, so a TCP probe on 3900 would have been the lie the plan warned about. CI: fixtures under ci/garage/ rather than an exclusion - with --single-node the pod reaches Ready on its own, which makes the install-test an assertion about the layout, the one-PVC subdirectory layout and readOnlyRootFilesystem instead of a formality. The Renovate block carries a regex versioning anchored to ^v<major>.<minor>.<patch>$, because the registry receives 40-character commit-SHA tags almost daily (measured 2026-10-03: the newest 22 tags are all of that kind) and the four v1.99.x-internal builds sort above the v1.3.x line. Part of #81
72 lines
3.9 KiB
YAML
72 lines
3.9 KiB
YAML
{{- /*
|
|
The 1Password items this chart materializes. Two of them, because they have
|
|
different audiences: the RPC secret belongs to this node alone, while the S3
|
|
access key is the credential the CONSUMER uses - the same item a consuming
|
|
chart (stalwart's secrets.s3SecretRef) points at, which is why it carries the
|
|
bucket name as a third field: endpoint, bucket, key and secret then have one
|
|
source of truth instead of being kept in step by hand.
|
|
|
|
WHAT THE FIELD NAMES ARE AND WHY THEY ARE NOT FREE-FORM: the operator
|
|
materializes one Secret key per FIELD of the item, and this chart references
|
|
those keys by name in the StatefulSet's env block. An item without the
|
|
expected field leaves the container in CreateContainerConfigError. NOTES.txt
|
|
lists all four fields for the installer.
|
|
|
|
AND THE LIMIT THAT CANNOT BE CLOSED: a change to the CONTENT of either item
|
|
restarts nothing. Helm never sees a secret the 1Password operator
|
|
materializes, so there is nothing to hash. What this chart does cover is the
|
|
op:// PATH, through checksum/secret-ref on the pod template. After rotating a
|
|
value INSIDE an item the manual step is
|
|
kubectl -n <ns> rollout restart statefulset/<release>-garage-sfs
|
|
and for the access key that is not enough on its own - see NOTES.txt, Garage
|
|
refuses to adopt a changed secret for an access key it already knows.
|
|
The 1Password operator's auto-restart annotation is NOT set here and must not
|
|
be: verified against operator 1.12.0 it enumerates Deployments only, so on a
|
|
StatefulSet it is a silent no-op advertising a guarantee it does not provide
|
|
(README, guideline 9).
|
|
*/ -}}
|
|
{{- $settings := fromYaml (include "garage.settings" .) -}}
|
|
apiVersion: onepassword.com/v1
|
|
kind: OnePasswordItem
|
|
metadata:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-rpc-secret
|
|
labels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
|
spec:
|
|
# Must carry the field: rpc-secret
|
|
#
|
|
# 32 bytes of random hex - Garage rejects anything else with "Invalid RPC
|
|
# secret key: expected 32 bytes of random hex". Generate it with
|
|
# openssl rand -hex 32
|
|
# It is required even for a single node: Garage uses it as the network key of
|
|
# its RPC layer and refuses to start without it ("rpc_secret value is
|
|
# missing, not present in config file or in environment").
|
|
itemPath: {{ required "You must provide an RPC secret reference (secrets.rpcSecretRef) - the 1Password item holding this cluster's 32-byte hex 'rpc-secret' field" .Values.secrets.rpcSecretRef }}
|
|
{{- $s3Needed := and $settings.singleNode (or $settings.defaultAccessKey $settings.defaultBucket) }}
|
|
{{- if or .Values.secrets.s3SecretRef $s3Needed }}
|
|
---
|
|
apiVersion: onepassword.com/v1
|
|
kind: OnePasswordItem
|
|
metadata:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-s3-secret
|
|
labels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
|
spec:
|
|
# Must carry the fields: access-key, secret-key, bucket
|
|
#
|
|
# THE DIRECTION IS INWARDS, and that is the part worth knowing: with
|
|
# garage.singleNode plus defaultAccessKey/defaultBucket, Garage IMPORTS these
|
|
# three values on start (--default-access-key / --default-bucket read them
|
|
# from the environment). The vault item is the source of truth, and nothing
|
|
# has to be copied out of the server afterwards.
|
|
#
|
|
# Garage validates all three on import and refuses to START on a malformed
|
|
# one, so they are not free-form labels (src/model/key_table.rs,
|
|
# src/model/bucket_alias_table.rs):
|
|
# access-key at least 8 characters, ASCII alphanumeric plus - _ .
|
|
# secret-key at least 16 characters, printable ASCII (U+0021-U+007E)
|
|
# bucket a valid S3 bucket name (3-63 characters, lowercase)
|
|
# NOTES.txt prints the generators.
|
|
itemPath: {{ required "You must provide an S3 key reference (secrets.s3SecretRef) - the 1Password item holding the fields access-key, secret-key and bucket. It is required while garage.singleNode is true together with defaultAccessKey or defaultBucket, because Garage imports those values on start" .Values.secrets.s3SecretRef }}
|
|
{{- end }}
|