jklein
79b7e44fdc
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 11s
Validate Helm Charts / Build garage (pull_request) Successful in 6s
Validate Helm Charts / Trivy config garage (pull_request) Successful in 16s
Install-test / Install-test garage (pull_request) Successful in 1m15s
Validate Helm Charts / Install-test (pull_request) Successful in 1m15s
One Garage node with replication_factor = 1, built from template-chart to the style guide rather than copied from upstream's in-tree chart (Gitea #81). The single-node trade is deliberate and its price is named at the value: the factor applies to the metadata as well, where upstream calls a corrupted database "generally not recoverable" and a replicated volume replicates the corruption faithfully. The three countermeasures that work at factor 1 are therefore mandatory, not optional - metadata_fsync = true (off in Garage's defaults), the automatic metadata snapshots, and those snapshots on a second PVC. Storage: PVC 1 carries metadata_dir AND data_dir as two subdirectories, because volume snapshots are atomic per volume and that is what keeps index and blocks consistent with each other; PVC 2 carries metadata_snapshots_dir, because a recovery point must not die with the volume whose corruption it should repair. Garage creates all three directories itself, which matters because the image is FROM scratch and has no mkdir. Two corrections to the approved plan, both verified against the source at tag v2.4.1 and decided on the issue before implementation: - The RPC secret arrives as GARAGE_RPC_SECRET via secretKeyRef, not through rpc_secret_file. The file form cannot work in a non-root pod: read_secret_file rejects any mode with a group or other bit set (src/garage/secrets.rs:144, mask 0o077) while the kubelet OR-s the group-read bit onto every file of a secret volume once fsGroup is set (pkg/volume/volume_linux.go), and Kubernetes can set the group of such a file but never its owner. The mechanism is documented at the value so the next round does not "fix" it back. - The first start is done by the server, not by hand: garage server --single-node creates the layout in-process before any listener opens, is idempotent on the layout version and has no --version argument that could be reused, and --default-access-key/--default-bucket import key and bucket FROM the vault item. Both premises of the original "manual first start" decision fell with that. Probes follow from the same source reading: readiness is GET /health on the admin port, which answers 503 while no layout exists; liveness and startup are a TCP connect, because a probe that can only fail would restart a node waiting for its layout forever (same lesson as zipline, README 12.3). The S3 listener binds independently of the layout, so a TCP probe on 3900 would have been the lie the plan warned about. CI: fixtures under ci/garage/ rather than an exclusion - with --single-node the pod reaches Ready on its own, which makes the install-test an assertion about the layout, the one-PVC subdirectory layout and readOnlyRootFilesystem instead of a formality. The Renovate block carries a regex versioning anchored to ^v<major>.<minor>.<patch>$, because the registry receives 40-character commit-SHA tags almost daily (measured 2026-10-03: the newest 22 tags are all of that kind) and the four v1.99.x-internal builds sort above the v1.3.x line. Part of #81
104 lines
4.6 KiB
YAML
104 lines
4.6 KiB
YAML
# CI values for the garage install-test.
|
|
#
|
|
# WHY THIS CHART IS TESTED RATHER THAN EXCLUDED, and what the test is worth:
|
|
# with garage.singleNode (the chart default) the server creates its own cluster
|
|
# layout before it opens any listener, so the pod reaches Ready without an
|
|
# operator. That is what makes an install-test possible here at all - without
|
|
# the flag the readiness probe would honestly never go green (no layout, no
|
|
# quorum, /health answers 503) and a green run would have required a probe that
|
|
# lies.
|
|
#
|
|
# WHAT A GREEN RUN PROVES - the four measurement points the plan for Gitea #81
|
|
# left open, three of which this closes on every PR:
|
|
#
|
|
# 1. Garage accepts metadata_dir and data_dir as two SUBDIRECTORIES OF ONE
|
|
# PVC and creates them itself. The image is FROM scratch and has no mkdir,
|
|
# so if the assumption were wrong the pod could not start at all.
|
|
# 2. readOnlyRootFilesystem: true holds with no emptyDir anywhere - the
|
|
# container writes only into the two volumes.
|
|
# 3. The layout is really applied in-process and the admin API answers 200 on
|
|
# /health afterwards: Ready is exactly that HTTP 200, so the rollout
|
|
# status IS the assertion.
|
|
# Plus the mechanical half: the Secret key names match what the StatefulSet
|
|
# references, the rendered garage.toml PARSES AS GARAGE READS IT (it opens
|
|
# LMDB, validates replication_factor against --single-node and refuses
|
|
# malformed keys), the access key and the bucket are imported from the vault
|
|
# fields, the resource quantities are valid, and the schema accepts a
|
|
# realistic non-default values file.
|
|
#
|
|
# WHAT IT DOES NOT PROVE, and no install-test here could:
|
|
# - that the metadata auto-snapshot really lands in metadata_snapshots_dir.
|
|
# The first snapshot is taken after interval/2, and the shortest interval
|
|
# that is not absurd is still minutes - longer than the run. A snapshot
|
|
# interval of seconds would test the timer, not the thing.
|
|
# - anything about Longhorn: the snapshot semantics the chart's whole storage
|
|
# layout is built on are a property of the storage layer, and the runner
|
|
# has local-path.
|
|
# - durability. metadata_fsync is set, and whether an fsync survives a power
|
|
# cut is not observable in a container.
|
|
|
|
secrets:
|
|
# Materialized by ci/garage/fixtures.yaml as ci-garage-rpc-secret and
|
|
# ci-garage-s3-secret (the OnePasswordItem stub CRD comes from ci/common/).
|
|
rpcSecretRef: op://ci/garage-rpc
|
|
s3SecretRef: op://ci/garage-s3
|
|
|
|
garage:
|
|
# Both ON, i.e. the chart's defaults, and they are the reason the run means
|
|
# something: the key and the bucket are imported FROM the fixture Secret, so
|
|
# a wrong field name or a malformed value fails the start rather than passing
|
|
# unnoticed.
|
|
singleNode: true
|
|
defaultAccessKey: true
|
|
defaultBucket: true
|
|
|
|
# ON against the chart's default of off, together with a root domain. It
|
|
# renders a second listener, a second containerPort, a Service port and the
|
|
# [s3_web] section of garage.toml - all of which Garage has to accept, and
|
|
# none of which the default install exercises.
|
|
web:
|
|
enabled: true
|
|
rootDomain: ".web.garage.ci.example.com"
|
|
|
|
s3:
|
|
region: garage-ci
|
|
rootDomain: ".s3.garage.ci.example.com"
|
|
|
|
# Shortened from the chart's "6h" so the snapshot WORKER is created and
|
|
# scheduled during the run. It still does not take a snapshot - the first one
|
|
# is due after interval/2, i.e. 15 minutes here - and that gap is named above
|
|
# rather than papered over with a one-second interval that would only prove
|
|
# the timer fires.
|
|
metadataAutoSnapshotInterval: "30m"
|
|
|
|
resources:
|
|
# Small, to stay inside the runner's budget, while still exercising the
|
|
# limit derivation: 256Mi x limitFactor 3 = 768Mi.
|
|
requests:
|
|
cpu: 100m
|
|
memory: 256Mi
|
|
ephemeral-storage: 32Mi
|
|
|
|
storage:
|
|
data:
|
|
amount: 512Mi
|
|
# k3d's dynamic provisioner. The intended production value,
|
|
# longhorn-encrypted-ns, does not exist on the runner - and a pre-bound
|
|
# hostPath PV would be worse than this one, because the kubelet does not
|
|
# apply fsGroup to a hostPath volume at all, so the test would silently
|
|
# stop exercising the ownership path that matters for a volume the
|
|
# container did not create.
|
|
storageClass: local-path
|
|
snapshots:
|
|
amount: 256Mi
|
|
storageClass: local-path
|
|
|
|
ingress:
|
|
# Left ON although no ingress controller answers in CI: the point is that the
|
|
# object is accepted and that the Service port it names by NAME ("s3")
|
|
# actually exists - a typo there is otherwise invisible until someone opens
|
|
# the URL.
|
|
enabled: true
|
|
domain: garage.ci.example.com
|
|
clusterIssuer: ci-selfsigned
|