1 Commits
Author SHA1 Message Date
jlab-renovate-bot 1ca22c4238 Update docker.io/searxng/searxng Docker tag to v2026.9.13-96479fff6
renovate/stability-days Updates have met minimum release age requirement
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 7s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Successful in 4s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Failing after 43s
Validate Helm Charts / Install-test ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }} (pull_request) Failing after 5s
2026-09-16 10:01:39 +00:00
212 changed files with 633 additions and 18145 deletions

No files matched your search

+4 -10
View File
@@ -38,13 +38,7 @@ on:
- searxng
- bentopdf
- ytdlp-web-player
- sure
- valkey
- stalwart
- garage
- keycloak
- bulwark
# freesailarr is deliberately absent: none of its ten images takes
# freesailarr is deliberately absent: none of its nine images takes
# its tag from appVersion (they are all pinned in its values.yaml),
# so this workflow would rewrite appVersion and republish the chart
# without changing a single deployed image. Update its images through
@@ -70,18 +64,18 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: "Configure OP-Connect"
uses: 1password/load-secrets-action/configure@70062d7a876d3eb6334754fa26efd2fbd90c32f2 # v5
uses: 1password/load-secrets-action/configure@v5
with:
connect-host: https://op-connect.jk-dev.de
- name: "Load GitHub Credentials"
id: load-jk-cluster-bot-creds
uses: 1password/load-secrets-action@70062d7a876d3eb6334754fa26efd2fbd90c32f2 # v5
uses: 1password/load-secrets-action@v5
env:
OP_CONNECT_TOKEN: ${{ secrets.OP_CONNECT_TOKEN }}
JK_CLUSTER_BOT_TOKEN: op://GitHub-Actions/github-helm-charts-push-token/credential
JK_CLUSTER_USERNAME: op://GitHub-Actions/github-helm-charts-push-token/username
JK_CLUSTER_EMAIL: op://GitHub-Actions/github-helm-charts-push-token/email
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- uses: actions/checkout@v7
with:
token: ${{ env.JK_CLUSTER_BOT_TOKEN }}
- name: "Get Chart Version"
@@ -1,216 +0,0 @@
# The all-charts install-test: the same job the pull-request pipeline runs, with
# a matrix over every installable chart instead of the changed ones (Gitea #52).
#
# The question it answers is not "did this change break a chart" - that is the
# pull-request pipeline's job - but "does the environment under an UNCHANGED
# chart still hold". The install-test matrix follows the charts a pull request
# touches, so a chart nobody has edited since the forge move carries a green
# history from a different runner, and that green says nothing about today.
#
# It exists because the answer arrived once by accident: Gitea PR #48 corrected
# a comment in 21 charts, which made every one of them "changed", and so became
# the first all-charts run after the move - 19 of 20 matrix charts came up in
# 12m41s in one job, and exactly one did not (rss-bridge, Gitea #51). That
# result was worth having and nobody had planned it. This workflow makes it
# repeatable.
#
# WHEN TO RUN IT: after a change to the runner, to K3S_IMAGE, to the pinned
# KUBECTL_VERSION, to the fixtures under ci/ - and after a forge move. Not on a
# schedule: it would spend compute every day on a question that changes rarely,
# and a red run nobody asked for is a red run nobody reads.
name: Install-test (all charts)
on:
workflow_dispatch:
inputs:
charts:
required: false
type: string
default: ""
description: "charts to test, separated by spaces or commas - empty means every installable chart"
jobs:
select-charts:
name: "Select charts"
runs-on: ubuntu-latest
outputs:
charts: ${{ steps.select.outputs.charts }}
count: ${{ steps.select.outputs.count }}
steps:
- name: "Checkout Repository"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: "Select the charts to test"
id: select
env:
# The dispatch input is read as an environment variable and never
# interpolated into the script body. A "${{ inputs.charts }}" inside
# the shell would let whoever triggers the run write commands into
# this step; it is a small door and there is no reason to leave it
# open. The name check further down is the second half of that: these
# values end up as a matrix entry, a k3d cluster name and an argument
# to ci/run-install-test.sh.
REQUESTED: ${{ inputs.charts }}
run: |
FAIL=0
# --- The candidates -------------------------------------------------
# Empty input: every directory with a Chart.yaml, which is the same
# definition detect-changed-charts uses for "this is a chart". No list
# to maintain - adding a chart enrols it, the way the quantity render
# test enrols one.
#
# A subset was asked for: exactly those names. This is what lets
# someone test a single chart against the runner, which is what Gitea
# #51 needs - and it is why an unknown or excluded name below is an
# error rather than a silent omission. Asking for one chart and
# getting a green run over zero charts would be the worst answer this
# workflow could give.
if [ -z "$(printf '%s' "$REQUESTED" | tr -d '[:space:],')" ]; then
SUBSET=false
CANDIDATES="$(ls -d -- */Chart.yaml 2>/dev/null | sed 's|/Chart.yaml$||' | sort)"
echo "No subset requested - every chart directory is a candidate."
else
SUBSET=true
CANDIDATES="$(printf '%s' "$REQUESTED" | tr ',' ' ' | tr -s '[:space:]' '\n' | grep -v '^$' | sort -u)"
echo "Requested subset: $(echo "$CANDIDATES" | tr '\n' ' ')"
fi
for C in $CANDIDATES; do
case "$C" in
*[!a-z0-9-]*)
echo "::error::'$C' is not a chart name - chart directories in this repository are lowercase letters, digits and dashes."
FAIL=1
continue
;;
esac
if [ ! -f "$C/Chart.yaml" ]; then
echo "::error::'$C' is not a chart directory of this repository (no $C/Chart.yaml)."
FAIL=1
fi
done
[ "$FAIL" -eq 0 ] || exit 1
CHARTS="$(printf '%s\n' $CANDIDATES | jq -R . | jq -sc 'unique')"
# --- The exclusion list, through the one parser ---------------------
# ci/run-install-test.sh --list-excluded emits "<chart><TAB><reason>"
# and is the only thing in this repository that reads the format of
# ci/excluded-charts.txt - here no more than in validate-charts.yml,
# because a second parser would be a second chance to select
# different charts, silently and in either direction (issue #231).
# The comparison is jq's "!=" on whole strings, so a chart name is
# never read as a pattern.
#
# The two callers differ in what they do with a hit, and on purpose.
# Over all charts, an excluded chart is dropped and named with its
# reason, exactly as in the pull-request pipeline. In a requested
# subset it is an ERROR: someone typed that name, and quietly
# returning a green run for a chart that was never attempted is the
# lie #231 removed from the check list.
SUMMARY_ROWS=""
while IFS=$'\t' read -r EX_NAME EX_REASON; do
[ -n "$EX_NAME" ] || continue
if [ "$(echo "$CHARTS" | jq -r --arg c "$EX_NAME" 'index($c) != null')" = "true" ]; then
if [ "$SUBSET" = true ]; then
echo "::error::chart '$EX_NAME' was requested but is listed in ci/excluded-charts.txt: $EX_REASON"
FAIL=1
fi
# A "|" in the justification would break out of the table cell.
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$EX_NAME\` | ${EX_REASON//|/\\|} |"$'\n'
echo "Excluded from the install-test: $EX_NAME - $EX_REASON"
fi
CHARTS="$(echo "$CHARTS" | jq -c --arg c "$EX_NAME" 'map(select(. != $c))')"
done <<< "$(ci/run-install-test.sh --list-excluded)"
[ "$FAIL" -eq 0 ] || exit 1
# An empty matrix does not fail - the install-test job would simply
# not be created and the run would end green, having tested nothing.
# That is the shape of signal this whole issue is about, so it is a
# loud error instead.
if [ "$CHARTS" = "[]" ]; then
echo "::error::no chart left to test - every candidate is listed in ci/excluded-charts.txt."
exit 1
fi
COUNT="$(echo "$CHARTS" | jq -r 'length')"
echo "charts=$CHARTS" >> "$GITHUB_OUTPUT"
echo "count=$COUNT" >> "$GITHUB_OUTPUT"
echo "Install-test matrix ($COUNT charts): $CHARTS"
# --- The run record, first half -------------------------------------
# What was asked for, when, and against what. The per-chart outcome
# and the environment each leg measured are written by the legs
# themselves (the called workflow's "Record the result" step), the
# verdict by the record job below. A green run nobody notes is
# unknown again next time.
{
echo "## All-charts install-test"
echo
echo "- Started: **$(date -u '+%Y-%m-%d %H:%M') UTC**"
echo "- Requested: $( [ "$SUBSET" = true ] && echo "subset \`$REQUESTED\`" || echo "**every installable chart**" )"
echo "- Matrix: **$COUNT charts** — $(echo "$CHARTS" | jq -r 'map("`" + . + "`") | join(", ")')"
echo "- Expected runtime, so a deviation is visible as one: **20 charts in 12m41s**, measured on Gitea PR #48 — one job, all legs concurrently on one Docker daemon. The job timeout is 30 minutes per leg."
echo "- One throwaway k3d cluster per chart, no reuse: a reused cluster would carry leftovers, a warm image cache and fixtures already applied, and \"came up from nothing\" is the statement this test exists to make."
echo
echo "Each chart's outcome, with the runner label, \`K3S_IMAGE\` and \`kubectl\` version it was measured against, is in the install-test job summaries; the verdict over the whole matrix is at the end."
echo
} >> "$GITHUB_STEP_SUMMARY"
if [ -n "$SUMMARY_ROWS" ]; then
{
echo "### Charts excluded from the install-test"
echo
echo "These charts are listed in \`ci/excluded-charts.txt\` and get **no** install-test job. There is no green tick for them, because nothing was attempted (issue #231)."
echo
echo "| Chart | Documented reason |"
echo "| --- | --- |"
printf '%s' "$SUMMARY_ROWS"
echo
} >> "$GITHUB_STEP_SUMMARY"
fi
# The same workflow validate-charts.yml calls, with the same inputs, differing
# only in the chart list. Not a copy of the job: see the header of
# .github/workflows/install-test.yml for what would drift.
all-charts:
name: "All charts"
needs: select-charts
uses: ./.github/workflows/install-test.yml
with:
charts: ${{ needs.select-charts.outputs.charts }}
secrets: inherit
# The verdict over the matrix as a whole. A matrix leg only ever sees its own
# job summary, so this is the one place that can say "all of them" - and the
# aggregate is what the question "has the environment changed" is answered
# with. It runs on failure too, hence "if: always()".
record:
name: "Record the run"
needs: [select-charts, all-charts]
if: always()
runs-on: ubuntu-latest
steps:
- name: "Write the verdict"
env:
RESULT: ${{ needs.all-charts.result }}
COUNT: ${{ needs.select-charts.outputs.count }}
run: |
case "$RESULT" in
success)
VERDICT="**all $COUNT charts installed and became Ready** on this runner, on $(date -u '+%Y-%m-%d')"
;;
failure)
VERDICT="**at least one chart did not come up.** Which ones, and whether the chart failed or the cluster never came up, is in the per-chart records above. That is a result and not an accident to re-run: rss-bridge failing this way on Gitea PR #48 is why Gitea #51 exists"
;;
skipped|cancelled)
VERDICT="**no verdict** - the matrix did not run ($RESULT). Nothing was tested; do not read this run as a green one"
;;
*)
VERDICT="**no verdict** - unexpected matrix result '$RESULT'"
;;
esac
{
echo "## Verdict"
echo
echo "$VERDICT"
echo
echo "Worth writing down where it belongs: a result from this run is what the next \"has anything drifted\" question is compared against. If it moved a chart forward or backward, say so on the chart's wiki page or in \`Lessons-Learned\`."
} >> "$GITHUB_STEP_SUMMARY"
-417
View File
@@ -1,417 +0,0 @@
# The install-test, as one job with two callers.
#
# Both callers install a chart the SAME way; they differ only in which charts
# they name. validate-charts.yml passes the charts a pull request changed;
# install-test-all-charts.yml passes every installable chart on demand, to ask
# whether the environment underneath an UNCHANGED chart still holds (Gitea
# #52) - a chart untouched since the forge move carries a green history from a
# different runner, and that says nothing about today.
#
# It is a called workflow and not a second job, for the same reason
# ci/excluded-charts.txt has exactly one parser: a second copy is a second
# chance to test DIFFERENTLY, unnoticed. The parts that would drift are worth
# naming, because each was paid for once already - the pinned kubectl with its
# skew check, envsubst, the detection of a containerised job, the per-leg
# cluster name. A copy that slowly disagrees about any of them does not go red;
# it turns a green run into a statement about something other than what was
# assumed.
#
# What stays with the callers: WHICH charts are tested, and whether the job
# runs at all (needs/if). Everything that decides HOW one is installed is here.
name: Install-test
on:
workflow_call:
inputs:
charts:
required: true
type: string
description: 'JSON array of chart names, e.g. ["cyberchef","zipline"]. The entries of ci/excluded-charts.txt must already be gone - see the matrix comment below.'
secrets:
OP_CONNECT_TOKEN:
# Needed by exactly one leg - freesailarr, for the 1Password Connect
# lookup of its CI Wireguard key. Declared OPTIONAL on purpose, where
# the shared publish block in workflow-util/pipelines declares its token
# required: on this instance a missing secret does not fail the job, it
# fails job CREATION, so the run shows no steps at all (README, "Where
# this repo lives"; the long form is on that repository's wiki). With
# required: true a pull request from a fork - which carries no secrets -
# would lose the entire install-test that way, every chart at once, with
# nothing naming the cause. Optional instead: only the freesailarr leg
# fails, loudly, at the ":?" guards that name the missing op:// field.
required: false
description: "1Password Connect token; only the freesailarr leg reads it"
jobs:
install-test:
name: "Install-test ${{ matrix.chart }}"
# The one job in this repo that does not simply say ubuntu-latest, and the
# expression is not decoration - the same label means different machines on
# the two forges this repo currently lives on (#225).
#
# This is the only job that needs a Docker socket: it creates a k3d
# cluster, and k3d is a Docker client. On GitHub's hosted runners every
# label carries Docker, so the question never came up. On the Gitea
# instance it was MEASURED on a probe pull request, per job:
# detect-changed-charts, build-chart, the quantity render and the Trivy
# scan all pass on `ubuntu-latest` - none of them needs Docker - while the
# install-test dies there at "Create k3d cluster" with "Cannot connect to
# the Docker daemon at unix:///var/run/docker.sock", an error that points
# at k3d and never at the runner label. `ubuntu-docker` is the label that
# carries a socket, and Gitea does evaluate this expression: the probe job
# was picked up by the docker runner with labels ["ubuntu-docker"].
#
# Why an expression and not a plain `ubuntu-docker`: that label does not
# exist on GitHub, so every chart PR would sit queued forever for as long
# as this repo still builds on github.com - blocking the merge with no
# error anywhere. github.server_url is the one context value that separates
# the two without naming our instance.
#
# The label is necessary and was, for a while, not sufficient:
# `ubuntu-docker`'s daemon reported "Cgroup Driver: none" plus "Running in
# rootless-mode without cgroups", and k3d got as far as starting the k3s
# node, which restarted immediately with an empty log because k3s needs
# cgroup delegation (#225). Nothing in this file addressed that - it was
# dealt with on the instance side. It no longer blocks, and the evidence is
# Gitea PR #48: 19 of 20 matrix charts installed and became Ready on this
# label. The twentieth, rss-bridge, failed and is Gitea #51.
runs-on: ${{ github.server_url == 'https://github.com' && 'ubuntu-latest' || 'ubuntu-docker' }}
# Blocking since Renovate automerges patch/minor updates: a chart that no
# longer starts must stop the PR before it reaches main and publishes. This
# ends the stabilization phase of issue #22 - the job reported failures
# without blocking while it proved itself across several chart PRs.
#
# The 30 minutes are per leg and hold for the all-charts caller too, with
# the measured figure to compare against: Gitea PR #48 ran 20 charts in
# 12m41s in one job, all legs concurrently on one Docker daemon. If a run
# ever comes near this timeout, it is raised HERE, once, for both callers -
# a value that differs per caller would mean the two tests give up at
# different points while claiming to be the same test.
timeout-minutes: 30
strategy:
fail-fast: false
# The charts arrive from the caller ALREADY filtered against
# ci/excluded-charts.txt: validate-charts.yml does it in
# detect-changed-charts, install-test-all-charts.yml in select-charts, and
# both go through ci/run-install-test.sh --list-excluded, the one parser
# of that file. Both also name every chart they dropped WITH its
# documented reason in their own job summary, because an absent matrix
# entry says "not run" and never why (issue #231).
#
# DO NOT move that decision in here, e.g. as an "if" on this job
# if: ... && !contains(fromJSON(inputs.excluded-charts), matrix.chart)
# to get a visible "Skipped" per chart instead of an absent entry.
# GitHub rejects it - jobs.<id>.if is evaluated BEFORE the matrix is
# expanded, so the matrix context does not exist there:
# context "matrix" is not allowed here.
# available contexts are "github", "inputs", "needs", "vars".
# Measured on run 33650724109 (PR #233, closed): the whole workflow ended
# in failure with zero jobs created. The per-chart value has to be gone
# from the matrix before this workflow is called.
matrix:
chart: ${{ fromJSON(inputs.charts) }}
env:
K3D_VERSION: v5.9.0
K3S_IMAGE: docker.io/rancher/k3s:v1.35.5-k3s1
# Pinned rather than taken from the runner image. The argument is not
# "Gitea's image has no kubectl" - that is only how it was noticed. A
# kubectl inherited from whatever the image happens to ship drifts
# unannounced with every image rebuild, and an install-test is supposed
# to run against a KNOWN client. That GitHub's hosted image already
# carries one does not make the pin unnecessary; it only makes the drift
# invisible there. The value mirrors K3S_IMAGE above (v1.35.5), which is
# the cluster this client talks to.
# renovate: datasource=github-releases depName=kubernetes/kubernetes
KUBECTL_VERSION: v1.35.9
# The same expression as runs-on above, written a second time because the
# env context is not available in runs-on - so it cannot be the other way
# round, and reading the label back off the runner is not possible either
# (RUNNER_NAME below names the machine, not the label that picked it).
# It exists for the run record at the end of this job: "on which runner
# was this green tick measured" is the whole question of Gitea #52. Kept
# right under runs-on, so a change to one that forgets the other is
# visible on a single screen.
RUNNER_LABEL: ${{ github.server_url == 'https://github.com' && 'ubuntu-latest' || 'ubuntu-docker' }}
steps:
- name: "Checkout Repository"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: "Install Helm"
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
- name: "Install kubectl"
run: |
# kubectl and the cluster are a pair: the client talks to the server
# K3S_IMAGE starts, and Kubernetes supports at most one minor of
# skew between them. Renovate updates KUBECTL_VERSION on its own
# while K3S_IMAGE is tracked by nothing, so without this check the
# two drift apart until the client is talking to a server it no
# longer supports - silently, because kubectl only warns. And it is
# not theoretical: measured by dry run, a lookup from v1.34.0
# offers v1.37.0 in ONE step, because Renovate fills the non-major
# bucket with the newest candidate rather than walking the minors.
# So this check WILL fire one day, on purpose, with a message that
# says which of the two pins to move.
#
# Deliberately not solved with an allowedVersions constraint pinning
# kubectl to the k3s minor: that is the nginx trap the README
# describes - a filter that silently stops offering updates the day
# the other side moves. A loud failure beats a quiet freeze.
read -r KMAJ KMIN <<<"$(printf '%s' "$KUBECTL_VERSION" | sed -E 's/^v([0-9]+)\.([0-9]+).*/\1 \2/')"
read -r SMAJ SMIN <<<"$(printf '%s' "$K3S_IMAGE" | sed -E 's/.*:v([0-9]+)\.([0-9]+).*/\1 \2/')"
SKEW=$(( (KMAJ * 100 + KMIN) - (SMAJ * 100 + SMIN) ))
if [ "${SKEW#-}" -gt 1 ]; then
echo "::error::kubectl $KUBECTL_VERSION and k3s $K3S_IMAGE are more than one minor apart. Bump K3S_IMAGE to match, or pin KUBECTL_VERSION back."
exit 1
fi
curl -sSfL -o /tmp/kubectl "https://dl.k8s.io/release/${KUBECTL_VERSION}/bin/linux/amd64/kubectl"
mkdir -p "$HOME/.local/bin"
install -m 0755 /tmp/kubectl "$HOME/.local/bin/kubectl"
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
"$HOME/.local/bin/kubectl" version --client
- name: "Provide envsubst"
run: |
# ci/run-install-test.sh renders ci/<chart>/fixtures.env.yaml with
# envsubst - today only freesailarr has one, and without the tool
# the fixtures collapse to "error: no objects passed to apply",
# gluetun never gets its Wireguard key, and nothing behind its kill
# switch becomes Ready. GitHub's hosted image ships gettext-base,
# the Gitea runner image does not. The script is fine as it is; a
# standard tool is simply missing.
#
# CHECKED FOR, NOT PINNED - and that is a different decision from
# kubectl above, on purpose. kubectl's VERSION changes what the test
# means: its distance to the API server is a real source of failure,
# which is why it is pinned, tracked and guarded. envsubst
# substitutes text; which release performs the substitution does not
# change the result. Pinning it would buy nothing and add a line for
# Renovate to carry. So: present - do nothing, which keeps the
# GitHub path free of any package install; missing - fetch it, and
# the difference between the two runners is stated rather than
# hidden behind an unconditional apt-get.
if command -v envsubst >/dev/null 2>&1; then
echo "envsubst present at $(command -v envsubst) - nothing to do"
exit 0
fi
echo "envsubst missing on this runner image - installing gettext-base"
if [ "$(id -u)" -eq 0 ]; then SUDO=""; else SUDO="sudo"; fi
$SUDO apt-get update -qq
$SUDO apt-get install -y -qq --no-install-recommends gettext-base
command -v envsubst
# freesailarr is the one chart whose install-test needs credentials:
# without a real Wireguard key gluetun never completes a handshake, and
# behind its kill switch not a single app container becomes Ready. The key
# comes from 1Password Connect like the registry credentials in the shared
# publish block, and it is a CI-ONLY Mullvad key: Mullvad ties an
# address to a key and sends the tunnel to the endpoint it saw last, so
# reusing the production key would let every PR run pull the tunnel out
# from under the running stack - with the symptom in the cluster and the
# cause in a pull request. Loaded only for this chart; no other chart run
# touches the VPN key.
- name: "Configure OP-Connect"
if: matrix.chart == 'freesailarr'
uses: 1password/load-secrets-action/configure@70062d7a876d3eb6334754fa26efd2fbd90c32f2 # v5
with:
connect-host: https://op-connect.jk-dev.de
- name: "Load freesailarr VPN credentials"
id: load-freesailarr-vpn
if: matrix.chart == 'freesailarr'
uses: 1password/load-secrets-action@70062d7a876d3eb6334754fa26efd2fbd90c32f2 # v5
env:
OP_CONNECT_TOKEN: ${{ secrets.OP_CONNECT_TOKEN }}
FREESAILARR_WG_KEY: op://GitHub-Actions/freesailarr-mullvad-test-config/credential
FREESAILARR_WG_ADDRESSES: op://GitHub-Actions/freesailarr-mullvad-test-config/ip-address
# The same trap the shared publish block documents at length - since
# Gitea #66 that long form lives on the wiki of workflow-util/pipelines,
# page "publish-helm-chart", not in this repository: v5 of
# load-secrets-action does NOT export the loaded values as environment
# variables for later steps, only its step outputs carry them, and the
# loading step logs "Populating variable" and stays green either way. So
# the values are mapped into each step's own env block via
# steps.<id>.outputs.* and read as quoted shell variables. The ":?" guards
# run before the cluster is even created and name the op:// field that is
# missing - an empty key would otherwise reach gluetun and look like a
# broken chart instead of a missing secret. They print the variable name,
# never its value.
- name: "Verify the VPN credentials arrived"
if: matrix.chart == 'freesailarr'
env:
FREESAILARR_WG_KEY: ${{ steps.load-freesailarr-vpn.outputs.FREESAILARR_WG_KEY }}
FREESAILARR_WG_ADDRESSES: ${{ steps.load-freesailarr-vpn.outputs.FREESAILARR_WG_ADDRESSES }}
run: |-
: "${FREESAILARR_WG_KEY:?empty - check op://GitHub-Actions/freesailarr-mullvad-test-config/credential}"
: "${FREESAILARR_WG_ADDRESSES:?empty - check op://GitHub-Actions/freesailarr-mullvad-test-config/ip-address}"
echo "Both VPN values arrived non-empty."
- name: "Install k3d"
run: curl -sSfL https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | TAG="$K3D_VERSION" bash
# One cluster per matrix leg, named after the chart. It used to be the
# fixed name "install-test", and that was wrong on GitHub too - it just
# could not show there, because GitHub gives every matrix leg its own VM
# with its own Docker daemon, so two legs never saw each other's
# cluster. The environment hid the bug for two years rather than
# preventing it. Gitea runs all legs of a matrix CONCURRENTLY in ONE job
# on ONE daemon, and there the fixed name collides at once: measured on
# the probe PR of #225, one leg logs "Re-using existing network
# 'k3d-install-test'", the other dies on "The container name
# /k3d-install-test-server-0 is already in use" - and then the first
# leg's rollback deletes the shared network out from under the second,
# which fails with "network k3d-install-test not found". Both legs red,
# neither cause in either message. DO NOT put the fixed name back.
#
# Why the bare chart name and not a prefixed one: k3d rejects a cluster
# name over 32 characters ("Cluster name must be <= 32 characters",
# measured against k3d v5.9.0 - 32 passes, 33 fails), and every
# container is named k3d-<cluster>-server-0 from it. The longest chart
# here is calibre-web-automated at 21, so "install-test-<chart>" would
# be 34 and break exactly that one chart and no other - the kind of
# prefix that is worse than the fixed name because it only sometimes
# breaks. The guard below states the limit instead of leaving it to be
# rediscovered.
- name: "Create k3d cluster"
run: |
CLUSTER="${{ matrix.chart }}"
if [ "${#CLUSTER}" -gt 32 ]; then
echo "::error::chart name '$CLUSTER' is ${#CLUSTER} characters and k3d allows at most 32 for a cluster name."
exit 1
fi
# Does THIS job run inside a container? That, and not which forge we
# are on, is what decides whether k3d's cluster is reachable (#227).
#
# k3d publishes the API server port on the Docker *host*. When the
# job runs straight on the host - GitHub's hosted runners - that is
# the same machine, and the "https://0.0.0.0:<port>" k3d writes into
# the kubeconfig works. When the job itself is a container, as every
# Gitea Actions job is, the host is the dind container and 0.0.0.0
# points at the job container itself, where nothing listens. The
# cluster comes up fine and the first kubectl call is refused.
#
# Deliberately NOT keyed on github.server_url, although that would
# pick the right branch today: a self-hosted GitHub runner in a
# container has exactly this problem, and a condition that says
# "GitHub" while meaning "not containerised" is the kind of lie this
# repo exists to remove. So the state is detected instead of
# inferred, and the detection is the same question the fix needs
# answered anyway: ask the daemon about ourselves. Inside a
# container that the daemon knows, this returns our network and k3d
# joins it, which puts k3s and this job on one bridge where Docker's
# own DNS resolves the container names. On a host it returns nothing,
# because the hostname is not a container, and the old path is taken
# unchanged.
# The "|| true" is load-bearing: this step runs under `bash -e -o
# pipefail`, and on a host `docker inspect` exits non-zero, which
# would abort the step instead of taking the else branch.
JOB_NET="$(docker inspect -f '{{range $k, $v := .NetworkSettings.Networks}}{{$k}} {{end}}' "$(hostname)" 2>/dev/null | awk '{print $1}' || true)"
if [ -n "$JOB_NET" ]; then
echo "Containerised job: $(hostname) on docker network '$JOB_NET' - k3d joins it."
k3d cluster create "$CLUSTER" --image "$K3S_IMAGE" --network "$JOB_NET" --wait --timeout 180s
# Same reason: the published port is useless from in here, so the
# kubeconfig is pointed at the load balancer container by name,
# over the shared network. If the API server certificate did not
# carry that name this would fail on the TLS handshake and the
# cluster-info below would say so - it holds without an extra
# --tls-san, measured on the probe PR of #225.
kubectl config set-cluster "k3d-$CLUSTER" --server="https://k3d-$CLUSTER-serverlb:6443"
else
echo "Job runs on the docker host - k3d's published port is reachable as written."
k3d cluster create "$CLUSTER" --image "$K3S_IMAGE" --wait --timeout 180s
fi
# Reachability is proven HERE, and the shape of this check is the
# product of getting it wrong twice. Both mistakes are written down
# because both look like something to simplify away.
#
# It is a LOOP and not a single call. k3d's --wait returns when the
# node container reports ready, but the API server can still answer
# discovery with 503 for a moment after that. Measured: a single
# `kubectl cluster-info` right here failed with "ServiceUnavailable:
# the server is currently unable to handle the request" on one of
# two legs of the SAME GitHub run while the other passed. A check
# that is itself racy produces exactly the sporadic red this job
# must never have (#227) - and a test people have learned to re-run
# is one they have stopped believing.
#
# It counts READY NODES and not "did kubectl answer". `kubectl get
# nodes` exits 0 on an EMPTY list, so the first version passed
# against a cluster whose API server was up and whose node had never
# registered: the log said "No resources found", the step went
# green, and the failure surfaced five minutes later as a pod stuck
# in Pending with no node and no scheduling event - nowhere near its
# cause. A check that a broken cluster passes is worse than no
# check. `kubectl wait --for=condition=Ready nodes --all` is not the
# shorter way to write this: it has the identical hole, because with
# zero nodes the condition is vacuously satisfied.
#
# The evidence that this version earns its keep: only after it
# counted nodes did the real cause become visible in the dump below
# - k3s could not start containerd ("error creating fsnotify
# watcher: too many open files", an inotify limit on the host), and
# nothing above this line had ever said so.
for ATTEMPT in $(seq 1 90); do
READY_NODES="$(kubectl get nodes --no-headers 2>/dev/null | awk '$2 == "Ready"' | wc -l | tr -d ' ')"
[ "$READY_NODES" -ge 1 ] && break
if [ "$ATTEMPT" -eq 90 ]; then
echo "::error::the k3d cluster '$CLUSTER' answers but has no Ready node after 180s"
kubectl get nodes -o wide || true
kubectl describe nodes || true
echo "--- k3s server log ---"
docker logs "k3d-$CLUSTER-server-0" 2>&1 | tail -80 || true
exit 1
fi
sleep 2
done
kubectl get nodes -o wide
# Every chart reaching this step is one the exclusion filter let through,
# so the script's SKIP branch is never taken here - it exists for local
# runs. The variables below are empty for every chart but the one that
# declares them in ci/<chart>/settings.env (REQUIRED_ENV); the script
# fails loudly if a declared one is missing, it never skips over it.
- name: "Install chart and wait for readiness"
# The id is read by the run record at the end of this job: its outcome
# is what separates "the chart did not come up" from "the cluster never
# came up", and the two mean opposite things for Gitea #52.
id: install
env:
FREESAILARR_WG_KEY: ${{ steps.load-freesailarr-vpn.outputs.FREESAILARR_WG_KEY }}
FREESAILARR_WG_ADDRESSES: ${{ steps.load-freesailarr-vpn.outputs.FREESAILARR_WG_ADDRESSES }}
run: ci/run-install-test.sh "${{ matrix.chart }}"
# Must carry the same per-leg name as the create step above: a blanket
# cleanup, or the old fixed name, would tear down the neighbouring leg's
# cluster on a runner that shares one Docker daemon - which is precisely
# the half of the collision that turned one red leg into two.
- name: "Delete k3d cluster"
if: always()
run: k3d cluster delete "${{ matrix.chart }}" || true
# The run record (Gitea #52). A green install-test says "this chart came
# up" and nothing about WHERE, and where is exactly what the question
# needs: a chart that has not been touched since the forge move carries a
# green history from another environment. So every leg writes down the
# four facts that decide what its result means - date, runner label,
# K3S_IMAGE, kubectl - next to its own outcome. A green run nobody notes
# is unknown again next time.
#
# Per leg, and not one table for the whole matrix, because there is no
# place to write one: a matrix leg only ever sees its own
# GITHUB_STEP_SUMMARY, on either forge. The aggregate verdict is written
# by the calling workflow, which does see the matrix as a whole.
#
# It reports steps.install.outcome and not the job status: a leg that died
# in "Create k3d cluster" never installed anything, and recording that as
# a failed chart would blame the chart for the environment - precisely the
# confusion this issue is about.
- name: "Record the result"
if: always()
env:
CHART: ${{ matrix.chart }}
INSTALL_OUTCOME: ${{ steps.install.outcome }}
run: |
case "$INSTALL_OUTCOME" in
success) RESULT="installed, every workload Ready" ;;
failure) RESULT="**FAILED** - the chart did not come up" ;;
*) RESULT="**FAILED before the install** - cluster or tooling, not the chart (install step outcome: '${INSTALL_OUTCOME:-none}')" ;;
esac
{
echo "### Install-test \`$CHART\`: $RESULT"
echo
echo "| Date (UTC) | Runner label | Runner name | K3S_IMAGE | kubectl |"
echo "| --- | --- | --- | --- | --- |"
echo "| $(date -u '+%Y-%m-%d %H:%M') | \`$RUNNER_LABEL\` | \`${RUNNER_NAME:-unknown}\` | \`$K3S_IMAGE\` | \`$KUBECTL_VERSION\` |"
echo
} >> "$GITHUB_STEP_SUMMARY"
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-apache-tika:
name: Publish Apache Tika Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'apache-tika'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-bentopdf:
name: Publish BentoPDF Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'bentopdf'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
-16
View File
@@ -1,16 +0,0 @@
name: Publish Bulwark Helm Chart
on:
workflow_dispatch:
push:
branches:
- main
paths:
- bulwark/**
jobs:
publish-bulwark:
name: Publish Bulwark Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'bulwark'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,8 +9,7 @@ on:
jobs:
publish-calibre-web-automated:
name: Publish Calibre Web Automated Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'calibre-web-automated'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-cyberchef:
name: Publish Cyberchef Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'cyberchef'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-freesailarr:
name: Publish Freesailarr Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'freesailarr'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
-16
View File
@@ -1,16 +0,0 @@
name: Publish Garage Helm Chart
on:
workflow_dispatch:
push:
branches:
- main
paths:
- garage/**
jobs:
publish-garage:
name: Publish Garage Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'garage'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-gotenberg:
name: Publish Gotenberg Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'gotenberg'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+149
View File
@@ -0,0 +1,149 @@
name: Publish Helm Chart with patched Application Version
on:
workflow_dispatch:
inputs:
chart-location:
required: false
type: string
default: "."
description: "location of the helm chart directory"
chart-name:
required: true
type: string
description: "name of the helm chart"
workflow_call:
inputs:
chart-location:
required: false
type: string
default: "."
description: "location of the helm chart directory"
chart-name:
required: true
type: string
description: "name of the helm chart"
outputs:
chart-version:
description: "the version of the published helm chart"
value: ${{ jobs.publish-helm-chart.outputs.chart-version }}
secrets:
OP_CONNECT_TOKEN:
required: true
jobs:
publish-helm-chart:
outputs:
chart-version: ${{ steps.get-chart-version.outputs.CHART_VERSION }}
runs-on: ubuntu-latest
steps:
- name: "Checkout Repository"
uses: actions/checkout@v7
- name: "Install Helm"
uses: azure/setup-helm@v5
- name: "Configure OP-Connect"
uses: 1password/load-secrets-action/configure@v5
with:
connect-host: https://op-connect.jk-dev.de
# helm package exits 1 with "found in Chart.yaml, but missing in charts/
# directory" while a declared dependency has not been fetched, so this
# comes first. It is a no-op for every chart without a dependencies:
# block; ci/helm-dependencies.sh explains why it resolves from Chart.yaml
# instead of a committed Chart.lock. Note the consequence for the
# published artifact: the subchart .tgz is packaged INTO ours, so the
# version pinned in Chart.yaml is the version operators get, and nothing
# downstream ever sees the dependency again.
- name: "Fetch chart dependencies"
run: ${{ github.workspace }}/ci/helm-dependencies.sh "${{ inputs.chart-name }}"
working-directory: ${{ inputs.chart-location }}
- name: "Package Helm Chart"
run: helm package ${{ inputs.chart-name }}
working-directory: ${{ inputs.chart-location }}
- name: "Get Helm Chart Version"
id: get-chart-version
run: |-
CHART_VERSION=$(grep '^version:' ${{ inputs.chart-name}}/Chart.yaml | awk '{print $2}')
echo $CHART_VERSION
echo "CHART_VERSION=$CHART_VERSION" >> $GITHUB_OUTPUT
working-directory: ${{ inputs.chart-location }}
- name: "Load Chart Registry Credentials"
id: load-registry-creds
uses: 1password/load-secrets-action@v5
env:
OP_CONNECT_TOKEN: ${{ secrets.OP_CONNECT_TOKEN }}
REGISTRY_PASSWORD: op://GitHub-Actions/jlab-registry-access/password
REGISTRY_USERNAME: op://GitHub-Actions/jlab-registry-access/username
REGISTRY_CHARTS_URL: op://GitHub-Actions/jlab-registry-access/charts-url
# The target is a classic, index-based Helm registry, not an OCI one, so
# there is no "helm registry login" session and no "helm push": the upload
# is a single HTTP POST of the packaged .tgz to <charts-url>/api/charts,
# authenticated per request. Three properties of this step are load-bearing
# and each of them was paid for once already.
#
# 1) The values are mapped into the step's own env block and read as quoted
# shell variables - never interpolated into the command line. load-secrets-
# action v5 (bumped in #140) no longer exports the loaded values as
# environment variables for later steps the way v4 did; only the step
# outputs carry them. Every "${{ env.* }}" reference therefore resolved to
# an empty string from that bump on, with nothing reporting an error - the
# loading step logs "Populating variable" and stays green either way. The
# ":?" guards fail loudly and name the op:// field that is missing, and
# quoting keeps an empty value from silently shifting arguments.
#
# 2) The credentials never reach the process list. "curl --user $U:$P"
# would put them in argv; "curl --config -" reads its options from stdin
# instead, so nothing sensitive appears in any argument. The price of that
# config file is its value syntax: an unquoted value ends at the first
# space (measured on curl 8.5.0 - "has space" arrived as "has"), so the
# step refuses whitespace in the credentials up front rather than letting
# a truncated password come back as a puzzling 401. A Gitea access token
# is 40 hex characters, so this only ever fires on a mis-entered item.
#
# 3) "--fail-with-body" is what makes a failed upload fail the job. Unlike
# "helm push", curl exits 0 on HTTP 401/403/409 unless asked not to - the
# publish would go green while nothing was uploaded. Measured against Gitea
# 1.24.7: a 401 without the flag is exit 0, with the flag exit 22, and the
# response body is still printed so the log names the cause.
#
# The file name is built from the chart name and the version read above
# instead of a "<chart>-*.tgz" glob: --upload-file takes exactly one file,
# and a glob matching two turns the second into curl's URL argument, while
# a glob matching none is passed through literally. Both fail obscurely.
# "helm package" keeps the "+" of a "1.4.0+up1.9.2" version verbatim in the
# file name, so "<name>-<version>.tgz" is an exact match (measured).
- name: "Upload Helm Chart to the Chart Registry"
env:
REGISTRY_PASSWORD: ${{ steps.load-registry-creds.outputs.REGISTRY_PASSWORD }}
REGISTRY_USERNAME: ${{ steps.load-registry-creds.outputs.REGISTRY_USERNAME }}
REGISTRY_CHARTS_URL: ${{ steps.load-registry-creds.outputs.REGISTRY_CHARTS_URL }}
CHART_NAME: ${{ inputs.chart-name }}
CHART_VERSION: ${{ steps.get-chart-version.outputs.CHART_VERSION }}
run: |-
: "${REGISTRY_CHARTS_URL:?empty - check op://GitHub-Actions/jlab-registry-access/charts-url}"
: "${REGISTRY_USERNAME:?empty - check op://GitHub-Actions/jlab-registry-access/username}"
: "${REGISTRY_PASSWORD:?empty - check op://GitHub-Actions/jlab-registry-access/password}"
: "${CHART_NAME:?empty - the chart-name input did not arrive}"
: "${CHART_VERSION:?empty - could not read version: from ${CHART_NAME}/Chart.yaml}"
# 1Password masks the full URL, but a curl connection error prints only
# "<host>:<port>", which that mask would not cover. Mask the host too.
CHARTS_HOST="${REGISTRY_CHARTS_URL#*://}"
echo "::add-mask::${CHARTS_HOST%%/*}"
case "${REGISTRY_USERNAME}${REGISTRY_PASSWORD}" in
*[[:space:]]*)
echo "::error::username/password in op://GitHub-Actions/jlab-registry-access must not contain whitespace - curl --config truncates an unquoted value at the first space"
exit 1
;;
esac
CHART_PACKAGE="${CHART_NAME}-${CHART_VERSION}.tgz"
test -f "$CHART_PACKAGE" || {
echo "::error::${CHART_PACKAGE} does not exist - helm package produced: $(ls -1 ./*.tgz 2>&1)"
exit 1
}
echo "uploading ${CHART_PACKAGE}"
printf 'user = %s:%s\nurl = %s\n' \
"$REGISTRY_USERNAME" "$REGISTRY_PASSWORD" "${REGISTRY_CHARTS_URL%/}/api/charts" |
curl --config - \
--fail-with-body \
--silent --show-error \
--request POST \
--upload-file "$CHART_PACKAGE" \
--write-out '\nHTTP %{http_code}\n'
working-directory: ${{ inputs.chart-location }}
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-homeassistant:
name: Publish Homeassistant Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'homeassistant'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
-16
View File
@@ -1,16 +0,0 @@
name: Publish Keycloak Helm Chart
on:
workflow_dispatch:
push:
branches:
- main
paths:
- keycloak/**
jobs:
publish-keycloak:
name: Publish Keycloak Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'keycloak'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-matrix-synapse:
name: Publish Matrix Synapse Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'matrix-synapse'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,8 +9,7 @@ on:
jobs:
publish-minecraft-router:
name: Publish Minecraft-Router Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'minecraft-router'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,8 +9,7 @@ on:
jobs:
publish-minecraft-server:
name: Publish Minecraft-Server Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'minecraft-server'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-outline:
name: Publish Outline Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'outline'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-paperless-ngx:
name: Publish Paperless-NGX Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'paperless-ngx'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-patchmon:
name: Publish Patchmon Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'patchmon'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-rallly:
name: Publish Rallly Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'rallly'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-rss-bridge:
name: Publish RSS-Bridge Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'rss-bridge'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-samba:
name: Publish Samba Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'samba'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-satisfactory:
name: Publish Satisfactory Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'satisfactory'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-searxng:
name: Publish SearXNG Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'searxng'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
-16
View File
@@ -1,16 +0,0 @@
name: Publish Stalwart Helm Chart
on:
workflow_dispatch:
push:
branches:
- main
paths:
- stalwart/**
jobs:
publish-stalwart:
name: Publish Stalwart Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'stalwart'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
-16
View File
@@ -1,16 +0,0 @@
name: Publish Sure Helm Chart
on:
workflow_dispatch:
push:
branches:
- main
paths:
- sure/**
jobs:
publish-sure:
name: Publish Sure Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'sure'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,8 +9,7 @@ on:
jobs:
publish-teamspeak-server:
name: Publish TeamSpeak Server Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'teamspeak-server'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,8 +9,7 @@ on:
jobs:
publish-urlaubsverwaltung:
name: Publish Urlaubsverwaltung Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'urlaubsverwaltung'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
-16
View File
@@ -1,16 +0,0 @@
name: Publish Valkey Helm Chart
on:
workflow_dispatch:
push:
branches:
- main
paths:
- valkey/**
jobs:
publish-valkey:
name: Publish Valkey Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: 'valkey'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-voucher-vault:
name: Publish VoucherVault Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'voucher-vault'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
@@ -9,8 +9,7 @@ on:
jobs:
publish-ytdlp-web-player:
name: Publish YTDLP Web Player Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'ytdlp-web-player'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
+1 -2
View File
@@ -9,8 +9,7 @@ on:
jobs:
publish-zipline:
name: Publish Zipline Helm Chart
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
uses: ./.github/workflows/publish-helm-chart.yml
with:
chart-name: 'zipline'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
-37
View File
@@ -1,37 +0,0 @@
name: SonarQube Analysis
# The analysis itself lives in workflow-util/pipelines, bound to a tag like
# every other call to that repository. This hull only decides WHEN it runs.
#
# No "paths" filter, unlike the publish-<chart>.yml hulls: the analysis looks at
# the whole tree - the five shell scripts under ci/, every workflow file, the
# YAML and Helm templates, the values.schema.json files, and the secret
# detection that covers everything. A filter would mean a change outside the
# filter leaves the last result standing while claiming to be current.
on:
# push on main, and only main, because the SonarQube Community Build analyses
# the main branch only - there is no pull-request analysis to hook into. The
# consequence belongs said out loud rather than discovered: a finding appears
# AFTER the merge. This is an observation of the state of main, not a gate,
# and the shared block deliberately carries no quality-gate step.
push:
branches:
- main
# workflow_dispatch, and it is not decoration. Three things about this path
# are unmeasured - whether node24 is in the runner image, whether the ~59 MiB
# scanner download gets through (a 17 MB one once did not, MTU 1500 against a
# pod network of 1450), and whether a repository variable reaches a workflow
# called from another repository. Any of them may need more than one attempt,
# and without a dispatch every attempt would cost a dummy commit on main.
# Only the maintainer can trigger it; the bot account cannot dispatch.
workflow_dispatch:
jobs:
sonar:
name: SonarQube Analysis
uses: workflow-util/pipelines/.gitea/workflows/sonar-analysis.yml@v4
# secrets: inherit passes SONAR_TOKEN through. The block declares it as
# required, and a missing required secret fails job CREATION on this
# instance, not the job - the run would stand there with no steps at all.
# SONAR_HOST_URL is a repository variable and needs nothing here.
secrets: inherit
+106 -86
View File
@@ -15,7 +15,7 @@ jobs:
any-install: ${{ steps.detect.outputs.any-install }}
steps:
- name: "Checkout Repository"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
uses: actions/checkout@v7
with:
fetch-depth: 0
- name: "Detect changed charts"
@@ -116,11 +116,11 @@ jobs:
chart: ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }}
steps:
- name: "Checkout Repository"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
uses: actions/checkout@v7
with:
fetch-depth: 0
- name: "Install Helm"
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
uses: azure/setup-helm@v5
# Both steps below refuse to run while a chart declares a dependency that
# is not in its charts/ directory - lint with "[WARNING] chart directory
# is missing these dependencies", package with "found in Chart.yaml, but
@@ -157,9 +157,8 @@ jobs:
# against - resource quantities formatted with printf "%v", where float64
# flips to exponential notation above ~1e6 - lives in the resources helper,
# and that helper is copied into every chart's templates/_helpers.tpl,
# because sharing it would mean a library chart (README, guideline 14). So
# it comes back by copy: a new chart started from an older one, or from
# template-chart before the fix.
# because charts here have no dependencies. So it comes back by copy: a new
# chart started from an older one, or from template-chart before the fix.
# In that PR the chart is new and its copy of the bug is in no diff anyone
# reads. Running over everything also means there is no list to maintain -
# adding a chart enrolls it. Two renders per chart, no cluster, a few
@@ -167,42 +166,12 @@ jobs:
# build-chart, so it adds no wall-clock time to the pipeline.
steps:
- name: "Checkout Repository"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
uses: actions/checkout@v7
- name: "Install Helm"
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
uses: azure/setup-helm@v5
- name: "Render every chart with bare integer quantities"
run: ci/run-quantity-render-test.sh
selector-match:
name: "Selector match"
runs-on: ubuntu-latest
# The second job that ignores the changed-chart matrix and runs over *every*
# chart (Gitea #96), and for the same reason as quantity-render above: the
# mistake arrives by copy into a chart whose own diff nobody reads for it.
#
# What it asserts: every PodDisruptionBudget selector and every
# topologySpreadConstraints labelSelector a chart renders really matches the
# labels of a pod template in the SAME render. Both objects fail SILENTLY
# when their selector matches nothing - the spread constraint becomes a
# condition over zero pods and is trivially satisfied, the budget does its
# arithmetic over zero pods and never blocks an eviction - so neither a
# render error nor the install-test can catch it, and with
# whenUnsatisfiable: DoNotSchedule the object reads like a guarantee while
# distributing nothing.
#
# It needs no cluster and no Trivy: ~14 seconds for the whole repository,
# depends on no other job, and finishes well inside build-chart. The YAML is
# parsed by Helm itself through ci/yaml2json, because the runners have
# neither pyyaml nor yq and a hand-written parser would be able to read
# nothing and report nothing - see the script's header.
steps:
- name: "Checkout Repository"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: "Install Helm"
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
- name: "Check every rendered selector against the rendered pod labels"
run: ci/run-selector-match-test.sh
trivy-config:
name: "Trivy config ${{ matrix.chart }}"
needs: detect-changed-charts
@@ -220,37 +189,21 @@ jobs:
chart: ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }}
env:
# renovate: datasource=github-releases depName=aquasecurity/trivy
TRIVY_VERSION: v0.75.0
TRIVY_VERSION: v0.74.0
# "report" always exits 0. Switch to "gate" to make a finding that no
# documented exception covers fail the job - the intended step once the
# seccomp gap (#84) is closed.
TRIVY_CONFIG_MODE: report
steps:
- name: "Checkout Repository"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
uses: actions/checkout@v7
- name: "Install Helm"
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
# A plain release download rather than aquasecurity/setup-trivy, and the
# reason is measured on the Gitea probe PR of #225, not assumed. That
# action runs actions/checkout against aquasecurity/trivy on github.com
# and hands it `github.token` - which on Gitea is a GITEA token.
# github.com answers 401, checkout retries twice and the job ends on
# "fatal: could not read Username for 'https://github.com': terminal
# prompts disabled", an error naming neither the action nor the token.
# Its actions/cache step is inert on the instance as well ("Cache action
# is only supported on GHES version >= 3.5"), and fetching the nested
# actions cost 4m30s before the job even started. One tarball behaves
# identically on both forges, and the `# renovate:` comment above still
# keeps TRIVY_VERSION current - verified by dry run: pinned to v0.72.0
# the lookup offers v0.74.0 as a minor update.
uses: azure/setup-helm@v5
- name: "Install Trivy"
run: |
curl -sSfL -o /tmp/trivy.tar.gz \
"https://github.com/aquasecurity/trivy/releases/download/${TRIVY_VERSION}/trivy_${TRIVY_VERSION#v}_Linux-64bit.tar.gz"
mkdir -p "$HOME/.local/bin"
tar -xzf /tmp/trivy.tar.gz -C "$HOME/.local/bin" trivy
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
"$HOME/.local/bin/trivy" --version
uses: aquasecurity/setup-trivy@v0.3.1
with:
version: ${{ env.TRIVY_VERSION }}
cache: true
# Findings land in the job summary, not only in the log: exceptions with
# their reasons come from ci/trivy/trivyignore.yaml, the trusted-registry
# list from ci/trivy/data/ksv0125.yaml.
@@ -258,30 +211,97 @@ jobs:
run: ci/run-trivy-config.sh "${{ matrix.chart }}"
install-test:
name: "Install-test"
name: "Install-test ${{ matrix.chart }}"
needs: [detect-changed-charts, build-chart]
if: needs.detect-changed-charts.outputs.any-install == 'true'
# The job itself lives in .github/workflows/install-test.yml and is called
# from here and from install-test-all-charts.yml (Gitea #52). Everything
# that decides HOW a chart is installed moved there unchanged - the runner
# label expression, the pinned kubectl with its skew check, envsubst, the
# containerised-job detection, the per-leg cluster name, the freesailarr VPN
# credentials. What stayed here is WHICH charts are tested and whether the
# job runs at all. A copied job would be a second chance to test
# differently, unnoticed, which is the same argument that gives
# ci/excluded-charts.txt exactly one parser.
#
# "install-charts", not "charts": the changed charts minus the ones in
# ci/excluded-charts.txt, filtered in detect-changed-charts, which also
# names every excluded chart of this pull request with its documented
# reason in its job summary (issue #231). The called workflow's matrix
# comment carries the rest of that lesson, including why the decision
# cannot be made per matrix leg instead.
uses: ./.github/workflows/install-test.yml
with:
charts: ${{ needs.detect-changed-charts.outputs.install-charts }}
# As the 24 publish-<chart>.yml hulls do, and measured to work on this
# instance (README, "Where this repo lives"). The called workflow declares
# exactly one secret, OP_CONNECT_TOKEN, and only its freesailarr leg reads
# it.
secrets: inherit
runs-on: ubuntu-latest
# Blocking since Renovate automerges patch/minor updates: a chart that no
# longer starts must stop the PR before it reaches main and publishes. This
# ends the stabilization phase of issue #22 - the job reported failures
# without blocking while it proved itself across several chart PRs.
timeout-minutes: 30
strategy:
fail-fast: false
# "install-charts", not "charts": the changed charts minus the ones in
# ci/excluded-charts.txt, filtered in detect-changed-charts. Excluded
# charts are named with their reason in that job's summary (issue #231).
#
# DO NOT try to move that decision into an "if" on this job, e.g.
# if: ... && !contains(fromJSON(needs.detect-changed-charts.outputs.excluded-charts), matrix.chart)
# to get a visible "Skipped" per chart instead of an absent entry.
# GitHub rejects it - jobs.<id>.if is evaluated BEFORE the matrix is
# expanded, so the matrix context does not exist there:
# context "matrix" is not allowed here.
# available contexts are "github", "inputs", "needs", "vars".
# Measured on run 33650724109 (PR #233, closed): the whole workflow ended
# in failure with zero jobs created. The per-chart value has to be gone
# from the matrix by the time this job is reached.
matrix:
chart: ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }}
env:
K3D_VERSION: v5.9.0
K3S_IMAGE: docker.io/rancher/k3s:v1.35.5-k3s1
steps:
- name: "Checkout Repository"
uses: actions/checkout@v7
- name: "Install Helm"
uses: azure/setup-helm@v5
# freesailarr is the one chart whose install-test needs credentials:
# without a real Wireguard key gluetun never completes a handshake, and
# behind its kill switch not a single app container becomes Ready. The key
# comes from 1Password Connect like the registry credentials in
# publish-helm-chart.yml, and it is a CI-ONLY Mullvad key: Mullvad ties an
# address to a key and sends the tunnel to the endpoint it saw last, so
# reusing the production key would let every PR run pull the tunnel out
# from under the running stack - with the symptom in the cluster and the
# cause in a pull request. Loaded only for this chart; no other chart run
# touches the VPN key.
- name: "Configure OP-Connect"
if: matrix.chart == 'freesailarr'
uses: 1password/load-secrets-action/configure@v5
with:
connect-host: https://op-connect.jk-dev.de
- name: "Load freesailarr VPN credentials"
id: load-freesailarr-vpn
if: matrix.chart == 'freesailarr'
uses: 1password/load-secrets-action@v5
env:
OP_CONNECT_TOKEN: ${{ secrets.OP_CONNECT_TOKEN }}
FREESAILARR_WG_KEY: op://GitHub-Actions/freesailarr-mullvad-test-config/credential
FREESAILARR_WG_ADDRESSES: op://GitHub-Actions/freesailarr-mullvad-test-config/ip-address
# The same trap publish-helm-chart.yml documents at length: v5 of
# load-secrets-action does NOT export the loaded values as environment
# variables for later steps, only its step outputs carry them, and the
# loading step logs "Populating variable" and stays green either way. So
# the values are mapped into each step's own env block via
# steps.<id>.outputs.* and read as quoted shell variables. The ":?" guards
# run before the cluster is even created and name the op:// field that is
# missing - an empty key would otherwise reach gluetun and look like a
# broken chart instead of a missing secret. They print the variable name,
# never its value.
- name: "Verify the VPN credentials arrived"
if: matrix.chart == 'freesailarr'
env:
FREESAILARR_WG_KEY: ${{ steps.load-freesailarr-vpn.outputs.FREESAILARR_WG_KEY }}
FREESAILARR_WG_ADDRESSES: ${{ steps.load-freesailarr-vpn.outputs.FREESAILARR_WG_ADDRESSES }}
run: |-
: "${FREESAILARR_WG_KEY:?empty - check op://GitHub-Actions/freesailarr-mullvad-test-config/credential}"
: "${FREESAILARR_WG_ADDRESSES:?empty - check op://GitHub-Actions/freesailarr-mullvad-test-config/ip-address}"
echo "Both VPN values arrived non-empty."
- name: "Install k3d"
run: curl -sSfL https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | TAG="$K3D_VERSION" bash
- name: "Create k3d cluster"
run: k3d cluster create install-test --image "$K3S_IMAGE" --wait --timeout 180s
# Every chart reaching this step is one the exclusion filter let through,
# so the script's SKIP branch is never taken here - it exists for local
# runs. The variables below are empty for every chart but the one that
# declares them in ci/<chart>/settings.env (REQUIRED_ENV); the script
# fails loudly if a declared one is missing, it never skips over it.
- name: "Install chart and wait for readiness"
env:
FREESAILARR_WG_KEY: ${{ steps.load-freesailarr-vpn.outputs.FREESAILARR_WG_KEY }}
FREESAILARR_WG_ADDRESSES: ${{ steps.load-freesailarr-vpn.outputs.FREESAILARR_WG_ADDRESSES }}
run: ci/run-install-test.sh "${{ matrix.chart }}"
- name: "Delete k3d cluster"
if: always()
run: k3d cluster delete install-test || true
+7 -55
View File
@@ -7,74 +7,26 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
The main Claude session acts as the **task manager** and does not implement issues itself:
1. **Plan first, get approval**: The manager presents a plan for the issue and waits for the user's explicit approval. No implementation before approval.
2. **Document the final plan on the issue**: Once approved, the plan is posted as a comment on the issue before implementation starts (skip if the issue body already contains the full plan; later decisions go there as comments too).
2. **Document the final plan on the issue**: Once approved, the plan is posted as a comment on the GitHub issue before implementation starts (skip if the issue body already contains the full plan; later decisions go there as comments too).
3. **Delegate implementation**: After approval, the manager launches a dedicated **development agent**. The agent implements the issue completely on its own branch and opens a PR.
4. **PR comment handling by the agent**: After opening the PR, the development agent watches it for review comments. New comments are addressed automatically (code changes, reply on the thread), and addressed threads are marked as **resolved**.
5. **Update the wiki — mandatory, and before the merge is requested.** The wiki is this repo's knowledge base; a change that lands in `main` while the wiki still describes the old state has not been finished, it has been half-finished in a way nothing fails on. The development agent edits the affected pages as part of its issue, so code and documentation are reviewed together. See "The wiki" below for what obliges an edit and what does not.
6. **Agent lifetime**: The development agent terminates itself once its PR has been merged (or closed).
7. Only after the PR has been **merged by the user** may work on the next issue begin. The user merges; never merge yourself.
8. **After the merge, verify the linked issue was closed** (auto-close via "Fixes #n"); close it manually if not. Gitea auto-closes on the same keywords, but only for an issue in the same repository — a migrated issue whose number changed is closed by hand. Check in the same step that the wiki edit really landed.
5. **Agent lifetime**: The development agent terminates itself once its PR has been merged (or closed).
6. Only after the PR has been **merged by the user** may work on the next issue begin. The user merges; never merge yourself.
7. **After the merge, verify the linked issue was closed** (auto-close via "Fixes #n"); close it manually if not.
Additional rules for all commits, PRs, issues, and comments:
- Do **not** add a `Co-Authored-By` trailer and do **not** append "Generated with Claude Code" footers.
- **Commits must be signed**, and in the Coder workspace they are: signing runs through its own SSH signer (`gpg.format = ssh`, `gpg.ssh.program = coder-git-ssh-sign`, `commit.gpgsign = true`), so no 1Password step is involved. Verify before opening the PR — `git log --format='%G?' -1` must print `G` — and if signing fails, stop and report rather than pushing unsigned work, because branch protection rejects it at merge time.
- **A PR that does not finish its issue writes `Part of #n`, never `Fixes #n`.** The keyword closes the issue on merge, so a `Fixes` on a PR that leaves work behind marks an open task as done — and reopening by hand is the only repair, which only happens if someone remembers to look. This is not hypothetical: Gitea PR #48 carried `Fixes #40` while `rss-bridge` had been deliberately left out of the sweep, and the merge closed Gitea #40 with one occurrence still open. Step 8 above is the other half of the same rule: the auto-close it verifies is only the right outcome when the keyword was the right one.
## The wiki
The knowledge base is the Gitea wiki at `https://git.jk-dev.de/jlab-cluster/helm-charts/wiki` — one page per chart (installation, required 1Password fields, every configurable value, the chart's traps) plus the long form of this workflow, the pipelines, the style guide and the incidents each rule was bought with. It is a **separate git repository**:
```bash
git clone ssh://git@git.jk-dev.de:30022/jlab-cluster/helm-charts.wiki.git
```
Its default branch is `main`, not `master`.
Flat layout, one Markdown file per page, page name = file name, links are relative page names without the extension (`[CI/CD](CI-CD)`). A new page must also be listed in `_Sidebar.md`, or it is reachable only by search.
**There is no PR flow here, so fetch and rebase before every push and never force-push** — two agents have already edited the same page minutes apart, and a force push deletes the other edit silently, with nothing left to report it.
**What obliges a wiki edit** (step 5 of the workflow above, same round of work, never a follow-up issue):
| Change | Page |
| --- | --- |
| a value added, removed, renamed, or its default changed | the chart's page — values table, and the operating notes if behaviour moved |
| runtime behaviour changed (probes, hardening, restart semantics, a value becoming required) | the chart's page, plus `Lessons-Learned` if the reason could be repeated elsewhere |
| a new chart | a new chart page, a row in `Charts`, an entry in `_Sidebar.md` |
| a convention added or changed | `Chart-Style-Guide` (and README, which stays the code-side copy) |
| pipeline, fixture or runner behaviour changed | `CI-CD` |
| this issue workflow changed | `Workflow` — **and** this file, which is the binding copy |
| a Renovate rule added or changed | `Renovate` |
| something broke in a way a rule now prevents | `Lessons-Learned` |
**What does not oblige an edit:** a plain app-version bump. Renovate automerges those continuously and never touches the wiki, which is why chart pages carry no version numbers at all and name `Chart.yaml` as the source of truth. An app update that *does* change values or behaviour is not a plain bump, and the table applies.
Wiki commits follow the same rules as repo commits: no `Co-Authored-By`, no "Generated with Claude Code" footer.
## Which forge, and with which tool
Everything runs on the Gitea instance JLab-Git: issues, pull requests, CI and Renovate. github.com is a push mirror with Actions and Renovate **switched off**, and it lags until the mirror runs. README, "Where this repo lives", holds the measured differences between a Gitea runner and a GitHub one.
- **`tea`, not `gh`.** Work happens in a **Coder workspace, and it has no `op`** — that is the normal case here, not a broken environment, so do not wait for the 1Password CLI or propose installing it. Call `tea` directly; it carries the workspace's stored login and opens PRs and issues under the **bot account**, which is accepted. Say so in the report when something is authored that way. (`op plugin run -- tea`, the older instruction, applies only where `op` actually exists.)
- **The remote is `origin`, and it points at Gitea** (`ssh://git@git.jk-dev.de:30022/jlab-cluster/helm-charts.git`). The checkout has exactly that one, and so does every worktree cut from it, so a bare `git push` goes to the leading copy; the github.com mirror is fed by Gitea itself and is never pushed to by hand. A clone elsewhere may carry the mirror as a second remote named `github` — wherever more than one exists, say which one you mean.
- **Cut every worktree from a freshly fetched `origin/main`.** `git fetch origin main` first, then branch from `origin/main` — never from the local `main` ref, and never assume the shared checkout is current: both can be many merges behind. The staleness is invisible where it matters most, and it outlives the fix: the project-instructions block a subagent receives is a **snapshot** of the spawning session's working directory, taken when **that session started** — not a live read. Fetching mid-session does not refresh it, so the file on disk can be correct while every agent started from then on is still handed the superseded rules. Measured on 2026-09-27: an agent whose own worktree was identical to `origin/main` (`git rev-list --left-right --count origin/main...HEAD` returned `0 0`) received a block from before Gitea #47 and reported a contradiction in this file that two PRs had already fixed, quoting the stale copy as binding. Two consequences, and only these two help: the spawning session's directory must be current **before** the session starts, and an agent checks a rule against the **file in its own worktree** plus a diff against `origin/main` — never against `main`, and never against its own context block, because the first consequence cannot be enforced from inside an agent and the second can.
- **Branch protection with a signature requirement is active**, and signatures verify on Gitea. A commit that is not signed does not reach `main` — check before opening a PR, do not discover it at merge time.
- **No merge queue.** Gitea has none, and its place is taken by **auto-merge on green checks**: the PR is armed once and merges itself when the required check passes **and the required review exists** — both are active on `main`, so an armed PR without a review sits there instead of merging. Rule 7 above is unchanged — the user merges, and an agent never merges its own PR or arms auto-merge on it.
- **A required check per chart is not possible.** Gitea runs a whole matrix as one job under one status context, so the install-test reports once for all charts. What is required on `main` is `validate-charts`, as that single context; per chart is the part that cannot be had. Relevant whenever branch protection is touched.
- **Issue numbers changed with the move, and `#n` in this repo's files means the GitHub tracker.** The issues open at the time are Gitea #5–#21 and #24; the mapping is a comment on Gitea #14. Closed ones were not migrated and exist only on GitHub. Beware that Gitea silently resolves a bare `#n` against its own counter, so an old reference can render as a link to something unrelated. When you write a *new* reference, use the Gitea number.
- **Renovate runs outside this repo**, as the shared `jlab-renovate-bot` instance; it reads our `renovate.json5` and its `postUpgradeTasks` hook is allow-listed there. **Do not add a Renovate workflow here** — a second runner means duplicate PRs and two schedulers rewriting each other's branches. README, "Renovate on Gitea", has the evidence.
## Keep the docs current
- Changes to this issue workflow always go into this CLAUDE.md file.
- General repository information (charts, CI pipelines, conventions, tooling) always lives in README.md. Any PR that changes repo-wide behavior must update README.md in the same PR.
- **Operator-facing knowledge lives in the wiki**, not in README.md: how a chart is installed, what every value does, what its traps are. The division is `CLAUDE.md` binds the agent, `README.md` explains the repository to someone reading the code, the wiki explains the system to whoever runs it, and a chart's `NOTES.txt` explains one release. Where README and the wiki cover the same rule, both are updated in the same PR — see "The wiki" above.
- Runtime verification of chart changes is done in a local, throwaway k3d cluster (k3d is installed on the maintainer's machine): stub CRD for `OnePasswordItem`, dummy secrets, throwaway infrastructure (postgres/redis) where needed.
## What this repo is
A collection of self-made Helm charts, one directory per chart, published to a private, classic (index-based) Helm registry by Gitea Actions — an HTTP `POST` of the packaged `.tgz`, not an OCI `helm push`; the target URL and its credentials live in 1Password and never in this repository. Validation happens on three levels: locally with Helm, in the PR pipeline (`validate-charts.yml`: strict lint, package, version-bump check), and in the PR install-test stage (k3d, fixtures under `ci/` — see README):
A collection of self-made Helm charts, one directory per chart, published to a private, classic (index-based) Helm registry via GitHub Actions — an HTTP `POST` of the packaged `.tgz`, not an OCI `helm push`; the target URL and its credentials live in 1Password and never in this repository. Validation happens on three levels: locally with Helm, in the PR pipeline (`validate-charts.yml`: strict lint, package, version-bump check), and in the PR install-test stage (k3d, fixtures under `ci/` — see README):
```bash
helm lint --strict <chart-name>
@@ -85,7 +37,7 @@ helm template <chart-name> --set ingress.domain=example.com --set ingress.cluste
Copy `template-chart/` and replace every `xxx` placeholder — in file names (`templates/xxx.deployment.yaml` → `templates/<app>.deployment.yaml`), in `Chart.yaml`, in the `values.yaml` keys and their `.Values.xxx.*` references inside templates, and in `values.schema.json` (both the values paths and the `xxx.` helper prefixes). The schema and the null-safe fallback helpers come with the template but their content is chart-specific — adapt them along the six points of the README section "Values that must not lie", and validate the result against the chart's real bundle values, not only against its defaults. Then wire up CI:
1. Add `.github/workflows/publish-<chart-name>.yml` (copy an existing one, e.g. `publish-outline.yml`) — it triggers on pushes to `main` touching `<chart-name>/**` and calls the reusable `publish-helm-chart.yml`, which lives in `workflow-util/pipelines` and is bound to a tag. Copying an existing hull gets the `uses:` path, `registry-owner: ${{ github.repository_owner }}` and `secrets: inherit` right; writing one from memory does not.
1. Add `.github/workflows/publish-<chart-name>.yml` (copy an existing one, e.g. `publish-outline.yml`) — it triggers on pushes to `main` touching `<chart-name>/**` and calls the reusable `publish-helm-chart.yml`.
2. Add the chart name to the `chart-name` choice list in `.github/workflows/change-app-version.yml`.
3. Add install-test fixtures under `ci/<chart-name>/` (`test-values.yaml`, `fixtures.yaml`) or an entry with justification in `ci/excluded-charts.txt` — the PR install-test fails if both are missing.
4. Add a `customManagers` block for the chart's main image in `renovate.json5` (one per chart, since the image repository lives in the template and the version in `Chart.yaml`) — without it Renovate never updates that chart's app version. See README section "Dependency updates (Renovate)".
@@ -117,4 +69,4 @@ The binding style guide for all current and future charts lives in **README.md (
- Resources: requests as values defaults; limits auto-computed by the `_helpers.tpl` helper (`limitFactor` default 3; no CPU limits unless set explicitly).
- Replicas per workload (explicit `0` allowed) plus a chart-global `scaleDown` flag that overrides them.
- `icon:` mandatory in every `Chart.yaml`; `helm lint --strict` without findings; CI fixtures under `ci/<chart>/` or a justified exclusion.
- **Chart dependencies are admissible but expensive** (the former "no dependencies" rule fell with #214; `ytdlp-web-player` was the first to use one, `stalwart` the second, `paperless-ngx` the third). A dependency has to be optional and off by default via `condition:`, and upstream's chart is consumed as it comes — where its defaults fall short of these conventions, that is **documented in our `values.yaml`, never patched**. The price to know before adding the next one: a subchart is invisible to everything downstream that watches Helm release versions, so a new upstream version — a security fix included — reaches a cluster only when the outer chart is republished (Renovate's built-in `helmv3` manager plus `ci/renovate-bump-chart.sh` is the whole safety net, and it belongs in a comment at the pinned version); and every helm command exits 1 while `charts/` is unfilled, which is why `ci/helm-dependencies.sh` runs before lint, package, render and install — including the render test that covers **all** charts on **every** PR. Full rules and the rest of the cost: README, "Chart dependencies".
- **Chart dependencies are admissible but expensive** (the former "no dependencies" rule fell with #214; `ytdlp-web-player` is the only chart using one). A dependency has to be optional and off by default via `condition:`, and upstream's chart is consumed as it comes — where its defaults fall short of these conventions, that is **documented in our `values.yaml`, never patched**. The price to know before adding the next one: a subchart is invisible to everything downstream that watches Helm release versions, so a new upstream version — a security fix included — reaches a cluster only when the outer chart is republished (Renovate's built-in `helmv3` manager plus `ci/renovate-bump-chart.sh` is the whole safety net, and it belongs in a comment at the pinned version); and every helm command exits 1 while `charts/` is unfilled, which is why `ci/helm-dependencies.sh` runs before lint, package, render and install — including the render test that covers **all** charts on **every** PR. Full rules and the rest of the cost: README, "Chart dependencies".
+28 -212
View File
@@ -1,48 +1,34 @@
# helm-charts
Collection of some self-made helm-charts. Each top-level directory is a standalone Helm chart that gets packaged and uploaded to a private, classic (index-based) Helm registry by CI.
> **This repo lives on the Gitea instance JLab-Git.** Issues, pull requests, CI and Renovate all run there, and `main` there is the leading copy. github.com is a push mirror with Actions and Renovate **switched off**. The workflow files under `.github/workflows/` are read by Gitea unchanged; "Where this repo lives" below names the places where a Gitea runner behaves differently from a GitHub one, each of them measured rather than assumed.
>
> **The knowledge base is the [wiki](https://git.jk-dev.de/jlab-cluster/helm-charts/wiki).** It holds one page per chart — prerequisites, an installation walkthrough, the complete list of configurable values and the chart's operating traps — plus the long form of the workflow, the pipelines, the style guide and the [lessons](https://git.jk-dev.de/jlab-cluster/helm-charts/wiki/Lessons-Learned) each rule was bought with. This README stays the overview for someone reading the code; anything an *operator* needs is in the wiki. Keeping it current is a required step of the workflow (`CLAUDE.md`), not an afterthought.
>
> **A reading note on `#n`.** Issue numbers in this file point at the **GitHub** tracker. Most of them name *closed* issues, which were not migrated and exist only there. Gitea resolves a bare `#n` against its own counter instead — where, for example, `#22` is an unrelated throwaway pull request rather than the install-test issue this file means. The seventeen issues that were still open at the move are Gitea #5–#21 and #24; the mapping is a comment on Gitea #14.
Collection of some self-made helm-charts. Each top-level directory is a standalone Helm chart that gets packaged and uploaded to a private, classic (index-based) Helm registry via GitHub Actions.
## Charts
Every chart has a wiki page of its own — `https://git.jk-dev.de/jlab-cluster/helm-charts/wiki/<chart>` — carrying the installation walkthrough, the required 1Password fields and the full values reference. The table below is the one-line version.
| Chart | Deploys | Notes |
|---|---|---|
| `apache-tika` | Apache Tika | Deployment + Service only (no ingress) |
| `bentopdf` | BentoPDF | Stateless Deployment + `config.json` ConfigMap, `subPath`-mounted into the web root and always rendered (empty renders as `{}`) |
| `bulwark` | Bulwark Webmail (JMAP client for Stalwart) | **The companion to `stalwart`, and a pure client**: no mail, no database, no cache, no broker — the messages belong to the Stalwart release named by `bulwark.jmapServerUrl`. StatefulSet with **one** PVC at `/app/data`, carrying all four of Bulwark's directories (admin config, admin state with the audit log, encrypted user settings, telemetry) as subdirectories — one volume because snapshots are atomic per volume and never across them (the `garage` lesson, #81). No ConfigMap, so no `checksum/config` (§9 has nothing to attach to); the `op://` paths are covered by `checksum/secret-ref`. **The defining property, and it governs every value: the VOLUME BEATS THE CHART.** `configManager.get` resolves `admin override > env var > default`, the admin dashboard may write *any* key, and nothing says it just disconnected the chart — so a `helm upgrade` with a changed value can succeed and change nothing. Upstream documents this only for the server URL; it is true of everything. Exactly three values escape it (`SESSION_SECRET(_FILE)`, `ADMIN_PASSWORD`, `BULWARK_TELEMETRY`), and `bulwark.adminConfigReadOnly` closes it at a price that is **threefold and measured**: it freezes the settings policy (and with it the only way to move Web Push off the vendor relay), it freezes an override that *already* exists rather than removing it, and on a **fresh** install it stops `admin.json` from being written at all — so the admin dashboard never comes up and answers HTTP 500 while the pod stays Ready. Hence the default `false`. Sign-in is **OIDC-only** (`oauth.only`), and the way back out of a lockout is **not** `helm upgrade` with it `false` — `oauthOnly` is read through the same precedence chain, so the repair is removing the override in the **admin dashboard**, which is why `ADMIN_PASSWORD` is set at all (it generates no random password and logs nothing — the `#65` analogy does not hold here). Its **redirect URI carries the UI language** (`/<locale>/auth/callback`, 27 locales, and the language is not in the visible URL), so a single registered URI is the likeliest cause of a failed first sign-in. Both `SESSION_SECRET_FILE` and `OAUTH_CLIENT_SECRET_FILE` exist and are used — the better path `garage` and `keycloak` did not have — mounted **without `subPath`** so that a rotation takes effect without a restart, since both are re-read on every use. Two backup statements, both sharp: without the session secret the synchronized settings are unreadable **out of an intact volume**, and if the dashboard's OAuth setup was ever used the client secret sits in `config.json` on that volume in **cleartext**. All three probes are a plain GET on `/api/health` — **not** HEAD, which a `HTTPGetAction` cannot send — and the limit is written at the value: it carries process, HTTP acceptance, a free event loop and Next.js routing, and **nothing** about Stalwart, so with the mail server gone the pod is green. Served only at a host **root** (`NEXT_PUBLIC_BASE_PATH` is compiled in), one replica (single RWO PVC), two emptyDirs for `readOnlyRootFilesystem` (`/tmp`, `/app/.next/cache` — image optimization is active because `images.unoptimized` is in the lite branch only) |
| `calibre-web-automated` | Calibre-Web Automated | StatefulSet, extra consume-PVC |
| `cyberchef` | CyberChef | Stateless Deployment |
| `freesailarr` | gluetun VPN media stack (qbittorrent, prowlarr, flaresolverr, radarr, recyclarr, sonarr, seerr, bazarr, and mediathekarr off by default) | Single-pod StatefulSet: all apps share gluetun's network namespace; config volumes via volumeClaimTemplates, media PVCs as existingClaims (optional per-category `subPath`, so one claim can serve all three categories). One ClusterIP Service per app (mediathekarr's carries both of its ports); no global `ingress.domain`. Ingress in two shapes that mix: one host per app (default, each app's host required), or one shared `ingress.host` with the apps as sub-paths (documented ingress exception below) |
| `garage` | Garage (S3-compatible object store) | **One** node, `replication_factor = 1` — a deliberate trade with its price named (Gitea #81): the volume layer underneath already replicates and encrypts, so Garage replication on top would be nine times the raw data across two layers that know nothing about each other. What that costs is that the factor applies to the **metadata** too, where a corrupted database is upstream-documented as "generally not recoverable" and a replicated volume replicates the corruption. Hence three settings that are mandatory rather than optional: `metadata_fsync = true` (**off** in Garage's defaults), the automatic metadata snapshots, and those snapshots on a **second PVC** — a file-system snapshot of a live metadata database is not a restore point, so the volume backup covers the blocks and not the index. The first PVC carries `metadata_dir` **and** `data_dir` as two subdirectories, because volume snapshots are atomic per volume: that is what keeps index and blocks consistent with each other. Two further properties shape the whole chart: the image is **`FROM scratch`** and contains exactly one file, `/garage` — no shell, so **exec probes are impossible here** and an init container would need a second image — and the RPC secret has to arrive as an **environment variable**, because `rpc_secret_file` rejects any file with a group bit set while the kubelet sets one (the reasoning sits at the value). Started with `garage server --single-node`, which creates the cluster layout in-process before any listener opens and is idempotent on the layout version, and with `--default-access-key`/`--default-bucket`, which **import** key and bucket from the 1Password item — so the vault is the source of truth instead of a copy of what the server generated. Readiness is the admin API's `/health`, liveness and startup deliberately are **not**: `/health` answers 503 until a layout exists, so a node waiting for one would be restarted forever |
| `freesailarr` | gluetun VPN media stack (qbittorrent, prowlarr, flaresolverr, radarr, recyclarr, sonarr, seerr, bazarr) | Single-pod StatefulSet: all apps share gluetun's network namespace; config volumes via volumeClaimTemplates, media PVCs as existingClaims (optional per-category `subPath`, so one claim can serve all three categories). One ClusterIP Service per app; no global `ingress.domain`. Ingress in two shapes that mix: one host per app (default, each app's host required), or one shared `ingress.host` with the apps as sub-paths (documented ingress exception below) |
| `gotenberg` | Gotenberg | Deployment + Service only (no ingress) |
| `homeassistant` | Home Assistant | StatefulSet; the only chart carrying Helm tests (`templates/tests/`) |
| `keycloak` | Keycloak (OpenID Connect / SAML identity provider) | **Two** replicas with `cache=ispn`, clustered through Keycloak's default `jdbc-ping` stack — discovery runs over the PostgreSQL that is there anyway, so no headless Service and no JGroups wiring. Deployment, **no PVC** (Keycloak 26 keeps everything in the database, user sessions included). **The chart has two init containers and needs both** (Gitea #83): `db`, `health-enabled` and `metrics-enabled` are Keycloak *build* options, the published image is built with their defaults, and a runtime change makes Keycloak rewrite `/opt/keycloak/lib/quarkus` at start — which `readOnlyRootFilesystem: true` forbids and which upstream describes as a start that "creates an image that is mutable". An empty `emptyDir` over that path does not work either, because `QuarkusEntryPoint` loads `quarkus/quarkus-application.dat` before it dispatches any command, `build` included. So one init container **seeds** the directory into a volume, the next runs `kc.sh build` into it, and the serving container runs `start --optimized` — every container keeping the full hardening baseline. Build options therefore exist in exactly **one** place and a build option in `keycloak.env` fails the render. First chart with a `PodDisruptionBudget` (§11) and first `quay.io` image (hence the entry in `ci/trivy/data/ksv0125.yaml`). Two Services (8080, and the management port 9000 **never** published — it serves `/health/*` and optionally `/metrics` unauthenticated) and two Ingresses. **The admin hostname is not an access control**: `hostname-admin` only decides which URLs the server generates, and upstream states that it "does not prevent accessing the Administration REST API endpoints via the frontend URL" — restricting that API is reverse-proxy work, and the chart renders no `/admin` deny rule. `ingress.domain` is read as `KC_HOSTNAME` (as a full URL, which upstream requires alongside `hostname-admin`), so it stays required with the Ingress off. Liveness is `/health/live` and deliberately not `/health` or `/health/ready` — those include the database, and a liveness probe on them restarts the whole cluster for as long as the database is away. **`keycloak.metrics.enabled` defaults to `true` for the readiness probe, not for the metrics**: upstream's table of health checks carries a "Requires Metrics" column and the *database* check is the row that says yes, so with it off `/health/ready` answers 200 while the database is gone — the `urlaubsverwaltung` shape (#218) with the dependency hidden behind an option whose name says observability. Two traps at the values: upstream's `db-pool-max-size` default of 100 would ask a default PostgreSQL for 200 connections across two replicas (the chart ships 20), and two releases sharing one database **merge into one Infinispan cluster** unless the cluster names differ |
| `matrix-synapse` | Matrix Synapse | StatefulSet plus an nginx delegation Deployment/Service |
| `minecraft-router` | itzg/mc-router | Deployment + routing ConfigMap + NodePort Service (TCP only, no ingress); `mappings` is a required value |
| `minecraft-server` | itzg/minecraft-server | StatefulSet + PVC + Service + init-script ConfigMap (no ingress); service type is a value (`ClusterIP` by default, routed via `minecraft-router`). **Only the newest Java line is maintained** (chart 5.0.0, #158): the directory carries one line, older ones stay published as frozen artifacts (java8/java17 at `4.0.0`, java21 at `4.1.0`) and receive no further chart changes — publishing once per line out of one directory is what made the lines drift. To run an older JVM with the current chart, set `image.tagOverride`; the chart then checks that `minecraft.version` fits the Java line of the **effective** tag and fails the render with both values named. Two cases pass unchecked on purpose: a tag without a `java<N>` part and a `minecraft.version` that is not a plain numeric `x[.y[.z]]`. `image.tagOverride` is deliberately outside Renovate's reach — `renovate.json5` tracks `appVersion` only. Since chart 2.0.0 the tag is an **immutable dated upstream release** (`appVersion: <release>-java8` / `-java17` / `-java21` / `-java25`, e.g. `2026.8.2-java8`) instead of the continuously rebuilt rolling line tag. Pilot for the strict `values.schema.json` (#68), which is now binding for every chart |
| `outline` | Outline | StatefulSet, separate DB/Redis secrets |
| `paperless-ngx` | Paperless-ngx | Largest chart: consume CronJob, SMB-connector PV/PVC, pre/post-consumption scripts ConfigMap. Optional `valkey` subchart — **this repo's own chart, from the published index**, off by default and gated on `valkeySubchart.enabled` (not `valkey.enabled`, which the subchart's strict schema rejects). Configured as a **queue** (PVC + `noeviction`), unlike `stalwart`'s cache: `PAPERLESS_REDIS` is the Celery broker, and `allkeys-lru` evicts across the whole keyspace, where a backlog of waiting consume tasks is exactly what LRU reads as cold. The password comes from the **same 1Password item** as the Redis connection, in a third field `auth-conf` holding the complete line `requirepass <password>` — so the password sits in that item **twice** and a rotation has to touch both fields. Two more traps: with the subchart on, `redis-connection-url` in the vault **must** be repointed at `<release>-valkey-service:6379` or the bundled Valkey runs unused, and `valkeySubchart:` must not be erased — a null override deletes the key, Helm reads an unresolvable `condition:` as *enabled*, so erasing the switch turns the subchart **on** (rejected by `values.schema.json`). Never change `valkey/` and this chart in the same PR, see "Chart dependencies" |
| `patchmon` | PatchMon | One Deployment (the 2.x server serves frontend and `/api/*` from one process) plus an optional guacd sidecar; OIDC-only sign-in |
| `paperless-ngx` | Paperless-ngx | Largest chart: consume CronJob, SMB-connector PV/PVC, pre/post-consumption scripts ConfigMap |
| `patchmon` | PatchMon | Two Deployments (frontend + backend), OIDC secret |
| `rallly` | Rallly | Deployment, DB/OIDC/SMTP secrets |
| `rss-bridge` | RSS-Bridge | Deployment + whitelist ConfigMap; `ingress.hosts` is a required list of host + optional `tlsSecretName`, so several hosts can keep their own existing certificates (documented deviation from the one-TLS-secret rule, see its `values.yaml`) |
| `samba` | Samba | StatefulSet, users secret + config ConfigMap |
| `satisfactory` | Satisfactory game server | StatefulSet, no ingress |
| `searxng` | SearXNG | Stateless Deployment + settings ConfigMap (`use_default_settings: true` plus a free-form `settings.extra` passthrough); the signing key is a required `OnePasswordItem`, an external Valkey an optional second one. `ingress.domain` is read twice — Ingress host and `SEARXNG_BASE_URL` — so it stays required with the Ingress switched off |
| `stalwart` | Stalwart mail server (SMTP, IMAP, POP3, JMAP, ManageSieve) | A webmail client for it is the separate `bulwark` chart (above), pointed at this release's JMAP endpoint — the two are only useful together, and nothing in either reports the other being away. **The chart configures almost nothing, and that is the chart's defining property.** Since v0.16 Stalwart keeps its whole configuration — every listener, the blob store, the in-memory store, TLS objects, domains and the cluster roles — inside the data store and manages it over JMAP; the only file on disk is a `config.json` that names the data store and nothing else. So the chart renders that ConfigMap (secret-free: `authSecret` uses the `EnvironmentVariable` variant, so the password arrives by `secretKeyRef` and `checksum/config` stays meaningful), one StatefulSet per entry of the `nodes` **map** with `STALWART_ROLE` and `STALWART_HOSTNAME`, one NodePort Service **per port** (not per group — a NodePort is cluster-unique, so `full` and `mx` can only take SMTP behind the same Service, selected by a per-port pod label), a per-group ClusterIP `http` Service for the Ingress, per-group headless Services and NetworkPolicies, and a cert-manager `Certificate` for the mail ports. Everything else is operator work in the WebUI, which is why `NOTES.txt` carries an eight-step first install. Four traps worth knowing before reading the chart: **the role lookup fails open** — a node whose `ClusterRole` is missing from the database enables every listener *and* every task while staying Running with a green probe, so `mx` ships at `replicas: 0` and the NetworkPolicy is a ceiling independent of what was bound; **`NET_BIND_SERVICE` is a documented deviation needed to `exec`, not to bind** (the image sets `setcap cap_net_bind_service=+ep`, so `drop: [ALL]` alone makes the kernel fail `execve` — removing it as "unused" stops the chart starting); **`externalTrafficPolicy` must be `Cluster`** (no pod runs on a control-plane node while the HA proxy delivers to them, so `Local` would drop every packet) and the source address it costs is restored only by the HA proxy's PROXY v2 header; and **`/healthz` is rate-limited to 100 req/60s per source IP**, which is a node IP for kubelet probes, so the probe intervals are calculated rather than chosen. A first install needs `bootstrap.enabled: true` because the setup wizard appears only when `config.json` is *absent*. Optional `valkey` subchart — **this repo's own chart, from the published index**, off by default and gated on `valkeySubchart.enabled` (not `valkey.enabled`, which the subchart's strict schema rejects); never change `valkey/` and this chart in the same PR, see "Chart dependencies" |
| `sure` | Sure (we-promise, the maintained Maybe-finance fork) | One StatefulSet with **two containers**: the Puma web process and a Sidekiq worker, sharing one ReadWriteOnce PVC at `/rails/storage` — that shared volume is the reason they live in one pod. Only the web container migrates (the image entrypoint runs `db:prepare` solely for the `./bin/rails server` command, so the worker is started through `args:` and skips it). Three 1Password items (app/database/redis); Postgres and Redis stay external. Readiness deliberately probes `/` rather than Rails' `/up`, which is a boot check and touches no database; the worker's three probes ask `sidekiqmon` for this pod's own heartbeat. `ingress.domain` is read for `APP_DOMAIN` and the WebAuthn relying party, so it stays required with the Ingress switched off |
| `teamspeak-server` | TeamSpeak server | StatefulSet, no ingress |
| `template-chart` | — | Scaffold for new charts (placeholder `xxx`), never published |
| `urlaubsverwaltung` | Urlaubsverwaltung | Deployment, DB + OIDC secrets |
| `valkey` | Valkey | One instance per application (cache or queue backend), replacing the shared `cluster-redis` whose single ACL file coupled six applications together. StatefulSet with one replica + ClusterIP Service, no cluster mode, no Ingress and no value to add one. The password arrives as a **finished configuration line** (`requirepass <secret>`) in the 1Password field `auth-conf`, is mounted as a file and `include`d by the config — never in argv, never in the environment; an init container refuses to start the pod unless that line is really there, because the alternative failure is an open server that looks healthy. `maxmemory` is **derived from the computed memory limit** and a configured value above it aborts the render. Persistence and eviction are a pair, not two switches (queue = PVC + `noeviction`, the default; cache = emptyDir + `allkeys-lru`). First chart carrying a **NetworkPolicy** (on by default, Gitea #5), and the first whose probes had to be proven capable of failing: `valkey-cli` exits 0 on an error reply unless `-e` is given |
| `voucher-vault` | VoucherVault | StatefulSet, DB + OIDC secrets |
| `ytdlp-web-player` | ytdlp_web_player | Stateless Deployment, video cache and tool cache as emptyDirs (no PVC). **The first chart with a dependency** (`stalwart` and `paperless-ngx` followed): an optional `oauth2-proxy` subchart, off by default, wired in front of the app because the application ships no authentication at all — see "Chart dependencies" below |
| `ytdlp-web-player` | ytdlp_web_player | Stateless Deployment, video cache and tool cache as emptyDirs (no PVC). **The only chart with a dependency**: an optional `oauth2-proxy` subchart, off by default, wired in front of the app because the application ships no authentication at all — see "Chart dependencies" below |
| `zipline` | Zipline | StatefulSet, DB secret |
## Versioning
@@ -52,77 +38,18 @@ Every chart has a wiki page of its own — `https://git.jk-dev.de/jlab-cluster/h
- Bump the **chart version** in every change that touches a chart directory — a push to `main` publishes the chart.
- **App-version updates go through PRs** (so the validate and install-test pipelines run). The manually dispatched `Change App-Version` workflow (rewrites `version:`/`appVersion:` and commits directly to `main`) exists but bypasses both pipelines. For app **major** jumps, update stepwise: first to the last release of the old major line (own PR, merge, publish), then to the newest major.
## CI/CD
## CI/CD (GitHub Actions)
- `publish-<chart>.yml` — one thin workflow per chart; triggers on pushes to `main` touching `<chart>/**` (or manually) and calls the reusable workflow below. **A push to a chart directory on `main` publishes that chart**, so the chart version must be bumped in the same change.
- `publish-helm-chart.yml` — **not in this repository any more.** The reusable workflow lives in [`workflow-util/pipelines`](https://git.jk-dev.de/workflow-util/pipelines), the shared building-block repo, and the callers here bind it to a tag: `uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3` (Gitea #66). It still does the same thing — `helm package`, then a single HTTP `POST` of the packaged `.tgz` to the private chart registry, which is a **classic, index-based** Helm registry and not an OCI one, so `helm push` is not used, there is no `helm registry login` session, and every request carries its own Basic auth. What changed is who supplies what:
- **`registry-owner: ${{ github.repository_owner }}`** in each caller's `with:`. The owner used to be derived inside the workflow (#63); it is now a required input with no default, so the expression is evaluated **here**, in the calling repository, where the value is measured. That is `pipelines`' house rule — a shared block never guesses an owner segment — and it is why every caller carries that one extra line.
- **`secrets: inherit`**, unchanged, for `OP_CONNECT_TOKEN`.
- the repository **variable `OP_CONNECT_HOST`**, which replaced the Connect host that used to be hardcoded in our copy. A shared block must not carry one tenant's infrastructure address; the block's first step fails and names the variable when it is unset.
- `publish-helm-chart.yml` — reusable workflow: `helm package`, then a single HTTP `POST` of the packaged `.tgz` to the private chart registry. The target is a **classic, index-based** Helm registry (Gitea's `/api/charts` endpoint), not an OCI one: `helm push` speaks only OCI and is not used, there is no `helm registry login` session, and every request carries its own Basic auth. Credentials **and the target URL** come from 1Password Connect (`OP_CONNECT_TOKEN` secret, item `op://GitHub-Actions/jlab-registry-access/` with `username`, `password`, `charts-url`); the workflow appends `/api/charts` to `charts-url`. No registry address ever appears in this repository. The loaded values are read via **`steps.<id>.outputs.*`**, mapped into the step's own `env:` block and used as quoted shell variables — never interpolated into the command line. `load-secrets-action` **v5 no longer exports them as environment variables** for later steps the way v4 did, so after the v5 bump every `${{ env.* }}` reference resolved to an empty string while the loading step still logged `Populating variable` and stayed green. Each value is additionally guarded with `:?`, because an empty one used to collapse into a missing argument rather than an error. Three properties of the upload step are load-bearing:
- **Credentials stay out of the process list.** `curl --user "$U:$P"` would put them in `argv`; the step uses `curl --config -` and pipes the options in on stdin instead.
- **A failed upload fails the job.** Unlike `helm push`, `curl` exits **0** on HTTP 401/403/409 — a publish would go green with nothing uploaded. `--fail-with-body` turns those into a non-zero exit and still prints the response body, so the log names the cause.
- **The file name is built, not globbed.** `<chart>-<version>.tgz` is assembled from the chart name and the version already read from `Chart.yaml`; `--upload-file` takes exactly one file, and a glob matching two silently turns the second into curl's URL argument. `helm package` keeps the `+` of a `1.4.0+up1.9.2` version verbatim in the file name, so the built name is an exact match.
The registry credentials and the target URL still come from 1Password Connect (item `op://GitHub-Actions/jlab-registry-access/` with `username`, `password`, `charts-url`). `charts-url` holds only the **base up to `/api/packages`** and the block composes `<charts-url>/<owner>/helm/api/charts`, so **the publish target and its credentials appear nowhere in this repository.** `ci/helm-dependencies.sh` is still run before `helm package`, and it is still **our** script: the block's checkout clones the calling repository, so it runs from this tree.
**What this does *not* claim any more**: that no registry address is in the repo at all. That wording was true until a chart took one of our own published charts as a subchart — `stalwart`'s `Chart.yaml` names `https://git.jk-dev.de/api/packages/jlab-cluster/helm` as a dependency `repository:`, because a Helm dependency is resolved from a URL in the chart and there is nowhere else to put it. The narrower claim above is the one that matters and it is unchanged: that is the **read** path, the index is anonymously readable (measured), so it carries **no credential** and grants nothing a `git clone` of this repo does not already imply. The host itself has always been in `CLAUDE.md`; what is new is only the read-only path segment.
The reasons behind the upload step are **not repeated here**, because they are no longer ours to keep current: why both halves of the composed URL are guarded, why the credentials go in through `curl --config -` instead of `--user`, why `--fail-with-body` is what makes a failed upload fail the job, and why the file name is built rather than globbed, are all on the [`publish-helm-chart` wiki page](https://git.jk-dev.de/workflow-util/pipelines/wiki/publish-helm-chart) of that repository. Nothing was dropped in the move — the upload step's shell body went across unchanged except for two error messages that name the owner as an input.
Note: the registry **accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact, so a republish of an existing version is still silent — see #160. That is also what makes a manual `workflow_dispatch` of any `publish-<chart>.yml` a harmless way to prove the shared block works.
Note: the registry **accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact, so a republish of an existing version is still silent — see #160.
- `change-app-version.yml` — manually dispatched app-version bump; the chart must be listed in its `chart-name` choice list.
- `ci/helm-dependencies.sh <chart-dir>` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but three: `ytdlp-web-player`, `stalwart`, `paperless-ngx`). It runs in **six** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in the shared publish block (which is why that block names this script as a prerequisite of its callers — see above), and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh`, `ci/run-selector-match-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs.
- `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (the same changed-charts matrix, minus the charts listed in `ci/excluded-charts.txt`) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix, or run the all-charts install-test after the merge. The **resource-quantity render test** and the **selector-match render test** are the two exceptions to that rule: both run over every chart on every PR — see below.
- `install-test.yml` — reusable workflow (`workflow_call`) holding the install-test **job** itself: the runner-label expression, the pinned `kubectl` with its skew check, `envsubst`, the containerised-job detection, the per-leg k3d cluster and the `freesailarr` VPN credentials. Its one input is a JSON array of chart names; its callers decide *which* charts are tested and whether the job runs at all, never *how*. Two callers: `validate-charts.yml` and `install-test-all-charts.yml`. **It is not copied for the second caller** — a second copy is a second chance to test differently, unnoticed, the same argument that gives `ci/excluded-charts.txt` exactly one parser.
**It stays in this repository, and that is a decision, not an oversight** (Gitea #66, decision 4). It is reusable in form but not in substance: it is welded to the `ci/` fixtures, to the `ci/excluded-charts.txt` parser and to this repository's chart matrix. A building block that only one repository can use is not shared, it is merely moved somewhere else — and moving it would buy nothing while putting a tag between us and a file we change often. `publish-helm-chart.yml` was the opposite case: it takes a chart name and a registry owner and knows nothing about this repository, which is exactly why it could go. This note exists so the question is not asked again in half a year.
- `install-test-all-charts.yml` — manually dispatched **all-charts install-test**: the same job, with a matrix over every installable chart instead of the changed ones. Optional input for a subset of charts (empty = all). See below for when to run it.
- `sonar.yml` — **SonarQube analysis**, a hull like the publish ones: it triggers on pushes to `main` (no `paths` filter — the analysis looks at the whole tree) or manually, and calls `workflow-util/pipelines/.gitea/workflows/sonar-analysis.yml@v4` with `secrets: inherit`. The project-specific half is `sonar-project.properties` in the repo root. **It is not a gate** and it does not block anything — see below.
### Where this repo lives, and what a Gitea runner does differently
Everything below was measured on this instance, not taken from documentation, and it is the reason for every deviation in `validate-charts.yml`. How the move got here is not repeated — that is Gitea #14.
**Reusable workflows with `secrets: inherit` work**, so the 25 thin `publish-<chart>.yml` hulls run unchanged — including now that the workflow they call lives in another repository (Gitea #66). One quirk worth knowing before debugging the wrong thing: a *missing* secret does not fail the job, it fails job **creation**, so the run shows no steps at all.
**Runner labels mean different machines than on GitHub.** `ubuntu-latest` here has **no Docker socket**; `ubuntu-docker` has one. Exactly one job cares — the install-test, because k3d is a Docker client — and its `runs-on` is therefore an expression rather than a fixed label, so the same file still works on a GitHub runner where `ubuntu-docker` does not exist. Every other job (lint/package, the quantity render, the Trivy scan) needs no socket and stays on `ubuntu-latest`.
**A matrix runs concurrently inside ONE job, under ONE status context.** This is the single most consequential difference and it has two halves. Operationally: all legs share one Docker daemon, which is what makes a per-leg resource name mandatory (see the cluster name below). For branch protection: **a required check per chart is not possible**, because there is only one context for the whole matrix — and its name carries the *unexpanded* matrix expression, e.g. `Validate Helm Charts / Install-test ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }} (pull_request)`. Required-check names have to be read off a real run; they cannot be copied from GitHub. That example name is from before the install-test job moved into the reusable `install-test.yml` (Gitea #52) — the matrix expression now reads `${{ fromJSON(inputs.charts) }}` and the context may carry the calling job's name as well, so **read the current name off the next chart PR before using it anywhere**. What *is* required on `main` today is **`validate-charts`**, as the single context it reports under, alongside a **review**: per chart is the part that is impossible, not required checks as such. Both are what makes Renovate's `platformAutomerge: true` safe — see "PR policy".
**Expected runtimes**, so that a later deviation is visible as one:
| | measured here | GitHub, same commit |
|---|---|---|
| `Create k3d cluster` | 19–25 s | 22–23 s |
| `install-test` job, two charts in the matrix (cyberchef + freesailarr) | 210–216 s over three runs, spread 6 s | ~145 s wall, two parallel VMs |
| `install-test` job, freesailarr alone, ten containers and a real tunnel | 225 s (measured on the MediathekArr change, Gitea #20) | not comparable — Actions there were already off |
| `install-test` job, **20 charts** in the matrix — the accidental all-charts run of Gitea PR #48 | **12m41s**, one job, all legs concurrently on one Docker daemon; 19 of 20 charts Ready, the twentieth is Gitea #51 | never ran there |
The runner is **not slower** — cluster creation is a touch faster. The difference in total time is **structural, not performance**: two matrix legs share one runner here, while GitHub gives each leg its own VM. Reading it as "Gitea is slower" leads to the wrong conclusion.
**`aquasecurity/setup-trivy` cannot be used.** It runs `actions/checkout` against `aquasecurity/trivy` on github.com and hands it `github.token` — which here is a *Gitea* token. github.com answers 401 and checkout ends on `fatal: could not read Username for 'https://github.com'`, naming neither the action nor the token. Its `actions/cache` step is inert here as well. The workflow downloads the release tarball instead: one request, same behaviour on either forge.
**Tools are installed, not inherited — and the two cases differ on purpose.** `kubectl` is **pinned** to match `K3S_IMAGE` and guarded against version skew, because a client's distance to the API server changes what the test *means*, and a version that arrives with the runner image drifts with every image rebuild. `envsubst` is only **checked for and fetched if absent**, because it substitutes text and the release doing so does not change the result; pinning it would be upkeep without value. That GitHub's image happens to ship both does not make either unnecessary — it only makes the drift invisible there.
**Containerisation is detected, not inferred from the platform.** k3d publishes the API server port on the Docker *host*; when the job is itself a container — as every Gitea Actions job is — that host is the dind container, so the `https://0.0.0.0:<port>` k3d writes into the kubeconfig points at the job container, where nothing listens. The workflow asks the daemon about its own container instead of asking which forge it is on: a self-hosted **GitHub** runner in a container has the identical problem, and a condition that says "GitHub" while meaning "not containerised" is the kind of lie this repo exists to remove.
Still unmeasured, because they never stood in the way: `/dev/fuse` and user namespaces. Said as unmeasured rather than as fine.
#### What a GitHub runner had been providing quietly
Getting the install-test to run here surfaced six assumptions that had never been written down anywhere. Five of them say something about the **runner**, one about **our code**, and the split is the point.
The five environment ones are the harmless kind: they fail loudly, immediately, and are fixed on the node side (`cluster-config`), not here — Docker bridges built at MTU 1500 against a pod network of 1450 (small responses fit in one packet, a 17 MB tarball does not, which is why it looked like a blocked host); `fs.inotify` limits too low for k3s, so containerd never started and the node never registered; no `kubectl` and no `envsubst` in the image; and the kubeconfig pointing at `0.0.0.0`.
The sixth is different, and it is the one to remember.
> **The k3d cluster name was hard-coded as `install-test` — and that was wrong on GitHub too.** It simply could never show there: each matrix leg gets its own VM with its own Docker daemon, so two legs never saw each other's cluster. Here, where all legs share one daemon, it collides immediately — one leg logs `Re-using existing network`, the other dies on `The container name /k3d-install-test-server-0 is already in use`, and the first leg's rollback then deletes the shared network out from under the second. Two red legs, and neither message names the cause.
>
> For two years the roomier environment carried that bug instead of showing it. **A generous environment is no friend when you want to know whether something is correct.**
#### Two rules for every check added here
Both were learned by getting them wrong in this repo, and both apply to any new verification step, not just to clusters.
1. **A check that a broken system passes is worse than no check.** `kubectl get nodes` exits **0 on an empty list**, so a readiness check written that way went green against a cluster whose node had never registered — and the failure then surfaced five minutes later, somewhere unrelated. `kubectl wait --for=condition=Ready nodes --all` is not the shorter way to write it: with zero nodes the condition is vacuously satisfied. Assert the thing you mean (*at least one node, and Ready*), not that the command answered.
2. **Green does not mean checked.** A passing job earns exactly as much trust as the assertions inside it. The open case of this kind is Gitea #24: the images in `ci/<chart>/fixtures.yaml` are not followed by Renovate, so the install-test can keep passing against fixture infrastructure that is years stale. **Not solved** — named here so it is not mistaken for solved.
- `ci/helm-dependencies.sh <chart-dir>` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one). It runs in **five** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in `publish-helm-chart.yml`, and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs.
- `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (the same changed-charts matrix, minus the charts listed in `ci/excluded-charts.txt`) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix. The **resource-quantity render test** is the exception to that rule: it runs over every chart on every PR — see below.
### Resource-quantity render test (all charts, every PR)
@@ -142,31 +69,10 @@ Two details worth knowing before touching it:
- **The resources paths come from `values.schema.json`, not `values.yaml`** — it is JSON, so `python3` reads it without pyyaml (same runner constraint as `ci/run-trivy-config.sh`), and every chart-owned key must appear there anyway. A block counts when it is `#/definitions/resources` or carries `limitFactor`/`requests` itself, which is what keeps `minecraft-server`'s JVM-sizing `minecraft.resources` out. A chart whose templates render a resources block while its schema declares none **fails** the test with exactly that message: the schema is then out of sync with the chart.
- **A chart that never reaches the helper is reported, not counted as passing.** `gotenberg` sets every computable limit explicitly in its fallback defaults, and since a `null` never deletes a default there, no override can free them — nothing is multiplied, so nothing is proven. Those charts appear under "helper not reached" in the job summary rather than as a green tick for a code path that was never entered.
### Selector-match render test (all charts, every PR)
`ci/run-selector-match-test.sh [chart …]` (no argument = all charts) renders every chart with the same fixtures the install-test uses and checks that **every label selector the render contains really matches the labels of a pod template in that same render**. Two kinds of object are covered, and both of them fail *silently* when their selector matches nothing:
| Object | What a non-matching selector does | Who notices today |
|---|---|---|
| `topologySpreadConstraints` | the constraint is a condition over **zero** pods and is therefore trivially satisfied: nothing is distributed, no event, no log line | nobody — and with `whenUnsatisfiable: DoNotSchedule` the object reads like a guarantee |
| `PodDisruptionBudget` | the budget does its arithmetic over zero pods and never blocks an eviction, so `kubectl drain` takes every replica at once | nobody — `kubectl get pdb` shows an object that looks like protection |
Neither is a render error, neither fails the install-test, and neither shows up in the diff of the PR that causes it: the chart that breaks it is usually a chart that **renamed a label somewhere else**. Charts answer the first half of that by keeping the selector and the pod labels in one helper (`stalwart.groupLabels`, `keycloak`'s `app` label) — but one definition only helps while nobody writes a second one, and that is the half a test can hold. Introduced with Gitea #96, where the chart-owned spread constraint made the trivially-satisfied case reachable by design rather than by accident.
Three things to know before touching it:
- **Helm is the YAML parser, through the one-template chart `ci/yaml2json`.** The runners have neither `pyyaml` nor `yq` (measured; the same constraint `ci/run-trivy-config.sh` and `ci/run-quantity-render-test.sh` work under), and a YAML reader written in `awk` was the obvious alternative and the wrong one: its failure mode is a parser that quietly reads nothing, finds no selector, reports no mismatch and leaves the test green for a chart it never looked at. So the rendered manifest stream is handed to that chart as a **values file** and comes back as JSON, which `python3` reads from the standard library. The one textual step is the document split on a line that is exactly `---` in column one, and getting *that* wrong does not produce a false pass — the fragment stops being valid YAML and `helm` exits 1.
- **What is deliberately not checked, named so the gap is not mistaken for coverage.** `podAffinity`/`podAntiAffinity` are out of scope: their semantics are relative to pods that exist, so a term matching nothing is a legitimate configuration rather than a mistake. And a selector expressed **only** through `matchExpressions` is **reported as "not checked", never counted as passing** — evaluating set-based requirements means reimplementing label-selector semantics, which is a second parser by another name.
- **Two anti-vacuum guards, because this test can be green over nothing.** Both objects are rendered only above a replica threshold, and most charts' CI fixtures sit below it. `ci/selector-match/<chart>.yaml` is a render-only overlay applied **last**, whose only job is to push a chart over that threshold — and a chart that *has* one and still renders no selector **fails**, since that overlay existing is the whole claim that something was checked. On top of that the run fails if **no selector at all** was checked anywhere. Today `stalwart` is exercised through its overlay (two replicas in both node groups plus an `externalServices.valkey.host`, without which the chart's own "cluster with no in-memory store" abort fires) and `keycloak` through its ordinary install-test fixture, which already runs two replicas.
Measured: 30 charts, **5 selectors verified in ~14 s**, no cluster. The fail path was exercised rather than assumed — renaming `node-group` to `nodegroup` in a hand-written copy of the PDB selector produced two `FAIL` lines naming both the selector and the labels actually present, and a non-zero exit; the same was done for the spread constraint's selector.
### Install-test (PR pipeline stage 2)
For every chart a PR changes, `ci/run-install-test.sh <chart>` runs against a throwaway k3d cluster: it applies the stub `OnePasswordItem` CRD plus the chart's fixtures, `helm install`s the chart with its CI values, and waits for all Deployments/StatefulSets to roll out (per-chart timeout). Since probes and security hardening are active on all workloads, "Ready" means the app actually answers. On failure the job dumps pod descriptions, events and container logs.
The job itself is the reusable workflow **`install-test.yml`**; `validate-charts.yml` calls it with the changed charts, `install-test-all-charts.yml` with all of them. Everything about *how* a chart is installed is in that one file, deliberately — see "All-charts install-test" below.
Fixtures live at the **repo root** under `ci/` (deliberately *not* inside the chart directories, so they are neither packaged into the chart tgz nor trigger the publish workflows on merge):
- `ci/common/` — fixtures applied for every chart (schema-less stub CRD for `OnePasswordItem`)
@@ -176,55 +82,15 @@ Fixtures live at the **repo root** under `ci/` (deliberately *not* inside the ch
- `ci/<chart>/settings.env` — optional overrides: `INSTALL_TIMEOUT` for slow starters, and `REQUIRED_ENV="VAR_A VAR_B"` to declare the variables `fixtures.env.yaml` needs. **A missing or empty declared variable fails the install-test loudly, naming the variable — it never skips the chart.** Skipping would restore exactly the green-but-meaningless signal this mechanism exists to remove; the one legitimate case of absent secrets, a PR from a fork, belongs in an `if` on the workflow job rather than in the script
- `ci/excluded-charts.txt` — charts that cannot run in CI, one per line as `<chart> <justification>`. Currently: `satisfactory` (multi-GB image + 10GB+ Steam download), `template-chart` (scaffold with placeholder image). The justification is mandatory: a line carrying only a chart name is deliberately **not** an exclusion, because a bare name would drop a chart out of CI with nothing to review. **An excluded chart gets no install-test job at all** — see below
`freesailarr` is the only chart using `fixtures.env.yaml` today: gluetun needs a real Wireguard key, or it never completes a handshake and no app container behind its kill switch becomes Ready. The install-test job loads the key from 1Password Connect for that chart only, using the same pattern as the shared publish block — `steps.<id>.outputs.*` into the step's own `env:`, quoted, with `:?` guards that name the missing `op://` field before the cluster is even created. The key in that item is a **CI-only Mullvad key**, deliberately not the production one: Mullvad ties an address to a key and routes the tunnel to the endpoint it saw last, so a shared key would make every PR run pull the tunnel out from under the running stack — with the symptom in the cluster and the cause in a pull request.
`freesailarr` is the only chart using `fixtures.env.yaml` today: gluetun needs a real Wireguard key, or it never completes a handshake and no app container behind its kill switch becomes Ready. The install-test job loads the key from 1Password Connect for that chart only, using the same pattern as `publish-helm-chart.yml` — `steps.<id>.outputs.*` into the step's own `env:`, quoted, with `:?` guards that name the missing `op://` field before the cluster is even created. The key in that item is a **CI-only Mullvad key**, deliberately not the production one: Mullvad ties an address to a key and routes the tunnel to the endpoint it saw last, so a shared key would make every PR run pull the tunnel out from under the running stack — with the symptom in the cluster and the cause in a pull request.
**An excluded chart is dropped from the matrix, not skipped inside its job (#231).** It used to get a job that ran the script, printed `SKIP` and exited 0 — and GitHub knows only the exit code, so the check list showed `Install-test satisfactory ✓` for a chart whose image was never pulled, indistinguishable from one that really came up in the cluster. `detect-changed-charts` now removes the excluded charts before the matrix is built, so no job exists for them and nothing claims they passed. Because an absence on its own says nothing, that job writes every excluded chart of the PR **with its documented justification** into the job summary — a `Skipped` would only say "not run", and why is the question a review actually asks. The list is parsed in exactly one place: `ci/run-install-test.sh --list-excluded` emits `<chart><TAB><reason>` and the workflow calls it instead of re-implementing the format in YAML, since a second parser would be a second chance to select different charts — silently, and in either direction. The script keeps its own `SKIP` branch for local runs. What cannot be done instead: deciding per chart in an `if` on the install-test job, to get a visible `Skipped` entry rather than an absent one. GitHub rejects it with `context "matrix" is not allowed here` — `jobs.<id>.if` is evaluated **before** the matrix is expanded, so the `matrix` context does not exist there, and the whole workflow ends in failure with zero jobs created (measured on the closed PR #233).
Fixture images avoid Docker Hub where possible (`public.ecr.aws`/`ghcr.io`) to dodge runner rate limits. The job **blocks the PR** on failure — the stabilization phase of #22 ended when Renovate started automerging patch/minor updates, since a chart that no longer starts must not reach `main` and publish unattended. When adding a new chart, add its `ci/<chart>/` fixtures (or an entry in `ci/excluded-charts.txt`) — the install-test fails if both are missing. The script also runs locally against any existing cluster: `k3d cluster create t && ci/run-install-test.sh <chart> && k3d cluster delete t`.
### All-charts install-test (on demand)
`install-test-all-charts.yml` runs **the same job** as the PR pipeline — the same `install-test.yml`, the same pinned `kubectl`, the same fixtures — with a matrix over **every installable chart** instead of the charts a PR changed. `workflow_dispatch` only, **no schedule**: a daily run would spend compute on a question that changes rarely, and a red run nobody asked for is a red run nobody reads.
It answers a different question from the PR pipeline. The PR install-test guards against what a change breaks; this one asks whether the environment under an **unchanged** chart still holds. The install-test matrix follows the charts a PR touches, so a chart nobody has edited since the forge move carries a green history from a *different runner* — and that green says nothing about today. **Run it after a change to the runner, to `K3S_IMAGE`, to the pinned `KUBECTL_VERSION`, to the fixtures under `ci/` — and after a forge move.** It is also the answer for a PR that only changes the harness and therefore tests nothing by itself: run it once the change is on `main`.
Why it exists at all: the answer arrived once by accident. Gitea PR #48 corrected a *comment* in 21 charts, which made all of them "changed" and turned the run into the first all-charts install-test after the move — **19 of 20 matrix charts came up in 12m41s** in one job, and exactly one did not (`rss-bridge`, whose last green run was from a GitHub runner; Gitea #51). Worth having, and nobody had planned it.
- **Input `charts`** — space- or comma-separated chart names, empty means all. That is what lets a single chart be tested against the runner, which is what Gitea #51 needs. A name that is not a chart directory, or one listed in `ci/excluded-charts.txt`, is a **loud error**: someone typed it, and a green run over zero charts would be the worst possible answer. An empty matrix fails the run for the same reason.
- **One throwaway cluster per chart, no reuse.** Reuse would save time and change what the test proves — leftovers from the previous install, a warm image cache, fixtures already applied. "Came up from nothing" is the statement.
- **Exclusions come from the one parser**, `ci/run-install-test.sh --list-excluded`, and the excluded charts appear in the job summary **with their justification**, exactly as in the PR pipeline.
- **The result is written down.** Every matrix leg records its own outcome next to the four facts that decide what the outcome means — date, runner label, `K3S_IMAGE`, `kubectl` version — and a final job records the verdict over the whole matrix. A green run nobody notes is unknown again next time. Per leg and not as one table, because a matrix leg only ever sees its own job summary.
- **Expected runtime: 20 charts in 12m41s** (Gitea PR #48, one job, all legs concurrently on one Docker daemon). The figure is there so a later deviation is visible as one; compare it against the chart count the run prints in its own summary — 24 at the time of writing, and one more with every new chart. The job timeout is 30 minutes per leg and lives in `install-test.yml`, i.e. it is the same for both callers on purpose.
### SonarQube analysis (after the merge, and not a gate)
`sonar.yml` sends the whole tree through a SonarQube analysis. The workflow itself is the shared block [`sonar-analysis.yml`](https://git.jk-dev.de/workflow-util/pipelines/wiki/sonar-analysis) in `workflow-util/pipelines`, bound to `@v4`; the only thing on this side is `sonar-project.properties` in the repo root, holding the project key `JLab-Helm-Charts`. **That key is read off the existing project in SonarQube, not derived** — it is the one the maintainer set up, and a guessed key (`jlab-cluster_helm-charts`, reasoned from the instance serving two owners) is what made the first run fail. The file carries **no `sonar.projectName`** on purpose: once the project exists, the display name belongs to the server, and a name sent with every analysis would overwrite a rename there silently.
**What is analysed — measured, not taken from a language list.** The first run (run 375) loaded exactly **two quality profiles, `json` and `yaml`**; the sensors that did work were **IaC Kubernetes** (3.4 s) and the **secrets detection** (`TextAndSecretsSensor`) over all 391 indexed files.
| What is in this repository | Analyser | Runs? |
|---|---|---|
| 238 YAML files and 26 Helm template directories | IaC Kubernetes, YAML | yes |
| 26 `values.schema.json` | JSON | yes |
| the whole tree (391 files) | Secrets | yes |
| the 5 shell scripts under `ci/` — the whole CI logic | Shell | **no** |
| the 30 workflow files | GitHub Actions | **no** |
**The two missing rows are not a configuration mistake, and nothing in this repository can fix them: analysers come from the *server*, not from the scanner.** The language list the earlier version of this table was built from describes the newest SonarQube release; this instance does not have the Shell and GitHub Actions plugins. The consequence is worth stating plainly rather than leaving it to be rediscovered — **the files carrying the actual logic, the five `ci/` scripts and the thirty workflows, are not analysed at all.** What runs is IaC over the charts plus secret detection over everything. Whether the instance gets upgraded or gains those plugins is a decision for whoever runs it, not something this repository settles; `shellcheck` and review remain the only check on the scripts until then.
The secrets row is the valuable one that *does* run. **The premise of this repository is that no credential ever lands in it** — every secret is a 1Password `OnePasswordItem` reference — and until now nothing checked that premise. What it actually reports is still unknown: run 375 built the report (603 kB) but never uploaded it, because the upload is where the wrong project key failed.
**It is not a gate, and that is a decision.** The SonarQube Community Build analyses the **main branch only**; there is no pull-request analysis to hook into, so the trigger is a push to `main` and **a finding appears after the merge**. At that point there is nothing left to block, which is why the shared block carries no quality-gate step: a red gate behind the merge would only colour `main` red without having prevented anything, and a permanent red is a thing people learn to ignore — taking the credibility of `validate-charts` with it. A gate becomes the right thing under one of two conditions: pull-request analysis is available, or a deployment hangs off the workflow. Until then, read the findings in SonarQube; nothing in this repository turns red because of them.
**No exclusions, for the first round.** Including the Helm templates, which are not valid YAML because of `{{ }}`. What the Helm and YAML analysers make of them is only learnable by letting them run over them once, and excluding them up front would discard that answer along with the bulk of the repository. If the first run produces noise genuinely not worth reading, the exclusions go into `sonar-project.properties` with the finding they answer named next to them — not before.
**Prerequisites**, both on the instance and neither in a file here: the repository **variable `SONAR_HOST_URL`** (a variable and not a secret on purpose — the server address is not a credential, and as a variable it stays readable in the log, which is what makes a wrong one diagnosable) and the **secret `SONAR_TOKEN`**, which reaches the block through `secrets: inherit`. The block guards both and names the setting and its place when one is empty. A *missing* `SONAR_TOKEN` never gets that far: the block declares it `required`, and a missing required secret fails job **creation** on this instance, so the run would show no steps at all — the same quirk as everywhere else here.
**Three things about this path were unmeasured, and run 375 answered all three — positively.** `node24` **is** in the runner image, so the JavaScript scan action runs. The scanner download **gets through**: ~59 MiB of archive plus its signature in roughly three seconds, so the MTU failure that once killed a 17 MB download on this runner (see "What a GitHub runner had been providing quietly") does not repeat at five times the size. And a repository **variable does reach a workflow called from another repository** — the block's guard passed and the scanner started, which it could not have done without `vars.SONAR_HOST_URL` arriving. That last one also settles the identical question the publish route carried for `OP_CONNECT_HOST`: same forge version, same mechanism, a repository variable read inside a workflow called from another repository. What the publish route still has unproven is the upload itself, which has never run. `workflow_dispatch` stays on the hull for the retries: without it every attempt costs a dummy commit on `main`.
## Dependency updates (Renovate)
Renovate runs as the shared `jlab-renovate-bot` instance on the Gitea side, which also serves the other repositories there — see "Renovate on Gitea" below. **There is no Renovate workflow in this repo, and none should be added.** (The `jlab-renovate` GitHub App that used to do this is switched off along with the rest of GitHub Actions.) What updates is unaffected either way: the config is **`renovate.json5`** in the repo root (JSON5 for the inline reasoning; requires Renovate >= 41 for `managerFilePatterns`/`customType`). There must be exactly one config file — do not add a second `renovate.json` or `.github/renovate.json`.
Renovate runs as the self-hosted `jlab-renovate` GitHub App. Its config is **`renovate.json5`** in the repo root (JSON5 for the inline reasoning; requires Renovate >= 41 for `managerFilePatterns`/`customType`). There must be exactly one config file — do not add a second `renovate.json` or `.github/renovate.json`.
What Renovate keeps current:
@@ -232,15 +98,15 @@ What Renovate keeps current:
| --- | --- |
| Chart app versions (`appVersion:` in `Chart.yaml`) | One `customManagers` entry per chart, hard-coding the image repository |
| Sidecar/init images in `values.yaml` (nginx, guacd, busybox, alpine) | Built-in `helm-values` manager |
| **All** images of `freesailarr`, including its main one | Built-in `helm-values` manager — the chart pins every tag in `values.yaml` and has no app-version block. One exception: `mediathekarr` is pinned by **digest** (its tags carry no version) and has its own `customManagers` block |
| **All** images of `freesailarr`, including its main one | Built-in `helm-values` manager — the chart pins every tag in `values.yaml` and has no app-version block |
| GitHub Actions | Built-in `github-actions` manager |
| Tool versions pinned as workflow env vars (`TRIVY_VERSION`) | `# renovate: datasource=… depName=…` comment above the line |
Each chart renders its main image as `<repo>:{{ .Chart.AppVersion }}` — repository in the template, version in `Chart.yaml`. Renovate cannot join two files, so the mapping lives in `renovate.json5`, one block per chart. It is deliberately *not* a `# renovate:` comment inside each `Chart.yaml`: that would touch all chart directories and force a version bump plus a publish per chart for a pure comment change. Charts whose tags carry a flavour or date suffix (`apache-tika`, `minecraft-server`, `rss-bridge`, `searxng`, `teamspeak-server`) override `versioning` in their block; the reasoning sits next to each one.
One rule those overrides share, learned from `searxng` and worth knowing before writing the next one: **a capture group is a claim about how the value is compared.** Renovate's regex versioning parses `major`, `minor`, `patch`, `build` and `revision` with `Number.parseInt` (`modules/versioning/regex/index.js`, `_parse`), so a non-numeric commit hash captured as `build` becomes whatever digits lead it — `9fea41204` is `9`, `777ba8fa4` is `777`, and a hash starting with a letter is `NaN`. Measured on renovate 41.173.1 and 44.48.0, that makes Renovate offer an image pushed *earlier the same day* as an update to a newer one — a downgrade wearing the clothes of a routine date bump, which is exactly what a review waves through and automerge then carries to `main`. Parts of a tag that carry no ordering belong **outside** every group: the anchored pattern still requires them to be there, they simply take no part in the comparison. `prerelease` is not the way out either — it sorts correctly but marks every tag `isStable === false`, which hands the whole chart to `ignoreUnstable`. `freesailarr` has no **app-version** block on purpose — see the images exception above; a block would either duplicate the `helm-values` updates or rewrite an `appVersion` that selects no image. It does carry one `customManagers` block, for `mediathekarr`: that image is pinned by digest because its tags carry no version (`latest` is a 2024 build, `beta` moves), and the built-in `helm-values` manager reads only `repository`/`tag` and skips it (`skipReason: invalid-value`), so the two cannot collide. Digest rules need the same guard as versioning rules and for a sharper reason: while the pin is current, a matching rule and a broken one produce the identical empty result. Verified with two dry-run fixtures — the real digest yields no update, a deliberately stale one yields exactly one `digest` update back to the pinned digest.
One rule those overrides share, learned from `searxng` and worth knowing before writing the next one: **a capture group is a claim about how the value is compared.** Renovate's regex versioning parses `major`, `minor`, `patch`, `build` and `revision` with `Number.parseInt` (`modules/versioning/regex/index.js`, `_parse`), so a non-numeric commit hash captured as `build` becomes whatever digits lead it — `9fea41204` is `9`, `777ba8fa4` is `777`, and a hash starting with a letter is `NaN`. Measured on renovate 41.173.1 and 44.48.0, that makes Renovate offer an image pushed *earlier the same day* as an update to a newer one, unattended under the patch/minor automerge rule. Parts of a tag that carry no ordering belong **outside** every group: the anchored pattern still requires them to be there, they simply take no part in the comparison. `prerelease` is not the way out either — it sorts correctly but marks every tag `isStable === false`, which hands the whole chart to `ignoreUnstable`. `freesailarr` has no block on purpose — see the images exception above; a block would either duplicate the `helm-values` updates or rewrite an `appVersion` that selects no image.
A consequence of pinning every tag in `values.yaml`: `ci/renovate-bump-chart.sh` sees an unchanged `appVersion` and bumps the chart **patch**. That holds for all ten images alike, the digest-pinned one included — none of them is special — and it is the intended outcome, since `appVersion` here is the chart's own marker rather than a tracked upstream version.
A consequence of pinning every tag in `values.yaml`: `ci/renovate-bump-chart.sh` sees an unchanged `appVersion` and bumps the chart **patch**. That holds for all nine images alike — none of them is special — and it is the intended outcome, since `appVersion` here is the chart's own marker rather than a tracked upstream version.
Two dependencies are constrained beyond their versioning. **nginx** (the Synapse delegation sidecar) is limited to **even minors** via `allowedVersions`. nginx numbers its release branches by minor — even is stable, odd is mainline — and the chart belongs on stable. Patch updates inside the stable branch still flow through normally. The constraint tolerates a variant suffix (`1.30.4-alpine`) on purpose: docker versioning only ever offers candidates carrying the current tag's suffix, so a suffix-blind constraint would silently freeze nginx the day the sidecar moves to a slimmer base image — a switch to `1.30.4-alpine` that #84 has commissioned a check on — with no error and no warning.
@@ -266,34 +132,17 @@ Everything is derived from the base commit, so the script is idempotent across t
>
> Without that entry Renovate skips the task without an error and every chart PR stays red on the version check.
If the dependency dashboard reports `Failed to look up github-releases package …: no-result` for third-party repos, that is the instance's github.com credentials, not this config. The instance needs a read-only PAT as `GITHUB_COM_TOKEN` for github.com datasource lookups; its own platform token is a Gitea token and is worth nothing there.
### Renovate on Gitea
**Do not add a Renovate workflow to this repository.** It is the obvious thing to reach for — Gitea has no app model, so a scheduled job in the repo looks like the only way to run the bot — and it is wrong here: a **shared `jlab-renovate-bot` instance** runs outside this repository and has it enrolled. A second runner against the same repo produces duplicate PRs and two schedulers rewriting each other's branches, which is not a safety net but a fault. This paragraph exists so the next reader who finds no workflow knows why instead of building one.
The instance is outside this repository, which has two consequences worth writing down:
- **It reads our `renovate.json5` and the per-chart rules do fire.** Measured, not assumed: its first run here opened PRs for `searxng` and `gotenberg` plus the dependency dashboard. Those two are `customManagers` targets of ours — no built-in manager finds an `appVersion:` in a `Chart.yaml` — so the config is demonstrably in effect.
- **`allowedCommands` is already granted there, and the version-bump hook runs.** This was the expensive unknown: `postUpgradeTasks` only executes when the command is allow-listed in the *instance* config, and a missing entry is skipped **silently**, leaving every chart PR red on the version check. The proof is in that searxng PR's diff, which carries `version: 1.0.4+up2026.9.11.ffe96f8a6` → `1.0.5+up2026.9.13.96479fff6` alongside the `appVersion` change. Nothing but `ci/renovate-bump-chart.sh` writes `version:`. The allow-list is not visible from this repo, though, so if it is ever dropped the symptom here will be red chart PRs with no explanation.
One item is easy to miss and is named in "The version-bump hook" above: the instance needs a `GITHUB_COM_TOKEN` for github.com datasource lookups. Every `github-releases`/`github-tags` lookup and the whole built-in `github-actions` manager resolve against github.com, the platform token is a Gitea one and worth nothing there, and unauthenticated lookups hit the rate limit inside a single pass. The symptom is a `no-result` line on the dependency dashboard, not an error.
If the dependency dashboard reports `Failed to look up github-releases package …: no-result` for third-party repos, that is the instance's GitHub credentials, not this config: a GitHub App installation token is not a good fit for looking up repos the app is not installed on. Give the instance a separate read-only PAT as `GITHUB_COM_TOKEN` for github.com datasource lookups.
### PR policy
The config is aligned with the one in `cluster-config`, because the same `jlab-renovate-bot` instance serves both and two repos that behave differently under one bot is a source of surprises rather than of safety.
- **Automerge is global** (`automerge: true`, `automergeType: "pr"`, `platformAutomerge: true`) — every update type, **majors included**. The safeguard is not the absence of automerge, it is the branch protection: `main` requires a **review** and requires `validate-charts` to pass. `platformAutomerge` arms the PR on the forge, and Gitea merges it when *both* conditions are met, so an armed PR without a review simply sits there. This is the setting the older `platformAutomerge: false` comment in `renovate.json5` named as the better one "once required checks exist on `main`" — they now do. Same reasoning as `cluster-config`, which retired its dashboard-approval step for it.
- **`minimumReleaseAge: "3 days"`** for `patch`, `minor` and `digest` updates. Renovate does not even open the PR before then. A merge here is never just a merge — a chart publish follows it straight away — so a release that gets yanked hours after it appeared must not reach the registry. Majors carry no age gate: they are held at the review, not by a delay.
- **Majors** arrive one line at a time (`separateMultipleMajor`): a v1 → v3 jump produces a PR for the latest v2 first and then one for v3 — the stepwise procedure the versioning section requires. `separateMinorPatch: true` likewise splits minor and patch into separate PRs, so one red install-test does not block unrelated updates.
- **No PR caps** (`prConcurrentLimit: 100`, `prHourlyLimit: 0`), as in `cluster-config`. The known cost, written down rather than discovered: the first run after a quiet week can open many PRs at once, and every chart PR starts a k3d install-test — which on a Gitea runner queues inside one job rather than fanning out (see "Where this repo lives"). That queue is the accepted trade for not having updates trickle in four per hour.
- **`config:best-practices`** is in `extends`. Two of its members matter here: `helpers:pinGitHubActionDigests` moves the workflow files from `actions/checkout@v7` to SHA pins (wanted), and `docker:pinDigests` is switched off again for `custom.regex` and `helm-values` — an `appVersion:` is a tag by construction (`<repo>:{{ .Chart.AppVersion }}`, and the `+up` suffix is built from the same string), so a digest has nowhere to go. Sidecar digests in `values.yaml` *could* be pinned; that is a separate decision, since each digest change would be a chart patch plus a publish.
- **Automerge** for `patch`, `minor` and `digest` updates, after a `minimumReleaseAge` of 3 days and only once the PR pipeline is green — install-test included, which is why that job no longer runs with `continue-on-error`. Minor is in scope because the sidecar images barely move in patch steps (`alpine 3.22` → `3.24`, `nginx 1.30.4` → `1.31.4` are both minor), so a patch-only rule would practically never fire. Renovate merges itself rather than using GitHub auto-merge, which would merge immediately as long as `main` has no required status checks. Majors are merged by hand.
- **Majors** arrive one line at a time (`separateMultipleMajor`): a v1 → v3 jump produces a PR for the latest v2 first and then one for v3 — the stepwise procedure the versioning section requires.
- `prConcurrentLimit: 10` / `prHourlyLimit: 4` keep the k3d install-tests from all starting at once.
- The manually dispatched `change-app-version.yml` still exists for one-off bumps, but the normal path for app updates is now a Renovate PR.
## Creating a new chart
The wiki carries the same procedure as a worked checklist: [Creating a new chart](https://git.jk-dev.de/jlab-cluster/helm-charts/wiki/New-Chart). A new chart also needs a wiki page of its own before its PR is merged.
1. Copy `template-chart/` to `<chart-name>/` and replace every `xxx` placeholder: template file names (`xxx.deployment.yaml` → `<app>.deployment.yaml`), `Chart.yaml`, the `values.yaml` keys, the `.Values.xxx.*` references inside the templates, **and both the `xxx.` helper prefixes and the values paths in `values.schema.json`**. Then walk the six points in "Values that must not lie" below — the schema and the null-safe fallbacks are copied with the template, but their content is chart-specific and wrong until you adapt it. The same goes for `templates/NOTES.txt`: it comes with the template as a worked example and is placeholder text until you replace it with what an operator of *your* app needs after the install (see "Everything else" below); its leading `{{/* … */}}` block explains the bar and is meant to be deleted.
2. Add `.github/workflows/publish-<chart-name>.yml` (copy an existing one, e.g. `publish-outline.yml`).
3. Add the chart name to the choice list in `.github/workflows/change-app-version.yml`.
@@ -313,7 +162,7 @@ For runtime verification (do the workloads actually start, do probes/security se
## Chart style guidelines
These guidelines are **binding for every current and future chart** in this repo. They are the consolidated end state of the conventions overhaul (#32, batches 0–4); `template-chart/` is the reference implementation. The wiki mirrors them as the [chart style guide](https://git.jk-dev.de/jlab-cluster/helm-charts/wiki/Chart-Style-Guide), with the incident behind each rule collected in [Lessons learned](https://git.jk-dev.de/jlab-cluster/helm-charts/wiki/Lessons-Learned) — change a rule here and update that page in the same PR.
These guidelines are **binding for every current and future chart** in this repo. They are the consolidated end state of the conventions overhaul (#32, batches 0–4); `template-chart/` is the reference implementation.
### 1. Naming
@@ -334,7 +183,7 @@ These guidelines are **binding for every current and future chart** in this repo
### 3. Images
- The main app image tag comes from `{{ .Chart.AppVersion }}` — never hardcoded.
- **Documented exception — freesailarr**: the chart bundles ten independently versioned upstreams (nine of them enabled by default), so none of them is "the" app version. **No** tag comes from `appVersion`; every image is pinned as `<app>.image.repository` / `<app>.image.tag` in `values.yaml`, which is also what puts them under the built-in `helm-values` manager. Its `appVersion` is the chart's own version marker, freely chosen and tied to no upstream — raised by hand when a new edition of the chart is marked as a whole. Consequently the chart has **no app-version** block in `renovate.json5` and is **not** in the `change-app-version.yml` choice list. `mediathekarr` adds a second deviation on top: its tags carry no version at all, so it is pinned by `image.digest` as well and is the one image in this chart with a `customManagers` block — a `repository`/`tag` pair alone would leave it without any update path.
- **Documented exception — freesailarr**: the chart bundles nine independently versioned upstreams, so none of them is "the" app version. **No** tag comes from `appVersion`; all nine images are pinned as `<app>.image.repository` / `<app>.image.tag` in `values.yaml`, which is also what puts every one of them under the built-in `helm-values` manager. Its `appVersion` is the chart's own version marker, freely chosen and tied to no upstream — raised by hand when a new edition of the chart is marked as a whole. Consequently the chart has **no** block in `renovate.json5` and is **not** in the `change-app-version.yml` choice list.
- **Documented exception — minecraft-server**: the only chart whose main image tag can be overridden, through `image.tagOverride` (#158, empty by default — the tag still comes from `appVersion`). `appVersion` is the tag and therefore fixes the JVM, and since chart 5.0.0 only the **newest** Java line is maintained out of this one directory; the older lines stay published as frozen chart versions (java8 and java17 at 4.0.0, java21 at 4.1.0) and never receive another change. Without the override, running an older JVM would mean installing a frozen chart and giving up every later fix — so the line is a decision at rollout time rather than a property of the chart. The override is named `tagOverride`, not `tag`, because it decouples the running image from the chart version, and it is deliberately outside Renovate's reach (`renovate.json5` tracks the `appVersion` in `Chart.yaml` and nothing else). The chart checks the pairing itself: the Java line of the **effective** tag against `minecraft.version`, failing the render with both values named — a mismatch used to install silently and only break the container afterwards.
- Auxiliary images (sidecars, init containers, secondary workloads — e.g. the delegation nginx in matrix-synapse, guacd in patchmon, busybox init/consume images in paperless-ngx) are pinned via values instead of being hardcoded in templates, and their pinned versions are reviewed alongside every app-version update round.
@@ -346,7 +195,7 @@ These guidelines are **binding for every current and future chart** in this repo
- The TLS secret is named `tls-<release>-<chart>-ingress`.
- **Documented exception — homeassistant**: keeps its upstream-style flexible ingress values (host/tls lists, no required values); only its `ingress.className` default follows the standard (`nginx`, clearable to omit the field).
- **A shared domain with sub-paths is the second admissible ingress shape.** Where a chart exposes several apps, they may either each take a host of their own or share one host and be told apart by path (`domain.tld/sonarr`, `domain.tld/radarr`, …) — one DNS name and one certificate for the whole bundle. The rules for the shared shape: the domain is a single value (`ingress.host`), each app's path is its own value defaulting to `/<app>`, the paths render in one Ingress object `<release>-<chart>-ingress` with the standard TLS secret `tls-<release>-<chart>-ingress`, and an app that carries a host of its own always wins and keeps its own object — so both shapes mix without extra values. An app that needs a per-path proxy annotation (`rewrite-target` is per **object**, not per path) gets its own object on the same host, repeating the same TLS secret name and **omitting** the cert-manager annotation, so that exactly one certificate is requested. Apps that cannot serve a base path must be rendered at `/` or refused with a `fail` — never rendered somewhere they provably cannot work.
- **Documented exception — freesailarr**: the chart bundles several independently exposed apps in one pod, so it has **no** `ingress.domain`. It offers both shapes. By default each app carries its own `<app>.ingress.enabled` plus a required `<app>.ingress.host` (required only while that app's ingress is enabled and it is not on the shared domain) and renders its own Ingress object `<release>-<chart>-<app>-ingress` with its own TLS secret `tls-<release>-<chart>-<app>-ingress`. Setting the global `ingress.host` switches every app without a host of its own to the shared-domain shape under `<app>.ingress.path`; qBittorrent (no base path, needs a prefix-stripping rewrite) gets the extra object described above, and seerr as well as mediathekarr are pinned to `/` — mediathekarr because its UI links assets with root-absolute paths, which a rewrite cannot fix; with both of them on one shared host the duplicate-path guard fires and one of them needs a host of its own (its ingress is therefore off by default). `clusterIssuer`, `className` and `annotations` stay global defaults under `ingress.` and are overridable per app.
- **Documented exception — freesailarr**: the chart bundles several independently exposed apps in one pod, so it has **no** `ingress.domain`. It offers both shapes. By default each app carries its own `<app>.ingress.enabled` plus a required `<app>.ingress.host` (required only while that app's ingress is enabled and it is not on the shared domain) and renders its own Ingress object `<release>-<chart>-<app>-ingress` with its own TLS secret `tls-<release>-<chart>-<app>-ingress`. Setting the global `ingress.host` switches every app without a host of its own to the shared-domain shape under `<app>.ingress.path`; qBittorrent (no base path, needs a prefix-stripping rewrite) gets the extra object described above, and seerr is pinned to `/`. `clusterIssuer`, `className` and `annotations` stay global defaults under `ingress.` and are overridable per app.
### 5. Security hardening
@@ -406,11 +255,6 @@ These guidelines are **binding for every current and future chart** in this repo
- Every Deployment/StatefulSet has a `replicas` value (default `1`, the values path per workload matches its section) that also accepts an explicit `0` for per-workload fine control.
- Every chart has a global `scaleDown` flag (default `false`): `scaleDown: true` takes precedence over all `replicas` values, renders every Deployment/StatefulSet with `replicas: 0` and suspends CronJobs. It **stops** the app but does not uninstall it — PVCs, Secrets, Services and Ingress stay in place; setting it back to `false` brings the workloads back up.
- Implemented via a per-chart `<chart>.replicas` helper in `_helpers.tpl` (reference in `template-chart`); the helper uses an explicit nil check (`kindIs "invalid"`) because Helm's `default` would swallow an explicit `0`.
- **A chart whose default is more than one replica ships a `PodDisruptionBudget`** — `policy/v1`, named `<release>-<chart>-pdb`, `minAvailable` as a value (default `1`), selector matching the workload's `app` label. `keycloak` (Gitea #83) is the first and sets the convention. The reason it is a rule and not a per-chart taste: a node drain is the most common reason a pod moves — it is what a node upgrade does — and without a budget the drain is free to take every replica at once, so the second replica buys nothing against the failure it was added for.
- **And it is rendered only while the effective replica count is `>= 2`.** `minAvailable: 1` with exactly one replica forbids every voluntary eviction, so `kubectl drain` blocks forever on a pod nothing may move — and the message names the PDB, not the replica count, which sends the reader to the wrong file. The same applies to `scaleDown: true`, where there are no pods and the object would describe a promise about nothing. So the condition is on the **computed** replica count (the `<chart>.replicas` helper), not on the raw value, and `NOTES.txt` prints which of the two happened rather than leaving it to be inferred from `kubectl get pdb`.
- **A chart whose replicas are a property of a *group* gets one PDB per group, with that group's own count** (Gitea #96; `stalwart` is the first, one node group per cluster role). The condition above then applies per group — with `full: 2` and `mx: 1` exactly one object is rendered — and `NOTES.txt` lists the outcome per group rather than once for the release. The name carries the group (`<release>-<chart>-<group>-pdb`), and so does the selector.
- **A budget without a distribution rule is half an answer.** The PDB keeps a drain from taking every replica at once; it says nothing about where the replicas were put in the first place. Where a chart's replicas are meant to survive the loss of a node, it also offers a **per-group switch that renders the `topologySpreadConstraints` entry itself, with a selector the chart fills from its own pod labels** — never one the operator writes, because a spread constraint whose selector matches no pod is trivially satisfied and distributes nothing, with a green rollout and no message. `stalwart` sets the convention: `nodes.<group>.spreadAcrossNodes` (default `true`, gated on the same computed `>= 2`), `maxSkew: 1` on `kubernetes.io/hostname`, `whenUnsatisfiable: DoNotSchedule` so the distribution is a promise rather than an effort, and a generic per-group pass-through (`affinity`, `nodeSelector`, `tolerations`, `topologySpreadConstraints`) for everything the switch does not cover. Three traps come with it and all three belong in the chart's `values.yaml`: `nodeAffinityPolicy: Ignore` must be set **explicitly** (its API default is `Honor`, under which a narrowing `nodeSelector` shrinks the set of counted domains until both replicas fit in one, giving skew 0 — co-location with no `Pending`, no event, and a selector that is *correct*, so no CI check can see it); that field is **beta** and is pruned without a word below Kubernetes 1.26, which brings that exact gap back; and the replica ceiling is **a rule, not a number** — as many replicas per group as there are schedulable workers, because `nodeTaintsPolicy` defaults to `Ignore` and control-plane domains therefore hold the minimum at 0 permanently. That ceiling comes from `nodeTaintsPolicy` and is **not** a consequence of setting `nodeAffinityPolicy`; the two are documented separately for that reason. A user entry colliding with the chart's own on `(topologyKey, whenUnsatisfiable)` **aborts the render naming both values**, because the two alternatives are an API error pointing at a list index and a silent merge over `patchMergeKey: topologyKey` that drops one selector and goes green.
- **Both selectors are held against the rendered pod labels on every PR** by `ci/run-selector-match-test.sh` — see "Selector-match render test" above. Needed because the shared-helper rule above is a convention, and a convention is the thing a copy breaks.
### 12. Values that must not lie (#68, #93, #99, #215)
@@ -428,19 +272,11 @@ Reference implementation: `template-chart/values.schema.json` and `template-char
**2. Probes, security contexts and resources go through the null-safe helper, never behind an `if`.** An override written without children is YAML null and erases the block: the container renders `securityContext: null` (unhardened, healthy, silent), a probe disappears, or `resources` loses its requests and computed limits. The `defaultedDict` helper defines the opposite semantics — an empty block means "chart defaults apply" — while an explicitly set value still wins, **including `false` and `0`** (hence the `kindIs "invalid"` check; `default` and `mergeOverwrite` both swallow zero values). Do **not** wrap these blocks in `{{- if }}`: a guard does not render a visible `null`, it removes the probe entirely, which is the same bug with no evidence left.
The rule is about the **shape, not about those three block kinds**. `satisfactory` shipped 7.0.0 with `game` null-safe and its two neighbours not (Gitea #16): `--set server=null` aborted the render, and a single erased key was the quieter half — a nulled `server.gamePort` rendered `port:` and `nodePort:` empty into the Service and an empty `containerPort` into the StatefulSet, rejected by nothing until the API server saw it. Every chart-owned block a template dereferences belongs behind the helper, and a chart where only *some* of them do is worse than one where none do: whoever sees `game=null` fall back cleanly assumes `server=null` does too.
**A deliberate exception is fine; an undocumented one is the bug.** `satisfactory.storage` is left un-defaulted on purpose — an erased size yields an invalid PVC that fails at apply time rather than a volume quietly sized by the chart — so `server: null` still renders a PVC without a size. That is a defensible trade and it is *not* the null-safety bug next to it; what makes it safe to leave standing is that the schema, `values.yaml` and the chart page all name it as an exception and point at the issue where it is argued (Gitea #39). Note also where the trade actually bites: only on a **new** install. For an existing release a fallback to the default size would fail too, because a PVC cannot shrink. Do not overturn a reasoned exception inside a bugfix patch — the patch then stops being one.
**And `helm template` does not render `NOTES.txt`.** A value read there — `server` is read in four files in that chart — stays broken after a fix that looks green in every render, until someone runs a real `helm install`. `helm install --dry-run=client` is not the way around it either: it still wants a reachable API server (measured on helm v3.22.0). What works without a cluster: in a throwaway copy of the chart, wrap the verbatim `NOTES.txt` body in a `{{- define }}` and `include` it into a scratch ConfigMap, which `helm template` *does* render — same engine, same root context, only the sink differs.
**3. Do not hang liveness on an external dependency where a dependency-free path exists.** `zipline` deliberately probes `/` for liveness and `/api/healthcheck` only for readiness and startup, so a database outage cannot restart the pod. Where no such path exists — `outline` returns 500 on `/` when the database is down — all three probes may share the endpoint, but the reason belongs in a comment on the helper, or the next reader will "fix" the inconsistency.
The other half of the same rule: readiness **must** hang on the dependency, and a separate path is not proof that it does. `urlaubsverwaltung` probed `/actuator/health/liveness` and `/actuator/health/readiness` for two years and still stayed 1/1 Ready through a total database outage (#218), because Spring Boot's `readiness` group contains `readinessState` **alone** — the split paths were right, the group content was empty of anything external. Whatever aggregates behind a probe path (Spring health groups, a `/healthz` a framework assembles for you) has to be read off the **running** endpoint, not off the path name; the chart names the members explicitly (`MANAGEMENT_ENDPOINT_HEALTH_GROUP_READINESS_INCLUDE=readinessState,db`) so the composition is in the template rather than in an upstream default.
**4. A fallback reproduces *this chart's* defaults, not the repo standard.** Several charts deviate from the hardening baseline on purpose (no count here — the previous wording named one and went stale the next time a chart landed). A "cleaned up" standard default is not tidying, it is an outage: `calibre-web-automated` must keep `readOnlyRootFilesystem: false`, because in read-only mode the image disables its PUID/PGID remapping and the runtime uid silently moves from 1000 to 911 — **making existing volume data inaccessible**. The same applies to non-standard values that look like mistakes: `satisfactory` keeps `limitFactor: 2` (that is what holds the memory limit at the published 8Gi) and `fsGroupChangePolicy: OnRootMismatch` (which avoids a recursive chown of multi-GB game files on every start).
**The sharpest case is a capability that is granted and never used.** `stalwart`'s fallback has to carry `capabilities.add: [NET_BIND_SERVICE]`, because the image sets `setcap cap_net_bind_service=+ep` on its binary — which makes it a capability-dumb binary, and `capabilities(7)` then has the kernel fail the **`execve` itself** when the bounding set does not carry the capability: *"If the process did not obtain the full set of file permitted capabilities, then `execve(2)` fails with the error `EPERM`."* So `drop: [ALL]` with the standard empty `add` does not produce an unhardened container, it produces `exec /usr/local/bin/stalwart: operation not permitted` and a chart that cannot start. The capability is then spent entirely on the exec check and is never used to bind anything, which is exactly what makes it a magnet for a hardening pass looking for dead grants. This is why a deviation is documented with its **mechanism** and not only its name: "needed for binding" would have been wrong and would have invited the removal.
**4. A fallback reproduces *this chart's* defaults, not the repo standard.** Five charts deviate from the hardening baseline on purpose. A "cleaned up" standard default is not tidying, it is an outage: `calibre-web-automated` must keep `readOnlyRootFilesystem: false`, because in read-only mode the image disables its PUID/PGID remapping and the runtime uid silently moves from 1000 to 911 — **making existing volume data inaccessible**. The same applies to non-standard values that look like mistakes: `satisfactory` keeps `limitFactor: 2` (that is what holds the memory limit at the published 8Gi) and `fsGroupChangePolicy: OnRootMismatch` (which avoids a recursive chown of multi-GB game files on every start).
**5. A probe default carries the probe's context, not just its timings.** Two different symptoms, both from real charts: `teamspeak-server` addresses its TCP port by the name `file-transfer` because the voice service is UDP and cannot be probed — a default without that name leaves the workload **with no working probe at all**. `patchmon` sends a `Host` header because the server answers **403** to any request whose Host is neither loopback nor a CORS_ORIGIN host — a default without it means the pod never becomes Ready. Port names, `Host` headers and `exec` commands belong in the fallback; where the context is dynamic, pass the root context in and rebuild it (patchmon derives the header from `ingress.domain`).
@@ -448,18 +284,12 @@ Reference implementation: `template-chart/values.schema.json` and `template-char
**Two things about the mechanics that are easy to lose:**
- **`null` reaches the schema in two cases out of three, and which one you are in depends on the chart's default, not on what the user wrote.** Measured on Helm 3.22.0 with a probe chart, because the wording here was wrong before and was applied to exactly the case it got wrong:
- a user `null` over a **non-null** chart default (`--set x=null`, or `x: null` in a values file) is treated as a deletion and the key is removed *before* schema validation — the schema never sees it, and only the helper can catch it;
- a **`null` chart default** survives into validation with or without a user `null` on top of it, because there is nothing to delete it against, and the schema sees the key present with value `null`;
- a key **without any default** (e.g. `resources.limits`) likewise survives, and a strict `"type": "object"` aborts the render.
That is why the helper **and** the widened `["object", "null"]` / `["string", "null"]` types are both needed: each covers a case the other cannot. Everything else stays strictly typed.
- **A value that has no valid default cannot be enforced by `values.schema.json` at all, and `required()` is the whole enforcement.** The chain is short and it has no exit (#18): the key ships as `null`, so it is *present* at validation time, and JSON Schema's `required` tests presence and not null-ness — the array never fires, under any input. The two ways to make it fire both break the rule that `helm lint --strict` passes on chart defaults, since lint validates those defaults against the schema: dropping the `null` default gives `missing property '<key>'`, and narrowing `["string", "null"]` to `"string"` gives `got null, want string` — each a red `validate-charts` run on a chart nobody touched. So such a key is guarded with `required()` in the template and **described** as required in the schema; a `required` array next to it would be a guard that cannot fire, which is the same defect as a value that does nothing, dressed as a safeguard. This is why no chart in this repo carries one.
- **`null` reaches the schema in one case and not the other.** For a key **with** a chart default, Helm's coalescing treats the `null` as a deletion and removes it *before* schema validation — the schema never sees it, and only the helper can catch it. For a key **without** a default (e.g. `resources.limits`), there is nothing to delete, the `null` survives into validation, and a strict `"type": "object"` aborts the render. That is why the helper **and** the widened `["object", "null"]` types are both needed: each covers a case the other cannot. Everything else stays strictly typed.
- **Validate every schema against the real values, not only against chart defaults.** Defaults are valid by construction, so they prove nothing. Render each chart against its bundle in `cluster-config/fleet-cd/` **and** against `helm get values` — the two differ more often than expected (four of ten bundles in one wave, always with the repo file being the worse one). This step caught a schema of mine that would have broken three live paperless releases, because all three write `tikaGotenberg.enabled` as the string `"true"` and the first draft demanded a boolean. Result per chart is a **list of rejected keys**, which is the cluster side's work order before the version is pulled up.
### 13. Everything else
- `icon:` is mandatory in every `Chart.yaml`. **Documented exception — freesailarr, the only chart without one**: the rule everywhere else is "the icon is this chart's one upstream's logo", and a chart bundling ten independently versioned projects has no such upstream — picking one of the ten would imply a "main app" the stack deliberately does not have. That is the whole criterion, and it applies to no other chart in the repo: a single-upstream chart without an icon is a missing icon, not an exception. The omission costs an INFO line in `helm lint --strict` (`icon is recommended`), which still exits 0.
- `icon:` is mandatory in every `Chart.yaml`. **Documented exception — freesailarr, the only chart without one**: the rule everywhere else is "the icon is this chart's one upstream's logo", and a chart bundling nine independently versioned projects has no such upstream — picking one of the nine would imply a "main app" the stack deliberately does not have. That is the whole criterion, and it applies to no other chart in the repo: a single-upstream chart without an icon is a missing icon, not an exception. The omission costs an INFO line in `helm lint --strict` (`icon is recommended`), which still exits 0.
- `helm lint --strict` must pass without findings.
- **Every chart ships a `templates/NOTES.txt`** (#43), `template-chart` included — so this is a requirement for a new chart, not a nice-to-have it can defer. No list of names here on purpose: a count goes stale the moment the next chart lands, which is exactly how the previous wording came to claim a number it never matched. Reference implementation: `template-chart/templates/NOTES.txt`, a placeholder-carrying template rather than finished prose, so a new chart adapts it instead of starting from an empty file. The bar it sets: print what an operator would otherwise have to look up in the templates (the rendered access URL, **which fields the 1Password item must carry**, the options actually active in this release, and the chart's known traps), conditional on the values really set. A `NOTES.txt` that only repeats the values is skimmed once and never read again. **No real domain anywhere** — the URL renders from `ingress.domain`, and where an example is unavoidable it is `example.com`. A `NOTES.txt` renders no Kubernetes object, so writing or reworking one is purely additive: a minor bump.
- CI: every installable chart has fixtures under `ci/<chart>/` (`test-values.yaml`, optionally `fixtures.yaml`/`fixtures.env.yaml`/`settings.env`) or a justified entry in `ci/excluded-charts.txt` — the install-test fails if both are missing.
@@ -467,20 +297,6 @@ Reference implementation: `template-chart/values.schema.json` and `template-char
Until `ytdlp-web-player` (#214) this section read "charts have no dependencies (`charts/` directories are empty)". That rule is gone, replaced by a bar rather than a ban — a dependency is **admissible, not free**, and the reason it stopped being a ban is worth keeping: the chart needed an oauth2-proxy in the data path in front of an application that ships no authentication at all, and proxy and app are only useful wired to each other.
Three charts carry one, and comparing them is the quickest way to see where the bar sits. `ytdlp-web-player` takes **upstream's** `oauth2-proxy` because the app has no authentication and the two are useless apart; there, point 2 above applies in full. `stalwart` (Gitea #65) takes **one of our own published charts**, `valkey`, because Stalwart's **cluster coordinator rides on the in-memory store connection**, so a multi-node install does not work without one. `paperless-ngx` (Gitea #78) takes the same one, because `PAPERLESS_REDIS` is the Celery broker and the application therefore cannot run without a Redis at all — before the subchart, every installation needed a second Helm release whose address lived in a 1Password item and in neither chart. All three are `condition:`-gated and off by default: the subchart is the path for an installation that has no Valkey yet, and one that already has points its connection value at that instead, so the dependency renders nothing.
**The same subchart, opposite configuration — and that comparison is the useful part.** `stalwart` runs it as a **cache** (`persistence.enabled: false` + `allkeys-lru`): its in-memory store holds rate-limiter counters, tokens and pub/sub, all reconstructible, and the cost it accepts is that the limiters fail open under memory pressure. `paperless-ngx` runs it as a **queue** (PVC + `noeviction`, the subchart's defaults): the broker's contents are work, and a dropped consume task is a document that is silently never processed. `allkeys-lru` would not be a trade there but actively wrong, because LRU evicts typeless across the whole keyspace and a backlog of waiting tasks is exactly what it reads as cold. The price `paperless-ngx` accepts instead: the same instance also serves the Django cache and the Celery result backend (both Redis in 3.x), so a full instance **refuses writes** — new tasks included — rather than dropping cold keys, and the answer is memory sizing, not a policy change. The valkey chart treats the two values as a **pair** and warns against mixing them; these two charts are the worked examples of picking either half deliberately.
**A dependency on one of our own charts is cheaper in one way and more dangerous in another.** Point 2 stops applying: there are no third-party defaults to leave unpatched, because the subchart follows these guidelines by being one of ours. In exchange it adds a hazard an external dependency does not have, and it is sharp enough to be a rule:
> **Never change a chart and a chart that depends on it in the same pull request.**
The dependency resolves from the **published index**, not from the working tree, and a chart is published only by a push to `main` that touches it. So a combined PR runs every `helm` command — lint, package, render **and the install-test** — against the **old** subchart, goes green on an artifact that is not the one in the diff, and leaves the subchart's own change exercised by nothing. Two consecutive PRs: change the subchart, merge, let it publish, then raise the `version:` in the dependent chart.
Two mechanical details of a self-referencing dependency, both measured for #65: the index is **anonymously readable**, so `ci/helm-dependencies.sh` needs no credential and resolves the exact build-metadata version (`1.0.0+up9.1.2.alpine`) from the URL alone with an empty `HELM_REPOSITORY_CONFIG` — the state a fresh runner is in. And Renovate's built-in `helmv3` manager does follow a self-hosted Gitea index: a dry run (Renovate 44.132.2) with the dependency pinned low reported `newVersion: "1.0.0+up9.1.2.alpine"`, `versioning: "helm"`, timestamp read from our index. Without that second result the published-version form would have had no update path at all, which is why it was measured rather than assumed.
One more trap, and it is the reason `stalwart` gates its subchart on `valkeySubchart.enabled` rather than the conventional `valkey.enabled`: **everything under the subchart's key is handed to the subchart as its values**, so a `condition:` key placed there is validated against *its* schema. Our charts ship `additionalProperties: false`, so `valkey.enabled` is rejected by the subchart — and because Helm skips a **disabled** subchart's schema validation, the error surfaces only for whoever first switches it on. Put the condition on a key of the parent's own.
**When a dependency earns its place.** All three, not two of three:
1. The dependent piece would otherwise be a second Helm release that only exists to serve this one, with the wiring between them living in neither chart.
+1 -1
View File
@@ -5,5 +5,5 @@ icon: https://www.apache.org/logos/res/tika/default.png
type: application
version: 10.0.2+up3.3.1.0.full
version: 10.0.1+up3.3.1.0.full
appVersion: "3.3.1.0-full"
+5 -8
View File
@@ -19,11 +19,9 @@ Usage (the values path to the resources dict varies per chart):
resources:
{{- include "apache-tika.resources" .Values.<app>.resources | nindent 12 }}
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart, and a dependency is admissible but expensive (README, guideline 14) -
this chart takes none. This file in template-chart is the reference
implementation.
NOTE: charts in this repo deliberately have no dependencies, so this helper is
duplicated into every chart's templates/_helpers.tpl with the chart's name as
define prefix. This file in template-chart is the reference implementation.
*/}}
{{- define "apache-tika.resources" -}}
{{- $resources := . | default dict -}}
@@ -170,9 +168,8 @@ Rules:
- keys only present in the override are passed through unchanged.
NOTE: like the helpers above, this is duplicated into every chart's
templates/_helpers.tpl with the chart's name as define prefix, because sharing
it would mean a library chart and this chart takes no dependency (README,
guideline 14).
templates/_helpers.tpl with the chart's name as define prefix, because charts
in this repo deliberately have no dependencies.
*/}}
{{- define "apache-tika.defaultedDict" -}}
{{- $defaults := .defaults | default dict -}}
+1 -1
View File
@@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/alam00000/bentopdf/main/public/images/fa
type: application
version: 1.0.3+up2.8.8
version: 1.0.2+up2.8.8
appVersion: "2.8.8"
maintainers:
+5 -8
View File
@@ -19,11 +19,9 @@ Usage (the values path to the resources dict varies per chart):
resources:
{{- include "bentopdf.resources" .Values.<app>.resources | nindent 12 }}
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart, and a dependency is admissible but expensive (README, guideline 14) -
this chart takes none. This file in template-chart is the reference
implementation.
NOTE: charts in this repo deliberately have no dependencies, so this helper is
duplicated into every chart's templates/_helpers.tpl with the chart's name as
define prefix. This file in template-chart is the reference implementation.
*/}}
{{- define "bentopdf.resources" -}}
{{- $resources := . | default dict -}}
@@ -170,9 +168,8 @@ Rules:
- keys only present in the override are passed through unchanged.
NOTE: like the helpers above, this is duplicated into every chart's
templates/_helpers.tpl with the chart's name as define prefix, because sharing
it would mean a library chart and this chart takes no dependency (README,
guideline 14).
templates/_helpers.tpl with the chart's name as define prefix, because charts
in this repo deliberately have no dependencies.
*/}}
{{- define "bentopdf.defaultedDict" -}}
{{- $defaults := .defaults | default dict -}}
-23
View File
@@ -1,23 +0,0 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
-36
View File
@@ -1,36 +0,0 @@
apiVersion: v2
name: bulwark
description: A Helm chart to deploy Bulwark Webmail, a JMAP webmail client (mail, calendar, contacts, files) for a Stalwart mail server
icon: https://avatars.githubusercontent.com/u/266882946
type: application
# appVersion is the exact image tag of ghcr.io/bulwarkmail/webmail.
#
# THE SUFFIX-FREE FORM, AND THAT IS A CHOICE RATHER THAN THE ONLY OPTION. This
# registry publishes every release TWICE - 1.12.0 and v1.12.0 are the same image
# under two names (counted 2026-10-05 against the ghcr.io tags API: 315 tags, and
# both families run the whole way back to 1.3.9). So unlike stalwart, garage,
# matrix-synapse and calibre-web-automated, where the "v" IS the tag and
# therefore part of appVersion and of the "+up" suffix, here it is optional and
# this chart pins the form without it. Do not "normalize" it to v1.12.0 later:
# that would rewrite appVersion and the chart version for an identical image.
#
# The tag census, Renovate's handling of each shape and the three dry-run
# fixtures that prove it are in renovate.json5's bulwark block rather than
# duplicated here. The short version: eight tag shapes, six of them things
# nobody should pin, and plain `docker` versioning excludes all six - so that
# block deliberately carries NO versioningTemplate.
version: 1.0.0+up1.12.0
appVersion: "1.12.0"
maintainers:
- name: jklein
# NO DEPENDENCIES, and nothing here wants one. Bulwark is a client: it holds no
# mail, has no database, no cache and no broker. The mail server it talks to is
# the `stalwart` chart (Gitea #65), a SEPARATE RELEASE reached over the network
# by the URL in bulwark.jmapServerUrl - not a subchart, because the two are
# installed, scaled, upgraded and backed up independently and a JMAP endpoint is
# an address rather than a bundled component. README, "Chart dependencies", for
# the bar a dependency has to clear.
-252
View File
@@ -1,252 +0,0 @@
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
{{- $replicas := int (include "bulwark.replicas" (dict "scaleDown" .Values.scaleDown "replicas" $settings.replicas)) -}}
{{- $sfs := printf "%s-%s-sfs" .Release.Name .Chart.Name -}}
{{- $svc := printf "%s-%s-service" .Release.Name .Chart.Name -}}
{{- $url := include "bulwark.publicUrl" . -}}
Bulwark Webmail {{ .Chart.AppVersion }} installed as release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
{{ $replicas }} replica(s). This is a CLIENT: it holds no mail and has no database.
The messages and the accounts belong to the Stalwart server at
{{ $settings.jmapServerUrl }}
If that server is away, this release is still Ready and still green - see
"WHAT THE PROBES DO NOT TELL YOU" below.
ENDPOINTS
{{ if .Values.ingress.enabled }} Web: {{ $url }}
{{ else }} Web: no Ingress (ingress.enabled: false) - in-cluster only
{{ end }} In-cluster: http://{{ $svc }}.{{ .Release.Namespace }}.svc.cluster.local:{{ $settings.port | int64 }}
{{ if .Values.ingress.enabled }} Admin: {{ $url }}/admin/login
{{ else }} Admin: /admin/login on the in-cluster address above. With no Ingress, reach it with
kubectl -n {{ .Release.Namespace }} port-forward statefulset/{{ $sfs }} 3000:{{ $settings.port | int64 }}
and open http://localhost:3000/admin/login
{{ end }}{{ if .Values.ingress.enabled }}
Bulwark is served at the ROOT of that host and cannot be moved under a path:
NEXT_PUBLIC_BASE_PATH is compiled into the image.
{{ end }}
THE 1PASSWORD ITEMS, AND WHICH FIELDS THEY MUST CARRY
The operator materializes one Secret key per item FIELD. A missing or misnamed
field here does NOT stop the pod - the two file-form secrets simply read as
absent - so the symptom is a feature that is quietly off rather than an error.
{{ .Values.secrets.appSecretRef | default "<secrets.appSecretRef NOT SET>" }}
session-secret openssl rand -base64 32
MANDATORY. Persistent sessions and the encryption of the
synchronized settings both hang on it, and the second
fails SILENTLY without it: settings sync is computed as
"enabled AND a secret exists", so with no secret it is
simply off and nothing says so.
admin-password the admin dashboard's initial password. CONSUMED ONCE -
see step 5 below.
{{ if $settings.oauth.enabled }}
{{ .Values.secrets.oauthSecretRef | default "<secrets.oauthSecretRef NOT SET>" }}
oauth-client-secret the confidential OIDC client's secret, as issued by
the identity provider when the client was registered.
{{ else }}
(no OAuth item - bulwark.oauth.enabled is false, so sign-in is by password)
{{ end }}
The file-form secret{{ if $settings.oauth.enabled }}s are{{ else }} is{{ end }} mounted as a DIRECTORY, not with subPath, and
re-read on every use - so rotating {{ if $settings.oauth.enabled }}either field{{ else }}that field{{ end }} in the vault takes effect
WITHOUT a restart. admin-password is the exception and no restart helps it.
{{ if $settings.oauth.enabled }}
BEFORE YOU INSTALL - THE OIDC CLIENT HAS TO EXIST FIRST
{{ if $settings.oauth.only }}
THIS IS NOT ADVICE AND IT IS NOT A FOLLOW-UP STEP. bulwark.oauth.only is ON,
which means the password form is hidden, which means that WITHOUT A WORKING
OIDC CLIENT THE FIRST SIGN-IN IS IMPOSSIBLE. There is no fallback for users.
{{ end }}
Register a CONFIDENTIAL client with the provider at
{{ $settings.oauth.issuerUrl }}
client id: {{ $settings.oauth.clientId }}
secret: into the vault item's oauth-client-secret field
THE REDIRECT URI CARRIES THE UI LANGUAGE, AND THAT IS THE ONE THING THAT WILL
COST YOU THE FIRST SIGN-IN IF YOU REGISTER A SINGLE URI. Bulwark builds it as
{{ $url }}/<locale>/auth/callback
where <locale> is the language the VISITOR'S BROWSER resolved to - so a German
browser sends /de/auth/callback and an English one /en/auth/callback, from the
same deployment. The language is not in the visible URL (the locale prefix is
"never" in this image), which is exactly why this is surprising: the address
bar shows no language and the redirect_uri contains one.
Register ONE PER LANGUAGE YOU EXPECT, or a wildcard if the provider allows it.
On Keycloak the wildcard is the practical answer:
{{ $url }}/*
Otherwise the 27 locales Bulwark supports are:
ar ca cs da de en es fa fr he hu it ja ko lv mn nb nl pl pt ro ru sk tr uk
zh zh-TW
If a sign-in fails with a redirect_uri mismatch, read the URI out of the
failed request rather than guessing which language it was - that is the value
the provider compares byte for byte.
POST-LOGOUT, IF YOU USE IT: bulwark.oauth.endSession is {{ $settings.oauth.endSession }}, and a
post-logout redirect URI is only sent when OAUTH_POST_LOGOUT_REDIRECT_URI is
set (this chart does not render it). If you add it through bulwark.env, it has
to be registered with the provider too.
The issuer must be reachable BY THE POD, and the endpoints it advertises must
share the issuer's origin or be publicly routable - Bulwark rejects a
discovered endpoint on a private address from a different origin. A
split-horizon DNS setup needs OAUTH_ALLOW_PRIVATE_ENDPOINTS through
bulwark.env; this chart does not render it.
{{ end }}
FIRST INSTALL
1. kubectl -n {{ .Release.Namespace }} rollout status statefulset/{{ $sfs }}
Ready means the HTTP server is accepting. It does NOT mean the mail server
is reachable - nothing in this chart checks that.
2. Confirm the setup wizard really is OFF. It is, as long as
bulwark.jmapServerUrl is set, and the chart refuses to render without it -
but this is the one thing worth seeing rather than trusting, because a
wizard that ran once changes the rules for good (step 4):
kubectl -n {{ .Release.Namespace }} logs statefulset/{{ $sfs }} | grep -i "SETUP REQUIRED"
Nothing found is the correct result. A "SETUP REQUIRED" banner with a token
means Bulwark is in bootstrap mode and something is wrong with the value.
3. Sign in{{ if $settings.oauth.only }} through the identity provider{{ end }} at {{ $url }}
4. READ THIS BEFORE YOU OPEN THE ADMIN DASHBOARD. The dashboard writes
OVERRIDES into config.json on the volume, and an override BEATS this chart
for that key - for good, and silently.
Priority is: admin override > environment variable > default.
So if you save a server URL in the dashboard, changing bulwark.jmapServerUrl
and running helm upgrade afterwards does nothing and reports success. The
same is true of every other key the dashboard can write. What the chart
sets is a DEFAULT, not a setting.
To see which keys are currently overridden:
kubectl -n {{ .Release.Namespace }} exec {{ $sfs }}-0 -- cat /app/data/admin/config.json
An empty object, or a file that does not exist, means this chart is in
force.
5. THE ADMIN PASSWORD IS CONSUMED EXACTLY ONCE. On the first start it is
hashed into /app/data/admin/admin.json and the environment variable is
never read again. Rotating the vault field afterwards has NO EFFECT -
changing the password goes through the dashboard, or by deleting admin.json
from the volume and restarting. Know that before you treat the vault field
as the source of truth.
IF NOBODY CAN SIGN IN - THE WAY BACK, AND IT IS NOT A helm upgrade
Read this now, not when it happens, because the obvious repair does not work.
The situation: the identity provider is down, or its configuration is wrong,
and bulwark.oauth.only hides the password form. No user can get in.
WHAT DOES NOT WORK RELIABLY: helm upgrade with bulwark.oauth.only=false.
oauthOnly is an ordinary config key, and every part of the app reads it
through the priority chain above - not from the environment. If the admin
dashboard's OAuth setup has ever been used, "oauthOnly": true is sitting in
config.json on the volume, and from then on the chart's value is ignored. The
upgrade succeeds and changes nothing.
WHAT DOES WORK, and it does not depend on the identity provider:
a) Open {{ $url }}/admin/login and sign in with the admin-password field
from the vault item. This door has its own password and is not behind
OIDC.{{ if eq $settings.stalwartAdminAccess "auto" }} (bulwark.stalwartAdminAccess is "auto", so a Stalwart
admin also gets in here without that password.){{ end }}
b) In the dashboard, REVERT the override - the dashboard's own "remove
override" on the OAuth settings. That is what puts the chart's value back
in charge.
IF THE DASHBOARD ITSELF IS UNREACHABLE, the last resort is the volume:
kubectl -n {{ .Release.Namespace }} exec {{ $sfs }}-0 -- cat /app/data/admin/config.json
# remove the "oauthOnly" key (and any other override in the way), then
kubectl -n {{ .Release.Namespace }} rollout restart statefulset/{{ $sfs }}
Deleting admin.json in the same directory resets the admin password to
whatever the vault field currently holds, on the next start.
WHAT THE PROBES DO NOT TELL YOU
All three probes are a plain HTTP GET on /api/health, and that endpoint returns
200 whatever state the application is in - for an unauthenticated caller the
body even says "healthy" unconditionally. So a green pod here means: the
process is alive, HTTP is being accepted, the event loop is free, and Next.js
can route.
IT SAYS NOTHING ABOUT THE MAIL SERVER. With Stalwart completely gone, this
release is Ready, green, and serves a web app that cannot fetch a single
message. Monitor the stalwart release for that - there is no endpoint here
that could cover it, and the one that would (/api/setup/test-jmap) sits behind
the setup token this chart disables.
It also says nothing about the reported state: /api/health reports "degraded"
on high V8 heap usage, but only to an authenticated caller asking for details,
and always with HTTP 200. A memory limit set too low therefore produces a pod
that is permanently degraded and permanently Ready.
BACKUPS - THE VOLUME ALONE IS NOT ONE, AND IT MAY NEED PROTECTING ITSELF
Two statements, both of them about {{ .Release.Name }}-{{ .Chart.Name }}-pvc:
1. A BACKUP OF THE VOLUME ALONE IS NOT A BACKUP.{{ if $settings.settingsSync }} The synchronized user
settings on it are encrypted with session-secret.{{ else }} The user settings are not
synchronized in this release, but persistent sessions still are.{{ end }} Lose the vault
item and that data is unreadable out of a perfectly intact volume. Back up
the 1Password item together with the volume, or the restore is a directory
of ciphertext.
2. A BACKUP OF THE VOLUME MAY CONTAIN A SECRET IN CLEARTEXT. If anybody has
used the dashboard's OAuth setup, the OIDC client secret is written as
PLAINTEXT into config.json on this volume. So the backup may need the same
protection the vault gets. Check with:
kubectl -n {{ .Release.Namespace }} exec {{ $sfs }}-0 -- grep -l oauthClientSecret /app/data/admin/config.json
TRAPS WORTH KNOWING BEFORE YOU NEED THEM
- WEB PUSH GOES THROUGH THE VENDOR'S RELAY until somebody changes it, and the
change is NOT a chart value. The built-in default is
https://notifications.relay.bulwarkmail.org and it is compiled into the
image. A self-hosted relay CAN be made the effective default and even
mandatory for all users - through the settings policy in the admin
dashboard, because that setting has no environment variable at all. So this
is a choice you have, not a limitation you are stuck with; it just lives in
the dashboard.
- THERE IS NO ConfigMap IN THIS CHART, so no checksum/config annotation, and
that is not an omission: there is no rendered config file whose change could
fail to restart anything. Repointing secrets.* at a different vault item
DOES roll the pod (checksum/secret-ref). Changing a value INSIDE an item
does not - but for the two file-form secrets it does not need to, because
they are re-read on every use.
{{ if $settings.adminConfigReadOnly }}
- bulwark.adminConfigReadOnly IS ON. The chart is authoritative and no
override can appear - and three things are frozen with it: the settings
policy (so Web Push cannot be moved to your own relay any more), any
override that was ALREADY on the volume (it cannot be removed either), and -
if this release has never started with it off - the admin password, which
means the admin dashboard never comes up and its login answers HTTP 500
while this pod stays Ready.
If you have just enabled this on a fresh install, turn it off, let the pod
start once so admin.json is written, and only then turn it back on.
{{ else }}
- bulwark.adminConfigReadOnly is OFF, which is the default and keeps the
dashboard working. The cost is the precedence trap of step 4: the volume can
override this chart at any time. Turning it on makes the chart authoritative
but freezes the settings policy, freezes any override already present, and
must never be the state of a FIRST install - see values.yaml for the
sequence.
{{ end }}
- THE SETTINGS POLICY, THE THEMES AND THE BRANDING ARE NOT IN THIS CHART.
They live in policy.json on the volume and are set in the dashboard. This
chart neither renders nor backs them up beyond the volume itself.
WHAT THIS CHART DOES NOT RENDER
- No ConfigMap (see above) and no NetworkPolicy. The pod needs egress to the
JMAP server and to the OIDC issuer; a policy added later has to allow both,
or sign-in breaks in a way that looks like a provider outage.
- No PodDisruptionBudget: with one replica a minAvailable of 1 would forbid
every voluntary eviction and make `kubectl drain` hang forever on a pod
nothing may move (README, guideline 11).
- No sub-path Ingress, and no value that could add one - the base path is
compiled into the image.
- No proxy-body-size annotation. Attachment uploads go through the Ingress and
nginx's default of 1m rejects larger ones with a 413 that reads like an
application fault; the right number is the mail server's limit, which this
chart does not know. Set it in ingress.annotations.
-464
View File
@@ -1,464 +0,0 @@
{{/*
Render the body of a container "resources:" block from a values dict of the form
resources:
limitFactor: 3 # optional, default 3
requests: { cpu, memory, ephemeral-storage / ephemeralStorage, ... }
limits: { ... } # optional, every entry optional
Rules (variant B, see issue #19):
- requests are rendered as given (camelCase "ephemeralStorage" is normalized
to the Kubernetes key "ephemeral-storage").
- an explicitly set limit always wins.
- a missing memory / ephemeral-storage limit is computed as
limitFactor x the corresponding request.
- a missing cpu limit is NOT set at all (the kubelet then treats CPU as
unlimited; CPU is compressible, so this cannot kill the workload).
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart, and a dependency is admissible but expensive (README, guideline 14) -
this chart takes none. The reference implementation is
template-chart/templates/_helpers.tpl.
*/}}
{{- define "bulwark.resources" -}}
{{- $resources := . | default dict -}}
{{- $factor := 3.0 -}}
{{- if not (kindIs "invalid" $resources.limitFactor) -}}
{{- $factor = float64 $resources.limitFactor -}}
{{- end -}}
{{- $requests := dict -}}
{{- range $key, $value := ($resources.requests | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $requests (include "bulwark.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- $limits := dict -}}
{{- range $key, $value := ($resources.limits | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $limits (include "bulwark.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- range $key, $request := $requests -}}
{{- if and (ne $key "cpu") (not (hasKey $limits $key)) -}}
{{- $_ := set $limits $key (include "bulwark.multiplyQuantity" (dict "quantity" $request "factor" $factor)) -}}
{{- end -}}
{{- end -}}
{{- $out := dict -}}
{{- with $requests }}{{- $_ := set $out "requests" . }}{{- end -}}
{{- with $limits }}{{- $_ := set $out "limits" . }}{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Normalize a resource name from values to its Kubernetes key.
*/}}
{{- define "bulwark.resourceKey" -}}
{{- if eq . "ephemeralStorage" -}}ephemeral-storage{{- else -}}{{ . }}{{- end -}}
{{- end -}}
{{/*
Multiply a Kubernetes quantity by a factor, keeping its suffix.
Input: dict "quantity" <string|number> "factor" <float64>.
*/}}
{{- define "bulwark.multiplyQuantity" -}}
{{- $quantity := include "bulwark.formatNumber" .quantity -}}
{{- $number := regexFind "^[0-9]+(\\.[0-9]+)?" $quantity -}}
{{- if not $number -}}
{{- fail (printf "cannot parse resource quantity %q" $quantity) -}}
{{- end -}}
{{- $suffix := trimPrefix $number $quantity -}}
{{- if not (regexMatch "^(m|k|M|G|T|P|E|Ki|Mi|Gi|Ti|Pi|Ei)?$" $suffix) -}}
{{- fail (printf "unsupported resource quantity suffix %q in %q" $suffix $quantity) -}}
{{- end -}}
{{- $product := mulf (float64 $number) .factor -}}
{{- if eq $product (floor $product) -}}
{{- printf "%d%s" (int64 $product) $suffix -}}
{{- else -}}
{{- printf "%s%s" (include "bulwark.formatNumber" $product) $suffix -}}
{{- end -}}
{{- end -}}
{{/*
Render a number as a plain decimal string, never in exponential notation
(issue #157): the null-safe fallback helpers pass values through fromYaml,
which parses every number as float64, and Go's "%v" switches float64 to
exponential notation from about 1e6 upwards - "1.048576e+06" is not a valid
Kubernetes quantity. Strings ("512Mi") are passed through untouched.
*/}}
{{- define "bulwark.formatNumber" -}}
{{- if kindIs "float64" . -}}
{{- regexReplaceAll "\\.?0+$" (printf "%.10f" .) "" -}}
{{- else -}}
{{- printf "%v" . -}}
{{- end -}}
{{- end -}}
{{/*
Render the replica count for the StatefulSet.
Input: dict "scaleDown" <flag|nil> "replicas" <value|nil>.
- scaleDown: true takes precedence and always yields 0.
- otherwise an explicitly set replicas value is used as given, including an
explicit 0 - Helm's "default" would swallow 0, hence the kindIs "invalid"
nil check.
- a missing replicas value defaults to 1.
A NOTE SPECIFIC TO THIS CHART: the schema caps this at 1, and the reason is the
single ReadWriteOnce PVC rather than anything about the application. Two pods
cannot both mount it, so a second replica would sit in ContainerCreating
forever - and the PVC is not optional here, because /app/data carries the admin
configuration, the audit log and the synchronized user settings. An RWX volume
would not make it a cluster either: nothing in Bulwark coordinates two
processes over that directory.
*/}}
{{- define "bulwark.replicas" -}}
{{- if .scaleDown -}}
0
{{- else if not (kindIs "invalid" .replicas) -}}
{{- .replicas -}}
{{- else -}}
1
{{- end -}}
{{- end -}}
{{/*
Merge a user-supplied values sub-tree onto a dict of chart defaults so that a
null / empty override falls back to the defaults instead of erasing them
(issues #61, #99). Input: dict "defaults" <dict> "given" <dict|nil> "path"
<string>. Output: YAML of the merged dict (consume it with fromYaml).
An override key written without children is YAML null, and Helm treats that
null as a deletion of the chart's default. This helper defines the opposite
semantics - an empty block means "chart defaults apply" - while an explicitly
set value still wins, including false and 0.
*/}}
{{- define "bulwark.defaultedDict" -}}
{{- $defaults := .defaults | default dict -}}
{{- $given := .given -}}
{{- $path := .path | default "value" -}}
{{- if kindIs "invalid" $given -}}
{{- $given = dict -}}
{{- end -}}
{{- if not (kindIs "map" $given) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" $path (kindOf $given) $given) -}}
{{- end -}}
{{- $out := deepCopy $defaults -}}
{{- range $key, $value := $given -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $default := index $defaults $key -}}
{{- if and (kindIs "map" $value) (kindIs "map" $default) -}}
{{- $_ := set $out $key (fromYaml (include "bulwark.defaultedDict" (dict "defaults" $default "given" $value "path" (printf "%s.%s" $path $key)))) -}}
{{- else if kindIs "map" $default -}}
{{- fail (printf "%s.%s must be a mapping or null, got %s (%v)" $path $key (kindOf $value) $value) -}}
{{- else -}}
{{- $_ := set $out $key $value -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Pick one sub-block out of a values block that may itself be null.
Input: dict "block" <dict|nil> "key" <string> "path" <string>.
Returns YAML of dict "value" <the sub-block, may be nil>.
*/}}
{{- define "bulwark.subBlock" -}}
{{- $block := .block -}}
{{- if kindIs "invalid" $block -}}
{{- $block = dict -}}
{{- end -}}
{{- if not (kindIs "map" $block) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" .path (kindOf $block) $block) -}}
{{- end -}}
{{- toYaml (dict "value" (index $block .key)) -}}
{{- end -}}
{{/*
THE EFFECTIVE BULWARK SETTINGS - everything the StatefulSet, the Service and
NOTES.txt read out of .Values.bulwark, merged onto this chart's defaults.
Why the whole settings block goes through the null-safe helper and not only the
three usual suspects (guideline 12.2: "the rule is about the shape, not about
those three block kinds"): every value below is dereferenced by a template, and
an erased one would not abort the render. An erased bulwark.port renders an
empty containerPort and an empty Service targetPort - rejected by nothing until
the API server sees it; an erased bulwark.oauth.enabled would silently turn OIDC
OFF on a release whose whole login path is OIDC, which is the quiet half and the
worse one. Satisfactory shipped exactly this shape with half its blocks
null-safe and half not (Gitea #16).
The defaults here MUST stay in sync with values.yaml, which remains the
documented, user-facing place for them; this copy is only the fallback.
Usage: {{ $settings := fromYaml (include "bulwark.settings" .) }}
*/}}
{{- define "bulwark.settings" -}}
{{- $given := (fromYaml (include "bulwark.subBlock" (dict "block" .Values "key" "bulwark" "path" ".Values"))).value -}}
{{- $defaults := dict
"replicas" 1
"port" 3000
"jmapServerUrl" nil
"appName" nil
"stalwartFeatures" true
"stalwartAdminAccess" "auto"
"settingsSync" true
"adminConfigReadOnly" false
"telemetry" false
"updateCheck" false
"logLevel" "info"
"logFormat" "json"
"writableCache" (dict
"tmpSizeLimit" "256Mi"
"nextCacheSizeLimit" "512Mi")
"oauth" (dict
"enabled" true
"only" true
"clientId" nil
"issuerUrl" nil
"scopes" nil
"endSession" true) -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" $given "path" "bulwark") -}}
{{- end -}}
{{/*
THE DATA DIRECTORY, defined once because two templates have to agree on it: the
StatefulSet mounts the PVC here and NOTES.txt names it.
/app/data, and it is NOT a value. Bulwark resolves its four directories
relative to the process working directory - getConfigDir() falls back to
<cwd>/data/admin and getStateDir() to <cwd>/data/admin-state (lib/admin/paths.ts
at tag 1.12.0) - and cwd is /app (the image's WORKDIR). A path that disagreed
with the mount would put the admin configuration and the audit log into the
container's ephemeral layer, which looks like it works until the pod is
replaced. The image creates all four subdirectories and chowns them to
1001:1001 (Dockerfile: mkdir -p /app/data/{settings,admin,admin-state,telemetry}
&& chown -R nextjs:nodejs /app/data) - but that happens in the IMAGE LAYER,
which the PVC mount covers, so the subdirectories are created again at runtime
by Bulwark itself (ensureConfigDir / ensureStateDir, both mkdir -p). fsGroup is
what makes the empty volume writable for uid 1001.
*/}}
{{- define "bulwark.dataDir" -}}/app/data{{- end -}}
{{/*
WHERE THE TWO FILE-FORM SECRETS ARE MOUNTED, and the one property of this mount
that must not be "tidied up": it is a WHOLE-DIRECTORY mount with NO subPath.
The kubelet never refreshes a subPath mount. Both files are re-read on every
use - getSessionSecret() calls readFileEnv on every call
(lib/auth/session-secret.ts) and getGlobalClientSecret() does the same on every
token exchange (lib/oauth/token-exchange.ts, no cache) - so rotating a field
inside the vault item takes effect WITHOUT a pod restart. With a subPath that
property is silently lost: the file in the container would keep the old value
forever while the vault, the Secret and `helm upgrade` all look correct.
*/}}
{{- define "bulwark.secretMountPath" -}}/etc/bulwark/secrets{{- end -}}
{{/*
The effective security contexts, probes and resources (issue #99).
The defaults below MUST stay in sync with values.yaml. They reproduce THIS
chart's defaults, not the repo standard.
*/}}
{{- define "bulwark.podSecurityContext" -}}
{{- $given := (fromYaml (include "bulwark.subBlock" (dict "block" . "key" "pod" "path" "bulwark.securityContext"))).value -}}
{{- $defaults := dict
"runAsNonRoot" true
"runAsUser" 1001
"runAsGroup" 1001
"fsGroup" 1001
"fsGroupChangePolicy" "Always"
"seccompProfile" (dict "type" "RuntimeDefault") -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" $given "path" "bulwark.securityContext.pod") -}}
{{- end -}}
{{- define "bulwark.containerSecurityContext" -}}
{{- $given := (fromYaml (include "bulwark.subBlock" (dict "block" . "key" "container" "path" "bulwark.securityContext"))).value -}}
{{- $defaults := dict
"allowPrivilegeEscalation" false
"readOnlyRootFilesystem" true
"runAsNonRoot" true
"capabilities" (dict "drop" (list "ALL") "add" (list)) -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" $given "path" "bulwark.securityContext.container") -}}
{{- end -}}
{{/*
ALL THREE PROBES ARE AN HTTP GET ON /api/health, AND WHAT THAT DOES AND DOES NOT
CARRY IS MEASURED AGAINST THE SOURCE AT TAG 1.12.0 - not read off the path name,
which is the mistake #218 and Gitea #83 each cost this repo once.
FIRST, WHY GET AND NOT HEAD, because the plan for this chart (Gitea #89) said
HEAD and HEAD IS NOT AVAILABLE TO A KUBERNETES PROBE AT ALL. The API type is
HTTPGetAction and it has no method field; the kubelet's prober issues a GET and
there is no value anywhere in a pod spec that changes that. The only way to send
a HEAD would be an exec probe shelling out to a HTTP client - and this image
contains no curl and no wget (npm and npx are deliberately removed from the
runtime layer), so it would mean `node -e` on every probe tick: a new Node
process every few seconds against an endpoint whose GET handler is already
cheap. That is strictly worse, so the chart uses the GET that Kubernetes
actually offers.
WHAT CHANGES BECAUSE OF THAT: almost nothing, and in the direction that helps.
Upstream's HEAD handler returns `new Response(null, { status: 200 })` and
computes nothing. The GET handler does gather diagnostics, which makes it very
slightly more capable than HEAD rather than less: its catch block answers 503
when `process.memoryUsage()` or `v8.getHeapStatistics()` itself throws. Every
limitation argued below is unchanged.
WHAT THE ENDPOINT DOES. The GET success path returns a hard `status: 200`
whatever state it computed, and for an UNAUTHENTICATED caller the body is
hard-coded to `healthy` as well (`status: detailed ? status : 'healthy'`);
`degraded` appears only with `?detailed=true` AND a caller that passes
canSeeInstanceDetails. Upstream tests exactly that -
lib/__tests__/health-route.test.ts expects HTTP 200 while `payload.status` is
`degraded`. A probe is unauthenticated and asks for no details, so it sees 200
and "healthy" always.
SO THE PROBE CANNOT REPORT THE APPLICATION'S STATE - and this is deliberately
NOT filed as "a probe that cannot fail", because that would be wrong and would
invite somebody to replace it with something worse. What a 200 here really
proves, all of it load-bearing: the Node process is alive, the HTTP server is
accepting, the event loop is not blocked, and Next.js can route a request. An
OOM kill, a crash, a wedged event loop and a container that never finished
booting all fail this probe. That is more than #218's endpoint carried and it is
exactly what a liveness and a startup probe are for.
WHAT IT DOES NOT CARRY, and no probe in this chart can: anything about
STALWART. Bulwark holds no mail and this endpoint does not ask the JMAP server
anything, so with the mail server gone /api/health is green and the pod stays
Ready. There is no better endpoint - /api/setup/test-jmap would really test the
connection, but it lives behind the bootstrap setup token and is unreachable in
this chart (the wizard is disabled by jmapServerUrl). The limit is written at
the readiness value in values.yaml because readiness is the probe whose promise
it weakens.
*/}}
{{- define "bulwark.livenessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/api/health" "port" "http" "scheme" "HTTP")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.livenessProbe") -}}
{{- end -}}
{{- define "bulwark.readinessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/api/health" "port" "http" "scheme" "HTTP")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.readinessProbe") -}}
{{- end -}}
{{/*
STARTUP: the same endpoint, and it gates the other two - which is why neither of
them carries an initialDelaySeconds.
BUDGET = periodSeconds x failureThreshold = 5 x 24 = 120s. What it has to cover
is not Bulwark's own boot, which is a Next.js standalone server starting in
about a second: it is attaching and mounting the network volume, and the
instrumentation hook that runs BEFORE the first request is served -
instrumentation.node.ts migrates the legacy admin layout, loads config.json and
policy.json off the volume, and on a first install hashes ADMIN_PASSWORD with
scrypt and writes admin.json. The scrypt hash is the one deliberately slow step
in there.
*/}}
{{- define "bulwark.startupProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/api/health" "port" "http" "scheme" "HTTP")
"periodSeconds" 5
"timeoutSeconds" 5
"failureThreshold" 24 -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.startupProbe") -}}
{{- end -}}
{{/*
The effective resources block, fed into the resources helper above so that an
erased "resources:" still yields the chart's requests and computed limits.
*/}}
{{- define "bulwark.effectiveResources" -}}
{{- $defaults := dict
"limitFactor" 3
"requests" (dict "cpu" "100m" "memory" "256Mi" "ephemeral-storage" "64Mi") -}}
{{- $merged := fromYaml (include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.resources")) -}}
{{- include "bulwark.resources" $merged -}}
{{- end -}}
{{/*
The additive env block, rendered through tpl with the value, secretKeyRef and
configMapKeyRef forms (guideline 10).
ADDITIVE MEANS ADDITIVE: an entry repeating a name this chart already sets does
not replace it. At install time both reach the pod and the later one wins; on an
EXISTING release the pair collapses to the chart's own value, because Kubernetes
merges env by name - and helm upgrade still reports success. So do not use this
block to override JMAP_SERVER_URL, OAUTH_ONLY or SESSION_SECRET_FILE; change the
values that produce them.
AND THERE IS A SECOND, SHARPER REASON NOT TO USE IT FOR BULWARK'S OWN SETTINGS,
specific to this application: an environment variable is only the MIDDLE layer
of three. configManager.get resolves "admin override > env var > default"
(lib/admin/config-manager.ts), so any key the admin dashboard has written into
config.json on the PVC beats whatever arrives here - silently. An entry added
through this block is therefore not a guarantee, it is a default that the volume
may already be overriding. values.yaml, under bulwark.adminConfigReadOnly, has
the whole precedence story.
*/}}
{{- define "bulwark.env" -}}
{{- $root := .root -}}
{{- range $value := .env }}
{{- if $value.value }}
- name: {{ $value.name | quote }}
# quote is required, not cosmetic: a value like "3000" or "true" renders
# unquoted as a YAML number/bool, and the API server then rejects the whole
# manifest with "cannot unmarshal number into Go struct field EnvVar.value of
# type string".
value: {{ tpl $value.value $root | quote }}
{{- else if $value.valueFrom }}
{{- if $value.valueFrom.secretKeyRef }}
- name: {{ $value.name }}
valueFrom:
secretKeyRef:
name: {{ tpl $value.valueFrom.secretKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.secretKeyRef.key $root | quote }}
{{- else if $value.valueFrom.configMapKeyRef }}
- name: {{ $value.name }}
valueFrom:
configMapKeyRef:
name: {{ tpl $value.valueFrom.configMapKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.configMapKeyRef.key $root | quote }}
{{- end }}
{{- end }}
{{- end }}
{{- end -}}
{{/*
The public base URL, derived from ingress.domain. NOTES.txt prints it and builds
the OIDC redirect URI from it, which is why it is a helper rather than several
copies of the same printf.
DELIBERATELY NO required() HERE, and that is the opposite of what it first looks
like it should have. The only caller is NOTES.txt, and `helm template` does not
render NOTES.txt (README, guideline 12.2) - so a required() in this helper would
be a guard that cannot fire in lint, in the render test or in CI, and would fire
for the first time during a real `helm install`. A guard that only runs in
production is worse than no guard.
The honest position for THIS chart, checked in both directions as guideline 12.1
asks: nothing outside the Ingress template FUNCTIONALLY reads ingress.domain.
Bulwark derives its own public URL from the request origin and takes no base-URL
variable, so with ingress.enabled: false the release is complete without a
domain - the calibre-web-automated case, where the gate frees both values. The
enforcement therefore sits entirely in bulwark.ingress.yaml, where it is reached
by every render that has the Ingress on (the default), and this helper renders a
visible placeholder instead of failing.
*/}}
{{- define "bulwark.publicUrl" -}}
{{- if .Values.ingress.domain -}}
https://{{ .Values.ingress.domain }}
{{- else -}}
https://<ingress.domain NOT SET>
{{- end -}}
{{- end -}}
-67
View File
@@ -1,67 +0,0 @@
{{- /*
ON BY DEFAULT (ingress.enabled: true): a webmail client nobody can reach is not
a webmail client.
AT THE ROOT OF THE HOST, AND THERE IS NO VALUE TO PUT IT UNDER A PATH.
NEXT_PUBLIC_BASE_PATH is compiled into the bundle - upstream: "These are
compiled into the bundle and cannot be changed at runtime" - so a sub-path
deployment needs a rebuilt image, which is outside what this repository does
(same limit as garage #81 and keycloak #83). A rewrite-target annotation is not a
way around it either: the bundle's own asset and API URLs are built for the root,
so stripping a prefix at the proxy produces a page that loads and then fetches
everything from the wrong place. The rule below is therefore "/" with pathType
Prefix, deliberately and permanently.
GUIDELINE 1, CHECKED IN BOTH DIRECTIONS (README, "Values that must not lie"),
AND FOR THIS CHART THE GATE FREES BOTH VALUES. Checked rather than assumed:
nothing outside this template FUNCTIONALLY reads ingress.domain. Bulwark derives
its own public URL from the request origin and takes no base-URL environment
variable - there is no KC_HOSTNAME equivalent here - so a release with
ingress.enabled: false is complete without a domain. That is the
calibre-web-automated case rather than the matrix-synapse one.
The one other READER is NOTES.txt, which prints the access URL and the OIDC
redirect URI. That read is deliberately NOT guarded with a required(): `helm
template` does not render NOTES.txt, so a guard there would fire for the first
time during a real `helm install` and never in lint, in the render test or in CI.
The bulwark.publicUrl helper renders a visible placeholder instead, and the
enforcement lives here - in the template that is reached by every render with the
Ingress on, which is the default.
*/ -}}
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-ingress
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
annotations:
cert-manager.io/cluster-issuer: {{ required "A Cluster-Issuer must be provided (ingress.clusterIssuer)" .Values.ingress.clusterIssuer }}
{{- /* Merged, not replaced. The one annotation worth adding here for this
application is nginx.ingress.kubernetes.io/proxy-body-size: attachment
uploads and the Files view go through this Ingress, and nginx's
default of 1m rejects anything larger with a 413 that reads like an
application fault. The chart does not set it because the right number
is the mail server's limit, which this chart does not know. */}}
{{- with .Values.ingress.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
ingressClassName: {{ .Values.ingress.className }}
rules:
- host: {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
http:
paths:
- backend:
service:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
port:
# BY NAME, so bulwark.port and this file cannot drift apart.
name: http
path: /
pathType: Prefix
tls:
- hosts:
- {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
secretName: tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
{{- end }}
-50
View File
@@ -1,50 +0,0 @@
{{- /*
ONE claim, mounted at /app/data, holding all FOUR of Bulwark's directories as
subdirectories of it:
admin config.json, policy.json, admin.json (the password hash),
plugin-config/, plugins/, themes/, branding uploads
admin-state admin-state.json (login timestamps), audit.log, the bootstrap
setup token
settings the encrypted per-account user settings
telemetry consent and instance id
ONE VOLUME AND NOT FOUR, because a snapshot is atomic per volume and never
across volumes - the durability lesson garage (#81) was built on. Split up, a
backup can capture an audit log that records logins the configuration does not
know about yet, or settings encrypted against a secret the configuration has
since been repointed from. There is no argument on the other side here: this is
a few hundred kilobytes of JSON plus an append-only log, not an object store.
Bulwark creates all four subdirectories itself on start (ensureConfigDir and
ensureStateDir, both mkdir -p). It has to: the image's own `mkdir -p
/app/data/...` ran in a layer that this mount covers. fsGroup is what makes the
freshly provisioned volume writable for uid 1001.
DELIBERATELY NOT null-SAFE: an erased size yields an invalid PVC that is
rejected at apply time, which is loud. A chart-side fallback would silently size
somebody's volume - and on an existing release it could not help anyway, because
a PVC cannot shrink. Same reasoning as garage's storage block and satisfactory's
documented exception (Gitea #39).
NEVER RENAMED. A renamed PVC makes Kubernetes provision a new, empty volume and
orphans the data - here that means admin.json, the only copy of the admin
password hash, and audit.log, which by its nature cannot be reconstructed.
*/ -}}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
accessModes:
# ReadWriteOnce, and the single replica follows from it rather than the other
# way round - see the bulwark.replicas helper.
- ReadWriteOnce
resources:
requests:
storage: {{ .Values.storage.amount }}
{{- if .Values.storage.storageClass }}
storageClassName: {{ .Values.storage.storageClass }}
{{- end }}
-109
View File
@@ -1,109 +0,0 @@
{{- /*
The 1Password items this chart materializes. TWO of them, split BY AUDIENCE
rather than by count:
<release>-bulwark-app-secret this release's own secrets - the session
secret and the admin password. Nobody else
knows or needs them.
<release>-bulwark-oauth-secret the OIDC client secret. NOT ours: the
identity provider (keycloak, #83) creates it
when the client is registered and rotates it
there, so it is the one of the three that a
second system also holds. A separate item is
what lets it be repointed and rotated without
touching the other two.
Same reasoning as garage's split between its RPC secret and its S3 key.
WHAT THE FIELD NAMES ARE AND WHY THEY ARE NOT FREE-FORM: the operator
materializes one Secret key per FIELD of the item, and this chart references
those keys by name - two as file paths in the StatefulSet's volumeMounts, one by
secretKeyRef. An item without the expected field leaves the container in
CreateContainerConfigError, or - worse for the file form - produces a container
that starts with a MISSING file, which readFileEnv answers with null rather than
an error. NOTES.txt lists every field for the installer.
TWO SECRETS, TWO DIFFERENT ROTATION STORIES, and the difference is a property of
the application rather than of this chart:
session-secret, oauth-client-secret FILE form. Re-read on every use with no
cache, so a rotation is live WITHOUT a
restart - see the whole-directory mount
in bulwark.sfs.yaml, which is what makes
that true.
admin-password CONSUMED ONCE. initAdminPassword hashes
it on the first start, writes admin.json
to the PVC, and never looks at the
variable again. Rotating the field after
that has NO effect, restart or not.
AND THE LIMIT THAT CANNOT BE CLOSED FOR EITHER: a change to the CONTENT of an
item restarts nothing, because Helm never sees a secret the 1Password operator
materializes and there is nothing to hash. What this chart does cover is the
op:// PATH, through checksum/secret-ref on the pod template. The operator's
auto-restart annotation is NOT set here and must not be: verified against
operator 1.12.0 it enumerates Deployments only, so on a StatefulSet it is a
silent no-op advertising a guarantee it does not provide (README, guideline 9).
*/ -}}
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-app-secret
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
# Must carry the fields: session-secret, admin-password
#
# session-secret: 32 bytes of randomness, base64 - `openssl rand -base64 32`.
# MANDATORY. Persistent "remember me" sessions and the encryption of the
# synchronized user settings both hang on it, and the second one fails
# SILENTLY without it: app/api/config/route.ts computes
# `settingsSyncEnabled: configManager.get(...) && hasSessionSecret()`, so
# bulwark.settingsSync: true plus a missing secret means synchronization is
# simply off, with no error anywhere.
#
# LOSING IT LOSES DATA THAT AN INTACT VOLUME STILL HOLDS: the settings on the
# PVC are encrypted with it. This item belongs in the backup next to the
# volume - a backup of the volume alone is not a backup.
#
# admin-password: the initial password of the ADMIN DASHBOARD.
# It is set because the dashboard is the way back out of an OIDC lockout
# (see bulwark.oauth.only in values.yaml), not because of any secret leak -
# measured, there is none: without ADMIN_PASSWORD Bulwark generates no
# random password and logs nothing, it just disables the dashboard.
#
# CONSUMED ON THE FIRST START ONLY. After admin.json exists on the PVC this
# field is never read again, so rotating it here changes nothing; the reset
# goes through the dashboard or by deleting admin.json.
itemPath: {{ required "You must provide an app secret reference (secrets.appSecretRef) - the 1Password item holding the fields 'session-secret' (openssl rand -base64 32; mandatory, the settings sync is silently disabled without it) and 'admin-password' (the admin dashboard, which is the only way back out of an OIDC lockout)" .Values.secrets.appSecretRef }}
{{- if $settings.oauth.enabled }}
---
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-oauth-secret
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
# Must carry the field: oauth-client-secret
#
# The confidential OIDC client's secret, as issued by the identity provider
# when the client is registered. The registration is a PRE-INSTALL step, not a
# follow-up: with bulwark.oauth.only (the default) and no working client, the
# first sign-in is impossible. NOTES.txt prints the redirect URI to register.
#
# It arrives as a FILE, through OAUTH_CLIENT_SECRET_FILE, so it is not in the
# container's process environment - which garage (#81) and keycloak (#83) could
# not achieve because neither application offered a file form. #81's other half
# was checked here and does not apply: lib/read-file-env.ts is readFileSync
# plus try/catch with no mode check, so the group-read bit fsGroup adds is
# harmless.
#
# AND THE ONE WAY A ROTATION SILENTLY DOES NOTHING: getGlobalClientSecret
# checks the ADMIN OVERRIDE FIRST. If the dashboard's OAuth setup has ever
# run, config.json on the PVC carries oauthClientSecret in PLAINTEXT and that
# value beats this file forever. That is also why a backup of the volume can
# contain this secret in the clear.
itemPath: {{ required "You must provide an OAuth client secret reference (secrets.oauthSecretRef) - the 1Password item holding the field 'oauth-client-secret', as issued by the identity provider when the OIDC client was registered. Required while bulwark.oauth.enabled is true; set it to false for a password-login install" .Values.secrets.oauthSecretRef }}
{{- end }}
-48
View File
@@ -1,48 +0,0 @@
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
selector:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
ports:
# The one port this application serves. targetPort addresses the container
# port BY NAME, so bulwark.port can be changed freely while renaming the port
# would break this and all three probes at once.
- protocol: TCP
port: {{ $settings.port | int64 }}
targetPort: http
name: http
# ClusterIP, and no value to change it. The way out of the cluster is the
# Ingress with a certificate: this is a webmail client that handles session
# cookies and mail content, so a NodePort would put it on a routable address
# over plain HTTP by accident.
type: ClusterIP
---
{{/*
The headless Service a StatefulSet needs for its serviceName. It exists because
a StatefulSet requires one, not because anything addresses the pod by its stable
DNS name - with one replica and no peer coordination there is nothing to address.
Kept separate from the ClusterIP Service above so that the latter stays a plain
load-balanced endpoint for the Ingress; pointing serviceName at a Service with a
ClusterIP is accepted by the API server but gives the pod no per-pod DNS record,
which is the kind of half-working arrangement that only shows up later.
*/}}
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-headless
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
clusterIP: None
selector:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
ports:
- protocol: TCP
port: {{ $settings.port | int64 }}
targetPort: http
name: http
-329
View File
@@ -1,329 +0,0 @@
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
{{- /*
A StatefulSet and not a Deployment, and the reason is the single ReadWriteOnce
PVC rather than any cluster identity: a rolling Deployment update blocks itself,
because the new pod waits for a volume the old pod still holds. The repo's
convention requires a StatefulSet for a stateful application anyway, and this one
is stateful - /app/data carries the admin configuration, the admin password hash,
the audit log and the encrypted user settings.
NO volumeClaimTemplates: this repo uses standalone PVC templates throughout, so
the StatefulSet can be renamed in a major PR with a --cascade=orphan note without
the data following the name (README, guideline 1).
NO INIT CONTAINER, and that is checked rather than skipped. Bulwark creates its
four data subdirectories itself on start (ensureConfigDir / ensureStateDir, both
mkdir -p), which it has to because the image's own mkdir ran in a layer this
mount covers; fsGroup makes the fresh volume writable for uid 1001. There is
nothing to seed and nothing to chown, so an init container would only be a second
image to pin and update.
GUARD AT RENDER TIME, not at start: a missing OIDC client id or issuer with
oauth.enabled would otherwise produce a release that installs cleanly, becomes
Ready, and then answers every sign-in attempt with a broken redirect - and with
oauth.only there is no password form to fall back on. The render fails instead,
naming the value. It lives HERE rather than in NOTES.txt because `helm template`
does not render NOTES.txt (guideline 12.2), so a guard there would pass every
check in CI while being absent.
*/ -}}
{{- if $settings.oauth.enabled }}
{{- if not $settings.oauth.clientId }}
{{- fail "bulwark.oauth.clientId is required while bulwark.oauth.enabled is true - without it the OIDC redirect is built against an empty client and every sign-in fails. With bulwark.oauth.only (the default) there is no password form to fall back to, so this would be an install nobody can log into. Register the client with the identity provider first; NOTES.txt prints the redirect URI." }}
{{- end }}
{{- if not $settings.oauth.issuerUrl }}
{{- fail "bulwark.oauth.issuerUrl is required while bulwark.oauth.enabled is true - Bulwark fetches the OIDC discovery document from it, and without it there is nothing to discover. For keycloak this is https://<host>/realms/<realm>." }}
{{- end }}
{{- end }}
{{- if not $settings.jmapServerUrl }}
{{- /* Not merely "the mail server is unset": an empty JMAP_SERVER_URL puts
detectSetupState() into "bootstrap" (lib/setup/state.ts), which serves the
FIRST-RUN WIZARD at /setup and 302s everything else there. The wizard then
writes its result into config.json on the PVC - and from that moment the
volume beats every value in this chart, permanently and silently. So an
unset value here does not produce a non-working release, it produces a
SILENTLY DIFFERENT ONE, which is worse. */}}
{{- fail "bulwark.jmapServerUrl is required - it is the address of the Stalwart server AND the switch that disables the first-run setup wizard. Left empty, Bulwark starts in bootstrap mode, serves its wizard at /setup, and whatever is clicked through there lands in config.json on the PVC - which from then on OVERRIDES every value this chart sets, silently. Set it to the stalwart release's JMAP endpoint." }}
{{- end }}
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
replicas: {{ include "bulwark.replicas" (dict "scaleDown" .Values.scaleDown "replicas" $settings.replicas) }}
serviceName: {{ .Release.Name }}-{{ .Chart.Name }}-headless
selector:
matchLabels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
template:
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
annotations:
# NO checksum/config ANNOTATION, AND THAT IS NOT AN OMISSION (#82).
# This chart renders no ConfigMap: Bulwark is configured through
# environment variables plus the two JSON files it writes itself onto the
# PVC. There is no rendered config file whose change could fail to
# restart the workload, so the rule has nothing to attach to. A changed
# environment variable changes the pod template directly and rolls the
# pod, which is what the annotation exists to achieve elsewhere.
#
# Hash of the rendered OnePasswordItems - the op:// PATHS, not their
# contents. Repointing secrets.appSecretRef or secrets.oauthSecretRef at
# a different vault item therefore rolls the pod instead of leaving the
# old credentials in the running process. A change INSIDE an item is not
# covered and cannot be, because Helm never sees it - but for the two
# FILE-form secrets it does not need to be: both are re-read on every use
# (see the secrets volume below), so a rotation is live without a
# restart. The exception is admin-password, which is consumed exactly
# once on the first start and which no restart brings back.
checksum/secret-ref: {{ include (print $.Template.BasePath "/bulwark.secret.yaml") . | sha256sum }}
spec:
automountServiceAccountToken: false
securityContext:
{{- include "bulwark.podSecurityContext" $.Values.bulwark.securityContext | nindent 8 }}
containers:
- name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
# Fully qualified. The suffix-free tag family, see Chart.yaml - this
# registry publishes every release twice (1.12.0 and v1.12.0) and the
# chart pins the form without the "v".
image: "ghcr.io/bulwarkmail/webmail:{{ .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
# No command and no args: the image's CMD is ["node", "server.js"] with
# WORKDIR /app, and the working directory is load-bearing here rather
# than incidental - Bulwark resolves its data directories relative to
# process.cwd(), so overriding either would move /app/data out from
# under the volume mount.
env:
# THE MAIL SERVER, AND THE SWITCH THAT KILLS THE SETUP WIZARD. With
# this set, detectSetupState() returns "env-managed" and /setup is
# not served at all, which is what makes this chart declarative.
#
# IT IS STILL BEATEN BY A SERVER URL SAVED IN THE ADMIN DASHBOARD -
# the one precedence trap upstream documents itself. See values.yaml.
- name: JMAP_SERVER_URL
value: {{ $settings.jmapServerUrl | quote }}
{{- with $settings.appName }}
- name: APP_NAME
value: {{ . | quote }}
{{- end }}
- name: STALWART_FEATURES
value: {{ $settings.stalwartFeatures | quote }}
- name: STALWART_ADMIN_ACCESS
value: {{ $settings.stalwartAdminAccess | quote }}
# The settings sync. Note that this flag alone is not enough: without
# a session secret, app/api/config/route.ts turns it off SILENTLY
# (`configManager.get(...) && hasSessionSecret()`). The secret below
# is therefore mandatory rather than recommended.
- name: SETTINGS_SYNC_ENABLED
value: {{ $settings.settingsSync | quote }}
# THE SESSION SECRET, AS A FILE PATH. Everything about why it is the
# file form and why the mount has no subPath is at the volume below.
#
# This is also ONE OF ONLY TWO VALUES IN THIS CHART THAT THE VOLUME
# CANNOT OVERRIDE: getSessionSecret (lib/auth/session-secret.ts) reads
# SESSION_SECRET, then SESSION_SECRET_FILE, and only then the admin
# override - the reverse of configManager.get's normal order.
- name: SESSION_SECRET_FILE
value: {{ include "bulwark.secretMountPath" . }}/session-secret
# THE ADMIN PASSWORD, AS AN ENVIRONMENT VARIABLE, AND THAT IS FORCED
# RATHER THAN CHOSEN: ADMIN_PASSWORD has no _FILE form in
# CONFIG_ENV_MAP, unlike the other two secrets here. The residual
# price, stated rather than hidden: it is in the container's process
# environment, so anyone who can exec into the pod or read its /proc
# can see it until the first start has hashed it into admin.json.
#
# WHY IT IS SET AT ALL: the admin dashboard is the only way back out
# of an OIDC lockout (OAUTH_ONLY below), and without this variable
# the dashboard is simply disabled - there is no random password and
# nothing in the log, measured in lib/admin/password.ts.
- name: ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-app-secret
key: admin-password
{{- if $settings.adminConfigReadOnly }}
# THE CHART IS AUTHORITATIVE FROM HERE ON - and this is the one value
# whose price is bigger than its name. It freezes the settings policy
# (and with it the only way to move Web Push off the vendor relay),
# it freezes an override that ALREADY exists instead of removing it,
# and on a FRESH install it prevents admin.json from being written at
# all - which takes the admin dashboard, i.e. the rescue door, away
# entirely while the pod stays Ready. values.yaml has the measurement
# and the only safe sequence.
- name: ADMIN_CONFIG_READONLY
value: "true"
{{- end }}
{{- if $settings.oauth.enabled }}
- name: OAUTH_ENABLED
value: "true"
# HIDES THE PASSWORD FORM. The way back out is NOT a helm upgrade
# with this false - oauthOnly is read through configManager.get by
# every consumer, so an override in config.json on the PVC wins. The
# repair is removing that override in the admin dashboard. NOTES.txt
# carries it.
- name: OAUTH_ONLY
value: {{ $settings.oauth.only | quote }}
- name: OAUTH_CLIENT_ID
value: {{ $settings.oauth.clientId | quote }}
- name: OAUTH_ISSUER_URL
value: {{ $settings.oauth.issuerUrl | quote }}
# The client secret as a FILE, so it stays out of the process
# environment - the better path garage (#81) and keycloak (#83) did
# not have. Re-read on every token exchange, so a rotation needs no
# restart; see the volume below for the subPath condition that makes
# that true.
- name: OAUTH_CLIENT_SECRET_FILE
value: {{ include "bulwark.secretMountPath" . }}/oauth-client-secret
{{- with $settings.oauth.scopes }}
- name: OAUTH_SCOPES
value: {{ . | quote }}
{{- end }}
- name: OAUTH_END_SESSION
value: {{ $settings.oauth.endSession | quote }}
{{- end }}
# Both OFF against the image's defaults, and deliberately: a mail
# client should not phone home from a private cluster unless somebody
# chose it, and Renovate already watches the image tag, so the
# version check would answer outbound a question already answered
# inbound.
- name: BULWARK_TELEMETRY
value: {{ ternary "on" "off" $settings.telemetry | quote }}
- name: BULWARK_UPDATE_CHECK
value: {{ ternary "on" "off" $settings.updateCheck | quote }}
- name: LOG_LEVEL
value: {{ $settings.logLevel | quote }}
- name: LOG_FORMAT
value: {{ $settings.logFormat | quote }}
# PORT and HOSTNAME are already set in the image (3000 / 0.0.0.0);
# PORT is repeated here only because bulwark.port makes it
# configurable, and the containerPort below has to agree with it.
- name: PORT
value: {{ $settings.port | quote }}
{{- with $.Values.bulwark.env }}
{{- include "bulwark.env" (dict "root" $ "env" .) | nindent 12 }}
{{- end }}
ports:
# The name "http" is what the Service and all three probes reference,
# so renaming it here breaks four things at once.
- name: http
containerPort: {{ $settings.port | int64 }}
protocol: TCP
# Probes, resources and both security contexts go through the null-safe
# helpers (#99): an erased block means "chart defaults apply" instead of
# a vanished probe or an unhardened container. Never wrap these in an
# "if" - a guard hides the erased case completely instead of rendering a
# visible null.
#
# All three are an HTTP GET on /api/health - not HEAD, because a
# Kubernetes probe cannot send one (HTTPGetAction has no method field).
# What that endpoint can and cannot report is argued at the helpers and
# at bulwark.readinessProbe in values.yaml; the short version is that it
# carries the process, HTTP acceptance, a free event loop and Next.js
# routing - and NOTHING about the mail server.
livenessProbe:
{{- include "bulwark.livenessProbe" $.Values.bulwark.livenessProbe | nindent 12 }}
readinessProbe:
{{- include "bulwark.readinessProbe" $.Values.bulwark.readinessProbe | nindent 12 }}
startupProbe:
{{- include "bulwark.startupProbe" $.Values.bulwark.startupProbe | nindent 12 }}
resources:
{{- include "bulwark.effectiveResources" $.Values.bulwark.resources | nindent 12 }}
securityContext:
{{- include "bulwark.containerSecurityContext" $.Values.bulwark.securityContext | nindent 12 }}
volumeMounts:
# THE ONE PVC, carrying all four of Bulwark's directories as
# subdirectories so that a volume snapshot catches configuration,
# runtime state, audit log and user settings atomically.
- name: data
mountPath: {{ include "bulwark.dataDir" . }}
# THE TWO FILE-FORM SECRETS, AS A WHOLE DIRECTORY AND NOT WITH
# subPath. This is the line a later round will want to "tidy up" into
# two subPath mounts, and that would silently break secret rotation:
# the kubelet never refreshes a subPath mount, while both values are
# re-read from disk on every use (getSessionSecret and
# getGlobalClientSecret both call readFileEnv with no cache). With
# this mount a rotated vault field is live without a restart; with
# subPath the container would hold the first value forever and
# nothing would report it.
- name: secrets
mountPath: {{ include "bulwark.secretMountPath" . }}
readOnly: true
# The two writable paths outside the volume, needed because
# readOnlyRootFilesystem is true. Both are CACHES, not state - see
# bulwark.writableCache in values.yaml for how the set was
# established and why neither belongs on the PVC.
- name: tmp
mountPath: /tmp
- name: next-cache
mountPath: /app/.next/cache
restartPolicy: Always
volumes:
- name: data
persistentVolumeClaim:
claimName: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
- name: secrets
# One projected volume over the two Secrets the 1Password operator
# materializes, so both files sit in one directory and the container
# needs one mount instead of two.
#
# defaultMode 0440 - owner and GROUP read, and the group bit is the
# part that must not be "tightened" to 0400. Files in a secret volume
# are owned by ROOT, not by the container's user; what gives uid 1001
# access is fsGroup, which the kubelet applies by chowning the files to
# group 1001. Without a group read bit, uid 1001 could not read either
# file - and the failure would be silent, because readFileEnv answers
# an EACCES with null exactly as it answers a missing file: the
# settings sync would switch itself off and every token exchange would
# send an empty client secret.
#
# (The kubelet ORs 0440 onto the mode of a secret volume once fsGroup
# is set, so 0400 would in practice be corrected to 0440 anyway. It is
# written out because a mode that only works because something else
# silently fixes it is not a mode anyone should have to re-derive.)
#
# #81's permission trap does not apply in the other direction either:
# lib/read-file-env.ts is readFileSync plus try/catch with no mode
# check, so unlike garage, Bulwark does not REJECT a group-readable
# secret file. That is what makes the file form usable here at all.
projected:
defaultMode: 0440
sources:
- secret:
name: {{ .Release.Name }}-{{ .Chart.Name }}-app-secret
items:
# Only the session secret becomes a file. admin-password is
# in the same Secret but arrives as an environment variable
# (it has no _FILE form), so projecting it here as well would
# put it on disk for no reader.
- key: session-secret
path: session-secret
# optional: the 1Password operator fills this Secret in-cluster
# AFTER Helm has applied the manifests, so on a first install
# the Secret does not exist yet. Without this the pod would sit
# in ContainerCreating until the operator caught up, and with a
# misconfigured operator it would sit there forever with the
# reason only in the pod's events. It starts instead, and the
# missing secret surfaces where it is actionable: the settings
# sync stays off and the log says so.
optional: true
{{- if $settings.oauth.enabled }}
- secret:
name: {{ .Release.Name }}-{{ .Chart.Name }}-oauth-secret
items:
- key: oauth-client-secret
path: oauth-client-secret
optional: true
{{- end }}
- name: tmp
emptyDir:
sizeLimit: {{ $settings.writableCache.tmpSizeLimit }}
# Next.js's own cache directory in standalone mode - the optimized-image
# cache and the fetch cache. It is a SUBDIRECTORY of .next, which is what
# makes this possible at all: /app/.next/static holds the build output and
# has to stay readable, and mounting the cache path does not shadow it.
- name: next-cache
emptyDir:
sizeLimit: {{ $settings.writableCache.nextCacheSizeLimit }}
-293
View File
@@ -1,293 +0,0 @@
{
"$schema": "https://json-schema.org/draft-07/schema#",
"title": "bulwark values",
"description": "Strict values schema (issue #68). additionalProperties is false on every chart-owned object level, so a key this chart does not read is rejected at render time instead of being silently ignored; the objects that go verbatim into the Kubernetes pod spec (probes, security contexts, global, free-form annotation maps) stay open, because the chart does not interpret their contents. The blocks the chart dereferences additionally accept null, because an erased block means 'chart defaults apply' (issue #99) rather than a vanished setting. Note what this schema CANNOT enforce: secrets.appSecretRef, bulwark.jmapServerUrl and the two OIDC values all ship as null and JSON Schema's 'required' tests presence, not null-ness, so a 'required' array would never fire - the enforcement is required() in bulwark.secret.yaml and fail() in bulwark.sfs.yaml (issue #18). And note what no schema can express at all for this chart: an admin override in config.json on the PVC beats every value below, silently (values.yaml, point 1).",
"type": "object",
"additionalProperties": false,
"properties": {
"global": {
"description": "Helm's conventional cross-chart values. Unused by this chart, kept open so an umbrella install cannot break on it.",
"type": "object"
},
"scaleDown": {
"description": "true renders the StatefulSet with replicas: 0 and takes precedence over bulwark.replicas. The PVC, Secrets, Service and Ingress stay in place.",
"type": "boolean"
},
"secrets": {
"type": "object",
"additionalProperties": false,
"properties": {
"appSecretRef": {
"description": "op:// item path. The item must carry the fields 'session-secret' and 'admin-password'. Required always. The session secret is mandatory rather than recommended: without it the settings sync is switched off SILENTLY, and the encrypted settings already on the volume become unreadable. The admin password is consumed exactly once, on the first start.",
"type": ["string", "null"]
},
"oauthSecretRef": {
"description": "op:// item path. The item must carry the field 'oauth-client-secret', as issued by the identity provider when the OIDC client was registered. Required while bulwark.oauth.enabled is true.",
"type": ["string", "null"]
}
}
},
"bulwark": {
"type": "object",
"additionalProperties": false,
"properties": {
"replicas": {
"description": "0 or 1. Capped at 1 because of the single ReadWriteOnce PVC, not because of the application: two pods cannot both mount it, so a second replica would sit in ContainerCreating forever. An RWX volume would not help either - nothing coordinates two processes over that directory, and the admin config is read into memory once at start.",
"type": "integer",
"minimum": 0,
"maximum": 1
},
"port": {
"$ref": "#/definitions/port"
},
"jmapServerUrl": {
"description": "The Stalwart JMAP endpoint. Required - and it does two things: it is the mail server's address AND the switch that disables the first-run setup wizard. Left empty, Bulwark starts in bootstrap mode and whatever is clicked through its wizard lands in config.json on the PVC, which then overrides every value here. Enforced by fail() in the StatefulSet, not by this schema (see the title).",
"type": ["string", "null"]
},
"appName": {
"description": "APP_NAME - the product name in the UI and the PWA. null leaves upstream's 'Webmail'.",
"type": ["string", "null"]
},
"stalwartFeatures": {
"description": "STALWART_FEATURES - the Stalwart-specific UI parts (password change, Sieve rules). Does not gate the server-side credential passthrough, which has its own upstream switch this chart does not render.",
"type": "boolean"
},
"stalwartAdminAccess": {
"description": "STALWART_ADMIN_ACCESS - who may open the admin dashboard, which after bulwark.oauth.only is the only way back into a broken login path. 'auto' (upstream's default, kept here) signs a Stalwart admin in without the Bulwark admin password; 'password' requires it; 'off' ignores Stalwart admin status entirely. The consequence of 'auto' is at the value in values.yaml: a Stalwart admin can write the config overrides that disconnect this chart from its own values.",
"type": "string",
"enum": ["auto", "password", "off"]
},
"settingsSync": {
"description": "SETTINGS_SYNC_ENABLED - per-account preferences follow the user instead of living in one browser. true means /app/data/settings on the PVC carries USER DATA, encrypted with the session secret. Without that secret this switch is ignored, silently.",
"type": "boolean"
},
"adminConfigReadOnly": {
"description": "ADMIN_CONFIG_READONLY - refuse every write to the admin config directory at the application layer, which makes this file authoritative and closes the precedence trap for good. FALSE by default because the price is threefold and measured: it also freezes the settings policy (and with it the only way to move Web Push off the vendor relay); it freezes an override that ALREADY exists rather than removing it, because removeAdminOverride is guarded by the same check; and on a FRESH install it prevents admin.json from being written at all, so the admin dashboard - the rescue door - never comes up and its login endpoint answers HTTP 500 while the pod stays Ready. values.yaml has the only safe sequence.",
"type": "boolean"
},
"telemetry": {
"description": "BULWARK_TELEMETRY. false. Upstream's default is already off (telemetry is opt-in), so what this buys is the LOCK: the environment beats the state file and upstream blocks the admin UI from changing it while the override is active.",
"type": "boolean"
},
"updateCheck": {
"description": "BULWARK_UPDATE_CHECK - the hourly version check. Unlike telemetry this is ON unless switched off, so false does change behaviour. Off because Renovate already watches the image tag inbound.",
"type": "boolean"
},
"logLevel": {
"type": "string",
"enum": ["error", "warn", "info", "debug"]
},
"logFormat": {
"description": "text or json. json here, against upstream's default of text, because the logs go to a cluster log collector rather than to a terminal.",
"type": "string",
"enum": ["text", "json"]
},
"oauth": {
"description": "May be null / empty: the chart then falls back to its defaults (OIDC on, password form hidden). That fallback is deliberate and is the safe direction - an erased block must not silently turn the whole sign-in path off on a release whose login is OIDC.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"enabled": {
"description": "OAUTH_ENABLED. Gates the OIDC values, the OAuth OnePasswordItem and the secret file mount.",
"type": "boolean"
},
"only": {
"description": "OAUTH_ONLY - hide the password form, so the identity provider is the only way in for a user. This chart's riskiest default, taken deliberately. The way back out of a lockout is NOT a helm upgrade with this false: oauthOnly is read through configManager.get, so an override in config.json on the PVC wins. The repair is removing that override in the admin dashboard.",
"type": "boolean"
},
"clientId": {
"description": "OAUTH_CLIENT_ID. Required while enabled - enforced by fail() in the StatefulSet.",
"type": ["string", "null"]
},
"issuerUrl": {
"description": "OAUTH_ISSUER_URL, the issuer Bulwark fetches the discovery document from (keycloak: https://<host>/realms/<realm>). Required while enabled - enforced by fail() in the StatefulSet. The endpoints it advertises must share the issuer's origin or be public, or Bulwark's discovery validator rejects them.",
"type": ["string", "null"]
},
"scopes": {
"description": "OAUTH_SCOPES - REPLACES the scope list rather than adding to it. null leaves upstream's default, which is the right starting point: an incomplete list here is a login that succeeds and then has no identity to work with. Use OAUTH_EXTRA_SCOPES through bulwark.env to add.",
"type": ["string", "null"]
},
"endSession": {
"description": "OAUTH_END_SESSION - signing out also ends the provider's session. true (upstream's default): on a shared browser a sign-out that leaves the provider signed in means the next click is straight back in.",
"type": "boolean"
}
}
},
"writableCache": {
"description": "The sizeLimit of the two emptyDirs that readOnlyRootFilesystem: true makes necessary - /tmp and Next.js's own cache at /app/.next/cache. Both are CACHES, not state: losing them on a restart costs one round of image re-optimization. They belong here rather than on the PVC, because a cache on a ReadWriteOnce network volume is slower than regenerating it and would grow the only volume that has to be backed up. These limits and the pod's ephemeral-storage request under bulwark.resources are two halves of one accounting and should move together.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"tmpSizeLimit": {
"type": ["string", "null"]
},
"nextCacheSizeLimit": {
"type": ["string", "null"]
}
}
},
"resources": {
"$ref": "#/definitions/resources"
},
"livenessProbe": {
"$ref": "#/definitions/probe"
},
"readinessProbe": {
"$ref": "#/definitions/probe"
},
"startupProbe": {
"$ref": "#/definitions/probe"
},
"securityContext": {
"$ref": "#/definitions/securityContext"
},
"env": {
"$ref": "#/definitions/env"
}
}
},
"storage": {
"description": "The single PVC at /app/data, carrying all four of Bulwark's directories as subdirectories - one volume, because a snapshot is atomic per volume and never across volumes (the lesson of Gitea #81). Deliberately NOT null-safe: an erased size yields an invalid PVC, which fails loudly at apply time. Issue #99 covers only the silent class, and on an existing release a chart-side fallback could not help anyway, because a PVC cannot shrink.",
"type": "object",
"additionalProperties": false,
"properties": {
"amount": {
"description": "What actually grows here is audit.log, and it grows with admin activity rather than with mail volume - the application does not rotate it.",
"type": ["string", "null"]
},
"storageClass": {
"description": "null uses the cluster's default StorageClass. ReadWriteOnce is enough and is what the single replica needs.",
"type": ["string", "null"]
}
}
},
"ingress": {
"type": "object",
"additionalProperties": false,
"properties": {
"enabled": {
"description": "Gates the whole Ingress template (issue #93). Checked in both directions for this chart: nothing outside that template FUNCTIONALLY reads ingress.domain - Bulwark derives its own public URL from the request origin and takes no base-URL variable - so switching the Ingress off frees domain as well as clusterIssuer. The only other reader is NOTES.txt, which renders a visible placeholder rather than carrying a required() that helm template could never reach.",
"type": "boolean"
},
"clusterIssuer": {
"type": ["string", "null"]
},
"domain": {
"description": "The public hostname. Bulwark is served at the ROOT of it and cannot be served under a path: NEXT_PUBLIC_BASE_PATH is compiled into the bundle, and a rewrite-target annotation does not help because the bundle's own asset URLs are built for the root.",
"type": ["string", "null"]
},
"className": {
"type": ["string", "null"]
},
"annotations": {
"description": "Extra Ingress annotations, merged with the cert-manager annotation the chart always sets. Free-form keys by nature, so this object stays open. The one worth adding for this application is nginx.ingress.kubernetes.io/proxy-body-size: attachment uploads go through this Ingress and nginx's default of 1m rejects anything larger with a 413 that reads like an application fault.",
"type": "object"
}
}
}
},
"definitions": {
"port": {
"type": ["integer", "null"],
"minimum": 1,
"maximum": 65535
},
"resources": {
"description": "May be null / empty: the chart then falls back to its default requests and computed limits (issue #99) instead of rendering a container without resource accounting. One thing specific to this chart: the memory LIMIT is what V8's heap limit is derived from, and /api/health reports 'degraded' once heap usage crosses its threshold - a report the probes cannot see, so a limit set too low produces a pod that is permanently degraded and permanently Ready.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"limitFactor": {
"type": "number",
"exclusiveMinimum": 0
},
"requests": {
"$ref": "#/definitions/resourceList"
},
"limits": {
"$ref": "#/definitions/resourceList"
}
}
},
"resourceList": {
"description": "Both spellings of the ephemeral-storage key are accepted because the resources helper normalizes camelCase to the Kubernetes key.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"cpu": {
"type": ["string", "number", "null"]
},
"memory": {
"type": ["string", "number", "null"]
},
"ephemeral-storage": {
"type": ["string", "number", "null"]
},
"ephemeralStorage": {
"type": ["string", "number", "null"]
}
}
},
"env": {
"description": "Additive container env entries. An entry repeating a name the chart already sets does not replace it - on an existing release the pair collapses to the chart's own value while helm upgrade reports success. And a second caveat specific to this application: an environment variable is only the MIDDLE of three layers, so anything set here is a default that an admin override in config.json on the PVC may already be beating, silently.",
"type": ["array", "null"],
"items": {
"type": "object",
"additionalProperties": false,
"required": ["name"],
"properties": {
"name": {
"type": "string"
},
"value": {
"type": "string"
},
"valueFrom": {
"type": "object",
"additionalProperties": false,
"properties": {
"secretKeyRef": {
"$ref": "#/definitions/keyRef"
},
"configMapKeyRef": {
"$ref": "#/definitions/keyRef"
}
}
}
}
}
},
"keyRef": {
"type": "object",
"additionalProperties": false,
"required": ["name", "key"],
"properties": {
"name": {
"type": "string"
},
"key": {
"type": "string"
}
}
},
"securityContext": {
"description": "May be null / empty at any level: the chart then falls back to its hardening defaults (issue #99). This chart's fallback reproduces ITS defaults, deviations included - uid/gid/fsGroup 1001 rather than this repo's 20000, because that is the user the image creates (addgroup --gid 1001 nodejs / adduser --uid 1001 nextjs) and the owner of /app/data in the image layer. fsGroupChangePolicy is the repo's 'Always' and is correct here rather than merely default: /app/data holds a few JSON files and an append-only log, so the recursive chown is cheap, and it is what makes a freshly provisioned volume writable on the first start.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"pod": {
"description": "Merged onto the chart's pod hardening defaults and rendered as the pod's securityContext; kept open because the chart does not interpret the individual fields.",
"type": ["object", "null"]
},
"container": {
"description": "Merged onto the chart's container hardening defaults and rendered as the container's securityContext; kept open because the chart does not interpret the individual fields.",
"type": ["object", "null"]
}
}
},
"probe": {
"description": "Merged onto the chart's probe defaults and rendered as a container probe; kept open because the chart does not interpret the individual fields. May be null / empty, which means the defaults apply (issue #99) rather than the probe disappearing. All three defaults carry the same CONTEXT: an HTTP GET on /api/health, addressed at the port NAME 'http'. Not HEAD, although Bulwark offers one and it would be cheaper - a Kubernetes HTTPGetAction has no method field and the kubelet always issues a GET, and the only alternative (an exec probe) would mean a Node process every few seconds in an image with no curl or wget. What that proves and what it does not is measured rather than read off the path - it carries the process, HTTP acceptance, a free event loop and Next.js routing, and it carries NOTHING about the reported application state (the handler returns a hard 200 and hard-codes the body to 'healthy' for an unauthenticated caller) and NOTHING about Stalwart (with the mail server gone the probe is green and the pod stays Ready). There is no better endpoint: /api/setup/test-jmap would really test the connection but sits behind the bootstrap setup token, which this chart disables.",
"type": ["object", "null"]
}
}
}
-737
View File
@@ -1,737 +0,0 @@
# Bulwark Webmail - a JMAP webmail client (mail, calendar, contacts, files) for
# a Stalwart mail server. It is a CLIENT: it holds no mail, has no database, no
# cache and no broker. The messages, the accounts and SMTP are stalwart's
# (Gitea #65), reached over the network by bulwark.jmapServerUrl.
#
# READ THIS FIRST. Four properties shape every value below, and not one of them
# is visible from the outside (Gitea #89). Everything here was measured against
# the source at tag 1.12.0, not taken from the documentation - the documentation
# was wrong about two of them.
#
# 1. THE VOLUME BEATS THE CHART, FOR EVERY VALUE, SILENTLY.
# This is the single most important sentence in this file.
# lib/admin/config-manager.ts resolves a config value in this order:
#
# admin override (config.json on the PVC) > environment variable > default
#
# Every setting this chart renders as an environment variable is therefore a
# DEFAULT that the admin dashboard may already be overriding. The override is
# a key in config.json on the PVC, the dashboard may write ANY key of
# CONFIG_ENV_MAP (app/api/admin/config/route.ts validates against exactly
# that key list), and nothing in the dashboard says that it has just
# disconnected the chart. A `helm upgrade` with a changed value then changes
# nothing, reports success, and leaves no trace of why.
#
# The documentation names this only for the server URL ("A server URL saved
# by the wizard takes priority over the environment variable"). It is not
# special to the server URL. It is the behaviour of every key.
#
# THERE ARE EXACTLY THREE EXCEPTIONS, all verified, and they are the only
# values in this chart that the volume cannot silently beat:
# - SESSION_SECRET / SESSION_SECRET_FILE: getSessionSecret()
# (lib/auth/session-secret.ts) reads the environment FIRST and the admin
# override last - the reverse of the normal order.
# - ADMIN_PASSWORD: initAdminPassword() reads the environment only. (It is
# still consumed exactly once - see secrets.appSecretRef.)
# - BULWARK_TELEMETRY: effectiveConsent() takes the environment over the
# state file, and upstream blocks the admin UI from changing it while the
# override is set.
# The OAuth client secret is NOT an exception, which is the one most likely to
# be assumed - see secrets.oauthSecretRef.
#
# The way out, if you want the chart to be authoritative, is
# bulwark.adminConfigReadOnly - and it has a price that is bigger than it
# looks. Read it there before setting it.
#
# 2. THERE IS NO ConfigMap IN THIS CHART, AND THAT IS NOT AN OMISSION.
# Bulwark is configured entirely through environment variables plus the two
# JSON files it writes itself onto the PVC (config.json, policy.json). No
# config file is rendered, mounted or hashed here, so the repo's
# checksum/config rule (#82) has nothing to attach to: there is no ConfigMap
# whose change could fail to restart the workload. What IS covered is the
# op:// PATHS of the vault items, through checksum/secret-ref on the pod
# template, so repointing a secrets.* value really does roll the pod. A
# change INSIDE an item is not covered and cannot be - Helm never sees it -
# and for the two FILE-form secrets it does not need to be: both are re-read
# on every use, so a rotation takes effect without a restart (point 3).
#
# 3. THE TWO FILE-FORM SECRETS ARE MOUNTED WITHOUT subPath, ON PURPOSE.
# SESSION_SECRET_FILE and OAUTH_CLIENT_SECRET_FILE both exist (upstream calls
# them the file form of SESSION_SECRET and OAUTH_CLIENT_SECRET), and taking
# them keeps both values out of the container's process environment, which is
# what garage (#81) and keycloak (#83) could not do because neither
# application offered a file form at all. Here the file form is also
# ROTATABLE: readFileEnv() is called on every use with no cache, so a changed
# field in the vault item takes effect without a pod restart - but only
# because the Secret is mounted as a whole directory. The kubelet never
# refreshes a subPath mount, so a subPath here would silently freeze both
# secrets at their first value while everything else looked correct.
# ADMIN_PASSWORD has NO file form and stays an environment variable.
#
# 4. THREE OPTIONS ARE BAKED IN AT BUILD TIME AND THIS CHART CANNOT MOVE THEM.
# Upstream: "These are compiled into the bundle and cannot be changed at
# runtime ... To change build-time variables with the published Docker image,
# rebuild with matching --build-arg." Building our own image is outside what
# this repository does - the same limit as garage (#81) and keycloak (#83).
# - NEXT_PUBLIC_BASE_PATH: so Bulwark is served at the ROOT of a hostname.
# There is no sub-path deployment and no value here that could make one.
# - NEXT_PUBLIC_DEFAULT_LOCALE: the fallback UI language stays "en". A
# visitor with a matching Accept-Language gets their own language;
# everybody else gets English.
# - NEXT_PUBLIC_PUSH_RELAY_URL: see bulwark.pushRelay below. This one is
# NOT the disclosure it looks like.
# Scale the whole chart down without uninstalling it: scaleDown: true renders
# the StatefulSet with replicas: 0 and takes precedence over bulwark.replicas.
# The PVC, the Secrets, the Service and the Ingress stay in place;
# scaleDown: false brings the workload back up.
scaleDown: false
secrets:
# op:// path of the 1Password item carrying THIS RELEASE'S OWN secrets.
# REQUIRED - the chart does not render without it.
#
# Must carry the fields:
# session-secret -> mounted as a file, read via SESSION_SECRET_FILE
# admin-password -> ADMIN_PASSWORD (environment variable, no file form)
#
# WHY session-secret IS MANDATORY AND NOT OPTIONAL. Two things hang on it, and
# one of them fails silently without it: persistent "remember me" sessions,
# and the encryption of the synchronized user settings. app/api/config/route.ts
# computes `settingsSyncEnabled: configManager.get(...) && hasSessionSecret()`
# - so with bulwark.settingsSync true and no secret, synchronization is
# switched OFF with no error anywhere. Generate it with
# openssl rand -base64 32
#
# AND THE BACKUP STATEMENT, WHICH IS TWO-PART AND IS THE SHARPEST THING ON
# THIS PAGE. It belongs wherever this release's backups are described:
#
# (a) A BACKUP OF THE VOLUME ALONE IS NOT A BACKUP. The synchronized
# settings on the PVC are encrypted with session-secret. Lose the vault
# item and they are unreadable out of a perfectly intact volume. The
# secret has to be in the backup just as much as the volume is.
# (b) A BACKUP OF THE VOLUME MAY CONTAIN A SECRET IN CLEARTEXT. If anybody
# has ever used the admin dashboard's OAuth setup,
# app/api/admin/oauth/setup/route.ts has written oauthClientSecret as
# PLAINTEXT into config.json on the PVC. So the volume backup is not
# merely useless without the vault - it may itself need the protection
# the vault gets.
#
# WHY admin-password IS SET AT ALL, and the reason is NOT the one the issue
# originally gave. Measured: without ADMIN_PASSWORD, Bulwark does NOT generate
# a random password and writes NOTHING to stdout - lib/admin/password.ts
# simply logs "Admin dashboard disabled (no ADMIN_PASSWORD set)" and the
# dashboard stays off. (The only random value in the project is the bootstrap
# setup token, which is written into the state directory and never logged.) So
# there is no leak into the pod log here and the analogy to stalwart's
# STALWART_RECOVERY_ADMIN (#65) does not hold.
#
# The reason it is set is point 1 at the top of this file combined with
# bulwark.oauth.only: THE ADMIN DASHBOARD IS THE ONLY WAY BACK OUT OF AN
# OIDC LOCKOUT. Switching it off means taking away the door.
#
# THE FIELD IS CONSUMED EXACTLY ONCE, which is the part that surprises people
# later: initAdminPassword() returns as soon as admin.json exists on the PVC
# and never looks at the variable again; on the first start it hashes it with
# scrypt and persists the hash. ROTATING THE VAULT FIELD AFTERWARDS HAS NO
# EFFECT. A reset goes through the dashboard itself, or by deleting admin.json
# from the PVC. Same cause as point 1: the volume beats the chart.
appSecretRef: null
# op:// path of the 1Password item carrying the OIDC CLIENT SECRET.
# Required while bulwark.oauth.enabled is true (the default).
#
# Must carry the field:
# oauth-client-secret -> mounted as a file, read via OAUTH_CLIENT_SECRET_FILE
#
# A SEPARATE ITEM FROM appSecretRef, BY AUDIENCE: this value is not ours, it
# is created by the identity provider when the client is registered (keycloak,
# #83), it is rotated there, and it is the one of the three that another
# system also knows. Two items mean it can be repointed and rotated without
# touching this release's own secrets. Same reasoning as garage's split
# between its RPC secret and its S3 key.
#
# ROTATION NEEDS NO RESTART - with one condition the chart satisfies.
# getGlobalClientSecret() (lib/oauth/token-exchange.ts) calls readFileEnv on
# EVERY token exchange with no cache, so a new value in the vault is live as
# soon as the operator has written it. The condition is the whole-directory
# mount of point 3; a subPath mount would never be refreshed.
#
# AND THE ONE WAY A ROTATION STILL DOES NOTHING, which is point 1 again and is
# why this is not filed under "rotation just works": getGlobalClientSecret
# checks the ADMIN OVERRIDE FIRST. If the admin dashboard's OAuth setup has
# ever run, config.json on the PVC carries oauthClientSecret in plaintext and
# that value wins over this file - forever, and with no error. The repair is
# the same as for every other override: remove it in the dashboard (see
# bulwark.oauth.only).
#
# #81 DOES NOT APPLY HERE, and it was checked rather than assumed, because it
# is exactly what forced garage onto an environment variable: lib/read-file-env.ts
# is readFileSync plus try/catch and nothing else - no statSync, no
# `mode & 0o077` permission check. The group-read bit the kubelet ORs onto
# every file of a secret volume once fsGroup is set is therefore harmless, and
# the file form is load-bearing rather than merely available.
oauthSecretRef: null
bulwark:
# ONE, and the schema caps it there. The reason is this chart's single
# ReadWriteOnce PVC, not anything about the application: two pods cannot both
# mount it, so a second replica would sit in ContainerCreating forever. An RWX
# volume would not help either - nothing in Bulwark coordinates two processes
# over that directory, and the admin config is read into memory once at start
# (configManager.load() in instrumentation.node.ts), so two pods would hold
# two divergent views of it.
#
# An explicit 0 is allowed and is the per-workload way to stop the release;
# the chart-global scaleDown flag does the same for everything at once.
replicas: 1
# THE JMAP SERVER - the stalwart release this client talks to. REQUIRED, and
# it does two things at once, which is why it has no default.
#
# (a) it is the address of the mail server. Without it there is no mail.
# (b) IT SWITCHES OFF THE FIRST-RUN SETUP WIZARD. detectSetupState()
# (lib/setup/state.ts) returns "env-managed" as soon as this variable is
# non-empty, and in that state /setup is not served at all. That is what
# makes this chart declarative instead of "install, then click through a
# wizard whose result lands on a volume".
#
# An in-cluster stalwart release is reached at its http Service, e.g.
# http://<stalwart-release>-stalwart-<group>-http.<namespace>.svc.cluster.local:8080
# A published one is reached at its public https URL. Either works - this is
# server-to-server, the browser never sees it.
#
# THE TRAP, AND IT IS THE ONE THE UPSTREAM DOCUMENTATION ACTUALLY WARNS ABOUT:
# a server URL saved in the admin dashboard WINS over this value (point 1 at
# the top). If the dashboard has ever been used to set it, changing this value
# and running `helm upgrade` moves nothing, succeeds, and says nothing. Read
# the effective value back off the running app rather than off the chart:
# kubectl -n <ns> exec <pod> -- cat /app/data/admin/config.json
# An absent jmapServerUrl key in that file means this value is in force.
jmapServerUrl: null
# The product name in the UI and the PWA. null leaves upstream's "Webmail".
appName: null
# STALWART_FEATURES - the Stalwart-specific parts of the UI (password change,
# Sieve rule management). True, which is upstream's default and is the point
# of pairing this chart with stalwart at all.
#
# Note what it does NOT gate: the server-side credential-bearing passthrough
# at /api/account/stalwart/jmap has its own switch upstream
# (STALWART_JMAP_PASSTHROUGH_ENABLED, default true). This chart does not render
# it; an installation that wants the UI features but not the passthrough sets
# it through bulwark.env and should read the precedence note on that value
# first.
stalwartFeatures: true
# WHO MAY OPEN THE ADMIN DASHBOARD, and this value matters more here than its
# name suggests, because after bulwark.oauth.only the dashboard is the ONLY
# door back into a broken login path.
#
# Upstream's three modes (STALWART_ADMIN_ACCESS):
# auto a Stalwart admin sees the shield and is signed into /admin
# WITHOUT the Bulwark admin password. Upstream's default and the
# legacy behaviour.
# password a Stalwart admin sees the shield but must enter the Bulwark
# admin password like everybody else.
# off Stalwart admin status is ignored; /admin is reachable only via
# /admin/login with the Bulwark admin password.
#
# "auto", i.e. upstream's default, is kept DELIBERATELY and the consequence is
# stated rather than left to be discovered: with it, anybody who is an admin
# on the stalwart side can write config overrides here - and an override is
# exactly the thing that can disconnect this chart from its own values
# (point 1) or lock the login path (bulwark.oauth.only). That is usually the
# same person, which is why it is not changed; if it is not, "password"
# narrows the door to whoever holds the vault item, and "off" also removes the
# shield from the UI. Both require admin-password to be configured - which
# this chart always does - or the dashboard would become unreachable.
stalwartAdminAccess: auto
# SETTINGS_SYNC_ENABLED - per-account user preferences follow the user between
# browsers instead of living in one browser's local storage.
#
# ON. The consequence is that /app/data/settings on the PVC carries USER DATA,
# encrypted with the session secret - which is where the two-part backup
# statement at secrets.appSecretRef comes from. It also means the secret is
# mandatory rather than recommended: without it this switch is ignored,
# silently.
settingsSync: true
# ADMIN_CONFIG_READONLY - refuse every write to the admin CONFIG directory at
# the application layer, which makes the values in this file authoritative and
# closes the precedence trap of point 1 for good: with no override able to
# appear, bulwark.oauth.only (and every other value here) always takes effect.
#
# OFF BY DEFAULT, AND THE PRICE IS WHY. It is not one cost but three, and the
# third was measured rather than inferred - it is bigger than the plan for
# this chart assumed:
#
# (a) POLICY, THEMES AND BRANDING FREEZE TOO. assertWritable
# (lib/admin/paths.ts) guards setPolicy, markSetupComplete, the plugin
# and theme registries and the branding upload - not just the config
# override. So the settings policy becomes unchangeable, and with it the
# ONLY way to point Web Push at your own relay (see bulwark.pushRelay).
# (b) AN OVERRIDE THAT ALREADY EXISTS IS FROZEN IN PLACE, not removed.
# removeAdminOverride is guarded by the same assertWritable. Turning this
# on while config.json already carries `oauthOnly: true` therefore makes
# that lockout permanent and takes away the documented repair at the same
# time. If you are turning this on, read config.json first.
# (c) ON A FRESH INSTALL IT TAKES THE ADMIN DASHBOARD AWAY ENTIRELY - i.e.
# exactly the door this chart keeps a password for. initAdminPassword
# persists the hashed ADMIN_PASSWORD by calling writeConfigData, which
# begins with assertWritable('save admin password'). With no admin.json
# on the volume yet, that throws: the startup hook logs "Admin dashboard
# init skipped", the retry inside /api/admin/auth throws again, and the
# login endpoint answers HTTP 500. The pod is Ready and serving mail the
# whole time, so nothing reports it.
#
# THE ONLY SAFE SEQUENCE, which is also upstream's own framing ("Operators set
# ADMIN_CONFIG_READONLY=true after running the setup wizard"):
# 1. install with this false, let the pod start once, so admin.json exists
# 2. set every policy you want (push relay included) - after step 3 you
# cannot
# 3. check that config.json carries no override you still need removed
# 4. then helm upgrade with this true
# The chart cannot verify step 1 for you: it has no way to know what is on the
# volume. NOTES.txt repeats the sequence.
adminConfigReadOnly: false
oauth:
# OIDC SIGN-IN against the identity provider (keycloak, #83). ON.
enabled: true
# OAUTH_ONLY - hide the password form entirely, so the identity provider is
# the only way for a USER to sign in. ON, and this is the chart's riskiest
# default, taken deliberately with the way back out written down.
#
# THE RISK: if the provider is down or the OIDC configuration is wrong,
# nobody reaches their mail, and there is no password form to fall back to.
#
# THE WAY BACK, AND IT IS NOT WHAT IT LOOKS LIKE. `helm upgrade` with this
# set to false is NOT reliably the repair, and believing it was would be the
# trap of point 1 sitting inside the rescue instruction itself: oauthOnly is
# an ordinary key of CONFIG_ENV_MAP, every consumer reads it through
# configManager.get (app/api/auth/session/route.ts,
# app/api/auth/verify/route.ts, app/api/config/route.ts - none of them reads
# process.env), and the admin dashboard's OAuth setup writes
# `oauthOnly: true` into config.json when it is used. Once that override is
# on the PVC, this value is ignored and the upgrade reports success.
#
# WHAT REALLY WORKS, and it does not depend on the identity provider:
# OPEN THE ADMIN DASHBOARD (its own password, from the vault item) AND
# REMOVE THE OVERRIDE - the dashboard's revert, i.e. DELETE on
# /api/admin/config, which calls configManager.removeAdminOverride.
# After that the chart's value applies again. If the dashboard itself is
# unreachable, the remaining repair is editing or deleting config.json on
# the PVC directly. Both are in NOTES.txt.
#
# AND THE PRE-INSTALL STEP THIS DEFAULT MAKES MANDATORY: the OIDC client
# must be registered with the provider BEFORE the first install, not after.
# With oauth.only true and no working client, the very first sign-in is
# impossible. NOTES.txt prints the redirect URI to register.
only: true
# OAUTH_CLIENT_ID. Required while oauth.enabled is true.
clientId: null
# OAUTH_ISSUER_URL - the issuer, from which Bulwark fetches the discovery
# document. Required while oauth.enabled is true. For keycloak (#83) this is
# https://<keycloak-host>/realms/<realm>
#
# It must be reachable BY THE POD and the endpoints it advertises must share
# the issuer's origin or be public: getDiscoveryValidator
# (lib/oauth/token-exchange.ts) rejects a discovered endpoint on a private
# address from a different origin unless OAUTH_ALLOW_PRIVATE_ENDPOINTS is
# set. The chart does not render that escape hatch - a split-horizon setup
# that needs it can add it through bulwark.env.
issuerUrl: null
# OAUTH_SCOPES - replaces the scope list Bulwark would otherwise request.
# null leaves upstream's default, which is the right starting point; an
# incomplete list here is a login that succeeds and then has no identity to
# work with.
scopes: null
# OAUTH_END_SESSION - signing out of Bulwark also ends the provider's
# session through its end_session_endpoint. True, upstream's default: on a
# shared browser, a sign-out that leaves the provider signed in means the
# next click is straight back in. Set it false where the provider is shared
# with applications that should stay signed in.
endSession: true
# WEB PUSH RELAY - read this before writing it down as a disclosure, because
# the obvious reading of the build-time variable is wrong and was corrected
# against the source.
#
# NEXT_PUBLIC_PUSH_RELAY_URL is baked into the bundle (point 4 at the top), so
# this chart cannot move the BUILT-IN entry, which is upstream's hosted relay
# at https://notifications.relay.bulwarkmail.org. It would be wrong to stop
# there and declare Web Push "not changeable with the published image": that
# variable only sets the FALLBACK. lib/push-relays.ts resolves the effective
# relay as `resolveDefaultRelayUrl(policy)`, which takes policy.pushRelayUrl
# whenever it is a valid URL and uses the built-in one only otherwise; with
# policy.pushRelayUrlLocked an operator can additionally pin every user to it.
#
# So the correct, two-part statement, and both halves matter:
# - WITHOUT further action, Web Push for this release goes through the
# VENDOR'S relay. For a mail client that is a fact worth knowing, and it
# stays in the users' relay list either way, because the list is built
# from the bundle.
# - A SELF-HOSTED RELAY CAN BE MADE THE EFFECTIVE DEFAULT AND MANDATORY -
# but through policy.json, i.e. the ADMIN DASHBOARD, not through a chart
# value. There is deliberately no value here: pushRelayUrl lives in
# SettingsPolicy and has no entry in CONFIG_ENV_MAP, so no environment
# variable reaches it and a chart value would be a switch wired to
# nothing.
#
# AND THE INTERACTION TO KEEP IN MIND: bulwark.adminConfigReadOnly freezes
# setPolicy, so turning that on before setting the relay makes this change
# impossible afterwards. Order matters - see the sequence at that value.
#
# THERE IS DELIBERATELY NO VALUE UNDER THIS COMMENT. A `pushRelayUrl:` key
# here would render nothing and change nothing, which is the exact defect
# guideline 12 exists to remove - a key that looks like configuration and does
# nothing. The knowledge belongs in the file an operator reads; the switch
# belongs where it actually is, in the dashboard.
# BULWARK_TELEMETRY - upstream's anonymous telemetry.
#
# FALSE, AND THAT IS NOT MERELY RESTATING THE IMAGE'S DEFAULT, which is the
# reason this value exists at all. Measured in lib/telemetry/state.ts: consent
# defaults to 'off' and telemetry really is opt-in (the comment in
# instrumentation.node.ts calling it "on by default" is wrong - DEFAULTS carries
# `consent: 'off'`). What rendering the variable buys is the LOCK:
# effectiveConsent() takes the environment over the state file, and upstream
# blocks the admin UI from changing it while the override is active ("UI changes
# are blocked when env override is active"). So this is one of the very few
# settings in this chart where the chart beats the volume rather than the other
# way round - see the exception list in point 1 at the top of this file.
#
# Setting it true opts in; /app/data/telemetry on the PVC then holds the
# consent record and the instance id, which is why that directory is inside the
# single volume rather than an emptyDir.
telemetry: false
# BULWARK_UPDATE_CHECK - an hourly version check against
# version.telemetry.bulwarkmail.org.
#
# FALSE, AND HERE THE DEFAULT REALLY IS THE OTHER WAY: unlike telemetry, this
# check is ON unless switched off (lib/version-check/state.ts, disabledByEnv()
# recognizes only off/false/0/no - an absent variable means enabled). So this
# value does change behaviour.
#
# Off for two reasons: a mail client should not phone home from a private
# cluster unless somebody chose it, and Renovate already watches the image tag
# (renovate.json5) - so the check would answer outbound a question that is
# already answered inbound, and answer it to a third party.
updateCheck: false
# LOG_LEVEL - error, warn, info or debug.
logLevel: info
# LOG_FORMAT - text or json. JSON here, against upstream's default of text,
# because the logs go to a cluster log collector rather than to a terminal.
logFormat: json
# The HTTP port. 3000, which is what the image binds (ENV PORT=3000,
# EXPOSE 3000) and what HOSTNAME=0.0.0.0 makes reachable. Both the Service and
# all three probes address this port BY NAME ("http"), so changing the number
# here is safe and renaming the port is not.
port: 3000
resources:
# Limits are optional: memory and ephemeral-storage limits default to
# limitFactor x the request, an explicitly set limit always wins, and a CPU
# limit is only set when configured explicitly - CPU is compressible, so
# leaving it unlimited throttles instead of OOM-killing.
#
# 256Mi x 3 = 768Mi. A Next.js standalone server serving a SPA and proxying
# JMAP holds no mail and caches no mailbox, so the footprint is the Node
# heap plus the image-optimization cache - not a function of how much mail
# the users have.
#
# THE MEMORY LIMIT IS LOAD-BEARING IN ONE DIRECTION, though: /api/health
# reports `degraded` once V8 heap usage crosses its warning threshold, and
# the heap limit is derived from the container limit. That report is not
# visible to the probes (see bulwark.readinessProbe), so a limit set too low
# produces a pod that is permanently degraded, permanently Ready, and says
# so only to an authenticated caller asking for details.
#
# The ephemeral-storage request covers the two emptyDirs (/tmp and the
# Next.js cache) plus container logs. It does NOT cover /app/data, which is
# the PVC - see storage.amount.
limitFactor: 3
requests:
cpu: 100m
memory: 256Mi
ephemeral-storage: 64Mi
# ALL THREE PROBES ARE AN HTTP GET ON /api/health, and the limit of that is
# written at the readiness probe below rather than left for somebody to assume.
#
# A NOTE ON GET VERSUS HEAD, so nobody "fixes" this back. Bulwark offers a HEAD
# handler on this path and it would be the cheaper call - but a KUBERNETES PROBE
# CANNOT SEND HEAD. The API type is HTTPGetAction, it has no method field, and
# the kubelet always issues a GET. The only way to send a HEAD is an exec probe
# calling an HTTP client, and this image has neither curl nor wget (npm and npx
# are removed from the runtime layer), so it would mean starting a Node process
# every few seconds instead of one cheap HTTP request. The GET handler does
# gather diagnostics, which makes it marginally MORE capable than HEAD, not
# less: it can answer 503 when the diagnostics themselves throw.
#
# LIVENESS: the process is alive, accepting HTTP, its event loop is free and
# Next.js can route. An OOM kill, a crash and a wedged event loop all fail it.
livenessProbe:
httpGet:
path: /api/health
port: http
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# READINESS: the same endpoint, and HERE IS THE LIMIT, stated at the value
# because a probe path is not a promise (README guideline 12.3; #218 and
# Gitea #83 are the two times this repo paid for assuming otherwise).
#
# WHAT THIS PROBE CARRIES: the Node process is alive, the HTTP server accepts,
# the event loop is not blocked, Next.js routes. That is real and it is not
# nothing.
#
# WHAT IT DOES NOT CARRY, measured in app/api/health/route.ts at tag 1.12.0:
# - THE REPORTED STATE. The success path returns a hard `status: 200`
# regardless of what it computed, and for an unauthenticated caller the
# body is hard-coded to "healthy" as well
# (`status: detailed ? status : 'healthy'`); "degraded" appears only with
# ?detailed=true AND a caller that passes canSeeInstanceDetails. Upstream
# tests precisely this - lib/__tests__/health-route.test.ts expects HTTP
# 200 while payload.status is "degraded". A probe is unauthenticated and
# asks for no details, so it sees 200 and "healthy" always. The only 503
# this endpoint can produce comes from its catch block, i.e. when
# process.memoryUsage() or v8.getHeapStatistics() itself throws.
# - ANYTHING ABOUT STALWART. Bulwark holds no mail and this endpoint asks the
# JMAP server nothing. WITH THE MAIL SERVER COMPLETELY GONE, THIS PROBE IS
# GREEN AND THE POD STAYS Ready. Monitor the stalwart release for that;
# nothing here can.
#
# AND WHY IT IS STILL THIS ENDPOINT: there is no better one.
# /api/setup/test-jmap really would test the connection, but it lives behind
# the bootstrap setup token and this chart disables the wizard, so it is
# unreachable. Pointing readiness at "/" instead would only add Next.js
# rendering to the same set of guarantees, and would start failing on an
# unauthenticated redirect rather than on anything about the mail path.
readinessProbe:
httpGet:
path: /api/health
port: http
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# STARTUP: gates the other two, which is why neither carries an
# initialDelaySeconds.
#
# BUDGET = periodSeconds x failureThreshold = 5 x 24 = 120s. What that has to
# cover is not the Next.js server, which starts in about a second: it is
# attaching and mounting the network volume, and the instrumentation hook that
# runs before the first request is served (instrumentation.node.ts migrates
# the legacy admin layout, loads config.json and policy.json off the volume,
# and on a FIRST install hashes ADMIN_PASSWORD with scrypt and writes
# admin.json - the scrypt hash being the one deliberately slow step).
#
# NOT MEASURED AGAINST A RUNNING POD AT THE TIME OF WRITING, and said so
# rather than presented as calibrated: the budget is derived from what the
# start does, with head room, and the cost of head room is nothing because a
# pod that IS healthy becomes Ready on the first probe either way.
startupProbe:
httpGet:
path: /api/health
port: http
periodSeconds: 5
timeoutSeconds: 5
failureThreshold: 24
securityContext:
pod:
runAsNonRoot: true
# uid/gid 1001, NOT this repo's 20000, and that is the image's own user
# rather than a free choice: the Dockerfile creates
# addgroup --system --gid 1001 nodejs
# adduser --system --uid 1001 nextjs
# and copies the application tree plus /app/data with --chown=nextjs:nodejs.
# 1001 therefore OWNS everything the process reads, and the four data
# subdirectories are chowned to it in the image. Read off the Dockerfile at
# tag 1.12.0, not guessed.
#
# fsGroupChangePolicy: Always, the repo standard, and here it is the right
# one rather than the default one: /app/data holds a handful of JSON files,
# an audit log and a small settings tree, so the recursive chown is
# cheap - unlike garage's block store or satisfactory's game files, where
# it is the reason both deviate to OnRootMismatch. It also matters on the
# FIRST start, because a freshly provisioned volume arrives owned by root
# and Bulwark has to create its four subdirectories in it.
runAsUser: 1001
runAsGroup: 1001
fsGroup: 1001
fsGroupChangePolicy: Always
seccompProfile:
type: RuntimeDefault
container:
allowPrivilegeEscalation: false
# TRUE, with two emptyDirs carrying the writes that are not state. Which
# paths those are, and how the set was established, is at
# bulwark.writableCache below - it is the one part of this chart that was
# derived from the source and then PROVEN by the install-test rather than
# the other way round.
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
# EMPTY, and it can stay empty: the process binds 3000, which is above
# 1024, and the entrypoint is `node server.js` with no file
# capabilities - so there is no execve-time capability check like
# stalwart's.
add: []
# THE WRITABLE PATHS OUTSIDE THE VOLUME, as emptyDirs, because
# readOnlyRootFilesystem is true and Next.js writes in two places that are not
# application state.
#
# HOW THIS SET WAS ESTABLISHED, and the honest order of events: it was DERIVED
# from next.config.ts at tag 1.12.0 - `output: "standalone"`, and
# `images: { unoptimized: true }` applies only in the isLite branch, so the
# published image has image optimization ACTIVE and writes its cache under
# /app/.next/cache - and it was then PROVEN by the install-test, which starts
# the real image with this exact hardening and requires the pod to become
# Ready. A missing writable path here is not a subtle defect: it is an EACCES
# or EROFS at the first write.
#
# /tmp the conventional scratch path. Node, Next.js and the
# image-optimization pipeline all use it.
# /app/.next/cache Next.js's own cache directory in standalone mode: the
# optimized-image cache and the fetch cache. It is a
# SUBDIRECTORY of .next, which is why it can be an
# emptyDir at all - /app/.next/static holds the build
# output and must stay readable, and mounting the cache
# path does not shadow it.
#
# Both are CACHES and neither is state: losing them on a restart costs one
# round of image re-optimization. Do not "simplify" either one onto the PVC -
# a cache on a ReadWriteOnce network volume is slower than regenerating it,
# and it would grow the only volume that has to be backed up.
writableCache:
# Size limits on both, so a runaway cache fills an emptyDir instead of the
# node's disk. These are the sizeLimit of each emptyDir; the pod's
# ephemeral-storage request/limit (bulwark.resources) is the other half of
# the same accounting and the two should move together.
tmpSizeLimit: 256Mi
nextCacheSizeLimit: 512Mi
# Additive environment entries for the Bulwark container, rendered through tpl
# and supporting the value, secretKeyRef and configMapKeyRef forms.
#
# ADDITIVE MEANS ADDITIVE: an entry repeating a name the chart already sets
# does not replace it. At install time both reach the pod and the later one
# wins; on an EXISTING release the pair collapses to the chart's own value,
# because Kubernetes merges env by name - and helm upgrade still reports
# success. So do not use this to override JMAP_SERVER_URL, OAUTH_ONLY or
# SESSION_SECRET_FILE; change the values that produce them.
#
# AND THE SECOND, SHARPER CAVEAT, SPECIFIC TO THIS APPLICATION: an environment
# variable is only the MIDDLE of three layers (point 1 at the top of this
# file). Anything set here is a default that an admin override in config.json
# on the PVC may already be beating, silently. An entry here is therefore not
# a guarantee - unless bulwark.adminConfigReadOnly is on, and that has its own
# price.
#
# Genuinely useful here, all of them CONFIG_ENV_MAP keys this chart does not
# render: APP_DESCRIPTION, LOGIN_COMPANY_NAME and the other LOGIN_* branding
# variables, COOKIE_SAME_SITE, STALWART_JMAP_PASSTHROUGH_ENABLED,
# OAUTH_EXTRA_SCOPES, WOPI_CLIENT_URL.
env: []
storage:
# THE ONE PVC, mounted at /app/data, and "one" is the decision rather than the
# obvious shape - Bulwark keeps FOUR directories under it and they could each
# have their own volume:
#
# /app/data/admin config.json, policy.json, admin.json (the password
# hash), plugins, themes, branding uploads
# /app/data/admin-state admin-state.json (login timestamps), audit.log, the
# bootstrap setup token
# /app/data/settings the encrypted per-account user settings
# /app/data/telemetry consent and instance id
#
# ONE VOLUME, BECAUSE A SNAPSHOT IS ATOMIC PER VOLUME AND NEVER ACROSS THEM -
# the lesson garage (#81) was built on. Split over four, a backup could
# capture an audit log that records logins the configuration does not know
# about yet, or settings encrypted against a secret the config has since been
# repointed from. One volume keeps configuration, runtime state, audit trail
# and user settings consistent with each other, and there is no performance
# argument on the other side: this is a few hundred kilobytes of JSON plus an
# append-only log.
#
# Bulwark creates all four subdirectories itself on start (ensureConfigDir /
# ensureStateDir, both mkdir -p), which it has to, because the image's own
# mkdir ran in a layer that this mount covers.
#
# THE PVC IS DELIBERATELY NOT null-SAFE: an erased size yields an invalid PVC
# that is rejected at apply time, which is loud. A chart-side fallback would
# silently size somebody's volume - and on an existing release it could not
# help anyway, because a PVC cannot shrink. Same reasoning as garage's storage
# block and satisfactory's documented exception (Gitea #39).
#
# IT IS ALSO NEVER RENAMED: a renamed PVC makes Kubernetes provision a new,
# empty volume and orphans the data - including admin.json, which is the only
# copy of the admin password hash, and the audit log, which by its nature
# cannot be reconstructed.
#
# 1Gi is generous for the JSON files and the settings tree; what actually
# grows is audit.log, and it grows with admin activity rather than with mail
# volume. There is no rotation for it in the application, so this is the value
# to watch on a long-lived release.
amount: 1Gi
# null uses the cluster's default StorageClass. ReadWriteOnce is enough and is
# what the single replica needs; see bulwark.replicas for why more than one is
# not on offer.
storageClass: null
ingress:
# ON by default: a webmail client nobody can reach is not a webmail client.
enabled: true
# The public hostname. REQUIRED, AND REQUIRED EVEN WITH ingress.enabled:
# false - this is guideline 12.1, checked in both directions for this chart.
# The value is read a SECOND time outside the Ingress template: NOTES.txt
# derives the OIDC redirect URI from it, and that URI is what has to be
# registered with the identity provider before the first install. Gating it
# away would produce a release that renders cleanly and whose operator is told
# to register `https:///auth/callback`.
#
# BULWARK IS SERVED AT THE ROOT OF THIS HOST AND CANNOT BE SERVED UNDER A
# PATH. NEXT_PUBLIC_BASE_PATH is compiled into the bundle (point 4 at the top
# of this file), so a sub-path deployment would need a rebuilt image. The
# Ingress rule is therefore "/" and there is no value to change it - a
# rewrite-target annotation would not help either, because the bundle's own
# asset URLs are built for the root.
domain: null
# cert-manager cluster issuer. Required while the Ingress is enabled.
clusterIssuer: null
className: nginx
# Extra annotations, merged with the cert-manager annotation the chart always
# sets.
#
# ONE WORTH KNOWING ABOUT FOR THIS APPLICATION: file attachments and the Files
# view upload through this Ingress, and nginx's default
# proxy-body-size (1m) will reject anything larger with a 413 that looks like
# an application fault. Set
# nginx.ingress.kubernetes.io/proxy-body-size: "25m"
# (or whatever the mail server accepts) to match stalwart's own limit. The
# chart does not set it, because the right number is the mail server's and
# this chart does not know it.
annotations: {}
+1 -1
View File
@@ -5,7 +5,7 @@ name: calibre-web-automated
description: A Helm chart to deploy calibere-web-automated
icon: https://raw.githubusercontent.com/crocodilestick/Calibre-Web-Automated/main/README_images/cwa-logo-round-dark.png
version: 7.0.1+upv4.0.6
version: 6.1.0+upv4.0.6
appVersion: v4.0.6
maintainers:
+7 -4
View File
@@ -61,7 +61,7 @@ THE FIRST START BUILDS A LIBRARY, AND THE STARTUP PROBE IS TIMING IT
kubectl -n {{ .Release.Namespace }} logs {{ $pod }} -f
SECRETS — THE FIELDS THE 1PASSWORD ITEM MUST CARRY
{{ .Values.secrets.cwaSecretRef }}
{{ if .Values.secrets.cwaSecretRef }}{{ .Values.secrets.cwaSecretRef }}{{ else }}(secrets.cwaSecretRef is unset — see below){{ end }}
→ Secret {{ .Release.Name }}-{{ .Chart.Name }}-secret
hardcover-api-token becomes the HARDCOVER_TOKEN environment variable
calibre-plugins mounted as /config/.config/calibre/customize.py.json,
@@ -74,10 +74,13 @@ SECRETS — THE FIELDS THE 1PASSWORD ITEM MUST CARRY
hardcover-api-token parks the pod in CreateContainerConfigError; a missing
calibre-plugins fails the mount instead ("couldn't find key calibre-plugins"),
and the container is never created at all.
{{- if not .Values.secrets.cwaSecretRef }}
secrets.cwaSecretRef itself is required: the template guards it with
required(), so a release without it does not render at all rather than
creating a OnePasswordItem with an empty itemPath.
secrets.cwaSecretRef IS UNSET. The OnePasswordItem has been created with an
empty itemPath, the operator has nothing to resolve, and the Secret above will
never appear — the pod cannot start. Set it to the item's op:// path and
upgrade.
{{- end }}
CHANGING THE ITEM'S CONTENT RESTARTS NOTHING
Helm never sees what is inside a OnePasswordItem, so it cannot hash it into the
+5 -8
View File
@@ -19,11 +19,9 @@ Usage (the values path to the resources dict varies per chart):
resources:
{{- include "calibre-web-automated.resources" .Values.<app>.resources | nindent 12 }}
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart, and a dependency is admissible but expensive (README, guideline 14) -
this chart takes none. This file in template-chart is the reference
implementation.
NOTE: charts in this repo deliberately have no dependencies, so this helper is
duplicated into every chart's templates/_helpers.tpl with the chart's name as
define prefix. This file in template-chart is the reference implementation.
*/}}
{{- define "calibre-web-automated.resources" -}}
{{- $resources := . | default dict -}}
@@ -169,9 +167,8 @@ Rules:
- keys only present in the override are passed through unchanged.
NOTE: like the helpers above, this is duplicated into every chart's
templates/_helpers.tpl with the chart's name as define prefix, because sharing
it would mean a library chart and this chart takes no dependency (README,
guideline 14).
templates/_helpers.tpl with the chart's name as define prefix, because charts
in this repo deliberately have no dependencies.
*/}}
{{- define "calibre-web-automated.defaultedDict" -}}
{{- $defaults := .defaults | default dict -}}
@@ -3,4 +3,4 @@ kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-secret
spec:
itemPath: {{ required "A 1Password item reference must be provided (secrets.cwaSecretRef)" .Values.secrets.cwaSecretRef }}
itemPath: {{ .Values.secrets.cwaSecretRef }}
-1
View File
@@ -17,7 +17,6 @@
"additionalProperties": false,
"properties": {
"cwaSecretRef": {
"description": "op:// item path carrying \"hardcover-api-token\" and \"calibre-plugins\". Required - the OnePasswordItem, the HARDCOVER_TOKEN env reference and the plugin volume are all rendered unconditionally. Enforced by required() in the template, not here: the null default keeps this type widened, and a schema \"required\" entry could never fire against a key that is present with value null.",
"type": ["string", "null"]
}
}
-9
View File
@@ -5,15 +5,6 @@
scaleDown: false
secrets:
# REQUIRED - op:// item path for the 1Password operator, carrying the fields
# "hardcover-api-token" and "calibre-plugins". Both are needed even if you
# never use Hardcover: the OnePasswordItem is created unconditionally, and so
# are the environment reference and the volume. Without this value the render
# aborts - required() in calibre-web-automated.secret.yaml is the whole
# enforcement, and values.schema.json cannot help here (README, guideline 12).
# Before the guard existed it produced an OnePasswordItem with an empty
# itemPath, the operator had nothing to resolve, the Secret never appeared and
# the pod could not start.
cwaSecretRef: null
cwa:
-69
View File
@@ -1,69 +0,0 @@
# Fixtures for the bulwark install-test: the two Secrets the 1Password operator
# would otherwise materialize from the OnePasswordItems the chart renders
# (release name: ci, stub CRD in ci/common/), plus a name for the JMAP endpoint
# to resolve to.
#
# NO DATABASE, NO CACHE, NO BROKER AND NO MAIL SERVER - and that is a property of
# the chart rather than a shortcut taken here. Bulwark is a client: it holds no
# mail and contacts the JMAP server per request, not at boot, so it starts and
# becomes Ready with nothing behind that URL. The flip side is the limitation the
# chart documents at bulwark.readinessProbe, and this file is the clearest
# illustration of it: everything needed to make this chart green is in these few
# objects, and none of it is a mail server.
---
apiVersion: v1
kind: Secret
metadata:
name: ci-bulwark-app-secret
stringData:
# The field NAMES are what matter here, not the values: the operator
# materializes one Secret key per item field, and the chart consumes these two
# in DIFFERENT ways - session-secret as a projected FILE, admin-password
# through a secretKeyRef.
#
# Only the second of those is actually asserted by the run. The projected
# volume is `optional: true` by design (so a first install does not hang
# waiting for the 1Password operator), so a misspelled session-secret yields a
# missing file and a pod that starts fine. A misspelled admin-password leaves
# the container in CreateContainerConfigError and fails the run.
#
# A real base64 32-byte value rather than a placeholder word: Bulwark derives
# encryption keys from it, and a value of the wrong shape would be the kind of
# thing that works in CI and not in production.
session-secret: Y2ktb25seS1zZXNzaW9uLXNlY3JldC1ub3QtYS1yZWFsLW9uZQ==
# Visibly throwaway. It is hashed with scrypt into admin.json on the first
# start and the cluster is deleted with the job; the only thing it protects is
# a dashboard that exists for a few minutes.
admin-password: ci-bulwark-admin-password
---
apiVersion: v1
kind: Secret
metadata:
name: ci-bulwark-oauth-secret
stringData:
# The OIDC client secret. Nothing in CI performs a token exchange - no identity
# provider runs here - so this exists to exercise the SECOND source of the
# chart's projected volume and the OAUTH_CLIENT_SECRET_FILE path, not the OAuth
# flow.
oauth-client-secret: ci-bulwark-oauth-client-secret
---
# The JMAP endpoint's name. A Service with NO SELECTOR and therefore no
# endpoints: the in-cluster DNS name resolves, and a connection to it is refused.
#
# DELIBERATELY NOT A DEPLOYMENT. ci/run-install-test.sh waits for every fixture
# Deployment and StatefulSet to roll out BEFORE it installs the chart, so a stub
# web server here would be one more workload able to fail a run that is not about
# it - and it would buy nothing, because a refused connection and a served 404
# are indistinguishable to a chart that never contacts this address at boot.
#
# It exists at all so the URL in test-values.yaml is a real name rather than an
# invented hostname, which keeps the fixture honest about what the chart does
# with it.
apiVersion: v1
kind: Service
metadata:
name: jmap-stub
spec:
ports:
- port: 8080
targetPort: 8080
-162
View File
@@ -1,162 +0,0 @@
# CI values for the bulwark install-test.
#
# WHY THIS CHART IS TESTED RATHER THAN EXCLUDED, and what a green run is worth.
# Bulwark needs no database, no cache and no broker, and it does not need a
# reachable JMAP server to START - the connection is made per request, not at
# boot - so a dummy URL is enough to bring the pod up. That makes this one of
# the cheapest install-tests in the repo and the only place where the two
# riskiest assumptions of the chart are exercised at all. Neither is observable
# in a render.
#
# WHAT A GREEN RUN PROVES:
#
# 1. readOnlyRootFilesystem: true HOLDS, WITH EXACTLY THE TWO emptyDirs THE
# CHART RENDERS - /tmp and /app/.next/cache. This is the measurement point
# the plan for Gitea #89 left open, and it is the reason this fixture
# exists: the set of writable paths was DERIVED from next.config.ts
# (output: "standalone", and images.unoptimized applies only in the isLite
# branch, so image optimization is active and writes under
# /app/.next/cache) and could not be measured without running the image.
# A missing writable path is not subtle - it is an EROFS or EACCES on the
# first write - so a pod that reaches Ready and stays Ready is the proof.
# A SUPERFLUOUS emptyDir would NOT be caught here; that half stays open and
# is named in the chart's wiki page.
# 2. THE PROBES CAN REALLY BE SATISFIED. All three are an HTTP GET on
# /api/health, which has to exist in the built bundle and answer 200
# unauthenticated. If it did not, the startup probe would exhaust its budget
# and the rollout would fail - and no render could have told us, because a
# path only exists in the built bundle.
# 3. The uid/gid 1001 read off the Dockerfile is right, and fsGroup makes a
# freshly provisioned volume writable for it: Bulwark creates its four
# subdirectories under /app/data itself on start, and the image's own mkdir
# ran in a layer this mount covers. Wrong uid, or no fsGroup, and the
# instrumentation hook fails to write admin.json.
# 4. ADMIN_PASSWORD is really consumed on the first start - the hook hashes it
# with scrypt and writes /app/data/admin/admin.json. That write is inside
# the startup budget, so a budget that was too short would fail here.
# Plus the mechanical half: the projected volume assembles, the Service port
# name matches the containerPort, the Ingress is accepted with its extra
# annotation alongside the cert-manager one, the resource quantities are valid,
# and the schema accepts a realistic non-default values file.
#
# ONE CLAIM THIS FIXTURE DELIBERATELY DOES NOT MAKE, because it would be
# false: it does not prove that the two FILE-form secret keys are spelled
# right. The projected volume marks both sources `optional: true` on purpose -
# so that a first install does not hang in ContainerCreating while the
# 1Password operator catches up - and the consequence is that a misspelled
# session-secret or oauth-client-secret key yields a MISSING FILE and a pod
# that starts perfectly. Only `admin-password` is proven, because it arrives
# through a non-optional secretKeyRef and a wrong name there leaves the
# container in CreateContainerConfigError.
#
# WHAT IT DOES NOT PROVE, and the first two matter most:
# - ANYTHING ABOUT JMAP. The URL below points at a Service that answers
# nothing useful. Bulwark comes up regardless, which is precisely the
# property the readiness probe cannot see (chart values.yaml,
# bulwark.readinessProbe) - so this test demonstrates the limitation rather
# than closing it. A green run here says nothing about mail working.
# - ANYTHING ABOUT OIDC. No identity provider runs in CI, so the sign-in path
# is never walked. What IS checked is that the chart's render guards fire on
# a missing client id or issuer (unit-testable at render time) and that the
# secret file is mounted where OAUTH_CLIENT_SECRET_FILE points.
# - THE PRECEDENCE TRAP. config.json never exists in this run, so the
# "volume beats the chart" behaviour - the chart's single most important
# operating caveat - is not exercised. It cannot be: it needs an admin
# writing an override, which needs a browser.
# - anything about an upgrade, or about a volume that already carries data.
# This installs once, from nothing.
secrets:
# Materialized by ci/bulwark/fixtures.yaml as ci-bulwark-app-secret and
# ci-bulwark-oauth-secret (the OnePasswordItem stub CRD comes from ci/common/).
appSecretRef: op://ci/bulwark-app
oauthSecretRef: op://ci/bulwark-oauth
bulwark:
# A dummy JMAP endpoint. It does NOT have to answer - Bulwark connects per
# request - but it does have to be SET, because an empty value puts the app
# into bootstrap mode and serves the setup wizard instead. So this value also
# exercises the path that keeps the chart declarative.
#
# It points at the selector-less Service in fixtures.yaml: a name that RESOLVES
# and then refuses the connection. That is deliberately not a Deployment -
# fixtures with a workload have to roll out before the chart is installed, so a
# stub serving nothing would be one more thing able to fail a run that is not
# about it. A refused connection and a served 404 are indistinguishable to this
# chart, because nothing here contacts the JMAP server at boot.
jmapServerUrl: http://jmap-stub.default.svc.cluster.local:8080
appName: Bulwark CI
# OIDC LEFT ON, i.e. the chart's default, although no provider answers in CI.
# That is deliberate: it is what makes the run exercise the OAuth
# OnePasswordItem, the second source of the projected volume and the
# OAUTH_CLIENT_SECRET_FILE path. Bulwark does not contact the issuer at boot,
# so an unreachable one costs nothing here - it would only fail a sign-in, and
# nothing signs in.
oauth:
enabled: true
only: true
clientId: bulwark-ci
issuerUrl: https://keycloak.ci.example.com/realms/ci
# ON, i.e. the chart's default, and it is what makes /app/data/settings get
# created and the session secret actually read: with the sync off, a missing or
# misnamed session-secret field would pass unnoticed.
settingsSync: true
# OFF - the chart's default, spelled out because it is load-bearing for what
# this test means. With it ON, a fresh install cannot write admin.json and the
# admin dashboard never comes up; the pod would still be Ready, so the run
# would be green while the chart's documented rescue door was missing. Leaving
# it off is what makes step 4 of "what a green run proves" an assertion.
adminConfigReadOnly: false
# Both off, as in the chart - and here with a second reason: a CI runner should
# not send a telemetry heartbeat or an update check to a vendor endpoint on
# every pull request.
telemetry: false
updateCheck: false
# debug, so that a failed run has something to read. The install-test dumps
# container logs on failure, and Bulwark's info level says very little about
# why a start went wrong.
logLevel: debug
# text rather than the chart's json: a human reads these logs out of a failed
# job, not a collector.
logFormat: text
resources:
# Small, to stay inside the runner's budget, while still exercising the limit
# derivation: 192Mi x limitFactor 3 = 576Mi. Not smaller than this on
# purpose - the memory LIMIT is what V8's heap limit is derived from, and a
# Next.js server with too little heap fails in a way that looks like an
# application fault.
requests:
cpu: 100m
memory: 192Mi
ephemeral-storage: 64Mi
storage:
amount: 256Mi
# k3d's dynamic provisioner. The intended production value does not exist on
# the runner - and a pre-bound hostPath PV would be worse than this one,
# because the kubelet does not apply fsGroup to a hostPath volume at all, so
# the test would silently stop exercising the ownership path that matters most
# here: Bulwark has to CREATE four subdirectories in a volume it did not make.
storageClass: local-path
ingress:
# Left ON although no ingress controller answers in CI: the point is that the
# object is accepted and that the Service port it names BY NAME ("http")
# actually exists - a typo there is otherwise invisible until someone opens the
# URL.
enabled: true
domain: bulwark.ci.example.com
clusterIssuer: ci-selfsigned
annotations:
# Exercises the annotation merge: this has to arrive ALONGSIDE the
# cert-manager annotation the chart always sets, not instead of it. It is
# also the annotation this chart's NOTES.txt tells operators to add, so the
# fixture is the shape a real install has.
nginx.ingress.kubernetes.io/proxy-body-size: "25m"
+1 -1
View File
@@ -5,7 +5,7 @@
# (REQUIRED_ENV), and pipes the result straight into kubectl.
#
# The values come from a 1Password item holding a CI-only Mullvad key, loaded in
# the install-test job of .github/workflows/install-test.yml. A CI-only key
# the install-test job of .github/workflows/validate-charts.yml. A CI-only key
# on purpose: Mullvad ties an address to a key and sends the tunnel to the
# endpoint it saw last, so a shared key would let every pull request pull the
# tunnel out from under the running stack.
+3 -6
View File
@@ -1,11 +1,8 @@
# Ten containers in one pod, several GB of images to pull, and the app
# Nine containers in one pod, several GB of images to pull, and the app
# containers only start once gluetun has completed its Wireguard handshake -
# each of them then runs its own startup probe (150s budget, 300s for seerr's
# first-boot migrations). mediathekarr adds ~850 MB of image and waits for
# gluetun's health server BEFORE its process starts (its own start gate), so its
# 300s startup budget only begins once the tunnel is up. The default of 300s is
# not enough for any of that.
INSTALL_TIMEOUT=1200
# first-boot migrations). The default of 300s is not enough for that.
INSTALL_TIMEOUT=900
# The gluetun credentials for ci/freesailarr/fixtures.env.yaml. Missing or empty
# values fail the install-test loudly instead of skipping it - see
+1 -9
View File
@@ -1,4 +1,4 @@
# freesailarr: the whole stack in one pod - gluetun plus nine app containers
# freesailarr: the whole stack in one pod - gluetun plus eight app containers
# that share its network namespace and sit behind its kill switch. Nothing here
# is a shortcut around that: the install-test brings up a real Wireguard tunnel
# with a CI-only Mullvad key (see ci/freesailarr/fixtures.env.yaml), because a
@@ -37,11 +37,3 @@ ingress:
# in the CI cluster, and the install-test checks readiness, not routing.
host: freesailarr.example.com
clusterIssuer: ci
mediathekarr:
# On in CI although it is off by default, so the install-test actually covers
# the tenth container: its digest-pinned image (single-arch amd64, ~850 MB),
# its start gate against gluetun's health server, and its probes on the two
# ports. Its ingress stays off - MediathekArr can only be served at the root
# of a host, and the root of the shared host above belongs to seerr.
enabled: true
-45
View File
@@ -1,45 +0,0 @@
# Fixtures for the garage install-test: the two Secrets the 1Password operator
# would otherwise materialize from the OnePasswordItems the chart renders
# (release name: ci, stub CRD in ci/common/).
#
# NO THROWAWAY INFRASTRUCTURE: Garage depends on nothing - no database, no
# cache, no other service. The whole deployment is one process and two volumes,
# which is also why this chart is cheap enough to test on every PR.
#
# THE FIELD NAMES MATTER AS MUCH AS THE VALUES, and here the VALUES matter too,
# which is unusual for a fixture: Garage validates all four of them on start and
# refuses to run on a malformed one. So these are not placeholders - a value
# that merely looks right (a 31-byte RPC secret, a 6-character access key) fails
# the run, which is exactly the check that makes the fixture worth having.
apiVersion: v1
kind: Secret
metadata:
name: ci-garage-rpc-secret
stringData:
# EXACTLY 32 bytes, hex-encoded = 64 hex characters. Garage decodes this and
# rejects anything else with "Invalid RPC secret key: expected 32 bytes of
# random hex" - it is the network key of the RPC layer, required even for a
# single node.
#
# A throwaway value, and a visibly throwaway one: this is not a secret, it is
# a test vector. Nothing here survives the run, the cluster is deleted with
# the job, and the node it keys has no peers.
rpc-secret: "c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1c1"
---
apiVersion: v1
kind: Secret
metadata:
name: ci-garage-s3-secret
stringData:
# The three fields Garage IMPORTS on start with --default-access-key and
# --default-bucket. Each one is validated (src/model/key_table.rs,
# src/model/bucket_alias_table.rs):
# access-key at least 8 characters, ASCII alphanumeric plus - _ .
# secret-key at least 16 characters, printable ASCII
# bucket a valid S3 bucket name, 3-63 characters, lowercase
# Shortening any of them below its minimum is the quickest way to check that
# this test really exercises the import path: the container exits instead of
# starting.
access-key: "ciGarageTestKey01"
secret-key: "c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2c2"
bucket: "ci-garage-test"
-103
View File
@@ -1,103 +0,0 @@
# CI values for the garage install-test.
#
# WHY THIS CHART IS TESTED RATHER THAN EXCLUDED, and what the test is worth:
# with garage.singleNode (the chart default) the server creates its own cluster
# layout before it opens any listener, so the pod reaches Ready without an
# operator. That is what makes an install-test possible here at all - without
# the flag the readiness probe would honestly never go green (no layout, no
# quorum, /health answers 503) and a green run would have required a probe that
# lies.
#
# WHAT A GREEN RUN PROVES - the four measurement points the plan for Gitea #81
# left open, three of which this closes on every PR:
#
# 1. Garage accepts metadata_dir and data_dir as two SUBDIRECTORIES OF ONE
# PVC and creates them itself. The image is FROM scratch and has no mkdir,
# so if the assumption were wrong the pod could not start at all.
# 2. readOnlyRootFilesystem: true holds with no emptyDir anywhere - the
# container writes only into the two volumes.
# 3. The layout is really applied in-process and the admin API answers 200 on
# /health afterwards: Ready is exactly that HTTP 200, so the rollout
# status IS the assertion.
# Plus the mechanical half: the Secret key names match what the StatefulSet
# references, the rendered garage.toml PARSES AS GARAGE READS IT (it opens
# LMDB, validates replication_factor against --single-node and refuses
# malformed keys), the access key and the bucket are imported from the vault
# fields, the resource quantities are valid, and the schema accepts a
# realistic non-default values file.
#
# WHAT IT DOES NOT PROVE, and no install-test here could:
# - that the metadata auto-snapshot really lands in metadata_snapshots_dir.
# The first snapshot is taken after interval/2, and the shortest interval
# that is not absurd is still minutes - longer than the run. A snapshot
# interval of seconds would test the timer, not the thing.
# - anything about Longhorn: the snapshot semantics the chart's whole storage
# layout is built on are a property of the storage layer, and the runner
# has local-path.
# - durability. metadata_fsync is set, and whether an fsync survives a power
# cut is not observable in a container.
secrets:
# Materialized by ci/garage/fixtures.yaml as ci-garage-rpc-secret and
# ci-garage-s3-secret (the OnePasswordItem stub CRD comes from ci/common/).
rpcSecretRef: op://ci/garage-rpc
s3SecretRef: op://ci/garage-s3
garage:
# Both ON, i.e. the chart's defaults, and they are the reason the run means
# something: the key and the bucket are imported FROM the fixture Secret, so
# a wrong field name or a malformed value fails the start rather than passing
# unnoticed.
singleNode: true
defaultAccessKey: true
defaultBucket: true
# ON against the chart's default of off, together with a root domain. It
# renders a second listener, a second containerPort, a Service port and the
# [s3_web] section of garage.toml - all of which Garage has to accept, and
# none of which the default install exercises.
web:
enabled: true
rootDomain: ".web.garage.ci.example.com"
s3:
region: garage-ci
rootDomain: ".s3.garage.ci.example.com"
# Shortened from the chart's "6h" so the snapshot WORKER is created and
# scheduled during the run. It still does not take a snapshot - the first one
# is due after interval/2, i.e. 15 minutes here - and that gap is named above
# rather than papered over with a one-second interval that would only prove
# the timer fires.
metadataAutoSnapshotInterval: "30m"
resources:
# Small, to stay inside the runner's budget, while still exercising the
# limit derivation: 256Mi x limitFactor 3 = 768Mi.
requests:
cpu: 100m
memory: 256Mi
ephemeral-storage: 32Mi
storage:
data:
amount: 512Mi
# k3d's dynamic provisioner. The intended production value,
# longhorn-encrypted-ns, does not exist on the runner - and a pre-bound
# hostPath PV would be worse than this one, because the kubelet does not
# apply fsGroup to a hostPath volume at all, so the test would silently
# stop exercising the ownership path that matters for a volume the
# container did not create.
storageClass: local-path
snapshots:
amount: 256Mi
storageClass: local-path
ingress:
# Left ON although no ingress controller answers in CI: the point is that the
# object is accepted and that the Service port it names by NAME ("s3")
# actually exists - a typo there is otherwise invisible until someone opens
# the URL.
enabled: true
domain: garage.ci.example.com
clusterIssuer: ci-selfsigned
-118
View File
@@ -1,118 +0,0 @@
# Fixtures for the keycloak install-test: a throwaway PostgreSQL plus the two
# Secrets the 1Password operator would otherwise materialize from the
# OnePasswordItems the chart renders (release name: ci, stub CRD in ci/common/).
#
# WHY A REAL DATABASE AND NOT A STUB: Keycloak does nothing without one. It
# creates its entire schema through Liquibase on first start, and - the reason
# this fixture is the valuable half of the test - jdbc-ping discovers the second
# replica THROUGH this database. A stub would turn a cluster test into a
# templating test.
#
# The image is pinned to a major line like every other fixture here
# (postgres:16-alpine); ci/** is excluded from Renovate on purpose, because
# this is test scaffolding and churning it only risks breaking the
# install-test.
apiVersion: v1
kind: Secret
metadata:
name: ci-keycloak-db-secret
stringData:
# The field NAMES are what matters here, not the values: the operator
# materializes one Secret key per item field, and the Deployment references
# these two by name. A rename on either side leaves the pod in
# CreateContainerConfigError, which is a state no render can show.
#
# They must agree with the POSTGRES_USER / POSTGRES_PASSWORD below, because
# Keycloak really connects.
username: keycloak
password: ci-keycloak-db-password
---
apiVersion: v1
kind: Secret
metadata:
name: ci-keycloak-bootstrap-admin-secret
stringData:
# The temporary bootstrap administrator. Keycloak creates it while the master
# realm is created, so these values are exercised on the very first start -
# and the variables are named KC_BOOTSTRAP_ADMIN_* only since Keycloak 26.
#
# A visibly throwaway password: nothing here survives the run, the cluster is
# deleted with the job, and the only thing it protects is a realm that exists
# for a few minutes.
username: ci-temp-admin
password: ci-bootstrap-admin-password
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
labels:
app: postgres
spec:
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: public.ecr.aws/docker/library/postgres:16-alpine
env:
- name: POSTGRES_USER
value: keycloak
- name: POSTGRES_PASSWORD
value: ci-keycloak-db-password
- name: POSTGRES_DB
value: keycloak
ports:
- containerPort: 5432
readinessProbe:
exec:
command: ["pg_isready", "-U", "keycloak"]
periodSeconds: 5
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
volumes:
# emptyDir: the database exists for the length of one job. Nothing here
# is meant to survive, and a PVC would only add a provisioner to the
# list of things that can fail a chart's test.
- name: data
emptyDir: {}
---
apiVersion: v1
kind: ConfigMap
metadata:
# A MINIMAL BUT REAL LOGIN THEME, so that the install-test actually walks the
# path keycloak.extraVolumes exists for (Gitea #88) instead of only rendering
# it. ci/keycloak/test-values.yaml mounts it on the SUBPATH
# /opt/keycloak/themes/ci-theme - never on /opt/keycloak/themes itself, which
# would hide the image's base, keycloak and keycloak.v2 themes.
name: ci-keycloak-theme
data:
# A theme's layout is <theme>/<type>/theme.properties, and a ConfigMap key
# cannot contain a slash - so the "login/" directory is produced by
# items[].path in the volume, which can. That indirection is exactly the part
# of the theme route a render cannot verify and a running pod can.
theme.properties: |
parent=keycloak.v2
# The reason the issue was opened: Keycloak has no logo field on an identity
# provider, so the icon on its login button comes only from a login theme.
# Nothing in CI drives a browser, so this line is carried rather than
# checked - what the run proves is that the theme reaches the container.
kcLogoIdP-ci-provider = fa fa-lock
---
apiVersion: v1
kind: Service
metadata:
name: postgres
spec:
selector:
app: postgres
ports:
- port: 5432
targetPort: 5432
-17
View File
@@ -1,17 +0,0 @@
# The default 300s is not enough here, and the three parts that fill the budget
# are worth naming so the number can be lowered again if one of them changes:
#
# 1. TWO INIT CONTAINERS PER POD. The build one runs a full Quarkus
# augmentation - upstream's own issue reports measure "Quarkus
# augmentation completed in" at roughly 13-23 seconds, and a CI runner is
# not a fast machine.
# 2. THE FIRST START CREATES THE WHOLE SCHEMA through Liquibase, and with two
# replicas the second pod waits on the first's migration lock. So the two
# pods do not become Ready in parallel.
# 3. The rollout is only complete when BOTH pods are Ready, which is
# deliberate (it is what makes cluster formation part of the assertion)
# and which means the slow parts add up rather than overlap.
#
# The chart's own startupProbe budget is 600s (10s x 60); this timeout has to
# be at least that, or the test gives up before the probe would have.
INSTALL_TIMEOUT=900
-222
View File
@@ -1,222 +0,0 @@
# CI values for the keycloak install-test.
#
# WHY THIS CHART IS TESTED RATHER THAN EXCLUDED, and what a green run is worth.
# Keycloak has no volume, needs one throwaway PostgreSQL and reaches Ready on
# its own, so an exclusion would be hard to justify. More importantly, this is
# the only place where the four riskiest assumptions of the chart are exercised
# at all - none of them is observable in a render.
#
# WHAT A GREEN RUN PROVES, point by point (the measurement points of Gitea #83):
#
# 1. THE TWO INIT CONTAINERS WORK AT ALL. The seed container copies the
# image's own augmentation output into the volume, the build container
# mounts that volume at /opt/keycloak/lib/quarkus and runs `kc.sh build`
# into it - both with readOnlyRootFilesystem: true. If the seeding
# assumption were wrong (QuarkusEntryPoint loads
# quarkus/quarkus-application.dat before it dispatches any command,
# `build` included), the build container could not start, and the pod
# would sit in Init with the reason only in its log.
# 2. readOnlyRootFilesystem: true HOLDS FOR THE SERVING CONTAINER, with
# emptyDirs only at /opt/keycloak/lib/quarkus, /opt/keycloak/data and
# /tmp - and NOT at /opt/keycloak/conf, which would hide the
# cache-ispn.xml that cache=ispn reads.
# 3. `start --optimized` ACCEPTS THE PRE-BUILT INSTALLATION and does not
# re-augment. A disagreement between the build options and the serving
# container would show up here as a crash loop rather than as a slow
# start, because the path it would want to rewrite is read-only.
# 4. TWO REPLICAS BOTH COME UP AND STAY UP against one database - the
# rollout is complete only when BOTH pods are Ready, so a second replica
# that crashed, deadlocked on the schema migration lock or fought the
# first over the database would fail the run. Measured in Gitea PR #87:
# 71.8s from `helm install` to both Ready, 0 restarts - three runs, all
# three 72s and 0 restarts.
# THAT WAS LUCK, OR A FAST WEEK, AND NOT AN INVARIANT - which took an A/B
# inside Gitea PR #92 to establish, because the first five runs there made
# it look like the new theme mount had destabilized the start:
# WITH the theme mount (spec of #88)
# run 585: 101s, ONE restart on one pod
# run 590: 82s, 0
# run 593: 191s, 0
# run 595: 101s, ONE
# run 597: 111s, ONE -> 3 of 5
# WITHOUT it (the same branch, the pair commented out; pod template
# hash 5844db56b6, i.e. BYTE-IDENTICAL to the spec PR #87 measured)
# run 598: 81s, 0
# run 599: 171s, ONE -> 1 of 2
# So the restart happens WITHOUT the mount as well, and #88 is not its
# cause. It is pre-existing, intermittent behaviour of THIS chart's first
# install: always exactly one restart on one pod, 35-80s before that pod
# reports Ready. It does not track the runner's speed either - the fastest
# (81s) and the slowest (191s) run of the whole series restarted nothing.
# WHAT THE RESTART IS HAS NOT BEEN ESTABLISHED, and that is the finding
# worth keeping: the harness dumps container logs only when a run FAILS
# (diagnose() hangs off the EXIT trap behind "RC -ne 0") and all seven runs
# PASSED - A GREEN RUN CANNOT EXPLAIN A RESTART IT REPORTS. Answering it
# means making the final-state step print the restart reasons whenever a
# counter is above zero; that is a change to ci/run-install-test.sh with a
# CI-CD wiki obligation of its own, and it was deliberately NOT made inside
# #88. A first install that serializes two replicas on the Liquibase
# migration lock is a plausible place for it - stated as the hypothesis it
# is, because nothing here measured it.
# THE METHOD IS THE PART TO REUSE: a counter that moved in the same round
# as a change is not evidence against the change until the control case has
# run on the SAME machine. Three clean runs two days earlier were not that
# control, and treating them as one would have cost this PR its conclusion
# in both directions.
# 5. A MOUNTED LOGIN THEME SURVIVES ALL OF THAT (Gitea #88) - the full
# argument sits at keycloak.extraVolumes further down, including what it
# still does not prove.
# Plus the mechanical half: the Secret key names match what the Deployment
# references, /health/started, /health/live and /health/ready all answer on
# the management port (so health-enabled really was baked in by the init
# container), the resource quantities are valid, and the schema accepts a
# realistic non-default values file.
#
# WHAT IT DOES NOT PROVE, and the first entry is the one not to misread:
# - THAT jdbc-ping ACTUALLY FOUND THE PEER. Two replicas both reaching Ready
# is NOT that proof, and believing it was is the mistake this paragraph
# exists to prevent: readiness reports this node's own health, so a node
# that never found its peer is Ready, healthy and alone - and two such
# nodes are two islands that each believe they are the cluster. The
# Infinispan view has to be read out of the log, which is what step 2 of
# NOTES.txt tells the operator to do. So measurement point 4 of the plan
# stays open after this test; what the test does rule out is the louder
# half (a second replica that cannot start or cannot share the database).
# - that hostname-admin serves the console cleanly on a second hostname. No
# ingress controller answers in CI and nothing drives a browser, so
# measurement point 5 stays open too. What IS checked is that both Ingress
# objects are accepted, that the Service port they name exists, and that
# Keycloak accepts both hostname values as full URLs at start.
# - anything about an upgrade. The rolling-update restriction between minor
# versions is a property of two versions meeting, and this test installs
# one.
# - that the PostgreSQL here resembles production. It is one pod with an
# emptyDir, so nothing about durability, connection limits or latency
# transfers.
secrets:
# Materialized by ci/keycloak/fixtures.yaml as ci-keycloak-db-secret and
# ci-keycloak-bootstrap-admin-secret (the OnePasswordItem stub CRD comes from
# ci/common/).
databaseSecretRef: op://ci/keycloak-db
bootstrapAdminSecretRef: op://ci/keycloak-bootstrap-admin
keycloak:
# TWO, i.e. the chart's default, and the whole reason this fixture is not a
# one-replica shortcut. With one replica the install-test would be green
# without ever showing whether the second node is found - and "a path the
# test does not walk is a path we have not proven" is the lesson Gitea #80
# paid for. The install-test script waits for the Deployment's rollout, and a
# rollout is only complete when BOTH pods are Ready, so cluster formation is
# inside the assertion rather than next to it.
replicas: 2
db:
# The throwaway PostgreSQL from fixtures.yaml. Plain Service name: both are
# in the same namespace.
host: postgres
database: keycloak
# Small but not degenerate: 2 replicas x 5 = 10 connections against
# postgres:16-alpine's default max_connections of 100, which leaves the
# fixture comfortable while still exercising the pool options.
pool:
initialSize: 1
minSize: 1
maxSize: 5
# The chart default, spelled out because it is load-bearing for what this
# test means: metrics-enabled is what puts the DATABASE check into
# /health/ready (upstream's check table, "Requires Metrics: Yes"). With it
# off, readiness would pass without ever touching the database, and a green
# run would say nothing about the connection at all.
#
# It is also a BUILD option, so with it true the init container has to build
# an installation differing from the published image in three options rather
# than two - which is what makes a build that silently did nothing fail the
# run instead of looking identical.
metrics:
enabled: true
# Set on purpose, although one release cannot collide with itself: it
# exercises the KC_CACHE_EMBEDDED_CLUSTER_NAME path, and a name Keycloak
# rejected would fail the start rather than pass unnoticed.
cache:
clusterName: ci-keycloak
resources:
# Below the chart's defaults to stay inside the runner's budget, and the
# memory number is chosen rather than rounded: the image sizes the heap at
# 70% of the LIMIT, so 600Mi x limitFactor 3 = 1800Mi gives a ~1260Mi heap.
# Upstream's floor is "at least 750 MB" for the limit; going much below
# this makes the JVM the reason a run fails.
requests:
cpu: 250m
memory: 600Mi
ephemeral-storage: 256Mi
build:
resources:
requests:
cpu: 250m
memory: 512Mi
ephemeral-storage: 256Mi
# THE EXTRA VOLUME PAIR, WALKED RATHER THAN RENDERED (Gitea #88). The theme
# ConfigMap comes from fixtures.yaml. Three things this puts inside the
# assertion that no render reaches:
#
# 1. A MOUNT ON A SUBPATH OF A DIRECTORY THAT EXISTS IN THE IMAGE IS
# ACCEPTED ALONGSIDE readOnlyRootFilesystem: true.
# /opt/keycloak/themes/ci-theme does not exist in the image, so the
# runtime has to create the mount point inside a root filesystem that is
# about to be read-only. If that did not work, every container of this
# pod would fail to start.
# 2. `kc.sh build` STILL SUCCEEDS WITH THE EXTRA MOUNT PRESENT. The mounts
# are rendered on the build init container too (that is what makes SPI
# providers work at all), so this fixture is also the test that the extra
# mount does not disturb the augmentation - a failure would leave the pod
# in Init with the reason only in a log.
# 3. `start --optimized` COMES UP WITH A RUNTIME THEME IN PLACE. The plan of
# #88 called that "a reasoned expectation, not verified"; a Ready pod with
# this mounted is the verification, and it is the one that matters for the
# unusual two-init-container construction of this chart.
#
# WHAT IT STILL DOES NOT PROVE, and it would be easy to overclaim here: that
# the LOGIN PAGE renders and that base/keycloak/keycloak.v2 are still listed.
# The health endpoints answer regardless of the themes directory, and nothing
# in this harness makes an HTTP request to / or runs a command in the pod. That
# the subpath hides nothing is a property of the mount path rather than a
# measurement - and the kcLogoIdP line needs a browser plus a realm setting,
# neither of which exists in CI.
extraVolumes:
- name: login-theme
configMap:
name: ci-keycloak-theme
items:
- key: theme.properties
path: login/theme.properties
extraVolumeMounts:
- name: login-theme
# THE SUBPATH, AND THE WHOLE POINT OF THE EXAMPLE: a volume on
# /opt/keycloak/themes would hide the image's own three themes and break
# the login page while the pod stayed Ready.
mountPath: /opt/keycloak/themes/ci-theme
readOnly: true
ingress:
# Left ON although no ingress controller answers in CI: the point is that
# both objects are accepted and that the Service port they name BY NAME
# ("http") actually exists - a typo there is otherwise invisible until
# somebody opens the URL.
enabled: true
domain: keycloak.ci.example.com
clusterIssuer: ci-selfsigned
admin:
# The second hostname, on as well, so the chart renders KC_HOSTNAME_ADMIN
# and Keycloak has to accept it as a full URL. That is the half of
# measurement point 5 a cluster-less test can reach: Keycloak validates
# both hostname values at start ("verified that full URL is used, incl.
# scheme and hostname"), so a chart that built a bare host here would fail
# the start instead of serving a broken console later.
enabled: true
domain: keycloak-admin.ci.example.com
+42 -43
View File
@@ -1,19 +1,11 @@
# Fixtures for the paperless-ngx install-test: a throwaway postgres, the
# secrets the 1Password operator would otherwise materialize (release name:
# ci), and storage plumbing. The chart hardcodes storageClassName
# Fixtures for the paperless-ngx install-test: throwaway postgres + redis,
# the secrets the 1Password operator would otherwise materialize (release
# name: ci), and storage plumbing. The chart hardcodes storageClassName
# "longhorn" (RWX for the consume PVC), which does not exist in k3d: a
# no-provisioner StorageClass named "longhorn" plus pre-bound hostPath PVs
# stand in for it. Paperless runs as uid 1000 with a read-only rootfs and
# hostPath volumes do not support fsGroup, so a node-setup Job pre-creates
# the exact subPath directories world-writable.
#
# THERE IS NO THROWAWAY REDIS HERE ANY MORE, and that is the change this file
# carries: the release brings its own Valkey through the subchart
# (ci/paperless-ngx/test-values.yaml says what that proves), so the broker under
# test is the chart's own object rather than a Deployment next to it. Leaving a
# redis Deployment in place would have been the worse of the two: the url would
# have had to name one of them, and whichever it named, the other would have sat
# there making the run look like it covered something it did not.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
@@ -84,43 +76,12 @@ spec:
path: /var/lib/ci-pv/paperless-ngx/consume
type: DirectoryOrCreate
---
# The two Secrets below stand in for ONE 1Password item. Both the chart's
# redis OnePasswordItem and the subchart's auth OnePasswordItem point at
# op://ci/paperless-redis, and a OnePasswordItem materializes EVERY field of the
# item it names - so in a real cluster both Secrets carry all three fields. They
# are written out that way here rather than trimmed to what each consumer reads,
# because the trimmed version would have been a fixture that is tidier than
# reality and would have hidden the only thing worth knowing: the extra keys are
# unused, not a leak and not a bug.
apiVersion: v1
kind: Secret
metadata:
name: ci-paperless-ngx-redis-secret
stringData:
# Names the SUBCHART's Service, which is the mandatory vault step of this
# configuration (NOTES.txt prints it). A wrong name here is the failure this
# install-test exists to catch: paperless would come up healthy and process
# nothing.
redis-connection-url: redis://:cipass@ci-valkey-service:6379
key-prefix: ci
# Same password as in the url above - that duplication is the chart's
# documented trap 1, and the fixture is the cheapest place to keep it honest:
# let the two drift and the run goes red on authentication.
auth-conf: requirepass cipass
---
# What the subchart's own OnePasswordItem (ci-valkey-secret) would become. The
# key the Valkey actually mounts is "auth-conf", and its value has to be a
# COMPLETE configuration line, because the Valkey configuration includes the
# file: give it anything that is not "requirepass <password>" and the init
# container refuses to start the pod, which is the failure that chart exists to
# make loud.
apiVersion: v1
kind: Secret
metadata:
name: ci-valkey-secret
stringData:
auth-conf: requirepass cipass
redis-connection-url: redis://:cipass@ci-valkey-service:6379
redis-connection-url: redis://:cipass@redis:6379
key-prefix: ci
---
apiVersion: v1
@@ -192,3 +153,41 @@ spec:
ports:
- port: 5432
targetPort: 5432
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
labels:
app: redis
spec:
replicas: 1
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: public.ecr.aws/docker/library/redis:7-alpine
args: ["redis-server", "--requirepass", "cipass"]
ports:
- containerPort: 6379
readinessProbe:
tcpSocket:
port: 6379
periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
name: redis
spec:
selector:
app: redis
ports:
- port: 6379
targetPort: 6379
-69
View File
@@ -1,36 +1,4 @@
# CI values for the paperless-ngx install-test.
#
# THE SUBCHART IS ON, AND THAT IS THE POINT OF THIS FILE. Gitea #65 added the
# same valkey dependency to stalwart but covered it by a local RENDER only -
# deliberately, because Stalwart's in-memory store is an object in its database,
# so a Valkey in that namespace would have been reachable and unused and the
# test would have looked like it covered the clustering path while covering
# nothing of it. Here it is the opposite: paperless-ngx reads PAPERLESS_REDIS
# from the vault item, so pointing that url at the subchart's Service makes the
# bundled Valkey the broker this release actually uses. So this is the first
# install of the valkey-subchart path anywhere, and what it proves is:
#
# 1. The dependency resolves from the published index on a real runner with no
# credentials, and lint/package/render/install all survive it.
# 2. The subchart's objects are accepted inside a parent release: its
# StatefulSet becomes Ready, which means its init container found a
# "requirepass " line in the mounted auth file and its three exec probes
# authenticated and got PONG back.
# 3. The PVC path of the subchart works under a parent release name
# (ci-valkey-*), i.e. the names really are <release>-valkey-<kind>.
# 4. paperless-ngx talks to it: the url in the fixture Secret names
# ci-valkey-service, so a wrong Service name or a password mismatch between
# the two fields of the "item" shows up as a failing run instead of as a
# quietly idle Valkey.
#
# What it still CANNOT prove is that documents are processed - that needs a real
# consume run and is out of scope here, as before.
secrets:
# One item for both, which is the configuration the chart is designed around:
# redis-connection-url, key-prefix AND auth-conf live in the same 1Password
# item. In CI the operator does not exist, so ci/paperless-ngx/fixtures.yaml
# stands in for it - and it stands in TWICE, because a OnePasswordItem
# materializes every field of the item it names. See the fixture.
redisSecretRef: op://ci/paperless-redis
databaseSecretRef: op://ci/paperless-db
paperlessSecretRef: op://ci/paperless
@@ -48,42 +16,5 @@ paperless:
# neither erase them nor break rendering.
barcodes:
valkeySubchart:
enabled: true
valkey:
secrets:
# The SAME op:// path as secrets.redisSecretRef above, typed out again
# because Helm cannot default one value from another (Gitea #78). Typing it
# here is also what exercises the parent's guard in the "satisfied"
# direction; the failing direction is covered by a local render, since the
# harness installs one configuration per chart.
authSecretRef: op://ci/paperless-redis
persistence:
# QUEUE, the chart's and the subchart's default - and the risky half, which
# is why it is not switched off here: a provisioned PVC, the pod's fsGroup
# against a volume it did not create, and an append-only file written and
# replayed on it.
enabled: true
appendonly: true
# k3d's dynamic provisioner. The subchart's default, longhorn-encrypted-ns,
# does not exist on the runner, and the no-provisioner "longhorn" class the
# paperless fixtures define is pre-bound to hostPath PVs for paperless' own
# claims - a hostPath volume gets no fsGroup from the kubelet at all, so
# reusing it here would silently stop testing the ownership path.
storageClass: local-path
size: 128Mi
valkey:
resources:
# Small, to stay inside the runner's budget, while still exercising the
# subchart's derivation: 64Mi x limitFactor 3 = 192Mi limit, maxmemory =
# 0.75 x that = 144Mi.
requests:
cpu: 50m
memory: 64Mi
ephemeral-storage: 32Mi
# smbConnection and oidc stay disabled (defaults): the SMB CSI driver and a
# real OIDC login flow are out of scope for the CI install-test.
+7 -10
View File
@@ -21,11 +21,9 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
CI_DIR="$ROOT/ci"
# --- The exclusion list, parsed in exactly one place ------------------------
# The consumers of ci/excluded-charts.txt are this script and the two jobs that
# build an install-test matrix - detect-changed-charts in
# .github/workflows/validate-charts.yml and select-charts in
# .github/workflows/install-test-all-charts.yml - both of which drop the
# excluded charts before the matrix exists (issue #231). Two parsers would be two
# Two consumers read ci/excluded-charts.txt: this script, and the install-test
# matrix in .github/workflows/validate-charts.yml, which drops the excluded
# charts before the matrix is built (issue #231). Two parsers would be two
# chances to select different charts - and the failure would be silent in the
# worst direction, a chart the workflow believes excluded and the script
# believes testable, or the reverse. So there is one parser, here, and the
@@ -49,10 +47,9 @@ excluded_reason() {
list_excluded | awk -F'\t' -v chart="$1" '$1 == chart { print $2; exit }'
}
# Emits "<chart><TAB><reason>" for every excluded chart. Read by both jobs that
# assemble an install-test matrix (detect-changed-charts in validate-charts.yml,
# select-charts in install-test-all-charts.yml) before any chart is picked,
# which is why this runs before the argument is required.
# Emits "<chart><TAB><reason>" for every excluded chart. Read by the detect job
# of validate-charts.yml before any chart is picked, which is why this runs
# before the argument is required.
if [ "${1:-}" = "--list-excluded" ]; then
list_excluded
exit 0
@@ -103,7 +100,7 @@ fi
# REQUIRED_ENV="FREESAILARR_WG_KEY FREESAILARR_WG_ADDRESSES"
#
# The values come from the environment - in CI from 1Password Connect, see the
# install-test job in .github/workflows/install-test.yml.
# install-test job in .github/workflows/validate-charts.yml.
#
# A missing or empty value is a hard error on purpose. Skipping the chart
# instead would restore exactly the green-but-meaningless signal this mechanism
-461
View File
@@ -1,461 +0,0 @@
#!/usr/bin/env bash
# Selector-match render test: every label selector a chart renders is held
# against the labels of the pod templates in the SAME render.
#
# Usage: ci/run-selector-match-test.sh [chart ...] (no argument = all charts)
#
# WHAT IT GUARDS AGAINST, AND WHY A GREEN ROLLOUT DOES NOT (Gitea #96)
#
# Two kinds of object in this repository select pods by label, and BOTH of them
# fail silently when the selector matches nothing:
#
# topologySpreadConstraints - a constraint whose labelSelector matches no pod
# is TRIVIALLY SATISFIED. It is a condition over zero pods, so every
# placement meets it: nothing is distributed, the rollout is green, no event
# is emitted and no log line appears. With whenUnsatisfiable: DoNotSchedule
# this is worse rather than better, because the object then reads like a
# guarantee that two replicas sit on different nodes.
# PodDisruptionBudget - a budget whose selector matches no pod does its
# arithmetic over zero pods and never blocks an eviction. `kubectl drain`
# takes the whole group at once while the object sits in the namespace
# looking like protection.
#
# Neither is a render error, neither is an install-test failure, and neither is
# visible in a diff: the chart that breaks it is usually a chart that RENAMED a
# label somewhere else. Charts keep their selector and their pod labels in one
# helper for that reason (stalwart.groupLabels, keycloak's app label) - but one
# definition only helps while nobody writes a second one, and that is the part a
# test can hold.
#
# Over ALL charts and not the changed ones, for the same reason as
# ci/run-quantity-render-test.sh: the mistake arrives by copy into a new chart,
# whose own diff nobody reads for it, and running over everything means there is
# no list to maintain - adding a chart enrols it.
#
# HOW THE YAML IS PARSED, AND WHY NOT WITH A PARSER WRITTEN HERE
#
# The runners have neither pyyaml nor yq - measured, the same constraint
# ci/run-trivy-config.sh and ci/run-quantity-render-test.sh work under. Writing
# a YAML reader in awk or sed was the obvious alternative and is the wrong one:
# its failure mode is a parser that quietly reads nothing, finds no selector,
# reports no mismatch, and leaves this test green for a chart it never looked
# at - the exact class of signal this repository removes rather than adds.
#
# So HELM is the parser, which is a YAML implementation that is already
# installed, already trusted, and already the one that produced the manifests.
# ci/yaml2json is a one-template chart that prints its values back as JSON; the
# rendered manifest stream is handed to it as a VALUES FILE and comes back as
# JSON that python3 reads with its standard library. What IS done textually is
# only the document split, on a line that is exactly "---" in column one, and
# the consequence of getting that wrong is not a false pass: the fragment stops
# being valid YAML and helm exits 1.
#
# WHAT IS DELIBERATELY NOT CHECKED
#
# - podAffinity / podAntiAffinity. They carry labelSelectors too, but their
# semantics are "relative to pods that exist", and an anti-affinity term
# matching nothing is a legitimate configuration rather than a mistake.
# Out of scope, and named so that the gap is not mistaken for coverage.
# - selectors expressed ONLY through matchExpressions. Evaluating a set-based
# requirement means reimplementing label-selector semantics, which is a
# second parser by another name. Those are REPORTED as "not checked" and
# are never counted as passing, because a green tick for a code path nobody
# entered is worse than an honest gap (README, "Two rules for every check
# added here").
#
# AND THE ANTI-VACUUM GUARDS, because this test can be green over nothing. A
# chart renders a PDB or a spread constraint only above a replica threshold, and
# every chart's CI fixtures sit below it. Two rules therefore apply:
#
# 1. ci/selector-match/<chart>.yaml is an overlay applied last, whose only job
# is to push a chart over that threshold. A chart that HAS one and still
# renders no selector FAILS - the overlay exists for exactly that, so an
# overlay that stopped working is a finding and not a shrug.
# 2. If no selector at all was checked across the whole run, the run fails.
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
CI_DIR="$ROOT/ci"
YAML2JSON="$CI_DIR/yaml2json"
OVERLAY_DIR="$CI_DIR/selector-match"
RELEASE="selector"
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
# --- Which charts -----------------------------------------------------------
CHARTS=()
if [ "$#" -gt 0 ]; then
CHARTS=("$@")
else
for DIR in "$ROOT"/*/; do
[ -f "$DIR/Chart.yaml" ] || continue
CHARTS+=("$(basename "$DIR")")
done
fi
if [ "${#CHARTS[@]}" -eq 0 ]; then
echo "ERROR: no chart directories found under $ROOT" >&2
exit 1
fi
FAILED=""
CHECKED=""
NOT_EXERCISED=""
WITH_GAPS=""
TOTAL_SELECTORS=0
SUMMARY_ROWS=""
for CHART in "${CHARTS[@]}"; do
if [ ! -f "$ROOT/$CHART/Chart.yaml" ]; then
echo "ERROR: '$CHART' is not a chart directory" >&2
exit 1
fi
# Same reason as in ci/run-quantity-render-test.sh: a chart with a declared
# dependency cannot be rendered at all while charts/ is empty, and this job
# runs over EVERY chart on EVERY pull request - so without this line a PR
# touching nothing but a README would go red on the charts that have one.
"$ROOT/ci/helm-dependencies.sh" "$ROOT/$CHART"
# Values: the same precedence the other render tests use, so a chart's CI
# values are maintained in one place and satisfy its required() values.
VALUES_ARGS=()
VALUES_LABEL="chart defaults"
if [ -f "$CI_DIR/$CHART/test-values.yaml" ]; then
VALUES_ARGS=(-f "$CI_DIR/$CHART/test-values.yaml")
VALUES_LABEL="ci/$CHART/test-values.yaml"
elif [ -f "$CI_DIR/trivy/values/$CHART.yaml" ]; then
VALUES_ARGS=(-f "$CI_DIR/trivy/values/$CHART.yaml")
VALUES_LABEL="ci/trivy/values/$CHART.yaml"
fi
HAS_OVERLAY=""
if [ -f "$OVERLAY_DIR/$CHART.yaml" ]; then
# Last, so it wins over the install-test values.
VALUES_ARGS+=(-f "$OVERLAY_DIR/$CHART.yaml")
VALUES_LABEL="$VALUES_LABEL + ci/selector-match/$CHART.yaml"
HAS_OVERLAY=1
fi
echo "::group::$CHART ($VALUES_LABEL)"
RENDER="$WORK/$CHART.manifest.yaml"
if ! helm template "$RELEASE" "$ROOT/$CHART" \
${VALUES_ARGS[@]+"${VALUES_ARGS[@]}"} >"$RENDER" 2>"$WORK/err"; then
sed 's/^/ /' "$WORK/err"
echo "::endgroup::"
echo "::error title=selector-match $CHART::$CHART does not render with its CI values, so no selector could be checked. Reproduce: helm template $RELEASE $CHART $(printf '%s ' "${VALUES_ARGS[@]+${VALUES_ARGS[@]}}")"
FAILED="$FAILED $CHART"
continue
fi
# --- The document split, and the only textual step in this script ----------
# One YAML sequence item per rendered document, content indented by four and
# the item marker on a line of its own - so a document whose first line is a
# comment ("# Source: ...") needs no special case. A wrong split does not
# produce a false pass: the result stops being parsable and helm exits 1.
DOCS="$WORK/$CHART.docs.yaml"
awk '
BEGIN { print "docs:"; started = 0 }
/^---[ \t]*$/ { print " -"; started = 1; next }
{
if (!started) { print " -"; started = 1 }
if (length($0) == 0) print ""; else print " " $0
}
' "$RENDER" >"$DOCS"
JSON="$WORK/$CHART.json"
if ! helm template y2j "$YAML2JSON" -f "$DOCS" 2>"$WORK/err" \
| awk '/^\{/ { print; exit }' >"$JSON"; then
sed 's/^/ /' "$WORK/err"
echo "::endgroup::"
echo "::error title=selector-match $CHART::the rendered manifests of $CHART could not be re-read as YAML by ci/yaml2json. That is a parse failure, not a selector finding: either the render contains a document this script's '---' split cut in the wrong place, or the manifest is not valid YAML. Helm's own message is in the job log."
FAILED="$FAILED $CHART"
continue
fi
if [ ! -s "$JSON" ]; then
echo "::endgroup::"
echo "::error title=selector-match $CHART::ci/yaml2json produced no JSON for $CHART - the manifests were not re-read, so nothing was checked."
FAILED="$FAILED $CHART"
continue
fi
# --- The comparison --------------------------------------------------------
# Inline, as in ci/run-quantity-render-test.sh, and on the standard library
# only - json, because the YAML is already gone by this point. It writes one
# "VERDICT|detail" line per selector and never decides the exit code itself:
# the loop below does, so a verdict can never be lost to a swallowed status.
set +e
python3 - "$CHART" "$JSON" >"$WORK/$CHART.out" 2>"$WORK/$CHART.pyerr" <<'PY'
import json, sys
chart, path = sys.argv[1], sys.argv[2]
docs = json.load(open(path)).get("docs") or []
# A pod template is any value of a key named "template" that carries a spec with
# containers. Found by walking rather than by enumerating kinds, so a CronJob's
# spec.jobTemplate.spec.template is covered without a special case, and so is
# whatever workload kind a chart adds next.
def pod_templates(node, path="$"):
out = []
if isinstance(node, dict):
for key, value in node.items():
here = "%s.%s" % (path, key)
if (key == "template" and isinstance(value, dict)
and isinstance(value.get("spec"), dict)
and "containers" in value["spec"]):
out.append((here, value))
out.extend(pod_templates(value, here))
elif isinstance(node, list):
for i, value in enumerate(node):
out.extend(pod_templates(value, "%s[%d]" % (path, i)))
return out
def labels_of(template):
meta = template.get("metadata")
labels = meta.get("labels") if isinstance(meta, dict) else None
return labels if isinstance(labels, dict) else {}
def name_of(doc):
meta = doc.get("metadata")
if isinstance(meta, dict) and meta.get("name"):
return str(meta["name"])
return "<unnamed %s>" % doc.get("kind", "object")
def matches(selector, labels):
"""matchLabels only. Every key must be present with the same value; the
comparison is on the string form, because YAML 'true' and the string
"true" are the same label value to Kubernetes."""
for key, value in selector.items():
if key not in labels or str(labels[key]) != str(value):
return False
return True
def classify(selector, where):
"""(matchLabels dict, None) when it can be evaluated, (None, reason) when
it cannot. A selector this script cannot evaluate is REPORTED, never
counted as a pass."""
if selector is None:
return None, ("%s has NO selector at all. For a topology spread "
"constraint that means it matches no pod and is "
"therefore vacuously satisfied; for a PodDisruptionBudget "
"it is rejected by the API server" % where)
if not isinstance(selector, dict):
return None, "%s selector is not a mapping (%r)" % (where, selector)
match_labels = selector.get("matchLabels")
if isinstance(match_labels, dict) and match_labels:
if selector.get("matchExpressions"):
# matchLabels AND matchExpressions: the matchLabels half is still
# a necessary condition, so checking it is sound - it just is not
# the whole selector. Said out loud rather than passed off as full
# coverage.
return match_labels, None
return match_labels, None
if selector.get("matchExpressions"):
return None, ("%s selects only through matchExpressions, which this "
"test does not evaluate - doing so would mean "
"reimplementing label-selector semantics, i.e. a second "
"parser. Reported, not counted as passing" % where)
if match_labels == {} or selector == {}:
return {}, None # the empty selector matches every pod
return None, "%s selector has neither matchLabels nor matchExpressions" % where
# Rooted at "<Kind>/<name>" rather than at "$", because a chart with one
# workload per node group renders several documents whose internal path is
# identical - and then every finding would name the same spot.
all_templates = []
for doc in docs:
if isinstance(doc, dict):
root = "%s/%s" % (doc.get("kind", "?"), name_of(doc))
all_templates.extend(pod_templates(doc, root))
lines = []
# 1. Every topologySpreadConstraints entry, against ITS OWN pod template. That
# is the strict and the correct comparison: the failure being guarded
# against is a constraint that does not count the very pods it sits on.
for where, template in all_templates:
spread = template.get("spec", {}).get("topologySpreadConstraints")
if not isinstance(spread, list):
continue
labels = labels_of(template)
for i, constraint in enumerate(spread):
if not isinstance(constraint, dict):
lines.append("FAIL|%s: topologySpreadConstraints[%d] at %s is not a mapping"
% (chart, i, where))
continue
key = constraint.get("topologyKey", "<none>")
tag = "spread[%d] on %s at %s" % (i, key, where)
match_labels, reason = classify(constraint.get("labelSelector"), tag)
if match_labels is None:
if constraint.get("labelSelector") is None:
lines.append("FAIL|%s: %s" % (chart, reason))
else:
lines.append("SKIP|%s: %s" % (chart, reason))
continue
if matches(match_labels, labels):
lines.append("OK|%s: %s matches its own pod template's labels (%s)"
% (chart, tag, ", ".join("%s=%s" % kv for kv in sorted(match_labels.items())) or "empty selector, matches every pod"))
else:
lines.append("FAIL|%s: %s selects %s but the pod template it is "
"attached to carries %s - the constraint therefore "
"matches NONE of its own pods, is trivially satisfied, "
"and distributes nothing while the rollout stays green. "
"Render the chart and compare the selector with the pod "
"template's labels; the usual cause is a label renamed "
"in one of the two places"
% (chart, tag,
", ".join("%s=%s" % kv for kv in sorted(match_labels.items())),
", ".join("%s=%s" % kv for kv in sorted(labels.items())) or "no labels at all"))
# 2. Every PodDisruptionBudget, against ALL pod templates of the render: a
# budget has no owning template, it only has to select some pod that exists.
pdbs = [d for d in docs if isinstance(d, dict) and d.get("kind") == "PodDisruptionBudget"]
for doc in pdbs:
tag = "PodDisruptionBudget %s" % name_of(doc)
spec = doc.get("spec") if isinstance(doc.get("spec"), dict) else {}
match_labels, reason = classify(spec.get("selector"), tag)
if match_labels is None:
lines.append(("FAIL|%s: %s" if spec.get("selector") is None else "SKIP|%s: %s")
% (chart, reason))
continue
if not all_templates:
lines.append("FAIL|%s: %s is rendered but this render contains no pod "
"template at all, so its selector cannot match anything"
% (chart, tag))
continue
hits = [w for w, t in all_templates if matches(match_labels, labels_of(t))]
if hits:
lines.append("OK|%s: %s matches %d pod template(s): %s"
% (chart, tag, len(hits), ", ".join(hits)))
else:
known = sorted({"%s=%s" % kv for _, t in all_templates for kv in labels_of(t).items()})
lines.append("FAIL|%s: %s selects %s, which matches NONE of the %d pod "
"template(s) in the same render. A PDB that matches no pod "
"never blocks an eviction, so a node drain takes every "
"replica at once while the object looks like protection. "
"Labels actually present on the pod templates: %s"
% (chart, tag,
", ".join("%s=%s" % kv for kv in sorted(match_labels.items())),
len(all_templates), "; ".join(known) or "none"))
for line in lines:
print(line)
PY
RC=$?
set -e
if [ "$RC" -gt 1 ]; then
sed 's/^/ /' "$WORK/$CHART.pyerr"
echo "::endgroup::"
echo "::error title=selector-match $CHART::the selector comparison itself failed for $CHART (python exit $RC). Treated as a failure, never as a pass."
FAILED="$FAILED $CHART"
continue
fi
CHART_FAIL=""
CHART_OK=0
CHART_SKIP=0
while IFS='|' read -r VERDICT DETAIL; do
case "$VERDICT" in
OK)
CHART_OK=$((CHART_OK + 1))
echo " ok $DETAIL"
;;
SKIP)
CHART_SKIP=$((CHART_SKIP + 1))
echo " gap $DETAIL"
;;
FAIL)
CHART_FAIL=1
echo " FAIL $DETAIL"
echo "::error title=selector-match $CHART::$DETAIL"
;;
*)
[ -n "$VERDICT" ] && echo " $VERDICT $DETAIL"
;;
esac
done <"$WORK/$CHART.out"
echo "::endgroup::"
TOTAL_SELECTORS=$((TOTAL_SELECTORS + CHART_OK))
if [ -n "$CHART_FAIL" ]; then
FAILED="$FAILED $CHART"
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | **failed** | $CHART_OK | $CHART_SKIP |"$'\n'
echo "$CHART: FAILED"
elif [ "$CHART_OK" -gt 0 ]; then
CHECKED="$CHECKED $CHART"
[ "$CHART_SKIP" -gt 0 ] && WITH_GAPS="$WITH_GAPS $CHART"
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | ok | $CHART_OK | $CHART_SKIP |"$'\n'
echo "$CHART: ok ($CHART_OK selector(s) matched, $CHART_SKIP not checked)"
elif [ -n "$HAS_OVERLAY" ]; then
# The overlay's ONE job is to push this chart over the threshold at which
# it renders a selector. If it did not, this test is vacuum-green for the
# chart it was written for, which is the failure the overlay prevents.
FAILED="$FAILED $CHART"
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | **failed** | 0 | $CHART_SKIP |"$'\n'
echo "::error title=selector-match $CHART::$CHART has an overlay at ci/selector-match/$CHART.yaml but rendered no PodDisruptionBudget and no topologySpreadConstraints, so nothing was checked. That overlay exists only to make those objects render - either it no longer reaches the threshold (replica count, a feature switch) or the chart stopped rendering them. A green tick here would be a tick for a check that ran over nothing."
echo "$CHART: FAILED (overlay present, no selector rendered)"
else
NOT_EXERCISED="$NOT_EXERCISED $CHART"
[ "$CHART_SKIP" -gt 0 ] && WITH_GAPS="$WITH_GAPS $CHART"
echo "$CHART: no PodDisruptionBudget and no topologySpreadConstraints rendered - nothing to check."
fi
done
# --- Report ------------------------------------------------------------------
count() { set -- $1; echo "$#"; }
listing() { set -- $1; [ "$#" -eq 0 ] && { echo "–"; return; }; printf '`%s`' "$1"; shift; for C in "$@"; do printf ', `%s`' "$C"; done; echo; }
SUMMARY="${GITHUB_STEP_SUMMARY:-/dev/null}"
{
echo "### Selector-match render test (Gitea #96)"
echo
echo "Every \`PodDisruptionBudget\` selector and every \`topologySpreadConstraints\` label"
echo "selector in a chart's render, held against the labels of the pod templates in the"
echo "**same** render. A selector that matches no pod is not a render error and not an"
echo "install-test failure: the spread constraint is then trivially satisfied and the"
echo "budget never blocks an eviction, both silently."
echo
echo "| Result | Count | Charts |"
echo "|---|---|---|"
echo "| Selectors checked and matching | $(count "$CHECKED") | $(listing "$CHECKED") |"
echo "| No such object rendered | $(count "$NOT_EXERCISED") | $(listing "$NOT_EXERCISED") |"
echo "| Failed | $(count "$FAILED") | $(listing "$FAILED") |"
echo
echo "$TOTAL_SELECTORS selector(s) verified in total."
if [ -n "$WITH_GAPS" ]; then
echo
echo "Charts with selectors this test does **not** evaluate (\`matchExpressions\` only — reported, never counted as passing): $(listing "$WITH_GAPS")."
fi
if [ -n "$SUMMARY_ROWS" ]; then
echo
echo "| Chart | Result | Checked | Not checked |"
echo "| --- | --- | --- | --- |"
printf '%s' "$SUMMARY_ROWS"
fi
} >>"$SUMMARY"
echo "===================== selector match test ======================"
echo "checked: $(count "$CHECKED")${CHECKED:+ -$CHECKED}"
echo "not exercised: $(count "$NOT_EXERCISED")${NOT_EXERCISED:+ -$NOT_EXERCISED}"
echo "failed: $(count "$FAILED")${FAILED:+ -$FAILED}"
echo "selectors verified: $TOTAL_SELECTORS"
echo "================================================================"
if [ -n "$FAILED" ]; then
echo "ERROR: selectors do not match the pod labels they are meant to select in:$FAILED" >&2
exit 1
fi
# The second anti-vacuum guard. Reaching zero means every chart fell into "no
# such object rendered", which is indistinguishable from a broken harness - and
# it is the state this test would silently decay into if the overlays were ever
# dropped.
if [ "$TOTAL_SELECTORS" -eq 0 ]; then
echo "ERROR: not a single selector was checked in this run. Either no chart renders a PodDisruptionBudget or a topologySpreadConstraints any more, or the overlays under ci/selector-match/ no longer push one over its replica threshold. A pass over zero assertions is not a pass." >&2
exit 1
fi
-38
View File
@@ -1,38 +0,0 @@
# Overlay for ci/run-selector-match-test.sh, applied ON TOP of
# ci/stalwart/test-values.yaml and nowhere else - not by the install-test, not by
# the Trivy scan, not by the quantity render test.
#
# WHY IT EXISTS. Both objects this test looks at are rendered only while a node
# group's COMPUTED replica count is >= 2, and the install-test fixture runs
# full: 1 / mx: 0 deliberately (a second pod would need a real in-memory store,
# and a running "mx" pod would fail-open into a second full node against an
# empty database). So without this file stalwart renders no PodDisruptionBudget
# and no topologySpreadConstraints, and the test would be green for the chart it
# was written for, having looked at nothing. ci/run-selector-match-test.sh
# therefore FAILS a chart that has an overlay and still renders no selector.
#
# This is a RENDER-ONLY overlay. Nothing here is installed anywhere, so the two
# reasons the install-test keeps mx at 0 do not apply.
nodes:
full:
replicas: 2
exposedPorts: [smtp, submissions, imaps, http]
mx:
# Two as well, so BOTH groups' selectors are checked. The per-group split is
# the whole point of this chart's version of the rule: one object per group,
# each selecting only its own pods. A single group would leave the case where
# a group's selector accidentally matches the OTHER group's pods untested.
replicas: 2
exposedPorts: [smtp, http]
externalServices:
valkey:
# REQUIRED HERE, and not cosmetic: with more than one Stalwart pod and no
# in-memory store address the chart ABORTS THE RENDER ("this release runs N
# Stalwart pods, which is a cluster, but no in-memory store has an
# address"). Nothing connects to this name - the render never resolves it -
# but without it this overlay would turn the whole job red instead of
# exercising it. valkeySubchart.enabled: true would be the other way out and
# is deliberately not taken: it would add a network fetch of the published
# valkey chart to a test that otherwise needs none.
host: valkey-selector-match-fixture
-104
View File
@@ -1,104 +0,0 @@
# Fixtures for the stalwart install-test: a throwaway Postgres, the one Secret
# the 1Password operator would otherwise materialize, and a stub CRD for
# cert-manager's Certificate (release name: ci; the OnePasswordItem stub CRD
# comes from ci/common/).
#
# NO VALKEY HERE, DELIBERATELY. The in-memory store is an object in Stalwart's
# DATABASE, not a chart value, so a Valkey in this namespace would be reachable
# and unused - it would make the test look like it covered the clustering path
# while covering nothing of it. The test runs a single node (nodes.full
# replicas 1, nodes.mx replicas 0), which is the configuration that needs no
# in-memory store at all. The subchart path is covered by a local render only,
# named as a known gap in the PR that introduced this chart.
#
# A stub CRD for Certificate rather than certificate.enabled: false, because
# switching the Certificate off would also drop the volume and volumeMount that
# carry it - and the mount is on the risky side of readOnlyRootFilesystem. With
# the stub the object applies, the Secret it names never appears, and the
# optional: true on the volume is exercised for real: the pods must start
# without it. That is the behaviour an operator meets on every first install,
# before cert-manager has answered the challenge.
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: certificates.cert-manager.io
spec:
group: cert-manager.io
names:
kind: Certificate
listKind: CertificateList
plural: certificates
singular: certificate
scope: Namespaced
versions:
- name: v1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
x-kubernetes-preserve-unknown-fields: true
---
apiVersion: v1
kind: Secret
metadata:
name: ci-stalwart-datastore-secret
stringData:
# The ONLY field this chart reads from the data store item. config.json
# references it by variable name through
# "authSecret": {"@type":"EnvironmentVariable","variableName":"STALWART_DB_PASSWORD"}
# so this value has to match the throwaway Postgres below. A real password
# rather than an empty string on purpose: upstream's resolver treats an empty
# environment variable as "not found" and fails the store open, so an empty
# value here would fail the run for the right reason in the wrong place.
password: ci-stalwart-throwaway-password
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
labels:
app: postgres
spec:
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: public.ecr.aws/docker/library/postgres:16-alpine
env:
- name: POSTGRES_USER
value: stalwart
- name: POSTGRES_PASSWORD
value: ci-stalwart-throwaway-password
- name: POSTGRES_DB
value: stalwart
ports:
- containerPort: 5432
readinessProbe:
exec:
command: ["pg_isready", "-U", "stalwart"]
periodSeconds: 5
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
volumes:
- name: data
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: postgres
spec:
selector:
app: postgres
ports:
- port: 5432
targetPort: 5432
-123
View File
@@ -1,123 +0,0 @@
# CI values for the stalwart install-test.
#
# WHAT THIS TEST CAN AND CANNOT PROVE — read this before trusting a green run.
#
# It CANNOT prove a working mail server, and no install-test here could. Since
# v0.16 every listener, the blob store, the in-memory store, the TLS object and
# the cluster roles live in Stalwart's DATABASE and are created by a human in
# the WebUI. Against an empty database there is no SMTP service to test, so a
# green run says nothing about mail.
#
# What it DOES prove is the part that is the chart's own responsibility, and the
# first item is the reason this chart is tested rather than excluded:
#
# 1. THE CONTAINER EXECS AT ALL. The image carries
# "setcap cap_net_bind_service=+ep" on its binary, which makes the kernel
# fail execve itself when the capability is not in the bounding set. The
# first draft of this chart had capabilities.drop: [ALL] with nothing
# added and would have failed with "operation not permitted" in every
# install - caught by reading the Dockerfile, but exactly the class of
# failure an install-test exists to catch rather than reason about.
# 2. readOnlyRootFilesystem: true holds with the two image volumes plus /tmp
# as emptyDirs, and the init container really can write config.json into
# the /etc/stalwart emptyDir.
# 3. The rendered config.json parses AS STALWART READS IT, not just as JSON:
# the server opens the data store, which means "@type": "PostgreSql", the
# integer-millisecond timeout and the authSecret EnvironmentVariable
# indirection are all accepted. This is the one check that the password
# really arrives without ever being in the ConfigMap.
# 4. The Secret key names match what the templates reference, the ConfigMap
# mount works, the NodePorts can be allocated, the resource quantities are
# valid, the schema accepts a realistic (non-default) values file, and the
# NetworkPolicy objects are accepted without cutting off the probes.
#
# RECOVERY MODE IS ON, and that is not a shortcut - it is the only mode that can
# work here. The bootstrap wizard appears only when config.json is ABSENT, and
# this chart always renders one; upstream's documented path for a templated
# config.json is to start once with STALWART_RECOVERY_MODE=1. Recovery mode
# brings up the management HTTP endpoint and keeps the mail ports closed, which
# is precisely the reachable surface the probes use.
#
# THE ONE THING THIS FILE COULD NOT BE VERIFIED AGAINST: whether /healthz/live
# and /healthz/ready are served on the RECOVERY listener. The source shows them
# handled in the general HTTP request path with no auth and no network
# restriction, but recovery mode binds a single listener and skips most config
# parsing, and there is no cluster in the authoring environment to try it on.
# If the install-test goes red with the pods never becoming Ready, that is the
# reason, and the fix is either an exec probe or an entry in
# ci/excluded-charts.txt - see the PR for #65.
bootstrap:
enabled: true
logLevel: debug
dataStore:
postgres:
host: postgres
database: stalwart
username: stalwart
tls:
# OFF, against the chart's own default of true. The throwaway Postgres in
# fixtures.yaml serves no TLS, and the alternative - allowInvalidCerts -
# would be worse: it would pass whether or not TLS worked, so a chart
# change that broke the TLS path would still go green. A deviation that
# fails loudly beats one that hides.
enable: false
allowInvalidCerts: false
secrets:
# Materialized by ci/stalwart/fixtures.yaml as ci-stalwart-datastore-secret.
dataStoreSecretRef: op://ci/stalwart-datastore
# Both deliberately unset: neither is needed to bring a pod up, and leaving
# them out exercises the conditional env and OnePasswordItem blocks in the
# "absent" direction, which is the shape a minimal install really has.
recoveryAdminSecretRef: null
s3SecretRef: null
ingress:
# Left ON although no ingress controller answers in CI: the point is that the
# object is accepted and that the per-group http Service it names actually
# exists - a selector typo there is otherwise invisible until someone opens
# the URL.
enabled: true
domain: stalwart.ci.example.com
clusterIssuer: ci-selfsigned
nodeGroup: full
certificate:
# On, against a STUB CRD (fixtures.yaml). Nothing issues the Secret it names,
# which is the interesting case: the volume is mounted optional, so the pods
# must start without it. That is what every real first install looks like
# before cert-manager has answered the challenge.
enabled: true
clusterIssuer: ci-selfsigned
networkPolicy:
# Left ON. probeSources stays empty, so the http rule renders without a
# "from" selector - the configuration this chart ships. Whether the policy
# actually FILTERS is a property of the cluster's CNI and is not claimed by
# this test; what is claimed is that the objects are accepted and that they
# do not keep the kubelet's probes from working.
enabled: true
nodes:
full:
replicas: 1
exposedPorts: [smtp, submissions, imaps, http]
resources:
# Small, to stay inside the runner's budget, while still exercising the
# limit derivation: 256Mi x limitFactor 3 = 768Mi.
requests:
cpu: 100m
memory: 256Mi
ephemeral-storage: 64Mi
mx:
# Stays at 0, which is also the chart default. Two reasons rather than one:
# a second node would need an in-memory store (the render aborts without
# one), and the "mx" ClusterRole does not exist in an empty database, so a
# running mx pod would fail-open into a second full node and the two would
# fight over the same NodePort-backed SMTP Service. The StatefulSet is
# still created and the harness still waits on it - a 0-replica rollout
# completes immediately, so this covers the explicit-zero path of the
# replicas helper.
replicas: 0
exposedPorts: [smtp, http]
-129
View File
@@ -1,129 +0,0 @@
# Fixtures for the sure install-test: throwaway PostgreSQL 16 (the version
# upstream's compose file uses) and Redis, plus the secrets the 1Password
# operator would otherwise materialize (release name: ci).
#
# Both are mandatory for this chart, and for different reasons: the web
# container migrates the schema at start and its readiness probe runs a query,
# while all three probes of the worker container ask Redis whether the Sidekiq
# process still has a heartbeat. A missing Redis therefore does not show up as
# "a background job is late" but as a pod that never turns Ready.
apiVersion: v1
kind: Secret
metadata:
name: ci-sure-secret
stringData:
# 64 bytes hex, as upstream's "openssl rand -hex 64" produces. Any value
# works for a throwaway database - it is the key the app derives its Active
# Record encryption keys from, and nothing here survives the run.
secret-key-base: 3f786850e387550fdab836ed7e6dc881de23001b3f786850e387550fdab836ed7e6dc881de23001b3f786850e387550fdab836ed7e6dc881de23001b3f786850
---
apiVersion: v1
kind: Secret
metadata:
name: ci-sure-db-secret
stringData:
host: postgres
port: "5432"
database: sure
username: ci
password: ci
---
apiVersion: v1
kind: Secret
metadata:
name: ci-sure-redis-secret
stringData:
connection-url: redis://:cipass@redis:6379/0
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
labels:
app: postgres
spec:
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: public.ecr.aws/docker/library/postgres:16-alpine
env:
- name: POSTGRES_USER
value: ci
- name: POSTGRES_PASSWORD
value: ci
- name: POSTGRES_DB
value: sure
ports:
- containerPort: 5432
readinessProbe:
# -d names the database the chart actually uses. Without it
# pg_isready asks for a database named after the user and the
# server logs 'FATAL: database "ci" does not exist' every five
# seconds - noise that reads like a chart problem in the diagnostics
# dump when something else fails.
exec:
command: ["pg_isready", "-U", "ci", "-d", "sure"]
periodSeconds: 5
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
volumes:
- name: data
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: postgres
spec:
selector:
app: postgres
ports:
- port: 5432
targetPort: 5432
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
labels:
app: redis
spec:
replicas: 1
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: public.ecr.aws/docker/library/redis:7-alpine
args: ["redis-server", "--requirepass", "cipass"]
ports:
- containerPort: 6379
readinessProbe:
tcpSocket:
port: 6379
periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
name: redis
spec:
selector:
app: redis
ports:
- port: 6379
targetPort: 6379
-6
View File
@@ -1,6 +0,0 @@
# The first start of the web container creates and migrates the whole Rails
# schema ("./bin/rails db:prepare", run by the image entrypoint) before Puma
# binds its port, and the worker container only registers with Sidekiq once
# that schema exists. Both startup probes budget 600s for it, so the rollout
# wait has to be longer than a single startup budget, not shorter.
INSTALL_TIMEOUT=900
-10
View File
@@ -1,10 +0,0 @@
secrets:
appSecretRef: op://ci/sure-app
dbSecretRef: op://ci/sure-db
redisSecretRef: op://ci/sure-redis
ingress:
domain: sure.ci.local
# clusterIssuer is strictly required (no default); the Ingress just has to
# apply - there is no cert-manager in the CI cluster.
clusterIssuer: ci
-6
View File
@@ -20,9 +20,3 @@ ksv0125:
- cr.jk-dev.de
- ghcr.io
- docker.io
# quay.io, added with the keycloak chart (Gitea #83) - Keycloak publishes
# its official image only there (quay.io/keycloak/keycloak), and it is so
# far the only chart in this repo using that registry. Listed rather than
# suppressed, for the reason the comment above gives: the check keeps being
# able to say something about an image from a genuinely foreign source.
- quay.io
+1 -14
View File
@@ -24,7 +24,7 @@ misconfigurations:
statement: "No CPU limits, on purpose: CPU is compressible, so a limit throttles where the missing limit merely shares - see README 'Resources (auto-limits)'. The chart helper renders a CPU limit only when one is set explicitly."
- id: AVD-KSV-0020
statement: "UIDs are pinned to the value the image expects, not raised above 10000: minecraft-server 1000 (gosu default), teamspeak-server 9987, voucher-vault 33, the matrix-synapse delegation nginx 101, sure 1000 (its Dockerfile declares USER 1000:1000). calibre-web-automated shows the risk - a runtime uid moving from 1000 to 911 makes existing volume data inaccessible, and every chart with a PVC carries that same risk class."
statement: "UIDs are pinned to the value the image expects, not raised above 10000: minecraft-server 1000 (gosu default), teamspeak-server 9987, voucher-vault 33, the matrix-synapse delegation nginx 101. calibre-web-automated shows the risk - a runtime uid moving from 1000 to 911 makes existing volume data inaccessible, and every chart with a PVC carries that same risk class."
- id: AVD-KSV-0021
statement: "GIDs follow the pinned UIDs above; the same image expectation and the same PVC ownership risk apply."
@@ -109,19 +109,6 @@ misconfigurations:
paths: ["**/homeassistant.yaml"]
statement: "NET_RAW is the capability in question and is added on purpose for device discovery; see AVD-KSV-0022."
# --- valkey: the two keyword findings on its ConfigMap ---------------------
# Both are keyword matches on the COMMENTS and the shell script in the
# ConfigMap, not on data. Worth suppressing with a reason rather than leaving
# standing, because "ConfigMap stores secrets" on this chart of all charts
# says the opposite of what it does: the whole design exists to keep the
# password out of everything Helm renders.
- id: AVD-KSV-0109
paths: ["**/valkey.yaml"]
statement: "False positive on the rendered ConfigMap. The matched strings are the comment '# Nothing here prints the password' and the line pass=\"$(sed -n 's/^requirepass...' \"$AUTH_FILE\")\" of the probe script - i.e. the code that READS the password out of the mounted Secret file at probe time. The ConfigMap contains no credential: the password lives only in the 1Password item, is materialized into a Secret by the operator, mounted as a file and 'include'd by the configuration, which is precisely why no requirepass line is rendered here. Verify by rendering the chart - no value from the vault appears in it."
- id: AVD-KSV-01010
paths: ["**/valkey.yaml"]
statement: "Same class as AVD-KSV-0109 above: the matched keys are 'valkey.conf' and an 'echo \"valkey...' line of the validation script. Both are chart-authored configuration and diagnostics text; nothing user- or credential-derived is stored in this ConfigMap."
# --- rss-bridge: port 80 is fixed by the image ----------------------------
- id: AVD-KSV-0117
paths: ["**/rss-bridge.yaml"]
-25
View File
@@ -1,25 +0,0 @@
# Fixtures for the valkey install-test: the one Secret the 1Password operator
# would otherwise materialize from the OnePasswordItem the chart renders
# (release name: ci, stub CRD in ci/common/).
#
# The field name matters as much as the value. The chart mounts exactly the key
# "auth-conf" and its content has to be a COMPLETE Valkey configuration line,
# because the configuration "include"s the file - so this fixture is also what
# keeps the init container's check honest: give it anything that is not a
# "requirepass <password>" line and the pod refuses to start, which is the
# failure this chart exists to make loud.
#
# No throwaway infrastructure here: Valkey depends on nothing. What this
# install-test does exercise, and why it is the persistent variant, is the risky
# path - a provisioned PVC, the pod's fsGroup against it, and an append-only
# file being created and replayed on a volume the container did not create.
apiVersion: v1
kind: Secret
metadata:
name: ci-valkey-secret
stringData:
# A throwaway password. Nothing here survives the run - but it is a real
# password rather than an empty string on purpose: an empty one would produce
# exactly the open server the chart refuses to start, and the run would fail
# for the right reason at the wrong place.
auth-conf: requirepass ci-valkey-throwaway-password
-37
View File
@@ -1,37 +0,0 @@
# CI values for the valkey install-test. The PERSISTENT variant on purpose: it is
# the chart's default and the only risky path there is (a provisioned PVC, the
# pod's fsGroup against it, an append-only file created on a volume the container
# did not create). The machinery can run one variant per chart, so the
# non-persistent path is covered by a local render only - named as a known gap in
# the PR that introduced this chart.
secrets:
authSecretRef: op://ci/valkey
persistence:
enabled: true
appendonly: true
# k3d's dynamic provisioner. The chart's default, longhorn-encrypted-ns, does
# not exist on the runner, and a pre-bound hostPath PV would be worse than this
# one: the kubelet does not apply fsGroup to a hostPath volume at all, so the
# test would silently stop exercising the ownership path it is here for.
storageClass: local-path
size: 128Mi
valkey:
# Small, so the whole thing stays inside the runner's budget. It still
# exercises the derivation: 64Mi x limitFactor 3 = 192Mi limit, maxmemory =
# 0.75 x that = 144Mi - and the render aborts if a configured maxmemory ever
# ends up above the limit.
resources:
requests:
cpu: 50m
memory: 64Mi
ephemeral-storage: 32Mi
# Left ON, although nothing in the test connects from another namespace: the
# point here is that the object is accepted and does not keep the pod's own
# traffic (the exec probes are local to the pod and unaffected) from working.
# Whether the policy actually FILTERS is a property of the cluster's CNI and is
# not claimed by this test.
networkPolicy:
enabled: true
-24
View File
@@ -1,24 +0,0 @@
apiVersion: v2
# NOT A CHART THIS REPOSITORY PUBLISHES. It is a YAML parser, and Helm is the
# only one the runners have.
#
# ci/run-selector-match-test.sh has to read label selectors out of rendered
# manifests, and the runners carry neither pyyaml nor yq - measured, and the
# same constraint ci/run-trivy-config.sh and ci/run-quantity-render-test.sh
# work under. The alternative to this chart is a YAML parser written in awk or
# sed, and the failure mode of that is the one this repository removes rather
# than adds: a parser that quietly reads nothing finds no selector, reports no
# mismatch, and the test is green for a chart it never looked at.
#
# So the manifest fragment is handed to Helm as a VALUES FILE, and the one
# template below prints it back as JSON, which python3 reads with its standard
# library. A fragment Helm cannot parse makes helm exit 1, which is loud.
#
# It lives under ci/ and therefore has no publish workflow, no version-bump
# requirement and no entry in change-app-version.yml: the chart-discovery loops
# in this repository look for a Chart.yaml one level below the repo root, and
# "ci" has none. The version below is meaningless for the same reason.
name: yaml2json
description: Internal CI helper - reprints a values file as JSON so python3 can read YAML without pyyaml
type: application
version: 0.0.0
-1
View File
@@ -1 +0,0 @@
{{ .Values | toJson }}
-4
View File
@@ -1,4 +0,0 @@
# Deliberately empty. Every value this chart renders comes from the -f file the
# caller passes, which is one document out of a rendered manifest stream. A
# default here would be merged into that document and would show up as a key
# the chart under test never rendered.
+1 -1
View File
@@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/gchq/CyberChef/master/src/web/static/ima
type: application
version: 9.0.2+up11.4.0
version: 9.0.1+up11.4.0
appVersion: "11.4.0"
maintainers:
+5 -8
View File
@@ -19,11 +19,9 @@ Usage (the values path to the resources dict varies per chart):
resources:
{{- include "cyberchef.resources" .Values.<app>.resources | nindent 12 }}
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart, and a dependency is admissible but expensive (README, guideline 14) -
this chart takes none. This file in template-chart is the reference
implementation.
NOTE: charts in this repo deliberately have no dependencies, so this helper is
duplicated into every chart's templates/_helpers.tpl with the chart's name as
define prefix. This file in template-chart is the reference implementation.
*/}}
{{- define "cyberchef.resources" -}}
{{- $resources := . | default dict -}}
@@ -170,9 +168,8 @@ Rules:
- keys only present in the override are passed through unchanged.
NOTE: like the helpers above, this is duplicated into every chart's
templates/_helpers.tpl with the chart's name as define prefix, because sharing
it would mean a library chart and this chart takes no dependency (README,
guideline 14).
templates/_helpers.tpl with the chart's name as define prefix, because charts
in this repo deliberately have no dependencies.
*/}}
{{- define "cyberchef.defaultedDict" -}}
{{- $defaults := .defaults | default dict -}}
+10 -12
View File
@@ -1,25 +1,23 @@
apiVersion: v2
name: freesailarr
description: A Helm chart to deploy a Wireguard-tunneled media automation stack on Kubernetes - a VPN gateway with indexer, downloader, library managers, subtitle fetcher, request front end and an optional public-broadcaster indexer in one pod, all reaching the internet only through the tunnel
description: A Helm chart to deploy a Wireguard-tunneled media automation stack on Kubernetes - a VPN gateway with indexer, downloader, library managers, subtitle fetcher and request front end in one pod, all reaching the internet only through the tunnel
# No icon on purpose: the stack has no single upstream, and every other chart's
# icon is its one project's logo. Picking any of the ten here would suggest
# icon is its one project's logo. Picking any of the nine here would suggest
# the same false "main app" the versioning deliberately avoids. The field is
# optional, so it is omitted rather than filled with an arbitrary choice.
type: application
version: 3.1.1+up1.0.0
version: 3.0.5+up1.0.0
# The chart's own version marker, freely chosen and tied to no upstream. The
# stack bundles ten independently versioned projects (nine of them on by
# default), so there is no single "app version" to track here.
# stack bundles nine independently versioned projects, so there is no single
# "app version" to track here.
#
# It is NOT the source of any image tag: all ten images are pinned in
# values.yaml, where Renovate's helm-values manager proposes them - except
# mediathekarr, which is pinned by digest and has its own customManagers block
# in renovate.json5, because its tags carry no version. Nothing rewrites this
# field automatically - the chart has no app-version block in renovate.json5
# and is not in the change-app-version.yml choice list. Raise it by hand when
# you want to mark a new edition of the chart as a whole.
# It is NOT the source of any image tag: all nine images are pinned in
# values.yaml, where Renovate's helm-values manager proposes them. Nothing
# rewrites this field automatically - the chart has no app-version block in
# renovate.json5 and is not in the change-app-version.yml choice list. Raise it
# by hand when you want to mark a new edition of the chart as a whole.
appVersion: "1.0.0"
maintainers:
+1 -96
View File
@@ -11,7 +11,7 @@
{{- $sharedNames := list -}}
{{- range $shared }}{{ $sharedNames = append $sharedNames .name }}{{ end -}}
freesailarr — release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
{{ if .Values.mediathekarr.enabled }}Ten{{ else }}Nine{{ end }} containers in ONE pod, sharing gluetun's network namespace.
Nine containers in ONE pod, sharing gluetun's network namespace.
NOTHING IS READY UNTIL THE TUNNEL IS UP
gluetun's readiness probe is its health server, which answers 200 only while
@@ -134,101 +134,6 @@ QBITTORRENT HAS NO SAVE PATH SET (qbittorrent.savePath is empty)
WebUI before starting any download.
{{- end }}
{{- end }}
{{- if .Values.mediathekarr.enabled }}
{{- $dir := .Values.mediathekarr.downloads.directory }}
{{- $gate := .Values.mediathekarr.startGate }}
{{- $mediaStartup := fromYaml (include "freesailarr.probeFor" (dict "root" $ "app" "mediathekarr" "kind" "startupProbe" "given" .Values.mediathekarr.startupProbe)) }}
MEDIATHEKARR: TWO PORTS, AND THE SABNZBD API IS NOT AT /api
Every container shares one network namespace, so both consumers reach it over
127.0.0.1 — no service name, no DNS, no ingress:
Prowlarr -> Newznab indexer, http://127.0.0.1:5008
Sonarr/Radarr -> SABnzbd host 127.0.0.1, port 5007, API path /download/api
A request to /api on 5007 answers with a Newznab validation error and sends
you looking at the wrong half of the app — the upstream documentation only
mentions port 5007 at all.
Sonarr and Radarr must import from
/downloads/{{ $dir }}/complete (one sub-directory per category below it)
which is exactly the path the chart hands MediathekArr and which it reports
back as the SABnzbd complete_dir. Any deviation — a remote path mapping, a
different mount — imports NOTHING, without an error.
The UI (setup wizard and download queue) has no ingress{{ if and .Values.mediathekarr.ingress.enabled .Values.mediathekarr.ingress.host }}, other than the one
you gave it on {{ .Values.mediathekarr.ingress.host }}{{ end }}:
kubectl -n {{ .Release.Namespace }} port-forward svc/{{ $name }}-mediathekarr-service 5007:5007
{{- if $gate.enabled }}
IT DELIBERATELY STARTS LATE, AND THAT IS THE POINT
The container waits (up to {{ $gate.timeoutSeconds }}s) for gluetun to report healthy before
it starts MediathekArr{{ if gt (int $gate.settleSeconds) 0 }}, plus {{ $gate.settleSeconds }}s of settling time{{ end }}. Started before the tunnel,
MediathekArr fails its one ruleset fetch, never retries it, and then answers
EVERY search with 0 results — container up, Ready, nothing in the log. That
is why a green pod proves nothing here and why the gate exits instead of
starting anyway on timeout: a crash-looping container is visible, a silently
empty indexer is not.
kubectl -n {{ .Release.Namespace }} logs {{ $name }}-sfs-0 -c mediathekarr | head -3
The startup probe has to outlast the gate ({{ mul (int $mediaStartup.periodSeconds) (int $mediaStartup.failureThreshold) }}s budget against a {{ $gate.timeoutSeconds }}s gate),
otherwise the kubelet kills the container while it is still waiting.
{{- else }}
THE START GATE IS OFF (mediathekarr.startGate.enabled: false)
MediathekArr may then start before the Wireguard tunnel is up. Its ruleset
fetch fails once, is never retried, and the indexer answers every search
with 0 results afterwards — measured over eleven minutes, with the container
up, Ready and no error anywhere. Nothing reports this state. Check a search
by hand after every restart, or switch the gate back on.
{{- end }}
A FAILED MEDIATHEKARR DOWNLOAD NEVER SAYS WHY
MediathekArr does not check the HTTP status code. On a geo-blocked fetch it
writes the ~476-byte HTML error page out as .mp4 and logs "Download
completed"; only mkvmerge then refuses the file and the status flips to
Failed — with fail_message: null. Measured A/B with one identical NZB:
direct 94,235,525 bytes and Completed, through a foreign exit 0 bytes and
Failed, fail_message null in BOTH cases.
So a geo block is indistinguishable from a broken server, an expired URL or a
full disk, in Sonarr and in the container log alike. And the protection is an
accident: it rests entirely on mkvmerge rejecting an HTML page. Anything a
geo endpoint returns that mkvmerge accepts is reported as Completed and walks
into your library.
WHETHER A SHOW WORKS THROUGH YOUR EXIT: READ THE MARKER, NOT A LIST
The tunnel currently exits {{ with .Values.gluetun.vpn.serverCities }}in {{ . }}{{ else }}{{ with .Values.gluetun.vpn.serverCountries }}in {{ . }}{{ else }}wherever gluetun picks{{ end }}{{ end }}. ARD and ZDF block parts of their
catalogue outside Germany, and every container hangs in gluetun's namespace —
there is no per-container exit.
A table of series goes stale; the marker in the media URL does not. It
predicted the outcome in 584 of 584 measured cases:
/de/, /geo/de/, /deutschlandweit/, geoavailability_D -> blocked outside DE
/weltweit/ -> open
/none/ (ZDF) -> open
/dach/, deChAt (WDR variant) -> open
The rule of thumb "own production = available worldwide" is MEASURABLY WRONG
(an ARD own production was blocked, a licensed series was open). Only the
marker counts.
The escape route is one value — gluetun.vpn.serverCountries/serverCities set
to a German exit — and its price is that it moves the WHOLE stack, all ten
containers, qBittorrent included, into that jurisdiction and onto those
servers. That is a deployment decision, not a chart setting to flip lightly.
OPEN, EXPLICITLY UNMEASURED: HLS ENTRIES
Part of the catalogue is served as an HLS playlist instead of a progressive
MP4. The image carries an ffmpeg-based M3U8 path, but whether such an entry
completes end to end has not been measured. If it does not, those entries fail
regardless of any geo block — and, per the paragraph above, without a reason.
{{- end }}
EVERY SERVICE POINTS AT THE SAME POD IP
One ClusterIP service per app{{ if .Values.recyclarr.enabled }} (recyclarr has none — no port in daemon mode){{ end }}, but all
+17 -78
View File
@@ -19,10 +19,9 @@ Usage:
resources:
{{- include "freesailarr.resources" .Values.<app>.resources | nindent 12 }}
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart, and a dependency is admissible but expensive (README, guideline 14) -
this chart takes none. The reference implementation lives in template-chart.
NOTE: charts in this repo deliberately have no dependencies, so this helper is
duplicated into every chart's templates/_helpers.tpl with the chart's name as
define prefix. The reference implementation lives in template-chart.
*/}}
{{- define "freesailarr.resources" -}}
{{- $resources := . | default dict -}}
@@ -136,7 +135,7 @@ implementation lives in template-chart.
{{/*
Render additive env entries (the repo-wide <app>.env convention: tpl-rendered,
supporting the value, secretKeyRef and configMapKeyRef forms). Shared as a
helper here because this chart has ten containers, each with its own env block.
helper here because this chart has nine containers, each with its own env block.
Usage:
{{- include "freesailarr.env" (dict "root" $ "env" .Values.<app>.env) | nindent 12 }}
@@ -177,14 +176,6 @@ recyclarr is deliberately absent: in daemon mode it serves no port, so it gets
no container port, no service and no ingress. flaresolverr is present - it
listens on 8191 and therefore gets a service - but it has no ingress block in
values and is never exposed from outside the cluster.
"port" is the app's PRIMARY port: the one its service is named after and the one
an ingress points at. mediathekarr is the only app with a second one, because it
runs two processes in one container - the SABnzbd role on 5007 (the UI, hence
the primary) and the Newznab role Prowlarr consumes on 5008. Optional
"extraPorts" entries ({name, port}) are additional ports of the SAME service and
are declared to gluetun's firewall alongside the primary; they never take part
in the ingress, which addresses a service port by name.
*/}}
{{- define "freesailarr.portApps" -}}
{{- $apps := list -}}
@@ -195,8 +186,7 @@ in the ingress, which addresses a service port by name.
(dict "name" "radarr" "port" 7878)
(dict "name" "sonarr" "port" 8989)
(dict "name" "seerr" "port" 5055)
(dict "name" "bazarr" "port" 6767)
(dict "name" "mediathekarr" "port" 5007 "extraPorts" (list (dict "name" "ma-newznab" "port" 5008))) -}}
(dict "name" "bazarr" "port" 6767) -}}
{{- if (index $.Values $app.name).enabled -}}
{{- $apps = append $apps $app -}}
{{- end -}}
@@ -224,9 +214,8 @@ ingress block with enabled: true. Used by the ingress template.
{{/*
The sub-path an app is served under in the shared-domain mode, e.g. "/sonarr".
Null-safe by construction: an absent or erased <app>.ingress.path falls back to
"/<app>", and to "/" for the two apps that can only live at the root - seerr,
which has no base-path support at all, and mediathekarr, whose UI links its
assets with root-absolute paths (see the guard in sharedIngressApps).
"/<app>", and to "/" for seerr, which has no base-path support at all and can
therefore only live at the root.
Input: dict "root" $ "app" <app name>.
*/}}
@@ -237,7 +226,7 @@ Input: dict "root" $ "app" <app name>.
{{- $path = $config.path -}}
{{- end -}}
{{- if or (kindIs "invalid" $path) (eq (toString $path) "") -}}
{{- if has .app (list "seerr" "mediathekarr") -}}
{{- if eq .app "seerr" -}}
{{- $path = "/" -}}
{{- else -}}
{{- $path = printf "/%s" .app -}}
@@ -261,12 +250,9 @@ Two guards live here rather than in the ingress template, so that every
consumer - the ingress objects, the UrlBase env vars and NOTES.txt - sees the
same validated list:
1. seerr or mediathekarr under any path other than "/". seerr has no base-path
support (upstream request open since May 2022) and MediathekArr's UI links
its assets with root-absolute paths, so a chart that rendered either under
a sub-path would produce something that provably cannot work. Both at "/"
on one host is caught by guard 2 below - one of them then needs a host of
its own.
1. seerr under any path other than "/". seerr has no base-path support
(upstream request open since May 2022), so a chart that rendered it under
a sub-path would produce something that provably cannot work.
2. Two apps claiming the same path on the shared host. Only paths on the
shared host can collide; apps with their own host are on a different host
and are therefore not part of this check.
@@ -283,14 +269,11 @@ same validated list:
{{- $config := (index $root.Values $app.name).ingress -}}
{{- if not ($config.host | default "") -}}
{{- $path := include "freesailarr.appPath" (dict "root" $root "app" $app.name) -}}
{{- $rootOnly := dict
"seerr" "seerr has no base-path support (https://github.com/Fallenbagel/jellyseerr/issues/97)"
"mediathekarr" "MediathekArr links its UI assets with root-absolute paths (/download/css/..., measured), so a sub-path stays broken even behind a prefix-stripping rewrite - the browser then asks for those assets at the host root" -}}
{{- if and (hasKey $rootOnly $app.name) (ne $path "/") -}}
{{- fail (printf "%s.ingress.path is %q, but %s and it can therefore only be served at \"/\". Either leave %s.ingress.path at \"/\" or give it its own %s.ingress.host." $app.name $path (index $rootOnly $app.name) $app.name $app.name) -}}
{{- if and (eq $app.name "seerr") (ne $path "/") -}}
{{- fail (printf "seerr.ingress.path is %q, but seerr has no base-path support and can only be served at \"/\" (https://github.com/Fallenbagel/jellyseerr/issues/97). Either leave seerr.ingress.path at \"/\" or give seerr its own seerr.ingress.host." $path) -}}
{{- end -}}
{{- if hasKey $seen $path -}}
{{- fail (printf "%s.ingress.path and %s.ingress.path are both %q on the shared host %q - every app under ingress.host needs its own path. Where neither can move (seerr and mediathekarr can only be served at \"/\"), give one of them its own <app>.ingress.host." (index $seen $path) $app.name $path $host) -}}
{{- fail (printf "%s.ingress.path and %s.ingress.path are both %q on the shared host %q - every app under ingress.host needs its own path" (index $seen $path) $app.name $path $host) -}}
{{- end -}}
{{- $_ := set $seen $path $app.name -}}
{{- $apps = append $apps (dict "name" $app.name "port" $app.port "path" $path) -}}
@@ -373,9 +356,6 @@ alive. It is not what opens the ports today.
{{- $ports := list -}}
{{- range $app := include "freesailarr.portApps" . | fromJsonArray -}}
{{- $ports = append $ports (int $app.port) -}}
{{- range $extra := $app.extraPorts | default list -}}
{{- $ports = append $ports (int $extra.port) -}}
{{- end -}}
{{- end -}}
{{- $ports = append $ports 9999 -}}
{{- join "," $ports -}}
@@ -386,7 +366,7 @@ Whether a media category is mounted at all, i.e. whether at least one app that
mounts it is enabled. "true" when it is, the empty string when it is not (an
empty string is falsy in a template "if", so the result can be used directly).
downloads: qbittorrent, radarr, sonarr, mediathekarr
downloads: qbittorrent, radarr, sonarr
movies: radarr, bazarr
tv: sonarr, bazarr
@@ -395,7 +375,7 @@ Input: dict "root" $ "category" <"downloads"|"movies"|"tv">.
{{- define "freesailarr.mediaCategoryEnabled" -}}
{{- $v := .root.Values -}}
{{- if eq .category "downloads" -}}
{{- if or $v.qbittorrent.enabled $v.radarr.enabled $v.sonarr.enabled $v.mediathekarr.enabled -}}true{{- end -}}
{{- if or $v.qbittorrent.enabled $v.radarr.enabled $v.sonarr.enabled -}}true{{- end -}}
{{- else if eq .category "movies" -}}
{{- if or $v.radarr.enabled $v.bazarr.enabled -}}true{{- end -}}
{{- else if eq .category "tv" -}}
@@ -905,47 +885,6 @@ bazarr:
- FOWNER
- SETGID
- SETUID
mediathekarr:
livenessProbe:
exec:
command:
- /bin/bash
- -c
- >-
wget -q -T 5 -O /dev/null "http://127.0.0.1:5008/api?t=caps" &&
wget -q -T 5 -O /dev/null "http://127.0.0.1:5007/download/api?mode=version&output=json&apikey=x"
periodSeconds: 30
timeoutSeconds: 15
failureThreshold: 3
readinessProbe:
httpGet:
path: /download/
port: 5007
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
startupProbe:
httpGet:
path: /download/
port: 5007
periodSeconds: 5
timeoutSeconds: 5
failureThreshold: 60
resources:
limitFactor: 3
requests:
cpu: 0.1
memory: 256Mi
ephemeralStorage: 100Mi
securityContext:
runAsUser: {{ .Values.puid }}
runAsGroup: {{ .Values.pgid }}
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
configInit:
resources:
limitFactor: 3
@@ -966,7 +905,7 @@ configInit:
{{/*
The effective pod security context. Separate from the per-container table
because it is the one block shared by all containers of the pod.
because it is the one block shared by all nine containers.
*/}}
{{- define "freesailarr.podSecurityContext" -}}
{{- $given := (fromYaml (include "freesailarr.subBlock" (dict "block" .Values.securityContext "key" "pod" "path" "securityContext"))).value -}}
@@ -21,16 +21,5 @@ spec:
port: {{ int $app.port }}
targetPort: {{ $app.name }}
name: {{ $app.name }}
{{- /* An app with more than one port keeps them in ONE service rather than
getting a second object: they belong to the same container and the
same pod IP, and only the primary port is ever addressed by an
ingress. mediathekarr is the only such app today - 5007 (SABnzbd
role, primary) and 5008 (Newznab role). */}}
{{- range $extra := $app.extraPorts | default list }}
- protocol: TCP
port: {{ int $extra.port }}
targetPort: {{ $extra.name }}
name: {{ $extra.name }}
{{- end }}
type: ClusterIP
{{- end }}
+3 -110
View File
@@ -521,109 +521,6 @@ spec:
subPath: {{ . | quote }}
{{- end }}
{{- end }}
{{- if .Values.mediathekarr.enabled }}
# MediathekArr, the tenth container and the only one off by default: a fake
# Usenet indexer for Prowlarr (Newznab, port 5008) and a fake SABnzbd
# for Sonarr/Radarr (port 5007), fetching from the ARD and ZDF media
# libraries. Both consumers sit in this pod and reach it over
# 127.0.0.1, so it needs neither a service nor DNS to do its job.
- name: mediathekarr
# Tag AND digest: the tag stays readable, the digest is what is
# actually pulled. "beta" moves, and "latest" is a 2024 build - see
# the values comment.
image: "{{ .Values.mediathekarr.image.repository }}:{{ .Values.mediathekarr.image.tag }}@{{ required "mediathekarr.image.digest is required - the tag alone is a moving reference" .Values.mediathekarr.image.digest }}"
imagePullPolicy: IfNotPresent
{{- if .Values.mediathekarr.startGate.enabled }}
# THE START GATE. Kubernetes cannot order two regular containers, and
# an initContainer waiting for gluetun would deadlock (gluetun only
# starts once the init containers are done). So the wait happens here,
# in front of the image entrypoint: poll gluetun's health server -
# 200 only while the tunnel actually works - and exec the entrypoint
# afterwards.
#
# Why this is not optional decoration: MediathekArr fetches its
# ruleset once at start and never retries. Started before the tunnel,
# it answered every Newznab search with 0 results for over eleven
# minutes - container up, Ready, nothing in the log. The failure case
# looks exactly like the healthy one, which is why the gate exits
# instead of starting anyway when the timeout passes: a crash-looping
# container is visible, a silently empty indexer is not.
#
# wget comes with the (Debian) image, bash is its entrypoint's own
# interpreter - no extra tooling and no shell built into the chart.
command:
- /bin/bash
- -c
- |
set -eu
deadline=$(( SECONDS + {{ int .Values.mediathekarr.startGate.timeoutSeconds }} ))
echo "mediathekarr: waiting for gluetun to report healthy (127.0.0.1:9999)"
until wget -q -T 5 -O /dev/null http://127.0.0.1:9999/; do
if [ "$SECONDS" -ge "$deadline" ]; then
echo "mediathekarr: gluetun did not report healthy within {{ int .Values.mediathekarr.startGate.timeoutSeconds }}s - exiting instead of starting without a tunnel, because a MediathekArr started too early answers every search with 0 results and reports nothing" >&2
exit 1
fi
sleep {{ int .Values.mediathekarr.startGate.intervalSeconds }}
done
echo "mediathekarr: gluetun healthy after ${SECONDS}s"
{{- if gt (int .Values.mediathekarr.startGate.settleSeconds) 0 }}
sleep {{ int .Values.mediathekarr.startGate.settleSeconds }}
{{- end }}
exec /app/docker_start.sh
{{- end }}
env:
- name: TZ
value: {{ .Values.timezone | quote }}
{{- /* Both download paths are set by the chart, not left to the
image: complete/ has to be visible to Sonarr/Radarr under
the SAME absolute path (they are handed this value as the
SABnzbd complete_dir, and a deviation imports nothing
without an error), and incomplete/ has to live on the media
claim rather than in an emptyDir, which would count against
the container's ephemeral-storage limit and let one large
download evict the whole pod. Setting them also keeps the
image's built-in /data/mediathek default out of play, which
the read-only root filesystem does not provide. */}}
- name: DOWNLOAD_INCOMPLETE_PATH
value: {{ printf "/downloads/%s/incomplete" .Values.mediathekarr.downloads.directory | quote }}
- name: DOWNLOAD_COMPLETE_PATH
value: {{ printf "/downloads/%s/complete" .Values.mediathekarr.downloads.directory | quote }}
{{- include "freesailarr.env" (dict "root" $ "env" .Values.mediathekarr.env) | nindent 12 }}
ports:
- containerPort: 5007
protocol: TCP
name: mediathekarr
- containerPort: 5008
protocol: TCP
name: ma-newznab
livenessProbe:
{{- include "freesailarr.probeFor" (dict "root" $ "app" "mediathekarr" "kind" "livenessProbe" "given" .Values.mediathekarr.livenessProbe) | nindent 12 }}
readinessProbe:
{{- include "freesailarr.probeFor" (dict "root" $ "app" "mediathekarr" "kind" "readinessProbe" "given" .Values.mediathekarr.readinessProbe) | nindent 12 }}
startupProbe:
{{- include "freesailarr.probeFor" (dict "root" $ "app" "mediathekarr" "kind" "startupProbe" "given" .Values.mediathekarr.startupProbe) | nindent 12 }}
resources:
{{- include "freesailarr.resourcesFor" (dict "root" $ "app" "mediathekarr" "given" .Values.mediathekarr.resources) | nindent 12 }}
securityContext:
{{- include "freesailarr.securityContextFor" (dict "root" $ "app" "mediathekarr" "given" .Values.mediathekarr.securityContext) | nindent 12 }}
volumeMounts:
- mountPath: /app/config
name: mediathekarr-config
{{- /* The same claim, the same mount path as sonarr/radarr - that
is what makes the complete_dir path identical on both
sides. */}}
- mountPath: /downloads
name: {{ include "freesailarr.mediaVolumeName" (dict "root" $ "category" "downloads") }}
{{- with .Values.media.downloads.subPath }}
subPath: {{ . | quote }}
{{- end }}
# Not a measured requirement: the container ran through with a
# read-only root and without a writable /tmp. Kept as cheap
# insurance for the .NET runtime, which is the kind of consumer that
# reaches for /tmp on a code path nobody exercised yet.
- mountPath: /tmp
name: mediathekarr-tmp
{{- end }}
restartPolicy: Always
volumes:
# The tun device gluetun needs to create the Wireguard tunnel.
@@ -645,11 +542,11 @@ spec:
setup with three separate claims still renders byte-identically to
2.0.1 and the diff itself proves the change is inert there.
*/}}
{{- if or .Values.qbittorrent.enabled .Values.radarr.enabled .Values.sonarr.enabled .Values.mediathekarr.enabled }}
{{- if or .Values.qbittorrent.enabled .Values.radarr.enabled .Values.sonarr.enabled }}
{{- if include "freesailarr.mediaVolumeOwned" (dict "root" $ "category" "downloads") }}
- name: media-downloads
persistentVolumeClaim:
claimName: {{ required "media.downloads.existingClaim is required when qbittorrent, radarr, sonarr or mediathekarr is enabled" .Values.media.downloads.existingClaim }}
claimName: {{ required "media.downloads.existingClaim is required when qbittorrent, radarr or sonarr is enabled" .Values.media.downloads.existingClaim }}
{{- end }}
{{- end }}
{{- if or .Values.radarr.enabled .Values.bazarr.enabled }}
@@ -682,16 +579,12 @@ spec:
- name: seerr-tmp
emptyDir: {}
{{- end }}
{{- if .Values.mediathekarr.enabled }}
- name: mediathekarr-tmp
emptyDir: {}
{{- end }}
# Config volumes for gluetun and the enabled apps. NOTE: volumeClaimTemplates
# are immutable - toggling an app's enabled flag later requires recreating
# the StatefulSet once (kubectl delete statefulset <name> --cascade=orphan
# before upgrading; existing PVCs and their data survive).
volumeClaimTemplates:
{{- range $app := list "gluetun" "qbittorrent" "prowlarr" "radarr" "recyclarr" "sonarr" "seerr" "bazarr" "mediathekarr" }}
{{- range $app := list "gluetun" "qbittorrent" "prowlarr" "radarr" "recyclarr" "sonarr" "seerr" "bazarr" }}
{{- $config := index $.Values $app }}
{{- if or (eq $app "gluetun") $config.enabled }}
- metadata:
-91
View File
@@ -535,76 +535,6 @@
"$ref": "#/definitions/containerSecurityContext"
}
}
},
"mediathekarr": {
"type": "object",
"additionalProperties": false,
"properties": {
"enabled": {
"type": "boolean"
},
"image": {
"$ref": "#/definitions/pinnedImage"
},
"ingress": {
"$ref": "#/definitions/appIngress"
},
"storage": {
"$ref": "#/definitions/appStorage"
},
"startGate": {
"description": "Delays the container until gluetun reports healthy. NOT null-safe on purpose and not merged through the fallback helper: this block decides whether the gate is rendered at all, and an erased block would silently produce a container that starts before the tunnel - the failure this chart writes three paragraphs about, and one that looks green.",
"type": "object",
"additionalProperties": false,
"properties": {
"enabled": {
"type": "boolean"
},
"intervalSeconds": {
"type": "integer",
"minimum": 1
},
"timeoutSeconds": {
"description": "Total wait before the container exits instead of starting without a tunnel. Keep the startup probe's budget (periodSeconds x failureThreshold) above it, or the kubelet kills the container while the gate is still waiting.",
"type": "integer",
"minimum": 1
},
"settleSeconds": {
"type": "integer",
"minimum": 0
}
}
},
"downloads": {
"type": "object",
"additionalProperties": false,
"properties": {
"directory": {
"description": "Sub-directory on the shared downloads claim holding incomplete/ and complete/. The chart derives DOWNLOAD_INCOMPLETE_PATH and DOWNLOAD_COMPLETE_PATH from it; complete/ is what Sonarr/Radarr are told to import from, under the identical absolute path.",
"type": "string",
"minLength": 1
}
}
},
"env": {
"$ref": "#/definitions/env"
},
"livenessProbe": {
"$ref": "#/definitions/probe"
},
"readinessProbe": {
"$ref": "#/definitions/probe"
},
"startupProbe": {
"$ref": "#/definitions/probe"
},
"resources": {
"$ref": "#/definitions/resources"
},
"securityContext": {
"$ref": "#/definitions/containerSecurityContext"
}
}
}
},
"definitions": {
@@ -816,27 +746,6 @@
"object",
"null"
]
},
"pinnedImage": {
"description": "Like image, plus the digest mediathekarr is pinned by: its tags are unusable (\"latest\" is a 2024 build, \"beta\" moves), so the tag stays as the readable label and the digest is what is actually pulled. Deliberately a separate definition instead of adding an optional digest to image: on every other image in this chart a digest key would be a value the templates never read - exactly the silent no-op the schema exists to reject (#68).",
"type": "object",
"additionalProperties": false,
"properties": {
"repository": {
"type": "string"
},
"tag": {
"type": [
"string",
"number"
]
},
"digest": {
"description": "Image digest, sha256:<64 hex>. Required - the template refuses to render without it.",
"type": "string",
"pattern": "^sha256:[0-9a-f]{64}$"
}
}
}
}
}
+4 -261
View File
@@ -4,11 +4,10 @@
# the Wireguard tunnel.
#
# Image pinning (documented deviation from the Chart.AppVersion standard):
# NO image tag comes from appVersion. All ten images, gluetun included, are
# NO image tag comes from appVersion. All nine images, gluetun included, are
# pinned here via <app>.image.repository / <app>.image.tag, because the stack
# combines ten independently versioned upstream projects and none of them is
# "the" one app version. mediathekarr additionally carries an image.digest -
# see the reasoning in its own block at the end of this file.
# combines nine independently versioned upstream projects and none of them is
# "the" one app version.
#
# That is also what puts every image under Renovate: the built-in helm-values
# manager reads image.repository/image.tag pairs out of this file and proposes
@@ -85,7 +84,7 @@ storage:
# Media libraries as pre-existing PVCs (shared with the separately deployed
# Jellyfin) - the chart mounts but never creates them. Each claim is required
# as soon as an app that mounts it is enabled:
# downloads: qbittorrent, radarr, sonarr, mediathekarr
# downloads: qbittorrent, radarr, sonarr
# movies: radarr, bazarr
# tv: sonarr, bazarr
#
@@ -875,259 +874,3 @@ bazarr:
- FOWNER
- SETGID
- SETUID
# MediathekArr: the tenth container, and the only one that is OFF by default. It fakes a Usenet indexer towards Prowlarr and a SABnzbd downloader
# towards Sonarr/Radarr, and fetches the content over plain HTTP from the ARD
# and ZDF media libraries. Both consumers already sit in this pod and share
# gluetun's network namespace, so they reach it over 127.0.0.1 - no service, no
# DNS, no ingress detour.
#
# Off by default because it is additive and nobody's existing release should
# grow a container on an upgrade: enabling it creates a new
# volumeClaimTemplate, which a StatefulSet cannot take without being recreated
# (see the NOTES and the volumeClaimTemplates comment in the StatefulSet).
#
# ----------------------------------------------------------------------------
# THE GEO CAVEAT, AND HOW TO CHECK IT YOURSELF
# ----------------------------------------------------------------------------
# The tunnel leaves through Zurich by this chart's default, and ARD and ZDF
# block parts of their catalogue outside Germany. Every container hangs in
# gluetun's namespace; there is no bypass for a single container.
#
# Measured, 480 URLs across 12 channels: 104 of 473 loadable URLs were blocked
# through Zurich (22.0%), none of them on the German side. The spread is very
# uneven - ONE 97.5%, ZDFneo 47.5%, ZDFinfo 37.5%, Das Erste 20.5%, ZDF 10.3%,
# BR 0% - so a single number tells you little about the show you actually want.
#
# THE PROCEDURE, NOT A TABLE OF SERIES. A list of titles is stale in half a
# year; the marker in the CDN path is not. Over two measurement rounds it
# predicted the outcome perfectly - 584 of 584 cases in the full survey, not a
# single misclassification. Look at the media URL of an entry and read the
# path:
#
# /de/, /geo/de/, /deutschlandweit/, geoavailability_D -> blocked outside DE
# /weltweit/ -> open
# /none/ (ZDF) -> open
# /dach/, deChAt (WDR variant) -> open, Switzerland
# is part of DACH
#
# The obvious rule of thumb "own production = available worldwide" is MEASURABLY
# WRONG and is named here so nobody re-invents it: an ARD own production was
# blocked, a licensed series was open. Only the marker counts.
#
# THE ESCAPE ROUTE AND ITS PRICE. The exit is a deployment decision, not a chart
# property: set gluetun.vpn.serverCountries/serverCities to a German exit and
# the blocks disappear. The price is that this moves the WHOLE stack, all ten
# containers - qBittorrent included - into that jurisdiction and onto those
# servers. There is no per-container exit, and this chart deliberately does not
# pretend otherwise.
#
# ----------------------------------------------------------------------------
# THE fail_message: null TRAP - READ THIS BEFORE DEBUGGING A FAILED DOWNLOAD
# ----------------------------------------------------------------------------
# MediathekArr never checks the HTTP status code. Measured A/B with an identical
# NZB: direct (DE) -> status Completed, 94,235,525 bytes, fail_message null;
# through Zurich -> status Failed, 0 bytes, fail_message ALSO null.
#
# On a geo-blocked fetch it writes the ~476-byte HTML error page out as .mp4 and
# logs "Download completed". Only mkvmerge then refuses the file (exit code 2)
# and flips the status to Failed. Two consequences:
#
# 1. Sonarr is told "failed" without a reason. A geo block looks exactly like
# a broken server, an expired URL or a full disk - and the container log
# does not mention the 403 either.
# 2. That protection is an accident, not error handling. It rests entirely on
# mkvmerge rejecting an HTML page. Any geo response mkvmerge would accept
# is reported as Completed, and then a broken file walks into the library.
# It did not happen in either measurement round (Akamai consistently
# returns HTML), but that is not something to rely on.
#
# ----------------------------------------------------------------------------
# OPEN QUESTION, EXPLICITLY UNMEASURED: HLS
# ----------------------------------------------------------------------------
# Part of the catalogue is served as an HLS playlist (application/x-mpegURL)
# instead of a progressive MP4. The image carries an ffmpeg-based M3U8 path
# (log lines "Starting M3U8 download" / "M3U8 download completed", and ffmpeg is
# present in the image), so the code is there - but whether such an entry
# actually completes end to end has NOT been measured. If it does not, those
# entries fail regardless of any geo block. Stated as an open question on
# purpose: no assumption in either direction.
mediathekarr:
enabled: false
# Pinned BY DIGEST, unlike every other image in this chart, because the tags
# are unusable: "latest" is from 2024-11-12 (= v0.2.5) and the upstream README
# points at "beta", which is a moving tag. tag + digest together keep the
# human-readable line and the immutable reference.
#
# The image is SINGLE-ARCH, linux/amd64 only - there is no arm64 variant. The
# target cluster is amd64 throughout, so this is a note, not a problem; on a
# mixed cluster the pod would need a nodeSelector this chart does not offer.
#
# Renovate: the built-in helm-values manager cannot follow this pair (the tag
# carries no version), so renovate.json5 has a customManagers block that
# matches exactly these three lines and proposes digest updates for the beta
# tag. Renaming or reordering the keys silently switches that off - the rule
# would still be valid, it just would not match any more (#158).
image:
repository: docker.io/pcjones/mediathekarr
tag: beta
digest: sha256:3ed7e1487810e1519301bd56a0c420e260e428295b029a47806c0397e4a52a33
# TWO ports, not one, and the upstream documentation only mentions the first:
# 5007 MediathekArrDownloader - the SABnzbd role and the web UI
# 5008 MediathekArrServer - the Newznab role Prowlarr talks to
# The SABnzbd API lives under /download/api, NOT under /api: a request to
# /api on 5007 answers with a Newznab validation error and sends you looking
# in the wrong place. Both ports are declared to gluetun's firewall
# automatically (see the firewallInputPorts helper).
#
# No ingress by default: MediathekArr's UI links its assets with root-absolute
# paths (/download/css/..., measured), so it can only be served at the ROOT of
# a host - a sub-path is broken even behind a prefix-stripping rewrite,
# because the browser then requests the assets at the host root. On the shared
# domain the root belongs to seerr, so enabling this ingress there needs
# either seerr switched off or a host of its own here. Both consumers reach
# MediathekArr over localhost and need no ingress at all; the UI is only
# needed to run the setup wizard, for which a port-forward is enough.
ingress:
enabled: false
host: null
# Read only in the shared-domain mode. "/" is the only value that works;
# anything else fails the render rather than producing a broken UI.
path: /
# clusterIssuer: null
# className: null
# annotations: {}
storage:
# Size of the MediathekArr config volume (/app/config, a small JSON).
size: 1Gi
# THE START GATE - the sharpest point of this container, and the one whose
# failure mode looks green.
#
# If MediathekArr starts before the tunnel is up, its ruleset fetch fails once
# and is never retried: the instance then answered every Newznab search with 0
# results for over eleven minutes, while the container was up, Ready and
# logged nothing wrong. A green pod proves nothing here. With the start
# delayed until gluetun reports healthy, the ruleset loads and the search
# matches an instance running with direct egress (the search itself is not
# geo-blocked - 56 hits through either exit).
#
# Kubernetes cannot express "start after that container": gluetun is a regular
# container, and an initContainer waiting for it would deadlock. So the gate
# lives in this container's command - it polls gluetun's health server on
# 127.0.0.1:9999 (the same endpoint gluetun's own probes use, 200 only while
# the tunnel works) and only then execs the image entrypoint.
#
# On timeout the container EXITS instead of starting anyway. That is
# deliberate: a crash-looping container is visible, a silently empty indexer
# is not.
#
# Keep startupProbe budget > timeoutSeconds + app start, or the kubelet kills
# the container while the gate is still waiting. Defaults: 180s gate,
# 5s * 60 = 300s startup budget.
startGate:
enabled: true
# Poll interval and total budget for gluetun becoming healthy. gluetun's own
# startup budget is 150s, so 180s covers a normal handshake with room.
intervalSeconds: 3
timeoutSeconds: 180
# Extra pause after gluetun reports healthy, before the app starts. NOT a
# measured requirement - gluetun's 200 is the actual condition - but a cheap
# buffer for DNS and routing to settle. 0 disables it.
settleSeconds: 10
# WHERE THE DOWNLOADS GO. Both directories live on the SHARED downloads claim
# (media.downloads), mounted at /downloads exactly like sonarr, radarr and
# qbittorrent mount it, under this sub-directory:
#
# /downloads/<directory>/incomplete work in progress
# /downloads/<directory>/complete finished, plus one sub-dir per category
#
# Two reasons, both of them the kind that fails silently:
#
# 1. complete/ MUST be visible to Sonarr/Radarr under the SAME absolute
# path. The chart hands that path to MediathekArr
# (DOWNLOAD_COMPLETE_PATH), which reports it to Sonarr as the SABnzbd
# complete_dir. Every container here mounts the claim at /downloads, so
# the paths are identical by construction - a deviation imports nothing,
# without an error.
# 2. incomplete/ must NOT be an emptyDir. An emptyDir counts against the
# container's ephemeral-storage limit, and one large download makes the
# kubelet evict the WHOLE pod - all ten containers. Same trap as the one
# documented in ytdlp-web-player.
#
# Do not set DOWNLOAD_COMPLETE_PATH or DOWNLOAD_INCOMPLETE_PATH in the env
# block below: the chart always sets both, and a second entry of the same name
# is not a defined state (see README, "Environment variables").
downloads:
directory: mediathekarr
# Extra environment variables, appended to the ones the chart always sets.
# Useful ones the image reads: CATEGORIES, MAX_PARALLEL_DOWNLOADS,
# MEDIATHEKARR_API_BASE_URL.
env: []
# Probes on measured 200 endpoints - MediathekArr has NO health endpoint at
# all (/health, /healthz, /api/health, /download/health, /status and /ping all
# answer 404).
#
# Liveness is an exec on purpose: one container runs TWO processes, and the
# kubelet allows one probe of each kind. The Newznab half can wedge while the
# downloader half still answers, and that is exactly the failure that makes
# searches come back empty. The command checks both ports and restarts the
# container when either stops answering. (The entrypoint already exits when
# one of the two processes CRASHES - the exec probe covers the case where it
# is alive but not serving.)
#
# Readiness and startup use the downloader port: that is the one the Service
# and any ingress point at, and /download/ answers 200 without a query string.
livenessProbe:
exec:
command:
- /bin/bash
- -c
- >-
wget -q -T 5 -O /dev/null "http://127.0.0.1:5008/api?t=caps" &&
wget -q -T 5 -O /dev/null "http://127.0.0.1:5007/download/api?mode=version&output=json&apikey=x"
periodSeconds: 30
timeoutSeconds: 15
failureThreshold: 3
readinessProbe:
httpGet:
path: /download/
port: 5007
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# The startup probe gates liveness/readiness AND has to outlast the start gate
# (see startGate above): 5s * 60 = 300s against a 180s gate.
startupProbe:
httpGet:
path: /download/
port: 5007
periodSeconds: 5
timeoutSeconds: 5
failureThreshold: 60
resources:
limitFactor: 3
# ephemeral-storage stays small on purpose: downloads land on the media
# claim, not on the container filesystem (see downloads above).
requests:
cpu: 0.1
memory: 256Mi
ephemeralStorage: 100Mi
# Measured, including a real download and the mkvmerge remux: the container
# runs fully hardened - non-root, uid/gid from puid/pgid,
# readOnlyRootFilesystem: true - as long as its writable paths are volumes.
# Those are /app/config (the config claim) and the two download directories on
# the media claim; the image's built-in default /data/mediathek is never
# touched, because the chart overrides both paths by environment (measured:
# read-only root, no /data/mediathek mount, no error).
# PUID/PGID are NOT needed (the image's gosu path is only taken when they are
# set), and the
# commented-out non-root block in its Dockerfile is outdated for the beta.
# runAsUser/runAsGroup come from the global puid/pgid values in the template -
# do not set them here.
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
capabilities:
drop:
- ALL
-23
View File
@@ -1,23 +0,0 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
-26
View File
@@ -1,26 +0,0 @@
apiVersion: v2
name: garage
description: A Helm chart to deploy Garage, an S3-compatible object store, as a single node on Kubernetes
icon: https://garagehq.deuxfleurs.fr/images/garage-logo.svg
type: application
# appVersion is the exact image tag, and the "v" is part of it - so it is part
# of appVersion and of the "+up" suffix. Precedent: stalwart (+upv0.16.24),
# calibre-web-automated, matrix-synapse, ytdlp-web-player.
#
# The image is docker.io/dxflrs/garage, the MULTI-ARCH one. Upstream's own chart
# pins dxflrs/amd64_garage instead; that is a per-architecture image and pinning
# it would tie the chart to amd64 for no gain.
#
# THE TAG TRAP THIS VERSION HAS TO SURVIVE: next to the release tags the
# registry continuously receives COMMIT-SHA TAGS (40 hex characters, the newest
# of them dated 2026-10-03). A Renovate rule without a version constraint would
# offer those as "updates", which is why the garage block in renovate.json5
# carries a versioningTemplate anchored to ^v<major>.<minor>.<patch>$ - same
# shape as the stalwart block, for the same reason.
version: 1.0.0+upv2.4.1
appVersion: "v2.4.1"
maintainers:
- name: jklein
-144
View File
@@ -1,144 +0,0 @@
{{- $settings := fromYaml (include "garage.settings" .) -}}
{{- $pod := printf "%s-%s-sfs-0" .Release.Name .Chart.Name -}}
{{- $svc := printf "%s-%s-service" .Release.Name .Chart.Name -}}
Garage {{ .Chart.AppVersion }} installed as release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
ONE node, replication_factor = {{ $settings.replicationFactor | int64 }}, metadata engine {{ $settings.dbEngine }}.
S3 ENDPOINT (in-cluster, path-style addressing):
http://{{ $svc }}.{{ .Release.Namespace }}.svc.cluster.local:{{ $settings.ports.s3 | int64 }}
region: {{ $settings.s3.region }}
{{- if .Values.ingress.enabled }}
published at: https://{{ .Values.ingress.domain }}
{{- end }}
{{- if $settings.web.enabled }}
static website endpoint on port {{ $settings.ports.web | int64 }}{{ with $settings.web.rootDomain }} (root domain {{ . }}){{ else }} - WITHOUT a rootDomain it serves nothing; set garage.web.rootDomain{{ end }}
{{- end }}
THE 1PASSWORD ITEMS, AND WHICH FIELDS THEY MUST CARRY
The operator materializes one Secret key per item FIELD, so a missing or
misnamed field leaves the pod in CreateContainerConfigError.
{{ .Values.secrets.rpcSecretRef }}
rpc-secret 32 bytes of random hex: openssl rand -hex 32
{{ if .Values.secrets.s3SecretRef }} {{ .Values.secrets.s3SecretRef }}
access-key >= 8 chars, [A-Za-z0-9-_.]: openssl rand -hex 12
secret-key >= 16 printable ASCII chars: openssl rand -hex 32
bucket a valid S3 bucket name (3-63 chars, lowercase)
{{ else }} (no S3 key item configured - secrets.s3SecretRef is unset)
{{ end }}
{{- if and $settings.singleNode (or $settings.defaultAccessKey $settings.defaultBucket) }}
THE DIRECTION IS INWARDS: Garage IMPORTS{{ if $settings.defaultAccessKey }} the access key{{ end }}{{ if $settings.defaultBucket }}{{ if $settings.defaultAccessKey }} and{{ end }} the bucket{{ end }} from
that item on start. The vault is the source of truth and nothing has to be
copied out of the server.
{{- end }}
FIRST START
{{ if $settings.singleNode }}
Nothing to do by hand. The server created its own cluster layout before it
opened any listener{{ if $settings.defaultAccessKey }}, imported the access key{{ end }}{{ if $settings.defaultBucket }} and created the bucket{{ end }}.
Verify it, in this order - all commands need /garage as the command, because
the image has no shell:
1. kubectl -n {{ .Release.Namespace }} rollout status statefulset/{{ .Release.Name }}-{{ .Chart.Name }}-sfs
Ready means the admin API answered 200 on /health, which means a layout
exists and the node has quorum. Not-Ready with a running container means
the layout is missing - check the logs for "Created initial layout".
2. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage status
One node, with a role (zone dc1, a capacity taken from the detected disk
size). A node WITHOUT a role means no layout was applied.
3. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage layout show
{{- if $settings.defaultBucket }}
4. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage bucket list
kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage bucket info <bucket>
The bucket exists and the imported key has read/write/owner on it.
{{- end }}
5. One write through the S3 API from a client pod, which is the only step
that proves the endpoint rather than the process.
{{ else }}
garage.singleNode is false, so THIS RELEASE SERVES NOTHING YET and the pod
will stay not-Ready until a layout exists - that is correct, not a defect: the
readiness probe reports /health, and /health answers 503 while there is no
layout. Apply one by hand, in this order:
1. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage status
Note the node id and the current layout version (0 on a fresh install).
2. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- \
/garage layout assign -z dc1 -c <capacity> <node-id>
The capacity is this node's usable size, roughly storage.data.amount
({{ .Values.storage.data.amount }}). It takes BYTE suffixes B, KB, MB, GB, TB, PB - not
Kubernetes' Gi/Mi, so write e.g. 10GB, not 10Gi.
3. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage layout show
Read it before applying. The next step is NOT reversible by repeating it.
4. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage layout apply --version <N+1>
N is the version from step 1, so a fresh cluster takes --version 1.
NEVER call this twice with the same --version: the flag is a guard
against applying a staged change you have not seen, and reusing a number
is how a layout is applied twice.
5. kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- /garage bucket create <bucket>
kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- \
/garage key import --yes <access-key> <secret-key>
kubectl -n {{ .Release.Namespace }} exec {{ $pod }} -- \
/garage bucket allow --read --write --owner --key <access-key> <bucket>
"key import" rather than "key create", so the credential still comes FROM
the vault item. "key create" generates one and then it has to travel the
other way, through a terminal, with the vault as a copy.
6. The pod becomes Ready within one probe period after step 4.
{{ end }}
WHAT THIS CHART DOES NOT RENDER, AND WHAT IS MISSING WITHOUT IT
- THE LONGHORN BACKUP LABELS. recurring-job.longhorn.io/daily-snapshot and
.../daily-backup belong to the consuming bundle, not to this chart. They do
NOT apply automatically: without them the off-site half of the backup plan
does not exist, it only looks like it does.
- The namespace's volume-encryption secret, if the StorageClass is
longhorn-encrypted-ns: it resolves
${pvc.namespace}-pvc-encryption-secret, and without that entry the PVCs
stay Pending.
- No kubernetes_discovery CRD and no bootstrap peers: a single node has no
members to find.
TRAPS WORTH KNOWING BEFORE YOU NEED THEM
- THE METADATA IS NOT IN YOUR VOLUME BACKUP in any usable form. A
file-system snapshot of a live metadata database is not a restore point -
taken during a write it is corrupt. What IS a restore point are Garage's
own snapshots, on the second PVC, taken every {{ $settings.metadataAutoSnapshotInterval }}.
- THE WINDOW IS SHORT: exactly TWO snapshots are kept, the first after half
the interval and every later one after the interval times a random
1.0-1.2. With metadata_auto_snapshot_interval = {{ $settings.metadataAutoSnapshotInterval }} the oldest
recovery point is therefore between one and two and a half intervals old,
never more. A corruption found later than that is older than every
snapshot that still exists.
- metadata_fsync is ON here and OFF in Garage's defaults. If you ever see it
missing from garage.toml, that is a regression, not a cleanup.
- replication_factor 1 applies to the METADATA as well. Upstream calls a
corrupted metadata database "generally not recoverable", and the volume
layer replicates a corrupted file as faithfully as a good one.
- ROTATING THE S3 SECRET IS NOT A RE-SAVE. Garage refuses to start when the
access key already exists with a different secret ("is associated with a
secret key different than the one given in GARAGE_DEFAULT_SECRET_KEY"). To
rotate: either put a NEW access-key id into the item (the old key stays and
can be deleted with "/garage key delete"), or delete the old key first and
then update both fields.
- A CHANGE INSIDE A VAULT ITEM RESTARTS NOTHING. Helm never sees it. After
changing a field:
kubectl -n {{ .Release.Namespace }} rollout restart statefulset/{{ .Release.Name }}-{{ .Chart.Name }}-sfs
(Repointing a secrets.*SecretRef at a different item DOES roll the pod -
that is covered by checksum/secret-ref.)
- THE IMAGE HAS NO SHELL. "kubectl exec -it ... -- sh" does not work and
never will; every command above uses /garage directly. Exec probes are
impossible in this chart for the same reason.
- THE RPC SECRET IS IN THE PROCESS ENVIRONMENT, not in a file. That is
forced, not chosen: Garage rejects a secret file with any group bit set,
and the kubelet sets one. values.yaml has the mechanism.
-392
View File
@@ -1,392 +0,0 @@
{{/*
Render the body of a container "resources:" block from a values dict of the form
resources:
limitFactor: 3 # optional, default 3
requests: { cpu, memory, ephemeral-storage / ephemeralStorage, ... }
limits: { ... } # optional, every entry optional
Rules (variant B, see issue #19):
- requests are rendered as given (camelCase "ephemeralStorage" is normalized
to the Kubernetes key "ephemeral-storage").
- an explicitly set limit always wins.
- a missing memory / ephemeral-storage limit is computed as
limitFactor x the corresponding request.
- a missing cpu limit is NOT set at all (the kubelet then treats CPU as
unlimited; CPU is compressible, so this cannot kill the workload).
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart, and a dependency is admissible but expensive (README, guideline 14) -
this chart takes none. The reference implementation is
template-chart/templates/_helpers.tpl.
*/}}
{{- define "garage.resources" -}}
{{- $resources := . | default dict -}}
{{- $factor := 3.0 -}}
{{- if not (kindIs "invalid" $resources.limitFactor) -}}
{{- $factor = float64 $resources.limitFactor -}}
{{- end -}}
{{- $requests := dict -}}
{{- range $key, $value := ($resources.requests | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $requests (include "garage.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- $limits := dict -}}
{{- range $key, $value := ($resources.limits | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $limits (include "garage.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- range $key, $request := $requests -}}
{{- if and (ne $key "cpu") (not (hasKey $limits $key)) -}}
{{- $_ := set $limits $key (include "garage.multiplyQuantity" (dict "quantity" $request "factor" $factor)) -}}
{{- end -}}
{{- end -}}
{{- $out := dict -}}
{{- with $requests }}{{- $_ := set $out "requests" . }}{{- end -}}
{{- with $limits }}{{- $_ := set $out "limits" . }}{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Normalize a resource name from values to its Kubernetes key.
*/}}
{{- define "garage.resourceKey" -}}
{{- if eq . "ephemeralStorage" -}}ephemeral-storage{{- else -}}{{ . }}{{- end -}}
{{- end -}}
{{/*
Multiply a Kubernetes quantity by a factor, keeping its suffix.
Input: dict "quantity" <string|number> "factor" <float64>.
*/}}
{{- define "garage.multiplyQuantity" -}}
{{- $quantity := include "garage.formatNumber" .quantity -}}
{{- $number := regexFind "^[0-9]+(\\.[0-9]+)?" $quantity -}}
{{- if not $number -}}
{{- fail (printf "cannot parse resource quantity %q" $quantity) -}}
{{- end -}}
{{- $suffix := trimPrefix $number $quantity -}}
{{- if not (regexMatch "^(m|k|M|G|T|P|E|Ki|Mi|Gi|Ti|Pi|Ei)?$" $suffix) -}}
{{- fail (printf "unsupported resource quantity suffix %q in %q" $suffix $quantity) -}}
{{- end -}}
{{- $product := mulf (float64 $number) .factor -}}
{{- if eq $product (floor $product) -}}
{{- printf "%d%s" (int64 $product) $suffix -}}
{{- else -}}
{{- printf "%s%s" (include "garage.formatNumber" $product) $suffix -}}
{{- end -}}
{{- end -}}
{{/*
Render a number as a plain decimal string, never in exponential notation
(issue #157): the null-safe fallback helpers pass values through fromYaml,
which parses every number as float64, and Go's "%v" switches float64 to
exponential notation from about 1e6 upwards - "1.048576e+06" is not a valid
Kubernetes quantity. Strings ("512Mi") are passed through untouched.
*/}}
{{- define "garage.formatNumber" -}}
{{- if kindIs "float64" . -}}
{{- regexReplaceAll "\\.?0+$" (printf "%.10f" .) "" -}}
{{- else -}}
{{- printf "%v" . -}}
{{- end -}}
{{- end -}}
{{/*
Render the replica count for the StatefulSet.
Input: dict "scaleDown" <flag|nil> "replicas" <value|nil>.
- scaleDown: true takes precedence and always yields 0.
- otherwise an explicitly set replicas value is used as given, including an
explicit 0 - Helm's "default" would swallow 0, hence the kindIs "invalid"
nil check.
- a missing replicas value defaults to 1.
A NOTE SPECIFIC TO THIS CHART: more than one replica is not a cluster. Garage
nodes find each other through RPC and a cluster LAYOUT, and this chart renders
neither peer bootstrapping nor the kubernetes_discovery CRD (see values.yaml).
A second replica would come up as a lone node with its own empty metadata db on
its own volume, and with --single-node it refuses to start at all once it sees
the other node. The value exists because guideline 11 requires it and because
0 is the documented way to stop the release; the schema caps it at 1.
*/}}
{{- define "garage.replicas" -}}
{{- if .scaleDown -}}
0
{{- else if not (kindIs "invalid" .replicas) -}}
{{- .replicas -}}
{{- else -}}
1
{{- end -}}
{{- end -}}
{{/*
Merge a user-supplied values sub-tree onto a dict of chart defaults so that a
null / empty override falls back to the defaults instead of erasing them
(issues #61, #99). Input: dict "defaults" <dict> "given" <dict|nil> "path"
<string>. Output: YAML of the merged dict (consume it with fromYaml).
An override key written without children is YAML null, and Helm treats that
null as a deletion of the chart's default. This helper defines the opposite
semantics - an empty block means "chart defaults apply" - while an explicitly
set value still wins, including false and 0.
*/}}
{{- define "garage.defaultedDict" -}}
{{- $defaults := .defaults | default dict -}}
{{- $given := .given -}}
{{- $path := .path | default "value" -}}
{{- if kindIs "invalid" $given -}}
{{- $given = dict -}}
{{- end -}}
{{- if not (kindIs "map" $given) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" $path (kindOf $given) $given) -}}
{{- end -}}
{{- $out := deepCopy $defaults -}}
{{- range $key, $value := $given -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $default := index $defaults $key -}}
{{- if and (kindIs "map" $value) (kindIs "map" $default) -}}
{{- $_ := set $out $key (fromYaml (include "garage.defaultedDict" (dict "defaults" $default "given" $value "path" (printf "%s.%s" $path $key)))) -}}
{{- else if kindIs "map" $default -}}
{{- fail (printf "%s.%s must be a mapping or null, got %s (%v)" $path $key (kindOf $value) $value) -}}
{{- else -}}
{{- $_ := set $out $key $value -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Pick one sub-block out of a values block that may itself be null.
Input: dict "block" <dict|nil> "key" <string> "path" <string>.
Returns YAML of dict "value" <the sub-block, may be nil>.
*/}}
{{- define "garage.subBlock" -}}
{{- $block := .block -}}
{{- if kindIs "invalid" $block -}}
{{- $block = dict -}}
{{- end -}}
{{- if not (kindIs "map" $block) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" .path (kindOf $block) $block) -}}
{{- end -}}
{{- toYaml (dict "value" (index $block .key)) -}}
{{- end -}}
{{/*
THE EFFECTIVE GARAGE SETTINGS - everything the ConfigMap and the StatefulSet
read out of .Values.garage, merged onto this chart's defaults.
Why the whole settings block goes through the null-safe helper and not only the
three usual suspects (guideline 12.2: "the rule is about the shape, not about
those three block kinds"): every value below is dereferenced by a template, and
an erased one would not abort the render - it would render a garage.toml with
an empty right-hand side, or a containerPort with no number. The first produces
a pod that crash-loops on a config parse error, the second a manifest the API
server rejects at apply time. Both are failures that arrive after the render
looked fine, which is exactly the class #99 is about. Satisfactory shipped the
same shape with half its blocks null-safe and half not (Gitea #16), and the
half that was not took the Service's port with it.
The defaults here MUST stay in sync with values.yaml, which remains the
documented, user-facing place for them; this copy is only the fallback.
Usage: {{ $settings := fromYaml (include "garage.settings" .) }}
*/}}
{{- define "garage.settings" -}}
{{- $given := (fromYaml (include "garage.subBlock" (dict "block" .Values "key" "garage" "path" ".Values"))).value -}}
{{- $defaults := dict
"singleNode" true
"defaultAccessKey" true
"defaultBucket" true
"replicationFactor" 1
"dbEngine" "lmdb"
"metadataFsync" true
"dataFsync" false
"metadataAutoSnapshotInterval" "6h"
"compressionLevel" 1
"blockSize" "1M"
"bindAddress" "[::]"
"adminBindAddress" "0.0.0.0"
"ports" (dict "s3" 3900 "rpc" 3901 "web" 3902 "admin" 3903)
"s3" (dict "region" "garage" "rootDomain" nil)
"web" (dict "enabled" false "rootDomain" nil) -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" $given "path" "garage") -}}
{{- end -}}
{{/*
The three directories, defined once because TWO templates have to agree on
them: garage.toml names them and the StatefulSet mounts the volumes underneath
them. They are deliberately NOT values - a path that disagrees with its mount
is a pod that writes its metadata into the container's ephemeral layer, which
looks like it works until the pod is replaced.
The split is the point of this chart's storage layout (see values.yaml):
metadata and data are two SUBDIRECTORIES OF ONE VOLUME, the metadata snapshots
sit on a SECOND one. Garage creates all three itself on start
(src/model/garage.rs, create_dir_all for metadata_dir and data_dir;
src/model/snapshot.rs for the snapshots dir), which matters here because the
image is FROM scratch and carries no mkdir - nothing else in this pod could
create them.
*/}}
{{- define "garage.dataVolumePath" -}}/mnt/garage{{- end -}}
{{- define "garage.metadataDir" -}}{{ include "garage.dataVolumePath" . }}/meta{{- end -}}
{{- define "garage.dataDir" -}}{{ include "garage.dataVolumePath" . }}/data{{- end -}}
{{- define "garage.snapshotsDir" -}}/mnt/garage-snapshots{{- end -}}
{{/*
The effective security contexts, probes and resources (issue #99).
The defaults below MUST stay in sync with values.yaml. They reproduce THIS
chart's defaults, not the repo standard - see the fsGroupChangePolicy note.
*/}}
{{- define "garage.podSecurityContext" -}}
{{- $given := (fromYaml (include "garage.subBlock" (dict "block" . "key" "pod" "path" "garage.securityContext"))).value -}}
{{- $defaults := dict
"runAsNonRoot" true
"runAsUser" 20000
"runAsGroup" 1000
"fsGroup" 1000
"fsGroupChangePolicy" "OnRootMismatch"
"seccompProfile" (dict "type" "RuntimeDefault") -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" $given "path" "garage.securityContext.pod") -}}
{{- end -}}
{{- define "garage.containerSecurityContext" -}}
{{- $given := (fromYaml (include "garage.subBlock" (dict "block" . "key" "container" "path" "garage.securityContext"))).value -}}
{{- $defaults := dict
"allowPrivilegeEscalation" false
"readOnlyRootFilesystem" true
"runAsNonRoot" true
"capabilities" (dict "drop" (list "ALL") "add" (list)) -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" $given "path" "garage.securityContext.container") -}}
{{- end -}}
{{/*
LIVENESS: a TCP connect to the admin port, and NOT the /health endpoint the
readiness probe uses. The reason is the one zipline wrote down (README,
guideline 12.3) in its sharpest form: /health answers 503 until the cluster
LAYOUT exists (src/rpc/system.rs, "Layout not yet configured, special case" ->
ClusterHealthStatus::Unavailable -> 503 in src/api/admin/special.rs). A node
waiting for its layout - every install with garage.singleNode: false, and any
install whose layout was lost - would therefore be killed and restarted
forever, and a restart is precisely what cannot fix it.
What the TCP probe does prove is what a liveness probe is for: the process is
alive and its listener is open. A wedged process that stops accepting is
restarted; a healthy process that has nothing to serve yet is not.
*/}}
{{- define "garage.livenessProbe" -}}
{{- $defaults := dict
"tcpSocket" (dict "port" "admin")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.livenessProbe") -}}
{{- end -}}
{{/*
READINESS: GET /health on the admin port - the one endpoint that answers the
question a readiness probe asks, namely "can this node serve requests".
Measured against the source at tag v2.4.1 rather than chosen:
- the S3 API server is spawned independently of the layout state
(src/garage/server.rs, the S3/web/admin servers are all spawned after
initial_config), so the S3 socket binds and ANSWERS - with errors - before
any layout exists. A tcpSocket probe on port 3900 would therefore report a
service ready that accepts nothing, which is the lie this chart is not
allowed to tell.
- /health needs no token (@special in src/api/admin/router_v2.rs) and maps
ClusterHealthStatus straight onto the status code: Healthy and Degraded ->
200, Unavailable -> 503 with "Quorum is not available for some/all
partitions, reads and writes will fail". An unconfigured layout is a
special case in src/rpc/system.rs and yields Unavailable.
So with garage.singleNode: true the pod becomes Ready a moment after start
(initial_config applies the layout BEFORE the servers are spawned), and with
singleNode: false it stays not-Ready until an operator applies a layout by hand
- which is the truth, not a defect: until then the S3 endpoint accepts nothing
and no Service should route to it.
*/}}
{{- define "garage.readinessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/health" "port" "admin")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.readinessProbe") -}}
{{- end -}}
{{/*
STARTUP: TCP again, for the same reason as liveness - a startup probe that
fails forever kills the container, so it must not depend on a layout that only
an operator can create.
Budget = periodSeconds * failureThreshold = 5 * 24 = 120s. It covers the slow
part of a real start, which is not Garage: attaching and mounting two network
volumes, plus opening (and on an unclean shutdown recovering) the LMDB
metadata database. Garage's own startup work after that is sub-second.
*/}}
{{- define "garage.startupProbe" -}}
{{- $defaults := dict
"tcpSocket" (dict "port" "admin")
"periodSeconds" 5
"timeoutSeconds" 5
"failureThreshold" 24 -}}
{{- include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.startupProbe") -}}
{{- end -}}
{{/*
The effective resources block, fed into the resources helper above so that an
erased "resources:" still yields the chart's requests and computed limits.
*/}}
{{- define "garage.effectiveResources" -}}
{{- $defaults := dict
"limitFactor" 3
"requests" (dict "cpu" "100m" "memory" "512Mi" "ephemeral-storage" "32Mi") -}}
{{- $merged := fromYaml (include "garage.defaultedDict" (dict "defaults" $defaults "given" . "path" "garage.resources")) -}}
{{- include "garage.resources" $merged -}}
{{- end -}}
{{/*
The additive env block, rendered through tpl with the value, secretKeyRef and
configMapKeyRef forms (guideline 10).
ADDITIVE MEANS ADDITIVE: an entry repeating a name this chart already sets does
not replace it. At install time both reach the pod and the later one wins; on
an EXISTING release the pair collapses to the chart's own value, because
Kubernetes merges env by name - and helm upgrade still reports success. So do
not use this block to override GARAGE_RPC_SECRET or the three
GARAGE_DEFAULT_* variables; change the values that produce them.
*/}}
{{- define "garage.env" -}}
{{- $root := .root -}}
{{- range $value := .env }}
{{- if $value.value }}
- name: {{ $value.name | quote }}
# quote is required, not cosmetic: a value like "3900" or "true" renders
# unquoted as a YAML number/bool, and the API server then rejects the whole
# manifest with "cannot unmarshal number into Go struct field EnvVar.value of
# type string".
value: {{ tpl $value.value $root | quote }}
{{- else if $value.valueFrom }}
{{- if $value.valueFrom.secretKeyRef }}
- name: {{ $value.name }}
valueFrom:
secretKeyRef:
name: {{ tpl $value.valueFrom.secretKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.secretKeyRef.key $root | quote }}
{{- else if $value.valueFrom.configMapKeyRef }}
- name: {{ $value.name }}
valueFrom:
configMapKeyRef:
name: {{ tpl $value.valueFrom.configMapKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.configMapKeyRef.key $root | quote }}
{{- end }}
{{- end }}
{{- end }}
{{- end -}}
-173
View File
@@ -1,173 +0,0 @@
{{- $settings := fromYaml (include "garage.settings" .) }}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-configmap
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
data:
# The complete garage.toml. It carries NO SECRET: the RPC secret arrives as
# the environment variable GARAGE_RPC_SECRET (see the StatefulSet and
# values.yaml for why the file form cannot be used here), so this ConfigMap
# is safe to read and the checksum/config annotation on the pod template
# still means "the configuration changed".
#
# It is mounted with subPath at /etc/garage.toml, which is the path Garage
# looks at without being told (-c defaults to /etc/garage.toml). A subPath
# mount is never refreshed by the kubelet, which is exactly why the checksum
# annotation is mandatory rather than decorative.
garage.toml: |
# Managed by the garage Helm chart - changes here are overwritten on the
# next helm upgrade. Edit the chart's values instead.
# ONE node, and the replication factor is 1 BY DECISION, with the risk
# named (Gitea #81): longhorn-encrypted-ns already keeps three replicas of
# the encrypted volume, and Garage replication 3 on top of that would be
# nine times the raw data for two layers that know nothing about each
# other.
#
# WHAT THAT COSTS, because it is not free: upstream says of
# replication_factor = 1 "Do not use this for anything else than test
# deployments", and the reason is on its known-issues page - the same
# factor applies to the METADATA, so a single-replica setup is especially
# exposed to metadata corruption, and a corrupted database file is
# "generally not recoverable". Longhorn does not help against that: it
# replicates a corrupted file three times. Redundancy against device
# failure, yes; against file corruption, no. The three settings below are
# the countermeasures that do work at factor 1, and none of them is
# optional.
replication_factor = {{ $settings.replicationFactor | int64 }}
# LMDB is Garage's default since 0.9 and the faster engine. The price is
# named: the reported corruption after an unclean shutdown stays possible,
# the settings below only lower the probability, and at factor 1 there is
# no peer to recover from - only the last snapshot. Upstream ties its LMDB
# recommendation to replication factor >= 2; running it at 1 is this
# chart's deliberate deviation.
#
# The alternative is "sqlite", which is slower on metadata throughput. With
# a single mail server as the only consumer that ceiling is unlikely to be
# reached, so sqlite is a defensible choice for anyone who weighs the
# durability of a single-file database higher. "fjall" is experimental.
db_engine = "{{ $settings.dbEngine }}"
# metadata_dir and data_dir are two SUBDIRECTORIES OF ONE VOLUME; the
# snapshots live on a second one. Longhorn snapshots are atomic PER VOLUME
# and never across volumes, so keeping the index and the blocks together is
# what prevents the one failure mode that matters for a mail store: a
# backup whose metadata points at objects whose blocks were not written yet
# ("the mailbox lists a message whose body is missing").
#
# The snapshots are on the OTHER volume for the opposite reason: a recovery
# point must not die with the volume whose corruption it is there to heal.
metadata_dir = "{{ include "garage.metadataDir" . }}"
data_dir = "{{ include "garage.dataDir" . }}"
metadata_snapshots_dir = "{{ include "garage.snapshotsDir" . }}"
# metadata_fsync IS OFF IN GARAGE'S DEFAULTS, and this is the single
# setting most easily lost in a "tidy up the obvious defaults" pass. It is
# set here on purpose: Longhorn's block layer cannot help if the engine
# never calls fsync - a write that was never handed to the kernel is not in
# any replica and not in any snapshot. Garage's own example configuration
# pairs it with metadata_auto_snapshot_interval, as here.
metadata_fsync = {{ $settings.metadataFsync }}
# data_fsync stays off, which is Garage's default and the right trade for
# the blocks: a lost block is one object, is detectable by a scrub, and is
# replaceable from the sender's copy; a lost metadata write can take the
# whole index with it. Turning it on costs an fsync per written block.
data_fsync = {{ $settings.dataFsync }}
# THE ONLY CONSISTENT RECOVERY POINT THIS DEPLOYMENT HAS. File-system
# snapshots of metadata_dir guarantee nothing - Garage says so itself: a
# snapshot taken during a metadata write is corrupt and useless as a
# restore point, which is why Garage recommends ITS OWN snapshot mechanism
# in all cases. This is the correction that reversed a premise of the
# original plan: Longhorn backs up the blocks, not the metadata.
#
# KNOW THE WINDOW: Garage keeps exactly TWO snapshots (KEEP_SNAPSHOTS = 2),
# the first one after interval/2 and every later one after interval times a
# random 1.0-1.2. At "6h" that means the first after ~3h and a restore
# window of roughly 6-14.4h afterwards. A corruption discovered later than
# that is older than every snapshot that still exists.
metadata_auto_snapshot_interval = "{{ $settings.metadataAutoSnapshotInterval }}"
block_size = "{{ $settings.blockSize }}"
# An integer is the zstd level; the literal "none" disables compression and
# has to be QUOTED to stay valid TOML. Garage's deserializer takes exactly
# those two forms - and 0 is not the second one: it is a valid integer that
# zstd reads as its default level (3), so writing 0 to "switch compression
# off" quietly turns it up.
{{- if kindIs "string" $settings.compressionLevel }}
compression_level = {{ $settings.compressionLevel | quote }}
{{- else }}
compression_level = {{ $settings.compressionLevel | int64 }}
{{- end }}
# Inter-node RPC. There are no other nodes here (no bootstrap_peers, no
# kubernetes_discovery - see values.yaml), but Garage binds this listener
# regardless and refuses to start without an rpc_secret, which arrives from
# the environment.
rpc_bind_addr = "{{ $settings.bindAddress }}:{{ $settings.ports.rpc | int64 }}"
[s3_api]
api_bind_addr = "{{ $settings.bindAddress }}:{{ $settings.ports.s3 | int64 }}"
# The region name clients must use. It is not a cloud region, it is a label
# both sides have to agree on - a consumer configured for a different
# region gets a signature error, not a redirect.
s3_region = "{{ $settings.s3.region }}"
{{- with $settings.s3.rootDomain }}
# Vhost-style bucket access (<bucket>.<root_domain>). Only useful with an
# Ingress and a wildcard DNS entry in front of it; path-style access works
# without it and is what an in-cluster consumer uses.
root_domain = {{ . | quote }}
{{- end }}
{{- if $settings.web.enabled }}
[s3_web]
# Static website hosting, OFF by default (see values.yaml): it is a second
# HTTP listener and serves nothing at all without a root_domain, while the
# consumer this chart was built for speaks S3.
bind_addr = "{{ $settings.bindAddress }}:{{ $settings.ports.web | int64 }}"
{{- with $settings.web.rootDomain }}
root_domain = {{ . | quote }}
{{- end }}
{{- end }}
[admin]
# Bound on 0.0.0.0 rather than [::] BECAUSE ALL THREE PROBES USE THIS PORT.
# A dual-stack listener on [::] only accepts the kubelet's IPv4 connection
# while net.ipv6.bindv6only is 0 - true by default on Linux, but a node
# sysctl this chart cannot see, and the failure would be a pod that never
# becomes Ready for a reason nothing in the manifest points at. The S3 and
# RPC listeners keep the upstream [::] default; set garage.bindAddress to
# "0.0.0.0" on an IPv4-only cluster, or garage.adminBindAddress to "[::]"
# on an IPv6-only one.
#
# NO admin_token IS SET, and that is not an omission: /health, the one
# endpoint the probes use, is explicitly token-free (@special in the admin
# router), and nothing in this deployment calls the authenticated API - the
# CLI inside the pod talks RPC, not HTTP. A token here would be a fifth
# vault field guarding nothing that is in use.
#
# WHAT IS AND IS NOT REACHABLE AS A RESULT, stated precisely because "no
# token" reads like "nothing is open":
# /health open, by design, and the probes depend on it
# /metrics OPEN TOO - the metrics_require_token switch defaults to
# false, so without a metrics credential the endpoint needs no
# authentication. That is internal metrics on a ClusterIP port;
# to close it, set that switch true and add a credential.
#
# (And a note for whoever edits these lines: do NOT write them
# as "<key> = <value>" with a credential-ish key name. Trivy's
# KSV-0109 reads an assignment inside this ConfigMap's DATA -
# comments included - as a stored secret and reports HIGH. It
# cost one run here, 2026-10-03.)
# everything else 401/403 until a token exists. Note that a config
# token is not the only way to get one: Garage 2.x also keeps
# admin tokens in the cluster ("garage admin-token create"),
# and those work without any entry in this file.
# If you do want a config token, it is admin_token_file / the
# GARAGE_ADMIN_TOKEN environment variable - and the file form carries the
# same permission trap as rpc_secret_file (see values.yaml).
api_bind_addr = "{{ $settings.adminBindAddress }}:{{ $settings.ports.admin | int64 }}"
-50
View File
@@ -1,50 +0,0 @@
{{- /*
OFF BY DEFAULT (ingress.enabled: false), because the consumer this chart was
built for speaks S3 inside the cluster and needs no published endpoint. The
values exist anyway, built to this repo's ingress convention, so that
publishing the endpoint later is a values change rather than a chart change.
GUIDELINE 1, CHECKED IN BOTH DIRECTIONS (README, "Values that must not lie"):
nothing outside this template reads ingress.domain. The S3 API's vhost-style
suffix is its own value, garage.s3.rootDomain, because the two are not the same
thing - a wildcard vhost domain and the single host an Ingress answers on. So
with the Ingress switched off, BOTH domain and clusterIssuer are free, and
their required() lives here where it can actually fire.
WHAT IT PUBLISHES: the S3 API. There is no web UI to expose - Garage's admin
API is JSON and the authenticated half is closed in this chart (no token), and
the project's web interface is a separate program that is not part of this
image. Anyone publishing this should know what the object they are putting on
the internet is: an S3 endpoint that authenticates every request with a
signature, whose credentials are the two fields of the s3 vault item.
*/ -}}
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-ingress
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
annotations:
cert-manager.io/cluster-issuer: {{ required "A Cluster-Issuer must be provided (ingress.clusterIssuer)" .Values.ingress.clusterIssuer }}
{{- with .Values.ingress.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
ingressClassName: {{ .Values.ingress.className }}
rules:
- host: {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
http:
paths:
- backend:
service:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
port:
name: s3
path: /
pathType: Prefix
tls:
- hosts:
- {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
secretName: tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
{{- end }}
Loaded 100 of 212 files, more files were not shown because too many files have changed in this diff. Show more