jklein
59d2614384
Validate Helm Charts / Detect changed charts (pull_request) Successful in 5s
Validate Helm Charts / Build ${{ matrix.chart }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
Validate Helm Charts / Resource quantities (pull_request) Successful in 18s
Validate Helm Charts / Selector match (pull_request) Successful in 14s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 14s
Validate Helm Charts / Bump level (pull_request) Successful in 7s
Validate Helm Charts / Trivy config cyberchef (pull_request) Successful in 20s
Validate Helm Charts / Trivy config matrix-synapse (pull_request) Successful in 22s
Validate Helm Charts / Trivy config samba (pull_request) Successful in 21s
SonarQube Analysis / SonarQube Analysis (push) Successful in 48s
The version-bump hook classified every app update that was not a major as a chart patch, so an app minor arrived as a patch - the mechanism behind the sonarr incident, where a security default was switched off inside a bump the conventions told the reviewer not to read (Gitea #128). ci/renovate-bump-chart.sh now takes the level from Renovate's update type, passed in as a second argument: major -> major, minor -> MINOR, patch/digest/pin -> patch. The level comes from Renovate rather than from comparing appVersion because an appVersion comparison is blind to the case that prompted the rule - freesailarr keeps a hand-maintained appVersion for ten upstreams, and prowlarr's 2.5.2 -> 2.6.5 moved an image tag in values.yaml while appVersion stood still. Three properties come with it: - Maximum, not replacement. The chart jump is the greater of the chart's own classification and the app jump, so a value that becomes required stays a major under an app patch. Mechanically it is also what makes a grouped branch deterministic: the hook runs once per update from the same base commit, so without it the last call would win. - Non-semver app versions stay at patch for a minor (seven charts). In CalVer and date tags the second position is a month; in a four-part or suffixed version the positions are not semver positions. An app major is not subject to it. - A missing or unknown update type falls back to the pre-#128 behaviour. postUpgradeTasks only runs where the command is allow-listed on the shared instance, which is not visible from here and fails silently, so the worst case is the old classification rather than every chart PR red on the version check. ci/run-bump-level-test.sh holds 49 cases against the version string the script writes, wired up as the bump-level job: a wrong level is a valid version string, so the PR would go green and the only symptom would be a number that understates what it publishes. The convention texts follow in CLAUDE.md and README, including the limit the decision deliberately keeps: the reading duty was rejected, so the incident stays repeatable and the honest number is the whole of the remedy. Fixes #128
230 lines
13 KiB
Bash
Executable File
230 lines
13 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Bump-level test: every case of ci/renovate-bump-chart.sh, held against the
|
|
# version string it writes into a throwaway chart.
|
|
#
|
|
# Usage: ci/run-bump-level-test.sh
|
|
#
|
|
# WHY THIS TEST EXISTS (Gitea #128)
|
|
#
|
|
# Until #128 the hook was a calculator with one branch: app major -> chart
|
|
# major, everything else -> chart patch. With the update type coming from
|
|
# Renovate it became DECIDING LOGIC — it is now the thing that says how big a
|
|
# change the cluster is being offered, and the chart version is the only signal
|
|
# anything downstream sees (README, "Chart dependencies": Fleet's `fleet.yaml`
|
|
# in `cluster-config` sees our chart version and nothing inside the package).
|
|
#
|
|
# And its failure mode is silence. A wrong level is a valid version string: the
|
|
# PR goes green, the chart publishes, and the only symptom is a number that
|
|
# understates what it ships. Nothing in the pipeline reads it, so nothing but a
|
|
# test can.
|
|
#
|
|
# HOW A CASE IS BUILT, AND WHY A REAL GIT REPO
|
|
#
|
|
# The script reads the BASE commit (`git show HEAD:<chart>/Chart.yaml`) and the
|
|
# working tree, so a case is not a function call — it is a repository with a
|
|
# commit and a dirty file. Each case therefore gets a throwaway `git init` in a
|
|
# unique temp directory, one commit carrying the base Chart.yaml, and the new
|
|
# Chart.yaml written over it uncommitted. That is exactly the state Renovate's
|
|
# `postUpgradeTasks` hook runs in.
|
|
#
|
|
# Signing is switched off per command (`-c commit.gpgsign=false`): this
|
|
# workspace signs every commit through an SSH signer, and a throwaway repo has
|
|
# no business invoking it. The temp root comes from `mktemp -d`, because /tmp is
|
|
# shared with whatever else runs on this machine.
|
|
#
|
|
# The runners have no pyyaml, no yq and no Docker — the same constraint
|
|
# ci/run-selector-match-test.sh and ci/run-quantity-render-test.sh work under.
|
|
# None is needed: the fixtures are six-line Chart.yaml files written by printf,
|
|
# and the assertion is one `version:` line read back with sed. Helm is not
|
|
# needed either, which is why this job carries no Helm step.
|
|
#
|
|
# WHAT IS ASSERTED, AND WHAT IS NOT
|
|
#
|
|
# Asserted: the full `version:` line (so the `+up` suffix and its dash-to-dot
|
|
# normalization are covered by every case), the exit code, and for the cases
|
|
# that must leave the chart alone, that the line is untouched.
|
|
#
|
|
# NOT asserted, and not assertable from here: that Renovate passes the update
|
|
# type at all. The hook only runs when the command is allow-listed in the
|
|
# *instance* config, which lives outside this repository — and a pattern that
|
|
# does not match is skipped silently. That is why a missing second argument is
|
|
# itself a case here (it must fall back, not fail), and why the PR that
|
|
# introduced this carries a dry-run instead of a claim.
|
|
set -euo pipefail
|
|
|
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
SCRIPT="$ROOT/ci/renovate-bump-chart.sh"
|
|
|
|
[ -x "$SCRIPT" ] || {
|
|
echo "FAIL: $SCRIPT is not executable" >&2
|
|
exit 1
|
|
}
|
|
|
|
TMP="$(mktemp -d "${TMPDIR:-/tmp}/bump-level-test.XXXXXX")"
|
|
trap 'rm -rf "$TMP"' EXIT
|
|
|
|
GIT_OPTS=(-c user.name=ci -c user.email=ci@example.invalid -c commit.gpgsign=false -c tag.gpgsign=false)
|
|
|
|
passed=0
|
|
failed=0
|
|
|
|
chart_yaml() { # <name> <version> <appVersion>
|
|
printf 'apiVersion: v2\nname: %s\ndescription: test fixture\ntype: application\nversion: %s\nappVersion: "%s"\nicon: https://example.com/icon.png\n' \
|
|
"$1" "$2" "$3"
|
|
}
|
|
|
|
# Builds a repository whose HEAD carries <base-content> for <dir>/Chart.yaml
|
|
# (skipped when empty, which is the "new chart on this branch" case) and whose
|
|
# working tree carries <tree-content>. Prints the repository path.
|
|
scenario() { # <dir> <base-content> <tree-content>
|
|
local dir=$1 base=$2 tree=$3 repo
|
|
repo="$(mktemp -d "$TMP/case.XXXXXX")"
|
|
git -C "$repo" init -q
|
|
printf 'root commit so HEAD exists\n' >"$repo/README.md"
|
|
git -C "$repo" add README.md
|
|
git -C "$repo" "${GIT_OPTS[@]}" commit -qm "root"
|
|
mkdir -p "$repo/$dir"
|
|
if [ -n "$base" ]; then
|
|
printf '%s' "$base" >"$repo/$dir/Chart.yaml"
|
|
git -C "$repo" add "$dir/Chart.yaml"
|
|
git -C "$repo" "${GIT_OPTS[@]}" commit -qm "base chart"
|
|
fi
|
|
[ -z "$tree" ] || printf '%s' "$tree" >"$repo/$dir/Chart.yaml"
|
|
printf '%s' "$repo"
|
|
}
|
|
|
|
version_of() { sed -n 's/^version:[[:space:]]*//p' "$1" | head -n 1; }
|
|
|
|
report() { # <name> <expected> <got> <expected-rc> <got-rc> <output>
|
|
if [ "$2" = "$3" ] && [ "$4" = "$5" ]; then
|
|
passed=$((passed + 1))
|
|
printf ' ok %s\n' "$1"
|
|
else
|
|
failed=$((failed + 1))
|
|
printf ' FAIL %s\n expected version %-24s rc %s\n got version %-24s rc %s\n output: %s\n' \
|
|
"$1" "$2" "$4" "$3" "$5" "$6"
|
|
fi
|
|
}
|
|
|
|
# The ordinary case: one chart, a base version, an app version that may or may
|
|
# not have moved, and an update type. UPDATE_TYPE of "-" calls the script with
|
|
# ONE argument (the allow-list / older-Renovate case), "" passes an empty
|
|
# second argument (a template field that resolved to nothing).
|
|
bump() { # <name> <base-version> <base-app> <tree-version> <tree-app> <update-type> <expected-version> [expected-rc]
|
|
local name=$1 base_version=$2 base_app=$3 tree_version=$4 tree_app=$5 utype=$6 expected=$7 rc_expected=${8:-0}
|
|
local repo out rc got
|
|
repo="$(scenario demo "$(chart_yaml demo "$base_version" "$base_app")" "$(chart_yaml demo "$tree_version" "$tree_app")")"
|
|
rc=0
|
|
if [ "$utype" = "-" ]; then
|
|
out="$(cd "$repo" && "$SCRIPT" demo 2>&1)" || rc=$?
|
|
else
|
|
out="$(cd "$repo" && "$SCRIPT" demo "$utype" 2>&1)" || rc=$?
|
|
fi
|
|
got="$(version_of "$repo/demo/Chart.yaml")"
|
|
report "$name" "$expected" "$got" "$rc_expected" "$rc" "$out"
|
|
}
|
|
|
|
echo "== update type decides the level (plain semver appVersion) =="
|
|
bump "major -> chart major" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 2.0.0 major 2.0.0+up2.0.0
|
|
bump "minor -> chart minor" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.5.0 minor 1.3.0+up1.5.0
|
|
bump "patch -> chart patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.4.1 patch 1.2.4+up1.4.1
|
|
bump "digest -> chart patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.4.0 digest 1.2.4+up1.4.0
|
|
bump "pin -> chart patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.4.0 pin 1.2.4+up1.4.0
|
|
bump "pinDigest -> chart patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.4.0 pinDigest 1.2.4+up1.4.0
|
|
bump "minor resets the patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.5.0 minor 1.3.0+up1.5.0
|
|
bump "major resets minor and patch" 4.5.6+up1.4.0 1.4.0 4.5.6+up1.4.0 2.0.0 major 5.0.0+up2.0.0
|
|
|
|
echo "== the case that prompted the rule: appVersion never moves (freesailarr) =="
|
|
# prowlarr 2.5.2 -> 2.6.5 is an image tag in values.yaml; freesailarr's
|
|
# appVersion is a hand-maintained marker and stays at 1.0.0. An appVersion
|
|
# comparison sees nothing and would bump the patch.
|
|
bump "sidecar/bundle minor, appVersion unchanged" 3.1.1+up1.0.0 1.0.0 3.1.1+up1.0.0 1.0.0 minor 3.2.0+up1.0.0
|
|
bump "sidecar/bundle patch, appVersion unchanged" 3.1.1+up1.0.0 1.0.0 3.1.1+up1.0.0 1.0.0 patch 3.1.2+up1.0.0
|
|
|
|
echo "== non-semver app versions stay at patch (all seven on origin/main) =="
|
|
bump "rss-bridge date tag, month roll" 2.1.1+up2026.09.30 2026-09-30 2.1.1+up2026.09.30 2026-10-05 minor 2.1.2+up2026.10.05
|
|
bump "homeassistant CalVer" 6.2.0+up2026.8.3 2026.8.3 6.2.0+up2026.8.3 2026.9.0 minor 6.2.1+up2026.9.0
|
|
bump "searxng CalVer plus hash" 1.1.0+up2026.9.13.3b8009065 2026.9.13-3b8009065 1.1.0+up2026.9.13.3b8009065 2026.10.1-aa12bb34c minor 1.1.1+up2026.10.1.aa12bb34c
|
|
bump "minecraft-server CalVer plus JRE" 6.0.1+up2026.8.2.java25 2026.8.2-java25 6.0.1+up2026.8.2.java25 2026.9.1-java25 minor 6.0.2+up2026.9.1.java25
|
|
bump "apache-tika four parts" 10.0.2+up3.3.1.0.full 3.3.1.0-full 10.0.2+up3.3.1.0.full 3.4.0.0-full minor 10.0.3+up3.4.0.0.full
|
|
bump "teamspeak prerelease line" 5.1.3+up6.0.0.beta13.1 6.0.0-beta13.1 5.1.3+up6.0.0.beta13.1 6.0.0-beta14.1 minor 5.1.4+up6.0.0.beta14.1
|
|
bump "valkey flavour suffix" 1.0.0+up9.1.2.alpine 9.1.2-alpine 1.0.0+up9.1.2.alpine 9.2.0-alpine minor 1.0.1+up9.2.0.alpine
|
|
# A major is NOT subject to the fallback - a year rolling over was a chart major
|
|
# before #128 as well.
|
|
bump "date tag, year roll, major" 2.1.1+up2026.09.30 2026-09-30 2.1.1+up2026.09.30 2027-01-05 major 3.0.0+up2027.01.05
|
|
bump "four parts, app major, major" 10.0.2+up3.3.1.0.full 3.3.1.0-full 10.0.2+up3.3.1.0.full 4.0.0.0-full major 11.0.0+up4.0.0.0.full
|
|
# The year guard is a guard, not a blanket: a v-prefixed plain line still counts.
|
|
bump "v-prefixed plain semver, minor" 7.2.0+upv1.159.0 v1.159.0 7.2.0+upv1.159.0 v1.160.0 minor 7.3.0+upv1.160.0
|
|
|
|
echo "== maximum, not replacement =="
|
|
# A chart-own bump already in the working tree stands, whatever the app jump is.
|
|
bump "chart major already on branch, app patch" 1.2.3+up1.4.0 1.4.0 2.0.0+up1.4.0 1.4.1 patch 2.0.0+up1.4.1
|
|
bump "chart minor already on branch, app patch" 1.2.3+up1.4.0 1.4.0 1.3.0+up1.4.0 1.4.1 patch 1.3.0+up1.4.1
|
|
# A grouped branch: the hook runs once per update from the same base, so the
|
|
# minor must survive a patch call that follows it.
|
|
bump "minor on branch, later patch call" 3.1.1+up1.0.0 1.0.0 3.2.0+up1.0.0 1.0.0 patch 3.2.0+up1.0.0
|
|
bump "patch on branch, later minor call" 3.1.1+up1.0.0 1.0.0 3.1.2+up1.0.0 1.0.0 minor 3.2.0+up1.0.0
|
|
bump "repeated minor call is idempotent" 3.1.1+up1.0.0 1.0.0 3.2.0+up1.0.0 1.0.0 minor 3.2.0+up1.0.0
|
|
bump "repeated major call is idempotent" 1.2.3+up1.4.0 1.4.0 2.0.0+up2.0.0 2.0.0 major 2.0.0+up2.0.0
|
|
bump "major on branch wins over a later minor" 1.2.3+up1.4.0 1.4.0 2.0.0+up1.4.0 1.5.0 minor 2.0.0+up1.5.0
|
|
# Stale branch after a rebase: the base moved ahead of what the branch carries,
|
|
# so the recomputed version wins and nothing is kept.
|
|
bump "stale branch version is not kept" 3.3.0+up1.0.0 1.0.0 3.1.2+up1.0.0 1.0.0 patch 3.3.1+up1.0.0
|
|
|
|
echo "== missing and unknown update type fall back to the pre-#128 behaviour =="
|
|
bump "no second argument, app major" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 2.0.0 - 2.0.0+up2.0.0
|
|
bump "no second argument, app minor" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.5.0 - 1.2.4+up1.5.0
|
|
bump "no second argument, app patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.4.1 - 1.2.4+up1.4.1
|
|
bump "no second argument, unchanged" 3.1.1+up1.0.0 1.0.0 3.1.1+up1.0.0 1.0.0 - 3.1.2+up1.0.0
|
|
bump "empty second argument, app major" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 2.0.0 "" 2.0.0+up2.0.0
|
|
bump "empty second argument, app minor" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.5.0 "" 1.2.4+up1.5.0
|
|
bump "unknown type, app minor" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.5.0 replacement 1.2.4+up1.5.0
|
|
bump "unknown type, app major" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 2.0.0 lockFileMaintenance 2.0.0+up2.0.0
|
|
bump "unknown type, maximum still holds" 1.2.3+up1.4.0 1.4.0 2.0.0+up1.4.0 1.4.1 nonsense 2.0.0+up1.4.1
|
|
|
|
echo "== charts the script must leave alone, and the hard failures =="
|
|
bump "unparsable chart semver fails loudly" 1.2+up1.4.0 1.4.0 1.2+up1.4.0 1.5.0 minor 1.2+up1.4.0 1
|
|
bump "missing appVersion fails loudly" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 "" minor 1.2.3+up1.4.0 1
|
|
|
|
new_chart_repo="$(scenario demo "" "$(chart_yaml demo 0.1.0+up1.0.0 1.0.0)")"
|
|
rc=0
|
|
out="$(cd "$new_chart_repo" && "$SCRIPT" demo minor 2>&1)" || rc=$?
|
|
report "a chart new on the branch keeps its version" \
|
|
"0.1.0+up1.0.0" "$(version_of "$new_chart_repo/demo/Chart.yaml")" 0 "$rc" "$out"
|
|
case "$out" in
|
|
*"new on this branch"*) passed=$((passed + 1)); printf ' ok and says so\n' ;;
|
|
*) failed=$((failed + 1)); printf ' FAIL new chart: no explanation in the output: %s\n' "$out" ;;
|
|
esac
|
|
|
|
tpl_repo="$(scenario template-chart "$(chart_yaml template-chart 4.2.0+upxxx xxx)" "$(chart_yaml template-chart 4.2.0+upxxx xxx)")"
|
|
rc=0
|
|
out="$(cd "$tpl_repo" && "$SCRIPT" template-chart minor 2>&1)" || rc=$?
|
|
report "template-chart is never touched" \
|
|
"4.2.0+upxxx" "$(version_of "$tpl_repo/template-chart/Chart.yaml")" 0 "$rc" "$out"
|
|
|
|
# Renovate calls the hook for every update it makes, chart or not.
|
|
other_repo="$(scenario demo "$(chart_yaml demo 1.2.3+up1.4.0 1.4.0)" "$(chart_yaml demo 1.2.3+up1.4.0 1.4.0)")"
|
|
for arg in ".github/workflows" "ci/gateway-api-crds.sh" "." "" "not-a-chart"; do
|
|
rc=0
|
|
out="$(cd "$other_repo" && "$SCRIPT" "$arg" major 2>&1)" || rc=$?
|
|
report "non-chart argument '$arg' is a no-op" \
|
|
"1.2.3+up1.4.0" "$(version_of "$other_repo/demo/Chart.yaml")" 0 "$rc" "$out"
|
|
done
|
|
|
|
rc=0
|
|
out="$(cd "$other_repo" && "$SCRIPT" 2>&1)" || rc=$?
|
|
report "no arguments at all is a no-op" \
|
|
"1.2.3+up1.4.0" "$(version_of "$other_repo/demo/Chart.yaml")" 0 "$rc" "$out"
|
|
|
|
rc=0
|
|
out="$(cd "$other_repo" && "$SCRIPT" demo/ patch 2>&1)" || rc=$?
|
|
report "a trailing slash is accepted" \
|
|
"1.2.4+up1.4.0" "$(version_of "$other_repo/demo/Chart.yaml")" 0 "$rc" "$out"
|
|
|
|
echo
|
|
if [ "$failed" -gt 0 ]; then
|
|
printf 'bump-level test: %d passed, %d FAILED\n' "$passed" "$failed"
|
|
exit 1
|
|
fi
|
|
printf 'bump-level test: %d cases passed\n' "$passed"
|