Make the all-charts install-test repeatable, as one shared job
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
Validate Helm Charts / Resource quantities (pull_request) Successful in 10s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 21s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
Validate Helm Charts / Resource quantities (pull_request) Successful in 10s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 21s
The install-test matrix follows the charts a pull request changes, so a chart nobody has touched since the forge move carries a green history from a different runner - and that green says nothing about today. Gitea PR #48 showed what the answer is worth: correcting a comment in 21 charts made every one of them "changed" and turned the run into the first all-charts install-test after the move, 19 of 20 matrix charts Ready in 12m41s and exactly one not (rss-bridge, Gitea #51). By accident. The install-test job moves into the reusable .github/workflows/ install-test.yml and is called twice: by validate-charts.yml with the charts a pull request changed, and by the new install-test-all-charts.yml with all of them. Not copied for the second caller - a second copy is a second chance to test differently, unnoticed, which is the argument that gives ci/excluded-charts.txt exactly one parser. The parts that would drift are the ones that were paid for once already: the pinned kubectl with its skew check, envsubst, the containerised-job detection, the per-leg cluster name. Nothing about how a chart is installed changed. The moved job differs from its predecessor in the matrix source (inputs.charts instead of the detect job's output), an id on the install step, and a step that records the result; every other step is byte-identical. The all-charts run is workflow_dispatch only - a schedule would spend compute every day on a question that changes rarely - with an optional subset input, which is what lets a single chart be tested against the runner as Gitea #51 needs. One throwaway cluster per chart, no reuse: reuse saves time and changes what the test proves. An unknown or excluded chart name, and an empty matrix, are loud errors rather than a green run over nothing. Every leg writes its own outcome into the job summary next to the four facts that decide what the outcome means - date, runner label, K3S_IMAGE, kubectl version - and a final job records the verdict over the matrix. A green run nobody notes is unknown again next time. The excluded charts appear with their justification, from the one parser, as in the pull request pipeline. 20 charts in 12m41s is written down as the expectation, so a later deviation is visible as one. Fixes #52
This commit was merged in pull request #54.
This commit is contained in:
6 files changed
+686
-318
No files matched your search
@@ -5,7 +5,7 @@
|
||||
# (REQUIRED_ENV), and pipes the result straight into kubectl.
|
||||
#
|
||||
# The values come from a 1Password item holding a CI-only Mullvad key, loaded in
|
||||
# the install-test job of .github/workflows/validate-charts.yml. A CI-only key
|
||||
# the install-test job of .github/workflows/install-test.yml. A CI-only key
|
||||
# on purpose: Mullvad ties an address to a key and sends the tunnel to the
|
||||
# endpoint it saw last, so a shared key would let every pull request pull the
|
||||
# tunnel out from under the running stack.
|
||||
|
||||
+10
-7
@@ -21,9 +21,11 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
CI_DIR="$ROOT/ci"
|
||||
|
||||
# --- The exclusion list, parsed in exactly one place ------------------------
|
||||
# Two consumers read ci/excluded-charts.txt: this script, and the install-test
|
||||
# matrix in .github/workflows/validate-charts.yml, which drops the excluded
|
||||
# charts before the matrix is built (issue #231). Two parsers would be two
|
||||
# The consumers of ci/excluded-charts.txt are this script and the two jobs that
|
||||
# build an install-test matrix - detect-changed-charts in
|
||||
# .github/workflows/validate-charts.yml and select-charts in
|
||||
# .github/workflows/install-test-all-charts.yml - both of which drop the
|
||||
# excluded charts before the matrix exists (issue #231). Two parsers would be two
|
||||
# chances to select different charts - and the failure would be silent in the
|
||||
# worst direction, a chart the workflow believes excluded and the script
|
||||
# believes testable, or the reverse. So there is one parser, here, and the
|
||||
@@ -47,9 +49,10 @@ excluded_reason() {
|
||||
list_excluded | awk -F'\t' -v chart="$1" '$1 == chart { print $2; exit }'
|
||||
}
|
||||
|
||||
# Emits "<chart><TAB><reason>" for every excluded chart. Read by the detect job
|
||||
# of validate-charts.yml before any chart is picked, which is why this runs
|
||||
# before the argument is required.
|
||||
# Emits "<chart><TAB><reason>" for every excluded chart. Read by both jobs that
|
||||
# assemble an install-test matrix (detect-changed-charts in validate-charts.yml,
|
||||
# select-charts in install-test-all-charts.yml) before any chart is picked,
|
||||
# which is why this runs before the argument is required.
|
||||
if [ "${1:-}" = "--list-excluded" ]; then
|
||||
list_excluded
|
||||
exit 0
|
||||
@@ -100,7 +103,7 @@ fi
|
||||
# REQUIRED_ENV="FREESAILARR_WG_KEY FREESAILARR_WG_ADDRESSES"
|
||||
#
|
||||
# The values come from the environment - in CI from 1Password Connect, see the
|
||||
# install-test job in .github/workflows/validate-charts.yml.
|
||||
# install-test job in .github/workflows/install-test.yml.
|
||||
#
|
||||
# A missing or empty value is a hard error on purpose. Skipping the chart
|
||||
# instead would restore exactly the green-but-meaningless signal this mechanism
|
||||
|
||||
Reference in new issue
Block a user