rss-bridge, CI-CD, Lessons learned: the green install-test reproduced, so "twice" rather than "one run"
The previous wiki commit recorded rss-bridge's install-test going green after months of failing on the Gitea runner, and hedged it with "it is one run". A second run on a second commit of the same PR came back green too - run 792 job 2623 and run 794 job 2638, both with the chart genuinely in the matrix, a k3d cluster built and the pod Ready. So the result is not a fluke, and the real caution is a different one: nobody has identified what changed on the runner. A green test does not explain itself, and the diagnostic programme in Gitea #51's body - docker info on the runner for userns and the dind configuration, the same image in a local k3d without remapping - remains unexecuted. That, and not the run count, is why the issue stays open. "Close an issue on a cause, not on a colour" is unaffected by the count.
3 files changed
+3
-3
No files matched your search
+1
-1
@@ -166,7 +166,7 @@ Group 3 took it to `56` assertions over `21` charts. Its 19 come from six charts
|
||||
|
||||
**The realm hostnames of Gitea #122 add `4`**, and they are objects rather than a new check: `keycloak`'s share goes from 4 to 8 — one `parentRefs` and one `backendRefs` assertion for each of its two realm routes — plus one "not checked" line each for their backend-less redirect rule (`keycloak: ok (8 assertion(s) held, 2 not checked)`). The #118 prefix/redirect pairing deliberately does **not** fire on them: it is scoped to rules that strip a prefix with a `ReplacePrefixMatch` rewrite, and a realm route strips nothing — it redirects the bare root of a hostname that serves one realm. **The two rounds were measured separately and neither total includes the other**: `58/22` was read on the `rss-bridge` branch and `60/21` on the realm-hostname branch, which did not contain `rss-bridge`. So the **delta** is the figure to trust from either, and the running total is whatever the first run on `main` prints once both are in — read it there rather than adding these up, which is the same rule that produced both numbers.
|
||||
|
||||
**That chart's install-test also went green in the same PR**, after failing on this runner since the move — the event #51 had designated as its own signal, and the reason the chart was deliberately never added to `ci/excluded-charts.txt`. Pod `1/1 Ready` in 22 s, 0 restarts, no `Permission denied` in the job. The chart is ruled out as the cause by measurement rather than by argument: that PR's rendered defaults are **byte-identical** to `origin/main`, so the pod spec that installed is the one that had been failing. The runner is the implicated layer. It is **one** run, though, and nobody has identified what changed there — so the issue is not closed on the strength of a colour, and a fresh red would be the control case this result is missing. See [rss-bridge](rss-bridge).
|
||||
**That chart's install-test also went green in the same PR**, after failing on this runner since the move — the event #51 had designated as its own signal, and the reason the chart was deliberately never added to `ci/excluded-charts.txt`. Pod `1/1 Ready` in 22 s, 0 restarts, no `Permission denied` in the job. The chart is ruled out as the cause by measurement rather than by argument: that PR's rendered defaults are **byte-identical** to `origin/main`, so the pod spec that installed is the one that had been failing. The runner is the implicated layer. It reproduced over two consecutive runs on two commits of that PR, so it is not a single fluke — but nobody has identified what changed on the runner, so the issue is not closed on the strength of a colour. See [rss-bridge](rss-bridge).
|
||||
|
||||
The fail path was exercised rather than assumed: a Service renamed in the route, a port the Service does not expose, a port left out entirely, an emptied `parentRefs` with the chart's own guard disabled, an overlay that renders no route, a run with zero assertions, and — for the pairing — both the redirect rule deleted outright and the redirect pointed at a path other than the one stripped; each produced a `FAIL` naming both sides and a non-zero exit.
|
||||
|
||||
|
||||
+1
-1
@@ -720,7 +720,7 @@ So the chart stayed in the matrix and the check stayed red, explicitly as the st
|
||||
- **An excluded test is not a quiet test, it is a blind one.** The cost of an exclusion is usually discussed as "we lose coverage while the bug exists". The larger cost is that you also lose the moment the bug stops existing — and nobody schedules a day to go and re-check a skipped test.
|
||||
- **Keep a known failure attached to the thing it is about.** Red-and-explained is a worse pipeline and a better instrument than green-and-skipped. That trade is only correct when the explanation is written down where the red will be seen, which is why the chart's page and the issue both carried it.
|
||||
- **A green that arrives on an unchanged input is worth more than a green that arrives with a fix**, because it localises the cause by elimination. Here the byte-identical render was produced for an unrelated reason — version-bump justification — and turned out to be the control case. Cheap evidence collected as a matter of routine is what makes an accident interpretable.
|
||||
- **One run is not a repair.** The colour changed; nobody has yet identified what changed on the runner. An intermittent effect needs several runs per side, so the issue stays open and a fresh red would be the missing control rather than a surprise. **Close an issue on a cause, not on a colour** — which was the standing instruction on this one before the green arrived, written precisely so that the green could not be mistaken for the answer.
|
||||
- **A reproduced colour is still not a repair.** It went green twice, on two consecutive runs of two different commits, so it is not a fluke — and nobody has identified what changed on the runner. A green test does not explain itself, and the diagnostic programme that would (`docker info` on the runner for `userns` and the dind configuration, the same image in a local k3d without remapping) remains unexecuted. So the issue stays open, and a fresh red would be a finding rather than a surprise. **Close an issue on a cause, not on a colour** — which was the standing instruction on this one before the green arrived, written precisely so that the green could not be mistaken for the answer.
|
||||
|
||||
## The precise message that hid the structural one
|
||||
|
||||
|
||||
+1
-1
@@ -276,7 +276,7 @@ Since the install-test installs the Ingress mode, the overlay is the only automa
|
||||
|
||||
Two cautions belong with the result, and they are the reason #51 was not simply closed:
|
||||
|
||||
- **It is one run.** An intermittent effect needs several runs per side before either answer is honest, and nobody has yet identified *what* changed on the runner.
|
||||
- **Nobody has identified what changed on the runner.** It reproduced over two consecutive runs on two different commits of the same PR, so it is not a single fluke — but a green test does not explain itself, and the check programme in #51's body (`docker info` on the runner for `userns` and the dind configuration, the same image in a local k3d without remapping) is still unexecuted.
|
||||
- **The colour is not the finding.** The standing instruction on #51 is that whoever sees this test green closes the issue *after checking the cause*, not because of the colour.
|
||||
|
||||
The history below is kept because it is the evidence that localised the defect, and because a recurrence would be read against it.
|
||||
|
||||
Reference in new issue
Block a user