Two assumptions about the runner environment, both found by the Gitea
probe of #225 and both wrong on GitHub as well - they simply could not
show there.
CLUSTER NAME. Every leg created a cluster called "install-test". On
GitHub each matrix leg gets its own VM with its own Docker daemon, so
two legs never saw each other and the fixed name looked fine for two
years. Gitea runs all legs of a matrix CONCURRENTLY in ONE job on ONE
daemon, and there it collides immediately: one leg logs "Re-using
existing network 'k3d-install-test'", the other dies on "The container
name /k3d-install-test-server-0 is already in use", and then the first
leg's rollback deletes the shared network under the second. Two red
legs, and neither message names the cause. The cluster is now named
after the chart, in the create step and in the delete step - a blanket
cleanup would tear down the neighbour's cluster, which was the half of
the collision that turned one failure into two. ci/run-install-test.sh
never touches k3d, so nothing else had to move.
The bare chart name rather than a prefixed one, measured not assumed:
k3d v5.9.0 accepts a cluster name of 32 characters and rejects 33
("Cluster name must be <= 32 characters"), and the longest chart here is
calibre-web-automated at 21. "install-test-<chart>" would be 34 - it
would break that one chart and no other, which is worse than the fixed
name because it only sometimes breaks. A guard states the limit instead
of leaving it to be rediscovered.
KUBECTL. It was inherited from the runner image. The Gitea image has
none ("kubectl: command not found", 0 seconds into the chart install),
but that is how it was noticed, not why it is being changed: a client
version that arrives with the image drifts with every image rebuild, and
an install-test should run against a known one. GitHub's image carrying
one only makes the drift invisible there. Pinned to v1.35.5 to match the
K3S_IMAGE it talks to, tracked by Renovate the same way TRIVY_VERSION
is, and guarded against skew: kubectl supports one minor away from the
server, Renovate moves KUBECTL_VERSION on its own, and K3S_IMAGE is
tracked by nothing. Dry run proves both directions - the rule extracts
(kubernetes/kubernetes, github-releases) and it fires (from v1.34.0 the
lookup offers v1.37.0, minor). That same measurement shows the guard is
not theoretical: Renovate jumps several minors in one PR, so it will go
red on purpose, naming which pin to move.
Groundwork for #225, built on a throwaway pull request against the Gitea
instance rather than on assumptions. The move itself is not part of this
change - no Actions are switched off, no mirror is enabled, no branch
protection is touched.
What the probe run measured, and what follows from it:
- Which jobs need a Docker socket: only the install-test. Measured per
job - detect-changed-charts, build-chart, the quantity render and the
Trivy config scan all pass on `ubuntu-latest`, which on that instance
has no socket. The install-test dies there at "Create k3d cluster"
with "Cannot connect to the Docker daemon", an error naming k3d and
never the runner label. Its `runs-on` is now an expression: the label
`ubuntu-docker` on Gitea, `ubuntu-latest` on GitHub, because the
former does not exist there and a chart PR would otherwise queue
forever during the handover. Gitea evaluates it - the probe job was
taken by the docker runner.
- The install-test still cannot pass on Gitea. That daemon is rootless
with "Cgroup Driver: none"; the k3s node restarts immediately with an
empty log because k3s needs cgroup delegation. The label is necessary
and not sufficient, and the fix is the maintainer's.
- aquasecurity/setup-trivy cannot be used on Gitea: it checks out
aquasecurity/trivy from github.com with `github.token`, which there is
a Gitea token, and ends on "could not read Username for
https://github.com". Replaced by the release tarball - one request,
same behaviour on both forges, green on the probe run.
Also in this change:
- .github/workflows/renovate.yml, Renovate in self-operation: a daily
scheduled run in the official container, inert on github.com via a
`github.server_url` guard. Every rule proved by dry run instead of
assumed - the pinned Renovate image is tracked by the built-in
github-actions manager (44.80.0 -> 44.87.1, minor), TRIVY_VERSION is
still tracked after setup-trivy is gone (v0.72.0 -> v0.74.0, minor),
and `ubuntu-docker` is not extracted by the github-runners manager at
all, so nothing will quietly "correct" it.
- ci/migrate-issues-to-gitea.sh, runnable, with --dry-run, and NOT run.
Its header names what it cannot do: comment authorship, the #<n>
references between issues, and attachments.
- CLAUDE.md and README.md on the new flow: which forge and which tool,
auto-merge on green checks in place of the merge queue Gitea does not
have, and the measured differences of the two CIs.
Refs #225 - deliberately no "Fixes": the move is not finished with this.
A chart in ci/excluded-charts.txt got an install-test job that ran the script, printed SKIP and exited 0. GitHub knows only the exit code, so the check list showed "Install-test satisfactory OK" for a chart whose image was never pulled - indistinguishable from one that really came up in the cluster (#231). detect-changed-charts now removes those charts before the matrix is built, so no job exists for them and nothing claims they passed.
An absence says nothing by itself, so the detect job writes every excluded chart of the PR with its documented justification into the job summary. A "Skipped" would only say "not run"; the reason is the question a review actually asks.
The list is not parsed in YAML. The workflow calls ci/run-install-test.sh --list-excluded, the one parser, and compares whole strings with jq, so a chart name is never read as a pattern. The script's own exclusion branch now goes through the same excluded_reason() helper instead of a grep that used the chart name as a regex; it stays for local runs, where the CI filter does not apply.
Not viable, and recorded at the matrix as a comment: deciding per chart in an "if" on the install-test job, which would give a visible Skipped entry instead of an absent one. GitHub rejects it - jobs.<id>.if is evaluated before the matrix is expanded, so the matrix context does not exist there ("context \"matrix\" is not allowed here. available contexts are \"github\", \"inputs\", \"needs\", \"vars\"."). Run 33650724109 on the closed PR #233 ended in failure with zero jobs created.
Deploys matszwe02/ytdlp_web_player, a Flask app that fetches videos with
yt-dlp, transcodes them with ffmpeg and serves them as HLS. It ships no
authentication of any kind, so the chart carries a preconfigured oauth2-proxy
as an optional subchart, in the data path in front of the app.
This is the first chart in the repo with a dependency, so the rule "charts
have no dependencies" falls here. It is replaced in CLAUDE.md and README.md by
a bar: a dependency has to be optional and off by default, upstream's chart is
consumed as it comes (documented where it falls short of our conventions,
never patched), and the price is written down - above all that a subchart is
invisible to everything downstream watching Helm release versions, so a new
proxy release only reaches a cluster when this chart is republished.
CI learns the dependency in five places, not the three the plan expected:
helm lint, package, template and install all exit 1 while charts/ is unfilled,
so ci/helm-dependencies.sh runs before lint and package in validate-charts.yml,
before package in publish-helm-chart.yml, and inside run-install-test.sh,
run-trivy-config.sh and run-quantity-render-test.sh - the last of which renders
every chart on every PR and would have gone red first.
Notable chart decisions, each measured rather than assumed:
- The image has no USER and runs as root by default, but not by necessity: it
runs under an arbitrary uid with readOnlyRootFilesystem and the full
hardening baseline. Only the pinned uid is a deviation, and it is documented.
- Two emptyDirs, each justified separately. /data holds the video cache
(DATA_PATH). /tmp is NOT mounted - measured across metadata extraction, HLS
transcoding, sprite generation and audio extraction, the image never writes
there. What it does write is $HOME/.cache, so /cache is mounted with HOME and
XDG_CACHE_HOME pointed at it; without it yt-dlp re-downloads its JavaScript
challenge-solver from github.com on every extraction.
- The proxy is wired through config.configFile, the only value the subchart
passes through tpl - config.upstreams is emitted with toJson and would carry
a literal {{ .Release.Name }} into the manifest.
- Rendering aborts when this chart's Ingress and the proxy's claim the same
host: ingress-nginx rejects the second one and a Fleet bundle then sits in
ErrApplied until forceSyncGeneration is bumped.
Fixes#214
Adds ci/run-quantity-render-test.sh and a validate-charts job that runs it
over every chart on every PR.
The bug behind #157 has a loud and a silent half. Loud: a bare integer request
of 1048576 is reformatted by printf "%v" to "1.048576e+06", the helper rejects
the suffix it produced itself and the render aborts. Silent: with a fractional
limitFactor the *product* is reformatted after the suffix check, the render
succeeds, and the manifest carries memory: "1.4999985e+06" - rejected by
nothing until the API server sees it at apply time.
So the test renders each chart twice and, for the silent case, searches the
rendered text rather than the exit code. It runs over all charts, not the
changed ones: the resources helper is copied into every chart's _helpers.tpl,
so the regression arrives by copy, in a PR whose diff shows a new chart and not
the bug inside it. The resources paths come from values.schema.json (JSON, so
python3 reads it without pyyaml), which also means a new chart is enrolled
automatically and there is no list to maintain.
Charts that pin every computable limit in their fallback defaults never reach
the helper (gotenberg, rss-bridge). They are reported as "not exercised"
instead of counted as passing.
freesailarr was excluded from the install-test because gluetun never becomes
healthy without a real Wireguard key and no app container behind its kill
switch can become Ready. The green "Install-test freesailarr pass" was a
25-second SKIP on the most complex chart in the repo - a signal that proved
nothing.
With a CI-only Mullvad key in 1Password the reason is gone:
- ci/run-install-test.sh learns a general rule: ci/<chart>/fixtures.env.yaml is
rendered with envsubst before it is applied, restricted to the variable names
a chart declares in REQUIRED_ENV. A missing or empty declared variable fails
the run loudly and names the variable - it never skips the chart.
- The install-test job loads the key from 1Password Connect for freesailarr
only, following the pattern publish-helm-chart.yml documents: values via
steps.<id>.outputs.*, mapped into the step's own env, quoted, with ":?"
guards that name the missing op:// field before a cluster is created.
- Fixtures for freesailarr: three media claims, the gluetun secret template
with variable names only, firewallOutboundSubnets so the kill switch does not
block cluster DNS, and INSTALL_TIMEOUT=900 for nine containers.
freesailarr/Chart.yaml goes 3.0.0+up1.0.0 -> 3.0.1+up1.0.0. That bump is
deliberately empty of content: the install-test matrix follows changed chart
directories, and without it this PR could not run the test it adds.
The two open Renovate PRs (#138 alpine 3.22 -> 3.24, #139 nginx 1.30.4 ->
1.31.4) never automerged because both are minor updates while the rule
only covered patch and digest. Sidecar images barely move in patch steps,
so patch-only would practically never fire.
Automerging a chart update publishes it, so the install-test can no
longer report failures without blocking: continue-on-error is gone and
issue #22's stabilization phase ends here. Majors stay manual.
Renovate only ever rewrites appVersion or an image tag, so every PR it
opened against a chart directory failed the build-chart version check
(#138, #139). ci/renovate-bump-chart.sh recomputes
version: <semver>+up<appVersion> as a postUpgradeTasks hook: chart major
when the app major moves, chart patch otherwise.
Each chart renders its main image as <repo>:{{ .Chart.AppVersion }}, so
repository and version live in different files and Renovate cannot join
them. The mapping is one customManager per chart in renovate.json5,
deliberately not a comment inside each Chart.yaml - that would touch all
20 chart directories and force 20 publishes for a comment change.
Patch and digest updates automerge after a 3-day minimumReleaseAge once
the pipeline is green; majors arrive one line at a time to match the
stepwise app-major procedure. ci/** stays excluded: the install-test
fixtures pin throwaway infrastructure to major lines on purpose.
Adds a trivy-config job to the PR pipeline that renders each changed chart
with Helm and runs `trivy config` over the rendered manifests. Reporting
only - TRIVY_CONFIG_MODE=report always exits 0, so the job never blocks a PR.
Rendered manifests rather than the chart directory: `trivy config <chart-dir>`
renders with default values, which fails on every chart here because of their
required() values - and Trivy does not fail with it, it logs "Skipping chart"
and prints an empty, green report.
Values come from the install-test fixtures (ci/<chart>/test-values.yaml), the
only committed per-chart values in the repo; none of them touches
securityContext, images or capabilities, so the scan sees the chart's own
hardening. Charts without fixtures get ci/trivy/values/<chart>.yaml.
Exceptions live in ci/trivy/trivyignore.yaml in Trivy's native ignore-file
schema, each with a statement - the script refuses to run without one and
reports suppressed findings with their reason instead of hiding them.
AVD-KSV-0125 is configured instead of excepted: ci/trivy/data/ksv0125.yaml
lists the trusted registries, so the check stays able to flag a genuinely
foreign source.
Refs #84.
New install-test job in validate-charts.yml: for exactly the charts a PR
changes, install them into an ephemeral k3d cluster (pinned k3d v5.9.0,
k3s v1.35.5-k3s1) and require every Deployment/StatefulSet to become
Ready. Runs non-blocking (continue-on-error) during the stabilization
phase.
Fixtures live at the repo root under ci/ (not inside chart directories,
so they neither trigger publishes on merge nor get packaged): a stub
OnePasswordItem CRD, per-chart test values and fixtures (dummy secrets
with in-cluster DSNs, throwaway postgres/redis/dex, synapse-generate
seed job, static hostPath PVs where charts pin storage classes or need
RWX), plus a shared ci/run-install-test.sh harness that also runs
locally.
Excluded with documented reasons (ci/excluded-charts.txt): satisfactory
(multi-GB image), teamspeak-server (TS6 beta license expired, image
crashes on start), template-chart (placeholder image).
All 14 non-excluded charts verified locally in k3d using only the
committed fixtures.
On every pull request, detect which chart directories were touched and
run helm lint --strict, helm package, and a chart-version bump check
for exactly those charts. PRs without chart changes skip the build.
Fixes#5
jklein
jlab-renovate[bot]