23 Commits

Include renames

Author	SHA1	Message	Date
jklein	b614b53602	ci: remove the temporary repository_owner probe, record the measurement ... github.repository_owner is populated on this Gitea instance: measured as "jlab-cluster" (12 characters) in run 282 of validate-charts.yml. The value is now in the comment above the upload step, so the fact survives without the step that produced it.	2026-09-30 18:51:55 +00:00
jklein	a12983fbea	ci(publish): compose the registry URL from a base and the repository owner ... charts-url in op://GitHub-Actions/jlab-registry-access held the full registry URL and publish-helm-chart.yml appended /api/charts to it. It now holds only the base up to /api/packages, and the owner segment comes from github.repository_owner, so the target is composed as <base>/<owner>/helm/api/charts. Nothing in this repository points at the registry any more, not even a path segment, and the second owner on the same instance needs no second setting. The 26 thin publish-<chart>.yml callers are unchanged. Two guards keep the switch from the old shape loud rather than a 404 against a masked URL: charts-url is rejected unless it ends in /api/packages, naming /helm, /api/charts or an included owner explicitly; and the derived owner is rejected when empty or when it is not a usable URL path segment. Everything the upload step already had is untouched - the per-field :? guards, the whitespace check on the credentials, the host masking, the built file name, --fail-with-body. The probe step in validate-charts.yml is temporary and measures whether this forge populates github.repository_owner at all; it is removed again once the value has been read off the run.	2026-09-30 18:49:56 +00:00
jklein	6a3b05503e	Make the all-charts install-test repeatable, as one shared job ... The install-test matrix follows the charts a pull request changes, so a chart nobody has touched since the forge move carries a green history from a different runner - and that green says nothing about today. Gitea PR #48 showed what the answer is worth: correcting a comment in 21 charts made every one of them "changed" and turned the run into the first all-charts install-test after the move, 19 of 20 matrix charts Ready in 12m41s and exactly one not (rss-bridge, Gitea #51). By accident. The install-test job moves into the reusable .github/workflows/ install-test.yml and is called twice: by validate-charts.yml with the charts a pull request changed, and by the new install-test-all-charts.yml with all of them. Not copied for the second caller - a second copy is a second chance to test differently, unnoticed, which is the argument that gives ci/excluded-charts.txt exactly one parser. The parts that would drift are the ones that were paid for once already: the pinned kubectl with its skew check, envsubst, the containerised-job detection, the per-leg cluster name. Nothing about how a chart is installed changed. The moved job differs from its predecessor in the matrix source (inputs.charts instead of the detect job's output), an id on the install step, and a step that records the result; every other step is byte-identical. The all-charts run is workflow_dispatch only - a schedule would spend compute every day on a question that changes rarely - with an optional subset input, which is what lets a single chart be tested against the runner as Gitea #51 needs. One throwaway cluster per chart, no reuse: reuse saves time and changes what the test proves. An unknown or excluded chart name, and an empty matrix, are loud errors rather than a green run over nothing. Every leg writes its own outcome into the job summary next to the four facts that decide what the outcome means - date, runner label, K3S_IMAGE, kubectl version - and a final job records the verdict over the matrix. A green run nobody notes is unknown again next time. The excluded charts appear with their justification, from the one parser, as in the pull request pipeline. 20 charts in 12m41s is written down as the expectation, so a later deviation is visible as one. Fixes #52	2026-09-27 14:30:39 +00:00
jklein	e0b552d93e	Replace the obsolete no-dependencies rationale in every chart helper ... The doc comments on the resources and defaultedDict helpers justified the copy-per-chart with "charts in this repo deliberately have no dependencies". That rule fell with #214: a dependency is admissible now, just expensive, and it sits in the README as guideline 14, not 13. The justification still holds, but its reason is a different one - sharing the helper would mean a library chart, a second artefact to publish, version and pull for something that changes about once a year. searxng got that wording in PR #38; this carries it to 21 further charts, 40 occurrences in all. rss-bridge is deliberately left out. Its install-test fails on the Gitea runner for a reason that predates this branch, and touching the chart would keep it in the install-test matrix and the check red. Its copy of the sentence is carried by #51, which has to open the chart anyway. Comments only: every touched chart renders byte-identical against its ci/<chart>/test-values.yaml. Each published chart takes a patch bump, appVersion and +up suffix untouched; template-chart is never published and takes none, but is corrected because every new chart inherits from it. Two repo files carried the same claim at no version cost: validate-charts.yml explained the all-charts quantity render test with it, and change-app-version.yml still counted nine images for freesailarr - MediathekArr made it ten.	2026-09-27 10:27:57 +00:00
jklein	fdd798e1d9	install-test: provide envsubst, and say why it is not pinned like kubectl ... ci/run-install-test.sh renders ci/<chart>/fixtures.env.yaml with envsubst. GitHub's hosted image ships gettext-base, the Gitea runner image does not, and without it the fixtures collapse to 'error: no objects passed to apply' - gluetun never gets its Wireguard key and nothing behind its kill switch becomes Ready. The script is left untouched; it does nothing wrong, a standard tool is simply absent. Checked for rather than pinned, and the difference from kubectl is written next to it: kubectl's version changes what the test MEANS, because its distance to the API server is a real source of failure, so it is pinned, tracked and guarded. envsubst substitutes text and the release performing it does not change the result, so a pin would be upkeep without value. Present means do nothing, which keeps the GitHub path free of any package install; missing means fetch it, so the difference between the two runners is stated instead of hidden behind an unconditional apt-get. Also rewrites the reachability comment into one block. It had grown into two stacked halves whose first half still described the behaviour of the version before last - both reasons (the loop against a discovery 503, counting Ready nodes against an empty list that exits 0) now stand together with the evidence that the second one is what made the inotify failure visible at all.	2026-09-17 01:00:38 +02:00
jklein	417dff0f80	install-test: wait for a Ready node, not for an answer ... Second defect in the same check, and the more serious one: kubectl get nodes exits 0 on an EMPTY list. The cluster on Gitea answered while its node had not registered, the step logged 'No resources found' and went green, and the failure surfaced five minutes later as a pod stuck in Pending with no node and no scheduling event - the cause nowhere near the symptom. kubectl wait --for=condition=Ready nodes --all has the same hole; with zero nodes it succeeds vacuously. It now counts Ready nodes, and on timeout dumps the node conditions and the k3s server log so the error picture is in the job that caused it.	2026-09-17 00:46:54 +02:00
jklein	4766126618	install-test: poll for the cluster instead of asking once ... The reachability check added with the containerisation fix was itself racy and went red on one of two legs of the same GitHub run: k3d's --wait returns when the node container reports ready, but the API server can still answer discovery with 503 for a moment after that, and kubectl cluster-info asked exactly once. A check that produces sporadic red is worse than no check - that is the one property #227 rules out explicitly. It now polls kubectl get nodes, which is the thing that has to work and exercises discovery on the way.	2026-09-17 00:38:41 +02:00
jklein	31e4260580	install-test: reach the k3d cluster when the job itself is a container ... Closes the last gap from #227. k3d publishes the API server port on the Docker HOST. On GitHub's hosted runners the job is that host, so the https://0.0.0.0:<port> k3d writes into the kubeconfig works. Every Gitea Actions job is itself a container, and there the host is the dind container: 0.0.0.0 points at the job container, where nothing listens. The cluster comes up cleanly and the first kubectl call is refused - as measured, surfacing two steps later as a validation error on the fixtures, which named nothing useful. The case is decided on whether THIS job runs in a container, not on which forge it runs on. github.server_url would pick the right branch today and still be wrong: a self-hosted GitHub runner in a container has the identical problem, and a condition that says GitHub while meaning not-containerised is the kind of lie this repo exists to remove. The detection is the same question the fix needs answered anyway - ask the daemon about ourselves. In a container it returns our network, k3d joins it, and Docker's DNS resolves the node names; on a host it returns nothing and the previous path is taken unchanged. kubectl cluster-info and get nodes now run inside the create step, so a cluster that is up but unreachable fails where it is caused instead of two steps downstream.	2026-09-17 00:29:08 +02:00
jklein	301fc0d4e5	install-test: one cluster per matrix leg, and a pinned kubectl ... Two assumptions about the runner environment, both found by the Gitea probe of #225 and both wrong on GitHub as well - they simply could not show there. CLUSTER NAME. Every leg created a cluster called "install-test". On GitHub each matrix leg gets its own VM with its own Docker daemon, so two legs never saw each other and the fixed name looked fine for two years. Gitea runs all legs of a matrix CONCURRENTLY in ONE job on ONE daemon, and there it collides immediately: one leg logs "Re-using existing network 'k3d-install-test'", the other dies on "The container name /k3d-install-test-server-0 is already in use", and then the first leg's rollback deletes the shared network under the second. Two red legs, and neither message names the cause. The cluster is now named after the chart, in the create step and in the delete step - a blanket cleanup would tear down the neighbour's cluster, which was the half of the collision that turned one failure into two. ci/run-install-test.sh never touches k3d, so nothing else had to move. The bare chart name rather than a prefixed one, measured not assumed: k3d v5.9.0 accepts a cluster name of 32 characters and rejects 33 ("Cluster name must be <= 32 characters"), and the longest chart here is calibre-web-automated at 21. "install-test-<chart>" would be 34 - it would break that one chart and no other, which is worse than the fixed name because it only sometimes breaks. A guard states the limit instead of leaving it to be rediscovered. KUBECTL. It was inherited from the runner image. The Gitea image has none ("kubectl: command not found", 0 seconds into the chart install), but that is how it was noticed, not why it is being changed: a client version that arrives with the image drifts with every image rebuild, and an install-test should run against a known one. GitHub's image carrying one only makes the drift invisible there. Pinned to v1.35.5 to match the K3S_IMAGE it talks to, tracked by Renovate the same way TRIVY_VERSION is, and guarded against skew: kubectl supports one minor away from the server, Renovate moves KUBECTL_VERSION on its own, and K3S_IMAGE is tracked by nothing. Dry run proves both directions - the rule extracts (kubernetes/kubernetes, github-releases) and it fires (from v1.34.0 the lookup offers v1.37.0, minor). That same measurement shows the guard is not theoretical: Renovate jumps several minors in one PR, so it will go red on purpose, naming which pin to move.	2026-09-17 00:22:16 +02:00
jklein	a6c27ebef3	Prepare the Gitea move: measured CI deviations, self-hosted Renovate, issue migration script ... Groundwork for #225, built on a throwaway pull request against the Gitea instance rather than on assumptions. The move itself is not part of this change - no Actions are switched off, no mirror is enabled, no branch protection is touched. What the probe run measured, and what follows from it: - Which jobs need a Docker socket: only the install-test. Measured per job - detect-changed-charts, build-chart, the quantity render and the Trivy config scan all pass on `ubuntu-latest`, which on that instance has no socket. The install-test dies there at "Create k3d cluster" with "Cannot connect to the Docker daemon", an error naming k3d and never the runner label. Its `runs-on` is now an expression: the label `ubuntu-docker` on Gitea, `ubuntu-latest` on GitHub, because the former does not exist there and a chart PR would otherwise queue forever during the handover. Gitea evaluates it - the probe job was taken by the docker runner. - The install-test still cannot pass on Gitea. That daemon is rootless with "Cgroup Driver: none"; the k3s node restarts immediately with an empty log because k3s needs cgroup delegation. The label is necessary and not sufficient, and the fix is the maintainer's. - aquasecurity/setup-trivy cannot be used on Gitea: it checks out aquasecurity/trivy from github.com with `github.token`, which there is a Gitea token, and ends on "could not read Username for https://github.com". Replaced by the release tarball - one request, same behaviour on both forges, green on the probe run. Also in this change: - .github/workflows/renovate.yml, Renovate in self-operation: a daily scheduled run in the official container, inert on github.com via a `github.server_url` guard. Every rule proved by dry run instead of assumed - the pinned Renovate image is tracked by the built-in github-actions manager (44.80.0 -> 44.87.1, minor), TRIVY_VERSION is still tracked after setup-trivy is gone (v0.72.0 -> v0.74.0, minor), and `ubuntu-docker` is not extracted by the github-runners manager at all, so nothing will quietly "correct" it. - ci/migrate-issues-to-gitea.sh, runnable, with --dry-run, and NOT run. Its header names what it cannot do: comment authorship, the #<n> references between issues, and attachments. - CLAUDE.md and README.md on the new flow: which forge and which tool, auto-merge on green checks in place of the merge queue Gitea does not have, and the measured differences of the two CIs. Refs #225 - deliberately no "Fixes": the move is not finished with this.	2026-09-14 17:07:31 +02:00
jklein	622e4c3ff8	ci: drop excluded charts from the install-test matrix ... A chart in ci/excluded-charts.txt got an install-test job that ran the script, printed SKIP and exited 0. GitHub knows only the exit code, so the check list showed "Install-test satisfactory OK" for a chart whose image was never pulled - indistinguishable from one that really came up in the cluster (#231). detect-changed-charts now removes those charts before the matrix is built, so no job exists for them and nothing claims they passed. An absence says nothing by itself, so the detect job writes every excluded chart of the PR with its documented justification into the job summary. A "Skipped" would only say "not run"; the reason is the question a review actually asks. The list is not parsed in YAML. The workflow calls ci/run-install-test.sh --list-excluded, the one parser, and compares whole strings with jq, so a chart name is never read as a pattern. The script's own exclusion branch now goes through the same excluded_reason() helper instead of a grep that used the chart name as a regex; it stays for local runs, where the CI filter does not apply. Not viable, and recorded at the matrix as a comment: deciding per chart in an "if" on the install-test job, which would give a visible Skipped entry instead of an absent one. GitHub rejects it - jobs.<id>.if is evaluated before the matrix is expanded, so the matrix context does not exist there ("context \"matrix\" is not allowed here. available contexts are \"github\", \"inputs\", \"needs\", \"vars\"."). Run 33650724109 on the closed PR #233 ended in failure with zero jobs created.	2026-09-04 16:58:14 +02:00
jklein	6b0d8c9232	ytdlp-web-player: new chart with an optional oauth2-proxy subchart (1.0.0+upv1.2.0) ... Deploys matszwe02/ytdlp_web_player, a Flask app that fetches videos with yt-dlp, transcodes them with ffmpeg and serves them as HLS. It ships no authentication of any kind, so the chart carries a preconfigured oauth2-proxy as an optional subchart, in the data path in front of the app. This is the first chart in the repo with a dependency, so the rule "charts have no dependencies" falls here. It is replaced in CLAUDE.md and README.md by a bar: a dependency has to be optional and off by default, upstream's chart is consumed as it comes (documented where it falls short of our conventions, never patched), and the price is written down - above all that a subchart is invisible to everything downstream watching Helm release versions, so a new proxy release only reaches a cluster when this chart is republished. CI learns the dependency in five places, not the three the plan expected: helm lint, package, template and install all exit 1 while charts/ is unfilled, so ci/helm-dependencies.sh runs before lint and package in validate-charts.yml, before package in publish-helm-chart.yml, and inside run-install-test.sh, run-trivy-config.sh and run-quantity-render-test.sh - the last of which renders every chart on every PR and would have gone red first. Notable chart decisions, each measured rather than assumed: - The image has no USER and runs as root by default, but not by necessity: it runs under an arbitrary uid with readOnlyRootFilesystem and the full hardening baseline. Only the pinned uid is a deviation, and it is documented. - Two emptyDirs, each justified separately. /data holds the video cache (DATA_PATH). /tmp is NOT mounted - measured across metadata extraction, HLS transcoding, sprite generation and audio extraction, the image never writes there. What it does write is $HOME/.cache, so /cache is mounted with HOME and XDG_CACHE_HOME pointed at it; without it yt-dlp re-downloads its JavaScript challenge-solver from github.com on every extraction. - The proxy is wired through config.configFile, the only value the subchart passes through tpl - config.upstreams is emitted with toJson and would carry a literal {{ .Release.Name }} into the manifest. - Rendering aborts when this chart's Ingress and the proxy's claim the same host: ingress-nginx rejects the second one and a Fleet bundle then sits in ErrApplied until forceSyncGeneration is bumped. Fixes #214	2026-08-30 17:34:49 +02:00
jklein	148e15a1f6	ci: guard resource quantities against exponential notation ... Adds ci/run-quantity-render-test.sh and a validate-charts job that runs it over every chart on every PR. The bug behind #157 has a loud and a silent half. Loud: a bare integer request of 1048576 is reformatted by printf "%v" to "1.048576e+06", the helper rejects the suffix it produced itself and the render aborts. Silent: with a fractional limitFactor the *product* is reformatted after the suffix check, the render succeeds, and the manifest carries memory: "1.4999985e+06" - rejected by nothing until the API server sees it at apply time. So the test renders each chart twice and, for the silent case, searches the rendered text rather than the exit code. It runs over all charts, not the changed ones: the resources helper is copied into every chart's _helpers.tpl, so the regression arrives by copy, in a PR whose diff shows a new chart and not the bug inside it. The resources paths come from values.schema.json (JSON, so python3 reads it without pyyaml), which also means a new chart is enrolled automatically and there is no list to maintain. Charts that pin every computable limit in their fallback defaults never reach the helper (gotenberg, rss-bridge). They are reported as "not exercised" instead of counted as passing.	2026-08-29 01:25:48 +02:00
jklein	69a280399c	ci: run the freesailarr install-test for real instead of skipping it ... freesailarr was excluded from the install-test because gluetun never becomes healthy without a real Wireguard key and no app container behind its kill switch can become Ready. The green "Install-test freesailarr pass" was a 25-second SKIP on the most complex chart in the repo - a signal that proved nothing. With a CI-only Mullvad key in 1Password the reason is gone: - ci/run-install-test.sh learns a general rule: ci/<chart>/fixtures.env.yaml is rendered with envsubst before it is applied, restricted to the variable names a chart declares in REQUIRED_ENV. A missing or empty declared variable fails the run loudly and names the variable - it never skips the chart. - The install-test job loads the key from 1Password Connect for freesailarr only, following the pattern publish-helm-chart.yml documents: values via steps.<id>.outputs.*, mapped into the step's own env, quoted, with ":?" guards that name the missing op:// field before a cluster is created. - Fixtures for freesailarr: three media claims, the gluetun secret template with variable names only, firewallOutboundSubnets so the kill switch does not block cluster DNS, and INSTALL_TIMEOUT=900 for nine containers. freesailarr/Chart.yaml goes 3.0.0+up1.0.0 -> 3.0.1+up1.0.0. That bump is deliberately empty of content: the install-test matrix follows changed chart directories, and without it this PR could not run the test it adds.	2026-08-26 14:30:42 +02:00
jklein	2e0e37d838	renovate: automerge minor updates, make the install-test blocking ... The two open Renovate PRs (#138 alpine 3.22 -> 3.24, #139 nginx 1.30.4 -> 1.31.4) never automerged because both are minor updates while the rule only covered patch and digest. Sidecar images barely move in patch steps, so patch-only would practically never fire. Automerging a chart update publishes it, so the install-test can no longer report failures without blocking: continue-on-error is gone and issue #22's stabilization phase ends here. Majors stay manual.	2026-08-24 12:01:08 +02:00
jklein	6b25d082a7	renovate: chart-aware config with an automatic version bump ... Renovate only ever rewrites appVersion or an image tag, so every PR it opened against a chart directory failed the build-chart version check (#138, #139). ci/renovate-bump-chart.sh recomputes version: <semver>+up<appVersion> as a postUpgradeTasks hook: chart major when the app major moves, chart patch otherwise. Each chart renders its main image as <repo>:{{ .Chart.AppVersion }}, so repository and version live in different files and Renovate cannot join them. The mapping is one customManager per chart in renovate.json5, deliberately not a comment inside each Chart.yaml - that would touch all 20 chart directories and force 20 publishes for a comment change. Patch and digest updates automerge after a 3-day minimumReleaseAge once the pipeline is green; majors arrive one line at a time to match the stepwise app-major procedure. ci/** stays excluded: the install-test fixtures pin throwaway infrastructure to major lines on purpose.	2026-08-24 11:42:43 +02:00
jklein	a176c7576c	Merge pull request #142 from jk-cluster/renovate/azure-setup-helm-5.x ... Update azure/setup-helm action to v5	2026-08-24 09:20:01 +00:00
jlab-renovate[bot]	5026622763	Update azure/setup-helm action to v5	2026-08-24 00:01:19 +00:00
jlab-renovate[bot]	35d381432c	Update actions/checkout action to v7	2026-08-23 23:01:23 +00:00
jklein	1ae5e1a7c3	CI: report Trivy misconfiguration findings for changed charts ... Adds a trivy-config job to the PR pipeline that renders each changed chart with Helm and runs `trivy config` over the rendered manifests. Reporting only - TRIVY_CONFIG_MODE=report always exits 0, so the job never blocks a PR. Rendered manifests rather than the chart directory: `trivy config <chart-dir>` renders with default values, which fails on every chart here because of their required() values - and Trivy does not fail with it, it logs "Skipping chart" and prints an empty, green report. Values come from the install-test fixtures (ci/<chart>/test-values.yaml), the only committed per-chart values in the repo; none of them touches securityContext, images or capabilities, so the scan sees the chart's own hardening. Charts without fixtures get ci/trivy/values/<chart>.yaml. Exceptions live in ci/trivy/trivyignore.yaml in Trivy's native ignore-file schema, each with a statement - the script refuses to run without one and reports suppressed findings with their reason instead of hiding them. AVD-KSV-0125 is configured instead of excepted: ci/trivy/data/ksv0125.yaml lists the trusted registries, so the check stays able to flag a genuinely foreign source. Refs #84.	2026-08-23 23:14:12 +02:00
jklein	cc6084ca58	Add install-test stage to the PR pipeline (k3d) ... New install-test job in validate-charts.yml: for exactly the charts a PR changes, install them into an ephemeral k3d cluster (pinned k3d v5.9.0, k3s v1.35.5-k3s1) and require every Deployment/StatefulSet to become Ready. Runs non-blocking (continue-on-error) during the stabilization phase. Fixtures live at the repo root under ci/ (not inside chart directories, so they neither trigger publishes on merge nor get packaged): a stub OnePasswordItem CRD, per-chart test values and fixtures (dummy secrets with in-cluster DSNs, throwaway postgres/redis/dex, synapse-generate seed job, static hostPath PVs where charts pin storage classes or need RWX), plus a shared ci/run-install-test.sh harness that also runs locally. Excluded with documented reasons (ci/excluded-charts.txt): satisfactory (multi-GB image), teamspeak-server (TS6 beta license expired, image crashes on start), template-chart (placeholder image). All 14 non-excluded charts verified locally in k3d using only the committed fixtures.	2026-08-21 23:12:38 +02:00
jklein	ded2f59020	Rename detect job output to any-chart-changed for clarity	2026-08-21 17:03:22 +02:00
jklein	79f9260f56	Add PR pipeline validating changed helm charts ... On every pull request, detect which chart directories were touched and run helm lint --strict, helm package, and a chart-version bump check for exactly those charts. PRs without chart changes skip the build. Fixes #5	2026-08-21 16:55:47 +02:00