Merge main into the MediathekArr branch
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 7s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Successful in 4s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 11s
Validate Helm Charts / Install-test ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }} (pull_request) Successful in 3m45s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 7s
Validate Helm Charts / Build ${{ fromJSON(needs.detect-changed-charts.outputs.charts) }} (pull_request) Successful in 4s
Validate Helm Charts / Trivy config ${{ fromJSON(needs.detect-changed-charts.outputs.scan-charts) }} (pull_request) Successful in 11s
Validate Helm Charts / Install-test ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }} (pull_request) Successful in 3m45s
The branch predates the Gitea move (#253) by 37 commits and both sides touched the same two files. Merged rather than rebased, so the signed commit of the original work stays intact, and resolved by combining rather than choosing - both statements are true at once: the chart has ten images now, and the repo lives on Gitea. README.md, the one real conflict: this branch rewrote the freesailarr image bullet from nine upstreams to ten, while main had grown a minecraft-server bullet next to it that this branch never had. Kept both - our "ten" wording and main's new bullet. renovate.json5 merged cleanly, but "cleanly" is not "correctly", so the result was read rather than trusted: the freesailarr exception now says ten and gained the pointer to the mediathekarr digest block, while the two things main added right beside it survived unchanged - the note that the post-upgrade hook is allow-listed on the shared Gitea instance, and the forge-neutral wording of the automerge rule. Nothing from before the move brought "GitHub" back into a sentence that had been made neutral.
This commit is contained in:
12 files changed
+442
-48
No files matched your search
@@ -1,6 +1,8 @@
|
||||
# helm-charts
|
||||
|
||||
Collection of some self-made helm-charts. Each top-level directory is a standalone Helm chart that gets packaged and uploaded to a private, classic (index-based) Helm registry via GitHub Actions.
|
||||
Collection of some self-made helm-charts. Each top-level directory is a standalone Helm chart that gets packaged and uploaded to a private, classic (index-based) Helm registry by CI.
|
||||
|
||||
> **The repository is moving from github.com to the Gitea instance JLab-Git (#225).** The code is already on both and identical; what still runs the show is GitHub. The workflow files under `.github/workflows/` are read by both forges unchanged, and "Where this repo lives" below names the handful of places where the two behave differently — each of them measured on a probe pull request, not assumed.
|
||||
|
||||
## Charts
|
||||
|
||||
@@ -38,7 +40,7 @@ Collection of some self-made helm-charts. Each top-level directory is a standalo
|
||||
- Bump the **chart version** in every change that touches a chart directory — a push to `main` publishes the chart.
|
||||
- **App-version updates go through PRs** (so the validate and install-test pipelines run). The manually dispatched `Change App-Version` workflow (rewrites `version:`/`appVersion:` and commits directly to `main`) exists but bypasses both pipelines. For app **major** jumps, update stepwise: first to the last release of the old major line (own PR, merge, publish), then to the newest major.
|
||||
|
||||
## CI/CD (GitHub Actions)
|
||||
## CI/CD
|
||||
|
||||
- `publish-<chart>.yml` — one thin workflow per chart; triggers on pushes to `main` touching `<chart>/**` (or manually) and calls the reusable workflow below. **A push to a chart directory on `main` publishes that chart**, so the chart version must be bumped in the same change.
|
||||
- `publish-helm-chart.yml` — reusable workflow: `helm package`, then a single HTTP `POST` of the packaged `.tgz` to the private chart registry. The target is a **classic, index-based** Helm registry (Gitea's `/api/charts` endpoint), not an OCI one: `helm push` speaks only OCI and is not used, there is no `helm registry login` session, and every request carries its own Basic auth. Credentials **and the target URL** come from 1Password Connect (`OP_CONNECT_TOKEN` secret, item `op://GitHub-Actions/jlab-registry-access/` with `username`, `password`, `charts-url`); the workflow appends `/api/charts` to `charts-url`. No registry address ever appears in this repository. The loaded values are read via **`steps.<id>.outputs.*`**, mapped into the step's own `env:` block and used as quoted shell variables — never interpolated into the command line. `load-secrets-action` **v5 no longer exports them as environment variables** for later steps the way v4 did, so after the v5 bump every `${{ env.* }}` reference resolved to an empty string while the loading step still logged `Populating variable` and stayed green. Each value is additionally guarded with `:?`, because an empty one used to collapse into a missing argument rather than an error. Three properties of the upload step are load-bearing:
|
||||
@@ -49,7 +51,23 @@ Collection of some self-made helm-charts. Each top-level directory is a standalo
|
||||
Note: the registry **accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact, so a republish of an existing version is still silent — see #160.
|
||||
- `change-app-version.yml` — manually dispatched app-version bump; the chart must be listed in its `chart-name` choice list.
|
||||
- `ci/helm-dependencies.sh <chart-dir>` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one). It runs in **five** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in `publish-helm-chart.yml`, and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs.
|
||||
- `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (same changed-charts matrix) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix. The **resource-quantity render test** is the exception to that rule: it runs over every chart on every PR — see below.
|
||||
- `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (the same changed-charts matrix, minus the charts listed in `ci/excluded-charts.txt`) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix. The **resource-quantity render test** is the exception to that rule: it runs over every chart on every PR — see below.
|
||||
|
||||
### Where this repo lives, and what Gitea Actions does differently
|
||||
|
||||
The move to the Gitea instance JLab-Git (#225) is in progress. Order matters and is not negotiable: **CI provably green on Gitea → GitHub Actions off → mirror on.** Any other order leaves a window in which either no CI exists or two of them publish into the same registry — and that registry accepts a re-upload of an existing version silently, replacing the artifact (#160).
|
||||
|
||||
Everything below was measured on a throwaway pull request against the Gitea instance, not taken from documentation. It is the reason for the two deviations now in `validate-charts.yml`.
|
||||
|
||||
- **Reusable workflows with `secrets: inherit` work.** The 23 thin `publish-<chart>.yml` hulls survive the move unchanged. (A missing secret does not fail the job — Gitea fails at *job creation*, so the run shows no steps at all, which is worth knowing before debugging the wrong thing.)
|
||||
- **`ubuntu-latest` on this instance has no Docker socket; `ubuntu-docker` has.** Same label, different meaning than on GitHub's hosted runners, and only one job in this repo cares: the install-test creates a k3d cluster, and k3d is a Docker client. Measured, per job: `detect-changed-charts`, `build-chart`, the quantity render and the Trivy config scan all pass on `ubuntu-latest` — none of them needs Docker. The install-test on `ubuntu-latest` dies at `Create k3d cluster` with `Cannot connect to the Docker daemon at unix:///var/run/docker.sock`, an error that names k3d and never the runner label.
|
||||
- **`ubuntu-docker` is necessary but not yet sufficient.** Its daemon (8 vCPU, 64 GB, Docker 29.5.2) is **rootless** and reports `Cgroup Driver: none` plus `WARNING: Running in rootless-mode without cgroups`. k3d gets as far as creating the network, pulling the images and starting the node; the k3s server container then restarts immediately with an empty log. k3s needs cgroup delegation. Until the instance's Docker-in-Docker gets it (rootful, or rootless with systemd delegation), **the install-test cannot run on Gitea at all** — see #225.
|
||||
- **`aquasecurity/setup-trivy` cannot be used.** It runs `actions/checkout` against `aquasecurity/trivy` on github.com with `github.token`, which on Gitea is a *Gitea* token; github.com answers 401 and checkout retries twice before ending on `fatal: could not read Username for 'https://github.com'`. Its `actions/cache` step is inert on the instance as well (`Cache action is only supported on GHES version >= 3.5`), and fetching the nested actions cost 4m30s before the job started. The workflow now downloads the release tarball instead — one request, identical behaviour on both forges, and the `# renovate:` comment above `TRIVY_VERSION` still keeps it current (proved by dry run: pinned to `v0.72.0`, the lookup offers `v0.74.0` as a minor).
|
||||
- **Matrix jobs lose their name in the status list.** Gitea does not expand `${{ matrix.chart }}` in a job `name:`; it substitutes the matrix *expression*, so the commit status reads `Validate Helm Charts / Install-test ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }} (pull_request)` and **all legs of a matrix share that one context**. Consequence for branch protection: a per-chart required check is not possible on Gitea, and the required-check names differ from GitHub's — they have to be read off a real run rather than copied.
|
||||
- **Actions are fetched from github.com anonymously and that works.** `actions/checkout`, `azure/setup-helm` and the 1Password action need no replacement; only an action that *itself* authenticates against github.com breaks (see Trivy above).
|
||||
- **No merge queue.** Gitea has none. Its place is taken by auto-merge on green checks, which for a single maintainer is equivalent in effect and different in one way worth remembering: auto-merge merges as soon as the checks it knows about are green, so it is only as strong as the *required* checks configured on `main`. With none configured it merges immediately.
|
||||
|
||||
Still on the maintainer's side and not delegable: branch protection plus the commit-signature requirement on the Gitea repo, the runner's cgroup problem above, and switching GitHub Actions off before enabling the mirror. The bot token gets `403` on `actions/runners`, `actions/secrets` and `branch_protections`, so none of that can be verified from here.
|
||||
|
||||
### Resource-quantity render test (all charts, every PR)
|
||||
|
||||
@@ -80,15 +98,17 @@ Fixtures live at the **repo root** under `ci/` (deliberately *not* inside the ch
|
||||
- `ci/<chart>/fixtures.yaml` — dummy secrets with in-cluster connection data, throwaway infrastructure (postgres/redis/dex), static hostPath PVs/StorageClasses where the chart pins a storage class or needs RWX, and seed jobs (e.g. `synapse generate`)
|
||||
- `ci/<chart>/fixtures.env.yaml` — fixtures that need a value which must not be committed. The file holds `${VARIABLE}` placeholders and nothing else; `ci/run-install-test.sh` renders it with `envsubst` and pipes the result straight into `kubectl` (never printed). Substitution is restricted to the names from `REQUIRED_ENV`, so an undeclared variable is never filled in and no unrelated `$…` in the manifest is touched
|
||||
- `ci/<chart>/settings.env` — optional overrides: `INSTALL_TIMEOUT` for slow starters, and `REQUIRED_ENV="VAR_A VAR_B"` to declare the variables `fixtures.env.yaml` needs. **A missing or empty declared variable fails the install-test loudly, naming the variable — it never skips the chart.** Skipping would restore exactly the green-but-meaningless signal this mechanism exists to remove; the one legitimate case of absent secrets, a PR from a fork, belongs in an `if` on the workflow job rather than in the script
|
||||
- `ci/excluded-charts.txt` — charts that cannot run in CI, each with a justification. Currently: `satisfactory` (multi-GB image + 10GB+ Steam download), `template-chart` (scaffold with placeholder image)
|
||||
- `ci/excluded-charts.txt` — charts that cannot run in CI, one per line as `<chart> <justification>`. Currently: `satisfactory` (multi-GB image + 10GB+ Steam download), `template-chart` (scaffold with placeholder image). The justification is mandatory: a line carrying only a chart name is deliberately **not** an exclusion, because a bare name would drop a chart out of CI with nothing to review. **An excluded chart gets no install-test job at all** — see below
|
||||
|
||||
`freesailarr` is the only chart using `fixtures.env.yaml` today: gluetun needs a real Wireguard key, or it never completes a handshake and no app container behind its kill switch becomes Ready. The install-test job loads the key from 1Password Connect for that chart only, using the same pattern as `publish-helm-chart.yml` — `steps.<id>.outputs.*` into the step's own `env:`, quoted, with `:?` guards that name the missing `op://` field before the cluster is even created. The key in that item is a **CI-only Mullvad key**, deliberately not the production one: Mullvad ties an address to a key and routes the tunnel to the endpoint it saw last, so a shared key would make every PR run pull the tunnel out from under the running stack — with the symptom in the cluster and the cause in a pull request.
|
||||
|
||||
**An excluded chart is dropped from the matrix, not skipped inside its job (#231).** It used to get a job that ran the script, printed `SKIP` and exited 0 — and GitHub knows only the exit code, so the check list showed `Install-test satisfactory ✓` for a chart whose image was never pulled, indistinguishable from one that really came up in the cluster. `detect-changed-charts` now removes the excluded charts before the matrix is built, so no job exists for them and nothing claims they passed. Because an absence on its own says nothing, that job writes every excluded chart of the PR **with its documented justification** into the job summary — a `Skipped` would only say "not run", and why is the question a review actually asks. The list is parsed in exactly one place: `ci/run-install-test.sh --list-excluded` emits `<chart><TAB><reason>` and the workflow calls it instead of re-implementing the format in YAML, since a second parser would be a second chance to select different charts — silently, and in either direction. The script keeps its own `SKIP` branch for local runs. What cannot be done instead: deciding per chart in an `if` on the install-test job, to get a visible `Skipped` entry rather than an absent one. GitHub rejects it with `context "matrix" is not allowed here` — `jobs.<id>.if` is evaluated **before** the matrix is expanded, so the `matrix` context does not exist there, and the whole workflow ends in failure with zero jobs created (measured on the closed PR #233).
|
||||
|
||||
Fixture images avoid Docker Hub where possible (`public.ecr.aws`/`ghcr.io`) to dodge runner rate limits. The job **blocks the PR** on failure — the stabilization phase of #22 ended when Renovate started automerging patch/minor updates, since a chart that no longer starts must not reach `main` and publish unattended. When adding a new chart, add its `ci/<chart>/` fixtures (or an entry in `ci/excluded-charts.txt`) — the install-test fails if both are missing. The script also runs locally against any existing cluster: `k3d cluster create t && ci/run-install-test.sh <chart> && k3d cluster delete t`.
|
||||
|
||||
## Dependency updates (Renovate)
|
||||
|
||||
Renovate runs as the self-hosted `jlab-renovate` GitHub App. Its config is **`renovate.json5`** in the repo root (JSON5 for the inline reasoning; requires Renovate >= 41 for `managerFilePatterns`/`customType`). There must be exactly one config file — do not add a second `renovate.json` or `.github/renovate.json`.
|
||||
Renovate runs as the self-hosted `jlab-renovate` GitHub App while the repo is on github.com, and on Gitea as the shared `jlab-renovate-bot` instance that already serves the other repositories there — see "Renovate on Gitea" below. **There is no Renovate workflow in this repo, and none should be added.** Which of the two runs it changes nothing about *what* gets updated: its config is **`renovate.json5`** in the repo root (JSON5 for the inline reasoning; requires Renovate >= 41 for `managerFilePatterns`/`customType`). There must be exactly one config file — do not add a second `renovate.json` or `.github/renovate.json`.
|
||||
|
||||
What Renovate keeps current:
|
||||
|
||||
@@ -132,9 +152,20 @@ Everything is derived from the base commit, so the script is idempotent across t
|
||||
|
||||
If the dependency dashboard reports `Failed to look up github-releases package …: no-result` for third-party repos, that is the instance's GitHub credentials, not this config: a GitHub App installation token is not a good fit for looking up repos the app is not installed on. Give the instance a separate read-only PAT as `GITHUB_COM_TOKEN` for github.com datasource lookups.
|
||||
|
||||
### Renovate on Gitea
|
||||
|
||||
**Do not add a Renovate workflow to this repository.** It is the obvious thing to reach for — Gitea has no app model, so a scheduled job in the repo looks like the only way to run the bot — and it is wrong here: a **shared `jlab-renovate-bot` instance already runs on the Gitea side** and already has this repository enrolled. A second runner against the same repo produces duplicate PRs and two schedulers rewriting each other's branches, which is not a safety net but a fault. This paragraph exists so the next reader who finds no workflow knows why instead of building one.
|
||||
|
||||
The instance is outside this repository, which has two consequences worth writing down:
|
||||
|
||||
- **It reads our `renovate.json5` and the per-chart rules do fire.** Measured, not assumed: its first run on the migrated repo opened PRs for `searxng` and `gotenberg` plus the dependency dashboard. Those two are `customManagers` targets of ours — no built-in manager finds an `appVersion:` in a `Chart.yaml` — so the config is demonstrably in effect.
|
||||
- **`allowedCommands` is already granted there, and the version-bump hook runs.** This was the expensive unknown: `postUpgradeTasks` only executes when the command is allow-listed in the *instance* config, and a missing entry is skipped **silently**, leaving every chart PR red on the version check. The proof is in that searxng PR's diff, which carries `version: 1.0.4+up2026.9.11.ffe96f8a6` → `1.0.5+up2026.9.13.96479fff6` alongside the `appVersion` change. Nothing but `ci/renovate-bump-chart.sh` writes `version:`. Nothing has to be configured for the move — but the allow-list is not visible from this repo either, so if it is ever dropped, the symptom here will be red chart PRs with no explanation.
|
||||
|
||||
One item does not come for free on either side and is named in "The version-bump hook" above: a `GITHUB_COM_TOKEN` for github.com datasource lookups. Under the GitHub App those rode on the installation token; a Gitea token is worth nothing at github.com, and unauthenticated lookups hit the rate limit inside a single pass. The symptom is a `no-result` line on the dependency dashboard, not an error.
|
||||
|
||||
### PR policy
|
||||
|
||||
- **Automerge** for `patch`, `minor` and `digest` updates, after a `minimumReleaseAge` of 3 days and only once the PR pipeline is green — install-test included, which is why that job no longer runs with `continue-on-error`. Minor is in scope because the sidecar images barely move in patch steps (`alpine 3.22` → `3.24`, `nginx 1.30.4` → `1.31.4` are both minor), so a patch-only rule would practically never fire. Renovate merges itself rather than using GitHub auto-merge, which would merge immediately as long as `main` has no required status checks. Majors are merged by hand.
|
||||
- **Automerge** for `patch`, `minor` and `digest` updates, after a `minimumReleaseAge` of 3 days and only once the PR pipeline is green — install-test included, which is why that job no longer runs with `continue-on-error`. Minor is in scope because the sidecar images barely move in patch steps (`alpine 3.22` → `3.24`, `nginx 1.30.4` → `1.31.4` are both minor), so a patch-only rule would practically never fire. Renovate merges itself (`platformAutomerge: false`) rather than handing the PR to the forge's own auto-merge, which on **both** GitHub and Gitea would merge immediately as long as `main` has no required status checks. That is a statement about the branch protection, not about the platform: once required checks exist on `main`, `platformAutomerge: true` becomes the better setting on either side. Majors are merged by hand.
|
||||
- **Majors** arrive one line at a time (`separateMultipleMajor`): a v1 → v3 jump produces a PR for the latest v2 first and then one for v3 — the stepwise procedure the versioning section requires.
|
||||
- `prConcurrentLimit: 10` / `prHourlyLimit: 4` keep the k3d install-tests from all starting at once.
|
||||
- The manually dispatched `change-app-version.yml` still exists for one-off bumps, but the normal path for app updates is now a Renovate PR.
|
||||
@@ -182,6 +213,7 @@ These guidelines are **binding for every current and future chart** in this repo
|
||||
|
||||
- The main app image tag comes from `{{ .Chart.AppVersion }}` — never hardcoded.
|
||||
- **Documented exception — freesailarr**: the chart bundles ten independently versioned upstreams (nine of them enabled by default), so none of them is "the" app version. **No** tag comes from `appVersion`; every image is pinned as `<app>.image.repository` / `<app>.image.tag` in `values.yaml`, which is also what puts them under the built-in `helm-values` manager. Its `appVersion` is the chart's own version marker, freely chosen and tied to no upstream — raised by hand when a new edition of the chart is marked as a whole. Consequently the chart has **no app-version** block in `renovate.json5` and is **not** in the `change-app-version.yml` choice list. `mediathekarr` adds a second deviation on top: its tags carry no version at all, so it is pinned by `image.digest` as well and is the one image in this chart with a `customManagers` block — a `repository`/`tag` pair alone would leave it without any update path.
|
||||
- **Documented exception — minecraft-server**: the only chart whose main image tag can be overridden, through `image.tagOverride` (#158, empty by default — the tag still comes from `appVersion`). `appVersion` is the tag and therefore fixes the JVM, and since chart 5.0.0 only the **newest** Java line is maintained out of this one directory; the older lines stay published as frozen chart versions (java8 and java17 at 4.0.0, java21 at 4.1.0) and never receive another change. Without the override, running an older JVM would mean installing a frozen chart and giving up every later fix — so the line is a decision at rollout time rather than a property of the chart. The override is named `tagOverride`, not `tag`, because it decouples the running image from the chart version, and it is deliberately outside Renovate's reach (`renovate.json5` tracks the `appVersion` in `Chart.yaml` and nothing else). The chart checks the pairing itself: the Java line of the **effective** tag against `minecraft.version`, failing the render with both values named — a mismatch used to install silently and only break the container afterwards.
|
||||
- Auxiliary images (sidecars, init containers, secondary workloads — e.g. the delegation nginx in matrix-synapse, guacd in patchmon, busybox init/consume images in paperless-ngx) are pinned via values instead of being hardcoded in templates, and their pinned versions are reviewed alongside every app-version update round.
|
||||
|
||||
### 4. Ingress
|
||||
|
||||
Reference in new issue
Block a user