jlab-cluster/helm-charts
3
0 Fork 0
Code Issues 15 Pull Requests 3 Actions Packages Wiki Activity

CI/CD: publish-helm-chart liegt jetzt in workflow-util/pipelines 

Die Seite beschrieb den Publish-Workflow als Workflow dieses Repositories. Er
liegt seit Gitea #66 in workflow-util/pipelines und wird an einen Tag gebunden
aufgerufen.

Neu ist ein Abschnitt, was ein Aufrufer hier beisteuert: registry-owner als
Pflichteingabe mit github.repository_owner, secrets: inherit, die
Repository-Variable OP_CONNECT_HOST, und ci/helm-dependencies.sh - das bleibt
unser Skript und laeuft aus unserem Checkout, womit es Teil des Vertrags ueber
die Repository-Grenze wird.

Die Begruendungen des Upload-Schritts stehen nicht mehr hier, sondern im Wiki
von pipelines, mit Verweis. Zwei Kopien einer Erklaerung heisst, dass eine von
beiden irgendwann falsch ist - und die Datei, die diese Saetze beschreiben, ist
nicht mehr unsere. Was hier bleibt, ist das Repository-Eigene: die erwartete
Form des Tresor-Felds, der zweite Eigentuemer auf derselben Instanz, der
Spiegel ohne Actions und der erneute Upload derselben Version.

Der Abschnitt zum Nachweis ist umgeschrieben: Ein manueller Dispatch einer
Huelle ist nach dem Umzug der einzige Weg, einen Publish von Hand zu starten,
und er ist das Einzige, was die drei offenen Punkte beweisen kann -
cross-repo-uses, eine Variable in einem fremd aufgerufenen Workflow, und der
Upload selbst.

Dazu nachgezogen: die Huellenzahl (25, nicht 26 oder 24), der Hinweis bei
helm-dependencies.sh, und in New-Chart der Schritt 1 - eine Huelle wird kopiert
und nicht aus dem Gedaechtnis geschrieben, weil eine fehlende Owner-Eingabe
erst beim echten Publish auffaellt.
jklein committed 2026-10-01 23:04:06 +00:00
1 parent d7ce2055a0
commit 18796c709a
2 files changed
+40 -27
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

CI-CD.md
+39 -26
@@ -4,12 +4,33 @@ Two pipelines: one that **validates** a pull request, one that **publishes** a c
## Publishing
- **`publish-<chart>.yml`** — one thin workflow per chart (26 of them). It triggers on pushes to `main` touching `<chart>/**`, or manually, and calls the reusable workflow below.
- **`publish-<chart>.yml`** — one thin workflow per chart (25 of them). It triggers on pushes to `main` touching `<chart>/**`, or manually, and calls the shared block below.
- **A push to a chart directory on `main` publishes that chart.** The chart version must be bumped in the same change; the PR pipeline enforces it.
- **`publish-helm-chart.yml`** — the reusable workflow: fetch dependencies, `helm package`, then a single HTTP `POST` of the `.tgz` to the registry.
- **`publish-helm-chart.yml`** — **not a workflow of this repository.** It lives in [`workflow-util/pipelines`](https://git.jk-dev.de/workflow-util/pipelines), the shared building-block repo, and does the same as before: fetch dependencies, `helm package`, then a single HTTP `POST` of the `.tgz` to the registry.
The registry is a **classic, index-based** Helm registry (Gitea's `/api/charts` endpoint), not an OCI one: `helm push` speaks only OCI and is not used, there is no `helm registry login` session, and every request carries its own Basic auth. Credentials **and the target URL** come from 1Password Connect (item `jlab-registry-access` with `username`, `password`, `charts-url`).
### What a caller here supplies
The hulls are bound to a tag, never to that repository's default branch — otherwise a change over there would change publishing here, unannounced, and the failure would appear where nobody touched anything. Each hull is four lines of substance:
```yaml
uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3
with:
chart-name: '<chart>'
registry-owner: ${{ github.repository_owner }}
secrets: inherit
```
| What | Why it is the caller's job |
| --- | --- |
| `registry-owner` | A required input with no default. The owner used to be derived inside the workflow; as an input the expression is evaluated **here**, where the value is measured. That repository's house rule is that a shared block never guesses an owner segment — the rule was bought by a hardcoded owner that sent every project's images to one wrong account. |
| `secrets: inherit` | The block needs `OP_CONNECT_TOKEN` from this repository. Unchanged by the move. |
| Variable `OP_CONNECT_HOST` | A repository variable, set on the instance. It replaced the Connect host that used to be hardcoded in our copy; a shared block must not carry one tenant's infrastructure address. The block's first step fails and names the variable when it is empty. |
| `ci/helm-dependencies.sh` | Still **our** script, and still run before `helm package`. The block's checkout clones the calling repository, so it runs from this tree — which also means deleting or renaming it breaks publishing from the other side of the tag. |
**Where the reasons live now.** Why the credentials go in through `curl --config -` rather than `--user`, why `--fail-with-body` is what makes a failed upload fail the job, why the file name is built rather than globbed, and how the loaded secrets are read: all of it is on the [`publish-helm-chart` page](https://git.jk-dev.de/workflow-util/pipelines/wiki/publish-helm-chart) of that repository's wiki. It is deliberately **not** repeated here — two copies of one explanation means one of them is wrong eventually, and the file those sentences describe is no longer ours. Nothing was lost in the move: the upload step's shell body went across unchanged apart from two error messages that name the owner as an input.
### The target URL is composed, not stored
`charts-url` holds only the **base up to `/api/packages`**. The workflow appends the rest:
@@ -19,37 +40,29 @@ op://GitHub-Actions/jlab-registry-access/charts-url = https://<host>/api/packa
target = <base>/<owner>/helm/api/charts
```
The owner segment comes from **`github.repository_owner`** — verified to be populated on this Gitea instance (measured as `jlab-cluster` on a temporary probe step, since removed). Consequences worth knowing:
The owner segment is the `registry-owner` input, which every hull fills with **`github.repository_owner`** — verified to be populated on this Gitea instance (measured as `jlab-cluster` on a temporary probe step, since removed). It used to be read inside the workflow; passing it in from here changed nothing about the resulting URL, only about where the expression is evaluated. Consequences worth knowing:
- **No registry address appears anywhere in the repository**, and since this change not even the owner path segment: it is derived from the repository rather than stored. The 26 thin callers need no setting at all.
- The **second owner on the same instance** (`suer`, for `suer-configurator`) gets the right target from the same workflow, without a second value to keep in step.
- On the **github.com mirror** the owner is a different one — but Actions are switched off there, so the publish never runs. If it ever did, the derived owner would be one the instance does not know: HTTP 404, loud, not a silent upload to the wrong place.
- **No registry address appears anywhere in the repository**, not even the owner path segment: it is derived from the repository rather than stored. The thin callers need no setting of their own beyond the one line.
- The **second owner on the same instance** (`suer`, for `suer-configurator`) gets the right target from the same block, without a second value to keep in step. Since the owner is an input rather than a derived value, that repository can use the block as it is.
- On the **github.com mirror** the owner is a different one — but Actions are switched off there, so the publish never runs. If it ever did, the owner would be one the instance does not know: HTTP 404, loud, not a silent upload to the wrong place.
Two guards make a mismatch between vault item and workflow loud rather than puzzling, because the only other signal is a 404 against a URL 1Password has masked:
**What has to be in the vault item**, because the block rejects anything else — the base only, no owner, no package type, no endpoint:
| Guard | Rejects |
| --- | --- |
| `charts-url` must not end in `/api/charts` | the old, full URL; the error names `/api/packages` as the shape to store |
| `charts-url` must not contain a `/helm` segment | the old URL's package-type segment |
| `charts-url` must end in `/api/packages` | everything else, a base that already carries the owner included |
| derived owner non-empty | an empty `github.repository_owner`, which would build `<base>//helm/api/charts` |
| derived owner is a plain path segment (`[A-Za-z0-9._-]`) | a `/`, a space or a `?`, which would build a different URL entirely |
```
op://GitHub-Actions/jlab-registry-access/charts-url = https://<host>/api/packages
```
**Changing this is a hard cut:** the maintainer shortens the vault item to the base **first**, then the workflow change is merged. Both mismatches fail with a 404 and neither publishes to a wrong place.
The block guards both halves of the composition and names the vault field in the error, because the only other signal would be a 404 against a URL 1Password has masked. The individual guards and the measurements behind them are on [its wiki page](https://git.jk-dev.de/workflow-util/pipelines/wiki/publish-helm-chart); what matters on this side is the shape above.
One more thing to know about proving it: a change to `publish-helm-chart.yml` touches no chart directory, so **merging it triggers no publish** and leaves the new path unproven until the next chart change. The end-to-end check is a single manual `workflow_dispatch` of `publish-helm-chart.yml` for any chart — the registry replaces an already published version on re-upload (HTTP 201, see below), so it is a safe test of the whole path. Only the maintainer can trigger it; the bot account has no write permission on the repository.
**The migration to that shape was a hard cut** (historical, done): the vault item was shortened to the base **first**, then the workflow change was merged. Both possible mismatches fail with a 404 and neither publishes to a wrong place.
### Three properties of the upload step, each paid for once
### Proving the path after a change to the publishing route
- **Credentials stay out of the process list.** `curl --user "$U:$P"` would put them in `argv`. The step uses `curl --config -` and pipes the options in on stdin.
- **A failed upload fails the job.** Unlike `helm push`, `curl` exits **0** on HTTP 401/403/409 — a publish would go green with nothing uploaded. `--fail-with-body` turns those into a non-zero exit and still prints the response body, so the log names the cause.
- **The file name is built, not globbed.** `<chart>-<version>.tgz` is assembled from the chart name and the version read from `Chart.yaml`. `--upload-file` takes exactly one file, and a glob matching two silently turns the second into curl's URL argument. `helm package` keeps the `+` of `1.4.0+up1.9.2` verbatim, so the built name matches exactly.
A change to the hulls or to the shared block **touches no chart directory, so merging it triggers no publish** and leaves the new route unproven until the next chart change happens to exercise it.
### How the secrets are read
The end-to-end check is a single manual `workflow_dispatch` of any `publish-<chart>.yml` — this is what those dispatch triggers are for, and after the move they are the only way to start a publish by hand. The registry replaces an already published version on re-upload (HTTP 201, below), so it is a safe test of the whole path. Only the maintainer can trigger it; the bot account cannot dispatch workflows.
Values are read via **`steps.<id>.outputs.*`**, mapped into the step's own `env:` block and used as quoted shell variables — never interpolated into the command line. Each is guarded with `:?`, because an empty value used to collapse into a missing argument rather than an error.
`load-secrets-action` **v5 no longer exports them as environment variables** for later steps the way v4 did. After the v5 bump every `${{ env.* }}` reference resolved to an empty string while the loading step still logged `Populating variable` and stayed green.
Three things only that run can prove, and they are the whole remaining risk of the move: that a cross-repository `uses:` resolves on this instance, that a repository variable reaches a workflow called from another repository, and that the upload itself still lands.
> **The registry accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact. A republish of an existing version is therefore still silent.
@@ -171,7 +184,7 @@ Not on a schedule: it would spend compute every day on a question that changes r
Not a workflow but the prerequisite of every helm command in this repo: it fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one).
It runs in **five** places, because every helm command exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in `publish-helm-chart.yml`, and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. One of those call sites is the render test that covers **every** chart on **every** PR — so a dependency-carrying chart can turn a third-party Helm repository's bad day into a red pipeline on an unrelated PR.
It runs in **five** places, because every helm command exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in the shared publish block, and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. The second of those is now across a repository boundary: the block calls this script from the calling repository's checkout, which makes the script part of the contract — renaming or deleting it breaks publishing from a place that no longer reads our diffs. One of those call sites is the render test that covers **every** chart on **every** PR — so a dependency-carrying chart can turn a third-party Helm repository's bad day into a red pipeline on an unrelated PR.
It runs `helm dependency update`, not `build`: `Chart.lock` is not committed, because Renovate rewrites `Chart.yaml` and cannot regenerate the lock, so a committed one would arrive stale.
@@ -183,7 +196,7 @@ Both matrices follow the changed **chart directories**, so a PR that only change
All measured on this instance, not taken from documentation, and the reason for every deviation in `validate-charts.yml`.
**Reusable workflows with `secrets: inherit` work**, so the 26 thin publish hulls run unchanged. One quirk: a *missing* secret does not fail the job, it fails job **creation** — the run then shows no steps at all.
**Reusable workflows with `secrets: inherit` work**, so the 25 thin publish hulls run unchanged — including now that the workflow they call lives in another repository. One quirk: a *missing* secret does not fail the job, it fails job **creation** — the run then shows no steps at all.
**Runner labels mean different machines.** `ubuntu-latest` has **no Docker socket** here; `ubuntu-docker` has one. Exactly one job cares — the install-test, because k3d is a Docker client — so its `runs-on` is an expression rather than a fixed label, and the same file still works on a GitHub runner where `ubuntu-docker` does not exist.
New-Chart.md
+1 -1
@@ -33,7 +33,7 @@ The schema and the null-safe fallback helpers come with the template, but their
## 3. Wire up CI — four steps, each silent when skipped
1. **`.github/workflows/publish-<chart-name>.yml`** — copy an existing one (e.g. `publish-outline.yml`). It triggers on pushes to `main` touching `<chart-name>/**` and calls the reusable `publish-helm-chart.yml`. Without it the chart is never published, and nothing complains.
1. **`.github/workflows/publish-<chart-name>.yml`** — copy an existing one (e.g. `publish-outline.yml`). It triggers on pushes to `main` touching `<chart-name>/**` and calls `publish-helm-chart.yml`, which lives in `workflow-util/pipelines` and is bound to a tag. **Copy, do not write from memory:** the `uses:` path, the tag, `registry-owner: ${{ github.repository_owner }}` and `secrets: inherit` all have to be right, and a hull missing the owner input fails only when the publish actually runs. Without the file at all the chart is never published, and nothing complains. See [CI/CD](CI-CD#what-a-caller-here-supplies).
2. **`.github/workflows/change-app-version.yml`** — add the chart to the `chart-name` choice list.
3. **`ci/<chart-name>/`** — install-test fixtures: `test-values.yaml` (required), plus `fixtures.yaml`, `fixtures.env.yaml` and `settings.env` where the chart needs them. Alternatively an entry **with justification** in `ci/excluded-charts.txt`. The install-test fails if both are missing. See [CI/CD](CI-CD#install-test-fixtures).
4. **`renovate.json5`** — a `customManagers` block for the chart's main image. Without it the app version is never updated. Check the image's tag list first: a flavour, date or prerelease suffix needs a `versioningTemplate` **with a comment saying why**, and floating tags like `latest`/`stable` belong outside the pattern. See [Renovate](Renovate#a-capture-group-is-a-claim-about-how-the-value-is-compared).