diff --git a/CI-CD.md b/CI-CD.md index 2bad7b0..1172c8c 100644 --- a/CI-CD.md +++ b/CI-CD.md @@ -4,12 +4,33 @@ Two pipelines: one that **validates** a pull request, one that **publishes** a c ## Publishing -- **`publish-.yml`** — one thin workflow per chart (26 of them). It triggers on pushes to `main` touching `/**`, or manually, and calls the reusable workflow below. +- **`publish-.yml`** — one thin workflow per chart (25 of them). It triggers on pushes to `main` touching `/**`, or manually, and calls the shared block below. - **A push to a chart directory on `main` publishes that chart.** The chart version must be bumped in the same change; the PR pipeline enforces it. -- **`publish-helm-chart.yml`** — the reusable workflow: fetch dependencies, `helm package`, then a single HTTP `POST` of the `.tgz` to the registry. +- **`publish-helm-chart.yml`** — **not a workflow of this repository.** It lives in [`workflow-util/pipelines`](https://git.jk-dev.de/workflow-util/pipelines), the shared building-block repo, and does the same as before: fetch dependencies, `helm package`, then a single HTTP `POST` of the `.tgz` to the registry. The registry is a **classic, index-based** Helm registry (Gitea's `/api/charts` endpoint), not an OCI one: `helm push` speaks only OCI and is not used, there is no `helm registry login` session, and every request carries its own Basic auth. Credentials **and the target URL** come from 1Password Connect (item `jlab-registry-access` with `username`, `password`, `charts-url`). +### What a caller here supplies + +The hulls are bound to a tag, never to that repository's default branch — otherwise a change over there would change publishing here, unannounced, and the failure would appear where nobody touched anything. Each hull is four lines of substance: + +```yaml +uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 +with: + chart-name: '' + registry-owner: ${{ github.repository_owner }} +secrets: inherit +``` + +| What | Why it is the caller's job | +| --- | --- | +| `registry-owner` | A required input with no default. The owner used to be derived inside the workflow; as an input the expression is evaluated **here**, where the value is measured. That repository's house rule is that a shared block never guesses an owner segment — the rule was bought by a hardcoded owner that sent every project's images to one wrong account. | +| `secrets: inherit` | The block needs `OP_CONNECT_TOKEN` from this repository. Unchanged by the move. | +| Variable `OP_CONNECT_HOST` | A repository variable, set on the instance. It replaced the Connect host that used to be hardcoded in our copy; a shared block must not carry one tenant's infrastructure address. The block's first step fails and names the variable when it is empty. | +| `ci/helm-dependencies.sh` | Still **our** script, and still run before `helm package`. The block's checkout clones the calling repository, so it runs from this tree — which also means deleting or renaming it breaks publishing from the other side of the tag. | + +**Where the reasons live now.** Why the credentials go in through `curl --config -` rather than `--user`, why `--fail-with-body` is what makes a failed upload fail the job, why the file name is built rather than globbed, and how the loaded secrets are read: all of it is on the [`publish-helm-chart` page](https://git.jk-dev.de/workflow-util/pipelines/wiki/publish-helm-chart) of that repository's wiki. It is deliberately **not** repeated here — two copies of one explanation means one of them is wrong eventually, and the file those sentences describe is no longer ours. Nothing was lost in the move: the upload step's shell body went across unchanged apart from two error messages that name the owner as an input. + ### The target URL is composed, not stored `charts-url` holds only the **base up to `/api/packages`**. The workflow appends the rest: @@ -19,37 +40,29 @@ op://GitHub-Actions/jlab-registry-access/charts-url = https:///api/packa target = //helm/api/charts ``` -The owner segment comes from **`github.repository_owner`** — verified to be populated on this Gitea instance (measured as `jlab-cluster` on a temporary probe step, since removed). Consequences worth knowing: +The owner segment is the `registry-owner` input, which every hull fills with **`github.repository_owner`** — verified to be populated on this Gitea instance (measured as `jlab-cluster` on a temporary probe step, since removed). It used to be read inside the workflow; passing it in from here changed nothing about the resulting URL, only about where the expression is evaluated. Consequences worth knowing: -- **No registry address appears anywhere in the repository**, and since this change not even the owner path segment: it is derived from the repository rather than stored. The 26 thin callers need no setting at all. -- The **second owner on the same instance** (`suer`, for `suer-configurator`) gets the right target from the same workflow, without a second value to keep in step. -- On the **github.com mirror** the owner is a different one — but Actions are switched off there, so the publish never runs. If it ever did, the derived owner would be one the instance does not know: HTTP 404, loud, not a silent upload to the wrong place. +- **No registry address appears anywhere in the repository**, not even the owner path segment: it is derived from the repository rather than stored. The thin callers need no setting of their own beyond the one line. +- The **second owner on the same instance** (`suer`, for `suer-configurator`) gets the right target from the same block, without a second value to keep in step. Since the owner is an input rather than a derived value, that repository can use the block as it is. +- On the **github.com mirror** the owner is a different one — but Actions are switched off there, so the publish never runs. If it ever did, the owner would be one the instance does not know: HTTP 404, loud, not a silent upload to the wrong place. -Two guards make a mismatch between vault item and workflow loud rather than puzzling, because the only other signal is a 404 against a URL 1Password has masked: +**What has to be in the vault item**, because the block rejects anything else — the base only, no owner, no package type, no endpoint: -| Guard | Rejects | -| --- | --- | -| `charts-url` must not end in `/api/charts` | the old, full URL; the error names `/api/packages` as the shape to store | -| `charts-url` must not contain a `/helm` segment | the old URL's package-type segment | -| `charts-url` must end in `/api/packages` | everything else, a base that already carries the owner included | -| derived owner non-empty | an empty `github.repository_owner`, which would build `//helm/api/charts` | -| derived owner is a plain path segment (`[A-Za-z0-9._-]`) | a `/`, a space or a `?`, which would build a different URL entirely | +``` +op://GitHub-Actions/jlab-registry-access/charts-url = https:///api/packages +``` -**Changing this is a hard cut:** the maintainer shortens the vault item to the base **first**, then the workflow change is merged. Both mismatches fail with a 404 and neither publishes to a wrong place. +The block guards both halves of the composition and names the vault field in the error, because the only other signal would be a 404 against a URL 1Password has masked. The individual guards and the measurements behind them are on [its wiki page](https://git.jk-dev.de/workflow-util/pipelines/wiki/publish-helm-chart); what matters on this side is the shape above. -One more thing to know about proving it: a change to `publish-helm-chart.yml` touches no chart directory, so **merging it triggers no publish** and leaves the new path unproven until the next chart change. The end-to-end check is a single manual `workflow_dispatch` of `publish-helm-chart.yml` for any chart — the registry replaces an already published version on re-upload (HTTP 201, see below), so it is a safe test of the whole path. Only the maintainer can trigger it; the bot account has no write permission on the repository. +**The migration to that shape was a hard cut** (historical, done): the vault item was shortened to the base **first**, then the workflow change was merged. Both possible mismatches fail with a 404 and neither publishes to a wrong place. -### Three properties of the upload step, each paid for once +### Proving the path after a change to the publishing route -- **Credentials stay out of the process list.** `curl --user "$U:$P"` would put them in `argv`. The step uses `curl --config -` and pipes the options in on stdin. -- **A failed upload fails the job.** Unlike `helm push`, `curl` exits **0** on HTTP 401/403/409 — a publish would go green with nothing uploaded. `--fail-with-body` turns those into a non-zero exit and still prints the response body, so the log names the cause. -- **The file name is built, not globbed.** `-.tgz` is assembled from the chart name and the version read from `Chart.yaml`. `--upload-file` takes exactly one file, and a glob matching two silently turns the second into curl's URL argument. `helm package` keeps the `+` of `1.4.0+up1.9.2` verbatim, so the built name matches exactly. +A change to the hulls or to the shared block **touches no chart directory, so merging it triggers no publish** and leaves the new route unproven until the next chart change happens to exercise it. -### How the secrets are read +The end-to-end check is a single manual `workflow_dispatch` of any `publish-.yml` — this is what those dispatch triggers are for, and after the move they are the only way to start a publish by hand. The registry replaces an already published version on re-upload (HTTP 201, below), so it is a safe test of the whole path. Only the maintainer can trigger it; the bot account cannot dispatch workflows. -Values are read via **`steps..outputs.*`**, mapped into the step's own `env:` block and used as quoted shell variables — never interpolated into the command line. Each is guarded with `:?`, because an empty value used to collapse into a missing argument rather than an error. - -`load-secrets-action` **v5 no longer exports them as environment variables** for later steps the way v4 did. After the v5 bump every `${{ env.* }}` reference resolved to an empty string while the loading step still logged `Populating variable` and stayed green. +Three things only that run can prove, and they are the whole remaining risk of the move: that a cross-repository `uses:` resolves on this instance, that a repository variable reaches a workflow called from another repository, and that the upload itself still lands. > **The registry accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact. A republish of an existing version is therefore still silent. @@ -171,7 +184,7 @@ Not on a schedule: it would spend compute every day on a question that changes r Not a workflow but the prerequisite of every helm command in this repo: it fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one). -It runs in **five** places, because every helm command exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in `publish-helm-chart.yml`, and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. One of those call sites is the render test that covers **every** chart on **every** PR — so a dependency-carrying chart can turn a third-party Helm repository's bad day into a red pipeline on an unrelated PR. +It runs in **five** places, because every helm command exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in the shared publish block, and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. The second of those is now across a repository boundary: the block calls this script from the calling repository's checkout, which makes the script part of the contract — renaming or deleting it breaks publishing from a place that no longer reads our diffs. One of those call sites is the render test that covers **every** chart on **every** PR — so a dependency-carrying chart can turn a third-party Helm repository's bad day into a red pipeline on an unrelated PR. It runs `helm dependency update`, not `build`: `Chart.lock` is not committed, because Renovate rewrites `Chart.yaml` and cannot regenerate the lock, so a committed one would arrive stale. @@ -183,7 +196,7 @@ Both matrices follow the changed **chart directories**, so a PR that only change All measured on this instance, not taken from documentation, and the reason for every deviation in `validate-charts.yml`. -**Reusable workflows with `secrets: inherit` work**, so the 26 thin publish hulls run unchanged. One quirk: a *missing* secret does not fail the job, it fails job **creation** — the run then shows no steps at all. +**Reusable workflows with `secrets: inherit` work**, so the 25 thin publish hulls run unchanged — including now that the workflow they call lives in another repository. One quirk: a *missing* secret does not fail the job, it fails job **creation** — the run then shows no steps at all. **Runner labels mean different machines.** `ubuntu-latest` has **no Docker socket** here; `ubuntu-docker` has one. Exactly one job cares — the install-test, because k3d is a Docker client — so its `runs-on` is an expression rather than a fixed label, and the same file still works on a GitHub runner where `ubuntu-docker` does not exist. diff --git a/New-Chart.md b/New-Chart.md index 7461842..1874667 100644 --- a/New-Chart.md +++ b/New-Chart.md @@ -33,7 +33,7 @@ The schema and the null-safe fallback helpers come with the template, but their ## 3. Wire up CI — four steps, each silent when skipped -1. **`.github/workflows/publish-.yml`** — copy an existing one (e.g. `publish-outline.yml`). It triggers on pushes to `main` touching `/**` and calls the reusable `publish-helm-chart.yml`. Without it the chart is never published, and nothing complains. +1. **`.github/workflows/publish-.yml`** — copy an existing one (e.g. `publish-outline.yml`). It triggers on pushes to `main` touching `/**` and calls `publish-helm-chart.yml`, which lives in `workflow-util/pipelines` and is bound to a tag. **Copy, do not write from memory:** the `uses:` path, the tag, `registry-owner: ${{ github.repository_owner }}` and `secrets: inherit` all have to be right, and a hull missing the owner input fails only when the publish actually runs. Without the file at all the chart is never published, and nothing complains. See [CI/CD](CI-CD#what-a-caller-here-supplies). 2. **`.github/workflows/change-app-version.yml`** — add the chart to the `chart-name` choice list. 3. **`ci//`** — install-test fixtures: `test-values.yaml` (required), plus `fixtures.yaml`, `fixtures.env.yaml` and `settings.env` where the chart needs them. Alternatively an entry **with justification** in `ci/excluded-charts.txt`. The install-test fails if both are missing. See [CI/CD](CI-CD#install-test-fixtures). 4. **`renovate.json5`** — a `customManagers` block for the chart's main image. Without it the app version is never updated. Check the image's tag list first: a flavour, date or prerelease suffix needs a `versioningTemplate` **with a comment saying why**, and floating tags like `latest`/`stable` belong outside the pattern. See [Renovate](Renovate#a-capture-group-is-a-claim-about-how-the-value-is-compared).