diff --git a/.github/workflows/publish-helm-chart.yml b/.github/workflows/publish-helm-chart.yml index 0592176..62337f9 100644 --- a/.github/workflows/publish-helm-chart.yml +++ b/.github/workflows/publish-helm-chart.yml @@ -64,52 +64,86 @@ jobs: echo $CHART_VERSION echo "CHART_VERSION=$CHART_VERSION" >> $GITHUB_OUTPUT working-directory: ${{ inputs.chart-location }} - - name: "Load Helm Registry Credentials" - id: load-jk-cr-creds + - name: "Load Chart Registry Credentials" + id: load-registry-creds uses: 1password/load-secrets-action@v5 env: OP_CONNECT_TOKEN: ${{ secrets.OP_CONNECT_TOKEN }} - JK_CR_TOKEN: op://GitHub-Actions/jk-registry-access/password - JK_CR_USERNAME: op://GitHub-Actions/jk-registry-access/username - JK_CR_CHARTS_LOCATION: op://GitHub-Actions/jk-registry-access/charts-location - JK_CR_HELM_REGISTRY: op://GitHub-Actions/jk-registry-access/helm-registry - # The values are mapped into each step's own env block and read as quoted - # shell variables - never interpolated into the command line. + REGISTRY_PASSWORD: op://GitHub-Actions/jlab-registry-access/password + REGISTRY_USERNAME: op://GitHub-Actions/jlab-registry-access/username + REGISTRY_CHARTS_URL: op://GitHub-Actions/jlab-registry-access/charts-url + # The target is a classic, index-based Helm registry, not an OCI one, so + # there is no "helm registry login" session and no "helm push": the upload + # is a single HTTP POST of the packaged .tgz to /api/charts, + # authenticated per request. Three properties of this step are load-bearing + # and each of them was paid for once already. # - # Two separate failures made this necessary. First, load-secrets-action v5 - # (bumped in #140) no longer exports the loaded values as environment - # variables for later steps the way v4 did; only the step outputs carry - # them. Every "${{ env.JK_CR_* }}" reference therefore resolved to an empty - # string from that bump on, with nothing reporting an error - the loading - # step logs "Populating variable" and stays green either way. + # 1) The values are mapped into the step's own env block and read as quoted + # shell variables - never interpolated into the command line. load-secrets- + # action v5 (bumped in #140) no longer exports the loaded values as + # environment variables for later steps the way v4 did; only the step + # outputs carry them. Every "${{ env.* }}" reference therefore resolved to + # an empty string from that bump on, with nothing reporting an error - the + # loading step logs "Populating variable" and stays green either way. The + # ":?" guards fail loudly and name the op:// field that is missing, and + # quoting keeps an empty value from silently shifting arguments. # - # Second, an empty value collapsed into a missing argument instead of an - # error: with an empty username, "-u" swallowed the following - # "--password-stdin", helm stopped reading the password from stdin, - # prompted interactively and died on the runner with "inappropriate ioctl - # for device" - a terminal error for what was really an unset variable. + # 2) The credentials never reach the process list. "curl --user $U:$P" + # would put them in argv; "curl --config -" reads its options from stdin + # instead, so nothing sensitive appears in any argument. The price of that + # config file is its value syntax: an unquoted value ends at the first + # space (measured on curl 8.5.0 - "has space" arrived as "has"), so the + # step refuses whitespace in the credentials up front rather than letting + # a truncated password come back as a puzzling 401. A Gitea access token + # is 40 hex characters, so this only ever fires on a mis-entered item. # - # The ":?" guards fail loudly and name the value that is missing, quoting - # keeps an empty one from shifting arguments, and the token no longer - # appears in the process list. - - name: "Login to JK-Registry" + # 3) "--fail-with-body" is what makes a failed upload fail the job. Unlike + # "helm push", curl exits 0 on HTTP 401/403/409 unless asked not to - the + # publish would go green while nothing was uploaded. Measured against Gitea + # 1.24.7: a 401 without the flag is exit 0, with the flag exit 22, and the + # response body is still printed so the log names the cause. + # + # The file name is built from the chart name and the version read above + # instead of a "-*.tgz" glob: --upload-file takes exactly one file, + # and a glob matching two turns the second into curl's URL argument, while + # a glob matching none is passed through literally. Both fail obscurely. + # "helm package" keeps the "+" of a "1.4.0+up1.9.2" version verbatim in the + # file name, so "-.tgz" is an exact match (measured). + - name: "Upload Helm Chart to the Chart Registry" env: - JK_CR_TOKEN: ${{ steps.load-jk-cr-creds.outputs.JK_CR_TOKEN }} - JK_CR_USERNAME: ${{ steps.load-jk-cr-creds.outputs.JK_CR_USERNAME }} - JK_CR_HELM_REGISTRY: ${{ steps.load-jk-cr-creds.outputs.JK_CR_HELM_REGISTRY }} + REGISTRY_PASSWORD: ${{ steps.load-registry-creds.outputs.REGISTRY_PASSWORD }} + REGISTRY_USERNAME: ${{ steps.load-registry-creds.outputs.REGISTRY_USERNAME }} + REGISTRY_CHARTS_URL: ${{ steps.load-registry-creds.outputs.REGISTRY_CHARTS_URL }} + CHART_NAME: ${{ inputs.chart-name }} + CHART_VERSION: ${{ steps.get-chart-version.outputs.CHART_VERSION }} run: |- - : "${JK_CR_HELM_REGISTRY:?empty - check op://GitHub-Actions/jk-registry-access/helm-registry}" - : "${JK_CR_USERNAME:?empty - check op://GitHub-Actions/jk-registry-access/username}" - : "${JK_CR_TOKEN:?empty - check op://GitHub-Actions/jk-registry-access/password}" - printf '%s' "$JK_CR_TOKEN" | helm registry login "$JK_CR_HELM_REGISTRY" -u "$JK_CR_USERNAME" --password-stdin - - name: "Push Helm Chart to JK-Container Registry" - env: - JK_CR_CHARTS_LOCATION: ${{ steps.load-jk-cr-creds.outputs.JK_CR_CHARTS_LOCATION }} - run: |- - : "${JK_CR_CHARTS_LOCATION:?empty - check op://GitHub-Actions/jk-registry-access/charts-location}" - helm push "${{ inputs.chart-name }}"-*.tgz "$JK_CR_CHARTS_LOCATION" + : "${REGISTRY_CHARTS_URL:?empty - check op://GitHub-Actions/jlab-registry-access/charts-url}" + : "${REGISTRY_USERNAME:?empty - check op://GitHub-Actions/jlab-registry-access/username}" + : "${REGISTRY_PASSWORD:?empty - check op://GitHub-Actions/jlab-registry-access/password}" + : "${CHART_NAME:?empty - the chart-name input did not arrive}" + : "${CHART_VERSION:?empty - could not read version: from ${CHART_NAME}/Chart.yaml}" + # 1Password masks the full URL, but a curl connection error prints only + # ":", which that mask would not cover. Mask the host too. + CHARTS_HOST="${REGISTRY_CHARTS_URL#*://}" + echo "::add-mask::${CHARTS_HOST%%/*}" + case "${REGISTRY_USERNAME}${REGISTRY_PASSWORD}" in + *[[:space:]]*) + echo "::error::username/password in op://GitHub-Actions/jlab-registry-access must not contain whitespace - curl --config truncates an unquoted value at the first space" + exit 1 + ;; + esac + CHART_PACKAGE="${CHART_NAME}-${CHART_VERSION}.tgz" + test -f "$CHART_PACKAGE" || { + echo "::error::${CHART_PACKAGE} does not exist - helm package produced: $(ls -1 ./*.tgz 2>&1)" + exit 1 + } + echo "uploading ${CHART_PACKAGE}" + printf 'user = %s:%s\nurl = %s\n' \ + "$REGISTRY_USERNAME" "$REGISTRY_PASSWORD" "${REGISTRY_CHARTS_URL%/}/api/charts" | + curl --config - \ + --fail-with-body \ + --silent --show-error \ + --request POST \ + --upload-file "$CHART_PACKAGE" \ + --write-out '\nHTTP %{http_code}\n' working-directory: ${{ inputs.chart-location }} - - name: "Logout from JK-Registry" - env: - JK_CR_HELM_REGISTRY: ${{ steps.load-jk-cr-creds.outputs.JK_CR_HELM_REGISTRY }} - run: helm registry logout "$JK_CR_HELM_REGISTRY" diff --git a/CLAUDE.md b/CLAUDE.md index e1d7cf5..451f3ab 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,7 +26,7 @@ Additional rules for all commits, PRs, issues, and comments: ## What this repo is -A collection of self-made Helm charts, one directory per chart, published to a private OCI registry via GitHub Actions. Validation happens on three levels: locally with Helm, in the PR pipeline (`validate-charts.yml`: strict lint, package, version-bump check), and in the PR install-test stage (k3d, fixtures under `ci/` — see README): +A collection of self-made Helm charts, one directory per chart, published to a private, classic (index-based) Helm registry via GitHub Actions — an HTTP `POST` of the packaged `.tgz`, not an OCI `helm push`; the target URL and its credentials live in 1Password and never in this repository. Validation happens on three levels: locally with Helm, in the PR pipeline (`validate-charts.yml`: strict lint, package, version-bump check), and in the PR install-test stage (k3d, fixtures under `ci/` — see README): ```bash helm lint --strict diff --git a/README.md b/README.md index 6c44459..2b82990 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # helm-charts -Collection of some self-made helm-charts. Each top-level directory is a standalone Helm chart that gets packaged and pushed to a private OCI registry (JK-Registry) via GitHub Actions. +Collection of some self-made helm-charts. Each top-level directory is a standalone Helm chart that gets packaged and uploaded to a private, classic (index-based) Helm registry via GitHub Actions. ## Charts @@ -41,7 +41,12 @@ Collection of some self-made helm-charts. Each top-level directory is a standalo ## CI/CD (GitHub Actions) - `publish-.yml` — one thin workflow per chart; triggers on pushes to `main` touching `/**` (or manually) and calls the reusable workflow below. **A push to a chart directory on `main` publishes that chart**, so the chart version must be bumped in the same change. -- `publish-helm-chart.yml` — reusable workflow: `helm package`, then `helm push` to the private JK-Registry. Registry credentials come from 1Password Connect (`OP_CONNECT_TOKEN` secret, items under `op://GitHub-Actions/`). The loaded values are read via **`steps..outputs.*`**, mapped into each step's own `env:` block and used as quoted shell variables — never interpolated into the command line. `load-secrets-action` **v5 no longer exports them as environment variables** for later steps the way v4 did, so after the v5 bump every `${{ env.JK_CR_* }}` reference resolved to an empty string while the loading step still logged `Populating variable` and stayed green. Each value is additionally guarded with `:?`, because an empty one used to collapse into a missing argument rather than an error: `-u` consumed the following `--password-stdin`, helm fell back to an interactive prompt and the job died with `inappropriate ioctl for device` — a terminal error for what was really an unset variable. +- `publish-helm-chart.yml` — reusable workflow: `helm package`, then a single HTTP `POST` of the packaged `.tgz` to the private chart registry. The target is a **classic, index-based** Helm registry (Gitea's `/api/charts` endpoint), not an OCI one: `helm push` speaks only OCI and is not used, there is no `helm registry login` session, and every request carries its own Basic auth. Credentials **and the target URL** come from 1Password Connect (`OP_CONNECT_TOKEN` secret, item `op://GitHub-Actions/jlab-registry-access/` with `username`, `password`, `charts-url`); the workflow appends `/api/charts` to `charts-url`. No registry address ever appears in this repository. The loaded values are read via **`steps..outputs.*`**, mapped into the step's own `env:` block and used as quoted shell variables — never interpolated into the command line. `load-secrets-action` **v5 no longer exports them as environment variables** for later steps the way v4 did, so after the v5 bump every `${{ env.* }}` reference resolved to an empty string while the loading step still logged `Populating variable` and stayed green. Each value is additionally guarded with `:?`, because an empty one used to collapse into a missing argument rather than an error. Three properties of the upload step are load-bearing: + - **Credentials stay out of the process list.** `curl --user "$U:$P"` would put them in `argv`; the step uses `curl --config -` and pipes the options in on stdin instead. + - **A failed upload fails the job.** Unlike `helm push`, `curl` exits **0** on HTTP 401/403/409 — a publish would go green with nothing uploaded. `--fail-with-body` turns those into a non-zero exit and still prints the response body, so the log names the cause. + - **The file name is built, not globbed.** `-.tgz` is assembled from the chart name and the version already read from `Chart.yaml`; `--upload-file` takes exactly one file, and a glob matching two silently turns the second into curl's URL argument. `helm package` keeps the `+` of a `1.4.0+up1.9.2` version verbatim in the file name, so the built name is an exact match. + + Note: the registry **accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact, so a republish of an existing version is still silent — see #160. - `change-app-version.yml` — manually dispatched app-version bump; the chart must be listed in its `chart-name` choice list. - `ci/helm-dependencies.sh ` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one). It runs in **five** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in `publish-helm-chart.yml`, and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs. - `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (same changed-charts matrix) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix. The **resource-quantity render test** is the exception to that rule: it runs over every chart on every PR — see below.