From 8878329f3166c159d34c5509fc0fb2585c4039fd Mon Sep 17 00:00:00 2001 From: Justin Klein Date: Thu, 1 Oct 2026 23:01:47 +0000 Subject: [PATCH] ci: publish charts through the shared block in workflow-util/pipelines The reusable publish workflow now lives in workflow-util/pipelines and is called pinned to @v3. All 25 thin hulls switch over and our copy is deleted in the same commit: a hull pointing at a deleted file, or an orphaned file nobody calls, would both be worse than the state before. Each hull gains one line, registry-owner: ${{ github.repository_owner }}. The owner used to be derived inside the workflow (#63); as a required input without a default it is evaluated here, in the calling repository, where the value is measured. That is the house rule of pipelines - a shared block never guesses an owner segment - and it keeps the block free of any context semantics. The workflow_dispatch and push triggers of every hull are untouched; after the move the dispatch is the only way to trigger a publish by hand. The Connect host is no longer wired into the block. It reads the repository variable OP_CONNECT_HOST, which the maintainer has set, and fails in its first step naming the variable when it is empty. Documentation follows the same split as the code. The README entry no longer describes a workflow of this repository; it names where the block lives, what each caller owes it, and points at the publish-helm-chart page of that repository's wiki for the reasons behind the upload step, which are no longer ours to keep current. Nothing of them was lost in the move - the upload step's shell body went across unchanged apart from two error messages that name the owner as an input. Also corrected, because the move made them wrong rather than merely stale: - the helm-dependencies.sh entry still counted the local publish workflow among the five places it runs; it is still five, one of them now the shared block, which is why that block names the script as a prerequisite of its callers; - two comments in install-test.yml pointed at publish-helm-chart.yml as a sibling file, one of them for a trap it documented "at length" - that long form is now on the pipelines wiki, and the comments say so; - the hull count read 24, and reusable workflows with secrets: inherit are now confirmed to work across repositories too; - the chart-creation step in CLAUDE.md said the reusable workflow is ours. install-test.yml stays here, with the reason written down (decision 4 on the issue): it is reusable in form but welded to the ci/ fixtures, the exclusion parser and this repository's chart matrix. A block only one repository can use is not shared, only moved. The note is there so the question is not asked again. Fixes #66 --- .github/workflows/install-test.yml | 15 +- .github/workflows/publish-apache-tika.yml | 3 +- .github/workflows/publish-bentopdf.yml | 3 +- .../publish-calibre-web-automated.yml | 3 +- .github/workflows/publish-cyberchef.yml | 3 +- .github/workflows/publish-freesailarr.yml | 3 +- .github/workflows/publish-gotenberg.yml | 3 +- .github/workflows/publish-helm-chart.yml | 199 ------------------ .github/workflows/publish-homeassistant.yml | 3 +- .github/workflows/publish-matrix-synapse.yml | 3 +- .../workflows/publish-minecraft-router.yml | 3 +- .../workflows/publish-minecraft-server.yml | 3 +- .github/workflows/publish-outline.yml | 3 +- .github/workflows/publish-paperless-ngx.yml | 3 +- .github/workflows/publish-patchmon.yml | 3 +- .github/workflows/publish-rallly.yml | 3 +- .github/workflows/publish-rss-bridge.yml | 3 +- .github/workflows/publish-samba.yml | 3 +- .github/workflows/publish-satisfactory.yml | 3 +- .github/workflows/publish-searxng.yml | 3 +- .github/workflows/publish-sure.yml | 3 +- .../workflows/publish-teamspeak-server.yml | 3 +- .../workflows/publish-urlaubsverwaltung.yml | 3 +- .github/workflows/publish-valkey.yml | 3 +- .github/workflows/publish-voucher-vault.yml | 3 +- .../workflows/publish-ytdlp-web-player.yml | 3 +- .github/workflows/publish-zipline.yml | 3 +- CLAUDE.md | 2 +- README.md | 22 +- 29 files changed, 74 insertions(+), 239 deletions(-) delete mode 100644 .github/workflows/publish-helm-chart.yml diff --git a/.github/workflows/install-test.yml b/.github/workflows/install-test.yml index 3f5750c..2bc4bbc 100644 --- a/.github/workflows/install-test.yml +++ b/.github/workflows/install-test.yml @@ -30,9 +30,10 @@ on: OP_CONNECT_TOKEN: # Needed by exactly one leg - freesailarr, for the 1Password Connect # lookup of its CI Wireguard key. Declared OPTIONAL on purpose, where - # publish-helm-chart.yml declares its token required: on this instance a - # missing secret does not fail the job, it fails job CREATION, so the - # run shows no steps at all (README, "Where this repo lives"). With + # the shared publish block in workflow-util/pipelines declares its token + # required: on this instance a missing secret does not fail the job, it + # fails job CREATION, so the run shows no steps at all (README, "Where + # this repo lives"; the long form is on that repository's wiki). With # required: true a pull request from a fork - which carries no secrets - # would lose the entire install-test that way, every chart at once, with # nothing naming the cause. Optional instead: only the freesailarr leg @@ -196,8 +197,8 @@ jobs: # freesailarr is the one chart whose install-test needs credentials: # without a real Wireguard key gluetun never completes a handshake, and # behind its kill switch not a single app container becomes Ready. The key - # comes from 1Password Connect like the registry credentials in - # publish-helm-chart.yml, and it is a CI-ONLY Mullvad key: Mullvad ties an + # comes from 1Password Connect like the registry credentials in the shared + # publish block, and it is a CI-ONLY Mullvad key: Mullvad ties an # address to a key and sends the tunnel to the endpoint it saw last, so # reusing the production key would let every PR run pull the tunnel out # from under the running stack - with the symptom in the cluster and the @@ -216,7 +217,9 @@ jobs: OP_CONNECT_TOKEN: ${{ secrets.OP_CONNECT_TOKEN }} FREESAILARR_WG_KEY: op://GitHub-Actions/freesailarr-mullvad-test-config/credential FREESAILARR_WG_ADDRESSES: op://GitHub-Actions/freesailarr-mullvad-test-config/ip-address - # The same trap publish-helm-chart.yml documents at length: v5 of + # The same trap the shared publish block documents at length - since + # Gitea #66 that long form lives on the wiki of workflow-util/pipelines, + # page "publish-helm-chart", not in this repository: v5 of # load-secrets-action does NOT export the loaded values as environment # variables for later steps, only its step outputs carry them, and the # loading step logs "Populating variable" and stays green either way. So diff --git a/.github/workflows/publish-apache-tika.yml b/.github/workflows/publish-apache-tika.yml index 329039e..4e89466 100644 --- a/.github/workflows/publish-apache-tika.yml +++ b/.github/workflows/publish-apache-tika.yml @@ -9,7 +9,8 @@ on: jobs: publish-apache-tika: name: Publish Apache Tika Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'apache-tika' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-bentopdf.yml b/.github/workflows/publish-bentopdf.yml index a5cf016..497b4db 100644 --- a/.github/workflows/publish-bentopdf.yml +++ b/.github/workflows/publish-bentopdf.yml @@ -9,7 +9,8 @@ on: jobs: publish-bentopdf: name: Publish BentoPDF Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'bentopdf' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-calibre-web-automated.yml b/.github/workflows/publish-calibre-web-automated.yml index 2af8291..9d8a850 100644 --- a/.github/workflows/publish-calibre-web-automated.yml +++ b/.github/workflows/publish-calibre-web-automated.yml @@ -9,7 +9,8 @@ on: jobs: publish-calibre-web-automated: name: Publish Calibre Web Automated Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'calibre-web-automated' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-cyberchef.yml b/.github/workflows/publish-cyberchef.yml index e07bb69..56ca47e 100644 --- a/.github/workflows/publish-cyberchef.yml +++ b/.github/workflows/publish-cyberchef.yml @@ -9,7 +9,8 @@ on: jobs: publish-cyberchef: name: Publish Cyberchef Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'cyberchef' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-freesailarr.yml b/.github/workflows/publish-freesailarr.yml index 8fe35ef..6daf19e 100644 --- a/.github/workflows/publish-freesailarr.yml +++ b/.github/workflows/publish-freesailarr.yml @@ -9,7 +9,8 @@ on: jobs: publish-freesailarr: name: Publish Freesailarr Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'freesailarr' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-gotenberg.yml b/.github/workflows/publish-gotenberg.yml index cd106b5..5304874 100644 --- a/.github/workflows/publish-gotenberg.yml +++ b/.github/workflows/publish-gotenberg.yml @@ -9,7 +9,8 @@ on: jobs: publish-gotenberg: name: Publish Gotenberg Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'gotenberg' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-helm-chart.yml b/.github/workflows/publish-helm-chart.yml deleted file mode 100644 index 069bb19..0000000 --- a/.github/workflows/publish-helm-chart.yml +++ /dev/null @@ -1,199 +0,0 @@ -name: Publish Helm Chart with patched Application Version -on: - workflow_dispatch: - inputs: - chart-location: - required: false - type: string - default: "." - description: "location of the helm chart directory" - chart-name: - required: true - type: string - description: "name of the helm chart" - workflow_call: - inputs: - chart-location: - required: false - type: string - default: "." - description: "location of the helm chart directory" - chart-name: - required: true - type: string - description: "name of the helm chart" - outputs: - chart-version: - description: "the version of the published helm chart" - value: ${{ jobs.publish-helm-chart.outputs.chart-version }} - secrets: - OP_CONNECT_TOKEN: - required: true -jobs: - publish-helm-chart: - outputs: - chart-version: ${{ steps.get-chart-version.outputs.CHART_VERSION }} - runs-on: ubuntu-latest - steps: - - name: "Checkout Repository" - uses: actions/checkout@v7 - - name: "Install Helm" - uses: azure/setup-helm@v5 - - name: "Configure OP-Connect" - uses: 1password/load-secrets-action/configure@v5 - with: - connect-host: https://op-connect.jk-dev.de - # helm package exits 1 with "found in Chart.yaml, but missing in charts/ - # directory" while a declared dependency has not been fetched, so this - # comes first. It is a no-op for every chart without a dependencies: - # block; ci/helm-dependencies.sh explains why it resolves from Chart.yaml - # instead of a committed Chart.lock. Note the consequence for the - # published artifact: the subchart .tgz is packaged INTO ours, so the - # version pinned in Chart.yaml is the version operators get, and nothing - # downstream ever sees the dependency again. - - name: "Fetch chart dependencies" - run: ${{ github.workspace }}/ci/helm-dependencies.sh "${{ inputs.chart-name }}" - working-directory: ${{ inputs.chart-location }} - - name: "Package Helm Chart" - run: helm package ${{ inputs.chart-name }} - working-directory: ${{ inputs.chart-location }} - - name: "Get Helm Chart Version" - id: get-chart-version - run: |- - CHART_VERSION=$(grep '^version:' ${{ inputs.chart-name}}/Chart.yaml | awk '{print $2}') - echo $CHART_VERSION - echo "CHART_VERSION=$CHART_VERSION" >> $GITHUB_OUTPUT - working-directory: ${{ inputs.chart-location }} - - name: "Load Chart Registry Credentials" - id: load-registry-creds - uses: 1password/load-secrets-action@v5 - env: - OP_CONNECT_TOKEN: ${{ secrets.OP_CONNECT_TOKEN }} - REGISTRY_PASSWORD: op://GitHub-Actions/jlab-registry-access/password - REGISTRY_USERNAME: op://GitHub-Actions/jlab-registry-access/username - REGISTRY_CHARTS_URL: op://GitHub-Actions/jlab-registry-access/charts-url - # The target is a classic, index-based Helm registry, not an OCI one, so - # there is no "helm registry login" session and no "helm push": the upload - # is a single HTTP POST of the packaged .tgz to - # //helm/api/charts, authenticated per request. - # - # That URL is composed, not stored (#63). charts-url holds only the base - # up to "/api/packages"; the owner segment comes from - # github.repository_owner - which Gitea does populate: measured as - # "jlab-cluster" in run 282 of validate-charts.yml, on a temporary step - # that was removed again. So no value pointing at the registry - not even - # a path segment - lives in this repository, and a second owner on the same - # instance needs no second setting. The mirror on github.com has Actions - # switched off, so the publish never runs there; if it ever did, the - # derived owner would be one the instance does not know and the upload - # would 404 rather than land somewhere wrong. Two guards make the switch - # from the old shape loud instead of puzzling: charts-url is rejected - # unless it ends in "/api/packages" (an old full URL still carrying - # "/helm", "/api/charts" or the owner is named explicitly), and the derived - # owner is rejected when empty or when it holds characters that are not a - # path segment - an empty one would build "//helm/api/charts", i.e. - # another bare 404 against a masked URL. - # - # Three further properties of this step are load-bearing and each of them - # was paid for once already. - # - # 1) The values are mapped into the step's own env block and read as quoted - # shell variables - never interpolated into the command line. load-secrets- - # action v5 (bumped in #140) no longer exports the loaded values as - # environment variables for later steps the way v4 did; only the step - # outputs carry them. Every "${{ env.* }}" reference therefore resolved to - # an empty string from that bump on, with nothing reporting an error - the - # loading step logs "Populating variable" and stays green either way. The - # ":?" guards fail loudly and name the op:// field that is missing, and - # quoting keeps an empty value from silently shifting arguments. - # - # 2) The credentials never reach the process list. "curl --user $U:$P" - # would put them in argv; "curl --config -" reads its options from stdin - # instead, so nothing sensitive appears in any argument. The price of that - # config file is its value syntax: an unquoted value ends at the first - # space (measured on curl 8.5.0 - "has space" arrived as "has"), so the - # step refuses whitespace in the credentials up front rather than letting - # a truncated password come back as a puzzling 401. A Gitea access token - # is 40 hex characters, so this only ever fires on a mis-entered item. - # - # 3) "--fail-with-body" is what makes a failed upload fail the job. Unlike - # "helm push", curl exits 0 on HTTP 401/403/409 unless asked not to - the - # publish would go green while nothing was uploaded. Measured against Gitea - # 1.24.7: a 401 without the flag is exit 0, with the flag exit 22, and the - # response body is still printed so the log names the cause. - # - # The file name is built from the chart name and the version read above - # instead of a "-*.tgz" glob: --upload-file takes exactly one file, - # and a glob matching two turns the second into curl's URL argument, while - # a glob matching none is passed through literally. Both fail obscurely. - # "helm package" keeps the "+" of a "1.4.0+up1.9.2" version verbatim in the - # file name, so "-.tgz" is an exact match (measured). - - name: "Upload Helm Chart to the Chart Registry" - env: - REGISTRY_PASSWORD: ${{ steps.load-registry-creds.outputs.REGISTRY_PASSWORD }} - REGISTRY_USERNAME: ${{ steps.load-registry-creds.outputs.REGISTRY_USERNAME }} - REGISTRY_CHARTS_URL: ${{ steps.load-registry-creds.outputs.REGISTRY_CHARTS_URL }} - REGISTRY_OWNER: ${{ github.repository_owner }} - CHART_NAME: ${{ inputs.chart-name }} - CHART_VERSION: ${{ steps.get-chart-version.outputs.CHART_VERSION }} - run: |- - : "${REGISTRY_CHARTS_URL:?empty - check op://GitHub-Actions/jlab-registry-access/charts-url}" - : "${REGISTRY_USERNAME:?empty - check op://GitHub-Actions/jlab-registry-access/username}" - : "${REGISTRY_PASSWORD:?empty - check op://GitHub-Actions/jlab-registry-access/password}" - : "${REGISTRY_OWNER:?empty - github.repository_owner is not populated on this forge; do not hardcode an owner, report it}" - : "${CHART_NAME:?empty - the chart-name input did not arrive}" - : "${CHART_VERSION:?empty - could not read version: from ${CHART_NAME}/Chart.yaml}" - # 1Password masks the full URL, but a curl connection error prints only - # ":", which that mask would not cover. Mask the host too. - CHARTS_HOST="${REGISTRY_CHARTS_URL#*://}" - echo "::add-mask::${CHARTS_HOST%%/*}" - # charts-url is the base only; this workflow appends the owner, the - # package type and the endpoint. Reject the old, full shape by name so - # the failure says what to change in the vault item instead of 404ing - # against a URL the log has masked. - CHARTS_BASE="${REGISTRY_CHARTS_URL%/}" - case "$CHARTS_BASE" in - */api/charts) - echo "::error::op://GitHub-Actions/jlab-registry-access/charts-url still ends in /api/charts - it must hold only the base up to /api/packages (https:///api/packages); this workflow appends //helm/api/charts itself" - exit 1 - ;; - */helm|*/helm/*) - echo "::error::op://GitHub-Actions/jlab-registry-access/charts-url still contains the /helm path segment - it must hold only the base up to /api/packages (https:///api/packages); this workflow appends //helm/api/charts itself" - exit 1 - ;; - */api/packages) ;; - *) - echo "::error::op://GitHub-Actions/jlab-registry-access/charts-url must end in /api/packages (https:///api/packages) - no owner, no package type, no endpoint; this workflow appends //helm/api/charts itself" - exit 1 - ;; - esac - # An empty owner would build "//helm/api/charts", a stray "/" or - # "?" would build something else entirely - both are bare 404s. - case "$REGISTRY_OWNER" in - *[!A-Za-z0-9._-]*) - echo "::error::the derived repository owner is not usable as a URL path segment: '${REGISTRY_OWNER}'" - exit 1 - ;; - esac - CHARTS_API_URL="${CHARTS_BASE}/${REGISTRY_OWNER}/helm/api/charts" - case "${REGISTRY_USERNAME}${REGISTRY_PASSWORD}" in - *[[:space:]]*) - echo "::error::username/password in op://GitHub-Actions/jlab-registry-access must not contain whitespace - curl --config truncates an unquoted value at the first space" - exit 1 - ;; - esac - CHART_PACKAGE="${CHART_NAME}-${CHART_VERSION}.tgz" - test -f "$CHART_PACKAGE" || { - echo "::error::${CHART_PACKAGE} does not exist - helm package produced: $(ls -1 ./*.tgz 2>&1)" - exit 1 - } - echo "uploading ${CHART_PACKAGE}" - printf 'user = %s:%s\nurl = %s\n' \ - "$REGISTRY_USERNAME" "$REGISTRY_PASSWORD" "$CHARTS_API_URL" | - curl --config - \ - --fail-with-body \ - --silent --show-error \ - --request POST \ - --upload-file "$CHART_PACKAGE" \ - --write-out '\nHTTP %{http_code}\n' - working-directory: ${{ inputs.chart-location }} diff --git a/.github/workflows/publish-homeassistant.yml b/.github/workflows/publish-homeassistant.yml index 89003f2..1baa830 100644 --- a/.github/workflows/publish-homeassistant.yml +++ b/.github/workflows/publish-homeassistant.yml @@ -9,7 +9,8 @@ on: jobs: publish-homeassistant: name: Publish Homeassistant Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'homeassistant' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-matrix-synapse.yml b/.github/workflows/publish-matrix-synapse.yml index b3e83a2..ea33d87 100644 --- a/.github/workflows/publish-matrix-synapse.yml +++ b/.github/workflows/publish-matrix-synapse.yml @@ -9,7 +9,8 @@ on: jobs: publish-matrix-synapse: name: Publish Matrix Synapse Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'matrix-synapse' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-minecraft-router.yml b/.github/workflows/publish-minecraft-router.yml index 1cc56d9..8bc2ef0 100644 --- a/.github/workflows/publish-minecraft-router.yml +++ b/.github/workflows/publish-minecraft-router.yml @@ -9,7 +9,8 @@ on: jobs: publish-minecraft-router: name: Publish Minecraft-Router Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'minecraft-router' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-minecraft-server.yml b/.github/workflows/publish-minecraft-server.yml index c6e57c5..294459f 100644 --- a/.github/workflows/publish-minecraft-server.yml +++ b/.github/workflows/publish-minecraft-server.yml @@ -9,7 +9,8 @@ on: jobs: publish-minecraft-server: name: Publish Minecraft-Server Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'minecraft-server' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-outline.yml b/.github/workflows/publish-outline.yml index 025fde6..64cba1e 100644 --- a/.github/workflows/publish-outline.yml +++ b/.github/workflows/publish-outline.yml @@ -9,7 +9,8 @@ on: jobs: publish-outline: name: Publish Outline Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'outline' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-paperless-ngx.yml b/.github/workflows/publish-paperless-ngx.yml index 8e4a24d..4ee0189 100644 --- a/.github/workflows/publish-paperless-ngx.yml +++ b/.github/workflows/publish-paperless-ngx.yml @@ -9,7 +9,8 @@ on: jobs: publish-paperless-ngx: name: Publish Paperless-NGX Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'paperless-ngx' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-patchmon.yml b/.github/workflows/publish-patchmon.yml index a0caf01..cb888d4 100644 --- a/.github/workflows/publish-patchmon.yml +++ b/.github/workflows/publish-patchmon.yml @@ -9,7 +9,8 @@ on: jobs: publish-patchmon: name: Publish Patchmon Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'patchmon' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-rallly.yml b/.github/workflows/publish-rallly.yml index 20a06de..4bd6243 100644 --- a/.github/workflows/publish-rallly.yml +++ b/.github/workflows/publish-rallly.yml @@ -9,7 +9,8 @@ on: jobs: publish-rallly: name: Publish Rallly Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'rallly' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-rss-bridge.yml b/.github/workflows/publish-rss-bridge.yml index 41d7208..b8dbbe8 100644 --- a/.github/workflows/publish-rss-bridge.yml +++ b/.github/workflows/publish-rss-bridge.yml @@ -9,7 +9,8 @@ on: jobs: publish-rss-bridge: name: Publish RSS-Bridge Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'rss-bridge' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-samba.yml b/.github/workflows/publish-samba.yml index 8f4179c..14484c0 100644 --- a/.github/workflows/publish-samba.yml +++ b/.github/workflows/publish-samba.yml @@ -9,7 +9,8 @@ on: jobs: publish-samba: name: Publish Samba Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'samba' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-satisfactory.yml b/.github/workflows/publish-satisfactory.yml index 5b2acf4..bb57e8d 100644 --- a/.github/workflows/publish-satisfactory.yml +++ b/.github/workflows/publish-satisfactory.yml @@ -9,7 +9,8 @@ on: jobs: publish-satisfactory: name: Publish Satisfactory Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'satisfactory' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-searxng.yml b/.github/workflows/publish-searxng.yml index 582125b..3662c04 100644 --- a/.github/workflows/publish-searxng.yml +++ b/.github/workflows/publish-searxng.yml @@ -9,7 +9,8 @@ on: jobs: publish-searxng: name: Publish SearXNG Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'searxng' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-sure.yml b/.github/workflows/publish-sure.yml index efa4ef0..1dd25e2 100644 --- a/.github/workflows/publish-sure.yml +++ b/.github/workflows/publish-sure.yml @@ -9,7 +9,8 @@ on: jobs: publish-sure: name: Publish Sure Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'sure' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-teamspeak-server.yml b/.github/workflows/publish-teamspeak-server.yml index 7a4796d..56ecc51 100644 --- a/.github/workflows/publish-teamspeak-server.yml +++ b/.github/workflows/publish-teamspeak-server.yml @@ -9,7 +9,8 @@ on: jobs: publish-teamspeak-server: name: Publish TeamSpeak Server Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'teamspeak-server' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-urlaubsverwaltung.yml b/.github/workflows/publish-urlaubsverwaltung.yml index 5e27447..072a25c 100644 --- a/.github/workflows/publish-urlaubsverwaltung.yml +++ b/.github/workflows/publish-urlaubsverwaltung.yml @@ -9,7 +9,8 @@ on: jobs: publish-urlaubsverwaltung: name: Publish Urlaubsverwaltung Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'urlaubsverwaltung' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-valkey.yml b/.github/workflows/publish-valkey.yml index 2216b2e..acebd95 100644 --- a/.github/workflows/publish-valkey.yml +++ b/.github/workflows/publish-valkey.yml @@ -9,7 +9,8 @@ on: jobs: publish-valkey: name: Publish Valkey Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'valkey' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-voucher-vault.yml b/.github/workflows/publish-voucher-vault.yml index 1586a9b..7003d90 100644 --- a/.github/workflows/publish-voucher-vault.yml +++ b/.github/workflows/publish-voucher-vault.yml @@ -9,7 +9,8 @@ on: jobs: publish-voucher-vault: name: Publish VoucherVault Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'voucher-vault' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-ytdlp-web-player.yml b/.github/workflows/publish-ytdlp-web-player.yml index 0b96767..1047f62 100644 --- a/.github/workflows/publish-ytdlp-web-player.yml +++ b/.github/workflows/publish-ytdlp-web-player.yml @@ -9,7 +9,8 @@ on: jobs: publish-ytdlp-web-player: name: Publish YTDLP Web Player Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'ytdlp-web-player' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/.github/workflows/publish-zipline.yml b/.github/workflows/publish-zipline.yml index 3380ee7..a807a44 100644 --- a/.github/workflows/publish-zipline.yml +++ b/.github/workflows/publish-zipline.yml @@ -9,7 +9,8 @@ on: jobs: publish-zipline: name: Publish Zipline Helm Chart - uses: ./.github/workflows/publish-helm-chart.yml + uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3 with: chart-name: 'zipline' + registry-owner: ${{ github.repository_owner }} secrets: inherit diff --git a/CLAUDE.md b/CLAUDE.md index 45b6fa4..e588c4b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -85,7 +85,7 @@ helm template --set ingress.domain=example.com --set ingress.cluste Copy `template-chart/` and replace every `xxx` placeholder — in file names (`templates/xxx.deployment.yaml` → `templates/.deployment.yaml`), in `Chart.yaml`, in the `values.yaml` keys and their `.Values.xxx.*` references inside templates, and in `values.schema.json` (both the values paths and the `xxx.` helper prefixes). The schema and the null-safe fallback helpers come with the template but their content is chart-specific — adapt them along the six points of the README section "Values that must not lie", and validate the result against the chart's real bundle values, not only against its defaults. Then wire up CI: -1. Add `.github/workflows/publish-.yml` (copy an existing one, e.g. `publish-outline.yml`) — it triggers on pushes to `main` touching `/**` and calls the reusable `publish-helm-chart.yml`. +1. Add `.github/workflows/publish-.yml` (copy an existing one, e.g. `publish-outline.yml`) — it triggers on pushes to `main` touching `/**` and calls the reusable `publish-helm-chart.yml`, which lives in `workflow-util/pipelines` and is bound to a tag. Copying an existing hull gets the `uses:` path, `registry-owner: ${{ github.repository_owner }}` and `secrets: inherit` right; writing one from memory does not. 2. Add the chart name to the `chart-name` choice list in `.github/workflows/change-app-version.yml`. 3. Add install-test fixtures under `ci//` (`test-values.yaml`, `fixtures.yaml`) or an entry with justification in `ci/excluded-charts.txt` — the PR install-test fails if both are missing. 4. Add a `customManagers` block for the chart's main image in `renovate.json5` (one per chart, since the image repository lives in the template and the version in `Chart.yaml`) — without it Renovate never updates that chart's app version. See README section "Dependency updates (Renovate)". diff --git a/README.md b/README.md index a56cace..3ab3bf5 100644 --- a/README.md +++ b/README.md @@ -51,23 +51,29 @@ Every chart has a wiki page of its own — `https://git.jk-dev.de/jlab-cluster/h ## CI/CD - `publish-.yml` — one thin workflow per chart; triggers on pushes to `main` touching `/**` (or manually) and calls the reusable workflow below. **A push to a chart directory on `main` publishes that chart**, so the chart version must be bumped in the same change. -- `publish-helm-chart.yml` — reusable workflow: `helm package`, then a single HTTP `POST` of the packaged `.tgz` to the private chart registry. The target is a **classic, index-based** Helm registry (Gitea's `/api/charts` endpoint), not an OCI one: `helm push` speaks only OCI and is not used, there is no `helm registry login` session, and every request carries its own Basic auth. Credentials **and the target URL** come from 1Password Connect (`OP_CONNECT_TOKEN` secret, item `op://GitHub-Actions/jlab-registry-access/` with `username`, `password`, `charts-url`). `charts-url` holds only the **base up to `/api/packages`**; the workflow composes the target as `//helm/api/charts`, and the owner comes from **`github.repository_owner`** (#63) — so the 26 thin callers stay as they are and a second owner on the same instance needs no second setting. **No registry address ever appears in this repository**, and since #63 not even the owner path segment: it is derived from the repository rather than stored. Both halves of that composition are guarded, because the only other signal would be a 404 against a URL the log has masked: `charts-url` is rejected unless it ends in `/api/packages` (an old, full URL still carrying `/helm`, `/api/charts` or the owner is named explicitly in the error), and the derived owner is rejected when empty or when it is not a usable path segment — an empty one would build `//helm/api/charts`. The loaded values are read via **`steps..outputs.*`**, mapped into the step's own `env:` block and used as quoted shell variables — never interpolated into the command line. `load-secrets-action` **v5 no longer exports them as environment variables** for later steps the way v4 did, so after the v5 bump every `${{ env.* }}` reference resolved to an empty string while the loading step still logged `Populating variable` and stayed green. Each value is additionally guarded with `:?`, because an empty one used to collapse into a missing argument rather than an error. Three properties of the upload step are load-bearing: - - **Credentials stay out of the process list.** `curl --user "$U:$P"` would put them in `argv`; the step uses `curl --config -` and pipes the options in on stdin instead. - - **A failed upload fails the job.** Unlike `helm push`, `curl` exits **0** on HTTP 401/403/409 — a publish would go green with nothing uploaded. `--fail-with-body` turns those into a non-zero exit and still prints the response body, so the log names the cause. - - **The file name is built, not globbed.** `-.tgz` is assembled from the chart name and the version already read from `Chart.yaml`; `--upload-file` takes exactly one file, and a glob matching two silently turns the second into curl's URL argument. `helm package` keeps the `+` of a `1.4.0+up1.9.2` version verbatim in the file name, so the built name is an exact match. +- `publish-helm-chart.yml` — **not in this repository any more.** The reusable workflow lives in [`workflow-util/pipelines`](https://git.jk-dev.de/workflow-util/pipelines), the shared building-block repo, and the callers here bind it to a tag: `uses: workflow-util/pipelines/.gitea/workflows/publish-helm-chart.yml@v3` (Gitea #66). It still does the same thing — `helm package`, then a single HTTP `POST` of the packaged `.tgz` to the private chart registry, which is a **classic, index-based** Helm registry and not an OCI one, so `helm push` is not used, there is no `helm registry login` session, and every request carries its own Basic auth. What changed is who supplies what: + - **`registry-owner: ${{ github.repository_owner }}`** in each caller's `with:`. The owner used to be derived inside the workflow (#63); it is now a required input with no default, so the expression is evaluated **here**, in the calling repository, where the value is measured. That is `pipelines`' house rule — a shared block never guesses an owner segment — and it is why every caller carries that one extra line. + - **`secrets: inherit`**, unchanged, for `OP_CONNECT_TOKEN`. + - the repository **variable `OP_CONNECT_HOST`**, which replaced the Connect host that used to be hardcoded in our copy. A shared block must not carry one tenant's infrastructure address; the block's first step fails and names the variable when it is unset. - Note: the registry **accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact, so a republish of an existing version is still silent — see #160. + The registry credentials and the target URL still come from 1Password Connect (item `op://GitHub-Actions/jlab-registry-access/` with `username`, `password`, `charts-url`). `charts-url` holds only the **base up to `/api/packages`** and the block composes `//helm/api/charts`, so **no registry address appears in this repository** — not even the owner path segment. `ci/helm-dependencies.sh` is still run before `helm package`, and it is still **our** script: the block's checkout clones the calling repository, so it runs from this tree. + + The reasons behind the upload step are **not repeated here**, because they are no longer ours to keep current: why both halves of the composed URL are guarded, why the credentials go in through `curl --config -` instead of `--user`, why `--fail-with-body` is what makes a failed upload fail the job, and why the file name is built rather than globbed, are all on the [`publish-helm-chart` wiki page](https://git.jk-dev.de/workflow-util/pipelines/wiki/publish-helm-chart) of that repository. Nothing was dropped in the move — the upload step's shell body went across unchanged except for two error messages that name the owner as an input. + + Note: the registry **accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact, so a republish of an existing version is still silent — see #160. That is also what makes a manual `workflow_dispatch` of any `publish-.yml` a harmless way to prove the shared block works. - `change-app-version.yml` — manually dispatched app-version bump; the chart must be listed in its `chart-name` choice list. -- `ci/helm-dependencies.sh ` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one). It runs in **five** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in `publish-helm-chart.yml`, and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs. +- `ci/helm-dependencies.sh ` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one). It runs in **five** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in the shared publish block (which is why that block names this script as a prerequisite of its callers — see above), and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs. - `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (the same changed-charts matrix, minus the charts listed in `ci/excluded-charts.txt`) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix, or run the all-charts install-test after the merge. The **resource-quantity render test** is the exception to that rule: it runs over every chart on every PR — see below. - `install-test.yml` — reusable workflow (`workflow_call`) holding the install-test **job** itself: the runner-label expression, the pinned `kubectl` with its skew check, `envsubst`, the containerised-job detection, the per-leg k3d cluster and the `freesailarr` VPN credentials. Its one input is a JSON array of chart names; its callers decide *which* charts are tested and whether the job runs at all, never *how*. Two callers: `validate-charts.yml` and `install-test-all-charts.yml`. **It is not copied for the second caller** — a second copy is a second chance to test differently, unnoticed, the same argument that gives `ci/excluded-charts.txt` exactly one parser. + + **It stays in this repository, and that is a decision, not an oversight** (Gitea #66, decision 4). It is reusable in form but not in substance: it is welded to the `ci/` fixtures, to the `ci/excluded-charts.txt` parser and to this repository's chart matrix. A building block that only one repository can use is not shared, it is merely moved somewhere else — and moving it would buy nothing while putting a tag between us and a file we change often. `publish-helm-chart.yml` was the opposite case: it takes a chart name and a registry owner and knows nothing about this repository, which is exactly why it could go. This note exists so the question is not asked again in half a year. - `install-test-all-charts.yml` — manually dispatched **all-charts install-test**: the same job, with a matrix over every installable chart instead of the changed ones. Optional input for a subset of charts (empty = all). See below for when to run it. ### Where this repo lives, and what a Gitea runner does differently Everything below was measured on this instance, not taken from documentation, and it is the reason for every deviation in `validate-charts.yml`. How the move got here is not repeated — that is Gitea #14. -**Reusable workflows with `secrets: inherit` work**, so the 24 thin `publish-.yml` hulls run unchanged. One quirk worth knowing before debugging the wrong thing: a *missing* secret does not fail the job, it fails job **creation**, so the run shows no steps at all. +**Reusable workflows with `secrets: inherit` work**, so the 25 thin `publish-.yml` hulls run unchanged — including now that the workflow they call lives in another repository (Gitea #66). One quirk worth knowing before debugging the wrong thing: a *missing* secret does not fail the job, it fails job **creation**, so the run shows no steps at all. **Runner labels mean different machines than on GitHub.** `ubuntu-latest` here has **no Docker socket**; `ubuntu-docker` has one. Exactly one job cares — the install-test, because k3d is a Docker client — and its `runs-on` is therefore an expression rather than a fixed label, so the same file still works on a GitHub runner where `ubuntu-docker` does not exist. Every other job (lint/package, the quantity render, the Trivy scan) needs no socket and stays on `ubuntu-latest`. @@ -144,7 +150,7 @@ Fixtures live at the **repo root** under `ci/` (deliberately *not* inside the ch - `ci//settings.env` — optional overrides: `INSTALL_TIMEOUT` for slow starters, and `REQUIRED_ENV="VAR_A VAR_B"` to declare the variables `fixtures.env.yaml` needs. **A missing or empty declared variable fails the install-test loudly, naming the variable — it never skips the chart.** Skipping would restore exactly the green-but-meaningless signal this mechanism exists to remove; the one legitimate case of absent secrets, a PR from a fork, belongs in an `if` on the workflow job rather than in the script - `ci/excluded-charts.txt` — charts that cannot run in CI, one per line as ` `. Currently: `satisfactory` (multi-GB image + 10GB+ Steam download), `template-chart` (scaffold with placeholder image). The justification is mandatory: a line carrying only a chart name is deliberately **not** an exclusion, because a bare name would drop a chart out of CI with nothing to review. **An excluded chart gets no install-test job at all** — see below -`freesailarr` is the only chart using `fixtures.env.yaml` today: gluetun needs a real Wireguard key, or it never completes a handshake and no app container behind its kill switch becomes Ready. The install-test job loads the key from 1Password Connect for that chart only, using the same pattern as `publish-helm-chart.yml` — `steps..outputs.*` into the step's own `env:`, quoted, with `:?` guards that name the missing `op://` field before the cluster is even created. The key in that item is a **CI-only Mullvad key**, deliberately not the production one: Mullvad ties an address to a key and routes the tunnel to the endpoint it saw last, so a shared key would make every PR run pull the tunnel out from under the running stack — with the symptom in the cluster and the cause in a pull request. +`freesailarr` is the only chart using `fixtures.env.yaml` today: gluetun needs a real Wireguard key, or it never completes a handshake and no app container behind its kill switch becomes Ready. The install-test job loads the key from 1Password Connect for that chart only, using the same pattern as the shared publish block — `steps..outputs.*` into the step's own `env:`, quoted, with `:?` guards that name the missing `op://` field before the cluster is even created. The key in that item is a **CI-only Mullvad key**, deliberately not the production one: Mullvad ties an address to a key and routes the tunnel to the endpoint it saw last, so a shared key would make every PR run pull the tunnel out from under the running stack — with the symptom in the cluster and the cause in a pull request. **An excluded chart is dropped from the matrix, not skipped inside its job (#231).** It used to get a job that ran the script, printed `SKIP` and exited 0 — and GitHub knows only the exit code, so the check list showed `Install-test satisfactory ✓` for a chart whose image was never pulled, indistinguishable from one that really came up in the cluster. `detect-changed-charts` now removes the excluded charts before the matrix is built, so no job exists for them and nothing claims they passed. Because an absence on its own says nothing, that job writes every excluded chart of the PR **with its documented justification** into the job summary — a `Skipped` would only say "not run", and why is the question a review actually asks. The list is parsed in exactly one place: `ci/run-install-test.sh --list-excluded` emits `` and the workflow calls it instead of re-implementing the format in YAML, since a second parser would be a second chance to select different charts — silently, and in either direction. The script keeps its own `SKIP` branch for local runs. What cannot be done instead: deciding per chart in an `if` on the install-test job, to get a visible `Skipped` entry rather than an absent one. GitHub rejects it with `context "matrix" is not allowed here` — `jobs..if` is evaluated **before** the matrix is expanded, so the `matrix` context does not exist there, and the whole workflow ends in failure with zero jobs created (measured on the closed PR #233).