From eef939472eb029bed3ed9eac64535c8c05fc7746 Mon Sep 17 00:00:00 2001 From: Justin Klein Date: Thu, 1 Oct 2026 23:50:05 +0000 Subject: [PATCH] CI-CD: die SonarQube-Analyse, und warum sie kein Torwaechter ist Neuer Abschnitt auf CI-CD: der Aufruf, die Analysatoren und was sie hier abdecken, die fehlenden Ausschluesse mit Begruendung, die Voraussetzungen und wie ein fehlender aussieht, dazu die drei Punkte, die der erste Lauf beantwortet. Die Einleitung der Seite stimmte nicht mehr: Sie nannte zwei Pipelines und einen manuell ausgeloesten Lauf. Jetzt sind es vier Laeufe, und die Unterscheidung, die man mitnehmen muss, steht dort - nur die PR-Pipeline kann einen Merge aufhalten. Die Analyse meldet einen Zustand, der schon gemergt ist. Der Abschnitt zum Nachweis des Publish-Wegs nennt eine der drei offenen Fragen nicht mehr als diesem Weg allein gehoerend: Ob eine Repository-Variable in einem aus einem fremden Repository aufgerufenen Workflow ankommt, beantwortet der erste Sonar-Lauf fuer beide. Was auf der Seite des Bausteins steht - warum fetch-depth: 0 nicht optional ist, warum kein Docker-Runner noetig ist, was der Scanner zur Laufzeit holt - ist bewusst nicht wiederholt, sondern verlinkt. Zwei Kopien einer Begruendung heisst, dass eine davon irgendwann falsch ist, und die Datei gehoert uns nicht. Zum SONAR_TOKEN festgehalten, dass es vorhanden ist und dass "vorhanden" hier die Bestaetigung des Betreuers meint und keine Pruefung durch ein Werkzeug: Secrets und Variablen sind von aussen nicht lesbar, tea antwortet mit "user should be the owner of the repo". --- CI-CD.md | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 74 insertions(+), 1 deletion(-) diff --git a/CI-CD.md b/CI-CD.md index 1172c8c..c492952 100644 --- a/CI-CD.md +++ b/CI-CD.md @@ -1,6 +1,8 @@ # CI/CD -Two pipelines: one that **validates** a pull request, one that **publishes** a chart after the merge. Both run on Gitea Actions and read the files under `.github/workflows/` unchanged. Next to them one manually dispatched run, the [all-charts install-test](#the-all-charts-install-test-on-demand), which asks a question no pull request asks: does the environment under an *unchanged* chart still hold. +Two pipelines: one that **validates** a pull request, one that **publishes** a chart after the merge. Both run on Gitea Actions and read the files under `.github/workflows/` unchanged. Next to them two runs that no pull request starts: the [all-charts install-test](#the-all-charts-install-test-on-demand), dispatched by hand, which asks whether the environment under an *unchanged* chart still holds — and the [SonarQube analysis](#sonarqube-analysis), which runs on every push to `main` and **blocks nothing**. + +Which of the four can stop a merge is the distinction to carry into every section below: only the PR pipeline can. The publish runs after the decision, the all-charts install-test runs when someone asks, and the analysis reports a state that is already merged. ## Publishing @@ -64,6 +66,8 @@ The end-to-end check is a single manual `workflow_dispatch` of any `publish- **The registry accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact. A republish of an existing version is therefore still silent. ## The PR pipeline — `validate-charts.yml` @@ -180,6 +184,75 @@ Not on a schedule: it would spend compute every day on a question that changes r | **The record** | per chart: outcome, date, runner label, `K3S_IMAGE`, `kubectl` version. Plus a final job with the verdict over the whole matrix. A green run nobody notes is unknown again next time. | | **Expected runtime** | **20 charts in 12m41s** (Gitea PR #48, one job, all legs concurrent on one Docker daemon). Written down so a later deviation is visible as one; compare it against the chart count the run prints in its own summary — 24 at the time of writing, and one more with every new chart. The timeout is 30 minutes per leg, in `install-test.yml`, hence identical for both callers. **A big image moves this less than it looks:** [convertx](convertx) is ~1.4 GiB compressed, by a wide margin the largest in the repo, and its first green leg measured **2m35s** including cluster creation (Gitea PR #61) — inside the existing envelope, not a new one. The legs run concurrently, so this figure tracks the *slowest* leg plus scheduling, never the sum of the pulls. | +## SonarQube analysis + +`sonar.yml` is a hull of the same kind as the publish ones: it decides *when*, and the workflow itself is the shared block [`sonar-analysis`](https://git.jk-dev.de/workflow-util/pipelines/wiki/sonar-analysis) in `workflow-util/pipelines`, bound to a tag. The project-specific half is `sonar-project.properties` in the repo root. Why `fetch-depth: 0` is not optional, why the block needs no Docker runner, and what the scanner downloads at run time are on that page and deliberately not repeated here. + +```yaml +on: + push: + branches: [main] + workflow_dispatch: +jobs: + sonar: + uses: workflow-util/pipelines/.gitea/workflows/sonar-analysis.yml@v4 + secrets: inherit +``` + +### It observes, it does not gate + +The Community Build analyses **the main branch only** — there is no pull-request analysis to hook into. So the trigger is a push to `main`, and **a finding appears after the merge**. At that point there is nothing left to block, which is why the block carries no quality-gate step. + +That is not a gap waiting to be filled. A red gate behind the merge would colour `main` red without having prevented anything, and a permanent red is a thing people learn to ignore — at which point it has spent the credibility of `validate-charts`, the one check here that *can* stop a merge. It becomes the right thing under one of two conditions, and both change the situation rather than the wiring: pull-request analysis is available, or a deployment hangs off the workflow. + +Practical consequence for whoever reads a chart PR: the analysis will never be in its check list. The findings are in SonarQube, about `main`, and they are read there. + +### What is analysed + +The languages were checked against SonarQube's list, not assumed — the fit turned out better than expected, Shell included. + +| What is in this repository | Analyser | +| --- | --- | +| the 5 shell scripts under `ci/` — the whole CI logic | Shell | +| ~30 workflow files | GitHub Actions | +| 238 YAML files and 26 Helm template directories | Kubernetes/Helm, YAML | +| 26 `values.schema.json` | JSON | +| the whole tree | Secrets | + +**The last row is the reason to bother.** The premise of this repository is that a credential never lands in it — every secret is a 1Password `OnePasswordItem` reference, materialised by the operator in the cluster — and until now *nothing checked that premise*. Every other row is a second opinion on something `helm lint --strict`, the render test or Trivy already touch; this one covers an assumption that was carried for two years on discipline alone. + +### No exclusions, for the first round + +Including the Helm templates, which are not valid YAML because of `{{ }}`. + +What the Helm and YAML analysers make of such a file is only learnable by letting them run over one. Excluding them up front would discard that answer and, with it, the bulk of the repository — and it would be an exclusion written on a guess, which hides whatever else happens to match it. There is no `paths` filter on the hull either, for the matching reason: a filter would leave the previous result standing after a change it did not cover, while still presenting it as current. + +If the first run produces noise genuinely not worth reading, the exclusions go into `sonar-project.properties` **with the finding they answer named next to them**. That file is ours; the shared block never carries an exclusion for one repository. + +### Prerequisites, and what a missing one looks like + +| Name | Art | | +| --- | --- | --- | +| `SONAR_HOST_URL` | repository variable | Set on the instance. A **variable and not a secret** on purpose: a server address is not a credential, and as a variable it stays readable in the log — which is the only thing that makes a wrong address diagnosable. Masked, it would produce a failure against an address nobody can see, the exact situation that forced an extra masking step into the publish block. | +| `SONAR_TOKEN` | repository secret | Confirmed present by the maintainer. Reaches the block through `secrets: inherit`. Needs the right to run analyses in SonarQube, and the right to create a project if the first run is to create it. | +| `sonar-project.properties` | file in the repo root | Project key `jlab-cluster_helm-charts`. The key carries the owner because the instance serves two (`jlab-cluster` and `suer`); a bare `helm-charts` is free today and collides the day the second owner has a repository of that name — and a key cannot be changed afterwards without losing the project's history. | + +Secrets and variables are not readable from outside, including by the bot: `tea actions secrets ls` answers `user should be the owner of the repo`, and variables are not listed at all. So "set" here means confirmed by the maintainer, not verified by a tool — and the block's guard is what turns a wrong assumption into a named error instead of a puzzling failure. + +The guard covers an empty variable, an empty secret and a forgotten `secrets: inherit`, and it checks both values before it exits, so a repository that has set up neither learns both in one run. What it cannot cover is a **missing** required secret: that fails job **creation** on this instance, not the job, and the run then shows no steps at all — the same quirk as everywhere else here, and the understandable case for this block, because without a token there is no analysis to run. + +### Three things the first run answers + +None of these is measured, and the merge of the change that added the hull is itself the first run — the trigger is a push to `main`, and a merge is such a push. Unlike the [publish route](#proving-the-path-after-a-change-to-the-publishing-route), which needs a manual dispatch because it touches no chart directory, this one proves itself on arrival. + +1. **Is `node24` in the runner image?** The scan action declares `runs: using: node24` and is therefore a JavaScript action. If it is absent, the failure will read as a problem with the action, not with the image — which is the whole reason this is written down. +2. **Does the scanner download get through?** The action fetches the Sonar Scanner CLI from `binaries.sonarsource.com` at run time and checks its GPG signature; measured at ~59 MiB. This is the likeliest first failure, and it has history: a 17 MB download once failed on this runner because a Docker bridge at MTU 1500 met a pod network of 1450 — see [What a GitHub runner had been providing quietly](#what-a-github-runner-had-been-providing-quietly). That was fixed on the node side, and the scanner is a multiple of it. +3. **Does a repository variable reach a workflow called from another repository?** The same open question the publish route still carries for `OP_CONNECT_HOST`, and this run answers it for both. If it does not arrive, the answer is loud: the guard fires and names the variable. The repair would be the one already made for the owner segment over there — pass the address in as an input, evaluated in the caller. + +Plus the one that is not a yes/no: **what the analysis actually reports**, in particular out of the Helm templates. That is what decides whether exclusions are needed, and it belongs in `sonar-project.properties` when it does. + +`workflow_dispatch` is on the hull for the retries these may need. Without it every attempt would cost a dummy commit on `main`. Only the maintainer can trigger it — the bot account cannot dispatch workflows. + ## `ci/helm-dependencies.sh` — the step they all share Not a workflow but the prerequisite of every helm command in this repo: it fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one).