jlab-cluster/helm-charts
3
0 Fork 0
Code Issues 15 Pull Requests 3 Actions Packages Wiki Activity

CI-CD: die SonarQube-Analyse, und warum sie kein Torwaechter ist 

Neuer Abschnitt auf CI-CD: der Aufruf, die Analysatoren und was sie hier
abdecken, die fehlenden Ausschluesse mit Begruendung, die Voraussetzungen und
wie ein fehlender aussieht, dazu die drei Punkte, die der erste Lauf beantwortet.

Die Einleitung der Seite stimmte nicht mehr: Sie nannte zwei Pipelines und
einen manuell ausgeloesten Lauf. Jetzt sind es vier Laeufe, und die
Unterscheidung, die man mitnehmen muss, steht dort - nur die PR-Pipeline kann
einen Merge aufhalten. Die Analyse meldet einen Zustand, der schon gemergt ist.

Der Abschnitt zum Nachweis des Publish-Wegs nennt eine der drei offenen Fragen
nicht mehr als diesem Weg allein gehoerend: Ob eine Repository-Variable in
einem aus einem fremden Repository aufgerufenen Workflow ankommt, beantwortet
der erste Sonar-Lauf fuer beide.

Was auf der Seite des Bausteins steht - warum fetch-depth: 0 nicht optional
ist, warum kein Docker-Runner noetig ist, was der Scanner zur Laufzeit holt -
ist bewusst nicht wiederholt, sondern verlinkt. Zwei Kopien einer Begruendung
heisst, dass eine davon irgendwann falsch ist, und die Datei gehoert uns nicht.

Zum SONAR_TOKEN festgehalten, dass es vorhanden ist und dass "vorhanden" hier
die Bestaetigung des Betreuers meint und keine Pruefung durch ein Werkzeug:
Secrets und Variablen sind von aussen nicht lesbar, tea antwortet mit
"user should be the owner of the repo".
jklein committed 2026-10-01 23:50:05 +00:00
1 parent cf31f0e2c5
commit eef939472e
1 file changed
+74 -1
Download Patch File Download Diff File Expand all files Collapse all files
CI-CD.md
+74 -1
@@ -1,6 +1,8 @@
# CI/CD
Two pipelines: one that **validates** a pull request, one that **publishes** a chart after the merge. Both run on Gitea Actions and read the files under `.github/workflows/` unchanged. Next to them one manually dispatched run, the [all-charts install-test](#the-all-charts-install-test-on-demand), which asks a question no pull request asks: does the environment under an *unchanged* chart still hold.
Two pipelines: one that **validates** a pull request, one that **publishes** a chart after the merge. Both run on Gitea Actions and read the files under `.github/workflows/` unchanged. Next to them two runs that no pull request starts: the [all-charts install-test](#the-all-charts-install-test-on-demand), dispatched by hand, which asks whether the environment under an *unchanged* chart still holds — and the [SonarQube analysis](#sonarqube-analysis), which runs on every push to `main` and **blocks nothing**.
Which of the four can stop a merge is the distinction to carry into every section below: only the PR pipeline can. The publish runs after the decision, the all-charts install-test runs when someone asks, and the analysis reports a state that is already merged.
## Publishing
@@ -64,6 +66,8 @@ The end-to-end check is a single manual `workflow_dispatch` of any `publish-<cha
Three things only that run can prove, and they are the whole remaining risk of the move: that a cross-repository `uses:` resolves on this instance, that a repository variable reaches a workflow called from another repository, and that the upload itself still lands.
The middle one is no longer exclusive to this route: the [SonarQube analysis](#sonarqube-analysis) reads `SONAR_HOST_URL` from `vars` inside a block in that same shared repository, so its first run answers the question for both. The other two stay with the dispatch.
> **The registry accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact. A republish of an existing version is therefore still silent.
## The PR pipeline — `validate-charts.yml`
@@ -180,6 +184,75 @@ Not on a schedule: it would spend compute every day on a question that changes r
| **The record** | per chart: outcome, date, runner label, `K3S_IMAGE`, `kubectl` version. Plus a final job with the verdict over the whole matrix. A green run nobody notes is unknown again next time. |
| **Expected runtime** | **20 charts in 12m41s** (Gitea PR #48, one job, all legs concurrent on one Docker daemon). Written down so a later deviation is visible as one; compare it against the chart count the run prints in its own summary — 24 at the time of writing, and one more with every new chart. The timeout is 30 minutes per leg, in `install-test.yml`, hence identical for both callers. **A big image moves this less than it looks:** [convertx](convertx) is ~1.4 GiB compressed, by a wide margin the largest in the repo, and its first green leg measured **2m35s** including cluster creation (Gitea PR #61) — inside the existing envelope, not a new one. The legs run concurrently, so this figure tracks the *slowest* leg plus scheduling, never the sum of the pulls. |
## SonarQube analysis
`sonar.yml` is a hull of the same kind as the publish ones: it decides *when*, and the workflow itself is the shared block [`sonar-analysis`](https://git.jk-dev.de/workflow-util/pipelines/wiki/sonar-analysis) in `workflow-util/pipelines`, bound to a tag. The project-specific half is `sonar-project.properties` in the repo root. Why `fetch-depth: 0` is not optional, why the block needs no Docker runner, and what the scanner downloads at run time are on that page and deliberately not repeated here.
```yaml
on:
push:
branches: [main]
workflow_dispatch:
jobs:
sonar:
uses: workflow-util/pipelines/.gitea/workflows/sonar-analysis.yml@v4
secrets: inherit
```
### It observes, it does not gate
The Community Build analyses **the main branch only** — there is no pull-request analysis to hook into. So the trigger is a push to `main`, and **a finding appears after the merge**. At that point there is nothing left to block, which is why the block carries no quality-gate step.
That is not a gap waiting to be filled. A red gate behind the merge would colour `main` red without having prevented anything, and a permanent red is a thing people learn to ignore — at which point it has spent the credibility of `validate-charts`, the one check here that *can* stop a merge. It becomes the right thing under one of two conditions, and both change the situation rather than the wiring: pull-request analysis is available, or a deployment hangs off the workflow.
Practical consequence for whoever reads a chart PR: the analysis will never be in its check list. The findings are in SonarQube, about `main`, and they are read there.
### What is analysed
The languages were checked against SonarQube's list, not assumed — the fit turned out better than expected, Shell included.
| What is in this repository | Analyser |
| --- | --- |
| the 5 shell scripts under `ci/` — the whole CI logic | Shell |
| ~30 workflow files | GitHub Actions |
| 238 YAML files and 26 Helm template directories | Kubernetes/Helm, YAML |
| 26 `values.schema.json` | JSON |
| the whole tree | Secrets |
**The last row is the reason to bother.** The premise of this repository is that a credential never lands in it — every secret is a 1Password `OnePasswordItem` reference, materialised by the operator in the cluster — and until now *nothing checked that premise*. Every other row is a second opinion on something `helm lint --strict`, the render test or Trivy already touch; this one covers an assumption that was carried for two years on discipline alone.
### No exclusions, for the first round
Including the Helm templates, which are not valid YAML because of `{{ }}`.
What the Helm and YAML analysers make of such a file is only learnable by letting them run over one. Excluding them up front would discard that answer and, with it, the bulk of the repository — and it would be an exclusion written on a guess, which hides whatever else happens to match it. There is no `paths` filter on the hull either, for the matching reason: a filter would leave the previous result standing after a change it did not cover, while still presenting it as current.
If the first run produces noise genuinely not worth reading, the exclusions go into `sonar-project.properties` **with the finding they answer named next to them**. That file is ours; the shared block never carries an exclusion for one repository.
### Prerequisites, and what a missing one looks like
| Name | Art | |
| --- | --- | --- |
| `SONAR_HOST_URL` | repository variable | Set on the instance. A **variable and not a secret** on purpose: a server address is not a credential, and as a variable it stays readable in the log — which is the only thing that makes a wrong address diagnosable. Masked, it would produce a failure against an address nobody can see, the exact situation that forced an extra masking step into the publish block. |
| `SONAR_TOKEN` | repository secret | Confirmed present by the maintainer. Reaches the block through `secrets: inherit`. Needs the right to run analyses in SonarQube, and the right to create a project if the first run is to create it. |
| `sonar-project.properties` | file in the repo root | Project key `jlab-cluster_helm-charts`. The key carries the owner because the instance serves two (`jlab-cluster` and `suer`); a bare `helm-charts` is free today and collides the day the second owner has a repository of that name — and a key cannot be changed afterwards without losing the project's history. |
Secrets and variables are not readable from outside, including by the bot: `tea actions secrets ls` answers `user should be the owner of the repo`, and variables are not listed at all. So "set" here means confirmed by the maintainer, not verified by a tool — and the block's guard is what turns a wrong assumption into a named error instead of a puzzling failure.
The guard covers an empty variable, an empty secret and a forgotten `secrets: inherit`, and it checks both values before it exits, so a repository that has set up neither learns both in one run. What it cannot cover is a **missing** required secret: that fails job **creation** on this instance, not the job, and the run then shows no steps at all — the same quirk as everywhere else here, and the understandable case for this block, because without a token there is no analysis to run.
### Three things the first run answers
None of these is measured, and the merge of the change that added the hull is itself the first run — the trigger is a push to `main`, and a merge is such a push. Unlike the [publish route](#proving-the-path-after-a-change-to-the-publishing-route), which needs a manual dispatch because it touches no chart directory, this one proves itself on arrival.
1. **Is `node24` in the runner image?** The scan action declares `runs: using: node24` and is therefore a JavaScript action. If it is absent, the failure will read as a problem with the action, not with the image — which is the whole reason this is written down.
2. **Does the scanner download get through?** The action fetches the Sonar Scanner CLI from `binaries.sonarsource.com` at run time and checks its GPG signature; measured at ~59 MiB. This is the likeliest first failure, and it has history: a 17 MB download once failed on this runner because a Docker bridge at MTU 1500 met a pod network of 1450 — see [What a GitHub runner had been providing quietly](#what-a-github-runner-had-been-providing-quietly). That was fixed on the node side, and the scanner is a multiple of it.
3. **Does a repository variable reach a workflow called from another repository?** The same open question the publish route still carries for `OP_CONNECT_HOST`, and this run answers it for both. If it does not arrive, the answer is loud: the guard fires and names the variable. The repair would be the one already made for the owner segment over there — pass the address in as an input, evaluated in the caller.
Plus the one that is not a yes/no: **what the analysis actually reports**, in particular out of the Helm templates. That is what decides whether exclusions are needed, and it belongs in `sonar-project.properties` when it does.
`workflow_dispatch` is on the hull for the retries these may need. Without it every attempt would cost a dummy commit on `main`. Only the maintainer can trigger it — the bot account cannot dispatch workflows.
## `ci/helm-dependencies.sh` — the step they all share
Not a workflow but the prerequisite of every helm command in this repo: it fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one).