diff --git a/CI-CD.md b/CI-CD.md index 1172c8c..c492952 100644 --- a/CI-CD.md +++ b/CI-CD.md @@ -1,6 +1,8 @@ # CI/CD -Two pipelines: one that **validates** a pull request, one that **publishes** a chart after the merge. Both run on Gitea Actions and read the files under `.github/workflows/` unchanged. Next to them one manually dispatched run, the [all-charts install-test](#the-all-charts-install-test-on-demand), which asks a question no pull request asks: does the environment under an *unchanged* chart still hold. +Two pipelines: one that **validates** a pull request, one that **publishes** a chart after the merge. Both run on Gitea Actions and read the files under `.github/workflows/` unchanged. Next to them two runs that no pull request starts: the [all-charts install-test](#the-all-charts-install-test-on-demand), dispatched by hand, which asks whether the environment under an *unchanged* chart still holds — and the [SonarQube analysis](#sonarqube-analysis), which runs on every push to `main` and **blocks nothing**. + +Which of the four can stop a merge is the distinction to carry into every section below: only the PR pipeline can. The publish runs after the decision, the all-charts install-test runs when someone asks, and the analysis reports a state that is already merged. ## Publishing @@ -64,6 +66,8 @@ The end-to-end check is a single manual `workflow_dispatch` of any `publish- **The registry accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact. A republish of an existing version is therefore still silent. ## The PR pipeline — `validate-charts.yml` @@ -180,6 +184,75 @@ Not on a schedule: it would spend compute every day on a question that changes r | **The record** | per chart: outcome, date, runner label, `K3S_IMAGE`, `kubectl` version. Plus a final job with the verdict over the whole matrix. A green run nobody notes is unknown again next time. | | **Expected runtime** | **20 charts in 12m41s** (Gitea PR #48, one job, all legs concurrent on one Docker daemon). Written down so a later deviation is visible as one; compare it against the chart count the run prints in its own summary — 24 at the time of writing, and one more with every new chart. The timeout is 30 minutes per leg, in `install-test.yml`, hence identical for both callers. **A big image moves this less than it looks:** [convertx](convertx) is ~1.4 GiB compressed, by a wide margin the largest in the repo, and its first green leg measured **2m35s** including cluster creation (Gitea PR #61) — inside the existing envelope, not a new one. The legs run concurrently, so this figure tracks the *slowest* leg plus scheduling, never the sum of the pulls. | +## SonarQube analysis + +`sonar.yml` is a hull of the same kind as the publish ones: it decides *when*, and the workflow itself is the shared block [`sonar-analysis`](https://git.jk-dev.de/workflow-util/pipelines/wiki/sonar-analysis) in `workflow-util/pipelines`, bound to a tag. The project-specific half is `sonar-project.properties` in the repo root. Why `fetch-depth: 0` is not optional, why the block needs no Docker runner, and what the scanner downloads at run time are on that page and deliberately not repeated here. + +```yaml +on: + push: + branches: [main] + workflow_dispatch: +jobs: + sonar: + uses: workflow-util/pipelines/.gitea/workflows/sonar-analysis.yml@v4 + secrets: inherit +``` + +### It observes, it does not gate + +The Community Build analyses **the main branch only** — there is no pull-request analysis to hook into. So the trigger is a push to `main`, and **a finding appears after the merge**. At that point there is nothing left to block, which is why the block carries no quality-gate step. + +That is not a gap waiting to be filled. A red gate behind the merge would colour `main` red without having prevented anything, and a permanent red is a thing people learn to ignore — at which point it has spent the credibility of `validate-charts`, the one check here that *can* stop a merge. It becomes the right thing under one of two conditions, and both change the situation rather than the wiring: pull-request analysis is available, or a deployment hangs off the workflow. + +Practical consequence for whoever reads a chart PR: the analysis will never be in its check list. The findings are in SonarQube, about `main`, and they are read there. + +### What is analysed + +The languages were checked against SonarQube's list, not assumed — the fit turned out better than expected, Shell included. + +| What is in this repository | Analyser | +| --- | --- | +| the 5 shell scripts under `ci/` — the whole CI logic | Shell | +| ~30 workflow files | GitHub Actions | +| 238 YAML files and 26 Helm template directories | Kubernetes/Helm, YAML | +| 26 `values.schema.json` | JSON | +| the whole tree | Secrets | + +**The last row is the reason to bother.** The premise of this repository is that a credential never lands in it — every secret is a 1Password `OnePasswordItem` reference, materialised by the operator in the cluster — and until now *nothing checked that premise*. Every other row is a second opinion on something `helm lint --strict`, the render test or Trivy already touch; this one covers an assumption that was carried for two years on discipline alone. + +### No exclusions, for the first round + +Including the Helm templates, which are not valid YAML because of `{{ }}`. + +What the Helm and YAML analysers make of such a file is only learnable by letting them run over one. Excluding them up front would discard that answer and, with it, the bulk of the repository — and it would be an exclusion written on a guess, which hides whatever else happens to match it. There is no `paths` filter on the hull either, for the matching reason: a filter would leave the previous result standing after a change it did not cover, while still presenting it as current. + +If the first run produces noise genuinely not worth reading, the exclusions go into `sonar-project.properties` **with the finding they answer named next to them**. That file is ours; the shared block never carries an exclusion for one repository. + +### Prerequisites, and what a missing one looks like + +| Name | Art | | +| --- | --- | --- | +| `SONAR_HOST_URL` | repository variable | Set on the instance. A **variable and not a secret** on purpose: a server address is not a credential, and as a variable it stays readable in the log — which is the only thing that makes a wrong address diagnosable. Masked, it would produce a failure against an address nobody can see, the exact situation that forced an extra masking step into the publish block. | +| `SONAR_TOKEN` | repository secret | Confirmed present by the maintainer. Reaches the block through `secrets: inherit`. Needs the right to run analyses in SonarQube, and the right to create a project if the first run is to create it. | +| `sonar-project.properties` | file in the repo root | Project key `jlab-cluster_helm-charts`. The key carries the owner because the instance serves two (`jlab-cluster` and `suer`); a bare `helm-charts` is free today and collides the day the second owner has a repository of that name — and a key cannot be changed afterwards without losing the project's history. | + +Secrets and variables are not readable from outside, including by the bot: `tea actions secrets ls` answers `user should be the owner of the repo`, and variables are not listed at all. So "set" here means confirmed by the maintainer, not verified by a tool — and the block's guard is what turns a wrong assumption into a named error instead of a puzzling failure. + +The guard covers an empty variable, an empty secret and a forgotten `secrets: inherit`, and it checks both values before it exits, so a repository that has set up neither learns both in one run. What it cannot cover is a **missing** required secret: that fails job **creation** on this instance, not the job, and the run then shows no steps at all — the same quirk as everywhere else here, and the understandable case for this block, because without a token there is no analysis to run. + +### Three things the first run answers + +None of these is measured, and the merge of the change that added the hull is itself the first run — the trigger is a push to `main`, and a merge is such a push. Unlike the [publish route](#proving-the-path-after-a-change-to-the-publishing-route), which needs a manual dispatch because it touches no chart directory, this one proves itself on arrival. + +1. **Is `node24` in the runner image?** The scan action declares `runs: using: node24` and is therefore a JavaScript action. If it is absent, the failure will read as a problem with the action, not with the image — which is the whole reason this is written down. +2. **Does the scanner download get through?** The action fetches the Sonar Scanner CLI from `binaries.sonarsource.com` at run time and checks its GPG signature; measured at ~59 MiB. This is the likeliest first failure, and it has history: a 17 MB download once failed on this runner because a Docker bridge at MTU 1500 met a pod network of 1450 — see [What a GitHub runner had been providing quietly](#what-a-github-runner-had-been-providing-quietly). That was fixed on the node side, and the scanner is a multiple of it. +3. **Does a repository variable reach a workflow called from another repository?** The same open question the publish route still carries for `OP_CONNECT_HOST`, and this run answers it for both. If it does not arrive, the answer is loud: the guard fires and names the variable. The repair would be the one already made for the owner segment over there — pass the address in as an input, evaluated in the caller. + +Plus the one that is not a yes/no: **what the analysis actually reports**, in particular out of the Helm templates. That is what decides whether exclusions are needed, and it belongs in `sonar-project.properties` when it does. + +`workflow_dispatch` is on the hull for the retries these may need. Without it every attempt would cost a dummy commit on `main`. Only the maintainer can trigger it — the bot account cannot dispatch workflows. + ## `ci/helm-dependencies.sh` — the step they all share Not a workflow but the prerequisite of every helm command in this repo: it fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one).