4 Commits
Author SHA1 Message Date
jlab-renovate-bot 1360553ccb Update dependency kubernetes/kubernetes to v1.37.1
Validate Helm Charts / Detect changed charts (pull_request) Successful in 5s
Validate Helm Charts / Build ${{ matrix.chart }} (pull_request) Skipped
Validate Helm Charts / Trivy config ${{ matrix.chart }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
renovate/stability-days Updates have met minimum release age requirement
Validate Helm Charts / Resource quantities (pull_request) Successful in 16s
Validate Helm Charts / Selector match (pull_request) Successful in 14s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 14s
Validate Helm Charts / Bump level (pull_request) Successful in 7s
Validate Helm Charts / YAML scalars (pull_request) Successful in 13s
2026-10-13 19:35:43 +00:00
jklein 39844fc92c ci: hold every scalar that must be a string, and the required podAffinity
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 15s
Validate Helm Charts / Build paperless-ngx (pull_request) Successful in 9s
Validate Helm Charts / Resource quantities (pull_request) Successful in 15s
Validate Helm Charts / Selector match (pull_request) Successful in 14s
Validate Helm Charts / Bump level (pull_request) Successful in 7s
Validate Helm Charts / YAML scalars (pull_request) Successful in 15s
Validate Helm Charts / Trivy config paperless-ngx (pull_request) Successful in 29s
Install-test / Install-test paperless-ngx (pull_request) Successful in 5m11s
Validate Helm Charts / Install-test (pull_request) Successful in 5m11s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 16s
Publish Paperless-NGX Helm Chart / Publish Paperless-NGX Helm Chart (push) Successful in 16s
SonarQube Analysis / SonarQube Analysis (push) Successful in 47s
The test Gitea #11 asked for ("whoever fixes this ships a test case that would
have caught it"), plus the guard for the fix #101 introduces.

ci/run-yaml-scalar-render-test.sh renders every chart with the fixtures the
install-test already uses and asserts three properties Kubernetes itself
requires, so no per-chart expectations are needed: a CronJob schedule is a
non-empty string, every env[].value of every container and initContainer is a
string, and every label and annotation value is a string. The first catches the
loud failure - the render aborting with a YAML error that names the template
and not the value - and the other two catch the quiet one, where the render
succeeds and the API server rejects the object at apply time.

WHY A NEW JOB AND NOT THE INSTALL-TEST. The object that carried the defect is
rendered by no fixture at all: paperless-ngx' consume CronJob sits behind
smbConnection.enabled, false in the chart defaults and false in its install-test
fixture, since the CI cluster has no SMB CSI driver. Nothing in this repository
ever produced it. So the overlays under ci/yaml-scalar/ are the test, not the
scaffolding: they switch such objects on AND feed them hostile values, and the
script fails when an overlay stops reaching its object or when the run asserted
nothing at all.

The fail path was exercised in all three shapes against a throwaway copy of the
chart as it stands on main, not reasoned about: the bare schedule aborted with
"yaml: line 6: did not find expected alphabetic or numeric character"; with only
the schedule quoted, the star-valued env entry aborted on the StatefulSet
instead; and with only the bool-looking value left - a render that succeeds -
the type assertion named the variable, the container and the API server's
message. Green repo-wide otherwise: 30 charts, 408 scalars, 0 failed.

ci/run-selector-match-test.sh additionally holds every
requiredDuringSchedulingIgnoredDuringExecution podAffinity term against all pod
templates of the same render (Gitea #101). That case is not the gap the script's
header describes: a preference that cannot be met is dropped and an
anti-affinity over zero pods is satisfied, but a REQUIRED term matching no pod
can never be satisfied and the pod stays Pending for good. paperless-ngx' mover
selects a label written in another template, which is exactly the copy this test
holds - renaming it would leave both templates rendering perfectly while the
mover silently stopped moving. Exercised by doing that rename in a throwaway
copy: one FAIL naming the selector, the pod templates it missed and the labels
actually present.

Six charts still render their env block bare (matrix-synapse, minecraft-router,
minecraft-server, rallly, satisfactory, teamspeak-server). They are green only
because no fixture of theirs sets a hostile value; README now says so instead of
repeating template-chart's claim that every chart quotes.

Part of #11
Part of #101
2026-10-13 19:10:26 +00:00
jklein 4b2bde505d feat(paperless-ngx)!: quote every operator scalar, pin the consume mover, drop oidcSecretRef
Three issues on one chart, bundled because they share values.yaml,
values.schema.json and the version line, and because the first half of #11
lives in the same template as #101.

Gitea #11, first half: the consume CronJob's schedule was rendered bare, so a
schedule in the commonest cron notation there is - a star, a slash, an interval
- aborted the render with a YAML parse error pointing at the template while the
operator's value appeared nowhere in the message. Quoting it in the values file
does not help; Helm passes the content through. Reproduced before the fix, at
the line the issue names, and fixed with | quote.

Gitea #11, second half: secrets.oidcSecretRef is removed, values and schema
both. No template ever read it, so no OnePasswordItem was created and nothing
reached the pod - the quiet kind of defect that looks like successful
configuration. It is not reimplemented, and the reason is in paperless: the
whole OIDC configuration travels in one json environment variable with no _FILE
variant, so a secretKeyRef cannot substitute a fragment into it. The
operational question is answered by #83 - Keycloak marks a client public per
client, so nothing forces a confidential one here.

In the same breath, oidc.provider.tokenAuthMethod is restricted to "none". Its
comment named client_secret_basic as the example, which asks the provider for
an authentication this chart cannot perform: it configures no client secret and
cannot. The two values read together looked like a working secret setup, and the
symptom was a login dying at the callback with invalid_client. No live release
sets it (measured against all three instances).

Gitea #112 asked for an extraEnv and the chart already had the standard
additive env block - the issue was measured against a deployed 12.2.0 and read
only the top-level keys. What was actually broken is the block itself: it
rendered its operand bare, so a "*"-leading value aborted the render and "true"
or "0755" arrived as a bool or an integer, which the API server rejects after a
green render. It is brought to the repository's documented quoted form
(template-chart carries the rule), which is what makes an arbitrary PAPERLESS_*
variable settable in practice. PAPERLESS_LOGGING_DIR is documented as the
worked example, with the cross-reference it needs: the root cause is the
NFS-exported RWX data volume in jk-cluster-config #360 and this does not fix it
- it only moves the log off that volume. The two Ingress upload-size
annotations had the same bare render and are quoted too.

Gitea #101: the consume mover now runs on the node of the StatefulSet pod, via
a required podAffinity on its label. It mounts the RWX consume volume, and
Longhorn's CSI driver refuses that mount unless it finds a workload pod in
Pending, so a mover on another node left the StatefulSet node's kubelet
retrying every two minutes - 291 events in 10 hours in both releases with SMB
on, none in the release without the CronJob. A static nodeSelector would be
wrong because the StatefulSet pod is not pinned either, and preferred would
reproduce the bug more rarely and harder to find.

The cost of a required term is handled rather than left open:
concurrencyPolicy: Forbid, a startingDeadlineSeconds that also keeps the
controller away from its 100-missed-schedules stop, and an activeDeadlineSeconds
that fails a job which cannot be placed instead of leaving one Pending pod per
schedule. scaleDown already suspends the CronJob; what remains is
paperless.replicas: 0 with SMB on, which now reports a bounded, visible failure.
successfulJobsHistoryLimit is deliberately untouched: Longhorn's check asks for
the existence of a Pending pod, not the absence of Succeeded ones, so those
three pods are the error message's evidence and not its cause.

Side defect fixed on the way: every read of smbConnection.cron dereferenced the
block directly, so an override that had erased it aborted with a nil-pointer
error naming an internal template path instead of falling back to the defaults
this chart promises everywhere else. Measured on origin/main; all reads now go
through a merged-defaults helper that preserves an explicit false.

Major: a value is removed and fails the strict schema afterwards, and runtime
placement changes.

Fixes #11
Fixes #101
Fixes #112
2026-10-13 19:09:45 +00:00
jklein 59d2614384 feat(ci): the chart bump follows the app bump
Validate Helm Charts / Detect changed charts (pull_request) Successful in 5s
Validate Helm Charts / Build ${{ matrix.chart }} (pull_request) Skipped
Validate Helm Charts / Install-test (pull_request) Skipped
Validate Helm Charts / Resource quantities (pull_request) Successful in 18s
Validate Helm Charts / Selector match (pull_request) Successful in 14s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 14s
Validate Helm Charts / Bump level (pull_request) Successful in 7s
Validate Helm Charts / Trivy config cyberchef (pull_request) Successful in 20s
Validate Helm Charts / Trivy config matrix-synapse (pull_request) Successful in 22s
Validate Helm Charts / Trivy config samba (pull_request) Successful in 21s
SonarQube Analysis / SonarQube Analysis (push) Successful in 48s
The version-bump hook classified every app update that was not a major
as a chart patch, so an app minor arrived as a patch - the mechanism
behind the sonarr incident, where a security default was switched off
inside a bump the conventions told the reviewer not to read (Gitea #128).

ci/renovate-bump-chart.sh now takes the level from Renovate's update
type, passed in as a second argument: major -> major, minor -> MINOR,
patch/digest/pin -> patch. The level comes from Renovate rather than
from comparing appVersion because an appVersion comparison is blind to
the case that prompted the rule - freesailarr keeps a hand-maintained
appVersion for ten upstreams, and prowlarr's 2.5.2 -> 2.6.5 moved an
image tag in values.yaml while appVersion stood still.

Three properties come with it:

- Maximum, not replacement. The chart jump is the greater of the
  chart's own classification and the app jump, so a value that becomes
  required stays a major under an app patch. Mechanically it is also
  what makes a grouped branch deterministic: the hook runs once per
  update from the same base commit, so without it the last call would
  win.
- Non-semver app versions stay at patch for a minor (seven charts). In
  CalVer and date tags the second position is a month; in a four-part
  or suffixed version the positions are not semver positions. An app
  major is not subject to it.
- A missing or unknown update type falls back to the pre-#128
  behaviour. postUpgradeTasks only runs where the command is
  allow-listed on the shared instance, which is not visible from here
  and fails silently, so the worst case is the old classification
  rather than every chart PR red on the version check.

ci/run-bump-level-test.sh holds 49 cases against the version string the
script writes, wired up as the bump-level job: a wrong level is a valid
version string, so the PR would go green and the only symptom would be
a number that understates what it publishes.

The convention texts follow in CLAUDE.md and README, including the
limit the decision deliberately keeps: the reading duty was rejected,
so the incident stays repeatable and the honest number is the whole of
the remedy.

Fixes #128
2026-10-13 18:47:44 +00:00
18 changed files with 1596 additions and 105 deletions

No files matched your search

+66
View File
@@ -237,6 +237,72 @@ jobs:
- name: "Check every rendered HTTPRoute against the rendered Services"
run: ci/run-httproute-match-test.sh
bump-level:
name: "Bump level"
runs-on: ubuntu-latest
# The fourth job that ignores the changed-chart matrix, and the only one
# that touches no chart at all: it tests ci/renovate-bump-chart.sh, the
# postUpgradeTasks hook that computes a chart's `version:` after Renovate
# moved an app version (Gitea #128).
#
# Why it is a job rather than a review: since #128 the hook takes the bump
# level from Renovate's update type, so an app minor becomes a chart minor.
# That makes it deciding logic, and its failure mode is silence - a wrong
# level is a VALID version string, so the PR goes green, the chart
# publishes, and the only symptom is a number that understates what it
# ships. Nothing else in this pipeline reads that number.
#
# It needs no Helm, no cluster and no Python: each case is a throwaway
# `git init` under mktemp with one commit plus a dirty Chart.yaml - the
# state the hook really runs in, since it reads the base commit - and the
# assertion is one `version:` line read back with sed. About a second for
# all 49 cases, and it depends on no other job.
steps:
- name: "Checkout Repository"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: "Check every bump level of the version-bump hook"
run: ci/run-bump-level-test.sh
yaml-scalar:
name: "YAML scalars"
runs-on: ubuntu-latest
# The fifth job that ignores the changed-chart matrix, and the fourth that
# runs over *every* chart (Gitea #11), for the same reason as the three
# render tests above: the unquoted
# form arrives by copy into a chart whose own diff nobody reads for it, and
# it is in seven charts right now.
#
# What it asserts: every CronJob schedule, every env[].value and every label
# and annotation value a chart renders is a STRING, which is what Kubernetes
# types all three as. A value rendered without | quote fails in one of two
# ways and the quiet one is the dangerous one: the render aborts with a YAML
# error naming the TEMPLATE while the operator's value appears nowhere in
# it, or - when the content is a legal plain scalar that simply is not a
# string - the render succeeds and the API server rejects the object at
# apply time with "cannot unmarshal bool into Go struct field
# EnvVar.value of type string".
#
# WHY THIS JOB EXISTS AT ALL, rather than the install-test covering it: the
# object that carried the defect is rendered by no fixture. paperless-ngx'
# consume CronJob sits behind smbConnection.enabled, false in the defaults
# and in every committed fixture, so nothing in this repository ever
# produced it - which is how an unquoted schedule shipped for months. The
# overlays under ci/yaml-scalar/ switch such objects on and feed them
# hostile values, and the script fails when an overlay stops reaching its
# object.
#
# No cluster and no Trivy; the whole repository in well under a minute,
# depends on no other job. The YAML is parsed by Helm through ci/yaml2json,
# because the runners have neither pyyaml nor yq and a hand-written parser
# could read nothing and report nothing - see the script's header.
steps:
- name: "Checkout Repository"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
- name: "Install Helm"
uses: azure/setup-helm@9bc31f4ebc9c6b171d7bfbaa5d006ae7abdb4310 # v5
- name: "Check every rendered scalar that must be a string"
run: ci/run-yaml-scalar-render-test.sh
trivy-config:
name: "Trivy config ${{ matrix.chart }}"
needs: detect-changed-charts
+8 -6
View File
@@ -48,7 +48,7 @@ Flat layout, one Markdown file per page, page name = file name, links are relati
| a Renovate rule added or changed | `Renovate` |
| something broke in a way a rule now prevents | `Lessons-Learned` |
**What does not oblige an edit:** a plain app-version bump. Renovate automerges those continuously and never touches the wiki, which is why chart pages carry no version numbers at all and name `Chart.yaml` as the source of truth. An app update that *does* change values or behaviour is not a plain bump, and the table applies.
**What does not oblige an edit:** an app-version bump, at any level. Renovate automerges those continuously and never touches the wiki, which is why chart pages carry no version numbers at all and name `Chart.yaml` as the source of truth. Since Gitea #128 **"plain" is mechanically defined instead of being a judgement**: the chart version now states the size of the jump, because `ci/renovate-bump-chart.sh` takes that size from Renovate's update type (app minor → chart minor). An app update that *does* change values or behaviour is not a plain bump and the table applies — but **nothing obliges anyone to go and find out**. That is the deliberate limit of #128, not an oversight: the reading duty was rejected, so the `sonarr` incident in `Lessons-Learned` stays repeatable and a bigger number is the only warning that is given.
Wiki commits follow the same rules as repo commits: no `Co-Authored-By`, no "Generated with Claude Code" footer.
@@ -94,11 +94,13 @@ Copy `template-chart/` and replace every `xxx` placeholder — in file names (`t
`Chart.yaml` versions follow `version: <chart-semver>+up<appVersion>` (e.g. `1.4.0+up1.9.2`), with `appVersion` matching the suffix (dashes in the app version are normalized to dots in the suffix). Any push to `main` touching a chart directory publishes that chart, so the version must be bumped in the same change.
Semver conventions for the chart part:
Semver conventions for the chart part. **The chart jump is the greater of the chart's own classification and the jump of what it ships** (Gitea #128) — a maximum, never a replacement, so a value that becomes required stays a major even under an app patch:
- **Patch**: bug fixes without behavior or values-schema change; plain app-version updates.
- **Minor**: additive, backward-compatible changes (new optional values with unchanged defaults).
- **Major**: runtime behavior changes or breaking values changes (e.g. probes, security hardening, computed limits, values that become required).
- **Patch**: bug fixes without behavior or values-schema change; an app **patch** or digest update.
- **Minor**: additive, backward-compatible changes (new optional values with unchanged defaults); an app **minor** update.
- **Major**: runtime behavior changes or breaking values changes (e.g. probes, security hardening, computed limits, values that become required); an app **major** update.
The app half of that is mechanical: `ci/renovate-bump-chart.sh` takes the level from **Renovate's update type**, passed in as `{{{updateType}}}`, and not from comparing `appVersion` — `freesailarr` pins every tag in `values.yaml` and its `appVersion` never moves, so a comparison is blind to exactly the case the rule exists for. Non-semver app versions (CalVer, date tags, four-part and suffixed tags — seven charts) stay at **patch** for a minor, because their second position is a month or a flavour and says nothing about the jump; an app **major** still produces a chart major. **It obliges nobody to read the release notes** — that variant was rejected in #128 — so an app minor that changes a default can still reach `main` unremarked; the number stops understating, that is all it does. See the wiki, `Lessons-Learned`.
App-version updates go through PRs (so the validate and install-test pipelines run), not through the `change-app-version.yml` dispatch workflow (which commits directly to main and bypasses both). For app **major** jumps, update stepwise: first to the last release of the old major line (own PR, merge, publish), then to the newest major.
@@ -117,4 +119,4 @@ The binding style guide for all current and future charts lives in **README.md (
- Resources: requests as values defaults; limits auto-computed by the `_helpers.tpl` helper (`limitFactor` default 3; no CPU limits unless set explicitly).
- Replicas per workload (explicit `0` allowed) plus a chart-global `scaleDown` flag that overrides them.
- `icon:` mandatory in every `Chart.yaml`; `helm lint --strict` without findings; CI fixtures under `ci/<chart>/` or a justified exclusion.
- **Chart dependencies are admissible but expensive** (the former "no dependencies" rule fell with #214; `ytdlp-web-player` was the first to use one, `stalwart` the second, `paperless-ngx` the third). A dependency has to be optional and off by default via `condition:`, and upstream's chart is consumed as it comes — where its defaults fall short of these conventions, that is **documented in our `values.yaml`, never patched**. The price to know before adding the next one: a subchart is invisible to everything downstream that watches Helm release versions, so a new upstream version — a security fix included — reaches a cluster only when the outer chart is republished (Renovate's built-in `helmv3` manager plus `ci/renovate-bump-chart.sh` is the whole safety net, and it belongs in a comment at the pinned version); and every helm command exits 1 while `charts/` is unfilled, which is why `ci/helm-dependencies.sh` runs before lint, package, render and install — including the render test that covers **all** charts on **every** PR. Full rules and the rest of the cost: README, "Chart dependencies".
- **Chart dependencies are admissible but expensive** (the former "no dependencies" rule fell with #214; `ytdlp-web-player` was the first to use one, `stalwart` the second, `paperless-ngx` the third). A dependency has to be optional and off by default via `condition:`, and upstream's chart is consumed as it comes — where its defaults fall short of these conventions, that is **documented in our `values.yaml`, never patched**. The price to know before adding the next one: a subchart is invisible to everything downstream that watches Helm release versions, so a new upstream version — a security fix included — reaches a cluster only when the outer chart is republished (Renovate's built-in `helmv3` manager plus `ci/renovate-bump-chart.sh` is the whole safety net, and since #128 that republish carries the **subchart's own level** — an upstream minor is a chart minor, which is the only place that jump becomes visible at all; it belongs in a comment at the pinned version); and every helm command exits 1 while `charts/` is unfilled, which is why `ci/helm-dependencies.sh` runs before lint, package, render and install — including the render test that covers **all** charts on **every** PR. Full rules and the rest of the cost: README, "Chart dependencies".
+62 -16
View File
@@ -68,8 +68,8 @@ Every chart has a wiki page of its own — `https://git.jk-dev.de/jlab-cluster/h
Note: the registry **accepts a re-upload of an already published version** (HTTP 201) and replaces the stored artifact, so a republish of an existing version is still silent — see #160. That is also what makes a manual `workflow_dispatch` of any `publish-<chart>.yml` a harmless way to prove the shared block works.
- `change-app-version.yml` — manually dispatched app-version bump; the chart must be listed in its `chart-name` choice list.
- `ci/helm-dependencies.sh <chart-dir>` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but three: `ytdlp-web-player`, `stalwart`, `paperless-ngx`). It runs in **seven** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in the shared publish block (which is why that block names this script as a prerequisite of its callers — see above), and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh`, `ci/run-selector-match-test.sh`, `ci/run-httproute-match-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs.
- `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (the same changed-charts matrix, minus the charts listed in `ci/excluded-charts.txt`) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix, or run the all-charts install-test after the merge. The **resource-quantity render test**, the **selector-match render test** and the **HTTPRoute render test** are the three exceptions to that rule: all of them run over every chart on every PR — see below.
- `ci/helm-dependencies.sh <chart-dir>` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but three: `ytdlp-web-player`, `stalwart`, `paperless-ngx`). It runs in **eight** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in the shared publish block (which is why that block names this script as a prerequisite of its callers — see above), and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh`, `ci/run-selector-match-test.sh`, `ci/run-httproute-match-test.sh`, `ci/run-yaml-scalar-render-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs.
- `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (the same changed-charts matrix, minus the charts listed in `ci/excluded-charts.txt`) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix, or run the all-charts install-test after the merge. The **resource-quantity render test**, the **selector-match render test**, the **HTTPRoute render test** and the **YAML-scalar render test** are the four exceptions to that rule: all of them run over every chart on every PR — see below. A fourth job, the **bump-level test**, ignores the matrix for a different reason: it tests no chart at all, but the `postUpgradeTasks` hook that computes their versions.
- `install-test.yml` — reusable workflow (`workflow_call`) holding the install-test **job** itself: the runner-label expression, the pinned `kubectl` with its skew check, `envsubst`, the containerised-job detection, the per-leg k3d cluster and the `freesailarr` VPN credentials. Its one input is a JSON array of chart names; its callers decide *which* charts are tested and whether the job runs at all, never *how*. Two callers: `validate-charts.yml` and `install-test-all-charts.yml`. **It is not copied for the second caller** — a second copy is a second chance to test differently, unnoticed, the same argument that gives `ci/excluded-charts.txt` exactly one parser.
**It stays in this repository, and that is a decision, not an oversight** (Gitea #66, decision 4). It is reusable in form but not in substance: it is welded to the `ci/` fixtures, to the `ci/excluded-charts.txt` parser and to this repository's chart matrix. A building block that only one repository can use is not shared, it is merely moved somewhere else — and moving it would buy nothing while putting a tag between us and a file we change often. `publish-helm-chart.yml` was the opposite case: it takes a chart name and a registry owner and knows nothing about this repository, which is exactly why it could go. This note exists so the question is not asked again in half a year.
@@ -156,10 +156,11 @@ Neither is a render error, neither fails the install-test, and neither shows up
Three things to know before touching it:
- **Helm is the YAML parser, through the one-template chart `ci/yaml2json`.** The runners have neither `pyyaml` nor `yq` (measured; the same constraint `ci/run-trivy-config.sh` and `ci/run-quantity-render-test.sh` work under), and a YAML reader written in `awk` was the obvious alternative and the wrong one: its failure mode is a parser that quietly reads nothing, finds no selector, reports no mismatch and leaves the test green for a chart it never looked at. So the rendered manifest stream is handed to that chart as a **values file** and comes back as JSON, which `python3` reads from the standard library. The one textual step is the document split on a line that is exactly `---` in column one, and getting *that* wrong does not produce a false pass — the fragment stops being valid YAML and `helm` exits 1.
- **What is deliberately not checked, named so the gap is not mistaken for coverage.** `podAffinity`/`podAntiAffinity` are out of scope: their semantics are relative to pods that exist, so a term matching nothing is a legitimate configuration rather than a mistake. And a selector expressed **only** through `matchExpressions` is **reported as "not checked", never counted as passing** — evaluating set-based requirements means reimplementing label-selector semantics, which is a second parser by another name.
- **Two anti-vacuum guards, because this test can be green over nothing.** Both objects are rendered only above a replica threshold, and most charts' CI fixtures sit below it. `ci/selector-match/<chart>.yaml` is a render-only overlay applied **last**, whose only job is to push a chart over that threshold — and a chart that *has* one and still renders no selector **fails**, since that overlay existing is the whole claim that something was checked. On top of that the run fails if **no selector at all** was checked anywhere. Today `stalwart` is exercised through its overlay (two replicas in both node groups plus an `externalServices.valkey.host`, without which the chart's own "cluster with no in-memory store" abort fires) and `keycloak` through its ordinary install-test fixture, which already runs two replicas.
- **What is deliberately not checked, named so the gap is not mistaken for coverage.** *Preferred* `podAffinity` and `podAntiAffinity` of either strength are out of scope: their semantics are relative to pods that exist, so a term matching nothing is a legitimate configuration rather than a mistake — a preference that cannot be met is dropped, and an anti-affinity over zero pods is satisfied. And a selector expressed **only** through `matchExpressions` is **reported as "not checked", never counted as passing** — evaluating set-based requirements means reimplementing label-selector semantics, which is a second parser by another name.
- **`requiredDuringSchedulingIgnoredDuringExecution` `podAffinity` *is* checked** (Gitea #101), and it is the one affinity case that is not a preference: a required term whose selector matches no pod in the release can never be satisfied, so the pod stays `Pending` for good. It is held against **all** pod templates of the render rather than its own, because selecting a *different* workload's pod is the entire purpose of such a term. First and so far only user: `paperless-ngx`' consume CronJob, pinned onto the node of its StatefulSet pod so Longhorn will mount the shared RWX consume volume — where the silent failure would be a mover that stops moving, one `Pending` pod per schedule, with both templates still rendering perfectly.
- **Two anti-vacuum guards, because this test can be green over nothing.** Both objects are rendered only above a replica threshold, and most charts' CI fixtures sit below it. `ci/selector-match/<chart>.yaml` is a render-only overlay applied **last**, whose only job is to push a chart over that threshold — and a chart that *has* one and still renders no selector **fails**, since that overlay existing is the whole claim that something was checked. On top of that the run fails if **no selector at all** was checked anywhere. Today `stalwart` is exercised through its overlay (two replicas in both node groups plus an `externalServices.valkey.host`, without which the chart's own "cluster with no in-memory store" abort fires), `keycloak` through its ordinary install-test fixture, which already runs two replicas, and `paperless-ngx` through `ci/selector-match/paperless-ngx.yaml`, whose only job is to switch `smbConnection` on so the consume CronJob — and with it the required `podAffinity` — renders at all.
Measured: 30 charts, **5 selectors verified in ~14 s**, no cluster. The fail path was exercised rather than assumed — renaming `node-group` to `nodegroup` in a hand-written copy of the PDB selector produced two `FAIL` lines naming both the selector and the labels actually present, and a non-zero exit; the same was done for the spread constraint's selector.
Measured: 30 charts, **6 selectors verified**, no cluster. The fail path was exercised rather than assumed — renaming `node-group` to `nodegroup` in a hand-written copy of the PDB selector produced two `FAIL` lines naming both the selector and the labels actually present, and a non-zero exit; the same was done for the spread constraint's selector. For the `podAffinity` check the same was done by renaming the StatefulSet's `app` label in a throwaway copy of `paperless-ngx` while leaving the CronJob's term alone: one `FAIL` naming the selector, the three pod templates it missed and the labels actually present, and a non-zero exit.
### HTTPRoute render test (all charts, every PR)
@@ -187,6 +188,38 @@ Measured in CI (run 719, the pilot): **30 charts, 4 assertions held**, whole job
The fail path was exercised rather than assumed: a Service renamed in the route, a port the Service does not expose, a port left out entirely, an emptied `parentRefs` with the chart's own guard disabled, an overlay that renders no route, a run with zero assertions, and — for the pairing — both the redirect rule deleted outright and the redirect pointed at a path other than the one stripped; each produced a `FAIL` naming both sides and a non-zero exit.
### Bump-level test (the version-bump hook, every PR)
`ci/run-bump-level-test.sh` (no arguments) holds every case of `ci/renovate-bump-chart.sh` against the version string it writes. Added with Gitea #128, which is what made it necessary: until then the hook was a calculator with one branch, and with the update type coming from Renovate it became the thing that *says how big a change a cluster is being offered* — while the chart version is the only signal anything downstream sees (see "Chart dependencies": Fleet reads our chart version and nothing inside the package).
Its failure mode is the silent kind this section's two rules are about: **a wrong level is a valid version string.** The PR goes green, the chart publishes, and the only symptom is a number that understates what it ships. Nothing in the pipeline reads that number, so nothing but a test can.
A case is not a function call, because the script reads the **base commit** (`git show HEAD:<chart>/Chart.yaml`) *and* the working tree: each of the 49 cases gets a throwaway `git init` under `mktemp -d`, one commit carrying the base `Chart.yaml`, and the new one written over it uncommitted — the state Renovate's `postUpgradeTasks` hook actually runs in. Signing is switched off per command, since this workspace signs through an SSH signer and a throwaway repo has no business invoking it. No Helm, no cluster, no `pyyaml`, no `yq`, no Docker: the fixtures are six-line `Chart.yaml` files written with `printf`, and the assertion is one `version:` line read back with `sed`. The job therefore carries no Helm step and takes about a second.
What the cases cover: each update type, the `freesailarr` shape (an app minor while `appVersion` stands still), all seven non-semver app versions plus the majors that must *not* fall back, the maximum rule in both directions including a stale branch version that must **not** be kept, a missing and an empty and an unknown second argument, and the inputs the script must leave alone — a chart new on the branch, `template-chart`, a `.github/` or `ci/` argument, no argument at all. The full `version:` line is asserted in every case, so the `+up` suffix and its dash-to-dot normalization are covered throughout.
**What it cannot assert, and does not pretend to:** that Renovate passes the update type at all. The hook runs only when the command is allow-listed on the shared instance, which is outside this repository — which is why a *missing* second argument is itself a case here, and why the change that introduced it carries a dry run instead of a claim (see "The version-bump hook").
### YAML-scalar render test (all charts, every PR)
`ci/run-yaml-scalar-render-test.sh [chart …]` (no argument = all charts) renders every chart with the same fixtures the install-test uses and checks that **every scalar Kubernetes types as a string really arrived as one**. Introduced with Gitea #11, where a value rendered without `| quote` had been shipping in a chart for months; #112 turned out to be the same defect in the same chart's `env` block. Three assertions, and all three are properties the API itself requires — there is no chart for which the opposite is a legitimate configuration, which is why this test needs no per-chart expectations:
| Assertion | What a bare render does | Who notices today |
|---|---|---|
| `CronJob.spec.schedule` is a non-empty string | a schedule beginning with `*` (the commonest cron notation there is) or with `@` **aborts the render** — YAML reads the leading character as an alias or a reserved indicator | the operator, eventually: the error names the **template** and their own value appears nowhere in it, and **quoting it in the values file does not help**, because Helm passes the string's content through |
| every `env[].value` of every container and initContainer is a string | `"true"` and `"0755"` are legal plain scalars that are **not strings**: the chart renders perfectly and the API server rejects the object with `cannot unmarshal bool into Go struct field EnvVar.value of type string` | nobody until apply time — which in this repo means the install-test or a cluster, never the render |
| every `metadata.labels` and `metadata.annotations` value is a string | both are typed `map[string]string`, so a unit-less `100` in an upload-size annotation is rejected whole | nobody in the render |
**Why this job exists next to the render tests that already run:** the object that carried the original defect is rendered by **no fixture at all**. `paperless-ngx`' consume CronJob sits behind `smbConnection.enabled`, which is `false` in the chart defaults and `false` in its install-test fixture — deliberately, since the CI cluster has no SMB CSI driver. So nothing in this repository ever produced that object: not lint, not the other render tests, not the install-test. A test that renders only what the fixtures already render would have been just as blind, which is the whole argument for the overlays below.
Three things to know before touching it:
- **Helm is the YAML parser**, through `ci/yaml2json`, for the same reason as the two tests above — the runners have neither `pyyaml` nor `yq`, and a parser written in `awk` fails by reading nothing and reporting nothing.
- **Two anti-vacuum guards.** `ci/yaml-scalar/<chart>.yaml` is a render-only overlay applied **last**, whose job is to switch such objects on *and to feed them the hostile values* — a schedule beginning with a star, an `env` value that is a bare `true`. A chart that *has* one and still renders none of the three kinds **fails**, and the run fails if **nothing at all** was asserted anywhere.
- **What is deliberately not checked**, named so the gaps are not mistaken for coverage. A value rendered into a scalar that is **hand-quoted in the template** (`"{{ .Values.x }}"`) is a string in the output by construction, so this test is blind to it — what it can still do is break the YAML when the value itself contains a quote or a backslash, and only `| quote` fixes that. Everything that is **not** a label, an annotation, an `env` value or a schedule: a bare `storage:` or `className:` is just as unquoted, but its value space is DNS labels and quantities, where no legal value is YAML-ambiguous. And **whether a cron schedule is a *valid* schedule** — that is the controller's judgement, and narrowing the schema to a regex was considered and rejected, because the defect was never a wrong schedule but a correct one rendered bare.
Measured locally: **30 charts, 408 scalars verified, 0 failed**, no cluster. The fail path was exercised rather than assumed, in all three of its shapes, against a throwaway copy of the chart as it stands on `main`: the unquoted schedule aborted the render with `yaml: line 6: did not find expected alphabetic or numeric character` and the test reported it; with only the schedule quoted, the star-valued `env` entry aborted the render on the StatefulSet instead; and with only the bool-looking value left — a render that **succeeds** — the type assertion produced a `FAIL` naming the variable, the container and the exact API-server message. **Seven charts still render their `env` block bare** (`matrix-synapse`, `minecraft-router`, `minecraft-server`, `rallly`, `satisfactory`, `teamspeak-server`, and `paperless-ngx` until #11); the test is green for them only because their fixtures set no hostile `env` value, and the overlay is what makes the difference between covered and merely green.
### Install-test (PR pipeline stage 2)
For every chart a PR changes, `ci/run-install-test.sh <chart>` runs against a throwaway k3d cluster: it applies the stub `OnePasswordItem` CRD plus the chart's fixtures, `helm install`s the chart with its CI values, and waits for all Deployments/StatefulSets to roll out (per-chart timeout). Since probes and security hardening are active on all workloads, "Ready" means the app actually answers. On failure the job dumps pod descriptions, events and container logs.
@@ -267,9 +300,9 @@ Each chart renders its main image as `<repo>:{{ .Chart.AppVersion }}` — reposi
One rule those overrides share, learned from `searxng` and worth knowing before writing the next one: **a capture group is a claim about how the value is compared.** Renovate's regex versioning parses `major`, `minor`, `patch`, `build` and `revision` with `Number.parseInt` (`modules/versioning/regex/index.js`, `_parse`), so a non-numeric commit hash captured as `build` becomes whatever digits lead it — `9fea41204` is `9`, `777ba8fa4` is `777`, and a hash starting with a letter is `NaN`. Measured on renovate 41.173.1 and 44.48.0, that makes Renovate offer an image pushed *earlier the same day* as an update to a newer one — a downgrade wearing the clothes of a routine date bump, which is exactly what a review waves through and automerge then carries to `main`. Parts of a tag that carry no ordering belong **outside** every group: the anchored pattern still requires them to be there, they simply take no part in the comparison. `prerelease` is not the way out either — it sorts correctly but marks every tag `isStable === false`, which hands the whole chart to `ignoreUnstable`. `freesailarr` has no **app-version** block on purpose — see the images exception above; a block would either duplicate the `helm-values` updates or rewrite an `appVersion` that selects no image. It does carry one `customManagers` block, for `mediathekarr`: that image is pinned by digest because its tags carry no version (`latest` is a 2024 build, `beta` moves), and the built-in `helm-values` manager reads only `repository`/`tag` and skips it (`skipReason: invalid-value`), so the two cannot collide. Digest rules need the same guard as versioning rules and for a sharper reason: while the pin is current, a matching rule and a broken one produce the identical empty result. Verified with two dry-run fixtures — the real digest yields no update, a deliberately stale one yields exactly one `digest` update back to the pinned digest.
A consequence of pinning every tag in `values.yaml`: `ci/renovate-bump-chart.sh` sees an unchanged `appVersion` and bumps the chart **patch**. That holds for all ten images alike, the digest-pinned one included — none of them is special — and it is the intended outcome, since `appVersion` here is the chart's own marker rather than a tracked upstream version.
A consequence of pinning every tag in `values.yaml`, and the measurement that forced the change in Gitea #128: `ci/renovate-bump-chart.sh` used to see an unchanged `appVersion` and bump the chart **patch** — for all ten images alike, the digest-pinned one included. `appVersion` here is the chart's own marker rather than a tracked upstream version, so a comparison had nothing to read: prowlarr's `2.5.2 → 2.6.5` was an upstream **minor** that arrived as a chart patch. Since #128 the level comes from Renovate's classification instead (`{{{updateType}}}`, the hook's second argument), so a minor among the ten makes a chart minor and a mixed group takes the highest level in it. The `+up` suffix still follows `appVersion` and therefore still stands still.
**An open point this leaves, recorded rather than solved.** That automatic patch is also a classification, and the convention exempts a "plain app-version bump" from the wiki obligation — so an image update that quietly changes a *default* is the one case where nothing reviews it and nothing documents it. It has already happened once: the `sonarr` 4.0.19 bump carried upstream's removal of the hard-coded trusted proxy networks, and it reached `main` unremarked (see [Lessons learned](https://git.jk-dev.de/jlab-cluster/helm-charts/wiki/Lessons-Learned)). Whether the convention should change is a separate decision and deliberately not proposed here.
**That open point is now decided (Gitea #128), and the decision is deliberately smaller than the problem.** The gap was real: the automatic patch was also a *classification*, and the convention exempts a "plain app-version bump" from the wiki obligation — so an image update that quietly changed a **default** was the one case where nothing reviewed it and nothing documented it. It had already happened: the `sonarr` 4.0.19 bump carried upstream's removal of the hard-coded trusted proxy networks and reached `main` unremarked (see [Lessons learned](https://git.jk-dev.de/jlab-cluster/helm-charts/wiki/Lessons-Learned)). What was decided is **the classification and not a reading duty**: the chart jump follows the app jump, so the number stops understating what it ships. The variant that would have caught that incident at its origin — *a chart minor or major obliges reading the upstream release notes, with the result stated in the PR even when it is "nothing found"* — was **rejected**, and its price is the part that stays: nobody is obliged to look behind the number, so an app minor that changes a default can still reach `main` unremarked. What the bigger number does buy is visibility where a patch has none — Fleet's `fleet.yaml` in `cluster-config` sees only our chart version (see "Chart dependencies"), so a minor is a number other gates can act on. Whether `cluster-config` actually distinguishes update types is **not** visible from this repository; that is the mechanism which makes the number effective, not a claim that it is in use.
Two dependencies are constrained beyond their versioning. **nginx** (the Synapse delegation sidecar) is limited to **even minors** via `allowedVersions`. nginx numbers its release branches by minor — even is stable, odd is mainline — and the chart belongs on stable. Patch updates inside the stable branch still flow through normally. The constraint tolerates a variant suffix (`1.30.4-alpine`) on purpose: docker versioning only ever offers candidates carrying the current tag's suffix, so a suffix-blind constraint would silently freeze nginx the day the sidecar moves to a slimmer base image — a switch to `1.30.4-alpine` that #84 has commissioned a check on — with no error and no warning.
@@ -296,13 +329,22 @@ Not managed: `ci/**` (install-test fixtures pin throwaway infrastructure to majo
### The version-bump hook
Renovate only rewrites `appVersion:` or an image tag; it knows nothing about `version: <chart-semver>+up<app-version>`. Left alone, every chart PR would fail the `build-chart` version check. `ci/renovate-bump-chart.sh <chart-dir>` closes that gap and is wired up as a `postUpgradeTasks` command:
Renovate only rewrites `appVersion:` or an image tag; it knows nothing about `version: <chart-semver>+up<app-version>`. Left alone, every chart PR would fail the `build-chart` version check. `ci/renovate-bump-chart.sh <chart-dir> [update-type]` closes that gap and is wired up as a `postUpgradeTasks` command. **Since Gitea #128 the chart jump follows the app jump**, which turned the script from a calculator into deciding logic:
- **app major changed** → chart major (`6.0.0` → `7.0.0`), matching "runtime behavior changes" in the semver conventions
- **anything else** (plain app update, sidecar image) → chart patch
- **update type `major`** → chart major (`6.0.0` → `7.0.0`), matching "runtime behavior changes" in the semver conventions
- **update type `minor`** → chart **minor** (`6.0.0` → `6.1.0`) — the part that was missing, and where the incident sat
- **`patch`, `digest`, `pin`, `pinDigest`** → chart patch
- **no or unknown update type** → the pre-#128 behaviour: app major changed → chart major, anything else → chart patch
- **never below what the branch already carries** — the maximum rule below
- the `+up` suffix is recomputed from the new `appVersion`, dashes normalized to dots
Everything is derived from the base commit, so the script is idempotent across the multiple updates a single Renovate branch can carry.
**The level comes from Renovate, not from `appVersion`,** and that is the whole point: `freesailarr` keeps a hand-maintained `appVersion` for ten upstreams, so the update that prompted the rule moved an image tag in `values.yaml` while `appVersion` stood still (see "The one grouped chart" above). Verified rather than assumed — Renovate 44.155.7, keycloak pinned one minor back: `--platform=local --dry-run=full` classifies the update (`"updateType": "minor"`) but **never reaches the branch worker**, so it cannot show the hook at all; driving Renovate's own post-upgrade executor against a throwaway checkout does, and logs `cmd: "ci/renovate-bump-chart.sh keycloak minor"` with the script answering `1.3.0+up26.7.0 -> 1.4.0+up26.8.1 (update type minor)`.
**Non-semver app versions stay at patch for a minor**, with the reason in the convention rather than in a footnote: in CalVer and date tags the second position is a *month*, so following it would mint a chart minor every month that says nothing about the risk, and in a four-part or suffixed version the positions are not semver positions at all. Seven charts on `main` are in that class — `rss-bridge` (`2026-09-30`), `homeassistant` (`2026.8.3`), `searxng` (`2026.9.13-3b8009065`), `minecraft-server` (`2026.8.2-java25`), `apache-tika` (`3.3.1.0-full`), `teamspeak-server` (`6.0.0-beta13.1`), `valkey` (`9.1.2-alpine`) — and an app **major** is not subject to it, so a year rolling over in a date tag is a chart major exactly as it was before. The honest edge: for `6.0.0-beta13.1` and `9.1.2-alpine` the numeric part *is* a semver line, so the fallback understates a real minor there. That is the accepted trade — conservative by default; a chart that wants the minor earns a documented exception rather than a widened pattern.
**Maximum, not replacement.** The chart jump is the *greater* of the chart's own classification and the app jump, so a value that becomes required stays a chart major under an app patch. Mechanically it is also what makes a grouped branch deterministic: the hook runs once per update and every call recomputes from the same base commit, so without the maximum the last call would win and `freesailarr`'s minor-among-patches group would take whatever level it happened to end on. The script only ever keeps a *higher* version than it computes, which is why repeated runs stay idempotent, and everything else is derived from the base commit for the same reason.
`ci/run-bump-level-test.sh` holds every one of those cases (49 of them) against the version string the script writes — a wrong level is a valid version string, so the PR would go green and the only symptom would be a number that understates what it publishes.
> **One-time setup outside this repo:** `postUpgradeTasks` only runs on self-hosted Renovate **and** only when the command is allow-listed in the *instance* config:
>
@@ -311,6 +353,8 @@ Everything is derived from the base commit, so the script is idempotent across t
> ```
>
> Without that entry Renovate skips the task without an error and every chart PR stays red on the version check.
>
> **The second argument depends on that pattern's shape, and the shape is not visible from here.** Renovate matches the allow-list against the **compiled** command (`allowedCommands.some((p) => regEx(p).test(compiledCmd))`), so the front-anchored form above still matches with an argument appended — measured, see the dry run noted in `renovate.json5`. A pattern anchored at the end on the instance (`…\.sh$`) would stop matching and produce the same silent skip a missing entry does. Only the maintainer can check that at the instance. It is why the script treats a **missing** update type as the pre-#128 behaviour instead of an error: the worst case of a narrower allow-list, an older Renovate or an unpopulated template field is the old classification, not red chart PRs.
If the dependency dashboard reports `Failed to look up github-releases package …: no-result` for third-party repos, that is the instance's github.com credentials, not this config. The instance needs a read-only PAT as `GITHUB_COM_TOKEN` for github.com datasource lookups; its own platform token is a Gitea token and is worth nothing there.
@@ -371,10 +415,11 @@ These guidelines are **binding for every current and future chart** in this repo
### 2. Versioning
- `Chart.yaml` uses `version: <chart-semver>+up<appVersion>`; the `+up` suffix matches `appVersion`, with dashes normalized to dots (e.g. `0.0.36-beta` → `+up0.0.36.beta`). `appVersion` is the **exact image tag** of the main app image.
- Semver rules for the chart part:
- **Patch**: bug fixes without behavior or values-schema change; plain app-version updates.
- **Minor**: additive, backward-compatible changes (new optional values with unchanged defaults).
- **Major**: runtime behavior changes or breaking values changes (e.g. resource renames, probes, security hardening, values that become required).
- Semver rules for the chart part. **The chart jump is the greater of the chart's own classification and the jump of what it ships** (Gitea #128) — a maximum, never a replacement, so a value that becomes required is a chart major even under an app patch:
- **Patch**: bug fixes without behavior or values-schema change; an app **patch** or digest update.
- **Minor**: additive, backward-compatible changes (new optional values with unchanged defaults); an app **minor** update.
- **Major**: runtime behavior changes or breaking values changes (e.g. resource renames, probes, security hardening, values that become required); an app **major** update.
- The app half of that is mechanical and belongs to the hook rather than to a reviewer's judgement: `ci/renovate-bump-chart.sh` takes the level from Renovate's update type, with non-semver app versions staying at patch for a minor and a missing update type falling back to the old behaviour. The full rule, the seven affected charts and what it deliberately does **not** oblige anybody to read are in "The version-bump hook" above.
- Any push to `main` touching a chart directory publishes that chart, so the version must be bumped in the same change.
### 3. Images
@@ -494,6 +539,7 @@ These guidelines are **binding for every current and future chart** in this repo
- Every workload container offers an additive `env` value block, appended to the env entries the chart always sets.
- Entries are rendered through `tpl` (so values may contain Helm templating) and support the `value`, `secretKeyRef` and `configMapKeyRef` forms (pattern: `template-chart/templates/xxx.deployment.yaml`).
- **`| quote` on the rendered `value`, and on the `name`/`key` of the ref forms, is part of the pattern and not cosmetic.** An env value is typed `string` by Kubernetes, so a value of `"true"` or `"0755"` rendered bare becomes a YAML bool or integer and the API server rejects the whole manifest with `cannot unmarshal bool into Go struct field EnvVar.value of type string` — after a perfectly green render. A value beginning with `*` is worse in the other direction: it aborts the render with a YAML error naming the template, while the operator's own value appears nowhere in the message, and quoting it in the values file does not help. **The rule is held by `ci/run-yaml-scalar-render-test.sh` on every PR** (Gitea #11/#112), which is also how the count below is known rather than assumed: `template-chart` carries the rule in a comment claiming "every chart in this repo quotes here", and **six still do not** — `matrix-synapse`, `minecraft-router`, `minecraft-server`, `rallly`, `satisfactory`, `teamspeak-server`. They are green only because no fixture of theirs sets a hostile value; bringing a chart to the quoted form is a **major** bump for it, since it changes rendered output for values that currently work.
- **Additive means additive: an entry repeating a name the chart already sets does not replace it, and what the pair does depends on when it arrives.** At install both entries reach the pod and the later one — the user's — wins. On an *existing* release the same entry is lost: Kubernetes merges `env` by name, the pair collapses to the chart's own value, and `helm upgrade` still reports success. Measured on `urlaubsverwaltung` (helm 3.19.0, Kubernetes 1.35): `helm get manifest` shows the variable twice while the live Deployment shows it once, carrying the chart's value. The failure mode is silence, so a chart whose `NOTES.txt` tells operators to override one of its own variables has to say that the override only takes at install time — and that a green upgrade proves nothing, the value belongs read back off the Deployment.
### 11. Scaling
@@ -584,7 +630,7 @@ One more trap, and it is the reason `stalwart` gates its subchart on `valkeySubc
**What it costs, all of it visible before the next one is added:**
- **The subchart is invisible to everything downstream.** Its version is inside our packaged `.tgz`, so anything that watches Helm release versions — Fleet's `fleet.yaml` in `cluster-config` included — sees only *our* chart version. A new upstream release, security fix included, reaches a cluster **only** when we publish a new version of the outer chart. Nothing reports the gap. Renovate's built-in `helmv3` manager opens the PR (no `customManagers` entry needed — the repository URL and version sit together in one file) and `ci/renovate-bump-chart.sh` bumps the chart patch that ships it; that chain is the entire safety net. State this in a comment at the pinned version, not only in a review.
- **The subchart is invisible to everything downstream.** Its version is inside our packaged `.tgz`, so anything that watches Helm release versions — Fleet's `fleet.yaml` in `cluster-config` included — sees only *our* chart version. A new upstream release, security fix included, reaches a cluster **only** when we publish a new version of the outer chart. Nothing reports the gap. Renovate's built-in `helmv3` manager opens the PR (no `customManagers` entry needed — the repository URL and version sit together in one file) and `ci/renovate-bump-chart.sh` bumps the chart version that ships it — since Gitea #128 at the **subchart's own level**, so an upstream minor is a chart minor even though our `appVersion` did not move. That chain is the entire safety net, and the number it writes is the only place the subchart's jump is visible at all. State this in a comment at the pinned version, not only in a review.
- **Every pipeline that touches the chart needs a network fetch first.** `helm lint`, `helm package`, `helm template` and `helm install` all exit 1 while a declared dependency is missing from `charts/`, so `ci/helm-dependencies.sh` runs before each of them (see CI/CD above). One of those call sites is the resource-quantity render test, which runs over **every** chart on **every** PR — a dependency-carrying chart can therefore turn a third-party Helm repository's bad day into a red pipeline on a PR that touched nothing related.
- **`charts/*.tgz` and `Chart.lock` are not committed** (`.gitignore` says why): the archive is a third-party binary nobody reviews in a diff, and a committed lock would arrive stale because Renovate rewrites `Chart.yaml` and cannot regenerate it — which is also why the pipelines run `helm dependency update` and not `helm dependency build`.
- **The repo's own mechanisms stop at the subchart boundary.** `checksum/config`, the 1Password auto-restart annotation and the `scaleDown` flag reach our templates only. A chart that renders a `OnePasswordItem` for its subchart documents the manual `kubectl rollout restart` that replaces the missing restart-on-change.
+189 -33
View File
@@ -11,23 +11,56 @@
# failing the `build-chart` version-bump check. This script closes that gap and
# is wired up as a `postUpgradeTasks` command in renovate.json5.
#
# Usage: ci/renovate-bump-chart.sh <chart-directory>
# Usage: ci/renovate-bump-chart.sh <chart-directory> [update-type]
#
# Bump level (README "Versioning"):
# - app major changed -> chart major, minor/patch reset to 0
# - anything else -> chart patch (plain app updates, sidecar images)
# Bump level (README "Versioning", decided in Gitea #128):
# - update type major -> chart major, minor/patch reset to 0
# - update type minor -> chart MINOR, patch reset to 0
# - patch / digest / pin -> chart patch
# - no or unknown update type -> the appVersion comparison below, i.e. the
# behaviour this script had before #128
# - and never BELOW what the branch already carries (see the maximum rule
# further down)
#
# THE UPDATE TYPE COMES FROM RENOVATE, AND THAT IS THE POINT (Gitea #128).
# Renovate has already classified the jump; passing `{{{updateType}}}` in is
# what makes a chart minor possible at all. Deriving it here from appVersion
# would be blind to exactly the case that prompted the rule: `freesailarr`
# carries a hand-maintained `appVersion: "1.0.0"` for ten upstreams and it never
# moves, so prowlarr's 2.5.2 -> 2.6.5 — an image tag in values.yaml — is a
# minor with an unchanged appVersion. README, "Dependency updates (Renovate)".
#
# WHAT A CHART MINOR BUYS, AND WHAT IT DOES NOT. It makes the number honest: an
# app minor stops arriving as a chart patch. It does NOT oblige anyone to read
# the release notes — that variant was decided against — so an app minor that
# changes a default can still reach `main` unremarked. What the number does is
# make the jump VISIBLE to everything downstream that sees only our chart
# version (Fleet's `fleet.yaml` in `cluster-config` among them), where a patch
# is invisible. See the wiki, Lessons-Learned.
#
# THE SECOND ARGUMENT IS OPTIONAL ON PURPOSE. `postUpgradeTasks` only runs when
# the command is allow-listed in the *instance* config, which is not visible
# from this repository, and a command that does not match the allow-list is
# skipped SILENTLY. So a missing update type — a narrower allow-list pattern, an
# older Renovate, a template field that is not populated — must not be a
# failure: without it the script falls back to what it did before #128 and every
# chart PR still gets its version bump.
#
# Everything is derived from the base commit (HEAD, which during
# postUpgradeTasks still points at the base branch), never from the working
# tree's `version:`. That makes repeated runs on the same branch idempotent —
# Renovate calls the script once per dependency update, and a branch may carry
# more than one update for the same chart.
# more than one update for the same chart. The one place the working tree IS
# read is the maximum rule below, and only to keep a higher version.
#
# Runs inside the Renovate container, so: POSIX sh, no bashisms, no jq/yq.
#
# Tested by ci/run-bump-level-test.sh, which is where the cases live.
set -eu
dir=${1:-}
update_type=${2:-}
[ -n "$dir" ] || exit 0
# Strip a trailing slash so "outline/" and "outline" behave the same.
@@ -49,6 +82,72 @@ field() {
sed -n "s/^$1:[[:space:]]*//p" | head -n 1 | tr -d "\"' "
}
# Splits "<major>.<minor>.<patch>" into p_major/p_minor/p_patch. Returns 1 if
# the string is not exactly three numeric parts, which is also how a four-part
# version (3.3.1.0) and a suffixed one (9.1.2-alpine) are rejected: the last
# part then is not numeric.
parse_semver() {
_sv=$1
case "$_sv" in
*.*.*) ;;
*) return 1 ;;
esac
p_major=${_sv%%.*}
_rest=${_sv#*.}
p_minor=${_rest%%.*}
p_patch=${_rest#*.}
for _part in "$p_major" "$p_minor" "$p_patch"; do
case "$_part" in
'' | *[!0-9]*) return 1 ;;
esac
done
return 0
}
# Is <a.b.c> strictly greater than <d.e.f>? Six arguments, componentwise.
semver_gt() {
if [ "$1" -ne "$4" ]; then
if [ "$1" -gt "$4" ]; then return 0; else return 1; fi
fi
if [ "$2" -ne "$5" ]; then
if [ "$2" -gt "$5" ]; then return 0; else return 1; fi
fi
if [ "$3" -gt "$6" ]; then return 0; else return 1; fi
}
# Does this app version's MINOR POSITION carry a statement about jump width?
# Only a plain [v]X.Y.Z does. Everything else keeps the pre-#128 behaviour, i.e.
# a patch, and the reason is in the convention rather than in a footnote: in
# CalVer and in date tags the second position is a MONTH, so following it would
# produce a chart minor every month that says nothing about the risk, and in a
# four-part or suffixed version the positions are not semver positions at all.
#
# Measured on origin/main, this covers seven charts and only those seven:
# rss-bridge 2026-09-30 (no dots at all)
# homeassistant 2026.8.3 (CalVer — caught by the year guard)
# searxng 2026.9.13-3b8009065 (CalVer plus a commit hash)
# minecraft-server 2026.8.2-java25 (CalVer plus a JRE suffix)
# apache-tika 3.3.1.0-full (four parts plus a flavour)
# teamspeak-server 6.0.0-beta13.1 (prerelease line)
# valkey 9.1.2-alpine (flavour suffix)
#
# The last two are the honest edge of this rule: their numeric part IS a semver
# line, so the fallback understates a real minor there. That is the decided
# trade — conservative by default, and a chart that wants the minor gets a
# documented exception rather than a widened pattern.
#
# A `major` update type is NOT subject to this: a year rolling over in a date
# tag arrives as a major, and that was a chart major before #128 too.
app_minor_is_meaningful() {
_av=${1#v}
_av=${_av#V}
parse_semver "$_av" || return 1
# A four-digit leading component from 2000 on is a calendar year, not a
# semver major.
if [ "$p_major" -ge 2000 ]; then return 1; fi
return 0
}
base=$(git show "HEAD:$chart" 2>/dev/null) || {
echo "renovate-bump: $chart is new on this branch, leaving its version alone"
exit 0
@@ -57,6 +156,7 @@ base=$(git show "HEAD:$chart" 2>/dev/null) || {
base_version=$(printf '%s\n' "$base" | field version)
base_app=$(printf '%s\n' "$base" | field appVersion)
new_app=$(field appVersion <"$chart")
tree_version=$(field version <"$chart")
if [ -z "$base_version" ] || [ -z "$new_app" ]; then
echo "renovate-bump: $chart has no version/appVersion to work with" >&2
@@ -65,27 +165,13 @@ fi
# <chart-semver> is everything before the +up build suffix.
semver=${base_version%%+*}
case "$semver" in
*.*.*) ;;
*)
echo "renovate-bump: cannot parse chart semver '$semver' in $chart" >&2
exit 1
;;
esac
major=${semver%%.*}
rest=${semver#*.}
minor=${rest%%.*}
patch=${rest#*.}
for part in "$major" "$minor" "$patch"; do
case "$part" in
'' | *[!0-9]*)
echo "renovate-bump: non-numeric component in chart semver '$semver'" >&2
exit 1
;;
esac
done
parse_semver "$semver" || {
echo "renovate-bump: cannot parse chart semver '$semver' in $chart" >&2
exit 1
}
major=$p_major
minor=$p_minor
patch=$p_patch
# Leading major of an app version: v1.159.0 -> 1, 3.3.1.0-full -> 3,
# 2025-06-03 -> 2025.
@@ -93,17 +179,87 @@ app_major() {
printf '%s' "$1" | sed -e 's/^[vV]//' -e 's/[^0-9].*$//'
}
if [ "$new_app" != "$base_app" ] && [ "$(app_major "$new_app")" != "$(app_major "$base_app")" ]; then
reason="app major $base_app -> $new_app"
new_semver="$((major + 1)).0.0"
else
reason="patch-level change"
new_semver="$major.$minor.$((patch + 1))"
# The pre-#128 level: app major changed -> major, anything else -> patch. Also
# the fallback whenever no usable update type arrives.
app_version_level() {
if [ "$new_app" != "$base_app" ] && [ "$(app_major "$new_app")" != "$(app_major "$base_app")" ]; then
echo major
else
echo patch
fi
}
case "$update_type" in
major)
level=major
reason="update type major"
;;
minor)
if app_minor_is_meaningful "$new_app"; then
level=minor
reason="update type minor"
else
level=$(app_version_level)
reason="update type minor, but appVersion '$new_app' is not a plain semver line -> $level"
fi
;;
patch | digest | pin | pinDigest)
level=patch
reason="update type $update_type"
;;
'')
level=$(app_version_level)
reason="no update type given, derived from appVersion -> $level"
;;
*)
level=$(app_version_level)
reason="unknown update type '$update_type', derived from appVersion -> $level"
;;
esac
case "$level" in
major)
new_major=$((major + 1))
new_minor=0
new_patch=0
;;
minor)
new_major=$major
new_minor=$((minor + 1))
new_patch=0
;;
*)
new_major=$major
new_minor=$minor
new_patch=$((patch + 1))
;;
esac
# THE MAXIMUM RULE (Gitea #128): the chart level is the GREATER of the chart's
# own classification and the app jump — never a replacement. Two cases, and the
# second is the mechanical one:
#
# - a chart-own bump already in the working tree stands. A value that becomes
# required is a chart major even when the app jump is a patch, and the chart
# version must not stop reporting the chart's own breaks.
# - a branch carrying SEVERAL updates. Renovate calls this script once per
# update and every call recomputes from the same base commit, so without
# this the last call would win: `freesailarr`'s grouped PR has arrived with
# three minors among five patches, and the chart level would then depend on
# the order the hook happened to run in.
#
# It only ever keeps a HIGHER version, so repeated runs stay idempotent.
if parse_semver "${tree_version%%+*}" &&
semver_gt "$p_major" "$p_minor" "$p_patch" "$new_major" "$new_minor" "$new_patch"; then
reason="$reason, kept the higher $p_major.$p_minor.$p_patch already on the branch"
new_major=$p_major
new_minor=$p_minor
new_patch=$p_patch
fi
# Dashes in the app version become dots in the +up suffix.
suffix=$(printf '%s' "$new_app" | tr '-' '.')
new_version="$new_semver+up$suffix"
new_version="$new_major.$new_minor.$new_patch+up$suffix"
sed -i.bak "s|^version:.*|version: $new_version|" "$chart"
rm -f "$chart.bak"
+229
View File
@@ -0,0 +1,229 @@
#!/usr/bin/env bash
# Bump-level test: every case of ci/renovate-bump-chart.sh, held against the
# version string it writes into a throwaway chart.
#
# Usage: ci/run-bump-level-test.sh
#
# WHY THIS TEST EXISTS (Gitea #128)
#
# Until #128 the hook was a calculator with one branch: app major -> chart
# major, everything else -> chart patch. With the update type coming from
# Renovate it became DECIDING LOGIC — it is now the thing that says how big a
# change the cluster is being offered, and the chart version is the only signal
# anything downstream sees (README, "Chart dependencies": Fleet's `fleet.yaml`
# in `cluster-config` sees our chart version and nothing inside the package).
#
# And its failure mode is silence. A wrong level is a valid version string: the
# PR goes green, the chart publishes, and the only symptom is a number that
# understates what it ships. Nothing in the pipeline reads it, so nothing but a
# test can.
#
# HOW A CASE IS BUILT, AND WHY A REAL GIT REPO
#
# The script reads the BASE commit (`git show HEAD:<chart>/Chart.yaml`) and the
# working tree, so a case is not a function call — it is a repository with a
# commit and a dirty file. Each case therefore gets a throwaway `git init` in a
# unique temp directory, one commit carrying the base Chart.yaml, and the new
# Chart.yaml written over it uncommitted. That is exactly the state Renovate's
# `postUpgradeTasks` hook runs in.
#
# Signing is switched off per command (`-c commit.gpgsign=false`): this
# workspace signs every commit through an SSH signer, and a throwaway repo has
# no business invoking it. The temp root comes from `mktemp -d`, because /tmp is
# shared with whatever else runs on this machine.
#
# The runners have no pyyaml, no yq and no Docker — the same constraint
# ci/run-selector-match-test.sh and ci/run-quantity-render-test.sh work under.
# None is needed: the fixtures are six-line Chart.yaml files written by printf,
# and the assertion is one `version:` line read back with sed. Helm is not
# needed either, which is why this job carries no Helm step.
#
# WHAT IS ASSERTED, AND WHAT IS NOT
#
# Asserted: the full `version:` line (so the `+up` suffix and its dash-to-dot
# normalization are covered by every case), the exit code, and for the cases
# that must leave the chart alone, that the line is untouched.
#
# NOT asserted, and not assertable from here: that Renovate passes the update
# type at all. The hook only runs when the command is allow-listed in the
# *instance* config, which lives outside this repository — and a pattern that
# does not match is skipped silently. That is why a missing second argument is
# itself a case here (it must fall back, not fail), and why the PR that
# introduced this carries a dry-run instead of a claim.
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SCRIPT="$ROOT/ci/renovate-bump-chart.sh"
[ -x "$SCRIPT" ] || {
echo "FAIL: $SCRIPT is not executable" >&2
exit 1
}
TMP="$(mktemp -d "${TMPDIR:-/tmp}/bump-level-test.XXXXXX")"
trap 'rm -rf "$TMP"' EXIT
GIT_OPTS=(-c user.name=ci -c user.email=ci@example.invalid -c commit.gpgsign=false -c tag.gpgsign=false)
passed=0
failed=0
chart_yaml() { # <name> <version> <appVersion>
printf 'apiVersion: v2\nname: %s\ndescription: test fixture\ntype: application\nversion: %s\nappVersion: "%s"\nicon: https://example.com/icon.png\n' \
"$1" "$2" "$3"
}
# Builds a repository whose HEAD carries <base-content> for <dir>/Chart.yaml
# (skipped when empty, which is the "new chart on this branch" case) and whose
# working tree carries <tree-content>. Prints the repository path.
scenario() { # <dir> <base-content> <tree-content>
local dir=$1 base=$2 tree=$3 repo
repo="$(mktemp -d "$TMP/case.XXXXXX")"
git -C "$repo" init -q
printf 'root commit so HEAD exists\n' >"$repo/README.md"
git -C "$repo" add README.md
git -C "$repo" "${GIT_OPTS[@]}" commit -qm "root"
mkdir -p "$repo/$dir"
if [ -n "$base" ]; then
printf '%s' "$base" >"$repo/$dir/Chart.yaml"
git -C "$repo" add "$dir/Chart.yaml"
git -C "$repo" "${GIT_OPTS[@]}" commit -qm "base chart"
fi
[ -z "$tree" ] || printf '%s' "$tree" >"$repo/$dir/Chart.yaml"
printf '%s' "$repo"
}
version_of() { sed -n 's/^version:[[:space:]]*//p' "$1" | head -n 1; }
report() { # <name> <expected> <got> <expected-rc> <got-rc> <output>
if [ "$2" = "$3" ] && [ "$4" = "$5" ]; then
passed=$((passed + 1))
printf ' ok %s\n' "$1"
else
failed=$((failed + 1))
printf ' FAIL %s\n expected version %-24s rc %s\n got version %-24s rc %s\n output: %s\n' \
"$1" "$2" "$4" "$3" "$5" "$6"
fi
}
# The ordinary case: one chart, a base version, an app version that may or may
# not have moved, and an update type. UPDATE_TYPE of "-" calls the script with
# ONE argument (the allow-list / older-Renovate case), "" passes an empty
# second argument (a template field that resolved to nothing).
bump() { # <name> <base-version> <base-app> <tree-version> <tree-app> <update-type> <expected-version> [expected-rc]
local name=$1 base_version=$2 base_app=$3 tree_version=$4 tree_app=$5 utype=$6 expected=$7 rc_expected=${8:-0}
local repo out rc got
repo="$(scenario demo "$(chart_yaml demo "$base_version" "$base_app")" "$(chart_yaml demo "$tree_version" "$tree_app")")"
rc=0
if [ "$utype" = "-" ]; then
out="$(cd "$repo" && "$SCRIPT" demo 2>&1)" || rc=$?
else
out="$(cd "$repo" && "$SCRIPT" demo "$utype" 2>&1)" || rc=$?
fi
got="$(version_of "$repo/demo/Chart.yaml")"
report "$name" "$expected" "$got" "$rc_expected" "$rc" "$out"
}
echo "== update type decides the level (plain semver appVersion) =="
bump "major -> chart major" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 2.0.0 major 2.0.0+up2.0.0
bump "minor -> chart minor" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.5.0 minor 1.3.0+up1.5.0
bump "patch -> chart patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.4.1 patch 1.2.4+up1.4.1
bump "digest -> chart patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.4.0 digest 1.2.4+up1.4.0
bump "pin -> chart patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.4.0 pin 1.2.4+up1.4.0
bump "pinDigest -> chart patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.4.0 pinDigest 1.2.4+up1.4.0
bump "minor resets the patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.5.0 minor 1.3.0+up1.5.0
bump "major resets minor and patch" 4.5.6+up1.4.0 1.4.0 4.5.6+up1.4.0 2.0.0 major 5.0.0+up2.0.0
echo "== the case that prompted the rule: appVersion never moves (freesailarr) =="
# prowlarr 2.5.2 -> 2.6.5 is an image tag in values.yaml; freesailarr's
# appVersion is a hand-maintained marker and stays at 1.0.0. An appVersion
# comparison sees nothing and would bump the patch.
bump "sidecar/bundle minor, appVersion unchanged" 3.1.1+up1.0.0 1.0.0 3.1.1+up1.0.0 1.0.0 minor 3.2.0+up1.0.0
bump "sidecar/bundle patch, appVersion unchanged" 3.1.1+up1.0.0 1.0.0 3.1.1+up1.0.0 1.0.0 patch 3.1.2+up1.0.0
echo "== non-semver app versions stay at patch (all seven on origin/main) =="
bump "rss-bridge date tag, month roll" 2.1.1+up2026.09.30 2026-09-30 2.1.1+up2026.09.30 2026-10-05 minor 2.1.2+up2026.10.05
bump "homeassistant CalVer" 6.2.0+up2026.8.3 2026.8.3 6.2.0+up2026.8.3 2026.9.0 minor 6.2.1+up2026.9.0
bump "searxng CalVer plus hash" 1.1.0+up2026.9.13.3b8009065 2026.9.13-3b8009065 1.1.0+up2026.9.13.3b8009065 2026.10.1-aa12bb34c minor 1.1.1+up2026.10.1.aa12bb34c
bump "minecraft-server CalVer plus JRE" 6.0.1+up2026.8.2.java25 2026.8.2-java25 6.0.1+up2026.8.2.java25 2026.9.1-java25 minor 6.0.2+up2026.9.1.java25
bump "apache-tika four parts" 10.0.2+up3.3.1.0.full 3.3.1.0-full 10.0.2+up3.3.1.0.full 3.4.0.0-full minor 10.0.3+up3.4.0.0.full
bump "teamspeak prerelease line" 5.1.3+up6.0.0.beta13.1 6.0.0-beta13.1 5.1.3+up6.0.0.beta13.1 6.0.0-beta14.1 minor 5.1.4+up6.0.0.beta14.1
bump "valkey flavour suffix" 1.0.0+up9.1.2.alpine 9.1.2-alpine 1.0.0+up9.1.2.alpine 9.2.0-alpine minor 1.0.1+up9.2.0.alpine
# A major is NOT subject to the fallback - a year rolling over was a chart major
# before #128 as well.
bump "date tag, year roll, major" 2.1.1+up2026.09.30 2026-09-30 2.1.1+up2026.09.30 2027-01-05 major 3.0.0+up2027.01.05
bump "four parts, app major, major" 10.0.2+up3.3.1.0.full 3.3.1.0-full 10.0.2+up3.3.1.0.full 4.0.0.0-full major 11.0.0+up4.0.0.0.full
# The year guard is a guard, not a blanket: a v-prefixed plain line still counts.
bump "v-prefixed plain semver, minor" 7.2.0+upv1.159.0 v1.159.0 7.2.0+upv1.159.0 v1.160.0 minor 7.3.0+upv1.160.0
echo "== maximum, not replacement =="
# A chart-own bump already in the working tree stands, whatever the app jump is.
bump "chart major already on branch, app patch" 1.2.3+up1.4.0 1.4.0 2.0.0+up1.4.0 1.4.1 patch 2.0.0+up1.4.1
bump "chart minor already on branch, app patch" 1.2.3+up1.4.0 1.4.0 1.3.0+up1.4.0 1.4.1 patch 1.3.0+up1.4.1
# A grouped branch: the hook runs once per update from the same base, so the
# minor must survive a patch call that follows it.
bump "minor on branch, later patch call" 3.1.1+up1.0.0 1.0.0 3.2.0+up1.0.0 1.0.0 patch 3.2.0+up1.0.0
bump "patch on branch, later minor call" 3.1.1+up1.0.0 1.0.0 3.1.2+up1.0.0 1.0.0 minor 3.2.0+up1.0.0
bump "repeated minor call is idempotent" 3.1.1+up1.0.0 1.0.0 3.2.0+up1.0.0 1.0.0 minor 3.2.0+up1.0.0
bump "repeated major call is idempotent" 1.2.3+up1.4.0 1.4.0 2.0.0+up2.0.0 2.0.0 major 2.0.0+up2.0.0
bump "major on branch wins over a later minor" 1.2.3+up1.4.0 1.4.0 2.0.0+up1.4.0 1.5.0 minor 2.0.0+up1.5.0
# Stale branch after a rebase: the base moved ahead of what the branch carries,
# so the recomputed version wins and nothing is kept.
bump "stale branch version is not kept" 3.3.0+up1.0.0 1.0.0 3.1.2+up1.0.0 1.0.0 patch 3.3.1+up1.0.0
echo "== missing and unknown update type fall back to the pre-#128 behaviour =="
bump "no second argument, app major" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 2.0.0 - 2.0.0+up2.0.0
bump "no second argument, app minor" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.5.0 - 1.2.4+up1.5.0
bump "no second argument, app patch" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.4.1 - 1.2.4+up1.4.1
bump "no second argument, unchanged" 3.1.1+up1.0.0 1.0.0 3.1.1+up1.0.0 1.0.0 - 3.1.2+up1.0.0
bump "empty second argument, app major" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 2.0.0 "" 2.0.0+up2.0.0
bump "empty second argument, app minor" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.5.0 "" 1.2.4+up1.5.0
bump "unknown type, app minor" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 1.5.0 replacement 1.2.4+up1.5.0
bump "unknown type, app major" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 2.0.0 lockFileMaintenance 2.0.0+up2.0.0
bump "unknown type, maximum still holds" 1.2.3+up1.4.0 1.4.0 2.0.0+up1.4.0 1.4.1 nonsense 2.0.0+up1.4.1
echo "== charts the script must leave alone, and the hard failures =="
bump "unparsable chart semver fails loudly" 1.2+up1.4.0 1.4.0 1.2+up1.4.0 1.5.0 minor 1.2+up1.4.0 1
bump "missing appVersion fails loudly" 1.2.3+up1.4.0 1.4.0 1.2.3+up1.4.0 "" minor 1.2.3+up1.4.0 1
new_chart_repo="$(scenario demo "" "$(chart_yaml demo 0.1.0+up1.0.0 1.0.0)")"
rc=0
out="$(cd "$new_chart_repo" && "$SCRIPT" demo minor 2>&1)" || rc=$?
report "a chart new on the branch keeps its version" \
"0.1.0+up1.0.0" "$(version_of "$new_chart_repo/demo/Chart.yaml")" 0 "$rc" "$out"
case "$out" in
*"new on this branch"*) passed=$((passed + 1)); printf ' ok and says so\n' ;;
*) failed=$((failed + 1)); printf ' FAIL new chart: no explanation in the output: %s\n' "$out" ;;
esac
tpl_repo="$(scenario template-chart "$(chart_yaml template-chart 4.2.0+upxxx xxx)" "$(chart_yaml template-chart 4.2.0+upxxx xxx)")"
rc=0
out="$(cd "$tpl_repo" && "$SCRIPT" template-chart minor 2>&1)" || rc=$?
report "template-chart is never touched" \
"4.2.0+upxxx" "$(version_of "$tpl_repo/template-chart/Chart.yaml")" 0 "$rc" "$out"
# Renovate calls the hook for every update it makes, chart or not.
other_repo="$(scenario demo "$(chart_yaml demo 1.2.3+up1.4.0 1.4.0)" "$(chart_yaml demo 1.2.3+up1.4.0 1.4.0)")"
for arg in ".github/workflows" "ci/gateway-api-crds.sh" "." "" "not-a-chart"; do
rc=0
out="$(cd "$other_repo" && "$SCRIPT" "$arg" major 2>&1)" || rc=$?
report "non-chart argument '$arg' is a no-op" \
"1.2.3+up1.4.0" "$(version_of "$other_repo/demo/Chart.yaml")" 0 "$rc" "$out"
done
rc=0
out="$(cd "$other_repo" && "$SCRIPT" 2>&1)" || rc=$?
report "no arguments at all is a no-op" \
"1.2.3+up1.4.0" "$(version_of "$other_repo/demo/Chart.yaml")" 0 "$rc" "$out"
rc=0
out="$(cd "$other_repo" && "$SCRIPT" demo/ patch 2>&1)" || rc=$?
report "a trailing slash is accepted" \
"1.2.4+up1.4.0" "$(version_of "$other_repo/demo/Chart.yaml")" 0 "$rc" "$out"
echo
if [ "$failed" -gt 0 ]; then
printf 'bump-level test: %d passed, %d FAILED\n' "$passed" "$failed"
exit 1
fi
printf 'bump-level test: %d cases passed\n' "$passed"
+65 -4
View File
@@ -52,10 +52,22 @@
#
# WHAT IS DELIBERATELY NOT CHECKED
#
# - podAffinity / podAntiAffinity. They carry labelSelectors too, but their
# semantics are "relative to pods that exist", and an anti-affinity term
# matching nothing is a legitimate configuration rather than a mistake.
# Out of scope, and named so that the gap is not mistaken for coverage.
# - preferred podAffinity and podAntiAffinity of either strength. They carry
# labelSelectors too, but their semantics are "relative to pods that exist",
# and a term matching nothing is a legitimate configuration rather than a
# mistake: a preference that cannot be met is dropped and an anti-affinity
# over zero pods is satisfied. Out of scope, and named so that the gap is
# not mistaken for coverage.
#
# requiredDuringSchedulingIgnoredDuringExecution podAffinity is NOT in that
# gap and is checked (Gitea #101). Its failure is not a preference quietly
# dropped: a required term whose selector matches no pod in the release can
# never be satisfied, so the pod stays Pending forever. For the workload
# that introduced one here - paperless-ngx' consume CronJob, pinned onto the
# node of its StatefulSet pod so that Longhorn will mount the shared RWX
# volume - that would mean a mover that silently stops moving, with one
# Pending pod per schedule and a green render behind it. The term names a
# label written in ANOTHER template, which is the copy this test holds.
# - selectors expressed ONLY through matchExpressions. Evaluating a set-based
# requirement means reimplementing label-selector semantics, which is a
# second parser by another name. Those are REPORTED as "not checked" and
@@ -343,6 +355,55 @@ for doc in pdbs:
", ".join("%s=%s" % kv for kv in sorted(match_labels.items())),
len(all_templates), "; ".join(known) or "none"))
# 3. Every REQUIRED podAffinity term, against ALL pod templates of the render
# (Gitea #101). Not against its own template: the whole point of such a term
# is to select a DIFFERENT workload's pod, so the correct question is whether
# anything in this release carries the labels it asks for. Preferred terms and
# anti-affinity of either strength are left alone - see the header.
for where, template in all_templates:
affinity = template.get("spec", {}).get("affinity")
if not isinstance(affinity, dict):
continue
pod_affinity = affinity.get("podAffinity")
if not isinstance(pod_affinity, dict):
continue
terms = pod_affinity.get("requiredDuringSchedulingIgnoredDuringExecution")
if not isinstance(terms, list):
continue
for i, term in enumerate(terms):
if not isinstance(term, dict):
lines.append("FAIL|%s: required podAffinity[%d] at %s is not a mapping"
% (chart, i, where))
continue
tag = "required podAffinity[%d] on %s at %s" % (
i, term.get("topologyKey", "<no topologyKey>"), where)
match_labels, reason = classify(term.get("labelSelector"), tag)
if match_labels is None:
if term.get("labelSelector") is None:
lines.append("FAIL|%s: %s" % (chart, reason))
else:
lines.append("SKIP|%s: %s" % (chart, reason))
continue
hits = [w for w, t in all_templates if matches(match_labels, labels_of(t))]
if hits:
lines.append("OK|%s: %s matches %d pod template(s): %s"
% (chart, tag, len(hits), ", ".join(hits)))
else:
known = sorted({"%s=%s" % kv for _, t in all_templates for kv in labels_of(t).items()})
lines.append("FAIL|%s: %s selects %s, which matches NONE of the %d "
"pod template(s) in the same render. A REQUIRED "
"podAffinity term that matches no pod can never be "
"satisfied, so the pod it is attached to stays Pending "
"forever - for a CronJob that means one Pending pod per "
"schedule and a workload that silently stopped running, "
"with nothing in the render to show for it. The usual "
"cause is a label renamed in the template that carries "
"the selected pod. Labels actually present on the pod "
"templates: %s"
% (chart, tag,
", ".join("%s=%s" % kv for kv in sorted(match_labels.items())),
len(all_templates), "; ".join(known) or "none"))
for line in lines:
print(line)
PY
+412
View File
@@ -0,0 +1,412 @@
#!/usr/bin/env bash
# YAML-scalar render test: an operator value that must arrive in the manifest as
# a STRING is held against the manifest it actually produced.
#
# Usage: ci/run-yaml-scalar-render-test.sh [chart ...] (no argument = all charts)
#
# WHAT IT GUARDS AGAINST (Gitea #11, and #112 turned out to be the same genus)
#
# A value is rendered into a YAML scalar without `| quote`. The operator quotes
# it correctly in their values file, Helm passes the string's CONTENT through,
# and the quotes are gone again in the manifest. Two outcomes, and the quiet one
# is the worse one:
#
# 1. LOUD, but misdirected - the content is not a legal plain scalar and the
# render aborts. Measured on paperless-ngx' consume CronJob, whose schedule
# was unquoted: a schedule in the commonest cron notation of all (a star, a
# slash, an interval) made `helm template` exit 1 with
#
# error converting YAML to JSON: yaml: line 6: did not find expected
# alphabetic or numeric character
#
# YAML read the leading "*" as an alias indicator. The message names the
# TEMPLATE and the operator's own value is nowhere in it, so the search
# starts in the wrong place. "@daily" fails the same way.
# 2. SILENT until the API server - the content IS a legal plain scalar, just
# not a string. "true" becomes a bool, "0755" becomes an integer, and the
# chart renders perfectly. Kubernetes then rejects the object with
# "cannot unmarshal number into Go struct field EnvVar.value of type
# string" - at apply time, which for this repository means in the
# install-test or in a cluster, never in the render.
#
# Neither is caught by "does helm template exit 0" alone: the first one is, but
# only if something actually renders the object, and the second one is not
# caught at all.
#
# WHAT IS ASSERTED, and all three are properties Kubernetes itself requires -
# there is no chart for which the opposite is a legitimate configuration, which
# is why this needs no per-chart expectations:
#
# 1. CronJob .spec.schedule is a non-empty string.
# 2. Every env[].value of every container and initContainer is a string.
# 3. Every metadata.labels and metadata.annotations value is a string
# (the API types both as map[string]string).
#
# WHY OVER ALL CHARTS, not the changed ones: the same reason as
# ci/run-quantity-render-test.sh and ci/run-selector-match-test.sh. The env
# block is copied into every chart rather than shared, so the unquoted form
# travels by copy into whatever chart is started from an older one - and that
# chart's own pull request is not where anyone would look for it. Running over
# everything also means there is no list to maintain: adding a chart enrols it.
#
# THE OVERLAYS, AND WHY THIS TEST IS USELESS WITHOUT THEM
#
# The defect above survived in a shipped chart for months for one reason: the
# object that carried it is rendered by NO fixture. paperless-ngx' CronJob sits
# behind smbConnection.enabled, which is false in the chart defaults and in
# every committed fixture, so neither lint, nor the render tests, nor the
# install-test ever produced it. A test that renders only what the fixtures
# already render would have been just as blind.
#
# So ci/yaml-scalar/<chart>.yaml is an overlay applied LAST whose job is to
# switch such objects on AND to feed them the hostile values - a schedule
# beginning with a star, an env value that is a bare "true". Two rules follow,
# the same ones ci/run-selector-match-test.sh works under:
#
# 1. A chart that HAS an overlay and still renders none of the three things
# above FAILS. The overlay exists for exactly that, so an overlay that
# stopped reaching its object is a finding and not a shrug.
# 2. If nothing at all was asserted across the whole run, the run fails. A
# pass over zero assertions is not a pass.
#
# WHAT IS DELIBERATELY NOT CHECKED, so the gap is not mistaken for coverage:
#
# - Values rendered into a scalar that is hand-quoted in the template
# ("{{ .Values.x }}"). Those are strings in the output, so this test is
# blind to them by construction; what they can still do is break the YAML
# when the value itself contains a quote or a backslash, and only `| quote`
# fixes that. Reported here rather than silently implied.
# - Everything that is not a label, an annotation, an env value or a schedule.
# A bare `storage:` or `className:` is just as unquoted, but its value space
# is DNS labels and quantities, where no legal value is YAML-ambiguous. The
# three above are where operator free text actually lands.
# - Whether a cron schedule is a VALID schedule. That is the cron
# controller's judgement, and narrowing the schema to a regex was considered
# and rejected: the defect was never a wrong schedule, it was a correct one
# rendered bare.
#
# How the YAML is parsed: with helm, via ci/yaml2json, exactly as in
# ci/run-selector-match-test.sh - the runners have neither pyyaml nor yq, and a
# YAML reader written in awk here would fail by quietly reading nothing and
# staying green. That script's header carries the full argument.
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
CI_DIR="$ROOT/ci"
YAML2JSON="$CI_DIR/yaml2json"
OVERLAY_DIR="$CI_DIR/yaml-scalar"
RELEASE="scalar"
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
# --- Which charts -----------------------------------------------------------
CHARTS=()
if [ "$#" -gt 0 ]; then
CHARTS=("$@")
else
for DIR in "$ROOT"/*/; do
[ -f "$DIR/Chart.yaml" ] || continue
CHARTS+=("$(basename "$DIR")")
done
fi
if [ "${#CHARTS[@]}" -eq 0 ]; then
echo "ERROR: no chart directories found under $ROOT" >&2
exit 1
fi
FAILED=""
CHECKED=""
NOT_EXERCISED=""
TOTAL_ASSERTIONS=0
SUMMARY_ROWS=""
for CHART in "${CHARTS[@]}"; do
if [ ! -f "$ROOT/$CHART/Chart.yaml" ]; then
echo "ERROR: '$CHART' is not a chart directory" >&2
exit 1
fi
# A chart with a declared dependency cannot be rendered at all while charts/
# is empty, and this job runs over EVERY chart on EVERY pull request.
"$ROOT/ci/helm-dependencies.sh" "$ROOT/$CHART"
VALUES_ARGS=()
VALUES_LABEL="chart defaults"
if [ -f "$CI_DIR/$CHART/test-values.yaml" ]; then
VALUES_ARGS=(-f "$CI_DIR/$CHART/test-values.yaml")
VALUES_LABEL="ci/$CHART/test-values.yaml"
elif [ -f "$CI_DIR/trivy/values/$CHART.yaml" ]; then
VALUES_ARGS=(-f "$CI_DIR/trivy/values/$CHART.yaml")
VALUES_LABEL="ci/trivy/values/$CHART.yaml"
fi
HAS_OVERLAY=""
if [ -f "$OVERLAY_DIR/$CHART.yaml" ]; then
# Last, so it wins over the install-test values.
VALUES_ARGS+=(-f "$OVERLAY_DIR/$CHART.yaml")
VALUES_LABEL="$VALUES_LABEL + ci/yaml-scalar/$CHART.yaml"
HAS_OVERLAY=1
fi
echo "::group::$CHART ($VALUES_LABEL)"
RENDER="$WORK/$CHART.manifest.yaml"
if ! helm template "$RELEASE" "$ROOT/$CHART" \
${VALUES_ARGS[@]+"${VALUES_ARGS[@]}"} >"$RENDER" 2>"$WORK/err"; then
sed 's/^/ /' "$WORK/err"
echo "::endgroup::"
echo "::error title=yaml-scalar $CHART::$CHART does not render with these values. If the error is a YAML parse error naming a template line, that IS the finding this test exists for: a value reached a scalar unquoted. Reproduce: helm template $RELEASE $CHART $(printf '%s ' "${VALUES_ARGS[@]+${VALUES_ARGS[@]}}")"
FAILED="$FAILED $CHART"
continue
fi
# The document split, the only textual step. A wrong split cannot produce a
# false pass: the result stops being parsable and helm exits 1.
DOCS="$WORK/$CHART.docs.yaml"
awk '
BEGIN { print "docs:"; started = 0 }
/^---[ \t]*$/ { print " -"; started = 1; next }
{
if (!started) { print " -"; started = 1 }
if (length($0) == 0) print ""; else print " " $0
}
' "$RENDER" >"$DOCS"
JSON="$WORK/$CHART.json"
if ! helm template y2j "$YAML2JSON" -f "$DOCS" 2>"$WORK/err" \
| awk '/^\{/ { print; exit }' >"$JSON"; then
sed 's/^/ /' "$WORK/err"
echo "::endgroup::"
echo "::error title=yaml-scalar $CHART::the rendered manifests of $CHART could not be re-read as YAML by ci/yaml2json. That is a parse failure, not a scalar finding."
FAILED="$FAILED $CHART"
continue
fi
if [ ! -s "$JSON" ]; then
echo "::endgroup::"
echo "::error title=yaml-scalar $CHART::ci/yaml2json produced no JSON for $CHART - the manifests were not re-read, so nothing was checked."
FAILED="$FAILED $CHART"
continue
fi
# --- The comparison --------------------------------------------------------
# Standard library only, and it never decides the exit code itself: it writes
# "VERDICT|detail" lines and the loop below decides, so a verdict can never
# be lost to a swallowed status.
set +e
python3 - "$CHART" "$JSON" >"$WORK/$CHART.out" 2>"$WORK/$CHART.pyerr" <<'PY'
import json, sys
chart, path = sys.argv[1], sys.argv[2]
docs = json.load(open(path)).get("docs") or []
lines = []
def name_of(doc):
meta = doc.get("metadata")
if isinstance(meta, dict) and meta.get("name"):
return str(meta["name"])
return "<unnamed %s>" % doc.get("kind", "object")
def typename(value):
return {bool: "bool", int: "int", float: "float", type(None): "null",
dict: "mapping", list: "list"}.get(type(value), type(value).__name__)
def pod_templates(node, path="$"):
"""Any value of a key named "template" carrying a spec with containers.
Found by walking, so a CronJob's spec.jobTemplate.spec.template needs no
special case and neither does whatever workload kind a chart adds next."""
out = []
if isinstance(node, dict):
for key, value in node.items():
here = "%s.%s" % (path, key)
if (key == "template" and isinstance(value, dict)
and isinstance(value.get("spec"), dict)
and "containers" in value["spec"]):
out.append((here, value))
out.extend(pod_templates(value, here))
elif isinstance(node, list):
for i, value in enumerate(node):
out.extend(pod_templates(value, "%s[%d]" % (path, i)))
return out
def metadatas(node, path="$"):
"""Every metadata mapping anywhere in the render - top level and inside pod
templates both, because labels and annotations exist at both levels."""
out = []
if isinstance(node, dict):
for key, value in node.items():
here = "%s.%s" % (path, key)
if key == "metadata" and isinstance(value, dict):
out.append((here, value))
out.extend(metadatas(value, here))
elif isinstance(node, list):
for i, value in enumerate(node):
out.extend(metadatas(value, "%s[%d]" % (path, i)))
return out
# --- 1. CronJob schedules ---------------------------------------------------
for doc in docs:
if not isinstance(doc, dict) or doc.get("kind") != "CronJob":
continue
where = "CronJob %s .spec.schedule" % name_of(doc)
spec = doc.get("spec") if isinstance(doc.get("spec"), dict) else {}
schedule = spec.get("schedule", "<absent>")
if isinstance(schedule, str) and schedule.strip():
lines.append("OK|%s: %s is the string %r" % (chart, where, schedule))
elif schedule == "<absent>":
lines.append("FAIL|%s: %s is missing entirely - a CronJob without a "
"schedule is rejected by the API server" % (chart, where))
else:
lines.append("FAIL|%s: %s rendered as %s (%r), not a string. A schedule "
"is operator free text and has to leave the template "
"through | quote; without it YAML re-reads the content and "
"a schedule beginning with a digit survives by luck while "
"one beginning with * or @ does not"
% (chart, where, typename(schedule), schedule))
# --- 2. env values ----------------------------------------------------------
for where, template in pod_templates({"docs": docs}):
spec = template.get("spec") if isinstance(template.get("spec"), dict) else {}
for kind in ("initContainers", "containers"):
for container in spec.get(kind) or []:
if not isinstance(container, dict):
continue
cname = container.get("name", "<unnamed>")
for entry in container.get("env") or []:
if not isinstance(entry, dict) or "value" not in entry:
continue # valueFrom entries carry no literal to check
value = entry["value"]
tag = "env %s of container %s at %s" % (
entry.get("name", "<unnamed>"), cname, where)
if isinstance(value, str):
lines.append("OK|%s: %s is a string" % (chart, tag))
else:
lines.append("FAIL|%s: %s rendered as %s (%r), not a string. "
"Kubernetes types env[].value as a string and "
"the API server rejects the whole manifest with "
"\"cannot unmarshal %s into Go struct field "
"EnvVar.value of type string\" - at apply time, "
"not in the render. The fix is | quote at the "
"render site; template-chart carries the form"
% (chart, tag, typename(value), value,
"number" if not isinstance(value, bool) else "bool"))
# --- 3. labels and annotations ----------------------------------------------
for where, meta in metadatas({"docs": docs}):
for field in ("labels", "annotations"):
block = meta.get(field)
if not isinstance(block, dict):
continue
for key, value in block.items():
tag = "%s[%r] at %s" % (field, key, where)
if isinstance(value, str):
lines.append("OK|%s: %s is a string" % (chart, tag))
else:
lines.append("FAIL|%s: %s rendered as %s (%r), not a string. "
"Kubernetes types %s as map[string]string, so the "
"API server rejects the object - the render stays "
"green. The fix is | quote at the render site"
% (chart, tag, typename(value), value, field))
for line in lines:
print(line)
PY
RC=$?
set -e
if [ "$RC" -gt 1 ]; then
sed 's/^/ /' "$WORK/$CHART.pyerr"
echo "::endgroup::"
echo "::error title=yaml-scalar $CHART::the scalar comparison itself failed for $CHART (python exit $RC). Treated as a failure, never as a pass."
FAILED="$FAILED $CHART"
continue
fi
CHART_FAIL=""
CHART_OK=0
while IFS='|' read -r VERDICT DETAIL; do
case "$VERDICT" in
OK)
CHART_OK=$((CHART_OK + 1))
echo " ok $DETAIL"
;;
FAIL)
CHART_FAIL=1
echo " FAIL $DETAIL"
echo "::error title=yaml-scalar $CHART::$DETAIL"
;;
*)
[ -n "$VERDICT" ] && echo " $VERDICT $DETAIL"
;;
esac
done <"$WORK/$CHART.out"
echo "::endgroup::"
TOTAL_ASSERTIONS=$((TOTAL_ASSERTIONS + CHART_OK))
if [ -n "$CHART_FAIL" ]; then
FAILED="$FAILED $CHART"
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | **failed** | $CHART_OK |"$'\n'
echo "$CHART: FAILED"
elif [ "$CHART_OK" -gt 0 ]; then
CHECKED="$CHECKED $CHART"
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | ok | $CHART_OK |"$'\n'
echo "$CHART: ok ($CHART_OK scalar(s) checked)"
elif [ -n "$HAS_OVERLAY" ]; then
FAILED="$FAILED $CHART"
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | **failed** | 0 |"$'\n'
echo "::error title=yaml-scalar $CHART::$CHART has an overlay at ci/yaml-scalar/$CHART.yaml but its render carried no schedule, no env value and no label or annotation at all, so nothing was checked. That overlay exists only to make such a scalar render; a green tick here would be a tick for a check that ran over nothing."
echo "$CHART: FAILED (overlay present, nothing asserted)"
else
NOT_EXERCISED="$NOT_EXERCISED $CHART"
echo "$CHART: nothing of the three kinds rendered - nothing to check."
fi
done
# --- Report ------------------------------------------------------------------
count() { set -- $1; echo "$#"; }
listing() { set -- $1; [ "$#" -eq 0 ] && { echo "–"; return; }; printf '`%s`' "$1"; shift; for C in "$@"; do printf ', `%s`' "$C"; done; echo; }
SUMMARY="${GITHUB_STEP_SUMMARY:-/dev/null}"
{
echo "### YAML-scalar render test (Gitea #11)"
echo
echo "Every \`CronJob.spec.schedule\`, every \`env[].value\` and every label and"
echo "annotation value in a chart's render, held against the type Kubernetes requires"
echo "for it. A value rendered without \`| quote\` either aborts the render with an"
echo "error pointing at the template instead of at the value, or arrives as a bool or"
echo "an integer and is rejected by the API server at apply time - never in the render."
echo
echo "| Result | Count | Charts |"
echo "|---|---|---|"
echo "| Scalars checked | $(count "$CHECKED") | $(listing "$CHECKED") |"
echo "| Nothing of the kind rendered | $(count "$NOT_EXERCISED") | $(listing "$NOT_EXERCISED") |"
echo "| Failed | $(count "$FAILED") | $(listing "$FAILED") |"
echo
echo "$TOTAL_ASSERTIONS scalar(s) verified in total."
if [ -n "$SUMMARY_ROWS" ]; then
echo
echo "| Chart | Result | Checked |"
echo "| --- | --- | --- |"
printf '%s' "$SUMMARY_ROWS"
fi
} >>"$SUMMARY"
echo "===================== yaml scalar render test ======================"
echo "checked: $(count "$CHECKED")${CHECKED:+ -$CHECKED}"
echo "not exercised: $(count "$NOT_EXERCISED")${NOT_EXERCISED:+ -$NOT_EXERCISED}"
echo "failed: $(count "$FAILED")${FAILED:+ -$FAILED}"
echo "scalars verified: $TOTAL_ASSERTIONS"
echo "==================================================================="
if [ -n "$FAILED" ]; then
echo "ERROR: a value that has to arrive as a string did not, in:$FAILED" >&2
exit 1
fi
if [ "$TOTAL_ASSERTIONS" -eq 0 ]; then
echo "ERROR: not a single scalar was checked in this run. Either no chart renders a schedule, an env value, a label or an annotation any more, or the overlays under ci/yaml-scalar/ no longer reach their objects. A pass over zero assertions is not a pass." >&2
exit 1
fi
+25
View File
@@ -0,0 +1,25 @@
# Selector-match overlay for paperless-ngx (Gitea #101).
#
# Its one job: make the consume CronJob render, so that the required podAffinity
# it carries is actually held against the StatefulSet's pod labels. The template
# sits behind smbConnection.enabled, false in the chart defaults and in
# ci/paperless-ngx/test-values.yaml, so without this overlay the affinity term
# exists in the repository and in no render - and the test would pass over it.
#
# What it guards: the term selects the StatefulSet pod by a label written in
# ANOTHER template (paperless-server.sfs.yaml). Renaming that label leaves both
# files rendering perfectly while the mover can never be scheduled again, one
# Pending pod per schedule, nothing in the render to show for it.
#
# Kept separate from ci/yaml-scalar/paperless-ngx.yaml although both switch SMB
# on: that one feeds the CronJob a hostile SCHEDULE and hostile env values for a
# different test, and folding the two would make each test's failure depend on
# the other's fixture.
smbConnection:
enabled: true
server: "//smb.ci.local/scans"
secrets:
# Satisfies the required() behind the SMB PersistentVolume's
# nodeStageSecretRef. Nothing is mounted in a render.
smbConnectionSecretRef: op://ci/paperless-smb
+53
View File
@@ -0,0 +1,53 @@
# YAML-scalar overlay for paperless-ngx (Gitea #11).
#
# THIS FILE IS THE TEST CASE the issue asked for, and its first job is to make
# an object exist at all. templates/paperless-consume.cronjob.yaml sits behind
# smbConnection.enabled, which is false in the chart defaults and false in
# ci/paperless-ngx/test-values.yaml - deliberately, because the install-test has
# no SMB CSI driver. So before this overlay NOTHING in the repository rendered
# that CronJob: not lint, not the render tests, not the install-test. That is
# the whole reason an unquoted schedule survived in a shipped chart, and it is
# why the fix comes with a render test rather than with a fixture change.
#
# The values below are hostile on purpose. Against the chart before 13.0.0 this
# overlay does not produce a finding - it produces an aborted render, which this
# test reports as the finding it is:
#
# Error: YAML parse error on paperless-ngx/templates/paperless-consume.cronjob.yaml:
# error converting YAML to JSON: yaml: line 6: did not find expected
# alphabetic or numeric character
smbConnection:
enabled: true
server: "//smb.ci.local/scans"
# A schedule beginning with "*" - the commonest cron notation there is, and
# the one that YAML reads as an alias indicator. Written here as a quoted
# string, which is exactly how an operator would write it and exactly what
# does NOT help: Helm passes the content through, and without | quote in the
# template the star stands bare in the manifest again.
cronJob: "*/15 * * * *"
secrets:
# The CronJob's SMB volume has a required() secret ref behind it; the
# PersistentVolume names it as nodeStageSecretRef. Nothing is mounted in a
# render, so no fixture Secret is needed - only a value that satisfies the
# required().
smbConnectionSecretRef: op://ci/paperless-smb
paperless:
# The silent half of the genus, on the chart's additive env block: both of
# these are STRINGS in this file and both are schema-valid, and before 13.0.0
# the first one aborted the render while the second arrived in the manifest as
# a YAML bool - which the API server rejects at apply time with "cannot
# unmarshal bool into Go struct field EnvVar.value of type string", so the
# render test is the only place it can be caught cheaply.
env:
- name: PAPERLESS_STAR_VALUE
value: "*never-interpreted*"
- name: PAPERLESS_BOOLEAN_LOOKING_VALUE
value: "true"
# The variable the whole env block was asked for (Gitea #112): the log off
# the NFS-backed RWX data volume and onto the /tmp emptyDir this pod already
# mounts. Harmless in a render, and it keeps the worked example honest by
# being the one an operator will actually copy.
- name: PAPERLESS_LOGGING_DIR
value: "/tmp/paperless-log"
+1 -1
View File
@@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/paperless-ngx/paperless-ngx/dev/src-ui/s
type: application
version: 12.3.0+up3.2.1
version: 13.0.0+up3.2.1
appVersion: 3.2.1
+70 -17
View File
@@ -8,6 +8,9 @@
{{- $valkeySvc := printf "%s-valkey-service" .Release.Name -}}
{{- $pre := default (list) $scripts.preConsumptionScripts -}}
{{- $post := default (list) $scripts.postConsumptionScripts -}}
{{- $cron := fromYaml (include "paperless-ngx.cron.scheduling" .Values.smbConnection.cron) -}}
{{- $cronPinned := $cron.pinToServerNode -}}
{{- $cronDeadline := include "paperless-ngx.formatNumber" $cron.activeDeadlineSeconds -}}
{{- $httpRoute := .Values.httpRoute | default dict -}}
{{- $hostnames := $httpRoute.hostnames | default list -}}
{{- if not $hostnames -}}
@@ -184,15 +187,14 @@ WHAT THE 1PASSWORD ITEMS MUST CONTAIN
here, so a wrong field name surfaces as a mount failure on the CronJob's
pod — a describe on that pod, never a missing variable.
{{- end }}
{{- if .Values.secrets.oidcSecretRef }}
secrets.oidcSecretRef IS SET AND IS READ BY NOTHING
"{{ .Values.secrets.oidcSecretRef }}" — no template in this chart references
that value. No OnePasswordItem is created for it and no Secret appears. The
OIDC client id travels as plain env from oidc.provider.clientId, and no
client secret is configured at all (see the OIDC section below). Whatever
you put in that item has no effect on this release.
{{- end }}
{{- /*
The warning that used to stand here - "secrets.oidcSecretRef IS SET AND IS READ
BY NOTHING" - is gone with the value itself (13.0.0, Gitea #11). It cannot be
printed any more and would not be reached: the key is out of values.schema.json
too, so a release that still sets it fails BEFORE any template renders, with
"additional properties 'oidcSecretRef' not allowed". The fix is to delete the
key; values.yaml says why it is not coming back.
*/}}
CHANGING A SECRET'S CONTENT RESTARTS NOTHING
Helm never sees the content of a OnePasswordItem, so it cannot hash it, and
@@ -340,12 +342,44 @@ HOW DOCUMENTS GET IN
move, not a copy: once the job has run, the share no longer holds the file
and the only copy is on the consume volume.
One thing to know before you edit that schedule: it is rendered into the
manifest unquoted, so a schedule that STARTS with "*" ("*/15 * * * *")
aborts the render with a YAML parse error on this template — YAML reads the
leading * as an alias. Quoting it in your values file does not help, the
quotes are gone by then. Use an equivalent that starts with a digit
("5,20,35,50 * * * *"); a "*" anywhere but in first position is fine.
The schedule is quoted on the way into the manifest since 13.0.0, so any
cron notation works — one starting with "*" or an "@daily" macro included.
Before that such a schedule aborted the render with a YAML parse error
pointing at the template, and the advice here was to avoid it; if you are
carrying a workaround schedule from that era ("5,20,35,50 * * * *" instead
of every 15 minutes), you can write what you meant now.
{{- if $cronPinned }}
PLACEMENT: the mover runs on the NODE OF THE STATEFULSET POD, via a
required podAffinity on its label (Gitea #101). Without that it lands
anywhere, and because it mounts the consume volume — RWX, from Longhorn —
the kubelet of the StatefulSet's node then retries the mount every two
minutes with "no Pending workload pods for volume ... to be mounted". It
was measured at 291 such events in 10 hours.
THE PRICE, and it is the one thing to know before scaling down: with no
StatefulSet pod the mover CANNOT BE SCHEDULED. scaleDown: true suspends
this CronJob outright, so that case is covered — but paperless.replicas: 0
with SMB still on is not, and there the job fails once per schedule after
activeDeadlineSeconds ({{ $cronDeadline }}s) with reason DeadlineExceeded. That is
deliberate: a bounded, visible failure rather than a Pending pod per
schedule that nobody sees. Switch smbConnection.cron.pinToServerNode to
false to go back to unpinned placement — and to the mount errors.
What the pinning does NOT fix: the consume volume stays RWX with a
share-manager behind it. Co-locating the two consumers removes the
cross-node mount, not the second consumer.
{{- else }}
PLACEMENT IS OFF (smbConnection.cron.pinToServerNode: false). The mover
will be scheduled wherever there is room, and when that is not the node of
the StatefulSet pod, Longhorn refuses to mount the shared RWX consume
volume there: the kubelet of the StatefulSet's node retries every two
minutes with "no Pending workload pods for volume ... to be mounted"
(Gitea #101, measured at 291 events in 10 hours). The errors stop when the
StatefulSet pod is deleted and comes back Pending, and return with the next
firing of this job.
{{- end }}
The job has NO probes (the documented exception to the chart conventions),
and it cannot fail usefully either: with nothing to move it exits 0, which
@@ -432,8 +466,13 @@ OIDC IS ON, AND IT IS PKCE-ONLY
{{- if not .Values.oidc.provider.tokenAuthMethod }}
oidc.provider.tokenAuthMethod is unset. Since the allauth update in 3.0 some
providers need it spelled out (client_secret_basic); the symptom is a login
that dies at the callback with invalid_client.
providers need it spelled out, and the symptom when they do is a login that
dies at the callback with invalid_client. The only value this chart accepts is
"none" — it states the public client explicitly. A method that authenticates
WITH a client secret cannot work here and is rejected by the schema (13.0.0,
Gitea #11): this release configures no client secret and cannot, because
paperless reads the whole OIDC configuration out of one json environment
variable with no _FILE variant.
{{- end }}
{{- if or .Values.oidc.autoLoginWithProvider .Values.oidc.disableRegularLogin }}
@@ -453,6 +492,20 @@ paperless.scripts IS SET AND DOES NOTHING
PAPERLESS_PRE_CONSUME_SCRIPT nor PAPERLESS_POST_CONSUME_SCRIPT is set on the
container. These values render no object and change no behaviour.
{{- end }}
{{- if .Values.paperless.env }}
paperless.env ADDS {{ len .Values.paperless.env }} VARIABLE(S) — AND AN OVERRIDE ONLY BITES AT INSTALL
{{ range $i, $e := .Values.paperless.env }}{{ if $i }}, {{ end }}{{ $e.name }}{{ end }}
These are appended AFTER the variables the chart sets itself, so at INSTALL an
entry repeating one of those names wins. On an upgrade of an EXISTING release
the same entry is lost: Kubernetes merges env by name, the pair collapses to
the chart's own value, and helm upgrade still reports success. Read the value
back off the StatefulSet, never off the upgrade's exit code:
kubectl -n {{ .Release.Namespace }} get statefulset {{ .Release.Name }}-{{ .Chart.Name }}-sfs \
-o jsonpath='{.spec.template.spec.containers[0].env}'
{{- end }}
{{- if gt (int $replicas) 1 }}
paperless.replicas IS {{ $replicas }} — EXPECT ONE POD TO WORK AND THE REST TO WAIT
+38
View File
@@ -340,3 +340,41 @@ The same three fallbacks for the SMB consume-mover CronJob. It has no probes
{{- $merged := fromYaml (include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "smbConnection.cron.resources")) -}}
{{- include "paperless-ngx.resources" $merged -}}
{{- end -}}
{{/*
The whole smbConnection.cron block, merged onto its defaults (YAML; consume
with fromYaml).
Input: .Values.smbConnection.cron (may be nil, empty or partial).
WHY EVERY READ OF THAT BLOCK GOES THROUGH HERE. The CronJob template used to
dereference .Values.smbConnection.cron.<key> directly, so an override that had
erased the block - the leftover childless
cron:
that issue #61 is about - did not fall back to the defaults the way this chart
promises everywhere else. It aborted the render with
nil pointer evaluating interface {}.securityContext
naming an internal template path and neither the value nor the cause. Measured
on origin/main before 13.0.0, so this is a fix and not a precaution.
The scalar defaults below MUST stay in sync with the smbConnection.cron block
in values.yaml, which stays the documented, user-facing place for them; this
copy is only the fallback. The sub-blocks (securityContext, resources, env)
keep their own defaults in the helpers above and are only passed through here.
An explicit "pinToServerNode: false" survives this, which is the whole reason
it is not a "default true" pipe: sprig's default would read false as unset and
silently turn the pinning back on - the one value whose explicit false has to
be obeyed, since it exists to switch off a hard placement rule.
*/}}
{{- define "paperless-ngx.cron.scheduling" -}}
{{- $defaults := dict
"pinToServerNode" true
"startingDeadlineSeconds" 600
"activeDeadlineSeconds" 3600 -}}
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "smbConnection.cron") -}}
{{- end -}}
@@ -1,26 +1,165 @@
{{- if .Values.smbConnection.enabled }}
{{- /*
The cron block, merged onto its defaults, so that an override which erased it
falls back instead of aborting with a nil-pointer error naming an internal
template path - which is what every direct .Values.smbConnection.cron.<key>
read did before 13.0.0. The helper carries the measurement.
*/}}
{{- $cron := fromYaml (include "paperless-ngx.cron.scheduling" .Values.smbConnection.cron) -}}
apiVersion: batch/v1
kind: CronJob
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-consume-cronjob
spec:
schedule: {{ .Values.smbConnection.cronJob }}
{{- /*
QUOTED, AND THAT IS THE WHOLE OF GITEA #11's FIRST HALF. The default
"0 0,12 * * *" renders safely only because it begins with a digit. The most
common cron notation of all - a star, a slash and an interval, every 15
minutes - begins with "*", which YAML reads as an alias indicator, and the
render ABORTS:
(and the notation cannot be written out in this comment, because the two
characters that start it would end a Go template comment right here)
Error: YAML parse error on paperless-ngx/templates/paperless-consume.cronjob.yaml:
error converting YAML to JSON: yaml: line 6: did not find expected
alphabetic or numeric character
The trap is that quoting it in the VALUES does not help: Helm passes the
string's content through, and the star stands bare in the manifest again. So
the operator writes a correctly quoted value and gets an error that points at
this template instead of at their value. "@daily" fails the same way ("@" is
a reserved YAML indicator).
ci/run-yaml-scalar-render-test.sh is the test that would have caught it, and
ci/yaml-scalar/paperless-ngx.yaml is the overlay that feeds it a star-leading
schedule - needed because this whole template sits behind
smbConnection.enabled, which is false in every other fixture, so nothing else
in the repository renders this object at all.
*/}}
schedule: {{ .Values.smbConnection.cronJob | quote }}
{{- /*
Forbid, hard-wired rather than a value: two movers running at once race on
the same files ("mv /mnt/smb/*.pdf" twice over one directory), and the loser
reports a file that is already gone. There is no configuration in which
overlapping runs of this job are wanted, so there is nothing to configure.
*/}}
concurrencyPolicy: Forbid
{{- /*
Bounds how late a missed run may still start, and it is here for a second
reason that has nothing to do with lateness: while the CronJob is suspended
(scaleDown) or its runs are being suppressed, the controller counts missed
schedules, and above 100 it STOPS SCHEDULING ALTOGETHER with only
"Cannot determine if job needs to be started: too many missed start times" in
the controller log. With this field set, the controller only counts misses
inside this window, so that state cannot be reached. The cost is stated in
values.yaml: a run that could not start inside the window is skipped, not
queued.
*/}}
startingDeadlineSeconds: {{ include "paperless-ngx.formatNumber" $cron.startingDeadlineSeconds }}
{{- if .Values.scaleDown }}
suspend: true
{{- end }}
jobTemplate:
spec:
{{- /*
THE SAFETY NET UNDER THE AFFINITY BELOW (Gitea #101). activeDeadlineSeconds
counts from the job's start, not from the pod's, so it also bounds a pod
that NEVER gets scheduled: when the StatefulSet pod is absent and the
required affinity cannot be satisfied, the job fails after this many
seconds with reason DeadlineExceeded instead of sitting Pending forever
and gaining a sibling every schedule. That turns the one new failure mode
this issue's fix could introduce into a visible, self-clearing one -
failedJobsHistoryLimit (Kubernetes' default, 1) keeps the evidence.
*/}}
activeDeadlineSeconds: {{ include "paperless-ngx.formatNumber" $cron.activeDeadlineSeconds }}
template:
spec:
{{- if $cron.pinToServerNode }}
{{- /*
GITEA #101: this pod mounts the consume volume, which is RWX and comes
from Longhorn. Without a placement rule the scheduler puts it on any
node, and Longhorn's CSI driver then refuses the mount on the node
where the StatefulSet pod runs:
MountVolume.SetUp failed ... code = Aborted desc = no Pending
workload pods for volume pvc-... to be mounted:
map[Running:[...-sfs-0] Succeeded:[...-consume-cronjob-...]]
Measured at every two minutes, 291 events in 10 hours, in both
releases that have SMB switched on; the release with
smbConnection.enabled: false and therefore no CronJob produced none of
them on identically built volumes.
WHY podAffinity AND NOT nodeSelector: the StatefulSet pod is not
pinned either - it sits wherever the scheduler put it and moves on
every restart. A static nodeSelector would be correct exactly until
then and silently wrong afterwards. The affinity follows the pod.
WHY required AND NOT preferred: preferred falls back to another node
in the very situation this exists for, which reproduces the bug more
rarely and therefore harder to find. The fallback is what went
unnoticed for ten hours.
The label is the one the StatefulSet puts on its pod template
(paperless-server.sfs.yaml, metadata.labels.app) and is read off the
running pod, not assumed.
WHAT THIS DOES NOT FIX: the volume stays RWX and keeps its
share-manager; co-locating the two consumers removes the CROSS-NODE
attach, not the second consumer. The design that removes it is moving
the mv into the StatefulSet pod, which cannot be done for the running
installations because RWX cannot be taken back off an existing PVC.
Gitea #101 names it as the alternative and not as the proposal.
*/}}
affinity:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
topologyKey: kubernetes.io/hostname
{{- end }}
automountServiceAccountToken: false
securityContext:
{{- include "paperless-ngx.cron.podSecurityContext" .Values.smbConnection.cron.securityContext | nindent 12 }}
{{- include "paperless-ngx.cron.podSecurityContext" $cron.securityContext | nindent 12 }}
containers:
- name: paperless-consume-cronjob
image: "{{ .Values.paperless.busybox.image.repository }}:{{ .Values.paperless.busybox.image.tag }}"
imagePullPolicy: IfNotPresent
securityContext:
{{- include "paperless-ngx.cron.containerSecurityContext" .Values.smbConnection.cron.securityContext | nindent 16 }}
{{- include "paperless-ngx.cron.containerSecurityContext" $cron.securityContext | nindent 16 }}
{{- if $cron.env }}
env:
{{- range $value := $cron.env }}
{{- if $value.value }}
{{- /*
quote is required, not cosmetic: a value like "8080" or "true"
renders unquoted as a YAML number/bool, and the API server then
rejects the whole manifest with "cannot unmarshal number into
Go struct field EnvVar.value of type string". A template
comment and not a "#" one, because a "#" line here is part of
the rendered manifest.
*/}}
- name: {{ $value.name | quote }}
value: {{ tpl $value.value $ | quote }}
{{- else if $value.valueFrom }}
{{- if $value.valueFrom.secretKeyRef }}
- name: {{ $value.name }}
valueFrom:
secretKeyRef:
name: {{ tpl $value.valueFrom.secretKeyRef.name $ | quote }}
key: {{ tpl $value.valueFrom.secretKeyRef.key $ | quote }}
{{- else if $value.valueFrom.configMapKeyRef }}
- name: {{ $value.name }}
valueFrom:
configMapKeyRef:
name: {{ tpl $value.valueFrom.configMapKeyRef.name $ | quote }}
key: {{ tpl $value.valueFrom.configMapKeyRef.key $ | quote }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
command:
- /bin/sh
- -c
@@ -33,7 +172,7 @@ spec:
mountPath: "/mnt/consume"
readOnly: false
resources:
{{- include "paperless-ngx.cron.effectiveResources" .Values.smbConnection.cron.resources | nindent 16 }}
{{- include "paperless-ngx.cron.effectiveResources" $cron.resources | nindent 16 }}
restartPolicy: Never
volumes:
- name: smb-share
@@ -42,4 +181,4 @@ spec:
- name: consume-folder
persistentVolumeClaim:
claimName: {{ .Release.Name }}-{{ .Chart.Name }}-consume-volume-claim
{{- end }}
{{- end }}
@@ -204,23 +204,41 @@ spec:
- name: PAPERLESS_DISABLE_REGULAR_LOGIN
value: "{{ .Values.oidc.disableRegularLogin }}"
{{- end }}
{{- /*
QUOTED SINCE 13.0.0, and it is a fix rather than a tidy-up (Gitea
#112, same genus as #11). This block is the chart's additive env
interface, and until 13.0.0 it rendered its operand bare. Two
measured consequences, both of them the operator's value arriving
as something else:
- a value beginning with "*" aborted the render, with the error
pointing at THIS template rather than at the value;
- "true" and "0755" rendered as a YAML bool and an integer, and
the API server rejects those with "cannot unmarshal number
into Go struct field EnvVar.value of type string" - the chart
renders, the install fails.
The quoted form is the repository's documented one
(template-chart/templates/xxx.deployment.yaml carries the rule);
this chart was one of seven that had not been brought to it.
*/}}
{{- range $value := .Values.paperless.env }}
{{- if $value.value }}
- name: {{ $value.name }}
value: {{ tpl $value.value $ }}
- name: {{ $value.name | quote }}
value: {{ tpl $value.value $ | quote }}
{{- else if $value.valueFrom }}
{{- if $value.valueFrom.secretKeyRef }}
- name: {{ $value.name }}
valueFrom:
secretKeyRef:
name: {{ tpl $value.valueFrom.secretKeyRef.name $ }}
key: {{ tpl $value.valueFrom.secretKeyRef.key $ }}
name: {{ tpl $value.valueFrom.secretKeyRef.name $ | quote }}
key: {{ tpl $value.valueFrom.secretKeyRef.key $ | quote }}
{{- else if $value.valueFrom.configMapKeyRef }}
- name: {{ $value.name }}
valueFrom:
configMapKeyRef:
name: {{ tpl $value.valueFrom.configMapKeyRef.name $ }}
key: {{ tpl $value.valueFrom.configMapKeyRef.key $ }}
name: {{ tpl $value.valueFrom.configMapKeyRef.name $ | quote }}
key: {{ tpl $value.valueFrom.configMapKeyRef.key $ | quote }}
{{- end }}
{{- end }}
{{- end }}
+10 -2
View File
@@ -23,8 +23,16 @@ metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-ingress
annotations:
cert-manager.io/cluster-issuer: {{ required "A Cluster-Issuer must be provided" .Values.ingress.clusterIssuer }}
nginx.ingress.kubernetes.io/proxy-body-size: {{ .Values.paperless.files.maxUploadFileSize }}
nginx.org/client-max-body-size: {{ .Values.paperless.files.maxUploadFileSize }}
{{- /*
Quoted for the same reason as the env block in the StatefulSet and the
schedule in the CronJob (Gitea #11's genus): an annotation value must be a
string, maxUploadFileSize is free operator text, and a unit-less "100" -
schema-valid, since the schema types it as a string - renders bare as an
integer and the API server rejects the whole Ingress. The default "10m"
happens to be safe, which is exactly how this survived.
*/}}
nginx.ingress.kubernetes.io/proxy-body-size: {{ .Values.paperless.files.maxUploadFileSize | quote }}
nginx.org/client-max-body-size: {{ .Values.paperless.files.maxUploadFileSize | quote }}
{{- with .Values.ingress.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
+19 -6
View File
@@ -69,9 +69,6 @@
},
"smbConnectionSecretRef": {
"type": ["string", "null"]
},
"oidcSecretRef": {
"type": ["string", "null"]
}
}
},
@@ -279,13 +276,28 @@
"type": ["string", "null"]
},
"cronJob": {
"description": "Cron schedule of the consume-mover job.",
"description": "Cron schedule of the consume-mover job. Deliberately without a pattern: cron syntax is richer than a regex here would admit, and the failure this value used to have was not a wrong schedule but a correct one rendered bare into the manifest - fixed at the render site with | quote (Gitea #11), not by narrowing what may be written.",
"type": ["string", "null"]
},
"cron": {
"type": "object",
"additionalProperties": false,
"properties": {
"pinToServerNode": {
"description": "Place the consume-mover pod on the node that carries the StatefulSet pod, via a required podAffinity on its label (Gitea #101). Default true: the mover mounts the RWX consume volume, and Longhorn's CSI driver refuses that mount unless it finds a workload pod in Pending, so a mover on another node leaves the StatefulSet node's kubelet retrying every two minutes. false restores the unpinned behaviour and with it those mount errors.",
"type": ["boolean", "null"]
},
"startingDeadlineSeconds": {
"description": "How late a missed run may still start. Also bounds the CronJob controller's missed-schedule count, which above 100 stops this job being scheduled at all, reported only in the controller's own log.",
"type": ["integer", "null"]
},
"activeDeadlineSeconds": {
"description": "Hard stop for one mover run, counted from the job's start, so it also bounds a pod that never gets scheduled - with pinToServerNode on and no StatefulSet pod, the job fails with DeadlineExceeded instead of leaving a Pending pod per schedule.",
"type": ["integer", "null"]
},
"env": {
"$ref": "#/definitions/env"
},
"securityContext": {
"$ref": "#/definitions/securityContext"
},
@@ -326,8 +338,9 @@
"type": ["string", "null"]
},
"tokenAuthMethod": {
"description": "Explicit token endpoint auth method (e.g. client_secret_basic). Some providers need this since the allauth update in paperless-ngx 3.0.",
"type": ["string", "null"]
"description": "Explicit token endpoint auth method. Restricted to 'none' in 13.0.0 (Gitea #11): this chart registers a PUBLIC client with PKCE and configures no client secret, so a method that authenticates with one cannot work - it produced a login that died at the callback with invalid_client while reading like a working secret setup. null omits the key; 'none' states the public client explicitly, which some providers need spelled out since the allauth update in paperless-ngx 3.0.",
"type": ["string", "null"],
"enum": ["none", null]
}
}
}
+121 -5
View File
@@ -13,7 +13,26 @@ secrets:
databaseSecretRef: null
paperlessSecretRef: null
smbConnectionSecretRef: null
oidcSecretRef: null
# REMOVED IN 13.0.0: secrets.oidcSecretRef (Gitea #11). It is gone from the
# schema too, so a release that still sets it now fails to render with
# additional properties 'oidcSecretRef' not allowed
# instead of being ignored - DELETE THE KEY from your values, nothing else is
# needed. It never did anything: no template read it, no OnePasswordItem was
# created for it, and whatever the item held never reached the pod.
#
# It is not coming back as a feature, and the reason is in paperless itself:
# the whole OIDC configuration travels in ONE json environment variable
# (SOCIALACCOUNT_PROVIDERS = json.loads(os.getenv(
# "PAPERLESS_SOCIALACCOUNT_PROVIDERS", "{}"))), and there is no _FILE variant
# of it. A secretKeyRef cannot substitute a fragment INTO a json variable, so
# an honest implementation would have to either inline the secret in the pod
# spec or push the complete json into the vault by hand - and the second one
# makes every required() in the oidc block below dead wood.
#
# The operational question behind it is answered rather than dodged: this
# chart registers a PUBLIC client with PKCE, Keycloak (Gitea #83) can mark a
# client public per client, and so no provider constraint forces a
# confidential client here. See the oidc block below.
ingress:
# Set to false to omit the Ingress entirely. NEW in the chart version that
@@ -179,7 +198,38 @@ paperless:
timeoutSeconds: 5
# Extra environment variables, appended to the ones the chart always sets.
# Values are rendered through tpl; value, secretKeyRef and configMapKeyRef
# forms are supported.
# forms are supported. paperless-ngx reads around 80 PAPERLESS_* variables and
# this chart maps a handful of them explicitly - everything else goes here.
#
# env:
# - name: PAPERLESS_LOGGING_DIR
# value: /tmp/paperless-log
#
# ADDITIVE MEANS ADDITIVE, and the timing matters: an entry repeating a name
# the chart already sets reaches the pod twice, and at INSTALL the later one -
# yours - wins. On an EXISTING release the same entry is lost, because
# Kubernetes merges env by name and the pair collapses to the chart's own
# value, while helm upgrade still reports success. Read an override back off
# the StatefulSet, never off the upgrade's exit code.
#
# VALUES MUST BE STRINGS (the schema enforces it) and they are quoted on the
# way out since 13.0.0. Before that a value beginning with "*" aborted the
# render and "true" or "0755" arrived as a bool or an integer, which the API
# server rejects - see the comment at the render site in
# templates/paperless-server.sfs.yaml.
#
# THE WORKED EXAMPLE, PAPERLESS_LOGGING_DIR, AND WHAT IT DOES NOT FIX
# (jk-cluster-config #360): with the data directory on an RWX volume that
# Longhorn exports over NFS, paperless' concurrent_log_handler cannot hold its
# lock file there and fails with "RuntimeError: Cannot acquire lock after 20
# attempts" - measured at 45 % of the application log across all three
# installations. Pointing PAPERLESS_LOGGING_DIR at a path under the /tmp
# emptyDir this pod already mounts takes the log off the NFS share and the
# noise stops. It does NOT fix the RWX data volume, which is the actual root
# cause and a cluster-config decision (the PVC asks for ReadWriteOnce and was
# provisioned ReadWriteMany; taking RWX back means recreating the volume).
# A log under /tmp does not survive a pod restart - no loss here, paperless
# also writes to stdout, which is what kubectl logs reads.
env: []
resources:
# Limits are optional: memory and ephemeral-storage limits default to
@@ -214,8 +264,58 @@ paperless:
smbConnection:
enabled: false
server: null
# Cron schedule of the consume mover. Quote it or not as you like - the chart
# quotes it on the way into the manifest (since 13.0.0, Gitea #11), which is
# what a schedule beginning with "*" or "@" needs. Before that such a
# schedule aborted the render with a YAML error pointing at the template.
cronJob: "0 0,12 * * *"
cron:
# Run the mover on the node that carries the StatefulSet pod (Gitea #101).
#
# WHY IT DEFAULTS TO TRUE: this pod mounts the consume volume, which is RWX
# and comes from Longhorn, and Longhorn's CSI driver refuses to mount an RWX
# volume unless it finds a workload pod in Pending - a mover pod that ran on
# a different node leaves the kubelet of the StatefulSet's node retrying
# every two minutes with "no Pending workload pods for volume ... to be
# mounted" (291 events in 10 hours, in both releases with SMB on; the
# release without the CronJob produced none).
#
# It is a switch and not a free-form affinity block on purpose: the default
# has to name the StatefulSet pod's label, that label contains the RELEASE
# NAME, and a values.yaml is never templated - so the useful default cannot
# be expressed here at all (the same property that forces
# valkey.secrets.authSecretRef to be typed out twice, further down).
#
# WHAT IT COSTS, AND IT IS A REAL COST: the affinity is required, so with
# the StatefulSet pod absent the mover cannot be scheduled. The two ways
# that happens are handled rather than left open - scaleDown: true suspends
# this CronJob outright, and activeDeadlineSeconds below fails a job that
# cannot be placed instead of letting Pending pods pile up. What stays: with
# paperless.replicas: 0 and SMB on, the mover does not run and reports one
# failed job per schedule. Set this to false to get the old, unpinned
# behaviour back - including the mount errors.
pinToServerNode: true
# How late a missed run may still start. Also keeps the CronJob controller
# from reaching "too many missed start times (> 100)", after which it stops
# scheduling this job entirely and says so only in its own log - reachable
# by leaving scaleDown: true in place for months. The cost of a window: a
# run that could not start inside it is skipped, not queued. Must stay well
# above 10s, below which the controller may never schedule at all.
startingDeadlineSeconds: 600
# Hard stop for one mover run, counted from the job's start - so it also
# bounds a pod that never gets scheduled (see pinToServerNode). Exceeding it
# shows up as a job with reason DeadlineExceeded, which is the point: a
# loud, self-clearing failure instead of a Pending pod that nobody sees.
# Raise it if a run legitimately moves more than an hour's worth of files;
# the mover copies across filesystems, so large scans are not instant.
activeDeadlineSeconds: 3600
# Extra environment variables for the mover container, same form as
# paperless.env above. Deliberately empty: the container is a busybox
# running one mv, so this exists for the odd proxy or locale variable and
# for the convention that every workload container offers the block - not
# because the mover needs configuration. Note that the cron SCHEDULE's time
# zone is the controller's and cannot be changed with a TZ entry here.
env: []
# The consume mover only shuffles files between the SMB share (mounted
# with 0777 modes) and the consume PVC, so it runs as the same non-root
# uid/gid as paperless itself with a fully hardened context.
@@ -251,9 +351,25 @@ oidc:
name: null
clientId: null
discoveryUrl: null
# Optional: explicit token endpoint auth method (e.g. client_secret_basic).
# Some providers need this since the allauth update in paperless-ngx 3.0
# (login fails at the callback with invalid_client otherwise).
# Optional: explicit token endpoint auth method. "none" is the only accepted
# value, and the schema enforces that (Gitea #11).
#
# WHY IT IS RESTRICTED, since the previous comment here named
# client_secret_basic as the example and that was actively misleading: this
# chart registers a PUBLIC client and configures NO client secret - it
# renders OAUTH_PKCE_ENABLED and nothing else, and secrets.oidcSecretRef was
# removed in 13.0.0 because no secret can reach paperless through a json
# environment variable anyway. A method that authenticates WITH a client
# secret therefore cannot work here; asking for one produced a login that
# died at the callback with invalid_client, while the two values read
# together looked like a working secret setup.
#
# So: null omits the key and lets allauth decide, "none" states the public
# client explicitly - which is what some providers need spelled out since
# the allauth update in paperless-ngx 3.0, and the symptom when they do is
# that same invalid_client at the callback. If a future paperless release
# supports an auth method that needs no secret, widening the enum is a minor
# bump; allowing one that needs a secret is not on the table.
tokenAuthMethod: null
# ---------------------------------------------------------------------------
+60 -4
View File
@@ -88,8 +88,55 @@
// line can only come from this hook. Nothing has to be configured for the
// move - but the allow-list lives on that instance, not in this repo, so it
// is also not visible from here if it is ever removed.
//
// THE SECOND ARGUMENT IS THE UPDATE TYPE (Gitea #128), and it is what lets
// the chart version follow the app version: major -> major, minor -> MINOR,
// patch/digest -> patch. It is passed rather than derived because Renovate
// has already classified the jump and the script cannot: `freesailarr` keeps
// a hand-maintained `appVersion: "1.0.0"` for ten upstreams, so prowlarr's
// 2.5.2 -> 2.6.5 moved an image tag in values.yaml while appVersion stood
// still - an appVersion comparison is blind to exactly the case the rule was
// written for (README, "The version-bump hook").
//
// VERIFIED, NOT ASSUMED, and it took two runs because one of them cannot
// answer the question. Renovate 44.155.7, keycloak pinned one minor back:
//
// 1. --platform=local --dry-run=full reads THIS file and classifies the
// update ("updateType": "minor", newValue 26.8.1, alongside a separate
// "patch" bucket for 26.7.5). What it does NOT do is reach the branch
// worker at all - there is not one "Processing branch" line in the log -
// so the hook never runs under the local platform and that run can
// prove the lookup and the config, never the command.
// 2. Renovate's own post-upgrade executor
// (dist/workers/repository/update/branch/execute-post-upgrade-commands)
// driven against a throwaway checkout with allowedCommands set to the
// pattern above:
// DEBUG: Executed post-upgrade task
// "cmd": "ci/renovate-bump-chart.sh keycloak minor"
// "stdout": "renovate-bump: keycloak/Chart.yaml 1.3.0+up26.7.0 ->
// 1.4.0+up26.8.1 (update type minor)"
// So the field arrives populated, the front-anchored allow-list accepts
// the command WITH the argument, and the result is a chart minor.
//
// `updateType` is in Renovate's template allowedFields (util/template:
// "One of digest, pin, rollback, patch, minor, major, replacement,
// pinDigest") and the command is compiled per UPDATE here, which is what
// executionMode: "update" below means.
//
// THE RISK THIS ADDS, and it cannot be settled from inside this repo: the
// allow-list is matched against the COMPILED command
// (`allowedCommands.some((p) => regEx(p).test(compiledCmd))`), so the
// front-anchored pattern above still matches with an argument appended. A
// pattern anchored at the END on the instance (`...\.sh$`) would stop
// matching and the task would be skipped with every chart PR red on the
// version check - the same silent failure a missing entry produces. Which
// form the instance uses is not visible from here. The script therefore
// treats a MISSING second argument as the pre-#128 behaviour instead of an
// error, so the worst case is the old classification rather than an outage.
postUpgradeTasks: {
commands: ["ci/renovate-bump-chart.sh {{{packageFileDir}}}"],
commands: [
"ci/renovate-bump-chart.sh {{{packageFileDir}}} {{{updateType}}}",
],
fileFilters: ["**/Chart.yaml"],
// "update" runs the script once per dependency update with the package
// file's directory, which is what the script needs to find the chart. The
@@ -169,9 +216,12 @@
"its size suggests: a subchart is invisible to everything downstream that watches",
"Helm release versions, so this PR is the ONLY way a new proxy version - a security",
"fix included - ever reaches a cluster. Automerge carries that PR to main once it is",
"reviewed and the check is green, and ci/renovate-bump-chart.sh bumps the chart patch",
"that publishes it (appVersion is unchanged, so it takes the patch branch). A major",
"goes the same way - it is the review, not a separate policy, that gives it a look.",
"reviewed and the check is green, and ci/renovate-bump-chart.sh bumps the chart version",
"that publishes it - since #128 at the SUBCHART'S level, so an oauth2-proxy minor is a",
"chart minor even though our appVersion did not move. That is the point of taking the",
"level from Renovate rather than from appVersion: the one number anything downstream",
"sees now reports the size of what is inside the package. A major goes the same way -",
"it is the review, not a separate policy, that gives it a look.",
],
matchManagers: ["helmv3"],
addLabels: ["subchart"],
@@ -262,6 +312,12 @@
"publish, one install-test run for the whole stack. Nothing is left to serialize, so",
"nothing is left to rebase.",
"",
"WHAT LEVEL A MIXED GROUP GETS, since #128 made that a question: the hook still runs once",
"per update and still recomputes from the same base, so the group takes the HIGHEST level",
"in it - three minors among five patches is a chart minor, whatever order the updates are",
"processed in. That is the maximum rule in ci/renovate-bump-chart.sh, and this branch is",
"the reason it is not optional: without it the last call would win.",
"",
"MATCHED BY FILE, NOT BY A LIST OF IMAGE NAMES. What creates the collision is 'touches",
"freesailarr/Chart.yaml', and that is true of every dependency Renovate finds in that",
"chart's values.yaml, whatever its manager, datasource or role in the pod. A",