feat(paperless-ngx)!: quote every operator scalar, pin the consume mover, drop oidcSecretRef #131

Merged

jklein merged 2 commits from feat/paperless-ngx-scalars-pinning-oidc into main 2026-10-13 21:29:32 +02:00

Conversation 2 Commits 2 Files Changed 14

+1049 -51

jlab-bot commented 2026-10-13 21:12:23 +02:00

Member

Copy link

Copy source

Three issues on one chart in one PR: they share values.yaml, values.schema.json and the version line, and the first half of #11 lives in the same template as #101.

Version 13.0.0+up3.2.1 (appVersion unchanged). Major because a value is removed and fails the strict schema afterwards, and because runtime placement changes. #112 alone would have been minor.

Read this first: #112 was not what it said

The issue reports that the chart has no way to set an arbitrary PAPERLESS_* variable. That is not true on main — paperless.env exists, is in the schema, renders after the fixed entries, and supports all three forms. It landed in "conventions batch 3", long before the issue. The issue measured helm show values of the deployed 12.2.0 and listed only the top-level keys; paperless.env sits inside the paperless block.

What was broken is the block itself, and it is the same defect as #11:

paperless.env:
  - name: PAPERLESS_STAR
    value: "*every*"     ->  render ABORTS (sfs.yaml, YAML alias indicator)
  - name: PAPERLESS_DEBUG
    value: "true"        ->  renders  value: true   (bool)
  - name: PAPERLESS_NUM
    value: "0755"        ->  renders  value: 0755   (int)

Both schema-valid strings, both arriving as something else. The last two render green and are rejected by the API server at apply time. So the chart did not lack the feature — it had one that could not be trusted with an arbitrary value, which is why "set PAPERLESS_LOGGING_DIR" happened to work (that value is quote-safe) while the next variable would not have.

The fix is the repository's documented quoted form — template-chart/templates/xxx.deployment.yaml carries the rule in a comment, and 23 of 30 charts follow it. paperless-ngx was one of seven that did not. No new value, no second name beside env.

PAPERLESS_LOGGING_DIR is documented as the worked example with its limits: the root cause is the NFS-exported RWX data volume in jk-cluster-config #360, and this does not fix it. It moves the log off that volume onto the /tmp emptyDir the pod already mounts. #360 is not closed by this and nothing in this repo touches that repository. The update strategy was left alone, as instructed.

#11, first half — the schedule

paperless-consume.cronjob.yaml:7 rendered its schedule bare. Reproduced before the fix, exactly as the issue's comment describes:

$ helm template ... --set smbConnection.enabled=true --set 'smbConnection.cronJob=*/15 * * * *'
Error: YAML parse error on paperless-ngx/templates/paperless-consume.cronjob.yaml:
error converting YAML to JSON: yaml: line 6: did not find expected alphabetic or numeric character

Fixed with | quote. @daily was broken the same way and now works too. The NOTES.txt paragraph that documented the bug as a workaround ("use an equivalent that starts with a digit") is replaced — it was telling operators to avoid the notation they wanted.

#11, second half — secrets.oidcSecretRef removed

Gone from values.yaml and from the schema, so a release that still sets it now fails with additional properties 'oidcSecretRef' not allowed instead of being ignored. Operator action: delete the key. It never did anything — no template read it, no OnePasswordItem was created.

Not reimplemented, for the reason in the issue comment: paperless reads the whole OIDC configuration from one JSON environment variable with no _FILE variant, so a secretKeyRef cannot substitute a fragment into it. The operational question is answered by #83 — Keycloak marks a client public per client, so nothing forces a confidential client.

oidc.provider.tokenAuthMethod is restricted to none (schema enum), which is the other half of the same decision. Its comment named client_secret_basic as the example: that asks the provider for an authentication this chart cannot perform, and the two values read together looked like a working secret setup. The symptom was a login dying at the callback with invalid_client.

Measured before restricting it: all three live instances (paperless-jk, paperless-cmk, paperless-ngx-tb) run OIDC as a public PKCE client and none of them sets token_auth_method — read off the live StatefulSets' env. So the enum breaks no running release.

#101 — the consume mover is pinned to the StatefulSet's node

A required podAffinity on the StatefulSet pod's label, topologyKey: kubernetes.io/hostname. Both levers were examined:

• Placement. A static nodeSelector is wrong because the StatefulSet pod is not pinned either — it moves on every restart, so the selector would be correct until then and silently wrong after. preferred falls back to another node in exactly the situation the rule exists for, which reproduces the bug more rarely and harder to find. Hence required, as the issue proposed.
• successfulJobsHistoryLimit: not part of the cause, and left untouched. Longhorn's predicate is the existence of a Pending pod, not the absence of Succeeded ones — the message says so itself (no Pending workload pods ... to be mounted). With the limit at 0 there would still be one Running pod and no Pending one, and the mount would still abort. Those three pods are the error message's evidence, not its mechanism. Changing it would have been a behaviour change with nothing behind it.

What happens when the StatefulSet pod is absent — the new failure mode, handled rather than left open:

Case	Behaviour
scaleDown: true	the CronJob is suspended (pre-existing), so no job is created at all
rolling restart	the cron pod waits and is scheduled once the StatefulSet pod exists; IgnoredDuringExecution never evicts a running mover
paperless.replicas: 0 with SMB on	the job fails after activeDeadlineSeconds (3600) with DeadlineExceeded — one bounded, visible failure per schedule, not a growing pile of Pending pods

Plus concurrencyPolicy: Forbid (two movers racing on the same mv is never wanted) and startingDeadlineSeconds: 600, which also keeps the controller away from its "too many missed start times (> 100)" stop — reachable by leaving scaleDown: true in place for months, and reported only in the controller's own log.

What the pinning does not fix, stated in values.yaml, in NOTES.txt and here: the consume volume stays RWX with a share-manager behind it. Co-locating the two consumers removes the cross-node mount, not the second consumer. The design that removes it is moving the mv into the StatefulSet pod, which cannot be done for the running installations because RWX cannot be taken back off an existing PVC — the issue names it as the alternative, not the proposal.

And what is not provable here at all: the install-test runs in k3d without Longhorn, so the RWX mount failure cannot be reproduced in CI. This fix is demonstrable at the render and placement level — the affinity is emitted, and its selector is held against the StatefulSet's actual pod labels by a test — not at the symptom.

The test #11 asked for

"Whoever fixes this ships a test case that would have caught it."

ci/run-yaml-scalar-render-test.sh (new job YAML scalars, all charts, every PR) asserts three properties Kubernetes itself requires, so it needs no per-chart expectations: a CronJob schedule is a non-empty string, every env[].value of every container and initContainer is a string, every label and annotation value is a string.

It is a render test and not a fixture change, deliberately. The install-test fixture keeps smbConnection off — the CI cluster has no SMB CSI driver, and this chart's install-test is already one of the more expensive ones (bundled Valkey). The hostile values live in ci/yaml-scalar/paperless-ngx.yaml, an overlay applied last. That overlay is the test: the reason the defect survived is that no fixture renders that CronJob at all, so a test over what the fixtures already produce would have been equally blind. A chart that has an overlay and still renders nothing of the three kinds fails, and a run that asserts nothing fails.

The fail path was executed, in all three shapes, against a throwaway copy of the chart as it stands on main:

1. bare schedule + */15 * * * * → render aborts, yaml: line 6: ..., test reports it → FAIL
2. only the schedule quoted → the *-valued env entry aborts on the StatefulSet instead → FAIL
3. only the bool-looking value left, so the render succeeds → the type assertion fires:

FAIL paperless-ngx: env PAPERLESS_BOOLEAN_LOOKING_VALUE of container scalar-paperless-ngx-sfs
  rendered as bool (True), not a string. ... "cannot unmarshal bool into Go struct field
  EnvVar.value of type string" - at apply time, not in the render.

Against this branch: 30 charts, 408 scalars, 0 failed.

ci/run-selector-match-test.sh additionally now holds every required podAffinity term against all pod templates of the same render — the guard for the #101 fix. That case was not in the script's documented gap: a preference that cannot be met is dropped and an anti-affinity over zero pods is satisfied, but a required term matching no pod can never be satisfied. The mover selects a label written in another template, which is the copy a test can hold. Exercised by renaming that label in a throwaway copy: one FAIL naming the selector, the pod templates it missed and the labels actually present.

Render diff against origin/main (not empty — that is the point)

With the chart's CI fixture plus smbConnection.enabled=true, default schedule:

 metadata:
   name: diff-paperless-ngx-consume-cronjob
 spec:
-  schedule: 0 0,12 * * *
+  schedule: "0 0,12 * * *"
+  concurrencyPolicy: Forbid
+  startingDeadlineSeconds: 600
   jobTemplate:
     spec:
+      activeDeadlineSeconds: 3600
       template:
         spec:
+          affinity:
+            podAffinity:
+              requiredDuringSchedulingIgnoredDuringExecution:
+                - labelSelector:
+                    matchLabels:
+                      app: diff-paperless-ngx-sfs
+                  topologyKey: kubernetes.io/hostname
           automountServiceAccountToken: false
@@
     cert-manager.io/cluster-issuer: ci
-    nginx.ingress.kubernetes.io/proxy-body-size: 10m
-    nginx.org/client-max-body-size: 10m
+    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
+    nginx.org/client-max-body-size: "10m"

With SMB off (the default fixture) the diff is the two annotation lines only. Every rationale in this PR is in a {{/* */}} template comment, not a # line: the rendered manifests were grepped for the new wording and carry none of it.

Side defect fixed on the way

Every read of smbConnection.cron dereferenced the block directly, so an override that had erased it did not fall back to the defaults this chart promises everywhere else (#61/#99) — it aborted:

Error: template: .../paperless-consume.cronjob.yaml:17:71: ... nil pointer
evaluating interface {}.securityContext

Measured on origin/main, so this is a fix and not a precaution. All reads now go through paperless-ngx.cron.scheduling, built on the chart's existing defaultedDict, which preserves an explicit false — the one value here whose explicit false must be obeyed, since it switches off a hard placement rule (default true would have silently flipped it back on).

The repo-wide checks #11 asked for

• Secret refs no template reads: swept all 46 secrets.* keys across 27 charts. paperless-ngx' oidcSecretRef was the only one in the repository. Fixed here; nothing to follow up.
• Unquoted scalars elsewhere: the only schedule: in the repo is this one (one CronJob total). Six charts still render their env block bare — matrix-synapse, minecraft-router, minecraft-server, rallly, satisfactory, teamspeak-server. They are green only because no fixture of theirs sets a hostile value; each is a major bump for that chart, so they are not folded into this PR. README now says this instead of repeating template-chart's claim that every chart quotes. Same genus, still open: proxy-body-size annotations in calibre-web-automated, outline and zipline (fixed here for paperless-ngx).

Verified

• helm lint --strict: 0 failed, with defaults and with the hostile overlay
• helm package → paperless-ngx-13.0.0+up3.2.1.tgz
• quantity-render 28/30 exercised, selector-match 6 selectors, httproute 64 assertions, yaml-scalar 408 — all 0 failed
• #128's new bump-level test: 49 cases passed (unrelated to this change; this PR is the first to run it)
• live-cluster facts re-measured before being relied on: three instances, replicas=1/RollingUpdate/OrderedReady, cron pods landing on jk-worker-1/jk-worker-2 while every StatefulSet pod sits on jk-worker-1, the consume PVC bound to pvc-8b7b112e-… (the volume named in #101), exactly one pod on the data volume, and a share-manager pod behind it
• commits signed (%G? = G), rebased onto 59d2614 after the #128 merge, keeping its README and workflow text

Fixes #11
Fixes #101
Fixes #112

Three issues on one chart in one PR: they share `values.yaml`, `values.schema.json` and the version line, and the first half of #11 lives in the same template as #101. **Version `13.0.0+up3.2.1`** (`appVersion` unchanged). Major because a value is **removed** and fails the strict schema afterwards, and because runtime placement changes. #112 alone would have been minor. ## Read this first: #112 was not what it said The issue reports that the chart has no way to set an arbitrary `PAPERLESS_*` variable. **That is not true on `main`** — `paperless.env` exists, is in the schema, renders after the fixed entries, and supports all three forms. It landed in "conventions batch 3", long before the issue. The issue measured `helm show values` of the deployed **12.2.0** and listed only the *top-level* keys; `paperless.env` sits inside the `paperless` block. What *was* broken is the block itself, and it is the same defect as #11: ``` paperless.env: - name: PAPERLESS_STAR value: "*every*" -> render ABORTS (sfs.yaml, YAML alias indicator) - name: PAPERLESS_DEBUG value: "true" -> renders value: true (bool) - name: PAPERLESS_NUM value: "0755" -> renders value: 0755 (int) ``` Both schema-valid strings, both arriving as something else. The last two render green and are rejected by the API server at apply time. So the chart did not lack the feature — it had one that could not be trusted with an arbitrary value, which is why "set `PAPERLESS_LOGGING_DIR`" happened to work (that value is quote-safe) while the next variable would not have. The fix is the repository's **documented** quoted form — `template-chart/templates/xxx.deployment.yaml` carries the rule in a comment, and 23 of 30 charts follow it. `paperless-ngx` was one of seven that did not. No new value, no second name beside `env`. `PAPERLESS_LOGGING_DIR` is documented as the worked example **with its limits**: the root cause is the NFS-exported RWX data volume in `jk-cluster-config` #360, and this does not fix it. It moves the log off that volume onto the `/tmp` emptyDir the pod already mounts. **#360 is not closed by this** and nothing in this repo touches that repository. The update strategy was left alone, as instructed. ## #11, first half — the schedule `paperless-consume.cronjob.yaml:7` rendered its schedule bare. Reproduced before the fix, exactly as the issue's comment describes: ``` $ helm template ... --set smbConnection.enabled=true --set 'smbConnection.cronJob=*/15 * * * *' Error: YAML parse error on paperless-ngx/templates/paperless-consume.cronjob.yaml: error converting YAML to JSON: yaml: line 6: did not find expected alphabetic or numeric character ``` Fixed with `| quote`. `@daily` was broken the same way and now works too. **The `NOTES.txt` paragraph that documented the bug as a workaround** ("use an equivalent that starts with a digit") is replaced — it was telling operators to avoid the notation they wanted. ## #11, second half — `secrets.oidcSecretRef` removed Gone from `values.yaml` and from the schema, so a release that still sets it now fails with `additional properties 'oidcSecretRef' not allowed` instead of being ignored. **Operator action: delete the key.** It never did anything — no template read it, no `OnePasswordItem` was created. Not reimplemented, for the reason in the issue comment: paperless reads the whole OIDC configuration from **one** JSON environment variable with no `_FILE` variant, so a `secretKeyRef` cannot substitute a fragment into it. The operational question is answered by #83 — Keycloak marks a client public per client, so nothing forces a confidential client. **`oidc.provider.tokenAuthMethod` is restricted to `none`** (schema `enum`), which is the other half of the same decision. Its comment named `client_secret_basic` as the example: that asks the provider for an authentication this chart cannot perform, and the two values read together looked like a working secret setup. The symptom was a login dying at the callback with `invalid_client`. **Measured before restricting it:** all three live instances (`paperless-jk`, `paperless-cmk`, `paperless-ngx-tb`) run OIDC as a public PKCE client and **none of them sets `token_auth_method`** — read off the live StatefulSets' env. So the enum breaks no running release. ## #101 — the consume mover is pinned to the StatefulSet's node A required `podAffinity` on the StatefulSet pod's label, `topologyKey: kubernetes.io/hostname`. Both levers were examined: - **Placement.** A static `nodeSelector` is wrong because the StatefulSet pod is not pinned either — it moves on every restart, so the selector would be correct until then and silently wrong after. `preferred` falls back to another node in exactly the situation the rule exists for, which reproduces the bug more rarely and harder to find. Hence `required`, as the issue proposed. - **`successfulJobsHistoryLimit`: not part of the cause, and left untouched.** Longhorn's predicate is the *existence of a Pending pod*, not the *absence of Succeeded ones* — the message says so itself (`no Pending workload pods ... to be mounted`). With the limit at 0 there would still be one `Running` pod and no `Pending` one, and the mount would still abort. Those three pods are the error message's evidence, not its mechanism. Changing it would have been a behaviour change with nothing behind it. **What happens when the StatefulSet pod is absent** — the new failure mode, handled rather than left open: | Case | Behaviour | |---|---| | `scaleDown: true` | the CronJob is **suspended** (pre-existing), so no job is created at all | | rolling restart | the cron pod waits and is scheduled once the StatefulSet pod exists; `IgnoredDuringExecution` never evicts a running mover | | `paperless.replicas: 0` with SMB on | the job **fails after `activeDeadlineSeconds` (3600) with `DeadlineExceeded`** — one bounded, visible failure per schedule, not a growing pile of `Pending` pods | Plus `concurrencyPolicy: Forbid` (two movers racing on the same `mv` is never wanted) and `startingDeadlineSeconds: 600`, which also keeps the controller away from its "too many missed start times (> 100)" stop — reachable by leaving `scaleDown: true` in place for months, and reported only in the controller's own log. **What the pinning does not fix**, stated in `values.yaml`, in `NOTES.txt` and here: the consume volume stays RWX with a share-manager behind it. Co-locating the two consumers removes the **cross-node** mount, not the second consumer. The design that removes it is moving the `mv` into the StatefulSet pod, which cannot be done for the running installations because RWX cannot be taken back off an existing PVC — the issue names it as the alternative, not the proposal. **And what is not provable here at all:** the install-test runs in k3d **without Longhorn**, so the RWX mount failure cannot be reproduced in CI. This fix is demonstrable at the render and placement level — the affinity is emitted, and its selector is held against the StatefulSet's actual pod labels by a test — not at the symptom. ## The test #11 asked for > "Whoever fixes this ships a test case that would have caught it." `ci/run-yaml-scalar-render-test.sh` (new job **YAML scalars**, all charts, every PR) asserts three properties Kubernetes itself requires, so it needs no per-chart expectations: a CronJob `schedule` is a non-empty string, every `env[].value` of every container and initContainer is a string, every label and annotation value is a string. **It is a render test and not a fixture change, deliberately.** The install-test fixture keeps `smbConnection` off — the CI cluster has no SMB CSI driver, and this chart's install-test is already one of the more expensive ones (bundled Valkey). The hostile values live in `ci/yaml-scalar/paperless-ngx.yaml`, an overlay applied last. That overlay **is** the test: the reason the defect survived is that *no fixture renders that CronJob at all*, so a test over what the fixtures already produce would have been equally blind. A chart that has an overlay and still renders nothing of the three kinds fails, and a run that asserts nothing fails. **The fail path was executed, in all three shapes, against a throwaway copy of the chart as it stands on `main`:** 1. bare schedule + `*/15 * * * *` → render aborts, `yaml: line 6: ...`, test reports it → **FAIL** 2. only the schedule quoted → the `*`-valued `env` entry aborts on the StatefulSet instead → **FAIL** 3. only the bool-looking value left, so the render **succeeds** → the type assertion fires: ``` FAIL paperless-ngx: env PAPERLESS_BOOLEAN_LOOKING_VALUE of container scalar-paperless-ngx-sfs rendered as bool (True), not a string. ... "cannot unmarshal bool into Go struct field EnvVar.value of type string" - at apply time, not in the render. ``` Against this branch: **30 charts, 408 scalars, 0 failed.** `ci/run-selector-match-test.sh` additionally now holds every **required** `podAffinity` term against all pod templates of the same render — the guard for the #101 fix. That case was not in the script's documented gap: a preference that cannot be met is dropped and an anti-affinity over zero pods is satisfied, but a *required* term matching no pod can never be satisfied. The mover selects a label written in **another** template, which is the copy a test can hold. Exercised by renaming that label in a throwaway copy: one `FAIL` naming the selector, the pod templates it missed and the labels actually present. ## Render diff against `origin/main` (not empty — that is the point) With the chart's CI fixture plus `smbConnection.enabled=true`, default schedule: ```diff metadata: name: diff-paperless-ngx-consume-cronjob spec: - schedule: 0 0,12 * * * + schedule: "0 0,12 * * *" + concurrencyPolicy: Forbid + startingDeadlineSeconds: 600 jobTemplate: spec: + activeDeadlineSeconds: 3600 template: spec: + affinity: + podAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchLabels: + app: diff-paperless-ngx-sfs + topologyKey: kubernetes.io/hostname automountServiceAccountToken: false @@ cert-manager.io/cluster-issuer: ci - nginx.ingress.kubernetes.io/proxy-body-size: 10m - nginx.org/client-max-body-size: 10m + nginx.ingress.kubernetes.io/proxy-body-size: "10m" + nginx.org/client-max-body-size: "10m" ``` With SMB off (the default fixture) the diff is the two annotation lines only. Every rationale in this PR is in a `{{/* */}}` template comment, **not** a `#` line: the rendered manifests were grepped for the new wording and carry none of it. ## Side defect fixed on the way Every read of `smbConnection.cron` dereferenced the block directly, so an override that had **erased** it did not fall back to the defaults this chart promises everywhere else (#61/#99) — it aborted: ``` Error: template: .../paperless-consume.cronjob.yaml:17:71: ... nil pointer evaluating interface {}.securityContext ``` Measured on `origin/main`, so this is a fix and not a precaution. All reads now go through `paperless-ngx.cron.scheduling`, built on the chart's existing `defaultedDict`, which preserves an explicit `false` — the one value here whose explicit `false` must be obeyed, since it switches off a hard placement rule (`default true` would have silently flipped it back on). ## The repo-wide checks #11 asked for - **Secret refs no template reads:** swept all 46 `secrets.*` keys across 27 charts. **`paperless-ngx`' `oidcSecretRef` was the only one in the repository.** Fixed here; nothing to follow up. - **Unquoted scalars elsewhere:** the only `schedule:` in the repo is this one (one CronJob total). **Six charts still render their `env` block bare** — `matrix-synapse`, `minecraft-router`, `minecraft-server`, `rallly`, `satisfactory`, `teamspeak-server`. They are green only because no fixture of theirs sets a hostile value; each is a major bump for that chart, so they are **not** folded into this PR. README now says this instead of repeating `template-chart`'s claim that every chart quotes. Same genus, still open: `proxy-body-size` annotations in `calibre-web-automated`, `outline` and `zipline` (fixed here for `paperless-ngx`). ## Verified - `helm lint --strict`: 0 failed, with defaults and with the hostile overlay - `helm package` → `paperless-ngx-13.0.0+up3.2.1.tgz` - quantity-render 28/30 exercised, selector-match 6 selectors, httproute 64 assertions, yaml-scalar 408 — all 0 failed - #128's new `bump-level` test: **49 cases passed** (unrelated to this change; this PR is the first to run it) - live-cluster facts re-measured before being relied on: three instances, `replicas=1`/`RollingUpdate`/`OrderedReady`, cron pods landing on `jk-worker-1`/`jk-worker-2` while every StatefulSet pod sits on `jk-worker-1`, the consume PVC bound to `pvc-8b7b112e-…` (the volume named in #101), exactly one pod on the data volume, and a share-manager pod behind it - commits signed (`%G?` = `G`), rebased onto `59d2614` after the #128 merge, keeping its README and workflow text Fixes #11 Fixes #101 Fixes #112

jlab-bot added 2 commits 2026-10-13 21:12:23 +02:00

feat(paperless-ngx)!: quote every operator scalar, pin the consume mover, drop oidcSecretRef ... 4b2bde505d

Three issues on one chart, bundled because they share values.yaml,
values.schema.json and the version line, and because the first half of #11
lives in the same template as #101.

Gitea #11, first half: the consume CronJob's schedule was rendered bare, so a
schedule in the commonest cron notation there is - a star, a slash, an interval
- aborted the render with a YAML parse error pointing at the template while the
operator's value appeared nowhere in the message. Quoting it in the values file
does not help; Helm passes the content through. Reproduced before the fix, at
the line the issue names, and fixed with | quote.

Gitea #11, second half: secrets.oidcSecretRef is removed, values and schema
both. No template ever read it, so no OnePasswordItem was created and nothing
reached the pod - the quiet kind of defect that looks like successful
configuration. It is not reimplemented, and the reason is in paperless: the
whole OIDC configuration travels in one json environment variable with no _FILE
variant, so a secretKeyRef cannot substitute a fragment into it. The
operational question is answered by #83 - Keycloak marks a client public per
client, so nothing forces a confidential one here.

In the same breath, oidc.provider.tokenAuthMethod is restricted to "none". Its
comment named client_secret_basic as the example, which asks the provider for
an authentication this chart cannot perform: it configures no client secret and
cannot. The two values read together looked like a working secret setup, and the
symptom was a login dying at the callback with invalid_client. No live release
sets it (measured against all three instances).

Gitea #112 asked for an extraEnv and the chart already had the standard
additive env block - the issue was measured against a deployed 12.2.0 and read
only the top-level keys. What was actually broken is the block itself: it
rendered its operand bare, so a "*"-leading value aborted the render and "true"
or "0755" arrived as a bool or an integer, which the API server rejects after a
green render. It is brought to the repository's documented quoted form
(template-chart carries the rule), which is what makes an arbitrary PAPERLESS_*
variable settable in practice. PAPERLESS_LOGGING_DIR is documented as the
worked example, with the cross-reference it needs: the root cause is the
NFS-exported RWX data volume in jk-cluster-config #360 and this does not fix it
- it only moves the log off that volume. The two Ingress upload-size
annotations had the same bare render and are quoted too.

Gitea #101: the consume mover now runs on the node of the StatefulSet pod, via
a required podAffinity on its label. It mounts the RWX consume volume, and
Longhorn's CSI driver refuses that mount unless it finds a workload pod in
Pending, so a mover on another node left the StatefulSet node's kubelet
retrying every two minutes - 291 events in 10 hours in both releases with SMB
on, none in the release without the CronJob. A static nodeSelector would be
wrong because the StatefulSet pod is not pinned either, and preferred would
reproduce the bug more rarely and harder to find.

The cost of a required term is handled rather than left open:
concurrencyPolicy: Forbid, a startingDeadlineSeconds that also keeps the
controller away from its 100-missed-schedules stop, and an activeDeadlineSeconds
that fails a job which cannot be placed instead of leaving one Pending pod per
schedule. scaleDown already suspends the CronJob; what remains is
paperless.replicas: 0 with SMB on, which now reports a bounded, visible failure.
successfulJobsHistoryLimit is deliberately untouched: Longhorn's check asks for
the existence of a Pending pod, not the absence of Succeeded ones, so those
three pods are the error message's evidence and not its cause.

Side defect fixed on the way: every read of smbConnection.cron dereferenced the
block directly, so an override that had erased it aborted with a nil-pointer
error naming an internal template path instead of falling back to the defaults
this chart promises everywhere else. Measured on origin/main; all reads now go
through a merged-defaults helper that preserves an explicit false.

Major: a value is removed and fails the strict schema afterwards, and runtime
placement changes.

Fixes #11
Fixes #101
Fixes #112

ci: hold every scalar that must be a string, and the required podAffinity ... 39844fc92c

The test Gitea #11 asked for ("whoever fixes this ships a test case that would
have caught it"), plus the guard for the fix #101 introduces.

ci/run-yaml-scalar-render-test.sh renders every chart with the fixtures the
install-test already uses and asserts three properties Kubernetes itself
requires, so no per-chart expectations are needed: a CronJob schedule is a
non-empty string, every env[].value of every container and initContainer is a
string, and every label and annotation value is a string. The first catches the
loud failure - the render aborting with a YAML error that names the template
and not the value - and the other two catch the quiet one, where the render
succeeds and the API server rejects the object at apply time.

WHY A NEW JOB AND NOT THE INSTALL-TEST. The object that carried the defect is
rendered by no fixture at all: paperless-ngx' consume CronJob sits behind
smbConnection.enabled, false in the chart defaults and false in its install-test
fixture, since the CI cluster has no SMB CSI driver. Nothing in this repository
ever produced it. So the overlays under ci/yaml-scalar/ are the test, not the
scaffolding: they switch such objects on AND feed them hostile values, and the
script fails when an overlay stops reaching its object or when the run asserted
nothing at all.

The fail path was exercised in all three shapes against a throwaway copy of the
chart as it stands on main, not reasoned about: the bare schedule aborted with
"yaml: line 6: did not find expected alphabetic or numeric character"; with only
the schedule quoted, the star-valued env entry aborted on the StatefulSet
instead; and with only the bool-looking value left - a render that succeeds -
the type assertion named the variable, the container and the API server's
message. Green repo-wide otherwise: 30 charts, 408 scalars, 0 failed.

ci/run-selector-match-test.sh additionally holds every
requiredDuringSchedulingIgnoredDuringExecution podAffinity term against all pod
templates of the same render (Gitea #101). That case is not the gap the script's
header describes: a preference that cannot be met is dropped and an
anti-affinity over zero pods is satisfied, but a REQUIRED term matching no pod
can never be satisfied and the pod stays Pending for good. paperless-ngx' mover
selects a label written in another template, which is exactly the copy this test
holds - renaming it would leave both templates rendering perfectly while the
mover silently stopped moving. Exercised by doing that rename in a throwaway
copy: one FAIL naming the selector, the pod templates it missed and the labels
actually present.

Six charts still render their env block bare (matrix-synapse, minecraft-router,
minecraft-server, rallly, satisfactory, teamspeak-server). They are green only
because no fixture of theirs sets a hostile value; README now says so instead of
repeating template-chart's claim that every chart quotes.

Part of #11
Part of #101

jlab-bot changed title from feat(paperless-ngx): quote every operator scalar, pin the consume mover, drop oidcSecretRef to feat(paperless-ngx)!: quote every operator scalar, pin the consume mover, drop oidcSecretRef 2026-10-13 21:12:41 +02:00

jlab-bot commented 2026-10-13 21:24:22 +02:00

Author

Member

Copy link

Copy source

CI green — run 850, all ten checks

Head 39844fc, rebased onto 59d2614 (the #128 merge). Job output rather than a colour:

YAML scalars — the new job, 15 s on the runner:

paperless-ngx: ok (27 scalar(s) checked)
===================== yaml scalar render test ======================
checked:       30 - apache-tika bentopdf bulwark calibre-web-automated cyberchef
                    freesailarr garage gotenberg homeassistant keycloak ...
not exercised: 0
failed:        0
scalars verified: 408

Selector match — the podAffinity check this PR adds, 14 s:

paperless-ngx: ok (1 selector(s) matched, 0 not checked)
failed:        0
selectors verified: 6

The one selector is the mover's required podAffinity held against the StatefulSet's rendered pod labels. Previous figure was 5; the delta is this chart, through ci/selector-match/paperless-ngx.yaml.

Install-test paperless-ngx — 5 m, the expensive one (bundled Valkey):

deployment "postgres" successfully rolled out
Waiting for 1 pods to be ready...
OK: chart 'paperless-ngx' installed and all workloads are Ready.

Bump level — #128's new job, 7 s, 49 cases passed. This PR is the first to run it; it is green and unrelated to this change.

Also green: Detect changed charts, Build paperless-ngx (lint, package, version bump 12.3.0+up3.2.1 → 13.0.0+up3.2.1), Resource quantities, HTTPRoute match, Trivy config paperless-ngx.

What this run does not prove, repeating it here so it is not lost in the body: the install-test cluster is k3d without Longhorn, so the RWX mount failure of #101 is not reproducible in CI. The placement fix is covered at the render level — the affinity is emitted, and its selector is mechanically matched against the StatefulSet's pod labels — not at the symptom.

Wiki updated in the same round (1caabc1): paperless-ngx, CI-CD, Chart-Style-Guide, Lessons-Learned. Two notes on the chart page were replaced rather than appended, because they had become wrong: "A schedule may not start with *" documented the bug as a workaround, and the probes-exception section claimed paperless.replicas: 0 keeps moving PDFs off the share — with the required affinity it no longer can.

## CI green — run 850, all ten checks Head `39844fc`, rebased onto `59d2614` (the #128 merge). Job output rather than a colour: **`YAML scalars` — the new job, 15 s on the runner:** ``` paperless-ngx: ok (27 scalar(s) checked) ===================== yaml scalar render test ====================== checked: 30 - apache-tika bentopdf bulwark calibre-web-automated cyberchef freesailarr garage gotenberg homeassistant keycloak ... not exercised: 0 failed: 0 scalars verified: 408 ``` **`Selector match` — the `podAffinity` check this PR adds, 14 s:** ``` paperless-ngx: ok (1 selector(s) matched, 0 not checked) failed: 0 selectors verified: 6 ``` The one selector is the mover's required `podAffinity` held against the StatefulSet's rendered pod labels. Previous figure was 5; the delta is this chart, through `ci/selector-match/paperless-ngx.yaml`. **`Install-test paperless-ngx` — 5 m, the expensive one (bundled Valkey):** ``` deployment "postgres" successfully rolled out Waiting for 1 pods to be ready... OK: chart 'paperless-ngx' installed and all workloads are Ready. ``` **`Bump level` — #128's new job, 7 s, 49 cases passed.** This PR is the first to run it; it is green and unrelated to this change. Also green: `Detect changed charts`, `Build paperless-ngx` (lint, package, version bump `12.3.0+up3.2.1` → `13.0.0+up3.2.1`), `Resource quantities`, `HTTPRoute match`, `Trivy config paperless-ngx`. **What this run does not prove**, repeating it here so it is not lost in the body: the install-test cluster is k3d **without Longhorn**, so the RWX mount failure of #101 is not reproducible in CI. The placement fix is covered at the render level — the affinity is emitted, and its selector is mechanically matched against the StatefulSet's pod labels — not at the symptom. Wiki updated in the same round (`1caabc1`): `paperless-ngx`, `CI-CD`, `Chart-Style-Guide`, `Lessons-Learned`. Two notes on the chart page were **replaced rather than appended**, because they had become wrong: "A schedule may not start with `*`" documented the bug as a workaround, and the probes-exception section claimed `paperless.replicas: 0` keeps moving PDFs off the share — with the required affinity it no longer can.

jklein approved these changes 2026-10-13 21:29:24 +02:00

jklein merged commit 39844fc92c into main 2026-10-13 21:29:32 +02:00

jklein deleted branch feat/paperless-ngx-scalars-pinning-oidc 2026-10-13 21:29:33 +02:00

jlab-bot referenced this pull request 2026-10-14 15:36:44 +02:00

Security: NetworkPolicies für alle Charts #5

Sign in to join this conversation.

Reviewers

No Reviewers

jklein

Labels

Clear labels

inhouse-chart

Inhouse-Chart aus der JLab-Git-Paketregistry

renovate

Von Renovate angelegt

No labels

Milestone

No items

No Milestone

Assignees

Clear assignees

jklein (Justin Klein) jlab-bot (jlab-git-bot) jlab-renovate-bot (jlab-renovate-bot)

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: jlab-cluster/helm-charts#131