6 Commits

Include renames

Author	SHA1	Message	Date
jklein	530227405c	fix(paperless-ngx): egress names the Tika/Gotenberg POD ports, not the Service port ... The rule allowed outbound 80, the port the apache-tika and gotenberg Services listen on, and justified it in a comment: an egress rule names the port on the destination as the client dials it, and the client dials the Service. That is wrong. kube-proxy does a ClusterIP Service's destination NAT BEFORE the egress policy is evaluated at the pod's network interface, so the policy sees podIP:targetPort and never clusterIP:port - and those two charts forward 80 to 9998 and 3000. Deployed as 14.0.0+up3.3.1 it cut Paperless off from both services on 2026-10-15: mail import, EML, HTML and Office documents stopped, plain PDFs and images kept working, and the error was httpx.ConnectTimeout with no mention of a NetworkPolicy. The rule now names 9998, 3000 and 80. The 80 stays because it costs nothing and covers a foreign Tika that really does listen on 80. The refuted sentence is quoted rather than deleted: it reads perfectly, which is how it survived review, and the timeout-not-refused symptom is written down with it. No policy switch of its own: the rule stays gated on paperless.files.tikaGotenberg.enabled, because a second value that can drift from the feature re-arms this very outage. New networkPolicy.egress.mailImport opens IMAPS 993, off by default - paperless' mail accounts are rows in its database, so the chart cannot derive the rule. Plain IMAP (143) and POP3 stay in extraPorts, and 993 is noted as a real destination port, because a mail server outside the cluster has no DNAT in the path. Minor: one additive value, and a fix that widens a rule. Nothing breaks, no value becomes required, appVersion unchanged. Part of #136	2026-10-15 17:39:24 +00:00
jklein	73c2ec704f	feat!: NetworkPolicy in the last five charts - the sweep is complete ... matrix-synapse, paperless-ngx, freesailarr, valkey, stalwart. One major each. All 29 installable charts plus template-chart now render a policy, and ci/run-networkpolicy-test.sh passes over all 30 with 206 assertions held. MATRIX-SYNAPSE renders TWO policies, one per workload, and the split is the decision in that file. Synapse federates - it must be able to reach any homeserver on the internet, so 443 and 8448 go to any destination - while the delegation nginx serves two static JSON documents and calls nobody, so its egress is DNS and nothing else. A single policy over both pods would have to permit the union, handing full federation reach and the database port to an nginx whose job is to return a constant, and it would read as one decision rather than as a widening. networkPolicy.egress.extraPorts deliberately does not reach the delegation policy for the same reason. PAPERLESS-NGX names the two pods of its own release that this object does NOT cover: the valkey subchart's pod, which carries its own labels and needs no cover because the valkey chart ships a tighter policy of its own; and the consume CronJob's pods, whose pod template carries NO LABELS AT ALL, so no podSelector can match them. That second one is a real gap in the convention. It is tolerable only because the job moves files between mounted volumes and reaches the SMB share through a CSI volume the node mounts - the pod opens no connection of its own - and if that ever changes the CronJob needs labels before it can need a policy. FREESAILARR is ten containers in one pod behind a VPN, and simpler than that sounds: they share a network namespace, so all inter-container traffic is localhost and no policy sees any of it. Because the namespace belongs to gluetun, everything the *arr containers send is encapsulated before it leaves, so three rules describe the whole outbound surface - DNS, UDP 51820 for the tunnel, and 443 for gluetun's server-list API, which cannot go through the tunnel because it runs before the tunnel exists. The tunnel rule is the one that can break this chart and also the best covered: gluetun's health server answers 200 only while the tunnel is up and all three of its probes address it, so a blocked 51820 is a pod that never goes Ready - in the install-test, which brings up a real tunnel. VALKEY AND STALWART, the two charts that had a policy before this issue, are pulled onto the convention without a single rule changing. What changed: - the reasoning moved out of "#" comments, which are part of the rendered object and travel into the cluster, into template comments. Both files failed the new check; that is what it is for. - stalwart gains networkPolicy.ingressController, which fixes a hole its own comment described without being able to solve: it advised putting the controller's namespace into allowedNamespaces, which is namespace-granular and would have opened the http port to every pod in kube-system. - stalwart's example was WRONG and is corrected. It suggested ["ingress-nginx"]; measured on 2026-10-14, ingress-nginx is gone from this cluster entirely and the controller is rke2-traefik in kube-system. A policy written against the old one would have been accepted, looked strict, and silently blocked all web traffic. Their justified specifics are untouched: valkey stays Ingress-only, because it opens no outbound connection and an egress rule would constrain nothing while making the object look stricter; and it keeps 6379 restricted by source with no probeSources at all, which is safe only because all three of its probes are exec probes and put no packet on the network. The convention was written to accommodate both, and homeassistant and satisfactory now invoke valkey's rule in the other direction. ONE CHANGE TO THE TEST, forced by a structural fact rather than by convenience: the rendered-comment assertion now skips documents from a SUBCHART. A dependency is fetched from the Helm registry, not from the copy in this repository, so the policy paperless-ngx and stalwart bundle is whatever valkey version is PUBLISHED - and nothing in the parent chart can change its rendered comments. Holding the parent responsible would have made the test unfixable until valkey's next release and would fail a chart for a file its own diff does not contain. The subchart's own pull request is where it is caught, because this test runs over every chart including valkey itself. All twelve failure paths were re-run against the change. Part of #5	2026-10-14 14:46:11 +00:00
jklein	4b2bde505d	feat(paperless-ngx)!: quote every operator scalar, pin the consume mover, drop oidcSecretRef ... Three issues on one chart, bundled because they share values.yaml, values.schema.json and the version line, and because the first half of #11 lives in the same template as #101. Gitea #11, first half: the consume CronJob's schedule was rendered bare, so a schedule in the commonest cron notation there is - a star, a slash, an interval - aborted the render with a YAML parse error pointing at the template while the operator's value appeared nowhere in the message. Quoting it in the values file does not help; Helm passes the content through. Reproduced before the fix, at the line the issue names, and fixed with | quote. Gitea #11, second half: secrets.oidcSecretRef is removed, values and schema both. No template ever read it, so no OnePasswordItem was created and nothing reached the pod - the quiet kind of defect that looks like successful configuration. It is not reimplemented, and the reason is in paperless: the whole OIDC configuration travels in one json environment variable with no _FILE variant, so a secretKeyRef cannot substitute a fragment into it. The operational question is answered by #83 - Keycloak marks a client public per client, so nothing forces a confidential one here. In the same breath, oidc.provider.tokenAuthMethod is restricted to "none". Its comment named client_secret_basic as the example, which asks the provider for an authentication this chart cannot perform: it configures no client secret and cannot. The two values read together looked like a working secret setup, and the symptom was a login dying at the callback with invalid_client. No live release sets it (measured against all three instances). Gitea #112 asked for an extraEnv and the chart already had the standard additive env block - the issue was measured against a deployed 12.2.0 and read only the top-level keys. What was actually broken is the block itself: it rendered its operand bare, so a "*"-leading value aborted the render and "true" or "0755" arrived as a bool or an integer, which the API server rejects after a green render. It is brought to the repository's documented quoted form (template-chart carries the rule), which is what makes an arbitrary PAPERLESS_* variable settable in practice. PAPERLESS_LOGGING_DIR is documented as the worked example, with the cross-reference it needs: the root cause is the NFS-exported RWX data volume in jk-cluster-config #360 and this does not fix it - it only moves the log off that volume. The two Ingress upload-size annotations had the same bare render and are quoted too. Gitea #101: the consume mover now runs on the node of the StatefulSet pod, via a required podAffinity on its label. It mounts the RWX consume volume, and Longhorn's CSI driver refuses that mount unless it finds a workload pod in Pending, so a mover on another node left the StatefulSet node's kubelet retrying every two minutes - 291 events in 10 hours in both releases with SMB on, none in the release without the CronJob. A static nodeSelector would be wrong because the StatefulSet pod is not pinned either, and preferred would reproduce the bug more rarely and harder to find. The cost of a required term is handled rather than left open: concurrencyPolicy: Forbid, a startingDeadlineSeconds that also keeps the controller away from its 100-missed-schedules stop, and an activeDeadlineSeconds that fails a job which cannot be placed instead of leaving one Pending pod per schedule. scaleDown already suspends the CronJob; what remains is paperless.replicas: 0 with SMB on, which now reports a bounded, visible failure. successfulJobsHistoryLimit is deliberately untouched: Longhorn's check asks for the existence of a Pending pod, not the absence of Succeeded ones, so those three pods are the error message's evidence and not its cause. Side defect fixed on the way: every read of smbConnection.cron dereferenced the block directly, so an override that had erased it aborted with a nil-pointer error naming an internal template path instead of falling back to the defaults this chart promises everywhere else. Measured on origin/main; all reads now go through a merged-defaults helper that preserves an explicit false. Major: a value is removed and fails the strict schema afterwards, and runtime placement changes. Fixes #11 Fixes #101 Fixes #112	2026-10-13 19:09:45 +00:00
jklein	bd1fdc09a0	feat: Gateway-API as an alternative to the Ingress in five annotation charts ... Group 2 of the #114 sweep: the charts whose behaviour hangs off an nginx annotation - calibre-web-automated, outline, paperless-ngx, zipline (proxy-body-size) and freesailarr (rewrite-target). Each gets the form the pilot (cyberchef, PR #115) fixed and group 1 (PR #116) repeated: an httpRoute block off by default, a template rendering ONLY an HTTPRoute with parentRefs - no Gateway, no gatewayClassName, no Certificate, no implementation-specific policy object - render aborts on both modes at once (naming both value paths), on an empty parentRefs and on a missing hostname, empty hostnames meaning [ingress.domain], a schema entry closed against unknown keys with an open parentRef definition, and a NOTES.txt that names the active mode and what the mode costs. Minor, additive, defaults unchanged: `helm template` with each chart's CI values is byte-identical to origin/main for all five. The upload limit is DOCUMENTED, not rebuilt (decision 2 of the issue). The Gateway API knows no body size; a policy object would bind the chart to one implementation. So the Gateway branch of NOTES.txt says, per chart and in the chart's own terms, that the value is NOT ENFORCED and names where to set it: an e-book library (calibre), an upload service (zipline), the web-UI half of the document intake while the consume directory keeps no limit at all (paperless), and the open #12 pair (outline). Four findings that were not in the plan, each argued in the file it changes: freesailarr gated freesailarr.sharedIngressApps on ingress.enabled, and that list is read by the <APP>__SERVER__URLBASE variables as well as by the routing objects. In Gateway mode it would have come out EMPTY while the route served radarr, sonarr and prowlarr under their prefixes: UrlBase unset, assets and API calls linked at the host root, broken UI behind a Ready pod. The helper's two guards (seerr/mediathekarr only at "/", no duplicate paths) would have gone unenforced there too. Fixed with freesailarr.published (ingress.enabled OR httpRoute.enabled) - the same shape as sure.publishedOverTls in group 1, and identical to ingress.enabled while httpRoute.enabled is false. Measured: on origin/main with ingress.enabled: false the render emits ZERO URLBASE variables; with this change in Gateway mode it emits three. paperless-ngx was the last chart in the repository rendering its Ingress UNCONDITIONALLY. Without an ingress.enabled switch there is no second value path for the mutual-exclusion abort to name, and the Ingress plus its unconditional required() on clusterIssuer would have kept rendering in Gateway mode - a value demanded for nothing, which is issue #114's "Folge 1" and README guideline 12. The switch was added, defaulting to true, so the render is unchanged. paperless-ngx PAPERLESS_URL had no required() of its own: it relied on the Ingress template's, which stops firing the moment the Ingress is switchable. With httpRoute.hostnames set and ingress.domain empty it would have rendered "https://" and refused every login POST as a CSRF failure. It now carries its own required(), naming what the value is for. calibre-web-automated claimed, in values.yaml and in its schema, that ingress.domain is read nowhere but in the Ingress template. The HTTPRoute reads it, so the claim now holds only while both modes are off. Corrected rather than left standing - the same correction garage needed in group 1. freesailarr is where the Gateway API is LESS machinery than what it replaces, not a translation of it. rewrite-target applies per Ingress OBJECT, so qBittorrent (no base path, prefix must be stripped) needed a second Ingress on the shared host, which then had to repeat the first object's TLS secret name, omit the cert-manager annotation conditionally so only one Certificate is requested, and match an ImplementationSpecific regex path for the rewrite's capture group. A URLRewrite filter applies per RULE: one route, one rule per app, the filter on qBittorrent's rule, a plain PathPrefix. All four crutches gone. It is also the only chart without httpRoute.hostnames - it has no ingress.domain either, and a chart-global hostname list could not say which of the ten apps it belonged to, so the hostnames stay in ingress.host and <app>.ingress.host. CI: one render overlay per chart under ci/httproute/, so the assertion is not vacuum-green - ci/run-httproute-match-test.sh goes from 20 held assertions over 10 charts to 36 over 15. freesailarr's overlay moves bazarr onto a host of its own, so the shared-host route AND the per-app route are both rendered; without that the per-app branch would be exercised by nothing. The install-test fixtures stay in Ingress mode: the install runs one configuration per chart, cyberchef already pays that price against the real Gateway-API CRDs, and for paperless-ngx that fixture is the only install of the valkey-subchart path anywhere. README also carries a stale figure fixed here: the HTTPRoute test was still documented at the pilot's 4 assertions over 2 charts, although group 1 had taken it to 20 over 10. Not proven, by construction: that traffic flows. No Gateway controller runs in CI. Not run locally either: k3d, Docker, Trivy. The Gateway branch of NOTES.txt is EXECUTED by the render test (a template error in it fails the job, verified) but its text is printed by no pipeline stage - helm template does not emit notes, and the install-test installs in Ingress mode. Part of #114	2026-10-12 18:15:29 +00:00
jklein	a9ca5302de	feat(paperless-ngx): optional valkey subchart, password from the redis item ... paperless-ngx cannot run without a Redis - PAPERLESS_REDIS is the Celery broker - so every installation needed a second Helm release whose address lived in a 1Password item and in neither chart. This adds our own published valkey chart as an optional dependency, off by default and gated on valkeySubchart.enabled (not valkey.enabled, which the subchart's strict schema rejects and which Helm would only have complained about for whoever first switched it on). Additive, default behaviour unchanged: minor, 12.1.3+up3.2.1 -> 12.2.0+up3.2.1. Configured as a QUEUE (PVC + noeviction, the subchart's defaults), unlike stalwart's cache. Read off the 3.2.1 source rather than assumed: one PAPERLESS_REDIS url serves five consumers - the Celery broker, the result backend (Redis in 3.x, the database before), the channels pub/sub layer and two Django caches. allkeys-lru would not be a trade here but actively wrong, because LRU evicts typeless across the whole keyspace and a backlog of waiting consume tasks is exactly what it reads as cold; a dropped task is a document silently never processed. The price, named in values.yaml rather than smoothed over: two of the five consumers are caches, so a full instance refuses writes - new tasks included - and the answer is memory sizing, not a policy change. All three cache-like consumers are TTL-bounded, which is what keeps that price small. The password comes from the SAME item as the Redis connection, in a third field auth-conf holding the complete line "requirepass <password>". It cannot DEFAULT to secrets.redisSecretRef, and that is a property of Helm, not an omission: a values.yaml is never templated and a parent cannot compute a subchart's value. Measured - a template expression there lands verbatim in the subchart's itemPath and fails as a YAML parse error naming neither the value nor the cause. Forgetting it is therefore caught in values.schema.json, by a conditional rule rather than a fail() in a template. Also measured, and the reason: Helm renders a subchart's templates BEFORE the parent's, so a guard in this chart loses the race to the subchart's own required(), which reports "secrets.authSecretRef" and cannot know the path belongs to the redis item. A check that cannot fire is the mistake found while testing #77, so the first draft of that guard was removed rather than shipped. The schema runs before any rendering and names /valkey/secrets/authSecretRef. One trap found while testing and closed here: "valkeySubchart: null" DELETES the key, Helm then cannot resolve the condition path and treats the dependency as ENABLED - so erasing the switch turned the subchart on with no password set. Unlike every other block in this chart (issue #99), that one is rejected by the schema instead of defaulting. The install-test now runs with the subchart ON and the throwaway redis Deployment removed: the url in the fixtures names ci-valkey-service, so the bundled Valkey is the broker the release actually uses. #65 covered this path by a render only - deliberately, since an unused Valkey would have made that test look like it covered clustering. This is the first real installation of the valkey-subchart path. Documented, not glossed: the password now sits in that item twice and a rotation has to touch both fields; with the subchart on the vault url is a mandatory step, not a hint; and a OnePasswordItem materializes every field of its item, so both Secrets carry all three keys - untidy, harmless, and written down so nobody "tidies it up". Fixes #78	2026-10-02 21:24:22 +00:00
jklein	ac53555e44	paperless-ngx: strict values.schema.json and null-safe hardening ... Largest values surface of the programme, and the first chart where the null-safety covers two workloads. 1. values.schema.json (#68): additionalProperties: false on every chart-owned object level, including files.barcodes, ocr (with enums for the 3.x modes), scripts, tikaGotenberg, the oidc provider block and the SMB CronJob. 2. Null-safety for hardening, probes and resources (#99), for the server AND the SMB consume-mover CronJob. It reuses the defaultedDict helper the barcodes block already introduced (#61) instead of adding a second mechanism - one merge helper, now with more callers. The server's pod context is consumed twice: as the pod securityContext and by the /run-ownership init container, which chowns /run to runAsUser:runAsGroup because s6-overlay refuses to start unless /run is owned by the runtime uid. Both reads now go through the same helper, so the chown cannot disagree with the context it has to match. Erasing that block used to abort rendering with a nil-pointer error; it now falls back to the defaults like every other chart. The CronJob has no probes (documented exception), so only its two contexts and resources are covered. files.storage stays deliberately not null-safe: an erased value yields an invalid PVC, which is the loud class #99 excludes. Rendering is byte-identical to 11.0.0, both with the CI values (which include the #61 regression guard) and with the production paperless-jk bundle values.	2026-08-23 21:59:06 +02:00