8 Commits

Include renames

Author	SHA1	Message	Date
jklein	73c2ec704f	feat!: NetworkPolicy in the last five charts - the sweep is complete ... matrix-synapse, paperless-ngx, freesailarr, valkey, stalwart. One major each. All 29 installable charts plus template-chart now render a policy, and ci/run-networkpolicy-test.sh passes over all 30 with 206 assertions held. MATRIX-SYNAPSE renders TWO policies, one per workload, and the split is the decision in that file. Synapse federates - it must be able to reach any homeserver on the internet, so 443 and 8448 go to any destination - while the delegation nginx serves two static JSON documents and calls nobody, so its egress is DNS and nothing else. A single policy over both pods would have to permit the union, handing full federation reach and the database port to an nginx whose job is to return a constant, and it would read as one decision rather than as a widening. networkPolicy.egress.extraPorts deliberately does not reach the delegation policy for the same reason. PAPERLESS-NGX names the two pods of its own release that this object does NOT cover: the valkey subchart's pod, which carries its own labels and needs no cover because the valkey chart ships a tighter policy of its own; and the consume CronJob's pods, whose pod template carries NO LABELS AT ALL, so no podSelector can match them. That second one is a real gap in the convention. It is tolerable only because the job moves files between mounted volumes and reaches the SMB share through a CSI volume the node mounts - the pod opens no connection of its own - and if that ever changes the CronJob needs labels before it can need a policy. FREESAILARR is ten containers in one pod behind a VPN, and simpler than that sounds: they share a network namespace, so all inter-container traffic is localhost and no policy sees any of it. Because the namespace belongs to gluetun, everything the *arr containers send is encapsulated before it leaves, so three rules describe the whole outbound surface - DNS, UDP 51820 for the tunnel, and 443 for gluetun's server-list API, which cannot go through the tunnel because it runs before the tunnel exists. The tunnel rule is the one that can break this chart and also the best covered: gluetun's health server answers 200 only while the tunnel is up and all three of its probes address it, so a blocked 51820 is a pod that never goes Ready - in the install-test, which brings up a real tunnel. VALKEY AND STALWART, the two charts that had a policy before this issue, are pulled onto the convention without a single rule changing. What changed: - the reasoning moved out of "#" comments, which are part of the rendered object and travel into the cluster, into template comments. Both files failed the new check; that is what it is for. - stalwart gains networkPolicy.ingressController, which fixes a hole its own comment described without being able to solve: it advised putting the controller's namespace into allowedNamespaces, which is namespace-granular and would have opened the http port to every pod in kube-system. - stalwart's example was WRONG and is corrected. It suggested ["ingress-nginx"]; measured on 2026-10-14, ingress-nginx is gone from this cluster entirely and the controller is rke2-traefik in kube-system. A policy written against the old one would have been accepted, looked strict, and silently blocked all web traffic. Their justified specifics are untouched: valkey stays Ingress-only, because it opens no outbound connection and an egress rule would constrain nothing while making the object look stricter; and it keeps 6379 restricted by source with no probeSources at all, which is safe only because all three of its probes are exec probes and put no packet on the network. The convention was written to accommodate both, and homeassistant and satisfactory now invoke valkey's rule in the other direction. ONE CHANGE TO THE TEST, forced by a structural fact rather than by convenience: the rendered-comment assertion now skips documents from a SUBCHART. A dependency is fetched from the Helm registry, not from the copy in this repository, so the policy paperless-ngx and stalwart bundle is whatever valkey version is PUBLISHED - and nothing in the parent chart can change its rendered comments. Holding the parent responsible would have made the test unfixable until valkey's next release and would fail a chart for a file its own diff does not contain. The subchart's own pull request is where it is caught, because this test runs over every chart including valkey itself. All twelve failure paths were re-run against the change. Part of #5	2026-10-14 14:46:11 +00:00
jklein	b4c9823e01	feat(stalwart): MTA-STS routes as a maintainable list, in both routing modes ... The policy route for mta-sts.<domain> belonged to another repository (jk-cluster-config) while pointing at this chart's own Service. It moves here as mtaSts.hosts, a list, so several domains - subdomains included - are maintainable in one place and in whichever routing mode is active. THE LIST IS THE SWITCH. No mtaSts.enabled: "enabled with an empty list" would be a state with no meaning that the chart would then have to catch. The schema rejects the key outright, so the rule is enforced rather than documented. Default [] renders nothing in either mode - the defaults and the unchanged CI fixture both produce a byte-identical helm template against origin/main. Entry form {host, tlsSecretName, clusterIssuer?}. The two TLS fields are read ONLY in Ingress mode; in Gateway mode TLS sits on the listener, so requiring them there would demand a value that does nothing. THE PATH IS HARDWIRED. RFC 8461 fixes /.well-known/mta-sts.txt, rendered with pathType Exact (type: Exact in the route). A values key for a protocol-mandated path can only ever be set wrong, and wrong is invisible from this side. Exact also matters in itself: a prefix on these hostnames would publish the mail server's whole management surface under them. ONE INGRESS PER CLUSTER ISSUER, NOT ONE OBJECT - and the plan's own three requirements forced that. cert-manager reads the two fields at different scopes: cluster-issuer is an ANNOTATION (object-scoped, and absent means nothing is issued for that object at all), while secretName is tls-entry-scoped and yields one Certificate per entry. So an object cannot both request a certificate and leave another host's secret alone, which is what an empty per-entry issuer has to mean. Measured in jk-cluster: ingress-shim names the Certificate AFTER THE SECRET and owns it from the Ingress, so a shared secret must sit in exactly one object (hence grouping the tls list by secret, reusing rss-bridge.tlsGroups' shape) and a secret may not span two issuer groups (hence the abort). Objects are named by issuer, not by index: an index is only free where the object holds no certificate, which is the precondition homeassistant's route naming states for itself. Gateway mode renders ONE HTTPRoute with every hostname - nothing differs per host there - attached to the same Gateway(s) as the WebUI route. WHAT THE CHART CANNOT PROMISE, AND THEREFORE NAMES: it registers a route, it does not create a domain. The domains live in Stalwart's data store, so an entry whose domain is missing there answers 404 at a correctly routed endpoint with every object green - and MTA-STS failures are silent on the SENDER's side, which delivers in the clear and logs nothing here. Same for the _mta-sts.<domain> TXT record and the policy's content. The subdomain trap is documented too: mta-sts.sub.<domain> is one label below *.<domain>, so a parent wildcard certificate does not cover it. Not folded into certificate.extraDnsNames deliberately - that would hang the MAIL certificate on this list, letting a bad policy entry take STARTTLS and IMAPS with it. Minor, additive, empty default: 1.2.0+upv0.16.25 -> 1.3.0+upv0.16.25. CI: ci/httproute/stalwart.yaml enters all three shapes of the entry form (with issuer, without, and a subdomain), taking the HTTPRoute render test from 56 assertions over 21 charts to 58 over 21 - stalwart from 2 to 4. The Ingress half cannot be covered there, since an overlay's whole job is to switch the mode off, so ci/stalwart/test-values.yaml carries two entries and the install-test has a real API server accept them. Fixes #123	2026-10-13 10:41:10 +00:00
jklein	3e969c5659	feat(stalwart): Gateway-API as an alternative to the Ingress ... Group 3 of the #114 sweep. ONE route - not one per node group. #114 describes this chart as rendering "per node group"; its SERVICES and StatefulSets are, the routing object is not and must not be. The route targets exactly ingress.nodeGroup's own http Service and keeps both of the Ingress's guards, because the WebUI's setup and management screens are not safe to spread over a group that may not run the management tasks. ingress.nodeGroup is therefore shared by both modes: it names where the management surface is published from, which is not a property of the Ingress. THE MAIL CERTIFICATE STAYS, and this is the only chart in the sweep where anything of the TLS path does. stalwart.certificate.yaml covers STARTTLS on 25 and implicit TLS on 465/993; those connections pass through neither an ingress controller nor a Gateway. So in Gateway mode ingress.domain is still required (the certificate's dnsNames read it), ingress.clusterIssuer is still required unless certificate.clusterIssuer is set, and the Certificate must NOT be removed as part of a migration - deleting it drops SMTP and IMAP back to Stalwart's self-signed certificate, with clients seeing a name mismatch while every Kubernetes object stays green. Verified in the render: Gateway mode emits Certificate + HTTPRoute and no Ingress. WHAT THIS MODE DROPS - AND HERE IT BREAKS A SECOND CHART OF THE SAME GROUP. Measured on 2026-10-12, the live Ingress on mail.bekemeyer-klein.de carried, all of it operator-supplied through ingress.annotations (this chart sets none of it): enable-cors: true cors-allow-origin: https://webmail.bekemeyer-klein.de cors-allow-methods: GET, POST, OPTIONS cors-allow-headers: Authorization, Content-Type That origin is BULWARK, the webmail chart in this same group. Without the CORS configuration the browser-side webmail cannot call this server's JMAP endpoint at all, and the failure is entirely client-side: a console error, nothing in the server's logs. AND "no counterpart" WOULD HAVE BEEN WRONG. Measured by grepping the CRDs this repo pins (gateway-api v1.6.3 standard channel, ci/gateway-api-crds.sh) rather than inferred: HTTPRoute's filters carry a full 'cors' field (GEP-1767) with allowOrigins/allowMethods/allowHeaders/allowCredentials/maxAge, and its documented level - verbatim from that CRD - is "Support: Extended". responseHeaderModifier is in the same CRD and is also Extended, not Core. The chart still does not render it, for three reasons in order: 1. The chart has NO CORS values. The live configuration is the operator's, and the chart sets no CORS annotations in Ingress mode either - so rendering a filter means inventing a values surface that would exist in Gateway mode only. That asymmetry is its own defect and its own work. 2. Extended support is not support. A rejected route is rejected WHOLE, which would take the WebUI and JMAP down together (the reasoning #119 reached for statusCode); and the observed failure is quieter - Cilium accepts filters.type: CORS and forwards the preflight upstream anyway (cilium/cilium#45272), i.e. Accepted: True with the behaviour absent. 3. Half a CORS is worse than none, and the translatable half is the useless one HERE. A responseHeaderModifier can add the Access-Control-Allow-* headers; the PREFLIGHT has to be answered BY THE GATEWAY, which the CORS filter's own contract states and a bare header filter cannot do. Because the allowed headers include Authorization, EVERY webmail request is preflighted - so the expressible half is exactly the half that does not help. Named, not rebuilt (decision 2 of #114), in values.yaml, in the schema and in the Gateway branch of NOTES.txt, with the operational instruction: configure CORS at the Gateway with that implementation's own support BEFORE moving this host, and test it from the webmail in a browser - curl sends no preflight, so it passes while the webmail fails. NOTES.txt now asks $published (ingress.enabled OR httpRoute.enabled) where it asked ingress.enabled, and two paragraphs that named the ingress controller specifically now name the Gateway in that mode: the auto-ban step (switching API does not avoid or reset it - one proxy address instead of another) and the NetworkPolicy paragraph, where the namespace to allow is the Gateway's data plane rather than an ingress controller's. Minor: additive, defaults unchanged. Empty helm template diff against origin/main with ci/stalwart/test-values.yaml. Part of #114	2026-10-13 08:59:09 +00:00
jlab-renovate-bot	b5b5d9a310	Update docker.io/stalwartlabs/stalwart Docker tag to v0.16.25	2026-10-08 07:35:13 +00:00
jklein	6ade74db80	feat(stalwart): spread each node group over nodes and give each one a PDB ... A two-replica node group was redundancy on paper: nothing kept both pods off one node, and no PodDisruptionBudget kept a drain from taking both at once. Both objects are now rendered per node group, gated on that group's COMPUTED replica count being >= 2. The distribution is a promise, not an effort: maxSkew 1 on kubernetes.io/hostname with whenUnsatisfiable DoNotSchedule, selecting only this group's own pods, so an "mx" and a "full" pod may share a node. The selector comes from the chart (stalwart.groupLabels), never from the operator - a spread constraint whose selector matches no pod is trivially satisfied, distributes nothing and reports nothing. nodeAffinityPolicy: Ignore is set explicitly, and that is the load-bearing detail. Its API default is Honor, under which a narrowing nodeSelector shrinks the set of counted domains until both replicas fit into one: skew 0, both pods co-located, no Pending, no event, and a selector that is correct - so no check can see it. With Ignore the same narrowing gives skew 2 and the second pod is Pending. A render abort was the alternative and would also have forbidden a nodeSelector spanning five nodes, which is legal. A user entry colliding with the chart's own on (topologyKey, whenUnsatisfiable) aborts the render naming both values, because the API rejects the duplicate with a message about a list index and an apply-based path merges the two over patchMergeKey: topologyKey silently, dropping one selector. Minor rather than major, proved with a diff rather than argued: with the shipped defaults (full: 1, mx: 0) and with ci/stalwart/test-values.yaml, helm template renders byte-for-byte what 1.0.1+upv0.16.24 rendered - both diffs empty. New in CI, over every chart on every PR: ci/run-selector-match-test.sh holds every rendered PodDisruptionBudget selector and every topologySpreadConstraints labelSelector against the labels of the pod templates in the same render. Helm itself is the YAML parser (ci/yaml2json), because the runners have neither pyyaml nor yq and a hand-written parser could read nothing and report nothing. podAffinity/podAntiAffinity are out of scope; matchExpressions-only selectors are reported as not checked rather than counted as passing. Two anti-vacuum guards: a chart with an overlay under ci/selector-match/ that renders no selector fails, and a run that checks no selector at all fails. Fixes #96	2026-10-07 19:36:04 +00:00
jlab-renovate-bot	bf2e1009e4	Update docker.io/library/busybox Docker tag to v1.38.0	2026-10-02 22:07:25 +00:00
jklein	07c75e80c1	feat(stalwart): take our own published valkey chart as the subchart ... Replaces the dependency on upstream's valkey chart with this repo's own, resolved from the published index: version: "1.0.0+up9.1.2.alpine" repository: "https://git.jk-dev.de/api/packages/jlab-cluster/helm" The condition is "valkeySubchart.enabled", NOT "valkey.enabled", and that is forced rather than stylistic. Everything under "valkey:" is handed to the subchart as its values and validated against ITS schema, and our valkey chart ships additionalProperties: false with no "enabled" property - so "valkey.enabled" is rejected by the subchart: valkey: - at '': additional properties 'enabled' not allowed Measured. And because Helm skips a DISABLED subchart's schema validation, that error would have surfaced only for whoever first set it to true: every render, lint and install-test in CI would have stayed green. Adding "enabled" to valkey/values.schema.json was rejected as the repair - that chart does not read the key, and it would have needed its own PR and a publish first. The host helper was also wrong for our chart and is fixed: our Service is "<release>-valkey-service", not upstream's bare "<release>-valkey". The values block is rewritten for our chart's schema, with the pair chosen rather than inherited: CACHE (persistence off + allkeys-lru), because everything Stalwart keeps in the in-memory store - rate limiters, auth tokens, sessions, auto-ban entries, coordination pub/sub - is reconstructible, and the node-id lease lives in the data store. The cost is named where it is set: with allkeys-lru the rate limiters and the auto-ban FAIL OPEN under memory pressure, so the answer is to size memory, not to switch to noeviction (which would make writes error and take out authentication instead). secrets.authSecretRef wants an item whose single field "auth-conf" holds the complete "requirepass <pw>" line, not a bare password. Chart.yaml's comments are corrected where they became false: nothing is "consumed as it comes" any more and there is no shortfall of someone else's to document. The Renovate reasoning stays and is now backed by measurement. New trap, in Chart.yaml and in three places in the wiki: NEVER change valkey/ and a chart depending on it in the same PR. The dependency resolves from the published index, so a combined PR lints, packages, renders AND install-tests against the OLD subchart - green on an artifact that is not the one in the diff. Two consecutive PRs. Two things measured, not assumed, because either could have broken the update path with no symptom but "no updates ever": - helm dependency update AND build resolve the exact build-metadata version from the URL alone with an EMPTY HELM_REPOSITORY_CONFIG - a fresh runner's state - so the anonymous index needs no credential and ci/helm-dependencies.sh is untouched; - Renovate 44.132.2 dry run with the dependency pinned low: manager helmv3 matched the file, took registryUrls from Chart.yaml and offered newVersion "1.0.0+up9.1.2.alpine", versioning "helm". The self-hosted index and the "+up..." suffix are both handled. README: the claim "no registry address appears in this repository" is narrowed to what still holds - the publish target and its credentials come from 1Password and are not in the repo. The read path is now in Chart.yaml by necessity, carries no credential, and the host was always in CLAUDE.md. Done here rather than later so the paperless PR finds it already corrected. Re-verified after the change: helm lint --strict (0 failed), helm package (the subchart is embedded in the .tgz), helm template with the subchart OFF and ON (ON was previously impossible), against CI test-values, the rendered config.json, ci/run-quantity-render-test.sh (0 failed), NOTES.txt through the scratch-ConfigMap trick, and the fail paths including the reworded abort. The rendered subchart Service name matches what the helper emits, the cache pairing really renders (maxmemory-policy allkeys-lru, appendonly no, save "", emptyDir) and no PVC appears. Part of #65	2026-10-02 20:59:17 +00:00
jklein	780cb8eafd	feat(stalwart): new chart for the Stalwart mail server ... Cluster-capable from the start, with roles rather than symmetry: one StatefulSet per entry of the "nodes" map, the key passed in STALWART_ROLE. Since v0.16 Stalwart keeps its whole configuration inside the data store and manages it over JMAP, so the chart's scope is deliberately narrow: it renders config.json (exactly the DataStore object), the workloads, the Services, the NetworkPolicies, the Ingress and a cert-manager Certificate. Listeners, blob store, in-memory store, TLS object and cluster roles are operator work in the WebUI, which is why NOTES.txt carries an eight-step first install. config.json stays secret-free: authSecret uses the EnvironmentVariable variant (verified against the serde definition of PostgreSqlStore at v0.16.24), so the password arrives through secretKeyRef and checksum/config stays meaningful. The username cannot follow it - authUsername is a plain String with no secret form - so it is a public value and says so. Services are per PORT, not per group: a NodePort is cluster-unique, so "full" and "mx" can only accept SMTP behind one Service, selected through a per-port pod label. The http port gets a per-group ClusterIP Service instead, so the Ingress reaches exactly ingress.nodeGroup. Four traps, documented rather than explained away: - The role lookup FAILS OPEN. A node whose ClusterRole is missing from the database enables every listener and every task while staying Running with a green probe, and the log line arrives after the listeners are bound. Hence mx at replicas 0, per-port Services, and a NetworkPolicy that is a ceiling independent of what was bound. - NET_BIND_SERVICE is a documented deviation needed to EXEC, not to bind. The image sets setcap cap_net_bind_service=+ep, which makes the kernel fail execve when the bounding set is empty. The values comment says both why it is there and that it is never used, because the second half is what stops the next hardening pass from removing it. - externalTrafficPolicy must be Cluster - no pod runs on a control-plane node while the HA proxy delivers to them, so Local would drop every packet - and the source address that costs is restored only by the HA proxy's PROXY v2 header. - /healthz is rate-limited to 100 requests/60s per source IP, which is a node IP for kubelet probes, so all pods on a node share one bucket. The probe intervals are calculated (6 req/min/pod, ~16 pods per node) and the sum is a comment in values.yaml. A first install needs bootstrap.enabled: true: the setup wizard appears only when config.json is ABSENT, and this chart always renders one, so recovery mode is upstream's documented path for a templated config.json. Second chart in the repo with a dependency - the Valkey project's own chart, off by default behind condition:, because Stalwart's cluster coordinator rides on the in-memory store connection. The render aborts when more than one pod runs with no in-memory store address anywhere. Also wired up: publish-stalwart.yml through the shared block in workflow-util/pipelines, the change-app-version.yml choice list, a Renovate customManagers block anchored on the v prefix (which excludes the moving latest / v0.16 / -alpine tags), CI fixtures, and the README and CLAUDE.md lines that claimed ytdlp-web-player was the only chart with a dependency. Verified locally: ci/helm-dependencies.sh, helm lint --strict (0 failed), helm package, helm template against chart defaults and against ci/stalwart/test-values.yaml, the rendered config.json parsed as JSON with the field types the serde definition requires, ci/run-quantity-render-test.sh (0 failed, both nodes.*.resources paths exercised), NOTES.txt rendered through a scratch ConfigMap, and the null-safety and fail paths (securityContext=null keeps NET_BIND_SERVICE; a cluster without an in-memory store, an unknown port name, a group without http and an unknown ingress.nodeGroup all abort with a named message; the schema rejects timeout, a string timeoutMs and a list-shaped nodes). NO runtime verification was possible - there is no k3d or Docker daemon in this workspace. Part of #65	2026-10-02 20:59:17 +00:00