diff --git a/README.md b/README.md index 78c0399..74bf255 100644 --- a/README.md +++ b/README.md @@ -231,6 +231,7 @@ Measured locally: **30 charts, 408 scalars verified, 0 failed**, no cluster. The | no ingress rule allows every source by accident | `from: []` allows everything, identically to having no `from` at all, while reading in a diff and in `kubectl` output as a restriction — the one mistake in this class that makes a policy *weaker* while looking *stricter* | nobody | | every port a **network** probe addresses is either unrestricted, or the chart offers `networkPolicy.probeSources` **and** says in `NOTES.txt` what a wrong CIDR costs | the kubelet probes from the IP of the worker node running the pod, which no chart can know. A restricted probe port without that address drops every probe: the pod never becomes `Ready`, the rollout times out, and the symptom reads as a broken application | the install-test, in production's CNI — **but not in CI**, see below | | **cross-chart egress**: where a chart dials another chart *of this repository*, its egress rule must contain the **pod** ports behind that chart's Services (#136) | an egress rule naming the **Service** port. kube-proxy does the ClusterIP's destination NAT *before* the policy is evaluated at the pod's interface, so the policy sees `podIP:targetPort` and the rule matches nothing wherever `port` and `targetPort` differ | nobody: the symptom is a **timeout**, not `connection refused`, and no error mentions a NetworkPolicy — `paperless-ngx` 14.0.0 shipped it and cost an outage on 2026-10-15 | +| **the ambient-mesh switch** (#143), on two renders per chart plus the chart's declared defaults: those defaults equal the port and `ipBlock` the test **pins**; with the switch off neither appears; with it on the pinned port is present and is still the **only** ingress port that is no `containerPort`, and a source-restricted rule carrying it also carries the `ipBlock` | the named exception decaying into the hole it was carved around. Without the off-half a chart could render the port unconditionally and nothing would notice; without the only-port half the exception becomes the free port key that was rejected; and **without the pin** — now that the port is a value — the exception would permit whatever the value said, so a typo would ride in through a configurable field | nobody — and in production as a pod unreachable in the mesh, or one whose every probe is dropped the first time `probeSources` is narrowed | Egress ports are otherwise **deliberately not** held against the pod's `containerPort`s: an egress port is a port on the *destination*, and that comparison would fail every chart that reaches a database. @@ -245,7 +246,9 @@ Four things to know before touching it: - **Subchart documents are skipped by the comment assertion**, for a structural reason and not a lenient one: a dependency is fetched from the Helm registry rather than from the copy in this repo, so the policy `paperless-ngx` and `stalwart` bundle is whatever `valkey` version is *published*, and nothing in the parent chart can change its rendered comments. The subchart's own PR is where it is caught, because this test runs over `valkey` too. - **Anti-vacuum guard**, as in the tests above: `ci/networkpolicy/.yaml` is a render-only overlay applied **last**, whose job is to render the **source-restricted probe rule** — a branch no chart's own defaults ever reach, and the one containing the expensive mistake (a `namespaceSelector` and a `podSelector` written as two list entries instead of one would be an OR, opening the port to every pod in `kube-system`). A chart that *has* an overlay and still renders no restricted **probe** rule fails. That counter is specifically the probe rule and not "any restricted rule", because `keycloak` restricts two rules in its own defaults and would have satisfied a looser guard while the branch went unrendered. -Measured locally: **30 charts, 210 assertions held, 0 failed**, no cluster, ~15 seconds. All twelve fail paths were exercised rather than assumed — and one earned its keep immediately: the `probeSources` check was a substring test, so a chart whose key had been **deleted** while the long comment above it still discussed the value passed it. It now looks for the key. The cross-chart assertion added two of those 210, and its own six fail paths were executed the same way: a required port removed from the rule, a wrong number in its place, a peer chart renamed away, the policy's name no longer following the `--` convention, the peers rendering no Service at all (which would make the requirement empty, and is treated as a failure because an empty requirement reads exactly like a satisfied one), and the gated rule made unconditional — which is the failure path of the *gate-off* half, since with the gate off and the rule properly gated the assertion must stay silent. +Measured locally: **30 charts, 333 assertions held, 0 failed**, no cluster, ~25 seconds (the ambient-mesh assertion added a second render per chart, and roughly 10 seconds with it). All twelve fail paths were exercised rather than assumed — and one earned its keep immediately: the `probeSources` check was a substring test, so a chart whose key had been **deleted** while the long comment above it still discussed the value passed it. It now looks for the key. The cross-chart assertion added two of those 210, and its own six fail paths were executed the same way: a required port removed from the rule, a wrong number in its place, a peer chart renamed away, the policy's name no longer following the `--` convention, the peers rendering no Service at all (which would make the requirement empty, and is treated as a failure because an empty requirement reads exactly like a satisfied one), and the gated rule made unconditional — which is the failure path of the *gate-off* half, since with the gate off and the rule properly gated the assertion must stay silent. + +**The ambient-mesh assertion (#143) was built the same way, and ten of its fail paths were executed**: a second, different port added under the exception (rejected — the exception covers exactly one number); `15008` rendered ungated, caught by assertion 2 in the main loop, which renders the switch at its default; the `ipBlock` rendered ungated, which **only** this assertion can catch, because assertion 2 never looks at `from` peers; the `ipBlock` half deleted while `15008` stayed, i.e. the half-set switch the design exists to make impossible, reachable because `ci/networkpolicy/template-chart.yaml` fills `probeSources`; the policy's name taken off the `--` convention; the value present in the schema but absent from `values.yaml`, which a `--set` render alone would not have noticed; a chart whose **declared default port deviates** (three independent checks fire at once: the declared-defaults comparison, the pinned port missing from the render, and the deviating port not being covered by the exception); a port-only chart **declaring `redirectedProbeSources`**, which reaches no rule there; a **deleted `ambientMesh` object** and one carrying only `enabled: true` (both must *render*, on the chart's own defaults — and the guards that make that work were each shown to be load-bearing by removing them); and a **release-level port override**, which must render and does, while the pinned exception demonstrably does **not** stretch to cover it. The name check earned its keep on the first run rather than in a rehearsal: it read the **directory** name, and `template-chart`'s chart is called `xxx`. **What this test does not and cannot cover, named so the gap is not mistaken for coverage:** that traffic still flows. No ingress controller runs in the install-test cluster, so the path from Traefik to a pod is proven nowhere in this pipeline — and, measured on the PR for #5, **k3s exempts the kubelet's probe traffic from policy**, so a blocked probe path does *not* turn the install-test red either (`cyberchef` became `Ready` in 69s with its probe port restricted to an address no probe can come from). What the install-test *does* prove is enforcement of everything the application itself needs: with `outline`'s DNS egress rule removed, its pod never became `Ready` and the job failed after 5m59s. So the net under this convention is real, but it is an **egress** net — which raises the stakes for assertion 2, since a wrong ingress port is caught mechanically here or not at all. @@ -693,7 +696,18 @@ Every port of a workload falls into one of four, and this is the part to get rig #### Values -`networkPolicy.enabled`, `.allowedNamespaces`, `.probeSources`, `.ingressController`, `.egress.extraPorts` — and a chart ships **only the ones that reach a rule**. A chart may add a **named** egress switch where a port depends on something the values cannot see: `paperless-ngx`' `.egress.mailImport` opens IMAPS `993`, off by default, because its mail accounts are rows in its own database and no value says whether a release fetches mail. The bar for one of those: it names what it opens, and it is **not** a second switch for a port some *other* value already decides — that is why the Tika/Gotenberg rule stayed gated on `paperless.files.tikaGotenberg.enabled` and got no policy switch of its own. Two values that can drift apart re-arm the outage the rule was fixed for. A value that silently does nothing is worse than one that is absent, because it reads as a control: `samba` and `satisfactory` carry `enabled` alone (every inbound rule is open to any source), and `teamspeak-server` has `allowedNamespaces` but no `probeSources`. +`networkPolicy.enabled`, `.allowedNamespaces`, `.probeSources`, `.ingressController`, `.ingress.ambientMesh.*`, `.egress.extraPorts` — and a chart ships **only the ones that reach a rule**, with `.enabled` and `.ingress.ambientMesh` the two every chart has. + +**Three of those names overlap as substrings and are three different things**, which is worth stating because a search for `ingress` matches all of them: top-level `ingress` is the Ingress **object** a chart renders, `networkPolicy.ingress` is the policy's **inbound side**, and `networkPolicy.ingressController` is the **source selector** for the controller's pods — a *sibling* of `networkPolicy.ingress` and no part of it. The 21 charts carrying `ingressController` had **no** inbound escape hatch before #143, and one search run during that work reported them as having one on the strength of the substring alone. + +**And a fourth name, whose confusion risk is the larger one**, because these two are both lists of probe sources and sit two levels apart: + +| Name | Whose addresses | Default | Empty means | +| --- | --- | --- | --- | +| `networkPolicy.probeSources` | **yours** — the worker node CIDRs, which no chart can know | `[]` | the probe rule is **open** | +| `networkPolicy.ingress.ambientMesh.redirectedProbeSources` | **Istio's** — the address its data plane rewrites the probe source onto | `[169.254.7.127/32]`, i.e. shipped *filled*, because unlike the node IPs it is a constant the chart **can** know | n/a — clearing it restores the default, deliberately: the two halves of the switch are not separable | + +It is named `redirectedProbeSources` and not `probeSources` for exactly that reason: one is where probes come from normally, the other where they come from *after Istio redirects them*. A chart may add a **named** egress switch where a port depends on something the values cannot see: `paperless-ngx`' `.egress.mailImport` opens IMAPS `993`, off by default, because its mail accounts are rows in its own database and no value says whether a release fetches mail. The bar for one of those: it names what it opens, and it is **not** a second switch for a port some *other* value already decides — that is why the Tika/Gotenberg rule stayed gated on `paperless.files.tikaGotenberg.enabled` and got no policy switch of its own. Two values that can drift apart re-arm the outage the rule was fixed for. A value that silently does nothing is worse than one that is absent, because it reads as a control: `samba` and `satisfactory` carry `enabled` alone (every inbound rule is open to any source), and `teamspeak-server` has `allowedNamespaces` but no `probeSources`. **No chart ships `allowedNamespaces` pre-filled, ever.** A chart that named `mc-router`, or `paperless-jk`, or `stalwart` in its defaults would be baking in an assumption about *this* cluster's namespace layout — which is exactly the kind of claim a chart cannot keep true. So the default is the empty list and the chart is **restrictive**; filling it is the **deployment adaptation**, made where the namespace layout is actually visible. That is why a sweep like this produces a list of release-side one-liners: those are the expected procedure, not a defect list, and the breaking-change major on each chart is what announces them. @@ -701,6 +715,55 @@ Every port of a workload falls into one of four, and this is the part to get rig `ingressController` is a separate value with two keys rather than a line in `allowedNamespaces` for one reason: **its two selectors render as one `from` entry, which is an AND** — those pods in that namespace. Written as two list entries they would be an OR, and putting `kube-system` into `allowedNamespaces` instead would open the port to every pod in the cluster's own namespace. Its default is **measured, not assumed**: `ingress-nginx` is gone from this cluster, the only `IngressClass` is `traefik`, and the controller is the DaemonSet `rke2-traefik` in `kube-system` — which also serves the Gateway API here, so one source covers the Ingress and the HTTPRoute path alike. `ingress.className` says nothing about it: a class names an implementation, not the pods the traffic comes from. +#### The ambient-mesh switch, and the one named exception in the render test (#143) + +**`networkPolicy.ingress.ambientMesh`, in all 30 charts, switched off by default.** Istio runs on this cluster in ambient mode, and Istio's own documentation is explicit that *"ambient does not and will never bypass Kubernetes `NetworkPolicy` enforcement"* — so until this value existed, **no chart here could put a pod into the mesh at all**. Additive, unchanged default, unchanged render: **minor** per chart, and the render was diffed against the previous release per chart rather than asserted. + +**The shape, and why both numbers are values rather than literals:** + +```yaml +networkPolicy: + ingress: + ambientMesh: + enabled: false + port: 15008 + redirectedProbeSources: + - 169.254.7.127/32 +``` + +**Both numbers are Istio *constants*, not standards** — and a value with a documented default is the better home for a vendor constant than a template literal, because an operator can correct it when Istio changes the number without waiting for a chart release. Their provenance, written on each one in every chart rather than assumed: + +- **`15008`** is Istio's HBONE port. Not registered with IANA for this, prescribed by no protocol, and **unknown to Kubernetes**. Istio picked it; Istio can change it. +- **`169.254.7.127`** is the address Istio's data plane rewrites the kubelet's probe source onto, and it is recognisable as vendor magic rather than protocol three ways: it is **link-local**, `169.254.0.0/16` (RFC 3927), a range reserved for node-local use and **never routed** — exactly the range one picks a magic address out of; Istio's IPv6 counterpart `fd16:9254:7127:1337:ffff:ffff:ffff:ffff` carries the **same digits** (`9254:7127` *is* `169.254.7.127`) plus `1337`, which is what vendor constants look like; and **Kubernetes knows the address in neither the NetworkPolicy API nor the kubelet** — only Istio's own data plane does. + +**Both read for ztunnel 1.30.6.** If Istio changes either, **the failure is silent**: a pod unreachable inside the mesh, or — for the address — probes dropped, and only where `probeSources` is non-empty, which is nowhere today. Nothing in this repository and nothing in Kubernetes would report it. What to re-check then: the port and address Istio's release notes name, this default in all 30 charts, **and** the pinned expectation in `ci/run-networkpolicy-test.sh`. + +`redirectedProbeSources` is a **list** and not a string, and this is the one case where that does not contradict "the value is a constant, not a configuration": a **dual-stack** cluster needs **both** of Istio's addresses at the same time, not one of two. That is a nameable case rather than speculation. This cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default — an `ipBlock` of the wrong family is accepted by the API server and matches nothing — and it is documented as the addition to make *alongside*, never *instead*. + +**Deleting the object is safe**, per the null-safety rule (#99): every read goes through a fallback reproducing the chart's own declared defaults, so `ambientMesh: null`, or an object carrying only `enabled: true`, renders the documented defaults. Both guards are load-bearing and were executed, not assumed — the direct chained read `$npIngress.ambientMesh.enabled` fails with `nil pointer evaluating interface {}.enabled` on a deleted object, which is the shape of the defect #136 shipped one value away; and without the port fallback a nilled `port` renders **`port: 0`**, an accepted and silently wrong policy. + +**It renders two things, and that they are one value is the whole design.** + +1. **`15008/TCP` inbound** — ztunnel's HBONE tunnel, where every inbound mesh connection arrives. +2. **`169.254.7.127/32` as an additional peer**, but only where the rule restricts sources at all. Inside the mesh the kubelet's probes are **redirected** and no longer arrive from the worker node's address. + +Point 2 is the one nobody thinks of, which is why the two halves cannot be had separately. An operator who adds `15008` to a port list by hand writes a policy that passes review — and the first time `probeSources` is narrowed, every probe is dropped, the pod never becomes `Ready`, and the symptom reads as a broken application rather than as a policy. That is the probe-path trap of this section one value further along. **A switch whose halves can only be had together cannot be set halfway**, so this is a named intention and not a port list. Charts with no `probeSources` at all render the port half only: their probe rule is never source-restricted, and an `ipBlock` for a redirected probe would match nothing while making the rule read as if it did something. + +**The interesting part is the CI, because the obvious implementation was the worst one.** Assertion 2 of `ci/run-networkpolicy-test.sh` holds every ingress port against the selected pod spec's `containerPort`s, and `15008` is deliberately none — ztunnel terminates it, not the application. A free `ingress.extraPorts` key would have had to exempt *operator-supplied ports* from that assertion, which opens it for **all 30 charts**: every mistyped ingress port, the NodePort mistake included, would then pass. So the assertion keeps its edge and gets **one named exception for one port number, gated on one value** — and it is not granted in the main loop at all, which renders every chart with the value at its default, so `15008` stays an ordinary failure there. + +**And since the port is a value, the exception cannot be checked against the rendered number.** "Whatever the value says" is design A's hole wearing design C's name: a mistyped port would pass simply by sitting in the configurable field. So **the expectation is pinned in the test** — `AMBIENT_PORT` and `AMBIENT_CIDR` — and assertion 7 checks two things against that statement: + +1. the render **at chart defaults** produces exactly that port and exactly that `ipBlock`, and +2. **every chart's `values.yaml` declares the same defaults**, read back through `ci/yaml2json` rather than by regex — because a substring test over a commented file is how the `probeSources` check was once fooled by a deleted key whose comment survived. + +**A chart that drifts is red.** If Istio changes a constant, it takes 30 values files **and** the two pinned lines — and the pinned lines are what force someone to look. That block is therefore also the single place in the repository that states what we hold Istio's numbers to be, which makes it the answer to "where does `169.254.7.127` come from" as well; the provenance is written there. The renders set **only** `enabled=true` and never a port override, deliberately: the pin is a statement about the defaults *we ship*, not about what a release may choose. An operator overriding the value in their own release is outside this test's reach, as is true of every value here — and that is not a new gap. + +**An exception without its own check is the hole it was carved around**, so assertion 7 holds it to six things on **two renders of every chart**: the declared defaults match the pin; with the switch off, neither the pinned port nor the `ipBlock` is present; with it on, the pinned port is present, it is still the **only** ingress port that is not a `containerPort`, and wherever the rule carrying it restricts sources it carries the `ipBlock` too. The probe-source half is conditional on the chart offering `probeSources`: the five charts whose probe rule is never source-restricted render no `ipBlock`, so for them the key must be **absent** — a value reaching no rule reads as a control that does nothing — and that absence is asserted rather than tolerated. The policy is found by the `--` name convention, because an exception must not be granted to an object the test could not identify. It lives in the existing script rather than a new one for the reason #136's cross-chart assertion does, only more so: **the exception weakens assertion 2, so it belongs beside it** — a separate script can be green while the exception it is paired with is wrong, and nothing would connect the two. + +**Two things this does not prove, and they are not footnotes.** That `15008` *alone* suffices comes from Istio's documentation; what is measured on this cluster is only which listeners ztunnel opens. And the probe-source half reaches no release, because `probeSources` is empty in every chart (`stalwart` sets it to `[]` explicitly) — CI proves its render through the overlays in `ci/networkpolicy`, not its effect. Both become provable at the first namespace to join the mesh, and **none carries `istio.io/dataplane-mode`** (measured 2026-10-19, 0 of 97). Whether ztunnel needs a source selector of its own is open for the same reason — but it is *not* the unknowable case it looked like: ztunnel runs with its own pod IPs in `istio-system` (no `hostNetwork` on the DaemonSet, Istio 1.30.6), so a peer for it would be a `namespaceSelector` plus `podSelector`, not a node-IP `ipBlock`. + +**Consumer side:** switching this on for a release is a value key *and* a chart version in **one** PR in `jk-cluster-config`, because Fleet does not retry a failed apply — and `networkPolicy` carries `additionalProperties: false` in all 30 charts, so an unknown key does not get ignored, it fails the render. + #### What these policies do not do — and it belongs in every chart, not just here - **Egress is coarse: by port, never by destination.** A `NetworkPolicy` matches `ipBlock` CIDRs and pod/namespace selectors and **never hostnames**, so "only to the database host" is not expressible at all. This cluster's CNI is Canal (Flannel plus Calico for policy) and has no FQDN rule; Calico's `GlobalNetworkSet` could express one but would bind every chart to Calico — the same argument that kept implementation-specific objects out of the Gateway-API sweep (#114) — so it is **not used**. This is a limit of the object, and each chart names what it costs *that* chart. diff --git a/apache-tika/Chart.yaml b/apache-tika/Chart.yaml index c6e5b5c..975097a 100644 --- a/apache-tika/Chart.yaml +++ b/apache-tika/Chart.yaml @@ -5,5 +5,5 @@ icon: https://www.apache.org/logos/res/tika/default.png type: application -version: 11.0.0+up3.3.1.0.full +version: 11.1.0+up3.3.1.0.full appVersion: "3.3.1.0-full" diff --git a/apache-tika/templates/apache-tika.networkpolicy.yaml b/apache-tika/templates/apache-tika.networkpolicy.yaml index 7d24aa0..ab230be 100644 --- a/apache-tika/templates/apache-tika.networkpolicy.yaml +++ b/apache-tika/templates/apache-tika.networkpolicy.yaml @@ -50,6 +50,25 @@ selects. {{- $ns := .Release.Namespace }} {{- $probeSources := $np.probeSources | default list }} {{- $allowedNs := $np.allowedNamespaces | default list }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -83,6 +102,37 @@ spec: - ports: - protocol: TCP port: 9998 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -110,6 +160,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/apache-tika/values.schema.json b/apache-tika/values.schema.json index f2d820e..7298850 100644 --- a/apache-tika/values.schema.json +++ b/apache-tika/values.schema.json @@ -31,6 +31,37 @@ "type": "string" } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/apache-tika/values.yaml b/apache-tika/values.yaml index b749e35..9ae1fab 100644 --- a/apache-tika/values.yaml +++ b/apache-tika/values.yaml @@ -115,6 +115,135 @@ networkPolicy: # everything else that needs the port has to be in it. The chart adds the own # namespace and allowedNamespaces to the same rule for exactly that reason. probeSources: [] + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/bentopdf/Chart.yaml b/bentopdf/Chart.yaml index a4d6091..00d9491 100644 --- a/bentopdf/Chart.yaml +++ b/bentopdf/Chart.yaml @@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/alam00000/bentopdf/main/public/images/fa type: application -version: 2.0.0+up2.8.8 +version: 2.1.0+up2.8.8 appVersion: "2.8.8" maintainers: diff --git a/bentopdf/templates/bentopdf.networkpolicy.yaml b/bentopdf/templates/bentopdf.networkpolicy.yaml index 55ad14a..d176a6e 100644 --- a/bentopdf/templates/bentopdf.networkpolicy.yaml +++ b/bentopdf/templates/bentopdf.networkpolicy.yaml @@ -44,6 +44,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -76,6 +95,37 @@ spec: - ports: - protocol: TCP port: 8080 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -118,6 +168,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/bentopdf/values.schema.json b/bentopdf/values.schema.json index 9cd201d..8e4ad2e 100644 --- a/bentopdf/values.schema.json +++ b/bentopdf/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/bentopdf/values.yaml b/bentopdf/values.yaml index b86f830..e0bd4b7 100644 --- a/bentopdf/values.yaml +++ b/bentopdf/values.yaml @@ -362,6 +362,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/bulwark/Chart.yaml b/bulwark/Chart.yaml index 2bae6a9..f03d14c 100644 --- a/bulwark/Chart.yaml +++ b/bulwark/Chart.yaml @@ -21,7 +21,7 @@ type: application # duplicated here. The short version: eight tag shapes, six of them things # nobody should pin, and plain `docker` versioning excludes all six - so that # block deliberately carries NO versioningTemplate. -version: 2.0.0+up1.12.0 +version: 2.1.0+up1.12.0 appVersion: "1.12.0" maintainers: diff --git a/bulwark/templates/bulwark.networkpolicy.yaml b/bulwark/templates/bulwark.networkpolicy.yaml index c8f60b5..71ff6fa 100644 --- a/bulwark/templates/bulwark.networkpolicy.yaml +++ b/bulwark/templates/bulwark.networkpolicy.yaml @@ -50,6 +50,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := .Values.ingress.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -84,6 +103,37 @@ spec: - ports: - protocol: TCP port: 3000 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -126,6 +176,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/bulwark/values.schema.json b/bulwark/values.schema.json index e036389..0097ae5 100644 --- a/bulwark/values.schema.json +++ b/bulwark/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/bulwark/values.yaml b/bulwark/values.yaml index aa294cf..783e12c 100644 --- a/bulwark/values.yaml +++ b/bulwark/values.yaml @@ -883,6 +883,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/calibre-web-automated/Chart.yaml b/calibre-web-automated/Chart.yaml index aee34f2..5e961e5 100644 --- a/calibre-web-automated/Chart.yaml +++ b/calibre-web-automated/Chart.yaml @@ -5,7 +5,7 @@ name: calibre-web-automated description: A Helm chart to deploy calibere-web-automated icon: https://raw.githubusercontent.com/crocodilestick/Calibre-Web-Automated/main/README_images/cwa-logo-round-dark.png -version: 8.0.0+upv4.0.6 +version: 8.1.0+upv4.0.6 appVersion: v4.0.6 maintainers: diff --git a/calibre-web-automated/templates/calibre-web-automated.networkpolicy.yaml b/calibre-web-automated/templates/calibre-web-automated.networkpolicy.yaml index cff3cb5..f176f9e 100644 --- a/calibre-web-automated/templates/calibre-web-automated.networkpolicy.yaml +++ b/calibre-web-automated/templates/calibre-web-automated.networkpolicy.yaml @@ -45,6 +45,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := .Values.ingress.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -79,6 +98,37 @@ spec: - ports: - protocol: TCP port: 8083 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -121,6 +171,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/calibre-web-automated/values.schema.json b/calibre-web-automated/values.schema.json index cfb24d7..cc5f825 100644 --- a/calibre-web-automated/values.schema.json +++ b/calibre-web-automated/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/calibre-web-automated/values.yaml b/calibre-web-automated/values.yaml index dd8a5e5..f30ec46 100644 --- a/calibre-web-automated/values.yaml +++ b/calibre-web-automated/values.yaml @@ -250,6 +250,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/ci/run-networkpolicy-test.sh b/ci/run-networkpolicy-test.sh index 4acfab7..f79c7a3 100755 --- a/ci/run-networkpolicy-test.sh +++ b/ci/run-networkpolicy-test.sh @@ -83,6 +83,15 @@ # because a green tick for a code path nobody entered is worse than an # honest gap (README, "Two rules for every check added here"). # - PORT RANGES (endPort). Same treatment: reported, never counted. +# - THAT 15008 IS ENOUGH TO REACH A POD IN THE AMBIENT MESH. Assertion 7 +# asserts the RENDER of networkPolicy.ingress.ambientMesh in both +# directions, and nothing beyond it. That the HBONE port alone suffices +# comes from Istio's documentation; measured on this cluster is only which +# listeners ztunnel opens, and no namespace carries istio.io/dataplane-mode +# yet, so there is no path on which it could be measured end to end. The +# probe-source half (169.254.7.127/32) is in the same position: its render is +# asserted, its effect is not, because probeSources is empty in every chart +# of this repository. # # AND THE ANTI-VACUUM GUARDS, because this test could be green over nothing: # @@ -492,6 +501,20 @@ for policy in policies: # port is a port on the DESTINATION and has nothing to do with this pod's # containerPorts, so holding egress against them would fail every chart that # reaches a database. + # + # THE ONE NAMED EXCEPTION, AND IT IS NOT GRANTED HERE (Gitea #143). Port + # 15008 - ztunnel's HBONE tunnel - is terminated by the mesh and not by the + # application, so it is legitimately no containerPort while + # networkPolicy.ingress.ambientMesh is true. This loop renders every chart + # with that value at its DEFAULT, which is false, so for this assertion + # 15008 is an ordinary number and stays an ordinary failure - that is the + # negative half of the exception and it costs nothing. The exception itself + # lives in assertion 7, on a SECOND render with the value on, and it permits + # exactly that one number. It is deliberately not implemented as "skip + # operator-supplied ports when a gate is set": that was design A in #143 and + # would have opened this assertion for all 30 charts, letting every mistyped + # ingress port through - including the NodePort mistake this assertion + # exists for. for i, rule in enumerate(spec.get("ingress") or []): if not isinstance(rule, dict): fail("%s ingress[%d] is not a mapping" % (tag, i)) @@ -870,6 +893,11 @@ EGRESS_PEER_ROWS="paperless-ngx|apache-tika,gotenberg|paperless.files.tikaGotenb # leaves the parsed JSON in $2. The render/split/parse steps are repeated here # instead of refactored out of the loop on purpose: the five assertions above # are load-bearing and were not worth disturbing for this. +# RCJ_WITH_OVERLAY=1 additionally applies ci/networkpolicy/.yaml, last, +# exactly as the main loop does. Off by default so the cross-chart rows below +# keep rendering the consumer the way its release does; assertion 7 switches it +# on, because the overlay is the ONLY way the source-restricted branch - and +# with it the probe-source half of the ambient switch - is reachable at all. render_chart_json() { RCJ_CHART="$1" RCJ_OUT="$2" @@ -883,6 +911,9 @@ render_chart_json() { RCJ_VALUES=(-f "$CI_DIR/trivy/values/$RCJ_CHART.yaml") RCJ_LABEL="ci/trivy/values/$RCJ_CHART.yaml" fi + if [ "${RCJ_WITH_OVERLAY:-}" = "1" ] && [ -f "$OVERLAY_DIR/$RCJ_CHART.yaml" ]; then + RCJ_VALUES+=(-f "$OVERLAY_DIR/$RCJ_CHART.yaml") + fi "$ROOT/ci/helm-dependencies.sh" "$ROOT/$RCJ_CHART" >/dev/null || return 1 RCJ_MANIFEST="$RCJ_OUT.manifest.yaml" helm template "$RELEASE" "$ROOT/$RCJ_CHART" \ @@ -1210,6 +1241,537 @@ PY fi done +# --- Assertion 7: the ambient-mesh switch (Gitea #143) ----------------------- +# networkPolicy.ingress.ambientMesh renders the two things a pod needs before it +# can be reached inside Istio's ambient mesh, and the whole design question of +# #143 was how to allow the first of them without blunting assertion 2: +# +# 15008/TCP inbound - ztunnel's HBONE tunnel. Terminated by the +# mesh, so it is NOT a containerPort of the +# application. Assertion 2 would reject it, and +# it is right to reject it by default. +# 169.254.7.127/32 as a peer - where probeSources is non-empty. Inside the +# mesh the kubelet's probes are redirected and +# no longer arrive from the worker node's +# address. +# +# WHAT THIS ASSERTION IS FOR. The exception assertion 2 grants is NAMED and +# single-valued: one port number, gated on one value. The danger of any +# exception is that it stops being checked and becomes the hole it was carved +# around, so the exception is held to five things at once, on TWO renders of +# every chart: +# +# the chart's DECLARED DEFAULTS (read back through ci/yaml2json) +# - enabled must default to false, and the port and the probe source must be +# exactly the numbers pinned below. A chart whose default drifted would +# otherwise ship a port no other chart ships while the exception kept being +# granted to the pinned one. +# switch OFF (the main loop's render, reused) +# - the pinned port must be ABSENT. Without this half the exception is a hole: a +# chart could render the port unconditionally and nothing would notice, +# because assertion 7 would find it present and assertion 2 is where it +# would have failed. +# - 169.254.7.127/32 must be absent too, for the same reason. +# switch ON (only "enabled=true" is set, never a port override - the pin is a +# statement about the defaults WE ship) +# - the PINNED port must be PRESENT, or the value is decorative. +# - the PINNED port must be the ONLY ingress port that is not a +# containerPort. This is the line between a named exception and design A +# of #143: every other mistyped or NodePort number still fails here, with +# the switch on. Note "pinned": since #143 made the port a VALUE, checking +# the exception against the rendered value would permit whatever the value +# said, which is design A's hole under design C's name. +# - wherever the rule carrying 15008 restricts sources, that rule's "from" +# must contain 169.254.7.127/32. This is the inseparability of the two +# halves, asserted rather than trusted - and it is REACHABLE, because +# these renders apply ci/networkpolicy/.yaml, which is where +# probeSources is non-empty (template-chart, gotenberg). +# +# And the policy is found by the NAME CONVENTION, --*, the same +# way assertion 6 finds it: an exception must not be granted to an object this +# test did not identify. A chart whose policy stops following the convention +# fails here instead of silently losing its coverage. +# +# TWO LIMITS OF THIS ASSERTION, named rather than left to be discovered: +# +# - ASSERTIONS 1, 3, 4 AND 5 ARE NOT RE-RUN ON THE SWITCHED-ON RENDER. Only +# assertion 2's subject matter is, because that is the one the exception +# touches. Assertion 3 is the interesting omission: with the switch on and +# probeSources empty, 15008 sits in a rule with no "from", and assertion 3's +# two justifications for an open rule - a network probe addresses the port, +# or a NodePort publishes it - cover neither. That is intended rather than +# overlooked. The port has to be reachable from wherever ztunnel dials from, +# and that address is not knowable from a render: ztunnel runs with its own +# pod IPs in istio-system (measured 2026-10-19, no hostNetwork on the +# DaemonSet, Istio 1.30.6), but whether a peer of its own is needed cannot +# be settled until a namespace actually joins the mesh. Teaching assertion 3 +# a third justification before then would be a rule written against a guess. +# - THE PROBE-SOURCE HALF IS CONDITIONAL ON THE CHART OFFERING probeSources, +# like assertion 4 and the overlay guard. A chart with exec-only probes has +# no probe on the network to redirect, so the ipBlock would match nothing. +# +# BUILT INTO THIS SCRIPT rather than added as a new test with its own pipeline +# job - the same decision as assertion 6 for #136, and here it is stronger than +# convenience. The exception WEAKENS assertion 2, so it belongs next to it: a +# separate script can be green while the exception it is paired with is wrong, +# and nothing would connect the two. Everything else is already here as well - +# the job runs over every chart on every pull request and so cannot be skipped +# by a path filter, helm is already the YAML parser through ci/yaml2json, and +# the verdict, grouping and summary plumbing is shared. +# +# EVERY CHART MUST OFFER THE VALUE, and a chart that does not fails. Same +# mechanism as assertion 0: the mistake arrives by copy into a NEW chart whose +# own diff nobody reads for it, which is exactly how garage (#81) and keycloak +# (#83) arrived with no policy at all after #5 had been filed. +# THE PINNED EXPECTATION, and it is pinned HERE rather than read from the chart +# on purpose (Gitea #143). Both numbers are values with documented defaults, so +# that an operator can correct an Istio constant without waiting for a chart +# release - but an exception that permits "whatever the value says" is design A's +# hole wearing design C's name: a mistyped port would pass simply by sitting in +# the configurable field, and that is the single reason design A was rejected. +# +# So the test states what WE believe Istio's numbers to be, and asserts two +# things against that statement: the render at CHART DEFAULTS produces exactly +# this port and exactly this ipBlock, AND every chart's values.yaml DECLARES +# these same defaults. A chart that drifts is red. +# +# Which makes this block the one place in the repository that says what we hold +# Istio's constants to be - and therefore also the answer to "where does +# 169.254.7.127 come from": +# +# 15008 Istio's HBONE port. Not registered with IANA for this, +# prescribed by no protocol, unknown to Kubernetes. Istio +# picked it and Istio can change it. +# 169.254.7.127/32 the address Istio's data plane rewrites the kubelet's probe +# source onto. LINK-LOCAL (169.254.0.0/16, RFC 3927), a range +# reserved for node-local use and never routed. Istio's IPv6 +# counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries +# the SAME DIGITS plus 1337, which is what vendor constants +# look like. Neither the NetworkPolicy API nor the kubelet has +# any notion of it. +# +# BOTH READ FOR ztunnel 1.30.6. If Istio changes either, the failure is silent - +# a pod unreachable in the mesh, or probes dropped where probeSources is +# non-empty, which is nowhere today. Changing them means 30 values files AND +# these two lines, and these two lines are what force someone to look. +# +# An operator who overrides the value in their own release is outside this test's +# reach. That is true of every value in this repository and is not a new gap - +# and it is why these renders set ONLY "enabled=true" and never a port override: +# the pin is a statement about the defaults WE ship, not about what a release may +# choose. +AMBIENT_PORT=15008 +AMBIENT_CIDR="169.254.7.127/32" +AMBIENT_VALUE="networkPolicy.ingress.ambientMesh" +for CHART in "${CHARTS[@]}"; do + # A chart whose main-loop render already failed has nothing to compare + # against; it is reported there and would only produce a second, derived + # error here. + case " $FAILED " in *" $CHART "*) continue ;; esac + [ -s "$WORK/$CHART.json" ] || continue + + echo "::group::ambient-mesh switch: $CHART" + ROW_FAIL="" + ON_JSON="$WORK/ambient-$CHART-on.json" + if ! RCJ_WITH_OVERLAY=1 render_chart_json "$CHART" "$ON_JSON" \ + --set "$AMBIENT_VALUE.enabled=true"; then + sed 's/^/ /' "$WORK/xerr" + echo " FAIL $CHART does not render with $AMBIENT_VALUE.enabled=true" + echo "::error title=ambient-mesh $CHART::$CHART does not render with $AMBIENT_VALUE.enabled=true. Since Gitea #143 every chart in this repository offers that value, and networkPolicy carries additionalProperties: false in all of them - so an unknown key does not get ignored, it fails the render. Add the object to values.yaml AND to values.schema.json (schema and template belong together per chart), with enabled: false, port: \$AMBIENT_PORT and redirectedProbeSources: [\$AMBIENT_CIDR], and render that port plus - inside the source-restricted branch - that ipBlock. Copy template-chart/templates/xxx.networkpolicy.yaml." + ROW_FAIL=1 + fi + + # The policy's name is built from Chart.yaml's name, which is NOT always the + # directory name - template-chart is called "xxx", and the whole point of that + # chart is that it is a copy template. Reading the directory name here found + # exactly that on the first run, so the name comes from Chart.yaml. + CHART_NAME="$(sed -n 's/^name:[[:space:]]*//p' "$ROOT/$CHART/Chart.yaml" \ + | head -1 | tr -d '[:space:]')" + if [ -z "$CHART_NAME" ]; then + echo " FAIL $CHART has no name in Chart.yaml" + echo "::error title=ambient-mesh $CHART::$CHART/Chart.yaml declares no name, so the NetworkPolicy's conventional name could not be derived." + ROW_FAIL=1 + fi + + # The chart's DECLARED defaults, read with the same parser as everything else + # (ci/yaml2json prints a values file back as JSON). Not a regex over + # values.yaml: the pinned expectation is about numbers and a list, and a + # substring test over a commented file is how the probeSources check in + # assertion 4 was once fooled by a deleted key whose comment survived. + VALUES_JSON="$WORK/$CHART.values.json" + if [ -z "$ROW_FAIL" ]; then + if ! helm template y2j "$YAML2JSON" -f "$ROOT/$CHART/values.yaml" 2>"$WORK/xerr" \ + | awk '/^\{/ { print; exit }' >"$VALUES_JSON" || [ ! -s "$VALUES_JSON" ]; then + sed 's/^/ /' "$WORK/xerr" + echo " FAIL $CHART/values.yaml could not be re-read as JSON" + echo "::error title=ambient-mesh $CHART::$CHART/values.yaml could not be re-read by ci/yaml2json, so the chart's DECLARED ambient-mesh defaults could not be compared with the numbers this test pins. That is a parse failure, not a policy finding." + ROW_FAIL=1 + fi + fi + + if [ -z "$ROW_FAIL" ]; then + set +e + python3 - "$CHART" "$RELEASE" "$AMBIENT_VALUE" "$WORK/$CHART.json" "$ON_JSON" \ + "$ROOT/$CHART" "$CHART_NAME" "$AMBIENT_PORT" "$AMBIENT_CIDR" "$VALUES_JSON" \ + >"$WORK/ambient-$CHART.out" 2>"$WORK/ambient-$CHART.pyerr" <<'PY' +import json, re, sys + +(chart, release, gate, off_path, on_path, chart_dir, chart_name, + pinned_port, pinned_cidr, values_json) = sys.argv[1:11] + +# Pinned by the caller, never read from the chart - see the long note above +# AMBIENT_PORT in this script. +PORT = int(pinned_port) +CIDR = pinned_cidr + +lines = [] +def ok(msg): lines.append("OK|%s: %s" % (chart, msg)) +def skip(msg): lines.append("SKIP|%s: %s" % (chart, msg)) +def fail(msg): lines.append("FAIL|%s: %s" % (chart, msg)) + + +def docs_of(path): + return [d for d in (json.load(open(path)).get("docs") or []) if isinstance(d, dict)] + + +def pod_templates(node): + out = [] + if isinstance(node, dict): + for key, value in node.items(): + if (key == "template" and isinstance(value, dict) + and isinstance(value.get("spec"), dict) + and "containers" in value["spec"]): + out.append(value) + out.extend(pod_templates(value)) + elif isinstance(node, list): + for value in node: + out.extend(pod_templates(value)) + return out + + +def labels_of(template): + meta = template.get("metadata") + labels = meta.get("labels") if isinstance(meta, dict) else None + return labels if isinstance(labels, dict) else {} + + +def matches(selector, labels): + for key, value in selector.items(): + if key not in labels or str(labels[key]) != str(value): + return False + return True + + +def container_ports(template): + out = [] + spec = template.get("spec") or {} + for key in ("containers", "initContainers"): + for container in spec.get(key) or []: + if not isinstance(container, dict): + continue + for port in container.get("ports") or []: + if isinstance(port, dict) and port.get("containerPort") is not None: + out.append((int(port["containerPort"]), port.get("name"))) + return out + + +def own_policies(path): + """This chart's OWN NetworkPolicies, by the -- name + convention - so a bundled subchart's policy can neither satisfy nor break an + assertion the parent chart is responsible for.""" + prefix = "%s-%s-" % (release, chart_name) + out = [] + for doc in docs_of(path): + if doc.get("kind") != "NetworkPolicy": + continue + name = ((doc.get("metadata") or {}).get("name")) or "" + if name.startswith(prefix): + out.append((name, doc)) + return out + + +def ingress_rules(doc): + spec = doc.get("spec") if isinstance(doc.get("spec"), dict) else {} + return [r for r in (spec.get("ingress") or []) if isinstance(r, dict)] + + +def rule_ports(rule): + """[(number_or_name, protocol)] for the rule's ports.""" + out = [] + for port in rule.get("ports") or []: + if not isinstance(port, dict): + continue + value = port.get("port") + proto = str(port.get("protocol") or "TCP") + if isinstance(value, int) or (isinstance(value, str) and value.isdigit()): + out.append((int(value), proto)) + elif value is not None: + out.append((value, proto)) + return out + + +def cidrs_of(rule): + out = [] + for peer in rule.get("from") or []: + if isinstance(peer, dict) and isinstance(peer.get("ipBlock"), dict): + out.append(str(peer["ipBlock"].get("cidr"))) + return out + + +# --- The chart DECLARES the pinned defaults ------------------------------- +# The render with the switch on already proves the SCHEMA declares the key - an +# undeclared key cannot pass additionalProperties: false. What that render +# cannot prove is what the chart's own defaults SAY, and since #143 made both +# Istio constants configurable, that is the half the pinned exception rests on: +# the exception below is granted to the number THIS SCRIPT names, so a chart +# whose default has drifted away from it must be red here rather than quietly +# carrying a different port than every other chart. +values_yaml = "" +try: + values_yaml = open("%s/values.yaml" % chart_dir, encoding="utf-8").read() +except IOError: + pass + +declared = json.load(open(values_json)) +node = (((declared.get("networkPolicy") or {}).get("ingress") or {}) + .get("ambientMesh")) +if not isinstance(node, dict): + fail("renders with %s.enabled=true - so the schema knows the key - but the " + "chart's own values.yaml declares no ambientMesh OBJECT (found %r). A " + "value that exists only in the schema is reachable through --set and " + "findable nowhere: the defaults are what an operator reads. Ship the " + "object with enabled: false, port: %d and redirectedProbeSources: " + "[%s], as in template-chart." % (gate, node, PORT, CIDR)) +else: + problems = [] + if node.get("enabled") not in (False, None): + problems.append("enabled is %r and must default to false - the switch is " + "additive and no release may get it by accident" + % node.get("enabled")) + if node.get("port") != PORT: + problems.append("port defaults to %r, but this test pins Istio's HBONE " + "port at %d" % (node.get("port"), PORT)) + # Conditional on the chart offering probeSources, exactly as the + # inseparability check below and assertion 4 are. A chart whose probe rule + # is never source-restricted renders no ipBlock, so this key would reach no + # rule - and a value that silently does nothing reads as a control, which is + # worse than one that is absent (README, "Values, and shipping only the live + # ones"). So for those charts the key must be ABSENT, and that is asserted + # rather than tolerated. + sources = node.get("redirectedProbeSources") + offers_ps = bool(re.search(r"^\s*probeSources\s*:", values_yaml, re.M)) + if offers_ps and sources != [CIDR]: + problems.append("redirectedProbeSources defaults to %r, but this test " + "pins Istio's redirected probe source at [%r]" + % (sources, CIDR)) + if not offers_ps and sources is not None: + problems.append("redirectedProbeSources is declared (%r) although this " + "chart offers no networkPolicy.probeSources, so its probe " + "rule is never source-restricted and no ipBlock is ever " + "rendered - the key would reach no rule and read as a " + "control that does nothing" % sources) + if problems: + fail("the chart's declared ambient-mesh defaults do not match the " + "numbers ci/run-networkpolicy-test.sh pins: %s. Both are ISTIO " + "CONSTANTS, and they are values precisely so an operator can " + "correct them without a chart release - but the exception assertion " + "2 grants the HBONE port is pinned to the number THIS TEST knows, " + "never to whatever the value says, because a value-driven exception " + "would let a mistyped port ride in through a configurable field. " + "That is the hole design A of #143 was rejected for. So: if Istio " + "changed the constant, change it in all 30 charts AND in the pinned " + "expectation here - the drift is meant to be loud." + % "; ".join(problems)) + else: + ok("declares the pinned ambient-mesh defaults verbatim - enabled false, " + "port %d, %s - so the named exception below is granted to a number " + "this chart really ships, and not to whatever a value happened to say" + % (PORT, ("redirectedProbeSources [%s]" % CIDR) if offers_ps else + "and no redirectedProbeSources, correctly, since this chart " + "renders no ipBlock at all")) + +off_policies = own_policies(off_path) +on_policies = own_policies(on_path) + +if not off_policies or not on_policies: + fail("renders no NetworkPolicy named %s-%s-* (off: %d, on: %d), so the " + "ambient-mesh exception could not be held against this chart's own " + "policy. The convention is {{ .Release.Name }}-{{ .Chart.Name }}-; " + "if the name changed, this assertion has to follow - it is not skipped, " + "because an exception granted to an object this test cannot identify is " + "worth less than no exception at all." + % (release, chart_name, len(off_policies), len(on_policies))) +else: + # --- The switch OFF half ------------------------------------------------ + off_hits, off_cidr_hits = [], [] + for name, doc in off_policies: + for i, rule in enumerate(ingress_rules(doc)): + for value, proto in rule_ports(rule): + if value == PORT: + off_hits.append("%s ingress[%d] %s/%s" % (name, i, value, proto)) + for cidr in cidrs_of(rule): + if cidr == CIDR: + off_cidr_hits.append("%s ingress[%d]" % (name, i)) + if off_hits: + fail("allows ingress port %d with %s.enabled at its default (false): %s. The " + "exception assertion 2 grants for %d is gated on that value, and " + "this is the half that keeps it from being a hole - a port rendered " + "unconditionally would otherwise be found present by this assertion " + "and never held against the pod spec by assertion 2. Gate the port " + "on the value." % (PORT, gate, ", ".join(off_hits), PORT)) + elif off_cidr_hits: + fail("carries the ipBlock %s with %s.enabled at its default (false): %s. The " + "probe-source half of the switch is for the mesh only, where the " + "kubelet's probes are redirected; outside it the address matches " + "nothing and makes the rule read as if it did something." + % (CIDR, gate, ", ".join(off_cidr_hits))) + else: + ok("with %s.enabled at its default the policy allows neither port %d nor the " + "ipBlock %s - the negative half of assertion 2's named exception, " + "without which the exception would be a hole" % (gate, PORT, CIDR)) + + # --- The switch ON half ------------------------------------------------- + templates = [] + for doc in docs_of(on_path): + templates.extend(pod_templates(doc)) + + found_port, bad_ports, restricted_without_cidr, carriers = [], [], [], [] + for name, doc in on_policies: + spec = doc.get("spec") if isinstance(doc.get("spec"), dict) else {} + selector = spec.get("podSelector") + match_labels = (selector or {}).get("matchLabels") or {} + selected = [t for t in templates if matches(match_labels, labels_of(t))] + declared = set() + for t in selected: + for number, pname in container_ports(t): + declared.add(number) + if pname: + declared.add(pname) + for i, rule in enumerate(ingress_rules(doc)): + for value, proto in rule_ports(rule): + if value == PORT: + found_port.append("%s ingress[%d] %d/%s" % (name, i, PORT, proto)) + carriers.append((name, i, rule)) + if proto != "TCP": + bad_ports.append("%s ingress[%d] renders %d as %s, but " + "HBONE is TCP" % (name, i, PORT, proto)) + elif value not in declared and selected: + bad_ports.append( + "%s ingress[%d] allows %r, which is no containerPort of " + "the selected pod spec" % (name, i, value)) + + if not found_port: + fail("renders no ingress port %d with %s.enabled=true, so the value is " + "decorative: the chart accepts it, the schema declares it, and the " + "policy that comes out is the same one as before. A pod whose " + "namespace has joined the ambient mesh is then unreachable, because " + "ambient does not bypass NetworkPolicy enforcement." % (PORT, gate)) + else: + ok("with %s.enabled=true the policy allows %s - ztunnel's HBONE tunnel, the one " + "port assertion 2 grants a named exception for" + % (gate, ", ".join(found_port))) + + if bad_ports: + fail("with %s.enabled=true the policy allows ingress port(s) that are NOT " + "containerPorts of the selected pod spec and are NOT the one number " + "the exception covers (%d): %s. This is the line between a named " + "exception and the free port key that was rejected as design A of " + "#143 - with the switch on, every OTHER wrong port still fails, " + "including a NodePort number written where a pod port belongs." + % (gate, PORT, "; ".join(bad_ports))) + elif found_port: + ok("with %s.enabled=true, port %d is still the ONLY ingress port of this chart " + "that is not a containerPort of the selected pod spec - the exception " + "covers exactly one number and nothing rode in beside it" + % (gate, PORT)) + + # The inseparability of the two halves, where the render can reach it. + # + # Conditional on the chart OFFERING probeSources, exactly as assertion 4 and + # the overlay guard are, and for the same reason: a chart with exec-only + # probes (valkey, minecraft-server) puts no probe packet on the network at + # all, so there is no probe to redirect and 169.254.7.127/32 would be an + # entry that matches nothing - the kind of strict-looking, empty rule this + # script exists to catch. Those charts render the port half only, and say so + # in their values.yaml. + offers_probe_sources = bool(re.search( + r"^\s*probeSources\s*:", values_yaml, re.M)) + for name, i, rule in carriers: + if not rule.get("from"): + continue + if not offers_probe_sources: + ok("%s ingress[%d] carries port %d in a source-restricted rule and " + "no ipBlock %s - correct here, because this chart offers no " + "networkPolicy.probeSources: its probes put no packet on the " + "network, so there is no redirected probe to allow" + % (name, i, PORT, CIDR)) + continue + if CIDR in cidrs_of(rule): + ok("%s ingress[%d] carries port %d AND the ipBlock %s in the same " + "\"from\" list - the two halves of the switch are inseparable, " + "which is the reason it is one value and not two" + % (name, i, PORT, CIDR)) + else: + restricted_without_cidr.append( + "%s ingress[%d] (peers: %s)" + % (name, i, ", ".join(cidrs_of(rule)) or "no ipBlock at all")) + if restricted_without_cidr: + fail("with %s.enabled=true the rule carrying port %d restricts its sources but " + "does not include the ipBlock %s: %s. Inside the mesh the kubelet's " + "probes are redirected and arrive from that address instead of from " + "the worker node's, so a restricted rule without it drops every " + "probe - the pod never becomes Ready and the symptom reads as a " + "broken application rather than as a policy. That is assertion 4 of " + "this script, re-created one value further along, and it is the " + "whole reason the two halves are a single switch." + % (gate, PORT, CIDR, "; ".join(restricted_without_cidr))) + if not any(r.get("from") for _n, _i, r in carriers): + skip("the rule carrying port %d has no \"from\" list in this render, so " + "the probe-source half (%s) is not reachable here and was not " + "checked. That is the state of every chart's DEFAULTS - probeSources " + "is empty everywhere - and it is why ci/networkpolicy/.yaml " + "exists: template-chart and gotenberg render the restricted branch " + "and the half IS checked there. Reported, not counted as passing." + % (PORT, CIDR)) + +for line in lines: + print(line) +PY + RC=$? + set -e + if [ "$RC" -gt 1 ]; then + sed 's/^/ /' "$WORK/ambient-$CHART.pyerr" + echo "::error title=ambient-mesh $CHART::the ambient-mesh comparison itself failed for $CHART (python exit $RC). Treated as a failure, never as a pass." + ROW_FAIL=1 + else + while IFS='|' read -r VERDICT DETAIL; do + case "$VERDICT" in + OK) + TOTAL_ASSERTIONS=$((TOTAL_ASSERTIONS + 1)) + echo " ok $DETAIL" + ;; + SKIP) echo " gap $DETAIL" ;; + FAIL) + ROW_FAIL=1 + echo " FAIL $DETAIL" + echo "::error title=ambient-mesh $CHART::$DETAIL" + ;; + *) [ -n "$VERDICT" ] && echo " $VERDICT $DETAIL" ;; + esac + done <"$WORK/ambient-$CHART.out" + fi + fi + + echo "::endgroup::" + if [ -n "$ROW_FAIL" ]; then + FAILED="$FAILED $CHART(ambient-mesh)" + echo "ambient-mesh $CHART: FAILED" + else + AMBIENT_OK="${AMBIENT_OK:-} $CHART" + echo "ambient-mesh $CHART: ok" + fi +done + # --- Report ------------------------------------------------------------------ count() { set -- $1; echo "$#"; } listing() { set -- $1; [ "$#" -eq 0 ] && { echo "–"; return; }; printf '`%s`' "$1"; shift; for C in "$@"; do printf ', `%s`' "$C"; done; echo; } @@ -1225,6 +1787,7 @@ SUMMARY="${GITHUB_STEP_SUMMARY:-/dev/null}" echo "| Result | Count | Charts |" echo "|---|---|---|" echo "| Policies checked | $(count "$CHECKED") | $(listing "$CHECKED") |" + echo "| Ambient-mesh switch held (Gitea #143) | $(count "${AMBIENT_OK:-}") | $(listing "${AMBIENT_OK:-}") |" echo "| Failed | $(count "$FAILED") | $(listing "$FAILED") |" echo echo "$TOTAL_ASSERTIONS assertion(s) held in total." diff --git a/cyberchef/Chart.yaml b/cyberchef/Chart.yaml index 7b6dec9..3e75799 100644 --- a/cyberchef/Chart.yaml +++ b/cyberchef/Chart.yaml @@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/gchq/CyberChef/master/src/web/static/ima type: application -version: 10.0.0+up11.4.0 +version: 10.1.0+up11.4.0 appVersion: "11.4.0" maintainers: diff --git a/cyberchef/templates/cyberchef.networkpolicy.yaml b/cyberchef/templates/cyberchef.networkpolicy.yaml index 5f95b85..0bcb73d 100644 --- a/cyberchef/templates/cyberchef.networkpolicy.yaml +++ b/cyberchef/templates/cyberchef.networkpolicy.yaml @@ -51,6 +51,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -85,6 +104,37 @@ spec: - ports: - protocol: TCP port: 8080 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -128,6 +178,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/cyberchef/values.schema.json b/cyberchef/values.schema.json index 6aace01..87755b6 100644 --- a/cyberchef/values.schema.json +++ b/cyberchef/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/cyberchef/values.yaml b/cyberchef/values.yaml index b2113d0..da06a9c 100644 --- a/cyberchef/values.yaml +++ b/cyberchef/values.yaml @@ -189,6 +189,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/freesailarr/Chart.yaml b/freesailarr/Chart.yaml index 6e14a3e..8a179bc 100644 --- a/freesailarr/Chart.yaml +++ b/freesailarr/Chart.yaml @@ -8,7 +8,7 @@ description: A Helm chart to deploy a Wireguard-tunneled media automation stack type: application -version: 5.0.0+up1.0.0 +version: 5.1.0+up1.0.0 # The chart's own version marker, freely chosen and tied to no upstream. The # stack bundles ten independently versioned projects (nine of them on by # default), so there is no single "app version" to track here. diff --git a/freesailarr/templates/freesailarr.networkpolicy.yaml b/freesailarr/templates/freesailarr.networkpolicy.yaml index 9ea4dfd..a65cc76 100644 --- a/freesailarr/templates/freesailarr.networkpolicy.yaml +++ b/freesailarr/templates/freesailarr.networkpolicy.yaml @@ -68,6 +68,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -125,6 +144,37 @@ spec: port: 6767 - protocol: TCP port: 5007 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -168,6 +218,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/freesailarr/values.schema.json b/freesailarr/values.schema.json index 4b580d1..5aac2bc 100644 --- a/freesailarr/values.schema.json +++ b/freesailarr/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/freesailarr/values.yaml b/freesailarr/values.yaml index c54e786..c1c5fed 100644 --- a/freesailarr/values.yaml +++ b/freesailarr/values.yaml @@ -1412,6 +1412,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/garage/Chart.yaml b/garage/Chart.yaml index a262667..8d6d0d6 100644 --- a/garage/Chart.yaml +++ b/garage/Chart.yaml @@ -19,7 +19,7 @@ type: application # offer those as "updates", which is why the garage block in renovate.json5 # carries a versioningTemplate anchored to ^v..$ - same # shape as the stalwart block, for the same reason. -version: 2.0.0+upv2.4.1 +version: 2.1.0+upv2.4.1 appVersion: "v2.4.1" maintainers: diff --git a/garage/templates/garage.networkpolicy.yaml b/garage/templates/garage.networkpolicy.yaml index f5a989b..285adf7 100644 --- a/garage/templates/garage.networkpolicy.yaml +++ b/garage/templates/garage.networkpolicy.yaml @@ -54,6 +54,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -140,6 +159,37 @@ spec: - ports: - protocol: TCP port: {{ $settings.ports.admin | int64 }} + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -167,6 +217,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/garage/values.schema.json b/garage/values.schema.json index 3248d4b..2654f84 100644 --- a/garage/values.schema.json +++ b/garage/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/garage/values.yaml b/garage/values.yaml index 36c5069..bdd32a7 100644 --- a/garage/values.yaml +++ b/garage/values.yaml @@ -576,6 +576,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/gotenberg/Chart.yaml b/gotenberg/Chart.yaml index bf1dd51..4573e43 100644 --- a/gotenberg/Chart.yaml +++ b/gotenberg/Chart.yaml @@ -5,7 +5,7 @@ icon: https://gotenberg.dev/img/logo.png type: application -version: 9.1.0+up8.38.1 +version: 9.2.0+up8.38.1 appVersion: "8.38.1" maintainers: diff --git a/gotenberg/templates/gotenberg.networkpolicy.yaml b/gotenberg/templates/gotenberg.networkpolicy.yaml index 8ad7d24..8da3f15 100644 --- a/gotenberg/templates/gotenberg.networkpolicy.yaml +++ b/gotenberg/templates/gotenberg.networkpolicy.yaml @@ -54,6 +54,25 @@ selects. {{- $ns := .Release.Namespace }} {{- $probeSources := $np.probeSources | default list }} {{- $allowedNs := $np.allowedNamespaces | default list }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -86,6 +105,37 @@ spec: - ports: - protocol: TCP port: 3000 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -113,6 +163,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/gotenberg/values.schema.json b/gotenberg/values.schema.json index e450c7a..5fc452b 100644 --- a/gotenberg/values.schema.json +++ b/gotenberg/values.schema.json @@ -31,6 +31,37 @@ "type": "string" } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/gotenberg/values.yaml b/gotenberg/values.yaml index 9794e8b..5ab18e7 100644 --- a/gotenberg/values.yaml +++ b/gotenberg/values.yaml @@ -140,6 +140,135 @@ networkPolicy: # everything else that needs the port has to be in it. The chart adds the own # namespace and allowedNamespaces to the same rule for exactly that reason. probeSources: [] + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # REMOTE ASSETS: lets Chromium fetch over TCP 80 and 443, to any # destination. OFF by default, and both halves of that are deliberate. diff --git a/homeassistant/Chart.yaml b/homeassistant/Chart.yaml index 9804892..a36f326 100644 --- a/homeassistant/Chart.yaml +++ b/homeassistant/Chart.yaml @@ -4,8 +4,7 @@ description: A Helm chart to deploy Home Assistant on Kubernetes icon: https://www.home-assistant.io/images/home-assistant-logo.svg type: application -version: 7.0.0+up2026.8.3 - +version: 7.1.0+up2026.8.3 appVersion: "2026.8.3" maintainers: diff --git a/homeassistant/templates/homeassistant.networkpolicy.yaml b/homeassistant/templates/homeassistant.networkpolicy.yaml index 069404f..a45fcae 100644 --- a/homeassistant/templates/homeassistant.networkpolicy.yaml +++ b/homeassistant/templates/homeassistant.networkpolicy.yaml @@ -56,6 +56,25 @@ selects. {{- $ns := .Release.Namespace }} {{- $probeSources := $np.probeSources | default list }} {{- $allowedNs := $np.allowedNamespaces | default list }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -86,6 +105,37 @@ spec: - ports: - protocol: TCP port: 8123 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -110,6 +160,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/homeassistant/values.schema.json b/homeassistant/values.schema.json index d148ceb..367d217 100644 --- a/homeassistant/values.schema.json +++ b/homeassistant/values.schema.json @@ -14,6 +14,37 @@ "type": "object", "additionalProperties": false, "properties": { + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "enabled": { "type": "boolean" }, diff --git a/homeassistant/values.yaml b/homeassistant/values.yaml index 656bc78..c004f13 100644 --- a/homeassistant/values.yaml +++ b/homeassistant/values.yaml @@ -288,3 +288,132 @@ networkPolicy: # everything else that needs the port has to be in it. The chart adds the own # namespace and allowedNamespaces to the same rule for exactly that reason. probeSources: [] + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + diff --git a/keycloak/Chart.yaml b/keycloak/Chart.yaml index 5cafd20..7b8106a 100644 --- a/keycloak/Chart.yaml +++ b/keycloak/Chart.yaml @@ -43,7 +43,7 @@ type: application # procedure lives in NOTES.txt and on the wiki page instead of in a # packageRules entry. Read the "MINOR UPGRADES" section of NOTES.txt before # merging an app-version PR that changes the minor. -version: 2.0.0+up26.8.0 +version: 2.1.0+up26.8.0 appVersion: "26.8.0" maintainers: diff --git a/keycloak/templates/keycloak.networkpolicy.yaml b/keycloak/templates/keycloak.networkpolicy.yaml index e58887a..8d64518 100644 --- a/keycloak/templates/keycloak.networkpolicy.yaml +++ b/keycloak/templates/keycloak.networkpolicy.yaml @@ -81,6 +81,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -166,6 +185,37 @@ spec: - ports: - protocol: TCP port: {{ $settings.ports.management | int64 }} + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -193,6 +243,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/keycloak/values.schema.json b/keycloak/values.schema.json index 2c56f73..403ed74 100644 --- a/keycloak/values.schema.json +++ b/keycloak/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/keycloak/values.yaml b/keycloak/values.yaml index c3fe0b3..d766169 100644 --- a/keycloak/values.yaml +++ b/keycloak/values.yaml @@ -967,6 +967,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/matrix-synapse/Chart.yaml b/matrix-synapse/Chart.yaml index afa2208..b3c0c1e 100644 --- a/matrix-synapse/Chart.yaml +++ b/matrix-synapse/Chart.yaml @@ -3,7 +3,7 @@ name: matrix-synapse description: A Helm chart to deploy a Matrix Synapse server on Kubernetes icon: https://raw.githubusercontent.com/element-hq/synapse/develop/docs/favicon.png type: application -version: 8.0.0+upv1.159.0 +version: 8.1.0+upv1.159.0 appVersion: "v1.159.0" maintainers: - name: jklein \ No newline at end of file diff --git a/matrix-synapse/templates/matrix-synapse.networkpolicy.yaml b/matrix-synapse/templates/matrix-synapse.networkpolicy.yaml index 79ecf57..34fffa4 100644 --- a/matrix-synapse/templates/matrix-synapse.networkpolicy.yaml +++ b/matrix-synapse/templates/matrix-synapse.networkpolicy.yaml @@ -49,6 +49,25 @@ WHAT THESE POLICIES DO NOT DO: {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -76,6 +95,37 @@ spec: - ports: - protocol: TCP port: 8008 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's @@ -121,6 +171,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/matrix-synapse/values.schema.json b/matrix-synapse/values.schema.json index cdfaf9f..2915e78 100644 --- a/matrix-synapse/values.schema.json +++ b/matrix-synapse/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/matrix-synapse/values.yaml b/matrix-synapse/values.yaml index 97782c2..7893c14 100644 --- a/matrix-synapse/values.yaml +++ b/matrix-synapse/values.yaml @@ -364,6 +364,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports FOR SYNAPSE ONLY. These deliberately do NOT # widen the delegation server's policy: an entry meant for Synapse's diff --git a/minecraft-router/Chart.yaml b/minecraft-router/Chart.yaml index bf555fc..7445877 100644 --- a/minecraft-router/Chart.yaml +++ b/minecraft-router/Chart.yaml @@ -5,7 +5,7 @@ icon: https://avatars.githubusercontent.com/u/988985 type: application -version: 5.1.0+up1.48.0 +version: 5.2.0+up1.48.0 appVersion: "1.48.0" maintainers: diff --git a/minecraft-router/templates/minecraft-router.networkpolicy.yaml b/minecraft-router/templates/minecraft-router.networkpolicy.yaml index 47e903a..144d8f5 100644 --- a/minecraft-router/templates/minecraft-router.networkpolicy.yaml +++ b/minecraft-router/templates/minecraft-router.networkpolicy.yaml @@ -54,6 +54,25 @@ selects. {{- $ns := .Release.Namespace }} {{- $probeSources := $np.probeSources | default list }} {{- $allowedNs := $np.allowedNamespaces | default list }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -105,6 +124,37 @@ spec: - ports: - protocol: TCP port: {{ .Values.server.apiPort | int64 }} + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -129,6 +179,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/minecraft-router/values.schema.json b/minecraft-router/values.schema.json index 51a8346..2cf7c12 100644 --- a/minecraft-router/values.schema.json +++ b/minecraft-router/values.schema.json @@ -31,6 +31,37 @@ "type": "string" } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/minecraft-router/values.yaml b/minecraft-router/values.yaml index 2ea58ef..b30e877 100644 --- a/minecraft-router/values.yaml +++ b/minecraft-router/values.yaml @@ -202,6 +202,135 @@ networkPolicy: # everything else that needs the port has to be in it. The chart adds the own # namespace and allowedNamespaces to the same rule for exactly that reason. probeSources: [] + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/minecraft-server/Chart.yaml b/minecraft-server/Chart.yaml index 74e393d..291e8fb 100644 --- a/minecraft-server/Chart.yaml +++ b/minecraft-server/Chart.yaml @@ -23,7 +23,7 @@ type: application # continuously, so a pod restart could pull a different image than the last one # without anything being recorded anywhere. The dated tags are immutable, so the # image is now pinned by the chart version like every other chart in this repo. -version: 7.0.0+up2026.8.2.java25 +version: 7.1.0+up2026.8.2.java25 appVersion: "2026.8.2-java25" maintainers: diff --git a/minecraft-server/templates/minecraft-server.networkpolicy.yaml b/minecraft-server/templates/minecraft-server.networkpolicy.yaml index c71c01d..8366395 100644 --- a/minecraft-server/templates/minecraft-server.networkpolicy.yaml +++ b/minecraft-server/templates/minecraft-server.networkpolicy.yaml @@ -53,6 +53,19 @@ selects. {{- $np := .Values.networkPolicy }} {{- $ns := .Release.Namespace }} {{- $allowedNs := $np.allowedNamespaces | default list }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same + thing as the top-level .Values.ingress (the Ingress object) - both match + a search for "ingress". */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -92,6 +105,49 @@ spec: - ports: - protocol: TCP port: {{ .Values.server.port | int64 }} + {{- if $ambientEnabled }} + {{- /* networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP is ztunnel's + HBONE tunnel, the port every inbound connection arrives on once the + namespace carries istio.io/dataplane-mode - ambient "does not and will + never bypass Kubernetes NetworkPolicy enforcement", so without it a pod + in the mesh is unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh holds a NAMED exception for exactly this number, gated on exactly + this value, and proves on a second render that it stays the only such + port and that it is absent while the value is false. + + THE PORT HALF ONLY, and that is not an omission. Elsewhere this switch + also adds 169.254.7.127/32 as a probe source, because inside the mesh the + kubelet's probes are redirected away from the worker node's address. + This chart offers no networkPolicy.probeSources at all, so the rule + carrying the probe port is never restricted by source - and an + unrestricted rule needs no probe address. Measured in the render: three + exec probes, which put no packet on the network at all. + WHICH RULE IT JOINS: + the game-port rule, in BOTH of its branches. That rule is open when the + Service is a NodePort (players arrive from outside, already masqueraded) + and restricted to the namespace otherwise, and the two branches are + separate rules in this template - so the port is added twice, or it + would be missing from whichever branch a release happens to render. + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- end }} {{- if not (eq .Values.server.service.type "NodePort") }} {{- /* @@ -107,6 +163,49 @@ spec: - ports: - protocol: TCP port: {{ .Values.server.port | int64 }} + {{- if $ambientEnabled }} + {{- /* networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP is ztunnel's + HBONE tunnel, the port every inbound connection arrives on once the + namespace carries istio.io/dataplane-mode - ambient "does not and will + never bypass Kubernetes NetworkPolicy enforcement", so without it a pod + in the mesh is unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh holds a NAMED exception for exactly this number, gated on exactly + this value, and proves on a second render that it stays the only such + port and that it is absent while the value is false. + + THE PORT HALF ONLY, and that is not an omission. Elsewhere this switch + also adds 169.254.7.127/32 as a probe source, because inside the mesh the + kubelet's probes are redirected away from the worker node's address. + This chart offers no networkPolicy.probeSources at all, so the rule + carrying the probe port is never restricted by source - and an + unrestricted rule needs no probe address. Measured in the render: three + exec probes, which put no packet on the network at all. + WHICH RULE IT JOINS: + the game-port rule, in BOTH of its branches. That rule is open when the + Service is a NodePort (players arrive from outside, already masqueraded) + and restricted to the namespace otherwise, and the two branches are + separate rules in this template - so the port is added twice, or it + would be missing from whichever branch a release happens to render. + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} from: - namespaceSelector: matchLabels: diff --git a/minecraft-server/values.schema.json b/minecraft-server/values.schema.json index 8816d55..8d9eb0a 100644 --- a/minecraft-server/values.schema.json +++ b/minecraft-server/values.schema.json @@ -24,6 +24,30 @@ "type": "string" } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/minecraft-server/values.yaml b/minecraft-server/values.yaml index bd1b6e7..6b1670e 100644 --- a/minecraft-server/values.yaml +++ b/minecraft-server/values.yaml @@ -352,6 +352,71 @@ networkPolicy: # matches nothing and is silently ineffective. Note the granularity: an entry # opens the port to EVERY pod in that namespace. allowedNamespaces: [] + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # It renders 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod whose + # namespace has joined the mesh is unreachable. + # + # ONE HALF SHORTER THAN ELSEWHERE, deliberately. In other charts this switch + # also adds 169.254.7.127/32 as a probe source, because inside the mesh the + # kubelet's probes are redirected away from the worker node's address. This + # chart has no networkPolicy.probeSources at all, so the rule carrying the + # probe port is never restricted by source - and an unrestricted rule needs + # no probe address. An ipBlock for one would match nothing while making the + # rule look like it did something, which is the kind of strict-looking empty + # rule ci/run-networkpolicy-test.sh exists to catch; that script ties the + # exemption to the absence of probeSources rather than to a chart name. + # + # NOT PROVEN: that 15008 alone is enough comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens, and no + # namespace carries istio.io/dataplane-mode yet. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/outline/Chart.yaml b/outline/Chart.yaml index 4fcf627..b5550e6 100644 --- a/outline/Chart.yaml +++ b/outline/Chart.yaml @@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/outline/outline/main/public/images/icon- type: application -version: 8.1.0+up1.11.0 +version: 8.2.0+up1.11.0 appVersion: 1.11.0 maintainers: diff --git a/outline/templates/outline.networkpolicy.yaml b/outline/templates/outline.networkpolicy.yaml index 3be11b3..9ae9c9b 100644 --- a/outline/templates/outline.networkpolicy.yaml +++ b/outline/templates/outline.networkpolicy.yaml @@ -51,6 +51,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -85,6 +104,37 @@ spec: - ports: - protocol: TCP port: 3000 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -127,6 +177,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/outline/values.schema.json b/outline/values.schema.json index 5087fe9..44900c7 100644 --- a/outline/values.schema.json +++ b/outline/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/outline/values.yaml b/outline/values.yaml index 4921a97..0b01618 100644 --- a/outline/values.yaml +++ b/outline/values.yaml @@ -241,6 +241,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/paperless-ngx/Chart.yaml b/paperless-ngx/Chart.yaml index e896ac3..f088939 100644 --- a/paperless-ngx/Chart.yaml +++ b/paperless-ngx/Chart.yaml @@ -5,8 +5,7 @@ icon: https://raw.githubusercontent.com/paperless-ngx/paperless-ngx/dev/src-ui/s type: application -version: 15.0.0+up3.3.1 - +version: 15.1.0+up3.3.1 appVersion: 3.3.1 maintainers: diff --git a/paperless-ngx/templates/paperless-server.networkpolicy.yaml b/paperless-ngx/templates/paperless-server.networkpolicy.yaml index 765b691..6687d62 100644 --- a/paperless-ngx/templates/paperless-server.networkpolicy.yaml +++ b/paperless-ngx/templates/paperless-server.networkpolicy.yaml @@ -79,6 +79,25 @@ that the egress rule below contains their containerPorts. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -113,6 +132,37 @@ spec: - ports: - protocol: TCP port: 8000 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -155,6 +205,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/paperless-ngx/values.schema.json b/paperless-ngx/values.schema.json index 5ff881a..e7f5d27 100644 --- a/paperless-ngx/values.schema.json +++ b/paperless-ngx/values.schema.json @@ -87,6 +87,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/paperless-ngx/values.yaml b/paperless-ngx/values.yaml index ee4cea0..27b5cef 100644 --- a/paperless-ngx/values.yaml +++ b/paperless-ngx/values.yaml @@ -636,6 +636,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Outbound IMAPS (993) for paperless' mail import. OFF, and it has to be a # value rather than something the chart derives: the mail accounts are ROWS diff --git a/patchmon/Chart.yaml b/patchmon/Chart.yaml index 74bde87..9e48ebe 100644 --- a/patchmon/Chart.yaml +++ b/patchmon/Chart.yaml @@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/PatchMon/PatchMon/main/frontend/public/a type: application -version: 8.0.0+up2.1.3 +version: 8.1.0+up2.1.3 appVersion: "2.1.3" maintainers: diff --git a/patchmon/templates/patchmon.networkpolicy.yaml b/patchmon/templates/patchmon.networkpolicy.yaml index bcef2e2..b99976c 100644 --- a/patchmon/templates/patchmon.networkpolicy.yaml +++ b/patchmon/templates/patchmon.networkpolicy.yaml @@ -45,6 +45,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := .Values.ingress.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -80,6 +99,37 @@ spec: - ports: - protocol: TCP port: 3000 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -122,6 +172,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/patchmon/values.schema.json b/patchmon/values.schema.json index 233df81..62dc4ea 100644 --- a/patchmon/values.schema.json +++ b/patchmon/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/patchmon/values.yaml b/patchmon/values.yaml index f471153..ce1f9c9 100644 --- a/patchmon/values.yaml +++ b/patchmon/values.yaml @@ -288,6 +288,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/rallly/Chart.yaml b/rallly/Chart.yaml index fb94316..a63543f 100644 --- a/rallly/Chart.yaml +++ b/rallly/Chart.yaml @@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/lukevella/rallly/main/apps/landing/publi type: application -version: 9.0.1+up4.15.5 +version: 9.1.0+up4.15.5 appVersion: "4.15.5" maintainers: diff --git a/rallly/templates/rallly.networkpolicy.yaml b/rallly/templates/rallly.networkpolicy.yaml index 1ac8d3e..a6380dd 100644 --- a/rallly/templates/rallly.networkpolicy.yaml +++ b/rallly/templates/rallly.networkpolicy.yaml @@ -45,6 +45,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := .Values.ingress.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -79,6 +98,37 @@ spec: - ports: - protocol: TCP port: 3000 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -121,6 +171,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/rallly/values.schema.json b/rallly/values.schema.json index 0038530..c2e7b35 100644 --- a/rallly/values.schema.json +++ b/rallly/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/rallly/values.yaml b/rallly/values.yaml index ea1ae68..af43d5f 100644 --- a/rallly/values.yaml +++ b/rallly/values.yaml @@ -252,6 +252,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/rss-bridge/Chart.yaml b/rss-bridge/Chart.yaml index 65d1d7d..aed1583 100644 --- a/rss-bridge/Chart.yaml +++ b/rss-bridge/Chart.yaml @@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/RSS-Bridge/rss-bridge/master/static/logo type: application -version: 3.0.0+up2026.09.30 +version: 3.1.0+up2026.09.30 appVersion: "2026-09-30" maintainers: diff --git a/rss-bridge/templates/rss-bridge.networkpolicy.yaml b/rss-bridge/templates/rss-bridge.networkpolicy.yaml index 0c0b988..acc38a2 100644 --- a/rss-bridge/templates/rss-bridge.networkpolicy.yaml +++ b/rss-bridge/templates/rss-bridge.networkpolicy.yaml @@ -48,6 +48,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -85,6 +104,37 @@ spec: - ports: - protocol: TCP port: 80 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -127,6 +177,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/rss-bridge/values.schema.json b/rss-bridge/values.schema.json index 1be3172..377b4a4 100644 --- a/rss-bridge/values.schema.json +++ b/rss-bridge/values.schema.json @@ -46,6 +46,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/rss-bridge/values.yaml b/rss-bridge/values.yaml index b765e20..4d0b327 100644 --- a/rss-bridge/values.yaml +++ b/rss-bridge/values.yaml @@ -419,6 +419,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/samba/Chart.yaml b/samba/Chart.yaml index 19f2e28..3080fb5 100644 --- a/samba/Chart.yaml +++ b/samba/Chart.yaml @@ -5,6 +5,5 @@ icon: https://avatars.githubusercontent.com/u/13281359 type: application -version: 8.0.0+up4.23.10 - +version: 8.1.0+up4.23.10 appVersion: 4.23.10 diff --git a/samba/templates/samba.networkpolicy.yaml b/samba/templates/samba.networkpolicy.yaml index 853ee04..6888a35 100644 --- a/samba/templates/samba.networkpolicy.yaml +++ b/samba/templates/samba.networkpolicy.yaml @@ -49,6 +49,19 @@ selects. */ -}} {{- if .Values.networkPolicy.enabled }} {{- $np := .Values.networkPolicy }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same + thing as the top-level .Values.ingress (the Ingress object) - both match + a search for "ingress". */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -90,6 +103,46 @@ spec: port: 445 - protocol: TCP port: 139 + {{- if $ambientEnabled }} + {{- /* networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP is ztunnel's + HBONE tunnel, the port every inbound connection arrives on once the + namespace carries istio.io/dataplane-mode - ambient "does not and will + never bypass Kubernetes NetworkPolicy enforcement", so without it a pod + in the mesh is unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh holds a NAMED exception for exactly this number, gated on exactly + this value, and proves on a second render that it stays the only such + port and that it is absent while the value is false. + + THE PORT HALF ONLY, and that is not an omission. Elsewhere this switch + also adds 169.254.7.127/32 as a probe source, because inside the mesh the + kubelet's probes are redirected away from the worker node's address. + This chart offers no networkPolicy.probeSources at all, so the rule + carrying the probe port is never restricted by source - and an + unrestricted rule needs no probe address. Measured in the render: three + tcpSocket probes, so these ARE network probes - they are simply never + restricted here, and 445 is additionally published as a NodePort. + WHICH RULE IT JOINS: + the only rule this policy has, the one carrying 445 and 139. + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} egress: {{- /* DNS, unconditionally. There is no value to switch this off, and that is diff --git a/samba/values.schema.json b/samba/values.schema.json index 633eb17..c080de6 100644 --- a/samba/values.schema.json +++ b/samba/values.schema.json @@ -17,6 +17,30 @@ "enabled": { "type": "boolean" }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/samba/values.yaml b/samba/values.yaml index dd0ca82..905679a 100644 --- a/samba/values.yaml +++ b/samba/values.yaml @@ -128,6 +128,71 @@ networkPolicy: # broadcasts on 137/138 UDP are not published by this chart's Service at all, # and clients reach the share by address rather than by browse list. enabled: true + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # It renders 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod whose + # namespace has joined the mesh is unreachable. + # + # ONE HALF SHORTER THAN ELSEWHERE, deliberately. In other charts this switch + # also adds 169.254.7.127/32 as a probe source, because inside the mesh the + # kubelet's probes are redirected away from the worker node's address. This + # chart has no networkPolicy.probeSources at all, so the rule carrying the + # probe port is never restricted by source - and an unrestricted rule needs + # no probe address. An ipBlock for one would match nothing while making the + # rule look like it did something, which is the kind of strict-looking empty + # rule ci/run-networkpolicy-test.sh exists to catch; that script ties the + # exemption to the absence of probeSources rather than to a chart name. + # + # NOT PROVEN: that 15008 alone is enough comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens, and no + # namespace carries istio.io/dataplane-mode yet. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/satisfactory/Chart.yaml b/satisfactory/Chart.yaml index d30ef8f..c6912e9 100644 --- a/satisfactory/Chart.yaml +++ b/satisfactory/Chart.yaml @@ -3,7 +3,7 @@ name: satisfactory description: A Helm chart to deploy a Satisfactory Gameserver on Kubernetes icon: https://raw.githubusercontent.com/wolveix/satisfactory-server/main/.github/logo.png type: application -version: 8.0.0+upv1.9.10 +version: 8.1.0+upv1.9.10 appVersion: "v1.9.10" maintainers: - name: jklein \ No newline at end of file diff --git a/satisfactory/templates/satisfactory.networkpolicy.yaml b/satisfactory/templates/satisfactory.networkpolicy.yaml index 4278d78..6b9c09d 100644 --- a/satisfactory/templates/satisfactory.networkpolicy.yaml +++ b/satisfactory/templates/satisfactory.networkpolicy.yaml @@ -57,6 +57,19 @@ every port here against the containerPorts of the pod spec this policy selects. */ -}} {{- if .Values.networkPolicy.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same + thing as the top-level .Values.ingress (the Ingress object) - both match + a search for "ingress". */}} +{{- $npIngress := .Values.networkPolicy.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -98,4 +111,45 @@ spec: port: {{ include "satisfactory.formatNumber" .Values.server.gamePort }} - protocol: TCP port: {{ include "satisfactory.formatNumber" .Values.server.messagingPort }} + {{- if $ambientEnabled }} + {{- /* networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP is ztunnel's + HBONE tunnel, the port every inbound connection arrives on once the + namespace carries istio.io/dataplane-mode - ambient "does not and will + never bypass Kubernetes NetworkPolicy enforcement", so without it a pod + in the mesh is unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh holds a NAMED exception for exactly this number, gated on exactly + this value, and proves on a second render that it stays the only such + port and that it is absent while the value is false. + + THE PORT HALF ONLY, and that is not an omission. Elsewhere this switch + also adds 169.254.7.127/32 as a probe source, because inside the mesh the + kubelet's probes are redirected away from the worker node's address. + This chart offers no networkPolicy.probeSources at all, so the rule + carrying the probe port is never restricted by source - and an + unrestricted rule needs no probe address. The chart's networkPolicy + block has nothing but enabled. Measured in the render: two exec probes + and one tcpSocket. + WHICH RULE IT JOINS: + the only rule this policy has, the one carrying the game and messaging + ports. + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- end }} diff --git a/satisfactory/values.schema.json b/satisfactory/values.schema.json index c34ae43..daa5e9b 100644 --- a/satisfactory/values.schema.json +++ b/satisfactory/values.schema.json @@ -14,6 +14,30 @@ "type": "object", "additionalProperties": false, "properties": { + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + } + } + } + } + }, "enabled": { "type": "boolean" } diff --git a/satisfactory/values.yaml b/satisfactory/values.yaml index b056b27..ceaee15 100644 --- a/satisfactory/values.yaml +++ b/satisfactory/values.yaml @@ -185,3 +185,68 @@ networkPolicy: # as both - a TCP-only rule would be accepted and would drop half the game # traffic. enabled: true + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # It renders 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod whose + # namespace has joined the mesh is unreachable. + # + # ONE HALF SHORTER THAN ELSEWHERE, deliberately. In other charts this switch + # also adds 169.254.7.127/32 as a probe source, because inside the mesh the + # kubelet's probes are redirected away from the worker node's address. This + # chart has no networkPolicy.probeSources at all, so the rule carrying the + # probe port is never restricted by source - and an unrestricted rule needs + # no probe address. An ipBlock for one would match nothing while making the + # rule look like it did something, which is the kind of strict-looking empty + # rule ci/run-networkpolicy-test.sh exists to catch; that script ties the + # exemption to the absence of probeSources rather than to a chart name. + # + # NOT PROVEN: that 15008 alone is enough comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens, and no + # namespace carries istio.io/dataplane-mode yet. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + diff --git a/searxng/Chart.yaml b/searxng/Chart.yaml index e7110d8..5725637 100644 --- a/searxng/Chart.yaml +++ b/searxng/Chart.yaml @@ -9,7 +9,7 @@ type: application # (2026.8.22-9fea41204) and "latest" moves daily. appVersion is therefore the # exact tag, and the +up suffix normalizes the dash to a dot as the versioning # scheme requires - the same shape rss-bridge and minecraft-server use. -version: 2.0.0+up2026.10.13.22c64b8d3 +version: 2.1.0+up2026.10.13.22c64b8d3 appVersion: "2026.10.13-22c64b8d3" maintainers: diff --git a/searxng/templates/searxng.networkpolicy.yaml b/searxng/templates/searxng.networkpolicy.yaml index f44b1ab..d5af784 100644 --- a/searxng/templates/searxng.networkpolicy.yaml +++ b/searxng/templates/searxng.networkpolicy.yaml @@ -46,6 +46,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -80,6 +99,37 @@ spec: - ports: - protocol: TCP port: 8080 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -122,6 +172,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/searxng/values.schema.json b/searxng/values.schema.json index 0182a86..755bcad 100644 --- a/searxng/values.schema.json +++ b/searxng/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/searxng/values.yaml b/searxng/values.yaml index 6ea00f4..a9b6e25 100644 --- a/searxng/values.yaml +++ b/searxng/values.yaml @@ -465,6 +465,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/stalwart/Chart.yaml b/stalwart/Chart.yaml index 2275b6d..a92fb7e 100644 --- a/stalwart/Chart.yaml +++ b/stalwart/Chart.yaml @@ -37,7 +37,7 @@ type: application # # Note for whoever bumps this next: the suffix tracks appVersion, so a major # here keeps the +up it was merged with. -version: 3.0.0+upv0.16.26 +version: 3.1.0+upv0.16.26 appVersion: "v0.16.26" maintainers: diff --git a/stalwart/templates/stalwart.networkpolicy.yaml b/stalwart/templates/stalwart.networkpolicy.yaml index 464f71e..2368ffb 100644 --- a/stalwart/templates/stalwart.networkpolicy.yaml +++ b/stalwart/templates/stalwart.networkpolicy.yaml @@ -61,6 +61,25 @@ unenforced policy is accepted and filters nothing. {{- $probeSources := .Values.networkPolicy.probeSources | default list -}} {{- $allowedNs := .Values.networkPolicy.allowedNamespaces | default list -}} {{- $ic := .Values.networkPolicy.ingressController | default dict -}} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := .Values.networkPolicy.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} {{- range $group, $spec := fromYaml (include "stalwart.nodes" .) }} {{- $name := printf "%s-%s" (include "stalwart.fullname" $root) $group }} --- @@ -105,6 +124,37 @@ spec: - ports: - protocol: {{ $http.protocol | default "TCP" }} port: {{ int $http.containerPort }} + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if or $probeSources $allowedNs }} {{- /* Restricted because networkPolicy.probeSources or .allowedNamespaces is @@ -147,6 +197,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/stalwart/values.schema.json b/stalwart/values.schema.json index 6a51942..b025dc2 100644 --- a/stalwart/values.schema.json +++ b/stalwart/values.schema.json @@ -334,6 +334,37 @@ "minLength": 1 } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "description": "Egress is restricted by PORT, not by destination: a NetworkPolicy matches CIDRs and selectors and never hostnames, so no host value from this file can appear in it.", "type": "object", diff --git a/stalwart/values.yaml b/stalwart/values.yaml index f95a308..2e38d46 100644 --- a/stalwart/values.yaml +++ b/stalwart/values.yaml @@ -541,6 +541,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + # Egress is restricted BY PORT, not by destination, and the reason is a limit # of the object rather than laziness: a NetworkPolicy matches ipBlock CIDRs # and pod/namespace selectors, never hostnames — so dataStore.postgres.host diff --git a/sure/Chart.yaml b/sure/Chart.yaml index d4d1a76..d1b4e6f 100644 --- a/sure/Chart.yaml +++ b/sure/Chart.yaml @@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/we-promise/sure/main/public/android-chro type: application -version: 2.0.0+up0.7.4 +version: 2.1.0+up0.7.4 appVersion: "0.7.4" maintainers: diff --git a/sure/templates/sure.networkpolicy.yaml b/sure/templates/sure.networkpolicy.yaml index b3a2d2b..3eab307 100644 --- a/sure/templates/sure.networkpolicy.yaml +++ b/sure/templates/sure.networkpolicy.yaml @@ -52,6 +52,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := .Values.ingress.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -86,6 +105,37 @@ spec: - ports: - protocol: TCP port: 3000 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -128,6 +178,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/sure/values.schema.json b/sure/values.schema.json index 81041bb..672c02e 100644 --- a/sure/values.schema.json +++ b/sure/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/sure/values.yaml b/sure/values.yaml index fe05e49..48cec07 100644 --- a/sure/values.yaml +++ b/sure/values.yaml @@ -443,6 +443,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/teamspeak-server/Chart.yaml b/teamspeak-server/Chart.yaml index 1ec1d89..c6b98bd 100644 --- a/teamspeak-server/Chart.yaml +++ b/teamspeak-server/Chart.yaml @@ -5,5 +5,5 @@ icon: https://avatars.githubusercontent.com/u/136759148 type: application -version: 6.0.0+up6.0.0.beta13.1 +version: 6.1.0+up6.0.0.beta13.1 appVersion: "6.0.0-beta13.1" diff --git a/teamspeak-server/templates/teamspeak-server.networkpolicy.yaml b/teamspeak-server/templates/teamspeak-server.networkpolicy.yaml index 771fb16..f292e5f 100644 --- a/teamspeak-server/templates/teamspeak-server.networkpolicy.yaml +++ b/teamspeak-server/templates/teamspeak-server.networkpolicy.yaml @@ -50,6 +50,19 @@ selects. {{- $np := .Values.networkPolicy }} {{- $ns := .Release.Namespace }} {{- $allowedNs := $np.allowedNamespaces | default list }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same + thing as the top-level .Values.ingress (the Ingress object) - both match + a search for "ingress". */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -96,6 +109,50 @@ spec: - ports: - protocol: TCP port: {{ .Values.server.fileTransferPort | int64 }} + {{- if $ambientEnabled }} + {{- /* networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP is ztunnel's + HBONE tunnel, the port every inbound connection arrives on once the + namespace carries istio.io/dataplane-mode - ambient "does not and will + never bypass Kubernetes NetworkPolicy enforcement", so without it a pod + in the mesh is unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh holds a NAMED exception for exactly this number, gated on exactly + this value, and proves on a second render that it stays the only such + port and that it is absent while the value is false. + + THE PORT HALF ONLY, and that is not an omission. Elsewhere this switch + also adds 169.254.7.127/32 as a probe source, because inside the mesh the + kubelet's probes are redirected away from the worker node's address. + This chart offers no networkPolicy.probeSources at all, so the rule + carrying the probe port is never restricted by source - and an + unrestricted rule needs no probe address. Measured in the render: three + tcpSocket probes, so these ARE network probes - they are simply never + restricted here. + WHICH RULE IT JOINS: + the file-transfer rule, the policy's open TCP rule. The voice rule is + UDP and the rule on 10080 is restricted to the namespace. A policy is an + OR over its rules, so which rule carries the port decides only which + sources reach it - and an open TCP rule is the honest default while it + is unknown where ztunnel dials from. + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- /* THE WEB QUERY INTERFACE, and the one rule in this chart with real security value. It has no Service and no probe, so it can be restricted, and it is: this diff --git a/teamspeak-server/values.schema.json b/teamspeak-server/values.schema.json index 5db644f..7f4b471 100644 --- a/teamspeak-server/values.schema.json +++ b/teamspeak-server/values.schema.json @@ -24,6 +24,30 @@ "type": "string" } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/teamspeak-server/values.yaml b/teamspeak-server/values.yaml index 13d6fb1..a2d6fea 100644 --- a/teamspeak-server/values.yaml +++ b/teamspeak-server/values.yaml @@ -132,6 +132,71 @@ networkPolicy: # matches nothing and is silently ineffective. Note the granularity: an entry # opens the port to EVERY pod in that namespace. allowedNamespaces: [] + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # It renders 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod whose + # namespace has joined the mesh is unreachable. + # + # ONE HALF SHORTER THAN ELSEWHERE, deliberately. In other charts this switch + # also adds 169.254.7.127/32 as a probe source, because inside the mesh the + # kubelet's probes are redirected away from the worker node's address. This + # chart has no networkPolicy.probeSources at all, so the rule carrying the + # probe port is never restricted by source - and an unrestricted rule needs + # no probe address. An ipBlock for one would match nothing while making the + # rule look like it did something, which is the kind of strict-looking empty + # rule ci/run-networkpolicy-test.sh exists to catch; that script ties the + # exemption to the absence of probeSources rather than to a chart name. + # + # NOT PROVEN: that 15008 alone is enough comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens, and no + # namespace carries istio.io/dataplane-mode yet. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/template-chart/templates/xxx.networkpolicy.yaml b/template-chart/templates/xxx.networkpolicy.yaml index 023f1fd..80843d2 100644 --- a/template-chart/templates/xxx.networkpolicy.yaml +++ b/template-chart/templates/xxx.networkpolicy.yaml @@ -96,6 +96,40 @@ spec this policy selects. {{- $probeSources := $np.probeSources | default list }} {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} +{{- /* networkPolicy.ingress - the policy's INGRESS DIRECTION, and not to be + confused with networkPolicy.ingressController one line up, nor with the + chart's top-level .Values.ingress. Three different things whose names + overlap as substrings: + + .Values.ingress the Ingress OBJECT this chart renders + .Values.networkPolicy.ingress switches on the policy's inbound side + .Values.networkPolicy.ingressController + the SOURCE SELECTOR for the ingress + controller's pods, a sibling of + networkPolicy.ingress and NO PART OF + IT + + Named here because a search for "ingress" matches all three, and one + round of this work already reported ingressController as an inbound + escape hatch on the strength of the substring alone. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE, and not as decoration (#99). ambientMesh is an + OBJECT, so every read below goes through a fallback that reproduces THIS + chart's declared defaults - because an operator who deletes the object, + or who writes only "enabled: true" and leaves the rest out, must land on + the defaults and not in a nil dereference. #136 shipped exactly that + defect one value away from here: the NOTES.txt branch dereferenced + .networkPolicy.egress directly, and a release that had dropped the key + failed to render. + + Note which fallback the probe-source list gets: "default" treats an EMPTY + list as unset, so clearing it restores the constant rather than removing + the ipBlock. That is deliberate - the two halves of this switch are not + separable, and the way to not open the port at all is enabled: false. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy @@ -129,6 +163,45 @@ spec: - protocol: {{ .protocol | default "TCP" }} port: {{ .containerPort }} {{- end }} + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh. The default is + 15008/TCP: ztunnel's HBONE tunnel, the port every inbound mesh + connection arrives on once the namespace carries + istio.io/dataplane-mode. Istio's own documentation is explicit + that ambient "does not and will never bypass Kubernetes + NetworkPolicy enforcement", so without this port a pod in the + mesh is simply unreachable. + + 15008 IS AN ISTIO CONSTANT, NOT A STANDARD. It is not registered + with IANA for this, Kubernetes knows nothing about it, and no + protocol prescribes it - Istio picked it for HBONE and Istio can + change it. That is precisely why it sits in a VALUE with a + documented default rather than as a literal here: an operator can + correct it without waiting for a chart release. + + It is the one port in this rule that is NOT a containerPort of + the selected pod spec - ztunnel terminates it, not the + application - so ci/run-networkpolicy-test.sh holds a NAMED + exception for it. Note what that exception is pinned to: the + number 15008 as written in THE TEST, plus an assertion that this + chart's values.yaml declares the same default. It is NOT "whatever + this value happens to say" - that would let a mistyped port ride + in through a configurable field, which is the hole design A of + #143 was rejected for. + + READ FOR ztunnel 1.30.6. If Istio changes the port, or the mesh is + replaced: the failure is a pod that cannot be reached inside the + mesh. Change the default in all 30 charts AND the pinned number in + the test - which is the thing that forces someone to look. + + NOT MEASURED, and the chart should not pretend otherwise: that + 15008 ALONE suffices comes from Istio's documentation. What has + been measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace that joins the mesh, and + no namespace carries the label yet. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes come from the IP of the WORKER NODE running the @@ -168,6 +241,65 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason + the switch is one value instead of two. Inside ISTIO's ambient + mesh the kubelet's probes are redirected BY ISTIO and no longer + arrive from the worker node's address: they arrive from + 169.254.7.127, which is therefore the probe source nobody would + think to add. + + WHERE THAT ADDRESS COMES FROM, because it is worth being able to + recognise as vendor magic rather than as protocol: + + - It is LINK-LOCAL, 169.254.0.0/16 (RFC 3927) - a range + reserved for node-local use that is never routed. Exactly + the range one picks a vendor's magic address out of. + - Istio's IPv6 counterpart gives the game away: + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME + DIGITS - 9254:7127 is 169.254.7.127 - plus 1337. Numbers + built that way are vendor constants, not protocols. + - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the + NetworkPolicy API nor the kubelet has any notion of it; only + Istio's own data plane rewrites the probe source onto it. + + READ FOR ztunnel 1.30.6. If Istio changes it, or the mesh is + replaced, THE FAILURE IS SILENT: probes would be dropped, and + only where probeSources is non-empty - which is nowhere today. + Nothing in this repository or in Kubernetes would report it. That + is why the address is a VALUE with a documented default, so it can + be corrected without a chart release, and why the test pins the + number it expects: changing it means touching 30 values files and + the test, and the test is what makes someone look. + + A LIST, not a string, and this is the one case where that does not + contradict "the value is a constant, not a configuration": on a + DUAL-STACK cluster BOTH entries are needed at the same time, not + one of two. This cluster is SingleStack IPv4, so the IPv6 + counterpart above is left out of the default - an ipBlock of the + wrong family is accepted by the API server and matches nothing. + Add it alongside, never instead, if the cluster gains IPv6. + + The two halves stay inseparable. An operator who only opens the + port by hand has a policy that passes every review - and the + moment probeSources is non-empty, every probe is dropped, the pod + never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh, re-created one value further along. + + Only rendered in this branch, because this is the only branch + with a "from" list at all: with probeSources empty the rule is + open and needs no peer. + + UNPROVEN IN OPERATION: probeSources is empty in every chart of + this repository (stalwart sets it to the empty list explicitly), + so no release reaches this branch today. What CI proves is its + render, through the overlay in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the cluster, because this port carries no NodePort. The diff --git a/template-chart/values.schema.json b/template-chart/values.schema.json index fcbcdef..125e4f7 100644 --- a/template-chart/values.schema.json +++ b/template-chart/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/template-chart/values.yaml b/template-chart/values.yaml index 85ef0de..83ed712 100644 --- a/template-chart/values.yaml +++ b/template-chart/values.yaml @@ -272,6 +272,123 @@ networkPolicy: podLabels: app.kubernetes.io/name: rke2-traefik + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are not the + # same thing: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the controller's + # pods. A SIBLING of this block and NO PART + # OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh. OFF by default, and additive: with + # it off this chart renders byte for byte as it did before the value existed. + # + # WHAT IT RENDERS, and the two halves are the point: + # + # 1. The HBONE port inbound (default 15008/TCP) - ztunnel's tunnel. Every + # inbound mesh connection arrives on it, and Istio's documentation is + # explicit that ambient "does not and will never bypass Kubernetes + # NetworkPolicy enforcement". Without this port a pod whose namespace + # has joined the mesh is unreachable. + # 2. The redirected probe source (default 169.254.7.127/32) - but ONLY + # where networkPolicy.probeSources is non-empty, because only then does + # the rule have a "from" list at all. Inside the mesh the kubelet's + # probes are redirected and no longer come from the worker node's + # address. + # + # WHY ONE SWITCH AND NOT TWO. Point 2 is the one nobody thinks of. An + # operator who adds "15008" to a port list by hand writes a policy that + # passes review, and the first time probeSources is narrowed every probe is + # dropped: the pod never becomes Ready and the symptom reads as a broken + # application rather than as a policy. A switch whose halves can only be had + # together cannot be set halfway - which is why the two live in ONE object + # under one "enabled", and not as separate values that can drift apart. + # + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else in this object is inert while it is false, + # and the chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: it is not + # registered with IANA for this, no protocol prescribes it, and Kubernetes + # knows nothing about it. Read for ztunnel 1.30.6 (measured on this + # cluster: ztunnel opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh simply cannot be reached. Change + # it here in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately the thing that + # forces someone to look, because the exception it grants this port is + # pinned to the number IT knows and not to whatever this value says. A + # value-driven exception would let a mistyped port through, which is the + # hole design A of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh; none carries + # istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM, so it is recognisable as vendor magic + # rather than as protocol: + # + # - It is LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, and this list in all 30 charts + # plus the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and this is the one case where that does not + # contradict "a constant, not a configuration": on a DUAL-STACK cluster + # BOTH entries are needed at the same time, not one of two. This cluster is + # SingleStack IPv4, so the IPv6 counterpart is left out - an ipBlock of the + # wrong family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the + # ipBlock, deliberately: the two halves of this switch are not separable. + # To not open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/urlaubsverwaltung/Chart.yaml b/urlaubsverwaltung/Chart.yaml index 2470ec7..b12d099 100644 --- a/urlaubsverwaltung/Chart.yaml +++ b/urlaubsverwaltung/Chart.yaml @@ -4,5 +4,5 @@ description: A Helm chart to deploy urlaubsverwaltung on Kubernetes icon: https://raw.githubusercontent.com/urlaubsverwaltung/urlaubsverwaltung/main/src/main/resources/static/favicons/apple-touch-icon.png type: application -version: 11.0.0+up6.17.0 +version: 11.1.0+up6.17.0 appVersion: 6.17.0 diff --git a/urlaubsverwaltung/templates/urlaubsverwaltung.networkpolicy.yaml b/urlaubsverwaltung/templates/urlaubsverwaltung.networkpolicy.yaml index 08f7513..dc47031 100644 --- a/urlaubsverwaltung/templates/urlaubsverwaltung.networkpolicy.yaml +++ b/urlaubsverwaltung/templates/urlaubsverwaltung.networkpolicy.yaml @@ -48,6 +48,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := .Values.ingress.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -82,6 +101,37 @@ spec: - ports: - protocol: TCP port: 8080 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -124,6 +174,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/urlaubsverwaltung/values.schema.json b/urlaubsverwaltung/values.schema.json index ed48c3e..63fb905 100644 --- a/urlaubsverwaltung/values.schema.json +++ b/urlaubsverwaltung/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/urlaubsverwaltung/values.yaml b/urlaubsverwaltung/values.yaml index 977fa89..974f519 100644 --- a/urlaubsverwaltung/values.yaml +++ b/urlaubsverwaltung/values.yaml @@ -262,6 +262,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/valkey/Chart.yaml b/valkey/Chart.yaml index a2109b6..df93a62 100644 --- a/valkey/Chart.yaml +++ b/valkey/Chart.yaml @@ -26,7 +26,7 @@ type: application # running image with no change in this repository and nothing reports it. The # remedy if that ever matters is pinDigests for this one image - see the valkey # block in renovate.json5. -version: 2.0.0+up9.1.2.alpine +version: 2.1.0+up9.1.2.alpine appVersion: "9.1.2-alpine" maintainers: diff --git a/valkey/templates/valkey.networkpolicy.yaml b/valkey/templates/valkey.networkpolicy.yaml index e72a2a1..c995c0d 100644 --- a/valkey/templates/valkey.networkpolicy.yaml +++ b/valkey/templates/valkey.networkpolicy.yaml @@ -32,6 +32,19 @@ So this policy is what the rest of the rollout aims at and mostly cannot reach: a port that is genuinely closed to everything but the namespaces named below. */ -}} {{- if .Values.networkPolicy.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same + thing as the top-level .Values.ingress (the Ingress object) - both match + a search for "ingress". */}} +{{- $npIngress := .Values.networkPolicy.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -68,4 +81,52 @@ spec: ports: - protocol: TCP port: 6379 + {{- if $ambientEnabled }} + {{- /* networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP is ztunnel's + HBONE tunnel, the port every inbound connection arrives on once the + namespace carries istio.io/dataplane-mode - ambient "does not and will + never bypass Kubernetes NetworkPolicy enforcement", so without it a pod + in the mesh is unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh holds a NAMED exception for exactly this number, gated on exactly + this value, and proves on a second render that it stays the only such + port and that it is absent while the value is false. + + THE PORT HALF ONLY, and that is not an omission. Elsewhere this switch + also adds 169.254.7.127/32 as a probe source, because inside the mesh the + kubelet's probes are redirected away from the worker node's address. + This chart offers no networkPolicy.probeSources at all, so the rule + carrying the probe port is never restricted by source - and an + unrestricted rule needs no probe address. Measured in the render: three + exec probes, which run inside the container through the kubelet's + runtime and put no packet on the network at all. + WHICH RULE IT JOINS: + the only rule this policy has, the one carrying 6379. That rule is + ALWAYS restricted to the release's own namespace plus allowedNamespaces, + so 15008 is restricted with it - and whether ztunnel is reached by those + peers is the open question of #143. ztunnel runs with its own pod IPs in + istio-system (measured 2026-10-19: no hostNetwork on the DaemonSet, + Istio 1.30.6), so if a peer of its own proves necessary it is + expressible as a namespaceSelector plus podSelector and not as an + unknowable node CIDR. It cannot be settled before a namespace joins the + mesh, and none has. + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- end }} diff --git a/valkey/values.schema.json b/valkey/values.schema.json index 888950b..b173b5b 100644 --- a/valkey/values.schema.json +++ b/valkey/values.schema.json @@ -108,6 +108,30 @@ "type": "object", "additionalProperties": false, "properties": { + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + } + } + } + } + }, "enabled": { "description": "On by default. Only a real boundary where the cluster's CNI enforces NetworkPolicies - an unenforced policy is accepted by the API server and filters nothing.", "type": "boolean" diff --git a/valkey/values.yaml b/valkey/values.yaml index 76896f8..6909b32 100644 --- a/valkey/values.yaml +++ b/valkey/values.yaml @@ -279,3 +279,68 @@ networkPolicy: # no typo protection: a namespace name that does not exist matches nothing and # is silently ineffective. allowedNamespaces: [] + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # It renders 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod whose + # namespace has joined the mesh is unreachable. + # + # ONE HALF SHORTER THAN ELSEWHERE, deliberately. In other charts this switch + # also adds 169.254.7.127/32 as a probe source, because inside the mesh the + # kubelet's probes are redirected away from the worker node's address. This + # chart has no networkPolicy.probeSources at all, so the rule carrying the + # probe port is never restricted by source - and an unrestricted rule needs + # no probe address. An ipBlock for one would match nothing while making the + # rule look like it did something, which is the kind of strict-looking empty + # rule ci/run-networkpolicy-test.sh exists to catch; that script ties the + # exemption to the absence of probeSources rather than to a chart name. + # + # NOT PROVEN: that 15008 alone is enough comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens, and no + # namespace carries istio.io/dataplane-mode yet. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + diff --git a/voucher-vault/Chart.yaml b/voucher-vault/Chart.yaml index c033cee..5913963 100644 --- a/voucher-vault/Chart.yaml +++ b/voucher-vault/Chart.yaml @@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/l4rm4nd/VoucherVault/main/myapp/static/a type: application -version: 7.0.0+up1.30.3 +version: 7.1.0+up1.30.3 appVersion: "1.30.3" maintainers: diff --git a/voucher-vault/templates/voucher-vault.networkpolicy.yaml b/voucher-vault/templates/voucher-vault.networkpolicy.yaml index 51d98e7..7977c94 100644 --- a/voucher-vault/templates/voucher-vault.networkpolicy.yaml +++ b/voucher-vault/templates/voucher-vault.networkpolicy.yaml @@ -41,6 +41,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := .Values.ingress.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -75,6 +94,37 @@ spec: - ports: - protocol: TCP port: 8000 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -117,6 +167,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/voucher-vault/values.schema.json b/voucher-vault/values.schema.json index cd9f3cd..892d1b7 100644 --- a/voucher-vault/values.schema.json +++ b/voucher-vault/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/voucher-vault/values.yaml b/voucher-vault/values.yaml index c6eb3b7..2acc305 100644 --- a/voucher-vault/values.yaml +++ b/voucher-vault/values.yaml @@ -338,6 +338,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/ytdlp-web-player/Chart.yaml b/ytdlp-web-player/Chart.yaml index c1ee8c6..2ef22b4 100644 --- a/ytdlp-web-player/Chart.yaml +++ b/ytdlp-web-player/Chart.yaml @@ -5,7 +5,7 @@ icon: https://raw.githubusercontent.com/matszwe02/ytdlp_web_player/main/src/stat type: application -version: 2.0.0+upv1.2.0 +version: 2.1.0+upv1.2.0 appVersion: "v1.2.0" maintainers: diff --git a/ytdlp-web-player/templates/ytdlp-web-player.networkpolicy.yaml b/ytdlp-web-player/templates/ytdlp-web-player.networkpolicy.yaml index db24ced..2f7d5bc 100644 --- a/ytdlp-web-player/templates/ytdlp-web-player.networkpolicy.yaml +++ b/ytdlp-web-player/templates/ytdlp-web-player.networkpolicy.yaml @@ -51,6 +51,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -86,6 +105,37 @@ spec: - ports: - protocol: TCP port: 5000 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -129,6 +179,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/ytdlp-web-player/values.schema.json b/ytdlp-web-player/values.schema.json index 57667d1..d4a0b71 100644 --- a/ytdlp-web-player/values.schema.json +++ b/ytdlp-web-player/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/ytdlp-web-player/values.yaml b/ytdlp-web-player/values.yaml index 21a1bc6..1b0a497 100644 --- a/ytdlp-web-player/values.yaml +++ b/ytdlp-web-player/values.yaml @@ -524,6 +524,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup diff --git a/zipline/Chart.yaml b/zipline/Chart.yaml index 77bbb18..d80c6ed 100644 --- a/zipline/Chart.yaml +++ b/zipline/Chart.yaml @@ -16,8 +16,7 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. # Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 8.0.0+up4.7.0 - +version: 8.1.0+up4.7.0 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. Versions are not expected to # follow Semantic Versioning. They should reflect the version the application is using. diff --git a/zipline/templates/zipline.networkpolicy.yaml b/zipline/templates/zipline.networkpolicy.yaml index 315b315..e6d8788 100644 --- a/zipline/templates/zipline.networkpolicy.yaml +++ b/zipline/templates/zipline.networkpolicy.yaml @@ -42,6 +42,25 @@ selects. {{- $allowedNs := $np.allowedNamespaces | default list }} {{- $ic := $np.ingressController | default dict }} {{- $routed := .Values.ingress.enabled }} +{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing + as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress + controller's pods, a SIBLING of it and no part of it) nor as the + top-level .Values.ingress (the Ingress object). All three match a search + for "ingress"; one round of this work already mistook the second for an + inbound escape hatch on the strength of the substring. */}} +{{- $npIngress := $np.ingress | default dict }} +{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes + through a fallback reproducing THIS chart's declared defaults. An + operator who deletes the object, or writes only "enabled: true", must + land on the defaults and not in a nil dereference - #136 shipped exactly + that defect one value away, where the NOTES.txt branch dereferenced + .networkPolicy.egress directly. Clearing the probe-source list restores + the constant rather than removing the ipBlock, deliberately: the two + halves are not separable, and enabled: false is how you open nothing. */}} +{{- $ambient := $npIngress.ambientMesh | default dict }} +{{- $ambientEnabled := $ambient.enabled }} +{{- $ambientPort := $ambient.port | default 15008 }} +{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: @@ -77,6 +96,37 @@ spec: - ports: - protocol: TCP port: 3000 + {{- if $ambientEnabled }} + {{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP + is ztunnel's HBONE tunnel, the port every inbound connection arrives on + once the namespace carries istio.io/dataplane-mode - ambient "does not + and will never bypass Kubernetes NetworkPolicy enforcement", so without + it a pod in the mesh is simply unreachable. + + It is the one port here that is NOT a containerPort of the selected pod + spec: ztunnel terminates it, not the application. ci/run-networkpolicy- + test.sh therefore holds a NAMED exception for exactly this number, gated + on exactly this value - and assertion 7 of that script proves, on a + second render, that 15008 stays the ONLY such port and that it is absent + while the value is false. + + AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with + IANA for this, no protocol prescribes it, and Kubernetes knows + nothing about it - Istio picked it for HBONE and Istio can change + it. Hence a VALUE with a documented default rather than a literal + here, so an operator can correct it without a chart release. Read + for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh + grants is pinned to the number THAT TEST names, plus a check that + this chart declares the same default - never to whatever the value + says, because that would let a mistyped port ride in through a + configurable field (design A of #143). + + NOT MEASURED: that 15008 alone suffices comes from Istio's + documentation. Measured on this cluster is only which listeners ztunnel + opens. Provable at the first namespace to join the mesh; none has. */}} + - protocol: TCP + port: {{ int $ambientPort }} + {{- end }} {{- if $probeSources }} {{- /* RESTRICTED, because the operator supplied probeSources. The kubelet's probes @@ -119,6 +169,48 @@ spec: - ipBlock: cidr: {{ . | quote }} {{- end }} + {{- if $ambientEnabled }} + {{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is + one value and not two. Inside the mesh the kubelet's probes are + redirected and no longer arrive from the worker node's address: they + arrive from 169.254.7.127. + + An operator who only opens 15008 by hand writes a policy that passes + review - and the moment probeSources is non-empty every probe is + dropped, the pod never becomes Ready, and the symptom reads as a broken + application rather than as a policy. That is assertion 4 of + ci/run-networkpolicy-test.sh one value further along. A switch whose + halves can only be had together cannot be set halfway. + + Only in this branch, because this is the only branch with a "from" list: + with no sources restricted the rule is open and needs no peer. The IPv6 + counterpart is deliberately absent - this cluster is SingleStack IPv4, + and an ipBlock of the wrong family is accepted and matches nothing. + + WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for + node-local use and never routed; Istio's IPv6 counterpart + fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the + NetworkPolicy API nor the kubelet - only Istio's data plane + rewrites the probe source onto it. Read for ztunnel 1.30.6, and if + Istio changes it THE FAILURE IS SILENT: probes dropped, and only + where probeSources is non-empty, which is nowhere today. + + A LIST, not a string, and the one case where that does not + contradict "a constant, not a configuration": a DUAL-STACK cluster + needs BOTH entries at once. This cluster is SingleStack IPv4, so + the IPv6 counterpart is left out - an ipBlock of the wrong family + is accepted and matches nothing. Add it ALONGSIDE, never instead. + + UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + release reaches this branch. CI proves its render, through the overlays + in ci/networkpolicy. */}} + {{- range $ambientProbeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- end }} {{- else }} {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the diff --git a/zipline/values.schema.json b/zipline/values.schema.json index 3ab4551..c272a0b 100644 --- a/zipline/values.schema.json +++ b/zipline/values.schema.json @@ -45,6 +45,37 @@ } } }, + "ingress": { + "description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "ambientMesh": { + "description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "enabled": { + "description": "The switch. Everything else in this object is inert while it is false.", + "type": ["boolean", "null"] + }, + "port": { + "description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.", + "type": ["integer", "null"], + "minimum": 1, + "maximum": 65535 + }, + "redirectedProbeSources": { + "description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.", + "type": ["array", "null"], + "items": { + "type": "string" + } + } + } + } + } + }, "egress": { "type": ["object", "null"], "additionalProperties": false, diff --git a/zipline/values.yaml b/zipline/values.yaml index 28b97a8..497e7dd 100644 --- a/zipline/values.yaml +++ b/zipline/values.yaml @@ -232,6 +232,135 @@ networkPolicy: namespace: kube-system podLabels: app.kubernetes.io/name: rke2-traefik + + # The policy's INBOUND side. Mind the three names that overlap as substrings, + # because a search for "ingress" matches all of them and they are different + # things: + # + # ingress (top level) the Ingress OBJECT this chart + # renders - host, TLS, className + # networkPolicy.ingress this block: what the policy lets IN + # networkPolicy.ingressController the SOURCE SELECTOR for the ingress + # controller's pods. A SIBLING of this block + # and NO PART OF IT, despite the name. + ingress: + # Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and + # additive: while it is false this chart renders byte for byte as it did + # before the value existed. + # + # IT RENDERS TWO THINGS, and that they come together is the point: + # + # 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh + # connection arrives on it, and ambient "does not and will never bypass + # Kubernetes NetworkPolicy enforcement", so without this port a pod + # whose namespace has joined the mesh is unreachable. + # 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule + # restricts sources at all, which here means probeSources is non-empty. + # Inside the mesh the kubelet's probes are redirected and no longer come + # from the worker node's address. + # + # WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever + # adds "15008" to a port list by hand writes a policy that passes review, and + # the first time probeSources is narrowed every probe is dropped - the pod + # never becomes Ready and the symptom reads as a broken application rather + # than as a policy. A switch whose halves can only be had together cannot be + # set halfway, which is why this is a named intention and not a port list. + # + # TWO THINGS IT DOES NOT PROVE: + # - That 15008 alone is enough. That comes from Istio's documentation; + # measured on this cluster is only which listeners ztunnel opens. + # Provable at the first namespace to join the mesh - none carries + # istio.io/dataplane-mode yet. + # - That the probe-source half works. probeSources is empty in every chart + # here, so no release reaches that branch. CI proves its render, not its + # effect. + # + # Note where the port lands: it joins the rule that carries the probe port, + # so once probeSources restricts that rule, 15008 is restricted with it. + # Whether ztunnel needs a peer of its own is open - it runs with its own pod + # IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio + # 1.30.6), so if one proves necessary it is expressible as a + # namespaceSelector plus podSelector and needs no unknowable node CIDR. + # BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values + # rather than template literals for exactly that reason: a vendor constant + # can change, and an operator should be able to correct it without waiting + # for a chart release. Their provenance is written on each one. + # + # DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults + # declared here (#99), so "ambientMesh: null", or an object carrying only + # "enabled: true", renders the documented defaults instead of failing. + ambientMesh: + # The switch. Everything else here is inert while it is false, and the + # chart renders exactly as it did before the object existed. + enabled: false + + # ISTIO'S HBONE PORT. A constant Istio chose, not a standard: not + # registered with IANA for this, prescribed by no protocol, and unknown to + # Kubernetes. Read for ztunnel 1.30.6 (measured on this cluster: ztunnel + # opens [::]:15008 as its inbound listener). + # + # If Istio changes it, a pod in the mesh cannot be reached. Change it here + # in all 30 charts AND in the pinned expectation inside + # ci/run-networkpolicy-test.sh - the test is deliberately what forces + # someone to look, because the exception it grants this port is pinned to + # the number IT knows and not to whatever this value says. A value-driven + # exception would let a mistyped port through, which is the hole design A + # of #143 was rejected for. + # + # NOT MEASURED: that this port ALONE suffices comes from Istio's + # documentation. Measured here is only which listeners ztunnel opens, and + # no namespace carries istio.io/dataplane-mode yet. + port: 15008 + + # SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM. + # + # NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the + # two are easy to confuse, so: + # + # networkPolicy.probeSources + # YOUR worker node CIDRs. Empty by default, and empty means the + # probe rule is OPEN. The chart cannot know these addresses. + # networkPolicy.ingress.ambientMesh.redirectedProbeSources + # ISTIO's rewrite address. Shipped filled, because unlike the node + # IPs it is a constant the chart CAN know. Only rendered where the + # rule above is restricted, i.e. where probeSources is non-empty. + # + # WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol: + # + # - LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for + # node-local use that is never routed - exactly the range one picks a + # vendor's magic address out of. + # - Istio's IPv6 counterpart gives it away: + # fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS + # (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are + # vendor constants, not protocols. + # - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API + # nor the kubelet has any notion of it; only Istio's own data plane + # rewrites the probe source onto it. + # + # Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT: + # probes would be dropped, and only where probeSources is non-empty - which + # is nowhere in this repository today - so nothing here and nothing in + # Kubernetes would report it. What to re-check then: the address Istio's + # release notes name for probe redirection, this list in all 30 charts, and + # the pinned expectation in ci/run-networkpolicy-test.sh. + # + # A LIST, not a string, and the one case where that does not contradict "a + # constant, not a configuration": on a DUAL-STACK cluster BOTH entries are + # needed at the same time, not one of two. This cluster is SingleStack + # IPv4, so the IPv6 counterpart is left out - an ipBlock of the wrong + # family is accepted by the API server and matches nothing. Add it + # ALONGSIDE, never instead, if the cluster gains IPv6. + # + # Clearing this list restores the default rather than removing the ipBlock, + # deliberately: the two halves of this switch are not separable. To not + # open the port at all, set enabled: false. + # + # UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no + # release reaches this branch. CI proves the render, not the effect. + redirectedProbeSources: + - 169.254.7.127/32 + egress: # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and # has no switch - a knob whose only other setting breaks every name lookup