jlab-cluster/helm-charts
3
0 Fork 0
Code Issues 15 Pull Requests 2 Actions Packages Wiki Activity

feat(template-chart)!: ambientMesh becomes an object, and the CI exception gets a pinned expectation

Browse Source 
Gitea #143, maintainer's instruction. Both numbers this switch renders are
ISTIO CONSTANTS rather than standards, and a value with a documented default
is the better home for a vendor constant than a template literal: an operator
can correct it when Istio changes the number, without waiting for a chart
release.

    networkPolicy:
      ingress:
        ambientMesh:
          enabled: false
          port: 15008
          redirectedProbeSources:
            - 169.254.7.127/32

One object under one switch, so the coupling argument survives the change
instead of being undermined by it - two separate values could drift apart,
which is the thing this design exists to prevent. redirectedProbeSources is a
LIST because a dual-stack cluster needs BOTH of Istio's addresses at once, not
one of two; that is a nameable case rather than speculation, so it does not
contradict reason 4 of the issue. Istio's IPv6 counterpart is documented as the
dual-stack addition, with the reason it is absent today unchanged: SingleStack
IPv4, and an ipBlock of the wrong family is accepted and matches nothing.

Named redirectedProbeSources, not probeSources, because the confusion risk with
networkPolicy.probeSources one level up is larger than the ingress/
ingressController one: that value holds YOUR worker node CIDRs, is empty by
default and means OPEN when empty; this one holds ISTIO's rewrite address and
ships filled, because unlike the node IPs it is a constant the chart can know.
The distinction is written where both appear.

PROVENANCE, which the previous round explained the coupling of but never
attributed. 15008 is Istio's HBONE port - not IANA-registered for this, not
prescribed by any protocol, unknown to Kubernetes. 169.254.7.127 is link-local
(169.254.0.0/16, RFC 3927, never routed - the range one picks vendor magic out
of); Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries
the same digits plus 1337, which is what vendor constants look like; and
Kubernetes knows the address in neither the NetworkPolicy API nor the kubelet -
only Istio's data plane rewrites the probe source onto it. Both read for
ztunnel 1.30.6. If either changes, the failure is SILENT: a pod unreachable in
the mesh, or probes dropped where probeSources is non-empty, which is nowhere
today.

NULL-SAFE, per #99 and because #136 shipped this exact defect one value away
(its NOTES.txt branch dereferenced .networkPolicy.egress directly): every read
goes through a fallback reproducing the chart's own declared defaults, so a
deleted object or one carrying only "enabled: true" renders the defaults rather
than failing. Clearing the probe-source list restores the constant instead of
removing the ipBlock, deliberately - the halves are not separable, and
enabled: false is how you open nothing.

THE CONSEQUENCE, and it is the point of the CI half. With the port
configurable, the exception can no longer be checked against the rendered
value: "whatever the value says" is design A's hole wearing design C's name,
and a mistyped port would pass by sitting in the configurable field. So the
expectation is PINNED IN THE TEST - AMBIENT_PORT and AMBIENT_CIDR - and
assertion 7 now checks two things instead of one: that the render at chart
defaults produces exactly that port and that ipBlock, and that every chart's
values.yaml DECLARES the same defaults, read back through ci/yaml2json rather
than by regex. A chart that drifts is red. Changing an Istio constant means
touching 30 values files AND the pinned lines, and the pinned lines are what
force someone to look.

That block is therefore also the single place saying what we hold Istio's
numbers to be, so it carries the provenance too. The renders set only
enabled=true and never a port override, on purpose: the pin is a statement
about the defaults we ship, not about what a release may choose - an operator
overriding a value is outside this test's reach, as for every value here.

The declared-defaults check is conditional on the chart offering probeSources,
like the inseparability check and assertion 4: the five charts whose probe rule
is never source-restricted render no ipBlock, so for them the key must be
ABSENT - a value reaching no rule reads as a control that does nothing - and
that absence is asserted rather than tolerated. Their schemas drop it too.

No version change: nothing of this is published, the switch stays additive with
its default off, and the render at defaults is byte-identical - re-diffed per
chart against origin/main, not assumed.
This commit is contained in:
jklein committed 2026-10-19 17:56:58 +00:00
1 parent f2fda15e57
commit 22977c183f
4 files changed
+358 -98
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

template-chart/templates/xxx.networkpolicy.yaml
+89 -27
View File
@@ -113,7 +113,23 @@ spec this policy selects.
round of this work already reported ingressController as an inbound
escape hatch on the strength of the substring alone. */}}
{{- $npIngress := $np.ingress | default dict }}
{{- $ambientMesh := $npIngress.ambientMesh }}
{{- /* NULL-SAFE ON PURPOSE, and not as decoration (#99). ambientMesh is an
OBJECT, so every read below goes through a fallback that reproduces THIS
chart's declared defaults - because an operator who deletes the object,
or who writes only "enabled: true" and leaves the rest out, must land on
the defaults and not in a nil dereference. #136 shipped exactly that
defect one value away from here: the NOTES.txt branch dereferenced
.networkPolicy.egress directly, and a release that had dropped the key
failed to render.
Note which fallback the probe-source list gets: "default" treats an EMPTY
list as unset, so clearing it restores the constant rather than removing
the ipBlock. That is deliberate - the two halves of this switch are not
separable, and the way to not open the port at all is enabled: false. */}}
{{- $ambient := $npIngress.ambientMesh | default dict }}
{{- $ambientEnabled := $ambient.enabled }}
{{- $ambientPort := $ambient.port | default 15008 }}
{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }}
{{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }}
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
@@ -147,20 +163,36 @@ spec:
- protocol: {{ .protocol | default "TCP" }}
port: {{ .containerPort }}
{{- end }}
{{- if $ambientMesh }}
{{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh. 15008/TCP is
ztunnel's HBONE tunnel, the port every inbound mesh connection
arrives on once the namespace carries istio.io/dataplane-mode.
Istio's own documentation is explicit that ambient "does not and
will never bypass Kubernetes NetworkPolicy enforcement", so
without this port a pod in the mesh is simply unreachable.
{{- if $ambientEnabled }}
{{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh. The default is
15008/TCP: ztunnel's HBONE tunnel, the port every inbound mesh
connection arrives on once the namespace carries
istio.io/dataplane-mode. Istio's own documentation is explicit
that ambient "does not and will never bypass Kubernetes
NetworkPolicy enforcement", so without this port a pod in the
mesh is simply unreachable.
15008 IS AN ISTIO CONSTANT, NOT A STANDARD. It is not registered
with IANA for this, Kubernetes knows nothing about it, and no
protocol prescribes it - Istio picked it for HBONE and Istio can
change it. That is precisely why it sits in a VALUE with a
documented default rather than as a literal here: an operator can
correct it without waiting for a chart release.
It is the one port in this rule that is NOT a containerPort of
the selected pod spec - ztunnel terminates it, not the
application - so ci/run-networkpolicy-test.sh holds a NAMED
exception for exactly this number, gated on exactly this value.
Anything else that is not a containerPort still fails, and so
does 15008 itself while the value is off.
exception for it. Note what that exception is pinned to: the
number 15008 as written in THE TEST, plus an assertion that this
chart's values.yaml declares the same default. It is NOT "whatever
this value happens to say" - that would let a mistyped port ride
in through a configurable field, which is the hole design A of
#143 was rejected for.
READ FOR ztunnel 1.30.6. If Istio changes the port, or the mesh is
replaced: the failure is a pod that cannot be reached inside the
mesh. Change the default in all 30 charts AND the pinned number in
the test - which is the thing that forces someone to look.
NOT MEASURED, and the chart should not pretend otherwise: that
15008 ALONE suffices comes from Istio's documentation. What has
@@ -168,7 +200,7 @@ spec:
opens. Provable at the first namespace that joins the mesh, and
no namespace carries the label yet. */}}
- protocol: TCP
port: 15008
port: {{ int $ambientPort }}
{{- end }}
{{- if $probeSources }}
{{- /* RESTRICTED, because the operator supplied probeSources. The
@@ -209,34 +241,64 @@ spec:
- ipBlock:
cidr: {{ . | quote }}
{{- end }}
{{- if $ambientMesh }}
{{- if $ambientEnabled }}
{{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason
the switch is one value instead of two. Inside the mesh the
kubelet's probes are redirected and no longer arrive from the
worker node's address: they arrive from 169.254.7.127, which is
therefore the probeSources entry nobody would think to add.
the switch is one value instead of two. Inside ISTIO's ambient
mesh the kubelet's probes are redirected BY ISTIO and no longer
arrive from the worker node's address: they arrive from
169.254.7.127, which is therefore the probe source nobody would
think to add.
The two halves are deliberately inseparable. An operator who
only opens 15008 by hand has a policy that passes every review -
and the moment probeSources is non-empty, every probe is dropped,
the pod never becomes Ready, and the symptom reads as a broken
WHERE THAT ADDRESS COMES FROM, because it is worth being able to
recognise as vendor magic rather than as protocol:
- It is LINK-LOCAL, 169.254.0.0/16 (RFC 3927) - a range
reserved for node-local use that is never routed. Exactly
the range one picks a vendor's magic address out of.
- Istio's IPv6 counterpart gives the game away:
fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME
DIGITS - 9254:7127 is 169.254.7.127 - plus 1337. Numbers
built that way are vendor constants, not protocols.
- KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the
NetworkPolicy API nor the kubelet has any notion of it; only
Istio's own data plane rewrites the probe source onto it.
READ FOR ztunnel 1.30.6. If Istio changes it, or the mesh is
replaced, THE FAILURE IS SILENT: probes would be dropped, and
only where probeSources is non-empty - which is nowhere today.
Nothing in this repository or in Kubernetes would report it. That
is why the address is a VALUE with a documented default, so it can
be corrected without a chart release, and why the test pins the
number it expects: changing it means touching 30 values files and
the test, and the test is what makes someone look.
A LIST, not a string, and this is the one case where that does not
contradict "the value is a constant, not a configuration": on a
DUAL-STACK cluster BOTH entries are needed at the same time, not
one of two. This cluster is SingleStack IPv4, so the IPv6
counterpart above is left out of the default - an ipBlock of the
wrong family is accepted by the API server and matches nothing.
Add it alongside, never instead, if the cluster gains IPv6.
The two halves stay inseparable. An operator who only opens the
port by hand has a policy that passes every review - and the
moment probeSources is non-empty, every probe is dropped, the pod
never becomes Ready, and the symptom reads as a broken
application rather than as a policy. That is assertion 4 of
ci/run-networkpolicy-test.sh, re-created one value further along.
One switch cannot be set halfway.
Only rendered in this branch, because this is the only branch
with a "from" list at all: with probeSources empty the rule is
open and needs no peer. The IPv6 counterpart
fd16:9254:7127:1337:ffff:ffff:ffff:ffff is deliberately absent -
this cluster is SingleStack IPv4, and an ipBlock of the wrong
family is accepted by the API server and matches nothing.
open and needs no peer.
UNPROVEN IN OPERATION: probeSources is empty in every chart of
this repository (stalwart sets it to the empty list explicitly),
so no release reaches this branch today. What CI proves is its
render, through the overlay in ci/networkpolicy. */}}
{{- range $ambientProbeSources }}
- ipBlock:
cidr: 169.254.7.127/32
cidr: {{ . | quote }}
{{- end }}
{{- end }}
{{- else }}
{{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only
template-chart/values.schema.json
+22 -2
View File
@@ -51,8 +51,28 @@
"additionalProperties": false,
"properties": {
"ambientMesh": {
"description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while it is false. Renders 15008/TCP inbound - ztunnel's HBONE tunnel, which ambient explicitly does not let bypass NetworkPolicy enforcement - AND, where probeSources is non-empty, 169.254.7.127/32 as an additional probe source, because inside the mesh the kubelet's probes are redirected away from the worker node's address. The two halves are inseparable on purpose: whoever opens 15008 by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Unproven in operation: that 15008 alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.",
"type": ["boolean", "null"]
"description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while enabled is false. Renders the HBONE port inbound AND, where networkPolicy.probeSources is non-empty, the redirected probe source in the same \"from\" list. The two halves live in one object under one switch on purpose: whoever opens the port by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Both numbers are ISTIO CONSTANTS rather than standards, which is why they are values with documented defaults and not template literals - a vendor constant can change, and an operator should be able to correct it without waiting for a chart release. Deleting the whole object is safe: every read falls back to the defaults the chart declares. Unproven in operation: that the HBONE port alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.",
"type": ["object", "null"],
"additionalProperties": false,
"properties": {
"enabled": {
"description": "The switch. Everything else in this object is inert while it is false.",
"type": ["boolean", "null"]
},
"port": {
"description": "Istio's HBONE port, default 15008. A constant Istio chose, not a standard: not registered with IANA for this, prescribed by no protocol, and unknown to Kubernetes. Read for ztunnel 1.30.6. It is the one ingress port of these charts that is deliberately no containerPort - ztunnel terminates it, not the application - and the exception ci/run-networkpolicy-test.sh grants it is pinned to the number THAT TEST knows, plus an assertion that this chart declares the same default. Not to whatever this value says: a value-driven exception would let a mistyped port ride in through a configurable field.",
"type": ["integer", "null"],
"minimum": 1,
"maximum": 65535
},
"redirectedProbeSources": {
"description": "Sources the kubelet's probes arrive from once Istio has redirected them, default [169.254.7.127/32]. NOT the same value as networkPolicy.probeSources one level up: that one holds YOUR worker node CIDRs, is empty by default and means OPEN when empty, and the chart cannot know those addresses; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. Recognisable as vendor magic rather than protocol: it is link-local (169.254.0.0/16, RFC 3927, never routed), Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, and Kubernetes itself knows the address in neither the NetworkPolicy API nor the kubelet. A list and not a string because a dual-stack cluster needs BOTH entries at once; this cluster is SingleStack IPv4, so the IPv6 counterpart is left out of the default - an ipBlock of the wrong family is accepted and matches nothing. If Istio changes the address the failure is SILENT: probes dropped, and only where probeSources is non-empty.",
"type": ["array", "null"],
"items": {
"type": "string"
}
}
}
}
}
},
template-chart/values.yaml
+95 -34
View File
@@ -283,50 +283,111 @@ networkPolicy:
# pods. A SIBLING of this block and NO PART
# OF IT, despite the name.
ingress:
# Opens the pod for Istio's ambient mesh. OFF by default, and additive:
# with it off this chart renders byte for byte as it did before the value
# existed.
# Opens the pod for Istio's ambient mesh. OFF by default, and additive: with
# it off this chart renders byte for byte as it did before the value existed.
#
# WHAT IT RENDERS, and the two halves are the point:
#
# 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh
# connection arrives on it, and Istio's documentation is explicit that
# ambient "does not and will never bypass Kubernetes NetworkPolicy
# enforcement". Without this port a pod whose namespace has joined the
# mesh is unreachable.
# 2. 169.254.7.127/32 as an additional probe source - but ONLY where
# probeSources is non-empty, because only then does the rule have a
# "from" list at all. Inside the mesh the kubelet's probes are
# redirected and no longer come from the worker node's address.
# 1. The HBONE port inbound (default 15008/TCP) - ztunnel's tunnel. Every
# inbound mesh connection arrives on it, and Istio's documentation is
# explicit that ambient "does not and will never bypass Kubernetes
# NetworkPolicy enforcement". Without this port a pod whose namespace
# has joined the mesh is unreachable.
# 2. The redirected probe source (default 169.254.7.127/32) - but ONLY
# where networkPolicy.probeSources is non-empty, because only then does
# the rule have a "from" list at all. Inside the mesh the kubelet's
# probes are redirected and no longer come from the worker node's
# address.
#
# WHY ONE SWITCH AND NOT TWO. Point 2 is the one nobody thinks of. An
# operator who adds "15008" to a port list by hand writes a policy that
# passes review, and the first time probeSources is narrowed every probe is
# dropped: the pod never becomes Ready and the symptom reads as a broken
# application rather than as a policy. A switch whose halves can only be
# had together cannot be set halfway - which is why this is a named
# intention and not a free port list.
# application rather than as a policy. A switch whose halves can only be had
# together cannot be set halfway - which is why the two live in ONE object
# under one "enabled", and not as separate values that can drift apart.
#
# TWO THINGS IT DOES NOT PROVE, and they belong next to the value rather
# than in a commit message:
# BOTH NUMBERS BELOW ARE ISTIO CONSTANTS, NOT STANDARDS, and they are values
# rather than template literals for exactly that reason: a vendor constant
# can change, and an operator should be able to correct it without waiting
# for a chart release. Their provenance is written on each one.
#
# - That 15008 alone is enough. That comes from Istio's documentation.
# Measured on this cluster is only which listeners ztunnel opens
# ([::]:15008 inbound, plus its own admin and stats ports). Provable at
# the first namespace that joins the mesh; none carries
# istio.io/dataplane-mode yet.
# - That the probe-source half works. probeSources is empty in every
# chart here, so no release reaches that branch. CI proves its render,
# not its effect.
#
# If the policy restricts sources, note that 15008 is restricted with them:
# the port joins the rule that carries the probe port, so it is reachable
# from that rule's peers only. Whether ztunnel needs a peer of its own is
# open - it runs with its own pod IPs in istio-system (measured: no
# hostNetwork on the DaemonSet, Istio 1.30.6), so if one turns out to be
# needed it is expressible as a namespaceSelector plus podSelector and does
# not need an unknowable node-IP CIDR.
ambientMesh: false
# DELETING THIS OBJECT IS SAFE. Every read falls back to the defaults
# declared here (#99), so "ambientMesh: null", or an object carrying only
# "enabled: true", renders the documented defaults instead of failing.
ambientMesh:
# The switch. Everything else in this object is inert while it is false,
# and the chart renders exactly as it did before the object existed.
enabled: false
# ISTIO'S HBONE PORT. A constant Istio chose, not a standard: it is not
# registered with IANA for this, no protocol prescribes it, and Kubernetes
# knows nothing about it. Read for ztunnel 1.30.6 (measured on this
# cluster: ztunnel opens [::]:15008 as its inbound listener).
#
# If Istio changes it, a pod in the mesh simply cannot be reached. Change
# it here in all 30 charts AND in the pinned expectation inside
# ci/run-networkpolicy-test.sh - the test is deliberately the thing that
# forces someone to look, because the exception it grants this port is
# pinned to the number IT knows and not to whatever this value says. A
# value-driven exception would let a mistyped port through, which is the
# hole design A of #143 was rejected for.
#
# NOT MEASURED: that this port ALONE suffices comes from Istio's
# documentation. Measured here is only which listeners ztunnel opens.
# Provable at the first namespace to join the mesh; none carries
# istio.io/dataplane-mode yet.
port: 15008
# SOURCES THE KUBELET'S PROBES ARRIVE FROM ONCE ISTIO HAS REDIRECTED THEM.
#
# NOT THE SAME VALUE AS networkPolicy.probeSources one level up, and the
# two are easy to confuse, so:
#
# networkPolicy.probeSources
# YOUR worker node CIDRs. Empty by default, and empty means the
# probe rule is OPEN. The chart cannot know these addresses.
# networkPolicy.ingress.ambientMesh.redirectedProbeSources
# ISTIO's rewrite address. Shipped filled, because unlike the node
# IPs it is a constant the chart CAN know. Only rendered where the
# rule above is restricted, i.e. where probeSources is non-empty.
#
# WHERE 169.254.7.127 COMES FROM, so it is recognisable as vendor magic
# rather than as protocol:
#
# - It is LINK-LOCAL, 169.254.0.0/16 (RFC 3927): a range reserved for
# node-local use that is never routed - exactly the range one picks a
# vendor's magic address out of.
# - Istio's IPv6 counterpart gives it away:
# fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS
# (9254:7127 is 169.254.7.127) plus 1337. Numbers built that way are
# vendor constants, not protocols.
# - KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the NetworkPolicy API
# nor the kubelet has any notion of it; only Istio's own data plane
# rewrites the probe source onto it.
#
# Read for ztunnel 1.30.6. IF ISTIO CHANGES IT, THE FAILURE IS SILENT:
# probes would be dropped, and only where probeSources is non-empty - which
# is nowhere in this repository today - so nothing here and nothing in
# Kubernetes would report it. What to re-check then: the address Istio's
# release notes name for probe redirection, and this list in all 30 charts
# plus the pinned expectation in ci/run-networkpolicy-test.sh.
#
# A LIST, not a string, and this is the one case where that does not
# contradict "a constant, not a configuration": on a DUAL-STACK cluster
# BOTH entries are needed at the same time, not one of two. This cluster is
# SingleStack IPv4, so the IPv6 counterpart is left out - an ipBlock of the
# wrong family is accepted by the API server and matches nothing. Add it
# ALONGSIDE, never instead, if the cluster gains IPv6.
#
# Clearing this list restores the default rather than removing the
# ipBlock, deliberately: the two halves of this switch are not separable.
# To not open the port at all, set enabled: false.
#
# UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no
# release reaches this branch. CI proves the render, not the effect.
redirectedProbeSources:
- 169.254.7.127/32
egress:
# Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and