feat(garage): networkPolicy.ingress.ambientMesh
Gitea #143. Additive, default false, so the render is byte-identical to the previous release - proven by diffing both renders against origin/main rather than asserted. Switched on it adds 15008/TCP, ztunnel's HBONE tunnel, to the rule carrying the probe port, and - inside that rule's source-restricted branch - 169.254.7.127/32, the address the kubelet's probes arrive from once the pod is in the mesh. Ambient does not bypass NetworkPolicy enforcement, so without the port a pod whose namespace has joined the mesh is unreachable. The two halves are one value because the second is the one nobody thinks of: 15008 added by hand leaves a policy that passes review and drops every probe the first time probeSources is narrowed, and a pod that never becomes Ready reads as a broken application rather than as a policy. Minor, not major: additive value, unchanged default, unchanged render.
This commit is contained in:
4 files changed
+115
-1
No files matched your search
+1
-1
@@ -19,7 +19,7 @@ type: application
|
||||
# offer those as "updates", which is why the garage block in renovate.json5
|
||||
# carries a versioningTemplate anchored to ^v<major>.<minor>.<patch>$ - same
|
||||
# shape as the stalwart block, for the same reason.
|
||||
version: 2.0.0+upv2.4.1
|
||||
version: 2.1.0+upv2.4.1
|
||||
appVersion: "v2.4.1"
|
||||
|
||||
maintainers:
|
||||
|
||||
@@ -54,6 +54,14 @@ selects.
|
||||
{{- $allowedNs := $np.allowedNamespaces | default list }}
|
||||
{{- $ic := $np.ingressController | default dict }}
|
||||
{{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }}
|
||||
{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing
|
||||
as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress
|
||||
controller's pods, a SIBLING of it and no part of it) nor as the
|
||||
top-level .Values.ingress (the Ingress object). All three match a search
|
||||
for "ingress"; one round of this work already mistook the second for an
|
||||
inbound escape hatch on the strength of the substring. */}}
|
||||
{{- $npIngress := $np.ingress | default dict }}
|
||||
{{- $ambientMesh := $npIngress.ambientMesh }}
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
@@ -140,6 +148,26 @@ spec:
|
||||
- ports:
|
||||
- protocol: TCP
|
||||
port: {{ $settings.ports.admin | int64 }}
|
||||
{{- if $ambientMesh }}
|
||||
{{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP
|
||||
is ztunnel's HBONE tunnel, the port every inbound connection arrives on
|
||||
once the namespace carries istio.io/dataplane-mode - ambient "does not
|
||||
and will never bypass Kubernetes NetworkPolicy enforcement", so without
|
||||
it a pod in the mesh is simply unreachable.
|
||||
|
||||
It is the one port here that is NOT a containerPort of the selected pod
|
||||
spec: ztunnel terminates it, not the application. ci/run-networkpolicy-
|
||||
test.sh therefore holds a NAMED exception for exactly this number, gated
|
||||
on exactly this value - and assertion 7 of that script proves, on a
|
||||
second render, that 15008 stays the ONLY such port and that it is absent
|
||||
while the value is false.
|
||||
|
||||
NOT MEASURED: that 15008 alone suffices comes from Istio's
|
||||
documentation. Measured on this cluster is only which listeners ztunnel
|
||||
opens. Provable at the first namespace to join the mesh; none has. */}}
|
||||
- protocol: TCP
|
||||
port: 15008
|
||||
{{- end }}
|
||||
{{- if $probeSources }}
|
||||
{{- /*
|
||||
RESTRICTED, because the operator supplied probeSources. The kubelet's probes
|
||||
@@ -167,6 +195,30 @@ spec:
|
||||
- ipBlock:
|
||||
cidr: {{ . | quote }}
|
||||
{{- end }}
|
||||
{{- if $ambientMesh }}
|
||||
{{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is
|
||||
one value and not two. Inside the mesh the kubelet's probes are
|
||||
redirected and no longer arrive from the worker node's address: they
|
||||
arrive from 169.254.7.127.
|
||||
|
||||
An operator who only opens 15008 by hand writes a policy that passes
|
||||
review - and the moment probeSources is non-empty every probe is
|
||||
dropped, the pod never becomes Ready, and the symptom reads as a broken
|
||||
application rather than as a policy. That is assertion 4 of
|
||||
ci/run-networkpolicy-test.sh one value further along. A switch whose
|
||||
halves can only be had together cannot be set halfway.
|
||||
|
||||
Only in this branch, because this is the only branch with a "from" list:
|
||||
with no sources restricted the rule is open and needs no peer. The IPv6
|
||||
counterpart is deliberately absent - this cluster is SingleStack IPv4,
|
||||
and an ipBlock of the wrong family is accepted and matches nothing.
|
||||
|
||||
UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no
|
||||
release reaches this branch. CI proves its render, through the overlays
|
||||
in ci/networkpolicy. */}}
|
||||
- ipBlock:
|
||||
cidr: 169.254.7.127/32
|
||||
{{- end }}
|
||||
{{- else }}
|
||||
{{- /*
|
||||
NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the
|
||||
|
||||
@@ -45,6 +45,17 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"ingress": {
|
||||
"description": "The policy's INBOUND side. Three names here overlap as substrings and are not the same thing: the top-level ingress is the Ingress OBJECT this chart renders, networkPolicy.ingress is this block, and networkPolicy.ingressController is the SOURCE SELECTOR for the ingress controller's pods - a SIBLING of this block and no part of it.",
|
||||
"type": ["object", "null"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"ambientMesh": {
|
||||
"description": "Opens the pod for Istio's ambient mesh; off by default and additive, so the render is byte-identical while it is false. Renders 15008/TCP inbound - ztunnel's HBONE tunnel, which ambient explicitly does not let bypass NetworkPolicy enforcement - AND, where probeSources is non-empty, 169.254.7.127/32 as an additional probe source, because inside the mesh the kubelet's probes are redirected away from the worker node's address. The two halves are inseparable on purpose: whoever opens 15008 by hand has not added the probe address, and the first narrowed probeSources then drops every probe, so the pod never becomes Ready and the symptom reads as a broken application rather than as a policy. Unproven in operation: that 15008 alone suffices comes from Istio's documentation, and no namespace of this cluster carries istio.io/dataplane-mode yet.",
|
||||
"type": ["boolean", "null"]
|
||||
}
|
||||
}
|
||||
},
|
||||
"egress": {
|
||||
"type": ["object", "null"],
|
||||
"additionalProperties": false,
|
||||
|
||||
@@ -576,6 +576,57 @@ networkPolicy:
|
||||
namespace: kube-system
|
||||
podLabels:
|
||||
app.kubernetes.io/name: rke2-traefik
|
||||
|
||||
# The policy's INBOUND side. Mind the three names that overlap as substrings,
|
||||
# because a search for "ingress" matches all of them and they are different
|
||||
# things:
|
||||
#
|
||||
# ingress (top level) the Ingress OBJECT this chart
|
||||
# renders - host, TLS, className
|
||||
# networkPolicy.ingress this block: what the policy lets IN
|
||||
# networkPolicy.ingressController the SOURCE SELECTOR for the ingress
|
||||
# controller's pods. A SIBLING of this block
|
||||
# and NO PART OF IT, despite the name.
|
||||
ingress:
|
||||
# Opens the pod for Istio's ambient mesh (Gitea #143). OFF by default and
|
||||
# additive: while it is false this chart renders byte for byte as it did
|
||||
# before the value existed.
|
||||
#
|
||||
# IT RENDERS TWO THINGS, and that they come together is the point:
|
||||
#
|
||||
# 1. 15008/TCP inbound - ztunnel's HBONE tunnel. Every inbound mesh
|
||||
# connection arrives on it, and ambient "does not and will never bypass
|
||||
# Kubernetes NetworkPolicy enforcement", so without this port a pod
|
||||
# whose namespace has joined the mesh is unreachable.
|
||||
# 2. 169.254.7.127/32 as an additional peer - but ONLY where the rule
|
||||
# restricts sources at all, which here means probeSources is non-empty.
|
||||
# Inside the mesh the kubelet's probes are redirected and no longer come
|
||||
# from the worker node's address.
|
||||
#
|
||||
# WHY ONE SWITCH AND NOT TWO: point 2 is the one nobody thinks of. Whoever
|
||||
# adds "15008" to a port list by hand writes a policy that passes review, and
|
||||
# the first time probeSources is narrowed every probe is dropped - the pod
|
||||
# never becomes Ready and the symptom reads as a broken application rather
|
||||
# than as a policy. A switch whose halves can only be had together cannot be
|
||||
# set halfway, which is why this is a named intention and not a port list.
|
||||
#
|
||||
# TWO THINGS IT DOES NOT PROVE:
|
||||
# - That 15008 alone is enough. That comes from Istio's documentation;
|
||||
# measured on this cluster is only which listeners ztunnel opens.
|
||||
# Provable at the first namespace to join the mesh - none carries
|
||||
# istio.io/dataplane-mode yet.
|
||||
# - That the probe-source half works. probeSources is empty in every chart
|
||||
# here, so no release reaches that branch. CI proves its render, not its
|
||||
# effect.
|
||||
#
|
||||
# Note where the port lands: it joins the rule that carries the probe port,
|
||||
# so once probeSources restricts that rule, 15008 is restricted with it.
|
||||
# Whether ztunnel needs a peer of its own is open - it runs with its own pod
|
||||
# IPs in istio-system (measured: no hostNetwork on the DaemonSet, Istio
|
||||
# 1.30.6), so if one proves necessary it is expressible as a
|
||||
# namespaceSelector plus podSelector and needs no unknowable node CIDR.
|
||||
ambientMesh: false
|
||||
|
||||
egress:
|
||||
# Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and
|
||||
# has no switch - a knob whose only other setting breaks every name lookup
|
||||
|
||||
Reference in new issue
Block a user