Gitea #143, maintainer's instruction. Both numbers this switch renders are ISTIO CONSTANTS rather than standards, and a value with a documented default is the better home for a vendor constant than a template literal: an operator can correct it when Istio changes the number, without waiting for a chart release. networkPolicy: ingress: ambientMesh: enabled: false port: 15008 redirectedProbeSources: - 169.254.7.127/32 One object under one switch, so the coupling argument survives the change instead of being undermined by it - two separate values could drift apart, which is the thing this design exists to prevent. redirectedProbeSources is a LIST because a dual-stack cluster needs BOTH of Istio's addresses at once, not one of two; that is a nameable case rather than speculation, so it does not contradict reason 4 of the issue. Istio's IPv6 counterpart is documented as the dual-stack addition, with the reason it is absent today unchanged: SingleStack IPv4, and an ipBlock of the wrong family is accepted and matches nothing. Named redirectedProbeSources, not probeSources, because the confusion risk with networkPolicy.probeSources one level up is larger than the ingress/ ingressController one: that value holds YOUR worker node CIDRs, is empty by default and means OPEN when empty; this one holds ISTIO's rewrite address and ships filled, because unlike the node IPs it is a constant the chart can know. The distinction is written where both appear. PROVENANCE, which the previous round explained the coupling of but never attributed. 15008 is Istio's HBONE port - not IANA-registered for this, not prescribed by any protocol, unknown to Kubernetes. 169.254.7.127 is link-local (169.254.0.0/16, RFC 3927, never routed - the range one picks vendor magic out of); Istio's IPv6 counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the same digits plus 1337, which is what vendor constants look like; and Kubernetes knows the address in neither the NetworkPolicy API nor the kubelet - only Istio's data plane rewrites the probe source onto it. Both read for ztunnel 1.30.6. If either changes, the failure is SILENT: a pod unreachable in the mesh, or probes dropped where probeSources is non-empty, which is nowhere today. NULL-SAFE, per #99 and because #136 shipped this exact defect one value away (its NOTES.txt branch dereferenced .networkPolicy.egress directly): every read goes through a fallback reproducing the chart's own declared defaults, so a deleted object or one carrying only "enabled: true" renders the defaults rather than failing. Clearing the probe-source list restores the constant instead of removing the ipBlock, deliberately - the halves are not separable, and enabled: false is how you open nothing. THE CONSEQUENCE, and it is the point of the CI half. With the port configurable, the exception can no longer be checked against the rendered value: "whatever the value says" is design A's hole wearing design C's name, and a mistyped port would pass by sitting in the configurable field. So the expectation is PINNED IN THE TEST - AMBIENT_PORT and AMBIENT_CIDR - and assertion 7 now checks two things instead of one: that the render at chart defaults produces exactly that port and that ipBlock, and that every chart's values.yaml DECLARES the same defaults, read back through ci/yaml2json rather than by regex. A chart that drifts is red. Changing an Istio constant means touching 30 values files AND the pinned lines, and the pinned lines are what force someone to look. That block is therefore also the single place saying what we hold Istio's numbers to be, so it carries the provenance too. The renders set only enabled=true and never a port override, on purpose: the pin is a statement about the defaults we ship, not about what a release may choose - an operator overriding a value is outside this test's reach, as for every value here. The declared-defaults check is conditional on the chart offering probeSources, like the inseparability check and assertion 4: the five charts whose probe rule is never source-restricted render no ipBlock, so for them the key must be ABSENT - a value reaching no rule reads as a control that does nothing - and that absence is asserted rather than tolerated. Their schemas drop it too. No version change: nothing of this is published, the switch stays additive with its default off, and the render at defaults is byte-identical - re-diffed per chart against origin/main, not assumed.
333 lines
18 KiB
YAML
333 lines
18 KiB
YAML
{{- /*
|
|
NetworkPolicy — the reference implementation for Gitea #5.
|
|
|
|
Copy this file with the rest of template-chart and replace every xxx: the
|
|
values paths (.Values.networkPolicy.*), the helper prefix and the podSelector
|
|
label, exactly as in the other template files.
|
|
|
|
EVERY COMMENT IN THIS FILE IS A TEMPLATE COMMENT, and that is not a style
|
|
choice. A "#" comment is part of the RENDERED manifest: it travels into the
|
|
cluster, shows up in `helm get manifest` and `kubectl get -o yaml`, and bloats
|
|
every diff a reviewer reads. Reasoning belongs in a Go template comment, which
|
|
renders nothing. This caught several rounds of this rollout.
|
|
|
|
And a second-order trap found while writing this header: the prose inside a
|
|
template comment must not spell out the comment's own closing delimiter. The
|
|
parser ends the comment at the first occurrence, and the remaining prose is
|
|
then emitted as manifest content - the render fails with "did not find
|
|
expected node content", pointing at the template and not at the sentence.
|
|
Describe the delimiters instead of quoting them.
|
|
|
|
WHAT THIS OBJECT DOES, AND - MORE IMPORTANTLY - WHAT IT DOES NOT
|
|
|
|
It is a CEILING ON REACHABILITY, not an authentication layer. Three limits are
|
|
real and worth having:
|
|
|
|
1. ONLY THE DECLARED PORTS ARE REACHABLE AT ALL. Whatever else the process
|
|
binds - a debug listener, a management interface the image opens by
|
|
default, a port that appears in the next app version - is blocked without
|
|
anyone having to notice it exists. This is the largest part of what a
|
|
default-on policy buys in this repository.
|
|
2. EGRESS IS CAPPED BY PORT, so a compromised pod cannot open arbitrary
|
|
outbound connections.
|
|
3. INGRESS IS CAPPED BY SOURCE for every port no network probe touches.
|
|
|
|
And three things it does NOT do, each of which has to be written down per
|
|
chart rather than quietly assumed:
|
|
|
|
a. EGRESS IS COARSE. A NetworkPolicy matches CIDRs and selectors and NEVER
|
|
hostnames, so "only to the database host" cannot be expressed. This
|
|
cluster's CNI is Canal (Flannel plus Calico for policy), which has no
|
|
FQDN rule; Calico's GlobalNetworkSet could express one but would bind
|
|
every chart to Calico - the same argument that kept implementation-
|
|
specific objects out of the Gateway-API sweep (Gitea #114). So each
|
|
egress rule below permits its PORT to any destination. That is a limit of
|
|
the object, not a shortcut.
|
|
b. THE PROBED PORT IS OPEN INSIDE THE CLUSTER BY DEFAULT. See the long note
|
|
on networkPolicy.probeSources in values.yaml. Narrowing it needs the
|
|
worker node IPs, which this chart cannot know.
|
|
c. A POLICY IS ONLY A BOUNDARY WHERE THE CNI ENFORCES ONE. An unenforced
|
|
policy is accepted by the API server and filters nothing, so everything
|
|
looks configured while nothing is filtered. Measured for this cluster
|
|
(Canal, 6/6) and for the CI cluster (k3d, which brings k3s' own policy
|
|
controller) - verify it rather than reading it off a value.
|
|
|
|
ONE CLUSTER SETTING THIS WHOLE CONVENTION DEPENDS ON AND CANNOT CHECK:
|
|
TRAEFIK MUST NOT RUN WITH hostNetwork.
|
|
|
|
Measured on 2026-10-14: the rke2-traefik DaemonSet has no hostNetwork, and its
|
|
pods carry real pod IPs (10.42.3.120, 10.42.4.94, 10.42.0.187; the pod CIDR is
|
|
10.42.0.0/16, while the nodes sit on entirely different addresses). ONLY
|
|
BECAUSE OF THAT does the podSelector in networkPolicy.ingressController match
|
|
it.
|
|
|
|
Switch the ingress controller to host networking and its pods share the node's
|
|
network namespace. They are then not selectable as pods at all, the
|
|
ingressController entry matches nothing, and EVERY WEB CHART IN THIS REPOSITORY
|
|
GOES DARK AT ONCE in the restricted configuration - while every policy is
|
|
still accepted and still reads as correct. The symptom to recognise is exactly
|
|
that shape: all web traffic stops together and every pod stays Ready, because
|
|
the probe path does not run through the controller.
|
|
|
|
Nothing here can detect this and nothing here can fail usefully on it, which is
|
|
the same class of dependency as --enable-certificate-owner-ref in #126. It is
|
|
written here, on the thing that depends on it, so whoever changes the
|
|
controller's networking meets the consequence.
|
|
|
|
AND A POLICY DOES APPLY TO TRAFFIC THAT CAME THROUGH A SERVICE, which is why
|
|
the ingressController rule works in the first place. A Service is only
|
|
destination NAT, and the policy is evaluated at the pod's interface, so a
|
|
ClusterIP exempts nothing. What the Service changes is the SOURCE ADDRESS: pod
|
|
-> ClusterIP -> pod PRESERVES it, so a podSelector or a namespaceSelector can
|
|
still match the sender. Traffic arriving from outside through a NodePort with
|
|
the default externalTrafficPolicy: Cluster is MASQUERADED to a node, which is
|
|
why such rules cannot be restricted by source at all and are left open instead.
|
|
|
|
THE PORT NUMBERS HERE ARE POD PORTS, NEVER NODEPORTS. A NetworkPolicy is
|
|
evaluated at the POD's network interface, after kube-proxy has already
|
|
translated a NodePort to the targetPort. Writing 30080 where the pod listens
|
|
on 8080 matches nothing: the object is accepted, looks strict, and blocks
|
|
every inbound connection instead. ci/run-networkpolicy-test.sh asserts this
|
|
mechanically - every port named here must exist as a containerPort of the pod
|
|
spec this policy selects.
|
|
*/ -}}
|
|
{{- if .Values.networkPolicy.enabled }}
|
|
{{- $np := .Values.networkPolicy }}
|
|
{{- $probeSources := $np.probeSources | default list }}
|
|
{{- $allowedNs := $np.allowedNamespaces | default list }}
|
|
{{- $ic := $np.ingressController | default dict }}
|
|
{{- /* networkPolicy.ingress - the policy's INGRESS DIRECTION, and not to be
|
|
confused with networkPolicy.ingressController one line up, nor with the
|
|
chart's top-level .Values.ingress. Three different things whose names
|
|
overlap as substrings:
|
|
|
|
.Values.ingress the Ingress OBJECT this chart renders
|
|
.Values.networkPolicy.ingress switches on the policy's inbound side
|
|
.Values.networkPolicy.ingressController
|
|
the SOURCE SELECTOR for the ingress
|
|
controller's pods, a sibling of
|
|
networkPolicy.ingress and NO PART OF
|
|
IT
|
|
|
|
Named here because a search for "ingress" matches all three, and one
|
|
round of this work already reported ingressController as an inbound
|
|
escape hatch on the strength of the substring alone. */}}
|
|
{{- $npIngress := $np.ingress | default dict }}
|
|
{{- /* NULL-SAFE ON PURPOSE, and not as decoration (#99). ambientMesh is an
|
|
OBJECT, so every read below goes through a fallback that reproduces THIS
|
|
chart's declared defaults - because an operator who deletes the object,
|
|
or who writes only "enabled: true" and leaves the rest out, must land on
|
|
the defaults and not in a nil dereference. #136 shipped exactly that
|
|
defect one value away from here: the NOTES.txt branch dereferenced
|
|
.networkPolicy.egress directly, and a release that had dropped the key
|
|
failed to render.
|
|
|
|
Note which fallback the probe-source list gets: "default" treats an EMPTY
|
|
list as unset, so clearing it restores the constant rather than removing
|
|
the ipBlock. That is deliberate - the two halves of this switch are not
|
|
separable, and the way to not open the port at all is enabled: false. */}}
|
|
{{- $ambient := $npIngress.ambientMesh | default dict }}
|
|
{{- $ambientEnabled := $ambient.enabled }}
|
|
{{- $ambientPort := $ambient.port | default 15008 }}
|
|
{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }}
|
|
{{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }}
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
|
labels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
|
spec:
|
|
{{- /* Selects the Deployment's pods. A chart that keeps the StatefulSet
|
|
instead has to move this selector along with the Service's, and
|
|
ci/run-networkpolicy-test.sh fails if it selects no pod spec of the
|
|
same render - which is the mistake that would otherwise leave a
|
|
NetworkPolicy in the namespace filtering nothing at all. */}}
|
|
podSelector:
|
|
matchLabels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
|
policyTypes:
|
|
- Ingress
|
|
{{- /* Egress is listed because there IS something to restrict here: this
|
|
workload needs DNS and nothing else by default. A chart whose
|
|
workload opens no outbound connection of its own must NOT list
|
|
Egress - valkey is the example - because an egress rule that
|
|
constrains nothing makes the object look stricter than it is. */}}
|
|
- Egress
|
|
ingress:
|
|
{{- /* The http port. It is the port all three probes address, so it is
|
|
the probe path, and the shape below is the whole reason
|
|
networkPolicy.probeSources exists. */}}
|
|
- ports:
|
|
{{- range .Values.xxx.ports }}
|
|
- protocol: {{ .protocol | default "TCP" }}
|
|
port: {{ .containerPort }}
|
|
{{- end }}
|
|
{{- if $ambientEnabled }}
|
|
{{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh. The default is
|
|
15008/TCP: ztunnel's HBONE tunnel, the port every inbound mesh
|
|
connection arrives on once the namespace carries
|
|
istio.io/dataplane-mode. Istio's own documentation is explicit
|
|
that ambient "does not and will never bypass Kubernetes
|
|
NetworkPolicy enforcement", so without this port a pod in the
|
|
mesh is simply unreachable.
|
|
|
|
15008 IS AN ISTIO CONSTANT, NOT A STANDARD. It is not registered
|
|
with IANA for this, Kubernetes knows nothing about it, and no
|
|
protocol prescribes it - Istio picked it for HBONE and Istio can
|
|
change it. That is precisely why it sits in a VALUE with a
|
|
documented default rather than as a literal here: an operator can
|
|
correct it without waiting for a chart release.
|
|
|
|
It is the one port in this rule that is NOT a containerPort of
|
|
the selected pod spec - ztunnel terminates it, not the
|
|
application - so ci/run-networkpolicy-test.sh holds a NAMED
|
|
exception for it. Note what that exception is pinned to: the
|
|
number 15008 as written in THE TEST, plus an assertion that this
|
|
chart's values.yaml declares the same default. It is NOT "whatever
|
|
this value happens to say" - that would let a mistyped port ride
|
|
in through a configurable field, which is the hole design A of
|
|
#143 was rejected for.
|
|
|
|
READ FOR ztunnel 1.30.6. If Istio changes the port, or the mesh is
|
|
replaced: the failure is a pod that cannot be reached inside the
|
|
mesh. Change the default in all 30 charts AND the pinned number in
|
|
the test - which is the thing that forces someone to look.
|
|
|
|
NOT MEASURED, and the chart should not pretend otherwise: that
|
|
15008 ALONE suffices comes from Istio's documentation. What has
|
|
been measured on this cluster is only which listeners ztunnel
|
|
opens. Provable at the first namespace that joins the mesh, and
|
|
no namespace carries the label yet. */}}
|
|
- protocol: TCP
|
|
port: {{ int $ambientPort }}
|
|
{{- end }}
|
|
{{- if $probeSources }}
|
|
{{- /* RESTRICTED, because the operator supplied probeSources. The
|
|
kubelet's probes come from the IP of the WORKER NODE running the
|
|
pod, so that address has to be in the list or the pod never
|
|
becomes Ready - and a pod that is never Ready reads as a broken
|
|
application rather than as a policy. The ingress controller is
|
|
added here too, or the Ingress goes dark the moment this list is
|
|
filled. */}}
|
|
from:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
{{- /* Set by Kubernetes itself on every namespace (>= 1.21), so
|
|
this needs no labelling work on namespaces this chart does
|
|
not own. */}}
|
|
kubernetes.io/metadata.name: {{ .Release.Namespace }}
|
|
{{- if and $routed $ic.namespace }}
|
|
{{- /* ONE from-entry with TWO selectors, which is an AND: pods
|
|
carrying these labels IN that namespace. Written as two list
|
|
entries it would be an OR and would open the port to the whole
|
|
controller namespace - kube-system here, i.e. to everything in
|
|
the cluster's own namespace. */}}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ $ic.namespace | quote }}
|
|
{{- with $ic.podLabels }}
|
|
podSelector:
|
|
matchLabels:
|
|
{{- toYaml . | nindent 14 }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- range $allowedNs }}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ . | quote }}
|
|
{{- end }}
|
|
{{- range $probeSources }}
|
|
- ipBlock:
|
|
cidr: {{ . | quote }}
|
|
{{- end }}
|
|
{{- if $ambientEnabled }}
|
|
{{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason
|
|
the switch is one value instead of two. Inside ISTIO's ambient
|
|
mesh the kubelet's probes are redirected BY ISTIO and no longer
|
|
arrive from the worker node's address: they arrive from
|
|
169.254.7.127, which is therefore the probe source nobody would
|
|
think to add.
|
|
|
|
WHERE THAT ADDRESS COMES FROM, because it is worth being able to
|
|
recognise as vendor magic rather than as protocol:
|
|
|
|
- It is LINK-LOCAL, 169.254.0.0/16 (RFC 3927) - a range
|
|
reserved for node-local use that is never routed. Exactly
|
|
the range one picks a vendor's magic address out of.
|
|
- Istio's IPv6 counterpart gives the game away:
|
|
fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME
|
|
DIGITS - 9254:7127 is 169.254.7.127 - plus 1337. Numbers
|
|
built that way are vendor constants, not protocols.
|
|
- KUBERNETES DOES NOT KNOW THIS ADDRESS. Neither the
|
|
NetworkPolicy API nor the kubelet has any notion of it; only
|
|
Istio's own data plane rewrites the probe source onto it.
|
|
|
|
READ FOR ztunnel 1.30.6. If Istio changes it, or the mesh is
|
|
replaced, THE FAILURE IS SILENT: probes would be dropped, and
|
|
only where probeSources is non-empty - which is nowhere today.
|
|
Nothing in this repository or in Kubernetes would report it. That
|
|
is why the address is a VALUE with a documented default, so it can
|
|
be corrected without a chart release, and why the test pins the
|
|
number it expects: changing it means touching 30 values files and
|
|
the test, and the test is what makes someone look.
|
|
|
|
A LIST, not a string, and this is the one case where that does not
|
|
contradict "the value is a constant, not a configuration": on a
|
|
DUAL-STACK cluster BOTH entries are needed at the same time, not
|
|
one of two. This cluster is SingleStack IPv4, so the IPv6
|
|
counterpart above is left out of the default - an ipBlock of the
|
|
wrong family is accepted by the API server and matches nothing.
|
|
Add it alongside, never instead, if the cluster gains IPv6.
|
|
|
|
The two halves stay inseparable. An operator who only opens the
|
|
port by hand has a policy that passes every review - and the
|
|
moment probeSources is non-empty, every probe is dropped, the pod
|
|
never becomes Ready, and the symptom reads as a broken
|
|
application rather than as a policy. That is assertion 4 of
|
|
ci/run-networkpolicy-test.sh, re-created one value further along.
|
|
|
|
Only rendered in this branch, because this is the only branch
|
|
with a "from" list at all: with probeSources empty the rule is
|
|
open and needs no peer.
|
|
|
|
UNPROVEN IN OPERATION: probeSources is empty in every chart of
|
|
this repository (stalwart sets it to the empty list explicitly),
|
|
so no release reaches this branch today. What CI proves is its
|
|
render, through the overlay in ci/networkpolicy. */}}
|
|
{{- range $ambientProbeSources }}
|
|
- ipBlock:
|
|
cidr: {{ . | quote }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- else }}
|
|
{{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only
|
|
inside the cluster, because this port carries no NodePort. The
|
|
alternative was to guess the worker node CIDRs, and a wrong guess
|
|
blocks every kubelet probe: the pod stays un-Ready, the rollout
|
|
times out, and the cause looks like the application. Note what an
|
|
EMPTY "from: []" would do instead: it also allows everything, but
|
|
while looking like a restriction. Hence no key at all, and
|
|
ci/run-networkpolicy-test.sh rejects the empty form. */}}
|
|
{{- end }}
|
|
egress:
|
|
{{- /* DNS, unconditionally. There is no value to switch this off, and
|
|
that is deliberate: a knob whose only other setting breaks every
|
|
name lookup in the pod is a trap and not a feature. */}}
|
|
- ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
{{- with $np.egress }}
|
|
{{- with .extraPorts }}
|
|
{{- /* Ports only - see note (a) in the header. Any destination. */}}
|
|
- ports:
|
|
{{- range . }}
|
|
- protocol: TCP
|
|
port: {{ int . }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|