jklein
8dd15d42d5
Gitea #143, maintainer's instruction. The HBONE port and the redirected probe source are ISTIO CONSTANTS, not standards, so they move from template literals into values with documented defaults - an operator can then correct them when Istio changes a number, without waiting for a chart release. One object under one "enabled", so the two halves still cannot be set separately; two values could drift apart, which is what this design exists to prevent. Every read is null-safe (#99), so a deleted object or one carrying only "enabled: true" falls back to the defaults declared here rather than dereferencing nil - the defect #136 shipped one value away. Provenance now written on each constant: 15008 is Istio's HBONE port, not IANA-registered for this and unknown to Kubernetes; 169.254.7.127 is link-local (RFC 3927, never routed), its IPv6 counterpart carries the same digits plus 1337, and neither the NetworkPolicy API nor the kubelet knows the address - only Istio's data plane rewrites the probe source onto it. Both read for ztunnel 1.30.6, and a change in either fails silently. The exception in ci/run-networkpolicy-test.sh stays pinned to the numbers that test names, and additionally asserts this chart declares the same defaults - never "whatever the value says", which would let a mistyped port in through a configurable field. No version change: nothing is published yet, the default is still off, and the render at defaults is byte-identical - re-diffed against origin/main.
296 lines
15 KiB
YAML
296 lines
15 KiB
YAML
{{- /*
|
|
One NetworkPolicy per node group (Gitea #5).
|
|
|
|
THIS CHART AND valkey ARE THE TWO THAT HAD A POLICY BEFORE #5, so the
|
|
convention was derived from them rather than imposed on them. Three things
|
|
changed here; everything else is the original and is right.
|
|
|
|
- THE REASONING MOVED OUT OF "#" COMMENTS into template comments. A "#"
|
|
comment is part of the RENDERED object: it travels into the cluster, shows
|
|
up in `helm get manifest` and `kubectl get -o yaml`, and lands in every
|
|
release diff. ci/run-networkpolicy-test.sh now fails on a rendered "#", and
|
|
this file was one of the two that failed it.
|
|
- networkPolicy.ingressController ARRIVED, and it fixes a real hole this
|
|
file's own comment described without being able to solve: it used to say
|
|
that setting probeSources means putting the ingress controller's namespace
|
|
into allowedNamespaces "or the Ingress goes dark". That works, but
|
|
allowedNamespaces is namespace-granular, so it would have opened the http
|
|
port to EVERY pod in the controller's namespace - kube-system, here. The
|
|
new value renders a namespaceSelector and a podSelector in ONE from-entry,
|
|
which is an AND, and narrows the source to the controller's pods.
|
|
- THE EXAMPLE IN values.yaml WAS WRONG AND IS CORRECTED. It suggested
|
|
allowedNamespaces: ["ingress-nginx"]. Measured on 2026-10-14: ingress-nginx
|
|
is GONE from this cluster - no DaemonSet, no pods - the only IngressClass is
|
|
traefik, and the controller is the DaemonSet rke2-traefik in kube-system. A
|
|
policy written against the old controller would have been accepted, would
|
|
have looked strict, and would have silently blocked all web traffic.
|
|
|
|
WHAT DID NOT CHANGE, because it is the reason this file was the model:
|
|
|
|
This object is the CEILING ON REACHABILITY, and that is its real job here: it
|
|
holds no matter which ports the process actually bound. A node whose ClusterRole
|
|
is missing from the database fail-opens into a full node and binds every
|
|
listener - including IMAP and the admin interface on a group meant to take SMTP
|
|
only - while staying Running with a green probe. The per-port Services publish
|
|
only what was asked for, and this policy is the second, independent limit. See
|
|
values.yaml for the source-level detail.
|
|
|
|
THE PORT NUMBERS HERE ARE POD PORTS (25/465/993/8080), NOT NODEPORTS.
|
|
|
|
This is the easiest mistake to make in this chart, so it is spelled out. A
|
|
NetworkPolicy is evaluated at the POD's network interface, after kube-proxy has
|
|
already translated the NodePort to the targetPort. Writing 30025 here would
|
|
match nothing: the packet arrives at the pod on 25. The policy would then look
|
|
strict, be accepted by the API server, and silently block every inbound
|
|
connection - or, if it is the only rule, block the lot.
|
|
ci/run-networkpolicy-test.sh now asserts this mechanically, against the
|
|
containerPorts of the pod spec each policy selects.
|
|
|
|
Egress is restricted BY PORT and not by destination, because a NetworkPolicy
|
|
matches CIDRs and selectors and never hostnames - so dataStore.postgres.host
|
|
and externalServices.*.host cannot appear in it. What it still buys: a
|
|
compromised pod cannot open arbitrary outbound ports.
|
|
|
|
And a NetworkPolicy is only a boundary where the cluster's CNI enforces one. An
|
|
unenforced policy is accepted and filters nothing.
|
|
*/ -}}
|
|
{{- if .Values.networkPolicy.enabled }}
|
|
{{- $root := . -}}
|
|
{{- $ports := fromYaml (include "stalwart.ports" .) -}}
|
|
{{- $egress := .Values.networkPolicy.egress -}}
|
|
{{- $probeSources := .Values.networkPolicy.probeSources | default list -}}
|
|
{{- $allowedNs := .Values.networkPolicy.allowedNamespaces | default list -}}
|
|
{{- $ic := .Values.networkPolicy.ingressController | default dict -}}
|
|
{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing
|
|
as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress
|
|
controller's pods, a SIBLING of it and no part of it) nor as the
|
|
top-level .Values.ingress (the Ingress object). All three match a search
|
|
for "ingress"; one round of this work already mistook the second for an
|
|
inbound escape hatch on the strength of the substring. */}}
|
|
{{- $npIngress := .Values.networkPolicy.ingress | default dict }}
|
|
{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes
|
|
through a fallback reproducing THIS chart's declared defaults. An
|
|
operator who deletes the object, or writes only "enabled: true", must
|
|
land on the defaults and not in a nil dereference - #136 shipped exactly
|
|
that defect one value away, where the NOTES.txt branch dereferenced
|
|
.networkPolicy.egress directly. Clearing the probe-source list restores
|
|
the constant rather than removing the ipBlock, deliberately: the two
|
|
halves are not separable, and enabled: false is how you open nothing. */}}
|
|
{{- $ambient := $npIngress.ambientMesh | default dict }}
|
|
{{- $ambientEnabled := $ambient.enabled }}
|
|
{{- $ambientPort := $ambient.port | default 15008 }}
|
|
{{- $ambientProbeSources := $ambient.redirectedProbeSources | default (list "169.254.7.127/32") }}
|
|
{{- range $group, $spec := fromYaml (include "stalwart.nodes" .) }}
|
|
{{- $name := printf "%s-%s" (include "stalwart.fullname" $root) $group }}
|
|
---
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: {{ $name }}-networkpolicy
|
|
labels:
|
|
app: {{ $name }}
|
|
spec:
|
|
podSelector:
|
|
matchLabels:
|
|
app: {{ $name }}-sfs
|
|
policyTypes:
|
|
- Ingress
|
|
- Egress
|
|
ingress:
|
|
{{- /*
|
|
The public mail ports: open to any source. They have to be - the senders are
|
|
the whole internet, and after NodePort masquerading the source address is a
|
|
control-plane node anyway.
|
|
*/}}
|
|
{{- $publicPorts := list }}
|
|
{{- range $portName := $spec.exposedPorts }}
|
|
{{- $port := index $ports $portName }}
|
|
{{- if $port.public }}{{- $publicPorts = append $publicPorts $portName }}{{- end }}
|
|
{{- end }}
|
|
{{- if $publicPorts }}
|
|
- ports:
|
|
{{- range $portName := $publicPorts }}
|
|
{{- $port := index $ports $portName }}
|
|
{{- /*
|
|
{{ $portName }}: published on a nodePort, reaching the pod on
|
|
{{ int $port.containerPort }} - which is the number that belongs here.
|
|
*/}}
|
|
- protocol: {{ $port.protocol | default "TCP" }}
|
|
port: {{ int $port.containerPort }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- /* The http port: probes plus the ingress controller. */}}
|
|
{{- $http := index $ports "http" }}
|
|
- ports:
|
|
- protocol: {{ $http.protocol | default "TCP" }}
|
|
port: {{ int $http.containerPort }}
|
|
{{- if $ambientEnabled }}
|
|
{{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP
|
|
is ztunnel's HBONE tunnel, the port every inbound connection arrives on
|
|
once the namespace carries istio.io/dataplane-mode - ambient "does not
|
|
and will never bypass Kubernetes NetworkPolicy enforcement", so without
|
|
it a pod in the mesh is simply unreachable.
|
|
|
|
It is the one port here that is NOT a containerPort of the selected pod
|
|
spec: ztunnel terminates it, not the application. ci/run-networkpolicy-
|
|
test.sh therefore holds a NAMED exception for exactly this number, gated
|
|
on exactly this value - and assertion 7 of that script proves, on a
|
|
second render, that 15008 stays the ONLY such port and that it is absent
|
|
while the value is false.
|
|
|
|
AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with
|
|
IANA for this, no protocol prescribes it, and Kubernetes knows
|
|
nothing about it - Istio picked it for HBONE and Istio can change
|
|
it. Hence a VALUE with a documented default rather than a literal
|
|
here, so an operator can correct it without a chart release. Read
|
|
for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh
|
|
grants is pinned to the number THAT TEST names, plus a check that
|
|
this chart declares the same default - never to whatever the value
|
|
says, because that would let a mistyped port ride in through a
|
|
configurable field (design A of #143).
|
|
|
|
NOT MEASURED: that 15008 alone suffices comes from Istio's
|
|
documentation. Measured on this cluster is only which listeners ztunnel
|
|
opens. Provable at the first namespace to join the mesh; none has. */}}
|
|
- protocol: TCP
|
|
port: {{ int $ambientPort }}
|
|
{{- end }}
|
|
{{- if or $probeSources $allowedNs }}
|
|
{{- /*
|
|
Restricted because networkPolicy.probeSources or .allowedNamespaces is
|
|
set. MIND THE KUBELET: probes come from the IP of the worker node running
|
|
the pod, so that address has to be in probeSources or the pod never
|
|
becomes Ready - and a pod that is never Ready reads as a broken
|
|
application rather than as a policy.
|
|
*/}}
|
|
from:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
{{- /*
|
|
Set by Kubernetes itself on every namespace (>= 1.21), so this
|
|
needs no labelling work.
|
|
*/}}
|
|
kubernetes.io/metadata.name: {{ $root.Release.Namespace }}
|
|
{{- if $ic.namespace }}
|
|
{{- /*
|
|
The ingress controller, as ONE from-entry with TWO selectors - which is
|
|
an AND: those pods in that namespace. Written as two list entries it
|
|
would be an OR and would open the http port to every pod in the
|
|
controller's namespace. This is what replaced the old advice of putting
|
|
kube-system into allowedNamespaces.
|
|
*/}}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ $ic.namespace | quote }}
|
|
{{- with $ic.podLabels }}
|
|
podSelector:
|
|
matchLabels:
|
|
{{- toYaml . | nindent 14 }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- range $allowedNs }}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ . | quote }}
|
|
{{- end }}
|
|
{{- range $probeSources }}
|
|
- ipBlock:
|
|
cidr: {{ . | quote }}
|
|
{{- end }}
|
|
{{- if $ambientEnabled }}
|
|
{{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is
|
|
one value and not two. Inside the mesh the kubelet's probes are
|
|
redirected and no longer arrive from the worker node's address: they
|
|
arrive from 169.254.7.127.
|
|
|
|
An operator who only opens 15008 by hand writes a policy that passes
|
|
review - and the moment probeSources is non-empty every probe is
|
|
dropped, the pod never becomes Ready, and the symptom reads as a broken
|
|
application rather than as a policy. That is assertion 4 of
|
|
ci/run-networkpolicy-test.sh one value further along. A switch whose
|
|
halves can only be had together cannot be set halfway.
|
|
|
|
Only in this branch, because this is the only branch with a "from" list:
|
|
with no sources restricted the rule is open and needs no peer. The IPv6
|
|
counterpart is deliberately absent - this cluster is SingleStack IPv4,
|
|
and an ipBlock of the wrong family is accepted and matches nothing.
|
|
|
|
WHERE 169.254.7.127 COMES FROM - vendor magic, not protocol:
|
|
it is LINK-LOCAL (169.254.0.0/16, RFC 3927), a range reserved for
|
|
node-local use and never routed; Istio's IPv6 counterpart
|
|
fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries the SAME DIGITS
|
|
plus 1337; and KUBERNETES DOES NOT KNOW THE ADDRESS - neither the
|
|
NetworkPolicy API nor the kubelet - only Istio's data plane
|
|
rewrites the probe source onto it. Read for ztunnel 1.30.6, and if
|
|
Istio changes it THE FAILURE IS SILENT: probes dropped, and only
|
|
where probeSources is non-empty, which is nowhere today.
|
|
|
|
A LIST, not a string, and the one case where that does not
|
|
contradict "a constant, not a configuration": a DUAL-STACK cluster
|
|
needs BOTH entries at once. This cluster is SingleStack IPv4, so
|
|
the IPv6 counterpart is left out - an ipBlock of the wrong family
|
|
is accepted and matches nothing. Add it ALONGSIDE, never instead.
|
|
|
|
UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no
|
|
release reaches this branch. CI proves its render, through the overlays
|
|
in ci/networkpolicy. */}}
|
|
{{- range $ambientProbeSources }}
|
|
- ipBlock:
|
|
cidr: {{ . | quote }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- else }}
|
|
{{- /*
|
|
NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the
|
|
cluster: http carries no nodePort, so nothing outside reaches it through
|
|
this chart. The alternative was to guess the worker node CIDRs, and a
|
|
wrong guess blocks every kubelet probe. Narrow it by setting
|
|
networkPolicy.probeSources; the ingress controller is then added from
|
|
networkPolicy.ingressController, so the Ingress does not go dark.
|
|
|
|
Note that an empty "from: []" would be worse than no key at all: it also
|
|
allows everything, but while reading as a restriction.
|
|
ci/run-networkpolicy-test.sh rejects that form.
|
|
*/}}
|
|
{{- end }}
|
|
egress:
|
|
{{- if $egress.dns }}
|
|
- ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
{{- end }}
|
|
{{- if $egress.smtp }}
|
|
{{- /*
|
|
Outbound delivery: port 25 on any host on the internet. Uniform for every
|
|
group on purpose - the chart does not know which groups run the outboundMta
|
|
task, the value would drift away from the database, and the failure mode of
|
|
getting it wrong is mail silently not leaving.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 25
|
|
{{- end }}
|
|
{{- $outPorts := list (int $root.Values.dataStore.postgres.port) }}
|
|
{{- with $root.Values.externalServices }}
|
|
{{- if .s3 }}{{- $outPorts = append $outPorts (int .s3.port) }}{{- end }}
|
|
{{- if .valkey }}{{- $outPorts = append $outPorts (int .valkey.port) }}{{- end }}
|
|
{{- end }}
|
|
{{- range $root.Values.networkPolicy.egress.extraPorts }}
|
|
{{- $outPorts = append $outPorts (int .) }}
|
|
{{- end }}
|
|
{{- /*
|
|
The data store, the blob store, the in-memory store and anything added
|
|
through networkPolicy.egress.extraPorts. Ports only - see the header. Unlike
|
|
most charts in this sweep these are the CONFIGURED ports rather than the
|
|
standard ones, because this chart holds them as values.
|
|
*/}}
|
|
- ports:
|
|
{{- range $outPorts | uniq }}
|
|
- protocol: TCP
|
|
port: {{ . }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|