jklein
fe573cc762
Gitea #143. Additive, default false, so the render is byte-identical to the previous release - proven by diffing both renders against origin/main rather than asserted. Switched on it adds 15008/TCP, ztunnel's HBONE tunnel, to the rule carrying the probe port, and - inside that rule's source-restricted branch - 169.254.7.127/32, the address the kubelet's probes arrive from once the pod is in the mesh. Ambient does not bypass NetworkPolicy enforcement, so without the port a pod whose namespace has joined the mesh is unreachable. The two halves are one value because the second is the one nobody thinks of: 15008 added by hand leaves a policy that passes review and drops every probe the first time probeSources is narrowed, and a pod that never becomes Ready reads as a broken application rather than as a policy. Minor, not major: additive value, unchanged default, unchanged render.
256 lines
12 KiB
YAML
256 lines
12 KiB
YAML
{{- /*
|
|
One NetworkPolicy per node group (Gitea #5).
|
|
|
|
THIS CHART AND valkey ARE THE TWO THAT HAD A POLICY BEFORE #5, so the
|
|
convention was derived from them rather than imposed on them. Three things
|
|
changed here; everything else is the original and is right.
|
|
|
|
- THE REASONING MOVED OUT OF "#" COMMENTS into template comments. A "#"
|
|
comment is part of the RENDERED object: it travels into the cluster, shows
|
|
up in `helm get manifest` and `kubectl get -o yaml`, and lands in every
|
|
release diff. ci/run-networkpolicy-test.sh now fails on a rendered "#", and
|
|
this file was one of the two that failed it.
|
|
- networkPolicy.ingressController ARRIVED, and it fixes a real hole this
|
|
file's own comment described without being able to solve: it used to say
|
|
that setting probeSources means putting the ingress controller's namespace
|
|
into allowedNamespaces "or the Ingress goes dark". That works, but
|
|
allowedNamespaces is namespace-granular, so it would have opened the http
|
|
port to EVERY pod in the controller's namespace - kube-system, here. The
|
|
new value renders a namespaceSelector and a podSelector in ONE from-entry,
|
|
which is an AND, and narrows the source to the controller's pods.
|
|
- THE EXAMPLE IN values.yaml WAS WRONG AND IS CORRECTED. It suggested
|
|
allowedNamespaces: ["ingress-nginx"]. Measured on 2026-10-14: ingress-nginx
|
|
is GONE from this cluster - no DaemonSet, no pods - the only IngressClass is
|
|
traefik, and the controller is the DaemonSet rke2-traefik in kube-system. A
|
|
policy written against the old controller would have been accepted, would
|
|
have looked strict, and would have silently blocked all web traffic.
|
|
|
|
WHAT DID NOT CHANGE, because it is the reason this file was the model:
|
|
|
|
This object is the CEILING ON REACHABILITY, and that is its real job here: it
|
|
holds no matter which ports the process actually bound. A node whose ClusterRole
|
|
is missing from the database fail-opens into a full node and binds every
|
|
listener - including IMAP and the admin interface on a group meant to take SMTP
|
|
only - while staying Running with a green probe. The per-port Services publish
|
|
only what was asked for, and this policy is the second, independent limit. See
|
|
values.yaml for the source-level detail.
|
|
|
|
THE PORT NUMBERS HERE ARE POD PORTS (25/465/993/8080), NOT NODEPORTS.
|
|
|
|
This is the easiest mistake to make in this chart, so it is spelled out. A
|
|
NetworkPolicy is evaluated at the POD's network interface, after kube-proxy has
|
|
already translated the NodePort to the targetPort. Writing 30025 here would
|
|
match nothing: the packet arrives at the pod on 25. The policy would then look
|
|
strict, be accepted by the API server, and silently block every inbound
|
|
connection - or, if it is the only rule, block the lot.
|
|
ci/run-networkpolicy-test.sh now asserts this mechanically, against the
|
|
containerPorts of the pod spec each policy selects.
|
|
|
|
Egress is restricted BY PORT and not by destination, because a NetworkPolicy
|
|
matches CIDRs and selectors and never hostnames - so dataStore.postgres.host
|
|
and externalServices.*.host cannot appear in it. What it still buys: a
|
|
compromised pod cannot open arbitrary outbound ports.
|
|
|
|
And a NetworkPolicy is only a boundary where the cluster's CNI enforces one. An
|
|
unenforced policy is accepted and filters nothing.
|
|
*/ -}}
|
|
{{- if .Values.networkPolicy.enabled }}
|
|
{{- $root := . -}}
|
|
{{- $ports := fromYaml (include "stalwart.ports" .) -}}
|
|
{{- $egress := .Values.networkPolicy.egress -}}
|
|
{{- $probeSources := .Values.networkPolicy.probeSources | default list -}}
|
|
{{- $allowedNs := .Values.networkPolicy.allowedNamespaces | default list -}}
|
|
{{- $ic := .Values.networkPolicy.ingressController | default dict -}}
|
|
{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same thing
|
|
as networkPolicy.ingressController (the SOURCE SELECTOR for the ingress
|
|
controller's pods, a SIBLING of it and no part of it) nor as the
|
|
top-level .Values.ingress (the Ingress object). All three match a search
|
|
for "ingress"; one round of this work already mistook the second for an
|
|
inbound escape hatch on the strength of the substring. */}}
|
|
{{- $npIngress := .Values.networkPolicy.ingress | default dict }}
|
|
{{- $ambientMesh := $npIngress.ambientMesh }}
|
|
{{- range $group, $spec := fromYaml (include "stalwart.nodes" .) }}
|
|
{{- $name := printf "%s-%s" (include "stalwart.fullname" $root) $group }}
|
|
---
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: {{ $name }}-networkpolicy
|
|
labels:
|
|
app: {{ $name }}
|
|
spec:
|
|
podSelector:
|
|
matchLabels:
|
|
app: {{ $name }}-sfs
|
|
policyTypes:
|
|
- Ingress
|
|
- Egress
|
|
ingress:
|
|
{{- /*
|
|
The public mail ports: open to any source. They have to be - the senders are
|
|
the whole internet, and after NodePort masquerading the source address is a
|
|
control-plane node anyway.
|
|
*/}}
|
|
{{- $publicPorts := list }}
|
|
{{- range $portName := $spec.exposedPorts }}
|
|
{{- $port := index $ports $portName }}
|
|
{{- if $port.public }}{{- $publicPorts = append $publicPorts $portName }}{{- end }}
|
|
{{- end }}
|
|
{{- if $publicPorts }}
|
|
- ports:
|
|
{{- range $portName := $publicPorts }}
|
|
{{- $port := index $ports $portName }}
|
|
{{- /*
|
|
{{ $portName }}: published on a nodePort, reaching the pod on
|
|
{{ int $port.containerPort }} - which is the number that belongs here.
|
|
*/}}
|
|
- protocol: {{ $port.protocol | default "TCP" }}
|
|
port: {{ int $port.containerPort }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- /* The http port: probes plus the ingress controller. */}}
|
|
{{- $http := index $ports "http" }}
|
|
- ports:
|
|
- protocol: {{ $http.protocol | default "TCP" }}
|
|
port: {{ int $http.containerPort }}
|
|
{{- if $ambientMesh }}
|
|
{{- /* FIRST HALF OF networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP
|
|
is ztunnel's HBONE tunnel, the port every inbound connection arrives on
|
|
once the namespace carries istio.io/dataplane-mode - ambient "does not
|
|
and will never bypass Kubernetes NetworkPolicy enforcement", so without
|
|
it a pod in the mesh is simply unreachable.
|
|
|
|
It is the one port here that is NOT a containerPort of the selected pod
|
|
spec: ztunnel terminates it, not the application. ci/run-networkpolicy-
|
|
test.sh therefore holds a NAMED exception for exactly this number, gated
|
|
on exactly this value - and assertion 7 of that script proves, on a
|
|
second render, that 15008 stays the ONLY such port and that it is absent
|
|
while the value is false.
|
|
|
|
NOT MEASURED: that 15008 alone suffices comes from Istio's
|
|
documentation. Measured on this cluster is only which listeners ztunnel
|
|
opens. Provable at the first namespace to join the mesh; none has. */}}
|
|
- protocol: TCP
|
|
port: 15008
|
|
{{- end }}
|
|
{{- if or $probeSources $allowedNs }}
|
|
{{- /*
|
|
Restricted because networkPolicy.probeSources or .allowedNamespaces is
|
|
set. MIND THE KUBELET: probes come from the IP of the worker node running
|
|
the pod, so that address has to be in probeSources or the pod never
|
|
becomes Ready - and a pod that is never Ready reads as a broken
|
|
application rather than as a policy.
|
|
*/}}
|
|
from:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
{{- /*
|
|
Set by Kubernetes itself on every namespace (>= 1.21), so this
|
|
needs no labelling work.
|
|
*/}}
|
|
kubernetes.io/metadata.name: {{ $root.Release.Namespace }}
|
|
{{- if $ic.namespace }}
|
|
{{- /*
|
|
The ingress controller, as ONE from-entry with TWO selectors - which is
|
|
an AND: those pods in that namespace. Written as two list entries it
|
|
would be an OR and would open the http port to every pod in the
|
|
controller's namespace. This is what replaced the old advice of putting
|
|
kube-system into allowedNamespaces.
|
|
*/}}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ $ic.namespace | quote }}
|
|
{{- with $ic.podLabels }}
|
|
podSelector:
|
|
matchLabels:
|
|
{{- toYaml . | nindent 14 }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- range $allowedNs }}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ . | quote }}
|
|
{{- end }}
|
|
{{- range $probeSources }}
|
|
- ipBlock:
|
|
cidr: {{ . | quote }}
|
|
{{- end }}
|
|
{{- if $ambientMesh }}
|
|
{{- /* SECOND HALF OF networkPolicy.ingress.ambientMesh, and the reason it is
|
|
one value and not two. Inside the mesh the kubelet's probes are
|
|
redirected and no longer arrive from the worker node's address: they
|
|
arrive from 169.254.7.127.
|
|
|
|
An operator who only opens 15008 by hand writes a policy that passes
|
|
review - and the moment probeSources is non-empty every probe is
|
|
dropped, the pod never becomes Ready, and the symptom reads as a broken
|
|
application rather than as a policy. That is assertion 4 of
|
|
ci/run-networkpolicy-test.sh one value further along. A switch whose
|
|
halves can only be had together cannot be set halfway.
|
|
|
|
Only in this branch, because this is the only branch with a "from" list:
|
|
with no sources restricted the rule is open and needs no peer. The IPv6
|
|
counterpart is deliberately absent - this cluster is SingleStack IPv4,
|
|
and an ipBlock of the wrong family is accepted and matches nothing.
|
|
|
|
UNPROVEN IN OPERATION: probeSources is empty in every chart here, so no
|
|
release reaches this branch. CI proves its render, through the overlays
|
|
in ci/networkpolicy. */}}
|
|
- ipBlock:
|
|
cidr: 169.254.7.127/32
|
|
{{- end }}
|
|
{{- else }}
|
|
{{- /*
|
|
NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the
|
|
cluster: http carries no nodePort, so nothing outside reaches it through
|
|
this chart. The alternative was to guess the worker node CIDRs, and a
|
|
wrong guess blocks every kubelet probe. Narrow it by setting
|
|
networkPolicy.probeSources; the ingress controller is then added from
|
|
networkPolicy.ingressController, so the Ingress does not go dark.
|
|
|
|
Note that an empty "from: []" would be worse than no key at all: it also
|
|
allows everything, but while reading as a restriction.
|
|
ci/run-networkpolicy-test.sh rejects that form.
|
|
*/}}
|
|
{{- end }}
|
|
egress:
|
|
{{- if $egress.dns }}
|
|
- ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
{{- end }}
|
|
{{- if $egress.smtp }}
|
|
{{- /*
|
|
Outbound delivery: port 25 on any host on the internet. Uniform for every
|
|
group on purpose - the chart does not know which groups run the outboundMta
|
|
task, the value would drift away from the database, and the failure mode of
|
|
getting it wrong is mail silently not leaving.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 25
|
|
{{- end }}
|
|
{{- $outPorts := list (int $root.Values.dataStore.postgres.port) }}
|
|
{{- with $root.Values.externalServices }}
|
|
{{- if .s3 }}{{- $outPorts = append $outPorts (int .s3.port) }}{{- end }}
|
|
{{- if .valkey }}{{- $outPorts = append $outPorts (int .valkey.port) }}{{- end }}
|
|
{{- end }}
|
|
{{- range $root.Values.networkPolicy.egress.extraPorts }}
|
|
{{- $outPorts = append $outPorts (int .) }}
|
|
{{- end }}
|
|
{{- /*
|
|
The data store, the blob store, the in-memory store and anything added
|
|
through networkPolicy.egress.extraPorts. Ports only - see the header. Unlike
|
|
most charts in this sweep these are the CONFIGURED ports rather than the
|
|
standard ones, because this chart holds them as values.
|
|
*/}}
|
|
- ports:
|
|
{{- range $outPorts | uniq }}
|
|
- protocol: TCP
|
|
port: {{ . }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|