jklein
f88e443a2e
Gitea #143, maintainer's instruction. The HBONE port and the redirected probe source are ISTIO CONSTANTS, not standards, so they move from template literals into values with documented defaults - an operator can then correct them when Istio changes a number, without waiting for a chart release. One object under one "enabled", so the two halves still cannot be set separately; two values could drift apart, which is what this design exists to prevent. Every read is null-safe (#99), so a deleted object or one carrying only "enabled: true" falls back to the defaults declared here rather than dereferencing nil - the defect #136 shipped one value away. Provenance now written on each constant: 15008 is Istio's HBONE port, not IANA-registered for this and unknown to Kubernetes; 169.254.7.127 is link-local (RFC 3927, never routed), its IPv6 counterpart carries the same digits plus 1337, and neither the NetworkPolicy API nor the kubelet knows the address - only Istio's data plane rewrites the probe source onto it. Both read for ztunnel 1.30.6, and a change in either fails silently. The exception in ci/run-networkpolicy-test.sh stays pinned to the numbers that test names, and additionally asserts this chart declares the same defaults - never "whatever the value says", which would let a mistyped port in through a configurable field. No version change: nothing is published yet, the default is still off, and the render at defaults is byte-identical - re-diffed against origin/main. No redirectedProbeSources here, in neither values.yaml nor the schema: this chart offers no networkPolicy.probeSources, so its probe rule is never source-restricted and no ipBlock is ever rendered. The key would reach no rule and read as a control that does nothing, which is worse than one that is absent. ci/run-networkpolicy-test.sh asserts that absence rather than tolerating it.
310 lines
15 KiB
YAML
310 lines
15 KiB
YAML
{{- /*
|
|
NetworkPolicy for minecraft-server (Gitea #5).
|
|
|
|
THIS IS THE CHART WHERE THE POLICY GETS GENUINELY TIGHT, and the reason is its
|
|
probes: all three are exec probes. An exec probe runs inside the container
|
|
through the kubelet's runtime and puts NO packet on the network, so there is no
|
|
kubelet traffic to accommodate on any port - which means every port here can be
|
|
restricted by source, with no probeSources needed at all. valkey is tight for
|
|
the same reason.
|
|
|
|
The one rule that cannot be fixed is the game port, and it depends on a value:
|
|
|
|
server.service.type == NodePort players arrive from the internet, already
|
|
masqueraded, so the rule must be OPEN.
|
|
anything else (the default) the only reachable path is in-cluster, i.e.
|
|
minecraft-router, so the rule is RESTRICTED
|
|
to this namespace plus allowedNamespaces -
|
|
and the router's namespace has to be in that
|
|
list.
|
|
|
|
Both branches are rendered from the same value the Service reads, so they cannot
|
|
disagree. The RCON and metrics ports are restricted in either case.
|
|
|
|
WHAT THIS OBJECT IS: a ceiling on reachability, not an authentication layer.
|
|
Only the ports declared below are reachable at all - whatever else the process
|
|
binds, now or after the next app update, is blocked without anyone having to
|
|
notice it exists.
|
|
|
|
WHAT IT IS NOT:
|
|
- EGRESS IS COARSE. A NetworkPolicy matches CIDRs and selectors and NEVER
|
|
hostnames, so "only to the database host" cannot be expressed at all. This
|
|
cluster's CNI is Canal (Flannel plus Calico for policy) and has no FQDN
|
|
rule; Calico's GlobalNetworkSet could express one but would bind this chart
|
|
to Calico, the same argument that kept implementation-specific objects out
|
|
of the Gateway-API sweep (#114). Each egress rule therefore permits its
|
|
PORT to any destination.
|
|
- NOT A BOUNDARY WHERE THE CNI DOES NOT ENFORCE ONE. An unenforced policy is
|
|
accepted by the API server and filters nothing, so everything reads as
|
|
configured while nothing is filtered.
|
|
- THE REASONING IS IN TEMPLATE COMMENTS, not in "#" comments: those are part
|
|
of the rendered object, travel into the cluster and appear in every release
|
|
diff. ci/run-networkpolicy-test.sh fails on a rendered "#".
|
|
|
|
THE PORT NUMBERS BELOW ARE POD PORTS, NEVER NODEPORTS. A NetworkPolicy is
|
|
evaluated at the pod's network interface, after kube-proxy has already
|
|
translated a NodePort to the targetPort, so a NodePort number here would match
|
|
nothing while looking strict - and, being the only rule for that port, would
|
|
block every inbound connection instead. ci/run-networkpolicy-test.sh asserts
|
|
every port here against the containerPorts of the pod spec this policy
|
|
selects.
|
|
*/ -}}
|
|
{{- if .Values.networkPolicy.enabled }}
|
|
{{- $np := .Values.networkPolicy }}
|
|
{{- $ns := .Release.Namespace }}
|
|
{{- $allowedNs := $np.allowedNamespaces | default list }}
|
|
{{- /* networkPolicy.ingress - the policy's INBOUND side, and not the same
|
|
thing as the top-level .Values.ingress (the Ingress object) - both match
|
|
a search for "ingress". */}}
|
|
{{- $npIngress := $np.ingress | default dict }}
|
|
{{- /* NULL-SAFE ON PURPOSE (#99): ambientMesh is an OBJECT, so every read goes
|
|
through a fallback reproducing THIS chart's declared defaults. An
|
|
operator who deletes the object, or writes only "enabled: true", must
|
|
land on the defaults and not in a nil dereference - #136 shipped exactly
|
|
that defect one value away, where the NOTES.txt branch dereferenced
|
|
.networkPolicy.egress directly. */}}
|
|
{{- $ambient := $npIngress.ambientMesh | default dict }}
|
|
{{- $ambientEnabled := $ambient.enabled }}
|
|
{{- $ambientPort := $ambient.port | default 15008 }}
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
|
labels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
|
spec:
|
|
{{- /*
|
|
Selects the StatefulSet's pods. ci/run-networkpolicy-test.sh fails if
|
|
this matches no pod spec of the same render - which is the mistake that
|
|
would otherwise leave a policy in the namespace filtering nothing, and
|
|
its usual cause is a label renamed in the workload template.
|
|
*/}}
|
|
podSelector:
|
|
matchLabels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
|
policyTypes:
|
|
- Ingress
|
|
{{- /*
|
|
Egress is listed because there is something real to restrict: a Minecraft
|
|
server needs DNS and HTTPS and nothing else in the normal case. Mod packs that
|
|
call out on other ports are the exception, and extraPorts is for them.
|
|
*/}}
|
|
- Egress
|
|
ingress:
|
|
{{- if eq .Values.server.service.type "NodePort" }}
|
|
{{- /*
|
|
THE GAME PORT, NODEPORT BRANCH (server.service.type is NodePort). Open with no
|
|
"from": the senders are Minecraft clients on the internet and, after NodePort
|
|
masquerading, the source address is a cluster node anyway.
|
|
|
|
THE NUMBER IS THE POD PORT (server.port), not server.service.outwardPort. A
|
|
NetworkPolicy is evaluated after kube-proxy has translated the NodePort, so the
|
|
NodePort number would match nothing here while looking strict - and being the
|
|
only rule for the port, it would block every player instead.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: {{ .Values.server.port | int64 }}
|
|
{{- if $ambientEnabled }}
|
|
{{- /* networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP is ztunnel's
|
|
HBONE tunnel, the port every inbound connection arrives on once the
|
|
namespace carries istio.io/dataplane-mode - ambient "does not and will
|
|
never bypass Kubernetes NetworkPolicy enforcement", so without it a pod
|
|
in the mesh is unreachable.
|
|
|
|
It is the one port here that is NOT a containerPort of the selected pod
|
|
spec: ztunnel terminates it, not the application. ci/run-networkpolicy-
|
|
test.sh holds a NAMED exception for exactly this number, gated on exactly
|
|
this value, and proves on a second render that it stays the only such
|
|
port and that it is absent while the value is false.
|
|
|
|
THE PORT HALF ONLY, and that is not an omission. Elsewhere this switch
|
|
also adds 169.254.7.127/32 as a probe source, because inside the mesh the
|
|
kubelet's probes are redirected away from the worker node's address.
|
|
This chart offers no networkPolicy.probeSources at all, so the rule
|
|
carrying the probe port is never restricted by source - and an
|
|
unrestricted rule needs no probe address. Measured in the render: three
|
|
exec probes, which put no packet on the network at all.
|
|
WHICH RULE IT JOINS:
|
|
the game-port rule, in BOTH of its branches. That rule is open when the
|
|
Service is a NodePort (players arrive from outside, already masqueraded)
|
|
and restricted to the namespace otherwise, and the two branches are
|
|
separate rules in this template - so the port is added twice, or it
|
|
would be missing from whichever branch a release happens to render.
|
|
AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with
|
|
IANA for this, no protocol prescribes it, and Kubernetes knows
|
|
nothing about it - Istio picked it for HBONE and Istio can change
|
|
it. Hence a VALUE with a documented default rather than a literal
|
|
here, so an operator can correct it without a chart release. Read
|
|
for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh
|
|
grants is pinned to the number THAT TEST names, plus a check that
|
|
this chart declares the same default - never to whatever the value
|
|
says, because that would let a mistyped port ride in through a
|
|
configurable field (design A of #143).
|
|
|
|
NOT MEASURED: that 15008 alone suffices comes from Istio's
|
|
documentation. Measured on this cluster is only which listeners ztunnel
|
|
opens. Provable at the first namespace to join the mesh; none has. */}}
|
|
- protocol: TCP
|
|
port: {{ int $ambientPort }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- if not (eq .Values.server.service.type "NodePort") }}
|
|
{{- /*
|
|
THE GAME PORT, CLUSTER-INTERNAL BRANCH (the default): the Service is not a
|
|
NodePort, so nothing outside the cluster can reach this port through this
|
|
chart, and the rule is restricted to this namespace plus allowedNamespaces.
|
|
|
|
PUT THE ROUTER'S NAMESPACE IN allowedNamespaces. In this cluster players reach
|
|
the server through minecraft-router, which runs in its own namespace - so
|
|
without that entry the server is unreachable and the symptom is a connection
|
|
timeout in a game client, with a healthy pod behind it.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: {{ .Values.server.port | int64 }}
|
|
{{- if $ambientEnabled }}
|
|
{{- /* networkPolicy.ingress.ambientMesh (Gitea #143). 15008/TCP is ztunnel's
|
|
HBONE tunnel, the port every inbound connection arrives on once the
|
|
namespace carries istio.io/dataplane-mode - ambient "does not and will
|
|
never bypass Kubernetes NetworkPolicy enforcement", so without it a pod
|
|
in the mesh is unreachable.
|
|
|
|
It is the one port here that is NOT a containerPort of the selected pod
|
|
spec: ztunnel terminates it, not the application. ci/run-networkpolicy-
|
|
test.sh holds a NAMED exception for exactly this number, gated on exactly
|
|
this value, and proves on a second render that it stays the only such
|
|
port and that it is absent while the value is false.
|
|
|
|
THE PORT HALF ONLY, and that is not an omission. Elsewhere this switch
|
|
also adds 169.254.7.127/32 as a probe source, because inside the mesh the
|
|
kubelet's probes are redirected away from the worker node's address.
|
|
This chart offers no networkPolicy.probeSources at all, so the rule
|
|
carrying the probe port is never restricted by source - and an
|
|
unrestricted rule needs no probe address. Measured in the render: three
|
|
exec probes, which put no packet on the network at all.
|
|
WHICH RULE IT JOINS:
|
|
the game-port rule, in BOTH of its branches. That rule is open when the
|
|
Service is a NodePort (players arrive from outside, already masqueraded)
|
|
and restricted to the namespace otherwise, and the two branches are
|
|
separate rules in this template - so the port is added twice, or it
|
|
would be missing from whichever branch a release happens to render.
|
|
AN ISTIO CONSTANT, NOT A STANDARD: 15008 is not registered with
|
|
IANA for this, no protocol prescribes it, and Kubernetes knows
|
|
nothing about it - Istio picked it for HBONE and Istio can change
|
|
it. Hence a VALUE with a documented default rather than a literal
|
|
here, so an operator can correct it without a chart release. Read
|
|
for ztunnel 1.30.6. The exception ci/run-networkpolicy-test.sh
|
|
grants is pinned to the number THAT TEST names, plus a check that
|
|
this chart declares the same default - never to whatever the value
|
|
says, because that would let a mistyped port ride in through a
|
|
configurable field (design A of #143).
|
|
|
|
NOT MEASURED: that 15008 alone suffices comes from Istio's
|
|
documentation. Measured on this cluster is only which listeners ztunnel
|
|
opens. Provable at the first namespace to join the mesh; none has. */}}
|
|
- protocol: TCP
|
|
port: {{ int $ambientPort }}
|
|
{{- end }}
|
|
from:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
{{- /*
|
|
Set by Kubernetes itself on every namespace (>= 1.21), so this needs no
|
|
labelling work on a namespace this chart does not own.
|
|
*/}}
|
|
kubernetes.io/metadata.name: {{ $ns }}
|
|
{{- range $allowedNs }}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ . | quote }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- if .Values.server.rcon.enabled }}
|
|
{{- /*
|
|
RCON - remote console, i.e. full administrative control of the server. No
|
|
Service and no probe, so it is restricted to this namespace plus
|
|
allowedNamespaces: the rule with the most security value in this chart, and it
|
|
costs nothing.
|
|
|
|
GATED ON server.rcon.enabled, AND THAT GATE IS NOT COSMETIC. The containerPort
|
|
itself is rendered only when RCON is on (see the StatefulSet), so without the
|
|
same condition here the policy would name a port the pod does not declare -
|
|
which is not merely untidy: ci/run-networkpolicy-test.sh rejects it, and it
|
|
rejected exactly this while the chart was being written. A rule for a port
|
|
nothing listens on is a rule nobody can verify.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: {{ .Values.server.rcon.port | int64 }}
|
|
from:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
{{- /*
|
|
Set by Kubernetes itself on every namespace (>= 1.21), so this needs no
|
|
labelling work on a namespace this chart does not own.
|
|
*/}}
|
|
kubernetes.io/metadata.name: {{ $ns }}
|
|
{{- range $allowedNs }}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ . | quote }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- /*
|
|
The metrics port. No Service, no probe, so restricted like RCON. A Prometheus
|
|
that scrapes this release from another namespace needs that namespace in
|
|
allowedNamespaces - and the failure is a target that silently stops reporting,
|
|
which is worth knowing before it happens.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 19565
|
|
from:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
{{- /*
|
|
Set by Kubernetes itself on every namespace (>= 1.21), so this needs no
|
|
labelling work on a namespace this chart does not own.
|
|
*/}}
|
|
kubernetes.io/metadata.name: {{ $ns }}
|
|
{{- range $allowedNs }}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ . | quote }}
|
|
{{- end }}
|
|
egress:
|
|
{{- /*
|
|
DNS, unconditionally. There is no value to switch this off, and that is
|
|
deliberate: a knob whose only other setting breaks every name lookup in
|
|
the pod is a trap and not a feature.
|
|
*/}}
|
|
- ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
{{- /*
|
|
Outbound HTTPS. Needed for Mojang's session server when online-mode is on - a
|
|
server that cannot reach it rejects every login - and for resource-pack and mod
|
|
downloads. To any destination, because those are hostnames.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 443
|
|
{{- with $np.egress }}
|
|
{{- with .extraPorts }}
|
|
{{- /*
|
|
Ports only, to any destination - see the header. This is the operator's
|
|
escape hatch from the egress cap, and the only alternative to switching
|
|
the whole policy off.
|
|
*/}}
|
|
- ports:
|
|
{{- range . }}
|
|
- protocol: TCP
|
|
port: {{ int . }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|