feat(template-chart): the reference NetworkPolicy, on by default
The reference implementation for Gitea #5: one NetworkPolicy per workload, networkPolicy.enabled defaulting to true, plus the values block, the schema entry and the NOTES.txt section that go with it. The shape is taken from the two charts that already had one - valkey and stalwart - rather than invented, because they carry the expensive lessons: policyTypes only where there is something to restrict, namespaces always spelled out because an empty "from" allows everything, kubernetes.io/metadata.name so no foreign namespace needs labelling, POD ports and never NodePorts, egress by port because the object cannot express a hostname, and the probe path left open by default because the kubelet probes from the node's own IP. Three things are written down rather than assumed. The ingress source is measured on the cluster and is NOT what the issue body assumes: ingress-nginx is gone, the only IngressClass is traefik, and the controller is the DaemonSet rke2-traefik in kube-system - which also serves the Gateway API here, so one source covers both the Ingress and the HTTPRoute path. Egress stays coarse: with Canal (Flannel plus Calico) there is no FQDN rule, and Calico's GlobalNetworkSet would bind every chart to Calico, the same argument that kept implementation-specific objects out of #114. And a policy is only a boundary where the CNI enforces one - unenforced, it is accepted and filters nothing. All reasoning sits in Go template comments, not in "#" comments: those are part of the rendered object and travel into the cluster. Writing this header also found the second-order trap, which is now in the header itself - prose inside a template comment must not spell out the comment's own closing delimiter, or the parser ends the comment early and the remaining prose is emitted as manifest content. Part of #5
This commit is contained in:
4 files changed
+388
No files matched your search
@@ -182,6 +182,57 @@ THIS CHART IS A SCAFFOLD — IT IS NOT MEANT TO RUN
|
||||
"{{ .Release.Name }}-{{ .Chart.Name }}-config-map" while the ConfigMap this chart creates is
|
||||
"{{ .Release.Name }}-{{ .Chart.Name }}-configmap": replace the whole block with your app's real
|
||||
variables rather than repairing the example.
|
||||
{{- if .Values.networkPolicy.enabled }}
|
||||
|
||||
NETWORKPOLICY IS ACTIVE — WHAT IT CAPS, AND WHAT IT DOES NOT
|
||||
{{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy selects this workload's pods. Only the
|
||||
container ports this chart declares are reachable at all; anything else the
|
||||
process binds is blocked without anyone having to notice it exists. Outbound
|
||||
traffic is capped at DNS{{ with .Values.networkPolicy.egress }}{{ with .extraPorts }} plus TCP {{ join ", " . }}{{ end }}{{ end }}.
|
||||
|
||||
Three limits belong in the same breath, because none of them is visible in
|
||||
the object:
|
||||
|
||||
- EGRESS IS BY PORT, NEVER BY DESTINATION. A NetworkPolicy matches CIDRs and
|
||||
selectors and cannot express a hostname, so every rule above permits its
|
||||
port to ANY address. This is a limit of the object, not a shortcut — and
|
||||
with this cluster's CNI (Canal: Flannel plus Calico) there is no FQDN rule
|
||||
to reach for. An outbound port the chart does not know about is blocked:
|
||||
add it to networkPolicy.egress.extraPorts, and expect to find out through
|
||||
an application-level connection error rather than through this object.
|
||||
{{- if .Values.networkPolicy.probeSources }}
|
||||
- THE PROBE PATH IS RESTRICTED in this release, because
|
||||
networkPolicy.probeSources is set to
|
||||
{{ join ", " .Values.networkPolicy.probeSources }}
|
||||
Probes come from the IP of the WORKER NODE running the pod. If none of
|
||||
those CIDRs covers the node this pod lands on, the kubelet's probes are
|
||||
dropped and THE POD NEVER BECOMES READY — the rollout then times out and
|
||||
the symptom looks like a broken application. Check with
|
||||
|
||||
kubectl -n {{ .Release.Namespace }} describe pod -l app={{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
||||
|
||||
and compare "Readiness probe failed" against `kubectl get node -o wide`.
|
||||
{{- else }}
|
||||
- THE PROBED PORT IS OPEN INSIDE THE CLUSTER. networkPolicy.probeSources is
|
||||
empty, so the rule for that port carries no source selector at all: any
|
||||
pod the CNI routes to can reach it. Not from the internet — the port has no
|
||||
NodePort — but this is NOT pod-level isolation. Narrowing it means listing
|
||||
the worker node IPs in networkPolicy.probeSources, because the kubelet
|
||||
probes from the node's own address and the chart cannot know it; a wrong
|
||||
guess there leaves the pod permanently un-Ready.
|
||||
{{- end }}
|
||||
- IT IS ONLY A BOUNDARY WHERE THE CNI ENFORCES POLICIES. Where it does not,
|
||||
this object is accepted and filters nothing, so everything reads as
|
||||
configured while nothing is filtered. Prove it rather than assume it:
|
||||
|
||||
kubectl -n {{ .Release.Namespace }} get networkpolicy {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
||||
|
||||
shows the object exists; only a connection attempt from an unrelated
|
||||
namespace shows that it bites.
|
||||
|
||||
Switch it off with networkPolicy.enabled: false{{ if .Values.networkPolicy.allowedNamespaces }}; the namespaces currently
|
||||
allowed in addition to {{ .Release.Namespace }} are {{ join ", " .Values.networkPolicy.allowedNamespaces }}{{ end }}.
|
||||
{{- end }}
|
||||
{{- if .Values.ingress.enabled }}
|
||||
{{- with .Values.ingress.annotations }}
|
||||
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
{{- /*
|
||||
NetworkPolicy — the reference implementation for Gitea #5.
|
||||
|
||||
Copy this file with the rest of template-chart and replace every xxx: the
|
||||
values paths (.Values.networkPolicy.*), the helper prefix and the podSelector
|
||||
label, exactly as in the other template files.
|
||||
|
||||
EVERY COMMENT IN THIS FILE IS A TEMPLATE COMMENT, and that is not a style
|
||||
choice. A "#" comment is part of the RENDERED manifest: it travels into the
|
||||
cluster, shows up in `helm get manifest` and `kubectl get -o yaml`, and bloats
|
||||
every diff a reviewer reads. Reasoning belongs in a Go template comment, which
|
||||
renders nothing. This caught several rounds of this rollout.
|
||||
|
||||
And a second-order trap found while writing this header: the prose inside a
|
||||
template comment must not spell out the comment's own closing delimiter. The
|
||||
parser ends the comment at the first occurrence, and the remaining prose is
|
||||
then emitted as manifest content - the render fails with "did not find
|
||||
expected node content", pointing at the template and not at the sentence.
|
||||
Describe the delimiters instead of quoting them.
|
||||
|
||||
WHAT THIS OBJECT DOES, AND - MORE IMPORTANTLY - WHAT IT DOES NOT
|
||||
|
||||
It is a CEILING ON REACHABILITY, not an authentication layer. Three limits are
|
||||
real and worth having:
|
||||
|
||||
1. ONLY THE DECLARED PORTS ARE REACHABLE AT ALL. Whatever else the process
|
||||
binds - a debug listener, a management interface the image opens by
|
||||
default, a port that appears in the next app version - is blocked without
|
||||
anyone having to notice it exists. This is the largest part of what a
|
||||
default-on policy buys in this repository.
|
||||
2. EGRESS IS CAPPED BY PORT, so a compromised pod cannot open arbitrary
|
||||
outbound connections.
|
||||
3. INGRESS IS CAPPED BY SOURCE for every port no network probe touches.
|
||||
|
||||
And three things it does NOT do, each of which has to be written down per
|
||||
chart rather than quietly assumed:
|
||||
|
||||
a. EGRESS IS COARSE. A NetworkPolicy matches CIDRs and selectors and NEVER
|
||||
hostnames, so "only to the database host" cannot be expressed. This
|
||||
cluster's CNI is Canal (Flannel plus Calico for policy), which has no
|
||||
FQDN rule; Calico's GlobalNetworkSet could express one but would bind
|
||||
every chart to Calico - the same argument that kept implementation-
|
||||
specific objects out of the Gateway-API sweep (Gitea #114). So each
|
||||
egress rule below permits its PORT to any destination. That is a limit of
|
||||
the object, not a shortcut.
|
||||
b. THE PROBED PORT IS OPEN INSIDE THE CLUSTER BY DEFAULT. See the long note
|
||||
on networkPolicy.probeSources in values.yaml. Narrowing it needs the
|
||||
worker node IPs, which this chart cannot know.
|
||||
c. A POLICY IS ONLY A BOUNDARY WHERE THE CNI ENFORCES ONE. An unenforced
|
||||
policy is accepted by the API server and filters nothing, so everything
|
||||
looks configured while nothing is filtered. Measured for this cluster
|
||||
(Canal, 6/6) and for the CI cluster (k3d, which brings k3s' own policy
|
||||
controller) - verify it rather than reading it off a value.
|
||||
|
||||
THE PORT NUMBERS HERE ARE POD PORTS, NEVER NODEPORTS. A NetworkPolicy is
|
||||
evaluated at the POD's network interface, after kube-proxy has already
|
||||
translated a NodePort to the targetPort. Writing 30080 where the pod listens
|
||||
on 8080 matches nothing: the object is accepted, looks strict, and blocks
|
||||
every inbound connection instead. ci/run-networkpolicy-test.sh asserts this
|
||||
mechanically - every port named here must exist as a containerPort of the pod
|
||||
spec this policy selects.
|
||||
*/ -}}
|
||||
{{- if .Values.networkPolicy.enabled }}
|
||||
{{- $np := .Values.networkPolicy }}
|
||||
{{- $probeSources := $np.probeSources | default list }}
|
||||
{{- $allowedNs := $np.allowedNamespaces | default list }}
|
||||
{{- $ic := $np.ingressController | default dict }}
|
||||
{{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }}
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
||||
labels:
|
||||
app: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
||||
spec:
|
||||
{{- /* Selects the Deployment's pods. A chart that keeps the StatefulSet
|
||||
instead has to move this selector along with the Service's, and
|
||||
ci/run-networkpolicy-test.sh fails if it selects no pod spec of the
|
||||
same render - which is the mistake that would otherwise leave a
|
||||
NetworkPolicy in the namespace filtering nothing at all. */}}
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment
|
||||
policyTypes:
|
||||
- Ingress
|
||||
{{- /* Egress is listed because there IS something to restrict here: this
|
||||
workload needs DNS and nothing else by default. A chart whose
|
||||
workload opens no outbound connection of its own must NOT list
|
||||
Egress - valkey is the example - because an egress rule that
|
||||
constrains nothing makes the object look stricter than it is. */}}
|
||||
- Egress
|
||||
ingress:
|
||||
{{- /* The http port. It is the port all three probes address, so it is
|
||||
the probe path, and the shape below is the whole reason
|
||||
networkPolicy.probeSources exists. */}}
|
||||
- ports:
|
||||
{{- range .Values.xxx.ports }}
|
||||
- protocol: {{ .protocol | default "TCP" }}
|
||||
port: {{ .containerPort }}
|
||||
{{- end }}
|
||||
{{- if $probeSources }}
|
||||
{{- /* RESTRICTED, because the operator supplied probeSources. The
|
||||
kubelet's probes come from the IP of the WORKER NODE running the
|
||||
pod, so that address has to be in the list or the pod never
|
||||
becomes Ready - and a pod that is never Ready reads as a broken
|
||||
application rather than as a policy. The ingress controller is
|
||||
added here too, or the Ingress goes dark the moment this list is
|
||||
filled. */}}
|
||||
from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
{{- /* Set by Kubernetes itself on every namespace (>= 1.21), so
|
||||
this needs no labelling work on namespaces this chart does
|
||||
not own. */}}
|
||||
kubernetes.io/metadata.name: {{ .Release.Namespace }}
|
||||
{{- if and $routed $ic.namespace }}
|
||||
{{- /* ONE from-entry with TWO selectors, which is an AND: pods
|
||||
carrying these labels IN that namespace. Written as two list
|
||||
entries it would be an OR and would open the port to the whole
|
||||
controller namespace - kube-system here, i.e. to everything in
|
||||
the cluster's own namespace. */}}
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: {{ $ic.namespace | quote }}
|
||||
{{- with $ic.podLabels }}
|
||||
podSelector:
|
||||
matchLabels:
|
||||
{{- toYaml . | nindent 14 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- range $allowedNs }}
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: {{ . | quote }}
|
||||
{{- end }}
|
||||
{{- range $probeSources }}
|
||||
- ipBlock:
|
||||
cidr: {{ . | quote }}
|
||||
{{- end }}
|
||||
{{- else }}
|
||||
{{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only
|
||||
inside the cluster, because this port carries no NodePort. The
|
||||
alternative was to guess the worker node CIDRs, and a wrong guess
|
||||
blocks every kubelet probe: the pod stays un-Ready, the rollout
|
||||
times out, and the cause looks like the application. Note what an
|
||||
EMPTY "from: []" would do instead: it also allows everything, but
|
||||
while looking like a restriction. Hence no key at all, and
|
||||
ci/run-networkpolicy-test.sh rejects the empty form. */}}
|
||||
{{- end }}
|
||||
egress:
|
||||
{{- /* DNS, unconditionally. There is no value to switch this off, and
|
||||
that is deliberate: a knob whose only other setting breaks every
|
||||
name lookup in the pod is a trap and not a feature. */}}
|
||||
- ports:
|
||||
- protocol: UDP
|
||||
port: 53
|
||||
- protocol: TCP
|
||||
port: 53
|
||||
{{- with $np.egress }}
|
||||
{{- with .extraPorts }}
|
||||
{{- /* Ports only - see note (a) in the header. Any destination. */}}
|
||||
- ports:
|
||||
{{- range . }}
|
||||
- protocol: TCP
|
||||
port: {{ int . }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
Reference in new issue
Block a user