diff --git a/template-chart/templates/NOTES.txt b/template-chart/templates/NOTES.txt index 4541cee..0993867 100644 --- a/template-chart/templates/NOTES.txt +++ b/template-chart/templates/NOTES.txt @@ -182,6 +182,57 @@ THIS CHART IS A SCAFFOLD — IT IS NOT MEANT TO RUN "{{ .Release.Name }}-{{ .Chart.Name }}-config-map" while the ConfigMap this chart creates is "{{ .Release.Name }}-{{ .Chart.Name }}-configmap": replace the whole block with your app's real variables rather than repairing the example. +{{- if .Values.networkPolicy.enabled }} + +NETWORKPOLICY IS ACTIVE — WHAT IT CAPS, AND WHAT IT DOES NOT + {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy selects this workload's pods. Only the + container ports this chart declares are reachable at all; anything else the + process binds is blocked without anyone having to notice it exists. Outbound + traffic is capped at DNS{{ with .Values.networkPolicy.egress }}{{ with .extraPorts }} plus TCP {{ join ", " . }}{{ end }}{{ end }}. + + Three limits belong in the same breath, because none of them is visible in + the object: + + - EGRESS IS BY PORT, NEVER BY DESTINATION. A NetworkPolicy matches CIDRs and + selectors and cannot express a hostname, so every rule above permits its + port to ANY address. This is a limit of the object, not a shortcut — and + with this cluster's CNI (Canal: Flannel plus Calico) there is no FQDN rule + to reach for. An outbound port the chart does not know about is blocked: + add it to networkPolicy.egress.extraPorts, and expect to find out through + an application-level connection error rather than through this object. +{{- if .Values.networkPolicy.probeSources }} + - THE PROBE PATH IS RESTRICTED in this release, because + networkPolicy.probeSources is set to + {{ join ", " .Values.networkPolicy.probeSources }} + Probes come from the IP of the WORKER NODE running the pod. If none of + those CIDRs covers the node this pod lands on, the kubelet's probes are + dropped and THE POD NEVER BECOMES READY — the rollout then times out and + the symptom looks like a broken application. Check with + + kubectl -n {{ .Release.Namespace }} describe pod -l app={{ .Release.Name }}-{{ .Chart.Name }}-deployment + + and compare "Readiness probe failed" against `kubectl get node -o wide`. +{{- else }} + - THE PROBED PORT IS OPEN INSIDE THE CLUSTER. networkPolicy.probeSources is + empty, so the rule for that port carries no source selector at all: any + pod the CNI routes to can reach it. Not from the internet — the port has no + NodePort — but this is NOT pod-level isolation. Narrowing it means listing + the worker node IPs in networkPolicy.probeSources, because the kubelet + probes from the node's own address and the chart cannot know it; a wrong + guess there leaves the pod permanently un-Ready. +{{- end }} + - IT IS ONLY A BOUNDARY WHERE THE CNI ENFORCES POLICIES. Where it does not, + this object is accepted and filters nothing, so everything reads as + configured while nothing is filtered. Prove it rather than assume it: + + kubectl -n {{ .Release.Namespace }} get networkpolicy {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy + + shows the object exists; only a connection attempt from an unrelated + namespace shows that it bites. + + Switch it off with networkPolicy.enabled: false{{ if .Values.networkPolicy.allowedNamespaces }}; the namespaces currently + allowed in addition to {{ .Release.Namespace }} are {{ join ", " .Values.networkPolicy.allowedNamespaces }}{{ end }}. +{{- end }} {{- if .Values.ingress.enabled }} {{- with .Values.ingress.annotations }} diff --git a/template-chart/templates/xxx.networkpolicy.yaml b/template-chart/templates/xxx.networkpolicy.yaml new file mode 100644 index 0000000..512df6c --- /dev/null +++ b/template-chart/templates/xxx.networkpolicy.yaml @@ -0,0 +1,169 @@ +{{- /* +NetworkPolicy — the reference implementation for Gitea #5. + +Copy this file with the rest of template-chart and replace every xxx: the +values paths (.Values.networkPolicy.*), the helper prefix and the podSelector +label, exactly as in the other template files. + +EVERY COMMENT IN THIS FILE IS A TEMPLATE COMMENT, and that is not a style +choice. A "#" comment is part of the RENDERED manifest: it travels into the +cluster, shows up in `helm get manifest` and `kubectl get -o yaml`, and bloats +every diff a reviewer reads. Reasoning belongs in a Go template comment, which +renders nothing. This caught several rounds of this rollout. + +And a second-order trap found while writing this header: the prose inside a +template comment must not spell out the comment's own closing delimiter. The +parser ends the comment at the first occurrence, and the remaining prose is +then emitted as manifest content - the render fails with "did not find +expected node content", pointing at the template and not at the sentence. +Describe the delimiters instead of quoting them. + +WHAT THIS OBJECT DOES, AND - MORE IMPORTANTLY - WHAT IT DOES NOT + +It is a CEILING ON REACHABILITY, not an authentication layer. Three limits are +real and worth having: + + 1. ONLY THE DECLARED PORTS ARE REACHABLE AT ALL. Whatever else the process + binds - a debug listener, a management interface the image opens by + default, a port that appears in the next app version - is blocked without + anyone having to notice it exists. This is the largest part of what a + default-on policy buys in this repository. + 2. EGRESS IS CAPPED BY PORT, so a compromised pod cannot open arbitrary + outbound connections. + 3. INGRESS IS CAPPED BY SOURCE for every port no network probe touches. + +And three things it does NOT do, each of which has to be written down per +chart rather than quietly assumed: + + a. EGRESS IS COARSE. A NetworkPolicy matches CIDRs and selectors and NEVER + hostnames, so "only to the database host" cannot be expressed. This + cluster's CNI is Canal (Flannel plus Calico for policy), which has no + FQDN rule; Calico's GlobalNetworkSet could express one but would bind + every chart to Calico - the same argument that kept implementation- + specific objects out of the Gateway-API sweep (Gitea #114). So each + egress rule below permits its PORT to any destination. That is a limit of + the object, not a shortcut. + b. THE PROBED PORT IS OPEN INSIDE THE CLUSTER BY DEFAULT. See the long note + on networkPolicy.probeSources in values.yaml. Narrowing it needs the + worker node IPs, which this chart cannot know. + c. A POLICY IS ONLY A BOUNDARY WHERE THE CNI ENFORCES ONE. An unenforced + policy is accepted by the API server and filters nothing, so everything + looks configured while nothing is filtered. Measured for this cluster + (Canal, 6/6) and for the CI cluster (k3d, which brings k3s' own policy + controller) - verify it rather than reading it off a value. + +THE PORT NUMBERS HERE ARE POD PORTS, NEVER NODEPORTS. A NetworkPolicy is +evaluated at the POD's network interface, after kube-proxy has already +translated a NodePort to the targetPort. Writing 30080 where the pod listens +on 8080 matches nothing: the object is accepted, looks strict, and blocks +every inbound connection instead. ci/run-networkpolicy-test.sh asserts this +mechanically - every port named here must exist as a containerPort of the pod +spec this policy selects. +*/ -}} +{{- if .Values.networkPolicy.enabled }} +{{- $np := .Values.networkPolicy }} +{{- $probeSources := $np.probeSources | default list }} +{{- $allowedNs := $np.allowedNamespaces | default list }} +{{- $ic := $np.ingressController | default dict }} +{{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }} +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy + labels: + app: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy +spec: + {{- /* Selects the Deployment's pods. A chart that keeps the StatefulSet + instead has to move this selector along with the Service's, and + ci/run-networkpolicy-test.sh fails if it selects no pod spec of the + same render - which is the mistake that would otherwise leave a + NetworkPolicy in the namespace filtering nothing at all. */}} + podSelector: + matchLabels: + app: {{ .Release.Name }}-{{ .Chart.Name }}-deployment + policyTypes: + - Ingress + {{- /* Egress is listed because there IS something to restrict here: this + workload needs DNS and nothing else by default. A chart whose + workload opens no outbound connection of its own must NOT list + Egress - valkey is the example - because an egress rule that + constrains nothing makes the object look stricter than it is. */}} + - Egress + ingress: + {{- /* The http port. It is the port all three probes address, so it is + the probe path, and the shape below is the whole reason + networkPolicy.probeSources exists. */}} + - ports: + {{- range .Values.xxx.ports }} + - protocol: {{ .protocol | default "TCP" }} + port: {{ .containerPort }} + {{- end }} + {{- if $probeSources }} + {{- /* RESTRICTED, because the operator supplied probeSources. The + kubelet's probes come from the IP of the WORKER NODE running the + pod, so that address has to be in the list or the pod never + becomes Ready - and a pod that is never Ready reads as a broken + application rather than as a policy. The ingress controller is + added here too, or the Ingress goes dark the moment this list is + filled. */}} + from: + - namespaceSelector: + matchLabels: + {{- /* Set by Kubernetes itself on every namespace (>= 1.21), so + this needs no labelling work on namespaces this chart does + not own. */}} + kubernetes.io/metadata.name: {{ .Release.Namespace }} + {{- if and $routed $ic.namespace }} + {{- /* ONE from-entry with TWO selectors, which is an AND: pods + carrying these labels IN that namespace. Written as two list + entries it would be an OR and would open the port to the whole + controller namespace - kube-system here, i.e. to everything in + the cluster's own namespace. */}} + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: {{ $ic.namespace | quote }} + {{- with $ic.podLabels }} + podSelector: + matchLabels: + {{- toYaml . | nindent 14 }} + {{- end }} + {{- end }} + {{- range $allowedNs }} + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: {{ . | quote }} + {{- end }} + {{- range $probeSources }} + - ipBlock: + cidr: {{ . | quote }} + {{- end }} + {{- else }} + {{- /* NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only + inside the cluster, because this port carries no NodePort. The + alternative was to guess the worker node CIDRs, and a wrong guess + blocks every kubelet probe: the pod stays un-Ready, the rollout + times out, and the cause looks like the application. Note what an + EMPTY "from: []" would do instead: it also allows everything, but + while looking like a restriction. Hence no key at all, and + ci/run-networkpolicy-test.sh rejects the empty form. */}} + {{- end }} + egress: + {{- /* DNS, unconditionally. There is no value to switch this off, and + that is deliberate: a knob whose only other setting breaks every + name lookup in the pod is a trap and not a feature. */}} + - ports: + - protocol: UDP + port: 53 + - protocol: TCP + port: 53 + {{- with $np.egress }} + {{- with .extraPorts }} + {{- /* Ports only - see note (a) in the header. Any destination. */}} + - ports: + {{- range . }} + - protocol: TCP + port: {{ int . }} + {{- end }} + {{- end }} + {{- end }} +{{- end }} diff --git a/template-chart/values.schema.json b/template-chart/values.schema.json index c278ae2..fcbcdef 100644 --- a/template-chart/values.schema.json +++ b/template-chart/values.schema.json @@ -9,6 +9,57 @@ "description": "Helm's conventional cross-chart values. Unused by this chart, kept open so an umbrella install cannot break on it.", "type": "object" }, + "networkPolicy": { + "description": "The chart's NetworkPolicy (Gitea #5), on by default. A ceiling on reachability and not an authentication layer: only the declared container ports are reachable at all, egress is capped by PORT (a NetworkPolicy matches CIDRs and selectors, never hostnames, so a destination host cannot be expressed), and ingress is capped by source for every port no network probe touches. Only a boundary where the cluster's CNI enforces policies - an unenforced policy is accepted by the API server and filters nothing. The template's header carries the full list of what it does not do.", + "type": "object", + "additionalProperties": false, + "properties": { + "enabled": { + "type": "boolean" + }, + "allowedNamespaces": { + "description": "Namespaces whose pods may reach this workload, in addition to the release's own namespace, which is always allowed. Matched on the kubernetes.io/metadata.name label Kubernetes sets on every namespace itself, so no labelling work is needed - and no typo protection either: a name that does not exist matches nothing and is silently ineffective. An entry opens the port to EVERY pod in that namespace; for the ingress controller use ingressController instead.", + "type": ["array", "null"], + "items": { + "type": "string" + } + }, + "probeSources": { + "description": "Source CIDRs allowed to reach the port the network probes address. EMPTY MEANS OPEN, not blocked: with no entry that rule is rendered without a \"from\" selector. The kubelet's probes come from the IP of the worker node running the pod, an address the chart cannot know and must not guess - a guessed CIDR that excludes the real node blocks every probe and the pod never becomes Ready, which reads as a broken application rather than as a policy. Setting this restricts the rule, which is why the chart adds the own namespace, ingressController and allowedNamespaces to the same rule.", + "type": ["array", "null"], + "items": { + "type": "string" + } + }, + "ingressController": { + "description": "The ingress controller allowed to reach the routed port once probeSources restricts it; rendered only while ingress.enabled or httpRoute.enabled is true. The two keys become ONE \"from\" entry and are therefore an AND - pods carrying podLabels IN namespace - which is the whole point: the controller's namespace in allowedNamespaces would open the port to every pod in kube-system. The default is measured on this cluster (DaemonSet rke2-traefik in kube-system, which also serves the Gateway API here), not assumed; a policy written against the wrong controller is accepted, looks strict and silently blocks all web traffic.", + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "namespace": { + "type": ["string", "null"] + }, + "podLabels": { + "description": "Label set the controller's pods carry, rendered verbatim as a podSelector's matchLabels. Kept open because the chart does not interpret the keys.", + "type": ["object", "null"] + } + } + }, + "egress": { + "type": ["object", "null"], + "additionalProperties": false, + "properties": { + "extraPorts": { + "description": "Additional outbound TCP ports, to ANY destination - ports and not destinations, because a NetworkPolicy cannot express a hostname. DNS is always allowed and has no switch. Without an entry here an outbound port the chart does not already derive is blocked, which is the cap's whole purpose and also its upgrade risk.", + "type": ["array", "null"], + "items": { + "type": "integer" + } + } + } + } + } + }, "scaleDown": { "type": "boolean" }, diff --git a/template-chart/values.yaml b/template-chart/values.yaml index f40b321..85ef0de 100644 --- a/template-chart/values.yaml +++ b/template-chart/values.yaml @@ -174,3 +174,120 @@ httpRoute: # is how a migration from the Ingress runs: the HTTPRoute takes a second # hostname while the Ingress keeps ingress.domain. hostnames: [] + +# --------------------------------------------------------------------------- +# NetworkPolicy (Gitea #5) — the reference block. Copy it with the template. +# --------------------------------------------------------------------------- +networkPolicy: + # ON BY DEFAULT, which is the decision this whole rollout rests on: a + # NetworkPolicy that ships switched off is reached only when somebody + # remembers to switch it on, and then the issue is closed while nothing is + # filtered. + # + # READ THIS BEFORE RELYING ON IT: a NetworkPolicy is only a boundary if the + # cluster's CNI enforces policies. Where it does not, the API server still + # accepts the object and everything looks configured while nothing is + # filtered - an unenforced policy is a false sense of security, not a weak + # one. Verify on the cluster (try to connect from an unrelated namespace) + # rather than reading it off this value. + # + # WHAT IS ACTUALLY CAPPED, and what is not, is written out in the template's + # header - read it once before assuming this value means "the workload is + # isolated". In short: only the declared ports are reachable, egress is + # capped by PORT (never by host - the object cannot express a hostname), and + # the probed port stays open inside the cluster until probeSources is set. + enabled: true + + # Namespaces whose pods may reach this workload, in addition to the release's + # own namespace - which is always allowed. Empty means: own namespace only. + # + # Matched by the "kubernetes.io/metadata.name" label, which Kubernetes sets on + # every namespace itself (>= 1.21), so no labelling work is needed on a + # namespace this chart does not own - but also no typo protection: a + # namespace name that does not exist matches nothing and is silently + # ineffective. + # + # NOTE THE GRANULARITY: an entry here opens the port to EVERY pod in that + # namespace. For the ingress controller use ingressController below instead, + # which pins the source down to the controller's pods. + allowedNamespaces: [] + + # Source CIDRs allowed to reach the port the probes address. This is the + # single most dangerous value in this block, in both directions, so it is + # spelled out in full. + # + # EMPTY MEANS OPEN, NOT BLOCKED. With no entry the rule for that port is + # rendered WITHOUT a "from" selector, so the port is reachable from anywhere + # the CNI routes to - which, since this port carries no NodePort, means from + # inside the cluster and not from the internet. + # + # That is the deliberate choice between two bad defaults. The kubelet's + # liveness/readiness/startup probes originate from the IP of the WORKER NODE + # running the pod, an address this chart cannot know and must not guess: a + # guessed CIDR that excludes the real node addresses blocks every probe, the + # pod never becomes Ready, the rollout times out, and the symptom reads as a + # broken application rather than as a policy. Shipping "open" instead is + # honest about what it does. + # + # Narrow it by listing the worker node IPs (e.g. ["10.0.1.11/32", + # "10.0.1.12/32"]) — and mind the companion trap: once this list is + # non-empty the rule IS restricted, which also cuts off everything else that + # reaches this port. The chart adds the release's own namespace, the + # ingressController below and allowedNamespaces to the same rule for exactly + # that reason. + # + # A probe that is an "exec" probe is NOT affected by any of this: it runs + # inside the container through the kubelet's runtime and puts no packet on + # the network, which is why a chart with exec-only probes (valkey) can + # restrict its port by source with no probeSources at all. + probeSources: [] + + # The ingress controller allowed to reach the routed port once probeSources + # restricts it. Rendered ONLY while ingress.enabled or httpRoute.enabled is + # true, and only inside the restricted branch - with the default (open) rule + # the controller reaches the port anyway and naming it would be noise. + # + # The two keys are rendered as ONE "from" entry, which makes them an AND: + # pods carrying podLabels IN namespace. That is the point of having them + # here at all - putting the controller's namespace into allowedNamespaces + # instead would open the port to every pod in kube-system. + # + # THE DEFAULT IS MEASURED ON THIS CLUSTER, NOT ASSUMED (2026-10-14): + # ingress-nginx is gone (no DaemonSet, no pods), the only IngressClass is + # "traefik" and it is the default, and the controller is the DaemonSet + # rke2-traefik in kube-system with the pod label + # app.kubernetes.io/name=rke2-traefik. The same Traefik also serves the + # Gateway API here (GatewayClass "traefik", controller + # traefik.io/gateway-controller, no pods of its own in the Gateway's + # namespace), so this one source covers BOTH the Ingress and the HTTPRoute + # path. + # + # A policy written against the WRONG controller is the expensive mistake in + # this block: it is accepted, it looks strict, and it silently blocks all + # web traffic. Check the cluster before changing these two keys, and note + # that ingress.className says nothing about them - the class names the + # controller's implementation, not the pods its traffic comes from. + ingressController: + namespace: kube-system + podLabels: + app.kubernetes.io/name: rke2-traefik + + egress: + # Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and + # has no switch - a knob whose only other setting breaks every name lookup + # in the pod is a trap, not a feature. + # + # PORTS, NOT DESTINATIONS, and that is a limit of the object rather than + # laziness: a NetworkPolicy matches ipBlock CIDRs and pod/namespace + # selectors, never hostnames, so "only to the database host" cannot be + # written at all. Each entry here permits its port to ANY destination. + # What the cap still buys is real: a compromised pod cannot open arbitrary + # outbound connections. + # + # Add what the deployment actually needs: 443 for an outbound HTTPS call + # (an OIDC provider's discovery document, an S3 endpoint, an app that + # fetches), 5432 for Postgres, 3306 for MariaDB, 6379 for Valkey/Redis, + # 587 or 465 for SMTP submission. A chart that KNOWS its outbound port + # from its own values should derive it in the template instead of asking + # for it twice (stalwart does this with dataStore.postgres.port). + extraPorts: []