feat(outline)!: NetworkPolicy, on by default
Major, as every chart in this rollout is. Inbound 3000 only; outbound DNS plus 5432, 6379 and 443. Those three egress ports are the STANDARD ones and not the configured ones, which is this chart's real limitation and is written into values.yaml, the template header and NOTES.txt rather than left to be discovered: Outline's database, Redis and OIDC provider all arrive as connection URLs inside 1Password items that the chart never parses, so it cannot know where they listen and cannot validate a guess. A dependency on another port is blocked, and because all three probes hit /_health - which checks Postgres and Redis - the symptom is a pod that never becomes Ready rather than an error anyone reads. networkPolicy.egress.extraPorts is the escape hatch. That same property is what makes this chart the right place for the enforcement experiment in the next commit. Also reverts the cyberchef fixture from the first experiment: with the probe port restricted to an address that is not the node's, the install-test went GREEN, so the claim that it catches a blocked probe path does not hold as written. The next commit establishes whether policies are enforced in CI at all. Part of #5
This commit is contained in:
6 files changed
+385
-25
No files matched your search
@@ -235,3 +235,57 @@ FORCE_HTTPS IS ON
|
||||
header. Turned on, Outline redirects to https itself — which loops
|
||||
endlessly if that header does not reach the pod.
|
||||
{{- end }}
|
||||
{{- if .Values.networkPolicy.enabled }}
|
||||
|
||||
NETWORKPOLICY IS ACTIVE — WHAT IT CAPS, AND WHAT IT DOES NOT
|
||||
{{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy selects this workload's pods.
|
||||
Inbound: TCP 3000 only. Outbound: DNS, TCP 5432 (Postgres), 6379 (Redis),
|
||||
443 (OIDC discovery){{ with .Values.networkPolicy.egress }}{{ with .extraPorts }}, plus TCP {{ join ", " . }}{{ end }}{{ end }}.
|
||||
|
||||
THOSE EGRESS PORTS ARE THE STANDARD ONES, NOT YOUR CONFIGURED ONES. The
|
||||
database, Redis and OIDC URLs all live inside 1Password items that this chart
|
||||
never parses, so it cannot know where your dependencies listen. A dependency
|
||||
on another port is blocked — and since all three probes hit /_health, which
|
||||
checks Postgres and Redis, the symptom is a pod that never becomes Ready
|
||||
rather than a connection error anyone sees. Add the real port to
|
||||
networkPolicy.egress.extraPorts.
|
||||
|
||||
Three limits belong in the same breath, because none of them is visible in
|
||||
the object itself:
|
||||
|
||||
- EGRESS IS BY PORT, NEVER BY DESTINATION. A NetworkPolicy matches CIDRs and
|
||||
selectors and cannot express a hostname, so every egress rule permits its
|
||||
port to ANY address. That is a limit of the object, not a shortcut — this
|
||||
cluster's CNI (Canal: Flannel plus Calico) has no FQDN rule either. An
|
||||
outbound port the chart does not know about is BLOCKED: add it to
|
||||
networkPolicy.egress.extraPorts, and expect to find out through an
|
||||
application-level connection error rather than through this object.
|
||||
{{- if .Values.networkPolicy.probeSources }}
|
||||
- THE PROBE PATH IS RESTRICTED in this release, because
|
||||
networkPolicy.probeSources is set to
|
||||
{{ join ", " .Values.networkPolicy.probeSources }}
|
||||
Probes come from the IP of the WORKER NODE running the pod. If none of those
|
||||
CIDRs covers the node this pod lands on, the kubelet's probes are dropped
|
||||
and THE POD NEVER BECOMES READY — the rollout then times out and the symptom
|
||||
looks like a broken application. Compare
|
||||
|
||||
kubectl -n {{ .Release.Namespace }} describe pod -l app={{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
||||
|
||||
against `kubectl get node -o wide`.
|
||||
{{- else }}
|
||||
- THE PROBED PORT IS OPEN INSIDE THE CLUSTER. networkPolicy.probeSources is
|
||||
empty, so that rule carries no source selector at all and any pod the CNI
|
||||
routes to can reach it. Not from the internet — but this is NOT pod-level
|
||||
isolation. Narrowing it means listing the worker node IPs in
|
||||
networkPolicy.probeSources, because the kubelet probes from the node's own
|
||||
address and the chart cannot know it; a wrong guess there leaves the pod
|
||||
permanently un-Ready.
|
||||
{{- end }}
|
||||
- IT IS ONLY A BOUNDARY WHERE THE CNI ENFORCES POLICIES. Where it does not,
|
||||
this object is accepted and filters nothing, so everything reads as
|
||||
configured while nothing is filtered. `kubectl -n {{ .Release.Namespace }} get networkpolicy`
|
||||
shows the object exists; only a connection attempt from an unrelated
|
||||
namespace shows that it bites.
|
||||
|
||||
Switch it off with networkPolicy.enabled: false.
|
||||
{{- end }}
|
||||
@@ -0,0 +1,192 @@
|
||||
{{- /*
|
||||
NetworkPolicy for outline (Gitea #5).
|
||||
|
||||
Outline is the first chart of this rollout whose policy has real work to do in
|
||||
BOTH directions, and it is also the one that proves the egress cap bites: all
|
||||
three of its probes hit /_health, which checks Postgres AND Redis, so a pod
|
||||
that cannot reach its data stores never becomes Ready.
|
||||
|
||||
THE EGRESS PORTS ARE THE CHART'S ONLY KNOWLEDGE OF ITS DEPENDENCIES, AND THEY
|
||||
ARE A GUESS AT THE STANDARD ONES. Outline's database, Redis and OIDC provider
|
||||
are configured through connection URLs inside 1Password items - the chart never
|
||||
sees a host or a port, cannot derive one, and cannot validate one. So the rules
|
||||
below name the standard ports (5432, 6379, 443) and nothing else. A deployment
|
||||
whose Postgres listens anywhere else is BLOCKED, and it finds out through an
|
||||
application-level connection error, not through this object. That is the
|
||||
upgrade risk of this chart's policy, stated plainly; the escape hatch is
|
||||
networkPolicy.egress.extraPorts.
|
||||
|
||||
WHAT THIS OBJECT IS: a ceiling on reachability, not an authentication layer.
|
||||
Only the ports declared below are reachable at all - whatever else the process
|
||||
binds, now or after the next app update, is blocked without anyone having to
|
||||
notice it exists.
|
||||
|
||||
WHAT IT IS NOT:
|
||||
- EGRESS IS COARSE. A NetworkPolicy matches CIDRs and selectors and NEVER
|
||||
hostnames, so "only to the database host" cannot be expressed at all. This
|
||||
cluster's CNI is Canal (Flannel plus Calico for policy) and has no FQDN
|
||||
rule; Calico's GlobalNetworkSet could express one but would bind this chart
|
||||
to Calico, the same argument that kept implementation-specific objects out
|
||||
of the Gateway-API sweep (#114). Each egress rule therefore permits its
|
||||
PORT to any destination.
|
||||
- NOT A BOUNDARY WHERE THE CNI DOES NOT ENFORCE ONE. An unenforced policy is
|
||||
accepted by the API server and filters nothing, so everything reads as
|
||||
configured while nothing is filtered.
|
||||
- THE REASONING IS IN TEMPLATE COMMENTS, not in "#" comments: those are part
|
||||
of the rendered object, travel into the cluster and appear in every release
|
||||
diff. ci/run-networkpolicy-test.sh fails on a rendered "#".
|
||||
|
||||
THE PORT NUMBERS BELOW ARE POD PORTS, NEVER NODEPORTS. A NetworkPolicy is
|
||||
evaluated at the pod's network interface, after kube-proxy has already
|
||||
translated a NodePort to the targetPort, so a NodePort number here would match
|
||||
nothing while looking strict - and, being the only rule for that port, would
|
||||
block every inbound connection instead. ci/run-networkpolicy-test.sh asserts
|
||||
every port here against the containerPorts of the pod spec this policy
|
||||
selects.
|
||||
*/ -}}
|
||||
{{- if .Values.networkPolicy.enabled }}
|
||||
{{- $np := .Values.networkPolicy }}
|
||||
{{- $ns := .Release.Namespace }}
|
||||
{{- $probeSources := $np.probeSources | default list }}
|
||||
{{- $allowedNs := $np.allowedNamespaces | default list }}
|
||||
{{- $ic := $np.ingressController | default dict }}
|
||||
{{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }}
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
||||
labels:
|
||||
app: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
||||
spec:
|
||||
{{- /*
|
||||
Selects the StatefulSet's pods. ci/run-networkpolicy-test.sh fails if
|
||||
this matches no pod spec of the same render - which is the mistake that
|
||||
would otherwise leave a policy in the namespace filtering nothing, and
|
||||
its usual cause is a label renamed in the workload template.
|
||||
*/}}
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
||||
policyTypes:
|
||||
- Ingress
|
||||
{{- /*
|
||||
Egress is listed because there IS something to restrict: this workload
|
||||
needs DNS and the ports named below, and nothing else. A chart whose
|
||||
workload opens no outbound connection of its own must NOT list Egress -
|
||||
valkey is the example - because a rule that constrains nothing makes the
|
||||
object look stricter than it is.
|
||||
*/}}
|
||||
- Egress
|
||||
ingress:
|
||||
{{- /*
|
||||
3000 - the only port this pod declares, the one the Service targets, and the
|
||||
one all three probes address (/_health). It is therefore the probe path.
|
||||
*/}}
|
||||
- ports:
|
||||
- protocol: TCP
|
||||
port: 3000
|
||||
{{- if $probeSources }}
|
||||
{{- /*
|
||||
RESTRICTED, because the operator supplied probeSources. The kubelet's probes
|
||||
come from the IP of the WORKER NODE running the pod, so that address has to be
|
||||
in the list or the pod never becomes Ready - and a pod that is never Ready
|
||||
reads as a broken application rather than as a policy.
|
||||
|
||||
The ingress controller is added to the same rule, or the Ingress goes dark the
|
||||
moment this list is filled.
|
||||
*/}}
|
||||
from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
{{- /*
|
||||
Set by Kubernetes itself on every namespace (>= 1.21), so this needs no
|
||||
labelling work on a namespace this chart does not own.
|
||||
*/}}
|
||||
kubernetes.io/metadata.name: {{ $ns }}
|
||||
{{- if and $routed $ic.namespace }}
|
||||
{{- /*
|
||||
ONE from-entry with TWO selectors, which is an AND: pods carrying these labels
|
||||
IN that namespace. Written as two list entries it would be an OR, and would
|
||||
open the port to every pod in kube-system.
|
||||
*/}}
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: {{ $ic.namespace | quote }}
|
||||
{{- with $ic.podLabels }}
|
||||
podSelector:
|
||||
matchLabels:
|
||||
{{- toYaml . | nindent 14 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- range $allowedNs }}
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: {{ . | quote }}
|
||||
{{- end }}
|
||||
{{- range $probeSources }}
|
||||
- ipBlock:
|
||||
cidr: {{ . | quote }}
|
||||
{{- end }}
|
||||
{{- else }}
|
||||
{{- /*
|
||||
NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the
|
||||
cluster, because this port carries no NodePort. The alternative was to guess
|
||||
the worker node CIDRs, and a wrong guess blocks every kubelet probe: the pod
|
||||
stays un-Ready, the rollout times out, and the cause looks like the
|
||||
application. An empty "from: []" would be worse than either - it allows
|
||||
everything too, but while reading as a restriction. Narrow it with
|
||||
networkPolicy.probeSources.
|
||||
*/}}
|
||||
{{- end }}
|
||||
egress:
|
||||
{{- /*
|
||||
DNS, unconditionally. There is no value to switch this off, and that is
|
||||
deliberate: a knob whose only other setting breaks every name lookup in
|
||||
the pod is a trap and not a feature.
|
||||
*/}}
|
||||
- ports:
|
||||
- protocol: UDP
|
||||
port: 53
|
||||
- protocol: TCP
|
||||
port: 53
|
||||
{{- /*
|
||||
Postgres. The port, not the host - outline.secrets.dbSecretRef carries a
|
||||
connection URL the chart never parses, so this is the STANDARD port and not
|
||||
the configured one. A Postgres on any other port is blocked and must be added
|
||||
to networkPolicy.egress.extraPorts.
|
||||
*/}}
|
||||
- ports:
|
||||
- protocol: TCP
|
||||
port: 5432
|
||||
{{- /*
|
||||
Redis/Valkey, same reasoning as Postgres above: the standard port, because
|
||||
outline.secrets.redisSecretRef holds a URL this chart does not read.
|
||||
*/}}
|
||||
- ports:
|
||||
- protocol: TCP
|
||||
port: 6379
|
||||
{{- /*
|
||||
Outbound HTTPS. Needed for the OIDC provider: Outline fetches the issuer's
|
||||
discovery document at boot, and the discovery URL also lives inside a secret.
|
||||
443 to any destination is as narrow as this object gets for that - a hostname
|
||||
cannot appear in a NetworkPolicy. An OIDC provider on a non-standard port (a
|
||||
CI dex on 5556, for instance) needs extraPorts.
|
||||
*/}}
|
||||
- ports:
|
||||
- protocol: TCP
|
||||
port: 443
|
||||
{{- with $np.egress }}
|
||||
{{- with .extraPorts }}
|
||||
{{- /*
|
||||
Ports only, to any destination - see the header. This is the operator's
|
||||
escape hatch from the egress cap, and the only alternative to switching
|
||||
the whole policy off.
|
||||
*/}}
|
||||
- ports:
|
||||
{{- range . }}
|
||||
- protocol: TCP
|
||||
port: {{ int . }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
Reference in new issue
Block a user