Install-test / Install-test outline (pull_request) Failing after 6m10s
Validate Helm Charts / Install-test (pull_request) Failing after 7m13s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Selector match (pull_request) Successful in 9s
Validate Helm Charts / Resource quantities (pull_request) Successful in 10s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 9s
Validate Helm Charts / Bump level (pull_request) Successful in 5s
Validate Helm Charts / YAML scalars (pull_request) Successful in 9s
Validate Helm Charts / NetworkPolicy (pull_request) Failing after 9s
Validate Helm Charts / Build cyberchef (pull_request) Successful in 5s
Validate Helm Charts / Trivy config cyberchef (pull_request) Successful in 14s
Validate Helm Charts / Build outline (pull_request) Successful in 5s
Validate Helm Charts / Build template-chart (pull_request) Successful in 6s
Validate Helm Charts / Trivy config template-chart (pull_request) Successful in 14s
Install-test / Install-test cyberchef (pull_request) Successful in 1m3s
Validate Helm Charts / Trivy config outline (pull_request) Successful in 14s
Both experiments are in, and together they answer the question the scope of #5 rests on - with an answer that is not the one the plan assumed. MEASURED, run 888 on this branch: Install-test cyberchef SUCCESS, 1m9s probe port 8080 restricted to 192.0.2.11/32, which cannot be the k3d node's address Install-test outline FAILURE, 5m59s DNS egress rule removed, so "postgres" and "redis" do not resolve and /_health cannot pass So: POLICIES ARE ENFORCED in the install-test - outline's pod never became Ready and rollout status timed out after six minutes, which is the net this rollout needs. But THE KUBELET'S PROBE TRAFFIC IS EXEMPT from policy in k3s: cyberchef became Ready in 69 seconds with its probe port restricted to an address no probe can come from. What that changes, and what it does not: - The plan's claim was "a policy that blocks the probe path makes the install-test red". That is NOT true here, and it is now measured rather than assumed. The install-test's net is for EGRESS and for anything the application itself needs, not for the probe path. - It does not undermine the rollout, because this convention never restricts a probe port by default - the rule is rendered without a "from" until an operator sets probeSources. The trap CI cannot see is therefore not triggered by this PR; it is triggered by a deliberate narrowing, which is what the NOTES.txt sentence and ci/run-networkpolicy-test.sh's fourth assertion exist for. - It raises the stakes for the render assertions. A wrong ingress port - the NodePort mistake above all - would leave the pod Ready in CI (probes exempt) and no ingress controller runs there either, so nothing in the pipeline would notice. That failure is caught mechanically by assertion 2 or not at all. - And it says nothing about the production cluster, whose CNI is Calico, not k3s'. There, probe traffic is subject to policy, which is exactly why the default stays open. Part of #5
193 lines
8.0 KiB
YAML
193 lines
8.0 KiB
YAML
{{- /*
|
|
NetworkPolicy for outline (Gitea #5).
|
|
|
|
Outline is the first chart of this rollout whose policy has real work to do in
|
|
BOTH directions, and it is also the one that proves the egress cap bites: all
|
|
three of its probes hit /_health, which checks Postgres AND Redis, so a pod
|
|
that cannot reach its data stores never becomes Ready.
|
|
|
|
THE EGRESS PORTS ARE THE CHART'S ONLY KNOWLEDGE OF ITS DEPENDENCIES, AND THEY
|
|
ARE A GUESS AT THE STANDARD ONES. Outline's database, Redis and OIDC provider
|
|
are configured through connection URLs inside 1Password items - the chart never
|
|
sees a host or a port, cannot derive one, and cannot validate one. So the rules
|
|
below name the standard ports (5432, 6379, 443) and nothing else. A deployment
|
|
whose Postgres listens anywhere else is BLOCKED, and it finds out through an
|
|
application-level connection error, not through this object. That is the
|
|
upgrade risk of this chart's policy, stated plainly; the escape hatch is
|
|
networkPolicy.egress.extraPorts.
|
|
|
|
WHAT THIS OBJECT IS: a ceiling on reachability, not an authentication layer.
|
|
Only the ports declared below are reachable at all - whatever else the process
|
|
binds, now or after the next app update, is blocked without anyone having to
|
|
notice it exists.
|
|
|
|
WHAT IT IS NOT:
|
|
- EGRESS IS COARSE. A NetworkPolicy matches CIDRs and selectors and NEVER
|
|
hostnames, so "only to the database host" cannot be expressed at all. This
|
|
cluster's CNI is Canal (Flannel plus Calico for policy) and has no FQDN
|
|
rule; Calico's GlobalNetworkSet could express one but would bind this chart
|
|
to Calico, the same argument that kept implementation-specific objects out
|
|
of the Gateway-API sweep (#114). Each egress rule therefore permits its
|
|
PORT to any destination.
|
|
- NOT A BOUNDARY WHERE THE CNI DOES NOT ENFORCE ONE. An unenforced policy is
|
|
accepted by the API server and filters nothing, so everything reads as
|
|
configured while nothing is filtered.
|
|
- THE REASONING IS IN TEMPLATE COMMENTS, not in "#" comments: those are part
|
|
of the rendered object, travel into the cluster and appear in every release
|
|
diff. ci/run-networkpolicy-test.sh fails on a rendered "#".
|
|
|
|
THE PORT NUMBERS BELOW ARE POD PORTS, NEVER NODEPORTS. A NetworkPolicy is
|
|
evaluated at the pod's network interface, after kube-proxy has already
|
|
translated a NodePort to the targetPort, so a NodePort number here would match
|
|
nothing while looking strict - and, being the only rule for that port, would
|
|
block every inbound connection instead. ci/run-networkpolicy-test.sh asserts
|
|
every port here against the containerPorts of the pod spec this policy
|
|
selects.
|
|
*/ -}}
|
|
{{- if .Values.networkPolicy.enabled }}
|
|
{{- $np := .Values.networkPolicy }}
|
|
{{- $ns := .Release.Namespace }}
|
|
{{- $probeSources := $np.probeSources | default list }}
|
|
{{- $allowedNs := $np.allowedNamespaces | default list }}
|
|
{{- $ic := $np.ingressController | default dict }}
|
|
{{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }}
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
|
labels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
|
spec:
|
|
{{- /*
|
|
Selects the StatefulSet's pods. ci/run-networkpolicy-test.sh fails if
|
|
this matches no pod spec of the same render - which is the mistake that
|
|
would otherwise leave a policy in the namespace filtering nothing, and
|
|
its usual cause is a label renamed in the workload template.
|
|
*/}}
|
|
podSelector:
|
|
matchLabels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
|
policyTypes:
|
|
- Ingress
|
|
{{- /*
|
|
Egress is listed because there IS something to restrict: this workload
|
|
needs DNS and the ports named below, and nothing else. A chart whose
|
|
workload opens no outbound connection of its own must NOT list Egress -
|
|
valkey is the example - because a rule that constrains nothing makes the
|
|
object look stricter than it is.
|
|
*/}}
|
|
- Egress
|
|
ingress:
|
|
{{- /*
|
|
3000 - the only port this pod declares, the one the Service targets, and the
|
|
one all three probes address (/_health). It is therefore the probe path.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 3000
|
|
{{- if $probeSources }}
|
|
{{- /*
|
|
RESTRICTED, because the operator supplied probeSources. The kubelet's probes
|
|
come from the IP of the WORKER NODE running the pod, so that address has to be
|
|
in the list or the pod never becomes Ready - and a pod that is never Ready
|
|
reads as a broken application rather than as a policy.
|
|
|
|
The ingress controller is added to the same rule, or the Ingress goes dark the
|
|
moment this list is filled.
|
|
*/}}
|
|
from:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
{{- /*
|
|
Set by Kubernetes itself on every namespace (>= 1.21), so this needs no
|
|
labelling work on a namespace this chart does not own.
|
|
*/}}
|
|
kubernetes.io/metadata.name: {{ $ns }}
|
|
{{- if and $routed $ic.namespace }}
|
|
{{- /*
|
|
ONE from-entry with TWO selectors, which is an AND: pods carrying these labels
|
|
IN that namespace. Written as two list entries it would be an OR, and would
|
|
open the port to every pod in kube-system.
|
|
*/}}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ $ic.namespace | quote }}
|
|
{{- with $ic.podLabels }}
|
|
podSelector:
|
|
matchLabels:
|
|
{{- toYaml . | nindent 14 }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- range $allowedNs }}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ . | quote }}
|
|
{{- end }}
|
|
{{- range $probeSources }}
|
|
- ipBlock:
|
|
cidr: {{ . | quote }}
|
|
{{- end }}
|
|
{{- else }}
|
|
{{- /*
|
|
NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the
|
|
cluster, because this port carries no NodePort. The alternative was to guess
|
|
the worker node CIDRs, and a wrong guess blocks every kubelet probe: the pod
|
|
stays un-Ready, the rollout times out, and the cause looks like the
|
|
application. An empty "from: []" would be worse than either - it allows
|
|
everything too, but while reading as a restriction. Narrow it with
|
|
networkPolicy.probeSources.
|
|
*/}}
|
|
{{- end }}
|
|
egress:
|
|
{{- /*
|
|
DNS, unconditionally. There is no value to switch this off, and that is
|
|
deliberate: a knob whose only other setting breaks every name lookup in
|
|
the pod is a trap and not a feature.
|
|
*/}}
|
|
- ports:
|
|
- protocol: UDP
|
|
port: 53
|
|
- protocol: TCP
|
|
port: 53
|
|
{{- /*
|
|
Postgres. The port, not the host - outline.secrets.dbSecretRef carries a
|
|
connection URL the chart never parses, so this is the STANDARD port and not
|
|
the configured one. A Postgres on any other port is blocked and must be added
|
|
to networkPolicy.egress.extraPorts.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 5432
|
|
{{- /*
|
|
Redis/Valkey, same reasoning as Postgres above: the standard port, because
|
|
outline.secrets.redisSecretRef holds a URL this chart does not read.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 6379
|
|
{{- /*
|
|
Outbound HTTPS. Needed for the OIDC provider: Outline fetches the issuer's
|
|
discovery document at boot, and the discovery URL also lives inside a secret.
|
|
443 to any destination is as narrow as this object gets for that - a hostname
|
|
cannot appear in a NetworkPolicy. An OIDC provider on a non-standard port (a
|
|
CI dex on 5556, for instance) needs extraPorts.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 443
|
|
{{- with $np.egress }}
|
|
{{- with .extraPorts }}
|
|
{{- /*
|
|
Ports only, to any destination - see the header. This is the operator's
|
|
escape hatch from the egress cap, and the only alternative to switching
|
|
the whole policy off.
|
|
*/}}
|
|
- ports:
|
|
{{- range . }}
|
|
- protocol: TCP
|
|
port: {{ int . }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|