Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Selector match (pull_request) Successful in 9s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 9s
Validate Helm Charts / Bump level (pull_request) Successful in 3s
Validate Helm Charts / YAML scalars (pull_request) Successful in 9s
Validate Helm Charts / Build cyberchef (pull_request) Successful in 5s
Validate Helm Charts / Resource quantities (pull_request) Successful in 10s
Validate Helm Charts / NetworkPolicy (pull_request) Failing after 10s
Validate Helm Charts / Trivy config cyberchef (pull_request) Successful in 14s
Validate Helm Charts / Build outline (pull_request) Successful in 5s
Validate Helm Charts / Build template-chart (pull_request) Successful in 5s
Validate Helm Charts / Trivy config outline (pull_request) Successful in 17s
Validate Helm Charts / Trivy config template-chart (pull_request) Successful in 15s
Install-test / Install-test cyberchef (pull_request) Successful in 1m9s
Install-test / Install-test outline (pull_request) Failing after 5m59s
Validate Helm Charts / Install-test (pull_request) Failing after 7m7s
The first experiment did not settle it. Restricting cyberchef's probe port 8080 to 192.0.2.11/32 - an address that cannot be the k3d node's - left the install-test GREEN, which has two very different causes: either k3s' policy controller exempts the kubelet's probe traffic (good news for this convention, whose default leaves the probe port open anyway), or it enforces nothing at all (no net under this rollout). This separates them without touching a probe. Outline's egress rule for DNS is removed, so the pod cannot resolve "postgres" or "redis" - and all three of its probes hit /_health, which checks both stores. Expected: RED, which proves egress is enforced. Green means nothing is enforced in CI and the scope has to be renegotiated. Reverted in the next commit either way. Part of #5
205 lines
8.9 KiB
YAML
205 lines
8.9 KiB
YAML
{{- /*
|
|
NetworkPolicy for outline (Gitea #5).
|
|
|
|
Outline is the first chart of this rollout whose policy has real work to do in
|
|
BOTH directions, and it is also the one that proves the egress cap bites: all
|
|
three of its probes hit /_health, which checks Postgres AND Redis, so a pod
|
|
that cannot reach its data stores never becomes Ready.
|
|
|
|
THE EGRESS PORTS ARE THE CHART'S ONLY KNOWLEDGE OF ITS DEPENDENCIES, AND THEY
|
|
ARE A GUESS AT THE STANDARD ONES. Outline's database, Redis and OIDC provider
|
|
are configured through connection URLs inside 1Password items - the chart never
|
|
sees a host or a port, cannot derive one, and cannot validate one. So the rules
|
|
below name the standard ports (5432, 6379, 443) and nothing else. A deployment
|
|
whose Postgres listens anywhere else is BLOCKED, and it finds out through an
|
|
application-level connection error, not through this object. That is the
|
|
upgrade risk of this chart's policy, stated plainly; the escape hatch is
|
|
networkPolicy.egress.extraPorts.
|
|
|
|
WHAT THIS OBJECT IS: a ceiling on reachability, not an authentication layer.
|
|
Only the ports declared below are reachable at all - whatever else the process
|
|
binds, now or after the next app update, is blocked without anyone having to
|
|
notice it exists.
|
|
|
|
WHAT IT IS NOT:
|
|
- EGRESS IS COARSE. A NetworkPolicy matches CIDRs and selectors and NEVER
|
|
hostnames, so "only to the database host" cannot be expressed at all. This
|
|
cluster's CNI is Canal (Flannel plus Calico for policy) and has no FQDN
|
|
rule; Calico's GlobalNetworkSet could express one but would bind this chart
|
|
to Calico, the same argument that kept implementation-specific objects out
|
|
of the Gateway-API sweep (#114). Each egress rule therefore permits its
|
|
PORT to any destination.
|
|
- NOT A BOUNDARY WHERE THE CNI DOES NOT ENFORCE ONE. An unenforced policy is
|
|
accepted by the API server and filters nothing, so everything reads as
|
|
configured while nothing is filtered.
|
|
- THE REASONING IS IN TEMPLATE COMMENTS, not in "#" comments: those are part
|
|
of the rendered object, travel into the cluster and appear in every release
|
|
diff. ci/run-networkpolicy-test.sh fails on a rendered "#".
|
|
|
|
THE PORT NUMBERS BELOW ARE POD PORTS, NEVER NODEPORTS. A NetworkPolicy is
|
|
evaluated at the pod's network interface, after kube-proxy has already
|
|
translated a NodePort to the targetPort, so a NodePort number here would match
|
|
nothing while looking strict - and, being the only rule for that port, would
|
|
block every inbound connection instead. ci/run-networkpolicy-test.sh asserts
|
|
every port here against the containerPorts of the pod spec this policy
|
|
selects.
|
|
*/ -}}
|
|
{{- if .Values.networkPolicy.enabled }}
|
|
{{- $np := .Values.networkPolicy }}
|
|
{{- $ns := .Release.Namespace }}
|
|
{{- $probeSources := $np.probeSources | default list }}
|
|
{{- $allowedNs := $np.allowedNamespaces | default list }}
|
|
{{- $ic := $np.ingressController | default dict }}
|
|
{{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }}
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: NetworkPolicy
|
|
metadata:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
|
labels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
|
spec:
|
|
{{- /*
|
|
Selects the StatefulSet's pods. ci/run-networkpolicy-test.sh fails if
|
|
this matches no pod spec of the same render - which is the mistake that
|
|
would otherwise leave a policy in the namespace filtering nothing, and
|
|
its usual cause is a label renamed in the workload template.
|
|
*/}}
|
|
podSelector:
|
|
matchLabels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
|
policyTypes:
|
|
- Ingress
|
|
{{- /*
|
|
Egress is listed because there IS something to restrict: this workload
|
|
needs DNS and the ports named below, and nothing else. A chart whose
|
|
workload opens no outbound connection of its own must NOT list Egress -
|
|
valkey is the example - because a rule that constrains nothing makes the
|
|
object look stricter than it is.
|
|
*/}}
|
|
- Egress
|
|
ingress:
|
|
{{- /*
|
|
3000 - the only port this pod declares, the one the Service targets, and the
|
|
one all three probes address (/_health). It is therefore the probe path.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 3000
|
|
{{- if $probeSources }}
|
|
{{- /*
|
|
RESTRICTED, because the operator supplied probeSources. The kubelet's probes
|
|
come from the IP of the WORKER NODE running the pod, so that address has to be
|
|
in the list or the pod never becomes Ready - and a pod that is never Ready
|
|
reads as a broken application rather than as a policy.
|
|
|
|
The ingress controller is added to the same rule, or the Ingress goes dark the
|
|
moment this list is filled.
|
|
*/}}
|
|
from:
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
{{- /*
|
|
Set by Kubernetes itself on every namespace (>= 1.21), so this needs no
|
|
labelling work on a namespace this chart does not own.
|
|
*/}}
|
|
kubernetes.io/metadata.name: {{ $ns }}
|
|
{{- if and $routed $ic.namespace }}
|
|
{{- /*
|
|
ONE from-entry with TWO selectors, which is an AND: pods carrying these labels
|
|
IN that namespace. Written as two list entries it would be an OR, and would
|
|
open the port to every pod in kube-system.
|
|
*/}}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ $ic.namespace | quote }}
|
|
{{- with $ic.podLabels }}
|
|
podSelector:
|
|
matchLabels:
|
|
{{- toYaml . | nindent 14 }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- range $allowedNs }}
|
|
- namespaceSelector:
|
|
matchLabels:
|
|
kubernetes.io/metadata.name: {{ . | quote }}
|
|
{{- end }}
|
|
{{- range $probeSources }}
|
|
- ipBlock:
|
|
cidr: {{ . | quote }}
|
|
{{- end }}
|
|
{{- else }}
|
|
{{- /*
|
|
NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the
|
|
cluster, because this port carries no NodePort. The alternative was to guess
|
|
the worker node CIDRs, and a wrong guess blocks every kubelet probe: the pod
|
|
stays un-Ready, the rollout times out, and the cause looks like the
|
|
application. An empty "from: []" would be worse than either - it allows
|
|
everything too, but while reading as a restriction. Narrow it with
|
|
networkPolicy.probeSources.
|
|
*/}}
|
|
{{- end }}
|
|
egress:
|
|
{{- /*
|
|
TEMPORARY - THIS COMMIT IS AN EXPERIMENT AND IS REVERTED IN THE NEXT ONE.
|
|
|
|
The DNS rule is REMOVED here on purpose. Everything about the scope of
|
|
Gitea #5 rests on the install-test enforcing NetworkPolicies, and the first
|
|
experiment did not settle it: restricting cyberchef's probe port to an
|
|
address that cannot be the node's left the install-test GREEN. That has two
|
|
possible causes and they are very different - either k3s' policy controller
|
|
exempts the kubelet's probe traffic, which would be good news for this
|
|
convention, or it enforces nothing at all, which would mean there is no net
|
|
under this rollout.
|
|
|
|
This separates them, unambiguously and without touching a probe. Outline
|
|
resolves "postgres" and "redis" by name, and all three of its probes hit
|
|
/_health, which checks BOTH stores. With no DNS rule and policyTypes
|
|
Egress, every lookup fails.
|
|
|
|
EXPECTED: the pod never becomes Ready, rollout status times out, the
|
|
outline install-test goes RED - which proves egress is enforced.
|
|
If it goes GREEN, nothing is enforced in CI and the scope of this issue has
|
|
to be renegotiated rather than pushed through.
|
|
*/}}
|
|
{{- /*
|
|
Postgres. The port, not the host - outline.secrets.dbSecretRef carries a
|
|
connection URL the chart never parses, so this is the STANDARD port and not
|
|
the configured one. A Postgres on any other port is blocked and must be added
|
|
to networkPolicy.egress.extraPorts.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 5432
|
|
{{- /*
|
|
Redis/Valkey, same reasoning as Postgres above: the standard port, because
|
|
outline.secrets.redisSecretRef holds a URL this chart does not read.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 6379
|
|
{{- /*
|
|
Outbound HTTPS. Needed for the OIDC provider: Outline fetches the issuer's
|
|
discovery document at boot, and the discovery URL also lives inside a secret.
|
|
443 to any destination is as narrow as this object gets for that - a hostname
|
|
cannot appear in a NetworkPolicy. An OIDC provider on a non-standard port (a
|
|
CI dex on 5556, for instance) needs extraPorts.
|
|
*/}}
|
|
- ports:
|
|
- protocol: TCP
|
|
port: 443
|
|
{{- with $np.egress }}
|
|
{{- with .extraPorts }}
|
|
{{- /*
|
|
Ports only, to any destination - see the header. This is the operator's
|
|
escape hatch from the egress cap, and the only alternative to switching
|
|
the whole policy off.
|
|
*/}}
|
|
- ports:
|
|
{{- range . }}
|
|
- protocol: TCP
|
|
port: {{ int . }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|