feat!: NetworkPolicy in the NodePort and game-server charts, on by default
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 23s
Validate Helm Charts / Bump level (pull_request) Successful in 4s
Validate Helm Charts / Selector match (pull_request) Successful in 12s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 10s
Validate Helm Charts / YAML scalars (pull_request) Successful in 9s
Validate Helm Charts / NetworkPolicy (pull_request) Failing after 10s
Validate Helm Charts / Trivy config apache-tika (pull_request) Successful in 23s
Validate Helm Charts / Build apache-tika (pull_request) Successful in 14s
Validate Helm Charts / Build bentopdf (pull_request) Successful in 6s
Validate Helm Charts / Build bulwark (pull_request) Successful in 6s
Validate Helm Charts / Build calibre-web-automated (pull_request) Successful in 5s
Validate Helm Charts / Build cyberchef (pull_request) Successful in 5s
Validate Helm Charts / Build homeassistant (pull_request) Successful in 10s
Validate Helm Charts / Build garage (pull_request) Successful in 6s
Validate Helm Charts / Build gotenberg (pull_request) Successful in 11s
Validate Helm Charts / Build keycloak (pull_request) Successful in 5s
Validate Helm Charts / Build minecraft-router (pull_request) Successful in 8s
Validate Helm Charts / Build patchmon (pull_request) Successful in 8s
Validate Helm Charts / Build rallly (pull_request) Successful in 5s
Validate Helm Charts / Build satisfactory (pull_request) Successful in 5s
Validate Helm Charts / Build minecraft-server (pull_request) Successful in 7s
Validate Helm Charts / Build outline (pull_request) Successful in 5s
Validate Helm Charts / Build rss-bridge (pull_request) Successful in 5s
Validate Helm Charts / Build samba (pull_request) Successful in 5s
Validate Helm Charts / Build searxng (pull_request) Successful in 5s
Validate Helm Charts / Build sure (pull_request) Successful in 11s
Validate Helm Charts / Build teamspeak-server (pull_request) Successful in 18s
Validate Helm Charts / Build template-chart (pull_request) Successful in 12s
Validate Helm Charts / Build urlaubsverwaltung (pull_request) Successful in 4s
Validate Helm Charts / Build ytdlp-web-player (pull_request) Successful in 5s
Validate Helm Charts / Build voucher-vault (pull_request) Successful in 5s
Validate Helm Charts / Build zipline (pull_request) Successful in 4s
Validate Helm Charts / Trivy config bentopdf (pull_request) Successful in 13s
Validate Helm Charts / Trivy config bulwark (pull_request) Successful in 15s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 12s
Validate Helm Charts / Trivy config minecraft-router (pull_request) Successful in 17s
Validate Helm Charts / Trivy config calibre-web-automated (pull_request) Successful in 11s
Validate Helm Charts / Trivy config cyberchef (pull_request) Successful in 13s
Validate Helm Charts / Trivy config garage (pull_request) Successful in 20s
Validate Helm Charts / Trivy config gotenberg (pull_request) Successful in 14s
Validate Helm Charts / Trivy config homeassistant (pull_request) Successful in 11s
Validate Helm Charts / Trivy config outline (pull_request) Successful in 14s
Validate Helm Charts / Trivy config rallly (pull_request) Successful in 15s
Validate Helm Charts / Trivy config minecraft-server (pull_request) Successful in 27s
Validate Helm Charts / Trivy config patchmon (pull_request) Successful in 12s
Validate Helm Charts / Trivy config rss-bridge (pull_request) Successful in 13s
Validate Helm Charts / Trivy config teamspeak-server (pull_request) Successful in 11s
Validate Helm Charts / Trivy config samba (pull_request) Successful in 16s
Validate Helm Charts / Trivy config satisfactory (pull_request) Successful in 22s
Validate Helm Charts / Trivy config searxng (pull_request) Successful in 22s
Validate Helm Charts / Trivy config sure (pull_request) Successful in 17s
Validate Helm Charts / Trivy config template-chart (pull_request) Successful in 14s
Validate Helm Charts / Trivy config urlaubsverwaltung (pull_request) Successful in 11s
Validate Helm Charts / Trivy config voucher-vault (pull_request) Successful in 12s
Validate Helm Charts / Trivy config ytdlp-web-player (pull_request) Successful in 17s
Validate Helm Charts / Trivy config zipline (pull_request) Successful in 18s
Install-test / Install-test bulwark (pull_request) Successful in 1m18s
Install-test / Install-test homeassistant (pull_request) Successful in 2m3s
Install-test / Install-test outline (pull_request) Successful in 2m9s
Install-test / Install-test rss-bridge (pull_request) Successful in 51s
Install-test / Install-test samba (pull_request) Successful in 1m14s
Install-test / Install-test apache-tika (pull_request) Successful in 1m7s
Install-test / Install-test bentopdf (pull_request) Successful in 1m5s
Install-test / Install-test calibre-web-automated (pull_request) Successful in 1m47s
Install-test / Install-test cyberchef (pull_request) Successful in 49s
Install-test / Install-test garage (pull_request) Successful in 1m10s
Install-test / Install-test gotenberg (pull_request) Successful in 1m35s
Install-test / Install-test keycloak (pull_request) Successful in 2m23s
Install-test / Install-test minecraft-router (pull_request) Successful in 37s
Install-test / Install-test minecraft-server (pull_request) Successful in 1m34s
Install-test / Install-test patchmon (pull_request) Successful in 1m17s
Install-test / Install-test rallly (pull_request) Successful in 1m30s
Install-test / Install-test searxng (pull_request) Successful in 1m9s
Install-test / Install-test sure (pull_request) Successful in 2m10s
Install-test / Install-test voucher-vault (pull_request) Successful in 1m48s
Install-test / Install-test ytdlp-web-player (pull_request) Successful in 59s
Install-test / Install-test zipline (pull_request) Successful in 1m59s
Validate Helm Charts / Install-test (pull_request) Successful in 33m44s
Install-test / Install-test teamspeak-server (pull_request) Successful in 1m15s
Install-test / Install-test urlaubsverwaltung (pull_request) Successful in 1m51s
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 23s
Validate Helm Charts / Bump level (pull_request) Successful in 4s
Validate Helm Charts / Selector match (pull_request) Successful in 12s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 10s
Validate Helm Charts / YAML scalars (pull_request) Successful in 9s
Validate Helm Charts / NetworkPolicy (pull_request) Failing after 10s
Validate Helm Charts / Trivy config apache-tika (pull_request) Successful in 23s
Validate Helm Charts / Build apache-tika (pull_request) Successful in 14s
Validate Helm Charts / Build bentopdf (pull_request) Successful in 6s
Validate Helm Charts / Build bulwark (pull_request) Successful in 6s
Validate Helm Charts / Build calibre-web-automated (pull_request) Successful in 5s
Validate Helm Charts / Build cyberchef (pull_request) Successful in 5s
Validate Helm Charts / Build homeassistant (pull_request) Successful in 10s
Validate Helm Charts / Build garage (pull_request) Successful in 6s
Validate Helm Charts / Build gotenberg (pull_request) Successful in 11s
Validate Helm Charts / Build keycloak (pull_request) Successful in 5s
Validate Helm Charts / Build minecraft-router (pull_request) Successful in 8s
Validate Helm Charts / Build patchmon (pull_request) Successful in 8s
Validate Helm Charts / Build rallly (pull_request) Successful in 5s
Validate Helm Charts / Build satisfactory (pull_request) Successful in 5s
Validate Helm Charts / Build minecraft-server (pull_request) Successful in 7s
Validate Helm Charts / Build outline (pull_request) Successful in 5s
Validate Helm Charts / Build rss-bridge (pull_request) Successful in 5s
Validate Helm Charts / Build samba (pull_request) Successful in 5s
Validate Helm Charts / Build searxng (pull_request) Successful in 5s
Validate Helm Charts / Build sure (pull_request) Successful in 11s
Validate Helm Charts / Build teamspeak-server (pull_request) Successful in 18s
Validate Helm Charts / Build template-chart (pull_request) Successful in 12s
Validate Helm Charts / Build urlaubsverwaltung (pull_request) Successful in 4s
Validate Helm Charts / Build ytdlp-web-player (pull_request) Successful in 5s
Validate Helm Charts / Build voucher-vault (pull_request) Successful in 5s
Validate Helm Charts / Build zipline (pull_request) Successful in 4s
Validate Helm Charts / Trivy config bentopdf (pull_request) Successful in 13s
Validate Helm Charts / Trivy config bulwark (pull_request) Successful in 15s
Validate Helm Charts / Trivy config keycloak (pull_request) Successful in 12s
Validate Helm Charts / Trivy config minecraft-router (pull_request) Successful in 17s
Validate Helm Charts / Trivy config calibre-web-automated (pull_request) Successful in 11s
Validate Helm Charts / Trivy config cyberchef (pull_request) Successful in 13s
Validate Helm Charts / Trivy config garage (pull_request) Successful in 20s
Validate Helm Charts / Trivy config gotenberg (pull_request) Successful in 14s
Validate Helm Charts / Trivy config homeassistant (pull_request) Successful in 11s
Validate Helm Charts / Trivy config outline (pull_request) Successful in 14s
Validate Helm Charts / Trivy config rallly (pull_request) Successful in 15s
Validate Helm Charts / Trivy config minecraft-server (pull_request) Successful in 27s
Validate Helm Charts / Trivy config patchmon (pull_request) Successful in 12s
Validate Helm Charts / Trivy config rss-bridge (pull_request) Successful in 13s
Validate Helm Charts / Trivy config teamspeak-server (pull_request) Successful in 11s
Validate Helm Charts / Trivy config samba (pull_request) Successful in 16s
Validate Helm Charts / Trivy config satisfactory (pull_request) Successful in 22s
Validate Helm Charts / Trivy config searxng (pull_request) Successful in 22s
Validate Helm Charts / Trivy config sure (pull_request) Successful in 17s
Validate Helm Charts / Trivy config template-chart (pull_request) Successful in 14s
Validate Helm Charts / Trivy config urlaubsverwaltung (pull_request) Successful in 11s
Validate Helm Charts / Trivy config voucher-vault (pull_request) Successful in 12s
Validate Helm Charts / Trivy config ytdlp-web-player (pull_request) Successful in 17s
Validate Helm Charts / Trivy config zipline (pull_request) Successful in 18s
Install-test / Install-test bulwark (pull_request) Successful in 1m18s
Install-test / Install-test homeassistant (pull_request) Successful in 2m3s
Install-test / Install-test outline (pull_request) Successful in 2m9s
Install-test / Install-test rss-bridge (pull_request) Successful in 51s
Install-test / Install-test samba (pull_request) Successful in 1m14s
Install-test / Install-test apache-tika (pull_request) Successful in 1m7s
Install-test / Install-test bentopdf (pull_request) Successful in 1m5s
Install-test / Install-test calibre-web-automated (pull_request) Successful in 1m47s
Install-test / Install-test cyberchef (pull_request) Successful in 49s
Install-test / Install-test garage (pull_request) Successful in 1m10s
Install-test / Install-test gotenberg (pull_request) Successful in 1m35s
Install-test / Install-test keycloak (pull_request) Successful in 2m23s
Install-test / Install-test minecraft-router (pull_request) Successful in 37s
Install-test / Install-test minecraft-server (pull_request) Successful in 1m34s
Install-test / Install-test patchmon (pull_request) Successful in 1m17s
Install-test / Install-test rallly (pull_request) Successful in 1m30s
Install-test / Install-test searxng (pull_request) Successful in 1m9s
Install-test / Install-test sure (pull_request) Successful in 2m10s
Install-test / Install-test voucher-vault (pull_request) Successful in 1m48s
Install-test / Install-test ytdlp-web-player (pull_request) Successful in 59s
Install-test / Install-test zipline (pull_request) Successful in 1m59s
Validate Helm Charts / Install-test (pull_request) Successful in 33m44s
Install-test / Install-test teamspeak-server (pull_request) Successful in 1m15s
Install-test / Install-test urlaubsverwaltung (pull_request) Successful in 1m51s
garage, minecraft-router, minecraft-server, samba, teamspeak-server,
satisfactory. One major each. These are the charts the plan said to leave for
last, and they earned it - three of them changed the convention rather than
just consuming it.
GARAGE is the clearest illustration after keycloak. Its four ports split three
ways: s3 and web are RESTRICTED by source (no probe touches them), admin is the
probe path and stays open inside the cluster - which is uncomfortable, because
admin is also the admin API, and that is said out loud instead of glossed - and
rpc gets the narrowest rule the object can express, a podSelector with no
namespaceSelector. All four numbers come from garage.settings.
MINECRAFT-SERVER is the tightest policy in the repository, and the reason is
worth knowing: all three of its probes are EXEC probes, which put no packet on
the network, so no port has to stay open for the kubelet and no probeSources
value is needed at all. RCON and metrics are restricted outright. The game port
has two branches rendered from the same value the Service reads -
server.service.type NodePort means players arrive from the internet and the
rule must be open; anything else means the only path is in-cluster, so the rule
is restricted and the router's namespace has to be in allowedNamespaces.
MINECRAFT-ROUTER derives its egress from the routing table instead of guessing:
server.backendPort plus the "port" override of every mapping that sets one. Add
a backend on a non-standard port and the policy follows. What it cannot express
is the destination - mappings name hosts, and this object matches no hostname -
so the rule is wider than the routing table, and that is written down.
SATISFACTORY gets policyTypes: [Ingress] and no egress section, and the second
half of the reason is the deciding one. Steam's content delivery is not a
pinnable port set (SteamCMD over TCP 80/443/27015-27050, master servers over
UDP 27000-27100 and 4380, changed by Valve, re-downloaded on every update) -
and this is the ONE chart with no install-test at all, because the image is
multiple GB and the first start pulls over 10GB. An egress cap here would be
the only one in the sweep with no empirical check behind it, on the chart whose
correct port set is least knowable.
SAMBA and TEAMSPEAK-SERVER carry the NodePort lesson explicitly: the rules name
445/139 and the pod's voice and file-transfer ports, never the NodePorts, and
teamspeak's file is explicit that its pod port and NodePort happening to share a
number is a coincidence of its values rather than a pattern to copy. Teamspeak's
voice rule is UDP, and the file says why that matters: a TCP rule on the same
number is accepted, looks complete, and drops every voice packet.
THREE THINGS THE CHECKS CAUGHT WHILE THIS BATCH WAS WRITTEN, all of them real:
- minecraft-server's RCON rule named a port the pod does not declare. The
containerPort is gated on server.rcon.enabled and the rule was not, so
assertion 2 rejected it. The rule now carries the same gate.
- samba and satisfactory would have shipped allowedNamespaces and
probeSources values that reach NO rule - every one of their inbound rules
is open to any source. A value that silently does nothing is worse than one
that is absent, because it reads as a control, so both are gone from those
two charts and from their schemas, with the reason in values.yaml.
teamspeak-server loses probeSources for the same reason and keeps
allowedNamespaces, which its web-query rule uses.
- the policy templates no longer declare template variables they do not use.
samba keeps one, satisfactory none.
Part of #5
This commit is contained in:
30 files changed
+1885
-6
No files matched your search
+1
-1
@@ -19,7 +19,7 @@ type: application
|
||||
# offer those as "updates", which is why the garage block in renovate.json5
|
||||
# carries a versioningTemplate anchored to ^v<major>.<minor>.<patch>$ - same
|
||||
# shape as the stalwart block, for the same reason.
|
||||
version: 1.1.0+upv2.4.1
|
||||
version: 2.0.0+upv2.4.1
|
||||
appVersion: "v2.4.1"
|
||||
|
||||
maintainers:
|
||||
|
||||
@@ -204,3 +204,63 @@ TRAPS WORTH KNOWING BEFORE YOU NEED THEM
|
||||
- THE RPC SECRET IS IN THE PROCESS ENVIRONMENT, not in a file. That is
|
||||
forced, not chosen: Garage rejects a secret file with any group bit set,
|
||||
and the kubelet sets one. values.yaml has the mechanism.
|
||||
{{- if .Values.networkPolicy.enabled }}
|
||||
|
||||
NETWORKPOLICY IS ACTIVE — WHAT IT CAPS, AND WHAT IT DOES NOT
|
||||
{{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy selects this workload's pods.
|
||||
Inbound, and the four ports split three ways:
|
||||
- s3 ({{ $settings.ports.s3 | int64 }}) and web ({{ $settings.ports.web | int64 }}): RESTRICTED BY SOURCE — this namespace,
|
||||
networkPolicy.allowedNamespaces, and the ingress controller.
|
||||
- admin ({{ $settings.ports.admin | int64 }}): open inside the cluster, because all three probes address
|
||||
it. This is the ADMIN API, so read that as a smaller ceiling rather than
|
||||
a closed door; networkPolicy.probeSources closes the rest.
|
||||
- rpc ({{ $settings.ports.rpc | int64 }}): this release's own pods only. It has no Service.
|
||||
Outbound: DNS and TCP {{ $settings.ports.rpc | int64 }} (the cluster bus, outbound as well as in){{ with .Values.networkPolicy.egress }}{{ with .extraPorts }},
|
||||
plus TCP {{ join ", " . }}{{ end }}{{ end }}.
|
||||
|
||||
CHECK YOUR S3 CONSUMERS. Anything in another namespace that talks to this
|
||||
endpoint on its in-cluster Service address — a backup job, a Zipline storing
|
||||
uploads, a Stalwart blob store — is blocked until that namespace is in
|
||||
networkPolicy.allowedNamespaces. Consumers using the published Ingress name
|
||||
arrive through the controller and are already covered. The failure is an
|
||||
application-level error with nothing pointing at this object.
|
||||
|
||||
Three limits belong in the same breath, because none of them is visible in
|
||||
the object itself:
|
||||
|
||||
- EGRESS IS BY PORT, NEVER BY DESTINATION. A NetworkPolicy matches CIDRs and
|
||||
selectors and cannot express a hostname, so every egress rule permits its
|
||||
port to ANY address. That is a limit of the object, not a shortcut — this
|
||||
cluster's CNI (Canal: Flannel plus Calico) has no FQDN rule either. An
|
||||
outbound port the chart does not know about is BLOCKED: add it to
|
||||
networkPolicy.egress.extraPorts, and expect to find out through an
|
||||
application-level connection error rather than through this object.
|
||||
{{- if .Values.networkPolicy.probeSources }}
|
||||
- THE PROBE PATH IS RESTRICTED in this release, because
|
||||
networkPolicy.probeSources is set to
|
||||
{{ join ", " .Values.networkPolicy.probeSources }}
|
||||
Probes come from the IP of the WORKER NODE running the pod. If none of those
|
||||
CIDRs covers the node this pod lands on, the kubelet's probes are dropped
|
||||
and THE POD NEVER BECOMES READY — the rollout then times out and the symptom
|
||||
looks like a broken application. Compare
|
||||
|
||||
kubectl -n {{ .Release.Namespace }} describe pod -l app={{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
||||
|
||||
against `kubectl get node -o wide`.
|
||||
{{- else }}
|
||||
- THE PROBED PORT IS OPEN INSIDE THE CLUSTER. networkPolicy.probeSources is
|
||||
empty, so that rule carries no source selector at all and any pod the CNI
|
||||
routes to can reach it. Not from the internet — but this is NOT pod-level
|
||||
isolation. Narrowing it means listing the worker node IPs in
|
||||
networkPolicy.probeSources, because the kubelet probes from the node's own
|
||||
address and the chart cannot know it; a wrong guess there leaves the pod
|
||||
permanently un-Ready.
|
||||
{{- end }}
|
||||
- IT IS ONLY A BOUNDARY WHERE THE CNI ENFORCES POLICIES. Where it does not,
|
||||
this object is accepted and filters nothing, so everything reads as
|
||||
configured while nothing is filtered. `kubectl -n {{ .Release.Namespace }} get networkpolicy`
|
||||
shows the object exists; only a connection attempt from an unrelated
|
||||
namespace shows that it bites.
|
||||
|
||||
Switch it off with networkPolicy.enabled: false.
|
||||
{{- end }}
|
||||
@@ -0,0 +1,232 @@
|
||||
{{- /*
|
||||
NetworkPolicy for garage (Gitea #5).
|
||||
|
||||
Garage declares four ports and they split three ways, which makes this the
|
||||
second-clearest illustration of the convention after keycloak:
|
||||
|
||||
s3 (3900) the S3 API. The Ingress points at it and no probe touches it,
|
||||
so it is RESTRICTED by source.
|
||||
web (3902) static website serving, same treatment as s3.
|
||||
admin (3903) the port all three probes address - tcpSocket twice and a
|
||||
httpGet /health - so it stays open inside the cluster. This is
|
||||
also the admin API, which is the uncomfortable part and is
|
||||
named in values.yaml: narrowing it means probeSources.
|
||||
rpc (3901) the node-to-node bus. It has NO Service at all, deliberately
|
||||
(see garage.service.yaml), and it gets the narrowest rule this
|
||||
object can express: the pods this policy itself selects.
|
||||
|
||||
All four numbers are derived from garage.settings, so moving a port in values
|
||||
moves the policy with it.
|
||||
|
||||
WHAT THIS OBJECT IS: a ceiling on reachability, not an authentication layer.
|
||||
Only the ports declared below are reachable at all - whatever else the process
|
||||
binds, now or after the next app update, is blocked without anyone having to
|
||||
notice it exists.
|
||||
|
||||
WHAT IT IS NOT:
|
||||
- EGRESS IS COARSE. A NetworkPolicy matches CIDRs and selectors and NEVER
|
||||
hostnames, so "only to the database host" cannot be expressed at all. This
|
||||
cluster's CNI is Canal (Flannel plus Calico for policy) and has no FQDN
|
||||
rule; Calico's GlobalNetworkSet could express one but would bind this chart
|
||||
to Calico, the same argument that kept implementation-specific objects out
|
||||
of the Gateway-API sweep (#114). Each egress rule therefore permits its
|
||||
PORT to any destination.
|
||||
- NOT A BOUNDARY WHERE THE CNI DOES NOT ENFORCE ONE. An unenforced policy is
|
||||
accepted by the API server and filters nothing, so everything reads as
|
||||
configured while nothing is filtered.
|
||||
- THE REASONING IS IN TEMPLATE COMMENTS, not in "#" comments: those are part
|
||||
of the rendered object, travel into the cluster and appear in every release
|
||||
diff. ci/run-networkpolicy-test.sh fails on a rendered "#".
|
||||
|
||||
THE PORT NUMBERS BELOW ARE POD PORTS, NEVER NODEPORTS. A NetworkPolicy is
|
||||
evaluated at the pod's network interface, after kube-proxy has already
|
||||
translated a NodePort to the targetPort, so a NodePort number here would match
|
||||
nothing while looking strict - and, being the only rule for that port, would
|
||||
block every inbound connection instead. ci/run-networkpolicy-test.sh asserts
|
||||
every port here against the containerPorts of the pod spec this policy
|
||||
selects.
|
||||
*/ -}}
|
||||
{{- if .Values.networkPolicy.enabled }}
|
||||
{{- $settings := fromYaml (include "garage.settings" .) }}
|
||||
{{- $np := .Values.networkPolicy }}
|
||||
{{- $ns := .Release.Namespace }}
|
||||
{{- $probeSources := $np.probeSources | default list }}
|
||||
{{- $allowedNs := $np.allowedNamespaces | default list }}
|
||||
{{- $ic := $np.ingressController | default dict }}
|
||||
{{- $routed := or .Values.ingress.enabled .Values.httpRoute.enabled }}
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
||||
labels:
|
||||
app: {{ .Release.Name }}-{{ .Chart.Name }}-networkpolicy
|
||||
spec:
|
||||
{{- /*
|
||||
Selects the StatefulSet's pods. ci/run-networkpolicy-test.sh fails if
|
||||
this matches no pod spec of the same render - which is the mistake that
|
||||
would otherwise leave a policy in the namespace filtering nothing, and
|
||||
its usual cause is a label renamed in the workload template.
|
||||
*/}}
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
||||
policyTypes:
|
||||
- Ingress
|
||||
{{- /*
|
||||
Egress is listed and it is narrow, which is the point: a storage node's only
|
||||
outbound traffic is DNS and its own cluster bus. Not permitted, and named in
|
||||
values.yaml because it is the plausible surprise: outbound HTTPS. Garage does
|
||||
not need it for S3 serving, but a deployment that configures remote replication
|
||||
or an external metrics sink does.
|
||||
*/}}
|
||||
- Egress
|
||||
ingress:
|
||||
{{- /*
|
||||
THE S3 API AND THE STATIC WEB PORT, both genuinely restricted by source,
|
||||
because no probe addresses either of them: this release's own namespace,
|
||||
whatever allowedNamespaces adds, and the ingress controller.
|
||||
|
||||
THE SOURCE LIST IS THE THING TO GET RIGHT HERE. Every consumer of this S3
|
||||
endpoint that lives in another namespace - a backup job, a Zipline using it for
|
||||
uploads, a Stalwart blob store - is a pod in that namespace and is BLOCKED
|
||||
unless the namespace is in allowedNamespaces. Consumers reaching the published
|
||||
Ingress name instead arrive through the controller and are already covered. A
|
||||
blocked S3 consumer fails at the application level, with nothing naming this
|
||||
object.
|
||||
*/}}
|
||||
- ports:
|
||||
- protocol: TCP
|
||||
port: {{ $settings.ports.s3 | int64 }}
|
||||
- protocol: TCP
|
||||
port: {{ $settings.ports.web | int64 }}
|
||||
from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
{{- /*
|
||||
Set by Kubernetes itself on every namespace (>= 1.21), so this needs no
|
||||
labelling work on a namespace this chart does not own.
|
||||
*/}}
|
||||
kubernetes.io/metadata.name: {{ $ns }}
|
||||
{{- if and $routed $ic.namespace }}
|
||||
{{- /*
|
||||
ONE from-entry with TWO selectors, which is an AND: pods carrying these labels
|
||||
IN that namespace. Written as two list entries it would be an OR, and would
|
||||
open the port to every pod in kube-system.
|
||||
*/}}
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: {{ $ic.namespace | quote }}
|
||||
{{- with $ic.podLabels }}
|
||||
podSelector:
|
||||
matchLabels:
|
||||
{{- toYaml . | nindent 14 }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- range $allowedNs }}
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: {{ . | quote }}
|
||||
{{- end }}
|
||||
{{- /*
|
||||
THE ADMIN PORT, and it is the probe path: liveness and startup open a TCP
|
||||
socket against it and readiness fetches /health from it. So it stays open
|
||||
inside the cluster by default.
|
||||
|
||||
That is worth stating rather than glossing, because this is the ADMIN API: key
|
||||
management and cluster layout live here. The policy caps it at "reachable from
|
||||
inside the cluster", not at "closed", and probeSources is what closes the rest -
|
||||
at the usual price of needing the worker node IPs.
|
||||
*/}}
|
||||
- ports:
|
||||
- protocol: TCP
|
||||
port: {{ $settings.ports.admin | int64 }}
|
||||
{{- if $probeSources }}
|
||||
{{- /*
|
||||
RESTRICTED, because the operator supplied probeSources. The kubelet's probes
|
||||
come from the IP of the WORKER NODE running the pod, so that address has to be
|
||||
in the list or the pod never becomes Ready - and a pod that is never Ready
|
||||
reads as a broken application rather than as a policy.
|
||||
|
||||
No ingress-controller entry on this rule: nothing should route to the admin API
|
||||
from outside.
|
||||
*/}}
|
||||
from:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
{{- /*
|
||||
Set by Kubernetes itself on every namespace (>= 1.21), so this needs no
|
||||
labelling work on a namespace this chart does not own.
|
||||
*/}}
|
||||
kubernetes.io/metadata.name: {{ $ns }}
|
||||
{{- range $allowedNs }}
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: {{ . | quote }}
|
||||
{{- end }}
|
||||
{{- range $probeSources }}
|
||||
- ipBlock:
|
||||
cidr: {{ . | quote }}
|
||||
{{- end }}
|
||||
{{- else }}
|
||||
{{- /*
|
||||
NO "from" SELECTOR, WHICH MEANS OPEN - deliberately, and only inside the
|
||||
cluster, because this port carries no NodePort. The alternative was to guess
|
||||
the worker node CIDRs, and a wrong guess blocks every kubelet probe: the pod
|
||||
stays un-Ready, the rollout times out, and the cause looks like the
|
||||
application. An empty "from: []" would be worse than either - it allows
|
||||
everything too, but while reading as a restriction. Narrow it with
|
||||
networkPolicy.probeSources.
|
||||
*/}}
|
||||
{{- end }}
|
||||
{{- /*
|
||||
THE NODE-TO-NODE RPC BUS, restricted to the pods this policy already selects -
|
||||
a podSelector with no namespaceSelector, which means "these labels, in this
|
||||
namespace".
|
||||
|
||||
This port deliberately has no Service (garage.service.yaml says why: a Service
|
||||
port would advertise a cluster interface that is not one), so this rule is the
|
||||
only description of it anywhere in the chart. The matching egress rule below is
|
||||
what lets a node dial OUT to its peers.
|
||||
*/}}
|
||||
- ports:
|
||||
- protocol: TCP
|
||||
port: {{ $settings.ports.rpc | int64 }}
|
||||
from:
|
||||
- podSelector:
|
||||
matchLabels:
|
||||
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
||||
egress:
|
||||
{{- /*
|
||||
DNS, unconditionally. There is no value to switch this off, and that is
|
||||
deliberate: a knob whose only other setting breaks every name lookup in
|
||||
the pod is a trap and not a feature.
|
||||
*/}}
|
||||
- ports:
|
||||
- protocol: UDP
|
||||
port: 53
|
||||
- protocol: TCP
|
||||
port: 53
|
||||
{{- /*
|
||||
Outbound RPC to the other nodes of this cluster. Needed as well as the inbound
|
||||
rule: policyTypes lists Egress, so a node dialling its peer is subject to this
|
||||
policy too, and with only the inbound rule the bus would be permitted in one
|
||||
direction - a layout that never converges while every node looks healthy.
|
||||
*/}}
|
||||
- ports:
|
||||
- protocol: TCP
|
||||
port: {{ $settings.ports.rpc | int64 }}
|
||||
{{- with $np.egress }}
|
||||
{{- with .extraPorts }}
|
||||
{{- /*
|
||||
Ports only, to any destination - see the header. This is the operator's
|
||||
escape hatch from the egress cap, and the only alternative to switching
|
||||
the whole policy off.
|
||||
*/}}
|
||||
- ports:
|
||||
{{- range . }}
|
||||
- protocol: TCP
|
||||
port: {{ int . }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
@@ -9,6 +9,57 @@
|
||||
"description": "Helm's conventional cross-chart values. Unused by this chart, kept open so an umbrella install cannot break on it.",
|
||||
"type": "object"
|
||||
},
|
||||
"networkPolicy": {
|
||||
"description": "The chart's NetworkPolicy (Gitea #5), on by default. A ceiling on reachability and not an authentication layer: only the declared container ports are reachable at all, egress is capped by PORT (a NetworkPolicy matches CIDRs and selectors, never hostnames, so a destination host cannot be expressed), and ingress is capped by source for every port no network probe touches. Only a boundary where the cluster's CNI enforces policies - an unenforced policy is accepted by the API server and filters nothing. The template's header carries the full list of what it does not do.",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"enabled": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"allowedNamespaces": {
|
||||
"description": "Namespaces whose pods may reach this workload, in addition to the release's own namespace, which is always allowed. Matched on the kubernetes.io/metadata.name label Kubernetes sets on every namespace itself, so no labelling work is needed - and no typo protection either: a name that does not exist matches nothing and is silently ineffective. An entry opens the port to EVERY pod in that namespace; for the ingress controller use ingressController instead.",
|
||||
"type": ["array", "null"],
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"probeSources": {
|
||||
"description": "Source CIDRs allowed to reach the port the network probes address. EMPTY MEANS OPEN, not blocked: with no entry that rule is rendered without a \"from\" selector. The kubelet's probes come from the IP of the worker node running the pod, an address the chart cannot know and must not guess - a guessed CIDR that excludes the real node blocks every probe and the pod never becomes Ready, which reads as a broken application rather than as a policy. Setting this restricts the rule, which is why the chart adds the own namespace, ingressController and allowedNamespaces to the same rule.",
|
||||
"type": ["array", "null"],
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"ingressController": {
|
||||
"description": "The ingress controller allowed to reach the routed port once probeSources restricts it; rendered only while ingress.enabled or httpRoute.enabled is true. The two keys become ONE \"from\" entry and are therefore an AND - pods carrying podLabels IN namespace - which is the whole point: the controller's namespace in allowedNamespaces would open the port to every pod in kube-system. The default is measured on this cluster (DaemonSet rke2-traefik in kube-system, which also serves the Gateway API here), not assumed; a policy written against the wrong controller is accepted, looks strict and silently blocks all web traffic.",
|
||||
"type": ["object", "null"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"namespace": {
|
||||
"type": ["string", "null"]
|
||||
},
|
||||
"podLabels": {
|
||||
"description": "Label set the controller's pods carry, rendered verbatim as a podSelector's matchLabels. Kept open because the chart does not interpret the keys.",
|
||||
"type": ["object", "null"]
|
||||
}
|
||||
}
|
||||
},
|
||||
"egress": {
|
||||
"type": ["object", "null"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"extraPorts": {
|
||||
"description": "Additional outbound TCP ports, to ANY destination - ports and not destinations, because a NetworkPolicy cannot express a hostname. DNS is always allowed and has no switch. Without an entry here an outbound port the chart does not already derive is blocked, which is the cap's whole purpose and also its upgrade risk.",
|
||||
"type": ["array", "null"],
|
||||
"items": {
|
||||
"type": "integer"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"scaleDown": {
|
||||
"description": "true renders the StatefulSet with replicas: 0 and takes precedence over garage.replicas.",
|
||||
"type": "boolean"
|
||||
|
||||
@@ -501,3 +501,92 @@ httpRoute:
|
||||
# garage.s3.rootDomain - the two still have to be set in step, as they do for
|
||||
# the Ingress.
|
||||
hostnames: []
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# NetworkPolicy (Gitea #5). On by default. The reference implementation with
|
||||
# the full reasoning is template-chart; the convention is in README, "Chart
|
||||
# style guidelines", and the operator's view on this chart's wiki page.
|
||||
# ---------------------------------------------------------------------------
|
||||
networkPolicy:
|
||||
# WHAT THIS POLICY ALLOWS, and Garage's four ports split three ways:
|
||||
# - s3 (3900) and web (3902): RESTRICTED BY SOURCE to this namespace,
|
||||
# allowedNamespaces and the ingress controller. No probe touches them,
|
||||
# which is what makes the restriction possible.
|
||||
# - admin (3903): open inside the cluster, because all three probes address
|
||||
# it. This is the ADMIN API - key management and cluster layout - so the
|
||||
# cap is "reachable from inside the cluster", not "closed". probeSources
|
||||
# closes the rest.
|
||||
# - rpc (3901): this release's own pods only. It has no Service at all.
|
||||
# Outbound: DNS and the rpc bus, nothing else.
|
||||
#
|
||||
# THE SOURCE LIST FOR s3 IS THE THING TO GET RIGHT. Every S3 consumer in
|
||||
# ANOTHER namespace - a backup job, a Zipline storing uploads, a Stalwart
|
||||
# blob store - is blocked unless that namespace is in allowedNamespaces.
|
||||
# Consumers that use the published Ingress name arrive through the controller
|
||||
# and are already covered. A blocked consumer fails at the application level,
|
||||
# with nothing naming this object.
|
||||
#
|
||||
# NOT allowed, and the plausible surprise: outbound HTTPS. Garage does not
|
||||
# need it to serve S3, but remote replication or an external metrics sink
|
||||
# does - see egress.extraPorts.
|
||||
enabled: true
|
||||
# Namespaces whose pods may reach this workload, in addition to the release's
|
||||
# own namespace, which is always allowed. Empty means: own namespace only.
|
||||
#
|
||||
# Matched on the "kubernetes.io/metadata.name" label Kubernetes sets on every
|
||||
# namespace itself, so no labelling work is needed on a namespace this chart
|
||||
# does not own - and no typo protection either: a name that does not exist
|
||||
# matches nothing and is silently ineffective. Note the granularity: an entry
|
||||
# opens the port to EVERY pod in that namespace.
|
||||
allowedNamespaces: []
|
||||
# Source CIDRs allowed to reach the admin port (garage.ports.admin, 3903 by default), which is the port all three probes address.
|
||||
#
|
||||
# EMPTY MEANS OPEN, NOT BLOCKED. With no entry that rule is rendered WITHOUT a
|
||||
# "from" selector, so the port is reachable from anywhere the CNI routes to -
|
||||
# inside the cluster, since it carries no NodePort.
|
||||
#
|
||||
# That is the deliberate choice between two bad defaults. The kubelet's probes
|
||||
# originate from the IP of the WORKER NODE running the pod, an address this
|
||||
# chart cannot know and must not guess: a CIDR that excludes the real node
|
||||
# blocks every probe, the pod never becomes Ready, the rollout times out, and
|
||||
# the symptom reads as a broken application rather than as a policy.
|
||||
#
|
||||
# Narrow it by listing the worker node IPs (e.g. ["10.0.1.11/32"]) - and mind
|
||||
# the companion trap: once this list is non-empty the rule IS restricted, so
|
||||
# everything else that needs the port has to be in it. The chart adds the own
|
||||
# namespace, the ingressController below and allowedNamespaces to the same rule for exactly that reason.
|
||||
probeSources: []
|
||||
# The ingress controller allowed to reach the routed port once probeSources
|
||||
# restricts it. Rendered only while ingress.enabled or httpRoute.enabled is true, and only in
|
||||
# the restricted branch - with the default (open) rule the controller reaches
|
||||
# the port anyway. The two keys become ONE "from" entry and are therefore an
|
||||
# AND: pods carrying podLabels IN namespace. That is the point of having them
|
||||
# at all - the controller's namespace in allowedNamespaces would open the port
|
||||
# to every pod in kube-system.
|
||||
#
|
||||
# MEASURED ON THIS CLUSTER, NOT ASSUMED (2026-10-14): ingress-nginx is gone,
|
||||
# the only IngressClass is "traefik" and it is the default, and the controller
|
||||
# is the DaemonSet rke2-traefik in kube-system. The same Traefik serves the
|
||||
# Gateway API here, so this one source covers the Ingress and the HTTPRoute
|
||||
# path alike. A policy written against the wrong controller is accepted, looks
|
||||
# strict, and silently blocks all web traffic - and ingress.className says
|
||||
# nothing about it, because a class names an implementation, not the pods the
|
||||
# traffic comes from.
|
||||
ingressController:
|
||||
namespace: kube-system
|
||||
podLabels:
|
||||
app.kubernetes.io/name: rke2-traefik
|
||||
egress:
|
||||
# Additional outbound TCP ports. DNS (UDP and TCP 53) is always allowed and
|
||||
# has no switch - a knob whose only other setting breaks every name lookup
|
||||
# in the pod is a trap, not a feature.
|
||||
#
|
||||
# PORTS, NOT DESTINATIONS, and that is a limit of the object rather than
|
||||
# laziness: a NetworkPolicy matches CIDRs and selectors and never hostnames,
|
||||
# so "only to the database host" cannot be written at all. Each entry
|
||||
# permits its port to ANY destination. What the cap still buys is real: a
|
||||
# compromised pod cannot open arbitrary outbound connections.
|
||||
#
|
||||
# 443 is the entry to add for remote replication or an external metrics
|
||||
# sink. Nothing is needed for plain S3 serving.
|
||||
extraPorts: []
|
||||
Reference in new issue
Block a user