test(ci): assert a chart's egress ports against the POD ports of the chart it dials
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 10s
Validate Helm Charts / Selector match (pull_request) Successful in 9s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 9s
Validate Helm Charts / NetworkPolicy (pull_request) Successful in 12s
Validate Helm Charts / Bump level (pull_request) Successful in 4s
Validate Helm Charts / YAML scalars (pull_request) Successful in 10s
Validate Helm Charts / Build paperless-ngx (pull_request) Successful in 6s
Validate Helm Charts / Trivy config paperless-ngx (pull_request) Successful in 20s
Install-test / Install-test paperless-ngx (pull_request) Successful in 3m2s
Validate Helm Charts / Install-test (pull_request) Successful in 3m2s

The four assertions from Gitea #5 hold ports against the pod spec the policy
SELECTS. That is right for ingress and meaningless for egress: an egress port
belongs to the destination, and for a database, a cache or the web that
destination is in no render this repository has. So the mistake behind #136 -
an egress rule naming a ClusterIP Service's port, which kube-proxy has already
translated away by the time the policy is evaluated at the pod's interface - was
not catchable within one chart.

It is catchable across two, and paperless-ngx -> apache-tika, gotenberg is the
only place in the repository where a chart dials another chart's Service (the
three other policies naming port 80 aim at the internet, where there is no DNAT
and 80 is the real port). The pairs are an explicit table rather than a
discovery, because no render can know which Service another chart dials - the
address is a value or a 1Password field - and a row naming a chart that no
longer exists fails instead of being skipped.

Each row is checked on two renders of the consumer: with the feature gate ON the
egress ports must CONTAIN every pod port behind the peers' Services, resolved
Service port -> targetPort -> containerPort, which is exactly the translation the
policy is evaluated after; with the gate OFF the requirement must not apply and
those ports must be absent, so a gated rule cannot quietly become one that
permits what the release cannot use. The evidence line prints the chain
("port 80 -> targetPort 'tika-port' -> pod port 9998"), so a reader sees the
DNAT rather than two numbers.

Built into run-networkpolicy-test.sh rather than added as a new script with its
own job: that job already runs over every chart on every PR, Helm is already the
YAML parser through ci/yaml2json, and the verdict and summary plumbing is
shared. python3 standard library only, no yq, no pyyaml, no Docker.

Six fail paths executed, not assumed: a required port removed, a wrong number in
its place, the peer chart renamed away, the policy name no longer following the
<release>-<chart>-<kind> convention, the peers rendering no Service at all (an
empty requirement reads like a satisfied one, so it fails), and the rule made
unconditional, which is the fail path of the gate-off half.

Measured: 30 charts, 210 assertions held (208 before), 0 failed, ~15s, no
cluster. README's stated figure was 206 and is corrected here.

Part of #136
This commit is contained in:
jklein committed 2026-10-15 17:46:45 +00:00
1 parent 530227405c
commit 1894e7d465
3 files changed
+409 -4

No files matched your search

+10
View File
@@ -321,6 +321,16 @@ jobs:
# Plus two guards of its own: a chart that renders NO policy fails, and a
# rendered "#" comment fails.
#
# And one assertion that spans TWO charts (Gitea #136), the only one here
# that does: where a chart dials the Service of another chart of this
# repository, its EGRESS rule must contain that chart's POD ports. kube-proxy
# does the ClusterIP's destination NAT before the policy is evaluated, so a
# rule naming the SERVICE port matches nothing wherever port and targetPort
# differ - paperless-ngx shipped exactly that and lost Tika and Gotenberg in
# production on 2026-10-15. The pairs are an explicit table in the script,
# because no render can discover which Service another chart dials, and the
# row is checked with the feature gate on AND off.
#
# Every one of those failures is SILENT. The API server accepts a policy
# whose selector matches nothing - it then sits in the namespace, is listed
# by kubectl, reads as a boundary and filters exactly nothing - just as
+8 -4
View File
@@ -230,8 +230,11 @@ Measured locally: **30 charts, 408 scalars verified, 0 failed**, no cluster. The
| every **ingress** port is a `containerPort` of the selected pods | a NodePort number where a pod port belongs. The policy is evaluated at the pod's interface, *after* kube-proxy translated the port, so `30025` matches nothing while looking strict — and being the only rule for that port, it blocks every inbound connection instead | nobody in CI (see below), and in production as a service that went mute |
| no ingress rule allows every source by accident | `from: []` allows everything, identically to having no `from` at all, while reading in a diff and in `kubectl` output as a restriction — the one mistake in this class that makes a policy *weaker* while looking *stricter* | nobody |
| every port a **network** probe addresses is either unrestricted, or the chart offers `networkPolicy.probeSources` **and** says in `NOTES.txt` what a wrong CIDR costs | the kubelet probes from the IP of the worker node running the pod, which no chart can know. A restricted probe port without that address drops every probe: the pod never becomes `Ready`, the rollout times out, and the symptom reads as a broken application | the install-test, in production's CNI — **but not in CI**, see below |
| **cross-chart egress**: where a chart dials another chart *of this repository*, its egress rule must contain the **pod** ports behind that chart's Services (#136) | an egress rule naming the **Service** port. kube-proxy does the ClusterIP's destination NAT *before* the policy is evaluated at the pod's interface, so the policy sees `podIP:targetPort` and the rule matches nothing wherever `port` and `targetPort` differ | nobody: the symptom is a **timeout**, not `connection refused`, and no error mentions a NetworkPolicy — `paperless-ngx` 14.0.0 shipped it and cost an outage on 2026-10-15 |
Egress ports are **deliberately not** held against the pod's `containerPort`s: an egress port is a port on the *destination*, and that comparison would fail every chart that reaches a database.
Egress ports are otherwise **deliberately not** held against the pod's `containerPort`s: an egress port is a port on the *destination*, and that comparison would fail every chart that reaches a database.
**The one exception is the fifth row, and it is the only assertion here that needs two charts.** For a database, a cache or the web the destination is in no render this repository has — but it *is* in a render when one chart dials the Service of another, and `paperless-ngx` → `apache-tika`, `gotenberg` is the single place in the repository where that happens (checked for #136: the three other charts naming `port: 80` in a policy are aiming at the **internet**, where there is no DNAT and 80 is the real port). So the pairs live in an explicit table (`EGRESS_PEER_ROWS`) rather than being discovered — a chart cannot know which Service another chart dials, because the address is a value or a 1Password field — and a row naming a chart that no longer exists **fails** instead of being skipped. Each row is checked on **two** renders: with the feature gate **on** the containment must hold, and with it **off** the requirement must not apply *and* those ports must be gone, so a gated rule cannot quietly become one that permits what the release cannot use.
Two guards beyond the four, both earned rather than designed. **A chart that renders no policy at all fails** — that is how this issue decayed in the first place: `garage` (#81) and `keycloak` (#83) were both created *after* #5 was filed and both arrived without a policy, `keycloak` with its unauthenticated management surface on 9000. And **a `#` comment inside a rendered policy fails**, because a `#` comment is part of the *object*: it travels into the cluster, shows up in `helm get manifest` and `kubectl get -o yaml`, and lands in every release diff. Reasoning belongs in Go template comments, which render nothing. Both precedent charts failed that check when it was first run, which is what it is for.
@@ -242,7 +245,7 @@ Four things to know before touching it:
- **Subchart documents are skipped by the comment assertion**, for a structural reason and not a lenient one: a dependency is fetched from the Helm registry rather than from the copy in this repo, so the policy `paperless-ngx` and `stalwart` bundle is whatever `valkey` version is *published*, and nothing in the parent chart can change its rendered comments. The subchart's own PR is where it is caught, because this test runs over `valkey` too.
- **Anti-vacuum guard**, as in the tests above: `ci/networkpolicy/<chart>.yaml` is a render-only overlay applied **last**, whose job is to render the **source-restricted probe rule** — a branch no chart's own defaults ever reach, and the one containing the expensive mistake (a `namespaceSelector` and a `podSelector` written as two list entries instead of one would be an OR, opening the port to every pod in `kube-system`). A chart that *has* an overlay and still renders no restricted **probe** rule fails. That counter is specifically the probe rule and not "any restricted rule", because `keycloak` restricts two rules in its own defaults and would have satisfied a looser guard while the branch went unrendered.
Measured locally: **30 charts, 206 assertions held, 0 failed**, no cluster, a few seconds. All twelve fail paths were exercised rather than assumed — and one earned its keep immediately: the `probeSources` check was a substring test, so a chart whose key had been **deleted** while the long comment above it still discussed the value passed it. It now looks for the key.
Measured locally: **30 charts, 210 assertions held, 0 failed**, no cluster, ~15 seconds. All twelve fail paths were exercised rather than assumed — and one earned its keep immediately: the `probeSources` check was a substring test, so a chart whose key had been **deleted** while the long comment above it still discussed the value passed it. It now looks for the key. The cross-chart assertion added two of those 210, and its own six fail paths were executed the same way: a required port removed from the rule, a wrong number in its place, a peer chart renamed away, the policy's name no longer following the `<release>-<chart>-<kind>` convention, the peers rendering no Service at all (which would make the requirement empty, and is treated as a failure because an empty requirement reads exactly like a satisfied one), and the gated rule made unconditional — which is the failure path of the *gate-off* half, since with the gate off and the rule properly gated the assertion must stay silent.
**What this test does not and cannot cover, named so the gap is not mistaken for coverage:** that traffic still flows. No ingress controller runs in the install-test cluster, so the path from Traefik to a pod is proven nowhere in this pipeline — and, measured on the PR for #5, **k3s exempts the kubelet's probe traffic from policy**, so a blocked probe path does *not* turn the install-test red either (`cyberchef` became `Ready` in 69s with its probe port restricted to an address no probe can come from). What the install-test *does* prove is enforcement of everything the application itself needs: with `outline`'s DNS egress rule removed, its pod never became `Ready` and the job failed after 5m59s. So the net under this convention is real, but it is an **egress** net — which raises the stakes for assertion 2, since a wrong ingress port is caught mechanically here or not at all.
@@ -690,7 +693,7 @@ Every port of a workload falls into one of four, and this is the part to get rig
#### Values
`networkPolicy.enabled`, `.allowedNamespaces`, `.probeSources`, `.ingressController`, `.egress.extraPorts` — and a chart ships **only the ones that reach a rule**. A value that silently does nothing is worse than one that is absent, because it reads as a control: `samba` and `satisfactory` carry `enabled` alone (every inbound rule is open to any source), and `teamspeak-server` has `allowedNamespaces` but no `probeSources`.
`networkPolicy.enabled`, `.allowedNamespaces`, `.probeSources`, `.ingressController`, `.egress.extraPorts` — and a chart ships **only the ones that reach a rule**. A chart may add a **named** egress switch where a port depends on something the values cannot see: `paperless-ngx`' `.egress.mailImport` opens IMAPS `993`, off by default, because its mail accounts are rows in its own database and no value says whether a release fetches mail. The bar for one of those: it names what it opens, and it is **not** a second switch for a port some *other* value already decides — that is why the Tika/Gotenberg rule stayed gated on `paperless.files.tikaGotenberg.enabled` and got no policy switch of its own. Two values that can drift apart re-arm the outage the rule was fixed for. A value that silently does nothing is worse than one that is absent, because it reads as a control: `samba` and `satisfactory` carry `enabled` alone (every inbound rule is open to any source), and `teamspeak-server` has `allowedNamespaces` but no `probeSources`.
**No chart ships `allowedNamespaces` pre-filled, ever.** A chart that named `mc-router`, or `paperless-jk`, or `stalwart` in its defaults would be baking in an assumption about *this* cluster's namespace layout — which is exactly the kind of claim a chart cannot keep true. So the default is the empty list and the chart is **restrictive**; filling it is the **deployment adaptation**, made where the namespace layout is actually visible. That is why a sweep like this produces a list of release-side one-liners: those are the expected procedure, not a defect list, and the breaking-change major on each chart is what announces them.
@@ -717,12 +720,13 @@ Every port of a workload falls into one of four, and this is the part to get rig
#### Mechanics
- **Pod ports, never NodePorts.** A `NetworkPolicy` is evaluated at the pod's network interface, *after* kube-proxy has translated a NodePort to the `targetPort`. `30025` where the pod listens on `25` matches nothing: accepted, strict-looking, and blocking every inbound connection instead. The render test asserts this mechanically against the selected pod spec's `containerPort`s.
- **And pod ports, never Service ports — in `egress` too (#136).** The same destination NAT runs on the way *out*: kube-proxy rewrites a ClusterIP's address and port **before** the egress policy is evaluated, so the policy sees `podIP:targetPort` and never `clusterIP:port`. An egress rule therefore names the **pod** port of its destination, and a Service port only ever matches by coincidence — wherever `port` and `targetPort` are the same number, which is the case for `5432`, `6379`, `993` and `443` and is why this mistake stayed invisible until a chart dialled a Service that forwards `80` to `9998`. It was argued the other way in `paperless-ngx` (*"an egress rule names the port on the destination as the client dials it, and the client dials the Service"*), shipped, and cost an outage. **The tell is the symptom:** a dropped packet runs into a **timeout**, a missing listener answers `connection refused` — so a timeout where the service demonstrably exists is a policy question, and no Kubernetes error will say so. For a destination *outside* the cluster there is no NAT in the path and the port dialled is the port to name; the deciding question is the DNAT, not the direction.
- **Protocol as well as number.** `teamspeak-server`'s voice port is UDP and `satisfactory`'s game port is both — a TCP-only rule on either is accepted, looks complete, and drops the traffic that actually arrives.
- **`kubernetes.io/metadata.name`** is set by Kubernetes itself on every namespace (≥ 1.21), so namespace selectors need no labelling work on namespaces the chart does not own — and get no typo protection either.
- **Never `from: []`.** It allows everything, identically to having no key, while reading as a restriction. Write no `from` where open is meant and say so in a template comment.
- **Reasoning goes in Go template comments, not `#` comments.** A `#` comment is part of the rendered object: it travels into the cluster, appears in `helm get manifest` and `kubectl get -o yaml`, and lands in every release diff. The render test fails on a rendered `#`. One second-order trap, found while writing the reference implementation: prose inside a template comment must not spell out the comment's own closing delimiter, or the parser ends the comment early and the rest is emitted as manifest content — the render then fails with `did not find expected node content`, pointing at the template and not at the sentence.
- **A conditional `containerPort` needs a conditional rule.** `minecraft-server`'s RCON port is rendered only when `server.rcon.enabled` is true; a rule without the same gate names a port the pod does not declare, and the render test rejects it.
- **A policy applies to traffic that arrived through a Service, and that is why the `ingressController` rule works at all.** A `NetworkPolicy` is evaluated at the pod's network interface and a Service is only destination NAT, so going through a ClusterIP does not exempt anything. What the Service *does* change is the **source address**: pod → ClusterIP → pod **preserves** it, which is what lets a `podSelector` match the Traefik pod or a `namespaceSelector` match a consumer's namespace. Traffic from outside through a `NodePort` with the default `externalTrafficPolicy: Cluster` is **masqueraded to a node**, which is why those rules cannot be restricted by source at all and are left open instead (`stalwart`'s template carries that second half; the first half is the load-bearing one).
- **A policy applies to traffic that arrived through a Service, and that is why the `ingressController` rule works at all.** A `NetworkPolicy` is evaluated at the pod's network interface and a Service is only destination NAT, so going through a ClusterIP does not exempt anything. What the Service *does* change is the **source address**: pod → ClusterIP → pod **preserves** it, which is what lets a `podSelector` match the Traefik pod or a `namespaceSelector` match a consumer's namespace. **And what it also changes is the destination port, which is the half #136 was paid for:** "a Service is only destination NAT" is exactly why an *egress* rule has to name the pod port — the same sentence, finished in the other direction. It was written here before the wrong rule was written there; the information was present and the conclusion was missing, which is the sort of gap no check catches. Traffic from outside through a `NodePort` with the default `externalTrafficPolicy: Cluster` is **masqueraded to a node**, which is why those rules cannot be restricted by source at all and are left open instead (`stalwart`'s template carries that second half; the first half is the load-bearing one).
#### The one cluster setting this convention depends on and does not control
+391
View File
@@ -819,6 +819,397 @@ PY
fi
done
# --- Assertion 6: the CROSS-CHART egress ports (Gitea #136) ------------------
# The only assertion here that needs more than one chart's render, and the
# reason it exists is an outage. paperless-ngx' egress rule for Tika and
# Gotenberg named 80 - the port those two charts' SERVICES listen on - and
# justified it in a template comment. kube-proxy does a ClusterIP Service's
# destination NAT BEFORE the egress policy is evaluated at the pod's network
# interface, so the policy sees podIP:targetPort; 80 matched nothing, and the
# release that shipped it cut Paperless off from both services on 2026-10-15.
# The symptom was a TIMEOUT, not "connection refused", and no error mentioned a
# NetworkPolicy.
#
# WHY ASSERTION 2 CANNOT CATCH THIS, and why that is not a defect in it: it
# holds ports against the pod spec the policy SELECTS, which is right for
# ingress and meaningless for egress - an egress port belongs to the
# DESTINATION. For a database, a cache or the web that destination is not in any
# render this repository has. It is in a render only when one chart of THIS
# repository dials the Service of another, and paperless-ngx -> apache-tika,
# gotenberg is the single place where that happens (checked across the
# repository for #136; every other egress port is either a 1:1 Service, so the
# mistake is invisible, or outside the cluster, where there is no DNAT at all
# and the port named IS the real one).
#
# So the table below is deliberately not a discovered list: a chart cannot know
# which Service another chart dials, because the address lives in a value or a
# 1Password item. Adding a row is the price of the next cross-chart egress rule,
# and a row that names a chart which no longer exists FAILS rather than being
# skipped - the failure path was executed, like the others.
#
# WHAT IS ASSERTED, per row, on TWO renders of the consumer:
# gate ON - the consumer's egress ports must CONTAIN every pod port behind
# the peers' Services, resolved through the peers' own renders
# (Service port -> targetPort -> containerPort, i.e. exactly the
# translation the policy is evaluated after).
# gate OFF - the containment requirement does NOT apply (the rule is gated,
# and a release without the feature must not carry a rule that
# permits nothing), and those ports must be absent.
#
# Built into THIS script rather than added as a new test with its own pipeline
# job, because everything it needs is already here: the job runs over every
# chart on every pull request (so it cannot be skipped by a path filter), Helm
# is already the YAML parser through ci/yaml2json, and the verdict, grouping and
# summary plumbing is shared. A second script would duplicate all of it to add
# one comparison.
#
# consumer|peer charts|gate value|extra --set needed to render the gate ON
EGRESS_PEER_ROWS="paperless-ngx|apache-tika,gotenberg|paperless.files.tikaGotenberg.enabled|paperless.files.tikaGotenberg.tikaEndpoint=http://tika.ci.local,paperless.files.tikaGotenberg.gotenbergEndpoint=http://gotenberg.ci.local"
# Renders one chart with the same values precedence the loop above uses and
# leaves the parsed JSON in $2. The render/split/parse steps are repeated here
# instead of refactored out of the loop on purpose: the five assertions above
# are load-bearing and were not worth disturbing for this.
render_chart_json() {
RCJ_CHART="$1"
RCJ_OUT="$2"
shift 2
RCJ_VALUES=()
RCJ_LABEL="chart defaults"
if [ -f "$CI_DIR/$RCJ_CHART/test-values.yaml" ]; then
RCJ_VALUES=(-f "$CI_DIR/$RCJ_CHART/test-values.yaml")
RCJ_LABEL="ci/$RCJ_CHART/test-values.yaml"
elif [ -f "$CI_DIR/trivy/values/$RCJ_CHART.yaml" ]; then
RCJ_VALUES=(-f "$CI_DIR/trivy/values/$RCJ_CHART.yaml")
RCJ_LABEL="ci/trivy/values/$RCJ_CHART.yaml"
fi
"$ROOT/ci/helm-dependencies.sh" "$ROOT/$RCJ_CHART" >/dev/null || return 1
RCJ_MANIFEST="$RCJ_OUT.manifest.yaml"
helm template "$RELEASE" "$ROOT/$RCJ_CHART" \
${RCJ_VALUES[@]+"${RCJ_VALUES[@]}"} "$@" >"$RCJ_MANIFEST" 2>"$WORK/xerr" || return 1
awk '
BEGIN { print "docs:"; started = 0 }
/^---[ \t]*$/ { print " -"; started = 1; next }
{
if (!started) { print " -"; started = 1 }
if (length($0) == 0) print ""; else print " " $0
}
' "$RCJ_MANIFEST" >"$RCJ_MANIFEST.docs"
helm template y2j "$YAML2JSON" -f "$RCJ_MANIFEST.docs" 2>"$WORK/xerr" \
| awk '/^\{/ { print; exit }' >"$RCJ_OUT" || return 1
[ -s "$RCJ_OUT" ] || return 1
return 0
}
CROSS_ROWS=0
for ROW in $EGRESS_PEER_ROWS; do
CONSUMER="$(printf '%s' "$ROW" | cut -d'|' -f1)"
PEERS="$(printf '%s' "$ROW" | cut -d'|' -f2 | tr ',' ' ')"
GATE="$(printf '%s' "$ROW" | cut -d'|' -f3)"
GATE_SETS="$(printf '%s' "$ROW" | cut -d'|' -f4)"
# Only when the consumer is in this run's chart list, so a single-chart
# invocation stays single-chart. The pipeline passes no arguments, so in CI
# every row always runs.
IN_RUN=""
for C in "${CHARTS[@]}"; do [ "$C" = "$CONSUMER" ] && IN_RUN=1; done
[ -n "$IN_RUN" ] || continue
echo "::group::cross-chart egress: $CONSUMER -> $PEERS"
CROSS_ROWS=$((CROSS_ROWS + 1))
ROW_FAIL=""
if [ ! -f "$ROOT/$CONSUMER/Chart.yaml" ]; then
echo " FAIL $CONSUMER is named in EGRESS_PEER_ROWS but is not a chart directory"
echo "::error title=cross-chart egress::$CONSUMER is named in EGRESS_PEER_ROWS in ci/run-networkpolicy-test.sh but is not a chart directory. If the chart was renamed or removed, update that table - a row that silently does nothing is how this check would decay back into the state that caused Gitea #136."
ROW_FAIL=1
fi
PEER_ARGS=()
for PEER in $PEERS; do
if [ ! -f "$ROOT/$PEER/Chart.yaml" ]; then
echo " FAIL peer chart $PEER does not exist"
echo "::error title=cross-chart egress::$CONSUMER's egress rule is checked against the chart $PEER, which is not a chart directory of this repository. Either it was renamed - in which case the row in EGRESS_PEER_ROWS in ci/run-networkpolicy-test.sh has to follow, and $CONSUMER's rule probably too - or the row is stale. Not skipped, because an unchecked row reads exactly like a passing one."
ROW_FAIL=1
continue
fi
if ! render_chart_json "$PEER" "$WORK/peer-$PEER.json"; then
sed 's/^/ /' "$WORK/xerr"
echo " FAIL peer chart $PEER does not render"
echo "::error title=cross-chart egress::the peer chart $PEER does not render with its CI values, so the pod ports behind its Services could not be read and $CONSUMER's egress rule could not be checked against them."
ROW_FAIL=1
continue
fi
PEER_ARGS+=("$PEER=$WORK/peer-$PEER.json")
done
ON_JSON="$WORK/consumer-$CONSUMER-on.json"
OFF_JSON="$WORK/consumer-$CONSUMER-off.json"
if [ -z "$ROW_FAIL" ]; then
SET_ARGS=(--set "$GATE=true")
OLD_IFS="$IFS"; IFS=','
for S in $GATE_SETS; do [ -n "$S" ] && SET_ARGS+=(--set "$S"); done
IFS="$OLD_IFS"
if ! render_chart_json "$CONSUMER" "$ON_JSON" "${SET_ARGS[@]}"; then
sed 's/^/ /' "$WORK/xerr"
echo " FAIL $CONSUMER does not render with $GATE=true"
echo "::error title=cross-chart egress::$CONSUMER does not render with $GATE=true and the --set values from EGRESS_PEER_ROWS, so its egress rule for $PEERS was never produced and nothing was checked. If the feature needs further values, add them to that row."
ROW_FAIL=1
elif ! render_chart_json "$CONSUMER" "$OFF_JSON" --set "$GATE=false"; then
sed 's/^/ /' "$WORK/xerr"
echo " FAIL $CONSUMER does not render with $GATE=false"
echo "::error title=cross-chart egress::$CONSUMER does not render with $GATE=false, so the gated-off case - in which this assertion must NOT fire - could not be checked."
ROW_FAIL=1
fi
fi
if [ -z "$ROW_FAIL" ]; then
set +e
python3 - "$CONSUMER" "$RELEASE" "$GATE" "$ON_JSON" "$OFF_JSON" "${PEER_ARGS[@]}" \
>"$WORK/cross-$CONSUMER.out" 2>"$WORK/cross-$CONSUMER.pyerr" <<'PY'
import json, sys
consumer, release, gate, on_path, off_path = sys.argv[1:6]
peers = [a.split("=", 1) for a in sys.argv[6:]]
lines = []
def ok(msg): lines.append("OK|%s: %s" % (consumer, msg))
def skip(msg): lines.append("SKIP|%s: %s" % (consumer, msg))
def fail(msg): lines.append("FAIL|%s: %s" % (consumer, msg))
def docs_of(path):
return [d for d in (json.load(open(path)).get("docs") or []) if isinstance(d, dict)]
def pod_templates(node):
out = []
if isinstance(node, dict):
for key, value in node.items():
if (key == "template" and isinstance(value, dict)
and isinstance(value.get("spec"), dict)
and "containers" in value["spec"]):
out.append(value)
out.extend(pod_templates(value))
elif isinstance(node, list):
for value in node:
out.extend(pod_templates(value))
return out
def labels_of(template):
meta = template.get("metadata")
labels = meta.get("labels") if isinstance(meta, dict) else None
return labels if isinstance(labels, dict) else {}
def matches(selector, labels):
for key, value in selector.items():
if key not in labels or str(labels[key]) != str(value):
return False
return True
def container_ports(template):
out = []
spec = template.get("spec") or {}
for key in ("containers", "initContainers"):
for container in spec.get(key) or []:
if not isinstance(container, dict):
continue
for port in container.get("ports") or []:
if isinstance(port, dict) and port.get("containerPort") is not None:
out.append((int(port["containerPort"]), port.get("name")))
return out
def service_pod_ports(path, chart):
"""The pod ports a client reaches THROUGH this chart's Services, as
{pod port: [evidence]}. Resolved Service port -> targetPort -> containerPort,
because that chain IS the destination NAT the egress policy is evaluated
after: the packet leaving the consumer carries the Service port, and the
packet the policy sees carries the pod port."""
docs = docs_of(path)
templates = []
for doc in docs:
templates.extend(pod_templates(doc))
out, unresolved = {}, []
for svc in docs:
if svc.get("kind") != "Service":
continue
spec = svc.get("spec") or {}
selector = spec.get("selector")
hits = [t for t in templates
if isinstance(selector, dict) and matches(selector, labels_of(t))]
by_name = dict((n, num) for t in hits for num, n in container_ports(t) if n)
svc_name = ((svc.get("metadata") or {}).get("name")) or "<unnamed Service>"
for port in spec.get("ports") or []:
if not isinstance(port, dict):
continue
target = port.get("targetPort")
if target is None:
target = port.get("port")
number = None
if isinstance(target, int):
number = target
elif isinstance(target, str):
number = int(target) if target.isdigit() else by_name.get(target)
if number is None:
unresolved.append("%s %s: targetPort %r" % (chart, svc_name, target))
continue
out.setdefault(number, []).append(
"%s %s: port %s -> targetPort %r -> pod port %d"
% (chart, svc_name, port.get("port"), target, number))
return out, unresolved
def egress_ports(path):
"""(numbers, policy names, has a rule with no ports at all) for the
consumer's OWN policy only - recognised by the <release>-<chart>- name
prefix, so a subchart's policy in the same render cannot satisfy a rule the
parent chart is responsible for."""
prefix = "%s-%s-" % (release, consumer)
numbers, names, open_rule = set(), [], False
for doc in docs_of(path):
if doc.get("kind") != "NetworkPolicy":
continue
name = ((doc.get("metadata") or {}).get("name")) or ""
if not name.startswith(prefix):
continue
names.append(name)
spec = doc.get("spec") if isinstance(doc.get("spec"), dict) else {}
for rule in spec.get("egress") or []:
if not isinstance(rule, dict):
continue
if not rule.get("ports"):
open_rule = True
continue
for port in rule["ports"]:
if not isinstance(port, dict):
continue
value = port.get("port")
if isinstance(value, int):
numbers.add(value)
elif isinstance(value, str) and value.isdigit():
numbers.add(int(value))
return numbers, names, open_rule
required, unresolved = {}, []
for chart, path in peers:
ports, bad = service_pod_ports(path, chart)
for number, evidence in ports.items():
required.setdefault(number, []).extend(evidence)
unresolved.extend(bad)
for note in unresolved:
skip("a Service targetPort of a peer chart could not be resolved to a "
"containerPort, so that port is not part of the requirement (%s). "
"Reported, not counted as passing." % note)
on_numbers, on_names, on_open = egress_ports(on_path)
off_numbers, off_names, _off_open = egress_ports(off_path)
if not required:
fail("the peer charts render no Service whose port resolves to a pod port, "
"so this assertion would have required nothing at all. That is "
"indistinguishable from a passing check and is treated as a failure: "
"either the peer charts stopped shipping a Service, or their targetPort "
"no longer resolves.")
elif not on_names:
fail("with %s=true the render contains no NetworkPolicy named %s-%s-*, so "
"the egress rule this assertion is about was not found. The policy's "
"name is the convention {{ .Release.Name }}-{{ .Chart.Name }}-<kind>; if "
"it changed, this check has to follow."
% (gate, release, consumer))
elif on_open:
skip("an egress rule of %s names NO port, which allows every port - so "
"containment of the peers' pod ports is satisfied without saying "
"anything. Reported, never counted as passing."
% ", ".join(on_names))
else:
missing = sorted(n for n in required if n not in on_numbers)
if missing:
fail("with %s=true, %s permits outbound %s but NOT %s - and those are the "
"POD ports behind the peer charts' Services: %s. An egress rule is "
"evaluated at the pod's network interface, AFTER kube-proxy has "
"translated the ClusterIP Service's destination (DNAT), so the policy "
"sees podIP:targetPort and a rule naming the SERVICE port matches "
"nothing wherever the two numbers differ. This is Gitea #136 "
"exactly: it cost an outage on 2026-10-15, and the symptom was a "
"TIMEOUT rather than \"connection refused\", with no error "
"mentioning a NetworkPolicy. Name the pod port(s) in the rule."
% (gate, ", ".join(on_names),
", ".join(str(n) for n in sorted(on_numbers)),
", ".join(str(n) for n in missing),
"; ".join(e for n in missing for e in required[n])))
else:
ok("with %s=true, %s permits every pod port behind the peer charts' "
"Services: %s"
% (gate, ", ".join(on_names),
"; ".join(e for n in sorted(required) for e in required[n])))
# The gated-off half. The requirement above must NOT apply here - a release
# without the feature is not expected to reach the peers - and the ports must
# be gone, or the rule is no longer gated and permits something the release
# has no use for.
present = sorted(n for n in required if n in off_numbers)
if not off_names:
skip("with %s=false the render contains no NetworkPolicy of this chart, "
"so the gated-off case could not be read." % gate)
elif present:
fail("with %s=false, %s still permits outbound %s, the peer charts' pod "
"port(s). Either the rule is no longer gated on that value - a rule "
"that permits what the release cannot use - or the CI values add "
"those ports by hand, which would make the gate-on assertion above "
"pass without the rule existing."
% (gate, ", ".join(off_names),
", ".join(str(n) for n in present)))
else:
ok("with %s=false the rule is absent (egress: %s), so the containment "
"requirement correctly does not apply and no release without the "
"feature carries a rule that permits nothing."
% (gate, ", ".join(str(n) for n in sorted(off_numbers)) or "none"))
for line in lines:
print(line)
PY
RC=$?
set -e
if [ "$RC" -gt 1 ]; then
sed 's/^/ /' "$WORK/cross-$CONSUMER.pyerr"
echo "::error title=cross-chart egress::the cross-chart egress comparison itself failed for $CONSUMER (python exit $RC). Treated as a failure, never as a pass."
ROW_FAIL=1
else
while IFS='|' read -r VERDICT DETAIL; do
case "$VERDICT" in
OK)
TOTAL_ASSERTIONS=$((TOTAL_ASSERTIONS + 1))
echo " ok $DETAIL"
;;
SKIP) echo " gap $DETAIL" ;;
FAIL)
ROW_FAIL=1
echo " FAIL $DETAIL"
echo "::error title=cross-chart egress::$DETAIL"
;;
*) [ -n "$VERDICT" ] && echo " $VERDICT $DETAIL" ;;
esac
done <"$WORK/cross-$CONSUMER.out"
fi
fi
echo "::endgroup::"
if [ -n "$ROW_FAIL" ]; then
FAILED="$FAILED $CONSUMER(cross-chart)"
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CONSUMER\` → \`$(printf '%s' "$PEERS" | tr ' ' ',')\` | **failed** | – | – |"$'\n'
echo "cross-chart egress $CONSUMER: FAILED"
else
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CONSUMER\` → \`$(printf '%s' "$PEERS" | tr ' ' ',')\` | ok | 2 | 0 |"$'\n'
echo "cross-chart egress $CONSUMER: ok"
fi
done
# --- Report ------------------------------------------------------------------
count() { set -- $1; echo "$#"; }
listing() { set -- $1; [ "$#" -eq 0 ] && { echo "–"; return; }; printf '`%s`' "$1"; shift; for C in "$@"; do printf ', `%s`' "$C"; done; echo; }