jklein
1894e7d465
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Validate Helm Charts / Resource quantities (pull_request) Successful in 10s
Validate Helm Charts / Selector match (pull_request) Successful in 9s
Validate Helm Charts / HTTPRoute match (pull_request) Successful in 9s
Validate Helm Charts / NetworkPolicy (pull_request) Successful in 12s
Validate Helm Charts / Bump level (pull_request) Successful in 4s
Validate Helm Charts / YAML scalars (pull_request) Successful in 10s
Validate Helm Charts / Build paperless-ngx (pull_request) Successful in 6s
Validate Helm Charts / Trivy config paperless-ngx (pull_request) Successful in 20s
Install-test / Install-test paperless-ngx (pull_request) Successful in 3m2s
Validate Helm Charts / Install-test (pull_request) Successful in 3m2s
The four assertions from Gitea #5 hold ports against the pod spec the policy SELECTS. That is right for ingress and meaningless for egress: an egress port belongs to the destination, and for a database, a cache or the web that destination is in no render this repository has. So the mistake behind #136 - an egress rule naming a ClusterIP Service's port, which kube-proxy has already translated away by the time the policy is evaluated at the pod's interface - was not catchable within one chart. It is catchable across two, and paperless-ngx -> apache-tika, gotenberg is the only place in the repository where a chart dials another chart's Service (the three other policies naming port 80 aim at the internet, where there is no DNAT and 80 is the real port). The pairs are an explicit table rather than a discovery, because no render can know which Service another chart dials - the address is a value or a 1Password field - and a row naming a chart that no longer exists fails instead of being skipped. Each row is checked on two renders of the consumer: with the feature gate ON the egress ports must CONTAIN every pod port behind the peers' Services, resolved Service port -> targetPort -> containerPort, which is exactly the translation the policy is evaluated after; with the gate OFF the requirement must not apply and those ports must be absent, so a gated rule cannot quietly become one that permits what the release cannot use. The evidence line prints the chain ("port 80 -> targetPort 'tika-port' -> pod port 9998"), so a reader sees the DNAT rather than two numbers. Built into run-networkpolicy-test.sh rather than added as a new script with its own job: that job already runs over every chart on every PR, Helm is already the YAML parser through ci/yaml2json, and the verdict and summary plumbing is shared. python3 standard library only, no yq, no pyyaml, no Docker. Six fail paths executed, not assumed: a required port removed, a wrong number in its place, the peer chart renamed away, the policy name no longer following the <release>-<chart>-<kind> convention, the peers rendering no Service at all (an empty requirement reads like a satisfied one, so it fails), and the rule made unconditional, which is the fail path of the gate-off half. Measured: 30 charts, 210 assertions held (208 before), 0 failed, ~15s, no cluster. README's stated figure was 206 and is corrected here. Part of #136
1266 lines
61 KiB
Bash
Executable File
1266 lines
61 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
# NetworkPolicy render test: every NetworkPolicy a chart renders is held against
|
||
# the pod specs, probes and Services of the SAME render (Gitea #5).
|
||
#
|
||
# Usage: ci/run-networkpolicy-test.sh [chart ...] (no argument = all charts)
|
||
#
|
||
# WHY THIS EXISTS, AND WHY A GREEN INSTALL-TEST DOES NOT COVER IT
|
||
#
|
||
# Since #5 every chart in this repository ships a NetworkPolicy, on by default.
|
||
# That is 29 objects whose failure modes are all SILENT in the same way: the API
|
||
# server accepts a policy that filters nothing, and it accepts a policy that
|
||
# filters everything. Four mistakes are reachable by copy, and each one has a
|
||
# shape a render can decide:
|
||
#
|
||
# A SELECTOR THAT MATCHES NO POD. The object sits in the namespace, is listed
|
||
# by kubectl, reads as a boundary, and filters nothing whatsoever. The usual
|
||
# cause is a label renamed in the workload template - a different file, a
|
||
# different diff. Nothing in a rollout notices, because nothing is denied.
|
||
# A NODEPORT NUMBER WHERE A POD PORT BELONGS. A NetworkPolicy is evaluated at
|
||
# the POD's network interface, after kube-proxy has already translated the
|
||
# NodePort to the targetPort. A rule naming 30025 where the pod listens on
|
||
# 25 matches nothing: accepted, strict-looking, and - being the only rule
|
||
# for that port - blocking every inbound connection instead. Four charts
|
||
# here publish NodePorts, so this is not a hypothetical.
|
||
# AN EMPTY "from: []". Allows every source, exactly like no "from" at all,
|
||
# while reading in a diff and in kubectl output as a restriction. The one
|
||
# mistake in this list that makes a policy weaker while looking stricter.
|
||
# A RESTRICTED PROBE PORT. The kubelet's liveness/readiness/startup probes
|
||
# come from the IP of the WORKER NODE running the pod, an address no chart
|
||
# can know. A rule that restricts a probed port by source without that
|
||
# address drops every probe: the pod never becomes Ready, the rollout times
|
||
# out, and the symptom reads as a broken application rather than as a
|
||
# policy. This is the one the install-test does catch - after minutes, at
|
||
# the end of a 29-chart matrix, with its cause three steps away.
|
||
#
|
||
# The install-test is the real net under the rollout and it is not replaced
|
||
# here: it proves ENFORCEMENT, on a live k3s whose policy controller is
|
||
# running. What it cannot do is cover satisfactory and template-chart, which
|
||
# have no install-test at all (ci/excluded-charts.txt), and it cannot run on a
|
||
# pull request that touches no chart. This test is the cheap half: no cluster,
|
||
# no Docker, a few seconds for the whole repository.
|
||
#
|
||
# Over ALL charts and not the changed ones, for the same reason as
|
||
# ci/run-selector-match-test.sh and ci/run-quantity-render-test.sh: the mistake
|
||
# arrives by copy into a NEW chart, whose own diff nobody reads for it, and
|
||
# running over everything means there is no list to maintain - adding a chart
|
||
# enrols it. That property is what the first assertion below leans on, and it
|
||
# is not theoretical: garage (#81) and keycloak (#83) were both created AFTER
|
||
# #5 was filed and both arrived with no policy at all, keycloak with an
|
||
# unauthenticated management port on 9000.
|
||
#
|
||
# HOW THE YAML IS PARSED, AND WHY NOT WITH A PARSER WRITTEN HERE
|
||
#
|
||
# The runners have neither pyyaml nor yq - measured, the same constraint
|
||
# ci/run-trivy-config.sh, ci/run-quantity-render-test.sh and
|
||
# ci/run-selector-match-test.sh work under. So HELM is the parser: ci/yaml2json
|
||
# is a one-template chart that prints its values back as JSON, the rendered
|
||
# manifest stream is handed to it as a VALUES FILE, and python3 reads the result
|
||
# with its standard library. What IS done textually is only the document split,
|
||
# on a line that is exactly "---" in column one, and the consequence of getting
|
||
# that wrong is not a false pass: the fragment stops being valid YAML and helm
|
||
# exits 1.
|
||
#
|
||
# Assertion 5 is the exception and works on the raw rendered YAML on purpose:
|
||
# comments are gone by the time yaml2json has read the stream, and a comment is
|
||
# exactly what it looks for.
|
||
#
|
||
# WHAT IS DELIBERATELY NOT CHECKED, named so the gaps are not mistaken for
|
||
# coverage:
|
||
#
|
||
# - THAT TRAFFIC STILL FLOWS. Nothing here, and nothing in CI, proves the path
|
||
# from Traefik to a pod: no ingress controller runs in the install-test
|
||
# cluster. What is proven is the probe path, acceptance by the API server,
|
||
# and the render-side assertions below.
|
||
# - EGRESS DESTINATIONS. A NetworkPolicy matches CIDRs and selectors and never
|
||
# hostnames, so "reaches its database" is not expressible in the object and
|
||
# not checkable here. Egress ports are deliberately NOT held against the
|
||
# pod's containerPorts either - an egress port is a port on the DESTINATION,
|
||
# and that comparison would fail every chart that reaches a database.
|
||
# - SELECTORS EXPRESSED ONLY THROUGH matchExpressions. Evaluating a set-based
|
||
# requirement means reimplementing label-selector semantics, i.e. a second
|
||
# parser. Those are REPORTED as "not checked" and never counted as passing,
|
||
# because a green tick for a code path nobody entered is worse than an
|
||
# honest gap (README, "Two rules for every check added here").
|
||
# - PORT RANGES (endPort). Same treatment: reported, never counted.
|
||
#
|
||
# AND THE ANTI-VACUUM GUARDS, because this test could be green over nothing:
|
||
#
|
||
# 1. A chart that renders NO NetworkPolicy fails. Since #5 that is a
|
||
# regression, not a style question - see the garage/keycloak note above.
|
||
# 2. ci/networkpolicy/<chart>.yaml is an overlay applied last, whose only job
|
||
# is to render a code path the chart's own defaults do not reach - above
|
||
# all the RESTRICTED probe rule, which is only rendered once probeSources
|
||
# is non-empty. A chart that HAS an overlay and still renders no restricted
|
||
# rule FAILS, because the overlay exists for exactly that.
|
||
# 3. If no assertion at all was evaluated across the whole run, the run fails.
|
||
set -euo pipefail
|
||
|
||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||
CI_DIR="$ROOT/ci"
|
||
YAML2JSON="$CI_DIR/yaml2json"
|
||
OVERLAY_DIR="$CI_DIR/networkpolicy"
|
||
RELEASE="netpol"
|
||
|
||
WORK="$(mktemp -d)"
|
||
trap 'rm -rf "$WORK"' EXIT
|
||
|
||
# --- Which charts -----------------------------------------------------------
|
||
CHARTS=()
|
||
if [ "$#" -gt 0 ]; then
|
||
CHARTS=("$@")
|
||
else
|
||
for DIR in "$ROOT"/*/; do
|
||
[ -f "$DIR/Chart.yaml" ] || continue
|
||
CHARTS+=("$(basename "$DIR")")
|
||
done
|
||
fi
|
||
if [ "${#CHARTS[@]}" -eq 0 ]; then
|
||
echo "ERROR: no chart directories found under $ROOT" >&2
|
||
exit 1
|
||
fi
|
||
|
||
FAILED=""
|
||
CHECKED=""
|
||
WITH_GAPS=""
|
||
TOTAL_ASSERTIONS=0
|
||
SUMMARY_ROWS=""
|
||
|
||
for CHART in "${CHARTS[@]}"; do
|
||
if [ ! -f "$ROOT/$CHART/Chart.yaml" ]; then
|
||
echo "ERROR: '$CHART' is not a chart directory" >&2
|
||
exit 1
|
||
fi
|
||
|
||
# Same reason as in ci/run-selector-match-test.sh: a chart with a declared
|
||
# dependency cannot be rendered at all while charts/ is empty, and this job
|
||
# runs over EVERY chart on EVERY pull request - so without this line a PR
|
||
# touching nothing but a README would go red on the charts that have one.
|
||
"$ROOT/ci/helm-dependencies.sh" "$ROOT/$CHART"
|
||
|
||
# Values: the same precedence the other render tests use, so a chart's CI
|
||
# values are maintained in one place and satisfy its required() values.
|
||
VALUES_ARGS=()
|
||
VALUES_LABEL="chart defaults"
|
||
if [ -f "$CI_DIR/$CHART/test-values.yaml" ]; then
|
||
VALUES_ARGS=(-f "$CI_DIR/$CHART/test-values.yaml")
|
||
VALUES_LABEL="ci/$CHART/test-values.yaml"
|
||
elif [ -f "$CI_DIR/trivy/values/$CHART.yaml" ]; then
|
||
VALUES_ARGS=(-f "$CI_DIR/trivy/values/$CHART.yaml")
|
||
VALUES_LABEL="ci/trivy/values/$CHART.yaml"
|
||
fi
|
||
HAS_OVERLAY=""
|
||
if [ -f "$OVERLAY_DIR/$CHART.yaml" ]; then
|
||
# Last, so it wins over the install-test values.
|
||
VALUES_ARGS+=(-f "$OVERLAY_DIR/$CHART.yaml")
|
||
VALUES_LABEL="$VALUES_LABEL + ci/networkpolicy/$CHART.yaml"
|
||
HAS_OVERLAY=1
|
||
fi
|
||
|
||
echo "::group::$CHART ($VALUES_LABEL)"
|
||
|
||
RENDER="$WORK/$CHART.manifest.yaml"
|
||
if ! helm template "$RELEASE" "$ROOT/$CHART" \
|
||
${VALUES_ARGS[@]+"${VALUES_ARGS[@]}"} >"$RENDER" 2>"$WORK/err"; then
|
||
sed 's/^/ /' "$WORK/err"
|
||
echo "::endgroup::"
|
||
echo "::error title=networkpolicy $CHART::$CHART does not render with its CI values, so no policy could be checked. Reproduce: helm template $RELEASE $CHART $(printf '%s ' "${VALUES_ARGS[@]+${VALUES_ARGS[@]}}")"
|
||
FAILED="$FAILED $CHART"
|
||
continue
|
||
fi
|
||
|
||
# --- The document split, and the only textual step before the parser -------
|
||
# One YAML sequence item per rendered document, content indented by four and
|
||
# the item marker on a line of its own - so a document whose first line is a
|
||
# comment ("# Source: ...") needs no special case. A wrong split does not
|
||
# produce a false pass: the result stops being parsable and helm exits 1.
|
||
DOCS="$WORK/$CHART.docs.yaml"
|
||
awk '
|
||
BEGIN { print "docs:"; started = 0 }
|
||
/^---[ \t]*$/ { print " -"; started = 1; next }
|
||
{
|
||
if (!started) { print " -"; started = 1 }
|
||
if (length($0) == 0) print ""; else print " " $0
|
||
}
|
||
' "$RENDER" >"$DOCS"
|
||
|
||
JSON="$WORK/$CHART.json"
|
||
if ! helm template y2j "$YAML2JSON" -f "$DOCS" 2>"$WORK/err" \
|
||
| awk '/^\{/ { print; exit }' >"$JSON"; then
|
||
sed 's/^/ /' "$WORK/err"
|
||
echo "::endgroup::"
|
||
echo "::error title=networkpolicy $CHART::the rendered manifests of $CHART could not be re-read as YAML by ci/yaml2json. That is a parse failure, not a policy finding: either the render contains a document this script's '---' split cut in the wrong place, or the manifest is not valid YAML. Helm's own message is in the job log."
|
||
FAILED="$FAILED $CHART"
|
||
continue
|
||
fi
|
||
if [ ! -s "$JSON" ]; then
|
||
echo "::endgroup::"
|
||
echo "::error title=networkpolicy $CHART::ci/yaml2json produced no JSON for $CHART - the manifests were not re-read, so nothing was checked."
|
||
FAILED="$FAILED $CHART"
|
||
continue
|
||
fi
|
||
|
||
# --- The assertions --------------------------------------------------------
|
||
# Inline and on the standard library only, as in ci/run-selector-match-test.sh
|
||
# and ci/run-quantity-render-test.sh: json, because the YAML is already gone
|
||
# by this point, plus re for the two textual checks that need the raw render.
|
||
# It writes one "VERDICT|detail" line per assertion and never decides the exit
|
||
# code itself - the loop below does, so a verdict can never be lost to a
|
||
# swallowed status.
|
||
set +e
|
||
python3 - "$CHART" "$JSON" "$RENDER" "$ROOT/$CHART" >"$WORK/$CHART.out" 2>"$WORK/$CHART.pyerr" <<'PY'
|
||
import json, re, sys
|
||
|
||
chart, json_path, yaml_path, chart_dir = sys.argv[1:5]
|
||
|
||
lines = []
|
||
def ok(msg): lines.append("OK|%s: %s" % (chart, msg))
|
||
def skip(msg): lines.append("SKIP|%s: %s" % (chart, msg))
|
||
def fail(msg): lines.append("FAIL|%s: %s" % (chart, msg))
|
||
|
||
docs = [d for d in (json.load(open(json_path)).get("docs") or []) if isinstance(d, dict)]
|
||
|
||
|
||
def pod_templates(node, path="$"):
|
||
"""Every pod template of the render, found by walking rather than by
|
||
enumerating kinds - so a CronJob's spec.jobTemplate.spec.template is covered
|
||
without a special case, and so is whatever workload kind a chart adds next.
|
||
Same shape as ci/run-selector-match-test.sh."""
|
||
out = []
|
||
if isinstance(node, dict):
|
||
for key, value in node.items():
|
||
here = "%s.%s" % (path, key)
|
||
if (key == "template" and isinstance(value, dict)
|
||
and isinstance(value.get("spec"), dict)
|
||
and "containers" in value["spec"]):
|
||
out.append((here, value))
|
||
out.extend(pod_templates(value, here))
|
||
elif isinstance(node, list):
|
||
for i, value in enumerate(node):
|
||
out.extend(pod_templates(value, "%s[%d]" % (path, i)))
|
||
return out
|
||
|
||
|
||
def name_of(doc):
|
||
meta = doc.get("metadata")
|
||
if isinstance(meta, dict) and meta.get("name"):
|
||
return str(meta["name"])
|
||
return "<unnamed %s>" % doc.get("kind", "object")
|
||
|
||
|
||
def labels_of(template):
|
||
meta = template.get("metadata")
|
||
labels = meta.get("labels") if isinstance(meta, dict) else None
|
||
return labels if isinstance(labels, dict) else {}
|
||
|
||
|
||
def matches(selector, labels):
|
||
"""matchLabels only. Every key must be present with the same value; the
|
||
comparison is on the string form, because YAML true and the string "true"
|
||
are the same label value to Kubernetes."""
|
||
for key, value in selector.items():
|
||
if key not in labels or str(labels[key]) != str(value):
|
||
return False
|
||
return True
|
||
|
||
|
||
all_templates = []
|
||
for doc in docs:
|
||
all_templates.extend(pod_templates(doc, "%s/%s" % (doc.get("kind", "?"), name_of(doc))))
|
||
|
||
policies = [d for d in docs if d.get("kind") == "NetworkPolicy"]
|
||
services = [d for d in docs if d.get("kind") == "Service"]
|
||
|
||
|
||
def container_ports(template):
|
||
"""[(number, protocol, name)] over containers AND initContainers. Init
|
||
containers are included because a sidecar-style one (restartPolicy: Always)
|
||
keeps listening for the life of the pod, and its port is as real as any
|
||
other."""
|
||
out = []
|
||
spec = template.get("spec") or {}
|
||
for key in ("containers", "initContainers"):
|
||
for container in spec.get(key) or []:
|
||
if not isinstance(container, dict):
|
||
continue
|
||
for port in container.get("ports") or []:
|
||
if not isinstance(port, dict) or port.get("containerPort") is None:
|
||
continue
|
||
out.append((int(port["containerPort"]),
|
||
str(port.get("protocol") or "TCP"), port.get("name")))
|
||
return out
|
||
|
||
|
||
def probe_ports(template):
|
||
"""Ports a NETWORK probe addresses, as {number: [who]}.
|
||
|
||
exec probes are deliberately absent, and that distinction is the whole
|
||
reason this is a function rather than a grep for "Probe": an exec probe runs
|
||
inside the container through the kubelet's runtime and puts no packet on the
|
||
network at all. A chart with exec-only probes (valkey, minecraft-server) can
|
||
therefore restrict its port by source with no probeSources whatsoever, and
|
||
treating its probe as network traffic would demand a value it does not need
|
||
and would weaken the policy it can have."""
|
||
found = {}
|
||
by_name = dict((n, num) for num, _p, n in container_ports(template) if n)
|
||
spec = template.get("spec") or {}
|
||
for key in ("containers", "initContainers"):
|
||
for container in spec.get(key) or []:
|
||
if not isinstance(container, dict):
|
||
continue
|
||
for probe_key in ("livenessProbe", "readinessProbe", "startupProbe"):
|
||
probe = container.get(probe_key)
|
||
if not isinstance(probe, dict):
|
||
continue
|
||
for handler in ("httpGet", "tcpSocket", "grpc"):
|
||
handler_spec = probe.get(handler)
|
||
if not isinstance(handler_spec, dict):
|
||
continue
|
||
raw = handler_spec.get("port")
|
||
number = None
|
||
if isinstance(raw, int):
|
||
number = raw
|
||
elif isinstance(raw, str) and raw.isdigit():
|
||
number = int(raw)
|
||
elif isinstance(raw, str) and raw in by_name:
|
||
number = by_name[raw]
|
||
elif isinstance(raw, str):
|
||
skip("%s of container %s names port %r, which is no "
|
||
"containerPort of the same pod spec - the probe port "
|
||
"could not be resolved, so the probe path was not "
|
||
"checked for it. Reported, not counted as passing."
|
||
% (probe_key, container.get("name"), raw))
|
||
continue
|
||
if number is not None:
|
||
found.setdefault(number, []).append(
|
||
"%s.%s/%s" % (container.get("name"), probe_key, handler))
|
||
return found
|
||
|
||
|
||
def nodeport_pod_ports():
|
||
"""Pod ports published through a NodePort Service of this render, as
|
||
{pod port: [service:nodePort]}. Traffic on these arrives from OUTSIDE the
|
||
cluster and is already masqueraded by the time it reaches the pod, so the
|
||
source address is a node and not the real client - which is why a rule for
|
||
such a port legitimately carries no source selector (assertion 3)."""
|
||
out = {}
|
||
for svc in services:
|
||
spec = svc.get("spec") or {}
|
||
if spec.get("type") != "NodePort" or not isinstance(spec.get("selector"), dict):
|
||
continue
|
||
hits = [t for _w, t in all_templates if matches(spec["selector"], labels_of(t))]
|
||
by_name = dict((n, num) for t in hits for num, _p, n in container_ports(t) if n)
|
||
for port in spec.get("ports") or []:
|
||
if not isinstance(port, dict) or not port.get("nodePort"):
|
||
continue
|
||
target = port.get("targetPort")
|
||
if target is None:
|
||
target = port.get("port")
|
||
number = None
|
||
if isinstance(target, int):
|
||
number = target
|
||
elif isinstance(target, str):
|
||
number = int(target) if target.isdigit() else by_name.get(target)
|
||
if number is not None:
|
||
out.setdefault(number, []).append(
|
||
"%s:%s" % (name_of(svc), port.get("nodePort")))
|
||
return out
|
||
|
||
|
||
nodeports = nodeport_pod_ports()
|
||
raw = open(yaml_path, encoding="utf-8").read()
|
||
|
||
|
||
def read(path):
|
||
try:
|
||
return open(path, encoding="utf-8").read()
|
||
except IOError:
|
||
return ""
|
||
|
||
|
||
notes = read("%s/templates/NOTES.txt" % chart_dir)
|
||
values_yaml = read("%s/values.yaml" % chart_dir)
|
||
# The KEY, not the word. A substring test here passed a chart whose
|
||
# probeSources key had been deleted while the long comment above it still
|
||
# talked about the value - found by running this assertion's failure path, which
|
||
# is the reason the repository runs them (README, "Two rules for every check
|
||
# added here").
|
||
offers_probe_sources = bool(re.search(r"^\s*probeSources\s*:", values_yaml, re.M))
|
||
# The phrase the convention requires NOTES.txt to carry, so that an operator who
|
||
# narrows probeSources has been told what a wrong CIDR costs. A phrase and not a
|
||
# vague "mentions probes", because the sentence that matters is the consequence.
|
||
notes_warns = bool(re.search(r"never becomes? ready", notes, re.I)) and "probeSources" in notes
|
||
|
||
RESTRICTED_RULES = 0
|
||
# Counted separately from RESTRICTED_RULES, and the overlay guard uses THIS one:
|
||
# a chart like keycloak restricts other rules by source in its own defaults, so
|
||
# "some rule has a from" would satisfy the guard without the probe branch ever
|
||
# being rendered - which is the only branch an overlay exists to reach.
|
||
PROBE_RESTRICTED = 0
|
||
|
||
# --- Assertion 0: the chart ships a policy at all ---------------------------
|
||
if not policies:
|
||
fail("renders no NetworkPolicy at all. Since Gitea #5 every chart in this "
|
||
"repository ships one, on by default, and this is the guard that keeps a "
|
||
"NEW chart from arriving without one - which is exactly how garage and "
|
||
"keycloak came to have none, both created after #5 was filed. Copy "
|
||
"templates/xxx.networkpolicy.yaml and the networkPolicy block from "
|
||
"template-chart, or change this test on purpose if the chart truly has "
|
||
"nothing to restrict.")
|
||
|
||
for policy in policies:
|
||
tag = "NetworkPolicy %s" % name_of(policy)
|
||
spec = policy.get("spec") if isinstance(policy.get("spec"), dict) else {}
|
||
policy_types = spec.get("policyTypes")
|
||
if not isinstance(policy_types, list):
|
||
policy_types = []
|
||
|
||
# --- Assertion 1: the podSelector hits a pod spec of the SAME render ----
|
||
selector = spec.get("podSelector")
|
||
selected = []
|
||
if selector is None:
|
||
fail("%s has no podSelector. That field is required by the API - a policy "
|
||
"without it is rejected - and 'podSelector: {}' is the spelling that "
|
||
"selects EVERY pod in the namespace." % tag)
|
||
elif not isinstance(selector, dict):
|
||
fail("%s podSelector is not a mapping (%r)" % (tag, selector))
|
||
elif selector.get("matchExpressions") and not selector.get("matchLabels"):
|
||
skip("%s selects only through matchExpressions, which this test does not "
|
||
"evaluate - doing so would mean reimplementing label-selector "
|
||
"semantics, i.e. a second parser. Reported, not counted as passing."
|
||
% tag)
|
||
selected = [t for _w, t in all_templates]
|
||
else:
|
||
match_labels = selector.get("matchLabels") or {}
|
||
hits = [(w, t) for w, t in all_templates if matches(match_labels, labels_of(t))]
|
||
selected = [t for _w, t in hits]
|
||
if not match_labels:
|
||
ok("%s has an empty podSelector, which selects every pod in the "
|
||
"namespace - legitimate for a namespace-wide policy, as long as "
|
||
"that is what was meant." % tag)
|
||
selected = [t for _w, t in all_templates]
|
||
elif hits:
|
||
ok("%s podSelector %s matches %d pod template(s): %s"
|
||
% (tag, ", ".join("%s=%s" % kv for kv in sorted(match_labels.items())),
|
||
len(hits), ", ".join(w for w, _t in hits)))
|
||
else:
|
||
known = sorted(set("%s=%s" % kv for _w, t in all_templates
|
||
for kv in labels_of(t).items()))
|
||
fail("%s podSelector selects %s, which matches NONE of the %d pod "
|
||
"template(s) in the same render. The object is then accepted, is "
|
||
"listed by kubectl, reads as a boundary and filters nothing at "
|
||
"all - and nothing else in this pipeline notices, because nothing "
|
||
"is denied. The usual cause is a label renamed in the workload "
|
||
"template. Labels actually present on the pod templates: %s"
|
||
% (tag, ", ".join("%s=%s" % kv for kv in sorted(match_labels.items())),
|
||
len(all_templates), "; ".join(known) or "none"))
|
||
|
||
declared = {}
|
||
for t in selected:
|
||
for number, proto, pname in container_ports(t):
|
||
declared.setdefault(number, set()).add(proto)
|
||
if pname:
|
||
declared.setdefault(pname, set()).add(proto)
|
||
probed = {}
|
||
for t in selected:
|
||
for number, who in probe_ports(t).items():
|
||
probed.setdefault(number, []).extend(who)
|
||
|
||
def rule_port_numbers(rule):
|
||
"""The pod ports a rule names, resolved through the selected pods' port
|
||
names. Returns (numbers, unresolved_names)."""
|
||
numbers, unresolved = set(), []
|
||
for port in rule.get("ports") or []:
|
||
if not isinstance(port, dict):
|
||
continue
|
||
value = port.get("port")
|
||
if isinstance(value, int):
|
||
numbers.add(value)
|
||
elif isinstance(value, str) and value.isdigit():
|
||
numbers.add(int(value))
|
||
elif isinstance(value, str):
|
||
hit = [num for t in selected for num, _p, n in container_ports(t)
|
||
if n == value]
|
||
if hit:
|
||
numbers.update(hit)
|
||
else:
|
||
unresolved.append(value)
|
||
return numbers, unresolved
|
||
|
||
# --- Assertion 2: every INGRESS port is a containerPort of those pods ---
|
||
# Ingress only, and that restriction is not an oversight: an egress rule's
|
||
# port is a port on the DESTINATION and has nothing to do with this pod's
|
||
# containerPorts, so holding egress against them would fail every chart that
|
||
# reaches a database.
|
||
for i, rule in enumerate(spec.get("ingress") or []):
|
||
if not isinstance(rule, dict):
|
||
fail("%s ingress[%d] is not a mapping" % (tag, i))
|
||
continue
|
||
for j, port in enumerate(rule.get("ports") or []):
|
||
if not isinstance(port, dict):
|
||
fail("%s ingress[%d].ports[%d] is not a mapping" % (tag, i, j))
|
||
continue
|
||
if port.get("endPort") is not None:
|
||
skip("%s ingress[%d].ports[%d] is a port RANGE (endPort %s), which "
|
||
"this test does not expand. Reported, not counted as passing."
|
||
% (tag, i, j, port.get("endPort")))
|
||
continue
|
||
value = port.get("port")
|
||
if value is None:
|
||
skip("%s ingress[%d].ports[%d] names no port, which allows EVERY "
|
||
"port for the given protocol. Reported, not counted as passing "
|
||
"- and worth a second look, because it is a far wider rule "
|
||
"than it looks." % (tag, i, j))
|
||
continue
|
||
key = int(value) if isinstance(value, int) or (
|
||
isinstance(value, str) and value.isdigit()) else value
|
||
if not selected:
|
||
fail("%s ingress[%d].ports[%d] names port %r, but the policy selects "
|
||
"no pod template at all, so no containerPort could be checked."
|
||
% (tag, i, j, value))
|
||
continue
|
||
if key in declared:
|
||
proto = str(port.get("protocol") or "TCP")
|
||
if proto in declared[key]:
|
||
ok("%s ingress[%d] port %s/%s is a containerPort of the selected "
|
||
"pod spec" % (tag, i, value, proto))
|
||
else:
|
||
fail("%s ingress[%d].ports[%d] allows %s/%s, but the selected pod "
|
||
"spec declares that port only as %s. A policy written for "
|
||
"the wrong protocol filters none of the traffic that "
|
||
"actually arrives."
|
||
% (tag, i, j, value, proto, "/".join(sorted(declared[key]))))
|
||
else:
|
||
numeric = sorted(k for k in declared if isinstance(k, int))
|
||
named = sorted(str(k) for k in declared if not isinstance(k, int))
|
||
hint = ""
|
||
if isinstance(key, int) and 30000 <= key <= 32767:
|
||
hint = (" THAT NUMBER IS IN THE NODEPORT RANGE, which makes this "
|
||
"almost certainly the NodePort mistake: a NetworkPolicy "
|
||
"is evaluated at the POD's interface, after kube-proxy "
|
||
"has already translated the NodePort to the targetPort. "
|
||
"The rule therefore matches nothing while looking strict "
|
||
"- and being the only rule for that port, it blocks every "
|
||
"inbound connection instead. Use the containerPort.")
|
||
fail("%s ingress[%d].ports[%d] allows port %r, which is NOT a "
|
||
"containerPort of any pod spec this policy selects.%s "
|
||
"containerPorts actually declared: %s%s"
|
||
% (tag, i, j, value, hint,
|
||
", ".join(str(n) for n in numeric) or "none",
|
||
(" (names: %s)" % ", ".join(named)) if named else ""))
|
||
|
||
# --- Assertion 3: no ingress rule that silently allows every source -----
|
||
# The plan's wording was "no rule with a missing or empty from". Implemented
|
||
# in the two halves it really has, because the precedents deliberately use
|
||
# one of them:
|
||
#
|
||
# from: [] - present but EMPTY. Always a failure. It allows every source,
|
||
# exactly like no key at all, while reading as a restriction.
|
||
# There is no case where it is the right way to write either
|
||
# intention.
|
||
# no "from" - the documented open form; valkey and stalwart both use it on
|
||
# purpose. Accepted ONLY where the render itself justifies
|
||
# being open: every port of the rule is addressed by a network
|
||
# probe of the selected pods (the probe comes from the node's
|
||
# IP, which no chart can know), or is published by a NodePort
|
||
# Service of the same render (the source is outside the cluster
|
||
# and already masqueraded). Anything else is an allow-all
|
||
# nobody chose, which is the case this assertion exists for.
|
||
if "Ingress" in policy_types or (spec.get("ingress") and not policy_types):
|
||
for i, rule in enumerate(spec.get("ingress") or []):
|
||
if not isinstance(rule, dict):
|
||
continue
|
||
if "from" in rule and rule["from"] == []:
|
||
fail("%s ingress[%d] has an EMPTY \"from: []\". That allows every "
|
||
"source - identically to having no \"from\" at all - while "
|
||
"reading in a diff and in kubectl output as a restriction. It "
|
||
"is the one mistake in this class that makes a policy weaker "
|
||
"while looking stricter. Write no \"from\" key where open is "
|
||
"meant, and say so in a template comment, or spell the sources "
|
||
"out." % (tag, i))
|
||
continue
|
||
if "from" in rule:
|
||
sources = rule["from"]
|
||
if not isinstance(sources, list):
|
||
fail("%s ingress[%d].from is not a list (%r)" % (tag, i, sources))
|
||
continue
|
||
empty = [k for k, s in enumerate(sources) if isinstance(s, dict) and not s]
|
||
if empty:
|
||
fail("%s ingress[%d].from contains the empty peer {} at index %s, "
|
||
"which matches every pod in every namespace and makes the "
|
||
"whole rule open while the surrounding list looks specific."
|
||
% (tag, i, ", ".join(str(k) for k in empty)))
|
||
else:
|
||
RESTRICTED_RULES += 1
|
||
ok("%s ingress[%d] restricts by source (%d peer(s))"
|
||
% (tag, i, len(sources)))
|
||
continue
|
||
numbers, unresolved = rule_port_numbers(rule)
|
||
if unresolved:
|
||
skip("%s ingress[%d] names port(s) %s that resolve to no "
|
||
"containerPort, so this rule's openness could not be judged. "
|
||
"Reported, not counted as passing."
|
||
% (tag, i, ", ".join(unresolved)))
|
||
continue
|
||
if not numbers:
|
||
fail("%s ingress[%d] has neither \"from\" nor \"ports\". That is the "
|
||
"widest rule the object can express - every port from every "
|
||
"source - and it makes every other rule in the policy "
|
||
"decorative." % (tag, i))
|
||
continue
|
||
unjustified = sorted(n for n in numbers
|
||
if n not in probed and n not in nodeports)
|
||
if unjustified:
|
||
fail("%s ingress[%d] carries no \"from\", so it is open to every "
|
||
"source the CNI routes to, and port(s) %s justify neither: no "
|
||
"network probe of the selected pods addresses them, and no "
|
||
"NodePort Service of this render publishes them. Those are the "
|
||
"only two reasons this convention leaves a rule open - the "
|
||
"kubelet probes from the node's own IP, which a chart cannot "
|
||
"know, and NodePort traffic arrives already masqueraded. "
|
||
"Restrict this rule by source, or split the genuinely open "
|
||
"ports into their own rule and say in a template comment why "
|
||
"they are open." % (tag, i, ", ".join(str(n) for n in unjustified)))
|
||
else:
|
||
why = []
|
||
for n in sorted(numbers):
|
||
if n in probed:
|
||
why.append("%d (probe path: %s)" % (n, ", ".join(probed[n])))
|
||
else:
|
||
why.append("%d (NodePort %s)" % (n, ", ".join(nodeports[n])))
|
||
ok("%s ingress[%d] is deliberately open, and the render justifies "
|
||
"it: %s" % (tag, i, "; ".join(why)))
|
||
|
||
# --- Assertion 4: the probe path -----------------------------------------
|
||
# For every port a NETWORK probe addresses, the policy either does not
|
||
# restrict it by source, or the chart offers probeSources AND its NOTES.txt
|
||
# says what a wrong CIDR costs. The most dangerous trap over 29 charts, and
|
||
# the only one the install-test shows only after minutes - as a rollout
|
||
# timeout whose cause is three steps away.
|
||
if selected and not probed:
|
||
ok("%s selects pods that have no network probe at all (exec probes, or "
|
||
"none), so no probe traffic crosses the network and every port may be "
|
||
"restricted by source without probeSources." % tag)
|
||
for port_number, who in sorted(probed.items()):
|
||
restricted, covered = [], False
|
||
for i, rule in enumerate(spec.get("ingress") or []):
|
||
if not isinstance(rule, dict):
|
||
continue
|
||
numbers, _unresolved = rule_port_numbers(rule)
|
||
if not rule.get("ports"):
|
||
numbers = set([port_number]) # a rule without ports covers all of them
|
||
if port_number not in numbers:
|
||
continue
|
||
covered = True
|
||
if rule.get("from"):
|
||
restricted.append(i)
|
||
if "Ingress" not in policy_types and not (spec.get("ingress") and not policy_types):
|
||
ok("%s does not restrict the Ingress direction at all, so the probe on "
|
||
"port %d (%s) is unaffected." % (tag, port_number, ", ".join(who)))
|
||
continue
|
||
if not covered:
|
||
fail("%s selects a pod whose probes address port %d (%s), but NO ingress "
|
||
"rule of this policy allows that port. With policyTypes Ingress the "
|
||
"kubelet's probe is then dropped and THE POD NEVER BECOMES READY - "
|
||
"the rollout times out and the symptom reads as a broken "
|
||
"application rather than as a policy."
|
||
% (tag, port_number, ", ".join(who)))
|
||
continue
|
||
if not restricted:
|
||
ok("%s leaves probe port %d (%s) unrestricted by source, which is the "
|
||
"safe default: the kubelet probes from the IP of the node running "
|
||
"the pod, and no chart can know it."
|
||
% (tag, port_number, ", ".join(who)))
|
||
continue
|
||
if offers_probe_sources and notes_warns:
|
||
PROBE_RESTRICTED += 1
|
||
ok("%s restricts probe port %d (%s) by source in ingress[%s], and the "
|
||
"chart carries what the convention requires with it: "
|
||
"networkPolicy.probeSources in values.yaml and the consequence "
|
||
"spelled out in NOTES.txt."
|
||
% (tag, port_number, ", ".join(who),
|
||
", ".join(str(r) for r in restricted)))
|
||
continue
|
||
missing = []
|
||
if not offers_probe_sources:
|
||
missing.append("values.yaml offers no networkPolicy.probeSources")
|
||
if not notes_warns:
|
||
missing.append("templates/NOTES.txt does not say that a probeSources list "
|
||
"missing the node's IP leaves the pod permanently un-Ready "
|
||
"(this test looks for the phrase \"never becomes Ready\" "
|
||
"together with the value's name)")
|
||
fail("%s restricts probe port %d (%s) by source in ingress[%s] while %s. A "
|
||
"restricted probe port is only defensible when the operator can widen "
|
||
"it AND has been told what happens if the CIDR is wrong: the kubelet "
|
||
"probes from the IP of the WORKER NODE running the pod, so a list that "
|
||
"misses it drops every probe and the pod never becomes Ready."
|
||
% (tag, port_number, ", ".join(who),
|
||
", ".join(str(r) for r in restricted), " and ".join(missing)))
|
||
|
||
# --- Assertion 5: no reasoning left in the rendered manifest ----------------
|
||
# Not one of the original four, and added because it hit several rounds of this
|
||
# rollout: a "#" comment is part of the RENDERED object. It travels into the
|
||
# cluster, shows up in `helm get manifest` and `kubectl get -o yaml`, and puts
|
||
# the chart's reasoning in front of everyone who diffs a release. Go template
|
||
# comments render nothing and are the place for it. It can only be checked on
|
||
# the raw YAML - the comments are gone once ci/yaml2json has read the stream.
|
||
# SUBCHART DOCUMENTS ARE EXCLUDED FROM THIS ONE, and the reason is structural
|
||
# rather than lenient. A dependency is fetched from the Helm registry, not from
|
||
# the copy in this repository, so the NetworkPolicy that paperless-ngx and
|
||
# stalwart bundle is whatever version of valkey is PUBLISHED - and nothing in
|
||
# the parent chart can change its rendered comments. Holding the parent
|
||
# responsible would make this test unfixable until the subchart's next release,
|
||
# and would fail a chart for a file its own diff does not contain. The
|
||
# subchart's own pull request is where that is caught, because this test runs
|
||
# over every chart including valkey itself.
|
||
#
|
||
# Recognised from the "# Source:" header Helm writes above each document: a
|
||
# subchart's path contains "/charts/".
|
||
if policies:
|
||
current, offenders = [], []
|
||
for line in raw.splitlines() + ["---"]:
|
||
if line.strip() == "---":
|
||
body = "\n".join(current)
|
||
source = re.search(r"^# Source:\s*(\S+)", body, re.M)
|
||
from_subchart = bool(source) and "/charts/" in source.group(1)
|
||
if (re.search(r"^kind:\s*NetworkPolicy\s*$", body, re.M)
|
||
and not from_subchart):
|
||
for raw_line in current:
|
||
stripped = raw_line.strip()
|
||
if stripped.startswith("#") and not stripped.startswith("# Source:"):
|
||
offenders.append(stripped)
|
||
current = []
|
||
else:
|
||
current.append(line)
|
||
if offenders:
|
||
fail("the rendered NetworkPolicy manifest(s) contain %d \"#\" comment line(s), "
|
||
"which are part of the OBJECT and are shipped to the cluster: they appear "
|
||
"in `helm get manifest` and in `kubectl get -o yaml`, and they put the "
|
||
"chart's reasoning into every release diff. Move them into Go template "
|
||
"comments, which render nothing. First offender: %s"
|
||
% (len(offenders), offenders[0][:160]))
|
||
else:
|
||
ok("the rendered NetworkPolicy manifest(s) carry no \"#\" comment - all "
|
||
"reasoning stayed in template comments")
|
||
|
||
print("OFFERSPS|%d" % (1 if offers_probe_sources else 0))
|
||
print("RESTRICTED|%d" % RESTRICTED_RULES)
|
||
print("PROBERESTRICTED|%d" % PROBE_RESTRICTED)
|
||
for line in lines:
|
||
print(line)
|
||
PY
|
||
RC=$?
|
||
set -e
|
||
if [ "$RC" -gt 1 ]; then
|
||
sed 's/^/ /' "$WORK/$CHART.pyerr"
|
||
echo "::endgroup::"
|
||
echo "::error title=networkpolicy $CHART::the policy comparison itself failed for $CHART (python exit $RC). Treated as a failure, never as a pass."
|
||
FAILED="$FAILED $CHART"
|
||
continue
|
||
fi
|
||
|
||
CHART_FAIL=""
|
||
CHART_OK=0
|
||
CHART_SKIP=0
|
||
CHART_RESTRICTED=0
|
||
CHART_PROBE_RESTRICTED=0
|
||
CHART_OFFERS_PS=0
|
||
while IFS='|' read -r VERDICT DETAIL; do
|
||
case "$VERDICT" in
|
||
OK)
|
||
CHART_OK=$((CHART_OK + 1))
|
||
echo " ok $DETAIL"
|
||
;;
|
||
SKIP)
|
||
CHART_SKIP=$((CHART_SKIP + 1))
|
||
echo " gap $DETAIL"
|
||
;;
|
||
RESTRICTED)
|
||
CHART_RESTRICTED="$DETAIL"
|
||
;;
|
||
PROBERESTRICTED)
|
||
CHART_PROBE_RESTRICTED="$DETAIL"
|
||
;;
|
||
OFFERSPS)
|
||
CHART_OFFERS_PS="$DETAIL"
|
||
;;
|
||
FAIL)
|
||
CHART_FAIL=1
|
||
echo " FAIL $DETAIL"
|
||
echo "::error title=networkpolicy $CHART::$DETAIL"
|
||
;;
|
||
*)
|
||
[ -n "$VERDICT" ] && echo " $VERDICT $DETAIL"
|
||
;;
|
||
esac
|
||
done <"$WORK/$CHART.out"
|
||
echo "::endgroup::"
|
||
|
||
TOTAL_ASSERTIONS=$((TOTAL_ASSERTIONS + CHART_OK))
|
||
|
||
if [ -n "$CHART_FAIL" ]; then
|
||
FAILED="$FAILED $CHART"
|
||
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | **failed** | $CHART_OK | $CHART_SKIP |"$'\n'
|
||
echo "$CHART: FAILED"
|
||
elif [ -n "$HAS_OVERLAY" ] && [ "$CHART_OFFERS_PS" -eq 1 ] && [ "$CHART_PROBE_RESTRICTED" -eq 0 ]; then
|
||
# The overlay's ONE job is to render a code path the chart's defaults do not
|
||
# reach - above all the RESTRICTED probe rule, which only appears once
|
||
# probeSources is non-empty. If no restricted rule came out, this test is
|
||
# vacuum-green for the very chart the overlay was written for.
|
||
FAILED="$FAILED $CHART"
|
||
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | **failed** | $CHART_OK | $CHART_SKIP |"$'\n'
|
||
echo "::error title=networkpolicy $CHART::$CHART has an overlay at ci/networkpolicy/$CHART.yaml but rendered no source-restricted PROBE rule, so the restricted branch - the one the chart's own defaults never reach - was not checked. That overlay exists for exactly this; either it no longer sets networkPolicy.probeSources where the chart reads it, or the template stopped rendering the restricted form. A green tick here would be a tick for a check that ran over nothing."
|
||
echo "$CHART: FAILED (overlay present, no restricted probe rule rendered)"
|
||
else
|
||
CHECKED="$CHECKED $CHART"
|
||
[ "$CHART_SKIP" -gt 0 ] && WITH_GAPS="$WITH_GAPS $CHART"
|
||
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | ok | $CHART_OK | $CHART_SKIP |"$'\n'
|
||
echo "$CHART: ok ($CHART_OK assertion(s) held, $CHART_SKIP not checked, $CHART_RESTRICTED source-restricted rule(s), $CHART_PROBE_RESTRICTED of them on a probe port)"
|
||
fi
|
||
done
|
||
|
||
# --- Assertion 6: the CROSS-CHART egress ports (Gitea #136) ------------------
|
||
# The only assertion here that needs more than one chart's render, and the
|
||
# reason it exists is an outage. paperless-ngx' egress rule for Tika and
|
||
# Gotenberg named 80 - the port those two charts' SERVICES listen on - and
|
||
# justified it in a template comment. kube-proxy does a ClusterIP Service's
|
||
# destination NAT BEFORE the egress policy is evaluated at the pod's network
|
||
# interface, so the policy sees podIP:targetPort; 80 matched nothing, and the
|
||
# release that shipped it cut Paperless off from both services on 2026-10-15.
|
||
# The symptom was a TIMEOUT, not "connection refused", and no error mentioned a
|
||
# NetworkPolicy.
|
||
#
|
||
# WHY ASSERTION 2 CANNOT CATCH THIS, and why that is not a defect in it: it
|
||
# holds ports against the pod spec the policy SELECTS, which is right for
|
||
# ingress and meaningless for egress - an egress port belongs to the
|
||
# DESTINATION. For a database, a cache or the web that destination is not in any
|
||
# render this repository has. It is in a render only when one chart of THIS
|
||
# repository dials the Service of another, and paperless-ngx -> apache-tika,
|
||
# gotenberg is the single place where that happens (checked across the
|
||
# repository for #136; every other egress port is either a 1:1 Service, so the
|
||
# mistake is invisible, or outside the cluster, where there is no DNAT at all
|
||
# and the port named IS the real one).
|
||
#
|
||
# So the table below is deliberately not a discovered list: a chart cannot know
|
||
# which Service another chart dials, because the address lives in a value or a
|
||
# 1Password item. Adding a row is the price of the next cross-chart egress rule,
|
||
# and a row that names a chart which no longer exists FAILS rather than being
|
||
# skipped - the failure path was executed, like the others.
|
||
#
|
||
# WHAT IS ASSERTED, per row, on TWO renders of the consumer:
|
||
# gate ON - the consumer's egress ports must CONTAIN every pod port behind
|
||
# the peers' Services, resolved through the peers' own renders
|
||
# (Service port -> targetPort -> containerPort, i.e. exactly the
|
||
# translation the policy is evaluated after).
|
||
# gate OFF - the containment requirement does NOT apply (the rule is gated,
|
||
# and a release without the feature must not carry a rule that
|
||
# permits nothing), and those ports must be absent.
|
||
#
|
||
# Built into THIS script rather than added as a new test with its own pipeline
|
||
# job, because everything it needs is already here: the job runs over every
|
||
# chart on every pull request (so it cannot be skipped by a path filter), Helm
|
||
# is already the YAML parser through ci/yaml2json, and the verdict, grouping and
|
||
# summary plumbing is shared. A second script would duplicate all of it to add
|
||
# one comparison.
|
||
#
|
||
# consumer|peer charts|gate value|extra --set needed to render the gate ON
|
||
EGRESS_PEER_ROWS="paperless-ngx|apache-tika,gotenberg|paperless.files.tikaGotenberg.enabled|paperless.files.tikaGotenberg.tikaEndpoint=http://tika.ci.local,paperless.files.tikaGotenberg.gotenbergEndpoint=http://gotenberg.ci.local"
|
||
|
||
# Renders one chart with the same values precedence the loop above uses and
|
||
# leaves the parsed JSON in $2. The render/split/parse steps are repeated here
|
||
# instead of refactored out of the loop on purpose: the five assertions above
|
||
# are load-bearing and were not worth disturbing for this.
|
||
render_chart_json() {
|
||
RCJ_CHART="$1"
|
||
RCJ_OUT="$2"
|
||
shift 2
|
||
RCJ_VALUES=()
|
||
RCJ_LABEL="chart defaults"
|
||
if [ -f "$CI_DIR/$RCJ_CHART/test-values.yaml" ]; then
|
||
RCJ_VALUES=(-f "$CI_DIR/$RCJ_CHART/test-values.yaml")
|
||
RCJ_LABEL="ci/$RCJ_CHART/test-values.yaml"
|
||
elif [ -f "$CI_DIR/trivy/values/$RCJ_CHART.yaml" ]; then
|
||
RCJ_VALUES=(-f "$CI_DIR/trivy/values/$RCJ_CHART.yaml")
|
||
RCJ_LABEL="ci/trivy/values/$RCJ_CHART.yaml"
|
||
fi
|
||
"$ROOT/ci/helm-dependencies.sh" "$ROOT/$RCJ_CHART" >/dev/null || return 1
|
||
RCJ_MANIFEST="$RCJ_OUT.manifest.yaml"
|
||
helm template "$RELEASE" "$ROOT/$RCJ_CHART" \
|
||
${RCJ_VALUES[@]+"${RCJ_VALUES[@]}"} "$@" >"$RCJ_MANIFEST" 2>"$WORK/xerr" || return 1
|
||
awk '
|
||
BEGIN { print "docs:"; started = 0 }
|
||
/^---[ \t]*$/ { print " -"; started = 1; next }
|
||
{
|
||
if (!started) { print " -"; started = 1 }
|
||
if (length($0) == 0) print ""; else print " " $0
|
||
}
|
||
' "$RCJ_MANIFEST" >"$RCJ_MANIFEST.docs"
|
||
helm template y2j "$YAML2JSON" -f "$RCJ_MANIFEST.docs" 2>"$WORK/xerr" \
|
||
| awk '/^\{/ { print; exit }' >"$RCJ_OUT" || return 1
|
||
[ -s "$RCJ_OUT" ] || return 1
|
||
return 0
|
||
}
|
||
|
||
CROSS_ROWS=0
|
||
for ROW in $EGRESS_PEER_ROWS; do
|
||
CONSUMER="$(printf '%s' "$ROW" | cut -d'|' -f1)"
|
||
PEERS="$(printf '%s' "$ROW" | cut -d'|' -f2 | tr ',' ' ')"
|
||
GATE="$(printf '%s' "$ROW" | cut -d'|' -f3)"
|
||
GATE_SETS="$(printf '%s' "$ROW" | cut -d'|' -f4)"
|
||
|
||
# Only when the consumer is in this run's chart list, so a single-chart
|
||
# invocation stays single-chart. The pipeline passes no arguments, so in CI
|
||
# every row always runs.
|
||
IN_RUN=""
|
||
for C in "${CHARTS[@]}"; do [ "$C" = "$CONSUMER" ] && IN_RUN=1; done
|
||
[ -n "$IN_RUN" ] || continue
|
||
|
||
echo "::group::cross-chart egress: $CONSUMER -> $PEERS"
|
||
CROSS_ROWS=$((CROSS_ROWS + 1))
|
||
ROW_FAIL=""
|
||
|
||
if [ ! -f "$ROOT/$CONSUMER/Chart.yaml" ]; then
|
||
echo " FAIL $CONSUMER is named in EGRESS_PEER_ROWS but is not a chart directory"
|
||
echo "::error title=cross-chart egress::$CONSUMER is named in EGRESS_PEER_ROWS in ci/run-networkpolicy-test.sh but is not a chart directory. If the chart was renamed or removed, update that table - a row that silently does nothing is how this check would decay back into the state that caused Gitea #136."
|
||
ROW_FAIL=1
|
||
fi
|
||
|
||
PEER_ARGS=()
|
||
for PEER in $PEERS; do
|
||
if [ ! -f "$ROOT/$PEER/Chart.yaml" ]; then
|
||
echo " FAIL peer chart $PEER does not exist"
|
||
echo "::error title=cross-chart egress::$CONSUMER's egress rule is checked against the chart $PEER, which is not a chart directory of this repository. Either it was renamed - in which case the row in EGRESS_PEER_ROWS in ci/run-networkpolicy-test.sh has to follow, and $CONSUMER's rule probably too - or the row is stale. Not skipped, because an unchecked row reads exactly like a passing one."
|
||
ROW_FAIL=1
|
||
continue
|
||
fi
|
||
if ! render_chart_json "$PEER" "$WORK/peer-$PEER.json"; then
|
||
sed 's/^/ /' "$WORK/xerr"
|
||
echo " FAIL peer chart $PEER does not render"
|
||
echo "::error title=cross-chart egress::the peer chart $PEER does not render with its CI values, so the pod ports behind its Services could not be read and $CONSUMER's egress rule could not be checked against them."
|
||
ROW_FAIL=1
|
||
continue
|
||
fi
|
||
PEER_ARGS+=("$PEER=$WORK/peer-$PEER.json")
|
||
done
|
||
|
||
ON_JSON="$WORK/consumer-$CONSUMER-on.json"
|
||
OFF_JSON="$WORK/consumer-$CONSUMER-off.json"
|
||
if [ -z "$ROW_FAIL" ]; then
|
||
SET_ARGS=(--set "$GATE=true")
|
||
OLD_IFS="$IFS"; IFS=','
|
||
for S in $GATE_SETS; do [ -n "$S" ] && SET_ARGS+=(--set "$S"); done
|
||
IFS="$OLD_IFS"
|
||
if ! render_chart_json "$CONSUMER" "$ON_JSON" "${SET_ARGS[@]}"; then
|
||
sed 's/^/ /' "$WORK/xerr"
|
||
echo " FAIL $CONSUMER does not render with $GATE=true"
|
||
echo "::error title=cross-chart egress::$CONSUMER does not render with $GATE=true and the --set values from EGRESS_PEER_ROWS, so its egress rule for $PEERS was never produced and nothing was checked. If the feature needs further values, add them to that row."
|
||
ROW_FAIL=1
|
||
elif ! render_chart_json "$CONSUMER" "$OFF_JSON" --set "$GATE=false"; then
|
||
sed 's/^/ /' "$WORK/xerr"
|
||
echo " FAIL $CONSUMER does not render with $GATE=false"
|
||
echo "::error title=cross-chart egress::$CONSUMER does not render with $GATE=false, so the gated-off case - in which this assertion must NOT fire - could not be checked."
|
||
ROW_FAIL=1
|
||
fi
|
||
fi
|
||
|
||
if [ -z "$ROW_FAIL" ]; then
|
||
set +e
|
||
python3 - "$CONSUMER" "$RELEASE" "$GATE" "$ON_JSON" "$OFF_JSON" "${PEER_ARGS[@]}" \
|
||
>"$WORK/cross-$CONSUMER.out" 2>"$WORK/cross-$CONSUMER.pyerr" <<'PY'
|
||
import json, sys
|
||
|
||
consumer, release, gate, on_path, off_path = sys.argv[1:6]
|
||
peers = [a.split("=", 1) for a in sys.argv[6:]]
|
||
|
||
lines = []
|
||
def ok(msg): lines.append("OK|%s: %s" % (consumer, msg))
|
||
def skip(msg): lines.append("SKIP|%s: %s" % (consumer, msg))
|
||
def fail(msg): lines.append("FAIL|%s: %s" % (consumer, msg))
|
||
|
||
|
||
def docs_of(path):
|
||
return [d for d in (json.load(open(path)).get("docs") or []) if isinstance(d, dict)]
|
||
|
||
|
||
def pod_templates(node):
|
||
out = []
|
||
if isinstance(node, dict):
|
||
for key, value in node.items():
|
||
if (key == "template" and isinstance(value, dict)
|
||
and isinstance(value.get("spec"), dict)
|
||
and "containers" in value["spec"]):
|
||
out.append(value)
|
||
out.extend(pod_templates(value))
|
||
elif isinstance(node, list):
|
||
for value in node:
|
||
out.extend(pod_templates(value))
|
||
return out
|
||
|
||
|
||
def labels_of(template):
|
||
meta = template.get("metadata")
|
||
labels = meta.get("labels") if isinstance(meta, dict) else None
|
||
return labels if isinstance(labels, dict) else {}
|
||
|
||
|
||
def matches(selector, labels):
|
||
for key, value in selector.items():
|
||
if key not in labels or str(labels[key]) != str(value):
|
||
return False
|
||
return True
|
||
|
||
|
||
def container_ports(template):
|
||
out = []
|
||
spec = template.get("spec") or {}
|
||
for key in ("containers", "initContainers"):
|
||
for container in spec.get(key) or []:
|
||
if not isinstance(container, dict):
|
||
continue
|
||
for port in container.get("ports") or []:
|
||
if isinstance(port, dict) and port.get("containerPort") is not None:
|
||
out.append((int(port["containerPort"]), port.get("name")))
|
||
return out
|
||
|
||
|
||
def service_pod_ports(path, chart):
|
||
"""The pod ports a client reaches THROUGH this chart's Services, as
|
||
{pod port: [evidence]}. Resolved Service port -> targetPort -> containerPort,
|
||
because that chain IS the destination NAT the egress policy is evaluated
|
||
after: the packet leaving the consumer carries the Service port, and the
|
||
packet the policy sees carries the pod port."""
|
||
docs = docs_of(path)
|
||
templates = []
|
||
for doc in docs:
|
||
templates.extend(pod_templates(doc))
|
||
out, unresolved = {}, []
|
||
for svc in docs:
|
||
if svc.get("kind") != "Service":
|
||
continue
|
||
spec = svc.get("spec") or {}
|
||
selector = spec.get("selector")
|
||
hits = [t for t in templates
|
||
if isinstance(selector, dict) and matches(selector, labels_of(t))]
|
||
by_name = dict((n, num) for t in hits for num, n in container_ports(t) if n)
|
||
svc_name = ((svc.get("metadata") or {}).get("name")) or "<unnamed Service>"
|
||
for port in spec.get("ports") or []:
|
||
if not isinstance(port, dict):
|
||
continue
|
||
target = port.get("targetPort")
|
||
if target is None:
|
||
target = port.get("port")
|
||
number = None
|
||
if isinstance(target, int):
|
||
number = target
|
||
elif isinstance(target, str):
|
||
number = int(target) if target.isdigit() else by_name.get(target)
|
||
if number is None:
|
||
unresolved.append("%s %s: targetPort %r" % (chart, svc_name, target))
|
||
continue
|
||
out.setdefault(number, []).append(
|
||
"%s %s: port %s -> targetPort %r -> pod port %d"
|
||
% (chart, svc_name, port.get("port"), target, number))
|
||
return out, unresolved
|
||
|
||
|
||
def egress_ports(path):
|
||
"""(numbers, policy names, has a rule with no ports at all) for the
|
||
consumer's OWN policy only - recognised by the <release>-<chart>- name
|
||
prefix, so a subchart's policy in the same render cannot satisfy a rule the
|
||
parent chart is responsible for."""
|
||
prefix = "%s-%s-" % (release, consumer)
|
||
numbers, names, open_rule = set(), [], False
|
||
for doc in docs_of(path):
|
||
if doc.get("kind") != "NetworkPolicy":
|
||
continue
|
||
name = ((doc.get("metadata") or {}).get("name")) or ""
|
||
if not name.startswith(prefix):
|
||
continue
|
||
names.append(name)
|
||
spec = doc.get("spec") if isinstance(doc.get("spec"), dict) else {}
|
||
for rule in spec.get("egress") or []:
|
||
if not isinstance(rule, dict):
|
||
continue
|
||
if not rule.get("ports"):
|
||
open_rule = True
|
||
continue
|
||
for port in rule["ports"]:
|
||
if not isinstance(port, dict):
|
||
continue
|
||
value = port.get("port")
|
||
if isinstance(value, int):
|
||
numbers.add(value)
|
||
elif isinstance(value, str) and value.isdigit():
|
||
numbers.add(int(value))
|
||
return numbers, names, open_rule
|
||
|
||
|
||
required, unresolved = {}, []
|
||
for chart, path in peers:
|
||
ports, bad = service_pod_ports(path, chart)
|
||
for number, evidence in ports.items():
|
||
required.setdefault(number, []).extend(evidence)
|
||
unresolved.extend(bad)
|
||
|
||
for note in unresolved:
|
||
skip("a Service targetPort of a peer chart could not be resolved to a "
|
||
"containerPort, so that port is not part of the requirement (%s). "
|
||
"Reported, not counted as passing." % note)
|
||
|
||
on_numbers, on_names, on_open = egress_ports(on_path)
|
||
off_numbers, off_names, _off_open = egress_ports(off_path)
|
||
|
||
if not required:
|
||
fail("the peer charts render no Service whose port resolves to a pod port, "
|
||
"so this assertion would have required nothing at all. That is "
|
||
"indistinguishable from a passing check and is treated as a failure: "
|
||
"either the peer charts stopped shipping a Service, or their targetPort "
|
||
"no longer resolves.")
|
||
elif not on_names:
|
||
fail("with %s=true the render contains no NetworkPolicy named %s-%s-*, so "
|
||
"the egress rule this assertion is about was not found. The policy's "
|
||
"name is the convention {{ .Release.Name }}-{{ .Chart.Name }}-<kind>; if "
|
||
"it changed, this check has to follow."
|
||
% (gate, release, consumer))
|
||
elif on_open:
|
||
skip("an egress rule of %s names NO port, which allows every port - so "
|
||
"containment of the peers' pod ports is satisfied without saying "
|
||
"anything. Reported, never counted as passing."
|
||
% ", ".join(on_names))
|
||
else:
|
||
missing = sorted(n for n in required if n not in on_numbers)
|
||
if missing:
|
||
fail("with %s=true, %s permits outbound %s but NOT %s - and those are the "
|
||
"POD ports behind the peer charts' Services: %s. An egress rule is "
|
||
"evaluated at the pod's network interface, AFTER kube-proxy has "
|
||
"translated the ClusterIP Service's destination (DNAT), so the policy "
|
||
"sees podIP:targetPort and a rule naming the SERVICE port matches "
|
||
"nothing wherever the two numbers differ. This is Gitea #136 "
|
||
"exactly: it cost an outage on 2026-10-15, and the symptom was a "
|
||
"TIMEOUT rather than \"connection refused\", with no error "
|
||
"mentioning a NetworkPolicy. Name the pod port(s) in the rule."
|
||
% (gate, ", ".join(on_names),
|
||
", ".join(str(n) for n in sorted(on_numbers)),
|
||
", ".join(str(n) for n in missing),
|
||
"; ".join(e for n in missing for e in required[n])))
|
||
else:
|
||
ok("with %s=true, %s permits every pod port behind the peer charts' "
|
||
"Services: %s"
|
||
% (gate, ", ".join(on_names),
|
||
"; ".join(e for n in sorted(required) for e in required[n])))
|
||
|
||
# The gated-off half. The requirement above must NOT apply here - a release
|
||
# without the feature is not expected to reach the peers - and the ports must
|
||
# be gone, or the rule is no longer gated and permits something the release
|
||
# has no use for.
|
||
present = sorted(n for n in required if n in off_numbers)
|
||
if not off_names:
|
||
skip("with %s=false the render contains no NetworkPolicy of this chart, "
|
||
"so the gated-off case could not be read." % gate)
|
||
elif present:
|
||
fail("with %s=false, %s still permits outbound %s, the peer charts' pod "
|
||
"port(s). Either the rule is no longer gated on that value - a rule "
|
||
"that permits what the release cannot use - or the CI values add "
|
||
"those ports by hand, which would make the gate-on assertion above "
|
||
"pass without the rule existing."
|
||
% (gate, ", ".join(off_names),
|
||
", ".join(str(n) for n in present)))
|
||
else:
|
||
ok("with %s=false the rule is absent (egress: %s), so the containment "
|
||
"requirement correctly does not apply and no release without the "
|
||
"feature carries a rule that permits nothing."
|
||
% (gate, ", ".join(str(n) for n in sorted(off_numbers)) or "none"))
|
||
|
||
for line in lines:
|
||
print(line)
|
||
PY
|
||
RC=$?
|
||
set -e
|
||
if [ "$RC" -gt 1 ]; then
|
||
sed 's/^/ /' "$WORK/cross-$CONSUMER.pyerr"
|
||
echo "::error title=cross-chart egress::the cross-chart egress comparison itself failed for $CONSUMER (python exit $RC). Treated as a failure, never as a pass."
|
||
ROW_FAIL=1
|
||
else
|
||
while IFS='|' read -r VERDICT DETAIL; do
|
||
case "$VERDICT" in
|
||
OK)
|
||
TOTAL_ASSERTIONS=$((TOTAL_ASSERTIONS + 1))
|
||
echo " ok $DETAIL"
|
||
;;
|
||
SKIP) echo " gap $DETAIL" ;;
|
||
FAIL)
|
||
ROW_FAIL=1
|
||
echo " FAIL $DETAIL"
|
||
echo "::error title=cross-chart egress::$DETAIL"
|
||
;;
|
||
*) [ -n "$VERDICT" ] && echo " $VERDICT $DETAIL" ;;
|
||
esac
|
||
done <"$WORK/cross-$CONSUMER.out"
|
||
fi
|
||
fi
|
||
|
||
echo "::endgroup::"
|
||
if [ -n "$ROW_FAIL" ]; then
|
||
FAILED="$FAILED $CONSUMER(cross-chart)"
|
||
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CONSUMER\` → \`$(printf '%s' "$PEERS" | tr ' ' ',')\` | **failed** | – | – |"$'\n'
|
||
echo "cross-chart egress $CONSUMER: FAILED"
|
||
else
|
||
SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CONSUMER\` → \`$(printf '%s' "$PEERS" | tr ' ' ',')\` | ok | 2 | 0 |"$'\n'
|
||
echo "cross-chart egress $CONSUMER: ok"
|
||
fi
|
||
done
|
||
|
||
# --- Report ------------------------------------------------------------------
|
||
count() { set -- $1; echo "$#"; }
|
||
listing() { set -- $1; [ "$#" -eq 0 ] && { echo "–"; return; }; printf '`%s`' "$1"; shift; for C in "$@"; do printf ', `%s`' "$C"; done; echo; }
|
||
SUMMARY="${GITHUB_STEP_SUMMARY:-/dev/null}"
|
||
{
|
||
echo "### NetworkPolicy render test (Gitea #5)"
|
||
echo
|
||
echo "Every \`NetworkPolicy\` a chart renders, held against the pod specs, probes and"
|
||
echo "Services of the **same** render. All of these failures are silent: the API server"
|
||
echo "accepts a policy that filters nothing just as readily as one that filters"
|
||
echo "everything."
|
||
echo
|
||
echo "| Result | Count | Charts |"
|
||
echo "|---|---|---|"
|
||
echo "| Policies checked | $(count "$CHECKED") | $(listing "$CHECKED") |"
|
||
echo "| Failed | $(count "$FAILED") | $(listing "$FAILED") |"
|
||
echo
|
||
echo "$TOTAL_ASSERTIONS assertion(s) held in total."
|
||
echo
|
||
echo "**Not covered here, and not by anything else in CI:** that traffic still flows."
|
||
echo "No ingress controller runs in the install-test cluster, so the path from Traefik"
|
||
echo "to a pod is proven nowhere. What *is* proven is the probe path (install-test),"
|
||
echo "acceptance by the API server, and the render-side assertions above."
|
||
if [ -n "$WITH_GAPS" ]; then
|
||
echo
|
||
echo "Charts with code paths this test does **not** evaluate (\`matchExpressions\`-only selectors, port ranges, rules naming no port — reported, never counted as passing): $(listing "$WITH_GAPS")."
|
||
fi
|
||
if [ -n "$SUMMARY_ROWS" ]; then
|
||
echo
|
||
echo "| Chart | Result | Held | Not checked |"
|
||
echo "| --- | --- | --- | --- |"
|
||
printf '%s' "$SUMMARY_ROWS"
|
||
fi
|
||
} >>"$SUMMARY"
|
||
|
||
echo "===================== networkpolicy test ======================="
|
||
echo "checked: $(count "$CHECKED")${CHECKED:+ -$CHECKED}"
|
||
echo "failed: $(count "$FAILED")${FAILED:+ -$FAILED}"
|
||
echo "assertions held: $TOTAL_ASSERTIONS"
|
||
echo "================================================================"
|
||
|
||
if [ -n "$FAILED" ]; then
|
||
echo "ERROR: NetworkPolicy findings in:$FAILED" >&2
|
||
exit 1
|
||
fi
|
||
|
||
# The third anti-vacuum guard. Reaching zero means no chart rendered a policy
|
||
# this script could say anything about, which is indistinguishable from a broken
|
||
# harness - and it is the state this test would silently decay into.
|
||
if [ "$TOTAL_ASSERTIONS" -eq 0 ]; then
|
||
echo "ERROR: not a single assertion was evaluated in this run. Either no chart renders a NetworkPolicy any more, or the harness stopped reading the manifests. A pass over zero assertions is not a pass." >&2
|
||
exit 1
|
||
fi
|