#!/usr/bin/env bash # NetworkPolicy render test: every NetworkPolicy a chart renders is held against # the pod specs, probes and Services of the SAME render (Gitea #5). # # Usage: ci/run-networkpolicy-test.sh [chart ...] (no argument = all charts) # # WHY THIS EXISTS, AND WHY A GREEN INSTALL-TEST DOES NOT COVER IT # # Since #5 every chart in this repository ships a NetworkPolicy, on by default. # That is 29 objects whose failure modes are all SILENT in the same way: the API # server accepts a policy that filters nothing, and it accepts a policy that # filters everything. Four mistakes are reachable by copy, and each one has a # shape a render can decide: # # A SELECTOR THAT MATCHES NO POD. The object sits in the namespace, is listed # by kubectl, reads as a boundary, and filters nothing whatsoever. The usual # cause is a label renamed in the workload template - a different file, a # different diff. Nothing in a rollout notices, because nothing is denied. # A NODEPORT NUMBER WHERE A POD PORT BELONGS. A NetworkPolicy is evaluated at # the POD's network interface, after kube-proxy has already translated the # NodePort to the targetPort. A rule naming 30025 where the pod listens on # 25 matches nothing: accepted, strict-looking, and - being the only rule # for that port - blocking every inbound connection instead. Four charts # here publish NodePorts, so this is not a hypothetical. # AN EMPTY "from: []". Allows every source, exactly like no "from" at all, # while reading in a diff and in kubectl output as a restriction. The one # mistake in this list that makes a policy weaker while looking stricter. # A RESTRICTED PROBE PORT. The kubelet's liveness/readiness/startup probes # come from the IP of the WORKER NODE running the pod, an address no chart # can know. A rule that restricts a probed port by source without that # address drops every probe: the pod never becomes Ready, the rollout times # out, and the symptom reads as a broken application rather than as a # policy. This is the one the install-test does catch - after minutes, at # the end of a 29-chart matrix, with its cause three steps away. # # The install-test is the real net under the rollout and it is not replaced # here: it proves ENFORCEMENT, on a live k3s whose policy controller is # running. What it cannot do is cover satisfactory and template-chart, which # have no install-test at all (ci/excluded-charts.txt), and it cannot run on a # pull request that touches no chart. This test is the cheap half: no cluster, # no Docker, a few seconds for the whole repository. # # Over ALL charts and not the changed ones, for the same reason as # ci/run-selector-match-test.sh and ci/run-quantity-render-test.sh: the mistake # arrives by copy into a NEW chart, whose own diff nobody reads for it, and # running over everything means there is no list to maintain - adding a chart # enrols it. That property is what the first assertion below leans on, and it # is not theoretical: garage (#81) and keycloak (#83) were both created AFTER # #5 was filed and both arrived with no policy at all, keycloak with an # unauthenticated management port on 9000. # # HOW THE YAML IS PARSED, AND WHY NOT WITH A PARSER WRITTEN HERE # # The runners have neither pyyaml nor yq - measured, the same constraint # ci/run-trivy-config.sh, ci/run-quantity-render-test.sh and # ci/run-selector-match-test.sh work under. So HELM is the parser: ci/yaml2json # is a one-template chart that prints its values back as JSON, the rendered # manifest stream is handed to it as a VALUES FILE, and python3 reads the result # with its standard library. What IS done textually is only the document split, # on a line that is exactly "---" in column one, and the consequence of getting # that wrong is not a false pass: the fragment stops being valid YAML and helm # exits 1. # # Assertion 5 is the exception and works on the raw rendered YAML on purpose: # comments are gone by the time yaml2json has read the stream, and a comment is # exactly what it looks for. # # WHAT IS DELIBERATELY NOT CHECKED, named so the gaps are not mistaken for # coverage: # # - THAT TRAFFIC STILL FLOWS. Nothing here, and nothing in CI, proves the path # from Traefik to a pod: no ingress controller runs in the install-test # cluster. What is proven is the probe path, acceptance by the API server, # and the render-side assertions below. # - EGRESS DESTINATIONS. A NetworkPolicy matches CIDRs and selectors and never # hostnames, so "reaches its database" is not expressible in the object and # not checkable here. Egress ports are deliberately NOT held against the # pod's containerPorts either - an egress port is a port on the DESTINATION, # and that comparison would fail every chart that reaches a database. # - SELECTORS EXPRESSED ONLY THROUGH matchExpressions. Evaluating a set-based # requirement means reimplementing label-selector semantics, i.e. a second # parser. Those are REPORTED as "not checked" and never counted as passing, # because a green tick for a code path nobody entered is worse than an # honest gap (README, "Two rules for every check added here"). # - PORT RANGES (endPort). Same treatment: reported, never counted. # - THAT 15008 IS ENOUGH TO REACH A POD IN THE AMBIENT MESH. Assertion 7 # asserts the RENDER of networkPolicy.ingress.ambientMesh in both # directions, and nothing beyond it. That the HBONE port alone suffices # comes from Istio's documentation; measured on this cluster is only which # listeners ztunnel opens, and no namespace carries istio.io/dataplane-mode # yet, so there is no path on which it could be measured end to end. The # probe-source half (169.254.7.127/32) is in the same position: its render is # asserted, its effect is not, because probeSources is empty in every chart # of this repository. # # AND THE ANTI-VACUUM GUARDS, because this test could be green over nothing: # # 1. A chart that renders NO NetworkPolicy fails. Since #5 that is a # regression, not a style question - see the garage/keycloak note above. # 2. ci/networkpolicy/.yaml is an overlay applied last, whose only job # is to render a code path the chart's own defaults do not reach - above # all the RESTRICTED probe rule, which is only rendered once probeSources # is non-empty. A chart that HAS an overlay and still renders no restricted # rule FAILS, because the overlay exists for exactly that. # 3. If no assertion at all was evaluated across the whole run, the run fails. set -euo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" CI_DIR="$ROOT/ci" YAML2JSON="$CI_DIR/yaml2json" OVERLAY_DIR="$CI_DIR/networkpolicy" RELEASE="netpol" WORK="$(mktemp -d)" trap 'rm -rf "$WORK"' EXIT # --- Which charts ----------------------------------------------------------- CHARTS=() if [ "$#" -gt 0 ]; then CHARTS=("$@") else for DIR in "$ROOT"/*/; do [ -f "$DIR/Chart.yaml" ] || continue CHARTS+=("$(basename "$DIR")") done fi if [ "${#CHARTS[@]}" -eq 0 ]; then echo "ERROR: no chart directories found under $ROOT" >&2 exit 1 fi FAILED="" CHECKED="" WITH_GAPS="" TOTAL_ASSERTIONS=0 SUMMARY_ROWS="" for CHART in "${CHARTS[@]}"; do if [ ! -f "$ROOT/$CHART/Chart.yaml" ]; then echo "ERROR: '$CHART' is not a chart directory" >&2 exit 1 fi # Same reason as in ci/run-selector-match-test.sh: a chart with a declared # dependency cannot be rendered at all while charts/ is empty, and this job # runs over EVERY chart on EVERY pull request - so without this line a PR # touching nothing but a README would go red on the charts that have one. "$ROOT/ci/helm-dependencies.sh" "$ROOT/$CHART" # Values: the same precedence the other render tests use, so a chart's CI # values are maintained in one place and satisfy its required() values. VALUES_ARGS=() VALUES_LABEL="chart defaults" if [ -f "$CI_DIR/$CHART/test-values.yaml" ]; then VALUES_ARGS=(-f "$CI_DIR/$CHART/test-values.yaml") VALUES_LABEL="ci/$CHART/test-values.yaml" elif [ -f "$CI_DIR/trivy/values/$CHART.yaml" ]; then VALUES_ARGS=(-f "$CI_DIR/trivy/values/$CHART.yaml") VALUES_LABEL="ci/trivy/values/$CHART.yaml" fi HAS_OVERLAY="" if [ -f "$OVERLAY_DIR/$CHART.yaml" ]; then # Last, so it wins over the install-test values. VALUES_ARGS+=(-f "$OVERLAY_DIR/$CHART.yaml") VALUES_LABEL="$VALUES_LABEL + ci/networkpolicy/$CHART.yaml" HAS_OVERLAY=1 fi echo "::group::$CHART ($VALUES_LABEL)" RENDER="$WORK/$CHART.manifest.yaml" if ! helm template "$RELEASE" "$ROOT/$CHART" \ ${VALUES_ARGS[@]+"${VALUES_ARGS[@]}"} >"$RENDER" 2>"$WORK/err"; then sed 's/^/ /' "$WORK/err" echo "::endgroup::" echo "::error title=networkpolicy $CHART::$CHART does not render with its CI values, so no policy could be checked. Reproduce: helm template $RELEASE $CHART $(printf '%s ' "${VALUES_ARGS[@]+${VALUES_ARGS[@]}}")" FAILED="$FAILED $CHART" continue fi # --- The document split, and the only textual step before the parser ------- # One YAML sequence item per rendered document, content indented by four and # the item marker on a line of its own - so a document whose first line is a # comment ("# Source: ...") needs no special case. A wrong split does not # produce a false pass: the result stops being parsable and helm exits 1. DOCS="$WORK/$CHART.docs.yaml" awk ' BEGIN { print "docs:"; started = 0 } /^---[ \t]*$/ { print " -"; started = 1; next } { if (!started) { print " -"; started = 1 } if (length($0) == 0) print ""; else print " " $0 } ' "$RENDER" >"$DOCS" JSON="$WORK/$CHART.json" if ! helm template y2j "$YAML2JSON" -f "$DOCS" 2>"$WORK/err" \ | awk '/^\{/ { print; exit }' >"$JSON"; then sed 's/^/ /' "$WORK/err" echo "::endgroup::" echo "::error title=networkpolicy $CHART::the rendered manifests of $CHART could not be re-read as YAML by ci/yaml2json. That is a parse failure, not a policy finding: either the render contains a document this script's '---' split cut in the wrong place, or the manifest is not valid YAML. Helm's own message is in the job log." FAILED="$FAILED $CHART" continue fi if [ ! -s "$JSON" ]; then echo "::endgroup::" echo "::error title=networkpolicy $CHART::ci/yaml2json produced no JSON for $CHART - the manifests were not re-read, so nothing was checked." FAILED="$FAILED $CHART" continue fi # --- The assertions -------------------------------------------------------- # Inline and on the standard library only, as in ci/run-selector-match-test.sh # and ci/run-quantity-render-test.sh: json, because the YAML is already gone # by this point, plus re for the two textual checks that need the raw render. # It writes one "VERDICT|detail" line per assertion and never decides the exit # code itself - the loop below does, so a verdict can never be lost to a # swallowed status. set +e python3 - "$CHART" "$JSON" "$RENDER" "$ROOT/$CHART" >"$WORK/$CHART.out" 2>"$WORK/$CHART.pyerr" <<'PY' import json, re, sys chart, json_path, yaml_path, chart_dir = sys.argv[1:5] lines = [] def ok(msg): lines.append("OK|%s: %s" % (chart, msg)) def skip(msg): lines.append("SKIP|%s: %s" % (chart, msg)) def fail(msg): lines.append("FAIL|%s: %s" % (chart, msg)) docs = [d for d in (json.load(open(json_path)).get("docs") or []) if isinstance(d, dict)] def pod_templates(node, path="$"): """Every pod template of the render, found by walking rather than by enumerating kinds - so a CronJob's spec.jobTemplate.spec.template is covered without a special case, and so is whatever workload kind a chart adds next. Same shape as ci/run-selector-match-test.sh.""" out = [] if isinstance(node, dict): for key, value in node.items(): here = "%s.%s" % (path, key) if (key == "template" and isinstance(value, dict) and isinstance(value.get("spec"), dict) and "containers" in value["spec"]): out.append((here, value)) out.extend(pod_templates(value, here)) elif isinstance(node, list): for i, value in enumerate(node): out.extend(pod_templates(value, "%s[%d]" % (path, i))) return out def name_of(doc): meta = doc.get("metadata") if isinstance(meta, dict) and meta.get("name"): return str(meta["name"]) return "" % doc.get("kind", "object") def labels_of(template): meta = template.get("metadata") labels = meta.get("labels") if isinstance(meta, dict) else None return labels if isinstance(labels, dict) else {} def matches(selector, labels): """matchLabels only. Every key must be present with the same value; the comparison is on the string form, because YAML true and the string "true" are the same label value to Kubernetes.""" for key, value in selector.items(): if key not in labels or str(labels[key]) != str(value): return False return True all_templates = [] for doc in docs: all_templates.extend(pod_templates(doc, "%s/%s" % (doc.get("kind", "?"), name_of(doc)))) policies = [d for d in docs if d.get("kind") == "NetworkPolicy"] services = [d for d in docs if d.get("kind") == "Service"] def container_ports(template): """[(number, protocol, name)] over containers AND initContainers. Init containers are included because a sidecar-style one (restartPolicy: Always) keeps listening for the life of the pod, and its port is as real as any other.""" out = [] spec = template.get("spec") or {} for key in ("containers", "initContainers"): for container in spec.get(key) or []: if not isinstance(container, dict): continue for port in container.get("ports") or []: if not isinstance(port, dict) or port.get("containerPort") is None: continue out.append((int(port["containerPort"]), str(port.get("protocol") or "TCP"), port.get("name"))) return out def probe_ports(template): """Ports a NETWORK probe addresses, as {number: [who]}. exec probes are deliberately absent, and that distinction is the whole reason this is a function rather than a grep for "Probe": an exec probe runs inside the container through the kubelet's runtime and puts no packet on the network at all. A chart with exec-only probes (valkey, minecraft-server) can therefore restrict its port by source with no probeSources whatsoever, and treating its probe as network traffic would demand a value it does not need and would weaken the policy it can have.""" found = {} by_name = dict((n, num) for num, _p, n in container_ports(template) if n) spec = template.get("spec") or {} for key in ("containers", "initContainers"): for container in spec.get(key) or []: if not isinstance(container, dict): continue for probe_key in ("livenessProbe", "readinessProbe", "startupProbe"): probe = container.get(probe_key) if not isinstance(probe, dict): continue for handler in ("httpGet", "tcpSocket", "grpc"): handler_spec = probe.get(handler) if not isinstance(handler_spec, dict): continue raw = handler_spec.get("port") number = None if isinstance(raw, int): number = raw elif isinstance(raw, str) and raw.isdigit(): number = int(raw) elif isinstance(raw, str) and raw in by_name: number = by_name[raw] elif isinstance(raw, str): skip("%s of container %s names port %r, which is no " "containerPort of the same pod spec - the probe port " "could not be resolved, so the probe path was not " "checked for it. Reported, not counted as passing." % (probe_key, container.get("name"), raw)) continue if number is not None: found.setdefault(number, []).append( "%s.%s/%s" % (container.get("name"), probe_key, handler)) return found def nodeport_pod_ports(): """Pod ports published through a NodePort Service of this render, as {pod port: [service:nodePort]}. Traffic on these arrives from OUTSIDE the cluster and is already masqueraded by the time it reaches the pod, so the source address is a node and not the real client - which is why a rule for such a port legitimately carries no source selector (assertion 3).""" out = {} for svc in services: spec = svc.get("spec") or {} if spec.get("type") != "NodePort" or not isinstance(spec.get("selector"), dict): continue hits = [t for _w, t in all_templates if matches(spec["selector"], labels_of(t))] by_name = dict((n, num) for t in hits for num, _p, n in container_ports(t) if n) for port in spec.get("ports") or []: if not isinstance(port, dict) or not port.get("nodePort"): continue target = port.get("targetPort") if target is None: target = port.get("port") number = None if isinstance(target, int): number = target elif isinstance(target, str): number = int(target) if target.isdigit() else by_name.get(target) if number is not None: out.setdefault(number, []).append( "%s:%s" % (name_of(svc), port.get("nodePort"))) return out nodeports = nodeport_pod_ports() raw = open(yaml_path, encoding="utf-8").read() def read(path): try: return open(path, encoding="utf-8").read() except IOError: return "" notes = read("%s/templates/NOTES.txt" % chart_dir) values_yaml = read("%s/values.yaml" % chart_dir) # The KEY, not the word. A substring test here passed a chart whose # probeSources key had been deleted while the long comment above it still # talked about the value - found by running this assertion's failure path, which # is the reason the repository runs them (README, "Two rules for every check # added here"). offers_probe_sources = bool(re.search(r"^\s*probeSources\s*:", values_yaml, re.M)) # The phrase the convention requires NOTES.txt to carry, so that an operator who # narrows probeSources has been told what a wrong CIDR costs. A phrase and not a # vague "mentions probes", because the sentence that matters is the consequence. notes_warns = bool(re.search(r"never becomes? ready", notes, re.I)) and "probeSources" in notes RESTRICTED_RULES = 0 # Counted separately from RESTRICTED_RULES, and the overlay guard uses THIS one: # a chart like keycloak restricts other rules by source in its own defaults, so # "some rule has a from" would satisfy the guard without the probe branch ever # being rendered - which is the only branch an overlay exists to reach. PROBE_RESTRICTED = 0 # --- Assertion 0: the chart ships a policy at all --------------------------- if not policies: fail("renders no NetworkPolicy at all. Since Gitea #5 every chart in this " "repository ships one, on by default, and this is the guard that keeps a " "NEW chart from arriving without one - which is exactly how garage and " "keycloak came to have none, both created after #5 was filed. Copy " "templates/xxx.networkpolicy.yaml and the networkPolicy block from " "template-chart, or change this test on purpose if the chart truly has " "nothing to restrict.") for policy in policies: tag = "NetworkPolicy %s" % name_of(policy) spec = policy.get("spec") if isinstance(policy.get("spec"), dict) else {} policy_types = spec.get("policyTypes") if not isinstance(policy_types, list): policy_types = [] # --- Assertion 1: the podSelector hits a pod spec of the SAME render ---- selector = spec.get("podSelector") selected = [] if selector is None: fail("%s has no podSelector. That field is required by the API - a policy " "without it is rejected - and 'podSelector: {}' is the spelling that " "selects EVERY pod in the namespace." % tag) elif not isinstance(selector, dict): fail("%s podSelector is not a mapping (%r)" % (tag, selector)) elif selector.get("matchExpressions") and not selector.get("matchLabels"): skip("%s selects only through matchExpressions, which this test does not " "evaluate - doing so would mean reimplementing label-selector " "semantics, i.e. a second parser. Reported, not counted as passing." % tag) selected = [t for _w, t in all_templates] else: match_labels = selector.get("matchLabels") or {} hits = [(w, t) for w, t in all_templates if matches(match_labels, labels_of(t))] selected = [t for _w, t in hits] if not match_labels: ok("%s has an empty podSelector, which selects every pod in the " "namespace - legitimate for a namespace-wide policy, as long as " "that is what was meant." % tag) selected = [t for _w, t in all_templates] elif hits: ok("%s podSelector %s matches %d pod template(s): %s" % (tag, ", ".join("%s=%s" % kv for kv in sorted(match_labels.items())), len(hits), ", ".join(w for w, _t in hits))) else: known = sorted(set("%s=%s" % kv for _w, t in all_templates for kv in labels_of(t).items())) fail("%s podSelector selects %s, which matches NONE of the %d pod " "template(s) in the same render. The object is then accepted, is " "listed by kubectl, reads as a boundary and filters nothing at " "all - and nothing else in this pipeline notices, because nothing " "is denied. The usual cause is a label renamed in the workload " "template. Labels actually present on the pod templates: %s" % (tag, ", ".join("%s=%s" % kv for kv in sorted(match_labels.items())), len(all_templates), "; ".join(known) or "none")) declared = {} for t in selected: for number, proto, pname in container_ports(t): declared.setdefault(number, set()).add(proto) if pname: declared.setdefault(pname, set()).add(proto) probed = {} for t in selected: for number, who in probe_ports(t).items(): probed.setdefault(number, []).extend(who) def rule_port_numbers(rule): """The pod ports a rule names, resolved through the selected pods' port names. Returns (numbers, unresolved_names).""" numbers, unresolved = set(), [] for port in rule.get("ports") or []: if not isinstance(port, dict): continue value = port.get("port") if isinstance(value, int): numbers.add(value) elif isinstance(value, str) and value.isdigit(): numbers.add(int(value)) elif isinstance(value, str): hit = [num for t in selected for num, _p, n in container_ports(t) if n == value] if hit: numbers.update(hit) else: unresolved.append(value) return numbers, unresolved # --- Assertion 2: every INGRESS port is a containerPort of those pods --- # Ingress only, and that restriction is not an oversight: an egress rule's # port is a port on the DESTINATION and has nothing to do with this pod's # containerPorts, so holding egress against them would fail every chart that # reaches a database. # # THE ONE NAMED EXCEPTION, AND IT IS NOT GRANTED HERE (Gitea #143). Port # 15008 - ztunnel's HBONE tunnel - is terminated by the mesh and not by the # application, so it is legitimately no containerPort while # networkPolicy.ingress.ambientMesh is true. This loop renders every chart # with that value at its DEFAULT, which is false, so for this assertion # 15008 is an ordinary number and stays an ordinary failure - that is the # negative half of the exception and it costs nothing. The exception itself # lives in assertion 7, on a SECOND render with the value on, and it permits # exactly that one number. It is deliberately not implemented as "skip # operator-supplied ports when a gate is set": that was design A in #143 and # would have opened this assertion for all 30 charts, letting every mistyped # ingress port through - including the NodePort mistake this assertion # exists for. for i, rule in enumerate(spec.get("ingress") or []): if not isinstance(rule, dict): fail("%s ingress[%d] is not a mapping" % (tag, i)) continue for j, port in enumerate(rule.get("ports") or []): if not isinstance(port, dict): fail("%s ingress[%d].ports[%d] is not a mapping" % (tag, i, j)) continue if port.get("endPort") is not None: skip("%s ingress[%d].ports[%d] is a port RANGE (endPort %s), which " "this test does not expand. Reported, not counted as passing." % (tag, i, j, port.get("endPort"))) continue value = port.get("port") if value is None: skip("%s ingress[%d].ports[%d] names no port, which allows EVERY " "port for the given protocol. Reported, not counted as passing " "- and worth a second look, because it is a far wider rule " "than it looks." % (tag, i, j)) continue key = int(value) if isinstance(value, int) or ( isinstance(value, str) and value.isdigit()) else value if not selected: fail("%s ingress[%d].ports[%d] names port %r, but the policy selects " "no pod template at all, so no containerPort could be checked." % (tag, i, j, value)) continue if key in declared: proto = str(port.get("protocol") or "TCP") if proto in declared[key]: ok("%s ingress[%d] port %s/%s is a containerPort of the selected " "pod spec" % (tag, i, value, proto)) else: fail("%s ingress[%d].ports[%d] allows %s/%s, but the selected pod " "spec declares that port only as %s. A policy written for " "the wrong protocol filters none of the traffic that " "actually arrives." % (tag, i, j, value, proto, "/".join(sorted(declared[key])))) else: numeric = sorted(k for k in declared if isinstance(k, int)) named = sorted(str(k) for k in declared if not isinstance(k, int)) hint = "" if isinstance(key, int) and 30000 <= key <= 32767: hint = (" THAT NUMBER IS IN THE NODEPORT RANGE, which makes this " "almost certainly the NodePort mistake: a NetworkPolicy " "is evaluated at the POD's interface, after kube-proxy " "has already translated the NodePort to the targetPort. " "The rule therefore matches nothing while looking strict " "- and being the only rule for that port, it blocks every " "inbound connection instead. Use the containerPort.") fail("%s ingress[%d].ports[%d] allows port %r, which is NOT a " "containerPort of any pod spec this policy selects.%s " "containerPorts actually declared: %s%s" % (tag, i, j, value, hint, ", ".join(str(n) for n in numeric) or "none", (" (names: %s)" % ", ".join(named)) if named else "")) # --- Assertion 3: no ingress rule that silently allows every source ----- # The plan's wording was "no rule with a missing or empty from". Implemented # in the two halves it really has, because the precedents deliberately use # one of them: # # from: [] - present but EMPTY. Always a failure. It allows every source, # exactly like no key at all, while reading as a restriction. # There is no case where it is the right way to write either # intention. # no "from" - the documented open form; valkey and stalwart both use it on # purpose. Accepted ONLY where the render itself justifies # being open: every port of the rule is addressed by a network # probe of the selected pods (the probe comes from the node's # IP, which no chart can know), or is published by a NodePort # Service of the same render (the source is outside the cluster # and already masqueraded). Anything else is an allow-all # nobody chose, which is the case this assertion exists for. if "Ingress" in policy_types or (spec.get("ingress") and not policy_types): for i, rule in enumerate(spec.get("ingress") or []): if not isinstance(rule, dict): continue if "from" in rule and rule["from"] == []: fail("%s ingress[%d] has an EMPTY \"from: []\". That allows every " "source - identically to having no \"from\" at all - while " "reading in a diff and in kubectl output as a restriction. It " "is the one mistake in this class that makes a policy weaker " "while looking stricter. Write no \"from\" key where open is " "meant, and say so in a template comment, or spell the sources " "out." % (tag, i)) continue if "from" in rule: sources = rule["from"] if not isinstance(sources, list): fail("%s ingress[%d].from is not a list (%r)" % (tag, i, sources)) continue empty = [k for k, s in enumerate(sources) if isinstance(s, dict) and not s] if empty: fail("%s ingress[%d].from contains the empty peer {} at index %s, " "which matches every pod in every namespace and makes the " "whole rule open while the surrounding list looks specific." % (tag, i, ", ".join(str(k) for k in empty))) else: RESTRICTED_RULES += 1 ok("%s ingress[%d] restricts by source (%d peer(s))" % (tag, i, len(sources))) continue numbers, unresolved = rule_port_numbers(rule) if unresolved: skip("%s ingress[%d] names port(s) %s that resolve to no " "containerPort, so this rule's openness could not be judged. " "Reported, not counted as passing." % (tag, i, ", ".join(unresolved))) continue if not numbers: fail("%s ingress[%d] has neither \"from\" nor \"ports\". That is the " "widest rule the object can express - every port from every " "source - and it makes every other rule in the policy " "decorative." % (tag, i)) continue unjustified = sorted(n for n in numbers if n not in probed and n not in nodeports) if unjustified: fail("%s ingress[%d] carries no \"from\", so it is open to every " "source the CNI routes to, and port(s) %s justify neither: no " "network probe of the selected pods addresses them, and no " "NodePort Service of this render publishes them. Those are the " "only two reasons this convention leaves a rule open - the " "kubelet probes from the node's own IP, which a chart cannot " "know, and NodePort traffic arrives already masqueraded. " "Restrict this rule by source, or split the genuinely open " "ports into their own rule and say in a template comment why " "they are open." % (tag, i, ", ".join(str(n) for n in unjustified))) else: why = [] for n in sorted(numbers): if n in probed: why.append("%d (probe path: %s)" % (n, ", ".join(probed[n]))) else: why.append("%d (NodePort %s)" % (n, ", ".join(nodeports[n]))) ok("%s ingress[%d] is deliberately open, and the render justifies " "it: %s" % (tag, i, "; ".join(why))) # --- Assertion 4: the probe path ----------------------------------------- # For every port a NETWORK probe addresses, the policy either does not # restrict it by source, or the chart offers probeSources AND its NOTES.txt # says what a wrong CIDR costs. The most dangerous trap over 29 charts, and # the only one the install-test shows only after minutes - as a rollout # timeout whose cause is three steps away. if selected and not probed: ok("%s selects pods that have no network probe at all (exec probes, or " "none), so no probe traffic crosses the network and every port may be " "restricted by source without probeSources." % tag) for port_number, who in sorted(probed.items()): restricted, covered = [], False for i, rule in enumerate(spec.get("ingress") or []): if not isinstance(rule, dict): continue numbers, _unresolved = rule_port_numbers(rule) if not rule.get("ports"): numbers = set([port_number]) # a rule without ports covers all of them if port_number not in numbers: continue covered = True if rule.get("from"): restricted.append(i) if "Ingress" not in policy_types and not (spec.get("ingress") and not policy_types): ok("%s does not restrict the Ingress direction at all, so the probe on " "port %d (%s) is unaffected." % (tag, port_number, ", ".join(who))) continue if not covered: fail("%s selects a pod whose probes address port %d (%s), but NO ingress " "rule of this policy allows that port. With policyTypes Ingress the " "kubelet's probe is then dropped and THE POD NEVER BECOMES READY - " "the rollout times out and the symptom reads as a broken " "application rather than as a policy." % (tag, port_number, ", ".join(who))) continue if not restricted: ok("%s leaves probe port %d (%s) unrestricted by source, which is the " "safe default: the kubelet probes from the IP of the node running " "the pod, and no chart can know it." % (tag, port_number, ", ".join(who))) continue if offers_probe_sources and notes_warns: PROBE_RESTRICTED += 1 ok("%s restricts probe port %d (%s) by source in ingress[%s], and the " "chart carries what the convention requires with it: " "networkPolicy.probeSources in values.yaml and the consequence " "spelled out in NOTES.txt." % (tag, port_number, ", ".join(who), ", ".join(str(r) for r in restricted))) continue missing = [] if not offers_probe_sources: missing.append("values.yaml offers no networkPolicy.probeSources") if not notes_warns: missing.append("templates/NOTES.txt does not say that a probeSources list " "missing the node's IP leaves the pod permanently un-Ready " "(this test looks for the phrase \"never becomes Ready\" " "together with the value's name)") fail("%s restricts probe port %d (%s) by source in ingress[%s] while %s. A " "restricted probe port is only defensible when the operator can widen " "it AND has been told what happens if the CIDR is wrong: the kubelet " "probes from the IP of the WORKER NODE running the pod, so a list that " "misses it drops every probe and the pod never becomes Ready." % (tag, port_number, ", ".join(who), ", ".join(str(r) for r in restricted), " and ".join(missing))) # --- Assertion 5: no reasoning left in the rendered manifest ---------------- # Not one of the original four, and added because it hit several rounds of this # rollout: a "#" comment is part of the RENDERED object. It travels into the # cluster, shows up in `helm get manifest` and `kubectl get -o yaml`, and puts # the chart's reasoning in front of everyone who diffs a release. Go template # comments render nothing and are the place for it. It can only be checked on # the raw YAML - the comments are gone once ci/yaml2json has read the stream. # SUBCHART DOCUMENTS ARE EXCLUDED FROM THIS ONE, and the reason is structural # rather than lenient. A dependency is fetched from the Helm registry, not from # the copy in this repository, so the NetworkPolicy that paperless-ngx and # stalwart bundle is whatever version of valkey is PUBLISHED - and nothing in # the parent chart can change its rendered comments. Holding the parent # responsible would make this test unfixable until the subchart's next release, # and would fail a chart for a file its own diff does not contain. The # subchart's own pull request is where that is caught, because this test runs # over every chart including valkey itself. # # Recognised from the "# Source:" header Helm writes above each document: a # subchart's path contains "/charts/". if policies: current, offenders = [], [] for line in raw.splitlines() + ["---"]: if line.strip() == "---": body = "\n".join(current) source = re.search(r"^# Source:\s*(\S+)", body, re.M) from_subchart = bool(source) and "/charts/" in source.group(1) if (re.search(r"^kind:\s*NetworkPolicy\s*$", body, re.M) and not from_subchart): for raw_line in current: stripped = raw_line.strip() if stripped.startswith("#") and not stripped.startswith("# Source:"): offenders.append(stripped) current = [] else: current.append(line) if offenders: fail("the rendered NetworkPolicy manifest(s) contain %d \"#\" comment line(s), " "which are part of the OBJECT and are shipped to the cluster: they appear " "in `helm get manifest` and in `kubectl get -o yaml`, and they put the " "chart's reasoning into every release diff. Move them into Go template " "comments, which render nothing. First offender: %s" % (len(offenders), offenders[0][:160])) else: ok("the rendered NetworkPolicy manifest(s) carry no \"#\" comment - all " "reasoning stayed in template comments") print("OFFERSPS|%d" % (1 if offers_probe_sources else 0)) print("RESTRICTED|%d" % RESTRICTED_RULES) print("PROBERESTRICTED|%d" % PROBE_RESTRICTED) for line in lines: print(line) PY RC=$? set -e if [ "$RC" -gt 1 ]; then sed 's/^/ /' "$WORK/$CHART.pyerr" echo "::endgroup::" echo "::error title=networkpolicy $CHART::the policy comparison itself failed for $CHART (python exit $RC). Treated as a failure, never as a pass." FAILED="$FAILED $CHART" continue fi CHART_FAIL="" CHART_OK=0 CHART_SKIP=0 CHART_RESTRICTED=0 CHART_PROBE_RESTRICTED=0 CHART_OFFERS_PS=0 while IFS='|' read -r VERDICT DETAIL; do case "$VERDICT" in OK) CHART_OK=$((CHART_OK + 1)) echo " ok $DETAIL" ;; SKIP) CHART_SKIP=$((CHART_SKIP + 1)) echo " gap $DETAIL" ;; RESTRICTED) CHART_RESTRICTED="$DETAIL" ;; PROBERESTRICTED) CHART_PROBE_RESTRICTED="$DETAIL" ;; OFFERSPS) CHART_OFFERS_PS="$DETAIL" ;; FAIL) CHART_FAIL=1 echo " FAIL $DETAIL" echo "::error title=networkpolicy $CHART::$DETAIL" ;; *) [ -n "$VERDICT" ] && echo " $VERDICT $DETAIL" ;; esac done <"$WORK/$CHART.out" echo "::endgroup::" TOTAL_ASSERTIONS=$((TOTAL_ASSERTIONS + CHART_OK)) if [ -n "$CHART_FAIL" ]; then FAILED="$FAILED $CHART" SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | **failed** | $CHART_OK | $CHART_SKIP |"$'\n' echo "$CHART: FAILED" elif [ -n "$HAS_OVERLAY" ] && [ "$CHART_OFFERS_PS" -eq 1 ] && [ "$CHART_PROBE_RESTRICTED" -eq 0 ]; then # The overlay's ONE job is to render a code path the chart's defaults do not # reach - above all the RESTRICTED probe rule, which only appears once # probeSources is non-empty. If no restricted rule came out, this test is # vacuum-green for the very chart the overlay was written for. FAILED="$FAILED $CHART" SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | **failed** | $CHART_OK | $CHART_SKIP |"$'\n' echo "::error title=networkpolicy $CHART::$CHART has an overlay at ci/networkpolicy/$CHART.yaml but rendered no source-restricted PROBE rule, so the restricted branch - the one the chart's own defaults never reach - was not checked. That overlay exists for exactly this; either it no longer sets networkPolicy.probeSources where the chart reads it, or the template stopped rendering the restricted form. A green tick here would be a tick for a check that ran over nothing." echo "$CHART: FAILED (overlay present, no restricted probe rule rendered)" else CHECKED="$CHECKED $CHART" [ "$CHART_SKIP" -gt 0 ] && WITH_GAPS="$WITH_GAPS $CHART" SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CHART\` | ok | $CHART_OK | $CHART_SKIP |"$'\n' echo "$CHART: ok ($CHART_OK assertion(s) held, $CHART_SKIP not checked, $CHART_RESTRICTED source-restricted rule(s), $CHART_PROBE_RESTRICTED of them on a probe port)" fi done # --- Assertion 6: the CROSS-CHART egress ports (Gitea #136) ------------------ # The only assertion here that needs more than one chart's render, and the # reason it exists is an outage. paperless-ngx' egress rule for Tika and # Gotenberg named 80 - the port those two charts' SERVICES listen on - and # justified it in a template comment. kube-proxy does a ClusterIP Service's # destination NAT BEFORE the egress policy is evaluated at the pod's network # interface, so the policy sees podIP:targetPort; 80 matched nothing, and the # release that shipped it cut Paperless off from both services on 2026-10-15. # The symptom was a TIMEOUT, not "connection refused", and no error mentioned a # NetworkPolicy. # # WHY ASSERTION 2 CANNOT CATCH THIS, and why that is not a defect in it: it # holds ports against the pod spec the policy SELECTS, which is right for # ingress and meaningless for egress - an egress port belongs to the # DESTINATION. For a database, a cache or the web that destination is not in any # render this repository has. It is in a render only when one chart of THIS # repository dials the Service of another, and paperless-ngx -> apache-tika, # gotenberg is the single place where that happens (checked across the # repository for #136; every other egress port is either a 1:1 Service, so the # mistake is invisible, or outside the cluster, where there is no DNAT at all # and the port named IS the real one). # # So the table below is deliberately not a discovered list: a chart cannot know # which Service another chart dials, because the address lives in a value or a # 1Password item. Adding a row is the price of the next cross-chart egress rule, # and a row that names a chart which no longer exists FAILS rather than being # skipped - the failure path was executed, like the others. # # WHAT IS ASSERTED, per row, on TWO renders of the consumer: # gate ON - the consumer's egress ports must CONTAIN every pod port behind # the peers' Services, resolved through the peers' own renders # (Service port -> targetPort -> containerPort, i.e. exactly the # translation the policy is evaluated after). # gate OFF - the containment requirement does NOT apply (the rule is gated, # and a release without the feature must not carry a rule that # permits nothing), and those ports must be absent. # # Built into THIS script rather than added as a new test with its own pipeline # job, because everything it needs is already here: the job runs over every # chart on every pull request (so it cannot be skipped by a path filter), Helm # is already the YAML parser through ci/yaml2json, and the verdict, grouping and # summary plumbing is shared. A second script would duplicate all of it to add # one comparison. # # consumer|peer charts|gate value|extra --set needed to render the gate ON EGRESS_PEER_ROWS="paperless-ngx|apache-tika,gotenberg|paperless.files.tikaGotenberg.enabled|paperless.files.tikaGotenberg.tikaEndpoint=http://tika.ci.local,paperless.files.tikaGotenberg.gotenbergEndpoint=http://gotenberg.ci.local" # Renders one chart with the same values precedence the loop above uses and # leaves the parsed JSON in $2. The render/split/parse steps are repeated here # instead of refactored out of the loop on purpose: the five assertions above # are load-bearing and were not worth disturbing for this. # RCJ_WITH_OVERLAY=1 additionally applies ci/networkpolicy/.yaml, last, # exactly as the main loop does. Off by default so the cross-chart rows below # keep rendering the consumer the way its release does; assertion 7 switches it # on, because the overlay is the ONLY way the source-restricted branch - and # with it the probe-source half of the ambient switch - is reachable at all. render_chart_json() { RCJ_CHART="$1" RCJ_OUT="$2" shift 2 RCJ_VALUES=() RCJ_LABEL="chart defaults" if [ -f "$CI_DIR/$RCJ_CHART/test-values.yaml" ]; then RCJ_VALUES=(-f "$CI_DIR/$RCJ_CHART/test-values.yaml") RCJ_LABEL="ci/$RCJ_CHART/test-values.yaml" elif [ -f "$CI_DIR/trivy/values/$RCJ_CHART.yaml" ]; then RCJ_VALUES=(-f "$CI_DIR/trivy/values/$RCJ_CHART.yaml") RCJ_LABEL="ci/trivy/values/$RCJ_CHART.yaml" fi if [ "${RCJ_WITH_OVERLAY:-}" = "1" ] && [ -f "$OVERLAY_DIR/$RCJ_CHART.yaml" ]; then RCJ_VALUES+=(-f "$OVERLAY_DIR/$RCJ_CHART.yaml") fi "$ROOT/ci/helm-dependencies.sh" "$ROOT/$RCJ_CHART" >/dev/null || return 1 RCJ_MANIFEST="$RCJ_OUT.manifest.yaml" helm template "$RELEASE" "$ROOT/$RCJ_CHART" \ ${RCJ_VALUES[@]+"${RCJ_VALUES[@]}"} "$@" >"$RCJ_MANIFEST" 2>"$WORK/xerr" || return 1 awk ' BEGIN { print "docs:"; started = 0 } /^---[ \t]*$/ { print " -"; started = 1; next } { if (!started) { print " -"; started = 1 } if (length($0) == 0) print ""; else print " " $0 } ' "$RCJ_MANIFEST" >"$RCJ_MANIFEST.docs" helm template y2j "$YAML2JSON" -f "$RCJ_MANIFEST.docs" 2>"$WORK/xerr" \ | awk '/^\{/ { print; exit }' >"$RCJ_OUT" || return 1 [ -s "$RCJ_OUT" ] || return 1 return 0 } CROSS_ROWS=0 for ROW in $EGRESS_PEER_ROWS; do CONSUMER="$(printf '%s' "$ROW" | cut -d'|' -f1)" PEERS="$(printf '%s' "$ROW" | cut -d'|' -f2 | tr ',' ' ')" GATE="$(printf '%s' "$ROW" | cut -d'|' -f3)" GATE_SETS="$(printf '%s' "$ROW" | cut -d'|' -f4)" # Only when the consumer is in this run's chart list, so a single-chart # invocation stays single-chart. The pipeline passes no arguments, so in CI # every row always runs. IN_RUN="" for C in "${CHARTS[@]}"; do [ "$C" = "$CONSUMER" ] && IN_RUN=1; done [ -n "$IN_RUN" ] || continue echo "::group::cross-chart egress: $CONSUMER -> $PEERS" CROSS_ROWS=$((CROSS_ROWS + 1)) ROW_FAIL="" if [ ! -f "$ROOT/$CONSUMER/Chart.yaml" ]; then echo " FAIL $CONSUMER is named in EGRESS_PEER_ROWS but is not a chart directory" echo "::error title=cross-chart egress::$CONSUMER is named in EGRESS_PEER_ROWS in ci/run-networkpolicy-test.sh but is not a chart directory. If the chart was renamed or removed, update that table - a row that silently does nothing is how this check would decay back into the state that caused Gitea #136." ROW_FAIL=1 fi PEER_ARGS=() for PEER in $PEERS; do if [ ! -f "$ROOT/$PEER/Chart.yaml" ]; then echo " FAIL peer chart $PEER does not exist" echo "::error title=cross-chart egress::$CONSUMER's egress rule is checked against the chart $PEER, which is not a chart directory of this repository. Either it was renamed - in which case the row in EGRESS_PEER_ROWS in ci/run-networkpolicy-test.sh has to follow, and $CONSUMER's rule probably too - or the row is stale. Not skipped, because an unchecked row reads exactly like a passing one." ROW_FAIL=1 continue fi if ! render_chart_json "$PEER" "$WORK/peer-$PEER.json"; then sed 's/^/ /' "$WORK/xerr" echo " FAIL peer chart $PEER does not render" echo "::error title=cross-chart egress::the peer chart $PEER does not render with its CI values, so the pod ports behind its Services could not be read and $CONSUMER's egress rule could not be checked against them." ROW_FAIL=1 continue fi PEER_ARGS+=("$PEER=$WORK/peer-$PEER.json") done ON_JSON="$WORK/consumer-$CONSUMER-on.json" OFF_JSON="$WORK/consumer-$CONSUMER-off.json" if [ -z "$ROW_FAIL" ]; then SET_ARGS=(--set "$GATE=true") OLD_IFS="$IFS"; IFS=',' for S in $GATE_SETS; do [ -n "$S" ] && SET_ARGS+=(--set "$S"); done IFS="$OLD_IFS" if ! render_chart_json "$CONSUMER" "$ON_JSON" "${SET_ARGS[@]}"; then sed 's/^/ /' "$WORK/xerr" echo " FAIL $CONSUMER does not render with $GATE=true" echo "::error title=cross-chart egress::$CONSUMER does not render with $GATE=true and the --set values from EGRESS_PEER_ROWS, so its egress rule for $PEERS was never produced and nothing was checked. If the feature needs further values, add them to that row." ROW_FAIL=1 elif ! render_chart_json "$CONSUMER" "$OFF_JSON" --set "$GATE=false"; then sed 's/^/ /' "$WORK/xerr" echo " FAIL $CONSUMER does not render with $GATE=false" echo "::error title=cross-chart egress::$CONSUMER does not render with $GATE=false, so the gated-off case - in which this assertion must NOT fire - could not be checked." ROW_FAIL=1 fi fi if [ -z "$ROW_FAIL" ]; then set +e python3 - "$CONSUMER" "$RELEASE" "$GATE" "$ON_JSON" "$OFF_JSON" "${PEER_ARGS[@]}" \ >"$WORK/cross-$CONSUMER.out" 2>"$WORK/cross-$CONSUMER.pyerr" <<'PY' import json, sys consumer, release, gate, on_path, off_path = sys.argv[1:6] peers = [a.split("=", 1) for a in sys.argv[6:]] lines = [] def ok(msg): lines.append("OK|%s: %s" % (consumer, msg)) def skip(msg): lines.append("SKIP|%s: %s" % (consumer, msg)) def fail(msg): lines.append("FAIL|%s: %s" % (consumer, msg)) def docs_of(path): return [d for d in (json.load(open(path)).get("docs") or []) if isinstance(d, dict)] def pod_templates(node): out = [] if isinstance(node, dict): for key, value in node.items(): if (key == "template" and isinstance(value, dict) and isinstance(value.get("spec"), dict) and "containers" in value["spec"]): out.append(value) out.extend(pod_templates(value)) elif isinstance(node, list): for value in node: out.extend(pod_templates(value)) return out def labels_of(template): meta = template.get("metadata") labels = meta.get("labels") if isinstance(meta, dict) else None return labels if isinstance(labels, dict) else {} def matches(selector, labels): for key, value in selector.items(): if key not in labels or str(labels[key]) != str(value): return False return True def container_ports(template): out = [] spec = template.get("spec") or {} for key in ("containers", "initContainers"): for container in spec.get(key) or []: if not isinstance(container, dict): continue for port in container.get("ports") or []: if isinstance(port, dict) and port.get("containerPort") is not None: out.append((int(port["containerPort"]), port.get("name"))) return out def service_pod_ports(path, chart): """The pod ports a client reaches THROUGH this chart's Services, as {pod port: [evidence]}. Resolved Service port -> targetPort -> containerPort, because that chain IS the destination NAT the egress policy is evaluated after: the packet leaving the consumer carries the Service port, and the packet the policy sees carries the pod port.""" docs = docs_of(path) templates = [] for doc in docs: templates.extend(pod_templates(doc)) out, unresolved = {}, [] for svc in docs: if svc.get("kind") != "Service": continue spec = svc.get("spec") or {} selector = spec.get("selector") hits = [t for t in templates if isinstance(selector, dict) and matches(selector, labels_of(t))] by_name = dict((n, num) for t in hits for num, n in container_ports(t) if n) svc_name = ((svc.get("metadata") or {}).get("name")) or "" for port in spec.get("ports") or []: if not isinstance(port, dict): continue target = port.get("targetPort") if target is None: target = port.get("port") number = None if isinstance(target, int): number = target elif isinstance(target, str): number = int(target) if target.isdigit() else by_name.get(target) if number is None: unresolved.append("%s %s: targetPort %r" % (chart, svc_name, target)) continue out.setdefault(number, []).append( "%s %s: port %s -> targetPort %r -> pod port %d" % (chart, svc_name, port.get("port"), target, number)) return out, unresolved def egress_ports(path): """(numbers, policy names, has a rule with no ports at all) for the consumer's OWN policy only - recognised by the -- name prefix, so a subchart's policy in the same render cannot satisfy a rule the parent chart is responsible for.""" prefix = "%s-%s-" % (release, consumer) numbers, names, open_rule = set(), [], False for doc in docs_of(path): if doc.get("kind") != "NetworkPolicy": continue name = ((doc.get("metadata") or {}).get("name")) or "" if not name.startswith(prefix): continue names.append(name) spec = doc.get("spec") if isinstance(doc.get("spec"), dict) else {} for rule in spec.get("egress") or []: if not isinstance(rule, dict): continue if not rule.get("ports"): open_rule = True continue for port in rule["ports"]: if not isinstance(port, dict): continue value = port.get("port") if isinstance(value, int): numbers.add(value) elif isinstance(value, str) and value.isdigit(): numbers.add(int(value)) return numbers, names, open_rule required, unresolved = {}, [] for chart, path in peers: ports, bad = service_pod_ports(path, chart) for number, evidence in ports.items(): required.setdefault(number, []).extend(evidence) unresolved.extend(bad) for note in unresolved: skip("a Service targetPort of a peer chart could not be resolved to a " "containerPort, so that port is not part of the requirement (%s). " "Reported, not counted as passing." % note) on_numbers, on_names, on_open = egress_ports(on_path) off_numbers, off_names, _off_open = egress_ports(off_path) if not required: fail("the peer charts render no Service whose port resolves to a pod port, " "so this assertion would have required nothing at all. That is " "indistinguishable from a passing check and is treated as a failure: " "either the peer charts stopped shipping a Service, or their targetPort " "no longer resolves.") elif not on_names: fail("with %s=true the render contains no NetworkPolicy named %s-%s-*, so " "the egress rule this assertion is about was not found. The policy's " "name is the convention {{ .Release.Name }}-{{ .Chart.Name }}-; if " "it changed, this check has to follow." % (gate, release, consumer)) elif on_open: skip("an egress rule of %s names NO port, which allows every port - so " "containment of the peers' pod ports is satisfied without saying " "anything. Reported, never counted as passing." % ", ".join(on_names)) else: missing = sorted(n for n in required if n not in on_numbers) if missing: fail("with %s=true, %s permits outbound %s but NOT %s - and those are the " "POD ports behind the peer charts' Services: %s. An egress rule is " "evaluated at the pod's network interface, AFTER kube-proxy has " "translated the ClusterIP Service's destination (DNAT), so the policy " "sees podIP:targetPort and a rule naming the SERVICE port matches " "nothing wherever the two numbers differ. This is Gitea #136 " "exactly: it cost an outage on 2026-10-15, and the symptom was a " "TIMEOUT rather than \"connection refused\", with no error " "mentioning a NetworkPolicy. Name the pod port(s) in the rule." % (gate, ", ".join(on_names), ", ".join(str(n) for n in sorted(on_numbers)), ", ".join(str(n) for n in missing), "; ".join(e for n in missing for e in required[n]))) else: ok("with %s=true, %s permits every pod port behind the peer charts' " "Services: %s" % (gate, ", ".join(on_names), "; ".join(e for n in sorted(required) for e in required[n]))) # The gated-off half. The requirement above must NOT apply here - a release # without the feature is not expected to reach the peers - and the ports must # be gone, or the rule is no longer gated and permits something the release # has no use for. present = sorted(n for n in required if n in off_numbers) if not off_names: skip("with %s=false the render contains no NetworkPolicy of this chart, " "so the gated-off case could not be read." % gate) elif present: fail("with %s=false, %s still permits outbound %s, the peer charts' pod " "port(s). Either the rule is no longer gated on that value - a rule " "that permits what the release cannot use - or the CI values add " "those ports by hand, which would make the gate-on assertion above " "pass without the rule existing." % (gate, ", ".join(off_names), ", ".join(str(n) for n in present))) else: ok("with %s=false the rule is absent (egress: %s), so the containment " "requirement correctly does not apply and no release without the " "feature carries a rule that permits nothing." % (gate, ", ".join(str(n) for n in sorted(off_numbers)) or "none")) for line in lines: print(line) PY RC=$? set -e if [ "$RC" -gt 1 ]; then sed 's/^/ /' "$WORK/cross-$CONSUMER.pyerr" echo "::error title=cross-chart egress::the cross-chart egress comparison itself failed for $CONSUMER (python exit $RC). Treated as a failure, never as a pass." ROW_FAIL=1 else while IFS='|' read -r VERDICT DETAIL; do case "$VERDICT" in OK) TOTAL_ASSERTIONS=$((TOTAL_ASSERTIONS + 1)) echo " ok $DETAIL" ;; SKIP) echo " gap $DETAIL" ;; FAIL) ROW_FAIL=1 echo " FAIL $DETAIL" echo "::error title=cross-chart egress::$DETAIL" ;; *) [ -n "$VERDICT" ] && echo " $VERDICT $DETAIL" ;; esac done <"$WORK/cross-$CONSUMER.out" fi fi echo "::endgroup::" if [ -n "$ROW_FAIL" ]; then FAILED="$FAILED $CONSUMER(cross-chart)" SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CONSUMER\` → \`$(printf '%s' "$PEERS" | tr ' ' ',')\` | **failed** | – | – |"$'\n' echo "cross-chart egress $CONSUMER: FAILED" else SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CONSUMER\` → \`$(printf '%s' "$PEERS" | tr ' ' ',')\` | ok | 2 | 0 |"$'\n' echo "cross-chart egress $CONSUMER: ok" fi done # --- Assertion 7: the ambient-mesh switch (Gitea #143) ----------------------- # networkPolicy.ingress.ambientMesh renders the two things a pod needs before it # can be reached inside Istio's ambient mesh, and the whole design question of # #143 was how to allow the first of them without blunting assertion 2: # # 15008/TCP inbound - ztunnel's HBONE tunnel. Terminated by the # mesh, so it is NOT a containerPort of the # application. Assertion 2 would reject it, and # it is right to reject it by default. # 169.254.7.127/32 as a peer - where probeSources is non-empty. Inside the # mesh the kubelet's probes are redirected and # no longer arrive from the worker node's # address. # # WHAT THIS ASSERTION IS FOR. The exception assertion 2 grants is NAMED and # single-valued: one port number, gated on one value. The danger of any # exception is that it stops being checked and becomes the hole it was carved # around, so the exception is held to five things at once, on TWO renders of # every chart: # # the chart's DECLARED DEFAULTS (read back through ci/yaml2json) # - enabled must default to false, and the port and the probe source must be # exactly the numbers pinned below. A chart whose default drifted would # otherwise ship a port no other chart ships while the exception kept being # granted to the pinned one. # switch OFF (the main loop's render, reused) # - the pinned port must be ABSENT. Without this half the exception is a hole: a # chart could render the port unconditionally and nothing would notice, # because assertion 7 would find it present and assertion 2 is where it # would have failed. # - 169.254.7.127/32 must be absent too, for the same reason. # switch ON (only "enabled=true" is set, never a port override - the pin is a # statement about the defaults WE ship) # - the PINNED port must be PRESENT, or the value is decorative. # - the PINNED port must be the ONLY ingress port that is not a # containerPort. This is the line between a named exception and design A # of #143: every other mistyped or NodePort number still fails here, with # the switch on. Note "pinned": since #143 made the port a VALUE, checking # the exception against the rendered value would permit whatever the value # said, which is design A's hole under design C's name. # - wherever the rule carrying 15008 restricts sources, that rule's "from" # must contain 169.254.7.127/32. This is the inseparability of the two # halves, asserted rather than trusted - and it is REACHABLE, because # these renders apply ci/networkpolicy/.yaml, which is where # probeSources is non-empty (template-chart, gotenberg). # # And the policy is found by the NAME CONVENTION, --*, the same # way assertion 6 finds it: an exception must not be granted to an object this # test did not identify. A chart whose policy stops following the convention # fails here instead of silently losing its coverage. # # TWO LIMITS OF THIS ASSERTION, named rather than left to be discovered: # # - ASSERTIONS 1, 3, 4 AND 5 ARE NOT RE-RUN ON THE SWITCHED-ON RENDER. Only # assertion 2's subject matter is, because that is the one the exception # touches. Assertion 3 is the interesting omission: with the switch on and # probeSources empty, 15008 sits in a rule with no "from", and assertion 3's # two justifications for an open rule - a network probe addresses the port, # or a NodePort publishes it - cover neither. That is intended rather than # overlooked. The port has to be reachable from wherever ztunnel dials from, # and that address is not knowable from a render: ztunnel runs with its own # pod IPs in istio-system (measured 2026-10-19, no hostNetwork on the # DaemonSet, Istio 1.30.6), but whether a peer of its own is needed cannot # be settled until a namespace actually joins the mesh. Teaching assertion 3 # a third justification before then would be a rule written against a guess. # - THE PROBE-SOURCE HALF IS CONDITIONAL ON THE CHART OFFERING probeSources, # like assertion 4 and the overlay guard. A chart with exec-only probes has # no probe on the network to redirect, so the ipBlock would match nothing. # # BUILT INTO THIS SCRIPT rather than added as a new test with its own pipeline # job - the same decision as assertion 6 for #136, and here it is stronger than # convenience. The exception WEAKENS assertion 2, so it belongs next to it: a # separate script can be green while the exception it is paired with is wrong, # and nothing would connect the two. Everything else is already here as well - # the job runs over every chart on every pull request and so cannot be skipped # by a path filter, helm is already the YAML parser through ci/yaml2json, and # the verdict, grouping and summary plumbing is shared. # # EVERY CHART MUST OFFER THE VALUE, and a chart that does not fails. Same # mechanism as assertion 0: the mistake arrives by copy into a NEW chart whose # own diff nobody reads for it, which is exactly how garage (#81) and keycloak # (#83) arrived with no policy at all after #5 had been filed. # THE PINNED EXPECTATION, and it is pinned HERE rather than read from the chart # on purpose (Gitea #143). Both numbers are values with documented defaults, so # that an operator can correct an Istio constant without waiting for a chart # release - but an exception that permits "whatever the value says" is design A's # hole wearing design C's name: a mistyped port would pass simply by sitting in # the configurable field, and that is the single reason design A was rejected. # # So the test states what WE believe Istio's numbers to be, and asserts two # things against that statement: the render at CHART DEFAULTS produces exactly # this port and exactly this ipBlock, AND every chart's values.yaml DECLARES # these same defaults. A chart that drifts is red. # # Which makes this block the one place in the repository that says what we hold # Istio's constants to be - and therefore also the answer to "where does # 169.254.7.127 come from": # # 15008 Istio's HBONE port. Not registered with IANA for this, # prescribed by no protocol, unknown to Kubernetes. Istio # picked it and Istio can change it. # 169.254.7.127/32 the address Istio's data plane rewrites the kubelet's probe # source onto. LINK-LOCAL (169.254.0.0/16, RFC 3927), a range # reserved for node-local use and never routed. Istio's IPv6 # counterpart fd16:9254:7127:1337:ffff:ffff:ffff:ffff carries # the SAME DIGITS plus 1337, which is what vendor constants # look like. Neither the NetworkPolicy API nor the kubelet has # any notion of it. # # BOTH READ FOR ztunnel 1.30.6. If Istio changes either, the failure is silent - # a pod unreachable in the mesh, or probes dropped where probeSources is # non-empty, which is nowhere today. Changing them means 30 values files AND # these two lines, and these two lines are what force someone to look. # # An operator who overrides the value in their own release is outside this test's # reach. That is true of every value in this repository and is not a new gap - # and it is why these renders set ONLY "enabled=true" and never a port override: # the pin is a statement about the defaults WE ship, not about what a release may # choose. AMBIENT_PORT=15008 AMBIENT_CIDR="169.254.7.127/32" AMBIENT_VALUE="networkPolicy.ingress.ambientMesh" for CHART in "${CHARTS[@]}"; do # A chart whose main-loop render already failed has nothing to compare # against; it is reported there and would only produce a second, derived # error here. case " $FAILED " in *" $CHART "*) continue ;; esac [ -s "$WORK/$CHART.json" ] || continue echo "::group::ambient-mesh switch: $CHART" ROW_FAIL="" ON_JSON="$WORK/ambient-$CHART-on.json" if ! RCJ_WITH_OVERLAY=1 render_chart_json "$CHART" "$ON_JSON" \ --set "$AMBIENT_VALUE.enabled=true"; then sed 's/^/ /' "$WORK/xerr" echo " FAIL $CHART does not render with $AMBIENT_VALUE.enabled=true" echo "::error title=ambient-mesh $CHART::$CHART does not render with $AMBIENT_VALUE.enabled=true. Since Gitea #143 every chart in this repository offers that value, and networkPolicy carries additionalProperties: false in all of them - so an unknown key does not get ignored, it fails the render. Add the object to values.yaml AND to values.schema.json (schema and template belong together per chart), with enabled: false, port: \$AMBIENT_PORT and redirectedProbeSources: [\$AMBIENT_CIDR], and render that port plus - inside the source-restricted branch - that ipBlock. Copy template-chart/templates/xxx.networkpolicy.yaml." ROW_FAIL=1 fi # The policy's name is built from Chart.yaml's name, which is NOT always the # directory name - template-chart is called "xxx", and the whole point of that # chart is that it is a copy template. Reading the directory name here found # exactly that on the first run, so the name comes from Chart.yaml. CHART_NAME="$(sed -n 's/^name:[[:space:]]*//p' "$ROOT/$CHART/Chart.yaml" \ | head -1 | tr -d '[:space:]')" if [ -z "$CHART_NAME" ]; then echo " FAIL $CHART has no name in Chart.yaml" echo "::error title=ambient-mesh $CHART::$CHART/Chart.yaml declares no name, so the NetworkPolicy's conventional name could not be derived." ROW_FAIL=1 fi # The chart's DECLARED defaults, read with the same parser as everything else # (ci/yaml2json prints a values file back as JSON). Not a regex over # values.yaml: the pinned expectation is about numbers and a list, and a # substring test over a commented file is how the probeSources check in # assertion 4 was once fooled by a deleted key whose comment survived. VALUES_JSON="$WORK/$CHART.values.json" if [ -z "$ROW_FAIL" ]; then if ! helm template y2j "$YAML2JSON" -f "$ROOT/$CHART/values.yaml" 2>"$WORK/xerr" \ | awk '/^\{/ { print; exit }' >"$VALUES_JSON" || [ ! -s "$VALUES_JSON" ]; then sed 's/^/ /' "$WORK/xerr" echo " FAIL $CHART/values.yaml could not be re-read as JSON" echo "::error title=ambient-mesh $CHART::$CHART/values.yaml could not be re-read by ci/yaml2json, so the chart's DECLARED ambient-mesh defaults could not be compared with the numbers this test pins. That is a parse failure, not a policy finding." ROW_FAIL=1 fi fi if [ -z "$ROW_FAIL" ]; then set +e python3 - "$CHART" "$RELEASE" "$AMBIENT_VALUE" "$WORK/$CHART.json" "$ON_JSON" \ "$ROOT/$CHART" "$CHART_NAME" "$AMBIENT_PORT" "$AMBIENT_CIDR" "$VALUES_JSON" \ >"$WORK/ambient-$CHART.out" 2>"$WORK/ambient-$CHART.pyerr" <<'PY' import json, re, sys (chart, release, gate, off_path, on_path, chart_dir, chart_name, pinned_port, pinned_cidr, values_json) = sys.argv[1:11] # Pinned by the caller, never read from the chart - see the long note above # AMBIENT_PORT in this script. PORT = int(pinned_port) CIDR = pinned_cidr lines = [] def ok(msg): lines.append("OK|%s: %s" % (chart, msg)) def skip(msg): lines.append("SKIP|%s: %s" % (chart, msg)) def fail(msg): lines.append("FAIL|%s: %s" % (chart, msg)) def docs_of(path): return [d for d in (json.load(open(path)).get("docs") or []) if isinstance(d, dict)] def pod_templates(node): out = [] if isinstance(node, dict): for key, value in node.items(): if (key == "template" and isinstance(value, dict) and isinstance(value.get("spec"), dict) and "containers" in value["spec"]): out.append(value) out.extend(pod_templates(value)) elif isinstance(node, list): for value in node: out.extend(pod_templates(value)) return out def labels_of(template): meta = template.get("metadata") labels = meta.get("labels") if isinstance(meta, dict) else None return labels if isinstance(labels, dict) else {} def matches(selector, labels): for key, value in selector.items(): if key not in labels or str(labels[key]) != str(value): return False return True def container_ports(template): out = [] spec = template.get("spec") or {} for key in ("containers", "initContainers"): for container in spec.get(key) or []: if not isinstance(container, dict): continue for port in container.get("ports") or []: if isinstance(port, dict) and port.get("containerPort") is not None: out.append((int(port["containerPort"]), port.get("name"))) return out def own_policies(path): """This chart's OWN NetworkPolicies, by the -- name convention - so a bundled subchart's policy can neither satisfy nor break an assertion the parent chart is responsible for.""" prefix = "%s-%s-" % (release, chart_name) out = [] for doc in docs_of(path): if doc.get("kind") != "NetworkPolicy": continue name = ((doc.get("metadata") or {}).get("name")) or "" if name.startswith(prefix): out.append((name, doc)) return out def ingress_rules(doc): spec = doc.get("spec") if isinstance(doc.get("spec"), dict) else {} return [r for r in (spec.get("ingress") or []) if isinstance(r, dict)] def rule_ports(rule): """[(number_or_name, protocol)] for the rule's ports.""" out = [] for port in rule.get("ports") or []: if not isinstance(port, dict): continue value = port.get("port") proto = str(port.get("protocol") or "TCP") if isinstance(value, int) or (isinstance(value, str) and value.isdigit()): out.append((int(value), proto)) elif value is not None: out.append((value, proto)) return out def cidrs_of(rule): out = [] for peer in rule.get("from") or []: if isinstance(peer, dict) and isinstance(peer.get("ipBlock"), dict): out.append(str(peer["ipBlock"].get("cidr"))) return out # --- The chart DECLARES the pinned defaults ------------------------------- # The render with the switch on already proves the SCHEMA declares the key - an # undeclared key cannot pass additionalProperties: false. What that render # cannot prove is what the chart's own defaults SAY, and since #143 made both # Istio constants configurable, that is the half the pinned exception rests on: # the exception below is granted to the number THIS SCRIPT names, so a chart # whose default has drifted away from it must be red here rather than quietly # carrying a different port than every other chart. values_yaml = "" try: values_yaml = open("%s/values.yaml" % chart_dir, encoding="utf-8").read() except IOError: pass declared = json.load(open(values_json)) node = (((declared.get("networkPolicy") or {}).get("ingress") or {}) .get("ambientMesh")) if not isinstance(node, dict): fail("renders with %s.enabled=true - so the schema knows the key - but the " "chart's own values.yaml declares no ambientMesh OBJECT (found %r). A " "value that exists only in the schema is reachable through --set and " "findable nowhere: the defaults are what an operator reads. Ship the " "object with enabled: false, port: %d and redirectedProbeSources: " "[%s], as in template-chart." % (gate, node, PORT, CIDR)) else: problems = [] if node.get("enabled") not in (False, None): problems.append("enabled is %r and must default to false - the switch is " "additive and no release may get it by accident" % node.get("enabled")) if node.get("port") != PORT: problems.append("port defaults to %r, but this test pins Istio's HBONE " "port at %d" % (node.get("port"), PORT)) # Conditional on the chart offering probeSources, exactly as the # inseparability check below and assertion 4 are. A chart whose probe rule # is never source-restricted renders no ipBlock, so this key would reach no # rule - and a value that silently does nothing reads as a control, which is # worse than one that is absent (README, "Values, and shipping only the live # ones"). So for those charts the key must be ABSENT, and that is asserted # rather than tolerated. sources = node.get("redirectedProbeSources") offers_ps = bool(re.search(r"^\s*probeSources\s*:", values_yaml, re.M)) if offers_ps and sources != [CIDR]: problems.append("redirectedProbeSources defaults to %r, but this test " "pins Istio's redirected probe source at [%r]" % (sources, CIDR)) if not offers_ps and sources is not None: problems.append("redirectedProbeSources is declared (%r) although this " "chart offers no networkPolicy.probeSources, so its probe " "rule is never source-restricted and no ipBlock is ever " "rendered - the key would reach no rule and read as a " "control that does nothing" % sources) if problems: fail("the chart's declared ambient-mesh defaults do not match the " "numbers ci/run-networkpolicy-test.sh pins: %s. Both are ISTIO " "CONSTANTS, and they are values precisely so an operator can " "correct them without a chart release - but the exception assertion " "2 grants the HBONE port is pinned to the number THIS TEST knows, " "never to whatever the value says, because a value-driven exception " "would let a mistyped port ride in through a configurable field. " "That is the hole design A of #143 was rejected for. So: if Istio " "changed the constant, change it in all 30 charts AND in the pinned " "expectation here - the drift is meant to be loud." % "; ".join(problems)) else: ok("declares the pinned ambient-mesh defaults verbatim - enabled false, " "port %d, %s - so the named exception below is granted to a number " "this chart really ships, and not to whatever a value happened to say" % (PORT, ("redirectedProbeSources [%s]" % CIDR) if offers_ps else "and no redirectedProbeSources, correctly, since this chart " "renders no ipBlock at all")) off_policies = own_policies(off_path) on_policies = own_policies(on_path) if not off_policies or not on_policies: fail("renders no NetworkPolicy named %s-%s-* (off: %d, on: %d), so the " "ambient-mesh exception could not be held against this chart's own " "policy. The convention is {{ .Release.Name }}-{{ .Chart.Name }}-; " "if the name changed, this assertion has to follow - it is not skipped, " "because an exception granted to an object this test cannot identify is " "worth less than no exception at all." % (release, chart_name, len(off_policies), len(on_policies))) else: # --- The switch OFF half ------------------------------------------------ off_hits, off_cidr_hits = [], [] for name, doc in off_policies: for i, rule in enumerate(ingress_rules(doc)): for value, proto in rule_ports(rule): if value == PORT: off_hits.append("%s ingress[%d] %s/%s" % (name, i, value, proto)) for cidr in cidrs_of(rule): if cidr == CIDR: off_cidr_hits.append("%s ingress[%d]" % (name, i)) if off_hits: fail("allows ingress port %d with %s.enabled at its default (false): %s. The " "exception assertion 2 grants for %d is gated on that value, and " "this is the half that keeps it from being a hole - a port rendered " "unconditionally would otherwise be found present by this assertion " "and never held against the pod spec by assertion 2. Gate the port " "on the value." % (PORT, gate, ", ".join(off_hits), PORT)) elif off_cidr_hits: fail("carries the ipBlock %s with %s.enabled at its default (false): %s. The " "probe-source half of the switch is for the mesh only, where the " "kubelet's probes are redirected; outside it the address matches " "nothing and makes the rule read as if it did something." % (CIDR, gate, ", ".join(off_cidr_hits))) else: ok("with %s.enabled at its default the policy allows neither port %d nor the " "ipBlock %s - the negative half of assertion 2's named exception, " "without which the exception would be a hole" % (gate, PORT, CIDR)) # --- The switch ON half ------------------------------------------------- templates = [] for doc in docs_of(on_path): templates.extend(pod_templates(doc)) found_port, bad_ports, restricted_without_cidr, carriers = [], [], [], [] for name, doc in on_policies: spec = doc.get("spec") if isinstance(doc.get("spec"), dict) else {} selector = spec.get("podSelector") match_labels = (selector or {}).get("matchLabels") or {} selected = [t for t in templates if matches(match_labels, labels_of(t))] declared = set() for t in selected: for number, pname in container_ports(t): declared.add(number) if pname: declared.add(pname) for i, rule in enumerate(ingress_rules(doc)): for value, proto in rule_ports(rule): if value == PORT: found_port.append("%s ingress[%d] %d/%s" % (name, i, PORT, proto)) carriers.append((name, i, rule)) if proto != "TCP": bad_ports.append("%s ingress[%d] renders %d as %s, but " "HBONE is TCP" % (name, i, PORT, proto)) elif value not in declared and selected: bad_ports.append( "%s ingress[%d] allows %r, which is no containerPort of " "the selected pod spec" % (name, i, value)) if not found_port: fail("renders no ingress port %d with %s.enabled=true, so the value is " "decorative: the chart accepts it, the schema declares it, and the " "policy that comes out is the same one as before. A pod whose " "namespace has joined the ambient mesh is then unreachable, because " "ambient does not bypass NetworkPolicy enforcement." % (PORT, gate)) else: ok("with %s.enabled=true the policy allows %s - ztunnel's HBONE tunnel, the one " "port assertion 2 grants a named exception for" % (gate, ", ".join(found_port))) if bad_ports: fail("with %s.enabled=true the policy allows ingress port(s) that are NOT " "containerPorts of the selected pod spec and are NOT the one number " "the exception covers (%d): %s. This is the line between a named " "exception and the free port key that was rejected as design A of " "#143 - with the switch on, every OTHER wrong port still fails, " "including a NodePort number written where a pod port belongs." % (gate, PORT, "; ".join(bad_ports))) elif found_port: ok("with %s.enabled=true, port %d is still the ONLY ingress port of this chart " "that is not a containerPort of the selected pod spec - the exception " "covers exactly one number and nothing rode in beside it" % (gate, PORT)) # The inseparability of the two halves, where the render can reach it. # # Conditional on the chart OFFERING probeSources, exactly as assertion 4 and # the overlay guard are, and for the same reason: a chart with exec-only # probes (valkey, minecraft-server) puts no probe packet on the network at # all, so there is no probe to redirect and 169.254.7.127/32 would be an # entry that matches nothing - the kind of strict-looking, empty rule this # script exists to catch. Those charts render the port half only, and say so # in their values.yaml. offers_probe_sources = bool(re.search( r"^\s*probeSources\s*:", values_yaml, re.M)) for name, i, rule in carriers: if not rule.get("from"): continue if not offers_probe_sources: ok("%s ingress[%d] carries port %d in a source-restricted rule and " "no ipBlock %s - correct here, because this chart offers no " "networkPolicy.probeSources: its probes put no packet on the " "network, so there is no redirected probe to allow" % (name, i, PORT, CIDR)) continue if CIDR in cidrs_of(rule): ok("%s ingress[%d] carries port %d AND the ipBlock %s in the same " "\"from\" list - the two halves of the switch are inseparable, " "which is the reason it is one value and not two" % (name, i, PORT, CIDR)) else: restricted_without_cidr.append( "%s ingress[%d] (peers: %s)" % (name, i, ", ".join(cidrs_of(rule)) or "no ipBlock at all")) if restricted_without_cidr: fail("with %s.enabled=true the rule carrying port %d restricts its sources but " "does not include the ipBlock %s: %s. Inside the mesh the kubelet's " "probes are redirected and arrive from that address instead of from " "the worker node's, so a restricted rule without it drops every " "probe - the pod never becomes Ready and the symptom reads as a " "broken application rather than as a policy. That is assertion 4 of " "this script, re-created one value further along, and it is the " "whole reason the two halves are a single switch." % (gate, PORT, CIDR, "; ".join(restricted_without_cidr))) if not any(r.get("from") for _n, _i, r in carriers): skip("the rule carrying port %d has no \"from\" list in this render, so " "the probe-source half (%s) is not reachable here and was not " "checked. That is the state of every chart's DEFAULTS - probeSources " "is empty everywhere - and it is why ci/networkpolicy/.yaml " "exists: template-chart and gotenberg render the restricted branch " "and the half IS checked there. Reported, not counted as passing." % (PORT, CIDR)) for line in lines: print(line) PY RC=$? set -e if [ "$RC" -gt 1 ]; then sed 's/^/ /' "$WORK/ambient-$CHART.pyerr" echo "::error title=ambient-mesh $CHART::the ambient-mesh comparison itself failed for $CHART (python exit $RC). Treated as a failure, never as a pass." ROW_FAIL=1 else while IFS='|' read -r VERDICT DETAIL; do case "$VERDICT" in OK) TOTAL_ASSERTIONS=$((TOTAL_ASSERTIONS + 1)) echo " ok $DETAIL" ;; SKIP) echo " gap $DETAIL" ;; FAIL) ROW_FAIL=1 echo " FAIL $DETAIL" echo "::error title=ambient-mesh $CHART::$DETAIL" ;; *) [ -n "$VERDICT" ] && echo " $VERDICT $DETAIL" ;; esac done <"$WORK/ambient-$CHART.out" fi fi echo "::endgroup::" if [ -n "$ROW_FAIL" ]; then FAILED="$FAILED $CHART(ambient-mesh)" echo "ambient-mesh $CHART: FAILED" else AMBIENT_OK="${AMBIENT_OK:-} $CHART" echo "ambient-mesh $CHART: ok" fi done # --- Report ------------------------------------------------------------------ count() { set -- $1; echo "$#"; } listing() { set -- $1; [ "$#" -eq 0 ] && { echo "–"; return; }; printf '`%s`' "$1"; shift; for C in "$@"; do printf ', `%s`' "$C"; done; echo; } SUMMARY="${GITHUB_STEP_SUMMARY:-/dev/null}" { echo "### NetworkPolicy render test (Gitea #5)" echo echo "Every \`NetworkPolicy\` a chart renders, held against the pod specs, probes and" echo "Services of the **same** render. All of these failures are silent: the API server" echo "accepts a policy that filters nothing just as readily as one that filters" echo "everything." echo echo "| Result | Count | Charts |" echo "|---|---|---|" echo "| Policies checked | $(count "$CHECKED") | $(listing "$CHECKED") |" echo "| Ambient-mesh switch held (Gitea #143) | $(count "${AMBIENT_OK:-}") | $(listing "${AMBIENT_OK:-}") |" echo "| Failed | $(count "$FAILED") | $(listing "$FAILED") |" echo echo "$TOTAL_ASSERTIONS assertion(s) held in total." echo echo "**Not covered here, and not by anything else in CI:** that traffic still flows." echo "No ingress controller runs in the install-test cluster, so the path from Traefik" echo "to a pod is proven nowhere. What *is* proven is the probe path (install-test)," echo "acceptance by the API server, and the render-side assertions above." if [ -n "$WITH_GAPS" ]; then echo echo "Charts with code paths this test does **not** evaluate (\`matchExpressions\`-only selectors, port ranges, rules naming no port — reported, never counted as passing): $(listing "$WITH_GAPS")." fi if [ -n "$SUMMARY_ROWS" ]; then echo echo "| Chart | Result | Held | Not checked |" echo "| --- | --- | --- | --- |" printf '%s' "$SUMMARY_ROWS" fi } >>"$SUMMARY" echo "===================== networkpolicy test =======================" echo "checked: $(count "$CHECKED")${CHECKED:+ -$CHECKED}" echo "failed: $(count "$FAILED")${FAILED:+ -$FAILED}" echo "assertions held: $TOTAL_ASSERTIONS" echo "================================================================" if [ -n "$FAILED" ]; then echo "ERROR: NetworkPolicy findings in:$FAILED" >&2 exit 1 fi # The third anti-vacuum guard. Reaching zero means no chart rendered a policy # this script could say anything about, which is indistinguishable from a broken # harness - and it is the state this test would silently decay into. if [ "$TOTAL_ASSERTIONS" -eq 0 ]; then echo "ERROR: not a single assertion was evaluated in this run. Either no chart renders a NetworkPolicy any more, or the harness stopped reading the manifests. A pass over zero assertions is not a pass." >&2 exit 1 fi