test(ci): assert a chart's egress ports against the POD ports of the chart it dials

The four assertions from Gitea #5 hold ports against the pod spec the policy
SELECTS. That is right for ingress and meaningless for egress: an egress port
belongs to the destination, and for a database, a cache or the web that
destination is in no render this repository has. So the mistake behind #136 -
an egress rule naming a ClusterIP Service's port, which kube-proxy has already
translated away by the time the policy is evaluated at the pod's interface - was
not catchable within one chart.

It is catchable across two, and paperless-ngx -> apache-tika, gotenberg is the
only place in the repository where a chart dials another chart's Service (the
three other policies naming port 80 aim at the internet, where there is no DNAT
and 80 is the real port). The pairs are an explicit table rather than a
discovery, because no render can know which Service another chart dials - the
address is a value or a 1Password field - and a row naming a chart that no
longer exists fails instead of being skipped.

Each row is checked on two renders of the consumer: with the feature gate ON the
egress ports must CONTAIN every pod port behind the peers' Services, resolved
Service port -> targetPort -> containerPort, which is exactly the translation the
policy is evaluated after; with the gate OFF the requirement must not apply and
those ports must be absent, so a gated rule cannot quietly become one that
permits what the release cannot use. The evidence line prints the chain
("port 80 -> targetPort 'tika-port' -> pod port 9998"), so a reader sees the
DNAT rather than two numbers.

Built into run-networkpolicy-test.sh rather than added as a new script with its
own job: that job already runs over every chart on every PR, Helm is already the
YAML parser through ci/yaml2json, and the verdict and summary plumbing is
shared. python3 standard library only, no yq, no pyyaml, no Docker.

Six fail paths executed, not assumed: a required port removed, a wrong number in
its place, the peer chart renamed away, the policy name no longer following the
<release>-<chart>-<kind> convention, the peers rendering no Service at all (an
empty requirement reads like a satisfied one, so it fails), and the rule made
unconditional, which is the fail path of the gate-off half.

Measured: 30 charts, 210 assertions held (208 before), 0 failed, ~15s, no
cluster. README's stated figure was 206 and is corrected here.

Part of #136

ci/run-networkpolicy-test.sh

@@ -819,6 +819,397 @@ PY
   fi
 done
 
+# --- Assertion 6: the CROSS-CHART egress ports (Gitea #136) ------------------
+# The only assertion here that needs more than one chart's render, and the
+# reason it exists is an outage. paperless-ngx' egress rule for Tika and
+# Gotenberg named 80 - the port those two charts' SERVICES listen on - and
+# justified it in a template comment. kube-proxy does a ClusterIP Service's
+# destination NAT BEFORE the egress policy is evaluated at the pod's network
+# interface, so the policy sees podIP:targetPort; 80 matched nothing, and the
+# release that shipped it cut Paperless off from both services on 2026-10-15.
+# The symptom was a TIMEOUT, not "connection refused", and no error mentioned a
+# NetworkPolicy.
+#
+# WHY ASSERTION 2 CANNOT CATCH THIS, and why that is not a defect in it: it
+# holds ports against the pod spec the policy SELECTS, which is right for
+# ingress and meaningless for egress - an egress port belongs to the
+# DESTINATION. For a database, a cache or the web that destination is not in any
+# render this repository has. It is in a render only when one chart of THIS
+# repository dials the Service of another, and paperless-ngx -> apache-tika,
+# gotenberg is the single place where that happens (checked across the
+# repository for #136; every other egress port is either a 1:1 Service, so the
+# mistake is invisible, or outside the cluster, where there is no DNAT at all
+# and the port named IS the real one).
+#
+# So the table below is deliberately not a discovered list: a chart cannot know
+# which Service another chart dials, because the address lives in a value or a
+# 1Password item. Adding a row is the price of the next cross-chart egress rule,
+# and a row that names a chart which no longer exists FAILS rather than being
+# skipped - the failure path was executed, like the others.
+#
+# WHAT IS ASSERTED, per row, on TWO renders of the consumer:
+#   gate ON  - the consumer's egress ports must CONTAIN every pod port behind
+#              the peers' Services, resolved through the peers' own renders
+#              (Service port -> targetPort -> containerPort, i.e. exactly the
+#              translation the policy is evaluated after).
+#   gate OFF - the containment requirement does NOT apply (the rule is gated,
+#              and a release without the feature must not carry a rule that
+#              permits nothing), and those ports must be absent.
+#
+# Built into THIS script rather than added as a new test with its own pipeline
+# job, because everything it needs is already here: the job runs over every
+# chart on every pull request (so it cannot be skipped by a path filter), Helm
+# is already the YAML parser through ci/yaml2json, and the verdict, grouping and
+# summary plumbing is shared. A second script would duplicate all of it to add
+# one comparison.
+#
+# consumer|peer charts|gate value|extra --set needed to render the gate ON
+EGRESS_PEER_ROWS="paperless-ngx|apache-tika,gotenberg|paperless.files.tikaGotenberg.enabled|paperless.files.tikaGotenberg.tikaEndpoint=http://tika.ci.local,paperless.files.tikaGotenberg.gotenbergEndpoint=http://gotenberg.ci.local"
+
+# Renders one chart with the same values precedence the loop above uses and
+# leaves the parsed JSON in $2. The render/split/parse steps are repeated here
+# instead of refactored out of the loop on purpose: the five assertions above
+# are load-bearing and were not worth disturbing for this.
+render_chart_json() {
+  RCJ_CHART="$1"
+  RCJ_OUT="$2"
+  shift 2
+  RCJ_VALUES=()
+  RCJ_LABEL="chart defaults"
+  if [ -f "$CI_DIR/$RCJ_CHART/test-values.yaml" ]; then
+    RCJ_VALUES=(-f "$CI_DIR/$RCJ_CHART/test-values.yaml")
+    RCJ_LABEL="ci/$RCJ_CHART/test-values.yaml"
+  elif [ -f "$CI_DIR/trivy/values/$RCJ_CHART.yaml" ]; then
+    RCJ_VALUES=(-f "$CI_DIR/trivy/values/$RCJ_CHART.yaml")
+    RCJ_LABEL="ci/trivy/values/$RCJ_CHART.yaml"
+  fi
+  "$ROOT/ci/helm-dependencies.sh" "$ROOT/$RCJ_CHART" >/dev/null || return 1
+  RCJ_MANIFEST="$RCJ_OUT.manifest.yaml"
+  helm template "$RELEASE" "$ROOT/$RCJ_CHART" \
+    ${RCJ_VALUES[@]+"${RCJ_VALUES[@]}"} "$@" >"$RCJ_MANIFEST" 2>"$WORK/xerr" || return 1
+  awk '
+    BEGIN { print "docs:"; started = 0 }
+    /^---[ \t]*$/ { print "  -"; started = 1; next }
+    {
+      if (!started) { print "  -"; started = 1 }
+      if (length($0) == 0) print ""; else print "    " $0
+    }
+  ' "$RCJ_MANIFEST" >"$RCJ_MANIFEST.docs"
+  helm template y2j "$YAML2JSON" -f "$RCJ_MANIFEST.docs" 2>"$WORK/xerr" \
+    | awk '/^\{/ { print; exit }' >"$RCJ_OUT" || return 1
+  [ -s "$RCJ_OUT" ] || return 1
+  return 0
+}
+
+CROSS_ROWS=0
+for ROW in $EGRESS_PEER_ROWS; do
+  CONSUMER="$(printf '%s' "$ROW" | cut -d'|' -f1)"
+  PEERS="$(printf '%s' "$ROW" | cut -d'|' -f2 | tr ',' ' ')"
+  GATE="$(printf '%s' "$ROW" | cut -d'|' -f3)"
+  GATE_SETS="$(printf '%s' "$ROW" | cut -d'|' -f4)"
+
+  # Only when the consumer is in this run's chart list, so a single-chart
+  # invocation stays single-chart. The pipeline passes no arguments, so in CI
+  # every row always runs.
+  IN_RUN=""
+  for C in "${CHARTS[@]}"; do [ "$C" = "$CONSUMER" ] && IN_RUN=1; done
+  [ -n "$IN_RUN" ] || continue
+
+  echo "::group::cross-chart egress: $CONSUMER -> $PEERS"
+  CROSS_ROWS=$((CROSS_ROWS + 1))
+  ROW_FAIL=""
+
+  if [ ! -f "$ROOT/$CONSUMER/Chart.yaml" ]; then
+    echo "  FAIL   $CONSUMER is named in EGRESS_PEER_ROWS but is not a chart directory"
+    echo "::error title=cross-chart egress::$CONSUMER is named in EGRESS_PEER_ROWS in ci/run-networkpolicy-test.sh but is not a chart directory. If the chart was renamed or removed, update that table - a row that silently does nothing is how this check would decay back into the state that caused Gitea #136."
+    ROW_FAIL=1
+  fi
+
+  PEER_ARGS=()
+  for PEER in $PEERS; do
+    if [ ! -f "$ROOT/$PEER/Chart.yaml" ]; then
+      echo "  FAIL   peer chart $PEER does not exist"
+      echo "::error title=cross-chart egress::$CONSUMER's egress rule is checked against the chart $PEER, which is not a chart directory of this repository. Either it was renamed - in which case the row in EGRESS_PEER_ROWS in ci/run-networkpolicy-test.sh has to follow, and $CONSUMER's rule probably too - or the row is stale. Not skipped, because an unchecked row reads exactly like a passing one."
+      ROW_FAIL=1
+      continue
+    fi
+    if ! render_chart_json "$PEER" "$WORK/peer-$PEER.json"; then
+      sed 's/^/  /' "$WORK/xerr"
+      echo "  FAIL   peer chart $PEER does not render"
+      echo "::error title=cross-chart egress::the peer chart $PEER does not render with its CI values, so the pod ports behind its Services could not be read and $CONSUMER's egress rule could not be checked against them."
+      ROW_FAIL=1
+      continue
+    fi
+    PEER_ARGS+=("$PEER=$WORK/peer-$PEER.json")
+  done
+
+  ON_JSON="$WORK/consumer-$CONSUMER-on.json"
+  OFF_JSON="$WORK/consumer-$CONSUMER-off.json"
+  if [ -z "$ROW_FAIL" ]; then
+    SET_ARGS=(--set "$GATE=true")
+    OLD_IFS="$IFS"; IFS=','
+    for S in $GATE_SETS; do [ -n "$S" ] && SET_ARGS+=(--set "$S"); done
+    IFS="$OLD_IFS"
+    if ! render_chart_json "$CONSUMER" "$ON_JSON" "${SET_ARGS[@]}"; then
+      sed 's/^/  /' "$WORK/xerr"
+      echo "  FAIL   $CONSUMER does not render with $GATE=true"
+      echo "::error title=cross-chart egress::$CONSUMER does not render with $GATE=true and the --set values from EGRESS_PEER_ROWS, so its egress rule for $PEERS was never produced and nothing was checked. If the feature needs further values, add them to that row."
+      ROW_FAIL=1
+    elif ! render_chart_json "$CONSUMER" "$OFF_JSON" --set "$GATE=false"; then
+      sed 's/^/  /' "$WORK/xerr"
+      echo "  FAIL   $CONSUMER does not render with $GATE=false"
+      echo "::error title=cross-chart egress::$CONSUMER does not render with $GATE=false, so the gated-off case - in which this assertion must NOT fire - could not be checked."
+      ROW_FAIL=1
+    fi
+  fi
+
+  if [ -z "$ROW_FAIL" ]; then
+    set +e
+    python3 - "$CONSUMER" "$RELEASE" "$GATE" "$ON_JSON" "$OFF_JSON" "${PEER_ARGS[@]}" \
+      >"$WORK/cross-$CONSUMER.out" 2>"$WORK/cross-$CONSUMER.pyerr" <<'PY'
+import json, sys
+
+consumer, release, gate, on_path, off_path = sys.argv[1:6]
+peers = [a.split("=", 1) for a in sys.argv[6:]]
+
+lines = []
+def ok(msg):   lines.append("OK|%s: %s" % (consumer, msg))
+def skip(msg): lines.append("SKIP|%s: %s" % (consumer, msg))
+def fail(msg): lines.append("FAIL|%s: %s" % (consumer, msg))
+
+
+def docs_of(path):
+    return [d for d in (json.load(open(path)).get("docs") or []) if isinstance(d, dict)]
+
+
+def pod_templates(node):
+    out = []
+    if isinstance(node, dict):
+        for key, value in node.items():
+            if (key == "template" and isinstance(value, dict)
+                    and isinstance(value.get("spec"), dict)
+                    and "containers" in value["spec"]):
+                out.append(value)
+            out.extend(pod_templates(value))
+    elif isinstance(node, list):
+        for value in node:
+            out.extend(pod_templates(value))
+    return out
+
+
+def labels_of(template):
+    meta = template.get("metadata")
+    labels = meta.get("labels") if isinstance(meta, dict) else None
+    return labels if isinstance(labels, dict) else {}
+
+
+def matches(selector, labels):
+    for key, value in selector.items():
+        if key not in labels or str(labels[key]) != str(value):
+            return False
+    return True
+
+
+def container_ports(template):
+    out = []
+    spec = template.get("spec") or {}
+    for key in ("containers", "initContainers"):
+        for container in spec.get(key) or []:
+            if not isinstance(container, dict):
+                continue
+            for port in container.get("ports") or []:
+                if isinstance(port, dict) and port.get("containerPort") is not None:
+                    out.append((int(port["containerPort"]), port.get("name")))
+    return out
+
+
+def service_pod_ports(path, chart):
+    """The pod ports a client reaches THROUGH this chart's Services, as
+    {pod port: [evidence]}. Resolved Service port -> targetPort -> containerPort,
+    because that chain IS the destination NAT the egress policy is evaluated
+    after: the packet leaving the consumer carries the Service port, and the
+    packet the policy sees carries the pod port."""
+    docs = docs_of(path)
+    templates = []
+    for doc in docs:
+        templates.extend(pod_templates(doc))
+    out, unresolved = {}, []
+    for svc in docs:
+        if svc.get("kind") != "Service":
+            continue
+        spec = svc.get("spec") or {}
+        selector = spec.get("selector")
+        hits = [t for t in templates
+                if isinstance(selector, dict) and matches(selector, labels_of(t))]
+        by_name = dict((n, num) for t in hits for num, n in container_ports(t) if n)
+        svc_name = ((svc.get("metadata") or {}).get("name")) or "<unnamed Service>"
+        for port in spec.get("ports") or []:
+            if not isinstance(port, dict):
+                continue
+            target = port.get("targetPort")
+            if target is None:
+                target = port.get("port")
+            number = None
+            if isinstance(target, int):
+                number = target
+            elif isinstance(target, str):
+                number = int(target) if target.isdigit() else by_name.get(target)
+            if number is None:
+                unresolved.append("%s %s: targetPort %r" % (chart, svc_name, target))
+                continue
+            out.setdefault(number, []).append(
+                "%s %s: port %s -> targetPort %r -> pod port %d"
+                % (chart, svc_name, port.get("port"), target, number))
+    return out, unresolved
+
+
+def egress_ports(path):
+    """(numbers, policy names, has a rule with no ports at all) for the
+    consumer's OWN policy only - recognised by the <release>-<chart>- name
+    prefix, so a subchart's policy in the same render cannot satisfy a rule the
+    parent chart is responsible for."""
+    prefix = "%s-%s-" % (release, consumer)
+    numbers, names, open_rule = set(), [], False
+    for doc in docs_of(path):
+        if doc.get("kind") != "NetworkPolicy":
+            continue
+        name = ((doc.get("metadata") or {}).get("name")) or ""
+        if not name.startswith(prefix):
+            continue
+        names.append(name)
+        spec = doc.get("spec") if isinstance(doc.get("spec"), dict) else {}
+        for rule in spec.get("egress") or []:
+            if not isinstance(rule, dict):
+                continue
+            if not rule.get("ports"):
+                open_rule = True
+                continue
+            for port in rule["ports"]:
+                if not isinstance(port, dict):
+                    continue
+                value = port.get("port")
+                if isinstance(value, int):
+                    numbers.add(value)
+                elif isinstance(value, str) and value.isdigit():
+                    numbers.add(int(value))
+    return numbers, names, open_rule
+
+
+required, unresolved = {}, []
+for chart, path in peers:
+    ports, bad = service_pod_ports(path, chart)
+    for number, evidence in ports.items():
+        required.setdefault(number, []).extend(evidence)
+    unresolved.extend(bad)
+
+for note in unresolved:
+    skip("a Service targetPort of a peer chart could not be resolved to a "
+         "containerPort, so that port is not part of the requirement (%s). "
+         "Reported, not counted as passing." % note)
+
+on_numbers, on_names, on_open = egress_ports(on_path)
+off_numbers, off_names, _off_open = egress_ports(off_path)
+
+if not required:
+    fail("the peer charts render no Service whose port resolves to a pod port, "
+         "so this assertion would have required nothing at all. That is "
+         "indistinguishable from a passing check and is treated as a failure: "
+         "either the peer charts stopped shipping a Service, or their targetPort "
+         "no longer resolves.")
+elif not on_names:
+    fail("with %s=true the render contains no NetworkPolicy named %s-%s-*, so "
+         "the egress rule this assertion is about was not found. The policy's "
+         "name is the convention {{ .Release.Name }}-{{ .Chart.Name }}-<kind>; if "
+         "it changed, this check has to follow."
+         % (gate, release, consumer))
+elif on_open:
+    skip("an egress rule of %s names NO port, which allows every port - so "
+         "containment of the peers' pod ports is satisfied without saying "
+         "anything. Reported, never counted as passing."
+         % ", ".join(on_names))
+else:
+    missing = sorted(n for n in required if n not in on_numbers)
+    if missing:
+        fail("with %s=true, %s permits outbound %s but NOT %s - and those are the "
+             "POD ports behind the peer charts' Services: %s. An egress rule is "
+             "evaluated at the pod's network interface, AFTER kube-proxy has "
+             "translated the ClusterIP Service's destination (DNAT), so the policy "
+             "sees podIP:targetPort and a rule naming the SERVICE port matches "
+             "nothing wherever the two numbers differ. This is Gitea #136 "
+             "exactly: it cost an outage on 2026-10-15, and the symptom was a "
+             "TIMEOUT rather than \"connection refused\", with no error "
+             "mentioning a NetworkPolicy. Name the pod port(s) in the rule."
+             % (gate, ", ".join(on_names),
+                ", ".join(str(n) for n in sorted(on_numbers)),
+                ", ".join(str(n) for n in missing),
+                "; ".join(e for n in missing for e in required[n])))
+    else:
+        ok("with %s=true, %s permits every pod port behind the peer charts' "
+           "Services: %s"
+           % (gate, ", ".join(on_names),
+              "; ".join(e for n in sorted(required) for e in required[n])))
+
+    # The gated-off half. The requirement above must NOT apply here - a release
+    # without the feature is not expected to reach the peers - and the ports must
+    # be gone, or the rule is no longer gated and permits something the release
+    # has no use for.
+    present = sorted(n for n in required if n in off_numbers)
+    if not off_names:
+        skip("with %s=false the render contains no NetworkPolicy of this chart, "
+             "so the gated-off case could not be read." % gate)
+    elif present:
+        fail("with %s=false, %s still permits outbound %s, the peer charts' pod "
+             "port(s). Either the rule is no longer gated on that value - a rule "
+             "that permits what the release cannot use - or the CI values add "
+             "those ports by hand, which would make the gate-on assertion above "
+             "pass without the rule existing."
+             % (gate, ", ".join(off_names),
+                ", ".join(str(n) for n in present)))
+    else:
+        ok("with %s=false the rule is absent (egress: %s), so the containment "
+           "requirement correctly does not apply and no release without the "
+           "feature carries a rule that permits nothing."
+           % (gate, ", ".join(str(n) for n in sorted(off_numbers)) or "none"))
+
+for line in lines:
+    print(line)
+PY
+    RC=$?
+    set -e
+    if [ "$RC" -gt 1 ]; then
+      sed 's/^/  /' "$WORK/cross-$CONSUMER.pyerr"
+      echo "::error title=cross-chart egress::the cross-chart egress comparison itself failed for $CONSUMER (python exit $RC). Treated as a failure, never as a pass."
+      ROW_FAIL=1
+    else
+      while IFS='|' read -r VERDICT DETAIL; do
+        case "$VERDICT" in
+          OK)
+            TOTAL_ASSERTIONS=$((TOTAL_ASSERTIONS + 1))
+            echo "  ok     $DETAIL"
+            ;;
+          SKIP) echo "  gap    $DETAIL" ;;
+          FAIL)
+            ROW_FAIL=1
+            echo "  FAIL   $DETAIL"
+            echo "::error title=cross-chart egress::$DETAIL"
+            ;;
+          *) [ -n "$VERDICT" ] && echo "  $VERDICT $DETAIL" ;;
+        esac
+      done <"$WORK/cross-$CONSUMER.out"
+    fi
+  fi
+
+  echo "::endgroup::"
+  if [ -n "$ROW_FAIL" ]; then
+    FAILED="$FAILED $CONSUMER(cross-chart)"
+    SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CONSUMER\` → \`$(printf '%s' "$PEERS" | tr ' ' ',')\` | **failed** | – | – |"$'\n'
+    echo "cross-chart egress $CONSUMER: FAILED"
+  else
+    SUMMARY_ROWS="${SUMMARY_ROWS}| \`$CONSUMER\` → \`$(printf '%s' "$PEERS" | tr ' ' ',')\` | ok | 2 | 0 |"$'\n'
+    echo "cross-chart egress $CONSUMER: ok"
+  fi
+done
+
 # --- Report ------------------------------------------------------------------
 count() { set -- $1; echo "$#"; }
 listing() { set -- $1; [ "$#" -eq 0 ] && { echo "–"; return; }; printf '`%s`' "$1"; shift; for C in "$@"; do printf ', `%s`' "$C"; done; echo; }