fix(paperless-ngx): egress names the Tika/Gotenberg POD ports, not the Service port

The rule allowed outbound 80, the port the apache-tika and gotenberg Services
listen on, and justified it in a comment: an egress rule names the port on the
destination as the client dials it, and the client dials the Service. That is
wrong. kube-proxy does a ClusterIP Service's destination NAT BEFORE the egress
policy is evaluated at the pod's network interface, so the policy sees
podIP:targetPort and never clusterIP:port - and those two charts forward 80 to
9998 and 3000. Deployed as 14.0.0+up3.3.1 it cut Paperless off from both
services on 2026-10-15: mail import, EML, HTML and Office documents stopped,
plain PDFs and images kept working, and the error was httpx.ConnectTimeout with
no mention of a NetworkPolicy.

The rule now names 9998, 3000 and 80. The 80 stays because it costs nothing and
covers a foreign Tika that really does listen on 80. The refuted sentence is
quoted rather than deleted: it reads perfectly, which is how it survived review,
and the timeout-not-refused symptom is written down with it.

No policy switch of its own: the rule stays gated on
paperless.files.tikaGotenberg.enabled, because a second value that can drift
from the feature re-arms this very outage.

New networkPolicy.egress.mailImport opens IMAPS 993, off by default - paperless'
mail accounts are rows in its database, so the chart cannot derive the rule.
Plain IMAP (143) and POP3 stay in extraPorts, and 993 is noted as a real
destination port, because a mail server outside the cluster has no DNAT in the
path.

Minor: one additive value, and a fix that widens a rule. Nothing breaks, no
value becomes required, appVersion unchanged.

Part of #136
This commit is contained in:
jklein committed 2026-10-15 17:39:24 +00:00
1 parent 1ef23e9ce9
commit 530227405c
5 files changed
+168 -43

No files matched your search

@@ -27,11 +27,12 @@ theoretical:
opens no connection that a policy would have anything to say about. If that
ever changes, the CronJob needs labels before it can need a policy.
3. MAIL FETCHING IS NOT PERMITTED. Paperless' mail accounts live in its
DATABASE, configured through the web UI, so this chart cannot see them. A
release that fetches mail needs IMAP (993 or 143) in
networkPolicy.egress.extraPorts, and the symptom otherwise is a mail rule
that silently stops importing.
3. MAIL FETCHING IS NOT PERMITTED UNLESS THE OPERATOR SAYS SO. Paperless' mail
accounts live in its DATABASE, configured through the web UI, so this chart
cannot see them and cannot derive the rule. networkPolicy.egress.mailImport
opens IMAPS (993) and is off by default; plain IMAP (143) and POP3 belong in
networkPolicy.egress.extraPorts. Left off on a release that does fetch mail,
the symptom is a mail rule that silently stops importing.
WHAT THIS OBJECT IS: a ceiling on reachability, not an authentication layer.
Only the ports declared below are reachable at all - whatever else the process
@@ -53,13 +54,22 @@ WHAT IT IS NOT:
of the rendered object, travel into the cluster and appear in every release
diff. ci/run-networkpolicy-test.sh fails on a rendered "#".
THE PORT NUMBERS BELOW ARE POD PORTS, NEVER NODEPORTS. A NetworkPolicy is
evaluated at the pod's network interface, after kube-proxy has already
translated a NodePort to the targetPort, so a NodePort number here would match
nothing while looking strict - and, being the only rule for that port, would
block every inbound connection instead. ci/run-networkpolicy-test.sh asserts
every port here against the containerPorts of the pod spec this policy
selects.
THE PORT NUMBERS BELOW ARE POD PORTS, NEVER NODEPORTS AND NEVER SERVICE PORTS.
A NetworkPolicy is evaluated at the pod's network interface, after kube-proxy
has already done its destination NAT - of a NodePort on the way in, and of a
ClusterIP on the way OUT. So a NodePort number in an ingress rule matches
nothing while looking strict (and, being the only rule for that port, blocks
every inbound connection instead), and a SERVICE port in an egress rule matches
nothing wherever the Service's port and its targetPort differ. The second half
is not hypothetical: it is Gitea #136, it cost an outage on 2026-10-15, and the
sentence that caused it is quoted as refuted at the Tika/Gotenberg rule below.
ci/run-networkpolicy-test.sh asserts every INGRESS port here against the
containerPorts of the pod spec this policy selects. Egress cannot be checked
that way - an egress port belongs to the DESTINATION, which for Postgres, Redis
and the web is not even in the render. The one exception is Tika and Gotenberg,
which are two charts of THIS repository: the same test renders them and asserts
that the egress rule below contains their containerPorts.
*/ -}}
{{- if .Values.networkPolicy.enabled }}
{{- $np := .Values.networkPolicy }}
@@ -88,9 +98,10 @@ spec:
- Ingress
{{- /*
Egress is listed because there is a lot to restrict and most of it is known.
What is NOT permitted and is the plausible surprise: IMAP. Paperless' mail
accounts are rows in its database, configured in the web UI, so the chart
cannot see them - see networkPolicy.egress.extraPorts.
What is NOT permitted by default and is the plausible surprise: mail. Paperless'
mail accounts are rows in its database, configured in the web UI, so the chart
cannot see them - networkPolicy.egress.mailImport opens IMAPS 993, and anything
else (143, POP3) goes into networkPolicy.egress.extraPorts.
*/}}
- Egress
ingress:
@@ -184,17 +195,51 @@ spec:
port: 6379
{{- if .Values.paperless.files.tikaGotenberg.enabled }}
{{- /*
TIKA AND GOTENBERG, on port 80 - which is what their Services listen on in this
repository, even though their containers use 9998 and 3000. That distinction is
the whole reason this rule says 80: an EGRESS rule names the port on the
DESTINATION as the client dials it, and the client dials the Service.
TIKA AND GOTENBERG, ON THEIR POD PORTS 9998 AND 3000 - and 80 as well, for a
third case named at the end. Gitea #136: this rule used to name 80 ALONE, and
the release that shipped it cut Paperless off from both services on
2026-10-15. Mail import, EML, HTML and Office documents stopped; plain PDFs
and images kept working, which is why it did not look like a network fault.
Rendered only while paperless.files.tikaGotenberg.enabled is true, so a release
without them carries no rule that permits nothing. Endpoints on another port -
the containers' own 9998 and 3000, if addressed directly rather than through
their Services - need networkPolicy.egress.extraPorts.
THE SENTENCE THAT STOOD HERE IS WRONG. It is quoted rather than deleted,
because it reads perfectly and that is precisely how it survived review:
"an EGRESS rule names the port on the DESTINATION as the client dials
it, and the client dials the Service."
kube-proxy translates the destination of a ClusterIP Service (DNAT) BEFORE
the egress policy is evaluated at the pod's network interface. The policy
therefore sees podIP:targetPort and never clusterIP:port, so an egress rule
names the POD port. A Service port only ever matches by coincidence, where
port and targetPort are the same number - which is the case for 5432, 6379
and 443 above and below, and is the reason nothing else here broke.
THE SYMPTOM IS A TIMEOUT, NOT "connection refused", and that is the half to
remember: a missing service REFUSES, a dropped packet RUNS INTO THE TIMEOUT.
What this produced was httpx.ConnectTimeout out of paperless' parser, naming
the Tika URL and no NetworkPolicy at all - nothing in the error points here.
WHY 80 STAYS: it costs nothing and covers an installation whose Tika really
does listen on 80 - a foreign one behind a proxy, where 80 IS the pod port.
WHY THERE IS NO SWITCH OF ITS OWN for this rule, and that is deliberate: it
is gated on the FEATURE value, because a second value that can drift from the
feature re-arms exactly this outage. Whoever enables Tika and forgets the
policy switch lands back here, with the same message that does not point at
the policy. Rendered only while paperless.files.tikaGotenberg.enabled is
true, so a release without them still carries no rule that permits nothing.
9998 and 3000 are the containerPorts the apache-tika and gotenberg charts of
this repository declare, so they are fixed rather than guessed - and
ci/run-networkpolicy-test.sh renders both of those charts and asserts that
this rule contains them. That is the one assertion a single-chart render
cannot make, and it is why this mistake was not caught the first time.
*/}}
- ports:
- protocol: TCP
port: 9998
- protocol: TCP
port: 3000
- protocol: TCP
port: 80
{{- end }}
@@ -207,6 +252,30 @@ spec:
- protocol: TCP
port: 443
{{- with $np.egress }}
{{- if .mailImport }}
{{- /*
IMAPS for mail import, OFF BY DEFAULT - because this chart cannot know
whether a release fetches mail. Paperless' mail accounts are rows in its
DATABASE, entered in the web UI, and nothing in the values hints that they
exist. So this value is an operator's statement ("this release imports
mail") and not a derivation, which is also why it cannot be folded into
another switch.
993 ONLY. Plain IMAP (143) and POP3 (110, 995) are deliberately NOT included:
a value named mailImport that quietly opened four ports would be wider than
its name, and the plaintext variants are not what a new account uses. They go
into networkPolicy.egress.extraPorts.
AND THIS PORT IS THE REAL DESTINATION PORT, unlike the Tika rule above. A
mail server outside the cluster is reached with no Service and therefore no
DNAT in the path, so 993 is both what paperless dials and what the policy
sees. What decides which number an egress rule needs is the DNAT, not the
direction and not the protocol - that is the whole lesson of #136.
*/}}
- ports:
- protocol: TCP
port: 993
{{- end }}
{{- with .extraPorts }}
{{- /*
Ports only, to any destination - see the header. This is the operator's