diff --git a/stalwart.md b/stalwart.md index 773a813..bdf08f2 100644 --- a/stalwart.md +++ b/stalwart.md @@ -14,7 +14,7 @@ So **this chart configures almost nothing, and that is its defining property, no | Versions | not listed on purpose — read the current pair off `stalwart/Chart.yaml`, see [Chart index](Charts) | | Workloads | one StatefulSet **per node group**, plus one config-seeding init container each | | Storage | **no PVC.** Both image volumes are emptyDirs — state lives in Postgres, S3 and Valkey | -| Ingress | yes, for the WebUI and JMAP only, pointed at a single node group | +| Routing | **two mutually exclusive modes** for the WebUI and JMAP only, Ingress (default) or Gateway API (`httpRoute.enabled`) — **one** object in either mode, pointed at a **single** node group (`ingress.nodeGroup`), never per group. SMTP, IMAP and ManageSieve pass through neither. In Gateway mode the **CORS configuration the webmail depends on is gone** — see [Routing](#routing-ingress-or-gateway-api) | | External dependencies | **Postgres** (required), **S3** (blob store), **Valkey** (in-memory store + cluster coordinator) | | Secrets | up to 3 × `OnePasswordItem` (data store, recovery admin, S3) | | Network | one NodePort Service **per public port** + per-group ClusterIP and headless Services + **NetworkPolicy on by default** | @@ -295,12 +295,22 @@ Setting a host here does **not** point Stalwart at it. The blob store and in-mem | Value | Default | Meaning | |---|---|---| -| `ingress.enabled` | `true` | Gates the Ingress only; **`ingress.domain` stays required either way** (the mail Certificate reads it) | -| `ingress.clusterIssuer` | `null` | **Required**, no default | -| `ingress.domain` | `null` | **Required**, no default; read twice | -| `ingress.className` | `nginx` | | -| `ingress.nodeGroup` | `full` | Which group the Ingress routes to, via that group's own `http` Service | -| `ingress.annotations` | `{}` | Merged with the cert-manager annotation | +| `ingress.enabled` | `true` | Gates the Ingress only; **`ingress.domain` stays required either way** (the mail Certificate reads it). **Mutually exclusive with `httpRoute.enabled`** | +| `ingress.clusterIssuer` | `null` | **Required**, no default — unless `certificate.clusterIssuer` is set, which it falls back to. Gateway mode does **not** free it for that reason: the mail Certificate still needs an issuer | +| `ingress.domain` | `null` | **Required**, no default; read **three** times since the Gateway support — the Ingress host, the mail Certificate's `dnsNames`, and the `HTTPRoute`'s hostname while `httpRoute.hostnames` is empty | +| `ingress.className` | `nginx` | No counterpart in Gateway mode — the class sits on the Gateway | +| `ingress.nodeGroup` | `full` | Which group the **routing object** points at, via that group's own `http` Service. **Shared by both modes on purpose**: it names the node group the management surface is published from, which is not a property of the Ingress. Must be a key of `nodes:` and must expose `http`; both are checked in both templates | +| `ingress.annotations` | `{}` | Merged with the cert-manager annotation. **No counterpart in Gateway mode**, and for this chart that is the consequential one: this installation's **CORS** configuration lives here. See [Routing](#routing-ingress-or-gateway-api) | + +### `httpRoute` + +The Gateway-API alternative to the Ingress, **off by default**. Added with group 3 of the Gateway-API sweep (Gitea #114); the form is the pilot's and is binding for every chart ([style guide §4](Chart-Style-Guide#4-ingress-and-gateway-api)). + +| Value | Default | Meaning | +|---|---|---| +| `httpRoute.enabled` | `false` | Renders `-stalwart-httproute` instead of the Ingress. **Mutually exclusive with `ingress.enabled`** — with both `true` the render aborts naming both values. **One** route, not one per node group | +| `httpRoute.parentRefs` | `[]` | The Gateway(s) the route attaches to — Gateway API `ParentReference` entries (`name`, and optionally `namespace`, `sectionName`, `port`, `group`, `kind`), passed through unchanged. **Required while `httpRoute.enabled` is true**: the render aborts on an empty list. The schema requires `name` on every entry and is otherwise left open, because Kubernetes owns that schema and adds fields to it across releases | +| `httpRoute.hostnames` | `[]` | Empty means `[ingress.domain]`, so the HTTP hostname has a single source of truth. Note that **the server's own hostname lives in the database**, not in any chart value: after bootstrap the WebUI is reached at `https:///admin`, so a second hostname can be reachable without the application agreeing that it is its name | ### `securityContext` and probes @@ -446,12 +456,66 @@ An earlier draft had the application bind 30025/30465/30993 so that nothing priv **And in the NetworkPolicy the pod ports apply, not the node ports** — 25/465/993, never 30025/30465/30993. A policy is evaluated at the pod's interface, after kube-proxy has already translated the node port to the target port. Writing 30025 there matches nothing: the packet arrives on 25. The policy then looks strict, is accepted by the API server, and silently blocks everything inbound. +### Routing: Ingress or Gateway API + +Two mutually exclusive modes for the HTTP surface — WebUI and JMAP — and exactly one of them may be on. `ingress.enabled` ships `true`, `httpRoute.enabled` ships `false`. + +**One route, not one per node group.** #114 describes this chart as rendering "per node group"; its **Services and StatefulSets** are per group, the routing object is not and must not be. The route targets exactly `ingress.nodeGroup`'s own `http` Service and keeps both of the Ingress's guards — the group must be a key of `nodes:` and must expose `http` — because the WebUI's setup and management screens are not safe to spread over a group that may not run the management tasks, and a port-label selector would send each request to a random group. + +| Rule | Why | +|---|---| +| `ingress.enabled` and `httpRoute.enabled` cannot both be true — the render aborts naming **both** value paths | two data paths onto one Service under one hostname is a conflict at DNS level already. For Stalwart there is a second edge: the auto-ban sees the **proxy's** address, so two data paths mean two addresses to get banned and a WebUI that fails for half the requests. **A migration runs over a second hostname** — and note that the server hostname configured *in the database* decides where the WebUI sends administrators, which no chart value can change | +| `httpRoute.parentRefs` must not be empty — the render aborts | a route without a parent attaches to **no** Gateway. The object is created, it is schema-valid, no controller rejects it, and it routes nothing: no event, no condition, and the only symptom is a hostname that does not answer | +| `ingress.nodeGroup` is shared by both modes | it names where the management surface is published from, which is not a property of the Ingress | + +**SMTP, IMAP and ManageSieve are not routed in either mode**, exactly as before: they keep arriving through the per-port NodePort Services behind the HA proxy. A Gateway could in principle carry them as `TLSRoute`/`TCPRoute` listeners; this chart renders neither, and doing so would be a separate decision. + +### The mail certificate stays in Gateway mode, and must not be cleaned up + +**This is the only chart in the whole sweep where anything of the TLS path stays with the chart.** `certificate.enabled` keeps rendering `-stalwart-mail-certificate` in Gateway mode, because that object has nothing to do with the Ingress or the Gateway: it covers **STARTTLS on 25 and implicit TLS on 465/993**, and those connections pass through no HTTP proxy at all ([TLS](#tls-cert-manager-not-stalwarts-own-acme)). + +So in Gateway mode: + +- `ingress.domain` is **still required** — the certificate's `dnsNames` read it; +- `ingress.clusterIssuer` is **still required** unless `certificate.clusterIssuer` is set, which it falls back to; +- and the Certificate **must not be removed** as part of a migration. Deleting it drops SMTP and IMAP back to whatever self-signed certificate Stalwart generates for itself: clients see a name mismatch while **every Kubernetes object in the release stays green**. + +What *does* move to the Gateway is the certificate for the **HTTP hostname** only, onto its listener. + +### Gateway mode drops the CORS configuration, and that breaks the webmail + +Measured on the live Ingress on 2026-10-12 — **operator-supplied** through `ingress.annotations`; this chart sets none of it: + +``` +nginx.ingress.kubernetes.io/enable-cors: true +nginx.ingress.kubernetes.io/cors-allow-origin: https:// +nginx.ingress.kubernetes.io/cors-allow-methods: GET, POST, OPTIONS +nginx.ingress.kubernetes.io/cors-allow-headers: Authorization, Content-Type +``` + +That origin is **[bulwark](bulwark)**, the webmail chart — so this is the one loss in the sweep that breaks a *second* chart. The CORS configuration is what allows the browser-side webmail to call this server's JMAP endpoint at all. Without it the webmail stops working, with a console error in the browser and **nothing whatsoever in this server's logs**. + +**And "no counterpart" would be the wrong thing to write here.** Measured by grepping the CRDs this repository pins (`ci/gateway-api-crds.sh`, gateway-api v1.6.3 standard channel) rather than inferred: + +- HTTPRoute's filters carry a full **`cors`** field (GEP-1767) with `allowOrigins`, `allowMethods`, `allowHeaders`, `allowCredentials` and `maxAge`. Its documented level, verbatim from that CRD: **`Support: Extended`**. +- `responseHeaderModifier` is in the same CRD and is **also `Support: Extended`**, not Core. + +There *is* a counterpart, then, and **this chart still does not render it**. Three reasons, in order: + +1. **The chart has no CORS values.** The live configuration is the operator's — a specific origin, a specific method list — and the chart sets no CORS annotation in Ingress mode either. Rendering a filter means inventing a values surface that would exist in Gateway mode only, and that asymmetry is its own defect and its own piece of work. +2. **Extended support is not support.** An implementation that does not support an Extended value reports it in the route's status, and a rejected route is rejected **whole** — which here would take the WebUI and JMAP down together (the reasoning [#119 reached for `statusCode`](freesailarr)). The observed failure is quieter still: Cilium accepts `filters.type: CORS` and forwards the browser's preflight upstream anyway (`cilium/cilium#45272`), i.e. `Accepted: True` with the behaviour absent. A chart cannot know which implementation it landed on. +3. **Half a CORS is worse than none, and the translatable half is the useless one here.** A `responseHeaderModifier` can add the three `Access-Control-Allow-*` headers to a response. The **preflight** is a different thing: the browser sends `OPTIONS` first and the **gateway**, not the backend, has to answer it — the CORS filter's own contract says a path carrying it must not pass the preflight upstream, which a bare header modifier cannot do. And this installation's header list includes **`Authorization`**, which makes **every** webmail request preflighted: the half that is expressible without the CORS filter is precisely the half that does not help. + +**So: configure CORS at the Gateway, with that implementation's own support, before you move this host. Then test it from the webmail in a browser — `curl` sends no preflight, so it will pass while the webmail fails.** + ### The Ingress ban trap The one the HA proxy does **not** solve. Stalwart sees the ingress controller's **pod IP** for every JMAP and WebUI request, not the client's, and the auto-ban eventually bans that address — at which point WebUI and JMAP are dead for everyone at once. For HTTP the remedy is not the PROXY protocol but Stalwart's `forwarded` handling with `X-Forwarded-For` evaluation, **while discarding forwarding headers the client brought itself**. A mandatory step (step 4), because the outage arrives weeks later and looks like a server fault. +**Gateway mode does not avoid this and does not reset it**: it is one proxy address instead of another — the Gateway's data-plane pods rather than the ingress controller's — and whether the Gateway sends an `X-Forwarded-For` at all is that implementation's business, not something the route can state. The same applies to `networkPolicy.allowedNamespaces`: in Gateway mode the namespace to allow is the **Gateway's data plane**, which usually lives in a namespace of its own, not an ingress controller's. + Second half of the same area: **after bootstrap, port 8080 is no longer the admin port.** Upstream moves administration to `https:///admin` once setup completes. The Ingress therefore only works while an HTTP `NetworkListener` on 8080 exists in the database — between finishing setup and creating that listener it resolves to a port that answers nothing. ### A missing `config.json` opens an ephemeral store and looks healthy @@ -586,6 +650,15 @@ What it does prove is the chart's own responsibility, and the first item is why **No Valkey in the fixtures, deliberately.** The in-memory store is a database object, so a Valkey in the namespace would be reachable and unused — making the test look as though it covered clustering while covering none of it. The test runs a single node, which needs no in-memory store. The subchart path is covered by a local render only; a known gap. +**The Gateway mode is held at render level**, by `ci/httproute/stalwart.yaml` plus `ci/run-httproute-match-test.sh`, which runs over every chart on every PR ([CI/CD](CI-CD#httproute-render-test)). The overlay switches this chart into Gateway mode; **2 assertions** — `parentRefs` not empty, and the `backendRefs` entry resolving to **that node group's** `http` Service in the same render. The port is held **by number** (`ports.http.containerPort`), and for this chart that is the check worth having: the NodePort Services publish a *different* number (`ports..nodePort`) than the container binds, so naming the wrong one in a route would be a plausible mistake. The one-route-not-one-per-group property is the other thing the overlay pins down, since it is exactly what a copy would get wrong. + +**What that does not cover**, named so the gaps are not mistaken for coverage: + +- **that the hostname answers.** No Gateway controller runs in CI and no route reaches `Accepted: True`. +- **the CORS configuration** — the thing this chart actually loses in that mode, and the one that breaks the [bulwark](bulwark) webmail. It lives in the operator's `ingress.annotations`, the chart has no values for it, and the Gateway API's own CORS filter is Extended-support with observed implementation variance. All of that is documented in `values.yaml` and printed by `NOTES.txt`, and documentation is not something a test can assert. +- **that SMTP/IMAP still work.** They never went through the Ingress and do not go through the route; the mail Certificate that serves them keeps being rendered in Gateway mode, which is this chart's distinguishing feature in the sweep, and no render test looks at whether it was left alone. +- **`NOTES.txt` itself.** `helm template` does not emit it and no script under `ci/` reads it, so its text carries review-only assurance. + ## See also - [Chart style guide](Chart-Style-Guide) — the conventions this chart follows and the two it deviates from