jlab-cluster/helm-charts
3
0 Fork 0
Code Issues 15 Pull Requests 3 Actions Packages Wiki Activity

matrix-synapse: the Gateway API as an alternative to the Ingress (#114) 

Two HTTPRoute objects, and the shape is forced rather than chosen: the Ingress
is ONE object with TWO host rules (homeserver at /, delegation nginx at
/.well-known/matrix), and an HTTPRoute's spec.hostnames applies to the whole
object, so two hostnames wanting different paths and different backends have
to be two objects. Noted on the page that #114 describes this chart as
rendering its Ingresses in a loop over several apps - that is freesailarr.

httpRoute.delegation.hostnames deliberately does not fall back to
httpRoute.hostnames: the server name is what remote homeservers resolve, and
the migration order depends on telling it apart from the homeserver's own
host.

One section is new in kind: Gateway mode is a BETTER FIT for Matrix
delegation. The server name is normally an apex domain already serving a
website, and ingress-nginx refuses a second Ingress on one hostname - which is
why the delegation rule has to sit inside this chart's single Ingress object.
HTTPRoutes on one listener and hostname are merged by the implementation,
longest match winning, so that route can sit beside whatever owns the apex.

And the loss section says which half matters. proxy-body-size 50m (plus the
nginx.org twin) is the real one, against a max_upload_size this chart cannot
see because the config comes from 1Password. The configuration-snippet is NOT:
more_set_headers is the headers-more module's OUTPUT-header directive, so
those headers go on the response and Synapse never sees them. The page says in
as many words not to re-create it at the Gateway and not to read its absence
as the cause of a rate-limiting problem - flagged as a reading of documented
semantics rather than an experiment.
jklein committed 2026-10-12 21:32:08 +00:00
1 parent 2adfe91a52
commit 5a4c4c274c
1 file changed
+75 -8
Download Patch File Download Diff File Expand all files Collapse all files
matrix-synapse.md
+75 -8
@@ -8,7 +8,7 @@ Deploys [Matrix Synapse](https://element-hq.github.io/synapse/) as a single-repl
| Versions | not listed on purpose — read the current pair off `matrix-synapse/Chart.yaml`, see [Chart index](Charts) |
| Workloads | StatefulSet (1 replica); optional delegation nginx Deployment + Service |
| Storage | one RWO PVC, two subPaths: `/data` and `/media_store` |
| Ingress | yes, gated by `ingress.enabled`; `ingress.domain` survives the gate — see [§12.1](Chart-Style-Guide#12-values-that-must-not-lie) |
| Routing | **two mutually exclusive modes**, Ingress (default) or Gateway API (`httpRoute.enabled`). `ingress.domain` survives both gates — see [§12.1](Chart-Style-Guide#12-values-that-must-not-lie). The Ingress is **one object with two host rules**; Gateway mode is **two `HTTPRoute` objects** — see [Routing](#routing-ingress-or-gateway-api). In Gateway mode the **media upload limit is not enforced** |
| External dependencies | whatever `database:` in the homeserver.yaml names (external PostgreSQL, or SQLite under `/data`) |
| Secrets | 1 × `OnePasswordItem` |
| Install-test | `ci/matrix-synapse/` (fixtures + seed job + test values, `INSTALL_TIMEOUT=600`) |
@@ -18,7 +18,7 @@ Deploys [Matrix Synapse](https://element-hq.github.io/synapse/) as a single-repl
- **1Password operator** in the cluster, plus one item carrying a field named literally `homeserver.yaml` — see [the fields below](#what-the-1password-item-must-contain).
- **A complete Synapse configuration**, including a signing key and a log config on the volume. `synapse generate` produces them; the chart does not.
- **A reachable database** as named in that file. Nothing here deploys one.
- **cert-manager** with the `ClusterIssuer` named in `ingress.clusterIssuer`, and DNS for **both** hostnames pointing at the ingress controller when the delegation is on.
- **cert-manager** with the `ClusterIssuer` named in `ingress.clusterIssuer`, and DNS for **both** hostnames pointing at the ingress controller when the delegation is on. In **Gateway mode** the cluster issuer is not needed from this release: both certificates sit on the Gateway's listeners (two listeners, or one wildcard covering both names). What is needed instead is the **Gateway-API CRDs** plus a Gateway whose listener admits routes from this namespace — and since Matrix federation is HTTPS-only, a listener without a valid certificate for these names is a federation outage that no probe in this release can see.
- **The server name decided before the first start.** It cannot be changed afterwards — see [The server name is decided once](#the-server-name-is-decided-once-and-not-by-matrixservername).
## Installation
@@ -137,16 +137,77 @@ The delegation Deployment has **no `env` block** — it is the one container in
| Key | Type | Default | Description |
|---|---|---|---|
| `ingress.enabled` | bool | `true` | `false` omits the Ingress and frees `clusterIssuer` — **but not `domain`**. |
| `ingress.domain` | string | `null` | **Required while the Ingress *or* the delegation is enabled.** Where Synapse actually answers, and the value baked into both `.well-known` documents. |
| `ingress.clusterIssuer` | string | `null` | **Required while the Ingress is enabled.** cert-manager `ClusterIssuer`. |
| `ingress.className` | string | `nginx` | Ingress class. |
| `ingress.annotations` | object | `{}` | Merged with the cert-manager annotation the chart always sets. |
| `ingress.enabled` | bool | `true` | `false` omits the Ingress and frees `clusterIssuer` — **but not `domain`**. **Mutually exclusive with `httpRoute.enabled`.** |
| `ingress.domain` | string | `null` | **Required while the Ingress, the HTTPRoute *or* the delegation is enabled.** Where Synapse actually answers, the value baked into both `.well-known` documents, and the homeserver `HTTPRoute`'s hostname while `httpRoute.hostnames` is empty. |
| `ingress.clusterIssuer` | string | `null` | **Required while the Ingress is enabled.** cert-manager `ClusterIssuer`. Unused *and* unnecessary in Gateway mode. |
| `ingress.className` | string | `nginx` | Ingress class. No counterpart in Gateway mode — the class sits on the Gateway. |
| `ingress.annotations` | object | `{}` | Merged with the cert-manager annotation the chart always sets. **No counterpart in Gateway mode**, and for this chart that matters twice over: the live Ingress carries a body size here *and* a `configuration-snippet`. See [What Gateway mode costs](#what-gateway-mode-costs-this-chart-and-one-of-the-two-is-not-what-it-looks-like). |
TLS secrets: `tls-<release>-matrix-synapse-ingress` for `ingress.domain`, and `tls-<release>-matrix-synapse-delegation` for `matrix.delegation.serverUrl`.
TLS secrets: `tls-<release>-matrix-synapse-ingress` for `ingress.domain`, and `tls-<release>-matrix-synapse-delegation` for `matrix.delegation.serverUrl`. In Gateway mode the chart renders neither — both certificates belong on the Gateway's listeners.
### `httpRoute`
The Gateway-API alternative to the Ingress, **off by default**. Added with group 3 of the Gateway-API sweep (Gitea #114); the form is the pilot's and is binding for every chart ([style guide §4](Chart-Style-Guide#4-ingress-and-gateway-api)).
| Key | Type | Default | Description |
|---|---|---|---|
| `httpRoute.enabled` | bool | `false` | Renders `<release>-matrix-synapse-httproute` and — while `matrix.delegation.enabled` — `<release>-matrix-synapse-delegation-httproute`, instead of the one Ingress. **Mutually exclusive with `ingress.enabled`**: with both `true` the render aborts naming both values. |
| `httpRoute.parentRefs` | list | `[]` | The Gateway(s) the **homeserver** route attaches to — Gateway API `ParentReference` entries (`name`, and optionally `namespace`, `sectionName`, `port`, `group`, `kind`), passed through unchanged. **Required while `httpRoute.enabled` is true**: the render aborts on an empty list. The schema requires `name` on every entry and is otherwise left open, because Kubernetes owns that schema and adds fields to it across releases. |
| `httpRoute.hostnames` | list | `[]` | Empty means `[ingress.domain]`, so the homeserver hostname has a single source of truth. **Does not cover the delegation host** — deliberately; see below. |
| `httpRoute.delegation.parentRefs` | list | `[]` | The delegation route's own parent. Empty falls back to `httpRoute.parentRefs`. Worth setting, because the server name is usually an apex domain published by a **different** Gateway — somebody else's website normally owns it. |
| `httpRoute.delegation.hostnames` | list | `[]` | Empty means `[matrix.delegation.serverUrl]`, and it deliberately does **not** fall back to `httpRoute.hostnames`: the server name is what *remote* homeservers resolve and is a different thing from the homeserver's own hostname. The migration order depends on telling them apart — move the homeserver host first, confirm it answers, move the server name last. A chart-global list covering both would erase that distinction. |
## Operating notes
### Routing: Ingress or Gateway API
Two mutually exclusive modes, and exactly one of them may be on. `ingress.enabled` ships `true`, `httpRoute.enabled` ships `false`.
**The shapes are not symmetrical, and that is forced rather than chosen.** The Ingress is **one object with two host rules**:
```
ingress.domain / synapse service
matrix.delegation.serverUrl /.well-known/matrix delegation service
```
In the Gateway API that has to become **two `HTTPRoute` objects**, because `spec.hostnames` applies to the *whole* object — one route can carry several hostnames only when they want the **same** rules, and these two want different paths and different backends.
> #114 describes this chart as rendering its Ingresses "in a loop over several apps". It does not; that is [freesailarr](freesailarr). This chart renders one object with two host rules.
| Rule | Why |
|---|---|
| `ingress.enabled` and `httpRoute.enabled` cannot both be true — the render aborts naming **both** value paths | two data paths onto one Service under one hostname is a conflict at DNS level already. For Matrix it is worse: `/.well-known/matrix` tells **every other homeserver in the federation** where to find this one, so a server name resolving to two data paths makes federation succeed or fail per remote server and per retry. **A migration runs over a second hostname**, and the delegation host moves **last**, after the homeserver host answers. |
| `httpRoute.parentRefs` must not be empty — the render aborts | a route without a parent attaches to **no** Gateway. The object is created, it is schema-valid, no controller rejects it, and it routes nothing: no event, no condition, and the only symptom is a hostname that does not answer — which here means federation is down, not just the web client. |
| The delegation route is gated on `matrix.delegation.enabled` | the same switch the Ingress's second host rule uses. Without the delegation workload there is nothing to route to. |
| No `URLRewrite` filter in either mode | the delegation nginx answers on the **full** `/.well-known/matrix/*` path, so nothing is stripped — which is also why the #118 prefix/redirect assurance has no precondition to meet here. |
### One thing Gateway mode makes easier, and it is the awkward part of Matrix delegation
The server name is normally an **apex domain that already serves a website**. Under ingress-nginx two Ingress objects cannot both claim one hostname — the admission webhook rejects the second claim on the same host and path — which is exactly why the delegation rule has to live *inside* this chart's single Ingress object rather than beside it.
HTTPRoutes do not work that way: several routes may attach to the same listener for the same hostname, and the implementation **merges** them by match specificity, a longer path winning. So in Gateway mode the `/.well-known/matrix` route can sit next to whatever else owns that apex, without either side knowing about the other. It is one of the two places in the whole sweep where the Gateway API is a better fit rather than a translation (the other is [freesailarr](freesailarr)'s per-rule rewrite).
### What Gateway mode costs this chart, and one of the two is not what it looks like
Measured on the live Ingress on 2026-10-12 — all of it **operator-supplied** through `ingress.annotations`; this chart sets none of it:
```
nginx.ingress.kubernetes.io/proxy-body-size: 50m
nginx.org/client-max-body-size: 50m
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "X-Forwarded-For $remote_addr";
more_set_headers "X-Forwarded-Proto $scheme";
more_set_headers "Host $host:$server_port";
```
**1. The body size is the real loss.** Synapse's media repository takes uploads and its own ceiling is `max_upload_size` in `homeserver.yaml` — which this chart **cannot see**, because the whole configuration arrives from 1Password. The effective limit is the smaller of the two, and in Gateway mode the proxy half becomes **the Gateway's default**: neither number anyone can point at is necessarily the one that decides. There is no body-size field in the Gateway API — it is implementation policy, not routing — so the limit is documented rather than rebuilt. Set the Gateway's half to match `max_upload_size` before pointing DNS at it. (`nginx.org/client-max-body-size` is the same limit spelled for NGINX Inc's ingress controller rather than ingress-nginx — a hedge across both — and it goes for the same reason.)
**2. The snippet is not load-bearing, and that took checking rather than assuming.** It *looks* like what Synapse's `x_forwarded` setting depends on — client IP for rate limiting, and federation. It is not: `more_set_headers` is the headers-more module's **output**-header directive ("Replaces (if any) or adds (if not any) the specified *output* headers"), while the request-side directive is a different one, `more_set_input_headers` ("it operates on input headers (or request headers)"). So those three headers are added to the **response to the client**; Synapse never sees them, and what it reads with `x_forwarded: true` is the **request's** `X-Forwarded-For`, which ingress-nginx populates itself.
So for a migration: **not translatable, and not needed.** The Gateway API's `ResponseHeaderModifier` could set static response headers but knows no nginx variables (`$remote_addr`, `$scheme`, `$host`), so rebuilding it is impossible anyway — and pointless. **Do not re-create it at the Gateway in the belief that Synapse needs it, and do not read its absence as the cause of a rate-limiting or federation problem.**
This is a reading of the directive's documented semantics, stated as such: nothing here observed a Synapse instance with and without the snippet.
### `ingress.domain` needs its own guard
This chart is the reason [§12.1](Chart-Style-Guide#12-values-that-must-not-lie) exists. The delegation nginx serves `ingress.domain` inside `/.well-known/matrix/{server,client}` — the endpoints through which **federation finds the homeserver**. Putting the Ingress behind `ingress.enabled` without a second `required()` on the domain would publish
@@ -228,6 +289,12 @@ Every pod of the StatefulSet mounts the same RWO claim and the same `homeserver.
**That fixture pins a Synapse image Renovate does not follow** (`ci/**` is deliberately unmanaged, so the throwaway infrastructure stays on its major lines). Bump `ghcr.io/element-hq/synapse` in `ci/matrix-synapse/fixtures.yaml` **by hand** when the chart moves a major, or the seed job generates a config for a different version than the one under test. See [Renovate](Renovate) and [CI/CD](CI-CD).
**The Gateway mode is held at render level**, by `ci/httproute/matrix-synapse.yaml` plus `ci/run-httproute-match-test.sh`, which runs over every chart on every PR ([CI/CD](CI-CD#httproute-render-test)). This chart is the one in its group where "both routing objects" means two different hostnames, two different paths and two different **backends**, and the overlay covers both: the fixture already sets `ingress.domain` and has the delegation on with `matrix.delegation.serverUrl: ci.local`, so the homeserver route renders on its host at `/` pointing at the Synapse Service, and the delegation route renders on *its* host matching `PathPrefix /.well-known/matrix` and pointing at the **delegation** Service. **4 assertions** — a `parentRefs` and a `backendRefs` check per route — and both references are held against the Services of the same render **by port number**, because `backendRefs` takes a `PortNumber` and cannot name the port `http` the way the Ingress does.
An overlay rather than a fixture change, because the install-test runs one configuration per chart and this chart's fixture is the one with the pre-bound PV and the `synapse generate` seed job.
**What that does not cover**: that either hostname answers — which for Matrix is also whether **federation** works. No Gateway controller runs in CI, no route reaches `Accepted: True`, and Matrix federation is HTTPS-only while nothing in this mode renders a certificate. Nor the body-size limit, which has no counterpart in that mode; that loss is documented in `values.yaml` and printed by `NOTES.txt`, and documentation is not something a test can assert. **Nothing in CI reads `NOTES.txt`**: `helm template` does not emit it and no script under `ci/` reads it, so its text carries review-only assurance.
## See also
- [Chart style guide](Chart-Style-Guide) — §12.1 is written from this chart