feat/bulwark: new chart for Bulwark Webmail 1.0.0+up1.12.0 #90

2026-10-05T23:29:58+02:00

jlab-bot commented

2026-10-05 23:29:58 +02:00

Builds the chart for Bulwark Webmail, the JMAP client for stalwart (Gitea #65).

StatefulSet, one PVC at /app/data, ClusterIP Service plus the headless one the StatefulSet needs, Ingress at a host root, two OnePasswordItems, three probes, computed limits, replicas + scaleDown, strict values.schema.json. keycloak/ was the pattern for hardening, schema and the null-safe fallbacks.

Everything below was measured against the source at tag 1.12.0. The upstream documentation was wrong about three of them, and the approved plan inherited two of those plus one thing the plan assumed about Kubernetes.

Five corrections to the plan, found while building

The first is the one that matters least; the last is the only deviation from a binding decision, and it is why this PR says Part of #89.

BULWARK_TELEMETRY is not on by default. The plan treated switching it off as a deviation from the image. Measured: DEFAULTS carries consent: 'off' — telemetry is opt-in, and the comment in instrumentation.node.ts calling it "on by default" is wrong. What rendering the variable actually buys is the lock: the environment beats the state file and upstream blocks the admin UI from changing it. That makes it a third exception to "the volume beats the chart", alongside the session secret and the admin password. BULWARK_UPDATE_CHECK is the opposite — on unless switched off — so the two values are not symmetrical.
adminConfigReadOnly costs more than the plan priced. Comment 5349 named one price (it freezes setPolicy, so policy/themes/branding and the push relay). There are two more, both from the same assertWritable:
- removeAdminOverride is guarded too, so turning it on freezes an existing override in place instead of clearing it — converting a recoverable lockout into a permanent one and removing the documented repair in the same move;
- initAdminPassword persists through writeConfigData, which begins with assertWritable('save admin password'). On a fresh install that throws: the startup hook logs Admin dashboard init skipped, the retry in /api/admin/auth throws again, and the login endpoint answers HTTP 500 — while the pod is Ready and serving mail. The value meant to protect the configuration would remove the rescue door.
The decision (ship it, default false) stands and is now better justified. The complete price and the only safe ordering are at the value and in NOTES.txt.
The OIDC redirect URI carries the UI language. It is https://<domain>/<locale>/auth/callback, where <locale> is what the visitor's browser resolved to — 27 locales, and the locale prefix is never in this image, so the language is not in the visible URL. Upstream's own admin helper registers one URI per locale for exactly this reason. A single registered URI is the likeliest cause of a failed first sign-in, and nothing had named it. NOTES.txt prints the wildcard and the full locale list.
The registry publishes every release twice. The earlier note said the tags carry no v prefix. Measured: 1.12.0 and v1.12.0 exist for the same image, both families running back to 1.3.9. The chart pins the suffix-free form; docker versioning normalizes the other to the same version, so it is never offered as an update. Full census in the Renovate block: 315 tags in eight shapes.
A Kubernetes probe cannot send HEAD — and 5349 fixes HEAD /api/health as binding. HTTPGetAction has no method field and the kubelet always issues a GET; the only alternative is an exec probe, and this image carries neither curl nor wget (npm/npx are removed from the runtime layer), so it would mean starting a Node process every few seconds. The chart uses GET. Every limit 5349 argues is unchanged — the handler still returns a hard 200, still hard-codes healthy for an unauthenticated caller, and still says nothing about Stalwart — and GET is marginally more capable than HEAD, since its catch block can 503 when the diagnostics themselves throw. This is the one decision I changed rather than asked about, and it wants ratification.

The chart's defining property

configManager.get resolves admin override > env var > default. Every value this chart sets is a default the admin dashboard may already be overriding — it may write any key of CONFIG_ENV_MAP, and nothing says it just disconnected the chart. A helm upgrade can succeed and change nothing. Upstream documents this for the server URL only; it is true of everything, including oauthOnly, which is why the plan's rescue instruction did not work (5342/5349, re-confirmed here against app/api/auth/session/route.ts, app/api/auth/verify/route.ts and app/api/config/route.ts).

Three values escape it: SESSION_SECRET(_FILE), ADMIN_PASSWORD, BULWARK_TELEMETRY. The OAuth client secret does not — getGlobalClientSecret checks the admin override first, so a plaintext oauthClientSecret in config.json beats the mounted file forever.

Other decisions worth reviewing

Both secrets use the file form (SESSION_SECRET_FILE, OAUTH_CLIENT_SECRET_FILE) — the better path garage (#81) and keycloak (#83) did not have — in one projected volume mounted without subPath, because both are re-read on every use and the kubelet never refreshes a subPath mount. Mode 0440, not 0400: secret files are owned by root and fsGroup supplies only the group, so without the group bit uid 1001 is denied — and readFileEnv answers EACCES exactly as it answers "absent", so the failure would be a feature quietly switching itself off. #81's permission trap is the mirror image and was checked: lib/read-file-env.ts has no statSync and no mode check, so unlike garage, Bulwark does not reject a group-readable file.
Two vault items, split by audience, like garage's RPC/S3 split: ours (session-secret, admin-password) and the provider's (oauth-client-secret).
No ConfigMap, so §9 has nothing to attach to — stated explicitly in values.yaml so it does not read as an omission. checksum/secret-ref covers the op:// paths.
Render-time guards in the StatefulSet (not NOTES.txt, which helm template never renders) for jmapServerUrl and the two OIDC values. The first is not merely "the mail server is unset": an empty value puts the app into bootstrap, serves the setup wizard, and whatever is clicked through there lands on the PVC and overrides the chart permanently. An unset value produces a silently different release, not a broken one.
No value for the push relay, on purpose. pushRelayUrl lives in SettingsPolicy and has no CONFIG_ENV_MAP entry, so a chart key would be a switch wired to nothing — the exact defect §12 exists to remove. The two-part statement from 5349 is a comment instead.
Ingress gating frees both domain and clusterIssuer (the calibre-web-automated case). Checked in both directions: nothing outside the Ingress template functionally reads the domain — Bulwark derives its public URL from the request origin and takes no base-URL variable. NOTES.txt reads it, so bulwark.publicUrl renders a visible placeholder rather than carrying a required() that no render could reach.
stalwartAdminAccess kept at upstream's auto, with the consequence written down rather than silently changed: with auto, any Stalwart admin reaches /admin without the Bulwark admin password — and /admin is what writes the overrides and can lock the login path. Usually the same person; password narrows it. Flagged rather than deviated from, since it was not in the plan.

Verified locally

Check	Result
`ci/helm-dependencies.sh bulwark`	exit 0 (no dependencies declared)
`helm lint --strict bulwark`	0 findings
`helm package`	`bulwark-1.0.0+up1.12.0.tgz`
`helm template` vs. defaults	fails, naming the missing value — as intended
`helm template` vs. `ci/bulwark/test-values.yaml`	7 objects: StatefulSet, PVC, 2 Services, Ingress, 2 OnePasswordItems
`ci/run-quantity-render-test.sh bulwark`	both cases exercise the helper, no exponential notation
`ci/run-quantity-render-test.sh` (all charts)	28 exercised, 0 failed
Schema, both directions	unknown key rejected (`additional properties 'nonsense' not allowed`); `replicas: 2` rejected (`maximum: got 2, want 1`)
Null-safety (#99)	`bulwark=null`, `securityContext=null`, `readinessProbe=null`, `resources=null`, `writableCache=null` all fall back to chart defaults
`scaleDown` / explicit `replicas: 0`	both render `replicas: 0`
`NOTES.txt`	rendered via the `define`+ConfigMap trick in a throwaway copy, in four value combinations
Renovate dry run	44.138.0, `--platform=local --dry-run=lookup`, reading this PR's `renovate.json5`: `1.12.0` → `[]`; `1.11.0` → exactly `1.11.2` (patch) + `1.12.0` (minor); `1.10.0` → exactly `1.12.0` (minor). No beta, no `-always`, no `v` form, no floating or SHA tag

Not verified — and the install-test is what settles the first one

No container runtime in this workspace (no Docker daemon; podman cannot set up user namespaces), no k3d, no Trivy. So:

The exact set of emptyDirs needed for readOnlyRootFilesystem: true is DERIVED, not measured. next.config.ts builds output: "standalone" and images.unoptimized applies only in the isLite branch, so image optimization is active and writes under /app/.next/cache; /tmp is the conventional scratch path. The install-test in this PR is the proof — a missing writable path is an EROFS/EACCES on the first write, so a pod that reaches and stays Ready settles it. The other half stays open permanently and is named on the wiki page: a superfluous emptyDir would not be caught.
No Trivy run. ghcr.io is already in ci/trivy/data/ksv0125.yaml, so no data change was needed.
Nothing about JMAP, OIDC or the precedence trap is exercised by CI — config.json never exists in a CI run. Named in ci/bulwark/test-values.yaml.
The two file-form secret key names are not proven by the install-test: the projected volume is optional: true by design, so a typo yields a missing file and a healthy pod. Only admin-password fails loudly.
The startup budget (120s) is reasoned from what the start does, not stopwatched.

Wiki

Done in this round, pushed to the wiki repo's main as 996f8c0 (signed, rebased, never force):

new page bulwark;
row in Charts — and the counter is now measured against */Chart.yaml instead of carried forward, because it had drifted. The table also still lists convertx, for which there is no chart directory in this repository. Flagged in a note rather than deleted: removing it is a decision about that chart, not a documentation fix. Worth a look.
entry in _Sidebar;
cross-reference on the stalwart page, since the two are only useful together;
four Lessons-Learned entries, all transferable: the recovery instruction that ran through the fault it repaired; the read-only switch whose write barrier also blocked the one-time bootstrap write; the probe method the platform does not offer; and the 0400 secret file that only works because the kubelet widens it.

Keyword

Part of #89, deliberately not Fixes. The chart is complete, but correction 5 changes a decision 5349 fixed as binding, and that belongs to you to confirm rather than to an auto-close. Everything else is either a correction that strengthens the plan or a measurement now recorded. Once you are happy with the GET-instead-of-HEAD call and the install-test is green, #89 can be closed by hand.

I have not merged anything and have not armed auto-merge.

Builds the chart for **Bulwark Webmail**, the JMAP client for `stalwart` (Gitea #65). StatefulSet, **one** PVC at `/app/data`, ClusterIP Service plus the headless one the StatefulSet needs, Ingress at a host **root**, two `OnePasswordItem`s, three probes, computed limits, `replicas` + `scaleDown`, strict `values.schema.json`. `keycloak/` was the pattern for hardening, schema and the null-safe fallbacks. Everything below was measured against the **source at tag `1.12.0`**. The upstream documentation was wrong about three of them, and the approved plan inherited two of those plus one thing the plan assumed about Kubernetes. ## Five corrections to the plan, found while building The first is the one that matters least; the last is the only deviation from a **binding** decision, and it is why this PR says `Part of #89`. 1. **`BULWARK_TELEMETRY` is not on by default.** The plan treated switching it off as a deviation from the image. Measured: `DEFAULTS` carries `consent: 'off'` — telemetry is opt-in, and the comment in `instrumentation.node.ts` calling it "on by default" is wrong. What rendering the variable actually buys is the **lock**: the environment beats the state file *and* upstream blocks the admin UI from changing it. That makes it a **third** exception to "the volume beats the chart", alongside the session secret and the admin password. `BULWARK_UPDATE_CHECK` is the opposite — on unless switched off — so the two values are not symmetrical. 2. **`adminConfigReadOnly` costs more than the plan priced.** Comment 5349 named one price (it freezes `setPolicy`, so policy/themes/branding and the push relay). There are two more, both from the same `assertWritable`: - `removeAdminOverride` is guarded too, so turning it on **freezes an existing override in place** instead of clearing it — converting a recoverable lockout into a permanent one and removing the documented repair in the same move; - `initAdminPassword` persists through `writeConfigData`, which begins with `assertWritable('save admin password')`. On a **fresh** install that throws: the startup hook logs `Admin dashboard init skipped`, the retry in `/api/admin/auth` throws again, and **the login endpoint answers HTTP 500** — while the pod is `Ready` and serving mail. The value meant to protect the configuration would remove the rescue door. The decision (ship it, default `false`) stands and is now better justified. The complete price and the only safe ordering are at the value and in `NOTES.txt`. 3. **The OIDC redirect URI carries the UI language.** It is `https://<domain>/<locale>/auth/callback`, where `<locale>` is what the visitor's browser resolved to — 27 locales, and the locale prefix is `never` in this image, so the language is **not** in the visible URL. Upstream's own admin helper registers one URI per locale for exactly this reason. A single registered URI is the likeliest cause of a failed first sign-in, and nothing had named it. `NOTES.txt` prints the wildcard and the full locale list. 4. **The registry publishes every release twice.** The earlier note said the tags carry no `v` prefix. Measured: `1.12.0` **and** `v1.12.0` exist for the same image, both families running back to `1.3.9`. The chart pins the suffix-free form; `docker` versioning normalizes the other to the same version, so it is never offered as an update. Full census in the Renovate block: 315 tags in eight shapes. 5. **A Kubernetes probe cannot send `HEAD`** — and 5349 fixes `HEAD /api/health` as binding. `HTTPGetAction` has no method field and the kubelet always issues a GET; the only alternative is an `exec` probe, and this image carries neither `curl` nor `wget` (`npm`/`npx` are removed from the runtime layer), so it would mean starting a Node process every few seconds. **The chart uses GET.** Every limit 5349 argues is unchanged — the handler still returns a hard 200, still hard-codes `healthy` for an unauthenticated caller, and still says nothing about Stalwart — and GET is marginally *more* capable than HEAD, since its catch block can 503 when the diagnostics themselves throw. **This is the one decision I changed rather than asked about, and it wants ratification.** ## The chart's defining property `configManager.get` resolves `admin override > env var > default`. Every value this chart sets is a **default** the admin dashboard may already be overriding — it may write any key of `CONFIG_ENV_MAP`, and nothing says it just disconnected the chart. A `helm upgrade` can succeed and change nothing. Upstream documents this for the server URL only; it is true of everything, including `oauthOnly`, which is why the plan's rescue instruction did not work (5342/5349, re-confirmed here against `app/api/auth/session/route.ts`, `app/api/auth/verify/route.ts` and `app/api/config/route.ts`). Three values escape it: `SESSION_SECRET(_FILE)`, `ADMIN_PASSWORD`, `BULWARK_TELEMETRY`. **The OAuth client secret does not** — `getGlobalClientSecret` checks the admin override first, so a plaintext `oauthClientSecret` in `config.json` beats the mounted file forever. ## Other decisions worth reviewing - **Both secrets use the file form** (`SESSION_SECRET_FILE`, `OAUTH_CLIENT_SECRET_FILE`) — the better path `garage` (#81) and `keycloak` (#83) did not have — in **one projected volume mounted without `subPath`**, because both are re-read on every use and the kubelet never refreshes a `subPath` mount. Mode **`0440`, not `0400`**: secret files are owned by `root` and `fsGroup` supplies only the group, so without the group bit uid 1001 is denied — and `readFileEnv` answers `EACCES` exactly as it answers "absent", so the failure would be a feature quietly switching itself off. #81's permission trap is the mirror image and was checked: `lib/read-file-env.ts` has no `statSync` and no mode check, so unlike garage, Bulwark does not *reject* a group-readable file. - **Two vault items, split by audience**, like garage's RPC/S3 split: ours (`session-secret`, `admin-password`) and the provider's (`oauth-client-secret`). - **No ConfigMap**, so §9 has nothing to attach to — stated explicitly in `values.yaml` so it does not read as an omission. `checksum/secret-ref` covers the `op://` paths. - **Render-time guards** in the StatefulSet (not `NOTES.txt`, which `helm template` never renders) for `jmapServerUrl` and the two OIDC values. The first is not merely "the mail server is unset": an empty value puts the app into `bootstrap`, serves the setup wizard, and whatever is clicked through there lands on the PVC and overrides the chart permanently. An unset value produces a **silently different** release, not a broken one. - **No value for the push relay, on purpose.** `pushRelayUrl` lives in `SettingsPolicy` and has no `CONFIG_ENV_MAP` entry, so a chart key would be a switch wired to nothing — the exact defect §12 exists to remove. The two-part statement from 5349 is a comment instead. - **Ingress gating frees both `domain` and `clusterIssuer`** (the `calibre-web-automated` case). Checked in both directions: nothing outside the Ingress template *functionally* reads the domain — Bulwark derives its public URL from the request origin and takes no base-URL variable. `NOTES.txt` reads it, so `bulwark.publicUrl` renders a visible placeholder rather than carrying a `required()` that no render could reach. - **`stalwartAdminAccess` kept at upstream's `auto`**, with the consequence written down rather than silently changed: with `auto`, any Stalwart admin reaches `/admin` **without** the Bulwark admin password — and `/admin` is what writes the overrides and can lock the login path. Usually the same person; `password` narrows it. Flagged rather than deviated from, since it was not in the plan. ## Verified locally | Check | Result | |---|---| | `ci/helm-dependencies.sh bulwark` | exit 0 (no dependencies declared) | | `helm lint --strict bulwark` | **0 findings** | | `helm package` | `bulwark-1.0.0+up1.12.0.tgz` | | `helm template` vs. defaults | fails, naming the missing value — as intended | | `helm template` vs. `ci/bulwark/test-values.yaml` | 7 objects: StatefulSet, PVC, 2 Services, Ingress, 2 OnePasswordItems | | `ci/run-quantity-render-test.sh bulwark` | both cases exercise the helper, no exponential notation | | `ci/run-quantity-render-test.sh` (all charts) | 28 exercised, **0 failed** | | Schema, both directions | unknown key rejected (`additional properties 'nonsense' not allowed`); `replicas: 2` rejected (`maximum: got 2, want 1`) | | Null-safety (#99) | `bulwark=null`, `securityContext=null`, `readinessProbe=null`, `resources=null`, `writableCache=null` all fall back to chart defaults | | `scaleDown` / explicit `replicas: 0` | both render `replicas: 0` | | `NOTES.txt` | rendered via the `define`+ConfigMap trick in a throwaway copy, in four value combinations | | Renovate dry run | 44.138.0, `--platform=local --dry-run=lookup`, reading this PR's `renovate.json5`: `1.12.0` → `[]`; `1.11.0` → exactly `1.11.2` (patch) + `1.12.0` (minor); `1.10.0` → exactly `1.12.0` (minor). No beta, no `-always`, no `v` form, no floating or SHA tag | ## Not verified — and the install-test is what settles the first one **No container runtime in this workspace** (no Docker daemon; podman cannot set up user namespaces), no k3d, no Trivy. So: - **The exact set of emptyDirs needed for `readOnlyRootFilesystem: true` is DERIVED, not measured.** `next.config.ts` builds `output: "standalone"` and `images.unoptimized` applies only in the `isLite` branch, so image optimization is active and writes under `/app/.next/cache`; `/tmp` is the conventional scratch path. **The install-test in this PR is the proof** — a missing writable path is an `EROFS`/`EACCES` on the first write, so a pod that reaches and stays `Ready` settles it. The other half stays open permanently and is named on the wiki page: a **superfluous** emptyDir would not be caught. - No Trivy run. `ghcr.io` is already in `ci/trivy/data/ksv0125.yaml`, so no data change was needed. - Nothing about JMAP, OIDC or the precedence trap is exercised by CI — `config.json` never exists in a CI run. Named in `ci/bulwark/test-values.yaml`. - The two file-form secret **key names** are not proven by the install-test: the projected volume is `optional: true` by design, so a typo yields a missing file and a healthy pod. Only `admin-password` fails loudly. - The startup budget (120s) is reasoned from what the start does, not stopwatched. ## Wiki Done in this round, pushed to the wiki repo's `main` as `996f8c0` (signed, rebased, never force): - new page **`bulwark`**; - row in **`Charts`** — and the counter is now **measured** against `*/Chart.yaml` instead of carried forward, because it had drifted. **The table also still lists `convertx`, for which there is no chart directory in this repository.** Flagged in a note rather than deleted: removing it is a decision about that chart, not a documentation fix. Worth a look. - entry in **`_Sidebar`**; - cross-reference on the **`stalwart`** page, since the two are only useful together; - four **`Lessons-Learned`** entries, all transferable: the recovery instruction that ran through the fault it repaired; the read-only switch whose write barrier also blocked the one-time bootstrap write; the probe method the platform does not offer; and the `0400` secret file that only works because the kubelet widens it. ## Keyword **`Part of #89`, deliberately not `Fixes`.** The chart is complete, but correction 5 changes a decision 5349 fixed as binding, and that belongs to you to confirm rather than to an auto-close. Everything else is either a correction that strengthens the plan or a measurement now recorded. Once you are happy with the GET-instead-of-HEAD call and the install-test is green, #89 can be closed by hand. I have not merged anything and have not armed auto-merge.

jlab-bot added 1 commit 2026-10-05 23:29:58 +02:00

feat(bulwark): new chart for Bulwark Webmail (1.0.0+up1.12.0)

Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s

Details

Validate Helm Charts / Resource quantities (pull_request) Successful in 13s

Details

Validate Helm Charts / Build bulwark (pull_request) Successful in 6s

Details

Validate Helm Charts / Trivy config bulwark (pull_request) Successful in 10s

Details

Install-test / Install-test bulwark (pull_request) Successful in 8m50s

Details

Validate Helm Charts / Install-test (pull_request) Successful in 8m50s

Details

75f2f31e60

A JMAP webmail client for the stalwart chart (Gitea #65): StatefulSet, one PVC
at /app/data, Service plus headless Service, Ingress at a host root, two
OnePasswordItems, three probes, computed limits, replicas plus scaleDown, strict
values.schema.json.

Everything below was measured against the source at tag 1.12.0 rather than read
out of the documentation, which was wrong about three of them.

THE DEFINING PROPERTY: THE VOLUME BEATS THE CHART, FOR EVERY VALUE, SILENTLY.
configManager.get resolves "admin override > env var > default", the admin
dashboard may write any key of CONFIG_ENV_MAP, and nothing reports that it just
disconnected the chart - so a helm upgrade can succeed and change nothing.
Upstream documents this for the server URL only; it is true of everything.
Exactly three values escape it (SESSION_SECRET(_FILE), ADMIN_PASSWORD,
BULWARK_TELEMETRY). The value that closes it, bulwark.adminConfigReadOnly,
defaults to false because its price is threefold: it freezes the settings policy
(and with it the only way to move Web Push off the vendor relay), it freezes an
override that ALREADY exists instead of removing it, and on a fresh install it
stops admin.json from being written at all - so the admin dashboard never comes
up and its login answers HTTP 500 while the pod stays Ready.

OIDC-only sign-in, with the real way back out of a lockout: not helm upgrade
with OAUTH_ONLY=false (oauthOnly goes through the same precedence chain) but
removing the override in the admin dashboard. That is why ADMIN_PASSWORD is set
- the dashboard is the door. It generates no random password and logs nothing,
so the #65 analogy does not apply.

SESSION_SECRET_FILE and OAUTH_CLIENT_SECRET_FILE are both used - the better path
garage (#81) and keycloak (#83) did not have - in one projected volume mounted
WITHOUT subPath, because both are re-read on every use and a subPath mount is
never refreshed, and at mode 0440 because secret files are owned by root and
fsGroup only supplies the group.

Two backup statements, both at the value: without the session secret the
synchronized settings are unreadable out of an intact volume, and if the
dashboard's OAuth setup was ever used the client secret sits in config.json on
that volume in cleartext.

Probes: all three a plain GET on /api/health, with the limit written at the
readiness value - it carries process, HTTP acceptance, a free event loop and
Next.js routing, and nothing about the reported state or about Stalwart.

No ConfigMap, so the checksum/config rule (#82) has nothing to attach to, and
values.yaml says so explicitly; the op:// paths are covered by
checksum/secret-ref.

Renovate: no versioningTemplate, measured rather than assumed. 315 tags in eight
shapes (releases, a second v-prefixed family of the same images, an -always
flavour, floating minors and names, commit SHAs, betas, attestations) and plain
docker versioning excludes all six unwanted ones. Dry runs at 1.12.0, 1.11.0 and
1.10.0 are recorded in the block.

Part of #89 - deliberately not "Fixes", because one binding decision of the
issue was changed out of necessity and wants ratification rather than an
automatic close: the plan specifies HEAD /api/health for all three probes, and a
Kubernetes probe cannot send HEAD at all (HTTPGetAction has no method field; the
kubelet always issues GET). The alternative, an exec probe, would mean starting a
Node process every few seconds in an image that carries no curl and no wget. GET
is used instead and is marginally more capable than HEAD, so every limit the
issue argues is unchanged - but the decision is the maintainer's to confirm.