jklein
b39f36ff5c
A JMAP webmail client for the stalwart chart (Gitea #65): StatefulSet, one PVC at /app/data, Service plus headless Service, Ingress at a host root, two OnePasswordItems, three probes, computed limits, replicas plus scaleDown, strict values.schema.json. Everything below was measured against the source at tag 1.12.0 rather than read out of the documentation, which was wrong about three of them. THE DEFINING PROPERTY: THE VOLUME BEATS THE CHART, FOR EVERY VALUE, SILENTLY. configManager.get resolves "admin override > env var > default", the admin dashboard may write any key of CONFIG_ENV_MAP, and nothing reports that it just disconnected the chart - so a helm upgrade can succeed and change nothing. Upstream documents this for the server URL only; it is true of everything. Exactly three values escape it (SESSION_SECRET(_FILE), ADMIN_PASSWORD, BULWARK_TELEMETRY). The value that closes it, bulwark.adminConfigReadOnly, defaults to false because its price is threefold: it freezes the settings policy (and with it the only way to move Web Push off the vendor relay), it freezes an override that ALREADY exists instead of removing it, and on a fresh install it stops admin.json from being written at all - so the admin dashboard never comes up and its login answers HTTP 500 while the pod stays Ready. OIDC-only sign-in, with the real way back out of a lockout: not helm upgrade with OAUTH_ONLY=false (oauthOnly goes through the same precedence chain) but removing the override in the admin dashboard. That is why ADMIN_PASSWORD is set - the dashboard is the door. It generates no random password and logs nothing, so the #65 analogy does not apply. SESSION_SECRET_FILE and OAUTH_CLIENT_SECRET_FILE are both used - the better path garage (#81) and keycloak (#83) did not have - in one projected volume mounted WITHOUT subPath, because both are re-read on every use and a subPath mount is never refreshed, and at mode 0440 because secret files are owned by root and fsGroup only supplies the group. Two backup statements, both at the value: without the session secret the synchronized settings are unreadable out of an intact volume, and if the dashboard's OAuth setup was ever used the client secret sits in config.json on that volume in cleartext. Probes: all three a plain GET on /api/health, with the limit written at the readiness value - it carries process, HTTP acceptance, a free event loop and Next.js routing, and nothing about the reported state or about Stalwart. No ConfigMap, so the checksum/config rule (#82) has nothing to attach to, and values.yaml says so explicitly; the op:// paths are covered by checksum/secret-ref. Renovate: no versioningTemplate, measured rather than assumed. 315 tags in eight shapes (releases, a second v-prefixed family of the same images, an -always flavour, floating minors and names, commit SHAs, betas, attestations) and plain docker versioning excludes all six unwanted ones. Dry runs at 1.12.0, 1.11.0 and 1.10.0 are recorded in the block. Part of #89 - deliberately not "Fixes", because one binding decision of the issue was changed out of necessity and wants ratification rather than an automatic close: the plan specifies HEAD /api/health for all three probes, and a Kubernetes probe cannot send HEAD at all (HTTPGetAction has no method field; the kubelet always issues GET). The alternative, an exec probe, would mean starting a Node process every few seconds in an image that carries no curl and no wget. GET is used instead and is marginally more capable than HEAD, so every limit the issue argues is unchanged - but the decision is the maintainer's to confirm.
330 lines
19 KiB
YAML
330 lines
19 KiB
YAML
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
|
|
{{- /*
|
|
A StatefulSet and not a Deployment, and the reason is the single ReadWriteOnce
|
|
PVC rather than any cluster identity: a rolling Deployment update blocks itself,
|
|
because the new pod waits for a volume the old pod still holds. The repo's
|
|
convention requires a StatefulSet for a stateful application anyway, and this one
|
|
is stateful - /app/data carries the admin configuration, the admin password hash,
|
|
the audit log and the encrypted user settings.
|
|
|
|
NO volumeClaimTemplates: this repo uses standalone PVC templates throughout, so
|
|
the StatefulSet can be renamed in a major PR with a --cascade=orphan note without
|
|
the data following the name (README, guideline 1).
|
|
|
|
NO INIT CONTAINER, and that is checked rather than skipped. Bulwark creates its
|
|
four data subdirectories itself on start (ensureConfigDir / ensureStateDir, both
|
|
mkdir -p), which it has to because the image's own mkdir ran in a layer this
|
|
mount covers; fsGroup makes the fresh volume writable for uid 1001. There is
|
|
nothing to seed and nothing to chown, so an init container would only be a second
|
|
image to pin and update.
|
|
|
|
GUARD AT RENDER TIME, not at start: a missing OIDC client id or issuer with
|
|
oauth.enabled would otherwise produce a release that installs cleanly, becomes
|
|
Ready, and then answers every sign-in attempt with a broken redirect - and with
|
|
oauth.only there is no password form to fall back on. The render fails instead,
|
|
naming the value. It lives HERE rather than in NOTES.txt because `helm template`
|
|
does not render NOTES.txt (guideline 12.2), so a guard there would pass every
|
|
check in CI while being absent.
|
|
*/ -}}
|
|
{{- if $settings.oauth.enabled }}
|
|
{{- if not $settings.oauth.clientId }}
|
|
{{- fail "bulwark.oauth.clientId is required while bulwark.oauth.enabled is true - without it the OIDC redirect is built against an empty client and every sign-in fails. With bulwark.oauth.only (the default) there is no password form to fall back to, so this would be an install nobody can log into. Register the client with the identity provider first; NOTES.txt prints the redirect URI." }}
|
|
{{- end }}
|
|
{{- if not $settings.oauth.issuerUrl }}
|
|
{{- fail "bulwark.oauth.issuerUrl is required while bulwark.oauth.enabled is true - Bulwark fetches the OIDC discovery document from it, and without it there is nothing to discover. For keycloak this is https://<host>/realms/<realm>." }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- if not $settings.jmapServerUrl }}
|
|
{{- /* Not merely "the mail server is unset": an empty JMAP_SERVER_URL puts
|
|
detectSetupState() into "bootstrap" (lib/setup/state.ts), which serves the
|
|
FIRST-RUN WIZARD at /setup and 302s everything else there. The wizard then
|
|
writes its result into config.json on the PVC - and from that moment the
|
|
volume beats every value in this chart, permanently and silently. So an
|
|
unset value here does not produce a non-working release, it produces a
|
|
SILENTLY DIFFERENT ONE, which is worse. */}}
|
|
{{- fail "bulwark.jmapServerUrl is required - it is the address of the Stalwart server AND the switch that disables the first-run setup wizard. Left empty, Bulwark starts in bootstrap mode, serves its wizard at /setup, and whatever is clicked through there lands in config.json on the PVC - which from then on OVERRIDES every value this chart sets, silently. Set it to the stalwart release's JMAP endpoint." }}
|
|
{{- end }}
|
|
apiVersion: apps/v1
|
|
kind: StatefulSet
|
|
metadata:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
|
labels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
|
spec:
|
|
replicas: {{ include "bulwark.replicas" (dict "scaleDown" .Values.scaleDown "replicas" $settings.replicas) }}
|
|
serviceName: {{ .Release.Name }}-{{ .Chart.Name }}-headless
|
|
selector:
|
|
matchLabels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
|
template:
|
|
metadata:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
|
labels:
|
|
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
|
annotations:
|
|
# NO checksum/config ANNOTATION, AND THAT IS NOT AN OMISSION (#82).
|
|
# This chart renders no ConfigMap: Bulwark is configured through
|
|
# environment variables plus the two JSON files it writes itself onto the
|
|
# PVC. There is no rendered config file whose change could fail to
|
|
# restart the workload, so the rule has nothing to attach to. A changed
|
|
# environment variable changes the pod template directly and rolls the
|
|
# pod, which is what the annotation exists to achieve elsewhere.
|
|
#
|
|
# Hash of the rendered OnePasswordItems - the op:// PATHS, not their
|
|
# contents. Repointing secrets.appSecretRef or secrets.oauthSecretRef at
|
|
# a different vault item therefore rolls the pod instead of leaving the
|
|
# old credentials in the running process. A change INSIDE an item is not
|
|
# covered and cannot be, because Helm never sees it - but for the two
|
|
# FILE-form secrets it does not need to be: both are re-read on every use
|
|
# (see the secrets volume below), so a rotation is live without a
|
|
# restart. The exception is admin-password, which is consumed exactly
|
|
# once on the first start and which no restart brings back.
|
|
checksum/secret-ref: {{ include (print $.Template.BasePath "/bulwark.secret.yaml") . | sha256sum }}
|
|
spec:
|
|
automountServiceAccountToken: false
|
|
securityContext:
|
|
{{- include "bulwark.podSecurityContext" $.Values.bulwark.securityContext | nindent 8 }}
|
|
containers:
|
|
- name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
|
|
# Fully qualified. The suffix-free tag family, see Chart.yaml - this
|
|
# registry publishes every release twice (1.12.0 and v1.12.0) and the
|
|
# chart pins the form without the "v".
|
|
image: "ghcr.io/bulwarkmail/webmail:{{ .Chart.AppVersion }}"
|
|
imagePullPolicy: IfNotPresent
|
|
# No command and no args: the image's CMD is ["node", "server.js"] with
|
|
# WORKDIR /app, and the working directory is load-bearing here rather
|
|
# than incidental - Bulwark resolves its data directories relative to
|
|
# process.cwd(), so overriding either would move /app/data out from
|
|
# under the volume mount.
|
|
env:
|
|
# THE MAIL SERVER, AND THE SWITCH THAT KILLS THE SETUP WIZARD. With
|
|
# this set, detectSetupState() returns "env-managed" and /setup is
|
|
# not served at all, which is what makes this chart declarative.
|
|
#
|
|
# IT IS STILL BEATEN BY A SERVER URL SAVED IN THE ADMIN DASHBOARD -
|
|
# the one precedence trap upstream documents itself. See values.yaml.
|
|
- name: JMAP_SERVER_URL
|
|
value: {{ $settings.jmapServerUrl | quote }}
|
|
{{- with $settings.appName }}
|
|
- name: APP_NAME
|
|
value: {{ . | quote }}
|
|
{{- end }}
|
|
- name: STALWART_FEATURES
|
|
value: {{ $settings.stalwartFeatures | quote }}
|
|
- name: STALWART_ADMIN_ACCESS
|
|
value: {{ $settings.stalwartAdminAccess | quote }}
|
|
# The settings sync. Note that this flag alone is not enough: without
|
|
# a session secret, app/api/config/route.ts turns it off SILENTLY
|
|
# (`configManager.get(...) && hasSessionSecret()`). The secret below
|
|
# is therefore mandatory rather than recommended.
|
|
- name: SETTINGS_SYNC_ENABLED
|
|
value: {{ $settings.settingsSync | quote }}
|
|
# THE SESSION SECRET, AS A FILE PATH. Everything about why it is the
|
|
# file form and why the mount has no subPath is at the volume below.
|
|
#
|
|
# This is also ONE OF ONLY TWO VALUES IN THIS CHART THAT THE VOLUME
|
|
# CANNOT OVERRIDE: getSessionSecret (lib/auth/session-secret.ts) reads
|
|
# SESSION_SECRET, then SESSION_SECRET_FILE, and only then the admin
|
|
# override - the reverse of configManager.get's normal order.
|
|
- name: SESSION_SECRET_FILE
|
|
value: {{ include "bulwark.secretMountPath" . }}/session-secret
|
|
# THE ADMIN PASSWORD, AS AN ENVIRONMENT VARIABLE, AND THAT IS FORCED
|
|
# RATHER THAN CHOSEN: ADMIN_PASSWORD has no _FILE form in
|
|
# CONFIG_ENV_MAP, unlike the other two secrets here. The residual
|
|
# price, stated rather than hidden: it is in the container's process
|
|
# environment, so anyone who can exec into the pod or read its /proc
|
|
# can see it until the first start has hashed it into admin.json.
|
|
#
|
|
# WHY IT IS SET AT ALL: the admin dashboard is the only way back out
|
|
# of an OIDC lockout (OAUTH_ONLY below), and without this variable
|
|
# the dashboard is simply disabled - there is no random password and
|
|
# nothing in the log, measured in lib/admin/password.ts.
|
|
- name: ADMIN_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-app-secret
|
|
key: admin-password
|
|
{{- if $settings.adminConfigReadOnly }}
|
|
# THE CHART IS AUTHORITATIVE FROM HERE ON - and this is the one value
|
|
# whose price is bigger than its name. It freezes the settings policy
|
|
# (and with it the only way to move Web Push off the vendor relay),
|
|
# it freezes an override that ALREADY exists instead of removing it,
|
|
# and on a FRESH install it prevents admin.json from being written at
|
|
# all - which takes the admin dashboard, i.e. the rescue door, away
|
|
# entirely while the pod stays Ready. values.yaml has the measurement
|
|
# and the only safe sequence.
|
|
- name: ADMIN_CONFIG_READONLY
|
|
value: "true"
|
|
{{- end }}
|
|
{{- if $settings.oauth.enabled }}
|
|
- name: OAUTH_ENABLED
|
|
value: "true"
|
|
# HIDES THE PASSWORD FORM. The way back out is NOT a helm upgrade
|
|
# with this false - oauthOnly is read through configManager.get by
|
|
# every consumer, so an override in config.json on the PVC wins. The
|
|
# repair is removing that override in the admin dashboard. NOTES.txt
|
|
# carries it.
|
|
- name: OAUTH_ONLY
|
|
value: {{ $settings.oauth.only | quote }}
|
|
- name: OAUTH_CLIENT_ID
|
|
value: {{ $settings.oauth.clientId | quote }}
|
|
- name: OAUTH_ISSUER_URL
|
|
value: {{ $settings.oauth.issuerUrl | quote }}
|
|
# The client secret as a FILE, so it stays out of the process
|
|
# environment - the better path garage (#81) and keycloak (#83) did
|
|
# not have. Re-read on every token exchange, so a rotation needs no
|
|
# restart; see the volume below for the subPath condition that makes
|
|
# that true.
|
|
- name: OAUTH_CLIENT_SECRET_FILE
|
|
value: {{ include "bulwark.secretMountPath" . }}/oauth-client-secret
|
|
{{- with $settings.oauth.scopes }}
|
|
- name: OAUTH_SCOPES
|
|
value: {{ . | quote }}
|
|
{{- end }}
|
|
- name: OAUTH_END_SESSION
|
|
value: {{ $settings.oauth.endSession | quote }}
|
|
{{- end }}
|
|
# Both OFF against the image's defaults, and deliberately: a mail
|
|
# client should not phone home from a private cluster unless somebody
|
|
# chose it, and Renovate already watches the image tag, so the
|
|
# version check would answer outbound a question already answered
|
|
# inbound.
|
|
- name: BULWARK_TELEMETRY
|
|
value: {{ ternary "on" "off" $settings.telemetry | quote }}
|
|
- name: BULWARK_UPDATE_CHECK
|
|
value: {{ ternary "on" "off" $settings.updateCheck | quote }}
|
|
- name: LOG_LEVEL
|
|
value: {{ $settings.logLevel | quote }}
|
|
- name: LOG_FORMAT
|
|
value: {{ $settings.logFormat | quote }}
|
|
# PORT and HOSTNAME are already set in the image (3000 / 0.0.0.0);
|
|
# PORT is repeated here only because bulwark.port makes it
|
|
# configurable, and the containerPort below has to agree with it.
|
|
- name: PORT
|
|
value: {{ $settings.port | quote }}
|
|
{{- with $.Values.bulwark.env }}
|
|
{{- include "bulwark.env" (dict "root" $ "env" .) | nindent 12 }}
|
|
{{- end }}
|
|
ports:
|
|
# The name "http" is what the Service and all three probes reference,
|
|
# so renaming it here breaks four things at once.
|
|
- name: http
|
|
containerPort: {{ $settings.port | int64 }}
|
|
protocol: TCP
|
|
# Probes, resources and both security contexts go through the null-safe
|
|
# helpers (#99): an erased block means "chart defaults apply" instead of
|
|
# a vanished probe or an unhardened container. Never wrap these in an
|
|
# "if" - a guard hides the erased case completely instead of rendering a
|
|
# visible null.
|
|
#
|
|
# All three are an HTTP GET on /api/health - not HEAD, because a
|
|
# Kubernetes probe cannot send one (HTTPGetAction has no method field).
|
|
# What that endpoint can and cannot report is argued at the helpers and
|
|
# at bulwark.readinessProbe in values.yaml; the short version is that it
|
|
# carries the process, HTTP acceptance, a free event loop and Next.js
|
|
# routing - and NOTHING about the mail server.
|
|
livenessProbe:
|
|
{{- include "bulwark.livenessProbe" $.Values.bulwark.livenessProbe | nindent 12 }}
|
|
readinessProbe:
|
|
{{- include "bulwark.readinessProbe" $.Values.bulwark.readinessProbe | nindent 12 }}
|
|
startupProbe:
|
|
{{- include "bulwark.startupProbe" $.Values.bulwark.startupProbe | nindent 12 }}
|
|
resources:
|
|
{{- include "bulwark.effectiveResources" $.Values.bulwark.resources | nindent 12 }}
|
|
securityContext:
|
|
{{- include "bulwark.containerSecurityContext" $.Values.bulwark.securityContext | nindent 12 }}
|
|
volumeMounts:
|
|
# THE ONE PVC, carrying all four of Bulwark's directories as
|
|
# subdirectories so that a volume snapshot catches configuration,
|
|
# runtime state, audit log and user settings atomically.
|
|
- name: data
|
|
mountPath: {{ include "bulwark.dataDir" . }}
|
|
# THE TWO FILE-FORM SECRETS, AS A WHOLE DIRECTORY AND NOT WITH
|
|
# subPath. This is the line a later round will want to "tidy up" into
|
|
# two subPath mounts, and that would silently break secret rotation:
|
|
# the kubelet never refreshes a subPath mount, while both values are
|
|
# re-read from disk on every use (getSessionSecret and
|
|
# getGlobalClientSecret both call readFileEnv with no cache). With
|
|
# this mount a rotated vault field is live without a restart; with
|
|
# subPath the container would hold the first value forever and
|
|
# nothing would report it.
|
|
- name: secrets
|
|
mountPath: {{ include "bulwark.secretMountPath" . }}
|
|
readOnly: true
|
|
# The two writable paths outside the volume, needed because
|
|
# readOnlyRootFilesystem is true. Both are CACHES, not state - see
|
|
# bulwark.writableCache in values.yaml for how the set was
|
|
# established and why neither belongs on the PVC.
|
|
- name: tmp
|
|
mountPath: /tmp
|
|
- name: next-cache
|
|
mountPath: /app/.next/cache
|
|
restartPolicy: Always
|
|
volumes:
|
|
- name: data
|
|
persistentVolumeClaim:
|
|
claimName: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
|
|
- name: secrets
|
|
# One projected volume over the two Secrets the 1Password operator
|
|
# materializes, so both files sit in one directory and the container
|
|
# needs one mount instead of two.
|
|
#
|
|
# defaultMode 0440 - owner and GROUP read, and the group bit is the
|
|
# part that must not be "tightened" to 0400. Files in a secret volume
|
|
# are owned by ROOT, not by the container's user; what gives uid 1001
|
|
# access is fsGroup, which the kubelet applies by chowning the files to
|
|
# group 1001. Without a group read bit, uid 1001 could not read either
|
|
# file - and the failure would be silent, because readFileEnv answers
|
|
# an EACCES with null exactly as it answers a missing file: the
|
|
# settings sync would switch itself off and every token exchange would
|
|
# send an empty client secret.
|
|
#
|
|
# (The kubelet ORs 0440 onto the mode of a secret volume once fsGroup
|
|
# is set, so 0400 would in practice be corrected to 0440 anyway. It is
|
|
# written out because a mode that only works because something else
|
|
# silently fixes it is not a mode anyone should have to re-derive.)
|
|
#
|
|
# #81's permission trap does not apply in the other direction either:
|
|
# lib/read-file-env.ts is readFileSync plus try/catch with no mode
|
|
# check, so unlike garage, Bulwark does not REJECT a group-readable
|
|
# secret file. That is what makes the file form usable here at all.
|
|
projected:
|
|
defaultMode: 0440
|
|
sources:
|
|
- secret:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-app-secret
|
|
items:
|
|
# Only the session secret becomes a file. admin-password is
|
|
# in the same Secret but arrives as an environment variable
|
|
# (it has no _FILE form), so projecting it here as well would
|
|
# put it on disk for no reader.
|
|
- key: session-secret
|
|
path: session-secret
|
|
# optional: the 1Password operator fills this Secret in-cluster
|
|
# AFTER Helm has applied the manifests, so on a first install
|
|
# the Secret does not exist yet. Without this the pod would sit
|
|
# in ContainerCreating until the operator caught up, and with a
|
|
# misconfigured operator it would sit there forever with the
|
|
# reason only in the pod's events. It starts instead, and the
|
|
# missing secret surfaces where it is actionable: the settings
|
|
# sync stays off and the log says so.
|
|
optional: true
|
|
{{- if $settings.oauth.enabled }}
|
|
- secret:
|
|
name: {{ .Release.Name }}-{{ .Chart.Name }}-oauth-secret
|
|
items:
|
|
- key: oauth-client-secret
|
|
path: oauth-client-secret
|
|
optional: true
|
|
{{- end }}
|
|
- name: tmp
|
|
emptyDir:
|
|
sizeLimit: {{ $settings.writableCache.tmpSizeLimit }}
|
|
# Next.js's own cache directory in standalone mode - the optimized-image
|
|
# cache and the fetch cache. It is a SUBDIRECTORY of .next, which is what
|
|
# makes this possible at all: /app/.next/static holds the build output and
|
|
# has to stay readable, and mounting the cache path does not shadow it.
|
|
- name: next-cache
|
|
emptyDir:
|
|
sizeLimit: {{ $settings.writableCache.nextCacheSizeLimit }}
|