jlab-cluster/helm-charts
3
0 Fork 0
Code Issues 15 Pull Requests 3 Actions Packages Wiki Activity

feat(bulwark): new chart for Bulwark Webmail (1.0.0+up1.12.0)
Validate Helm Charts / Detect changed charts (pull_request) Successful in 3s
Details
Validate Helm Charts / Resource quantities (pull_request) Successful in 13s
Details
Validate Helm Charts / Build bulwark (pull_request) Successful in 6s
Details
Validate Helm Charts / Trivy config bulwark (pull_request) Successful in 10s
Details
Install-test / Install-test bulwark (pull_request) Successful in 8m50s
Details
Validate Helm Charts / Install-test (pull_request) Successful in 8m50s
Details

Browse Source 
A JMAP webmail client for the stalwart chart (Gitea #65): StatefulSet, one PVC
at /app/data, Service plus headless Service, Ingress at a host root, two
OnePasswordItems, three probes, computed limits, replicas plus scaleDown, strict
values.schema.json.

Everything below was measured against the source at tag 1.12.0 rather than read
out of the documentation, which was wrong about three of them.

THE DEFINING PROPERTY: THE VOLUME BEATS THE CHART, FOR EVERY VALUE, SILENTLY.
configManager.get resolves "admin override > env var > default", the admin
dashboard may write any key of CONFIG_ENV_MAP, and nothing reports that it just
disconnected the chart - so a helm upgrade can succeed and change nothing.
Upstream documents this for the server URL only; it is true of everything.
Exactly three values escape it (SESSION_SECRET(_FILE), ADMIN_PASSWORD,
BULWARK_TELEMETRY). The value that closes it, bulwark.adminConfigReadOnly,
defaults to false because its price is threefold: it freezes the settings policy
(and with it the only way to move Web Push off the vendor relay), it freezes an
override that ALREADY exists instead of removing it, and on a fresh install it
stops admin.json from being written at all - so the admin dashboard never comes
up and its login answers HTTP 500 while the pod stays Ready.

OIDC-only sign-in, with the real way back out of a lockout: not helm upgrade
with OAUTH_ONLY=false (oauthOnly goes through the same precedence chain) but
removing the override in the admin dashboard. That is why ADMIN_PASSWORD is set
- the dashboard is the door. It generates no random password and logs nothing,
so the #65 analogy does not apply.

SESSION_SECRET_FILE and OAUTH_CLIENT_SECRET_FILE are both used - the better path
garage (#81) and keycloak (#83) did not have - in one projected volume mounted
WITHOUT subPath, because both are re-read on every use and a subPath mount is
never refreshed, and at mode 0440 because secret files are owned by root and
fsGroup only supplies the group.

Two backup statements, both at the value: without the session secret the
synchronized settings are unreadable out of an intact volume, and if the
dashboard's OAuth setup was ever used the client secret sits in config.json on
that volume in cleartext.

Probes: all three a plain GET on /api/health, with the limit written at the
readiness value - it carries process, HTTP acceptance, a free event loop and
Next.js routing, and nothing about the reported state or about Stalwart.

No ConfigMap, so the checksum/config rule (#82) has nothing to attach to, and
values.yaml says so explicitly; the op:// paths are covered by
checksum/secret-ref.

Renovate: no versioningTemplate, measured rather than assumed. 315 tags in eight
shapes (releases, a second v-prefixed family of the same images, an -always
flavour, floating minors and names, commit SHAs, betas, attestations) and plain
docker versioning excludes all six unwanted ones. Dry runs at 1.12.0, 1.11.0 and
1.10.0 are recorded in the block.

Part of #89 - deliberately not "Fixes", because one binding decision of the
issue was changed out of necessity and wants ratification rather than an
automatic close: the plan specifies HEAD /api/health for all three probes, and a
Kubernetes probe cannot send HEAD at all (HTTPGetAction has no method field; the
kubelet always issues GET). The alternative, an exec probe, would mean starting a
Node process every few seconds in an image that carries no curl and no wget. GET
is used instead and is marginally more capable than HEAD, so every limit the
issue argues is unchanged - but the decision is the maintainer's to confirm.
This commit is contained in:
jklein committed 2026-10-05 21:19:56 +00:00
1 parent 28567842f3
commit 75f2f31e60
17 files changed
+2719 -1
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

bulwark/templates/NOTES.txt
+252
View File
@@ -0,0 +1,252 @@
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
{{- $replicas := int (include "bulwark.replicas" (dict "scaleDown" .Values.scaleDown "replicas" $settings.replicas)) -}}
{{- $sfs := printf "%s-%s-sfs" .Release.Name .Chart.Name -}}
{{- $svc := printf "%s-%s-service" .Release.Name .Chart.Name -}}
{{- $url := include "bulwark.publicUrl" . -}}
Bulwark Webmail {{ .Chart.AppVersion }} installed as release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
{{ $replicas }} replica(s). This is a CLIENT: it holds no mail and has no database.
The messages and the accounts belong to the Stalwart server at
{{ $settings.jmapServerUrl }}
If that server is away, this release is still Ready and still green - see
"WHAT THE PROBES DO NOT TELL YOU" below.
ENDPOINTS
{{ if .Values.ingress.enabled }} Web: {{ $url }}
{{ else }} Web: no Ingress (ingress.enabled: false) - in-cluster only
{{ end }} In-cluster: http://{{ $svc }}.{{ .Release.Namespace }}.svc.cluster.local:{{ $settings.port | int64 }}
{{ if .Values.ingress.enabled }} Admin: {{ $url }}/admin/login
{{ else }} Admin: /admin/login on the in-cluster address above. With no Ingress, reach it with
kubectl -n {{ .Release.Namespace }} port-forward statefulset/{{ $sfs }} 3000:{{ $settings.port | int64 }}
and open http://localhost:3000/admin/login
{{ end }}{{ if .Values.ingress.enabled }}
Bulwark is served at the ROOT of that host and cannot be moved under a path:
NEXT_PUBLIC_BASE_PATH is compiled into the image.
{{ end }}
THE 1PASSWORD ITEMS, AND WHICH FIELDS THEY MUST CARRY
The operator materializes one Secret key per item FIELD. A missing or misnamed
field here does NOT stop the pod - the two file-form secrets simply read as
absent - so the symptom is a feature that is quietly off rather than an error.
{{ .Values.secrets.appSecretRef | default "<secrets.appSecretRef NOT SET>" }}
session-secret openssl rand -base64 32
MANDATORY. Persistent sessions and the encryption of the
synchronized settings both hang on it, and the second
fails SILENTLY without it: settings sync is computed as
"enabled AND a secret exists", so with no secret it is
simply off and nothing says so.
admin-password the admin dashboard's initial password. CONSUMED ONCE -
see step 5 below.
{{ if $settings.oauth.enabled }}
{{ .Values.secrets.oauthSecretRef | default "<secrets.oauthSecretRef NOT SET>" }}
oauth-client-secret the confidential OIDC client's secret, as issued by
the identity provider when the client was registered.
{{ else }}
(no OAuth item - bulwark.oauth.enabled is false, so sign-in is by password)
{{ end }}
The file-form secret{{ if $settings.oauth.enabled }}s are{{ else }} is{{ end }} mounted as a DIRECTORY, not with subPath, and
re-read on every use - so rotating {{ if $settings.oauth.enabled }}either field{{ else }}that field{{ end }} in the vault takes effect
WITHOUT a restart. admin-password is the exception and no restart helps it.
{{ if $settings.oauth.enabled }}
BEFORE YOU INSTALL - THE OIDC CLIENT HAS TO EXIST FIRST
{{ if $settings.oauth.only }}
THIS IS NOT ADVICE AND IT IS NOT A FOLLOW-UP STEP. bulwark.oauth.only is ON,
which means the password form is hidden, which means that WITHOUT A WORKING
OIDC CLIENT THE FIRST SIGN-IN IS IMPOSSIBLE. There is no fallback for users.
{{ end }}
Register a CONFIDENTIAL client with the provider at
{{ $settings.oauth.issuerUrl }}
client id: {{ $settings.oauth.clientId }}
secret: into the vault item's oauth-client-secret field
THE REDIRECT URI CARRIES THE UI LANGUAGE, AND THAT IS THE ONE THING THAT WILL
COST YOU THE FIRST SIGN-IN IF YOU REGISTER A SINGLE URI. Bulwark builds it as
{{ $url }}/<locale>/auth/callback
where <locale> is the language the VISITOR'S BROWSER resolved to - so a German
browser sends /de/auth/callback and an English one /en/auth/callback, from the
same deployment. The language is not in the visible URL (the locale prefix is
"never" in this image), which is exactly why this is surprising: the address
bar shows no language and the redirect_uri contains one.
Register ONE PER LANGUAGE YOU EXPECT, or a wildcard if the provider allows it.
On Keycloak the wildcard is the practical answer:
{{ $url }}/*
Otherwise the 27 locales Bulwark supports are:
ar ca cs da de en es fa fr he hu it ja ko lv mn nb nl pl pt ro ru sk tr uk
zh zh-TW
If a sign-in fails with a redirect_uri mismatch, read the URI out of the
failed request rather than guessing which language it was - that is the value
the provider compares byte for byte.
POST-LOGOUT, IF YOU USE IT: bulwark.oauth.endSession is {{ $settings.oauth.endSession }}, and a
post-logout redirect URI is only sent when OAUTH_POST_LOGOUT_REDIRECT_URI is
set (this chart does not render it). If you add it through bulwark.env, it has
to be registered with the provider too.
The issuer must be reachable BY THE POD, and the endpoints it advertises must
share the issuer's origin or be publicly routable - Bulwark rejects a
discovered endpoint on a private address from a different origin. A
split-horizon DNS setup needs OAUTH_ALLOW_PRIVATE_ENDPOINTS through
bulwark.env; this chart does not render it.
{{ end }}
FIRST INSTALL
1. kubectl -n {{ .Release.Namespace }} rollout status statefulset/{{ $sfs }}
Ready means the HTTP server is accepting. It does NOT mean the mail server
is reachable - nothing in this chart checks that.
2. Confirm the setup wizard really is OFF. It is, as long as
bulwark.jmapServerUrl is set, and the chart refuses to render without it -
but this is the one thing worth seeing rather than trusting, because a
wizard that ran once changes the rules for good (step 4):
kubectl -n {{ .Release.Namespace }} logs statefulset/{{ $sfs }} | grep -i "SETUP REQUIRED"
Nothing found is the correct result. A "SETUP REQUIRED" banner with a token
means Bulwark is in bootstrap mode and something is wrong with the value.
3. Sign in{{ if $settings.oauth.only }} through the identity provider{{ end }} at {{ $url }}
4. READ THIS BEFORE YOU OPEN THE ADMIN DASHBOARD. The dashboard writes
OVERRIDES into config.json on the volume, and an override BEATS this chart
for that key - for good, and silently.
Priority is: admin override > environment variable > default.
So if you save a server URL in the dashboard, changing bulwark.jmapServerUrl
and running helm upgrade afterwards does nothing and reports success. The
same is true of every other key the dashboard can write. What the chart
sets is a DEFAULT, not a setting.
To see which keys are currently overridden:
kubectl -n {{ .Release.Namespace }} exec {{ $sfs }}-0 -- cat /app/data/admin/config.json
An empty object, or a file that does not exist, means this chart is in
force.
5. THE ADMIN PASSWORD IS CONSUMED EXACTLY ONCE. On the first start it is
hashed into /app/data/admin/admin.json and the environment variable is
never read again. Rotating the vault field afterwards has NO EFFECT -
changing the password goes through the dashboard, or by deleting admin.json
from the volume and restarting. Know that before you treat the vault field
as the source of truth.
IF NOBODY CAN SIGN IN - THE WAY BACK, AND IT IS NOT A helm upgrade
Read this now, not when it happens, because the obvious repair does not work.
The situation: the identity provider is down, or its configuration is wrong,
and bulwark.oauth.only hides the password form. No user can get in.
WHAT DOES NOT WORK RELIABLY: helm upgrade with bulwark.oauth.only=false.
oauthOnly is an ordinary config key, and every part of the app reads it
through the priority chain above - not from the environment. If the admin
dashboard's OAuth setup has ever been used, "oauthOnly": true is sitting in
config.json on the volume, and from then on the chart's value is ignored. The
upgrade succeeds and changes nothing.
WHAT DOES WORK, and it does not depend on the identity provider:
a) Open {{ $url }}/admin/login and sign in with the admin-password field
from the vault item. This door has its own password and is not behind
OIDC.{{ if eq $settings.stalwartAdminAccess "auto" }} (bulwark.stalwartAdminAccess is "auto", so a Stalwart
admin also gets in here without that password.){{ end }}
b) In the dashboard, REVERT the override - the dashboard's own "remove
override" on the OAuth settings. That is what puts the chart's value back
in charge.
IF THE DASHBOARD ITSELF IS UNREACHABLE, the last resort is the volume:
kubectl -n {{ .Release.Namespace }} exec {{ $sfs }}-0 -- cat /app/data/admin/config.json
# remove the "oauthOnly" key (and any other override in the way), then
kubectl -n {{ .Release.Namespace }} rollout restart statefulset/{{ $sfs }}
Deleting admin.json in the same directory resets the admin password to
whatever the vault field currently holds, on the next start.
WHAT THE PROBES DO NOT TELL YOU
All three probes are a plain HTTP GET on /api/health, and that endpoint returns
200 whatever state the application is in - for an unauthenticated caller the
body even says "healthy" unconditionally. So a green pod here means: the
process is alive, HTTP is being accepted, the event loop is free, and Next.js
can route.
IT SAYS NOTHING ABOUT THE MAIL SERVER. With Stalwart completely gone, this
release is Ready, green, and serves a web app that cannot fetch a single
message. Monitor the stalwart release for that - there is no endpoint here
that could cover it, and the one that would (/api/setup/test-jmap) sits behind
the setup token this chart disables.
It also says nothing about the reported state: /api/health reports "degraded"
on high V8 heap usage, but only to an authenticated caller asking for details,
and always with HTTP 200. A memory limit set too low therefore produces a pod
that is permanently degraded and permanently Ready.
BACKUPS - THE VOLUME ALONE IS NOT ONE, AND IT MAY NEED PROTECTING ITSELF
Two statements, both of them about {{ .Release.Name }}-{{ .Chart.Name }}-pvc:
1. A BACKUP OF THE VOLUME ALONE IS NOT A BACKUP.{{ if $settings.settingsSync }} The synchronized user
settings on it are encrypted with session-secret.{{ else }} The user settings are not
synchronized in this release, but persistent sessions still are.{{ end }} Lose the vault
item and that data is unreadable out of a perfectly intact volume. Back up
the 1Password item together with the volume, or the restore is a directory
of ciphertext.
2. A BACKUP OF THE VOLUME MAY CONTAIN A SECRET IN CLEARTEXT. If anybody has
used the dashboard's OAuth setup, the OIDC client secret is written as
PLAINTEXT into config.json on this volume. So the backup may need the same
protection the vault gets. Check with:
kubectl -n {{ .Release.Namespace }} exec {{ $sfs }}-0 -- grep -l oauthClientSecret /app/data/admin/config.json
TRAPS WORTH KNOWING BEFORE YOU NEED THEM
- WEB PUSH GOES THROUGH THE VENDOR'S RELAY until somebody changes it, and the
change is NOT a chart value. The built-in default is
https://notifications.relay.bulwarkmail.org and it is compiled into the
image. A self-hosted relay CAN be made the effective default and even
mandatory for all users - through the settings policy in the admin
dashboard, because that setting has no environment variable at all. So this
is a choice you have, not a limitation you are stuck with; it just lives in
the dashboard.
- THERE IS NO ConfigMap IN THIS CHART, so no checksum/config annotation, and
that is not an omission: there is no rendered config file whose change could
fail to restart anything. Repointing secrets.* at a different vault item
DOES roll the pod (checksum/secret-ref). Changing a value INSIDE an item
does not - but for the two file-form secrets it does not need to, because
they are re-read on every use.
{{ if $settings.adminConfigReadOnly }}
- bulwark.adminConfigReadOnly IS ON. The chart is authoritative and no
override can appear - and three things are frozen with it: the settings
policy (so Web Push cannot be moved to your own relay any more), any
override that was ALREADY on the volume (it cannot be removed either), and -
if this release has never started with it off - the admin password, which
means the admin dashboard never comes up and its login answers HTTP 500
while this pod stays Ready.
If you have just enabled this on a fresh install, turn it off, let the pod
start once so admin.json is written, and only then turn it back on.
{{ else }}
- bulwark.adminConfigReadOnly is OFF, which is the default and keeps the
dashboard working. The cost is the precedence trap of step 4: the volume can
override this chart at any time. Turning it on makes the chart authoritative
but freezes the settings policy, freezes any override already present, and
must never be the state of a FIRST install - see values.yaml for the
sequence.
{{ end }}
- THE SETTINGS POLICY, THE THEMES AND THE BRANDING ARE NOT IN THIS CHART.
They live in policy.json on the volume and are set in the dashboard. This
chart neither renders nor backs them up beyond the volume itself.
WHAT THIS CHART DOES NOT RENDER
- No ConfigMap (see above) and no NetworkPolicy. The pod needs egress to the
JMAP server and to the OIDC issuer; a policy added later has to allow both,
or sign-in breaks in a way that looks like a provider outage.
- No PodDisruptionBudget: with one replica a minAvailable of 1 would forbid
every voluntary eviction and make `kubectl drain` hang forever on a pod
nothing may move (README, guideline 11).
- No sub-path Ingress, and no value that could add one - the base path is
compiled into the image.
- No proxy-body-size annotation. Attachment uploads go through the Ingress and
nginx's default of 1m rejects larger ones with a 413 that reads like an
application fault; the right number is the mail server's limit, which this
chart does not know. Set it in ingress.annotations.
bulwark/templates/_helpers.tpl
+464
View File
@@ -0,0 +1,464 @@
{{/*
Render the body of a container "resources:" block from a values dict of the form
resources:
limitFactor: 3 # optional, default 3
requests: { cpu, memory, ephemeral-storage / ephemeralStorage, ... }
limits: { ... } # optional, every entry optional
Rules (variant B, see issue #19):
- requests are rendered as given (camelCase "ephemeralStorage" is normalized
to the Kubernetes key "ephemeral-storage").
- an explicitly set limit always wins.
- a missing memory / ephemeral-storage limit is computed as
limitFactor x the corresponding request.
- a missing cpu limit is NOT set at all (the kubelet then treats CPU as
unlimited; CPU is compressible, so this cannot kill the workload).
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
the chart's name as define prefix, because sharing it would mean a library
chart, and a dependency is admissible but expensive (README, guideline 14) -
this chart takes none. The reference implementation is
template-chart/templates/_helpers.tpl.
*/}}
{{- define "bulwark.resources" -}}
{{- $resources := . | default dict -}}
{{- $factor := 3.0 -}}
{{- if not (kindIs "invalid" $resources.limitFactor) -}}
{{- $factor = float64 $resources.limitFactor -}}
{{- end -}}
{{- $requests := dict -}}
{{- range $key, $value := ($resources.requests | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $requests (include "bulwark.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- $limits := dict -}}
{{- range $key, $value := ($resources.limits | default dict) -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $_ := set $limits (include "bulwark.resourceKey" $key) $value -}}
{{- end -}}
{{- end -}}
{{- range $key, $request := $requests -}}
{{- if and (ne $key "cpu") (not (hasKey $limits $key)) -}}
{{- $_ := set $limits $key (include "bulwark.multiplyQuantity" (dict "quantity" $request "factor" $factor)) -}}
{{- end -}}
{{- end -}}
{{- $out := dict -}}
{{- with $requests }}{{- $_ := set $out "requests" . }}{{- end -}}
{{- with $limits }}{{- $_ := set $out "limits" . }}{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Normalize a resource name from values to its Kubernetes key.
*/}}
{{- define "bulwark.resourceKey" -}}
{{- if eq . "ephemeralStorage" -}}ephemeral-storage{{- else -}}{{ . }}{{- end -}}
{{- end -}}
{{/*
Multiply a Kubernetes quantity by a factor, keeping its suffix.
Input: dict "quantity" <string|number> "factor" <float64>.
*/}}
{{- define "bulwark.multiplyQuantity" -}}
{{- $quantity := include "bulwark.formatNumber" .quantity -}}
{{- $number := regexFind "^[0-9]+(\\.[0-9]+)?" $quantity -}}
{{- if not $number -}}
{{- fail (printf "cannot parse resource quantity %q" $quantity) -}}
{{- end -}}
{{- $suffix := trimPrefix $number $quantity -}}
{{- if not (regexMatch "^(m|k|M|G|T|P|E|Ki|Mi|Gi|Ti|Pi|Ei)?$" $suffix) -}}
{{- fail (printf "unsupported resource quantity suffix %q in %q" $suffix $quantity) -}}
{{- end -}}
{{- $product := mulf (float64 $number) .factor -}}
{{- if eq $product (floor $product) -}}
{{- printf "%d%s" (int64 $product) $suffix -}}
{{- else -}}
{{- printf "%s%s" (include "bulwark.formatNumber" $product) $suffix -}}
{{- end -}}
{{- end -}}
{{/*
Render a number as a plain decimal string, never in exponential notation
(issue #157): the null-safe fallback helpers pass values through fromYaml,
which parses every number as float64, and Go's "%v" switches float64 to
exponential notation from about 1e6 upwards - "1.048576e+06" is not a valid
Kubernetes quantity. Strings ("512Mi") are passed through untouched.
*/}}
{{- define "bulwark.formatNumber" -}}
{{- if kindIs "float64" . -}}
{{- regexReplaceAll "\\.?0+$" (printf "%.10f" .) "" -}}
{{- else -}}
{{- printf "%v" . -}}
{{- end -}}
{{- end -}}
{{/*
Render the replica count for the StatefulSet.
Input: dict "scaleDown" <flag|nil> "replicas" <value|nil>.
- scaleDown: true takes precedence and always yields 0.
- otherwise an explicitly set replicas value is used as given, including an
explicit 0 - Helm's "default" would swallow 0, hence the kindIs "invalid"
nil check.
- a missing replicas value defaults to 1.
A NOTE SPECIFIC TO THIS CHART: the schema caps this at 1, and the reason is the
single ReadWriteOnce PVC rather than anything about the application. Two pods
cannot both mount it, so a second replica would sit in ContainerCreating
forever - and the PVC is not optional here, because /app/data carries the admin
configuration, the audit log and the synchronized user settings. An RWX volume
would not make it a cluster either: nothing in Bulwark coordinates two
processes over that directory.
*/}}
{{- define "bulwark.replicas" -}}
{{- if .scaleDown -}}
0
{{- else if not (kindIs "invalid" .replicas) -}}
{{- .replicas -}}
{{- else -}}
1
{{- end -}}
{{- end -}}
{{/*
Merge a user-supplied values sub-tree onto a dict of chart defaults so that a
null / empty override falls back to the defaults instead of erasing them
(issues #61, #99). Input: dict "defaults" <dict> "given" <dict|nil> "path"
<string>. Output: YAML of the merged dict (consume it with fromYaml).
An override key written without children is YAML null, and Helm treats that
null as a deletion of the chart's default. This helper defines the opposite
semantics - an empty block means "chart defaults apply" - while an explicitly
set value still wins, including false and 0.
*/}}
{{- define "bulwark.defaultedDict" -}}
{{- $defaults := .defaults | default dict -}}
{{- $given := .given -}}
{{- $path := .path | default "value" -}}
{{- if kindIs "invalid" $given -}}
{{- $given = dict -}}
{{- end -}}
{{- if not (kindIs "map" $given) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" $path (kindOf $given) $given) -}}
{{- end -}}
{{- $out := deepCopy $defaults -}}
{{- range $key, $value := $given -}}
{{- if not (kindIs "invalid" $value) -}}
{{- $default := index $defaults $key -}}
{{- if and (kindIs "map" $value) (kindIs "map" $default) -}}
{{- $_ := set $out $key (fromYaml (include "bulwark.defaultedDict" (dict "defaults" $default "given" $value "path" (printf "%s.%s" $path $key)))) -}}
{{- else if kindIs "map" $default -}}
{{- fail (printf "%s.%s must be a mapping or null, got %s (%v)" $path $key (kindOf $value) $value) -}}
{{- else -}}
{{- $_ := set $out $key $value -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- toYaml $out -}}
{{- end -}}
{{/*
Pick one sub-block out of a values block that may itself be null.
Input: dict "block" <dict|nil> "key" <string> "path" <string>.
Returns YAML of dict "value" <the sub-block, may be nil>.
*/}}
{{- define "bulwark.subBlock" -}}
{{- $block := .block -}}
{{- if kindIs "invalid" $block -}}
{{- $block = dict -}}
{{- end -}}
{{- if not (kindIs "map" $block) -}}
{{- fail (printf "%s must be a mapping or null, got %s (%v)" .path (kindOf $block) $block) -}}
{{- end -}}
{{- toYaml (dict "value" (index $block .key)) -}}
{{- end -}}
{{/*
THE EFFECTIVE BULWARK SETTINGS - everything the StatefulSet, the Service and
NOTES.txt read out of .Values.bulwark, merged onto this chart's defaults.
Why the whole settings block goes through the null-safe helper and not only the
three usual suspects (guideline 12.2: "the rule is about the shape, not about
those three block kinds"): every value below is dereferenced by a template, and
an erased one would not abort the render. An erased bulwark.port renders an
empty containerPort and an empty Service targetPort - rejected by nothing until
the API server sees it; an erased bulwark.oauth.enabled would silently turn OIDC
OFF on a release whose whole login path is OIDC, which is the quiet half and the
worse one. Satisfactory shipped exactly this shape with half its blocks
null-safe and half not (Gitea #16).
The defaults here MUST stay in sync with values.yaml, which remains the
documented, user-facing place for them; this copy is only the fallback.
Usage: {{ $settings := fromYaml (include "bulwark.settings" .) }}
*/}}
{{- define "bulwark.settings" -}}
{{- $given := (fromYaml (include "bulwark.subBlock" (dict "block" .Values "key" "bulwark" "path" ".Values"))).value -}}
{{- $defaults := dict
"replicas" 1
"port" 3000
"jmapServerUrl" nil
"appName" nil
"stalwartFeatures" true
"stalwartAdminAccess" "auto"
"settingsSync" true
"adminConfigReadOnly" false
"telemetry" false
"updateCheck" false
"logLevel" "info"
"logFormat" "json"
"writableCache" (dict
"tmpSizeLimit" "256Mi"
"nextCacheSizeLimit" "512Mi")
"oauth" (dict
"enabled" true
"only" true
"clientId" nil
"issuerUrl" nil
"scopes" nil
"endSession" true) -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" $given "path" "bulwark") -}}
{{- end -}}
{{/*
THE DATA DIRECTORY, defined once because two templates have to agree on it: the
StatefulSet mounts the PVC here and NOTES.txt names it.
/app/data, and it is NOT a value. Bulwark resolves its four directories
relative to the process working directory - getConfigDir() falls back to
<cwd>/data/admin and getStateDir() to <cwd>/data/admin-state (lib/admin/paths.ts
at tag 1.12.0) - and cwd is /app (the image's WORKDIR). A path that disagreed
with the mount would put the admin configuration and the audit log into the
container's ephemeral layer, which looks like it works until the pod is
replaced. The image creates all four subdirectories and chowns them to
1001:1001 (Dockerfile: mkdir -p /app/data/{settings,admin,admin-state,telemetry}
&& chown -R nextjs:nodejs /app/data) - but that happens in the IMAGE LAYER,
which the PVC mount covers, so the subdirectories are created again at runtime
by Bulwark itself (ensureConfigDir / ensureStateDir, both mkdir -p). fsGroup is
what makes the empty volume writable for uid 1001.
*/}}
{{- define "bulwark.dataDir" -}}/app/data{{- end -}}
{{/*
WHERE THE TWO FILE-FORM SECRETS ARE MOUNTED, and the one property of this mount
that must not be "tidied up": it is a WHOLE-DIRECTORY mount with NO subPath.
The kubelet never refreshes a subPath mount. Both files are re-read on every
use - getSessionSecret() calls readFileEnv on every call
(lib/auth/session-secret.ts) and getGlobalClientSecret() does the same on every
token exchange (lib/oauth/token-exchange.ts, no cache) - so rotating a field
inside the vault item takes effect WITHOUT a pod restart. With a subPath that
property is silently lost: the file in the container would keep the old value
forever while the vault, the Secret and `helm upgrade` all look correct.
*/}}
{{- define "bulwark.secretMountPath" -}}/etc/bulwark/secrets{{- end -}}
{{/*
The effective security contexts, probes and resources (issue #99).
The defaults below MUST stay in sync with values.yaml. They reproduce THIS
chart's defaults, not the repo standard.
*/}}
{{- define "bulwark.podSecurityContext" -}}
{{- $given := (fromYaml (include "bulwark.subBlock" (dict "block" . "key" "pod" "path" "bulwark.securityContext"))).value -}}
{{- $defaults := dict
"runAsNonRoot" true
"runAsUser" 1001
"runAsGroup" 1001
"fsGroup" 1001
"fsGroupChangePolicy" "Always"
"seccompProfile" (dict "type" "RuntimeDefault") -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" $given "path" "bulwark.securityContext.pod") -}}
{{- end -}}
{{- define "bulwark.containerSecurityContext" -}}
{{- $given := (fromYaml (include "bulwark.subBlock" (dict "block" . "key" "container" "path" "bulwark.securityContext"))).value -}}
{{- $defaults := dict
"allowPrivilegeEscalation" false
"readOnlyRootFilesystem" true
"runAsNonRoot" true
"capabilities" (dict "drop" (list "ALL") "add" (list)) -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" $given "path" "bulwark.securityContext.container") -}}
{{- end -}}
{{/*
ALL THREE PROBES ARE AN HTTP GET ON /api/health, AND WHAT THAT DOES AND DOES NOT
CARRY IS MEASURED AGAINST THE SOURCE AT TAG 1.12.0 - not read off the path name,
which is the mistake #218 and Gitea #83 each cost this repo once.
FIRST, WHY GET AND NOT HEAD, because the plan for this chart (Gitea #89) said
HEAD and HEAD IS NOT AVAILABLE TO A KUBERNETES PROBE AT ALL. The API type is
HTTPGetAction and it has no method field; the kubelet's prober issues a GET and
there is no value anywhere in a pod spec that changes that. The only way to send
a HEAD would be an exec probe shelling out to a HTTP client - and this image
contains no curl and no wget (npm and npx are deliberately removed from the
runtime layer), so it would mean `node -e` on every probe tick: a new Node
process every few seconds against an endpoint whose GET handler is already
cheap. That is strictly worse, so the chart uses the GET that Kubernetes
actually offers.
WHAT CHANGES BECAUSE OF THAT: almost nothing, and in the direction that helps.
Upstream's HEAD handler returns `new Response(null, { status: 200 })` and
computes nothing. The GET handler does gather diagnostics, which makes it very
slightly more capable than HEAD rather than less: its catch block answers 503
when `process.memoryUsage()` or `v8.getHeapStatistics()` itself throws. Every
limitation argued below is unchanged.
WHAT THE ENDPOINT DOES. The GET success path returns a hard `status: 200`
whatever state it computed, and for an UNAUTHENTICATED caller the body is
hard-coded to `healthy` as well (`status: detailed ? status : 'healthy'`);
`degraded` appears only with `?detailed=true` AND a caller that passes
canSeeInstanceDetails. Upstream tests exactly that -
lib/__tests__/health-route.test.ts expects HTTP 200 while `payload.status` is
`degraded`. A probe is unauthenticated and asks for no details, so it sees 200
and "healthy" always.
SO THE PROBE CANNOT REPORT THE APPLICATION'S STATE - and this is deliberately
NOT filed as "a probe that cannot fail", because that would be wrong and would
invite somebody to replace it with something worse. What a 200 here really
proves, all of it load-bearing: the Node process is alive, the HTTP server is
accepting, the event loop is not blocked, and Next.js can route a request. An
OOM kill, a crash, a wedged event loop and a container that never finished
booting all fail this probe. That is more than #218's endpoint carried and it is
exactly what a liveness and a startup probe are for.
WHAT IT DOES NOT CARRY, and no probe in this chart can: anything about
STALWART. Bulwark holds no mail and this endpoint does not ask the JMAP server
anything, so with the mail server gone /api/health is green and the pod stays
Ready. There is no better endpoint - /api/setup/test-jmap would really test the
connection, but it lives behind the bootstrap setup token and is unreachable in
this chart (the wizard is disabled by jmapServerUrl). The limit is written at
the readiness value in values.yaml because readiness is the probe whose promise
it weakens.
*/}}
{{- define "bulwark.livenessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/api/health" "port" "http" "scheme" "HTTP")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.livenessProbe") -}}
{{- end -}}
{{- define "bulwark.readinessProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/api/health" "port" "http" "scheme" "HTTP")
"periodSeconds" 10
"timeoutSeconds" 5
"failureThreshold" 3 -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.readinessProbe") -}}
{{- end -}}
{{/*
STARTUP: the same endpoint, and it gates the other two - which is why neither of
them carries an initialDelaySeconds.
BUDGET = periodSeconds x failureThreshold = 5 x 24 = 120s. What it has to cover
is not Bulwark's own boot, which is a Next.js standalone server starting in
about a second: it is attaching and mounting the network volume, and the
instrumentation hook that runs BEFORE the first request is served -
instrumentation.node.ts migrates the legacy admin layout, loads config.json and
policy.json off the volume, and on a first install hashes ADMIN_PASSWORD with
scrypt and writes admin.json. The scrypt hash is the one deliberately slow step
in there.
*/}}
{{- define "bulwark.startupProbe" -}}
{{- $defaults := dict
"httpGet" (dict "path" "/api/health" "port" "http" "scheme" "HTTP")
"periodSeconds" 5
"timeoutSeconds" 5
"failureThreshold" 24 -}}
{{- include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.startupProbe") -}}
{{- end -}}
{{/*
The effective resources block, fed into the resources helper above so that an
erased "resources:" still yields the chart's requests and computed limits.
*/}}
{{- define "bulwark.effectiveResources" -}}
{{- $defaults := dict
"limitFactor" 3
"requests" (dict "cpu" "100m" "memory" "256Mi" "ephemeral-storage" "64Mi") -}}
{{- $merged := fromYaml (include "bulwark.defaultedDict" (dict "defaults" $defaults "given" . "path" "bulwark.resources")) -}}
{{- include "bulwark.resources" $merged -}}
{{- end -}}
{{/*
The additive env block, rendered through tpl with the value, secretKeyRef and
configMapKeyRef forms (guideline 10).
ADDITIVE MEANS ADDITIVE: an entry repeating a name this chart already sets does
not replace it. At install time both reach the pod and the later one wins; on an
EXISTING release the pair collapses to the chart's own value, because Kubernetes
merges env by name - and helm upgrade still reports success. So do not use this
block to override JMAP_SERVER_URL, OAUTH_ONLY or SESSION_SECRET_FILE; change the
values that produce them.
AND THERE IS A SECOND, SHARPER REASON NOT TO USE IT FOR BULWARK'S OWN SETTINGS,
specific to this application: an environment variable is only the MIDDLE layer
of three. configManager.get resolves "admin override > env var > default"
(lib/admin/config-manager.ts), so any key the admin dashboard has written into
config.json on the PVC beats whatever arrives here - silently. An entry added
through this block is therefore not a guarantee, it is a default that the volume
may already be overriding. values.yaml, under bulwark.adminConfigReadOnly, has
the whole precedence story.
*/}}
{{- define "bulwark.env" -}}
{{- $root := .root -}}
{{- range $value := .env }}
{{- if $value.value }}
- name: {{ $value.name | quote }}
# quote is required, not cosmetic: a value like "3000" or "true" renders
# unquoted as a YAML number/bool, and the API server then rejects the whole
# manifest with "cannot unmarshal number into Go struct field EnvVar.value of
# type string".
value: {{ tpl $value.value $root | quote }}
{{- else if $value.valueFrom }}
{{- if $value.valueFrom.secretKeyRef }}
- name: {{ $value.name }}
valueFrom:
secretKeyRef:
name: {{ tpl $value.valueFrom.secretKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.secretKeyRef.key $root | quote }}
{{- else if $value.valueFrom.configMapKeyRef }}
- name: {{ $value.name }}
valueFrom:
configMapKeyRef:
name: {{ tpl $value.valueFrom.configMapKeyRef.name $root | quote }}
key: {{ tpl $value.valueFrom.configMapKeyRef.key $root | quote }}
{{- end }}
{{- end }}
{{- end }}
{{- end -}}
{{/*
The public base URL, derived from ingress.domain. NOTES.txt prints it and builds
the OIDC redirect URI from it, which is why it is a helper rather than several
copies of the same printf.
DELIBERATELY NO required() HERE, and that is the opposite of what it first looks
like it should have. The only caller is NOTES.txt, and `helm template` does not
render NOTES.txt (README, guideline 12.2) - so a required() in this helper would
be a guard that cannot fire in lint, in the render test or in CI, and would fire
for the first time during a real `helm install`. A guard that only runs in
production is worse than no guard.
The honest position for THIS chart, checked in both directions as guideline 12.1
asks: nothing outside the Ingress template FUNCTIONALLY reads ingress.domain.
Bulwark derives its own public URL from the request origin and takes no base-URL
variable, so with ingress.enabled: false the release is complete without a
domain - the calibre-web-automated case, where the gate frees both values. The
enforcement therefore sits entirely in bulwark.ingress.yaml, where it is reached
by every render that has the Ingress on (the default), and this helper renders a
visible placeholder instead of failing.
*/}}
{{- define "bulwark.publicUrl" -}}
{{- if .Values.ingress.domain -}}
https://{{ .Values.ingress.domain }}
{{- else -}}
https://<ingress.domain NOT SET>
{{- end -}}
{{- end -}}
bulwark/templates/bulwark.ingress.yaml
+67
View File
@@ -0,0 +1,67 @@
{{- /*
ON BY DEFAULT (ingress.enabled: true): a webmail client nobody can reach is not
a webmail client.
AT THE ROOT OF THE HOST, AND THERE IS NO VALUE TO PUT IT UNDER A PATH.
NEXT_PUBLIC_BASE_PATH is compiled into the bundle - upstream: "These are
compiled into the bundle and cannot be changed at runtime" - so a sub-path
deployment needs a rebuilt image, which is outside what this repository does
(same limit as garage #81 and keycloak #83). A rewrite-target annotation is not a
way around it either: the bundle's own asset and API URLs are built for the root,
so stripping a prefix at the proxy produces a page that loads and then fetches
everything from the wrong place. The rule below is therefore "/" with pathType
Prefix, deliberately and permanently.
GUIDELINE 1, CHECKED IN BOTH DIRECTIONS (README, "Values that must not lie"),
AND FOR THIS CHART THE GATE FREES BOTH VALUES. Checked rather than assumed:
nothing outside this template FUNCTIONALLY reads ingress.domain. Bulwark derives
its own public URL from the request origin and takes no base-URL environment
variable - there is no KC_HOSTNAME equivalent here - so a release with
ingress.enabled: false is complete without a domain. That is the
calibre-web-automated case rather than the matrix-synapse one.
The one other READER is NOTES.txt, which prints the access URL and the OIDC
redirect URI. That read is deliberately NOT guarded with a required(): `helm
template` does not render NOTES.txt, so a guard there would fire for the first
time during a real `helm install` and never in lint, in the render test or in CI.
The bulwark.publicUrl helper renders a visible placeholder instead, and the
enforcement lives here - in the template that is reached by every render with the
Ingress on, which is the default.
*/ -}}
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-ingress
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
annotations:
cert-manager.io/cluster-issuer: {{ required "A Cluster-Issuer must be provided (ingress.clusterIssuer)" .Values.ingress.clusterIssuer }}
{{- /* Merged, not replaced. The one annotation worth adding here for this
application is nginx.ingress.kubernetes.io/proxy-body-size: attachment
uploads and the Files view go through this Ingress, and nginx's
default of 1m rejects anything larger with a 413 that reads like an
application fault. The chart does not set it because the right number
is the mail server's limit, which this chart does not know. */}}
{{- with .Values.ingress.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
ingressClassName: {{ .Values.ingress.className }}
rules:
- host: {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
http:
paths:
- backend:
service:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
port:
# BY NAME, so bulwark.port and this file cannot drift apart.
name: http
path: /
pathType: Prefix
tls:
- hosts:
- {{ required "A Domain/Host must be provided (ingress.domain)" .Values.ingress.domain }}
secretName: tls-{{ .Release.Name }}-{{ .Chart.Name }}-ingress
{{- end }}
bulwark/templates/bulwark.pvc.yaml
+50
View File
@@ -0,0 +1,50 @@
{{- /*
ONE claim, mounted at /app/data, holding all FOUR of Bulwark's directories as
subdirectories of it:
admin config.json, policy.json, admin.json (the password hash),
plugin-config/, plugins/, themes/, branding uploads
admin-state admin-state.json (login timestamps), audit.log, the bootstrap
setup token
settings the encrypted per-account user settings
telemetry consent and instance id
ONE VOLUME AND NOT FOUR, because a snapshot is atomic per volume and never
across volumes - the durability lesson garage (#81) was built on. Split up, a
backup can capture an audit log that records logins the configuration does not
know about yet, or settings encrypted against a secret the configuration has
since been repointed from. There is no argument on the other side here: this is
a few hundred kilobytes of JSON plus an append-only log, not an object store.
Bulwark creates all four subdirectories itself on start (ensureConfigDir and
ensureStateDir, both mkdir -p). It has to: the image's own `mkdir -p
/app/data/...` ran in a layer that this mount covers. fsGroup is what makes the
freshly provisioned volume writable for uid 1001.
DELIBERATELY NOT null-SAFE: an erased size yields an invalid PVC that is
rejected at apply time, which is loud. A chart-side fallback would silently size
somebody's volume - and on an existing release it could not help anyway, because
a PVC cannot shrink. Same reasoning as garage's storage block and satisfactory's
documented exception (Gitea #39).
NEVER RENAMED. A renamed PVC makes Kubernetes provision a new, empty volume and
orphans the data - here that means admin.json, the only copy of the admin
password hash, and audit.log, which by its nature cannot be reconstructed.
*/ -}}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
accessModes:
# ReadWriteOnce, and the single replica follows from it rather than the other
# way round - see the bulwark.replicas helper.
- ReadWriteOnce
resources:
requests:
storage: {{ .Values.storage.amount }}
{{- if .Values.storage.storageClass }}
storageClassName: {{ .Values.storage.storageClass }}
{{- end }}
bulwark/templates/bulwark.secret.yaml
+109
View File
@@ -0,0 +1,109 @@
{{- /*
The 1Password items this chart materializes. TWO of them, split BY AUDIENCE
rather than by count:
<release>-bulwark-app-secret this release's own secrets - the session
secret and the admin password. Nobody else
knows or needs them.
<release>-bulwark-oauth-secret the OIDC client secret. NOT ours: the
identity provider (keycloak, #83) creates it
when the client is registered and rotates it
there, so it is the one of the three that a
second system also holds. A separate item is
what lets it be repointed and rotated without
touching the other two.
Same reasoning as garage's split between its RPC secret and its S3 key.
WHAT THE FIELD NAMES ARE AND WHY THEY ARE NOT FREE-FORM: the operator
materializes one Secret key per FIELD of the item, and this chart references
those keys by name - two as file paths in the StatefulSet's volumeMounts, one by
secretKeyRef. An item without the expected field leaves the container in
CreateContainerConfigError, or - worse for the file form - produces a container
that starts with a MISSING file, which readFileEnv answers with null rather than
an error. NOTES.txt lists every field for the installer.
TWO SECRETS, TWO DIFFERENT ROTATION STORIES, and the difference is a property of
the application rather than of this chart:
session-secret, oauth-client-secret FILE form. Re-read on every use with no
cache, so a rotation is live WITHOUT a
restart - see the whole-directory mount
in bulwark.sfs.yaml, which is what makes
that true.
admin-password CONSUMED ONCE. initAdminPassword hashes
it on the first start, writes admin.json
to the PVC, and never looks at the
variable again. Rotating the field after
that has NO effect, restart or not.
AND THE LIMIT THAT CANNOT BE CLOSED FOR EITHER: a change to the CONTENT of an
item restarts nothing, because Helm never sees a secret the 1Password operator
materializes and there is nothing to hash. What this chart does cover is the
op:// PATH, through checksum/secret-ref on the pod template. The operator's
auto-restart annotation is NOT set here and must not be: verified against
operator 1.12.0 it enumerates Deployments only, so on a StatefulSet it is a
silent no-op advertising a guarantee it does not provide (README, guideline 9).
*/ -}}
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-app-secret
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
# Must carry the fields: session-secret, admin-password
#
# session-secret: 32 bytes of randomness, base64 - `openssl rand -base64 32`.
# MANDATORY. Persistent "remember me" sessions and the encryption of the
# synchronized user settings both hang on it, and the second one fails
# SILENTLY without it: app/api/config/route.ts computes
# `settingsSyncEnabled: configManager.get(...) && hasSessionSecret()`, so
# bulwark.settingsSync: true plus a missing secret means synchronization is
# simply off, with no error anywhere.
#
# LOSING IT LOSES DATA THAT AN INTACT VOLUME STILL HOLDS: the settings on the
# PVC are encrypted with it. This item belongs in the backup next to the
# volume - a backup of the volume alone is not a backup.
#
# admin-password: the initial password of the ADMIN DASHBOARD.
# It is set because the dashboard is the way back out of an OIDC lockout
# (see bulwark.oauth.only in values.yaml), not because of any secret leak -
# measured, there is none: without ADMIN_PASSWORD Bulwark generates no
# random password and logs nothing, it just disables the dashboard.
#
# CONSUMED ON THE FIRST START ONLY. After admin.json exists on the PVC this
# field is never read again, so rotating it here changes nothing; the reset
# goes through the dashboard or by deleting admin.json.
itemPath: {{ required "You must provide an app secret reference (secrets.appSecretRef) - the 1Password item holding the fields 'session-secret' (openssl rand -base64 32; mandatory, the settings sync is silently disabled without it) and 'admin-password' (the admin dashboard, which is the only way back out of an OIDC lockout)" .Values.secrets.appSecretRef }}
{{- if $settings.oauth.enabled }}
---
apiVersion: onepassword.com/v1
kind: OnePasswordItem
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-oauth-secret
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
# Must carry the field: oauth-client-secret
#
# The confidential OIDC client's secret, as issued by the identity provider
# when the client is registered. The registration is a PRE-INSTALL step, not a
# follow-up: with bulwark.oauth.only (the default) and no working client, the
# first sign-in is impossible. NOTES.txt prints the redirect URI to register.
#
# It arrives as a FILE, through OAUTH_CLIENT_SECRET_FILE, so it is not in the
# container's process environment - which garage (#81) and keycloak (#83) could
# not achieve because neither application offered a file form. #81's other half
# was checked here and does not apply: lib/read-file-env.ts is readFileSync
# plus try/catch with no mode check, so the group-read bit fsGroup adds is
# harmless.
#
# AND THE ONE WAY A ROTATION SILENTLY DOES NOTHING: getGlobalClientSecret
# checks the ADMIN OVERRIDE FIRST. If the dashboard's OAuth setup has ever
# run, config.json on the PVC carries oauthClientSecret in PLAINTEXT and that
# value beats this file forever. That is also why a backup of the volume can
# contain this secret in the clear.
itemPath: {{ required "You must provide an OAuth client secret reference (secrets.oauthSecretRef) - the 1Password item holding the field 'oauth-client-secret', as issued by the identity provider when the OIDC client was registered. Required while bulwark.oauth.enabled is true; set it to false for a password-login install" .Values.secrets.oauthSecretRef }}
{{- end }}
bulwark/templates/bulwark.service.yaml
+48
View File
@@ -0,0 +1,48 @@
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-service
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
selector:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
ports:
# The one port this application serves. targetPort addresses the container
# port BY NAME, so bulwark.port can be changed freely while renaming the port
# would break this and all three probes at once.
- protocol: TCP
port: {{ $settings.port | int64 }}
targetPort: http
name: http
# ClusterIP, and no value to change it. The way out of the cluster is the
# Ingress with a certificate: this is a webmail client that handles session
# cookies and mail content, so a NodePort would put it on a routable address
# over plain HTTP by accident.
type: ClusterIP
---
{{/*
The headless Service a StatefulSet needs for its serviceName. It exists because
a StatefulSet requires one, not because anything addresses the pod by its stable
DNS name - with one replica and no peer coordination there is nothing to address.
Kept separate from the ClusterIP Service above so that the latter stays a plain
load-balanced endpoint for the Ingress; pointing serviceName at a Service with a
ClusterIP is accepted by the API server but gives the pod no per-pod DNS record,
which is the kind of half-working arrangement that only shows up later.
*/}}
apiVersion: v1
kind: Service
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-headless
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
clusterIP: None
selector:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
ports:
- protocol: TCP
port: {{ $settings.port | int64 }}
targetPort: http
name: http
bulwark/templates/bulwark.sfs.yaml
+329
View File
@@ -0,0 +1,329 @@
{{- $settings := fromYaml (include "bulwark.settings" .) -}}
{{- /*
A StatefulSet and not a Deployment, and the reason is the single ReadWriteOnce
PVC rather than any cluster identity: a rolling Deployment update blocks itself,
because the new pod waits for a volume the old pod still holds. The repo's
convention requires a StatefulSet for a stateful application anyway, and this one
is stateful - /app/data carries the admin configuration, the admin password hash,
the audit log and the encrypted user settings.
NO volumeClaimTemplates: this repo uses standalone PVC templates throughout, so
the StatefulSet can be renamed in a major PR with a --cascade=orphan note without
the data following the name (README, guideline 1).
NO INIT CONTAINER, and that is checked rather than skipped. Bulwark creates its
four data subdirectories itself on start (ensureConfigDir / ensureStateDir, both
mkdir -p), which it has to because the image's own mkdir ran in a layer this
mount covers; fsGroup makes the fresh volume writable for uid 1001. There is
nothing to seed and nothing to chown, so an init container would only be a second
image to pin and update.
GUARD AT RENDER TIME, not at start: a missing OIDC client id or issuer with
oauth.enabled would otherwise produce a release that installs cleanly, becomes
Ready, and then answers every sign-in attempt with a broken redirect - and with
oauth.only there is no password form to fall back on. The render fails instead,
naming the value. It lives HERE rather than in NOTES.txt because `helm template`
does not render NOTES.txt (guideline 12.2), so a guard there would pass every
check in CI while being absent.
*/ -}}
{{- if $settings.oauth.enabled }}
{{- if not $settings.oauth.clientId }}
{{- fail "bulwark.oauth.clientId is required while bulwark.oauth.enabled is true - without it the OIDC redirect is built against an empty client and every sign-in fails. With bulwark.oauth.only (the default) there is no password form to fall back to, so this would be an install nobody can log into. Register the client with the identity provider first; NOTES.txt prints the redirect URI." }}
{{- end }}
{{- if not $settings.oauth.issuerUrl }}
{{- fail "bulwark.oauth.issuerUrl is required while bulwark.oauth.enabled is true - Bulwark fetches the OIDC discovery document from it, and without it there is nothing to discover. For keycloak this is https://<host>/realms/<realm>." }}
{{- end }}
{{- end }}
{{- if not $settings.jmapServerUrl }}
{{- /* Not merely "the mail server is unset": an empty JMAP_SERVER_URL puts
detectSetupState() into "bootstrap" (lib/setup/state.ts), which serves the
FIRST-RUN WIZARD at /setup and 302s everything else there. The wizard then
writes its result into config.json on the PVC - and from that moment the
volume beats every value in this chart, permanently and silently. So an
unset value here does not produce a non-working release, it produces a
SILENTLY DIFFERENT ONE, which is worse. */}}
{{- fail "bulwark.jmapServerUrl is required - it is the address of the Stalwart server AND the switch that disables the first-run setup wizard. Left empty, Bulwark starts in bootstrap mode, serves its wizard at /setup, and whatever is clicked through there lands in config.json on the PVC - which from then on OVERRIDES every value this chart sets, silently. Set it to the stalwart release's JMAP endpoint." }}
{{- end }}
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
spec:
replicas: {{ include "bulwark.replicas" (dict "scaleDown" .Values.scaleDown "replicas" $settings.replicas) }}
serviceName: {{ .Release.Name }}-{{ .Chart.Name }}-headless
selector:
matchLabels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
template:
metadata:
name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
labels:
app: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
annotations:
# NO checksum/config ANNOTATION, AND THAT IS NOT AN OMISSION (#82).
# This chart renders no ConfigMap: Bulwark is configured through
# environment variables plus the two JSON files it writes itself onto the
# PVC. There is no rendered config file whose change could fail to
# restart the workload, so the rule has nothing to attach to. A changed
# environment variable changes the pod template directly and rolls the
# pod, which is what the annotation exists to achieve elsewhere.
#
# Hash of the rendered OnePasswordItems - the op:// PATHS, not their
# contents. Repointing secrets.appSecretRef or secrets.oauthSecretRef at
# a different vault item therefore rolls the pod instead of leaving the
# old credentials in the running process. A change INSIDE an item is not
# covered and cannot be, because Helm never sees it - but for the two
# FILE-form secrets it does not need to be: both are re-read on every use
# (see the secrets volume below), so a rotation is live without a
# restart. The exception is admin-password, which is consumed exactly
# once on the first start and which no restart brings back.
checksum/secret-ref: {{ include (print $.Template.BasePath "/bulwark.secret.yaml") . | sha256sum }}
spec:
automountServiceAccountToken: false
securityContext:
{{- include "bulwark.podSecurityContext" $.Values.bulwark.securityContext | nindent 8 }}
containers:
- name: {{ .Release.Name }}-{{ .Chart.Name }}-sfs
# Fully qualified. The suffix-free tag family, see Chart.yaml - this
# registry publishes every release twice (1.12.0 and v1.12.0) and the
# chart pins the form without the "v".
image: "ghcr.io/bulwarkmail/webmail:{{ .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
# No command and no args: the image's CMD is ["node", "server.js"] with
# WORKDIR /app, and the working directory is load-bearing here rather
# than incidental - Bulwark resolves its data directories relative to
# process.cwd(), so overriding either would move /app/data out from
# under the volume mount.
env:
# THE MAIL SERVER, AND THE SWITCH THAT KILLS THE SETUP WIZARD. With
# this set, detectSetupState() returns "env-managed" and /setup is
# not served at all, which is what makes this chart declarative.
#
# IT IS STILL BEATEN BY A SERVER URL SAVED IN THE ADMIN DASHBOARD -
# the one precedence trap upstream documents itself. See values.yaml.
- name: JMAP_SERVER_URL
value: {{ $settings.jmapServerUrl | quote }}
{{- with $settings.appName }}
- name: APP_NAME
value: {{ . | quote }}
{{- end }}
- name: STALWART_FEATURES
value: {{ $settings.stalwartFeatures | quote }}
- name: STALWART_ADMIN_ACCESS
value: {{ $settings.stalwartAdminAccess | quote }}
# The settings sync. Note that this flag alone is not enough: without
# a session secret, app/api/config/route.ts turns it off SILENTLY
# (`configManager.get(...) && hasSessionSecret()`). The secret below
# is therefore mandatory rather than recommended.
- name: SETTINGS_SYNC_ENABLED
value: {{ $settings.settingsSync | quote }}
# THE SESSION SECRET, AS A FILE PATH. Everything about why it is the
# file form and why the mount has no subPath is at the volume below.
#
# This is also ONE OF ONLY TWO VALUES IN THIS CHART THAT THE VOLUME
# CANNOT OVERRIDE: getSessionSecret (lib/auth/session-secret.ts) reads
# SESSION_SECRET, then SESSION_SECRET_FILE, and only then the admin
# override - the reverse of configManager.get's normal order.
- name: SESSION_SECRET_FILE
value: {{ include "bulwark.secretMountPath" . }}/session-secret
# THE ADMIN PASSWORD, AS AN ENVIRONMENT VARIABLE, AND THAT IS FORCED
# RATHER THAN CHOSEN: ADMIN_PASSWORD has no _FILE form in
# CONFIG_ENV_MAP, unlike the other two secrets here. The residual
# price, stated rather than hidden: it is in the container's process
# environment, so anyone who can exec into the pod or read its /proc
# can see it until the first start has hashed it into admin.json.
#
# WHY IT IS SET AT ALL: the admin dashboard is the only way back out
# of an OIDC lockout (OAUTH_ONLY below), and without this variable
# the dashboard is simply disabled - there is no random password and
# nothing in the log, measured in lib/admin/password.ts.
- name: ADMIN_PASSWORD
valueFrom:
secretKeyRef:
name: {{ .Release.Name }}-{{ .Chart.Name }}-app-secret
key: admin-password
{{- if $settings.adminConfigReadOnly }}
# THE CHART IS AUTHORITATIVE FROM HERE ON - and this is the one value
# whose price is bigger than its name. It freezes the settings policy
# (and with it the only way to move Web Push off the vendor relay),
# it freezes an override that ALREADY exists instead of removing it,
# and on a FRESH install it prevents admin.json from being written at
# all - which takes the admin dashboard, i.e. the rescue door, away
# entirely while the pod stays Ready. values.yaml has the measurement
# and the only safe sequence.
- name: ADMIN_CONFIG_READONLY
value: "true"
{{- end }}
{{- if $settings.oauth.enabled }}
- name: OAUTH_ENABLED
value: "true"
# HIDES THE PASSWORD FORM. The way back out is NOT a helm upgrade
# with this false - oauthOnly is read through configManager.get by
# every consumer, so an override in config.json on the PVC wins. The
# repair is removing that override in the admin dashboard. NOTES.txt
# carries it.
- name: OAUTH_ONLY
value: {{ $settings.oauth.only | quote }}
- name: OAUTH_CLIENT_ID
value: {{ $settings.oauth.clientId | quote }}
- name: OAUTH_ISSUER_URL
value: {{ $settings.oauth.issuerUrl | quote }}
# The client secret as a FILE, so it stays out of the process
# environment - the better path garage (#81) and keycloak (#83) did
# not have. Re-read on every token exchange, so a rotation needs no
# restart; see the volume below for the subPath condition that makes
# that true.
- name: OAUTH_CLIENT_SECRET_FILE
value: {{ include "bulwark.secretMountPath" . }}/oauth-client-secret
{{- with $settings.oauth.scopes }}
- name: OAUTH_SCOPES
value: {{ . | quote }}
{{- end }}
- name: OAUTH_END_SESSION
value: {{ $settings.oauth.endSession | quote }}
{{- end }}
# Both OFF against the image's defaults, and deliberately: a mail
# client should not phone home from a private cluster unless somebody
# chose it, and Renovate already watches the image tag, so the
# version check would answer outbound a question already answered
# inbound.
- name: BULWARK_TELEMETRY
value: {{ ternary "on" "off" $settings.telemetry | quote }}
- name: BULWARK_UPDATE_CHECK
value: {{ ternary "on" "off" $settings.updateCheck | quote }}
- name: LOG_LEVEL
value: {{ $settings.logLevel | quote }}
- name: LOG_FORMAT
value: {{ $settings.logFormat | quote }}
# PORT and HOSTNAME are already set in the image (3000 / 0.0.0.0);
# PORT is repeated here only because bulwark.port makes it
# configurable, and the containerPort below has to agree with it.
- name: PORT
value: {{ $settings.port | quote }}
{{- with $.Values.bulwark.env }}
{{- include "bulwark.env" (dict "root" $ "env" .) | nindent 12 }}
{{- end }}
ports:
# The name "http" is what the Service and all three probes reference,
# so renaming it here breaks four things at once.
- name: http
containerPort: {{ $settings.port | int64 }}
protocol: TCP
# Probes, resources and both security contexts go through the null-safe
# helpers (#99): an erased block means "chart defaults apply" instead of
# a vanished probe or an unhardened container. Never wrap these in an
# "if" - a guard hides the erased case completely instead of rendering a
# visible null.
#
# All three are an HTTP GET on /api/health - not HEAD, because a
# Kubernetes probe cannot send one (HTTPGetAction has no method field).
# What that endpoint can and cannot report is argued at the helpers and
# at bulwark.readinessProbe in values.yaml; the short version is that it
# carries the process, HTTP acceptance, a free event loop and Next.js
# routing - and NOTHING about the mail server.
livenessProbe:
{{- include "bulwark.livenessProbe" $.Values.bulwark.livenessProbe | nindent 12 }}
readinessProbe:
{{- include "bulwark.readinessProbe" $.Values.bulwark.readinessProbe | nindent 12 }}
startupProbe:
{{- include "bulwark.startupProbe" $.Values.bulwark.startupProbe | nindent 12 }}
resources:
{{- include "bulwark.effectiveResources" $.Values.bulwark.resources | nindent 12 }}
securityContext:
{{- include "bulwark.containerSecurityContext" $.Values.bulwark.securityContext | nindent 12 }}
volumeMounts:
# THE ONE PVC, carrying all four of Bulwark's directories as
# subdirectories so that a volume snapshot catches configuration,
# runtime state, audit log and user settings atomically.
- name: data
mountPath: {{ include "bulwark.dataDir" . }}
# THE TWO FILE-FORM SECRETS, AS A WHOLE DIRECTORY AND NOT WITH
# subPath. This is the line a later round will want to "tidy up" into
# two subPath mounts, and that would silently break secret rotation:
# the kubelet never refreshes a subPath mount, while both values are
# re-read from disk on every use (getSessionSecret and
# getGlobalClientSecret both call readFileEnv with no cache). With
# this mount a rotated vault field is live without a restart; with
# subPath the container would hold the first value forever and
# nothing would report it.
- name: secrets
mountPath: {{ include "bulwark.secretMountPath" . }}
readOnly: true
# The two writable paths outside the volume, needed because
# readOnlyRootFilesystem is true. Both are CACHES, not state - see
# bulwark.writableCache in values.yaml for how the set was
# established and why neither belongs on the PVC.
- name: tmp
mountPath: /tmp
- name: next-cache
mountPath: /app/.next/cache
restartPolicy: Always
volumes:
- name: data
persistentVolumeClaim:
claimName: {{ .Release.Name }}-{{ .Chart.Name }}-pvc
- name: secrets
# One projected volume over the two Secrets the 1Password operator
# materializes, so both files sit in one directory and the container
# needs one mount instead of two.
#
# defaultMode 0440 - owner and GROUP read, and the group bit is the
# part that must not be "tightened" to 0400. Files in a secret volume
# are owned by ROOT, not by the container's user; what gives uid 1001
# access is fsGroup, which the kubelet applies by chowning the files to
# group 1001. Without a group read bit, uid 1001 could not read either
# file - and the failure would be silent, because readFileEnv answers
# an EACCES with null exactly as it answers a missing file: the
# settings sync would switch itself off and every token exchange would
# send an empty client secret.
#
# (The kubelet ORs 0440 onto the mode of a secret volume once fsGroup
# is set, so 0400 would in practice be corrected to 0440 anyway. It is
# written out because a mode that only works because something else
# silently fixes it is not a mode anyone should have to re-derive.)
#
# #81's permission trap does not apply in the other direction either:
# lib/read-file-env.ts is readFileSync plus try/catch with no mode
# check, so unlike garage, Bulwark does not REJECT a group-readable
# secret file. That is what makes the file form usable here at all.
projected:
defaultMode: 0440
sources:
- secret:
name: {{ .Release.Name }}-{{ .Chart.Name }}-app-secret
items:
# Only the session secret becomes a file. admin-password is
# in the same Secret but arrives as an environment variable
# (it has no _FILE form), so projecting it here as well would
# put it on disk for no reader.
- key: session-secret
path: session-secret
# optional: the 1Password operator fills this Secret in-cluster
# AFTER Helm has applied the manifests, so on a first install
# the Secret does not exist yet. Without this the pod would sit
# in ContainerCreating until the operator caught up, and with a
# misconfigured operator it would sit there forever with the
# reason only in the pod's events. It starts instead, and the
# missing secret surfaces where it is actionable: the settings
# sync stays off and the log says so.
optional: true
{{- if $settings.oauth.enabled }}
- secret:
name: {{ .Release.Name }}-{{ .Chart.Name }}-oauth-secret
items:
- key: oauth-client-secret
path: oauth-client-secret
optional: true
{{- end }}
- name: tmp
emptyDir:
sizeLimit: {{ $settings.writableCache.tmpSizeLimit }}
# Next.js's own cache directory in standalone mode - the optimized-image
# cache and the fetch cache. It is a SUBDIRECTORY of .next, which is what
# makes this possible at all: /app/.next/static holds the build output and
# has to stay readable, and mounting the cache path does not shadow it.
- name: next-cache
emptyDir:
sizeLimit: {{ $settings.writableCache.nextCacheSizeLimit }}