Cluster-capable from the start, with roles rather than symmetry: one StatefulSet per entry of the "nodes" map, the key passed in STALWART_ROLE. Since v0.16 Stalwart keeps its whole configuration inside the data store and manages it over JMAP, so the chart's scope is deliberately narrow: it renders config.json (exactly the DataStore object), the workloads, the Services, the NetworkPolicies, the Ingress and a cert-manager Certificate. Listeners, blob store, in-memory store, TLS object and cluster roles are operator work in the WebUI, which is why NOTES.txt carries an eight-step first install. config.json stays secret-free: authSecret uses the EnvironmentVariable variant (verified against the serde definition of PostgreSqlStore at v0.16.24), so the password arrives through secretKeyRef and checksum/config stays meaningful. The username cannot follow it - authUsername is a plain String with no secret form - so it is a public value and says so. Services are per PORT, not per group: a NodePort is cluster-unique, so "full" and "mx" can only accept SMTP behind one Service, selected through a per-port pod label. The http port gets a per-group ClusterIP Service instead, so the Ingress reaches exactly ingress.nodeGroup. Four traps, documented rather than explained away: - The role lookup FAILS OPEN. A node whose ClusterRole is missing from the database enables every listener and every task while staying Running with a green probe, and the log line arrives after the listeners are bound. Hence mx at replicas 0, per-port Services, and a NetworkPolicy that is a ceiling independent of what was bound. - NET_BIND_SERVICE is a documented deviation needed to EXEC, not to bind. The image sets setcap cap_net_bind_service=+ep, which makes the kernel fail execve when the bounding set is empty. The values comment says both why it is there and that it is never used, because the second half is what stops the next hardening pass from removing it. - externalTrafficPolicy must be Cluster - no pod runs on a control-plane node while the HA proxy delivers to them, so Local would drop every packet - and the source address that costs is restored only by the HA proxy's PROXY v2 header. - /healthz is rate-limited to 100 requests/60s per source IP, which is a node IP for kubelet probes, so all pods on a node share one bucket. The probe intervals are calculated (6 req/min/pod, ~16 pods per node) and the sum is a comment in values.yaml. A first install needs bootstrap.enabled: true: the setup wizard appears only when config.json is ABSENT, and this chart always renders one, so recovery mode is upstream's documented path for a templated config.json. Second chart in the repo with a dependency - the Valkey project's own chart, off by default behind condition:, because Stalwart's cluster coordinator rides on the in-memory store connection. The render aborts when more than one pod runs with no in-memory store address anywhere. Also wired up: publish-stalwart.yml through the shared block in workflow-util/pipelines, the change-app-version.yml choice list, a Renovate customManagers block anchored on the v prefix (which excludes the moving latest / v0.16 / -alpine tags), CI fixtures, and the README and CLAUDE.md lines that claimed ytdlp-web-player was the only chart with a dependency. Verified locally: ci/helm-dependencies.sh, helm lint --strict (0 failed), helm package, helm template against chart defaults and against ci/stalwart/test-values.yaml, the rendered config.json parsed as JSON with the field types the serde definition requires, ci/run-quantity-render-test.sh (0 failed, both nodes.*.resources paths exercised), NOTES.txt rendered through a scratch ConfigMap, and the null-safety and fail paths (securityContext=null keeps NET_BIND_SERVICE; a cluster without an in-memory store, an unknown port name, a group without http and an unknown ingress.nodeGroup all abort with a named message; the schema rejects timeout, a string timeoutMs and a list-shaped nodes). NO runtime verification was possible - there is no k3d or Docker daemon in this workspace. Part of #65
124 lines
4.3 KiB
YAML
124 lines
4.3 KiB
YAML
{{- /*
|
|
Three kinds of Service, and the split is forced rather than stylistic.
|
|
|
|
1. ONE NodePort Service PER PUBLIC PORT, not per node group. A NodePort is
|
|
unique across the whole cluster, so "full" and "mx" can only accept SMTP
|
|
behind the SAME Service object - two Services asking for 30025 is a
|
|
conflict. Hence the selector matches a per-port LABEL on the pods instead
|
|
of a node group, which lets one Service cover every group that lists the
|
|
port and still exclude the ones that do not.
|
|
|
|
Putting all the ports into one Service does not work either: a Service has
|
|
exactly one selector, so 30993 would then be routed to "mx" pods as well,
|
|
where nothing is listening.
|
|
|
|
2. ONE ClusterIP http Service PER NODE GROUP. The Ingress must reach exactly
|
|
one group (ingress.nodeGroup): the management surface is not safe to spread
|
|
across a group that may not run the management tasks, and a port-label
|
|
selector would send each request to a random group. So http gets a
|
|
per-group Service and the Ingress names one of them.
|
|
|
|
3. ONE headless Service PER NODE GROUP, for the StatefulSet's stable pod DNS.
|
|
*/ -}}
|
|
{{- $root := . -}}
|
|
{{- $ports := fromYaml (include "stalwart.ports" .) -}}
|
|
{{- $nodes := fromYaml (include "stalwart.nodes" .) -}}
|
|
{{- $release := include "stalwart.fullname" . -}}
|
|
|
|
{{- /* 1. Per-port NodePort Services. */ -}}
|
|
{{- range $portName, $port := $ports }}
|
|
{{- if $port.public }}
|
|
{{- $groups := list }}
|
|
{{- range $group, $spec := $nodes }}
|
|
{{- if has $portName $spec.exposedPorts }}{{- $groups = append $groups $group }}{{- end }}
|
|
{{- end }}
|
|
{{- if $groups }}
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: {{ $release }}-{{ $portName }}-service
|
|
labels:
|
|
app: {{ $release }}
|
|
{{- with $root.Values.service.annotations }}
|
|
annotations:
|
|
{{- toYaml . | nindent 4 }}
|
|
{{- end }}
|
|
spec:
|
|
type: NodePort
|
|
# Cluster, never Local. No Stalwart pod is ever scheduled on a control-plane
|
|
# node, and the HA proxy delivers to the control-plane nodes - under Local
|
|
# every packet would be dropped, not some. The cost is that kube-proxy
|
|
# masquerades the source address; the PROXY v2 header the HA proxy prepends
|
|
# is what makes that harmless. values.yaml has the full reasoning and the
|
|
# list of what a lost source address breaks on a mail server.
|
|
externalTrafficPolicy: {{ $root.Values.service.externalTrafficPolicy }}
|
|
selector:
|
|
# Scoped to this release, then to the port. Node groups serving this port:
|
|
# {{ $groups | sortAlpha | join ", " }}
|
|
stalwart.jk-dev.de/release: {{ $release }}
|
|
{{ include "stalwart.portLabel" $portName }}: "true"
|
|
ports:
|
|
- name: {{ $portName }}
|
|
protocol: {{ $port.protocol | default "TCP" }}
|
|
# port and nodePort are the published number; targetPort is the number
|
|
# the application binds inside the pod. They differ here on purpose.
|
|
port: {{ int $port.nodePort }}
|
|
nodePort: {{ int $port.nodePort }}
|
|
targetPort: {{ $portName }}
|
|
{{- end }}
|
|
{{- end }}
|
|
{{- end }}
|
|
|
|
{{- /* 2. + 3. Per-group http ClusterIP and headless Services. */ -}}
|
|
{{- range $group, $spec := $nodes }}
|
|
{{- $name := printf "%s-%s" $release $group }}
|
|
{{- if has "http" $spec.exposedPorts }}
|
|
{{- $http := index $ports "http" }}
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: {{ $name }}-http-service
|
|
labels:
|
|
app: {{ $name }}
|
|
{{- with $root.Values.service.annotations }}
|
|
annotations:
|
|
{{- toYaml . | nindent 4 }}
|
|
{{- end }}
|
|
spec:
|
|
type: ClusterIP
|
|
selector:
|
|
app: {{ $name }}-sfs
|
|
ports:
|
|
- name: http
|
|
protocol: {{ $http.protocol | default "TCP" }}
|
|
port: {{ int $http.containerPort }}
|
|
targetPort: http
|
|
{{- end }}
|
|
---
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: {{ $name }}-headless
|
|
labels:
|
|
app: {{ $name }}
|
|
spec:
|
|
# Headless: this exists for the StatefulSet's stable per-pod DNS names
|
|
# (<pod>.<release>-stalwart-<group>-headless), not for load balancing.
|
|
clusterIP: None
|
|
# So the DNS names resolve before a pod is Ready - a cluster member has to be
|
|
# findable by its peers while it is still starting up.
|
|
publishNotReadyAddresses: true
|
|
selector:
|
|
app: {{ $name }}-sfs
|
|
ports:
|
|
{{- range $portName := $spec.exposedPorts }}
|
|
{{- $port := index $ports $portName }}
|
|
- name: {{ $portName }}
|
|
protocol: {{ $port.protocol | default "TCP" }}
|
|
port: {{ int $port.containerPort }}
|
|
targetPort: {{ $portName }}
|
|
{{- end }}
|
|
{{- end }}
|