feat(stalwart): spread each node group over nodes and give each one a PDB
A two-replica node group was redundancy on paper: nothing kept both pods off one node, and no PodDisruptionBudget kept a drain from taking both at once. Both objects are now rendered per node group, gated on that group's COMPUTED replica count being >= 2. The distribution is a promise, not an effort: maxSkew 1 on kubernetes.io/hostname with whenUnsatisfiable DoNotSchedule, selecting only this group's own pods, so an "mx" and a "full" pod may share a node. The selector comes from the chart (stalwart.groupLabels), never from the operator - a spread constraint whose selector matches no pod is trivially satisfied, distributes nothing and reports nothing. nodeAffinityPolicy: Ignore is set explicitly, and that is the load-bearing detail. Its API default is Honor, under which a narrowing nodeSelector shrinks the set of counted domains until both replicas fit into one: skew 0, both pods co-located, no Pending, no event, and a selector that is correct - so no check can see it. With Ignore the same narrowing gives skew 2 and the second pod is Pending. A render abort was the alternative and would also have forbidden a nodeSelector spanning five nodes, which is legal. A user entry colliding with the chart's own on (topologyKey, whenUnsatisfiable) aborts the render naming both values, because the API rejects the duplicate with a message about a list index and an apply-based path merges the two over patchMergeKey: topologyKey silently, dropping one selector. Minor rather than major, proved with a diff rather than argued: with the shipped defaults (full: 1, mx: 0) and with ci/stalwart/test-values.yaml, helm template renders byte-for-byte what 1.0.1+upv0.16.24 rendered - both diffs empty. New in CI, over every chart on every PR: ci/run-selector-match-test.sh holds every rendered PodDisruptionBudget selector and every topologySpreadConstraints labelSelector against the labels of the pod templates in the same render. Helm itself is the YAML parser (ci/yaml2json), because the runners have neither pyyaml nor yq and a hand-written parser could read nothing and report nothing. podAffinity/podAntiAffinity are out of scope; matchExpressions-only selectors are reported as not checked rather than counted as passing. Two anti-vacuum guards: a chart with an overlay under ci/selector-match/ that renders no selector fails, and a run that checks no selector at all fails. Fixes #96
This commit was merged in pull request #97.
This commit is contained in:
1 parent
1698fc0aeb
commit
6ade74db80
14 files changed
+1125
-8
No files matched your search
@@ -191,6 +191,24 @@
|
||||
"exposedPorts": {
|
||||
"$ref": "#/definitions/exposedPorts"
|
||||
},
|
||||
"spreadAcrossNodes": {
|
||||
"$ref": "#/definitions/spreadAcrossNodes"
|
||||
},
|
||||
"podDisruptionBudget": {
|
||||
"$ref": "#/definitions/podDisruptionBudget"
|
||||
},
|
||||
"affinity": {
|
||||
"$ref": "#/definitions/podSpecObject"
|
||||
},
|
||||
"nodeSelector": {
|
||||
"$ref": "#/definitions/podSpecObject"
|
||||
},
|
||||
"tolerations": {
|
||||
"$ref": "#/definitions/podSpecArray"
|
||||
},
|
||||
"topologySpreadConstraints": {
|
||||
"$ref": "#/definitions/topologySpreadConstraints"
|
||||
},
|
||||
"resources": {
|
||||
"$ref": "#/definitions/resources"
|
||||
},
|
||||
@@ -212,6 +230,24 @@
|
||||
"exposedPorts": {
|
||||
"$ref": "#/definitions/exposedPorts"
|
||||
},
|
||||
"spreadAcrossNodes": {
|
||||
"$ref": "#/definitions/spreadAcrossNodes"
|
||||
},
|
||||
"podDisruptionBudget": {
|
||||
"$ref": "#/definitions/podDisruptionBudget"
|
||||
},
|
||||
"affinity": {
|
||||
"$ref": "#/definitions/podSpecObject"
|
||||
},
|
||||
"nodeSelector": {
|
||||
"$ref": "#/definitions/podSpecObject"
|
||||
},
|
||||
"tolerations": {
|
||||
"$ref": "#/definitions/podSpecArray"
|
||||
},
|
||||
"topologySpreadConstraints": {
|
||||
"$ref": "#/definitions/topologySpreadConstraints"
|
||||
},
|
||||
"resources": {
|
||||
"$ref": "#/definitions/resources"
|
||||
},
|
||||
@@ -233,6 +269,24 @@
|
||||
"exposedPorts": {
|
||||
"$ref": "#/definitions/exposedPorts"
|
||||
},
|
||||
"spreadAcrossNodes": {
|
||||
"$ref": "#/definitions/spreadAcrossNodes"
|
||||
},
|
||||
"podDisruptionBudget": {
|
||||
"$ref": "#/definitions/podDisruptionBudget"
|
||||
},
|
||||
"affinity": {
|
||||
"$ref": "#/definitions/podSpecObject"
|
||||
},
|
||||
"nodeSelector": {
|
||||
"$ref": "#/definitions/podSpecObject"
|
||||
},
|
||||
"tolerations": {
|
||||
"$ref": "#/definitions/podSpecArray"
|
||||
},
|
||||
"topologySpreadConstraints": {
|
||||
"$ref": "#/definitions/topologySpreadConstraints"
|
||||
},
|
||||
"resources": {
|
||||
"$ref": "#/definitions/resources"
|
||||
},
|
||||
@@ -410,6 +464,31 @@
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"spreadAcrossNodes": {
|
||||
"description": "Renders a topologySpreadConstraints entry for THIS group on topologyKey kubernetes.io/hostname, maxSkew 1, whenUnsatisfiable DoNotSchedule, nodeAffinityPolicy Ignore, with a labelSelector the chart fills from its own pod labels - the user never writes it, because a selector that matches no pod is trivially satisfied, distributes nothing and reports nothing. Rendered only while the group's COMPUTED replica count is >= 2, so scaleDown: true removes it as well. DoNotSchedule makes it a promise rather than an effort, and the price is a Pending pod: when two of three workers are gone (while their Node objects still exist), or when a nodeSelector/affinity narrows the eligible nodes below the replica count. nodeAffinityPolicy is beta - on by default from Kubernetes 1.26, GA with the NodeInclusionPolicyInPodTopologySpread gate in 1.33 - and is pruned SILENTLY on anything older, which brings back exactly the co-location it was set against. The replica ceiling (as many per group as there are schedulable workers, because control-plane domains hold the minimum at 0) comes from nodeTaintsPolicy and is NOT a consequence of that field. The whole rule holds only while this chart stays PVC-free - see open measurement 1 on #65.",
|
||||
"type": "boolean"
|
||||
},
|
||||
"podDisruptionBudget": {
|
||||
"description": "One PodDisruptionBudget per node group, rendered only while that group's COMPUTED replica count is >= 2 (so scaleDown: true renders none). The condition is the point, not tidiness: minAvailable 1 with exactly one replica forbids every voluntary eviction, so kubectl drain blocks forever on a pod nothing may move - and the message names the PDB rather than the replica count. minAvailable rather than maxUnavailable, as in keycloak: the promise is about the service. The selector comes from the same definition as the pod template's labels, because a PDB matching no pod never blocks an eviction while looking like protection.",
|
||||
"type": ["object", "null"],
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"enabled": { "type": "boolean" },
|
||||
"minAvailable": { "type": "integer", "minimum": 1 }
|
||||
}
|
||||
},
|
||||
"podSpecObject": {
|
||||
"description": "Passed verbatim into this group's pod spec (affinity, nodeSelector). Loosely typed on purpose, as with the probes and the security contexts: Kubernetes owns this schema, the chart reads none of the contents, and a stricter copy here would reject valid configurations rather than catch mistakes. Declaring the key is not optional even so, because the level above carries additionalProperties: false. Empty renders no field at all. Mind that narrowing the eligible nodes below a group's replica count while spreadAcrossNodes is on leaves a replica permanently Pending - intended, and loud by design.",
|
||||
"type": ["object", "null"]
|
||||
},
|
||||
"podSpecArray": {
|
||||
"description": "Passed verbatim into this group's pod spec (tolerations). Loosely typed for the same reason as podSpecObject. Note that tolerating a taint does not raise the replica ceiling: with nodeTaintsPolicy at its default of Ignore an untolerated node already counts as an empty domain.",
|
||||
"type": ["array", "null"]
|
||||
},
|
||||
"topologySpreadConstraints": {
|
||||
"description": "Additional constraints for this group, APPENDED after the chart's own and really AND-ed with it, so a second key such as topology.kubernetes.io/zone works as expected. Loosely typed like the other pass-throughs. An entry on kubernetes.io/hostname with whenUnsatisfiable DoNotSchedule - or omitted, which IS DoNotSchedule - aborts the render naming both this value and spreadAcrossNodes: Kubernetes keys a constraint on that pair, kubectl create rejects the duplicate with a message about a list index instead of a value, and an apply-based path merges the two over patchMergeKey: topologyKey silently, dropping one labelSelector for a green result.",
|
||||
"type": ["array", "null"]
|
||||
},
|
||||
"externalService": {
|
||||
"type": ["object", "null"],
|
||||
"additionalProperties": false,
|
||||
|
||||
Reference in new issue
Block a user