jklein
a9ca5302de
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 12s
Validate Helm Charts / Build paperless-ngx (pull_request) Successful in 6s
Validate Helm Charts / Trivy config paperless-ngx (pull_request) Successful in 22s
Install-test / Install-test paperless-ngx (pull_request) Successful in 3m31s
Validate Helm Charts / Install-test (pull_request) Successful in 3m31s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
Publish Paperless-NGX Helm Chart / Publish Paperless-NGX Helm Chart (push) Successful in 10s
SonarQube Analysis / SonarQube Analysis (push) Successful in 33s
paperless-ngx cannot run without a Redis - PAPERLESS_REDIS is the Celery broker - so every installation needed a second Helm release whose address lived in a 1Password item and in neither chart. This adds our own published valkey chart as an optional dependency, off by default and gated on valkeySubchart.enabled (not valkey.enabled, which the subchart's strict schema rejects and which Helm would only have complained about for whoever first switched it on). Additive, default behaviour unchanged: minor, 12.1.3+up3.2.1 -> 12.2.0+up3.2.1. Configured as a QUEUE (PVC + noeviction, the subchart's defaults), unlike stalwart's cache. Read off the 3.2.1 source rather than assumed: one PAPERLESS_REDIS url serves five consumers - the Celery broker, the result backend (Redis in 3.x, the database before), the channels pub/sub layer and two Django caches. allkeys-lru would not be a trade here but actively wrong, because LRU evicts typeless across the whole keyspace and a backlog of waiting consume tasks is exactly what it reads as cold; a dropped task is a document silently never processed. The price, named in values.yaml rather than smoothed over: two of the five consumers are caches, so a full instance refuses writes - new tasks included - and the answer is memory sizing, not a policy change. All three cache-like consumers are TTL-bounded, which is what keeps that price small. The password comes from the SAME item as the Redis connection, in a third field auth-conf holding the complete line "requirepass <password>". It cannot DEFAULT to secrets.redisSecretRef, and that is a property of Helm, not an omission: a values.yaml is never templated and a parent cannot compute a subchart's value. Measured - a template expression there lands verbatim in the subchart's itemPath and fails as a YAML parse error naming neither the value nor the cause. Forgetting it is therefore caught in values.schema.json, by a conditional rule rather than a fail() in a template. Also measured, and the reason: Helm renders a subchart's templates BEFORE the parent's, so a guard in this chart loses the race to the subchart's own required(), which reports "secrets.authSecretRef" and cannot know the path belongs to the redis item. A check that cannot fire is the mistake found while testing #77, so the first draft of that guard was removed rather than shipped. The schema runs before any rendering and names /valkey/secrets/authSecretRef. One trap found while testing and closed here: "valkeySubchart: null" DELETES the key, Helm then cannot resolve the condition path and treats the dependency as ENABLED - so erasing the switch turned the subchart on with no password set. Unlike every other block in this chart (issue #99), that one is rejected by the schema instead of defaulting. The install-test now runs with the subchart ON and the throwaway redis Deployment removed: the url in the fixtures names ci-valkey-service, so the bundled Valkey is the broker the release actually uses. #65 covered this path by a render only - deliberately, since an unused Valkey would have made that test look like it covered clustering. This is the first real installation of the valkey-subchart path. Documented, not glossed: the password now sits in that item twice and a rotation has to touch both fields; with the subchart on the vault url is a mandatory step, not a hint; and a OnePasswordItem materializes every field of its item, so both Secrets carry all three keys - untidy, harmless, and written down so nobody "tidies it up". Fixes #78
343 lines
14 KiB
Smarty
343 lines
14 KiB
Smarty
{{/*
|
|
Render the body of a container "resources:" block from a values dict of the form
|
|
|
|
resources:
|
|
limitFactor: 3 # optional, default 3
|
|
requests: { cpu, memory, ephemeral-storage / ephemeralStorage, ... }
|
|
limits: { ... } # optional, every entry optional
|
|
|
|
Rules (variant B, see issue #19):
|
|
- requests are rendered as given (camelCase "ephemeralStorage" is normalized
|
|
to the Kubernetes key "ephemeral-storage").
|
|
- an explicitly set limit always wins.
|
|
- a missing memory / ephemeral-storage limit is computed as
|
|
limitFactor x the corresponding request.
|
|
- a missing cpu limit is NOT set at all (the kubelet then treats CPU as
|
|
unlimited; CPU is compressible, so this cannot kill the workload).
|
|
|
|
Usage (the values path to the resources dict varies per chart):
|
|
resources:
|
|
{{- include "paperless-ngx.resources" .Values.<app>.resources | nindent 12 }}
|
|
|
|
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
|
|
the chart's name as define prefix, because sharing it would mean a LIBRARY
|
|
chart, and a dependency is admissible but expensive (README, guideline 14).
|
|
This chart does take one - the optional valkey subchart - which changes
|
|
nothing here: a library chart for one helper would still be the wrong trade,
|
|
and the subchart carries its own copy of this helper anyway. This file in
|
|
template-chart is the reference implementation.
|
|
*/}}
|
|
{{- define "paperless-ngx.resources" -}}
|
|
{{- $resources := . | default dict -}}
|
|
{{- $factor := 3.0 -}}
|
|
{{- if not (kindIs "invalid" $resources.limitFactor) -}}
|
|
{{- $factor = float64 $resources.limitFactor -}}
|
|
{{- end -}}
|
|
{{- $requests := dict -}}
|
|
{{- range $key, $value := ($resources.requests | default dict) -}}
|
|
{{- if not (kindIs "invalid" $value) -}}
|
|
{{- $_ := set $requests (include "paperless-ngx.resourceKey" $key) $value -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
{{- $limits := dict -}}
|
|
{{- range $key, $value := ($resources.limits | default dict) -}}
|
|
{{- if not (kindIs "invalid" $value) -}}
|
|
{{- $_ := set $limits (include "paperless-ngx.resourceKey" $key) $value -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
{{- range $key, $request := $requests -}}
|
|
{{- if and (ne $key "cpu") (not (hasKey $limits $key)) -}}
|
|
{{- $_ := set $limits $key (include "paperless-ngx.multiplyQuantity" (dict "quantity" $request "factor" $factor)) -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
{{- $out := dict -}}
|
|
{{- with $requests }}{{- $_ := set $out "requests" . }}{{- end -}}
|
|
{{- with $limits }}{{- $_ := set $out "limits" . }}{{- end -}}
|
|
{{- toYaml $out -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Normalize a resource name from values to its Kubernetes key.
|
|
*/}}
|
|
{{- define "paperless-ngx.resourceKey" -}}
|
|
{{- if eq . "ephemeralStorage" -}}ephemeral-storage{{- else -}}{{ . }}{{- end -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Multiply a Kubernetes quantity by a factor, keeping its suffix.
|
|
Input: dict "quantity" <string|number> "factor" <float64>.
|
|
Handles plain numbers ("2", "0.5", 1073741824), decimal ("1.5Gi"), binary
|
|
("512Mi", "1Gi", "10Ki") and milli/SI suffixes ("100m", "500M"). Integer
|
|
results are printed without a decimal point.
|
|
*/}}
|
|
{{- define "paperless-ngx.multiplyQuantity" -}}
|
|
{{- $quantity := include "paperless-ngx.formatNumber" .quantity -}}
|
|
{{- $number := regexFind "^[0-9]+(\\.[0-9]+)?" $quantity -}}
|
|
{{- if not $number -}}
|
|
{{- fail (printf "cannot parse resource quantity %q" $quantity) -}}
|
|
{{- end -}}
|
|
{{- $suffix := trimPrefix $number $quantity -}}
|
|
{{- if not (regexMatch "^(m|k|M|G|T|P|E|Ki|Mi|Gi|Ti|Pi|Ei)?$" $suffix) -}}
|
|
{{- fail (printf "unsupported resource quantity suffix %q in %q" $suffix $quantity) -}}
|
|
{{- end -}}
|
|
{{- $product := mulf (float64 $number) .factor -}}
|
|
{{- if eq $product (floor $product) -}}
|
|
{{- printf "%d%s" (int64 $product) $suffix -}}
|
|
{{- else -}}
|
|
{{- printf "%s%s" (include "paperless-ngx.formatNumber" $product) $suffix -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Render a number as a plain decimal string, never in exponential notation.
|
|
|
|
Why this exists (issue #157): the null-safe fallback helpers pass values
|
|
through fromYaml, which parses every number as float64. Go's "%v" switches
|
|
float64 to exponential notation from about 1e6 upwards, so a bare byte count
|
|
like 1048576 became "1.048576e+06" - not a valid Kubernetes quantity, and the
|
|
render aborted. Strings ("512Mi") are passed through untouched.
|
|
*/}}
|
|
{{- define "paperless-ngx.formatNumber" -}}
|
|
{{- if kindIs "float64" . -}}
|
|
{{- regexReplaceAll "\\.?0+$" (printf "%.10f" .) "" -}}
|
|
{{- else -}}
|
|
{{- printf "%v" . -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Render the replica count for a Deployment/StatefulSet.
|
|
|
|
Input: dict
|
|
"scaleDown" the chart-global .Values.scaleDown flag (may be nil)
|
|
"replicas" the workload's replicas value (may be nil)
|
|
|
|
Rules (issue #25):
|
|
- scaleDown: true takes precedence and always yields 0.
|
|
- otherwise an explicitly set replicas value is used as given, including an
|
|
explicit 0 — Helm's "default" would swallow 0, hence the explicit
|
|
kindIs "invalid" nil check.
|
|
- a missing replicas value defaults to 1.
|
|
|
|
Usage (the values path to the workload's replicas varies per chart):
|
|
replicas: {{ include "paperless-ngx.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.<app>.replicas) }}
|
|
|
|
NOTE: like the resources helper above, this is duplicated into every chart's
|
|
templates/_helpers.tpl with the chart's name as define prefix. This file in
|
|
template-chart is the reference implementation.
|
|
*/}}
|
|
{{- define "paperless-ngx.replicas" -}}
|
|
{{- if .scaleDown -}}
|
|
0
|
|
{{- else if not (kindIs "invalid" .replicas) -}}
|
|
{{- .replicas -}}
|
|
{{- else -}}
|
|
1
|
|
{{- end -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Merge a user-supplied values sub-tree onto a dict of chart defaults so that a
|
|
null / empty override falls back to the defaults instead of erasing them.
|
|
|
|
Input: dict "defaults" <dict> "given" <dict|nil> "path" <string, optional
|
|
values path used in error messages>. Output: YAML of the merged dict (consume
|
|
it with fromYaml).
|
|
|
|
Why this exists (issue #61): Helm merges values maps recursively, but an
|
|
override key written without children — e.g. a leftover
|
|
|
|
barcodes:
|
|
|
|
after deleting its last remaining sub-key — is YAML null and replaces the
|
|
whole default sub-tree. The chart then renders with no barcode settings at
|
|
all. This helper defines the opposite semantics: an empty block means "chart
|
|
defaults apply".
|
|
|
|
Rules:
|
|
- a nil "given" is treated as an empty dict (all defaults apply).
|
|
- a key whose override value is null keeps the default (nulls never delete).
|
|
- a key present in both as a map is merged recursively, so a partially
|
|
filled block only overrides the keys it actually sets.
|
|
- a scalar override for a key the defaults define as a map is rejected with
|
|
a message naming the values path, instead of failing later with a template
|
|
field error.
|
|
- any other set value wins as given, including an explicit false or 0 —
|
|
Helm's "default" and sprig's "mergeOverwrite" would both swallow those
|
|
zero values, hence the explicit kindIs "invalid" nil check.
|
|
- keys only present in the override are passed through unchanged.
|
|
*/}}
|
|
{{- define "paperless-ngx.defaultedDict" -}}
|
|
{{- $defaults := .defaults | default dict -}}
|
|
{{- $given := .given -}}
|
|
{{- $path := .path | default "value" -}}
|
|
{{- if kindIs "invalid" $given -}}
|
|
{{- $given = dict -}}
|
|
{{- end -}}
|
|
{{- if not (kindIs "map" $given) -}}
|
|
{{- fail (printf "%s must be a mapping or null, got %s (%v)" $path (kindOf $given) $given) -}}
|
|
{{- end -}}
|
|
{{- $out := deepCopy $defaults -}}
|
|
{{- range $key, $value := $given -}}
|
|
{{- if not (kindIs "invalid" $value) -}}
|
|
{{- $default := index $defaults $key -}}
|
|
{{- if and (kindIs "map" $value) (kindIs "map" $default) -}}
|
|
{{- $_ := set $out $key (fromYaml (include "paperless-ngx.defaultedDict" (dict "defaults" $default "given" $value "path" (printf "%s.%s" $path $key)))) -}}
|
|
{{- else if kindIs "map" $default -}}
|
|
{{- fail (printf "%s.%s must be a mapping or null, got %s (%v)" $path $key (kindOf $value) $value) -}}
|
|
{{- else -}}
|
|
{{- $_ := set $out $key $value -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
{{- toYaml $out -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
The effective barcode settings, rendered as YAML (consume with fromYaml).
|
|
|
|
Input: .Values.paperless.files.barcodes (may be nil, empty or partial).
|
|
|
|
The defaults below MUST stay in sync with the barcodes block in values.yaml,
|
|
which stays the documented, user-facing place for them; this copy is only the
|
|
fallback for the case where an override has erased the block.
|
|
*/}}
|
|
{{- define "paperless-ngx.barcodes" -}}
|
|
{{- $defaults := dict
|
|
"enabled" true
|
|
"barcodeString" "PATCHT"
|
|
"retainBarcodePage" false
|
|
"asn" (dict "enabled" true) -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "paperless.files.barcodes") -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Pick one sub-block out of a values block that may itself be null.
|
|
|
|
Input: dict "block" <dict|nil> "key" <string> "path" <string>.
|
|
Returns YAML of dict "value" <the sub-block, may be nil>, after rejecting a
|
|
non-mapping block with a message that names the values path. Needed because
|
|
".Values.paperless.securityContext" can itself be erased.
|
|
*/}}
|
|
{{- define "paperless-ngx.subBlock" -}}
|
|
{{- $block := .block -}}
|
|
{{- if kindIs "invalid" $block -}}
|
|
{{- $block = dict -}}
|
|
{{- end -}}
|
|
{{- if not (kindIs "map" $block) -}}
|
|
{{- fail (printf "%s must be a mapping or null, got %s (%v)" .path (kindOf $block) $block) -}}
|
|
{{- end -}}
|
|
{{- toYaml (dict "value" (index $block .key)) -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
The effective security contexts, probes and resources of both workloads
|
|
(issue #99), built on the same defaultedDict the barcodes block already uses
|
|
(issue #61) - one mechanism, not two.
|
|
|
|
The defaults below MUST stay in sync with the corresponding blocks in
|
|
values.yaml, which stays the documented, user-facing place for them; this copy
|
|
is only the fallback for the case where an override has erased the block.
|
|
|
|
The pod context is consumed twice: as the pod's securityContext and by the
|
|
/run-ownership init container, which chowns the /run emptyDir to
|
|
runAsUser:runAsGroup because s6-overlay refuses to start unless /run is OWNED
|
|
by the runtime uid. Both reads go through this helper, so the chown can never
|
|
disagree with the context it is supposed to match.
|
|
*/}}
|
|
{{- define "paperless-ngx.podSecurityContext" -}}
|
|
{{- $given := (fromYaml (include "paperless-ngx.subBlock" (dict "block" . "key" "pod" "path" "paperless.securityContext"))).value -}}
|
|
{{- $defaults := dict
|
|
"runAsNonRoot" true
|
|
"runAsUser" 1000
|
|
"runAsGroup" 1000
|
|
"fsGroup" 1000
|
|
"fsGroupChangePolicy" "Always" -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" $given "path" "paperless.securityContext.pod") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.containerSecurityContext" -}}
|
|
{{- $given := (fromYaml (include "paperless-ngx.subBlock" (dict "block" . "key" "container" "path" "paperless.securityContext"))).value -}}
|
|
{{- $defaults := dict
|
|
"allowPrivilegeEscalation" false
|
|
"readOnlyRootFilesystem" true
|
|
"runAsNonRoot" true
|
|
"capabilities" (dict "drop" (list "ALL")) -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" $given "path" "paperless.securityContext.container") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.livenessProbe" -}}
|
|
{{- $defaults := dict
|
|
"httpGet" (dict "path" "/" "port" 8000)
|
|
"periodSeconds" 30
|
|
"failureThreshold" 3
|
|
"successThreshold" 1
|
|
"timeoutSeconds" 5 -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "paperless.livenessProbe") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.readinessProbe" -}}
|
|
{{- $defaults := dict
|
|
"httpGet" (dict "path" "/" "port" 8000)
|
|
"periodSeconds" 10
|
|
"failureThreshold" 3
|
|
"successThreshold" 1
|
|
"timeoutSeconds" 5 -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "paperless.readinessProbe") -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
The startup budget is deliberately large (10s x 60 = 600s): paperless runs
|
|
database migrations on startup, and since 3.0 the first start after an upgrade
|
|
also rebuilds the search index from scratch (Whoosh -> Tantivy).
|
|
*/}}
|
|
{{- define "paperless-ngx.startupProbe" -}}
|
|
{{- $defaults := dict
|
|
"httpGet" (dict "path" "/" "port" 8000)
|
|
"periodSeconds" 10
|
|
"failureThreshold" 60
|
|
"timeoutSeconds" 5 -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "paperless.startupProbe") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.effectiveResources" -}}
|
|
{{- $defaults := dict
|
|
"limitFactor" 3
|
|
"requests" (dict "cpu" 0.2 "memory" "1024Mi" "ephemeralStorage" "200Mi") -}}
|
|
{{- $merged := fromYaml (include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "paperless.resources")) -}}
|
|
{{- include "paperless-ngx.resources" $merged -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
The same three fallbacks for the SMB consume-mover CronJob. It has no probes
|
|
(documented exception), so only the two contexts and resources apply.
|
|
*/}}
|
|
{{- define "paperless-ngx.cron.podSecurityContext" -}}
|
|
{{- $given := (fromYaml (include "paperless-ngx.subBlock" (dict "block" . "key" "pod" "path" "smbConnection.cron.securityContext"))).value -}}
|
|
{{- $defaults := dict
|
|
"runAsNonRoot" true
|
|
"runAsUser" 1000
|
|
"runAsGroup" 1000
|
|
"fsGroup" 1000
|
|
"fsGroupChangePolicy" "Always" -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" $given "path" "smbConnection.cron.securityContext.pod") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.cron.containerSecurityContext" -}}
|
|
{{- $given := (fromYaml (include "paperless-ngx.subBlock" (dict "block" . "key" "container" "path" "smbConnection.cron.securityContext"))).value -}}
|
|
{{- $defaults := dict
|
|
"allowPrivilegeEscalation" false
|
|
"readOnlyRootFilesystem" true
|
|
"runAsNonRoot" true
|
|
"capabilities" (dict "drop" (list "ALL")) -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" $given "path" "smbConnection.cron.securityContext.container") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.cron.effectiveResources" -}}
|
|
{{- $defaults := dict
|
|
"limitFactor" 3
|
|
"requests" (dict "cpu" "10m" "memory" "16Mi" "ephemeralStorage" "10Mi") -}}
|
|
{{- $merged := fromYaml (include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "smbConnection.cron.resources")) -}}
|
|
{{- include "paperless-ngx.resources" $merged -}}
|
|
{{- end -}}
|