jklein
4b2bde505d
Three issues on one chart, bundled because they share values.yaml, values.schema.json and the version line, and because the first half of #11 lives in the same template as #101. Gitea #11, first half: the consume CronJob's schedule was rendered bare, so a schedule in the commonest cron notation there is - a star, a slash, an interval - aborted the render with a YAML parse error pointing at the template while the operator's value appeared nowhere in the message. Quoting it in the values file does not help; Helm passes the content through. Reproduced before the fix, at the line the issue names, and fixed with | quote. Gitea #11, second half: secrets.oidcSecretRef is removed, values and schema both. No template ever read it, so no OnePasswordItem was created and nothing reached the pod - the quiet kind of defect that looks like successful configuration. It is not reimplemented, and the reason is in paperless: the whole OIDC configuration travels in one json environment variable with no _FILE variant, so a secretKeyRef cannot substitute a fragment into it. The operational question is answered by #83 - Keycloak marks a client public per client, so nothing forces a confidential one here. In the same breath, oidc.provider.tokenAuthMethod is restricted to "none". Its comment named client_secret_basic as the example, which asks the provider for an authentication this chart cannot perform: it configures no client secret and cannot. The two values read together looked like a working secret setup, and the symptom was a login dying at the callback with invalid_client. No live release sets it (measured against all three instances). Gitea #112 asked for an extraEnv and the chart already had the standard additive env block - the issue was measured against a deployed 12.2.0 and read only the top-level keys. What was actually broken is the block itself: it rendered its operand bare, so a "*"-leading value aborted the render and "true" or "0755" arrived as a bool or an integer, which the API server rejects after a green render. It is brought to the repository's documented quoted form (template-chart carries the rule), which is what makes an arbitrary PAPERLESS_* variable settable in practice. PAPERLESS_LOGGING_DIR is documented as the worked example, with the cross-reference it needs: the root cause is the NFS-exported RWX data volume in jk-cluster-config #360 and this does not fix it - it only moves the log off that volume. The two Ingress upload-size annotations had the same bare render and are quoted too. Gitea #101: the consume mover now runs on the node of the StatefulSet pod, via a required podAffinity on its label. It mounts the RWX consume volume, and Longhorn's CSI driver refuses that mount unless it finds a workload pod in Pending, so a mover on another node left the StatefulSet node's kubelet retrying every two minutes - 291 events in 10 hours in both releases with SMB on, none in the release without the CronJob. A static nodeSelector would be wrong because the StatefulSet pod is not pinned either, and preferred would reproduce the bug more rarely and harder to find. The cost of a required term is handled rather than left open: concurrencyPolicy: Forbid, a startingDeadlineSeconds that also keeps the controller away from its 100-missed-schedules stop, and an activeDeadlineSeconds that fails a job which cannot be placed instead of leaving one Pending pod per schedule. scaleDown already suspends the CronJob; what remains is paperless.replicas: 0 with SMB on, which now reports a bounded, visible failure. successfulJobsHistoryLimit is deliberately untouched: Longhorn's check asks for the existence of a Pending pod, not the absence of Succeeded ones, so those three pods are the error message's evidence and not its cause. Side defect fixed on the way: every read of smbConnection.cron dereferenced the block directly, so an override that had erased it aborted with a nil-pointer error naming an internal template path instead of falling back to the defaults this chart promises everywhere else. Measured on origin/main; all reads now go through a merged-defaults helper that preserves an explicit false. Major: a value is removed and fails the strict schema afterwards, and runtime placement changes. Fixes #11 Fixes #101 Fixes #112
381 lines
16 KiB
Smarty
381 lines
16 KiB
Smarty
{{/*
|
|
Render the body of a container "resources:" block from a values dict of the form
|
|
|
|
resources:
|
|
limitFactor: 3 # optional, default 3
|
|
requests: { cpu, memory, ephemeral-storage / ephemeralStorage, ... }
|
|
limits: { ... } # optional, every entry optional
|
|
|
|
Rules (variant B, see issue #19):
|
|
- requests are rendered as given (camelCase "ephemeralStorage" is normalized
|
|
to the Kubernetes key "ephemeral-storage").
|
|
- an explicitly set limit always wins.
|
|
- a missing memory / ephemeral-storage limit is computed as
|
|
limitFactor x the corresponding request.
|
|
- a missing cpu limit is NOT set at all (the kubelet then treats CPU as
|
|
unlimited; CPU is compressible, so this cannot kill the workload).
|
|
|
|
Usage (the values path to the resources dict varies per chart):
|
|
resources:
|
|
{{- include "paperless-ngx.resources" .Values.<app>.resources | nindent 12 }}
|
|
|
|
NOTE: this helper is duplicated into every chart's templates/_helpers.tpl with
|
|
the chart's name as define prefix, because sharing it would mean a LIBRARY
|
|
chart, and a dependency is admissible but expensive (README, guideline 14).
|
|
This chart does take one - the optional valkey subchart - which changes
|
|
nothing here: a library chart for one helper would still be the wrong trade,
|
|
and the subchart carries its own copy of this helper anyway. This file in
|
|
template-chart is the reference implementation.
|
|
*/}}
|
|
{{- define "paperless-ngx.resources" -}}
|
|
{{- $resources := . | default dict -}}
|
|
{{- $factor := 3.0 -}}
|
|
{{- if not (kindIs "invalid" $resources.limitFactor) -}}
|
|
{{- $factor = float64 $resources.limitFactor -}}
|
|
{{- end -}}
|
|
{{- $requests := dict -}}
|
|
{{- range $key, $value := ($resources.requests | default dict) -}}
|
|
{{- if not (kindIs "invalid" $value) -}}
|
|
{{- $_ := set $requests (include "paperless-ngx.resourceKey" $key) $value -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
{{- $limits := dict -}}
|
|
{{- range $key, $value := ($resources.limits | default dict) -}}
|
|
{{- if not (kindIs "invalid" $value) -}}
|
|
{{- $_ := set $limits (include "paperless-ngx.resourceKey" $key) $value -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
{{- range $key, $request := $requests -}}
|
|
{{- if and (ne $key "cpu") (not (hasKey $limits $key)) -}}
|
|
{{- $_ := set $limits $key (include "paperless-ngx.multiplyQuantity" (dict "quantity" $request "factor" $factor)) -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
{{- $out := dict -}}
|
|
{{- with $requests }}{{- $_ := set $out "requests" . }}{{- end -}}
|
|
{{- with $limits }}{{- $_ := set $out "limits" . }}{{- end -}}
|
|
{{- toYaml $out -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Normalize a resource name from values to its Kubernetes key.
|
|
*/}}
|
|
{{- define "paperless-ngx.resourceKey" -}}
|
|
{{- if eq . "ephemeralStorage" -}}ephemeral-storage{{- else -}}{{ . }}{{- end -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Multiply a Kubernetes quantity by a factor, keeping its suffix.
|
|
Input: dict "quantity" <string|number> "factor" <float64>.
|
|
Handles plain numbers ("2", "0.5", 1073741824), decimal ("1.5Gi"), binary
|
|
("512Mi", "1Gi", "10Ki") and milli/SI suffixes ("100m", "500M"). Integer
|
|
results are printed without a decimal point.
|
|
*/}}
|
|
{{- define "paperless-ngx.multiplyQuantity" -}}
|
|
{{- $quantity := include "paperless-ngx.formatNumber" .quantity -}}
|
|
{{- $number := regexFind "^[0-9]+(\\.[0-9]+)?" $quantity -}}
|
|
{{- if not $number -}}
|
|
{{- fail (printf "cannot parse resource quantity %q" $quantity) -}}
|
|
{{- end -}}
|
|
{{- $suffix := trimPrefix $number $quantity -}}
|
|
{{- if not (regexMatch "^(m|k|M|G|T|P|E|Ki|Mi|Gi|Ti|Pi|Ei)?$" $suffix) -}}
|
|
{{- fail (printf "unsupported resource quantity suffix %q in %q" $suffix $quantity) -}}
|
|
{{- end -}}
|
|
{{- $product := mulf (float64 $number) .factor -}}
|
|
{{- if eq $product (floor $product) -}}
|
|
{{- printf "%d%s" (int64 $product) $suffix -}}
|
|
{{- else -}}
|
|
{{- printf "%s%s" (include "paperless-ngx.formatNumber" $product) $suffix -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Render a number as a plain decimal string, never in exponential notation.
|
|
|
|
Why this exists (issue #157): the null-safe fallback helpers pass values
|
|
through fromYaml, which parses every number as float64. Go's "%v" switches
|
|
float64 to exponential notation from about 1e6 upwards, so a bare byte count
|
|
like 1048576 became "1.048576e+06" - not a valid Kubernetes quantity, and the
|
|
render aborted. Strings ("512Mi") are passed through untouched.
|
|
*/}}
|
|
{{- define "paperless-ngx.formatNumber" -}}
|
|
{{- if kindIs "float64" . -}}
|
|
{{- regexReplaceAll "\\.?0+$" (printf "%.10f" .) "" -}}
|
|
{{- else -}}
|
|
{{- printf "%v" . -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Render the replica count for a Deployment/StatefulSet.
|
|
|
|
Input: dict
|
|
"scaleDown" the chart-global .Values.scaleDown flag (may be nil)
|
|
"replicas" the workload's replicas value (may be nil)
|
|
|
|
Rules (issue #25):
|
|
- scaleDown: true takes precedence and always yields 0.
|
|
- otherwise an explicitly set replicas value is used as given, including an
|
|
explicit 0 — Helm's "default" would swallow 0, hence the explicit
|
|
kindIs "invalid" nil check.
|
|
- a missing replicas value defaults to 1.
|
|
|
|
Usage (the values path to the workload's replicas varies per chart):
|
|
replicas: {{ include "paperless-ngx.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.<app>.replicas) }}
|
|
|
|
NOTE: like the resources helper above, this is duplicated into every chart's
|
|
templates/_helpers.tpl with the chart's name as define prefix. This file in
|
|
template-chart is the reference implementation.
|
|
*/}}
|
|
{{- define "paperless-ngx.replicas" -}}
|
|
{{- if .scaleDown -}}
|
|
0
|
|
{{- else if not (kindIs "invalid" .replicas) -}}
|
|
{{- .replicas -}}
|
|
{{- else -}}
|
|
1
|
|
{{- end -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Merge a user-supplied values sub-tree onto a dict of chart defaults so that a
|
|
null / empty override falls back to the defaults instead of erasing them.
|
|
|
|
Input: dict "defaults" <dict> "given" <dict|nil> "path" <string, optional
|
|
values path used in error messages>. Output: YAML of the merged dict (consume
|
|
it with fromYaml).
|
|
|
|
Why this exists (issue #61): Helm merges values maps recursively, but an
|
|
override key written without children — e.g. a leftover
|
|
|
|
barcodes:
|
|
|
|
after deleting its last remaining sub-key — is YAML null and replaces the
|
|
whole default sub-tree. The chart then renders with no barcode settings at
|
|
all. This helper defines the opposite semantics: an empty block means "chart
|
|
defaults apply".
|
|
|
|
Rules:
|
|
- a nil "given" is treated as an empty dict (all defaults apply).
|
|
- a key whose override value is null keeps the default (nulls never delete).
|
|
- a key present in both as a map is merged recursively, so a partially
|
|
filled block only overrides the keys it actually sets.
|
|
- a scalar override for a key the defaults define as a map is rejected with
|
|
a message naming the values path, instead of failing later with a template
|
|
field error.
|
|
- any other set value wins as given, including an explicit false or 0 —
|
|
Helm's "default" and sprig's "mergeOverwrite" would both swallow those
|
|
zero values, hence the explicit kindIs "invalid" nil check.
|
|
- keys only present in the override are passed through unchanged.
|
|
*/}}
|
|
{{- define "paperless-ngx.defaultedDict" -}}
|
|
{{- $defaults := .defaults | default dict -}}
|
|
{{- $given := .given -}}
|
|
{{- $path := .path | default "value" -}}
|
|
{{- if kindIs "invalid" $given -}}
|
|
{{- $given = dict -}}
|
|
{{- end -}}
|
|
{{- if not (kindIs "map" $given) -}}
|
|
{{- fail (printf "%s must be a mapping or null, got %s (%v)" $path (kindOf $given) $given) -}}
|
|
{{- end -}}
|
|
{{- $out := deepCopy $defaults -}}
|
|
{{- range $key, $value := $given -}}
|
|
{{- if not (kindIs "invalid" $value) -}}
|
|
{{- $default := index $defaults $key -}}
|
|
{{- if and (kindIs "map" $value) (kindIs "map" $default) -}}
|
|
{{- $_ := set $out $key (fromYaml (include "paperless-ngx.defaultedDict" (dict "defaults" $default "given" $value "path" (printf "%s.%s" $path $key)))) -}}
|
|
{{- else if kindIs "map" $default -}}
|
|
{{- fail (printf "%s.%s must be a mapping or null, got %s (%v)" $path $key (kindOf $value) $value) -}}
|
|
{{- else -}}
|
|
{{- $_ := set $out $key $value -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
{{- end -}}
|
|
{{- toYaml $out -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
The effective barcode settings, rendered as YAML (consume with fromYaml).
|
|
|
|
Input: .Values.paperless.files.barcodes (may be nil, empty or partial).
|
|
|
|
The defaults below MUST stay in sync with the barcodes block in values.yaml,
|
|
which stays the documented, user-facing place for them; this copy is only the
|
|
fallback for the case where an override has erased the block.
|
|
*/}}
|
|
{{- define "paperless-ngx.barcodes" -}}
|
|
{{- $defaults := dict
|
|
"enabled" true
|
|
"barcodeString" "PATCHT"
|
|
"retainBarcodePage" false
|
|
"asn" (dict "enabled" true) -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "paperless.files.barcodes") -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
Pick one sub-block out of a values block that may itself be null.
|
|
|
|
Input: dict "block" <dict|nil> "key" <string> "path" <string>.
|
|
Returns YAML of dict "value" <the sub-block, may be nil>, after rejecting a
|
|
non-mapping block with a message that names the values path. Needed because
|
|
".Values.paperless.securityContext" can itself be erased.
|
|
*/}}
|
|
{{- define "paperless-ngx.subBlock" -}}
|
|
{{- $block := .block -}}
|
|
{{- if kindIs "invalid" $block -}}
|
|
{{- $block = dict -}}
|
|
{{- end -}}
|
|
{{- if not (kindIs "map" $block) -}}
|
|
{{- fail (printf "%s must be a mapping or null, got %s (%v)" .path (kindOf $block) $block) -}}
|
|
{{- end -}}
|
|
{{- toYaml (dict "value" (index $block .key)) -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
The effective security contexts, probes and resources of both workloads
|
|
(issue #99), built on the same defaultedDict the barcodes block already uses
|
|
(issue #61) - one mechanism, not two.
|
|
|
|
The defaults below MUST stay in sync with the corresponding blocks in
|
|
values.yaml, which stays the documented, user-facing place for them; this copy
|
|
is only the fallback for the case where an override has erased the block.
|
|
|
|
The pod context is consumed twice: as the pod's securityContext and by the
|
|
/run-ownership init container, which chowns the /run emptyDir to
|
|
runAsUser:runAsGroup because s6-overlay refuses to start unless /run is OWNED
|
|
by the runtime uid. Both reads go through this helper, so the chown can never
|
|
disagree with the context it is supposed to match.
|
|
*/}}
|
|
{{- define "paperless-ngx.podSecurityContext" -}}
|
|
{{- $given := (fromYaml (include "paperless-ngx.subBlock" (dict "block" . "key" "pod" "path" "paperless.securityContext"))).value -}}
|
|
{{- $defaults := dict
|
|
"runAsNonRoot" true
|
|
"runAsUser" 1000
|
|
"runAsGroup" 1000
|
|
"fsGroup" 1000
|
|
"fsGroupChangePolicy" "Always" -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" $given "path" "paperless.securityContext.pod") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.containerSecurityContext" -}}
|
|
{{- $given := (fromYaml (include "paperless-ngx.subBlock" (dict "block" . "key" "container" "path" "paperless.securityContext"))).value -}}
|
|
{{- $defaults := dict
|
|
"allowPrivilegeEscalation" false
|
|
"readOnlyRootFilesystem" true
|
|
"runAsNonRoot" true
|
|
"capabilities" (dict "drop" (list "ALL")) -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" $given "path" "paperless.securityContext.container") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.livenessProbe" -}}
|
|
{{- $defaults := dict
|
|
"httpGet" (dict "path" "/" "port" 8000)
|
|
"periodSeconds" 30
|
|
"failureThreshold" 3
|
|
"successThreshold" 1
|
|
"timeoutSeconds" 5 -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "paperless.livenessProbe") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.readinessProbe" -}}
|
|
{{- $defaults := dict
|
|
"httpGet" (dict "path" "/" "port" 8000)
|
|
"periodSeconds" 10
|
|
"failureThreshold" 3
|
|
"successThreshold" 1
|
|
"timeoutSeconds" 5 -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "paperless.readinessProbe") -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
The startup budget is deliberately large (10s x 60 = 600s): paperless runs
|
|
database migrations on startup, and since 3.0 the first start after an upgrade
|
|
also rebuilds the search index from scratch (Whoosh -> Tantivy).
|
|
*/}}
|
|
{{- define "paperless-ngx.startupProbe" -}}
|
|
{{- $defaults := dict
|
|
"httpGet" (dict "path" "/" "port" 8000)
|
|
"periodSeconds" 10
|
|
"failureThreshold" 60
|
|
"timeoutSeconds" 5 -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "paperless.startupProbe") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.effectiveResources" -}}
|
|
{{- $defaults := dict
|
|
"limitFactor" 3
|
|
"requests" (dict "cpu" 0.2 "memory" "1024Mi" "ephemeralStorage" "200Mi") -}}
|
|
{{- $merged := fromYaml (include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "paperless.resources")) -}}
|
|
{{- include "paperless-ngx.resources" $merged -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
The same three fallbacks for the SMB consume-mover CronJob. It has no probes
|
|
(documented exception), so only the two contexts and resources apply.
|
|
*/}}
|
|
{{- define "paperless-ngx.cron.podSecurityContext" -}}
|
|
{{- $given := (fromYaml (include "paperless-ngx.subBlock" (dict "block" . "key" "pod" "path" "smbConnection.cron.securityContext"))).value -}}
|
|
{{- $defaults := dict
|
|
"runAsNonRoot" true
|
|
"runAsUser" 1000
|
|
"runAsGroup" 1000
|
|
"fsGroup" 1000
|
|
"fsGroupChangePolicy" "Always" -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" $given "path" "smbConnection.cron.securityContext.pod") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.cron.containerSecurityContext" -}}
|
|
{{- $given := (fromYaml (include "paperless-ngx.subBlock" (dict "block" . "key" "container" "path" "smbConnection.cron.securityContext"))).value -}}
|
|
{{- $defaults := dict
|
|
"allowPrivilegeEscalation" false
|
|
"readOnlyRootFilesystem" true
|
|
"runAsNonRoot" true
|
|
"capabilities" (dict "drop" (list "ALL")) -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" $given "path" "smbConnection.cron.securityContext.container") -}}
|
|
{{- end -}}
|
|
|
|
{{- define "paperless-ngx.cron.effectiveResources" -}}
|
|
{{- $defaults := dict
|
|
"limitFactor" 3
|
|
"requests" (dict "cpu" "10m" "memory" "16Mi" "ephemeralStorage" "10Mi") -}}
|
|
{{- $merged := fromYaml (include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "smbConnection.cron.resources")) -}}
|
|
{{- include "paperless-ngx.resources" $merged -}}
|
|
{{- end -}}
|
|
|
|
{{/*
|
|
The whole smbConnection.cron block, merged onto its defaults (YAML; consume
|
|
with fromYaml).
|
|
|
|
Input: .Values.smbConnection.cron (may be nil, empty or partial).
|
|
|
|
WHY EVERY READ OF THAT BLOCK GOES THROUGH HERE. The CronJob template used to
|
|
dereference .Values.smbConnection.cron.<key> directly, so an override that had
|
|
erased the block - the leftover childless
|
|
|
|
cron:
|
|
|
|
that issue #61 is about - did not fall back to the defaults the way this chart
|
|
promises everywhere else. It aborted the render with
|
|
|
|
nil pointer evaluating interface {}.securityContext
|
|
|
|
naming an internal template path and neither the value nor the cause. Measured
|
|
on origin/main before 13.0.0, so this is a fix and not a precaution.
|
|
|
|
The scalar defaults below MUST stay in sync with the smbConnection.cron block
|
|
in values.yaml, which stays the documented, user-facing place for them; this
|
|
copy is only the fallback. The sub-blocks (securityContext, resources, env)
|
|
keep their own defaults in the helpers above and are only passed through here.
|
|
|
|
An explicit "pinToServerNode: false" survives this, which is the whole reason
|
|
it is not a "default true" pipe: sprig's default would read false as unset and
|
|
silently turn the pinning back on - the one value whose explicit false has to
|
|
be obeyed, since it exists to switch off a hard placement rule.
|
|
*/}}
|
|
{{- define "paperless-ngx.cron.scheduling" -}}
|
|
{{- $defaults := dict
|
|
"pinToServerNode" true
|
|
"startingDeadlineSeconds" 600
|
|
"activeDeadlineSeconds" 3600 -}}
|
|
{{- include "paperless-ngx.defaultedDict" (dict "defaults" $defaults "given" . "path" "smbConnection.cron") -}}
|
|
{{- end -}}
|