feat(paperless-ngx): optional valkey subchart, password from the redis item
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 12s
Validate Helm Charts / Build paperless-ngx (pull_request) Successful in 6s
Validate Helm Charts / Trivy config paperless-ngx (pull_request) Successful in 22s
Install-test / Install-test paperless-ngx (pull_request) Successful in 3m31s
Validate Helm Charts / Install-test (pull_request) Successful in 3m31s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
Publish Paperless-NGX Helm Chart / Publish Paperless-NGX Helm Chart (push) Successful in 10s
SonarQube Analysis / SonarQube Analysis (push) Successful in 33s

paperless-ngx cannot run without a Redis - PAPERLESS_REDIS is the Celery
broker - so every installation needed a second Helm release whose address
lived in a 1Password item and in neither chart. This adds our own published
valkey chart as an optional dependency, off by default and gated on
valkeySubchart.enabled (not valkey.enabled, which the subchart's strict
schema rejects and which Helm would only have complained about for whoever
first switched it on). Additive, default behaviour unchanged: minor,
12.1.3+up3.2.1 -> 12.2.0+up3.2.1.

Configured as a QUEUE (PVC + noeviction, the subchart's defaults), unlike
stalwart's cache. Read off the 3.2.1 source rather than assumed: one
PAPERLESS_REDIS url serves five consumers - the Celery broker, the result
backend (Redis in 3.x, the database before), the channels pub/sub layer and
two Django caches. allkeys-lru would not be a trade here but actively wrong,
because LRU evicts typeless across the whole keyspace and a backlog of
waiting consume tasks is exactly what it reads as cold; a dropped task is a
document silently never processed. The price, named in values.yaml rather
than smoothed over: two of the five consumers are caches, so a full instance
refuses writes - new tasks included - and the answer is memory sizing, not a
policy change. All three cache-like consumers are TTL-bounded, which is what
keeps that price small.

The password comes from the SAME item as the Redis connection, in a third
field auth-conf holding the complete line "requirepass <password>". It cannot
DEFAULT to secrets.redisSecretRef, and that is a property of Helm, not an
omission: a values.yaml is never templated and a parent cannot compute a
subchart's value. Measured - a template expression there lands verbatim in
the subchart's itemPath and fails as a YAML parse error naming neither the
value nor the cause.

Forgetting it is therefore caught in values.schema.json, by a conditional
rule rather than a fail() in a template. Also measured, and the reason:
Helm renders a subchart's templates BEFORE the parent's, so a guard in this
chart loses the race to the subchart's own required(), which reports
"secrets.authSecretRef" and cannot know the path belongs to the redis item.
A check that cannot fire is the mistake found while testing #77, so the
first draft of that guard was removed rather than shipped. The schema runs
before any rendering and names /valkey/secrets/authSecretRef.

One trap found while testing and closed here: "valkeySubchart: null" DELETES
the key, Helm then cannot resolve the condition path and treats the
dependency as ENABLED - so erasing the switch turned the subchart on with no
password set. Unlike every other block in this chart (issue #99), that one
is rejected by the schema instead of defaulting.

The install-test now runs with the subchart ON and the throwaway redis
Deployment removed: the url in the fixtures names ci-valkey-service, so the
bundled Valkey is the broker the release actually uses. #65 covered this
path by a render only - deliberately, since an unused Valkey would have made
that test look like it covered clustering. This is the first real
installation of the valkey-subchart path.

Documented, not glossed: the password now sits in that item twice and a
rotation has to touch both fields; with the subchart on the vault url is a
mandatory step, not a hint; and a OnePasswordItem materializes every field
of its item, so both Secrets carry all three keys - untidy, harmless, and
written down so nobody "tidies it up".

Fixes #78
This commit was merged in pull request #80.
This commit is contained in:
jklein committed 2026-10-02 21:24:22 +00:00
1 parent a288ee231a
commit a9ca5302de
10 files changed
+521 -57

No files matched your search

+78 -4
View File
@@ -3,6 +3,9 @@
{{- $barcodes := fromYaml (include "paperless-ngx.barcodes" .Values.paperless.files.barcodes) -}}
{{- $pod := printf "%s-%s-sfs-0" .Release.Name .Chart.Name -}}
{{- $scripts := .Values.paperless.scripts | default dict -}}
{{- $valkeySub := (.Values.valkeySubchart | default dict).enabled -}}
{{- $valkeyAuth := ((.Values.valkey | default dict).secrets | default dict).authSecretRef -}}
{{- $valkeySvc := printf "%s-valkey-service" .Release.Name -}}
{{- $pre := default (list) $scripts.preConsumptionScripts -}}
{{- $post := default (list) $scripts.postConsumptionScripts -}}
Paperless-ngx {{ .Chart.AppVersion }} — release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
@@ -70,6 +73,12 @@ WHAT THE 1PASSWORD ITEMS MUST CONTAIN
key-prefix PAPERLESS_REDIS_PREFIX — set this per release when
several releases share one Redis, or two instances
will consume each other's task queue
auth-conf the COMPLETE line "requirepass PASSWORD", not a
bare password. Read by the bundled Valkey only, so
it is {{ if $valkeySub }}REQUIRED for this release — see the Valkey
section below{{ else }}unused while valkeySubchart.enabled is
false, and the item needs it only once you switch
the bundled Valkey on{{ end }}.
secrets.paperlessSecretRef → {{ .Values.secrets.paperlessSecretRef }}
Secret {{ .Release.Name }}-{{ .Chart.Name }}-secret, fields:
@@ -117,9 +126,9 @@ CHANGING A SECRET'S CONTENT RESTARTS NOTHING
fields points the release at whatever that new endpoint contains, empty or
not, and the next start migrates it.
POSTGRES AND REDIS ARE NOT PART OF THIS CHART
Both have to exist and be reachable before the first start. They fail in
opposite ways, and only one of them is loud:
POSTGRES IS NOT PART OF THIS CHART{{ if not $valkeySub }}, AND NEITHER IS REDIS{{ end }}
{{ if $valkeySub }}The database{{ else }}Both{{ end }} ha{{ if $valkeySub }}s{{ else }}ve{{ end }} to exist and be reachable before the first start.{{ if not $valkeySub }} They fail in
opposite ways, and only one of them is loud:{{ end }}
- Without the database paperless does not come up at all. All three probes
ask for "/" on port 8000, so you get no useful error: the pod never turns
@@ -129,7 +138,72 @@ POSTGRES AND REDIS ARE NOT PART OF THIS CHART
broker: consumption, OCR, the scheduled tasks and the index updates simply
never run, and documents dropped into the consume directory stay there.
A wrong redis-connection-url costs you the whole processing side and
nothing turns red.
nothing turns red.{{ if $valkeySub }} This release brings its own Valkey (below), so
the address is in your hands — which is exactly why the url is the one
thing to check first.{{ end }}
{{ if $valkeySub }}
THIS RELEASE BRINGS ITS OWN VALKEY — ONE MANDATORY STEP IS IN THE VAULT
valkeySubchart.enabled is true, so the release contains a Valkey of its own:
StatefulSet {{ .Release.Name }}-valkey-sfs, Service {{ $valkeySvc }}, its own
NetworkPolicy, and a PersistentVolumeClaim (persistence is on — this is a
queue, see below).
1. THE CONNECTION URL MUST POINT AT IT. This is a step, not a hint: the url
lives in the vault and the chart cannot write it. In the item
"{{ .Values.secrets.redisSecretRef }}", field redis-connection-url:
redis://:PASSWORD@{{ $valkeySvc }}:6379
Pointing elsewhere is not an error anywhere: paperless talks to that other
Redis, the Valkey next to it runs along unused, and nothing reports it.
2. THE PASSWORD IS IN THE ITEM TWICE — once inside redis-connection-url and
once in the "requirepass PASSWORD" line of auth-conf. Both are the same
password, and a rotation has to touch BOTH fields. Change only one and
paperless can no longer authenticate against its own Valkey. After any
such change, restart both workloads by hand (Helm cannot see inside a
1Password item, so nothing rolls automatically):
kubectl -n {{ .Release.Namespace }} rollout restart statefulset/{{ .Release.Name }}-{{ .Chart.Name }}-sfs
kubectl -n {{ .Release.Namespace }} rollout restart statefulset/{{ .Release.Name }}-valkey-sfs
3. AUTH-CONF IS A CONFIGURATION LINE, NOT A PASSWORD. The Valkey mounts the
field as a file and includes it, so the password never reaches a process
list or an environment variable. An init container refuses to start the
pod unless a line beginning with "requirepass " is really there — without
that check a typo would leave a Valkey running with NO password and
looking perfectly healthy.
{{- if and $valkeyAuth (ne ($valkeyAuth | toString) ($.Values.secrets.redisSecretRef | toString)) }}
NOTE: valkey.secrets.authSecretRef points at a DIFFERENT item than
secrets.redisSecretRef:
valkey.secrets.authSecretRef → {{ $valkeyAuth }}
secrets.redisSecretRef → {{ .Values.secrets.redisSecretRef }}
That is supported and may well be deliberate — the bundled Valkey is then fed
from its own item. It is only worth knowing that you now keep TWO items in
step: the password in auth-conf over there and the one inside
redis-connection-url here have to stay identical.
{{- end }}
BOTH ITEMS' FIELDS LAND IN BOTH SECRETS. A OnePasswordItem materializes every
field of the item it names, so with one shared item the Valkey's Secret also
carries redis-connection-url and key-prefix, and this chart's redis Secret
also carries auth-conf. Untidy and harmless — nothing mounts a field it does
not reference. It is mentioned so it does not get "cleaned up".
IT IS CONFIGURED AS A QUEUE, and that is a decision: persistence on, eviction
policy "noeviction". PAPERLESS_REDIS is the Celery broker, so its contents are
work that must not vanish — "allkeys-lru" would evict across the whole
keyspace and a backlog of waiting tasks is exactly what LRU reads as cold.
The price: the same instance also serves the Django cache, so when it fills,
writes are REFUSED rather than old keys dropped — new tasks included. The
answer is to give it more memory (valkey.valkey.resources.requests.memory),
never to switch the policy. values.yaml argues this in full.
SCALEDOWN DOES NOT REACH IT. scaleDown: true scales paperless to zero and
leaves the Valkey running, which is the right way round for a queue: the
backlog stays where it is.
{{- end }}
THE FIRST START MIGRATES, AND THE STARTUP PROBE IS THE CLOCK
Every start runs the database migrations before the server answers, and the