valkey: one instance per application, replacing the shared cluster-redis (1.0.0+up9.1.2.alpine) #53

Merged

jklein merged 1 commits from feat/valkey-chart into main 2026-09-27 16:14:59 +02:00

Conversation 3 Commits 1 Files Changed 19

+1687

jlab-bot commented 2026-09-27 13:05:27 +02:00

Member

Copy link

Copy source

Fixes #46

A new chart valkey for one instance per application, replacing the shared cluster-redis. The four decisions of the plan comments are implemented as decided, with decision 1 in its revised form (appVersion: 9.1.2-alpine, chart 1.0.0+up9.1.2.alpine, no versioningTemplate).

The two things that were measured, not assumed

1. The uid: 999, and where it comes from

Confirmed by reading the upstream build, not by trying it: valkey-io/valkey-container, 9.1/alpine/Dockerfile, final stage:

addgroup -S -g 1000 valkey; \
adduser -S -G valkey -u 999 valkey

(gid 1000, not 999, with the upstream comment "alpine already has a gid 999, so we'll use the next id".) There is no USER directive, so the image starts as root, and docker-entrypoint.sh steps down itself:

if [ "$1" = 'valkey-server' ]; then
  if [ "$(id -u)" = '0' ]; then
    find . \! -user valkey -exec chown valkey '{}' +
    exec setpriv --reuid=valkey --regid=valkey --clear-groups -- "$0" "$@"

Two corrections to the plan text, both harmless but worth having right: on the alpine variant the step-down is setpriv, not gosu (that is the Debian flavour), and the chown /data in that branch is the part the chart skips by running as uid 999 directly. That is why fsGroup: 1000 is not decoration — it is the only thing that makes the volume writable. /data is created mode 1777 in the image, so the non-persistent path is writable regardless.

Also read off the same Dockerfile and relevant to the password: the image patches protected-mode off (sed -ri 's!...createBoolConfig("protected-mode", ... 1 ...!...0...!'). "No password" therefore really does mean "reachable from the whole pod network", which is what the init container exists to prevent.

2. The probe: valkey-cli ping really does exit 0 on NOAUTH

From valkey 9.1.2, src/valkey-cli.c. An error reply becomes a non-zero exit in exactly one place:

} else if (!config.interactive && config.set_errcode && reply->type == VALKEY_REPLY_ERROR) {
    fprintf(stderr, "%s\n", reply->str);
    exit(1);

and config.set_errcode is set by nothing but -e ("-e Return exit error code when command execution fails."). So a probe written as valkey-cli ping prints NOAUTH Authentication required. and exits 0 — the always-green assertion. Second finding from the same file: with VALKEYCLI_AUTH set to a wrong password, cliAuth fails and prints AUTH failed: WRONGPASS ..., but main ignores cliConnect's return value, so the command is sent anyway and the failure arrives as the NOAUTH reply — i.e. the same path. The auth env var is VALKEYCLI_AUTH, with REDISCLI_AUTH still honoured as the legacy spelling (#define CLI_AUTH_ENV / OLD_CLI_AUTH_ENV).

The probe is therefore VALKEYCLI_AUTH="$(…)" valkey-cli -e -p 6379 ping | grep -qx PONG, with the password read out of the same mounted auth file the server was configured from (never -a, never in argv).

Proof that it can fail — local, with valkey-cli stubbed (the stub mimics the source above; the run also greps every output for the password and found none):

case	exit
right password, -e	0
extra whitespace after requirepass	0
double-quoted password	0
wrong password	1
no requirepass line at all	1
wrong password, without -e	1 (the grep catches it)
right password, without -e	0

The last two rows are why both halves are in the command and why the comment in the ConfigMap says so precisely: grep -qx PONG alone would already catch an error reply, and -e is what puts the server's own message (NOAUTH…, WRONGPASS…) on stderr and into the probe's failure output instead of silently on stdout.

The wrong-password test against a real server has been run on this PR — see the evidence comment below. Summary: with readiness authenticating with a deliberately wrong password, the pod stayed 0/1 Running with Readiness probe failed: AUTH failed: WRONGPASS ... / NOAUTH Authentication required. and RESTARTS 0, and the install-test went red; the two temporary experiment commits have been removed from the branch and the final run is green. There is no Docker daemon in the work environment, so that run is the only runtime proof there can be.

What the init container rejects

check-auth.sh (in the ConfigMap, run by an init container from the same image, as uid 999 with everything dropped) greps the mounted file for ^requirepass[[:space:]][[:space:]]*[^[:space:]]. Verified locally against the real script:

auth file content	result
requirepass <pw>	passes
requirepass <pw> on the second line	passes
requirepas <pw> (misspelled)	rejected
requirepass (no value)	rejected
requirepass (whitespace only)	rejected
empty field	rejected
requirepass <pw> (leading space — Valkey ignores it too)	rejected
file missing entirely	rejected

It never prints the file; the message names the 1Password field instead. What it deliberately does not do is compose the line — the vault stays the only place the password exists in.

The rest of the implementation

• StatefulSet, one replica (valkey.replicas capped at 1 by the schema — two pods would fight over one RWO volume and clients behind the Service would silently see two datasets), ClusterIP Service, chart-global scaleDown. No Ingress and no value to add one.
• Password: 1Password field auth-conf, holding the finished line requirepass <password>, mounted as a file and included by the config. The include is rendered last, so nothing in valkey.extraConfig can override it — stated in values.yaml as a property, not as advice.
• checksum/config over the ConfigMap and checksum/secret-ref over the rendered OnePasswordItem. operator.1password.io/auto-restart is not set (silent no-op on a StatefulSet).
• maxmemory derived from the rendered memory limit (read back out of the resources helper, so it cannot drift from what the container gets): maxmemoryFactor default 0.75. A configured valkey.maxmemory above the limit aborts the render with both byte counts named; 0 aborts too, because in Valkey that means unlimited. Defaults render maxmemory 603979776 against a 768Mi limit.
• Persistence: longhorn-encrypted-ns, 1Gi, appendonly a value with the reason in the comment. With persistence.enabled: false the chart renders appendonly no and save "", so a pod without a volume writes nothing to ephemeral storage. The two sensible pairs are documented at the top of values.yaml, and "no persistence + noeviction" gets a warning both there and in the rendered NOTES.txt.
• NetworkPolicy on by default, networkPolicy.allowedNamespaces for consumers elsewhere, and the note that it is only a boundary where the CNI enforces policies. Ingress-only on purpose (this instance opens no outbound connection).
• Hardening: runAsNonRoot, 999/1000, fsGroup 1000, readOnlyRootFilesystem, all caps dropped, seccompProfile: RuntimeDefault in the pod context so the init container is covered. /data is the only writable path — no extra emptyDir was needed, because the server writes nowhere else (logs go to stdout, and neither pidfile nor unixsocket is configured).
• Resources per convention: requests as defaults, memory/ephemeral-storage limits computed, no CPU limit.
• values.schema.json with additionalProperties: false at every chart-owned level; maxmemoryPolicy is an enum of the eight real policies.

Verification

Run locally, all green:

• helm lint --strict valkey (defaults and with the CI values)
• render with only the required value; with persistence.enabled=false; with scaleDown: true (replicas: 0); with valkey.maxmemory=2Gi → render aborts: "valkey.maxmemory is 2Gi (2147483648 bytes) and thus ABOVE the container's memory limit (805306368 bytes)…"; with --set-string valkey.maxmemory=0 → aborts as unlimited
• null-safety: valkey.securityContext=null, valkey.resources=null and all three probes =null fall back to the chart's own defaults (uid 999 included)
• schema: persistence.enable typo, valkey.replicas=2, an unknown maxmemoryPolicy and a Valkey-style 512mb are each rejected by name
• ci/run-quantity-render-test.sh valkey — both cases exercise the helper; and the whole repo still passes (24 exercised, 0 failed). The fraction case also checked by hand: the derived maxmemory renders as 1124998, not in exponential notation
• helm package valkey → valkey-1.0.0+up9.1.2.alpine.tgz
• NOTES.txt rendered in all its branches (queue, cache, the mistaken pair, scaled to zero, NetworkPolicy on/off) with the scratch-ConfigMap trick from the README, since helm template never renders it

What I could not verify

Said in those words, because the work environment has neither a Docker daemon nor node:

• The install-test is the only runtime proof, and it has now run (see the evidence comment). Nothing was started in the work environment itself: not the server, not the init container, not the probes against a real valkey-cli. The stub-based table above proves the scripts, the source quotes prove valkey-cli, and the three cluster runs prove the two meeting in one pod.
• The install-test can only cover one variant. It runs the persistent path (PVC, fsGroup, AOF), which is the risky one and the chart's default. The non-persistent path is rendered locally only and is a known gap in exactly those words.
• fsGroup itself is not proven by the install-test. k3d's local-path volumes are node-directory backed and created world-writable, so the container can write whether or not the kubelet applied fsGroup. The test proves writability; the ownership mechanism is unverified. A hostPath PV fixture would have been worse — the kubelet applies fsGroup to those not at all.
• Whether the NetworkPolicy filters anything is a property of the cluster's CNI. The test only shows the object is accepted and does not break the pod's own traffic.
• The Renovate block is unproven by a dry run (no node/npx here, as established in #34). It needs none of the usual guard, though, and that is the whole reason decision 1 was revised: plain docker versioning with no capture groups and a suffix that never changes. First evidence will be valkey appearing on the dependency dashboard.
• The rotated-password self-healing described in values.yaml and NOTES.txt (kubelet refreshes the mounted file → liveness fails → container restarts with the new password) is labelled UNVERIFIED there, and the manual rollout restart stays the documented procedure.

Wiki

valkey page added (installation, the 1Password field, the full values reference, the traps), plus a row in Charts and an entry in _Sidebar.md.

Fixes #46 A new chart `valkey` for one instance per application, replacing the shared `cluster-redis`. The four decisions of the plan comments are implemented as decided, with decision 1 in its revised form (`appVersion: 9.1.2-alpine`, chart `1.0.0+up9.1.2.alpine`, no `versioningTemplate`). ## The two things that were measured, not assumed ### 1. The uid: 999, and where it comes from Confirmed by reading the upstream build, not by trying it: `valkey-io/valkey-container`, `9.1/alpine/Dockerfile`, final stage: ``` addgroup -S -g 1000 valkey; \ adduser -S -G valkey -u 999 valkey ``` (gid 1000, not 999, with the upstream comment *"alpine already has a gid 999, so we'll use the next id"*.) There is no `USER` directive, so the image starts as root, and `docker-entrypoint.sh` steps down itself: ```sh if [ "$1" = 'valkey-server' ]; then if [ "$(id -u)" = '0' ]; then find . \! -user valkey -exec chown valkey '{}' + exec setpriv --reuid=valkey --regid=valkey --clear-groups -- "$0" "$@" ``` Two corrections to the plan text, both harmless but worth having right: on the **alpine** variant the step-down is `setpriv`, not `gosu` (that is the Debian flavour), and the `chown /data` in that branch is the part the chart skips by running as uid 999 directly. That is why `fsGroup: 1000` is not decoration — it is the only thing that makes the volume writable. `/data` is created mode `1777` in the image, so the non-persistent path is writable regardless. Also read off the same Dockerfile and relevant to the password: the image **patches `protected-mode` off** (`sed -ri 's!...createBoolConfig("protected-mode", ... 1 ...!...0...!'`). "No password" therefore really does mean "reachable from the whole pod network", which is what the init container exists to prevent. ### 2. The probe: `valkey-cli ping` really does exit 0 on `NOAUTH` From `valkey` 9.1.2, `src/valkey-cli.c`. An error reply becomes a non-zero exit in **exactly one** place: ```c } else if (!config.interactive && config.set_errcode && reply->type == VALKEY_REPLY_ERROR) { fprintf(stderr, "%s\n", reply->str); exit(1); ``` and `config.set_errcode` is set by nothing but `-e` (`"-e Return exit error code when command execution fails."`). So a probe written as `valkey-cli ping` prints `NOAUTH Authentication required.` **and exits 0** — the always-green assertion. Second finding from the same file: with `VALKEYCLI_AUTH` set to a wrong password, `cliAuth` fails and prints `AUTH failed: WRONGPASS ...`, but `main` **ignores** `cliConnect`'s return value, so the command is sent anyway and the failure arrives as the `NOAUTH` reply — i.e. the same path. The auth env var is `VALKEYCLI_AUTH`, with `REDISCLI_AUTH` still honoured as the legacy spelling (`#define CLI_AUTH_ENV` / `OLD_CLI_AUTH_ENV`). The probe is therefore `VALKEYCLI_AUTH="$(…)" valkey-cli -e -p 6379 ping | grep -qx PONG`, with the password read out of the same mounted auth file the server was configured from (never `-a`, never in argv). **Proof that it can fail — local, with `valkey-cli` stubbed** (the stub mimics the source above; the run also greps every output for the password and found none): | case | exit | | --- | --- | | right password, `-e` | 0 | | extra whitespace after `requirepass` | 0 | | double-quoted password | 0 | | **wrong password** | **1** | | **no `requirepass` line at all** | **1** | | wrong password, **without** `-e` | 1 (the `grep` catches it) | | right password, without `-e` | 0 | The last two rows are why both halves are in the command and why the comment in the ConfigMap says so precisely: `grep -qx PONG` alone would already catch an error reply, and `-e` is what puts the server's own message (`NOAUTH…`, `WRONGPASS…`) on stderr and into the probe's failure output instead of silently on stdout. **The wrong-password test against a real server has been run on this PR** — see the evidence comment below. Summary: with readiness authenticating with a deliberately wrong password, the pod stayed `0/1 Running` with `Readiness probe failed: AUTH failed: WRONGPASS ... / NOAUTH Authentication required.` and `RESTARTS 0`, and the install-test went red; the two temporary experiment commits have been removed from the branch and the final run is green. There is no Docker daemon in the work environment, so that run is the only runtime proof there can be. ## What the init container rejects `check-auth.sh` (in the ConfigMap, run by an init container from the same image, as uid 999 with everything dropped) greps the mounted file for `^requirepass[[:space:]][[:space:]]*[^[:space:]]`. Verified locally against the real script: | auth file content | result | | --- | --- | | `requirepass <pw>` | passes | | `requirepass <pw>` on the second line | passes | | `requirepas <pw>` (misspelled) | rejected | | `requirepass` (no value) | rejected | | `requirepass ` (whitespace only) | rejected | | empty field | rejected | | ` requirepass <pw>` (leading space — Valkey ignores it too) | rejected | | file missing entirely | rejected | It never prints the file; the message names the 1Password field instead. What it deliberately does **not** do is compose the line — the vault stays the only place the password exists in. ## The rest of the implementation - **StatefulSet**, one replica (`valkey.replicas` capped at 1 by the schema — two pods would fight over one RWO volume and clients behind the Service would silently see two datasets), ClusterIP Service, chart-global `scaleDown`. No Ingress and no value to add one. - **Password**: 1Password field **`auth-conf`**, holding the finished line `requirepass <password>`, mounted as a file and `include`d by the config. The include is rendered **last**, so nothing in `valkey.extraConfig` can override it — stated in `values.yaml` as a property, not as advice. - **`checksum/config`** over the ConfigMap and **`checksum/secret-ref`** over the rendered `OnePasswordItem`. `operator.1password.io/auto-restart` is not set (silent no-op on a StatefulSet). - **`maxmemory`** derived from the *rendered* memory limit (read back out of the resources helper, so it cannot drift from what the container gets): `maxmemoryFactor` default `0.75`. A configured `valkey.maxmemory` above the limit aborts the render with both byte counts named; `0` aborts too, because in Valkey that means unlimited. Defaults render `maxmemory 603979776` against a `768Mi` limit. - **Persistence**: `longhorn-encrypted-ns`, 1Gi, `appendonly` a value with the reason in the comment. With `persistence.enabled: false` the chart renders `appendonly no` **and** `save ""`, so a pod without a volume writes nothing to ephemeral storage. The two sensible pairs are documented at the top of `values.yaml`, and "no persistence + noeviction" gets a warning both there and in the rendered `NOTES.txt`. - **NetworkPolicy** on by default, `networkPolicy.allowedNamespaces` for consumers elsewhere, and the note that it is only a boundary where the CNI enforces policies. Ingress-only on purpose (this instance opens no outbound connection). - **Hardening**: `runAsNonRoot`, 999/1000, `fsGroup` 1000, `readOnlyRootFilesystem`, all caps dropped, `seccompProfile: RuntimeDefault` in the **pod** context so the init container is covered. `/data` is the only writable path — no extra emptyDir was needed, because the server writes nowhere else (logs go to stdout, and neither `pidfile` nor `unixsocket` is configured). - **Resources** per convention: requests as defaults, memory/ephemeral-storage limits computed, no CPU limit. - `values.schema.json` with `additionalProperties: false` at every chart-owned level; `maxmemoryPolicy` is an enum of the eight real policies. ## Verification Run locally, all green: - `helm lint --strict valkey` (defaults and with the CI values) - render with only the required value; with `persistence.enabled=false`; with `scaleDown: true` (`replicas: 0`); with `valkey.maxmemory=2Gi` → **render aborts**: *"valkey.maxmemory is 2Gi (2147483648 bytes) and thus ABOVE the container's memory limit (805306368 bytes)…"*; with `--set-string valkey.maxmemory=0` → aborts as unlimited - null-safety: `valkey.securityContext=null`, `valkey.resources=null` and all three probes `=null` fall back to the chart's own defaults (uid 999 included) - schema: `persistence.enable` typo, `valkey.replicas=2`, an unknown `maxmemoryPolicy` and a Valkey-style `512mb` are each rejected by name - `ci/run-quantity-render-test.sh valkey` — both cases exercise the helper; and the whole repo still passes (24 exercised, 0 failed). The fraction case also checked by hand: the derived `maxmemory` renders as `1124998`, not in exponential notation - `helm package valkey` → `valkey-1.0.0+up9.1.2.alpine.tgz` - `NOTES.txt` rendered in all its branches (queue, cache, the mistaken pair, scaled to zero, NetworkPolicy on/off) with the scratch-ConfigMap trick from the README, since `helm template` never renders it ## What I could not verify Said in those words, because the work environment has neither a Docker daemon nor node: - **The install-test is the only runtime proof, and it has now run** (see the evidence comment). Nothing was started in the work environment itself: not the server, not the init container, not the probes against a real `valkey-cli`. The stub-based table above proves the *scripts*, the source quotes prove `valkey-cli`, and the three cluster runs prove the two meeting in one pod. - **The install-test can only cover one variant.** It runs the persistent path (PVC, `fsGroup`, AOF), which is the risky one and the chart's default. The **non-persistent path is rendered locally only and is a known gap** in exactly those words. - **`fsGroup` itself is not proven by the install-test.** k3d's `local-path` volumes are node-directory backed and created world-writable, so the container can write whether or not the kubelet applied `fsGroup`. The test proves *writability*; the ownership mechanism is unverified. A hostPath PV fixture would have been worse — the kubelet applies `fsGroup` to those not at all. - **Whether the NetworkPolicy filters** anything is a property of the cluster's CNI. The test only shows the object is accepted and does not break the pod's own traffic. - **The Renovate block is unproven by a dry run** (no node/npx here, as established in #34). It needs none of the usual guard, though, and that is the whole reason decision 1 was revised: plain docker versioning with no capture groups and a suffix that never changes. First evidence will be `valkey` appearing on the dependency dashboard. - **The rotated-password self-healing** described in `values.yaml` and `NOTES.txt` (kubelet refreshes the mounted file → liveness fails → container restarts with the new password) is labelled UNVERIFIED there, and the manual `rollout restart` stays the documented procedure. ## Wiki `valkey` page added (installation, the 1Password field, the full values reference, the traps), plus a row in `Charts` and an entry in `_Sidebar.md`.

jklein force-pushed feat/valkey-chart from 93ce5f8ef2 to 5048ce953c 2026-09-27 13:18:18 +02:00 Compare

jlab-bot commented 2026-09-27 13:22:26 +02:00

Author

Member

Copy link

Copy source

The runtime proof, in the cluster

Three install-test runs on this PR. Runs 233 and 234 are deliberately red experiments, pushed as temporary commits and removed again; the branch is back at the single reviewable commit and run 235 is green. The two temporary commits are gone from the branch (force-pushed away), so there is nothing to un-merge.

1. Baseline — the chart comes up (run 232, repeated as run 235)

NAME              READY   STATUS    RESTARTS   AGE
ci-valkey-sfs-0   1/1     Running   0          56s
OK: chart 'valkey' installed and all workloads are Ready.

So: uid 999 against a provisioned PVC works, the init container passes, the server starts from the mounted configuration with the included auth file, and the three authenticated probes succeed. NOTES.txt rendered in the install output as expected, naming the shape and the derived numbers:

THIS RELEASE IS CONFIGURED AS A QUEUE
  persistence.enabled:        true
  valkey.maxmemoryPolicy:     noeviction
  memory limit (computed):    192.00Mi (201326592 bytes)
  maxmemory:                  144.00Mi (150994944 bytes), derived as maxmemoryFactor 0.75 x the limit

2. The probe CAN fail — wrong password (run 233, red on purpose)

The temporary commit gave readiness only a deliberately wrong password while the server kept the right one:

readinessProbe:
  exec:
    command: ["/bin/sh", "-ec", "VALKEYCLI_AUTH=deliberately-wrong-password valkey-cli -e -p 6379 ping | grep -qx PONG"]

Result:

NAME              READY   STATUS    RESTARTS   AGE
ci-valkey-sfs-0   0/1     Running   0          62s
error: timed out waiting for the condition

Warning  Unhealthy  5s (x2 over 15s)  kubelet  spec.containers{ci-valkey-sfs}: Readiness probe failed: AUTH failed: WRONGPASS invalid username-password pair or user is disabled.
NOAUTH Authentication required.

Readiness went down, the pod stayed 0/1, and the install-test blocked. Three details worth having on the record:

• The failure message names the cause (WRONGPASS / NOAUTH) in the pod's events — that is the -e doing its work, since without it valkey-cli writes the error to stdout and exits 0.
• RESTARTS 0: liveness kept using the password from the mounted file and stayed green, so the pod was held out of the Service instead of being restarted in a loop. That is the intended split.
• Readiness failing is sufficient to block a rollout here, which is what makes this probe the one worth breaking on purpose.

So the assertion is real: the earlier local table (stubbed valkey-cli) said the script fails on a wrong password, and the cluster now says the probe does too, against the actual binary.

3. The init container rejects a malformed field (run 234, red on purpose)

Temporary commit: the fixture's auth-conf field spelled requirepas — the typo that would otherwise produce an open server that looks healthy.

NAME              READY   STATUS       RESTARTS      AGE
ci-valkey-sfs-0   0/1     Init:Error   2 (22s ago)   61s

Warning  BackOff  13s (x3 over 27s)  kubelet  spec.initContainers{ci-valkey-check-auth}: Back-off restarting failed container ci-valkey-check-auth

[pod/ci-valkey-sfs-0/ci-valkey-check-auth] valkey: the mounted auth file carries no 'requirepass <password>' line.
[pod/ci-valkey-sfs-0/ci-valkey-check-auth]   Expected the field 'auth-conf' of the 1Password item to contain
[pod/ci-valkey-sfs-0/ci-valkey-check-auth]   Refusing to start: Valkey would come up without a password, report ...

The server container never started, and the message names the 1Password field without printing its contents.

What is still not proven, unchanged from the PR body

• The non-persistent variant is rendered locally only — the machinery runs one variant per chart. Known gap, in those words.
• fsGroup itself is not proven: k3d's local-path volumes are node-directory backed and world-writable, so the container can write regardless of whether the kubelet applied fsGroup. Writability is proven, the ownership mechanism is not.
• Whether the NetworkPolicy filters is a property of the CNI; the run only shows the object is accepted and does not disturb the pod.
• The Renovate block has had no dry run (no node here, as in #34) and needs none by design — first evidence will be valkey on the dependency dashboard.

## The runtime proof, in the cluster Three install-test runs on this PR. Runs 233 and 234 are **deliberately red** experiments, pushed as temporary commits and removed again; the branch is back at the single reviewable commit and run 235 is green. The two temporary commits are gone from the branch (force-pushed away), so there is nothing to un-merge. ### 1. Baseline — the chart comes up (run 232, repeated as run 235) ``` NAME READY STATUS RESTARTS AGE ci-valkey-sfs-0 1/1 Running 0 56s OK: chart 'valkey' installed and all workloads are Ready. ``` So: uid 999 against a provisioned PVC works, the init container passes, the server starts from the mounted configuration with the `include`d auth file, and the three authenticated probes succeed. `NOTES.txt` rendered in the install output as expected, naming the shape and the derived numbers: ``` THIS RELEASE IS CONFIGURED AS A QUEUE persistence.enabled: true valkey.maxmemoryPolicy: noeviction memory limit (computed): 192.00Mi (201326592 bytes) maxmemory: 144.00Mi (150994944 bytes), derived as maxmemoryFactor 0.75 x the limit ``` ### 2. The probe CAN fail — wrong password (run 233, red on purpose) The temporary commit gave **readiness only** a deliberately wrong password while the server kept the right one: ```yaml readinessProbe: exec: command: ["/bin/sh", "-ec", "VALKEYCLI_AUTH=deliberately-wrong-password valkey-cli -e -p 6379 ping | grep -qx PONG"] ``` Result: ``` NAME READY STATUS RESTARTS AGE ci-valkey-sfs-0 0/1 Running 0 62s error: timed out waiting for the condition Warning Unhealthy 5s (x2 over 15s) kubelet spec.containers{ci-valkey-sfs}: Readiness probe failed: AUTH failed: WRONGPASS invalid username-password pair or user is disabled. NOAUTH Authentication required. ``` Readiness went down, the pod stayed **0/1**, and the install-test blocked. Three details worth having on the record: - The failure message names the cause (`WRONGPASS` / `NOAUTH`) in the pod's events — that is the `-e` doing its work, since without it valkey-cli writes the error to stdout and exits 0. - **`RESTARTS 0`**: liveness kept using the password from the mounted file and stayed green, so the pod was held out of the Service instead of being restarted in a loop. That is the intended split. - Readiness failing is *sufficient* to block a rollout here, which is what makes this probe the one worth breaking on purpose. So the assertion is real: the earlier local table (stubbed `valkey-cli`) said the script fails on a wrong password, and the cluster now says the probe does too, against the actual binary. ### 3. The init container rejects a malformed field (run 234, red on purpose) Temporary commit: the fixture's `auth-conf` field spelled `requirepas` — the typo that would otherwise produce an open server that looks healthy. ``` NAME READY STATUS RESTARTS AGE ci-valkey-sfs-0 0/1 Init:Error 2 (22s ago) 61s Warning BackOff 13s (x3 over 27s) kubelet spec.initContainers{ci-valkey-check-auth}: Back-off restarting failed container ci-valkey-check-auth [pod/ci-valkey-sfs-0/ci-valkey-check-auth] valkey: the mounted auth file carries no 'requirepass <password>' line. [pod/ci-valkey-sfs-0/ci-valkey-check-auth] Expected the field 'auth-conf' of the 1Password item to contain [pod/ci-valkey-sfs-0/ci-valkey-check-auth] Refusing to start: Valkey would come up without a password, report ... ``` The server container never started, and the message names the 1Password field without printing its contents. ### What is still not proven, unchanged from the PR body - The **non-persistent variant** is rendered locally only — the machinery runs one variant per chart. Known gap, in those words. - **`fsGroup` itself** is not proven: k3d's `local-path` volumes are node-directory backed and world-writable, so the container can write regardless of whether the kubelet applied `fsGroup`. Writability is proven, the ownership mechanism is not. - **Whether the NetworkPolicy filters** is a property of the CNI; the run only shows the object is accepted and does not disturb the pod. - The **Renovate block** has had no dry run (no node here, as in #34) and needs none by design — first evidence will be `valkey` on the dependency dashboard.

jklein added 1 commit 2026-09-27 13:24:57 +02:00

feat(valkey): one Valkey instance per application (1.0.0+up9.1.2.alpine) ... 813c796a7b

A new chart for a single Valkey instance per application, replacing the
shared cluster-redis whose separation rests on one ACL file in one
1Password field - a construction in which every change touches six
applications and which cost a month of Paperless standstill.

Shape: StatefulSet with one replica, ClusterIP Service, no cluster mode,
no Ingress and no value to add one.

The password arrives as a finished configuration line ("requirepass
<secret>") in the 1Password field auth-conf, is mounted as a file and
included by the main configuration - never in argv, never in the
environment. The include is rendered last, so nothing in extraConfig can
override it. Because a misspelled or empty field would leave an open
server that looks perfectly healthy, an init container greps the mounted
file for the line and refuses to start the pod without it; it never
prints the file.

maxmemory is derived from the computed memory limit (maxmemoryFactor,
default 0.75) and a configured value above that limit aborts the render:
otherwise the kubelet's OOM kill decides what the eviction policy was
configured to decide. Persistence and eviction are documented as pairs -
queue (PVC + noeviction, the default) and cache (emptyDir +
allkeys-lru) - and the combination "no persistence + noeviction" carries
a warning in values.yaml and in NOTES.txt.

Probes: valkey-cli ping as an exec probe, authenticated through
VALKEYCLI_AUTH read out of the mounted file, with "-e" and an assertion
on the answer PONG. Without -e valkey-cli exits 0 on an error reply
(valkey 9.1.2, src/valkey-cli.c: the config.set_errcode branch in
cliReadReply), which would make the probe unable to fail.

Hardening: uid 999 / gid 1000 - the image's own valkey user - with
fsGroup matching, runAsNonRoot, readOnlyRootFilesystem, all capabilities
dropped and seccompProfile RuntimeDefault in the pod context so the init
container is covered too. /data is the only writable path.

NetworkPolicy on by default (first chart in this repo with one), with a
switch and a list of additional namespaces; values.yaml names that it is
only a boundary where the CNI enforces policies.

Wiring: publish-valkey.yml, the chart-name choice list, ci/valkey
fixtures for the persistent variant, a plain docker-versioning Renovate
block, README chart row.

jklein force-pushed feat/valkey-chart from 5048ce953c to 813c796a7b 2026-09-27 13:24:57 +02:00 Compare

jlab-bot commented 2026-09-27 13:27:36 +02:00

Author

Member

Copy link

Copy source

Addendum: the two Trivy findings on the ConfigMap are documented exceptions now

The first run reported two keyword findings against the rendered ConfigMap, and one of them said something misleading about this chart in particular:

• KSV-0109 (HIGH, "ConfigMap stores secrets") — matched the comment # Nothing here prints the password and the line pass="$(sed -n ...)" of the probe script.
• KSV-01010 (MEDIUM, "sensitive contents") — matched the key valkey.conf and an echo "valkey... line of the validation script.

Both are keyword matches on comments and shell code, not on data. The ConfigMap holds no credential — that is the entire point of keeping the password in a mounted Secret file that the configuration includes. Leaving a HIGH "ConfigMap stores secrets" standing on the one chart built to avoid exactly that would be a misleading signal, so both are now entries in ci/trivy/trivyignore.yaml, scoped to **/valkey.yaml, each with its reason — the house rule being that no exception exists without a statement.

The scan now reports valkey: 0 to act on, 8 excepted; the other six are the repo-wide conventions (no CPU limit, the pinned uid/gid, the namespace artefact). Install-test green in the same run, all five checks green on the current head.

### Addendum: the two Trivy findings on the ConfigMap are documented exceptions now The first run reported two keyword findings against the rendered ConfigMap, and one of them said something misleading about this chart in particular: - `KSV-0109` (HIGH, "ConfigMap stores secrets") — matched the comment `# Nothing here prints the password` and the line `pass="$(sed -n ...)"` of the probe script. - `KSV-01010` (MEDIUM, "sensitive contents") — matched the key `valkey.conf` and an `echo "valkey...` line of the validation script. Both are keyword matches on comments and shell code, not on data. The ConfigMap holds no credential — that is the entire point of keeping the password in a mounted Secret file that the configuration `include`s. Leaving a HIGH "ConfigMap stores secrets" standing on the one chart built to avoid exactly that would be a misleading signal, so both are now entries in `ci/trivy/trivyignore.yaml`, scoped to `**/valkey.yaml`, each with its reason — the house rule being that no exception exists without a statement. The scan now reports `valkey: 0 to act on, 8 excepted`; the other six are the repo-wide conventions (no CPU limit, the pinned uid/gid, the namespace artefact). Install-test green in the same run, all five checks green on the current head.

jklein approved these changes 2026-09-27 16:14:53 +02:00

jklein merged commit 813c796a7b into main 2026-09-27 16:14:59 +02:00

jklein deleted branch feat/valkey-chart 2026-09-27 16:15:00 +02:00

jlab-bot referenced this pull request 2026-09-27 16:33:27 +02:00

Install-test over all charts, on demand, through one shared job #54

jlab-bot referenced this pull request 2026-09-27 22:45:59 +02:00

Neuer Chart: valkey - eine Instanz je Anwendung statt der geteilten Redis #46

Sign in to join this conversation.

Reviewers

No Reviewers

jklein

Labels

Clear labels

inhouse-chart

Inhouse-Chart aus der JLab-Git-Paketregistry

renovate

Von Renovate angelegt

No labels

Milestone

No items

No Milestone

Assignees

Clear assignees

jklein (Justin Klein) jlab-bot (jlab-git-bot) jlab-renovate-bot (jlab-renovate-bot)

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: jlab-cluster/helm-charts#53