jklein
a9ca5302de
Validate Helm Charts / Detect changed charts (pull_request) Successful in 4s
Validate Helm Charts / Resource quantities (pull_request) Successful in 12s
Validate Helm Charts / Build paperless-ngx (pull_request) Successful in 6s
Validate Helm Charts / Trivy config paperless-ngx (pull_request) Successful in 22s
Install-test / Install-test paperless-ngx (pull_request) Successful in 3m31s
Validate Helm Charts / Install-test (pull_request) Successful in 3m31s
Publish Helm Chart with patched Application Version / publish-helm-chart (push) Successful in 10s
Publish Paperless-NGX Helm Chart / Publish Paperless-NGX Helm Chart (push) Successful in 10s
SonarQube Analysis / SonarQube Analysis (push) Successful in 33s
paperless-ngx cannot run without a Redis - PAPERLESS_REDIS is the Celery broker - so every installation needed a second Helm release whose address lived in a 1Password item and in neither chart. This adds our own published valkey chart as an optional dependency, off by default and gated on valkeySubchart.enabled (not valkey.enabled, which the subchart's strict schema rejects and which Helm would only have complained about for whoever first switched it on). Additive, default behaviour unchanged: minor, 12.1.3+up3.2.1 -> 12.2.0+up3.2.1. Configured as a QUEUE (PVC + noeviction, the subchart's defaults), unlike stalwart's cache. Read off the 3.2.1 source rather than assumed: one PAPERLESS_REDIS url serves five consumers - the Celery broker, the result backend (Redis in 3.x, the database before), the channels pub/sub layer and two Django caches. allkeys-lru would not be a trade here but actively wrong, because LRU evicts typeless across the whole keyspace and a backlog of waiting consume tasks is exactly what it reads as cold; a dropped task is a document silently never processed. The price, named in values.yaml rather than smoothed over: two of the five consumers are caches, so a full instance refuses writes - new tasks included - and the answer is memory sizing, not a policy change. All three cache-like consumers are TTL-bounded, which is what keeps that price small. The password comes from the SAME item as the Redis connection, in a third field auth-conf holding the complete line "requirepass <password>". It cannot DEFAULT to secrets.redisSecretRef, and that is a property of Helm, not an omission: a values.yaml is never templated and a parent cannot compute a subchart's value. Measured - a template expression there lands verbatim in the subchart's itemPath and fails as a YAML parse error naming neither the value nor the cause. Forgetting it is therefore caught in values.schema.json, by a conditional rule rather than a fail() in a template. Also measured, and the reason: Helm renders a subchart's templates BEFORE the parent's, so a guard in this chart loses the race to the subchart's own required(), which reports "secrets.authSecretRef" and cannot know the path belongs to the redis item. A check that cannot fire is the mistake found while testing #77, so the first draft of that guard was removed rather than shipped. The schema runs before any rendering and names /valkey/secrets/authSecretRef. One trap found while testing and closed here: "valkeySubchart: null" DELETES the key, Helm then cannot resolve the condition path and treats the dependency as ENABLED - so erasing the switch turned the subchart on with no password set. Unlike every other block in this chart (issue #99), that one is rejected by the schema instead of defaulting. The install-test now runs with the subchart ON and the throwaway redis Deployment removed: the url in the fixtures names ci-valkey-service, so the bundled Valkey is the broker the release actually uses. #65 covered this path by a render only - deliberately, since an unused Valkey would have made that test look like it covered clustering. This is the first real installation of the valkey-subchart path. Documented, not glossed: the password now sits in that item twice and a rotation has to touch both fields; with the subchart on the vault url is a mandatory step, not a hint; and a OnePasswordItem materializes every field of its item, so both Secrets carry all three keys - untidy, harmless, and written down so nobody "tidies it up". Fixes #78
121 lines
18 KiB
Markdown
121 lines
18 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## Working on issues — mandatory workflow
|
||
|
||
The main Claude session acts as the **task manager** and does not implement issues itself:
|
||
|
||
1. **Plan first, get approval**: The manager presents a plan for the issue and waits for the user's explicit approval. No implementation before approval.
|
||
2. **Document the final plan on the issue**: Once approved, the plan is posted as a comment on the issue before implementation starts (skip if the issue body already contains the full plan; later decisions go there as comments too).
|
||
3. **Delegate implementation**: After approval, the manager launches a dedicated **development agent**. The agent implements the issue completely on its own branch and opens a PR.
|
||
4. **PR comment handling by the agent**: After opening the PR, the development agent watches it for review comments. New comments are addressed automatically (code changes, reply on the thread), and addressed threads are marked as **resolved**.
|
||
5. **Update the wiki — mandatory, and before the merge is requested.** The wiki is this repo's knowledge base; a change that lands in `main` while the wiki still describes the old state has not been finished, it has been half-finished in a way nothing fails on. The development agent edits the affected pages as part of its issue, so code and documentation are reviewed together. See "The wiki" below for what obliges an edit and what does not.
|
||
6. **Agent lifetime**: The development agent terminates itself once its PR has been merged (or closed).
|
||
7. Only after the PR has been **merged by the user** may work on the next issue begin. The user merges; never merge yourself.
|
||
8. **After the merge, verify the linked issue was closed** (auto-close via "Fixes #n"); close it manually if not. Gitea auto-closes on the same keywords, but only for an issue in the same repository — a migrated issue whose number changed is closed by hand. Check in the same step that the wiki edit really landed.
|
||
|
||
Additional rules for all commits, PRs, issues, and comments:
|
||
|
||
- Do **not** add a `Co-Authored-By` trailer and do **not** append "Generated with Claude Code" footers.
|
||
- **Commits must be signed**, and in the Coder workspace they are: signing runs through its own SSH signer (`gpg.format = ssh`, `gpg.ssh.program = coder-git-ssh-sign`, `commit.gpgsign = true`), so no 1Password step is involved. Verify before opening the PR — `git log --format='%G?' -1` must print `G` — and if signing fails, stop and report rather than pushing unsigned work, because branch protection rejects it at merge time.
|
||
- **A PR that does not finish its issue writes `Part of #n`, never `Fixes #n`.** The keyword closes the issue on merge, so a `Fixes` on a PR that leaves work behind marks an open task as done — and reopening by hand is the only repair, which only happens if someone remembers to look. This is not hypothetical: Gitea PR #48 carried `Fixes #40` while `rss-bridge` had been deliberately left out of the sweep, and the merge closed Gitea #40 with one occurrence still open. Step 8 above is the other half of the same rule: the auto-close it verifies is only the right outcome when the keyword was the right one.
|
||
|
||
## The wiki
|
||
|
||
The knowledge base is the Gitea wiki at `https://git.jk-dev.de/jlab-cluster/helm-charts/wiki` — one page per chart (installation, required 1Password fields, every configurable value, the chart's traps) plus the long form of this workflow, the pipelines, the style guide and the incidents each rule was bought with. It is a **separate git repository**:
|
||
|
||
```bash
|
||
git clone ssh://git@git.jk-dev.de:30022/jlab-cluster/helm-charts.wiki.git
|
||
```
|
||
|
||
Its default branch is `main`, not `master`.
|
||
|
||
Flat layout, one Markdown file per page, page name = file name, links are relative page names without the extension (`[CI/CD](CI-CD)`). A new page must also be listed in `_Sidebar.md`, or it is reachable only by search.
|
||
|
||
**There is no PR flow here, so fetch and rebase before every push and never force-push** — two agents have already edited the same page minutes apart, and a force push deletes the other edit silently, with nothing left to report it.
|
||
|
||
**What obliges a wiki edit** (step 5 of the workflow above, same round of work, never a follow-up issue):
|
||
|
||
| Change | Page |
|
||
| --- | --- |
|
||
| a value added, removed, renamed, or its default changed | the chart's page — values table, and the operating notes if behaviour moved |
|
||
| runtime behaviour changed (probes, hardening, restart semantics, a value becoming required) | the chart's page, plus `Lessons-Learned` if the reason could be repeated elsewhere |
|
||
| a new chart | a new chart page, a row in `Charts`, an entry in `_Sidebar.md` |
|
||
| a convention added or changed | `Chart-Style-Guide` (and README, which stays the code-side copy) |
|
||
| pipeline, fixture or runner behaviour changed | `CI-CD` |
|
||
| this issue workflow changed | `Workflow` — **and** this file, which is the binding copy |
|
||
| a Renovate rule added or changed | `Renovate` |
|
||
| something broke in a way a rule now prevents | `Lessons-Learned` |
|
||
|
||
**What does not oblige an edit:** a plain app-version bump. Renovate automerges those continuously and never touches the wiki, which is why chart pages carry no version numbers at all and name `Chart.yaml` as the source of truth. An app update that *does* change values or behaviour is not a plain bump, and the table applies.
|
||
|
||
Wiki commits follow the same rules as repo commits: no `Co-Authored-By`, no "Generated with Claude Code" footer.
|
||
|
||
## Which forge, and with which tool
|
||
|
||
Everything runs on the Gitea instance JLab-Git: issues, pull requests, CI and Renovate. github.com is a push mirror with Actions and Renovate **switched off**, and it lags until the mirror runs. README, "Where this repo lives", holds the measured differences between a Gitea runner and a GitHub one.
|
||
|
||
- **`tea`, not `gh`.** Work happens in a **Coder workspace, and it has no `op`** — that is the normal case here, not a broken environment, so do not wait for the 1Password CLI or propose installing it. Call `tea` directly; it carries the workspace's stored login and opens PRs and issues under the **bot account**, which is accepted. Say so in the report when something is authored that way. (`op plugin run -- tea`, the older instruction, applies only where `op` actually exists.)
|
||
- **The remote is `origin`, and it points at Gitea** (`ssh://git@git.jk-dev.de:30022/jlab-cluster/helm-charts.git`). The checkout has exactly that one, and so does every worktree cut from it, so a bare `git push` goes to the leading copy; the github.com mirror is fed by Gitea itself and is never pushed to by hand. A clone elsewhere may carry the mirror as a second remote named `github` — wherever more than one exists, say which one you mean.
|
||
- **Cut every worktree from a freshly fetched `origin/main`.** `git fetch origin main` first, then branch from `origin/main` — never from the local `main` ref, and never assume the shared checkout is current: both can be many merges behind. The staleness is invisible where it matters most, and it outlives the fix: the project-instructions block a subagent receives is a **snapshot** of the spawning session's working directory, taken when **that session started** — not a live read. Fetching mid-session does not refresh it, so the file on disk can be correct while every agent started from then on is still handed the superseded rules. Measured on 2026-09-27: an agent whose own worktree was identical to `origin/main` (`git rev-list --left-right --count origin/main...HEAD` returned `0 0`) received a block from before Gitea #47 and reported a contradiction in this file that two PRs had already fixed, quoting the stale copy as binding. Two consequences, and only these two help: the spawning session's directory must be current **before** the session starts, and an agent checks a rule against the **file in its own worktree** plus a diff against `origin/main` — never against `main`, and never against its own context block, because the first consequence cannot be enforced from inside an agent and the second can.
|
||
- **Branch protection with a signature requirement is active**, and signatures verify on Gitea. A commit that is not signed does not reach `main` — check before opening a PR, do not discover it at merge time.
|
||
- **No merge queue.** Gitea has none, and its place is taken by **auto-merge on green checks**: the PR is armed once and merges itself when the required check passes **and the required review exists** — both are active on `main`, so an armed PR without a review sits there instead of merging. Rule 7 above is unchanged — the user merges, and an agent never merges its own PR or arms auto-merge on it.
|
||
- **A required check per chart is not possible.** Gitea runs a whole matrix as one job under one status context, so the install-test reports once for all charts. What is required on `main` is `validate-charts`, as that single context; per chart is the part that cannot be had. Relevant whenever branch protection is touched.
|
||
- **Issue numbers changed with the move, and `#n` in this repo's files means the GitHub tracker.** The issues open at the time are Gitea #5–#21 and #24; the mapping is a comment on Gitea #14. Closed ones were not migrated and exist only on GitHub. Beware that Gitea silently resolves a bare `#n` against its own counter, so an old reference can render as a link to something unrelated. When you write a *new* reference, use the Gitea number.
|
||
- **Renovate runs outside this repo**, as the shared `jlab-renovate-bot` instance; it reads our `renovate.json5` and its `postUpgradeTasks` hook is allow-listed there. **Do not add a Renovate workflow here** — a second runner means duplicate PRs and two schedulers rewriting each other's branches. README, "Renovate on Gitea", has the evidence.
|
||
|
||
## Keep the docs current
|
||
|
||
- Changes to this issue workflow always go into this CLAUDE.md file.
|
||
- General repository information (charts, CI pipelines, conventions, tooling) always lives in README.md. Any PR that changes repo-wide behavior must update README.md in the same PR.
|
||
- **Operator-facing knowledge lives in the wiki**, not in README.md: how a chart is installed, what every value does, what its traps are. The division is `CLAUDE.md` binds the agent, `README.md` explains the repository to someone reading the code, the wiki explains the system to whoever runs it, and a chart's `NOTES.txt` explains one release. Where README and the wiki cover the same rule, both are updated in the same PR — see "The wiki" above.
|
||
- Runtime verification of chart changes is done in a local, throwaway k3d cluster (k3d is installed on the maintainer's machine): stub CRD for `OnePasswordItem`, dummy secrets, throwaway infrastructure (postgres/redis) where needed.
|
||
|
||
## What this repo is
|
||
|
||
A collection of self-made Helm charts, one directory per chart, published to a private, classic (index-based) Helm registry by Gitea Actions — an HTTP `POST` of the packaged `.tgz`, not an OCI `helm push`; the target URL and its credentials live in 1Password and never in this repository. Validation happens on three levels: locally with Helm, in the PR pipeline (`validate-charts.yml`: strict lint, package, version-bump check), and in the PR install-test stage (k3d, fixtures under `ci/` — see README):
|
||
|
||
```bash
|
||
helm lint --strict <chart-name>
|
||
helm template <chart-name> --set ingress.domain=example.com --set ingress.clusterIssuer=letsencrypt # render locally; required values vary per chart
|
||
```
|
||
|
||
## Creating a new chart
|
||
|
||
Copy `template-chart/` and replace every `xxx` placeholder — in file names (`templates/xxx.deployment.yaml` → `templates/<app>.deployment.yaml`), in `Chart.yaml`, in the `values.yaml` keys and their `.Values.xxx.*` references inside templates, and in `values.schema.json` (both the values paths and the `xxx.` helper prefixes). The schema and the null-safe fallback helpers come with the template but their content is chart-specific — adapt them along the six points of the README section "Values that must not lie", and validate the result against the chart's real bundle values, not only against its defaults. Then wire up CI:
|
||
|
||
1. Add `.github/workflows/publish-<chart-name>.yml` (copy an existing one, e.g. `publish-outline.yml`) — it triggers on pushes to `main` touching `<chart-name>/**` and calls the reusable `publish-helm-chart.yml`, which lives in `workflow-util/pipelines` and is bound to a tag. Copying an existing hull gets the `uses:` path, `registry-owner: ${{ github.repository_owner }}` and `secrets: inherit` right; writing one from memory does not.
|
||
2. Add the chart name to the `chart-name` choice list in `.github/workflows/change-app-version.yml`.
|
||
3. Add install-test fixtures under `ci/<chart-name>/` (`test-values.yaml`, `fixtures.yaml`) or an entry with justification in `ci/excluded-charts.txt` — the PR install-test fails if both are missing.
|
||
4. Add a `customManagers` block for the chart's main image in `renovate.json5` (one per chart, since the image repository lives in the template and the version in `Chart.yaml`) — without it Renovate never updates that chart's app version. See README section "Dependency updates (Renovate)".
|
||
|
||
## Versioning scheme
|
||
|
||
`Chart.yaml` versions follow `version: <chart-semver>+up<appVersion>` (e.g. `1.4.0+up1.9.2`), with `appVersion` matching the suffix (dashes in the app version are normalized to dots in the suffix). Any push to `main` touching a chart directory publishes that chart, so the version must be bumped in the same change.
|
||
|
||
Semver conventions for the chart part:
|
||
|
||
- **Patch**: bug fixes without behavior or values-schema change; plain app-version updates.
|
||
- **Minor**: additive, backward-compatible changes (new optional values with unchanged defaults).
|
||
- **Major**: runtime behavior changes or breaking values changes (e.g. probes, security hardening, computed limits, values that become required).
|
||
|
||
App-version updates go through PRs (so the validate and install-test pipelines run), not through the `change-app-version.yml` dispatch workflow (which commits directly to main and bypasses both). For app **major** jumps, update stepwise: first to the last release of the old major line (own PR, merge, publish), then to the newest major.
|
||
|
||
## Chart conventions
|
||
|
||
The binding style guide for all current and future charts lives in **README.md ("Chart style guidelines")**; `template-chart/` is the reference implementation. Compact summary:
|
||
|
||
- Resources named `{{ .Release.Name }}-{{ .Chart.Name }}-<kind>`; template files `<app>.<kind>.yaml`; stateful apps use a StatefulSet (`<app>.sfs.yaml`). Documented exception: historical PVC names stay (renames would orphan data); StatefulSet renames only with a `--cascade=orphan` migration note in a major PR.
|
||
- Main image tag from `{{ .Chart.AppVersion }}`; auxiliary/sidecar/init images pinned via values and reviewed on app-update rounds.
|
||
- Ingress: `ingress.domain` and `ingress.clusterIssuer` `required` **without defaults**; `ingress.className` defaults to `nginx`; optional `ingress.annotations` merged with the cert-manager annotation; TLS secret `tls-<release>-<chart>-ingress`. Documented exception: homeassistant keeps its flexible upstream ingress style (className default `nginx`).
|
||
- Secrets are 1Password `OnePasswordItem` CRs: values expose `secrets.<name>SecretRef` holding an `op://` item path; the operator materializes the Kubernetes Secret.
|
||
- Config changes must restart the workload (#82): every pod template consuming a ConfigMap carries `checksum/config: {{ include (print $.Template.BasePath "/<chart>.configmap.yaml") . | sha256sum }}` (one line per ConfigMap). Without it a changed ConfigMap restarts nothing — `subPath` mounts are never refreshed and apps read config only at start. Adding it to an existing chart is a **major** bump. Secrets from `OnePasswordItem` cannot be hashed (Helm never sees their content); charts mounting them document the limitation in `values.yaml`.
|
||
- Every container offers an additive `env` block rendered through `tpl` with `value`, `secretKeyRef`, and `configMapKeyRef` forms.
|
||
- Hardening: `automountServiceAccountToken: false`, non-root, `readOnlyRootFilesystem: true`, caps dropped, `allowPrivilegeEscalation: false`; contexts values-configurable (`securityContext.pod` / `securityContext.container`); writable paths as emptyDirs; image-forced deviations MUST be documented with a comment in the chart's `values.yaml`.
|
||
- Probes: liveness/readiness/startup on every workload (exception: paperless consume CronJob), values-configurable with calibrated defaults.
|
||
- Resources: requests as values defaults; limits auto-computed by the `_helpers.tpl` helper (`limitFactor` default 3; no CPU limits unless set explicitly).
|
||
- Replicas per workload (explicit `0` allowed) plus a chart-global `scaleDown` flag that overrides them.
|
||
- `icon:` mandatory in every `Chart.yaml`; `helm lint --strict` without findings; CI fixtures under `ci/<chart>/` or a justified exclusion.
|
||
- **Chart dependencies are admissible but expensive** (the former "no dependencies" rule fell with #214; `ytdlp-web-player` was the first to use one, `stalwart` the second, `paperless-ngx` the third). A dependency has to be optional and off by default via `condition:`, and upstream's chart is consumed as it comes — where its defaults fall short of these conventions, that is **documented in our `values.yaml`, never patched**. The price to know before adding the next one: a subchart is invisible to everything downstream that watches Helm release versions, so a new upstream version — a security fix included — reaches a cluster only when the outer chart is republished (Renovate's built-in `helmv3` manager plus `ci/renovate-bump-chart.sh` is the whole safety net, and it belongs in a comment at the pinned version); and every helm command exits 1 while `charts/` is unfilled, which is why `ci/helm-dependencies.sh` runs before lint, package, render and install — including the render test that covers **all** charts on **every** PR. Full rules and the rest of the cost: README, "Chart dependencies".
|