ytdlp-web-player: new chart with an optional oauth2-proxy subchart (1.0.0+upv1.2.0)
Deploys matszwe02/ytdlp_web_player, a Flask app that fetches videos with
yt-dlp, transcodes them with ffmpeg and serves them as HLS. It ships no
authentication of any kind, so the chart carries a preconfigured oauth2-proxy
as an optional subchart, in the data path in front of the app.
This is the first chart in the repo with a dependency, so the rule "charts
have no dependencies" falls here. It is replaced in CLAUDE.md and README.md by
a bar: a dependency has to be optional and off by default, upstream's chart is
consumed as it comes (documented where it falls short of our conventions,
never patched), and the price is written down - above all that a subchart is
invisible to everything downstream watching Helm release versions, so a new
proxy release only reaches a cluster when this chart is republished.
CI learns the dependency in five places, not the three the plan expected:
helm lint, package, template and install all exit 1 while charts/ is unfilled,
so ci/helm-dependencies.sh runs before lint and package in validate-charts.yml,
before package in publish-helm-chart.yml, and inside run-install-test.sh,
run-trivy-config.sh and run-quantity-render-test.sh - the last of which renders
every chart on every PR and would have gone red first.
Notable chart decisions, each measured rather than assumed:
- The image has no USER and runs as root by default, but not by necessity: it
runs under an arbitrary uid with readOnlyRootFilesystem and the full
hardening baseline. Only the pinned uid is a deviation, and it is documented.
- Two emptyDirs, each justified separately. /data holds the video cache
(DATA_PATH). /tmp is NOT mounted - measured across metadata extraction, HLS
transcoding, sprite generation and audio extraction, the image never writes
there. What it does write is $HOME/.cache, so /cache is mounted with HOME and
XDG_CACHE_HOME pointed at it; without it yt-dlp re-downloads its JavaScript
challenge-solver from github.com on every extraction.
- The proxy is wired through config.configFile, the only value the subchart
passes through tpl - config.upstreams is emitted with toJson and would carry
a literal {{ .Release.Name }} into the manifest.
- Rendering aborts when this chart's Ingress and the proxy's claim the same
host: ingress-nginx rejects the second one and a Fleet bundle then sits in
ErrApplied until forceSyncGeneration is bumped.
Fixes #214
This commit is contained in:
23 files changed
+1706
-8
No files matched your search
@@ -28,6 +28,7 @@ Collection of some self-made helm-charts. Each top-level directory is a standalo
|
||||
| `template-chart` | — | Scaffold for new charts (placeholder `xxx`), never published |
|
||||
| `urlaubsverwaltung` | Urlaubsverwaltung | Deployment, DB + OIDC secrets |
|
||||
| `voucher-vault` | VoucherVault | StatefulSet, DB + OIDC secrets |
|
||||
| `ytdlp-web-player` | ytdlp_web_player | Stateless Deployment, video cache and tool cache as emptyDirs (no PVC). **The only chart with a dependency**: an optional `oauth2-proxy` subchart, off by default, wired in front of the app because the application ships no authentication at all — see "Chart dependencies" below |
|
||||
| `zipline` | Zipline | StatefulSet, DB secret |
|
||||
|
||||
## Versioning
|
||||
@@ -42,6 +43,7 @@ Collection of some self-made helm-charts. Each top-level directory is a standalo
|
||||
- `publish-<chart>.yml` — one thin workflow per chart; triggers on pushes to `main` touching `<chart>/**` (or manually) and calls the reusable workflow below. **A push to a chart directory on `main` publishes that chart**, so the chart version must be bumped in the same change.
|
||||
- `publish-helm-chart.yml` — reusable workflow: `helm package`, then `helm push` to the private JK-Registry. Registry credentials come from 1Password Connect (`OP_CONNECT_TOKEN` secret, items under `op://GitHub-Actions/`). The loaded values are read via **`steps.<id>.outputs.*`**, mapped into each step's own `env:` block and used as quoted shell variables — never interpolated into the command line. `load-secrets-action` **v5 no longer exports them as environment variables** for later steps the way v4 did, so after the v5 bump every `${{ env.JK_CR_* }}` reference resolved to an empty string while the loading step still logged `Populating variable` and stayed green. Each value is additionally guarded with `:?`, because an empty one used to collapse into a missing argument rather than an error: `-u` consumed the following `--password-stdin`, helm fell back to an interactive prompt and the job died with `inappropriate ioctl for device` — a terminal error for what was really an unset variable.
|
||||
- `change-app-version.yml` — manually dispatched app-version bump; the chart must be listed in its `chart-name` choice list.
|
||||
- `ci/helm-dependencies.sh <chart-dir>` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one). It runs in **five** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in `publish-helm-chart.yml`, and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs.
|
||||
- `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (same changed-charts matrix) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix. The **resource-quantity render test** is the exception to that rule: it runs over every chart on every PR — see below.
|
||||
|
||||
### Resource-quantity render test (all charts, every PR)
|
||||
@@ -55,7 +57,7 @@ Collection of some self-made helm-charts. Each top-level directory is a standalo
|
||||
|
||||
The two cases exist because the bug behind #157 — quantities formatted with `printf "%v"`, where the `float64` the null-safe fallback helpers produce flips to exponential notation from about `1e6` — has a loud and a silent half. Loud: the reformatted quantity carries the suffix `e+06`, the helper rejects it, `helm template` exits non-zero. Silent: the *product* of a fractional `limitFactor` and a bare number is reformatted **after** the suffix check, so the render succeeds and the manifest carries `memory: "1.4999985e+06"` — rejected by nothing until the API server sees it at apply time. A check on the exit code alone therefore proves only half the case; the second case searches the **rendered text** for a resource quantity in exponential notation. `999999` stays below `1e6` on purpose, so the fraction case isolates the second formatting site instead of tripping over the first.
|
||||
|
||||
Why all charts rather than the changed ones: charts here have no dependencies, so the resources helper is copied into every chart's `templates/_helpers.tpl`. The regression arrives by copy — a new chart started from an older chart, or from `template-chart` before the fix — and in that PR the copied bug is in no diff anyone reads. Running over everything also removes the list nobody would maintain: adding a chart enrolls it automatically. Two renders per chart, no cluster, a few seconds for the whole repo; the job depends on nothing else and finishes well inside the `build-chart` runtime.
|
||||
Why all charts rather than the changed ones: the resources helper is copied into every chart's `templates/_helpers.tpl` rather than shared through a library chart. The regression arrives by copy — a new chart started from an older chart, or from `template-chart` before the fix — and in that PR the copied bug is in no diff anyone reads. Running over everything also removes the list nobody would maintain: adding a chart enrolls it automatically. Two renders per chart, no cluster, a few seconds for the whole repo; the job depends on nothing else and finishes well inside the `build-chart` runtime.
|
||||
|
||||
Two details worth knowing before touching it:
|
||||
|
||||
@@ -137,7 +139,8 @@ If the dependency dashboard reports `Failed to look up github-releases package
|
||||
1. Copy `template-chart/` to `<chart-name>/` and replace every `xxx` placeholder: template file names (`xxx.deployment.yaml` → `<app>.deployment.yaml`), `Chart.yaml`, the `values.yaml` keys, the `.Values.xxx.*` references inside the templates, **and both the `xxx.` helper prefixes and the values paths in `values.schema.json`**. Then walk the five points in "Values that must not lie" below — the schema and the null-safe fallbacks are copied with the template, but their content is chart-specific and wrong until you adapt it. The same goes for `templates/NOTES.txt`: it comes with the template as a worked example and is placeholder text until you replace it with what an operator of *your* app needs after the install (see "Everything else" below); its leading `{{/* … */}}` block explains the bar and is meant to be deleted.
|
||||
2. Add `.github/workflows/publish-<chart-name>.yml` (copy an existing one, e.g. `publish-outline.yml`).
|
||||
3. Add the chart name to the choice list in `.github/workflows/change-app-version.yml`.
|
||||
4. Add a `customManagers` block for the chart's main image in `renovate.json5` (copy a neighbouring one, swap `managerFilePatterns` and `depNameTemplate`) — without it the chart's app version is never updated. Check the image's tag list first: if it carries a flavour, date or prerelease suffix, add a `versioningTemplate` with a comment saying why.
|
||||
4. Add a `customManagers` block for the chart's main image in `renovate.json5` (copy a neighbouring one, swap `managerFilePatterns` and `depNameTemplate`) — without it the chart's app version is never updated. Check the image's tag list first: if it carries a flavour, date or prerelease suffix, add a `versioningTemplate` with a comment saying why. **Floating tags belong outside the pattern**: `ytdlp-web-player` anchors its versioning regex on the `v` prefix precisely so `latest` and `stable` — both of which move — can never be selected.
|
||||
5. Only if the chart needs a **dependency**: read "Chart dependencies" in the style guidelines first, it is a bar rather than a formality. Nothing has to be added to the pipelines — `ci/helm-dependencies.sh` already runs everywhere and keys off the `dependencies:` block — but the subchart's own updates then depend entirely on Renovate's built-in `helmv3` manager plus a chart-version bump, which is the part to write down at the pinned version.
|
||||
|
||||
Validate locally with:
|
||||
|
||||
@@ -201,7 +204,7 @@ These guidelines are **binding for every current and future chart** in this repo
|
||||
### 7. Resources (auto-limits)
|
||||
|
||||
- Resource requests ship as chart defaults in `values.yaml` and are user-overridable.
|
||||
- Limits are computed by a per-chart `_helpers.tpl` helper (reference implementation in `template-chart/templates/_helpers.tpl`): an explicitly set limit always wins, missing memory and ephemeral-storage limits default to `resources.limitFactor` × the request (factor default `3`), and a CPU limit is only rendered when set explicitly — otherwise CPU stays unlimited (it is compressible, throttling instead of OOM-killing). The helper is duplicated into every chart because charts have no dependencies.
|
||||
- Limits are computed by a per-chart `_helpers.tpl` helper (reference implementation in `template-chart/templates/_helpers.tpl`): an explicitly set limit always wins, missing memory and ephemeral-storage limits default to `resources.limitFactor` × the request (factor default `3`), and a CPU limit is only rendered when set explicitly — otherwise CPU stays unlimited (it is compressible, throttling instead of OOM-killing). The helper is copied into every chart rather than shared through a library chart — which would be a second artifact to publish, version and pull for a helper that changes about once a year; the resource-quantity render test above exists because a copy can carry an old bug into a diff nobody reads.
|
||||
|
||||
### 8. Secrets
|
||||
|
||||
@@ -277,4 +280,23 @@ Reference implementation: `template-chart/values.schema.json` and `template-char
|
||||
- `helm lint --strict` must pass without findings.
|
||||
- **Every chart ships a `templates/NOTES.txt`** (#43), `template-chart` included — so this is a requirement for a new chart, not a nice-to-have it can defer. No list of names here on purpose: a count goes stale the moment the next chart lands, which is exactly how the previous wording came to claim a number it never matched. Reference implementation: `template-chart/templates/NOTES.txt`, a placeholder-carrying template rather than finished prose, so a new chart adapts it instead of starting from an empty file. The bar it sets: print what an operator would otherwise have to look up in the templates (the rendered access URL, **which fields the 1Password item must carry**, the options actually active in this release, and the chart's known traps), conditional on the values really set. A `NOTES.txt` that only repeats the values is skimmed once and never read again. **No real domain anywhere** — the URL renders from `ingress.domain`, and where an example is unavoidable it is `example.com`. A `NOTES.txt` renders no Kubernetes object, so writing or reworking one is purely additive: a minor bump.
|
||||
- CI: every installable chart has fixtures under `ci/<chart>/` (`test-values.yaml`, optionally `fixtures.yaml`/`fixtures.env.yaml`/`settings.env`) or a justified entry in `ci/excluded-charts.txt` — the install-test fails if both are missing.
|
||||
- Charts have no dependencies (`charts/` directories are empty).
|
||||
### 14. Chart dependencies
|
||||
|
||||
Until `ytdlp-web-player` (#214) this section read "charts have no dependencies (`charts/` directories are empty)". That rule is gone, replaced by a bar rather than a ban — a dependency is **admissible, not free**, and the reason it stopped being a ban is worth keeping: the chart needed an oauth2-proxy in the data path in front of an application that ships no authentication at all, and proxy and app are only useful wired to each other.
|
||||
|
||||
**When a dependency earns its place.** All three, not two of three:
|
||||
|
||||
1. The dependent piece would otherwise be a second Helm release that only exists to serve this one, with the wiring between them living in neither chart.
|
||||
2. Upstream is the right owner of it. We consume their chart as it comes: **we do not patch it, do not re-harden it, do not re-implement its values.** Where their defaults fall short of the guidelines above, that is **documented in our `values.yaml`**, not fixed by us — and a values-level fix stays available to whoever installs it.
|
||||
3. It is **optional and off by default** (`condition:` in `Chart.yaml`), so the chart is complete without it and a default install renders none of its objects.
|
||||
|
||||
**What it costs, all of it visible before the next one is added:**
|
||||
|
||||
- **The subchart is invisible to everything downstream.** Its version is inside our packaged `.tgz`, so anything that watches Helm release versions — Fleet's `fleet.yaml` in `cluster-config` included — sees only *our* chart version. A new upstream release, security fix included, reaches a cluster **only** when we publish a new version of the outer chart. Nothing reports the gap. Renovate's built-in `helmv3` manager opens the PR (no `customManagers` entry needed — the repository URL and version sit together in one file) and `ci/renovate-bump-chart.sh` bumps the chart patch that ships it; that chain is the entire safety net. State this in a comment at the pinned version, not only in a review.
|
||||
- **Every pipeline that touches the chart needs a network fetch first.** `helm lint`, `helm package`, `helm template` and `helm install` all exit 1 while a declared dependency is missing from `charts/`, so `ci/helm-dependencies.sh` runs before each of them (see CI/CD above). One of those call sites is the resource-quantity render test, which runs over **every** chart on **every** PR — a dependency-carrying chart can therefore turn a third-party Helm repository's bad day into a red pipeline on a PR that touched nothing related.
|
||||
- **`charts/*.tgz` and `Chart.lock` are not committed** (`.gitignore` says why): the archive is a third-party binary nobody reviews in a diff, and a committed lock would arrive stale because Renovate rewrites `Chart.yaml` and cannot regenerate it — which is also why the pipelines run `helm dependency update` and not `helm dependency build`.
|
||||
- **The repo's own mechanisms stop at the subchart boundary.** `checksum/config`, the 1Password auto-restart annotation and the `scaleDown` flag reach our templates only. A chart that renders a `OnePasswordItem` for its subchart documents the manual `kubectl rollout restart` that replaces the missing restart-on-change.
|
||||
|
||||
Two implementation notes that cost time otherwise: a values key containing a dash is unreachable as `.Values.oauth2-proxy` (that parses as a subtraction) and needs `index .Values "oauth2-proxy"`, while `condition:` in `Chart.yaml` splits on dots only and needs no escaping. And inside a subchart `.Chart.Name` is the **subchart's** name — a value templated into it (`config.configFile` in this case) must spell the parent chart out literally; only `.Release.Name` is shared.
|
||||
|
||||
Charts that need no dependency still have none, and `charts/` stays empty for all of them. `template-chart` remains dependency-free: the scaffold should not carry a `dependencies:` block a new chart has to remember to delete.
|
||||
Reference in new issue
Block a user