From a6c27ebef3fa355c6c860da5f1822ce557875193 Mon Sep 17 00:00:00 2001 From: MeepMr <67143880+MeepMr@users.noreply.github.com> Date: Mon, 14 Sep 2026 17:07:31 +0200 Subject: [PATCH] Prepare the Gitea move: measured CI deviations, self-hosted Renovate, issue migration script Groundwork for #225, built on a throwaway pull request against the Gitea instance rather than on assumptions. The move itself is not part of this change - no Actions are switched off, no mirror is enabled, no branch protection is touched. What the probe run measured, and what follows from it: - Which jobs need a Docker socket: only the install-test. Measured per job - detect-changed-charts, build-chart, the quantity render and the Trivy config scan all pass on `ubuntu-latest`, which on that instance has no socket. The install-test dies there at "Create k3d cluster" with "Cannot connect to the Docker daemon", an error naming k3d and never the runner label. Its `runs-on` is now an expression: the label `ubuntu-docker` on Gitea, `ubuntu-latest` on GitHub, because the former does not exist there and a chart PR would otherwise queue forever during the handover. Gitea evaluates it - the probe job was taken by the docker runner. - The install-test still cannot pass on Gitea. That daemon is rootless with "Cgroup Driver: none"; the k3s node restarts immediately with an empty log because k3s needs cgroup delegation. The label is necessary and not sufficient, and the fix is the maintainer's. - aquasecurity/setup-trivy cannot be used on Gitea: it checks out aquasecurity/trivy from github.com with `github.token`, which there is a Gitea token, and ends on "could not read Username for https://github.com". Replaced by the release tarball - one request, same behaviour on both forges, green on the probe run. Also in this change: - .github/workflows/renovate.yml, Renovate in self-operation: a daily scheduled run in the official container, inert on github.com via a `github.server_url` guard. Every rule proved by dry run instead of assumed - the pinned Renovate image is tracked by the built-in github-actions manager (44.80.0 -> 44.87.1, minor), TRIVY_VERSION is still tracked after setup-trivy is gone (v0.72.0 -> v0.74.0, minor), and `ubuntu-docker` is not extracted by the github-runners manager at all, so nothing will quietly "correct" it. - ci/migrate-issues-to-gitea.sh, runnable, with --dry-run, and NOT run. Its header names what it cannot do: comment authorship, the # references between issues, and attachments. - CLAUDE.md and README.md on the new flow: which forge and which tool, auto-merge on green checks in place of the merge queue Gitea does not have, and the measured differences of the two CIs. Refs #225 - deliberately no "Fixes": the move is not finished with this. --- .github/workflows/renovate.yml | 108 +++++++++++ .github/workflows/validate-charts.yml | 54 +++++- CLAUDE.md | 17 +- README.md | 36 +++- ci/migrate-issues-to-gitea.sh | 256 ++++++++++++++++++++++++++ renovate.json5 | 15 +- 6 files changed, 469 insertions(+), 17 deletions(-) create mode 100644 .github/workflows/renovate.yml create mode 100755 ci/migrate-issues-to-gitea.sh diff --git a/.github/workflows/renovate.yml b/.github/workflows/renovate.yml new file mode 100644 index 0000000..2d967f0 --- /dev/null +++ b/.github/workflows/renovate.yml @@ -0,0 +1,108 @@ +# Renovate, self-hosted (issue #225). +# +# On github.com Renovate arrived as the `jlab-renovate` GitHub App and this +# repo only ever carried renovate.json5. Gitea has no app model and no hosted +# Renovate, so after the move the bot has to be *run* by this repo: a scheduled +# job with its own token. That is the only change to how dependencies are +# updated - renovate.json5 stays the single source of what gets updated, and +# everything in it works unchanged on the Gitea platform. +# +# What the maintainer has to create once on the instance (the token this +# workflow cannot create for itself): +# +# RENOVATE_TOKEN Gitea access token of the bot account, scopes +# read:user, write:issue, write:repository. Renovate opens +# the PRs and the dependency dashboard issue with it, so +# the PRs will be authored by whoever owns the token. +# GITHUB_COM_TOKEN Read-only github.com PAT (public_repo / no scopes at all +# is enough). NOT optional here: every `github-releases` and +# `github-tags` lookup in renovate.json5 - and the whole +# built-in github-actions manager, which resolves +# actions/checkout and friends on github.com - hits +# github.com's unauthenticated rate limit within one run +# otherwise. On the App this came for free; here it does not. +# README already names this for the `no-result` symptom. +# +# There is deliberately no `allowedCommands` to set up by hand any more: the +# postUpgradeTasks hook renovate.json5 needs (ci/renovate-bump-chart.sh, which +# recomputes `version:` after an appVersion change) is allow-listed by the +# RENOVATE_ALLOWED_COMMANDS env below, because this workflow *is* the instance +# config now. Without it Renovate skips the task silently and every chart PR +# stays red on the version-bump check - see README, "The version-bump hook". +name: Renovate + +on: + schedule: + # 03:40 UTC, once a day. Renovate's own `timezone` in renovate.json5 + # (Europe/Berlin) governs the PR schedules; this cron is plain UTC. + - cron: "40 3 * * *" + workflow_dispatch: + inputs: + dryRun: + description: "Dry run: 'full' looks up and renders everything but writes nothing" + type: choice + default: "full" + options: ["full", "lookup", "extract", "disabled"] + logLevel: + description: "Renovate log level" + type: choice + default: "info" + options: ["info", "debug"] + +jobs: + renovate: + name: Renovate + # The guard that keeps this workflow inert on github.com. The file reaches + # GitHub as well (that is where the migration PR is merged), and a + # scheduled run there would authenticate a *Gitea* token against a Gitea + # endpoint from a runner that has neither - failing nightly for a reason + # nobody is watching for. GITHUB_SERVER_URL is the one context value that + # differs between the two platforms without naming our instance: Gitea + # sets it to its own base URL. Delete this line once github.com is a + # push mirror without Actions. + if: github.server_url != 'https://github.com' + # No Docker socket needed: the Gitea runner starts every job in a container + # itself, and `container:` is the runner's job to create, not the job's. + # Measured on the probe PR of #225 - the same distinction that puts + # install-test on ubuntu-docker leaves this job on ubuntu-latest. + runs-on: ubuntu-latest + # The official image rather than `npx renovate`: it ships the git, node and + # package-manager versions Renovate expects, and pulling ~200 MB from npm + # on every scheduled run is the alternative. Renovate's built-in + # github-actions manager keeps this tag current (proved by dry run, see the + # migration PR) - it reads `container:` images, not only `uses:`. + container: ghcr.io/renovatebot/renovate:44.87.1 + steps: + - name: "Checkout Repository" + uses: actions/checkout@v7 + + - name: "Run Renovate" + env: + # Platform. The endpoint is derived from the run itself so the + # instance address stays out of this repository, the same rule the + # publish workflow follows for the registry URL. + RENOVATE_PLATFORM: gitea + RENOVATE_ENDPOINT: ${{ github.server_url }}/api/v1 + RENOVATE_TOKEN: ${{ secrets.RENOVATE_TOKEN }} + GITHUB_COM_TOKEN: ${{ secrets.GITHUB_COM_TOKEN }} + + # Exactly this repository. Autodiscovery would enrol every repo the + # token can see, and the token is an account token, not an app + # installation scoped to one repo. + RENOVATE_AUTODISCOVER: "false" + RENOVATE_REPOSITORIES: ${{ github.repository }} + + # Who the commits are by. The e-mail must not be a real address of a + # person - Gitea attributes a commit to the account whose verified + # address matches, and these commits are the bot's. + RENOVATE_GIT_AUTHOR: "Renovate Bot " + + # The instance-level half of renovate.json5's postUpgradeTasks. A + # command not listed here is skipped WITHOUT an error message. + RENOVATE_ALLOWED_COMMANDS: '["^ci/renovate-bump-chart\\.sh"]' + + # Manual runs default to a dry run; the schedule is the only thing + # that writes on its own. + RENOVATE_DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dryRun || 'null' }} + LOG_LEVEL: ${{ github.event_name == 'workflow_dispatch' && inputs.logLevel || 'info' }} + run: renovate diff --git a/.github/workflows/validate-charts.yml b/.github/workflows/validate-charts.yml index 2488734..bafcc42 100644 --- a/.github/workflows/validate-charts.yml +++ b/.github/workflows/validate-charts.yml @@ -199,11 +199,27 @@ jobs: uses: actions/checkout@v7 - name: "Install Helm" uses: azure/setup-helm@v5 + # A plain release download rather than aquasecurity/setup-trivy, and the + # reason is measured on the Gitea probe PR of #225, not assumed. That + # action runs actions/checkout against aquasecurity/trivy on github.com + # and hands it `github.token` - which on Gitea is a GITEA token. + # github.com answers 401, checkout retries twice and the job ends on + # "fatal: could not read Username for 'https://github.com': terminal + # prompts disabled", an error naming neither the action nor the token. + # Its actions/cache step is inert on the instance as well ("Cache action + # is only supported on GHES version >= 3.5"), and fetching the nested + # actions cost 4m30s before the job even started. One tarball behaves + # identically on both forges, and the `# renovate:` comment above still + # keeps TRIVY_VERSION current - verified by dry run: pinned to v0.72.0 + # the lookup offers v0.74.0 as a minor update. - name: "Install Trivy" - uses: aquasecurity/setup-trivy@v0.3.1 - with: - version: ${{ env.TRIVY_VERSION }} - cache: true + run: | + curl -sSfL -o /tmp/trivy.tar.gz \ + "https://github.com/aquasecurity/trivy/releases/download/${TRIVY_VERSION}/trivy_${TRIVY_VERSION#v}_Linux-64bit.tar.gz" + mkdir -p "$HOME/.local/bin" + tar -xzf /tmp/trivy.tar.gz -C "$HOME/.local/bin" trivy + echo "$HOME/.local/bin" >> "$GITHUB_PATH" + "$HOME/.local/bin/trivy" --version # Findings land in the job summary, not only in the log: exceptions with # their reasons come from ci/trivy/trivyignore.yaml, the trusted-registry # list from ci/trivy/data/ksv0125.yaml. @@ -214,7 +230,35 @@ jobs: name: "Install-test ${{ matrix.chart }}" needs: [detect-changed-charts, build-chart] if: needs.detect-changed-charts.outputs.any-install == 'true' - runs-on: ubuntu-latest + # The one job in this repo that does not simply say ubuntu-latest, and the + # expression is not decoration - the same label means different machines on + # the two forges this repo currently lives on (#225). + # + # This is the only job that needs a Docker socket: it creates a k3d + # cluster, and k3d is a Docker client. On GitHub's hosted runners every + # label carries Docker, so the question never came up. On the Gitea + # instance it was MEASURED on a probe pull request, per job: + # detect-changed-charts, build-chart, the quantity render and the Trivy + # scan all pass on `ubuntu-latest` - none of them needs Docker - while the + # install-test dies there at "Create k3d cluster" with "Cannot connect to + # the Docker daemon at unix:///var/run/docker.sock", an error that points + # at k3d and never at the runner label. `ubuntu-docker` is the label that + # carries a socket, and Gitea does evaluate this expression: the probe job + # was picked up by the docker runner with labels ["ubuntu-docker"]. + # + # Why an expression and not a plain `ubuntu-docker`: that label does not + # exist on GitHub, so every chart PR would sit queued forever for as long + # as this repo still builds on github.com - blocking the merge with no + # error anywhere. github.server_url is the one context value that separates + # the two without naming our instance. + # + # KNOWN, and not fixed by this line: `ubuntu-docker`'s daemon is ROOTLESS + # and reports "Cgroup Driver: none" plus "Running in rootless-mode without + # cgroups". k3d then gets as far as starting the k3s node, which restarts + # immediately with an empty log, because k3s needs cgroup delegation. Until + # the instance's Docker-in-Docker provides it, this job cannot pass on + # Gitea - see #225. The label is necessary, it is just not sufficient. + runs-on: ${{ github.server_url == 'https://github.com' && 'ubuntu-latest' || 'ubuntu-docker' }} # Blocking since Renovate automerges patch/minor updates: a chart that no # longer starts must stop the PR before it reaches main and publishes. This # ends the stabilization phase of issue #22 - the job reported failures diff --git a/CLAUDE.md b/CLAUDE.md index 3b33edf..2e6b88b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -7,16 +7,27 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co The main Claude session acts as the **task manager** and does not implement issues itself: 1. **Plan first, get approval**: The manager presents a plan for the issue and waits for the user's explicit approval. No implementation before approval. -2. **Document the final plan on the issue**: Once approved, the plan is posted as a comment on the GitHub issue before implementation starts (skip if the issue body already contains the full plan; later decisions go there as comments too). +2. **Document the final plan on the issue**: Once approved, the plan is posted as a comment on the issue before implementation starts (skip if the issue body already contains the full plan; later decisions go there as comments too). 3. **Delegate implementation**: After approval, the manager launches a dedicated **development agent**. The agent implements the issue completely on its own branch and opens a PR. 4. **PR comment handling by the agent**: After opening the PR, the development agent watches it for review comments. New comments are addressed automatically (code changes, reply on the thread), and addressed threads are marked as **resolved**. 5. **Agent lifetime**: The development agent terminates itself once its PR has been merged (or closed). 6. Only after the PR has been **merged by the user** may work on the next issue begin. The user merges; never merge yourself. -7. **After the merge, verify the linked issue was closed** (auto-close via "Fixes #n"); close it manually if not. +7. **After the merge, verify the linked issue was closed** (auto-close via "Fixes #n"); close it manually if not. Gitea auto-closes on the same keywords, but only for an issue in the same repository — a migrated issue whose number changed is closed by hand. Additional rules for all commits, PRs, issues, and comments: - Do **not** add a `Co-Authored-By` trailer and do **not** append "Generated with Claude Code" footers. +- **Commits must be signed.** Verify the signature before opening the PR; if `op-ssh-sign` fails, stop and report rather than pushing unsigned work. + +## Which forge, and with which tool + +The repository is moving from github.com to the Gitea instance JLab-Git (#225). The move is **not finished**, so both are live and the steps below differ by where the work happens. README, "Where this repo lives", holds the measured differences between the two CIs; this section holds only what changes about the *workflow*. + +- **Remotes are named `github` and `JLab-Git`** — there is no `origin`. Say which one you mean; `git push` without a remote is ambiguous here. +- **Today the merge flow still runs on GitHub.** Issues, pull requests and merges happen there until the maintainer switches Actions off and turns on the mirror. Until then, **do not push to `JLab-Git/main`**: with publish workflows armed on both sides, two CIs publish into the same registry, and that registry accepts a re-upload of an existing version silently, replacing the artifact (#160). Pushing a *branch* to Gitea is fine — only `main` triggers a publish. +- **`gh` only as `op plugin run -- gh`**, `tea` only as `op plugin run -- tea`. A bare `gh` falls back to the wrong identity. Every call costs a 1Password confirmation, so bundle them and never poll. +- **After the move: no merge queue.** Gitea has none. Its place is taken by **auto-merge on green checks** — the PR is armed once and merges itself when the required checks pass. The manager still decides *whether* to arm it; rule 6 above is unchanged: the user merges, and an agent never merges its own PR. +- **Issue numbers will not survive the move.** Gitea renumbers, and issues and PRs share one counter. A `#n` written today keeps pointing at the GitHub issue; `ci/migrate-issues-to-gitea.sh` writes the old→new mapping but deliberately rewrites nothing. ## Keep the docs current @@ -26,7 +37,7 @@ Additional rules for all commits, PRs, issues, and comments: ## What this repo is -A collection of self-made Helm charts, one directory per chart, published to a private, classic (index-based) Helm registry via GitHub Actions — an HTTP `POST` of the packaged `.tgz`, not an OCI `helm push`; the target URL and its credentials live in 1Password and never in this repository. Validation happens on three levels: locally with Helm, in the PR pipeline (`validate-charts.yml`: strict lint, package, version-bump check), and in the PR install-test stage (k3d, fixtures under `ci/` — see README): +A collection of self-made Helm charts, one directory per chart, published to a private, classic (index-based) Helm registry by CI (GitHub Actions today, Gitea Actions after #225; the workflow files are read by both unchanged) — an HTTP `POST` of the packaged `.tgz`, not an OCI `helm push`; the target URL and its credentials live in 1Password and never in this repository. Validation happens on three levels: locally with Helm, in the PR pipeline (`validate-charts.yml`: strict lint, package, version-bump check), and in the PR install-test stage (k3d, fixtures under `ci/` — see README): ```bash helm lint --strict diff --git a/README.md b/README.md index 149f8c5..3cb6b63 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,8 @@ # helm-charts -Collection of some self-made helm-charts. Each top-level directory is a standalone Helm chart that gets packaged and uploaded to a private, classic (index-based) Helm registry via GitHub Actions. +Collection of some self-made helm-charts. Each top-level directory is a standalone Helm chart that gets packaged and uploaded to a private, classic (index-based) Helm registry by CI. + +> **The repository is moving from github.com to the Gitea instance JLab-Git (#225).** The code is already on both and identical; what still runs the show is GitHub. The workflow files under `.github/workflows/` are read by both forges unchanged, and "Where this repo lives" below names the handful of places where the two behave differently — each of them measured on a probe pull request, not assumed. ## Charts @@ -38,7 +40,7 @@ Collection of some self-made helm-charts. Each top-level directory is a standalo - Bump the **chart version** in every change that touches a chart directory — a push to `main` publishes the chart. - **App-version updates go through PRs** (so the validate and install-test pipelines run). The manually dispatched `Change App-Version` workflow (rewrites `version:`/`appVersion:` and commits directly to `main`) exists but bypasses both pipelines. For app **major** jumps, update stepwise: first to the last release of the old major line (own PR, merge, publish), then to the newest major. -## CI/CD (GitHub Actions) +## CI/CD - `publish-.yml` — one thin workflow per chart; triggers on pushes to `main` touching `/**` (or manually) and calls the reusable workflow below. **A push to a chart directory on `main` publishes that chart**, so the chart version must be bumped in the same change. - `publish-helm-chart.yml` — reusable workflow: `helm package`, then a single HTTP `POST` of the packaged `.tgz` to the private chart registry. The target is a **classic, index-based** Helm registry (Gitea's `/api/charts` endpoint), not an OCI one: `helm push` speaks only OCI and is not used, there is no `helm registry login` session, and every request carries its own Basic auth. Credentials **and the target URL** come from 1Password Connect (`OP_CONNECT_TOKEN` secret, item `op://GitHub-Actions/jlab-registry-access/` with `username`, `password`, `charts-url`); the workflow appends `/api/charts` to `charts-url`. No registry address ever appears in this repository. The loaded values are read via **`steps..outputs.*`**, mapped into the step's own `env:` block and used as quoted shell variables — never interpolated into the command line. `load-secrets-action` **v5 no longer exports them as environment variables** for later steps the way v4 did, so after the v5 bump every `${{ env.* }}` reference resolved to an empty string while the loading step still logged `Populating variable` and stayed green. Each value is additionally guarded with `:?`, because an empty one used to collapse into a missing argument rather than an error. Three properties of the upload step are load-bearing: @@ -51,6 +53,22 @@ Collection of some self-made helm-charts. Each top-level directory is a standalo - `ci/helm-dependencies.sh ` — not a workflow but the step they all share: fetches a chart's declared dependencies into `charts/`, and returns silently for the charts that declare none (all but one). It runs in **five** places, because every helm command this repo builds on exits 1 while a declared dependency is missing: before lint and package in `validate-charts.yml`, before package in `publish-helm-chart.yml`, and inside `ci/run-install-test.sh`, `ci/run-quantity-render-test.sh` and `ci/run-trivy-config.sh`. See "Chart dependencies" in the style guidelines for what that costs. - `validate-charts.yml` — PR pipeline: detects which charts a PR touches and, for exactly those, runs `helm lint --strict`, `helm package`, and a chart-version bump check (`template-chart` is exempt from the version check). PRs without chart changes skip the build matrix. After the build stage, an **install-test** job (the same changed-charts matrix, minus the charts listed in `ci/excluded-charts.txt`) installs each changed chart into an ephemeral [k3d](https://k3d.io) cluster on the runner and requires every workload to become Ready — see below. Both matrices follow the changed chart directories, so a PR that only changes the harness under `ci/` exercises nothing by itself; the Trivy scan has a fixed-sample fallback for that case, the install-test has none — bump the affected chart's patch version to pull it into the matrix. The **resource-quantity render test** is the exception to that rule: it runs over every chart on every PR — see below. +### Where this repo lives, and what Gitea Actions does differently + +The move to the Gitea instance JLab-Git (#225) is in progress. Order matters and is not negotiable: **CI provably green on Gitea → GitHub Actions off → mirror on.** Any other order leaves a window in which either no CI exists or two of them publish into the same registry — and that registry accepts a re-upload of an existing version silently, replacing the artifact (#160). + +Everything below was measured on a throwaway pull request against the Gitea instance, not taken from documentation. It is the reason for the two deviations now in `validate-charts.yml`. + +- **Reusable workflows with `secrets: inherit` work.** The 23 thin `publish-.yml` hulls survive the move unchanged. (A missing secret does not fail the job — Gitea fails at *job creation*, so the run shows no steps at all, which is worth knowing before debugging the wrong thing.) +- **`ubuntu-latest` on this instance has no Docker socket; `ubuntu-docker` has.** Same label, different meaning than on GitHub's hosted runners, and only one job in this repo cares: the install-test creates a k3d cluster, and k3d is a Docker client. Measured, per job: `detect-changed-charts`, `build-chart`, the quantity render and the Trivy config scan all pass on `ubuntu-latest` — none of them needs Docker. The install-test on `ubuntu-latest` dies at `Create k3d cluster` with `Cannot connect to the Docker daemon at unix:///var/run/docker.sock`, an error that names k3d and never the runner label. +- **`ubuntu-docker` is necessary but not yet sufficient.** Its daemon (8 vCPU, 64 GB, Docker 29.5.2) is **rootless** and reports `Cgroup Driver: none` plus `WARNING: Running in rootless-mode without cgroups`. k3d gets as far as creating the network, pulling the images and starting the node; the k3s server container then restarts immediately with an empty log. k3s needs cgroup delegation. Until the instance's Docker-in-Docker gets it (rootful, or rootless with systemd delegation), **the install-test cannot run on Gitea at all** — see #225. +- **`aquasecurity/setup-trivy` cannot be used.** It runs `actions/checkout` against `aquasecurity/trivy` on github.com with `github.token`, which on Gitea is a *Gitea* token; github.com answers 401 and checkout retries twice before ending on `fatal: could not read Username for 'https://github.com'`. Its `actions/cache` step is inert on the instance as well (`Cache action is only supported on GHES version >= 3.5`), and fetching the nested actions cost 4m30s before the job started. The workflow now downloads the release tarball instead — one request, identical behaviour on both forges, and the `# renovate:` comment above `TRIVY_VERSION` still keeps it current (proved by dry run: pinned to `v0.72.0`, the lookup offers `v0.74.0` as a minor). +- **Matrix jobs lose their name in the status list.** Gitea does not expand `${{ matrix.chart }}` in a job `name:`; it substitutes the matrix *expression*, so the commit status reads `Validate Helm Charts / Install-test ${{ fromJSON(needs.detect-changed-charts.outputs.install-charts) }} (pull_request)` and **all legs of a matrix share that one context**. Consequence for branch protection: a per-chart required check is not possible on Gitea, and the required-check names differ from GitHub's — they have to be read off a real run rather than copied. +- **Actions are fetched from github.com anonymously and that works.** `actions/checkout`, `azure/setup-helm` and the 1Password action need no replacement; only an action that *itself* authenticates against github.com breaks (see Trivy above). +- **No merge queue.** Gitea has none. Its place is taken by auto-merge on green checks, which for a single maintainer is equivalent in effect and different in one way worth remembering: auto-merge merges as soon as the checks it knows about are green, so it is only as strong as the *required* checks configured on `main`. With none configured it merges immediately. + +Still on the maintainer's side and not delegable: branch protection plus the commit-signature requirement on the Gitea repo, the runner's cgroup problem above, and switching GitHub Actions off before enabling the mirror. The bot token gets `403` on `actions/runners`, `actions/secrets` and `branch_protections`, so none of that can be verified from here. + ### Resource-quantity render test (all charts, every PR) `ci/run-quantity-render-test.sh [chart …]` (no argument = all charts) renders every chart twice with the fixtures the install-test and the Trivy scan already use, injecting a bare integer memory request at **every** `resources` block the chart's `values.schema.json` declares: @@ -90,7 +108,7 @@ Fixture images avoid Docker Hub where possible (`public.ecr.aws`/`ghcr.io`) to d ## Dependency updates (Renovate) -Renovate runs as the self-hosted `jlab-renovate` GitHub App. Its config is **`renovate.json5`** in the repo root (JSON5 for the inline reasoning; requires Renovate >= 41 for `managerFilePatterns`/`customType`). There must be exactly one config file — do not add a second `renovate.json` or `.github/renovate.json`. +Renovate runs as the self-hosted `jlab-renovate` GitHub App while the repo is on github.com, and as a scheduled job of its own (`.github/workflows/renovate.yml`) on Gitea, where no app model exists — see "Renovate on Gitea" below. Which of the two runs it changes nothing about *what* gets updated: its config is **`renovate.json5`** in the repo root (JSON5 for the inline reasoning; requires Renovate >= 41 for `managerFilePatterns`/`customType`). There must be exactly one config file — do not add a second `renovate.json` or `.github/renovate.json`. What Renovate keeps current: @@ -134,9 +152,19 @@ Everything is derived from the base commit, so the script is idempotent across t If the dependency dashboard reports `Failed to look up github-releases package …: no-result` for third-party repos, that is the instance's GitHub credentials, not this config: a GitHub App installation token is not a good fit for looking up repos the app is not installed on. Give the instance a separate read-only PAT as `GITHUB_COM_TOKEN` for github.com datasource lookups. +### Renovate on Gitea + +There is no Renovate app for Gitea, so after the move the bot is run by this repository: `.github/workflows/renovate.yml`, a daily schedule plus a manual dispatch that defaults to a dry run. It executes the official `ghcr.io/renovatebot/renovate` image; no Docker socket is involved, because starting a job's `container:` is the runner's own work — the job only lives in it. The file carries an `if: github.server_url != 'https://github.com'` guard so the same workflow stays inert on github.com, where a Gitea token and a Gitea endpoint would fail nightly for a reason nobody is watching for. + +Three things about it are easy to get wrong and cost nothing to get right: + +- **`GITHUB_COM_TOKEN` is not optional here.** Every `github-releases`/`github-tags` lookup and the whole built-in `github-actions` manager resolve against github.com. Under the App those calls rode on the installation token; a Gitea token is worth nothing there, and unauthenticated github.com runs into the rate limit inside a single pass. The symptom is the `no-result` line on the dependency dashboard, not an error. +- **`allowedCommands` moved into the workflow.** `postUpgradeTasks` only runs when the command is allow-listed in the *instance* config, which this workflow now is: `RENOVATE_ALLOWED_COMMANDS='["^ci/renovate-bump-chart\.sh"]'`. Without it Renovate skips the version-bump hook **silently** and every chart PR stays red on the version check. +- **Renovate keeps its own image current.** The built-in `github-actions` manager reads `container:` images, not only `uses:`. Proved by dry run rather than assumed: pinned to `44.80.0`, the lookup returns `44.87.1` as a `minor` update on branch `renovate/ghcr.io-renovatebot-renovate-44.x`, with `pendingChecks: true` from the 3-day `minimumReleaseAge`. The `ubuntu-docker` runner label is **not** touched by the `github-runners` manager — it is not extracted at all, so nothing will quietly "fix" it back to a version label. + ### PR policy -- **Automerge** for `patch`, `minor` and `digest` updates, after a `minimumReleaseAge` of 3 days and only once the PR pipeline is green — install-test included, which is why that job no longer runs with `continue-on-error`. Minor is in scope because the sidecar images barely move in patch steps (`alpine 3.22` → `3.24`, `nginx 1.30.4` → `1.31.4` are both minor), so a patch-only rule would practically never fire. Renovate merges itself rather than using GitHub auto-merge, which would merge immediately as long as `main` has no required status checks. Majors are merged by hand. +- **Automerge** for `patch`, `minor` and `digest` updates, after a `minimumReleaseAge` of 3 days and only once the PR pipeline is green — install-test included, which is why that job no longer runs with `continue-on-error`. Minor is in scope because the sidecar images barely move in patch steps (`alpine 3.22` → `3.24`, `nginx 1.30.4` → `1.31.4` are both minor), so a patch-only rule would practically never fire. Renovate merges itself (`platformAutomerge: false`) rather than handing the PR to the forge's own auto-merge, which on **both** GitHub and Gitea would merge immediately as long as `main` has no required status checks. That is a statement about the branch protection, not about the platform: once required checks exist on `main`, `platformAutomerge: true` becomes the better setting on either side. Majors are merged by hand. - **Majors** arrive one line at a time (`separateMultipleMajor`): a v1 → v3 jump produces a PR for the latest v2 first and then one for v3 — the stepwise procedure the versioning section requires. - `prConcurrentLimit: 10` / `prHourlyLimit: 4` keep the k3d install-tests from all starting at once. - The manually dispatched `change-app-version.yml` still exists for one-off bumps, but the normal path for app updates is now a Renovate PR. diff --git a/ci/migrate-issues-to-gitea.sh b/ci/migrate-issues-to-gitea.sh new file mode 100755 index 0000000..a5aed2f --- /dev/null +++ b/ci/migrate-issues-to-gitea.sh @@ -0,0 +1,256 @@ +#!/usr/bin/env bash +# Copy this repository's GitHub issues into the Gitea repository, over the API. +# +# Part of the move to JLab-Git (#225). The Gitea repo already exists and keeps +# its history, so Gitea's own "migrate from GitHub" is not available - it +# insists on creating the repository itself. What is left is this: read the +# issues from GitHub, write them to Gitea. +# +# WHAT THIS SCRIPT CANNOT DO - read this before running it +# +# Three losses are inherent to the API path, not bugs to be fixed later. They +# are named here because each of them is invisible in the result: the migrated +# issue looks complete, and only the detail is gone. +# +# 1. COMMENT AUTHORSHIP. Every issue and every comment is created by the +# token's user. Gitea has no "create as another user" outside the admin +# sudo API, and the GitHub accounts do not exist on the instance anyway. +# The script therefore prepends an attribution line to each body naming +# the original author and date. A thread of three people still reads as +# three people - but the avatar, the @mention and every "authored by" +# filter say the token's user wrote all of it. +# +# 2. CROSS-REFERENCES BETWEEN ISSUES. Gitea hands out numbers sequentially, +# and issues and pull requests share one counter on both systems, so +# GitHub's #218 does not become Gitea's #218. Every "#" inside a body, +# a comment, a values.yaml, a NOTES.txt or a commit message then points at +# a different entry - silently, because a wrong number still renders as a +# valid link. This script does NOT rewrite them. It writes the mapping to +# --map-file (oldnew, one per line) so the rewrite can be decided and +# done separately; #225 lists the options and none of them is decided. +# +# 3. ATTACHMENTS. Images and files in a body live behind GitHub's +# user-content CDN. The markdown is copied verbatim, so the links keep +# working exactly as long as the GitHub repo stays reachable - and break +# the day it does not. Re-uploading them is a separate job (download, +# POST to Gitea's attachment API, rewrite the link) and deliberately not +# part of this one. +# +# Smaller ones, for completeness: milestones, assignees, reactions, review +# threads and the closed pull requests are not migrated. Pull requests are +# skipped entirely - a PR without its branch and diff is a dead entry, and +# every branch it referred to is merged or gone. +# +# USAGE +# +# GITEA_URL=https:// GITEA_TOKEN= \ +# ci/migrate-issues-to-gitea.sh --dry-run +# +# --state open|closed|all which issues (default: open) +# --limit N stop after N issues (a trial run on real data) +# --map-file PATH where old->new lands (default: issue-map.tsv) +# --source OWNER/REPO GitHub source (default: jk-cluster/helm-charts) +# --target OWNER/REPO Gitea target (default: jlab-cluster/helm-charts) +# +# The instance address comes from the environment, never from this file - the +# same rule the publish workflow follows for the registry URL. +# +# IDEMPOTENCE. The map file is also the guard: an issue already listed there is +# skipped. Delete the file and the next run creates everything a second time, +# so keep it next to the migration and do not regenerate it from scratch. +# +# REQUIREMENTS. `gh` authenticated against github.com (in this setup that means +# `op plugin run -- gh`, otherwise the identity falls back to the wrong user), +# plus `curl`, `jq` and a Gitea token with write access to issues on the target +# repo. + +set -euo pipefail + +DRY_RUN=0 +STATE=open +LIMIT=0 +MAP_FILE=issue-map.tsv +SOURCE_REPO=jk-cluster/helm-charts +TARGET_REPO=jlab-cluster/helm-charts + +while [ $# -gt 0 ]; do + case "$1" in + --dry-run) DRY_RUN=1 ;; + --state) STATE="${2:?--state needs open|closed|all}"; shift ;; + --limit) LIMIT="${2:?--limit needs a number}"; shift ;; + --map-file) MAP_FILE="${2:?--map-file needs a path}"; shift ;; + --source) SOURCE_REPO="${2:?--source needs OWNER/REPO}"; shift ;; + --target) TARGET_REPO="${2:?--target needs OWNER/REPO}"; shift ;; + -h | --help) sed -n '2,66p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + *) echo "unknown argument: $1" >&2; exit 2 ;; + esac + shift +done + +case "$STATE" in + open | closed | all) ;; + *) echo "--state must be open, closed or all (got: $STATE)" >&2; exit 2 ;; +esac + +for tool in gh curl jq; do + command -v "$tool" >/dev/null 2>&1 || { + echo "missing required tool: $tool" >&2 + exit 1 + } +done + +# The target address is required even for a dry run: a dry run that skips the +# address proves less than it appears to, and the run would then differ from +# the real one in more than the writes. +: "${GITEA_URL:?set GITEA_URL to the instance base URL, e.g. https://git.example.com}" +GITEA_URL="${GITEA_URL%/}" +if [ "$DRY_RUN" -eq 0 ]; then + : "${GITEA_TOKEN:?set GITEA_TOKEN to a token with issue write access on $TARGET_REPO}" +fi + +API="$GITEA_URL/api/v1" + +# The token goes into a curl config file rather than onto the command line, so +# it never appears in the process list - the same reason publish-helm-chart.yml +# pipes its options into `curl --config -`. Here it is a file and not stdin, +# because stdin carries the JSON body. mktemp creates it 0600. +CURL_CFG="$(mktemp)" +trap 'rm -f "$CURL_CFG"' EXIT +if [ "$DRY_RUN" -eq 0 ]; then + printf 'header = "Authorization: token %s"\n' "$GITEA_TOKEN" >"$CURL_CFG" +fi + +# --fail-with-body turns 401/403/422 into a non-zero exit and still prints the +# response. Without it curl exits 0 on all of them and the migration would +# report success having written nothing. +gitea() { + local method="$1" path="$2" + if [ "$method" = GET ]; then + curl --config "$CURL_CFG" --silent --show-error --fail-with-body \ + --request GET "$API/$path" + else + curl --config "$CURL_CFG" --silent --show-error --fail-with-body \ + --request "$method" \ + --header "Content-Type: application/json" \ + --data-binary @- "$API/$path" + fi +} + +say() { printf '%s\n' "$*"; } + +[ "$DRY_RUN" -eq 1 ] && say "DRY RUN - nothing is written to $GITEA_URL" +say "source: github.com/$SOURCE_REPO target: $GITEA_URL/$TARGET_REPO" +say "state: $STATE map file: $MAP_FILE" +say "" + +touch "$MAP_FILE" + +# --- labels ------------------------------------------------------------------ +# +# A label that does not exist on the target is dropped by Gitea's issue API +# without an error, so they are created on demand below. Names are compared +# case-sensitively, which is how both systems store them. +existing_labels="" +if [ "$DRY_RUN" -eq 0 ]; then + existing_labels="$(gitea GET "repos/$TARGET_REPO/labels?limit=100" | jq -r '.[].name')" +fi + +# --- issues ------------------------------------------------------------------ +# +# --paginate walks every page; PRs are filtered out here rather than by an API +# flag, because GitHub's issues endpoint returns them too and only the +# .pull_request key tells them apart. Oldest first, so the Gitea numbers come +# out in the same order as the originals even though the values differ. +issues_json="$(gh api --paginate \ + -H "Accept: application/vnd.github+json" \ + "repos/$SOURCE_REPO/issues?state=$STATE&per_page=100&sort=created&direction=asc" | + jq -s 'add | map(select(.pull_request == null))')" + +total="$(jq 'length' <<<"$issues_json")" +say "found $total issue(s) on GitHub (pull requests excluded)" +say "" + +count=0 +while read -r number; do + [ -n "$number" ] || continue + + if grep -qE "^${number} " "$MAP_FILE"; then + say "#$number already migrated (in $MAP_FILE) - skipped" + continue + fi + + if [ "$LIMIT" -gt 0 ] && [ "$count" -ge "$LIMIT" ]; then + say "--limit $LIMIT reached, stopping" + break + fi + count=$((count + 1)) + + issue="$(jq --argjson n "$number" '.[] | select(.number == $n)' <<<"$issues_json")" + title="$(jq -r '.title' <<<"$issue")" + state="$(jq -r '.state' <<<"$issue")" + labels="$(jq -c '[.labels[].name]' <<<"$issue")" + + # The provenance line is the only edit made to a body. It carries the old + # number, which is what makes an un-rewritten "#" elsewhere traceable at + # all, and it names the original author because the API cannot. + body="$(jq -r ' + "_Migriert von GitHub: " + .html_url + " - urspruenglich von @" + .user.login + + " am " + (.created_at | split("T")[0]) + "._\n\n" + + (.body // "")' <<<"$issue")" + + say "#$number $title [$state, labels: $labels]" + + if [ "$DRY_RUN" -eq 1 ]; then + comment_count="$(jq -r '.comments' <<<"$issue")" + suffix="" + [ "$state" = closed ] && suffix=", then close it" + say " would create the issue, $comment_count comment(s)$suffix" + continue + fi + + # Create any label this issue needs and the target does not have yet. + while read -r label; do + [ -n "$label" ] || continue + if ! grep -qxF "$label" <<<"$existing_labels"; then + jq -n --arg n "$label" '{name: $n, color: "#ededed"}' | + gitea POST "repos/$TARGET_REPO/labels" >/dev/null + existing_labels="$existing_labels"$'\n'"$label" + say " created label: $label" + fi + done < <(jq -r '.[]' <<<"$labels") + + created="$(jq -n --arg t "$title" --arg b "$body" --argjson l "$labels" \ + '{title: $t, body: $b, labels: $l}' | + gitea POST "repos/$TARGET_REPO/issues")" + new_number="$(jq -r '.number' <<<"$created")" + say " -> $GITEA_URL/$TARGET_REPO/issues/$new_number" + + # Comments in order. Each gets the same attribution header as the body; see + # loss (1) in the header of this file. + while read -r comment; do + [ -n "$comment" ] || continue + comment_body="$(jq -r ' + "_@" + .user.login + " am " + (.created_at | split("T")[0]) + + " (migriert von GitHub)._\n\n" + (.body // "")' <<<"$comment")" + jq -n --arg b "$comment_body" '{body: $b}' | + gitea POST "repos/$TARGET_REPO/issues/$new_number/comments" >/dev/null + done < <(gh api --paginate -H "Accept: application/vnd.github+json" \ + "repos/$SOURCE_REPO/issues/$number/comments?per_page=100" | + jq -s -c 'add // [] | .[]') + + if [ "$state" = closed ]; then + jq -n '{state: "closed"}' | + gitea PATCH "repos/$TARGET_REPO/issues/$new_number" >/dev/null + say " closed" + fi + + printf '%s\t%s\n' "$number" "$new_number" >>"$MAP_FILE" +done < <(jq -r '.[].number' <<<"$issues_json") + +say "" +if [ "$DRY_RUN" -eq 1 ]; then + say "dry run finished - $count issue(s) would have been created" +else + say "finished - $count issue(s) created, mapping appended to $MAP_FILE" + say "the mapping is the input for the # rewrite, which this script does not do" +fi diff --git a/renovate.json5 b/renovate.json5 index 51ed472..e8d8754 100644 --- a/renovate.json5 +++ b/renovate.json5 @@ -49,7 +49,9 @@ // allowedCommands: ["^ci/renovate-bump-chart\\.sh"] // // Without that entry Renovate skips the task silently and every chart PR - // stays red on the version-bump check. + // stays red on the version-bump check. On Gitea, where this repo runs + // Renovate itself, "the instance config" is .github/workflows/renovate.yml - + // it sets RENOVATE_ALLOWED_COMMANDS to exactly that value. postUpgradeTasks: { commands: ["ci/renovate-bump-chart.sh {{{packageFileDir}}}"], fileFilters: ["**/Chart.yaml"], @@ -80,10 +82,13 @@ matchUpdateTypes: ["patch", "minor", "digest"], automerge: true, automergeType: "pr", - // Renovate waits for the checks itself instead of handing the PR to - // GitHub's auto-merge. GitHub would merge immediately when the branch has - // no required status checks configured — and validate-charts is not (yet) - // a required check. + // Renovate waits for the checks itself instead of handing the PR to the + // forge's own auto-merge. Both of them — GitHub's, and the auto-merge + // that takes the merge queue's place on Gitea, which has none — merge + // immediately when the branch has no required status checks configured, + // and validate-charts is not (yet) a required check on either side. This + // is a statement about the branch protection, not about the platform: + // once required checks exist on `main`, true is the better value. platformAutomerge: false, // A release that is hours old should not walk into main unattended; a // chart publish follows the merge straight away.