jklein
320257c742
Two corrections found while verifying the first CI run, not from the plan. THE DEFAULT THAT MADE THE READINESS PROBE TOOTHLESS. Keycloak's DATABASE health check is the one row of upstream's "Available Checks" table whose "Requires Metrics" column says yes; Cluster, Graceful shutdown and Keycloak Initialized say no. So with metrics-enabled=false - this chart's default until now - /health/ready answers 200 while the database is gone, and the pod keeps taking logins it cannot serve. The probe path was right and the aggregate behind it contained nothing external, which is the urlaubsverwaltung defect (#218) in a new hiding place: not a health group this time, but an option named for observability, where nobody looks when reviewing a probe. keycloak.metrics.enabled therefore defaults to TRUE, and its comment argues about readiness rather than about scraping. The price is stated at the value: /metrics then answers on the management port, unauthenticated, carrying realm names, client ids and login counters - on a port that is ClusterIP-only and in no Ingress. Switching it off is now documented as a decision about whether readiness can fail at all. The CI fixture keeps it on and says why, so a green run really does touch the database. THE STARTUP BUDGET NOW CARRIES ITS MEASUREMENT. From the install-test of PR #87 on the Gitea docker runner (k3s v1.35.5, two replicas, empty database, image pulled during the run): 71.8 seconds from `helm install` to BOTH pods Ready - first at ~70.8s, second one second later, 0 restarts. The 600s budget stays, with the reason for the headroom written down: everything in that run was the cheap case, and a first install against a real database serializes the second replica behind the first's Liquibase migration lock instead of letting it arrive a second later. AND AN OVERCLAIM OF MY OWN, REMOVED. ci/keycloak/test-values.yaml said a green run proves jdbc-ping found the second replica. It does not: readiness reports one node's own health, so a node that never found its peer is Ready, healthy and alone - and two such nodes are two islands that each believe they are the cluster. What the test does rule out is the louder half, a replica that cannot start or cannot share the database. The Infinispan view has to be read out of the log, which is what step 2 of NOTES.txt tells the operator to do, now with the note that no probe and no install-test replaces that step. Measurement point 4 of the plan therefore stays open. Part of #83
144 lines
7.2 KiB
YAML
144 lines
7.2 KiB
YAML
# CI values for the keycloak install-test.
|
|
#
|
|
# WHY THIS CHART IS TESTED RATHER THAN EXCLUDED, and what a green run is worth.
|
|
# Keycloak has no volume, needs one throwaway PostgreSQL and reaches Ready on
|
|
# its own, so an exclusion would be hard to justify. More importantly, this is
|
|
# the only place where the four riskiest assumptions of the chart are exercised
|
|
# at all - none of them is observable in a render.
|
|
#
|
|
# WHAT A GREEN RUN PROVES, point by point (the measurement points of Gitea #83):
|
|
#
|
|
# 1. THE TWO INIT CONTAINERS WORK AT ALL. The seed container copies the
|
|
# image's own augmentation output into the volume, the build container
|
|
# mounts that volume at /opt/keycloak/lib/quarkus and runs `kc.sh build`
|
|
# into it - both with readOnlyRootFilesystem: true. If the seeding
|
|
# assumption were wrong (QuarkusEntryPoint loads
|
|
# quarkus/quarkus-application.dat before it dispatches any command,
|
|
# `build` included), the build container could not start, and the pod
|
|
# would sit in Init with the reason only in its log.
|
|
# 2. readOnlyRootFilesystem: true HOLDS FOR THE SERVING CONTAINER, with
|
|
# emptyDirs only at /opt/keycloak/lib/quarkus, /opt/keycloak/data and
|
|
# /tmp - and NOT at /opt/keycloak/conf, which would hide the
|
|
# cache-ispn.xml that cache=ispn reads.
|
|
# 3. `start --optimized` ACCEPTS THE PRE-BUILT INSTALLATION and does not
|
|
# re-augment. A disagreement between the build options and the serving
|
|
# container would show up here as a crash loop rather than as a slow
|
|
# start, because the path it would want to rewrite is read-only.
|
|
# 4. TWO REPLICAS BOTH COME UP AND STAY UP against one database - the
|
|
# rollout is complete only when BOTH pods are Ready, so a second replica
|
|
# that crashed, deadlocked on the schema migration lock or fought the
|
|
# first over the database would fail the run. Measured in Gitea PR #87:
|
|
# 71.8s from `helm install` to both Ready, 0 restarts.
|
|
# Plus the mechanical half: the Secret key names match what the Deployment
|
|
# references, /health/started, /health/live and /health/ready all answer on
|
|
# the management port (so health-enabled really was baked in by the init
|
|
# container), the resource quantities are valid, and the schema accepts a
|
|
# realistic non-default values file.
|
|
#
|
|
# WHAT IT DOES NOT PROVE, and the first entry is the one not to misread:
|
|
# - THAT jdbc-ping ACTUALLY FOUND THE PEER. Two replicas both reaching Ready
|
|
# is NOT that proof, and believing it was is the mistake this paragraph
|
|
# exists to prevent: readiness reports this node's own health, so a node
|
|
# that never found its peer is Ready, healthy and alone - and two such
|
|
# nodes are two islands that each believe they are the cluster. The
|
|
# Infinispan view has to be read out of the log, which is what step 2 of
|
|
# NOTES.txt tells the operator to do. So measurement point 4 of the plan
|
|
# stays open after this test; what the test does rule out is the louder
|
|
# half (a second replica that cannot start or cannot share the database).
|
|
# - that hostname-admin serves the console cleanly on a second hostname. No
|
|
# ingress controller answers in CI and nothing drives a browser, so
|
|
# measurement point 5 stays open too. What IS checked is that both Ingress
|
|
# objects are accepted, that the Service port they name exists, and that
|
|
# Keycloak accepts both hostname values as full URLs at start.
|
|
# - anything about an upgrade. The rolling-update restriction between minor
|
|
# versions is a property of two versions meeting, and this test installs
|
|
# one.
|
|
# - that the PostgreSQL here resembles production. It is one pod with an
|
|
# emptyDir, so nothing about durability, connection limits or latency
|
|
# transfers.
|
|
|
|
secrets:
|
|
# Materialized by ci/keycloak/fixtures.yaml as ci-keycloak-db-secret and
|
|
# ci-keycloak-bootstrap-admin-secret (the OnePasswordItem stub CRD comes from
|
|
# ci/common/).
|
|
databaseSecretRef: op://ci/keycloak-db
|
|
bootstrapAdminSecretRef: op://ci/keycloak-bootstrap-admin
|
|
|
|
keycloak:
|
|
# TWO, i.e. the chart's default, and the whole reason this fixture is not a
|
|
# one-replica shortcut. With one replica the install-test would be green
|
|
# without ever showing whether the second node is found - and "a path the
|
|
# test does not walk is a path we have not proven" is the lesson Gitea #80
|
|
# paid for. The install-test script waits for the Deployment's rollout, and a
|
|
# rollout is only complete when BOTH pods are Ready, so cluster formation is
|
|
# inside the assertion rather than next to it.
|
|
replicas: 2
|
|
|
|
db:
|
|
# The throwaway PostgreSQL from fixtures.yaml. Plain Service name: both are
|
|
# in the same namespace.
|
|
host: postgres
|
|
database: keycloak
|
|
# Small but not degenerate: 2 replicas x 5 = 10 connections against
|
|
# postgres:16-alpine's default max_connections of 100, which leaves the
|
|
# fixture comfortable while still exercising the pool options.
|
|
pool:
|
|
initialSize: 1
|
|
minSize: 1
|
|
maxSize: 5
|
|
|
|
# The chart default, spelled out because it is load-bearing for what this
|
|
# test means: metrics-enabled is what puts the DATABASE check into
|
|
# /health/ready (upstream's check table, "Requires Metrics: Yes"). With it
|
|
# off, readiness would pass without ever touching the database, and a green
|
|
# run would say nothing about the connection at all.
|
|
#
|
|
# It is also a BUILD option, so with it true the init container has to build
|
|
# an installation differing from the published image in three options rather
|
|
# than two - which is what makes a build that silently did nothing fail the
|
|
# run instead of looking identical.
|
|
metrics:
|
|
enabled: true
|
|
|
|
# Set on purpose, although one release cannot collide with itself: it
|
|
# exercises the KC_CACHE_EMBEDDED_CLUSTER_NAME path, and a name Keycloak
|
|
# rejected would fail the start rather than pass unnoticed.
|
|
cache:
|
|
clusterName: ci-keycloak
|
|
|
|
resources:
|
|
# Below the chart's defaults to stay inside the runner's budget, and the
|
|
# memory number is chosen rather than rounded: the image sizes the heap at
|
|
# 70% of the LIMIT, so 600Mi x limitFactor 3 = 1800Mi gives a ~1260Mi heap.
|
|
# Upstream's floor is "at least 750 MB" for the limit; going much below
|
|
# this makes the JVM the reason a run fails.
|
|
requests:
|
|
cpu: 250m
|
|
memory: 600Mi
|
|
ephemeral-storage: 256Mi
|
|
|
|
build:
|
|
resources:
|
|
requests:
|
|
cpu: 250m
|
|
memory: 512Mi
|
|
ephemeral-storage: 256Mi
|
|
|
|
ingress:
|
|
# Left ON although no ingress controller answers in CI: the point is that
|
|
# both objects are accepted and that the Service port they name BY NAME
|
|
# ("http") actually exists - a typo there is otherwise invisible until
|
|
# somebody opens the URL.
|
|
enabled: true
|
|
domain: keycloak.ci.example.com
|
|
clusterIssuer: ci-selfsigned
|
|
admin:
|
|
# The second hostname, on as well, so the chart renders KC_HOSTNAME_ADMIN
|
|
# and Keycloak has to accept it as a full URL. That is the half of
|
|
# measurement point 5 a cluster-less test can reach: Keycloak validates
|
|
# both hostname values at start ("verified that full URL is used, incl.
|
|
# scheme and hostname"), so a chart that built a bare host here would fail
|
|
# the start instead of serving a broken console later.
|
|
enabled: true
|
|
domain: keycloak-admin.ci.example.com
|