jklein
0c403b6100
The computed limits (limitFactor 3 x request = 1536Mi memory, 30Mi ephemeral storage) were too small for real documents: a 12 MB HTML already gets the container OOM-killed, and the caller sees no HTTP error at all, just a dropped connection and a rising RESTARTS counter. Rather than raise the requests and reserve conversion-sized capacity on the node around the clock, requests and limits are decoupled: the requests stay as they were, and both limits are now set explicitly. The chart already supported this - an explicit limit wins over the limitFactor computation. Measured in a throwaway k3d cluster with the official image (HTML via Chromium), building on the series in the issue: 18 MB HTML, 2Gi -> OOMKilled 18 MB HTML, 2560Mi -> OOMKilled 18 MB HTML, 3Gi -> HTTP 200 in ~22s, 21 MB PDF 12 MB HTML, 1536Mi -> OOMKilled (the old computed limit) 12 MB HTML, 2Gi -> HTTP 200 in ~13s Memory beyond 3Gi buys nothing: gotenberg's own request timeout is 30s and the chart passes no flags to change it, so a document needing more memory has already run out of time (a 25 MB HTML misses the 30s even at 4Gi). The ephemeral-storage limit is raised in the same change, because raising memory alone only moves the failure. With 3Gi memory and the old 30Mi limit, /tmp peaked at 40 MB on a single conversion and at ~125 MB over a short series, and the pod was evicted: Pod ephemeral local storage usage exceeds the total limit of containers 30Mi That failure is worse than the OOM-kill: the pod does not restart, it is replaced, and the old one is left behind as Evicted. The defaults in the effectiveResources helper are updated in lockstep with values.yaml, so an erased "resources:" block (#99) falls back to the new explicit limits instead of the old computed ones. NOTES.txt is pulled along. Its numbers came from effectiveResources already, but the text described a threshold that has moved and advice that no longer holds: raising the request no longer drags the limit with it now that the limits are explicit. The warning is kept but no longer claims memory is the first limit you hit - with these limits, the 30s timeout is. Major, because the runtime behaviour changes and the pod may occupy more memory during large conversions than it was allowed to before. Fixes #106
83 lines
4.4 KiB
Plaintext
83 lines
4.4 KiB
Plaintext
{{- $replicas := include "gotenberg.replicas" (dict "scaleDown" .Values.scaleDown "replicas" .Values.gotenberg.replicas) -}}
|
|
{{- /* effectiveResources, not resources: an erased "resources:" block falls back
|
|
to the chart defaults in the Deployment (#99), so the notes must report the
|
|
limits the release actually gets, not "(none set)". */ -}}
|
|
{{- $limits := (include "gotenberg.effectiveResources" .Values.gotenberg.resources | fromYaml).limits | default dict -}}
|
|
{{- $svc := printf "%s-%s-service" .Release.Name .Chart.Name -}}
|
|
Gotenberg {{ .Chart.AppVersion }} — release {{ .Release.Name }} in namespace {{ .Release.Namespace }}.
|
|
|
|
ACCESS
|
|
This chart creates no Ingress. The API is cluster-internal:
|
|
|
|
http://{{ $svc }}.{{ .Release.Namespace }}.svc.cluster.local
|
|
|
|
Mind the port. The Service listens on 80 and forwards to the container's
|
|
3000, so consumers are configured without a port — e.g. paperless-ngx:
|
|
|
|
PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://{{ $svc }}.{{ .Release.Namespace }}.svc.cluster.local
|
|
|
|
To reach it from your machine:
|
|
|
|
kubectl -n {{ .Release.Namespace }} port-forward svc/{{ $svc }} 3000:80
|
|
curl http://localhost:3000/health
|
|
|
|
The health route reports Chromium and LibreOffice separately — it is the
|
|
quickest way to tell a broken converter from a broken caller. This chart has
|
|
no secrets and no volumes.
|
|
{{ if eq $replicas "0" }}
|
|
SCALED TO ZERO
|
|
{{ if .Values.scaleDown }}scaleDown: true{{ else }}gotenberg.replicas: 0{{ end }} — the deployment runs with replicas: 0. The Service
|
|
keeps its ClusterIP but has no endpoints, so callers fail with "connection
|
|
refused" rather than a timeout. Set it back and upgrade to return.
|
|
{{ end }}
|
|
NO AUTHENTICATION, AND THIS CHART CANNOT ADD ANY
|
|
Gotenberg's basic auth and OIDC, its SSRF guards for the downloadFrom
|
|
feature, and its request timeout are all command-line flags. This chart
|
|
passes no flags — it only offers gotenberg.env — so none of them can be
|
|
switched on through values. Everything that reaches the Service may convert,
|
|
and the URL routes make the server fetch whatever URL they are handed. Keep
|
|
it cluster-internal; do not put an Ingress in front of it without auth.
|
|
|
|
BIG CONVERSIONS RUN OUT OF TIME BEFORE THEY RUN OUT OF MEMORY
|
|
Chromium and LibreOffice hold the whole document in memory, so a conversion
|
|
is a short, memory-hungry burst. This release limits the container to
|
|
|
|
memory: {{ index $limits "memory" | default "(none set)" }}
|
|
ephemeral-storage: {{ index $limits "ephemeral-storage" | default "(none set)" }}
|
|
|
|
against deliberately smaller requests: the pod reserves little while it
|
|
idles and grows into the limits only while converting.
|
|
|
|
Those limits are set so the 30-second request timeout below is what a
|
|
too-large document hits first. An 18 MB HTML converts in about 22 seconds
|
|
and stays well inside them; a document heavy enough to exhaust the memory
|
|
limit would have run past the 30 seconds anyway, so raising memory further
|
|
does not turn a failing conversion into a successful one — splitting the
|
|
document does.
|
|
|
|
If you do need to change them, set gotenberg.resources.limits.memory and
|
|
.limits.ephemeralStorage. Both are set explicitly now, so raising the
|
|
matching request no longer drags the limit along at limitFactor x request.
|
|
|
|
Should a conversion still hit a limit, the two failures look different and
|
|
neither of them reports an HTTP error to the caller:
|
|
|
|
- Out of memory: the connection is dropped mid-request and the pod's
|
|
RESTARTS counter goes up. Confirm with
|
|
|
|
kubectl -n {{ .Release.Namespace }} get pod -l app={{ .Release.Name }}-{{ .Chart.Name }}-deployment -o jsonpath='{.items[*].status.containerStatuses[*].lastState.terminated.reason}'
|
|
|
|
which then prints OOMKilled.
|
|
|
|
- Out of ephemeral storage: /tmp and /home/gotenberg are emptyDirs and count
|
|
against the ephemeral-storage limit — Chromium's profile alone takes about
|
|
4 MB, and the uploaded file plus the rendered PDF come on top. The pod does
|
|
not restart, it disappears: a fresh one takes its place and the old one is
|
|
left in status Evicted, with the message naming ephemeral-storage.
|
|
|
|
REQUESTS TIME OUT AFTER 30 SECONDS
|
|
That is Gotenberg's own default, and since the chart passes no flags it
|
|
cannot be changed here. This is the ceiling large conversions actually meet:
|
|
an 18 MB HTML needs about 22 seconds, and a 25 MB one does not finish inside
|
|
the 30. Callers that need longer must split the document instead.
|