Deploy with Helm

This guide deploys grex on Kubernetes with the official Helm chart. Use it when you already have a cluster and want a production-shaped install (named ports, ConfigMap config, optional TLS, Ingress, and Prometheus Operator scrapes). For a laptop lab with collectors included, prefer the Compose stack instead.

Prerequisites

• Kubernetes 1.25+
• Helm 3.14+
• A grex container image your cluster can pull

Release images on GHCR land with the GoReleaser milestone. Until then, build and push (or load into a kind/minikube node) yourself:

docker build -t ghcr.io/dennisme/grex:0.1.0 .
# push or kind load docker-image ghcr.io/dennisme/grex:0.1.0

Add the chart repository

The chart is published on the same GitHub Pages site as the documentation and static demo, under a dedicated /charts/ path so the three do not collide:

Path	Content
/	MkDocs documentation
/demo/	Static fleet UI demo
/charts/	Helm repository index (index.yaml; each .tgz downloads from its own GitHub Release asset)

helm repo add grex https://dennisme.github.io/grex/charts/
helm repo update
helm search repo grex

From a git checkout you can install the chart directory directly:

helm install grex ./deploy/charts/grex --namespace grex --create-namespace

Minimal install

helm install grex grex/grex \
  --namespace grex \
  --create-namespace \
  --set image.repository=ghcr.io/dennisme/grex \
  --set image.tag=0.1.0

Verify:

kubectl -n grex get pods,svc
kubectl -n grex port-forward svc/grex 8080:8080 9090:9090
curl -sS http://127.0.0.1:9090/healthz   # ok
open http://127.0.0.1:8080/              # fleet UI

Collectors should dial the OpAMP Service (default port 4320):

ws://grex.grex.svc.cluster.local:4320/v1/opamp

Production-shaped install

Create a values file (for example grex-values.yaml):

image:
  repository: ghcr.io/dennisme/grex
  tag: "0.1.0"

replicaCount: 1   # fleet state is in-memory; do not scale out yet

config:
  log:
    level: info
    format: json
  fleet:
    heartbeat_interval: 30s
    stale_missed_heartbeats: 3
    required_attributes:
      - deployment.environment
      - service.namespace

tls:
  enabled: true
  existingSecret: grex-opamp-tls
  # Secret keys default to tls.crt, tls.key, ca.crt

ingress:
  enabled: true
  className: nginx
  hosts:
    - host: grex.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: grex-ui-tls
      hosts:
        - grex.example.com

serviceMonitor:
  enabled: true
  # labels:
  #   release: kube-prometheus-stack

Create the OpAMP TLS Secret first (server cert, key, and client CA for mTLS):

kubectl -n grex create secret generic grex-opamp-tls \
  --from-file=tls.crt=./server.pem \
  --from-file=tls.key=./server-key.pem \
  --from-file=ca.crt=./ca.pem

Install or upgrade:

helm upgrade --install grex grex/grex \
  --namespace grex \
  --create-namespace \
  -f grex-values.yaml

With TLS enabled, collectors use:

wss://grex.grex.svc.cluster.local:4320/v1/opamp

and must present a client certificate signed by the CA in ca.crt.

What gets deployed

Resource	Role
Deployment grex	Single grex process, three container ports
Service grex	Named ports opamp, ui, telemetry
ConfigMap	grex YAML at /etc/grex/config.yaml
ServiceAccount	Dedicated SA (token automount optional)
Ingress (optional)	Routes to the UI port only
ServiceMonitor ×2 (optional)	/metrics and /metrics/fleet
OpAMP gateway (optional)	Multiplexing collector; off by default

Probes use the telemetry listener: liveness → /healthz, readiness → /readyz. Termination grace defaults to 30s, which covers grex’s drain delay (5s) plus shutdown grace (10s). See Health and lifecycle.

Configuration knobs

Concern	Values keys	Notes
Image	image.*	Tag defaults to chart appVersion
grex YAML	config.*, listeners.*	Rendered into the ConfigMap
Env overrides	extraEnv	GREX_* after file load
OpAMP TLS	tls.*	Requires tls.existingSecret when enabled
UI exposure	ingress.* or service.type	Do not put OpAMP on HTTP Ingress casually
Scrapes	serviceMonitor.* or external jobs	Two jobs, same telemetry port
Gateway	opampGateway.enabled	Needs a gateway-capable image

Full field list: Helm chart reference. Application settings: Configuration.

Single replica (important)

grex does not persist fleet state. Each process holds its own registry. Running replicaCount > 1 without sticky sessions and a shared store splits the fleet view across pods. Keep one replica until HA is designed. The chart’s HPA is off by default and capped at one for the same reason.

OpAMP gateway (optional)

For large fleets, enable the optional gateway Deployment so agents dial the gateway and the gateway multiplexes upstream to grex:

opampGateway:
  enabled: true
  image:
    repository: ghcr.io/dennisme/grex-opamp-gateway
    tag: "0.1.0"
  # Supply a full collector config, or use the chart’s minimal default:
  # config: |
  #   extensions:
  #     opampgateway: ...

The default gateway config is a starting point only. Production configs should match your collector distribution and the patterns in Scaling with gateways and deploy/compose/opamp-gateway.yaml.

Scraping metrics

Prefer Prometheus Operator ServiceMonitors:

serviceMonitor:
  enabled: true

That creates two ServiceMonitors (server /metrics, fleet /metrics/fleet) with independent intervals, matching Scraping.

Without the operator, either set metrics.serviceAnnotations.enabled=true for a single /metrics annotation scrape, or point static scrape configs at the telemetry Service port with two jobs.

Security checklist

• UI/API is open until authentication ships (issue #11). Restrict with network policy, mesh, or Ingress auth.
• Prefer mTLS on OpAMP (tls.enabled + client CA) for any non-lab fleet.
• Keep the telemetry port off the public internet (especially if config.debug.pprof_enabled is true).
• Pod defaults: non-root, read-only root filesystem, dropped capabilities.

Upgrade and uninstall

helm repo update
helm upgrade grex grex/grex -n grex -f grex-values.yaml

helm uninstall grex -n grex
# Secrets you created (TLS) are not removed by the chart.

ConfigMap changes roll the Deployment via a content checksum annotation.

Local cluster smoke test (kind / k3d)

From a git checkout you can exercise a real install without a remote registry. The script builds the Dockerfile image, loads it into a disposable cluster, installs the chart with deploy/charts/ci/values-e2e.yaml, asserts /healthz, /readyz, metrics, /api/status, the UI, and runs helm test.

Prerequisites: Docker, kubectl, Helm, and either kind or k3d (k3s-in-Docker).

just helm-e2e            # auto-pick kind or k3d
just helm-e2e-kind       # force kind
just helm-e2e-k3d        # force k3d (k3s)

# or:
./deploy/charts/smoke.sh --provider kind
./deploy/charts/smoke.sh --provider k3d --keep   # leave cluster for debugging
./deploy/charts/smoke.sh --skip-build            # reuse grex:e2e image

CI runs the same script against kind (.github/workflows/helm.yml job e2e-kind). Compose remains the multi-collector lab; this path validates the Kubernetes package only (no agents).

Troubleshooting

Symptom	Check
Pod CrashLoopBackOff	kubectl -n grex logs deploy/grex — often missing TLS files or bad YAML
Image pull errors	Tag/registry, imagePullSecrets, or load local image into the node
Empty fleet UI	Collectors not reaching OpAMP Service; path must include /v1/opamp
Ready never true	Readiness is /readyz on telemetry; ensure port 9090 is free and probes match
ServiceMonitor ignored	Prometheus Operator CRDs missing, or label selectors do not match your Prometheus
kind/k3d smoke fails	kubectl -n grex-e2e describe pod, logs; ensure Docker can pull busybox for helm test

Validate a values file without installing:

helm template grex grex/grex -f grex-values.yaml | less
helm lint deploy/charts/grex   # from a git checkout
just helm-lint

Next steps

• Helm chart reference — every values key
• TLS and mTLS — certificate layout
• Configuration — grex YAML fields
• Scraping — dual Prometheus jobs
• Compose stack — local multi-collector lab