Skip to main content

Production Installation

Cosmonic Control separates operator and developer concerns: Wasm component developers use their own tooling to build and publish Wasm components, while operators use standard Kubernetes-native pipelines and tooling to deploy and manage them.

This page covers installing and configuring Cosmonic Control on a production Kubernetes cluster. See also:

Prerequisites​

Installing Cosmonic Control​

Cosmonic Control is distributed as an OCI Helm chart at oci://ghcr.io/cosmonic/cosmonic-control. The chart deploys the following components into the cosmonic-system namespace:

ComponentRole
operatorRuntime operator — reconciles CRDs and manages wasmCloud workloads
nexusNATS message bus — internal communication backbone
envoyHTTP ingress proxy — routes external traffic to Wasm workloads
opentelemetry-collectorReceives OTLP telemetry from all components
prometheusMetrics store
lokiLog store
tempoTrace store
persesObservability dashboard UI

Cloud clusters (EKS, GKE, AKS)​

For cloud clusters, set envoy.service.type=LoadBalancer. Most providers will provision a load balancer automatically. Use envoy.service.annotations to control cloud-specific load balancer behavior.

AWS (Network Load Balancer):

helm install cosmonic-control oci://ghcr.io/cosmonic/cosmonic-control \
  --version 0.6.1 \
  --namespace cosmonic-system \
  --create-namespace \
  --set envoy.service.type=LoadBalancer \
  --set-json 'envoy.service.annotations={"service.beta.kubernetes.io/aws-load-balancer-type":"nlb","service.beta.kubernetes.io/aws-load-balancer-scheme":"internet-facing"}'

GKE / AKS (standard cloud load balancer):

helm install cosmonic-control oci://ghcr.io/cosmonic/cosmonic-control \
  --version 0.6.1 \
  --namespace cosmonic-system \
  --create-namespace \
  --set envoy.service.type=LoadBalancer

On-premises and bare-metal​

For clusters without a cloud load balancer controller, expose Envoy as a NodePort and route external traffic to that port yourself:

helm install cosmonic-control oci://ghcr.io/cosmonic/cosmonic-control \
  --version 0.6.1 \
  --namespace cosmonic-system \
  --create-namespace \
  --set envoy.service.type=NodePort \
  --set envoy.service.httpNodePort=30950

Traffic must reach port 30950 on any node in the cluster. Configure your external load balancer, firewall rules, or ingress proxy accordingly.

Using a values file​

For anything beyond a simple install, use a values.yaml file to manage configuration:

# cosmonic-control-values.yaml
envoy:
  service:
    type: LoadBalancer
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-type: nlb
      service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
helm install cosmonic-control oci://ghcr.io/cosmonic/cosmonic-control \
  --version 0.6.1 \
  --namespace cosmonic-system \
  --create-namespace \
  -f cosmonic-control-values.yaml

Wait for readiness​

kubectl rollout status deploy -l app.kubernetes.io/instance=cosmonic-control -n cosmonic-system

Installing HostGroups​

A HostGroup is a group of one or more wasmCloud host pods that run Wasm workloads. Every Cosmonic Control installation needs at least one HostGroup. HostGroups connect to the nexus NATS server and register themselves as available hosts for workload scheduling.

Install the default HostGroup:

helm install hostgroup oci://ghcr.io/cosmonic/cosmonic-control-hostgroup \
  --version 0.6.1 \
  --namespace cosmonic-system

Wait for it to be ready:

kubectl rollout status deploy -l app.kubernetes.io/instance=hostgroup -n cosmonic-system

Scaling HostGroups​

HostGroups are standard Kubernetes Deployments and can be scaled horizontally. Set replicaCount to run multiple host replicas:

helm install hostgroup oci://ghcr.io/cosmonic/cosmonic-control-hostgroup \
  --version 0.6.1 \
  --namespace cosmonic-system \
  --set replicaCount=3

Cosmonic Control automatically load-balances workloads across all available hosts in a HostGroup (round-robin). If a host crashes, its workloads are redistributed to remaining hosts.

Multiple HostGroups​

Deploy multiple HostGroups with different names and host labels to create distinct scheduling zones—for example, separating general-purpose workloads from GPU workloads, or isolating workloads by team or environment:

# General-purpose HostGroup
helm install hostgroup-default oci://ghcr.io/cosmonic/cosmonic-control-hostgroup \
  --version 0.6.1 \
  --namespace cosmonic-system \
  --set hostgroup=default \
  --set replicaCount=2

# GPU-enabled HostGroup
helm install hostgroup-gpu oci://ghcr.io/cosmonic/cosmonic-control-hostgroup \
  --version 0.6.1 \
  --namespace cosmonic-system \
  --set hostgroup=gpu \
  --set gpu=true \
  --set runtimeClassName=nvidia \
  --set replicaCount=1

Workload placement is controlled through host labels on WorkloadDeployment manifests. See Multi-tenancy and RBAC for details.

GPU HostGroups​

Setting gpu: true on a HostGroup passes --wasi-webgpu to the control-host container, enabling Wasm workloads on that HostGroup to call the wasi:webgpu WIT interface.

To schedule GPU pods onto NVIDIA GPU nodes, set runtimeClassName: nvidia. Configure the NVIDIA device plugin and a matching Kubernetes RuntimeClass on the cluster beforehand; the Helm value must match the RuntimeClass name.

To make host-side artifacts (model files, precompiled kernels, driver libraries) available inside the HostGroup pod, set volumes and volumeMounts. These accept the standard Kubernetes Volume and VolumeMount specs and are available on any HostGroup, not only GPU ones:

# hostgroup-gpu-values.yaml
hostgroup: gpu
gpu: true
runtimeClassName: nvidia
replicaCount: 1

volumes:
  - name: models
    hostPath:
      path: /opt/models
      type: Directory

volumeMounts:
  - name: models
    mountPath: /var/models
    readOnly: true
helm install hostgroup-gpu oci://ghcr.io/cosmonic/cosmonic-control-hostgroup \
  --version 0.6.1 \
  --namespace cosmonic-system \
  -f hostgroup-gpu-values.yaml

Separating the data-plane NATS​

By default, the wasmCloud host inside a HostGroup pod uses one NATS connection — to the Nexus service — for both control-plane traffic (workload scheduling, host heartbeats) and data-plane traffic (wasmcloud:messaging, wasi:keyvalue, wasi:blobstore). For deployments that want to keep workload-generated NATS traffic off the control-plane bus, the HostGroup chart's dataNats.url value opens a second NATS connection used only for the data-plane plugins:

# hostgroup-values.yaml
dataNats:
  url: "nats://workload-nats.shared.svc.cluster.local:4222"

When dataNats.url is empty (the default), the host reuses the Nexus connection so existing releases behave identically. When the URL points anywhere other than the Nexus URL, the host dials a fresh connection at startup; if it can't, the pod fails to come up.

Typical reasons to split:

  • The data-plane NATS lives outside the cluster (a managed NATS account, or a separate cluster) and you don't want workload pods talking to it via the control-plane Nexus.
  • You want JetStream KV usage to scale independently of operator/host scheduling traffic.
  • You want to size, persist, or replicate the data-plane bus differently from the control-plane bus — for example, a clustered Nexus for HA scheduling and a separate single-node NATS for ephemeral workload messaging.

wasi:keyvalue buckets created by workloads land on whichever NATS connection serves the data plane, so switching dataNats.url after workloads are running moves those buckets to the new server and the old buckets are no longer visible. Drain workloads using wasi:keyvalue before changing the URL.

Passing extra args and env to the host​

Three additive HostGroup values let you extend the host container without forking the chart:

  • env — appended after the chart's own environment variables, so entries can reference earlier vars via $(VAR) and override them on duplicate names. Accepts the standard Kubernetes env shape including valueFrom for ConfigMap / Secret references.
  • envFrom — populates the host container from one or more ConfigMaps / Secrets.
  • extraArgs — appended verbatim to the host CLI, for host flags the chart does not template (custom host builds, experimental flags).
# hostgroup-values.yaml
env:
  - name: RUST_LOG
    value: debug
envFrom:
  - secretRef:
      name: host-extra-secrets
extraArgs:
  - "--allow-insecure-registries"

Resource sizing​

Control plane​

The control plane components are each deployed as single replicas. The v0.4.x chart exposes a resources block on every component. Defaults are empty (no requests or limits); set them per component in a values file as needed:

# cosmonic-control-values.yaml
operator:
  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      cpu: 500m
      memory: 512Mi

nexus:
  resources:
    requests:
      cpu: 100m
      memory: 256Mi
    limits:
      cpu: 500m
      memory: 512Mi

envoy:
  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      cpu: 500m
      memory: 256Mi

The resources block is accepted under operator, nexus, envoy, opentelemetryCollector, prometheus, loki, tempo, and perses. Each follows the standard Kubernetes ResourceRequirements shape.

note

Most control-plane components currently run as single replicas. Nexus can run as a clustered StatefulSet for HA — see Nexus persistence and HA. For other components that need HA in production, contact support@cosmonic.com.

HostGroups​

HostGroup pods run Wasm workloads and typically benefit from tuning. Set resources in the HostGroup values file:

# hostgroup-values.yaml
replicaCount: 3

resources:
  requests:
    cpu: 500m
    memory: 512Mi
  limits:
    cpu: 2
    memory: 2Gi

As a starting point: a single HostGroup pod can comfortably run dozens of concurrent Wasm components given their small footprint (typically sub-millisecond startup, kilobyte-scale memory per component). Scale out by increasing replicaCount rather than individual pod resource limits.

Nexus persistence and HA​

Nexus runs an embedded NATS server, and JetStream state — every wasi:keyvalue bucket created by workloads, plus the ARTIFACTS object store backing the Artifact CRD — lives in /data inside the Nexus pod. By default, /data is an emptyDir and that state is destroyed on pod restart.

Durable single-node Nexus​

For workloads that need durable wasi:keyvalue state but don't require failover, enable persistence with the default replica count:

# cosmonic-control-values.yaml
nexus:
  persistence:
    enabled: true
    size: 10Gi
    storageClassName: ""   # empty = cluster default StorageClass

This switches Nexus from a single-replica Deployment (the default emptyDir layout) to a StatefulSet with a volumeClaimTemplate. The PVC is named data-nexus-0 and survives helm uninstall so a reinstall can reattach to existing JetStream state.

Clustered Nexus for HA​

For high availability, scale nexus.replicas to an odd number so JetStream's RAFT meta-group has a quorum:

# cosmonic-control-values.yaml
nexus:
  replicas: 3
  persistence:
    enabled: true
    size: 10Gi

The chart fails the install at template time if replicas is even, or if replicas > 1 is paired with persistence disabled, so misconfiguration cannot reach the cluster. Each replica gets its own PVC (data-nexus-0, data-nexus-1, data-nexus-2); a headless nexus-headless Service provides per-pod DNS and cluster routes are auto-generated.

Two follow-on details when running clustered:

  • The operator passes -artifact-replicas={{ nexus.replicas }}, so the ARTIFACTS object store is replicated to match. The operator reconciles the bucket's replica count on every start, so scaling Nexus from 1 to 3 brings the Artifact bucket along.
  • Server clustering does not replicate R=1 data on its own. JetStream streams and KV buckets need their replica count set at or above the desired survival threshold to outlast a node loss — the ARTIFACTS bucket gets this automatically, but any KV bucket the host opens on behalf of a workload keeps whatever replica count was set at its creation.

Upgrading an existing persistent install​

The v0.6.0 chart replaces the previous Deployment + nexus-data PVC layout with a StatefulSet + data-nexus-0 PVC, and nexus.persistence.existingClaim is removed. JetStream state on the old PVC does not roll forward on its own. To preserve it, copy /data from the old nexus-data PVC into data-nexus-0 (typically via a one-shot Job that mounts both) before scaling the StatefulSet up. See the v0.6.0 upgrade notes for the full migration story.

Multi-tenancy and RBAC​

Host labels for workload placement​

HostGroups expose a hostLabels map that is propagated to every wasmCloud host in the group. These labels are used in WorkloadDeployment manifests to control which HostGroup runs a given workload:

# hostgroup-team-a-values.yaml
hostgroup: team-a
hostLabels:
  team: a
  environment: production
replicaCount: 2
helm install hostgroup-team-a oci://ghcr.io/cosmonic/cosmonic-control-hostgroup \
  --version 0.6.1 \
  --namespace cosmonic-system \
  -f hostgroup-team-a-values.yaml

Reference the labels in a WorkloadDeployment using the hostSelector field to pin workloads to specific HostGroups.

Kubernetes RBAC​

Cosmonic Control creates one ClusterRole for the operator service account, plus two namespace-scoped Role objects in cosmonic-system. The ClusterRole grants:

  • control.cosmonic.io: ProjectEnvironment, HTTPTrigger (plus their status subresources)
  • runtime.wasmcloud.dev: Artifact, Host, Workload, WorkloadReplicaSet, WorkloadDeployment (plus status and finalizers subresources)
  • "" (core): ConfigMap, Secret, Namespace (read-only); Service (read-only); Pod (get, list, patch, watch)

The namespace-scoped Roles cover leader election (coordination.k8s.io/leases in cosmonic-system) and host finalizer reconciliation (a second Pod grant scoped to cosmonic-system). The Pod permissions let the operator patch the finalizer on a wasmCloud host pod when it terminates, so workloads reschedule immediately instead of waiting for the reconciliation loop.

Tenant isolation is enforced by Kubernetes namespace-scoped RBAC. Grant teams access to their own namespaces with standard Role/RoleBinding objects scoped to the control.cosmonic.io and runtime.wasmcloud.dev API groups. See Tenant RBAC for the full RBAC reference and worked examples.

Further reading​

Last updated on by github-actions[bot]