Modelplane v0.7 is out, and it brings CoreWeave into the fleet. A platform team can now provision CoreWeave Kubernetes Service (CKS) clusters right alongside EKS, AKS, GKE, and the other neoclouds Modelplane supports. AMD GPU pools also share nodes between small models now, as NVIDIA pools have since v0.6.
With KubeCon in Salt Lake City this week, we spent the rest of the release running real fleets and smoothing out what we found. Rollouts finish on a full GPU pool, a model's endpoint picker stays up through a node drain, and GPU metrics tell you which deployment is using each GPU.
CoreWeave
Set an InferenceCluster's source to CoreWeave and Modelplane provisions a
CKS cluster with its VPC, a system node pool, and the GPU node pools you list,
then installs the serving stack. Here's one node of eight NVIDIA RTX PRO 6000
Blackwell GPUs, from the getting-started guide:
apiVersion: modelplane.ai/v1alpha1
kind: InferenceClass
metadata:
name: rtxp6000-8x
spec:
description: "CoreWeave rtxp6000-8x, 8x NVIDIA RTX PRO 6000 Blackwell 96GB"
provisioning:
provider: CoreWeave
coreweave:
# CKS instance types vary by zone.
instanceType: rtxp6000-8x
accelerator:
type: nvidia-rtx-pro-6000
count: 8
devices:
- name: gpu
claim: DRA
driver: gpu.nvidia.com
deviceClassName: gpu.nvidia.com
count: 8
attributes:
architecture: { string: Blackwell }
cudaComputeCapability: { version: "12.0.0" }
capacity:
memory: { value: "97887Mi" } # RTX PRO 6000's real reported VRAM
---
apiVersion: modelplane.ai/v1alpha1
kind: InferenceCluster
metadata:
name: coreweave-us-east-04
labels:
modelplane.ai/region: us-east-04
spec:
cluster:
source: CoreWeave
coreweave:
zone: US-EAST-04A
nodePools:
- name: gpu-rtxp6000
className: rtxp6000-8x
nodeCount: 1
minNodeCount: 1
maxNodeCount: 1The ML team doesn't change a thing. A ModelDeployment that runs on EKS or
Nebius runs on CoreWeave too, on any pool whose devices match its selectors.
Thanks to CoreWeave for the GPU capacity we built and tested this on. Bringing up a new cloud always shakes out a few bugs, and working through them with CoreWeave's engineers has been a lot of fun. Their networking team is tracking one where deleting a VPC right after its cluster can leave the VPC behind (provider-upjet-coreweave#4). They also spotted our provider updating some VPCs in a tight loop and asked us about it (provider-upjet-coreweave#6). We've sent them a fix in terraform-provider-coreweave#554.
Rollouts that finish on a full pool
GPUs are expensive, so pools tend to run full. That used to make rolling out a new engine version awkward. An engine's Deployment started a new pod before stopping an old one, and on a pool with no free GPU the new pod waited for a GPU the old pod would never release. Modelplane now replaces one copy at a time, on the GPU the old copy frees. LeaderWorkerSet and Grove gangs already worked this way.
A deployment also reports ready only once every copy runs the current spec,
rather than partway through a rollout. Switching an engine from Standalone to
a Leader and Worker gang cleans up the old Deployment too, so it doesn't
hold on to the GPUs its replacement needs.
Serving through node drains
Draining a node used to take a model offline for minutes. Each replica's endpoint picker, which chooses the engine copy that serves each request, ran as one pod, and the cluster's gateway won't route without it. On Nebius a replacement picker took about four minutes to start. Modelplane now runs two pickers per replica, spread across nodes, with a disruption budget that keeps a drain from evicting both.
New gateways are safer to route to, too. The first ModelRoute on a new
InferenceGateway could report ready while Envoy AI Gateway was still rolling
the gateway, so requests to it returned HTTP 500s. A gateway now waits for that
rollout before it reports ready. It also keeps a placeholder route attached, so
the first real route doesn't trigger another rollout.
Which deployment is using that GPU?
GPU metrics have the same deployment, replica, role, and engine labels
as engine metrics. A dashboard can break GPU utilisation down by deployment and
show it next to that deployment's queue depth and latency.
Inference clusters are lighter too. Modelplane's telemetry collector scrapes engines and GPUs itself, which means clusters don't need a Prometheus stack of their own anymore.
Try it on your laptop
The new local demo runs Modelplane on two kind clusters on your laptop, with fake GPUs and models small enough for a CPU. You play both teams, building a small fleet and then upgrading a model behind the same name.
When you're ready for real GPUs, the getting-started guide covers Vultr and Civo end to end, and a new recipe serves Moonshot AI's Kimi K3.1 Pro with SGLang on one 8x B300 node.
Come find us at KubeCon
The full release notes are on GitHub. Modelplane is Apache 2.0 at github.com/modelplaneai/modelplane, and now has a charter setting out the project's scope, plus a governance model adapted from Crossplane's. Questions are welcome in Slack. If you're at KubeCon this week, come find us at the Upbound booth.
