An Intellyx BrainBlog by Jason English, for Rafay
By now, everybody knows AI technology is hot, and that it runs better on GPUs (graphics processing units) instead of the CPUs (central processing units) that run standard computers. Hence, all of the enthusiasm and rising stock prices for NVIDIA and other AI-Native technology vendors that leverage GPUs.
Therefore, it stands to reason that if you wanted to support AI, you might rent GPU capacity from a hyperscaler or neocloud vendor and hope the opex costs remain sustainable. Or, if you wanted to maintain tighter control, you would go buy a bunch of GPUs and stick them into a rack in a supercooled data center somewhere.
GPUs are similar to CPUs in that they are both types of computer chips that handle complex processing tasks – but that’s where the similarity ends. Adding GPUs to a traditional data center involves more than simply adding new hardware. It requires an entire support ecosystem that aggregates, secures, manages, and configures GPU resources for their intended purpose.
What are the primary buyer considerations for leveraging GPUs within a modern AI application infrastructure?
Self-service cloud ease-of-use
“You must be able to go to a cloud provider’s page, click a button, and get an AI use case in a multi-tenant fashion. I don’t want to talk to anybody. If you can do that, you’re a cloud. If you can’t do that, you’re not a cloud. You’re custom infrastructure.”
– Haseeb Budhani, CEO, Rafay
The major cloud hyperscalers have spent more than 15 years developing thousands of new features and ambiguously-named services, but the most fundamental aspect of cloud computing remains user experience. An engineer should be able to just create an account on a credit card, set up their own VPC, populate it with resources and capacity, and start building and elastically scaling multi-tenant applications, without worrying about infrastructure.
This Infrastructure-as-a-Service (IaaS) model served the industry well for years because general-purpose compute became ever faster to deliver on abundant and cheap commodity CPUs. If one box in the cloud went lights-out, just toss it, and another one picks up the workload where it left off. Cloud native development further abstracted CPU and hardware resource concerns in general, with a “cattle not pets” mentality for provisioning and de-provisioning containers and Kubernetes pods for ephemeral workloads.
Unfortunately, unlike CPUs, GPUs are not commodities, they are precious assets: pricey to acquire, in limited supply, and not expendable. Whoever owns and operates the GPUs will seek to maximize their value, by limiting unproductive use and idle time, while increasing their utilization for the most productive AI applications.
GPUs are also not interchangeable with CPUs. GPUs present unique management concerns for operators. However, none of those details should be of any concern to an AI developer or data scientist who is simply looking for GPU-powered resources and expecting the ease of use of any other cloud IaaS.
Operationalizing a GPU platform-as-a-service
Whether delivering GPU access through a customer account within a public cloud IaaS, or provisioning resources through a development or data team within your own organization, capabilities for operationalizing the service for business need to be in place.
Often this initiative is referred to as platform engineering: where a centralized group or IT operations committee aggregates the available resources including GPUs, then allocates and stands up the cloud instances, ideally in a self-service fashion that doesn’t require too many forms, trouble tickets and/or email requests.
The platform engineering team, in turn, needs to validate that allocations of precious GPU resources are authorized for each particular team, gain visibility into how they are being utilized and for what purpose, and account for their cost, whether there is a direct payment required or a departmental chargeback for accounting purposes.
The Rafay platform gives cloud providers and enterprise IT shops a multitenant Platform-as-a-Service (PaaS) for establishing a self-service GPU cloud or platform engineering portal with access controls, cost and customer chargebacks, policy enforcement, and automated resource delivery pipelines. Users can select from several popular flavors of GPU deployments, with models and guardrails for AI use cases.
Figure 1. Rafay PaaS reference architecture for GPU Clouds including NVIDIA GPU hardware and software, and an array of AI software components, controls, and configurable templates for specific industries and use cases. Source: GPU PaaS Reference Architecture, Rafay, February 2026.
A GPU PaaS like Rafay eases the adoption curve for AI, allowing platform users to start small and dip a toe in the water to prove the value. As new functions and services are developed and added using familiar CI/CD and GitOps pipelines, the platform can naturally scale up GPU consumption without a rearchitecture project.
An optimized GPU platform allows focus on AI innovation
While an architect or manager may be keeping a close eye on GPU utilization and cost, developers should focus on innovation that serves customers, without worrying about the underlying architecture, whether the instance runs on bare metal, Kubernetes clusters in a private data center, or virtual clusters on a public cloud service.
Here’s some examples of Rafay’s PaaS in the field supporting unique GPU instances.
- The Australian neocloud Sharon AI uses Rafay as a centralized control plane to orchestrate and govern its AI Factory environments for customers with a localized sovereign architecture designed to manage its 150,000 GPUs.
- Cisco is delivering AI PODs as a GPU Cloud with Rafay as the governance and consumption layer, using automated multi-tenancy and provisioning of workload SKUs, essential for packaging and pricing their racked infrastructure in a way that developers can use a self-service portal or API to access GPU environments without needing to file support tickets.
- Cassava AI is abstracting bare metal and NVIDIA servers as an automated GPU-as-a-Service offering across sub-Saharan Africa with ready-to-use AI workspaces and preconfigured notebook environments. Developers and data scientists access a low-latency local Rafay hub that protects sovereign data, handles resource auto-scaling, and meters costs without relying on global hyperscalers.
For the back office, Rafay Observability tracks inventory, SKU usage, and tenant-level performance across the full stack; while triages, incidents, and agent-driven workflows turn alerts into auditable, approval-gated remediation. Downstream enterprise ITSM and FinOps platforms used by executives and managers integrate with Rafay for conducting design, tracking, and cost/benefit analysis workflows for the GPU estate.
The Intellyx Take
Neoclouds, telcos, and enterprise platform engineering teams don’t need to reinvent the cloud computing paradigm to curate, configure, and deliver GPU-powered AI use cases for each buyer persona, with a familiar, cloud-like customer experience.
The usual value metric for AI infrastructure investments these days is utilization, and that’s still very important, as GPUs are expensive and depreciating assets.
The real measure of success over the long haul is customer value. GPU utilization needs to match up with business utility in order to make an AI product offering that is adopted, essential to customers, and sustainable.
Copyright ©2026 Intellyx B.V. At the time of writing, Rafay is an Intellyx customer. Intellyx retains final editorial control of this content. A human wrote every word of this article. Image credits: Rafay (diagram). Adobe Express (editorial image).
