NVIDIA’s CUDA Rust Push Brings GPU Kernels to Rust

Picture a Rust service that handles model requests, manages memory, and coordinates inference. The application is written in one language—until the performance-critical GPU kernel appears. Traditionally, that small function runs on the GPU but must be written in CUDA C++ or another specialized language.

NVIDIA’s September 8, 2026 CUDA Rust announcement targets that boundary. Its new Rust projects compile GPU kernels written in Rust into PTX, the assembly-like intermediate format used by NVIDIA GPUs. This is not a Rust wrapper around a C++ kernel. The kernel itself can now begin as Rust code. (developer.nvidia.com)

Why GPU kernels are a difficult Rust problem

Rust’s compiler checks how data is borrowed and shared before a program runs. That prevents many bugs involving dangling references, invalid memory access, and two parts of a program modifying the same data unexpectedly. On a GPU, thousands of threads may execute the same kernel at once, so a small mistake in ownership can become a race that appears only under particular workloads.

What does native GPU programming in Rust actually change? It gives the compiler more of the program to inspect. Instead of using Rust for the host application and treating the GPU as a foreign island, developers can carry familiar ownership rules into the kernel—and choose how much control they want over threads and memory.

NVIDIA’s CUDA Rust work follows two CUDA programming models. SIMT, short for single instruction, multiple threads, describes the traditional approach: you write what one thread should do, then launch many copies of that work. Tile programming works at a higher level. You describe an operation over a block, or tile, of data, while the compiler decides how that tile maps onto real GPU threads.

The SIMT path: cuda-oxide

The first route, cuda-oxide, is aimed at developers who want CUDA-style control in Rust. It is a custom rustc code-generation backend—a compiler component that translates selected Rust functions into GPU code. Its pipeline moves device functions through Rust’s intermediate representation, Pliron, LLVM, and finally PTX, while the host-side portions continue through the ordinary Rust toolchain.

A small vector-add kernel shows the model:

#[kernel]
fn add(a: &[f32], b: &[f32], mut out: DisjointSlice<f32>) {
 let lane = thread::index_1d;

 if let Some(slot) = out.get_mut(lane) {
 let i = lane.get;
 *slot = a[i] + b[i];
 }
}

The function describes one thread’s job. Each thread reads one position from the input slices and writes one position in the output. The unusual-looking DisjointSlice type carries an important promise: every thread receives exclusive access to its own output element instead of sharing one unrestricted mutable slice.

The launch configuration is part of the safety story too. A launch contract can state that the kernel expects one-dimensional indexing and blocks of 256 threads. The generated host API checks the requested launch against that contract and the device’s limits before the kernel runs. Rust does not remove the need to understand GPU execution, but it can move several failure modes from runtime debugging into compile-time or launch-time checks. (developer.nvidia.com)

The Tile path: cutile-rs

cutile-rs takes a different route. Instead of asking you to calculate a thread index for every scalar, it lets you operate on tensor tiles. A host-side partition gives each tile ownership of a separate region of the output, and the Tile compiler determines how many hardware threads should execute the work.

A launch might look conceptually like this:

let output = zeros::<f32>(&[1024]).partition([128]);

let result = kernel::add(output, x, y)
.first
.unpartition;

Here, the output is divided into eight 128-element pieces. That partition does more than describe data layout. It establishes which tile may write to which region, supplies the tile size to the kernel, and determines the launch grid. The program does not separately calculate a block count and hope it agrees with the kernel’s indexing rules.

The cutile-rs macro captures the kernel structure and uses CUDA Tile IR, NVIDIA’s tile-level intermediate representation, to compile the GPU code when it is first needed. That process is called just-in-time compilation, or JIT compilation. The result is a higher-level programming experience: fewer explicit thread decisions, less shared-memory bookkeeping, and a smaller surface for thread races. (developer.nvidia.com)

Which CUDA Rust track fits?

Tile is the natural starting point when a computation can be expressed as operations over blocks of data. NVIDIA recommends it first because the compiler can adapt the mapping between tiles and GPU architectures without forcing the source code to encode every hardware choice.

SIMT remains valuable when you need precise control over thread indexing, shared memory, warp operations, atomics, or specialized scheduling. Shared memory is a fast on-chip region used by threads in a block, and controlling it directly can be the difference between an ordinary kernel and a carefully tuned one. That control also brings more responsibility; in the current SIMT path, some shared-memory operations still require unsafe Rust.

The choice of language and the choice of programming model are separate decisions. A team can prefer Rust for its host application while choosing Tile for one kernel and SIMT for another. NVIDIA has also described plans for interoperability among CUDA Rust, CUDA C++, and CUDA Python, so adopting one frontend is not meant to isolate a project from the rest of the CUDA ecosystem.

The toolchain is promising, but still moving

As of September 17, 2026, the setup remains demanding. Current cuda-oxide installation notes list Linux, an Ampere-or-newer GPU with compute capability sm_80 or higher, CUDA Toolkit 13.0 or newer, LLVM 21 or newer with NVIDIA PTX support, Clang 21, and a pinned Rust nightly toolchain. cutile-rs targets Linux as well, recommends CUDA 13.3, requires Rust 1.89 or newer, and uses stable Rust without requiring developers to install their own LLVM. (github.com)

Neither project should be treated as a finished replacement for CUDA C++. NVIDIA describes both as early-stage. cuda-oxide has progressed to a v0.2.0 community release, while cutile-rs has published beta releases with stable host- and device-side APIs, but compiler coverage and interfaces are still changing.

The more significant shift is architectural. The current cuda-oxide repository already documents interoperation that can chain a Tile kernel from cutile-rs with a SIMT kernel from cuda-oxide over shared device tensors and the same CUDA stream. That points toward a future where Rust is not another isolated GPU experiment, but one frontend inside a mixed CUDA application.

NVIDIA’s CUDA Rust announcement does not make GPU programming effortless, and it does not erase the need to understand parallel execution. It does something more useful: it gives Rust developers a native path into the kernel itself, with one option for hardware-level control and another for compiler-managed tiles. The long-term promise is not that every CUDA program will become Rust. It is that Rust no longer has to stop at the edge of the GPU.