Nvidia has now banned running CUDA-based software on other hardware platforms using translation layers in its updated licensing terms. This appears to be designed to prevent both the ZLUDA initiative and, perhaps more critically, some Chinese GPU makers from utilizing CUDA code with translation layers. We've pinged Nvidia for comment and will update you with additional details or clarifications when we get a response.
Longhorn, a software engineer, noticed the updated terms. "You may not reverse engineer, decompile or disassemble any portion of the output generated using Software elements for the purpose of translating such output artifacts to target a non-Nvidia platform," a new clause in CUDA 11.5 reads.
Being a leader has a good side and a bad side. On the one hand, everyone depends on you; on the other hand, everyone wants to stand on your shoulders. The latter is apparently what has happened with CUDA. Because the combination of CUDA and Nvidia hardware has proven to be incredibly efficient, tons of programs rely on it. However, as more competitive hardware enters the market, more users are inclined to run their CUDA programs on competing platforms. There are two ways to do it: recompile the code (available to developers of the respective programs) or use a translation layer.
For obvious reasons, using a translation layer like ZLUDA is the easiest way to run a CUDA program on non-Nvidia hardware. All one has to do is take already-compiled binaries and run them using ZLUDA or other translation layers. ZLUDA appears to be floundering now, with both AMD and Intel having passed on the opportunity to develop it further, but that doesn't mean translation isn't viable.
Several Chinese GPU makers, including one funded by the Chinese government, claim to run CUDA code. Denglin Technology designs processors featuring a "computing architecture compatible with programming models like CUDA/OpenCL." Given that reverse engineering of an Nvidia GPU is hard (unless one already somehow has all the low-level details about Nvidia GPU architectures), we are probably dealing with some sort of translation layer here, too.
One of the largest Chinese GPU makers, Moore Threads, also has a MUSIFY translation tool designed to allow CUDA code to work with its GPUs. However, whether or not MUSIFY falls under the classification of a complete translation layer remains to be seen (some of the aspects of MUSIFY could involve porting code). As such, it isn't entirely clear if the Nvidia ban on translation layers is a direct response to these initiatives or a pre-emptive strike against future developments.
For obvious reasons, using translation layers threatens Nvidia's hegemony in the accelerated computing space, particularly with AI applications. This is probably the impetus behind Nvidia's decision to ban running their CUDA applications on other hardware platforms using translation layers starting from CUDA 11.5.
The clause was absent in the CUDA 11.4 release, so it looks like running applications compiled using CUDA 11.4 and earlier compilers on non-Nvidia processors using translation layers is still fine. To that end, Nvidia won't achieve its goal of preventing everyone from running software developed for its hardware on other hardware platforms using layers like ZLUDA in the short term. Longer-term, the company will certainly place legal barriers for running CUDA programs via translation layers on third-party hardware, which could have a positive effect for Nvidia and a negative one for AMD, Intel, Biren, and other developers of AI compute hardware.
Recompiling existing CUDA programs remains perfectly legal. To simplify this, both AMD and Intel have tools to port CUDA programs to their ROCm (1) and OpenAPI platforms, respectively.
As AMD, Intel, Tenstorrent, and other companies develop better hardware, more software developers will be inclined to design for these platforms, Nvidia's CUDA dominance could ease over time. Furthermore, programs specifically developed and compiled for particular processors will inevitably work better than software run via translation layers, which means better competitive positioning for AMD, Intel, Tenstorrent, and others against Nvidia — if they can get software developers on board. GPGPU remains an important and highly competitive arena, and we'll be keeping an eye on how the situation progresses in the future.
