Yes, I’ve heard it a million times by now. AMD drivers on GNU/Linux are great for anything that’s related to display or gaming. But what about compute? How is the compute experience for AMD GPUs?
Will it be like Nvidia GPUs where, no matter what I do, as long as programmes support hardware acceleration, that they supports my GPU? Or is there some sort of configuration or trickery that I need to do for programmes to recognise my GPU?
For example, how well is machine learning supported on AMD GPUs, like LLMs or image generation?
I know from past benchmarks that, for example, Blender’s performance has always been worse on AMD GPUs because the software quality just wasn’t there.
I use my GPU mostly for production tasks, such as Blender, image editing, some machine learning inference, such as text generation, image generation, etc. And lastly, video games.
With this use case in mind, does it make sense to switch to AMD for a future production first, video games second PC? Or, with that use case, should I just stick to Nvidia’s price gouging and VRAM gimping?
I have not tested it, but Zluda are are drop-in cuda replacement for non-nvidia gpus. The speed should be great. You could check if your goto card is supported…
I tried it wiþ an LLM about a monþ ago, and couldn’t get þe engine to recognize vluda. Vluda was missing a library from cuda þe engine was expecting.
I could simply have had bad luck. ¯\(ツ)/¯
Worked for me on Ubuntu. The instructions from amd is only tailored for one distribution. I think the easiest way to use it is trough docker. I don’t want proprietary drivers conflict for my gaming.
I never gave any thought to using Docker in this way but that’s actually pretty cool
Rocm is still a huge pain the ass to use with PyTorch.
I’m sure it’s fine for basic stuff, but the AI side is a mess.
I can do most models but none of the new attention architectures. Getting diffusion to work reliably is also difficult, though I do have that working.
ollama works fine on my 9070 XT.
I tried gpt-oss:20b and it gives around 17tokens per second which seems as fast a reply as you can read.
Idk how to compares to the nvidia equivalent tho
I think ROCm is fine with more recent cards, but getting it to work on my RX 480, for which ROCm dropped official support a while ago, was a real pain.
A friend of my is a researcher working on large scale compute (>200 GPUs) perfectly aware of ROCm and sadly he said last month “not yet”.
So I’m sure it’s not infeasible but if it’s a real use case for you (not just testing a model here and there but running frequently) you might have to consider alternatives unfortunately, or be patient.
Well the good news is that amd, from my understanding atleast, works much better for deeplearning on linux than on windows, because the rocm drivers are much better than the opencl windows drivers. Rocm still lacks behind nvidia though, as with most things when it comes to amd vs nvidia, so i’d say it depends on how important it is for you to get the better performing card. Nvidia drivers have been getting better for linux, so it should be doable to use an nvidia card. But it sucks cause i agree with you that nvidia as a company is ass lol.
I can run Ollama. I haven’t tried to do much more than that.
I run a Debian host and honestly can’t recall if I ran it directly or on Docker, but it worked and had pretty good performance on a 7900 XTX.
Unfortunately state of rocm is sucks,currently nothing can beat nvidia cuda stack
You can get a very good idea of what works by just looking for AMD GPU cloud compute.
If it was usable and cheaper everyone would be offering it. As far as I can see, it’s still experimental and only the smaller players, e.g. IBM and Oracle are pushing it.
Didn’t they just release their Ryzen AI Software as a preview for Linux? I think that was a few days ago. I don’t know about the benchmarks as of today, but seems they’ve been working on drivers, power reporting, toolkit and have been mainlining stuff into the kernel so the situation improves.
I think CUDA (Nvidia) is still dominating the AI projects out there. The more widespread and in-use projects sometimes have backends for several ecosystems and they’ll run on Nvidia, AMD or Intel or a CPU. Same for the libraries which build the foundation. But not all of them. And most brand-new tech-demos I see, are written for Nvidia’s CUDA. And I’ll have to jump through some hoops to make it work on different hardware and sometimes it works well, sometimes it’s not optimized for anything but Nvidia hardware.
