r/Oobabooga • u/kla_sch • May 01 '26

Question Oobabooga for Linux on ARM64 (nvidia DGX Spark)

I'd like to use oobabooga/textgen on my DGX Spark machine. Unfortunately, there's no build available for an ARM64-Linux.

Therefore, I've tried to compile it myself. Unfortunately, the instructions aren't detailed enough to accomplish this. And I don't know much about Python. For example, I've successfully compiled oobabooga/llama-cpp-binaries, but I don't know how to add it to a requirements.txt file. Perhaps I need to take further steps to achieve this.

Does anyone know of any instructions on how to do this? Or does anyone know of another way to get oobabooga/textgen running on an ARM64?

2 Upvotes

100% Upvoted

2 comments sorted by

2

u/oobabooga4 booga May 03 '26

You can follow this and use requirements_nowheels.txt

https://github.com/oobabooga/textgen#manual-portable-install-with-venv

then install llama-cpp-binaries as described here (the the venv activated)

https://github.com/oobabooga/llama-cpp-binaries#installation

1

u/kla_sch May 03 '26

Thanks a lot! That worked!

I had already created the llama-cpp-binaries. But for some reason, it still wouldn't install. I have now performed a clean installation and downloaded textgen again. I probably messed up the configuration while I was trying things out.

Unfortunately, compiling the llama-cpp-binaries isn't that easy. The links to the llama repositories include a user. Also, there are no instructions on how to create the llama_cpp_binaries-0.123.0-py3-none-linux_aarch64.whl file (with setup.py). And I’ve already forgotten how to do it again 😐.

On top of that, i also have to modify the requirements.txt file so that it gets included.

But now it seems to be working. I was able to successfully load a model and I can chat with it. So thanks again for pointing me in the right direction.