What is the issue?

I have been suffering 3 hours this morning to make nvidia work with ollama fresh install. Whatever model i tried It did not use the nvidia H100 GPUs even if the systemctl status ollama is nicely showing the GPUs. For this you need to install nvidia toolkit. I have picked the latest of driver, toolkit, cuda and ollama did not load in the GPUs. Then i discovered I dont have the AVX enabled in my CPU of the VM. So, i said, its really good to have, so i enabled it and BINGO !!!! , ollama got loaded into the GPU!! Now all good!!

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.5.4