Setting up NVIDIA docker & podman (Ubuntu 24.04, Rev: 06/27)

Preamble

Confirming the Nvidia driver is available

• On Ubuntu Desktop, install from Software & Updates’s Additional Drivers and reboot.

• On Ubuntu Server, confirm the device is available using ubuntu-drivers devices and install the recommended “server” driver: sudo apt install nvidia-driver-535-server, then reboot.
• if you have an aplay error, you can sudo apt-get install alsa-utils

Using a more recent driver

• Check which driver is currently in use:

apt list --installed | grep nvidia | grep modules

• Seek for an available package that matches this format for 550:

apt search nvidia | grep 550 | grep modules | grep generic-hwe

• This gives us multiple options. The one that matches my use case (not a server, for example) is linux-modules-nvidia-550-generic-hwe-24.04, which we will install:

sudo apt install linux-modules-nvidia-550-generic-hwe-24.04

• You can then check that the modules for your kernel were installed by running:

sudo apt-cache policy linux-modules-nvidia-550-$(uname -r)

• The Installed and Candidate versions should match. You can also test reinstallation using sudo apt install linux-modules-nvidia-550-$(uname -r)

• After installing the kernel modules, install the driver meta-package which will install the required additional packages for the expected kernel:

sudo apt install nvidia-driver-550

• A reboot is required to load the new driver.

• After reboot, you will be able to confirm that the driver is running by using nvidia-smi which in this case now returns:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.90.07              Driver Version: 550.90.07      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+

Docker setup (from docker.io)

for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

sudo docker run hello-world

sudo usermod -aG docker $USER

Install podman

sudo apt install podman

podman run hello-world

podman run --rm -it docker.io/ubuntu:24.04 /bin/bash

NVIDIA Container Toolkit

• supported Linux distributions

• NVIDIA driver requirements and minimal hardware supported

• Docker versions

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

For Docker

sudo nvidia-ctk runtime configure --runtime=docker

sudo systemctl restart docker

docker run --rm --runtime=nvidia --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi

[...]
    "runtimes": {
        "nvidia": {
            "args": [],
            "path": "nvidia-container-runtime"
        }
    }

For Podman

nvidia-ctk --version

sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml

grep "  name:" /etc/cdi/nvidia.yaml

podman run --rm --device nvidia.com/gpu=all ubuntu nvidia-smi

Revision History

• 20240626-0: Added content to update Nvidia driver from 535 to 550 (from supporting CUDA 12.2 to 12.4 in containers)

• 20240526-0: Added a note about using podman only on hosts where you intend to use bridged VMs.

• 20240523-0: Updated content for Ubuntu 24.04

• 20240225-0: Initial release (for Ubuntu 22.04)