The NVIDIA GPU Container Runtime is a plugin that enables container platforms to securely access and manage NVIDIA GPUs as part of a containerized application environment. Docker is an open-source platform that automates the deployment, scaling, and management of applications within lightweight, portable containers. Podman is an open-source, daemonless container engine designed for developing, managing, and running OCI Containers, functioning as a drop-in replacement for Docker. This post contains the setup instructions for NVIDIA GPU container toolkits on Linux hosts running Ubuntu 22.04 for use with docker and podman.
docker and podman.init: 20240224)docker and podman.
We note that NVIDIA’s Container Toolkit officially only supports Ubuntu LTS release, but see this if your system is a 23.04 for example.nvidia-driver.Software & Updates’s Additional Drivers and reboot.ubuntu-drivers devices and install the recommended “server” driver: sudo apt install nvidia-driver-535-server, then reboot.nvidia-smi from a terminal; if a valid prompt shows up, you will have information on the Driver Version and the supported CUDA Version for future running GPU-enabled containers.apt registry option.
Details can be found at https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository.for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done
sudo apt-get update sudo apt-get install ca-certificates curl gnupg sudo install -m 0755 -d /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg sudo chmod a+r /etc/apt/keyrings/docker.gpg echo \ "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
docker is functional by checking if we get the Hello from Docker! message running its hello-world:sudo docker run hello-world
docker available without sudo, which has some security implications, as detailed in https://docs.docker.com/engine/install/linux-postinstall/. docker run hello-world without the need for a sudo:sudo usermod -aG docker $USER
apt search podman returns version 3 of podman. We need a version of podman above 4.1.0 to use the Container Device Interface (CDI) for nvidia-container-toolkit.podman.brew (per Homebrew’s instructions):/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Next Steps section of the installation process (“to add Homebrew to your PATH”, …).brew command and all the Hombebrew installations are relative to the location of /home/linuxbrew, for example,/home/linuxbrew/.linuxbrew/bin/brew.brew is available in your PATH using which brew and is functional with brew update.brew, Installing podman is as simple as:brew install podman
podman:==> Fetching dependencies for podman: bzip2, zlib, pcre2, expat, dbus, ncurses, readline, libxcrypt, util-linux, libffi, mpdecimal, ca-certificates, openssl@3, sqlite, xz, berkeley-db@5, libedit, krb5, libtirpc, libnsl, unzip, [email protected], glib, libseccomp, libcap, lz4, zstd, systemd, conmon, yajl, crun, libfuse, fuse-overlayfs, gmp, libunistring, libidn2, libtasn1, nettle, p11-kit, libevent, libnghttp2, unbound, gnutls, libgpg-error, libassuan, libgcrypt, libksba, libusb, npth, openldap, libsecret, pinentry, gnupg, gpgme, libslirp and slirp4netns [...] ==> Installing podman ==> Pouring podman--4.9.3.x86_64_linux.bottle.tar.gz
Caveats section of the installation, it is recommended to brew services start podman to have podman run as a service now and after reboot, which we want. We recommend you run this command.which podman points us to /home/linuxbrew/.linuxbrew/bin/podman.podman to be functional, we still need to install newuidmap for “rootless mode” (Linux-native “fake root” for rootless containers) bysudo apt install rootlesskit
podman:podman run hello-world
podman runs similarly to docker; for example:podman run --rm -it ubuntu:22.04 /bin/bash
ubuntu:22.04, give you a bash shell prompt in an interactive session, and will delete the created container when you exit the shell.podman has a /home/linuxbrew/.linuxbrew/etc/containers/registries.conf file already configured with unqualified-search-registries=["docker.io"]. More details on that topic at https://podman.io/docs/installation#registriesconf.podman.distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \ && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \ && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \ sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update sudo apt-get install -y nvidia-container-toolkit
docker to recognize the toolkit:sudo nvidia-ctk runtime configure --runtime=docker
dockersudo systemctl restart docker
docker (no sudo needed if you made the optional step in the last section) sees any GPU that you have running on your system by having it run nvidia-smi. Note that docker will need both --runtime=nvidia and --gpus all to use the proper runtime and have access to all the GPUsdocker run --rm --runtime=nvidia --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
/etc/docker/daemon.json file to see that the nvidia-container-runtime is added:[...] "runtimes": { "nvidia": { "args": [], "path": "nvidia-container-runtime" } }
"default-runtime": "nvidia", (after the first {) and sudo systemctl restart docker. You should not have to add --runtime=nvidia to the CLI anymore.podman we will base the instructions for Container Device Interface (CDI) setup on https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/cdi-support.html.nvidia-ctk --version
NVIDIA Container Toolkit CLI version 1.14.5sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
INFO[0001] Generated CDI spec with version 0.5.00 and all)grep " name:" /etc/cdi/nvidia.yaml
podman (no sudo needed) using the proper devices --device nvidia.com/gpu=allpodman run --rm --device nvidia.com/gpu=all ubuntu nvidia-smi
Docker has become a cornerstone of the modern development stack: how applications are built, shipped, and run. It is an excellent solution to the portability problem: containers run consistently on any machine, eliminating the "it works on my machine" problem. It leverages containers: self-contained units of software that package the components needed for an application to run. This primer introduces its core concepts.
Instructions for a Linux host running Ubuntu 24.04 to install the Nvidia runtime for docker and podman.
Kolla Ansible provides production-ready containers (here Docker) and deployment tools for operating OpenStack clouds. This guide explains how to install a single host (all-in-one) OpenStack Cloud on a Ubuntu 22.04 server Linux Operating System on a private network. We specify values and variables early that can easily be adapted to others’ networks. We do not address encryption for the different OpenStack services and will use an HTTPS reverse proxy for access to the dashboard. Please note that this setup requires two physical NICs in the computer you will use.