CUDA Toolkit Install Fails on Ubuntu: Fix Guide

Installation Failure Symptoms

You are trying to install the CUDA toolkit on your Ubuntu GPU server and the installation fails with one of these:

E: Unable to locate package cuda-toolkit-12-4

dpkg: error processing package nvidia-dkms-550 (--configure):
 dependency problems - leaving unconfigured

ERROR: An NVIDIA kernel module 'nvidia-drm' appears to already be loaded.
Installation cannot continue.

CUDA installation on Ubuntu is notoriously fragile because it involves kernel modules, shared libraries, and package manager repositories that can conflict with each other. The path to a clean install depends entirely on which specific failure you encounter.

Cause 1: NVIDIA Repository Not Added

The Unable to locate package error means your apt sources do not include NVIDIA’s repository. Fix it:

# Download and install the CUDA repository pin
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update

# Now install
sudo apt install cuda-toolkit-12-4

Replace ubuntu2204 with ubuntu2004 or ubuntu2404 depending on your version. Check with lsb_release -rs.

Cause 2: Broken Dependencies

If apt reports dependency problems, the most reliable approach is to clean up before retrying:

# Remove partially installed NVIDIA packages
sudo apt --fix-broken install
sudo dpkg --configure -a

# If that fails, purge and start fresh
sudo apt purge 'nvidia-*' 'cuda-*' 'libcudnn*' -y
sudo apt autoremove -y
sudo apt update

# Reinstall
sudo apt install cuda-toolkit-12-4 nvidia-driver-550

This nuclear option removes all NVIDIA software and reinstalls cleanly. It is the fastest path out of dependency conflicts on a dedicated server where you can afford a reboot.

Cause 3: Existing Kernel Module Blocks Installation

The runfile installer refuses to proceed if NVIDIA kernel modules are already loaded. You need to stop any processes using the GPU and unload the modules:

# Stop display manager (if running)
sudo systemctl stop gdm3  # or lightdm

# Kill any GPU processes
sudo fuser -v /dev/nvidia* 2>/dev/null
sudo kill $(fuser /dev/nvidia* 2>/dev/null) 2>/dev/null

# Unload modules
sudo rmmod nvidia_drm nvidia_modeset nvidia_uvm nvidia

# Now run the installer
sudo sh cuda_12.4.0_550.54.14_linux.run

Cause 4: Missing Kernel Headers

DKMS needs kernel headers to compile the NVIDIA driver module. If headers are missing:

sudo apt install linux-headers-$(uname -r)
sudo apt install nvidia-driver-550  # Retry installation

On dedicated GPU servers, ensure your kernel is not custom-compiled without headers. The stock Ubuntu kernel always has headers available via apt.

Recommended Installation Method

After years of CUDA installation pain, the cleanest approach on Ubuntu is the apt-based method with NVIDIA’s official repository. It handles all dependencies automatically:

# 1. Add repository
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb

# 2. Update and install
sudo apt update
sudo apt install cuda-toolkit-12-4 cuda-drivers

# 3. Add to PATH
echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

# 4. Reboot
sudo reboot

Our comprehensive NVIDIA CUDA installation guide covers each step in full detail, including post-installation verification.

Post-Installation Verification

# Driver
nvidia-smi

# CUDA compiler
nvcc --version

# PyTorch integration
pip install torch --index-url https://download.pytorch.org/whl/cu124
python -c "import torch; print(torch.cuda.is_available())"

If all three checks pass, CUDA is correctly installed. For Docker-based workflows, install the NVIDIA Container Toolkit next — our Docker GPU guide covers this. For production setups running vLLM or PyTorch services, proceed to configure monitoring and API security.