The Linux package supply chain is more vulnerable than you think, and attackers are increasingly targeting it.
Let’s watch apt in action, but not with just any packages. We’re going to verify packages signed by a specific key.
Imagine you’re about to install nginx on your Ubuntu server. When you run sudo apt update && sudo apt install nginx, apt doesn’t just blindly download and install the nginx package. It’s performing a series of checks, and one of the most critical, yet often overlooked, is package signing.
Here’s a simplified view of what happens under the hood when apt installs a signed package:
- Repository Metadata Download:
aptfirst downloadsReleasefiles from the repository. These files contain hashes of other index files (likePackagesfiles) and are themselves signed by the repository’s GPG key. - Release File Verification:
aptchecks the signature of theReleasefile against the GPG keys it trusts. If the signature is invalid,aptwill refuse to proceed, preventing you from using potentially compromised index files. - Package Index Verification:
aptthen downloads thePackagesfile (which lists available packages, their versions, and their dependencies) and other index files. The hashes for these files are listed in the signedReleasefile.aptcalculates the hash of the downloaded index files and compares it to the hash provided in theReleasefile. If they don’t match, the index files have been tampered with. - Package File Verification: Finally, when you request to install a specific package,
aptdownloads the.debfile. ThePackagesfile contains the hash of the actual.debfile.aptcalculates the hash of the downloaded.debfile and compares it. If they match,aptknows the package hasn’t been altered in transit.
This multi-layered verification ensures that the packages you install originate from a trusted source and haven’t been modified by an attacker.
Let’s look at the configuration that makes this possible. On your system, apt’s trust in repository keys is managed in /etc/apt/trusted.gpg.d/. Each .gpg file in this directory is a public GPG key that apt will use to verify repository signatures. For example, if you’re using the official Ubuntu repositories, you’ll see keys like ubuntu-keyring-v2.gpg or similar.
When a repository is added (e.g., via add-apt-repository), its signing key is typically imported into this directory. The Release file from the repository will have a signature generated using the corresponding private GPG key. apt uses the public key stored locally to verify this signature.
The problem this solves is the "man-in-the-middle" (MITM) attack on package repositories. Without signing and verification, an attacker could intercept your connection to a repository, replace legitimate packages with malicious ones, and your system would happily install them.
Here’s how you’d manually verify a package’s signature yourself, beyond what apt does automatically. Let’s say you’ve downloaded a .deb file, my-package_1.0.0_amd64.deb, and you want to be extra sure. You’d first need the repository’s signing key. Let’s assume you have the public key in a file named repo.gpg.
# Import the public key into your GPG keyring (if not already done by apt)
gpg --import repo.gpg
# Verify the signature of the Release file (assuming it's in the same directory as the package)
# First, get the Release file and its signature
wget http://example.com/ubuntu/dists/jammy/Release
wget http://example.com/ubuntu/dists/jammy/Release.gpg
# Verify the signature
gpg --verify Release.gpg Release
If the gpg --verify command outputs "Good signature from 'Repository Name email@example.com'", then the Release file is authentic. Now, you’d need to check the hash of the .deb file against the Release file.
# Calculate the SHA256 hash of the downloaded .deb file
sha256sum my-package_1.0.0_amd64.deb
# Find the entry for my-package in the Release file and compare the hashes.
# You'd typically look for a line like:
# SHA256: <hash_of_my-package_1.0.0_amd64.deb> my-package_1.0.0_amd64.deb
This manual process mirrors what apt automates. The key is that apt has a pre-configured list of trusted keys for each repository it uses.
The one thing most people don’t realize is that apt doesn’t just check the signature of the .deb file itself. The .deb file is essentially an archive. The signature is applied to the index files (Release file, Packages file, etc.) that point to the .deb file. If an attacker compromises the repository server itself and replaces the Release file with one pointing to a malicious .deb with a matching hash, apt’s verification chain would still hold if the attacker can also forge the Release.gpg signature. However, the primary defense is against MITM attacks on the network or compromised mirrors, where the attacker cannot forge the repository’s private key.
The next step in securing your software is understanding and managing repository sources themselves, ensuring you’re only adding trusted repositories to your system.