Remotely Unlock LVM on LUKS over SSH on AlmaLinux 9 and Rocky Linux 9
Two ways to unlock a LUKS-encrypted root volume on a remote AlmaLinux 9 or Rocky Linux 9 server: SSH in the initramfs with dracut-sshd, or automatic unlocking with Clevis and Tang.
A server whose root filesystem sits on LVM inside a LUKS container stops at a passphrase prompt on every boot. On a remote machine that normally means opening your provider's web console each time. This guide shows two ways around that on AlmaLinux 9, Rocky Linux 9 and CentOS Stream 9: an SSH server inside the initramfs, so you can log in and type the passphrase before the root volume exists, and automatic unlocking against a key server with Clevis and Tang.
The older version of this procedure was written for CentOS 7, which reached end of life in June 2024, and used the dracut-crypt-ssh module with a hand-built partition layout. On the RHEL 9 family the installer creates LVM on LUKS for you, and the maintained equivalents are dracut-sshd and Clevis.
Which method to use
| Method | How it unlocks | Where it comes from |
|---|---|---|
dracut-sshd | You SSH into the initramfs and enter the passphrase yourself | Community project, packaged in EPEL 9. Not part of the distribution and not covered by Red Hat's documentation. |
| Clevis and Tang (network-bound disk encryption) | The server unlocks itself when it can reach a Tang server on the network | Shipped in the distribution's own repositories and documented in the Red Hat Enterprise Linux 9 Security hardening guide. |
Use dracut-sshd when a person should approve every boot and you have only one machine. Use Clevis and Tang when you have a second, trusted machine on the same private network and want unattended reboots. Both leave your LUKS passphrase working, so the web console remains a fallback.
Requirements
- AlmaLinux 9, Rocky Linux 9 or CentOS Stream 9 installed with an encrypted root volume. In the installer's storage screen, selecting Encrypt my data with the default LVM scheme gives you LVM on LUKS with an unencrypted
/boot. - A user with
sudorights (this guide usesdeploy) and an SSH public key. - Access to your provider's web console. You need it if the early-boot network or SSH setup does not work the first time.
Confirm the layout before you change anything:
lsblk -o NAME,TYPE,FSTYPE,MOUNTPOINTS
You should see a partition with the file system type crypto_LUKS, a crypt device below it, and your LVM logical volumes below that. Note the name of the LUKS partition, for example /dev/vda3.
Option 1: unlock over SSH with dracut-sshd
Install the packages
dracut-sshd is in the EPEL repository. It adds the system's OpenSSH server to the initramfs; dracut-network provides the early-boot networking.
sudo dnf install -y epel-release
sudo dnf install -y dracut-sshd dracut-network
Choose which keys may log in
The module copies the first file it finds from this list into the initramfs: /root/.ssh/dracut_authorized_keys, /etc/dracut-sshd/authorized_keys, /root/.ssh/authorized_keys. Only key authentication is available, and you log in as root. To use the same key as your deploy user:
sudo mkdir -p /etc/dracut-sshd
sudo cp /home/deploy/.ssh/authorized_keys /etc/dracut-sshd/authorized_keys
sudo chmod 600 /etc/dracut-sshd/authorized_keys
The root account must not be locked with ! in /etc/shadow, or the initramfs SSH server refuses the login. Check it, and if the second field of the output is L or LK, replace the lock with *, which still blocks password logins:
sudo passwd -S root
sudo usermod -p '*' root
Bring the network up in early boot
The initramfs needs an address before the root volume is open. Create a dracut configuration file that adds the network module and the kernel parameters for DHCP:
sudo tee /etc/dracut.conf.d/90-network.conf > /dev/null <<'EOF'
kernel_cmdline="rd.neednet=1 ip=dhcp"
add_dracutmodules+=" network "
EOF
If the server has a static address, put it on the kernel command line with grubby instead of using ip=dhcp above. The format is ip=<address>::<gateway>:<netmask>:<hostname>:<interface>:none:
sudo grubby --update-kernel=ALL --args="rd.neednet=1 ip=203.0.113.10::203.0.113.1:255.255.255.0:server1:eth0:none"
Replace eth0 with the interface name shown by ip link. In that case keep only the add_dracutmodules line in the configuration file.
Rebuild the initramfs
sudo dracut -f -v
sudo lsinitrd | grep 'authorized\|bin/sshd'
The second command must list sshd and an authorized_keys file. If it prints nothing, the module was not included; check that the package is installed and repeat.
Reboot and unlock
Keep the web console open in another window, then reboot:
sudo reboot
After a few seconds, connect as root and start the password agent:
ssh root@203.0.113.10
systemd-tty-ask-password-agent
Type the LUKS passphrase at the prompt. When every pending prompt is answered, the boot continues and the SSH session closes on its own. Wait a moment and log in again as deploy; you are now on the fully booted system.
About the host key
By default the module copies the server's normal SSH host keys into the initramfs, so your client sees the same fingerprint in both stages. The initramfs lives on the unencrypted /boot partition, so anyone who can read the disk offline can read those keys. If that matters in your threat model, create a separate key that is used only in early boot. The module picks up files named /etc/ssh/dracut_ssh_host_*_key instead of the system keys:
sudo ssh-keygen -t ed25519 -N '' -f /etc/ssh/dracut_ssh_host_ed25519_key
sudo dracut -f -v
Your SSH client will then see two different keys for the same address, so give the early-boot connection its own Host entry and known-hosts file in ~/.ssh/config.
Option 2: automatic unlocking with Clevis and Tang
Network-bound disk encryption ties the volume to a network instead of a person. A Tang server holds keys; the Clevis client on the encrypted machine adds an extra LUKS key slot that can only be reconstructed while Tang is reachable. Take the disk out of that network and it stays locked. Tang should live on a private or otherwise trusted network segment, not on the public internet.
Set up the Tang server
On a second AlmaLinux 9 or Rocky Linux 9 machine (198.51.100.20 in this example):
sudo dnf install -y tang
sudo systemctl enable --now tangd.socket
sudo firewall-cmd --add-service=http
sudo firewall-cmd --runtime-to-permanent
sudo tang-show-keys
Tang listens on the HTTP port by default. The last command prints the thumbprint of the signing key; keep it for the next step.
Bind the LUKS volume on the encrypted server
sudo dnf install -y clevis clevis-luks clevis-dracut
sudo clevis luks bind -d /dev/vda3 tang '{"url":"http://198.51.100.20"}'
Clevis shows the signing key thumbprint advertised by the Tang server and asks whether to trust it. Compare it with the output of tang-show-keys, answer y, then enter your existing LUKS passphrase. The binding is stored in a new key slot; the passphrase slot is left as it is. Check the result:
sudo clevis luks list -d /dev/vda3
Rebuild the initramfs with networking
Early boot needs the Clevis module and a network connection. Rebuilding with --hostonly-cmdline stores the required rd.neednet=1 parameter in the image:
sudo dracut -fv --regenerate-all --hostonly-cmdline
For a static address, add the same ip= parameter shown in Option 1 with grubby. Reboot and watch the console: the passphrase prompt appears briefly and disappears once Clevis has fetched the key. If Tang cannot be reached, the prompt stays and you can type the passphrase as before.
Troubleshooting
Permission denied (publickey)in early boot: the root account is locked with!, or the wrongauthorized_keysfile was picked up. Fix it, then runsudo dracut -f -vagain.- No answer on port 22 before unlocking: the initramfs has no network. Check the
ip=parameter and interface name from the web console, unlock there, and correct the configuration. - Need a shell in the initramfs: add
rd.breakto the kernel command line from the boot menu, then inspectsystemctl status sshd.serviceandjournalctl -u sshd. - Clevis stopped unlocking after Tang keys were rotated: run
sudo clevis luks report -d /dev/vda3 -s 1(use the slot number fromclevis luks list) and accept the offer to regenerate the binding.
References: the dracut-sshd project and Red Hat's policy-based decryption chapter.
Frequently asked questions
Does this protect against someone with physical access to the server?
Only partly. LUKS protects data on a powered-off disk. The kernel and initramfs on /boot are not encrypted, so an attacker who can modify them could capture the passphrase on the next unlock. Defending against that needs a verified boot chain as well as disk encryption.
Do I have to repeat the setup after a kernel update?
No. The files under /etc/dracut.conf.d, the installed dracut modules and the kernel arguments set with grubby --update-kernel=ALL apply to initramfs images built for new kernels. If you change the authorized keys or host keys, rebuild with sudo dracut -f --regenerate-all.
Can I use both methods on the same server?
The two address different needs, and your passphrase remains valid with either. A common arrangement is Clevis and Tang for routine reboots, with the web console as the manual fallback. Test any combination with a reboot while you still have console access.
How do I remove a Clevis binding?
Find the slot with sudo clevis luks list -d /dev/vda3 and remove it with sudo clevis luks unbind -d /dev/vda3 -s 1, replacing 1 with the slot number. Make sure you still know the passphrase first.