Harden SSH on Linux Servers: Keys, Configs, and Fail2Ban (2026)

SSH is surprisingly fragile for how much we rely on it.

Let’s get your SSH server locked down. We’ll cover key-based authentication, essential sshd_config tweaks, and then hammer the final nail in the coffin with Fail2Ban.

SSH Key Authentication

First, ditch password logins entirely. It’s just not worth the risk.

1. Generate Your SSH Key Pair

On your local machine (not the server), run:

ssh-keygen -t ed25519 -C "your_email@example.com"

This creates ~/.ssh/id_ed25519 (private key) and ~/.ssh/id_ed25519.pub (public key). Guard your private key like it’s your root password.

2. Copy Your Public Key to the Server

Use ssh-copy-id:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@your_server_ip

This appends your public key to ~/.ssh/authorized_keys on the server.

3. Test and Disable Password Authentication

Log in to the server using your key:

ssh -i ~/.ssh/id_ed25519 user@your_server_ip

If that works, it’s time to disable password auth on the server.

Hardening sshd_config

The SSH daemon’s configuration file (/etc/ssh/sshd_config) is your primary control panel.

1. Edit the Configuration

Open the file with root privileges:

sudo nano /etc/ssh/sshd_config

2. Key Directives to Change

• Port 22 -> Port 2222: Change the default port. This isn’t security by obscurity, but it does cut down on automated bot scans.

  • Why it works: Bots are hardcoded to scan port 22. Changing it means you’re only visible to those who know your specific port.
  • Fix: Find Port 22, change it to Port 2222 (or any unused port above 1024).
  • Important: You’ll need to allow this new port in your firewall before restarting SSH. For ufw: sudo ufw allow 2222/tcp.

• PermitRootLogin yes -> PermitRootLogin no: Never allow direct root login.

  • Why it works: If a compromised user account is the only way in, attackers can’t immediately gain root privileges. They must first sudo or su.
  • Fix: Change PermitRootLogin yes to PermitRootLogin no.

• PasswordAuthentication yes -> PasswordAuthentication no: This is the crucial step after key auth is working.

  • Why it works: Completely disables password-based logins, making brute-force attacks on passwords useless.
  • Fix: Change PasswordAuthentication yes to PasswordAuthentication no.

• PermitEmptyPasswords no: Ensure this is set.

  • Why it works: Prevents accounts with no password set from being used for SSH login.
  • Fix: Ensure PermitEmptyPasswords no is present and uncommented.

• ChallengeResponseAuthentication no: Usually disabled by default, but worth checking.

  • Why it works: Disables older, less secure authentication methods like PAM challenges.
  • Fix: Set ChallengeResponseAuthentication no.

• UsePAM yes: Keep this enabled if you use PAM for other services.

  • Why it works: Allows SSH to integrate with Pluggable Authentication Modules, which can provide additional security layers or logging.

• AllowUsers user1 user2: Restrict SSH access to specific users.

  • Why it works: Even if an account is compromised, only listed users can log in via SSH.
  • Fix: Uncomment and add AllowUsers your_username to the end of the file.

3. Validate and Restart SSH

Before restarting, check your config syntax:

sudo sshd -t

If it reports no errors, restart the SSH service:

sudo systemctl restart sshd

Crucially, test your SSH connection again from a new terminal window before closing your current one. If you messed up sshd_config and can’t log in, you’ll be locked out.

Fail2Ban: The Automated Defender

Fail2Ban scans log files (like /var/log/auth.log) and bans IP addresses that show malicious signs — too many password failures, seeking exploits, etc.

1. Install Fail2Ban

sudo apt update && sudo apt install fail2ban

(Use yum install epel-release && yum install fail2ban for RHEL/CentOS).

2. Configure Fail2Ban

Never edit jail.conf. Instead, create a jail.local file to override settings.

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local

Key Settings in jail.local:

• [DEFAULT] section:

  • bantime = 1h: How long an IP is banned. 10m (10 minutes), 1d (1 day), 1w (1 week). Start with 1h.
  • findtime = 10m: The window of time to look for failed attempts.
  • maxretry = 5: Number of failed attempts before a ban.
  • ignoreip = 127.0.0.1/8 ::1 your_home_ip: IPs to never ban. Add your static home/office IP here.

• [sshd] section:

  • enabled = true: Make sure this is set to true to activate SSH protection.
  • port = 2222: Update this to your custom SSH port!
  • logpath = %(sshd_log)s: Usually fine, but ensure it points to your SSH log.
  • backend = %(sshd_backend)s: Let Fail2Ban auto-detect, but if you have issues, systemd or auto are common.

3. Start and Enable Fail2Ban

sudo systemctl start fail2ban
sudo systemctl enable fail2ban

4. Monitor Fail2Ban

Check the status and see who’s been banned:

sudo fail2ban-client status
sudo fail2ban-client status sshd

You can unban an IP if needed:

sudo fail2ban-client set sshd unbanip <IP_ADDRESS>

After implementing these steps, your SSH server will be significantly more resilient to common attacks. The next hurdle is understanding how to manage SSH agent forwarding securely.