VPS Security Hardening: Mitigating Brute-Force Attacks with Fail2Ban

Exposing critical network services (such as SSH, FTP, or SMTP) to the internet makes your server a target for automated security scans. Botnets scan global IP ranges constantly, executing dictionary-based attacks to guess user credentials and gain unauthorized access to servers. If left unmanaged, these brute-force attacks can consume CPU resources, flood system logs, and increase the risk of credential theft.

For systems administrators, implementing brute-force protection is a key step in hardening VPS configurations. The standard tool for mitigating automated attacks on Linux servers is Fail2Ban. Fail2Ban monitors system logs for repeated failed login attempts and blocks the offending IP addresses using firewall rules. In this guide, we will analyze how Fail2Ban works, configure security jails, and walk through the SSH commands to manage banned IP lists and secure your server.

1. How Fail2Ban Protects Linux Servers

Fail2Ban runs as a system daemon that monitors server log files (such as /var/log/auth.log or /var/log/secure) for patterns representing failed login attempts. It uses filters to parse logs and detect patterns (like repeated SSH login failures).

When an IP address triggers a filter rule by exceeding the maximum number of failed attempts within a set time frame (the findtime), Fail2Ban updates your firewall rules (using iptables or firewalld) to block incoming connections from that IP for a specified ban period (the bantime). This helps protect your services from automated attacks without requiring manual administrator intervention.

2. Step-by-Step Security Configuration via SSH

To configure and manage Fail2Ban, log into your server via SSH as the root user. Run these commands to install the package, configure jail parameters, and manage banned IP lists.

Step 2.1: Install Fail2Ban on Your Server

Install Fail2Ban using your operating system's package manager:

yum install epel-release -y  # Enable EPEL repo on RHEL/AlmaLinux
yum install fail2ban -y      # Install Fail2Ban
# Or on Debian/Ubuntu
apt-get update && apt-get install fail2ban -y

Enable and start the service daemon:

systemctl enable --now fail2ban

Step 2.2: Configure Local Jail Settings

Do not modify Fail2Ban's default configuration file (/etc/fail2ban/jail.conf), as it can be overwritten during package updates. Instead, create a local override file called jail.local to manage your configurations:

cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
nano /etc/fail2ban/jail.local

In the [DEFAULT] section, configure the baseline parameters for whitelisting, ban times, and login attempt limits:

[DEFAULT]
ignoreip = 127.0.0.1/8 192.168.1.0/24   # Whitelist trusted IPs
bantime  = 1h                            # Ban duration
findtime = 10m                           # Time window to count failures
maxretry = 5                             # Max failed attempts before ban

Step 2.3: Configure the SSH Security Jail

Scroll down to the SSH jail section (usually labeled [sshd]) in your local configuration file. Enable the jail and configure it to monitor your SSH port:

[sshd]
enabled = true
port    = ssh                            # Or your custom SSH port
logpath = %(sshd_log)s
backend = %(sshd_backend)s

Save changes and restart Fail2Ban to apply the new jail configuration:

systemctl restart fail2ban

Step 2.4: Monitor Jails and Manage Banned IPs

Fail2Ban includes a command-line client tool to inspect active jails, check stats, and manage banned IP lists. Run these commands to monitor the service:

fail2ban-client status                  # View active jails list
fail2ban-client status sshd             # View SSH jail stats and banned IPs

To unban an IP address that was blocked by mistake, run:

fail2ban-client set sshd unbanip 192.168.1.150

To manually ban an IP address using Fail2Ban rules, run:

fail2ban-client set sshd banip 192.168.1.200

3. Fail2Ban Configuration Parameters

Frequently Asked Questions (FAQ)

Will Fail2Ban block me if I forget my password?

Yes. If you exceed the configured maxretry limit during the findtime window, Fail2Ban will block your IP address. To prevent lockouts, whitelist your local static IP address in the ignoreip parameter of your configuration file.

How does Fail2Ban block IP addresses?

Fail2Ban does not block traffic itself. It interacts with your server's system firewall (such as iptables, firewalld, or nftables) to dynamically create and remove firewall rules that block incoming traffic from banned IP addresses.

Can Fail2Ban protect web applications like WordPress?

Yes. Fail2Ban can protect web applications by parsing web server logs (like Nginx or Apache access logs) for repeated POST requests to login pages (e.g., wp-login.php) and banning IPs that trigger the filter rules.