Fail2ban 0.10 on CentOS6 - yes we can! And my notes on successful manual upgrades

Fail2ban's official compile for CentOS6 has never advanced beyond 0.9.6-1.el6.1. While 0.9.6 works, it's old and has a few major inconveniences:

  1. No IPv6 support 🙁
  2. A tendency for the daemon to die when parsing many logs or a high volume of activity
  3. Quite slow to parse logs when restarting after a config change

So, what can we do? Well, compile from source and upgrade with the supplied python script! It's easier than I anticipated, but there's still a few things you need to watch out for.

Prerequisite Alert! This guide assumes you're already running the latest version from EPEL, installed via `yum`. If not, do that first; it'll set up a few useful things (init scripts, paths and application locations, config structure).

Upgrade Steps

  1. Stop your existing fail2ban installation:
    service fail2ban stop
  2. Back up your existing fail2ban installation, most likely in /etc/fail2ban! Example:
    sudo mkdir /etc/f2b-backup && sudo cp -r /etc/fail2ban /etc/f2b-backup
  3. Head to https://github.com/fail2ban/fail2ban/releases and grab the zip of the latest 0.10.4 release (NB that 0.11 is in development)
  4. With Python 2 ('python', as opposed to 'python3') installed, extract fail2ban then compile and install:
    wget https://github.com/fail2ban/fail2ban/archive/0.10.4.zip
    unzip 0.10.4.zip
    cd fail2ban-0.10.4
    python setup.py install
  5. The installer should drop every binary into the correct place, no changing of $PATH required.

However, CentOS7/Fedora changed a few things under the surface from CentOS6; fail2ban's configs are now intended for use on (at least) a CentOS 7/Fedora 19 system. As such, some of things like 'backends' (sources of log files) are defined differently as CentOS7 comes with systemd instead of initd. You'll need to make some minor modifications to config files to get F2B running again. I needed to do the following:

To start the SSHD jail, modify /etc/fail2ban/paths-fedora.conf, commenting out the following lines (courtesy of John Smith on SO):
syslog_backend = systemd
sshd_backend = systemd
dropbear_backend = systemd
proftpd_backend = systemd
pureftpd_backend = systemd
wuftpd_backend = systemd
postfix_backend = systemd
dovecot_backend = systemd

NB: in this specific instance you cannot specify a paths-fedora.local override file due to how fail2ban loads these core config files, so remember to check the file after your next update from source.

Start fail2ban with service fail2ban start and you should see it start up. If possible, run a second SSH session so you can watch fail2ban's log as it runs through log files during startup: tail -fn 100 /var/log/fail2ban.log

Making pesky SSH chancers go away

This week my system has been experiencing botnet-originated SSH drive-by login attempts using dictionary lists of usernames. Only one try per username means it wasn't tripping the normal ban-after-ban-after-three-tries jail for invalid passwords. To mitigate this, I added a new jail called "sshd-invalidusers", based on cweiske's guide on AskUbuntu.

Create the file /etc/fail2ban/filter.d/sshd-invaliduser.conf with the following:

[INCLUDES]
before = common.conf

[Definition]
_daemon = sshd

failregex = ^%(__prefix_line)s[iI](?:llegal|nvalid) user .*? from <HOST>(?: port \d+)?\s*$
            ^%(__prefix_line)sFailed password for invalid user .*? from <HOST>(?: port \d+)?\s*$

ignoreregex =

[Init]
journalmatch = _SYSTEMD_UNIT=sshd.service + _COMM=sshd

Then, add the jail to your /etc/fail2ban/jail.local override file:

[sshd-invaliduser]
enabled = true
maxretry = 1
port    = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s

Always use .local files, not .conf!

While tempting to modify the various .conf files, you're setting yourself up for a fall. Do not directly modify /etc/fail2ban/jail.conf(or any .conf file), as the default fileset they are overwritten by any future upgrades.

If you are making any modifications to jail.conf, first copy that file to jail.local (cp jail.conf jail.local) and work off the .local file in future. Same goes for any .conf file, the .local file sitting next to it will always take precedence when fail2ban loads its configs.

A few other useful things

This server runs an Apache VirtualHost multi-domain setup; each site's access, error and combined logs, for both HTTP and HTTPS, get their own file. I also run some management software which handles Apache configurations on a per-user basis, so there's lots of nested folders - first of users, then inside those each user's sites' log files. The structure is /var/log/httpd/username/sitename-category.log. I find this useful for organisation and monitoring. I can write separately about how I did this if anyone's interested - leave a comment or send me a Tweet.

A few users run WordPress sites, so I'd like to monitor all the logs using the wordpress, apache-wp-login and apache-wp-xmlrpc jails. This is actually easier than you'd think thanks to wildcard syntax...

NB: each jail has an additional action to report repeat abusive traffic to AbuseIPDB using their API (action_abuseipdb). Remove this line from each jail if you don't have this setup on your system.

[wordpress]
enabled = true
filter = wordpress
logpath = /var/log/httpd/domains/*/*.log
port = http,https
maxretry = 8

[apache-wp-login]
enabled = true
port    = http,https
filter  = apache-wp-login
logpath = /var/log/httpd/domains/*/*.log
maxretry = 6
findtime = 3600
bantime = 36000
action = %(action_)s
        %(action_abuseipdb)s[abuseipdb_apikey="myapikey", abuseipdb_category="10,18,20,21"]

[apache-wp-xmlrpc]
enabled = true
port    = http,https
filter  = apache-wp-login
logpath = /var/log/httpd/domains/*/*.log
maxretry = 6
findtime = 3600
bantime = 36000
action = %(action_mwl)s
        %(action_abuseipdb)s[abuseipdb_apikey="myapikey", abuseipdb_category="10,18,20,21"]

Other good reads

Handy things for testing...

Any thoughts, suggestions or errors? Please Tweet me or leave a comment!

Leave a Reply

Your email address will not be published. Required fields are marked *

Notify me of followup comments via email. You can also subscribe without commenting.

I