Fail2ban's official compile for CentOS6 has never advanced beyond 0.9.6-1.el6.1. While 0.9.6 works, it's old and has a few major inconveniences:
- No IPv6 support 🙁
- A tendency for the daemon to die when parsing many logs or a high volume of activity
- Quite slow to parse logs when restarting after a config change
So, what can we do? Well, compile from source and upgrade with the supplied python script! It's easier than I anticipated, but there's still a few things you need to watch out for.
Prerequisite Alert! This guide assumes you're already running the latest version from EPEL, installed via `yum`. If not, do that first; it'll set up a few useful things (init scripts, paths and application locations, config structure).
- Stop your existing fail2ban installation:
service fail2ban stop
- Back up your existing fail2ban installation, most likely in
sudo mkdir /etc/f2b-backup && sudo cp -r /etc/fail2ban /etc/f2b-backup
- Head to https://github.com/fail2ban/fail2ban/releases and grab the zip of the latest 0.10.4 release (NB that 0.11 is in development)
- With Python 2 ('python', as opposed to 'python3') installed, extract fail2ban then compile and install:
python setup.py install
- The installer should drop every binary into the correct place, no changing of
However, CentOS7/Fedora changed a few things under the surface from CentOS6; fail2ban's configs are now intended for use on (at least) a CentOS 7/Fedora 19 system. As such, some of things like 'backends' (sources of log files) are defined differently as CentOS7 comes with
systemd instead of
initd. You'll need to make some minor modifications to config files to get F2B running again. I needed to do the following:
To start the SSHD jail, modify
/etc/fail2ban/paths-fedora.conf, commenting out the following lines (courtesy of John Smith on SO):
syslog_backend = systemd
sshd_backend = systemd
dropbear_backend = systemd
proftpd_backend = systemd
pureftpd_backend = systemd
wuftpd_backend = systemd
postfix_backend = systemd
dovecot_backend = systemd
NB: in this specific instance you cannot specify a
paths-fedora.local override file due to how fail2ban loads these core config files, so remember to check the file after your next update from source.
Start fail2ban with
service fail2ban start and you should see it start up. If possible, run a second SSH session so you can watch fail2ban's log as it runs through log files during startup:
tail -fn 100 /var/log/fail2ban.log
Making pesky SSH chancers go away
This week my system has been experiencing botnet-originated SSH drive-by login attempts using dictionary lists of usernames. Only one try per username means it wasn't tripping the normal ban-after-ban-after-three-tries jail for invalid passwords. To mitigate this, I added a new jail called "sshd-invalidusers", based on cweiske's guide on AskUbuntu.
Create the file /etc/fail2ban/filter.d/sshd-invaliduser.conf with the following:
[INCLUDES] before = common.conf [Definition] _daemon = sshd failregex = ^%(__prefix_line)s[iI](?:llegal|nvalid) user .*? from <HOST>(?: port \d+)?\s*$ ^%(__prefix_line)sFailed password for invalid user .*? from <HOST>(?: port \d+)?\s*$ ignoreregex = [Init] journalmatch = _SYSTEMD_UNIT=sshd.service + _COMM=sshd
Then, add the jail to your
/etc/fail2ban/jail.local override file:
[sshd-invaliduser] enabled = true maxretry = 1 port = ssh logpath = %(sshd_log)s backend = %(sshd_backend)s
Always use .local files, not .conf!
While tempting to modify the various .conf files, you're setting yourself up for a fall. Do not directly modify
/etc/fail2ban/jail.conf(or any .conf file), as the default fileset they are overwritten by any future upgrades.
If you are making any modifications to jail.conf, first copy that file to jail.local (
cp jail.conf jail.local) and work off the .local file in future. Same goes for any .conf file, the .local file sitting next to it will always take precedence when fail2ban loads its configs.
A few other useful things
This server runs an Apache VirtualHost multi-domain setup; each site's access, error and combined logs, for both HTTP and HTTPS, get their own file. I also run some management software which handles Apache configurations on a per-user basis, so there's lots of nested folders - first of users, then inside those each user's sites' log files. The structure is
/var/log/httpd/username/sitename-category.log. I find this useful for organisation and monitoring. I can write separately about how I did this if anyone's interested - leave a comment or send me a Tweet.
A few users run WordPress sites, so I'd like to monitor all the logs using the
apache-wp-xmlrpc jails. This is actually easier than you'd think thanks to wildcard syntax...
NB: each jail has an additional action to report repeat abusive traffic to AbuseIPDB using their API (
action_abuseipdb). Remove this line from each jail if you don't have this setup on your system.
[wordpress] enabled = true filter = wordpress logpath = /var/log/httpd/domains/*/*.log port = http,https maxretry = 8 [apache-wp-login] enabled = true port = http,https filter = apache-wp-login logpath = /var/log/httpd/domains/*/*.log maxretry = 6 findtime = 3600 bantime = 36000 action = %(action_)s %(action_abuseipdb)s[abuseipdb_apikey="myapikey", abuseipdb_category="10,18,20,21"] [apache-wp-xmlrpc] enabled = true port = http,https filter = apache-wp-login logpath = /var/log/httpd/domains/*/*.log maxretry = 6 findtime = 3600 bantime = 36000 action = %(action_mwl)s %(action_abuseipdb)s[abuseipdb_apikey="myapikey", abuseipdb_category="10,18,20,21"]
Other good reads
- The Art of Web has a comprehensive guide for 0.8 and 0.9 fail2ban; while some syntax has changed since then the core principles are the same, and there's a good deal of explanation of various options, variables and config structure.
- If your system produces log files with a non-standard datestamp format, consider defining a custom regex pattern in filter.d/common.local... (I didn't need to, YMMV)
- Wondering why fail2ban never bans? Perhaps your regex syntax needs checking, this ServerFault thread may give you some ideas.
- Need to monitor Bind9? ixnfo has a handy step-by-step Bind9 logging guide for use with the
named-refusedjails (UDP and TCP).
Handy things for testing...
iptables -nvLwill show you a tabulated list of all jails with IP addresses. Not specifying
-nwill make iptables resolve every IP's reverse DNS, dramatically slowing down output when you start to get more than a handful of banned IPs.
ignoreipvariable in each jail's config is great to avoid inadvertently banning yourself while testing. I recommend you use it for (at least) SSH.
- Avoid using service
fail2ban restart, instead use
service fail2ban stopthen
service fail2ban start.
- Struggle with iptables? Perturb's IPTables rule generator is very useful.
- Some regex helpers and explainers:
- The automagical http://txt2re.com/ (via kottke.org)
- An example fail2ban regex
- Fail2Ban's official documentation on developing filters
- The very handy RegExr
- The also very handy RegexTester
- David Egan's Dev Notes: overriding fail2ban settings article
Any thoughts, suggestions or errors? Please Tweet me or leave a comment!