Creating fail2ban rules for HTTP error 404 (page not found) for popular web frontends: Nginx, HAProxy, Apache2.
First of all you can check how to create custom rules article.
Nginx
Let’s create rule for Nginx HTTP error 404.
1. Log line:
123.44.55.66 - - [18/Jan/2019:10:16:53 +0000] GET /index.asp HTTP/1.1 "404" 1065 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" "-"
2. Filter
Create filter.d/nginx-404.conf:
[Definition] # depending on your log format failregex = - - \[.*\] "(GET|POST|HEAD).*HTTP.* 404 #failregex = ^ - - .*(GET|POST) .* "(404|444|403|400)" .*$
test it:
fail2ban-regex /var/log/nginx/access.log /etc/fail2ban/filter.d/nginx-404.conf
3. Rule
Create jail.d/nginx.conf:
[nginx-404] enabled = true port = http,https filter = nginx-404 logpath = /var/log/nginx/*.log maxretry = 7 findtime = 300 bantime = 600
HAProxy
1. Log line:
Jan 18 14:26:10 proxy haproxy[25781]: 123.44.55.66:52886 [18/Jan/2019:14:26:10.569] www-https~ http-backend/web-01 0/0/1/1/2 404 454 - - --NN 6/5/0/1/0 0/0 "GET /index.asp HTTP/1.1"
2. Filter
Create file filter.d/haproxy-404.conf
[Definition] failregex = .*:<HOST>(?::\d+)?\s+.*\d* 404
test it:
fail2ban-regex /var/log/haproxy.log /etc/fail2ban/filter.d/haproxy-404.conf
3. Rule
Create file jail.d/haproxy-404.conf
[haproxy-404] enabled = true port = http,https filter = haproxy-404 logpath = /var/log/haproxy.log maxretry = 10 findtime = 300 bantime = 600
Apache2
1. Log line
123.44.55.66 - - [19/Jan/2019:06:33:01 +0000] "GET /apple-touch-icon-precomposed.png HTTP/1.1" 404 23818 "-" "MobileSafari/604.1 CFNetwork/974.2.1 Darwin/18.0.0"
2. Filter
Bonus! I want to add some ignore regular expressions to exclude errors like above (with icons).
[Definition] failregex = (?P<host>[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) .+ 404 [0-9]+ " ignoreregex = /apple-touch-icon.* /favicon\.ico
Test it with ignore regex as third parameter to fail2ban-regex:
fail2ban-regex --print-all-matched /var/log/apache2/access.log /etc/fail2ban/filter.d/apache-404.conf /etc/fail2ban/filter.d/apache-404.conf
3. Rule
[apache-404] enabled = true port = http,https filter = apache-404 logpath = /var/log/apache2/*.log maxretry = 12 findtime = 300 bantime = 600
See also: How to re-ban hackers.