Creating fail2ban rules for HTTP error 404 (page not found) for popular web frontends: Nginx, HAProxy, Apache2.

First of all you can check how to create custom rules article.

Nginx

Let’s create rule for Nginx HTTP error 404.

1. Log line:

123.44.55.66 - - [18/Jan/2019:10:16:53 +0000] GET /index.asp HTTP/1.1 "404" 1065 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko" "-"

2. Filter

Create filter.d/nginx-404.conf:

[Definition]
# depending on your log format
failregex = - - \[.*\] "(GET|POST|HEAD).*HTTP.* 404
#failregex = ^ - - .*(GET|POST) .* "(404|444|403|400)" .*$

test it:

fail2ban-regex /var/log/nginx/access.log /etc/fail2ban/filter.d/nginx-404.conf

3. Rule

Create jail.d/nginx.conf:

[nginx-404]
enabled  = true
port     = http,https
filter   = nginx-404
logpath  = /var/log/nginx/*.log
maxretry = 7
findtime = 300
bantime = 600

HAProxy

1. Log line:

Jan 18 14:26:10 proxy haproxy[25781]: 123.44.55.66:52886 [18/Jan/2019:14:26:10.569] www-https~ http-backend/web-01 0/0/1/1/2 404 454 - - --NN 6/5/0/1/0 0/0 "GET /index.asp HTTP/1.1"

2. Filter

Create file filter.d/haproxy-404.conf

[Definition]
failregex = .*:<HOST>(?::\d+)?\s+.*\d* 404

test it:

fail2ban-regex /var/log/haproxy.log /etc/fail2ban/filter.d/haproxy-404.conf

3. Rule

Create file jail.d/haproxy-404.conf

[haproxy-404]
enabled  = true
port     = http,https
filter   = haproxy-404
logpath  = /var/log/haproxy.log
maxretry = 10
findtime = 300
bantime = 600

Apache2

1. Log line

123.44.55.66 - - [19/Jan/2019:06:33:01 +0000] "GET /apple-touch-icon-precomposed.png HTTP/1.1" 404 23818 "-" "MobileSafari/604.1 CFNetwork/974.2.1 Darwin/18.0.0"

2. Filter

Bonus! I want to add some ignore regular expressions to exclude errors like above (with icons).

[Definition]
failregex = (?P<host>[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) .+ 404 [0-9]+ "
ignoreregex = /apple-touch-icon.*
              /favicon\.ico

Test it with ignore regex as third parameter to fail2ban-regex:

fail2ban-regex --print-all-matched /var/log/apache2/access.log /etc/fail2ban/filter.d/apache-404.conf /etc/fail2ban/filter.d/apache-404.conf

3. Rule

[apache-404]
enabled  = true
port     = http,https
filter   = apache-404
logpath  = /var/log/apache2/*.log
maxretry = 12
findtime = 300
bantime = 600

See also: How to re-ban hackers.