How to install a complete mailserver on Debian 8/9, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.
~ the howto that actually works ~
Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments
On this page
Installation
Block random address spammers
Users entering a wrong username/password combination for x times
Comments are on the last page.

Installation
Fail2ban reads logfiles and acts based on their entries. For example, it can recognize when someone has entered a wrong password six times in two minutes and lock them out for half an hour.
[code]
# aptitude install fail2ban
[/code]
Have Fail2ban start automatically at boot:
[code]
# systemctl enable fail2ban.service
[/code]
Copy the conf file to a local differential file:
[code]
# cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
[/code]
In /etc/fail2ban/jail.local set the following values:
[code]
backend = polling
[/code]
(Alternatively set to auto but that made Fail2ban complain about pyinotify not being installed. That’s not problem: Fail2ban just tries all options but I don’t like complaints in my logfiles.)
[code]
mta = sendmail
destemail = your@email.here
[/code]
Set action:
action_ = just ban
action_mw = ban and mail
action_mwl = ban and mail with whois report and relevant log lines
I suggest _mwl but change it to your needs.
[code]
action = %(action_mwl)s
[/code]
Start Fail2ban:
[code]
# fail2ban-client start
[/code]
If it was already running:
[code]
# service fail2ban restart
[/code]
You should receive a mail notification at your specified e-mail address that Fail2ban has started.
Block random address spammers
Some spammers/phishers send mail to common e-mail addresses (john@; admin@) in the hope these addresses exist. If a sender sends mail to a bunch of non-existant addresses at the same time you may as well stop accepting mail from that sender.
In /etc/fail2ban/filter.d/postfix.conf add these lines under failrexeg:
[code]
reject: RCPT from (.*)\[<HOST>\]: 550 5.1.1
reject: RCPT from (.*)\[<HOST>\]: 450 4.7.1
reject: RCPT from (.*)\[<HOST>\]: 554 5.7.1
[/code]
(Source: http://www.fail2ban.org/wiki/index.php/Postfix)
in /etc/fail2ban/jail.local under [Postfix] set
[code]
enabled = true
[/code]
Users entering a wrong username/password combination for x times
This can indicate a brute-force attack. It’s up to you to decide if you want to use it. Personally I like to use it but set the limit higher than the default by adding
[code]
maxretry = 10
[/code]
to the jail definition in /etc/fail2ban/jail.local.
For Roundcube:
In /etc/fail2ban/jail.local under [roundcube-auth] set
[code]
enabled = true
[/code]
Also set
[code]
logpath = /var/log/roundcube/errors
[/code]
In /etc/fail2ban/filter.d/roundcube-auth.conf set:
[code]
failregex = IMAP Error: (FAILED login|Login failed) for .*? from <HOST>
[/code]
Likely someone will come up with a better regex to identify logon failures but for me this works.
For other imap clients:
In /etc/fail2ban/jail.local under [dovecot] set
[code]
enabled = true
[/code]
Afterwards do
[code]
# service fail2ban reload
[/code]
To manually unban a client do
[code]
# fail2ban-client set owncloud unbanip 192.168.1.2
[/code]
To manually check the Fail2ban’s ownCloud jail:
[code]
# fail2ban-client status roundcube-auth
# fail2ban-client status dovecot
# fail2ban-client status postfix
[/code]