Installing a mailserver on Debian 8/9 – Part 6: Spamfiltering: SpamAssassin

How to install a complete mailserver on Debian 8/9, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ the howto that actually works ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

Installing Spamassassin

[code]
# aptitude install spamassassin spamc
[/code]

Create a user for spamc. From the spamc man page: “Spamc is the client half of the spamc/spamd pair. It should be used in place of “spamassassin” in scripts to process mail. It will read the mail from STDIN, and spool it to its connection to spamd, then read the result back and print it to STDOUT. Spamc has extremely low overhead in loading, so it should be much faster to load than the whole spamassassin program.”

[code]
# adduser spamd –disabled-login
[/code]
Just press enter at each question.

Have Spamassassin start on boot:
[code]
# systemctl enable spamassassin.service
[/code]

Have Spamassassin update its rules automatically every night. In /etc/default/spamassassin:
[code]
CRON=1
[/code]

Configuration of Spamassassin is done in /etc/spamassassin/local.cf. The defaults are ok. After the filter has had a chance to learn what is spam and what is not (say, after two weeks of use) add

[code]
use_bayes_rules 1
[/code]
to /etc/spamassassin/local.cf. This will have Spamassassin actually use Bayesian the rules it learned.

Info: http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Conf.html

Start the spam filter:
[code]
# service spamassassin start
[/code]

Tell Postfix we’re running a content filter. In /etc/postfix/master.cf find the line that reads

[code]
smtp inet n – – – – smtpd
[/code]

and directly underneath it add:
[code]
-o content_filter=spamassassin
[/code]

Note that the line beginning with -o must start with one more whitespaces. My Syntax Highlighter seems to be stripping them off. If it doesn’t, reloading Postfix will say: “/usr/sbin/postconf: fatal: invalid type field”. (Again, countless hours of fun.)

When reviewing this article I found this didn’t work. What I did wrong was I added the line not under
[code]
smtp inet n – – – – smtpd
[/code]
but under
[code]
submission inet n – y – – smtpd
[/code]
It’s an easy mistake to make because of the file’s layout.

At the end of /etc/postfix/master.cf add:
[code]
spamassassin unix – n n – – pipe
user=spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}
[/code]

Again note that the second line must start with one or more whitespaces.

[code]
# service spamassassin start
# postfix reload
[/code]

Once you’ve installed and configured Postfix send a test mail to and from your account in Roundcube and make sure it still works.
If it doesn’t, remove the newest entries from /etc/postfix/master.cf, restart Postfix and try again. See if you have made a typo and make sure restarting Postfix does not throw any errors. Only if sending and receiving works continue.

Testing the spamfilter

Send a mail to your recipient with the GTUBE. The GTUBE is EICAR for spamfilters: it is a special string of characters which spamfilters will recognize as spam for the purpose of testing. More on the GTUBE: http://spamassassin.apache.org/gtube/

So send a mail with this string in its body:
[code]
XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X
[/code]

Open the mail from Roundcube. It should start with:
Spam detection software, running on the system “mail.localdomain”,
has identified this incoming email as possible spam.

The same mail not containing the GTUBE should not contain that warning.

I like to see the original spam message, not a message containing the spam mail as attachment so I set Spamassassin to tag the mail header and not put it inside an other mail as attachment. In /etc/spamassassin/local.cf change:
[code]
report_safe 0
[/code]

[code]
# service spamassassin reload
[/code]

Send another GTUBE mail, open it in Roundcube and view its source.

View the e-mail's source — View the e-mail’s source

It should read:
[code]
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mail.localdomain
X-Spam-Flag: YES
X-Spam-Level: **************************************************
X-Spam-Status: Yes, score=1000.0 required=5.0 tests=GTUBE,HTML_MESSAGE
autolearn=no autolearn_force=no version=3.4.0
X-Spam-Report:
* 1000 GTUBE BODY: Generic Test for Unsolicited Bulk Email
* 0.0 HTML_MESSAGE BODY: HTML included in message
[/code]

Moving spam to the Junk folder automatically

This is actually part of something larger, namely a mail filtering system. Using filters (also known as rules) you can apply actions to mail using logic, for example move mail that’s tagged as spam to the Junk folder.

Automatically subscribe users to the special IMAP folders

First we need to make sure the IMAP client is subscribed to the folder the spam will be moved to.

If you open Roundcube you’ll notice that there are two folders: Inbox and Sent. I like to automatically subscribe users to a bunch of special IMAP folders so they’ll always be there. Open up /etc/dovecot/conf.d/15-mailboxes.conf and have it read:

[code]
namespace inbox {
mailbox Drafts {
special_use = \Drafts
auto = subscribe
}
mailbox Junk {
special_use = \Junk
auto = subscribe
}
mailbox Trash {
special_use = \Trash
auto = subscribe
}
mailbox Archive {
special_use = \Archive
auto = subscribe
}
mailbox Sent {
special_use = \Sent
auto = subscribe
}
mailbox "Sent Messages" {
special_use = \Sent
}
}
[/code]

Apart from copying and pasting the above text also read the commentary in that file. It’s quite useful.

Do
[code]
# dovecot reload
[/code]

and log out and back in to Roundcube because the special folders are created at login.

Automatically subscribed to special folders

Sieve

Dovecot has a plugin called Sieve that handles rules. Sieve actually is a programming language designed for mail filtering rules.

Let’s enable it.
In /etc/dovecot/conf.d/15-lda.conf:

[code]
protocol lda {
mail_plugins = $mail_plugins sieve
}
[/code]

in /etc/dovecot/conf.d/20-lmtp.conf:
[code]
protocol lmtp {
mail_plugins = $mail_plugins sieve
postmaster_address = tinus@example.com
}
[/code]

It’s a good idea to have a local postmaster address to prevent mail loops. Also it is required by the smtp rfc (though we’re technically not using smtp but lmtp here – still it’s a good idea).

In /etc/dovecot/conf.d/90-sieve.conf:
[code]
plugin {
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
}
[/code]

[code]
# service dovecot restart
[/code]

Test if Sieve is running:
[code]
# telnet localhost 4190
[/code]

If you get
[code]
Connected to localhost.
Escape character is ‘^]’.
"IMPLEMENTATION" "Dovecot Pigeonhole"
"SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave"
"NOTIFY" "mailto"
"SASL" "PLAIN LOGIN"
"VERSION" "1.0"
OK "Dovecot ready."
[/code]

then Sieve is working. Tap Enter three times to exit.

If you get
[code]
Trying 127.0.0.1…
telnet: Unable to connect to remote host: Connection refused
[/code]

then Sieve is not working. Check your settings, restart the Dovecot service and try again.

Do
[code]
doveconf -n
[/code]
to check for typos and such.

It is possible to create default sieve rules. This has one huge drawback: as soon as the user creates her own sieve rules (for example she sets an out of office autoreply) the default rules are no longer carried out even after deleting the personal rule.

Set up the spam rule globally

We’ll set up a global spam rule that will move spam to the user’s Junk folder automatically.

In /etc/dovecot/conf.d/90-sieve.conf set:
[code]
sieve_after = /etc/dovecot/sieve/spamfilter.sieve
[/code]
The above folder and filename are arbitrary but they seem logical to me.

Note: the sieve_global_dir directive is not related to this feature.

Why sieve_after? If you’d use sieve_before then any applicable (to that particular mail) rules the user would have would not be executed. By using sieve_after the user’s rules are executed first so they can override the sieve_after rules.

Create the folder:
[code]
mkdir /etc/dovecot/sieve
[/code]

In /etc/dovecot/sieve/spamfilter.sieve:
[code]
require ["fileinto"];
# rule:[SPAM]
if header :contains "X-Spam-Level" "*" {
fileinto "Junk";
}
[/code]

This rule checks for a header called ‘X-Spam-Level’. If its value contains * then the message is stored in the folder called Junk. Note that ‘**’ and ‘***’ also contain ‘*’ so you can set the amount of spam-certainty required for a message to be regarded spam. The scale is 1 to 50, 50 being the most certain.

Reload Dovecot and send a test message containing the GTUBE string to check if it works.

Automatically delete spam after 30 days

Using doveadm we can automatically delete all mail in the Junk folder older than a certain time, for example 30 days.

Doveadm is Dovecot’s administration utility. Read man doveadm for more information on doveadm. Other relevant reading for this case: man doveadm-search and man doveadm-search-query.

Let’s say you want to search all documents in all inboxes older than four hours:
[code]
# doveadm search -A mailbox Inbox savedbefore 4h
tinus@example.com 74b4db2757f9f05661350000595b2a1f 1
tinus@example.com 74b4db2757f9f05661350000595b2a1f 2
[/code]

-A means to look in all mailboxes, not just the one we specify. Remember the iteration query? This is what that is good for. Had you been using a static user query in /etc/dovecot/dovecot-sql.conf.ext then doveadm search -A would have thrown an error.

We don’t want to search the Inbox however and we want mail older than 30 days:
[code]
# doveadm search -A mailbox Junk savedbefore 30d
tinus@example.com 4adeeb2ca5c1ca565c3b000038e0142a 1
tinus@example.com 4adeeb2ca5c1ca565c3b000038e0142a 2
tinus@example.com 4adeeb2ca5c1ca565c3b000038e0142a 3
[/code]

[code]
$ man doveadm-search-query
[/code]
tells you more about the query format.

We don’t just want to search but actually delete the mails. (Do use search first to verify you get the results you were expecting.)

[code]
# doveadm expunge -A mailbox Junk savedbefore 30d
[/code]

You don’t want to run this manually every day. Let’s schedule it:
[code]
# crontab -e
[/code]

Add this line:
[code]
@hourly /usr/bin/doveadm expunge -A mailbox Junk savedbefore 30d
[/code]

If you don’t want the cronjob’s output mailed to you change to
[code]
@hourly /usr/bin/doveadm expunge -A mailbox Junk savedbefore 30d > /dev/null 2>&1
[/code]

This will run the command every hour. Change to
[code]
@daily /usr/bin/doveadm expunge -A mailbox Junk savedbefore 30d > /dev/null 2>&1
[/code]

To run it once every day.