Installing a mailserver on Debian 8/9 – Part 1: Introduction

How to install a complete mailserver on Debian 8/9, featuring Postfix, Dovecot, MySQL, Spamassassin, ClamAV, Roundcube and Fail2ban.

~ the howto that actually works ~

Part 1: Introduction
Part 2: Preparations: Apache, Let’s Encrypt, MySQL and phpMyAdmin
Part 3: MTA: Postfix
Part 4: IMAP server: Dovecot
Part 5: Web interface: Roundcube
Part 6: Spam filtering: SpamAsasssin
Part 7: Antivirus: ClamAV and ClamSMTP
Part 8: Quota and other Roundcube settings
Part 9: Using mail with a remote IMAP client (i.e. Thunderbird)
Part 10: Counter brute-force attacks with Fail2ban
Part 11: Sources, config files, colouring and comments

Target audience

These articles are intended for those who want to learn about mailservers on Linux. It covers applications, not hardware and operating system setup. If you’re going to implement a mailserver in a business environment read up on server hardening, backing up and restoring your setup and high availability techniques.

If you follow this article you will end up with a working mailserver and gain some insight in what you are doing, which you can build on to further increase your expertise.

Article structure: topic based layout

My instructions are based on topics, not software. A lot of howtos on Postfix and Dovecot will provide information based on files: here is file A, change this or that. Here is file B, comment out line number twelve. While that may or may not help you set up your mailserver this approach will a) not bring you much insight in what exactly you are doing and b) make troubleshooting a lot harder because mail not getting sent or received can be caused by any number of settings.

In this article series we will set up a basic mailsystem and test if it works. I’ll provide the necessary tools to troubleshoot the setup. Only if it works we’ll build further and after every step we stop, test and troubleshoot if necessary. If something breaks you will know exactly where to look and how to look for it.

Do read the documentation
If you want to understand the components – which you do – you should really read the reading suggestions. Man pages are a good place to start and most of the software we’ll be using has excellent online documentation. You don’t need to learn it all by heart. Reading documentation before installation will get you an impression of what the software is capable of.

Comments

Comments are on the last page of the article. I thought it would be good to see what others have asked before you post your own question.

Requirements

You need a domain name with access to its DNS settings. You’ll be out a fistfull of euro’s.

I assume you’re running on a freshly installed Debian 8 or Debian 9 server with the LAMP stack installed.

There are several approaches to our goal but like we discussed, my approach is to build one component at a time rather than everything at once so you’ll know what you’re doing. Effectively this means you may need to edit the same configuration file more than once and even undo some changes while you’re progressing.

If you’re going to use ClamAV you will need at least 2GB of RAM or it will complain about not being able to allocate memory.

A note on Debian 8 vs. Debian 9

In August 2017 I’ve updated this article to include Debian 9. The most important difference is I switched from MySQL to MariaDB, the way MySQL/MariaDB authenticates local users and a dependency of Roundcube’s Managesieve plugin on a php plugin. Because Roundcube is in Debian 9’s repositories you can ignore the part about backports if you are using Debian 9.

Backports

I’m using Debian 8.3 with all packages from the Debian repositories. At the moment of writing Roundcube is not available in Debian’s default repos but it is from the backports. I’m using the Debian packages because they’re well-maintained (for the software we’ll be using anyway) and because I value stability over newer features.

To enable backports on Debian 8 do:
[code]
# echo deb ftp://ftp.nl.debian.org/debian jessie-backports main contrib non-free > /etc/apt/sources.list.d/backports.list
# aptitude update
[/code]

More on backports: http://backports.debian.org/Instructions/

Test domain

For this series I’m using the fictional domain example.com. Be sure to replace it with your own domain. Also make sure your DNS records are in order for your domain, especially your MX and SPF records. Missing SPF is not a technical problem but a misconfigured SPF record can be.

Where I use your@email.here you should use your own e-mail address. And don’t use the P@ssw0rd password I use in the examples…

Port 25

I’m installing on a consumer connection which I can because my ISP gives me a semi-static IP address. Some providers block outgoing traffic on port 25 by default, and with good reason. Most will open port 25 for outgoing traffic on your connection if you ask them politely.

To prevent spam from being sent from other hosts on my network I closed off port 25 for outgoing traffic on my firewall for all hosts except my mailserver. However that alone may not be enough. Spamhaus lists IP addresses that are infected with certain malware that may have nothing to do with e-mail, such as the Ponmocup trojan downloader. Spamfilters looking at Spamhaus will then mark mail from those addresses as spam so not only must your mailservice be in order, you must also make sure the rest of your network behaves.

Logging and troubleshooting

I like to run my sessions in tmux. Tmux allows running multiple terminal sessions in one SSH connection. The lower third of my screen runs the mail logfile and I keep an eye on the Postfix config with watch -d postconf -n in another pane of tmux. This setup allows for very easy troubleshooting. If you haven’t worked with tmux before now is a good time to start. Its learning curve will be very much worth your while.

To install it do
[code]# aptitude install tmux[/code]

More on tmux: https://en.wikipedia.org/wiki/Tmux and
[code]
$ man tmux
[/code]

Where to get clues for troubleshooting
Logfiles are your starting point for troubleshooting. If things don’t work, check your logs. By convention log files go into /var/log/ on Debian (based) systems. Many programs allow you to set a custom log file path so if you can’t find its logfile see if you can find its location in the program’s configuration files. If a logfile isn’t created at all see if the account that should create/add to it has permissions to do so.

I like to keep logfiles running while testing so I can see what’s going on. To do so do
[code]
# tail -f /var/log/mail.log
[/code]
for example. You can stop it with Ctrl + C.

Furthermore Postfix can show you these things (from postconf’s man page):
postconf -p Show main.cf parameter settings. This is the default. [Use in conjunction with grep.]
postconf -n Show only configuration parameters that have explicit name=value settings in main.cf. Specify -nf to fold long lines for human readability

Dovecot has a similar feature (from doveconf’s man page):
doveconf -a Show all settings with their currently configured values.
doveconf -d Show the setting’s default value instead of the one currently configured.
doveconf -n Show only settings with non-default values.

Both postconf and doveconf have more options; read their respective man pages.

—

I wish you a lot of fun following this howto, and don’t forget to colour the colouring picture in the last installment!

On this page