This article describes how to install ASSP on Ubuntu Server 14.04 LTS.

If you find it useful please consider making a donation by clicking the PayPal button on the right or just click some ads here. Thanks!

*** An update to this article is available here: ASSP on Debian 8/Ubuntu 16.04. ***
Part 1: Introduction
Part 2: Installation
Part 3: Managing the server
Part 4: Working with ASSP
Part 5: Troubleshooting
Sources

Part 1: Introduction

ASSP is an open source spam filter that can be deployed as a relaying proxy between a mail server and the internet. ASSP is not an MTA – it cannot send mail itself. We’ll be using a locked-down version of Postfix as a local MTA however any MTA will do, for example your isp’s relay server or Microsoft Windows Server SMTP service.

Strategies
ASSP uses a growing set of spam filtering strategies that can each either block an incoming e-mail entirely or assign a score to it. In the end of the array of strategies all scores are added up and actions can be based on it or the end users may set their own filter actions. For example: discard any mail with an 80% or higher spam probability but put lower scoring spam in a dedicated folder. Don’t do anything with 20% or less scoring mail.

An autowhitelisting system prevents incoming mail senders from being scanned for spam so once a user has sent an e-mail to an external contact that contact will never be considered spam. Outgoing mail is added to the Bayesian and Markov statistical corpuses. Once the Bayesian and Markov databases have acquired enough data they can quite reliably distinguish spam from ham.

Virusscanner
ClamAV or any local virusscanner can be integrated to scan mail for viruses.

Web interface
A web interface allows configuring and managing ASSP. It offers extensive statistics, insight in smtp traffic and an automated investigation page to check wether a certain e-mail would or would not be considered spam and if so why exactly.

And more
It offers dnsbl methods, uribl, backscatter detection, an e-mail based end-user spam reporting interface, virusscanning, very extensive header analitics, attachment blocking, supports regular expressions for all values, optionally file based lists, greylisting, spf, dmarc, srs, Helo and sender validation, and more.

ASSP is a full featured, highly customizable, enterprise strength spamfilter.

About this page and my site
I sincerely apologize for the way I had to maim my screenshots by watermarking them. Other articles from my site have been blatantly copied and pasted on other sites without any credit. If I ever catch whoever does this I’ll personally come to their house and reset their router.

Network environment

We’ll be installing ASSP in a network with the example.com. dns suffix. The mailserver is Domino 6.5. Yeah, I know this is ancient. Send me screenshots and I’ll put them up. Since ASSP will be deployed as a relay server any mailserver will do.

Domain name: example.com.

function	IP address	hostname
ASSP spamfilter	192.168.1.36	ASSP
Domino mailserver	192.168.1.10	SERVER10
Secondary mailserver	192.168.1.3	SERVER03
primary dns server	192.168.1.12	SERVER12
secondary dns server	192.168.1.13	SERVER13
backup server	192.168.1.17	SERVER17

Part 2: Installation

Mail flow

Incoming mail is received by the spamfilter. The filter divides spam from ham and forwards one or both to the mailserver. The mailsever redistributes the mail to the receivers’ mailboxes.
Outgoing mail is added to the Bayesian and Markov statistiscal corpuses and the whitelist.

Operating system: Ubuntu Server 14.04.03 LTS

In theory any system supporting Perl is able to host ASSP. I’ve ran it on Windows, Linux and FreeBSD. In practice Linux is by far the easiest system to set it up on.

ActiveState produces the commercial ActivePerl for Microsoft Windows. The Business Edition costs US$ 999 per year. They make a Community Edition which is crippled and in my opinion unusable for ASSP. The Enlightened Perl Organisation creates a free of charge Strawberry Perl. Last time I tried it it was incomplete and not very compatible with CPAN. That may have improved and by all means try and run it on Windows if you like. I invite you to share your experience in the comment section under this article.

Ubuntu 14.04 LTS supports Perl out of the box. Both are free of charge. Canonical offers commercial support packages for Ubuntu; community support is provided free of charge and generally very good. Ubuntu 14.04 LTS is guaranteed to receive support until the end of 2019 and is widely regarded to be an industry standard.

The version we’re installing is Ubuntu 14.04.03 “Trusty” LTS 64bit.

On installation I chose:

• Basic Ubuntu server
• OpenSSH server
• Samba file server

If at any point you need to add or remove parts of the installation run
[code]# tasksel[/code]

We’ll use Samba for easy access from Windows clients to the rather file-based layout of the ASSP installation. It isn’t necessary for ASSP management, merely a convenience.

Upon completion of the installation run
[code]# aptitude update && aptitude dist-upgrade[/code]

MTA: Postfix

Postfix is our MTA of choice because it’s easy to install, old and proven and safe. We’ll have it listen for outgoing mail on port 125 so we can install it on the same machine ASSP will be running on. To install:
[code]# aptitude install postfix[/code]

When asked choose the default Internet Site configuration.

When asked for the System mail name fill out your domain name. Like the installer sais: if a mail address on the local host is foo@example.org, enter example.org here.

In the file /etc/postfix/master.cf find:
[code]smtp inet n – – – – smtpd[/code]

and change it to:
[code]125 inet n – – – – smtpd[/code]

Check Postfix’s maximum message size setting:
[code]# postconf | grep message_size_limit[/code]

Keep in mind that this size will be composed of the header size + body size + attachment size + anything else that is sent along with the mail. If you need to be able to send attachments up to 20MB in size set message_size_limit to something a bit higher than 20MB, for example 25MB.

In the file /etc/postfix/main.cf change the following value (or add it if it doesn’t exist):
message_size_limit = 26214400

Secure so only the ASSP server may use Postfix: in main.cf change mynetworks to:
[code]mynetworks = 127.0.0.0/8 192.168.1.36/32 [::ffff:127.0.0.0]/104 [::1]/128[/code]

at the end of the file add:
[code]smtpd_client_restrictions = permit_mynetworks, reject
smtpd_delay_reject = no
transport_maps = hash:/etc/postfix/transport[/code]
Look for the value ‘mydestination’ and remove your mail domain:
From
[code]
mydestination = example.com, assp.testnet.lab, localhost.testnet.lab, localhost
[/code]
To:
[code]
mydestination = assp.testnet.lab, localhost.testnet.lab, localhost
[/code]

Create the file /etc/postfix/transport and add to it:
[code]example.com smtp:192.168.1.10[/code]

Load the transport file in Postfix and reload Postfix:
[code]# postmap transport
# postfix reload[/code]

Virus scanner: ClamAV

Install ClamAV by running
[code]# aptitude install clamav clamav-daemon
# freshclam[/code]

Edit ClamAV’s config file /etc/clamav/clamd.conf;
alternatively run dpkg-reconfigure:
[code]# dpkg-reconfigure clamav-base[/code]

Socket type: UNIX (TCP will probably also work but it says a socket file is safer)
Local (UNIX) socket clamd will listen on: /var/run/clamav/clamd.ctl (this is the default at the time of writing. Just make sure you enter this exact value in the ASSP web interface under ClamAV and FileScan > Port or file socket for ClamAV (AvClamdPort).
User to run clamav-daemon as: clamav
Groups for clamav-daemon: clamav

Make sure FreshClam is scheduled to download new virus definitions periodically. Check /etc/clamav/freshclam.conf or configure it in Webmin.

You can have freshclam check for db updates every hour (Checks 24). If you set the following values in /etc/clamav/freshclam.conf you may check up to four times per hour (Checks 96):
[code]DNSDatabaseInfo current.cvd.clamav.net
DatabaseMirror db.XY.clamav.net
DatabaseMirror database.clamav.net[/code]
Replace XY with your country code.

Restart the Freshclam service:
[code]# service clamav-freshclam restart[/code]

Source: http://www.clamav.net/doc/cvd.html

Optional webbased management tool: Webmin

It’s always an option to manage the server with Webmin. Webmin is a webbased administration tool for servers. It runs fine on Ubuntu. Create the file /etc/apt/sources.list.d/webmin.list and add:
[code]deb http://download.webmin.com/download/repository sarge contrib
deb http://webmin.mirror.somersettechsolutions.co.uk/repository sarge contrib
[/code]
Download and install the repo’s key:
[code]
# cd /root
# wget http://www.webmin.com/jcameron-key.asc
# apt-key add jcameron-key.asc
[/code]
Update the repository cache and install:
[code]
# aptitude update
# aptitude install webmin[/code]

Source: http://www.webmin.com/deb.html

It is not necessary for the administration of the ASSP server to run Webmin. You can manage the server more completely from the command line but sometimes Webmin is just handy. Webmin is completely optional.

You will find Webmin on port 10000. Use your own credentials to log in.

Perl modules

ASSP’s readme file states that the preferred way to install the Perl modules is to use native packages where possible to prevent unnecessary complications with upgrading.
These are native Ubuntu packages that provide some of the Perl modules used by ASSP:

• libnet-dns-perl
• libauthen-sasl-perl
• libmail-spf-perl
• libregexp-optimizer-perl
• libfile-readbackwards-perl
• libnetaddr-ip-perl
• libnet-cidr-lite-perl
• libmail-dkim-perl
• libnet-ldap-perl
• libunicode-string-perl
• libemail-send-perl
• libemail-mime-perl
• libtext-unidecode-perl
• liblingua-stem-snowball-perl
• libsys-cpu-perl
• libthreads-perl
• libschedule-cron-perl
• libdigest-sha-perl
• libmime-types-perl
• libclamav-client-perl
• libarchive-zip-perl
• libberkeleydb-perl
• liblingua-identify-perl
• libsys-cpuload-perl
• libthreads-shared-perl
• libunicode-linebreak-perl
• libconvert-tnef-perl
• libimage-magick-perl

Install them by doing
[code]# aptitude install libnet-dns-perl libauthen-sasl-perl libmail-spf-perl libregexp-optimizer-perl libfile-readbackwards-perl libnetaddr-ip-perl libnet-cidr-lite-perl libmail-dkim-perl libnet-ldap-perl libunicode-string-perl libemail-send-perl libemail-mime-perl libtext-unidecode-perl liblingua-stem-snowball-perl libsys-cpu-perl libthreads-perl libschedule-cron-perl libdigest-sha-perl libmime-types-perl libclamav-client-perl libarchive-zip-perl libberkeleydb-perl liblingua-identify-perl libsys-cpuload-perl libthreads-shared-perl libunicode-linebreak-perl libconvert-tnef-perl libimage-magick-perl[/code]

Install more Perl modules:
(The first time you run cpan it will ask you some configuration questions. If in doubt just follow its lead. Press Enter to accept the suggested answer but do read the questions.)
[code]# cpan Text::Glob Number::Compare Compress::Zlib Convert::TNEF Digest::MD5 Digest::SHA1 Email::MIME::Modifier Email::Send Email::Valid File::ReadBackwards File::Scan::ClamAV LWP::Simple MIME::Types Mail::SPF Mail::SRS Net::CIDR::Lite Net::DNS Net::IP::Match::Regexp Net::LDAP Net::SMTP Net::SenderBase Net::Syslog PerlIO::scalar threads threads::shared Thread::Queue Thread::State Tie::DBI Time::HiRes Schedule::Cron IO::Socket::SSL Crypt::CBC Crypt::OpenSSL::AES DBD::mysql YAML DateTime Time::Format Module::Signature File::PathInfo File::Find::Rule File::Slurp File::Which LEOCHARRE::DEBUG File::chmod Linux::usermod LEOCHARRE::CLI Crypt::RC4 Text::PDF Smart::Comments CAM::PDF PDF::API2 PDF::Burst PDF::GetImages Image::OCR::Tesseract PDF::OCR PDF::OCR2 Mail::DKIM::Verifier Mail::SPF::Query Regexp::Optimizer Unicode::GCString Text::Unidecode Lingua::Stem::Snowball Lingua::Identify Archive::Zip[/code]

Sys::MemInfo will fail to install. The solution is to install build-essential first:
[code]# apt-get -y install libgd2-xpm-dev build-essential[/code]

Then install these modules:
[code]# cpan Sys::MemInfo Sys::CPU Sys::CpuAffinity Tie::RDBM Thread::State Digest::SHA1[/code]

Mail::SPF::Query

Mail::SPF::Query fails to install. I’m not sure it hasn’t been superseded by libmail-spf-query and ASSP just does a wrong check. In that case forcibly installing it from the CPAN prompt would fix ASSP’s misinterpretation of the facts:
[code]# cpan
cpan[1]> force install Mail::SPF::Query[/code]

If ASSP *is* correct however this would be an unholy route. In my setup it works. Do the force install and if it works it works. **Update** According to Silvenga in the comments forcing it is the way to go.

File::Scan::ClamAV

File::Scan::ClamAV fails to install because the tests it does are outdated. You can either force install it:
[code]
# cpan
cpan[1]> force install File::Scan::ClamAV
[/code]

Or change the tests as per http://www.gossamer-threads.com/lists/clamav/users/61844:

As root:
[code]# cpan
# install File::Scan::ClamAV[/code]
(this will fail)

[code]# look File::Scan::ClamAV[/code]
(this puts you in the build directory for this module within cpan) e.g.:

[code]cpan[2]> look File::Scan::ClamAV
Running look for module ‘File::Scan::ClamAV’

Trying to open a subshell in the build directory…
Working directory is /root/.cpan/build/File-Scan-ClamAV-1.91-O_dSGE
xxxx [at] xxx:.cpan/build/File-Scan-ClamAV-1.91-O_dSGE#[/code]

[code]#step 2
cpan[2]> cd t[/code]
(go into the test directory)

in mkconf.pl change:
[code]cpan[2]> my $dir = cwd;[/code]
to
[code]cpan[2]> my $dir = ‘/tmp’;[/code]
(Note: in the ‘gossamer’ link the previous line contains an error.)

#step 3
In 01ping.t, 02reload.t, 03quit.t, 04scan.t and 05streamscan.t files:
1) change all references to clamsock to /tmp/clamsock (should be 3
places in each file: in the loop, during File::Scan::ClamAV instantiation and unlink)
[code]cpan[2]> sed -i — ‘s|clamsock|’/tmp/clamsock’|g’ *.t
cpan[2]> cd ..
cpan[2]> make test
cpan[2]> make install[/code]

ASSP

Download the latest ASSP__install.zip from http://sourceforge.net/projects/assp/.
[code]$ wget -O assp.zip\
http://sourceforge.net/projects/assp/files/latest/download[/code]

Install unzip:
[code]# aptitude install unzip[/code]

Extract to /usr/share/assp/. You could put it anywhere but I’m using this path.
[code]# unzip -d /usr/share assp.zip[/code]

You can delete four textfiles in the zipfile’s root or keep them for reference. Anyway they don’t need to stay in /usr/share.

The folder structure will need to end up like this:
[code]usr
+- share
+- assp
+- certs
+- dkim
+- docs
+- files
+- blackre.txt
+- blockreportuser.txt
etc[/code]

Make the Perl scripts executable:
[code]# chmod +x /usr/share/assp/*.pl[/code]

Create a dedicated system user for assp:
[code]# useradd assp -r[/code]

You can pick your own accountname but make sure it is properly reflected in ASSP’s web interface under [Server Setup] ‘Run as UID’ and ‘Run as GID’.

Vipul’s Razor plugin

Vipul’s Razor checks checksums of incoming mail against an online database.

Note: version numbers may vary. Substitute where appropriate.

From the SourceForce ASSP site > ASSP V2 multithreading > razor2_for_assp download razor-agents.src.tar.gz. Untar:
[code]
# tar -xzvf razor-agents_src.tar.gz
[/code]

Compile the Perl module:
[code]
# cd razor-agents-2.84_assp
# perl Makefile.PL
# make
# make test
# make install
[/code]

If the making and installing went well, from the SF ASSP site > ASSP V2 multithreading > Plugins, download ASSP_Razor – Razor2 Plugin for ASSP. The file is called ASSP_Razor_1.09.zip.

Extract the file and move the resulting .pm file to /usr/share/assp/plugins:
[code]
# unzip ASSP_Razor_1.09.zip
# mv ASSP_Razor.pm /usr/share/assp/plugins
[/code]

Restart the ASSP server:
[code]
# service assp restart
[/code]

You may now configure the Razor plugin from the web interface.

Runlevel scripts

Create a file /etc/init.d/assp and make it executable.
[code]
#!/bin/sh -e
# Start or stop ASSP
#
# original version by Ivo Schaap had issues on Debian4.
# Modified by atramos.
# 2015-09-01 some error correction by Martin Huijgen
#
### BEGIN INIT INFO
# Provides: ASSP (Anti-Spam SMTP Proxy)
# Required-Start: $syslog, $local_fs
# Required-Stop: $syslog, $local_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start ASSP
# Description: Enable service provided by daemon.
### END INIT INFO
PATH=/bin:/usr/bin:/sbin:/usr/sbin
case “$1” in
start)
printf “Starting the Anti-Spam SMTP Proxy\n”
cd /usr/share/assp
perl assp.pl 2>&1 > /dev/null &
;;
stop)
printf “Stopping the Anti-Spam SMTP Proxy\n”
kill -9 `ps ax|grep “perl assp.pl”|grep -v grep|awk ‘{ print $1 }’`
;;
restart)
$0 stop || true
$0 start
;;
*)
printf “Usage: /etc/init.d/assp {start|stop|restart}\n”
exit 1
;;
esac
exit 0[/code]

[code]# chmod +x /etc/init.d/assp[/code]

Have it start AFTER clamav-daemon (check back here later if you haven’t installed ClamAV yet). Look in /etc/rc2.d and look at the number before clamav-daemon. Let’s say this is 20: ‘S20clamav-daemon’ –> ../init.d/clamav-daemon

Give assp a higher number than clamav-daemon, for example 30: ‘S30assp’. Then run update-rc.d:
[code]# update-rc.d assp defaults 30[/code]

Web interface configuration

You can find the web interface on http://192.168.1.36:55555. The default user is root and the default password is nospam4me.

You may change the settings either in de web interface or the config file (/usr/share/assp/assp.cfg). The web interface’s advantage is that is contains a lot of explanatory text so for first time users I would recommend to use the web interface. After changing the values click the Apply Changes button bottom right.

[code]
[Network Setup]
SMTP Listen Port (listenPort): 25
SMTP Destination (smtpDestination): 125

[SMTP Session Limits]
Max Size of Local Message (maxSize): 20971520
Max Size of External Message (maxSizeExternal): 20971520

[Recipients/Local Domains]
Local Domains (localDomains): example.com=>
SERVER10.SERVER.local|morexamples.com
[x] Verify Recipients with SMTP-VRFY (DoVRFY)

[TestModes]
Prepend Spam Subject (spamSubject): [SPAM]
[x] Prepend Spam Tag (spamTag)
[x] All Test Mode ON (allTestMode)

[DNS Setup]
DNS Name Servers (DNSServers): 192.168.1.12|192.168.1.13

[Logging]
Notification Email To (Notify): helpdesk@example.com

[Server Setup]
Run ASSP as a Daemon (AsADaemon):
Yes – run AutoRestartCmd on restart and wait
OS-shell command for AutoRestart (AutoRestartCmd):
service assp restart
Run as UID (RunAsUser): assp
Run as GID (RunAsGroup): assp
My Name (myName): mail.example.com
My Helo (myHelo): SENDERHELO – IP – MYNAME – FQDN | MYNAME
Override the Server SMTP Greeting (myGreeting): MYNAME

[Relaying]
Allow All Mail (acceptAllMail): 192.168.1.10|192.168.1.3
Relay Host (relayHost): 127.0.0.1:125
Relay Port (relayPort): 225
Allow Relay Connection from these IP’s (allowRelayCon):
192.168.1.10|192.168.1.3

[ClamAV and FileScan]
Send Virus Report To This Address: helpdesk@example.com
[x] Use ClamAV

***CHECK THIS IN /etc/clamav/clamd.conf!***
Port or file socket for ClamAV (AvClamdPort):
/var/run/clamav/clamd.ctl
*******************************************

[Attachment Blocking]
External Attachment Blocking (DoBlockExes): score
External Attachment Blocking Level (BlockExes): Level 2

Level 1 rejected File Extensions (BadAttachL1): exe\-bin|exe|scr|pif|vb[es]?|jse?|ws[cfh]?|sh[sb]?|li?nk|bat|cmd|com|ht[ab]|ps1?
Level 2 rejected File Extensions (BadAttachL2): ade|adp|bas|chm|cpl|crt|dll|hlp|hta|inf|ins|isp|jar|js|lnk|mdb|msc|msi|mst|pcd|reg|src|sct

[SenderBase / Whois]
Do Country Blocking (DoSenderBase): disabled

[File Paths and Database]
Extension for Mail Files (maillogExt): .mht
#This allows opening the mail in Windows by doubleclicking
it. To paste the mail in the Mail Analyzer do not
doubleclick it but right-click it and view it in a
plain-text editor.
database driver name (Dbdriver): BerkeleyDB
Spam/HMM Bayesian Database Files (spamdb): DB:

[PenaltyBox / Message and IP Scoring]
PenaltyBox Database (pbdb): DB:
Email Whitelist Database (whitelistdb): DB:
Email Redlist Database File (redlistdb): DB:
[/code]

E-mail archival

From the ASSP website download the Archive Plugin (ASSP_ARC) and put it in the Plugins folder in the folder where ASSP is installed, for example /usr/share/assp/Plugins/ASSP_ARC.pm

Restart the ASSP service:
[code]# service assp restart[/code]

The web interface will now have a section called ASSP_ARC-Plugin.

Configuring your mail server

How you configure your mailserver depends on the mailserver you’re using. I’ll demonstrate for IBM Domino but obviously since we’re creating a relay server you can configure any mailserver to work with ASSP. Have your mailserver accept incoming mail from your ASSP server and set your mailserver to relay any outgoing mail to ASSP. Some mailservers call this a smarthost.

In Domino Administrator under the Configuration tab expand Messaging > Configurations > select server and click Edit Configuration.

Router/SMTP > Basics > Relay host for messages leaving the local internet domain: ASSP:225

Part 3: Managing the server

Backing up the server

For disaster recovery use image based backups, like with Clonezilla or Ghost.

Additionally we can backup the following configuration files and directories:
[code]/etc/postfix/transport
/etc/postfix/main.cf
/usr/share/assp/
/etc/clamav/*.conf[/code]

For backup systems I like to keep things as simple as possible. I’m copying everything to a tarball and copy the tarball to a backup server which handles (offsite) backups.

The backup server is a Windows machine so we’re using smbclient to copy the file. The sharename it e$ and there’s a folder called “ASSP Backups”. Because there’s a space in it it needs to be surrounded by double quotes which need to be escaped:
\”ASSP Backups\”

Begin by specifying your smb credentials in /root/.smbcreds. Make sure that file is only readable by root! As root, in /root/.smbcreds write:
[code]username=administrator
password=The4dminPa$$word
domain=SERVER.local[/code]
The domain line is optional.

Make the credentials file readable to root only, just to make sure:
[code]# chmod 600 /root/.smbcreds[/code]

Here is the script I use to copy the file to the backup server.
[code]#! /bin/bash

NOW=$(date +”%Y-%m-%d”)

# Create backupdir
mkdir ./$NOW

# Write list of installed packages to backupdir
dpkg –get-selections | grep -v deinstall > ./$NOW/packageslist.txt

# Copy data to backupdir
cp -R /etc/ ./$NOW
cp -R /usr/share/assp ./$NOW

# Copy backupdir to tarball
tar -cvpzf backup-$NOW.tar.gz ./$NOW

# Copy tarball to backup server
smbclient //SERVER24/e$ -l 192.168.1.24 -A /root/.smbcreds -c “cd \”ASSP Backups\”; put backup-$NOW.tar.gz backup-$NOW.tar.gz”

# Remove backupdir
rm -R ./$NOW
[/code]

Scheduling the backup

Schedule the backup using cron
[code]# crontab -e[/code]
and specify a time, for example every day at 23.59 hrs (so the date represents the day the changes in the files were made):
[code]0 23 * * * sh /usr/share/backup.sh[/code]

Alternatively use Webmin to schedule the backup.

The backup file is probably not restorable on Windows machines because of the path length so test the backup on a Linux machine or by extracting some random files instead of complete paths.

Restoring a backup

There are three ways to restore a backup:

1. Load a disk image. This is an easy way with a little bit of downtime depending on your hardware and disk size. Not much can go wrong here. You would use this for disaster recovery purposes as you problably won’t keep many versions lying around.
2. Restore individual files from the backup. Best stop relevant services and verify things afterwards. Advantage: less downtime. Small disadvantage: perhaps the system will be inconsistent so you need to check if things still work afterwards. Most of the time this is not a problem.
3. Restore the complete file based backup. You would do this if you need an old backup as these backups are complete but smaller as they leave out the operating system and software so you can store a lot of them.

To restore the file based backup (option #3) either pick out individual files and overwrite existing ones or:

1. Reinstall the system and run dist-upgrade.
2. Reinstall the software using the packages list from the backup file:
   [code]# dpkg –set-selections < packageslist.txt
   # apt-get -u dselect-upgrade[/code]
3. Recreate the backup system.
4. For good measure, reboot.

Upgrading the server

Before upgrading make sure to create an image first. Afterwards do:
[code]# aptitude upgrade[/code]
or
[code]# aptitude dist-upgrade[/code]
if you’re feeling adventurous.

Or use Webmin to install the updates/upgrades.

Upgrading Perl modules

In ASSP’s web interface click Info and Stats (in the top banner). In the Perl Modules section ASSP will tell you which modules and versions are installed and which are required or recommended. Note that ‘required’ should not be taken very literally. In most cases the older versions work just fine. I suggest upgrading only if at least one of the following applies:

• it is a security update
• you are certain that it will not break anything
• not upgrading breaks functionality

To upgrade a Perl module not in Ubuntu’s repositories just install it again in CPAN:
[code]# CPAN Text::Glob[/code]
If it’s in the repositories it’s best to have the OS update it automatically.

Upgrading ASSP itself

To upgrade ASSP itself, i.e. the Perl script assp.pl:

1. Stop the ASSP service.
2. Download the latest version of the script from SourceForge.
3. Rename assp.pl to assp.pl.old
4. Extract the new assp.pl file to the ASSP directory
5. Start the ASSP service.

Do keep the old version around for a bit in case the new version doesn’t work right.

Part 4: Working with ASSP

Monitoring

ASSP general health

A coloured dot on the web interface in front of the word ASSP informs you of ASSP’s general health status. Click the coloured dot to open the health page.

This will open the ASSP Worker/DB/Regex Status page which contains status information, error if any exist and recommendations.

SMTP Connections List

Click SMTP Connections in the top bar to open the SMTP Connections List. This page shows the current SMTP connections.

Please note:

• You must not make changes to ASSP in the main ASSP configuration window while the SMTP Connections List is is open.
• The SMTP Connections List only autorefreshes when it has focus.
• Click the Stop button (top right) make changes in the refresh rate to enable or disable colouring, then press the Start button.

Blue is outgoing mail, green incoming ham, red spam and white is not yet determined. The gradient says something about the various lists (RP/RY/NP/WL) the respective addresses are on.

Statistics

The Info and Stats link in the top banner shows detailed statistics on just about anything related to the system. Among them:

• ASSP proxy uptime
• Number of messages processed
• SMTP connections received and rejected
• Message scoring statistics
• Number of viruses detected

Working with white and blacklists

In general, addresses and domains on a whitelist are not checked for spam. ASSP uses a host of whitelists; among them:

• a list of whitelisted IP addresses (whiteListedIPs)
• a list of regular expressions to identify non-spam (whiteRe)
• whitelisted domains and addresses (whiteListedDomains)
• Addresses to which mail is sent from inside will be automatically added to the whitelist for 365 days.

Addresses and domains on blacklists will generally be blocked. Both lists can have exceptions.

There are a number of ways to edit these lists.

From the main configuration page

You can find all whitelists on the main configuration page in the Whitelisting section.

From the SMTP Connections List page

On the SMTP Connections List page click an address to open the ‘Add/remove address from lists’ window. You can add the address or domain to various lists. Also realtime information is shown on the occurance of the address in lists.

From the Maillog tail

In the top bar click View Maillog Tail. This will show you the most recent bit of the maillog file. Click any e-mail or IP address here to open the ‘Add/remove addresses from lists’ window.

The redlist

The redlist is a special kind of list. Addresses on the redlist do not contribute to the whitelist. This is particularly useful in case of users who are on vacation and set an out of office autoresponder to reply to any incoming mail, including spam. Normally the addresses this autoresponder sends mail to would end up on the whitelist because your colleage is a local user and contributes to the whitelist.

‘redRe’, the value in ‘[SPAM Control] Regular Expression to Identify Redlisted Mail’ contains a set of regular expressions to identify this kind of mail.

To manually add or a remove a user to or from the redlist (or whitelist) click White/Redlist/Tuplets in the top bar. You can add, remove or verify one or more users against the white and redlist here.

If a user goes on holiday add her to the redlist. If she returns remove her.

Mail analyzer

The Mail Analyzer in the top bar will take you to the ASSP Mail Analyzer. Clicking a green or red link in the Maillog Tail will pop up a window containing, among other options, an analyze button. Clicking this button will also open the analyzer.

If you opened the analyzer from the Maillog Tail viewer the analyze box will already have been filled and the analysis is presented. If you went there ‘manually’ you can paste an e-mail’s headers in the box and click the Analyze button.

A detailed analysis of the mail and why it is or is not considered spam will be given.

E-mail interface

The various lists can be managed via e-mail. Special e-mail addresses provide ways to report spam or ham and a whole lot more operations.

Having end users mail errors in spam and ham identification helps improving the Bayesian (and Markov, if used) analysis. Note you can also move the mails manually by dropping them in the errors/spam or errors/notspam folder.

Some addresses are useful for end users; some may be too technical. Some require special privileges. Operations performed by end users will be reported back to the user as well as to the helpdesk account.

address (@assp.local)	function
For all local users – bold = most often used
help	Request help text
Spam	Report forwarded spam
Notspam	Report forwarded false positive (ham)
Whitelist	Add sender of forwarded mail to the whitelist
Notwhitelist	Remove sender of forwarded mail from the whitelist
Persblack	Add sender of forwarded mail to the user’s personal blacklist
Notpersblack	Remove sender of forwarded mail from the user’s personal blacklist
Analyze	Return analysis on forwarded mail
For privileged users only
Blacklist	Add sender of forwarded mail to the blacklist
Notblacklist	Remove sender of forwarded mail from the blacklist
Redlist	Add sender of forwarded mail to the redlist
Notred	Remove sender of forwarded mail from the redlist
Spamlover	Add sender of forwarded mail to the spamlover list
Notspamlover	Remove sender of forwarded mail from the spamlover list
Noprocessing	Add sender of forwarded mail to the no processing list
Notnoprocessing	Remove sender of forwarded mail from the no processing list
Blockreport	Request a block report for selected users

The Analyze address works but the Unicode and Bayesian Analyses are illegible. Using ASSP’s web interface is a simple workaround.

Folders

ASSP uses various folders for storing spam and ham. Since we also use the ASSP_ARC plugin to archive all mail duplication will occur.

Folder	Contains
/var/assp/archive	Archives of all incoming and outgoing mail including spam
/usr/share/assp/logs	Logfiles
/usr/share/assp/resendmail	Mail that is to be re-sent to the original recipient
/usr/share/assp/okmail	Incoming mail that is not deemed spam and non-processed mail
/usr/share/assp/spam	Collection of mail that’s deemed spam
/usr/share/assp/notspam	Collection of mail that’s not deemed spam
/usr/share/assp/errors/notspam	False positives as reported or moved here by users or admins (counts 4x)
/usr/share/assp/errors/spam	False negatives as reported or moved here by users or admins (counts 2x)
/usr/share/assp/errors/notspam/ newManuallyAdded	Same as parent folder but processed immediately
/usr/share/assp/errors/spam/newManuallyAdded
/usr/share/assp/discarded	Spam not used for the spam collection

The archive folder differs from okmail, spam and notspam in that not all okmail, spam and notspam are stored in their respective folders. freqNonSpam and freqSpam (both default to 1) define the number or messages that are stored (every nth message).

The archive folder stores all incoming and outgoing mail that is not blocked at the SMTP level (e.g. not blocked by the Spamhaus blacklist or blacklisted domains).

Mailflow cases

A user wants to know if a certain mail was blocked as spam.

• Web interface: View Maillog Tail > search on subject, sender or receiver. If ASSP finds nothing increase the number of lines ASSP searches in. The logfile will mention wether the mail was sorted into the spam or another folder. Spam messages will be marked red, notspam green and okmail grey.

  

• Command line:
  [code]# cat /usr/share/assp/logs/mail.log | grep -i ‘SEARCHTERMS'[/code]
  Replace SEARCHTERM by the string you are searching for. The escape character is blackslash ( \ ).
• Folders: look for the mail subject in the spam, notspam and ok folders. Alternatively search the archive folder and paste the contents of the mail in the Mail Analyzer.

A user receives an untagged spam message.

Does the message contain a “[SPAM]” tag in its subject? If so the user can either ignore it or set up a local filtering rule to automatically move messages marked “[SPAM]”. If not we have a false negative: an unidentified spam message. In that case on of the following actions can be taken:

• The user can forward the spam message to spam@assp.local. (It is advisable to leave out the user’s default signature.) This will point out the error to the Bayesian engine. The user and helpdesk@example.com will be notified. The notification is informational; no further action is required.
• An administrator can forward the spam message to spam@assp.local.
• An administrator can move the mail from either okmail or notspam to errors/spam/newManuallyAdded (or just errors/spam; the difference is that Manually is processed internally immediately while the regular folder’s contents will be processed daily).
• An admin can find the mail in the Maillog Tail and click the green or grey archival link. In the popup window select the action ‘Copy file to correctedspam’ and click the Do It! Button.

  

An incoming mail was erroneously identified as spam.

False positive. Have the user forward the mail to notspam@assp.local or either:

• Move the file from spam to /errors/notspam
• Forward the mail to notspam@assp.local yourself
• Perhaps add the sender’s address or domain to the whitelist

A user wants a copy of a picture in an e-mail

• Copy the mail from the user’s mailbox to your own. Open in Outlook, copy the picure, send it back to the user. [We use this procedure because certain pictures cannot be opened in our corporate client.]
• Find the mail in the archive, spam, notspam or okmail folder (if you don’t know in which folder to look search for the subject, sender or receiver in the Maillog Tail first) and double click it to open it in Internet Explorer. Copy the picture, send it back to the user.

A user wants to add a sender or domain to the blacklist

There are two sorts of blacklists: personal and global. Global blacklists divide further into different lists; personal blacklists are addresses or domains that are blocked on a per-user basis. Users can manipulate their own personal blocklists; admins can manipulate all blocklists.

Global blacklists:

• Web interface:
  • IP Blocking (IP based only)
• E-mail interface: forward a mail to blacklist@assp.local. The original sender’s e-mail address will be added to the global whitelist. notblacklist@assp.local delists the address. Wildcards are supported.

Personal blacklists:

• E-mail interface: the user must send a mail to persblack@assp.local listing addresses he wants to add to his personal blacklist. If the first line is ‘reportpersblack’ (without the quotes) or if the body is empty a complete report about all his personal black list entries is returned. notpersblack@assp.local delists the address. Wildcards are supported. Removing tinus.detester@example.com would also remove *@example.com.

Adding certain words to the blacklists

In the web interface: Regex Filter / Spambomb

• Regular Expression to Identify Spam in Subject (bombSubjectRe) – expressions to look for in the mail’s header
• Regular Expression for Header and Data Part (bombRe) – expressions to look for in the mail’s body and attachments

A user wants to add a sender or domain to the whitelist

• Automatically: any address anyone from inside sends an e-mail to is automatically listed to the autowhitelist for a year.
• E-mail interface: the user or an admin can send an e-mail to whitelist@assp.local to add addresses to the whitelist. In the mail’s body list the address(es) you want added. Notwhitelist@assp.local delists the address.
• Manually: only admins can manually add entries to the whitelist from the web interface: manipulate the No Processing lists or Whitelists. No Processing means just that: no processing. Mail is not checked except for virusses. Whitelists are somewhat more dynamic.
  If in doubt use one of the following:
  • Whitelisting > Whitelisted Ips (whiteListedIPs)
  • Regular Expressions to Identify Non-Spam (whiteRe)
  • Whitelisted Domains and Addresses (whiteListedDomains)

Part 5: Troubleshooting

Logfiles

Postfix

The Postfix logfiles are /var/log/mail.err and /var/log/mail.log.

ASSP

Look in /usr/share/assp/moduleLoadErrors.txt for errors regarding loading Perl modules. Also in the ASSP web interface, in the top banner, click ‘Info and Stats’ and check the Perl Modules section for more information on Perl modules.

ASSP’s mail processing logfiles are in /usr/share/assp/logs.

Have cron log to the system logging server:
Edit the file /etc/rsyslog.d/50-default.conf and change
[code]#cron.* /var/log/cron.log[/code]
to
[code]cron.* /var/log/cron.log[/code]

Restart rsyslog and cron:
[code]# service rsyslog restart
# service cron restart[/code]

General

Find out which program is using a port (for example port 25):
[code]# netstat -tapen | grep “:25 “[/code]

If ASSP does things it shouldn’t do try restarting it:
[code]# service assp restart[/code]

Check if /usr/share/assp/assp.cfg matches ASSP’s web interface values. Sometimes it needs a kick in the butt for the values to stick.

Do not make changes in the web interface when the SMTP Sessions monitor window is opened. The SMTP Sessions monitor window is only automatically refreshed if it has focus.

If mail in the SMTP Connections List remains in the MAIL FROM status you may need to check your Postfix’s message_size_limit:
[code]# postconf -n | grep message_size_limit[/code]

If it’s too low or absent (=default; 10MB at the time of writing) define it in /etc/postfix/main.cf.

Search in mail logfile from ASSP: click ‘View Maillog Tail’ top right in the ASSP web interface.

Error: “Invalid Helo string” or “Invalid SMTP Banner”: change myGreeting in ASSP or check /etc/postfix/main.cf and change myhostname to the FQDN of the server (mail.example.com). If you changed the Postfix config do
[code]# postfix reload[/code]

If your spamfilter’s public IP is 123.234.345.67 and its name is mail.example.com check if these match up.

Windows:
[code]C:\>ping mail.example.com

Pinging mail.example.com [123.234.345.67] with 32 bytes of data:
Reply from 123.234.345.67: bytes=32 time=1ms TTL=63
Reply from 123.234.345.67: bytes=32 time=1ms TTL=63
Reply from 123.234.345.67: bytes=32 time=1ms TTL=63
Reply from 123.234.345.67: bytes=32 time=1ms TTL=63

Ping statistics for 123.234.345.67:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 1ms, Maximum = 1ms, Average = 1ms

C:\>ping -a 123.234.345.67

Pinging mail.example.com [123.234.345.67] with 32 bytes of data:
Reply from 123.234.345.67: bytes=32 time=1ms TTL=63
Reply from 123.234.345.67: bytes=32 time=1ms TTL=63
Reply from 123.234.345.67: bytes=32 time=1ms TTL=63
Reply from 123.234.345.67: bytes=32 time=1ms TTL=63

Ping statistics for 123.234.345.67:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 1ms, Maximum = 1ms, Average = 1ms[/code]

Linux:
[code]vorkbaard@xbtest:~$ ping mail.example.com
PING mail.example.com (123.234.345.67) 56(84) bytes of data.
64 bytes from mail.example.com (123.234.345.67): icmp_seq=1 ttl=61 time=2.93 ms
64 bytes from mail.example.com (123.234.345.67): icmp_seq=2 ttl=61 time=2.04 ms
^C
— mail.example.com ping statistics —
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 2.047/2.492/2.937/0.445 ms

vorkbaard@xbtest:~$ nslookup 123.234.345.67
Server: 127.0.1.1
Address: 127.0.1.1#53

Non-authoritative answer:
76.543.432.321.IN-ADDR.ARPA name = mail.example.com.[/code]

If the reverse lookup (nslookup or ping -a) doesn’t resolve to mail.example.com but to something like static-126-146-117-81.thenetworkfactory.nl you need to contact your internet provider and ask them to add a reverse dns entry for your IP address. Your internet provider may or may not be the same entity as your DNS provider.

Setting static-126-146-117-81.thenetworkfactory.nl (in this instance) as your Helo greeting (myGreeting in ASSP) will work as well but you wouldn’t be in control over that name. Note that myGreeting is a different field than myName and myHelo. They can be different values but they need not be.

Useful websites

Is Not Spam – analyses your mail and explains why it is or isn’t deemed spam.
MX Toolbox – analyse SMTP settings and more by checking DNS MX entries against your SMTP server. To check for reverse DNS mismatches choose the Test Email Server option in MX Toolbox.
DNS Queries – check the health status of an SMTP server

Sources

Software sources
http://www.ubuntu.com/
http://sourceforge.net/projects/assp/

File permissions
https://forum.directadmin.com/showthread.php?t=18112

General installation
http://www.techrepublic.com/blog/diy-it-guy/diy-add-a-spam-filter-to-your-mail-server-with-assp/
https://www.howtoforge.com/installing-assp-anti-spam-smtp-proxy-on-ubuntu-server-10.04-debian-5.0
http://blog.aceshigh.net/2013/11/assp-anti-spam-smtp-proxy-server/

Mail::SPF failing to install
https://www.ndchost.com/wiki/cpanel/mail-spf-fails-to-install
https://forums.cpanel.net/threads/mail-spf-fails-to-install.142285/

Modules
http://www.ninthavenue.com.au/how-to-install-assp-2-x-on-debian-7-wheezy
http://customerhelp.co.za/linux/ubuntu-security/logwatch-missing-cpu-meminfo-cpan-ubuntu-14-04.html
http://www.gossamer-threads.com/lists/clamav/users/61844