home | tech | misc | code | bookmarks (broken) | contact | README


NetBSD mail server with Postfix, BIND (for DNS), Dovecot, Pigeonhole (Sieve), SSL, DKIM and SPF

Disclaimer

This is not a quick HOWTO on how to build a mail server with NetBSD. This is a detailed documentation on how to build a mail server with NetBSD. You won't be able to make it work without understand some important concepts, so read it carefully and take some time (days?) to study concepts, read the references and work on it.

The last update to this document was in 2017-09-24.

Note

I'm not an experienced sysadmin and I have never setup a mail server. This document is an effort to document what I learnt. If you see any error on have any suggestion to improve this guide, please, let me know.

Introduction

Why this guide?

I took a glance at several guides on the internet, with different operating systems. I finally found a very good one, Home mail server with Postfix + Dovecot on NetBSD (that some guides on the NetBSD IRC channel -- #netbsd at irc.freenode.org -- recommended) but it is still incomplete. I followed it but it still has some holes I intend to fill.

Also, documenting stuff is the way I found to learn the steps I did in loco and also to help people with what I learnt.

Software needed

I did the configuration in this guide with the following software:

  • NetBSD 7.0.2
  • Postfix 2.11.4 (the one that comes with NetBSD 7.0.2)
  • BIND 9.10.4-P3 (the one that comes with NetBSD 7.0.2)
  • OpenSSL 1.0.1u (the one that comes with NetBSD 7.0.2)
  • Dovecot 2.2.30.2 (c0463e)
  • OpenDKIM Filter v2.10.3

A bit of history

I decided it would be useful to have a section about mail history because it helps understand why it works the way it works. E-mail is a set of protocols and extensions piled up over time. For those who grew up in the age of social networks, it might to be difficult to understand some stuff without some historic background. For instance: it was always a mystery for me why mail server and IMAP server were different things.

So, let's begin on what e-mail is. I strongly recommend reading the Wikipedia page on Email that has a nice background. For us it is enough to know that communication via what today is known as "e-mail" began by 1960s but "modern" e-mail was standardized in 1982, with the SMTP protocol.

What was happening at that time? People didn't have a lot of computers every time like nowadays. Although personal computers were going to be mainstream very soon, computers were still limited to some niches and their place were universities, big companies and so on.

But, at these places, people still didn't have workstation computers like we have on every desk nowadays. They used mainframes. And how they connected to mainframes? Using a computer terminal.

So a typical setup would look like this:

+--------+ :
|alice's | : +-------------+
|terminal|---|             |
+--------+   |             |
             | mainframe A |
+--------+   |             |
| bob's  |---|             |
|terminal| : +-------------+
+--------+ :
            .RS 232

So Alice and Bob had to connect their terminals to the mainframe they used. This connection was made in different ways but RS 232 became the most used standard. It is interesting to see that this explains why Unix (and other operating systems at that time, like Multics) had have such a improvement on user isolation and time and resource sharing for the hundreds (thousands?) of users on the same computer [1].

[1]Some time ago I found an old video from AT&T starring Brian Kernighan, Dennis Ritchie and others. That video shows a typical office at that time, with people using terminals to connect to a Unix system. Take a look! https://www.youtube.com/watch?v=tc4ROCJYbm0

Although Alice and Bob are connected to the same computer, they could be in different locations. People developed programs so they could send messages to each other (Does anybody know more? Please, let me know :-]). What if we could also send messages to people connected to other computers? No problem! Some message delivery systems were designed but SMTP got standardized in 1982. If two mainframes were running mail server systems, they could exchange messages using SMTP using the port 25:

+--------+                                              +--------+
|alice's |   +-------------+          +-------------+   |chris's |
|terminal|---|             |          |             |---|terminal|
+--------+   |             |   SMTP   |             |   +--------+
             | mainframe A |----------| mainframe B |
+--------+   |             |  port 25 |             |   +--------+
| bob's  |---|             |          |             |---|daniel's|
|terminal| : +-------------+          +-------------+   |terminal|
+--------+ :                                            +--------+
            .RS 232

This is what a mail server (an SMTP server) does: it sends message from one server to another and, at the receiving side, it delivers to the users mailbox. Period. At that time people didn't laptops, computers and smartphones to fetch their e-mail boxes. One should simple login into the mainframe, using the computer terminal, and send an e-mail. Sysadmins trusted each other, they had control over computers and there were not many of them, even less connected to what we call "the Internet".

But soon this reality changed. Computers were getting more numerous. DNS, the Domain Name System was invented, and the invention of the World Wide Web made computers rise in popularity.

Nowadays one person can have many computers. Mainframes got scarce and people have more processing power than mainframes. We are connected to the Internet 24 hours per day. The situation looks like this:

.          +----------+                         +----------+
           |  alice's |                         |  bob's   |
           |smartphone|                         |smartphone|
           +----------+                         +----------+

           +---------+                          +---------+
           | alice's |                          |  bob's  |
           | laptop  |                          | laptop  |
   alice   +---------+                          +---------+   bob
     o                                                         o
    -|-    +---------+                          +---------+   -|-
    / \    | alice's |                          |  bob's  |   / \
           | home pc |                          | home pc |
           +---------+                          +---------+

           +---------+                          +---------+
           | alice's |                          |  bob's  |
           | work pc |                          | work pc |
           +---------+                          +---------+

We now have many problems. Two important ones are:

  1. How one should send an e-mail to other if the receiver now has many computers? What computer should send it (and what should receive)?
  2. People are now admins of their own set of computers and the internet wizards have no control of it, like they had on the mainframes age. We then had started to have problems with spam, etc.

So, many other communication tools showed up: social networks, VoIP software, etc., much of them centralized stuff. But the good and old e-mail survived! Extensions were developed (while keeping protocols backward compatible) to make it safer and avoid spam. Now we have something like this:

.          +----------+                         +----------+
           |  alice's |                         |  bob's   |
           |smartphone|                         |smartphone|
           +----------+                         +----------+
                                    . SMTP port 25
           +---------+             :            +---------+
           | alice's |  +-------+  : +-------+  |  bob's  |
           | laptop  |  |foo.com|  : |bar.com|  | laptop  |
   alice   +---------+  |  mail |----|  mail |  +---------+   bob
     o                  | server|    | server|                 o
    -|-    +---------+  +-------+    +-------+  +---------+   -|-
    / \    | alice's |  /                   \   |  bob's  |   / \
           | home pc |--.                    -- | home pc |
           +---------+  :                       +---------+
                        :
           +---------+   . SMTP port 587        +---------+
           | alice's |     to send              |  bob's  |
           | work pc |     POP3 to receive or   | work pc |
           +---------+     IMAP to receive      +---------+

Mail servers still use port 25 to communicate with each other. But users use other ports and protocols developed later to fetch e-mails from the mail server to their devices, or to send e-mails [2].

[2]There are also other manners to access your e-mail. With the web, web clients sprung up and it is nowadays one of the most common ways to access your inbox (not to mention proprietary API's that most applications use). Gmail is a good example of that. There is also web clients that are free software like Roundcube but, as I said earlier, they are not the focus of this article. You can refer to the great Home mail server with Postfix + Dovecot on NetBSD for more information.

What we are learning in this guide is to make the mail server communicate with other mail servers and also configure software to allow users to fetch their mail server.

Part 1: A simple mail server

In this part we are going to make our mail server talk to other mail servers. For this to work, we are going to need support for other system. Probably, the most important one is the DNS.

Before we begin, let's make some important definitions:

  1. We are going to setup a server called example.com. Our main user is "alice" (so we want to setup the e-mail alice@example.com. In a company environment, DNS, mail server, IMAP server, etc., is kept separately for security and other reasons. Since it is a personal server, we are going to install every service we need to the same machine.
  2. You can do it in your home, but you we'll need a fixed IP mainly because of Reverse DNS lookup. I find it easier (and maybe cheaper!) to just rent a small virtual private server. Let's suppose our IP is 123.45.67.89.

Setting up DNS with BIND

Important

Until now, I was just a DNS user. My first BIND configuration is this and there may be (serious) errors. Please, let me know if you see something wrong!

We are not enter the details of how DNS works. I had a user view of that and I had to discover a bit more to configure the e-mail server, but is still a very limited view. I will, though, list the wrong concepts I had because I believe other people may have the same difficulties.

Note

Everything I know here is because I read the great book DNS and BIND, by Cricket Liu and Paul Albitz. It explains very well what DNS is and how to configure BIND (probably the most used DNS server). At the time I write this I have read only the first 5 chapters, which was enough for a basic setup, but the whole book seems promising!

I though that the sole purpose of DNS was to to tell a requester the IP address of a given name, but it can inform many more things and some of them are used for the e-mail service.

Let's make some tests. Type in your terminal:

$ dig +short gmail.com
216.58.202.165

The dig tool [3] [4] will show the IP address associated with the gmail.com domain. This is called the A registry (A stands for Address). Looking for the A registry is the default but we can issue the command with it explicitly:

$ dig +short A gmail.com
216.58.202.165
[3]The dig(1) tool is a DNS lookup utility. There are more references to the more famous nslookup(1) but I prefer using dig. We are going to use the +short flag because we are not interested in the details of DNS query for now.
[4]It will use the default DNS server configured in your system (in /etc/resolv.conf. If you want to use another, use the @ character. For instance, to run the same command using the Google DNS server, type: dig @8.8.8.8 +short gmail.com.

What does it mean? If we send an e-mail to foo@gmail.com, will it be sent to IP address 216.58.202.165? No! The mail system actually queries for another DNS registry called MX. Let's take a look:

$ dig +short MX gmail.com
40 alt4.gmail-smtp-in.l.google.com.
20 alt2.gmail-smtp-in.l.google.com.
30 alt3.gmail-smtp-in.l.google.com.
10 alt1.gmail-smtp-in.l.google.com.
5 gmail-smtp-in.l.google.com.

Those are the servers that can receive e-mails when we send something to the Gmail servers. We also need to specify this in our DNS setting.

Will we run our own server? Yes, and we'll show how to make that with BIND.

Note

A note on registrars: the company you registered your domain in may have their own DNS server. The company where I registered mine had, but it was buggy and had a broken interface, so I decided to setup my own DNS server.

First, configure the /etc/named.conf file. NetBSD defaults are OK, but you have to add your own zone:

zone "example.com" {
        type master;
        notify no;
        file "example.com";
};

Now we need to create the zone file, /etc/namedb/example.com:

$TTL    3600
@       IN      SOA     example.com. hostmaster.example.com.  (
                                2017090901      ; Serial
                                3600            ; Refresh
                                300             ; Retry
                                3600000         ; Expire
                                3600 )          ; Minimum
                IN      NS      ns.example.com.
                IN      NS      puck.nether.net.
                IN      MX      1 mail.example.com.
mail            IN      A       123.45.67.89
ns              IN      A       123.45.67.89
example.com.    IN      A       123.45.67.89

This is where magic happens. We are not explaining every detail of this file. For more information, refer to the chapter 4 of DNS and BIND. Some lines deserve special attention, though:

  1. Line 2 (the one that starts with "@"): Just add your domainname. Don't forget the trailing dot [5].

  2. Lines 3 to 7: that follow are for DNS synchronization between servers. Defaults are ok. Increment the first number whenever you make a change to this zone, so other DNS servers know when you have newer information to be synced.

  3. Line 8: This is important: Here is where we configure the NS registry for this DNS entry. "NS" stands for Name Server and it is a server that will be queried when a client wants to know the IP address of any subdomains of example.com domain.

    It is important to explain things a bit: let's imagine someone is querying for server "smtp.example.com". Your ISP DNS server knows that the server responsible for the zone (in DNS parlance, it is called "authoritative server" for a given zone) is "ns.example.com", i.e., it has every information about any subdomains for "example.com". So things work like this: user queries his ISP DNS that queries "ns.example.com" that answers to the ISP DNS that answers to the user. That is why we need to set this line.

    In your registrar, you need to add the nameservers responsible for your zone. In my registrar I add the name of the nameserver ("ns.example.com") and its IP addres.

  4. Line 9: This is the name for the secondary DNS server. Registrars usually [6] require at least two DNS servers. Unfortunatelly we can't afford a second one. So we use one of the free secondary DNS servers that exist. I use PUCK Free Secondary DNS Service but you'll find others in Free Secondary DNS list. After you setup your secondary DNS server, it will easily synchronize information from the master server ("ns.example.com"). [7].

  5. Line 10: The MX record is where we specify our mail server. Any message addressed to "anybody@example.com" is delivered in one of the MX servers. You can have more than one server (if some is down, others can take over -- just remind the example we did before: dig +short MX gmail.com). But since we can't afford a backup server, we'll work with one only.

  6. Lines 11 to 13: Now we specify the A records, i.e., the IP address of each server. Unfortunatelly we have no money and they are actually the same! Could they have the same domain name? Yes, but we would have to give different names if we use different IP addresses for each server later, so it is better to have different names even if they have the same IP to make further improvements easier. [8].

[5]We use the trailing dot whenever we want to specify an "absolute domain" (like an absolute path in filesystem that starts with a trailing slash, absolute domains end with a trailing dot). Again, refer to chapter 4 of DNS and BIND.
[6]See that we have relative domain names ("smtp" becomes "smtp.example.com.") and absolute ("example.com." is just "example.com.").
[7]Always?
[8]For more information about DNS synchronization, see chapter 2 of DNS and BIND.

Important

At the time of this writing, I was not sure if I should include the PTR records. The company where I rent the VPS already had a reverse DNS record for this machine (with a different address, though). Is it enough? If someone know that, please, let me know.

After all this, we just add BIND daemon (named) to /etc/rc.conf and start it:

# echo 'named=YES' >> /etc/rc.conf
# /etc/rc.d/named start

Don't forget to tell your registrar about your nameserver and the secondary nameserver, too.

Setting up Postfix

We now have a our DNS records working. Without them (specially the MX records) it would be impossible to have a mail server. [9]

[9]Or at least very difficult? Mail servers predate DNS, but nowadays I find it very difficult to setup one without DNS.

Now, let's add configuration to the Postfix mail server, that comes by default in NetBSD. Again, almost all configuration here follows the rules described in Home mail server with Postfix + Dovecot on NetBSD.

Let's first change /etc/postfix/main.cf file. We are going to list every option to add to the file and explain them:

myhostname = smtp.example.com
mydestination = $mydomain, $myhostname, localhost.$mydomain, localhost
home_mailbox = Maildir/
  1. Line 1 and 2: Just the domain name and the name of this host. [10].
  2. Line 3: Who we want to deliver messages to.
  3. Line 4: We want to user Maildir to stored e-mails instead of more traditional (and limited) mbox.
[10]Does it have to be the same of the output of the hostname(1) command or does it have to be the entry of it in the DNS?

There is one more setting you have to make. Now in file /etc/postfix/master.cf. Change line:

#smtp      inet  n       -       n       -       -       smtpd

To:

smtp      inet  n       -       n       -       -       smtpd -o myhostname=example.com

Important

Replace example.com by the name you get from reverse DNS lookup. Sometimes it is configured by the company you contract your VPS from. To discover the domain name related to your IP address, use the dig(1) tool:

$ dig +short +x 123.45.67.89

This is really a very important setting since that receivers will query for address related to your IP and compare this to your e-mail headers Postfix add using this setting. If they don't match, mail serves closes the connection and don't receive the message. This is a common protection against spam that fakes DNS names.

VoilĂ ! This configuration is enough for a 1982 configuration. And, believe or not, you can send and receive e-mails to and from your mail server. There are two problems, though: one is that we'll send plain text message, without any cryptography. The other is that chances that your e-mail fall into the receiver's spam folder are high because we are not using modern stuff here. Let's learn how to configure them.

Part 2: Improving our mail server

Enabling SSL for Postfix

We are going to use openssl(1) to create our public and private keys. As Home mail server with Postfix + Dovecot on NetBSD pointed, let's create our keys.

We can do that with two ways. The first one is to have a self-signed certificate and the second one is ask for a company to generate the certificate for you.

Generating a self signed certificate is simple:

# mkdir -p /etc/ssl/{certs,private}
# cp /usr/share/examples/openssl/openssl.cnf /etc/openssl/
# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/example.com.key -out /etc/ssl/certs/example.crt
# chmod 600 /etc/ssl/private/example.com.key

If you want a company to sign your keys (which people recommend [11]), you'll probably have to pay. There are, though, some that provide that service for free or cheap enough. Check StartSSL for an example.

[11]Check this discussion for more information on the advantages or disadvantages of using a self-signed certificate.

Note

If you use StartSSL, they will generate the certificates and give you a zip file with them. Inside the zip file there are already directories with the certificates ready for common applications like Apache. We have not a specific directory for Postfix but we have a directory called OtherServer. You will find three files there: 1_Intermediate.crt, 2_example.com.crt and root.crt. The certificate wil are going to generate is a concateation of 2_example.com.cr and 1_Intermediate.crt (attention: the order matters). So, generate that:

# cat 2_example.com.crt 1_Intermediate.crt > /etc/ssl/certs/example.com.crt

After that, we should add more settings to Postfix /etc/postfix/main.cf file:

smtpd_tls_security_level=may
smtpd_use_tls=yes
smtpd_tls_auth_only=yes
smtpd_tls_cert_file=/etc/ssl/certs/example.com.crt
smtpd_tls_key_file=/etc/ssl/private/example.com.key
smtpd_tls_loglevel=1

You are not explaining these options since most of them are self-explanatory. But if you want to know the details, check the Postfix documentation.

Now, just start postfix!

# /etc/rc.d/postfix start

You are now able to send and receive e-mails from and to this machine. Make a test: 1. login into this machine (this is like connecting via a serial line to your mainframe! :-]), chose an existing e-mail address (your's or of a friend) and send it an e-mail:

$ echo "message body comes here" | mail -s "subject" myfriend@email.com

It will receive the e-mail. If it don't, check logs for postfix in /var/log/maillog. Also, never forget to check the spam folder. Some later configurations are still needed if we want reduce the probability of letting our e-mails be dropped to the spam folder.

Allowing devices check our e-mail with IMAP

We are now able to send and receive people e-mail. E-mail you receive shall appear in /var/mail/yourusername. We are in a 1982 configuration with some extensions: SSL mainly. Without it is almost impossible to exchange e-mail with other servers nowadays.

But we don't want that configuration. We don't want to login to this machine in a console or ssh whenever we need to check our e-mails. Also, we want to sync all our devices to fetch and send e-mails. So we are going to configure our IMAP server.

As I mentioned, Postfix is e-mail server, i.e., it handles the SMTP protocol (and extensions) and does e-mail exchanging with other servers. It doesn't provide any support for IMAP. To make IMAP to work, we'll need to install another daemon and configure Postfix to talk to it. We are going to install Dovecot.

For a bit more detailed guide on how to configure Dovecot with SASL with Dovecot, see Postfix with SASL.

Build it from source via pkgsrc (mail/dovecot2) or just install the binary package. Check pkgsrc documentation for details.

Configuration of Dovecot is straightforward. Open the /usr/pkg/etc/dovecot/dovecot.conf file in your favorite editor. First, change the line:

protocols = imap pop3 lmtp

to:

protocols = imap lmtp

We are not going to use POP3 (anybody uses?). Now, add or uncomment the following lines:

ssl = yes
ssl_cert = </etc/ssl/certs/example.com.crt
ssl_key = </etc/ssl/private/example.com.key

Now edit file /usr/pkg/etc/conf.d/10-mail.conf and add (or uncomment) this line:

mail_location = maildir:~/Maildir

It indicates to store users' e-mails in their home directories, using the Maildir format.

Now, just start Dovecot:

# cp /usr/pkg/share/examples/rc.d/dovecot /etc/rc.d
# echo 'dovecot=YES' >> /etc/rc.conf
# /etc/rc.d/dovecot start

There are one important things to note here: with this setup, Postfix and Dovecot are not talking to each other. What happens here is that Postfix deliver e-mails to users' home directory and Dovecot reads from the same location. It works but this setup has some limitations, like not allowing e-mail filtering. To know the advantages and how to setup LMTP take a look in section Making Postfix talk to Dovecot via LMTP.

Allowing devices to send e-mail using SMTP

We can now configure our devices to fetch e-mail from our server using the IMAP protocol but we cannot send yet. The SMTP port Postfix is using, port 25, is solely for other server communication purposes. But, for this, Postfix will enable us to use another port, mainly 587. This is called submission, i.e., e-mail submission and port 587 is reserved for that. [12]

[12]Other ports and proposals for e-mail submission have existed. See this link for more information.

Configure that is also pretty simple. First, open file /etc/postfix/master.cf again and uncomment the submission line and add some lines after that. It will be like:

submission inet n       -       n       -       -       smtpd
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth
  -o smtpd_sasl_security_options=noanonymous
  -o smtpd_sasl_local_domain=$myhostname
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o smtpd_recipient_restrictions=reject_non_fqdn_recipient,reject_unknown_recipient_domain,permit_sasl_authenticated,reject

Important

Don't remove spaces at the beginning of the line. They mean the continuation of the command on the line before.

Also add this line to /etc/postfix/main.cf:

smtp_tls_security_level = may

Realize that smtp* is different from smtpd* (see the trailing d) settings. The former is for the submission port (587), the later is for SMTP port (25).

Restart Postfix:

# /etc/rc.d/postfix restart

Now you have a full e-mail server! But it is not done yet, we'll have to configure all the extensions to it so we don't be marked as spam.

Making Postfix talk to Dovecot via LMTP

As we stated in section Allowing devices check our e-mail with IMAP, Postfix and Dovecot just write/read e-mails from the same location but they don't talk to each other. This is simple but it doesn't allow us to use any of the advanced features Dovecot provides. To fix this, we are going to setup both Postfix and Dovecot to use the LMTP protocol. So, Postfix will deliver e-mail to Dovecot explicitly:

e-mail                  delivers
incoming   +---------+  e-mail    +---------+
---------->| Postfix |----------->| Dovecot |
via        | server  |  via       | server  |
port 25    +---------+  LMTP      +---------+

Let's first start setting up Dovecot. It is a very simple configuration. According to the Dovecot with Postfix LMTP, we need to change file /usr/pkg/etc/dovecot/conf.d/10-master.conf, configuring LMTP service:

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
      mode = 0600
      user = postfix
  }
}

In this example, we are using a Unix Domain Socket (it is useful specially if Dovecot is installed at the same machine as Postfix), but you can use a inet socket too. See Dovecot with Postfix LMTP for details and don't forget to set permissions and ownership correctly.

It is enough at the Dovecot side. Now let's do that at the Postfix side. Edit file /etc/posfix/main.cf. First, since we are delivering our e-mail to Dovecot via LMTP, setting home_mailbox (that we configured at Setting up Postfix is now useless. You can commment or exclude it. Now we have to add the mailbox_transport setting:

mailbox_transport = lmtp:unix:private/dovecot-lmtp

You can also delete the home_mailbox from /etc/main.cf because we won't use it anymore.

Note

By default, user authentication is done with the full e-mail (i.e., clients should provide the full e-mail for login information, and not only the username. We recommend keeping this as is, specially if you want to user virtual domains as we explain in Setting up virtual users (and virtual domains). If you really change to change it, according to this page, edit /usr/pkg/etc/dovecot/conf.d/10-auth.conf and change variable auth_username_format:

auth_username_format = %Ln

Now, don't forget to restart Postfix and Dovecot:

# /etc/rc.d/postfix restart
# /etc/rc.d/dovecot restart

To make sure your LMTP setting works, always take a look at the /var/log/maillog file.

E-mail filtering with Pigeonhole (Sieve)

The Sieve programming language was created specially for e-mail filtering. In the Dovecot world it is done by the Pigeonhole project. We'll describe how to setup it here.

First, make sure your LMTP setting works. See Making Postfix talk to Dovecot via LMTP for more information. Then, install the mail/dovecot2-pigeonhole package.

Once it is working, let's enable the sieve plugin for LMTP. Edit file /usr/pkg/etc/dovecot/conf.d/20-lmtp.conf, locate line starting with mail_plugins and add it:

protocol lmtp {
  mail_plugins = $mail_plugins sieve
}

And just restart Dovecot:

# /etc/rc.d/dovecot restart

Simple, ahm?

Now, include your rules in your ~/sieve file. For example:

require ["fileinto"];
if header :contains "subject" "test" {
    fileinto "test";
    discard;
    stop;
}

This will make that every e-mail with the "test" word in the subject header is put into the "test" mailbox and discarded from inbox.

For more information about configuring the Pigeonhole Sieve plugin, see Pigeonhole/Sieve/Configuration and for troubleshooting see Sieve Troubleshooting and never forget to look at /var/log/maillog.

Setting up virtual users (and virtual domains)

For now, we now have a full workable e-mail server with different capabilities plugged in, but we are still using system users. If you want to add a new e-mail user you'll have to add it as a Unix system user, creating all the environment for a normal user, with home directory, login shell and so on. If you need to do so but don't want to let your users login via ssh, you can set their shells to /sbin/nologin.

But this solution doesn't scale. For this, both Postfix and Dovecot provide a feature called "virtual users" that you can use. There is also one more reason to use virtual users: you can also set virtual domains together and provide e-mail service for more than one domain, in the same server! A similar feature to virtual hosts of HTTP servers.

Postfix virtual users are completely unrelated to Dovecot virtual users. So, we can have different setups:

  1. Postfix without virtual users and Dovecot with virtual users.
  2. Both Postfix and Dovecot with virtual users.

The first one, AFAIK, let you use system users but allow you to setup a different password that users use for IMAP and system access. The second one is useful if you want to provide true virtual e-mail users and also virtual domains support. We are going to explain how to setup the second case.

The page how to build a virtual user e-mail server has helped a lot on how to understand and setup this. We are going to use the simple "passwd-file" method, on which we store user and password information in a plain text file, but several other methods, like storing users and password information in SQL database or LDAP.

First, we need to make a few changes to /etc/postfix/main.cf:

myhostname = other.com
virtual_mailbox_domains = /etc/postfix/vhosts
virtual_mailbox_base = /home/maildeliverer
virtual_mailbox_maps = hash:/etc/postfix/vmaps
virtual_uid_maps = static:1008
virtual_gid_maps = static:1008
virtual_transport = lmtp:unix:private/dovecot-lmtp
# Also, you can now delete line mailbox_transport we added before.

So, let's explain what each variable mean:

myhostname
We need to set myhostname to a different domain we are going to use in our e-mails. All virtual domains are all going to the /etc/postfix/vhosts file.
virtual_mailbox_domains
The file with a list of virtual domains we are going to work with. See below.
virtual_mailbox_base
The directory where e-mails will be stored. Let's create a system user for this and call it "maildeliverer" and let our virtual users boxes located at /home/maildeliverer.
virtual_mailbox_maps
The file with a list of virtual users. See below.
virtual_uid_maps
The uid (user id) virtual mail boxes will be created with. The uid of "maildeliverer".
virtual_gid_maps
The gid (gropu id) virtual mail boxes will be created with. The gid of "maildeliverer". In NetBSD, every new user is, by default, part of the "users" group (gid = 100) but it is wise to let a group that don't mix with other users groups. In this example we create a user with gid = 1008.
virtual_transport
Now that we are using virtual users we have to set this variable. It has the same value as the mailbox_transport variable. By the way, we are not going to need mailbox_transport anymore, so you can delete it.

The content of /etc/postfix/vhosts is just a list of virtual hosts per line:

example.com
example2.com

And the content of /etc/postfix/vmaps is a list of records per line, where each record is compound with user login information and where the e-mail will be stored:

alice@example.com       Maildir-alice
bob@example.com         Maildir-bob

The mailboxes will be created relative to value of variable virtual_mailbox_base with uid and gid set to virtual_uid_maps and virtual_gid_maps.

To add groups and user commands, use commands groupadd(8) and useradd(8) commands.

After you have created the /etc/postfix/vmaps file, enter the directory and execute the portmap program:

# cd /etc/postfix
# portmap vmaps

It will create file vmaps.db that postfix will use to load virtual user information.

It is worth to note that, although we have options such as virtual_mailbox_base, virtual_uid_maps, virtual_gid_maps and also the mailboxes in file /etc/postfix/vmaps, they are not being used (really?). We added this to show how postfix can work with virtual users alone but we are passing e-mails to Dovecot with LMTP, setting virtual_transport variable. So, those setting are just ignored (can anybody confirm this?).

So, let's now see how we configure Dovecot virtual users.

In file /usr/pkg/etc/dovecot/conf.d/10-auth.conf, disable or comment line:

#!include auth-system.conf.ext

And uncomment or add line:

!include auth-passwdfile.conf.ext

Then, let's configure file /usr/pkg/etc/dovecot/conf.d/auth-passwdfile.conf:

mail_location = maildir:/home/maildeliverer/%n/Maildir

passwd {
  driver = passwd-file
  args = scheme=CRYPT username_format=%u /usr/pkg/etc/dovecot/users
}

userdb {
  driver = passwd-file
  args = username_format=%u /usr/pkg/etc/dovecot/users
}

Note

This configuration let us configure virtual users mailbox at /home/maildeliverer/<username>/Maildir. %n is replaced by the username part of the e-mail (the one before @). If we have different users in different domains that have the same user name (e.g.: alice@foo.com and alice@bar.com) you'd better include the domain name in the mail_location information, like this:

mail_location = maildir:/home/maildeliverer/%d/%n/Maildir

We are pointing to file /usr/pkg/etc/dovecot/users. Let's create this file with this content:

alice@example.com:{CRAM-MD5}e02d374fde0dc75a17a557039a3a5338c7743304777dccd376f332bee68d2cf6:1008:1008::/home/maildeliverer/alice
bob@example.com:{CRAM-MD5}124ff4a48a846c9bf4146c32ba7a57ad2fb2b30cfef8d06c0f1c35eccb1598bc:1008:1008::/home/maildeliverer/bob

The format of this file is:

username@domain:crypt-password:uid:gid::virtualuserhome

To generate the password part, use the command:

# doveadm pw

Make sure the uid and gid parameters are set to the user you want to have access to the virtual boxes.

Finally, restart Postfix and Dovecot:

# /etc/rc.d/postfix restart
# /etc/rc.d/dovecot restart

Part 3: Improving it even more: E-mail authentication

Having a mail server configured with SSL is an important step to make your server be recognized as "legit" for other mail servers. Other crucial feature is reverse DNS. But those features are not enough. As time passed, other features were developed to make spam and fake e-mails difficult. The ones we are going to show here are SPF, DKIM and DMARC.

We are not going to explain what SPF, DKIM and DMARC are. For a great explanation about that see the page Why your Marketing Email will Land in my Spam Folder. For a discussion about this page see this entry in Hacker News.

SPF

SPF stands for Sender Policy Framework and it is a simple setting you add to your DNS setting so e-mail filters can query who is able to send e-mails signed as your domain.

Basically, you list in a DNS entry what servers are able to send e-mail using your domain. If we are using one simple e-mail server (if you can afford, choose to have redundancy) it is as simple as:

example.com.    IN      TXT     "v=spf1 mx -all"

With mx, it will look for domains of all MX servers you declared in your DNS.

Pay attention to the -all part. Basically, this line tells you that mail host, and only mail, can send e-mail. Everything else should be ignored.

Since we changed our DNS zone, reload nameserver configuration:

# /etc/rc.d/named reload

You can debug SPF problems with the SPF Record Lookup page.

DKIM

DKIM is a kind of e-mail authentication mechanism. It uses your private key to create a signature of your e-mail that is added to the e-mail headers. Other servers use your public key (that is added as a TXT record of your DNS server) to check the signature. It is a very effective way to check your e-mail is authentic and not a fake one. [13]

[13]Again, see Why your Marketing Email will Land in my Spam Folder for more details.

I followed the guide at How To Install and Configure DKIM with Postfix on Debian Wheezy for configuring DKIM.

First install it with pkgsrc (mail/opendkim). Then, generate the keys:

# cd /var/db/opendkim
# opendkim-genkey -s mail -d example.com

It will generate, inside var/db/opendkim, two files: mail.private, where your private key exists, and mail.txt where your DNS entry with public key exists.

mail.txt has the exact entry for DNS. Just append it to your DNS zone configuration and reload the DNS server changes:

# cat mail.txt >> /etc/namedb/example.com
# /etc/rc.d/named reload

Then, edit the OpenDKIM configuration file. Add (or uncomment) the following settings:

Domain      example.com
KeyFile     /var/db/opendkim/mail.private
Selector    mail
Socket      inet:12301@localhost

Basically, it will create a socket that we'll use to comunicate with Postfix. Another option would be to create a Unix Domain Socket with:

Socket local:/var/db/opendkim/dkim-socket

If you chose to do so, don't forget to change UserID and maybe UMask options.

After that, add the following lines to /etc/postfix/main.cf:

milter_protocol = 2
milter_default_action = accept
smtpd_milters = inet:localhost:12301
non_smtpd_milters = inet:localhost:12301

Now, start OpenDKIM:

# echo 'opendkim=YES' >> /etc/rc.conf
# /etc/rc.d/opendkim start

And reload Postfix configuration:

# /etc/rc.d/postfix reload

A very good tool to debug DKIM is DKIM Test page.

DMARC

TODO