home | tech | misc | code | bookmarks (broken) | contact | README

# NetBSD mail server with Postfix, BIND (for DNS), Dovecot, Pigeonhole (Sieve), SSL, DKIM and SPF

Note

This is not a quick HOWTO on how to build a mail server with NetBSD. This is a detailed documentation on how to build a mail server with NetBSD. You won't be able to make it work without understand some important concepts, so read it carefully and take some time (days?) to study concepts, read the references and work on it.

Note

I'm not an experienced sysadmin and I have never setup a mail server. This document is an effort to document what I learnt. If you see any error on have any suggestion to improve this guide, please, let me know.

Note

This is a detailed guide on how to configure Postfix, Dovecot and other tools to have a full email server in NetBSD. For information about common tools and other information, see my Postfix notes.

The last update to this document was on 2019-03-19.

## Introduction

### Why this guide?

I took a glance at several guides on the internet, with different operating systems. I finally found a very good one, Home mail server with Postfix + Dovecot on NetBSD (that some guides on the NetBSD IRC channel -- #netbsd at irc.freenode.org -- recommended) but it is still incomplete. I followed it but it still has some holes I intend to fill.

Also, documenting stuff is the way I found to learn the steps I did in loco and also to help people with what I learnt.

TODO

### Software needed

I did the configuration in this guide with the following software:

• NetBSD 7.0.2
• Postfix 2.11.4 (the one that comes with NetBSD 7.0.2)
• BIND 9.10.4-P3 (the one that comes with NetBSD 7.0.2)
• OpenSSL 1.0.1u (the one that comes with NetBSD 7.0.2)
• Dovecot 2.2.30.2 (c0463e)
• OpenDKIM Filter v2.10.3

### A bit of history

I decided it would be useful to have a section about mail history because it helps understand why it works the way it works. E-mail is a set of protocols and extensions piled up over time. For those who grew up in the age of social networks, it might to be difficult to understand some stuff without some historic background. For instance: it was always a mystery for me why mail server and IMAP server were different things.

So, let's begin on what e-mail is. I strongly recommend reading the Wikipedia page on Email that has a nice background. For us it is enough to know that communication via what today is known as "e-mail" began by 1960s but "modern" e-mail was standardized in 1982, with the SMTP protocol.

What was happening at that time? People didn't have a lot of computers every time like nowadays. Although personal computers were going to be mainstream very soon, computers were still limited to some niches and their place were universities, big companies and so on.

But, at these places, people still didn't have workstation computers like we have on every desk nowadays. They used mainframes. And how they connected to mainframes? Using a computer terminal.

So a typical setup would look like this:

+--------+ :
|alice's | : +-------------+
|terminal|---|             |
+--------+   |             |
| mainframe A |
+--------+   |             |
| bob's  |---|             |
|terminal| : +-------------+
+--------+ :
.RS 232


So Alice and Bob had to connect their terminals to the mainframe they used. This connection was made in different ways but RS 232 became the most used standard. It is interesting to see that this explains why Unix (and other operating systems at that time, like Multics) had have such a improvement on user isolation and time and resource sharing for the hundreds (thousands?) of users on the same computer [1].

 [1] Some time ago I found an old video from AT&T starring Brian Kernighan, Dennis Ritchie and others. That video shows a typical office at that time, with people using terminals to connect to a Unix system. Take a look! https://www.youtube.com/watch?v=tc4ROCJYbm0

Although Alice and Bob are connected to the same computer, they could be in different locations. People developed programs so they could send messages to each other (Does anybody know more? Please, let me know :-]). What if we could also send messages to people connected to other computers? No problem! Some message delivery systems were designed but SMTP got standardized in 1982. If two mainframes were running mail server systems, they could exchange messages using SMTP using the port 25:

+--------+                                              +--------+
|alice's |   +-------------+          +-------------+   |chris's |
|terminal|---|             |          |             |---|terminal|
+--------+   |             |   SMTP   |             |   +--------+
| mainframe A |----------| mainframe B |
+--------+   |             |  port 25 |             |   +--------+
| bob's  |---|             |          |             |---|daniel's|
|terminal| : +-------------+          +-------------+   |terminal|
+--------+ :                                            +--------+
.RS 232


This is what a mail server (an SMTP server) does: it sends message from one server to another and, at the receiving side, it delivers to the users mailbox. Period. At that time people didn't laptops, computers and smartphones to fetch their e-mail boxes. One should simple login into the mainframe, using the computer terminal, and send an e-mail. Sysadmins trusted each other, they had control over computers and there were not many of them, even less connected to what we call "the Internet".

But soon this reality changed. Computers were getting more numerous. DNS, the Domain Name System was invented, and the invention of the World Wide Web made computers rise in popularity.

Nowadays one person can have many computers. Mainframes got scarce and people have more processing power than mainframes. We are connected to the Internet 24x7. The situation looks like this:

.          +----------+                         +----------+
|  alice's |                         |  bob's   |
|smartphone|                         |smartphone|
+----------+                         +----------+

+---------+                          +---------+
| alice's |                          |  bob's  |
| laptop  |                          | laptop  |
alice   +---------+                          +---------+   bob
o                                                         o
-|-    +---------+                          +---------+   -|-
/ \    | alice's |                          |  bob's  |   / \
| home pc |                          | home pc |
+---------+                          +---------+

+---------+                          +---------+
| alice's |                          |  bob's  |
| work pc |                          | work pc |
+---------+                          +---------+


We now have many problems. Two important ones are:

1. How one should send an e-mail to other if the receiver now has many computers? What computer should send it (and what should receive)?
2. People are now admins of their own set of computers and the internet wizards have no control of it, like they had on the mainframes age. We then had started to have problems with spam, etc.

So, many other communication tools showed up: social networks, VoIP software, etc., much of them centralized stuff. But the good and old e-mail survived! Extensions were developed (while keeping protocols backward compatible) to make it safer and avoid spam. Now we have something like this:

.          +----------+                         +----------+
|  alice's |                         |  bob's   |
|smartphone|                         |smartphone|
+----------+                         +----------+
. SMTP port 25
+---------+             :            +---------+
| alice's |  +-------+  : +-------+  |  bob's  |
| laptop  |  |foo.com|  : |bar.com|  | laptop  |
alice   +---------+  |  mail |----|  mail |  +---------+   bob
o                  | server|    | server|                 o
-|-    +---------+  +-------+    +-------+  +---------+   -|-
/ \    | alice's |  /                   \   |  bob's  |   / \
| home pc |--.                    -- | home pc |
+---------+  :                       +---------+
:
+---------+   . SMTP port 587        +---------+
| alice's |     to send              |  bob's  |
| work pc |     POP3 to receive or   | work pc |


Mail servers still use port 25 to communicate with each other. But users use other ports and protocols developed later to fetch e-mails from the mail server to their devices, or to send e-mails [2].

 [2] There are also other manners to access your e-mail. With the web, web clients sprung up and it is nowadays one of the most common ways to access your inbox (not to mention proprietary API's that most applications use). Gmail is a good example of that. There is also web clients that are free software like Roundcube but, as I said earlier, they are not the focus of this article. You can refer to the great Home mail server with Postfix + Dovecot on NetBSD for more information.

What we are learning in this guide is to make the mail server communicate with other mail servers and also configure software to allow users to fetch their mail server.

## Part 1: A simple mail server

In this part we are going to make our mail server talk to other mail servers. For this to work, we are going to need support for other system. Probably, the most important one is the DNS.

Before we begin, let's make some important definitions:

1. We are going to setup a server called example.com. Our main user is "alice" (so we want to setup the e-mail alice@example.com. In a company environment, DNS, mail server, IMAP server, etc., is kept separately for security and other reasons. Since it is a personal server, we are going to install every service we need to the same machine.
2. You can do it in your home, but you we'll need a fixed IP mainly because of Reverse DNS lookup. I find it easier (and maybe cheaper!) to just rent a small virtual private server. Let's suppose our IP is 123.45.67.89.

### Setting up DNS with BIND

Important

Until now, I was just a DNS user. My first BIND configuration is this and there may be (serious) errors. Please, let me know if you see something wrong!

We are not enter the details of how DNS works. I had a user view of that and I had to discover a bit more to configure the e-mail server, but is still a very limited view. I will, though, list the wrong concepts I had because I believe other people may have the same difficulties.

Note

Everything I know here is because I read the great book DNS and BIND, by Cricket Liu and Paul Albitz. It explains very well what DNS is and how to configure BIND (probably the most used DNS server). At the time I write this I have read only the first 5 chapters, which was enough for a basic setup, but the whole book seems promising!

I though that the sole purpose of DNS was to to tell a requester the IP address of a given name, but it can inform many more things and some of them are used for the e-mail service.

Let's make some tests. Type in your terminal:

$dig +short gmail.com 216.58.202.165  The dig tool [3] [4] will show the IP address associated with the gmail.com domain. This is called the A registry (A stands for Address). Looking for the A registry is the default but we can issue the command with it explicitly: $ dig +short A gmail.com
216.58.202.165

 [3] The dig(1) tool is a DNS lookup utility. There are more references to the more famous nslookup(1) but I prefer using dig. We are going to use the +short flag because we are not interested in the details of DNS query for now.
 [4] It will use the default DNS server configured in your system (in /etc/resolv.conf. If you want to use another, use the @ character. For instance, to run the same command using the Google DNS server, type: dig @8.8.8.8 +short gmail.com.

What does it mean? If we send an e-mail to foo@gmail.com, will it be sent to IP address 216.58.202.165? No! The mail system actually queries for another DNS registry called MX. Let's take a look:

$dig +short MX gmail.com 40 alt4.gmail-smtp-in.l.google.com. 20 alt2.gmail-smtp-in.l.google.com. 30 alt3.gmail-smtp-in.l.google.com. 10 alt1.gmail-smtp-in.l.google.com. 5 gmail-smtp-in.l.google.com.  Those are the servers that can receive e-mails when we send something to the Gmail servers. We also need to specify this in our DNS setting. Will we run our own server? Yes, and we'll show how to make that with BIND. Note A note on registrars: the company you registered your domain in may have their own DNS server. The company where I registered mine had, but it was buggy and had a broken interface, so I decided to setup my own DNS server. First, configure the /etc/named.conf file. NetBSD defaults are OK, but you have to add your own zone: zone "example.com" { type master; notify no; file "example.com"; };  Now we need to create the zone file, /etc/namedb/example.com: $TTL    3600
@       IN      SOA     example.com. hostmaster.example.com.  (
2017090901      ; Serial
3600            ; Refresh
300             ; Retry
3600000         ; Expire
3600 )          ; Minimum
IN      NS      ns.example.com.
IN      NS      puck.nether.net.
IN      MX      1 mail.example.com.
mail            IN      A       123.45.67.89
ns              IN      A       123.45.67.89
example.com.    IN      A       123.45.67.89


This is where magic happens. We are not explaining every detail of this file. For more information, refer to the chapter 4 of DNS and BIND. Some lines deserve special attention, though:

1. Line 2 (the one that starts with "@"): Just add your domainname. Don't forget the trailing dot [5].

2. Lines 3 to 7: that follow are for DNS synchronization between servers. Defaults are ok. Increment the first number whenever you make a change to this zone, so other DNS servers know when you have newer information to be synced.

3. Line 8: This is important: Here is where we configure the NS registry for this DNS entry. "NS" stands for Name Server and it is a server that will be queried when a client wants to know the IP address of any subdomains of example.com domain.

It is important to explain things a bit: let's imagine someone is querying for server "smtp.example.com". Your ISP DNS server knows that the server responsible for the zone (in DNS parlance, it is called "authoritative server" for a given zone) is "ns.example.com", i.e., it has every information about any subdomains for "example.com". So things work like this: user queries his ISP DNS that queries "ns.example.com" that answers to the ISP DNS that answers to the user. That is why we need to set this line.

In your registrar, you need to add the nameservers responsible for your zone. In my registrar I add the name of the nameserver ("ns.example.com") and its IP addres.

4. Line 9: This is the name for the secondary DNS server. Registrars usually [6] require at least two DNS servers. Unfortunatelly we can't afford a second one. So we use one of the free secondary DNS servers that exist. I use PUCK Free Secondary DNS Service but you'll find others in Free Secondary DNS list. After you setup your secondary DNS server, it will easily synchronize information from the master server ("ns.example.com"). [7].

5. Line 10: The MX record is where we specify our mail server. Any message addressed to anybody@example.com is delivered in one of the MX servers. You can have more than one server (if some is down, others can take over -- just remind the example we did before: dig +short MX gmail.com). But since we can't afford a backup server, we'll work with one only.

6. Lines 11 to 13: Now we specify the A records, i.e., the IP address of each server. Unfortunatelly we have no money and they are actually the same! Could they have the same domain name? Yes, but we would have to give different names if we use different IP addresses for each server later, so it is better to have different names even if they have the same IP to make further improvements easier. [8].

 [5] We use the trailing dot whenever we want to specify an "absolute domain" (like an absolute path in filesystem that starts with a trailing slash, absolute domains end with a trailing dot). Again, refer to chapter 4 of DNS and BIND.
 [6] See that we have relative domain names ("smtp" becomes "smtp.example.com.") and absolute ("example.com." is just "example.com.").
 [7] Always?

Important

At the time of this writing, I was not sure if I should include the PTR records. The company where I rent the VPS already had a reverse DNS record for this machine (with a different address, though). Is it enough? If someone know that, please, let me know.

After all this, we just add BIND daemon (named) to /etc/rc.conf and start it:

# echo 'named=YES' >> /etc/rc.conf
# /etc/rc.d/named start


### Setting up Postfix

We now have a our DNS records working. Without them (specially the MX records) it would be impossible to have a mail server. [9]

 [9] Or at least very difficult? Mail servers predate DNS, but nowadays I find it very difficult to setup one without DNS.

Now, let's add configuration to the Postfix mail server, that comes by default in NetBSD. Again, almost all configuration here follows the rules described in Home mail server with Postfix + Dovecot on NetBSD.

Let's first change /etc/postfix/main.cf file. We are going to list every option to add to the file and explain them:

myhostname = smtp.example.com
mydestination = $mydomain,$myhostname, localhost.$mydomain, localhost home_mailbox = Maildir/ mynetworks_style = host  1. Line 1: Just the domain name and the name of this host. [10]. 2. Line 2: Who we want to deliver messages to. 3. Line 3: We want to user Maildir to stored e-mails instead of more traditional (and limited) mbox. 4. Line 4: We can specify who we can relay e-mail from. By default, it is "subnet". If you are hosting your e-mail server in a VPS [11], like me, other machines on the same host can be on the same network, so we set it to host, allowing only ourselves to relay e-mail. See [HILDEBRANDT2005], page 55).  [10] Does it have to be the same of the output of the hostname(1) command or does it have to be the entry of it in the DNS?  [11] Virtual Private Server There is one more setting you have to make. Now in file /etc/postfix/master.cf. Change line: #smtp inet n - n - - smtpd  To: smtp inet n - n - - smtpd -o myhostname=example.com  Important Replace example.com by the name you get from reverse DNS lookup. Sometimes it is configured by the company you contract your VPS from. To discover the domain name related to your IP address, use the dig(1) tool: $ dig +short +x 123.45.67.89


This is really a very important setting since that receivers will query for address related to your IP and compare this to your e-mail headers Postfix add using this setting. If they don't match, mail serves closes the connection and don't receive the message. This is a common protection against spam that fakes DNS names.

Voilà! This configuration is enough for a 1982 configuration. And, believe or not, you can send and receive e-mails to and from your mail server. There are two problems, though: one is that we'll send plain text message, without any cryptography. The other is that chances that your e-mail fall into the receiver's spam folder are high because we are not using modern stuff here. Let's learn how to configure them.

## Part 2: Improving our mail server

### Enabling SSL for Postfix

We are going to use openssl(1) to create our public and private keys. As Home mail server with Postfix + Dovecot on NetBSD pointed, let's create our keys.

We can do that with two ways. The first one is to have a self-signed certificate and the second one is ask for a company to generate the certificate for you.

Generating a self signed certificate is simple:

# mkdir -p /etc/ssl/{certs,private}
# cp /usr/share/examples/openssl/openssl.cnf /etc/openssl/
# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/example.com.key -out /etc/ssl/certs/example.crt
# chmod 600 /etc/ssl/private/example.com.key


If you want a company to sign your keys (which people recommend [12]), you'll probably have to pay. There are, though, some that provide that service for free or cheap enough. Check StartSSL for an example.

Note

If you use StartSSL, they will generate the certificates and give you a zip file with them. Inside the zip file there are already directories with the certificates ready for common applications like Apache. We have not a specific directory for Postfix but we have a directory called OtherServer. You will find three files there: 1_Intermediate.crt, 2_example.com.crt and root.crt. The certificate wil are going to generate is a concateation of 2_example.com.cr and 1_Intermediate.crt (attention: the order matters). So, generate that:

# cat 2_example.com.crt 1_Intermediate.crt > /etc/ssl/certs/example.com.crt


After that, we should add more settings to Postfix /etc/postfix/main.cf file:

smtpd_tls_security_level=may
smtpd_use_tls=yes
smtpd_tls_auth_only=yes
smtpd_tls_cert_file=/etc/ssl/certs/example.com.crt
smtpd_tls_key_file=/etc/ssl/private/example.com.key
smtpd_tls_loglevel=1


You are not explaining these options since most of them are self-explanatory. But if you want to know the details, check the Postfix documentation.

Now, just start postfix!

# /etc/rc.d/postfix start


You are now able to send and receive e-mails from and to this machine. Make a test: 1. login into this machine (this is like connecting via a serial line to your mainframe! :-]), chose an existing e-mail address (your's or of a friend) and send it an e-mail:

$echo "message body comes here" | mail -s "subject" myfriend@email.com  It will receive the e-mail. If it don't, check logs for postfix in /var/log/maillog. Also, never forget to check the spam folder. Some later configurations are still needed if we want reduce the probability of letting our e-mails be dropped to the spam folder. ### Allowing devices check our e-mail with IMAP We are now able to send and receive people e-mail. E-mail you receive shall appear in /var/mail/yourusername. We are in a 1982 configuration with some extensions: SSL mainly. Without it is almost impossible to exchange e-mail with other servers nowadays. But we don't want that configuration. We don't want to login to this machine in a console or ssh whenever we need to check our e-mails. Also, we want to sync all our devices to fetch and send e-mails. So we are going to configure our IMAP server. As I mentioned, Postfix is e-mail server, i.e., it handles the SMTP protocol (and extensions) and does e-mail exchanging with other servers. It doesn't provide any support for IMAP. To make IMAP to work, we'll need to install another daemon and configure Postfix to talk to it. We are going to install Dovecot. For a bit more detailed guide on how to configure Dovecot with SASL with Dovecot, see Postfix with SASL. Build it from source via pkgsrc (mail/dovecot2) or just install the binary package. Check pkgsrc documentation for details. Configuration of Dovecot is straightforward. Open the /usr/pkg/etc/dovecot/dovecot.conf file in your favorite editor. First, change the line: protocols = imap pop3 lmtp  to: protocols = imap lmtp  We are not going to use POP3 (anybody uses?). Now, add or uncomment the following lines: ssl = yes ssl_cert = </etc/ssl/certs/example.com.crt ssl_key = </etc/ssl/private/example.com.key  Now edit file /usr/pkg/etc/dovecot/conf.d/10-mail.conf and add (or uncomment) this line: mail_location = maildir:~/Maildir  It indicates to store users' e-mails in their home directories, using the Maildir format. Now, just start Dovecot: # cp /usr/pkg/share/examples/rc.d/dovecot /etc/rc.d # echo 'dovecot=YES' >> /etc/rc.conf # /etc/rc.d/dovecot start  There are one important things to note here: with this setup, Postfix and Dovecot are not talking to each other. What happens here is that Postfix deliver e-mails to users' home directory and Dovecot reads from the same location. It works but this setup has some limitations, like not allowing e-mail filtering. To know the advantages and how to setup LMTP take a look in section Making Postfix talk to Dovecot via LMTP. ### Allowing devices to send e-mail using SMTP We can now configure our devices to fetch e-mail from our server using the IMAP protocol but we cannot send yet. The SMTP port Postfix is using, port 25, is solely for other server communication purposes. But, for this, Postfix will enable us to use another port, mainly 587. This is called submission, i.e., e-mail submission and port 587 is reserved for that. [13]  [13] Other ports and proposals for e-mail submission have existed. See this link for more information. Configure that is also pretty simple. First, open file /etc/postfix/master.cf again and uncomment the submission line and add some lines after that. It will be like: submission inet n - n - - smtpd -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_sasl_type=dovecot -o smtpd_sasl_path=private/auth -o smtpd_sasl_security_options=noanonymous -o smtpd_sasl_local_domain=$myhostname
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
-o smtpd_recipient_restrictions=reject_non_fqdn_recipient,reject_unknown_recipient_domain,permit_sasl_authenticated,reject


Important

Don't remove spaces at the beginning of the line. They mean the continuation of the command on the line before.

Also add this line to /etc/postfix/main.cf:

smtp_tls_security_level = may


Realize that smtp* is different from smtpd* (see the trailing d) settings. The former is for the submission port (587), the later is for SMTP port (25).

Restart Postfix:

# /etc/rc.d/postfix restart


Now you have a full e-mail server! But it is not done yet, we'll have to configure all the extensions to it so we don't be marked as spam.

### Making Postfix talk to Dovecot via LMTP

As we stated in section Allowing devices check our e-mail with IMAP, Postfix and Dovecot just write/read e-mails from the same location but they don't talk to each other. This is simple but it doesn't allow us to use any of the advanced features Dovecot provides. To fix this, we are going to setup both Postfix and Dovecot to use the LMTP protocol. So, Postfix will deliver e-mail to Dovecot explicitly:

e-mail                  delivers
incoming   +---------+  e-mail    +---------+
---------->| Postfix |----------->| Dovecot |
via        | server  |  via       | server  |
port 25    +---------+  LMTP      +---------+


Let's first start setting up Dovecot. It is a very simple configuration. According to the Dovecot with Postfix LMTP, we need to change file /usr/pkg/etc/dovecot/conf.d/10-master.conf, configuring LMTP service:

service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}


In this example, we are using a Unix Domain Socket (it is useful specially if Dovecot is installed at the same machine as Postfix), but you can use a inet socket too. See Dovecot with Postfix LMTP for details and don't forget to set permissions and ownership correctly.

It is enough at the Dovecot side. Now let's do that at the Postfix side. Edit file /etc/posfix/main.cf. First, since we are delivering our e-mail to Dovecot via LMTP, setting home_mailbox (that we configured at Setting up Postfix is now useless. You can commment or exclude it. Now we have to add the mailbox_transport setting:

mailbox_transport = lmtp:unix:private/dovecot-lmtp


You can also delete the home_mailbox from /etc/main.cf because we won't use it anymore.

Note

By default, user authentication is done with the full e-mail (i.e., clients should provide the full e-mail for login information, and not only the username. We recommend keeping this as is, specially if you want to user virtual domains as we explain in Setting up virtual users (and virtual domains). If you really change to change it, according to this page, edit /usr/pkg/etc/dovecot/conf.d/10-auth.conf and change variable auth_username_format:

auth_username_format = %Ln


Now, don't forget to restart Postfix and Dovecot:

# /etc/rc.d/postfix restart
# /etc/rc.d/dovecot restart


To make sure your LMTP setting works, always take a look at the /var/log/maillog file.

### E-mail filtering with Pigeonhole (Sieve)

The Sieve programming language was created specially for e-mail filtering. In the Dovecot world it is done by the Pigeonhole project. We'll describe how to setup it here.

First, make sure your LMTP setting works. See Making Postfix talk to Dovecot via LMTP for more information. Then, install the mail/dovecot2-pigeonhole package.

Once it is working, let's enable the sieve plugin for LMTP. Edit file /usr/pkg/etc/dovecot/conf.d/20-lmtp.conf, locate line starting with mail_plugins and add it:

protocol lmtp {
mail_plugins = $mail_plugins sieve }  And just restart Dovecot: # /etc/rc.d/dovecot restart  Simple, ahm? Now, include your rules in your ~/sieve file. For example: require ["fileinto"]; if header :contains "subject" "test" { fileinto "test"; discard; stop; }  This will make that every e-mail with the "test" word in the subject header is put into the "test" mailbox and discarded from inbox. To validate your sieve script, use the sieve-filter(1) command. A good example of this command is here: $ sieve-filter -v -C -u test /path/to/sieve/example.sieve 'INBOX'


It will print what the filter would do in all messages that are in box INBOX. If you want to execute the rules, pass -e and -W flags. But be careful! The man page states that it can be very destructive. So, be sure of what you are doing!

For more information about configuring the Pigeonhole Sieve plugin, see Pigeonhole/Sieve/Configuration and for troubleshooting see Sieve Troubleshooting and never forget to look at /var/log/maillog.

### Setting up virtual users (and virtual domains)

For now, we now have a full workable e-mail server with different capabilities plugged in, but we are still using system users. If you want to add a new e-mail user you'll have to add it as a Unix system user, creating all the environment for a normal user, with home directory, login shell and so on. If you need to do so but don't want to let your users login via ssh, you can set their shells to /sbin/nologin.

But this solution doesn't scale. For this, both Postfix and Dovecot provide a feature called "virtual users" that you can use. There is also one more reason to use virtual users: you can also set virtual domains together and provide e-mail service for more than one domain, in the same server! A similar feature to virtual hosts of HTTP servers.

Postfix virtual users are completely unrelated to Dovecot virtual users. So, we can have different setups:

1. Postfix without virtual users and Dovecot with virtual users.
2. Both Postfix and Dovecot with virtual users.

The first one, AFAIK, let you use system users but allow you to setup a different password that users use for IMAP and system access. The second one is useful if you want to provide true virtual e-mail users and also virtual domains support. We are going to explain how to setup the second case.

The page how to build a virtual user e-mail server has helped a lot on how to understand and setup this. We are going to use the simple "passwd-file" method, on which we store user and password information in a plain text file, but several other methods, like storing users and password information in SQL database or LDAP.

First, we need to make a few changes to /etc/postfix/main.cf:

myhostname = other.com
virtual_mailbox_domains = /etc/postfix/vhosts
virtual_mailbox_base = /home/maildeliverer
virtual_mailbox_maps = hash:/etc/postfix/vmaps
virtual_uid_maps = static:1008
virtual_gid_maps = static:1008
virtual_transport = lmtp:unix:private/dovecot-lmtp
# Also, you can now delete line mailbox_transport we added before.


So, let's explain what each variable mean:

myhostname
We need to set myhostname to a different domain we are going to use in our e-mails. All virtual domains are all going to the /etc/postfix/vhosts file.
myorigin
Set it to either $mydomain or$myhostname or, if the hostname is different from the domain you want to get e-mail delivered to (this can happen if you are using virtual domains), set it to the domain name. This is good to delivering e-mail correctly if internal tools (such as cron(8) wants to send e-mails to users like root. After doing that, you still probably don't have a "root@example.com" account, so you can redirect it to whoever you want by appending a line to /etc/postfix/aliases (See [HILDEBRANDT2005], page 20.). Check Configuring aliases for our virtual users for more information.
virtual_mailbox_domains
The file with a list of virtual domains we are going to work with. See below.
virtual_mailbox_base
The directory where e-mails will be stored. Let's create a system user for this and call it "maildeliverer" and let our virtual users boxes located at /home/maildeliverer.
virtual_mailbox_maps
The file with a list of virtual users. See below.
virtual_uid_maps
The uid (user id) virtual mail boxes will be created with. The uid of "maildeliverer".
virtual_gid_maps
The gid (gropu id) virtual mail boxes will be created with. The gid of "maildeliverer". In NetBSD, every new user is, by default, part of the "users" group (gid = 100) but it is wise to let a group that don't mix with other users groups. In this example we create a user with gid = 1008.
virtual_transport
Now that we are using virtual users we have to set this variable. It has the same value as the mailbox_transport variable. By the way, we are not going to need mailbox_transport anymore, so you can delete it.

The content of /etc/postfix/vhosts is just a list of virtual hosts per line:

example.com
example2.com


And the content of /etc/postfix/vmaps is a list of records per line, where each record is compound with user login information and where the e-mail will be stored:

alice@example.com       Maildir-alice
bob@example.com         Maildir-bob


The mailboxes will be created relative to value of variable virtual_mailbox_base with uid and gid set to virtual_uid_maps and virtual_gid_maps.

After you have created the /etc/postfix/vmaps file, enter the directory and execute the postmap program:

# cd /etc/postfix
# postmap vmaps


It will create file vmaps.db that postfix will use to load virtual user information.

It is worth to note that, although we have options such as virtual_mailbox_base, virtual_uid_maps, virtual_gid_maps and also the mailboxes in file /etc/postfix/vmaps, they are not being used (really?). We added this to show how postfix can work with virtual users alone but we are passing e-mails to Dovecot with LMTP, setting virtual_transport variable. So, those setting are just ignored (can anybody confirm this?).

So, let's now see how we configure Dovecot virtual users.

In file /usr/pkg/etc/dovecot/conf.d/10-auth.conf, disable or comment line:

#!include auth-system.conf.ext


!include auth-passwdfile.conf.ext


Then, let's configure file /usr/pkg/etc/dovecot/conf.d/auth-passwdfile.conf:

mail_location = maildir:/home/maildeliverer/%n/Maildir

passwd {
driver = passwd-file
}

userdb {
driver = passwd-file
}


Note

This configuration let us configure virtual users mailbox at /home/maildeliverer/<username>/Maildir. %n is replaced by the username part of the e-mail (the one before @). If we have different users in different domains that have the same user name (e.g.: alice@foo.com and alice@bar.com) you'd better include the domain name in the mail_location information, like this:

mail_location = maildir:/home/maildeliverer/%d/%n/Maildir


We are pointing to file /usr/pkg/etc/dovecot/users. Let's create this file with this content:

alice@example.com:{CRAM-MD5}e02d374fde0dc75a17a557039a3a5338c7743304777dccd376f332bee68d2cf6:1008:1008::/home/maildeliverer/alice


The format of this file is:

username@domain:crypt-password:uid:gid::virtualuserhome


To generate the password part, use the command:

# doveadm pw


Finally, restart Postfix and Dovecot:

# /etc/rc.d/postfix restart
# /etc/rc.d/dovecot restart


### Configuring aliases for our virtual users

Once you have configured virtual users you can now configure virtual aliases, which are straightforward. You first need to tell where your aliases file is. First, add the following line to /etc/postfix/master.cf:

virtual_alias_map = hash:/etc/postfix/virtual_aliases


Now, create the /etc/postfix/virtual_aliases file with the content you wish:

secondary@example.com   alice@example.com


In NetBSD, if you want to receive e-mails from root (the system security framework itself sends e-mails to the local user root), also add to the virtual_aliases file:

root@example.com        foo@example.com


To make root@example.com work, do not forget to set myorigin option. See the Setting up virtual users (and virtual domains) section.

Now, generate the /etc/postfix/virtual_aliases.db file with:

# postmap virtual_aliases


They might start working without the need to restart postfix.

Note

Aliases (which you'd set with the alias_maps option) and virtual aliases (which you'd set with the virtual_alias_maps option) are different things. There is a nice answer about that here.

### Mailbox permissions in Dovecot

I don't like to let clients create their own mailboxes. Some are broken and create things without my permission. Also, when migrating a different account to your new server, or changing your mailboxes names, you may want to restrict this.

In file /usr/pkg/etc/dovecot/conf.d/20-imap.conf, make sure you have the following line:

mail_plugins = acl


And add the plugin configurations in /usr/pkg/etc/dovecot/conf.d/90-acl.conf:

plugin {
acl = vfile:/usr/pkg/etc/dovecot/global-acls:cache_secs=300
}


In this example we specify a global ACL file but other settings are possible. The contents of this file is something like this:

* user=username lrwstipek


Where each letter at the end of the line means something. See the Dovecot wiki page about ACL for the meaning of the letters. The existence of k letter, for instance, grants mailbox creation permission so if you want to forbid users to create mailboxes, remove the k letter and restart the Dovecot server.

### Changing default message size limit

By default, Postfix limits message size at 10 MB which is fine for most uses but it is a low limit for today's nonsense web age (see this and this). It will not deliver your message and send an error e-mail to the sender, telling her/him the message is too big.

To change that, just add or change parameter message_size_limit in main.cf:

# 30 MB message size limit -- oh god!  how we end up here?!
message_size_limit = 31457280


Note that the size is in bytes.

## Part 3: Improving it even more: E-mail authentication

Having a mail server configured with SSL is an important step to make your server be recognized as "legit" for other mail servers. Other crucial feature is reverse DNS. But those features are not enough. As time passed, other features were developed to make spam and fake e-mails difficult. The ones we are going to show here are SPF, DKIM and DMARC.

We are not going to explain what SPF, DKIM and DMARC are. For a great explanation about that see the page Why your Marketing Email will Land in my Spam Folder. For a discussion about this page see this entry in Hacker News.

### SPF

SPF stands for Sender Policy Framework and it is a simple setting you add to your DNS setting so e-mail filters can query who is able to send e-mails signed as your domain.

Basically, you list in a DNS entry what servers are able to send e-mail using your domain. If we are using one simple e-mail server (if you can afford, choose to have redundancy) it is as simple as:

example.com.    IN      TXT     "v=spf1 mx -all"


With mx, it will look for domains of all MX servers you declared in your DNS.

Pay attention to the -all part. Basically, this line tells you that mail host, and only mail, can send e-mail. Everything else should be ignored.

Since we changed our DNS zone, don't forget to increase the zone file serial number, so this change gets.

You can debug SPF problems with the SPF Record Lookup page.

You can now use dig to check for your SPF record:



TODO

## Reputation Block Lists (DNS Block Lists)

Reputation Block Lists, or RBLs, records spammers IP addresses so legit mail servers (like ours!) can protect against spammers. Although they are also called DNS Block Lists, they don't use domain addresses, but only IP addresses instead.

To query a block list you don't need any special tool besides your normal DNS tools. We are going to use host(1).

The block lists works the following way: you query, via DNS, for the reverse IP of the probable spammer.

There are several block lists. We are going to use b.barracudacentra.org. For information about this list, check their website at http://www.barracudacentral.org/ . They ask you to register your IP address (or the IP address of your mail server) but I could query it without having to register myself first,

Once I received an spam from a e-mail server with IP 89.46.222.110. To check whether or not it is listed in the block list, reverse the IP address and prepend it to the block list address. Then, issue a DNS request:

$host 110.222.46.89 b.barracudacentral.org 110.222.46.89.b.barracudacentral.org has address 127.0.0.2  It returned IP address 127.0.0.2. It means the IP is listed on the block list. To add this check in postfix, add the following option to /etc/postfix/main.cf: smtpd_client_restrictions = reject_rbl_client b.barracudacentral.org  Or just append it to the smtpd_client_restrictions if you have other options as well. ## Spam filtering TODO ## Part 5: Not necessary (but useful) things ### Using recipient_delimiter Posfix and Dovecot have a powerful feature that can be set with variable recipient_delimiter. You can setup a character to that text between the character and @ is stripped before delivering the e-mail. It is useful for address of the form: foo+bar@example.com  If recipient_delimiter is +, the e-mail will be delivered to user foo@example.com. I use it mainly to sign up to sites (e.g. foo+sitename@example.com), so I know if some of them is selling my e-mail to spammers, but it can be used to anything you can imagine, like delivering the e-mail to folders which name is the same name of the part that was stripped, like this. To enable that, configure recipient_delimiter in /etc/postfix/main.cf. If you are using Dovecot virtual users, don't forget to uncomment or add the recipient_delimiter variable in /usr/pkg/etc/dovecot/conf.d/15-lda.conf. Make sure the delimiter character for both postfix and dovecot configuration are the same. ### Implementing the "snooze" feature Some e-mail providers or plugin developers offer the "snooze" feature for e-mail: you can "set" your e-mail to hide in your INBOX and it will return at a specific time later. It is useful if you have a workflow like mine: I let all pending question in my INBOX but I will do some of them some days later. I don't want them to clutter my INBOX, though. Unfortunatelly solutions focus on specific products. So, we just use doveadm move. We first create a set of boxes that we want to use as "temporary boxes": "1d", "2d", "3d", etc. For instance, if I move an e-mail to the "7d" box, the script will return it back to the INBOX 7 days later. It can be implemented easily with a shell script. See: #!/bin/sh doveadm -v move -u user INBOX mailbox 1d all doveadm -v move -u user 1d mailbox 2d all doveadm -v move -u user 2d mailbox 3d all doveadm -v move -u user 3d mailbox 4d all doveadm -v move -u user 4d mailbox 5d all doveadm -v move -u user 5d mailbox 6d all doveadm -v move -u user 6d mailbox 7d all doveadm -v move -u user 7d mailbox 14d savedbefore 6days doveadm -v move -u user 14d mailbox 30d savedbefore 15days doveadm -v move -u user 30d mailbox 60d savedbefore 30days doveadm -v move -u user 60d mailbox 90d savedbefore 30days  Then I add it to the crontab: 0 3 * * *$HOME/do_snooze.sh


So it runs every day, at 3 AM.

Basically it just moves an e-mail from "7d" to "6d" and then "6d" to "5d" the next time the script is runs. The order is very important. If it was reverted, on the same execution the e-mail would be moved from "7d" to "6d", then from "6d" to "5d" and we want this to happen only once per day.

Other mailboxes, "14d", "30d" and "60d" don't have intermediate boxes to rely on, so we use another feature ("savedbefore") that takes in account the time the e-mail was saved in the box.

This solution has disadvantages: it is not possible to exactly know in what box the e-mail is (unless you implemented a search feature that you can run from a web client, for example), but it is a simple solution and it works regardless of the client.

### E-mail rewriting with Postfix

Postfix has advanced features, including some for e-mail rewriting. Those include cleanup(8) methods, header_checks(5), milters and more.

i'm now aware of everything nor the details of them. My need is a bit strange: I have different accounts (can be different or the same domain) for this Postfix server. My personal account and other accounts for mail list discussion. I don't want to use my personal account for mail list, but I also don't want to setup different accounts in my MUA: I want to receive everything in my personal account and I want to answer them from my personal account as well, but, when answering the e-mail, I want it to be rewritten as it appears to have been sent from another account.

an example: my personal e-mail is alice@example.com, but I also have foolist@example.com to send and receive e-mails to and from the Foo project.

postfix people on IRC channel at freenode recommend me not to do that and told me setting up this configuration on MUA was a better choice. I insisted, though.

let me first show you what I've tried but it didn't work:

aliases
Setting up a Postfix alias (foolist@example.com to alice@example.com) would work at the receiving part, but not for sending, since it translates foolist@example.com to alice@example.com, but not the other way around.
header_checks looks great and can indeed rewrite headers (writting headers, not the body, is enough), but I soon discovered (link) header_checks(5) rules don't "save state" between lines, i.e., I cannot mix on the same rule both To: and From: headers.

after some researching, I decided to investigate how to write a milter and came across this page: Postfix After-Queue Content Filter.

this page has examples on how to make your own filter for rewriting an e-mail. anyway, I'm going to describe what I did.

myrewrite unix  -       n       n       -       10      pipe  flags=Rq user=myrewrite null_sender= argv=/home/myrewrite/myrewrite.sh -f ${sender} --${recipient}


i called it "myrewrite". Following Postfix After-Queue Content Filter, for security reasons, I created a dedicated user for this also called "myrewrite".

now, still at /etc/postfix/master.cf, you should add a -o option to the server that you want it to call the myrewrite.sh script. The Postfix after-Queue Content Filter guide appends it to the smtp server, but since we want it to run just after we send it, we are going to append it to submission:

submission inet ... do not change what comes here ...
... nor on lines within ...
-o content_filter=myrewrite:dummy


We added the last line. Please, note there exists two white space before -o, meaning that we are appending it to the line just before.

Also, don't forget to create new account (for the foolist@example.com e-mail), no matter if you are using local Unix accounts, Postfix vmaps or Dovecot accounts.

Let's finally take a look at the myrewrite.sh script:

#!/bin/sh

# Based on example from http://www.postfix.org/FILTER_README.html

SENDMAIL="/usr/sbin/sendmail -G -i"

# E-mails we want to detect in "To:" header to enable rewritting.
to_field='maillist@project.org'

# Translation rules for e-mail
from_field_old='alice@example.com'
from_field_new='foolist@example.com'

# Temporary mail file.
mailfile="/var/spool/myrewrite/mail.$(date +%Y%m%d%H%M%S).$$" # Clean up when done or when aborting. trap "rm -f$mailfile" 0 1 2 3 15

# Does the rewritting
rewrite ()
{
# Awk script that does the actual rewrite.  We pass variables defined in
# this shell script to awk variables.
awk -v from_field_old="$from_field_old" \ -v from_field_new="$from_field_new" \
'
BEGIN {
}

/^[^[:space:]]+:/ && on_headers == 1 {
# variable holds the name of the current header.  It
# also works for multiline headers.
split($0, fields, ":") header = fields[1] } /^$/ {
# An empty line marks the beggining of the body part of
# the message.  We are not in headers region anymore.
}

{
# For every line (header and body) do:
line = $0 # Replace the old e-mail for the new e-mail gsub(from_field_old, from_field_new, line) # Remove "DKIM-Signature" header (i.e., dont print it). # This is specially important for two reasons: # # 1. We are going to rewrite the e-mail (body included) # so the signature changes. The signature just added # is not valid anymore. # # 2. When sending the modified e-mail using sendmail, # another DKIM-Signature will be appended (specially # if we are using another domain). If we dont # remove the first one, the e-mail will have two # DKIM-Signature. if (header == "DKIM-Signature") next # Finally, print the modified line. print line } ' "$mailfile" > "$mailfile.tmp" mv "$mailfile.tmp" "$mailfile" } # Writting the receiving e-mail to the "$mailfile" file.
cat >"$mailfile" # Rewritting is disabled by default rewrite=no # If we see we want to send to "$to_field", rewrite
grep -q "^To: .*$to_field" "$mailfile" && rewrite=yes

# If rewritting is enabled
if [ "$rewrite" = 'yes' ]; then # Rewrite the e-mail rewrite # The following changes the option list ("$@") to replace
# $from_field_old for$from_field_new.  Rewritting headers is not enough
# because the "Return-Path" header is writting using the value passed to
# -f flag of sendmail.
set -- $(echo "$@" | sed "s/$from_field_old/$from_field_new/g")
fi

# Finally, send it.
$SENDMAIL "$@" < "$mailfile" exit$?


## References

The important references cited in this guide are:

Books:

 [HILDEBRANDT2005] The Book of Postfix, 2005, by Ralf Hildebrandt and Patrick Koetter