SMTP Mail Routing using Exim and LDAP

kos-igetenoughspamasitis-thankyou@bastard.net

Introduction

This document describes how to configure and operate an SMTP mail hub which provides support for multiple domains, authenticated SMTP, POP3 and delegated domain administration. An emphasis is put in the security and scalability of the solution.

Goals

Software components have to be freely available, well understood and maintained
Support for multiple, dynamically allocatable virtual domains
Delegated administration: Each domain ought to be administered separately by a designated group of administrators
Authenticated SMTP and POP3
SSL/TLS support for incoming and outgoing traffic
Support for email aliases: Entities that have no physical mailbox associated with them
Support for virtual users: Entities with physical storage requirements
Mail Spool versatility: optional support for cyrus and maildir
Easy administration through a web interface

Architecture

The basic components of the system are outlined below:

Mail Transport Agent: The core of the system, the MTA accepts mail, manages the mail spool and delivers mail to its destination.
POP3/IMAP Server: Such servers provide a standards-based interface to mail storage on the server.
Configuration store: The information store that contains all instructions needed to deliver mail to a managed destination.
Mail spool: The information store to cover mail storage requirements.

Implementation

There are many available options that will cover the above requirements. This implementation cookbook concentrates on the following:

Configuration store: OpenLDAP for an LDAP user database
Mail spool: Unix file system with BSD mailboxes
MTA: Exim
POP3: popa3d + LDAP authentication

Below is an outline of the architecture of the system. Mail clients get load balanced across a set of mail servers that consist the first tier of the application. The mail servers obtain configuration information from the LDAP database and access the mail store.

LDAP Implementation

OpenLDAP (http://www.openldap.org/) is a freely available implementation of LDAPv3 with TLS and schema support. LDAP is a good solution for managing read-mostly user databases and the hierarchical object store it implements provides ample support for a virtual domain system. Its widespread support by various applications such as web servers (apache), ftp servers (proftpd), IMAP servers (courier-imap) translates to a viable upward path of only having a single LDAP database for across-the-board authentication and authorization needs. This implementation uses a custom schema for storage of information. It is derived from the corresponding schema that can be found in the Qmail LDAP patches.

In accordance to having all configuration data reside in the LDAP database, the toplevel DN of the database is used for the following:

Global administrators
System parameters (list of domains to accept relay for, blacklisted domains/ips)

Within the toplevel DN, entries to support virtual domains can be found. An overview of the final structure is shown below:

Cynics will be delighted to find that the hierarchical LDAP store is only used to store two levels of configuration: System wide and domain wide. This is true; subdomain support is still there however, as each subdomain can be stored as a full domain. It's ugly but simplifies LDAP client support immensely.

Basic LDAP Objects

The following describes the component objects that have been used. They employ definitions found in the default OpenLDAP schemas core.schema, cosine.schema, inetorgperson.schema and nis.schema. All of the above need to be enabled in slapd.conf for schema validation to let you use them.

top, objectClass: top Placeholder for the root node of the LDAP tree.
```
    dn: o=top
    objectClass:top
    
```

system parameter, objectClass: mailSetting Lame name for a system parameter object. Is used to store all kinds of Exim parameters, such as hosts we accept relaying from, spammer blacklists, backup MX domains etc.

    dn: mailSetting=host_accept_relay,o=top
    objectClass: mailSetting
    mailSetting: host_accept_relay
    mailSettingValue: localhost
    mailSettingValue: 127.0.0.1
    mailSettingValue: 207.46.197.102

toplevel admin, objectClass: person As with all systems, this one needs overall administrators too! The primary function of this administrator who can manage all virtual domains is to create new ones, the rest is usually done by the domain administrators themselves.
```
    dn: uid=admin,o=top
    objectClass: person
    sn: admin
    cn: admin
    userPassword: null
    
```
domain, objectClass: mailDomain A domain entry. Pretty simplistic, huh? It basically flags an SMTP domain for which we handle mail for. Underneath this object we will store all its users and aliases.
```
    dn: dc=domain1.com,o=top
    objectClass: mailDomain
    dc: domain1.com
    
```

alias, objectClass: mailAlias An email alias within a domain. No storage is associated with it, and may expand to more than one entries to implement a poor mans mailing list.

    dn: mailAlias=user1,dc=domain1.com,o=top
    objectClass: mailAlias
    mailAlias: user1
    dc: domain1.com
    mailAliasedName: me@yahoo.com
    mailAliasedName: backup@overthere.com

user, objectClass: mailUser An actual user. Notice that the uid, gid parameters govern what credentials Exim may assume when delivering mail, and mailMessageStore is used to point to the users mailbox.

    dn: uid=postmaster,dc=domain1.com,o=top
    objectClass: mailUser
    objectClass: posixAccount
    uid: postmaster
    dc: domain1.com
    sn: Postmaster
    cn: Postmaster
    userPassword: null
    loginShell: /bin/true
    uidNumber: 8
    gidNumber: 12
    mailMessageStore: /opt/mailspool/domain1.com/postmaster
    mailQuota: 32M
    homeDirectory: /nonexistant

group, objectClass: groupOfNames A domain can be managed by a group of administrators. This is an example group entry:

    dn: cn=admin,dc=domain1.com,o=top
    objectClass: groupOfNames
    cn: admin
    description: Group of Administrators for the domain1.com domain
    dc: domain1.com
    member: uid=kos,dc=domain1.com,o=top

LDAP Access Control

The security of the system is implemented in the LDAP database itself, for maximum versatility and cleanliness of the implementation. As a direct consequence, no setuid CGI scripts or hardcoded passwords are needed to produce a web management interface to this solution.

Password change support Users ought to be able to change their passwords. It is also very nice to have users be able to authenticate, so this gives compare access to the password attribute.
```
     access to attr=userPassword
     by self write
     by anonymous auth
     
```

Domain access Each domain can be managed by a postmaster user, an administrator group and the toplevel admin group.

     access to dn="dc=([^,]+),o=([^,]+)"
     by dn="uid=postmaster,dc=$1,o=$2" write
     by group="cn=admin,dc=$1,o=$2" write
     by group="cn=admin,o=$2" write
     by users read
     by anonymous read

Root access Toplevel admin users have access to the whole tree.

     access to dn="o=([^,]+)"
     by group="cn=admin,o=$1" write
     by users read
     by anonymous read

User modifyable Users can change a few attributes in their object, we don't want to be bothered for spelling errors.
```
     access to attrs=cn,sn
     by self write
     by users read
     by anonymous read
     
```

Generic access Fallback entry: Everything else gets read only access.

     access to *
     by self write
     by users read
     by anonymous read

MTA Implementation

Exim (http://www.exim.org/) is a relatively new offering in the Unix MTA arena. Inspired by the simplicity of Smail, Exim provides a lot of sendmail interface compatibility and a great deal of flexibility: It supports multiple mail spool formats (BSD /var/mail, Maildir, Cyrus) and multiple information stores for configuration (flat file, hashes, LDAP, relational databases).

For this task, Exim is configured to retrieve almost all information from the LDAP database. As of Exim 3.33, the only data that regrettably can't be stored there is primary_hostname (the main hostname of the box) and qualify_domain. They don't really matter in a virtual domain configuration anyway, so set them to something that looks ok for your Received: headers and forget.

Exim is configured to support in-session SSL/TLS (you need a recent version for this) and SMTP authentication to allow relaying. Outlook Express and Netscape Navigator have been tested with SMTP authentication.

The Directors portion of the Exim configuration file dictates the available options Exim has to deliver a particular piece of mail. They are scanned in turn and the first one that declares capability of handling the mail will be given a chance to do so. We configure to support the following:

.forward User forward files may contain a program to pipe mail to, email aliases and so on. They're only relevant if you want to support old skool power users that like to log in and read mail using pine.
aliases A search is done in LDAP for a matching alias entry for that domain. If found, the To: address gets rewritten to contain whatever the LDAP query returned (which may be more than one entry) and the result is requeued for delivery.
users If the user search matches, the physical mailbox location is retrieved from the LDAP object and mail is delivered to that BSD mailbox.
defaultalias If the default alias exists for that domain, it is retrieved and expanded in the same manner as any alias. This allows for a mailbox to receive mail for any other username within a domain.

POP3 Server Implementation

popa3d (http://www.openwall.com/popa3d/) is a simple and secure implementation of the POP3 protocol for Unix systems. Its design sets it apart in that security was the foremost concern, a goal which is not easily realizable with the Unix security model. This implementation uses popa3d, with some additional patches to support virtual users and authentication against an LDAP database.

For this scheme to work, we accept usernames to be of the form user@domain, construct the LDAP bind DN and attempt a bind as that user. On successful authentication, we query the user object that we logged in as, and retrieve the uid and gid fields. We set the unix credentials accordingly and access the mail spool.

Mail spool Implementation

As with most ambitious designs, this one also mandates a single unix user id and group id for mail storage. It does allow for exceptions; All components can query LDAP for that information, so if you need to modify them to accomodate shell users, you can. If you don't have such users, hardcode the user id and group id values in Exim's configuration file and enjoy the security benefits.

LDAP is an elegant solution for authentication, but doesn't provide the flexibility and performance required to store mail. With this reasoning it was decided to use generic file storage to implement a mail spooling system. The solution can still be made to scale without resorting to a relational database: High-end dedicated NFS servers can provide a scalable and highly available solution to storing mail (think Network Appliance).

Naturally, don't even think of deploying an NFS based solution without perimeter security or equivalent measure in place beforehand.

Since the aim is to provide a service which is primarily used for mail routing, aliases and few storage-capable mailboxes per domain, POP3 based access was chosen. The POP3 protocol doesn't provide facilities such as subfolders so simple BSD /var/mail storage was deemed adequate. To support virtual domains, we set a template for a user mailbox to be, for example,

/opt/mailspool/$domain/$user

where $domain is the virtual domain and $user is the username. Exim can be configured so that the location of the mailbox is retrieved from the database itself. However this will allow security breaches since domain administrators can modify all user parameters. For this reason, the path is normally constructed at real time by Exim, based on information in the SMTP envelope.

By taking advantage of the versatility of Exim with regard to mail spool, a system which is capable of fully supporting IMAP4 can be built by delivering to Maildir folders, even following server side rules a la Cyrus. The only IMAP4 server to my knowledge that supports both Maildir and LDAP authentication is Courier-IMAP which is too complex looking for what it does.

Web interface Implementation

Domain administration is done via a PHP4 script called vdom, which supports most of the available functionality in the system.

Web email access is provided by a slightly modified version of nocc to support virtual domains.

Reference

Full Exim configuration file that implements LDAP integration

OpenLDAP configuration file

popa3d 0.4 LDAP authentication patch

Schema mailrouter.schema to implement the custom namespace.

vdom 1.42, a PHP4 virtual domain manager that works with the above schema

NOCC multidomain patch, required to use NOCC with this system.

Simple Qmail aliases and users converters in perl.

Future work

imap4 + maildir.
next step: rdbms + back-rdbms
server-side filtering rules

$Id: index.html,v 1.7 2002/12/01 04:40:03 kos Exp $