Currently our mail servers are magnesium, calcium, and strontium. We use the following software: qmail, Binc IMAP, Dovecot, DSPAM, and procmail.

1. qmail

We use qmail as our MTA and MDA. It will relay messages from Computer Club IP addresses, and delivers messages to users' mail AFS volume.

1.1. Patches

We are currently using the following patches to qmail:

• netqmail

• tls-smtpauth

• qmail-spf

• qmail-bigdns

Additionally, there are some Computer Club specific hacks.

• install binaries, documentation, and man pages to /usr/local rather than /var/qmail, call qmail-local-shim
• use rename() rather than link() for the Maildir delivery protocol (needed for AFS)
• disable CRAM-MD5 authentication (we can't support it, since we don't have the actual text of user passwords lying around)

1.2. Prerequisites

Qmail requires some other software:

• DJB's daemontools

• DJB's ucspi-tcpi

Bad things will likely happen if the following software hasn't been installed first:

• DSPAM
• ezmlm
• procmail
• zephyr

Qmail also needs the following users in /etc/passwd.system:

alias:x:1000:1002::/var/qmail/alias:
qmaild:x:1001:1002::/var/qmail:
qmaill:x:1002:1002::/var/qmail:
qmailp:x:1003:1002::/var/qmail:
qmailq:x:1004:1003::/var/qmail:
qmailr:x:1005:1003::/var/qmail:
qmails:x:1006:1003::/var/qmail:

And the following groups in /etc/group:

nofiles:x:1002:
qmail:x:1003:

1.3. Building

1.3.1. Set Up Procedure Used for the Collection

This is not a description how to build the collection. For that, see below.

This gives an overview of the procedure I used in preparing the qmail-004 collection.

cd /afs/club/system/src/local/qmail/004
tar -zxvf dist/netqmail-1.05.tar.gz
cd netqmail-1.05
./collate.sh
cd ..
mv netqmail-1.05/netqmail-1.05/* .
rm -r netqmail-1.05

Next, I prepared a club directory, containing most of the magic for the collection. This involved merging patches, including the source for qmail-local-shim, writing a Makefile for qmail-local-shim, and a script to apply the patches and change a couple other files. There is also a note about AFS not supporting named pipes, and instructions on how to generate a /var/qmail hierarchy.

club/setup.sh

Lastly, I wrote a SMakefile. It has a fairly complicated INSTCOMMAND, and no MFCOMMAND. The INSTCOMMAND installs qmail-local-shim into the dest directory, and creates a tarball of the /var/qmail hierarchy (it can't be stored in the dest directory, since AFS can't store named pipes).

1.3.2. Building the Collection

Building qmail should be fairly straightforward. It shouldn't require anything more than:

cd /afs/club/system/src/local/qmail/004
smake init
smake mk
smake install

1.4. Installation

Copy the qmail dest directory to /usr/local/stow. It is important that the permissions, groups, and users of the files are preserved.

# rsync -v -a /afs/club/system/dest/@sys/local/qmail/004 /usr/local/stow/qmail-004

It doesn't appear that AFS will store a SUID bit. So, you'll need to manually fix the permissions on the qmail-queue binary.

# chmod u+s /usr/local/stow/qmail-004/bin/qmail-queue

Create /var/qmail hierarchy. It is in a tarball, since AFS can't store named pipes.

# cd /var
# tar -xvf /usr/local/stow/qmail-004/root.var.tar
# rm /usr/local/stow/qmail-004/root.var.tar

Stow qmail-004 in /usr/local.

# cd /usr/local/stow
# stow qmail-004

Create the log directories for qmail and smtpd.

# mkdir /var/log/qmail
# mkdir /var/log/smtpd

Create the supervise service directories for qmail and smtpd.

# cd /var/qmail

# mkdir -p qmail/log

# cat > qmail/run << "EOF"
> #!/bin/sh
> exec /usr/local/bin/qmail-start ./Maildir/
> EOF

# chmod 755 qmail/run
# touch qmail/down

# cat > qmail/log/run << "EOF"
> #!/bin/sh
> exec multilog t n50 /var/log/qmail
> EOF

# chmod 755 qmail/log/run
# ln -s /var/qmail/qmail /var/service

# mkdir -p smtpd/log

# cat > smtpd/run << "EOF"
> #!/bin/sh
> exec /usr/local/bin/tcpserver -v -x /var/qmail/tcp.smtp.cdb -u 1001 -g 1002 \
>                               0 smtp /usr/local/bin/qmail-smtpd 2>&1
> EOF

# chmod 755 smtpd/run
# touch smtpd/down

# cat > smtpd/log/run << "EOF"
> #!/bin/sh
> exec multilog t n50 /var/log/smtpd
> EOF

# chmod 755 smtpd/log/run
# ln -s /var/qmail/smtpd /var/service

Create the qmail startup scripts.

# cd /etc/init.d
# svinitd-create qmail > qmail
# chmod 755 qmail
# svinitd-create smtpd > smtpd
# chmod 755 smtpd

Create symbolic links for the qmail startup scripts.

# update-rc.d qmail defaults
# update-rc.d smtpd start 21 2 3 4 5 . stop 16 0 1 6 .

Setup qmail cron jobs.

# crontab -l > /tmp/qmail-crontab

# cat >> /tmp/qmail-crontab << "EOF"
>
> # qmail
> 0 * * * *        /afs/club/system/scripts/perl/mailassign.pl "/afs/club/user" > \
>                          /var/qmail/users/assign && /usr/local/bin/qmail-newu
> 2 * * * *        /afs/club/system/scripts/sh/update-alias.sh
> 30 * * * *      /afs/club.cc.cmu.edu/system/scripts/sh/update-mailtabs.sh
> 0 0 * * *        /usr/local/bin/update_tmprsadh > /dev/null 2>&1
> EOF

# crontab /tmp/qmail-crontab
# rm /tmp/qmail-crontab

Copy configuration files from an existing mail server.

# rsync -e ssh -a magnesium.club.cc.cmu.edu:/var/qmail/control/ /var/qmail/control
# rsync -e ssh -a magnesium.club.cc.cmu.edu:/var/qmail/tcp.smtp /var/qmail

Change /var/qmail/control/me and compile the rules for qmail-smtpd's tcpserver.

# cd /var/qmail
# hostname > control/me
# tcprules tcp.smtp.cdb tcp.smtp.tmp < tcp.smtp

Modify /var/qmail/control/locals and copy the new version to all mail servers.

# cd /var/qmail/control
# hostname >> locals
# rsync -e ssh -a locals magnesium.club.cc.cmu.edu:/var/qmail/control/locals
# rsync -e ssh -a locals calcium.club.cc.cmu.edu:/var/qmail/control/locals

Copy key files from an existing mail server. (The update-mailtabs script will keep this updated, but needs to be bootstrapped.)

# rsync -e ssh -a magnesium.club.cc.cmu.edu:/var/keys /var

Make sure the cron jobs have run at least once.

Start qmail and smtpd.

# /etc/init.d/qmail start
# /etc/init.d/smtpd start

1.5. SMTP Auth

Requires a checkpassword-compatible program. http://checkpasswd-pam.sourceforge.net/ is that for PAM (which can use krb5). Install it, make sure it is setuid.

Will need to modify /var/qmail/smtpd/run: append "/usr/bin/checkpasswd-pam -s smtp /bin/true", right after qmail-smtpd.

2. ezmlm

ezmlm is our mailing list manager and is remarkably understandable considering it's from djb. There are extensive manpages, which should be the first place to look. Some questions are only answered by looking at the source, which isn't that ugly.

Currently, we keep most (all?) of our mailing lists under the "ezmlm" user's Maildir.

2.1. Quick tips

Mailing lists are identified by the directory in which the data is stored. In our case, we use directories under /afs/club.cc.cmu.edu/usr/ezmlm/Maildir/, such as /afs/club.cc.cmu.edu/usr/ezmlm/Maildir/announcef07.

2.1.1. Creating a mailing list

To create an announcement-style mailing list, the following will usually work (swapping the name of the list for announcef07):

# ezmlm-make -5 gripe@club.cc.cmu.edu -m /afs/club.cc.cmu.edu/usr/ezmlm/Maildir/announcef07 /afs/club.cc.cmu.edu/usr/ezmlm/.qmail-announcef07 announcef07 club.cc.cmu.edu

After this, one needs to add this list to /afs/club/service/mail/subusers to add this as subuser of ezmlm. An (currently) hourly cronjob then uses this to update /var/qmail/users/assign and regenerate the corresponding cdb with qmail-newu.

2.1.2. Adding someone to a mailing list

To add someone to a mailing list, run:

# ezmlm-sub <mailing list directory> <email address>

3. Wheezy

This is a work-in-progress.

3.1. Packages

3.1.1. Unmodified debian packages

• daemontools
• svtools
• ucspi-tcp (TBD: may be able to use ucspi-tcp-ipv6)

3.1.2. Packages that required minor modifications

• procmail
  • Had to change a few #define's to default to ~/Maildir/ delivery
  • Removed setuid bits

  • The unmodified Debian version can be made work with some coaxing, it just will obnoxiously complain every time it is invoked; sigh...

• dspam; libdspam7-drv-mysql (source: dspam)
  • No source modifications were required; just had to rebuild with the "virtual users" feature disabled
    Toggle line numbers

    dch --local +cclub. # Identify as cclub build in package version; adjust club revision appropriately
    env DEB_BUILD_OPTIONS=disable_virtual_users dpkg-buildpackage -rfakeroot -us -uc -b
    
    

• daemontools-run (source: daemontools)
  • Had to fix its postinst script:

    • Put the SV:12345:respawn:/usr/bin/svscanboot line earlier in /etc/inittab

    • Without the fix, svscanboot doesn't start until after all the init scripts have run

3.1.3. Packages that required major modifications or were not in Debian

• qmail; qmail-uids-gids (source: netqmail)

  • There is a Debian package, but made significant changes

• qmail-mail-transport-agent
  • Glue package of our own invention
  • Similar in spirit to qmail-run, but it doesn't pull in as much cruft
  • Does some simple things so that Debian works with qmail
    • Provide the mail-transport-agent virtual package
    • Correctly handles system-generated mail (sendmail symlinks)
• ezmlm-idx
  • No Debian pacakge existed, so we made one
  • Patches were required to handle the Debian qmail package putting binaries in /usr/{s,}bin
• checkpassword-pam
  • No Debian package existed, but the upstream source had a debian/ directory
  • Made some simple modifications to upstream's packaging attempts

3.2. Custom programs and scripts for /usr/local

• ezmlm-issubn.pl

  • Ezmlm-idx helpfully changed the way several programs are invoked

    • program /absolute/path/to/subdir1 /absolute/path/to/subdir2 → program /absolute/path/to subdir1 subdir2

    • Generally, this only affects commands that would be invoked manually (e.g, ezmlm-sub, ezmlm-unsub, ezmlm-list)
    • However, it can affect a certain invocation of ezmlm-issubn in .qmail files
  • Wrapper script detects the old invocation, and rewrites the arguments to work with the new ezmlm-issubn binary

  • Script is in Subversion: https://svn.club.cc.cmu.edu/cclub/scripts/trunk/perl/ezmlm-issubn.pl

• wrapit
  • Since we're switching to debs for wheezy, many programs that were in /usr/local are now in /usr
  • Wrapit will log when such programs are invoked with absolute paths into /usr/local, so we can find and fix scripts and .qmail files

  • Source is in Subversion: https://svn.club.cc.cmu.edu/cclub/wrapit/trunk/

  • Symlinks for use with Wrapit: wrapper-symlinks.tar.gz

4. Installation

Install the Debian packages listed above.

Be sure to save off the users the packages create into /etc/passwd.system!

grep '^\(dspam\|alias\|qmail.\):' /etc/passwd >> /etc/passwd.system

Build and install the wrappers:

cd /tmp
svn co https://svn.club.cc.cmu.edu/cclub/wrapit/trunk ./wrapit
cd wrapit
make

# Remaining steps require root

mkdir -p /usr/local/stow/wrappers-001
make DESTDIR=/usr/local/stow/wrappers-001 install

cd /usr/local/stow/wrappers-001
mv ./usr/local/* .
rmdir ./usr/local
rmdir ./usr
mv ./var/log/wrapit /var/log
rmdir ./var/log
rmdir ./var

ln -s /var/log/wrapit/svc /etc/service/wrapit-logs

cd ./bin
svn export https://svn.club.cc.cmu.edu/cclub/scripts/trunk/perl/ezmlm-issubn.pl ./ezmlm-issubn.pl
# Assumes you've downloaded the wrapper-symlinks.tar.gz, which is attached to this page, somewhere locally
tar -zxvf /path/to/wrapper-symlinks.tar.gz

cd /usr/local
stow wrappers-001

5. Configuration

The easiest way to configure qmail for cclub, is to start with configuration from an existing mail server.

5.1. /var/qmail/control

Most of qmail's configuration exists as several files in the /var/qmail/control directory. For a list of such files, see qmail-control (5), though please note that our qmail supports additional control files (see the actual man page on one of the mailservers for a complete list).

5.1.1. Things that need to be changed if you start with an existing mail server's files

Edit me --> local FQDN.

Edit locals --> add local FQDN to the list. Copy the new version of locals to all the other mail servers.

5.1.2. Descriptions of the control files, some with cclub's values

Here are some important control files, and how they are configured for the cclub environment:

concurrencylocal

maximum number of local deliveries to perform simultaneously

5

concurrencyremote

maximum number of remote deliveries to perform simultaneously

30

defaultdomain

domain to use when a recipient address is given without a domain

club.cc.cmu.edu

locals

domains that are handled by local delivery (list below is valid as of 10 Feb 2014):

localhost
club.cc.cmu.edu
«hostname».club.cc.cmu.edu
thorin.dementia.org
thorin.club.cc.cmu.edu
aberrant.org
cmucc.org

me

the fully-qualified hostname of the machine qmail is running on

«hostname».club.cc.cmu.edu

plusdomain

what should be appended to recipient addresses given with a trailing '+' character

cmu.edu

queuelifetime

how long to keep a message in the queue before treating temporary delivery failures as permanent

345600

rcpthosts

domains for which mail will be accepted, with wildcards indicated by a leading '.' character

This list is quite lengthy (at some point we may want to go through this and identify domains in it that we are no longer hosting mail for, and remove them). I'm not including it here, due to its length. It should be fine to copy the file off of some existing mail server.

servercert.pem

concatenated, PEM encoded SSL private key, CA key chain, and SSL certificate

spfbehavior

controls SPF validation: whether it's done, and how to treat failures

1

timeoutremote

how long to wait for responses from remote SMTP servers

300

virtualdomains

maps "virtual" users and domains to local user accounts

This list is quite lengthy (at some point we may want to go through this and identify domains in it that we are no longer hosting mail for, and remove them). I'm not including it here, due to its length. It should be fine to copy the file off of some existing mail server.

5.2. Setup

Needs to be updated!

5.2.1. /var/qmail/control

Copy from one of the existing mail servers.

Edit me --> local FQDN.

Edit locals --> add local FQDN to the list.

5.2.2. /var/keys

Create the directory.

Extract the "mailtabs" keytab to /var/keys/mailtabs. Something was kind of strange when I did those though. It appeared there were duplicated keys for some of the keys types. I was able to remove them with ktutil, and things worked swimmingly after that.

5.2.3. Cron jobs

These all run as root. Probably a good idea to run all of them first manually.

# Mail stuff
00 *            * * *   /afs/club.cc.cmu.edu/system/scripts/perl/mailassign.pl /afs/club/user > /var/qmail/users/assign && /usr/sbin/qmail-newu
02 *            * * *   /afs/club.cc.cmu.edu/system/scripts/sh/update-alias.sh
30 *            * * *   /afs/club.cc.cmu.edu/system/scripts/sh/update-mailtabs.sh
00 0            * * *   /usr/sbin/update_tmprsadh > /dev/null 2>&1

CategoryServices CategoryMemberServices