Currently our mail servers are magnesium, calcium, and strontium. We use the following software: qmail, Binc IMAP, Dovecot, DSPAM, and procmail.
Contents
1. qmail
We use qmail as our MTA and MDA. It will relay messages from Computer Club IP addresses, and delivers messages to users' mail AFS volume.
1.1. Patches
We are currently using the following patches to qmail:
Additionally, there are some Computer Club specific hacks.
- install binaries, documentation, and man pages to /usr/local rather than /var/qmail, call qmail-local-shim
- use rename() rather than link() for the Maildir delivery protocol (needed for AFS)
- disable CRAM-MD5 authentication (we can't support it, since we don't have the actual text of user passwords lying around)
1.2. Prerequisites
Qmail requires some other software:
DJB's daemontools
DJB's ucspi-tcpi
Bad things will likely happen if the following software hasn't been installed first:
- DSPAM
- ezmlm
- procmail
- zephyr
Qmail also needs the following users in /etc/passwd.system:
alias:x:1000:1002::/var/qmail/alias: qmaild:x:1001:1002::/var/qmail: qmaill:x:1002:1002::/var/qmail: qmailp:x:1003:1002::/var/qmail: qmailq:x:1004:1003::/var/qmail: qmailr:x:1005:1003::/var/qmail: qmails:x:1006:1003::/var/qmail:
And the following groups in /etc/group:
nofiles:x:1002: qmail:x:1003:
1.3. Building
1.3.1. Set Up Procedure Used for the Collection
This is not a description how to build the collection. For that, see below.
This gives an overview of the procedure I used in preparing the qmail-004 collection.
cd /afs/club/system/src/local/qmail/004 tar -zxvf dist/netqmail-1.05.tar.gz cd netqmail-1.05 ./collate.sh cd .. mv netqmail-1.05/netqmail-1.05/* . rm -r netqmail-1.05
Next, I prepared a club directory, containing most of the magic for the collection. This involved merging patches, including the source for qmail-local-shim, writing a Makefile for qmail-local-shim, and a script to apply the patches and change a couple other files. There is also a note about AFS not supporting named pipes, and instructions on how to generate a /var/qmail hierarchy.
club/setup.sh
Lastly, I wrote a SMakefile. It has a fairly complicated INSTCOMMAND, and no MFCOMMAND. The INSTCOMMAND installs qmail-local-shim into the dest directory, and creates a tarball of the /var/qmail hierarchy (it can't be stored in the dest directory, since AFS can't store named pipes).
1.3.2. Building the Collection
Building qmail should be fairly straightforward. It shouldn't require anything more than:
cd /afs/club/system/src/local/qmail/004 smake init smake mk smake install
1.4. Installation
Copy the qmail dest directory to /usr/local/stow. It is important that the permissions, groups, and users of the files are preserved.
# rsync -v -a /afs/club/system/dest/@sys/local/qmail/004 /usr/local/stow/qmail-004
It doesn't appear that AFS will store a SUID bit. So, you'll need to manually fix the permissions on the qmail-queue binary.
# chmod u+s /usr/local/stow/qmail-004/bin/qmail-queue
Create /var/qmail hierarchy. It is in a tarball, since AFS can't store named pipes.
# cd /var # tar -xvf /usr/local/stow/qmail-004/root.var.tar # rm /usr/local/stow/qmail-004/root.var.tar
Stow qmail-004 in /usr/local.
# cd /usr/local/stow # stow qmail-004
Create the log directories for qmail and smtpd.
# mkdir /var/log/qmail # mkdir /var/log/smtpd
Create the supervise service directories for qmail and smtpd.
# cd /var/qmail # mkdir -p qmail/log # cat > qmail/run << "EOF" > #!/bin/sh > exec /usr/local/bin/qmail-start ./Maildir/ > EOF # chmod 755 qmail/run # touch qmail/down # cat > qmail/log/run << "EOF" > #!/bin/sh > exec multilog t n50 /var/log/qmail > EOF # chmod 755 qmail/log/run # ln -s /var/qmail/qmail /var/service # mkdir -p smtpd/log # cat > smtpd/run << "EOF" > #!/bin/sh > exec /usr/local/bin/tcpserver -v -x /var/qmail/tcp.smtp.cdb -u 1001 -g 1002 \ > 0 smtp /usr/local/bin/qmail-smtpd 2>&1 > EOF # chmod 755 smtpd/run # touch smtpd/down # cat > smtpd/log/run << "EOF" > #!/bin/sh > exec multilog t n50 /var/log/smtpd > EOF # chmod 755 smtpd/log/run # ln -s /var/qmail/smtpd /var/service
Create the qmail startup scripts.
# cd /etc/init.d # svinitd-create qmail > qmail # chmod 755 qmail # svinitd-create smtpd > smtpd # chmod 755 smtpd
Create symbolic links for the qmail startup scripts.
# update-rc.d qmail defaults # update-rc.d smtpd start 21 2 3 4 5 . stop 16 0 1 6 .
Setup qmail cron jobs.
# crontab -l > /tmp/qmail-crontab # cat >> /tmp/qmail-crontab << "EOF" > > # qmail > 0 * * * * /afs/club/system/scripts/perl/mailassign.pl "/afs/club/user" > \ > /var/qmail/users/assign && /usr/local/bin/qmail-newu > 2 * * * * /afs/club/system/scripts/sh/update-alias.sh > 30 * * * * /afs/club.cc.cmu.edu/system/scripts/sh/update-mailtabs.sh > 0 0 * * * /usr/local/bin/update_tmprsadh > /dev/null 2>&1 > EOF # crontab /tmp/qmail-crontab # rm /tmp/qmail-crontab
Copy configuration files from an existing mail server.
# rsync -e ssh -a magnesium.club.cc.cmu.edu:/var/qmail/control/ /var/qmail/control # rsync -e ssh -a magnesium.club.cc.cmu.edu:/var/qmail/tcp.smtp /var/qmail
Change /var/qmail/control/me and compile the rules for qmail-smtpd's tcpserver.
# cd /var/qmail # hostname > control/me # tcprules tcp.smtp.cdb tcp.smtp.tmp < tcp.smtp
Modify /var/qmail/control/locals and copy the new version to all mail servers.
# cd /var/qmail/control # hostname >> locals # rsync -e ssh -a locals magnesium.club.cc.cmu.edu:/var/qmail/control/locals # rsync -e ssh -a locals calcium.club.cc.cmu.edu:/var/qmail/control/locals
Copy key files from an existing mail server. (The update-mailtabs script will keep this updated, but needs to be bootstrapped.)
# rsync -e ssh -a magnesium.club.cc.cmu.edu:/var/keys /var
Make sure the cron jobs have run at least once.
Start qmail and smtpd.
# /etc/init.d/qmail start # /etc/init.d/smtpd start
1.5. SMTP Auth
Requires a checkpassword-compatible program. http://checkpasswd-pam.sourceforge.net/ is that for PAM (which can use krb5). Install it, make sure it is setuid.
Will need to modify /var/qmail/smtpd/run: append "/usr/bin/checkpasswd-pam -s smtp /bin/true", right after qmail-smtpd.
2. ezmlm
ezmlm is our mailing list manager and is remarkably understandable considering it's from djb. There are extensive manpages, which should be the first place to look. Some questions are only answered by looking at the source, which isn't that ugly.
Currently, we keep most (all?) of our mailing lists under the "ezmlm" user's Maildir.
2.1. Quick tips
Mailing lists are identified by the directory in which the data is stored. In our case, we use directories under /afs/club.cc.cmu.edu/usr/ezmlm/Maildir/, such as /afs/club.cc.cmu.edu/usr/ezmlm/Maildir/announcef07.
2.1.1. Creating a mailing list
To create an announcement-style mailing list, the following will usually work (swapping the name of the list for announcef07):
# ezmlm-make -5 gripe@club.cc.cmu.edu -m /afs/club.cc.cmu.edu/usr/ezmlm/Maildir/announcef07 /afs/club.cc.cmu.edu/usr/ezmlm/.qmail-announcef07 announcef07 club.cc.cmu.edu
After this, one needs to add this list to /afs/club/service/mail/subusers to add this as subuser of ezmlm. An (currently) hourly cronjob then uses this to update /var/qmail/users/assign and regenerate the corresponding cdb with qmail-newu.
2.1.2. Adding someone to a mailing list
To add someone to a mailing list, run:
# ezmlm-sub <mailing list directory> <email address>
3. Wheezy
This is a work-in-progress.
3.1. Packages
3.1.1. Unmodified debian packages
- daemontools
- svtools
- ucspi-tcp (TBD: may be able to use ucspi-tcp-ipv6)
3.1.2. Packages that required minor modifications
- procmail
- Had to change a few #define's to default to ~/Maildir/ delivery
- Removed setuid bits
The unmodified Debian version can be made work with some coaxing, it just will obnoxiously complain every time it is invoked; sigh...
- dspam; libdspam7-drv-mysql (source: dspam)
- No source modifications were required; just had to rebuild with the "virtual users" feature disabled Toggle line numbers
dch --local +cclub. # Identify as cclub build in package version; adjust club revision appropriately env DEB_BUILD_OPTIONS=disable_virtual_users dpkg-buildpackage -rfakeroot -us -uc -b
- No source modifications were required; just had to rebuild with the "virtual users" feature disabled
- daemontools-run (source: daemontools)
- Had to fix its postinst script:
Put the SV:12345:respawn:/usr/bin/svscanboot line earlier in /etc/inittab
- Without the fix, svscanboot doesn't start until after all the init scripts have run
- Had to fix its postinst script:
3.1.3. Packages that required major modifications or were not in Debian
- qmail; qmail-uids-gids (source: netqmail)
There is a Debian package, but made significant changes
- qmail-mail-transport-agent
- Glue package of our own invention
- Similar in spirit to qmail-run, but it doesn't pull in as much cruft
- Does some simple things so that Debian works with qmail
- Provide the mail-transport-agent virtual package
- Correctly handles system-generated mail (sendmail symlinks)
- ezmlm-idx
- No Debian pacakge existed, so we made one
- Patches were required to handle the Debian qmail package putting binaries in /usr/{s,}bin
- checkpassword-pam
- No Debian package existed, but the upstream source had a debian/ directory
- Made some simple modifications to upstream's packaging attempts
3.2. Custom programs and scripts for /usr/local
- ezmlm-issubn.pl
Ezmlm-idx helpfully changed the way several programs are invoked
program /absolute/path/to/subdir1 /absolute/path/to/subdir2 → program /absolute/path/to subdir1 subdir2
- Generally, this only affects commands that would be invoked manually (e.g, ezmlm-sub, ezmlm-unsub, ezmlm-list)
- However, it can affect a certain invocation of ezmlm-issubn in .qmail files
- Wrapper script detects the old invocation, and rewrites the arguments to work with the new ezmlm-issubn binary
Script is in Subversion: https://svn.club.cc.cmu.edu/cclub/scripts/trunk/perl/ezmlm-issubn.pl
- wrapit
- Since we're switching to debs for wheezy, many programs that were in /usr/local are now in /usr
- Wrapit will log when such programs are invoked with absolute paths into /usr/local, so we can find and fix scripts and .qmail files
Source is in Subversion: https://svn.club.cc.cmu.edu/cclub/wrapit/trunk/
Symlinks for use with Wrapit: wrapper-symlinks.tar.gz
4. Installation
All club users must be present on mail servers. To do this (as root):
touch /etc/passwd.user
/afs/club/system/scripts/sh/passwd-update.sh
Install the Debian packages listed above.
Be sure to save off the users the packages create into /etc/passwd.system!
grep '^\(dspam\|alias\|qmail.\):' /etc/passwd >> /etc/passwd.system
Build and install the wrappers:
cd /tmp
svn co https://svn.club.cc.cmu.edu/cclub/wrapit/trunk ./wrapit
cd wrapit
make
# Remaining steps require root
mkdir -p /usr/local/stow/wrappers-001
make DESTDIR=/usr/local/stow/wrappers-001 install
cd /usr/local/stow/wrappers-001
mv ./usr/local/* .
rmdir ./usr/local
rmdir ./usr
mv ./var/log/wrapit /var/log
rmdir ./var/log
rmdir ./var
ln -s /var/log/wrapit/svc /etc/service/wrapit-logs
cd ./bin
svn export https://svn.club.cc.cmu.edu/cclub/scripts/trunk/perl/ezmlm-issubn.pl ./ezmlm-issubn.pl
# Assumes you've downloaded the wrapper-symlinks.tar.gz, which is attached to this page, somewhere locally
tar -zxvf /path/to/wrapper-symlinks.tar.gz
cd /usr/local/stow
stow wrappers-001
5. Configuration
The easiest way to configure qmail for cclub, is to start with configuration from an existing mail server.
5.1. /var/qmail/control
Most of qmail's configuration exists as several files in the /var/qmail/control directory. For a list of such files, see qmail-control (5), though please note that our qmail supports additional control files (see the actual man page on one of the mailservers for a complete list).
5.1.1. Things that need to be changed if you start with an existing mail server's files
Edit me --> local FQDN.
Edit locals --> add local FQDN to the list. Copy the new version of locals to all the other mail servers.
5.1.2. Descriptions of the control files, some with cclub's values
Here are some important control files, and how they are configured for the cclub environment:
- concurrencylocal
- maximum number of local deliveries to perform simultaneously
5
- concurrencyremote
- maximum number of remote deliveries to perform simultaneously
30
- defaultdomain
- domain to use when a recipient address is given without a domain
club.cc.cmu.edu
- locals
- domains that are handled by local delivery (list below is valid as of 10 Feb 2014):
localhost club.cc.cmu.edu «hostname».club.cc.cmu.edu thorin.dementia.org thorin.club.cc.cmu.edu aberrant.org cmucc.org
- me
- the fully-qualified hostname of the machine qmail is running on
«hostname».club.cc.cmu.edu
- plusdomain
- what should be appended to recipient addresses given with a trailing '+' character
cmu.edu
- queuelifetime
- how long to keep a message in the queue before treating temporary delivery failures as permanent
345600
- rcpthosts
domains for which mail will be accepted, with wildcards indicated by a leading '.' character
This list is quite lengthy (at some point we may want to go through this and identify domains in it that we are no longer hosting mail for, and remove them). I'm not including it here, due to its length. It should be fine to copy the file off of some existing mail server.- servercert.pem
- concatenated, PEM encoded SSL private key, CA key chain, and SSL certificate
- spfbehavior
- controls SPF validation: whether it's done, and how to treat failures
1
- timeoutremote
- how long to wait for responses from remote SMTP servers
300
- virtualdomains
maps "virtual" users and domains to local user accounts
This list is quite lengthy (at some point we may want to go through this and identify domains in it that we are no longer hosting mail for, and remove them). I'm not including it here, due to its length. It should be fine to copy the file off of some existing mail server.
5.2. /var/keys
Create the directory.
Extract the "mailtabs" keytab to /var/keys/mailtabs.
mkdir -p /var/keys
kinit -S kadmin/admin «username»/admin
kadmin ext -k /var/keys/mailtabs mailtabs
kdestroy
The mailtabs will be added to the directory by update-mailtabs.sh.
5.3. Cron jobs
These all run as root. Probably a good idea to run all of them first manually.
1 # Mail stuff
2 00 * * * * /afs/club.cc.cmu.edu/system/scripts/perl/mailassign.pl /afs/club/user > /var/qmail/users/assign && /usr/sbin/qmail-newu
3 02 * * * * /afs/club.cc.cmu.edu/system/scripts/sh/update-alias.sh
4 30 * * * * /afs/club.cc.cmu.edu/system/scripts/sh/update-mailtabs.sh
5 00 0 * * * /usr/sbin/update_tmprsadh > /dev/null 2>&1
5.4. SMTP Access Control
See tcprules(1) for information on the rule files' syntax.
For the standard SMTP service (port 25), allow relaying for club machines. For non-club machines, optionally support authentication and relay for authenticated users. Otherwise, non-club machines are only allowed to send mail addressed to hosts/domains in rcpthosts.
/var/qmail/tcp.smtp:
# B6 machines 128.237.157.:allow,RELAYCLIENT="" # GERMANIUM 128.2.204.94:allow,RELAYCLIENT="" # TECHNETIUM + DomUs 128.2.204.148-153:allow,RELAYCLIENT="" # TRANSIT-1 128.2.207.106:allow,RELAYCLIENT="" # loopback IPs 127.:allow,RELAYCLIENT="" # catch-all :allow,SMTPAUTH=""
For the submission service (port 587), always require authentication.
/var/qmail/tcp.submission:
:allow,SMTPAUTH="!"
To use the rules with tcpserver, they need to be compiled into a .cdb:
tcprules /var/qmail/tcp.smtp.{cdb,tmp} < /var/qmail/tcp.smtp
tcprules /var/qmail/tcp.submission.{cdb,tmp} < /var/qmail/tcp.submission
5.5. SMTP Authentication
We use checkpassword-pam to authenticate club users connecting to SMTP. This requires a modified PAM configuration.
/etc/pam.d/smtpd:
#
# /etc/pam.d/smtpd - PAM settings for SMTP-AUTH
#
# Unfortunately, for this case, we can't use any of the common-* files.
# - pam_krb5.so, by default, wants to create FILE ccaches and chown them to
# the authenticated user; that would fail, since we don't run as root
# - we want to support mail passwords; setting that up requires additional
# arguments for pam_krb5.so
# - there's no need to create a PAG or get AFS tokens
auth required pam_krb5.so minimum_uid=110 alt_auth_map=%s/email \
keytab=/var/qmail/smtp.keytab ccache=MEMORY:
account required pam_krb5.so minimum_uid=110 alt_auth_map=%s/email \
keytab=/var/qmail/smtp.keytab ccache=MEMORY:
password required pam_deny.so
session required pam_permit.so
session optional pam_krb5.so minimum_uid=110 alt_auth_map=%s/email \
keytab=/var/qmail/smtp.keytab ccache=MEMORY:What's this all about?
- alt_auth_map=%s/email
- This allows us to provide alternate passwords that users can use for email. E.g., this would allow users to use a different password than there regular cclub password for email access from mobile devices.
- keytab=/var/qmail/smtp.keytab
- A Kerberos-enabled server uses a shared keytab with the KDC in order to authenticate the KDC (i.e., to prevent somebody to succeed in authenticating by spoofing the KDC). Usually /etc/krb5.keytab is used, but since checkpassword-pam is not run as root, it is not readable. To work around, we create a dedicated keytab for the smtpd to use.
- ccache=MEMORY:
- For a login user's convenience, Kerberos authentication will create a ticket file for the authentication user in /tmp, and then chown it to the authenticated user. This does not work when checkpassword-pam is not run from. For mail, the tickets are needed, so we can solve the problem by using an in-memory only credentials cache.
The next step is actually creating the above-mentioned keytab:
kinit -S kadmin/admin user/admin
kadmin ank -r SMTP/hostname.club.cc.cmu.edu
kadmin ext -k /var/qmail/smtp.keytab SMTP/hostname.club.cc.cmu.edu
chown qmaild:root /var/qmail/smtp.keytab