| Deletions are marked like this. | Additions are marked like this. |
| Line 52: | Line 52: |
| * qmail-qfilter |
Currently our mail servers are mx1, mx2, and mx3.
We use qmail as our MTA, and have it configured to deliver messages to users' mail volumes in AFS (mounted in $HOME/Maildir).
Contents
1. Mail (qmail)
1.1. Packages
1.1.1. Unmodified debian packages
- daemontools
- daemontools-run
- svtools
- ucspi-tcp-ipv6
1.1.2. Packages that required minor modifications
- procmail
- Had to change a few #define's to default to ~/Maildir/ delivery
- Removed setuid bits
The unmodified Debian version can be made work with some coaxing, it just will obnoxiously complain every time it is invoked; sigh...
1.1.3. Packages that required major modifications or were not in Debian
- qmail; qmail-uids-gids (source: netqmail)
There was a Debian package, but I made significant changes
- See debian/cclub-patches in the source package for the gory details
- qmail-mail-transport-agent
- Glue package of our own invention
- Similar in spirit to qmail-run, but it doesn't pull in as much cruft
- Does some simple things so that Debian works with qmail
- Provide the mail-transport-agent virtual package
- Correctly handles system-generated mail (sendmail symlinks)
- ezmlm-idx
- No Debian package existed, so we made one
- Patches were required to handle our qmail package putting binaries in /usr/{s,}bin
- We're now running code from the git master branch, plus some patches of our own to fix issues with the build and From-header rewriting
- checkpassword-pam
- No Debian package existed, but the upstream source had a debian/ directory
- Made some modifications to upstream's packaging attempts
- Now using a version from git identifying itself as 1.00rc
- dspam; libdspam7-drv-mysql (source: dspam)
- Looks like the project is mostly abandoned now
I made updates to the last Debian package I could find, bringing in the latest upstream version (3.10.1 -> 3.10.2)
- Had some difficulty getting it to build on Debian 12
- Because of the adjustments I had to make, I disabled the "virtual users" feature by default when building
- qmail-qfilter
daemontools-run (source: daemontools)
Had to fix its postinst script:
Put the SV:12345:respawn:/usr/bin/svscanboot line earlier in /etc/inittab
Without the fix, svscanboot doesn't start until after all the init scripts have run
- This is probably one of the only things systemd has ever made better
- The way the systemd runs the scriptlets created by svinitd-create does not have the problem we were seeing with later versions of the sysvinit scripts
- I believe there are no issues with just using the normal Debian package now
1.2. Custom programs and scripts for /usr/local
- ezmlm-issubn.pl
Ezmlm-idx helpfully changed the way several programs are invoked
program /absolute/path/to/subdir1 /absolute/path/to/subdir2 → program /absolute/path/to subdir1 subdir2
- Generally, this only affects commands that would be invoked manually (e.g, ezmlm-sub, ezmlm-unsub, ezmlm-list)
- However, it can affect a certain invocation of ezmlm-issubn in .qmail files
- Wrapper script detects the old invocation, and rewrites the arguments to work with the new ezmlm-issubn binary
Script is in Subversion: https://svn.club.cc.cmu.edu/cclub/scripts/trunk/perl/ezmlm-issubn.pl
- wrapit
- Since we're switching to debs for wheezy, many programs that were in /usr/local are now in /usr
- Wrapit will log when such programs are invoked with absolute paths into /usr/local, so we can find and fix scripts and .qmail files
Source is in Subversion: https://svn.club.cc.cmu.edu/cclub/wrapit/trunk/
Symlinks for use with Wrapit: wrapper-symlinks.tar.gz
1.3. Installation
Note that you need to be a little careful during clubification, where cclub-base-configuration would by default install exim4-- the basic MTA we run on general purpose (i.e., non-mail) servers.
If you clubify normally, you can switch from using exim4 as the mail-transport-agent provider to using qmail, with:
apt-get install -exim4 -exim4-base -exim4-daemon-light +qmail +qmail-mail-transport-agent
Alternatively, you can install qmail and qmail-mail-transport-agent together with cclub-base-configuration. E.g., a full command performing clubification might look like:
apt-get install cclub-base-configuration cclub-afs-client-configuration cclub-passwd-update-configuration cclub-xen-pvh-domu-configuration qmail qmail-mail-transport-agent
When you install qmail, it will pull in the qmail-uids-gids package as a dependency. It creates users with uids that are not in the standard range for system users. This means you will need to manually save off the users to /etc/passwd.system!
grep '^\(alias\|qmail.\):' /etc/passwd /etc/passwd.OLD | sort | uniq >> /etc/passwd.system
All club users must be present on mail servers. To do this (as root):
touch /etc/passwd.user
rm /etc/passwd.core
/usr/share/cclub-scripts/passwd_update_v2.sh
Install the rest of the Debian packages listed above.
Build and install the wrappers:
cd /tmp
svn co https://svn.club.cc.cmu.edu/cclub/wrapit/trunk ./wrapit
cd wrapit
make
# Remaining steps require root
mkdir -p /usr/local/stow/wrappers-001
make DESTDIR=/usr/local/stow/wrappers-001 install
cd /usr/local/stow/wrappers-001
mv ./usr/local/* .
rmdir ./usr/local
rmdir ./usr
mv ./var/log/wrapit /var/log
rmdir ./var/log
rmdir ./var
ln -s /var/log/wrapit/svc /etc/service/wrapit-logs
cd ./bin
svn export https://svn.club.cc.cmu.edu/cclub/scripts/trunk/perl/ezmlm-issubn.pl ./ezmlm-issubn.pl
# Assumes you've downloaded the wrapper-symlinks.tar.gz, which is attached to this page, somewhere locally
tar -zxvf /path/to/wrapper-symlinks.tar.gz
cd /usr/local/stow
stow wrappers-001
1.4. Configuration
The easiest way to configure qmail for cclub, is to start with configuration from an existing mail server.
1.4.1. /var/qmail/control
Most of qmail's configuration exists as several files in the /var/qmail/control directory. For a list of such files, see qmail-control (5), though please note that our qmail supports additional control files (see the actual man page on one of the mailservers for a complete list).
1.4.1.1. Things that need to be changed if you start with an existing mail server's files
Edit me --> local FQDN.
Edit locals --> add local FQDN to the list. Copy the new version of locals to all the other mail servers.
1.4.1.2. Descriptions of the control files, some with cclub's values
Here are some important control files, and how they are configured for the cclub environment:
- concurrencylocal
- maximum number of local deliveries to perform simultaneously
5
- concurrencyremote
- maximum number of remote deliveries to perform simultaneously
30
- defaultdomain
- domain to use when a recipient address is given with an unqualified hostname
club.cc.cmu.edu
- defaulthost
- domain to use when a recipient address is given without any hostname
club.cc.cmu.edu
- locals
- domains that are handled by local delivery (list below is valid as of 10 Feb 2014):
localhost club.cc.cmu.edu «hostname».club.cc.cmu.edu thorin.dementia.org thorin.club.cc.cmu.edu aberrant.org cmucc.org
- me
- the fully-qualified hostname of the machine qmail is running on
«hostname».club.cc.cmu.edu
- plusdomain
- what should be appended to recipient addresses given with a trailing '+' character
cmu.edu
- queuelifetime
- how long to keep a message in the queue before treating temporary delivery failures as permanent
345600
- rcpthosts
domains for which mail will be accepted, with wildcards indicated by a leading '.' character
This list is quite lengthy (at some point we may want to go through this and identify domains in it that we are no longer hosting mail for, and remove them). I'm not including it here, due to its length. It should be fine to copy the file off of some existing mail server.- servercert.pem
- concatenated, PEM encoded SSL private key, CA key chain, and SSL certificate
- spfbehavior
- controls SPF validation: whether it's done, and how to treat failures
1
- timeoutremote
- how long to wait for responses from remote SMTP servers
300
- virtualdomains
maps "virtual" users and domains to local user accounts
This list is quite lengthy (at some point we may want to go through this and identify domains in it that we are no longer hosting mail for, and remove them). I'm not including it here, due to its length. It should be fine to copy the file off of some existing mail server.
1.4.2. /var/keys
Create the directory.
Extract the "mailtabs" keytab to /var/keys/mailtabs.
mkdir -p /var/keys
kinit -S kadmin/admin «username»/admin
kadmin ext -k /var/keys/mailtabs mailtabs
kdestroy
The mailtabs will be added to the directory by update-mailtabs.sh.
1.4.3. Cron jobs
These all run as root. Probably a good idea to run all of them first manually.
# Mail stuff 00 * * * * /afs/club.cc.cmu.edu/system/scripts/perl/mailassign.pl /afs/club/user > /var/qmail/users/assign && /usr/sbin/qmail-newu 02 * * * * /afs/club.cc.cmu.edu/system/scripts/sh/update-alias.sh 30 * * * * /afs/club.cc.cmu.edu/system/scripts/sh/update-mailtabs.sh 00 0 * * * /usr/sbin/update_tmprsadh > /dev/null 2>&1
1.4.4. SMTP Access Control
See tcprules(1) for information on the rule files' syntax. See qmail-smtpd(8) for the behaviors the environment variables control.
For the standard SMTP service (port 25), allow relaying for club machines. For non-club machines, optionally support authentication and relay for authenticated users. Otherwise, non-club machines are only allowed to send mail addressed to hosts/domains in rcpthosts.
/var/qmail/tcp.smtp:
# B6 machines 128.237.157.:allow,RELAYCLIENT="" 172.29.24.:allow,RELAYCLIENT="" # Gates machines 128.2.220.21:allow,RELAYCLIENT="" 128.2.220.33:allow,RELAYCLIENT="" 128.2.220.55:allow,RELAYCLIENT="" 128.2.220.97:allow,RELAYCLIENT="" 128.2.220.100:allow,RELAYCLIENT="" 128.2.220.175:allow,RELAYCLIENT="" # loopback IPs 127.:allow,RELAYCLIENT="" # catch-all :allow,SMTPAUTH=""
For the submission service (port 587), always require authentication.
/var/qmail/tcp.submission:
:allow,SMTPAUTH="!"
To use the rules with tcpserver, they need to be compiled into a .cdb:
tcprules /var/qmail/tcp.smtp.{cdb,tmp} < /var/qmail/tcp.smtp
tcprules /var/qmail/tcp.submission.{cdb,tmp} < /var/qmail/tcp.submission
1.4.5. SMTP Authentication
We use checkpassword-pam to authenticate club users connecting to SMTP. This requires a modified PAM configuration.
/etc/pam.d/smtpd:
#
# /etc/pam.d/smtpd - PAM settings for SMTP-AUTH
#
# Unfortunately, for this case, we can't use any of the common-* files.
# - pam_krb5.so, by default, wants to create FILE ccaches and chown them to
# the authenticated user; that would fail, since we don't run as root
# - we want to support mail passwords; setting that up requires additional
# arguments for pam_krb5.so
# - there's no need to create a PAG or get AFS tokens
auth required pam_krb5.so minimum_uid=110 alt_auth_map=%s/email \
keytab=/var/qmail/smtp.keytab ccache=MEMORY:
account required pam_krb5.so minimum_uid=110 alt_auth_map=%s/email \
keytab=/var/qmail/smtp.keytab ccache=MEMORY:
password required pam_deny.so
session required pam_permit.so
session optional pam_krb5.so minimum_uid=110 alt_auth_map=%s/email \
keytab=/var/qmail/smtp.keytab ccache=MEMORY:What's this all about?
- alt_auth_map=%s/email
- This allows us to provide alternate passwords that users can use for email. E.g., this would allow users to use a different password than there regular cclub password for email access from mobile devices.
- keytab=/var/qmail/smtp.keytab
- A Kerberos-enabled server uses a shared keytab with the KDC in order to authenticate the KDC (i.e., to prevent somebody to succeed in authenticating by spoofing the KDC). Usually /etc/krb5.keytab is used, but since checkpassword-pam is not run as root, it is not readable. To work around, we create a dedicated keytab for the smtpd to use.
- ccache=MEMORY:
- For a login user's convenience, Kerberos authentication will create a ticket file for the authentication user in /tmp, and then chown it to the authenticated user. This does not work when checkpassword-pam is not run from. For mail, the tickets are needed, so we can solve the problem by using an in-memory only credentials cache.
The next step is actually creating the above-mentioned keytab:
kinit -S kadmin/admin user/admin
ktutil -k /var/qmail/smtp.keytab get SMTP/freedom.club.cc.cmu.edu
chmod 400 /var/qmail/smtp.keytab
chown qmaild:root /var/qmail/smtp.keytab
1.4.6. DSPAM Configuration
A few things need to change for the Cclub environment:
- /etc/dspam/dspam.conf
StorageDriver ---> /usr/lib/x86_64-linux-gnu/dspam/libmysql_drv.so
TrustedDeliveryAgent ---> "/bin/cat"
Add: UntrustedDeliveryAgent "/bin/cat"
Remove: all Trust lines except for Trust root- /etc/dspam/dspam.d/mysql.conf
- Fill in with the details for the club MySQL server.
1.5. Startup
To have qmail start on boot, create supervise service directories, link to them from /etc/services, and then create wrappers in /etc/init.d.
1.5.1. Create supervise service directories
cd /var/qmail
########################################################################
# qmail system
########################################################################
mkdir -p svc/qmail/log/main
cat > svc/qmail/run << EOF
#!/bin/sh
exec /usr/sbin/qmail-start ./Maildir/
EOF
chmod 0755 svc/qmail/run
touch svc/qmail/down
cat > svc/qmail/log/run << EOF
#!/bin/sh
umask 0077
exec multilog t s10485760 n15 ./main
EOF
chmod 0755 svc/qmail/log/run
ln -s /var/qmail/svc/qmail/log/main /var/log/qmail
########################################################################
# normal smtpd
########################################################################
mkdir -p svc/smtp/log/main
cat > svc/smtp/run << EOF
#!/bin/sh
exec /usr/bin/tcpserver -v -R -x /var/qmail/tcp.smtp.cdb \\
-u 1001 -g 1002 0 smtp \\
/usr/bin/rblsmtpd -b -r "zen.spamhaus.org" \\
/usr/sbin/qmail-smtpd \\
/usr/bin/checkpassword-pam -e -s smtpd /bin/true 2>&1
EOF
chmod 0755 svc/smtp/run
touch svc/smtp/down
cat > svc/smtp/log/run << EOF
#!/bin/sh
umask 0077
exec multilog t n50 ./main
EOF
chmod 0755 svc/smtp/log/run
########################################################################
# submission smtpd
########################################################################
mkdir -p svc/submission/log/main
cat > svc/submission/run << EOF
#!/bin/sh
exec /usr/bin/tcpserver -v -R -x /var/qmail/tcp.submission.cdb \\
-u 1001 -g 1002 0 submission \\
/usr/sbin/qmail-smtpd \\
/usr/bin/checkpassword-pam -e -s smtpd /bin/true 2>&1
EOF
chmod 0755 svc/submission/run
touch svc/submission/down
cat > svc/submission/log/run << EOF
#!/bin/sh
umask 0077
exec multilog t n50 ./main
EOF
chmod 0755 svc/submission/log/run
1.5.2. Link to them from /etc/services
ln -s /var/qmail/svc/qmail /etc/service/qmail
ln -s /var/qmail/svc/smtp /etc/service/smtp
ln -s /var/qmail/svc/submission /etc/service/submission
1.5.3. Create wrappers in /etc/init.d
svinitd-create qmail | sed -e '/^#!/a\' \
-e '\' \
-e '### BEGIN INIT INFO\' \
-e '# Provides: qmail\' \
-e '# Required-Start: $local_fs $network daemontools-run\' \
-e '# Required-Stop: $network\' \
-e '# Short-Description: qmail\' \
-e '# Description: The qmail mail transport agent queue management and\' \
-e '# delivery processes.\' \
-e '### END INIT INFO' \
> /etc/init.d/qmail
chmod 0755 /etc/init.d/qmail
svinitd-create smtp submission | sed -e '/^#!/a\' \
-e '\' \
-e '### BEGIN INIT INFO\' \
-e '# Provides: smtpd\' \
-e '# Required-Start: $local_fs $network daemontools-run\' \
-e '# Required-Stop: $network\' \
-e '# Short-Description: smtpd\' \
-e '# Description: The qmail mail transport agent smtp daemon, receiving\' \
-e '# mail via (E)SMTP(S)(A) on ports 25 and 587.\' \
-e '### END INIT INFO' \
> /etc/init.d/smtpd
chmod 0755 /etc/init.d/smtpd
1.5.4. Enable the services at boot
update-rc.d qmail defaults
update-rc.d smtpd defaults
2. Mail for Orgs
I set up mail this way for the KGB.
Unless otherwise noted, each step requires admin tokens.
- Create PTS users for the organization's mail service
pts createuser ${org}mail # Note the pts ID assigned to the user. E.g., # ---- # kbare@conch:~$ pts createuser ${org}mail # User ${org}mail has id 1891 # ^^^^ pts createuser ${org}mail.mail - Add the organization's user to passwd.user
echo "${org}mail:x:${uid}:20:${long_org_name} Mail Forwarding:${org_subdir_in_andrew_afs}:/bin/false" \ >> /afs/club.cc.cmu.edu/service/etc/passwd.user - Create a symlink in /afs/club.cc.cmu.edu/usr pointing to the org's directory
ln -s ${org_subdir_in_andrew_afs} /afs/.club.cc.cmu.edu/usr/${org}mail vos release club.usr - Create a Kerberos mail principal, extract it to the mailtabs directory
kadmin ank -r --use-defaults ${org}mail/mail kadmin ext -k /afs/club.cc.cmu.edu/service/mail/mailtabs/${org}mail ${org}mail/mail chown ${uid}:dialout /afs/club.cc.cmu.edu/service/mail/mailtabs/${org}mail - Add the hosted domain to the mailserver configurations
# XXX: if we end up doing a lot of org mail hosting, consider some automated # way to handle this. # Do the following as root on *EACH* of the mail servers. # Currently the mail servers are: MX1, MX2, MX3, and WHEEZY-TEST-MX echo -e "${orgdomain}\n.${orgdomain}" >> /var/qmail/control/rcpthosts echo "${orgdomain}:${org}mail" >> /var/qmail/control/virtualdomains - Things won't work until various things sync (via cron jobs)
- passwd-update.sh on all the mail servers (happens on the hour every hour)
- update-mailtabs.sh on all the mail servers (happens on the half hour every hour)
3. Mailing Lists (ezmlm)
ezmlm is our mailing list manager and is remarkably understandable considering it's from djb. There are extensive manpages, which should be the first place to look. Some questions are only answered by looking at the source, which isn't that ugly.
Currently, we keep most (all?) of our mailing lists under the "ezmlm" user's Maildir.
3.1. Quick tips
Mailing lists are identified by the directory in which the data is stored. In our case, we use directories under /afs/club.cc.cmu.edu/usr/ezmlm/Maildir/, such as /afs/club.cc.cmu.edu/usr/ezmlm/Maildir/announcef07.
3.1.1. Creating a mailing list
To create an announcement-style mailing list, the following will usually work (swapping the name of the list for announcef07):
# ezmlm-make -5 gripe@club.cc.cmu.edu -m /afs/club.cc.cmu.edu/usr/ezmlm/Maildir/announcef07 /afs/club.cc.cmu.edu/usr/ezmlm/.qmail-announcef07 announcef07 club.cc.cmu.edu
After this, one needs to add this list to /afs/club/service/mail/subusers to add this as subuser of ezmlm. An (currently) hourly cronjob then uses this to update /var/qmail/users/assign and regenerate the corresponding cdb with qmail-newu.
3.1.2. Adding someone to a mailing list
To add someone to a mailing list, run:
# ezmlm-sub <mailing list directory> <email address>
4. TODOS (when upgrade to wheezy is complete)
Email Aaron before and after putting the last wheezy MX into place.
Update the DSPAM database to the new schema (changes some strings into integers).
Ensure the DSPAM-purge job is running from cron somewhere (once MAGNESIUM is retired).
Start updating old absolute paths into /usr/local.
5. TODOs
5.1. for qmail package
Update the SMTP TLS patch -> 20231230
- Remove /var/qmail/bin
- Incorporate qmail-mail-transport-agent into the same notqmail source package
- Remove wrappers that do maildir2box and exec an MUA
- Separate package for man pages and utilities that could be useful on the shells
5.2. for wrapper-symlinks
- Remove ezmlm-to40x-mysql
5.3. for wrapit
- Consider spawning a background process from /var/log/wrapit/svc/run that will keep /var/log/wrapit/fifo open for writing, which will prevent the multilog process from restarting after each log entry is written by wrapit.
5.4. for dspam
- It installs a script in /etc/cron.daily running /usr/bin/dspam_maintenance
It probably doesn't make sense to run it from all the mail servers at the same time
- And IIRC, at least in the distant past, the normal binary for database management didn't work well for the MySQL driver, and it was better to run several SQL statements.
- For now, I'm removing the execute bit from dspam_maintenance to prevent the cron job from running on the Debian 12 test host
- Apparently dspam_clean is also a thing (what I remembered the SQL script superseding...) and it's a different thing than dspam_maintenance