4.11. What about performance?

There are many aspects to tuning a Mailman system, but they essentially resolve to:

1. Mailman
2. MTA
3. Supporting infrastructure.

Taking them in order:

Tuning Mailman is normally not needed. The only real control which is of significance is SMTP_MAX_RCPTS in ~mailman/Mailman/mm_cfg.py (see ~mailman/Mailman/Defaults.py for details). The optimal value will vary depending on your lists, the distribution of their memberships across domains, and the responsiveness of the various domain's MXes. Typically the sweet spot is in the range of 2 - 5. See Chuq von Rospach's analysis of VERP expense in FAQ 4.12 at 4.12 What about VERP? for further details.

MTA tuning is incredibly subject to local conditions, MX demographics of your lists, list load and traffic patterns etc. For MTA-specific tuning tips see MTA Performance Tuning Tips for EXIM, MTA Performance Tuning Tips for Sendmail, MTA Performance Tuning Tips for Postfix, and MTA Performance Tuning Tips for Qmail.

There are however a some critical common aspects:

• Configure your MTA to not do DNS verifies on receipt from localhost. Most MTAs default to doing verifies by default. Leaving that turned on will slow delivery rates from mailman to your MTA significantly, especially for larger lists.

EXIM

  In Exim, the value to edit is receiver_verify_hosts. See
  README.EXIM in the Mailman distribution, or
  http://www.exim.org/howto/mailman.html for details.

POSTFIX

QMAIL

SENDMAIL

    From: http://mail.python.org/pipermail/mailman-developers/2001-August/009329.html

    You can do this without modifying your sendmail files at all.
    Instead, in your startup script, add:

        /usr/sbin/sendmail -bd -ODeliveryMode=defer \
               -ODaemonPortOptions=Name=MSA,Port=NNNN,M=E,Addr=127.0.0.1

    Where NNNN is some port number not otherwise used (you can test
    if something's in use by doing "telnet localhost NNNN" -- if it's
    refused, there's no daemon listening)

    This sets up a sendmail process listening to the alternate port,
    in DEFER mode, but set to talk only to the localhost interface,
    so it's not accessible by anyoneother than your local machine:
    no open relay problems.

    To make mailman access that port, add this to your mm_cfg.py:

        # define alternate SMTP port
        SMTPPORT = 1313

• Watch your mail spools and observe the percentage and number of slow MXes you have for your lists. Based on that you may want to tune your retry times/rates and total time a message can live in your spool (minimum 4 days). If you have large populations of slow MXes, there can be significant advantage to putting in a domain routing rule in your MTA to send all that traffic to another mail server running as a smarthost which is specifically configured/tuned for slow MXes,

MTAs are inherently IO bound, especially disk IO bound. There are a number of things you can do at the system level to help alleviate this:

• Run a local caching nameserver. MTAs do very large numbers of DNS lookups. Taking network latency out of the picture can speed performance significantly. See Improving performance by local DNS caching.

• Directories that have too many files in them can take too long to search. For most versions of Unix, if you have more than 1000 files in a directory, it can start seriously slowing down the performance of an MTA. Moreover, once a directory has had a large number of files in it, that directory will never get smaller. So, it will always take a long time to scan, even if it is currently empty. Don't let your directories get large, and make sure that your MTA is configured to use a directory hashing scheme. This also helps alleviate the problem of excessive synchronous meta-data operations causing filesystem thrashing, because now you can have multiple MTA processes that are each doing their own synchronous meta-data operations in their own directories - or at least with fewer collisions.
• If you are using syslog(8) logging with your MTA, make sure that your syslog.conf is not configured to sync() after every write to your mail logs. For Linux, you can do this by prefixing the entry in syslog.conf with a '-'. See the man page for syslog.conf(5) for details. If you are doing syslog over the network, make sure that you use a version of syslog that uses TCP and not UDP - I've seen 75% or more packet loss with UDP syslog. Unfortunately, most versions of syslog that are shipped by default are not capable of doing syslog over TCP, so this will most likely require that you consider a replacement such as syslog-ng, ssyslog, or nsyslog.
• Put your syslog logs and your MTA spool on physically different spindles (ie different drives). It makes little sense to have two very IO active processes competing for who gets to read or write from the same device at any given time.
• After carefully checking your MTA documentation, consider mounting your MTA spool device with the "noatime" option. This can significantly reduce IO levels to Inodes, which will help the general IO load of your mail spool device.
• If designing a new system for this purpose, optimise for synchronous disk transactions - ie your disks should have low latency, the ability to handle several queued operations (i.e., SCSI, not IDE), and be arranged in a configuration that reduces latency (ie RAID0 or RAID1+0, not RAID5).
• If you're running a server that is capable of using a journaling filesystem (recent versions of linux with ext3fs, Solaris with VxFS, etc...), try putting the journal on a separate fast device - either a very fast disk or solid state disk (still expensive, but cheaper than everything on solid state). Of course, not all journaling filesystems will allow you to put the journal on a separate device.
• If you're using FreeBSD, NetBSD, OpenBSD, etc... ensure that the filesystem for your MTA mail spool has "softupdates" enabled. More recent versions of these OSes should come with softupdates enabled by default, but you should check. The advantage of softupdates over journaling is that journaling doesn't eliminate synchronous meta-data operations, it merely delays and orders them, while softupdates can completely eliminate many synchronous meta-data operations if the file(s) is/are created, used, and deleted quickly enough (which is typical for an MTA mail spool). The key to softupdates is that it does this in a way that will always leave the filesystem in a consistent state, and should be able to avoid fsck on boot. With softupdates on a suitably configured system, you can get near RAM disk levels of performance, without taking the risk of having your filesystems corrupted or significant data loss in the case of power outage.
• If you are running extremely high mail loads (think tens of millions of deliveries per day) you may want to consider using a solid state disk for your mail spool instead of a normal magnetic drive. Much faster. Much lower latency. Very expensive.
• If you don't care about reliability (e.g., you're sending out daily reports, and if today's report doesn't get there in a certain amount of time then it is useless anyway), you could consider putting your mail spool on a RAM disk. This is similar to the concept of a solid-state disk, except that it is kept in main system RAM, and if the machine should suddenly reboot then everything on the RAM disk will be lost. Contrariwise, with a solid-state disk, everything written to the disk would be safe even if power was lost.

Converted from the Mailman FAQ Wizard

This is one of many Frequently Asked Questions.