6.8. Improving performance by local DNS caching

The MTA used by Mailman will in general require to do a whole pile of DNS lookups to route the addresses handed to it by Mailman (there are exceptions, like if you just push all the addresses to an upstream smart host or over UUCP - in which case you have moved the problem to the next hop).

To improve the performance of the MTA - especially the latency on message sending - it is worth ensuring that there is a local fast DNS cache. By local we mean on the same machine where the MTA is running. Another alternative, usually not as good, would be on a local 100Mbit network (or faster), so within a millisecond of ping time. Since the MTA for any resonably loaded lists will tend to do a whole bunch of very similar lookups in any period, adding a local cache for the DNS lookups can be a huge performance boost - to the point where its worth putting the cache on the local machine even though it is stealing some cycles from the MTA itself. You could even do a two-level system of caching, where the DNS server on the local machine forwards unknown queries to a central caching DNS server on the network.

The selection and configuration of such a DNS cache is outside what can be explained here. There are two primary options that you can use:

1. Dan Bernstein's DNS cache - part of his djbdns tools. This is a small single function program which performs as a DNS cache for this purpose and is written to be secure. The downside is that some people have problems with the licensing of DJB's code, and it works in its own little universe of programs with strange methodology for those coming at it from outside. See <http://cr.yp.to/djbdns.html> for details.

2. The standard BIND DNS server, configured as a DNS cache. This is typically not configured to be secure "out-of-the-box", has been in existence much longer and therefore has historically had more security issues, and may use larger amounts of RAM on your system. Your Unix distribution should provide appropriate packages for this – but make sure any configuration is for caching only restricted to local hosts (see the Team Cymru "Secure BIND Template" for one example of how to configure your BIND server securely, at <http://www.cymru.com/Documents/secure-bind-template.html>.

3. Also note others, http://en.wikipedia.org/wiki/Category:DNS_software

Note that BIND is much better documented than djbdns, given that there are many books written on this subject (especially "DNS and BIND" by Albitz and Liu from O'Reilly & Assoc. at <http://www.oreilly.com/catalog/dns4/index.html>, and "DNS and BIND Cookbook", by Cricket Liu from O'Reilly & Assoc. at <http://www.oreilly.com/catalog/dnsbindckbk/index.html>, as well as the main BIND page from the Internet Systems Corporation at <http://www.isc.org/sw/bind/>. There is also commercial support available for BIND, both from the ISC (see <http://www.isc.org/ops/support/> and other vendors. Finally, there are many commercial packages from other vendors which are based on BIND, see <http://www.isc.org/sw/bind/vendorware.php>.

Moreover, real-world experience with regards to performance of these two options can differ. Read the slides for the talk by Brad Knowles entitled "Domain Name Server Comparison: BIND 8 vs. BIND 9 vs. djbdns vs. ???" (see http://www.shub-internet.org/brad/papers/dnscomparison/) and note that, on the test systems in question, dnscache never came close to the level of performance provided by BIND 8 and BIND 9.

For example, for the root zone, dnscache did 98 queries per second vs. almost 500 queries per second for BIND 8 and over 300 queries per second for BIND 9, and for the .tv zone dnscache could only do 16 queries per second vs. 90 for BIND 8 and 55 for BIND 9. See page 90 of the version of these slides presented at the RIPE 44 meeting in Amsterdam for more information.

In addition, djbdns (which includes dnscache) depends on a number of external packages which must also be installed in order for the software to function. It appears to have been developed in response to security issues in much older versions of BIND. However, while BIND has adapted and evolved (to make most of these concerns a non-issue), djbdns has not.

Moreover, there are many aspects to the DNS protocol that djbdns does not support, at least not by default. If you want or need any of these features, then you have to figure out how to turn them on, or find a patch from a third-party (not accepted by the author, otherwise it would have been in the package) which adds the feature in question.

Converted from the Mailman FAQ Wizard

This is one of many Frequently Asked Questions.