6.16. Understanding group mismatch errors - how mailman implements security

If you have ever seen a group mismatch error and would like to fix it, or are just wondering how mailman implements part of its security then read on.

Mailman implements most of its security via group membership. Permission to perform certain operations is determined by membership in a group. There are 3 major software components that interact in a mailman installation, mailman, the email server (MTA), and the web server. Both the MTA and the web server will invoke programs that are part of mailman to perform operations. Mailman wants to be sure whoever invokes it is trusted and has permission to perform these operations. This security "check" is performed by "wrapper" programs.

The MTA invokes the "mailman" wrapper program, and the web server invokes CGI programs in the cgi-bin directory. The security check is identical for both the MTA and web server, differing only in the Group ID (gid) that is being validated. For the MTA this gid is set with the configure parameter --with-mail-gid and for the web server it is set with the --with-cgi-gid parameter. The rest of this discussion will use the MTA, the mailman progam, and the mail-gid as an example, the web server and cgi-gid are handled identically.

When the wrapper is built the mail-gid (via --with-mail-gid) is hardcoded into the source code of the wrapper.

Normally when a process is invoked it is run with the owner and group of the process that invoked it. It does not execute with the owner and group belonging to the executable (unless it is setuid or setgid respectively).

The mailman executable is setgid mailman. This means no matter who runs it, it will execute with its group set to mailman. Mailman's security is group based, anything mailman attempts to do will only succeed if the process attempting to perform the operation is a member of the mailman group. This is why the mailman "wrapper" is setgid mailman. No matter who invokes it, it runs as if it were a member of the mailman group (not the group of process that invoked it). Thus it has permission to perform mailman operations because it is executing as a member of the mailman group.

But wait! That means anybody can invoke the mailman wrapper program and perform mailman operations because the wrapper when it starts to execute will immediately assume the mailman group identity granting it full mailman permissions. Thus we need a way to say "only a select set of trusted processes can invoke me". In other words, if somebody askes me to run and do mailman operations, do I trust the entity that asked me to do this? The trust question is answered by identifying the group of the process that asked me to run, in short, "if you're not a member of a group I trust I refuse to perform mailman operations".

This check is performed by looking at the real gid (not the effective gid) of the wrapper once it starts running. The real gid will be the gid of the process that invoked it. For the mailman wrapper this would be the gid of the MTA. The wrapper asks the simple question, is the real gid I'm executing under identical to the gid I was told belongs to the MTA? Remember this gid is hardcoded via the --with-mail-gid configure parameter. If they match mailman considers this a trusted invocation and the wrapper continues to execute. Recall that because the wrapper is setgid mailman it executes with its effective group set to mailman, not the MTA's group. This is what we want, we want mailman to perform its operations as if it were a member of the mailman group no matter who invoked it. We've also validated whoever invoked us is trusted. If the group check fails you get a group mismatch error.

To fix a group mismatch error you essentially have two choices. Either rebuild mailman with the --with-*-gid parameter set to match the group your MTA and/or web server invokes it's child processes under. Or you can configure your MTA or web server to use the group mailman was built with. Not all MTA or web servers can do this and the mechanics of how to configure the MTA or web server are unique to the server, you'll have to consult your servers documentation.

Converted from the Mailman FAQ Wizard

This is one of many Frequently Asked Questions.