How to Install Mailman 3 on a Debian 10 Server.

This document is a How-To on installing Mailman 3 on a Debian 10 server. The server environment will include Debian 10, NGINX, Let's Encrypt, Postfix, and PostgreSQL.

Prepare Server Environment

1. Create a VPS running Debian 10. I recommend using Linode. The server should be a minimum of 1 CPU and 2 Gigabyte of Memory. 25-50 Gb of storage should be sufficient.

2. Log into the server as root.

$ ssh root@your_server_ip

3. Create a New User: mailman

# adduser mailman

You will be asked some questions about this new user when running the above command. Please make sure you use a strong password for mailman as it will be typically known that any server running Mailman 3 will have a mailman user setup on it.

4. I recommend setting up a basic firewall to tighten up security on this Mailman 3 server. UFW is one that can be installed on a Debian 10 server:

# apt update
# apt install ufw

Once UFW is installed, you need to make sure you allow the various applications that Mailman 3 will be using through UFW. Some of these applications are the following: OpenSSH, Nginx Full, and Postfix.

So here is an example of what to do immediately after you install UFW:

# ufw app list

Output
Available applications:
 . . .
 OpenSSH
 . . .

This tells us what available applications we can configure UFW to allow. So let's add OpenSSH to UFW:

# ufw allow OpenSSH

Now we can enable UFW:

# ufw enable

You can see what is allowed through the UFW firewall by running the following command:

# ufw status

Output
Status: active

To             Action   From
--             ------   ----
OpenSSH          ALLOW    Anywhere
OpenSSH (v6)     ALLOW    Anywhere (v6)

Webserver Installation: NGINX

1. Both Django, Postorius, and Hyperkitty requires a web server. So let's install NGINX.

apt install nginx

2. Now that Nginx is installed, we need to allow it through the UFW Firewall.

# ufw allow 'Nginx Full'

Let's verify the change.

# ufw status

Output
Status: active

To                         Action      From
--                         ------      ----
OpenSSH                    ALLOW       Anywhere                  
Nginx HTTP                 ALLOW       Anywhere                  
OpenSSH (v6)               ALLOW       Anywhere (v6)             
Nginx HTTP (v6)            ALLOW       Anywhere (v6)

3. Let's make sure Nginx is running.

# systemctl status nginx

Output
● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2019-07-03 12:52:54 UTC; 4min 23s ago
     Docs: man:nginx(8)
 Main PID: 3942 (nginx)
    Tasks: 3 (limit: 4719)
   Memory: 6.1M
   CGroup: /system.slice/nginx.service
           ├─3942 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
           ├─3943 nginx: worker process
           └─3944 nginx: worker process

If your output has Active: active (running) then you are good to go.

With Nginx confirmed to be running you should be able to access it via your server' IP address:

http://your_server_ipaddress

Create a NGINX Server Block For Your Mailman 3 Domain

This section shows how I set up a Mailman 3 domain for my Debian 10 Mailman 3/Postorius/Hyperkitty installations. YMMV.

# nano /etc/nginx/conf.d/listdomain.com.conf

Add the following to listdomain.com.conf

server {
        listen 80;
        listen [::]:80;

        server_name listdomain.com;

        location / {
                proxy_pass http://127.0.0.1:8000;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Host $host;
                proxy_set_header X-Forwarded-Server $host;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_redirect off;
        }

        location /static/ {
                alias /opt/mailman/mm/static/;
        }


}

# nginx -t

# nginx -s reload

Securing Your Nginx Installation: Let's Encrypt

1. The first step in securing Nginx is to install Certbot. This means first installing the dependencies for Certbot.

# apt install python3-acme python3-certbot python3-mock python3-openssl python3-pkg-resources python3-pyparsing python3-zope.interface

Now we are ready to install Certbot:

# apt install python3-certbot-nginx

Normally you would need to allow SSL traffic through the AFW firewall but we have already done previously in this how-to so we can continue on.

2. The next step is to fetch a SSL certificate for your Mailman 3 list domain name. This how-to assumes you know how to add a server block to Nginx. I keep my server blocks in /etc/nginx/conf.d/. You will need to make sure you have a server block setup already for your domain name before proceeding.

To obtain a SSL certificate for your domain name run:

# certbot --nginx -d listdomain.com

Certbox will go through its motions. At the end it is going to ask you whether you want to redirect all HTTP traffic to HTTPS. Choose 2 if yes which is what I recommend doing.

That's it! We are now ready to install Postfix because after all, Mailman 3 needs to send out mail.

Mail Server Installation: Postfix

1. Ok, we are now ready to install Postfix. The first thing we do here is install mailtuils:

# apt install mailutils

Now we install Postfix:

# apt install postfix

2. Now you will be presented with an interactive dialogue:

General type of mail configuration: Internet Site

System Mail Name: server hostname

3. Let's configure Postfix now. Open the main Postfix configuration file:

# nano /etc/postfix/main.cf

4. Make sure inet_interfaces is set to all:

inet_interfaces = all

5. Now let's set myhostname and mydestination:

myhostname = server_hostname
mydestination = $myhostname, localhost.$myhostname, localhost

6. Add some additional settings for Mailman 3 to work and quality of life after the following line:

inet_protocols = all

unknown_local_recipient_reject_code = 550
owner_request_special = no
always_add_missing_headers = yes

transport_maps =
    hash:/opt/mailman/mm/var/data/postfix_lmtp
local_recipient_maps =
    hash:/opt/mailman/mm/var/data/postfix_lmtp
relay_domains =
    hash:/opt/mailman/mm/var/data/postfix_domains
default_destination_recipient_limit = 30
default_destination_concurrency_limit = 15

header_checks = regexp:/etc/postfix/header_checks

Save and close the file.

7. Run #nano /etc/postfix/header_checks and add the following:

/^Subject:/     WARN

Save and close the file.

8. Time to apply the changes:

# systemctl restart postfix

9. The final step is to allow postfix through the firewall:

# ufw allow postfix

Now you have a working SMTP server and it's time to install and set up a database server for Mailman 3.

Database Server Installation: Postgres

1. So, the first step is to install several packages, some of these are needed for later steps:

# apt install python3-pip python3-dev libpq-dev postgresql postgresql-contrib curl

2. Now that PostgreSQL is installed, let's create the Mailman database and user. First let's open up a Postgres session by typing the following:

# sudo -u postgres psql

Now let's create the Mailman database:

postgres=# CREATE DATABASE mailman;

Now let's create the Mailman user:

postgres=# CREATE USER mailman WITH PASSWORD 'secure-password';

The following is recommended for setting up a Django project which is what is needed for the installation of Postorius and Hyperkitty:

postgres=# ALTER ROLE mailman SET client_encoding TO 'utf8';
postgres=# ALTER ROLE mailman SET default_transaction_isolation TO 'read committed';
postgres=# ALTER ROLE mailman SET timezone TO 'UTC';

The timezone part can be customized to reflect your geographical setting.

The last step is to give access to the mailman user to administer the mailman database:

postgres=# GRANT ALL PRIVILEGES ON DATABASE mailman TO mailman;

To finish this section out, exit the PostgreSQL prompt:

postgres=# \q

Setup of a Python Virtual Environment

Let's make sure we have Python 3 installed.

# apt install python3 python3-venv

Now we are going to be using a virtual environment to install Mailman 3. So let's install that too:

# apt install virtualenv python3-virtualenv

Installation of Potentially Needed Applications

Now that we have a working web, email and database server it’s time to install a number of potentially needed applications. Install the following applications as root:

# apt install sassc lynx
# apt install memcached
# apt install fail2ban
# apt install gettext

Install Dart Sass

# cd /usr/local/lib
# wget https://github.com/sass/dart-sass/releases/download/1.32.5/dart-sass-1.32.5-linux-x64.tar.gz
# tar -xf dart-sass-1.32.5-linux-x64.tar.gz
# chmod -R o-w dart-sass
# ln -s /usr/local/lib/dart-sass/sass /usr/local/bin/sass
# rm -f dart-sass-1.32.5-linux-x64.tar.gz

Test Dart Sass Installation

# sass --version

If you are using Dart Sass, then make sure you set COMPRESS_PRECOMPILERS within settings_local.py to the following:

COMPRESS_PRECOMPILERS = (
   ('text/x-scss', 'sass --style compressed {infile} {outfile}'),
   ('text/x-sass', 'sass --style compressed {infile} {outfile}'),
)

Setup Directories and Files for Mailman 3

Ok, now we need to create some directories and files.

Directories:

# mkdir -p /opt/mailman/mm/bin/ 
# mkdir /opt/mailman/mm/var/

Files:

# touch /opt/mailman/mm/init.py (empty)

# touch /opt/mailman/mm/mailman.cfg sample

# touch /opt/mailman/mm/mailman-hyperkitty.cfg sample

# touch /opt/mailman/mm/settings_local.py sample

# touch /opt/mailman/mm/settings.py (copy of the mailman-suite settings.py file but with DEBUG = False)

# touch /opt/mailman/mm/urls.py sample

# touch /opt/mailman/mm/wsgi.py sample

# touch /opt/mailman/mm/gunicorn.conf sample

Now let's create a symlink for Mailman 3 logs:

# cd /opt/mailman/mm
# ln -s /opt/mailman/mm/var/logs logs 

This symlink will be broken. Don't freak out! The /opt/mailman/mm/var/logs directory will be created when we install Mailman 3.

Now we need to create some executables for Mailman 3.

/opt/mailman/mm/bin/django-admin sample Script to run Django management commands.

/opt/mailman/mm/bin/mailman sample Script to run mailman commands.

/opt/mailman/mm/bin/mailman-post-update sample Script to update static web and run migrations following a software update.

/opt/mailman/mm/bin/gunicorn sample Script to start Gunicorn.

Make sure /opt/mailman and everything below is owned by mailman.mailman.

# chown -R mailman.mailman /opt/mailman

Modify the mailman.cfg, mailman-hyperkitty.cfg, and settings_local.py files to match your particular server setup. Make sure you add the Postgresql password for the mailman db user in mailman.cfg as that is hard to miss.

Create and Enter Virtual Environment

Make sure you are doing this step as the mailman user:

# su mailman

# python3 -m venv /opt/mailman/mm/venv

# source /opt/mailman/mm/venv/bin/activate

To leave the virtual environment:

# deactivate

Installation of Additional Applications via PIP

Before proceeding, make sure you are logged in as the mailman user AND you are in your virtual environment. The directory you should be in is /opt/mailman/mm.

# pip install wheel

# pip install psycopg2-binary

# pip install pylibmc

# pip install gunicorn

# pip install mailman

# pip install hyperkitty

When installing Hyperkitty via pip, you may run into the following error:

• error: Can not find Rust compiler

This is due to changes in the Cryptography module which is required by Django. If you run into this error, here is how to fix it.

A. Install the latest version of Rust (ref: https://www.rust-lang.org/tools/install):

# cd ~ (done as root user)
# curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# rustc --version (to verify installation)

B. Now install additional software:

# apt-get install build-essential libssl-dev libffi-dev python3-dev cargo

Now proceed to reinstalling the Hyperkitty module and everything else:

# pip install hyperkitty

# pip install postorius

# pip install mailman-hyperkitty

# pip install whoosh

# pip install importlib_resources==1.1.0

Once the above packages are installed run “mailman info” in your virtual environment. That will install Mailman core into your /opt/mailman/mm/var directory.

Setup Startup Scripts

Setup your systemd startup scripts and place them in /lib/systemd/system/. Finally, we need to arrange that all the services are running. Generally, the OS has taken care of init, upstart or systemd scripts for Postfix, the web server, PostgreSQL and memcached, but other things that need to be run are Mailman core, Django's qcluster and Gunicorn if that is used.

• Systemd mailman.service

• Systemd qcluster.service

• Systemd gunicorn.service

Post Installation Tasks

Now it is time to run the Mailman Post Update script:

(venv) # /opt/mailman/mm/bin/mailman-post-update

Create Django Superuser

The next item to do is create an admin user for the Django interface. To do that run:

# /opt/mailman/mm/bin/django-admin createsuperuser

It will prompt you for a username, email address and password. After the above script finishes its course, its time to get back to root:

(venv) # deactivate

mailman@ # exit

Start Mailman 3 Core, Django, and Gunicorn

Start mailman, django, and gunicorn using the above created system scripts:

Mailman 3 Core

# systemctl start mailman

Django

# systemctl start qcluster

Gunicorn

# systemctl start gunicorn

Setup Cron Jobs

Log in as Mailman user:

# su mailman

crontab -e

Add the following cron entries:

@hourly  /opt/mailman/mm/bin/django-admin runjobs hourly
@daily   /opt/mailman/mm/bin/django-admin runjobs daily
@weekly  /opt/mailman/mm/bin/django-admin runjobs weekly
@monthly /opt/mailman/mm/bin/django-admin runjobs monthly
@yearly  /opt/mailman/mm/bin/django-admin runjobs yearly
0,15,30,45 * * * * /opt/mailman/mm/bin/django-admin runjobs quarter_hourly
* * * * * /opt/mailman/mm/bin/django-admin runjobs minutely

# Send periodic digests.
30 3 * * * /opt/mailman/mm/bin/mailman digests --periodic

# Send request reminder for MM 3. Like the checkdbs job for 2.1
0 8 * * * /opt/mailman/mm/bin/mailman notify

Setup Xapian for Indexing Archives

How to keep Mailman 3 Updated