Differences between revisions 12 and 14 (spanning 2 versions)
Revision 12 as of 2018-05-29 23:48:41
Size: 3453
Editor: msapiro
Comment: Starting to flesh out new page
Revision 14 as of 2018-10-19 23:23:54
Size: 6133
Editor: msapiro
Comment: Getting closer to current state.
Deletions are marked like this. Additions are marked like this.
Line 13: Line 13:
I have done a significant reinstallation on both servers. <<Action(recall,The prior version of this page,rev=10)>> is no longer applicable to these servers. I will be updating this page with more information about the current installs as time goes forward. I have done a significant reinstallation on both servers. <<Action(recall,The prior version of this page,rev=10)>> is no longer applicable to these servers. I have also installed MM 3 on a third server. I have now evolved my process and installations to a stable point. It is this configuration that I will describe.
Line 16: Line 16:
Both servers already had sass installed via Both servers already had sass and memcached installed via
Line 18: Line 18:
`apt install ruby-sass` `apt install ruby-sass`<<BR>>
`apt install memcached`

I installed these on the third server.
Line 22: Line 25:
The third server is Ubuntu 16.04 with Python 3.5, so it was OK as is.
Line 23: Line 28:
For these installs, I opted to install the entire Mailman suite in a virtualenv. The biggest motivation for this choice was the fact that there are already production lists on both servers and using a virtualenv allowed me to do a lot of the work in advance without interfering with the production install. For these installs, I opted to install the entire Mailman suite in a virtualenv. The biggest motivation for this choice was the fact that there are already production lists on both servers and using a virtualenv allowed me to do a lot of the work in advance without interfering with the production install. Possibly influenced by these experiences, I am now using virtualenv for all installs.
Line 27: Line 32:
I already had some things set up in `/opt/mailman` including a `git` subdirectory containing clones of the gitlab `mailman-hyperkitty`, `mailman`, `mailmanclient`, `hyperkitty`, `django-mailman3` and `postorius` projects. I already had some things set up in `/opt/mailman` including a `git` subdirectory containing clones of the !GitLab `mailman`, `mailmanclient`, `mailman-hyperkitty`, `hyperkitty`, `django-mailman3` and `postorius` projects. On the third server I have added `mailman-suite` because I use the settings.py from that project as the basis for mine.
Line 35: Line 40:
on both servers. These are dependencies for using the PostgreSQL database, memcached and the Whoosh backend for HyperKitty archive search. on both servers. These are dependencies for using the PostgreSQL database, memcached and the Whoosh backend for !HyperKitty archive search.

I also did the same, but in a Python 3.5 virtualenv on the third server.
Line 43: Line 51:

The third server uses Apache and Gunicorn so I did

`pip install gunicorn`

there.

=== Migration ===
The prior install on both servers used Mailman Bundler which is obsolete, however for consistency I kept some of the same names. Everything is in the /opt/mailman/ directory. I maintained the prior git/ subdirectory along with two bash scripts, [[attachment:pull_script.txt|pull|view]] and [[attachment:build_script.txt|build|view]] which are used to pull changes from !GitLab and build everything in the venv.

I maintained this on the third server as well.

=== Recommended Configuration ===

Here are my recommendations for the rest of the configuration. The older installs don't adhere to this exactly, but this is what I'm doing going forward.

Create directories:
 * /opt/mailman/mm/bin/
 * /opt/mailman/mm/var/
and files:
 * /opt/mailman/mm/__init__.py (empty)
 * /opt/mailman/mm/mailman.cfg ([[attachment:mailman.cfg|sample|view]])
 * /opt/mailman/mm/mailman-hyperkitty.cfg ([[attachment:mailman-hyperkitty.cfg|sample|view]])
 * /opt/mailman/mm/settings_local.py ([[attachment:settings_local.py|sample|view]])
 * /opt/mailman/mm/settings.py (copy of the mailman-suite settings.py file but with `DEBUG = False`)
 * /opt/mailman/mm/urls.py ([[attachment:urls.py|sample|view]])
 * /opt/mailman/mm/wsgi.py ([[attachment:wsgi.py|sample|view]])
if using Gunicorn:
 * /opt/mailman/mm/gunicorn.conf ([[attachment:gunicorn.conf|sample|view]])
and a symlink:
 * /opt/mailman/mm/logs -> /opt/mailman/mm/var/logs
In the /opt/mailman/mm/bin directory, create these executables:
 * /opt/mailman/mm/bin/django-admin ([[attachment:django-admin.txt|sample|view]]) Script to run Django management commands.
 * /opt/mailman/mm/bin/mailman ([[attachment:mailman.txt|sample|view]]) Script to run mailman commands.
 * /opt/mailman/mm/bin/mailman-post-update ([[attachment:mailman-post-update.txt|sample|view]]) Script to update static web and run migrations following a software update.
if using Gunicorn:
 * /opt/mailman/mm/bin/gunicorn ([[attachment:gunicorn.txt|sample|view]]) Script to start Gunicorn.
=== More ===
More to come, stay tuned.

Introduction

I have now installed Mailman 3 for production on two different servers. This is intended to be documentation of my decisions and experiences in doing so. It will be more of a narrative than a how-to, but I hope a how-to can be distilled from it.

The two servers are lists.mailman3.org (lm3o) and mail.python.org (mpo). Pre-existing things on those servers forced some decisions which were different between them, but the actual MM 3 installation was pretty much up to me, yet I did it differently the second time due to things I thought I'd learned the first time.

A fundamental step in any installation is to ensure that you have the necessary infrastructure for reliably sending mail. This mostly involves DNS records. I won't go into detail, but you need an A and maybe AAAA record for your MAIL FROM domain and rDNS PTR records for the IPs pointing back to the domain. You may also need an MX record for your email domain pointing to the Mailman server. Also good is a mechanism for DKIM signing outgoing mail and DKIM and SPF records for the domain.

This already existed on mpo, but needed to be set up on lm3o.

lm3o is the same server as mirror.list.org/mirror.mailman3.org. This is a mirror of the GNU Mailman web site and was already set up using nginx as a web server. Thus, the MM 3 installation here uses nginx as the web server proxying to gunicorn for WSGI support.

Update

I have done a significant reinstallation on both servers. The prior version of this page is no longer applicable to these servers. I have also installed MM 3 on a third server. I have now evolved my process and installations to a stable point. It is this configuration that I will describe.

Preliminaries

Both servers already had sass and memcached installed via

apt install ruby-sass
apt install memcached

I installed these on the third server.

Also, the servers are both Ubuntu 14.04 and the native Python 3 is Python 3.4, so I had previously installed Python 3.6 from source. It turned out that I also couldn't install libapache2-mod-wsgi-py3 on mpo via apt because that installed a Python 3.4 version. Ultimately on mpo I installed mod_wsgi via pip, but this required re-running configure and make install for Python 3.6 because I hadn't specified --enable-shared initially and that is required to build mod_wsgi.

The third server is Ubuntu 16.04 with Python 3.5, so it was OK as is.

Installation

For these installs, I opted to install the entire Mailman suite in a virtualenv. The biggest motivation for this choice was the fact that there are already production lists on both servers and using a virtualenv allowed me to do a lot of the work in advance without interfering with the production install. Possibly influenced by these experiences, I am now using virtualenv for all installs.

Note that all of this is done as the mailman user.

I already had some things set up in /opt/mailman including a git subdirectory containing clones of the GitLab mailman, mailmanclient, mailman-hyperkitty, hyperkitty, django-mailman3 and postorius projects. On the third server I have added mailman-suite because I use the settings.py from that project as the basis for mine.

I then created a /opt/mailman/mm directory and within that a Python 3.6 virtualenv named /opt/mailman/mm/venv. I then activated the virtualenv and did

pip install psycopg2-binary
pip install pylibmc
pip install Whoosh

on both servers. These are dependencies for using the PostgreSQL database, memcached and the Whoosh backend for HyperKitty archive search.

I also did the same, but in a Python 3.5 virtualenv on the third server.

Then because lm3o uses nginx and Gunicorn to support wsgi apps, I did

pip install gunicorn

there, and on mpo which uses Apache and mod_wsgi, I did

pip install mod_wsgi

The third server uses Apache and Gunicorn so I did

pip install gunicorn

there.

Migration

The prior install on both servers used Mailman Bundler which is obsolete, however for consistency I kept some of the same names. Everything is in the /opt/mailman/ directory. I maintained the prior git/ subdirectory along with two bash scripts, pull and build which are used to pull changes from GitLab and build everything in the venv.

I maintained this on the third server as well.

Here are my recommendations for the rest of the configuration. The older installs don't adhere to this exactly, but this is what I'm doing going forward.

Create directories:

  • /opt/mailman/mm/bin/
  • /opt/mailman/mm/var/

and files:

  • /opt/mailman/mm/init.py (empty)

  • /opt/mailman/mm/mailman.cfg (sample)

  • /opt/mailman/mm/mailman-hyperkitty.cfg (sample)

  • /opt/mailman/mm/settings_local.py (sample)

  • /opt/mailman/mm/settings.py (copy of the mailman-suite settings.py file but with DEBUG = False)

  • /opt/mailman/mm/urls.py (sample)

  • /opt/mailman/mm/wsgi.py (sample)

if using Gunicorn:

  • /opt/mailman/mm/gunicorn.conf (sample)

and a symlink:

  • /opt/mailman/mm/logs -> /opt/mailman/mm/var/logs

In the /opt/mailman/mm/bin directory, create these executables:

  • /opt/mailman/mm/bin/django-admin (sample) Script to run Django management commands.

  • /opt/mailman/mm/bin/mailman (sample) Script to run mailman commands.

  • /opt/mailman/mm/bin/mailman-post-update (sample) Script to update static web and run migrations following a software update.

if using Gunicorn:

  • /opt/mailman/mm/bin/gunicorn (sample) Script to start Gunicorn.

More

More to come, stay tuned.

MailmanWiki: DOC/Mailman 3 installation experience (last edited 2023-11-24 16:16:40 by msapiro)