Google Summer of Code 2016 Ideas Page
Mailman is free software for managing electronic mail discussion and e-newsletter lists. Mailman is integrated with the web, making it easy for users to manage their accounts and for list owners to administer their lists. Mailman supports built-in archiving, automatic bounce processing, content filtering, digest delivery, spam filters, and more. Many open source denizens will be familiar with Mailman 2.1, which is used to manage many project mailing lists. All of our current work is on Mailman 3, which was released in 2015, but there's still lots of room for new features and ideas!
We're in! GNU Mailman has been selected to participate in Google Summer of Code (GSoC) 2016! This page presents of some of the project ideas and guidelines on how to participate as a student or mentor.
Not sure about how GSoC works? Check out the GSoC student Guide
Contents
Getting Started
The GSoC Student Guide has some great general GSoC information. Read it first!
Before you ask any questions about how to do anything in git or initialize your repository please read through the git howto and see if that answers your questions.
Recommended Skills
Here's a few skills that will make it easier for you to get started as a Mailman developer:
Python programming. You can be fairly new to Python, but you're going to have to be comfortable reading the Mailman code and asking questions if you aren't sure how it works.
Familiarity with any version control. We use git on Gitlab, but any experience will help you here.
Ability to setup a development environment for Mailman. You'll need access to a Linux machine or VM, and you can get help setting it up.
Ability to communicate with the Mailman developers. You'll be expected to post to the public mailing lists, ask questions, and describe the work you're doing.
Specific projects may have additional desired skills listed along with them.
What is Mailman Suite?
In Mailman 3, we've divided up the code into a number of sub-projects. We refer to the whole package as "Mailman Suite" and there's a few really important pieces you should know:
Mailman Core - This is the part that actually sends and receives mail and handles subscriber and list information.
Postorius - A web interface for managing Mailman lists (e.g. subscribing, changing preferences)
Hyperkitty - A web interface to access GNU Mailman v3 archives.
Mailman Bundler - An installer to help people who want to use those 3 big pieces together.
There's also a number of smaller projects that provide the glue to make these pieces work well together, or allow them to be used separately.
All the Mailman source is now on GitLab.
Documentation & Installation
Here are some useful links to get you started with Mailman Development:
- TODO: link more developer documentation here
Mailman is written in Python. Mailman 3 core is compliant only with Python 3 (MM3.0 works with Python 3.4 only, but MM3.1 will work with 3.4 and 3.5) and the rest of the projects work on python2 (2.7+). You are not required to use a virtual machine for Mailman development, although many people prefer to do so.
Development work on Mailman 2.1 has been frozen for some time, so all new project ideas must be related to Mailman 3.
Contacting us
The most successful GSoC students are the ones who work closely with the Mailman team. As you might expect from a group that makes mailing list software, our preferred method of communication is our mailing list, but we also have an IRC channel. Please do not send private messages or emails unless asked to do so or the subject is personal rather than technical, since we get a lot of similar questions and would rather the questions and answers happen in public so everyone can benefit.
Mailing list to use for GSoC discussions: Mailman developers mailing list
- IRC channel: #mailman on irc.freenode.org
Writing Your Application
The GSoC Student Guide has some great information
Tips for Prospective Google Summer of Code Students (general good advice)
Tips for writing a good GSoC proposal (advice on structuring and composing the document)
You might also want to look at Last year's page
FAQ
* I am interested to contribute to Mailman and participate in GSOC, how do I start?
This page might be of some help.
This question has been asked tons of times on the mailing list (mentioned above) and IRC Channel (again, mentioned above). Both of them are logged publicly and are searchable. It would be really nice to go through them once before you ask the same question again. More specific questions are encouraged and receive more attention than "How do I start?". Mentors do try hard to reply to each and every email to the developers list, but in case you don't get any replies on trivial questions, don't be discouraged. Also, in case you are a student, helping others with small problems that you know about is also a great way to get noticed. Don't worry too much about giving wrong answers, list is constantly monitored by the core developers and your mistakes would be corrected.
* I am a beginner, what is the best way for me to contribute?
Issues, bugs and tests are obviously a good way to contribute. But, writing/improving documentation is another great way to contribute for beginners. It doesn't have to be a real commit or patch, write up a detailed blog post about what problems you faced while setting up mailman for development. What parts of mailman you find are difficult to understand? What are awesome things about mailman?
* My MR has been pending from a long time, what should I do?
In most of the cases the reason for a MR to be pending is that there is either something missing or something incorrect. In either of those cases if you think you need more feedback, contact me (maxking) and I can look into it, or ask the relevant person. I don't mean that you do that for each one, just the ones that haven't been noticed by anyone for a *long time*.
Project Ideas
If you have other idea you'd like to propose , please send it to mailman-developers@python.org for discussion! You can also look at the to-do list for Mailman 3.0 here, and see if anything is interesting enough that you would like to work on it through the summer.
Preset list settings templates (aka list styles)
Mailman can be used for a variety of different purposes: Discussion lists, announcement lists, newsletters and more. Each of those need different settings that currently need to be applied for each list separately.
The idea of this project is to make it easier for site admins to apply certain list styles to a list (or even multiple lists at once) by providing a number of fixed templates or styles that take care of the list configuration. e.g. When you want an announcement list, you'd be able to select "announcement list" and it would set it up so all posts must be vetted by the admin, posting instructions could be updated in subscriber emails, etc.
Ideally, this might also have a web interface (as part of Postorius?) where administrators can manage styles without having shell access to the machine. Minimally, we'd want to let web interface users choose from a set of templates provided by the admin.
Note that we are NOT talking about visual styles here. This project is about functional differences in the way the lists work. Barry called them "list styles" in Mailman core so you'll see them referred to that way, though.
This project will likely touch Mailman Core and Postorius.
- Difficulty
- Easy/Beginner-friendly
- Mentors
- Abhilash, Florian, Terri. All students interested in this task should discuss it on the mailman-developers mailing list.
GitLab/development tools integration
Per https://twitter.com/gvwilson/status/535279362871144448 -- a tool that will take a thread from a Mailman mailing list and turn it into a thread on a GitLab issue. Or any kind of GitLab API integration tool (e.g., to automatically mail a Mailman list when a repo gets a new GitLab pull request)
This doesn't necessarily have to be gitlab -- launchpad, github, gerrit, jira, bitbucket, etc. would all be valid targets. Mailman uses GitLab, so we'd love to see that one in particular (because we'd be able to use it ourselves!), but as GitHub is extremely popular, it might be a good second target (or even first if you want to maximize general appeal!)
- Difficulty
Easy/Beginner-friendly (for the thread->issue part; if you add in some other integration tools you can make this task more challenging!)
- Mentors
- Any mentor can help with this task. All students interested in this task should discuss it on the mailman-developers mailing list.
Shared bookmarking toolkit
A shared bookmarking tool that listens to a Mailman list, pulls out URLs that people mention, and adds them to an OPML file or Pinboard.
This idea by itself is may not be enough for a Summer of Code project, so interested students should brainstorm about some related tools that might be useful and discuss them with the developers, or select a set of smaller projects that could all be done by a single student.
WARNING: Currently this is the only such small project on the Mailman ideas list; you may want to wait until there are a few smaller projects available before seriously putting together a project proposal with several smaller pieces. At the very least, discuss any other small project ideas with the developers: we get a lot of suggested ideas that aren't suitable for Mailman and don't want you to waste time developing them if they're things we'll never be able to integrate. Thanks!
- Difficulty
- Easy/Beginner-friendly
- Mentors
- Any mentor can help with this task. All students interested in this task should discuss it on the mailman-developers mailing list.
Message queue based email archiver
Hyperkitty is mailman's official archiver, however it would be a good idea to add an alternative to it based on message queues, either publish-subscribe or a fifo message queue. Possibly an implementation which could be extended to use several different backends (like zeromq, redis etc) of choice would be best. (Support or thoughts on pub/sub support for other system events in mailman would be a plus, though not necessary).
- Difficulty Level
- Easy to moderate, depending on the actual proposal of the student.
- Mentors
- Florian, Aurelian. All students interested in this task should discuss it on the mailman-developers mailing list.
- Skills
- Knowledge about message/event queues and the pub/sub messaging pattern
- More reading
- * Look for IArchiver interface in mailman's source code.
Implement module to process ARC headers
From Mailman's point of view, Authenticated Received Chain (ARC) is a protocol that can help mitigate denial of service to subscribed addresses at Yahoo!, AOL, and other sites that have a "reject" DMARC policy. In trying to track spam, we often trace the Received header fields to find out where the spam most likely came from. However, as with any header field, Received is easy to forge. ARC prevents forgery by digitally signing an ARC-Seal header that provides the necessary trace information, as well as a DKIM-like ARC-Message-Signature field that signs the message itself as transmitted by the ARC host. Finally, the first ARC host to receive custody of the message also provides an ARC-Authentication-Results field which gives the result of the DMARC From alignment test at that host. A recipient which chooses to trust the host that verified the ARC-Authentication-Results may then decide to accept that as evidence of From alignment even though the originator's signature is invalidated by later changes to the message. (This is how DMARC denial of service can be alleviated.)
Like most such protocols, it is generally preferable that ARC be implemented in the MTA (eg, Postfix or Exim). However, experience with spamfiltering shows that such features are unavailable in many hosting environments, or admins prefer to put them all in Mailman for some reason. Also, ARC is an experimental, unproven protocol, and it is likely that many hosts will not implement it for some time.
This project involves implementing the ARC protocol specification, based on the DKIM signature specification. The canonicalization rules in the ARC specification are clearly explained, but must be precisely implemented; any mistake will result in signatures that can't be validated. However, the canonicalization process can be modeled on existing DKIM packages from PyPI. The digital signature itself will also be generated using available modules (DKIM and OpenSSL, for example).
Additional background reading that is not necessary for implementation, but may be of interest if you'd like to learn more about how we're trying to secure the venerable email system: Recommended Usage of the Authenticated Received Chain (ARC), Interoperability Issues Between DMARC and Indirect Email Flows, and Domain-based Message Authentication, Reporting, and Conformance (DMARC), as well as references contained in these documents. Feel free to explain your interest to Stephen and ask if it would be useful for you to read any particular document.
- Difficulty
Moderate. An implementer will need to learn to read and understand RFC-style protocol specifications, and implement them very precisely. For integration extensive testing will be required. Algorithms are all specified by the protocol, no algorithm design and little optimization will be necessary. Required Mailman background will likely be restricted to understanding the architecture and APIs of chains and pipelines, and adding a configuration option to enable the feature.
- Mentors
- Stephen, Barry
Discourse integration
Some people like web forums and others like email discussions. Discourse is FLOSS discussion forum software and it would be interesting to integrate this with GNU Mailman so that users could choose whichever interface they prefer, and still participate in a single, integrated thread. This project idea may be ambitious and is definitely a bit speculative, so the prospective student will have to do some research on feasibility and general technical approach.
This project will almost certainly involve the core, and may also touch on HyperKitty or Postorius.
- Difficulty
- Advanced
- Mentors
- Barry, Florian
More project ideas coming soon!
For Mentors
If you're interested in mentoring, please join mailman-developers@python.org and let us know!
Current list of prospective mentors
- Terri Oda (terri on IRC)
- Abhilash Raj (maxking on IRC)
- Stephen J. Turnbull (yaseppochi on IRC)
- Florian Fuchs (florianf on IRC)
- Barry Warsaw (barry on IRC)
- Sneha Priscilla Makini (stylistica on IRC)