Differences between revisions 13 and 14
Revision 13 as of 2013-03-21 10:55:41
Size: 16428
Editor: terri
Comment:
Revision 14 as of 2013-03-21 11:00:01
Size: 16577
Editor: f@state-of-mind
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
#pragma page-filename DEV/versions/15958105 #pragma page-filename DEV/versions/15958107
Line 53: Line 53:

=== Authenticated REST-API in Postorius/Django ===

Implement a public facing REST API with authentication/authorization, possibly using OAuth.

GNU Mailman is hoping to participate in Google Summer of Code in 2013.  This page captures some ideas for student projects, links to setup guides, etc.

Table of Contents:

Required Skills

Incoming students will need the following skills:

  • Intermediate python programming
  • Familiarity with source control (Mailman core and Postorius use bzr, Hyperkitty uses git)
  • Ability to set up your own development environment for Mailman 

Additional desired skills may be listed with specific projects.

We're happy to help you get up to speed, but the more you are able to demonstrate ability in advance, the more likely we are to choose your application!

Getting Started

Interested in getting started with Mailman development?   Here are some links for potential students and mentors:

Mailman is written in Python.  For developers, the path of least resistance is probably to use python 2.7+ (sadly not all our requirements handle python 3 yet) and work under Ubuntu Linux, although it is definitely possible to do development on other platforms.  It is definitely possible and encouraged to run your dev environment in a VM, although we do not currently have one set up and available to download.  (If you'd like to set one up and make it available for future developers, please email mailman-developers@python.org so we can help!)

Development work on Mailman 2.1 has been frozen for some time, so all new project ideas should be related to Mailman 3.

Getting in touch with the developers

If you're interested in participating in GSoC 2013 as a student, mentor, or interested community member, you should join the Mailman Developers mailing list: http://mail.python.org/mailman/listinfo/mailman-developers

Post any questions, comments, etc. to mailman-developers@python.org.

In addition you may be able to find us on IRC at #mailman on irc.freenode.org.  If no one is available to answer your question, please be patient and post it to the mailing list as well.  (We *are* the developers for Mailman -- unsurprisingly, most of us prefer to communicate via email.)

Project Ideas

Here are a few ideas for projects we would love to see completed.  If you have another idea you'd like to propose (either as a prospective student, mentor, or interested community member), please send it to mailman-developers@python.org for discussion!

Web Posting Interface

One of the hopes for Mailman 3 is that we can have the archives function seamlessly from the web for people who prefer a "web bulletin board" style of interface.  This means that Mailman could be used as a replacement for something like phpbb.  To achieve this goal, we need to create an interface allowing logged in users to post messages from the web interface, and integrate this into the appropriate pages (e.g. the hyperkitty archives, possibly the postorius list settings pages).  The interface itself may not take the whole summer, in which case you should be prepared to build other features to make the mailman web interface usable as a web forum.  Talk to the developers/users about what might be useful and suggest a few of these when you write your GSoC application.

Task Level: Intermediate

Desired skills (in addition to those named above): experience with django (not necessary, but helpful), web design (css/html/javascript)

Authenticated REST-API in Postorius/Django

Implement a public facing REST API with authentication/authorization, possibly using OAuth.

Scripts for migrating from Mailman 2.1 to Mailman 3

Mailman 3 is nearing release, but we need someone to write scripts to migrate from Mailman 2.1 to Mailman 3, including all list settings, user settings and archives.  Some of these will be easy choices because the settings are similar, but some scripts may be more complex since the way users are dealt with has changed considerably and we may want to merge and normalize settings.

Task level: Advanced beginner

Desired skills (in addition to those named above): none

Better User Settings management

In Mailman 3, users have the ability to change settings for all their lists, for individual lists, and for individual managed email addresses, but handling all of these settings could potentially be a user interface nightmare. 

For example,

  • a list member might want to subscribe an additional address so she can post using multiple identities, but not receive email there.
  • a list member might go on vacation and want to temporarily suspend mail from some mailing lists, but not others
  • a list member might want to receive digests of some lists but not all lists

We need someone to expose these options without making it unbearably confusing for users.  This project will involve not only creating new settings pages, and probably some summary pages where users can see all their settings, but also doing some basic user testing to ensure that the result is actually usable.

Task level: Intermediate

Desired skills (in addition to those named above): User interface design, django (not necessary, but helpful)

Design interface "themes" for specific types of list

The Mailman admin interface exposes a huge number of settings, but many lists will never use them.  We would like to have a variety of simplified interfaces specialized to various uses of Mailman. 

Some examples:

  • An announcement list might require member management, management of authorized posters to the list, and moderation of list postings.
  • A "simple" discussion list might require member management, moderation of list postings, changes to list description, and changes to things that might trigger moderation

Although it is possible to build entirely separate new interfaces to mailman, we realize that many people will want to just use postorius and configure it to suit their needs, so ideally we would have a way of creating these themes from the main interface.  We might also want to use this theme system for more traditional theme elements such as colours and icons.  It's not yet clear the best way to do this: we might be able to achieve it using nothing more than CSS templates (this would be very lightweight and convenient in many circumstances; a fast way for anyone to see an interface customized for their most often used options) but there may also be cases where the site admin will want to disable options in the web interface for some users and not have it possible for them to circumvent it. 

We want someone to take a look at Postorius, the current Mailman web interface, and develop several proof of concept interface themes or views as well as a more general templating system to create more such themes in the future.

Task level: Intermediate

Desired skills (in addition to those named above): intermediate python, experience with django (not necessary, but helpful), web design (css/html/javascript), familiarity with common uses of mailing lists, ability to move past decision paralysis and make executive decisions (we expect there to be some community argument about the best choice when designing the architecture of this system: you need to be prepared to just choose something and do it.)

Enhance List Style Capabilities

There are lots of different ways to use a mailing list: discussion lists, announcement-only lists, anonymous lists, etc.   Mailman 3 supports these as list "styles" and you can see the IStyle interface in src/mailman/interfaces/styles.py for the API and src/mailman/styles/default.py for the two built-in styles, both essentially legacy styles.

Right now, the styles are only applied when the list is created, and the list doesn't record what style was applied when it was made, however it is possible that administrators might like to change them and have relevant lists update.  This project is to do basically that: find a way to maintain styles and update them in a useful way, create a web interface for administrators to modify and apply styles, and create a stable of popular styles for new lists.  We may want a way to update styles without over-riding admin-set options, or even apply multiple styles to a list.  An experienced student may be able to complete this project early, so applicants may want to suggest other features they would like to work on after mid-terms.

Task level: Intermediate

Desired skills (in addition to those named above): understanding of how mailing lists are used in practice,  ui design & django would be helpful for the interface changes, but should not be necessary.

Log monitor

(Suggested by rsk)

Write Python scripts that monitor all of Mailman's logs, integrate what they find, and complain to the appropriate -owner addresses when bad things start happening. Bad things could be any of:

- repeated attempts to subscribe from same IP address/network block

- sudden increase in inbound traffic to otherwise-quiet list

- spike in unsubscriptions (maybe someone is forging them)

- etc.

Bonus points: integrate with HTTP and SMTP logs from which web server and whichever MTA are in play. (But this might might be better as a followon the next year. Given the diversity of the software in play and thus the diversity of the logs, it's a lot. So maybe the goal should be "don't design or code in a way that rules out doing this in the future".)

Task level: Intermediate

Desired skills (in addition to those named above): shell scripting, experience with simple visualizations might be helpful

Boilerplate stripper

(Suggested by rsk)

An increasing number of devices and services are starting to forcibly append boilerplate to users' email messages. This is annoying.

I'm talking about:

        Sent from my Android
        Sent from my Droid
        Sent from my HTC
        Sent from my Nexus

as well as the stuff appended by some freemail services.

 

The interface itself may not take the whole summer, in which case you should be prepared to build other features to make the mailman web interface usable as a web forum.  Talk to the developers/users about what might be useful and suggest a few of these when you write your GSoC application.

Note: This may be a project that can be finished before the end of the GSoC period, so any student proposing on this should have a list of other bugs or features they'd like to work on if this is finished early.


Task level: Intermediate

Desired skills (in addition to those named above): regular expressions would probably be helpful.

Anti-spam/anti-abuse in Mailman

(Suggested by rsk)

One of the general principles of anti-spam work is that abuse control is best done as close to the origination point as possible: thus, stopping spam at your network perimeter is better than stopping it in your firewall is better than stopping it in your MTA is better than stopping it in Mailman.

However...Mailman has localized knowledge that none of the others do, so it's time to craft some serious anti-abuse clue into it -- not just to defend against incoming spam, but against *outgoing* spam, so that list operators don't find their operations blacklisted. What's needed is a flexible piece of code that holds dubious messages for moderation.

An example: freemail account hijackers are sending out messages where the entire body consists solely of a URL. That's an easy test and will stop a lot of spam that will otherwise pass, even on subscriber-only lists (because it's coming from a subscriber's account). Another example: "test" messages. They're either spam or annoying. Another example: messages whose Subject line matches certain key phrases (which I won't include here, in case your incoming mail is content filtered -- mine's not, I don't use it). Another example: any message from often-forged bogus addresses like fbi@gmail.com.

So basically the idea is: have a laundry list of tests, run them sequentially, if there are any hits, hold the message and tell the right -owner about it. The good news is that this is simple and fast: most tests are a regexp match. Also the false positive rate on some of these is zero, as in 0.000. The bad news is that once in a while there will be a false positive, but if it's rare enough and if the message need only be owner-approved to go through, then no harm.

Why *not* do this in the MTA?

  1. MTA doesn't know about lists and subscribers and patterns.
  2. Some servers mix mailing lists and end user traffic and different kinds of anti-spam are appropriate for each.
  3. Messages going *out* need much much more scrutiny than messages coming in -- much more important not to be a spam source/relay then to protect own addresses.
  4. As Mailman gains mindshare, it will come under increasing scrutiny and attack. Now would be a good time to get in front of it. Waiting until the inevitable happens will be too late.
  5. Mailing list owners often don't run the MTA.

Task level: Intermediate

Desired skills (in addition to those named above): familiarity with antispam work

Administrative email message log

(Suggested by rsk)

Add a configurable set of options that will instruct Mailman to keep a log of

(a) all incoming adminstrative messages and

(b) all outgoing administrative messages.

(a) can be kinda sorta done by configuring the MTA to send second copies elsewhere, but (b) can't...and this really should be done inside Mailman.

The log should consist of full copies of the messages in mbox format. This is useful for debugging, for dealing with attacks, AND for proving, when necessary, that someone really did opt-in or really did unsubscribe. (By administrative messages I mean anything to -request, -owner, etc., as well as outbound subscription confirmation requests.)

Task level:  Intermediate

Desired skills (in addition to those named above): none

Better content-filtering/handling error messages

(Suggested by rsk)

When messages have content stripped, make sure that the log includes the Message-ID, the MIME type, the file extension, the rule that fired, etc. so that if this was not what was intended, it's much easier to debug.

Task level: Advanced beginner (the actual technical work should not be too difficult, but you will need to be adept at reading code to find the correct places to make changes)

Desired skills (in addition to those named above): none

Not-fully-defined Project Ideas

These are some ideas that are not as well-defined, kept here for inspiration.  If you want to discuss any of these ideas, please send email to mailman-developers@python.org.

  1. Fixing bugs/adding features from the bug queues

    All of the components of Mailman have reasonably-maintained  bug queues.  It would be possible to build a GSoC project around a few wishlist items or desired bug fixes if none of the above projects appeals to you.  Do make sure to talk to the mailman-developers list to find out if any given set of bugs would be suitable.

  2. Make your own project! 
    Got a feature you've always wanted to see in Mailman?  We're always open for suggestions, just stop by the mailman-developers list to discuss your project idea before you submit an application so we can help you make sure it's the right size for gsoc and that we have the mentoring support available to help you build it!

MailmanWiki: DEV/Google Summer of Code 2013 (last edited 2015-03-04 06:03:22 by msapiro)