Google Summer of Code 2015 Ideas Page

GNU Mailman plans to participate in Google Summer of Code (GSoC) 2015. This page presents of some of the project ideas and guidelines on how to participate.

Contents

  1. Google Summer of Code 2015 Ideas Page
    1. Skills Required
    2. Expectations
    3. Getting Started with Mailman
      1. Links to Mailman3 source code
    4. Want to get involved?
      1. Where to get help?
      2. Mentors Are Not Teachers
    5. Project Ideas
      1. A Dashboard for Admins/Owners/Moderators
      2. Subscriber profile pages
      3. GitHub/development tools integration
      4. Shared bookmarking toolkit
      5. Better handling of list styles
      6. Continuous Integration for Mailman Suite
      7. Mailman Client written in Javascript
      8. Anonymous lists
      9. Dynamic Sublists
      10. Build 2-3 Mailman plugins
      11. Message queue based email archiver
      12. Improve and integrate Vote-MM patch
    6. More ideas
      1. Propose your own idea!
    7. Mentor List

Skills Required

These skills are appreciated but their absence in no way disqualifies anyone from participating. Specific projects may have their desired skills listed along with them.

  • Python programming. You can be fairly new to Python, but you're going to have to be comfortable reading the Mailman code and asking questions if you aren't sure how it works.
  • Familiarity with any version control (Mailman core and Postorius use bzr, Hyperkitty uses git).
  • Ability to setup your own development environment for Mailman

Note: You can use a version control system of your own choice for playing around, but we accept merge requests for Mailman core/.client and Postorius only through Launchpad (which uses bzr).

We encourage you to discuss your questions with us so that we can help you get up to speed. But the more you are able to demonstrate ability in advance, the more likely we are to choose your application!

Expectations

GSoC is a big chance to hack on an open source project, work with experienced developers, learn things that you might not find in a text book, get your code out there -- and get paid for all that! If you already look at it like that then the following list of expectations will probably not trouble you. But we have to set up some basic rules for working together, and it is important that you follow them:

  • 40hrs/week on an average. The exact time may be more or less depending on the requirement of the project. You can compensate for slack in next or even previous weeks, but GSoC is a full-time commitment.

  • We must know about any other involvements (of more than a day or two), so mention them in your application.

  • Your mentor(s) need to know where you stand. They are happy to spend a significant amount of time helping you -- use it!
  • Bi-weekly blog posts on the work you have been doing. Progress doesn't mean getting tasks done every day. There will be times where you get stuck. That's normal. Letting us know what's difficult is equally important as telling us about a cool new feature you just finished.
  • Regular communication with the community on our mailing list. Open source is equally about writing software as about discussing it with the community.

Also see this page for PSF expectations.

Important: One of the biggest no-nos during GSoC is disappearing without your mentor(s) knowing about it. Communication is the only way for us to effectively measure what you are working on, so please know that not delivering regular updates will lead to a failure at midterm.

Getting Started with Mailman

Here are some useful links to get you started with Mailman Development in GSoC.

Mailman is written in Python. Mailman 3 core is compliant only with python3.4 and the rest of the projects work on python2 (2.7+).You are not required to use a virtual machine for Mailman development, although many people prefer to do so.

Development work on Mailman 2.1 has been frozen for some time, so all new project ideas must be related to Mailman 3.

All of the above listed projects are a part of Mailman Project. Your proposal may involve any one of them or a combination of them.

Want to get involved?

If you're interested in participating in GSoC 2015 as a student, we suggest that you fix at least one bug for any of the projects mentioned above before submitting your application. There are several beginner friendly bugs for each of the projects listed on their respective launchpad bug trackers (Hyperkitty has separate one here).

Where to get help?

If you're interested in participating in GSoC 2015 as a student, mentor, or interested community member, you should join the Mailman Developers mailing list: http://mail.python.org/mailman/listinfo/mailman-developers and post any questions, comments, etc. to mailman-developers@python.org . But we want you to go through the resources mentioned above before asking questions, most of the times the answer is in there somewhere. Developers would be more enthusiastic to help you if you yourself have done some ground work.

In addition you may be able to find us on IRC at #mailman on irc.freenode.org. If no one is available to answer your question, please be patient and post it to the mailing list as well. (We *are* the developers for Mailman -- unsurprisingly, most of us prefer to communicate via email.)

Mailman has in the past participated under the Python Software Foundation's umbrella, and will again this year. The PSF has some additional information (including other IRC channels and mailing lists) here: https://wiki.python.org/moin/SummerOfCode/2015

Mentors Are Not Teachers

GSoC mentors are mentors, not teachers. A mentor teaches, it's true. The difference is partly in content and partly in the educational model. Teachers implement a narrow "push model" of education. By "narrow" I mean that they mostly teach technical content. By "push model" I mean they start from a fixed syllabus of topics, that's all they teach, and then they test you on what they decided to teach you. What mentors do is based on a broader "pull model": they help you with human relations (including with yourself, making and keeping schedules, honesty about progress, as well as dealing with other developers and the users whose requirements you are implementing), with design and artistic aspects that are hard to convey with a "lecture and test" model, as well as with the technical details. But it's a pull model in the sense that the student needs to get started on the task, do the best they can within a reasonable time (schedules!), and then when done or blocked, get advice from the mentor. The goal for the student is not to get high marks from the mentor, but to establish a working relationship that will enable her to complete the project with the highest quality product she is capable of.

Teachers tell you what you need to do, then grade you on how much of it you did, and did "right". Mentors don't. Mentors let you decide what to do (and make mistakes even at that stage!), then help you to fix it (without explicitly giving you a grade). Of course we do give you a grade in the end, but we do both a lot more work (the "broad" aspect) and a lot less work (the "wait for a pull from the student" aspect) than classrom teachers do in the process.

At this stage, there are two important implications for your proposal (including preliminary discussion on the mailing list or IRC).

  1. Even if the task description is sketchy, it's your job to fill it out with subtasks or stages until it becomes a summer's worth of work. If you can't do that, you should consider thinking about a different project. (On the other hand, if the task description is sketchy, your plan to get it done can contain equally sketchy steps and proposals if that's the best you can do.)

  2. Providing a schedule is crucial. Making a schedule is hard (that's one of the things that mentors teach that teachers rarely if ever do), but we need to know how hard you think the work you propose is, and how long you think it's going to take. How you respond to advice about your schedule is important information for our decisions about who to take on as GSoC interns.

Bottom line: Don't post "I'm interested in task X. Tell me how to do X." That's guaranteed to make at least one mentor (the author of this section) immediately lose interest in you. You need to convince us you have a plan to do the project. For example, write something along the lines of "to do a good job of X, I need to (a) study similar implementations, (b) specify my requirements, (c) find packages that do these things Y and Z that X needs to do, (d) write this code to link Y to Mailman's related API, (e) write that code to convert Mailman list properties to Z API" etc, etc. If you can't get that detailed, well, do the best you can. "Sketchy" is OK at this stage! (You do need to be more concrete in your proposal.)

Be active, not passive, about defining your own project! That's what we value most, more than past experience in number of projects or claimed coding skills.

Project Ideas

If you have other idea you'd like to propose , please send it to mailman-developers@python.org for discussion! You can also look at the to-do list for Mailman 3.0 here, and see if anything is interesting enough that you would like to work on it through the summer.

A Dashboard for Admins/Owners/Moderators

A single page that lists all to dos in one place (subscription requests, held messages). Probably useful for owners/moderators of multiple lists. Right now, each list gets its own page so owners of multiple lists must click through each one to make changes.

This task is primarily going to be part of Postorius.

Skills
Javascript, HTML.
Difficulty
Easy/Beginner-friendly
Mentors
Florian, Terri. All students interested in this task should discuss it on the mailman-developers mailing list.

Subscriber profile pages

Let users opt-in to show their membership to other list members. Possibly along with some additional user information. This could be useful for more private lists if you want to get an idea who your audience is.

This task will probably span both Postorius and perhaps also Hyperkitty. (see HyperKitty bug 74 and Postorius bug 1104497).

Skills
Javascript, HTML.
Difficulty
Easy/Beginner-friendly
Mentors
Florian, Terri, Aurelian. All students interested in this task should discuss it on the mailman-developers mailing list.

GitHub/development tools integration

Per https://twitter.com/gvwilson/status/535279362871144448 -- a tool that will take a thread from a Mailman mailing list and turn it into a thread on a GitHub issue. Or any kind of GitHub API integration tool (e.g., to automatically mail a Mailman list when a repo gets a new GitHub pull request)

This doesn't necessarily have to be github -- launchpad, gerrit, jira, bitbucket, gitlab, etc. would all be valid targets. But as GitHub is popular and familiar to many developers, it may be a good place to start.

Note that some Mailman developers may be moving some of their own working branches to GitLab (as a GNU Project we prefer to avoid services based on closed/proprietary software for Mailman work, so GitHub is inappropriate). With the acquisition of Gitosis by GitLab, GitLab is the obvious alternative for free software development. For this reason we would like to see GitLab integration ourselves. This doesn't mean you shouldn't work on GitHub or other closed services if that's what you need, just that if you don't have a strong preference, we do prefer GitLab.

Difficulty
Easy/Beginner-friendly
Mentors
Any mentor can help with this task. All students interested in this task should discuss it on the mailman-developers mailing list.

Shared bookmarking toolkit

A shared bookmarking tool that listens to a Mailman list, pulls out URLs that people mention, and adds them to an OPML file or Pinboard.

This idea by itself is may not be enough for a Summer of Code project, so interested students should brainstorm about some related tools that might be useful and discuss them with the developers.

Difficulty
Easy/Beginner-friendly
Mentors
Any mentor can help with this task. All students interested in this task should discuss it on the mailman-developers mailing list.

Better handling of list styles

Mailman can be used for a variety of different purposes: Discussion lists, announcement lists, newsletters and more. Each of those need different settings that currently need to be applied for each list separately. Note that we are NOT talking about visual styles here. This project is about functional differences in the way the lists work.

The idea of this project is to make it easier for site admins to apply certain list styles to a list (or even multiple lists at once) by providing a number of fixed templates or styles that take care of the list configuration. Ideally, this might also have a web interface (as part of Postorius?) where administrators can manage styles without having shell access to the machine.

This project will likely touch Mailman Core and Postorius.

Difficulty
Easy/Beginner-friendly
Mentors
Abhilash, Florian, Terri. All students interested in this task should discuss it on the mailman-developers mailing list.

Continuous Integration for Mailman Suite

Setting up continuous integration for all mailman projects with support for different database backends (sqlite and pgsql are a must, others should be easy to add). It should also enable developers to test their separate branch. Look up travis-ci if you have no idea what continuous integration means.

Difficulty Level
Easy/Beginner-friendly
Mentors
Any mentor can help with this task. All students interested in this task should discuss it on the mailman-developers mailing list.
More reading
* Look into archives of mailman-developers list for previous discussions on the topic during last GSoC.
  • Buildbot - An open source testing framework (you can obviously look for better alternatives if you want).

Mailman Client written in Javascript

mailman.client is the client library that talks to core's REST API through HTTP. Right now we have one official client in python but it would be nice to have it in Javascript as well so mailman could be integrated with NodeJS/IO.js.

Difficulty Level
This is an intermediate level task, best suited for those having prior experience with building NodeJS applications.
Mentors
Florian, others. All students interested in this task should discuss it on the mailman-developers mailing list.
Skills
Some level of experience in working with Javascript is required (since it would involve a fairly good amount of JS coding).
More Reading

* Source and documentation of the current mailman-client

  • Look for REST Server documentation in mailman core.

Anonymous lists

When a user posts to such lists his email-address is disguised as <foo>@<bar>, and when someone replies to the same address it should be delivered back to the intended recipient.

Difficulty Level
Intermediate
Mentors
Abhilash, Steve, others. All students interested in this task should discuss it on the mailman-developers mailing list.

Dynamic Sublists

Dynamic Sublists is an addition to Mailman by Systers that allows the dynamic creation of topics/sublists that users can then subscribe/unsubscribe from and filter more easily.

More details can be found on the Dynamic Sublists wiki page

We'd like to use dlists in Mailman 3 as a replacement for the old topics system of Mailman 2. This project is to port the existing mailman 2 code for dlists to mailman 3, which is a fairly complex task in and of itself. There was a Systers project to do this a few years ago, so you may find references to that. The project was never completed.

The existing lists code is available in Systers' launchpad repositories. If you find the branch with only the dlists changes (there used to be one), please link it here.

Difficulty Level
This is at least an intermediate task, as the ideal student will have to follow this through Mailman Core, Postorius (to add a web interface for subscribing/unsubscribing) and Hyperkitty (to make the archives aware of the dlists). You will also need to be able to read code built for Mailman 2 and translate to Mailman 3.
Mentors
Terri. All students interested in this task should discuss it on the mailman-developers mailing list.
Skills
Understanding of mailing list communication. For example, you should have a rough idea of some answers to the following questions: "How do people talk on mailing lists? What sort of threads would they want to unsubscribe from? What sort of interface would work best for this?" (although be open to working with real users!)
More Reading

* Dynamic Sublists wiki page

  • Mailman bug "Rip out or redesign the topic system" (see bug 975705)

  • "Dynamic Sublists: Scaling Unmoderated Mailing Lists" LISA paper from 2001 by Ellen Spertus, Robin Jeffries, and Kiem Sie. This paper describes the dynamic sublists system in detail.

  • The systers-dev list is a public list that uses the code, if you want to examine it on a live server. (The folk who know the dlists code and have worked with previous attempts at porting it are also all subscribed to this list.)

Build 2-3 Mailman plugins

Note: This project is controversial among the mentors, because most of the proposals so far don't seem to have enough content to fill up a summer, but GSoC doesn't encourage doing multiple unrelated tasks. Read the previous discussions on the mailing list!

Mailman 3 has a new plugin system that allows people to build rules that run before mail is delivered, but it's not entirely obvious to people how this will work since there aren't any complex plugins built yet. This project would involve choosing a couple of interesting plugins and building them both as useful pieces in their own right but also as examples for developers to build upon later.

Interested in this idea? Make sure you go read up on what the plugin system does and does not do before you propose an idea, as we're hoping for at least one of your plugins to work entirely within what's possible now. (The documentation is stored near the code in Mailman Core) Thinking of ideas that extend the system is great, but you might want to think of them as "stretch goals" for your GSoC application.

For example, you probably want to read:

  • src/Mailman/mailman/src/mailman/core/chains.py
  • src/Mailman/mailman/src/mailman/core/pipelines.py

Some plugin ideas::

  • Spam Assassin integration. This would involve making it possible for list admins to set a threshold for their list and have Mailman discard any messages with a Spam Assassin score higher than the threshold.
    • This would involve some work in Postorius to add an interface as well as work with Mailman Core for the plugin.
    • Setting up Spam Assassin can be non-trivial, so make sure you know how to do this before proposing this plugin idea!
  • Banned word list. This plugin would check messages against a banned word list (e.g. racial slurs, words related to discussions that need a cooling-off period) and either hold or discard messages based on what the admin wants done.
  • Checking for personal information. On public lists, sometimes people mistakenly post phone numbers or addresses to the list and want them removed from archives when they notice. This plugin would look for things that look like personal information (e.g. check for common phone number, email address, postal address formats) and take appropriate action (perhaps mailing the person to ask them to confirm that they really want this posted? perhaps emailing the admin? discarding the post?)
  • Add your own here!
    • Difficulty Level
    Intermediate or advanced, since you will be the first person to use the plugin system and will have to be comfortable with suggesting architectural changes if needed
    Mentors
    Barry, Stephen, others. All students interested in this task should discuss it on the mailman-developers mailing list.
    More reading
    * zope.interfaces and its implementation on mailman.

Message queue based email archiver

Hyperkitty is mailman's official archiver, however it would be a good idea to add an alternative to it based on message queues, either publish-subscribe or a fifo message queue. Possibly an implementation which could be extended to use several different backends (like zeromq, redis etc) of choice would be best. (Support or thoughts on pub/sub support for other system events in mailman would be a plus, though not necessary).

Difficulty Level
Intermediate or advanced depending on the actual proposal of the student.
Mentors
Florian, Aurelian. All students interested in this task should discuss it on the mailman-developers mailing list.
Skills
Knowledge about message/event queues and the pub/sub messaging pattern
More reading
* Look for IArchiver interface in mailman's source code.

Improve and integrate Vote-MM patch

Vote-MM adds voting and RSS support to Mailman 2. This task might involve

  1. Determining whether and how to adapt it to Mailman 3 (HyperKitty already has some voting feature I believe, and probably has RSS). It seems likely that the RSS capability could usefully be added to the IListArchiver interface. I suspect voting belongs in the archiver implementation (eg, HyperKitty), so if HyperKitty already has it, there's nothing to be done. But it's worth a thought.

  2. Updating the patch for Mailman 2. (Optional but requested by users.)
  3. Finding an appropriate way to distribute it (Mailman 2 is basically end-of-life, and a GSoC project should not expect to meet Mark Sapiro's deadline for the final release of Mailman2.1). (If subtask 2 is accepted.)
  4. Designing the RSS feature (and maybe the voting feature) for Mailman 3.
  5. Evaluating libraries for implementing the features (the ones used in the Vote-MM patch may not be available for Python 3, or be end-of-life, etc).
  6. Implementation.
    7. Difficulty level

    Intermediate (requires careful assessment of whether the project is worth doing at all -- subtask 1 needs to be done in the proposal. Research the HyperKitty features first!)

    Mentors
    Stephen
    Skills
    Refactoring (you'll learn as you go along), PyPI, maybe some web skills (RSS)
    More reading

    Ron Guerin's post: https://www.mail-archive.com/mailman-users@python.org/msg66077.html

More ideas

This is a tentative brainstorming list! Before dedicating yourself to any of these, please talk to the mailman-developers list. These are ideas that are potentially GSoC-worthy, but need more research before they could be proposed.

Propose your own idea!

You can also propose your own idea, if you think you have a great one. Make sure to talk about it on Mailman Developers <mailman-developers@python.org> first, though, to make sure that it's suitable for Summer of Code, meets the needs of enough Mailman users, and has at least one mentor interested enough to commit to it for the summer.

Mentor List

The following folks are mentors and administrators for Mailman's 2015 Google Summer of Code effort. Most mentors, even if they list addresses here, probably prefer that you post to Mailman Developers <mailman-developers@python.org> instead of sending private mail. That gives you the broadest exposure to all mentors (improves chance of selection, we are not going to get a slot per mentor!), and probably somebody can respond to you relatively quickly if the "obvious suspect" doesn't.

Remember that participating in mailing lists is a very important part of being a Mailman developer, so consistent public participation on the mailman-developers mailing list will make you more likely to get hired!

  • Abhilash Raj <raj DOT abhilash1 NO AT HARVEST gmail DOT com> (Org admin, maxking on #mailman)

  • Barry Warsaw <barry NO AT HARVEST list DOT org> (barry on #mailman) - MM core co-mentor

  • Florian Fuchs <flo DOT fuchs NO AT HARVEST gmail DOT com> (Org admin, nick: florianf on freenode)

  • Aurelien Bompard <aurelien NO AT HARVEST bompard DOT org> (abompard on #mailman)

  • Terri Oda <terri NO AT HARVEST toybox DOT ca> (terri on #mailman)

  • Stephen J. Turnbull aka Steve <stephen NO AT HARVEST xemacs DOT org> (yaseppochi on #mailman)

  • Sneha Priscilla Makini <24.sneha NO AT HARVEST gmail DOT com> (stylistica on #mailman)

  • Aamir Khan <syst3m.w0rm NO AT HARVEST gmail DOT com> (syst3mw0rm on #mailman)

MailmanWiki: DEV/Google_Summer_of_Code_2015 (last edited 2016-03-24 01:40:26 by stephen@xemacs.org)