|
Size: 8552
Comment:
|
Size: 6746
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| #pragma page-filename DEV/versions/786626 | #pragma page-filename DEV/versions/786622 |
| Line 3: | Line 3: |
| This is where I am collecting the feature list and other artifacts for Mailman 3.0 which will be a major upgrade for the project. Many things that people have been wanting for years will be addressed, most notably a unified user database, true virtual domain support, and backing the Mailman data in a real database layer. From a development perspective, we will be adopting a very strict test-driven development model, utilizing modern Python technology and coding styles. The focus will be on providing core Mailman as a library for easy integration into other frameworks and sites, while continuing the tradition of providing a turnkey solution with all the necessary parts, making it even easier to download, install, and go. | |
| Line 5: | Line 4: |
| There is currently no ETA for Mailman 3, but development is moving quickly. Contributors are welcome and encouraged! Discussions will happen on the [[mailto:mailman-developers@python.org|mailman-developers mailing list]]. Anyone can check out and use the [[../MailmanBranches|current development source branches]]. The list below is not a commitment of features (unless marked witha '''Done''' :)). Add your own as comments and I will move as appropriate into the wish list. You might also want to consider looking at the much more modest [[../Mailman 2.2|Mailman 2.2]] branch. | We aim to release Mailman 3.0 on Thursday, April 16, 2015. |
| Line 7: | Line 6: |
| {{{#!wiki caution You'll notice some overlap between this list and the one for Mailman 2.2. That's because this page was originally copied from the MM2.2 wish list, and I haven't yet finished reorganizing things since the [[http://mail.python.org/pipermail/mailman-developers/2007-July/019641.html|new roadmap]] was announced. }}} |
This page is where we are collecting the feature list and other artifacts for Mailman 3.0, which will be a major upgrade for the project. Many things that people have been wanting for years will be addressed, most notably a unified user database, true virtual domain support, and backing the Mailman data in a real database layer. From a development perspective, we will be adopting a very strict test-driven development model, utilizing modern Python technology and coding styles. The focus will be on providing core Mailman functionality through a REST API, architecturally separate components loosing communicating for better system administration and development lifecycles, while continuing the tradition of providing a turnkey solution with all the necessary parts, making it even easier to download, install, and go. |
| Line 11: | Line 8: |
| Contributors are welcome and encouraged! Discussions will happen on the [[mailto:mailman-developers@python.org|mailman-developers mailing list]]. Anyone can check out and use the [[../MailmanBranches|current development source branches]]. The TODO list below is not a commitment of features (unless marked with a '''Done''' ). == Development and status == Mailman 3 is essentially five projects: * [[https://launchpad.net/~mailman-coders/mailman/3.0/ | Mailman Core]] * [[DEV/Postorius Web Interface|Postorius - The Web UI for Mailman]] ([[https://launchpad.net/postorius|code]]) * [[https://launchpad.net/mailman.client | Mailman Client - The REST API Client]] * [[HyperKitty|HyperKitty - The Archiver for Mailman]] ([[https://github.com/hyperkitty|code]]) * [[https://launchpad.net/mailman-bundler | Mailman Bundler - Installer for Mailman Suite including all above projects]] The suite as a whole is currently [[https://mail.python.org/pipermail/mailman-announce/2014-December/000197.html|at 3.0b5]]. [[https://mail.python.org/pipermail/mailman-developers/2015-January/024147.html|We aim]] to get a 3.0 release of the suite out by the end of the [[DEV/PyCon 2015 Sprint|PyCon sprints, April 13-16 2015]]. Mailman 3.0 is most suitable to new installations of Mailman. We do not guarantee that it will work well for migrations and upgrades of Mailman 2.x; we believe that will work, but encourage you to make backups and test your migrations first. It is not yet at full feature parity with Mailman 2.x, but we're working on that for 3.1. Developer resources: * [[http://aosabook.org/en/mailman.html|Architectural overview]] * [[https://launchpad.net/mailman/3.0|Mailman 3 code "series" in Launchpad]] * [[https://bugs.launchpad.net/mailman/+bugs?field.tag=mailman3|bugs affecting the Mailman 3.x series]] * [[https://bugs.launchpad.net/postorius/+bugs?field.tag=mailman3-suite-blocker|bugs blocking the release of Mailman 3]] * [[DEV/A 5 minute guide to get the Mailman web UI running|a 5-minute guide to get Mailman core + Postorius running]] * [[https://mail.python.org/pipermail/mailman-announce/2014-April/000191.html|announcement of beta 1 preview (April 2014)]] * [[https://mail.python.org/pipermail/mailman-announce/2008-April/000104.html|announcement of alpha 1 (April 2008)]] <<Anchor(todo)>> |
|
| Line 12: | Line 36: |
| Here are things we need to do before MM3 can be released. | |
| Line 14: | Line 37: |
| * Defend against runaway .bak file restore loops in the Switchboard. = (Potential) feature set = == Compatibility and code cleaning == * '''Python 2.5 or newer will be required.''' * Switch to email 4 ('''Done''') * Use Python's standard [[http://www.python.org/doc/current/lib/module-logging.html|logging]] package throughout. ('''Done''') * Convert the last of the string exceptions in Errors.py to class exceptions * Remove all unnecessary ''''future'''' statements ('''Done''') * Remove {{{True}}}/{{{False}}} hackery ('''Done''') * Fix [[../TestSuite|unit test suite]]! ('''Done''') * Eliminate the use of {{{types}}} module as much as possible ('''Done''') * Use the [[http://www.python.org/doc/current/lib/module-optparse.html|optparse]] module throughout instead of getopt. ('''Under way''') * Use the [[http://www.python.org/doc/current/lib/module-subprocess.html|subprocess]] module instead of things like {{{os.popen()}}}. == Wart removal == * Get rid of the "one list per site with the same name" restriction (cf. [[http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.047.htp|FAQ on virtual hosting]], Hans Ulrich Niedermann's [[http://nix.lauft.net/mailman/|vhost Branch]]) ('''Done''') * Easy configuration of list styles (this might be a MM3 thing). == Message cleanup == * Strip nodup recipients from both the {{{To}}} and {{{Cc}}} headers, and juggle recipient headers so that the list address is always in the {{{To}}} field. This might address the other major complaint against no-reply-to-munging, recipient proliferation. == Web U/I == A web UI improvement project is underway at [[../Summer of Code|summer of code]]: please contribute ideas/thoughts/etc there. |
== 3.0 == |
| Line 35: | Line 39: |
| == Privacy == * Add an option to allow ''verified'' non-members to post to the list. A verified non-member is someone who posts to the list and responds in the affirmative to a confirmation message. A verified member can post in the future (think [[http://www.gmane.org|Gmane]]). * Rosters should not be public by default. * Get rid of password reminders altogether. Encrypt member passwords and use a password reset feature instead of a reminder. ('''Under way''') == Bounce processing == * Fix the bounce probe problem, and make bounce processing more efficient. Right now, it sucks. == Internationalization == * Clean up our Unicode story! All strings should be Unicode internally, with conversion at the boundaries of the system. IWBNI we could get rid of all the catches of UnicodeErrors, however we may have to make changes to things like the email library. * Switch to a new templating system that lets us share templates across languages, but extract messages for the catalog. * Finish the transition to [[../Managing Translations|an externally managed translation system]] * Switch to $strings (i.e. {{{string.Template}}} instances) for all i18n substitutions. We should be able to mechanically update all existing translations. ('''Under way''') * Language variants (e.g. en_UK) == Spam defenses == * Possibly integrate [[http://www.spambayes.org|Spambayes]] * Possibly provide a standard SpamAssassin handler * Provide some mechanism whereby the MTA can query Mailman to ask whether the email sender is allowed to send email to the list, so that (when the answer is no) the MTA can reject the email at SMTP time. Currently, Mailman does email rejection after the MTA has accepted email, and all the options available for suspected spam (send an email to the forged sender address, send an email to the list owner, throw the email away) are undesirable. The ideal here would be a daemon that can answer SMTP callout queries, but acceptable would be a small python script which took the sender address and list name as arguments. == Archiving == Note that what we can accomplish here will be based on the availability of an Archiving Champion. |
Here are things we need to do before Mailman 3.0 Suite final can be released. We started this list with the leftovers from [[../PyCon Sprint 2013|PyCon Sprint 2013]] and updated it at the [[../PyCon 2015 Sprint|PyCon Sprint 2015]]. |
| Line 54: | Line 41: |
| * Reconsider using a 3rd-party archiver * Perhaps URLs to messages should be based on message-ids instead of message numbers so that regenerating archives can't break links. '''This must include backward compatible links''' * Ditch direct access and vend all archive messages through CGI so that we can do address obfuscation, and message deletion, etc. on the fly (with caching of course, but have to worry about web crawlers). * [[http://lists.w3.org/Archives/Public/www-rdf-interest/2003Feb/0047.html|Add RSS feed]] * Allow for admins to remove or edit messages through the web. * Move archive threads into another list? * Put archives in the list/''mylist'' directory. * Add a search option * Make archives default template look and feel similar to Web UI (whatever it looks like after the [[../Summer of Code|Summer of Code]] project is done) * Make archive templatable (at least by changing CSS) so they can match people's existing site look-and-feel * MUAs usually make URLs clickable. An new Archive could be used when posts are distributed, in the footer, so that each message has a link to the whole thread in the Archive. * Present all messages in a thread at once, and offer plaintext download of the whole thread * Put messages into a database and/or move away from mbox as the canonical storage format. == Operational/Performance/Administration == * Improve admin statistics gather (# of posts sent, # bounced, # of domains accessed, which lists are quiet and which are busy) and make these available via the web interface. * Get rid of pickles-as-database, probably settling on something like [[http://www.sqlite.org|SQLite]] as a default database, but also investigate [[http://www.sqlobject.org|SQLObject]] and [[http://www.sqlalchemy.org|SQLAlchemy]] ('''Under way''') * Perhaps also include other database adapters if there are champions to support them. * Caps for list member count and outgoing messages per time slice. * Better feedback to users who have bounced (e.g. put a link to their last bounced message on their member page). * The 'digest option' needs to be improved to delete 'double messages' * Add a datetime of when the user subscribed to the database, and make this visible. * Provide a better list-data export and import mechanism via XML. ''Possibly'' allow for data export via the web. * For Postfix, switch to [[http://www.postfix.org/VIRTUAL_README.html|separate domains, non-UNIX accounts]] virtual domains and maildir delivery by default. Delivery mechanics for other MTAs may or may not change. * Partition queue directories into hashed subdirectories so that all queue files do not live in the same directory (which increases contention on the directory inodes and reduces overall performance). |
* '''Mailman Suite''' * Update installation HOWTO(s) * Write release notes and announcement * Test new installs of 3.0 suite * '''Core''' * [[https://code.launchpad.net/~barry/mailman/subpolicy|Subscription management workflow/policy - WIP branch]] (see [[https://bugs.launchpad.net/postorius/+bug/1095552|bug 1095552]]) * '''Postorius''' * subscription moderation (see [[https://bugs.launchpad.net/postorius/+bug/1058449|bug 1058449]]) * '''Websites''' * Post HOWTO, release notes & announcement == 3.1 == TODOs for the next point release of 3.x: * Mailman Suite: * Internationalization - there must be a way for the community to contribute translations and for us to integrate them into releases ([[https://bugs.launchpad.net/mailman/+bug/1414154|bug 1414154]]) * Migration scripts for current Mailman 2 installations ([[https://bugs.launchpad.net/mailman/+bug/965532|bug 965532]]) * Test migration from 2 to 3 * Mailman 2 & 3 co-existing * Make it easier to install with Docker * Core * Subscription policy: dealing with permission to unsubscribe * Postorius * Implementation of django-browserid + custom audience checking * Add user settings page (Likely to be completed as part of stylistica's summer of code work; may need temporary fix sooner) * Add ability to remove moderators/owners from a list (see [[https://bugs.launchpad.net/postorius/+bug/1062889|bug 1062889]]) * non-member disposition [Put list of non-members in list settings with allow/deny, etc] (see [[https://bugs.launchpad.net/mailman/+bug/1414149|bug 1414149]]) * Pull list of supported languages for a domain from Mailman core via API (see [[https://bugs.launchpad.net/mailman/+bug/1414298|bug 1414298]]) * Hyperkitty * if a nonmember replies via the web interface, subscribe them as nondelivery member * Websites * Update visual style and information architecture of list.org and the wiki to feel more unified, be more navigable, and present a modern appearance * Convert www site generation from ancient ht2html scripts to something more modern like [[http://getnikola.com/|Nikola]] or [[http://blog.getpelican.com/|Pelican]] (no CMS required!) |
| Line 81: | Line 79: |
== Installation HowTo == Currently the best path for installing the entire Mailman 3 suite at once is Mailman Bundler: [[http://mailman-bundler.readthedocs.org/en/latest/|Mailman Bundler setup instructions on ReadTheDocs]] If you're installing on Ubuntu, see [[DEV/Mailman 3.0/Mailman 3.0 Suite Install on Ubuntu]] for the commands to install the prerequisites. We do eventually hope to have appropriate packages for some popular distros, so if you prefer packages only, please be patient! If you want to try Mailman 3 suite out in Docker, you might want to see [[DEV/Mailman 3.0/Mailman 3.0 Suite Dockerfile]]. I am still looking for someone to help provide nice instructions on how to use this, including connecting port 8000 inside the container to a port on the outside so that the tutorial can end with "and now go to http://localhost:8000/mailman3 to check out the web interface!" |
Mailman 3.0
We aim to release Mailman 3.0 on Thursday, April 16, 2015.
This page is where we are collecting the feature list and other artifacts for Mailman 3.0, which will be a major upgrade for the project. Many things that people have been wanting for years will be addressed, most notably a unified user database, true virtual domain support, and backing the Mailman data in a real database layer. From a development perspective, we will be adopting a very strict test-driven development model, utilizing modern Python technology and coding styles. The focus will be on providing core Mailman functionality through a REST API, architecturally separate components loosing communicating for better system administration and development lifecycles, while continuing the tradition of providing a turnkey solution with all the necessary parts, making it even easier to download, install, and go.
Contributors are welcome and encouraged! Discussions will happen on the mailman-developers mailing list. Anyone can check out and use the current development source branches. The TODO list below is not a commitment of features (unless marked with a Done ).
Development and status
Mailman 3 is essentially five projects:
The suite as a whole is currently at 3.0b5. We aim to get a 3.0 release of the suite out by the end of the PyCon sprints, April 13-16 2015.
Mailman 3.0 is most suitable to new installations of Mailman. We do not guarantee that it will work well for migrations and upgrades of Mailman 2.x; we believe that will work, but encourage you to make backups and test your migrations first. It is not yet at full feature parity with Mailman 2.x, but we're working on that for 3.1.
Developer resources:
To Do List
3.0
Here are things we need to do before Mailman 3.0 Suite final can be released. We started this list with the leftovers from PyCon Sprint 2013 and updated it at the PyCon Sprint 2015.
Mailman Suite
- Update installation HOWTO(s)
- Write release notes and announcement
- Test new installs of 3.0 suite
Core
Postorius
subscription moderation (see bug 1058449)
Websites
Post HOWTO, release notes & announcement
3.1
TODOs for the next point release of 3.x:
- Mailman Suite:
Internationalization - there must be a way for the community to contribute translations and for us to integrate them into releases (bug 1414154)
Migration scripts for current Mailman 2 installations (bug 965532)
- Test migration from 2 to 3
Mailman 2 & 3 co-existing
- Make it easier to install with Docker
- Core
- Subscription policy: dealing with permission to unsubscribe
- Postorius
- Implementation of django-browserid + custom audience checking
- Add user settings page (Likely to be completed as part of stylistica's summer of code work; may need temporary fix sooner)
Add ability to remove moderators/owners from a list (see bug 1062889)
non-member disposition [Put list of non-members in list settings with allow/deny, etc] (see bug 1414149)
Pull list of supported languages for a domain from Mailman core via API (see bug 1414298)
- Hyperkitty
- if a nonmember replies via the web interface, subscribe them as nondelivery member
- Websites
Installation HowTo
Currently the best path for installing the entire Mailman 3 suite at once is Mailman Bundler:
Mailman Bundler setup instructions on ReadTheDocs
If you're installing on Ubuntu, see DEV/Mailman 3.0/Mailman 3.0 Suite Install on Ubuntu for the commands to install the prerequisites.
We do eventually hope to have appropriate packages for some popular distros, so if you prefer packages only, please be patient!
If you want to try Mailman 3 suite out in Docker, you might want to see DEV/Mailman 3.0/Mailman 3.0 Suite Dockerfile. I am still looking for someone to help provide nice instructions on how to use this, including connecting port 8000 inside the container to a port on the outside so that the tutorial can end with "and now go to http://localhost:8000/mailman3 to check out the web interface!"