Differences between revisions 24 and 26 (spanning 2 versions)
Revision 24 as of 2006-09-27 23:16:00
Size: 5498
Editor: barry
Comment:
Revision 26 as of 2007-01-02 13:52:13
Size: 6015
Editor: barry
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
#pragma page-filename DEV/versions/243 #pragma page-filename DEV/versions/672
Line 21: Line 21:
 * Use the [[http://www.python.org/doc/current/lib/module-optparse.html|optparse]] module throughout instead of getopt.  * Use the [[http://www.python.org/doc/current/lib/module-optparse.html|optparse]] module throughout instead of getopt. ('''Under way''')
Line 24: Line 24:
 * Get rid of the "one list per site with the same name" restriction (cf. [[http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.047.htp|FAQ on virtual hosting]], Hans Ulrich Niedermann's [[http://nix.lauft.net/mailman/|vhost Branch]])  * Get rid of the "one list per site with the same name" restriction (cf. [[http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.047.htp|FAQ on virtual hosting]], Hans Ulrich Niedermann's [[http://nix.lauft.net/mailman/|vhost Branch]]) ('''Done''')
Line 32: Line 32:
 * Get rid of password reminders altogether. Encrypt member passwords and use a password reset feature instead of a reminder.  * Get rid of password reminders altogether. Encrypt member passwords and use a password reset feature instead of a reminder. ('''Under way''')
Line 39: Line 39:
 * Switch to $strings (i.e. {{{string.Template}}} instances) for all i18n substitutions. We should be able to mechanically update all existing translations.  * Switch to $strings (i.e. {{{string.Template}}} instances) for all i18n substitutions. We should be able to mechanically update all existing translations. ('''Under way''')
Line 55: Line 55:
== Operational ==
 * Get rid of pickles-as-database, probably settling on something like [[http://www.sqlite.org|SQLite]] as a default database, but also investigate [[http://www.sqlobject.org|SQLObject]] and [[http://www.sqlalchemy.org|SQLAlchemy]]
== Operational/Performance ==
 * Get rid of pickles-as-database, probably settling on something like [[http://www.sqlite.org|SQLite]] as a default database, but also investigate [[http://www.sqlobject.org|SQLObject]] and [[http://www.sqlalchemy.org|SQLAlchemy]] ('''Under way''')
Line 62: Line 62:
 * For Postfix, switch to [[http://www.postfix.org/VIRTUAL_README.html|separate domains, non-UNIX accounts]] virtual domains and maildir delivery by default. Delivery mechanics for other MTAs may or may not change.
 * Partition queue directories into hashed subdirectories so that all queue files do not live in the same directory (which increases contention on the directory inodes and reduces overall performance).

Mailman 2.2

This is where I am collecting the feature list and other artifacts for Mailman 2.2, which should be a modest, but well-deserved update for Mailman 2.1. It won't be as ambitious as Mailman 3, which is both good and bad, but I hope that it will help us get Mailman 2.2 out in a reasonable amount of time.

So far, none of these should be taken as commitments (unless marked with a Done :-)); manyare resurrected from the old zope.org wiki page), but currently this is the best place to collect the wish list. If you have specific requests, add them as comments to this page and I'll move them up into the main page as they get accepted for Mailman 2.2.

To Do List

Here are things we need to do before MM2.2 can be released.

  • Defend against runaway .bak file restore loops in the Switchboard.

(Potential) feature set

Compatibility and code cleaning

  • Python 2.4 or newer will be required.

  • Switch to email 4.0 (Done)

  • Use Python's standard logging package throughout. (Done)

  • Convert the last of the string exceptions in Errors.py to class exceptions
  • Remove all unnecessary 'future' statements (Done)

  • Remove True/False hackery (Done)

  • Fix unit test suite! (Done)

  • Eliminate the use of types module as much as possible (Done)

  • Use the optparse module throughout instead of getopt. (Under way)

  • Use the subprocess module instead of things like os.popen().

Wart removal

  • Get rid of the "one list per site with the same name" restriction (cf. FAQ on virtual hosting, Hans Ulrich Niedermann's vhost Branch) (Done)

  • Easy configuration of list styles (this might be a MM3 thing).

Web U/I

A web UI improvement project is underway at summer of code: please contribute ideas/thoughts/etc there.

Privacy

  • Add an option to allow verified non-members to post to the list. A verified non-member is someone who posts to the list and responds in the affirmative to a confirmation message. A verified member can post in the future (think Gmane).

  • Rosters should not be public by default.
  • Get rid of password reminders altogether. Encrypt member passwords and use a password reset feature instead of a reminder. (Under way)

Bounce processing

  • Fix the bounce probe problem, and make bounce processing more efficient. Right now, it sucks.

Internationalization

  • Clean up our Unicode story! All strings should be Unicode internally, with conversion at the boundaries of the system. IWBNI we could get rid of all the catches of UnicodeErrors, however we may have to make changes to things like the email library.

  • Switch to a new templating system that lets us share templates across languages, but extract messages for the catalog.
  • Finish the transition to an externally managed translation system

  • Switch to $strings (i.e. string.Template instances) for all i18n substitutions. We should be able to mechanically update all existing translations. (Under way)

  • Language variants (e.g. en_UK)

Spam defenses

Archiving

Note that what we can accomplish here will be based on the availability of an Archiving Champion.

  • Reconsider using a 3rd-party archiver
  • Perhaps URLs to messages should be based on message-ids instead of message numbers so that regenerating archives can't break links. This must include backward compatible links

  • Ditch direct access and vend all archive messages through CGI so that we can do address obfuscation, and message deletion, etc. on the fly.
  • Add RSS feed
  • Allow for admins to remove or edit messages through the web
  • Put archives in the list/mylist directory.

  • Add a search option
  • Make archives look and feel similar to Web UI (whatever it looks like after the Summer of Code project is done)

  • Make archive templatable (at least by changing CSS) so they can match people's existing site look-and-feel

Operational/Performance

  • Get rid of pickles-as-database, probably settling on something like SQLite as a default database, but also investigate SQLObject and SQLAlchemy (Under way)

  • Perhaps also include other database adapters if there are champions to support them.
  • Caps for list member count and outgoing messages per time slice.
  • Better feedback to users who have bounced (e.g. put a link to their last bounced message on their member page).
  • Add a datetime of when the user subscribed to the database, and make this visible.
  • Provide a better list-data export and import mechanism via XML. Possibly allow for data export via the web.

  • For Postfix, switch to separate domains, non-UNIX accounts virtual domains and maildir delivery by default. Delivery mechanics for other MTAs may or may not change.

  • Partition queue directories into hashed subdirectories so that all queue files do not live in the same directory (which increases contention on the directory inodes and reduces overall performance).

Anti-features

These are things we're not going to do in Mailman 2.2

  • Unified user database.



Comments

David Gibbs

In the Spam Defense section

  • Integrate with SpamAssassin (spamd)

Terri Oda

Just for the record, I'm still interested in helping with the web ui redesign, and I may be able to get some bona-fide human factors people from the local university to help give it a serious work-over.

Brad Knowles

Can we go ahead and incorporate Richard Barrett's patches for integrating ht:dig, or do something comparable?  It's a pain for him to have to continue to maintain these patches and update them for every new version of Mailman that is released, yet these are probably one of the most popular patches in the system.  IMO, it does a dis-service to the community to not provide these features out-of-the-box -- people shouldn't have to patch our code in order to get major functionality like this.

Glenn Sieb

I'd like to suggest that, since we have the code for sending out monthly mailings, instead of trashing it, we alter it, so that mailing list owners can have the option of having a monthly mailer--reminders of list rules, how to unsubscribe/propose new members, stuff like that.

This is what YahooGroups/eGroups/etc have been doing since their beginnings, and yes, I understand that Mailman isn't YG/eG/etc, however... This has to be the biggest feature, IMHO, that I've missed in Mailman. Just a box, in the General page, so the owner can put in the text that gets mailed out. Make it a "Do you want a monthly mailing?" option or something--but I think this functionality could win over a lot more people to Mailman's charms.

Anywho.. that's my idea--What do you think?

Best,
--Glenn

Barry Warsaw

Brad, I would definitely like to see something like Richard's ht:dig work integrated in 2.2. It's definitely time we had basic search capabilities in the archives. I think this is something we should coordinate with Ethan (our SoC student) who is working on the archive web pages.

Barry Warsaw

Glenn, you make an interesting point and I don't think this would be too hard to do. However, the current password reminder stuff really isn't suited to be adapted this way, and it's better off gone. I'll keep this in mind though because I do think it would be a fairly easy feature to add.

Glenn Sieb

Barry, that's great! Thanks!

Brad Knowles

For the archive handling stuff, we've got a couple of issues that I think need to be addressed, certainly for 2.2 if not earlier.

First off, I think there's a problem where the date information is corrected, if you use ARCHIVER_CLOBBER_DATE_POLICY=1. In particular, I believe that this is being corrected as part of the archive process, and if you were to manually re-generate the archives any previous messages that came in with bogus dates would now get "re-corrected" to be the local date/time of the manual archive generation, and not the date/time of message reception. We need to move this code up the chain, so that it will correct messages as they come in, before they get written to the archive, and before they get sent out to the recipients.

Secondly, I think that the code should be modified to look for the most recent "Received:" header, and to use that as a first fallback point to correcting an incorrect "Date:" header on the message. This header should have been put on the message by the local MTA on the Mailman server, so it should be mostly the same as using a localtime system call, except that you avoid any additional delays that may have been caused by the message sitting in a queue, etc.... This also helps resolve the above issue where the message needs to be corrected on input, as opposed to being corrected on insertion into the archive.

See FAQ 3.63, if you're interested.

Lindsay Haisley

I'm working on some utilities to integrate the Namazu search engine with Mailman, along the lines of Tom Morrison's very nice instruction posted at http://mail.python.org/pipermail/mailman-users/2004-June/037580.html.  The integration on the Namazu side is pretty straightforward and I've already written a piece that customizes Namazu templates on a per-list basis with information from Mailman's MailList module.  See nmzproc at http://www.fmp.com/namazu

There's only one thing holding up the train on the Mailman end, and that's the fact that the only list name substitution token available in archtoc.html and archtocnombox.html is %(listname)s, which is replaced with the lists real_name, including any capitalization which the list admin may chose to use in the "pretty" name for the list.  The Mailman templates need to be modified to generate a small search form, as per Tom Morrison's instructions, and the action attribute of the form needs to be a full path to a directory containing Namazu components, e.g. action="/mailman/%(listname)s/namazu.cgi".  If Namazu is set up for a list, and the list admin decides he wants to change the real_name for the list to display all in caps, the link to Namazu will break.

One line of code would fix this:

--- HyperArch.py.orig   2006-10-16 20:25:22.000000000 -0500
+++ HyperArch.py        2006-10-16 20:28:35.000000000 -0500
@@ -752,6 +752,7 @@
         listname = mlist.internal_name()
         mbox = os.path.join(mlist.archive_dir()'.mbox', listname'.mbox')
         d = {"listname": mlist.real_name,
+             "internal": mlist.internal_name(),
              "listinfo": mlist.GetScriptURL('listinfo', absolute=1),
              "fullarch": '../%s.mbox/%s.mbox' % (listname, listname),
              "size": sizeof(mbox, mlist.preferred_language),

This would give us a %(internal)s name for the list in the TOC index.html file which would be lower case and independent of administrative changes to the displayed list name.

I can do this easily for myself, but I'd like to put together and release a set of tools to integrate Namazu with Mailman.  Telling people to use alternate Mailman templates, or giving them a patch for existing templates, is one thing.  Telling them to patch the Mailman Archiver code is rather more radical, and if this small change could get the blessing of the Mailman
developers going forward it would make my work on the Namazu integration stuff a lot easier.

Kevin Cole

REJECTION NOTICE TEMPLATE?

This may exist elsewhere and I've just missed how to do it but... I want to have a stock customized rejection notice for messages that are explicitly rejected by a moderator. Something more than "This message was deemed inappropriate by the moderators." (or whatever the one-liner is now).

For example, a list recently set up to reconcile differences between "warring factions" in a community is likely to generate heated debate at its politest times.  We expect the community to cross the line and want to have a stock response when hitting the "reject" button for such messages.  I had assumed that's what the rejection notice box was for, but apparently that is only for messages automatically rejected. 

Bart

Hi,

I have a big client in the Netherlands using Mailman. The question that we have is whether or not the Mailman templates (subscribe to newsletter/Forum) can be changed with different template/design.
excuse me as I am not technical; just inquiring if it is possible and if so; what kind of person can do this job and how fast.

Thank you for your answers in advance.

Bart
info@socofi.nl

Martin Dennett

OK - I don't know how much of this has already been requested, just having signed up, but here's a couple of things I'd like to see:

A "Yahoo" (sorry for swearing) type admin page for approving messages and members. By that I mean columns that show if you wish to Approve/Discard/Reject/Defer all messages and members in one fell swoop instead of having to individually click each entry. Also the ability to ban the address from subscribing and to always carry out the same action for mails from that address.

Message numbering facility in an archive search.

Date & Time on messages in archive search.

A stats facility to show how many messages have been posted by subscribed members since joining.

A variable to be added that would allow me to see who the mail was actually sent to. Useful in times of spam being reported. To be only added as an option though, as I understand that this *can* be done via a configuration amendment, but I don't have command line access.

Hope that's not too much to be asking for.

Rgds

MD

Dave Yost

About that unified user database.

It would be most useful to see a table with a line for each email address and a column for each mailing list. Each cell would contain an 'enabled' checkbox, if they're on the list.

Beyond that, a database with a line per user and a list of their email addresses would be a wonderful enhancement.

We have 5 lists and only 20 people, and even with that small operation, it's a nightmare keeping track.

Thanks

Dave Yost

The membership list would be nicer if it were more like this: http://www.google.com/alerts/manage?hl=en

Look what happens when you click edit.

Thanks

Martin Dennett

Couple of other things:

1)  The ability to be able to delete individual messages from the archive and,

2) The cursor to be already positioned in the password box when the admin page is accessed (very frustrating!)

AdvThanksance

MD

Gabor Kiss

When I set up Namazu as described by Tom Morrison and Lindsay Haisley I needed a set-gid
CGI wrapper like private or rmlist.
This is because I added user authentication to Lindsay's Python script that is a wrapper
on top of namazu.cgi and it has to access config.pck of private lists.

I just patched an existing CGI wrapper in binary (I did not want to recompile the whole stuff):

perl -p -e 's/rmlist/search/g' rmlist > search

Then I renamed Lindsay's modified nmz_wrapper.cgi to search.py. It works great.

I just suggest to add a CGI wrapper called search to the code base that could be a general
hook for any third party search engine.

Luca Venturini

I do not think integration with Spambayesis the right way for a good spam defense. Instead we need a good spamassassin interface. A huge, increasing, number of server run spamassassin as spam filter. IMHO spamassassin integration is a priority.

Terri Oda

Suggestion from some of my users: having individual admin passwords so when an admin leaves, his/her access can be revoked without requiring all other admins to memorize a new password.

Brad Knowles

The proper place for anti-spam integration is the MTA, not Mailman. If you want to improve what Mailman can do as a response to what the anti-spam software has done to the incoming messages, that's great – but that's not the same thing as integrating or providing hooks for an anti-spam solution.

That said, it may also be useful for Mailman to provide hooks to integrate various anti-spam solutions. The standard Pythonic anti-spam solution would be SpamBayes, but I think we should definitely do what we can to support others as well.

Steve Beattie

It'd be nice if there was a way to send multiple "confirm <hash>" commands in a single email. I use a script to reject and (on occasion) accept messages to the lists I moderate; from within my email client I tag all the messages I wish to reject, and then use a macro to invoke my script once on all of those messages. Unfortunately, if I have 200 spam messages over 4 lists to reject (roughly my daily moderation load, or perhaps higher), my script has to send 200 "confirm" messages rather than 4 messages, one per list being moderated.

Thanks. 

David Buntin

For those using virtual hosting, I suggest integrating Hans Ulrich Niedermann's vhost Branch into 2.2. (Thanks for already adding it to 3.0.)

David Buntin

Tobias Conradi

Make pipermail (urls at least) ISO 8601 compliant? http://wiki.list.org/display/DEV/ISO+8601+date

MailmanWiki: DEV/Mailman 2.2 (last edited 2016-04-17 15:39:45 by msapiro)