Mailman - The GNU Mailing List Management System

https://gitlab.com/mailman/mailman/badges/master/build.svg https://readthedocs.org/projects/mailman/badge http://img.shields.io/pypi/v/mailman.svg http://img.shields.io/pypi/dm/mailman.svg

Copyright (C) 1998-2017 by the Free Software Foundation, Inc.

This is GNU Mailman, a mailing list management system distributed under the terms of the GNU General Public License (GPL) version 3 or later. The name of this software is spelled “Mailman” with a leading capital ‘M’ but with a lower case second ‘m’. Any other spelling is incorrect.

Technically speaking, you are reading the documentation for Mailman Core. The full Mailman 3 suite includes a web user interface called Postorius, a web archiver called HyperKitty, and a few other components. If you’re looking for instructions on installing the full suite, read that documentation.

Mailman is written in Python which is available for all platforms that Mailman is supported on, including GNU/Linux and most other Unix-like operating systems (e.g. Solaris, *BSD, MacOSX, etc.). Mailman is not supported on Windows, although web and mail clients on any platform should be able to interact with Mailman just fine.

The Mailman home page is:

and there is a community driven wiki at

For more information on Mailman, see the above web sites, or the documentation provided with this software.

Table of Contents

Mailman - The GNU Mailing List Management System

This is GNU Mailman, a mailing list management system distributed under the terms of the GNU General Public License (GPL) version 3 or later.

Mailman is written in Python, a free object-oriented programming language. Python is available for all platforms that Mailman is supported on, which includes GNU/Linux and most other Unix-like operating systems (e.g. Solaris, *BSD, MacOSX, etc.). Mailman is not supported on Windows, although web and mail clients on any platform should be able to interact with Mailman just fine.

GNU Mailman 3 consists of a suite of programs that work together:

  • Mailman Core; the core delivery engine. This is where you are right now.
  • Postorius; the web user interface for list members and administrators.
  • HyperKitty; the web-based archiver
  • Mailman client; the official Python bindings for talking to the Core’s REST administrative API.

Only the Core is required. You can write or integrate your own web user interface or archiver that speaks to the Core over its REST API. The REST API is a pure HTTP-based API so you don’t have to use Python, or even our official bindings. And you can deploy whatever components you want using whatever mechanisms you want.

But we really like Postorius and HyperKitty and hope you will too!

Spelling

The name of this software is spelled Mailman with a leading capital M but with a lower case second m. Any other spelling is incorrect. Its full name is GNU Mailman but is often referred colloquially as Mailman.

History

Mailman was originally developed by John Viega in the mid-1990s. Subsequent development (through version 1.0b3) was by Ken Manheimer. Further work towards the 1.0 final release was a group effort, with the core contributors being: Barry Warsaw, Ken Manheimer, Scott Cotton, Harald Meland, and John Viega. Since version 1.0 the project has been led by Barry Warsaw with contributions from many; see the ACKNOWLEDGMENTS file for details. Jeremy Hylton helped considerably with the Pipermail code in Mailman 2.0. Mailman 2.1 is primarily maintained by Mark Sapiro, with previous help by Tokio Kikuchi (RIP). Barry Warsaw is the lead developer on Mailman 3. Aurélien Bompard and Florian Fuchs lead development of Postorius and HyperKitty. Abhilash Raj is a core developer contributing to all the bits and maintains the CI infrastructure.

Project details

(GNU Mailman 2.1 is still maintained on Launchpad.)

There are two mailing lists you can use to contact the Mailman 3 developers. The general development mailing list:

and the Mailman 3 users list:

For now, please leave the older mailman-users mailing list for Mailman 2.

Release notes

Mailman 3 is a fully rewritten code base. The developers believe it has sufficient functionality to provide full mailing list services. It should be ready for production use by experienced system developers, but it may not be easy to install or run by novices.

It should also be possible to migrate Mailman 2.1 mailing lists to Mailman 3. Caution, backups, and testing is recommended.

We expect it to be possible to run Mailman 3 and Mailman 2.1 together on the same systems, but you may need to be quite experienced with configuring your mail server and web infrastructure.

Mailman 3 may have bugs.

Mailman 3 is not yet feature complete with Mailman 2.1.

The documentation here describes the Mailman Core in great detail. Postorius, Hyperkitty, and mailman.client are described and developed elsewhere.

More release notes are maintained on the Mailman wiki.

Installing Mailman 3

Copyright (C) 2008-2017 by the Free Software Foundation, Inc.

Requirements

For the Core, Python 3.4 or newer is required. It can either be the default ‘python3’ on your $PATH or it can be accessible via the python3.4, python3.5, or python3.6 binary. If your operating system does not include Python 3, see http://www.python.org for information about downloading installers (where available) and installing it from source (when necessary or preferred). Python 2 is not supported by the Core.

You may need some additional dependencies, which are either available from your OS vendor, or can be downloaded automatically from the Python Cheeseshop.

Documentation

The documentation for Mailman 3 is distributed throughout the sources. The core documentation (such as this file) is found in the src/mailman/docs directory, but much of the documentation is in module-specific places. Online versions of the Mailman 3 Core documentation is available online.

Also helpful might be Mark Sapiro’s documentation on building out the mailman3.org server.

Get the sources

The Mailman 3 source code is version controlled using Git. You can get a local copy by running this command:

$ git clone https://gitlab.com/mailman/mailman.git

or if you have a GitLab account and prefer ssh:

$ git clone git@gitlab.com:mailman/mailman.git

Running Mailman 3

You will need to set up a configuration file to override the defaults and set things up for your environment. Mailman is configured using an “ini”-style configuration system. Usually this means creating a mailman.cfg file and putting it in a standard search location. See the configuration documentation for details.

By default, all runtime files are put under a var directory in the current working directory.

Run the mailman info command to see which configuration file Mailman is using, and where it will put its database file. The first time you run this, Mailman will also create any necessary run-time directories and log files.

Try mailman --help for more details. You can use the commands mailman start to start the runner subprocess daemons, and of course mailman stop to stop them.

Note that you can also run Mailman from one of the virtual environments created by tox, e.g.:

$ tox -e py35-nocov --notest -r
$ .tox/py35-nocov/bin/mailman info

Mailman Shell

This documentation has examples which use the Mailman shell to interact with Mailman. To start the shell type mailman shell in your terminal.

There are some testings functions which need to be imported first before you use them. They can be imported from the modules available in mailman.testing. For example, to use dump_list you first need to import it from the mailman.testing.documentation module.

The shell automatically initializes the Mailman system, loads all the available interfaces, and configures the Zope Component Architecture (ZCA) which is used to access all the software components in Mailman. So for example, if you wanted to get access to the list manager component, you could do:

$ mailman shell
Welcome to the GNU Mailman shell

>>> list_manager = getUtility(IListManager)

Configuring Mailman

Mailman is configured via an “ini”-style configuration file, usually called mailman.cfg. Most of the defaults produce a usable system, but you will almost certainly have to set up a few things before you run Mailman for the first time. You only need to include those settings which you want to change; everything else is inherited.

These file system paths are searched in the following order to find your site’s custom mailman.cfg file. The first file found is used.

  • The file system path specified by the environment variable $MAILMAN_CONFIG_FILE
  • mailman.cfg in the current working directory
  • var/etc/mailman.cfg relative to the current working directory
  • $HOME/.mailman.cfg
  • /etc/mailman.cfg
  • /etc/mailman3/mailman.cfg
  • ../../etc/mailman.cfg relative to the working directory of argv[0]

You can also use the -C option to specify an explicit path, and this always takes precedence. See mailman --help for more details.

You must restart Mailman for any changes to take effect.

Which configuration file is in use?

Mailman itself will tell you which configuration file is being used when you run the mailman info command:

$ mailman info
GNU Mailman 3.1.0b4 (Between The Wheels)
Python 3.5.3 (default, Jan 19 2017, 14:11:04)
[GCC 6.3.0 20170118]
config file: /home/mailman/var/etc/mailman.cfg
db url: sqlite:////home/mailman/var/data/mailman.db
devmode: DISABLED
REST root url: http://localhost:8001/3.1/
REST credentials: restadmin:restpass

The first time you run this command it will create the configuration file and directory using the built-in defaults, so use -C to specify an alternative location. Of course the info subcommand shows you other interesting things about your Mailman instance.

Schemas, templates, and master sections

Mailman’s configuration system is built on top of lazr.config although in general the details aren’t important. Basically there is a schema.cfg file included in the source tree, which defines all the available sections and variables, along with global defaults. There is a built-in base mailman.cfg file also included in the source tree, which further refines the defaults.

Your custom mailman.cfg file, found using the search locations described above, provides the final override for these settings.

The schema.cfg file describes every section, variable, and permissible values, so you should consult this for more details. The schema.cfg file is included verbatim below.

You will notice two types of special sections in the schema.cfg files; those that end with the .template suffix, and others which end in a .master suffix. There are no other special sections.

Templates provide exactly that: a template for other similarly named sections. So for example, you will see a section labeled logging.template which provides some configuration variables and some basic defaults. You will also see a section called logging.bounce which refines the logging.template section by overriding one or more settings.

If you wanted to change the default logging level for the database component in Mailman, say from warn to info, you would add this to your mailman.cfg file:

[logging.database]
level: info

Generally you won’t add new template specialization sections; everything you need is already defined.

You will also see sections labeled with the .master suffix. For the most part you can treat these exactly the same as .template sections; the differences are only relevant for Mailman developers [1]. An example of a .master section is [runner.master] which is used to define the defaults for all the runner processes. This is specialized in the built-in mailman.cfg file, where you’ll see sections like [runner.archive] and [runner.in]. You won’t need to specially the master section yourself, but instead you can override some settings in the individual runner sections.

How do I change a setting?

If you think you want to change something, it can be a little tricky to find exactly the setting you’ll need. The first step is to use the mailman conf command to print all the current variables and their values. With no options, this will print all the hundreds of (sorted!) available settings to standard output. You can narrow this down in two ways. You can print just the values of a particular section:

$ mailman conf -s webservice
[webservice] admin_pass: restpass
[webservice] admin_user: restadmin
[webservice] api_version: 3.1
[webservice] hostname: localhost
[webservice] port: 8001
[webservice] show_tracebacks: yes
[webservice] use_https: no

Let’s say you wanted to change the port the REST API listens on. Just add this to your mailman.cfg file:

[webservice]
port: 8080

You can also search for a specific setting:

$ mailman conf -k prompt
[shell] prompt: >>>

The mailman conf command does not provide documentation about sections or variables. In order to get more information about what a particular variable controls, read the schema.cfg and built-in base mailman.cfg file.

schema.cfg

schema.cfg defines the ini-file schema and contains documentation for every section and configuration variable.

# Copyright (C) 2008-2017 by the Free Software Foundation, Inc.
#
# This file is part of GNU Mailman.
#
# GNU Mailman is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation, either version 3 of the License, or (at your option)
# any later version.
#
# GNU Mailman is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
# more details.
#
# You should have received a copy of the GNU General Public License along with
# GNU Mailman.  If not, see <http://www.gnu.org/licenses/>.

# This is the GNU Mailman configuration schema.  It defines the default
# configuration options for the core system and plugins.  It uses ini-style
# formats under the lazr.config regime to define all system configuration
# options.  See <https://launchpad.net/lazr.config> for details.


[mailman]
# This address is the "site owner" address.  Certain messages which must be
# delivered to a human, but which can't be delivered to a list owner (e.g. a
# bounce from a list owner), will be sent to this address.  It should point to
# a human.
site_owner: changeme@example.com

# This is the local-part of an email address used in the From field whenever a
# message comes from some entity to which there is no natural reply recipient.
# Mailman will append '@' and the host name of the list involved.  This
# address must not bounce and it must not point to a Mailman process.
noreply_address: noreply

# The default language for this server.
default_language: en

# Membership tests for posting purposes are usually performed by looking at a
# set of headers, passing the test if any of their values match a member of
# the list.  Headers are checked in the order given in this variable.  The
# value From_ means to use the envelope sender.  Field names are case
# insensitive.  This is a space separate list of headers.
sender_headers: from from_ reply-to sender

# Mail command processor will ignore mail command lines after designated max.
email_commands_max_lines: 10

# Default length of time a pending request is live before it is evicted from
# the pending database.
pending_request_life: 3d

# How long should files be saved before they are evicted from the cache?
cache_life: 7d

# A callable to run with no arguments early in the initialization process.
# This runs before database initialization.
pre_hook:

# A callable to run with no arguments late in the initialization process.
# This runs after adapters are initialized.
post_hook:

# Which paths.* file system layout to use.
layout: here

# Can MIME filtered messages be preserved by list owners?
filtered_messages_are_preservable: no

# How should text/html parts be converted to text/plain when the mailing list
# is set to convert HTML to plaintext?  This names a command to be called,
# where the substitution variable $filename is filled in by Mailman, and
# contains the path to the temporary file that the command should read from.
# The command should print the converted text to stdout.
html_to_plain_text_command: /usr/bin/lynx -dump $filename

# Specify what characters are allowed in list names.  Characters outside of
# the class [-_.+=!$*{}~0-9a-z] matched case insensitively are never allowed,
# but this specifies a subset as the only allowable characters.  This must be
# a valid character class regexp or the effect on list creation is
# unpredictable.
listname_chars: [-_.0-9a-z]


[shell]
# `mailman shell` (also `withlist`) gives you an interactive prompt that you
# can use to interact with an initialized and configured Mailman system.  Use
# --help for more information.  This section allows you to configure certain
# aspects of this interactive shell.

# Customize the interpreter prompt.
prompt: >>>

# Banner to show on startup.
banner: Welcome to the GNU Mailman shell

# Use IPython as the shell, which must be found on the system.  Valid values
# are `no`, `yes`, and `debug` where the latter is equivalent to `yes` except
# that any import errors will be displayed to stderr.
use_ipython: no

# Set this to allow for command line history if readline is available.  This
# can be as simple as $var_dir/history.py to put the file in the var directory.
history_file:


[paths.master]
# Important directories for Mailman operation.  These are defined here so that
# different layouts can be supported.   For example, a developer layout would
# be different from a FHS layout.  Most paths are based off the var_dir, and
# often just setting that will do the right thing for all the other paths.
# You might also have to set spool_dir though.
#
# Substitutions are allowed, but must be of the form $var where 'var' names a
# configuration variable in the paths.* section.  Substitutions are expanded
# recursively until no more $-variables are present.  Beware of infinite
# expansion loops!
#
# This is the root of the directory structure that Mailman will use to store
# its run-time data.
var_dir: /var/tmp/mailman
# This is where the Mailman queue files directories will be created.
queue_dir: $var_dir/queue
# This is the directory containing the Mailman 'runner' and 'master' commands
# if set to the string '$argv', it will be taken as the directory containing
# the 'mailman' command.
bin_dir: $argv
# All list-specific data.
list_data_dir: $var_dir/lists
# Directory where log files go.
log_dir: $var_dir/logs
# Directory for system-wide locks.
lock_dir: $var_dir/locks
# Directory for system-wide data.
data_dir: $var_dir/data
# Cache files.
cache_dir: $var_dir/cache
# Directory for configuration files and such.
etc_dir: $var_dir/etc
# Directory containing Mailman plugins.
ext_dir: $var_dir/ext
# Directory where the default IMessageStore puts its messages.
messages_dir: $var_dir/messages
# Directory for archive backends to store their messages in.  Archivers should
# create a subdirectory in here to store their files.
archive_dir: $var_dir/archives
# Root directory for site-specific template override files.
template_dir: $var_dir/templates
# There are also a number of paths to specific file locations that can be
# defined.  For these, the directory containing the file must already exist,
# or be one of the directories created by Mailman as per above.
#
# This is where PID file for the master runner is stored.
pid_file: $var_dir/master.pid
# Lock file.
lock_file: $lock_dir/master.lck


[devmode]
# Setting enabled to true enables certain safeguards and other behavior
# changes that make developing Mailman easier.  For example, it forces the
# SMTP RCPT TO recipients to be a test address so that no messages are
# accidentally sent to real addresses.
enabled: no

# Set this to an address to force the SMTP RCPT TO recipents when devmode is
# enabled.  This way messages can't be accidentally sent to real addresses.
recipient:

# This gets set by the testing layers so that the runner subprocesses produce
# predictable dates and times.
testing: no

# Time-outs for starting up various test subprocesses, such as the LMTP and
# REST servers.  This is only used for the test suite, so if you're seeing
# test failures, try increasing the wait time.
wait: 60s


[passwords]
# Where can we find the passlib configuration file?  The path can be either a
# file system path or a Python import path.  If the value starts with python:
# then it is a Python import path, otherwise it is a file system path.  File
# system paths must be absolute since no guarantees are made about the current
# working directory.  Python paths should not include the trailing .cfg, which
# the file must end with.
configuration: python:mailman.config.passlib

# When Mailman generates them, this is the default length of passwords.
password_length: 8


[runner.master]
# Define which runners, and how many of them, to start.

# The full import path to the class for this runner.
class: mailman.core.runner.Runner

# The queue directory path that this runner scans.  This is ignored for
# runners that don't manage a queue directory.
path: $QUEUE_DIR/$name

# The number of parallel runners.  This must be a power of 2.  This is ignored
# for runners that don't manage a queue directory.
instances: 1

# Whether to start this runner or not.
start: yes

# The maximum number of restarts for this runner.  When the runner exits
# because of an error or other unexpected problem, it is automatically
# restarted, until the maximum number of restarts has been reached.
max_restarts: 10

# The sleep interval for the runner.  It wakes up once every interval to
# process the files in its slice of the queue directory.  Some runners may
# ignore this.
sleep_time: 1s


[database]
# The class implementing the IDatabase.
class: mailman.database.sqlite.SQLiteDatabase

# Use this to set the Storm database engine URL.  You generally have one
# primary database connection for all of Mailman.  List data and most rosters
# will store their data in this database, although external rosters may access
# other databases in their own way.  This string supports standard
# 'configuration' substitutions.
url: sqlite:///$DATA_DIR/mailman.db
debug: no


[logging.template]
# This defines various log settings.  The options available are:
#
# - level     -- Overrides the default level; this may be any of the
#                standard Python logging levels, case insensitive.
# - format    -- Overrides the default format string
# - datefmt   -- Overrides the default date format string
# - path      -- Overrides the default logger path.  This may be a relative
#                path name, in which case it is relative to Mailman's LOG_DIR,
#                or it may be an absolute path name.  You cannot change the
#                handler class that will be used.
# - propagate -- Boolean specifying whether to propagate log message from this
#                logger to the root "mailman" logger.  You cannot override
#                settings for the root logger.
#
# In this section, you can define defaults for all loggers, which will be
# prefixed by 'mailman.'.  Use subsections to override settings for specific
# loggers.  The names of the available loggers are:
#
# - archiver        --  All archiver output
# - bounce          --  All bounce processing logs go here
# - config          --  Configuration issues
# - database        --  Database logging (SQLAlchemy and Alembic)
# - debug           --  Only used for development
# - error           --  All exceptions go to this log
# - fromusenet      --  Information related to the Usenet to Mailman gateway
# - http            --  Internal wsgi-based web interface
# - locks           --  Lock state changes
# - mischief        --  Various types of hostile activity
# - runner          --  Runner process start/stops
# - smtp            --  Successful SMTP activity
# - smtp-failure    --  Unsuccessful SMTP activity
# - subscribe       --  Information about leaves/joins
# - vette           --  Message vetting information
format: %(asctime)s (%(process)d) %(message)s
datefmt: %b %d %H:%M:%S %Y
propagate: no
level: info
path: mailman.log

[logging.root]

[logging.archiver]

[logging.bounce]
path: bounce.log

[logging.config]

[logging.database]
level: warn

[logging.debug]
path: debug.log
level: info

[logging.error]

[logging.fromusenet]

[logging.http]

[logging.locks]

[logging.mischief]

[logging.runner]

[logging.smtp]
path: smtp.log

# The smtp logger defines additional options for handling the logging of each
# attempted delivery.  These format strings specify what information is logged
# for every message, every successful delivery, every refused delivery and
# every recipient failure.  To disable a status message, set the value to 'no'
# (without the quotes).
#
# These template strings accept the following set of substitution
# placeholders, if available.
#
# msgid     -- the Message-ID of the message in question
# listname  -- the fully-qualified list name
# sender    -- the sender if available
# recip     -- the recipient address if available, or the number of
#              recipients being delivered to
# size      -- the approximate size of the message in bytes
# seconds   -- the number of seconds the operation took
# refused   -- the number of refused recipients
# smtpcode  -- the SMTP success or failure code
# smtpmsg   -- the SMTP success or failure message

every: $msgid smtp to $listname for $recip recips, completed in $time seconds
success: $msgid post to $listname from $sender, $size bytes
refused: $msgid post to $listname from $sender, $size bytes, $refused failures
failure: $msgid delivery to $recip failed with code $smtpcode, $smtpmsg

[logging.subscribe]

[logging.vette]


[webservice]
# The hostname at which admin web service resources are exposed.
hostname: localhost

# The port at which the admin web service resources are exposed.
port: 8001

# Whether or not requests to the web service are secured through SSL.
use_https: no

# Whether or not to show tracebacks in an HTTP response for a request that
# raised an exception.
show_tracebacks: yes

# The API version number for the current (highest) API.
api_version: 3.1

# The administrative username.
admin_user: restadmin

# The administrative password.
admin_pass: restpass


[language.master]
# Template for language definitions.  The section name must be [language.xx]
# where xx is the 2-character ISO code for the language.

# The English name for the language.
description: English (USA)
# And the default character set for the language.
charset: us-ascii
# Whether the language is enabled or not.
enabled: yes

# Language charsets as imported from Mailman 2.1 defaults
# Ref: http://www.lingoes.net/en/translator/langcode.htm
[language.ar]
description: Arabic
charset: utf-8
enabled: yes

[language.ast]
description: Asturian
charset: iso-8859-1
enabled: yes

[language.ca]
description: Catalan
charset: utf-8
enabled: yes

[language.cs]
description: Czech
charset: iso-8859-2
enabled: yes

[language.da]
description: Danish
charset: iso-8859-1
enabled: yes

[language.de]
description: German
charset: iso-8859-1
enabled: yes

[language.el]
description: Greek
charset: iso-8859-7
enabled: yes

[language.es]
description: Spanish
charset: iso-8859-1
enabled: yes

[language.et]
description: Estonian
charset: iso-8859-15
enabled: yes

[language.eu]
# Basque
description: Euskara
charset: iso-8859-15
enabled: yes

[language.fi]
description: Finnish
charset: iso-8859-1
enabled: yes

[language.fr]
description: French
charset: iso-8859-1
enabled: yes

[language.gl]
description: Galician
charset: utf-8
enabled: yes

[language.he]
description: Hebrew
charset: utf-8
enabled: yes

[language.hr]
description: Croatian
charset: iso-8859-2
enabled: yes

[language.hu]
description: Hungarian
charset: iso-8859-2
enabled: yes

[language.ia]
description: Interlingua
charset: iso-8859-15
enabled: yes

[language.it]
description: Italian
charset: iso-8859-1
enabled: yes

[language.ja]
description: Japanese
charset: euc-jp
enabled: yes

[language.ko]
description: Korean
charset: euc-kr
enabled: yes

[language.lt]
description: Lithuanian
charset: iso-8859-13
enabled: yes

[language.nl]
description: Dutch
charset: iso-8859-1
enabled: yes

[language.no]
description: Norwegian
charset: iso-8859-1
enabled: yes

[language.pl]
description: Polish
charset: iso-8859-2
enabled: yes

[language.pt]
description: Protuguese
charset: iso-8859-1
enabled: yes

[language.pt_BR]
description: Protuguese (Brazil)
charset: iso-8859-1
enabled: yes

[language.ro]
description: Romanian
charset: iso-8859-2
enabled: yes

[language.ru]
description: Russian
charset: koi8-r
enabled: yes

[language.sk]
description: Slovak
charset: utf-8
enabled: yes

[language.sl]
description: Slovenian
charset: iso-8859-2
enabled: yes

[language.sr]
description: Serbian
charset: utf-8
enabled: yes

[language.sv]
description: Swedish
charset: iso-8859-1
enabled: yes

[language.tr]
description: Turkish
charset: iso-8859-9
enabled: yes

[language.uk]
description: Ukrainian
charset: utf-8
enabled: yes

[language.vi]
description: Vietnamese
charset: utf-8
enabled: yes

[language.zh_CN]
description: Chinese
charset: utf-8
enabled: yes

[language.zh_TW]
description: Chinese (Taiwan)
charset: utf-8
enabled: yes


[antispam]
# This section defines basic antispam detection settings.

# This value contains lines which specify RFC 822 headers in the email to
# check for spamminess.  Each line contains a `key: value` pair, where the key
# is the header to check and the value is a Python regular expression to match
# against the header's value.  Multiple checks should be entered as multiline
# value with leading spaces:
#
# header_checks:
#   X-Spam: (yes|maybe)
#   Authentication-Results: mail.example.com; dmarc=(fail|quarantine)
#
# The header value and regular expression are always matched
# case-insensitively.
header_checks:

# The chain to jump to if any of the header patterns matches.  This must be
# the name of an existing chain such as 'discard', 'reject', 'hold', or
# 'accept', otherwise 'hold' will be used.
jump_chain: hold


[mta]
# The class defining the interface to the incoming mail transport agent.
incoming: mailman.mta.postfix.LMTP

# The callable implementing delivery to the outgoing mail transport agent.
# This must accept three arguments, the mailing list, the message, and the
# message metadata dictionary.
outgoing: mailman.mta.deliver.deliver

# How to connect to the outgoing MTA.  If smtp_user and smtp_pass is given,
# then Mailman will attempt to log into the MTA when making a new connection.
smtp_host: localhost
smtp_port: 25
smtp_user:
smtp_pass:

# Where the LMTP server listens for connections.  Use 127.0.0.1 instead of
# localhost for Postfix integration, because Postfix only consults DNS
# (e.g. not /etc/hosts).
lmtp_host: 127.0.0.1
lmtp_port: 8024

# Ceiling on the number of recipients that can be specified in a single SMTP
# transaction.  Set to 0 to submit the entire recipient list in one
# transaction.
max_recipients: 500

# Ceiling on the number of SMTP sessions to perform on a single socket
# connection.  Some MTAs have limits.  Set this to 0 to do as many as we like
# (i.e. your MTA has no limits).  Set this to some number great than 0 and
# Mailman will close the SMTP connection and re-open it after this number of
# consecutive sessions.
max_sessions_per_connection: 0

# Maximum number of simultaneous subthreads that will be used for SMTP
# delivery.  After the recipients list is chunked according to max_recipients,
# each chunk is handed off to the SMTP server by a separate such thread.  If
# your Python interpreter was not built for threads, this feature is disabled.
# You can explicitly disable it in all cases by setting max_delivery_threads
# to 0.
max_delivery_threads: 0

# How long should messages which have delivery failures continue to be
# retried?  After this period of time, a message that has failed recipients
# will be dequeued and those recipients will never receive the message.
delivery_retry_period: 5d

# These variables control the format and frequency of VERP-like delivery for
# better bounce detection.  VERP is Variable Envelope Return Path, defined
# here:
#
# http://cr.yp.to/proto/verp.txt
#
# This involves encoding the address of the recipient as Mailman knows it into
# the envelope sender address (i.e. RFC 5321 MAIL FROM).  Thus, no matter what
# kind of forwarding the recipient has in place, should it eventually bounce,
# we will receive an unambiguous notice of the bouncing address.
#
# However, we're technically only "VERP-like" because we're doing the envelope
# sender encoding in Mailman, not in the MTA.  We do require cooperation from
# the MTA, so you must be sure your MTA can be configured for extended address
# semantics.
#
# The first variable describes how to encode VERP envelopes.  It must contain
# these three string interpolations, which get filled in by Mailman:
#
# $bounces -- the list's -bounces robot address will be set here
# $local   -- the recipient address's local mailbox part will be set here
# $domain  -- the recipient address's domain name will be set here
#
# This example uses the default below.
#
# FQDN list address is: mylist@dom.ain
# Recipient is:         aperson@a.nother.dom
#
# The envelope sender will be mylist-bounces+aperson=a.nother.dom@dom.ain
#
# Note that your MTA /must/ be configured to deliver such an addressed message
# to mylist-bounces!
verp_delimiter: +
verp_format: ${bounces}+${local}=${domain}

# For nicer confirmation emails, use a VERP-like format which encodes the
# confirmation cookie in the reply address.  This lets us put a more user
# friendly Subject: on the message, but requires cooperation from the MTA.
# Format is like verp_format, but with the following substitutions:
#
# $address  -- the list-confirm address
# $cookie   -- the confirmation cookie
verp_confirm_format: $address+$cookie

# This regular expression unambiguously decodes VERP addresses, which will be
# placed in the To: (or other, depending on the MTA) header of the bounce
# message by the bouncing MTA.  Getting this right is critical -- and tricky.
# Learn your Python regular expressions.  It must define exactly three named
# groups, `bounces`, `local` and `domain`, with the same definition as above.
# It will be compiled case-insensitively.
verp_regexp: ^(?P<bounces>[^+]+?)\+(?P<local>[^=]+)=(?P<domain>[^@]+)@.*$

# This is analogous to verp_regexp, but for splitting apart the
# verp_confirm_format.  MUAs have been observed that mung
#
# From: local_part@host
#
# into
#
# To: "local_part" <local_part@host>
#
# when replying, so we skip everything up to '<' if any.
verp_confirm_regexp: ^(.*<)?(?P<addr>[^+]+?)\+(?P<cookie>[^@]+)@.*$

# Set this to 'yes' to enable VERP-like (more user friendly) confirmations.
verp_confirmations: no

# Another good opportunity is when regular delivery is personalized.  Here
# again, we're already incurring the performance hit for addressing each
# individual recipient.  Set this to 'yes' to enable VERPs on all personalized
# regular deliveries (personalized digests aren't supported yet).
verp_personalized_deliveries: no

# And finally, we can VERP normal, non-personalized deliveries.  However,
# because it can be a significant performance hit, we allow you to decide how
# often to VERP regular deliveries.  This is the interval, in number of
# messages, to do a VERP recipient address.  The same variable controls both
# regular and digest deliveries.  Set to 0 to disable occasional VERPs, set to
# 1 to VERP every delivery, or to some number > 1 for only occasional VERPs.
verp_delivery_interval: 0

# VERP format and regexp for probe messages.
verp_probe_format: $bounces+$token@$domain
verp_probe_regexp: ^(?P<bounces>[^+]+?)\+(?P<token>[^@]+)@.*$
# Set this 'yes' to activate VERP probe for disabling by bounce.
verp_probes: no

# This is the maximum number of automatic responses sent to an address because
# of -request messages or posting hold messages.  This limit prevents response
# loops between Mailman and misconfigured remote email robots.  Mailman
# already inhibits automatic replies to any message labeled with a header
# "Precendence: bulk|list|junk".  This is a fallback safety valve so it should
# be set fairly high.  Set to 0 for no limit (probably useful only for
# debugging).
max_autoresponses_per_day: 10

# Some list posts and mail to the -owner address may contain DomainKey or
# DomainKeys Identified Mail (DKIM) signature headers <http://www.dkim.org/>.
# Various list transformations to the message such as adding a list header or
# footer or scrubbing attachments or even reply-to munging can break these
# signatures.  It is generally felt that these signatures have value, even if
# broken and even if the outgoing message is resigned.  However, some sites
# may wish to remove these headers by setting this to 'yes'.
remove_dkim_headers: no

# Where can we find the mail server specific configuration file?  The path can
# be either a file system path or a Python import path.  If the value starts
# with python: then it is a Python import path, otherwise it is a file system
# path.  File system paths must be absolute since no guarantees are made about
# the current working directory.  Python paths should not include the trailing
# .cfg, which the file must end with.
configuration: python:mailman.config.postfix


[bounces]
# How often should the bounce runner process queued detected bounces?
register_bounces_every: 15m


[archiver.master]
# To add new archivers, define a new section based on this one, overriding the
# following values.

# The class implementing the IArchiver interface.
class:

# Set this to 'yes' to enable the archiver.
enable: no

# Additional configuration for the archiver.  The path can be either a file
# system path or a Python import path.  If the value starts with python: then
# it is a Python import path, otherwise it is a file system path.  File system
# paths must be absolute since no guarantees are made about the current
# working directory.  Python paths should not include the trailing .cfg, which
# the file must end with.
configuration: changeme

# When sending the message to the archiver, you have the option of
# "clobbering" the Date: header, specifically to make it more sane.  Some
# archivers can't handle dates that are wildly off from reality.  This does
# not change the Date: header for any other delivery vector except this
# specific archive.
#
# When the original Date header is clobbered, it will always be stored in
# X-Original-Date.  The new Date header will always be set to the date at
# which the messages was received by the Mailman server, in UTC.
#
# Your options here are:
# * never  -- Leaves the original Date header alone.
# * always -- Always override the Date header.
# * maybe  -- Override the Date only if it is outside the clobber_skew period.
clobber_date: maybe
clobber_skew: 1d

[archiver.mhonarc]
# This is the stock MHonArc archiver.
class: mailman.archiving.mhonarc.MHonArc
configuration: python:mailman.config.mhonarc

[archiver.mail_archive]
# This is the stock mail-archive.com archiver.
class: mailman.archiving.mailarchive.MailArchive
configuration: python:mailman.config.mail_archive

[archiver.prototype]
# This is a prototypical sample archiver.
class: mailman.archiving.prototype.Prototype


[styles]
# Python import paths inside which components are searched for which implement
# the IStyle interface.  Use one path per line.
paths:
    mailman.styles

# The default style to apply if nothing else was requested.  The value is the
# name of an existing style.  If no such style exists, no style will be
# applied.
default: legacy-default


[digests]
# Headers which should be kept in both RFC 1153 (plain) and MIME digests.  RFC
# 1153 also specifies these headers in this exact order, so order matters.
# These are space separated and case insensitive.
mime_digest_keep_headers:
    Date From To Cc Subject Message-ID Keywords
    In-Reply-To References Content-Type MIME-Version
    Content-Transfer-Encoding Precedence Reply-To
    Message List-Post

plain_digest_keep_headers:
    Message Date From
    Subject To Cc
    Message-ID Keywords
    Content-Type


[nntp]
# Set these variables if you need to authenticate to your NNTP server for
# Usenet posting or reading.  Leave these blank if no authentication is
# necessary.
user:
password:

# Host and port of the NNTP server to connect to.  Leave these blank to use
# the default localhost:119.
host:
port:

# This controls how headers must be cleansed in order to be accepted by your
# NNTP server.  Some servers like INN reject messages containing prohibited
# headers, or duplicate headers.  The NNTP server may reject the message for
# other reasons, but there's little that can be programmatically done about
# that.
#
# These headers (case ignored) are removed from the original message.  This is
# a whitespace separate list of headers.
remove_headers:
    nntp-posting-host nntp-posting-date x-trace
    x-complaints-to xref date-received posted
    posting-version relay-version received

# These headers are left alone, unless there are duplicates in the original
# message.  Any second and subsequent headers are rewritten to the second
# named header (case preserved).  This is a list of header pairs, one pair per
# line.
rewrite_duplicate_headers:
    To X-Original-To
    CC X-Original-CC
    Content-Transfer-Encoding X-Original-Content-Transfer-Encoding
    MIME-Version X-MIME-Version


[dmarc]
# RFC 7489 - Domain-based Message Authentication, Reporting, and Conformance.
# https://en.wikipedia.org/wiki/DMARC

# Parameters for DMARC DNS lookups.  If you are seeing 'DNSException: Unable
# to query DMARC policy ...' entries in your error log, you may need to adjust
# these.
#
# The time to wait for a response from a name server before timeout.
resolver_timeout: 3s
# The total time to spend trying to get an answer to the DNS question.
resolver_lifetime: 5s

# A URL from which to retrieve the data for the algorithm that computes
# Organizational Domains for DMARC policy lookup purposes.  This can be
# anything handled by the Python urllib.request.urlopen function.  See
# https://publicsuffix.org/list/ for info.
org_domain_data_url: https://publicsuffix.org/list/public_suffix_list.dat

# How long should the local suffix list be used before it's considered out of
# date.  After this amount of time a new list will be downloaded, but if it
# can't be accessed, old data will still be used.
cache_lifetime: 7d

mailman.cfg

Configuration settings provided in the built-in base mailman.cfg file overrides those provided in schema.cfg.

# Copyright (C) 2008-2017 by the Free Software Foundation, Inc.
#
# This file is part of GNU Mailman.
#
# GNU Mailman is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free
# Software Foundation, either version 3 of the License, or (at your option)
# any later version.
#
# GNU Mailman is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
# more details.
#
# You should have received a copy of the GNU General Public License along with
# GNU Mailman.  If not, see <http://www.gnu.org/licenses/>.

# This is the absolute bare minimum base configuration file.  User supplied
# configurations are pushed onto this.

[paths.local]
# Directories as specified in schema.cfg, putting most stuff in
# /var/tmp/mailman

[paths.dev]
# Convenient development layout where everything is put in a directory above
# where the mailman.cfg file lives.
var_dir: $cfg_file/../..

[paths.here]
# Layout where the var directory is put in the current working directory.
var_dir: $cwd/var

[paths.fhs]
# Filesystem Hiearchy Standard 2.3
# http://www.pathname.com/fhs/pub/fhs-2.3.html
bin_dir: /sbin
var_dir: /var/lib/mailman
queue_dir: /var/spool/mailman
log_dir: /var/log/mailman
lock_dir: /var/lock/mailman
etc_dir: /etc
ext_dir: /etc/mailman.d
pid_file: /var/run/mailman/master.pid

[language.en]

[runner.archive]
class: mailman.runners.archive.ArchiveRunner

[runner.bad]
class: mailman.runners.fake.BadRunner
# The bad runner is just a placeholder for its switchboard.
start: no

[runner.bounces]
class: mailman.runners.bounce.BounceRunner

[runner.command]
class: mailman.runners.command.CommandRunner

[runner.in]
class: mailman.runners.incoming.IncomingRunner

[runner.lmtp]
class: mailman.runners.lmtp.LMTPRunner
path:

[runner.nntp]
class: mailman.runners.nntp.NNTPRunner

[runner.out]
class: mailman.runners.outgoing.OutgoingRunner

[runner.pipeline]
class: mailman.runners.pipeline.PipelineRunner

[runner.rest]
class: mailman.runners.rest.RESTRunner
path:

[runner.retry]
class: mailman.runners.retry.RetryRunner
sleep_time: 15m

[runner.shunt]
class: mailman.runners.fake.ShuntRunner
# The shunt runner is just a placeholder for its switchboard.
start: no

[runner.virgin]
class: mailman.runners.virgin.VirginRunner

[runner.digest]
class: mailman.runners.digest.DigestRunner
[1]The technical differences are described in the lazr.config package, upon which Mailman’s configuration system is based.

Setting up your database

Mailman uses the SQLAlchemy ORM to provide persistence of data in a relational database. By default, Mailman uses Python’s built-in SQLite3 database, however, SQLAlchemy is compatible with PostgreSQL and MySQL, among possibly others.

Currently, Mailman is known to work with the SQLite3, PostgreSQL, and MySQL databases. (Volunteers to port it to other databases are welcome!). If you want to use SQLite3, you generally don’t need to change anything, but if you want Mailman to use PostgreSQL or MySQL, you’ll need to set those up first, and then change a configuration variable in your /etc/mailman.cfg file.

Two configuration variables control which database Mailman uses. The first names the class implementing the database interface. The second names the URL for connecting to the database. Both variables live in the [database] section of the configuration file.

SQLite3

As mentioned, if you want to use SQLite3 in the default configuration, you generally don’t need to change anything. However, if you want to change where the SQLite3 database is stored, you can change the url variable in the [database] section. By default, the database is stored in the data directory in the mailman.db file. Here’s how to tell Mailman to store its database in /var/lib/mailman/sqlite.db file:

[database]
url: sqlite:////var/lib/mailman/sqlite.db

PostgreSQL

First, you need to configure PostgreSQL itself. This Ubuntu article may help. Let’s say you create the mailman database in PostgreSQL via:

$ sudo -u postgres createdb -O $USER mailman

You would also need the Python driver psycopg2 for PostgreSQL:

$ pip install psycopg2

You would then need to set both the class and url variables in mailman.cfg like so:

[database]
class: mailman.database.postgresql.PostgreSQLDatabase
url: postgres://myuser:mypassword@mypghost/mailman

If you have any problems, you may need to delete the database and re-create it:

$ sudo -u postgres dropdb mailman
$ sudo -u postgres createdb -O myuser mailman

Many thanks to Stephen A. Goss for his contribution of PostgreSQL support.

MySQL

First you need to configure MySQL itself. Lets say you create the mailman database in MySQL via:

mysql> CREATE DATABASE mailman;

You would also need the Python driver pymysql for MySQL.:

$ pip install pymysql

You would then need to set both the class and url variables in mailman.cfg like so:

[database]
class: mailman.database.mysql.MySQLDatabase
url: mysql+pymysql://myuser:mypassword@mymysqlhost/mailman?charset=utf8&use_unicode=1

The last part of the url specifies the charset that client expects from the server and to use Unicode via the flag use_unicode. You can find more about these options on the SQLAlchemy’s MySQL page.

If you have any problems, you may need to delete the database and re-create it:

mysql> DROP DATABASE mailman;
mysql> CREATE DATABASE mailman;

Database Migrations

Mailman uses Alembic to manage database migrations. Let’s say you change something in the models, what steps are needed to reflect that change in the database schema? You need to create and enter a virtual environment, install Mailman into that, and then run the alembic command. For example:

$ python3 -m venv /tmp/mm3
$ source /tmp/mm3/bin/activate
$ python setup.py develop
$ mailman info
$ alembic -c src/mailman/config/alembic.cfg revision --autogenerate -m
  "<migration_name>"
$ deactivate

This would create a new migration which would be applied to the database automatically on the next run of Mailman.

People upgrading Mailman from previous versions need not do anything manually, as soon as a new migration is added in the sources, it will be automatically reflected in the schema on first-run post-update.

Note: When auto-generating migrations using Alembic, be sure to check the created migration before adding it to the version control. You will have to manually change some of the special data types defined in mailman.database.types. For example, mailman.database.types.Enum() needs to be changed to sa.Integer(), as the Enum type stores just the integer in the database. A more complex migration would be needed for UUID depending upon the database layer to be used.

Hooking up your mail server

Mailman needs to communicate with your MTA (mail transport agent or mail server, the software which handles sending mail across the Internet), both to accept incoming mail and to deliver outgoing mail. Mailman itself never delivers messages to the end user. It sends them to its immediate upstream MTA, which delivers them. In the same way, Mailman never receives mail directly. Mail from outside always comes via the MTA.

Mailman accepts incoming messages from the MTA using the Local Mail Transfer Protocol (LMTP) interface. LMTP is much more efficient than spawning a process just to do the delivery. Most open source MTAs support LMTP for local delivery. If yours doesn’t, and you need to use a different interface, please ask on the mailing list or on IRC.

Mailman passes all outgoing messages to the MTA using the Simple Mail Transfer Protocol (SMTP).

Cooperation between Mailman and the MTA requires some configuration of both. MTA configuration differs for each of the available MTAs, and there is a section for each one. Instructions for Postfix and Exim (v4) are given below. We would really appreciate a contribution of a configuration for Sendmail, and welcome information about other popular open source mail servers.

Configuring Mailman to communicate with the MTA is straightforward, and basically the same for all MTAs. Here are the default settings; if you need to change them, edit your mailman.cfg file:

[mta]
incoming: mailman.mta.postfix.LMTP
outgoing: mailman.mta.deliver.deliver
lmtp_host: 127.0.0.1
lmtp_port: 8024
smtp_host: localhost
smtp_port: 25
configuration: python:mailman.config.postfix

This configuration is for a system where Mailman and the MTA are on the same host.

Note that the modules that configure the communication protocol (especially incoming) are full-fledged Python modules, and may use these configuration parameters to automatically configure the MTA to recognize the list addresses and other attributes of the communication channel. This is why some constraints on the format of attributes arise (e.g., lmtp_host), even though Mailman itself has no problem with them.

It is possible (although not documented here) to completely replace or override the default mechanisms to handle both incoming and outgoing mail. Mailman is highly customizable here!

The incoming and outgoing parameters identify the Python objects used to communicate with the MTA. The python: scheme indicates that the paths should be a dotted Python module specification. The deliver module used in outgoing should be satisfactory for most MTAs. The postfix module in incoming is specific to the Postfix MTA. See the section for your MTA below for details on these parameters.

lmtp_host and lmtp_port are parameters which are used by Mailman, but also will be passed to the MTA to identify the Mailman host. The “same host” case is special; some MTAs (including Postfix) do not recognize “localhost”, and need the numerical IP address. If they are on different hosts, lmtp_host should be set to the domain name or IP address of the Mailman host. lmtp_port is fairly arbitrary (there is no standard port for LMTP). Use any port convenient for your site. “8024” is as good as any, unless another service is using it.

smtp_host and smtp_port are parameters used to identify the MTA to Mailman. If the MTA and Mailman are on separate hosts, smtp_host should be set to the domain name or IP address of the MTA host. smtp_port will almost always be 25, which is the standard port for SMTP. (Some special site configurations set it to a different port. If you need this, you probably already know that, know why, and what to do, too!)

Mailman also provides many other configuration variables that you can use to tweak performance for your operating environment. See the src/mailman/config/schema.cfg file for details.

Postfix

Postfix is an open source mail server by Wietse Venema.

Mailman settings

You need to tell Mailman that you are using the Postfix mail server. In your mailman.cfg file, add the following section:

[mta]
incoming: mailman.mta.postfix.LMTP
outgoing: mailman.mta.deliver.deliver
lmtp_host: mail.example.com
lmtp_port: 8024
smtp_host: mail.example.com
smtp_port: 25

Some of these settings are already the default, so take a look at Mailman’s src/mailman/config/schema.cfg file for details. You’ll need to change the lmtp_host and smtp_host to the appropriate host names of course. Generally, Postfix will listen for incoming SMTP connections on port 25. Postfix will deliver via LMTP over port 24 by default, however if you are not running Mailman as root, you’ll need to change this to a higher port number, as shown above.

Basic Postfix connections

There are several ways to hook Postfix up to Mailman, so here are the simplest instructions. The following settings should be added to Postfix’s main.cf file.

Mailman supports a technique called Variable Envelope Return Path (VERP) to disambiguate and accurately record bounces. By default Mailman’s VERP delimiter is the + sign, so adding this setting allows Postfix to properly handle Mailman’s VERP’d messages:

# Support the default VERP delimiter.
recipient_delimiter = +

In older versions of Postfix, unknown local recipients generated a temporary failure. It’s much better (and the default in newer Postfix releases) to treat them as permanent failures. You can add this to your main.cf file if needed (use the postconf command to check the defaults):

unknown_local_recipient_reject_code = 550

While generally not necessary if you set recipient_delimiter as described above, it’s better for Postfix to not treat owner- and -request addresses specially:

owner_request_special = no
Transport maps

By default, Mailman works well with Postfix transport maps as a way to deliver incoming messages to Mailman’s LMTP server. Mailman will automatically write the correct transport map when its mailman aliases command is run, or whenever a mailing list is created or removed via other commands. Mailman supports two type of transport map tables for Postfix, namely hash and regexp. Tables using hash are processed by postmap command. To use this format, you should have postmap command available on the host running Mailman. It is also the default one of the two. To connect Postfix to Mailman’s LMTP server, add the following to Postfix’s main.cf file:

transport_maps =
    hash:/path-to-mailman/var/data/postfix_lmtp
local_recipient_maps =
    hash:/path-to-mailman/var/data/postfix_lmtp
relay_domains =
    hash:/path-to-mailman/var/data/postfix_domains

where path-to-mailman is replaced with the actual path that you’re running Mailman from. Setting local_recipient_maps as well as transport_maps allows Postfix to properly reject all messages destined for non-existent local users. Setting relay_domains means Postfix will start to accept mail for newly added domains even if they are not part of mydestination.

Note that if you are not using virtual domains, then relay_domains isn’t strictly needed (but it is harmless). All you need to do in this scenario is to make sure that Postfix accepts mail for your one domain, normally by including it in mydestination.

Regular Expression tables remove the additional dependency of having postmap command available to Mailman. If you want to use regexp or Regular Expression tables, then add the following to Postfix’s main.cf file:

transport_maps =
    regexp:/path-to-mailman/var/data/postfix_lmtp
local_recipient_maps =
    regexp:/path-to-mailman/var/data/postfix_lmtp
relay_domains =
    regexp:/path-to-mailman/var/data/postfix_domains

You will also have to instruct Mailman to generate regexp tables instead of hash tables by adding the following configuration to mailman.cfg:

[mta]
incoming: mailman.mta.postfix.LMTP
outgoing: mailman.mta.deliver.deliver
lmtp_host: mail.example.com
lmtp_port: 8024
smtp_host: mail.example.com
smtp_port: 25
configuration: /path/to/postfix-mailman.cfg

Also you will have to create another configuration file called as postfix-mailman.cfg and add its path to the configuration parameter above. The postfix-mailman.cfg would look like this:

[postfix]
transport_file_type: regex
Postfix documentation

For more information regarding how to configure Postfix, please see the Postfix documentation at:

Exim

Exim 4 is an MTA maintained by the University of Cambridge and distributed by most open source OS distributions.

Mailman settings

Add or edit a stanza like this in mailman.cfg:

[mta]
# For all Exim4 installations.
incoming: mailman.mta.exim4.LMTP
outgoing: mailman.mta.deliver.deliver
# Typical single host with MTA and Mailman configuration.
# Adjust to your system's configuration.
# Exim happily works with the "localhost" alias rather than IP address.
lmtp_host: localhost
smtp_host: localhost
# Mailman should not be run as root.
# Use any convenient port > 1024.  8024 is a convention, but can be
# changed if there is a conflict with other software using that port.
lmtp_port: 8024
# smtp_port rarely needs to be set.
smtp_port: 25
# Exim4-specific configuration parameter defaults.  Currently empty.
configuration: python:mailman.config.exim4

For further information about these settings, see mailman/config/schema.cfg.

Exim4 configuration

The configuration presented below is mostly boilerplate that allows Exim to automatically discover your list addresses, and route both posts and administrative messages to the right Mailman services. For this reason, the mailman.mta.exim4 module ends up with all methods being no-ops.

This configuration is field-tested in a Debian “conf.d”-style Exim installation, with multiple configuration files that are assembled by a Debian-specific script. If your Exim v4 installation is structured differently, ignore the comments indicating location in the Debian installation.

# /etc/exim4/conf.d/main/25_mm3_macros
# The colon-separated list of domains served by Mailman.
domainlist mm_domains=list.example.net

MM3_LMTP_PORT=8024

# MM3_HOME must be set to mailman's var directory, wherever it is
# according to your installation.
MM3_HOME=/opt/mailman/var
MM3_UID=list
MM3_GID=list

################################################################
# The configuration below is boilerplate:
# you should not need to change it.

# The path to the list receipt (used as the required file when
# matching list addresses)
MM3_LISTCHK=MM3_HOME/lists/${local_part}.${domain}

# /etc/exim4/conf.d/router/455_mm3_router
mailman3_router:
  driver = accept
  domains = +mm_domains
  require_files = MM3_LISTCHK
  local_part_suffix_optional
  local_part_suffix = \
     -bounces   : -bounces+* : \
     -confirm   : -confirm+* : \
     -join      : -leave     : \
     -owner     : -request   : \
     -subscribe : -unsubscribe
  transport = mailman3_transport

# /etc/exim4/conf.d/transport/55_mm3_transport
mailman3_transport:
  driver = smtp
  protocol = lmtp
  allow_localhost
  hosts = localhost
  port = MM3_LMTP_PORT
  rcpt_include_affixes = true
Troubleshooting

The most likely causes of failure to deliver to Mailman are typos in the configuration, and errors in the MM3_HOME macro or the mm_domains list. Mismatches in the LMTP port could be a cause. Finally, Exim’s router configuration is order-sensitive. Especially if you are being tricky and supporting Mailman 2 and Mailman 3 at the same time, you could have one shadow the other.

Exim 4 documentation

There is copious documentation for Exim. The parts most relevant to configuring communication with Mailman 3 are the chapters on the accept router and the LMTP transport. Unless you are already familiar with Exim configuration, you probably want to start with the chapter on how Exim receives and delivers mail.

qmail

qmail is a MTA written by djb and, though old and not updated, still bulletproof and occassionally in use.

Mailman settings

Mostly defaults in mailman.cfg:

[mta]
# NullMTA is just implementing the interface and thus satisfying Mailman
# without doing anything fancy
incoming: mailman.mta.null.NullMTA
# Mailman should not be run as root.
# Use any convenient port > 1024.  8024 is a convention, but can be
# changed if there is a conflict with other software using that port.
lmtp_port: 8024

This will listen on localhost:8024 with LMTP and deliver outgoing messages to localhost:25. See mailman/config/schema.cfg for more information on these settings.

qmail configuration

It is assumed that qmail is configured to use the .qmail* files in a user’s home directory, however the instructions should easily be adaptable to other qmail configurations. However, it is required that Mailman has a (sub)domain respectively a namespace on its own. A helper script called qmail-lmtp is needed and can be found in the contrib/ directory of the Mailman source tree and assumed to be on $PATH here.

As qmail puts every namespace in the address, we have to filter it out again. If your main domain is example.com and you assign lists.example.com to the user mailman, qmail will give you the destination address mailman-spam@lists.example.com while it should actually be spam@lists.example.com. The second argument to qmail-lmtp defines how many parts (separated by dashes) to filter out. The first argument specifies the LMTP port of Mailman. Long story short, as user mailman:

% chmod +t "$HOME"
% echo '|qmail-lmtp 1 8042' > .qmail # put appropriate values here
% ln -sf .qmail .qmail-default
% chmod -t "$HOME"

Sendmail

The core Mailman developers generally do not use Sendmail, so experience is limited. Any and all contributions are welcome! The follow information from a post by Gary Algier <gaa@ulticom.com> may be useful as a starting point, although it describes Mailman 2:

I have it working fine. I recently replaced a very old implementation of sendmail and Mailman 2 on Solaris with a new one on CentOS 6. When I did so, I used the POSTFIX_ALIAS_CMD mechanism to automatically process the aliases. See:

https://mail.python.org/pipermail/mailman-users/2004-June/037518.html

In mm_cfg.py:

MTA='Postfix'
POSTFIX_ALIAS_CMD = '/usr/bin/sudo /etc/mail/import-mailman-aliases'

/etc/mail/import-mailman-aliases contains:

#! /bin/sh
/bin/cp /etc/mailman/aliases /etc/mail/mailman.aliases
/usr/bin/newaliases

In /etc/sudoers.d/mailman:

Cmnd_Alias IMPORT_MAILMAN_ALIASES = /etc/mail/import-mailman-aliases
apache ALL= NOPASSWD: IMPORT_MAILMAN_ALIASES
mailman ALL= NOPASSWD: IMPORT_MAILMAN_ALIASES
Defaults!IMPORT_MAILMAN_ALIASES !requiretty

In the sendmail.mc file I changed:

define(`ALIAS_FILE', `/etc/aliases')dnl

to:

define(`ALIAS_FILE', `/etc/aliases,/etc/mail/mailman.aliases')dnl

so that the Mailman aliases would be in a separate file.

The main issue here is that Mailman 2 expects to receive messages from the MTA via pipes, whereas Mailman 3 uses LMTP exclusively. Recent Sendmail does support LMTP, so it’s a matter of configuring a stock Sendmail. But rather than using aliases, it needs to be configured to relay to the LMTP port of Mailman.

Set up Postorius (web interface)

Postorius is Mailman 3’s Django-based web interface. You can run Mailman 3 without it, but most people prefer to use the web interface for changing list settings, viewing information about available lists, and subscribing or unsubscribing.

To set up Postorius, please see the Postorius documentation.

Set up HyperKitty (archiver)

The HyperKitty application aims at providing an interface to visualize and explore Mailman archives. This is a Django project.

To set up HyperKitty, please see the Hyperkitty documentation.

Contributing to Mailman 3

Copyright (C) 2008-2017 by the Free Software Foundation, Inc.

How to contribute

We accept merge requests and bug reports on GitLab. We prefer if every merge request is linked to a bug report, because we can more easily manage the priority of bug reports. For more substantial contributions, we may ask you to sign a copyright assignment to the Free Software Foundation, the owner of the GNU Mailman copyright. If you’d like to jump start your copyright assignment, please contact the GNU Mailman steering committee.

Please read the GNU Mailman Coding Style Guide for required coding style guidelines.

Contact Us

Contributions of code, problem reports, and feature requests are welcome. Please submit bug reports on the Mailman bug tracker at https://gitlab.com/mailman/mailman/issues (you need to have a login on GitLab to do so). You can also send email to the mailman-developers@python.org mailing list, or ask on IRC channel #mailman on Freenode.

Get the sources

The Mailman 3 source code is version controlled using Git. You can get a local copy by running this command:

$ git clone https://gitlab.com/mailman/mailman.git

or if you have a GitLab account and prefer ssh:

$ git clone git@gitlab.com:mailman/mailman.git

Testing Mailman 3

To run the Mailman test suite, just use the tox command:

$ tox

tox creates a virtual environment (virtualenv) for you, installs all the dependencies into that virtualenv, and runs the test suite from that virtualenv. By default it does not use the –system-site-packages so it downloads everything from the Python Cheeseshop.

A bare tox command will try to run several test suites, which might take a long time, and/or require versions of Python or other components you might not have installed. You can run tox -l to list the test suite environments available. Very often, when you want to run the full test suite in the quickest manner with components that should be available everywhere, run one of these command, depending on which version of Python 3 you have:

$ tox -e py36-nocov
$ tox -e py35-nocov
$ tox -e py34-nocov

You can run individual tests in any given environment by providing additional positional arguments. For example, to run only the tests that match a specific pattern:

$ tox -e py35-nocov -- -P user

You can see all the other arguments supported by the test suite by running:

$ tox -e py35-nocov -- --help

You also have access to the virtual environments created by tox, and you can use this run the virtual environment’s Python executable, or run the mailman command locally, e.g.:

$ .tox/py35-nocov/bin/python
$ .tox/py35-nocov/bin/mailman --help

If you want to set up the virtual environment without running the full test suite, you can do this:

$ tox -e py35-nocov --notest -r

Testing with PostgreSQL and MySQL

By default, the test suite runs with the built-in SQLite database engine. If you want to run the full test suite against the PostgreSQL or MySQL databases, set the database up as described in Setting up your database.

For PostgreSQL, then create a postgres.cfg file any where you want. This postgres.cfg file will contain the [database] section for PostgreSQL, e.g.:

[database]
class: mailman.database.postgresql.PostgreSQLDatabase
url: postgres://myuser:mypassword@mypghost/mailman

Then run the test suite like so:

$ MAILMAN_EXTRA_TESTING_CFG=/path/to/postgres.cfg tox -e py35-pg

You can combine these ways to invoke Mailman, so if you want to run an individual test against PostgreSQL, you could do:

$ MAILMAN_EXTRA_TESTING_CFG=/path/to/postgres.cfg tox -e py35-pg -- -P user

Note that the path specified in MAILMAN_EXTRA_TESTING_CFG must be an absolute path or some tests will fail.

Building for development

To build Mailman for development purposes, you can create a virtual environment outside of tox. You need to have the virtualenv program installed, or you can use Python 3’s built-in pyvenv command.

First, create a virtual environment (venv). The directory you install the venv into is up to you, but for purposes of this document, we’ll install it into /tmp/mm3:

$ python3 -m venv /tmp/mm3

Now, activate the virtual environment and set it up for development:

% source /tmp/mm3/bin/activate
% python setup.py develop

Sit back and have some Kombucha while you wait for everything to download and install.

Building the documentation

To build the documentation, you need some additional dependencies. The only one you probably need from your OS vendor is graphiz. E.g. On Debian or Ubuntu, you can do:

$ sudo apt install graphiz

All other dependencies should be automatically installed as needed. Build the documentation by running:

$ tox -e docs

Then visit:

build/sphinx/html/index.html

Mailman Shell

This documentation has examples which use the Mailman shell to interact with Mailman. To start the shell type mailman shell in your terminal.

There are some testings functions which need to be imported first before you use them. They can be imported from the modules available in mailman.testing. For example, to use dump_list you first need to import it from the mailman.testing.documentation module.

The shell automatically initializes the Mailman system, loads all the available interfaces, and configures the Zope Component Architecture (ZCA) which is used to access all the software components in Mailman. So for example, if you wanted to get access to the list manager component, you could do:

$ mailman shell
Welcome to the GNU Mailman shell

>>> list_manager = getUtility(IListManager)

GNU Mailman Coding Style Guide

Copyright (C) 2002-2017 Barry A. Warsaw

Python coding style guide for GNU Mailman Core

This document contains a style guide for Python programming, as used in GNU Mailman Core. PEP 8 is the basis for this style guide so its recommendations should be followed except for the differences outlined here. Core is a Python 3 application, so this document assumes the use of Python 3.

Much of the style guide is enforced by the command tox -e qa.

  • When creating new modules, start with the GNU Mailman Python template as an example.

  • Public module-global names should be exported in the __all__ but use the @public decorator from the public package to do this for all classes and functions.

  • Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants.

    Imports should be grouped, with the order being:

    1. non-from imports, grouped from shortest module name to longest module name, with ties being broken by alphabetical order.
    2. from-imports grouped alphabetically.

    Put a single blank line between the non-from import and the from-imports.

  • Right hanging comments are discouraged, in favor of preceding comments. E.g. bad:

    foo = baz(bar)  # This has a side-effect of fooing the bar.

    Good:

    # This has a side-effect of fooing the bar.
foo = blarzigop(bar)

    Comments should always be complete sentences, with proper capitalization and full stops (periods) at the end. We use two spaces after periods.

  • Put two blank lines between any top level construct or block of code (e.g. after import blocks). Put only one blank line between methods in a class. No blank lines between the class definition and the first method in the class. No blank lines between a class/method and its docstrings.

  • Try to minimize the vertical whitespace in a class or function. If you’re inclined to separate stanzas of code for readability, consider putting a comment in describing what the next stanza’s purpose is. Don’t put useless or obvious comments in just to avoid vertical whitespace though.

  • Unless internal quote characters would mess things up, the general rule is that single quotes should be used for short strings and double quotes for triple-quoted multi-line strings and docstrings. E.g.:

    foo = 'a foo thing'
warn = "Don't mess things up"
notice = """Our three chief weapons are:
         - surprise
         - deception
         - an almost fanatical devotion to the pope
         """

  • Write docstrings for modules, functions, classes, and methods. Docstrings can be omitted for special methods (e.g. __init__() or __str__()) where the meaning is obvious.

  • PEP 257 describes good docstrings conventions. Note that most importantly, the """ that ends a multiline docstring should be on a line by itself, e.g.:

    """Return a foobang

Optional plotz says to frobnicate the bizbaz first.
"""

  • For one liner docstrings, keep the closing """ on the same line.

  • fill-column for docstrings should be 78.

  • When testing the emptiness of sequences, use if len(seq) == 0 instead of relying on the falseness of empty sequences. However, if a variable can be one of several false values, it’s okay to just use if seq, though a preceding comment is usually in order.

  • Always decide whether a class’s methods and instance variables should be public or non-public.

    Single leading underscores are generally preferred for non-public attributes. Use double leading underscores only in classes designed for inheritance to ensure that truly private attributes will never name clash. These should be rare.

    Public attributes should have no leading or trailing underscores unless they conflict with reserved words, in which case, a single trailing underscore is preferable to a leading one, or a corrupted spelling, e.g. class_ rather than klass.

Mailman 3 Internationalization

Mailman does not yet support IDNA (internationalized domain names, RFC 5890) or internationalized mailboxes (RFC 6531) in email addresses. But display names and descriptions are fully internationalized in Mailman, using Unicode. Email content is handled by the Python email package, which provides robust handling of internationalized content conforming to the MIME standard (RFCs 2045-2049 and others).

The encoding of URI components addressing a REST endpoint is Unicode UTF-8. Mailman does not currently handle normalization, and we recommend consistently using normal form NFC. (For some languages NFKC is risky, as some users’ personal names may be corrupted by this normalization.) Mailman does not check for confusables or check repertoire.

Introduction to Unicode Concepts

The Unicode Standard is intended to provide an universal set of characters with a single, standard encoding providing an invertible mapping of characters to integers (called code points in this context).

Repertoires

A set of characters is called a repertoire. Unicode itself is intended to provide an universal repertoire sufficient to represent all words in all written languages, but a system may handle a restricted repertoire and still be considered conformant, as long as it does not corrupt characters it does not handle, and does not emit non-character code points.

Convertibility

Unicode is intended to provide a character for each character defined in a national character set standard. This is often controversial: Chinese characters are often unified with Japanese characters that appear somewhat different when displayed, while the Cyrillic and Greek equivalents of the Latin character “A” are treated as separate characters despite being pronounced the same way and being displayed as identical glyphs. These judgments are informed by the notion that a text should round-trip. That is, when a text is converted from Unicode to another encoding, and then back to Unicode, the result should be identical to the source text.

Normalization

For several reasons, Unicode provides for construction of characters by appending composable characters (such as accents) to base characters (typically letters). But since most languages assign a code point to each accented letter, the “round-tripping” requirement described above implies that Unicode should provide a code point for that accented letter, called a precomposed character. This means that for most accented characters, there are two or more ways to represent them, using various combinations of base characters, precomposed characters, and composable characters.

There are also a number of cases where equivalent characters have different code points (in a few extreme cases, the same character has different code points because the original national standard had duplicates). These cases are called compatibility characters.

The Unicode Standard requires that the compose character sequence be treated identically to the precomposed (single) character by all text-processing algorithms. For convenience in matching, an application may choose to normalize texts. There are two normalizations. The NFC normal form requires that all compositions to precomposed characters that can be done should be done. It has the advantage that the length of a word in characters is the number of code points in the word. The NFD normal form requires that all precomposed characters be decomposed into a sequence of a base character followed by composable characters. It useful in contexts where fuzzy matches (i.e., ignoring accents) are desired.

Finally, in each of these two forms a compatibility character may be replaced by its canonical equivalent, denoted NFKC and NFKD, respectively.

Using Unicode in Mailman

In most cases in Mailman it is highly recommended that input be encoded as UTF-8 in NFC format. Although highly conformant systems are becoming more common, there are still many systems that assume that one code point is translated to one glyph on display. On such systems NFC will provide a smoother user experience than NFD. Since much of the text data that Mailman handles is user names, and users frequently strongly prefer a particular compatibility character to its canonical equivalent, NFKC (or NFKD) should be avoided.

There are two other considerations in using Unicode in Mailman. The first is the problem of confusables. Confusables are characters which are considered different but whose glyphs are indistinguishable, such as Latin capital letter A and Greek capital letter Alpha. Similarly, many code points in Unicode are not yet assigned characters, or even defined as non-characters, and thus are not part of the repertoire of characters represented by Unicode.

Mailman makes no attempt to detect inappropriate use of confusables or non-characters (for example, to redirect users to a domain disseminating malware). The risks at present are vanishingly small because the necessary support in the mail system itself is not yet widespread, but this is likely to change in the near future.

Localization

We have it! We just don’t have proper documentation here yet.

Mailman 3 Core architecture

This is a brief overview of the internal architecture of the Mailman 3 core delivery engine. You should start here if you want to understand how Mailman works at the 1000 foot level. Another good source of architectural information is available in the chapter written by Barry Warsaw for the Architecture of Open Source Applications.

User model

Every major component of the system is defined by an interface. Look through src/mailman/interfaces for an understanding of the system components. Mailman objects which are stored in the database, are defined by model classes. Objects such as mailing lists, users, members, and addresses are primary objects within the system.

The mailing list is the central object which holds all the configuration settings for a particular mailing list. A mailing list is associated with a domain, and all mailing lists are managed (i.e. created, destroyed, looked up) via the mailing list manager.

Users represent people, and have a user id and a display name. Users are linked to addresses which represent a single email address. One user can be linked to many addresses, but an address is only linked to one user. Addresses can be verified or not verified. Mailman will deliver email only to verified addresses.

Users and addresses are managed by the user manager.

A member is created by linking a subscriber to a mailing list. Subscribers can be:

  • A user, which become members through their preferred address.
  • An address, which can be linked or unlinked to a user, but must be verified.

Members also have a role, representing regular members, digest members, list owners, and list moderators. Members can even have the non-member role (i.e. people not yet subscribed to the mailing list) for various moderation purposes.

Process model

Messages move around inside the Mailman system by way of queue directories managed by the switchboard. For example, when a message is first received by Mailman, it is moved to the in (for “incoming”) queue. During the processing of this message, it -or copies of it- may be moved to other queues such as the out queue (for outgoing email), the archive queue (for sending to the archivers), the digest queue (for composing digests), etc.

A message in a queue is represented by a single file, a .pck file. This file contains two objects, serialized as Python pickles. The first object is the message being processed, already parsed into a more efficient internal representation. The second object is a metadata dictionary that records additional information about the message as it is being processed.

.pck files only exist for messages moving between different system queues. There is no .pck file for messages while they are actively being processed.

Each queue directory is associated with a runner process which wakes up every so often. When the runner wakes up, it examines all the .pck files in FIFO order, deserializing the message and metadata objects, and processing them. If the message needs further processing in a different queue, it will be re-serialized back into a .pck file. If not (e.g. because processing of the message is complete), then no .pck file is written.

The Mailman system uses a few other runners which don’t process messages in a queue. You can think of these as fairly typical server process, and examples include the LMTP server, and the HTTP server for processing REST commands.

All of the runners are managed by a master watcher process. When you type mailman start you are actually starting the master. Based on configuration options, the master will start the appropriate runners as subprocesses, and it will watch for the clean exiting of these subprocesses when mailman stop is called.

Rules and chains

When a message is first received for posting to a mailing list, Mailman processes the message to determine whether the message is appropriate for the mailing list. If so, it accepts the message and it gets posted. Mailman can discard the message so that no further processing occurs. Mailman can also reject the message, bouncing it back to the original sender, usually with some indication of why the message was rejected. Or, Mailman can hold the message for moderator approval.

Moderation is the phase of processing that determines which of the above four dispositions will occur for the newly posted message. Moderation does not generally change the message, but it may record information in the metadata dictionary. Moderation is performed by the in queue runner.

Each step in the moderation phase applies a rule to the message and asks whether the rule hits or misses. Each rule is linked to an action which is taken if the rule hits (i.e. matches). If the rule misses (i.e. doesn’t match), then the next rule is tried. All of the rule/action links are strung together sequentially into a chain, and every mailing list has a start chain where rule processing begins.

Actually, every mailing list has two start chains, one for regular postings to the mailing list, and another for posting to the owners of the mailing list.

To recap: when a message comes into Mailman for posting to a mailing list, the incoming runner finds the destination mailing list, determines whether the message is for the entire list membership, or the list owners, and retrieves the appropriate start chain. The message is then passed to the chain, where each link in the chain first checks to see if its rule matches, and if so, it executes the linked action. This action is usually one of accept, reject, discard, and hold, but other actions are possible, such as executing a function, deferring action, or jumping to another chain.

As you might imagine, you can write new rules, compose them into new chains, and configure a mailing list to use your custom chain when processing the message during the moderation phase.

Pipeline of handlers

Once a message is accepted for posting to the mailing list, the message is usually modified in a number of different ways. For example, some message headers may be added or removed, some MIME parts might be scrubbed, added, or rearranged, and various informative headers and footers may be added to the message.

The process of preparing the message for the list membership (as well as the digests, archivers, and NNTP) falls to the pipeline of handlers managed by the pipeline queue.

The pipeline of handlers is similar to the processing chain, except here, a handler can make any modifications to the message it wants, and there is no rule decision or action. The message and metadata simply flow through a sequence of handlers arranged in a named pipeline. Some of the handlers modify the message in ways described above, and others copy the message to the outgoing, NNTP, archiver, or digester queues.

As with chains, each mailing list has two pipelines, one for posting to the list membership, and the other for posting to the list’s owners.

Of course, you can define new handlers, compose them into new pipelines, and change a mailing list’s pipelines.

Integration and control

Humans and external programs can interact with a running Core system in may different ways. There’s an extensive command line interface that provides useful options to a system administrator. For external applications such as the Postorius web user interface, and the HyperKitty archiver, the administrative REST API <rest-api> is the most common way to get information into and out of the Core.

Note: The REST API is an administrative API and as such it must not be exposed to the public internet. By default, the REST server only listens on localhost.

Internally, the Python API is extensive and well-documented. Most objects in the system are accessed through the Zope Component Architecture (ZCA). If your Mailman installation is importable, you can write scripts directly against the internal public Python API.

Other bits and pieces

There are lots of other pieces to the Mailman puzzle, such as the set of core functionality (logging, initialization, event handling, etc.), mailing list styles, the API for integrating external archivers and mail servers. The database layer is an critical piece, and Mailman has an extensive set of command line commands, and email commands.

Almost the entire system is documented in these pages, but it maybe be a bit of a spelunking effort to find it. Improvements are welcome!

Notes from the PyCon 2012 Mailman Sprint

These are notes from the Mailman sprint at PyCon 2012. They are not terribly well organized, nor fully fleshed out. Please edit and push branches to `Gitlab`_ or post patches to the Mailman bug tracker at <https://gitlab.com/mailman/mailman/issues>.

The intent of this document is to provide a view of Mailman 3’s workflow and structures from “eight miles high”.

Basic Messaging Handling Workflow

Mailman accepts a message via the LMTP protocol (RFC 2033). It implements a simple LMTP server internally based on the LMTP server provided in the Python standard library. The LMTP server’s responsibility is to parse the message into a tuple (mlist, msg, msgdata). If the parse fails (including messages which Mailman considers to be invalid due to lack of Message-Id as strongly recommended by RFC 2822 and RFC 5322), the message will be rejected, otherwise the parsed message and metadata dictionary are pickled, and the resulting message pickle added to one of the in, command, or bounce processing queues.

digraph msgflow { rankdir = LR; node [shape=box, color=lightblue, style=filled]; msg [shape=ellipse, color=black, fillcolor=white]; lmtpd [label="LMTP\nSERVER"]; rts [label="Return\nto Sender"]; msg -> MTA [label="SMTP"]; MTA -> lmtpd [label="LMTP"]; lmtpd -> MTA [label="reject"]; lmtpd -> IN -> PIPELINE [label=".pck"]; IN -> rts; lmtpd -> BOUNCES [label=".pck"]; lmtpd -> COMMAND [label=".pck"]; }

The in queue is processed by filter chains (explained below) to determine whether the post (or administrative request) will be processed. If not allowed, the message pickle is discarded, rejected (returned to sender), or held (saved for moderator approval – not shown). Otherwise the message is added to the pipeline (i.e. posting) queue. (Note that rejecting at this stage is not equivalent to rejecting during LMTP processing. This issue is currently unresolved.)

Each of the command, bounce, and pipeline queues is processed by a pipeline of handlers as in Mailman 2’s pipeline. (Some functions such as spam detection that were handled in the Mailman 2 pipeline are now in the filter chains.)

Handlers may copy messages to other queues (e.g., archive), and eventually posted messages for distribution to the list membership end up in the out queue for injection into the MTA.

The virgin queue (not depicted above) is a special queue for messages created by Mailman.

digraph pipeline { node [shape=box, style=rounded, group=0] { "MIME\ndelete" -> "cleanse headers" -> "add headers" -> "calculate\nrecipients" -> "to digest" -> "to archive" -> "to outgoing" } node [shape=box, color=lightblue, style=filled, group=1] { rank=same; PIPELINE -> "MIME\ndelete" } { rank=same; "to digest" -> DIGEST } { rank=same; "to archive" -> ARCHIVE } { rank=same; "to outgoing" -> OUT } }

Message Filtering

Once a message has been classified as a post or administrivia, rules are applied to determine whether the message should be distributed or acted on. Rules include things like “if the message’s sender is a non-member, hold it for moderation”, or “if the message contains an Approved header with a valid password, allow it to be posted”. A rule may also make no decision, in which case message processing is passed on to the next rule in the filter chain. The default set of rules looks something like this:

Configuration

Mailman 3 uses lazr.config, essentially an “ini”-style configuration format.

Each Runner’s configuration object knows whether it should be started when the Mailman daemon starts, and what queue the Runner manages.

Shell Commands

mailman: This is an ubercommand, with subcommands for all the various things admins might want to do, similar to Mailman 2’s mailmanctl, but with more functionality.

bin/master: The runner manager: starts, watches, stops the runner daemons.

bin/runner: Individual runner daemons. Each instance is configured with arguments specified on the command line.

User Model

A user represents a person. A user has an id and a display name, and optionally a list of linked addresses.

Each address is a separate object, linked to no more than one user.

A list member associates an address with a mailing list. Each list member has a id, a mailing list name, an address (which may be None, representing the user’s preferred address), a list of preferences, and a role such as “owner” or “moderator”. Roles are used to determine what kinds of mail the user receives via that membership. Owners will receive mail to list-owner, but not posts and moderation traffic, for example. A user with multiple roles on a single list will therefore have multiple memberships in that list, one for each role.

Roles are implemented by “magical, invisible” rosters which are objects representing queries on the membership database.

List Styles

Each list style is a named object. Its attributes are functions used to apply the relevant style settings to the mailing list at creation time. Since these are functions, they can be composed in various ways, to create substyles, etc.

GNU Mailman 3 changes

Copyright (C) 1998-2017 by the Free Software Foundation, Inc. 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA

Here is a history of user visible changes to Mailman.

3.1.1

(2017-11-17)

Bugs
  • An AttributeError: ‘str’ object has no attribute ‘decode’ exception in subject prefixing is fixed. (Closes #359)
  • Syntactically invalid sender addresses are now ignored. (Closes #229)
  • Messages with no syntactically valid senders are now automatically discarded. (Closes #369)
  • Generated regexp tables for Postfix now account for possible +extra additions to the -bounces and -confirm addresses. (Closes #401)
  • A list whose name is one of the admin, bounces, confirm, etc. subaddresses can now be posted to. (Closes #433)
  • Various message holds and rejects that gave ‘N/A’ as a reason now give an appropriate reason. (Closes #368)
  • A missing html_to_plain_text_command is now properly detected and logged. (closes #345)
  • Bounce messages are now composed for proper translations.
Configuration
  • Mailman now also searches at /etc/mailman3/mailman.cfg for the configuration file.
Interfaces
  • A new template list:user:notice:rejected has been added for customizing the bounce message rejection notice.
REST
  • Allow a mailing list’s acceptable aliases to be cleared by calling DELETE on the list’s config/acceptable_aliases resource. (Closes #394)

3.1.0 – “Between The Wheels”

(2017-05-25)

Bugs
  • When the mailing list’s admin_notify_mchanges is True, the list owners now get the subscription notification. (Closes: #1)
  • Fix the traceback that occurred when trying to convert a text/html subpart to plaintext via the mimedel handler. Now, a configuration variable [mailman]html_to_plain_text_command in the mailman.cfg file defines the command to use. It defaults to lynx. (Closes: #109)
  • Confirmation messages should not be Precedence: bulk. (Closes #75)
  • Fix constraint violations on mailing list deletes affecting PostgreSQL. Given by Abhilash Raj. (Closes #115)
  • mailman command with no subcommand now prints the help text. Given by Abhilash Raj. (Closes #137)
  • The MHonArc archiver must set stdin=PIPE when calling the subprocess. Given by Walter Doekes.
  • For now, treat DeliveryMode.summary_digests the same as .mime_digests. (Closes #141). Also, don’t enqueue a particular digest if there are no recipients for that digest.
  • For Python versions earlier than 3.5, use a compatibility layer for a backported smtpd module which can accept non-UTF-8 data. (Closes #140)
  • Bulk emails are now decorated with headers and footers. Given by Aurélien Bompard. (Closes #145)
  • Core no longer depends on the standalone mock module. (Closes: #146)
  • The logging of moderation reasons has been fixed. Given by Aurélien Bompard.
  • Collapse multiple Re: in Subject headers. Given by Mark Sapiro. (Closes: #147)
  • Added Trove classifiers to setup.py. (Closes: #152)
  • Fix the processing of subscription confirmation messages when the mailing list is set to confirm-then-moderate. (Closes #114)
  • Fix UnicodeEncodeError in the hold chain when sending the authorization email to the mailing list moderators. (Closes: #144)
  • Fix traceback in approved handler when the moderator password is None. Given by Aurélien Bompard.
  • Fix IntegrityErrors raised under PostreSQL when deleting users and addresses. Given by Aurélien Bompard.
  • Allow mailing lists to have localhost names with a suffix matching the subcommand extensions. Given by Aurélien Bompard. (Closes: #168)
  • Don’t traceback if a nonexistent message-id is deleted from the message store. Given by Aurélien Bompard, tweaked by Barry Warsaw. (Closes: #167)
  • Fix a bug in SubscriptionService.find_members() when searching for a subscribed address that is not linked to a user. Given by Aurélien Bompard.
  • Fix a REST server crash when trying to subscribe a user without a preferred address. (Closes #185)
  • Fix membership query when multiple users are subscribed to a mailing list. Reported by Darrell Kresge. (Closes: #190)
  • Prevent moderation of messages held for a different list. (Closes: #161)
  • When approving a subscription request via the REST API, for a user who is already a member, return an HTTP 409 Conflict code instead of the previous server traceback (and resulting HTTP 500 code). (Closes: #193)
  • In decoration URIs (e.g. IMailingList.header_uri and .footer_uri) you should now use the mailing list’s List-ID instead of the fqdn-listname. The latter is deprecated. (Closes #196)
  • Trying to subscribe an address as a list owner (or moderator or nonmember) which is already subscribed with that role produces a server error. Originally given by Anirudh Dahiya. (Closes #198)
  • Cross-posting messages held on both lists no longer fails. (Closes #176)
  • Don’t let unknown charsets crash the “approved” rule. Given by Aurélien Bompard. (Closes #203)
  • Don’t let crashes in IArchiver plugins break handlers or runners. (Closes #208)
  • Fix “None” as display name in welcome message. Given by Aditya Divekar. (Closes #194)
  • Fix mailman shell processing of $PYTHONSTARTUP. (Closes #224)
  • Fix query bug for SubscriptionService.find_members() leading to the incorrect number of members being returned. Given by Aurélien Bompard. (Closes #227)
  • Fix header match rule suffix inflation. Given by Aurélien Bompard. (Closes #226)
  • MIME digests now put the individual message/rfc822 messages inside a multipart/digest subpart. (Closes #234)
  • Nonmember subscriptions are removed when one of the addresses controlled by a user is subscribed as a member. Given by Aditya Divekar. (Closes #237)
  • Email address validation is now more compliant with RFC 5321. (Closes #266)
  • A mailing list’s description must not contain newlines. Given by Aurélien Bompard. (Closes: #273)
  • Allow MailingList.info to be set using the REST API. Given by Aurélien Bompard.
  • Extend header filters to also check sub-part headers. (Closes #280)
  • Allow REST API to PUT and PATCH domain attributes. Allows Postorius domain edit to work. (Closes: #290)
  • Prevent posting from banned addresses. Given by Aurélien Bompard. (Closes: #283)
  • Remove the digest mbox files after the digests are sent. Given by Aurélien Bompard. (Closes: #259)
  • Transmit the moderation reason and expose it in the REST API as the reason attribute. Given by Aurélien Bompard.
  • Don’t return a 500 error from the REST API when trying to handle a held message with defective content. Given by Abhilash Raj. (Closes: #256)
  • Delete subscription requests when a mailing list is deleted. Given by Abhilash Raj. (Closes: #214)
  • Messages were shunted when non-ASCII characters appeared in a mailing list’s description. Given by Mark Sapiro. (Closes: #215)
  • Fix confirmation of unsubscription requests. (Closes: #294)
  • Fix mailman stop not stopping some runners due to PEP 475 interaction. (Closes: #255)
  • Update documentation links for config.cfg settings. (Closes: #306)
  • Disallow problematic characters in listnames. (Closes: #311)
  • Forward port several content filtering fixes from the 2.1 branch. (Closes: #330, #331, #332 and #334)
Configuration
  • Mailing lists can now have their own header matching rules, although site-defined rules still take precedence. Importing a Mailman 2.1 list with header matching rules defined will create them in Mailman 3, albeit with a few unsupported corner cases. Definition of new header matching rules is not yet exposed through the REST API. Given by Aurélien Bompard.
  • The default languages from Mailman 2.1 have been ported over. Given by Aurélien Bompard.
  • There is now a configuration setting to limit the characters that can be used in list names.
Command line
  • mailman create <listname@dom.ain> will now create missing domains by default. The -d/--domain option is kept for backward compatibility, but now there is a -D/--no-domain option to prevent missing domains from being create, forcing an error in those cases. Given by Gurkirpal Singh. (Closes #39)
  • mailman subcommands now properly commit any outstanding transactions. (Closes #223)
  • mailman digests has grown --verbose and -dry-run options.
  • mailman shell now supports readline history if you set the [shell]history_file variable in mailman.cfg. Also, many useful names are pre-populated in the namespace of the shell. (Closes: #228)
Database
  • MySQL is now an officially supported database. Given by Abhilash Raj.
  • Fix a problem with tracebacks when a PostgreSQL database is power cycled while Mailman is still running. This ports an upstream SQLAlchemy fix to Mailman in lieu of a future SQLAlchemy 1.2 release. (Closes: #313)
Interfaces
  • Implement reasons for why a message is being held for moderator approval. Given by Aurélien Bompard, tweaked by Barry Warsaw.
  • The default postauth.txt and postheld.txt templates now no longer include the inaccurate admindb and confirmation urls.
  • Messages now include a Message-ID-Hash as the replacement for X-Message-ID-Hash although the latter is still included for backward compatibility. Also be sure that all places which add the header use the same algorithm. (Closes #118)
  • IMessageStore.delete_message() no longer raises a LookupError when you attempt to delete a nonexistent message from the message store.
  • ISubscriptionService.find_members() accepts asterisks as wildcards in the subscriber argument string. Given by Aurélien Bompard.
  • ISubscriptionService now supports mass unsubscribes. Given by Harshit Bansal.
Message handling
  • New DMARC mitigations have been added. Given by Mark Sapiro. (Closes #247)
  • New placeholders have been added for message headers and footers. You can use a placeholder of the format $<archiver-name>_url to insert the permalink to the message in the named archiver, for any archiver enabled for the mailing list. Given by Abhilash Raj.
  • The default posting chain has been modified so that the header-match chain and nonmember-moderation rule are processed before “hold” rules are processed. This allows for better anti-spam defenses and rejecting non-member posts instead of always holding them for moderator review. Given by Aurélien Bompard. (Closes #163)
  • Bounces can now contain rejection messages. Given by Aurélien Bompard.
  • The moderation_action for members and nonmember can now be None which signals falling back to the appropriate list default action, e.g. default_member_action and default_nonmember_action. Given by Aurélien Bompard. (Closes #189)
  • Ensure that postings from alternative emails aren’t held for moderator approval. For example, if a user is subscribed with one email but posts with a second email that they control, the message should be processed as a posting from a member. Given by Aditya Divekar. (Closes #222)
  • The default message footer has been improved to include a way to unsubscribe via the -leave address. Given by Francesco Ariis.
REST
  • REST API version 3.1 introduced. Mostly backward compatible with version 3.0 except that UUIDs are represented as hex strings instead of 128-bit integers, since the latter are not compatible with all versions of JavaScript. (Closes #121)
  • REST clients must minimally support HTTP/1.1. (Closes #288)
  • Experimental Gunicorn support. See contrib/gunicorn.py docstring for details. With assistance from Eric Searcy. (Closes #287)
  • The new template system is introduced for API 3.1. See src/mailman/rest/docs/templates.rst for details. (Closes #249)
  • When creating a user via REST using an address that already exists, but isn’t linked, the address is linked to the new user. Given by Aurélien Bompard.
  • The REST API incorrectly parsed is_server_owner values when given explicitly in the POST that creates a user. (Closes #136)
  • A new top-level resource <api>/owners can be used to get the list of server owners as IUser s. (Closes #135)
  • By POSTing to a user resource with an existing unlinked address, you can link the address to the user. Given by Abhilash Raj.
  • Fix pagination values start and total_size in the REST API. Given by Aurélien Bompard. (Closes: #154)
  • JSON representations for held message now include a self_link.
  • When [devmode]enabled is set, the JSON output is sorted. Given by Aurélien Bompard.
  • A member’s moderation action can be changed via the REST API. Given by Aurélien Bompard.
  • Fixed a number of corner cases for the return codes when PUTing or PATCHing list configuration variables. (Closes: #182)
  • Expose digest_send_periodic, digest_volume_frequency, and digests_enabled (renamed from digestable) to the REST API. (Closes: #159)
  • Expose the “bump digest” and “send digest” functionality though the REST API via the <api>/lists/<list-id>/digest end-point. GETting this resource returns the next_digest_number and volume as the same values accessible through the list’s configuraiton resource. POSTing to the resource with either send=True, bump=True, or both invokes the given action.
  • Global and list-centric bans can now be managed through the REST API. Given by Aurélien Bompard.
  • <api>/members/find accepts GET query parameters in addition to POST arguments. Given by Aurélien Bompard.
  • Header match rules for individual mailing lists are now exposed in the REST API. Given by Aurélien Bompard. (Closes: #192)
  • Expose goodbye_message_uri in the REST API. Given by Harshit Bansal.
  • New subscription requests are rejected if there is already one pending. With thanks to Anirudh Dahiya. (Closes #199)
  • Expose the system pipelines and chains via <api>/system/pipelines and <api>/system/chains respectively. Given by Simon Hanna. (Closes #66)
  • Support mass unsubscription of members via DELETE on the <api>/lists/<list-id>/roster/member resource. Given by Harshit Bansal. (Closes #171)
  • It is now possible to merge users when creating them via REST. When you POST to <api>/users/<address>/addresses and the address given in the email parameter already exists, instead of getting a 400 error, if you set absorb_existing=True in the POST data, the existing user will be merged into the newly created on. Given by Aurélien Bompard.
  • Port to Falcon 1.0 (Closes #20)
  • A member’s moderation_action can be reset, allowing fallback to the list’s default_member_action by setting the attribute to the empty string in the REST API. Given by Aurélien Bompard.
  • A list’s moderator_password can be set via the REST API. Given by Andrew Breksa. (Closes #207)
  • The ban manager now returns a pageable, sorted sequence. Given by Amit and Aurélien Bompard. (Closes #284)
  • Query parameters now allow you to filter mailing lists by the advertised boolean parameter. Given by Aurélien Bompard.
  • Only the system-enabled archivers are returned in the REST API. Given by Aurélien Bompard.
  • Backward incompatibility: mild Held message resources now have an original_subject key which is the raw value of the Subject: header (i.e. without any RFC 2047 decoding). The subject key is RFC 2047 decoded. Given by Simon Hanna. (Closes #219)
Other
  • Add official support for Python 3.5 and 3.6. (Closes #295)
  • A handful of unused legacy exceptions have been removed. The redundant MailmanException has been removed; use MailmanError everywhere.
  • Drop the use of the lazr.smtptest library, which is based on the asynchat/asyncore-based smtpd.py stdlib module. Instead, use the asyncio-based aiosmtpd package.
  • Improvements in importing Mailman 2.1 lists, given by Aurélien Bompard.
  • The prototype archiver is not web accessible so it does not have a list_url or permalink. Given by Aurélien Bompard.
  • Large performance improvement in SubscriptionService.find_members(). Given by Aurélien Bompard.
  • Rework the digest machinery, and add a new digests subcommand, which can be used from the command line or cron to immediately send out any partially collected digests, or bump the digest and volume numbers.
  • The mailing list “data directory” has been renamed. Instead of using the fqdn listname, the subdirectory inside [paths]list_data_dir now uses the List-ID.
  • The mailman members command can now be used to display members based on subscription roles. Also, the positional “list” argument can now accept list names or list-ids.
  • Unsubscriptions can now be confirmed and/or moderated. (Closes #213)

3.0.0 – “Show Don’t Tell”

(2015-04-28)

Architecture
  • Domains now have a list of owners, which are IUser objects, instead of the single contact_address they used to have. IUser objects now also have a is_server_owner flag (defaulting to False) to indicate whether they have superuser privileges. Give by Abhliash Raj, with fixes and refinements by Barry Warsaw. (LP: #1423756)
  • Mailing list subscription policy work flow has been completely rewritten. It now properly supports email verification and subscription confirmation by the user, and approval by the moderator using unique tokens. IMailingList objects now have a subscription_policy attribute. (LP: #1095552)
  • Port the REST machinery to Falcon 0.3. (LP: #1446881)
Bugs
  • Fix calculation of default configuration file to use when the $var_dir is created by mailman start. (LP: #1411435)
  • When creating a user with an email address, do not create the user record if the email address already exists. Given by Andrew Stuart. (LP: #1418280)
  • When deleting a user via REST, make sure all linked addresses are deleted. Found by Andrew Stuart. (LP: #1419519)
  • When trying to subscribe an address to a mailing list through the REST API where a case-differing version of the address is already subscribed, return a 409 error instead of a 500 error. Found by Ankush Sharma. (LP: #1425359)
  • mailman lists --domain was not properly handling its arguments. Given by Manish Gill. (LP: #1166911)
  • When deleting a user object, make sure their preferences are also deleted. Given by Abhishek. (LP: #1418276)
  • Be sure a mailing list’s acceptable aliases are deleted when the mailing list itself is deleted. (LP: #1432239)
  • The built-in example IArchiver implementations now explicitly return None. (LP: #1203359)
  • The test suite now runs successfully again with PostgreSQL. Given by Aurélien Bompard. (LP: #1435941)
Configuration
  • When specifying a file system path in the [paths.*] section, $cfg_file can be used to expand into the path of the -C option if given. In the default [paths.dev] section, $var_dir is now specified relative to $cfg_file so that it won’t accidentally be relative to the current working directory, if -C is given.
  • $cwd is now an additional substitution variable for the mailman.cfg file’s [paths.*] sections. A new [paths.here] section is added, which puts the var_dir in $cwd. It is made the default layout.
Documentation
  • Improve the documentation describing how to run Alembic to add new schema migrations. Given by Abhilash Raj.
REST
  • Backward incompatible change: The JSON representation for pending mailing list subscription hold now no longer includes the password key. Also, the address key has been renamed email for consistent terminology and other usage.
  • You can now view the contents of, inject messages into, and delete messages from the various queue directories via the <api>/queues resource.
  • You can now DELETE an address. If the address is linked to a user, the user is not delete, it is just unlinked.
  • A new API is provided to support non-production testing infrastructures, allowing a client to cull all orphaned UIDs via DELETE on <api>/reserved/uids/orphans. Note that no guarantees of API stability will ever be made for resources under reserved. (LP: #1420083)
  • Domains can now optionally be created with owners; domain owners can be added after the fact; domain owners can be deleted. Also, users now have an is_server_owner flag as part of their representation, which defaults to False, and can be PUT and PATCH’d. Given by Abhilash Raj, with fixes and refinements by Barry Warsaw. (LP: #1423756)

3.0 beta 5 – “Carve Away The Stone”

(2014-12-29)

Bugs
  • Fixed Unicode errors in the digest runner and when sending messages to the site owner as a fallback. Given by Aurélien Bompard. (LP: #1130957).
  • Fixed Unicode errors when a message being added to the digest has non-ascii characters in its payload, but no Content-Type header defining a charset. Given by Aurélien Bompard. (LP: #1170347)
  • Fixed messages without a text/plain part crashing the Approved rule. Given by Aurélien Bompard. (LP: #1158721)
  • Fixed getting non-ASCII filenames from RFC 2231 i18n’d messages. Given by Aurélien Bompard. (LP: #1060951)
  • Fixed AttributeError on MIME digest messages. Given by Aurélien Bompard. (LP: #1130696)
Commands
  • The mailman conf command no longer takes the -t/–sort option; the output is always sorted.
Configuration
  • The [database]migrations_path setting is removed.
Database
  • The ORM layer, previously implemented with Storm, has been replaced by SQLAlchemy, thanks to the fantastic work by Abhilash Raj and Aurélien Bompard. Alembic is now used for all database schema migrations.
  • The new logger mailman.database logs any errors at the database layer.
Development
  • Python 3.4 is now the minimum requirement.
  • You no longer have to create a virtual environment separately when running the test suite. Just use tox.
  • You no longer have to edit src/mailman/testing/testing.cfg to run the test suite against PostgreSQL. See src/mailman/docs/START.rst for details.
Interfaces
  • The RFC 2369 headers added to outgoing messages are now added in sorted order.
  • Several changes to the internal API:
    • IListManager.mailing_lists is guaranteed to be sorted in List-ID order.
    • IDomains.mailing_lists is guaranteed to be sorted in List-ID order.
    • Iteration over domains via the IDomainManager is guaranteed to be sorted by IDomain.mail_host order.
    • ITemporaryDatabase interface and all implementations are removed.
REST
  • The Falcon Framework has replaced restish as the REST layer. This is an internal change only.
  • The JSON representation http_etag key uses an algorithm that is insensitive to Python’s dictionary sort order.
  • The address resource now has an additional ‘/user’ sub-resource which can be used to GET the address’s linked user if there is one. This sub-resource also supports POST to link an unlinked address (with an optional ‘auto_create’ flag), and PUT to link the address to a different user. It also supports DELETE to unlink the address. (LP: #1312884) Given by Aurélien Bompard based on work by Nicolas Karageuzian.
  • The /3.0/system path is deprecated; use /3.0/system/versions to get the system version information.
  • You can access the system configuration via the resource path /3.0/system/configuration/<section>. This returns a dictionary with the keys being the section’s variables and the values being their value from mailman.cfg as verbatim strings. You can get a list of all section names via /3.0/system/configuration which returns a dictionary containing the http_etag and the section names as a sorted list under the sections key. The system configuration resource is read-only.
  • Member resource JSON now include the member_id as a separate key.

3.0 beta 4 – “Time and Motion”

(2014-04-22)

Development
  • Mailman 3 no longer uses zc.buildout and tests are now run by the nose2 test runner. See src/mailman/docs/START.rst for details on how to build Mailman and run the test suite. Also, use -P to select a test pattern and -E to enable stderr debugging in runners.
  • Use the enum34 package instead of flufl.enum.
  • Use setuptools instead of distribute, since the latter is defunct.
REST
  • Add reply_to_address and first_strip_reply_to as writable attributes of a mailing list’s configuration. (LP: #1157881)
  • Support pagination of some large collections (lists, users, members). [Florian Fuchs] (LP: #1156529)
  • Expose hide_address to the .../preferences REST API. [Sneha Priscilla.] (LP: #1203519)
  • Mailing lists can now individually enable or disable any archiver available site-wide. [Joanna Skrzeszewska] (LP: #1158040)
  • Addresses can be added to existing users, including display names, via the REST API. [Florian Fuchs]
  • Fixed a crash in the REST server when searching for nonmembers via /find which we’ve never seen before, because those members only have an address record, not a user record. This requires a small change in the API where the JSON response’s address key now contains the URL to the address resource, the new email key contains the email address as a string, and the user key is optional.
Commands
  • mailman conf now has a -t/–sort flag which sorts the output by section and then key. [Karl-Aksel Puulmann and David Soto] (LP: 1162492)
  • Greatly improve the fidelity of the Mailman 2.1 list importer functionality (i.e. mailman import21). [Aurélien Bompard].
Configuration
  • Add support for the Exim 4 MTA. [Stephen Turnbull]
  • When creating the initial file system layout in var, e.g. via bin/mailman info, add an var/etc/mailman.cfg file if one does not already exist. Also, when initializing the system, look for that file as the configuration file, just after ./mailman.cfg and before ~/.mailman.cfg. (LP: #1157861)
Database
  • The bounceevent table now uses list-ids to cross-reference the mailing list, to match other tables. Similarly for the IBounceEvent interface.
  • Added a listarchiver table to support list-specific archivers.
Bugs
  • Non-queue runners should not create var/queue subdirectories. [Sandesh Kumar Agrawal] (LP: #1095422)
  • Creation of lists with upper case names should be coerced to lower case. (LP: #1117176)
  • Fix REST server crash on mailman reopen due to no interception of signals. (LP: #1184376)
  • Add subject_prefix to the IMailingList interface, and clarify the docstring for display_name. (LP: #1181498)
  • Fix importation from MM2.1 to MM3 of the archive policy. [Aurélien Bompard] (LP: #1227658)
  • Fix non-member moderation rule to prefer a member sender if both members and non-members are in the message’s sender list. [Aurélien Bompard] (LP: #1291452)
  • Fix IntegrityError (against PostgreSQL) when deleting a list with content filters. [Aurélien Bompard] (LP: #1117174)
  • Fix test isolation bug in languages.rst. [Piotr Kasprzyk] (LP: #1308769)

3.0 beta 3 – “Here Again”

(2012-12-31)

Compatibility
  • Python 2.7 is now required. Python 2.6 is no longer officially supported. The code base is now also python2.7 -3 clean, although there are still some warnings in 3rd party dependencies. (LP: #1073506)
REST
  • API change: The JSON representation for held messages no longer includes the data key. The values in this dictionary are flatted into the top-level JSON representation. The key key is remove since it’s redundant. Use message_id for held messages, and address for held subscriptions/unsubscriptions. The following _mod_* keys are inserted without the _mod_ prefix:
    • _mod_subject -> subject
    • _mod_hold_date -> hold_date
    • _mod_reason -> reason
    • _mod_sender -> sender
    • _mod_message_id -> message_id
  • List styles are supported through the REST API. Get the list of available styles (by name) via …/lists/styles. Create a list in a specific style by using POST data style_name=<style>. (LP: #975692)
  • Allow the getting/setting of IMailingList.subject_prefix via the REST API (given by Terri Oda). (LP: #1062893)
  • Expose a REST API for membership change (subscriptions and unsubscriptions) moderation. (LP: #1090753)
  • Add list_id to JSON representation for a mailing list (given by Jimmy Bergman).
  • The canonical resource for a mailing list (and thus its self_link) is now the URL with the list-id. To reference a mailing list, the list-id url is preferred, but for backward compatibility, the posting address is still accepted.
  • You can now PUT and PATCH on user resources to change the user’s display name or password. For passwords, you pass in the clear text password and Mailman will hash it before storing.
  • You can now verify and unverify an email address through the REST API. POST to …/addresses/<email>/verify and …/addresses/<email>/unverify respectively. The POST data is ignored. It is not an error to verify or unverify an address more than once, but verifying an already verified address does not change its .verified_on date. (LP: #1054730)
  • Deleting a user through the REST API also deletes all the user’s linked addresses and memberships. (LP: #1074374)
  • A user’s password can be verified by POSTing to …/user/<id>/login. The data must contain a single parameter cleartext_password and if this matches, a 204 (No Content) will be returned, otherwise a 403 (Forbidden) is returned. (LP: #1065447)
Configuration
  • [passlib]path configuration variable renamed to [passlib]configuration.
  • Postfix-specific configurations in the [mta] section are moved to a separate file, named by the [mta]configuration variable.
  • In the new postfix.cfg file, postfix_map_cmd is renamed to postmap_command.
  • The default list style is renamed to legacy-default and a new legacy-announce style is added. This is similar to the legacy-default except set up for announce-only lists.
Database
  • The ban table now uses list-ids to cross-reference the mailing list, since these cannot change even if the mailing list is moved or renamed.
  • The following columns were unused and have been removed:
    • mailinglist.new_member_options
    • mailinglist.send_reminders
    • mailinglist.subscribe_policy
    • mailinglist.unsubscribe_policy
    • mailinglist.subscribe_auto_approval
    • mailinglist.private_roster
    • mailinglist.admin_member_chunksize
Interfaces
  • The IBanManager is no longer a global utility. Instead, you adapt an IMailingList to an IBanManager to manage the bans for a specific mailing list. To manage the global bans, adapt None.
Commands
  • bin/mailman aliases loses the –output, –format, and –simple arguments, and adds a –directory argument. This is necessary to support the Postfix relay_domains support.
  • bin/mailman start was passing the wrong relative path to its runner subprocesses when -C was given. (LP: #982551)
  • bin/runner command has been simplified and its command line options reduced. Now, only one -r/–runner option may be provided and the round-robin feature has been removed.
Other
  • Added support for Postfix relay_domains setting for better virtual domain support. [Jimmy Bergman].
  • Two new events are triggered on membership changes: SubscriptionEvent when a new member joins a mailing list, and an UnsubscriptionEvent when a member leaves a mailing list. (LP: #1047286)
  • Improve the –help text for the start, stop, restart, and reopen subcommands. (LP: #1035033)
Bugs
  • Fixed send_goodbye_message(). (LP: #1091321)
  • Fixed REST server crash on reopen command. Identification and test provided by Aurélien Bompard. (LP: #1184376)

3.0 beta 2 – “Freeze”

(2012-09-05)

Architecture
  • The link between members and the mailing lists they are subscribed to, is now via the RFC 2369 list_id instead of the fqdn listname (i.e. posting address). This is because while the posting address can change if the mailing list is moved to a new server, the list id is fixed. (LP: #1024509)
    • IListManager.get_by_list_id() added.
    • IListManager.list_ids added.
    • IMailingList.list_id added.
    • Several internal APIs that accepted fqdn list names now require list ids, e.g. ISubscriptionService.join() and .find_members().
    • IMember.list_id attribute added; .mailing_list is now an alias that retrieves and returns the IMailingList.
  • passlib is now used for all password hashing instead of flufl.password. The default hash is sha512_crypt. (LP: #1015758)
  • Internally, all datetimes are kept in the UTC timezone, however because of LP: #280708, they are stored in the database in naive format.
  • received_time is now added to the message metadata by the LMTP runner instead of by Switchboard.enqueue(). This latter no longer depends on received_time in the metadata.
  • The ArchiveRunner no longer acquires a lock before it calls the individual archiver implementations, since not all of them need a lock. If they do, the implementations must acquire said lock themselves.
  • The news runner and queue has been renamed to the more accurate nntp. The runner has also been ported to Mailman 3 (LP: #967409). Beta testers can safely remove $var_dir/queue/news.
  • A mailing list’s moderator password is no longer stored in the clear; it is hashed with the currently selected scheme.
  • An AddressVerificationEvent is triggered when an IAddress is verified or unverified. (LP: #975698)
  • A PasswordChangeEvent is triggered when an IUser’s password changes. (LP: #975700)
  • When a queue runner gets an exception in its _dispose() method, a RunnerCrashEvent is triggered, which contains references to the queue runner, mailing list, message, metadata, and exception. Interested parties can subscribe to that zope.event for notification.
  • Events renamed and moved: * mailman.chains.accept.AcceptNotification * mailman.chains.base.ChainNotification * mailman.chains.discard.DiscardNotification * mailman.chains.hold.HoldNotification * mailman.chains.owner.OwnerNotification * mailman.chains.reject.RejectNotification changed to (respectively): * mailman.interfaces.chains.AcceptEvent * mailman.interfaces.chains.ChainEvent * mailman.interfaces.chains.DiscardEvent * mailman.interfaces.chains.HoldEvent * mailman.interfaces.chains.AcceptOwnerEvent * mailman.interfaces.chains.RejectEvent
  • A ConfigurationUpdatedEvent is triggered when the system-wide global configuration stack is pushed or popped.
  • The policy for archiving has now been collapsed into a single enum, called ArchivePolicy. This describes the three states of never archive, archive privately, and archive_publicly. (LP: #967238)
Database
  • Schema migrations (LP: #971013)
    • mailinglist.include_list_post_header -> allow_list_posts
    • mailinglist.news_prefix_subject_too -> nntp_prefix_subject_too
    • mailinglist.news_moderation -> newsgroup_moderation
    • mailinglist.archive and mailinglist.archive_private have been collapsed into archive_policy.
    • mailinglist.nntp_host has been removed.
    • mailinglist.generic_nonmember_action has been removed (LP: #975696)
  • Schema migrations (LP: #1024509) - member.mailing_list -> list_id
  • The PostgreSQL port of the schema accidentally added a moderation_callback column to the mailinglist table. Since this is unused in Mailman, it was simply commented out of the base schema for PostgreSQL.
REST
  • Expose archive_policy in the REST API. Contributed by Alexander Sulfrian. (LP: #1039129)
Configuration
  • New configuration variables clobber_date and clobber_skew supported in every [archiver.<name>] section. These are used to determine under what circumstances a message destined for a specific archiver should have its Date: header clobbered. (LP: #963612)
  • With the switch to passlib, [passwords]password_scheme has been removed. Instead use [passwords]path to specify where to find the passlib.cfg file. See the comments in schema.cfg for details.
  • Configuration schema variable changes: * [nntp]username -> [nntp]user * [nntp]port (added)
  • Header check specifications in the mailman.cfg file have changed quite bit. The previous [spam.header.foo] sections have been removed. Instead, there’s a new [antispam] section that contains a header_checks variable. This variable takes multiple lines of Header: regexp values, one per line. There is also a new jump_chain variable which names the chain to jump to should any of the header checks (including the list-specific, and programmatically added ones) match.
Documentation
  • Some additional documentation on related components such as Postorius and hyperkitty have been added, given by Stephen J Turnbull.
Bug fixes
  • Fixed the RFC 1153 digest footer to be compliant. (LP: #887610)
  • Fixed a UnicodeError with non-ascii message bodies in the approved rule, given by Mark Sapiro. (LP: #949924)
  • Fixed a typo when returning the configuration file’s header match checks. (LP: #953497)
  • List-Post should be NO when posting is not allowed. (LP: #987563)
  • Non-unicode values in msgdata broke pending requests. (LP: #1031391)
  • Show devmode in bin/mailman info output. (LP: #1035028)
  • Fix residual references to the old IMailingList archive variables. (LP: #1031393)

3.0 beta 1 – “The Twilight Zone”

(2012-03-23)

Architecture
  • Schema migrations have been implemented.
  • Implement the style manager as a utility instead of an attribute hanging off the mailman.config.config object.
  • PostgreSQL support contributed by Stephen A. Goss. (LP: #860159)
  • Separate out the RFC 2369 header adding handler.
  • Dynamically calculate the List-Id header instead of storing it in the database. This means it cannot be changed.
  • Major redesign of the template search system, fixing LP: #788309. $var_dir is now used when search for all template overrides, site, domain, or mailing list. The in-tree English templates are used only as a last fallback.
  • Support downloading templates by URI, including mailman:// URIs. This is used in welcome and goodbye messages, as well as regular and digest headers and footers, and supports both language and mailing list specifications. E.g. mailman:///test@example.com/it/welcome.txt
  • $user_password is no longer supported as a placeholder in headers and footers.
  • Mailing lists get multiple chains and pipelines. For example, normal postings go through the posting_chain while messages to owners to through owners_chain. The default built-in chain is renamed to default-posting-chain while the built-in pipeline is renamed default-posting-pipeline.
  • The experimental maildir runner is removed. Use LMTP.
  • The LMTP server now requires that the incoming message have a Message-ID, otherwise it rejects the message with a 550 error. Also, the LMTP server adds the X-Message-ID-Hash header automatically. The inject cli command will also add the X-Message-ID-Hash header, but it will craft a Message-ID header first if one is missing from the injected text. Also, inject will always set the correct value for the original_size attribute on the message object, instead of trusting a possibly incorrect value if it’s already set. The individual IArchiver implementations no longer set the X-Message-ID-Hash header.
  • The Prototype archiver now stores its files in maildir format inside of $var_dir/archives/prototype, given by Toshio Kuratomi.
  • Improved “8 mile high” document distilled by Stephen J Turnbull from the Pycon 2012 Mailman 3 sprint. Also improvements to the Sphinx build given by Andrea Crotti (LP: #954718).
  • Pipermail has been eradicated.
  • Configuration variable [mailman]filtered_messages_are_preservable controls whether messages which have their top-level Content-Type filtered out can be preserved in the bad queue by list owners.
  • Configuration section [scrubber] removed, as is the scrubber handler. This handler was essentially incompatible with Mailman 3 since it required coordination with Pipermail to store attachments on disk.
Database
  • Schema changes: - welcome_msg -> welcome_message_uri - goodbye_msg -> goodbye_message_uri - send_welcome_msg -> send_welcome_message - send_goodbye_msg -> send_goodbye_message - msg_header -> header_uri - msg_footer -> footer_uri - digest_header -> digest_header_uri - digest_footer -> digest_footer_uri - start_chain -> posting_chain - pipeline -> posting_pipeline - real_name -> display_name (mailinglist, user, address)
  • Schema additions: - mailinglist.filter_action - mailinglist.owner_chain - mailinglist.owner_pipeline
REST
  • Held messages can now be moderated through the REST API. Mailing list resources now accept a held path component. GETing this returns all held messages for the mailing list. POSTing to a specific request id under this url can dispose of the message using Action enums.
  • Mailing list resources now have a member_count attribute which gives the number of subscribed members. Given by Toshio Kuratomi.
Interfaces
  • Add property IUserManager.members to return all IMembers in the system.
  • Add property IListmanager.name_components which returns 2-tuples for every mailing list as (list_name, mail_host).
  • Remove previously deprecated IListManager.get_mailing_lists().
  • IMailTransportAgentAliases now explicitly accepts duck-typed arguments.
  • IRequests interface is removed. Now just use adaptation from IListRequests directly (which takes an IMailingList object).
  • handle_message() now allows for Action.hold which is synonymous with Action.defer (since the message is already being held).
  • IListRequests.get_request() now takes an optional request_type argument to narrow the search for the given request.
  • New ITemplateLoader utility.
  • ILanguageManager.add() returns the ILanguage object just created.
  • IMailinglist.decorators removed; it was unused
  • IMailingList.real_name -> IMailingList.display_name
  • IUser.real_name -> IUser.display_name
  • IAddress.real_name -> IAddress.display_name
  • Add property IRoster.member_count.
Commands
  • IPython support in bin/mailman shell contributed by Andrea Crotti. (LP: #949926).
  • The mailman.cfg configuration file will now automatically be detected if it exists in an etc directory which is a sibling of argv0.
  • bin/mailman shell is an alias for withlist.
  • The confirm email command now properly handles Re:-like prefixes, even if they contain non-ASCII characters. (LP: #685261)
  • The join email command no longer accepts an address= argument. Its digest= argument now accepts the following values: no (for regular delivery), mime, or plain.
  • Added a help email command.
  • A welcome message is sent when the user confirms their subscription via email.
  • Global -C option now accepts an absolute path to the configuration file. Given by Andrea Crotti. (LP: #953707)
Bug fixes
  • Subscription disabled probe warning notification messages are now sent without a Precedence: header. Given by Mark Sapiro. (LP: #808821)
  • Fixed KeyError in retry runner, contributed by Stephen A. Goss. (LP: #872391)
  • Fixed bogus use of bounce_processing attribute (should have been process_bounces, with thanks to Vincent Fretin. (LP: #876774)
  • Fix test_moderation for timezones east of UTC+0000, given by blacktav. (LP: #890675)

3.0 alpha 8 – “Where’s My Thing?”

(2011-09-23)

Architecture
  • Factor out bounce detection to flufl.bounce.
  • Unrecognized bounces can now also be forwarded to the site owner.
  • mailman.qrunner log is renamed to mailman.runner
  • master-qrunner.lck -> master.lck
  • master-qrunner.pid -> master.pid
  • Four new events are created, and notifications are sent during mailing list lifecycle changes: - ListCreatingEvent - sent before the mailing list is created - ListCreatedEvent - sent after the mailing list is created - ListDeletingEvent - sent before the mailing list is deleted - ListDeletedEvent - sent after the mailing list is deleted
  • Four new events are created, and notifications are sent during domain lifecycle changes: - DomainCreatingEvent - sent before the domain is created - DomainCreatedEvent - sent after the domain is created - DomainDeletingEvent - sent before the domain is deleted - DomainDeletedEvent - sent after the domain is deleted
  • Using the above events, when a domain is deleted, associated mailing lists are deleted. (LP: #837526)
  • IDomain.email_host -> .mail_host (LP: #831660)
  • User and Member ids are now proper UUIDs.
  • Improved the way enums are stored in the database, so that they are more explicitly expressed in the code, and more database efficient.
REST
  • Preferences for addresses, users, and members can be accessed, changed, and deleted through the REST interface. Hierarchical, combined preferences for members, and system preferences can be read through the REST interface. (LP: #821438)
  • The IMailingList attribute host_name has been renamed to mail_host for consistency. This changes the REST API for mailing list resources. (LP: #787599)
  • New REST resource http://…/members/find can be POSTed to in order to find member records. Optional arguments are subscriber (email address to search for), fqdn_listname, and role (i.e. MemberRole). (LP: #799612)
  • You can now query or change a member’s delivery_mode attribute through the REST API (LP: #833132). Given by Stephen A. Goss.
  • New REST resource http://…/<domain>/lists can be GETed in order to find all the mailing lists in a specific domain (LP: #829765). Given by Stephen A. Goss.
  • Fixed /lists/<fqdn_listname>/<role>/<email> (LP: #825570)
  • Remove role plurals from /lists/<fqdn_listname/rosters/<role>
  • Fixed incorrect error code for /members/<bogus> (LP: #821020). Given by Stephen A. Goss.
  • DELETE users via the REST API. (LP: #820660)
  • Moderators and owners can be added via REST (LP: #834130). Given by Stephen A. Goss.
  • Getting the roster or configuration of a nonexistent list did not give a 404 error (LP: #837676). Given by Stephen A. Goss.
  • PATCHing an invalid attribute on a member did not give a 400 error (LP: #833376). Given by Stephen A. Goss.
  • Getting the memberships for a non-existent address did not give a 404 error (LP: #848103). Given by Stephen A. Goss.
Commands
  • bin/qrunner is renamed to bin/runner.
  • bin/mailman aliases gains -f and -s options.
  • bin/mailman create no longer allows a list to be created with bogus owner addresses. (LP: #778687)
  • bin/mailman start –force option is fixed. (LP: #869317)
Documentation
  • Update the COPYING file to contain the GPLv3. (LP: #790994)
  • Major terminology change: ban the terms “queue runners” and “qrunners” since not all runners manage queue directories. Just call them “runners”. Also, the master is now just called “the master runner”.
Testing
  • New configuration variable in [devmode] section, called wait which sets the timeout value used in the test suite for starting up subprocesses.
  • Handle SIGTERM in the REST server so that the test suite always shuts down correctly. (LP: #770328)
Other bugs and changes
  • Moderating a message with Action.accept now sends the message. (LP: #827697)
  • Fix AttributeError triggered by i18n call in autorespond_to_sender() (LP: #827060)
  • Local timezone in X-Mailman-Approved-At caused test failure. (LP: #832404)
  • InvalidEmailAddressError no longer repr()’s its value.
  • Rewrote a test for compatibility between Python 2.6 and 2.7. (LP: #833208)
  • Fixed Postfix alias file generation when more than one mailing list exists. (LP: #874929). Given by Vincent Fretin.

3.0 alpha 7 – “Mission”

(2011-04-29)

Architecture
  • Significant updates to the subscription model. Members can now subscribe with a preferred address, and changes to that will be immediately reflected in mailing list subscriptions. Users who subscribe with an explicit address can easily change to a different address, as long as that address is verified. (LP: #643949)
  • IUsers and IMembers are now assigned a unique, random, immutable id.
  • IUsers now have created_on and .preferred_address properties.
  • IMembers now have a .user attribute for easy access to the subscribed user.
  • When created with add_member(), passwords are always stored encrypted.
  • In all interfaces, “email” refers to the textual email address while “address” refers to the IAddress object.
  • mailman.chains.base.Chain no longer self registers.
  • New member and nonmember moderation rules and chains. This effectively ports moderation rules from Mailman 2 and replaces attributes such as member_moderation_action, default_member_moderation, and generic_nonmember_action. Now, nonmembers exist as subscriptions on a mailing list and members have a moderation_action attribute which describes the disposition for postings from that address.
  • Member.is_moderated was removed because of the above change.
  • default_member_action and default_nonmember_action were added to mailing lists.
  • All sender addresses are registered (unverified) with the user manager by the incoming queue runner. This way, nonmember moderation rules will always have an IAddress that they can subscribe to the list (as MemberRole.nonmember).
  • Support for SMTP AUTH added via smtp_user and smtp_pass configuration variables in the [mta] section. (LP: #490044)
  • IEmailValidator interface for pluggable validation of email addresses.
  • .subscribe() is moved from the IAddress to the IMailingList
  • IAddresses get their registered_on attribute set when the object is created.
Configuration
  • [devmode] section gets a new ‘testing’ variable.
  • Added password_scheme and password_length settings for defining the default password encryption scheme.
  • creator_pw_file and site_pw_file are removed.
Commands
  • ‘bin/mailman start’ does a better job of producing an error when Mailman is already running.
  • ‘bin/mailman status’ added for providing command line status on the master queue runner watcher process.
  • ‘bin/mailman info’ now prints the REST root url and credentials.
  • mmsitepass removed; there is no more site password.
REST
  • Add Basic Auth support for REST API security. (Jimmy Bergman)
  • Include the fqdn_listname and email address in the member JSON representation.
  • Added reply_goes_to_list, send_welcome_msg, welcome_msg, default_member_moderation to the mailing list’s writable attributes in the REST service. (Jimmy Bergman)
  • Expose the new membership model to the REST API. Canonical member resource URLs are now much shorter and live in their own top-level namespace instead of within the mailing list’s namespace.
  • /addresses/<email>/memberships gets all the memberships for a given email address.
  • /users is a new top-level URL under which user information can be accessed. Posting to this creates new users.
  • Users can subscribe to mailing lists through the REST API.
  • Domains can be deleted via the REST API.
  • PUT and PATCH to a list configuration now returns a 204 (No Content).
Build
  • Support Python 2.7. (LP: #667472)
  • Disable site-packages in buildout.cfg because of LP: #659231.
  • Don’t include eggs/ or parts/ in the source tarball. (LP: #656946)
  • flufl.lock is now required instead of locknix.
Bugs fixed
  • Typo in scan_message(). (LP: #645897)
  • Typo in add_member(). (LP: #710182) (Florian Fuchs)
  • Re-enable bounce detectors. (LP: #756943)
  • Clean up many pyflakes problems; ditching pylint.

3.0 alpha 6 – “Cut to the Chase”

(2010-09-20)

Commands
  • The functionality of ‘bin/list_members’ has been moved to ‘bin/mailman members’.
  • ‘bin/mailman info’ -v/–verbose output displays the file system layout paths Mailman is currently configured to use.
Configuration
  • You can now configure the paths Mailman uses for queue files, lock files, data files, etc. via the configuration file. Define a file system ‘layout’ and then select that layout in the [mailman] section. Default layouts include ‘local’ for putting everything in /var/tmp/mailman, ‘dev’ for local development, and ‘fhs’ for Filesystem Hierarchy Standard 2.3 (LP #490144).
  • Queue file directories now live in $var_dir/queues.
REST
  • lazr.restful has been replaced by restish as the REST publishing technology used by Mailman.
  • New REST API for getting all the members of a roster for a specific mailing list.
  • New REST API for getting and setting a mailing list’s configuration. GET and PUT are supported to retrieve the current configuration, and set all the list’s writable attributes in one request. PATCH is supported to partially update a mailing list’s configuration. Individual options can be set and retrieved by using subpaths.
  • Subscribing an already subscribed member via REST now returns a 409 HTTP error. LP: #552917
  • Fixed a bug when deleting a list via the REST API. LP: #601899
Architecture
  • X-BeenThere header is removed.
  • Mailman no longer touches the Sender or Errors-To headers.
  • Chain actions can now fire Zope events in their _process() implementations.
  • Environment variable $MAILMAN_VAR_DIR can be used to control the var/ directory for Mailman’s runtime files. New environment variable $MAILMAN_UNDER_MASTER_CONTROL is used instead of the qrunner’s –subproc/-s option.
Miscellaneous
  • Allow X-Approved and X-Approve headers, equivalent to Approved and Approve. LP: #557750
  • Various test failure fixes. LP: #543618, LP: #544477
  • List-Post header is retained in MIME digest messages. LP: #526143
  • Importing from a Mailman 2.1.x list is partially supported.

3.0 alpha 5 – “Distant Early Warning”

(2010-01-18)

REST
  • Add REST API for subscription services. You can now:
    • list all members in all mailing lists
    • subscribe (and possibly register) an address to a mailing list
    • unsubscribe an address from mailing list
Commands
  • ‘bin/dumpdb’ is now ‘bin/mailman qfile’
  • ‘bin/unshunt’ is now ‘bin/mailman unshunt’
  • Mailman now properly handles the ‘-join’, ‘-leave’, and ‘-confirm’ email commands and sub-addresses. ‘-subscribe’ and ‘-unsubscribe’ are aliases for ‘-join’ and ‘-leave’ respectively.
Configuration
  • devmode settings now live in their own [devmode] section.
  • Mailman now searches for a configuration file using this search order. The first file that exists is used.
    • -C config command line argument
    • $MAILMAN_CONFIG_FILE environment variable
    • ./mailman.cfg
    • ~/.mailman.cfg
    • /etc/mailman.cfg

3.0 alpha 4 – “Vital Signs”

(2009-11-28)

Commands
  • ‘bin/inject’ is now ‘bin/mailman inject’, with some changes
  • ‘bin/mailmanctl’ is now ‘bin/mailman start|stop|reopen|restart’
  • ‘bin/mailman version’ is added (output same as ‘bin/mailman –version’)
  • ‘bin/mailman members’ command line arguments have changed. It also now ignores blank lines and lines that start with #. It also no longer quits when it sees an address that’s already subscribed.
  • ‘bin/withlist’ is now ‘bin/mailman withlist’, and its command line arguments have changed.
  • ‘bin/mailman lists’ command line arguments have changed.
  • ‘bin/genaliases’ is now ‘bin/mailman aliases’
Architecture
  • A near complete rewrite of the low-level SMTP delivery machinery. This greatly improves readability, testability, reuse and extensibility. Almost all the old functionality has been retained. The smtp_direct.py handler is gone.
  • Refactor model objects into the mailman.model subpackage.
  • Refactor most of the i18n infrastructure into a separate flufl.i18n package.
  • Switch from setuptools to distribute.
  • Remove the dependency on setuptools_bzr
  • Do not create the .mo files during setup.
Configuration
  • All log files now have a ‘.log’ suffix by default.
  • The substitution placeholders in the verp_format configuration variable have been renamed.
  • Add a devmode configuration variable that changes some basic behavior. Most importantly, it allows you to set a low-level SMTP recipient for all mail for testing purposes. See also devmode_recipient.

3.0 alpha 3 – “Working Man”

(2009-08-21)

Configuration
  • Configuration is now done through lazr.config. Defaults.py is dead. lazr.config files are essentially hierarchical ini files.
  • Domains are now stored in the database instead of in the configuration file.
  • pre- and post- initialization hooks are now available to plugins. Specify additional hooks to run in the configuration file.
  • Add the environment variable $MAILMAN_CONFIG_FILE which overrides the -C command line option.
  • Make LMTP more compliant with Postfix docs (Patrick Koetter)
  • Added a NullMTA for mail servers like Exim which just work automatically.
Architecture
  • ‘bin/mailman’ is a new super-command for managing Mailman from the command line. Some older bin scripts have been converted, with more to come.
  • Mailman now has an administrative REST interface which can be used to get information from and manage Mailman remotely.
  • Back port of Mailman 2.1’s limit on .bak file restoration. After 3 restores, the file is moved to the bad queue, with a .psv extension. (Mark Sapiro)
  • Digest creation is moved into a new queue runner so it doesn’t block main message processing.
Other changes
  • bin/make_instance is no longer necessary, and removed
  • The debug log is turned up to info by default to reduce log file spam.
Building and installation
  • All doc tests can now be turned into documentation, via Sphinx. Just run bin/docs after bin/buildout.

3.0 alpha 2 – “Grand Designs”

(03-Jan-2009)

Licensing
  • Mailman 3 is now licensed under the GPLv3.
Bug fixes
  • Changed bin/arch to attempt to open the mbox before wiping the old archive. Launchpad bug #280418.
  • Added digest.mbox and pending.pck to the ‘list’ files checked by check_perms. Launchpad bug #284802.
Architecture
  • Converted to using zope.testing as the test infrastructure. Use bin/test now to run the full test suite. <http://pypi.python.org/pypi/zope.testing/3.7.1>
  • Partially converted to using lazr.config as the new configuration regime. Not everything has been converted yet, so some manual editing of mailman/Defaults.py is required. This will be rectified in future versions. <http://launchpad.net/lazr.config>
  • All web-related stuff is moved to its own directory, effectively moving it out of the way for now.
  • The email command infrastructure has been reworked to play more nicely with the plug-in architecture. Not all commands have yet been converted.
Other changes
  • The LMTP server now properly calculates the message’s original size.
  • For command line scripts, -C names the configuration file to use. For convenient testing, if -C is not given, then the environment variable MAILMAN_CONFIG_FILE is consulted.
  • Support added for a local MHonArc archiver, as well as archiving automatically in the remote Mail-Archive.com service.
  • The permalink proposal for supporting RFC 5064 has been adopted.
  • Mailing lists no longer have a .web_page_url attribute; this is taken from the mailing list’s domain’s base_url attribute.
  • Incoming MTA selection is now taken from the config file instead of plugins. An MTA for Postfix+LMTP is added. bin/genaliases works again.
  • If a message has no Message-ID, the stock archivers will return None for the permalink now instead of raising an assertion.
  • IArchiver no longer has an is_enabled property; this is taken from the configuration file now.
Installation

3.0 alpha 1 – “Leave That Thing Alone”

(08-Apr-2008)

User visible changes
  • So called ‘new style’ subject prefixing is the default now, and the only option. When a list’s subject prefix is added, it’s always done so before any Re: tag, not after. E.g. ‘[My List] Re: The subject’.
  • RFC 2369 headers List-Subscribe and List-Unsubscribe now use the preferred -join and -leave addresses instead of the -request address with a subject value.
Configuration

  • There is no more separate configure; make; make install step. Mailman 3.0 is a setuptools package.

  • Mailman can now be configured via a ‘mailman.cfg’ file which lives in $VAR_PREFIX/etc. This is used to separate the configuration from the source directory. Alternative configuration files can be specified via -C/–config for most command line scripts. mailman.cfg contains Python code. mm_cfg.py is no more. You do not need to import Defaults.py in etc/mailman.cfg. You should still consult Defaults.py for the list of site configuration variables available to you.

    See the etc/mailman.cfg.sample file.

  • PUBLIC_ARCHIVE_URL and DEFAULT_SUBJECT_PREFIX now takes $-string substitutions instead of %-string substitutions. See documentation in Defaults.py.in for details.

  • Message headers and footers now only accept $-string substitutions; %-strings are no longer supported. The substitution variable ‘_internal_name’ has been removed; use $list_name or $real_name instead. The substitution variable $fqdn_listname has been added. DEFAULT_MSG_FOOTER in Defaults.py.in has been updated accordingly.

  • The KNOWN_SPAMMERS global variable is replaced with HEADER_MATCHES. The mailing list’s header_filter_rules variable is replaced with header_matches which has the same semantics as HEADER_MATCHES, but is list-specific.

  • DEFAULT_MAIL_COMMANDS_MAX_LINES -> EMAIL_COMMANDS_MAX_LINES

  • All SMTP_LOG_* templates use $-strings and all consistently write the Message-ID as the first item in the log entry.

  • DELIVERY_MODULE now names a handler, not a module (yes, this is a misnomer, but it will likely change again before the final release).

Architecture

  • Internally, all strings are Unicodes.

  • Implementation of a chain-of-rules based approach for deciding whether a message should initially be accepted, held for approval, rejected/bounced, or discarded. This replaces most of the disposition handlers in the pipeline. The IncomingRunner now only processes message through the rule chains, and once accepted, places the message in a new queue processed by the PipelineRunner.

  • Substantially reworked the entire queue runner process management, including mailmanctl, a new master script, and the qrunners. This should be much more robust and reliable now.

  • The Storm ORM is used for data storage, with the SQLite backend as the default relational database.

  • Zope interfaces are used to describe the major components.

  • Users are now stored in a unified database, and shared across all mailing lists.

  • Mailman’s web interface is now WSGI compliant. WSGI is a Python standard (PEP 333) allowing web applications to be (more) easily integrated with any number of existing Python web application frameworks. For more information see:

    http://www.wsgi.org/wsgi http://www.python.org/dev/peps/pep-0333/

    Mailman can still be run as a traditional CGI program of course.

  • Mailman now provides an LMTP server for more efficient integration with supporting mail servers (e.g. Postfix, Sendmail). The Local Mail Transport Protocol is defined in RFC 2033:

    http://www.faqs.org/rfcs/rfc2033.html

  • Virtual domains are now fully supported in that mailing lists of the same name can exist in more than one domain. This is accomplished by renaming the lists/ and archives/ subdirectories after the list’s posting address. For example, data for list foo in example.com and list foo in example.org will be stored in lists/foo@example.com and lists/foo@example.org.

    For Postfix or manual MTA users, you will need to regenerate your mail aliases. Use bin/genaliases.

    VIRTUAL_HOST_OVERVIEW has been removed, effectively Mailman now operates as if it were always enabled. If your site has more than one domain, you must configure all domains by using add_domain() in your etc/mailman.cfg flie (see below – add_virtual() has been removed).

  • If you had customizations based on Site.py, you will need to re-implement them. Site.py has been removed.

  • The site list is no more. You can remove your ‘mailman’ site list unless you want to retain it for other purposes, but it is no longer used (or required) by Mailman. You should set NO_REPLY_ADDRESS to an address that throws away replies, and you should set SITE_OWNER_ADDRESS to an email address that reaches the person ultimately responsible for the Mailman installation. The MAILMAN_SITE_LIST variable has been removed.

  • qrunners no longer restart on SIGINT; SIGUSR1 is used for that now.

Internationalization Big Changes
  • Translators should work only on messages/<lang>/LC_MESSAGES/mailman.po. Templates files are generated from mailman.po during the build process.
New Features
  • Confirmed member change of address is logged in the ‘subscribe’ log, and if admin_notify_mchanges is true, a notice is sent to the list owner using a new adminaddrchgack.txt template.
  • There is a new list attribute ‘subscribe_auto_approval’ which is a list of email addresses and regular expressions matching email addresses whose subscriptions are exempt from admin approval. RFE 403066.
Command line scripts
  • Most scripts have grown a -C/–config flag to allow you to specify a different configuration file. Without this, the default etc/mailman.cfg file will be used.
  • the -V/–virtual-host-overview switch in list_lists has been removed, while -d/–domain and -f/–full have been added.
  • bin/newlist is renamed bin/create_list and bin/rmlist is renamed bin/remove_list. Both take fully-qualified list names now (i.e. the list’s posting address), but also accept short names, in which case the default domain is used. newlist’s -u/–urlhost and -e/–emailhost switches have been removed. The domain that the list is being added to must already exist.
  • Backport the ability to specify additional footer interpolation variables by the message metadata ‘decoration-data’ key.
Bug fixes and other patches
  • Removal of DomainKey/DKIM signatures is now controlled by Defaults.py mm_cfg.py variable REMOVE_DKIM_HEADERS (default = No).
  • Queue runner processing is improved to log and preserve for analysis in the shunt queue certain bad queue entries that were previously logged but lost. Also, entries are preserved when an attempt to shunt throws an exception (1656289).
  • The processing of Topics regular expressions has changed. Previously the Topics regexp was compiled in verbose mode but not documented as such which caused some confusion. Also, the documentation indicated that topic keywords could be entered one per line, but these entries were not handled properly. Topics regexps are now compiled in non-verbose mode and multi- line entries are ‘ored’. Existing Topics regexps will be converted when the list is updated so they will continue to work.
  • The List-Help, List-Subscribe, and List-Unsubscribe headers were incorrectly suppressed in messages that Mailman sends directly to users.
  • The ‘adminapproved’ metadata key is renamed ‘moderator_approved’.

GNU Mailman Acknowledgments

Copyright (C) 1998-2017 by the Free Software Foundation, Inc.

Governance

GNU Mailman was invented by John Viega. Barry Warsaw is the current project leader. Aurélien Bompard leads HyperKitty and bundler development. Florian Fuchs leads Postorius development. Development of mailman.client is a group effort.

All project decisions are made by consensus via the Mailman Cabal, er, Steering Committee which can be contacted directly via mailman-cabal@python.org

Core Developers

The following folks are or have been core developers of Mailman (in alphabetical order):

  • Abhilash Raj, Mailman’s Youngest Core Dev
  • Aurélien Bompard, Pizzaman
  • Barry Warsaw, Mailman’s yappy guard dog
  • Florian Fuchs
  • Harald Meland, Norse Mailman
  • John Viega, Mailman’s inventor
  • Ken Manheimer, Mailman’s savior
  • Mark Sapiro, Mailman’s compulsive responder
  • Scott Cotton, Cookie-Monster
  • Stephen J. Turnbull, Standards Otaku
  • Terri Oda
  • Thomas Wouters, Mailman’s Dutch treat
  • Tokio Kikuchi (RIP), Mailman’s weatherman

Special Thanks

Very special thanks to Andrija Arsic for his winning GNU Mailman logos.

Thanks also go to the following people for their important contributions in other aspects of the Mailman project:

  • Brad Knowles
  • Clytie Siddall
  • JC Dill

Thanks also to Dragon for his original Mailman logo contribution, and to Terri Oda for the neat shortcut icon and the member documentation.

Control.com sponsored development of several Mailman 2.1 features, including topics filters, external membership sources, and initial virtual mailing list support. My thanks especially to Dan Pierson and Ken Crater from Control.com.

Here is the list of other people who have contributed useful ideas, suggestions, bug fixes, testing, etc., or who have been very helpful in answering questions on mailman-users. Please let me know if you have been left off the list!

  • “office”
  • Ademar de Souza Reis, Jr.
  • Alessio Bragadini
  • Alexander Sulfrian
  • Andrew Kuchling
  • Andrew Martynov
  • Anti Veeranna
  • Anton Antonov
  • Antonis Limperis
  • Ashley M. Kirchner
  • Balazs Nagy
  • Bartosz Sawicki
  • Ben Burnett
  • Bernhard Reiter
  • Bernhard Schmidt
  • Bert Hubert
  • Bill Wagner
  • Blair Zajac
  • Bob Fleck
  • Bob Puff
  • Bojan
  • Cabel Sasser
  • Carson Gaspar
  • Chris Kolar
  • Chris Pepper
  • Chris Ryan
  • Chris Snell
  • Christian F Buser
  • Christian Reis
  • Christopher P. Lindsey
  • Chuq Von Rospach
  • Claudio Cattazzo
  • Dai Xiaoguang
  • Dale Newfield
  • Dale Stimson
  • Dan Mick
  • Dan Ohnesorg
  • Dan Wilder
  • Daniel Buchmann
  • Daniel Zeiss
  • Danil Smirnov
  • Danny Terweij
  • Dario Lopez-Kästen
  • Darrell Fuhriman
  • David Abrahams
  • David B. O’Donnell
  • David Blomquist
  • David Champion
  • David Gibbs
  • David Habben
  • David Martínez Moreno
  • David Soto
  • David T-G
  • Diego Francisco de Gastal Morales
  • Dirk Mueller
  • Dmitri I GOULIAEV
  • Don Porter
  • Donn Cave
  • Ed Lau
  • Eddie Kohler
  • Egon Frerich
  • Emerson Ribeiro de Mello
  • Emilio Delgado
  • Eric D. Christensen
  • Erik Forsberg
  • Erik Myllymaki
  • Eva Österlind
  • Fabian Wenk
  • Federico Grau
  • Fil
  • Florian Weimer
  • Francesco Potortì
  • Francis Jorissen
  • Franck Martin
  • Fred Drake
  • Gabriel P. Silva
  • Garey Mills
  • Gari Araolaza
  • Geoff Mayes
  • Gerald Oskoboiny
  • Gergely Madarasz
  • Gleydson Mazioli da Silva
  • Grant Bowman
  • Greg Lindahl
  • Greg Stein
  • Greg Ward
  • Guido van Rossum
  • Harald Koch
  • Heiko Rommel
  • Henny Huisman
  • Hrvoje Niksic
  • Hugo Koji Kobayashi
  • Hye-Shik Chang
  • Ikeda Soji
  • J C Lawrence
  • J. D. Bronson
  • James Henstridge
  • Jan Veuger
  • Jason R. Mastaler
  • Javad Hoseini
  • Javier Rial Rodríguez
  • Jay Luker
  • Jeff Berliner
  • Jeff Hahn
  • Jens Vagelpohl
  • Jeremy Hylton
  • Jim Popovitch
  • Jim Tittsler
  • Jimmy Bergman
  • Joe Peterson
  • John A. Martin
  • John Carnes
  • John Dennis
  • John Read
  • Jon Parise
  • Jonas Muerer
  • Jonas Smedegaard
  • Joni Töyrylä
  • Jose Paulo Moitinho de Almeida
  • Julio A. Cartaya
  • Kai Schaetzl
  • Karl Chen
  • Karoly Segesdi
  • Kathleen Webb
  • Kerem Erkan
  • Kleber A. Benatti
  • L’homme Moderne
  • Les Niles
  • Lindsay Haisley
  • Lionel Elie Mamane
  • Luca Maranzano
  • Luigi Rosa
  • Mahyar Moghimi
  • Marc MERLIN
  • Marcos Costales
  • Mark Weaver
  • Martijn Dekker
  • Martin ‘Joey’ Schulze
  • Martin Matuska
  • Martin Mokrejs
  • Martin Pool
  • Martin von Loewis
  • Matthias Andree
  • Matthias Juchem
  • Matthias Klose
  • Maxim Dzumanenko
  • Maxime Carron
  • Maximillian Dornseif
  • Mentor Cana
  • Michael Fischer v. Mollard
  • Michael Mclay
  • Michael Meltzer
  • Michael Ranner
  • Michael Yount
  • Mike Avery
  • Mike Noyes
  • Mikhail Sobolev
  • Mikhail Zabaluev
  • Miloslav Trmac
  • Mirian Margiani
  • Moreno Baricevic
  • Moritz Naumann
  • Ned Dawes
  • Nicholas Russo
  • Nigel Metheringham
  • Nino Katic
  • Noam Zeilberger
  • Ousmane Wilane
  • Owen Taylor
  • Pascal GEORGE
  • Pasi Sjöholm
  • Patrick Finnerty
  • Patrick Koetter
  • Paul Cox
  • Paul Hebble
  • Peer Heinlein
  • Pekka Haavisto
  • Phil Pennock
  • Piarres Beobide Egaña
  • PieterB
  • Ping Yeh
  • Ralf Doeblitz
  • Ralf Hildebrandt
  • Ricardo Kustner
  • Rob Ellis
  • Robert Daeley
  • Robert Garrigós
  • Rodolfo Pilas
  • Roger Tsang
  • Ron Jarrell
  • Rostyk Ivantsiv
  • SATOH Fumiyasu
  • SHIGENO Kazutaka
  • Sean Reifschneider
  • Seb Wills
  • Skye Poier
  • Stan Bubrouski
  • Stefan Divjak
  • Stefan Förster
  • Stefan Plewako
  • Stefaniu Criste
  • Stephan Richter
  • Stig Hackvan
  • Stonewall Ballard
  • Stuart Bishop
  • Students of HIT <mailman-cn@mail.cs.hit.edu.cn>
  • Sven Anderson
  • Sylvain Langlade
  • Szabolcs Szigeti
  • Søren Bondrup
  • Tamito KAJIYAMA
  • Tanner Lovelace
  • Ted Cabeen
  • Terry Allen
  • Terry Grace
  • Terry Hardie
  • Thijs Kinkhorst
  • Tim Peters
  • Timothy O’Malley
  • Todd (Freedom Lover)
  • Todd Vierling
  • Todd Zullinger
  • Tollef Fog Heen
  • Tom G. Christensen
  • Tomasz Chmielewski
  • Toni Panadès
  • Tristan Roddis
  • Uros Kositer
  • Vadim Getmanshchuk
  • Valia V. Vaneeva
  • Vizi Szilard
  • Walter Hop
  • William Ahern
  • YASUDA Yukihiro
  • Yasuhito FUTATSUKI

And everyone else on mailman-developers@python.org and mailman-users@python.org! Thank you, all.

Mailman modules

These documents are generated from the internal module documentation.

Models

Email addresses

Addresses represent a text email address, along with some meta data about those addresses, such as their registration date, and whether and when they’ve been validated. Addresses may be linked to the users that Mailman knows about. Addresses are subscribed to mailing lists though members.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)
Creating addresses

Addresses are created directly through the user manager, which starts out with no addresses.

>>> dump_list(address.email for address in user_manager.addresses)
*Empty*

Creating an unlinked email address is straightforward.

>>> address_1 = user_manager.create_address('aperson@example.com')
>>> dump_list(address.email for address in user_manager.addresses)
aperson@example.com

However, such addresses have no real name.

>>> print(address_1.display_name)
<BLANKLINE>

You can also create an email address object with a real name.

>>> address_2 = user_manager.create_address(
...     'bperson@example.com', 'Ben Person')
>>> dump_list(address.email for address in user_manager.addresses)
aperson@example.com
bperson@example.com
>>> dump_list(address.display_name for address in user_manager.addresses)
<BLANKLINE>
Ben Person

The str() of the address is the RFC 2822 preferred originator format, while the repr() carries more information.

>>> print(str(address_2))
Ben Person <bperson@example.com>
>>> print(repr(address_2))
<Address: Ben Person <bperson@example.com> [not verified] at 0x...>

You can assign real names to existing addresses.

>>> address_1.display_name = 'Anne Person'
>>> dump_list(address.display_name for address in user_manager.addresses)
Anne Person
Ben Person

These addresses are not linked to users, and can be seen by searching the user manager for an associated user.

>>> print(user_manager.get_user('aperson@example.com'))
None
>>> print(user_manager.get_user('bperson@example.com'))
None

You can create email addresses that are linked to users by using a different interface.

>>> user_1 = user_manager.create_user(
...     'cperson@example.com', u'Claire Person')
>>> dump_list(address.email for address in user_1.addresses)
cperson@example.com
>>> dump_list(address.email for address in user_manager.addresses)
aperson@example.com
bperson@example.com
cperson@example.com
>>> dump_list(address.display_name for address in user_manager.addresses)
Anne Person
Ben Person
Claire Person

And now you can find the associated user.

>>> print(user_manager.get_user('aperson@example.com'))
None
>>> print(user_manager.get_user('bperson@example.com'))
None
>>> user_manager.get_user('cperson@example.com')
<User "Claire Person" (...) at ...>
Deleting addresses

You can remove an unlinked address from the user manager.

>>> user_manager.delete_address(address_1)
>>> dump_list(address.email for address in user_manager.addresses)
bperson@example.com
cperson@example.com
>>> dump_list(address.display_name for address in user_manager.addresses)
Ben Person
Claire Person

Deleting a linked address does not delete the user, but it does unlink the address from the user.

>>> dump_list(address.email for address in user_1.addresses)
cperson@example.com
>>> user_1.controls('cperson@example.com')
True
>>> address_3 = list(user_1.addresses)[0]
>>> user_manager.delete_address(address_3)
>>> dump_list(address.email for address in user_1.addresses)
*Empty*
>>> user_1.controls('cperson@example.com')
False
>>> dump_list(address.email for address in user_manager.addresses)
bperson@example.com
Registration and verification

Addresses have two dates, the date the address was registered on and the date the address was validated on. The former is set when the address is created, but the latter must be set explicitly.

>>> address_4 = user_manager.create_address(
...     'dperson@example.com', 'Dan Person')
>>> print(address_4.registered_on)
2005-08-01 07:49:23
>>> print(address_4.verified_on)
None

The verification date records when the user has completed a mail-back verification procedure. It takes a datetime object.

>>> from mailman.utilities.datetime import now
>>> address_4.verified_on = now()
>>> print(address_4.verified_on)
2005-08-01 07:49:23

The address shows the verified status in its representation.

>>> address_4
<Address: Dan Person <dperson@example.com> [verified] at ...>

An event is triggered when the address gets verified.

>>> saved_event = None
>>> address_5 = user_manager.create_address(
...     'eperson@example.com', 'Elle Person')
>>> def save_event(event):
...     global saved_event
...     saved_event = event
>>> from mailman.testing.helpers import event_subscribers
>>> with event_subscribers(save_event):
...     address_5.verified_on = now()
>>> print(saved_event)
<AddressVerificationEvent eperson@example.com 2005-08-01 07:49:23>

An event is also triggered when the address is unverified. In this case, check the event’s address’s verified_on attribute; if this is None, then the address is being unverified.

>>> with event_subscribers(save_event):
...     address_5.verified_on = None
>>> print(saved_event)
<AddressVerificationEvent eperson@example.com unverified>
>>> print(saved_event.address.verified_on)
None
Case-preserved addresses

Technically speaking, email addresses are case sensitive in the local part. Mailman preserves the case of addresses and uses the case preserved version when sending the user a message, but it treats addresses that are different in case equivalently in all other situations.

>>> address_6 = user_manager.create_address(
...     'FPERSON@example.com', 'Frank Person')

The str() of such an address prints the RFC 2822 preferred originator format with the original case-preserved address. The repr() contains all the gory details.

>>> print(str(address_6))
Frank Person <FPERSON@example.com>
>>> print(repr(address_6))
<Address: Frank Person <FPERSON@example.com> [not verified]
          key: fperson@example.com at 0x...>

Both the case-insensitive version of the address and the original case-preserved version are available on attributes of the IAddress object.

>>> print(address_6.email)
fperson@example.com
>>> print(address_6.original_email)
FPERSON@example.com

Because addresses are case-insensitive for all other purposes, you cannot create an address that differs only in case. You can get the address using either the lower cased version or case-preserved version. In fact, searching for an address is case insensitive.

>>> print(user_manager.get_address('fperson@example.com').email)
fperson@example.com
>>> print(user_manager.get_address('FPERSON@example.com').email)
fperson@example.com
Automatic responder

In various situations, Mailman will send an automatic response to the author of an email message. For example, if someone sends a command to the -request address, Mailman will send a response, but to cut down on third party spam, the sender will only get a certain number of responses per day.

First, given a mailing list you need to adapt it to an IAutoResponseSet.

>>> mlist = create_list('test@example.com')
>>> from mailman.interfaces.autorespond import IAutoResponseSet
>>> response_set = IAutoResponseSet(mlist)

>>> from zope.interface.verify import verifyObject
>>> verifyObject(IAutoResponseSet, response_set)
True

You can’t adapt other objects to an IAutoResponseSet.

>>> IAutoResponseSet(object())
Traceback (most recent call last):
...
TypeError: ('Could not adapt', ...

There are various kinds of response types. For example, Mailman will send an automatic response when messages are held for approval, or when it receives an email command. You can find out how many responses for a particular address have already been sent today.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> address = getUtility(IUserManager).create_address(
...     'aperson@example.com')

>>> from mailman.interfaces.autorespond import Response
>>> print(response_set.todays_count(address, Response.hold))
0
>>> print(response_set.todays_count(address, Response.command))
0

Using the response set, we can record that a hold response is sent to the address.

>>> response_set.response_sent(address, Response.hold)
>>> print(response_set.todays_count(address, Response.hold))
1
>>> print(response_set.todays_count(address, Response.command))
0

We can also record that a command response was sent.

>>> response_set.response_sent(address, Response.command)
>>> print(response_set.todays_count(address, Response.hold))
1
>>> print(response_set.todays_count(address, Response.command))
1

Let’s send one more.

>>> response_set.response_sent(address, Response.command)
>>> print(response_set.todays_count(address, Response.hold))
1
>>> print(response_set.todays_count(address, Response.command))
2

Now the day flips over and all the counts reset.

>>> from mailman.utilities.datetime import factory
>>> factory.fast_forward()

>>> print(response_set.todays_count(address, Response.hold))
0
>>> print(response_set.todays_count(address, Response.command))
0
Response dates

You can also use the response set to get the date of the last response sent.

>>> response = response_set.last_response(address, Response.hold)
>>> response.mailing_list
<mailing list "test@example.com" at ...>
>>> response.address
<Address: aperson@example.com [not verified] at ...>
>>> response.response_type
<Response.hold: 1>
>>> response.date_sent
datetime.date(2005, 8, 1)

When another response is sent today, that becomes the last one sent.

>>> response_set.response_sent(address, Response.command)
>>> response_set.last_response(address, Response.command).date_sent
datetime.date(2005, 8, 2)

>>> factory.fast_forward(days=3)
>>> response_set.response_sent(address, Response.command)
>>> response_set.last_response(address, Response.command).date_sent
datetime.date(2005, 8, 5)

If there’s been no response sent to a particular address, None is returned.

>>> address = getUtility(IUserManager).create_address(
...     'bperson@example.com')
>>> print(response_set.todays_count(address, Response.command))
0
>>> print(response_set.last_response(address, Response.command))
None
Bounces

When a message to an email address bounces, Mailman’s bounce runner will register a bounce event. This registration is done through a utility.

>>> from zope.component import getUtility
>>> from zope.interface.verify import verifyObject
>>> from mailman.interfaces.bounce import IBounceProcessor
>>> processor = getUtility(IBounceProcessor)
>>> verifyObject(IBounceProcessor, processor)
True
Registration

When a bounce occurs, it’s always within the context of a specific mailing list.

>>> mlist = create_list('test@example.com')

The bouncing email contains useful information that will be registered as well. In particular, the Message-ID is a key piece of data that needs to be recorded.

>>> msg = message_from_string("""\
... From: mail-daemon@example.org
... To: test-bounces@example.com
... Message-ID: <first>
...
... """)

There is a suite of bounce detectors that are used to heuristically extract the bouncing email addresses. Various techniques are employed including VERP, DSN, and magic. It is the bounce runner’s responsibility to extract the set of bouncing email addresses. These are passed one-by-one to the registration interface.

>>> event = processor.register(mlist, 'anne@example.com', msg)
>>> print(event.list_id)
test.example.com
>>> print(event.email)
anne@example.com
>>> print(event.message_id)
<first>

Bounce events have a timestamp.

>>> print(event.timestamp)
2005-08-01 07:49:23

Bounce events have a flag indicating whether they’ve been processed or not.

>>> event.processed
False

When a bounce is registered, you can indicate the bounce context.

>>> msg = message_from_string("""\
... From: mail-daemon@example.org
... To: test-bounces@example.com
... Message-ID: <second>
...
... """)

If no context is given, then a default one is used.

>>> event = processor.register(mlist, 'bart@example.com', msg)
>>> print(event.message_id)
<second>
>>> print(event.context)
BounceContext.normal

A probe bounce carries more weight than just a normal bounce.

>>> from mailman.interfaces.bounce import BounceContext
>>> event = processor.register(
...     mlist, 'bart@example.com', msg, BounceContext.probe)
>>> print(event.message_id)
<second>
>>> print(event.context)
BounceContext.probe
Domains

Domains are how Mailman interacts with email host names and web host names.

>>> from operator import attrgetter
>>> def show_domains(*, with_owners=False):
...     if len(manager) == 0:
...         print('no domains')
...         return
...     for domain in sorted(manager, key=attrgetter('mail_host')):
...         print(domain)
...     owners = sorted(owner.addresses[0].email
...                     for owner in domain.owners)
...     for owner in owners:
...         print('- owner:', owner)

>>> show_domains()
no domains

Adding a domain requires some basic information, of which the email host name is the only required piece. The other parts are inferred from that.

>>> manager.add('example.org')
<Domain example.org>
>>> show_domains()
<Domain example.org>

We can remove domains too.

>>> manager.remove('example.org')
<Domain example.org>
>>> show_domains()
no domains

Sometimes the email host name is different than the base url for hitting the web interface for the domain.

>>> manager.add('example.com')
<Domain example.com>
>>> show_domains()
<Domain example.com>

Domains can have explicit descriptions, and can be created with one or more owners.

>>> manager.add(
...     'example.net',
...     description='The example domain',
...     owners=['anne@example.com'])
<Domain example.net, The example domain>

>>> show_domains(with_owners=True)
<Domain example.com>
<Domain example.net, The example domain>
- owner: anne@example.com

Domains can have multiple owners, ideally one of the owners should have a verified preferred address. However this is not checked right now and the configuration’s default contact address may be used as a fallback.

>>> net_domain = manager['example.net']
>>> net_domain.add_owner('bart@example.org')
>>> show_domains(with_owners=True)
<Domain example.com>
<Domain example.net, The example domain>
- owner: anne@example.com
- owner: bart@example.org

Domains can list all associated mailing lists with the mailing_lists property.

>>> def show_lists(domain):
...     mlists = list(domain.mailing_lists)
...     for mlist in mlists:
...         print(mlist)
...     if len(mlists) == 0:
...         print('no lists')

>>> net_domain = manager['example.net']
>>> com_domain = manager['example.com']
>>> show_lists(net_domain)
no lists

>>> create_list('test@example.net')
<mailing list "test@example.net" at ...>
>>> transaction.commit()
>>> show_lists(net_domain)
<mailing list "test@example.net" at ...>

>>> show_lists(com_domain)
no lists

In the global domain manager, domains are indexed by their email host name.

>>> for domain in sorted(manager, key=attrgetter('mail_host')):
...     print(domain.mail_host)
example.com
example.net

>>> print(manager['example.net'])
<Domain example.net, The example domain>

As with dictionaries, you can also get the domain. If the domain does not exist, None or a default is returned.

>>> print(manager.get('example.net'))
<Domain example.net, The example domain>

>>> print(manager.get('doesnotexist.com'))
None

>>> print(manager.get('doesnotexist.com', 'blahdeblah'))
blahdeblah
Languages

Mailman is multilingual. A language manager handles the known set of languages at run time, as well as enabling those languages for use in a running Mailman instance.

>>> from mailman.interfaces.languages import ILanguageManager
>>> from zope.component import getUtility
>>> from zope.interface.verify import verifyObject

>>> mgr = getUtility(ILanguageManager)
>>> verifyObject(ILanguageManager, mgr)
True

# Make a copy of the language manager's dictionary, so we can restore it
# after the test.  Currently the test layer doesn't manage this.
>>> saved = mgr._languages.copy()

# The language manager component comes pre-populated; clear it out.
>>> mgr.clear()

A language manager keeps track of the languages it knows about.

>>> list(mgr.codes)
[]
>>> list(mgr.languages)
[]
Adding languages

Adding a new language requires three pieces of information, the 2-character language code, the English description of the language, and the character set used by the language. The language object is returned.

>>> mgr.add('en', 'us-ascii', 'English')
<Language [en] English>
>>> mgr.add('it', 'iso-8859-1', 'Italian')
<Language [it] Italian>

And you can get information for all known languages.

>>> print(mgr['en'].description)
English
>>> print(mgr['en'].charset)
us-ascii
>>> print(mgr['it'].description)
Italian
>>> print(mgr['it'].charset)
iso-8859-1
Other iterations

You can iterate over all the known language codes.

>>> mgr.add('pl', 'iso-8859-2', 'Polish')
<Language [pl] Polish>
>>> sorted(mgr.codes)
['en', 'it', 'pl']

You can iterate over all the known languages.

>>> from operator import attrgetter
>>> languages = sorted((language for language in mgr.languages),
...                    key=attrgetter('code'))
>>> for language in languages:
...     print(language.code, language.charset, language.description)
en us-ascii English
it iso-8859-1 Italian
pl iso-8859-2 Polish

You can ask whether a particular language code is known.

>>> 'it' in mgr
True
>>> 'xx' in mgr
False

You can get a particular language by its code.

>>> print(mgr['it'].description)
Italian
>>> print(mgr['xx'].code)
Traceback (most recent call last):
...
KeyError: 'xx'
>>> print(mgr.get('it').description)
Italian
>>> print(mgr.get('xx'))
None
>>> print(mgr.get('xx', 'missing'))
missing
Clearing the known languages

The language manager can forget about all the language codes it knows about.

>>> 'en' in mgr
True

>>> mgr.clear()
>>> 'en' in mgr
False

# Restore the data.
>>> mgr._languages = saved
The mailing list manager

The IListManager is how you create, delete, and retrieve mailing list objects.

>>> from mailman.interfaces.listmanager import IListManager
>>> from zope.component import getUtility
>>> list_manager = getUtility(IListManager)
Creating a mailing list

Creating the list returns the newly created IMailList object.

>>> from mailman.interfaces.mailinglist import IMailingList
>>> mlist = list_manager.create('test@example.com')
>>> IMailingList.providedBy(mlist)
True

All lists with identities have a short name, a host name, a fully qualified listname, and an RFC 2369 list id. This latter will not change even if the mailing list moves to a different host, so it is what uniquely distinguishes the mailing list to the system.

>>> print(mlist.list_name)
test
>>> print(mlist.mail_host)
example.com
>>> print(mlist.fqdn_listname)
test@example.com
>>> print(mlist.list_id)
test.example.com
Deleting a mailing list

Use the list manager to delete a mailing list.

>>> list_manager.delete(mlist)
>>> sorted(list_manager.names)
[]

After deleting the list, you can create it again.

>>> mlist = list_manager.create('test@example.com')
>>> print(mlist.fqdn_listname)
test@example.com
Retrieving a mailing list

When a mailing list exists, you can ask the list manager for it and you will always get the same object back.

>>> mlist_2 = list_manager.get('test@example.com')
>>> mlist_2 is mlist
True

You can also get a mailing list by it’s list id.

>>> mlist_2 = list_manager.get_by_list_id('test.example.com')
>>> mlist_2 is mlist
True

If you try to get a list that doesn’t existing yet, you get None.

>>> print(list_manager.get('test_2@example.com'))
None
>>> print(list_manager.get_by_list_id('test_2.example.com'))
None

You also get None if the list name is invalid.

>>> print(list_manager.get('foo'))
None
Iterating over all mailing lists

Once you’ve created a bunch of mailing lists, you can use the list manager to iterate over the mailing list objects, the list posting addresses, or the list address components.

>>> mlist_3 = list_manager.create('test_3@example.com')
>>> mlist_4 = list_manager.create('test_4@example.com')

>>> for name in sorted(list_manager.names):
...     print(name)
test@example.com
test_3@example.com
test_4@example.com

>>> for list_id in sorted(list_manager.list_ids):
...     print(list_id)
test.example.com
test_3.example.com
test_4.example.com

>>> for fqdn_listname in sorted(m.fqdn_listname
...                             for m in list_manager.mailing_lists):
...     print(fqdn_listname)
test@example.com
test_3@example.com
test_4@example.com

>>> for list_name, mail_host in sorted(list_manager.name_components):
...     print(list_name, '@', mail_host)
test   @ example.com
test_3 @ example.com
test_4 @ example.com
Mailing lists

The mailing list is a core object in Mailman. It is uniquely identified in the system by its list-id which is derived from its posting address, i.e. the email address you would send a message to in order to post a message to the mailing list. The list id is defined in RFC 2369.

>>> mlist = create_list('aardvark@example.com')
>>> print(mlist.list_id)
aardvark.example.com
>>> print(mlist.fqdn_listname)
aardvark@example.com

The mailing list also has convenient attributes for accessing the list’s short name (i.e. local part) and host name.

>>> print(mlist.list_name)
aardvark
>>> print(mlist.mail_host)
example.com
Rosters

Mailing list membership is represented by rosters. Each mailing list has several rosters of members, representing the subscribers to the mailing list, the owners, the moderators, and so on. The rosters are defined by a membership role.

Addresses can be explicitly subscribed to a mailing list. By default, a subscription puts the address in the member role, meaning that address will receive a copy of any message sent to the mailing list.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)

>>> aperson = user_manager.create_address('aperson@example.com')
>>> bperson = user_manager.create_address('bperson@example.com')
>>> mlist.subscribe(aperson)
<Member: aperson@example.com on aardvark@example.com as MemberRole.member>
>>> mlist.subscribe(bperson)
<Member: bperson@example.com on aardvark@example.com as MemberRole.member>

Both addresses appear on the roster of members.

>>> from operator import attrgetter
>>> sort_key = attrgetter('address.email')

>>> for member in sorted(mlist.members.members, key=sort_key):
...     print(member)
<Member: aperson@example.com on aardvark@example.com as MemberRole.member>
<Member: bperson@example.com on aardvark@example.com as MemberRole.member>

By explicitly specifying the role of the subscription, an address can be added to the owner and moderator rosters.

>>> from mailman.interfaces.member import MemberRole
>>> mlist.subscribe(aperson, MemberRole.owner)
<Member: aperson@example.com on aardvark@example.com as MemberRole.owner>
>>> cperson = user_manager.create_address('cperson@example.com')
>>> mlist.subscribe(cperson, MemberRole.owner)
<Member: cperson@example.com on aardvark@example.com as MemberRole.owner>
>>> mlist.subscribe(cperson, MemberRole.moderator)
<Member: cperson@example.com on aardvark@example.com
         as MemberRole.moderator>

A Person is now both a member and an owner of the mailing list. C Person is an owner and a moderator.

>>> for member in sorted(mlist.owners.members, key=sort_key):
...     print(member)
<Member: aperson@example.com on aardvark@example.com as MemberRole.owner>
<Member: cperson@example.com on aardvark@example.com as MemberRole.owner>

>>> for member in mlist.moderators.members:
...     print(member)
<Member: cperson@example.com on aardvark@example.com
         as MemberRole.moderator>

All rosters can also be accessed indirectly.

>>> roster = mlist.get_roster(MemberRole.member)
>>> for member in sorted(roster.members, key=sort_key):
...     print(member)
<Member: aperson@example.com on aardvark@example.com as MemberRole.member>
<Member: bperson@example.com on aardvark@example.com as MemberRole.member>

>>> roster = mlist.get_roster(MemberRole.owner)
>>> for member in sorted(roster.members, key=sort_key):
...     print(member)
<Member: aperson@example.com on aardvark@example.com as MemberRole.owner>
<Member: cperson@example.com on aardvark@example.com as MemberRole.owner>

>>> roster = mlist.get_roster(MemberRole.moderator)
>>> for member in roster.members:
...     print(member)
<Member: cperson@example.com on aardvark@example.com
         as MemberRole.moderator>
Subscribing users

An alternative way of subscribing to a mailing list is as a user with a preferred address. This way the user can change their subscription address just by changing their preferred address.

The user must have a preferred address.

>>> from mailman.utilities.datetime import now
>>> user = user_manager.create_user('dperson@example.com', 'Dave Person')
>>> address = list(user.addresses)[0]
>>> address.verified_on = now()
>>> user.preferred_address = address

The preferred address is used in the subscription.

>>> mlist.subscribe(user)
<Member: Dave Person <dperson@example.com> on aardvark@example.com
         as MemberRole.member>
>>> for member in sorted(mlist.members.members, key=sort_key):
...     print(member)
<Member: aperson@example.com on aardvark@example.com as MemberRole.member>
<Member: bperson@example.com on aardvark@example.com as MemberRole.member>
<Member: Dave Person <dperson@example.com> on aardvark@example.com
         as MemberRole.member>

If the user’s preferred address changes, their subscribed email address also changes automatically.

>>> new_address = user.register('dave.person@example.com')
>>> new_address.verified_on = now()
>>> user.preferred_address = new_address

>>> for member in sorted(mlist.members.members, key=sort_key):
...     print(member)
<Member: aperson@example.com on aardvark@example.com as MemberRole.member>
<Member: bperson@example.com on aardvark@example.com as MemberRole.member>
<Member: dave.person@example.com on aardvark@example.com
         as MemberRole.member>

A user is allowed to explicitly subscribe again with a specific address, even if this address is their preferred address.

>>> mlist.subscribe(user.preferred_address)
<Member: dave.person@example.com
         on aardvark@example.com as MemberRole.member>
List memberships

Users represent people in Mailman, members represent subscriptions. Users control email addresses, and rosters are collections of members. A member ties a subscribed email address to a role, such as member, administrator, or moderator. Even non-members are represented by a roster.

Roster sets are collections of rosters and a mailing list has a single roster set that contains all its members, regardless of that member’s role.

Mailing lists and roster sets have an indirect relationship, through the roster set’s name. Roster also have names, but are related to roster sets by a more direct containment relationship. This is because it is possible to store mailing list data in a different database than user data.

When we create a mailing list, it starts out with no members, owners, moderators, administrators, or nonmembers.

>>> mlist = create_list('ant@example.com')
>>> dump_list(mlist.members.members)
*Empty*
>>> dump_list(mlist.owners.members)
*Empty*
>>> dump_list(mlist.moderators.members)
*Empty*
>>> dump_list(mlist.administrators.members)
*Empty*
>>> dump_list(mlist.nonmembers.members)
*Empty*
Administrators

A mailing list’s administrators are defined as union of the list’s owners and moderators. We can add new owners or moderators to this list by assigning roles to users. First we have to create the user, because there are no users in the user database yet.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)
>>> user_1 = user_manager.create_user('aperson@example.com', 'Anne Person')
>>> print(user_1)
<User "Anne Person" (...) at ...>

We can add Anne as an owner of the mailing list, by creating a member role for her.

>>> from mailman.interfaces.member import MemberRole
>>> address_1 = list(user_1.addresses)[0]
>>> mlist.subscribe(address_1, MemberRole.owner)
<Member: Anne Person <aperson@example.com> on
         ant@example.com as MemberRole.owner>
>>> dump_list(member.address for member in mlist.owners.members)
Anne Person <aperson@example.com>

Adding Anne as a list owner also makes her an administrator, but does not make her a moderator. Nor does it make her a member of the list.

>>> dump_list(member.address for member in mlist.administrators.members)
Anne Person <aperson@example.com>
>>> dump_list(member.address for member in mlist.moderators.members)
*Empty*
>>> dump_list(member.address for member in mlist.members.members)
*Empty*

Bart becomes a moderator of the list.

>>> user_2 = user_manager.create_user('bperson@example.com', 'Bart Person')
>>> print(user_2)
<User "Bart Person" (...) at ...>
>>> address_2 = list(user_2.addresses)[0]
>>> mlist.subscribe(address_2, MemberRole.moderator)
<Member: Bart Person <bperson@example.com>
         on ant@example.com as MemberRole.moderator>
>>> dump_list(member.address for member in mlist.moderators.members)
Bart Person <bperson@example.com>

Now, both Anne and Bart are list administrators.

>>> from operator import attrgetter
>>> def dump_members(roster):
...     all_addresses = list(member.address for member in roster)
...     sorted_addresses = sorted(all_addresses, key=attrgetter('email'))
...     dump_list(sorted_addresses)

>>> dump_members(mlist.administrators.members)
Anne Person <aperson@example.com>
Bart Person <bperson@example.com>
Members

Similarly, list members are born of users being subscribed with the proper role.

>>> user_3 = user_manager.create_user(
...     'cperson@example.com', 'Cris Person')
>>> address_3 = list(user_3.addresses)[0]
>>> member = mlist.subscribe(address_3, MemberRole.member)
>>> member
<Member: Cris Person <cperson@example.com>
         on ant@example.com as MemberRole.member>

Cris’s user record can also be retrieved from her member record.

>>> member.user
<User "Cris Person" (3) at ...>

Cris will be a regular delivery member but not a digest member.

>>> dump_members(mlist.members.members)
Cris Person <cperson@example.com>
>>> dump_members(mlist.regular_members.members)
Cris Person <cperson@example.com>
>>> dump_members(mlist.digest_members.addresses)
*Empty*

It’s easy to make the list administrators members of the mailing list too.

>>> members = []
>>> for address in mlist.administrators.addresses:
...     member = mlist.subscribe(address, MemberRole.member)
...     members.append(member)
>>> dump_list(members, key=attrgetter('address.email'))
<Member: Anne Person <aperson@example.com> on
         ant@example.com as MemberRole.member>
<Member: Bart Person <bperson@example.com> on
         ant@example.com as MemberRole.member>
>>> dump_members(mlist.members.members)
Anne Person <aperson@example.com>
Bart Person <bperson@example.com>
Cris Person <cperson@example.com>
>>> dump_members(mlist.regular_members.members)
Anne Person <aperson@example.com>
Bart Person <bperson@example.com>
Cris Person <cperson@example.com>
>>> dump_members(mlist.digest_members.members)
*Empty*
Nonmembers

Nonmembers are used to represent people who have posted to the mailing list but are not subscribed to the mailing list. These may be legitimate users who have found the mailing list and wish to interact without a direct subscription, or they may be spammers who should never be allowed to contact the mailing list. Because all the same moderation rules can be applied to nonmembers, we represent them as the same type of object but with a different role.

>>> user_6 = user_manager.create_user('fperson@example.com', 'Fred Person')
>>> address_6 = list(user_6.addresses)[0]
>>> member_6 = mlist.subscribe(address_6, MemberRole.nonmember)
>>> member_6
<Member: Fred Person <fperson@example.com> on ant@example.com
         as MemberRole.nonmember>
>>> dump_members(mlist.nonmembers.members)
Fred Person <fperson@example.com>

Nonmembers do not get delivery of any messages.

>>> dump_members(mlist.members.members)
Anne Person <aperson@example.com>
Bart Person <bperson@example.com>
Cris Person <cperson@example.com>
>>> dump_members(mlist.regular_members.members)
Anne Person <aperson@example.com>
Bart Person <bperson@example.com>
Cris Person <cperson@example.com>
>>> dump_members(mlist.digest_members.members)
*Empty*
Finding members

You can find the IMember object that is a member of a roster for a given text email address by using the IRoster.get_member() method.

>>> mlist.owners.get_member('aperson@example.com')
<Member: Anne Person <aperson@example.com> on
         ant@example.com as MemberRole.owner>
>>> mlist.administrators.get_member('aperson@example.com')
<Member: Anne Person <aperson@example.com> on
         ant@example.com as MemberRole.owner>
>>> mlist.members.get_member('aperson@example.com')
<Member: Anne Person <aperson@example.com> on
         ant@example.com as MemberRole.member>
>>> mlist.nonmembers.get_member('fperson@example.com')
<Member: Fred Person <fperson@example.com> on
         ant@example.com as MemberRole.nonmember>

However, if the address is not subscribed with the appropriate role, then None is returned.

>>> print(mlist.administrators.get_member('zperson@example.com'))
None
>>> print(mlist.moderators.get_member('aperson@example.com'))
None
>>> print(mlist.members.get_member('zperson@example.com'))
None
>>> print(mlist.nonmembers.get_member('aperson@example.com'))
None
All subscribers

There is also a roster containing all the subscribers of a mailing list, regardless of their role.

>>> def sortkey(member):
...     return (member.address.email, member.role.value)
>>> for member in sorted(mlist.subscribers.members, key=sortkey):
...     print(member.address.email, member.role)
aperson@example.com MemberRole.member
aperson@example.com MemberRole.owner
bperson@example.com MemberRole.member
bperson@example.com MemberRole.moderator
cperson@example.com MemberRole.member
fperson@example.com MemberRole.nonmember
Subscriber type

Members can be subscribed to a mailing list either via an explicit address, or indirectly through a user’s preferred address. Sometimes you want to know which one it is.

Herb subscribes to the mailing list via an explicit address.

>>> herb = user_manager.create_address(
...     'hperson@example.com', 'Herb Person')
>>> herb_member = mlist.subscribe(herb)

Iris subscribes to the mailing list via her preferred address.

>>> iris = user_manager.make_user(
...     'iperson@example.com', 'Iris Person')
>>> preferred = list(iris.addresses)[0]
>>> from mailman.utilities.datetime import now
>>> preferred.verified_on = now()
>>> iris.preferred_address = preferred
>>> iris_member = mlist.subscribe(iris)

When we need to know which way a member is subscribed, we can look at the this attribute.

>>> herb_member.subscriber
<Address: Herb Person <hperson@example.com> [not verified] at ...>
>>> iris_member.subscriber
<User "Iris Person" (5) at ...>
Moderation actions

All members of any role have a moderation action which specifies how postings from that member are handled. By default, owners and moderators are automatically accepted for posting to the mailing list.

>>> for member in sorted(mlist.administrators.members,
...                      key=attrgetter('address.email')):
...     print(member.address.email, member.role, member.moderation_action)
aperson@example.com MemberRole.owner     Action.accept
bperson@example.com MemberRole.moderator Action.accept

By default, members and nonmembers have their action set to None, meaning that the mailing list’s default_member_action or default_nonmember_action will be used.

>>> for member in sorted(mlist.members.members,
...                      key=attrgetter('address.email')):
...     print(member.address.email, member.role, member.moderation_action)
aperson@example.com MemberRole.member None
bperson@example.com MemberRole.member None
cperson@example.com MemberRole.member None
hperson@example.com MemberRole.member None
iperson@example.com MemberRole.member None

>>> for member in mlist.nonmembers.members:
...     print(member.address.email, member.role, member.moderation_action)
fperson@example.com MemberRole.nonmember None

The mailing list’s default action for members is deferred, which specifies that the posting should go through the normal moderation checks. Its default action for nonmembers is to hold for moderator approval.

Changing subscriptions

When a user is subscribed to a mailing list via a specific address they control (as opposed to being subscribed with their preferred address), they can change their delivery address by setting the appropriate parameter. Note though that the address they’re changing to must be verified.

>>> bee = create_list('bee@example.com')
>>> gwen = user_manager.create_user('gwen@example.com')
>>> gwen_address = list(gwen.addresses)[0]
>>> gwen_member = bee.subscribe(gwen_address)
>>> for m in bee.members.members:
...     print(m.member_id.int, m.mailing_list.list_id, m.address.email)
9 bee.example.com gwen@example.com

Gwen gets a email address.

>>> new_address = gwen.register('gperson@example.com')

Gwen verifies her email address, and updates her membership.

>>> from mailman.utilities.datetime import now
>>> new_address.verified_on = now()
>>> gwen_member.address = new_address

Now her membership reflects the new address.

>>> for m in bee.members.members:
...     print(m.member_id.int, m.mailing_list.list_id, m.address.email)
9 bee.example.com gperson@example.com
Events

An event is triggered when a new member is subscribed to a mailing list.

>>> from mailman.testing.helpers import event_subscribers
>>> def handle_event(event):
...     print(event)

>>> cat = create_list('cat@example.com')
>>> herb = user_manager.create_address('herb@example.com')
>>> with event_subscribers(handle_event):
...     member = cat.subscribe(herb)
herb@example.com joined cat.example.com

An event is triggered when a member is unsubscribed from a mailing list.

>>> with event_subscribers(handle_event):
...     member.unsubscribe()
herb@example.com left cat.example.com
The message store

The message store is a collection of messages keyed off of Message-ID and X-Message-ID-Hash headers. Either of these values can be combined with the message’s List-Archive header to create a globally unique URI to the message object in the internet facing interface of the message store. The X-Message-ID-Hash is the base-32 SHA1 hash of the Message-ID.

>>> from mailman.interfaces.messages import IMessageStore
>>> from zope.component import getUtility
>>> message_store = getUtility(IMessageStore)

A message with a Message-ID header can be stored.

>>> msg = message_from_string("""\
... Subject: An important message
... Message-ID: <87myycy5eh.fsf@uwakimon.sk.tsukuba.ac.jp>
...
... This message is very important.
... """)
>>> x_message_id_hash = message_store.add(msg)
>>> print(x_message_id_hash)
JJIGKPKB6CVDX6B2CUG4IHAJRIQIOUTP
>>> print(msg.as_string())
Subject: An important message
Message-ID: <87myycy5eh.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID-Hash: JJIGKPKB6CVDX6B2CUG4IHAJRIQIOUTP
X-Message-ID-Hash: JJIGKPKB6CVDX6B2CUG4IHAJRIQIOUTP
<BLANKLINE>
This message is very important.
<BLANKLINE>
Finding messages

There are several ways to find a message given either the Message-ID or X-Message-ID-Hash headers. In either case, if no matching message is found, None is returned.

>>> print(message_store.get_message_by_id('nothing'))
None
>>> print(message_store.get_message_by_hash('nothing'))
None

Given an existing Message-ID, the message can be found.

>>> message = message_store.get_message_by_id(msg['message-id'])
>>> print(message.as_string())
Subject: An important message
Message-ID: <87myycy5eh.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID-Hash: JJIGKPKB6CVDX6B2CUG4IHAJRIQIOUTP
X-Message-ID-Hash: JJIGKPKB6CVDX6B2CUG4IHAJRIQIOUTP
<BLANKLINE>
This message is very important.
<BLANKLINE>

Similarly, we can find messages by the X-Message-ID-Hash:

>>> message = message_store.get_message_by_hash(msg['x-message-id-hash'])
>>> print(message.as_string())
Subject: An important message
Message-ID: <87myycy5eh.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID-Hash: JJIGKPKB6CVDX6B2CUG4IHAJRIQIOUTP
X-Message-ID-Hash: JJIGKPKB6CVDX6B2CUG4IHAJRIQIOUTP
<BLANKLINE>
This message is very important.
<BLANKLINE>
Iterating over all messages

The message store provides a means to iterate over all the messages it contains.

>>> messages = list(message_store.messages)
>>> len(messages)
1
>>> print(messages[0].as_string())
Subject: An important message
Message-ID: <87myycy5eh.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID-Hash: JJIGKPKB6CVDX6B2CUG4IHAJRIQIOUTP
X-Message-ID-Hash: JJIGKPKB6CVDX6B2CUG4IHAJRIQIOUTP
<BLANKLINE>
This message is very important.
<BLANKLINE>
Deleting messages from the store

You delete a message from the storage service by providing the Message-ID for the message you want to delete.

>>> message_id = message['message-id']
>>> message_store.delete_message(message_id)
>>> list(message_store.messages)
[]
>>> print(message_store.get_message_by_id(message_id))
None
>>> print(message_store.get_message_by_hash(message['x-message-id-hash']))
None
Mailing list addresses

Every mailing list has a number of addresses which are publicly available. These are defined in the IMailingListAddresses interface.

>>> mlist = create_list('_xtest@example.com')

The posting address is where people send messages to be posted to the mailing list. This is exactly the same as the fully qualified list name.

>>> print(mlist.fqdn_listname)
_xtest@example.com
>>> print(mlist.posting_address)
_xtest@example.com

Messages to the mailing list’s no reply address always get discarded without prejudice.

>>> print(mlist.no_reply_address)
noreply@example.com

The mailing list’s owner address reaches the human moderators.

>>> print(mlist.owner_address)
_xtest-owner@example.com

The request address goes to the list’s email command robot.

>>> print(mlist.request_address)
_xtest-request@example.com

The bounces address accepts and processes all potential bounces.

>>> print(mlist.bounces_address)
_xtest-bounces@example.com

The join (a.k.a. subscribe) address is where someone can email to get added to the mailing list. The subscribe alias is a synonym for join, but it’s deprecated.

>>> print(mlist.join_address)
_xtest-join@example.com
>>> print(mlist.subscribe_address)
_xtest-subscribe@example.com

The leave (a.k.a. unsubscribe) address is where someone can email to get removed from the mailing list. The unsubscribe alias is a synonym for leave, but it’s deprecated.

>>> print(mlist.leave_address)
_xtest-leave@example.com
>>> print(mlist.unsubscribe_address)
_xtest-unsubscribe@example.com
Email confirmations

Email confirmation messages are sent when actions such as subscriptions need to be confirmed. It requires that a cookie be provided, which will be included in the local part of the email address. The exact format of this is dependent on the verp_confirm_format configuration variable.

>>> print(mlist.confirm_address('cookie'))
_xtest-confirm+cookie@example.com
>>> print(mlist.confirm_address('wookie'))
_xtest-confirm+wookie@example.com

>>> config.push('test config', """
... [mta]
... verp_confirm_format: $address---$cookie
... """)
>>> print(mlist.confirm_address('cookie'))
_xtest-confirm---cookie@example.com
>>> config.pop('test config')
The pending database

The pending database is where various types of events which need confirmation are stored. These can include email address registration events, held messages (but only for user confirmation), auto-approvals, and probe bounces. This is not where messages held for administrator approval are kept.

In order to pend an event, you first need a pending database.

>>> from mailman.interfaces.pending import IPendings
>>> from zope.component import getUtility
>>> pendingdb = getUtility(IPendings)

There are nothing in the pendings database.

>>> pendingdb.count
0

The pending database can add any IPendable to the database, returning a token that can be used in urls and such.

>>> from zope.interface import implementer
>>> from mailman.interfaces.pending import IPendable
>>> @implementer(IPendable)
... class SimplePendable(dict):
...     PEND_TYPE = 'subscription'

>>> subscription = SimplePendable(
...     address='aperson@example.com',
...     display_name='Anne Person',
...     language='en',
...     password='xyz')
>>> token = pendingdb.add(subscription)
>>> len(token)
40

There’s exactly one entry in the pendings database now.

>>> pendingdb.count
1

You can confirm the pending, which means returning the IPendable structure (as a dictionary) from the database that matches the token.

All IPendable classes have a PEND_TYPE attribute which must be a string. It is used to identify and query pendables in the database, and will be returned as the type key in the dictionary. Thus type is a reserved key and pendables may not otherwise set it.

If the token isn’t in the database, None is returned.

>>> pendable = pendingdb.confirm(b'missing')
>>> print(pendable)
None
>>> pendable = pendingdb.confirm(token)
>>> dump_msgdata(pendable)
address     : aperson@example.com
display_name: Anne Person
language    : en
password    : xyz
type        : subscription

After confirmation, the token is no longer in the database.

>>> print(pendingdb.confirm(token))
None

There are a few other things you can do with the pending database. When you confirm a token, you can leave it in the database, or in other words, not expunge it.

>>> event_1 = SimplePendable(type='one')
>>> token_1 = pendingdb.add(event_1)
>>> event_2 = SimplePendable(type='two')
>>> token_2 = pendingdb.add(event_2)
>>> event_3 = SimplePendable(type='three')
>>> token_3 = pendingdb.add(event_3)
>>> pendable = pendingdb.confirm(token_1, expunge=False)
>>> dump_msgdata(pendable)
type: one
>>> pendable = pendingdb.confirm(token_1, expunge=True)
>>> dump_msgdata(pendable)
type: one
>>> print(pendingdb.confirm(token_1))
None

You can iterate over all the pendings in the database.

>>> pendables = list(pendingdb)
>>> def sort_key(item):
...     token, pendable = item
...     return pendable['type']
>>> sorted_pendables = sorted(pendables, key=sort_key)
>>> for token, pendable in sorted_pendables:
...     print(pendable['type'])
three
two

An event can be given a lifetime when it is pended, otherwise it just uses a default lifetime.

>>> from datetime import timedelta
>>> yesterday = timedelta(days=-1)
>>> event_4 = SimplePendable(type='four')
>>> token_4 = pendingdb.add(event_4, lifetime=yesterday)

Every once in a while the pending database is cleared of old records.

>>> pendingdb.evict()
>>> print(pendingdb.confirm(token_4))
None
>>> pendable = pendingdb.confirm(token_2)
>>> dump_msgdata(pendable)
type: two
Moderator requests

Various actions will be held for moderator approval, such as subscriptions to closed lists, or postings by non-members. The requests database is the low level interface to these actions requiring approval.

An application level interface for holding messages and membership changes is also available.

Mailing list-centric

A set of requests are always related to a particular mailing list. Adapt the mailing list to get its requests.

>>> from mailman.interfaces.requests import IListRequests
>>> from zope.interface.verify import verifyObject

>>> mlist = create_list('test@example.com')
>>> requests = IListRequests(mlist)
>>> verifyObject(IListRequests, requests)
True
>>> requests.mailing_list
<mailing list "test@example.com" at ...>
Holding requests

The list’s requests database starts out empty.

>>> print(requests.count)
0
>>> dump_list(requests.held_requests)
*Empty*

At the lowest level, the requests database is very simple. Holding a request requires a request type (as an enum value), a key, and an optional dictionary of associated data. The request database assigns no semantics to the held data, except for the request type.

>>> from mailman.interfaces.requests import RequestType

We can hold messages for moderator approval.

>>> requests.hold_request(RequestType.held_message, 'hold_1')
1

We can hold subscription requests for moderator approval.

>>> requests.hold_request(RequestType.subscription, 'hold_2')
2

We can hold unsubscription requests for moderator approval.

>>> requests.hold_request(RequestType.unsubscription, 'hold_3')
3
Getting requests

We can see the total number of requests being held.

>>> print(requests.count)
3

We can also see the number of requests being held by request type.

>>> print(requests.count_of(RequestType.subscription))
1
>>> print(requests.count_of(RequestType.unsubscription))
1

We can also see when there are multiple held requests of a particular type.

>>> print(requests.hold_request(RequestType.held_message, 'hold_4'))
4
>>> print(requests.count_of(RequestType.held_message))
2

We can ask the requests database for a specific request, by providing the id of the request data we want. This returns a 2-tuple of the key and data we originally held.

>>> key, data = requests.get_request(2)
>>> print(key)
hold_2

There was no additional data associated with request 2.

>>> print(data)
None

If we ask for a request that is not in the database, we get None back.

>>> print(requests.get_request(801))
None
Additional data

When a request is held, additional data can be associated with it, in the form of a dictionary with string values.

>>> data = dict(foo='yes', bar='no')
>>> requests.hold_request(RequestType.held_message, 'hold_5', data)
5

The data is returned when the request is retrieved. The dictionary will have an additional key which holds the name of the request type.

>>> key, data = requests.get_request(5)
>>> print(key)
hold_5
>>> dump_msgdata(data)
_request_type: held_message
bar          : no
foo          : yes
type         : data
Iterating over requests

To make it easier to find specific requests, the list requests can be iterated over by type.

>>> print(requests.count_of(RequestType.held_message))
3
>>> for request in requests.of_type(RequestType.held_message):
...     key, data = requests.get_request(request.id)
...     print(request.id, request.request_type, key)
...     if data is not None:
...         for key in sorted(data):
...             print('    {0}: {1}'.format(key, data[key]))
1 RequestType.held_message hold_1
4 RequestType.held_message hold_4
5 RequestType.held_message hold_5
    _request_type: held_message
    bar: no
    foo: yes
    type: data
Deleting requests

Once a specific request has been handled, it can be deleted from the requests database.

>>> print(requests.count)
5
>>> requests.delete_request(2)
>>> print(requests.count)
4

Request 2 is no longer in the database.

>>> print(requests.get_request(2))
None

>>> for request in requests.held_requests:
...     requests.delete_request(request.id)
>>> print(requests.count)
0
Subscriptions

When a user wants to join a mailing list, they must register and verify their email address. Then depending on how the mailing list is configured, they may need to confirm their subscription and have it approved by the list moderator. The ISubscriptionManager interface manages this work flow.

>>> from mailman.interfaces.subscriptions import ISubscriptionManager

To begin, adapt a mailing list to an ISubscriptionManager. This is a named interface because the same interface manages both subscriptions and unsubscriptions.

>>> mlist = create_list('ant@example.com')
>>> manager = ISubscriptionManager(mlist)

Either addresses or users with a preferred address can be registered.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> anne = getUtility(IUserManager).create_address(
...     'anne@example.com', 'Anne Person')
Subscribing an email address

The subscription process requires that the email address be verified. It may also require confirmation or moderator approval depending on the mailing list’s subscription policy. For example, an open subscription policy does not require confirmation or approval, but the email address must still be verified, and the process will pause until these steps are completed.

>>> from mailman.interfaces.mailinglist import SubscriptionPolicy
>>> mlist.subscription_policy = SubscriptionPolicy.open

Anne attempts to join the mailing list. A unique token is created which represents this work flow.

>>> token, token_owner, member = manager.register(anne)

Because her email address has not yet been verified, she has not yet become a member of the mailing list.

>>> print(member)
None
>>> print(mlist.members.get_member('anne@example.com'))
None

Once she verifies her email address, she will become a member of the mailing list. When the subscription policy requires confirmation, the verification process implies that she also confirms her wish to join the mailing list.

>>> token, token_owner, member = manager.confirm(token)
>>> member
<Member: Anne Person <anne@example.com> on ant@example.com
    as MemberRole.member>
>>> mlist.members.get_member('anne@example.com')
<Member: Anne Person <anne@example.com> on ant@example.com
    as MemberRole.member>
Subscribing a user

Users can also register, but they must have a preferred address. The mailing list will deliver messages to this preferred address.

>>> bart = getUtility(IUserManager).make_user(
...     'bart@example.com', 'Bart Person')

Bart verifies his address and makes it his preferred address.

>>> from mailman.utilities.datetime import now
>>> preferred = list(bart.addresses)[0]
>>> preferred.verified_on = now()
>>> bart.preferred_address = preferred

The mailing list’s subscription policy does not require Bart to confirm his subscription, but the moderate does want to approve all subscriptions.

>>> mlist.subscription_policy = SubscriptionPolicy.moderate

Now when Bart registers as a user for the mailing list, a token will still be generated, but this is only used by the moderator. At first, Bart is not subscribed to the mailing list.

>>> token, token_owner, member = manager.register(bart)
>>> print(member)
None
>>> print(mlist.members.get_member('bart@example.com'))
None

When the moderator confirms Bart’s subscription, he joins the mailing list.

>>> token, token_owner, member = manager.confirm(token)
>>> member
<Member: Bart Person <bart@example.com> on ant@example.com
    as MemberRole.member>
>>> mlist.members.get_member('bart@example.com')
<Member: Bart Person <bart@example.com> on ant@example.com
    as MemberRole.member>
Unsubscribing

Similarly, unsubscribing a user depends on the mailing list’s unsubscription policy. Of course, since the address or user is already subscribed, implying that their email address is already verified, that step is not required. To begin with unsubscribing, you need to adapt the mailing list to the same interface, but with a different name.

>>> manager = ISubscriptionManager(mlist)

If the mailing list’s unsubscription policy is open, unregistering the subscription takes effect immediately.

>>> mlist.unsubscription_policy = SubscriptionPolicy.open
>>> token, token_owner, member = manager.unregister(anne)
>>> print(mlist.members.get_member('anne@example.com'))
None

Usually though, the member must confirm their unsubscription request, to prevent an attacker from unsubscribing them from the list without their knowledge.

>>> mlist.unsubscription_policy = SubscriptionPolicy.confirm
>>> token, token_owner, member = manager.unregister(bart)

Bart hasn’t confirmed yet, so he’s still a member of the list.

>>> mlist.members.get_member('bart@example.com')
<Member: Bart Person <bart@example.com> on ant@example.com
    as MemberRole.member>

Once Bart confirms, he’s unsubscribed from the mailing list.

>>> token, token_owner, member = manager.confirm(token)
>>> print(mlist.members.get_member('bart@example.com'))
None
Subscription services

The ISubscriptionService utility provides higher level convenience methods useful for searching, retrieving, iterating, and removing memberships across all mailing lists on the system.

>>> from mailman.interfaces.subscriptions import ISubscriptionService
>>> service = getUtility(ISubscriptionService)

You can use the service to get all members of all mailing lists, for any membership role. At first, there are no memberships.

>>> service.get_members()
[]
>>> sum(1 for member in service)
0
>>> from uuid import UUID
>>> print(service.get_member(UUID(int=801)))
None
Listing members

When there are some members, of any role on any mailing list, they can be retrieved through the subscription service.

>>> from mailman.app.lifecycle import create_list
>>> ant = mlist
>>> bee = create_list('bee@example.com')
>>> cat = create_list('cat@example.com')

Some people become members.

>>> from mailman.interfaces.member import MemberRole
>>> from mailman.testing.helpers import subscribe
>>> anne_1 = subscribe(ant, 'Anne')
>>> anne_2 = subscribe(ant, 'Anne', MemberRole.owner)
>>> bart_1 = subscribe(ant, 'Bart', MemberRole.moderator)
>>> bart_2 = subscribe(bee, 'Bart', MemberRole.owner)
>>> anne_3 = subscribe(cat, 'Anne', email='anne@example.com')
>>> cris_1 = subscribe(cat, 'Cris')

The service can be used to iterate over them.

>>> for member in service.get_members():
...     print(member)
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.owner>
<Member: Bart Person <bperson@example.com>
    on ant@example.com as MemberRole.moderator>
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.member>
<Member: Bart Person <bperson@example.com>
    on bee@example.com as MemberRole.owner>
<Member: Anne Person <anne@example.com>
    on cat@example.com as MemberRole.member>
<Member: Cris Person <cperson@example.com>
    on cat@example.com as MemberRole.member>

The service can also be used to get the information about a single member.

>>> print(service.get_member(bart_2.member_id))
<Member: Bart Person <bperson@example.com>
    on bee@example.com as MemberRole.owner>

There is an iteration shorthand for getting all the members.

>>> for member in service:
...     print(member)
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.owner>
<Member: Bart Person <bperson@example.com>
    on ant@example.com as MemberRole.moderator>
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.member>
<Member: Bart Person <bperson@example.com>
    on bee@example.com as MemberRole.owner>
<Member: Anne Person <anne@example.com>
    on cat@example.com as MemberRole.member>
<Member: Cris Person <cperson@example.com>
    on cat@example.com as MemberRole.member>
Searching for members

The subscription service can be used to find memberships based on specific search criteria. For example, we can find all the mailing lists that Anne is a member of with her aperson@example.com address.

>>> for member in service.find_members('aperson@example.com'):
...     print(member)
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.member>
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.owner>

There may be no matching memberships.

>>> list(service.find_members('dave@example.com'))
[]

The address may contain asterisks, which will be interpreted as a wildcard in the search pattern.

>>> for member in service.find_members('*person*'):
...     print(member)
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.member>
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.owner>
<Member: Bart Person <bperson@example.com>
    on ant@example.com as MemberRole.moderator>
<Member: Bart Person <bperson@example.com>
    on bee@example.com as MemberRole.owner>
<Member: Cris Person <cperson@example.com>
    on cat@example.com as MemberRole.member>

Memberships can also be searched for by user id.

>>> for member in service.find_members(anne_1.user.user_id):
...     print(member)
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.member>
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.owner>

You can find all the memberships for a specific mailing list.

>>> for member in service.find_members(list_id='ant.example.com'):
...     print(member)
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.member>
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.owner>
<Member: Bart Person <bperson@example.com>
    on ant@example.com as MemberRole.moderator>

You can find all the memberships for an address on a specific mailing list, but you have to give it the list id, not the fqdn listname since the former is stable but the latter could change if the list is moved.

>>> for member in service.find_members(
...         'bperson@example.com', 'ant.example.com'):
...     print(member)
<Member: Bart Person <bperson@example.com>
    on ant@example.com as MemberRole.moderator>

You can find all the memberships for an address with a specific role.

>>> for member in service.find_members(
...         list_id='ant.example.com', role=MemberRole.owner):
...     print(member)
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.owner>

You can also find a specific membership by all three criteria.

>>> for member in service.find_members(
...         'bperson@example.com', 'bee.example.com', MemberRole.owner):
...     print(member)
<Member: Bart Person <bperson@example.com>
    on bee@example.com as MemberRole.owner>
Finding a single member

If you expect only zero or one member to match your criteria, you can use a the more efficient find_member() method. This takes exactly the same criteria as find_members().

There may be no matching members.

>>> print(service.find_member('dave@example.com'))
None

But if there is exactly one membership, it is returned.

>>> service.find_member('bperson@example.com', 'ant.example.com')
<Member: Bart Person <bperson@example.com>
    on ant@example.com as MemberRole.moderator>
Removing members

Members can be removed via this service.

>>> len(service.get_members())
6
>>> service.leave('cat.example.com', 'cperson@example.com')
>>> len(service.get_members())
5
>>> for member in service:
...     print(member)
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.owner>
<Member: Bart Person <bperson@example.com>
    on ant@example.com as MemberRole.moderator>
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.member>
<Member: Bart Person <bperson@example.com>
    on bee@example.com as MemberRole.owner>
<Member: Anne Person <anne@example.com>
    on cat@example.com as MemberRole.member>
Mass Removal

The subscription service can be used to perform mass removals. You are required to pass the list id of the respective mailing list and a list of email addresses to be removed.

>>> bart_2 = subscribe(ant, 'Bart')
>>> cris_2 = subscribe(ant, 'Cris')
>>> for member in service:
...     print(member)
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.owner>
<Member: Bart Person <bperson@example.com>
    on ant@example.com as MemberRole.moderator>
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.member>
<Member: Bart Person <bperson@example.com>
    on ant@example.com as MemberRole.member>
<Member: Cris Person <cperson@example.com>
    on ant@example.com as MemberRole.member>
<Member: Bart Person <bperson@example.com>
    on bee@example.com as MemberRole.owner>
<Member: Anne Person <anne@example.com>
    on cat@example.com as MemberRole.member>

There are now two more memberships.

>>> len(service.get_members())
7

But this address is not subscribed to any mailing list.

>>> print(service.find_member('bogus@example.com'))
None

We can unsubscribe some addresses from the ant mailing list. Note that even though Anne is subscribed several times, only her ant membership with role member will be removed.

>>> success, fail = service.unsubscribe_members(
...     'ant.example.com', [
...         'aperson@example.com',
...         'cperson@example.com',
...         'bogus@example.com',
...         ])

There were some successes…

>>> dump_list(success)
aperson@example.com
cperson@example.com

…and some failures.

>>> dump_list(fail)
bogus@example.com

And now there are 5 memberships again.

>>> for member in service:
...     print(member)
<Member: Anne Person <aperson@example.com>
    on ant@example.com as MemberRole.owner>
<Member: Bart Person <bperson@example.com>
    on ant@example.com as MemberRole.moderator>
<Member: Bart Person <bperson@example.com>
    on ant@example.com as MemberRole.member>
<Member: Bart Person <bperson@example.com>
    on bee@example.com as MemberRole.owner>
<Member: Anne Person <anne@example.com>
    on cat@example.com as MemberRole.member>
>>> len(service.get_members())
5
The user manager

The IUserManager is how you create, delete, and manage users.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)
Creating users

There are several ways you can create a user object. The simplest is to create a blank user by not providing an address or real name at creation time. This user will have an empty string as their real name, but will not have a password.

>>> from mailman.interfaces.user import IUser
>>> from zope.interface.verify import verifyObject
>>> user = user_manager.create_user()
>>> verifyObject(IUser, user)
True

>>> dump_list(address.email for address in user.addresses)
*Empty*
>>> print(user.display_name)

>>> print(user.password)
None

The user has preferences, but none of them will be specified.

>>> print(user.preferences)
<Preferences ...>

A user can be assigned a real name.

>>> user.display_name = 'Anne Person'
>>> dump_list(user.display_name for user in user_manager.users)
Anne Person

A user can be assigned a password.

>>> user.password = 'secret'
>>> dump_list(user.password for user in user_manager.users)
secret

You can also create a user with an address to start out with.

>>> user_2 = user_manager.create_user('bperson@example.com')
>>> verifyObject(IUser, user_2)
True
>>> dump_list(address.email for address in user_2.addresses)
bperson@example.com
>>> dump_list(user.display_name for user in user_manager.users)
<BLANKLINE>
Anne Person

As above, you can assign a real name to such users.

>>> user_2.display_name = 'Ben Person'
>>> dump_list(user.display_name for user in user_manager.users)
Anne Person
Ben Person

You can also create a user with just a real name.

>>> user_3 = user_manager.create_user(display_name='Claire Person')
>>> verifyObject(IUser, user_3)
True
>>> dump_list(address.email for address in user.addresses)
*Empty*
>>> dump_list(user.display_name for user in user_manager.users)
Anne Person
Ben Person
Claire Person

Finally, you can create a user with both an address and a real name.

>>> user_4 = user_manager.create_user('dperson@example.com', 'Dan Person')
>>> verifyObject(IUser, user_3)
True
>>> dump_list(address.email for address in user_4.addresses)
dperson@example.com
>>> dump_list(address.display_name for address in user_4.addresses)
Dan Person
>>> dump_list(user.display_name for user in user_manager.users)
Anne Person
Ben Person
Claire Person
Dan Person
Deleting users

You delete users by going through the user manager. The deleted user is no longer available through the user manager iterator.

>>> user_manager.delete_user(user)
>>> dump_list(user.display_name for user in user_manager.users)
Ben Person
Claire Person
Dan Person
Finding users

You can ask the user manager to find the IUser that controls a particular email address. You’ll get back the original user object if it’s found. Note that the .get_user() method takes a string email address, not an IAddress object.

>>> address = list(user_4.addresses)[0]
>>> found_user = user_manager.get_user(address.email)
>>> found_user
<User "Dan Person" (...) at ...>
>>> found_user is user_4
True

If the address is not in the user database or does not have a user associated with it, you will get None back.

>>> print(user_manager.get_user('zperson@example.com'))
None
>>> user_4.unlink(address)
>>> print(user_manager.get_user(address.email))
None

Users can also be found by their unique user id.

>>> found_user = user_manager.get_user_by_id(user_4.user_id)
>>> user_4
<User "Dan Person" (...) at ...>
>>> found_user
<User "Dan Person" (...) at ...>
>>> user_4.user_id == found_user.user_id
True

If a non-existent user id is given, None is returned.

>>> from uuid import UUID
>>> print(user_manager.get_user_by_id(UUID(int=801)))
None
Finding all members

The user manager can return all the members known to the system.

>>> mlist = create_list('test@example.com')
>>> mlist.subscribe(list(user_2.addresses)[0])
<Member: bperson@example.com on test@example.com as MemberRole.member>
>>> mlist.subscribe(user_manager.create_address('eperson@example.com'))
<Member: eperson@example.com on test@example.com as MemberRole.member>
>>> mlist.subscribe(user_manager.create_address('fperson@example.com'))
<Member: fperson@example.com on test@example.com as MemberRole.member>

Bart is also the owner of the mailing list.

>>> from mailman.interfaces.member import MemberRole
>>> mlist.subscribe(list(user_2.addresses)[0], MemberRole.owner)
<Member: bperson@example.com on test@example.com as MemberRole.owner>

There are now four members in the system. Sort them by address then role.

>>> def sort_key(member):
...     return (member.address.email, member.role.name)
>>> members = sorted(user_manager.members, key=sort_key)
>>> for member in members:
...     print(member.mailing_list.list_id, member.address.email,
...           member.role)
test.example.com bperson@example.com MemberRole.member
test.example.com bperson@example.com MemberRole.owner
test.example.com eperson@example.com MemberRole.member
test.example.com fperson@example.com MemberRole.member
Creating a new user

A common situation (especially during the subscription life cycle) is to create a user linked to an address, with a preferred address. Say for example, we are asked to subscribe a new address we have never seen before.

>>> cris = user_manager.make_user('cris@example.com', 'Cris Person')

Since we’ve never seen cris@example.com before, this call creates a new user with the given email and display name.

>>> cris
<User "Cris Person" (5) at ...>

The user has a single unverified address object.

>>> for address in cris.addresses:
...     print(repr(address))
<Address: Cris Person <cris@example.com> [not verified] at ...>
Server owners

Some users are designated as server owners. At first there are no server owners.

>>> len(list(user_manager.server_owners))
0

Dan is made a server owner.

>>> user_4.is_server_owner = True
>>> owners = list(user_manager.server_owners)
>>> len(owners)
1
>>> owners[0]
<User "Dan Person" (...) at ...>

Now Ben and Claire are also made server owners.

>>> user_2.is_server_owner = True
>>> user_3.is_server_owner = True
>>> owners = list(user_manager.server_owners)
>>> len(owners)
3
>>> from operator import attrgetter
>>> for user in sorted(owners, key=attrgetter('display_name')):
...     print(user)
<User "Ben Person" (...) at ...>
<User "Claire Person" (...) at ...>
<User "Dan Person" (...) at ...>

Clair retires as a server owner.

>>> user_3.is_server_owner = False
>>> owners = list(user_manager.server_owners)
>>> len(owners)
2
>>> for user in sorted(owners, key=attrgetter('display_name')):
...     print(user)
<User "Ben Person" (...) at ...>
<User "Dan Person" (...) at ...>
Users

Users are entities that represent people. A user has a real name and a optional encoded password. A user may also have an optional preferences and a set of addresses they control. They can even have a preferred address, i.e. one that they use by default.

See usermanager.txt for examples of how to create, delete, and find users.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)
User data

Users may have a real name and a password.

>>> user_1 = user_manager.create_user()
>>> user_1.password = 'my password'
>>> user_1.display_name = 'Zoe Person'
>>> dump_list(user.display_name for user in user_manager.users)
Zoe Person
>>> dump_list(user.password for user in user_manager.users)
my password

The password and real name can be changed at any time.

>>> user_1.display_name = 'Zoe X. Person'
>>> user_1.password = 'another password'
>>> dump_list(user.display_name for user in user_manager.users)
Zoe X. Person
>>> dump_list(user.password for user in user_manager.users)
another password

When the user’s password is changed, an event is triggered.

>>> saved_event = None
>>> def save_event(event):
...     global saved_event
...     saved_event = event
>>> from mailman.testing.helpers import event_subscribers
>>> with event_subscribers(save_event):
...     user_1.password = 'changed again'
>>> print(saved_event)
<PasswordChangeEvent Zoe X. Person>

The event holds a reference to the IUser that changed their password.

>>> print(saved_event.user.display_name)
Zoe X. Person
>>> print(saved_event.user.password)
changed again
Basic user identification

Although rarely visible to users, every user has a unique immutable ID. This ID is generated randomly at the time the user is created, and is represented by a UUID.

>>> print(user_1.user_id)
00000000-0000-0000-0000-000000000001

User records also have a date on which they where created.

# The test suite uses a predictable timestamp. >>> print(user_1.created_on) 2005-08-01 07:49:23
Users addresses

One of the pieces of information that a user links to is a set of email addresses they control, in the form of IAddress objects. A user can control many addresses, but addresses may be linked to only one user.

The easiest way to link a user to an address is to just register the new address on a user object.

>>> user_1.register('zperson@example.com', 'Zoe Person')
<Address: Zoe Person <zperson@example.com> [not verified] at 0x...>
>>> user_1.register('zperson@example.org')
<Address: zperson@example.org [not verified] at 0x...>
>>> dump_list(address.email for address in user_1.addresses)
zperson@example.com
zperson@example.org
>>> dump_list(address.display_name for address in user_1.addresses)
<BLANKLINE>
Zoe Person

You can also create the address separately and then link it to the user.

>>> address_1 = user_manager.create_address('zperson@example.net')
>>> user_1.link(address_1)
>>> dump_list(address.email for address in user_1.addresses)
zperson@example.com
zperson@example.net
zperson@example.org
>>> dump_list(address.display_name for address in user_1.addresses)
<BLANKLINE>
<BLANKLINE>
Zoe Person

You can also ask whether a given user controls a given address.

>>> user_1.controls(address_1.email)
True
>>> user_1.controls('bperson@example.com')
False

Given a text email address, the user manager can find the user that controls that address.

>>> user_manager.get_user('zperson@example.com') is user_1
True
>>> user_manager.get_user('zperson@example.net') is user_1
True
>>> user_manager.get_user('zperson@example.org') is user_1
True
>>> print(user_manager.get_user('bperson@example.com'))
None

Addresses can also be unlinked from a user.

>>> user_1.unlink(address_1)
>>> user_1.controls('zperson@example.net')
False
>>> print(user_manager.get_user('aperson@example.net'))
None
Preferred address

Users can register a preferred address. When subscribing to a mailing list, unless some other address is explicitly specified, the user will be subscribed with their preferred address. This allows them to change their preferred address once, and have all their subscriptions automatically track this change.

By default, a user has no preferred address.

>>> user_2 = user_manager.create_user()
>>> print(user_2.preferred_address)
None

Even when a user registers an address, this address will not be set as the preferred address.

>>> anne = user_2.register('anne@example.com', 'Anne Person')
>>> print(user_2.preferred_address)
None

Once the address has been verified, it can be set as the preferred address, but only if the address is either controlled by the user or uncontrolled. In the latter case, setting it as the preferred address makes it controlled by the user.

>>> from mailman.utilities.datetime import now
>>> anne.verified_on = now()
>>> anne
<Address: Anne Person <anne@example.com> [verified] at ...>
>>> user_2.controls(anne.email)
True
>>> user_2.preferred_address = anne
>>> user_2.preferred_address
<Address: Anne Person <anne@example.com> [verified] at ...>

>>> aperson = user_manager.create_address('aperson@example.com')
>>> user_2.controls(aperson.email)
False
>>> aperson.verified_on = now()
>>> user_2.preferred_address = aperson
>>> user_2.controls(aperson.email)
True

A user can disavow their preferred address.

>>> user_2.preferred_address
<Address: aperson@example.com [verified] at ...>
>>> del user_2.preferred_address
>>> print(user_2.preferred_address)
None

The preferred address always shows up in the set of addresses controlled by this user.

>>> from operator import attrgetter
>>> for address in sorted(user_2.addresses, key=attrgetter('email')):
...     print(address.email)
anne@example.com
aperson@example.com
Users and preferences

This is a helper function for the following section.

>>> def show_prefs(prefs):
...     print('acknowledge_posts    :', prefs.acknowledge_posts)
...     print('preferred_language   :', prefs.preferred_language)
...     print('receive_list_copy    :', prefs.receive_list_copy)
...     print('receive_own_postings :', prefs.receive_own_postings)
...     print('delivery_mode        :', prefs.delivery_mode)

Users have preferences, but these preferences have no default settings.

>>> from mailman.interfaces.preferences import IPreferences
>>> show_prefs(user_1.preferences)
acknowledge_posts    : None
preferred_language   : None
receive_list_copy    : None
receive_own_postings : None
delivery_mode        : None

Some of these preferences are booleans and they can be set to True or False.

>>> from mailman.core.constants import DeliveryMode
>>> prefs = user_1.preferences
>>> prefs.acknowledge_posts = True
>>> prefs.preferred_language = 'it'
>>> prefs.receive_list_copy = False
>>> prefs.receive_own_postings = False
>>> prefs.delivery_mode = DeliveryMode.regular
>>> show_prefs(user_1.preferences)
acknowledge_posts    : True
preferred_language   : <Language [it] Italian>
receive_list_copy    : False
receive_own_postings : False
delivery_mode        : DeliveryMode.regular
Subscriptions

Users know which mailing lists they are subscribed to, regardless of membership role.

>>> user_1.link(address_1)
>>> dump_list(address.email for address in user_1.addresses)
zperson@example.com
zperson@example.net
zperson@example.org
>>> com = user_manager.get_address('zperson@example.com')
>>> org = user_manager.get_address('zperson@example.org')
>>> net = user_manager.get_address('zperson@example.net')

>>> mlist_1 = create_list('xtest_1@example.com')
>>> mlist_2 = create_list('xtest_2@example.com')
>>> mlist_3 = create_list('xtest_3@example.com')
>>> from mailman.interfaces.member import MemberRole

>>> mlist_1.subscribe(com, MemberRole.member)
<Member: Zoe Person <zperson@example.com> on xtest_1@example.com as
    MemberRole.member>
>>> mlist_2.subscribe(org, MemberRole.member)
<Member: zperson@example.org on xtest_2@example.com as MemberRole.member>
>>> mlist_2.subscribe(org, MemberRole.owner)
<Member: zperson@example.org on xtest_2@example.com as MemberRole.owner>
>>> mlist_3.subscribe(net, MemberRole.moderator)
<Member: zperson@example.net on xtest_3@example.com as
    MemberRole.moderator>

>>> memberships = user_1.memberships
>>> from mailman.interfaces.roster import IRoster
>>> from zope.interface.verify import verifyObject
>>> verifyObject(IRoster, memberships)
True
>>> def sortkey(member):
...     return member.address.email, member.mailing_list, member.role.value
>>> members = sorted(memberships.members, key=sortkey)
>>> len(members)
4
>>> for member in sorted(members, key=sortkey):
...     print(member.address.email, member.mailing_list.list_id,
...           member.role)
zperson@example.com xtest_1.example.com MemberRole.member
zperson@example.net xtest_3.example.com MemberRole.moderator
zperson@example.org xtest_2.example.com MemberRole.member
zperson@example.org xtest_2.example.com MemberRole.owner
Server owners

Some users are server owners. Zoe is not yet a server owner.

>>> user_1.is_server_owner
False

So, let’s make her one.

>>> user_1.is_server_owner = True
>>> user_1.is_server_owner
True

Runners

Alias Overview

A typical Mailman list exposes nine aliases which point to seven different wrapped scripts. E.g. for a list named mylist, you’d have:

mylist-bounces -> bounces
mylist-confirm -> confirm
mylist-join    -> join    (-subscribe is an alias)
mylist-leave   -> leave   (-unsubscribe is an alias)
mylist-owner   -> owner
mylist         -> post
mylist-request -> request

-request, -join, and -leave are a robot addresses; their sole purpose is to process emailed commands, although the latter two are hardcoded to subscription and unsubscription requests. -bounces is the automated bounce processor, and all messages to list members have their return address set to -bounces. If the bounce processor fails to extract a bouncing member address, it can optionally forward the message on to the list owners.

-owner is for reaching a human operator with minimal list interaction (i.e. no bounce processing). -confirm is another robot address which processes replies to VERP-like confirmation notices.

So delivery flow of messages look like this:

joerandom ---> mylist ---> list members
   |                           |
   |                           |[bounces]
   |        mylist-bounces <---+ <-------------------------------+
   |              |                                              |
   |              +--->[internal bounce processing]              |
   |              ^                |                             |
   |              |                |    [bounce found]           |
   |         [bounces *]           +--->[register and discard]   |
   |              |                |                      |      |
   |              |                |                      |[*]   |
   |        [list owners]          |[no bounce found]     |      |
   |              ^                |                      |      |
   |              |                |                      |      |
   +-------> mylist-owner <--------+                      |      |
   |                                                      |      |
   |           data/owner-bounces.mbox <--[site list] <---+      |
   |                                                             |
   +-------> mylist-join--+                                      |
   |                      |                                      |
   +------> mylist-leave--+                                      |
   |                      |                                      |
   |                      v                                      |
   +-------> mylist-request                                      |
   |              |                                              |
   |              +---> [command processor]                      |
   |                            |                                |
   +-----> mylist-confirm ----> +---> joerandom                  |
                                          |                      |
                                          |[bounces]             |
                                          +----------------------+

A person can send an email to the list address (for posting), the -owner address (to reach the human operator), or the -confirm, -join, -leave, and -request mailbots. Message to the list address are then forwarded on to the list membership, with bounces directed to the -bounces address.

[*] Messages sent to the -owner address are forwarded on to the list owner/moderators. All -owner destined messages have their bounces directed to the site list -bounces address, regardless of whether a human sent the message or the message was crafted internally. The intention here is that the site owners want to be notified when one of their list owners’ addresses starts bouncing (yes, the will be automated in a future release).

Any messages to site owners has their bounces directed to a special loop killer address, which just dumps the message into data/owners-bounces.mbox.

Finally, message to any of the mailbots causes the requested action to be performed. Results notifications are sent to the author of the message, which all bounces pointing back to the -bounces address.

The command runner

This runner’s purpose is to process and respond to email commands. Commands are extensible using the Mailman plug-in system, but Mailman comes with a number of email commands out of the box. These are processed when a message is sent to the list’s -request address.

>>> mlist = create_list('test@example.com')
>>> mlist.send_welcome_messages = False
A command in the Subject

For example, the echo command simply echoes the original command back to the sender. The command can be in the Subject header.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test-request@example.com
... Subject: echo hello
... Message-ID: <aardvark>
...
... """)

>>> from mailman.app.inject import inject_message
>>> filebase = inject_message(mlist, msg, switchboard='command')
>>> from mailman.runners.command import CommandRunner
>>> from mailman.testing.helpers import make_testable_runner
>>> command = make_testable_runner(CommandRunner)
>>> command.run()

And now the response is in the virgin queue.

>>> from mailman.testing.helpers import get_queue_messages
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1

>>> print(messages[0].msg.as_string())
Subject: The results of your email commands
From: test-bounces@example.com
To: aperson@example.com
...

The results of your email command are provided below.

- Original message details:
    From: aperson@example.com
    Subject: echo hello
    Date: ...
    Message-ID: <aardvark>

- Results:
echo hello

- Done.


>>> dump_msgdata(messages[0].msgdata)
_parsemsg           : False
listid              : test.example.com
nodecorate          : True
recipients          : {'aperson@example.com'}
reduced_list_headers: True
version             : ...
A command in the body

The command can also be found in the body of the message, as long as the message is plain text.

>>> msg = message_from_string("""\
... From: bperson@example.com
... To: test-request@example.com
... Message-ID: <bobcat>
...
... echo foo bar
... """)

>>> filebase = inject_message(mlist, msg, switchboard='command')
>>> command.run()
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1

>>> print(messages[0].msg.as_string())
Subject: The results of your email commands
From: test-bounces@example.com
To: bperson@example.com
...
Precedence: bulk

The results of your email command are provided below.

- Original message details:
    From: bperson@example.com
    Subject: n/a
    Date: ...
    Message-ID: <bobcat>

- Results:
echo foo bar

- Done.
Implicit commands

For some commands, specifically for joining and leaving a mailing list, there are email aliases that act like commands, even when there’s nothing else in the Subject or body. For example, to join a mailing list, a user need only email the -join address or -subscribe address (the latter is deprecated).

Because Dirk has never registered with Mailman before, he gets two responses. The first is a confirmation message so that Dirk can validate his email address, and the other is the results of his email command.

>>> msg = message_from_string("""\
... From: Dirk Person <dperson@example.com>
... To: test-join@example.com
...
... """)

>>> filebase = inject_message(
...     mlist, msg, switchboard='command', subaddress='join')
>>> command.run()
>>> messages = get_queue_messages('virgin', sort_on='subject')
>>> len(messages)
2

>>> from mailman.interfaces.subscriptions import ISubscriptionManager

>>> manager = ISubscriptionManager(mlist)
>>> for item in messages:
...     subject = item.msg['subject']
...     print('Subject:', subject)
...     if 'confirm' in str(subject):
...         token = str(subject).split()[1].strip()
...         new_token, token_owner, member = manager.confirm(token)
...         assert new_token is None, 'Confirmation failed'
Subject: The results of your email commands
Subject: confirm ...

Similarly, to leave a mailing list, the user need only email the -leave or -unsubscribe address (the latter is deprecated).

>>> msg = message_from_string("""\
... From: dperson@example.com
... To: test-leave@example.com
...
... """)

>>> from mailman.interfaces.mailinglist import SubscriptionPolicy
>>> mlist.unsubscription_policy = SubscriptionPolicy.open
>>> filebase = inject_message(
...     mlist, msg, switchboard='command', subaddress='leave')
>>> command.run()
>>> messages = get_queue_messages('virgin')
>>> len(messages)
2

>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
...
Subject: You have been unsubscribed from the Test mailing list
From: test-bounces@example.com
To: dperson@example.com
...

>>> print(messages[1].msg.as_string())
Subject: The results of your email commands
From: test-bounces@example.com
To: dperson@example.com
...

- Results:
Dirk Person <dperson@example.com> left test@example.com

- Done.

The -confirm address is also available as an implicit command.

>>> msg = message_from_string("""\
... From: dperson@example.com
... To: test-confirm+123@example.com
...
... """)

>>> filebase = inject_message(
...     mlist, msg, switchboard='command', subaddress='confirm')
>>> command.run()
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1

>>> print(messages[0].msg.as_string())
Subject: The results of your email commands
From: test-bounces@example.com
To: dperson@example.com
...

The results of your email command are provided below.

- Original message details:
From: dperson@example.com
Subject: n/a
Date: ...
Message-ID: ...

- Results:
Confirmation token did not match

- Done.
Stopping command processing

The end command stops email processing, so that nothing following is looked at by the command queue.

>>> msg = message_from_string("""\
... From: cperson@example.com
... To: test-request@example.com
... Message-ID: <caribou>
...
... echo foo bar
... end ignored
... echo baz qux
... """)

>>> filebase = inject_message(mlist, msg, switchboard='command')
>>> command.run()
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1

>>> print(messages[0].msg.as_string())
Subject: The results of your email commands
...

- Results:
echo foo bar

- Unprocessed:
echo baz qux

- Done.

The stop command is an alias for end.

>>> msg = message_from_string("""\
... From: cperson@example.com
... To: test-request@example.com
... Message-ID: <caribou>
...
... echo foo bar
... stop ignored
... echo baz qux
... """)

>>> filebase = inject_message(mlist, msg, switchboard='command')
>>> command.run()
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1

>>> print(messages[0].msg.as_string())
Subject: The results of your email commands
...

- Results:
echo foo bar

- Unprocessed:
echo baz qux

- Done.
Digesting

Mailman crafts and sends digests by a separate digest runner process. This starts by a number of messages being posted to the mailing list.

>>> mlist = create_list('test@example.com')
>>> mlist.digest_size_threshold = 0.6
>>> mlist.volume = 1
>>> mlist.next_digest_number = 1
>>> mlist.send_welcome_message = False

>>> from string import Template
>>> process = config.handlers['to-digest'].process

>>> def fill_digest():
...     size = 0
...     for i in range(1, 5):
...         text = Template("""\
... From: aperson@example.com
... To: xtest@example.com
... Subject: Test message $i
... List-Post: <test@example.com>
...
... Here is message $i
... """).substitute(i=i)
...         msg = message_from_string(text)
...         process(mlist, msg, {})
...         size += len(text)
...         if size >= mlist.digest_size_threshold * 1024:
...             break

>>> fill_digest()

The runner gets kicked off when a marker message gets dropped into the digest queue. The message metadata points to the mailbox file containing the messages to put in the digest.

>>> digestq = config.switchboards['digest']
>>> len(digestq.files)
1

>>> from mailman.testing.helpers import get_queue_messages
>>> entry = get_queue_messages('digest')[0]

The marker message is empty.

>>> print(entry.msg.as_string())

But the message metadata has a reference to the digest file.

>>> dump_msgdata(entry.msgdata)
_parsemsg    : False
digest_number: 1
digest_path  : .../lists/test.example.com/digest.1.1.mmdf
listid       : test.example.com
version      : 3
volume       : 1

There are 4 messages in the digest.

>>> from mailman.utilities.mailbox import Mailbox
>>> sum(1 for item in Mailbox(entry.msgdata['digest_path']))
4

When the runner runs, it processes the digest mailbox, crafting both the plain text (RFC 1153) digest and the MIME digest.

>>> from mailman.runners.digest import DigestRunner
>>> from mailman.testing.helpers import make_testable_runner
>>> runner = make_testable_runner(DigestRunner)
>>> runner.run()

If there are no members receiving digests, none are sent.

>>> messages = get_queue_messages('virgin')
>>> len(messages)
0

Once some users are subscribed and receiving digests, the digest runner places both digests into the virgin queue for final delivery.

>>> from mailman.testing.helpers import subscribe
>>> from mailman.interfaces.member import DeliveryMode

>>> anne = subscribe(mlist, 'Anne')
>>> anne.preferences.delivery_mode = DeliveryMode.mime_digests
>>> bart = subscribe(mlist, 'Bart')
>>> bart.preferences.delivery_mode = DeliveryMode.plaintext_digests

>>> fill_digest()
>>> runner.run()
>>> messages = get_queue_messages('virgin')
>>> len(messages)
2

Anne and Bart unsubscribe from the mailing list.

>>> anne.unsubscribe()
>>> bart.unsubscribe()

The MIME digest is a multipart, and the RFC 1153 digest is the other one.

>>> def mime_rfc1153(messages):
...     if messages[0].msg.is_multipart():
...         return messages[0], messages[1]
...     return messages[1], messages[0]

>>> mime, rfc1153 = mime_rfc1153(messages)

The MIME digest has lots of good stuff, all contained in the multipart.

>>> print(mime.msg.as_string())
Content-Type: multipart/mixed; boundary="===============...=="
MIME-Version: 1.0
From: test-request@example.com
Subject: Test Digest, Vol 1, Issue 2
To: test@example.com
Reply-To: test@example.com
Date: ...
Message-ID: ...
<BLANKLINE>
--===============...==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Description: Test Digest, Vol 1, Issue 2
<BLANKLINE>
Send Test mailing list submissions to
    test@example.com
<BLANKLINE>
To subscribe or unsubscribe via email, send a message with subject or
body 'help' to
    test-request@example.com
<BLANKLINE>
You can reach the person managing the list at
    test-owner@example.com
<BLANKLINE>
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Test digest..."
--===============...==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Description: Today's Topics (4 messages)
<BLANKLINE>
Today's Topics:
<BLANKLINE>
   1. Test message 1 (aperson@example.com)
   2. Test message 2 (aperson@example.com)
   3. Test message 3 (aperson@example.com)
   4. Test message 4 (aperson@example.com)
<BLANKLINE>
--===============...==
Content-Type: multipart/digest; boundary="===============...=="
MIME-Version: 1.0
<BLANKLINE>
--===============...==
Content-Type: message/rfc822
MIME-Version: 1.0
<BLANKLINE>
From: aperson@example.com
To: xtest@example.com
Subject: Test message 1
List-Post: <test@example.com>
<BLANKLINE>
Here is message 1
<BLANKLINE>
--===============...==
Content-Type: message/rfc822
MIME-Version: 1.0
<BLANKLINE>
From: aperson@example.com
To: xtest@example.com
Subject: Test message 2
List-Post: <test@example.com>
<BLANKLINE>
Here is message 2
<BLANKLINE>
--===============...==
Content-Type: message/rfc822
MIME-Version: 1.0
<BLANKLINE>
From: aperson@example.com
To: xtest@example.com
Subject: Test message 3
List-Post: <test@example.com>
<BLANKLINE>
Here is message 3
<BLANKLINE>
--===============...==
Content-Type: message/rfc822
MIME-Version: 1.0
<BLANKLINE>
From: aperson@example.com
To: xtest@example.com
Subject: Test message 4
List-Post: <test@example.com>
<BLANKLINE>
Here is message 4
<BLANKLINE>
--===============...==--
<BLANKLINE>
--===============...==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Description: Digest Footer
<BLANKLINE>
_______________________________________________
Test mailing list -- test@example.com
To unsubscribe send an email to test-leave@example.com
<BLANKLINE>
--===============...==--
<BLANKLINE>

The RFC 1153 contains the digest in a single plain text message.

>>> print(rfc1153.msg.as_string())
From: test-request@example.com
Subject: Test Digest, Vol 1, Issue 2
To: test@example.com
Reply-To: test@example.com
Date: ...
Message-ID: ...
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
<BLANKLINE>
Send Test mailing list submissions to
    test@example.com
<BLANKLINE>
To subscribe or unsubscribe via email, send a message with subject or
body 'help' to
    test-request@example.com
<BLANKLINE>
You can reach the person managing the list at
    test-owner@example.com
<BLANKLINE>
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Test digest..."
<BLANKLINE>
Today's Topics:
<BLANKLINE>
   1. Test message 1 (aperson@example.com)
   2. Test message 2 (aperson@example.com)
   3. Test message 3 (aperson@example.com)
   4. Test message 4 (aperson@example.com)
<BLANKLINE>
<BLANKLINE>
----------------------------------------------------------------------
<BLANKLINE>
From: aperson@example.com
Subject: Test message 1
To: xtest@example.com
<BLANKLINE>
Here is message 1
<BLANKLINE>
------------------------------
<BLANKLINE>
From: aperson@example.com
Subject: Test message 2
To: xtest@example.com
<BLANKLINE>
Here is message 2
<BLANKLINE>
------------------------------
<BLANKLINE>
From: aperson@example.com
Subject: Test message 3
To: xtest@example.com
<BLANKLINE>
Here is message 3
<BLANKLINE>
------------------------------
<BLANKLINE>
From: aperson@example.com
Subject: Test message 4
To: xtest@example.com
<BLANKLINE>
Here is message 4
<BLANKLINE>
------------------------------
<BLANKLINE>
Subject: Digest Footer
<BLANKLINE>
_______________________________________________
Test mailing list -- test@example.com
To unsubscribe send an email to test-leave@example.com
<BLANKLINE>
<BLANKLINE>
------------------------------
<BLANKLINE>
End of Test Digest, Vol 1, Issue 2
**********************************
<BLANKLINE>
Digest delivery

A mailing list’s members can choose to receive normal delivery, plain text digests, or MIME digests.

>>> len(get_queue_messages('virgin'))
0

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)

>>> from mailman.interfaces.member import DeliveryMode, MemberRole
>>> def subscribe(email, mode):
...     address = user_manager.create_address(email)
...     member = mlist.subscribe(address, MemberRole.member)
...     member.preferences.delivery_mode = mode
...     return member

Two regular delivery members subscribe to the mailing list.

>>> member_1 = subscribe('uperson@example.com', DeliveryMode.regular)
>>> member_2 = subscribe('vperson@example.com', DeliveryMode.regular)

Two MIME digest members subscribe to the mailing list.

>>> member_3 = subscribe('wperson@example.com', DeliveryMode.mime_digests)
>>> member_4 = subscribe('xperson@example.com', DeliveryMode.mime_digests)

One RFC 1153 digest member subscribes to the mailing list.

>>> member_5 = subscribe(
...     'yperson@example.com', DeliveryMode.plaintext_digests)
>>> member_6 = subscribe(
...     'zperson@example.com', DeliveryMode.plaintext_digests)

When a digest gets sent, the appropriate recipient list is chosen.

>>> mlist.preferred_language = 'en'
>>> mlist.digest_size_threshold = 0.5
>>> fill_digest()
>>> runner.run()

The digests are sitting in the virgin queue. One of them is the MIME digest and the other is the RFC 1153 digest.

>>> messages = get_queue_messages('virgin')
>>> len(messages)
2

>>> mime, rfc1153 = mime_rfc1153(messages)

Only wperson and xperson get the MIME digests.

>>> sorted(mime.msgdata['recipients'])
['wperson@example.com', 'xperson@example.com']

Only yperson and zperson get the RFC 1153 digests.

>>> sorted(rfc1153.msgdata['recipients'])
['yperson@example.com', 'zperson@example.com']

Now uperson decides that they would like to start receiving digests too.

>>> member_1.preferences.delivery_mode = DeliveryMode.mime_digests
>>> fill_digest()
>>> runner.run()

>>> messages = get_queue_messages('virgin')
>>> len(messages)
2

>>> mime, rfc1153 = mime_rfc1153(messages)
>>> sorted(mime.msgdata['recipients'])
['uperson@example.com', 'wperson@example.com', 'xperson@example.com']

>>> sorted(rfc1153.msgdata['recipients'])
['yperson@example.com', 'zperson@example.com']

At this point, both uperson and wperson decide that they’d rather receive regular deliveries instead of digests. uperson would like to get any last digest that may be sent so that she doesn’t miss anything. wperson does care as much and does not want to receive one last digest.

>>> mlist.send_one_last_digest_to(
...     member_1.address, member_1.preferences.delivery_mode)

>>> member_1.preferences.delivery_mode = DeliveryMode.regular
>>> member_3.preferences.delivery_mode = DeliveryMode.regular

>>> fill_digest()
>>> runner.run()

>>> messages = get_queue_messages('virgin')
>>> mime, rfc1153 = mime_rfc1153(messages)
>>> sorted(mime.msgdata['recipients'])
['uperson@example.com', 'xperson@example.com']

>>> sorted(rfc1153.msgdata['recipients'])
['yperson@example.com', 'zperson@example.com']

Since uperson has received their last digest, they will not get any more of them.

>>> fill_digest()
>>> runner.run()

>>> messages = get_queue_messages('virgin')
>>> len(messages)
2

>>> mime, rfc1153 = mime_rfc1153(messages)
>>> sorted(mime.msgdata['recipients'])
['xperson@example.com']

>>> sorted(rfc1153.msgdata['recipients'])
['yperson@example.com', 'zperson@example.com']
The incoming runner

This runner’s sole purpose in life is to decide the disposition of the message. It can either be accepted for delivery, rejected (i.e. bounced), held for moderator approval, or discarded.

The runner operates by processing chains on a message/metadata pair in the context of a mailing list. Each mailing list has a default chain for messages posted to the mailing list. This chain is processed with the message eventually ending up in one of the four disposition states described above.

>>> mlist = create_list('test@example.com')
>>> print(mlist.posting_chain)
default-posting-chain
Sender addresses

The incoming runner ensures that the sender addresses on the message are registered with the system. This is used for determining nonmember posting privileges. The addresses will not be linked to a user and will be unverified, so if the real user comes along later and claims the address, it will be linked to their user account (and must be verified).

While configurable, the sender addresses by default are those named in the From:, Sender:, and Reply-To: headers, as well as the envelope sender (though we won’t worry about the latter).

>>> msg = message_from_string("""\
... From: zperson@example.com
... Reply-To: yperson@example.com
... Sender: xperson@example.com
... To: test@example.com
... Subject: This is spiced ham
... Message-ID: <bogus>
...
... """)

>>> from zope.component import getUtility
>>> from mailman.interfaces.usermanager import IUserManager
>>> user_manager = getUtility(IUserManager)
>>> print(user_manager.get_address('xperson@example.com'))
None
>>> print(user_manager.get_address('yperson@example.com'))
None
>>> print(user_manager.get_address('zperson@example.com'))
None

Inject the message into the incoming queue, similar to the way the upstream mail server normally would.

>>> from mailman.app.inject import inject_message
>>> filebase = inject_message(mlist, msg)

The incoming runner runs until it is empty.

>>> from mailman.runners.incoming import IncomingRunner
>>> from mailman.testing.helpers import make_testable_runner
>>> incoming = make_testable_runner(IncomingRunner, 'in')
>>> incoming.run()

And now the addresses are known to the system. As mentioned above, they are not linked to a user and are unverified.

>>> for localpart in ('xperson', 'yperson', 'zperson'):
...     email = '{0}@example.com'.format(localpart)
...     address = user_manager.get_address(email)
...     print('{0}; verified? {1}; user? {2}'.format(
...           address.email,
...           ('No' if address.verified_on is None else 'Yes'),
...           user_manager.get_user(email)))
xperson@example.com; verified? No; user? None
yperson@example.com; verified? No; user? None
zperson@example.com; verified? No; user? None
Accepted messages

We have a message that is going to be sent to the mailing list. Once Anne is a member of the mailing list, this message is so perfectly fine for posting that it will be accepted and forward to the pipeline queue.

>>> from mailman.testing.helpers import subscribe
>>> subscribe(mlist, 'Anne')
<Member: Anne Person <aperson@example.com> on test@example.com
         as MemberRole.member>

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: My first post
... Message-ID: <first>
...
... First post!
... """)

Inject the message into the incoming queue and run until the queue is empty.

>>> filebase = inject_message(mlist, msg)
>>> incoming.run()

There are no messages left in the incoming queue.

>>> get_queue_messages('in')
[]

Now the message is in the pipeline queue.

>>> messages = get_queue_messages('pipeline')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
From: aperson@example.com
To: test@example.com
Subject: My first post
Message-ID: <first>
Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
Date: ...
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency;
    loop; banned-address; member-moderation; nonmember-moderation;
    administrivia; implicit-dest; max-recipients; max-size;
    news-moderation; no-subject; suspicious-header
<BLANKLINE>
First post!
<BLANKLINE>
>>> dump_msgdata(messages[0].msgdata)
_parsemsg    : False
envsender    : noreply@example.com
...
Held messages

The list moderator sets the emergency flag on the mailing list. The built-in chain will now hold all posted messages, so nothing will show up in the pipeline queue.

>>> from mailman.interfaces.chain import ChainEvent
>>> def on_chain(event):
...     if isinstance(event, ChainEvent):
...         print(event)
...         print(event.chain)
...         print('From: {0}\nTo: {1}\nMessage-ID: {2}'.format(
...             event.msg['from'], event.msg['to'],
...             event.msg['message-id']))

>>> mlist.emergency = True

>>> from mailman.testing.helpers import event_subscribers
>>> with event_subscribers(on_chain):
...     filebase = inject_message(mlist, msg)
...     incoming.run()
<mailman.interfaces.chain.HoldEvent ...>
<mailman.chains.hold.HoldChain ...>
From: aperson@example.com
To: test@example.com
Message-ID: <first>

>>> mlist.emergency = False
Discarded messages

Another possibility is that the message would get immediately discarded. The built-in chain does not have such a disposition by default, so let’s craft a new chain and set it as the mailing list’s start chain.

>>> from mailman.chains.base import Chain, Link
>>> from mailman.interfaces.chain import LinkAction
>>> def make_chain(name, target_chain):
...     test_chain = Chain(name, 'Testing {}'.format(target_chain))
...     config.chains[test_chain.name] = test_chain
...     link = Link('truth', LinkAction.jump, target_chain)
...     test_chain.append_link(link)
...     return test_chain

>>> test_chain = make_chain('always-discard', 'discard')
>>> mlist.posting_chain = test_chain.name

>>> msg.replace_header('message-id', '<second>')
>>> with event_subscribers(on_chain):
...     filebase = inject_message(mlist, msg)
...     incoming.run()
<mailman.interfaces.chain.DiscardEvent ...>
<mailman.chains.discard.DiscardChain ...>
From: aperson@example.com
To: test@example.com
Message-ID: <second>

>>> del config.chains[test_chain.name]
Rejected messages

Similar to discarded messages, a message can be rejected, or bounced back to the original sender. Again, the built-in chain doesn’t support this so we’ll just create a new chain that does.

>>> test_chain = make_chain('always-reject', 'reject')
>>> mlist.posting_chain = test_chain.name

>>> msg.replace_header('message-id', '<third>')
>>> with event_subscribers(on_chain):
...     filebase = inject_message(mlist, msg)
...     incoming.run()
<mailman.interfaces.chain.RejectEvent ...>
<mailman.chains.reject.RejectChain ...>
From: aperson@example.com
To: test@example.com
Message-ID: <third>

The rejection message is sitting in the virgin queue waiting to be delivered to the original sender.

>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
Subject: My first post
From: test-owner@example.com
To: aperson@example.com
...
<BLANKLINE>
--===============...
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
<BLANKLINE>
[No bounce details are available]
--===============...
Content-Type: message/rfc822
MIME-Version: 1.0
<BLANKLINE>
From: aperson@example.com
To: test@example.com
Subject: My first post
Message-ID: <third>
Date: ...
<BLANKLINE>
First post!
<BLANKLINE>
--===============...

>>> del config.chains['always-reject']
LMTP server

Mailman can accept messages via LMTP (RFC 2033). Most modern mail servers support LMTP local delivery, so this is a very portable way to connect Mailman with your mail server.

Our LMTP server is fairly simple though; all it does is make sure that the message is destined for a valid endpoint, e.g. mylist-join@example.com, that the message bytes can be parsed into a message object, and that the message has a Message-ID header.

Let’s start a testable LMTP runner.

>>> from mailman.testing import helpers
>>> master = helpers.TestableMaster()
>>> master.start('lmtp')

It also helps to have a nice LMTP client.

>>> lmtp = helpers.get_lmtp_client()
(220, b'... GNU Mailman LMTP runner 2.0')
>>> lmtp.lhlo('remote.example.org')
(250, ...)
Posting address

Once the mailing list is created, the posting address is valid, and messages can be sent to the list.

>>> create_list('mylist@example.com')
<mailing list "mylist@example.com" at ...>

>>> transaction.commit()
>>> lmtp.sendmail(
...     'anne.person@example.com',
...     ['mylist@example.com'], """\
... From: anne.person@example.com
... To: mylist@example.com
... Subject: An interesting message
... Message-ID: <badger>
...
... This is an interesting message.
... """)
{}

Since the message itself is valid, it gets parsed and lands in the incoming queue.

>>> from mailman.testing.helpers import get_queue_messages
>>> messages = get_queue_messages('in')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
From: anne.person@example.com
To: mylist@example.com
Subject: An interesting message
Message-ID: <badger>
Message-ID-Hash: JYMZWSQ4IC2JPKK7ZUONRFRVC4ZYJGKJ
X-Message-ID-Hash: JYMZWSQ4IC2JPKK7ZUONRFRVC4ZYJGKJ
X-MailFrom: anne.person@example.com
<BLANKLINE>
This is an interesting message.
>>> dump_msgdata(messages[0].msgdata)
_parsemsg    : False
listid       : mylist.example.com
original_size: ...
to_list      : True
version      : ...
Sub-addresses

The LMTP server understands each of the list’s sub-addreses, such as -join, -leave, -request and so on. The message is accepted if posted to a valid sub-address.

>>> lmtp.sendmail(
...     'anne.person@example.com',
...     ['mylist-request@example.com'], """\
... From: anne.person@example.com
... To: mylist-request@example.com
... Subject: Help
... Message-ID: <dog>
...
... Please help me.
... """)
{}
Request subaddress

Depending on the subaddress, there is a message in the appropriate queue for later processing. For example, all -request messages are put into the command queue for processing.

>>> messages = get_queue_messages('command')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
From: anne.person@example.com
To: mylist-request@example.com
Subject: Help
Message-ID: <dog>
Message-ID-Hash: 4SKREUSPI62BHDMFBSOZ3BMXFETSQHNA
X-Message-ID-Hash: 4SKREUSPI62BHDMFBSOZ3BMXFETSQHNA
X-MailFrom: anne.person@example.com
<BLANKLINE>
Please help me.
>>> dump_msgdata(messages[0].msgdata)
_parsemsg    : False
listid       : mylist.example.com
original_size: ...
subaddress   : request
version      : ...
Bounce processor

A message to the -bounces address goes to the bounce processor.

>>> lmtp.sendmail(
...     'mail-daemon@example.com',
...     ['mylist-bounces@example.com'], """\
... From: mail-daemon@example.com
... To: mylist-bounces@example.com
... Subject: A bounce
... Message-ID: <elephant>
...
... Bouncy bouncy.
... """)
{}
>>> messages = get_queue_messages('bounces')
>>> len(messages)
1
>>> dump_msgdata(messages[0].msgdata)
_parsemsg    : False
listid       : mylist.example.com
original_size: ...
subaddress   : bounces
version      : ...
Command processor

Confirmation messages go to the command processor…

>>> lmtp.sendmail(
...     'anne.person@example.com',
...     ['mylist-confirm@example.com'], """\
... From: anne.person@example.com
... To: mylist-confirm@example.com
... Subject: A bounce
... Message-ID: <falcon>
...
... confirm 123
... """)
{}
>>> messages = get_queue_messages('command')
>>> len(messages)
1
>>> dump_msgdata(messages[0].msgdata)
_parsemsg    : False
listid       : mylist.example.com
original_size: ...
subaddress   : confirm
version      : ...

…as do join messages…

>>> lmtp.sendmail(
...     'anne.person@example.com',
...     ['mylist-join@example.com'], """\
... From: anne.person@example.com
... To: mylist-join@example.com
... Message-ID: <giraffe>
...
... """)
{}
>>> messages = get_queue_messages('command')
>>> len(messages)
1
>>> dump_msgdata(messages[0].msgdata)
_parsemsg    : False
listid       : mylist.example.com
original_size: ...
subaddress   : join
version      : ...

>>> lmtp.sendmail(
...     'anne.person@example.com',
...     ['mylist-subscribe@example.com'], """\
... From: anne.person@example.com
... To: mylist-subscribe@example.com
... Message-ID: <hippopotamus>
...
... """)
{}
>>> messages = get_queue_messages('command')
>>> len(messages)
1
>>> dump_msgdata(messages[0].msgdata)
_parsemsg    : False
listid       : mylist.example.com
original_size: ...
subaddress   : join
version      : ...

…and leave messages.

>>> lmtp.sendmail(
...     'anne.person@example.com',
...     ['mylist-leave@example.com'], """\
... From: anne.person@example.com
... To: mylist-leave@example.com
... Message-ID: <iguana>
...
... """)
{}
>>> messages = get_queue_messages('command')
>>> len(messages)
1
>>> dump_msgdata(messages[0].msgdata)
_parsemsg    : False
listid       : mylist.example.com
original_size: ...
subaddress   : leave
version      : ...

>>> lmtp.sendmail(
...     'anne.person@example.com',
...     ['mylist-unsubscribe@example.com'], """\
... From: anne.person@example.com
... To: mylist-unsubscribe@example.com
... Message-ID: <jackal>
...
... """)
{}
>>> messages = get_queue_messages('command')
>>> len(messages)
1
>>> dump_msgdata(messages[0].msgdata)
_parsemsg    : False
listid       : mylist.example.com
original_size: ...
subaddress   : leave
version      : ...
Incoming processor

Messages to the -owner address also go to the incoming processor.

>>> lmtp.sendmail(
...     'anne.person@example.com',
...     ['mylist-owner@example.com'], """\
... From: anne.person@example.com
... To: mylist-owner@example.com
... Message-ID: <kangaroo>
...
... """)
{}
>>> messages = get_queue_messages('in')
>>> len(messages)
1
>>> dump_msgdata(messages[0].msgdata)
_parsemsg    : False
envsender    : noreply@example.com
listid       : mylist.example.com
original_size: ...
subaddress   : owner
to_owner     : True
version      : ...
The NNTP runner

The NNTP runner gateways mailing list messages to an NNTP newsgroup.

>>> mlist = create_list('test@example.com')
>>> mlist.linked_newsgroup = 'comp.lang.python'

Get a handle on the NNTP server, which we’ll use later to verify the posted messages.

>>> from mailman.testing.helpers import get_nntp_server
>>> nntpd = get_nntp_server(cleanups)

A message gets posted to the mailing list. It may contain some headers which are prohibited by NNTP servers such as INN.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... NNTP-Posting-Host: news.example.com
... NNTP-Posting-Date: today
... X-Trace: blah blah
... X-Complaints-To: abuse@dom.ain
... Xref: blah blah
... Xref: blah blah
... Date-Received: yesterday
... Posted: tomorrow
... Posting-Version: 99.99
... Relay-Version: 88.88
... Received: blah blah
...
... A message
... """)

The message gets copied to the NNTP queue for preparation and posting.

>>> filebase = config.switchboards['nntp'].enqueue(
...     msg, listid='test.example.com')
>>> from mailman.testing.helpers import make_testable_runner
>>> from mailman.runners.nntp import NNTPRunner
>>> runner = make_testable_runner(NNTPRunner, 'nntp')
>>> runner.run()

The message was successfully posted the NNTP server.

>>> print(nntpd.get_message().as_string())
From: aperson@example.com
To: test@example.com
Newsgroups: comp.lang.python
Message-ID: ...
Lines: 1
<BLANKLINE>
A message
<BLANKLINE>
Outgoing runner

The outgoing runner is the process that delivers messages to the directly upstream SMTP server. It is this upstream SMTP server that performs final delivery to the intended recipients.

Messages that appear in the outgoing queue are processed individually through a delivery module, essentially a pluggable interface for determining how the recipient set will be batched, whether messages will be personalized and VERP’d, etc. The outgoing runner doesn’t itself support retrying but it can move messages to the ‘retry queue’ for handling delivery failures.

>>> mlist = create_list('test@example.com')

>>> from mailman.testing.helpers import subscribe
>>> subscribe(mlist, 'Anne')
<Member: Anne Person <aperson@example.com>
         on test@example.com as MemberRole.member>

>>> subscribe(mlist, 'Bart')
<Member: Bart Person <bperson@example.com>
         on test@example.com as MemberRole.member>

>>> subscribe(mlist, 'Cris')
<Member: Cris Person <cperson@example.com>
         on test@example.com as MemberRole.member>

Normally, messages would show up in the outgoing queue after the message has been processed by the rule set and pipeline. But we can simulate that here by injecting a message directly into the outgoing queue. First though, we must call the member-recipients handler so that the message metadata will be populated with the list of addresses to deliver the message to.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: My first post
... Message-ID: <first>
...
... First post!
... """)

>>> msgdata = {}
>>> handler = config.handlers['member-recipients']
>>> handler.process(mlist, msg, msgdata)
>>> outgoing_queue = config.switchboards['out']

The to-outgoing handler populates the message metadata with the destination mailing list name. Simulate that here too.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     tolist=True,
...     listid=mlist.list_id)

Running the outgoing runner processes the message, delivering it to the upstream SMTP.

>>> from mailman.runners.outgoing import OutgoingRunner
>>> from mailman.testing.helpers import make_testable_runner
>>> outgoing = make_testable_runner(OutgoingRunner, 'out')
>>> outgoing.run()

Every recipient got the same copy of the message.

>>> messages = list(smtpd.messages)
>>> len(messages)
1

>>> print(messages[0].as_string())
From: aperson@example.com
To: test@example.com
Subject: My first post
Message-ID: <first>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: aperson@example.com, bperson@example.com, cperson@example.com

First post!
Personalization

Mailman supports sending individual messages to each recipient by personalizing delivery. This increases the bandwidth between Mailman and the upstream mail server, and between the upstream mail server and the remote recipient mail servers. The benefit is that personalization provides for a much better user experience, because the messages can be tailored for each recipient.

>>> from mailman.interfaces.mailinglist import Personalization
>>> mlist.personalize = Personalization.individual
>>> transaction.commit()

Now when we send the message, our mail server will get three copies instead of just one.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
3

Since we’ve done no other configuration, the only difference in the messages is the recipient address. Specifically, the Sender header is the same for all recipients.

>>> from operator import itemgetter
>>> def show_headers(messages):
...     for message in sorted(messages, key=itemgetter('x-rcptto')):
...         print(message['X-RcptTo'], message['X-MailFrom'])

>>> show_headers(messages)
aperson@example.com   test-bounces@example.com
bperson@example.com   test-bounces@example.com
cperson@example.com   test-bounces@example.com
VERP

An even more interesting personalization opportunity arises if VERP is enabled. Here, Mailman takes advantage of the fact that it’s sending individualized messages anyway, so it also encodes the recipients address in the Sender header.

Forcing VERP

A handler can force VERP by setting the verp key in the message metadata.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     verp=True,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> show_headers(messages)
aperson@example.com   test-bounces+aperson=example.com@example.com
bperson@example.com   test-bounces+bperson=example.com@example.com
cperson@example.com   test-bounces+cperson=example.com@example.com
VERP personalized deliveries

The site administrator can enable VERP whenever messages are personalized.

>>> config.push('verp', """
... [mta]
... verp_personalized_deliveries: yes
... """)

Again, we get three individual messages, with VERP’d Sender headers.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> show_headers(messages)
aperson@example.com   test-bounces+aperson=example.com@example.com
bperson@example.com   test-bounces+bperson=example.com@example.com
cperson@example.com   test-bounces+cperson=example.com@example.com

>>> config.pop('verp')
>>> mlist.personalize = Personalization.none
>>> transaction.commit()
VERP every once in a while

Perhaps personalization is too much of an overhead, but the list owners would still like to occasionally get the benefits of VERP. The site administrator can enable occasional VERPing of messages every so often, by setting a delivery interval. Every N non-personalized deliveries turns on VERP for just the next one.

>>> config.push('verp occasionally', """
... [mta]
... verp_delivery_interval: 3
... """)

# Reset the list's post_id, which is used to calculate the intervals.
>>> mlist.post_id = 1
>>> transaction.commit()

The first message is sent to the list, and it is neither personalized nor VERP’d.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
1

>>> show_headers(messages)
aperson@example.com, bperson@example.com, cperson@example.com
test-bounces@example.com

# Perform post-delivery bookkeeping.
>>> after = config.handlers['after-delivery']
>>> after.process(mlist, msg, msgdata)
>>> transaction.commit()

The second message sent to the list is also not VERP’d.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
1

>>> show_headers(messages)
aperson@example.com, bperson@example.com, cperson@example.com
test-bounces@example.com

# Perform post-delivery bookkeeping.
>>> after.process(mlist, msg, msgdata)
>>> transaction.commit()

The third message though is VERP’d.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> show_headers(messages)
aperson@example.com   test-bounces+aperson=example.com@example.com
bperson@example.com   test-bounces+bperson=example.com@example.com
cperson@example.com   test-bounces+cperson=example.com@example.com

# Perform post-delivery bookkeeping.
>>> after.process(mlist, msg, msgdata)
>>> transaction.commit()

The next one is back to bulk delivery.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
1

>>> show_headers(messages)
aperson@example.com, bperson@example.com, cperson@example.com
test-bounces@example.com

>>> config.pop('verp occasionally')
VERP every time

If the site administrator wants to enable VERP for every delivery, even if no personalization is going on, they can set the interval to 1.

>>> config.push('always verp', """
... [mta]
... verp_delivery_interval: 1
... """)

# Reset the list's post_id, which is used to calculate the intervals.
>>> mlist.post_id = 1
>>> transaction.commit()

The first message is VERP’d.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> show_headers(messages)
aperson@example.com   test-bounces+aperson=example.com@example.com
bperson@example.com   test-bounces+bperson=example.com@example.com
cperson@example.com   test-bounces+cperson=example.com@example.com

# Perform post-delivery bookkeeping.
>>> after.process(mlist, msg, msgdata)
>>> transaction.commit()

As is the second message.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> show_headers(messages)
aperson@example.com   test-bounces+aperson=example.com@example.com
bperson@example.com   test-bounces+bperson=example.com@example.com
cperson@example.com   test-bounces+cperson=example.com@example.com

# Perform post-delivery bookkeeping.
>>> after.process(mlist, msg, msgdata)
>>> transaction.commit()

And the third message.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> show_headers(messages)
aperson@example.com   test-bounces+aperson=example.com@example.com
bperson@example.com   test-bounces+bperson=example.com@example.com
cperson@example.com   test-bounces+cperson=example.com@example.com

# Perform post-delivery bookkeeping.
>>> after.process(mlist, msg, msgdata)
>>> transaction.commit()

>>> config.pop('always verp')
Never VERP

Similarly, the site administrator can disable occasional VERP’ing of non-personalized messages by setting the interval to zero.

>>> config.push('never verp', """
... [mta]
... verp_delivery_interval: 0
... """)

# Reset the list's post_id, which is used to calculate the intervals.
>>> mlist.post_id = 1
>>> transaction.commit()

Neither the first message…

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
1

>>> show_headers(messages)
aperson@example.com, bperson@example.com, cperson@example.com
test-bounces@example.com

…nor the second message is VERP’d.

>>> ignore = outgoing_queue.enqueue(
...     msg, msgdata,
...     listid=mlist.list_id)
>>> outgoing.run()
>>> messages = list(smtpd.messages)
>>> len(messages)
1

>>> show_headers(messages)
aperson@example.com, bperson@example.com, cperson@example.com
test-bounces@example.com
REST server

Mailman is controllable through an administrative REST HTTP server.

>>> from mailman.testing import helpers
>>> master = helpers.TestableMaster(helpers.wait_for_webservice)
>>> master.start('rest')

The RESTful server can be used to access basic version information.

>>> dump_json('http://localhost:9001/3.1/system')
api_version: 3.1
http_etag: "..."
mailman_version: GNU Mailman 3...
python_version: ...
self_link: http://localhost:9001/3.1/system/versions

Previous versions of the REST API can also be accessed.

>>> dump_json('http://localhost:9001/3.0/system')
api_version: 3.0
http_etag: "..."
mailman_version: GNU Mailman 3...
python_version: ...
self_link: http://localhost:9001/3.0/system/versions
Clean up
>>> master.stop()

Chains

Moderation

Posts by members and nonmembers are subject to moderation checks during incoming processing. Different situations can cause such posts to be held for moderator approval.

>>> mlist = create_list('test@example.com')

Members and nonmembers have a moderation action which can shortcut the normal moderation checks. The built-in chain does just a few checks first, such as seeing if the message has a matching Approved: header, or if the emergency flag has been set on the mailing list, or whether a mail loop has been detected.

Mailing lists have a default moderation action, one for members and another for nonmembers. If a member’s moderation action is None, then the member moderation check falls back to the appropriate list default.

A moderation action of defer means that no explicit moderation check is performed and the rest of the rule chain processing proceeds as normal. But it is also common for first-time posters to have a hold action, meaning that their messages are held for moderator approval for a while.

Nonmembers almost always have a hold action, though some mailing lists may choose to set this default action to discard, meaning their posts would be immediately thrown away.

Member moderation

Posts by list members are moderated if the member’s moderation action is not deferred. The default setting for the moderation action of new members is determined by the mailing list’s settings. By default, a mailing list is not set to moderate new member postings.

>>> print(mlist.default_member_action)
Action.defer

In order to find out whether the message is held or accepted, we can subscribe to internal events that are triggered on each case.

>>> from mailman.interfaces.chain import ChainEvent
>>> def on_chain(event):
...     if isinstance(event, ChainEvent):
...         print(event)
...         print(event.chain)
...         print('Subject:', event.msg['subject'])
...         print('Hits:')
...         for hit in event.msgdata.get('rule_hits', []):
...             print('   ', hit)
...         print('Misses:')
...         for miss in event.msgdata.get('rule_misses', []):
...             print('   ', miss)

Anne is a list member with moderation action of None so that moderation will fall back to the mailing list’s default_member_action.

>>> from mailman.testing.helpers import subscribe
>>> member = subscribe(mlist, 'Anne', email='anne@example.com')
>>> member
<Member: Anne Person <anne@example.com> on test@example.com
         as MemberRole.member>
>>> print(member.moderation_action)
None

Anne’s post to the mailing list runs through the incoming runner’s default built-in chain. No rules hit and so the message is accepted.

>>> msg = message_from_string("""\
... From: anne@example.com
... To: test@example.com
... Subject: aardvark
...
... This is a test.
... """)

>>> from mailman.core.chains import process
>>> from mailman.testing.helpers import event_subscribers
>>> with event_subscribers(on_chain):
...     process(mlist, msg, {}, 'default-posting-chain')
<mailman.interfaces.chain.AcceptEvent ...>
<mailman.chains.accept.AcceptChain ...>
Subject: aardvark
Hits:
Misses:
    dmarc-mitigation
    no-senders
    approved
    emergency
    loop
    banned-address
    member-moderation
    nonmember-moderation
    administrivia
    implicit-dest
    max-recipients
    max-size
    news-moderation
    no-subject
    suspicious-header

However, when Anne’s moderation action is set to hold, her post is held for moderator approval.

>>> from mailman.interfaces.action import Action
>>> member.moderation_action = Action.hold

>>> msg = message_from_string("""\
... From: anne@example.com
... To: test@example.com
... Subject: badger
...
... This is a test.
... """)

>>> with event_subscribers(on_chain):
...     process(mlist, msg, {}, 'default-posting-chain')
<mailman.interfaces.chain.HoldEvent ...>
<mailman.chains.hold.HoldChain ...>
Subject: badger
Hits:
    member-moderation
Misses:
    dmarc-mitigation
    no-senders
    approved
    emergency
    loop
    banned-address

Anne’s moderation action can also be set to discard…

>>> member.moderation_action = Action.discard

>>> msg = message_from_string("""\
... From: anne@example.com
... To: test@example.com
... Subject: cougar
...
... This is a test.
... """)

>>> with event_subscribers(on_chain):
...     process(mlist, msg, {}, 'default-posting-chain')
<mailman.interfaces.chain.DiscardEvent ...>
<mailman.chains.discard.DiscardChain ...>
Subject: cougar
Hits:
    member-moderation
Misses:
    dmarc-mitigation
    no-senders
    approved
    emergency
    loop
    banned-address

… or reject.

>>> member.moderation_action = Action.reject

>>> msg = message_from_string("""\
... From: anne@example.com
... To: test@example.com
... Subject: dingo
...
... This is a test.
... """)

>>> with event_subscribers(on_chain):
...     process(mlist, msg, {}, 'default-posting-chain')
<mailman.interfaces.chain.RejectEvent ...>
<mailman.chains.reject.RejectChain ...>
Subject: dingo
Hits:
    member-moderation
Misses:
    dmarc-mitigation
    no-senders
    approved
    emergency
    loop
    banned-address
Nonmembers

Registered nonmembers are handled very similarly to members, except that a different list default setting is used when moderating nonmemberds. This is how the incoming runner adds sender addresses as nonmembers.

>>> from zope.component import getUtility
>>> from mailman.interfaces.usermanager import IUserManager
>>> user_manager = getUtility(IUserManager)
>>> address = user_manager.create_address('bart@example.com')
>>> address
<Address: bart@example.com [not verified] at ...>

When the moderation rule runs on a message from this sender, this address will be registered as a nonmember of the mailing list, and it will be held for moderator approval.

>>> msg = message_from_string("""\
... From: bart@example.com
... To: test@example.com
... Subject: elephant
...
... """)

>>> with event_subscribers(on_chain):
...     process(mlist, msg, {}, 'default-posting-chain')
<mailman.interfaces.chain.HoldEvent ...>
<mailman.chains.hold.HoldChain ...>
Subject: elephant
Hits:
    nonmember-moderation
Misses:
    dmarc-mitigation
    no-senders
    approved
    emergency
    loop
    banned-address
    member-moderation

>>> nonmember = mlist.nonmembers.get_member('bart@example.com')
>>> nonmember
<Member: bart@example.com on test@example.com as MemberRole.nonmember>

When a nonmember’s default moderation action is None, the rule will use the mailing list’s default_nonmember_action.

>>> print(nonmember.moderation_action)
None
>>> print(mlist.default_nonmember_action)
Action.hold

Rules

Rules are applied to each message as part of a rule chain. Individual rules simply return a boolean specifying whether the rule matches or not. Chain links determine what happens when a rule matches.

All rules

Rules are maintained in the configuration object as a dictionary mapping rule names to rule objects.

>>> from zope.interface.verify import verifyObject
>>> from mailman.interfaces.rules import IRule
>>> for rule_name in sorted(config.rules):
...     rule = config.rules[rule_name]
...     print(rule_name, verifyObject(IRule, rule))
administrivia True
any True
approved True
banned-address True
dmarc-mitigation True
emergency True
implicit-dest True
loop True
max-recipients True
max-size True
member-moderation True
news-moderation True
no-senders True
no-subject True
nonmember-moderation True
suspicious-header True
truth True

You can get a rule by name.

>>> rule = config.rules['emergency']
>>> verifyObject(IRule, rule)
True
Rule checks

Individual rules can be checked to see if they match, by running the rule’s check() method. This returns a boolean indicating whether the rule was matched or not.

>>> mlist = create_list('_xtest@example.com')
>>> msg = message_from_string("""\
... From: aperson@example.com
...
... An important message.
... """)

For example, the emergency rule just checks to see if the emergency flag is set on the mailing list, and the message has not been pre-approved by the list administrator.

>>> print(rule.name)
emergency
>>> mlist.emergency = False
>>> rule.check(mlist, msg, {})
False
>>> mlist.emergency = True
>>> rule.check(mlist, msg, {})
True
>>> rule.check(mlist, msg, dict(moderator_approved=True))
False
Administrivia

The administrivia rule matches when the message contains some common email commands in the Subject: header or first few lines of the payload. This is used to catch messages posted to the list which should have been sent to the -request robot address.

>>> mlist = create_list('_xtest@example.com')
>>> mlist.administrivia = True
>>> rule = config.rules['administrivia']
>>> print(rule.name)
administrivia

For example, if the Subject: header contains the word unsubscribe, the rule matches.

>>> msg_1 = message_from_string("""\
... From: aperson@example.com
... Subject: unsubscribe
...
... """)
>>> rule.check(mlist, msg_1, {})
True

Similarly, if the body of the message contains the word subscribe in the first few lines of text, the rule matches.

>>> msg_2 = message_from_string("""\
... From: aperson@example.com
... Subject: I wish to join your list
...
... subscribe
... """)
>>> rule.check(mlist, msg_2, {})
True

In both cases, administrivia checking can be disabled.

>>> mlist.administrivia = False
>>> rule.check(mlist, msg_1, {})
False
>>> rule.check(mlist, msg_2, {})
False

To make the administrivia heuristics a little more robust, the rule actually looks for a minimum and maximum number of arguments, so that it really does seem like a mis-addressed email command. In this case, the confirm command requires at least one argument. We don’t give that here so the rule will not match.

>>> mlist.administrivia = True
>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: confirm
...
... """)
>>> rule.check(mlist, msg, {})
False

But a real confirm message will match.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: confirm 12345
...
... """)
>>> rule.check(mlist, msg, {})
True

We don’t show all the other possible email commands, but you get the idea.

Non-administrivia

Of course, messages that don’t contain administrivia, don’t match the rule.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: examine
...
... persuade
... """)
>>> rule.check(mlist, msg, {})
False

Also, only text/plain parts are checked for administrivia, so any email commands in other content type subparts are ignored.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: some administrivia
... Content-Type: text/x-special
...
... subscribe
... """)
>>> rule.check(mlist, msg, {})
False
Pre-approved postings

Messages can contain a pre-approval, which is used to bypass the normal message approval queue. This has several use cases:

  • A list administrator can send an emergency message to the mailing list from an unregistered address, for example if they are away from their normal email.
  • An automated script can be programmed to send a message to an otherwise moderated list.

In order to support this, a mailing list can be given a moderator password which is shared among all the administrators.

>>> mlist = create_list('test@example.com')

This password will not be stored in clear text, so it must be hashed using the configured hash protocol.

>>> mlist.moderator_password = config.password_context.encrypt(
...     'super secret')

The approved rule determines whether the message contains the proper approval or not.

>>> rule = config.rules['approved']
>>> print(rule.name)
approved
No approval

The preferred header to check for approval is Approved:. If the message does not have this header, the rule will not match.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... An important message.
... """)
>>> rule.check(mlist, msg, {})
False

If the rule has an Approved header, but the value of this header does not match the moderator password, the rule will not match. Note that the header must contain the clear text version of the password.

>>> msg['Approved'] = 'not the password'
>>> rule.check(mlist, msg, {})
False
The message is approved

By adding an Approved header with a matching password, the rule will match.

>>> del msg['approved']
>>> msg['Approved'] = 'super secret'
>>> rule.check(mlist, msg, {})
True
Alternative headers

Other headers can be used to stash the moderator password. This rule also checks the Approve header.

>>> del msg['approved']
>>> msg['Approve'] = 'super secret'
>>> rule.check(mlist, msg, {})
True

Similarly, an X-Approved header can be used.

>>> del msg['approve']
>>> msg['X-Approved'] = 'super secret'
>>> rule.check(mlist, msg, {})
True

And finally, an X-Approve header can be used.

>>> del msg['x-approved']
>>> msg['X-Approve'] = 'super secret'
>>> rule.check(mlist, msg, {})
True
Removal of header

Technically, rules should not have side-effects, however this rule does remove the Approved header (LP: #973790) when it matches.

>>> del msg['x-approved']
>>> msg['Approved'] = 'super secret'
>>> rule.check(mlist, msg, {})
True
>>> print(msg['approved'])
None

It also removes the header when it doesn’t match. If the rule didn’t do this, then the mailing list could be probed for its moderator password.

>>> msg['Approved'] = 'not the password'
>>> rule.check(mlist, msg, {})
False
>>> print(msg['approved'])
None
Using a pseudo-header

Mail programs have varying degrees to which they support custom headers like Approved:. For this reason, Mailman also supports using a pseudo-header, which is really just the first non-whitespace line in the payload of the message. If this pseudo-header looks like a matching Approved: header, the message is similarly allowed to pass.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... Approved: super secret
... An important message.
... """)
>>> rule.check(mlist, msg, {})
True

The pseudo-header is always removed from the body of plain text messages.

>>> print(msg.as_string())
From: aperson@example.com
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
<BLANKLINE>
An important message.
<BLANKLINE>

As before, a mismatch in the pseudo-header does not approve the message, but the pseudo-header line is still removed.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... Approved: not the password
... An important message.
... """)
>>> rule.check(mlist, msg, {})
False

>>> print(msg.as_string())
From: aperson@example.com
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

An important message.
MIME multipart support

Mailman searches for the pseudo-header as the first non-whitespace line in the first text/plain message part of the message. This allows the feature to be used with MIME documents.

>>> msg = message_from_string("""\
... From: aperson@example.com
... MIME-Version: 1.0
... Content-Type: multipart/mixed; boundary="AAA"
...
... --AAA
... Content-Type: application/x-ignore
...
... Approved: not the password
... The above line will be ignored.
...
... --AAA
... Content-Type: text/plain
...
... Approved: super secret
... An important message.
... --AAA--
... """)
>>> rule.check(mlist, msg, {})
True

Like before, the pseudo-header is removed, but only from the text parts.

>>> print(msg.as_string())
From: aperson@example.com
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="AAA"
<BLANKLINE>
--AAA
Content-Type: application/x-ignore
<BLANKLINE>
Approved: not the password
The above line will be ignored.
<BLANKLINE>
--AAA
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
<BLANKLINE>
An important message.
--AAA--
<BLANKLINE>

If the correct password is in the non-text/plain part, it is ignored.

>>> msg = message_from_string("""\
... From: aperson@example.com
... MIME-Version: 1.0
... Content-Type: multipart/mixed; boundary="AAA"
...
... --AAA
... Content-Type: application/x-ignore
...
... Approved: super secret
... The above line will be ignored.
...
... --AAA
... Content-Type: text/plain
...
... Approved: not the password
... An important message.
... --AAA--
... """)
>>> rule.check(mlist, msg, {})
False

Pseudo-header is still stripped, but only from the text/plain part.

>>> print(msg.as_string())
From: aperson@example.com
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="AAA"
<BLANKLINE>
--AAA
Content-Type: application/x-ignore
<BLANKLINE>
Approved: super secret
The above line will be ignored.
<BLANKLINE>
--AAA
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
<BLANKLINE>
An important message.
--AAA--
Stripping text/html parts

Because some mail programs will include both a text/plain part and a text/html alternative, the rule must search the alternatives and strip anything that looks like an Approved: header.

>>> msg = message_from_string("""\
... From: aperson@example.com
... MIME-Version: 1.0
... Content-Type: multipart/mixed; boundary="AAA"
...
... --AAA
... Content-Type: text/html
...
... <html>
... <head></head>
... <body>
... <b>Approved: super secret</b>
... <p>The above line will be ignored.
... </body>
... </html>
...
... --AAA
... Content-Type: text/plain
...
... Approved: super secret
... An important message.
... --AAA--
... """)
>>> rule.check(mlist, msg, {})
True

And the header-like text in the text/html part was stripped.

>>> print(msg.as_string())
From: aperson@example.com
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="AAA"
<BLANKLINE>
--AAA
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Content-Type: text/html; charset="us-ascii"
<BLANKLINE>
<html>
<head></head>
<body>
<b></b>
<p>The above line will be ignored.
</body>
</html>
<BLANKLINE>
--AAA
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
<BLANKLINE>
An important message.
--AAA--
<BLANKLINE>

This is true even if the rule does not match (i.e. the incorrect password was given).

>>> msg = message_from_string("""\
... From: aperson@example.com
... MIME-Version: 1.0
... Content-Type: multipart/mixed; boundary="AAA"
...
... --AAA
... Content-Type: text/html
...
... <html>
... <head></head>
... <body>
... <b>Approved: not the password</b>
... <p>The above line will be ignored.
... </body>
... </html>
...
... --AAA
... Content-Type: text/plain
...
... Approved: not the password
... An important message.
... --AAA--
... """)
>>> rule.check(mlist, msg, {})
False

>>> print(msg.as_string())
From: aperson@example.com
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="AAA"

--AAA
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Content-Type: text/html; charset="us-ascii"

<html>
<head></head>
<body>
<b></b>
<p>The above line will be ignored.
</body>
</html>

--AAA
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"

An important message.
--AAA--
DMARC mitigation

This rule only matches in order to jump to the moderation chain to reject or discard the message. The rule looks at the list’s dmarc_mitigate_action and if it is other than no_mitigation, it checks the domain of the From: address for a DMARC policy. Depending on various settings, reject or discard the message, or just flag it for the dmarc handler to apply DMARC mitigations to the message.

>>> mlist = create_list('ant@example.com')
>>> rule = config.rules['dmarc-mitigation']
>>> print(rule.name)
dmarc-mitigation

First we set up a mock to return predictable responses to DNS lookups. This returns p=reject for the example.biz domain and not for any others.

>>> from mailman.rules.tests.test_dmarc import get_dns_resolver
>>> ignore = cleanups.enter_context(get_dns_resolver())

Use test data for the organizational domain suffixes.

>>> from mailman.rules.tests.test_dmarc import use_test_organizational_data
>>> cleanups.enter_context(use_test_organizational_data())

A message From: a domain without a DMARC policy does not set any flags.

>>> from mailman.interfaces.mailinglist import DMARCMitigateAction
>>> mlist.dmarc_mitigate_action = DMARCMitigateAction.munge_from
>>> msg = message_from_string("""\
... From: aperson@example.org
... To: ant@example.com
... Subject: A posted message
...
... """)
>>> msgdata = {}
>>> rule.check(mlist, msg, msgdata)
False
>>> msgdata
{}

Even if the From: domain publishes p=reject, no flags are set if the list’s action is no_mitigation.

>>> mlist.dmarc_mitigate_action = DMARCMitigateAction.no_mitigation
>>> msg = message_from_string("""\
... From: aperson@example.biz
... To: ant@example.com
... Subject: A posted message
...
... """)
>>> msgdata = {}
>>> rule.check(mlist, msg, msgdata)
False
>>> msgdata
{}

With a mitigation strategy chosen, the message is flagged.

>>> mlist.dmarc_mitigate_action = DMARCMitigateAction.munge_from
>>> msg = message_from_string("""\
... From: aperson@example.biz
... To: ant@example.com
... Subject: A posted message
...
... """)
>>> msgdata = {}
>>> rule.check(mlist, msg, msgdata)
False
>>> msgdata
{'dmarc': True}

Subdomains which don’t have a policy will check the organizational domain.

>>> msg = message_from_string("""\
... From: aperson@sub.domain.example.biz
... To: ant@example.com
... Subject: A posted message
...
... """)
>>> msgdata = {}
>>> rule.check(mlist, msg, msgdata)
False
>>> msgdata
{'dmarc': True}

The list’s action can also be set to immediately discard or reject the message.

>>> mlist.dmarc_mitigate_action = DMARCMitigateAction.discard
>>> msg = message_from_string("""\
... From: aperson@example.biz
... To: ant@example.com
... Subject: A posted message
... Message-ID: <xxx_message_id@example.biz>
...
... """)
>>> msgdata = {}
>>> rule.check(mlist, msg, msgdata)
True
>>> dump_msgdata(msgdata)
dmarc             : True
dmarc_action      : discard
moderation_reasons: ['DMARC moderation']
moderation_sender : aperson@example.biz

We can reject the message with a default reason.

>>> mlist.dmarc_mitigate_action = DMARCMitigateAction.reject
>>> msg = message_from_string("""\
... From: aperson@example.biz
... To: ant@example.com
... Subject: A posted message
... Message-ID: <xxx_message_id@example.biz>
...
... """)
>>> msgdata = {}
>>> rule.check(mlist, msg, msgdata)
True
>>> dump_msgdata(msgdata)
dmarc             : True
dmarc_action      : reject
moderation_reasons: ['You are not allowed to post to this mailing list...
moderation_sender : aperson@example.biz

And, we can reject with a custom message.

>>> mlist.dmarc_moderation_notice = 'A silly reason'
>>> msg = message_from_string("""\
... From: aperson@example.biz
... To: ant@example.com
... Subject: A posted message
... Message-ID: <xxx_message_id@example.biz>
...
... """)
>>> msgdata = {}
>>> rule.check(mlist, msg, msgdata)
True
>>> dump_msgdata(msgdata)
dmarc             : True
dmarc_action      : reject
moderation_reasons: ['A silly reason']
moderation_sender : aperson@example.biz
Emergency

When the mailing list has its emergency flag set, all messages posted to the list are held for moderator approval.

>>> mlist = create_list('test@example.com')
>>> rule = config.rules['emergency']
>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: My first post
... Message-ID: <first>
...
... An important message.
... """)

By default, the mailing list does not have its emergency flag set.

>>> mlist.emergency
False
>>> rule.check(mlist, msg, {})
False

The emergency rule matches if the flag is set on the mailing list.

>>> mlist.emergency = True
>>> rule.check(mlist, msg, {})
True

However, if the message metadata has a moderator_approved key set, then even if the mailing list has its emergency flag set, the message still goes through to the membership.

>>> rule.check(mlist, msg, dict(moderator_approved=True))
False
Header matching

Mailman can do pattern based header matching during its normal rule processing. There is a set of site-wide default header matches specified in the configuration file under the [antispam] section.

>>> mlist = create_list('test@example.com')

In this section, the variable header_checks contains a list of the headers to check, and the patterns to check them against. By default, this list is empty.

It is also possible to programmatically extend these header checks. Here, we’ll extend the checks with a pattern that matches 4 or more stars.

>>> chain = config.chains['header-match']
>>> chain.extend('x-spam-score', '[*]{4,}')

First, if the message has no X-Spam-Score: header, the message passes through the chain with no matches.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: Not spam
... Message-ID: <ant>
...
... This is a message.
... """)

By looking at the message metadata after chain processing, we can see that none of the rules matched.

>>> from mailman.core.chains import process
>>> msgdata = {}
>>> process(mlist, msg, msgdata, 'header-match')
>>> hits_and_misses(msgdata)
No rules hit
Rule misses:
    x-spam-score: [*]{4,}

The header may exist but does not match the pattern.

>>> msg['X-Spam-Score'] = '***'
>>> msgdata = {}
>>> process(mlist, msg, msgdata, 'header-match')
>>> hits_and_misses(msgdata)
No rules hit
Rule misses:
    x-spam-score: [*]{4,}

The header may exist and match the pattern. By default, when the header matches, it gets held for moderator approval.

>>> from mailman.interfaces.chain import ChainEvent
>>> from mailman.testing.helpers import event_subscribers
>>> def handler(event):
...     if isinstance(event, ChainEvent):
...         print(event.__class__.__name__,
...               event.chain.name, event.msg['message-id'])

>>> del msg['x-spam-score']
>>> msg['X-Spam-Score'] = '*****'
>>> msgdata = {}
>>> with event_subscribers(handler):
...     process(mlist, msg, msgdata, 'header-match')
HoldEvent hold <ant>

>>> hits_and_misses(msgdata)
Rule hits:
    x-spam-score: [*]{4,}
No rules missed

The configuration file can also specify a different final disposition for messages that match their header checks. For example, we may just want to discard such messages.

>>> from mailman.testing.helpers import configuration
>>> msgdata = {}
>>> with event_subscribers(handler):
...     with configuration('antispam', jump_chain='discard'):
...         process(mlist, msg, msgdata, 'header-match')
DiscardEvent discard <ant>

These programmatically added headers can be removed by flushing the chain. Now, nothing with match this message.

>>> chain.flush()
>>> msgdata = {}
>>> process(mlist, msg, msgdata, 'header-match')
>>> hits_and_misses(msgdata)
No rules hit
No rules missed
List-specific header matching

Each mailing list can also be configured with a set of header matching regular expression rules. These can be used to impose list-specific header filtering with the same semantics as the global [antispam] section, or to have a different action.

To follow the global antispam action, the header match rule must not specify a chain to jump to. If the default antispam action is changed in the configuration file and Mailman is restarted, those rules will get the new jump action.

The list administrator wants to match not on four stars, but on three plus signs, but only for the current mailing list.

>>> from mailman.interfaces.mailinglist import IHeaderMatchList
>>> header_matches = IHeaderMatchList(mlist)
>>> header_matches.append('x-spam-score', '[+]{3,}')

A message with a spam score of two pluses does not match.

>>> msgdata = {}
>>> del msg['x-spam-score']
>>> msg['X-Spam-Score'] = '++'
>>> process(mlist, msg, msgdata, 'header-match')
>>> hits_and_misses(msgdata)
No rules hit
Rule misses:
    x-spam-score: [+]{3,}

But a message with a spam score of three pluses does match. Because a message with the previous Message-Id is already in the moderation queue, we need to give this message a new Message-Id.

>>> msgdata = {}
>>> del msg['x-spam-score']
>>> msg['X-Spam-Score'] = '+++'
>>> del msg['message-id']
>>> msg['Message-Id'] = '<bee>'
>>> process(mlist, msg, msgdata, 'header-match')
>>> hits_and_misses(msgdata)
Rule hits:
    x-spam-score: [+]{3,}
No rules missed

As does a message with a spam score of four pluses.

>>> msgdata = {}
>>> del msg['x-spam-score']
>>> msg['X-Spam-Score'] = '++++'
>>> del msg['message-id']
>>> msg['Message-Id'] = '<cat>'
>>> process(mlist, msg, msgdata, 'header-match')
>>> hits_and_misses(msgdata)
Rule hits:
    x-spam-score: [+]{3,}
No rules missed

Now, the list administrator wants to match on three plus signs, but wants those emails to be discarded instead of held.

>>> header_matches.remove('x-spam-score', '[+]{3,}')
>>> header_matches.append('x-spam-score', '[+]{3,}', 'discard')

A message with a spam score of three pluses will still match, and the message will be discarded.

>>> msgdata = {}
>>> del msg['x-spam-score']
>>> msg['X-Spam-Score'] = '+++'
>>> del msg['message-id']
>>> msg['Message-Id'] = '<dog>'
>>> with event_subscribers(handler):
...     process(mlist, msg, msgdata, 'header-match')
DiscardEvent discard <dog>
>>> hits_and_misses(msgdata)
Rule hits:
    x-spam-score: [+]{3,}
No rules missed
Implicit destination

The implicit-dest rule matches when the mailing list’s posting address is not explicitly mentioned in the set of message recipients.

>>> mlist = create_list('_xtest@example.com')
>>> rule = config.rules['implicit-dest']
>>> print(rule.name)
implicit-dest

In order to check for implicit destinations, we need to adapt the mailing list to the appropriate interface.

>>> from mailman.interfaces.mailinglist import IAcceptableAliasSet
>>> alias_set = IAcceptableAliasSet(mlist)

This rule matches messages that have an implicit destination, meaning that the mailing list’s posting address isn’t included in the explicit recipients.

>>> mlist.require_explicit_destination = True
>>> alias_set.clear()

>>> msg = message_from_string("""\
... From: aperson@example.org
... Subject: An implicit message
...
... """)
>>> rule.check(mlist, msg, {})
True

You can disable implicit destination checks for the mailing list.

>>> mlist.require_explicit_destination = False
>>> rule.check(mlist, msg, {})
False

Even with some recipients, if the posting address is not included, the rule will match.

>>> mlist.require_explicit_destination = True
>>> msg['To'] = 'myfriend@example.com'
>>> rule.check(mlist, msg, {})
True

Add the posting address as a recipient and the rule will no longer match.

>>> msg['Cc'] = '_xtest@example.com'
>>> rule.check(mlist, msg, {})
False

Alternatively, if one of the acceptable aliases is in the recipients list, then the rule will not match.

>>> del msg['cc']
>>> rule.check(mlist, msg, {})
True

>>> alias_set.add('myfriend@example.com')
>>> rule.check(mlist, msg, {})
False

A message gated from NNTP will obviously have an implicit destination. Such gated messages will not be held for implicit destination because it’s assumed that Mailman pulled it from the appropriate news group.

>>> rule.check(mlist, msg, dict(from_usenet=True))
False

Additional aliases can be added.

>>> alias_set.add('other@example.com')
>>> del msg['to']
>>> rule.check(mlist, msg, {})
True

>>> msg['To'] = 'other@example.com'
>>> rule.check(mlist, msg, {})
False

Aliases can be removed.

>>> alias_set.remove('other@example.com')
>>> rule.check(mlist, msg, {})
True

Aliases can also be cleared.

>>> msg['Cc'] = 'myfriend@example.com'
>>> rule.check(mlist, msg, {})
False

>>> alias_set.clear()
>>> rule.check(mlist, msg, {})
True
Alias patterns

It’s also possible to specify an alias pattern, i.e. a regular expression to match against the recipients. For example, we can say that if there is a recipient in the example.net domain, then the rule does not match.

>>> alias_set.add('^.*@example.net')
>>> rule.check(mlist, msg, {})
True

>>> msg['To'] = 'you@example.net'
>>> rule.check(mlist, msg, {})
False
Bad aliases

You cannot add an alias that looks like neither a pattern nor an email address.

>>> alias_set.add('foobar')
Traceback (most recent call last):
...
ValueError: foobar
Posting loops

To avoid a posting loop, Mailman has a rule to check for the existence of an RFC 2369 List-Post: header with the value of the list’s posting address.

>>> mlist = create_list('_xtest@example.com')
>>> rule = config.rules['loop']
>>> print(rule.name)
loop

The header could be missing, in which case the rule does not match.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... An important message.
... """)
>>> rule.check(mlist, msg, {})
False

The header could be present, but not match the list’s posting address.

>>> msg['List-Post'] = 'not-this-list@example.com'
>>> rule.check(mlist, msg, {})
False

If the header is present and does match the posting address, the rule matches.

>>> del msg['list-post']
>>> msg['List-Post'] = mlist.posting_address
>>> rule.check(mlist, msg, {})
True

Even if there are multiple List-Post: headers, as long as one with the posting address exists, the rule matches.

>>> msg = message_from_string("""\
... From: aperson@example.com
... List-Post: not-this-list@example.com
... List-Post: _xtest@example.com
... List-Post: foo@example.com
...
... An important message.
... """)
>>> rule.check(mlist, msg, {})
True
Message size

The message-size rule matches when the posted message is bigger than a specified maximum. Generally this is used to prevent huge attachments from getting posted to the list. This value is calculated in terms of KB (1024 bytes).

>>> mlist = create_list('_xtest@example.com')
>>> rule = config.rules['max-size']
>>> print(rule.name)
max-size

For example, setting the maximum message size to 1 means that any message bigger than that will match the rule.

>>> mlist.max_message_size = 1 # 1024 bytes
>>> one_line = 'x' * 79
>>> big_body = '\n'.join([one_line] * 15)
>>> msg = message_from_string("""\
... From: aperson@example.com
... To: _xtest@example.com
...
... """ + big_body)
>>> rule.check(mlist, msg, {})
True

Setting the maximum message size to zero means no size check is performed.

>>> mlist.max_message_size = 0
>>> rule.check(mlist, msg, {})
False

Of course, if the maximum size is larger than the message’s size, then it’s still okay.

>>> mlist.max_message_size = msg.original_size/1024.0 + 1
>>> rule.check(mlist, msg, {})
False
Moderation

All members and nonmembers have a moderation action, which defaults to the appropriate list’s default action. When the action is not defer, the moderation rule flags the message as needing moderation. This might be to automatically accept, discard, reject, or hold the message.

Two separate rules check for member and nonmember moderation. Member moderation happens early in the built-in chain, while nonmember moderation happens later in the chain, after normal moderation checks.

>>> mlist = create_list('test@example.com')
Member moderation 
>>> member_rule = config.rules['member-moderation']
>>> print(member_rule.name)
member-moderation

Anne, a mailing list member, sends a message to the mailing list. Her moderation action is not set.

>>> from mailman.testing.helpers import subscribe
>>> member = subscribe(mlist, 'Anne')
>>> member
<Member: Anne Person <aperson@example.com> on test@example.com
         as MemberRole.member>
>>> print(member.moderation_action)
None

Because the list’s default member action is set to defer, Anne’s posting is not moderated.

>>> print(mlist.default_member_action)
Action.defer

Because Anne is not moderated, the member moderation rule does not match.

>>> member_msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: A posted message
...
... """)
>>> member_rule.check(mlist, member_msg, {})
False

Once the member’s moderation action is set to something other than defer or None (given the list’s current default member moderation action), the rule matches. Also, the message metadata has a few extra pieces of information for the eventual moderation chain.

>>> from mailman.interfaces.action import Action
>>> member.moderation_action = Action.hold
>>> msgdata = {}
>>> member_rule.check(mlist, member_msg, msgdata)
True
>>> dump_msgdata(msgdata)
member_moderation_action: hold
moderation_reasons      : ['The message comes from a moderated member']
moderation_sender       : aperson@example.com
Nonmembers

Nonmembers are handled in a similar way, although by default, nonmember postings are held for moderator approval.

>>> nonmember_rule = config.rules['nonmember-moderation']
>>> print(nonmember_rule.name)
nonmember-moderation

Bart, who is not a member of the mailing list, sends a message to the list. He has no explicit nonmember moderation action.

>>> from mailman.interfaces.member import MemberRole
>>> nonmember = subscribe(mlist, 'Bart', MemberRole.nonmember)
>>> nonmember
<Member: Bart Person <bperson@example.com> on test@example.com
         as MemberRole.nonmember>
>>> print(nonmember.moderation_action)
None

The list’s default nonmember moderation action is to hold postings by nonmembers.

>>> print(mlist.default_nonmember_action)
Action.hold

Since Bart is registered as a nonmember of the list, and his moderation action is set to None, the action falls back to the list’s default nonmember moderation action, which is to hold the post for moderator approval. Thus the rule matches and the message metadata again carries some useful information.

>>> nonmember_msg = message_from_string("""\
... From: bperson@example.com
... To: test@example.com
... Subject: A posted message
...
... """)
>>> msgdata = {}
>>> nonmember_rule.check(mlist, nonmember_msg, msgdata)
True
>>> dump_msgdata(msgdata)
member_moderation_action: hold
moderation_reasons      : ['The message is not from a list member']
moderation_sender       : bperson@example.com

Of course, the nonmember action can be set to defer the decision, in which case the rule does not match.

>>> nonmember.moderation_action = Action.defer
>>> nonmember_rule.check(mlist, nonmember_msg, {})
False
Unregistered nonmembers

The incoming runner ensures that all sender addresses are registered in the system, but it is the moderation rule that subscribes nonmember addresses to the mailing list if they are not already subscribed.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> address = getUtility(IUserManager).create_address(
...     'cperson@example.com')
>>> address
<Address: cperson@example.com [not verified] at ...>

>>> msg = message_from_string("""\
... From: cperson@example.com
... To: test@example.com
... Subject: A posted message
...
... """)

cperson is neither a member, nor a nonmember of the mailing list.

>>> def memberkey(member):
...     return member.mailing_list, member.address.email, member.role.value

>>> dump_list(mlist.members.members, key=memberkey)
<Member: Anne Person <aperson@example.com>
         on test@example.com as MemberRole.member>
>>> dump_list(mlist.nonmembers.members, key=memberkey)
<Member: Bart Person <bperson@example.com>
         on test@example.com as MemberRole.nonmember>

However, when the nonmember moderation rule runs, it adds the cperson as a nonmember of the list. The rule also matches.

>>> msgdata = {}
>>> nonmember_rule.check(mlist, msg, msgdata)
True
>>> dump_msgdata(msgdata)
member_moderation_action: hold
moderation_reasons      : ['The message is not from a list member']
moderation_sender       : cperson@example.com

>>> dump_list(mlist.members.members, key=memberkey)
<Member: Anne Person <aperson@example.com>
         on test@example.com as MemberRole.member>
>>> dump_list(mlist.nonmembers.members, key=memberkey)
<Member: Bart Person <bperson@example.com>
         on test@example.com as MemberRole.nonmember>
<Member: cperson@example.com
         on test@example.com as MemberRole.nonmember>
Cross-membership checks

Of course, the member moderation rule does not match for nonmembers…

>>> member_rule.check(mlist, nonmember_msg, {})
False
>>> nonmember_rule.check(mlist, member_msg, {})
False
Newsgroup moderation

The news-moderation rule matches all messages posted to mailing lists that gateway to a moderated newsgroup. The reason for this is that such messages must get forwarded on to the newsgroup moderator. From there it will get posted to the newsgroup, and from there, gated to the mailing list. It’s a circuitous route, but it works nonetheless by holding all messages posted directly to the mailing list.

>>> mlist = create_list('_xtest@example.com')
>>> rule = config.rules['news-moderation']
>>> print(rule.name)
news-moderation

Set the list configuration variable to enable newsgroup moderation.

>>> from mailman.interfaces.nntp import NewsgroupModeration
>>> mlist.newsgroup_moderation = NewsgroupModeration.moderated

And now all messages will match the rule.

>>> msg = message_from_string("""\
... From: aperson@example.org
... Subject: An announcement
...
... Great things are happening.
... """)
>>> rule.check(mlist, msg, {})
True

When moderation is turned off, the rule does not match.

>>> mlist.newsgroup_moderation = NewsgroupModeration.none
>>> rule.check(mlist, msg, {})
False
No Subject header

This rule matches if the message has no Subject: header, or if the header is the empty string when stripped.

>>> mlist = create_list('_xtest@example.com')
>>> rule = config.rules['no-subject']
>>> print(rule.name)
no-subject

A message with a non-empty subject does not match the rule.

>>> msg = message_from_string("""\
... From: aperson@example.org
... To: _xtest@example.com
... Subject: A posted message
...
... """)
>>> rule.check(mlist, msg, {})
False

Delete the Subject: header and the rule matches.

>>> del msg['subject']
>>> rule.check(mlist, msg, {})
True

Even a Subject: header with only whitespace still matches the rule.

>>> msg['Subject'] = '    '
>>> rule.check(mlist, msg, {})
True
Maximum number of recipients

This rule matches when there are more than the maximum allowed number of explicit recipients addressed by the message.

>>> mlist = create_list('_xtest@example.com')
>>> rule = config.rules['max-recipients']
>>> print(rule.name)
max-recipients

In this case, we’ll create a message with five recipients. These include all addresses in the To: and CC: headers.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: _xtest@example.com, bperson@example.com
... Cc: cperson@example.com
... Cc: dperson@example.com (Dan Person)
... To: Elly Q. Person <eperson@example.com>
...
... Hey folks!
... """)

For backward compatibility, the message must have fewer than the maximum number of explicit recipients.

>>> mlist.max_num_recipients = 5
>>> rule.check(mlist, msg, {})
True

>>> mlist.max_num_recipients = 6
>>> rule.check(mlist, msg, {})
False

Zero means any number of recipients are allowed.

>>> mlist.max_num_recipients = 0
>>> rule.check(mlist, msg, {})
False
Suspicious headers

Suspicious headers are a way for Mailman to hold messages that match a particular regular expression. This mostly historical feature is fairly confusing to users, and the list attribute that controls this is misnamed.

>>> mlist = create_list('_xtest@example.com')
>>> rule = config.rules['suspicious-header']
>>> print(rule.name)
suspicious-header

Set the so-called suspicious header configuration variable.

>>> mlist.bounce_matching_headers = 'From: .*person@(blah.)?example.com'
>>> msg = message_from_string("""\
... From: aperson@example.com
... To: _xtest@example.com
... Subject: An implicit message
...
... """)
>>> rule.check(mlist, msg, {})
True

But if the header doesn’t match the regular expression, the rule won’t match. This one comes from a .org address.

>>> msg = message_from_string("""\
... From: aperson@example.org
... To: _xtest@example.com
... Subject: An implicit message
...
... """)
>>> rule.check(mlist, msg, {})
False
Truth

This rule always matches. This makes it useful as a terminus rule for unconditionally jumping to another chain.

>>> rule = config.rules['truth']
>>> rule.check(False, False, False)
True

Handlers

Acknowledgment headers

Messages that flow through the global pipeline get their headers cooked, which basically means that their headers go through several mostly unrelated transformations. Some headers get added, others get changed. Some of these changes depend on mailing list settings and others depend on how the message is getting sent through the system. We’ll take things one-by-one.

>>> mlist = create_list('_xtest@example.com')
>>> mlist.subject_prefix = ''

When the message’s metadata has a noack key set, an X-Ack: no header is added.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... A message of great import.
... """)

>>> from mailman.handlers.cook_headers import process
>>> process(mlist, msg, dict(noack=True))
>>> print(msg.as_string())
From: aperson@example.com
X-Ack: no
...

Any existing X-Ack header in the original message is removed.

>>> msg = message_from_string("""\
... X-Ack: yes
... From: aperson@example.com
...
... A message of great import.
... """)
>>> process(mlist, msg, dict(noack=True))
>>> print(msg.as_string())
From: aperson@example.com
X-Ack: no
...
Message acknowledgment

When a user posts a message to a mailing list, and that user has chosen to receive acknowledgments of their postings, Mailman will sent them such an acknowledgment.

>>> mlist = create_list('test@example.com')
>>> mlist.display_name = 'Test'
>>> mlist.preferred_language = 'en'
>>> mlist.send_welcome_message = False
>>> # XXX This will almost certainly change once we've worked out the web
>>> # space layout for mailing lists now.

>>> # Ensure that the virgin queue is empty, since we'll be checking this
>>> # for new auto-response messages.
>>> from mailman.testing.helpers import get_queue_messages
>>> get_queue_messages('virgin')
[]

Subscribe a user to the mailing list.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)

>>> from mailman.interfaces.member import MemberRole
>>> user_1 = user_manager.create_user('aperson@example.com')
>>> address_1 = list(user_1.addresses)[0]
>>> mlist.subscribe(address_1, MemberRole.member)
<Member: aperson@example.com on test@example.com as MemberRole.member>
Non-member posts

Non-members can’t get acknowledgments of their posts to the mailing list.

>>> msg = message_from_string("""\
... From: bperson@example.com
...
... """)

>>> handler = config.handlers['acknowledge']
>>> handler.process(mlist, msg, {})
>>> get_queue_messages('virgin')
[]

We can also specify the original sender in the message’s metadata. If that person is also not a member, no acknowledgment will be sent either.

>>> msg = message_from_string("""\
... From: bperson@example.com
...
... """)
>>> handler.process(mlist, msg,
...     dict(original_sender='cperson@example.com'))
>>> get_queue_messages('virgin')
[]
No acknowledgment requested

Unless the user has requested acknowledgments, they will not get one.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... """)
>>> handler.process(mlist, msg, {})
>>> get_queue_messages('virgin')
[]

Similarly if the original sender is specified in the message metadata, and that sender is a member but not one who has requested acknowledgments, none will be sent.

>>> user_2 = user_manager.create_user('dperson@example.com')
>>> address_2 = list(user_2.addresses)[0]
>>> mlist.subscribe(address_2, MemberRole.member)
<Member: dperson@example.com on test@example.com as MemberRole.member>

>>> handler.process(mlist, msg,
...     dict(original_sender='dperson@example.com'))
>>> get_queue_messages('virgin')
[]
Requested acknowledgments

If the member requests acknowledgments, Mailman will send them one when they post to the mailing list.

>>> user_1.preferences.acknowledge_posts = True

The receipt will include the original message’s subject in the response body,

>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: Something witty and insightful
...
... """)
>>> handler.process(mlist, msg, {})
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> dump_msgdata(messages[0].msgdata)
_parsemsg           : False
listid              : test.example.com
nodecorate          : True
recipients          : {'aperson@example.com'}
reduced_list_headers: True
...
>>> print(messages[0].msg.as_string())
...
MIME-Version: 1.0
...
Subject: Test post acknowledgment
From: test-bounces@example.com
To: aperson@example.com
...
Precedence: bulk
<BLANKLINE>
Your message entitled
<BLANKLINE>
    Something witty and insightful
<BLANKLINE>
was successfully received by the Test mailing list.

If there is no subject, then the receipt will use a generic message.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... """)
>>> handler.process(mlist, msg, {})
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> dump_msgdata(messages[0].msgdata)
_parsemsg           : False
listid              : test.example.com
nodecorate          : True
recipients          : {'aperson@example.com'}
reduced_list_headers: True
...
>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
...
Subject: Test post acknowledgment
From: test-bounces@example.com
To: aperson@example.com
...
Precedence: bulk
<BLANKLINE>
Your message entitled
<BLANKLINE>
    (no subject)
<BLANKLINE>
was successfully received by the Test mailing list.
After delivery

After a message is delivered, or more correctly, after it has been processed by the rest of the handlers in the incoming queue pipeline, a couple of bookkeeping pieces of information are updated.

>>> from datetime import timedelta
>>> from mailman.utilities.datetime import now
>>> mlist = create_list('_xtest@example.com')
>>> post_time = now() - timedelta(minutes=10)
>>> mlist.last_post_time = post_time
>>> mlist.post_id = 10

Processing a message with this handler updates the last_post_time and post_id attributes.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... Something interesting.
... """)

>>> handler = config.handlers['after-delivery']
>>> handler.process(mlist, msg, {})
>>> mlist.last_post_time > post_time
True
>>> mlist.post_id
11
Archives

Updating the archives with posted messages is handled by a separate queue, which allows for better memory management and prevents blocking the main delivery processes while messages are archived. This also allows external archivers to work in a separate process from the main Mailman delivery processes.

>>> handler = config.handlers['to-archive']
>>> mlist = create_list('_xtest@example.com')
>>> switchboard = config.switchboards['archive']

A helper function.

>>> def clear():
...     for filebase in switchboard.files:
...         msg, msgdata = switchboard.dequeue(filebase)
...         switchboard.finish(filebase)

The purpose of this handler is to make a simple decision as to whether the message should get archived and if so, to drop the message in the archiving queue. Really the most important things are to determine when a message should not get archived.

For example, no digests should ever get archived.

>>> from mailman.interfaces.archiver import ArchivePolicy
>>> mlist.archive_policy = ArchivePolicy.public
>>> msg = message_from_string("""\
... Subject: A sample message
...
... A message of great import.
... """)
>>> handler.process(mlist, msg, dict(isdigest=True))
>>> switchboard.files
[]

If the mailing list is not configured to archive, then even regular deliveries won’t be archived.

>>> mlist.archive_policy = ArchivePolicy.never
>>> handler.process(mlist, msg, {})
>>> switchboard.files
[]

There are two de-facto standards for a message to indicate that it does not want to be archived. We’ve seen both in the wild so both are supported. The X-No-Archive: header can be used to indicate that the message should not be archived. Confusingly, this header’s value is actually ignored.

>>> mlist.archive_policy = ArchivePolicy.public
>>> msg = message_from_string("""\
... Subject: A sample message
... X-No-Archive: YES
...
... A message of great import.
... """)
>>> handler.process(mlist, msg, dict(isdigest=True))
>>> switchboard.files
[]

Even a no value will stop the archiving of the message.

>>> msg = message_from_string("""\
... Subject: A sample message
... X-No-Archive: No
...
... A message of great import.
... """)
>>> handler.process(mlist, msg, dict(isdigest=True))
>>> switchboard.files
[]

Another header that’s been observed is the X-Archive: header. Here, the header’s case folded value must be no in order to prevent archiving.

>>> msg = message_from_string("""\
... Subject: A sample message
... X-Archive: No
...
... A message of great import.
... """)
>>> handler.process(mlist, msg, dict(isdigest=True))
>>> switchboard.files
[]

But if the value is yes, then the message will be archived.

>>> msg = message_from_string("""\
... Subject: A sample message
... X-Archive: Yes
...
... A message of great import.
... """)
>>> handler.process(mlist, msg, {})
>>> len(switchboard.files)
1
>>> filebase = switchboard.files[0]
>>> qmsg, qdata = switchboard.dequeue(filebase)
>>> switchboard.finish(filebase)
>>> print(qmsg.as_string())
Subject: A sample message
X-Archive: Yes
<BLANKLINE>
A message of great import.
<BLANKLINE>
>>> dump_msgdata(qdata)
_parsemsg: False
version  : 3

Without either archiving header, and all other things being the same, the message will get archived.

>>> msg = message_from_string("""\
... Subject: A sample message
...
... A message of great import.
... """)
>>> handler.process(mlist, msg, {})
>>> len(switchboard.files)
1
>>> filebase = switchboard.files[0]
>>> qmsg, qdata = switchboard.dequeue(filebase)
>>> switchboard.finish(filebase)
>>> print(qmsg.as_string())
Subject: A sample message
<BLANKLINE>
A message of great import.
<BLANKLINE>
>>> dump_msgdata(qdata)
_parsemsg: False
version  : 3
Avoid duplicates

This handler implements several strategies to reduce the reception of duplicate messages. It does this by removing certain recipients from the list of recipients calculated earlier.

>>> mlist = create_list('_xtest@example.com')

Create some members we’re going to use.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)

>>> address_a = user_manager.create_address('aperson@example.com')
>>> address_b = user_manager.create_address('bperson@example.com')

>>> from mailman.interfaces.member import MemberRole
>>> member_a = mlist.subscribe(address_a, MemberRole.member)
>>> member_b = mlist.subscribe(address_b, MemberRole.member)
>>> # This is the message metadata dictionary as it would be produced by
>>> # the CalcRecips handler.
>>> recips = dict(
...     recipients=['aperson@example.com', 'bperson@example.com'])
Short circuiting

The module short-circuits if there are no recipients.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: A message of great import
...
... Something
... """)
>>> msgdata = {}

>>> handler = config.handlers['avoid-duplicates']
>>> handler.process(mlist, msg, msgdata)
>>> msgdata
{}
>>> print(msg.as_string())
From: aperson@example.com
Subject: A message of great import

Something
Suppressing the list copy

Members can elect not to receive a list copy of any message on which they are explicitly named as a recipient. This is done by setting their receive_list_copy preference to False. However, if they aren’t mentioned in one of the recipient headers (i.e. To, CC, Resent-To, or Resent-CC), then they will get a list copy.

>>> member_a.preferences.receive_list_copy = False
>>> msg = message_from_string("""\
... From: Claire Person <cperson@example.com>
...
... Something of great import.
... """)
>>> msgdata = recips.copy()
>>> handler.process(mlist, msg, msgdata)
>>> sorted(msgdata['recipients'])
['aperson@example.com', 'bperson@example.com']
>>> print(msg.as_string())
From: Claire Person <cperson@example.com>
<BLANKLINE>
Something of great import.
<BLANKLINE>

If they’re mentioned on the CC line, they won’t get a list copy.

>>> msg = message_from_string("""\
... From: Claire Person <cperson@example.com>
... CC: aperson@example.com
...
... Something of great import.
... """)
>>> msgdata = recips.copy()
>>> handler.process(mlist, msg, msgdata)
>>> sorted(msgdata['recipients'])
['bperson@example.com']
>>> print(msg.as_string())
From: Claire Person <cperson@example.com>
CC: aperson@example.com
<BLANKLINE>
Something of great import.
<BLANKLINE>

But if they’re mentioned on the CC line and have receive_list_copy set to True (the default), then they still get a list copy.

>>> msg = message_from_string("""\
... From: Claire Person <cperson@example.com>
... CC: bperson@example.com
...
... Something of great import.
... """)
>>> msgdata = recips.copy()
>>> handler.process(mlist, msg, msgdata)
>>> sorted(msgdata['recipients'])
['aperson@example.com', 'bperson@example.com']
>>> print(msg.as_string())
From: Claire Person <cperson@example.com>
CC: bperson@example.com
<BLANKLINE>
Something of great import.
<BLANKLINE>

Other headers checked for recipients include the To…

>>> msg = message_from_string("""\
... From: Claire Person <cperson@example.com>
... To: aperson@example.com
...
... Something of great import.
... """)
>>> msgdata = recips.copy()
>>> handler.process(mlist, msg, msgdata)
>>> sorted(msgdata['recipients'])
['bperson@example.com']
>>> print(msg.as_string())
From: Claire Person <cperson@example.com>
To: aperson@example.com
<BLANKLINE>
Something of great import.
<BLANKLINE>

… Resent-To …

>>> msg = message_from_string("""\
... From: Claire Person <cperson@example.com>
... Resent-To: aperson@example.com
...
... Something of great import.
... """)
>>> msgdata = recips.copy()
>>> handler.process(mlist, msg, msgdata)
>>> sorted(msgdata['recipients'])
['bperson@example.com']
>>> print(msg.as_string())
From: Claire Person <cperson@example.com>
Resent-To: aperson@example.com
<BLANKLINE>
Something of great import.
<BLANKLINE>

…and Resent-CC headers.

>>> msg = message_from_string("""\
... From: Claire Person <cperson@example.com>
... Resent-Cc: aperson@example.com
...
... Something of great import.
... """)
>>> msgdata = recips.copy()
>>> handler.process(mlist, msg, msgdata)
>>> sorted(msgdata['recipients'])
['bperson@example.com']
>>> print(msg.as_string())
From: Claire Person <cperson@example.com>
Resent-Cc: aperson@example.com
<BLANKLINE>
Something of great import.
<BLANKLINE>
Cleansing headers

All messages posted to a list get their headers cleansed. Some headers are related to additional permissions that can be granted to the message and other headers can be used to fish for membership.

>>> mlist = create_list('_xtest@example.com')

Headers such as Approved, Approve, (as well as their X- variants) and Urgent are used to grant special permissions to individual messages. All may contain a password; the first two headers are used by list administrators to pre-approve a message normal held for approval. The latter header is used to send a regular message to all members, regardless of whether they get digests or not. Because all three headers contain passwords, they must be removed from any posted message.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Approved: foobar
... Approve: barfoo
... X-Approved: bazbar
... X-Approve: barbaz
... Urgent: notreally
... Subject: A message of great import
...
... Blah blah blah
... """)

>>> handler = config.handlers['cleanse']
>>> handler.process(mlist, msg, {})
>>> print(msg.as_string())
From: aperson@example.com
Subject: A message of great import

Blah blah blah

Other headers can be used by list members to fish the list for membership, so we don’t let them go through. These are a mix of standard headers and custom headers supported by some mail readers. For example, X-PMRC is supported by Pegasus mail. I don’t remember what program uses X-Confirm-Reading-To though (Some Microsoft product perhaps?).

>>> msg = message_from_string("""\
... From: bperson@example.com
... Reply-To: bperson@example.org
... Sender: asystem@example.net
... Return-Receipt-To: another@example.com
... Disposition-Notification-To: athird@example.com
... X-Confirm-Reading-To: afourth@example.com
... X-PMRQC: afifth@example.com
... Subject: a message to you
...
... How are you doing?
... """)
>>> handler.process(mlist, msg, {})
>>> print(msg.as_string())
From: bperson@example.com
Reply-To: bperson@example.org
Sender: asystem@example.net
Subject: a message to you
<BLANKLINE>
How are you doing?
<BLANKLINE>
Anonymous lists

Anonymous mailing lists also try to cleanse certain identifying headers from the original posting, so that it is at least a bit more difficult to determine who sent the message. This isn’t perfect though, for example, the body of the messages are never scrubbed (though that might not be a bad idea). The From and Reply-To headers in the posted message are taken from list attributes.

Hotmail apparently sets X-Originating-Email.

>>> mlist.anonymous_list = True
>>> mlist.description = 'A Test Mailing List'
>>> mlist.preferred_language = 'en'
>>> msg = message_from_string("""\
... From: bperson@example.com
... Reply-To: bperson@example.org
... Sender: asystem@example.net
... X-Originating-Email: cperson@example.com
... Subject: a message to you
...
... How are you doing?
... """)
>>> handler.process(mlist, msg, {})
>>> print(msg.as_string())
Subject: a message to you
From: A Test Mailing List <_xtest@example.com>
Reply-To: _xtest@example.com
<BLANKLINE>
How are you doing?
<BLANKLINE>
Cooking headers

Messages that flow through the global pipeline get their headers ‘cooked’, which basically means that their headers go through several mostly unrelated transformations. Some headers get added, others get changed. Some of these changes depend on mailing list settings and others depend on how the message is getting sent through the system. We’ll take things one-by-one.

>>> mlist = create_list('test@example.com')
>>> mlist.subject_prefix = ''
Saving the original sender

Because the original sender headers may get deleted or changed, this handler will place the sender in the message metadata for safe keeping.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... A message of great import.
... """)
>>> msgdata = {}

>>> from mailman.handlers.cook_headers import process
>>> process(mlist, msg, msgdata)
>>> print(msgdata['original_sender'])
aperson@example.com

But if there was no original sender, then the empty string will be saved.

>>> msg = message_from_string("""\
... Subject: No original sender
...
... A message of great import.
... """)
>>> msgdata = {}
>>> process(mlist, msg, msgdata)
>>> print(msgdata['original_sender'])
<BLANKLINE>
Mailman version header

Mailman will also insert an X-Mailman-Version header…

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... A message of great import.
... """)
>>> process(mlist, msg, {})
>>> from mailman.version import VERSION
>>> msg['x-mailman-version'] == VERSION
True

…but only if one doesn’t already exist.

>>> msg = message_from_string("""\
... From: aperson@example.com
... X-Mailman-Version: 3000
...
... A message of great import.
... """)
>>> process(mlist, msg, {})
>>> print(msg['x-mailman-version'])
3000
Precedence header

Mailman will insert a Precedence header, which is a de-facto standard for telling automatic reply software (e.g. vacation(1)) not to respond to this message.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... A message of great import.
... """)
>>> process(mlist, msg, {})
>>> print(msg['precedence'])
list

But Mailman will only add that header if the original message doesn’t already have one of them.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Precedence: junk
...
... A message of great import.
... """)
>>> process(mlist, msg, {})
>>> print(msg['precedence'])
junk
Personalization

The To field normally contains the list posting address. However when messages are fully personalized, that header will get overwritten with the address of the recipient. The list’s posting address will be added to one of the recipient headers so that users will be able to reply back to the list.

>>> from mailman.interfaces.mailinglist import (
...     Personalization, ReplyToMunging)
>>> mlist.personalize = Personalization.full
>>> mlist.reply_goes_to_list = ReplyToMunging.no_munging
>>> mlist.description = 'My test mailing list'
>>> msg = message_from_string("""\
... From: aperson@example.com
...
... """)
>>> process(mlist, msg, {})
>>> print(msg.as_string())
From: aperson@example.com
X-Mailman-Version: ...
Precedence: list
Cc: My test mailing list <test@example.com>
<BLANKLINE>
<BLANKLINE>
Message decoration

Message decoration is the process of adding headers and footers to the original message. A handler module takes care of this based on the settings of the mailing list and the type of message being processed.

>>> mlist = create_list('ant@example.com')
>>> msg_text = """\
... From: aperson@example.org
...
... Here is a message.
... """
>>> msg = message_from_string(msg_text)
Short circuiting

Digest messages get decorated during the digest creation phase so no extra decorations are added for digest messages.

>>> from mailman.handlers.decorate import process
>>> process(mlist, msg, dict(isdigest=True))
>>> print(msg.as_string())
From: aperson@example.org

Here is a message.

>>> process(mlist, msg, dict(nodecorate=True))
>>> print(msg.as_string())
From: aperson@example.org

Here is a message.
Simple decorations

Message decorations are specified by URI and can be specialized by the mailing list and language. Internal Mailman decorations can be referenced by using the mailman:/// URL scheme. Here we create a simple English header and footer for all mailing lists in our site.

>>> import os, tempfile
>>> template_dir = tempfile.mkdtemp()
>>> site_dir = os.path.join(template_dir, 'site', 'en')
>>> os.makedirs(site_dir)
>>> config.push('templates', """
... [paths.testing]
... template_dir: {}
... """.format(template_dir))

>>> myheader_path = os.path.join(site_dir, 'myheader.txt')
>>> with open(myheader_path, 'w') as fp:
...     print('header', file=fp)
>>> myfooter_path = os.path.join(site_dir, 'myfooter.txt')
>>> with open(myfooter_path, 'w') as fp:
...     print('footer', file=fp)

Adding these template URIs to the template manager sets the mailing list up to use these templates. Since these are site-global templates, we can use a shorter path.

>>> from mailman.interfaces.template import ITemplateManager
>>> from zope.component import getUtility
>>> manager = getUtility(ITemplateManager)
>>> manager.set('list:member:regular:header',
...             mlist.list_id, 'mailman:///myheader.txt')
>>> manager.set('list:member:regular:footer',
...             mlist.list_id, 'mailman:///myfooter.txt')

Text messages that have no declared content type are, by default encoded in ASCII. When the mailing list’s preferred language is en (i.e. English), the character set of the mailing list and of the message will match, allowing Mailman to simply prepend the header and append the footer verbatim.

>>> mlist.preferred_language = 'en'
>>> process(mlist, msg, {})
>>> print(msg.as_string())
From: aperson@example.org
...
<BLANKLINE>
header
Here is a message.
footer

Mailman supports a number of interpolation variables, placeholders in the header and footer for information to be filled in with mailing list specific data. An example of such information is the mailing list’s real name (a short descriptive name for the mailing list).

>>> with open(myheader_path, 'w') as fp:
...     print('$display_name header', file=fp)
>>> with open(myfooter_path, 'w') as fp:
...     print('$display_name footer', file=fp)

>>> msg = message_from_string(msg_text)
>>> mlist.display_name = 'Ant'
>>> process(mlist, msg, {})
>>> print(msg.as_string())
From: aperson@example.org
...
Ant header
Here is a message.
Ant footer

You can’t just pick any interpolation variable though; if you do, the variable will remain in the header or footer unchanged.

>>> with open(myheader_path, 'w') as fp:
...     print('$dummy header', file=fp)
>>> with open(myfooter_path, 'w') as fp:
...     print('$dummy footer', file=fp)

>>> msg = message_from_string(msg_text)
>>> process(mlist, msg, {})
>>> print(msg.as_string())
From: aperson@example.org
...
$dummy header
Here is a message.
$dummy footer
Handling RFC 3676 ‘format=flowed’ parameters

RFC 3676 describes a standard by which text/plain messages can marked by generating MUAs for better readability in compatible receiving MUAs. The format parameter on the text/plain Content-Type header gives hints as to how the receiving MUA may flow and delete trailing whitespace for better display in a proportional font.

When Mailman sees text/plain messages with such RFC 3676 parameters, it preserves these parameters when it concatenates headers and footers to the message payload.

>>> with open(myheader_path, 'w') as fp:
...     print('header', file=fp)
>>> with open(myfooter_path, 'w') as fp:
...     print('footer', file=fp)

>>> mlist.preferred_language = 'en'
>>> msg = message_from_string("""\
... From: aperson@example.org
... Content-Type: text/plain; format=flowed; delsp=no
...
... Here is a message\x20
... with soft line breaks.
... """)
>>> process(mlist, msg, {})
>>> # Don't use 'print' here as above because it won't be obvious from the
>>> # output that the soft-line break space at the end of the 'Here is a
>>> # message' line will be retained in the output.
>>> print(msg['content-type'])
text/plain; format="flowed"; delsp="no"; charset="us-ascii"
>>> for line in msg.get_payload().splitlines():
...     print('>{0}<'.format(line))
>header<
>Here is a message <
>with soft line breaks.<
>footer<
Decorating mixed-charset messages

When a message has no explicit character set, it is assumed to be ASCII. However, if the mailing list’s preferred language has a different character set, Mailman will still try to concatenate the header and footer, but it will convert the text to utf-8 and base-64 encode the message payload.

# 'ja' = Japanese; charset = 'euc-jp'
>>> mlist.preferred_language = 'ja'

>>> with open(myheader_path, 'w') as fp:
...     print('$description header', file=fp)
>>> with open(myfooter_path, 'w') as fp:
...     print('$description footer', file=fp)
>>> mlist.description = '\u65e5\u672c\u8a9e'

>>> from email.message import Message
>>> msg = Message()
>>> msg.set_payload('Fran\xe7aise', 'iso-8859-1')
>>> print(msg.as_string())
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<BLANKLINE>
Fran=E7aise
>>> process(mlist, msg, {})
>>> print(msg.as_string())
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
<BLANKLINE>
5pel5pys6KqeIGhlYWRlcgpGcmFuw6dhaXNlCuaXpeacrOiqniBmb290ZXIK

Sometimes the message even has an unknown character set. In this case, Mailman has no choice but to decorate the original message with MIME attachments.

>>> mlist.preferred_language = 'en'
>>> with open(myheader_path, 'w') as fp:
...     print('header', file=fp)
>>> with open(myfooter_path, 'w') as fp:
...     print('footer', file=fp)

>>> msg = message_from_string("""\
... From: aperson@example.org
... Content-Type: text/plain; charset=unknown
... Content-Transfer-Encoding: 7bit
...
... Here is a message.
... """)

>>> process(mlist, msg, {})
>>> msg.set_boundary('BOUNDARY')
>>> print(msg.as_string())
From: aperson@example.org
Content-Type: multipart/mixed; boundary="BOUNDARY"

--BOUNDARY
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

header
--BOUNDARY
Content-Type: text/plain; charset=unknown
Content-Transfer-Encoding: 7bit

Here is a message.

--BOUNDARY
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

footer
--BOUNDARY--
Decorating multipart messages

Multipart messages have to be decorated differently. The header and footer cannot be simply concatenated into the payload because that will break the MIME structure of the message. Instead, the header and footer are attached as separate MIME subparts.

When the outer part is multipart/mixed, the header and footer can have a Content-Disposition of inline so that MUAs can display these headers as if they were simply concatenated.

>>> part_1 = message_from_string("""\
... From: aperson@example.org
...
... Here is the first message.
... """)
>>> part_2 = message_from_string("""\
... From: bperson@example.com
...
... Here is the second message.
... """)
>>> from email.mime.multipart import MIMEMultipart
>>> msg = MIMEMultipart('mixed', boundary='BOUNDARY',
...                     _subparts=(part_1, part_2))
>>> process(mlist, msg, {})
>>> print(msg.as_string())
Content-Type: multipart/mixed; boundary="BOUNDARY"
MIME-Version: 1.0
<BLANKLINE>
--BOUNDARY
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
<BLANKLINE>
header
--BOUNDARY
From: aperson@example.org
<BLANKLINE>
Here is the first message.
<BLANKLINE>
--BOUNDARY
From: bperson@example.com
<BLANKLINE>
Here is the second message.
<BLANKLINE>
--BOUNDARY
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
<BLANKLINE>
footer
--BOUNDARY--
Decorating other content types

Non-multipart non-text content types will get wrapped in a multipart/mixed so that the header and footer can be added as attachments.

>>> msg = message_from_string("""\
... From: aperson@example.org
... Content-Type: image/x-beautiful
...
... IMAGEDATAIMAGEDATAIMAGEDATA
... """)
>>> process(mlist, msg, {})
>>> msg.set_boundary('BOUNDARY')
>>> print(msg.as_string())
From: aperson@example.org
...
--BOUNDARY
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
<BLANKLINE>
header
--BOUNDARY
Content-Type: image/x-beautiful
<BLANKLINE>
IMAGEDATAIMAGEDATAIMAGEDATA
<BLANKLINE>
--BOUNDARY
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
<BLANKLINE>
footer
--BOUNDARY--
Digests

Digests are a way for a user to receive list traffic in collections instead of as individual messages when immediately posted. There are several forms of digests, although only two are currently supported: MIME digests and RFC 1153 (a.k.a. plain text) digests.

>>> mlist = create_list('xtest@example.com')

This is a helper function used to iterate through all the accumulated digest messages, in the order in which they were posted. This makes it easier to update the tests when we switch to a different mailbox format.

>>> from mailman.testing.helpers import digest_mbox
>>> from itertools import count
>>> from string import Template

>>> def message_factory():
...     for i in count(1):
...         text = Template("""\
... From: aperson@example.com
... To: xtest@example.com
... Subject: Test message $i
...
... Here is message $i
... """).substitute(i=i)
...         yield message_from_string(text)
>>> message_factory = message_factory()
Short circuiting

When a message is posted to the mailing list, it is generally added to a mailbox, unless the mailing list does not allow digests.

>>> mlist.digests_enabled = False
>>> msg = next(message_factory)
>>> process = config.handlers['to-digest'].process
>>> process(mlist, msg, {})
>>> sum(1 for msg in digest_mbox(mlist))
0
>>> digest_queue = config.switchboards['digest']
>>> digest_queue.files
[]

…or they may allow digests but the message is already a digest.

>>> mlist.digests_enabled = True
>>> process(mlist, msg, dict(isdigest=True))
>>> sum(1 for msg in digest_mbox(mlist))
0
>>> digest_queue.files
[]
Sending a digest

For messages which are not digests, but which are posted to a digesting mailing list, the messages will be stored until they reach a criteria triggering the sending of the digest. If none of those criteria are met, then the message will just sit in the mailbox for a while.

>>> mlist.digest_size_threshold = 10000
>>> process(mlist, msg, {})
>>> digest_queue.files
[]
>>> digest = digest_mbox(mlist)
>>> sum(1 for msg in digest)
1
>>> import os
>>> os.remove(digest._path)

When the size of the digest mailbox reaches the maximum size threshold, a marker message is placed into the digest runner’s queue. The digest is not actually crafted by the handler.

>>> mlist.digest_size_threshold = 1
>>> mlist.volume = 2
>>> mlist.next_digest_number = 10
>>> digest_path = os.path.join(mlist.data_path, 'digest.mmdf')
>>> size = 0
>>> for msg in message_factory:
...     process(mlist, msg, {})
...     # When the digest reaches the proper size, it is renamed.  So we
...     # can break out of this list when the file disappears.
...     if not os.path.exists(digest_path):
...         break

>>> sum(1 for msg in digest_mbox(mlist))
0
>>> len(digest_queue.files)
1

The digest has been moved to a unique file.

>>> from mailman.utilities.mailbox import Mailbox
>>> from mailman.testing.helpers import get_queue_messages
>>> item = get_queue_messages('digest')[0]
>>> for msg in Mailbox(item.msgdata['digest_path']):
...     print(msg['subject'])
Test message 2
Test message 3
Test message 4
Test message 5
Test message 6
Test message 7
Test message 8
Test message 9

Digests are actually crafted and sent by a separate digest runner.

DMARC Mitigations

In order to mitigate the effects of DMARC on mailing list traffic, list administrators have the ability to apply transformations to messages delivered to list members. These transformations are applied only to individual messages sent to list members and not to messages in digests, archives, or gated via NNTP.

The messages can be transformed by either munging the From: header and putting original From: in Cc: or Reply-To:, or by wrapping the original message in an outer message From: the list.

Exactly which transformations are applied depends on a number of list settings.

The settings and their effects are:

anonymous_list
If True, no mitigations are ever applied because the message is already From: the list.
dmarc_mitigate_action
The action to apply to messages From: a domain publishing a DMARC policy of reject or quarantine, or to all messages depending on the setting of dmarc_mitigate_unconditionally.
dmarc_mitigate_unconditionally
If True, apply dmarc_mitigate_action to all messages, but only if dmarc_mitigate_action is neither reject or discard.
dmarc_moderation_notice
Text to include in any rejection notice to be sent when dmarc_policy_mitigation of reject applies. This overrides the built-in default text.
dmarc_wrapped_message_text
Text to be added as a separate text/plain MIME part preceding the original message part in the wrapped message when a wrap_message mitigation applies. If this is not provided the separate text/plain MIME part is not added.
reply_goes_to_list
If this is set to other than no-munging of Reply-To:, the original From: goes in Cc: rather than Reply-To:. This is intended to make MUA functions of reply and reply-all have the same effect with messages to which mitigations have been applied as they do with other messages.

The possible actions for dmarc_mitigate_action are:

no_mitigation
Make no transformation to the message.
munge_from
Change the From: header and put the original From: in Reply-To: or in some cases Cc:.
wrap_message
Wrap the message in an outer message with headers from the original message as in munge_from.
reject
Bounce the message back to the sender with a default reason or one supplied in dmarc_moderation_notice.
discard
Silently discard the message.

Here’s what happens when we munge the From:.

>>> from mailman.interfaces.mailinglist import (
...     DMARCMitigateAction, ReplyToMunging)

>>> mlist = create_list('ant@example.com')
>>> mlist.dmarc_mitigate_action = DMARCMitigateAction.munge_from
>>> mlist.reply_goes_to_list = ReplyToMunging.no_munging

>>> msg = message_from_string("""\
... From: Anne Person <aperson@example.com>
... To: ant@example.com
...
... A message of great import.
... """)
>>> msgdata = dict(dmarc=True, original_sender='aperson@example.com')

>>> from mailman.handlers.dmarc import process
>>> process(mlist, msg, msgdata)
>>> print(msg.as_string())
To: ant@example.com
From: Anne Person via Ant <ant@example.com>
Reply-To: Anne Person <aperson@example.com>

A message of great import.

Here we wrap the message without adding a text part.

>>> mlist.dmarc_mitigate_action = DMARCMitigateAction.wrap_message
>>> mlist.dmarc_wrapped_message_text = ''

>>> msg = message_from_string("""\
... From: Anne Person <aperson@example.com>
... To: ant@example.com
...
... A message of great import.
... """)
>>> msgdata = dict(dmarc=True, original_sender='aperson@example.com')

>>> process(mlist, msg, msgdata)
>>> print(msg.as_string())
To: ant@example.com
MIME-Version: 1.0
Message-ID: <...>
From: Anne Person via Ant <ant@example.com>
Reply-To: Anne Person <aperson@example.com>
Content-Type: message/rfc822
Content-Disposition: inline

From: Anne Person <aperson@example.com>
To: ant@example.com

A message of great import.

And here’s a wrapped message with an added text part.

>>> mlist.dmarc_wrapped_message_text = 'The original message is attached.'

>>> msg = message_from_string("""\
... From: Anne Person <aperson@example.com>
... To: ant@example.com
...
... A message of great import.
... """)
>>> msgdata = dict(dmarc=True, original_sender='aperson@example.com')

>>> process(mlist, msg, msgdata)
>>> print(msg.as_string())
To: ant@example.com
MIME-Version: 1.0
Message-ID: <...>
From: Anne Person via Ant <ant@example.com>
Reply-To: Anne Person <aperson@example.com>
Content-Type: multipart/mixed; boundary="..."

--...
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

The original message is attached.
--...
Content-Type: message/rfc822
MIME-Version: 1.0
Content-Disposition: inline

From: Anne Person <aperson@example.com>
To: ant@example.com

A message of great import.

--...--
File recipients

Mailman can calculate the recipients for a message from a Sendmail-style include file. This file must be called members.txt and it must live in the list’s data directory.

>>> mlist = create_list('_xtest@example.com')
Short circuiting

If the message’s metadata already has recipients, this handler immediately returns.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... A message.
... """)
>>> msgdata = {'recipients': 7}

>>> handler = config.handlers['file-recipients']
>>> handler.process(mlist, msg, msgdata)
>>> print(msg.as_string())
From: aperson@example.com

A message.

>>> dump_msgdata(msgdata)
recipients: 7
Existing file

If the file exists, it contains a list of addresses, one per line. These addresses are returned as the set of recipients.

>>> import os
>>> file_path = os.path.join(mlist.data_path, 'members.txt')
>>> with open(file_path, 'w', encoding='utf-8') as fp:
...     print('bperson@example.com', file=fp)
...     print('cperson@example.com', file=fp)
...     print('dperson@example.com', file=fp)
...     print('eperson@example.com', file=fp)
...     print('fperson@example.com', file=fp)
...     print('gperson@example.com', file=fp)

>>> msgdata = {}
>>> handler.process(mlist, msg, msgdata)
>>> dump_list(msgdata['recipients'])
bperson@example.com
cperson@example.com
dperson@example.com
eperson@example.com
fperson@example.com
gperson@example.com

However, if the sender of the original message is a member of the list and their address is in the include file, the sender’s address is not included in the recipients list.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> address_1 = getUtility(IUserManager).create_address(
...     'cperson@example.com')

>>> from mailman.interfaces.member import MemberRole
>>> mlist.subscribe(address_1, MemberRole.member)
<Member: cperson@example.com on _xtest@example.com as MemberRole.member>

>>> msg = message_from_string("""\
... From: cperson@example.com
...
... A message.
... """)
>>> msgdata = {}
>>> handler.process(mlist, msg, msgdata)
>>> dump_list(msgdata['recipients'])
bperson@example.com
dperson@example.com
eperson@example.com
fperson@example.com
gperson@example.com
Content filtering

Mailman can filter the content of messages posted to a mailing list by stripping MIME subparts, and possibly reorganizing the MIME structure of a message.

>>> mlist = create_list('test@example.com')

Several mailing list options control content filtering. First, the feature must be enabled, then there are two options that control which MIME types get filtered and which get passed. Finally, there is an option to control whether text/html parts will get converted to plain text. Let’s set up some defaults for these variables, then we’ll explain them in more detail below.

>>> mlist.filter_content = True
>>> mlist.filter_types = []
>>> mlist.pass_types = []
>>> mlist.convert_html_to_plaintext = False
Filtering the outer content type

A simple filtering setting will just search the content types of the messages parts, discarding all parts with a matching MIME type. If the message’s outer content type matches the filter, the entire message will be discarded. However, if we turn off content filtering altogether, then the handler short-circuits.

>>> from mailman.interfaces.mime import FilterAction

>>> mlist.filter_types = ['image/jpeg']
>>> mlist.filter_action = FilterAction.discard

>>> msg = message_from_string("""\
... From: aperson@example.com
... Content-Type: image/jpeg
... MIME-Version: 1.0
...
... xxxxx
... """)

>>> process = config.handlers['mime-delete'].process
>>> mlist.filter_content = False
>>> msgdata = {}
>>> process(mlist, msg, msgdata)
>>> print(msg.as_string())
From: aperson@example.com
Content-Type: image/jpeg
MIME-Version: 1.0

xxxxx
>>> msgdata
{}

Similarly, no content filtering is performed on digest messages, which are crafted internally by Mailman.

>>> mlist.filter_content = True
>>> msgdata = {'isdigest': True}
>>> process(mlist, msg, msgdata)
>>> print(msg.as_string())
From: aperson@example.com
Content-Type: image/jpeg
MIME-Version: 1.0
<BLANKLINE>
xxxxx
>>> dump_msgdata(msgdata)
isdigest: True
Simple multipart filtering

If one of the subparts in a multipart message matches the filter type, then just that subpart will be stripped. If that leaves just a single subpart, the multipart will be replaced by the subpart.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Content-Type: multipart/mixed; boundary=BOUNDARY
... MIME-Version: 1.0
...
... --BOUNDARY
... Content-Type: image/jpeg
... MIME-Version: 1.0
...
... xxx
...
... --BOUNDARY
... Content-Type: image/gif
... MIME-Version: 1.0
...
... yyy
... --BOUNDARY--
... """)

>>> process(mlist, msg, {})
>>> print(msg.as_string())
From: aperson@example.com
MIME-Version: 1.0
Content-Type: image/gif
X-Content-Filtered-By: Mailman/MimeDel ...

yyy
Collapsing multipart/alternative messages

When content filtering encounters a multipart/alternative part, and the results of filtering leave only one of the subparts, then the multipart/alternative may be collapsed. For example, in the following message, the outer content type is a multipart/mixed. Inside this part is just a single subpart that has a content type of multipart/alternative. This inner multipart has two subparts, a jpeg and a gif.

Content filtering will remove the jpeg part, leaving the multipart/alternative with only a single gif subpart. Because there’s only one subpart left, the MIME structure of the message will be reorganized, removing the inner multipart/alternative so that the outer multipart/mixed has just a single gif subpart, and then the multipart is recast as just the subpart.

>>> mlist.collapse_alternatives = True
>>> msg = message_from_string("""\
... From: aperson@example.com
... Content-Type: multipart/mixed; boundary=BOUNDARY
... MIME-Version: 1.0
...
... --BOUNDARY
... Content-Type: multipart/alternative; boundary=BOUND2
... MIME-Version: 1.0
...
... --BOUND2
... Content-Type: image/jpeg
... MIME-Version: 1.0
...
... xxx
...
... --BOUND2
... Content-Type: image/gif
... MIME-Version: 1.0
...
... yyy
... --BOUND2--
...
... --BOUNDARY--
... """)
>>> process(mlist, msg, {})
>>> print(msg.as_string())
From: aperson@example.com
MIME-Version: 1.0
Content-Type: image/gif
X-Content-Filtered-By: Mailman/MimeDel ...
<BLANKLINE>
yyy

When the outer part is a multipart/alternative and filtering leaves this outer part with just one subpart, the entire message is converted to the left over part’s content type. In other words, the left over inner part is promoted to being the outer part.

>>> mlist.filter_types = ['image/jpeg', 'text/html']
>>> msg = message_from_string("""\
... From: aperson@example.com
... Content-Type: multipart/alternative; boundary=AAA
...
... --AAA
... Content-Type: text/html
...
... <b>This is some html</b>
... --AAA
... Content-Type: text/plain
...
... This is plain text
... --AAA--
... """)

>>> process(mlist, msg, {})
>>> print(msg.as_string())
From: aperson@example.com
Content-Type: text/plain
X-Content-Filtered-By: Mailman/MimeDel ...

This is plain text

Clean up.

>>> mlist.filter_types = ['image/jpeg']
Conversion to plain text

Some mailing lists prohibit HTML email, and in fact, such email can be a phishing or spam vector. However, many mail readers will send HTML email by default because users think it looks pretty. One approach to handling this would be to filter out text/html parts and rely on multipart/alternative collapsing to leave just a plain text part. This works because many mail readers that send HTML email actually send a plain text part in the second subpart of such multipart/alternatives.

While this is a good suggestion for plain text-only mailing lists, often a mail reader will send only a text/html part with no plain text alternative. in this case, the site administer can enable text/html to text/plain conversion by defining a conversion command. A list administrator still needs to enable such conversion for their list though.

>>> mlist.convert_html_to_plaintext = True

By default, Mailman sends the message through lynx, but since this program is not guaranteed to exist, we’ll craft a simple, but stupid script to simulate the conversion process. The script expects a single argument, which is the name of the file containing the message payload to filter.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Content-Type: text/html
... MIME-Version: 1.0
...
... <html><head></head>
... <body></body></html>
... """)

>>> from mailman.handlers.tests.test_mimedel import dummy_script
>>> with dummy_script():
...     process(mlist, msg, {})
>>> print(msg.as_string())
From: aperson@example.com
MIME-Version: 1.0
Content-Type: text/plain
X-Content-Filtered-By: Mailman/MimeDel ...

Converted text/html to text/plain
Filename: ...
Discarding empty parts

Similarly, if after filtering a multipart section ends up empty, then the entire multipart is discarded. For example, here’s a message where an inner multipart/mixed contains two jpeg subparts. Both jpegs are filtered out, so the entire inner multipart/mixed is discarded.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Content-Type: multipart/mixed; boundary=AAA
...
... --AAA
... Content-Type: multipart/mixed; boundary=BBB
...
... --BBB
... Content-Type: image/jpeg
...
... xxx
... --BBB
... Content-Type: image/jpeg
...
... yyy
... --BBB---
... --AAA
... Content-Type: multipart/alternative; boundary=CCC
...
... --CCC
... Content-Type: text/html
...
... <h2>This is a header</h2>
...
... --CCC
... Content-Type: text/plain
...
... A different message
... --CCC--
... --AAA
... Content-Type: image/gif
...
... zzz
... --AAA
... Content-Type: image/gif
...
... aaa
... --AAA--
... """)

>>> with dummy_script():
...     process(mlist, msg, {})

>>> print(msg.as_string())
From: aperson@example.com
Content-Type: multipart/mixed; boundary=AAA
X-Content-Filtered-By: Mailman/MimeDel ...

--AAA
MIME-Version: 1.0
Content-Type: text/plain

Converted text/html to text/plain
Filename: ...

--AAA
Content-Type: image/gif

zzz
--AAA
Content-Type: image/gif

aaa
--AAA--
Passing MIME types

XXX Describe the pass_mime_types setting and how it interacts with filter_mime_types.

Calculating recipients

Every message that makes it through to the list membership gets sent to a set of recipient addresses. These addresses are calculated by one of the handler modules and depends on a host of factors.

>>> mlist = create_list('test@example.com')

Recipients are calculate from the list membership, so first some people subscribe to the mailing list…

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)

>>> address_a = user_manager.create_address('aperson@example.com')
>>> address_b = user_manager.create_address('bperson@example.com')
>>> address_c = user_manager.create_address('cperson@example.com')
>>> address_d = user_manager.create_address('dperson@example.com')
>>> address_e = user_manager.create_address('eperson@example.com')
>>> address_f = user_manager.create_address('fperson@example.com')

…then subscribe these addresses to the mailing list as members…

>>> from mailman.interfaces.member import MemberRole
>>> member_a = mlist.subscribe(address_a, MemberRole.member)
>>> member_b = mlist.subscribe(address_b, MemberRole.member)
>>> member_c = mlist.subscribe(address_c, MemberRole.member)
>>> member_d = mlist.subscribe(address_d, MemberRole.member)
>>> member_e = mlist.subscribe(address_e, MemberRole.member)
>>> member_f = mlist.subscribe(address_f, MemberRole.member)

…then make some of the members digest members.

>>> from mailman.interfaces.member import DeliveryMode
>>> member_d.preferences.delivery_mode = DeliveryMode.plaintext_digests
>>> member_e.preferences.delivery_mode = DeliveryMode.mime_digests
>>> member_f.preferences.delivery_mode = DeliveryMode.summary_digests
Regular delivery recipients

Regular delivery recipients are those people who get messages from the list as soon as they are posted. In other words, these folks are not digest members.

>>> msg = message_from_string("""\
... From: Xavier Person <xperson@example.com>
...
... Something of great import.
... """)
>>> msgdata = {}
>>> handler = config.handlers['member-recipients']
>>> handler.process(mlist, msg, msgdata)
>>> dump_list(msgdata['recipients'])
aperson@example.com
bperson@example.com
cperson@example.com

Members can elect not to receive a list copy of their own postings.

>>> member_c.preferences.receive_own_postings = False
>>> msg = message_from_string("""\
... From: Claire Person <cperson@example.com>
...
... Something of great import.
... """)
>>> msgdata = {}
>>> handler.process(mlist, msg, msgdata)
>>> dump_list(msgdata['recipients'])
aperson@example.com
bperson@example.com

Members can also elect not to receive a list copy of any message on which they are explicitly named as a recipient. However, see the avoid duplicates handler for details.

Digest recipients

XXX Test various digest deliveries.

Urgent messages
XXX Test various urgent deliveries:
  • test_urgent_moderator()
  • test_urgent_admin()
  • test_urgent_reject()
NNTP Gateway

Mailman has an NNTP gateway, whereby messages posted to the mailing list can be forwarded onto an NNTP newsgroup.

>>> mlist = create_list('test@example.com')

Gatewaying from the mailing list to the newsgroup happens through a separate nntp queue and happen immediately when the message is posted through to the list. Note that gatewaying from the newsgroup to the list happens via a separate process.

There are several situations which prevent a message from being gatewayed to the newsgroup. The feature could be disabled, as is the default.

>>> mlist.gateway_to_news = False
>>> msg = message_from_string("""\
... Subject: An important message
...
... Something of great import.
... """)

>>> handler = config.handlers['to-usenet']
>>> handler.process(mlist, msg, {})
>>> from mailman.testing.helpers import get_queue_messages
>>> get_queue_messages('nntp')
[]

Even if enabled, messages that came from the newsgroup are never gated back to the newsgroup.

>>> mlist.gateway_to_news = True
>>> handler.process(mlist, msg, dict(fromusenet=True))
>>> get_queue_messages('nntp')
[]

Neither are digests ever gated to the newsgroup.

>>> handler.process(mlist, msg, dict(isdigest=True))
>>> get_queue_messages('nntp')
[]

However, other posted messages get gated to the newsgroup via the nntp queue. The list owner can set the linked newsgroup and the nntp host that its messages are gated to.

>>> mlist.linked_newsgroup = 'comp.lang.thing'
>>> mlist.nntp_host = 'news.example.com'
>>> handler.process(mlist, msg, {})
>>> messages = get_queue_messages('nntp')
>>> len(messages)
1

>>> print(messages[0].msg.as_string())
Subject: An important message

Something of great import.


>>> dump_msgdata(messages[0].msgdata)
_parsemsg: False
listid   : test.example.com
version  : 3
List owner recipients

When a message is posted to a mailing list’s -owners address, all of the list’s administrators will receive a copy. The administrators are defined as the set of owners and moderators.

>>> mlist_1 = create_list('alpha@example.com')

Anne is the owner of the list and Bart is a moderator of the list.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)
>>> anne_addr = user_manager.create_address('anne@example.com')
>>> bart_addr = user_manager.create_address('bart@example.com')
>>> from mailman.interfaces.member import MemberRole
>>> anne = mlist_1.subscribe(anne_addr, MemberRole.owner)
>>> bart = mlist_1.subscribe(bart_addr, MemberRole.moderator)

The recipients list for the -owners address includes both Anne and Bart.

>>> msg = message_from_string("""\
... From: Xavier Person <xperson@example.com>
... To: alpha@example.com
...
... """)
>>> msgdata = {}
>>> handler = config.handlers['owner-recipients']
>>> handler.process(mlist_1, msg, msgdata)
>>> dump_list(msgdata['recipients'])
anne@example.com
bart@example.com

Anne disables her owner delivery, so she will not receive -owner emails.

>>> from mailman.interfaces.member import DeliveryStatus
>>> anne.preferences.delivery_status = DeliveryStatus.by_user
>>> msgdata = {}
>>> handler.process(mlist_1, msg, msgdata)
>>> dump_list(msgdata['recipients'])
bart@example.com

If Bart also disables his owner delivery, then no one could contact the list’s owners. Since this is unacceptable, the site owner is used as a fallback.

>>> bart.preferences.delivery_status = DeliveryStatus.by_user
>>> msgdata = {}
>>> handler.process(mlist_1, msg, msgdata)
>>> dump_list(msgdata['recipients'])
noreply@example.com

For mailing lists which have no owners at all, the site owner is also used as a fallback.

>>> mlist_2 = create_list('beta@example.com')
>>> print(mlist_2.administrators.member_count)
0
>>> msgdata = {}
>>> handler.process(mlist_2, msg, msgdata)
>>> dump_list(msgdata['recipients'])
noreply@example.com
Reply-to munging

Messages that flow through the global pipeline get their headers cooked, which basically means that their headers go through several mostly unrelated transformations. Some headers get added, others get changed. Some of these changes depend on mailing list settings and others depend on how the message is getting sent through the system. We’ll take things one-by-one.

>>> mlist = create_list('_xtest@example.com')

Reply-to munging refers to the behavior where a mailing list can be configured to change or augment an existing Reply-To header in a message posted to the list. Reply-to munging is fairly controversial, with arguments made either for or against munging.

The Mailman developers, and I believe the majority consensus is to do no reply-to munging, under several principles. Primarily, most reply-to munging is requested by people who do not have both a Reply and Reply All button on their mail reader. If you do not munge Reply-To, then these buttons will work properly, but if you munge the header, it is impossible for these buttons to work right, because both will reply to the list. This leads to unfortunate accidents where a private message is accidentally posted to the entire list.

However, Mailman gives list owners the option to do reply-To munging anyway, mostly as a way to shut up the really vocal minority who seem to insist on this mis-feature.

Reply to list

A list can be configured to add a Reply-To header pointing back to the mailing list’s posting address. If there’s no Reply-To header in the original message, the list’s posting address simply gets inserted.

>>> from mailman.interfaces.mailinglist import ReplyToMunging
>>> mlist.reply_goes_to_list = ReplyToMunging.point_to_list
>>> mlist.preferred_language = 'en'
>>> mlist.description = ''
>>> msg = message_from_string("""\
... From: aperson@example.com
...
... """)

>>> from mailman.handlers.cook_headers import process
>>> process(mlist, msg, {})
>>> len(msg.get_all('reply-to'))
1
>>> print(msg['reply-to'])
_xtest@example.com

It’s also possible to strip any existing Reply-To header first, before adding the list’s posting address.

>>> mlist.first_strip_reply_to = True
>>> msg = message_from_string("""\
... From: aperson@example.com
... Reply-To: bperson@example.com
...
... """)
>>> process(mlist, msg, {})
>>> len(msg.get_all('reply-to'))
1
>>> print(msg['reply-to'])
_xtest@example.com

If you don’t first strip the header, then the list’s posting address will just get appended to whatever the original version was.

>>> mlist.first_strip_reply_to = False
>>> msg = message_from_string("""\
... From: aperson@example.com
... Reply-To: bperson@example.com
...
... """)
>>> process(mlist, msg, {})
>>> len(msg.get_all('reply-to'))
1
>>> print(msg['reply-to'])
bperson@example.com, _xtest@example.com
Explicit Reply-To

The list can also be configured to have an explicit Reply-To header.

>>> mlist.reply_goes_to_list = ReplyToMunging.explicit_header
>>> mlist.reply_to_address = 'my-list@example.com'
>>> msg = message_from_string("""\
... From: aperson@example.com
...
... """)
>>> process(mlist, msg, {})
>>> len(msg.get_all('reply-to'))
1
>>> print(msg['reply-to'])
my-list@example.com

And as before, it’s possible to either strip any existing Reply-To header…

>>> mlist.first_strip_reply_to = True
>>> msg = message_from_string("""\
... From: aperson@example.com
... Reply-To: bperson@example.com
...
... """)
>>> process(mlist, msg, {})
>>> len(msg.get_all('reply-to'))
1
>>> print(msg['reply-to'])
my-list@example.com

…or not.

>>> mlist.first_strip_reply_to = False
>>> msg = message_from_string("""\
... From: aperson@example.com
... Reply-To: bperson@example.com
...
... """)
>>> process(mlist, msg, {})
>>> len(msg.get_all('reply-to'))
1
>>> print(msg['reply-to'])
my-list@example.com, bperson@example.com
Automatic response handler

Mailman has a autoreply handler that sends automatic responses to messages it receives on its posting address, owner address, or robot address. Automatic responses are subject to various conditions, such as headers in the original message or the amount of time since the last auto-response.

>>> mlist = create_list('_xtest@example.com')
>>> mlist.display_name = 'XTest'
Basic automatic responding

Basic automatic responding occurs when the list is set up to respond to either its -owner address, its -request address, or to the posting address, and a message is sent to one of these addresses. A mailing list also has an automatic response grace period which specifies how much time must pass before a second response will be sent, with 0 meaning “there is no grace period”.

>>> from datetime import timedelta
>>> from mailman.interfaces.autorespond import ResponseAction

>>> mlist.autorespond_owner = ResponseAction.respond_and_continue
>>> mlist.autoresponse_grace_period = timedelta()
>>> mlist.autoresponse_owner_text = 'owner autoresponse text'

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: _xtest-owner@example.com
...
... help
... """)

The preceding message to the mailing list’s owner will trigger an automatic response.

>>> from mailman.testing.helpers import get_queue_messages

>>> handler = config.handlers['replybot']
>>> handler.process(mlist, msg, dict(to_owner=True))
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1

>>> dump_msgdata(messages[0].msgdata)
_parsemsg           : False
listid              : _xtest.example.com
nodecorate          : True
recipients          : {'aperson@example.com'}
reduced_list_headers: True
version             : 3

>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Auto-response for your message to the "XTest" mailing list
From: _xtest-bounces@example.com
To: aperson@example.com
X-Mailer: The Mailman Replybot
X-Ack: No
Message-ID: <...>
Date: ...
Precedence: bulk

owner autoresponse text
Short circuiting

Several headers in the original message determine whether an automatic response should even be sent. For example, if the message has an X-Ack: No header, no auto-response is sent.

>>> msg = message_from_string("""\
... From: aperson@example.com
... X-Ack: No
...
... help me
... """)

>>> handler.process(mlist, msg, dict(to_owner=True))
>>> get_queue_messages('virgin')
[]

Mailman itself can suppress automatic responses for certain types of internally crafted messages, by setting the noack metadata key.

>>> msg = message_from_string("""\
... From: mailman@example.com
...
... help for you
... """)

>>> handler.process(mlist, msg, dict(noack=True, to_owner=True))
>>> get_queue_messages('virgin')
[]

If there is a Precedence: header with any of the values bulk, junk, or list, then the automatic response is also suppressed.

>>> msg = message_from_string("""\
... From: asystem@example.com
... Precedence: bulk
...
... hey!
... """)

>>> handler.process(mlist, msg, dict(to_owner=True))
>>> get_queue_messages('virgin')
[]

>>> msg.replace_header('precedence', 'junk')
>>> handler.process(mlist, msg, dict(to_owner=True))
>>> get_queue_messages('virgin')
[]

>>> msg.replace_header('precedence', 'list')
>>> handler.process(mlist, msg, dict(to_owner=True))
>>> get_queue_messages('virgin')
[]

Unless the X-Ack: header has a value of yes, in which case, the Precedence header is ignored.

>>> msg['X-Ack'] = 'yes'
>>> handler.process(mlist, msg, dict(to_owner=True))
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1

>>> dump_msgdata(messages[0].msgdata)
_parsemsg           : False
listid              : _xtest.example.com
nodecorate          : True
recipients          : {'asystem@example.com'}
reduced_list_headers: True
version             : 3

>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Auto-response for your message to the "XTest" mailing list
From: _xtest-bounces@example.com
To: asystem@example.com
X-Mailer: The Mailman Replybot
X-Ack: No
Message-ID: <...>
Date: ...
Precedence: bulk

owner autoresponse text
Available auto-responses

As shown above, a message sent to the -owner address will get an auto-response with the text set for owner responses. Two other types of email will get auto-responses: those sent to the -request address…

>>> mlist.autorespond_requests = ResponseAction.respond_and_continue
>>> mlist.autoresponse_request_text = 'robot autoresponse text'

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: _xtest-request@example.com
...
... help me
... """)

>>> handler.process(mlist, msg, dict(to_request=True))
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1

>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Auto-response for your message to the "XTest" mailing list
From: _xtest-bounces@example.com
To: aperson@example.com
X-Mailer: The Mailman Replybot
X-Ack: No
Message-ID: <...>
Date: ...
Precedence: bulk

robot autoresponse text

…and those sent to the posting address.

>>> mlist.autorespond_postings = ResponseAction.respond_and_continue
>>> mlist.autoresponse_postings_text = 'postings autoresponse text'

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: _xtest@example.com
...
... help me
... """)

>>> handler.process(mlist, msg, dict(to_list=True))
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1

>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Auto-response for your message to the "XTest" mailing list
From: _xtest-bounces@example.com
To: aperson@example.com
X-Mailer: The Mailman Replybot
X-Ack: No
Message-ID: <...>
Date: ...
Precedence: bulk

postings autoresponse text
Grace periods

Automatic responses have a grace period, during which no additional responses will be sent. This is so as not to bombard the sender with responses. The grace period is measured in days.

>>> mlist.autoresponse_grace_period = timedelta(days=10)

When a response is sent to a person via any of the owner, request, or postings addresses, the response date is recorded. The grace period is usually measured in days.

>>> msg = message_from_string("""\
... From: bperson@example.com
... To: _xtest-owner@example.com
...
... help
... """)

This is the first response to bperson, so it gets sent.

>>> handler.process(mlist, msg, dict(to_owner=True))
>>> len(get_queue_messages('virgin'))
1

But with a grace period greater than zero, no subsequent response will be sent right now.

>>> handler.process(mlist, msg, dict(to_owner=True))
>>> len(get_queue_messages('virgin'))
0

Fast forward 9 days and you still don’t get a response.

>>> from mailman.utilities.datetime import factory
>>> factory.fast_forward(days=9)

>>> handler.process(mlist, msg, dict(to_owner=True))
>>> len(get_queue_messages('virgin'))
0

But tomorrow, the sender will get a new auto-response.

>>> factory.fast_forward()
>>> handler.process(mlist, msg, dict(to_owner=True))
>>> len(get_queue_messages('virgin'))
1

Of course, everything works the same way for messages to the request address, even if the sender is the same person…

>>> msg = message_from_string("""\
... From: bperson@example.com
... To: _xtest-request@example.com
...
... help
... """)

>>> handler.process(mlist, msg, dict(to_request=True))
>>> len(get_queue_messages('virgin'))
1

>>> handler.process(mlist, msg, dict(to_request=True))
>>> len(get_queue_messages('virgin'))
0

>>> factory.fast_forward(days=9)
>>> handler.process(mlist, msg, dict(to_request=True))
>>> len(get_queue_messages('virgin'))
0

>>> factory.fast_forward()
>>> handler.process(mlist, msg, dict(to_request=True))
>>> len(get_queue_messages('virgin'))
1

…and for messages to the posting address.

>>> msg = message_from_string("""\
... From: bperson@example.com
... To: _xtest@example.com
...
... help
... """)

>>> handler.process(mlist, msg, dict(to_list=True))
>>> len(get_queue_messages('virgin'))
1

>>> handler.process(mlist, msg, dict(to_list=True))
>>> len(get_queue_messages('virgin'))
0

>>> factory.fast_forward(days=9)
>>> handler.process(mlist, msg, dict(to_list=True))
>>> len(get_queue_messages('virgin'))
0

>>> factory.fast_forward()
>>> handler.process(mlist, msg, dict(to_list=True))
>>> len(get_queue_messages('virgin'))
1
RFC 2919 and 2369 headers

RFC 2919 and RFC 2369 define headers for mailing list actions. These headers generally start with the List- prefix.

>>> mlist = create_list('test@example.com')
>>> mlist.preferred_language = 'en'
>>> from mailman.interfaces.archiver import ArchivePolicy
>>> mlist.archive_policy = ArchivePolicy.never

The rfc-2369 handler adds the List- headers. List-Id is always added.

>>> from mailman.handlers.rfc_2369 import process
>>> msg = message_from_string("""\
... From: aperson@example.com
...
... """)
>>> process(mlist, msg, {})
>>> list_headers(msg, 'list-id')
---start---
list-id: <test.example.com>
---end---
Fewer headers

Some people don’t like these headers because their mail readers aren’t good about hiding them. A list owner can turn these headers off.

>>> mlist.include_rfc2369_headers = False
>>> msg = message_from_string("""\
... From: aperson@example.com
...
... """)
>>> process(mlist, msg, {})
>>> list_headers(msg)
---start---
---end---

Messages which Mailman generates itself, such as user or owner notifications, have a reduced set of List- headers. Specifically, there is no List-Post, List-Archive or Archived-At header. ..

>>> mlist.include_rfc2369_headers = True
>>> msg = message_from_string("""\
... From: aperson@example.com
...
... """)

>>> process(mlist, msg, dict(reduced_list_headers=True))
>>> list_headers(msg)
---start---
list-help: <mailto:test-request@example.com?subject=help>
list-id: <test.example.com>
list-subscribe: <mailto:test-join@example.com>
list-unsubscribe: <mailto:test-leave@example.com>
---end---
List-Post header

Discussion lists, to which any subscriber can post, also have a List-Post header which contains the mailto: URL used to send messages to the list.

>>> mlist.include_rfc2369_headers = True
>>> mlist.allow_list_posts = True
>>> msg = message_from_string("""\
... From: aperson@example.com
...
... """)
>>> process(mlist, msg, {})
>>> list_headers(msg)
---start---
list-help: <mailto:test-request@example.com?subject=help>
list-id: <test.example.com>
list-post: <mailto:test@example.com>
list-subscribe: <mailto:test-join@example.com>
list-unsubscribe: <mailto:test-leave@example.com>
---end---

Some mailing lists are announce, or one-way lists, not discussion lists. Because the general membership cannot post to these mailing lists, the list owner can set a flag which adds a special List-Post header value, according to RFC 2369.

>>> mlist.allow_list_posts = False
>>> msg = message_from_string("""\
... From: aperson@example.com
...
... """)
>>> process(mlist, msg, {})
>>> list_headers(msg)
---start---
list-help: <mailto:test-request@example.com?subject=help>
list-id: <test.example.com>
list-post: NO
list-subscribe: <mailto:test-join@example.com>
list-unsubscribe: <mailto:test-leave@example.com>
---end---
List-Id header

If the mailing list has a description, then it is included in the List-Id header.

>>> mlist.allow_list_posts = True
>>> mlist.description = 'My test mailing list'
>>> msg = message_from_string("""\
... From: aperson@example.com
...
... """)
>>> process(mlist, msg, {})
>>> list_headers(msg)
---start---
list-help: <mailto:test-request@example.com?subject=help>
list-id: My test mailing list <test.example.com>
list-post: <mailto:test@example.com>
list-subscribe: <mailto:test-join@example.com>
list-unsubscribe: <mailto:test-leave@example.com>
---end---

Any existing List-Id headers are removed from the original message.

>>> msg = message_from_string("""\
... From: aperson@example.com
... List-ID: <123.456.789>
...
... """)

>>> process(mlist, msg, {})
>>> list_headers(msg, only='list-id')
---start---
list-id: My test mailing list <test.example.com>
---end---
Archive headers

When the mailing list is configured to enable archiving, List-Archive headers will be added for each web accessible archiver that is enabled.

RFC 5064 defines the Archived-At header which contains the url to the individual message in the archives. Archivers which don’t support pre-calculation of the archive url cannot add the Archived-At header. However, other archivers can calculate the url, and do add this header.

If the mailing list isn’t being archived, neither the List-Archive nor Archived-At headers will be added.

Subject prefixes

Mailing lists can define a subject prefix which gets added to the front of any Subject text. This can be used to quickly identify which mailing list the message was posted to.

>>> mlist = create_list('test@example.com')

The default list style gives the mailing list a default prefix.

>>> print(mlist.subject_prefix)
[Test]

This can be changed to anything, but typically ends with a trailing space.

>>> mlist.subject_prefix = '[XTest] '
>>> process = config.handlers['subject-prefix'].process
No Subject

If the original message has no Subject, then a canned one is used.

>>> msg = message_from_string("""\
... From: aperson@example.com
...
... A message of great import.
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'])
[XTest] (no subject)
Inserting a prefix

If the original message had a Subject header, then the prefix is inserted at the beginning of the header’s value.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: Something important
...
... A message of great import.
... """)
>>> msgdata = {}
>>> process(mlist, msg, msgdata)
>>> print(msg['subject'])
[XTest] Something important

The original Subject is available in the metadata.

>>> print(msgdata['original_subject'])
Something important

If a Subject header already has a prefix, usually following a Re: marker, another one will not be added but the prefix will be moved to the front of the header text.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: Re: [XTest] Something important
...
... A message of great import.
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'])
[XTest] Re: Something important

If the Subject header has a prefix at the front of the header text, that’s where it will stay.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: [XTest] Re: Something important
...
... A message of great import.
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'])
[XTest] Re: Something important

Sometimes the incoming Subject header has a pathological sequence of Re: like markers. These should all be collapsed up to the first non-Re: marker.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: [XTest] Re: RE : Re: Re: Re: Re: Re: Something important
...
... A message of great import.
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'])
[XTest] Re: Something important
Internationalized headers

Internationalization adds some interesting twists to the handling of subject prefixes. Part of what makes this interesting is the encoding of i18n headers using RFC 2047, and lists whose preferred language is in a different character set than the encoded header.

>>> msg = message_from_string("""\
... Subject: =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
...
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'].encode())
[XTest] =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
>>> print(str(msg['subject']))
[XTest] メールマン
Prefix numbers

Subject prefixes support a placeholder for the numeric post id. Every time a message is posted to the mailing list, a post id gets incremented. This is a purely sequential integer that increases monotonically. By added a %d placeholder to the subject prefix, this post id can be included in the prefix.

>>> mlist.subject_prefix = '[XTest %d] '
>>> mlist.post_id = 456
>>> msg = message_from_string("""\
... Subject: Something important
...
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'])
[XTest 456] Something important

This works even when the message is a reply, except that in this case, the numeric post id in the generated subject prefix is updated with the new post id.

>>> msg = message_from_string("""\
... Subject: [XTest 123] Re: Something important
...
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'])
[XTest 456] Re: Something important

If the Subject header had old style prefixing, the prefix is moved to the front of the header text.

>>> msg = message_from_string("""\
... Subject: Re: [XTest 123] Something important
...
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'])
[XTest 456] Re: Something important

And of course, the proper thing is done when posting id numbers are included in the subject prefix, and the subject is encoded non-ASCII.

>>> msg = message_from_string("""\
... Subject: =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
...
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'].encode())
[XTest 456] =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
>>> print(msg['subject'])
[XTest 456] メールマン

Even more fun is when the internationalized Subject header already has a prefix, possibly with a different posting number.

>>> msg = message_from_string("""\
... Subject: [XTest 123] Re: =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
...
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'].encode())
[XTest 456] Re: =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
>>> print(msg['subject'])
[XTest 456] Re: メールマン

As before, old style subject prefixes are re-ordered.

>>> msg = message_from_string("""\
... Subject: Re: [XTest 123] =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
...
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'].encode())
[XTest 456] Re:
  =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
>>> print(msg['subject'])
[XTest 456]  Re: メールマン

In this test case, we get an extra space between the prefix and the original subject. It’s because the original is crooked. Note that a Subject starting with ‘n ‘ is generated by some version of Eudora Japanese edition.

>>> mlist.subject_prefix = '[XTest] '
>>> msg = message_from_string("""\
... Subject:
...  Important message
...
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'])
[XTest]  Important message

And again, with an RFC 2047 encoded header.

>>> msg = message_from_string("""\
... Subject:
...  =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
...
... """)
>>> process(mlist, msg, {})
>>> print(msg['subject'].encode())
[XTest] =?iso-2022-jp?b?GyRCJWEhPCVrJV4lcxsoQg==?=
>>> print(msg['subject'])
[XTest]  メールマン
Message tagger

Mailman has a topics system which works like this: a mailing list administrator sets up one or more topics, which is essentially a named regular expression. The topic name can be any arbitrary string, and the name serves double duty as the topic tag. Each message that flows the mailing list has its Subject: and Keywords: headers compared against these regular expressions. The message then gets tagged with the topic names of each hit.

>>> mlist = create_list('_xtest@example.com')

Topics must be enabled for Mailman to do any topic matching, even if topics are defined.

>>> mlist.topics = [('bar fight', '.*bar.*', 'catch any bars', False)]
>>> mlist.topics_enabled = False
>>> mlist.topics_bodylines_limit = 0

>>> msg = message_from_string("""\
... Subject: foobar
... Keywords: barbaz
...
... """)
>>> msgdata = {}

>>> from mailman.handlers.tagger import process
>>> process(mlist, msg, msgdata)
>>> print(msg.as_string())
Subject: foobar
Keywords: barbaz


>>> msgdata
{}

However, once topics are enabled, message will be tagged. There are two artifacts of tagging; an X-Topics: header is added with the topic name, and the message metadata gets a key with a list of matching topic names.

>>> mlist.topics_enabled = True
>>> msg = message_from_string("""\
... Subject: foobar
... Keywords: barbaz
...
... """)
>>> msgdata = {}
>>> process(mlist, msg, msgdata)
>>> print(msg.as_string())
Subject: foobar
Keywords: barbaz
X-Topics: bar fight
<BLANKLINE>
<BLANKLINE>
>>> msgdata['topichits']
['bar fight']
Scanning body lines

The tagger can also look at a certain number of body lines, but only for Subject: and Keyword: header-like lines. When set to zero, no body lines are scanned.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: nothing
... Keywords: at all
...
... X-Ignore: something else
... Subject: foobar
... Keywords: barbaz
... """)
>>> msgdata = {}
>>> process(mlist, msg, msgdata)
>>> print(msg.as_string())
From: aperson@example.com
Subject: nothing
Keywords: at all
<BLANKLINE>
X-Ignore: something else
Subject: foobar
Keywords: barbaz
<BLANKLINE>
>>> msgdata
{}

But let the tagger scan a few body lines and the matching headers will be found.

>>> mlist.topics_bodylines_limit = 5
>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: nothing
... Keywords: at all
...
... X-Ignore: something else
... Subject: foobar
... Keywords: barbaz
... """)
>>> msgdata = {}
>>> process(mlist, msg, msgdata)
>>> print(msg.as_string())
From: aperson@example.com
Subject: nothing
Keywords: at all
X-Topics: bar fight
<BLANKLINE>
X-Ignore: something else
Subject: foobar
Keywords: barbaz
<BLANKLINE>
>>> msgdata['topichits']
['bar fight']

However, scanning stops at the first body line that doesn’t look like a header.

>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: nothing
... Keywords: at all
...
... This is not a header
... Subject: foobar
... Keywords: barbaz
... """)
>>> msgdata = {}
>>> process(mlist, msg, msgdata)
>>> print(msg.as_string())
From: aperson@example.com
Subject: nothing
Keywords: at all
<BLANKLINE>
This is not a header
Subject: foobar
Keywords: barbaz
>>> msgdata
{}

When set to a negative number, all body lines will be scanned.

>>> mlist.topics_bodylines_limit = -1
>>> lots_of_headers = '\n'.join(['X-Ignore: zip'] * 100)
>>> msg = message_from_string("""\
... From: aperson@example.com
... Subject: nothing
... Keywords: at all
...
... %s
... Subject: foobar
... Keywords: barbaz
... """ % lots_of_headers)
>>> msgdata = {}
>>> process(mlist, msg, msgdata)
>>> # Rather than print out 100 X-Ignore: headers, let's just prove that
>>> # the X-Topics: header exists, meaning that the tagger did its job.
>>> print(msg['x-topics'])
bar fight
>>> msgdata['topichits']
['bar fight']
Scanning sub-parts

The tagger will also scan the body lines of text subparts in a multipart message, using the same rules as if all those body lines lived in a single text payload.

>>> msg = message_from_string("""\
... Subject: Was
... Keywords: Raw
... Content-Type: multipart/alternative; boundary="BOUNDARY"
...
... --BOUNDARY
... From: sabo
... To: obas
...
... Subject: farbaw
... Keywords: barbaz
...
... --BOUNDARY--
... """)
>>> msgdata = {}
>>> process(mlist, msg, msgdata)
>>> print(msg.as_string())
Subject: Was
Keywords: Raw
Content-Type: multipart/alternative; boundary="BOUNDARY"
X-Topics: bar fight
<BLANKLINE>
--BOUNDARY
From: sabo
To: obas
<BLANKLINE>
Subject: farbaw
Keywords: barbaz
<BLANKLINE>
--BOUNDARY--
<BLANKLINE>
>>> msgdata['topichits']
['bar fight']

But the tagger will not descend into non-text parts.

>>> msg = message_from_string("""\
... Subject: Was
... Keywords: Raw
... Content-Type: multipart/alternative; boundary=BOUNDARY
...
... --BOUNDARY
... From: sabo
... To: obas
... Content-Type: message/rfc822
...
... Subject: farbaw
... Keywords: barbaz
...
... --BOUNDARY
... From: sabo
... To: obas
... Content-Type: message/rfc822
...
... Subject: farbaw
... Keywords: barbaz
...
... --BOUNDARY--
... """)
>>> msgdata = {}
>>> process(mlist, msg, msgdata)
>>> print(msg['x-topics'])
None
>>> msgdata
{}
The outgoing handler

Mailman’s outgoing queue is used as the wrapper around SMTP delivery to the upstream mail server. The to-outgoing handler does little more than drop the message into the outgoing queue.

>>> mlist = create_list('test@example.com')

Craft a message destined for the outgoing queue. Include some random metadata as if this message had passed through some other handlers.

>>> msg = message_from_string("""\
... Subject: Here is a message
...
... Something of great import.
... """)

>>> msgdata = dict(foo=1, bar=2, verp=True)
>>> handler = config.handlers['to-outgoing']
>>> handler.process(mlist, msg, msgdata)

While the queued message will not be changed, the queued metadata will have an additional key set: the mailing list name.

>>> from mailman.testing.helpers import get_queue_messages
>>> messages = get_queue_messages('out')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
Subject: Here is a message
<BLANKLINE>
Something of great import.
>>> dump_msgdata(messages[0].msgdata)
_parsemsg: False
bar      : 2
foo      : 1
listid   : test.example.com
verp     : True
version  : 3

Mailman 3 Core administrative REST API

Here is extensive documentation on the Mailman Core administrative REST API.

The REST server

Mailman exposes a REST HTTP server for administrative control.

The server listens for connections on a configurable host name and port.

It is always protected by HTTP basic authentication using a single global user name and password. The credentials are set in the [webservice] section of the configuration using the admin_user and admin_pass properties.

Because the REST server has full administrative access, it should never be exposed to the public internet. By default it only listens to connections on localhost. Don’t change this unless you really know what you’re doing. In addition you should set the user name and password to secure values and distribute them to any REST clients with reasonable precautions.

The Mailman major and minor version numbers are in the URL.

You can write your own HTTP clients to speak this API, or you can use the official Python bindings.

Basic operation

The encoding of URI components addressing a REST endpoint is Unicode UTF-8. There is more information about internationalization in Mailman.

In order to do anything with the REST API, you need to know its Basic AUTH credentials, and the version of the API you wish to speak to.

Credentials

If you include the proper basic authorization credentials, the request succeeds.

>>> import requests
>>> response = requests.get(
...     'http://localhost:9001/3.0/system/versions',
...     auth=('restadmin', 'restpass'))
>>> print(response.status_code)
200
System version information

System version information can be retrieved from the server, in the form of a JSON encoded response.

>>> dump_json('http://localhost:9001/3.0/system/versions')
api_version: 3.0
http_etag: "..."
mailman_version: GNU Mailman 3...
python_version: ...
self_link: http://localhost:9001/3.0/system/versions
API Versions

The REST API exposes two versions which are almost completely identical. As you’ve seen above, the 3.0 API is the base API. There is also a 3.1 API, which can be used interchangably:

>>> dump_json('http://localhost:9001/3.1/system/versions')
api_version: 3.1
http_etag: "..."
mailman_version: GNU Mailman 3...
python_version: ...
self_link: http://localhost:9001/3.1/system/versions

The only difference is the way UUIDs are represented. UUIDs are 128-bit unique ids for objects such as users and members. In version 3.0 of the API, UUIDs are represented as 128-bit integers, but these were found to be incompatible for some versions of JavaScript, so in API version 3.1 UUIDs are represented as hex strings.

Choose whichever API version makes sense for your application. In general, we recommend using API 3.1, but most of the current documentation describes API 3.0. Just make the mental substitution as you read along.

Collections and Pagination

All collections automatically support pagination. You can use this to limit the number of items of the collection that get returned, and you can page through the results by incrementing the page counter.

For example, let’s say we have 50 mailing lists.

>>> from mailman.app.lifecycle import create_list
>>> for i in range(50):
...     mlist = create_list('list{:02d}@example.com'.format(i))
>>> transaction.commit()

We can get the first 10 lists by asking for the first page of items.

>>> json = call_http('http://localhost:9001/3.0/lists?count=10&page=1')
>>> for entry in json['entries']:
...     print(entry['list_id'])
list00.example.com
list01.example.com
list02.example.com
list03.example.com
list04.example.com
list05.example.com
list06.example.com
list07.example.com
list08.example.com
list09.example.com

We can also ask for the third set of 10 mailing lists.

>>> json = call_http('http://localhost:9001/3.0/lists?count=10&page=3')
>>> for entry in json['entries']:
...     print(entry['list_id'])
list20.example.com
list21.example.com
list22.example.com
list23.example.com
list24.example.com
list25.example.com
list26.example.com
list27.example.com
list28.example.com
list29.example.com

Of course, we can also adjust the page size and ask for the tenth page of 5 mailing lists.

>>> json = call_http('http://localhost:9001/3.0/lists?count=5&page=10')
>>> for entry in json['entries']:
...     print(entry['list_id'])
list45.example.com
list46.example.com
list47.example.com
list48.example.com
list49.example.com
The size of a collection

This same idiom allows you to get just the size of the collection. You do this by asking for a page of size zero.

>>> dump_json('http://localhost:9001/3.0/lists?count=0&page=1')
http_etag: ...
start: 0
total_size: 50
Page start

Notice the start element in the returned JSON. This tells you which item of the collection the page starts on.

>>> dump_json('http://localhost:9001/3.0/lists?count=2&page=15')
entry 0:
    display_name: List28
    ...
entry 1:
    display_name: List29
    ...
http_etag: ...
start: 28
total_size: 50
REST API helpers

There are a number of helpers that make building out the REST API easier.

Etags

HTTP etags are a way for clients to decide whether their copy of a resource has changed or not. Mailman’s REST API calculates this in a cheap and dirty way. Pass in the dictionary representing the resource and that dictionary gets modified to contain the etag under the http_etag key.

>>> from mailman.rest.helpers import etag
>>> resource = dict(geddy='bass', alex='guitar', neil='drums')
>>> json_data = etag(resource)
>>> print(resource['http_etag'])
"6929ecfbda2282980a4818fb75f82e812077f77a"

For convenience, the etag function also returns the JSON representation of the dictionary after tagging, since that’s almost always what you want.

>>> import json
>>> data = json.loads(json_data)

# This is pretty close to what we want, so it's convenient to use.
>>> dump_msgdata(data)
alex     : guitar
geddy    : bass
http_etag: "6929ecfbda2282980a4818fb75f82e812077f77a"
neil     : drums
POST and PUT unpacking

Another helper unpacks POST and PUT request variables, validating and converting their values.

>>> from mailman.rest.validator import Validator
>>> validator = Validator(one=int, two=str, three=bool)

>>> class FakeRequest:
...     params = None
>>> FakeRequest.params = dict(one='1', two='two', three='yes')

On valid input, the validator can be used as a **keyword argument.

>>> def print_request(one, two, three):
...     print(repr(one), repr(two), repr(three))
>>> print_request(**validator(FakeRequest))
1 'two' True

On invalid input, an exception is raised.

>>> FakeRequest.params['one'] = 'hello'
>>> print_request(**validator(FakeRequest))
Traceback (most recent call last):
...
ValueError: Cannot convert parameters: one

On missing input, an exception is raised.

>>> del FakeRequest.params['one']
>>> print_request(**validator(FakeRequest))
Traceback (most recent call last):
...
ValueError: Missing parameters: one

If more than one key is missing, it will be reflected in the error message.

>>> del FakeRequest.params['two']
>>> print_request(**validator(FakeRequest))
Traceback (most recent call last):
...
ValueError: Missing parameters: one, two

Extra keys are also not allowed.

>>> FakeRequest.params = dict(one='1', two='two', three='yes',
...                           four='', five='')
>>> print_request(**validator(FakeRequest))
Traceback (most recent call last):
...
ValueError: Unexpected parameters: five, four

However, if optional keys are missing, it’s okay.

>>> validator = Validator(one=int, two=str, three=bool,
...                       four=int, five=int,
...                       _optional=('four', 'five'))

>>> FakeRequest.params = dict(one='1', two='two', three='yes',
...                           four='4', five='5')
>>> def print_request(one, two, three, four=None, five=None):
...     print(repr(one), repr(two), repr(three), repr(four), repr(five))
>>> print_request(**validator(FakeRequest))
1 'two' True 4 5

>>> del FakeRequest.params['four']
>>> print_request(**validator(FakeRequest))
1 'two' True None 5

>>> del FakeRequest.params['five']
>>> print_request(**validator(FakeRequest))
1 'two' True None None

But if the optional values are present, they must of course also be valid.

>>> FakeRequest.params = dict(one='1', two='two', three='yes',
...                           four='no', five='maybe')
>>> print_request(**validator(FakeRequest))
Traceback (most recent call last):
...
ValueError: Cannot convert parameters: five, four
Arrays

Some POST forms include more than one value for a particular key. This is how lists and arrays are modeled. The validator does the right thing with such form data. Specifically, when a key shows up multiple times in the form data, a list is given to the validator.

# We can't use a normal dictionary because we'll have multiple keys, but
# the validator only wants to call .items() on the object.
>>> class MultiDict:
...     def __init__(self, *params): self.values = list(params)
...     def items(self): return iter(self.values)
>>> form_data = MultiDict(
...     ('one', '1'),
...     ('many', '3'),
...     ('many', '4'),
...     ('many', '5'),
...     )

This is a validation function that ensures the value is a list.

>>> def must_be_list(value):
...     if not isinstance(value, list):
...         raise ValueError('not a list')
...     return [int(item) for item in value]

This is a validation function that ensure the value is not a list.

>>> def must_be_scalar(value):
...     if isinstance(value, list):
...         raise ValueError('is a list')
...     return int(value)

And a validator to pull it all together.

>>> validator = Validator(one=must_be_scalar, many=must_be_list)
>>> FakeRequest.params = form_data
>>> values = validator(FakeRequest)
>>> print(values['one'])
1
>>> print(values['many'])
[3, 4, 5]

The list values are guaranteed to be in the same order they show up in the form data.

>>> FakeRequest.params = MultiDict(
...     ('one', '1'),
...     ('many', '3'),
...     ('many', '5'),
...     ('many', '4'),
...     )
>>> values = validator(FakeRequest)
>>> print(values['one'])
1
>>> print(values['many'])
[3, 5, 4]
System configuration

The entire system configuration is available through the REST API. You can get a list of all defined sections.

>>> dump_json('http://localhost:9001/3.0/system/configuration')
http_etag: ...
sections: ['antispam', 'archiver.mail_archive', 'archiver.master', ...
self_link: http://localhost:9001/3.0/system/configuration

You can also get all the values for a particular section, such as the [mailman] section…

>>> dump_json('http://localhost:9001/3.0/system/configuration/mailman')
cache_life: 7d
default_language: en
email_commands_max_lines: 10
filtered_messages_are_preservable: no
html_to_plain_text_command: /usr/bin/lynx -dump $filename
http_etag: ...
layout: testing
listname_chars: [-_.0-9a-z]
noreply_address: noreply
pending_request_life: 3d
post_hook:
pre_hook:
self_link: http://localhost:9001/3.0/system/configuration/mailman
sender_headers: from from_ reply-to sender
site_owner: noreply@example.com

…or the [dmarc] section (or any other).

>>> dump_json('http://localhost:9001/3.0/system/configuration/dmarc')
cache_lifetime: 7d
http_etag: ...
org_domain_data_url: https://publicsuffix.org/list/public_suffix_list.dat
resolver_lifetime: 5s
resolver_timeout: 3s
self_link: http://localhost:9001/3.0/system/configuration/dmarc

Dotted section names work too, for example, to get the French language settings section.

>>> dump_json('http://localhost:9001/3.0/system/configuration/language.fr')
charset: iso-8859-1
description: French
enabled: yes
http_etag: ...
self_link: http://localhost:9001/3.0/system/configuration/language.fr
Domains

Domains are how Mailman interacts with email host names and web host names.

# The test framework starts out with an example domain, so let's delete
# that first.
>>> from mailman.interfaces.domain import IDomainManager
>>> from zope.component import getUtility
>>> domain_manager = getUtility(IDomainManager)

>>> domain_manager.remove('example.com')
<Domain example.com...>
>>> transaction.commit()

The REST API can be queried for the set of known domains, of which there are initially none.

>>> dump_json('http://localhost:9001/3.0/domains')
http_etag: "..."
start: 0
total_size: 0

Once a domain is added, it is accessible through the API.

>>> domain_manager.add('example.com', 'An example domain')
<Domain example.com, An example domain>
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/domains')
entry 0:
    description: An example domain
    http_etag: "..."
    mail_host: example.com
    self_link: http://localhost:9001/3.0/domains/example.com
http_etag: "..."
start: 0
total_size: 1

At the top level, all domains are returned as separate entries.

>>> domain_manager.add('example.org',)
<Domain example.org>
>>> domain_manager.add(
...     'lists.example.net',
...     'Porkmasters')
<Domain lists.example.net, Porkmasters>
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/domains')
entry 0:
    description: An example domain
    http_etag: "..."
    mail_host: example.com
    self_link: http://localhost:9001/3.0/domains/example.com
entry 1:
    description: None
    http_etag: "..."
    mail_host: example.org
    self_link: http://localhost:9001/3.0/domains/example.org
entry 2:
    description: Porkmasters
    http_etag: "..."
    mail_host: lists.example.net
    self_link: http://localhost:9001/3.0/domains/lists.example.net
http_etag: "..."
start: 0
total_size: 3
Individual domains

The information for a single domain is available by following one of the self_links from the above collection.

>>> dump_json('http://localhost:9001/3.0/domains/lists.example.net')
description: Porkmasters
http_etag: "..."
mail_host: lists.example.net
self_link: http://localhost:9001/3.0/domains/lists.example.net

You can also list all the mailing lists for a given domain. At first, the example.com domain does not contain any mailing lists.

>>> dump_json('http://localhost:9001/3.0/domains/example.com/lists')
http_etag: "..."
start: 0
total_size: 0

>>> dump_json('http://localhost:9001/3.0/lists', {
...           'fqdn_listname': 'test-domains@example.com',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/lists/test-domains.example.com
...

>>> dump_json('http://localhost:9001/3.0/domains/example.com/lists')
entry 0:
    display_name: Test-domains
    fqdn_listname: test-domains@example.com
    http_etag: "..."
    ...
    member_count: 0
    self_link: http://localhost:9001/3.0/lists/test-domains.example.com
    volume: 1
http_etag: "..."
start: 0
total_size: 1

Other domains continue to contain no mailing lists.

>>> dump_json('http://localhost:9001/3.0/domains/lists.example.net/lists')
http_etag: "..."
start: 0
total_size: 0
Creating new domains

New domains can be created by posting to the domains url.

>>> dump_json('http://localhost:9001/3.0/domains', {
...           'mail_host': 'lists.example.com',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/domains/lists.example.com
...

Now the web service knows about our new domain.

>>> dump_json('http://localhost:9001/3.0/domains/lists.example.com')
description: None
http_etag: "..."
mail_host: lists.example.com
self_link: http://localhost:9001/3.0/domains/lists.example.com

And the new domain is in our database.

>>> domain_manager['lists.example.com']
<Domain lists.example.com>

# Unlock the database.
>>> transaction.abort()

You can also create a new domain with a description and a contact address.

>>> dump_json('http://localhost:9001/3.0/domains', {
...           'mail_host': 'my.example.com',
...           'description': 'My new domain',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/domains/my.example.com
...

>>> dump_json('http://localhost:9001/3.0/domains/my.example.com')
description: My new domain
http_etag: "..."
mail_host: my.example.com
self_link: http://localhost:9001/3.0/domains/my.example.com

>>> domain_manager['my.example.com']
<Domain my.example.com, My new domain>

# Unlock the database.
>>> transaction.abort()
Deleting domains

Domains can also be deleted via the API.

>>> dump_json('http://localhost:9001/3.0/domains/lists.example.com',
...           method='DELETE')
content-length: 0
date: ...
server: ...
status: 204
Domain owners

Domains can have owners. By posting some addresses to the owners resource, you can add some domain owners. Currently our domain has no owners:

>>> dump_json('http://localhost:9001/3.0/domains/my.example.com/owners')
http_etag: ...
start: 0
total_size: 0

Anne and Bart volunteer to be a domain owners.

>>> dump_json('http://localhost:9001/3.0/domains/my.example.com/owners', (
...     ('owner', 'anne@example.com'), ('owner', 'bart@example.com')
...     ))
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/domains/my.example.com/owners')
entry 0:
    created_on: 2005-08-01T07:49:23
    http_etag: ...
    is_server_owner: False
    self_link: http://localhost:9001/3.0/users/1
    user_id: 1
entry 1:
    created_on: 2005-08-01T07:49:23
    http_etag: ...
    is_server_owner: False
    self_link: http://localhost:9001/3.0/users/2
    user_id: 2
http_etag: ...
start: 0
total_size: 2

We can delete all the domain owners.

>>> dump_json('http://localhost:9001/3.0/domains/my.example.com/owners',
...           method='DELETE')
content-length: 0
date: ...
server: ...
status: 204

Now there are no owners.

>>> dump_json('http://localhost:9001/3.0/domains/my.example.com/owners')
http_etag: ...
start: 0
total_size: 0

New domains can be created with owners.

>>> dump_json('http://localhost:9001/3.0/domains', (
...           ('mail_host', 'your.example.com'),
...           ('owner', 'anne@example.com'),
...           ('owner', 'bart@example.com'),
...           ))
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/domains/your.example.com
server: ...
status: 201

The new domain has the expected owners.

>>> dump_json('http://localhost:9001/3.0/domains/your.example.com/owners')
entry 0:
    created_on: 2005-08-01T07:49:23
    http_etag: ...
    is_server_owner: False
    self_link: http://localhost:9001/3.0/users/1
    user_id: 1
entry 1:
    created_on: 2005-08-01T07:49:23
    http_etag: ...
    is_server_owner: False
    self_link: http://localhost:9001/3.0/users/2
    user_id: 2
http_etag: ...
start: 0
total_size: 2
Mailing lists

The REST API can be queried for the set of known mailing lists. There is a top level collection that can return all the mailing lists. There aren’t any yet though.

>>> dump_json('http://localhost:9001/3.0/lists')
http_etag: "..."
start: 0
total_size: 0

Create a mailing list in a domain and it’s accessible via the API.

>>> mlist = create_list('ant@example.com')
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/lists')
entry 0:
    display_name: Ant
    fqdn_listname: ant@example.com
    http_etag: "..."
    list_id: ant.example.com
    list_name: ant
    mail_host: example.com
    member_count: 0
    self_link: http://localhost:9001/3.0/lists/ant.example.com
    volume: 1
http_etag: "..."
start: 0
total_size: 1

You can also query for lists from a particular domain.

>>> dump_json('http://localhost:9001/3.0/domains/example.com/lists')
entry 0:
    display_name: Ant
    fqdn_listname: ant@example.com
    http_etag: "..."
    list_id: ant.example.com
    list_name: ant
    mail_host: example.com
    member_count: 0
    self_link: http://localhost:9001/3.0/lists/ant.example.com
    volume: 1
http_etag: "..."
start: 0
total_size: 1

Advertised lists can be filtered using the advertised query parameter.

>>> mlist = create_list('elk@example.com')
>>> mlist.advertised = False
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/lists?advertised=true')
entry 0:
    ...
    list_id: ant.example.com
    ...
http_etag: "..."
start: 0
total_size: 1

The same applies to lists from a particular domain.

>>> dump_json('http://localhost:9001/3.0/domains/example.com'
...           '/lists?advertised=true')
entry 0:
    ...
    list_id: ant.example.com
    ...
http_etag: "..."
start: 0
total_size: 1
Paginating over list records

Instead of returning all the list records at once, it’s possible to return them in pages by adding the GET parameters count and page to the request URI. Page 1 is the first page and count defines the size of the page.

>>> dump_json('http://localhost:9001/3.0/domains/example.com/lists'
...           '?count=1&page=1')
entry 0:
    display_name: Ant
    fqdn_listname: ant@example.com
    http_etag: "..."
    list_id: ant.example.com
    list_name: ant
    mail_host: example.com
    member_count: 0
    self_link: http://localhost:9001/3.0/lists/ant.example.com
    volume: 1
http_etag: "..."
start: 0
total_size: 2

>>> dump_json('http://localhost:9001/3.0/domains/example.com/lists'
...           '?count=1&page=2')
entry 0:
    display_name: Elk
    fqdn_listname: elk@example.com
    http_etag: "..."
    list_id: elk.example.com
    list_name: elk
    mail_host: example.com
    member_count: 0
    self_link: http://localhost:9001/3.0/lists/elk.example.com
    volume: 1
http_etag: "..."
start: 1
total_size: 2
Creating lists via the API

New mailing lists can also be created through the API, by posting to the lists URL.

>>> dump_json('http://localhost:9001/3.0/lists', {
...           'fqdn_listname': 'bee@example.com',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/lists/bee.example.com
...

The mailing list exists in the database.

>>> from mailman.interfaces.listmanager import IListManager
>>> from zope.component import getUtility
>>> list_manager = getUtility(IListManager)

>>> bee = list_manager.get('bee@example.com')
>>> bee
<mailing list "bee@example.com" at ...>

The mailing list was created using the default style, which allows list posts.

>>> bee.allow_list_posts
True

It is also available in the REST API via the location given in the response.

>>> dump_json('http://localhost:9001/3.0/lists/bee.example.com')
display_name: Bee
fqdn_listname: bee@example.com
http_etag: "..."
list_id: bee.example.com
list_name: bee
mail_host: example.com
member_count: 0
self_link: http://localhost:9001/3.0/lists/bee.example.com
volume: 1

Normally, you access the list via its RFC 2369 list-id as shown above, but for backward compatibility purposes, you can also access it via the list’s posting address, if that has never been changed (since the list-id is immutable, but the posting address is not).

>>> dump_json('http://localhost:9001/3.0/lists/bee@example.com')
display_name: Bee
fqdn_listname: bee@example.com
http_etag: "..."
list_id: bee.example.com
list_name: bee
mail_host: example.com
member_count: 0
self_link: http://localhost:9001/3.0/lists/bee.example.com
volume: 1
Apply a style at list creation time

List styles allow you to more easily create mailing lists of a particular type, e.g. discussion lists. We can see which styles are available, and which is the default style.

>>> dump_json('http://localhost:9001/3.0/lists/styles')
default: legacy-default
http_etag: "..."
style_names: ['legacy-announce', 'legacy-default']

When creating a list, if we don’t specify a style to apply, the default style is used. However, we can provide a style name in the POST data to choose a different style.

>>> dump_json('http://localhost:9001/3.0/lists', {
...           'fqdn_listname': 'cat@example.com',
...           'style_name': 'legacy-announce',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/lists/cat.example.com
...

We can tell that the list was created using the legacy-announce style, because announce lists don’t allow posting by the general public.

>>> cat = list_manager.get('cat@example.com')
>>> cat.allow_list_posts
False
Deleting lists via the API

Existing mailing lists can be deleted through the API, by doing an HTTP DELETE on the mailing list URL.

>>> dump_json('http://localhost:9001/3.0/lists/bee.example.com',
...           method='DELETE')
content-length: 0
date: ...
server: ...
status: 204

The mailing list does not exist.

>>> print(list_manager.get('bee@example.com'))
None

For backward compatibility purposes, you can delete a list via its posting address as well.

>>> dump_json('http://localhost:9001/3.0/lists/ant@example.com',
...           method='DELETE')
content-length: 0
date: ...
server: ...
status: 204

The mailing list does not exist.

>>> print(list_manager.get('ant@example.com'))
None
Managing mailing list archivers

The Mailman system has some site-wide enabled archivers, and each mailing list can enable or disable these archivers individually. This gives list owners control over where traffic to their list is archived. You can see which archivers are available, and whether they are enabled for this mailing list.

>>> mlist = create_list('dog@example.com')
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/lists/dog@example.com/archivers')
http_etag: "..."
mail-archive: True
mhonarc: True

You can set all the archiver states by putting new state flags on the resource.

>>> dump_json(
...     'http://localhost:9001/3.0/lists/dog@example.com/archivers', {
...         'mail-archive': False,
...         'mhonarc': True,
...         }, method='PUT')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/lists/dog@example.com/archivers')
http_etag: "..."
mail-archive: False
mhonarc: True

You can change the state of a subset of the list archivers.

>>> dump_json(
...     'http://localhost:9001/3.0/lists/dog@example.com/archivers', {
...         'mhonarc': False,
...         }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/lists/dog@example.com/archivers')
http_etag: "..."
mail-archive: False
mhonarc: False
List digests

A list collects messages and prepares a digest which can be periodically sent to all members who elect to receive digests. Digests are usually sent whenever their size has reached a threshold, but you can force a digest to be sent immediately via the REST API.

Let’s create a mailing list that has a digest recipient.

>>> from mailman.interfaces.member import DeliveryMode
>>> from mailman.testing.helpers import subscribe
>>> emu = create_list('emu@example.com')
>>> emu.send_welcome_message = False
>>> anne = subscribe(emu, 'Anne')
>>> anne.preferences.delivery_mode = DeliveryMode.plaintext_digests

The mailing list has a fairly high size threshold so that sending a single message through the list won’t trigger an automatic digest. The threshold is the maximum digest size in kibibytes (1024 bytes).

>>> emu.digest_size_threshold = 100
>>> transaction.commit()

We send a message through the mailing list to start collecting for a digest.

>>> from mailman.runners.digest import DigestRunner
>>> from mailman.testing.helpers import make_testable_runner
>>> msg = message_from_string("""\
... From: anne@example.com
... To: emu@example.com
... Subject: Message #1
...
... """)
>>> config.handlers['to-digest'].process(emu, msg, {})
>>> runner = make_testable_runner(DigestRunner, 'digest')
>>> runner.run()

No digest was sent because it didn’t reach the size threshold.

>>> from mailman.testing.helpers import get_queue_messages
>>> len(get_queue_messages('virgin'))
0

By POSTing to the list’s digest end-point with the send parameter set, we can force the digest to be sent.

>>> dump_json('http://localhost:9001/3.0/lists/emu.example.com/digest', {
...           'send': True,
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...

Once the runner does its thing, the digest message will be sent.

>>> runner.run()
>>> items = get_queue_messages('virgin')
>>> len(items)
1
>>> print(items[0].msg)
From: emu-request@example.com
Subject: Emu Digest, Vol 1, Issue 1
To: emu@example.com
...
From: anne@example.com
Subject: Message #1
To: emu@example.com
...
End of Emu Digest, Vol 1, Issue 1
*********************************
<BLANKLINE>

Digests also have a volume number and digest number which can be bumped, also by POSTing to the REST API. Bumping the digest for this list will increment the digest volume and reset the digest number to 1. We have to fake that the last digest was sent a couple of days ago.

>>> from datetime import timedelta
>>> from mailman.interfaces.digests import DigestFrequency
>>> emu.digest_volume_frequency = DigestFrequency.daily
>>> emu.digest_last_sent_at -= timedelta(days=2)
>>> transaction.commit()

Before bumping, we can get the next digest volume and number. Doing a GET on the digest resource is just a shorthand for getting some interesting information about the digest. Note that volume and next_digest_number can also be retrieved from the list’s configuration resource.

>>> dump_json('http://localhost:9001/3.0/lists/emu.example.com/digest')
http_etag: ...
next_digest_number: 2
volume: 1

Let’s bump the digest.

>>> dump_json('http://localhost:9001/3.0/lists/emu.example.com/digest', {
...           'bump': True,
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...

And now the next digest to be sent will have a new volume number.

>>> dump_json('http://localhost:9001/3.0/lists/emu.example.com/digest')
http_etag: ...
next_digest_number: 1
volume: 2
Mailing list configuration

Mailing lists can be configured via the REST API.

>>> mlist = create_list('ant@example.com')
>>> transaction.commit()
Reading a configuration

All readable attributes for a list are available on a sub-resource.

>>> dump_json('http://localhost:9001/3.0/lists/ant@example.com/config')
acceptable_aliases: []
admin_immed_notify: True
admin_notify_mchanges: False
administrivia: True
advertised: True
allow_list_posts: True
anonymous_list: False
archive_policy: public
autorespond_owner: none
autorespond_postings: none
autorespond_requests: none
autoresponse_grace_period: 90d
autoresponse_owner_text:
autoresponse_postings_text:
autoresponse_request_text:
bounces_address: ant-bounces@example.com
collapse_alternatives: True
convert_html_to_plaintext: False
created_at: 20...T...
default_member_action: defer
default_nonmember_action: hold
description:
digest_footer_uri:
digest_header_uri:
digest_last_sent_at: None
digest_send_periodic: True
digest_size_threshold: 30.0
digest_volume_frequency: monthly
digests_enabled: True
display_name: Ant
dmarc_mitigate_action: no_mitigation
dmarc_mitigate_unconditionally: False
dmarc_moderation_notice:
dmarc_wrapped_message_text:
filter_content: False
first_strip_reply_to: False
footer_uri:
fqdn_listname: ant@example.com
goodbye_message_uri:
header_uri:
http_etag: "..."
include_rfc2369_headers: True
info:
join_address: ant-join@example.com
last_post_at: None
leave_address: ant-leave@example.com
list_name: ant
mail_host: example.com
moderator_password: None
next_digest_number: 1
no_reply_address: noreply@example.com
owner_address: ant-owner@example.com
post_id: 1
posting_address: ant@example.com
posting_pipeline: default-posting-pipeline
reply_goes_to_list: no_munging
reply_to_address:
request_address: ant-request@example.com
send_welcome_message: True
subject_prefix: [Ant]
subscription_policy: confirm
volume: 1
welcome_message_uri:
Changing the full configuration

Not all of the readable attributes can be set through the web interface. The ones that can, can either be set via PUT or PATCH. PUT changes all the writable attributes in one request.

When using PUT, all writable attributes must be included.

>>> dump_json('http://localhost:9001/3.0/lists/'
...           'ant@example.com/config',
...           dict(
...             acceptable_aliases=['one@example.com', 'two@example.com'],
...             admin_immed_notify=False,
...             admin_notify_mchanges=True,
...             administrivia=False,
...             advertised=False,
...             anonymous_list=True,
...             archive_policy='never',
...             autorespond_owner='respond_and_discard',
...             autorespond_postings='respond_and_continue',
...             autorespond_requests='respond_and_discard',
...             autoresponse_grace_period='45d',
...             autoresponse_owner_text='the owner',
...             autoresponse_postings_text='the mailing list',
...             autoresponse_request_text='the robot',
...             display_name='Fnords',
...             description='This is my mailing list',
...             include_rfc2369_headers=False,
...             info='This is the mailing list information',
...             allow_list_posts=False,
...             digest_send_periodic=False,
...             digest_size_threshold=10.5,
...             digest_volume_frequency='yearly',
...             digests_enabled=False,
...             dmarc_mitigate_action='munge_from',
...             dmarc_mitigate_unconditionally=False,
...             dmarc_moderation_notice='Some moderation notice',
...             dmarc_wrapped_message_text='some message text',
...             posting_pipeline='virgin',
...             filter_content=True,
...             first_strip_reply_to=True,
...             convert_html_to_plaintext=True,
...             collapse_alternatives=False,
...             reply_goes_to_list='point_to_list',
...             reply_to_address='bee@example.com',
...             send_welcome_message=False,
...             subject_prefix='[ant]',
...             subscription_policy='moderate',
...             default_member_action='hold',
...             default_nonmember_action='discard',
...             moderator_password='password',
...             ),
...           'PUT')
content-length: 0
date: ...
server: WSGIServer/...
status: 204

These values are changed permanently.

>>> dump_json('http://localhost:9001/3.0/lists/'
...           'ant@example.com/config')
acceptable_aliases: ['one@example.com', 'two@example.com']
admin_immed_notify: False
admin_notify_mchanges: True
administrivia: False
advertised: False
allow_list_posts: False
anonymous_list: True
archive_policy: never
autorespond_owner: respond_and_discard
autorespond_postings: respond_and_continue
autorespond_requests: respond_and_discard
autoresponse_grace_period: 45d
autoresponse_owner_text: the owner
autoresponse_postings_text: the mailing list
autoresponse_request_text: the robot
...
collapse_alternatives: False
convert_html_to_plaintext: True
...
default_member_action: hold
default_nonmember_action: discard
description: This is my mailing list
...
digest_send_periodic: False
digest_size_threshold: 10.5
digest_volume_frequency: yearly
digests_enabled: False
display_name: Fnords
dmarc_mitigate_action: munge_from
dmarc_mitigate_unconditionally: False
dmarc_moderation_notice: Some moderation notice
dmarc_wrapped_message_text: some message text
filter_content: True
first_strip_reply_to: True
footer_uri:
fqdn_listname: ant@example.com
...
include_rfc2369_headers: False
...
moderator_password: {plaintext}password
...
posting_pipeline: virgin
reply_goes_to_list: point_to_list
reply_to_address: bee@example.com
...
send_welcome_message: False
subject_prefix: [ant]
subscription_policy: moderate
...
Changing a partial configuration

Using PATCH, you can change just one attribute.

>>> dump_json('http://localhost:9001/3.0/lists/'
...           'ant@example.com/config',
...           dict(display_name='My List'),
...           'PATCH')
content-length: 0
date: ...
server: ...
status: 204

These values are changed permanently.

>>> print(mlist.display_name)
My List
Sub-resources

Mailing list configuration variables are actually available as sub-resources on the mailing list. Their values can be retrieved and set through the sub-resource.

Simple resources

You can view the current value of the sub-resource.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/config/display_name')
display_name: My List
http_etag: ...

The resource can be changed by PUTting to it. Note that the value still requires a dictionary, and that dictionary must have a single key matching the name of the resource.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/config/display_name',
...           dict(display_name='Your List'),
...           'PUT')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/config/display_name')
display_name: Your List
http_etag: ...

PATCH works the same way, with the same effect, so you can choose to use either method.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/config/display_name',
...           dict(display_name='Their List'),
...           'PATCH')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/config/display_name')
display_name: Their List
http_etag: ...
Acceptable aliases

These are recipient aliases that can be used in the To: and CC: headers instead of the posting address. They are often used in forwarded emails. By default, a mailing list has no acceptable aliases.

>>> from mailman.interfaces.mailinglist import IAcceptableAliasSet
>>> IAcceptableAliasSet(mlist).clear()
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/lists/'
...           'ant@example.com/config/acceptable_aliases')
acceptable_aliases: []
http_etag: "..."

We can add a few by PUT-ing them on the sub-resource. The keys in the dictionary are ignored.

>>> dump_json('http://localhost:9001/3.0/lists/'
...           'ant@example.com/config/acceptable_aliases',
...           dict(acceptable_aliases=['foo@example.com',
...                                    'bar@example.net']),
...           'PUT')
content-length: 0
date: ...
server: WSGIServer/...
status: 204

You can get all the mailing list’s acceptable aliases through the REST API.

>>> response = call_http(
...     'http://localhost:9001/3.0/lists/'
...     'ant@example.com/config/acceptable_aliases')
>>> for alias in response['acceptable_aliases']:
...     print(alias)
bar@example.net
foo@example.com

The mailing list has its aliases set.

>>> from mailman.interfaces.mailinglist import IAcceptableAliasSet
>>> aliases = IAcceptableAliasSet(mlist)
>>> for alias in sorted(aliases.aliases):
...     print(alias)
bar@example.net
foo@example.com

The aliases can be removed by using DELETE.

>>> response = call_http(
...     'http://localhost:9001/3.0/lists/'
...     'ant@example.com/config/acceptable_aliases',
...     method='DELETE')
content-length: 0
date: ...
server: WSGIServer/...
status: 204

Now the mailing list has no aliases.

>>> aliases = IAcceptableAliasSet(mlist)
>>> print(len(list(aliases.aliases)))
0
Header matches

Mailman can do pattern based header matching during its normal rule processing. Each mailing list can also be configured with a set of header matching regular expression rules. These can be used to impose list-specific header filtering with the same semantics as the global [antispam] section, or to have a different action.

The list of header matches for a mailing list are returned on the header-matches child of this list.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches')
http_etag: "..."
start: 0
total_size: 0

New header matches can be created by POSTing to the resource.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches', {
...           'header': 'X-Spam-Flag',
...           'pattern': '^Yes',
...           })
content-length: 0
...
location: .../3.0/lists/ant.example.com/header-matches/0
...
status: 201

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches/0')
header: x-spam-flag
http_etag: "..."
pattern: ^Yes
position: 0
self_link: http://localhost:9001/3.0/lists/ant.example.com/header-matches/0

To follow the global antispam action, the header match rule must not specify an action key, which names the chain to jump to if the rule matches. If the default antispam action is changed in the configuration file and Mailman is restarted, those rules will get the new jump action. If a specific action is desired, the action key must name a valid chain to jump to.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches', {
...           'header': 'X-Spam-Status',
...           'pattern': '^Yes',
...           'action': 'discard',
...           })
content-length: 0
...
location: .../3.0/lists/ant.example.com/header-matches/1
...
status: 201

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches/1')
action: discard
header: x-spam-status
http_etag: "..."
pattern: ^Yes
position: 1
self_link: http://localhost:9001/3.0/lists/ant.example.com/header-matches/1

The resource can be changed by PATCHing it. The position key can be used to change the priority of the header match in the list. If it is not supplied, the priority is not changed.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches/1',
...           dict(pattern='^No', action='accept'),
...           'PATCH')
content-length: 0
date: ...
server: ...
status: 204
>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches/1')
action: accept
header: x-spam-status
http_etag: "..."
pattern: ^No
position: 1
self_link: http://localhost:9001/3.0/lists/ant.example.com/header-matches/1

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches/1',
...           dict(position=0),
...           'PATCH')
content-length: 0
date: ...
server: ...
status: 204
>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches')
entry 0:
    action: accept
    header: x-spam-status
    http_etag: "..."
    pattern: ^No
    position: 0
    self_link: .../lists/ant.example.com/header-matches/0
entry 1:
    header: x-spam-flag
    http_etag: "..."
    pattern: ^Yes
    position: 1
    self_link: .../lists/ant.example.com/header-matches/1
http_etag: "..."
start: 0
total_size: 2

The PUT method can replace an entire header match. The position key is optional; if it is omitted, the order will not be changed.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches/1',
...           dict(header='X-Spam-Status',
...                pattern='^Yes',
...                action='hold',
...           ), 'PUT')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches/1')
action: hold
header: x-spam-status
http_etag: "..."
pattern: ^Yes
position: 1
self_link: http://localhost:9001/3.0/lists/ant.example.com/header-matches/1

A header match can be removed using the DELETE method.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches/1',
...           method='DELETE')
content-length: 0
...
status: 204

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches')
entry 0:
    action: accept
    header: x-spam-status
    http_etag: "..."
    pattern: ^No
    position: 0
    self_link: .../lists/ant.example.com/header-matches/0
http_etag: "..."
start: 0
total_size: 1

The mailing list’s header matches can be cleared by issuing a DELETE request on the top resource.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches',
...           method='DELETE')
content-length: 0
...
status: 204

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/header-matches')
http_etag: "..."
start: 0
total_size: 0
Addresses

The REST API can be used to manage addresses.

There are no addresses yet.

>>> dump_json('http://localhost:9001/3.0/addresses')
http_etag: "..."
start: 0
total_size: 0

When an address is created via the internal API, it is available in the REST API.

>>> from zope.component import getUtility
>>> from mailman.interfaces.usermanager import IUserManager
>>> user_manager = getUtility(IUserManager)
>>> anne = user_manager.create_address('anne@example.com')
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/addresses')
entry 0:
    email: anne@example.com
    http_etag: "..."
    original_email: anne@example.com
    registered_on: 2005-08-01T07:49:23
    self_link: http://localhost:9001/3.0/addresses/anne@example.com
http_etag: "..."
start: 0
total_size: 1

Anne’s address can also be accessed directly.

>>> dump_json('http://localhost:9001/3.0/addresses/anne@example.com')
email: anne@example.com
http_etag: "..."
original_email: anne@example.com
registered_on: 2005-08-01T07:49:23
self_link: http://localhost:9001/3.0/addresses/anne@example.com

Bart registers with a mixed-case address. The canonical URL always includes the lower-case version.

>>> bart = user_manager.create_address('Bart.Person@example.com')
>>> transaction.commit()
>>> dump_json(
...     'http://localhost:9001/3.0/addresses/bart.person@example.com')
email: bart.person@example.com
http_etag: "..."
original_email: Bart.Person@example.com
registered_on: 2005-08-01T07:49:23
self_link: http://localhost:9001/3.0/addresses/bart.person@example.com

But his address record can be accessed with the case-preserved version too.

>>> dump_json(
...     'http://localhost:9001/3.0/addresses/Bart.Person@example.com')
email: bart.person@example.com
http_etag: "..."
original_email: Bart.Person@example.com
registered_on: 2005-08-01T07:49:23
self_link: http://localhost:9001/3.0/addresses/bart.person@example.com

When an address has a real name associated with it, this is also available in the REST API.

>>> cris = user_manager.create_address('cris@example.com', 'Cris Person')
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com')
display_name: Cris Person
email: cris@example.com
http_etag: "..."
original_email: cris@example.com
registered_on: 2005-08-01T07:49:23
self_link: http://localhost:9001/3.0/addresses/cris@example.com
Verifying

When the address gets verified, this attribute is available in the REST representation.

>>> from mailman.utilities.datetime import now
>>> anne.verified_on = now()
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/addresses/anne@example.com')
email: anne@example.com
http_etag: "..."
original_email: anne@example.com
registered_on: 2005-08-01T07:49:23
self_link: http://localhost:9001/3.0/addresses/anne@example.com
verified_on: 2005-08-01T07:49:23

Addresses can also be verified through the REST API, by POSTing to the ‘verify’ sub-resource. The POST data is ignored.

>>> dump_json('http://localhost:9001/3.0/addresses/'
...           'cris@example.com/verify', {})
content-length: 0
date: ...
server: ...
status: 204

Now Cris’s address is verified.

>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com')
display_name: Cris Person
email: cris@example.com
http_etag: "..."
original_email: cris@example.com
registered_on: 2005-08-01T07:49:23
self_link: http://localhost:9001/3.0/addresses/cris@example.com
verified_on: 2005-08-01T07:49:23

If you should ever need to ‘unverify’ an address, POST to the ‘unverify’ sub-resource. Again, the POST data is ignored.

>>> dump_json('http://localhost:9001/3.0/addresses/'
...           'cris@example.com/unverify', {})
content-length: 0
date: ...
server: ...
status: 204

Now Cris’s address is unverified.

>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com')
display_name: Cris Person
email: cris@example.com
http_etag: "..."
original_email: cris@example.com
registered_on: 2005-08-01T07:49:23
self_link: http://localhost:9001/3.0/addresses/cris@example.com
The user

To link an address to a user, a POST request can be sent to the /user sub-resource of the address. If the user does not exist, it will be created.

>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com/user',
...           {'display_name': 'Cris X. Person'})
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/users/1
server: ...
status: 201

The user is now created and the address is linked to it:

>>> cris.user
<User "Cris X. Person" (1) at 0x...>
>>> cris_user = user_manager.get_user('cris@example.com')
>>> cris_user
<User "Cris X. Person" (1) at 0x...>
>>> cris.user == cris_user
True
>>> [a.email for a in cris_user.addresses]
['cris@example.com']

A link to the user resource is now available as a sub-resource.

>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com')
display_name: Cris Person
email: cris@example.com
http_etag: "..."
original_email: cris@example.com
registered_on: 2005-08-01T07:49:23
self_link: http://localhost:9001/3.0/addresses/cris@example.com
user: http://localhost:9001/3.0/users/1

To prevent automatic user creation from taking place, add the auto_create parameter to the POST request and set it to False.

>>> dump_json('http://localhost:9001/3.0/addresses/anne@example.com/user',
...           {'display_name': 'Anne User', 'auto_create': False})
Traceback (most recent call last):
...
urllib.error.HTTPError: HTTP Error 403: ...

A request to the /user sub-resource will return the linked user’s representation:

>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com/user')
created_on: 2005-08-01T07:49:23
display_name: Cris X. Person
http_etag: "..."
is_server_owner: False
password: ...
self_link: http://localhost:9001/3.0/users/1
user_id: 1

The address and the user can be unlinked by sending a DELETE request on the /user resource. The user itself is not deleted, only the link.

>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com/user',
...           method='DELETE')
content-length: 0
date: ...
server: ...
status: 204
>>> transaction.abort()
>>> cris.user == None
True
>>> from uuid import UUID
>>> user_manager.get_user_by_id(UUID(int=1))
<User "Cris X. Person" (1) at 0x...>
>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com/user')
Traceback (most recent call last):
...
urllib.error.HTTPError: HTTP Error 404: ...

You can link an existing user to an address by passing the user’s ID in the POST request.

>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com/user',
...           {'user_id': 1})
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
server: ...
status: 200

>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com/user')
created_on: ...
display_name: Cris X. Person
http_etag: ...
password: ...
self_link: http://localhost:9001/3.0/users/1
user_id: 1

To link an address to a different user, you can either send a DELETE request followed by a POST request, or you can send a PUT request.

>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com/user',
...           {'display_name': 'Cris Q Person'}, method="PUT")
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/users/2
server: ...
status: 201

>>> dump_json('http://localhost:9001/3.0/addresses/cris@example.com/user')
created_on: ...
display_name: Cris Q Person
http_etag: ...
password: ...
self_link: http://localhost:9001/3.0/users/2
user_id: 2
User addresses

Users control addresses. The canonical URLs for these user-controlled addresses live in the /addresses namespace.

>>> dave = user_manager.create_user('dave@example.com', 'Dave Person')
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/users/dave@example.com/addresses')
entry 0:
    display_name: Dave Person
    email: dave@example.com
    http_etag: "..."
    original_email: dave@example.com
    registered_on: 2005-08-01T07:49:23
    self_link: http://localhost:9001/3.0/addresses/dave@example.com
    user: http://localhost:9001/3.0/users/3
http_etag: "..."
start: 0
total_size: 1

>>> dump_json('http://localhost:9001/3.0/addresses/dave@example.com')
display_name: Dave Person
email: dave@example.com
http_etag: "..."
original_email: dave@example.com
registered_on: 2005-08-01T07:49:23
self_link: http://localhost:9001/3.0/addresses/dave@example.com
user: http://localhost:9001/3.0/users/3

A user can be associated with multiple email addresses. You can add new addresses to an existing user.

>>> dump_json(
...     'http://localhost:9001/3.0/users/dave@example.com/addresses', {
...           'email': 'dave.person@example.org'
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/addresses/dave.person@example.org
server: ...
status: 201

When you add the new address, you can give it an optional display name.

>>> dump_json(
...     'http://localhost:9001/3.0/users/dave@example.com/addresses', {
...           'email': 'dp@example.org',
...           'display_name': 'Davie P',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/addresses/dp@example.org
server: ...
status: 201

The user controls these new addresses.

>>> dump_json('http://localhost:9001/3.0/users/dave@example.com/addresses')
entry 0:
    email: dave.person@example.org
    http_etag: "..."
    original_email: dave.person@example.org
    registered_on: 2005-08-01T07:49:23
    self_link: http://localhost:9001/3.0/addresses/dave.person@example.org
    user: http://localhost:9001/3.0/users/3
entry 1:
    display_name: Dave Person
    email: dave@example.com
    http_etag: "..."
    original_email: dave@example.com
    registered_on: 2005-08-01T07:49:23
    self_link: http://localhost:9001/3.0/addresses/dave@example.com
    user: http://localhost:9001/3.0/users/3
entry 2:
    display_name: Davie P
    email: dp@example.org
    http_etag: "..."
    original_email: dp@example.org
    registered_on: 2005-08-01T07:49:23
    self_link: http://localhost:9001/3.0/addresses/dp@example.org
    user: http://localhost:9001/3.0/users/3
http_etag: "..."
start: 0
total_size: 3
Memberships

Addresses can be subscribed to mailing lists. When they are, all the membership records for that address are easily accessible via the REST API.

Elle registers several email addresses.

>>> elle = user_manager.create_user('elle@example.com', 'Elle Person')
>>> subscriber = list(elle.addresses)[0]
>>> elle.register('eperson@example.com')
<Address: eperson@example.com [not verified] at ...>
>>> elle.register('elle.person@example.com')
<Address: elle.person@example.com [not verified] at ...>

Elle subscribes to two mailing lists with one of her addresses.

>>> ant = create_list('ant@example.com')
>>> bee = create_list('bee@example.com')
>>> ant.subscribe(subscriber)
<Member: Elle Person <elle@example.com> on ant@example.com
         as MemberRole.member>
>>> bee.subscribe(subscriber)
<Member: Elle Person <elle@example.com> on bee@example.com
         as MemberRole.member>
>>> transaction.commit()

Elle can get her memberships for each of her email addresses.

>>> dump_json('http://localhost:9001/3.0/addresses/'
...           'elle@example.com/memberships')
entry 0:
    address: http://localhost:9001/3.0/addresses/elle@example.com
    delivery_mode: regular
    email: elle@example.com
    http_etag: "..."
    list_id: ant.example.com
    member_id: 1
    role: member
    self_link: http://localhost:9001/3.0/members/1
    user: http://localhost:9001/3.0/users/4
entry 1:
    address: http://localhost:9001/3.0/addresses/elle@example.com
    delivery_mode: regular
    email: elle@example.com
    http_etag: "..."
    list_id: bee.example.com
    member_id: 2
    role: member
    self_link: http://localhost:9001/3.0/members/2
    user: http://localhost:9001/3.0/users/4
http_etag: "..."
start: 0
total_size: 2

>>> dump_json('http://localhost:9001/3.0/addresses/'
...           'eperson@example.com/memberships')
http_etag: "..."
start: 0
total_size: 0

When Elle subscribes to the bee list again with a different address, this does not show up in the list of memberships for his other address.

>>> subscriber = user_manager.get_address('eperson@example.com')
>>> bee.subscribe(subscriber)
<Member: eperson@example.com on bee@example.com as MemberRole.member>
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/addresses/'
...           'elle@example.com/memberships')
entry 0:
    address: http://localhost:9001/3.0/addresses/elle@example.com
    delivery_mode: regular
    email: elle@example.com
    http_etag: "..."
    list_id: ant.example.com
    member_id: 1
    role: member
    self_link: http://localhost:9001/3.0/members/1
    user: http://localhost:9001/3.0/users/4
entry 1:
    address: http://localhost:9001/3.0/addresses/elle@example.com
    delivery_mode: regular
    email: elle@example.com
    http_etag: "..."
    list_id: bee.example.com
    member_id: 2
    role: member
    self_link: http://localhost:9001/3.0/members/2
    user: http://localhost:9001/3.0/users/4
http_etag: "..."
start: 0
total_size: 2

>>> dump_json('http://localhost:9001/3.0/addresses/'
...           'eperson@example.com/memberships')
entry 0:
    address: http://localhost:9001/3.0/addresses/eperson@example.com
    delivery_mode: regular
    email: eperson@example.com
    http_etag: "..."
    list_id: bee.example.com
    member_id: 3
    role: member
    self_link: http://localhost:9001/3.0/members/3
    user: http://localhost:9001/3.0/users/4
http_etag: "..."
start: 0
total_size: 1
Deleting

Addresses can be deleted via the REST API.

>>> fred = user_manager.create_address('fred@example.com', 'Fred Person')
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/addresses/fred@example.com')
display_name: Fred Person
email: fred@example.com
http_etag: "..."
original_email: fred@example.com
registered_on: 2005-08-01T07:49:23
self_link: http://localhost:9001/3.0/addresses/fred@example.com

>>> dump_json('http://localhost:9001/3.0/addresses/fred@example.com',
...     method='DELETE')
content-length: 0
date: ...
server: ...
status: 204
>>> transaction.abort()

>>> print(user_manager.get_address('fred@example.com'))
None

If an address is linked to a user, deleting the address does not delete the user, it just unlinks it.

>>> gwen = user_manager.create_user('gwen@example.com', 'Gwen Person')
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/users/5/addresses')
entry 0:
    display_name: Gwen Person
    email: gwen@example.com
    http_etag: "..."
    original_email: gwen@example.com
    registered_on: 2005-08-01T07:49:23
    self_link: http://localhost:9001/3.0/addresses/gwen@example.com
    user: http://localhost:9001/3.0/users/5
http_etag: "795b0680c57ec2df3dceb68ccce2619fecdc7225"
start: 0
total_size: 1

>>> dump_json('http://localhost:9001/3.0/addresses/gwen@example.com',
...     method='DELETE')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/users/5/addresses')
http_etag: "..."
start: 0
total_size: 0
Users

The REST API can be used to add and remove users, add and remove user addresses, and change their preferred address, password, or name. The API can also be used to verify a user’s password.

Users are different than members; the latter represents an email address subscribed to a specific mailing list. Users are just people that Mailman knows about.

There are no users yet.

>>> dump_json('http://localhost:9001/3.0/users')
http_etag: "..."
start: 0
total_size: 0

Anne is added, with an email address. Her user record gets a user_id.

>>> from zope.component import getUtility
>>> from mailman.interfaces.usermanager import IUserManager
>>> user_manager = getUtility(IUserManager)
>>> anne = user_manager.create_user('anne@example.com', 'Anne Person')
>>> transaction.commit()
>>> int(anne.user_id.int)
1

Anne’s user record is returned as an entry into the collection of all users.

>>> dump_json('http://localhost:9001/3.0/users')
entry 0:
    created_on: 2005-08-01T07:49:23
    display_name: Anne Person
    http_etag: "..."
    is_server_owner: False
    self_link: http://localhost:9001/3.0/users/1
    user_id: 1
http_etag: "..."
start: 0
total_size: 1

A user might not have a display name, in which case, the attribute will not be returned in the REST API.

>>> bart = user_manager.create_user('bart@example.com')
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/users')
entry 0:
    created_on: 2005-08-01T07:49:23
    display_name: Anne Person
    http_etag: "..."
    is_server_owner: False
    self_link: http://localhost:9001/3.0/users/1
    user_id: 1
entry 1:
    created_on: 2005-08-01T07:49:23
    http_etag: "..."
    is_server_owner: False
    self_link: http://localhost:9001/3.0/users/2
    user_id: 2
http_etag: "..."
start: 0
total_size: 2
Paginating over user records

Instead of returning all the user records at once, it’s possible to return them in pages by adding the GET parameters count and page to the request URI. Page 1 is the first page and count defines the size of the page.

>>> dump_json('http://localhost:9001/3.0/users?count=1&page=1')
entry 0:
    created_on: 2005-08-01T07:49:23
    display_name: Anne Person
    http_etag: "..."
    is_server_owner: False
    self_link: http://localhost:9001/3.0/users/1
    user_id: 1
http_etag: "..."
start: 0
total_size: 2

>>> dump_json('http://localhost:9001/3.0/users?count=1&page=2')
entry 0:
    created_on: 2005-08-01T07:49:23
    http_etag: "..."
    is_server_owner: False
    self_link: http://localhost:9001/3.0/users/2
    user_id: 2
http_etag: "..."
start: 1
total_size: 2
Creating users

New users can be created by POSTing to the users collection. At a minimum, the user’s email address must be provided.

>>> dump_json('http://localhost:9001/3.0/users', {
...           'email': 'cris@example.com',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/users/3
server: ...
status: 201

Cris is now a user known to the system, but he has no display name.

>>> user_manager.get_user('cris@example.com')
<User "" (3) at ...>

Cris’s user record can also be accessed via the REST API, using her user id. Note that because no password was given when the record was created, a random one was assigned to her.

>>> dump_json('http://localhost:9001/3.0/users/3')
created_on: 2005-08-01T07:49:23
http_etag: "..."
is_server_owner: False
password: {plaintext}...
self_link: http://localhost:9001/3.0/users/3
user_id: 3

Because email addresses just have an @ sign in then, there’s no confusing them with user ids. Thus, Cris’s record can be retrieved via her email address.

>>> dump_json('http://localhost:9001/3.0/users/cris@example.com')
created_on: 2005-08-01T07:49:23
http_etag: "..."
is_server_owner: False
password: {plaintext}...
self_link: http://localhost:9001/3.0/users/3
user_id: 3
Providing a display name

When a user is added, a display name can be provided.

>>> transaction.abort()
>>> dump_json('http://localhost:9001/3.0/users', {
...           'email': 'dave@example.com',
...           'display_name': 'Dave Person',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/users/4
server: ...
status: 201

Dave’s user record includes his display name.

>>> dump_json('http://localhost:9001/3.0/users/4')
created_on: 2005-08-01T07:49:23
display_name: Dave Person
http_etag: "..."
is_server_owner: False
password: {plaintext}...
self_link: http://localhost:9001/3.0/users/4
user_id: 4
Providing passwords

To avoid getting assigned a random, and irretrievable password (but one which can be reset), you can provide a password when the user is created. By default, the password is provided in plain text, and it is hashed by Mailman before being stored.

>>> transaction.abort()
>>> dump_json('http://localhost:9001/3.0/users', {
...           'email': 'elly@example.com',
...           'display_name': 'Elly Person',
...           'password': 'supersekrit',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/users/5
server: ...
status: 201

When we view Elly’s user record, we can tell that her password has been hashed because it has the hash algorithm prefix (i.e. the {plaintext} marker).

>>> dump_json('http://localhost:9001/3.0/users/5')
created_on: 2005-08-01T07:49:23
display_name: Elly Person
http_etag: "..."
is_server_owner: False
password: {plaintext}supersekrit
self_link: http://localhost:9001/3.0/users/5
user_id: 5
Updating users

Dave’s display name can be changed through the REST API.

>>> dump_json('http://localhost:9001/3.0/users/4', {
...           'display_name': 'David Person',
...           }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

Dave’s display name has been updated.

>>> dump_json('http://localhost:9001/3.0/users/dave@example.com')
created_on: 2005-08-01T07:49:23
display_name: David Person
http_etag: "..."
is_server_owner: False
password: {plaintext}...
self_link: http://localhost:9001/3.0/users/4
user_id: 4

Dave can also be assigned a new password by providing in the new cleartext password. Mailman will hash this before it is stored internally.

>>> dump_json('http://localhost:9001/3.0/users/4', {
...           'cleartext_password': 'clockwork angels',
...           }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

As described above, even though you see {plaintext}clockwork angels below, it has still been hashed before storage. The default hashing algorithm for the test suite is a plain text hash, but you can see that it works by the addition of the algorithm prefix.

>>> dump_json('http://localhost:9001/3.0/users/4')
created_on: 2005-08-01T07:49:23
display_name: David Person
http_etag: "..."
is_server_owner: False
password: {plaintext}clockwork angels
self_link: http://localhost:9001/3.0/users/4
user_id: 4

You can change both the display name and the password by PUTing the full resource.

>>> dump_json('http://localhost:9001/3.0/users/4', {
...           'cleartext_password': 'the garden',
...           'display_name': 'David Personhood',
...           'is_server_owner': False,
...           }, method='PUT')
content-length: 0
date: ...
server: ...
status: 204

Dave’s user record has been updated.

>>> dump_json('http://localhost:9001/3.0/users/dave@example.com')
created_on: 2005-08-01T07:49:23
display_name: David Personhood
http_etag: "..."
is_server_owner: False
password: {plaintext}the garden
self_link: http://localhost:9001/3.0/users/4
user_id: 4
Deleting users via the API

Users can also be deleted via the API.

>>> dump_json('http://localhost:9001/3.0/users/cris@example.com',
...           method='DELETE')
content-length: 0
date: ...
server: ...
status: 204
User addresses

Fred may have any number of email addresses associated with his user account, and we can find them all through the API.

Through some other means, Fred registers a bunch of email addresses and associates them with his user account.

>>> fred = user_manager.create_user('fred@example.com', 'Fred Person')
>>> fred.register('fperson@example.com')
<Address: fperson@example.com [not verified] at ...>
>>> fred.register('fred.person@example.com')
<Address: fred.person@example.com [not verified] at ...>
>>> fred.register('Fred.Q.Person@example.com')
<Address: Fred.Q.Person@example.com [not verified]
          key: fred.q.person@example.com at ...>
>>> transaction.commit()

When we access Fred’s addresses via the REST API, they are sorted in lexical order by original (i.e. case-preserved) email address.

>>> dump_json('http://localhost:9001/3.0/users/fred@example.com/addresses')
entry 0:
    email: fred.q.person@example.com
    http_etag: "..."
    original_email: Fred.Q.Person@example.com
    registered_on: 2005-08-01T07:49:23
    self_link:
        http://localhost:9001/3.0/addresses/fred.q.person@example.com
    user: http://localhost:9001/3.0/users/6
entry 1:
    email: fperson@example.com
    http_etag: "..."
    original_email: fperson@example.com
    registered_on: 2005-08-01T07:49:23
    self_link: http://localhost:9001/3.0/addresses/fperson@example.com
    user: http://localhost:9001/3.0/users/6
entry 2:
    email: fred.person@example.com
    http_etag: "..."
    original_email: fred.person@example.com
    registered_on: 2005-08-01T07:49:23
    self_link: http://localhost:9001/3.0/addresses/fred.person@example.com
    user: http://localhost:9001/3.0/users/6
entry 3:
    display_name: Fred Person
    email: fred@example.com
    http_etag: "..."
    original_email: fred@example.com
    registered_on: 2005-08-01T07:49:23
    self_link: http://localhost:9001/3.0/addresses/fred@example.com
    user: http://localhost:9001/3.0/users/6
http_etag: "..."
start: 0
total_size: 4

In fact, since these are all associated with Fred’s user account, any of the addresses can be used to look up Fred’s user record.

>>> dump_json('http://localhost:9001/3.0/users/fred@example.com')
created_on: 2005-08-01T07:49:23
display_name: Fred Person
http_etag: "..."
is_server_owner: False
self_link: http://localhost:9001/3.0/users/6
user_id: 6

>>> dump_json('http://localhost:9001/3.0/users/fred.person@example.com')
created_on: 2005-08-01T07:49:23
display_name: Fred Person
http_etag: "..."
is_server_owner: False
self_link: http://localhost:9001/3.0/users/6
user_id: 6

>>> dump_json('http://localhost:9001/3.0/users/fperson@example.com')
created_on: 2005-08-01T07:49:23
display_name: Fred Person
http_etag: "..."
is_server_owner: False
self_link: http://localhost:9001/3.0/users/6
user_id: 6

>>> dump_json('http://localhost:9001/3.0/users/Fred.Q.Person@example.com')
created_on: 2005-08-01T07:49:23
display_name: Fred Person
http_etag: "..."
is_server_owner: False
self_link: http://localhost:9001/3.0/users/6
user_id: 6
Verifying passwords

A user’s password is stored internally in hashed form. Logging in a user is the process of verifying a provided clear text password against the hashed internal password.

When Elly was added as a user, she provided a password in the clear. Now the password is hashed and getting her user record returns the hashed password.

>>> dump_json('http://localhost:9001/3.0/users/5')
created_on: 2005-08-01T07:49:23
display_name: Elly Person
http_etag: "..."
is_server_owner: False
password: {plaintext}supersekrit
self_link: http://localhost:9001/3.0/users/5
user_id: 5

Unless the client can run the hashing algorithm on the login text that Elly provided, and do its own comparison, the client should let the REST API handle password verification.

This time, Elly successfully logs into Mailman.

>>> dump_json('http://localhost:9001/3.0/users/5/login', {
...           'cleartext_password': 'supersekrit',
...           }, method='POST')
content-length: 0
date: ...
server: ...
status: 204
Server owners

Users can be designated as server owners. Elly is not currently a server owner.

>>> dump_json('http://localhost:9001/3.0/users/5')
created_on: 2005-08-01T07:49:23
display_name: Elly Person
http_etag: "..."
is_server_owner: False
password: {plaintext}supersekrit
self_link: http://localhost:9001/3.0/users/5
user_id: 5

Let’s make her a server owner.

>>> dump_json('http://localhost:9001/3.0/users/5', {
...           'is_server_owner': True,
...           }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/users/5')
created_on: 2005-08-01T07:49:23
display_name: Elly Person
http_etag: "..."
is_server_owner: True
password: {plaintext}supersekrit
self_link: http://localhost:9001/3.0/users/5
user_id: 5

Elly later retires as server owner.

>>> dump_json('http://localhost:9001/3.0/users/5', {
...           'is_server_owner': False,
...           }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/users/5')
created_on: 2005-08-01T07:49:23
display_name: Elly Person
http_etag: "..."
is_server_owner: False
password: {plaintext}...
self_link: http://localhost:9001/3.0/users/5
user_id: 5

Gwen, a new users, takes over as a server owner.

>>> dump_json('http://localhost:9001/3.0/users', {
...           'display_name': 'Gwen Person',
...           'email': 'gwen@example.com',
...           'is_server_owner': True,
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/users/7
server: ...
status: 201

>>> dump_json('http://localhost:9001/3.0/users/7')
created_on: 2005-08-01T07:49:23
display_name: Gwen Person
http_etag: "..."
is_server_owner: True
password: {plaintext}...
self_link: http://localhost:9001/3.0/users/7
user_id: 7
Linking users

If an address already exists, but is not yet linked to a user, and a new user is requested for that address, the user will be linked to the existing address.

Herb’s address already exists, but no user is linked to it.

>>> herb = user_manager.create_address('herb@example.com')
>>> print(herb.user)
None
>>> transaction.commit()

Now, a user creation request is received, using Herb’s email address.

>>> dump_json('http://localhost:9001/3.0/users', {
...           'email': 'herb@example.com',
...           'display_name': 'Herb Person',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/users/8
server: ...
status: 201

Herb’s email address is now linked to the new user.

>>> herb.user
<User "Herb Person" (8) at ...
Membership

The REST API can be used to subscribe and unsubscribe users to mailing lists. A subscribed user is called a member. There is a top level collection that returns all the members of all known mailing lists.

There are no mailing lists and no members yet.

>>> dump_json('http://localhost:9001/3.0/members')
http_etag: "..."
start: 0
total_size: 0

We create a mailing list, which starts out with no members.

>>> bee = create_list('bee@example.com')
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/members')
http_etag: "..."
start: 0
total_size: 0
Subscribers

After Bart subscribes to the mailing list, his subscription is available via the REST interface.

>>> from mailman.interfaces.member import MemberRole
>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)

>>> from mailman.testing.helpers import subscribe
>>> subscribe(bee, 'Bart')
<Member: Bart Person <bperson@example.com> on bee@example.com
         as MemberRole.member>

>>> dump_json('http://localhost:9001/3.0/members')
entry 0:
    address: http://localhost:9001/3.0/addresses/bperson@example.com
    delivery_mode: regular
    email: bperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 1
    role: member
    self_link: http://localhost:9001/3.0/members/1
    user: http://localhost:9001/3.0/users/1
http_etag: "..."
start: 0
total_size: 1

Bart’s specific membership can be accessed directly:

>>> dump_json('http://localhost:9001/3.0/members/1')
address: http://localhost:9001/3.0/addresses/bperson@example.com
delivery_mode: regular
email: bperson@example.com
http_etag: ...
list_id: bee.example.com
member_id: 1
role: member
self_link: http://localhost:9001/3.0/members/1
user: http://localhost:9001/3.0/users/1

When Cris also joins the mailing list, her subscription is also available via the REST interface.

>>> subscribe(bee, 'Cris')
<Member: Cris Person <cperson@example.com> on bee@example.com
         as MemberRole.member>

>>> dump_json('http://localhost:9001/3.0/members')
entry 0:
    address: http://localhost:9001/3.0/addresses/bperson@example.com
    delivery_mode: regular
    email: bperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 1
    role: member
    self_link: http://localhost:9001/3.0/members/1
    user: http://localhost:9001/3.0/users/1
entry 1:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 2
    role: member
    self_link: http://localhost:9001/3.0/members/2
    user: http://localhost:9001/3.0/users/2
http_etag: "..."
start: 0
total_size: 2

The subscribed members are returned in alphabetical order, so when Anna subscribes, she is returned first.

>>> subscribe(bee, 'Anna')
<Member: Anna Person <aperson@example.com> on bee@example.com
         as MemberRole.member>

>>> dump_json('http://localhost:9001/3.0/members')
entry 0:
    address: http://localhost:9001/3.0/addresses/aperson@example.com
    delivery_mode: regular
    email: aperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 3
    role: member
    self_link: http://localhost:9001/3.0/members/3
    user: http://localhost:9001/3.0/users/3
entry 1:
    address: http://localhost:9001/3.0/addresses/bperson@example.com
    delivery_mode: regular
    email: bperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 1
    role: member
    self_link: http://localhost:9001/3.0/members/1
    user: http://localhost:9001/3.0/users/1
entry 2:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 2
    role: member
    self_link: http://localhost:9001/3.0/members/2
    user: http://localhost:9001/3.0/users/2
http_etag: "..."
start: 0
total_size: 3

Subscriptions are also returned alphabetically by mailing list posting address. Anna and Cris subscribe to this new mailing list.

>>> ant = create_list('ant@example.com')
>>> subscribe(ant, 'Anna')
<Member: Anna Person <aperson@example.com> on ant@example.com
         as MemberRole.member>
>>> subscribe(ant, 'Cris')
<Member: Cris Person <cperson@example.com> on ant@example.com
         as MemberRole.member>

User ids are different than member ids.

>>> dump_json('http://localhost:9001/3.0/members')
entry 0:
    address: http://localhost:9001/3.0/addresses/aperson@example.com
    delivery_mode: regular
    email: aperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 4
    role: member
    self_link: http://localhost:9001/3.0/members/4
    user: http://localhost:9001/3.0/users/3
entry 1:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 5
    role: member
    self_link: http://localhost:9001/3.0/members/5
    user: http://localhost:9001/3.0/users/2
entry 2:
    address: http://localhost:9001/3.0/addresses/aperson@example.com
    delivery_mode: regular
    email: aperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 3
    role: member
    self_link: http://localhost:9001/3.0/members/3
    user: http://localhost:9001/3.0/users/3
entry 3:
    address: http://localhost:9001/3.0/addresses/bperson@example.com
    delivery_mode: regular
    email: bperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 1
    role: member
    self_link: http://localhost:9001/3.0/members/1
    user: http://localhost:9001/3.0/users/1
entry 4:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 2
    role: member
    self_link: http://localhost:9001/3.0/members/2
    user: http://localhost:9001/3.0/users/2
http_etag: "..."
start: 0
total_size: 5

We can also get just the members of a single mailing list.

>>> dump_json(
...     'http://localhost:9001/3.0/lists/ant@example.com/roster/member')
entry 0:
    address: http://localhost:9001/3.0/addresses/aperson@example.com
    delivery_mode: regular
    email: aperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 4
    role: member
    self_link: http://localhost:9001/3.0/members/4
    user: http://localhost:9001/3.0/users/3
entry 1:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 5
    role: member
    self_link: http://localhost:9001/3.0/members/5
    user: http://localhost:9001/3.0/users/2
http_etag: ...
start: 0
total_size: 2
Paginating over member records

Instead of returning all the member records at once, it’s possible to return them in pages by adding the GET parameters count and page to the request URI. Page 1 is the first page and count defines the size of the page.

>>> dump_json(
...     'http://localhost:9001/3.0/lists/ant@example.com/roster/member'
...     '?count=1&page=1')
entry 0:
    address: http://localhost:9001/3.0/addresses/aperson@example.com
    delivery_mode: regular
    email: aperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 4
    role: member
    self_link: http://localhost:9001/3.0/members/4
    user: http://localhost:9001/3.0/users/3
http_etag: ...
start: 0
total_size: 2

This works with members of a single list as well as with all members.

>>> dump_json(
...     'http://localhost:9001/3.0/members?count=1&page=1')
entry 0:
    address: http://localhost:9001/3.0/addresses/aperson@example.com
    delivery_mode: regular
    email: aperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 4
    role: member
    self_link: http://localhost:9001/3.0/members/4
    user: http://localhost:9001/3.0/users/3
http_etag: ...
start: 0
total_size: 5
Owners and moderators

Mailing list owners and moderators also show up in the REST API. Cris becomes an owner of the ant mailing list and Dave becomes a moderator of the bee mailing list.

>>> dump_json('http://localhost:9001/3.0/members', {
...           'list_id': 'ant.example.com',
...           'subscriber': 'dperson@example.com',
...           'role': 'moderator',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/members/6
server: ...
status: 201

>>> dump_json('http://localhost:9001/3.0/members', {
...           'list_id': 'bee.example.com',
...           'subscriber': 'cperson@example.com',
...           'role': 'owner',
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/members/7
server: ...
status: 201

>>> dump_json('http://localhost:9001/3.0/members')
entry 0:
    address: http://localhost:9001/3.0/addresses/dperson@example.com
    delivery_mode: regular
    email: dperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 6
    moderation_action: accept
    role: moderator
    self_link: http://localhost:9001/3.0/members/6
    user: http://localhost:9001/3.0/users/4
entry 1:
    address: http://localhost:9001/3.0/addresses/aperson@example.com
    delivery_mode: regular
    email: aperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 4
    role: member
    self_link: http://localhost:9001/3.0/members/4
    user: http://localhost:9001/3.0/users/3
entry 2:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 5
    role: member
    self_link: http://localhost:9001/3.0/members/5
    user: http://localhost:9001/3.0/users/2
entry 3:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 7
    moderation_action: accept
    role: owner
    self_link: http://localhost:9001/3.0/members/7
    user: http://localhost:9001/3.0/users/2
entry 4:
    address: http://localhost:9001/3.0/addresses/aperson@example.com
    delivery_mode: regular
    email: aperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 3
    role: member
    self_link: http://localhost:9001/3.0/members/3
    user: http://localhost:9001/3.0/users/3
entry 5:
    address: http://localhost:9001/3.0/addresses/bperson@example.com
    delivery_mode: regular
    email: bperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 1
    role: member
    self_link: http://localhost:9001/3.0/members/1
    user: http://localhost:9001/3.0/users/1
entry 6:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 2
    role: member
    self_link: http://localhost:9001/3.0/members/2
    user: http://localhost:9001/3.0/users/2
http_etag: "..."
start: 0
total_size: 7

We can access all the owners of a list.

>>> dump_json(
...     'http://localhost:9001/3.0/lists/bee@example.com/roster/owner')
entry 0:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 7
    moderation_action: accept
    role: owner
    self_link: http://localhost:9001/3.0/members/7
    user: http://localhost:9001/3.0/users/2
http_etag: ...
start: 0
total_size: 1
Finding members

A specific member can always be referenced by their role and address.

>>> dump_json('http://localhost:9001/3.0/lists/'
...           'bee@example.com/owner/cperson@example.com')
address: http://localhost:9001/3.0/addresses/cperson@example.com
delivery_mode: regular
email: cperson@example.com
http_etag: ...
list_id: bee.example.com
member_id: 7
moderation_action: accept
role: owner
self_link: http://localhost:9001/3.0/members/7
user: http://localhost:9001/3.0/users/2

You can find a specific member based on several different criteria. For example, we can search for all the memberships of a particular address.

>>> dump_json('http://localhost:9001/3.0/members/find', {
...           'subscriber': 'aperson@example.com',
...           })
entry 0:
    address: http://localhost:9001/3.0/addresses/aperson@example.com
    delivery_mode: regular
    email: aperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 4
    role: member
    self_link: http://localhost:9001/3.0/members/4
    user: http://localhost:9001/3.0/users/3
entry 1:
    address: http://localhost:9001/3.0/addresses/aperson@example.com
    delivery_mode: regular
    email: aperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 3
    role: member
    self_link: http://localhost:9001/3.0/members/3
    user: http://localhost:9001/3.0/users/3
http_etag: ...
start: 0
total_size: 2

Or, we can find all the memberships for a particular mailing list.

>>> dump_json('http://localhost:9001/3.0/members/find', {
...           'list_id': 'bee.example.com',
...           })
entry 0:
    address: http://localhost:9001/3.0/addresses/aperson@example.com
    delivery_mode: regular
    email: aperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 3
    role: member
    self_link: http://localhost:9001/3.0/members/3
    user: http://localhost:9001/3.0/users/3
entry 1:
    address: http://localhost:9001/3.0/addresses/bperson@example.com
    delivery_mode: regular
    email: bperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 1
    role: member
    self_link: http://localhost:9001/3.0/members/1
    user: http://localhost:9001/3.0/users/1
entry 2:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 2
    role: member
    self_link: http://localhost:9001/3.0/members/2
    user: http://localhost:9001/3.0/users/2
entry 3:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 7
    moderation_action: accept
    role: owner
    self_link: http://localhost:9001/3.0/members/7
    user: http://localhost:9001/3.0/users/2
http_etag: "..."
start: 0
total_size: 4

Or, we can find all the memberships for an address on a particular mailing list.

>>> dump_json('http://localhost:9001/3.0/members/find', {
...           'subscriber': 'cperson@example.com',
...           'list_id': 'bee.example.com',
...           })
entry 0:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 2
    role: member
    self_link: http://localhost:9001/3.0/members/2
    user: http://localhost:9001/3.0/users/2
entry 1:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 7
    moderation_action: accept
    role: owner
    self_link: http://localhost:9001/3.0/members/7
    user: http://localhost:9001/3.0/users/2
http_etag: ...
start: 0
total_size: 2

Or, we can find all the memberships for an address with a specific role.

>>> dump_json('http://localhost:9001/3.0/members/find', {
...           'subscriber': 'cperson@example.com',
...           'role': 'member',
...           })
entry 0:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 5
    role: member
    self_link: http://localhost:9001/3.0/members/5
    user: http://localhost:9001/3.0/users/2
entry 1:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 2
    role: member
    self_link: http://localhost:9001/3.0/members/2
    user: http://localhost:9001/3.0/users/2
http_etag: ...
start: 0
total_size: 2

Finally, we can search for a specific member given all three criteria.

>>> dump_json('http://localhost:9001/3.0/members/find', {
...           'subscriber': 'cperson@example.com',
...           'list_id': 'bee.example.com',
...           'role': 'member',
...           })
entry 0:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 2
    role: member
    self_link: http://localhost:9001/3.0/members/2
    user: http://localhost:9001/3.0/users/2
http_etag: ...
start: 0
total_size: 1

Search can also be performed using HTTP GET queries.

>>> dump_json('http://localhost:9001/3.0/members/find'
...           '?subscriber=cperson@example.com'
...           '&list_id=bee.example.com'
...           '&role=member'
...           )
entry 0:
    address: http://localhost:9001/3.0/addresses/cperson@example.com
    delivery_mode: regular
    email: cperson@example.com
    http_etag: ...
    list_id: bee.example.com
    member_id: 2
    role: member
    self_link: http://localhost:9001/3.0/members/2
    user: http://localhost:9001/3.0/users/2
http_etag: ...
start: 0
total_size: 1
Joining a mailing list

A user can be subscribed to a mailing list via the REST API, either by a specific address, or more generally by their preferred address. A subscribed user is called a member.

Elly subscribes to the ant mailing list. Since her email address is not yet known to Mailman, a user is created for her. By default, she gets a regular delivery.

By pre-verifying her subscription, we don’t require Elly to verify that her email address is valid. By pre-confirming her subscription too, no confirmation email will be sent. Pre-approval means that the list moderator won’t have to approve her subscription request.

>>> dump_json('http://localhost:9001/3.0/members', {
...           'list_id': 'ant.example.com',
...           'subscriber': 'eperson@example.com',
...           'display_name': 'Elly Person',
...           'pre_verified': True,
...           'pre_confirmed': True,
...           'pre_approved': True,
...           })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/members/8
server: ...
status: 201

Elly is now a known user, and a member of the mailing list.

>>> elly = user_manager.get_user('eperson@example.com')
>>> elly
<User "Elly Person" (...) at ...>

>>> set(member.list_id for member in elly.memberships.members)
{'ant.example.com'}

>>> dump_json('http://localhost:9001/3.0/members')
entry 0:
...
entry 3:
    address: http://localhost:9001/3.0/addresses/eperson@example.com
    delivery_mode: regular
    email: eperson@example.com
    http_etag: ...
    list_id: ant.example.com
    member_id: 8
    role: member
    self_link: http://localhost:9001/3.0/members/8
    user: http://localhost:9001/3.0/users/5
...

Gwen is a user with a preferred address. She subscribes to the ant mailing list with her preferred address.

>>> from mailman.utilities.datetime import now
>>> gwen = user_manager.create_user('gwen@example.com', 'Gwen Person')
>>> preferred = list(gwen.addresses)[0]
>>> preferred.verified_on = now()
>>> gwen.preferred_address = preferred

# Note that we must extract the user id before we commit the transaction.
# This is because accessing the .user_id attribute will lock the database
# in the testing process, breaking the REST queue process.  Also, the
# user_id is a UUID internally, but an integer (represented as a string)
# is required by the REST API.
>>> user_id = gwen.user_id.int
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/members', {
...     'list_id': 'ant.example.com',
...     'subscriber': user_id,
...     'pre_verified': True,
...     'pre_confirmed': True,
...     'pre_approved': True,
...     })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/members/9
server: ...
status: 201

>>> dump_json('http://localhost:9001/3.0/members')
entry 0:
...
entry 4:
    address: http://localhost:9001/3.0/addresses/gwen@example.com
    delivery_mode: regular
    email: gwen@example.com
    http_etag: "..."
    list_id: ant.example.com
    member_id: 9
    role: member
    self_link: http://localhost:9001/3.0/members/9
    user: http://localhost:9001/3.0/users/6
...
total_size: 9

When Gwen changes her preferred address, her subscription automatically tracks the new address.

>>> new_preferred = gwen.register('gwen.person@example.com')
>>> new_preferred.verified_on = now()
>>> gwen.preferred_address = new_preferred
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/members')
entry 0:
...
entry 4:
    address: http://localhost:9001/3.0/addresses/gwen.person@example.com
    delivery_mode: regular
    email: gwen.person@example.com
    http_etag: "..."
    list_id: ant.example.com
    member_id: 9
    role: member
    self_link: http://localhost:9001/3.0/members/9
    user: http://localhost:9001/3.0/users/6
...
total_size: 9
Leaving a mailing list

Elly decides she does not want to be a member of the mailing list after all, so she leaves from the mailing list.

# Ensure our previous reads don't keep the database lock.
>>> transaction.abort()
>>> dump_json('http://localhost:9001/3.0/members/8',
...           method='DELETE')
content-length: 0
...
status: 204

Elly is no longer a member of the mailing list.

>>> set(member.mailing_list for member in elly.memberships.members)
set()
Changing delivery address

As shown above, Gwen is subscribed to a mailing list with her preferred email address. If she changes her preferred address, this automatically changes the address she will receive deliveries at for all such memberships.

However, when Herb subscribes to a couple of mailing lists with explicit addresses, he must change each subscription explicitly.

Herb controls multiple email addresses. All of these addresses are verified.

>>> herb = user_manager.create_user('herb@example.com', 'Herb Person')
>>> herb_1 = list(herb.addresses)[0]
>>> herb_2 = herb.register('hperson@example.com')
>>> herb_3 = herb.register('herb.person@example.com')
>>> for address in herb.addresses:
...     address.verified_on = now()

Herb subscribes to both the ant and bee mailing lists with one of his addresses.

>>> ant.subscribe(herb_1)
<Member: Herb Person <herb@example.com> on
         ant@example.com as MemberRole.member>
>>> bee.subscribe(herb_1)
<Member: Herb Person <herb@example.com> on
         bee@example.com as MemberRole.member>
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/members')
entry 0:
...
entry 4:
    address: http://localhost:9001/3.0/addresses/herb@example.com
    delivery_mode: regular
    email: herb@example.com
    http_etag: "..."
    list_id: ant.example.com
    member_id: 10
    role: member
    self_link: http://localhost:9001/3.0/members/10
    user: http://localhost:9001/3.0/users/7
...
entry 9:
    address: http://localhost:9001/3.0/addresses/herb@example.com
    delivery_mode: regular
    email: herb@example.com
    http_etag: "..."
    list_id: bee.example.com
    member_id: 11
    role: member
    self_link: http://localhost:9001/3.0/members/11
    user: http://localhost:9001/3.0/users/7
http_etag: "..."
start: 0
total_size: 10

In order to change all of his subscriptions to use a different email address, Herb must iterate through his memberships explicitly.

>>> from mailman.testing.helpers import call_api
>>> content, response = call_api('http://localhost:9001/3.0/addresses/'
...                              'herb@example.com/memberships')
>>> memberships = [entry['self_link'] for entry in content['entries']]
>>> for url in sorted(memberships):
...     print(url)
http://localhost:9001/3.0/members/10
http://localhost:9001/3.0/members/11

For each membership resource, the subscription address is changed by PATCH’ing the address attribute.

>>> dump_json('http://localhost:9001/3.0/members/10', {
...           'address': 'hperson@example.com',
...           }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/members/11', {
...           'address': 'hperson@example.com',
...           }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

Herb’s memberships with the old address are gone.

>>> dump_json('http://localhost:9001/3.0/addresses/'
...           'herb@example.com/memberships')
http_etag: "..."
start: 0
total_size: 0

Herb’s memberships have been updated with his new email address. Of course, his membership ids have not changed.

>>> dump_json('http://localhost:9001/3.0/addresses/'
...           'hperson@example.com/memberships')
entry 0:
    address: http://localhost:9001/3.0/addresses/hperson@example.com
    delivery_mode: regular
    email: hperson@example.com
    http_etag: "..."
    list_id: ant.example.com
    member_id: 10
    role: member
    self_link: http://localhost:9001/3.0/members/10
    user: http://localhost:9001/3.0/users/7
entry 1:
    address: http://localhost:9001/3.0/addresses/hperson@example.com
    delivery_mode: regular
    email: hperson@example.com
    http_etag: "..."
    list_id: bee.example.com
    member_id: 11
    role: member
    self_link: http://localhost:9001/3.0/members/11
    user: http://localhost:9001/3.0/users/7
http_etag: "..."
start: 0
total_size: 2

When changing his subscription address, Herb may also decide to change his mode of delivery.

>>> dump_json('http://localhost:9001/3.0/members/11', {
...           'address': 'herb@example.com',
...           'delivery_mode': 'mime_digests',
...           }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/addresses/'
...           'herb@example.com/memberships')
entry 0:
    address: http://localhost:9001/3.0/addresses/herb@example.com
    delivery_mode: mime_digests
    email: herb@example.com
    http_etag: "..."
    list_id: bee.example.com
    member_id: 11
    role: member
    self_link: http://localhost:9001/3.0/members/11
    user: http://localhost:9001/3.0/users/7
http_etag: "..."
start: 0
total_size: 1
Moderating a member

The moderation action for a member can be changed by PATCH’ing the moderation_action attribute. When the member action falls back to the list default, there is no such attribute in the resource.

>>> dump_json('http://localhost:9001/3.0/members/10')
address: http://localhost:9001/3.0/addresses/hperson@example.com
delivery_mode: regular
email: hperson@example.com
http_etag: "..."
list_id: ant.example.com
member_id: 10
role: member
self_link: http://localhost:9001/3.0/members/10
user: http://localhost:9001/3.0/users/7

Patching the moderation action both changes it for the given user, and adds the attribute to the member’s resource.

>>> dump_json('http://localhost:9001/3.0/members/10', {
...           'moderation_action': 'hold',
...           }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/members/10')
address: http://localhost:9001/3.0/addresses/hperson@example.com
...
moderation_action: hold
...

It can be reset to the list default by patching an empty value.

>>> dump_json('http://localhost:9001/3.0/members/10', {
...           'moderation_action': '',
...           }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/members/10')
address: http://localhost:9001/3.0/addresses/hperson@example.com
delivery_mode: regular
email: hperson@example.com
http_etag: "..."
list_id: ant.example.com
member_id: 10
role: member
self_link: http://localhost:9001/3.0/members/10
user: http://localhost:9001/3.0/users/7
Handling the list of banned addresses

To ban an address from subscribing you can POST to the /bans child of any list using the REST API.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com/bans',
...           {'email': 'banned@example.com'})
content-length: 0
...
location: .../3.0/lists/ant.example.com/bans/banned@example.com
...
status: 201

This address is now banned, and you can get the list of banned addresses by issuing a GET request on the /bans child.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com/bans')
entry 0:
    email: banned@example.com
    http_etag: "..."
    list_id: ant.example.com
    self_link: .../3.0/lists/ant.example.com/bans/banned@example.com
...

You can always GET a single banned address.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/bans/banned@example.com')
email: banned@example.com
http_etag: "..."
list_id: ant.example.com
self_link: .../3.0/lists/ant.example.com/bans/banned@example.com

Unbanning addresses is also possible by issuing a DELETE request.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com'
...           '/bans/banned@example.com',
...           method='DELETE')
content-length: 0
...
status: 204

After unbanning, the address is not shown in the ban list anymore.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com/bans')
http_etag: "..."
start: 0
total_size: 0

Global bans prevent an address from subscribing to any mailing list, and they can be added via the top-level bans resource.

>>> dump_json('http://localhost:9001/3.0/bans',
...           {'email': 'banned@example.com'})
content-length: 0
...
location: http://localhost:9001/3.0/bans/banned@example.com
...
status: 201

Note that entries in the global bans do not have a list_id field.

>>> dump_json('http://localhost:9001/3.0/bans')
entry 0:
    email: banned@example.com
    http_etag: "..."
    self_link: http://localhost:9001/3.0/bans/banned@example.com
...

>>> dump_json('http://localhost:9001/3.0/bans/banned@example.com')
email: banned@example.com
http_etag: "..."
self_link: http://localhost:9001/3.0/bans/banned@example.com

As with list-centric bans, you can delete a global ban.

>>> dump_json('http://localhost:9001/3.0/bans/banned@example.com',
...           method='DELETE')
content-length: 0
...
status: 204

>>> dump_json('http://localhost:9001/3.0/bans/banned@example.com')
Traceback (most recent call last):
...
urllib.error.HTTPError: HTTP Error 404: ...
>>> dump_json('http://localhost:9001/3.0/bans')
http_etag: "..."
start: 0
total_size: 0
Mass Unsubscriptions

A batch of users can be unsubscribed from the mailing list via the REST API just by supplying their email addresses.

>>> cat = create_list('cat@example.com')
>>> subscribe(cat, 'Isla')
<Member: Isla Person <iperson@example.com> on
         cat@example.com as MemberRole.member>
>>> subscribe(cat, 'John')
<Member: John Person <jperson@example.com> on
         cat@example.com as MemberRole.member>
>>> subscribe(cat, 'Kate')
<Member: Kate Person <kperson@example.com> on
         cat@example.com as MemberRole.member>

There are three new members of the mailing list. We try to mass delete them, plus one other address that isn’t a member of the list. We get back a dictionary mapping email addresses to the success or failure of the removal operation. It’s okay that one of the addresses is removed twice.

>>> dump_json(
...     'http://localhost:9001/3.0/lists/cat.example.com/roster/member', {
...     'emails': ['iperson@example.com',
...                'jperson@example.com',
...                'iperson@example.com',
...                'zperson@example.com',
...                ]},
...     'DELETE')
http_etag: "..."
iperson@example.com: True
jperson@example.com: True
zperson@example.com: False

And now only Kate is still a member.

>>> dump_json(
...     'http://localhost:9001/3.0/lists/cat.example.com/roster/member')
entry 0:
    address: http://localhost:9001/3.0/addresses/kperson@example.com
    delivery_mode: regular
    email: kperson@example.com
    http_etag: "..."
    list_id: cat.example.com
    member_id: 14
    role: member
    self_link: http://localhost:9001/3.0/members/14
    user: http://localhost:9001/3.0/users/10
...
total_size: 1
Queues

You can get information about what messages are currently in the Mailman queues by querying the top-level queues resource. Of course, this information may be out-of-date by the time you receive a response, since queue management is asynchronous, but the information will be as current as possible.

You can get the list of all queue names.

>>> dump_json('http://localhost:9001/3.0/queues')
entry 0:
    count: 0
    directory: .../queue/archive
    files: []
    http_etag: ...
    name: archive
    self_link: http://localhost:9001/3.0/queues/archive
entry 1:
    count: 0
    directory: .../queue/bad
    files: []
    http_etag: ...
    name: bad
    self_link: http://localhost:9001/3.0/queues/bad
entry 2:
    count: 0
    directory: .../queue/bounces
    files: []
    http_etag: ...
    name: bounces
    self_link: http://localhost:9001/3.0/queues/bounces
entry 3:
    count: 0
    directory: .../queue/command
    files: []
    http_etag: ...
    name: command
    self_link: http://localhost:9001/3.0/queues/command
entry 4:
    count: 0
    directory: .../queue/digest
    files: []
    http_etag: ...
    name: digest
    self_link: http://localhost:9001/3.0/queues/digest
entry 5:
    count: 0
    directory: .../queue/in
    files: []
    http_etag: ...
    name: in
    self_link: http://localhost:9001/3.0/queues/in
entry 6:
    count: 0
    directory: .../queue/nntp
    files: []
    http_etag: ...
    name: nntp
    self_link: http://localhost:9001/3.0/queues/nntp
entry 7:
    count: 0
    directory: .../queue/out
    files: []
    http_etag: ...
    name: out
    self_link: http://localhost:9001/3.0/queues/out
entry 8:
    count: 0
    directory: .../queue/pipeline
    files: []
    http_etag: ...
    name: pipeline
    self_link: http://localhost:9001/3.0/queues/pipeline
entry 9:
    count: 0
    directory: .../queue/retry
    files: []
    http_etag: ...
    name: retry
    self_link: http://localhost:9001/3.0/queues/retry
entry 10:
    count: 0
    directory: .../queue/shunt
    files: []
    http_etag: ...
    name: shunt
    self_link: http://localhost:9001/3.0/queues/shunt
entry 11:
    count: 0
    directory: .../queue/virgin
    files: []
    http_etag: ...
    name: virgin
    self_link: http://localhost:9001/3.0/queues/virgin
http_etag: ...
self_link: http://localhost:9001/3.0/queues
start: 0
total_size: 12

Query an individual queue to get a count of, and the list of file base names in the queue. There are currently no files in the bad queue.

>>> dump_json('http://localhost:9001/3.0/queues/bad')
count: 0
directory: .../queue/bad
files: []
http_etag: ...
name: bad
self_link: http://localhost:9001/3.0/queues/bad

We can inject a message into the bad queue. It must be destined for an existing mailing list.

>>> dump_json('http://localhost:9001/3.0/lists', {
...     'fqdn_listname': 'ant@example.com',
...     })
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/lists/ant.example.com
server: WSGIServer/0.2 CPython/...
status: 201

While list creation takes an FQDN list name, injecting a message to the queue requires a List ID.

>>> dump_json('http://localhost:9001/3.0/queues/bad', {
...     'list_id': 'ant.example.com',
...     'text': """\
... From: anne@example.com
... To: ant@example.com
... Subject: Testing
...
... """})
content-length: 0
content-type: application/json; charset=UTF-8
date: ...
location: http://localhost:9001/3.0/queues/bad/...
server: ...
status: 201

And now the bad queue has at least one message in it.

>>> dump_json('http://localhost:9001/3.0/queues/bad')
count: 1
directory: .../queue/bad
files: ['...']
http_etag: ...
name: bad
self_link: http://localhost:9001/3.0/queues/bad

We can delete the injected message.

>>> json = call_http('http://localhost:9001/3.0/queues/bad')
>>> len(json['files'])
1
>>> dump_json('http://localhost:9001/3.0/queues/bad/{}'.format(
...           json['files'][0]),
...           method='DELETE')
content-length: 0
date: ...
server: ...
status: 204

And now the queue has no files.

>>> dump_json('http://localhost:9001/3.0/queues/bad')
count: 0
directory: .../queue/bad
files: []
http_etag: ...
name: bad
self_link: http://localhost:9001/3.0/queues/bad
Server owners

Certain users can be designated as server owners. This role has no direct function in the core, but it can be used by clients of the REST API to determine additional permissions. For example, Postorius might allow server owners to create new domains.

Initially, there are no server owners.

>>> dump_json('http://localhost:9001/3.0/owners')
http_etag: "..."
start: 0
total_size: 0

When new users are created in the core, they do not become server owners by default.

>>> from zope.component import getUtility
>>> from mailman.interfaces.usermanager import IUserManager
>>> user_manager = getUtility(IUserManager)
>>> anne = user_manager.create_user('anne@example.com', 'Anne Person')
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/owners')
http_etag: "..."
start: 0
total_size: 0

Anne’s server owner flag is set.

>>> anne.is_server_owner = True
>>> transaction.commit()

And now we can find her user record.

>>> dump_json('http://localhost:9001/3.0/owners')
entry 0:
    created_on: 2005-08-01T07:49:23
    display_name: Anne Person
    http_etag: "..."
    is_server_owner: True
    self_link: http://localhost:9001/3.0/users/1
    user_id: 1
http_etag: "..."
start: 0
total_size: 1

Bart and Cate are also users, but not server owners.

>>> bart = user_manager.create_user('bart@example.com', 'Bart Person')
>>> cate = user_manager.create_user('cate@example.com', 'Cate Person')
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/owners')
entry 0:
    created_on: 2005-08-01T07:49:23
    display_name: Anne Person
    http_etag: "..."
    is_server_owner: True
    self_link: http://localhost:9001/3.0/users/1
    user_id: 1
http_etag: "..."
start: 0
total_size: 1

Anne retires as a server owner, with Bart and Cate taking over.

>>> anne.is_server_owner = False
>>> bart.is_server_owner = True
>>> cate.is_server_owner = True
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/owners')
entry 0:
    created_on: 2005-08-01T07:49:23
    display_name: Bart Person
    http_etag: "..."
    is_server_owner: True
    self_link: http://localhost:9001/3.0/users/2
    user_id: 2
entry 1:
    created_on: 2005-08-01T07:49:23
    display_name: Cate Person
    http_etag: "..."
    is_server_owner: True
    self_link: http://localhost:9001/3.0/users/3
    user_id: 3
http_etag: "..."
start: 0
total_size: 2
Post Moderation

Messages which are held for approval can be accepted, rejected, discarded, or deferred by the list moderators.

Viewing the list of held messages

Held messages can be moderated through the REST API. A mailing list starts with no held messages.

>>> ant = create_list('ant@example.com')
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/lists/ant@example.com/held')
http_etag: "..."
start: 0
total_size: 0

When a message gets held for moderator approval, it shows up in this list.

>>> msg = message_from_string("""\
... From: anne@example.com
... To: ant@example.com
... Subject: Something
... Message-ID: <alpha>
...
... Something else.
... """)

>>> from mailman.app.moderator import hold_message
>>> request_id = hold_message(ant, msg, {'extra': 7}, 'Because')
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/lists/ant@example.com/held')
entry 0:
    extra: 7
    hold_date: 2005-08-01T07:49:23
    http_etag: "..."
    message_id: <alpha>
    msg: From: anne@example.com
To: ant@example.com
Subject: Something
Message-ID: <alpha>
Message-ID-Hash: XZ3DGG4V37BZTTLXNUX4NABB4DNQHTCP
X-Message-ID-Hash: XZ3DGG4V37BZTTLXNUX4NABB4DNQHTCP

Something else.

    original_subject: Something
    reason: Because
    request_id: 1
    self_link: http://localhost:9001/3.0/lists/ant.example.com/held/1
    sender: anne@example.com
    subject: Something
http_etag: "..."
start: 0
total_size: 1

You can get an individual held message by providing the request id for that message. This will include the text of the message.

>>> def url(request_id):
...     return ('http://localhost:9001/3.0/lists/'
...             'ant@example.com/held/{0}'.format(request_id))

>>> dump_json(url(request_id))
extra: 7
hold_date: 2005-08-01T07:49:23
http_etag: "..."
message_id: <alpha>
msg: From: anne@example.com
To: ant@example.com
Subject: Something
Message-ID: <alpha>
Message-ID-Hash: XZ3DGG4V37BZTTLXNUX4NABB4DNQHTCP
X-Message-ID-Hash: XZ3DGG4V37BZTTLXNUX4NABB4DNQHTCP

Something else.

original_subject: Something
reason: Because
request_id: 1
self_link: http://localhost:9001/3.0/lists/ant.example.com/held/1
sender: anne@example.com
subject: Something
Disposing of held messages

Individual messages can be moderated through the API by POSTing back to the held message’s resource. The POST data requires an action of one of the following:

  • discard - throw the message away.
  • reject - bounces the message back to the original author.
  • defer - defer any action on the message (continue to hold it)
  • accept - accept the message for posting.

Let’s see what happens when the above message is deferred.

>>> dump_json(url(request_id), {
...     'action': 'defer',
...     })
content-length: 0
date: ...
server: ...
status: 204

The message is still in the moderation queue.

>>> dump_json(url(request_id))
extra: 7
hold_date: 2005-08-01T07:49:23
http_etag: "..."
message_id: <alpha>
msg: From: anne@example.com
To: ant@example.com
Subject: Something
Message-ID: <alpha>
Message-ID-Hash: XZ3DGG4V37BZTTLXNUX4NABB4DNQHTCP
X-Message-ID-Hash: XZ3DGG4V37BZTTLXNUX4NABB4DNQHTCP
<BLANKLINE>
Something else.
<BLANKLINE>
original_subject: Something
reason: Because
request_id: 1
self_link: http://localhost:9001/3.0/lists/ant.example.com/held/1
sender: anne@example.com
subject: Something

The held message can be discarded.

>>> dump_json(url(request_id), {
...     'action': 'discard',
...     })
content-length: 0
date: ...
server: ...
status: 204

Messages can also be accepted via the REST API. Let’s hold a new message for moderation.

>>> del msg['message-id']
>>> msg['Message-ID'] = '<bravo>'
>>> request_id = hold_message(ant, msg)
>>> transaction.commit()

>>> results = call_http(url(request_id))
>>> print(results['message_id'])
<bravo>

>>> dump_json(url(request_id), {
...     'action': 'accept',
...     })
content-length: 0
date: ...
server: ...
status: 204

>>> from mailman.testing.helpers import get_queue_messages
>>> messages = get_queue_messages('pipeline')
>>> len(messages)
1
>>> print(messages[0].msg['message-id'])
<bravo>

Messages can be rejected via the REST API too. These bounce the message back to the original author.

>>> del msg['message-id']
>>> msg['Message-ID'] = '<charlie>'
>>> request_id = hold_message(ant, msg)
>>> transaction.commit()

>>> results = call_http(url(request_id))
>>> print(results['message_id'])
<charlie>

>>> dump_json(url(request_id), {
...     'action': 'reject',
...     })
content-length: 0
date: ...
server: ...
status: 204

>>> from mailman.testing.helpers import get_queue_messages
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg['subject'])
Request to mailing list "Ant" rejected

The subject of the message is decoded and the original subject is accessible under original_subject.

>>> msg = message_from_string("""\
... From: anne@example.com
... To: ant@example.com
... Subject: =?iso-8859-1?q?p=F6stal?=
... Message-ID: <beta>
...
... Something else.
... """)

>>> from mailman.app.moderator import hold_message
>>> request_id = hold_message(ant, msg, {'extra': 7}, 'Because')
>>> transaction.commit()

>>> results = call_http(url(request_id))
>>> print(results['subject'])
pöstal
>>> print(results['original_subject'])
=?iso-8859-1?q?p=F6stal?=
Preferences

Addresses have preferences.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)

>>> anne = user_manager.create_address('anne@example.com')
>>> transaction.commit()

Although to start with, an address has no preferences.

>>> dump_json('http://localhost:9001/3.0/addresses/anne@example.com'
...           '/preferences')
http_etag: "..."
self_link: http://localhost:9001/3.0/addresses/anne@example.com/preferences

Once the address is given some preferences, they are available through the REST API.

>>> anne.preferences.acknowledge_posts = True
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/addresses/anne@example.com'
...           '/preferences')
acknowledge_posts: True
http_etag: "..."
self_link: http://localhost:9001/3.0/addresses/anne@example.com/preferences

Similarly, users have their own set of preferences, which also start out empty.

>>> bart = user_manager.create_user('bart@example.com')
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/addresses/bart@example.com'
...           '/preferences')
http_etag: "..."
self_link: http://localhost:9001/3.0/addresses/bart@example.com/preferences

Setting a preference on the user’s address does not set them on the user.

>>> list(bart.addresses)[0].preferences.acknowledge_posts = True
>>> transaction.commit()

>>> dump_json('http://localhost:9001/3.0/addresses/bart@example.com'
...           '/preferences')
acknowledge_posts: True
http_etag: "..."
self_link: http://localhost:9001/3.0/addresses/bart@example.com/preferences

>>> dump_json('http://localhost:9001/3.0/users/1/preferences')
http_etag: "..."
self_link: http://localhost:9001/3.0/users/1/preferences

Users have their own set of preferences.

>>> bart.preferences.receive_own_postings = False
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/users/1/preferences')
http_etag: "..."
receive_own_postings: False
self_link: http://localhost:9001/3.0/users/1/preferences

Similarly, members have their own separate set of preferences, and just like the above, setting a preference on the member’s address or user does not set the preference on the member.

>>> from mailman.interfaces.member import MemberRole
>>> mlist = create_list('test@example.com')
>>> bart_member = mlist.subscribe(list(bart.addresses)[0])
>>> bart_member.preferences.receive_list_copy = False
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/members/1/preferences')
http_etag: "..."
receive_list_copy: False
self_link: http://localhost:9001/3.0/members/1/preferences
Changing preferences

Preferences for the address, user, or member can be changed through the API. You can change all the preferences for a particular object by using an HTTP PUT operation.

>>> dump_json('http://localhost:9001/3.0/addresses/bart@example.com'
...           '/preferences', {
...           'acknowledge_posts': True,
...           'delivery_mode': 'plaintext_digests',
...           'delivery_status': 'by_user',
...           'hide_address': False,
...           'preferred_language': 'ja',
...           'receive_list_copy': True,
...           'receive_own_postings': False,
...           }, method='PUT')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/addresses/bart@example.com'
...           '/preferences')
acknowledge_posts: True
delivery_mode: plaintext_digests
delivery_status: by_user
hide_address: False
http_etag: "..."
preferred_language: ja
receive_list_copy: True
receive_own_postings: False
self_link: http://localhost:9001/3.0/addresses/bart@example.com/preferences

You can also update just a few of the attributes using PATCH.

>>> dump_json('http://localhost:9001/3.0/addresses/bart@example.com'
...           '/preferences', {
...           'delivery_mode': 'plaintext_digests',
...           'receive_list_copy': False,
...           }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/addresses/bart@example.com'
...           '/preferences')
acknowledge_posts: True
delivery_mode: plaintext_digests
delivery_status: by_user
hide_address: False
http_etag: "..."
preferred_language: ja
receive_list_copy: False
receive_own_postings: False
self_link: http://localhost:9001/3.0/addresses/bart@example.com/preferences
Deleting preferences

Preferences for any of the levels, member, user, or address, can be entirely deleted.

>>> dump_json('http://localhost:9001/3.0/addresses/anne@example.com'
...           '/preferences', {
...           'preferred_language': 'ja',
...           }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/addresses/anne@example.com'
...           '/preferences')
acknowledge_posts: True
http_etag: "..."
preferred_language: ja
self_link: http://localhost:9001/3.0/addresses/anne@example.com/preferences

>>> dump_json('http://localhost:9001/3.0/addresses/anne@example.com'
...           '/preferences', method='DELETE')
content-length: 0
date: ...
server: ...
status: 204

>>> dump_json('http://localhost:9001/3.0/addresses/anne@example.com'
...           '/preferences')
http_etag: "..."
self_link: http://localhost:9001/3.0/addresses/anne@example.com/preferences
Combined member preferences

The member resource provides a way to access the set of preference in effect for a specific subscription. This stacks the preferences, so that a value is always available. The preference value is looked up first on the member, falling back to the address, then user, then system preference.

Preferences accessed through this interface are always read only.

>>> dump_json('http://localhost:9001/3.0/members/1/all/preferences')
acknowledge_posts: True
delivery_mode: plaintext_digests
delivery_status: by_user
http_etag: "..."
preferred_language: ja
receive_list_copy: False
receive_own_postings: False
self_link: http://localhost:9001/3.0/members/1/all/preferences
System preferences

The Mailman system itself has a default set of preference. All preference look ups fall back to these values, which are read-only.

>>> dump_json('http://localhost:9001/3.0/system/preferences')
acknowledge_posts: False
delivery_mode: regular
delivery_status: enabled
hide_address: True
http_etag: "..."
preferred_language: en
receive_list_copy: True
receive_own_postings: True
self_link: http://localhost:9001/3.0/system/preferences
Subscription moderation

Subscription (and sometimes unsubscription) requests can similarly be accepted, discarded, rejected, or deferred by the list moderators.

Viewing subscription requests

A mailing list starts with no pending subscription or unsubscription requests.

>>> ant = create_list('ant@example.com')
>>> ant.admin_immed_notify = False
>>> from mailman.interfaces.mailinglist import SubscriptionPolicy
>>> ant.subscription_policy = SubscriptionPolicy.moderate
>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com/requests')
http_etag: "..."
start: 0
total_size: 0

When Anne tries to subscribe to the Ant list, her subscription is held for moderator approval. Her email address is pre-verified and her subscription request is pre-confirmed, but because the mailing list is moderated, a token is returned to track her subscription request.

>>> dump_json('http://localhost:9001/3.0/members', {
...           'list_id': 'ant.example.com',
...           'subscriber': 'anne@example.com',
...           'display_name': 'Anne Person',
...           'pre_verified': True,
...           'pre_confirmed': True,
...           })
http_etag: ...
token: 0000000000000000000000000000000000000001
token_owner: moderator

The subscription request can be viewed in the REST API.

>>> transaction.commit()
>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com/requests')
entry 0:
    display_name: Anne Person
    email: anne@example.com
    http_etag: "..."
    list_id: ant.example.com
    token: 0000000000000000000000000000000000000001
    token_owner: moderator
    type: subscription
    when: 2005-08-01T07:49:23
http_etag: "..."
start: 0
total_size: 1
Viewing individual requests

You can view an individual membership change request by providing the token (a.k.a. request id). Anne’s subscription request looks like this.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com/'
...           'requests/0000000000000000000000000000000000000001')
display_name: Anne Person
email: anne@example.com
http_etag: "..."
list_id: ant.example.com
token: 0000000000000000000000000000000000000001
token_owner: moderator
type: subscription
when: 2005-08-01T07:49:23
Disposing of subscription requests

Moderators can dispose of held subscription requests by POSTing back to the request’s resource. The POST data requires an action of one of the following:

  • discard - throw the request away.
  • reject - the request is denied and a notification is sent to the email
    address requesting the membership change.
  • defer - defer any action on this membership change (continue to hold it).
  • accept - accept the membership change.

Anne’s subscription request is accepted.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com/requests'
...           '/0000000000000000000000000000000000000001',
...           {'action': 'accept'})
content-length: 0
date: ...
server: ...
status: 204

Anne is now a member of the mailing list.

>>> ant.members.get_member('anne@example.com')
<Member: Anne Person <anne@example.com> on ant@example.com
         as MemberRole.member>

There are no more membership change requests.

>>> dump_json('http://localhost:9001/3.0/lists/ant.example.com/requests')
http_etag: "..."
start: 0
total_size: 0
Templates

In Mailman 3.1 a new template system was introduced to allow for maximum flexibility in the format and content of messages sent by and through Mailman. For example, when a new member joins a list, a welcome message is sent to that member. The welcome message is created from a template found by a URL associated with a template name and a context.

So if for example, you want to include links to pages on you website, you can create a custom template, make it available via download from a URL, and then associate that URL with a mailing list’s welcome message. Some standard placeholders can be defined in the template, and these will be filled in by Mailman when the welcome message is sent.

The URL itself can have placeholders, and this allows for additional flexibility when looking up the content.

Examples

Let’s say you have a mailing list:

>>> ant = create_list('ant@example.com')

The standard welcome message doesn’t have any links to it because by default Mailman doesn’t know about any web user interface front-end. When Anne is subscribed to the mailing list, she sees this plain welcome message.

>>> anne = subscribe(ant, 'Anne')
>>> items = get_queue_messages('virgin')
>>> print(items[0].msg)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Welcome to the "Ant" mailing list
From: ant-request@example.com
To: Anne Person <aperson@example.com>
...
<BLANKLINE>
Welcome to the "Ant" mailing list!
<BLANKLINE>
To post to this list, send your email to:
<BLANKLINE>
  ant@example.com
<BLANKLINE>
You can make such adjustments via email by sending a message to:
<BLANKLINE>
  ant-request@example.com
<BLANKLINE>
with the word 'help' in the subject or body (don't include the
quotes), and you will get back a message with instructions.  You will
need your password to change your options, but for security purposes,
this email is not included here.  If you have forgotten your password you
will need to click on the 'Forgot Password?' link on the login page.

Let’s say though that you wanted to provide a link to a Code of Conduct in the welcome message. You publish both the code of conduct and the welcome message pointing to the code on your website. Now you can tell the mailing list to use this welcome message instead of the default one.

>>> call_http('http://localhost:9001/3.1/lists/ant.example.com/uris', {
...     'list:user:notice:welcome': 'http://localhost:8180/welcome_1.txt',
...     }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

The name of the template corresponding to the welcome message is list:user:notice:welcome and the location of your new welcome message text is at http://localhost:8180/welcome_1.txt.

Now when a new member subscribes to the mailing list, they’ll see the new welcome message.

>>> bill = subscribe(ant, 'Bill')
>>> items = get_queue_messages('virgin')
>>> print(items[0].msg)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Welcome to the "Ant" mailing list
From: ant-request@example.com
To: Bill Person <bperson@example.com>
...
<BLANKLINE>
Welcome to the "Ant" mailing list!
<BLANKLINE>
To post to this list, send your email to:
<BLANKLINE>
  ant@example.com
<BLANKLINE>
There is a Code of Conduct for this mailing list which you can view at
http://www.example.com/code-of-conduct.html

It’s even possible to require a username and password (Basic Auth) for retrieving the welcome message.

>>> call_http('http://localhost:9001/3.1/lists/ant.example.com/uris', {
...     'list:user:notice:welcome': 'http://localhost:8180/welcome_2.txt',
...     'username': 'anne',
...     'password': 'is special',
...     }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

The username and password will be used to retrieve the welcome text.

>>> cris = subscribe(ant, 'Cris')
>>> items = get_queue_messages('virgin')
>>> print(items[0].msg)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Welcome to the "Ant" mailing list
From: ant-request@example.com
To: Cris Person <cperson@example.com>
...
<BLANKLINE>
I'm glad you made it!

The text is cached so subsequent uses don’t necessarily need to hit the internet.

>>> dave = subscribe(ant, 'Dave')
>>> items = get_queue_messages('virgin')
>>> print(items[0].msg)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Welcome to the "Ant" mailing list
From: ant-request@example.com
To: Dave Person <dperson@example.com>
...
<BLANKLINE>
I'm glad you made it!
Template format

Mailman expects the templates to be return as content type text/plain; charset=”UTF-8”.

Template URLs can be any of the following schemes:

  • http:// - standard scheme supported by the requests library;
  • https:// - standard scheme also supported by requests;
  • file:/// - any path on the local file system; UTF-8 contents by default;
  • mailman:/// - a path defined within the Mailman source code tree. It is not recommended that you use these; they are primarily provided for Mailman’s internal use.

Generally, if a template is not defined or not found, the empty string is used. IOW, a missing template does not cause an error, it simply causes the named template to be blank.

URL placeholders

The URLs themselves can contain placeholders, and this can be used to provide even more flexibility in the way the template texts are retrieved. Two common placeholders include the List-ID and the mailing list’s preferred language code.

>>> ant.preferred_language = 'fr'
>>> call_http('http://localhost:9001/3.1/lists/ant.example.com/uris', {
...     'list:user:notice:welcome':
...     'http://localhost:8180/$list_id/$language/welcome_3.txt',
...     }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

The next person to subscribe will get a French welcome message.

>>> dave = subscribe(ant, 'Elle')
>>> items = get_queue_messages('virgin')
>>> print(items[0].msg)
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Subject: =?iso-8859-1?q?Welcome_to_the_=22Ant=22_mailing_list?=
From: ant-request@example.com
To: Elle Person <eperson@example.com>
...
<BLANKLINE>
Je suis heureux que vous pouvez nous rejoindre!

Standard URL substitutions include:

  • $list_id - The mailing list’s List-ID (ant.example.com)
  • $listname - The mailing list’s fully qualified list name (ant@example.com)
  • $domain_name - The mailing list’s domain name (example.com)
  • $language - The language code for the mailing list’s preferred language (fr)
Template contexts

When Mailman is looking for a template, it always searches for it in up to three contexts, and you can set the template for any of these three contexts: a mailing list, a domain, the site.

Most templates are searched first by the mailing list, then by domain, then by site. One notable exception is the domain:admin:notice:new-list template, which is sent when a new mailing list is created. Because (modulo any style default settings) there won’t be a template for the newly created mailing list, this template is always searched for first in the domain, and then in the site.

In fact, this illustrates a common naming scheme for templates. The colon-separated sections usually follow the form <context>:<recipient>:<type>:<name> where context would be “domain” or “list, <recipient> would be “admin”, “user”, or “member”, and <type> can be “action” or “notice”. This isn’t a strict naming scheme, but it does give you some indication as to the use of the template. All template names used internally by Mailman are given below.

You’ve already seen how the mailing list context works above. Let’s look at the domain and site contexts next.

Domain context

Let’s say you want all mailing lists in a given domain to share exactly the same welcome message template. Remember that Mailman will insert substitutions into the templates themselves to customize them for each mailing list, so in general a single template can be shared by all mailing lists in the domain.

The first thing to do is to set the URI for the welcome message in the domain to be shared.

>>> call_http('http://localhost:9001/3.1/domains/example.com/uris', {
...     'list:user:notice:welcome':
...     'http://localhost:8180/welcome_4.txt',
...     }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

And let’s create a new mailing list in this domain.

>>> bee = create_list('bee@example.com')

Now when Anne subscribes to the Bee mailing list, she will get this domain-wide welcome message.

>>> anne = subscribe(bee, 'Anne')
>>> items = get_queue_messages('virgin')
>>> print(items[0].msg)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Welcome to the "Bee" mailing list
From: bee-request@example.com
To: Anne Person <aperson@example.com>
...
Welcome to the Bee list in the example.com domain.

So far so good. What happens if Fred subscribes to the Ant mailing list?

>>> fred = subscribe(ant, 'Fred')
>>> items = get_queue_messages('virgin')
>>> print(items[0].msg)
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Subject: =?iso-8859-1?q?Welcome_to_the_=22Ant=22_mailing_list?=
From: ant-request@example.com
To: Fred Person <fperson@example.com>
...
<BLANKLINE>
Je suis heureux que vous pouvez nous rejoindre!

Okay, that’s strange! Why did Fred get the French welcome message? It’s because the mailing list context overrides the domain context! Similarly, a domain context overrides a site context. This allows you to provide generic templates to be used as a default, with specific overrides where necessary.

Let’s delete the Ant list’s override.

>>> ant.preferred_language = 'en'
>>> call_http('http://localhost:9001/3.1/lists/ant.example.com/uris'
...           '/list:user:notice:welcome',
...           method='DELETE')
content-length: 0
date: ...
server: ...
status: 204

Now when Gwen subscribes to the Ant list, she gets the domain’s welcome message.

>>> gwen = subscribe(ant, 'Gwen')
>>> items = get_queue_messages('virgin')
>>> print(items[0].msg)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Welcome to the "Ant" mailing list
From: ant-request@example.com
To: Gwen Person <gperson@example.com>
...
<BLANKLINE>
Welcome to the Ant list in the example.com domain.
Site context

Let’s say we want the same welcome template for every mailing list on our Mailman installation. For this we use the site context.

First, let’s delete the domain context we set previously. Note that previously we used a DELETE method on the list’s welcome template resource, but we could have also done this by PATCHing an empty string for the URI, which Mailman’s REST API interprets as a deletion too. Let’s use this approach to delete the domain welcome message.

>>> call_http('http://localhost:9001/3.1/domains/example.com/uris', {
...     'list:user:notice:welcome': '',
...     }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

Now let’s set a new welcome template URI for the site.

>>> call_http('http://localhost:9001/3.1/uris', {
...     'list:user:notice:welcome':
...     'http://localhost:8180/welcome_5.txt',
...     }, method='PATCH')
content-length: 0
date: ...
server: ...
status: 204

Now Herb subscribes to both the Ant…

>>> herb = subscribe(ant, 'Herb')
>>> items = get_queue_messages('virgin')
>>> print(items[0].msg)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Welcome to the "Ant" mailing list
From: ant-request@example.com
To: Herb Person <hperson@example.com>
...
<BLANKLINE>
Yay! You joined the ant@example.com mailing list.

…and Bee mailing lists.

>>> herb = subscribe(bee, 'Herb')
>>> items = get_queue_messages('virgin')
>>> print(items[0].msg)
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Welcome to the "Bee" mailing list
From: bee-request@example.com
To: Herb Person <hperson@example.com>
...
<BLANKLINE>
Yay! You joined the bee@example.com mailing list.
Templated texts

All the texts that Mailman uses to create or decorate messages can be associated with a URL. Mailman looks up templates by name and downloads it via that URL. The retrieved text supports placeholders which are filled in by Mailman. There are a common set of placeholders most templates support:

  • listname - fully qualified list name (e.g. ant@example.com)
  • list_id - the List-ID header (e.g. ant.example.com)
  • display_name - the display name of the mailing list (e.g. Ant)
  • short_listname - the local part of the list name (e.g. ant)
  • domain - the domain name part of the list name (e.g. example.com)
  • description - the mailing list’s short description text
  • info - the mailing list’s longer descriptive text
  • request_email - the email address for the -request alias
  • owner_email - the email address for the -owner alias
  • site_email - the email address to reach the owners of the site
  • language - the two letter language code for the list’s preferred language (e.g. en, it, fr)

Other template substitutions are described below the template name listed below. Here are all the supported template names:

  • domain:admin:notice:new-list

    Sent to the administrators of any newly created mailing list.

  • list:admin:action:post

    Sent to the list administrators when moderator approval for a posting is required.

    • subject - the original Subject of the message
    • sender_email - the poster’s email address
    • reasons - some reasons why the post is being held for approval
  • list:admin:action:subscribe

    Sent to the list administrators when moderator approval for a subscription request is required.

    • member - display name and email address of the subscriber
  • list:admin:action:unsubscribe

    Sent to the list administrators when moderator approval for an unsubscription request is required.

    • member - display name and email address of the subscriber
  • list:admin:notice:subscribe

    Sent to the list administrators to notify them when a new member has been subscribed.

    • member - display name and email address of the subscriber
  • list:admin:notice:unrecognized

    Sent to the list administrators when a bounce message in an unrecognized format has been received.

  • list:admin:notice:unsubscribe

    Sent to the list administrators to notify them when a member has been unsubscribed.

    • member - display name and email address of the subscriber
  • list:member:digest:footer

    The footer for a digest message.

  • list:member:digest:header

    The header for a digest message.

  • list:member:digest:masthead

    The digest “masthead”; i.e. a common introduction for all digest messages.

  • list:member:regular:footer

    The footer for a regular (non-digest) message.

    When personalized deliveries are enabled, these substitution variables are also defined:

    • member - display name and email address of the subscriber
    • user_email - the email address of the recipient
    • user_delivered_to - the case-preserved email address of the recipient
    • user_language - the description of the user’s preferred language (e.g. “French”, “English”, “Italian”)
    • user_name - the recipient’s display name if available
  • list:member:regular:header

    The header for a regular (non-digest) message.

    When personalized deliveries are enabled, these substitution variables are also defined:

    • member - display name and email address of the subscriber
    • user_email - the email address of the recipient
    • user_delivered_to - the case-preserved email address of the recipient
    • user_language - the description of the user’s preferred language (e.g. “French”, “English”, “Italian”)
    • user_name - the recipient’s display name if available
  • list:user:action:subscribe

    The message sent to subscribers when a subscription confirmation is required.

    • token - the unique confirmation token
    • subject - the Subject heading for the confirmation email, which includes the confirmation token
    • confirm_email - the email address to send the confirmation response to; this corresponds to the Reply-To header
    • user_email - the email address being confirmed
  • list:user:action:unsubscribe

    The message sent to subscribers when an unsubscription confirmation is required.

    • token - the unique confirmation token
    • subject - the Subject heading for the confirmation email, which includes the confirmation token
    • confirm_email - the email address to send the confirmation response to; this corresponds to the Reply-To header
    • user_email - the email address being confirmed
  • list:user:notice:goodbye

    The notice sent to a member when they unsubscribe from a mailing list.

  • list:user:notice:hold

    The notice sent to a poster when their message is being held or moderator approval.

    • subject - the original Subject of the message
    • sender_email - the poster’s email address
    • reasons - some reasons why the post is being held for approval
  • list:user:notice:no-more-today

    Sent to a user when the maximum number of autoresponses has been reached for that day.

    • sender_email - the email address of the poster
    • count - the number of autoresponse messages sent to the user today
  • list:user:notice:post

    Notice sent to a poster when their message has been received by the mailing list.

    • subject - the Subject field of the received message
  • list:user:notice:probe

    A bounce probe sent to a member when their subscription has been disabled due to bounces.

    • sender_email - the email address of the bouncing member
  • list:user:notice:refuse

    Notice sent to a poster when their message has been rejected by the list’s moderator.

    • request - the type of request being rejected
    • reason - the reason for the rejection, as provided by the list’s moderators
  • list:user:notice:welcome

    The notice sent to a member when they are subscribed to the mailing list.

    • user_name - the display name of the new member
    • user_email - the email address of the new member

Core

Chains

When a new message is posted to a mailing list, Mailman uses a set of rule chains to decide whether the message gets accepted for posting, rejected, discarded, or held for moderator approval.

There are a number of built-in chains available that act as end-points in the processing of messages.

The Discard chain

The discard chain simply throws the message away.

>>> chain = config.chains['discard']
>>> print(chain.name)
discard
>>> print(chain.description)
Discard a message and stop processing.

>>> mlist = create_list('test@example.com')
>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: My first post
... Message-ID: <first>
...
... An important message.
... """)

>>> def print_msgid(event):
...     print('{0}: {1}'.format(
...         event.chain.name.upper(), event.msg.get('message-id', 'n/a')))

>>> from mailman.core.chains import process
>>> from mailman.testing.helpers import event_subscribers

>>> with event_subscribers(print_msgid):
...     process(mlist, msg, {}, 'discard')
DISCARD: <first>
The Reject chain

The reject chain bounces the message back to the original sender, and logs this action.

>>> chain = config.chains['reject']
>>> print(chain.name)
reject
>>> print(chain.description)
Reject/bounce a message and stop processing.

>>> with event_subscribers(print_msgid):
...     process(mlist, msg, {}, 'reject')
REJECT: <first>

The bounce message is now sitting in the virgin queue.

>>> from mailman.testing.helpers import get_queue_messages
>>> qfiles = get_queue_messages('virgin')
>>> len(qfiles)
1
>>> print(qfiles[0].msg.as_string())
Subject: My first post
From: test-owner@example.com
To: aperson@example.com
...
[No bounce details are available]
...
Content-Type: message/rfc822
MIME-Version: 1.0
<BLANKLINE>
From: aperson@example.com
To: test@example.com
Subject: My first post
Message-ID: <first>
<BLANKLINE>
An important message.
<BLANKLINE>
...
The Hold Chain

The hold chain places the message into the administrative request database and depending on the list’s settings, sends a notification to both the original sender and the list moderators.

>>> chain = config.chains['hold']
>>> print(chain.name)
hold
>>> print(chain.description)
Hold a message and stop processing.

>>> with event_subscribers(print_msgid):
...     process(mlist, msg, {}, 'hold')
HOLD: <first>

There are now two messages in the virgin queue, one to the list moderators and one to the original author.

>>> qfiles = get_queue_messages('virgin', sort_on='to')
>>> len(qfiles)
2

One of the message is addressed to the mailing list moderators, and the other is addressed to the original sender.

>>> from operator import itemgetter
>>> messages = sorted((item.msg for item in qfiles),
...                   key=itemgetter('to'), reverse=True)

This one is addressed to the list moderators.

>>> print(messages[0].as_string())
Subject: test@example.com post from aperson@example.com requires approval
From: test-owner@example.com
To: test-owner@example.com
MIME-Version: 1.0
...
As list administrator, your authorization is requested for the
following mailing list posting:
<BLANKLINE>
    List:    test@example.com
    From:    aperson@example.com
    Subject: My first post
<BLANKLINE>
The message is being held because:
<BLANKLINE>
    N/A
At your convenience, visit your dashboard to approve or deny the
request.
<BLANKLINE>
...
Content-Type: message/rfc822
MIME-Version: 1.0
<BLANKLINE>
From: aperson@example.com
To: test@example.com
Subject: My first post
Message-ID: <first>
Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
<BLANKLINE>
An important message.
<BLANKLINE>
...
Content-Type: message/rfc822
MIME-Version: 1.0
<BLANKLINE>
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: confirm ...
From: test-request@example.com
...
<BLANKLINE>
If you reply to this message, keeping the Subject: header intact,
Mailman will discard the held message.  Do this if the message is
spam.  If you reply to this message and include an Approved: header
with the list password in it, the message will be approved for posting
to the list.  The Approved: header can also appear in the first line
of the body of the reply.
...

This message is addressed to the sender of the message.

>>> print(messages[1].as_string())
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Your message to test@example.com awaits moderator approval
From: test-bounces@example.com
To: aperson@example.com
...
Your mail to 'test@example.com' with the subject
<BLANKLINE>
    My first post
<BLANKLINE>
Is being held until the list moderator can review it for approval.
<BLANKLINE>
The message is being held because:
<BLANKLINE>
    N/A
<BLANKLINE>
Either the message will get posted to the list, or you will receive
notification of the moderator's decision.
The Accept chain

The accept chain sends the message on the pipeline queue, where it will be processed and sent on to the list membership.

>>> chain = config.chains['accept']
>>> print(chain.name)
accept
>>> print(chain.description)
Accept a message.

>>> with event_subscribers(print_msgid):
...     process(mlist, msg, {}, 'accept')
ACCEPT: <first>

>>> qfiles = get_queue_messages('pipeline')
>>> len(qfiles)
1
>>> print(qfiles[0].msg.as_string())
From: aperson@example.com
To: test@example.com
Subject: My first post
Message-ID: <first>
Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB

An important message.
Run-time chains

We can also define chains at run time, and these chains can be mutated. Run-time chains are made up of links where each link associates both a rule and a jump. The rule is really a rule name, which is looked up when needed. The jump names a chain which is jumped to if the rule matches.

There is one built-in posting chain. This is the default chain to use when no other input chain is defined for a mailing list. It runs through the default rules.

>>> chain = config.chains['default-posting-chain']
>>> print(chain.name)
default-posting-chain
>>> print(chain.description)
The built-in moderation chain.

Once the sender is a member of the mailing list, the previously created message is innocuous enough that it should pass through all default rules. This message will end up in the pipeline queue.

>>> from mailman.testing.helpers import subscribe
>>> subscribe(mlist, 'Anne')
<Member: aperson@example.com on test@example.com as MemberRole.member>

>>> with event_subscribers(print_msgid):
...     process(mlist, msg, {})
ACCEPT: <first>

>>> qfiles = get_queue_messages('pipeline')
>>> len(qfiles)
1
>>> print(qfiles[0].msg.as_string())
From: aperson@example.com
To: test@example.com
Subject: My first post
Message-ID: <first>
Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency;
    loop; banned-address; member-moderation; nonmember-moderation;
    administrivia; implicit-dest; max-recipients; max-size;
    news-moderation; no-subject; suspicious-header

An important message.

In addition, the message metadata now contains lists of all rules that have hit and all rules that have missed.

>>> dump_list(qfiles[0].msgdata['rule_hits'])
*Empty*
>>> dump_list(qfiles[0].msgdata['rule_misses'])
administrivia
approved
banned-address
dmarc-mitigation
emergency
implicit-dest
loop
max-recipients
max-size
member-moderation
news-moderation
no-senders
no-subject
nonmember-moderation
suspicious-header
Runners

The runners are the processes that perform long-running tasks, such as moving messages around the Mailman queues. Some runners don’t manage queues, such as the LMTP and REST API handling runners. Each runner that manages a queue directory though, is responsible for a slice of the hash space. It processes all the files in its slice, sleeps a little while, then wakes up and runs through its queue files again.

Basic architecture

The basic architecture of runner is implemented in the base class that all runners inherit from. This base class implements a .run() method that runs continuously in a loop until the .stop() method is called.

>>> mlist = create_list('test@example.com')

Here is a very simple derived runner class. Runners use a configuration section in the configuration files to determine run characteristics, such as the queue directory to use. Here we push a configuration section for the test runner.

>>> config.push('test-runner', """
... [runner.test]
... max_restarts: 1
... """)

>>> from mailman.core.runner import Runner
>>> class TestableRunner(Runner):
...     def _dispose(self, mlist, msg, msgdata):
...         self.msg = msg
...         self.msgdata = msgdata
...         return False
...
...     def _do_periodic(self):
...         self.stop()
...
...     def _snooze(self, filecnt):
...         return

>>> runner = TestableRunner('test')

This runner doesn’t do much except run once, storing the message and metadata on instance variables.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
...
... A test message.
... """)
>>> switchboard = config.switchboards['test']
>>> filebase = switchboard.enqueue(msg, listid=mlist.list_id,
...                                foo='yes', bar='no')
>>> runner.run()
>>> print(runner.msg.as_string())
From: aperson@example.com
To: test@example.com
<BLANKLINE>
A test message.
<BLANKLINE>
>>> dump_msgdata(runner.msgdata)
_parsemsg: False
bar      : no
foo      : yes
lang     : en
listid   : test.example.com
version  : 3

XXX More of the Runner API should be tested.

The switchboard

The switchboard is subsystem that moves messages between queues. Each instance of a switchboard is responsible for one queue directory.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: _xtest@example.com
...
... A test message.
... """)

Create a switchboard by giving its queue name and directory.

>>> import os
>>> queue_directory = os.path.join(config.QUEUE_DIR, 'test')
>>> from mailman.core.switchboard import Switchboard
>>> switchboard = Switchboard('test', queue_directory)
>>> print(switchboard.name)
test
>>> switchboard.queue_directory == queue_directory
True

Here’s a helper function for ensuring things work correctly.

>>> def check_qfiles(directory=None):
...     if directory is None:
...         directory = queue_directory
...     files = {}
...     for qfile in os.listdir(directory):
...         root, ext = os.path.splitext(qfile)
...         files[ext] = files.get(ext, 0) + 1
...     if len(files) == 0:
...         print('empty')
...     for ext in sorted(files):
...         print('{0}: {1}'.format(ext, files[ext]))
Enqueing and dequeing

The message can be enqueued with metadata specified in the passed in dictionary.

>>> filebase = switchboard.enqueue(msg)
>>> check_qfiles()
.pck: 1

To read the contents of a queue file, dequeue it.

>>> msg, msgdata = switchboard.dequeue(filebase)
>>> print(msg.as_string())
From: aperson@example.com
To: _xtest@example.com
<BLANKLINE>
A test message.
<BLANKLINE>
>>> dump_msgdata(msgdata)
_parsemsg: False
version  : 3
>>> check_qfiles()
.bak: 1

To complete the dequeing process, removing all traces of the message file, finish it (without preservation).

>>> switchboard.finish(filebase)
>>> check_qfiles()
empty

When enqueing a file, you can provide additional metadata keys by using keyword arguments.

>>> filebase = switchboard.enqueue(msg, {'foo': 1}, bar=2)
>>> msg, msgdata = switchboard.dequeue(filebase)
>>> switchboard.finish(filebase)
>>> dump_msgdata(msgdata)
_parsemsg: False
bar      : 2
foo      : 1
version  : 3

Keyword arguments override keys from the metadata dictionary.

>>> filebase = switchboard.enqueue(msg, {'foo': 1}, foo=2)
>>> msg, msgdata = switchboard.dequeue(filebase)
>>> switchboard.finish(filebase)
>>> dump_msgdata(msgdata)
_parsemsg: False
foo      : 2
version  : 3
Iterating over files

There are two ways to iterate over all the files in a switchboard’s queue. Normally, queue files end in .pck (for ‘pickle’) and the easiest way to iterate over just these files is to use the .files attribute.

>>> filebase_1 = switchboard.enqueue(msg, foo=1)
>>> filebase_2 = switchboard.enqueue(msg, foo=2)
>>> filebase_3 = switchboard.enqueue(msg, foo=3)
>>> filebases = sorted((filebase_1, filebase_2, filebase_3))
>>> sorted(switchboard.files) == filebases
True
>>> check_qfiles()
.pck: 3

You can also use the .get_files() method if you want to iterate over all the file bases for some other extension.

>>> for filebase in switchboard.get_files():
...     msg, msgdata = switchboard.dequeue(filebase)
>>> bakfiles = sorted(switchboard.get_files('.bak'))
>>> bakfiles == filebases
True
>>> check_qfiles()
.bak: 3
>>> for filebase in switchboard.get_files('.bak'):
...     switchboard.finish(filebase)
>>> check_qfiles()
empty
Recovering files

Calling .dequeue() without calling .finish() leaves .bak backup files in place. These can be recovered when the switchboard is instantiated.

>>> filebase_1 = switchboard.enqueue(msg, foo=1)
>>> filebase_2 = switchboard.enqueue(msg, foo=2)
>>> filebase_3 = switchboard.enqueue(msg, foo=3)
>>> for filebase in switchboard.files:
...     msg, msgdata = switchboard.dequeue(filebase)
...     # Don't call .finish()
>>> check_qfiles()
.bak: 3
>>> switchboard_2 = Switchboard('test', queue_directory, recover=True)
>>> check_qfiles()
.pck: 3

The files can be recovered explicitly.

>>> for filebase in switchboard.files:
...     msg, msgdata = switchboard.dequeue(filebase)
...     # Don't call .finish()
>>> check_qfiles()
.bak: 3
>>> switchboard.recover_backup_files()
>>> check_qfiles()
.pck: 3

But the files will only be recovered at most three times before they are considered defective. In order to prevent mail bombs and loops, once this maximum is reached, the files will be preserved in the ‘bad’ queue.

>>> for filebase in switchboard.files:
...     msg, msgdata = switchboard.dequeue(filebase)
...     # Don't call .finish()
>>> check_qfiles()
.bak: 3
>>> switchboard.recover_backup_files()
>>> check_qfiles()
empty

>>> bad = config.switchboards['bad']
>>> check_qfiles(bad.queue_directory)
.psv: 3
Clean up
>>> for file in os.listdir(bad.queue_directory):
...     os.remove(os.path.join(bad.queue_directory, file))
>>> check_qfiles(bad.queue_directory)
empty
Queue slices

XXX Add tests for queue slices.

App

Banning email addresses

Email addresses can be banned from ever subscribing, either to a specific mailing list or globally within the Mailman system. Both explicit email addresses and email address patterns can be banned.

Bans are managed through the Ban Manager. There are ban managers for specific lists, and there is a global ban manager. To get access to the global ban manager, adapt None.

>>> from mailman.interfaces.bans import IBanManager
>>> global_bans = IBanManager(None)

At first, no email addresses are banned globally.

>>> global_bans.is_banned('anne@example.com')
False

To get a list-specific ban manager, adapt the mailing list object.

>>> mlist = create_list('test@example.com')
>>> test_bans = IBanManager(mlist)

There are no bans for this particular list.

>>> test_bans.is_banned('bart@example.com')
False
Specific bans

An email address can be banned from a specific mailing list by adding a ban to the list’s ban manager.

>>> test_bans.ban('cris@example.com')
>>> test_bans.is_banned('cris@example.com')
True
>>> test_bans.is_banned('bart@example.com')
False

However, this is not a global ban.

>>> global_bans.is_banned('cris@example.com')
False
Global bans

An email address can be banned globally, so that it cannot be subscribed to any mailing list.

>>> global_bans.ban('dave@example.com')

Because there is a global ban, Dave is also banned from the mailing list.

>>> test_bans.is_banned('dave@example.com')
True

Even when a new mailing list is created, Dave is still banned from this list because of his global ban.

>>> sample = create_list('sample@example.com')
>>> sample_bans = IBanManager(sample)
>>> sample_bans.is_banned('dave@example.com')
True

Dave is of course banned globally.

>>> global_bans.is_banned('dave@example.com')
True

Cris however is not banned globally.

>>> global_bans.is_banned('cris@example.com')
False

Even though Cris is not banned globally, we can add a global ban for her.

>>> global_bans.ban('cris@example.com')
>>> global_bans.is_banned('cris@example.com')
True

Cris is now banned from all mailing lists.

>>> test_bans.is_banned('cris@example.com')
True
>>> sample_bans.is_banned('cris@example.com')
True

We can remove the global ban to once again just ban her address from just the test list.

>>> global_bans.unban('cris@example.com')
>>> global_bans.is_banned('cris@example.com')
False
>>> test_bans.is_banned('cris@example.com')
True
>>> sample_bans.is_banned('cris@example.com')
False
Regular expression bans

Entire email address patterns can be banned, both for a specific mailing list and globally, just as specific addresses can be banned. Use this for example, when an entire domain is a spam faucet. When using a pattern, the email address must start with a caret (^).

>>> test_bans.ban('^.*@example.org')

Now, no one from example.org can subscribe to the test mailing list.

>>> test_bans.is_banned('elle@example.org')
True
>>> test_bans.is_banned('eperson@example.org')
True

example.com addresses are not banned.

>>> test_bans.is_banned('elle@example.com')
False

example.org addresses are not banned globally, nor for any other mailing list.

>>> sample_bans.is_banned('elle@example.org')
False
>>> global_bans.is_banned('elle@example.org')
False

Of course, we can ban everyone from example.org globally too.

>>> global_bans.ban('^.*@example.org')
>>> sample_bans.is_banned('elle@example.org')
True
>>> global_bans.is_banned('elle@example.org')
True

We can remove the mailing list ban on the pattern, though the global ban will still be in place.

>>> test_bans.unban('^.*@example.org')
>>> test_bans.is_banned('elle@example.org')
True
>>> sample_bans.is_banned('elle@example.org')
True
>>> global_bans.is_banned('elle@example.org')
True

But once the global ban is removed, everyone from example.org can subscribe to the mailing lists.

>>> global_bans.unban('^.*@example.org')
>>> test_bans.is_banned('elle@example.org')
False
>>> sample_bans.is_banned('elle@example.org')
False
>>> global_bans.is_banned('elle@example.org')
False
Adding and removing bans

It is not an error to add a ban more than once. These are just ignored.

>>> test_bans.ban('fred@example.com')
>>> test_bans.ban('fred@example.com')
>>> test_bans.is_banned('fred@example.com')
True

Nor is it an error to remove a ban more than once.

>>> test_bans.unban('fred@example.com')
>>> test_bans.unban('fred@example.com')
>>> test_bans.is_banned('fred@example.com')
False
Bounces

An important feature of Mailman is automatic bounce processing.

Bounces, or message rejection

Mailman can bounce messages back to the original sender. This is essentially equivalent to rejecting the message with notification. Mailing lists can bounce a message with an optional error message.

>>> mlist = create_list('ant@example.com')

Any message can be bounced.

>>> msg = message_from_string("""\
... To: ant@example.com
... From: aperson@example.com
... Subject: Something important
...
... I sometimes say something important.
... """)

Bounce a message by passing in the original message, and an optional error message. The bounced message ends up in the virgin queue, awaiting sending to the original message author.

>>> from mailman.app.bounces import bounce_message
>>> bounce_message(mlist, msg)
>>> from mailman.testing.helpers import get_queue_messages
>>> items = get_queue_messages('virgin')
>>> len(items)
1
>>> print(items[0].msg.as_string())
Subject: Something important
From: ant-owner@example.com
To: aperson@example.com
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="..."
Message-ID: ...
Date: ...
Precedence: bulk
<BLANKLINE>
--...
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
<BLANKLINE>
[No bounce details are available]
--...
Content-Type: message/rfc822
MIME-Version: 1.0
<BLANKLINE>
To: ant@example.com
From: aperson@example.com
Subject: Something important
<BLANKLINE>
I sometimes say something important.
<BLANKLINE>
--...--

An error message can be given when the message is bounced, and this will be included in the payload of the text/plain part. The error message must be passed in as an instance of a RejectMessage exception.

>>> from mailman.interfaces.pipeline import RejectMessage
>>> error = RejectMessage("This wasn't very important after all.")
>>> bounce_message(mlist, msg, error)
>>> items = get_queue_messages('virgin', expected_count=1)
>>> print(items[0].msg.as_string())
Subject: Something important
From: ant-owner@example.com
To: aperson@example.com
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="..."
Message-ID: ...
Date: ...
Precedence: bulk
<BLANKLINE>
--...
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
<BLANKLINE>
This wasn't very important after all.
--...
Content-Type: message/rfc822
MIME-Version: 1.0
<BLANKLINE>
To: ant@example.com
From: aperson@example.com
Subject: Something important
<BLANKLINE>
I sometimes say something important.
<BLANKLINE>
--...--

The RejectMessage exception can also include a set of reasons, which will be interpolated into the message using the {reasons} placeholder.

>>> error = RejectMessage("""This message is rejected because:
...
... $reasons
... """, [
...     'I am not happy',
...     'You are not happy',
...     'We are not happy'])
>>> bounce_message(mlist, msg, error)
>>> items = get_queue_messages('virgin', expected_count=1)
>>> print(items[0].msg.as_string())
Subject: Something important
From: ant-owner@example.com
To: aperson@example.com
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="..."
Message-ID: ...
Date: ...
Precedence: bulk
<BLANKLINE>
--...
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
<BLANKLINE>
This message is rejected because:
<BLANKLINE>
I am not happy
You are not happy
We are not happy
<BLANKLINE>
--...
Content-Type: message/rfc822
MIME-Version: 1.0
<BLANKLINE>
To: ant@example.com
From: aperson@example.com
Subject: Something important
<BLANKLINE>
I sometimes say something important.
<BLANKLINE>
--...
<BLANKLINE>
Hooks

Mailman defines two initialization hooks, one which is run early in the initialization process and the other run late in the initialization process. Hooks name an importable callable so it must be accessible on sys.path.

>>> import os, sys
>>> from mailman.config import config
>>> config_directory = os.path.dirname(config.filename)
>>> sys.path.insert(0, config_directory)

>>> hook_path = os.path.join(config_directory, 'hooks.py')
>>> with open(hook_path, 'w') as fp:
...     print("""\
... counter = 1
... def pre_hook():
...     global counter
...     print('pre-hook:', counter)
...     counter += 1
...
... def post_hook():
...     global counter
...     print('post-hook:', counter)
...     counter += 1
... """, file=fp)
>>> fp.close()
Pre-hook

We can set the pre-hook in the configuration file.

>>> config_path = os.path.join(config_directory, 'hooks.cfg')
>>> with open(config_path, 'w') as fp:
...     print("""\
... [meta]
... extends: test.cfg
...
... [mailman]
... pre_hook: hooks.pre_hook
... """, file=fp)

The hooks are run in the second and third steps of initialization. However, we can’t run those initialization steps in process, so call a command line script that will produce no output to force the hooks to run.

>>> import subprocess
>>> from mailman.testing.layers import ConfigLayer
>>> def call():
...     exe = os.path.join(os.path.dirname(sys.executable), 'mailman')
...     env = dict(MAILMAN_CONFIG_FILE=config_path,
...                PYTHONPATH=config_directory)
...     test_cfg = os.environ.get('MAILMAN_EXTRA_TESTING_CFG')
...     if test_cfg is not None:
...         env['MAILMAN_EXTRA_TESTING_CFG'] = test_cfg
...     proc = subprocess.Popen(
...         [exe, 'lists', '--domain', 'ignore', '-q'],
...         cwd=ConfigLayer.root_directory, env=env,
...         universal_newlines=True,
...         stdout=subprocess.PIPE, stderr=subprocess.PIPE)
...     stdout, stderr = proc.communicate()
...     assert proc.returncode == 0, stderr
...     print(stdout)

>>> call()
pre-hook: 1


>>> os.remove(config_path)
Post-hook

We can set the post-hook in the configuration file.

>>> with open(config_path, 'w') as fp:
...     print("""\
... [meta]
... extends: test.cfg
...
... [mailman]
... post_hook: hooks.post_hook
... """, file=fp)

>>> call()
post-hook: 1


>>> os.remove(config_path)
Running both hooks

We can set the pre- and post-hooks in the configuration file.

>>> with open(config_path, 'w') as fp:
...     print("""\
... [meta]
... extends: test.cfg
...
... [mailman]
... pre_hook: hooks.pre_hook
... post_hook: hooks.post_hook
... """, file=fp)

>>> call()
pre-hook: 1
post-hook: 2
Application level list life cycle

The low-level way to create and delete a mailing list is to use the IListManager interface. This interface simply adds or removes the appropriate database entries to record the list’s creation.

There is a higher level interface for creating and deleting mailing lists which performs additional tasks such as:

  • validating the list’s posting address (which also serves as the list’s fully qualified name);
  • ensuring that the list’s domain is registered;
  • applying a list style to the new list;
  • creating and assigning list owners;
  • notifying watchers of list creation;
  • creating ancillary artifacts (such as the list’s on-disk directory)
Creating a list with owners

You can also specify a list of owner email addresses. If these addresses are not yet known, they will be registered, and new users will be linked to them.

>>> owners = [
...     'aperson@example.com',
...     'bperson@example.com',
...     'cperson@example.com',
...     'dperson@example.com',
...     ]

>>> ant = create_list('ant@example.com', owners)
>>> dump_list(address.email for address in ant.owners.addresses)
aperson@example.com
bperson@example.com
cperson@example.com
dperson@example.com

None of the owner addresses are verified.

>>> any(address.verified_on is not None
...     for address in ant.owners.addresses)
False

However, all addresses are linked to users.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> user_manager = getUtility(IUserManager)
>>> for address in owners:
...     user = user_manager.get_user(address)
...     print(int(user.user_id.int), list(user.addresses)[0])
1 aperson@example.com
2 bperson@example.com
3 cperson@example.com
4 dperson@example.com

If you create a mailing list with owner addresses that are already known to the system, they won’t be created again.

>>> bee = create_list('bee@example.com', owners)
>>> from operator import attrgetter
>>> for user in sorted(bee.owners.users, key=attrgetter('user_id')):
...     print(int(user.user_id.int), list(user.addresses)[0])
1 aperson@example.com
2 bperson@example.com
3 cperson@example.com
4 dperson@example.com
Deleting a list

Removing a mailing list deletes the list, all its subscribers, and any related artifacts.

>>> from mailman.app.lifecycle import remove_list
>>> remove_list(bee)

>>> from mailman.interfaces.listmanager import IListManager
>>> print(getUtility(IListManager).get('bee@example.com'))
None

We should now be able to completely recreate the mailing list.

>>> buzz = create_list('bee@example.com', owners)
>>> dump_list(address.email for address in bee.owners.addresses)
aperson@example.com
bperson@example.com
cperson@example.com
dperson@example.com
Messages

Mailman has its own Message classes, derived from the standard email.message.Message class, but providing additional useful methods.

User notifications

When Mailman needs to send a message to a user, it creates a UserNotification instance, and then calls the .send() method on this object. This method requires a mailing list instance.

>>> mlist = create_list('test@example.com')

The UserNotification constructor takes the recipient address, the sender address, an optional subject, optional body text, and optional language.

>>> from mailman.email.message import UserNotification
>>> msg = UserNotification(
...     'aperson@example.com',
...     'test@example.com',
...     'Something you need to know',
...     'I needed to tell you this.')
>>> msg.send(mlist)

The message will end up in the virgin queue.

>>> from mailman.testing.helpers import get_queue_messages
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Subject: Something you need to know
From: test@example.com
To: aperson@example.com
Message-ID: ...
Date: ...
Precedence: bulk
<BLANKLINE>
I needed to tell you this.

The message above got a Precedence: bulk header added by default. If the message we’re sending already has a Precedence: header, it shouldn’t be changed.

>>> del msg['precedence']
>>> msg['Precedence'] = 'list'
>>> msg.send(mlist)

Again, the message will end up in the virgin queue but with the original Precedence: header.

>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg['precedence'])
list

Sometimes we want to send the message without a Precedence: header such as when we send a probe message.

>>> del msg['precedence']
>>> msg.send(mlist, add_precedence=False)

Again, the message will end up in the virgin queue but without the Precedence: header.

>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg['precedence'])
None

However, if the message already has a Precedence: header, setting the precedence=False argument will have no effect.

>>> msg['Precedence'] = 'junk'
>>> msg.send(mlist, add_precedence=False)
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg['precedence'])
junk
Application level moderation

At an application level, moderation involves holding messages and membership changes for moderator approval. This utilizes the lower level interface for list-centric moderation requests.

Moderation is always mailing list-centric.

>>> mlist = create_list('ant@example.com')
>>> mlist.preferred_language = 'en'
>>> mlist.display_name = 'A Test List'
>>> mlist.admin_immed_notify = False

We’ll use the lower level API for diagnostic purposes.

>>> from mailman.interfaces.requests import IListRequests
>>> requests = IListRequests(mlist)
Message moderation
Holding messages

Anne posts a message to the mailing list, but she is not a member of the list, so the message is held for moderator approval.

>>> msg = message_from_string("""\
... From: anne@example.org
... To: ant@example.com
... Subject: Something important
... Message-ID: <aardvark>
...
... Here's something important about our mailing list.
... """)

Holding a message means keeping a copy of it that a moderator must approve before the message is posted to the mailing list. To hold the message, the message, its metadata, and a reason for the hold must be provided. In this case, we won’t include any additional metadata.

>>> from mailman.app.moderator import hold_message
>>> hold_message(mlist, msg, {}, 'Needs approval')
1

We can also hold a message with some additional metadata.

>>> msg = message_from_string("""\
... From: bart@example.org
... To: ant@example.com
... Subject: Something important
... Message-ID: <badger>
...
... Here's something important about our mailing list.
... """)
>>> msgdata = dict(sender='anne@example.com', approved=True)

>>> hold_message(mlist, msg, msgdata, 'Feeling ornery')
2
Disposing of messages

The moderator can select one of several dispositions:

  • discard - throw the message away.
  • reject - bounces the message back to the original author.
  • defer - defer any action on the message (continue to hold it)
  • accept - accept the message for posting.

The most trivial is to simply defer a decision for now.

>>> from mailman.interfaces.action import Action
>>> from mailman.app.moderator import handle_message
>>> handle_message(mlist, 1, Action.defer)

This leaves the message in the requests database.

>>> key, data = requests.get_request(1)
>>> print(key)
<aardvark>

The moderator can also discard the message.

>>> handle_message(mlist, 1, Action.discard)
>>> print(requests.get_request(1))
None

The message can be rejected, which bounces the message back to the original sender.

>>> handle_message(mlist, 2, Action.reject, 'Off topic')

The message is no longer available in the requests database.

>>> print(requests.get_request(2))
None

And there is one message in the virgin queue - the rejection notice.

>>> from mailman.testing.helpers import get_queue_messages
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
...
Subject: Request to mailing list "A Test List" rejected
From: ant-bounces@example.com
To: bart@example.org
...
<BLANKLINE>
Your request to the ant@example.com mailing list
<BLANKLINE>
    Posting of your message titled "Something important"
<BLANKLINE>
has been rejected by the list moderator.  The moderator gave the
following reason for rejecting your request:
<BLANKLINE>
"Off topic"
<BLANKLINE>
Any questions or comments should be directed to the list administrator
at:
<BLANKLINE>
    ant-owner@example.com
<BLANKLINE>

The bounce gets sent to the original sender.

>>> for recipient in sorted(messages[0].msgdata['recipients']):
...     print(recipient)
bart@example.org

Or the message can be approved.

>>> msg = message_from_string("""\
... From: cris@example.org
... To: ant@example.com
... Subject: Something important
... Message-ID: <caribou>
...
... Here's something important about our mailing list.
... """)
>>> id = hold_message(mlist, msg, {}, 'Needs approval')
>>> handle_message(mlist, id, Action.accept)

This places the message back into the incoming queue for further processing, however the message metadata indicates that the message has been approved.

>>> messages = get_queue_messages('pipeline')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
From: cris@example.org
To: ant@example.com
Subject: Something important
...

>>> dump_msgdata(messages[0].msgdata)
_parsemsg         : False
approved          : True
moderator_approved: True
type              : data
version           : 3
Forwarding the message

The message can be forwarded to another address. This is helpful for getting the message into the inbox of one of the moderators.

>>> msg = message_from_string("""\
... From: elly@example.org
... To: ant@example.com
... Subject: Something important
... Message-ID: <elephant>
...
... Here's something important about our mailing list.
... """)
>>> req_id = hold_message(mlist, msg, {}, 'Needs approval')
>>> handle_message(mlist, req_id, Action.discard,
...                forward=['zack@example.com'])

The forwarded message is in the virgin queue, destined for the moderator.

>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
Subject: Forward of moderated message
From: ant-bounces@example.com
To: zack@example.com
...

>>> for recipient in sorted(messages[0].msgdata['recipients']):
...     print(recipient)
zack@example.com
Holding unsubscription requests

Some lists require moderator approval for unsubscriptions. In this case, only the unsubscribing address is required.

Fred is a member of the mailing list…

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> mlist.send_welcome_message = False
>>> fred = getUtility(IUserManager).create_address(
...     'fred@example.com', 'Fred Person')
>>> from mailman.interfaces.subscriptions import ISubscriptionManager
>>> registrar = ISubscriptionManager(mlist)
>>> token, token_owner, member = registrar.register(
...     fred, pre_verified=True, pre_confirmed=True, pre_approved=True)
>>> member
<Member: Fred Person <fred@example.com> on ant@example.com
         as MemberRole.member>

…but now that he wants to leave the mailing list, his request must be approved.

>>> from mailman.app.moderator import hold_unsubscription
>>> req_id = hold_unsubscription(mlist, 'fred@example.com')

As with subscription requests, the unsubscription request can be deferred.

>>> from mailman.app.moderator import handle_unsubscription
>>> handle_unsubscription(mlist, req_id, Action.defer)
>>> print(mlist.members.get_member('fred@example.com').address)
Fred Person <fred@example.com>

The held unsubscription can also be discarded, and the member will remain subscribed.

>>> handle_unsubscription(mlist, req_id, Action.discard)
>>> print(mlist.members.get_member('fred@example.com').address)
Fred Person <fred@example.com>

The request can be rejected, in which case a message is sent to the member, and the person remains a member of the mailing list.

>>> req_id = hold_unsubscription(mlist, 'fred@example.com')
>>> handle_unsubscription(mlist, req_id, Action.reject, 'No can do')
>>> print(mlist.members.get_member('fred@example.com').address)
Fred Person <fred@example.com>

Fred gets a rejection notice.

>>> messages = get_queue_messages('virgin')
>>> len(messages)
1

>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
...
Subject: Request to mailing list "A Test List" rejected
From: ant-bounces@example.com
To: fred@example.com
...
Your request to the ant@example.com mailing list

    Unsubscription request

has been rejected by the list moderator.  The moderator gave the
following reason for rejecting your request:

"No can do"
...

The unsubscription request can also be accepted. This removes the member from the mailing list.

>>> req_id = hold_unsubscription(mlist, 'fred@example.com')
>>> mlist.send_goodbye_message = False
>>> handle_unsubscription(mlist, req_id, Action.accept)
>>> print(mlist.members.get_member('fred@example.com'))
None
Notifications
Membership change requests

Usually, the list administrators want to be notified when there are membership change requests they need to moderate. These notifications are sent when the list is configured to send them.

>>> from mailman.interfaces.mailinglist import SubscriptionPolicy
>>> mlist.admin_immed_notify = True
>>> mlist.subscription_policy = SubscriptionPolicy.moderate

Gwen tries to subscribe to the mailing list.

>>> gwen = getUtility(IUserManager).create_address(
...     'gwen@example.com', 'Gwen Person')
>>> token, token_owner, member = registrar.register(
...     gwen, pre_verified=True, pre_confirmed=True)

Her subscription must be approved by the list administrator, so she is not yet a member of the mailing list.

>>> print(member)
None
>>> print(mlist.members.get_member('gwen@example.com'))
None

There’s now a message in the virgin queue, destined for the list owner.

>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
...
Subject: New subscription request to A Test List from gwen@example.com
From: ant-owner@example.com
To: ant-owner@example.com
...
Your authorization is required for a mailing list subscription request
approval:
<BLANKLINE>
    For:  Gwen Person <gwen@example.com>
    List: ant@example.com

Similarly, the administrator gets notifications on unsubscription requests. Jeff is a member of the mailing list, and chooses to unsubscribe.

>>> unsub_req_id = hold_unsubscription(mlist, 'jeff@example.org')
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
...
Subject: New unsubscription request from A Test List by jeff@example.org
From: ant-owner@example.com
To: ant-owner@example.com
...
Your authorization is required for a mailing list unsubscription
request approval:
<BLANKLINE>
    For:  jeff@example.org
    List: ant@example.com
<BLANKLINE>
Membership changes

When a new member request is accepted, the mailing list administrators can receive a membership change notice.

>>> mlist.admin_notify_mchanges = True
>>> mlist.admin_immed_notify = False
>>> token, token_owner, member = registrar.confirm(token)
>>> member
<Member: Gwen Person <gwen@example.com> on ant@example.com
         as MemberRole.member>
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
...
Subject: A Test List subscription notification
From: noreply@example.com
To: ant-owner@example.com
...
Gwen Person <gwen@example.com> has been successfully subscribed to A
Test List.

Similarly when an unsubscription request is accepted, the administrators can get a notification.

>>> req_id = hold_unsubscription(mlist, 'gwen@example.com')
>>> handle_unsubscription(mlist, req_id, Action.accept)
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
...
Subject: A Test List unsubscription notification
From: noreply@example.com
To: ant-owner@example.com
...
Gwen Person <gwen@example.com> has been removed from A Test List.
Welcome messages

When a member is subscribed to the mailing list, they can get a welcome message.

>>> mlist.admin_notify_mchanges = False
>>> mlist.send_welcome_message = True
>>> herb = getUtility(IUserManager).create_address(
...     'herb@example.com', 'Herb Person')
>>> token, token_owner, member = registrar.register(
...     herb, pre_verified=True, pre_confirmed=True, pre_approved=True)
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
...
Subject: Welcome to the "A Test List" mailing list
From: ant-request@example.com
To: Herb Person <herb@example.com>
...
Welcome to the "A Test List" mailing list!
...
Goodbye messages

Similarly, when the member’s unsubscription request is approved, she’ll get a goodbye message.

>>> mlist.send_goodbye_message = True
>>> req_id = hold_unsubscription(mlist, 'herb@example.com')
>>> handle_unsubscription(mlist, req_id, Action.accept)
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
MIME-Version: 1.0
...
Subject: You have been unsubscribed from the A Test List mailing list
From: ant-bounces@example.com
To: herb@example.com
...
Pipelines

Pipelines process messages that have been accepted for posting, applying any modifications and also sending copies of the message to the archives, digests, NNTP, and outgoing queues. Pipelines are named and consist of a sequence of handlers, each of which is applied in turn. Unlike rules and chains, there is no way to stop a pipeline from processing the message once it’s started.

>>> mlist = create_list('test@example.com')
>>> print(mlist.posting_pipeline)
default-posting-pipeline
>>> from mailman.core.pipelines import process

For the purposes of these examples, we’ll enable just one archiver.

>>> from mailman.interfaces.mailinglist import IListArchiverSet
>>> for archiver in IListArchiverSet(mlist).archivers:
...     archiver.is_enabled = (archiver.name == 'mhonarc')
Processing a message

Messages hit the pipeline after they’ve been accepted for posting.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: My first post
... Message-ID: <first>
... X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
...
... First post!
... """)
>>> msgdata = {}
>>> process(mlist, msg, msgdata, mlist.posting_pipeline)

The message has been modified with additional headers, footer decorations, etc.

>>> print(msg.as_string())
From: aperson@example.com
To: test@example.com
Message-ID: <first>
X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
X-Mailman-Version: ...
Precedence: list
Subject: [Test] My first post
List-Id: <test.example.com>
Archived-At: <http://example.com/.../4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB>
List-Archive: <http://example.com/archives/test@example.com>
List-Help: <mailto:test-request@example.com?subject=help>
List-Post: <mailto:test@example.com>
List-Subscribe: <mailto:test-join@example.com>
List-Unsubscribe: <mailto:test-leave@example.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
<BLANKLINE>
First post!
_______________________________________________
Test mailing list -- test@example.com
To unsubscribe send an email to test-leave@example.com
<BLANKLINE>

The message metadata has information about recipients and other stuff. However there are currently no recipients for this message.

>>> dump_msgdata(msgdata)
original_sender : aperson@example.com
original_subject: My first post
recipients      : set()
stripped_subject: My first post

After pipeline processing, the message is now sitting in various other processing queues.

>>> from mailman.testing.helpers import get_queue_messages
>>> messages = get_queue_messages('archive')
>>> len(messages)
1

>>> print(messages[0].msg.as_string())
From: aperson@example.com
To: test@example.com
Message-ID: <first>
X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
X-Mailman-Version: ...
Precedence: list
Subject: [Test] My first post
List-Id: <test.example.com>
...

First post!


>>> dump_msgdata(messages[0].msgdata)
_parsemsg       : False
original_sender : aperson@example.com
original_subject: My first post
recipients      : set()
stripped_subject: My first post
version         : 3

This mailing list is not linked to an NNTP newsgroup, so there’s nothing in the outgoing nntp queue.

>>> messages = get_queue_messages('nntp')
>>> len(messages)
0

The outgoing queue will hold the copy of the message that will actually get delivered to end recipients.

>>> messages = get_queue_messages('out')
>>> len(messages)
1

>>> print(messages[0].msg.as_string())
From: aperson@example.com
To: test@example.com
Message-ID: <first>
X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
X-Mailman-Version: ...
Precedence: list
Subject: [Test] My first post
List-Id: <test.example.com>
...

First post!

_______________________________________________
Test mailing list -- test@example.com
To unsubscribe send an email to test-leave@example.com

>>> dump_msgdata(messages[0].msgdata)
_parsemsg       : False
listid          : test.example.com
original_sender : aperson@example.com
original_subject: My first post
recipients      : set()
stripped_subject: My first post
version         : 3

There’s now one message in the digest mailbox, getting ready to be sent.

>>> from mailman.testing.helpers import digest_mbox
>>> digest = digest_mbox(mlist)
>>> sum(1 for mboxmsg in digest)
1

>>> print(list(digest)[0].as_string())
From: aperson@example.com
To: test@example.com
Message-ID: <first>
X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
X-Mailman-Version: ...
Precedence: list
Subject: [Test] My first post
List-Id: <test.example.com>
...

First post!
System versions

Mailman system information is available through the system object, which implements the ISystem interface.

>>> from mailman.interfaces.system import ISystem
>>> from mailman.core.system import system
>>> from zope.interface.verify import verifyObject

>>> verifyObject(ISystem, system)
True

The Mailman version is also available via the system object.

>>> print(system.mailman_version)
GNU Mailman ...

The Python version running underneath is also available via the system object.

# The entire python_version string is variable, so this is the best test
# we can do.
>>> import sys
>>> system.python_version == sys.version
True

List styles

List styles are a way to name and apply a template of attribute settings to new mailing lists. Every style has a name, which must be unique.

Styles are generally only applied when a mailing list is created, although there is no reason why styles can’t be applied to an existing mailing list. However, when a style changes, the mailing lists using that style are not automatically updated. Instead, think of styles as the initial set of defaults for just about any mailing list attribute. In fact, application of a style to a mailing list can really modify the mailing list in any way.

To start with, there are a few legacy styles.

>>> from zope.component import getUtility
>>> from mailman.interfaces.styles import IStyleManager
>>> manager = getUtility(IStyleManager)
>>> for style in manager.styles:
...     print(style.name)
legacy-announce
legacy-default

When you create a mailing list through the low-level IListManager API, no style is applied.

>>> from mailman.interfaces.listmanager import IListManager
>>> mlist = getUtility(IListManager).create('ant@example.com')
>>> print(mlist.display_name)
None

The legacy default style sets the list’s display name.

>>> manager.get('legacy-default').apply(mlist)
>>> print(mlist.display_name)
Ant
Registering styles

New styles must implement the IStyle interface.

>>> from zope.interface import implementer
>>> from mailman.interfaces.styles import IStyle
>>> @implementer(IStyle)
... class TestStyle:
...     name = 'a-test-style'
...     def apply(self, mailing_list):
...         # Just does something very simple.
...         mailing_list.display_name = 'TEST STYLE LIST'

You can register a new style with the style manager.

>>> manager.register(TestStyle())

All registered styles are returned in alphabetical order by style name.

>>> for style in manager.styles:
...     print(style.name)
a-test-style
legacy-announce
legacy-default

You can also ask the style manager for the style, by name.

>>> test_style = manager.get('a-test-style')
>>> print(test_style.name)
a-test-style
Unregistering styles

You can unregister a style, making it unavailable in the future.

>>> manager.unregister(test_style)
>>> for style in manager.styles:
...     print(style.name)
legacy-announce
legacy-default

Asking for a missing style returns None.

>>> print(manager.get('a-test-style'))
None
Apply styles at list creation

You can specify a style to apply when creating a list through the high-level API. Let’s start by registering the test style.

>>> manager.register(test_style)

Now, when we use the high level API, we can ask for the style to be applied.

>>> from mailman.app.lifecycle import create_list
>>> mlist = create_list('bee@example.com', style_name=test_style.name)

The style has been applied.

>>> print(mlist.display_name)
TEST STYLE LIST

If no style name is provided when creating the list, the system default style (taken from the configuration file) is applied.

>>> @implementer(IStyle)
... class AnotherStyle:
...     name = 'another-style'
...     def apply(self, mailing_list):
...         # Just does something very simple.
...         mailing_list.display_name = 'ANOTHER STYLE LIST'
>>> another_style = AnotherStyle()

We’ll set up the system default to apply this newly registered style if no other style is explicitly given.

>>> from mailman.testing.helpers import configuration
>>> with configuration('styles', default=another_style.name):
...     manager.register(another_style)
...     mlist = create_list('cat@example.com')
>>> print(mlist.display_name)
ANOTHER STYLE LIST

Archivers

Mailman supports pluggable archivers, and it comes with several default archivers.

>>> mlist = create_list('test@example.com')
>>> msg = message_from_string("""\
... From: aperson@example.org
... To: test@example.com
... Subject: An archived message
... Message-ID: <12345>
... X-Message-ID-Hash: RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE
...
... Here is an archived message.
... """)

Archivers support an interface which provides the RFC 2369 List-Archive: header, and one that provides a permalink to the specific message object in the archive. This latter is appropriate for the message footer or for the RFC 5064 Archived-At: header.

If the archiver is not network-accessible, it will return None and the headers will not be added.

Mailman defines a draft spec for how list servers and archivers can interoperate.

>>> archivers = {}
>>> from operator import attrgetter
>>> for archiver in sorted(config.archivers, key=attrgetter('name')):
...     print(archiver.name)
...     print('   ', archiver.list_url(mlist))
...     print('   ', archiver.permalink(mlist, msg))
...     archivers[archiver.name] = archiver
mail-archive
    http://go.mail-archive.dev/test%40example.com
    http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE
mhonarc
    http://example.com/.../test@example.com
    http://example.com/.../RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE
prototype
    None
    None
Sending the message to the archiver

The prototype archiver archives messages to a maildir.

>>> import os
>>> archivers['prototype'].archive_message(mlist, msg)
>>> archive_path = os.path.join(
...     config.ARCHIVE_DIR, 'prototype', mlist.fqdn_listname, 'new')
>>> len(os.listdir(archive_path))
1
The Mail-Archive.com

The Mail Archive is a public archiver that can be used to archive message for free. Mailman comes with a plugin for this archiver; by enabling it messages to public lists will get sent there automatically.

>>> archiver = archivers['mail-archive']
>>> print(archiver.list_url(mlist))
http://go.mail-archive.dev/test%40example.com
>>> print(archiver.permalink(mlist, msg))
http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE

To archive the message, the archiver actually mails the message to a special address at The Mail Archive. The message gets no header or footer decoration.

>>> from mailman.interfaces.archiver import ArchivePolicy
>>> mlist.archive_policy = ArchivePolicy.public
>>> archiver.archive_message(mlist, msg)

>>> from mailman.runners.outgoing import OutgoingRunner
>>> from mailman.testing.helpers import make_testable_runner
>>> outgoing = make_testable_runner(OutgoingRunner, 'out')
>>> outgoing.run()

>>> from operator import itemgetter
>>> messages = list(smtpd.messages)
>>> len(messages)
1

>>> print(messages[0].as_string())
From: aperson@example.org
To: test@example.com
Subject: An archived message
Message-ID: <12345>
X-Message-ID-Hash: RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: archive@mail-archive.dev

Here is an archived message.

>>> smtpd.clear()

However, if the mailing list is not public, the message will never be archived at this service.

>>> mlist.archive_policy = ArchivePolicy.private
>>> print(archiver.list_url(mlist))
None
>>> print(archiver.permalink(mlist, msg))
None
>>> archiver.archive_message(mlist, msg)
>>> list(smtpd.messages)
[]

Additionally, this archiver can handle malformed Message-IDs.

>>> from mailman.utilities.email import add_message_hash
>>> mlist.archive_policy = ArchivePolicy.public
>>> del msg['message-id']
>>> del msg['x-message-id-hash']
>>> msg['Message-ID'] = '12345>'
>>> add_message_hash(msg)
'YJIGBYRWZFG5LZEBQ7NR25B5HBR2BVD6'
>>> print(archiver.permalink(mlist, msg))
http://go.mail-archive.dev/YJIGBYRWZFG5LZEBQ7NR25B5HBR2BVD6

>>> del msg['message-id']
>>> del msg['x-message-id-hash']
>>> msg['Message-ID'] = '<12345'
>>> add_message_hash(msg)
'XUFFJNJ2P2WC4NDPQRZFDJMV24POP64B'
>>> print(archiver.permalink(mlist, msg))
http://go.mail-archive.dev/XUFFJNJ2P2WC4NDPQRZFDJMV24POP64B

>>> del msg['message-id']
>>> del msg['x-message-id-hash']
>>> msg['Message-ID'] = '12345'
>>> add_message_hash(msg)
'RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE'
>>> print(archiver.permalink(mlist, msg))
http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE

>>> del msg['message-id']
>>> del msg['x-message-id-hash']
>>> add_message_hash(msg)
>>> msg['Message-ID'] = '    12345    '
>>> add_message_hash(msg)
'RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE'
>>> print(archiver.permalink(mlist, msg))
http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE
MHonArc

A MHonArc archiver is also available.

>>> archiver = archivers['mhonarc']
>>> print(archiver.name)
mhonarc

Messages sent to a local MHonArc instance are added to its archive via a subprocess call.

>>> from mailman.testing.helpers import LogFileMark
>>> mark = LogFileMark('mailman.archiver')
>>> archiver.archive_message(mlist, msg)
>>> print('LOG:', mark.readline())
LOG: ... /usr/bin/mhonarc
     -add
     -dbfile .../test@example.com.mbox/mhonarc.db
     -outdir .../mhonarc/test@example.com
     -stderr .../logs/mhonarc
     -stdout .../logs/mhonarc -spammode -umask 022

Mail Transport Agents

SMTP authentication

The SMTP server may require authentication. Mailman supports setting the SMTP user name and password. The actual authentication mechanism used is determined by Python’s smtplib module, which tries the more secure CRAM-MD5 method first, followed by the less secure mechanisms PLAIN and LOGIN.

When sending authentication data between Mailman and the MTA over an unsecured network, the submission (mail) server should offer CRAM-MD5 as mechanism to have Python’s smtplib module automatically choose the more secure mechanism.

When the user name and password match what’s expected by the server, everything is a-okay.

>>> mlist = create_list('test@example.com')

By default there is no user name and password, but this matches what’s expected by the test server.

>>> config.push('auth', """
... [mta]
... smtp_user: testuser
... smtp_pass: testpass
... """)

Attempting delivery first must authorize with the mail server.

>>> from mailman.mta.bulk import BulkDelivery
>>> bulk = BulkDelivery()

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: My first post
... Message-ID: <first>
...
... First post!
... """)

>>> bulk.deliver(mlist, msg, dict(recipients=['bperson@example.com']))
{}

>>> print(smtpd.get_authentication_credentials())
AHRlc3R1c2VyAHRlc3RwYXNz
>>> config.pop('auth')

But if the user name and password does not match, the connection will fail.

>>> config.push('auth', """
... [mta]
... smtp_user: baduser
... smtp_pass: badpass
... """)

>>> bulk = BulkDelivery()
>>> response = bulk.deliver(
...     mlist, msg, dict(recipients=['bperson@example.com']))
>>> dump_msgdata(response)
bperson@example.com: (571, b'Bad authentication')

>>> config.pop('auth')
Standard bulk delivery

Mailman has several built in strategies for completing the actual delivery of messages to the immediate upstream mail transport agent, which completes the actual final delivery to recipients.

Bulk delivery attempts to deliver as few copies of the identical message as possible to as many recipients as possible. By grouping recipients this way, bandwidth between Mailman and the MTA, and consequently between the MTA and remote mail servers, can be greatly reduced. The downside is the messages cannot be personalized. See verp.txt for an alternative strategy.

>>> from mailman.mta.bulk import BulkDelivery

The standard bulk deliverer takes as an argument the maximum number of recipients per session. The default is to deliver the message in one chunk, containing all recipients.

>>> bulk = BulkDelivery()

Delivery strategies must implement the proper interface.

>>> from mailman.interfaces.mta import IMailTransportAgentDelivery
>>> from zope.interface.verify import verifyObject
>>> verifyObject(IMailTransportAgentDelivery, bulk)
True
Chunking recipients

The set of final recipients is contained in the recipients key in the message metadata. When max_recipients is specified as zero, then the bulk deliverer puts all recipients into one big chunk.

>>> from string import ascii_letters
>>> recipients = set(letter + 'person@example.com'
...                  for letter in ascii_letters)

>>> chunks = list(bulk.chunkify(recipients))
>>> len(chunks)
1
>>> len(chunks[0])
52

Let say the maximum number of recipients allowed is 4, then no chunk will have more than 4 recipients, though they can have fewer (but still not zero).

>>> bulk = BulkDelivery(4)
>>> chunks = list(bulk.chunkify(recipients))
>>> len(chunks)
13
>>> all(0 < len(chunk) <= 4 for chunk in chunks)
True

The chunking algorithm sorts recipients by top level domain by length.

>>> recipients = set([
...     'anne@example.com',
...     'bart@example.org',
...     'cate@example.net',
...     'dave@example.com',
...     'elle@example.org',
...     'fred@example.net',
...     'gwen@example.com',
...     'herb@example.us',
...     'ione@example.net',
...     'john@example.com',
...     'kate@example.com',
...     'liam@example.ca',
...     'mary@example.us',
...     'neil@example.net',
...     'ocho@example.org',
...     'paco@example.xx',
...     'quaq@example.zz',
...     ])

>>> bulk = BulkDelivery(4)
>>> chunks = list(bulk.chunkify(recipients))
>>> len(chunks)
6

We can’t make any guarantees about sorting within each chunk, but we can tell a few things. For example, the first two chunks will be composed of .net (4) and .org (3) domains (for a total of 7).

>>> len(chunks[0])
4
>>> len(chunks[1])
3

>>> for address in sorted(chunks[0].union(chunks[1])):
...     print(address)
bart@example.org
cate@example.net
elle@example.org
fred@example.net
ione@example.net
neil@example.net
ocho@example.org

We also know that the next two chunks will contain .com (5) addresses.

>>> len(chunks[2])
4
>>> len(chunks[3])
1

>>> for address in sorted(chunks[2].union(chunks[3])):
...     print(address)
anne@example.com
dave@example.com
gwen@example.com
john@example.com
kate@example.com

The next chunk will contain the .us (2) and .ca (1) domains.

>>> len(chunks[4])
3
>>> for address in sorted(chunks[4]):
...     print(address)
herb@example.us
liam@example.ca
mary@example.us

The final chunk will contain the outliers, .xx (1) and .zz (2).

>>> len(chunks[5])
2
>>> for address in sorted(chunks[5]):
...     print(address)
paco@example.xx
quaq@example.zz
Bulk delivery

The set of recipients for bulk delivery comes from the message metadata. If there are no calculated recipients, nothing gets sent.

>>> mlist = create_list('test@example.com')
>>> msg = message_from_string("""\
... From: aperson@example.org
... To: test@example.com
... Subject: test one
... Message-ID: <aardvark>
...
... This is a test.
... """)

>>> bulk = BulkDelivery()
>>> bulk.deliver(mlist, msg, {})
{}
>>> len(list(smtpd.messages))
0

>>> bulk.deliver(mlist, msg, dict(recipients=set()))
{}
>>> len(list(smtpd.messages))
0

With bulk delivery and no maximum number of recipients, there will be just one message sent, with all the recipients packed into the envelope recipients (i.e. RCTP TO).

>>> recipients = set('person_{0:02d}'.format(i) for i in range(100))
>>> msgdata = dict(recipients=recipients)
>>> bulk.deliver(mlist, msg, msgdata)
{}

>>> messages = list(smtpd.messages)
>>> len(messages)
1
>>> print(messages[0].as_string())
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
...
X-RcptTo: person_...
    person_...
...

This is a test.

The X-RcptTo: header contains the set of recipients, in sorted order.

>>> len(messages[0]['x-rcptto'].split(','))
100

When the maximum number of recipients is set to 20, 5 messages will be sent, each with 20 addresses in the RCPT TO.

>>> bulk = BulkDelivery(20)
>>> bulk.deliver(mlist, msg, msgdata)
{}

>>> messages = list(smtpd.messages)
>>> len(messages)
5
>>> for message in messages:
...     x_rcptto = message['x-rcptto']
...     print('Number of recipients:', len(x_rcptto.split(',')))
Number of recipients: 20
Number of recipients: 20
Number of recipients: 20
Number of recipients: 20
Number of recipients: 20
Delivery headers

The sending agent shows up in the RFC 5321 MAIL FROM, which shows up in the X-MailFrom: header in the sample message.

The bulk delivery module calculates the sending agent address first from the message metadata…

>>> bulk = BulkDelivery()
>>> recipients = set(['aperson@example.com'])
>>> msgdata = dict(recipients=recipients,
...                sender='asender@example.org')
>>> bulk.deliver(mlist, msg, msgdata)
{}

>>> message = list(smtpd.messages)[0]
>>> print(message.as_string())
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: asender@example.org
X-RcptTo: aperson@example.com

This is a test.

…followed by the mailing list’s bounces robot address…

>>> del msgdata['sender']
>>> bulk.deliver(mlist, msg, msgdata)
{}

>>> message = list(smtpd.messages)[0]
>>> print(message.as_string())
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: aperson@example.com

This is a test.

…and finally the site owner, if there is no mailing list target for this message.

>>> config.push('site-owner', """\
... [mailman]
... site_owner: site-owner@example.com
... """)

>>> bulk.deliver(None, msg, msgdata)
{}

>>> message = list(smtpd.messages)[0]
>>> print(message.as_string())
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: site-owner@example.com
X-RcptTo: aperson@example.com

This is a test.

# Remove test configuration.
>>> config.pop('site-owner')
Delivery failures

Mailman does not do final delivery. Instead, it sends mail through a site local mail server which manages queuing and final delivery. However, even this local mail server can produce delivery failures visible to Mailman in certain situations.

For example, there could be a problem delivering to any of the specified recipients.

# Tell the mail server to fail on the next 3 RCPT TO commands, one for
# each recipient in the following message.
>>> smtpd.err_queue.put(('rcpt', 500))
>>> smtpd.err_queue.put(('rcpt', 500))
>>> smtpd.err_queue.put(('rcpt', 500))

>>> recipients = set([
...     'aperson@example.org',
...     'bperson@example.org',
...     'cperson@example.org',
...     ])
>>> msgdata = dict(recipients=recipients)

>>> msg = message_from_string("""\
... From: aperson@example.org
... To: test@example.com
... Subject: test three
... Message-ID: <camel>
...
... This is a test.
... """)

>>> failures = bulk.deliver(mlist, msg, msgdata)
>>> for address in sorted(failures):
...     print(address, failures[address][0],
...                    failures[address][1].decode('ascii'))
aperson@example.org 500 Error: SMTPRecipientsRefused
bperson@example.org 500 Error: SMTPRecipientsRefused
cperson@example.org 500 Error: SMTPRecipientsRefused

>>> messages = list(smtpd.messages)
>>> len(messages)
0

Or there could be some other problem causing an SMTP response failure.

# Tell the mail server to register a temporary failure on the next MAIL
# FROM command.
>>> smtpd.err_queue.put(('mail', 450))

>>> failures = bulk.deliver(mlist, msg, msgdata)
>>> for address in sorted(failures):
...     print(address, failures[address][0],
...                    failures[address][1].decode('ascii'))
aperson@example.org 450 Error: SMTPResponseException
bperson@example.org 450 Error: SMTPResponseException
cperson@example.org 450 Error: SMTPResponseException

# Tell the mail server to register a permanent failure on the next MAIL
# FROM command.
>>> smtpd.err_queue.put(('mail', 500))

>>> failures = bulk.deliver(mlist, msg, msgdata)
>>> for address in sorted(failures):
...     print(address, failures[address][0],
...                    failures[address][1].decode('ascii'))
aperson@example.org 500 Error: SMTPResponseException
bperson@example.org 500 Error: SMTPResponseException
cperson@example.org 500 Error: SMTPResponseException

XXX Untested: socket.error, IOError, smtplib.SMTPException.

MTA connections

Outgoing connections to the outgoing mail transport agent (MTA) are mitigated through a Connection class, which can transparently manage multiple sessions in a single connection.

>>> from mailman.mta.connection import Connection

The number of sessions per connections is specified when the Connection object is created, as is the host and port number of the SMTP server. Zero means there’s an unlimited number of sessions per connection.

>>> connection = Connection(
...     config.mta.smtp_host, int(config.mta.smtp_port), 0)

At the start, there have been no connections to the server.

>>> smtpd.get_connection_count()
0

By sending a message to the server, a connection is opened.

>>> connection.sendmail('anne@example.com', ['bart@example.com'], """\
... From: anne@example.com
... To: bart@example.com
... Subject: aardvarks
...
... """)
{}

>>> smtpd.get_connection_count()
1

We can reset the connection count back to zero.

>>> from smtplib import SMTP
>>> def reset():
...     smtpd = SMTP()
...     smtpd.connect(config.mta.smtp_host, int(config.mta.smtp_port))
...     smtpd.docmd('RSET')

>>> reset()
>>> smtpd.get_connection_count()
0

>>> connection.quit()

By providing an SMTP user name and password in the configuration file, Mailman will authenticate with the mail server after each new connection.

>>> config.push('auth', """
... [mta]
... smtp_user: testuser
... smtp_pass: testpass
... """)

>>> connection = Connection(
...     config.mta.smtp_host, int(config.mta.smtp_port), 0,
...     config.mta.smtp_user, config.mta.smtp_pass)
>>> connection.sendmail('anne@example.com', ['bart@example.com'], """\
... From: anne@example.com
... To: bart@example.com
... Subject: aardvarks
...
... """)
{}
>>> print(smtpd.get_authentication_credentials())
AHRlc3R1c2VyAHRlc3RwYXNz

>>> reset()
>>> config.pop('auth')
Sessions per connection

Let’s say we specify a maximum number of sessions per connection of 2. When the third message is sent, the connection is torn down and a new one is created.

The connection count starts at zero.

>>> connection = Connection(
...     config.mta.smtp_host, int(config.mta.smtp_port), 2)

>>> smtpd.get_connection_count()
0

We send two messages through the Connection object. Only one connection is opened.

>>> connection.sendmail('anne@example.com', ['bart@example.com'], """\
... From: anne@example.com
... To: bart@example.com
... Subject: aardvarks
...
... """)
{}

>>> smtpd.get_connection_count()
1

>>> connection.sendmail('anne@example.com', ['bart@example.com'], """\
... From: anne@example.com
... To: bart@example.com
... Subject: aardvarks
...
... """)
{}

>>> smtpd.get_connection_count()
1

The third message would cause a third session, exceeding the maximum. So the current connection is closed and a new one opened.

>>> connection.sendmail('anne@example.com', ['bart@example.com'], """\
... From: anne@example.com
... To: bart@example.com
... Subject: aardvarks
...
... """)
{}

>>> smtpd.get_connection_count()
2

A fourth message does not cause a new connection to be made.

>>> connection.sendmail('anne@example.com', ['bart@example.com'], """\
... From: anne@example.com
... To: bart@example.com
... Subject: aardvarks
...
... """)
{}

>>> smtpd.get_connection_count()
2

But a fifth one does.

>>> connection.sendmail('anne@example.com', ['bart@example.com'], """\
... From: anne@example.com
... To: bart@example.com
... Subject: aardvarks
...
... """)
{}

>>> smtpd.get_connection_count()
3
No maximum

A value of zero means that there is an unlimited number of sessions per connection.

>>> connection = Connection(
...     config.mta.smtp_host, int(config.mta.smtp_port), 0)
>>> reset()

Even after ten messages are sent, there’s still been only one connection to the server.

>>> connection.debug = True
>>> for i in range(10):
...     # Ignore the results.
...     results = connection.sendmail(
...         'anne@example.com', ['bart@example.com'], """\
... From: anne@example.com
... To: bart@example.com
... Subject: aardvarks
...
... """)

>>> smtpd.get_connection_count()
1
Development mode

By putting Mailman into development mode, you can force the recipients to a given hard-coded address. This allows you to test Mailman without worrying about accidental deliveries to unintended recipients.

>>> config.push('devmode', """
... [devmode]
... enabled: yes
... recipient: zperson@example.com
... """)

>>> smtpd.clear()
>>> connection.sendmail(
...     'anne@example.com',
...     ['bart@example.com', 'cate@example.com'], """\
... From: anne@example.com
... To: bart@example.com
... Subject: aardvarks
...
... """)
{}

>>> messages = list(smtpd.messages)
>>> len(messages)
1
>>> print(messages[0].as_string())
From: anne@example.com
To: bart@example.com
Subject: aardvarks
X-Peer: ...
X-MailFrom: anne@example.com
X-RcptTo: zperson@example.com, zperson@example.com



>>> config.pop('devmode')
Personalized decoration

Personalized messages can be decorated by headers and footers containing information specific to the recipient.

>>> from mailman.mta.decorating import DecoratingDelivery
>>> decorating = DecoratingDelivery()

Delivery strategies must implement the proper interface.

>>> from mailman.interfaces.mta import IMailTransportAgentDelivery
>>> from zope.interface.verify import verifyObject
>>> verifyObject(IMailTransportAgentDelivery, decorating)
True
Decorations

Decorations are added when the mailing list had a header and/or footer defined, and the decoration handler is told to do personalized decorations. We start by writing the site-global header and footer template.

>>> import os, tempfile
>>> template_dir = tempfile.mkdtemp()
>>> site_dir = os.path.join(template_dir, 'site', 'en')
>>> os.makedirs(site_dir)
>>> config.push('templates', """
... [paths.testing]
... template_dir: {}
... """.format(template_dir))

>>> myheader_path = os.path.join(site_dir, 'myheader.txt')
>>> with open(myheader_path, 'w') as fp:
...     print("""\
... Delivery address: $user_address
... Subscribed address: $user_delivered_to
... """, file=fp)
>>> myfooter_path = os.path.join(site_dir, 'myfooter.txt')
>>> with open(myfooter_path, 'w') as fp:
...     print("""\
... User name: $user_name
... Language: $user_language
... """, file=fp)

Then create a mailing list which will use this header and footer. Because these are site-global templates, we can use a shorted URL.

>>> mlist = create_list('test@example.com')
>>> from mailman.interfaces.template import ITemplateManager
>>> from zope.component import getUtility
>>> manager = getUtility(ITemplateManager)
>>> manager.set('list:member:regular:header', mlist.list_id,
...             'mailman:///myheader.txt')
>>> manager.set('list:member:regular:footer', mlist.list_id,
...             'mailman:///myfooter.txt')
>>> transaction.commit()

>>> msg = message_from_string("""\
... From: aperson@example.org
... To: test@example.com
... Subject: test one
... Message-ID: <aardvark>
...
... This is a test.
... """)

>>> recipients = {
...     'aperson@example.com',
...     'bperson@example.com',
...     'cperson@example.com',
...     }

>>> msgdata = dict(
...     recipients=recipients,
...     personalize=True,
...     )

More information is included when the recipient is a member of the mailing list.

>>> from mailman.interfaces.member import MemberRole
>>> from mailman.interfaces.usermanager import IUserManager
>>> user_manager = getUtility(IUserManager)

>>> anne = user_manager.create_user('aperson@example.com', 'Anne Person')
>>> mlist.subscribe(list(anne.addresses)[0], MemberRole.member)
<Member: Anne Person <aperson@example.com> ...

>>> bart = user_manager.create_user('bperson@example.com', 'Bart Person')
>>> mlist.subscribe(list(bart.addresses)[0], MemberRole.member)
<Member: Bart Person <bperson@example.com> ...

>>> cris = user_manager.create_user('cperson@example.com', 'Cris Person')
>>> mlist.subscribe(list(cris.addresses)[0], MemberRole.member)
<Member: Cris Person <cperson@example.com> ...

The decorations happen when the message is delivered.

>>> decorating.deliver(mlist, msg, msgdata)
{}
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> from operator import itemgetter
>>> for message in sorted(messages, key=itemgetter('x-rcptto')):
...     print(message.as_string())
...     print('----------')
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: aperson@example.com

Delivery address: aperson@example.com
Subscribed address: aperson@example.com
This is a test.
User name: Anne Person
Language: English (USA)
----------
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: bperson@example.com

Delivery address: bperson@example.com
Subscribed address: bperson@example.com
This is a test.
User name: Bart Person
Language: English (USA)
----------
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: cperson@example.com

Delivery address: cperson@example.com
Subscribed address: cperson@example.com
This is a test.
User name: Cris Person
Language: English (USA)
----------
Decorate only once

Do not decorate a message twice. Decorators must insert the decorated key into the message metadata.

>>> msgdata['nodecorate'] = True
>>> decorating.deliver(mlist, msg, msgdata)
{}
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> for message in sorted(messages, key=itemgetter('x-rcptto')):
...     print(message.as_string())
...     print('----------')
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: aperson@example.com

This is a test.
----------
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: bperson@example.com

This is a test.
----------
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: cperson@example.com

This is a test.
----------
Fully personalized delivery

Fully personalized mail delivery is an enhancement over VERP delivery where the To: field of the message is replaced with the recipient’s address. A typical email message is sent to the mailing list’s posting address and copied to the list membership that way. Some people like the more personal address.

Personalized delivery still does VERP.

>>> from mailman.mta.personalized import PersonalizedDelivery
>>> personalized = PersonalizedDelivery()

Delivery strategies must implement the proper interface.

>>> from mailman.interfaces.mta import IMailTransportAgentDelivery
>>> from zope.interface.verify import verifyObject
>>> verifyObject(IMailTransportAgentDelivery, personalized)
True
No personalization

By default, the To: header is not personalized.

>>> mlist = create_list('test@example.com')
>>> msg = message_from_string("""\
... From: aperson@example.org
... To: test@example.com
... Subject: test one
... Message-ID: <aardvark>
...
... This is a test.
... """)

>>> recipients = set([
...     'aperson@example.com',
...     'bperson@example.com',
...     'cperson@example.com',
...     ])

>>> personalized.deliver(mlist, msg, dict(recipients=recipients))
{}
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> from operator import itemgetter
>>> for message in sorted(messages, key=itemgetter('x-rcptto')):
...     print(message.as_string())
...     print('----------')
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: aperson@example.com

This is a test.
----------
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: bperson@example.com

This is a test.
----------
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: cperson@example.com

This is a test.
----------
To header

When the mailing list requests personalization, the To: header is replaced with the recipient’s address and name.

>>> from mailman.interfaces.mailinglist import Personalization
>>> mlist.personalize = Personalization.full
>>> transaction.commit()

>>> personalized.deliver(mlist, msg, dict(recipients=recipients))
{}
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> for message in sorted(messages, key=itemgetter('to')):
...     print(message.as_string())
...     print('----------')
From: aperson@example.org
To: aperson@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: aperson@example.com

This is a test.
----------
From: aperson@example.org
To: bperson@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: bperson@example.com

This is a test.
----------
From: aperson@example.org
To: cperson@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: cperson@example.com

This is a test.
----------

If the recipient is a user registered with Mailman, and the user has an associated real name, then this name also shows up in the To: header.

>>> from zope.component import getUtility
>>> from mailman.interfaces.usermanager import IUserManager
>>> user_manager = getUtility(IUserManager)

>>> bill = user_manager.create_user('bperson@example.com', 'Bill Person')
>>> cate = user_manager.create_user('cperson@example.com', 'Cate Person')
>>> transaction.commit()

>>> personalized.deliver(mlist, msg, dict(recipients=recipients))
{}
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> from operator import itemgetter
>>> for message in sorted(messages, key=itemgetter('x-rcptto')):
...     print(message.as_string())
...     print('----------')
From: aperson@example.org
To: aperson@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: aperson@example.com

This is a test.
----------
From: aperson@example.org
To: Bill Person <bperson@example.com>
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: bperson@example.com

This is a test.
----------
From: aperson@example.org
To: Cate Person <cperson@example.com>
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: cperson@example.com

This is a test.
----------
Standard VERP delivery

Variable Envelope Return Path (VERP) delivery is an alternative to bulk delivery, where an individual message is crafted uniquely for each recipient.

The cost of enabling VERP is that Mailman must send to the upstream MTA, one message per recipient. Under bulk delivery, an exact copy of one message can be sent to many recipients, greatly reducing the bandwidth for delivery.

In Mailman, enabling VERP delivery for bounce detection brings with it a side benefit: the message which must be crafted uniquely for each recipient, can be further personalized to include all kinds of information unique to that recipient. In the simplest case, the message can contain footer information, e.g. pointing the user to their account URL or including a user-specific unsubscription link. In theory, VERP delivery means we can do sophisticated mail merge operations.

Mailman’s use of the term VERP really means message personalization.

>>> from mailman.mta.verp import VERPDelivery
>>> verp = VERPDelivery()

Delivery strategies must implement the proper interface.

>>> from mailman.interfaces.mta import IMailTransportAgentDelivery
>>> from zope.interface.verify import verifyObject
>>> verifyObject(IMailTransportAgentDelivery, verp)
True
No recipients

The message metadata specifies the set of recipients to send this message to. If there are no recipients, there’s nothing to do.

>>> mlist = create_list('test@example.com')
>>> msg = message_from_string("""\
... From: aperson@example.org
... To: test@example.com
... Subject: test one
... Message-ID: <aardvark>
...
... This is a test.
... """)

>>> verp.deliver(mlist, msg, {})
{}
>>> len(list(smtpd.messages))
0

>>> verp.deliver(mlist, msg, dict(recipients=set()))
{}
>>> len(list(smtpd.messages))
0
Individual copy

Each recipient of the message gets an individual, personalized copy of the message, with their email address encoded into the envelope sender. This is so the return path will point back to Mailman but allow for decoding of the intended recipient’s delivery address.

>>> recipients = set([
...     'aperson@example.com',
...     'bperson@example.com',
...     'cperson@example.com',
...     ])

VERPing is only actually done if the metadata requests it.

>>> msgdata = dict(recipients=recipients, verp=True)
>>> verp.deliver(mlist, msg, msgdata)
{}
>>> messages = list(smtpd.messages)
>>> len(messages)
3

>>> from operator import itemgetter
>>> for message in sorted(messages, key=itemgetter('x-rcptto')):
...     print(message.as_string())
...     print('----------')
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces+aperson=example.com@example.com
X-RcptTo: aperson@example.com

This is a test.
----------
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces+bperson=example.com@example.com
X-RcptTo: bperson@example.com

This is a test.
----------
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
X-Peer: ...
X-MailFrom: test-bounces+cperson=example.com@example.com
X-RcptTo: cperson@example.com

This is a test.
----------

The deliverer made a copy of the original message, so it wasn’t changed.

>>> print(msg.as_string())
From: aperson@example.org
To: test@example.com
Subject: test one
Message-ID: <aardvark>
<BLANKLINE>
This is a test.
<BLANKLINE>

Mailman runner control

Mailman has a number of runner subprocesses which perform long-running tasks such as listening on an LMTP port, processing REST API requests, or processing messages in a queue directory. In normal operation, the mailman command is used to start, stop and manage the runners. This is just a wrapper around the real master watcher, which handles runner starting, stopping, exiting, and log file reopening.

>>> from mailman.testing.helpers import TestableMaster

Start the master in a sub-thread.

>>> master = TestableMaster()
>>> master.start()

There should be a process id for every runner that claims to be startable.

>>> from lazr.config import as_boolean
>>> startable_runners = [conf for conf in config.runner_configs
...                      if as_boolean(conf.start)]
>>> len(list(master.runner_pids)) == len(startable_runners)
True

Now verify that all the runners are running.

>>> import os

# This should produce no output.
>>> for pid in master.runner_pids:
...     os.kill(pid, 0)

Stop the master process, which should also kill (and not restart) the child runner processes.

>>> master.stop()

None of the children are running now.

>>> import errno
>>> from contextlib import suppress
>>> for pid in master.runner_pids:
...     with suppress(ProcessLookupError):
...         os.kill(pid, 0)
...         print('Process did not exit:', pid)

Commands

Generating aliases

For some mail servers, Mailman must generate data files that are used to hook Mailman up to the mail server. The details of this differ for each mail server. Generally these files are automatically kept up-to-date when mailing lists are created or removed, but you might occasionally need to manually regenerate the file. The mailman aliases command does this.

>>> class FakeArgs:
...     directory = None
>>> from mailman.commands.cli_aliases import Aliases
>>> command = Aliases()

For example, connecting Mailman to Postfix is generally done through the LMTP protocol. Mailman starts an LMTP server and Postfix delivers messages to Mailman as an LMTP client. By default this is done through Postfix transport maps.

Selecting Postfix as the source of incoming messages enables transport map generation.

>>> config.push('postfix', """
... [mta]
... incoming: mailman.mta.postfix.LMTP
... lmtp_host: lmtp.example.com
... lmtp_port: 24
... """)

Let’s create a mailing list and then display the transport map for it. We’ll write the appropriate files to a temporary directory.

>>> import os, shutil, tempfile
>>> output_directory = tempfile.mkdtemp()
>>> ignore = cleanups.callback(shutil.rmtree, output_directory)

>>> FakeArgs.directory = output_directory
>>> mlist = create_list('test@example.com')
>>> command.process(FakeArgs)

For Postfix, there are two files in the output directory.

>>> files = sorted(os.listdir(output_directory))
>>> for file in files:
...     print(file)
postfix_domains
postfix_lmtp

The transport map file contains all the aliases for the mailing list.

>>> with open(os.path.join(output_directory, 'postfix_lmtp')) as fp:
...     print(fp.read())
# AUTOMATICALLY GENERATED BY MAILMAN ON ...
...
test@example.com               lmtp:[lmtp.example.com]:24
test-bounces@example.com       lmtp:[lmtp.example.com]:24
test-confirm@example.com       lmtp:[lmtp.example.com]:24
test-join@example.com          lmtp:[lmtp.example.com]:24
test-leave@example.com         lmtp:[lmtp.example.com]:24
test-owner@example.com         lmtp:[lmtp.example.com]:24
test-request@example.com       lmtp:[lmtp.example.com]:24
test-subscribe@example.com     lmtp:[lmtp.example.com]:24
test-unsubscribe@example.com   lmtp:[lmtp.example.com]:24
<BLANKLINE>

The relay domains file contains a list of all the domains.

>>> with open(os.path.join(output_directory, 'postfix_domains')) as fp:
...     print(fp.read())
# AUTOMATICALLY GENERATED BY MAILMAN ON ...
...
example.com example.com
Display configuration values

Just like the Postfix command postconf(1), the mailman conf command lets you dump one or more Mailman configuration variables to standard output or a file.

Mailman’s configuration is divided in multiple sections which contain multiple key-value pairs. The mailman conf command allows you to display a specific key-value pair, or several key-value pairs.

>>> class FakeArgs:
...     key = None
...     section = None
...     output = None
>>> from mailman.commands.cli_conf import Conf
>>> command = Conf()

To get a list of all key-value pairs of any section, you need to call the command without any options.

>>> command.process(FakeArgs)
[antispam] header_checks:
...
[logging.bounce] level: info
...
[mailman] site_owner: noreply@example.com
...

You can list all the key-value pairs of a specific section.

>>> FakeArgs.section = 'shell'
>>> command.process(FakeArgs)
[shell] banner: Welcome to the GNU Mailman shell
[shell] history_file:
[shell] prompt: >>>
[shell] use_ipython: no

You can also pass a key and display all key-value pairs matching the given key, along with the names of the corresponding sections.

>>> FakeArgs.section = None
>>> FakeArgs.key = 'path'
>>> command.process(FakeArgs)
[logging.archiver] path: mailman.log
[logging.bounce] path: bounce.log
[logging.config] path: mailman.log
[logging.database] path: mailman.log
[logging.debug] path: debug.log
[logging.error] path: mailman.log
[logging.fromusenet] path: mailman.log
[logging.http] path: mailman.log
[logging.locks] path: mailman.log
[logging.mischief] path: mailman.log
[logging.root] path: mailman.log
[logging.runner] path: mailman.log
[logging.smtp] path: smtp.log
[logging.subscribe] path: mailman.log
[logging.vette] path: mailman.log

If you specify both a section and a key, you will get the corresponding value.

>>> FakeArgs.section = 'mailman'
>>> FakeArgs.key = 'site_owner'
>>> command.process(FakeArgs)
noreply@example.com
Starting and stopping Mailman

The Mailman daemon processes can be started and stopped from the command line.

Set up

All we care about is the master process; normally it starts a bunch of runners, but we don’t care about any of them, so write a test configuration file for the master that disables all the runners.

>>> from mailman.commands.tests.test_control import make_config
Starting
>>> from mailman.commands.cli_control import Start
>>> start = Start()

>>> class FakeArgs:
...     force = False
...     run_as_user = True
...     quiet = False
...     config = make_config()
>>> args = FakeArgs()

Starting the daemons prints a useful message and starts the master watcher process in the background.

>>> start.process(args)
Starting Mailman's master runner

>>> from mailman.commands.tests.test_control import find_master

The process exists, and its pid is available in a run time file.

>>> pid = find_master()
>>> pid is not None
True
Stopping

You can also stop the master watcher process from the command line, which stops all the child processes too.

>>> from mailman.commands.cli_control import Stop
>>> stop = Stop()
>>> stop.process(args)
Shutting down Mailman's master runner

>>> from datetime import datetime, timedelta
>>> import os
>>> import time
>>> import errno
>>> def bury_master():
...     until = timedelta(seconds=2) + datetime.now()
...     while datetime.now() < until:
...         time.sleep(0.1)
...         try:
...             os.kill(pid, 0)
...             os.waitpid(pid, os.WNOHANG)
...         except OSError as error:
...             if error.errno == errno.ESRCH:
...                 # The process has exited.
...                 print('Master process went bye bye')
...                 return
...             else:
...                 raise
...     else:
...         raise AssertionError('Master process lingered')

>>> bury_master()
Master process went bye bye

XXX We need tests for restart (SIGUSR1) and reopen (SIGHUP).

Command line list creation

A system administrator can create mailing lists by the command line.

>>> class FakeArgs:
...     language = None
...     owners = []
...     quiet = False
...     domain = True
...     listname = None
...     notify = False

You cannot create a mailing list in an unknown domain.

>>> from mailman.commands.cli_lists import Create
>>> command = Create()

>>> class FakeParser:
...     def error(self, message):
...         print(message)
>>> command.parser = FakeParser()

>>> FakeArgs.domain = False
>>> FakeArgs.listname = ['test@example.xx']
>>> command.process(FakeArgs)
Undefined domain: example.xx

By default, Mailman will create the domain if it doesn’t exist.

>>> FakeArgs.domain = True
>>> command.process(FakeArgs)
Created mailing list: test@example.xx

Now both the domain and the mailing list exist in the database.

>>> from mailman.interfaces.listmanager import IListManager
>>> from zope.component import getUtility
>>> list_manager = getUtility(IListManager)
>>> list_manager.get('test@example.xx')
<mailing list "test@example.xx" at ...>

>>> from mailman.interfaces.domain import IDomainManager
>>> getUtility(IDomainManager).get('example.xx')
<Domain example.xx>

You can prevent the creation of the domain in existing domains by using the -D or --no-domain flag. Although the --no-domain flag is not required when domain already exists it can be used to force an error when domain doesn’t exist.

>>> FakeArgs.domain = False
>>> FakeArgs.listname = ['test1@example.com']
>>> command.process(FakeArgs)
Created mailing list: test1@example.com

>>> list_manager.get('test1@example.com')
<mailing list "test1@example.com" at ...>

The command can also operate quietly.

>>> FakeArgs.quiet = True
>>> FakeArgs.listname = ['test2@example.com']
>>> command.process(FakeArgs)

>>> mlist = list_manager.get('test2@example.com')
>>> mlist
<mailing list "test2@example.com" at ...>
Setting the owner

By default, no list owners are specified.

>>> dump_list(mlist.owners.addresses)
*Empty*

But you can specify an owner address on the command line when you create the mailing list.

>>> FakeArgs.quiet = False
>>> FakeArgs.listname = ['test4@example.com']
>>> FakeArgs.owners = ['foo@example.org']
>>> command.process(FakeArgs)
Created mailing list: test4@example.com

>>> mlist = list_manager.get('test4@example.com')
>>> dump_list(repr(address) for address in mlist.owners.addresses)
<Address: foo@example.org [not verified] at ...>

You can even specify more than one address for the owners.

>>> FakeArgs.owners = ['foo@example.net',
...                    'bar@example.net',
...                    'baz@example.net']
>>> FakeArgs.listname = ['test5@example.com']
>>> command.process(FakeArgs)
Created mailing list: test5@example.com

>>> mlist = list_manager.get('test5@example.com')
>>> from operator import attrgetter
>>> dump_list(repr(address) for address in mlist.owners.addresses)
<Address: bar@example.net [not verified] at ...>
<Address: baz@example.net [not verified] at ...>
<Address: foo@example.net [not verified] at ...>
Setting the language

You can set the default language for the new mailing list when you create it. The language must be known to Mailman.

>>> FakeArgs.listname = ['test3@example.com']
>>> FakeArgs.language = 'ee'
>>> command.process(FakeArgs)
Invalid language code: ee

>>> from mailman.interfaces.languages import ILanguageManager
>>> getUtility(ILanguageManager).add('ee', 'iso-8859-1', 'Freedonian')
<Language [ee] Freedonian>

>>> FakeArgs.quiet = False
>>> FakeArgs.listname = ['test3@example.com']
>>> FakeArgs.language = 'fr'
>>> command.process(FakeArgs)
Created mailing list: test3@example.com

>>> mlist = list_manager.get('test3@example.com')
>>> print(mlist.preferred_language)
<Language [fr] French>
>>> FakeArgs.language = None
Notifications

When told to, Mailman will notify the list owners of their new mailing list.

>>> FakeArgs.listname = ['test6@example.com']
>>> FakeArgs.notify = True
>>> command.process(FakeArgs)
Created mailing list: test6@example.com

The notification message is in the virgin queue.

>>> from mailman.testing.helpers import get_queue_messages
>>> messages = get_queue_messages('virgin')
>>> len(messages)
1

>>> for message in messages:
...     print(message.msg.as_string())
MIME-Version: 1.0
...
Subject: Your new mailing list: test6@example.com
From: noreply@example.com
To: foo@example.net, bar@example.net, baz@example.net
...

The mailing list 'test6@example.com' has just been created for you.
The following is some basic information about your mailing list.

There is an email-based interface for users (not administrators) of
your list; you can get info about using it by sending a message with
just the word 'help' as subject or in the body, to:

    test6-request@example.com

Please address all questions to noreply@example.com.
The ‘echo’ command

The mail command ‘echo’ simply replies with the original command and arguments to the sender.

>>> command = config.commands['echo']
>>> print(command.name)
echo
>>> print(command.argument_description)
[args]
>>> print(command.description)
Echo back your arguments.

The original message is ignored, but the results receive the echoed command.

>>> mlist = create_list('test@example.com')

>>> from mailman.runners.command import Results
>>> results = Results()

>>> from mailman.email.message import Message
>>> print(command.process(mlist, Message(), {}, ('foo', 'bar'), results))
ContinueProcessing.yes
>>> print(str(results))
The results of your email command are provided below.

echo foo bar
The ‘end’ command

The mail command processor recognized an ‘end’ command which tells it to stop processing email messages.

>>> command = config.commands['end']
>>> print(command.name)
end
>>> print(command.description)
Stop processing commands.

The ‘end’ command takes no arguments.

>>> print('DESCRIPTION:', command.argument_description)
DESCRIPTION:

The command itself is fairly simple; it just stops command processing, and the message isn’t even looked at.

>>> mlist = create_list('test@example.com')
>>> from mailman.email.message import Message
>>> print(command.process(mlist, Message(), {}, (), None))
ContinueProcessing.no

The ‘stop’ command is a synonym for ‘end’.

>>> command = config.commands['stop']
>>> print(command.name)
stop
>>> print(command.description)
An alias for 'end'.
>>> print('DESCRIPTION:', command.argument_description)
DESCRIPTION:
>>> print(command.process(mlist, Message(), {}, (), None))
ContinueProcessing.no
Email command help

You can get some help about the various email commands that are available by sending the word help to a mailing list’s -request address.

>>> mlist = create_list('test@example.com')
>>> from mailman.commands.eml_help import Help
>>> help = Help()
>>> print(help.name)
help
>>> print(help.description)
Get help about available email commands.
>>> print(help.argument_description)
[command]

With no arguments, help provides a list of the available commands and a short description of each of them.

>>> from mailman.runners.command import Results
>>> results = Results()

>>> from mailman.email.message import Message
>>> print(help.process(mlist, Message(), {}, (), results))
ContinueProcessing.yes
>>> print(results)
The results of your email command are provided below.

confirm     - Confirm a subscription request.
echo        - Echo back your arguments.
end         - Stop processing commands.
help        - Get help about available email commands.
join        - Join this mailing list.
leave       - Leave this mailing list.
stop        - An alias for 'end'.
subscribe   - An alias for 'join'.
unsubscribe - An alias for 'leave'.

With an argument, you can get more detailed help about a specific command.

>>> results = Results()
>>> print(help.process(mlist, Message(), {}, ('help',), results))
ContinueProcessing.yes
>>> print(results)
The results of your email command are provided below.
<BLANKLINE>
help [command]
Get help about available email commands.
<BLANKLINE>

Some commands have even more detailed help.

>>> results = Results()
>>> print(help.process(mlist, Message(), {}, ('join',), results))
ContinueProcessing.yes
>>> print(results)
The results of your email command are provided below.
<BLANKLINE>
join [digest=<no|mime|plain>]
Join this mailing list.
<BLANKLINE>
You will be asked to confirm your subscription request and you may be
issued a provisional password.
<BLANKLINE>
By using the 'digest' option, you can specify whether you want digest
delivery or not.  If not specified, the mailing list's default delivery
mode will be used.
<BLANKLINE>
Importing list data

If you have the config.pck file for a version 2.1 mailing list, you can import that into an existing mailing list in Mailman 3.0.

>>> from mailman.commands.cli_import import Import21
>>> command = Import21()

>>> class FakeArgs:
...     listname = None
...     pickle_file = None

>>> class FakeParser:
...     def error(self, message):
...         print(message)
>>> command.parser = FakeParser()

You must specify the mailing list you are importing into, and it must exist.

>>> command.process(FakeArgs)
List name is required

>>> FakeArgs.listname = ['import@example.com']
>>> command.process(FakeArgs)
No such list: import@example.com

When the mailing list exists, you must specify a real pickle file to import from.

>>> mlist = create_list('import@example.com')
>>> command.process(FakeArgs)
config.pck file is required

>>> FakeArgs.pickle_file = [__file__]
>>> command.process(FakeArgs)
Not a Mailman 2.1 configuration file: .../import.rst

Now we can import the test pickle file. As a simple illustration of the import, the mailing list’s ‘real name’ has changed.

>>> from pkg_resources import resource_filename
>>> FakeArgs.pickle_file = [
...     resource_filename('mailman.testing', 'config.pck')]

>>> print(mlist.display_name)
Import

>>> command.process(FakeArgs)
>>> print(mlist.display_name)
Test
Getting information

You can get information about Mailman’s environment by using the command line script mailman info. By default, the info is printed to standard output.

>>> from mailman.commands.cli_info import Info
>>> command = Info()

>>> class FakeArgs:
...     output = None
...     verbose = None
>>> args = FakeArgs()

>>> command.process(args)
GNU Mailman 3...
Python ...
...
config file: .../test.cfg
db url: ...
REST root url: http://localhost:9001/3.1/
REST credentials: restadmin:restpass

By passing in the -o/--output option, you can print the info to a file.

>>> from mailman.config import config
>>> import os
>>> output_path = os.path.join(config.VAR_DIR, 'output.txt')
>>> args.output = output_path
>>> command.process(args)
>>> with open(output_path) as fp:
...     print(fp.read())
GNU Mailman 3...
Python ...
...
config file: .../test.cfg
db url: ...
devmode: DISABLED
REST root url: http://localhost:9001/3.1/
REST credentials: restadmin:restpass

You can also get more verbose information, which contains a list of the file system paths that Mailman is using.

>>> args.output = None
>>> args.verbose = True
>>> config.create_paths = False
>>> config.push('fhs', """
... [mailman]
... layout: fhs
... """)
>>> ignore = cleanups.callback(config.pop, 'fhs')
>>> config.create_paths = True

The Filesystem Hierarchy Standard layout is the same everywhere by definition.

>>> command.process(args)
GNU Mailman 3...
Python ...
...
File system paths:
    ARCHIVE_DIR     = /var/lib/mailman/archives
    BIN_DIR         = /sbin
    CACHE_DIR       = /var/lib/mailman/cache
    CFG_FILE        = .../test.cfg
    DATA_DIR        = /var/lib/mailman/data
    ETC_DIR         = /etc
    EXT_DIR         = /etc/mailman.d
    LIST_DATA_DIR   = /var/lib/mailman/lists
    LOCK_DIR        = /var/lock/mailman
    LOCK_FILE       = /var/lock/mailman/master.lck
    LOG_DIR         = /var/log/mailman
    MESSAGES_DIR    = /var/lib/mailman/messages
    PID_FILE        = /var/run/mailman/master.pid
    QUEUE_DIR       = /var/spool/mailman
    TEMPLATE_DIR    = .../mailman/templates
    VAR_DIR         = /var/lib/mailman
Command line message injection

You can inject a message directly into a queue directory via the command line.

>>> from mailman.commands.cli_inject import Inject
>>> command = Inject()

>>> class FakeArgs:
...     queue = None
...     show = False
...     filename = None
...     listname = None
...     keywords = []
>>> args = FakeArgs()

>>> class FakeParser:
...     def error(self, message):
...         print(message)
>>> command.parser = FakeParser()

It’s easy to find out which queues are available.

>>> args.show = True
>>> command.process(args)
Available queues:
    archive
    bad
    bounces
    command
    digest
    in
    nntp
    out
    pipeline
    retry
    shunt
    virgin

>>> args.show = False

Usually, the text of the message to inject is in a file.

>>> import os, tempfile
>>> fd, filename = tempfile.mkstemp()
>>> with os.fdopen(fd, 'w') as fp:
...     print("""\
... From: aperson@example.com
... To: test@example.com
... Subject: testing
... Message-ID: <aardvark>
...
... This is a test message.
... """, file=fp)

However, the mailing list name is always required.

>>> args.filename = filename
>>> command.process(args)
List name is required

Let’s provide a list name and try again.

>>> mlist = create_list('test@example.com')
>>> transaction.commit()
>>> from mailman.testing.helpers import get_queue_messages

>>> get_queue_messages('in')
[]
>>> args.listname = ['test@example.com']
>>> command.process(args)

By default, the incoming queue is used.

>>> items = get_queue_messages('in')
>>> len(items)
1
>>> print(items[0].msg.as_string())
From: aperson@example.com
To: test@example.com
Subject: testing
Message-ID: ...
Date: ...

This is a test message.



>>> dump_msgdata(items[0].msgdata)
_parsemsg    : False
listid       : test.example.com
original_size: 253
version      : 3

But a different queue can be specified on the command line.

>>> args.queue = 'virgin'
>>> command.process(args)

>>> get_queue_messages('in')
[]
>>> items = get_queue_messages('virgin')
>>> len(items)
1
>>> print(items[0].msg.as_string())
From: aperson@example.com
To: test@example.com
Subject: testing
Message-ID: ...
Date: ...

This is a test message.



>>> dump_msgdata(items[0].msgdata)
_parsemsg    : False
listid       : test.example.com
original_size: 253
version      : 3
Standard input

The message text can also be provided on standard input.

>>> from io import StringIO

>>> standard_in = StringIO(str("""\
... From: bperson@example.com
... To: test@example.com
... Subject: another test
... Message-ID: <badger>
...
... This is another test message.
... """))

>>> import sys
>>> sys.stdin = standard_in
>>> args.filename = '-'
>>> args.queue = None

>>> command.process(args)
>>> items = get_queue_messages('in')
>>> len(items)
1
>>> print(items[0].msg.as_string())
From: bperson@example.com
To: test@example.com
Subject: another test
Message-ID: ...
Date: ...

This is another test message.



>>> dump_msgdata(items[0].msgdata)
_parsemsg    : False
listid       : test.example.com
original_size: 261
version      : 3
Metadata

Additional metadata keys can be provided on the command line. These key/value pairs get added to the message metadata dictionary when the message is injected.

>>> args = FakeArgs()
>>> args.filename = filename
>>> args.listname = ['test@example.com']
>>> args.keywords = ['foo=one', 'bar=two']
>>> command.process(args)

>>> items = get_queue_messages('in')
>>> dump_msgdata(items[0].msgdata)
_parsemsg    : False
bar          : two
foo          : one
listid       : test.example.com
original_size: 253
version      : 3
Errors

It is an error to specify a queue that doesn’t exist.

>>> args.queue = 'xxbogusxx'
>>> command.process(args)
No such queue: xxbogusxx

It is also an error to specify a mailing list that doesn’t exist.

>>> args.queue = None
>>> args.listname = ['bogus']
>>> command.process(args)
No such list: bogus
Command line list display

A system administrator can display all the mailing lists via the command line. When there are no mailing lists, a helpful message is displayed.

>>> class FakeArgs:
...     advertised = False
...     names = False
...     descriptions = False
...     quiet = False
...     domains = None

>>> from mailman.commands.cli_lists import Lists
>>> command = Lists()
>>> command.process(FakeArgs)
No matching mailing lists found

When there are a few mailing lists, they are shown in alphabetical order by their fully qualified list names, with a description.

>>> from mailman.interfaces.domain import IDomainManager
>>> from zope.component import getUtility
>>> getUtility(IDomainManager).add('example.net')
<Domain example.net...>

>>> mlist_1 = create_list('list-one@example.com')
>>> mlist_1.description = 'List One'

>>> mlist_2 = create_list('list-two@example.com')
>>> mlist_2.description = 'List Two'

>>> mlist_3 = create_list('list-one@example.net')
>>> mlist_3.description = 'List One in Example.Net'

>>> command.process(FakeArgs)
3 matching mailing lists found:
list-one@example.com
list-one@example.net
list-two@example.com
Names

You can display the mailing list names with their posting addresses, using the --names/-n switch.

>>> FakeArgs.names = True
>>> command.process(FakeArgs)
3 matching mailing lists found:
list-one@example.com [List-one]
list-one@example.net [List-one]
list-two@example.com [List-two]
Descriptions

You can also display the mailing list descriptions, using the --descriptions/-d option.

>>> FakeArgs.descriptions = True
>>> command.process(FakeArgs)
3 matching mailing lists found:
list-one@example.com [List-one] - List One
list-one@example.net [List-one] - List One in Example.Net
list-two@example.com [List-two] - List Two

Maybe you want the descriptions but not the names.

>>> FakeArgs.names = False
>>> command.process(FakeArgs)
3 matching mailing lists found:
list-one@example.com - List One
list-one@example.net - List One in Example.Net
list-two@example.com - List Two
Less verbosity

There’s also a --quiet/-q switch which reduces the verbosity a bit.

>>> FakeArgs.quiet = True
>>> FakeArgs.descriptions = False
>>> command.process(FakeArgs)
list-one@example.com
list-one@example.net
list-two@example.com
Specific domain

You can narrow the search down to a specific domain with the –domain option. A helpful message is displayed if no matching domains are given.

>>> FakeArgs.quiet = False
>>> FakeArgs.domain = ['example.org']
>>> command.process(FakeArgs)
No matching mailing lists found

But if a matching domain is given, only mailing lists in that domain are shown.

>>> FakeArgs.domain = ['example.net']
>>> command.process(FakeArgs)
1 matching mailing lists found:
list-one@example.net

More than one –domain argument can be given; then all mailing lists in matching domains are shown.

>>> FakeArgs.domain = ['example.com', 'example.net']
>>> command.process(FakeArgs)
3 matching mailing lists found:
list-one@example.com
list-one@example.net
list-two@example.com
Advertised lists

Mailing lists can be ‘advertised’ meaning their existence is public knowledge. Non-advertised lists are considered private. Display through the command line can select on this attribute.

>>> FakeArgs.domain = []
>>> FakeArgs.advertised = True
>>> mlist_1.advertised = False

>>> command.process(FakeArgs)
2 matching mailing lists found:
list-one@example.net
list-two@example.com
Managing members

The mailman members command allows a site administrator to display, add, and remove members from a mailing list.

>>> ant = create_list('ant@example.com')

>>> class FakeArgs:
...     input_filename = None
...     output_filename = None
...     list = []
...     regular = False
...     digest = None
...     nomail = None
...     role = None
>>> args = FakeArgs()

>>> from mailman.commands.cli_members import Members
>>> command = Members()
Listing members

You can list all the members of a mailing list by calling the command with no options. To start with, there are no members of the mailing list.

>>> args.list = ['ant.example.com']
>>> command.process(args)
ant.example.com has no members

Once the mailing list add some members, they will be displayed.

>>> from mailman.testing.helpers import subscribe
>>> subscribe(ant, 'Anne', email='anne@example.com')
<Member: Anne Person <anne@example.com> on ant@example.com
         as MemberRole.member>
>>> subscribe(ant, 'Bart', email='bart@example.com')
<Member: Bart Person <bart@example.com> on ant@example.com
         as MemberRole.member>
>>> command.process(args)
Anne Person <anne@example.com>
Bart Person <bart@example.com>

Members are displayed in alphabetical order based on their address.

>>> subscribe(ant, 'Anne', email='anne@aaaxample.com')
<Member: Anne Person <anne@aaaxample.com> on ant@example.com
         as MemberRole.member>
>>> command.process(args)
Anne Person <anne@aaaxample.com>
Anne Person <anne@example.com>
Bart Person <bart@example.com>

You can also output this list to a file.

>>> from tempfile import NamedTemporaryFile
>>> with NamedTemporaryFile() as outfp:
...     args.output_filename = outfp.name
...     command.process(args)
...     with open(args.output_filename) as infp:
...         print(infp.read())
Anne Person <anne@aaaxample.com>
Anne Person <anne@example.com>
Bart Person <bart@example.com>
>>> args.output_filename = None

The output file can also be standard out.

>>> args.output_filename = '-'
>>> command.process(args)
Anne Person <anne@aaaxample.com>
Anne Person <anne@example.com>
Bart Person <bart@example.com>
>>> args.output_filename = None
Filtering on delivery mode

You can limit output to just the regular non-digest members…

>>> from mailman.interfaces.member import DeliveryMode
>>> args.regular = True
>>> member = ant.members.get_member('anne@example.com')
>>> member.preferences.delivery_mode = DeliveryMode.plaintext_digests
>>> command.process(args)
Anne Person <anne@aaaxample.com>
Bart Person <bart@example.com>

…or just the digest members. Furthermore, you can either display all digest members…

>>> member = ant.members.get_member('anne@aaaxample.com')
>>> member.preferences.delivery_mode = DeliveryMode.mime_digests
>>> args.regular = False
>>> args.digest = 'any'
>>> command.process(args)
Anne Person <anne@aaaxample.com>
Anne Person <anne@example.com>

…just plain text digest members…

>>> args.digest = 'plaintext'
>>> command.process(args)
Anne Person <anne@example.com>

…just MIME digest members.

>>> args.digest = 'mime'
>>> command.process(args)
Anne Person <anne@aaaxample.com>

# Reset for following tests.
>>> args.digest = None
Filtering on delivery status

You can also filter the display on the member’s delivery status. By default, all members are displayed, but you can filter out only those whose delivery status is enabled…

>>> from mailman.interfaces.member import DeliveryStatus

>>> member = ant.members.get_member('anne@aaaxample.com')
>>> member.preferences.delivery_status = DeliveryStatus.by_moderator
>>> member = ant.members.get_member('bart@example.com')
>>> member.preferences.delivery_status = DeliveryStatus.by_user

>>> member = subscribe(ant, 'Cris', email='cris@example.com')
>>> member.preferences.delivery_status = DeliveryStatus.unknown
>>> member = subscribe(ant, 'Dave', email='dave@example.com')
>>> member.preferences.delivery_status = DeliveryStatus.enabled
>>> member = subscribe(ant, 'Elle', email='elle@example.com')
>>> member.preferences.delivery_status = DeliveryStatus.by_bounces

>>> args.nomail = 'enabled'
>>> command.process(args)
Anne Person <anne@example.com>
Dave Person <dave@example.com>

…or disabled by the user…

>>> args.nomail = 'byuser'
>>> command.process(args)
Bart Person <bart@example.com>

…or disabled by the list administrator (or moderator)…

>>> args.nomail = 'byadmin'
>>> command.process(args)
Anne Person <anne@aaaxample.com>

…or by the bounce processor…

>>> args.nomail = 'bybounces'
>>> command.process(args)
Elle Person <elle@example.com>

…or for unknown (legacy) reasons.

>>> args.nomail = 'unknown'
>>> command.process(args)
Cris Person <cris@example.com>

You can also display all members who have delivery disabled for any reason.

>>> args.nomail = 'any'
>>> command.process(args)
Anne Person <anne@aaaxample.com>
Bart Person <bart@example.com>
Cris Person <cris@example.com>
Elle Person <elle@example.com>

# Reset for following tests.
>>> args.nomail = None
Adding members

You can add members to a mailing list from the command line. To do so, you need a file containing email addresses and full names that can be parsed by email.utils.parseaddr().

>>> bee = create_list('bee@example.com')
>>> with NamedTemporaryFile('w', buffering=1, encoding='utf-8') as fp:
...     for address in ('aperson@example.com',
...                     'Bart Person <bperson@example.com>',
...                     'cperson@example.com (Cate Person)',
...                     ):
...         print(address, file=fp)
...     fp.flush()
...     args.input_filename = fp.name
...     args.list = ['bee.example.com']
...     command.process(args)

>>> from operator import attrgetter
>>> dump_list(bee.members.addresses, key=attrgetter('email'))
aperson@example.com
Bart Person <bperson@example.com>
Cate Person <cperson@example.com>

You can also specify - as the filename, in which case the addresses are taken from standard input.

>>> from io import StringIO
>>> fp = StringIO()
>>> for address in ('dperson@example.com',
...                 'Elly Person <eperson@example.com>',
...                 'fperson@example.com (Fred Person)',
...                 ):
...         print(address, file=fp)
>>> args.input_filename = '-'
>>> filepos = fp.seek(0)
>>> import sys
>>> try:
...     stdin = sys.stdin
...     sys.stdin = fp
...     command.process(args)
... finally:
...     sys.stdin = stdin

>>> dump_list(bee.members.addresses, key=attrgetter('email'))
aperson@example.com
Bart Person <bperson@example.com>
Cate Person <cperson@example.com>
dperson@example.com
Elly Person <eperson@example.com>
Fred Person <fperson@example.com>

Blank lines and lines that begin with ‘#’ are ignored.

>>> with NamedTemporaryFile('w', buffering=1, encoding='utf-8') as fp:
...     for address in ('gperson@example.com',
...                     '# hperson@example.com',
...                     '   ',
...                     '',
...                     'iperson@example.com',
...                     ):
...         print(address, file=fp)
...     args.input_filename = fp.name
...     command.process(args)

>>> dump_list(bee.members.addresses, key=attrgetter('email'))
aperson@example.com
Bart Person <bperson@example.com>
Cate Person <cperson@example.com>
dperson@example.com
Elly Person <eperson@example.com>
Fred Person <fperson@example.com>
gperson@example.com
iperson@example.com

Addresses which are already subscribed are ignored, although a warning is printed.

>>> with NamedTemporaryFile('w', buffering=1, encoding='utf-8') as fp:
...     for address in ('gperson@example.com',
...                     'aperson@example.com',
...                     'jperson@example.com',
...                     ):
...         print(address, file=fp)
...     args.input_filename = fp.name
...     command.process(args)
Already subscribed (skipping): gperson@example.com
Already subscribed (skipping): aperson@example.com

>>> dump_list(bee.members.addresses, key=attrgetter('email'))
aperson@example.com
Bart Person <bperson@example.com>
Cate Person <cperson@example.com>
dperson@example.com
Elly Person <eperson@example.com>
Fred Person <fperson@example.com>
gperson@example.com
iperson@example.com
jperson@example.com
Displaying members

With no arguments, the command displays all members of the list.

>>> args.input_filename = None
>>> command.process(args)
aperson@example.com
Bart Person <bperson@example.com>
Cate Person <cperson@example.com>
dperson@example.com
Elly Person <eperson@example.com>
Fred Person <fperson@example.com>
gperson@example.com
iperson@example.com
jperson@example.com
Membership changes via email

Membership changes such as joining and leaving a mailing list, can be effected via the email interface. The Mailman email commands join, leave, and confirm are used.

Joining a mailing list

The mail command join subscribes an email address to the mailing list. subscribe is an alias for join.

>>> from mailman.commands.eml_membership import Join
>>> from mailman.utilities.string import wrap
>>> join = Join()
>>> print(join.name)
join
>>> print(wrap(join.description))
You will be asked to confirm your subscription request and you may be
issued a provisional password.
<BLANKLINE>
By using the 'digest' option, you can specify whether you want digest
delivery or not.  If not specified, the mailing list's default
delivery mode will be used.
>>> print(join.argument_description)
[digest=<no|mime|plain>]
No address to join 
>>> mlist = create_list('alpha@example.com')
>>> mlist.send_welcome_message = False

When no address argument is given, the message’s From address will be used. If that’s missing though, then an error is returned.

>>> from mailman.runners.command import Results
>>> results = Results()

>>> from mailman.email.message import Message
>>> print(join.process(mlist, Message(), {}, (), results))
ContinueProcessing.no
>>> print(results)
The results of your email command are provided below.

join: No valid address found to subscribe

The subscribe command is an alias.

>>> from mailman.commands.eml_membership import Subscribe
>>> subscribe = Subscribe()
>>> print(subscribe.name)
subscribe
>>> results = Results()
>>> print(subscribe.process(mlist, Message(), {}, (), results))
ContinueProcessing.no
>>> print(results)
The results of your email command are provided below.
<BLANKLINE>
subscribe: No valid address found to subscribe
<BLANKLINE>
Joining the sender

When the message has a From field, that address will be subscribed.

>>> msg = message_from_string("""\
... From: Anne Person <anne@example.com>
...
... """)
>>> results = Results()
>>> print(join.process(mlist, msg, {}, (), results))
ContinueProcessing.yes
>>> print(results)
The results of your email command are provided below.
<BLANKLINE>
Confirmation email sent to Anne Person <anne@example.com>
<BLANKLINE>

Anne is not yet a member of the mailing list because she must confirm her subscription request first.

>>> print(mlist.members.get_member('anne@example.com'))
None

Mailman has sent her the confirmation message.

>>> from mailman.testing.helpers import get_queue_messages
>>> items = get_queue_messages('virgin')
>>> len(items)
1
>>> print(items[0].msg.as_string())
MIME-Version: 1.0
...
Subject: confirm ...
From: alpha-confirm+...@example.com
To: anne@example.com
...
<BLANKLINE>
Email Address Registration Confirmation
<BLANKLINE>
Hello, this is the GNU Mailman server at example.com.
<BLANKLINE>
We have received a registration request for the email address
<BLANKLINE>
    anne@example.com
<BLANKLINE>
Before you can start using GNU Mailman at this site, you must first confirm
that this is your email address.  You can do this by replying to this me...
keeping the Subject header intact.
<BLANKLINE>
If you do not wish to register this email address, simply disregard this
message.  If you think you are being maliciously subscribed to the list, or
have any other questions, you may contact
<BLANKLINE>
    alpha-owner@example.com
<BLANKLINE>

Anne confirms her registration.

>>> def extract_token(message):
...     return str(message['subject']).split()[1].strip()
>>> token = extract_token(items[0].msg)

>>> from mailman.commands.eml_confirm import Confirm
>>> confirm = Confirm()
>>> msg = message_from_string("""\
... To: alpha-confirm+{token}@example.com
... From: anne@example.com
... Subject: Re: confirm {token}
...
... """.format(token=token))

>>> results = Results()
>>> print(confirm.process(mlist, msg, {}, (token,), results))
ContinueProcessing.yes
>>> print(results)
The results of your email command are provided below.

Confirmed

Anne is now a member of the mailing list.

>>> mlist.members.get_member('anne@example.com')
<Member: Anne Person <anne@example.com>
         on alpha@example.com as MemberRole.member>
Joining a second list 
>>> mlist_2 = create_list('baker@example.com')
>>> msg = message_from_string("""\
... From: Anne Person <anne@example.com>
...
... """)
>>> print(join.process(mlist_2, msg, {}, (), Results()))
ContinueProcessing.yes

Anne is not a member of the mailing list.

>>> print(mlist_2.members.get_member('anne@example.com'))
None

One Anne confirms this subscription, she becomes a member of the mailing list.

>>> items = get_queue_messages('virgin')
>>> len(items)
1
>>> token = extract_token(items[0].msg)
>>> msg = message_from_string("""\
... To: baker-confirm+{token}@example.com
... From: anne@example.com
... Subject: Re: confirm {token}
...
... """.format(token=token))

>>> results = Results()
>>> print(confirm.process(mlist_2, msg, {}, (token,), results))
ContinueProcessing.yes
>>> print(results)
The results of your email command are provided below.

Confirmed


>>> print(mlist_2.members.get_member('anne@example.com'))
<Member: Anne Person <anne@example.com>
         on baker@example.com as MemberRole.member>
Leaving a mailing list

The mail command leave unsubscribes an email address from the mailing list. unsubscribe is an alias for leave.

>>> from mailman.commands.eml_membership import Leave
>>> leave = Leave()
>>> print(leave.name)
leave
>>> print(leave.description)
Leave this mailing list.
<BLANKLINE>
You may be asked to confirm your request.

Anne is a member of the baker@example.com mailing list, when she decides to leave it. Because the mailing list allows for open unsubscriptions (i.e. no confirmation is needed), when she sends a message to the -leave address for the list, she is immediately removed.

>>> from mailman.interfaces.mailinglist import SubscriptionPolicy
>>> mlist_2.unsubscription_policy = SubscriptionPolicy.open
>>> mlist.unsubscription_policy = SubscriptionPolicy.open
>>> results = Results()
>>> print(leave.process(mlist_2, msg, {}, (), results))
ContinueProcessing.yes
>>> print(results)
The results of your email command are provided below.
<BLANKLINE>
Anne Person <anne@example.com> left baker@example.com
<BLANKLINE>

Anne is no longer a member of the mailing list.

>>> print(mlist_2.members.get_member('anne@example.com'))
None

Anne does not need to leave a mailing list with the same email address she’s subscribe with. Any of her registered, linked, and validated email addresses will do.

>>> from mailman.interfaces.usermanager import IUserManager
>>> from zope.component import getUtility
>>> anne = getUtility(IUserManager).get_user('anne@example.com')
>>> address = anne.register('anne.person@example.org')

>>> results = Results()
>>> print(mlist.members.get_member('anne@example.com'))
<Member: Anne Person <anne@example.com>
         on alpha@example.com as MemberRole.member>

>>> msg = message_from_string("""\
... To: alpha-leave@example.com
... From: anne.person@example.org
...
... """)

Since Anne’s alternative address has not yet been verified, it can’t be used to unsubscribe Anne from the alpha mailing list.

>>> print(leave.process(mlist, msg, {}, (), results))
ContinueProcessing.no

>>> print(results)
The results of your email command are provided below.

Invalid or unverified email address: anne.person@example.org


>>> print(mlist.members.get_member('anne@example.com'))
<Member: Anne Person <anne@example.com>
         on alpha@example.com as MemberRole.member>

Once Anne has verified her alternative address though, it can be used to unsubscribe her from the list.

>>> from mailman.utilities.datetime import now
>>> address.verified_on = now()

>>> results = Results()
>>> print(leave.process(mlist, msg, {}, (), results))
ContinueProcessing.yes

>>> print(results)
The results of your email command are provided below.

Anne Person <anne.person@example.org> left alpha@example.com


>>> print(mlist.members.get_member('anne@example.com'))
None
Confirmations

Bart wants to join the alpha list, so he sends his subscription request.

>>> msg = message_from_string("""\
... From: Bart Person <bart@example.com>
...
... """)

>>> print(join.process(mlist, msg, {}, (), Results()))
ContinueProcessing.yes

There are two messages in the virgin queue, one of which is the confirmation message.

>>> for item in get_queue_messages('virgin'):
...     if str(item.msg['subject']).startswith('confirm'):
...         break
... else:
...     raise AssertionError('No confirmation message')
>>> token = extract_token(item.msg)

Bart replies to the original message, specifically keeping the Subject header intact except for any prefix. Mailman matches the token and confirms Bart as a user of the system.

>>> msg = message_from_string("""\
... From: Bart Person <bart@example.com>
... To: alpha-confirm+{token}@example.com
... Subject: Re: confirm {token}
...
... """.format(token=token))

>>> results = Results()
>>> print(confirm.process(mlist, msg, {}, (token,), results))
ContinueProcessing.yes

>>> print(results)
The results of your email command are provided below.

Confirmed

Now Bart is now a member of the mailing list.

>>> print(mlist.members.get_member('bart@example.com'))
<Member: Bart Person <bart@example.com>
         on alpha@example.com as MemberRole.member>
Dumping queue files

The qfile command dumps the contents of a queue pickle file. This is especially useful when you have shunt files you want to inspect.

XXX Test the interactive operation of qfile

Pretty printing

By default, the qfile command pretty prints the contents of a queue pickle file to standard output.

>>> from mailman.commands.cli_qfile import QFile
>>> command = QFile()

>>> class FakeArgs:
...     interactive = False
...     doprint = True
...     qfile = []

Let’s say Mailman shunted a message file.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: Uh oh
...
... I borkeded Mailman.
... """)

>>> shuntq = config.switchboards['shunt']
>>> basename = shuntq.enqueue(msg, foo=7, bar='baz', bad='yes')

Once we’ve figured out the file name of the shunted message, we can print it.

>>> from os.path import join
>>> qfile = join(shuntq.queue_directory, basename + '.pck')

>>> FakeArgs.qfile = [qfile]
>>> command.process(FakeArgs)
[----- start pickle -----]
<----- start object 1 ----->
From: aperson@example.com
To: test@example.com
Subject: Uh oh

I borkeded Mailman.

<----- start object 2 ----->
{'_parsemsg': False, 'bad': 'yes', 'bar': 'baz', 'foo': 7, 'version': 3}
[----- end pickle -----]

Maybe we don’t want to print the contents of the file though, in case we want to enter the interactive prompt.

>>> FakeArgs.doprint = False
>>> command.process(FakeArgs)
Command line list removal

A system administrator can remove mailing lists by the command line.

>>> create_list('test@example.com')
<mailing list "test@example.com" at ...>

>>> from mailman.interfaces.listmanager import IListManager
>>> from zope.component import getUtility
>>> list_manager = getUtility(IListManager)
>>> list_manager.get('test@example.com')
<mailing list "test@example.com" at ...>

>>> class FakeArgs:
...     quiet = False
...     archives = False
...     listname = ['test@example.com']
>>> args = FakeArgs()

>>> from mailman.commands.cli_lists import Remove
>>> command = Remove()
>>> command.process(args)
Removed list: test@example.com

>>> print(list_manager.get('test@example.com'))
None

You can also remove lists quietly.

>>> create_list('test@example.com')
<mailing list "test@example.com" at ...>

>>> args.quiet = True
>>> command.process(args)

>>> print(list_manager.get('test@example.com'))
None
Getting status

The status of the Mailman master process can be queried from the command line. It’s clear at this point that nothing is running.

>>> from mailman.commands.cli_status import Status
>>> status = Status()

>>> class FakeArgs:
...     pass

The status is printed to stdout and a status code is returned.

>>> status.process(FakeArgs)
GNU Mailman is not running
0

We can simulate the master starting up by acquiring its lock.

>>> from flufl.lock import Lock
>>> lock = Lock(config.LOCK_FILE)
>>> lock.lock()

Getting the status confirms that the master is running.

>>> status.process(FakeArgs)
GNU Mailman is running (master pid: ...

We shut down the master and confirm the status.

>>> lock.unlock()
>>> status.process(FakeArgs)
GNU Mailman is not running
0
Unshunt

When errors occur while processing email messages, the messages will end up in the shunt queue. The unshunt command allows system administrators to manage the shunt queue.

>>> from mailman.commands.cli_unshunt import Unshunt
>>> command = Unshunt()

>>> class FakeArgs:
...     discard = False

Let’s say there is a message in the shunt queue.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: A broken message
... Message-ID: <aardvark>
...
... """)

>>> shuntq = config.switchboards['shunt']
>>> len(list(shuntq.files))
0
>>> base_name = shuntq.enqueue(msg, {})
>>> len(list(shuntq.files))
1

The unshunt command by default moves the message back to the incoming queue.

>>> inq = config.switchboards['in']
>>> len(list(inq.files))
0

>>> command.process(FakeArgs)

>>> from mailman.testing.helpers import get_queue_messages
>>> items = get_queue_messages('in')
>>> len(items)
1
>>> print(items[0].msg.as_string())
From: aperson@example.com
To: test@example.com
Subject: A broken message
Message-ID: <aardvark>

unshunt moves all shunt queue messages.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: A broken message
... Message-ID: <badgers>
...
... """)
>>> base_name = shuntq.enqueue(msg, {})

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: A broken message
... Message-ID: <crow>
...
... """)
>>> base_name = shuntq.enqueue(msg, {})

>>> len(list(shuntq.files))
2

>>> command.process(FakeArgs)
>>> items = get_queue_messages('in')
>>> len(items)
2

>>> sorted(item.msg['message-id'] for item in items)
['<badgers>', '<crow>']
Return to the original queue

While the messages in the shunt queue are generally returned to the incoming queue, if the error occurred while the message was being processed from a different queue, it will be returned to the queue it came from.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: A broken message
... Message-ID: <dingo>
...
... """)

The queue that the message comes from is in message metadata.

>>> base_name = shuntq.enqueue(msg, {}, whichq='bounces')

>>> len(list(shuntq.files))
1
>>> len(list(config.switchboards['bounces'].files))
0

The message is automatically re-queued to the bounces queue.

>>> command.process(FakeArgs)
>>> len(list(shuntq.files))
0
>>> items = get_queue_messages('bounces')
>>> len(items)
1

>>> print(items[0].msg.as_string())
From: aperson@example.com
To: test@example.com
Subject: A broken message
Message-ID: <dingo>
Discarding all shunted messages

If you don’t care about the shunted messages, just discard them.

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: A broken message
... Message-ID: <elephant>
...
... """)
>>> base_name = shuntq.enqueue(msg, {})

>>> FakeArgs.discard = True
>>> command.process(FakeArgs)

The messages are now gone.

>>> items = get_queue_messages('in')
>>> len(items)
0
Printing the version

You can print the Mailman version number.

>>> from mailman.commands.cli_version import Version
>>> command = Version()

>>> command.process(None)
GNU Mailman 3...
Operating on mailing lists

The shell (alias: withlist) command is a pretty powerful way to operate on mailing lists from the command line. This command allows you to interact with a list at a Python prompt, or process one or more mailing lists through custom made Python functions.

Getting detailed help

Because withlist is so complex, you need to request detailed help.

>>> from mailman.commands.cli_withlist import Withlist
>>> command = Withlist()

>>> class FakeArgs:
...     interactive = False
...     run = None
...     details = True
...     listname = []

>>> class FakeParser:
...     def error(self, message):
...         print(message)
>>> command.parser = FakeParser()

>>> args = FakeArgs()
>>> command.process(args)
This script provides you with a general framework for interacting with a
mailing list.
...
Running a command

By putting a Python function somewhere on your sys.path, you can have withlist call that function on a given mailing list. The function takes a single argument, the mailing list.

>>> import os, sys
>>> old_path = sys.path[:]
>>> sys.path.insert(0, config.VAR_DIR)

>>> with open(os.path.join(config.VAR_DIR, 'showme.py'), 'w') as fp:
...     print("""\
... def showme(mailing_list):
...     print("The list's name is", mailing_list.fqdn_listname)
...
... def displayname(mailing_list):
...     print("The list's display name is", mailing_list.display_name)
... """, file=fp)

If the name of the function is the same as the module, then you only need to name the function once.

>>> mlist = create_list('aardvark@example.com')
>>> args.details = False
>>> args.run = 'showme'
>>> args.listname = 'aardvark@example.com'
>>> command.process(args)
The list's name is aardvark@example.com

The function’s name can also be different than the modules name. In that case, just give the full module path name to the function you want to call.

>>> args.run = 'showme.displayname'
>>> command.process(args)
The list's display name is Aardvark
Multiple lists

You can run a command over more than one list by using a regular expression in the listname argument. To indicate a regular expression is used, the string must start with a caret.

>>> mlist_2 = create_list('badger@example.com')
>>> mlist_3 = create_list('badboys@example.com')

>>> args.listname = '^.*example.com'
>>> command.process(args)
The list's display name is Aardvark
The list's display name is Badboys
The list's display name is Badger

>>> args.listname = '^bad.*'
>>> command.process(args)
The list's display name is Badboys
The list's display name is Badger

>>> args.listname = '^foo'
>>> command.process(args)
Error handling

You get an error if you try to run a function over a non-existent mailing list.

>>> args.listname = 'mystery@example.com'
>>> command.process(args)
No such list: mystery@example.com

You also get an error if no mailing list is named.

>>> args.listname = None
>>> command.process(args)
--run requires a mailing list name
Interactive use

You can also get an interactive prompt which allows you to inspect a live Mailman system directly. Through the mailman.cfg file, you can set the prompt and banner, and you can choose between the standard Python REPL or IPython.

If the GNU readline library is available, it will be enabled automatically, giving you command line editing and other features. You can also set the [shell]history_file variable in the mailman.cfg file and when the normal Python REPL is used, your interactive commands will be written to and read from this file.

Note that the $PYTHONSTARTUP environment variable will also be honored if set, and any file named by this variable will be read at start up time. It’s common practice to also enable GNU readline history in a $PYTHONSTARTUP file and if you do this, be aware that it will interact badly with [shell]history_file, causing your history to be written twice. To disable this when using the interactive shell command, do something like:

$ PYTHONSTARTUP= mailman shell

to temporarily unset the environment variable.

IPython

You can use IPython as the interactive shell by setting the [shell]use_ipython variables in your mailman.cfg file to yes. IPython must be installed and available on your system

When using IPython, the [shell]history_file is not used.

Community Contributions

This is a directory contain various contributions from the fantastic GNU Mailman community. We welcome and appreciate your contributions.

Please be aware that the files in this directory are not officially supported by the core Mailman development team. They are also covered by their own license terms; consult those files for details. We do however require GPL compatible licenses for any contributed code appearing here.

If you have any questions about these files, please contact the author of that file. The core Mailman development team may not be able to answer your questions, but we’ll still try to review any issues or merge proposals related to them.

Indices and tables