man ezmlm (5): format of ezmlm directory

OVERVIEW

An ezmlm directory, dir, stores information about an ezmlm mailing list. ezmlm-make creates dir along with the files necessary to support the other programs; ezmlm-sub and ezmlm-unsub manipulate the subscriber list(s) stored under dir; ezmlm-list lists the subscriber list(s); ezmlm-manage handles administrative requests automatically; ezmlm-send sends a message to all subscribers listed in dir and also maintains a message archive and message subject index if the list is configured to do so. ezmlm-reject rejects messages that that are unsuitable for distribution to the mailing list; ezmlm-return handles bounces; ezmlm-weed weeds out useless messages; ezmlm-warn warns users for which messages bounce and eventually removes them from the subscriber list. ezmlm-idx can create a subject index from an existing list archive, and ezmlm-archive creates and updates thread and author indices for the archive. ezmlm-get manages message, index, and thread retrieval from the archive, as well as the generation of message digests; ezmlm-cron provides a restricted interface to cron for the generation of digest generation trigger messages; ezmlm-store queues messages of moderated lists and sends a moderation request to the moderator(s); ezmlm-moderate processes moderation requests to accept the queued message to the list via ezmlm-send or to return the message to the sender; ezmlm-confirm does similar for user confirmation requests; ezmlm-clean cleans up the moderation queue and returns to the sender any messages that have timed-out; ezmlm-gate posts messages that come from a SENDER in an address database, and sends remaining messages out for moderation; ezmlm-check is used to diagnose problems with ezmlm mailing list configuration; ezmlm-checksub and ezmlm-issubn determine if a SENDER is a subscriber or a member of a collection of addresses; ezmlm-tstdig determines if it is time to create a new digest based on the number and volume of messages and the amount of time that has passed since the last digest was issued; ezmlm-request can be used to answer ezmlm commands in the subject line easing migration from other mailing list managers. It can also function as a global interface mimicking the interface of other mailing list manager. ezmlm-glmake can set up the global interface, and ezmlm-glconf can create a configuration file for the global interface from your lists.

SUBSCRIBERS

dir/subscribers is a directory containing the subscriber list. ezmlm-manage allows automatic subscription if dir/public exists.

The list is hashed into 53 files, named @ through t in ASCII. A nonexistent file is treated as an empty file.

Each file contains a series of addresses. An address can be any string of non-NUL characters up to 400 bytes long. Each address is preceded by the letter T and followed by a NUL.

For reliability, when an address is added to or removed from the mailing list, the relevant file is recreated under a temporary name inside dir/subscribers and then moved into place.

dir/subdb contains subscriber database access information for lists that are configured to use an alternate database plugin (such as mysql, pgsql, or sqlite3) for storage of mailing lists. Its format is plugin:host:port:user:password:database:table. The database and table names both default to ezmlm. If this file does not exist but dir/sql exists, its contents are used with a plugin name of sql. If no such file exists, the standard lists stored in dir/subscribers etc are used.

BOUNCES

dir/bounce is a directory containing bounce messages. ezmlm-return stores several types of files here.

DELIVERY INSTRUCTIONS

ezmlm-make sets up four files to control mailing list deliveries. Each file is a series of delivery instructions in dot-qmail format.

dir/editor handles incoming mailing list submissions. ezmlm-make sets up dir/editor to invoke ezmlm-send to immediately forward each message to all subscribers and then to run ezmlm-warn.

dir/owner handles incoming messages for the mailing list's owner. ezmlm-make sets up dir/owner to store messages in dir/Mailbox and then to run ezmlm-warn.

dir/bouncer handles incoming bounce messages. ezmlm-make sets up dir/bouncer to invoke ezmlm-return. ezmlm-warn is no longer invoked here due to the load it places on systems with many large lists with many bounces.

dir/confirmer handles incoming message confirm and discard requests for sender confirmed lists. ezmlm-make sets up dir/confirmer to invoke ezmlm-confirm, ezmlm-archive, and then ezmlm-clean.

dir/manager handles incoming administrative requests. ezmlm-make sets up dir/manager to invoke ezmlm-get, ezmlm-manage, ezmlm-request, and then ezmlm-warn.

dir/moderator handles incoming message accept and reject requests for moderated lists. ezmlm-make sets up dir/moderator to invoke ezmlm-weed, ezmlm-moderate, ezmlm-archive, and ezmlm-clean.

DIGESTS

ezmlm-get can create digests if it is invoked from the command line, from dir/editor, or from dir/manager. The program functions in slightly different ways in these 3 settings (see ezmlm-get(1)).

To enable automatic digests for a mailing list, use the ezmlm-make -d switch or create the dir/digested file. To also enable the generation of digests at specific times dictated by mailed trigger messages, a digestcode should be specified in the dir/digestcode file. This can be done by specifying digestcode as a fifth argument to ezmlm-make when setting up the list. digestcode must be alphanumeric and is case-insensitive.

To generate trigger messages, use ezmlm-cron(1) as an interface to crond(8) or use crond directly.

ezmlm-get is able to create digests with a variety of different formats which may be specified on the command line for ezmlm-get or in the file dir/digformat.

dir/num contains the number of the last message processed by ezmlm-send, followed by ':' and a number that is increased by 1 for each 256 bytes of message body text processed. The latter number is used by ezmlm-tstdig to determine if a new digest is due.

dir/dignum contains the contents of dir/num at the time of the last regular digest creation, followed by a ':', followed by a timestamp. It is updated after each regular digest is sent.

dir/digissue contains the issue number of the last regular digest. It is incremented for each regular digest sent.

The following user crontab entry (all on one line) generates a digest of the list [email protected] at 1600 every day:

00 16 * * * /var/qmail/bin/qmail-inject list-dig.digestcode

Alternatively, ezmlm-cron can be used:

% ezmlm-cron -t 16:00 list@host digestcode

ezmlm-get can also be run from the shell: To generate a digest to list-digest@host from the list housed in ~joe/list:

% ezmlm-get ~joe/list

Like other ezmlm-get replies, digest can be sent in several formats. See ezmlm-get(1) for more info.

MODERATION

There are four aspects of moderation: sender confirmation of posts (also known as "user confirmation" or "self moderation"), moderation of posts, moderation of subscriptions, and "remote administration", i.e. giving the moderator the right to (un)subscribe any user. ezmlm handles these four aspects separately. The first three aspects enhance security, while the last decreases security, but makes list administration considerably easier. By default, the moderator database is the same for all three functions. While "remote administration" and subscription moderation always use the same database, the moderators for message moderation can be different.

Even with subscription moderation, the user has to verify the request. This is to ensure that the user initiating the request really controls the address. ezmlm-manage options exist to disable the user handshake, which may be useful in some circumstances.

For standard moderation options, the moderators are by stored in a subscriber list in moddir/subscribers. By default moddir is dir/mod.

Moderators can be added and removed with:

ezmlm-sub dir mod moderator@host

ezmlm-unsub dir mod moderator@host

For subscription moderation, touch dir/modsub after adding moderator(s). For remote administration, touch dir/remote. If the contents of these files contain a subdirectory name, it is used as the name of the mod address list directory for subscription moderation. If both files exist and contain a subdirectory name, the dir/remote contents are ignored. Moderator addresses are stored as indicated in the SUBSCRIBERS section above. If no directory names are specified, the default, dir/mod, is used. In all cases, the named subscriber list must exist.

Sender confirmation is achieved by creating dir/confirmpost and moderation of posts is achieved by creating dir/modpost. In either case, modify dir/editor to invoke ezmlm-store. For sender confirmation, ezmlm-store stores the message in dir/mod/unconfirmed and sends a confirmation request to the sender. For moderation, ezmlm-store stores the message in dir/mod/pending and sends a moderation request to all moderators stored in mod. If moderation is enabled and dir/modpostonly exists, messages from non-moderators are rejected.

If neither dir/confirmpost nor dir/modpost exist, ezmlm-store posts messages directly (via ezmlm-send), and ezmlm-clean does nothing.

If dir/modpost contains a subdirectory name this directory is used as the mod subscriber list for message moderation. Moderators are stored in a subscriber list according to the SUBSCRIBERS section above. If no directory names are specified, the default, dir/mod, is used.

dir/confirmer is linked to dot-confirm-default and dir-discard-default. It handles replies for sender confirmation. dir/moderator is linked to dot-accept-default and dot-reject-default. It handles replies from the moderators.

In addition to a moderator list, the directories dir/mod/accepted, dir/mod/pending, dir/mod/rejected, and dir/mod/unconfirmed must exist. These directories contain the message moderation queue.

If dir/mod/modtime it determines the minimal time in hours that messages wait in the moderation queue, before they are returned to sender with the message in dir/text/mod-timeout.

If a -help command is sent for a moderator and dir/modsub or dir/remote exist, a more detailed help message stored in dir/text/mod-help will be sent together with the regular help. This text should not contain secrets. If dir/text/mod-help does not exist, dir/text/help will be sent.

If a -list command is sent for a moderator and dir/modsub or dir/remote exist, and either the dir/modcanlist file exists or the ezmlm-manage -l command line switch is specified, a subscriber list will be returned.

If an -edit.file command is sent for a moderator and dir/remote exist, and either the dir/modcanedit file exists or the ezmlm-manage -d or -e command line switches are specified, text/file is returned together with an ezmlm cookie. The remote administrator may return an edited version of the file, which will be stored, provided that the cookie is valid. See ezmlm-manage(1) for more info.

TEXT

text is a directory containing files sent out in messages generated by ezmlm in response to administrative requests. These files may be located in one of three locations: in the dir/text directory; in the alternate directory lang/text; or in the default directory /etc/ezmlm/default/text. The lang parameter in the second path is the contents of the dir/ezmlmrc file, which is created by ezmlm-make. By default, ezmlm-make does not install any of these text files into dir. Instead, it relies on the use of the alternate and default paths to look up text messages.

TEXT FILES

top: Introducing ezmlm. This is placed at the top of each response.
bottom: Explaining how to use ezmlm. This is placed at the bottom of each response.
sub-confirm: Explaining how to confirm a subscription request.
sub-ok: Acknowledging successful subscription.
sub-nop: Acknowledging a subscription request for an address already on the mailing list.
sub-bad: Rejecting a bad subscription confirmation number.
unsub-confirm: Explaining how to confirm an unsubscription request, and explaining how to figure out the subscription address.
unsub-ok: Acknowledging successful unsubscription.
unsub-nop: Acknowledging an unsubscription request for an address not on the mailing list.
unsub-bad: Rejecting a bad unsubscription confirmation number.
get-bad: Rejecting a bad archive retrieval request.
digest: Text copied into the Administrativia section of the digest. Usually, this will contain subscription info for the digest, as well as information on how to post to the list.
trailer: If this files exists, it is copied to the end of all messages to the list.
faq: Sent in response to the faq command. Usually contains frequently asked questions and answers specific for the mailing list.
info: Sent in response to the info command. Usually contains a descripition, policy, etc, for the list. The first line should in itself be a very brief description of the list.
help: General help in response to a misdirected or misspelled request.
bounce-warn: Pointing out that messages have bounced.
bounce-probe: Pointing out that a warning message has bounced.
bounce-num: Explaining that ezmlm-return has kept a list of bounced message numbers.
dig-bounce-num: Explaining that digest messages have bounced. All other text files are used for both the main list and the digest list.
bounce-bottom: Separating the bounce message.
mod-help: is set to list moderators issuing a -help command. It contains instructions for moderators, but it is relatively trivial for a non-moderator to read it. Don't put secrets here.
mod-reject: is the returned to the sender of a rejected post.
mod-timeout: is returned if the message timed-out without moderator action.
mod-sub: is added to the text confirming subscription and unsubscription instead of bottom and the requesting message, for actions that were approved by a moderator. Not copying the requesting message hides the moderator identity from the subscriber.
mod-request: is the text sent to the moderators to request moderator action on a posted message.
mod-sub-confirm: Requesting that the moderator confirm a request to subscribe. If this file does not exist, sub-confirm will be used.
mod-unsub-confirm: Requesting that the moderator confirm a request to unsubscribe. If this file does not exist, unsub-confirm will be used.
post-confirm: Requesting that the sender confirms that a posted message did originate from them.
edit-do: Instructions sent to the remote administrator together with a copy of a dir/text file and editing instructions.
edit-list: A list of editable files in dir/text with a one-line description send to a remote administrator in response to a -edit command.
edit-done: Sent to the remote administrator after an edited dir/text file has been successfully saved.

SUBSTITUTIONS

Several tags in the text files are replaced by ezmlm programs. Tags may appear anywhere on a line and multiple tags may appear on the same line.

<#L#>: The unmodified name of the list, as defined by dir/outlocal
<#l#>: The name of the list or the list-digest, as appropriate for the request. The use of <#l#> is to allow the same text file to be used for requests pertaining to both the main list and the digest list.
<#H#>: The hostname for the list, as defined by dir/outhost
<#h#>: The hostname for the list
<#n#>: The current message number in ezmlm-send, and the number of the first message in the digest in ezmlm-get
<#A#>: The moderation accept or (un)subscription target address (described below)
<#a#>: The local part of the moderation accept address
<#t#>: The subscription target address, with "@" replaced with "="
<#R#>: The moderation reject or (un)subscription reply address (described below), equivalent to <#r#>@<#h#>
<#r#>: The local part of the reject or reply address, equivalent to <#l#>-<#c#>
<#c#>: The cryptographic "cookie" in the reject or reply address
<#d#>: dir

The subscription target address is the address that has requested subscription to or unsubscription from the list in ezmlm-manage. The same tag is used in ezmlm-store for the address to which a reply must be sent to accept the original post.

The subscription reply address is the address to which a reply must be sent to confirm a subscription in ezmlm-manage. The same tag is used in ezmlm-store for the address to which a reply may be sent to reject the original post.

For backwards compatibility, the lines !A and !R are replaced with the value of <#A#> and <#R#> respectively.

TEXT/MESSAGES

One of the text files, text/messages, has special handling. It is used when creating short messages within the ezmlm programs, such as error messages, subject lines, and several others. Each line of this file contains a message name and the contents of that message, separated by a colon. Individual messages are loaded from all three locations described above instead of just the first file that is found, allowing for partial sets of customizations. Additionally, the programs have an internal table of messages as a final fallback.

In addition to the substitions listed above, the tags <#1#> through <#9#> are used by certain messages for file names and other parameters specific to the message. The default messages in /etc/ezmlm/default/text/messages should have a complete set of messages with all parameters used.

OUTGOING MESSAGE EDITING

dir/headerkeep is a list of good header field names, one per line, and dir/headerremove is a list of bad header field names. If dir/headerkeep is present, ezmlm-send removes all header fields but those that are listed from outgoing messages; otherwise ezmlm-send removes the header fields listed in dir/headerremove from all outgoing messages. ezmlm-make sets up dir/headerremove to remove Return-Path, Return-Receipt-To, and Return-Path fields.

dir/headeradd is a list of new header fields. ezmlm-send adds these fields to every outgoing message. ezmlm-send sets up dir/headeradd to add X-No-Archive: yes and Precedence:bulk.

If dir/headerreject exists, and the ezmlm-reject dir argument is specified, messages containing any of the listed headers are rejected.

If dir/mimekeep exists, ezmlm-send removes parts except those with corresponding content-types from composite MIME messages. Otherwise, if dir/mimeremove exists, ezmlm-send removes parts with the corresponding content-types. If the ezmlm-reject dir argument is specified, messages consisting only of disallowed content-types are rejected.

If dir/mimereject exists, and the ezmlm-reject dir argument is specified, simple MIME messages of these content-types, or composite MIME messages with any body part of these content-types are rejected.

If dir/sequence exists, the first line is added as a header to all outgoing messages, followed by a space and the message number. The message number is useful for archive retrievals, since some mail systems do not reveal the return-path to the user. NOTE: Sublists have their own message counter. Adding a sequence header from a sublists will give you the sublist message number which is different from the main list message number.

dir/prefix is a subject prefix. If this file exists, its contents are prefixed to the subject of the post in the outgoing message. The archived message is not processed. Attempts are made to not duplicate an existing prefix in replies. Think twice before using this option. A prefix takes unnecessary space on the subject line and most mail clients can easily filter on other headers, such as 'Mailing-List:'. If dir/prefix contains a single '#', this will be replaced by the message number. The use of this feature is inadvisable and violates internet mail standards. However, it is very popular in e.g. Japan. If you must use this feature, make sure you are aware that you may be causing problems to users, sublists, etc.

dir/text/trailer is a message trailer. If this file exists, it's contents are copied to the end of outgoing messages. Only lines terminated with new-line are copied. No trailer is copied to the archived version of the message.

MISCELLANY

If the allow address list exists, ezmlm will allow any sender found in that list to post even if they are not subscribers. If the deny address list exists, ezmlm will block all senders found in that list from posting to the list. Addresses in either list that start with a @ will allow or deny all senders at the following domain name. Addresses can be added and removed from these lists similarly to the moderator examples above.

If dir/listid exists, ezmlm programs create a new List-ID field, showing the contents of the first line of dir/listid, in every outgoing message. The list-id should be unique and within name space controlled by the owner. It should remain constant even if lists move and be of the format

List-ID: optional_text <unique_id.domain>

This header would result from a dir/listid file containing ``optional_text <unique_id.domain>''. See RFC 2919 at http://www.ietf.org/rfc/rfc2919.txt for more info.

The first lines of dir/outlocal and dir/outhost give the outgoing name of the mailing list. These are used by ezmlm-manage and ezmlm-send to construct sender addresses for outgoing messages.

If dir/sublist exists, this mailing list is a sublist, redistributing messages from a parent mailing list. The first line of dir/sublist is the name of the parent list. This affects the behavior of ezmlm-send.

If dir/qmqpservers exists, all ezmlm programs will use qmail-qmqpc(1) to send messages. If qmail-qmqpc is modified correctly, server IP addresses listed one per line in dir/qmqpsevers will be tried in order, rather than the default servers specified in /var/qmail/control.

If dir/msgsize exists, it is assumed to contain ``max:min'', where ``max'' is the maximum size in bytes of an acceptable message body, and ``min'' the corresponding minimal size. Either will be ignored if zero or omitted. If the ezmlm-reject command line specifies the list directory, messages not meeting the size criteria are rejected.

If dir/charset exists, the first line is assumed to represent a valid MIME character set, which is used for all outgoing MIME messages sent by ezmlm-get and the message moderation programs. The character set string may be suffixed with ':' and 'Q' or 'B' to send all outgoing text (ezmlm messages, digest table-of-contents, moderation requests, etc) encoded in ``Quoted-Printable'' or ``base64'' encoding. By default, no encoding is done, which may result in the transmission of characters with the high bit set. When encoding is specified, trigger messages and other parts of the reply that should not be encoded are sent as separate MIME parts.

dir/lock is an empty file. Any program that reads or writes the subscriber list, or adds messages to the archive, locks dir/lock.

dir/Log is an advisory log of subscription and unsubscription actions. WARNING: Log is not protected against system crashes. Log entries may be missing or corrupted if the system goes down. There is Log for each of the accessory address databases as well. Thus, the log for digest subscribers is dir/digest/Log. If enabled, these logs can be retrieved by remote administrators (see ezmlm-manage(1)).

If dir/omitbottom exists, will suppress the administrative information found in dir/text/bottom and the copy of the request that is normally copied into outgoing automatic responses. This may make it harder for the recipient to diagnose problems and learn commands.

dir/copylines specifies how many lines from the body of the original request to copy into outgoing automatic responses. If this file is not present or is empty, a value of 0 is used. In any case, the entire header is copied.

dir/digest contains items specific for the digest list.

dir/digest/subscribers contains hash files of digest subscriber addresses.

dir/digest/Log, dir/digest/bounce, dir/digest/lockbounce, and dir/digest/lock have functions for the digest list that mirror that of the corresponding files in dir.

dir/digheaders may contain a list of headers to include in the "m" format digests. Headers should be listed one per line not including the colon.

dir/digcount, dir/digsize, and dir/digtime control when ezmlm-tstdig will allow ezmlm-get to create a digest message. dir/tstdig is a timestamp used temporarily by ezmlm-tstdig to coordinate digesting.

dir/archnum contains the number of the last message processed by ezmlm-archive. Normally, ezmlm-archive will process entries for messages from one above the contents of this file up to an including the message number in dir/num. The default ezmlmrc template sets up ezmlm-archive to run only if the dir/threaded file exists.

If dir/noreturnposts exists, ezmlm-clean will not return timed-out posts to their senders.

If dir/nosubconfirm exists, ezmlm-manage will not require confirmation from the subscription target before subscribing it. Similarly, if dir/nounsubconfirm exists, ezmlm-manage will not require confirmation from the unsubscription target before unsubscribing it.

If dir/modgetonly exists, ezmlm-get will only allow moderators to retrieve data from the archive, even if dir/public exists. If dir/subgetonly exists, ezmlm-get will only allow subscribers to retrieve data from the archive.

If dir/nowarn exists, no warnings of any kind are sent by ezmlm-warn.

dir/key is a binary file used to create confirmation codes. Anyone who can guess the contents of dir/key can forge subscription requests. ezmlm-make does not put much effort into making dir/key difficult to guess; for better security, you should add some more secure random data to dir/key.

dir/flags contains the option flags that were passed to ezmlm-make when the list was created or last edited. It is used by programs that generate email messages to select which sections in text messages should be output. This is a new file introduced in version 5. Prior to this, the flags were stored in the first line of the dir/config file, along with other data.

dir/ezmlmrc contains the path to the directory in which the original ezmlmrc file was found. It is used to create alternate paths for text files.