DESCRIPTION

Mail Avenger is a highly-configurable MTA-independent SMTP (Simple Mail Transport Protocol) server designed to let you filter and fight SPAM before accepting incoming mail from a client machine. avenger is the script run on behalf of each user to decide whether to accept incoming mail.

When a client attempts to send mail to a user on the system, the avenger SMTP daemon, asmtpd, runs avenger to process the file .avenger/rcpt in the user's home directory. That file, a shell script with access to special functions, determines how the SMTP server should proceed. The possible outcomes are:

• Provisionally accept the mail, falling back to system-default rules
• Accept the mail immediately with no further checks
• Reject the mail immediately
• Defer the mail, telling the client to re-send it later
• Redirect the processing to another local name. The name can be another email address belonging to the current user, or an email address belonging to the special AvengerUser user. In the later case, avenger will be re-run with a different user ID, and hence can, for example, employ utilities that maintain state across multiple users (assuming they all redirect processing the same way).
• Run a ``bodytest'' rule. With this outcome, the the SMTP transaction continues on to receive the entire contents of the mail message, after which a program is run on the contents of the mail message. That program can decide, based on the contents, whether to accept, reject, defer, or silently discard the message.

Mail Avenger should typically be configured to have a Separator character, allowing each user to maintain multiple email addresses. With sendmail, Separator is typically "+", with qmail it is typically "-". If the separator is "+", then any email sent to user+ext@your-host will be processed by files in user's .avenger directory.

Avenger first checks for a file named rcpt+ext in a user's .avenger directory, then for rcpt+default. If ext itself contains the separator character, for example user+ext1+ext2@your-host, avenger will check first for rcpt+ext1+ext2, then for rcpt+ext1+default, then for rcpt+default. The same algorithm is extended for arbitrarily many separator characters. (If separator is "-", simply replace "+" with "-" throughout the above description, including in the names of files such as rcpt-default.)

If mail is rejected by the recipient checks but the sender address of a message is local and UserMail is 1 in asmtpd.conf (which is not the default), then before rejecting mail, avenger will be run on behalf of the sending user. In this case, the address will be parsed as above, but avenger will look for rules in files beginning mail instead of rcpt. This mechanism can be used by local users who want to relay mail through the server from an untrusted IP address.

Using the mail configuration files, each user can, for instance, configure a mail+... file to accept mail from an IP address he or she trusts, even if that address is not trusted by all users. (Alternatively, using tools such as macutil, a user might set up relaying of mail in which the envelope sender contains a cryptographic code, checked by the mail+... script.)

Error output of an avenger script rcpt+ext or mail+ext is redirected to a file called log+ext in the same directory, for use in debugging.

AVENGER SYNTAX

Avenger configuration files are simply shell scripts, using the syntax described in sh(1). Each line of the file contains a variable assignment, command, or function to run. Scripts can additionally make use of a number of avenger-specific functions and variables. This section describes avenger functions. The next two sections describe variables.

errcheck

Certain error conditions result in Mail Avenger rejecting mail by default, unless the message is explicitly accepted through an accept or successful bodytest check. These conditions are indicated by the MAIL_ERROR environment variable described below. If your script either rejects mail or falls through to the default behavior, there is often no reason to run tests on a message that will end up being rejected either way. errcheck exits immediately with the default error if the default would be to reject or defer the mail.

accept [message]

Immediately accepts the message (without falling back to any default rules). If message is supplied, it will be returned to the SMTP client. The default message is "ok".

reject [message]

Reject the mail, with message. (The default message is "command rejected for policy reasons").

defer [message]

Reject the mail with a temporary error code, so that a legitimate mail client will attempt to re-send it later. The default for message is "temporary error in processing".

bodytest command [arg ...]

Accept the current SMTP "RCPT" command. However, once the whole mail message has been received with the SMTP "DATA" command, run command with the message as its standard input. Depending on the exit status of command return to the client's "DATA" command either success, temporary, or permanent failure. Exit code 0 means accept the mail, 100 means reject, 111 means reject with a temporary error code (i.e., defer the mail). See the description of bodytest in the asmtpd/avenger interface description for more information on bodytest (since this function directly invokes bodytest in asmtpd).

Error output from command will be redirected to the same log file as output from the rcpt+... avenger script invoking the bodytest function. Standard output of command will be included as a diagnostic the bounce message if the exit code defers or rejects the mail.

Note that command and the arguments passed to bodytest will be run by the shell. Thus, it is important not to pass any arguments that might contain shell metacharacters such as ">" and "$".

redirect local

Finish processing, and re-run avenger as if mail were being sent to a different username local (possibly belonging to the special AvengerUser user). See the description of redirect in the asmtpd/avenger interface description for more information on redirect (since this function directly invokes redirect in asmtpd).

greylist [sender-key]

This command defers mail the first time mail is received from a particular sender at a particular IP address. However, after a certain interval, greylist_delay, if the client re-sends the mail, it will be accepted. Furthermore, from that point on, all mail will be immediately accepted from that sender and IP address, unless the sender stops sending mail for a period of greylist_ttl2 or more. If, however, after sending the initial, defered piece of mail, the client does not try again within a period of greylist_ttl1, then any record of the client will be erased, and the next time it tries to send mail it will be defered again.

The parameters can be tuned by setting variables in the script. The default values are:

    greylist_delay=30m  # Time to wait before allowing message
    greylist_ttl1=5h    # How long to remember first-time senders
    greylist_ttl2=36D   # How long to remember ok senders

m means minutes, h hours, and D days. For a complete list of allowed suffixes, see the documentation for dbutil(1) (in particular for the --expire option).

sender-key, if supplied, is used to identify the sender. The default value is "$CLIENT_IP $RECIPIENT $SENDER". If, for example, you wanted to record only the first 24-bits of IP address and didn't care about the recipient, you could use the command:

greylist ``${CLIENT_IP%.*} $SENDER''

setvars

All functions that set a variable by means of an external query to asmtpd are performed asynchronously. setvars actually waits for results and sets the values of those variables. In this way, a number of potentially slow requests (such as DNS lookups) can be initiated concurrently, and their latencies overlapped. However, one must remember to call setvars, or else variables that should contain the results of operations will remain unset.

dns var type domain-name

Performs a DNS lookup of domain-name for records of type type, and assigns the result to variable var when you call setvars. type must be one of a, mx, ptr, or txt (lower-case only).

rbl [-ipf] var domain

Looks up the current mail sender in a real-time blackhole list (RBL). domain is the domain name of the RBL (e.g., "bl.spamcop.net"). If the sender is listed, set var to the result of the DNS lookup when you next call setvars. -i looks up the sender's IP address (the default if no options are specified). -p looks up the sender's domain name (verified DNS PTR record). -f looks up the envelope sender domain name in the RBL.

spf0 var [spf-mechanism ...]

spf var [spf-mechanism ...]

Tests the sender against an arbitrary query formulated in the SPF language. This is a powerful way to whitelist or blacklist particular senders. For example, suppose you want to accept any mail from machines in the list maintained by trusted-forwarder.org, accept mail from any machine name ending "yahoo.com" reject any mail from users in the spamcop RBL, and for other users fall back to the default system-wide rules. You might use the following rcpt file:

    spf MYSPF +include:spf.trusted-forwarder.org \
        +ptr:yahoo.com -exists:%{ir}.bl.spamcop.net ?all
    setvars
    case "$MYSPF" in
        pass)
            accept "I like you"
            ;;
        fail)
            reject "I don't like you"
            ;;
        error)
            # Note, could instead fall through to default here
            defer "Temporary DNS error"
            ;;
    esac

Note that commands spf0 and spf are synonymous, but spf is deprecated, because in a later release of Mail Avenger spf will become synonymous with spf1.

spf1 var [spf-mechanism ...]

Performs the same tests as the spf directive, but returns the result strings None, Neutral, Pass, Fail, SoftFail, TempError, and PermError instead of none, neutral, pass, fail, softfail, error, and unknown.

AVENGER VARIABLES

These variables are set by the avenger script. In addition, asmtpd sets a number of environment variables before running avenger. These are documented in the next section, ENVIRONMENT.

FILEX

The extension on the file currently being processed. For example, if file rcpt+ext is being processed, will be set to "+ext". Empty when processing just rcpt (or mail). May also contain default when a default rule file for some suffix is being run.

PREFIX

SUFFIX

Assuming the separator is "+", when processing a file rcpt+base+default or mail+base+default, PREFIX is set to base, while SUFFIX is set to the portion of the name for which default was substituted. When the file does not end with default, SUFFIX is empty. When the file is just rcpt with no extension, both PREFIX and SUFFIX are empty. When SUFFIX itself contains a "+" character, SUFFIX1 contains to the part of SUFFIX after the first "+" character, SUFFIX2 contains the part after the second "+", and so on for each "+" character in suffix.

AVENGER/ASMTPD INTERFACE

avenger is just a simple shell script. You can inspect the file to see what it is doing. Most of the interesting operations happen in either asmtpd, or in external programs spawned from avenger. This section documents the interface between asmtpd and avenger.

avenger inherits a unix-domain socket connected to asmtpd on its standard input and output. It sends commands to asmtpd over this socket, and similarly reads replies from it. In order to avoid mixing messages to and from asmtpd with the output of other programs you run, however, the avenger shell script reorganizes its file descriptors so that all communication to and from asmtpd happens over file descriptor number 3.

Each command consists of a single line, followed by a newline (except the return command, which can optionally take multiple lines). There may or may not be a reply, possibly depending on the outcome of the command. Most replies consist of zero or more lines of the form

VARIABLE=value

VARIABLE is typically a variable name that was supplied as part of the command. The avenger shell script records results by setting the environment variable VARIABLE to value, so that it can be accessed by subsequent lines of the script.

Replies are sent in the order in which the corresponding commands were received. However, asmtpd executes requests asynchronously. Thus, one can perform several concurrent operations (such as DNS requests or SPF tests) by simply writing multiple commands to asmtpd before receiving any of the responses.

The "." command is a no-op, but asmtpd echoes the "." back to avenger as the reply. This allows one to synchronize the avenger process's state after issuing one or more commands. For example, one might issue several DNS lookups to check various RBLs (real-time blackhole lists), then issue a . command, then wait for replies. When the . comes back, all previous commands will also have completed. The avenger setvars command simply sends a ".", then loops until it reads back the ".", setting variables from any previous commands whose replies it reads in the process.

The following commands are available:

.

The . command is simply echoed back by asmtpd.

bodytest command

Ends the current avenger script. Specifies that asmtpd should receive the entire body of the message, then run command (under the same user ID as the current avenger script) with the entire mail message as its standard input. asmtpd then replies to the SMTP "DATA" command based on the exit status of command as follows:

0

If command exits with status 0, asmtpd will reply to the "DATA" command with success (SMTP code 250), and will pass the message to sendmail (or whatever you have configured as Sendmail in asmtpd.conf) for delivery.

99

If command exits with status 99, asmtpd will still reply to the "DATA" command with a successful 250 reply code, but will not spool the data. Either command must have done something with the data, or the message will be lost.

100 (also 64, 65, 70, 76, 77, 78, 112)

If command exits with status 100 (or any of the above exit statuses), avenger will reject the mail with a hard SMTP error (code 554). If command wrote output to its standard output, this output will be passed back to the mail client. Otherwise, asmtpd will supply the text ``message contents rejected.''

111 (or any other exit status)

If command exits with status 111, the result is the same as exit status 100, except that asmtpd will use a temporary error code (451) instead of 554.

signal

If command exits abnormally because of a signal, asmtpd will also use 451, but in this case will not pass the program's output back to the client. It will instead pass back a description of the problem.

Note that asmtpd can only run one bodytest command per message. If there are multiple recipients of a message, all must run the same bodytest under the same user ID. If two users wish to run different bodytest commands, or even run the same command under different user IDs, asmtpd will defer the second SMTP "RCPT" command with the message:

452 send a separate copy of the message to this user

This will cause the mail client to re-send the message later to the second user. To avoid forcing clients to send multiple copies of messages, you can place bodytest commands in system wide files (such as the default rule file), or use a redirect command to redirect to the AvengerUser, so that commands for multiple users can be run under the AvengerUser user ID.

Note that file descriptor 0 inherited by command is opened for both reading and writing. Thus, it is possible to modify the message before it is spooled by the local MTA. The command edinplace(1) is useful for running messages through spam filters that annotate messages before spooling them.

dns-a VARIABLE domain-name

Requests that asmtpd perform a DNS lookup for A (IPv4 address) records on domain-name. If such an A record exists, the reply is a list of one or more IP addresses:

VARIABLE=IP-address ...

If no such A record exists, the reply is simply:

VARIABLE=

With the standard avenger script, this sets VARIABLE to the empty string. If there is a temporary error in DNS name resolution, there is no reply, and hence with the default avenger script VARIABLE will remain unset.

When checking such things as RBLs, it is advisable not to reject mail because of a temporary DNS error. You can use the shell construct ${VARIABLE-default}$ to return $VARIABLE when VARIABLE is set, and default when VARIABLE is not set. Similarly ${VARIABLE+set} returns set if VARIABLE is set, and the empty string otherwise.

For example, if bad-senders.org contained an RBL of undesirable sender hosts:

    echo dns-a BADSENDER "$SENDER_HOST".bad-senders.org >&3
    setvars
    test -n "$BADSENDER" && reject "$SENDER_HOST is a bad sender"
    test -z "${BADSENDER+set}" \
        && defer "$SENDER_HOST.bad-senders.org: DNS error"

Note that when using the avenger script, there is already a function rbl to check RBLs.

dns-mx VARIABLE domain-name

Similar to dns-a, but looks up MX records. A successful reply is of the form:

VARIABLE=priority-1:host-1 [priority-2:host-2 ...]

Where priority-1 is the MX priority of host-1. As before, an empty string indicates no MX records exist, and no reply indicates an error.

dns-ptr VARIABLE IP-address

Returns a list of verified DNS hostnames for IP-address. As before, an empty string for VARIABLE indicates no PTR records exist, and no reply indicates an error.

dns-txt VARIABLE domain-name

Similar to the other dns commands, but looks up a record of type TXT. If multiple TXT records exist, returns only one. Places some restrictions on the TXT records, for example will not return one that contains a newline character.

netpath VARIABLE IP-address

Maps out the network hops to IP-address (this is similar to the traceroute system utility, but more efficient). The reply is of the form:

VARIABLE=#hops hop1 hop2 ...

#hops is the total number of network hops to IP-address if asmtpd can figure this out. (It won't always be able to if IP-address is behind a firewall.) If asmtpd cannot figure this out, the value is -1. hop1 and the remaining arguments are the addresses of routers along the way to IP-address.

redirect local

Terminates the current avenger process, and instead processes the mail as though it is being sent to local. This command is only available in ``rcpt'' mode, as opposed to ``mail'' mode (in which asmtpd runs avenger to see if it should relay mail for a local user on a non-local client machine).

local can be a local user name, or a local user name followed by the separator character and an extension. The name is mapped using the aliases (specified by AliasFile in asmtpd.conf).

Note that while the AvengerUser user can redirect to other users, ordinary users can only redirect to themselves or the AvengerUser.

return code explanation

or

return code-explanation

code-explanation

code explanation

Specifies the SMTP reponse desired. Also avoids further processing of the message with system-wide default rulesets (as typically happens when avenger simply exits with status 0). code must be a three digit number beginning 2, 4, or 5. (usually 250 for success, 451 to defer mail, and 554 to reject mail).

The first form of this command (with a space between code and explanation) gives a single line explanation along with the result code. In the second form, avenger specifies a multi-line response. In this case all but the last line must contain a - between the code and explanation, while the last line must contain a space. (Note that the return keyword only appears on the first line; after starting to issue a return command, no further commands can be issued.)

spf VARIABLE SPF-mechanism ...

spf0 VARIABLE SPF-mechanism ...

spf1 VARIABLE SPF-mechanism ...

Evaluates the mail client based on SPF mechanisms. It will return:

VARIABLE=disposition

where, for spf0, disposition is one of: none, neutral, pass, fail, softfail, error, or unknown (though the disposition none is actually impossible). For spf1, the equivalent disposition names are None, Neutral, Pass, Fail, SoftFail, TempError, PermError. (Currently spf is a synonym for spf0, but it is recommended that you avoid using spf as in a future release it may become an alias for spf1.)

As an example, suppose that your username is "joe", Separator is "+", and you have subscribed to a number of yahoo mailing lists using email address "joe+yahoo". If spammers started sending mail to "joe+yahoo", you would want to reject all mail to that address except that originating from yahoo's computers. Yahoo's computers might correspond to anything ending ".yahoo.com" or sharing a 24-bit IP-address prefix with any of yahoo.com's MX records. This can be accomplished with the following script in $HOME/.avenger/rcpt+yahoo:

    echo spf YAHOO ptr:yahoo.com mx:yahoo.com/24 -all >&3
    setvars
    case "$YAHOO" in
    fail)
        reject "Sorry, this private alias for Yahoo lists only"
        ;;
    error)
        defer "Sorry, temporary DNS error"
        ;;
    esac

EXAMPLES

If you never use your email address as an envelope sender, you can reject all bounces to that address with these commands in your rcpt file:

    test -z "$SENDER" \
        && reject "<$RECIPIENT> not a valid sender;" \
        " should not receive bounces"

The following script runs spamassassin (a popular spam filter, available from <http://www.spamassassin.org/>) on the body of a message, unless the sender of the message has an SPF disposition of pass or is already going to be rejected by default.

    # The next line immediately falls through to the default reject
    # disposition when mail has an SPF disposition of fail or the
    # sender does not accept bounce messages.
    errcheck
    test "$SPF" = pass \
        || bodytest edinplace -x 111 spamassassin -e 100

The following script immediately accepts any mail from any machine at MIT or NYU (provided MAIL_ERROR is not set), ``greylists'' machines not in one of those domains, and if the greylist passes, falls through to the the default, system-wide rules:

    errcheck
    spf TRUSTED ptr:nyu.edu ptr:mit.edu ?all
    setvars
    test pass = "$TRUSTED" && accept Trusted sender OK
    greylist_delay=5m
    greylist

The following script rejects mail from clients that have issued an SMTP ``POST'' command (which doesn't exist) or used aggressive, premature pipelining of commands. If the client put a space after the colon in the MAIL FROM: or RCPT TO: SMTP commands, it greylists the message using a key that includes the SYN fingerprint and first 24-bits of the IP address. If the SPF disposition of the message is error, it defers the message. If the SPF disposition of the message is softfail or none, it runs the body of the message through spamassassin.

    errcheck
    test -n "$CLIENT_POST" -o -n "$CLIENT_PIPELINING" \
        && reject "no spam please"
    test -n "$CLIENT_COLONSPACE" \
        && greylist "${CLIENT_IP%.*} $CLIENT_SYNFP $SENDER"
    case "$SPF" in
        error)
            defer "Temporary error in SPF record processing"
            ;;
        softfail|none)
            bodytest edinplace -x 111 spamassassin -e 100
            ;;
    esac

If you set your MACUTIL_SENDER environment variable to be "user+bounce+*@your.host.com" and send mail with macutil --sendmail, you can create the following rcpt+bounce+default to accept mail only to valid bounce addresses.

    macutil --check "$SUFFIX" > /dev/null \
        || reject "<$RECIPIENT>.. user unknown"

In conjunction with this script, you may want to reject bounce messages to your regular email addresss with your rcpt script, as described in the first example.

This example is slightly more complicated, and shows how to use a bodytest to reject mail based on message contents. The goal of this set-up is to check each message with the ClamAV anti-virus software (from <http://www.clamav.net/>) and the spamassassin mail filter. If the message contains a virus or is flagged as spam, it should be rejected with an explanation of the problem. We construct a shell script, $HOME/.avenger/body, to run these tests on message bodies. The script can be invoked with the line

bodytest $HOME/.avenger/body

in your $HOME/.avenger/rcpt file. Or, alternatively the script could be configured to run in the system-wide /etc/avenger/default file (in which case you want to make sure that the AvengerUser can write its own home directory, so as to store spamassassin files). The script is as follows:

    #!/bin/sh
    out="`clamscan -i --no-summary --mbox -  2>&1`"
    if test "$?" = 1; then
        echo This message appears to be infected with a virus
        printf "%s\n" "$out" \
            | sed -e '/Warning:/d' -e 's/^[^:]*: //' | sort -u
        exit 100
    fi
    out="`edinplace -x 111 spamassassin -e 100`"
    case "$?" in
        0)
            exit 0
            ;;
        100)
            echo Sorry, spamassassin has flagged your message as spam
            while read a b c; do
                test "$a $b" = "Content analysis" && break
            done
            read a
            read a
            read a
            while read a b c; do
                case "$a" in
                "")
                    break
                    ;;
                -*)
                    ;;
                [0-9]*)
                    printf "  %s\n" "$c"
                    ;;
                *)
                    printf "    %s\n" "$a $b $c"
                    ;;
                esac
            done
            exit 100
            ;;
        *)
            if test -n "$out"; then
                echo spamassassin failure:
                printf "%s\n" "$out"
            else
                echo system error in spamassassin
            fi
            exit 111
            ;;
    esac

The first half of this script runs the clamscan virus checker, storing the output in variable out. clamscan exits with code 1 when a virus is found, exits 0 on success, and uses other error codes to indicate various system errors. We only want to reject mail if clamscan exits with code 1. When this happens, we take the output of clamscan, format it in a more pleasing way (stripping out warnings), and send it to standard output. An example of an SMTP transaction using this bodytest and detecting a virus will look like this (tested with the special EICAR test string that flags a positive with most virus checkers):

    DATA
    354 enter mail, end with "." on a line by itself
    Subject: eicar test
    X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*
    .
    554-This message appears to be infected with a virus
    554 Eicar-Test-Signature FOUND

If the virus check fails, the script runs the message through spamassassin to check for spam. Note that spamassassin modifies the mail message, so that we must run it with edinplace. Note also that clamscan will read to the end of the input file, but this is okay since edinplace rewinds its standard input. We use the -e flag to tell spamassassin to exit 100 on spam. Then, if spamassassin exits 0, we accept the mail. If it exits with anything but 100, something went wrong and we temporarily defer the mail. Note that it might also be possible to accept the mail at this point, but since spamassassin edits the file in place, the message may be truncated if spamassassin exits unexpectedly.

If spamassassin exits 100, we reject the mail. We also report on why spamassassin has rejected the mail. Here again we take advantage of the fact that edinplace rewinds its standard input both before and after processing a message. Because the file descriptor has been rewound, we can start processing the message one line at a time with the shell script. Spamassassin by default (if you have not configred it with "report_safe 0") contains a spam report like this:

 Content analysis details:   (11.7 points, 5.0 required)
  pts rule name        description
 ---- --------------- --------------------------------------------------
  1.0 RATWARE_RCVD_AT Bulk email fingerprint (Received @) found
  4.2 X_MESSAGE_INFO  Bulk email fingerprint (X-Message-Info) found
  0.0 MONEY_BACK      BODY: Money back guarantee
  0.5 BIZ_TLD         URI: Contains a URL in the BIZ top-level domain
  0.6 URIBL_SBL       Contains a URL listed in the SBL blocklist
                      [URIs: crocpeptide.biz]
  0.5 URIBL_WS_SURBL  Contains a URL listed in the WS SURBL blocklist
                      [URIs: crocpeptide.biz]
 ...

We skip over the headers, and for each result, print it to the SMTP session. Negative/whitelist results (those starting -), we do not report, and comment lines (not starting with a number) we print indented. A typical SMTP session looks like this (using the special GTUBE test line that triggers spam filters):

    DATA
    354 enter mail, end with "." on a line by itself
    Subject: gtube test
    XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X
    .
    554-Sorry, spamassassin has flagged your message as spam
    554-  Missing Date: header
    554   BODY: Generic Test for Unsolicited Bulk Email

Here's an example of how to use SSL client certificates for authentication. If you have a private CA with common name ``My CA'' that signs the certificates of all your authorized mail clients, you can place the following in /etc/avenger/relay to permit those clients to relay:

    test "My CA" = "$SSL_ISSUER" \
        && accept "Relaying permitted for client $SSL_SUBJECT"
    reject "relaying denied"

FILES

/usr/local/libexec/avenger, /etc/avenger/default, $HOME/.avenger/rcpt, $HOME/.avenger/rcpt* $HOME/.avenger/mail, $HOME/.avenger/mail*

        bodytest "echo $VAR > $PWD/log"

        bodytest echo `escape "$VAR"` ">" $PWD/log

AUTHOR

David Mazieres