SYNOPSIS

uqwk [a-plethora-of-options]

DESCRIPTION

uqwk is a program which collects all a user's unread news and/or mail, and formats it into a packet for offline reading, saving telephone and connect charges. Simple Offline Usenet Packet (SOUP), ZipNews, and the deprecated QWK packet formats are supported. Mail can be read from classic Unix mbox format mail files, MMDF mail files, or maildir mailboxes. News is either read directly from a locally available file system, or obtained using NNTP.

uqwk also accepts reply packets, so replies can be mailed or posted, depending whether the message is marked private (email) or public (news).

uqwk also supports a small offline command language, so the contents of the user's .newsrc file can be viewed and manipulated offline.

GENERAL OPTIONS

Note that many varieties of uqwk can be built: not all options are applicable to (and available) in every one. Specifically, the NNTP-related options are only available in uqwk builds using NNTP, the options related to reading news directly from disk only to non-NNTP builds. Furthermore, the options concerning scoring and NNTP authentication are only present if support for scoring, resp. NNTP authentication, was compiled in.

-p

Print options. uqwk will show its version number and the options that were compiled in, will examine all appropriate environment variables and command line options, print the values of all run-time options, then exit. This is useful to see what uqwk thinks it is going to do before you actually run it (and what setup it is offering).

-v

identical to "-p".

-?

same as "-v".

+r

Read only. Normally, uqwk will empty the user's mail spool file and update the user's .newsrc file to reflect the fact that mail and news have been read. If this switch is specified, uqwk will not touch these files. This is useful for testing.

-r

Do not execute in read-only mode. This is the default.

+m

Do mail. uqwk will process all mail in the user's mail spool file and convert it into an offline packet.

-m

Don't do mail. This is the default.

+n

Do news. Using the user's .newsrc file and the news system's active file, uqwk will collect all unread news articles in all subscribed newsgroups into the packet. This can generate a large number of messages and large packet files.

-n

Don't do news. This is the default.

+L

Operate in SOUP (also known as HDPF or SLNP) mode. The files comprising a Simple Offline Usenet Packet (SOUP), specifically, AREAS and *.MSG, will be created instead of a QWK packet. The reply file, if any, will be interpreted as an SOUP "replies" file instead of a QWK reply packet. This is the default.

-L

Do not operate in SOUP mode. If (and only if) neither SOUP nor ZipNews mode is selected, uqwk will operate in QWK mode.

+z

Operate in ZipNews mode. The files comprising a ZipNews news packet, specifically, "user.JN", "user.NWS", and "user.IDX", will be created instead of a QWK packet.

-z

Do not operate in ZipNews mode. If (and only if) neither SOUP nor ZipNews mode is selected, uqwk will operate in QWK mode. This is the default.

-Iposting_agent

Specifies the program used when posting messages (usually /usr/lib/news/inews). uqwk adds a `-h' flag when calling it, except in plain (non-expert) QWK mode.

+Vn

The value of n determines whether articles are validated before posting, whether this checking is verbose, and whether the result is respected or ignored:

  +V  Equivalent to `+V3' (default)
  +V0 No article validation
  +V1 Verbose article validation, ignoring the outcome
  +V2 Silent article validation, failing articles are
      not posted
  +V3 Verbose article validation, articles containing
      fatal errors are not posted

Whenever checking is performed, bad articles are appended to the file `dead.articles' in the uqwk home directory. This option is only available when uqwk was compiled with the `GNKSA' option, in which case the default value is `3'.

-V

Equivalent to `+V0', as defined above.

+x

Honor Xref header lines in news articles. When this switch is in effect, uqwk will only pack a crossposted article in the first newsgroup in which it is encountered. This can reduce the size of the packet but may introduce unexpected behaviour on the part of the offline reader. This is the default.

-x

Do not honor Xref lines.

+F

Ignore error on nonexisting news messages; proceed instead of aborting. This is the default.

-F

Signal an error and abort on missing news messages.

-Mmailspool

Look in the directory fImailspool for the mailbox file (or maildir mailbox). The default is /var/spool/mail.

-fmailfile

Look for mail in the file mailfile. The default is a file (or maildir mailbox) with the same name as the user's login name in fImailspool; however, the MAIL environment variable is used when set. This switch overrides the -M switch.

-aactivefile

Use activefile for the list of all available newsgroups and article numbers. The default is /usr/local/news/lib/active. This option is not available if uqwk obtains articles from a news server using NNTP.

-Snewsdir

Look for news articles in the directory newsdir. The default is /var/spool/news. This option is not available if uqwk obtains articles from a news server using NNTP.

-Nnewsrcfile

Use newsrcfile for the list of all newsgroups to which the user is subscribed and the list of article numbers which have been read. The default is $HOME/.newsrc.

-uusername

By default uqwk uses the getpwuid() system call to determine the proper user name to use. This switch overrides the results of that call.

-Hhomedir

By default, uqwk uses the getpwuid() system call to determine the user's home directory. The home directory is where the files comprising the offline packet will be created. It is also where uqwk looks for the .newsrc file. This switch may be used to override the results of the getpwuid() call.

-Bmaxblocks

Stop processing news articles if the size of the offline packet exceeds maxblocks 128-byte blocks. This is useful since large amounts of unread news can create large packets. Use a maxblocks value of zero to suppress this check. The default is 4096 blocks (half a megabyte).

-Dngfile

Process newsgroups in the order specified in ngfile. ngfile is expected to contain one newsgroup name per line.

-Usumfile

Operate in "summary" mode. Instead of packing articles, uqwk creates a summary file, sumfile, which consists of the newsgroup name, article number, and subject line of each unread article. All summarized articles are marked as read.

-Eselfile

Operate in "selection" mode. Instead of using the .newsrc file to determine which articles to pack, uqwk reads selfile, a file in the same format as the summary file. (The actual subjects are optional in selfile.) The idea is that a summary may be created by using -U, downloaded, edited or processed by additional software to select desired articles, uploaded, and fed to the -E switch for offline article selection.

-Rreplyfile

Process replyfile as a reply packet. Messages and articles created by the offline reader must be uploaded as a reply packet and then processed by this switch to be mailed or posted. If operating in SOUP mode, this file should be the REPLIES file. uqwk will remove replyfile when it has been processed unless it is running in read-only mode. If operating in ZipNews mode, this switch should specify the directory in which the ZipNews reply files reside. No default reply file is set.

-dhostname

Use the specified hostname when processing ZipNews reply packets. This should be the fully-qualified domain name of the Unix host. The default is "nowhere".

QWK MODE OPTIONS

+h

Include headers. Since the QWK specification places limits on the sizes of certain header fields such as To:, From:, and Subject:, uqwk can include all message and article headers in the body of the message so all fields are visible. This is the default. (This switch only applies to the QWK packet format.)

-h

Do not include headers in messages. (QWK only.)

-llen

Truncate the names of newsgroups to len characters. Some offline readers cannot handle long newsgroup names. Newsgroup names are truncated after having been translated. The default is 15 characters. Specify a value of zero for unlimited newsgroup name lengths. (QWK only.)

+e

Include an entry for every subscribed newsgroup in the CONTROL.DAT file. (QWK only).

-e

Do not include an entry in CONTROL.DAT for empty newsgroups. This is the default. (QWK only.)

+W

Activate the Blue Wave kludge. The Blue Wave QWK reader does not like the personal.ndx file that uqwk creates by default. If this switch is specified uqwk will create a regular ???.ndx file that Blue Wave likes. (QWK only.)

-W

Do not activate the Blue Wave kludge. This is the default.

+X

Operate in expert QWK mode. The QWK header is ignored and all header lines are taken from the body of the message. In the hands of an expert, this permits the creation of RFC822 compliant mail messages and RFC1036 compliant news articles using QWK readers, which is difficult or impossible otherwise in many cases. This is intended for people who really know what they're doing. (QWK only.)

-X

Do not operate in expert QWK mode. This is the default.

-ttablefile

Translate newsgroup names as specified in tablefile. Each line in tablefile is expected to have two fields separated by white space: The old name of the newsgroup and the name to which it will be translated. (QWK only.)

-bbbsname

Specify the BBS Name to be entered into the QWK packet. The default is "unknown BBS". (QWK only.)

-ccity

Specify the BBS City to be entered into the QWK packet. The default is "Anytown, USA". (QWK only.)

-Pphone

Specify the BBS Phone Number to be entered into the QWK packet. The default is "555-1212". (QWK only.)

-ssysop

Specify the BBS Sysop Name to be entered into the QWK packet. The default is "Joe Sysop". (QWK only.)

-ibbsid

Specify the BBS ID to be entered into the QWK packet. The BBS ID is important since it will be checked against the BBS ID string in any incoming reply packets. If the two do not match, the reply packet will not be processed. The BBS ID consists of an integer, a comma, and a string of less than nine characters, with no spaces. The default is "0,SOMEBBS". (QWK only.)

SCORING OPTIONS

+k

Perform kill/score processing. The global score file will be read (if it exists), as well as the score file for the current group, and each article will becored'. See the "USING SCORE FILES" section below. This is the default.

-k

Do not perform kill/score processing.

-Tn

Set global threshold to signed integer n. The default is 0.

-Kkilldir

Look for score files in killdir. The default is $HOME/.uqwk.

NNTP AUTHENTICATION OPTIONS

+Ausername:password

Perform NNTP authentication, specifying the username and password to use for that purpose. (NNTP only)

-A

Don't do NNTP authentication. This is the default. (NNTP only)

USING SCORE FILES

If support for scoring was compiled in, uqwk can "score articles" while collecting news, so that articles whose "score" is below a certain threshold can be excluded from the packet. The rules according to which articles are scored are specified by the user in so-called score files.

The format of these score files is based on the score files used by some offline readers (notably YARN). Each newsgroup can have a score file; its name is the name of the group. There is also a global score file, that applies to all groups, which is called "global".

Getting started

If you find some class of articles you wish to exclude from a particular newsgroup, use any text editor to create a file named after it. Each line of such a file contains a score, a place to look for a keyword or regular expression, and the keyword.

For example, to exclude all crossposted articles from talk.bizarre, create a file called "talk.bizarre" in the directory $HOME/.uqwk, with the following line in it:

  -10 Newsgroups: ,

The effect of this line in this file would be that all articles in talk.bizarre which' "Newsgroups: " header contains a `,' would get -10 added to their initial score of 0. With a threshold of 0, that means that unless other lines add enought to compensate, these articles will not be included.

Score file format

Each score file carries the name of the group it applies to, and consists of a series of score lines. Blank lines and lines beginning with `#' are ignored. Lines can have any of the following formats:

killthreshold n

specifies an integer score n that articles must meet or exceed in order to be included in the packet. This line is optional, if not present the global kill threshold is used. (which defaults to 0 if not set).

score header pattern

constitutes an actual scoring line; score is added to an article's score if the remainder of the line matches. To match, the specified pattern must match with the articles's indicated header. The specified pattern is always interpreted as a regular expression; for reasons of compatibility patterns can be preceded with the word "pattern".

The global score file is processed first. Its patterns are applied to all articles. Score lines are processed in sequence. An article that scores below the kill threshold will not be included.

Example alt.usenet.offline-reader score file

  # only include articles explicitly selected
  killthreshold 1

  # select messages about FurriNews
  +2 Subject: FurriNews
  # select messages about uqwk
  +2 Subject: uqwk

  # eliminate messages about crapware
  -666 Subject: microsoft
  -666 Subject: netscape

Limitations, additional notes

It should be noted that only headers are scanned for computing scores. Furthermore, all scanning is done using POSIX regex functions -- even when a simple substring search could have sufficed.

COMMAND LANGUAGE

If, while processing a reply packet, uqwk encounters a message to the username "UQWK", the body of the message will be interpreted as a small command language, used to display newsgroup names and subscribe or unsubscribe to newsgroups. The results of the execution of the commands will be mailed back to the originating user.

This is what the command language looks like:

HELP

List all the available commands.

SUBSCRIBE newsgroup

Subscribe to the named newsgroup.

UNSUBSCRIBE newsgroup

Unsubscribe from the named newsgroup. UNSUBSCRIBE ALL may be used to unsubscribe from all newsgroups. UNSUBSCRIBE ALL is also the only way to create a new .newsrc if it does not already exist.

CATCHUP newsgroup

Mark all articles in the named newsgroup as read.

GROUPS

List all newsgroups to which the user is currently subscribed.

ALLGROUPS

List all the available newsgroups and the number of articles in each one.

SHELL command

Execute the specified shell command. Anything written to standard output will be mailed back to the originating user.

NOTES

In QWK mode, it is possible to send mail to long email addresses by including, as the first thing in the body of the message, a line of the form:

  To: [email protected]

(The space after the colon is important.)

SAMPLE SCRIPTS

It may be convenient for users to employ simple shell scripts for preparing packets using uqwk, and for handling reply packets. A few examples might prove instructive.

getsoup - collect mail and news into a SOUP packet

  #!/bin/sh

  UQWKARGS="-B0 -m +L"

  # What to do?  Set variables to enable.
  #UQ_DO_MAIL=1
  UQ_DO_NEWS=1
  export UQ_DO_MAIL UQ_DO_NEWS

  # NNTPAUTH="yes please!"

  # When reading mail from the system mailbox, locking
  # will have to be done somehow.  If your Unix system
  # does not have lockfile, but does have /usr/ucb/mail,
  # use UQ_LOCKING="ucbmail".  See below.

  UQ_LOCKING="lockfile"

  #
  # No user servicable parts below (I hope)
  #

  prg=`basename $0`

  if [ -z "$UQ_DO_MAIL" -a -z "$UQ_DO_NEWS" ]
  then
    echo "$prg: nothing to do"                          >&2
    exit
  fi

  UQ_HOME_DIR=$HOME/.uqwk
  UQ_NRC_FILE=$HOME/.newsrc
  export UQ_HOME_DIR UQ_NRC_FILE

  # Do things in a nice, cozy subdirectory
  if [ ! -d $UQ_HOME_DIR ]
  then
    echo "$prg: $UQ_HOME_DIR does not exist, creating it"
    mkdir -p $UQ_HOME_DIR
    if [ ! -d $UQ_HOME_DIR ]
    then
      echo "$prg: failed to create $UQ_HOME_DIR"        >&2
      echo "$prg: bailing out"                          >&2
      exit 1
    fi
  fi
  cd $UQ_HOME_DIR

  soupfile=soup.zip

  if [ -f $soupfile ] ; then
    echo "Hmmm, $soupfile already exists."
    rm -i $soupfile
    if [ -f $soupfile ] ; then
      exit 1
    fi
  fi

  if [ -n "$UQ_DO_NEWS" ]
  then
    UQWKARGS="$UQWKARGS +n"
    # Get auth stuff (nntp auth mode only)
    if [ -n "$NNTPAUTH" ]
    then
      while [ -z "$UQ_AUTH_PASS" ]
      do
        echo "Authentication for server $NNTPSERVER;"
        echo "Username: $USER, password: \c"
        stty -echo
          read UQ_AUTH_PASS; echo
        stty echo
      done
      # Pass these through environment
      UQ_AUTH_USER=$USER; export UQ_AUTH_USER UQ_AUTH_PASS
    fi
  fi

  if [ -n "$UQ_DO_MAIL" ]
  then
    UQWKARGS="$UQWKARGS +m"

    # Read mail from the system mail box?
    if [ -z "$UQ_MAIL_FILE" ]
    then
      # uqwk does not do mailbox locking.  Locking options:
      #  - procmail's nifty "lockfile" utility
      #  - /usr/ucb/mail ("ucbmail")
      UQ_MAIL_FILE=/tmp/$prg.$USER.mbx.$$;
      export UQ_MAIL_FILE

      case $UQ_LOCKING in
      "lockfile")
        lockfile -l666 -ml; trap "lockfile -mu" 1 2 3 13 15
         mv $MAIL $UQ_MAIL_FILE
         lockfile -mu; trap "" 1 2 3 13 15
         ;;
      "ucbmail")
         ( echo 's1-$' $UQ_MAIL_FILE ; echo q )          |
           /usr/ucb/mail >/dev/null
         ;;
      *)
         echo "$prg: no, or unknown, locking method"     >&2
         echo "$prg: bailing out"                        >&2
         exit 1
         ;;
      esac
    fi
  fi

  #
  # Now we know how to call uqwk, proper args 'n' all
  #
  uqwk $UQWKARGS; err=$?
  if [ $err -ne 0 ]
  then
    echo "$prg: fatal error in uqwk"                    >&2
    echo "$prg: proceed with caution"                   >&2
  fi

  # Create the SOUP packet
  #
  if [ "`echo *.MSG`" = "*.MSG" ]
  then
    echo "Sorry, nothing to get."
    rm -f AREAS
    exit
  fi
  zip -m $soupfile AREAS *.MSG
  mv $soupfile $HOME

  echo "Done, you can upload
    <ftp://$USER@ftp.`domainname`/$soupfile>
  now."

putsoup - process a SOUP reply packet

  #!/bin/sh

  prg=`basename $0`
  rf=reply.zip; cd $UQ_HOME_DIR

  if [ ! -f $rf ]; then
    echo "$prg: no reply packet found"                  >&2
    exit 1
  fi

  unzip -ju $rf                                 2>/dev/null

  if uqwk -m -n +L -RREPLIES
  then
    echo "$prg: fatal uqwk error, proceed with caution" >&2
  fi

  echo "Done. If all went well, $rf can safely be deleted."
  rm -i $rf

ACKNOWLEDGEMENTS

uqwk was inspired by a conversation with Harry Lockwood <[email protected]>.

QWK format was designed by Mark "Sparky" Herring <[email protected]>. The creator of SOUP is Rhys Weatherley <[email protected]>. The creator of ZipNews is Jack Kilday <[email protected]>.

Thanks to Patrick Y. Lee <[email protected]> for the QWK documentation. Many thanks also to the beta-testers: Karl J. Vesterling <[email protected]> and Brian J. Coan <[email protected]>.

Many thanks to Ken Whedbee <[email protected]> for the NNTP modifications.

Many other people, too numerous to list here, contributed bug fixes and suggestions for improvement.

AUTHOR

Steve Belczyk, [email protected], [email protected].

Copyright (C) 1993-1994 by steve belczyk. Permission to use, copy, modify and distribute this software and its documentation for any purpose and without fee is hereby granted, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation. This software is provided "as is" without expressed or implied warranty.

MODIFIED BY

Jeroen Scheerder, [email protected], [email protected], [email protected].

MORE ACKNOWLEDGEMENTS

Jim Tittsler <[email protected]> came up with decent scoring code. Jos den Bekker <josdb@xs4all> provided valuable suggestions and bugfixes, most notably regarding error handling.