adt-testreport-onepackage(1) choose one package and test it with adt-run

SYNOPSYS

adt-testreport-onepackage [variable=value|settings-file]...

DESCRIPTION

adt-testreport-onepackage is a rather ad-hoc script which chooses a package to test, calls adt-run, and reports the results. In a service installation a-t-o is usually called from adt-testreport-runloop.

a-t-o has the following functionality:

a-t-o downloads the Sources or Packages file, chooses a package to test (see below), downloads its source, invokes adt-run, writes a report describing the outcome, and emails the report to some appropriate address.

WARNING - it is essential to reconfigure this program before use, as the supplied examples will send emails to people who will become very upset with you!

CONFIGURATION

Command-line arguments containing = are interpreted by a-t-o with eval; other arguments are sourced using ., in each case interpreted as bash script. Configuration fragment authors should bear in mind that a-t-o uses set -e.

Various variables can be set for useful effect. A selection of the most usual ones, with default values, can be seen at the top of the a-t-o script itself. Certain others are mentioned in this manpage. There is no exhaustive list and no documentation of the semantics.

PACKAGE AND TEST SELECTION

The configuration variable target specifies whether a source package, or a binary package, is to be tested. It should be set to source or binary-arch.

a-t-o maintains a record of when each package was last tested, what version was tested, and whether the package had any tests.

This record is used to choose which package should be tested next: packages which have changed since the last test are preferred, as are packages which didn't lack tests last time they were examined.

If the configuration sets pkg to a package name (for example if pkg=package is passed as an argument), this overrides a-t-o's package selection algorithm.

REPORTING

If interactive is set to true (the default), a-t-o will not redirect the output from adt-run and will not email anyone, although the start of the email which would have been sent is sent to stdout. If interactive is false, a-t-o sends an email to its administrator (if a-t-o considers that the testing machinery is broken, or if maintainer_email_override is set), or otherwise to the package maintainer.

No email is sent if there were no tests, or all the tests skipped or passed.

The emailed report contains a summary of the outcome of the tests, and the output from adt-run. Additionally, if an email is sent, the build trees, test results, and so forth, will be uploaded to the location specified with the dest* options (see the onepackage-config example configuration file).

Some vague progress information is printed to a-t-o's stdout.

AUTOMATIC BUG REPORTING

With suitable configuration, a-t-o can be made to automatically file bugs when it detects a problem with a package.

It is necessary to avoid the submission of multiple reports for the same problem. This is done by fetching somehow a list of the open bug reports from the relevant bug system and extracting a list of the source package names. These are to be listed in the file named by the suppresspackages configuration variable. Testing frequency of these source packages is reduced, and when a problem is found the email is not sent.

When the system does decide to send an email about a package it will normally email the maintainer as determined from the package's control file. To submit bugs instead, set maintainer_email_override to the submission email address for the bug system. You must also set email_sourcepackage_headerandemail_binarypackage_header to appropriate templates. These templates specify what information should be prepended at the top of the body (not the RFC822 header) of the mail to be sent. In these templates @p stands for the package name being tested (the source package name for source packages or the binary package name for binary package tests), @v stands for the version being tested, @s stands for the source package name (even if a binary package generated by that source is being tested) and @a stands for a single at sign @.

If you an appropriately useable GPG signing key is available, you may specify email_signing_key which is passed to gpg's -u option. Otherwise the email is not signed.

FILES

The tmp=tmp and var=var configuration variables name two directories which must belong solely to this instance of a-t-o.

tmp will be erased by a-t-o at startup and then used for temporary files. It is not cleaned up afterwards; it is assumed that the next run of a-t-o will do that. tmp should not be made to refer to a symlink or mountpoint as a-t-o would break the link or attempt to remove the mountpoint; specify a subdirectory instead.

var is used to record the state, including the information about each package used for package selection. var/log contains a one-line record of each test run and may need to be rotated.

tmp and var default to tmp and var in a-t-o's starting directory.

EXIT STATUS

0      a package was apparently tested

20     adt-run reported an unexpected failure

other  other unexpected failures including bad usage, disk full, etc.

When a-t-o is run in a loop, the loop should stop pending administrator intervention if a-t-o exits nonzero.

FILES

tmp (default: ./tmp)
Temporary directory for working files. Do not specify /tmp.
var (default: ./var)
Storage directory for package testing history, score reports, logfiles, etc.
var/lastinfo-target
Information about the last test and outcome for each package. This is used for package scoring and selection.
var/scores-target
Report giving the score for each package, and the factors which influenced that score.
var/emailed/last-pkg,emailaddress
The summary of the last message sent to emailaddress about pkg. If the configuration variable suppressrepeatedemails is true then this is used to ensure that a maintainer (or the test installation operator) is only emailed when the situation for a particular package changes.
var/emailed/diff-pkg
Changes to the summary for the last email sent regarding pkg. This can be used to find out why an email was sent.
var/log
Logfile from a-t-o. If a-t-runloop is being used its stdout output often makes a better logfile.
destrsynchead/destdirfin/pkg
Full logfile and other information is rsync'd here. It is assumed that this area will be published via a webserver.

BUGS

It is essential not to use the example configuration! This configuration is likely to send inappropriate emails!

There is no documentation for the variable settings, and no error handling for detecting attempts to set incorrectly named variables.

The tuning parameters for the package selection algorithm cannot be adjusted. This algorithm is not hugely sophisticated. There should be a facility for explicitly requesting retests (other than setting pkg); there should be some kind of manual prioritisation queue.

Distributions whose management systems permit archive skew (that is, distributions whose contents sometimes contains unsatisfiable dependencies) cause pathological behaviour. a-t-o will prefer to test the newly-changed packages and immediately detect and report the archive skew as test failures. It would be better if it would detect this situation and not report a failure unless the problem persisted. And there ought to be an option to have it not try to test a package until its version has been stable for a nominated period.

If multiple distributions (multiple values of distro) are used, a-t-o will conflate their last-tested versions. It should use separate files.

Cross-architecture testing (where host and target have differing Debian architectures) has not been tested and is likely to have bugs both in interfaces and implementation.

a-t-o should be capable of automatically closing bug reports if it retests a package and finds that the bug is fixed.

The script lacks flexibility in many respects; many additional configuration settings and hooks are needed.

There is no facility for sharing test queues across multiple machines. This means that it is difficult to share the testing load.

AUTHORS AND COPYRIGHT

This manpage is part of autopkgtest, a tool for testing Debian binary packages. autopkgtest is Copyright (C) 2006-2007 Canonical Ltd and others.

See /usr/share/doc/autopkgtest/CREDITS for the list of contributors and full copying conditions.