Test::LectroTest::Property(3) Properties that make testable claims about your software

VERSION

version 0.5001

SYNOPSIS


use MyModule; # provides my_function_to_test
use Test::LectroTest::Generator qw( :common );
use Test::LectroTest::Property qw( Test );
use Test::LectroTest::TestRunner;
my $prop_non_neg = Property {
##[ x <- Int, y <- Int ]##
$tcon->label("negative") if $x < 0;
$tcon->label("odd") if $x % 2;
$tcon->retry if $y == 0; # 0 can't be used in test
my_function_to_test( $x, $y ) >= 0;
}, name => "my_function_to_test output is non-negative";
my $runner = Test::LectroTest::TestRunner->new();
$runner->run_suite(
$prop_non_neg,
# ... more properties here ...
);

DESCRIPTION

STOP! If you're just looking for an easy way to write and run unit tests, see Test::LectroTest first. Once you're comfortable with what is presented there and ready to delve into the full offerings of properties, this is the document for you.

This module allows you to define Properties that can be checked automatically by Test::LectroTest. A Property is a specification of your software's required behavior over a given set of conditions. The set of conditions is given by a generator-binding specification. The required behavior is defined implicitly by a block of code that tests your software for a given set of generated conditions; if your software matches the expected behavor, the block of code returns true; otherwise, false.

This documentation serves as reference documentation for LectroTest Properties. If you don't understand the basics of Properties yet, see ``OVERVIEW'' in Test::LectroTest::Tutorial before continuing.

Two ways to create Properties

There are two ways to create a property:
1.
Use the "Property" function to promote a block of code that contains both a generator-binding specification and a behavior test into a Test::LectroTest::Property object. This is the preferred method. Example: 

  my $prop1 = Property {
      ##[ x <- Int ]##
      thing_to_test($x) >= 0;
  }, name => "thing_to_test is non-negative";
2.
Use the "new" method of Test::LectroTest::Property and provide it with the necessary ingredients via named parameters: 

  my $prop2 = Test::LectroTest::Property->new(
      inputs => [ x => Int ],
      test   => sub { my ($tcon,$x) = @_;
                      thing_to_test($x) >= 0 },
      name   => "thing_to_test is non-negative"
  );

Both are equivalent, but the first is concise, easier to read, and lets LectroTest do some of the heavy lifting for you. The second is probably better, however, if you are constructing property specifications programmatically.

Generator-binding specification

The generator-binding specification declares that certain variables are to be bound to certain kinds of random-value generators during the tests of your software's behavior. The number and kind of generators define the ``condition space'' that is examined during property checks.

If you use the "Property" function to create your properties, your generator-binding specification must come first in your code block, and you must use the following syntax: 

  ##[ var1 <- gen1, var2 <- gen2, ... ]##

Comments are not allowed within the specification, but you may break it across multiple lines: 

  ##[ var1 <- gen1,
      var2 <- gen2, ...
  ]##

or 

  ##[
      var1 <- gen1,
      var2 <- gen2, ...
  ]##

Further, for better integration with syntax-highlighting IDEs, the terminating "]##" delimiter may be preceded by a hash symbol "#" and optional whitespace to make it appear like a comment: 

  ##[
      var1 <- gen1,
      var2 <- gen2, ...
  # ]##

On the other hand, if you use "Test::LectroTest::Property->new()" to create your objects, the generator-binding specification takes the form of an array reference containing variable-generator pairs that is passed to "new()" via the parameter named "inputs": 

  inputs => [ var1 => gen1, var2 => gen2, ... ]

Normal Perl syntax applies here.

Specifying multiple sets of generator bindings

Sometimes you may want to repeat a property check with multiple sets of generator bindings. This can happen, for instance, when your condition space is vast and you want to ensure that a particular portion of it receives focused coverage while still sampling the overall space. For times like this, you can list multiple sets of bindings within the "##[" and "]##" delimiters, like so: 

  ##[ var1 <- gen1A, ... ],
    [ var1 <- gen1B, ... ],
    ... more sets of bindings ...
    [ var1 <- gen1N, ... ]##

Note that only the first and last set need the special delimiters.

The equivalent when using "new()" is as follows: 

  inputs => [ [ var1 => gen1A, ... ],
              [ var1 => gen1B, ... ],
              ...
              [ var1 => gen1N, ... ] ]

Regardless of how you declare the sets of bindings, each set must provide bindings for the exact same set of variables. (The generators, of course, can be different.) For example, this kind of thing is illegal: 

  ##[ x <- Int ], [ y <- Int ]##

The above is illegal because both sets of bindings must use x or both must use y; they can't each use a different variable. 

  ##[ x <- Int             ],
    [ x <- Int, y <- Float ]##

The above is illegal because the second set has an extra variable that isn't present in the first. Both sets must use exactly the same variables. None of the variables may be extra, none may be missing, and all must be named identically across the sets of bindings.

Behavior test

The behavior test is a subroutine that accepts a test-controller object and a given set of input conditions, tests your software's observed behavior against the required behavior with respect to the input conditions, and returns true or false to indicate acceptance or rejection. If you are using the "Property" function to create your property objects, lexically bound variables are created and loaded with values automatically, per your input-generator specification, so you can just go ahead and use the variables immediately: 

  my $prop = Property {
    ##[ i <- Int, delta <- Float(range=>[0,1]) ]##
    my $lo_val = my_thing_to_test($i);
    my $hi_val = my_thing_to_test($i + $delta);
    $lo_val == $hi_val;
  }, name => "my_thing_to_test ignores fractions" ;

On the other hand, if you are using "Test::LectroTest::Property->new()", you must declare and initialize these variables manually from Perl's @_ variable in lexicographically increasing order after receiving $tcon, the test controller object. (This inconvenience, by the way, is why the former method is preferred.) The hard way: 

  my $prop = Test::LectroTest::Property->new(
    inputs => [ i => Int, delta => Float(range=>[0,1]) ],
    test => sub {
        my ($tcon, $delta, $i) = @_;
        my $lo_val = my_thing_to_test($i);
        my $hi_val = my_thing_to_test($i + $delta);
        $lo_val == $hi_val
    },
    name => "my_thing_to_test ignores fractions"
  ) ;

Control logic, retries, and labeling

Inside the behavior test, you have access to a special variable $tcon that allows you to interact with the test controller. Through $tcon you can do the following:
  • retry the current trial with different inputs (if you don't like the inputs you were given at first)
  • add labels to the current trial for reporting purposes
  • attach notes and variable dumps to the current trial for diagnostic purposes, should the trial fail

(For the full details of what you can do with $tcon see the ``testcontroller'' section of Test::LectroTest::TestRunner.)

For example, let's say that we have written a function "my_sqrt" that returns the square root of its input. In order to check whether our implementation fulfills the mathematical definition of square root, we might specify the following property: 

  my $epsilon = 0.000_001;
  Property {
      ##[ x <- Float ]##
      return $tcon->retry if $x < 0;
      $tcon->label("less than one") if $x < 1;
      my $sx = my_sqrt( $x );
      abs($sx * $sx - $x) < $epsilon;
  }, name => "my_sqrt satisfies defn of square root";

Because we don't want to deal with imaginary numbers, our square-root function is defined only over non-negative numbers. To make sure we don't accidentally check our property ``at'' a negative number, we use the following line to re-start the trial with a different input should the input we are given at first be negative: 

      return $tcon->retry if $x < 0;

An interesting fact is that for all values x between zero and one, the square root of x is larger than x itself. Perhaps our implementation treats such values as a special case. In order to be confident that we are checking this case, we added the following line: 

      $tcon->label("less than one") if $x < 1;

In the property-check output, we can see what percentage of the trials checked this case: 

  1..1
  ok 1 - 'my_sqrt satisfies defn of square root' (1000 attempts)
  #   1% less than one

Trivial cases

Random-input generators may create some inputs that are trivial and don't provide much testing value. To make it easy to label such cases, you can use the following from within your behavior tests: 

    $tcon->trivial if ... ;

The above is exactly equivalent to the following: 

    $tcon->label("trivial") if ... ;

HERE BE SOURCE FILTERS

The special syntax used to specify generator bindings relies upon a source filter (see Filter::Util::Call). If you don't want to use the syntax, you can disable the filter like so: 

    use Test::LectroTest::Property qw( NO_FILTER );

AUTHOR

Tom Moertel ([email protected])

INSPIRATION

The LectroTest project was inspired by Haskell's QuickCheck module by Koen Claessen and John Hughes: http://www.cs.chalmers.se/~rjmh/QuickCheck/.

COPYRIGHT and LICENSE

Copyright (c) 2004-13 by Thomas G Moertel. All rights reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

LAST SEARCHED