man Catmandu::Fix (3): a Catmandu class used for data crunching

SYNOPSIS

use Catmandu::Fix;
my $fixer = Catmandu::Fix->new(fixes => ['upcase("job")','remove_field("test")']);
or
my $fixer = Catmandu::Fix->new(fixes => ['fix_file.txt']);
my $arr = $fixer->fix([ ... ]);
my $hash = $fixer->fix({ ... });
my $it = Catmandu::Importer::YAML(file => '...');
$fixer->fix($it)->each(sub {
...
});
or
use Catmandu::Fix::upcase as => 'my_upcase';
use Catmandu::Fix::remove_field as => 'my_remove';
my $hash = { 'job' => 'librarian' , deep => { nested => '1'} };
my_upcase($hash,'job');
my_remove($hash,'deep.nested');

DESCRIPTION

Catmandu::Fixes can be used for easy data manipulation by non programmers. Using a small Perl DSL language end-users can use Fix routines to manipulate data objects. A plain text file of fixes can be created to specify all the routines needed to tranform the data into the desired format.

PATHS

All the Fix routines in Catmandu::Fix use a TT2 type reference to point to values in a Perl Hash. E.g. 'foo.2.bar' is a key 'bar' which is the 3-rd value of the key 'foo'.

A special case is when you want to point to all items in an array. In this case the wildcard '*' can be used. E.g. 'foo.*' points to all the items in the 'foo' array.

For array values there are special wildcards available:

 * $append   - Add a new item at the end of an array
 * $prepend  - Add a new item at the start of an array
 * $first    - Syntactic sugar for index '0' (the head of the array)
 * $last     - Syntactic sugar for index '-1' (the tail of the array)

E.g.

 # Create { mods => { titleInfo => [ { 'title' => 'a title' }] } };
 add_field('mods.titleInfo.$append.title', 'a title');
 # Create { mods => { titleInfo => [ { 'title' => 'a title' } , { 'title' => 'another title' }] } };
 add_field('mods.titleInfo.$append.title', 'another title');
 # Create { mods => { titleInfo => [ { 'title' => 'foo' } , { 'title' => 'another title' }] } };
 add_field('mods.titleInfo.$first.title', 'foo');
 # Create { mods => { titleInfo => [ { 'title' => 'foo' } , { 'title' => 'bar' }] } };
 add_field('mods.titleInfo.$last.title', 'bar');

Read more about the Fix language at our Wiki: <https://github.com/LibreCat/Catmandu/wiki/Fixes>

PUBLIC METHODS

new(fixes => [ FIX , ...])

Create a new Catmandu::Fix which will execute every FIX into a consecutive order. A FIX can be the name of a Catmandu::Fix::* routine, or the path to a plain text file containing all the fixes to be executed or a path to any executable if Catmandu::Fix::cmd is installed.

fix(HASH)

Execute all the fixes on a HASH. Returns the fixed HASH.

fix(ARRAY)

Execute all the fixes on every element in the ARRAY. Returns an ARRAY of fixes.

fix(Catmandu::Iterator)

Execute all the fixes on every item in an Catmandu::Iterator. Returns a (lazy) iterator on all the fixes.

fix(sub {})

Executes all the fixes on a generator function. Returns a new generator with fixed data.

log

Return the current logger. See Catmandu for activating the logger in your main code.

EXTEND

One can extend the Fix language by creating own custom-made fixes. Two methods are available to create an own Fix function:

  * Quick and easy: create a class that implements a C<fix> method.
  * Advanced: create a class that emits Perl code that will be evaled by the Fix module.

Both methods will be explained shortly.

Quick and easy

A Fix function is a Perl class in the "Catmandu::Fix" namespace that implements a "fix" method. The "fix" methods accepts a Perl hash as input and returns a (fixed) Perl hash as output. As an example, the code belows implements the "meow" Fix which inserts a 'meow' field with value 'purrrrr'.

    package Catmandu::Fix::meow;
    use Moo;
    sub fix {
        my ($self,$data) = @_;
        $data->{meow} = 'purrrrr';
        $data;
    }
    1;

Given this Perl class, the following fix statement can be used in your application:

    # Will add 'meow' = 'purrrrr' to the data
    meow()

Use the quick and easy method when your fixes are not dependent on reading or writing data from/to a JSON path. Your Perl classes need to implement their own logic to read or write data into the given Perl hash.

Fix arguments are passed as arguments to the "new" function of the Perl class. As in

    # In the fix file...
    meow('test123', -count => 4)
    # ...will be translated into this pseudo code
    my $fix = Catmandu::Fix::meow->new('test123', '-count', 4);

Using Moo these arguments can be catched with Catmandu::Fix::Has package:

    package Catmandu::Fix::meow;
    use Catmandu::Sane;
    use Moo;
    use Catmandu::Fix::Has;
    has msg   => (fix_arg => 1); # required parameter 1
    has count => (fix_opt => 1, default => sub { 4 }); # optional parameter 'count' with default value 4
    sub fix {
        my ($self,$data) = @_;
        $data->{meow} = $self->msg x $self->count;
        $data;
    }
    1;

Using this code the fix statement can be used like:

    # Will add 'meow' = 'purrpurrpurrpurr'
    meow('purr', -count => 4)

Advanced

The advanced method is required when one needs to read or write values from/to deeply nested JSON paths. One could parse JSON paths using the quick and easy Perl class above, but this would require a lot of inefficient for-while loops. The advanced method emits Perl code that gets compiled. This compiled code is evaled against all Perl hashes in the unput.The best way to learn this method is by inspecting some example Fix commands.

To ease the implementation of Fixed that emit Perl code some helper methods are created. Many Fix functions require a transformation of one or more values on a JSON Path. The Catmandu::Fix::SimpleGetValue provides an easy way to create such as script. In the example below we'll set the value at a JSON Path to 'purrrrr':

    package Catmandu::Fix::purrrrr;
    use Catmandu::Sane;
    use Moo;
    use Catmandu::Fix::Has;
    has path => (fix_arg => 1);
    with 'Catmandu::Fix::SimpleGetValue';
    sub emit_value {
        my ($self, $var, $fixer) = @_;
        "${var} = 'purrrrr';";
    }
    1;

Run this command as:

    # Set the value(s) of an existing path to 'purrr'
    purrrrr(my.deep.nested.path)
    purrrrr(all.my.values.*)

Notice how the "emit_value" of the Catmandu::Fix::purrrrr package returns Perl code and doesn't operate directy on the Perl data. The parameter $var contains only the name of a temporary variable that will hold the value of the JSON path after compiling the code into Perl.

Use Catmandu::Fix::Has to add more arguments to this fix:

    package Catmandu::Fix::purrrrr;
    use Catmandu::Sane;
    use Moo;
    use Catmandu::Fix::Has;
    has path => (fix_arg => 1);
    has msg  => (fix_opt => 1 , default => sub { 'purrrrr' });
    with 'Catmandu::Fix::SimpleGetValue';
    sub emit_value {
        my ($self, $var, $fixer) = @_;
        my $msg = $fixer->emit_string($self->msg);
        "${var} = ${msg};";
    }
    1;