SYNOPSIS
package Catmandu::Importer::Hello;
use Catmandu::Sane;
use Moo;
with 'Catmandu::Importer';
sub generator {
my ($self) = @_;
state $fh = $self->fh;
my $n = 0;
return sub {
$self->log->debug("generating record " . ++$n);
my $name = $self->readline;
return defined $name ? { "hello" => $name } : undef;
};
}
package main;
use Catmandu;
my $importer = Catmandu->importer('Hello', file => '/tmp/names.txt');
$importer->each(sub {
my $items = shift;
.
.
.
});
# Or on the command line
$ catmandu convert Hello to YAML < /tmp/names.txt
# Fetch remote content
$ catmandu convert JSON --file http://example.com/data.json to YAML
DESCRIPTION
A Catmandu::Importer is a Perl package that can import data from an external source (a file, the network, ...). Most importers read from an input stream, such as STDIN, a given file, or an URL to fetch data from, so this base class provides helper method for consuming the input stream once.Every Catmandu::Importer is a Catmandu::Fixable and thus inherits a 'fix' parameter that can be set in the constructor. When given then each item returned by the generator will be automatically Fixed using one or more Catmandu::Fixes. E.g.
my $importer = Catmandu->importer('Hello',fix => ['upcase(hello)']);
$importer->each( sub {
my $item = shift ; # Every item will be upcased...
} );
Every Catmandu::Importer is a Catmandu::Iterable and inherits the methods ("first", "each", "to_array"...) etc.
CONFIGURATION
- file
- Read input from a local file given by its path. If the path looks like a url, the content will be fetched first and then passed to the importer. Alternatively a scalar reference can be passed to read from a string.
- fh
- Read input from an IO::Handle. If not specified, Catmandu::Util::io is used to create the input stream from the "file" argument or by using STDIN.
- encoding
- Binmode of the input stream "fh". Set to ":utf8" by default.
- fix
- An ARRAY of one or more fixes or file scripts to be applied to imported items.
- data_path
-
The data at "data_path" is imported instead of the original data.
# given this imported item: {abc => [{a=>1},{b=>2},{c=>3}]} # with data_path 'abc', this item gets imported instead: [{a=>1},{b=>2},{c=>3}] # with data_path 'abc.*', 3 items get imported: {a=>1} {b=>2} {c=>3} - variables
-
Variables given here will interpolate the "file" and "http_body" options. The
syntax is the same as URI::Template.
# named arguments my $importer = Catmandu->importer('Hello', file => 'http://example.com/{id}', variables => {id => 1234}, ); # positional arguments {variables => "1234,768"} # or {variables => [1234,768]}
HTTP CONFIGURATION
These options are only relevant if "file" is a url. See LWP::UserAgent for details about these options.- http_method
- http_headers
- http_agent
- http_max_redirect
- http_timeout
- http_verify_hostname