SYNOPSIS

use PDL;
use PDL::NetCDF;
use PDL::Char;
my $ncobj = PDL::NetCDF->new ("test.nc", {REVERSE_DIMS => 1}); # New file
my $pdl = pdl [[1, 2, 3], [4, 5, 6]];
# Specify variable name to put PDL in, plus names of the dimensions. Dimension
# lengths are taken from the PDL, in this case, dim1 = 2 and dim2 = 3.
$ncobj->put ('var1', ['dim1', 'dim2'], $pdl);
# or for netcdf4 files
# $ncobj->put ('var1', ['dim1', 'dim2'], $pdl, {DEFLATE => 9});
# get the deflate level (for any fileformat)
my ($deflate, $shuffle) = $ncobj->getDeflateShuffle('var1');
# $pdlout = [[1, 2, 3], [4, 5, 6]]
my $pdlout = $ncobj->get ('var1');
# Store textual NetCDF arrays using perl strings: (This is a bit primitive, but works)
my $str = "Station1 Station2 Station3 ";
$obj->puttext('textvar', ['n_station', 'n_string'], [3,10], $str);
my $outstr = $obj->gettext('textvar');
# $outstr = "Station1 Station2 Station3 "
# Now textual NetCDF arrays can be stored with PDL::Char style PDLs. This is much
# more natural and flexible than the above method.
$str = PDL::Char->new (['Station1', 'Station2', 'Station3']);
$obj->put ('stations', ['dim_station', 'dim_charlen'], $str);
$outstr = $obj->get('stations');
print $outstr;
# Prints: ['Station1', 'Station2', 'Station3']
# For more info on PDL::Char variables see PDL::Char(3), or perldoc PDL::Char
# $dim1size = 2
my $dim1size = $ncobj->dimsize('dim1');
# A slice of the netCDF variable.
# [0,0] is the starting point, [1,2] is the count.
# $slice = [1,2]
my $slice = $ncobj->get ('var1', [0,0], [1,2]);
# Attach a double attribute of size 3 to var1
$ncobj->putatt (double([1,2,3]), 'double_attribute', 'var1');
# $attr1 = [1,2,3]
my $attr1 = $ncobj->getatt ('double_attribute', 'var1');
# $type = PDL::double
my $type = $ncobj->getvariabletype('var1');
# Write a textual, global attribute. 'attr_name' is the attribute name.
$ncobj->putatt ('The text of the global attribute', 'attr_name');
# $attr2 = 'The text of the global attribute'
my $attr2 = $ncobj->getatt ('attr_name');
# Close the netCDF file. The file is also automatically closed in a DESTROY block
# when it passes out of scope. This just makes is explicit.
$ncobj->close;

For (much) more information on NetCDF, see

http://www.unidata.ucar.edu/packages/netcdf/index.html

Also see the test file, test.pl in this distribution for some working examples.

DESCRIPTION

This is the PDL interface to the Unidata NetCDF library. It uses the netCDF version 3 library to make a subset of netCDF functionality available to PDL users in a clean, object-oriented interface.

Another NetCDF perl interface, which allows access to the entire range of netCDF functionality (but in a non-object-oriented style which uses perl arrays instead of PDLs) is available through Unidata at http://www.unidata.ucar.edu/packages/netcdf/index.html).

The NetCDF standard allows N-dimensional binary data to be efficiently stored, annotated and exchanged between many platforms.

When one creates a new netCDF object, this object is associated with one netCDF file.

FUNCTIONS

isNetcdf4

Check if compiled against netcdf4

Arguments: none

  if (PDL::NetCDF::isNetcdf4) {
        # open netcdf4 file
  }

defaultFormat

Get or change the default format when creating a netcdf-file. This can be overwritten by the NC_FORMAT option for new. Possible values are: PDL::NetCDF::NC_FORMAT_CLASSIC, PDL::NetCDF::NC_FORMAT_64BIT, PDL::NetCDF::NC_FORMAT_NETCDF4, PDL::NetCDF::NC_FORMAT_NETCDF4_CLASSIC

 Arguments:
 1) new format (constant)
 Return:
 old format as one of the NC_FORMAT_* constants

new

Create an object representing a netCDF file.

 Arguments:  
 1) The name of the file.
 2) optional:  A hashref containing options.  Currently defined are:
    TEMPLATE:
    An existing netCDF object for a file with
    identical layout.  This allows one to read in many similar netCDF
    files without incurring the overhead of reading in all variable
    and dimension names and IDs each time.  Caution:  Undefined
    weirdness may occur if you pass the netCDF object from a dissimilar
    file!
    MODE:
    use sysopen file-opening arguments, O_RDONLY, O_RDWR, O_CREAT, O_EXCL
    when used, this will overwrite the '>file.nc' type of opening
    see L<perlopentut> for usage of O_RDONLY...
    REVERSE_DIMS:
    this will turn the order of the dimension-names of
    netcdf-files. Even with this option the 'put' function will write
    variables in FORTRAN order (as before) and will reverse the
    dimension names so they fit this order.  With this option, the
    'putslice' function will write varibles in the same way as 'put'.
    You should use this option if your planning to work with other
    netcdf-programs (ncview, NCL) or if you are planning to combine
    putslice and slice.  You should _not_ use this option, if you need
    compatibility to older versions of PDL::NetCDF.
    NC_FORMAT:
    set the file format for a new netcdf file, see defaultFormat()
    SLOW_CHAR_FETCH:
    If this option is set, then a 'get' into a PDL::Char will be done
    one string at a time instead of all text data at once.  This
    is necessary if there are NULLs (hex 0) values embedded in the string
    arrays.  This takes longer, but gives the correct results.  If
    the fetch of a string array yields only the first element, try setting
    this option.

Example:

  my $nc = PDL::NetCDF->new ("file1.nc", {REVERSE_DIMS => 1});
  ...
  foreach my $ncfile (@a_bunch_of_similar_format_netcdf_files) {
    $nc = PDL::NetCDF->new("file2.nc", {TEMPLATE => $nc});  # These calls to 'new' are *much* faster
    ...
  }
  # opening using MODE
  use Fcntl; # define O_CREAT...
  # opening a completely new file (deleting if it exists!)
  my $newnc = PDL::NetCDF->new ("file2.nc", {MODE => O_CREAT|O_RDWR,
                                             REVERSE_DIMS => 1, NC_FORMAT => PDL::NetCDF::NC_FORMAT_NETCDF4});
  # opening existing file for reading and writing
  $nc = PDL::NetCDF->new ("file2.nc", {MODE => O_RDWR}
                        REVERSE_DIMS => 1});
  # opening existing file for reading only
  $nc = PDL::NetCDF->new ("file2.nc", {MODE => O_RDONLY,
                                       REVERSE_DIMS => 1});

If this file exists and you want to write to it, prepend the name with the '>' character: ``>name.nc''

Returns: The netCDF object. Barfs if there is an error.

getFormat

Get the format of a netcdf file

Arguments: none

Returns: @ integer equal to one of the PDL::NetCDF::NC_FORMAT_* constants.

put

Put a PDL matrix to a netCDF variable.

Arguments:

1) The name of the variable to create

2) A reference to a list of dimension names for this variable

3) The PDL to put. It must have the same number of dimensions as specified in the dimension name list.

4) Optional options hashref: {SHUFFLE => 1, DEFLATE => 7, COMPRESS => 0}

Returns: None.

  my $pdl = pdl [[1, 2, 3], [4, 5, 6]];
  # Specify variable name to put PDL in, plus names of the dimensions.  Dimension         
  # lengths are taken from the PDL, in this case, dim1 = 2 and dim2 = 3.      
  $ncobj->put ('var1', ['dim1', 'dim2'], $pdl);                                               
                                            
  # Now textual NetCDF arrays can be stored with PDL::Char style PDLs.  
  $str = PDL::Char->new (['Station1', 'Station2', 'Station3']);
  $obj->put ('stations', ['dim_station', 'dim_charlen'], $str);
  $outstr = $obj->get('stations');
  print $outstr;
  # Prints: ['Station1', 'Station2', 'Station3']
  # For more info on PDL::Char variables see PDL::Char(3), or perldoc PDL::Char

putslice

Put a PDL matrix to a slice of a NetCDF variable

Arguments:

1) The name of the variable to create

2) A reference to a list of dimension names for this variable

3) A reference to a list of dimensions for this variable

4) A reference to a list which specifies the N dimensional starting point of the slice.

5) A reference to a list which specifies the N dimensional count of the slice.

6) The PDL to put. It must conform to the size specified by the 4th and 5th
   arguments.  The 2nd and 3rd argument are optional if the variable is already
   defined in the netcdf object.

7) Optional options: {DEFLATE => 7, SHUFFLE => 0/1} will use gzip compression (level 7)
   on that variable and shuffle will not/will use the shuffle filter. These options are
   only valid for netcdf4 files. If you are unsure, test with 
   ($nc->getFormat >= PDL::NetCDF::NC_FORMAT::NC_FORMAT_NETCDF4)

Returns: None.

  my $pdl = pdl [[1, 2, 3], [4, 5, 6]];
  # Specify variable name to put PDL in, plus names of the dimensions.  Dimension         
  # lengths are taken from the PDL, in this case, dim1 = 2 and dim2 = 3.      
  $ncobj->putslice ('var1', ['dim1', 'dim2', 'dim3'], [2,3,3], [0,0,0], [2,3,1], $pdl);                                               
  $ncobj->putslice ('var1', [], [], [0,0,2], [2,3,1], $pdl);                                               
  my $pdl2 = $ncobj->get('var1');
  print $pdl2;
  [
 [
  [          1 9.96921e+36           1]
  [          2 9.96921e+36           2]
  [          3 9.96921e+36           3]
 ]
 [
  [          4 9.96921e+36           4]
  [          5 9.96921e+36           5]
  [          6 9.96921e+36           6]
 ]
]
 note that the netcdf missing value (not 0) is filled in.

sync

Synchronize the data to the disk. Use this if you want to read the file from another process without closing the file. This makes only sense after put, puttext, putslice, putatt operations

Returns: nothing. Barfs on error.

  $ncobj->sync

get

Get a PDL matrix from a netCDF variable.

Arguments:

1) The name of the netCDF variable to fetch. If this is the only argument, then the entire variable will be returned.

To fetch a slice of the netCDF variable, optional 2nd and 3rd argments must be specified:

2) A pdl which specifies the N dimensional starting point of the slice.

3) A pdl which specifies the N dimensional count of the slice.

Also, an options hashref may be passed. The only option currently is 'NOCOMPRESS' which tells PDL::NetCDF to *not* try to uncompress an compressed variable. See the COMPRESS option on 'put' and 'putslice' for more info.

Returns: The PDL representing the netCDF variable. Barfs on error.

  # A slice of the netCDF variable.
  # [0,0] is the starting point, [1,2] is the count.
  my $slice  = $ncobj->get ('var1', [0,0], [1,2], {NOCOMPRESS => 1});
  # If var1 contains this:  [[1, 2, 3], [4, 5, 6]]
  # Then $slice contains: [1,2] (Size '1' dimensions are eliminated).

putatt

putatt --- Attach a numerical or textual attribute to a NetCDF variable or the entire file.

Arguments:

1) The attribute. Either: A one dimensional PDL (perhaps contining only one number) or a string.

2) The name to give the attribute in the netCDF file. Many attribute names have pre-defined meanings. See the netCDF documentation for more details.

3) Optionally, you may specify the name of the pre-defined netCDF variable to associate this attribute with. If this is left off, the attribute is a global one, pertaining to the entire netCDF file.

Returns: Nothing. Barfs on error.

  # Attach a double attribute of size 3 to var1
  $ncobj->putatt (double([1,2,3]), 'double_attribute', 'var1');
  # Write a textual, global attribute.  'attr_name' is the attribute name.
  $ncobj->putatt ('The text of the global attribute', 'attr_name');

getatt

Get an attribute from a netCDF object.

Arguments:

1) The name of the attribute (a text string).

2) The name of the variable this attribute is attached to. If this argument is not specified, this function returns a global attribute of the input name.

  # Get a global attribute
  my $attr2 = $ncobj->getatt ('attr_name');
  # Get an attribute associated with the variable 'var1'
  my $attr1 = $ncobj->getatt ('double_attribute', 'var1');

getDeflateShuffle

Get the deflate level and the shuffle flag for a variable.

Can be called on all files, although only netcdf4 files support shuffle and deflate.

Arguments:

1) The name of the variable.

Returns:

($deflate, $shuffle)

  my ($deflate, $shuffle) = $nc->getDeflateShuffle('varName');

getvariabletype

Get a type of a variable from a netCDF object.

Arguments:

1) The name of the variable.

Returns: PDL::type or undef, when variable not defined

  # Get a type
  my $type = $ncobj->getvariabletype ('var1');

puttext

Put a perl text string into a multi-dimensional NetCDF array.

Arguments:

1) The name of the variable to be created (a text string).

2) A reference to a perl list of dimension names to use in creating this NetCDF array.

3) A reference to a perl list of dimension lengths.

4) A perl string to put into the netCDF array. If the NetCDF array is 3 x 10, then the string must
   have 30 charactars.

5) Optional nc4 options: {DEFLATE => 7, SHUFFLE => 0}

  my $str = "Station1  Station2  Station3  ";
  $obj->puttext('textvar', ['n_station', 'n_string'], [3,10], $str);

gettext

Get a multi-dimensional NetCDF array into a perl string.

Arguments:

1) The name of the NetCDF variable.

  my $outstr = $obj->gettext('textvar');

dimsize

Get the size of a dimension from a netCDF object.

Arguments:

1) The name of the dimension.

Returns: The size of the dimension.

  my $dim1size = $ncobj->dimsize('dim1');

close

Close a NetCDF object, writing out the file.

Arguments: None

Returns: Nothing

This closing of the netCDF file can be done explicitly though the 'close' method. Alternatively, a DESTROY block does an automatic close whenever the netCDF object passes out of scope.

  $ncobj->close();

getdimensionnames ([$varname])

Get all the dimension names from an open NetCDF object. If a variable name is specified, just return dimension names for *that* variable.

Arguments: none

Returns: An array reference of dimension names

  my $varlist = $ncobj->getdimensionnames();
  foreach(@$varlist){
    print "Found dim $_\n";
  }

getattributenames

Get the attribute names for a given variable from an open NetCDF object.

Arguments: Optional variable name, with no arguments it will return the objects global netcdf attributes.

Returns: An array reference of attribute names

  my $attlist = $ncobj->getattributenames('var1');

getvariablenames

Get all the variable names for an open NetCDF object.

Arguments:
 none.

Returns: An array reference of variable names

  my $varlist = $ncobj->getvariablenames();

setrec

Set up a 'record' of several 1D netCDF variables with the same dimension. Once this is set up, quick reading/writing of one element from all variables can be put/get from/to a perl list.

Arguments:

 1) The names of all the netCDF variables to group into a record

Returns:
 A record name to use in future putrec/getrec calls

  my $rec = $ncobj->setrec('var1', 'var2', 'var3');

getrec

Gets a 'record' (one value from each of several 1D netCDF variables) previously set up using 'setrec'. These values are returned in a perl list.

Arguments:

 1) The name of the record set up in 'setrec'.
 2) The index to fetch.

Returns:
 A perl list of all values.  Note that these variables
 can be of different types: float, double, integer, string.

  my @rec = $ncobj->getrec($rec, 5);

putrec

Puts a 'record' (one value from each of several 1D netCDF variables) previously set up using 'setrec'. These values are supplied as a perl list reference.

Arguments:

 1) The name of the record set up in 'setrec'.
 2) The index to set.
 3) A perl list ref containing the values.

Returns:
 None.

  $ncobj->putrec($rec, 5, \@values);

WRITING NetCDF-FILES EFFICIENTLY

Writing several variables to NetCDF-files can take a long time. When a new variable is attached by "put" to a file, the attribute header has to be written. This might force the internal netcdf-library to restructure the complete file, and thus might take very much IO-resources. By pre-defining the dimensions, attributes, and variables, much time can be saved. Essentially the rule of thumb is to define and write the data in the order it will be laid out in the file. Talking PDL::NetCDF, this means the following:

Open the netcdf file

    my $nc = new PDL::NetCDF('test.nc', {MODE => O_CREAT|O_RDWR,
                                         REVERSE_DIMS => 1});

Write the global attributes

    $nc->putatt (double([1,2,3]), 'double_attribute');

Define all variables, make use of the NC_UNLIMITED dimension

   # here it is possible to choose float/double/short/long
   $pdl_init = long ([]);  
   for (my $i=0; $i<$it; $i++) {
       my $out2 = $nc->putslice("VAR$i",
                             ['x','y','z','t'],
                             [150,100,20,PDL::NetCDF::NC_UNLIMITED()],
                             [0,0,0,0],[1,0,0,0],$pdl_init);
   }

Write the variable-attributes

   $nc->putatt ("var-attr", 'attribute', 'VAR0');

Write data with putslice

    $nc->putslice("VAR5",[],[],[0,0,0,0],[$datapdl->dims],$datapdl);

AUTHOR

Doug Hunt, dhunt\@ucar.edu.

CONTRIBUTORS

Heiko Klein, heiko.klein\@met.no Edward Baudrez, Royal Meteorological Institute of Belgium, edward.baudrez\@meteo.be