PDL::IO::Dumper(3) data dumping for structs with PDLs

DESCRIPTION

This package allows you cleanly to save and restore complex data structures which include PDLs, as ASCII strings and/or transportable ASCII files. It exports four functions into your namespace: sdump, fdump, frestore, and deep_copy.

PDL::IO::Dumper traverses the same types of structure that Data::Dumper knows about, because it uses a call to Data::Dumper. Unlike Data::Dumper it doesn't crash when accessing PDLs.

The PDL::IO::Dumper routines have a slightly different syntax than Data::Dumper does: you may only dump a single scalar perl expression rather than an arbitrary one. Of course, the scalar may be a ref to whatever humongous pile of spaghetti you want, so that's no big loss.

The output string is intended to be about as readable as Dumper's output is for non-PDL expressions. To that end, small PDLs (up to 8 elements) are stored as inline perl expressions, midsized PDLs (up to 200 elements) are stored as perl expressions above the main data structure, and large PDLs are stored as FITS files that are uuencoded and included in the dump string. (You have to have access to either uuencode(1) or the CPAN module Convert::UU for this to work).

No attempt is made to shrink the output string --- for example, inlined PDL expressions all include explicit reshape() and typecast commands, and uuencoding expands stuff by a factor of about 1.5. So your data structures will grow when you dump them.

Bugs

It's still possible to break this code and cause it to dump core, for the same reason that Data::Dumper crashes. In particular, other external-hook variables aren't recognized (for that a more universal Dumper would be needed) and will still exercise the Data::Dumper crash. This is by choice: (A) it's difficult to recognize which objects are actually external, and (B) most everyday objects are quite safe.

Another shortfall of Data::Dumper is that it doesn't recognize tied objects. This might be a Good Thing or a Bad Thing depending on your point of view, but it means that PDL::IO::Dumper includes a kludge to handle the tied Astro::FITS::Header objects associated with FITS headers (see the rfits documentation in PDL::IO::Misc for details).

There's currently no reference recursion detection, so a non-treelike reference topology will cause Dumper to buzz forever. That will likely be fixed in a future version. Meanwhile a warning message finds likely cases.

Author, copyright, no warranty

Copyright 2002, Craig DeForest.

This code may be distributed under the same terms as Perl itself (license available at <http://ww.perl.org>). Copying, reverse engineering, distribution, and modification are explicitly allowed so long as this notice is preserved intact and modified versions are clearly marked as such.

This package comes with NO WARRANTY.

HISTORY

  • 1.0: initial release
  • 1.1 (26-Feb-2002): Shorter form for short PDLs; more readability
  • 1.2 (28-Feb-2002): Added deep_copy() --- exported convenience function
      for ``eval sdump''
  • 1.3 (15-May-2002): Added checking for tied objects in gethdr()
      [workaround for hole in Data::Dumper]
  • 1.4 (15-Jan-2003): Added support for Convert::UU as well as
      command-line uu{en|de}code

FUNCTIONS

sdump

Dump a data structure to a string. 

  use PDL::IO::Dumper;
  $s = sdump(<VAR>);
  ...
  <VAR> = eval $s;

sdump dumps a single complex data structure into a string. You restore the data structure by eval-ing the string. Since eval is a builtin, no convenience routine exists to use it.

fdump

Dump a data structure to a file 

  use PDL::IO::Dumper;
  fdump(<VAR>,$filename);
  ...
  <VAR> = frestore($filename);

fdump dumps a single complex data structure to a file. You restore the data structure by eval-ing the perl code put in the file. A convenience routine (frestore) exists to do it for you.

I suggest using the extension '.pld' or (for non-broken OS's) '.pdld' to distinguish Dumper files. That way they are reminiscent of .pl files for perl, while still looking a little different so you can pick them out. You can certainly feed a dump file straight into perl (for syntax checking) but it will not do much for you, just build your data structure and exit.

frestore

Restore a dumped file 

  use PDL::IO::Dumper;
  fdump(<VAR>,$filename);
  ...
  <VAR> = frestore($filename);

frestore() is a convenience function that just reads in the named file and executes it in an eval. It's paired with fdump().

deep_copy

Convenience function copies a complete perl data structure by the brute force method of ``eval sdump''.

PDL::IO::Dumper::big_PDL

Identify whether a PDL is ``big'' [Internal routine]

Internal routine takes a PDL and returns a boolean indicating whether it's small enough for direct insertion into the dump string. If 0, it can be inserted. Larger numbers yield larger scopes of PDL. 1 implies that it should be broken out but can be handled with a couple of perl commands; 2 implies full uudecode treatment.

PDLs with Astro::FITS::Header objects as headers are taken to be FITS files and are always treated as huge, regardless of size.

PDL::IO::Dumper::stringify_PDL

Turn a PDL into a 1-part perl expr [Internal routine]

Internal routine that takes a PDL and returns a perl string that evals to the PDL. It should be used with care because it doesn't dump headers and it doesn't check number of elements. The point here is that numbers are dumped with the correct precision for their storage class. Things we don't know about get stringified element-by-element by their builtin class, which is probably not a bad guess.

PDL::IO::Dumper::uudecode_PDL

Recover a PDL from a uuencoded string [Internal routine]

This routine encapsulates uudecoding of the dumped string for large piddles. It's separate to encapsulate the decision about which method of uudecoding to try (both the built-in Convert::UU and the shell command uudecode(1) are supported).

PDL::IO::Dumper::dump_PDL

Generate 1- or 2-part expr for a PDL [Internal routine]

Internal routine that produces commands defining a PDL. You supply (<PDL>, <name>) and get back two strings: a prepended command string and an expr that evaluates to the final PDL. PDL is the PDL you want to dump. <inline> is a flag whether dump_PDL is being called inline or before the inline dump string (0 for before; 1 for in). <name> is the name of the variable to be assigned (for medium and large PDLs, which are defined before the dump string and assigned unique IDs).

PDL::IO::Dumper::find_PDLs

Walk a data structure and dump PDLs [Internal routine]

Walks the original data structure and generates appropriate exprs for each PDL. The exprs are inserted into the Data::Dumper output string. You shouldn't call this unless you know what you're doing. (see sdump, above).

LAST SEARCHED