SYNOPSIS

tmx-tokenize file.tmx # creates t_file.tmx
tmx-tokenize -o=out.tmx file.tmx

DESCRIPTION

Although this script is bundled in "XML::TMX", it has a soft dependency on "Lingua::FreeLing3". Soft means that the dependency is not ensured at install time, and other features of the module can still be used without "Lingua::FreeLing3". Nevertheless, if you want to use this tool you should install that module.

At the moment the supported languages are the same as supported by FreeLing3: English, Spanish, Russian, Portuguese and Italian.

It your TMX file includes any other language, they will be maintained without a change. This behavior can change in the future, as a basic regexp based tokenizer might be implemented.

AUTHOR

Alberto Manuel Brandão Simões, <[email protected]>

COPYRIGHT AND LICENSE

Copyright (C) 2012 by Alberto Manuel Brandão Simões