ECMWF::ECaccess(3) Perl extension to access the ECMWF ECaccess Web Services (or ECaccess API)

SYNOPSIS


What are the ECaccess Web Services?
How to use it?
ECaccess Authentication
ECaccess Token
Control Channel
Data Channel
Upload/Download FILEs
Release Token
ECaccess File System
Where can I find examples?
See Also

DESCRIPTION

ECMWF::ECaccess is a Perl module which provides access to the ECMWF ECaccess Web Services (or ECaccess API).

WHAT ARE THE ECACCESS WEB SERVICES?

The SOAP ECaccess API was created for developers and researchers interested in using ECMWF facilities in their applications. Developers write software programs that connect remotely to the SOAP ECaccess Service. Communication is performed via SOAP, an XML-based mechanism for exchanging typed information.

Developers can issue requests to the following SOAP ECaccess Methods:

Gateways Requests

  String getGatewayName();
  Boolean gatewayIsConnected();
  GatewayResponse getGateway(String token, String gateway);
  GatewayResponse[] getGatewayList(String token);

Certificates and Tokens Requests

  byte[] createCertificate(String ecuser, String passcode);
  String getTokenFromCertificate(byte[] certificate);
  String getTokenFromUserPasscode(String ecuser, String passcode);
  OperationResponse[] getOperationList(String token);
  OperationResponse getOperation(String token, String operationName);
  Boolean releaseToken(String token);

FILEs Requests

  Boolean changeFileMode(String token, Integer mode, String path);
  Boolean deleteFile(String token, String source, Boolean force);
  Boolean makeDirectory(String token, String dir);
  Boolean removeDirectory(String token, String dir);
  String getFileLastModified(String token, String source);
  Long getFileSize(String token, String source);
  Boolean copyFile(String token, String source, String target, Boolean erase);
  Boolean moveFile(String token, String source, String target);
  DirResponse[] getDirList(String token, String path, Boolean dir);
  String getInputFileHandle(String token, String source, Long offset);
  String getOutputFileHandle(String token, String target, Long offset, Integer umask);
  String getTemporaryFile(String token);

JOBs Requests

  String getJobOutputHandle(String token, String jobid);
  String getJobInputHandle(String token, String jobid);
  String getJobErrorHandle(String token, String jobid);
  JobResponse getJob(String token, String jobid);
  JobResponse[] getJobList(String token);
  Boolean deleteJob(String token, String jobid);
  String submitJob(String token, JobRequest request);
  String restartJob(String token, String jobId);
  QueueResponse[] getQueueList(String token);
  QueueDetailResponse[] getQueueDetail(String token, String queueName);

Events Requests

  EventResponse[] getEventList(String token);
  EventResponse getEvent(String token, String eventId);
  Integer sendEvent(String token, EventRequest request);

ECtrans Associations Requests

  AssociationResponse[] getAssociationList(String token,String gateway);
  AssociationResponse getAssociation(String token, String gateway, String name, Boolean template);
  Boolean putAssociation(String token, String gateway, AssociationRequest association);
  Boolean deleteAssociation(String token, String gateway, String name);

ECtrans Transfers Requests

  TransferResponse[] getTransferList(String token);
  TransferResponse getTransfer(String token, String transferId);
  Boolean deleteTransfer(String token, String transferId);
  String requestTransfer(String token, TransferRequest request);
  Boolean restartTransfer(String token, TransferRequest request, String transferId);

IOs Requests (on FILEs and JOBs transfers)

  Boolean writeStringHandle(String handle, String string);
  Boolean writeBytesHandle(String handle, byte[] bytes);
  String readStringHandle(String handle, Integer size);
  byte[] readBytesHandle(String handle, Integer size);
  Boolean closeHandle(String handle);

CosInfo Request

  String getCosInfo(String token);

The following link provides the WSDL file you can use to generate code if your environment supports it: 

  http://ecaccess.ecmwf.int/axis2/services/ECaccessService?wsdl

HOW TO USE IT?

ECaccess Authentication

Using the ECMWF::ECaccess Module requires a valid certificate. Certificates can be created with the ``ecaccess-certificate-create'' command from an ECMWF user identifier and a PASSCODE (using a security token), it generates a certificate in ``.eccert.crt'' in the user's home directory.

You need to ensure the following environment parameters are set with the correct values: 

  http_ecaccess=gateway.meteo.ms:9080
  https_ecaccess=gateway.meteo.ms:9443

(e.g. if your local ECaccess Gateways name is ``gateway.meteo.ms'' and you are using the default ECaccess http/s ports 9080/9443)

The default values are pointing to the ecaccess.ecmwf.int server.

If you access your ECaccess Gateway through a proxy then you should also set the following: 

  http_proxy=http://proxy.meteo.ms:port
  https_proxy=https://proxy.meteo.ms:port

ECaccess Token

You should generally request a token for the majority of methods available. For example, if your application requires submitting a new job, you should usually request a token from your certificate which grants access to the jobs management methods (e.g. submitJob, getJobList, deleteJob or getJobResult).

When you obtain an ECacces Token for a user, you must use that ECaccess Token for all future interactions with the API on behalf of the user.

In order to obtain an ECaccess Token you must use the following code: 

  use ECMWF::ECaccess;
  my $ecaccess = ECMWF::ECaccess->new(); # Create the ECaccess Controler
  my $token = $ecaccess->getToken(); # Get an ECaccess Token

Control Channel

Once you have an ECaccess Token you must request a Control Channel to access the API: 

  my $controlChannel = $ecaccess->getControlChannel(); # Get the Control Channel

This Control Channel will allow you to call any method of the API. For example, let's move a FILE from ECfs to the super-computer: 

  controlChannel->moveFile($token,'ec:test/a.out','c2a:/c2a/tmp/systems/syi/a.out');

Or let's get the CosInfo from ECMWF: 

  print $controlChannel->getCosInfo($token)->result."\n";

Data Channel

In order to Download/Upload Data to/from ECMWF you are required to use a Data Channel: 

  my $dataChannel = $ecaccess->getDataChannel(); # Get the Data Channel

The methods available on the Data Channel are the following: 

  Boolean writeStringHandle(String handle, String string);
  Boolean writeBytesHandle(String handle, byte[] bytes);
  String readStringHandle(String handle, Integer size);
  byte[] readBytesHandle(String handle, Integer size);

Upload/Download FILEs

Suppose you want to Upload a text FILE you would do the following: 

  $handle = $controlChannel->getOutputFileHandle($token,'home:/test.txt',0,600)->result;
  open FILE, 'test.txt' or die $!;
  while (read(FILE, $data, 524288) > 0) {
    $dataChannel->writeStringHandle($handle,$data);
  }
  $controlChannel->closeHandle($handle);
  close FILE;

In order to Download a binary FILE you would do the following: 

  $handle = $controlChannel->getInputFileHandle($token,'home:/a.out',0)->result;
  open FILE, ">", $target or die $!;
  binmode FILE;
  while (length($data = decode_base64($dataChannel->readBytesHandle($handle, 524288)->result)) > 0) {
    print FILE $data;
  }
  $controlChannel->closeHandle($handle);
  close FILE;

In both cases you get a FILE Handle through the Control Channel and then you access the Data through the Data Channel. Once the Download/Upload is completed you close the Handle with the 'closeHandle' method.

Release Token

Once you have finished with your session you must release your token using the following code: 

  $ecaccess->releaseToken($token); # Logout

Please note that you can also use the Control Channel to Download/Upload Data if you require a secure connection for Data Transfers (https vs. http).

ECaccess File System

When accessing FILEs at ECMWF the following domains are available to the user logged-in: 

  HOME: the $HOME directory (home:)
  SCRATCH: the $SCRATCH directory (scratch:)
  ECFS: the ECFS directory (ec:)
  ECTMP: the ECTMP directory (ectmp:)
  HOST: any server at ECMWF ({host-name}:)

The format of the path is the following: [domain:][/user-id/]path

If no user-id is specified then the current user-id is selected by default. The user-id parameter is not valid with the HOST domain.

If no domain is specified then an absolute path will translate to an absolute path on the ecgate server and a relative path will translate to a path in the HOME directory of the current user.

A few examples: 

  "bin/a.out"                           a.out file in the $HOME/bin directory of the current user
  "home:bin/a.out"                      a.out file in the $HOME/bin directory of the current user
  "/tmp/a.out"                          a.out file in the /tmp directory on ecgate
  "home:/syi/bin/a.out"                 a.out file in the $HOME/bin directory of user syi
  "ec:bin/a.out"                        a.out file in the ECFS bin directory of the current user
  "ec:/syi/bin/a.out"                   a.out file in the ECFS bin directory of user syi
  "c2a:/c2a/tmp/systems/syi/a.out"      a.out file in the /c2a/tmp/systems/syi/ directory of c2a

WHERE CAN I FIND EXAMPLES?

The scripts which are provided along with this Module are good examples of what can be achieved with the ECMWF::ECacces Module. They use all the methods available through the ECaccess API and show the options available as well as the format of the messages which are return by the server.

AUTHOR

Laurent Gougeon, <[email protected]>

COPYRIGHT AND LICENSE

Copyright (C) 2010 by ECMWF ([email protected])

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.8 or, at your option, any later version of Perl 5 you may have available.

LAST SEARCHED