s3ls(1) List S3 buckets and bucket contents

SYNOPSIS

s3ls [options]

s3ls [options] [ [ bucket | bucket/item ] ...]


Options:
--access-key AWS Access Key ID
--secret-key AWS Secret Access Key
--long
Environment:
AWS_ACCESS_KEY_ID
AWS_ACCESS_KEY_SECRET

OPTIONS

--help
Print a brief help message and exits.
--man
Prints the manual page and exits.
--verbose
Output what is being done as it is done.
--access-key and --secret-key
Specify the ``AWS Access Key Identifiers'' for the AWS account. --access-key is the ``Access Key ID'', and --secret-key is the ``Secret Access Key''. These are effectively the ``username'' and ``password'' to the AWS account, and should be kept confidential.

The access keys MUST be specified, either via these command line parameters, or via the AWS_ACCESS_KEY_ID and AWS_ACCESS_KEY_SECRET environment variables.

Specifying them on the command line overrides the environment variables.

--secure
Uses SSL/TLS HTTPS to communicate with the AWS service, instead of HTTP.
--long

ENVIRONMENT VARIABLES

AWS_ACCESS_KEY_ID and AWS_ACCESS_KEY_SECRET
Specify the ``AWS Access Key Identifiers'' for the AWS account. AWS_ACCESS_KEY_ID contains the ``Access Key ID'', and AWS_ACCESS_KEY_SECRET contains the ``Secret Access Key''. These are effectively the ``username'' and ``password'' to the AWS service, and should be kept confidential.

The access keys MUST be specified, either via these environment variables, or via the --access-key and --secret-key command line parameters.

If the command line parameters are set, they override these environment variables.

CONFIGURATION FILE

The configuration options will be read from the file "~/.s3-tools" if it exists. The format is the same as the command line options with one option per line. For example, the file could contain: 

    --access-key <AWS access key>
    --secret-key <AWS secret key>
    --secure

This example configuration file would specify the AWS access keys and that a secure connection using HTTPS should be used for all communications.

DESCRIPTION

Lists the buckets owned by the user, or all the item keys in a given bucket, or attributes associated with a given item.

If no buckets or bucket/itemkey is specified on the command line, all the buckets owned by the user are listed.

If the "--long" option is specified, the creation date of each bucket is also output.

If a bucket name is specified on the command line, all the item keys in that bucket are listed.

If the "--long" option is specified, the ID and display string of the item owner, the creation date, the MD5, and the size of the item are also output.

If a bucket name and an item key, separated by a slash character, is specified on the command line, then the bucket name and the item key are output. This is useful to check that the item actually exists.

If the "--long" option is specified, all the HTTP attributes of the item are also output. This will include Content-Length, Content-Type, ETag (which is the MD5 of the item contents), and Last-Modifed.

It may also include the HTTP attributes Content-Language, Expires, Cache-Control, Content-Disposition, and Content-Encoding.

It will also include any x-amz- metadata headers.

BUGS

Report bugs to Mark Atwood [email protected].

Occasionally the S3 service will randomly fail for no externally apparent reason. When that happens, this tool should retry, with a delay and a backoff.

Access to the S3 service can be authenticated with a X.509 certificate, instead of via the ``AWS Access Key Identifiers''. This tool should support that.

It might be useful to be able to specify the ``AWS Access Key Identifiers'' in the user's "~/.netrc" file. This tool should support that.

Errors and warnings are very ``Perl-ish'', and can be confusing.

Trying to access a bucket or item that does not exist or is not accessible by the user generates less than helpful error messages.

This tool does not efficiently handle listing huge buckets, as it downloads and parses the entire bucket listing, before it outputs anything.

This tool does not take advantage of the prefix, delimiter, and hierarchy features of the AWS S3 key listing API.

AUTHOR

Written by Mark Atwood [email protected].

Many thanks to Wotan LLC <http://wotanllc.com>, for supporting the development of these S3 tools.

Many thanks to the Amazon AWS engineers for developing S3.

LAST SEARCHED