Installing AWS CLI, S3-curl, and s3cmd on Mac OSX

Summary

The AWS Command Line Interface helps you manage your AWS services using the command line. You can configure your local machine to be able to send commands against AWS services from the command line. This guide will show you how to set it up on your Mac OSX, but you can follow the same instructions for Linux.

Requirements

• Java

  $java -version

  If Java is not installed, please follow these instructions for OSX.
• Curl

  $which curl

• Python - Download it from here

Installation

With the prerequisites installed, download the Amazon EC2 API Tools and extract the file. Follow the instructions from AWS for configuration. You will need to setup the following variables in your ~/.bash_profile file:

• EC2_HOME
• Updated PATH variable
• AWS_ACCESS_KEY
• AWS_SECRET_KEY

(Optional) Set the Region By default, the Amazon EC2 CLI tools use the US East (Northern Virginia) region (us-east-1) with the ec2.us-east-1.amazonaws.com service endpoint URL. To access a different region with the CLI tools, you must set the EC2_URL environment variable to the proper service endpoint URL. From AWS:

To set the service endpoint URL To list your available service endpoint URLs, call the ec2-describe-regions command, as shown in the previous section. Set the EC2_URL environment variable using the service endpoint URL returned from the ec2-describe-regions command as follows.

$ export EC2_URL=https://service_endpoint

For us-west-2, you would use

export EC2_URL=https://ec2.us-west-2.amazonaws.com

Once you added those environment variables, run source ~/.bash_profile to load them. Assuming your AWS credentials have the proper permissions, you should be able to run commands like this:

ec2-describe-instances i-54344e43

Let's download the Amazon S3 Authentication Tool for Curl. Unzip the downloaded files. Go into your home directory and edit ~/.s3curl. If the file does not exist, create it. Include the following:

%awsSecretAccessKeys = (
    # personal account
    personal => {
        id => 'YOUR_PESONAL_KEY_HERE',
        key => 'YOUR_PESONAL_SECRET_KEY_HERE',
    },

   # corporate account
   company => {
        id => 'YOUR_KEY_HERE',
        key => 'YOUR_SECRET_KEY_HERE',
    },
);

chmod +x s3curl.pl to make it executable. Now you can run

./s3curl.pl --id=personal -- http://YOUR_BUCKET_NAME.s3.amazonaws.com

s3curl is very useful, but the XML formatted response is not easy to interpret from the command line. This is where s3cmd helps. Download the package and extract it. Browse to the newly created directory and install it.

python setup.py install

You should be able to run the s3cmd. Let's configure it.

 s3cmd --configure

You will be prompted to enter your access and secret key. Once configured, you can run s3cmd ls to get a list of all the buckets in a readable format. This how you get the size of a bucket:

s3cmd du -H s3://YOUR_BUCKET_NAME

Feedback is always welcome.

Additional Information

• Tools for AWS Services
• AWS CLI
• Other s3cmd uses
• Installing s3cmd on Linux