Using s3cmd with ViPR

Sometimes, it can be handy to have a command-line tool to test and/or create simple scripts with.  S3cmd is a command-line utility written in Python (2.4+) that can perform most S3-related operations.

 

After extracting and installing the package you can configure it as normal, but you should skip the test because it will fail since it assumes you're connecting to AWS (we'll fix that next).

 

mbp2:~ cwikj$ s3cmd --configure

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.


Access key and Secret key are your identifiers for Amazon S3
Access Key [root]:
Secret Key [ok46444AdDZkUKqrrrrNeDoPod74J78uiTtSY2Du]:


Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password:
Path to GPG program [None]:


When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP and can't be used if you're behind a proxy
Use HTTPS protocol [No]:


On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't conect to S3 directly
HTTP Proxy server name:


New settings:
  Access Key: root
  Secret Key: ok46444AdDZkUKqrrrrNeDoPod74J78uiTtSY2Du
  Encryption password:
  Path to GPG program: None
  Use HTTPS protocol: False
  HTTP Proxy server name:
  HTTP Proxy server port: 0


Test access with supplied credentials? [Y/n] n


Save settings? [y/N] y
Configuration saved to '/Users/cwikj/.s3cfg'

 

Next, we'll need to change the endpoint from Amazon to your ViPR instance.  You can edit the .s3cfg file created during the configuration step and look for the following settings:

 

host_base = s3.amazonaws.com
host_bucket = %(bucket)s.s3.amazonaws.com

 

You can then change them to point at your ViPR instance:

 

host_base = s3.mydomain.com
host_bucket = %(bucket)s.s3.mydomain.com

 

Or, if you're connecting directly to a ViPR node(s), you can use port 9020 for S3:

 

host_base = s3.mydomain.com:9020
host_bucket = %(bucket)s.s3.mydomain.com:9020

 

After configuration is complete, you can test creating a bucket and uploading a file:

 

mbp2:~ cwikj$ s3cmd mb s3://bucket1
Bucket 's3://bucket1/' created

mbp2:~ cwikj$ s3cmd put README s3://bucket1
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
README -> s3://bucket1/README  [1 of 1]
13130 of 13130   100% in    0s    55.74 kB/s  done