Getting Started with ECS - Python + Swift (python-swiftclient)

If you're a Python programmer and want to use ECS for Swift-compatible storage, you can use the python-swiftclient package to easily connect to ECS.

 

First, you'll need to install the python-swiftclient package.  You can do this through pip using "pip install python-swiftclient":

 

macpro:vipr-python-swiftclient cwikj$ sudo pip install python-swiftclient
Downloading/unpacking python-swiftclient
  Downloading python_swiftclient-2.2.0-py2.py3-none-any.whl (40kB): 40kB downloaded
Downloading/unpacking simplejson>=2.0.9 (from python-swiftclient)
  Downloading simplejson-3.6.3.tar.gz (73kB): 73kB downloaded
  Running setup.py (path:/private/tmp/pip_build_root/simplejson/setup.py) egg_info for package simplejson
   
Downloading/unpacking six>=1.5.2 (from python-swiftclient)
  Downloading six-1.7.3-py2.py3-none-any.whl
Downloading/unpacking requests>=1.1 (from python-swiftclient)
  Downloading requests-2.3.0-py2.py3-none-any.whl (452kB): 452kB downloaded
Installing collected packages: python-swiftclient, simplejson, six, requests
  Running setup.py install for simplejson
    building 'simplejson._speedups' extension
    cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c simplejson/_speedups.c -o build/temp.macosx-10.9-intel-2.7/simplejson/_speedups.o
    simplejson/_speedups.c:2640:42: warning: implicit conversion loses integer precision: 'long' to 'int' [-Wshorten-64-to-32]
            int int_as_string_bitcount_val = PyLong_AsLong(int_as_string_bitcount);
                ~~~~~~~~~~~~~~~~~~~~~~~~~~   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    1 warning generated.
    cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/simplejson/_speedups.o -o build/lib.macosx-10.9-intel-2.7/simplejson/_speedups.so
   
Successfully installed python-swiftclient simplejson six requests
Cleaning up...

Note: on Mac OSX, I also had to install pbr through pip as well (pip install pbr).

 

Next, you'll need to gather your ECS configuration.  By default, the Swift protocol runs on ECS nodes on port 9024 for HTTP and 9025 for HTTPS.  You will also need to create a swift password for your user.

 

To create a swiftclient instance, you use your ECS username and your swift password as the 'key'.  The swauth interface runs on the Swift endpoint under /auth/v1.0, so you can create the authurl by appending to your endpoint.  Finally, you need to set "insecure" to True if you're connecting on port 9024.

 

from swiftclient import Connection
from pprint import pprint

# Setup your endpoint and credentials here
endpoint = 'http://10.0.0.24:9024'
user = 'jason'
key = 'password'

# This uses swauth (v1.0) auth
swift = Connection(
                authurl=endpoint + "/auth/v1.0",
                user=user,
                key=key,
                auth_version="1",
                insecure=True)

 

Once you have your Swift connection object, you can easily CRUD containers and objects:

 

# Config for testing
container_name = 'python-test'
object_name = 'hello.txt'

(headers, containers) = swift.get_account()
print 'account headers: '
pprint(headers)
print 'account containers: '
pprint(containers)

# Create container
print 'creating container %s...' % container_name
swift.put_container(container=container_name)
print 'done!'
print '------------------------\n'

# Upload object to container
print 'creating object %s in %s...' % (object_name, container_name)
swift.put_object(container=container_name,
                      obj=object_name,
                      contents='Hello World!',
                      content_type='text/plain')
print 'done!'
print '------------------------\n'

# Read object back
print 'reading object %s in %s...\n' % (object_name, container_name)
(headers,content) = swift.get_object(container=container_name, obj=object_name)
print 'content: %s\n' % content
print 'headers: '
pprint(headers)
print 'done!'
print '------------------------\n'

# Delete object
print 'Deleting object %s in %s...\n' % (object_name, container_name)
swift.delete_object(container=container_name, obj=object_name)
print 'done!'
print '------------------------\n'

# Delete container
print 'deleting container %s...' % container_name
swift.delete_container(container=container_name)
print 'done!'
print '------------------------\n'

 

Running this script should produce output similar to the following:

 

macpro:vipr-python-swiftclient cwikj$ python viprtest.py
Logged in using swauth, listing containers

account headers:
{'content-type': 'application/json',
'date': 'Wed, 20 Aug 2014 18:18:45 GMT',
'server': 'Jetty(7.6.4.v20120524)',
'transfer-encoding': 'chunked'}
account containers:
[]
creating container python-test...
done!
------------------------

creating object hello.txt in python-test...
done!
------------------------

reading object hello.txt in python-test...

content: Hello World!

headers:
{'content-length': '12',
'content-type': 'text/plain',
'date': 'Wed, 20 Aug 2014 18:18:46 GMT',
'etag': 'ed076287532e86365e841e92bfc50d8c',
'last-modified': 'Wed, 20 Aug 2014 18:18:46 GMT',
'server': 'Jetty(7.6.4.v20120524)',
'x-emc-mtime': '1408558726068',
'x-timestamp': '1408558726'}
done!
------------------------

Deleting object hello.txt in python-test...
done!
------------------------

deleting container python-test...
done!
------------------------

 

The source code for the sample is attached to this document.  Please refer to the python-swiftclient documentation for details on the operations for buckets and objects.