Getting Started with ECS - Java + S3 (ViPR SDK)

NOTE: The ViPR S3 client is deprecated and has been replaced with the ECS Java Object Client: ECS Object Client SDK for Java

GitHub: Home · EMCECS/ecs-object-client-java Wiki · GitHub

 

 

 

The ViPR S3 client extends Amazon’s own S3 client implementation included in the AWS SDK for Java.  As such, it depends on the AWS SDK library and all of its transitive dependencies.

 

Enhancements

 

The ViPR S3 client includes the following ECS-specific enhancements to the standard S3 API published by Amazon. If you wish to leverage any of these enhancements, you will need to use the ViPR S3 SDK.  If none of these enhancements apply to you, you may consider using the standard AWS SDK instead.  If you isolate the client creation appropriately, the code you write using the AWS SDK will be compatible should you choose to add these features later.

 

All of the below features are described in more detail in the ViPR S3 Extensions tutorial.

 

Object Modification

ECS allows object modification by including ranges in your PUT requests.  You can modify any part of an object, append to the end of an object or even create a “sparse” object by writing past the end of its existing data size.

 

Atomic Append

There may be cases where you need to append to an object, but determining the exact byte offset is inefficient and isn’t vital to your use-case.  For this scenario, ECS provides the ability to atomically append data to the object without specifying an offset (the correct offset is returned to you in the response).

 

Namespaces

ECS provides namespaces that isolate an entire bucket/key space.  A bucket/key in one namespace is not accessible and may even represent a completely different object in another namespace.  You can optionally specify a namespace via DNS hostname.  If you do not use DNS to specify the namespace for a request, you will need to use the ViPR S3 client and call .setNamespace() before interacting with objects.

 

Maven Setup

 

If your project uses maven or ivy for dependency management, then adding the ViPR S3 Client is quite easy. Simply include the following dependency in your pom.xml file:

 

<dependency>
    <groupId>com.emc.vipr</groupId>
    <artifactId>s3-client</artifactId>
    <version>2.0.1</version>
</dependency> 
















Manual Setup

 

If you are manually managing your dependencies, then you will need to download the latest distribution zip (s3-client-x.x.x.zip) from github:

 

ViPR Services SDK Latest Release

https://github.com/emcvipr/dataservices-sdk-java/releases

 

Configuration

 

Load Balanced endpoint

To instantiate and configure the ViPR S3 client with a load balanced endpoint, use the ViPRS3Client constructor and pass in your BasicAWSCredentials as you would with the AmazonS3Client.  The only additional required step is to specify your endpoint (the ECS  load balancer).  Then you must specify a namespace by calling setNamespace.  That last step is required even if you are using DNS namespaces.

 

BasicAWSCredentials creds = new BasicAWSCredentials(accessKey, secretKey);
ViPRS3Client vipr = new ViPRS3Client(endpoint, creds);

// to set namespace:
vipr.setNamespace(namespace);








 

Smart Client (client-side load balancing)

To enable the smart client, pass a ViPRConfig instance to the ViPRS3Client constructor.  Check out this separate post as well.

 

BasicAWSCredentials creds = new BasicAWSCredentials(accessKey, secretKey);

ViPRS3Config viprConfig = new ViPRS3Config();
viprConfig.withS3Endpoints("http://vipr-ds1.company.com:9020,http://vipr-ds2.company.com:9020");
viprConfig.setCredentialsProvider(new StaticCredentialsProvider(creds));

ViPRS3Client vipr = new ViPRS3Client(viprConfig);








 

The client extends AmazonS3Client and implements both the ViPRS3 and AmazonS3 interfaces, so any existing S3 code will continue to work with this instance.

 

Conditionalizing ECS-specific calls

 

In cases where you can’t anticipate your target environment, or where you must execute against both S3 and ViPR, you can branch your logic based on whether you are provided a ECS client or not.

 

if (client instanceof ViPRS3) {
    ((ViPRS3) client).updateObject(bucket, key, inputStream,
                                  metadata, startOffset);
} else {
    // recreate new complete object
    client.putObject(bucket, key, inputStream, metadata);
}
















Sample Project

 

A sample Eclipse project is provided to demonstrate all of the ECS enhancements with working examples. You can download the samples as a zip file here: ViPR S3 Java Samples