ViPR Controller Java Client - Cookbook

This document is a guide to using the ViPR Controller Java Client by example. The ViPR Controller Java Client is available with ViPR v1.1 and later.

Construction of the Client

The main entry point to the client is through the ViPRCoreClient class. The only required configuration parameter for a client is the hostname or IP address of the virtual IP of the ViPR instance. The client can be constructed either with just the hostname or a ClientConfig object.

The ClientConfig object contains all configuration available for the client. It can either be used as a JavaBean by calling set*() operations to configure settings or it can be used as a builder, chaining settings with with*() calls. Using the JavaBean style is easier to use from a dependency injection tool where the builder style is easier to use programatically in a single expression.

// Construction by hostname only

// Construction by hostname only
ViPRCoreClient client = new ViPRCoreClient("vipr-vip", true);

// Construction by ClientConfig
ViPRCoreClient client = new ViPRCoreClient(new ClientConfig().withHost("vipr-vip").withIgnoringCertificates(true));

// Construction of ClientConfig using JavaBeans
ClientConfig config = new ClientConfig();
config.setHost("vipr-vip");
config.setRequestLoggingEnabled(false);
config.setMaxRetries(10);
config.setMediaType("application/json");
ViPRCoreClient client = new ViPRCoreClient(config);

// Construction of ClientConfig using builder style
ViPRCoreClient client = new ViPRCoreClient(new ClientConfig()
    .withHost("vipr-vip")
    .withRequestLoggingDisabled()
    .withMaxRetries(10)
    .withMediaType("application/json"));




Debug Logging

The client can automatically log all XML or JSON going to and from the ViPR API. This is controlled by the configuration option setRequestLoggingEnabled and defaults to true. When enabled, inputs and outputs are logged using SLF4J.

Authentication

Authentication can be done by using the AuthCilent. This can either be created standalone the same way as the ViPRCoreClient or can be retrieved from the main client. Using the auth client, you can call the login method to receive an authentication token. This token can be reused until the token expires or the user calls logout.

// Creating a new AuthClient
AuthClient auth = new AuthClient("vipr-vip");

// Retrieving from ViPRCoreClient
AuthClient auth = client.auth();

// Login and retrieve the auth token
String token = auth.login("root", "ChangeMe");




In addition to the login call returning the token, it is also stored in the state of the client. If login was done somewhere else (perhaps another service or thread), then the token needs to be set on the client.

// Token created in a different client
String token = auth.login("root", "ChangeMe");
// ...
ViPRCoreClient client = new ViPRCoreClient("vipr-vip");
client.setAuthToken(token);

// Login done in the same client
ViPRCoreClient client = new ViPRCoreClient("vipr-vip");
client.auth().login("root", "ChangeMe");




For convenience, builder style convenience methods are provided on the ViPRCoreClient. These allow chaining performing a login or setting the token with construction of the client.

// Construct client and set auth token
ViPRCoreClient client = new ViPRCoreClient("vipr-vip").withAuthToken("token");

// Construct client, perform a login and set token
ViPRCoreClient client = new ViPRCoreClient("vipr-vip").withLogin("root", "ChangeMe");




Resources

The client is organized by different resource types. These are available by methods of the same name as the resource type. Every resource client follows a consistent structure for retrieving data.

The following describes different operations available to many resources.

  • .get(String id) - Gets are single record using the call GET RESOURCE/{id}
  • .list() - Gets a list of name/id pairs using the call GET RESOURCE. Note that this will return resources that are pending delete (inactive = true).
  • .getByIds() .getByRefs() - Queries multiple resources in a single call. If the target resource supports bulk operations, this uses the bulk APIs to retrieve resources for optimal performance. This call always filters out inactive resources.
  • .getAll() - This is a convenience method that does a list() followed by a getByIds(). It allows retrieving many resources in a single call. This call always filters out inactive resources.
  • .findBy*() - The use of the word find here indicates that the call leverages a search operation for optimal performance. Often times additional processing is done in addition to the search alone.
// Retrieve a list of projects. Note the call getByUserTenant() is a shortcut for querying the user's tenant and calling getByTenant()
List<ProjectRestRep> projects = client.projects().getByUserTenant();

// Retrieve a single project
ProjectRestRep project = client.projects().get(projectId);

// Get a list of all Virtual Arrays
List<VirtualArrayRestRep> virtualArrays = client.varrays().getAll();

// Retrieve a list of Block Virtual Pools for a given Virtual Array
List<BlockVirtualPoolRestRep> virtualPools = client.blockVpools().findByVirtualArray(virtualArrayId);




Filtering

A core capability provided by the client is the ability to filter results. Filtering allows removing items from result based on user defined criteria. There are also a number of built in filters for common operations. Filtering is processed when each bulk operation is performed to the API so it can reduce the memory requirements on the client when doing large queries. For example, you can make a call to retrieve 10k volumes at once if you are only interested in a small number of them. Consider a filter that will match exactly 5 of these volumes. The highest number volumes that will be stored in memory at any given time is the number of volumes retrieved per bulk request.

Custom filters can be created by implementing the ResourceFilter interface. Typically it is easiest to extend from DefaultResourceFilter. There are two methods for performing filtering:

  • boolean acceptId(URI id) - Return true if this ID should be queried.
  • boolean accept(T resource) - Return true if this resource should be returned in the final result set.
// Example using built in host type filter. Returns all physical hosts
List<HostRestRep> hosts = client.hosts().getByUserTenant(HostTypeFilter.ESX.not());

// Find all volumes in a project that have snapshots.
List<VolumeRestRep> volumes = client.blockVolumes().findByProject(project, new DefaultResourceFilter<VolumeRestRep>() {
    public boolean accept(VolumeRestRep item) {
        return !client.blockSnapshots().getByVolume(item.getId()).isEmpty();
    }
});






Searching

The client provides the ability to perform search queries using the search API. The client provides a type safe way to perform searches using a search builder. In addition, filters can be combined with searching to do more complex queries than a simple search alone. Searching calls on the client always filter out inactive results since they use the getByIds() call underneath.

// Search for a project by name. Finds all projects with 'Development' in the name
List<ProjectRestRep> projects = client.projects().search().byName("Development").run();

// Adds a filter to find projects with the exact name 'Development'. first() is a convenience call to return only the first result
ProjectRestRep project = client.projects().search().byName("Development").filter(new NameFilter<T>(name)).first();

// Since the above is so common, a convenience method for doing the same
ProjectRestRep project = client.projects().search().byExactName("Development").first();

// Find a volume by WWN
BlockVolumeRestRep volume = client.blockVolumes().search().byWwn(wwidOfVolume).first();




Create and Deactivate

The following is an example of working with some common operations.

// Create a Project. Note convenience method is provided without specifying the tenant will create in the user tenant
ProjectRestRep project = client.projects().create(new ProjectParam("project name"));

// Deactivate a project
client.projects().deactivate(project.getId());




Tasks

The client provides a lot of tools for dealing with asynchronous tasks. Anything that may take some time to complete will return a Task or Tasks object.

// Example creating a volume
VolumeCreate input = new VolumeCreate();
input.setName("VolumeName");
input.setVarray(virtualArrayId);
input.setVpool(virtualPoolId);
input.setProject(projectId);
input.setSize("2GB");
input.setCount(1);

// Note that create volume returns many tasks. Number of tasks is equal to the count parameter
// .firstTask() is a shortcut to get a single task on a Tasks object
Task<VolumeRestRep> task = client.blockVolumes().create(input).firstTask();

// Get is a convenience method that will:
// - Perform a waitFor() call (waits until the task is complete, error or ready state). Throws an exception if in error state.
// - Performs a query on the resource after the task completes, returning the VolumeRestRep
VolumeRestRep volume = task.get();


// Example creating an export for a volume
ExportCreateParam input = new ExportCreateParam();
input.setName("ExportName");
input.setType("Host");
input.addHost(hostId));
input.setVarray(virtualArrayId);
input.addVolume(volumeId);
input.setProject(projectId);


Task<ExportGroupRestRep> task = client.blockExports().create(input);
// Convenience method to get the ID of the created resource. Can be done before the task completes
task.getResourceId();

// The caller does not need to wait for the operation to complete.
// The following waits for the task to complete, throwing an error if one occurred. Will timeout after 60 seconds
task.waitFor(60000);




Create Volume and Export

The following is an example of a use case implemented in the ViPR UI. The use case is creating a block volume and exporting it to a host. This shows query for valid options for a user to select (for example, only allow the user to select a VirtualArray that is reachable from the selected host).

package com.emc.vipr.client;
import java.net.URI;
import java.util.List;
import com.emc.storageos.model.block.VolumeCreate;
import com.emc.storageos.model.block.VolumeRestRep;
import com.emc.storageos.model.block.export.ExportCreateParam;
import com.emc.storageos.model.block.export.ExportGroupRestRep;
import com.emc.storageos.model.block.export.ExportUpdateParam;
import com.emc.storageos.model.host.HostRestRep;
import com.emc.storageos.model.project.ProjectRestRep;
import com.emc.storageos.model.varray.VirtualArrayRestRep;
import com.emc.storageos.model.vpool.BlockVirtualPoolRestRep;
import com.emc.vipr.client.core.filters.HostTypeFilter;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;


public class ViPRClientApp {
    private ViPRCoreClient client;
    public ViPRClientApp(ViPRCoreClient client) {
        this.client = client;
    }
    public HostRestRep chooseHost(List<HostRestRep> hosts) {
        if (hosts.isEmpty()) {
            throw new IllegalArgumentException("No hosts");
        }
        return hosts.get(0);
    }
    public VirtualArrayRestRep chooseVirtualArray(List<VirtualArrayRestRep> virtualArrays) {
        if (virtualArrays.isEmpty()) {
            throw new IllegalArgumentException("No virtualArrays");
        }
        return virtualArrays.get(0);
    }
    public BlockVirtualPoolRestRep chooseVirtualPool(List<BlockVirtualPoolRestRep> virtualPools) {
        if (virtualPools.isEmpty()) {
            throw new IllegalArgumentException("No virtualPools");
        }
        return virtualPools.get(0);
    }
    public ProjectRestRep chooseProject(List<ProjectRestRep> projects) {
        if (projects.isEmpty()) {
            throw new IllegalArgumentException("No projects");
        }
        return projects.get(0);
    }
    public static void main(String[] args) {
        Logger.getRootLogger().setLevel(Level.INFO);
        ViPRCoreClient client = new ViPRCoreClient("localhost").withLogin("root", "ChangeMe");
        try {
            ViPRClientApp application = new ViPRClientApp(client);
            application.createBlockVolumeForHost();
        }
        finally {
            client.auth().logout();
        }
    }
    public void createBlockVolumeForHost() {
        List<HostRestRep> hosts = client.hosts().getByUserTenant(HostTypeFilter.ESX.not());
        // User choice
        HostRestRep selectedHost = chooseHost(hosts);
        List<VirtualArrayRestRep> virtualArrays = client.varrays().findByConnectedHost(selectedHost);
        // User choice
        VirtualArrayRestRep selectedVirtualArray = chooseVirtualArray(virtualArrays);
        List<BlockVirtualPoolRestRep> virtualPools = client.blockVpools().findByVirtualArray(selectedVirtualArray.getId());
        // User choice
        BlockVirtualPoolRestRep selectedVirtualPool = chooseVirtualPool(virtualPools);
        List<ProjectRestRep> projects = client.projects().getByUserTenant();
        // User choice
        ProjectRestRep selectedProject = chooseProject(projects);
        URI volumeId = createVolume(selectedVirtualArray, selectedVirtualPool, selectedProject);
        List<ExportGroupRestRep> exports = client.blockExports().findByHostOrCluster(selectedHost.getId(), selectedVirtualArray.getId(),
            selectedProject.getId());
        if (exports.isEmpty()) {
            createExport(volumeId, selectedHost, selectedVirtualArray, selectedProject);
        }
        else {
            addVolumeToExport(volumeId, exports.get(0));
        }
    }
    public URI createVolume(VirtualArrayRestRep virtualArray, BlockVirtualPoolRestRep virtualPool, ProjectRestRep project) {
        VolumeCreate input = new VolumeCreate();
        input.setName("SDSClientApp_Volume_" + System.currentTimeMillis());
        input.setVarray(virtualArray.getId());
        input.setVpool(virtualPool.getId());
        input.setSize("2GB");
        input.setCount(1);
        input.setProject(project.getId());
        Task<VolumeRestRep> task = client.blockVolumes().create(input).firstTask();
        VolumeRestRep volume = task.get();
        System.out.println("Created Volume: " + volume.getId());
        return volume.getId();
    }
    public URI createExport(URI volumeId, HostRestRep host, VirtualArrayRestRep virtualArray,
            ProjectRestRep project) {
        ExportCreateParam input = new ExportCreateParam();
        input.setName("SDSClientApp_Export");
        input.setType("Host");
        input.addHost(host.getId());
        input.setVarray(virtualArray.getId());
        input.addVolume(volumeId);
        input.setProject(project.getId());
        ExportGroupRestRep export = client.blockExports().create(input).get();
        System.out.println("Created Export Group: " + export.getId());
        return export.getId();
    }
    public void addVolumeToExport(URI volumeId, ExportGroupRestRep export) {
        ExportUpdateParam input = new ExportUpdateParam();
        input.addVolume(volumeId);
        client.blockExports().update(export.getId(), input);
    }
}




Catalog Client

The following examples show the basics of working with the Catalog client. Construction of the catalog client works much like the ViPRCoreClient. The catalog client requires an authentication token. This can be created through the auth client as described in earlier sections.

ViPRCatalogClient client = new ViPRCatalogClient("vipr-vip").withAuthToken(authToken);




Browsing Service Catalog

The service catalog is a hierarchical structure made up of services and categories. Categories are like directories which may contain both sub-categories and services. There is an API to query categories or services by their ID but they can also be queried by their 'path'.

// Browsing the catalog from the root
CategoryInfo category = client.catalog().browse();
// The above is equivalent to
category = client.catalog().browseCategory("");

// Browsing the contents of path BlockStorageServices
client.catalog().browseCategory("/BlockStorageServices");

// Browsing a particular service
client.catalog().browseService("/BlockStorageServices/CreateVolume");




Retrieving the Service Descriptor

Every service has a service descriptor. This is a JSON file that the portal uses to create a form. It contains the information on what fields are required for a service and their types.

// Querying a service by ID
client.catalog().getService("urn:storageos:CatalogService:ac3899b1-03e0-4b87-8706-c14d2455afa1:");

// Retrieving Service Descriptor
client.catalog().getServiceDescriptor("urn:storageos:CatalogService:ac3899b1-03e0-4b87-8706-c14d2455afa1:");




This is an example of the service descriptor for create block volume

{
  "serviceId": "CreateVolume",
  "category": "Block Services",
  "title": "Create Block Volume",
  "description": "Create a Block Volume",
  "roles": [],
  "destructive": false,
  "fields": {
    "virtualArray": {
      "name": "virtualArray",
      "label": "Virtual Array",
      "type": "assetType.bourne.virtualArray",
      "required": true,
      "select": "one",
      "lockable": true,
      "validation": {
        "min": 0,
        "max": 2147483647,
        "regEx": "",
        "failureMessage": ""
      },
      "options": {}
    },
    "virtualPool": {
      "name": "virtualPool",
      "label": "Virtual Pool",
      "type": "assetType.bourne.blockVirtualPool",
      "required": true,
      "select": "one",
      "lockable": true,
      "validation": {
        "min": 0,
        "max": 2147483647,
        "regEx": "",
        "failureMessage": ""
      },
      "options": {}
    },
    "project": {
      "name": "project",
      "label": "Project",
      "type": "assetType.bourne.project",
      "required": true,
      "select": "one",
      "lockable": true,
      "validation": {
        "min": 0,
        "max": 2147483647,
        "regEx": "",
        "failureMessage": ""
      },
      "options": {}
    },
    "name": {
      "name": "name",
      "label": "Name",
      "type": "text",
      "description": "User assigned description of the volume",
      "required": true,
      "select": "one",
      "lockable": false,
      "validation": {
        "min": 2,
        "max": 128,
        "regEx": "",
        "failureMessage": ""
      },
      "options": {}
    },
    "consistencyGroup": {
      "name": "consistencyGroup",
      "label": "Consistency Group",
      "type": "assetType.bourne.consistencyGroup",
      "required": false,
      "select": "one",
      "lockable": false,
      "validation": {
        "min": 0,
        "max": 2147483647,
        "regEx": "",
        "failureMessage": ""
      },
      "options": {}
    },
    "numberOfVolumes": {
      "name": "numberOfVolumes",
      "label": "Number Of Volumes",
      "type": "number",
      "required": true,
      "initialValue": "1",
      "select": "one",
      "lockable": false,
      "validation": {
        "min": 1,
        "max": 2147483647,
        "regEx": "",
        "failureMessage": ""
      },
      "options": {}
    },
    "size": {
      "name": "size",
      "label": "Size (GB)",
      "type": "storageSize",
      "required": true,
      "select": "one",
      "lockable": false,
      "validation": {
        "min": 1,
        "max": 2147483647,
        "regEx": "",
        "failureMessage": ""
      },
      "options": {}
    }
  }
}




As seen from this descriptor, the following fields are required.

  • virtualArray - Type of 'assetType.bourne.virtualArray'
  • virtualPool - Type of 'assetType.bourne.blockVirtualPool'
  • project - Type of 'assetType.bourne.project'
  • name - String
  • consistencyGroup - Type of 'assetType.bourne.consistencyGroup' optional
  • numberOfVolumes - Number
  • size - number

Querying Asset Options

Any asset types that begin with 'assetType' have dynamic inputs. There is a special 'asset options' API that retrieve possible values for these fields. Some of these types depend on the values of other types.

// Querying asset options for virtualArray
client.catalog().getAssetOptions("bourne.virtualArray");
// Sample Response is [urn:storageos:VirtualArray:951df984-d20b-42ba-b834-6a10c03e89f7: BrocadeVArray]

// Querying asset options for blockVirtualPool. Passing previous parameters.
Map<String,Object> params = new HashMap<String,Object>();
params.put("virtualArray", "urn:storageos:VirtualArray:951df984-d20b-42ba-b834-6a10c03e89f7:");
client.catalog().getAssetOptions("bourne.blockVirtualPool", params);




Ordering Services

Ordering services can either be done by service ID or by using the path to the service. Some services can take multiple parameters. Instead of passing a map in for the parameters, you can use javax.ws.rs.core.MultivaluedMap.

// These are the full parameters to the order
Map<String,Object> params = new HashMap<String,Object>();
params.put("virtualArray", "urn:storageos:VirtualArray:951df984-d20b-42ba-b834-6a10c03e89f7:");
params.put("virtualPool", "urn:storageos:VirtualPool:2b274abf-4e79-41fc-803e-04a1cacf0cfd:");
params.put("project", "urn:storageos:Project:823665f1-1676-4b01-8b6b-114d7705ac0e:");
params.put("name", "vipr-client1");
params.put("numberOfVolumes", 1);
params.put("size", 1);

// Order a service by ID
OrderInfo order = client.catalog().order("urn:storageos:CatalogService:ac3899b1-03e0-4b87-8706-c14d2455afa1:", params);

// Order a service by path
OrderInfo order = client.catalog().orderByPath("/BlockStorageServices/CreateVolume", params);

// Check status of an order
OrderInfo order = client.orders().get("urn:storageos:Order:70b707ab-803c-4c86-ae54-0842ce6a3056:);