Folks, just a heads up for the latest information and blogs from me, check out my new wordpress blog. @Rawstorage – Managing VMAX and PowerMax Storage in a modern IT world
With more and more people asking about REST and how can we to get started I decided to have a look and see if someone with limited programming ability would be able to do anything useful with it. This post will not teach you Python, and assumes basic programming skills.
I recommend learn Python the Hard Way - www.learnpythonthehardway.org, it's totally free if you want to dig deeper and invest in yourself RealPython.com, the $60 course is awesome.
The examples I will give here could perhaps be done by someone skilled in the dark arts of programming in a much more efficient way. Code shown here are functional examples.
First off I would like to credit, Michael McAleer , Ciara Stacke and James Forde some very talented developers in EMC who put me on the right path and put together the bulk of the python code I used in these examples. They have been patient and answered all of my questions to get going.
Ok, so here goes. I'm not going to go too in-depth into REST itself there's plenty of information on the web that will give better explanations than I ever could. Let me google that for you I'll summarize a bit below to give an introduction, but if you already know about REST and just want to know you can skip straight to the Accessing the VMAX REST API section.
So What is REST API anyway?
REST, is an architectural style of shaping webservices, REST stands for Representational State Transfer. It is not a standard, rather a style describing the act of transferring a state of something by its representation. An API (Application program interface) is a set of routines, protocols, and tools for building software applications. An API specifies how software components should interact and APIs are used when programming graphical user interface (GUI) components. Basically REST API gives you a way of interacting with an application without having to know every detail about how the underlying application is implemented.
REST is widely used in the industry, look at any major webapp out there and you'll probably find it has a REST API e.g twitter, instagram, maps.com, weatherchannel.com to name but a few. REST is very lightweight and easy to get started with. Most programming languages provide a way to interface with REST Python provides requests library which lets you do everything you need to do with REST.
This YouTube video gives a great explanation of other REST API's and some ideas as to how to use them REST API concepts and examples - YouTube
Also check out API University | ProgrammableWeb
Providing a REST interface into your product makes it very easy for people to plug in and build on the work that's already engineered into the system. It also enables other vendors and customers to build tools that interface with the VMAX REST API and increase options for customers. In the past we have had storage industry specific standards remember SMI-S. REST unlike SMI-S is widely used in other industries and knowing how to work with REST is a skill that is transferable, development teams don't need to make a significant investment in learning something that is only specific to storage. Once they know REST they can pick up the documentation for any REST API and figure out the REST calls they need to accomplish their end goals and build scripts really easily. For VMAX the heavy lifting is done on the Unisphere server using the same workflows and automation you see in the GUI, it's just a different way of accessing it. All EMC products are providing a REST interface these days.
The Unisphere GUI is designed to be easy to use and provides a great user experience however there may be repetitive tasks that might be better dealt with in a script rather than do through the UI. Another case is that you may want to pull information from the system into another interface for data mining maybe SPLUNK ?.
The VMAX REST API -
It might be news to some but the VMAX software team have been shipping a REST API bundled with Unisphere for many years, it was present in Unisphere 1.6 and provided a means of getting performance information from the Unisphere Performance Analyzer database, back then known as Symmetrix Performance Analyzer. When Unisphere version 8 was being written a decision was made to expand on the functionality of the REST API and add active management, Pretty much everything you can do in the GUI you will be able to do in REST, also if a new feature comes on board it will have a REST interface.
Accessing the VMAX REST API
VMAX REST API is available direct from your Unisphere installation or embedded Unisphere running on the array via the following Base URL
The architecture is as follows.
We also provide a nifty tool that lets you walk the REST schema and is really helpful when building your scripts and methods. server, this is available from the community page below, and the documentation in the form of navigable html pages are available from the server,
Ok, so you know how to access REST and where to get the documentation. Lets look at doing something with it
Unisphere for VMAX's REST API supports the following types of REST calls.
GET - Get state information on objects
POST - Calls to Create an object
PUT - This is making an edit to an objects state, normally you will need to do a get to ensure you have all the before you can do a PUT.
DELETE - Remove Item.
The REST client can be used to help figure out what REST calls you want to run. The example below shows a REST call to pull the Serial Numbers of all systems visible to this instance of Unisphere. The Resources Tree on the left guides you through the resources and you can expand to walk the tree to find the URI you are interested in and the calls each support. The Client shows the exact URI endpoints in the address bar which you can copy to help you build your scripts and methods.
All REST calls will start with the same URI base URL of https://ipaddress:8443/univmax/restapi shown here is the response from the call (in the scripts this is referred to as base_uri).
The example call show will list all Symmetrix monitored by Unisphere the resultant JSON reponse is returned. This is basically a dictionary in JSON payload. Depending on the call you may get multiple lists in your JSON response which is easily parsed.
Note if you are using REST to provision on VMAX2 Arrays use the provisioning resource, if you are provisioning on VMAX3 All Flash or Later use SLO provisioning.
Storage Group Based Control - Unisphere for VMAX is designed around the concept of the storage group for management of applications. A storage group is a container for a set of devices that are related in some way usually representative of an application or a tenant with data on the VMAX array. The majority of operations using REST API are built to take advantage of the storage group as a control entity e.g. SnapVX, SRDF. Using a storage group for management reduces the number of objects that storage administrators need to manage. A storage group must exist for masking devices to a host for access via the Storage Area Network so REST can take advantage of this construct. This will be covered off in more detail in the storage provisioning workflows.
Now that we've looked at how REST works in it's basic form it's probably a good idea to look at doing something useful with it. For this I will present some examples in Python. Python is really easy to get started with. Download and Install, I'm using version 3.6.
PyU4V is available for install from pypi and will install any dependencies for you. Version 3.0 is the latest available and the one we would recommend using if you're getting started. I'll go through it here but more information is available in the Quick Start Guide — PyU4V 3.0.0 documentation
Now you have the PyU4V function library installed you should be good to start writing your scripts. PyU4V can now be imported giving access to 250+ pre-written functions organized into 6 function modules contained in PyU4V, these are
- univmax_conn - Utility functions sets up your connection and reads configuration information from config file
- common - Useful functions for a variety of tasks, e.g. checking asynchronously executed jobs for completion
- performance - Performance Functions, e.g. get_fe_port_util_last4hrs
- provisioning - Provisioning Functions for VMAX All Flash and Hybrid Arrays
- replication - Local and Remote Replication Functions.
- rest_requests - Call Handler Module, called by other function modules to execute GET, PUT, POST and DELETE calls.
You may want to install a good IDE to help you write the code (IDE is integrated Development Environment, it helps with keeping track of variables and autocomplete on method names and variable calls etc.), I really like pycharm, I'm sure there are others but this one has served me well.
As with any programming task, we usually start with Hello World!. In this case I'm going to connect you to your Unisphere instance.
Below is a sample also available in the ProgrammersGuideExamples directory at
The script below
Line 3 Imports PyU4V function library
Line 5/6 Configures Connection and instantiates the session for the script
Line 9 Makes REST call using common function get_uni_version() to get the version of Unipshere server
Line 12 Makes REST call using common function get_array_list() to get a list of all managed arrays by the Unisphere server
The remaining lines of the script just print the details, and finish our Hello Unisphere Script.
Note, I've blurred my Unisphere Server IP as internal people tend to be sneaky and would attach and make changes without my say so.
So you'll probably note I'm passing a whole lot of information via the connection setup, this can also be set using a configuration file, which can be protected to ensure there are no clear text, IPs, passwords or usernames in scripts. Check out the quick start documentation on this.
We've looked at how we can make calls using the common functions, up until now these are all GET calls, these calls did not require us to pass any information as we are just querying for the state of objects that already exist. In the next half of the post I will go into more details about how to request information given a set of constraints and passing parameters for POST calls, to get performance statistics back.
In the next example I am just going to request a number of metrics from the performance REST calls, but you will see from the example you can gather more if you like by editing the JSON payload sent with the POST call.
All performance queries are POST calls, we need to POST specific parameters to the Unisphere Server to tell it what information we want to get back. You can check the API documentation to get the parameters you need to POST or you can simply use the supplied REST client as I am using in the example.
NOTE!!!!! Time and Date for performance are recorded in EPOCH time so you must do a conversion. Python will do the conversion for you in your scripts, but for the Client you will need to do this by hand I am using Epoch Converter - Unix Timestamp Converter to do this, also, Timestamps are in milliseconds so you may need to add 000 to the end, also probably a good idea to make sure that the host you are running has the correct Time and Date so you don't get any funny business due to time being set incorrectly.
Step 1. Build your REST call
Link here for downloading Unisphere for VMAX REST Client
Lets start with the REST client and build up a valid call for gathering Average Host MBs for one of the arrays.
- I've used the EPOCH converter to get time stamps for startDate and endDate one hour apart.
- Filling in the arrayParam section on the Client builds the necessary JSON payload on the right.
- Payload and URI information can be copied into the methods for the scripts.
Step 2. Build your script methods.
At this point believe it or not we have enough information to start building a method for this call in python that we can call in a script. So lets switch to the python IDE, I'm in pycharm here working in my file Get_FE_Stats_LastHour.py. Note this function is already available as part of the performance module which can be imported.
We need to import PyU4V to handle the setup of the session and deal with the actual posting of REST calls, I am also calling the time function as I want to do some epoch stuff later and argparse in case I want to pass in variables when calling the script.
So Back to work. Next thing I need to do is create a method to gather the performance information I wanted and return the result. Sounds easy (It actually is)
Ok.. Next I will simply create a method, for what I need.
In this case I called it get_array_metrics it accepts start_date and end_date. The test scripts automatically generates the datestamps using the time function in python standard library.
To create the function below, I've simply copied and pasted the payload from the REST client and replaced values with variables that can be passed in, python treats the JSON as a dictionary object so no need to do anything fancy with the pasted text. The target URI to my endpoint is also pasted from the client. When the function runs the URL and payload are passed to rest_requests to run as a POST call.
If you note in the json_payload I've added an extra metric to the request that I pasted from REST Client, so the function will return HostMBs and HostIOs for the array for the last hour, potentially I could grab more, but for this example two is plenty, if you look at the actual function in the PyU4V source it's getting many more and this is typically what we recommend with performance calls, rather than make 100 separate REST calls request them all at once.
Explore the functions in performance.py, you'll quickly see they are essentially all similar.
Step 3 Build a Test Script.
To test the method just created, you'll need to create a test file so that you can run it. It's only a few lines of code as everything is done in PyU4V and rest_requests. Reduce, Reuse, Recycle!!!
Running this script will return the same JSON seen in the output from the REST Client although there will be an extra metric returned and multiple values with the timestamps for the 5 minute interval for each.
To store the information I could create a simple cronjob or scheduled job and output to a filename of my choosing, This file could be then imported into another application for parsing or graphing. In my later post I cover this is a bit more detail and show you how to wrap information around the returned JSON and pull out the bits you want.
Please be sure to check out my other posts and examples on the Github.
One last point to mention is that all the documentation is shipped with the unisphere server so just point your browser to
https://IPADDRESS:8443/univmax/restapi/docs and you can download a copy. It's a searchable doc library. I'll cover this off in a future post.
A quick note on versioning, from Version 8.4 all active managment endpoints are transitioned to a versioned endpoint e.g. /84/sloprovisioning/storagegroup when version 9.0 is released all these endpoints will continue to work but if you want to take advantage of new features and functionality available in 9.0 version you will need to update your scripts to use the /90/sloprovisioning/storage group endpoints. We have a changelog to tell you what's changed between versions, so check that out. PyU4V will default to the version of the call for the Unisphere version you are working with.
If getting started with Unisphere REST API it is recommended to use the most recent version of the REST API supported by your Unisphere Server.
I really hope you found this useful. If there are questions you'd like me to try and answer feel free to log a comment and I'll answer.
Follow the links to the right for more examples. For more details on how to use the latest version of PyU4V Python Library Quick Start Guide — PyU4V 3.0.0 documentation
We now have a VMAX Plugin for vRealize Orchestrator – DELL EMC VMAX with VMware This plugin utilizes the VMAX REST APIs to be able to automate the management of your VMware landscape. Check out Drew Tonnesen's blog on this for more information.
Note:- I will to be at Dell Technologies World again this year, if you want to sync up or just say hi, I'm usually around the hands on lab areas.