Find Communities by: Category | Product

PowerMax

14 Posts

In Unisphere for PowerMax 9.1 release we have increased the integration with PowerPath.  With a new PowerMax array with a FX/Pro license (premium) you are entitled up to 75 host licenses. PP will now automatically acquire the FX license from the array for all OSes supported by PowerPath 6.3 and later. The license will appear as “FX” in Unisphere, PowerPath Management Appliance (PPMA) and the array.

 

With over 25 years of engineering and development behind it PowerPath is a family of multipathing software products that automate and optimize I/O performance in physical and virtual environments. PowerPath intelligently manages I/O path loads and failover to increase application availability and reduce latency. Proprietary optimization for Dell EMC PowerMax and VMAX increases I/O performance 2X - £3X over native multipathing.

 

We have heard from customers they want to see more integration with our existing software products in order to help their ongoing automation drive and provide enhanced troubleshooting capabilities from our side.

 

Flaky Path Reporting

The isolation of a flaky path can be a lengthy manual process, therefore PowerPath has been identifying paths as Flaky for years and reporting them in PowerPath GUI. Starting with PowerPath 6.5 – the Flaky path reporting will be sent and saved on the array. The array stores the HBA/WWN/FA Port, Host details and the timestamp in order to monitor the status of the path.

 

Every 24 hours, PP re-tests to see if the previously reported flaky path is still having issues. Then if the array is notified within 25 hours that the flaky path is still problematic the array dials home with the appropriate error code.

 

1.png

 

Now let’s look at Unisphere to see how its reported. In the example above we have two initiators on the same physical host connecting to the same FA port through the same switch on Fabric A.

 

With multiple physical HBAs reporting the issue against the same FA port, and with the H/W lab having checked the array side of the fabric and rule it out - the customer may choose to look at the switch port that FA 1d:30 connects to in order to find the root cause of the reported flaky path problem.

 

Oracle Instance Name


The objective of this feature is to allow users to identify which PP devices are being used to store Oracle data and log files and they can then use this device list to create consistent or recovery snapshots of the database. Here we can see from the Oracle side the instance name “testdb”

 

2.jpg

 

Now in Solutions Enabler you can track the Oracle instance name in a symdev list command with the command structure outlined.

 

3.png

 

In Unisphere for PowerMax you can track the Oracle instance name in the storage group>volume view.

 

4.jpg

 

Path Redundancy Verification Oracle Instance Name


This is a really useful feature which will allow customers to validate that 2 or more paths exist from a device to a host in that it will allow you to display multiple HBAs, multiple Fabrics/multiple switches within a fabric, multiple FA ports.

On an initial view at the cli output below, a customer may determine that they have no single point of failure in their setup below which has paths to 2 different FA’s over 2 different HBA’s.

 

6.jpg

 

However, when looking at the Masking View Path Details screen, we can see that while the paths are across two different switches (FCID 0x21100 and 0x29900), both of these switches are actually on the same fabric as indicated by the Fabric column on the right-hand side of the screen.  Therefore, the screen below shows the fabric as the single point of failure.

 

7.jpg

 

Symppath command


The symppath executable is a new SYMCLI command included with the SE 9.1 release and was created to support future expansion of SE related PP commands.

 

Previously symcfg was used to turn on/off PP related functionality.  This functionality still exists in the symcfg command for SE 9.1. To find out more just run symppath -h from the command line.

 

I hope you found these use cases helpful and if you have any questions please let me know. We plan to keep enhancing PowerPath functionality in the future and build tighter integration with our management software.

 

I would like to thank some learned PowerPath colleagues in preparing this, Owen Crowley in QE and Eric Don in Engineering.

In Unisphere for PowerMax 9.1 we are excited to release the ability for our customers to tag their storage groups and arrays. Previously people were made to rely on naming conventions, and this was open to human error and people forgetting to align to agree upon procedures. Tagging is a powerful feature which will be enable you to view and filter your objects also. To avail of the feature, you will need to be at Unisphere 9.1 and Hypermax O/S code 5978.444.444.

 

A user can add a tag to an object for the purpose of identification or to give other information. An association between objects can be inferred by adding the same tag to each object. Users can then search by tag to retrieve related objects. Tags can be added to and removed from objects. When a tag is no longer associated with an object it no longer exists.

 

Let’s begin to visually investigate what Tagging can offer by going to the SG list view and selecting a SG:

 

 

1.PNG.png

 

 

 

Here we have selected the SG called FinanceOracle and we can see it has a tag Oracle on it. If we require, we can filter(button highlighted in red to the right) on the oracle tag if that is the area we want to focus on if we have a large list of SG’s:

 

 

2.PNG.png

 

 

For FinanceOracle we have decided that it needs additional tags, so we need to add them, to do this. Select the SG itself and go to view details. The next step would be to select the tags Icon (highlighted in red to the left):

 

 

3.PNG.png

 

 

We have decided to add tags “new york” and “trading” so when we go back to the SG list view, we see the SG with the new tags we have created:

 

 

4.PNG.png

 

 

Now with the benefit of tagging we can see that we have a much more organized view of our storage groups and can filter by location application or team as required:

 

 

6.PNG.png

 

 

As well as SG’s you can tag arrays as you like, in this example we just tag by Geography:

 

 

7.PNG.png

 

 

Finally, you can also search by tag in our search facility, if you want to see all SG’s used by the billing team you can search for that tag:

 

 

8.PNG.png

 

 

 

Hopefully I’ve convinced you here of the benefit of using tags in the future and moving away from naming conventions! We have shown you some of the benefits in terms of your SG list where you filter and search per tag and by doing so it will allow you to be able to track and locate your storage objects.

 

 

 

General Rules on Tagging

  1. Symmetrix arrays and Storage Groups can be tagged only

  2. 64 Character limit of letters, numbers, - and _

  3. 500 unique tag names per array

  4. 8 tags per object

  5. Tags can be mixed case and are case sensitive

Note

Unisphere for PowerMax tags have no relationship with Unisphere 360 tags

In Unisphere for PowerMax 9.0 and PowerMax O/S 5978 we introduced a new feature called Storage Group Real Time which allows customers a more granular view of their performance data on the array.

 

Traditionally we offer the ability to view System wide performance data at the director level at 5 seconds but this new ability refers to the SG level whereby we can now see data at 30 second intervals rather than 5 minutes. This enhanced ability gives you the ability to more easily identify performance spikes as they occur.

 

There are certain caveats in order to use SG Real Time:

  • Unisphere for PowerMax 9.0 Required
  • PowerMax O/S 5978
  • 1 Array at a time can be monitored per Unisphere Instance.
  • Maximum of 5 SG’s at a time
  • Limited set of KPI Metrics (Host I/O’s/Sec, Host MB’s/Sec, Host Reads/Sec, Host Writes/Sec, Read RT (ms), Response Time (ms), Write RT (ms).

 

To begin let’s see how we register an array for SG Real Time, we go to Settings:

 

1.png

 

To register the array select Performance>System Registrations. You can see in the 4th column we have SG Real Time enabled on array 0156 as it is running the level of code required. By selecting the icon “Real Time Storage Groups” we can make our selection of what 5 SG’s we wish to monitor.

 

2.PNG.png

 

Presently we are monitoring 2 SG’s and if required we can 3 more if required.

 

Now let’s go to the charting section to see how it works in practice:

 

3.PNG.png

 

In the new chart section you can see we have called our chart RT1 and specified the last hour. In terms of objects we have selected Storage Groups and then we select the SG we can to chart on. Finally we have 7 KPI metrics of which we have chosen 3 here to draw our chart on.

 

4.png

 

Here on the X axis you can see how the time values are reduced down to minutes and we can get finer granularity as required if we are in troubleshooting mode.

 

I hope you find this useful and for a video on this please see here Unisphere for PowerMax Real Time.

In the Unisphere for PowerMax 9.0 release we have increased integration with Powerpath hosts that are discovered on the arrays we monitor in Unisphere.  With a new PowerMax array with a FX/Pro license (premium) you are entitled up to 75 ESXi host licenses. These PowerPath licenses are now automatically deployed for VMware, Windows and Linux hosts with PowerPath 6.3 and later.

 

With over 25 years of engineering and development behind it PowerPath is a family of multipathing software products that automate and optimize I/O performance in physical and virtual environments. PowerPath intelligently manages I/O path loads and failover to increase application availability and reduce latency. Proprietary optimization for Dell EMC PowerMax and VMAX increases I/O performance 2X - £3X over native multipathing.

 

We have heard from customers they want to see more integration with our existing software products in order to help their ongoing automation drive and also provide enhanced troubleshooting capabilities from our side.

 

PowerPath Hosts Landing Page:


In Unisphere on the left hand navigation you will find it under Hosts>PowerPath Hosts.

 

pp1.PNG.png

 

This view gives you a single pane of glass view of all your PowerPath hosts that are connected on arrays that this specific Unisphere instance can see.

 

In this view you are presented with a lot of useful information if you are in the midst of a troubleshooting scenario. By selecting the lcvt1032 host you are presented with a lot more information on the right hand navigation. The PowerPath version and patch level, OS version, revision and patch level, Server hardware, MFG and serial number, PowerPath license type and connectivity type (FC, iSCSI or FCOE) are all displayed.

 

The objects Initiators, Hosts/Host Groups, Masking Views, Storage Groups and Volumes are displayed in blue and can be selected in order for you to drill down to the specific object. At the bottom of the list is the number of VM’s on this ESXi server. Selecting the VM count highlighted in blue will get you to a list of the VM’s as we see here.


pp 1.5.PNG.png

 

 

So let’s say we need to quickly find out what are the initiators for lcvt1032 as we are investigating an internal ticket for a performance issue.

 

pp2.PNG.png

 

Now in this view to drill down on your performance problem you can select volumes to see what volumes are assigned to this host.

 

pp3.PNG.png

 

When you select an individual volume you can see the right hand pane shows a wealth of good information including whether if the volume is mounted. In this case it is not so that may indicate the root of the problem. If the dev was mounted, below the mount status will be the name of the host process using the dev (for ESXi it’s the VM name). The host process/vm name allows you to figure out what application is using the device. At the bottom of the device information is the date when date when the device last received I/O from any host. This allows you to find the devices that were not used for a while, and the combination of host name and process/vm name allows you to identify who owns the dormant device.

 

I hope I’ve displayed some of the power of the troubleshooting you can potentially do from the PowerPath landing page when you are investigating a potential PowerPath issue.

 

Auto Initiator Group (IG) creation using PowerPath Host Name:

On the main landing page we also have a new option called create host which will allow Auto IG creation:

 

pp4.PNG.png

 

On 5978 HyperMax code and Unisphere/Solutions Enabler 9.0 allows for the hostname to be provided when creating an IG. Unisphere/SE picks the host HBA WWN’s from the Host registration data provided by PowerPath and add them to the IG for you. This significantly helps you in the overall provisioning flow if adding a new PowerPath host.

 

The initiator array switch that controls AUTO IG creation is disabled by default, but you can turn it on by selecting it in the settings section of Unisphere.

 

pp 4.5.jpg

 

Auto IG Creation works in tandem with the PowerPath Host registration, if the PowerPath host registration switch is disabled, then the Auto IG Creation will also be disabled.

 

As of now Linux and AIX PowerPath will automatically detect Devices added or deleted on a certain path following masking or zoning changes (ESXi and Windows already does this). Once PowerPath Devices have changed it will run a host scan and the O/S will discover the new Devices.

 

This significantly helps you in the overall provisioning flow if adding a new PowerPath host. It achieves this by automatically pulling the WWN’s registered with that hostname provided the correct settings are enabled as outlined below. This reduces some of the configuration work that the user has to perform when setting up a host.

 

Device in Use Details:


Another really powerful feature is the Device in use example. By using Unisphere or SE you can easily see if a device is mounted and active by looking at the volume level. In Unisphere you would navigate down to the volume level:


pp5.png

 

On selecting a specific device the right hand pane will show more detailed information. We can see here that the device mounted state is true and is used by a process called consist_lun_x (for ESXi it would be the vm name). Therefore we can see if a device was used lately and who is the owner (host, mount status, application or vm name). The same information can also be viewed in SE with symdev list –sid xxxx –ppi.

 

SE screenshot.png

 

The use case we were looking to resolve here is that often SA’s get requested for resources on the array by application owners and they do so to future proof themselves for projects. Although from an application owner this makes perfect sense but from a SA whose job is to manage array usage and consumption it is not best practice. In certain cases people who manage the capacity of the array can leave or transfer and the capacity they owned goes unused. Up until this feature there was no good way from the storage side to track it down.

 

This information is updated every 24 hours and if the SA feels that application owners are taking liberties by not mounting volumes or using storage they can have a frank discussion with the application owner!

 

PowerPath Metrics:


Another new feature we have added is we’ve exposed some PowerPath metrics in our Performance section. These metrics are % PowerPath Observes Relative RT (PowerPath reported RT divided by array RT), PowerPath Average Response Time (ms) and PowerPath Observed Delta RT (ms) (PowerPath reported RT minus the array RT).

 

pp6.png

 

These metrics can be accessed in the all metrics section by using the radio button slider. These new metrics are available at the SG and Thin volume object type level. Here’s an example showing the differences in the three measurements vs array response time.

 

pp 7.png

 

In this example the array response is .41 ms, while the average I/o Response time as measured by PowerPath from the host to the array is .60 ms. This .20 ms (observed delta) and 147% relative delta (%PowerPath observed Relative RT) difference can help identify where the slowdown is occurring – not at the array level here! A spike in PowerPath observed Relative RT % may indicate SAN slowdown or a possible slow drain problem which occurs outside of the array.

 

I hope you found these use cases helpful and if you have any questions please let me know.

 

I would like to thank some learned PowerPath colleagues in preparing this content, Owen Crowley in QE, Eric Don in Engineering and Robert Lonadier in PM.

Application awareness for VMware is a new feature of Unisphere for PowerMax, which allows users to troubleshoot storage related problems to ESXi servers.

 

Using a read only vCenter user account, a vCenter or an individual ESXi server can be added to Unisphere.

 

The discovered items are then associated with local storage devices to Unisphere and mapped to storage related objects making it easier to investigate possible storage related problems.

 

The following are some of the actions a user can view on this new feature:

 

  • Masking Views related to ESXi server
  • Performance KPIs related to ESXi Server
  • Details of ESXi Server
  • Virtual Machines on ESXi Server
  • Storage Groups related to ESXi Server
  • Local Storage Capacity of ESXi Server
  • Details of Virtual Disk under a VM
  • Allows you to find the ESXi Server a VM resides on and related storage objects.

 

It is easy to identify potential storage related issues associated with your ESXi server by examining the main vCentre/ESXi server list view.

 

If any Storage Group is out of compliance an indication will display too highlight, also if there is a problem with Front End Ports utilization then this will able be indicated. Clear relationships between the storage objects and vCenter/Esxi server(s) make diagnosis of a potential problem swift.

 

The ability to locate an ESXi when given a particular VM name accelerates troubleshooting efforts and can rule in/out the problem having anything to do with the local storage array.

 

Discovery of VMware related objects can be scheduled by using the Job feature of Unisphere for PowerMax.

 

** Please Note currently no VMware related statistics are retrieved.

 

Getting Started with VMware Unisphere

 

Login to Unisphere for PowerMax

 

Select HOME > VMWARE > vCenters and ESXi

 

image1.png

 

The user can then register a whole vCenter or add individual ESXi servers. Registering is easy!

 

All that is required is the vCenter/ESXi server name or IP address and a username and password.

 

The user then has the option to run the discovery now or Add to Job List and run at the required time.

 

image2.png

Once done you get can view the esxi servers listed for the vCenter - note this only shows Esxi servers local to PowerMAX!

 

Storage group compliance and Front End colour coded status indicators draws your eye quickly to any potential issue. More information is readily available such as the VMware version and build number, CPU cores, Memory by double clicking one to the esxi server.

 

image3.png

 

Investigate a potential SG Compliance Problem

 

In this use case the user is presented with a yellow warning triangle in the Storage Group compliance column in the main list view.

 

This means this Storage Group is outside of the maximum response time for it's defined service level and might indicate a potential problem. Workload Planner calculates its weighted response time for the past 4 hours and for the past 2 weeks, and then compares the two values to the maximum response time associated with its given service level.

 

If one of them is in compliance and the other is out of compliance, then the compliance state is MARGINAL. If both are out of compliance, then the compliance state is CRITICAL.

 

image4.png

 

To investigate further you can View All Details of an ESXi and navigate to the Performance tab.

 

image5.png

 

Here you can see more KPIs and some other related performance objects.

 

Once the potential troublesome SG is determined the user can also now navigate to the Performance section of Unisphere and investigate further.

 

image6.png

 

A tip/trick I found useful when trying to locate an Esxi server or VM is to use the Unisphere for PowerMax search icon.

 

image7.png

 

 

Great to hear your thoughts on this fantastic VMware application awareness in Unisphere!

 

Blog Authored by Derek O'Mahony @DerekOMah

Getting The Most From Unisphere Database Storage Analyzer

 

Database Storage Analyzer (DSA) is a feature of Dell EMC Unisphere for PowerMax, that provides a full representation of the mapping and performance between database objects and Dell EMC PowerMax storage devices.

 

DSA can clearly highlight the location of a potential problem with a database (Oracle/Microsoft SQL Server) or associated local Dell EMC PowerMax storage array.With no additional cost associated it’s definitely worth kicking the Unisphere Database Storage Analyzer tires!

 

Array properties, devices, storage groups and allocated capacity associated with the Database storage are just some information available in DSA. The DSA Storage group(s) link allows you to drill down into the performance and masking view (host initiator, ports etc) details of your environment.

 

Database objects with the most IO wait time associated host, host device, sessions, wait classes focusing troubleshooting and diagnostic efforts for a particular database can all be easily identified.

 

As well as viewing performance, the DSA FAST hinting feature on PowerMax Hybrid arrays allows you to accelerate your mission critical processes while maintaining the designated service level enabled by the FAST engine on your Storage Groups.

 

WHERE TO LOCATE DSA?


Simply login to Unisphere for PowerMax

 

Select HOME > DATABASES

 

where is dsa.png

 

Initial Setup/Registration of your Database with DSA

 

To add Oracle or MS SQL Server Database(s) use the Register Monitored Environment wizard.As root (or superuser) access over SSH is often disabled, but DSA can also cater for SUDO users where the DB Host Administrator would set up a designated account and grant them the necessary subset of the root privileges via SUDO. This /etc/sudoers config allows the qa11 user to run the mapper_agent.sh as root without a password, from ALL hosts.

 

"qa11 ALL=NOPASSWD:/var/tmp/UnisphereMapper_Unispherehostname_ora33_on_dbhostname/mapper_agent.sh map -file dbfiles.dat -log_file dbc_mapper.log, /bin/ksh"


From a security view point, DSA uses role based access, so restricting who does what with respect to the database and storage areas of Unisphere for PowerMax, very useful indeed!

 

Let’s take a look at some Database Storage Analyzer use cases :

 

Use Case #1:  Host CPU impacting Database Performance

 

 

The Database Storage Analyzer list view clearly indicates that the Oracle database ora33 is experiencing poor performance response time by colouring part of the database response time status red.

The storage read response time appears to be as expected however the database read response time is high at 84.83.

 

image2.png

 

Select the VIEW ALL DETAILS button to investigate further.

 

The Active Session Wait chart illustrates that there is a large amount of CPU Count and OS CPU activity on the database host.

 

image3.png

 

For further confirmation that the storage is performing as expected the database and storage reads, writes and redo writes can be investigated on the Response Time and IOPS charts.

 

image4.png

 

Use Case #2: Determine the cause of  high response time for both DB and storage, potentially caused by a long running Database backup.

 

image5.png

 

Database ocrlhercules has a high DB and Storage Read Response time

 

image6.png

 

View details, graph titled ‘Response Time ms’ showing the high DB and Storage reads response times for a defined time period.

 

image7.png

 

Select the Analytics tab, the database object with the largest IO wait time can be quickly identified.

 

image8.png

 

Details about this object such as the program with the largest IO wait time is also ease to find.

 

image9.png

 

The associated devices for this database object are obtained by clicking the devices tab.

Shout, if you require any assistance getting started with Database Storage Analyzer!

 

Hopefully you enjoyed this blog !

 

For a video on DSA please visit here: Database Storage Analyzer - Troubleshooting a DB RT Issue

 

Blog Authored by Derek O'Mahony @DerekOMah

An essential part of Array management is monitoring for specific events or error conditions. When an event or error does occur Unisphere displays an alert and, if configured to do so, notifies you of the alert by the way of Email, SNMP or Syslog. If you wish to find out more about configuring SNMP and the Syslog please refer to this guide.

 

For a brief over view please click here.

 

Alert Policies


To begin with let’s start with the Alert Policies section. In the following examples we have focused on a 5977 PowerMax array. To navigate to here please select the Settings button as displayed by a “cog” icon in the upper right hand corner of Unisphere.

 

alert policies.PNG.png

 

When you install Unisphere some of these categories come enabled by default. However in order to capture a complete picture of the health, capacity and behaviour of the array we would recommend the following categories you can see to be enabled:

 

  • Array Component Events
  • Array Events
  • Device Config Change
  • Director Status
  • Hotspare Invoked
  • Port Link Status
  • Port Status
  • SRDF Link Status
  • Storage Pool Projection Alert

You can see here that the notifications turned on is email. It can also be via SNMP or the syslog.

 

Notifications:


To configure these please select the Notifications on the left hand side navigation:

 

notifications.PNG.png

 

Here select the configure button below email in order to set it up. You can configure email addresses to which notifications, alerts, and reports are sent. You can configure a single email address for all notification instances, or you can use different email addresses for different notifications on different storage systems.

 

configure email.PNG.png

 

In the Outgoing Mail Server (SMTP) section you need to specify the IP Address or Host and also the port you are using. In the User Information you will need to provide a sender email address that will make sense and here we have just called it out as Array0111@dell.com. You select create to configure an email address that will receive the alerts, in general it will be the storage team monitoring this so storagegang@acme.com would be a valid and relevant mail address.

 

Finally select which array and category which is relevant to you so here we have selected System, Performance and reports as we want to be alerted on these.

 

Compliance Alert Policies


This section is concerned with monitoring the service level compliance state of our storage groups.

 

compliance alerts.PNG.png

 

Here we have selected all the storage groups on array 0111 and the state we wish to monitor for is marginal or critical as we are not concerned with a stable SG state.

 

compliance alerts1.PNG.png

 

As you can see here we can have enabled email notifications for a certain number of SG’s if their state goes marginal or critical.

 

Performance Threshold and Alerts


This category will allow you to monitor the overall performance of the array and alert on it accordingly.


performance tresholds and alerts.PNG.png


In this example we are focused on the 0111 Array. Here we are focused on the array category which will allow us to track and alert when we go over a certain number in terms of a threshold across a variety of categories that we can see on the list. You can see that this list includes the KPI’s and generally the KPI’s that we pre-configure for the customer will satisfy most people’s needs but you may want to adjust the values based on your own preference.

 

As well as Array there are 5 other categories such as Storage & Hosts, Directors & Ports, Storage Resources, Data Protection and System Internals.

 

So I wanted to show a use case example here so I will select Storage & Hosts > Storage Groups.

 

performance tresholds and alerts-SGs.PNG.png

 

So for this example I am very sensitive about the Response Time of my SG’s on this array so I choose the pencil icon next to RT so I can edit it.

 

edit performance treshold alert.PNG.png

 

The default values were 20 and 30 so I have reduced them to 10 and 20. Next I select the trigger alert on the lower part of the screen.

 

trigger alert.PNG.png

 

I have selected the checkbox for trigger an alert and my first threshold is just a warning but the 2nd is a critical alert. The number of occurrences in the data samples which must happen before the alert is triggered. For example, if the threshold is breached 3 times out of 5 samples, an alert is imitated. The number of occurrences in the data samples which must happen before the alert is triggered. For example, if the threshold is breached 3 times out of 5 samples, an alert is initiated.


As I stated previously most people are happy to accept the pre-configured thresholds that are automatically setup when you install Unisphere. If you do wish to change these you have that ability and we have shown you a practical example above.


Symmetrix Threshold Alerts


These are set of alerts based on certain categories such as capacity consumption, Meta Data Utilization and Local Replication Utilization.

 

symm treshholds and alerts.PNG.png

 

In this example the array is a 0111 PowerMax box so we have 6 categories Storage Resource Pool Utilization, Storage Container Utilization, Frontend Meta Data Usage, Backend Meta Data Usage, Local Replication Utilization and System Meta Data Utilization. There are different categories for older arrays such as Fast and thin pools but are not applicable to the newer arrays.

 

In terms of the 3 various thresholds Warning, Critical and Fatal we have gone with the default values. We can of course customize these if we wish also and in terms of notifications we have again chose email but SNMP and syslog are also available.

 

Following these guidelines will enable you to successfully monitor your array's performance. I hope you have found this blog helpful and if you have any questions please feel free to reach out with questions.

This blog is intended to give you an overview of the new release Unisphere for PowerMax 9.0. This release has given us a great opportunity as we moved from the old Flex framework to the more modern and secure HTML5 design. With this green field canvas while writing a new code base we engaged pro-actively with our customers in order to have a release that is highly intuitive and offers enhanced troubleshooting capabilities while retaining its core functionality from prior releases.

 

This Agile process allowed us to receive feedback from customers while in development which allowed us to put this feedback directly into the product before we launched which allowed our customers a true sense of being listened to and having their input appear in the GA product. Significant application responsiveness has been observed in Beta customers environments already.  If you are interested in a overview video please follow this link.

 

In the upper right hand side we also have a “What’s New” section which describes some of the new features we have included in this release.

 

Upon login you are presented with the Datacentre view of Unisphere which allows you to see all of the arrays both local and remote that the Unisphere instance can see. This view comes from Unisphere360 that allows you a single pane of glass view of your environment. In the upper right hand side you have a sorter which allows you to filter by Compliance, Capacity, Health Score, IOPS, Throughput and Efficiency.

 

pic1.png

 

On the left hand side Menu you will also see new options covering VMware and Databases. We have labelled these together as Application Awareness and we have endeavoured to give our customers more of an end to end view of their storage from their application down to the storage device. By entering read only vCenter details you can get more information on how your Esxi servers are configured with masking and mapping details down to the storage. To learn more please see here.

 

Databases encompasses Database Storage Analyser which in the previous release was a separate link and launch but has now been further integrated into the product and this feature allows customers to do advanced troubleshooting on potential Database issues. For additional details on both these topics please see this blog.

 

If we select an array we are taken to the array dashboard view which will allow you a more in-depth view of you array.

 

pic2.png

 

With this sharp new design we are presented with an overall summary of your array across a number of categories. On the left hand side we have the various categories similar to previous versions. In the capacity section we display how much space has been subscribed and what usable capacity is left. Additionally we display efficiency scores which you can learn more about this topic here.

 

In the performance section here we present 3 KPI’s Host I/O’s, Latency and Throughput in a 4 hour window so that you can view any potential spikes in recent activity, this time window can be extended to 1 day and 2 weeks also.

 

Across the top of the screen we have 4 separate tiles System Health, SG Compliance, Capacity and Replication which I will go into more detail now.

 

pic3.PNG.png

 

The health score gives you a % rating score for the health of your array based on a traffic light system with Green being good at 95-100%, Orange is fair at 74-94% and Poor being Red at 0-73%. This is based on 5 core factors: Configuration, Capacity, System Utilization, and Service Level Compliance and SH Response Time. The area with the highest risk to your system health will hurt your score until actions are taken towards remediation.

 

In this example we have a score of 90% as we have an SG out of compliance which has reduced it by 10%. If for example here I were to resolve this by placing this SG at a higher service level or placing a Data Exclusion window on it, my score would improve to 100%. Your score is based on taking the worst score from the issues list, -10 here and subtracting it from 100%.

 

pic4.PNG.png

 

The next section is SG Compliance where Storage Groups listed by response times whereby you can see which SG’s are consuming the most resources on the array, you can sort alphabetically here but the feedback received from customers indicates that RT is preferred here. In the left hand column we can see which status of various SG’s are currently in. We can see we have 5 marginal SG’S and if we click here we are presented a list of the 5 SG’s in a marginal state.

 

pic5.PNG.png

 

The next section is capacity which allows you to quickly see how much capacity your array is subscribed for, and how much of that subscribed capacity has been allocated (written). The view also shows you how efficient your array is in using Data Reduction Technologies. In this section you have the option of a toggle button where you can have a more detailed view of what’s being consumed or saved.  For more information on Data Reduction please see the following whitepaper.

 

pic6.PNG.png

 

In the replication section we have new functionality in the Topology/Migration view where we have looked to represent a SRDF Topology view which shows a health status for your RA groups or SRDF links. Please note the ability to expand this view if needed in the upper right hand corner if you wish to view it in full size mode. If your links are down and no data is going across the link then you will go red as you will need to take corrective action. If a singular link is down or there is a problem with an RA group then the colour will go Orange and finally if all green your links are up and data is being copied across the link.

 

On the bottom of this view you will see a legend which enables you to see what mode of SRDF you are running between arrays. Finally on the left hand SG summary box you can select a SG if it is in error and drill down from here to take corrective action. If you are managing a NDM migration you can switch over in order to monitor that as it progresses.

 

pic7.png

 

In the upper right hand side as represented by a cog icon we have the new settings section which groups together a set of systems wide preferences and configurations such as User Management, Performance Collection settings, Licensing and Alerts. Settings are available all of the time and easily accessible without navigating away from the current view or task.

 

The Role Based Access Control (RBAC) or User Authorization feature released in Unisphere for PowerMax 9.0 enables you to restrict the management operations that individual users or groups of users may perform on their Storage Arrays.

 

pic8.png

 

Storage Administrators can have responsibility for managing a lot of backups for application owners and this work can be time consuming even with using scripts as they may get unique requests that fall outside defined windows. In order to reduce their workload and hand a certain degree of responsibility over to application owners we have created 2 new roles LocalReplication and RemoteReplication in order to provide them with the ability to perform their local and/or remote backups. These roles are strictly replication based and as always active management falls under the admin role.

 

pic9.png

 

pic10.png

 

In the top right hand corner we have a global search facility which allows you to search across all of the arrays your Unisphere can see for a specific set of objects including Storage Groups, Initiators, Hosts, VM’s or ESXi Servers. Once selected it will open that object’s details view. In this example I am searching for SG’s called esx across my arrays.

 

pic11.png

 

We have also provided the ability to save a provisioned Storage Group as a Provisioning Template. This will really help you when you want to provision multiple instances of similar workloads. The template takes into account the capacity and performance profile of the original saved workload, allowing you to check the system to see if another workload of this type will fit. Video example here.

 

pic12.png

 

With this release we have given users the ability to search in columns to target what they want, here I am in the Storage Group list view and I’m looking for a SG called test and I now can just search any SG with the keyword “test”.

 

I mentioned earlier that we have retained all our core functionality that existed in previous releases but we have made significant improvements in the Performance Section.

 

On the left hand menu we have 7 categories Storage, Hosts, Data Protection, Performance, System, Events and Support. These have been improved on from the previous version but the core functionality has remained the same in these sections.

 

There are separate blogs on performance below as we have made significant changes to this section with superior usability and functionality for the customer.

 

pic13.png

 

Performance Troubleshooting with Charts

Using the Heatmap, Reports and Plan Features

Performance troubleshooting with Dashboards

Performance Troubleshooting with Analyse

 

I hope this gave you a good introduction into the new release and for a full list of all of our content please visit our community page here.

Feature Overview


The Role Based Access Control (RBAC) or User Authorization feature released in Unisphere for PowerMax 9.0 enables you to restrict the management operations that individual users or groups of users may perform on their Storage Arrays.

In previous versions of Unisphere authentication was array wide. This set of enhancements will provide the following:

  • More granular support by giving the rights to operate on individual applications (Storage Groups) to application administrators, but not the entire array. 
  • Provides the administrator with the ability to further target user access to specific replication roles whether that is local or remote.
  • RESTAPI integration allowing associated scripts to take advantage of these RBAC controls to simplify the management stack and overall maintenance as well as eliminate the need to deploy Solutions Enabler gatekeeper devices.

 

RBAC Roles

 

It is managed using Unisphere for VMAX, Unisphere for PowerMax, or the Solutions Enabler CLI symauth command. Using symauth, a user or group of users, may be mapped to a specific access role, which defines the operations that these users are permitted to perform on the entire VMAX array.

There are currently 7 user defined roles that are available with RBAC: None, Monitor, PerfMonitor, StorageAdmin, SecurityAdmin, Admin, and Auditor. Listed below are the base capabilities of these current roles:

 

  • None No capabilities
  • Monitor Performs read-only operations on an array excluding the ability to read the audit log or Access Control definitions.
  • PerfMonitor Includes Monitor role permissions and grants additional privileges within the performance component of Unisphere for VMAX application to set up various alerts and update thresholds to monitor array performance.
  • StorageAdmin Perform all management and control functions. Please see specific section pertaining to this role below.
  • Auditor Grants the ability to view, but not modify, security settings for an array (including reading the audit log, symacl list and symauth) in addition to all monitor operations. This is the minimum role required to view the array audit log.


It’s important to clarify that your Storage_Admin role will remain your “Super user” and will remain sole control of provisioning storage on the array. To clarify this further here is a diagram which will outline the various roles and how they interact with one another:


RBAC Overview.png


How to configure RBAC


In order to set RBAC you will need go to the settings section and then Users and Groups and then Local Users.

 

RBAC-settings.png

 

To outline the feature I will now outline a number of the potential use cases it will be used for. 

 

Performance Monitor Role


In a situation whereby a new junior storage administrator joins the team you want them to learn more about the array but you are also conscious of giving them too much responsibility before they have a good understanding of the storage. By allowing them these privileges you enable them to do performance troubleshooting within Unisphere and also adjust various thresholds and alerts if required. This will allow them to get a good grounding of the architecture before they move on to active management of the array.

 

security.png

perfmonitor.png

 

Security Role


  Increasingly security plays an integral part of the management of the today’s data center. In order to allow the security team to do their job they need the ability to view certain logs and run certain query commands such as symaudit in order to check the system to see if there are any potential vulnerabilities or if some user has been doing something they should not have been performing due to the incorrect permissions being set. The 2 roles SecurityAdmin and Auditor should satisfy the security team’s needs in this respect. They will not have any active management or replications roles as they don’t require them to do their jobs. 


Application Owner Role


Storage Administrators can have responsibility for managing a lot of backups for application owners and this work can be time consuming even with using scripts as they may get unique requests that fall outside defined windows. In order to reduce their workload and hand a certain degree of responsibility over to application owners we have created 2 new roles LocalReplication and RemoteReplication in order to provide them with the ability to perform their local and/or remote backups. These roles are strictly replication based and as always active management falls under the admin role. 

 

appowner.png

 

For these replication roles we allow you the granularity of selecting individual SG’s. Here I have selected App1_SG as that is the one the user is responsible for. Here you also have a wildcard option whereby if you had an application owner with multiple sg’s that were labelled oracle_trading you could assign privileges to all of these in 1 click.

 

sglevel.png

 

You also have the ability to manage RBAC through your RESTAPI or symcli as required. For a useful video on RBAC please check this out: https://www.youtube.com/watch?v=2V7KidifeA4

 

For a more detailed deep dive on RBAC please see this whitepaper: https://www.emc.com/collateral/technical-documentation/h17132-role-based-access-controls-rbac-technical-overview-and-enhancements.pdf

One of the most critical areas for any Storage Administrator is monitoring and planning the capacity growth of your system so that you can plan your needs accordingly. To make this easier you can create reports within Unisphere with the categories you wish to monitor. This allows you to automate the process and get a tailored report sent to you each week and also other colleagues as required.

 

create report.PNG.png

 

To begin we just give the report a name and description which makes it identifiable, here we have chosen Capacity Report 0156. For the time frame we have adjusted it to cover the time period of the last week. Also for generated time zone you need to adjust it the time zone the array is located so the numbers make sense.

 

create query.PNG.png

 

Next we need to add the metrics that we want to focus on around capacity such as Usable, Subscribed and snapshot Capacity. In Data Format we want to capture average and maximum values so we select both.

 

set schedule.PNG.png

 

In the set schedule screen we need to select it to be run evrey week and then specify when to run it. We have choosen 04:59 each Sunday night as this is outside our normal change control window and at a time where the system will not be heavily used.

 

 

enable email.PNG.png

 

In order to specify who the report will be emailed to we just need to specify a valid email address and it will land in that person’s email inbox.

 

report created.PNG.png

 

Alerting on your Capacity growth in Unisphere 9.0


Another critical factor in monitoring your capacity growth is to enable alerts to allow you to get notifications on when you reach certain levels of utilization on your system. To do this you need to go to the settings icon in the upper right hand corner.

 

settings-alerts.PNG.png

 

We have a number of options here but for this particular use case we want to focus on Storage Resource Pool Utilization.


create symm treshhold alert.PNG.png

 

Here we select the SRP we want to monitor and we have 3 separate watermarks Warning, Critical and Fatal. You can adjust your settings as you require in order to inform you as the capacity of your array grows.

 

Hopefully you found this useful and it will help you track and monitor your Array Capacity.

Customers has told us the heat map is still popular and they like a physical representation of how busy the array is like we see here below. The variance in colors represents utilization levels. Clicking on a square will give you further metrics relevant to that object and you can drill down further on this if you wish.

 

17.PNG.png

 

In this release we have included a planning ability whereby you can project potentially how busy an object will be based on current usage out to 6 months, this will allow customers to place their workload on ports that are not expected to be busy over the coming months which should stop performance problems from even occurring.

 

18.PNG.png

 

Additionally we have allowed you to filter based on storage group. In a troubleshooting scenario you can easily select your storage group such as irco2sd15 here and filter the heat map based on the Fa’s that storage group is mapped down just as we see here:

 

19.PNG.png

 

20.PNG.png

 

Finally we allow users to drill down on specific objects within the heat map as required, here that director 2B is turning red and is quite busy so if we hover over and select it we get a picture of how busy it has been over the last 4 hours. If we wish to navigate to analyse or charts we can so from here and it lands us here in context with that particular director already selected.

 

The Reports section has not changed much in terms of content but it’s presented in a more consumable format. You can produce a report on demand on items such as capacity or you can choose to schedule it to be run at a certain time. Finally if you want a report to be emailed to you or a colleague you can also configure this.

 

21.PNG.png

 

The plan section displays predicted future data, based on linear projection. You can use the information contained in the projection dashboard to assist when planning and to avoid potential problems. It allows you to estimate and chart capacity growth based on your SRP or your thin pool.

 

It will give you a days to full value and a capacity figure over the course of 1 week, 1 month, 3 months and 6 months. You do require 10 days of performance data in order to generate this chart but it is a really powerful tool for storage administrators who want to view and chart their capacity consumption.

 

22.PNG.png

 

Here we are focused on SRP1 and a 6 month time frame, in the chart the yellow line represents the projected capacity growth for this box based on current capacity usage. Also highlighted in red we see are given a days to full value based on current trends.

 

For a video outlying these properties see here: https://www.youtube.com/edit?o=U&video_id=ZIBbvDULdhQ

 

I hope you found this useful and if you have any questions please let me know.

The Analyse section provides 3 views into your storage data. You can use real time and diagnostic views to debug and find causes for performance problems. You can then use historical to show long term trends and plan for future needs.

 

  • Real Time – Collects Data between 2 and 5 seconds for a limited set of metrics at an array level and is available for 1 hour.
  • Diagnostic – Collects data every 5 minutes for Root cause analysis and is available for the previous 7 days.
  • Historical – Collects data every 15 minutes and is available for the previous year.

 

Real Time is focused on an array level view where your focus is on FE/BE Directors and RDF Directors, you can also look at external directors if desired.

 

11.PNG.png

 

As highlighted in red you have the ability to capture a trace if required and display it in the charts section so you have a visual picture of your performance. Bear in mind this is displaying metrics at between 2-5 seconds so it offers you an insight into how the box is behaving at present in a really granular format. To capture the trace you merely select the individual director or all of the directors and then select capture trace.


12.PNG.png


To display this trace in a chart you merely change views to the charting section and select real time and then trace and you have the option to select 9 KPI’s related to directors so you can focus on metrics like BE IO’s per sec and % Cache WP.

Diagnostic provides high-level data about all the storage systems. The specific information depends on the models of the selected storage arrays.

 

13.PNG.png

 

Highlighted in red to the left we have the category of objects that we can select and drill down on. The list is ranked in terms of how busy the SG based across the values displayed. To the right you have the ability to change the time frame from 4 out to 24 hours. Additionally with this list having over 500 objects you have the ability to filter on it if you require.

 

I wanted to go over some of the potential use cases also here which will aid you in your troubleshooting. Typically when a storage administrator begins a performance troubleshooting task their starting point from an application perspective is the host.

 

14.PNG.png

 

To do this quickly we can leverage the Analyze section to select hosts and our scope is the last 12 hours, in this particular example the application owner is running on host irco2sd15 and although the host IO’s/Sec and host MBs/Sec are high the host is meeting its response time as we are green in the 3rd column.

A very useful section here is the events and alerts piece if selected it gives you a view into what events and alerts occurred in the last 24 hours.

 

15.PNG.png

 

Here we have the ability to go back to the last 24 hours and also customize the view so if we have a large number of configuration changes we can select a certain timeframe such as 2:00 to 2:30 last night. For this example I have received a call that someone has lost access to their host CA and so I can look back to the changes and I notice that someone ran a delete operation on the port group CA which might explain the loss of service.

 

Finally we have the Historical view which provides high-level data about all storage systems. This allows you to take a more long term view of how certain elements in your storage array have been performing over a longer time frame going back to 12 months. I just wanted to show you 1 potential use case here for a host.

 

16.PNG.png

 

You can see by the breadcrumb selection that I have drilled to focus on the initiator ports 1D and 3D as while investigating a performance problem with a host the application owner mentioned that performance has been slow for some time. Focusing on the response time for these FA’s we can see that over the course of the last 12 months it has been within expected parameters.

 

I hope you found this helpful, if you have any questions please let me know.

With HTML5 we have made significant improvements in the charting section which offers our customers a greater deal of functionality in allowing more flexibility in how they interact with the product combined with a reduction in the time spent identifying potential Array performance problems.

 

This release is significantly different from the previous in terms of layout and usage so I wanted to touch on certain key elements before I delved into some of the potential use cases.

 

6.PNG.png

 

You have the ability to name the chart if you wish but it’s optional, in the timeframe drop down you have the following options:

Realtime – the last hour at 30 second granularity for up to 5 SG’s at a time on 1 array covering certain KPI’s or run a trace at the system level.

 

Diagnostic – 1, 2, 4, 8, 12, 24 hours or customizable allowing you to focus on a certain timeframe such as 03:15 – 03:45 a.m. last night.

 

Historical – last 24 hours, last week, last month, last 6 months, last 12 months or customizable allowing you to focus on a certain timeframe such as 12:00 – 05:00 on the 24th of last month.

 

In Objects you have the ability to select from Array, Storage & Hosts, Directors & Ports, Storage Resources, Data Protection and System internals where by you can focus on the specific category you need to target.

If you notice the radio button highlighted in red we have given you the ability to toggle between average and max values here. Also in the metric list section you can choose just to display KPI Metrics or all metrics.

 

Additionally for the icons below the default tab I wanted to outline and explain what their function is:

 

7.PNG.png

 

  • Save as Template – allows you to save it as a template if you need to focus on certain KPI Metrics for a particular object like metadata usage on an array.
  • Save as PDF – Charts – This allows you to save a chart and send it to a colleague in a visual format if you want to illustrate a certain pattern like capacity growth in a the last month.
  • Save as PDF – Charts & Data – like above this gives you back the visual charts and also the figures behind that.
  • Copy Charts to a Tab – allows you to copy charts over to a different tab while retaining the originals in the current tab
  • Move Charts to a Tab – allows you to move charts completely over to a different tab

 

8.PNG.png

 

In the previous version of the software the navigation was along the top of the screen but we have changed this to a left hand side navigation view so customers can more easily interact with the product. In this example I have created a chart called PerfIssue03/20 whereby I have selected the timeframe of the last 4 hours for the array 0647 and selected the KPI’s % reads and % Writes and drew my chart accordingly.

 

As a potential use case let’s say we were focusing on a potential issue from last night and we began by checking the last 4 hours and we found nothing of note so we decided to expand our scope.  To do this we copied out chart with the pencil icon and we changed the time frame to the last 8 hours and added 2 additional metrics called BE IO’s Per Sec and BE READ Req’s Per Sec and then draw a new chart.

 

charts2.PNG.png

 

 

Some of this functionality was in the previous product but we have reduced the input required from the end user and allowed them to merely copy a chart and edit it if required.

 

Additionally we have allowed you to add more tabs in the same way you do with a web browser so if you were comparing performance over a couple of days similar to what you see in the screenshot you can label charts by day and group accordingly.

 

For a video on this please see here: https://www.youtube.com/edit?o=U&video_id=9192_w1iwco


I hope you found this overview helpful and if you would like to see more focus on certain areas please let us know.

This series of blog posts are dedicated to the performance section for Unisphere for PowerMax 9.0. We have made significant improvements in this particular area in terms of usability and functionality and I wanted to take some time to explain the new functionality and user flows.

I wanted to begin with the Dashboard section where we have our “Pre-Canned” Dashboards which provides you with the ability to interrogate across a certain range of objects which will allow you to troubleshoot a potential performance issue on an array with specific KPI’s concentrating on the most common areas of focus. In total we have 12 Dashboards including Array, Storage Groups, Hosts, Thin Pools, Storage Containers, Port Groups, FE Directors, BE Directors, RDF Directors, Disk Groups, Disk Technology and External Directors. The great thing about dashboards is it takes the hard work out of troubleshooting for customers in that it presents you with a set of charts that will help you identify a potential problem if there is one.

 

I wanted to focus on the Array and Storage Group Dashboard here and talk a little more about them in greater detail.

 

1.PNG.png

 

For this dashboard the timeframe is set for the last 4 hours in the upper right hand corner but the time range can be expanded as you wish if you need to see earlier timeframe over the last hours, days or months going back over the last 12 months. You can also change the graph style as you wish. These are average values but there is a toggle button whereby you can select a Maximum value if desired.

Also in the upper right hand corner we have some useful functionality including the following:

  • Ability to save this dashboard as a custom dashboard or as a template.
  • Schedule array report or schedule array overview report
  • Run PDF report now, Run PDF report now with data and Run array overview PDF report now.
  • Navigate to the charts or analyse section

 

 

There are 3 separate views for this dashboard across the upper left hand side including Array Workload, Array Efficiency and Performance thresholds. These have been chose in order to give you a broad picture of how the array is performing at present in terms of its workload and performance.

In the previous screenshot you were looking at an array workload view looking at Host and BE Reqs/sec, Host MB’s/sec, % Cache WP and Host and BE IO’s/sec. In the array efficiency we focus on Subscribed and Usable capacity, Snapshot Capacity and % Metadata usage in order to give you a view into consumption patterns on the array.

 

2.PNG.png

 

3.PNG.png

In the next example I have selected the Storage Group Dashboard and selected a SG which has been seeing heavy Read and Write activity over the last 4 hours. This dashboard you can see 4 separate tiles our current View SG Workload, I/O Profile, Performance Thresholds and SG Noisy Neighbour. The SG Workload tile gives you an indication of read and write activity that is going to the SG across 6 different charts.

 

I wanted to spend some time on the new functionality we have introduced in 9.0 with the Noisy Neighbor Dashboard in order to represent potential FA contention.

 

4.PNG.png

 

In our conversations with customers one of the most common issues they encounter in their environments is the Noisy Neighbour scenario. For a given Storage Group the Dashboard will chart key performance metrics, and also detail the relationship between the storage group and the associated front end director and ports. The Dashboard will also show other SG’s that are sharing the same ports and could be contributing to performance issues.

There are 6 charts here with 3 graphs on the top which focus on Reads and Writes. Along the bottom we see the FE Directors and FE Ports that this SG is mapped down. Also we have a complete list of all SG’s that share these mappings so if there were a “rogue” SG which was consuming a lot of resources and causing a knock on performance problem you could easily see who the offender was on this list.

 

5.PNG.png

Finally I just wanted to show you templates and custom dashboards. The template I’m displaying here is called MetaData Usage and in my case it’s something I need to monitor closely so I have created a chart with all of our Meta Data metrics and saved this is as a template so I can easily view it any time.

 

Also in the custom dashboard I can save specific objects such as SG “Tier 1 Trading Application” as it is one of my mission critical SG’s that I need to keep a close eye on so by saving it as a custom dashboard I can access it easily.

 

This concludes this blog on the dashboard section, next I will cover the charting section where we have made huge improvements in the functionality and usability of the product.

 

Noisy Neighbor Video Example

 

https://www.youtube.com/watch?v=GHGcDHboR7c