Find Communities by: Category | Product

Liberty release is out https://www.openstack.org/software/liberty/ and as always there are  new capabilities and enhancements across all the projects along with stabilization and optimization of the code. In this blog we will walk through what  EMC has contributed to OpenStack both from product and core features as  part of Liberty release into Cinder and Manila. To refresh Cinder is the block storage service and Manila is the Shared File Service


Before we get into the EMC part here are the key features that came into Liberty for Cinder and Manila.

Cinder:

     https://wiki.openstack.org/wiki/ReleaseNotes/Liberty#OpenStack_Block_Storage_.28Cinder.29

 

Manila:

     https://wiki.openstack.org/wiki/ReleaseNotes/Liberty#OpenStack_Shared_File_System_.28Manila.29

 

On the EMC front in addition to contributing towards our product drivers support for OpenStack, EMC has been doing core contribution to various projects like Cinder, Manila, and Barbican and are core members on the projects.  We have been contributing  to OpenStack since the Grizzly release and our contributions from commits, bug resolution, blueprints, lines of code, documentation etc has been increasing every release and will continue to do so for future releases as well.  EMC is vested in OpenSource and contributing to the OpenStack community is high up on the list.

 

Shown below are some of  core contribution that EMC has  done in Liberty release related to Cinder  and Manila projects.

 

Cinder

• Non-disruptive backup :

     http://docs.openstack.org/admin-guide-cloud/blockstorage_volume_backups.html

• Clone Consistency Group API :

     http://docs.openstack.org/admin-guide-cloud/blockstorage-consistency-groups.html

 

Manila

• Oversubscription in thin provisioning :

     https://review.openstack.org/#/c/192044/

 


Next let’s take a look at the enhancements made to some of the EMC products  in Liberty release wrt  Cinder and Manila . One of the key addition is integration of ScaleIO cinder driver in Liberty upstream.

Screen Shot 2015-10-23 at 2.39.21 PM.png

If you would like more details on each of the EMC Product drivers take a look in the Liberty configuration documents here: Volume drivers - OpenStack Configuration Reference  - liberty

 

To summarize we looked at EMC contribution both from product drivers and core contribution in Liberty  release. EMC will continue to:

 

  • Demonstrate a strong commitment to the OpenStack community by contributing to OpenStack projects.
  • Integrating our best-in-class Enterprise class storage products with OpenStack
  • Work with Ecosystem partners to provide a total solution for Customers to deploy and operate an OpenStack Cloud.

 

If you happen to be attending the  OpenStack Summit in Tokyo , October 27-30th 2015, visit us at the Booth P1 to learn more about our products and solutions that EMC has around OpenStack  as well as new demonstrations.

mkhalid

ScaleIO and OpenStack

Posted by mkhalid Oct 23, 2015

Over the past few years EMC has been investing a lot of resources in supporting OpenStack across our product portfolio and integrating them into the upstream release of OpenStack.  In Liberty release  of OpenStack we had added support upstream for ScaleIO.

 

ScaleIO is an enterprise class software defined block storage solution, which enables easy, flexible deployment , and provides high performance and massive scalability in a Cloud Scale environment .  It includes

• Software that creates a server-based SAN

• ScaleOut Block storage for commodity hardware

• Hyper-converged…OR NOT , Hardware & hypervisor AGNOSTIC

 

You can get more information at : http://www.emc.com/storage/scaleio/index.htm

 

ScaleIO has been supporting OpenStack since Havana release . In release Kilo and prior ScaleIO Cinder drivers had to  be downloaded  via EMC’s Git . With ScaleIO now being integrated into the Liberty release of OpenStack  you can download Liberty release and the Cinder drivers are present . Shown below is a chart indicating  ScaleIO support of OpenStack and highlighted in yellow is where the driver is incorporated into upstream Liberty release.

 

scaleioblog1.png

 

The Cinder volume operations that can be performed on ScaleIO are as follows:

 

scaleio2.png

The configuration details for Scale IO from a Cinder perspective are now included within OpenStack documentation located here:

 

http://docs.openstack.org/liberty/config-reference/content/ScaleIO-cinder-driver.html

 

 

ScaleIO   is available for Customers as free and frictionless download for test and development . If you are looking to kick the tires with ScaleIO in conjunction with OpenStack please download it and give it a whirl.  The software can be downloaded from the following link:

 

http://www.emc.com/products-solutions/trial-software-download/scaleio.htm?PID=STORE-DL-SCIO

 

 

In addition to this  we are working with partners to make it easier easier for customer to deploy ScaleIO in OpenStack. One such example is  where the EMC Code team working together with Mirantis engineering to provide a Fuel Plugin for ScaleIO.  Fuel is an open source deployment and management tool for OpenStack. Developed as an OpenStack community effort lead by Mirantis, it provides an intuitive, GUI-driven experience for automating the deployment and management of OpenStack, related community projects and plug-ins.   Here is a link to the code, release notes and how to deploy the Fuel plugins in your OpenStack environment.

 

Fuel plugin ScaleIO Cinder:

– Development repo (latest): https://github.com/emccode/fuel-plugin-scaleio-cinder-test

– OpenStack repo: https://github.com/openstack/fuel-plugin-scaleio-cinder

Fuel plugin ScaleIO:

– Development repo (latest): https://github.com/emccode/fuel-plugin-scaleio

– OpenStack repo: https://github.com/openstack/fuel-plugin-scaleio

 

By the way If you happen to be attending the  OpenStack Summit in Tokyo visit us at the Booth P1 to learn more about ScaleIO. Also be sure to check out the other products and solutions that EMC has around OpenStack  as well as new demonstrations.

EMC and the Federation have been actively contributing to the OpenStack community for quite some time through product integration into OpenStack  but also community participation on various other projects.

 

As OpenStack has continued to gain momentum in the market, we at EMC have increased our investment in OpenStack. Our focus has been to provide customer solutions that enable easier deployment and operation of an OpenStack Cloud. One of our investments is the Reference Architecture Program.

 

As part of the Reference Architecture Program EMC has been partnering with key OpenStack distribution partners (Canonical, Mirantis and Red Hat) to develop Reference Architectures that include their respective OpenStack distributions with EMC arrays.   The program enables customers who need help in deploying and integrating across the cloud supply chain to operate and manage an OpenStack cloud with EMC storage.

 

blogra.png


The three key OpenStack distribution vendors Canonical, Mirantis and RedHat bring a lot of value to customers through their tools and experience in OpenStack deployments. They have experience in guiding customer through the process of building an end to end value chain based on OpenStack. As part of the partner engagement our key focus is to provide the customer with a solution encompassing:

  • A reference architecture design guide to deploy Partners distribution of OpenStack with Certified EMC storage drivers.
  • Integration with partner tools which eases deployment and management of infrastructure services.
  • Joint support and services

 

The certified reference architecture for Mirantis  is available here:

https://www.emc.com/collateral/technical-documentation/h13933-emc-mirantis-openstack-ra.pdf.

Canonical reference architecture guide is available here:

https://www.emc.com/collateral/technical-documentation/h14174-emc-canonical-openstack-ra.pdf

and RedHat reference architecture guide is available here:

http://www.emc.com/collateral/technicaldocument/h14282-emc-red-hat-openstack-ra.pdf

The reference architecture guides describes the solution for managing storage life cycle (Cinder) using EMC storage technologies and our Partners OpenStack distribution based on Juno release. The document goes through the main features and functionality of the solution, the solution architecture and components including systems from other eco-system partners like Brocade, and the validated hardware and software environments.

 

Overall, we at EMC have invested in OpenStack and will continue our strong commitment to the OpenStack community by  contributing to a variety of OpenStack projects, integrating our best-in-class storage products with OpenStack and designing customer-centric solutions with the right eco-system partners to deploy and operate an OpenStack based Cloud in an efficient way.

The next release of OpenStack , codename Kilo is scheduled to come out shortly. Kilo has new capabilities and enhancement across all the projects along with stabilization and optimization of the code. In this blog lets walk through some of the new enhancement  that are coming up in BlockStorage (Cinder) and then move onto what EMC has been doing as part of the Kilo release.


A quick refresher on Cinder.  Cinder is the block storage service within OpenStack, which provides persistent storage to compute instances.  The key capabilities that Cinder offers are managing Volume , Snapshots and Backups.


In the Kilo release some of the improvements and enhanced capabilities for Cinder are as follows.


Key Improvement: Rolling Upgrades

One of the most important improvement that went into Kilo as part of Cinder improvement  (mentioned by Mike Perez PTL Cinder Project) during EMC sponsored CrowdChat (Talking a Ton about Kilo)   “Cinder's biggest improvement was to make rolling upgrades better for all projects in OpenStack. We worked on code that came from Nova, made it generic to allow services to be independent of schema upgrades from this point forward.  For details on the work check the following: https://review.openstack.org/#/c/131873/”.


https://blueprints.launchpad.net/cinder/+spec/cinder-objects



Incremental backup:

Backup is one of the capabilities provided by Cinder and pre-Kilo release every backup done is a full backup and so as the volume grows performing full backups on volume become resource intensive. This functionality enables incremental backup.  The customer now has the capability of taking  full backup and incremental backup. The enhancement also included  performing a backup from a snapshots. Snapshot are taken of the volume and backup of the snapshot is done, including taking incremental backups of snapshots. Check the blueprints for more details.


https://blueprints.launchpad.net/cinder/+spec/incremental-backup


Backup support for encrypted volumes: 

As we are in the subject of backup, until Juno release we had the capability to encrypt a volume but what was missing was the capability of backing up an encrypted volume. That functionality has been enabled in the Juno release.  The enhancement also includes the capability to restore the encrypted backup. Further details on the “backup support for encrypted volumes” and caveats would recommend you to look at the following:


https://blueprints.launchpad.net/cinder/+spec/backup-support-for-encrypted-volumes

 

 

Consistency Group:

Consistency group was introduced in Juno. To give a brief synopsis of what Consistency Group is today in Cinder (Block Storage Service), every operation happens at the volume level.  Consistency groups are needed to group volumes together for the purpose of data protection (snapshots, backups) and remote replication for disaster recovery. This functionality is geared towards data protection. http://docs.openstack.org/admin-guide-cloud/content/consistency-groups.html


The works is being done in phases, in the first phase the focus is on Snapshots.  In Kilo there are more enhancements going in to support  Snapshots in consistency group:

  • Allowing a user to create ConsistencyGroup from ConsistencyGroup snapshots
  • Add existing volumes to Consistency Group and removing volume from the group.

 

https://blueprints.launchpad.net/cinder/+spec/consistency-groups-kilo-update


Oversubscription Support for Thin provisioning:

Today (pre-Kilo)  in Juno when the backend storage supports thin provisioning, you can define the volume type as Thin provision but the scheduler does not take advantage of the capability of  over subscribing the storage backend which is “thin provisioned”. Over subscription allows flexibility in storage allocation and usage and is an important concept of thin provisioning capability.  This new functionality provides the cinder-scheduler the capability to oversubscribe the backend storage.  New configuration data (provisioned_capacity, max_over subscription ratio, max_used_ratio,) has been added for better decision-making. For details as how all of this comes together with detailed description look at the following:


https://github.com/openstack/cinder-specs/blob/master/specs/kilo/over-subscription-in-thin-provisioning.rst

 

Over subscription in thin provisioning : Blueprints : Cinder


Private Volumes: 

This is an interesting capability , when you create a volume type today in OpenStack it is visible to all tenants (projects) and there is no way you can restrict specific volume-type to specific tenants (projects).  The use cases for this functionality is when you are trying some new storage technology and you want specific user base to have access to it and second use case could be in case of applications that have need of high performance volume compared to others.  Take a look "

 

https://blueprints.launchpad.net/cinder/+spec/private-volume-types


There are a few other capabilities that have gone into Kilo release would recommend folks to look at the Kilo blueprints and find out about the implementation details.


We looked generically at a few Cinder capabilities coming in Kilo .  EMC has contributed quite a bit of work on few of the capabilities namely  Consistency group enhancement and the Oversubscription Support for Thin provisioning, as the next step lets look at what EMC has been doing around products in Cinder for Kilo:


PRODUCTS

CAPABILITIES

VNX

VMAX

XTREMIO


The focus thru the blog has been on Cinder but EMC has also been contributing to other Storage related OpenStack projects like Manila (File Shared Services ) which is an official OpenStack project.  As part of Manila in Kilo we have added Isilon support to Manila with the following capabilities.

  • Create/delete NFS/CIFS share
  • Create/delete snapshot.


To summarize we looked at some of the important capabilities that are coming as part of the Cinder project in the Kilo release and we looked at how EMC contributes to the OpenStack Community by enhancing and adding new capabilities to our products. Overall, we at EMC have serious investment in OpenStack and will continue to:

  • Demonstrate a strong commitment to the OpenStack community by contributing to a variety of OpenStack projects.
  • Integrating our best-in-class storage products with OpenStack,
  • Design customer centric solutions with Federation and the right eco-system partners to deploy and operate an OpenStack based Cloud in an efficient way.

 

Make sure to come by and check us out at the  OpenStack Summit in Vancouver on May 18th to 21st  2015 to have more detailed conversation around our contributions and solutions.

Filter Blog

By date:
By tag: