Find Communities by: Category | Product

Liberty release is out https://www.openstack.org/software/liberty/ and as always there are  new capabilities and enhancements across all the projects along with stabilization and optimization of the code. In this blog we will walk through what  EMC has contributed to OpenStack both from product and core features as  part of Liberty release into Cinder and Manila. To refresh Cinder is the block storage service and Manila is the Shared File Service


Before we get into the EMC part here are the key features that came into Liberty for Cinder and Manila.

Cinder:

     https://wiki.openstack.org/wiki/ReleaseNotes/Liberty#OpenStack_Block_Storage_.28Cinder.29

 

Manila:

     https://wiki.openstack.org/wiki/ReleaseNotes/Liberty#OpenStack_Shared_File_System_.28Manila.29

 

On the EMC front in addition to contributing towards our product drivers support for OpenStack, EMC has been doing core contribution to various projects like Cinder, Manila, and Barbican and are core members on the projects.  We have been contributing  to OpenStack since the Grizzly release and our contributions from commits, bug resolution, blueprints, lines of code, documentation etc has been increasing every release and will continue to do so for future releases as well.  EMC is vested in OpenSource and contributing to the OpenStack community is high up on the list.

 

Shown below are some of  core contribution that EMC has  done in Liberty release related to Cinder  and Manila projects.

 

Cinder

• Non-disruptive backup :

     http://docs.openstack.org/admin-guide-cloud/blockstorage_volume_backups.html

• Clone Consistency Group API :

     http://docs.openstack.org/admin-guide-cloud/blockstorage-consistency-groups.html

 

Manila

• Oversubscription in thin provisioning :

     https://review.openstack.org/#/c/192044/

 


Next let’s take a look at the enhancements made to some of the EMC products  in Liberty release wrt  Cinder and Manila . One of the key addition is integration of ScaleIO cinder driver in Liberty upstream.

Screen Shot 2015-10-23 at 2.39.21 PM.png

If you would like more details on each of the EMC Product drivers take a look in the Liberty configuration documents here: Volume drivers - OpenStack Configuration Reference  - liberty

 

To summarize we looked at EMC contribution both from product drivers and core contribution in Liberty  release. EMC will continue to:

 

  • Demonstrate a strong commitment to the OpenStack community by contributing to OpenStack projects.
  • Integrating our best-in-class Enterprise class storage products with OpenStack
  • Work with Ecosystem partners to provide a total solution for Customers to deploy and operate an OpenStack Cloud.

 

If you happen to be attending the  OpenStack Summit in Tokyo , October 27-30th 2015, visit us at the Booth P1 to learn more about our products and solutions that EMC has around OpenStack  as well as new demonstrations.

mkhalid

ScaleIO and OpenStack

Posted by mkhalid Oct 23, 2015

Over the past few years EMC has been investing a lot of resources in supporting OpenStack across our product portfolio and integrating them into the upstream release of OpenStack.  In Liberty release  of OpenStack we had added support upstream for ScaleIO.

 

ScaleIO is an enterprise class software defined block storage solution, which enables easy, flexible deployment , and provides high performance and massive scalability in a Cloud Scale environment .  It includes

• Software that creates a server-based SAN

• ScaleOut Block storage for commodity hardware

• Hyper-converged…OR NOT , Hardware & hypervisor AGNOSTIC

 

You can get more information at : http://www.emc.com/storage/scaleio/index.htm

 

ScaleIO has been supporting OpenStack since Havana release . In release Kilo and prior ScaleIO Cinder drivers had to  be downloaded  via EMC’s Git . With ScaleIO now being integrated into the Liberty release of OpenStack  you can download Liberty release and the Cinder drivers are present . Shown below is a chart indicating  ScaleIO support of OpenStack and highlighted in yellow is where the driver is incorporated into upstream Liberty release.

 

scaleioblog1.png

 

The Cinder volume operations that can be performed on ScaleIO are as follows:

 

scaleio2.png

The configuration details for Scale IO from a Cinder perspective are now included within OpenStack documentation located here:

 

http://docs.openstack.org/liberty/config-reference/content/ScaleIO-cinder-driver.html

 

 

ScaleIO   is available for Customers as free and frictionless download for test and development . If you are looking to kick the tires with ScaleIO in conjunction with OpenStack please download it and give it a whirl.  The software can be downloaded from the following link:

 

http://www.emc.com/products-solutions/trial-software-download/scaleio.htm?PID=STORE-DL-SCIO

 

 

In addition to this  we are working with partners to make it easier easier for customer to deploy ScaleIO in OpenStack. One such example is  where the EMC Code team working together with Mirantis engineering to provide a Fuel Plugin for ScaleIO.  Fuel is an open source deployment and management tool for OpenStack. Developed as an OpenStack community effort lead by Mirantis, it provides an intuitive, GUI-driven experience for automating the deployment and management of OpenStack, related community projects and plug-ins.   Here is a link to the code, release notes and how to deploy the Fuel plugins in your OpenStack environment.

 

Fuel plugin ScaleIO Cinder:

– Development repo (latest): https://github.com/emccode/fuel-plugin-scaleio-cinder-test

– OpenStack repo: https://github.com/openstack/fuel-plugin-scaleio-cinder

Fuel plugin ScaleIO:

– Development repo (latest): https://github.com/emccode/fuel-plugin-scaleio

– OpenStack repo: https://github.com/openstack/fuel-plugin-scaleio

 

By the way If you happen to be attending the  OpenStack Summit in Tokyo visit us at the Booth P1 to learn more about ScaleIO. Also be sure to check out the other products and solutions that EMC has around OpenStack  as well as new demonstrations.

Much of the success and popularity of open source technologies is due to new, large web-scale companies, like Facebook and Amazon, building better, more innovative data center designs with open source technologies. These new designs, while having some similar patterns to the traditional, offer the business the agility, scale, and flexibility it needs to make the digital transformation all without being locked-in to their technology decisions with specific vendors.

100575589-300dpi.jpgTraditional companies see the success and value of these new designs and look to provide the same benefits to their business by updating their environment. Randy Bias explains this view of cloud computing and what EMC’s role is in the open source ecosystem. EMC has and will continue to adopt open source with these changing times. The new techniques on display from these newer, web-scale companies built on open source technologies are being brought to the traditional enterprises of the world and EMC, as a leading technology provider, is helping their customers make this transition.

EMC already has a history of contributing to the OpenStack community and offering the necessary drivers to make sure their products can connect and work in an OpenStack environment. However, EMC is expanding their contribution and increasing their commitment. The announcement of CoprHD, EMC open sourcing of their software-defined storage product ViPR, is a big step in taking down the walls and changing the perception of the old, proprietary-only, closed EMC. EMC also offers a free, non-production version of their ScaleIO software for customers to experience.

2 hands clouds.jpg     In addition, EMC is partnering with the right players and distributors of OpenStack to help our customers build their open sourced clouds with EMC storage through reference architectures with Mirantis, Canonical, and RedHat. Overall, EMC understands it is no longer about a single platform, but rather the integration into the entire open source ecosystem. Listen to Dorian Naveh explain EMC’s strategic history with OpenStack.


Other companies are following suit with their own open source strategy. The big, traditional technology providers like HP and IBM are also making significant investments in the open source ecosystem. The trend continues to gain momentum, the writing is on the wall, OpenStack is maturing, and none of the big players want to miss out.

On Thursday June 4th, we reviewed the key themes from OpenStack Summit Vancouver. The conversation was led by EMC's own Mohamed Khalid with participation from our partners at theCube, Sean Winn, Egle Sigler from OpenStack Board of Directors, and Jesse Proudman from BlueBox.

 

The CrowdChat started with everyone sharing their big observations from the show, whether it be containerization or their trip to Grouse Mountain to visit the grizzly bear. People shared which sessions were 'absolutely can't miss' and included theCube interviews that were happening throughout the week from all types of companies and vendors.

 

One big topic of discussion was around interoperability and how much the DefCore work is helping this cause. Most people agreed the work is a good start but there is still plenty to be accomplished. My personal favorite topic was the Community App Catalog, whose efforts were demonstrated at the show. Some folks agreed with my sentiment that the development of an Enterprise App Store would be a huge advantage for OpenStack over someone like an AWS.

 

But these are just the highlights! We had an hour long twitter conversation about all of these topics and more. So if you want to catch up on what you missed just click on this link #openEMC - CrowdChat. What was your favorite OpenStack Summit topic?!

 

Don't forget to keep an eye for our next #openEMC conversation on CrowdChat!

Last November, EMC announced its increased investment in OpenStack and launched the EMC OpenStack Reference Architecture Program with inaugural partners Canonical, Mirantis, and Red Hat.  Founded on EMC’s core tenet of providing choice to its customers, EMC’s OpenStack Reference Architecture Program would enable customers to build OpenStack-based Hybrid Clouds in addition to EMC’s Federation Enterprise Hybrid Cloud based on VMware vCloud components.

 

EMC is now delivering against that promise by releasing new reference architectures for OpenStack clouds with Canonical, Mirantis, and Red Hat.  These reference architectures will facilitate the deployment and operations of OpenStack Clouds utilizing EMC technology by providing customers with solutions that have been tested, qualified, validated, and certified by the partners and EMC.  The reference architectures are complemented with best practices and deployment tools from our partners built upon experience with large scale cloud deployments.

 

Through close collaboration with each Reference Architecture Program partner and our longstanding strategic partner Brocade, EMC built fully functional lab environments and implemented standard “out-of-the-box” OpenStack distributions from our partners with EMC VNX and XtremIO arrays, EMC’s ScaleIO Software Defined Storage, and Brocade’s 6510 switches.  The EMC drivers used in the Reference Architecture Program are available directly with your preferred partner distribution.

 

Each configuration was then optimized to demonstrate how to make deployment a repeatable and reliable process, showcase the functionality of the solution, and simplify the implementation process itself.  Once completed, each reference architecture was documented and certified by the partners to provide customers the peace of mind that they've come to expect from EMC.  See the supporting coverage from Canonical (blog) and Mirantis (blog & press release) that support the completion and availability of these new solutions or reach out and talk to us directly on twitter by using the #OpenEMC hashtag.

 

EMC’s Open Source efforts are on fire and include dramatic increases in contributions as well as exciting new offerings with tangible customer benefits.  Stay tuned for more news from EMC this May at OpenStack Summit in Vancouver and follow #EMC and #OpenStackSummit on Twitter for real-time updates.

EMC and the Federation have been actively contributing to the OpenStack community for quite some time through product integration into OpenStack  but also community participation on various other projects.

 

As OpenStack has continued to gain momentum in the market, we at EMC have increased our investment in OpenStack. Our focus has been to provide customer solutions that enable easier deployment and operation of an OpenStack Cloud. One of our investments is the Reference Architecture Program.

 

As part of the Reference Architecture Program EMC has been partnering with key OpenStack distribution partners (Canonical, Mirantis and Red Hat) to develop Reference Architectures that include their respective OpenStack distributions with EMC arrays.   The program enables customers who need help in deploying and integrating across the cloud supply chain to operate and manage an OpenStack cloud with EMC storage.

 

blogra.png


The three key OpenStack distribution vendors Canonical, Mirantis and RedHat bring a lot of value to customers through their tools and experience in OpenStack deployments. They have experience in guiding customer through the process of building an end to end value chain based on OpenStack. As part of the partner engagement our key focus is to provide the customer with a solution encompassing:

  • A reference architecture design guide to deploy Partners distribution of OpenStack with Certified EMC storage drivers.
  • Integration with partner tools which eases deployment and management of infrastructure services.
  • Joint support and services

 

The certified reference architecture for Mirantis  is available here:

https://www.emc.com/collateral/technical-documentation/h13933-emc-mirantis-openstack-ra.pdf.

Canonical reference architecture guide is available here:

https://www.emc.com/collateral/technical-documentation/h14174-emc-canonical-openstack-ra.pdf

and RedHat reference architecture guide is available here:

http://www.emc.com/collateral/technicaldocument/h14282-emc-red-hat-openstack-ra.pdf

The reference architecture guides describes the solution for managing storage life cycle (Cinder) using EMC storage technologies and our Partners OpenStack distribution based on Juno release. The document goes through the main features and functionality of the solution, the solution architecture and components including systems from other eco-system partners like Brocade, and the validated hardware and software environments.

 

Overall, we at EMC have invested in OpenStack and will continue our strong commitment to the OpenStack community by  contributing to a variety of OpenStack projects, integrating our best-in-class storage products with OpenStack and designing customer-centric solutions with the right eco-system partners to deploy and operate an OpenStack based Cloud in an efficient way.

The next release of OpenStack , codename Kilo is scheduled to come out shortly. Kilo has new capabilities and enhancement across all the projects along with stabilization and optimization of the code. In this blog lets walk through some of the new enhancement  that are coming up in BlockStorage (Cinder) and then move onto what EMC has been doing as part of the Kilo release.


A quick refresher on Cinder.  Cinder is the block storage service within OpenStack, which provides persistent storage to compute instances.  The key capabilities that Cinder offers are managing Volume , Snapshots and Backups.


In the Kilo release some of the improvements and enhanced capabilities for Cinder are as follows.


Key Improvement: Rolling Upgrades

One of the most important improvement that went into Kilo as part of Cinder improvement  (mentioned by Mike Perez PTL Cinder Project) during EMC sponsored CrowdChat (Talking a Ton about Kilo)   “Cinder's biggest improvement was to make rolling upgrades better for all projects in OpenStack. We worked on code that came from Nova, made it generic to allow services to be independent of schema upgrades from this point forward.  For details on the work check the following: https://review.openstack.org/#/c/131873/”.


https://blueprints.launchpad.net/cinder/+spec/cinder-objects



Incremental backup:

Backup is one of the capabilities provided by Cinder and pre-Kilo release every backup done is a full backup and so as the volume grows performing full backups on volume become resource intensive. This functionality enables incremental backup.  The customer now has the capability of taking  full backup and incremental backup. The enhancement also included  performing a backup from a snapshots. Snapshot are taken of the volume and backup of the snapshot is done, including taking incremental backups of snapshots. Check the blueprints for more details.


https://blueprints.launchpad.net/cinder/+spec/incremental-backup


Backup support for encrypted volumes: 

As we are in the subject of backup, until Juno release we had the capability to encrypt a volume but what was missing was the capability of backing up an encrypted volume. That functionality has been enabled in the Juno release.  The enhancement also includes the capability to restore the encrypted backup. Further details on the “backup support for encrypted volumes” and caveats would recommend you to look at the following:


https://blueprints.launchpad.net/cinder/+spec/backup-support-for-encrypted-volumes

 

 

Consistency Group:

Consistency group was introduced in Juno. To give a brief synopsis of what Consistency Group is today in Cinder (Block Storage Service), every operation happens at the volume level.  Consistency groups are needed to group volumes together for the purpose of data protection (snapshots, backups) and remote replication for disaster recovery. This functionality is geared towards data protection. http://docs.openstack.org/admin-guide-cloud/content/consistency-groups.html


The works is being done in phases, in the first phase the focus is on Snapshots.  In Kilo there are more enhancements going in to support  Snapshots in consistency group:

  • Allowing a user to create ConsistencyGroup from ConsistencyGroup snapshots
  • Add existing volumes to Consistency Group and removing volume from the group.

 

https://blueprints.launchpad.net/cinder/+spec/consistency-groups-kilo-update


Oversubscription Support for Thin provisioning:

Today (pre-Kilo)  in Juno when the backend storage supports thin provisioning, you can define the volume type as Thin provision but the scheduler does not take advantage of the capability of  over subscribing the storage backend which is “thin provisioned”. Over subscription allows flexibility in storage allocation and usage and is an important concept of thin provisioning capability.  This new functionality provides the cinder-scheduler the capability to oversubscribe the backend storage.  New configuration data (provisioned_capacity, max_over subscription ratio, max_used_ratio,) has been added for better decision-making. For details as how all of this comes together with detailed description look at the following:


https://github.com/openstack/cinder-specs/blob/master/specs/kilo/over-subscription-in-thin-provisioning.rst

 

Over subscription in thin provisioning : Blueprints : Cinder


Private Volumes: 

This is an interesting capability , when you create a volume type today in OpenStack it is visible to all tenants (projects) and there is no way you can restrict specific volume-type to specific tenants (projects).  The use cases for this functionality is when you are trying some new storage technology and you want specific user base to have access to it and second use case could be in case of applications that have need of high performance volume compared to others.  Take a look "

 

https://blueprints.launchpad.net/cinder/+spec/private-volume-types


There are a few other capabilities that have gone into Kilo release would recommend folks to look at the Kilo blueprints and find out about the implementation details.


We looked generically at a few Cinder capabilities coming in Kilo .  EMC has contributed quite a bit of work on few of the capabilities namely  Consistency group enhancement and the Oversubscription Support for Thin provisioning, as the next step lets look at what EMC has been doing around products in Cinder for Kilo:


PRODUCTS

CAPABILITIES

VNX

VMAX

XTREMIO


The focus thru the blog has been on Cinder but EMC has also been contributing to other Storage related OpenStack projects like Manila (File Shared Services ) which is an official OpenStack project.  As part of Manila in Kilo we have added Isilon support to Manila with the following capabilities.

  • Create/delete NFS/CIFS share
  • Create/delete snapshot.


To summarize we looked at some of the important capabilities that are coming as part of the Cinder project in the Kilo release and we looked at how EMC contributes to the OpenStack Community by enhancing and adding new capabilities to our products. Overall, we at EMC have serious investment in OpenStack and will continue to:

  • Demonstrate a strong commitment to the OpenStack community by contributing to a variety of OpenStack projects.
  • Integrating our best-in-class storage products with OpenStack,
  • Design customer centric solutions with Federation and the right eco-system partners to deploy and operate an OpenStack based Cloud in an efficient way.

 

Make sure to come by and check us out at the  OpenStack Summit in Vancouver on May 18th to 21st  2015 to have more detailed conversation around our contributions and solutions.

If you missed our lively CrowdChat today on OpenStack Operations, you can read up on it here.

 

Check back here for future CrowdChats on other OpenStack topics!

The OpenStack news in the EMC world keeps coming fast and furious. A quick Recap:

 

Today we're announcing the creation of the EMC OpenStack Reference Architecture Program, along with the founding members:

  • Mirantis
  • Canonical
  • Red Hat

 

Check out Kenneth Hui's blog for more information here.

Tyler_Britten

Welcome Cloudscaling!

Posted by Tyler_Britten Oct 29, 2014

If you missed the announcement, EMC has acquired leading OpenStack provider, Cloudscaling.

 

Here's some great thoughts on the new additions to the OpenStack team at EMC:

 

Chad Sakac (@sakacc) : http://t.co/wmOtLQGvyS

Randy Bias (@randybias): http://t.co/FNc5lxfE6b

Ken Hui (@hui_kenneth): http://t.co/VZXDncEBBs

Tyler_Britten

EMC and OpenStack

Posted by Tyler_Britten Jan 27, 2014

Updated 9/26/2014 with latest driver information.

 

When customers build their clouds, EMC believes in choice at each layer of the stack. To that end, EMC joined the OpenStack Foundation in 2012 as a corporate sponsor. Since then we’ve be contributing drivers as of the Grizzly release, and have made them available in a number of fashions including Github.

 

Going forward EMC plans to continue to contribute. As an example, we’re working with NetApp and Mirantis on a new OpenStack project specifically for file storage called Manila. While there has been some basic NFS support in OpenStack since Grizzly, Manila will expand that support and also add CIFS; in short Manila will do for file what Cinder has done for block.

 

Running OpenStack on EMC

 

So what drivers and integrations are available for EMC customers running OpenStack?

 

          

Array
Supported Release
Protocol
Volume Functions
Location
VMAXGrizzly, Havana, IceHouseiSCSI, FC

All required functions, except ‘Create Vol

from snapshot’, ‘Extend vol’

Distribution
VNXGrizzly, Havana, IceHouseiSCSI, FCAll required functionsDistribution
XtremIOHavana, IceHouseiSCSIAll required functionsGitHub
ScaleIOHavana, IceHouseSDC/SDSAll required functionsEMC
IsilonHavana, IceHouseNFS (Cinder)All required functionsEMC
ViPR ControllerHavana, IceHouseiSCSI, FCAll required functionsGitHub

Note: GithubDrivers available at: github.com/emc-openstack

 

As you see, we have a combination iSCSI and FibreChannel Cinder driver available for VMAX platforms that leverage an SMI provider that EMC customers are familiar with called Solutions Enabler. It’s available for download on support.emc.com.

 

If you’re a VNX customer, you can deploy our iSCSI & FibreChannel driver. It leverages a command line tool called naviseccli, which is available on GitHub as well as on support.emc.com.

The iSCSI driver for our XtremIO all-flash array is available on GitHub site, and our Isilon NFS  and ScaleIO drivers are available by contacting EMC.

 

The ViPR driver requires the deployment of our software defined storage platform, ViPR. The advantage of using ViPR is it can abstract and pool both EMC and non-EMC storage and allow you to provision with a single ViPR cinder driver. ViPR is available as a free, non-commercial download here and the cinder driver is on github.

The full commercial version of ViPR also includes EMC support and Data Services capabilities. One of the available data services is the Object Service that gives you the ability to provide Swift, S3, or Atmos object storage on existing file platforms.

 

 

Filter Blog

By date:
By tag: