One of the biggest challenges that storage administrators face is balancing the storage space that is required by various applications in their data centers. Administrators are typically forced to allocate space based on anticipated storage growth. Such growth frequently is poorly understood and significantly overstated. Storage administrators do this to reduce the management expense and application downtime required to add storage later. Over-provisioning leads to increased costs, power, cooling and floor space requirements. It also leads to lower capacity utilization rates. The EMC Virtual Provisioning feature was introduced to address concerns such as this.
Virtual provisioning extends the concept of provisioning to mean that the device or file system created represents a particular size, but does not occupy that space on disk. While it may consume that space over time, it initially uses a fraction of the size representation. Through virtual provisioning the administrator chooses an initial size, a maximum size and a high water mark. The user or application sees the maximum size as if it was actually allocated to them. However, only the initial size is allocated on the storage system. The space that is actually allocated on the storage system automatically increases on an as-needed basis. This as-needed basis is determined by the high water mark (for example, by default if 90% of the provisioned storage on a virtually provisioned file system is being used, it will automatically extend). The storage that is allocated can continue to extend until the maximum size is reached.
The feature also provides alert notifications that can be used to track the physical allocation of the file system or its corresponding storage pool. If either the file system or the storage pool is nearing their maximum size, an alert is triggered so that the administrator can take action. This action can include deleting unused files, turning on deduplication or tiering, adding more disks, extending the maximum size of the file system, and so on.
A typical use case for Virtual Provisioning is the use of Virtual Machines (VMs) in virtually provisioned environments. VM images reside on NFS or iSCSI storage. As more VMs are created on the storage, the file systems are allowed to extend to the space needed to contain the VMs. Initially, the file systems can be small in size and then gradually grow as the need for more VMs arise. VMs are a good example of data that start small and grow corresponding to the growth of your organization.
Another use case for Virtual Provisioning is a CIFS share that many users from a group use to share various documents, videos, etc… It has a slow to moderate growth rate but the group requested a larger size than was immediately required (in this case 500GB) because of the perceived difficulty in filing another request later to extend it. With Virtual Provisioning the group is given a share that they see as 500GB but only occupies 100GB of physical storage.
When rolling out Virtual Provisioning, remember that:
- Virtual Provisioning can be activated on new file systems or existing file systems.
- For existing file systems, the initial size for Virtual Provisioning is equivalent to the current logical size of the existing file system. The administrator only needs to specify the high water mark and maximum size of the file system.
The following considerations must be understood before using the Virtual Provisioning feature:
- Investigate the usage pattern of the environment before deploying the feature. This is important because in special circumstances it can impact data accessibility and performance if the initial size is set too small.
- Be aware where virtual provisioning provides the most value:
- Environments where storage demands are high, but actual space usage is low
- Environments that use Home directories because users tend to demand more storage than they really need
- Test and development areas because users tend to demand more storage than they really need
- Document repositories because of the low rate of change
- Take precautions to prevent running low on physical storage space.
- Set the appropriate high watermarks
- Set storage usage notifications that alert administrators in time
- Set storage projection notifications with enough buffer for administrators to address impending storage full situations
- Integrate storage usage and trend reports into regular NAS reports
- Export the graph data to build report/charts
EMC recommends the following best practices for administrators implementing Virtual Provisioning:
- Storage usage notifications should be configured on a file system or storage pool to send an email or an SNMP trap and notify the administrator when a certain percentage of the current size or maximum size is used.
- Administrators should understand the growth rate of their pools and file systems to know what percentage is practical and allows enough time for them to react and address potential over-subscription issues.
- Storage projection notifications should be configured on a file system or storage pool to send an email or an SNMP trap and notify the administrator a certain amount of days before the current size or maximum size is projected to be full.
- Storage usage and storage projection notifications should be configured.
- Too many auto extensions could impact performance. It is not a good practice to start with a very small file system, for example 5 GB, which can extend to a large file system, for example 500 GB. This causes the file system to be fragmented.
- EMC CLARiiON Virtual Provisioning NEW!
- EMC CLARiiON LUN Shrinking with Microsoft Exchange 2007
- Thin Provisioning for Microsoft Exchange 2007 with EMC CLARiiON Virtual Provisioning and VMware vSphere 4
- Implementing Virtual Provisioning on EMC CLARiiON and Celerra with VMware Infrastructure
- Leveraging EMC CLARiiON CX4 Virtual Provisioning for Database Deployments
- Maximize the Performance Benefit from Enterprise Flash Drives by Sharing Through Virtual Provisioning in EMC CLARiiON CX4
- Virtual Provisioning Options for EMC Unified Storage Platforms