DELL EMC Cloud Disaster Recovery
In the data protection world, disaster recovery is one of the most important factor organizations plan. RPO and RTO are most important parameters of disaster recovery and can vary among environments. RPO and RTO are two key values that organizations must consider in order to develop an appropriate disaster recovery plan that can maintain business continuity after an unexpected event.
A single outage can make business owners aware of the importance of a DR plan. An effective DR plan or strategy is much more than simply anticipating a data center outage. It is more about ensuring minimal business impact.
RPO (Recovery Point Objective): By definition, the recovery point objective (RPO) is the maximum acceptable amount of data loss measured in time. It is the age of data in backup storage required to resume normal operations if a computer system or network failure occurs.
RTO (Recovery Time Objective): By definition, the recovery time objective (RTO) is the targeted duration of time and a service level within which a business process must be restored after a disaster (or disruption) in order to avoid unacceptable consequences associated with a break in business continuity.
RPO is usually pretty easy to measure like last successful backup of catalogs, or control data, metadata.
As an example if you have a backup solution and you backup the master server catalogs once a day then your RPO is 24 hours and a backup administrator understands, that if the backup system encounters a DR situation then maximum time we can go back is that last catalog back, whatever data was backed up before the last catalog backup is restorable, and any data backed up after catalog backup to the DR event is lost.
However defining RTO is not easy and it depends on lots of factors and even after defining it, and thoroughly exploring it, things can change.
As the cloud is currently the trend and being adopted by lots of organizations as a destination for long term data, or hosting their data centers into the cloud, or even as a DR option, DELL EMC Cloud DR solution works with DELL EMC data protection software or hardware and tremendously minimizes recovery time objective (RTO). It allows customers to make the cloud as a secondary or only DR option.
What is Data Domain Cloud Disaster Recovery solution?
Cloud Disaster Recovery allows organizations to efficiently extend their on-premises data protection to the public cloud or in other words,
Cloud Disaster Recovery is a solution that allows customers to save their VMware virtual machine backups to the cloud.
So that when a customer’s on-premises environment encounters disaster and could not be recovered in place, they still
have the option to recover their business in the Cloud.
Cloud DR provides data protection of on-premises virtual machines to the cloud provides familiar user experience, thus requiring minimal education and training, resulting in lower TCO and higher ROI. Using Cloud DR, you can leverage the agility and scale of public cloud for DR, with minimal cloud footprint (no additional compute is required during routine replication, and minimal compute is required in case of test or recovery). Cloud DR provides a unique capability of orchestrating not only the failover but also the failback of recovered workloads from the public cloud back to a new VM copy on-premises. Cloud DR enables orchestrated DR of multiple VMs using preconfigured DR plans and short RTO for critical VMs by enabling Rapid Recovery function to those VMs.
Example: If a customer lost a data center or a DR situation has arisen, they can start their business applications into the cloud as VM instances and run their business instantly without waiting for on-premises disaster recovery to happen first. This minimizes RTO heavily.
Note: The Cloud DR is not available in Greater China, Russia, Belarus, and Kazakhstan.
Note: The DELL EMC Powerprotect would be soon supported with Cloud DR solution.
The architecture of Cloud DR is divided into 2 modes
Standard Mode (Default)
Advanced Mode (only for AWS S3)
You can switch from Standard mode to Advanced mode, however, you cannot revert back.
Note: Starting Cloud DR 19.1, you would notice a change in CDRA deployment. The Advanced mode is not an option upfront, Standard mode is the default option and one wants to switch to Advanced mode, they can do it post-deployment under the settings tab in CDRA UI.
Base Warranty: 90-Day software warranty.
The Cloud DR is not available in Greater China, Russia, Belarus, and Kazakhstan.
Depending on the on-premises data protection solution, either the Cloud DR Add-on
(CDRA) or the virtual RecoverPoint Appliance (vRPA) manages the deployment of on-premises resources and the CDRS, which runs in the cloud. The CDRA and vRPA are referred to as on-premises sources.
Note: Multiple on-premises sources (CDRAs and vRPAs) can connect to a single CDRS, but an on-premises source cannot connect to multiple CDRS's.
Cloud DR was also known as Data Domain Cloud DR or DDCDR, however, there seem to be some naming changes recently.
DELL EMC announced the version 19.1 in the month of May 2019 and the new features of this release are as follows
• Support for AWS GovCloud: Cloud DR Standard mode supports AWS GovCloud.
• Support recovery of virtual machines with UEFI settings: Cloud DR supports direct recovery to vCenter/VMC with UEFI mode in BIOS.
• Support for VMware HA: Cloud DR supports VMware High Availability feature.
• Dual NIC with IPv6 Support: Cloud DR supports dual NIC configuration with IPv6 settings.
• Log collection in CDRS: Logs can be collected in CDRS UI. The logs are stored in the connected cloud location.
The backup software's which Cloud DR supports is (Avamar+DD), this could be a complete virtual solution like AVE and DDVE or Physical+virtual or Physical only. IDPA is also supported, however, this is again Avamar+DD Combo. RP4VM can be also used with DD CDR.
For deployment, I have used Amazon S3 for Cloud DR testing using Standard mode.
What is Amazon S3?
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance.
Before we deploy, let us understand the pre-requisites and architecture for a successful deployment.
Amazon Import/Export requirements
Cloud DR documentation, downloads, knowledgebase, etc can be found here
Download the Cloud DR OVF file
Size: ~ 4 GB.
Deploy CDRA on vSphere by choosing Deploy OVF Template like any virtual appliance.
Choose Eager Lazy Zero provisioning.
Once deployed, open a browser and type
Note: CDRA is deployed with 2 NICS. One NIC is used for external communication (Cloud and Data) and the other for internal communication. Keep in mind there is no option to deploy CDRA with 1 NIC only.
The default credentials are
On the very first login, the system will prompt you to change password.
I choose the standard for this demo.
On next screen define hostname, timezone, NTP, DNS, etc and click on save button.
Next, add a cloud account.
Note: For AWS GovCloud, only the US-West region is supported now. It is possible to create an S3 bucket on the US-East region, however, that region is not supported on CDR GovCloud.
Note: Advanced mode is only supported for AWS and not AWS GovCloud or Azure.
Before you are entering Access Key and secret key please click on Show IAM policy option.
Copy the IAM Policy
Some of you must be wondering what is IAM policy, I did the same when I heard it for the first time.
Basically, IAM policies define permissions for any action regardless of the method that you use to perform the operation.
You manage access in AWS by creating policies and attaching them to IAM identities (users, groups of users, or roles) or AWS resources. A policy is an object in AWS that, when associated with an identity or resource, defines their permissions. AWS evaluates these policies when a principal entity (user or role) makes a request. Permissions in the policies determine whether the request is allowed or denied. Most policies are stored in AWS as JSON documents. AWS supports six types of policies: identity-based policies, resource-based policies, permissions boundaries, Organizations SCPs, ACLs, and session policies.
Login to AWS console as the root user.
The region depends on your environment
Services >Security, Identity, & Compliance > IAM >Policies >Create Policy
Delete the existing content in JSON Tab & paste the content which we copied earlier "COPY IAM POLICY"
Name the policy, and click on Create policy.
Add a User by click on Users Tab
Click on Next: Permissions tab and select the Attach existing policies directly option & select your policy “cdra-policy”.
In this screen, you have an optional feature of using Tags if you wish.
Review and click on the Save tab.
Download the Access & Secret keys and keep them safe.
Now go back to CDRA GUI and enter these keys (Access and Secret) and click on Verify and save.
Next login to your AWS account using new IAM user credentials, created above.
Now let us create Bucket.
What is the Bucket?
These are unique containers to store your data or logical destination of your data on the cloud.
Navigate to Services > Storage > S3
Add your settings, and then finally click on Create tab.
Once created go back to CDRA console and here add a Cloud Target.
CDRA Console > Configuration> Deploy> Add Cloud Target
The Advanced security Option displays the encryption options.
SSE stands for server-side encryption.
You can protect data at rest in Amazon S3 by using three different modes of server-side encryption:
SSE-S3, SSE-C, or SSE-KMS.
With Cloud DR, SSE-C is not an option.
SSE-S3 requires that Amazon S3 manage the data and master encryption keys.
Server-side encryption protects data at rest. Amazon S3 encrypts each object with a unique key. As an additional safeguard, it encrypts the key itself with a master key that it rotates regularly. Amazon S3 server-side encryption uses one of the strongest block ciphers available, 256-bit Advanced Encryption Standard (AES-256), to encrypt your data.
Server-side encryption encrypts only the object data, not object metadata.
SSE-KMS (Server-side encryption-Key Management service).
SSE-KMS is very similar to SSE-AES.
The only difference is that the secret key (aka AWS managed Customer Master Key (CMK)) is provided by the KMS service and not by S3.
For more details, kindly refer here
Note: If you select the SSE-KMS encryption method, only the default customer-managed key is supported.
Changing the encryption key might cause errors with the files in the Amazon S3 bucket.
Let us recap what all we have deployed and configured.
1. Deployed and configured CDRA virtual appliance.
2. CDRA policy and CDRA user.
3. Cloud Account, Bucket, and Cloud Target.
Next, we will setup CDRS (Cloud Disaster Recover Server) in AWS.
Open CDRA console,
Navigate to Configuration > Deploy > select Cloud DR Server.
➢ If no CDRS has been deployed, the Deploy Cloud DR Server page appears.
➢ If the CDRS has already been deployed, the Cloud DR Server page appears.
You are not permitted to deploy additional CDRS instances
CDRS deployment is always using a public IP. The option to select an existing VPC is designed for VPN connections, and then you need to provide a legal range of IPs for the subnet the deployment will create and run the CDRS in. If you chose to create a new VPC, that’s fine too, the only reason to use an existing VPC is for VPN.
In the IPV4 CIDR Range section, the CIDR prefix for the CDRS is prepopulated, and you may retain the given value or change it.
Note: The CIDR range defines the number of IP addresses within the VPC.
The range is allocated by CDRS to the CDRS subnet and two RDS's (the second RDS is a backup for high availability).
Each RDS is created in its own Availability Zone and private subnet. If you selected an existing VPC in the previous step, ensure that the IP addresses within the CIDR range are available for use. If you specify a range of addresses that is not available (meaning that these IP addresses may already be in use), then the deployment process will not start.
In the User Configuration section, enter and confirm passwords for the CDRS Admin and CDRS Monitor users.
The passwords must be at least eight characters in length Contain characters of a minimum of three of the following types:
• English uppercase: A-Z
• English lowercase: a-z
• Numeric character: 0–9
• Special (non-alphanumeric) characters.
• Enter and confirm passwords for the CDRS Admin and CDRS Monitor users.
• Enter an email address for Cloud DR password reset requests.
When the Cloud DR Server is successfully deployed, AWS sends an email to this address for verification. Follow the instructions in the email within 24 hours of deployment.
Note: If you update the password, the new password must be different than the previous password.
To confirm that you accept the marketplace terms, click the I have accepted the AWS Marketplace terms checkbox and
Click Deploy Cloud DR Server.
Note: You can’t change CIDR /26 as this is by default option.
The CDRA begins deployment of the CDRS to the Cloud DR target. Deploying the CDRS may take up to 30-40 minutes.
The M4 Large instance type is used for the CDRS instance. To reduce deployment costs, you may want to purchase reserved instances from AWS; otherwise, an OnDemand instance is used. An elastic IP address is automatically assigned to the CDRS instance. You cannot change this IP address. If the deployment is successful, the Cloud DR Server page appears, listing the hostname of the CDRS host and the region. You can access the Cloud DR Server by clicking the CDRS Hostname link, but protection and disaster recovery are not supported until you complete all CDRA configuration steps.
During the Deployment process, the deployment process creates the following
- CDRS Deployment Stack
- New VPC called “CDRS-Vpc” with IP4 CIDR 10.0.0.0/26
- Three Subnets (One Public + Two Private in different zones)
- One Elastic IP ❖ One Internet gateway attached to the public subnet
- Security Group with 443 ports allowed.
Note: Adding a VPN gateway is only supported for Advanced Mode.
Next is to add a vCenter with administrator user credentials.
Open CDRA Console
Configuration > Deployment > vCenter Servers
Add hostname, port 443, administrator credentials.
Click Save and Connect. This will display a certificate for you to confirm.
Next, define recovery staging area
Once done with all settings, click on the Save tab.
Finally, add an Avamar server and Data Domain through CDRA console, using Local Backup tab.
Now the deployment is completed!!
Let us now have a look at protecting VM's in the cloud.
Login to Avamar administrative console with MCUser credentials.
Next, create a policy and define its parameters and finish the wizard.
You can run the backup manually or wait for the scheduler to trigger it. Once the backup completes, CDRA will copy the data to cloud S3 bucket.
Rapid recovery for protected assets
You can accelerate the recovery process ahead of time by creating rapid recovery copies for protected assets.
Creating a rapid recovery copy reduces the RTO for a protected asset but consumes additional cloud resources and incurs additional costs.
Through the CDRS UI, you can accelerate the recovery process by creating rapid recovery copies for protected VMs. Creating a rapid recovery copy starts a rehydration process and converts the VMDK files to the required format depending on the cloud provider environment. The recovery process then only needs to launch the recovered instance.
CDRS monitors available copies and orchestration activities in the cloud. The CDRS user interface can be used for disaster recovery testing and failover. A DR test enables temporary access to a virtual cloud instance to retrieve specific data or verify that the recovered VM is working before running a failover. You would start a failover when the on-premises production environment experiences a disaster or the VM is not running.
You can set the minimum time interval during which rapid recovery copies are not created.
The minimum time interval is 6 hours, and the maximum is 24 hours. The default setting is 12 hours.
To configure Rapid Recovery, login to CDRA console.
Navigate to Protection > Asset Protection
Select the Asset (s) and click on SET RAPID RECOVERY IMAGES
Now select the number of images to preserve between 1 to 5. Each RR copy will store an AMI on AWS.
Click on Set.
Recovery and Failback Test
- Failback initiated from the CDRS UI.
- CDRS launches a restore Service instance.
- Restore Service creates compressed and encrypted segments from the volumes and saves them on S3.
- CDRA creates a local restore virtual machine.
- When done writing the local disks, the CDRA powers off the Restore VM deletes its boot disk and reconfigures the VM to have the same CPU and memory as the Failover EC2 instance had. The VM is now failed back successfully with its disks that include all changes made on the failover EC2 instance.
- The CDRS performs cleanup and deletes snapshots, volumes, and segments from S3.
More to Come Soon.