Data Domain Cloud tier : integrating data domain with amazon aws S3

           

   Article Number:     534069                                   Article Version: 2     Article Type:    How To 
   

 


Product:

 

Data Domain Cloud DR,Data Domain

 

Instructions:

 

 

The following article walks you through the needed steps to configure Data Domain cloud tier capabilities with amazon aws S3 . This guide is mainly divided into 4 main parts:   

         
  •         Adding the required amazon aws user credentials from aws "IAM"       
  •      
  •         Importing the CA certificate to enable the communication between Data Domain and S3     
  •      
  •         Adding the cloud unit from Data Domain      
  •      
  •         Naming of the cloud unit      
  •    
First :Adding "IAM" user credentials   
   
    The first step in integrating Data Domain cloud tier with amazon AWS S3 is to add the required AWS user credentials from aws "IAM" . This user credentials will be imported to the data domain system to authorize the communicate with the amazon S3    
   
    The AWS user credentials must have permissions to :   
         
  •         create and delete buckets     
  •      
  •          add, modify, and delete files within the buckets they create.     
  •    
S3FullAccess is preferred, but these are the minimum requirements :   
         
  •         CreateBucket      
  •      
  •         ListBucket     
  •      
  •         DeleteBucket     
  •      
  •         ListAllMyBuckets     
  •      
  •         GetObject     
  •      
  •         PutObject     
  •      
  •         DeleteObject     
  •    
    A. go to :https://aws.amazon.com/  and log in to the AWS console or create a new account if this is your first time .   
   
    sign it to amazon aws console   
   
    B. from the top left corner choose services , and search for IAM  (AWS Identity and Access Management ), so we can create and manage AWS users and groups, and use permissions to allow and deny their access to AWS resources.    
   
    IAM access   
   
    C. from the IAM page select "users" from the left menu then select " add user" :   
   
    User-added image   
   
    D. give your new user a name , for example :"DD_S3_cloudtier" .   
    Select the access type to give it programmatic access , then click Next:   
   
     User-added image   
   
    E. now we need to give this user the required permissions to use S3 resources . select add user to group , and then select create group :   
   
    User-added image   
   
    F. give a unique name for the group , for example :"S3FullAccess_DD_cloudtier" and then search for "AmazonS3FullAccess" when the option appears in the result menu select it and then click create group :    
   
    User-added image   
   
    G. new you will be prompted back to previous menu  ,select the group we just created "S3FullAccess_DD_cloudtier" then click next :    
   
    User-added image   
   
    H. click next for the tags menu we don't need to make changes , so e reach the review menu , double check that  the details you entered are correct then click "create user "  :   
        
    User-added image   
   
    I. we reach an important page :    
    you have now the user "access key ID" and "secret access key" ,  you will be using them to integrate the Data Domain with your S3  resources , click "download .csv" and save this CSV file in a secure place and copy the access key ID and secret access key because we will be using them in a while in Data Domain .    
   
    User-added image   
   
   
    Second: Importing CA certificate   
   
    you need to Import the CA certificate to enable the communication between your Data Domain system and amazon S3   
   
    A. to download  AWS root certificate , go to :https://www.digicert.com/digicert-root-certificates.htm   
    and then select the Baltimore CyberTrust Root certificate .   
   
    User-added image   
        
         
  •         If your downloaded certificate has a .CRT extension, it will need to be converted to a PEM-encoded certificate. If so, use OpenSSL to convert the file from .crt format to .pem     
  •      
  •         (for example, openssl x509 -inform der -in BaltimoreCyberTrustRoot.crt -out BaltimoreCyberTrustRoot.pem).     
  •      
  •         You can know more about how to convert the certificate to PEM from the following KB: https://support.emc.com/kb/488482     
  •    
   
    B. another option is to go to the following page : https://baltimore-cybertrust-root.chain-demos.digicert.com/info/index.html   
    and copy the certificate to paste it in the Data Domain system as we will do next    
   
    User-added image   
   
   
    C. Now go to Data Domain GUI and follow the following procedure :    
         
  •         1. Select Data Management > File System > Cloud Units.     
  •      
  •         2. In the tool bar, click Manage Certificates. The Manage Certificates for Cloud dialog is displayed.     
  •      
  •         3. Click Add.     
  •      
  •         4. Select one of these options:                
               
    •             I want to upload the certificate as a .pem file.         
    •        
                  
  •    
   
                Browse to and select the certificate file.   
   
         
  •          I want to copy and paste the certificate text.     
  •    
                       Copy the contents of the .pem  file to your copy buffer.   
                        Paste the buffer into the dialog.   
         
  •         5. Click Add.     
  •    
User-added image   
   
   
    we are done with adding the CA certificate , next we are going to add our S3 cloud unit from Data Domain GUI :    
   
    Third:  Adding the clout unit to Data Domain   
   
    here is a quick comparison of some of the differences between DDOS releases and their cloud tier options available :   
                                                                                                                                                      
            DDOS Versio                      Capabilites         
6.0              
                 
  •                 Only supports "S3 standart storage" class             
  •              
  •                 Doesn't have a cloud provider verification method              
  •              
  •                 Doesn't support large object size feature              
  •            
              
6.1              
                 
  •                 Supports "standard" and "Standard-Infrequent Access (S3 Standard-IA)" storage classes             
  •              
  •                 6.1.1.5 >= : Have the cloud provider verification method              
  •              
  •                 supports large object size feature              
  •            
              
6.2              
                 
  •                 Supports "Standard" , "Standard-IA" and "One Zone-Infrequent Access (S3 One Zone-IA)"             
  •              
  •                 have the cloud verification method              
  •              
  •                 supports large object size feature              
  •            
              
        
    from Data domain GUI , follow this procedure to add the S3 cloud unit :   
         
  •         1. Select Data Management > File System > Cloud Units.     
  •      
  •         2. Click Add. The Add Cloud Unit dialog is displayed.     
  •      
  •         3. Enter a name for this cloud unit. Only alphanumeric characters are allowed. The remaining fields in the Add Cloud Unit dialog pertain to the cloud provider account.     
  •      
  •         4. For Cloud provider, select Amazon Web Services S3 from the drop-down list.     
  •      
  •         5. Select the storage class from the drop-down list : based on the version of the DDOS you are having , you will find different options based on the table above .     
  •    
               know more details about different supported S3 storage classes from the following link to choose the storage class best suitable               for your backup needs :   
               https://aws.amazon.com/s3/storage-classes/   
                  
         
  •         6. Select the appropriate Storage region from the drop-down list.     
  •      
  •         7. Enter the provider Access key "as password text" , the one we obtained from amazon IAM in step 1      
  •      
  •         8. Enter the provider Secret key "as password text" ,the one we obtained from amazon IAM in step 1      
  •      
  •         9. Ensure that port 443 (HTTPS) is not blocked in firewalls. Communication with the AWS cloud provider occurs on port 443.     
  •      
  •         10. If an HTTP proxy server is required to get around a firewall for this provider, click Configure for HTTP Proxy Server. Enter the proxy hostname, port, user, and password.     
  •    
   
    User-added image   
        
         
  •         11. if you are having DDOS >= 6.1.1.5 then click the cloud verification button ,     
  •    
             more details about Data Domain cloud verification tool could be found here :https://support.emc.com/kb/521796   
                 
              if your DDOS version is 6.0 then click add as the cloud verification option is not available in this release .    
         
  •         12. Click Add. The File System main window now displays summary information for the new cloud unit as well a control for enabling  and disabling the cloud unit.     
  •      
  •               
  •    
note :     
    you can update the S3 cloud unit access key and secret access key ID afterwards from Data Domain GUI easily if needed .   
   
   
    Third:  Naming of the cloud unit   
   
    If we go back now to amazon S3 , we will find that the Data Domain system created 3 buckets for this cloud unit .   
   
    User-added image   
   
    The naming convention for the 3 buckets are as follows:   
         
  •         A 16 character hexadecimal string     
  •      
  •         A dash character ('-')     
  •      
  •         Another 16 character hexadecimal string , the hexadecimal string is unique for this cloud unit      
  •      
  •         Another dash character ('-')     
  •      
  •         The buckets will end with the string '-d0', '-c0' and '-m0'.     
  •      
  •         The bucket ending with the string '-d0' is used for data segments.     
  •      
  •         The bucket ending with the string '-c0' is used for configuration data.     
  •      
  •         The bucket ending with the string '-m0' is used for metadata.     
  •    
    for more details about the naming of the cloud units check the following KB :https://support.emc.com/kb/487833   
   
   
    You are now done with creating S3 cloud unit that is integrated with your Data Domain system , and ready to start applying data movement policies for your Mtrees to migrate the data to the newly created cloud tier unit .                                                                                                                           

 

 

Notes:

 

 

   

         
  •         For better cloud tier capabilities, we would recommend upgrading to  DDOS 6.1.2.0 and later to benefit from "Large Object Size for Cloud Tier" feature added in theses releases for better cost and space optimization :     
  •    
               check the following KB for more details :https://support.emc.com/kb/522706   
        
         
  •         How to remove cloud tier unit from Data domain:     
  •    
              check the following KB for more details :https://support.emc.com/kb/488612   
        
         
  •         Configuring the Data movement policy , and more details about cloud tier  :     
  •    
               check the following admin guide (starting from page 427 for the data-movement policy configuration)  :   
               https://support.emc.com/docu78746_Data-Domain-Operating-System-6.0-Administration-Guide.pdf?language=en_US   
        
         
  •         Cloud tier cleaning :      
  •    
               check the following KB for more details :https://support.emc.com/kb/487657