ECS: CAS retention in ECS

           

   Article Number:     538782                                   Article Version: 3     Article Type:    How To 
   

 


Product:

 

ECS Appliance,Elastic Cloud Storage

 

Instructions:

 

 

   

      This article is about CAS retention in ECS. It also describes advanced retention for CAS applications: event-based retention, litigation hold, and the min/max governor     
      It is an extract from ECS Data Access Guide what can be found and downloaded here: https://www.dell.com/support/home/us/en/19/product-support/product/ecs-appliance-/docs   
   
          
   

     
      CAS retention in ECS   

   
    A CAS C-Clip can have a retention period that governs the length of time the associated object is   
    retained in ECS storage before an application can delete it.   
   
   
    Retention periods   

      Retention periods are assigned in the C-Clip for the object by the CAS application.   

    For example, if a financial document must be retained for three years from its creation date, then a   
    three-year retention period is specified in the C-Clip associated with the financial document. It is   
    also possible to specify that the document is retained indefinitely.   
   
    Retention policies (retention classes)   

      Note: The Centera concept of retention classes maps to retention policies in ECS. This     
      documentation uses retention policies.   

    Retention policies enable retention use cases to be captured and applied to C-Clips. For example,   
    different types of documents could have different retention periods. You could require the   
    following retention periods:   
         
  •         Financial: 3 years     
  •      
  •         Legal: 5 years     
  •      
  •         Email: 6 months     
  •    
   

      When a retention policy is applied to a number of C-Clips, by changing the policy, the retention     
      period changes for all objects to which the policy applies.   

    Retention policies are associated with namespaces in ECS and are recognized by the CAS   
    application as retention classes.   
        

      ECS bucket-level retention and CAS   

    Bucket-level retention is not the default pool retention in Centera. In ECS, CAS default retention is   
    constantly zero.   
        

      Default retention period in objects written without object-level retention in Compliance       
        namespaces
   

    Starting with ECS 3.0, when an application writes C-Clips with no object retention to an ECS CAS   
    bucket in a Compliance namespace, and the bucket has a retention value (6 months, for example),   
    the default retention period of infinite (-1) will be assigned to the C-Clips. The C-Clips can never   
    be deleted because their effective retention period is the longest one between the two: the   
    bucket-level retention period and the default object-level retention.   
        

      CAS precedence   

    When multiple retention periods are applied to a CAS object in ECS, the retention period with the   
    higher value has precedence no matter how the retention was applied.   
        

      How to apply CAS retention   

   

      You can define retention polices for namespaces in the ECS Portal or with the ECS Management     
      API. See Set up namespace retention policies.   

   

      Your external CAS application can assign a fixed retention period or a retention policy to the C-Clip     
      during its creation.   

   

      When applying retention periods through APIs, specify the period in seconds.   

    Note that ECS CAS takes the creation time of the C-Clip for all retention related calculations and   
    not the migration time.   

     
      How to create retention policies with the ECS Management API.     
     
      You can create retention periods and policies using the ECS, a summary of which is provided     
      below.   

                                                                                                                                                                                                     
            Method                      Description         
PUT /object/bucket/{bucketName}/retentionThe retention value for a bucket defines a mandatory           
               retention period which is applied to every object within a           
               bucket. If you set a retention period of 1 year, an object           
               from the bucket cannot be deleted for one year.
GET /object/bucket/{bucketName}/retentionReturns the retention period that is currently set for a           
               specified bucket.
POST /object/namespaces/namespace/{namespace}/           
               retention
For namespaces, the retention setting acts like a policy,           
               where each policy is a <Name>:<Retention period> pair.           
               You can define a number of retention policies for a           
               namespace and you can assign a policy, by name, to an           
               object within the namespace. This allows you to change           
               the retention period of a set of objects that have the           
               same policy assigned by changing the corresponding           
               policy.
PUT /object/namespaces/namespace/{namespace}/           
               retention/{class}
Updates the period for a retention period that is           
               associated with a namespace.
GET /object/namespaces/namespace/{namespace}/           
               retention
Returns the retention policy defined for a namespace.
   

      You can find more information about the ECS Management API in ECS Management REST API      
      introduction at ECS Data Access Guide. The online reference is here: ECS API Reference.   

   

     
      Advanced retention for CAS applications: event-based retention, litigation hold, and the min/max governor   

   

      Describes advanced retention features available in the CAS API that are supported by ECS.   

    Customer applications use the CAS API to enable retention strategies. When CAS workloads are   
    migrated to ECS, ECS awareness of CAS API features allow the customer applications to continue   
    working with the migrated data. In ECS, the following advanced retention management (ARM)   
    features are available without a separate license:   
         
  •         Event-based retention: the ability to configure an object through its C-Clip to apply (trigger) a       
             retention period or retention policy when the CAS application receives a specified event.     
  •      
  •         Litigation hold: the ability to prevent deletion of an object if the CAS application has applied a       
             litigation hold to the object through its C-Clip. The CAS application can apply up to 100       
             litigation holds to an object by creating and applying unique litigation hold IDs.     
  •      
  •         Min/Max governor: The ability for an administrator to set bucket-level limits for fixed retention       
             period or variable retention period. A variable retention period is one that is set to support       
             event-based retention. In ECS, System or Namespace Admins can set the values with the ECS       
             Portal. Programmers can use the ECS Management API to set the values.     
  •    
Note: ARM is supported for legacy CAS data written with any naming scheme that is migrated   
    to ECS.   

      Min/max governor for CAS bucket-level retention   

    From the ECS Portal, locate a CAS bucket and select Edit. All the features shown on the screen   
    below are CAS-only features except for the Bucket Retention Period feature. Bucket Retention     
      Period
is the standard ECS bucket retention feature supported on all ECS bucket types.   
   
       

     
      The CAS bucket retention features are explained in the following table.     
          

                                                                                                                                                                                                                               
            Feature                      Description         
Enforce RetentionIf this feature is turned on, no CAS object can be created without retention information           
               (period or policy). An attempt to save such an object will return an error. If it is turned on, it           
               is possible not to configure Bucket Retention Period even in a compliance-enabled           
               environment.           
               Note: When a CE+ mode Centera is migrated to ECS, Enforce Retention is turned on           
               by default on the bucket.
Bucket Retention           
               Period
If a bucket retention period is specified, then the longer period will be enforced if there is           
               both a bucket-level and an object-level retention period. In a Compliance-enabled           
               environment Bucket Retention Period is mandatory unless retention information in the           
               object is enforced. However, once configured the Bucket Retention Period cannot be           
               reset even when retention information in the object is enforced.
Minimum Fixed           
               Retention Period
This feature governs the retention periods specified in objects. If an object's retention           
               period is outside of the bounds specified here, then an attempt to write the object fails.           
               When using retention policies, the min/max settings are not enforced. Selecting Infinite for           
               Minimum Fixed Retention Period means all retention values must be infinite. Selecting if           
               for Maximum Fixed Retention Period means there is no maximum limit. Min/max           
               retention constrains are applied to any C-Clip written to a bucket. If a clip is migrated by           
               any SDK-based third-party tool the retention should be within bounds, otherwise an error is           
               thrown.
Maximum Fixed           
               Retention Period
Minimum Variable           
               Retention Period
This feature governs variable retention periods specified in objects using event-based           
               retention (EBR). In EBR, a base retention period is set and the programmed trigger function           
               has the ability to increase the retention period when the trigger fires. If an object's new           
               retention period is outside of the bounds specified here, then an attempt to write the object           
               in response to the trigger fails. When using retention policies, the min/max settings are not           
               enforced. Selecting Infinite for Minimum Variable Retention Period means all retention           
               values must be infinite. Selecting if for Maximum Variable Retention Period means there           
               is no maximum limit. Min/max retention constrains are applied to any C-Clip written to a           
               bucket. If a clip is migrated by any SDK-based third-party tool the retention should be           
               within bounds, otherwise an error is thrown.
Maximum Variable           
               Retention Period
   

      Note: If the System Administrator or programmer has not set any values for the fixed and     
      variable retention periods, the ECS Management API get function will not return values for     
      the min/max settings. The Enforce Retention Information in C-Clip will return a default     
      value of false.     
          

   

      Event-based retention   

    Event-based retention (EBR) is an instruction specifying that a record cannot be deleted before an   
    event and during a specified period after the event. In CAS, EBR is a C-Clip with a specified base   
    retention period or retention policy and an application-defined trigger that can set a longer   
    retention period when the trigger fires. The retention period only begins when the trigger fires.   
    When a C-Clip is marked for EBR, it cannot be deleted prior to the event unless a privileged delete   
    is used.   

      When using EBR, the C-Clip life-cycle is as follows:   

   
         
  •         Create: the application creates a new C-Clip and marks it as being under EBR. The application       
             can provide a fixed retention period which acts as a minimum retention and it must provide an       
             event based retention period or policy.     
  •      
  •         Trigger Event: The application triggers the event, which is the starting point of the eventbased       
             retention period or retention policy. At this point the application can assign a new eventbased       
             retention period, provided that it is longer than the one assigned at the time of the CClip       
             creation.     
  •      
  •         Delete: When the application tries to delete the C-Clip, the following conditions must be met:       
             Policy (Namespace) retention has expired       
             Bucket retention has expired       
             Fixed retention has expired       
             The event has been triggered       
             Both the EBR set at the time of creation and any subsequent changes (extensions) at the time of the event have expired     
  •    
    The following figure shows the three possible scenarios for a C-Clip under EBR:   
         
  •         C1 has a fixed or minimal retention which already expired before the event was triggered.     
  •      
  •         C2 has a fixed or minimal retention which will expire before the EBR expires.     
  •      
  •         C3 has a fixed or minimal retention which will expire after the EBR expires.     
  •    
     User-added image   

      For non-compliant namespaces, privileged delete commands can override fixed and variable     
      retention for EBR.   

   

      When applying EBR retention, it must comply with the Min/Max Governor settings for the variable     
      retention period.   

    The table shows the CAS API functions for event-based retention                                                                                                                                                                                                                                                                                       
FunctionDescription
FPClip_EnableEBRWithClassThis function sets a C-Clip to be eligible to receive a future event and           
               enables an event-based retention (EBR) class to be assigned to the CClip           
               during C-Clip creation time.
FPClip_EnableEBRWithPeriodThis function sets a C-Clip to be eligible to receive a future event and           
               enables an event-based retention (EBR) period to be assigned to the           
               C-Clip during C-Clip creation time.
FPClip_IsEBREnabledThis function returns a Boolean value to indicate whether or not a CClip           
               is enabled for event-based retention (EBR).
FPClip_GetEBRClassNameThis function retrieves the name of the event-based retention (EBR)           
               policy assigned to the C-Clip.
FPClip_GetEBREventTimeThis function returns the event time set on a C-Clip when the eventbased           
               retention (EBR) event for that C-Clip was triggered.
FPClip_GetEBRPeriodThis function returns the value (in seconds) of the event-based           
               retention (EBR) period associated with a C-Clip.
FPClip_TriggerEBREventThis function triggers the event of a C-Clip for which event-based           
               retention (EBR) was enabled.
FPClip_TriggerEBREventWithClassThis function triggers the event of a C-Clip for which event-based           
               retention (EBR) was enabled and assigns a new EBR policy to the CClip.
FPClip_TriggerEBREventWithPeriodThis function triggers the event of a C-Clip for which event-based           
               retention (EBR) was enabled and assigns a new EBR period to the CClip.
   

          

        

      Litigation hold   

    Litigation hold allows CAS applications to temporarily prevent deletion of a C-Clip. Litigation hold is   
    useful for data that is subject to an official investigation, subpoena, or inquiry and that may not be   
    deleted until the investigation is over. Once there is no need to hold the data, the litigation hold   
    can be released by the application and normal retention behavior resumes. The CAS application   
    places and removes a litigation hold at the C-Clip level.   

      Note: Even a privileged delete cannot delete a C-Clip under litigation hold.   

    One C-Clip can be under several litigation holds. The application must generate unique litigation   
    hold IDs and be able to track the specific litigation holds associated with a C-Clip. The application   
    cannot query a C-Clip for this information. There is only a function that determines the litigation   
    hold state of the C-Clip. If there is one or several litigation holds on the C-Clip, this function   
    returns true, otherwise, it is false.   

      When using litigation hold, the C-Clip life-cycle is as follows:   

   
         
  •         Create: The application creates a new C-Clip and provides a fixed and/or event-based       
             retention period.     
  •      
  •         Set litigation hold: An application puts the C-Clip on hold. This application can be different       
             from the application that wrote the C-Clip.     
  •      
  •         Release litigation hold: An application releases the C-Clip. This application can be different       
             from the application that sets the litigation hold or wrote the C-Clip.     
  •      
  •         Delete: When the application tries to delete the C-Clip, the following conditions must be satisfied:       
             There are no other litigation holds outstanding on the C-Clip.       
             Policy retention has expired.       
             Standard bucket retention has expired. (Standard bucket retention is available to all ECS object types, but is not recommended for CAS.)       
             Fixed retention period has expired (CAS-only feature).       
             Event-based retention has expired (CAS-only feature).     
  •    
   

      The following figure shows the three possible scenarios for a C-Clip put under litigation hold:   

   
         
  •         C1 has a fixed retention that already expired when put under hold.     
  •      
  •         C2 has a fixed retention that expires during the hold.     
  •      
  •         C3 has a fixed retention that will expire after release of the hold.     
  •    
   

      User-added image   

    A C-Clip can have multiple litigation holds assigned to it. If this is the case, each litigation hold   
    requires a separate API call with a unique identifier for the litigation hold.   

      Note: The maximum size of litigation hold ID is 64 characters. The maximum litigation hold IDs     
      per C-Clip is 100. These limitations are enforced by the CAS API.     
     
      The table shows the CAS API functions for litigation hold     
          

                                                                                                                       
            Function                      Description         
FPClip_GetRetentionHoldThis function determines the hold state of the C-Clip and returns           
               true or false.
FPClip_SetRetentionHoldThis function sets or resets a retention hold on a C-Clip. For           
               multiple litigation holds, provide a unique litigation hold ID for           
               each hold. For multiple holds, make one call per ID.
                                                                                                                             

 

 

Notes:

 

 

The CAS related topics below are detailed in ECS Data Access Guide and will be separated into different KB's:   

         
  •         ECS: Setting up CAS support in ECS - KB 538783        
               
    •             Setting up CAS support in ECS         
    •          
    •             Cold Storage         
    •        
                  
  •      
  •         ECS: Compliance for CAS - KB 538784        
               
    •             Platform hardening and Compliance         
    •          
    •             Compliance and retention policy         
    •          
    •             Compliance agent         
    •        
                  
  •      
  •         ECS: CAS retention in ECS - KB 538782        
               
    •             CAS retention         
    •          
    •             Advanced retention for CAS applications: event-based retention, litigation hold, and the min/max governor         
    •        
                  
  •      
  •         ECS: Set up namespace retention policies for CAS - KB 538786     
  •      
  •         ECS: CAS User & Bucket Administration Best Practices - KB 478327        
               
    •             Create and set up a bucket for a CAS user         
    •          
    •             Set up a CAS object user         
    •        
                  
  •      
  •         ECS: Set up bucket ACLs for CAS - KB 538789     
  •      
  •         ECS: ECS Management APIs that support CAS users - KB 538790     
  •      
  •         ECS: Content Addressable Storage (CAS) SDK API support - KB 538791     
  •      
  •         ECS: CAS (Centera) SDK error codes - KB 427034     
  •