ECS: CAS Migration Errors (-10014) Due to Prev.Clip Metadata


   Article Number:     534005                                   Article Version: 2     Article Type:    Break Fix 




ECS SD,Elastic Cloud Storage,ECS Appliance Hardware Series,ECS Appliance Software with Encryption,ECS Appliance Software without Encryption,ECS Software,ECS Software 2.2,ECS Software 3.1,ECS Software 3.2





At code versions lower than 3.3 GP1 an ECS non-native CAS migration could have failures due to the prev.clip value in the c-clip metadata.    
    During the migration -10014 errors would be sent by the ECS and those would correspond with errors in dataheadsvc.log of the nodes that received this transaction.   

[1186605060,sdk,app/59/WRITE_CLIP,-49da99c0:169b6429916:5adb,1] INFO (line 24) WRITE MIGRATIONAPP 54JGRJQE4U9I7e756JSU146JT05NOTAREALCLIP0B6EGMUNFALQFHC -10014 76 1859    






This issue happens because during the write process of a c-clip that has the metadata value prev.clip populated, the ECS dataheadsvc sees this value and tries to confirm that the previous clip exists. However, since this is a new write that check fails and then causes the entire write to fail which is the source of the -10014 error. In the example below the object C5EP91SF2U31HeNOTACLIPDK7P8G41AHJUV7FA0B6EGMUNFALQFHC was a value in the metadata of clip 54JGRJQE4U9I7e756JSU146JT05NOTAREALCLIP0B6EGMUNFALQFHC and gets rejected in the logs with the error NOT_FOUND.   

[1186605060,sdk,app/73/WRITE_CLIP,-49da99c9:169b6429016:5ae6,1-0a325806:1696888061c:227a:37a32] INFO (line 244) Method setHeadSystemMetadataForObject ObjectInfo: [id=9e18b175b50175e7f727527anotanobjectba2798fce6119ef4c8eb, CoS=urn:storageos:VirtualArray:ec07772e-3608-4b99-bb1c-4575891d611c.urn:storageos:ReplicationGroupInfo:xadhttreal-109b-4fce-af68-d2db86a948fc:global, FAMode=disabled, PreviousFAMode=, FAStartVersion=0, FAEndVersion=0, ObjectOwnerZone=null, objectOwnerRecordListnull, keyName=C5EP91SF2U31HeNOTACLIPK7P8G41AHJUV7FA0B6EGMUNFALQFHC] hit error, failed due to directory server returns error ERROR_OBJECT_NOT_FOUND requestId 0a325806:1696888061c:227a:37a33[1186605060,sdk,c003n01/73/WRITE_CLIP,-49da99c9:169b6429016:5ae6,1] ERROR (line 263) app/73/WRITE_CLIP: Failed to update object metadata, error is NOT_FOUND[1186605060,sdk,app/73/WRITE_CLIP,-49da99c9:169b6429016:5ae6,1] ERROR (line 136) Failed to execute the transaction, aborting.[1186605060,sdk,app/59/WRITE_CLIP,-49da99c9:169b6429016:5adb,1] INFO (line 24) WRITE MIGRATIONAPP 54JGRJQE4U9I7e756JSU146JT05NOTAREALCLIP0B6EGMUNFALQFHC -10014 76 1859    






This issue is resolved after an upgrade to 3.3 GP1.