|Article Number: 539102||Article Version: 3||Article Type: Break Fix|
ECS Appliance,ECS Appliance Gen 1,ECS Appliance Gen 2,ECS Appliance Gen 3,ECS Appliance Software with Encryption,ECS Appliance Software without Encryption,ECS Software
If a normal PUT request is used to replace an object that already exists, and is in retention, the PUT request fails with this S3 error:
ObjectUnderRetention If a multi-part (MPU) PUT is used to replace an object that is in retention, it receives this error:
AccessDenied The issue here is that a normal PUT and a MPU PUT should return the same error but do not.
It is possible that a PUT may fail with a 500 error. If that happens, the sender sees it is an error even if the files was actually uploaded successfully.
Any error tells GeoDrive that the file was unsuccessful, and it will be queued for a retry.
If during the retry, it receives "ObjectUnderRetention", GeoDrive will now understand that the file has been uploaded and will adjust the local file to reflect that.
Since multi-part PUT returns "AccessDenied", GeoDrive will continually attempt to retry the upload.
1. From the GeoDrive console - Confirm if either of the errors below are being produced:
i. Double-click on a sample MPU error to confirm the error output:
File cannot be uploaded to ECS server. Retries are exhausted (ECSxxxxx_ECSxxxxx): e:\x\x\... Site=x File=x Line=x Revision=-v184.108.40.206 Severity=WARNING Category=0 Process=Server Error=2149122451:2149122451 - Forbidden (403). Time=22/11/2019 2:36:39 988 ms Thread=15108Error during multipart upload (e:\xxx) in ECS server ECSxxxxx_ECSxxxxx: 2149122451 - Forbidden (403).ECS: 2001 - Access DeniedHttp: 403 - Forbidden: The server understood the request, but refuses to fulfill it.S3 Code: AccessDeniedS3 Resource: xxxS3 RequestID: xxxxxxxx:xxxxxxxxxxx:xxxxx:xxx ii. Alternative sample console output indicating this issue:
File cannot be uploaded to ECS server. Retries are exhausted (ECSxxxxx_ECSxxxxx): e:\xCryptographic Services failed while processing the OnIdentity() call in the System Writer Object.Error during multipart upload (e:\x) in ECS server ECSxxxxx_ECSxxxxx: 2149122451 - Forbidden (403).ECS: 2001 - Access Denied
2. Gather and confirm the following system and configuration information to be included the Service Request with ECS Support:
i. Version of GeoDrive being used
ii. Name of the affected ECS bucket and the object user
iii. The affected drive and any error output included in the GeoDrive console
iv. If there were any recent changes made to the system's configuration
v. If the affected files are larger than 128 MB (This is the default size where MPUs are used)
vi. Has this value been changed (Option - Performance: The threshold is at the bottom of the dialog)
3. If all the affected files are 128 MB or larger, please increase the MPU threshold (If not, please notify ECS Support through a Service Request and include all prior and subsequent information)
- This increment can be done through Options, then Performance as below (Of the two fields below, the "Minimum Threshold" is the field to be changed):
- This should cause the larger files to avoid MPU and therefore return the correct error ("ObjectUnderRetention")
- All files should work on retry
- When a file is queued for upload, if it fails it will be queued for a retry. If it tries 3 times and fails each time, it gets put on a "failed upload" list. Each failed upload will log the error - After the 3rd retry, it logs an additional error:
File cannot be uploaded to ECS server. Retries are exhausted (ECSxxxxx_ECSxxxxx): e:\x\x\...
- There is a button on the main UI screen that appears if this happens, and the user can press this button to get a list of failed uploads.
- If the user ignores it and does not try to force a retry - After 6 hours all failed uploads are put back in the upload queue for 3 more times (All files therefore retry 3 times every 6 hours).
4. If the above does not resolve this issue, please notify ECS Support through a Service Request and include all prior information.