ECS: swift bucket listing is showing all objects as content-type application/octet-stream

           

   Article Number:     535868                                   Article Version: 3     Article Type:    Break Fix 
   

 


Product:

 

ECS Appliance Software without Encryption

 

Issue:

 

 

When uploading objects with specifying the content-type such as image/jpeg the content-type shows as application/octet-stream when the bucket is listed.   

         
  1.         Generate an authentication token the will be used for the following examples). For this the SWIFT object user + Password is needed.     
  2.    
Command:   
    # curl -v -H 'X-Auth-User:<OBJECT USER>' -H 'X-Auth-Key:<PASSWORD>' http://<ECS DATA IP ADDRESS>:9024/auth/v1.0   
    Example:   
admin@ecs:~> curl -v -H 'X-Auth-User:swift_ns' -H 'X-Auth-Key:password' http://10.x.x.x:9024/auth/v1.0                                        * Hostname was NOT found in DNS cache*   Trying xxx.xxx.xxx.xxx...* Connected to xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx) port 9024 (#0)> GET /auth/v1.0 HTTP/1.1> User-Agent: curl/7.37.0> Host: xxx.xxx.xxx.xxx:9024> Accept: */*> X-Auth-User:swift_ns> X-Auth-Key:password> < HTTP/1.1 200 OK< Date: Sun, 11 Aug 2019 10:52:58 GMT< X-Storage-Url: http://10.x.x.x:9024/v1/swift_ns< X-Auth-Token: ECS_AfTLH6DaPi8ZdU7WbyslZjLOUsOXUlCSsTIP6XyCrQNACBTT7DHT4UN3D0cf3yc4l/l1FPkGxy4qgM+yt14wy3c= N1WjzSUQ36wkfr3EbbejvDuMsRVXJw3vAN0+IyTFC5Y= 71f13042c949ad41f621b0a8662e943fcd48376e9715dbcd727db62a5de59ecb< X-Storage-Token: ECS_AfTLH6DaPi8ZdU7WbyslZjLOUsOXUlCSsTIP6XyCrQNACBTT7DHT4UN3D0cf3yc4l/l1FPkGxy4qgM+yt14wy3c= N1WjzSUQ36wkfr3EbbejvDuMsRVXJw3vAN0+IyTFC5Y= 71f13042c949ad41f621b0a8662e943fcd48376e9715dbcd727db62a5de59ecb< x-emc-request-id: 0a3c129f:16c7386948a:216a:0< X-Auth-Token-Expires: 86399< Content-Type: text/html< Content-Length: 0< * Connection #0 to host 10.x.x.x left intactadmin@ecs:~>    
   
         
  1.         Example for uploading object with specifying the content-type as image/jpeg. Use the authentication token obtained from 1st step in the curl command:     
  2.    
Command:   
    # curl -v -X PUT -T <Object to Upload> -i -H 'X-Auth-Token: <Generated Authentication Token from Step 1>' http://<ECS DATA IP ADDRESS>:9024/v1/<Name Space>/<Container>/<Object Name> -H "Content-Type: image/jpeg"    
    Example:   
admin@ecs:~> curl -v -X PUT -T hello.txt -i -H 'X-Auth-Token: ECS_AfTLH6DaPi8ZdU7WbyslZjLOUsOXUlCSsTIP6XyCrQNACBTT7DHT4UN3D0cf3yc4l/l1FPkGxy4qgM+yt14wy3c= N1WjzSUQ36wkfr3EbbejvDuMsRVXJw3vAN0+IyTFC5Y= 71f13042c949ad41f621b0a8662e943fcd48376e9715dbcd727db62a5de59ecb' http://10.x.x.x:9024/v1/swift_ns/swift-con2/object4 -H "Content-Type: image/jpeg"* Hostname was NOT found in DNS cache*   Trying xxx.xxx.xxx.xxx...* Connected to xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx) port 9024 (#0)> PUT /v1/swift_ns/swift-con2/object4 HTTP/1.1> User-Agent: curl/7.37.0> Host: xxx.xxx.xxx.xxx:9024> Accept: */*> X-Auth-Token: ECS_AfTLH6DaPi8ZdU7WbyslZjLOUsOXUlCSsTIP6XyCrQNACBTT7DHT4UN3D0cf3yc4l/l1FPkGxy4qgM+yt14wy3c= N1WjzSUQ36wkfr3EbbejvDuMsRVXJw3vAN0+IyTFC5Y= 71f13042c949ad41f621b0a8662e943fcd48376e9715dbcd727db62a5de59ecb> Content-Type: image/jpeg> Content-Length: 17> Expect: 100-continue>< HTTP/1.1 100 ContinueHTTP/1.1 100 Continue* We are completely uploaded and fine< HTTP/1.1 201 CreatedHTTP/1.1 201 Created< Date: Wed, 17 Jul 2019 11:16:30 GMTDate: Wed, 17 Jul 2019 11:16:30 GMT< x-emc-request-id: 0af69973:16bdcb38ba0:3c09:0x-emc-request-id: 0af69973:16bdcb38ba0:3c09:0< X-Trans-Id: tx0af6997316bdcb38ba03c-09000000000X-Trans-Id: tx0af6997316bdcb38ba03c-09000000000< ETag: 248ea0bd4f456e01ea247331890bae5aETag: 248ea0bd4f456e01ea247331890bae5a< Content-Type: text/htmlContent-Type: text/html< Last-Modified: Wed, 17 Jul 2019 11:16:30 GMTLast-Modified: Wed, 17 Jul 2019 11:16:30 GMT< X-Timestamp: 1563362190X-Timestamp: 1563362190< x-emc-mtime: 1563362190964x-emc-mtime: 1563362190964< Location: http://10.x.x.x:9024/v1/swift_ns/swift-con2/object4Location: http://10.x.x.x:9024/v1/swift_ns/swift-con2/object4< Content-Length: 0Content-Length: 0<* Connection #0 to host 10.x.x.x left intact    
   
         
  1.         Bucket listing shows the content type as application/octet-stream:     
  2.    
Command:   
    # curl -v -X GET -H 'X-Auth-Token: <Generated Authentication Token from Step 1>' http://<ECS DATA IP ADDRESS>:9024/v1/<Name Space>/<Container>/\?format=json   
    Example:   
dmin@ecs:~> curl -v -X GET -H 'X-Auth-Token: ECS_AfTLH6DaPi8ZdU7WbyslZjLOUsOXUlCSsTIP6XyCrQNACBTT7DHT4UN3D0cf3yc4l/l1FPkGxy4qgM+yt14wy3c= N1WjzSUQ36wkfr3EbbejvDuMsRVXJw3vAN0+IyTFC5Y= 71f13042c949ad41f621b0a8662e943fcd48376e9715dbcd727db62a5de59ecb' http://xxx.xxx.xxx.xxx:9024/v1/swift_ns/swift-con2/\?format=json* Hostname was NOT found in DNS cache*   Trying xxx.xxx.xxx.xxx...* Connected to xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx) port 9024 (#0)> GET /v1/swift_ns/swift-con2/?format=json HTTP/1.1> User-Agent: curl/7.37.0> Host: xxx.xxx.xxx.xxx:9024> Accept: */*> X-Auth-Token: ECS_AfTLH6DaPi8ZdU7WbyslZjLOUsOXUlCSsTIP6XyCrQNACBTT7DHT4UN3D0cf3yc4l/l1FPkGxy4qgM+yt14wy3c= N1WjzSUQ36wkfr3EbbejvDuMsRVXJw3vAN0+IyTFC5Y= 71f13042c949ad41f621b0a8662e943fcd48376e9715dbcd727db62a5de59ecb>< HTTP/1.1 200 OK< Date: Wed, 17 Jul 2019 11:17:58 GMT< X-Timestamp: 1563361880117< X-Container-Object-Count: 0< X-Container-Bytes-Used: 0< x-emc-retention-period: 0< x-emc-is-stale-allowed: false< Accept-Ranges: bytes< x-emc-request-id: 0af69973:16bdcb38ba0:3c09:12< X-Trans-Id: tx0af6997316bdcb38ba03c-091200000000< Content-Type: application/json< Content-Length: 313<* Connection #0 to host 10.x.x.x left intact[{"content_type":"application/octet-stream","name":"object3","hash":"248ea0bd4f456e01ea247331890bae5a","bytes":17,"last_modified":"2019-07-17T11:13:35.325"},{"content_type":"application/octet-stream","name":"object4","hash":"248ea0bd4f456e01ea247331890bae5a","bytes":17,"last_modified":"2019-07-17T11:16:30.964"}]admin@ecs:~>    
   
         
  1.         Direct Object call shows the correct content type:     
  2.    
Command:   
    # curl -v -X GET -H 'X-Auth-Token: <Generated Authentication Token from Step 1>' http://<ECS DATA IP ADDRESS>:9024/v1/<Name Space>/<Container>/<Object Name>/\?format=json *   
    Example:   
admin@ecs:~> curl -v -X GET -H 'X-Auth-Token: ECS_AfTLH6DaPi8ZdU7WbyslZjLOUsOXUlCSsTIP6XyCrQNACBTT7DHT4UN3D0cf3yc4l/l1FPkGxy4qgM+yt14wy3c= N1WjzSUQ36wkcd727db62a5de59ecb' http://10.x.x.x:9024/v1/swift_ns/swift-con2/object4\?format=json* Hostname was NOT found in DNS cache*   Trying xxx.xxx.xxx.xxx...* Connected to xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx) port 9024 (#0)> GET /v1/swift_ns/swift-con2/object4?format=json HTTP/1.1> User-Agent: curl/7.37.0> Host: xxx.xxx.xxx.xxx:9024> Accept: */*> X-Auth-Token: ECS_AfTLH6DaPi8ZdU7WbyslZjLOUsOXUlCSsTIP6XyCrQNACBTT7DHT4UN3D0cf3yc4l/l1FPkGxy4qgM+yt14wy3c= N1WjzSUQ36wkfr3EbbejvDuMsRVXJw3vAN0+IyTFC5Y= 71f13042c949ad41f621b0a8662e943fcd48376e9715dbcd727db62a5de59ecb>< HTTP/1.1 200 OK< Date: Wed, 17 Jul 2019 11:19:10 GMT< x-emc-request-id: 0af69973:16bdcb38ba0:3c24:0< X-Trans-Id: tx0af6997316bdcb38ba03c-24000000000< Accept-Ranges: bytes< X-Timestamp: 1563362190< ETag: 248ea0bd4f456e01ea247331890bae5a< Last-Modified: Wed, 17 Jul 2019 11:16:30 GMT< x-emc-mtime: 1563362190964< Content-Type: image/jpeg< Content-Length: 17<2341234223456782* Connection #0 to host 10.x.x.x left intact    
                                                             

 

 

Cause:

 

 

This is a known issue in ECS 3.2 causing swift container listing showing all objects as content-type application/octet-stream when the bucket initially was created via ECS Management UI.   
   
    The API Type for the swift container is shown as S3:   

admin@ecs:~> svc_bucket info  swift-con2svc_bucket v1.0.21 (svc_tools v1.5.4)                 Started 2019-07-17 10:58:22Bucket ID                        swift_ns.swift-con2  Name                             swift-con2  Namespace                        swift_ns  Owner User                       swift_ns  Owner VDC Name                   VDC1  Owner zone/VDC ID                urn:storageos:VirtualDataCenterData:7cc29918-6a4f-471a-b37d-0b80c4456b2d  Keypool Hash ID                  31ff7027fd6f29147db9c67ee23b65c1743d9fd26b61ceb4f09fb619a14399e1  Bucket ACL:    Type       Affects    Name                                Access Rights    user       file/dir   swift_ns                   FULL_CONTROL  Replication Group (vpool) Name   rg-ams  Replication Group (vpool) ID     urn:storageos:ReplicationGroupInfo:f77c41f0-533c-435c-a349-088b3dc198ca:global  Bucket Creation Date             2019-06-27 11:26:52 (1561634812024)  Temp Failed (TSO)                False  API Type                         S3  FS Access Enabled                False  Encryption Enabled               True  Versioning State                 Disabled  ADO Enabled                      True  ADO Read-Only                    False  Default FS Access permissions (for owner group):    File                        Dir    Read    Write   Exec       Read    Write   Exec    Unk     Unk     Unk        Unk     Unk     Unk  Object count                     2  Total Object Size                0 GB    
   
                                                                

 

 

Change:

 

 

Code upgrade from ECS 3.2 to 3.3 or greater.                                                           

 

 

Resolution:

 

 

If a Bucket is created in UI, the bucket API Type will be S3 and not SWIFT. Workaround: Use API for creating buckets, what will set bucket API type to SWIFT as needed.