ECS: [ATMOS_test.py (oc_map)] ATMOS API CRUD test (writes, reads, deletes, performance, specify file count, size, keep files, etc..)

           

   Article Number:     540007                                   Article Version: 3     Article Type:    Break Fix 
   

 


Product:

 

ECS Appliance,ECS Appliance Software with Encryption,ECS Appliance Software without Encryption,ECS Software

 

Issue:

 

 

   

         
  •         Inability to test writes, reads, deletes, performance, etc.., via ATMOS API.     
  •      
  •         Inability to compare checksum(s) before a write, and after a read.     
  •      
  •         Inability to create and keep specific amount of files of specific size.     
  •    
                                                             

 

 

Cause:

 

 

   

         
  •         Tedious manual efforts     
  •      
  •         Lack of automation     
  •    
                                                             

 

 

Change:

 

 

 Automation through "oc_map"                                                           

 

 

Resolution:

 

 

   

         
  •         Download "oc_map" (Contains "s3_test.py" under "~/oc_map/suite/").  Download locations under "Notes" section.     
  •      
  •         To get usage:     
  •    
   
#python s3_test.py -h    
   
         
  •         Examples / Typical Usage:     
  •    
   
~/oc_map/suite> python atmos_test.py -h** ATMOS test usage / example:   #python atmos_test.py --atmos_uid <subtenant>/<user> --ss <secret_key> --chksum    
   
         
  •         Example output:     
  •    
   
~/oc_map/suite> python atmos_test.py --data_creds atmos_creds.txt --atmos_type objects --chksum --f_count 2 --f_size 1024atmos_test_v1.0.0.0 (oc_map_v3.2.3.2_RC_3), Started: 01-04-2020 16:47:46- Performing API test (write, read, delete)  -- Creating file "file_0"...  DONE, Size = "1024", sha256 = "cbca95186c7c4f00d052a2cfe70c21da401cd90614fdd1afc9185269877d3012"  -- Creating file "file_1"...  DONE, Size = "1024", sha256 = "7a2577d5ab09db2d5307bad565609c1cf6605aea5418786c00f01c9611ca2d0c"    --> DUR = 0 (Days) 00:00:00.001 (hh:mm:ss.ms)  -- Write all files...    - Performing ATMOS API call "POST" on file list "/home/admin/oc_map/suite/oc_cache/01-04-2020/test_files_tmp" against bucket "626b78f471a44abca3c07d1e2859df46".      --> Updates will occur every 1000 objects.       [SUCCESS]: 201 [OBJECT]: /home/admin/oc_map/suite/oc_cache/01-04-2020/file_0 : [OID]: 4763b98acadc71d8e2ffbe3ad3815c3f43ee5979c52bb865fe0fe5fbb75f0c9b       [SUCCESS]: 201 [OBJECT]: /home/admin/oc_map/suite/oc_cache/01-04-2020/file_1 : [OID]: b16d0dafcaed025747329682ad2d87f8aae8909324a5b563e6e74c1d5c7a3422    -- Total Objects Scanned: 2       --> SUCCESS: 2       --> ERROR:   0       --> DUR:     0 (Days) 00:00:00.168 (hh:mm:ss.ms)  -- Cleanup files...  DONE    --> DUR = 000 (ms)  -- Read files...    - Performing ATMOS API call "GET" on file list "/home/admin/oc_map/suite/oc_cache/01-04-2020/test_oids" against bucket "626b78f471a44abca3c07d1e2859df46".      --> Updates will occur every 1000 objects.       [SUCCESS]: 200 [OBJECT]: b16d0dafcaed025747329682ad2d87f8aae8909324a5b563e6e74c1d5c7a3422       [SUCCESS]: 200 [OBJECT]: 4763b98acadc71d8e2ffbe3ad3815c3f43ee5979c52bb865fe0fe5fbb75f0c9b    -- Total Objects Scanned: 2       --> SUCCESS: 2       --> ERROR:   0       --> DUR:     0 (Days) 00:00:00.077 (hh:mm:ss.ms)  -- Compare checksums...       [MATCH]:       [CURRENT] cbca95186c7c4f00d052a2cfe70c21da401cd90614fdd1afc9185269877d3012 [ORIGINAL] cbca95186c7c4f00d052a2cfe70c21da401cd90614fdd1afc9185269877d3012       [MATCH]:       [CURRENT] 7a2577d5ab09db2d5307bad565609c1cf6605aea5418786c00f01c9611ca2d0c [ORIGINAL] 7a2577d5ab09db2d5307bad565609c1cf6605aea5418786c00f01c9611ca2d0c    --> DUR = 000 (ms)  -- Delete files...    - Performing ATMOS API call "DELETE" on file list "/home/admin/oc_map/suite/oc_cache/01-04-2020/16:47:46_626b78f471a44abca3c07d1e2859df46_api_list.log" against bucket "626b78f471a44abca3c07d1e2859df46".      --> Updates will occur every 1000 objects.       [SUCCESS]: 204 [OBJECT]: b16d0dafcaed025747329682ad2d87f8aae8909324a5b563e6e74c1d5c7a3422       [SUCCESS]: 204 [OBJECT]: 4763b98acadc71d8e2ffbe3ad3815c3f43ee5979c52bb865fe0fe5fbb75f0c9b    -- Total Objects Scanned: 2       --> SUCCESS: 2       --> ERROR:   0       --> DUR:     0 (Days) 00:00:00.090 (hh:mm:ss.ms)  -- Cleanup files...  DONE    --> DUR = 000 (ms)  -- Test Duration = 0 (Days) 00:00:00.458 (hh:mm:ss.ms)- Finished: 01-04-2020 16:47:48,  Will now exit.    
   
         
  •         Options:     
  •    
   
~/oc_map/suite> python atmos_test.py -h* Data API test options:  --data_test                 Test "Write, Read, Delete" (Required ONLY in ".oc_map.py")  --atmos_uid                 <subtenant>/<s3_ecs_user>  --atmos_ip                  <ECS_data_IP>:<9022 or 9023>  --ss                        s3 secret key  --atmos_type                "namespace" or "objects" or "object" **(Required)  --atmos_share               <share name>. Slashes not needed.  Use to create or read from share.  Needed to upload full paths                                          --> Not needed for basename file  - Optional:      --chksum                    Generate checksum of test files, before and after write.  Compare and flag on mismatch      --f_count                   How many files to generate (1 - 100000, default = 1)      --f_size                    Size of each file measured in Bytes (1 (1 Byte) - 1073741824 (1 GB), default = 1024)      --create_bkts               Create buckets      --b_count                   How many buckets to create      --keep_file                 Keep file copied back      --w_to_log                  Output to default log ** Enabled by default      --no_log                    Disable logging      --no_read                   Disable Read Test      --no_delete                 Disable Delete Test      --no_bkt_delete             Disable deletion of buckets created with "--create_bkts"      --no_cleanup                Disable local file cleanup      -q  --quiet                 Suppress printing          --quiet_off             Disable automatic quiet      --range_bytes               <bytes-bytes> or <none>, Range in bytes      --add_header                <x-emc_header:value> add "x-emc" header to new file (avoid spaces)                                  --> Cannot be used with options "--s3_create_bkt" and "--atmos_create_bkt"                                  --> Use with "--api_put" and "--api_put_list"      --update_header             <x-emc_header:value> add "x-emc" header to existing file or new bucket (avoid spaces)                                  --> Can be used with options "--s3_create_bkt" and "--atmos_create_bkt"                                  --> Use with "--api_put" and "--api_put_list"      --limited_off           Display Response / Header details on all objects.  Default is to display only on single object* Advanced / Secure Options:      --secure_options            Reference KB 517148      --advanced_options          Advanced options    
                                                             

 

 

Notes: