14 Replies Latest reply: Mar 15, 2011 11:37 AM by Clinton Kitson RSS

Powershell scripts for Collecting and Processing vscsistats

Clinton Kitson

These scripts are meant to assist in collecting and processing vscsistats data based on operations to virtual machines disk files on a VMware ESX/i host.

 

The tool vscsistats is a native tool from VMware that collects information on virtual disk operations. However, the tool displays information in running totals which makes the data less useful. The main intention of these scripts is to leverage native vscsistats counters and do collections and calculations on sleep intervals which allows for outputting samples with metric per second values.

 

There are three scripts that are used together to get from collection to displaying data. One that runs as a job in the background collecting vscsistats output, the other that retrieves this output, and another that helps to parse and standardize the output for 3rd party tools and viewing from powershell.

 

 

 

 

Updated 3/7/11 - update to allow for ESX/i 4.0

Updated 3/7/11 - update to fix windows xp/2003 powershell job bug

Updated 3/7/11 - changed name of get_vscsistats.ps1 to collect_vscsistats.ps1

Updated 3/8/11 - bugfixes

 

MD5: 29950E9B1CC53C2B1233032B2CC6D509

  • 1. Re: Powershell scripts for Collecting and Processing vscsistats
    jasonboche

    I was unable to retrieve data.  When I run the script:

     

    C:\junk\test 2 vscsistats> .\get_vscsistats.ps1 -esxip 192.168.110.5 -username root -password xxxx

     

    It pauses for a few moments and returns:

    "not running anymore so quit"

     

    Preflight check was successful.

     

    See attachment

  • 2. Re: Powershell scripts for Collecting and Processing vscsistats
    Clinton Kitson

    The "not running anymore so quit" dialog refers to the script trying to receive results but the vscsistats is outputting nothing for the specified world/diskhandlid's.  Any chance you're running 4.0 (prolly not)? One thing to mention with this script is that it doesn't manage which VMs and disks are being monitored.  It assumes that you have ran the commands from the document to kick the monitoring process off.  The script then logins into the ESX console and receives the metrics via SSH for one session and then repetitive commands every x seconds.   If you log in through SSH and run the command that receives the output of a running vscsistats session, do you see anything?  If not then this is why the script will report that the not running anymore.

     

    Thanks for the feedback, let me know.

  • 3. Re: Powershell scripts for Collecting and Processing vscsistats
    jasonboche

    Yes I had kicked off vscsiStats on the ESX/ESXi host first.  That was my first thought based on the output from the script so I ran the vscsiStats -p all command to make sure data was being collected on the 3 virtual disks for the VM being monitored.

     

    ESX/ESXi 4.1 Update 1

     

    We can talk more tomorrow.

  • 4. Re: Powershell scripts for Collecting and Processing vscsistats
    mryom

    Hi 

     

    I get this error while running the script, is that a problem ?

     

    Receive-Job : Destination array was not long enough. Check destIndex and length
    , and the array's lower bounds.
    At C:\Users\Administrator\Desktop\pshell_get_vscsistats_022011\get_vscsistats.p
    s1:126 char:55
    +     if ($joboutput = (get-job -name job1 | receive-job <<<<  -keep | where {$
    _.field -match '\*\* JOB DONE \*\*' }) | select -last 1) {
        + CategoryInfo          : NotSpecified: (:) [Receive-Job], ArgumentExcepti
       on
        + FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Co
       mmands.ReceiveJobCommand

  • 5. Re: Powershell scripts for Collecting and Processing vscsistats
    Clinton Kitson

    I just posted an updated copy of the vscsistats script that allows for ESX/i 4.0 and fixes some XP/2003 job compatibility issues.  Also make sure to use "set-executionpolicy  bypass" if you are getting security warnings when executing the script.  Security warnings can cause the script to hang in the background as it kicks off the collection jobs.

  • 6. Re: Powershell scripts for Collecting and Processing vscsistats
    gwblok

    Hi, I just downloaded the zip file with the differnet powershell scripts.  I did not see a get_vscsistats.ps1 in the folder.

    When extracted, it came with the following powershell scripts:
    collect_vscsistats.ps1

    exec_vscsistats.ps1

    show_vscsistats.ps1

    ssh_function.ps1

     

    As I was reading through the Instruction PDF, it only mentions Get_vscsistats.ps1

    If I'm totally missing this, I'm sorry, I'm very new to VMWare, and never have done anything from PowerShell before yesterday.

    Any help would be appreciated.  Thanks

     

    Edit:

    In the manual, should i just be replaceing the get_vscsistats.ps1 with collect_vscsistats.ps1?

    I actually tried this command: collect_vscsistats.ps1 -esxip 10.X.X.X -username root - password password

    It returned: need an -outcsv

  • 7. Re: Powershell scripts for Collecting and Processing vscsistats
    Clinton Kitson

    The document was just updated to match the most recent version.  The -outcsv is a requirement, and the script was renamed to collect_vscsistats.ps1 more closely reference what it really does and to be standarized across other similar ones.

     

     

    Thanks

  • 8. Re: Powershell scripts for Collecting and Processing vscsistats
    gwblok

    Thanks, I'm retrying now.

    I'm using the same command but added -outcsv c:\temp  at the end.

    After about 5 Minutes it came back with an error. (attached)

    I'm also assuming I'll have to run this command on all of my esx servers once I figure out what is going on, correct?

     

    Ok... I see I need to install vscsistats on my ESX hosts.. looks like something to do when I'm doing maintance on them sometime.

    Thanks for your help.

  • 9. Re: Powershell scripts for Collecting and Processing vscsistats
    mryom

    Hi Clinton

     

    Im having some problems with the script... The script is running fine, but at some point I start getting an 'error' that there is no io from vm...

     

    I cant figure out if its a timeout or what... The shortest vscsistats i ran lasted only 6 minutes before the message came... The longest ran close to an hour...

     

    Any idea ?

  • 10. Re: Powershell scripts for Collecting and Processing vscsistats
    Clinton Kitson

    This refers to a situation where there is either no active IO occuring, or all vscsistats sessions have ended on the esx host.  Vscsistats has been a bit unpredictable for me at times in terms of length of runs.  Jason Boche mentioned a 30 minute default timeout for the session, but I've seen it last longer and shorter as well.  If anyone has any insight into why a vscsistats session would end without a forced stop I would be interested to hear.

     

    For now you can kick off the command on the ESX host to start back up the session and vscsistats will continue recording the output.  A management script for automagically kicking off and ensuring it is running might be needed to overcome the challenge.

  • 11. Re: Powershell scripts for Collecting and Processing vscsistats
    jasonboche

    Perhaps if the VM migrates to another host.

  • 12. Re: Powershell scripts for Collecting and Processing vscsistats
    mryom

    Thought about that my self... but havnt had the time to look into it...

  • 13. Re: Powershell scripts for Collecting and Processing vscsistats
    landog

    What do you make of this?

     

    On our ESX 4.0 host, in order to monitor a particular VM:  /usr/lib/vmware/bin/vscsiStats -s -w 6526

     

    View vscsistats output:  /usr/lib/vmware/bin/vscsiStats -p all

    Output (in part):

     

    Histogram: number of outstanding Read IOs when a new Read IO is issued for virtual machine worldGroupID : 6649, virtual disk handleID : 8655 {
    min : 15
    max : 2137702820906
    mean : 645656707
    count : 3320
       {
          0                  (<=                  1)
          0                  (<=                 10)
          228                (<=                100)
          39                 (<=                500)
          561                (<=               1000)
          965                (<=               5000)
          39                 (<=              15000)
          21                 (<=              30000)
          23                 (<=              50000)
          34                 (<=             100000)
          1410               (<= 9223372036854775807)
          352225972977664    (<=                 64)
          0                  (>                  64)
       }

    The collect_vscsistats.ps1 script  then errors:  Cannot convert value "9223372036854775807" to type "System.Int32"

     

    Should the  "/usr/lib/vmware/bin/vscsiStats -p all" command show output from world IDs other than the one I entered in the "/usr/lib/vmware/bin/vscsiStats -s -w 6526" command??  (It does.)

     

    Thanks,

    -dog

     


  • 14. Re: Powershell scripts for Collecting and Processing vscsistats
    Clinton Kitson

    Thanks for the data.  It looks like the output of vscsistats is a bit off in this situation.  You said that it is collecting for a VM that you didn't start it for?  Does the worldid shown in the data represent an actual VM? (-l and grep).   Looks like I might need to add some extra logic to handle a vscsistats bug as what you're showing doesn't look accurate (histogram #s increase then go whacky, then end around 64).

     

    I will post an update the allows for the bigger integers (thanks Ryanb).  In your case the bug from vscsistats is going to record the extra junk in the CSV or on the screen, but it can be filtered later before graphing so not a huge problem.  As far as I know ESX/i 4.1 has been flawless for vscsistats and this seems to be a 4.0 issue.