VPLEX: Event 0x8a4861d7 FRU Faulted reported for SPS, Power-Supply or UPS

           

   Article Number:     489789                                   Article Version: 4     Article Type:    Break Fix 
   

 


Product:

 

VPLEX Series,VPLEX Local,VPLEX Metro,VPLEX Geo,VPLEX VS1,VPLEX VS2,VPLEX VS6,VPLEX GeoSynchrony 5.1,VPLEX GeoSynchrony 5.1 Patch 1,VPLEX GeoSynchrony 5.1 Patch 2,VPLEX GeoSynchrony 5.1 Patch 3,VPLEX GeoSynchrony 5.1 Patch 4,VPLEX GeoSynchrony 5.2

 

Issue:

 

 

Impacted VPLEX Hardware:   
    EMC Hardware: VPLEX Series   
    EMC Hardware: VPLEX VS1   
    EMC Hardware: VPLEX VS2   
    EMC Hardware: VPLEX VS6   
    EMC Hardware: VPLEX-Local   
    EMC Hardware: VPLEX-Metro   
    EMC Hardware: VPLEX-Geo   
   
    Impacted VPLEX GeoSynchrony code versions:   
    EMC Software: GeoSynchrony 5.1   
    EMC Software: GeoSynchrony 5.1 Patch 1   
    EMC Software: GeoSynchrony 5.1 Patch 2   
    EMC Software: GeoSynchrony 5.1 Patch 3   
    EMC Software: GeoSynchrony 5.1 Patch 4   
    EMC Software: GeoSynchrony 5.2   
    EMC Software: GeoSynchrony 5.2 Patch 1   
    EMC Software: GeoSynchrony 5.2 Service Pack 1   
    EMC Software: GeoSynchrony 5.2 Service Pack 1 Patch 1   
    EMC Software: GeoSynchrony 5.2 Service Pack 1 Patch 2   
    EMC Software: GeoSynchrony 5.2 Service Pack 1 Patch 3   
    EMC Software: GeoSynchrony 5.3   
    EMC Software: GeoSynchrony 5.3 Patch 1   
    EMC Software: GeoSynchrony 5.3 Patch 2   
    EMC Software: GeoSynchrony 5.3 Patch 3   
    EMC Software: GeoSynchrony 5.3 Patch 4   
    EMC Software: GeoSynchrony 5.4   
    EMC Software: GeoSynchrony 5.4 Service Pack 1   
    EMC Software: GeoSynchrony 5.4 Service Pack 1 Patch 1   
    EMC Software: GeoSynchrony 5.4 Service Pack 1 Patch 3   
    EMC Software: GeoSynchrony 5.4 Service Pack 1 Patch 4   
    EMC Software: GeoSynchrony 5.5   
    EMC Software: GeoSynchrony 5.5 Patch 1   
    EMC Software: GeoSynchrony 5.5 Service Pack 1   
    EMC Software: GeoSynchrony 5.5 Service Pack 1 Patch 1   
    EMC Software: GeoSynchrony 5.5 Service Pack 1 Patch 2   
    EMC Software: GeoSynchrony 5.5 Service Pack 2   
    EMC Software: GeoSynchrony 5.5 Service Pack 2 Patch 1   
    EMC Software: GeoSynchrony 5.5 Service Pack 2 Patch 2   
    EMC Software: GeoSynchrony 6.0   
    EMC Software: GeoSynchrony 6.0 Patch 1   
    EMC Software: GeoSynchrony 6.0 Patch 2   
   
    Received one or more of the following dial homes for any of the listed FRUs,   
   
    Standby-Power-Supplies-A or –B (SPS-A or SPS-B):   
    sps@stand-by-power-supply-A: PartNo 078-000-050 /SerialNo xxxxxxxxxxxxxx /RevNo xxx :The operational state of the specified fru is Faulted. [Versions:MS{Dxx.xx.x.x.x, Dxx.xx.x.x, Dxx.xx.x.x}, Director{x.x.xxx.xx.x}, ClusterWitnessServer{unknown}] RCA: The state of the specified FRU is faulted Remedy: Contact EMC Customer Support.   
   
    sps@stand-by-power-supply-B: PartNo 078-000-050 /SerialNo xxxxxxxxxxxxxx /RevNo xxx :The operational state of the specified fru is Faulted. [Versions:MS{Dxx.xx.x.x.x, Dxx.xx.x.x, Dxx.xx.x.x}, Director{x.x.xxx.xx.x}, ClusterWitnessServer{Dxx.xx.x.x}] RCA: The state of the specified FRU is faulted Remedy: Contact EMC Customer Support.   
   
    Sometimes may see remote cluster reporting on its peer:   
    peer-sps@stand-by-power-supply-A: :The specified FRU Faulted. [Versions:MS{Dxx.xx.x.xx.x, Dxx.xx.x.x, Dxx.xx.x.xx}, Director{x.x.xxx.xx.x}, ClusterWitnessServer{Dxx.xx.x.x}] RCA: The specified FRU is not present.   
   
   
    Power-Supplies (psa# or psb# or power-supply-a# or b#):   
    /engine/psa0: PartNo 071-000-548 / SerialNo xxxxxxxxxxxxxx /RevNo xxx :The operational state of the specified fru is Faulted. [Versions:MS{Dxx.xx.x.x.x, Dxx.xx.x.x, Dxx.xx.x.x}, Director{x.x.xxx.xx.x}, ClusterWitnessServer{unknown}] RCA: The state of the specified FRU is faulted Remedy: Contact EMC Customer Support.   
   
    /engine/psa1: PartNo 071-000-543 / SerialNo xxxxxxxxxxxxxx /RevNo xxx :The operational state of the specified fru is Faulted. [Versions:MS{Dxx.xx.x.x.x, Dxx.xx.x.x, Dxx.xx.x.x}, Director{x.x.xxx.xx.x}, ClusterWitnessServer{Dxx.xx.x.x}] RCA: The state of the specified FRU is faulted Remedy: Contact EMC Customer Support.   
   
    /engine/psb0: PartNo 071-000-548 / SerialNo xxxxxxxxxxxxxx /RevNo xxx :The operational state of the specified fru is Faulted. [Versions:MS{Dxx.xx.x.x.x, Dxx.xx.x.x, Dxx.xx.x.x}, Director{x.x.xxx.xx.x}, ClusterWitnessServer{Dxx.xx.x.x}] RCA: The state of the specified FRU is faulted Remedy: Contact EMC Customer Support.   
   
    /engine/psb1: PartNo 071-000-543 /SerialNo xxxxxxxxxxxxxx /RevNo xxx : The operational state the of specified FRU is Faulted. [Versions:MS{Dxx.xx.x.xx.x, Dxx.xx.x.x, Dxx.xx.x.xx}, Director{x.x.xxx.xx.x}, ClusterWitnessServer{Dxx.xx.x.x}] RCA: : The state of the specified FRU is faulted Remedy: Contact EMC Customer Support.   
   
    power-supply-a1 : PartNo 071-000-543 / SerialNo xxxxxxxxxxxxxx /RevNo xxx:The operational state the of specified FRU is Faulted. Versions:MS{Dxx.xx.x.xx.x, Dxx.xx.x.x, Dxx.xx.x.xx}, Director{x.x.xxx.xx.x}, ClusterWitnessServer{Dxx.xx.x.x}] RCA: The state of the specified FRU is faulted   
   
    power-supply-b0 : PartNo 071-000-543 / SerialNo xxxxxxxxxxxxxx /RevNo xxx:The operational state the of specified FRU is Faulted. Versions:MS{Dxx.xx.x.xx.x, Dxx.xx.x.x, Dxx.xx.x.xx}, Director{x.x.xxx.xx.x}, ClusterWitnessServer{Dxx.xx.x.x}] RCA: The state of the specified FRU is faulted   
   
    Uninterruptable-Power-Supplies (UPS):   
    ups@UPS-A: PartNo 078-000-052 / SerialNo XXXXXXXXXXXX /RevNo xxxx:The operational state of the specified fru is Faulted. [Versions:MS{Dxx.xx.x.xx.x, Dxx.xx.x.x, Dxx.xx.x.xx}, Director{x.x.xxx.xx.x}, ClusterWitnessServer{Dxx.xx.x.x}] RCA: The state of the specified FRU is faulted Remedy: Contact EMC Customer Support.   
   
    ups@UPS-B: PartNo 078-000-052 / SerialNo XXXXXXXXXXXX /RevNo xxxx:The operational state of the specified fru is Faulted. [Versions:MS{Dxx.xx.x.xx.x, Dxx.xx.x.x, Dxx.xx.x.xx}, Director{x.x.xxx.xx.x}, ClusterWitnessServer{Dxx.xx.x.x}] RCA: The state of the specified FRU is faulted Remedy: Contact EMC Customer Support.   
   
     
                                                           

 

 

Cause:

 

 

The call homes listed in the Issue section may be due to tests, or checks, by those components and as the FRU monitoring portion of the code is not aware of these tests, or checks by the components, the code only knows when it sees what it knows as an error to report the event via the call home mechanism.                                                           

 

 

Resolution:

 

 

User needs to check and confirm the state of the reported FRU being reported as "faulted" covered in this article.         
         
          Steps for checking state of reported FRU:
   
   
    User to login to the VPLEX when any of the call homes listed in the Issue Section of this article have been reported and check the status of the reported FRU. If the FRU the issue was reported for is okay the call-home can be ignored. If, when checking the status of a FRU, it is in any other state than indicated in the sample outputs of a properly working state you will need to open an SR with EMC Customer Support to report the issue.   
   
    Follow the steps below to check for the state of the FRUs mentioned in this article,   
   
    To check the state of any of the below FRUs requires logging into the management server of the cluster the reported FRU event from. The following steps explain how to find which cluster and component the event was reported from and then how to log into the management server and VPlexcli.   
   
    To determine which cluster and component the call was reported from:   
   
    1] In the call home alert message to find out what cluster and component the event was reported for   
        look for the SubComponentID the issue was reported for, the first number listed is the cluster ID.   
   
       Example of call home:   
       SymptomCode: 0x8a4861d7     
         Category:      Status     
         Severity:      Error     
         Status:        Failed     
         Component:     DIRECTOR     
         ComponentID:   director-2-2-B
< <cluster-2, director-2-2-B was where the issue was   
        SubComponent:  ZPEM             reported from   
       SubComponentID:     
         CallHome:      Yes     
         FirstTime:     2015-09-29T15:27:35.096Z     
         LastTime:      2016-04-13T17:26:48.047Z     
         Count:         1
   
       CDATA: ups@UPS-B: PartNo 078-000-052 /SerialNo QSxxxxxxxxxx /RevNo 1XE1     
       :The operational state of the specified fru is Faulted.     
       [Versions:MS{D35.37.0.8.0, D35.37.0.5, D35.37.0.9},     
       Director{6.5.351.58.0}, ClusterWitnessServer{D35.27.0.2}] RCA: The state     
       of the specified FRU is faulted Remedy: Contact EMC Customer Support.
   
   
    Logging into the VPLEX   
   
    2] Log into the management server for the VPLEX cluster the call was reported from,   
   
        Sample output:   
        login as: service     
        Using keyboard-interactive authentication.     
        Password:     
        Last login: Wed Apr 13 18:05:04 2016 from xx.xx.xx.xx     
        service@ManagementServer:~>
   
   
    3] Then access the VPlexcli.   
   
         Sample output:   
         service@ ManagementServer:~> vplexcli     
         Trying ::1...     
         Connected to localhost.     
         Escape character is '^]'.     
     
         Enter User Name: service     
     
         Password:     
         Creating logfile:/var/log/VPlex/cli/session.log_service_localhost_     
         Logfile_T24531_YYYYMMDDHHMMSS     
     
         VPlexcli:/>
   
   
    Below are the steps for how to check the state of each FRU that this call home could be for.   
   
    Standby-Power-Supply (SPS-A or SPS-B)   
            
    4] Follow the steps to login as listed above under “Logging into the VPLEX”   
   
    5] Run the following vplexcli command to check the state of the Standby-Power-Supplies,   
   
        ll /engines/*/stand-by-power-supplies/*   
   
        Sample Output as reported for cluster-1:   
        VPlexcli:/> ll /engines/*/stand-by-power-supplies/*     
     
       /engines/engine-1-1/stand-by-power-supplies/stand-by-power-supply-a:     
     
       Attributes:     
       Name                    Value     
       ----------------------  --------------     
       battery-status          fully-charged
<< check this value for the reported FRU   
     battery-time-remaining  300 seconds     
       operational-status      online 
<< check this value for the reported FRU   
     part-number             078-000-050     
       revision-number         A08     
       serial-number           ACLSPxxxxxxxxx     
            
       Contexts:     
       Name     
       ------------     
       conditioning     
     
       /engines/engine-1-1/stand-by-power-supplies/stand-by-power-supply-b:     
     
        Attributes:     
        Name                    Value     
        ----------------------  --------------     
        battery-status          fully-charged     
        battery-time-remaining  300 seconds     
        operational-status      online     
        part-number             078-000-049     
        revision-number         A06     
        serial-number           ACxxxxxxxxxxxx
   
          
    6] If the ‘battery-status’ shows ‘charging’ or ‘unknown’ and the ‘operational status’ shows ‘offline’   
        or ‘removed’ or anything other than what is listed above for these two states there is a   
        possible issue and you will need contact EMC Customer Support for further assistance.   
   
    7] If the SPS states shows as listed above this issue can be ignored.   
   
    Power-Supplies (psa/psb)   
    Follow steps under “Logging into the VPLEX” to access the VPLEX   
   
    8] Run the following vplexcli command to check the current state of the power-supplies,   
   
        ll engines/*/power-supplies/*   
   
        Sample Output as reported for cluster-1:   
      VPlexcli:/> ll engines/*/power-supplies/*     
     
        /engines/engine-1-1/power-supplies/power-supply-a0:     
        Name                            Value     
        ------------------------------  --------------     
        onDC                            false
<< check this value for the reported FRU   
      operational-status              online << check this value for the reported FRU   
      part-number                     071-000-543     
        revision-number                 A08     
        serial-number                   ASTACxxxxxxxxx     
        temperature-threshold-exceeded  false     
     
        /engines/engine-1-1/power-supplies/power-supply-a1:     
        Name                            Value     
        ------------------------------  --------------     
        onDC                            false     
        operational-status              online     
        part-number                     071-000-543     
        revision-number                 A08     
        serial-number                   ASTACxxxxxxxxx     
        temperature-threshold-exceeded  false     
     
        /engines/engine-1-1/power-supplies/power-supply-b0:     
        Name                            Value     
        ------------------------------  --------------     
        onDC                            false     
        operational-status              online     
        part-number                     071-000-543     
        revision-number                 A08     
        serial-number                   ASTACxxxxxxxxx     
        temperature-threshold-exceeded  false     
     
        /engines/engine-1-1/power-supplies/power-supply-b1:     
        Name                            Value     
        ------------------------------  --------------     
        onDC                            false     
        operational-status              online     
        part-number                     071-000-543     
        revision-number                 A08     
        serial-number                   ASTACxxxxxxxxx     
        temperature-threshold-exceeded  false
   
   
      9] If the ‘onDC’ shows as ‘false’ and the ‘operational-status’ shows as ‘online’ this issue can be   
           ignored. The alert was most likely a false positive in that there was a battery test occurring when   
           the FRU monitor portion of the code ran it’s check of the FRU states.   
   
    10] If the ‘onDC’ value shows ‘true’ and the ‘operational-status’ shows ‘failure’ this could indicate an   
          issue. You will need to contact EMC Customer Support for further assistance.   
   
    Uninterruptable Power Supply (UPS)   
    Follow the steps to login as listed above under “Logging into the VPLEX”   
   
    11] Run the following vplexcli command to check the current state of the   
           uninterruptable-power-supplies,   
   
           ll clusters/*/uninterruptible-power-supplies/*   
   
         Sample Output as reported for cluster-1:   
       VPlexcli:/> ll clusters/*/uninterruptible-power-supplies/*     
     
         /clusters/cluster-1/uninterruptible-power-supplies/ups-2-a:     
         Name                      Value     
         ------------------------  -------------     
         battery-replacement-date  03/25/13     
         battery-status            fully-charged
<< checked this value for the reported FRU   
       battery-time-remaining    5min     
         operational-status        online
<< checked this value for the reported FRU   
       revision-number           1XE1     
         serial-number             QSxxxxxxxxxx     
     
         /clusters/cluster-1/uninterruptible-power-supplies/ups-2-b:     
         Name                      Value     
         ------------------------  -------------     
         battery-replacement-date  03/24/13     
         battery-status            fully-charged     
         battery-time-remaining    5min     
         operational-status        online     
         revision-number           1XE1     
         serial-number             QSxxxxxxxxxx
   
   
    12] If the ‘battery-status’ shows as 'fully-charged' and the ‘operational-status’ shows as 'online' this   
          issue can be ignored. The alert was most likely a false positive in that there was a battery test   
          occurring when the FRU monitor portion of the code ran it’s check of the FRU states.   
   
    13] If the ‘battery-status’  shows ‘charging’ and the ‘operational-status’ shows ‘online’ check again   
          in about 5 minutes as the battery test may still be in-progress, if after five minutes and the check   
          shows the same status’ contact EMC Customer Support for further assistance.   
   
    14] If the ‘battery-status’ shows ‘charging’ and the ‘operational-status’ shows ‘offline’ or ‘failure’   
          this may indicate an issue. Please contact EMC Customer Support for further assistance.   
   
    Related KB:   
    483311 - VPLEX: Event 0x8a4861d6 FRU is not present reported for SPS, Power-Supply and/or UPS