VPLEX: Error 0x8a36901a Dial Home for Rx Power low

Environment:

EMC VPLEX

EMC VPLEX 4.0.0.00.00.11

 

Description:

FC port becomes disabled
FC port goes down

Error code: 0x8a36901a / 2318831642 decimal

 

Resolution:

A degraded Back-End port can cause random and/or continuous access issues to disks with read error messages such as "lost access to disk" or "no mirrors exists". When the RX Power level on a VPLEX Fibre Channel port goes below the low limit, the VPLEX code will disable the port affected.

  • On a Symmetrix target, this maybe indicated by a "0x4400 INTERNAL  TARGET FAILURE asc/ascq" response on the I/O.
  • CLARiiON target does not give such a clear sign. However, it may be indicated by the following events reported in the firmware log:
    • Some tach/19 events of varying severity of I/O timeout percentage.
    • Logical unit resets indicated by scsi/28 events.
    • A small number of scsi/27 events with 0x2900 asc/ascqs.
    • stdf/10 TMF Abort Task events on the front-end.

 

 

These log activities can come in bursts with gaps of a couple of minutes in between. Eventually, such bursts can result in disks being marked dead. For these disks, there can be a high number of tach/19 events reported.

 

For Event message reporting RX Power low, CDATA[tach18 (B4-FC02): RxPower Low: 199, turning OFF port.

Follow the steps below, doing only one at a time, and after doing each step, checking the port to see if the issue is resolved. It is necessary to manually re-enable the port after each step to check if the issue was resolved as the port will not come back online on it's own.

 

There are two ways to manually enable a port:

  • From the VPlexcli enter the following command. The port specified in the command below is from the event message above:

set /**/B4-FC02::enabled true

  • Access the VPLEX GUI to enable the port.

a) Open a web browser.

b) In the address URL field, enter https://IP<address of the management server>. The following message will appear:

 

"We recommend that you close this web page and do not continue to this website.
           Click here to close this web page.
           Continue to this website (not recommended).
           More information" 


c) Choose "Continue to this website".
d) The EMC VPLEX Management Console login screen will appear. Log in using the default username and password used to access the VPLEX from the CLI.
e) Welcome to VPLEX 4.0 page will appear. Click on the tab at the top titled "Provision Storage".
f) From the "Provision Storage" page, find the cluster-id the port was disabled on and under clusters "Host" selection, click on "Ports". In the right hand pane, in the "Enabled?" column, look for the port that was disabled. It will be listed in the "Enabled?" column as "False".
g) Left click on the port name in the "Name" column. This will bring up the "Port Properties" window.
h) The second selection down is the "Enabled?" selection. Click on the box so a check appears, then click "OK". The "Port Properties" window will close and the port should now show as "True", meaning it is now enabled.


One thing to keep in mind, when enabling the port, it may come up "True" but, if the issue is not really fixed, the RX Power issue will return and the port will become disabled again. It is necessary to monitor the port, perhaps for the rest of the day or over a couple of days, to confirm whether the issue is fixed. If the issue returns, go to the next steps listed below. It may be necessary to repeat this resolution process after each enabling of the port until the issue is truly resolved.

Steps to follow:

a).Identify the physical port specified in the event. In the above event message, the port affected is B4-FC02. Follow the cable and find the SAN switch port the fibre is connected to or, if there is a topology map, see if the port to port information is included.
 
b).Clean and re-seat the cable on both ends.
 
c).Re-seat the SFP on both ends.
 
d).Replace the SFP on both ends.
 
e).Contact EMC support for proper FRU replacement procedure of the I/O SLIC.

 

Reference:

EMC Support Solution Number: 69658