VPLEX: Health-check --full reports Call Home "Error" state post NDU

           

   Article Number:     523118                                   Article Version: 3     Article Type:    Break Fix 
   

 


Product:

 

VPLEX GeoSynchrony,VPLEX Local,VPLEX Metro,VPLEX Series,VPLEX VS2,VPLEX VS6

 

Issue:

 

 

   

      An Error is reporting in the command health-check --full post upgrade but the Call Home functions properly.   

   
         
  •                 

              Pre NDU Health-check --full doesn't report an error.       

                  
  •      
  •                 

              Post NDU Health-check --full reports "Checking Call Home Status" as Error.       

                  
  •      
  •                 

              ConnectEMC_config.xml file looks the same as pre NDU as post NDU.       

                  
  •      
  •                 

              No issues seen in connectemc related logs.       

                  
  •      
  •                 

              The SMTP service is reachable and non-blocked.       

                  
  •      
  •                 

              Call-Home works right, for every triggered call-home test.       

                  
  •      
  •                 

              SYR / CLM system determine call home alerts have being correctly received. Hence, confirming Connecthome is received.        

                  
  •    
   

      Comparing PRE & POST Non-Disruptive Upgrade (NDU)   

   

      PRE NDU         
         
          VPlexcli:/> health-check --full
   

   

      Configuration (CONF):   

   

      Checking VPlexCli connectivity to directors.................... OK   

   

      Checking Directors Commission.................................. OK   

   

      Checking Directors Communication Status........................ OK   

   

      Checking Directors Operation Status............................ OK   

   

      Checking Inter-director management connectivity................ OK   

   

      Checking ports status.......................................... OK   

   

      Checking Call Home............................................. OK   

   

      Checking Connectivity.......................................... OK   

   

          

   

      POST NDU   

   

      VPlexcli:/> health-check --full   

   

      Configuration (CONF):   

   

      Checking VPlexCli connectivity to directors.................... OK   

   

      Checking Directors Commission.................................. OK   

   

      Checking Directors Communication Status........................ OK   

   

      Checking Directors Operation Status............................ OK   

   

      Checking Inter-director management connectivity................ OK   

   

      Checking ports status.......................................... OK   

   

      Checking Call Home Status...................................... Error   

   

          

   

      service@vplexMM:/var/log/VPlex/cli> more  health_check_full_scan.log   

   

      Configuration (CONF):   

   

      Checking VPlexCli connectivity to directors.................... OK   

   

      Checking Directors Commission.................................. OK   

   

      Checking Directors Communication Status........................ OK   

   

      Checking Directors Operation Status............................ OK   

   

      Checking Inter-director management connectivity................ OK   

   

      Checking ports status.......................................... OK   

   

      Checking Call Home Status...................................... Error   

   

      Email Server under Notification type: 'onSuccess/onFailure' is either   

   

      Not reachable or invalid.   

   

      Check if Email Server IP address: '10.1.111.100' is reachable and valid.   

   

      Email Server under Notification type: 'Primary' and 'Failover' are either   

   

      Not reachable or invalid.   

   

      Check if Email Server IP address: '10.1.111.100' and '10.1.111.100' are   

   

      Reachable and valid.   

   

          

   

      service@vplexMM:/opt/emc/connectemc> cat ConnectEMC_config.xml   

   

      <?xml version="1.0" encoding="UTF-8" standalone="no" ?>   

   

      <ConnectEMCConfig SchemaVersion="1.1.0">   

   

        <ConnectConfig Type="Email">   

   

          <Retries>7</Retries>   

   

          <Notification>Primary</Notification>   

   

          <Timeout>700</Timeout>   

   

          <Description></Description>   

   

          <BsafeEncrypt>no</BsafeEncrypt>   

   

          <IPProtocol>IPV4</IPProtocol>   

   

          <EmailServer>10.1.111.100</EmailServer>   

   

          <EmailAddress>emailalert@EMC.com</EmailAddress>   

   

          <EmailSender>VPlex_CKM00000000999@EMC.com</EmailSender>   

   

          <EmailFormat>ASCII</EmailFormat>   

   

          <EmailSubject>Call Home</EmailSubject>   

   

          <STARTTLS>no</STARTTLS>   

   

          <IncludeCallHomeData>no</IncludeCallHomeData>   

   

          <InsertBefore></InsertBefore>   

   

          <PreProcess></PreProcess>   

   

          <PostProcess></PostProcess>   

   

          <HeloParameter></HeloParameter>   

   

        </ConnectConfig>   

   

        <ConnectConfig Type="Email">   

   

          <Retries>7</Retries>   

   

          <Notification>Failover</Notification>   

   

          <Timeout>700</Timeout>   

   

          <Description></Description>   

   

          <BsafeEncrypt>no</BsafeEncrypt>   

   

          <IPProtocol>IPV4</IPProtocol>   

   

          <EmailServer>10.1.111.100</EmailServer>   

   

          <EmailAddress>emailalert@EMC.com</EmailAddress>   

   

          <EmailSender> VPlex_CKM00000000999@EMC.com</EmailSender>   

   

          <EmailFormat>ASCII</EmailFormat>   

   

          <EmailSubject>Call Home</EmailSubject>   

   

          <STARTTLS>no</STARTTLS>   

   

          <IncludeCallHomeData>no</IncludeCallHomeData>   

   

          <InsertBefore></InsertBefore>   

   

          <PreProcess></PreProcess>   

   

          <PostProcess></PostProcess>   

   

          <HeloParameter></HeloParameter>   

   

        </ConnectConfig>   

   

        <ConnectConfig Type="Email">   

   

          <Retries>7</Retries>   

   

          <Notification>onSuccess/onFailure</Notification>   

   

          <Timeout>700</Timeout>   

   

          <Description></Description>   

   

          <BsafeEncrypt>no</BsafeEncrypt>   

   

          <IPProtocol>IPV4</IPProtocol>   

   

          <EmailServer>10.1.111.100</EmailServer>   

   

          <EmailAddress>customer@genericemailaddress.com</EmailAddress>   

   

          <EmailSender>VPlex_CKM00000000999@EMC.com</EmailSender>   

   

          <EmailFormat>ASCII</EmailFormat>   

   

          <EmailSubject>Call Home</EmailSubject>   

   

          <STARTTLS>no</STARTTLS>   

   

          <IncludeCallHomeData>yes</IncludeCallHomeData>   

   

          <InsertBefore></InsertBefore>   

   

          <PreProcess></PreProcess>   

   

          <PostProcess></PostProcess>   

   

          <HeloParameter></HeloParameter>   

   

        </ConnectConfig>   

   

      </ConnectEMCConfig>   

   

          

   

      service@vplexMM:/var/log/ConnectEMC/logs> ping 10.1.111.100   

   

      PING 10.1.111.100 (10.1.111.100) 56(84) bytes of data.   

   

      --- 10.1.111.100 ping statistics ---   

   

      6 packets transmitted, 0 received, 100% packet loss, time 5010ms   

   

          

   

      service@vplexMM:~> telnet 10.1.111.100 25   

   

      Trying 10.1.111.100...   

   

      Connected to 10.1.111.100   

   

      Escape character is '^]'.   

   

      220 emc.com   

   

      helo localhost   

   

      250 emc.com   

   

      mail from: VPlex_CKM00000000999@EMC.com   

   

      250 2.1.0 Ok   

   

      rcpt to:customer@genericemailaddress.com   

   

      250 2.1.0 Ok   

   

          

   

      VPlexcli:/notifications/call-home> test   

   

      call-home test was successful.       
       
       
        As per the above information, this means that the customer is allowing the SMTP service on port "25" only and not the ICMP "ping".
   

   

      This error is expected and can be ignored once you verify that the test call home is working and appearing under /opt/emc/connectemc/archive   

   

          

   

      service@vplexMM:/opt/emc/connectemc/archive> ll   

   

      -rw-r----- 1 service users 2814 Jun 25 13:17 RSC_CKM00000000999_062518_011656000.xml   

   

      -rw-r----- 1 service users 2814 Jun 25 10:54 RSC_CKM00000000999_062518_105401000.xml   

   

      -rw-r----- 1 service users 2814 Jun 25 11:11 RSC_CKM00000000999_062518_111102000.xml   

   

      -rw-r----- 1 service users 2814 Jun 25 11:48 RSC_CKM00000000999_062518_114834000.xml   

                                                             

 

 

Cause:

 

 

   

      Checking call home status is part of the health-check -- full script which does the following:   

   

      1- Check the email server for each notification type in /opt/emc/connectemc/ConnectEMC_config.xml   

   

      2- Ping the server. If the server is not pingable for any reason (not reachable via network, server is shutdown, ICMP service is blocked via firewall, the <EmailServer> is a DNS name instead of the name in the ConnectEMC_config.xml file).   

   

      As a result, the command health-check --full script will fail and will show the following error:   

   

      Checking Call Home Status...................................... Error    

   

      The current healthcheck script checks if call home is enabled and generates a "Warning" state if it's disabled.     
          

   

      The healthcheck script also checks if call home has been functioning properly with several verifications such as: checking call homes have been generated; the call home emails have been sent successfully sent; or if SMTP server ping is alive.   

   

      If any of these verifications fail, the script's result will be flagged with an error as shown:   

   

      Checking Call Home Status...................................... Error   

                                                             

 

 

Resolution:

 

 

   

      After enabling the ICMP protocol on the firewall level between the VPLEX management server and their selected email server used (ESRS, customer's email server), the Call Home "Error" status is now clean:     
          

   

      VPlexcli:/> health-check --full   

   

      Configuration (CONF):   

   

      Checking VPlexCli connectivity to directors.................... OK   

   

      Checking Directors Commission.................................. OK   

   

      Checking Directors Communication Status........................ OK   

   

      Checking Directors Operation Status............................ OK   

   

      Checking Inter-director management connectivity................ OK   

   

      Checking ports status.......................................... OK   

   

      Checking Call Home Status...................................... OK   

   

      Checking Connectivity.......................................... OK   

   

      Checking COM Port Power Level.................................. OK   

   

      Checking Meta Data Backup...................................... OK   

   

      Checking Meta Data Slot Usage.................................. OK