Recoverpoint: "Error: An internal error has occurred" during RP Log collection

Article Number: 497110                              Article Version: 3                               Article Type: Break Fix


Product:

RecoverPoint SE,RecoverPoint SE 4.0,RecoverPoint SE 4.1,RecoverPoint SE 4.4,RecoverPoint SE 5.0


Issue:

"Error: An internal error has occurred. Please contact Customer Support."

or "internal_error"

was logged on Splitter Connectivity Status during RP Log collection

in RP/SE VNX Splitter configuration.

 

Example:

System component collection results

|---------------------------------------------------------------------------|

| Source          | Cluster         | Result                                |

|-----------------|-----------------|---------------------------------------|

| RPA 1           | Prod_Cluster    | Action succeeded                      |

| RPA 2           | Prod_Cluster    | Action succeeded                      |

| RPA 1           | DR_Cluster      | Action succeeded                      |

| RPA 2           | DR_Cluster      | Action succeeded                      |

| drvnx-A         | DR_Cluster      | Error: An internal error has occurred.|

|                 |                 | Please contact Customer Support.      |

| prodvnx-A       | Prod_Cluster    | Error: An internal error has occurred.|

|                 |                 | Please contact Customer Support.      |

| drvnx-B         | DR_Cluster      | internal_error                        |

| prodvnx-B       | Prod_Cluster    | Error: An internal error has occurred.|

|                 |                 | Please contact Customer Support.      |

|-----------------|-----------------|---------------------------------------|

 

 

Cause:

VNX splitter Log collection process took longer than default timeout specify by RP

Default Timeout params
tweak.params.infra:t_clariionConfTimeout = 1800 (seconds)
tweak.params.scsi_initiator:t_clariionMgr_logCollectionTimeout = 1200 (seconds)
tweak.params.scsi_initiator:t_clariionMgrDeadlockConfig = DLConfig(60, 2, 2000, 0, true) (seconds)


Note: Issue may impact one SP while the other SP was fine

 

Change:

VNX Splitter Log collection took longer to produce or Array was busy

 

Resolution:

Workaround:

 

Increase timeout value

1. t_clariionMgr_logCollectionTimeout to 3600 (1 hr) from 1200 via tweak.params.scsi_initiator
2. t_clariionMgrDeadlockConfig to DLConfig(60, 2, 3800, 0, true) (1 hr) via tweak.params.scsi_initiator
3. t_clariionConfTimeout to 3600 from 1800 seconds via tweak.params.infra

Putty SSH to all RPAs with boxmgmt account
[2] Setup ->[8] Advanced options ->[4] Run script
apply the following script and reboot RPA one at a time

MGY4YmY2Y2EzODQ4N2NlMGYzMWE0MDI0NWM5N2ZlOGIKdW5saW1pdGVkCm5vdF9yZXN0cmljdGVk
ClRoZSBpZCBvZiB0aGUgc2NyaXB0IGlzOjQzODkyCkNoYW5naW5nIDQgdHdlYWtzIC0gdF9JQ19t
YXhDbGFyaWlvblNwbGl0dGVySW5mb1NpemUgdG8gMTIwMCBmcm9tIDMwMCwgdF9jbGFyaWlvbk1n
cl9sb2dDb2xsZWN0aW9uVGltZW91dCB0byAzNjAwIGZyb20gMTIwMCwgdF9jbGFyaWlvbk1nckRl
YWRsb2NrQ29uZmlnIHRvIERMQ29uZmlnKDYwLCAyLCAyMDAwLCAwLCB0cnVlKSBmcm9tIERMQ29u
ZmlnKDYwLCAyLCAxMjAwLCAwLCB0cnVlKSBhbmQgdF9jbGFyaWlvbkNvbmZUaW1lb3V0IHRvIDM2
MDAgZnJvbSAxODAwLgpZYWtpciBNb3JkZWhheQojIS9iaW4vYmFzaApjZCAvaG9tZS9rb3Mva2Jv
eC9zcmMvaW5pdGlhbGl6YXRpb24vdHdlYWtfcGFyYW1zCmlmIFtbIGB6Z3JlcCAidF9JQ19tYXhD
bGFyaWlvblNwbGl0dGVySW5mb1NpemUgPSAzMDAiIHR3ZWFrLnBhcmFtcy5pbmZyYWAgJiYgYHpn
cmVwICJ0X2NsYXJpaW9uTWdyX2xvZ0NvbGxlY3Rpb25UaW1lb3V0ID0gMTIwMCIgdHdlYWsucGFy
YW1zLnNjc2lfaW5pdGlhdG9yYCAmJiBgemdyZXAgInRfY2xhcmlpb25NZ3JEZWFkbG9ja0NvbmZp
ZyA9IERMQ29uZmlnKDYwLCAyLCAxMjAwLCAwLCB0cnVlKSIgdHdlYWsucGFyYW1zLnNjc2lfaW5p
dGlhdG9yYCAmJiBgemdyZXAgInRfY2xhcmlpb25Db25mVGltZW91dCA9IDE4MDAiIHR3ZWFrLnBh
cmFtcy5pbmZyYWAgXV07IHRoZW4KI0NoYW5nZSB0X0lDX21heENsYXJpaW9uU3BsaXR0ZXJJbmZv
U2l6ZSB0byAxMjAwCiAgICAgICAgY3AgLWYgL2hvbWUva29zL2tib3gvc3JjL2luaXRpYWxpemF0
aW9uL3R3ZWFrX3BhcmFtcy90d2Vhay5wYXJhbXMuaW5mcmEgL2hvbWUva29zL2tib3gvc3JjL2lu
aXRpYWxpemF0aW9uL3R3ZWFrX3BhcmFtcy9iYWNrdXBfdHdlYWsucGFyYW1zLmluZnJhCiAgICAg
ICAgc2VkIC1pICdzL3RfSUNfbWF4Q2xhcmlpb25TcGxpdHRlckluZm9TaXplID0gMzAwL3RfSUNf
bWF4Q2xhcmlpb25TcGxpdHRlckluZm9TaXplID0gMTIwMC9nJyAvaG9tZS9rb3Mva2JveC9zcmMv
aW5pdGlhbGl6YXRpb24vdHdlYWtfcGFyYW1zL3R3ZWFrLnBhcmFtcy5pbmZyYQojQ2hhbmdlIHRf
Y2xhcmlpb25NZ3JfbG9nQ29sbGVjdGlvblRpbWVvdXQgdG8gMzYwMAogICAgICAgIGNwIC1mIC9o
b21lL2tvcy9rYm94L3NyYy9pbml0aWFsaXphdGlvbi90d2Vha19wYXJhbXMvdHdlYWsucGFyYW1z
LnNjc2lfaW5pdGlhdG9yIC9ob21lL2tvcy9rYm94L3NyYy9pbml0aWFsaXphdGlvbi90d2Vha19w
YXJhbXMvYmFja3VwX3R3ZWFrLnBhcmFtcy5zY3NpX2luaXRpYXRvcgogICAgICAgIHNlZCAtaSAn
cy90X2NsYXJpaW9uTWdyX2xvZ0NvbGxlY3Rpb25UaW1lb3V0ID0gMTIwMC90X2NsYXJpaW9uTWdy
X2xvZ0NvbGxlY3Rpb25UaW1lb3V0ID0gMzYwMC9nJyAvaG9tZS9rb3Mva2JveC9zcmMvaW5pdGlh
bGl6YXRpb24vdHdlYWtfcGFyYW1zL3R3ZWFrLnBhcmFtcy5zY3NpX2luaXRpYXRvcgojQ2hhbmdl
IHRfY2xhcmlpb25NZ3JEZWFkbG9ja0NvbmZpZyB0byAiRExDb25maWcoNjAsIDIsIDIwMDAsIDAs
IHRydWUpIgogICAgICAgICMgRG9uJ3QgbWFrZSBhIGJhY2t1cCBzaW5jZSB3ZSBkaWQgaW4gbGlu
ZSA4CiAgICAgICAgc2VkIC1pICdzL3RfY2xhcmlpb25NZ3JEZWFkbG9ja0NvbmZpZyA9IERMQ29u
ZmlnKDYwLCAyLCAxMjAwLCAwLCB0cnVlKS90X2NsYXJpaW9uTWdyRGVhZGxvY2tDb25maWcgPSBE
TENvbmZpZyg2MCwgMiwgMjAwMCwgMCwgdHJ1ZSkvZycgL2hvbWUva29zL2tib3gvc3JjL2luaXRp
YWxpemF0aW9uL3R3ZWFrX3BhcmFtcy90d2Vhay5wYXJhbXMuc2NzaV9pbml0aWF0b3IKI0NoYW5n
ZSB0X2NsYXJpaW9uQ29uZlRpbWVvdXQgdG8gMzYwMAogICAgICAgICMgRG9uJ3QgbWFrZSBhIGJh
Y2t1cCBzaW5jZSB3ZSBkaWQgaW4gbGluZSA1CiAgICAgICAgc2VkIC1pICdzL3RfY2xhcmlpb25D
b25mVGltZW91dCA9IDE4MDAvdF9jbGFyaWlvbkNvbmZUaW1lb3V0ID0gMzYwMC9nJyAvaG9tZS9r
b3Mva2JveC9zcmMvaW5pdGlhbGl6YXRpb24vdHdlYWtfcGFyYW1zL3R3ZWFrLnBhcmFtcy5pbmZy
YQoJCWVjaG8gIlJQQSBtdXN0IGJlIHJlYm9vdGVkIHRvIGFwcGx5IGNoYW5nZXMuIgplbHNlCiAg
ICAgICAgZWNobyAiVGhpcyBzY3JpcHQgd29uJ3Qgd29yayEiCiAgICAgICAgZXhpdApmaQo=#