|Article Number: 512581||Article Version: 3||Article Type: Break Fix|
Avamar Data Store Gen4T,Avamar,Avamar Server
Avamar release notes (7.5, 7.4) mentions the following:
As part of every Avamar release, the product is scanned for vulnerabilities using at least two common vulnerability assessments tools. This release was scanned with Foundstone and Nessus. The Avamar solution has also been scanned by various customers by using tools such as eEye Retina without issue. However, it is possible that the usage of other port/vulnerability scanners might cause disruption to normal operation of the Avamar server. Therefore, it might be necessary to disable scanning of the Avamar server if problems occur.
For Avamar Gen4T hardware, the vulnerability scanning results in the following two scenarios:
1. Avamar Administrator GUI shows the following notification message about "BMC Internal Watchdog Trip"
grep -i "BMC.*Watchdog" /var/log/messages
May 23 04:41:58 test-gen4t-02 ipmiutil: igetevent-gen4t: 0082 05/23/17 04:41:43 MAJ BMC BMC Internal HW Watchdog Trip #e1 Cold Reset Mode (asserted) 6f [00 ff ff]
Note: In the above output, this message was generated from the node with hostname "test-gen4t-02"
The above messages also generate an automatic dial home message.
2. Avamar Administrator GUI repeatedly displays old hardware messages whenever the Avamar hardware monitoring services are restarted due to the vulnerability scanning:
Following messages (or similar messages) can be seen in the /data01/cur/err.log (on the storage node)
Oct 12 00:19:45 test-gen4t-00 ipmiutil: igetevent-gen4t: 0044 06/09/17 21:15:40 MAJ BIOS Processor Errors #05 CPU Socket Socket:1 70 [82 01 ff] - ELOG(27) Empty CPU Socket: 00000001
Note that in the above messages, the actual error occurred on the date "06/09/17" but was repeated again on Sept 21st and Oct 12th
In both these cases, syslogs will have corresponding messages related to vulnerability scanning.
grep -i "sshd.*invalid user" /var/log/messages
Sep 21 02:50:04 test-gen4t-00 sshd: Failed none for invalid user NoSuchUser from 10.10.10.245 port 42885 ssh2
Sep 21 02:51:02 test-gen4t-00 sshd: Invalid user NoSuchUser from 10.10.10.245
Sep 21 02:51:02 test-gen4t-00 sshd: Failed none for invalid user NoSuchUser from 10.10.10.245 port 48180 ssh2
Oct 11 22:03:09 test-gen4t-00 sshd: Invalid user NoSuchUser from 10.10.10.245
Oct 11 22:03:09 test-gen4t-00 sshd: Failed none for invalid user NoSuchUser from 10.10.10.245 port 50579 ssh2
Oct 11 22:04:54 test-gen4t-00 sshd: Invalid user NoSuchUser from 10.10.10.245
Oct 11 22:04:54 test-gen4t-00 sshd: Failed none for invalid user NoSuchUser from 10.10.10.245 port 36524 ssh2
Vulnerability scanning causes Avamar Gen4T hardware monitoring services to crash and report false positive messages.
Avamar engineering team has recreated this issue and is working on a permanent fix for this issue.
It is recommended to whitelist the Avamar Servers from vulnerability scanning / security scanning / port scanning. That will ensure that the false positive messages are not generated