After gaining some operational experience with the previously introduced Nagios plugin for Infortrend EonStor and EonStor DS storage arrays, some issues became apparent which needed to be addressed in an update. The updated version of the Nagios plugin check_infortrend.sh fixes the following issues:

1. Newer array firmware versions use different severity strings for event log entries. The Nagios plugin has been adapted to be able to cope with these.

2. Due to a limitation in the array firmware, event log entries available via SNMP are not cleared when the corresponding event log entries are cleared via RAIDWatch or SANWatch. They are only cleared during a controller reboot. A support case with Infortrend confirmed this and it is currently under investigation whether this feature could be added for future releases of the array firmware.

   In the meantime a workaround to address this behaviour has been added to the Nagios plugin. In order to not constantly get alarms about already addressed events, the date of the event will now – along with the already used severity of the event – be taken into consideration. By default only events within the past 60 minutes will be evaluated. The extent of time to look into the past can be overridden from the default with the new “-t <minutes>” command line option of the Nagios plugin. E.g. “-t 120” will evaluate events within the past 2 hours. You might need to adjust the definition of the check_infortrend_events command to suite your environment, e.g.:

   # check Infortrend ESDS cache status
   define command {
       command_name    check_infortrend_events
       command_line    $USER1$/check_infortrend.sh -H $HOSTNAME$ -C events -t <minutes>
   }

   There are several pitfalls here though:

   • Make sure the timeframe aligns with your Nagios configuration options max_check_attempts, normal_check_interval and retry_check_interval to actually allow non-normal events to trigger an alarm. Generally speaking, the cumulated timeframe of the three Nagios configuration options of your Check_IFT_Events service check must be smaller than the timeframe given in the definition of the check_infortrend_events command.

   • Make sure you set the date and time on the array to the correct values or preferably use a SNTP server. Also, make sure the Nagios server and the array use the same timezone, preferably UTC in both cases.

3. Not exactly an issue with the Nagios plugin per se, but related to the monitoring of Infortrend storage arrays in general, is an adapted SNMPTT configuration. Since all SNMP traps generated from events of Infortrend arrays are indiscriminately reported with the same OID, a bit more extensive parsing of the SNMP traps needs to occur. In order to achieve this, the following entries should be used in the snmptt.conf.infortrend SNMPTT configuration file:

   /opt/snmptt/conf/snmptt.conf.infortrend

   EVENT iftEventText .1.3.6.1.4.1.1714.0.1 "Status Events" Critical
   FORMAT The description of the event $*
   MATCH $*: ( \[Alert Condition\])
   MATCH $*: ( \[Critical\])
   MATCH $*: ( \[Critical Error\])
   MATCH $*: ( \[Error\])
   SDESC
   The description of the event
   Variables:
   EDESC
    
   EVENT iftEventText .1.3.6.1.4.1.1714.0.1 "Status Events" Warning
   FORMAT The description of the event $*
   MATCH $*: ( \[Warning Condition\])
   MATCH $*: ( \[Warning\])
   SDESC
   The description of the event
   Variables:
   EDESC
    
   EVENT iftEventText .1.3.6.1.4.1.1714.0.1 "Status Events" Normal
   FORMAT The description of the event $*
   MATCH $*: ( \[Information\])
   MATCH $*: ( \[Notification\])
   SDESC
   The description of the event
   Variables:
   EDESC

   and the SNMPTT daemon should be restarted.

If you happen to find any additional issues with monitoring your Infortrend systems with this Nagios plugin, please feel free to leave a comment or drop me a note via email.

Leave a comment…