bityard Blog

2014-01-12 // Nagios Monitoring - Infortrend EonStor

Please be sure to also read the update Nagios Monitoring - Infortrend EonStor (Update) to this blog post.

We use several Infortrend EonStor and EonStor DS storage arrays as low-cost, bulk storage units in our datacenters. With check_infortrend.pl, check_infortrend and check_ift_{dev|hdd|ld}.pl there are already several Nagios plugin to monitor Infortrend EonStor storage arrays. Since i wanted a low overhead, shell-based plugin with support for performance data, i decided to write my own version check_infortrend.sh. In order to run the Nagios plugin, you need to have SNMP activated on the Infortrend storage array and a network connection from the Nagios system to the Infortrend device on port UDP/161 must be allowed.

The whole setup looks like this:

  1. Enable SNMP queries on the Infortrend storage array. Login via Telnet or SSH and navigate to: 

    -> view and edit Configuration parameters
   -> Communication Parameters
      -> Network Protocol Support
         -> SNMP - Disabled
            -> Enable SNMP Protocol?
               -> Yes

    Verify the port UDP/161 on the Infortrend device can be reached from the Nagios system.

  2. Optional: Enable SNMP traps to be sent to the Nagios system on the Infortrend storage array. This requires SNMPD and SNMPTT to be already setup on the Nagios system. Verify the port UDP/162 on the Nagios system can be reached from the Infortrend device.

  3. Download the Nagios plugin check_infortrend.sh and place it in the plugins directory of your Nagios system, in this example /usr/lib/nagios/plugins/: 

    $ mv -i check_infortrend.sh /usr/lib/nagios/plugins/
$ chmod 755 /usr/lib/nagios/plugins/check_infortrend.sh

  4. Define the following Nagios commands. In this example this is done in the file /etc/nagios-plugins/config/check_infortrend.cfg: 

    # check Infortrend ESDS cache status
define command {
    command_name    check_infortrend_cache
    command_line    $USER1$/check_infortrend.sh -H $HOSTNAME$ -C cache
}
# check Infortrend ESDS controller status
define command {
    command_name    check_infortrend_controller
    command_line    $USER1$/check_infortrend.sh -H $HOSTNAME$ -C controller
}
# check Infortrend ESDS disk status
define command {
    command_name    check_infortrend_disk
    command_line    $USER1$/check_infortrend.sh -H $HOSTNAME$ -C disk
}
# check Infortrend ESDS logicaldrive status
define command {
    command_name    check_infortrend_logicaldrive
    command_line    $USER1$/check_infortrend.sh -H $HOSTNAME$ -C logicaldrive
}
# check Infortrend ESDS logicalunit status
define command {
    command_name    check_infortrend_logicalunit
    command_line    $USER1$/check_infortrend.sh -H $HOSTNAME$ -C logicalunit
}
# check Infortrend ESDS event status
define command {
    command_name    check_infortrend_events
    command_line    $USER1$/check_infortrend.sh -H $HOSTNAME$ -C events
}

  5. Define a group of services in your Nagios configuration to be checked for each Infortrend system: 

    # check snmpd
define service {
    use                     generic-service
    hostgroup_name          infortrend
    service_description     Check_SNMPD
    check_command           check_snmpd
}
# check_infortrend_cache
define service {
    use                     generic-service-pnp
    hostgroup_name          infortrend
    service_description     Check_IFT_Cache
    check_command           check_infortrend_cache
}
# check_infortrend_controller
define service {
    use                     generic-service
    hostgroup_name          infortrend
    service_description     Check_IFT_Controller
    check_command           check_infortrend_controller
}
# check_infortrend_disk
define service {
    use                     generic-service-pnp
    hostgroup_name          infortrend
    service_description     Check_IFT_Disk
    check_command           check_infortrend_disk
}
# check_infortrend_logicaldrive
define service {
    use                     generic-service-pnp
    hostgroup_name          infortrend
    service_description     Check_IFT_LogicalDrive
    check_command           check_infortrend_logicaldrive
}
# check_infortrend_logicalunit
define service {
    use                     generic-service-pnp
    hostgroup_name          infortrend
    service_description     Check_IFT_LogicalUnit
    check_command           check_infortrend_logicalunit
}
# check_infortrend_events
define service {
    use                     generic-service
    hostgroup_name          infortrend
    service_description     Check_IFT_Events
    check_command           check_infortrend_events
}

    Replace generic-service with your Nagios service template. Replace generic-service-pnp with your Nagios service template that has performance data processing enabled.

  6. Define a service dependency to run the above checks only if the Check_SNMPD was run successfully: 

    # Infortrend SNMPD dependencies
define servicedependency {
    hostgroup_name                  infortrend
    service_description             Check_SNMPD
    dependent_service_description   Check_IFT_.*
    execution_failure_criteria      c,p,u,w
    notification_failure_criteria   c,p,u,w
}

  7. Define hosts in your Nagios configuration for each Infortrend device. In this example its named esds1: 

    define host {
    use         disk
    host_name   esds1
    alias       Infortrend Disk Storage 1
    address     10.0.0.1
    parents     parent_lan
}

    Replace disk with your Nagios host template for storage devices. Adjust the address and parents parameters according to your environment.

  8. Define a hostgroup in your Nagios configuration for all Infortrend devices. In this example it is named infortrend. The above checks are run against each member of the hostgroup: 

    define hostgroup {
    hostgroup_name  infortrend
    alias           Infortrend Disk Storages
    members         esds1
}

  9. Run a configuration check and if successful reload the Nagios process: 

    $ /usr/sbin/nagios3 -v /etc/nagios3/nagios.cfg
$ /etc/init.d/nagios3 reload

The new hosts and services should soon show up in the Nagios web interface.

If the optional step number 2 in the above list was done, SNMPTT also needs to be configured to be able to understand the incoming SNMP traps from Infortrend devices. This can be achieved by the following steps:

  1. Request a current version of the Infortrend SNMP MIB file from Infortrend support. In this example it's IFT_MIB_v1.40A02.mib. Transfer the file IFT_MIB_v1.40A02.mib to the Nagios server.

  2. Convert the SNMP MIB definitions in IFT_MIB_v1.40A02.mib into a format that SNMPTT can understand. 

    $ /opt/snmptt/snmpttconvertmib --in=MIB/IFT_MIB_v1.40A02.mib --out=/opt/snmptt/conf/snmptt.conf.infortrend
...
Done

Total translations:        1
Successful translations:   1
Failed translations:       0

  3. Edit the trap severity according to your requirements, e.g.: 

    $ vim /opt/snmptt/conf/snmptt.conf.infortrend

...
EVENT iftEventText .1.3.6.1.4.1.1714.2.1.1 "Status Events" Warning
...

  4. Add the new configuration file to be included in the global SNMPTT configuration and restart the SNMPTT daemon: 

    $ vim /opt/snmptt/snmptt.ini

...
[TrapFiles]
snmptt_conf_files = <<END
...
/etc/snmptt/conf.d/snmptt.conf.infortrend
...
END

$ /etc/init.d/snmptt reload

  5. Download the Nagios plugin check_snmp_traps.sh and place it in the plugins directory of your Nagios system, in this example /usr/lib/nagios/plugins/: 

    $ mv -i check_snmp_traps.sh /usr/lib/nagios/plugins/
$ chmod 755 /usr/lib/nagios/plugins/check_snmp_traps.sh

  6. Define the following Nagios command to check for SNMP traps in the SNMPTT database. In this example this is done in the file /etc/nagios-plugins/config/check_snmp_traps.cfg: 

    # check for snmp traps
define command {
    command_name    check_snmp_traps
    command_line    $USER1$/check_snmp_traps.sh -H $HOSTNAME$:$HOSTADDRESS$ -u <user> -p <pass> -d <snmptt_db>
}

    Replace user, pass and snmptt_db with values suitable for your SNMPTT database environment.

  7. Add another service in your Nagios configuration to be checked for each Infortrend device: 

    # check snmptraps
define service {
    use                     generic-service
    hostgroup_name          infortrend
    service_description     Check_SNMP_traps
    check_command           check_snmp_traps
}

  8. Optional: Define a serviceextinfo to display a folder icon next to the Check_SNMP_traps service check for each Infortrend device. This icon provides a direct link to the SNMPTT web interface with a filter for the selected host: 

    define serviceextinfo {
    hostgroup_name          infortrend
    service_description     Check_SNMP_traps
    notes                   SNMP Alerts
    #notes_url               http://<hostname>/nagios3/nagtrap/index.php?hostname=$HOSTNAME$
    #notes_url               http://<hostname>/nagios3/nsti/index.php?perpage=100&hostname=$HOSTNAME$
}

    Uncomment the notes_url depending on which web interface (nagtrap or nsti) is used. Replace hostname with the FQDN or IP address of the server running the web interface.

  9. Run a configuration check and if successful reload the Nagios process: 

    $ /usr/sbin/nagios3 -v /etc/nagios3/nagios.cfg
$ /etc/init.d/nagios3 reload

  10. Optional: If you're running PNP4Nagios v0.6 or later to graph Nagios performance data, you can use the check_infortrend_cache.php, check_infortrend_disk.php, check_infortrend_logicaldrive.php and check_infortrend_logicalunit.php PNP4Nagios templates to beautify the graphs. Download the PNP4Nagios templates check_infortrend_cache.php, check_infortrend_disk.php, check_infortrend_logicaldrive.php and check_infortrend_logicalunit.php and place them in the PNP4Nagios template directory, in this example /usr/share/pnp4nagios/html/templates/: 

    $ mv -i check_infortrend_*. /usr/share/pnp4nagios/html/templates/
$ chmod 644 /usr/share/pnp4nagios/html/templates/check_infortrend_*.php

    The following image shows an example of what the PNP4Nagios graphs look like for a Infortrend EonStor unit:

    PNP4Nagios graph for the cache usage in a Infortrend EonStor unit
    PNP4Nagios graph for the physical disk throughput in a Infortrend EonStor unit
    PNP4Nagios graphs for the logical disk throughput usage in a Infortrend EonStor unit
    PNP4Nagios graph for a temperature sensor in a Infortrend EonStor unit
    PNP4Nagios graph for a voltage sensor in a Infortrend EonStor unit

All done, you should now have a complete Nagios-based monitoring solution for your Infortrend EonStor systems.

2014-01-12 @ 15:01 | written by Frank Fegert (XING)
Tags:

Comments

Sylvain
No. 1 @ 2016/05/12 14:50

Hi,

I tried your useful plugin check_infortrend.sh, but in our context we only have 8 HDD. In this case, when i check the LD, I get a critical status.

Could you help me tweak your code for this case ?

I just want to check 8 HDD if these 8 HHD are good then return OK.

Thank in advance for your help.

Sincerely

Sylvain

» Reply
Frank Fegert
No. 2 @ 2016/05/13 06:39

Hello Sylvain,

thank you very much for your nice feedback! Unfortunately i currently have no access to a Infortrend storage system anymore. In order to debug this remotely can you please - preferably via email - send me the output of the following command: 

snmpwalk -On -v 2c -c $COMMUNITY $HOST .1.3.6.1.4.1.1714.1.1.2

Thanks & best regards,

Frank

» Reply

Leave a comment…

X G V X K
  • E-Mail address will not be published.
  • Formatting:
    //italic//  __underlined__
    **bold**  ''preformatted''
  • Links:
    [[http://example.com]]
    [[http://example.com|Link Text]]
  • Quotation:
    > This is a quote. Don't forget the space in front of the text: "> "
  • Code:
    <code>This is unspecific source code</code>
    <code [lang]>This is specifc [lang] code</code>
    <code php><?php echo 'example'; ?></code>
    Available: html, css, javascript, bash, cpp, …
  • Lists:
    Indent your text by two spaces and use a * for
    each unordered list item or a - for ordered ones.
This website uses cookies. By using the website, you agree with storing cookies on your computer. Also you acknowledge that you have read and understand our Privacy Policy. If you do not agree leave the website. More information about cookies