bityard Blog

2018-11-18 // SAP Cloud Connector - Configuring Multiple Local Administrative Users

Out of the box, the SAP Cloud Connector does not support the creation of multiple local users for administration purposes through its WebUI. The standard and documented way provides only the use of LDAP in conjunction with an external user directory in case multiple administrative users are necessary. This in turn introduces an unnecessary overhead and external dependency. It also has rather strict limitations on the names of the groups (admin or sccadmin) that can used to authorize users for administrative tasks in the SAP Cloud Connector. There is a simple workaround for this limitation, which will be described in this blog post.

Since the SAP Cloud Connector uses an embedded Tomcat servlet container as an application server, it also uses some of the infrastructure provided by Tomcat. This includes the local file-based user directory. In order to use multiple, distinctly named administrative users, they simply need to be manually added to this local file-based user directory.

The first step though, is to create a password hash that is compatible with the Tomcat servlet container. This can be accomplished with the use of the Tomcat command line tool digest.sh or digest.bat. Depending on your operating system this tool might have other names, e.g. /usr/bin/tomcat-digest on RedHat Enterprise Linux. On any system that has said command line tool available (e.g. on a RHEL 7 system with the RPM tomcat installed), run the following command: 

user@host:$ /usr/bin/tomcat-digest -a sha-256 '<PASSWORD>'

Substitute the placeholder <PASSWORD> with the actual passwort that will be used for the new administrative user. The output of the above command will be in the form of <PASSWORD>:<HASH>. For the following step, only the <HASH> part of the output will be necessary.

The next step is to manually add the new adminstrative user to the local file-based user directory of the embedded Tomcat. This is achieved by editing the file config/users.xml in your SAP Cloud Connector installation. In this example the installation was performed in the standard path for Linux systems, which is /opt/sap/scc. Open the file config/users.xml in an editor of your choice: 

root@host:# vi /opt/sap/scc/config/users.xml

Alter its contents by adding a new line, like shown in the following example:

/opt/sap/scc/config/users.xml
<?xml version='1.0' encoding='utf-8'?>
<tomcat-users>
  <role rolename="admin"/>
  <group groupname="initial"/>
  <user username="Administrator" password="********" roles="admin"/>
  <user username="<USERNAME>" password="<HASH>" roles="admin"/>
  [...]
</tomcat-users>

Substitute the placeholder <USERNAME> with the desired username of the new administrative user and substitute the placeholder <HASH> with the password hash generated above.

Restart the SAP Cloud Connector.

The newly created user should now be able to log in via the WebUI of the SAP Cloud Connector.

This kind of local file-based user directory is fairly simple to manage and to understand. I keep wondering why SAP would not document or even integrate this kind of user management in the WebUI of the SAP Cloud Connector. Another issue is the use of only one authorization role (admin or sccadmin), which is independent of the particular user directory used. This authorization role grants full administrative rights to the SAP Cloud Connector for a given user. A slightly more differentiated authorization scheme with roles for e.g. operations (connector restart and monitoring, no configuration changes) or monitoring (only monitoring, no connector restart, no configuration changes) purposes would be much more suitable for purposes in an enterprise environment.

0 Comments | 2018-11-18 written by Frank Fegert (XING) | Permanentlink
Tags:

2018-02-10 // Brocade Fabric OS Authentication Failure with SSH Public Key

With the update to Fabric OS v7.4.1d on Brocade fibre channel SAN switches, the CLI login via SSH public key authentication will sometimes be broken for administrative users. This blog post describes a manual workaround which can be used in order to temporarily correct this issue without the immediate need for another Fabric OS update.

During the preparation phase for migrating from our aging Brocade 5100 and 5300 Gen4 fibre channel SAN switches to the shiny new Brocade G620 Gen6 fibre channel SAN Switches, we needed to update the Fabric OS on the old switches to a v7.4.x version. Due to compatibility and support constraints with the IBM SAN Volume Controller (SVC), we decided to go with the Fabric OS v7.4.1d version.

After the successful Fabric OS update, the CLI login via SSH public key authentication was broken for some, but not all users with admin level priviledges on some but not all switches. A re-upload of the SSH public key for those users with the sshUtil importpubkey command didn't solve the issue. Debugging this further with a strace attached to the SSH daemon process on an affected switch revealed why the SSH public key authentication was failing: 

[...]
[pid 27941] connect(8, {sa_family=AF_FILE, path="/dev/log"}, 16) = -1 EPROTOTYPE (Protocol wrong type for socket)
[pid 27941] close(8)                    = 0
[pid 27941] socket(PF_FILE, SOCK_STREAM, 0) = 8
[pid 27941] fcntl64(8, F_SETFD, FD_CLOEXEC) = 0
[pid 27941] connect(8, {sa_family=AF_FILE, path="/dev/log"}, 16) = 0
[pid 27941] send(8, "<39>Feb  1 22:07:00 sshd[27941]: debug1: trying public key file /fabos/users/admin/.ssh/authorized_keys.<USERNAME>\0", 117, MSG_NOSIGNAL) = 117
[pid 27941] close(8)                    = 0
[pid 27941] open("/fabos/users/admin/.ssh/authorized_keys.<USERNAME_2>", O_RDONLY|O_NONBLOCK|O_LARGEFILE) = -1 EACCES (Permission denied)
[...]

This was done by using the root account on the Brocade switch. The same account was also used for the following research and the temporary workaround derived from this. Beware that using the root account on Brocade switches might have serious implications on the warranty or support for the devices. Be extra careful what you are doing as root on the Brocade switch, since it might easily affect the operational status of the device.

The last line from the above snippet of the strace output show, that the cause of the issue with SSH public key authentication was in the permissions of the users authorized_keys file. Looking at the permissions of the directory containing the file and the file itself showed: 

switch:FID128:root> cd /fabos/users/admin/
switch:FID128:root> ls -al
total 28
drwxr-xr-x   4 root     admin        4096 Jan 23 12:36 ./
drwxr-xr-x  12 root     sys          4096 Jul 15  2016 ../
-rw-r--r--   1 root     admin         507 Jul 15  2016 .bash_logout
-rw-r--r--   1 root     admin          27 Jul 15  2016 .inputrc
-rw-r--r--   1 root     admin        1275 Jul 15  2016 .profile
drwxr-xr-x   2 root     admin        4096 Feb  1 22:54 .ssh/
drwxrwxrwx   3 root     sys          4096 Aug 11  2011 .terminfo/

switch:FID128:root> cd .ssh
switch:FID128:root> pwd
/fabos/users/admin/.ssh

switch:FID128:root> ls -al
total 44
drwxr-xr-x   2 root     admin        4096 Feb  1 22:54 ./
drwxr-xr-x   4 root     admin        4096 Jan 23 12:36 ../
-rw-r--r--   1 root     admin       10240 Feb  1 22:54 authorizedKeys.tar
-rw-------   1 root     root          408 Jan 23 12:43 authorized_keys
-rw-r--r--   1 root     admin         755 Dec  8 11:13 authorized_keys.<USERNAME_1>
-rw-------   1 root     admin        1230 Feb  1 22:54 authorized_keys.<USERNAME_2>
-rw-------   1 root     root          408 Mar 22  2016 authorized_keys.<USERNAME_3>
-rw-r--r--   1 root     root          605 Sep 19 11:06 authorized_keys.<USERNAME_4>
-rw-r--r--   1 root     admin         134 Jul 15  2016 environment

switch:FID128:root> tar tvf authorizedKeys.tar
-rw-r--r-- root/admin      755 2017-12-08 11:13:06 authorized_keys.<USERNAME_1>
-rw------- root/admin     1230 2018-02-01 22:54:01 authorized_keys.<USERNAME_2>
-rw------- root/root       408 2016-03-22 10:12:37 authorized_keys.<USERNAME_3>
-rw-r--r-- root/root       606 2017-09-19 11:06:10 authorized_keys.<USERNAME_4>

switch:FID128:root> tar tvf /mnt/fabos/users/admin/.ssh/authorizedKeys.tar
-rw-r--r-- root/admin      755 2017-12-08 11:13:06 authorized_keys.<USERNAME_1>
-rw------- root/admin     1230 2018-02-01 22:54:01 authorized_keys.<USERNAME_2>
-rw------- root/root       408 2016-03-22 10:12:37 authorized_keys.<USERNAME_3>
-rw-r--r-- root/root       606 2017-09-19 11:06:10 authorized_keys.<USERNAME_4>

The permissions on some, but not all of the authorized_keys.<USERNAME_*> files were being too restrictive, since the SSH daemon was trying to read them as an effective user of the admin group. An immediate fix for this issue was to alter the permissions on the authorized_keys.<USERNAME_*> files in order to allow the admin group to read the content of the files: 

switch:FID128:root> cd /fabos/users/admin/
switch:FID128:root> chmod 640 authorized_keys.*
switch:FID128:root> chown root:admin authorized_keys.*
switch:FID128:root> tar cpf authorizedKeys.tar authorized_keys.*

switch:FID128:root> cd /mnt/fabos/users/admin/.ssh/
switch:FID128:root> chmod 640 authorized_keys.*
switch:FID128:root> chown root:admin authorized_keys.*
switch:FID128:root> tar cpf authorizedKeys.tar authorized_keys.*

Again looking at the permissions of the directory containing the authorized_keys.<USERNAME_*> files and the files itself showed now: 

switch:FID128:root> cd /fabos/users/admin/
switch:FID128:root> ls -la
total 44
drwxr-xr-x   2 root     admin        4096 Feb  1 22:54 ./
drwxr-xr-x   4 root     admin        4096 Jan 23 12:36 ../
-rw-r--r--   1 root     admin       10240 Feb  9 06:59 authorizedKeys.tar
-rw-------   1 root     root          408 Jan 23 12:43 authorized_keys
-rw-r-----   1 root     admin         755 Dec  8 11:13 authorized_keys.<USERNAME_1>
-rw-r-----   1 root     admin        1230 Feb  1 22:54 authorized_keys.<USERNAME_2>
-rw-r-----   1 root     admin         408 Mar 22  2016 authorized_keys.<USERNAME_3>
-rw-r-----   1 root     admin         605 Sep 19 11:06 authorized_keys.<USERNAME_4>
-rw-r--r--   1 root     admin         134 Jul 15  2016 environment

switch:FID128:root> tar tvf /fabos/users/admin/.ssh/authorizedKeys.tar
-rw-r----- root/admin      755 2017-12-08 11:13:06 authorized_keys.<USERNAME_1>
-rw-r----- root/admin     1230 2018-02-01 22:54:01 authorized_keys.<USERNAME_2>
-rw-r----- root/admin      408 2016-03-22 10:12:37 authorized_keys.<USERNAME_3>
-rw-r----- root/admin      606 2017-09-19 11:06:10 authorized_keys.<USERNAME_4>

switch:FID128:root> cd /mnt/fabos/users/admin/.ssh/
switch:FID128:root> ls -la
total 44
drwxr-xr-x   2 root     admin        4096 Feb  1 22:54 ./
drwxr-xr-x   4 root     admin        4096 Jan 23 12:50 ../
-rw-r--r--   1 root     admin       10240 Feb  1 22:54 authorizedKeys.tar
-rw-------   1 root     root          408 Jan 23 12:43 authorized_keys
-rw-r-----   1 root     admin         755 Dec  8 11:13 authorized_keys.<USERNAME_1>
-rw-r-----   1 root     admin        1230 Feb  1 22:54 authorized_keys.<USERNAME_2>
-rw-r-----   1 root     admin         408 Mar 22  2016 authorized_keys.<USERNAME_3>
-rw-r-----   1 root     admin         605 Sep 19 11:06 authorized_keys.<USERNAME_4>
-rw-r--r--   1 root     admin         134 Jan 23 12:50 environment

switch:FID128:root> tar tvf /mnt/fabos/users/admin/.ssh/authorizedKeys.tar
-rw-r----- root/admin      755 2017-12-08 11:13:06 authorized_keys.<USERNAME_1>
-rw-r----- root/admin     1230 2018-02-01 22:54:01 authorized_keys.<USERNAME_2>
-rw-r----- root/admin      408 2016-03-22 10:12:37 authorized_keys.<USERNAME_3>
-rw-r----- root/admin      606 2017-09-19 11:06:10 authorized_keys.<USERNAME_4>

With the corrected permissions on the authorized_keys.<USERNAME_*> files, CLI login via SSH public key authentication was now possible again.

Unfortunately this is only a temporary workaround, since the next upload of a SSH public key with the sshUtil importpubkey command will likely set the wrong permissions on the newly created or replaced authorized_keys.<USERNAME_*> file. This is due to the root cause of the issue actually being with the sshUtil importpubkey command. The snippet of a strace output show below was captured from a running sshUtil importpubkey command: 

[...]
chdir("/fabos/users/admin/.ssh")        = 0
[...]
[pid 10611] execve("/bin/cat", ["cat", "<USERNAME_2>_brocade_dsa.pub"], [/* 45 vars */]) = 0
[...]
[pid 10612] execve("/bin/chmod", ["/bin/chmod", "600", "authorized_keys.<USERNAME_2>"], [/* 45 vars */]) = 0
[...]
[pid 10612] lstat64("authorized_keys.<USERNAME_2>", {st_mode=S_IFREG|0640, st_size=1230, ...}) = 0
[pid 10612] chmod("authorized_keys.<USERNAME_2>", 0600) = 0
[...]
[pid 10613] execve("/bin/cp", ["cp", "-f", "authorized_keys.<USERNAME_2>", "/mnt/fabos/users/admin/.ssh/"], [/* 45 vars */]) = 0
[...]
[pid 10613] chmod("/mnt/fabos/users/admin/.ssh/authorized_keys.<USERNAME_2>", 0100600) = 0
[...]
[pid 10618] execve("/bin/tar", ["tar", "-cf", "authorizedKeys.tar", "authorized_keys.<USERNAME_1>", "authorized_keys.<USERNAME_2>", "authorized_keys.<USERNAME_3>"], [/* 45 vars */]) = 0
[...]

The /bin/chmod command on the third line of the above strace output shows that the file permission for the authorized_keys.<USERNAME_2> file is mistakenly set to 600 (-rw-------) instead to at least 640 (-rw-r-----). Exactly why this is sometimes happening can't be further analyzed, since the source code to the sshUtil command is not available.

A permanent resolution to this issue will be to update to at least Fabric OS v7.4.1e. The Release Notes for Fabric OS v7.4.1e indicate this in the following known defect:

Defect ID: DEFECT000616486
Technical Severity: Medium
Probability: Medium
Product: Brocade Fabric OS
Technology Group: Security
Reported In Release: FOS7.4.1
Technology: SSH - Secure Shell
Symptom: Unable to authenticate an SSH session after importing public key to switch.
Condition: This is encountered by admin level users on a switch running Fabric OS v7.4.1d
0 Comments | 2018-02-10 written by Frank Fegert (XING) | Permanentlink
Tags:

2015-11-13 // QLogic iSCSI HBA and Limitations in Bi-Directional Authentication

In the past the QLogic QConvergeConsole (qaucli) was used as an administration tool for the hardware initiator part of the QLogic 4000 and QLogic 8200 Series network adapters and iSCSI HBAs. Unfortunately this tool was only supported on the so-called “enterprise Linux distributions” like RHEL and SLES. If you were running any other Linux distribution like e.g. Debian or even one of the BSD distributions you were out of luck.

Thankfully QLogic addressed this support issue indirectly, by first announcing and since then by actually moving from a IOCTL based management method towards the Open-iSCSI based management method via the iscsiadm command. The announcement QLogic iSCSI Solution for Transitioning to the Open-iSCSI Model and the User's Guide IOCTL to Open-iSCSI Interface can be found at the QLogic web site.

While trying to test and use the new management method for the hardware initiator via the Open-iSCSI iscsiadm command, i soon ran into the issue that the packaged version of Open-iSCSI, which is shipped with Debian Wheezy, is based on the last stable release v2.0.873 from Open-iSCSI and is thus hopelessly out of date. The Open-iSCSI package shipped with Debian Jessie is a bit better, since it's already based on a newer version from the projects GitHub repository. Still, the Git commit used there dates back to August 23rd of 2013, which is also fairly old. After updating my system to Debian Jessie, i soon decided to rebuild the Open-iSCSI package from a much more recent version from the projects GitHub repository. With this, the management of the QLogic hardware initiators worked very well via the Open-iSCSI iscsiadm command and its now enhanced host mode.

In the host mode there are now three sub-modes chap, flashnode, stats. See man iscsiadm and /usr/share/doc/open-iscsi/README.gz for more details on how to use them. By first calling the host mode without any sub-mode, iscsiadm prints a list of available iSCSI HBAs along with the host number – shown in the first pair of square brackets – associated with each host by the OS kernel: 

root@host:~$ iscsiadm -m host
qla4xxx: [1] 10.0.0.5,[84:8f:69:35:fc:70],<empty> iqn.2000-04.com.qlogic:isp8214.000e1e37da2c.4
qla4xxx: [2] 10.0.0.6,[84:8f:69:35:fc:71],<empty> iqn.2000-04.com.qlogic:isp8214.000e1e37da2d.5

The host number – in the above example 1 and 2 – is used in the following examples showing the three sub-modes:

  • The stats sub-mode displays various statistics values, like e.g. TCP/IP and iSCSI sessions, of the given HBA port: 

    root@host:~$ iscsiadm -m host -H 1 -C stats

Host Statistics:
    mactx_frames: 2351750
    mactx_bytes: 233065914
    mactx_multicast_frames: 1209409
    mactx_broadcast_frames: 0
    mactx_pause_frames: 0
    mactx_control_frames: 0
    mactx_deferral: 0
    mactx_excess_deferral: 0
    mactx_late_collision: 0
    mactx_abort: 0
    mactx_single_collision: 0
    mactx_multiple_collision: 0
    mactx_collision: 0
    mactx_frames_dropped: 0
    mactx_jumbo_frames: 0
    macrx_frames: 4037613
    macrx_bytes: 1305799553
    macrx_unknown_control_frames: 0
    macrx_pause_frames: 0
    macrx_control_frames: 0
    macrx_dribble: 0
    macrx_frame_length_error: 0
    macrx_jabber: 0
    macrx_carrier_sense_error: 0
    macrx_frame_discarded: 0
    macrx_frames_dropped: 2409752
    mac_crc_error: 0
    mac_encoding_error: 0
    macrx_length_error_large: 0
    macrx_length_error_small: 0
    macrx_multicast_frames: 0
    macrx_broadcast_frames: 0
    iptx_packets: 1694187
    iptx_bytes: 112412836
    iptx_fragments: 0
    iprx_packets: 1446806
    iprx_bytes: 721191324
    iprx_fragments: 0
    ip_datagram_reassembly: 0
    ip_invalid_address_error: 0
    ip_error_packets: 0
    ip_fragrx_overlap: 0
    ip_fragrx_outoforder: 0
    ip_datagram_reassembly_timeout: 0
    ipv6tx_packets: 0
    ipv6tx_bytes: 0
    ipv6tx_fragments: 0
    ipv6rx_packets: 0
    ipv6rx_bytes: 0
    ipv6rx_fragments: 0
    ipv6_datagram_reassembly: 0
    ipv6_invalid_address_error: 0
    ipv6_error_packets: 0
    ipv6_fragrx_overlap: 0
    ipv6_fragrx_outoforder: 0
    ipv6_datagram_reassembly_timeout: 0
    tcptx_segments: 1694187
    tcptx_bytes: 69463008
    tcprx_segments: 1446806
    tcprx_byte: 692255204
    tcp_duplicate_ack_retx: 8
    tcp_retx_timer_expired: 28
    tcprx_duplicate_ack: 0
    tcprx_pure_ackr: 0
    tcptx_delayed_ack: 247594
    tcptx_pure_ack: 247710
    tcprx_segment_error: 0
    tcprx_segment_outoforder: 0
    tcprx_window_probe: 0
    tcprx_window_update: 2248673
    tcptx_window_probe_persist: 0
    ecc_error_correction: 0
    iscsi_pdu_tx: 1446486
    iscsi_data_bytes_tx: 30308
    iscsi_pdu_rx: 1446510
    iscsi_data_bytes_rx: 622721801
    iscsi_io_completed: 253632
    iscsi_unexpected_io_rx: 0
    iscsi_format_error: 0
    iscsi_hdr_digest_error: 0
    iscsi_data_digest_error: 0
    iscsi_sequence_error: 0

  • The chap sub-mode displays and alters a table containing authentication information. Calling this sub-mode with the -o show option displays the current contents of the table: 

    root@host:~$ iscsiadm -m host -H 1 -C chap -o show
# BEGIN RECORD 2.0-873
host.auth.tbl_idx = 0
host.auth.username_in = <empty>
host.auth.password_in = <empty>
# END RECORD
# BEGIN RECORD 2.0-873
host.auth.tbl_idx = 1
host.auth.username = <empty>
host.auth.password = <empty>
# END RECORD

[...]

    Why show isn't the default option in the context of the chap sub-mode, like it is in many other iscsiadm modes and sub-modes is something i haven't quite understood yet. Maybe it's a security measure to not accidentially divulge sensitive information, maybe it has just been overlooked by the developers.

    Usually, there are already two initial records with the indexes 0 and 1 present on a HBA. As shown in the example above, each authentication record consists of three parameters. A record index host.auth.tbl_idx to reference it, a username host.auth.username or host.auth.username_in and a password host.auth.password or host.auth.password_in. Depending on whether the record is used for outgoing authentication of an initiator against a target or the other way around for incoming authentication of a target against an initiator, the parameter pairs username/password or username_in/password_in are used. Apparently both types of parameter pairs – incoming and outgoing – cannot be mixed together in a single record. My guess is that this isn't a limitation in Open-iSCSI, but rather a limitation in the specification and/or of the underlying hardware.

    New authentication records can be added with the -o new option: 

    root@host:~$ iscsiadm -m host -H 1 -C chap -x 2 -o new
    root@host:~$ iscsiadm -m host -H 1 -C chap -o show
# BEGIN RECORD 2.0-873
host.auth.tbl_idx = 0
host.auth.username_in = <empty>
host.auth.password_in = <empty>
# END RECORD
# BEGIN RECORD 2.0-873
host.auth.tbl_idx = 1
host.auth.username = <empty>
host.auth.password = <empty>
# END RECORD
# BEGIN RECORD 2.0-873
host.auth.tbl_idx = 2
host.auth.username = <empty>
host.auth.password = <empty>
# END RECORD

[...]

    Parameters of existing authentication records can be set or updated with the -o update option. The particular record to be set or to be updated is selected in with the -x <host.auth.tbl_idx> option, which references the records host.auth.tbl_idx value. Multiple parameters can be set or updated with a single iscsiadm command by calling it with multiple pairs of -n <parameter-name> and -v <parameter-value>: 

    root@host:~$ iscsiadm -m host -H 1 -C chap -x 2 -o update -n host.auth.username -v testuser -n host.auth.password -v testpassword
    root@host:~$ iscsiadm -m host -H 1 -C chap -o show
# BEGIN RECORD 2.0-873
host.auth.tbl_idx = 0
host.auth.username_in = <empty>
host.auth.password_in = <empty>
# END RECORD
# BEGIN RECORD 2.0-873
host.auth.tbl_idx = 1
host.auth.username = <empty>
host.auth.password = <empty>
# END RECORD
# BEGIN RECORD 2.0-873
host.auth.tbl_idx = 2
host.auth.username = testuser
host.auth.password = testpassword
# END RECORD

[...]

    Finally, existing authentication records can be deleted with the -o delete option: 

    root@host:~$ iscsiadm -m host -H 1 -C chap -x 2 -o delete

  • The flashnode sub-mode displays and alters a table containing information about the iSCSI targets. Calling this sub-mode without any other options displays an overview of the currently configured flash nodes (i.e. targets) on a particular HBA: 

    root@host:~$ iscsiadm -m host -H 1 -C flashnode
qla4xxx: [0] 10.0.0.2:3260,0 iqn.2001-05.com.equallogic:0-fe83b6-a35c152cc-c72004e10ff558d4-lun-000002

    Similar to the previously mentioned host mode, each output line of the flashnode sub-mode contains an index number for each flash node entry (i.e. iSCSI target), which is shown in the first pair of square brackets. With this index number the individual flash node entries are referenced in all further operations.

    New flash nodes or target entries can be added with the -o new option. This operation also needs the information on whether the target addressed via the flash node will be reached via IPv4 or IPv6 addresses. This is accomplished with the -A ipv4 or -A ipv6 option: 

    root@host:~$ iscsiadm -m host -H 1 -C flashnode -o new -A ipv4
Create new flashnode for host 1.
New flashnode for host 1 added at index 1.

    If the operation of adding a new flash node is successful, the index under which the new flash node is addressable is returned. 

    root@host:~$ iscsiadm -m host -H 1 -C flashnode
qla4xxx: [0] 10.0.0.2:3260,0 iqn.2001-05.com.equallogic:0-fe83b6-a35c152cc-c72004e10ff558d4-lun-000002
qla4xxx: [1] 0.0.0.0:3260,0 <empty>

    Unlike authentication records, the flash node or target records contain a lot more parameters. They can be displayed by selecting a specific record by its index with the -x <flashnode_idx> option. The -o show option is the default and is thus optional: 

    root@host:~$ iscsiadm -m host -H 1 -C flashnode -x 1
# BEGIN RECORD 2.0-873
flashnode.session.auto_snd_tgt_disable = 0
flashnode.session.discovery_session = 0
flashnode.session.portal_type = ipv4
flashnode.session.entry_enable = 0
flashnode.session.immediate_data = 0
flashnode.session.initial_r2t = 0
flashnode.session.data_seq_in_order = 1
flashnode.session.data_pdu_in_order = 1
flashnode.session.chap_auth_en = 1
flashnode.session.discovery_logout_en = 0
flashnode.session.bidi_chap_en = 0
flashnode.session.discovery_auth_optional = 0
flashnode.session.erl = 0
flashnode.session.first_burst_len = 0
flashnode.session.def_time2wait = 0
flashnode.session.def_time2retain = 0
flashnode.session.max_outstanding_r2t = 0
flashnode.session.isid = 000e1e17da2c
flashnode.session.tsid = 0
flashnode.session.max_burst_len = 0
flashnode.session.def_taskmgmt_tmo = 10
flashnode.session.targetalias = <empty>
flashnode.session.targetname = <empty>
flashnode.session.discovery_parent_idx = 0
flashnode.session.discovery_parent_type = Sendtarget
flashnode.session.tpgt = 0
flashnode.session.chap_out_idx = 2
flashnode.session.chap_in_idx = 65535
flashnode.session.username = <empty>
flashnode.session.username_in = <empty>
flashnode.session.password = <empty>
flashnode.session.password_in = <empty>
flashnode.session.is_boot_target = 0
flashnode.conn[0].is_fw_assigned_ipv6 = 0
flashnode.conn[0].header_digest_en = 0
flashnode.conn[0].data_digest_en = 0
flashnode.conn[0].snack_req_en = 0
flashnode.conn[0].tcp_timestamp_stat = 0
flashnode.conn[0].tcp_nagle_disable = 0
flashnode.conn[0].tcp_wsf_disable = 0
flashnode.conn[0].tcp_timer_scale = 0
flashnode.conn[0].tcp_timestamp_en = 0
flashnode.conn[0].fragment_disable = 0
flashnode.conn[0].max_xmit_dlength = 0
flashnode.conn[0].max_recv_dlength = 65536
flashnode.conn[0].keepalive_tmo = 0
flashnode.conn[0].port = 3260
flashnode.conn[0].ipaddress = 0.0.0.0
flashnode.conn[0].redirect_ipaddr = 0.0.0.0
flashnode.conn[0].max_segment_size = 0
flashnode.conn[0].local_port = 0
flashnode.conn[0].ipv4_tos = 0
flashnode.conn[0].ipv6_traffic_class = 0
flashnode.conn[0].ipv6_flow_label = 0
flashnode.conn[0].link_local_ipv6 = <empty>
flashnode.conn[0].tcp_xmit_wsf = 0
flashnode.conn[0].tcp_recv_wsf = 0
flashnode.conn[0].statsn = 0
flashnode.conn[0].exp_statsn = 0
# END RECORD

    From the various parameters of a flash node or target record, the following are the most relevant in day to day use:

    • flashnode.session.chap_auth_en: Controls whether the initiator should authenticate against the target. This is enabled by default.

    • flashnode.session.bidi_chap_en: Controls whether the target should also authenticate itself against the initiator. This is disabled by default.

    • flashnode.session.targetname: The IQN of the target to be logged into and to be accessed.

    • flashnode.session.chap_out_idx: The index number (i.e. the value of the host.auth.tbl_idx parameter) of the authentication record to be used for authentication of the initiator against the target.

    • flashnode.conn[0].port: The TCP port of the target portal. The default is port 3260.

    • flashnode.conn[0].ipaddress: The IP address of the target portal.

    The parameter pairs flashnode.session.username/flashnode.session.password and flashnode.session.username_in/flashnode.session.password_in are handled differently than all the other parameters. They are not set or updated directly, but are rather filled in automatically. This is done by setting the respective flashnode.session.chap_out_idx or flashnode.session.chap_in_idx parameter to a value which references the index (i.e. the value host.auth.tbl_idx parameter) of an appropriate authentication record.

    Parameters of existing flash nodes or target entries can be set or updated with the -o update option. The particular record of which the parameters are to be set or to be updated is selected with the -x <flashnode_idx> option. This references an index number gathered from the list of flash nodes or a index number returned at the time of creation of a particular flash node. Multiple parameters can be set or updated with a single iscsiadm command by calling it with multiple pairs of -n <parameter-name> and -v <parameter-value>: 

    root@host:~$ iscsiadm -m host -H 1 -C flashnode -x 1 -o update -n flashnode.session.chap_out_idx -v 2 -n flashnode.session.targetname -v iqn.2001-05.com.equallogic:0-fe83b6-d63c152cc-7ce004e1102558d4-lun-000003 -n flashnode.conn[0].ipaddress -v 10.0.0.2

    The flash node or target entry updated by this command is shown below in a cut-down fashion for brevity: 

    root@host:~$ iscsiadm -m host -H 1 -C flashnode -x 1
# BEGIN RECORD 2.0-873
[...]
flashnode.session.chap_auth_en = 1
flashnode.session.discovery_logout_en = 0
flashnode.session.bidi_chap_en = 0
[...]
flashnode.session.targetname = iqn.2001-05.com.equallogic:0-fe83b6-d63c152cc-7ce004e1102558d4-lun-000003
[...]
flashnode.session.chap_out_idx = 4
flashnode.session.chap_in_idx = 65535
flashnode.session.username = testuser
flashnode.session.username_in = <empty>
flashnode.session.password = testpassword
flashnode.session.password_in = <empty>
[...]
flashnode.conn[0].port = 3260
flashnode.conn[0].ipaddress = 10.0.0.2
flashnode.conn[0].redirect_ipaddr = 0.0.0.0
[...]
# END RECORD

    Once configured, login and logout actions can be performed on the flash node (i.e. target) with the repective command options: 

    root@host:~$ iscsiadm -m host -H 1 -C flashnode -x 1 -o login
root@host:~$ iscsiadm -m host -H 1 -C flashnode -x 1 -o logout

    Unfortunately the iscsiadm command will return with a success status on the login and logout actions once it has passed them successfully to the HBA. This does not reflect on the status of the actual login and logout actions subsequently taken by the HBA against the target configured in the respective flash node! To my knowledge there is currently no information passed back to the command line about the result of the login and logout actions at the HBA levels.

    Newly established as well as already existing iSCSI sessions via the hardware initiator which were set up with the login action shown above, are shown along with all other Open-iSCSI session information in the output of the iscsiadm command in its session mode: 

    root@host:~$ iscsiadm -m session
qla4xxx: [1] 10.0.0.2:3260,1 iqn.2001-05.com.equallogic:0-fe83b6-a35c152cc-c72004e10ff558d4-lun-000002 (flash)
qla4xxx: [2] 10.0.0.2:3260,1 iqn.2001-05.com.equallogic:0-fe83b6-a35c152cc-c72004e10ff558d4-lun-000002 (flash)

[...]

    The fact that the session information is about a iSCSI session established via a hardware initiator is only signified by the flash label in the parentheses at the end of each line. In case of a iSCSI session established via a software initiator, the label in the parentheses reads non-flash.

    Finally, existing flash nodes can be deleted with the -o delete option: 

    root@host:~$ iscsiadm -m host -H 1 -C flashnode -x 1 -o delete

The records from both, the chap and the flashnode table, are stored in the HBAs flash memory. For the limits on how many entries can be stored in each table, see the specification of the particular HBA.

In my opinion, the integration of management of the QLogic hardware initators into the Open-iSCSI iscsiadm command improves and simplifies the administration and configuration a lot over the previous IOCTL based management method via the QLogic QConvergeConsole (qaucli). It finally opens management access to the QLogic hardware initiators to non-“enterprise Linux distributions” like Debian. Definately a big step in the right direction! The importance of using a current version of Open-iSCSI can – in my experience – not be stressed enough. Building and maintaining a package based on a current version from the projects GitHub repository is definitely worth the effort.

One thing i couldn't get to work though was the incoming part of a bi-directional CHAP authentication. In this scenario, not only does the initiator authenticate itself at the target for a iSCSI session to be successfully established, the target also has to authenticate itself against the initiator. My initial thought was, that a setup with bi-directional CHAP authentication should be easily accomplished by just performing the following three steps:

  1. creating an incoming authentication record with the value of the parameters host.auth.username_in and host.auth.password_in set to the respective values configured at the target storage system.

  2. setting the value of the flash node parameter flashnode.session.bidi_chap_en to 1.

  3. setting the value of the flash node parameter flashnode.session.chap_in_idx to the value of the parameter host.auth.tbl_idx, gathered from the newly created incoming authentication record in step 1.

The first two of the above tasks were indeed easily accomplished. The third one seemed easy too, but turned out to be more of a challenge. Setting the flash node parameter flashnode.session.chap_in_idx to the value of the host.auth.tbl_idx parameter from the previously created incoming authentication record just didn't work. Any attempt to change the default value 65535 failed. Neither was the flashnode.session.username_in/flashnode.session.password_in parameter pair automatically updated with the values from the parameters host.auth.username_in and host.auth.password_in. Oddly enough bi-directional CHAP authentication worked as long as there only was one storage system with one set of incoming authentication credentials! Adding another set of flash nodes for a second storage system with its own set of incoming authentication credentials would cause the bi-directional CHAP authentication to fail for all targets on this second storage system.

Being unable to debug this weird behaviour any further on my own, i turned to the Open-iSCSI mailing list for help. See the thread on the Open-iSCSI mailing list for the details. Don't be confused by the fact that the thread at the mailing list was initially about getting to work the network communication with the use of jumbo frames. This initial issue was resolved for me by switching to a more recent Open-iSCSI version as already mentioned above. My last question there was not answered publicly on the mailing list, but Adheer Chandravanshi from development at QLogic got in touch via email. Here's his explanation of the observed behaviour:

[…]

I see that you have added multiple incoming CHAP entries for both hosts 1 and 2.
But looks like even in case of multiple incoming CHAP entries in flash only the first entry takes effect for that particular host for all
the target node entries in flash.
This seems to be a limitation with the flash target node entries.
So you cannot map different incoming CHAP entry for different target nodes in HBA flash.

In your case, only following incoming CHAP entry will be effective for Host 1 as it's the first incoming chap entry. Same goes for Host
2.
# BEGIN RECORD 2.0-873
host.auth.tbl_idx = 2
host.auth.username_in = <username-from-target1>
host.auth.password_in = <password-from-target1>
# END RECORD


Try using only one incoming CHAP entry per host, you can have different outgoing CHAP entries though for each flash target node.

[…]

To sum this up, the QLogic HBAs basically use a first match approach when it comes to the incoming part of a bi-directional CHAP authentication. After finding the first incoming authentication record that is configured, it uses the credentials stored there. Any other – and possibly more suitable – records for incoming authentication are ignored. There's also no way to override this behaviour on a case by case basis via the flash node entries (i.e. iSCSI targets).

In my humble opinion this is a rather serious limitation of the security features in the QLogic hardware initiators. No security policy i have ever encountered in any organisation would allow for the reuse of authentication credentials over different systems. Unfortunately i have no further information as to why the implementation turned out this way. Maybe there was no feature request for this yet, maybe it was just an oversight or maybe there is a limitation in the hardware, preventing a more flexible implementation. Unfortunately my reply to the above email with an inquiry whether such a feature would possibly be implemented in future firmware versions has – up to now – not been answered.

0 Comments | 2015-11-13 written by Frank Fegert (XING) | Permanentlink
Tags:
This website uses cookies. By using the website, you agree with storing cookies on your computer. Also you acknowledge that you have read and understand our Privacy Policy. If you do not agree leave the website. More information about cookies