HMC Update to 7.7.7.0 SP1

After only a short intermission with the HMC Update to 7.7.6.0 SP1, the recent HMC update to v7.7.7.0 SP1 was again not installable directly from the ISO images via the HMC GUI. As previously described in HMC Update to 7.7.5.0 there are two options on how to update the HMC to the new version. For the first option one has to have direct physical access to the HMC and use a media with the downloaded ISO images burned on it. For the second option – which is more remote admin friendly – one can use the network upgrade files, which IBM cleverly fails to mention in the release notes and hides here. The whole network upgrade procedure is described in more detail in TechDoc 635220142. Either way – and not really understanable to me – the upgrade will completely whipe out the HMC along with all your settings and basically do a fresh install. Because of this “feature” you'll have to make sure to back up your partition profile data as well as your HMC configuration, in case you want to roll back to the previous HMC version.

As before, i went with the second option, the network upgrade. Unfortunately IBM also fails to make a statement as to which fixpack partially or completely includes and thus supercedes other fixpacks. Even looking into the ISO image and comparing the RPM files didn't clear things up for me, since different versions of the same RPM come with different fixpacks. To be on the save side i decided to apply the whole chain of MH01343 (aka V7R7.7.0M0), MH01345 (aka "Mandatory efix for HMC V7R770"), MH01355 (aka "eFix for HMC V7R770") and finally MH01346 (aka "V7R7.7.0M1"). Also as with earlier upgrades, in a dual HMC setup like mine, the second HMC has to be shut down, disconnected or otherwise prevented from accessing the managed systems while it's still running at a lower code version than the first HMC which is currently being updated or already running at a newer version. This was in my case ensured by disabling the switchport which the second HMC uses to connect to the service processor network.

The upgrade procedure i performed was a bit different from the one described in TechDoc 635220142, since it replaces most of the HMC GUI activity with commands on the HMC shell:

  1. Create a backup of the managed system profile data (replaces steps 1 to 7 in TechDoc 635220142):

    hscroot@hmc:~$ BK_DATE="`date '+%Y%m%d'`"; export BK_DATE
    hscroot@hmc:~$ lssyscfg -r sys -F name | sort | while read MSYS; do
    bkprofdata -m ${MSYS} -f ${MSYS}_${BK_DATE}.bak --force
    done
  2. Create a backup of the HMC data:

    hscroot@hmc:~$ bkconsdata -r ftp -h <ftp host> -u <user> -d <directory>

    Replace “<ftp host>” and “<user>” with a user on a host running a FTP server and being accessible from the HMC via network. Replace “<directory>” with a directory on that host which has enough space for an approximately 4GB large backup dump.

  3. Save the HMC configuration, users and managed system profile data to a hidden partition (replaces steps 8 to 13 in TechDoc 635220142):

    hscroot@hmc:~$ saveupgdata -r disk

    The data is stored on the filesystem:

    /dev/sda5    /mnt/upgrade  ext3     rw,noauto,owner,exec,nosuid,dev 0 0

    which cannot be mounted by the usual HMC shell users. If you have access to the HMC as root, the contents of /mnt/upgrade should look something like this:

    drwxr-xr-x 3 root root     4096 2013-04-30 15:00 .
    drwxr-xr-x 4 root root     4096 2012-06-19 20:54 ..
    -rw-rw-r-- 1 root ccfw    10240 2013-04-30 23:02 ACMSaveData.tar
    -rw-rw-r-- 1 root ccfw        0 2013-04-30 23:02 doRestore
    drwx------ 2 root root    16384 2011-02-10 21:58 lost+found
    -rw-rw-r-- 1 root ccfw   614400 2013-04-30 23:01 RSCTSaveUpgrade.tar
    -rw-rw-r-- 1 root ccfw    20480 2013-04-30 23:02 SaveCCFWUpgradeData.tar
    -rw-rw-r-- 1 root ccfw 71618560 2013-04-30 23:01 SaveHSCSystemUpgradeData.tar
    -rw-rw-r-- 1 root ccfw 14264320 2013-04-30 23:01 SaveProfileDataUpgrade.tar
    -rw-rw-r-- 1 root ccfw   112640 2013-04-30 23:02 SaveSysplanFiles.tar
  4. Download the network upgrade files from the link above and store them on a FTP server that is accessible from the HMC via network. On the HMC start the transfer of the downloaded files to the HMC (replaces step 14 in TechDoc 635220142):

    hscroot@hmc:~$ getupgfiles -h <ftp host> -u <user> -d <directory>

    Replace “<ftp host>” and “<user>” with a user on the host running the FTP server. Replace “<directory>” with the directory path where the downloaded files were stored. The getupgfiles command mounts the filesystem:

    /dev/sda6    /hmcdump      ext3     rw,noauto,owner,exec,nosuid,dev 0 0

    and stores the network upgrade files in it. This filesystem cannot be mounted by the usual HMC shell users. If you have access to the HMC as root, the contents of /hmcdump should look something like this:

    -rw-r--r-- 1 root hmc     2708320 2013-04-30 12:37 bzImage
    -rw-r--r-- 1 root hmc   808497152 2013-04-30 12:37 disk1.img
    -rw-r--r-- 1 root hmc  1142493184 2013-04-30 12:38 disk2.img
    -rw-r--r-- 1 root hmc  1205121024 2013-04-30 12:49 disk3.img
    -rw-r--r-- 1 root hmc          78 2013-04-30 12:37 hmcnetworkfiles.sum
    -rw-r--r-- 1 root hmc    34160044 2013-04-30 12:38 initrd.gz
    drwx------ 2 root root      16384 2011-02-10 21:58 lost+found
  5. Enable the upgrade procedure on the next reboot:

    hscroot@hmc:~$ lshmc -r
    ssh=enable,[...],netboot=disable,altdiskboot=disable,[...]
    
    hscroot@hmc:~$ chhmc -c altdiskboot -s enable --mode upgrade
    hscroot@hmc:~$ lshmc -r
    ssh=enable,[...],netboot=disable,altdiskboot=enable,[...]
  6. Reboot and start the upgrade process:

    hscroot@hmc:~$ hmcshutdown -t now -r

    If you have a KVM connection to the HMC, you should be able to follow the upgrade process with it. Here are some example screen shots:

    HMC network based upgrade to v7.7.7.0 - 1

    HMC network based upgrade to v7.7.7.0 - 2

    HMC network based upgrade to v7.7.7.0 - 3

After the upgrade to V7R7.7.0M0 is complete, you can apply the additional efixes and the service pack in the usual way via the HMC GUI. For me the additional efixes and the service pack showed the following output during the update process:

  1. MH01345:

    Mangement console corrective service installation in progress. Please wait...
    Corrective service file offload from remote server in progress...
    The corrective service file offload was successful. Continuing with HMC service installation...
    Verifying Certificate Information
    Authenticating Install Packages
    Installing Packages
    --- Installing ptf-req ....
    --- Installing RSCT ....
    src-3.1.4.2-13008
    rsct.core.utils-3.1.4.2-13008
    rsct.core-3.1.4.2-13008
    rsct.service-3.5.0.0-1
    rsct.basic-3.1.4.2-13008
    --- Installing CSM ....
    csm.core-1.7.1.20-1
    csm.deploy-1.7.1.20-1
    csm_hmc.server-1.7.1.20-1
    csm_hmc.hdwr_svr-7.0-3.4.0
    csm_hmc.client-1.7.1.20-1
    csm.server.hsc-1.7.1.20-1
    --- Installing LPARCMD ....
    hsc.lparcmd-3.0.0.1-1
    ln: creating symbolic link `/usr/hmcrbin/lsnodeid': File exists
    ln: creating symbolic link `/usr/hmcrbin/lsrsrc-api': File exists
    ln: creating symbolic link `/usr/hmcrbin/mkrsrc-api': File exists
    ln: creating symbolic link `/usr/hmcrbin/rmrsrc-api': File exists
    --- Installing InventoryScout ....
    --- Installing Pegasus ....
    --- Updating baseOS ....
    Corrective service installation was successful.
  2. MH01355:

    Mangement console corrective service installation in progress. Please wait...
    Corrective service file offload from remote server in progress...
    The corrective service file offload was successful. Continuing with HMC service installation...
    Verifying Certificate Information
    Authenticating Install Packages
    Installing Packages
    --- Installing ptf-req ....
    Corrective service installation was successful.
  3. MH01346:

    Mangement console corrective service installation in progress. Please wait...
    Corrective service file offload from remote server in progress...
    The corrective service file offload was successful. Continuing with HMC service installation...
    Verifying Certificate Information
    Authenticating Install Packages
    Installing Packages
    --- Installing ptf-req ....
    --- Installing RSCT ....
    src-3.1.4.2-13008
    rsct.core.utils-3.1.4.2-13008
    rsct.core-3.1.4.2-13008
    rsct.service-3.5.0.0-1
    rsct.basic-3.1.4.2-13008
    --- Installing CSM ....
    csm.core-1.7.1.20-1
    csm.deploy-1.7.1.20-1
    csm_hmc.server-1.7.1.20-1
    csm_hmc.hdwr_svr-7.0-3.4.0
    csm_hmc.client-1.7.1.20-1
    csm.server.hsc-1.7.1.20-1
    --- Installing LPARCMD ....
    hsc.lparcmd-3.0.0.1-1
    ln: creating symbolic link `/usr/hmcrbin/lsnodeid': File exists
    ln: creating symbolic link `/usr/hmcrbin/lsrsrc-api': File exists
    ln: creating symbolic link `/usr/hmcrbin/mkrsrc-api': File exists
    ln: creating symbolic link `/usr/hmcrbin/rmrsrc-api': File exists
    --- Installing InventoryScout ....
    --- Installing Pegasus ....
    --- Updating baseOS ....
    Corrective service installation was successful.

The MH01345 and MH01346 updates still show error messages with regard to symlink creation appearing during the update process. After the earlier post HMC Update to 7.7.6.0 SP1 in which i did a more detailed analysis of the issue, i opened a PMR with IBM support to get this fixed. The PMR was returned from development on 2013/02/18 with the comment:

This issue is resolved in 770.

and was eventually closed. Although – or actually because – this issue is rather trivial and non-critical to the HMC function, it's still another case in recent history where i wonder why we even bother to pay premium rates for IBM support in the first place! Well, lets re-open the PMR and find out what IBM thinks about the issue this time.

Aside from that, we're currently not experiencing any issues with the new HMC version.