2016-08-28 // Backporting Open-iSCSI to Debian 8 "Jessie"
The Debian Open-iSCSI package is now based on current upstream version of Open-iSCSI. Open-iSCSIs iscsiuio is now provided through its own Debian package. Several improvements (Git commit d05fe0e1, Git commit 6004a7e7) have been made in handling hardware initiator based iSCSI sessions.
Thanks to Christian Seiler for his work on bringing the Debian Open-iSCSI package up to a current upstream version and for helping to sort our some issues related to the use of hardware initiators!
In the previous article Debugging Segfaults in Open-iSCSIs iscsiuio on Intel Broadwell i mentioned using a backported version of Open-iSCSI on Debian 8 (“Jessie”). This new post describes the backport and the changes provided by it in greater detail. All the changes to the original Debian package from “unstable” (“Sid”) can be found in my Debian Open-iSCSI Packaging repository on GitHub.
Starting point was a clone of the Debian Open-iSCSI Packaging repository at Git commit df150d90. Mind though, that in the meantime between creating the backport and writing this, the Debian Open-iSCSI maintainers have been busy and a more recent version of the Debian Open-iSCSI package from “unstable” (“Sid”) is now available.
Within this particular version of the Debian Open-iSCSI package, i first enabled the build of Open-iSCSIs iscsiuio. On the one hand, this was done in order to ensure that the iscsiuio code would successfully build even at this old level of the Open-iSCSI code. On the other hand, this would be used as a differentiator for any issues surfacing later on, after the move to the more recent upstream Open-iSCSI sources, indicating the root cause of those would then solely be with the newer upstream version of Open-iSCSI. Some integration into the general system environment was also added at this point. In detail the changes were:
Git commit 32c96e6c removes the Debian patch
05-disable-iscsiuio.patchwhich disables the build of iscsiuio.Git commit 984344a1 enables the build of iscsiuio, extends the cleanup build targets and adds iscsiuio to the dh_systemd build targets.
Git commit 89d845a9 adds the results from the successful build – the iscsiuio binary, the iscsiuio manual page, a readme file and a logrotate configuration file – to the Debian package. It also adds the kernel modules
bnx2iandcnicto the list of kernel modules to be loaded at installation time.Git commit 89195bbe adds the systemd service and socket unit files for iscsiuio. Those files have been taken from this discussion on the Open-iSCSI mailing list and have slightly been altered.
With the above changes a intermediary package was build for testing purposes. During the following tests sometimes all currently mounted filesystems – even those distinctly not based on iSCSI volumes – would suddenly be unmounted. For some filesystems this would succeed, for others, like e.g. the /var and the root filesystem, this would fail due to them being currently in use. The issue particularly occured while stopping the open-iscsi service either via its regular init script or via its systemd service. This is usually done at system shutdown or during uninstall of the Open-iSCSI package. Tracking down the root cause of this issue led to an unhandled case in the umountiscsi.sh script, which is called while stopping the open-iscsi service. Specifically, the following code section is responsible for the observed behaviour:
- debian/extra/umountiscsi.sh
256 if [ $HAVE_LVM -eq 1 ] ; then 257 # Look for all LVM volume groups that have a backing store 258 # on any iSCSI device we found. Also, add $LVMGROUPS set in 259 # /etc/default/open-iscsi (for more complicated stacking 260 # configurations we don't automatically detect). 261 for _vg in $(cd /dev ; $PVS --noheadings -o vg_name $iscsi_disks $iscsi_partitions $iscsi_multipath_disks $iscsi_multipath_partitions 2>/dev/null) $LVMGROUPS ; do 262 add_to_set iscsi_lvm_vgs "$_vg" 263 done
The heuristic of the umountiscsi.sh script are trying to identify iSCSI based disk devices which are valid candidates for proper deactivation upon system shutdown. It turned out that in LVM based setups where there are currently no iSCSI based disk devices present, the variables $iscsi_disks, $iscsi_partitions, $iscsi_multipath_disks and $iscsi_multipath_partitions are left empty by the scripts logic. In line 261 in the above code snippet, this leads to a call to the pvs --noheadings -o vg_name command without any additional arguments limiting its output of volume groups. Hence, the returned output is instead a complete list of all volume groups currently present on the system. Based on this list, the associated logical volumes for each volume group are determined and added to the list of devices to be unmounted. Finally all devices in this list are actually unmounted.
Without making too invasive changes to the script logic of umountiscsi.sh a quick'n'dirty solution was to introduce a check before the call to pvs which would determine whether the variables $iscsi_disks, $iscsi_partitions, $iscsi_multipath_disks and $iscsi_multipath_partitions are all empty. If this is the case, the call to pvs is simply skipped. The following patch shows the necessary code changes which are also available in Git commit 5118af7f:
- umountiscsi.sh.patch
diff --git a/debian/extra/umountiscsi.sh b/debian/extra/umountiscsi.sh index 1206fa1..485069c 100755 --- a/debian/extra/umountiscsi.sh +++ b/debian/extra/umountiscsi.sh @@ -258,9 +258,11 @@ enumerate_iscsi_devices() { # on any iSCSI device we found. Also, add $LVMGROUPS set in # /etc/default/open-iscsi (for more complicated stacking # configurations we don't automatically detect). - for _vg in $(cd /dev ; $PVS --noheadings -o vg_name $iscsi_disks $iscsi_partitions $iscsi_multipath_disks $iscsi_multipath_partitions 2>/dev/null) $LVMGROUPS ; do - add_to_set iscsi_lvm_vgs "$_vg" - done + if [ -n "$iscsi_disks" -o -n "$iscsi_partitions" -o -n "$iscsi_multipath_disks" -o -n "$iscsi_multipath_partitions" ]; then + for _vg in $(cd /dev ; $PVS --noheadings -o vg_name $iscsi_disks $iscsi_partitions $iscsi_multipath_disks $iscsi_multipath_partitions 2>/dev/null) $LVMGROUPS ; do + add_to_set iscsi_lvm_vgs "$_vg" + done + fi # $iscsi_lvm_vgs is now unique list for _vg in $iscsi_lvm_vgs ; do
After this was fixed, the last step was to finally move to the more recent upstream Open-iSCSI sources. In detail the changes in this last step were:
Git commit f5ab51ff moves the code to version 2.0.873+git1.1dfb88a4 which is based upon the upstream Git commit 1dfb88a4. This is the last commit before the externalization of the Open-iSNS library. Since i didn't want to also backport the Open-iSNS packages from Debian “unstable” (“Sid”), i decided to just skip the next two upstream commits 76832662 and c6d1117b and stick with the locally delivered Open-iSNS library.
Git commit 8c1e6974 removes the local Debian patches
01_spelling-errors-and-manpage-hyphen-fixes.patch,02_make-iscsistart-a-dynamic-binary.patchand03_respect-build-flags.patchwhich have already been merged into the more recent upstream Open-iSCSI sources. The remaining local Debian patches were renamed and reordered to01_fix_iscsi_path.patch,02_var-lock_var-run_transition.patchand03_makefile_reproducibility_issues.patch. A whole bunch of new patches named{04,05,06,07,08,09,10,11,12,13,14,15}_upstream_git_commit_<Git commit ID>.patchwere added in order to bring the sources up to the – by then most recent – upstream Git commit 0fa43f29.Git commit d051dece removes some files from the Debian package, which were dynamically generated during the build of iscsiuio.
Finally Git commit 0fabb948 deals with the issue described in Debugging Segfaults in Open-iSCSIs iscsiuio on Intel Broadwell.
With the steps and changes described above, a backported version of Open-iSCSI using its most recent sources was created as a package for Debian 8 (“Jessie”). This package also supports offloaded iSCSI connections via the Broadcom BCM577xx and BCM578xx iSOEs with the use of iscsiuio. The package has been in production use for over a month now and no major issues – neither with the newer upstream Open-iSCSI sources, nor with use of Broadcom BCM577xx and BCM578xx iSOEs through iscsiuio – have emerged so far.
2013-12-16 // Debian Wheezy on IBM Power LPAR with Multipath Support
Earlier, in the post Debian Wheezy on IBM Power LPAR, i wrote about installing Debian on an IBM Power LPAR. Admittedly the previous post was a bit sparse, especially about the hoops one has to jump through when installing Debian in a multipathed setup like this:
To adress this here's a more complete, step by step guide on how to successfully install Debian Wheezy with multipathing enabled. The environment is basically the same as described before in Debian Wheezy on IBM Power LPAR:
Prepare an LPAR. In this example the parition ID is “4”. The VSCSI server adapters on the two VIO servers have the adapter ID “2”.
Prepare the VSCSI disk mappings on the two VIO servers. The hdisks are backed by a SAN / IBM SVC setup. In this example the disk used is
hdisk230on both VIO servers, it has the UUID332136005076801918127980000000000032904214503IBMfcp.Disk mapping on VIOS #1:
root@vios1-p730-342:/$ lscfg -vl hdisk230 hdisk230 U5877.001.0080249-P1-C9-T1-W5005076801205A2F-L7A000000000000 MPIO FC 2145 Manufacturer................IBM Machine Type and Model......2145 ROS Level and ID............0000 Device Specific.(Z0)........0000063268181002 Device Specific.(Z1)........0200646 Serial Number...............60050768019181279800000000000329 root@vios1-p730-342:/$ /usr/ios/cli/ioscli mkvdev -vdev hdisk230 -vadapter vhost0 root@vios1-p730-342:/$ /usr/ios/cli/ioscli lsmap -all | less SVSA Physloc Client Partition ID --------------- -------------------------------------------- ------------------ vhost0 U8231.E2D.06AB34T-V1-C2 0x00000004 VTD vtscsi0 Status Available LUN 0x8100000000000000 Backing device hdisk230 Physloc U5877.001.0080249-P1-C9-T1-W5005076801205A2F-L7A000000000000 Mirrored false
Disk mapping on VIOS #2:
root@vios2-p730-342:/$ lscfg -vl hdisk230 hdisk230 U5877.001.0080249-P1-C6-T1-W5005076801305A2F-L7A000000000000 MPIO FC 2145 Manufacturer................IBM Machine Type and Model......2145 ROS Level and ID............0000 Device Specific.(Z0)........0000063268181002 Device Specific.(Z1)........0200646 Serial Number...............60050768019181279800000000000329 root@vios2-p730-342:/$ /usr/ios/cli/ioscli mkvdev -vdev hdisk230 -vadapter vhost0 root@vios2-p730-342:/$ /usr/ios/cli/ioscli lsmap -all | less SVSA Physloc Client Partition ID --------------- -------------------------------------------- ------------------ vhost0 U8231.E2D.06AB34T-V2-C2 0x00000004 VTD vtscsi0 Status Available LUN 0x8100000000000000 Backing device hdisk230 Physloc U5877.001.0080249-P1-C6-T1-W5005076801305A2F-L7A000000000000 Mirrored false
Prepare the installation media, as a virtual target optical (VTOPT) device on one of the VIO servers:
root@vios1-p730-342:/$ /usr/ios/cli/ioscli mkrep -sp rootvg -size 2G root@vios1-p730-342:/$ df -g /var/vio/VMLibrary Filesystem GB blocks Free %Used Iused %Iused Mounted on /dev/VMLibrary 2.00 1.24 39% 7 1% /var/vio/VMLibrary source-system$ scp debian-testing-powerpc-netinst_20131214.iso root@vios1-p730-342:/var/vio/VMLibrary/ root@vios1-p730-342:/$ /usr/ios/cli/ioscli lsrep Size(mb) Free(mb) Parent Pool Parent Size Parent Free 2041 1269 rootvg 32256 5120 Name File Size Optical Access debian-7.2.0-powerpc-netinst.iso 258 None rw root@vios1-p730-342:/$ /usr/ios/cli/ioscli mkvdev -vadapter vhost0 -fbo root@vios1-p730-342:/$ /usr/ios/cli/ioscli loadopt -vtd vtopt0 -disk debian-7.2.0-powerpc-netinst.iso root@vios1-p730-342:/$ /usr/ios/cli/ioscli lsmap -all | less SVSA Physloc Client Partition ID --------------- -------------------------------------------- ------------------ vhost0 U8231.E2D.06AB34T-V1-C2 0x00000004 VTD vtopt0 Status Available LUN 0x8200000000000000 Backing device /var/vio/VMLibrary/debian-7.2.0-powerpc-netinst.iso Physloc Mirrored N/A VTD vtscsi0 Status Available LUN 0x8100000000000000 Backing device hdisk230 Physloc U5877.001.0080249-P1-C9-T1-W5005076801205A2F-L7A000000000000 Mirrored
Boot the LPAR and enter the SMS menu. Select the “SCSI CD-ROM” device as a boot device:
Select “
5. Select Boot Options”:PowerPC Firmware Version AL770_052 SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved. ------------------------------------------------------------------------------- Main Menu 1. Select Language 2. Setup Remote IPL (Initial Program Load) 3. Change SCSI Settings 4. Select Console 5. Select Boot Options ------------------------------------------------------------------------------- Navigation Keys: X = eXit System Management Services ------------------------------------------------------------------------------- Type menu item number and press Enter or select Navigation key: 5
Select “
1.Select Install/Boot Device”:PowerPC Firmware Version AL770_052 SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved. ------------------------------------------------------------------------------- Multiboot 1. Select Install/Boot Device 2. Configure Boot Device Order 3. Multiboot Startup <OFF> 4. SAN Zoning Support ------------------------------------------------------------------------------- Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services ------------------------------------------------------------------------------- Type menu item number and press Enter or select Navigation key: 1
Select “
7. List all Devices”:PowerPC Firmware Version AL770_052 SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved. ------------------------------------------------------------------------------- Select Device Type 1. Diskette 2. Tape 3. CD/DVD 4. IDE 5. Hard Drive 6. Network 7. List all Devices ------------------------------------------------------------------------------- Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services ------------------------------------------------------------------------------- Type menu item number and press Enter or select Navigation key: 7
Select “
2. SCSI CD-ROM”:PowerPC Firmware Version AL770_052 SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved. ------------------------------------------------------------------------------- Select Device Device Current Device Number Position Name 1. - Interpartition Logical LAN ( loc=U8231.E2D.06AB34T-V4-C4-T1 ) 2. 3 SCSI CD-ROM ( loc=U8231.E2D.06AB34T-V4-C2-T1-L8200000000000000 ) ------------------------------------------------------------------------------- Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services ------------------------------------------------------------------------------- Type menu item number and press Enter or select Navigation key: 2
Select “
2. Normal Mode Boot”:PowerPC Firmware Version AL770_052 SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved. ------------------------------------------------------------------------------- Select Task SCSI CD-ROM ( loc=U8231.E2D.06AB34T-V4-C2-T1-L8200000000000000 ) 1. Information 2. Normal Mode Boot 3. Service Mode Boot ------------------------------------------------------------------------------- Navigation keys: M = return to Main Menu ESC key = return to previous screen X = eXit System Management Services ------------------------------------------------------------------------------- Type menu item number and press Enter or select Navigation key: 2
Select “
1. Yes”:PowerPC Firmware Version AL770_052 SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved. ------------------------------------------------------------------------------- Are you sure you want to exit System Management Services? 1. Yes 2. No ------------------------------------------------------------------------------- Navigation Keys: X = eXit System Management Services ------------------------------------------------------------------------------- Type menu item number and press Enter or select Navigation key: 1
At the Debian installer yaboot boot prompt enter “
expert disk-detect/multipath/enable=true” to boot the installer image with multipath support enabled:IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM STARTING SOFTWARE IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM PLEASE WAIT... IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM - Elapsed time since release of system processors: 150814 mins 23 secs Config file read, 1337 bytes Welcome to Debian GNU/Linux wheezy! This is a Debian installation CDROM, built on 20131012-15:01. [...] Welcome to yaboot version 1.3.16 Enter "help" to get some basic usage information boot: expert disk-detect/multipath/enable=true
Go through the following installer dialogs:
Choose language Configure the keyboard Detect and mount CD-ROM
and select the configuration items according to your needs and environment.
In the installer dialog:
Load installer components from CD
select:
+----------------+ [?] Load installer components from CD +----------------+ | | | All components of the installer needed to complete the install will | | be loaded automatically and are not listed here. Some other | | (optional) installer components are shown below. They are probably | | not necessary, but may be interesting to some users. | | | | Note that if you select a component that requires others, those | | components will also be loaded. | | | | Installer components to load: | | | | [*] multipath-modules-3.2.0-4-powerpc64-di: Multipath support | | [*] network-console: Continue installation remotely using SSH | | [*] openssh-client-udeb: secure shell client for the Debian installer | | | | <Go Back> <Continue> | | | +-------------------------------------------------------------------------+
Go through the following installer dialogs:
Detect network hardware Configure the network Continue installation remotely using SSH Set up users and passwords Configure the clock
and configure the debian installer network and user settings according to your environment.
Log into the system via SSH and replace the
find-partitionsbinary on the installer:~ $ /usr/lib/partconf/find-partitions --flag prep Warning: The driver descriptor says the physical block size is 512 bytes, but Linux says it is 2048 bytes. A bug has been detected in GNU Parted. Refer to the web site of parted http://www.gnu.org/software/parted/parted.html for more information of what could be useful for bug submitting! Please email a bug report to bug-parted@gnu.org containing at least the version (2.3) and the following message: Assertion (disk != NULL) at ../../libparted/disk.c:1548 in function ped_disk_next_partition() failed. Aborted ~ $ mv -i /usr/lib/partconf/find-partitions /usr/lib/partconf/find-partitions.orig ~ $ scp user@source-system:~/find-partitions /usr/lib/partconf/ ~ $ chmod 755 /usr/lib/partconf/find-partitions ~ $ /usr/lib/partconf/find-partitions --flag prep
While still in the SSH login to the debian installer, manually load the
multipathkernel module:~ $ cd /lib/modules/3.2.0-4-powerpc64/kernel/drivers/md ~ $ modprobe multipath
Return to the virtual terminal and select the installer dialog:
Detect disks
and acknowledge the question regarding parameters for the
dm-emckernel module.Select the installer dialog:
Partition disks
and select the:
Manual
paritioning method. Then select the entry:
LVM VG mpatha, LV mpatha - 38.7 GB Linux device-mapper (multipath)
Do NOT select one of the entries starting with “SCSI”! If there is no entry offering a “mpath” device path, the most likely cause is that the above step no. 10, loading the multipath kernel module, failed. In this case check the file
/var/log/syslogfor error messages.Select “
Yes”“ to create a new partition table+-----------------------+ [!!] Partition disks +------------------------+ | | | You have selected an entire device to partition. If you proceed with | | creating a new partition table on the device, then all current | | partitions will be removed. | | | | Note that you will be able to undo this operation later if you wish. | | | | Create new empty partition table on this device? | | | | <Go Back> <Yes> <No> | | | +-----------------------------------------------------------------------+
Select ”
msdos“ as a partition table type:+-------------+ [.] Partition disks +--------------+ | | | Select the type of partition table to use. | | | | Partition table type: | | | | aix | | amiga | | bsd | | dvh | | gpt | | mac | | msdos | | sun | | loop | | | | <Go Back> | | | +--------------------------------------------------+
Select the ”
FREE SPACE“ entry on the mpath device to create new partitions:+------------------------+ [!!] Partition disks +-------------------------+ | | | This is an overview of your currently configured partitions and mount | | points. Select a partition to modify its settings (file system, mount | | point, etc.), a free space to create partitions, or a device to | | initialize its partition table. | | | | Guided partitioning | | Configure software RAID | | Configure the Logical Volume Manager | | Configure encrypted volumes | | | | LVM VG mpatha, LV mpatha - 38.7 GB Linux device-mapper (multipath | | pri/log 38.7 GB FREE SPACE | | SCSI1 (0,1,0) (sda) - 38.7 GB AIX VDASD | | SCSI2 (0,1,0) (sdb) - 38.7 GB AIX VDASD | | | | | | <Go Back> | | | +-------------------------------------------------------------------------+
Select ”
Create a new partition“:+-----------+ [!!] Partition disks +-----------+ | | | How to use this free space: | | | | Create a new partition | | Automatically partition the free space | | Show Cylinder/Head/Sector information | | | | <Go Back> | | | +----------------------------------------------+
Enter “8” to create a parition of about 7MB size:
+-----------------------+ [!!] Partition disks +------------------------+ | | | The maximum size for this partition is 38.7 GB. | | | | Hint: "max" can be used as a shortcut to specify the maximum size, or | | enter a percentage (e.g. "20%") to use that percentage of the maximum | | size. | | | | New partition size: | | | | 8____________________________________________________________________ | | | | <Go Back> <Continue> | | | +-----------------------------------------------------------------------+
Select ”
Primary“ as a partition type:+-----+ [!!] Partition disks +------+ | | | Type for the new partition: | | | | Primary | | Logical | | | | <Go Back> | | | +-----------------------------------+
Select ”
Beginning“ as the location for the new parition:+------------------------+ [!!] Partition disks +-------------------------+ | | | Please choose whether you want the new partition to be created at the | | beginning or at the end of the available space. | | | | Location for the new partition: | | | | Beginning | | End | | | | <Go Back> | | | +-------------------------------------------------------------------------+
Select ”
PowerPC PReP boot partition“ as the parition usage and set the ”Bootable flag“ to ”on“:+------------------------+ [!!] Partition disks +-------------------------+ | | | You are editing partition #1 of LVM VG mpatha, LV mpatha. No existing | | file system was detected in this partition. | | | | Partition settings: | | | | Use as: PowerPC PReP boot partition | | | | Bootable flag: on | | | | Copy data from another partition | | Delete the partition | | Done setting up the partition | | | | <Go Back> | | | +-------------------------------------------------------------------------+
Repeat the above steps two more times and create the following parition layout. Then select ”
Configure the Logical Volume Manager“ and acknowledge the partition scheme to be written to disk:+------------------------+ [!!] Partition disks +-------------------------+ | | | This is an overview of your currently configured partitions and mount | | points. Select a partition to modify its settings (file system, mount | | point, etc.), a free space to create partitions, or a device to | | initialize its partition table. | | | | Guided partitioning | | Configure software RAID | | Configure the Logical Volume Manager | | Configure encrypted volumes | | | | LVM VG mpatha, LV mpatha - 38.7 GB Linux device-mapper (multipath | | > #1 primary 7.3 MB B K | | > #2 primary 511.7 MB f ext4 /boot | | > #5 logical 38.1 GB K lvm | | SCSI1 (0,1,0) (sda) - 38.7 GB AIX VDASD | | | | <Go Back> | | | +-------------------------------------------------------------------------+
Select ”
Create volume group“:+----------+ [!!] Partition disks +-----------+ | | | Summary of current LVM configuration: | | | | Free Physical Volumes: 1 | | Used Physical Volumes: 0 | | Volume Groups: 0 | | Logical Volumes: 0 | | | | LVM configuration action: | | | | Display configuration details | | Create volume group | | Finish | | | | <Go Back> | | | +---------------------------------------------+
Enter a volume group name, here ”
vg00“:+-----------------------+ [!!] Partition disks +------------------------+ | | | Please enter the name you would like to use for the new volume group. | | | | Volume group name: | | | | vg00_________________________________________________________________ | | | | <Go Back> <Continue> | | | +-----------------------------------------------------------------------+
Select the device ”
/dev/mapper/mpathap5“ created above as a physical volume for the volume group:+-----------------+ [!!] Partition disks +------------------+ | | | Please select the devices for the new volume group. | | | | You can select one or more devices. | | | | Devices for the new volume group: | | | | [ ] /dev/mapper/mpathap1 (7MB) | | [ ] /dev/mapper/mpathap2 (511MB; ext4) | | [*] /dev/mapper/mpathap5 (38133MB) | | [ ] (38133MB; lvm) | | | | <Go Back> <Continue> | | | +-----------------------------------------------------------+
Create logical volumes according to your needs and environment. See the following list as an example. Note that the logical volume for the root filesystem resides inside the volume group:
+----------------------+ [!!] Partition disks +----------------------+ | | | Current LVM configuration: | | Unallocated physical volumes: | | * none | | | | Volume groups: | | * vg00 (38130MB) | | - Uses physical volume: /dev/mapper/mpathap5 (38130MB) | | - Provides logical volume: home (1023MB) | | - Provides logical volume: root (1023MB) | | - Provides logical volume: srv (28408MB) | | - Provides logical volume: swap (511MB) | | - Provides logical volume: tmp (1023MB) | | - Provides logical volume: usr (4093MB) | | - Provides logical volume: var (2046MB) | | | | <Continue> | | | +--------------------------------------------------------------------+
Assign filesystems to each of the logical volumes created above. See the following list as an example. Note that only the “PowerPC PReP boot partition” and the partition for the ”
/boot“ filesystem resides outside the volume group:+------------------------+ [!!] Partition disks +-------------------------+ | | | This is an overview of your currently configured partitions and mount | | points. Select a partition to modify its settings (file system, mount | | point, etc.), a free space to create partitions, or a device to | | initialize its partition table. | | | | Guided partitioning | | Configure software RAID | | Configure the Logical Volume Manager | | Configure encrypted volumes | | | | LVM VG mpatha, LV mpatha - 38.7 GB Linux device-mapper (multipath | | > #1 primary 7.3 MB B K | | > #2 primary 511.7 MB ext4 | | > #5 logical 38.1 GB K lvm | | LVM VG mpathap1, LV mpathap1 - 7.3 MB Linux device-mapper (linear | | LVM VG mpathap2, LV mpathap2 - 511.7 MB Linux device-mapper (line | | > #1 511.7 MB K ext4 /boot | | LVM VG mpathap5, LV mpathap5 - 38.1 GB Linux device-mapper (linea | | LVM VG vg00, LV home - 1.0 GB Linux device-mapper (linear) | | > #1 1.0 GB f ext4 /home | | LVM VG vg00, LV root - 1.0 GB Linux device-mapper (linear) | | > #1 1.0 GB f ext4 / | | LVM VG vg00, LV srv - 28.4 GB Linux device-mapper (linear) | | > #1 28.4 GB f ext4 /srv | | LVM VG vg00, LV swap - 511.7 MB Linux device-mapper (linear) | | > #1 511.7 MB f swap swap | | LVM VG vg00, LV tmp - 1.0 GB Linux device-mapper (linear) | | > #1 1.0 GB f ext4 /tmp | | LVM VG vg00, LV usr - 4.1 GB Linux device-mapper (linear) | | > #1 4.1 GB f ext4 /usr | | LVM VG vg00, LV var - 2.0 GB Linux device-mapper (linear) | | > #1 2.0 GB f ext4 /var | | SCSI1 (0,1,0) (sda) - 38.7 GB AIX VDASD | | SCSI2 (0,1,0) (sdb) - 38.7 GB AIX VDASD | | | | Undo changes to partitions | | Finish partitioning and write changes to disk | | | | <Go Back> | | | +-------------------------------------------------------------------------+
Select ”
Finish partitioning and write changes to disk“.Select the installer dialog:
Install the base system
Choose a Linux kernel to be installed and select a “targeted” initrd to be created.
Log into the system via SSH again and manually install the necessary multipath packages in the target system:
~ $ chroot /target /bin/bash root@ststdebian01:/$ mount /sys root@ststdebian01:/$ mount /proc root@ststdebian01:/$ apt-get install multipath-tools multipath-tools-boot Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: kpartx libaio1 The following NEW packages will be installed: kpartx libaio1 multipath-tools multipath-tools-boot 0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded. Need to get 0 B/258 kB of archives. After this operation, 853 kB of additional disk space will be used. Do you want to continue [Y/n]? y Preconfiguring packages ... Can not write log, openpty() failed (/dev/pts not mounted?) Selecting previously unselected package libaio1:powerpc. (Reading database ... 14537 files and directories currently installed.) Unpacking libaio1:powerpc (from .../libaio1_0.3.109-3_powerpc.deb) ... Selecting previously unselected package kpartx. Unpacking kpartx (from .../kpartx_0.4.9+git0.4dfdaf2b-7~deb7u1_powerpc.deb) ... Selecting previously unselected package multipath-tools. Unpacking multipath-tools (from .../multipath-tools_0.4.9+git0.4dfdaf2b-7~deb7u1_powerpc.deb) ... Selecting previously unselected package multipath-tools-boot. Unpacking multipath-tools-boot (from .../multipath-tools-boot_0.4.9+git0.4dfdaf2b-7~deb7u1_all.deb) ... Processing triggers for man-db ... Can not write log, openpty() failed (/dev/pts not mounted?) Setting up libaio1:powerpc (0.3.109-3) ... Setting up kpartx (0.4.9+git0.4dfdaf2b-7~deb7u1) ... Setting up multipath-tools (0.4.9+git0.4dfdaf2b-7~deb7u1) ... [ ok ] Starting multipath daemon: multipathd. Setting up multipath-tools-boot (0.4.9+git0.4dfdaf2b-7~deb7u1) ... update-initramfs: deferring update (trigger activated) Processing triggers for initramfs-tools ... update-initramfs: Generating /boot/initrd.img-3.2.0-4-powerpc64 cat: /sys/devices/vio/modalias: No such device root@ststdebian01:/$ vi /etc/multipath.conf defaults { getuid_callout "/lib/udev/scsi_id -g -u -d /dev/%n" no_path_retry 10 user_friendly_names yes } blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z][[0-9]*]" devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]" } multipaths { multipath { wwid 360050768019181279800000000000329 } }
Adjust the configuration items in
/etc/multipath.confaccording to your needs and environment, especially the disk UUID value in thewwidfield.Return to the virtual terminal and go through the following installer dialogs:
Configure the package manager Select and install software
and select the configuration items according to your needs and environment.
Select the installer dialog:
Install yaboot on a hard disk
and select
/dev/mapper/mpathap1as the device where the yaboot boot loader should be installed:+-----------------+ [!!] Install yaboot on a hard disk +------------------+ | | | Yaboot (the Linux boot loader) needs to be installed on a hard disk | | partition in order for your system to be bootable. Please choose the | | destination partition from among these partitions that have the | | bootable flag set. | | | | Warning: this will erase all data on the selected partition! | | | | Device for boot loader installation: | | | | /dev/mapper/mpathap1 | | | | <Go Back> | | | +-------------------------------------------------------------------------+
After the yaboot boot loader has been successfully installed, log into the system via SSH again. Change the device path for the boot partition, to reflect the naming scheme that will be present on the target system once it is running:
~ $ chroot /target /bin/bash root@ststdebian01:/$ vi /etc/fstab #/dev/mapper/mpathap2 /boot ext4 defaults 0 2 /dev/mapper/mpatha-part2 /boot ext4 defaults 0 2
Check the initrd for the inclusion of the multipath tools installed in the previous step:
root@ststdebian01:/$ gzip -dc /boot/initrd.img-3.2.0-4-powerpc64 | cpio -itvf - | grep -i multipath -rwxr-xr-x 1 root root 18212 Oct 15 05:16 sbin/multipath -rw-r--r-- 1 root root 392 Dec 16 18:03 etc/multipath.conf -rw-r--r-- 1 root root 328 Oct 15 05:16 lib/udev/rules.d/60-multipath.rules drwxr-xr-x 2 root root 0 Dec 16 18:03 lib/multipath -rw-r--r-- 1 root root 5516 Oct 15 05:16 lib/multipath/libpriohp_sw.so -rw-r--r-- 1 root root 9620 Oct 15 05:16 lib/multipath/libcheckrdac.so -rw-r--r-- 1 root root 9632 Oct 15 05:16 lib/multipath/libpriodatacore.so -rw-r--r-- 1 root root 13816 Oct 15 05:16 lib/multipath/libchecktur.so -rw-r--r-- 1 root root 9664 Oct 15 05:16 lib/multipath/libcheckdirectio.so -rw-r--r-- 1 root root 5516 Oct 15 05:16 lib/multipath/libprioemc.so -rw-r--r-- 1 root root 9576 Oct 15 05:16 lib/multipath/libcheckhp_sw.so -rw-r--r-- 1 root root 9616 Oct 15 05:16 lib/multipath/libpriohds.so -rw-r--r-- 1 root root 9672 Oct 15 05:16 lib/multipath/libprioiet.so -rw-r--r-- 1 root root 5404 Oct 15 05:16 lib/multipath/libprioconst.so -rw-r--r-- 1 root root 9608 Oct 15 05:16 lib/multipath/libcheckemc_clariion.so -rw-r--r-- 1 root root 5516 Oct 15 05:16 lib/multipath/libpriordac.so -rw-r--r-- 1 root root 5468 Oct 15 05:16 lib/multipath/libpriorandom.so -rw-r--r-- 1 root root 9620 Oct 15 05:16 lib/multipath/libprioontap.so -rw-r--r-- 1 root root 5488 Oct 15 05:16 lib/multipath/libcheckreadsector0.so -rw-r--r-- 1 root root 9656 Oct 15 05:16 lib/multipath/libprioweightedpath.so -rw-r--r-- 1 root root 9692 Oct 15 05:16 lib/multipath/libprioalua.so -rw-r--r-- 1 root root 9592 Oct 15 05:16 lib/multipath/libcheckcciss_tur.so -rw-r--r-- 1 root root 43552 Sep 20 06:13 lib/modules/3.2.0-4-powerpc64/kernel/drivers/md/dm-multipath.ko -rw-r--r-- 1 root root 267620 Oct 15 05:16 lib/libmultipath.so.0 -rwxr-xr-x 1 root root 984 Oct 14 22:25 scripts/local-top/multipath -rwxr-xr-x 1 root root 624 Oct 14 22:25 scripts/init-top/multipath
If the multipath tools, which were installed in the previous step, are missing, update the initrd to include them:
root@ststdebian01:/$ update-initramfs -u -k all
Check the validity of the yaboot configuration, especially the configuration items marked with ”
⇐= !!!“:root@ststdebian01:/$ vi /etc/yaboot.conf boot="/dev/mapper/mpathap1" <== !!! partition=2 root="/dev/mapper/vg00-root" <== !!! timeout=50 install=/usr/lib/yaboot/yaboot enablecdboot image=/vmlinux <== !!! label=Linux read-only initrd=/initrd.img <== !!! image=/vmlinux.old <== !!! label=old read-only initrd=/initrd.img.old <== !!!
If changes were made to the yaboot configuration, install the boot loader again:
root@ststdebian01:/$ ybin -v
In order to have the root filesystem on LVM and yaboot still be able to access its necessary configuration, the
yaboot.confhas to be placed in a subdirectory ”/etc/“ on a non-LVM device. See HOWTO-Booting with Yaboot on PowerPC for a detailed explaination which is also true for theyaboot.conf:It's worth noting thatyabootlocates the kernel image within a partition's filesystem without regard to where that partition will eventually be mounted within the Linux root filesystem. So, for example, if you've placed a kernel image or symlink at /boot/vmlinux, but /boot is actually a separate partition on your system, then the image path foryabootwill just beimage=/vmlinux.In this case the kernel and initrd images reside on the ”
/boot“ filesystem, so the path ”/etc/yaboot.conf“ has to be placed below ”/boot“. A symlink pointing from ”/etc/yaboot.conf“ to ”/boot/etc/yaboot.conf“ is added for the convenience of tools that expect it still to be in ”/etc“:root@ststdebian01:/$ mkdir /boot/etc root@ststdebian01:/$ mv -i /etc/yaboot.conf /boot/etc/ root@ststdebian01:/$ ln -s /boot/etc/yaboot.conf /etc/yaboot.conf root@ststdebian01:/$ ls -al /boot/etc/yaboot.conf /etc/yaboot.conf -rw-r--r-- 1 root root 547 Dec 16 18:20 /boot/etc/yaboot.conf lrwxrwxrwx 1 root root 21 Dec 16 18:48 /etc/yaboot.conf -> /boot/etc/yaboot.conf
Select the installer dialog:
Finish the installation
After a successful reboot, change the yaboot ”
boot=“ configuration directive (marked with ”⇐= !!!“) to use the device path name present in the now running system:root@ststdebian01:/$ vi /etc/yaboot.conf boot="/dev/mapper/mpatha-part1" <== !!! partition=2 root="/dev/mapper/vg00-root" timeout=50 install=/usr/lib/yaboot/yaboot enablecdboot image=/vmlinux label=Linux read-only initrd=/initrd.img image=/vmlinux.old label=old read-only initrd=/initrd.img.old root@ststdebian01:/$ ybin -v
Check the multipath status of the newly installed, now running system:
root@ststdebian01:~$ multipath -ll mpatha (360050768019181279800000000000329) dm-0 AIX,VDASD size=36G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active |- 0:0:1:0 sda 8:0 active ready running `- 1:0:1:0 sdb 8:16 active ready running root@ststdebian01:~$ df -h Filesystem Size Used Avail Use% Mounted on rootfs 961M 168M 745M 19% / udev 10M 0 10M 0% /dev tmpfs 401M 188K 401M 1% /run /dev/mapper/vg00-root 961M 168M 745M 19% / tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 801M 0 801M 0% /run/shm /dev/mapper/mpatha-part2 473M 28M 421M 7% /boot /dev/mapper/vg00-home 961M 18M 895M 2% /home /dev/mapper/vg00-srv 27G 172M 25G 1% /srv /dev/mapper/vg00-tmp 961M 18M 895M 2% /tmp /dev/mapper/vg00-usr 3.8G 380M 3.2G 11% /usr /dev/mapper/vg00-var 1.9G 191M 1.6G 11% /var root@ststdebian01:~$ mount | grep mapper /dev/mapper/vg00-root on / type ext4 (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered) /dev/mapper/mpatha-part2 on /boot type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered) /dev/mapper/vg00-home on /home type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered) /dev/mapper/vg00-srv on /srv type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered) /dev/mapper/vg00-tmp on /tmp type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered) /dev/mapper/vg00-usr on /usr type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered) /dev/mapper/vg00-var on /var type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered) root@ststdebian01:~$ pvdisplay --- Physical volume --- PV Name /dev/mapper/mpatha-part5 VG Name vg00 PV Size 35.51 GiB / not usable 3.00 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 9091 Free PE 0 Allocated PE 9091 PV UUID gO0PTK-xi9b-H4om-BRgj-rEy2-NSzC-UokjEw
Although it is a rather manual and error prone process, it is nonetheless possible to install Debian Wheezy with multipathing enabled from the start. Another and probably easier method would be to map the disk through only one VIO server during the installation phase and add the second mapping as well as the necessary multipath configuration after the installed system has been successfully brought up.
Some of the manual steps above should IMHO be handled automatically by the Debian installer. For example, why the missing packages multipath-tools and multipath-tools-boot aren't installed automatically eludes me. So there seems to be at least some room for improvement.
First tests with the Debian installer of the upcoming “Jessie” release were promising, but the completion of an install fails at the moment due to unresolved package dependencies. But that'll be the subject for future posts … 
2013-10-06 // Live Partition Mobility (LPM) with Debian on a IBM Power LPAR - Part 1
Live partition mobility (LPM) in the IBM Power environment is roughtly the same as vMotion in the VMware ESX world. It allows the movement of a LPAR from one IBM Power hardware system to another without (major) interruption to the system running within the LPAR. We use this feature on a regular basis for our AIX LPARs and it has simplified our work to a great extent, since we no longer need downtimes for a lot our regular administrative work. LPM also works for LPARs running Linux as an OS, but since IBM only supports the SuSE and Red Hat enterprise distributions, the necessary service and productivity tools to successfully perform LPM – and also DLPAR – operations are not readyly available to users of the Debian distribution. I still wanted to be able to do DLPAR and LPM operations on our Debian LPARs as well. To that effect, i did a conversion of the necessary RPM packages from the service and productivity tools mentioned before, to the DEB package format. Most of the conversion work was done by the alien tool provided by the Debian distribution. Besides that, there still were some manual patches necessary to adjust the components to the specifics of the Debian environment. A current version of the Linux kernel and a rebuild of the kernel package with some PPC specific options enabled was also necessary. Here are the individual steps:
Install the prerequisite Debian packages:
libstdc++5 ksh uuid libsgutils1 libsqlite3-0
$ apt-get install libstdc++5 ksh uuid libsgutils1 libsqlite3-0
The
libsqlite3-0is currently only necessary for thelibservicelog-1-1-1-32bitpackage. The packageslibservicelogandservicelog, which also have a dependency tolibsqlite3-0, contain binaries and libraries that are build for a 64bit userland (ppc64) which is currently not available for Debian. Using the binaries or libraries fromlibservicelogorservicelogwill therefore result in an error about unresolved symbols.Convert or download the IBM service and productivity tools:
Option 1: Download the already converted IBM service and productivity tools:
Option 2: Convert the IBM service and productivity tools from RPM to DEB:
Install the prerequisite Debian packages:
alien
$ apt-get install alien
Download the necessary patch files:
Filename Filesize Last modified DynamicRM.patch 3.7 KiB 2013/10/06 17:17 devices.chrp.base.ServiceRM.patch 2.9 KiB 2013/10/06 17:17 librtas-32bit.patch 453.0 B 2013/10/06 17:17 librtas.patch 434.0 B 2013/10/06 17:17 libservicelog-1_1-1-32bit.patch 582.0 B 2013/10/06 17:17 libservicelog-1_1-1.patch 553.0 B 2013/10/06 17:17 libservicelog.patch 522.0 B 2013/10/06 17:17 libvpd2.patch 494.0 B 2013/10/06 17:17 lsvpd.patch 789.0 B 2013/10/06 17:17 rsct.core.patch 10.6 KiB 2013/10/06 17:17 rsct.core.utils.patch 7.5 KiB 2013/10/06 17:17 servicelog.patch 487.0 B 2013/10/06 17:17 src.patch 7.2 KiB 2013/10/06 17:17 Convert
librtas:$ alien -gc librtas-32bit-1.3.6-4.ppc64.rpm $ patch -p 0 < librtas-32bit.patch $ cd librtas-32bit-1.3.6 $ ./debian/rules binary $ alien -gc librtas-1.3.6-3.ppc64.rpm $ patch -p 0 < librtas.patch $ cd librtas-1.3.6 $ ./debian/rules binary
Convert
src:$ alien -gc src-1.3.1.1-11277.ppc.rpm $ patch -p 0 < src.patch $ rm src-1.3.1.1/debian/postrm $ cd src-1.3.1.1 $ perl -i -p -e 's/ppc64/powerpc/' ./debian/control $ ./debian/rules binary
Convert RSCT core and utils:
$ alien -gc rsct.core.utils-3.1.0.7-11277.ppc.rpm $ patch -p 0 < rsct.core.utils.patch $ cd rsct.core.utils-3.1.0.7 $ ./debian/rules binary $ alien -gc rsct.core-3.1.0.7-11277.ppc.rpm $ patch -p 0 < rsct.core.patch $ cd rsct.core-3.1.0.7 $ ./debian/rules binary
Convert ServiceRM:
$ alien -gc devices.chrp.base.ServiceRM-2.3.0.0-11231.ppc.rpm $ patch -p 0 < devices.chrp.base.ServiceRM.patch $ cd devices.chrp.base.ServiceRM-2.3.0.0 $ ./debian/rules binary
Convert DynamicRM:
$ alien -gc DynamicRM-1.3.9-7.ppc64.rpm $ patch -p 0 < DynamicRM.patch $ cd DynamicRM-1.3.9 $ ./debian/rules binary
Convert
lsvpdandlibvpd:$ alien -gc libvpd2-2.1.3-3.ppc64.rpm $ patch -p 0 < libvpd.patch $ cd libvpd2-2.1.3 $ ./debian/rules binary $ alien -gc lsvpd-1.6.11-4.ppc64.rpm $ patch -p 0 < lsvpd.patch $ cd lsvpd-1.6.11 $ ./debian/rules binary
Build, package and/or install PowerPC Utils:
Install from sources at sourceforge.net or use custom build DEB package from:
Filename Filesize Last modified powerpc-ibm-utils_1.2.12-1_powerpc.deb 216.3 KiB 2013/10/06 16:43 powerpc-ibm-utils_1.2.12-1_powerpc.deb.sha1sum 81.0 B 2013/10/06 16:43 Convert
libservicelogandservicelog:$ alien -gc libservicelog-1_1-1-1.1.11-9.ppc64.rpm $ patch -p 0 < libservicelog-1_1-1.patch $ cd libservicelog-1_1-1-1.1.11 $ ./debian/rules binary $ alien -gc libservicelog-1.1.11-9.ppc64.rpm $ patch -p 0 < libservicelog.patch $ cd libservicelog-1.1.11 $ ./debian/rules binary $ alien -gc libservicelog-1_1-1-32bit-1.1.11-9.ppc.rpm $ patch -p 0 < libservicelog-1_1-1-32bit.patch $ cd libservicelog-1_1-1-32bit-1.1.11 $ ./debian/rules binary $ alien -gc servicelog-1.1.9-8.ppc64.rpm $ patch -p 0 < servicelog.patch $ cd servicelog-1.1.9 $ ./debian/rules binary
Install the IBM service and productivity tools DEB packages:
$ dpkg -i librtas_1.3.6-4_powerpc.deb librtas-32bit_1.3.6-5_powerpc.deb \ src_1.3.1.1-11278_powerpc.deb rsct.core.utils_3.1.0.7-11278_powerpc.deb \ rsct.core_3.1.0.7-11278_powerpc.deb devices.chrp.base.servicerm_2.3.0.0-11232_powerpc.deb \ dynamicrm_1.3.9-8_powerpc.deb libvpd2_2.1.3-4_powerpc.deb lsvpd_1.6.11-5_powerpc.deb \ powerpc-ibm-utils_1.2.12-1_powerpc.deb libservicelog-1-1-1-32bit_1.1.11-10_powerpc.deb \ libservicelog_1.1.11-10_powerpc.deb servicelog_1.1.9-9_powerpc.deb
Rebuild the stock Debian kernel package as described in HowTo Rebuild An Official Debian Kernel Package. I've confirmed DLPAR and LPM to successfully work with at least the Debian kernel packages versions
2.6.39-3~bpo60+1and3.2.46-1~bpo60+1. On themake menuconfigstep make sure the following kernel configuration options are selected:CONFIG_MIGRATION=y CONFIG_PPC_PSERIES=y CONFIG_PPC_SPLPAR=y CONFIG_LPARCFG=y CONFIG_PPC_SMLPAR=y CONFIG_PPC_RTAS=y CONFIG_RTAS_PROC=y CONFIG_NUMA=y # CONFIG_SPARSEMEM_VMEMMAP is not set CONFIG_MEMORY_HOTPLUG=y CONFIG_MEMORY_HOTPLUG_SPARSE=y CONFIG_MEMORY_HOTREMOVE=y CONFIG_ARCH_MEMORY_PROBE=y CONFIG_HOTPLUG_PCI=y CONFIG_HOTPLUG_PCI_RPA=y CONFIG_HOTPLUG_PCI_RPA_DLPAR=y
Or use one of the following trimmed down kernel configuration files:
Filename Filesize Last modified config-2.6.39-bpo.2-powerpc.config 54.6 KiB 2013/10/06 20:48 config-3.2.0-0.bpo.4.ssb.1-powerpc64.config 57.3 KiB 2013/10/06 20:48 Install the newly build kernel package, reboot and select the new kernel to be loaded.
A few minutes after the system been started, DLPAR and LPM operations on the LPAR should now be possible from the HMC. A good indication from the HMC GUI is a properly filled field in the “OS Version” column. From the HMC CLI you can check with:
$ lspartition -dlpar ... <#108> Partition:<45*8231-E2D*06AB35T, ststnagios02.lan.ssbag, 10.8.32.46> Active:<1>, OS:<Linux/Debian, 3.2.0-0.bpo.4.ssb.1-powerUnknown, Unknown>, DCaps:<0x2c7f>, CmdCaps:<0x19, 0x19>, PinnedMem:<0> ...
The LPAR should show up in the output and the value of
DCapsshould be different from0x0.After a successful LPM operation the output of
dmesgfrom within the OS should look like this:... [539043.613297] calling ibm,suspend-me on cpu 4 [539043.960651] EPOW <0x6240040000000b8 0x0 0x0> [539043.960665] ibmvscsi 30000003: Re-enabling adapter! [539043.961606] RTAS: event: 21, Type: EPOW, Severity: 1 [539043.962920] ibmvscsi 30000002: Re-enabling adapter! [539044.175848] property parse failed in parse_next_property at line 230 [539044.485745] ibmvscsi 30000002: partner initialization complete [539044.485815] ibmvscsi 30000002: host srp version: 16.a, host partition vios1-p730-222 (1), OS 3, max io 262144 [539044.485892] ibmvscsi 30000002: Client reserve enabled [539044.485907] ibmvscsi 30000002: sent SRP login [539044.485964] ibmvscsi 30000002: SRP_LOGIN succeeded [539044.525723] ibmvscsi 30000003: partner initialization complete [539044.525779] ibmvscsi 30000003: host srp version: 16.a, host partition vios2-p730-222 (2), OS 3, max io 262144 [539044.525884] ibmvscsi 30000003: Client reserve enabled [539044.525897] ibmvscsi 30000003: sent SRP login [539044.525943] ibmvscsi 30000003: SRP_LOGIN succeeded [539044.884514] property parse failed in parse_next_property at line 230 ...
Although this was done some time ago and there now have already been several new versions of the packages from the service and productivity tools, DLPAR and LPM still work with this setup. There will be another installment of this post in the future with updated package versions. Another item on my ToDo list is to provide those components from the service and productivity tools which are available in source code as native Debian packages.
2013-02-17 // Debian Wheezy on IBM Power LPAR
At work we're running several IBM Power LPARs with Debian as operating system. The current Debian version we're running is “squeeze” (v6.0.6), which was upgraded from the initially installed “lenny” (v5.0). Although there was some hassle with the yaboot package, the stock “lenny” Debian-Installer ISO image used back then worked as far as installing the base OS was concerned. For some reason, starting with the Debian “squeeze” release, the Debian-Installer stopped working in our setup, which looks like this:
The SAN (IBM SVC) based LUNs are mapped to two VIOS per hardware server. At the VIOS the LUNs show up as usual AIX hdisks, which are mapped to the “client” LPARs as virtual target SCSI devices (vtscsi). The LPARs have virtual client SCSI devices through which they access the mapped hdisks/LUNs. For access to CD/DVD devices from the LPAR, there are two options. Either map a physical CD/DVD device directly to the LPAR – needless to say, not very scalable or comfortable. Or place your CD/DVD media as ISO images on one of the VIOS and map those image files to the LPARs through virtual target optical devices (vtopt). The LPARs access them just like the hdisks/LUNs mentioned before, through their virtual client SCSI devices. From inside a LPAR running Linux, the setup shown above looks like this:
... [ 0.379011] scsi 0:0:1:0: Direct-Access AIX VDASD 0001 PQ: 0 ANSI: 3 [ 0.379271] scsi 0:0:2:0: CD-ROM AIX VOPTA PQ: 0 ANSI: 4 [ 0.398197] scsi 1:0:1:0: Direct-Access AIX VDASD 0001 PQ: 0 ANSI: 3 ... [ 2.738953] sd 0:0:1:0: Attached scsi generic sg0 type 0 [ 2.739123] sr 0:0:2:0: Attached scsi generic sg1 type 5 [ 2.739267] sd 1:0:1:0: Attached scsi generic sg2 type 0 ...
The targets 0:0:1:0 and 1:0:1:0 are the two paths to the mapped hdisk/LUN, one path for each VIOS. The target 0:0:2:0 is the mapped ISO image, it has only one path, since only one VIOS maps it to the LPAR.
Now back to the Debian-Installer, which would fail in the above setup with the following error message:
[!!] Install yaboot on a hard disk No bootstrap partition found No hard disks were found which have an "Apple_Bootstrap" partition. You must create an 819200-byte partition with type "Apple_Bootstrap".
This error message is actually caused by the GNU Parted command find-partitions:
~ # /usr/lib/partconf/find-partitions --flag prep Warning: The driver descriptor says the physical block size is 512 bytes, but Linux says it is 2048 bytes. A bug has been detected in GNU Parted. Refer to the web site of parted http://www.gnu.org/software/parted/parted.html for more information of what could be useful for bug submitting! Please email a bug report to address@hidden containing at least the version (2.3) and the following message: Assertion (disk != NULL) at ../../libparted/disk.c:1548 in function ped_disk_next_partition() failed. Aborted
A test with the current “whezzy” Debian-Installer quickly showed the same behaviour. Running the above find-partitions command manually within a shell from the running Debian-Installer and with a copied over strace sched some more light on the root cause of the problem:
~ # /tmp/strace -e trace=!_llseek,read /usr/lib/partconf/find-partitions --flag prep ... open("/dev/mapper/vg00-swap", O_RDWR|O_LARGEFILE) = 3 mmap(NULL, 528384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf7dda000 munmap(0xf7dda000, 528384) = 0 mmap(NULL, 528384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf7dda000 munmap(0xf7dda000, 528384) = 0 mmap(NULL, 528384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf7dda000 munmap(0xf7dda000, 528384) = 0 mmap(NULL, 528384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf7dda000 munmap(0xf7dda000, 528384) = 0 mmap(NULL, 528384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf7dda000 munmap(0xf7dda000, 528384) = 0 mmap(NULL, 528384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf7dda000 munmap(0xf7dda000, 528384) = 0 fsync(3) = 0 close(3) = 0 open("/dev/sr0", O_RDWR|O_LARGEFILE) = 3 fsync(3) = 0 close(3) = 0 open("/dev/sr0", O_RDWR|O_LARGEFILE) = 3 write(2, "Warning: ", 9Warning: ) = 9 write(2, "The driver descriptor says the p"..., 98The driver descriptor says the physical block size is 512 bytes, but Linux says it is 2048 bytes. ) = 98 fsync(3) = 0 close(3) = 0 write(2, "A bug has been detected in GNU P"..., 299A bug has been detected in GNU Parted. Refer to the web site of parted http://www.gnu.org/software/parted/parted.html for more information of what could be useful for bug submitting! Please email a bug report to) and the following message: ) = 299st the version (2.3--More-- write(2, "Assertion (disk != NULL) at ../."..., 102Assertion (disk != NULL) at ../../libparted/disk.c:1548 in function ped_disk_next_partition() failed. ) = 102 rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0 gettid() = 13874 tgkill(13874, 13874, SIGABRT) = 0 --- SIGABRT (Aborted) @ 0 (0) --- +++ killed by SIGABRT +++
Processing stopped at /dev/sr0 which is the Debian-Installer ISO image mapped through the vtopt device to the LPAR. Apparently the ibmvscsi kernel module reports a block size of 512 bytes for the device, but as it is a ISO image of a CD/DVD, the logical block size is 2048 bytes. The GNU parted libraries used by find-partitions stumble upon this discrepancy.
Since in case of the Debian-Installer the CD/DVD media the installer is booted from isn't a target one would be interested installing the system on, the easy fix would be to just skip all CD/DVD type devices. This could be done by the following quick'n'dirty patch to find-partitions:
- find-partitions_d-i_install_LPAR_from_vopt.patch
--- partconf.orig/find-parts.c 2013-02-15 16:35:30.394743001 +0100 +++ partconf/find-parts.c 2013-02-16 16:58:17.090742993 +0100 diff -rui partconf.orig/find-parts.c partconf/find-parts.c @@ -175,6 +175,7 @@ PedDevice *dev = NULL; PedDisk *disk; PedPartition *part; + struct stat st; ped_device_probe_all(); while ((dev = ped_device_get_next(dev)) != NULL) { @@ -182,6 +183,9 @@ continue; if (strstr(dev->path, "/dev/mtd") == dev->path) continue; + if (stat(dev->path, &st) == 0) + if (major(st.st_rdev) == 11) + continue; if (!ped_disk_probe(dev)) continue; disk = ped_disk_new(dev);
A patched find-partitions can be downloaded here and should be copied over into the directory /usr/lib/partconf/ of the running Debian-Installer system, once the error shown above is occurring. Hopefully a fixed version of find-partitions will be accepted for the final release of the wheezy installer and this workaround won't be necessary anymore. For further information see the discussion on the debian-powerpc mailing list and the corresponding Debian bug reports 332227, 350372 and 352914.
Besides this issue, the current RC of the Debian “wheezy” installer worked very well on a test LPAR on current Power7 hardware.
