bityard Blog

// Debian Wheezy on IBM Power LPAR with Multipath Support

Earlier, in the post Debian Wheezy on IBM Power LPAR, i wrote about installing Debian on an IBM Power LPAR. Admittedly the previous post was a bit sparse, especially about the hoops one has to jump through when installing Debian in a multipathed setup like this:

Debian Storage Setup in Dual VIOS Configuration on IBM Power Hardware

To adress this here's a more complete, step by step guide on how to successfully install Debian Wheezy with multipathing enabled. The environment is basically the same as described before in Debian Wheezy on IBM Power LPAR:

  1. Prepare an LPAR. In this example the parition ID is “4”. The VSCSI server adapters on the two VIO servers have the adapter ID “2”.

  2. Prepare the VSCSI disk mappings on the two VIO servers. The hdisks are backed by a SAN / IBM SVC setup. In this example the disk used is hdisk230 on both VIO servers, it has the UUID 332136005076801918127980000000000032904214503IBMfcp.

    Disk mapping on VIOS #1:

    root@vios1-p730-342:/$ lscfg -vl hdisk230
    
      hdisk230         U5877.001.0080249-P1-C9-T1-W5005076801205A2F-L7A000000000000  MPIO FC 2145
    
            Manufacturer................IBM
            Machine Type and Model......2145
            ROS Level and ID............0000
            Device Specific.(Z0)........0000063268181002
            Device Specific.(Z1)........0200646
            Serial Number...............60050768019181279800000000000329
    
    root@vios1-p730-342:/$ /usr/ios/cli/ioscli mkvdev -vdev hdisk230 -vadapter vhost0
    root@vios1-p730-342:/$ /usr/ios/cli/ioscli lsmap -all | less
    
    SVSA            Physloc                                      Client Partition ID
    --------------- -------------------------------------------- ------------------
    vhost0          U8231.E2D.06AB34T-V1-C2                      0x00000004
    
    VTD                   vtscsi0
    Status                Available
    LUN                   0x8100000000000000
    Backing device        hdisk230
    Physloc               U5877.001.0080249-P1-C9-T1-W5005076801205A2F-L7A000000000000
    Mirrored              false
    

    Disk mapping on VIOS #2:

    root@vios2-p730-342:/$ lscfg -vl hdisk230
    
      hdisk230         U5877.001.0080249-P1-C6-T1-W5005076801305A2F-L7A000000000000  MPIO FC 2145
    
            Manufacturer................IBM
            Machine Type and Model......2145
            ROS Level and ID............0000
            Device Specific.(Z0)........0000063268181002
            Device Specific.(Z1)........0200646
            Serial Number...............60050768019181279800000000000329
    
    root@vios2-p730-342:/$ /usr/ios/cli/ioscli mkvdev -vdev hdisk230 -vadapter vhost0
    root@vios2-p730-342:/$ /usr/ios/cli/ioscli lsmap -all | less
    
    SVSA            Physloc                                      Client Partition ID
    --------------- -------------------------------------------- ------------------
    vhost0          U8231.E2D.06AB34T-V2-C2                      0x00000004
    
    VTD                   vtscsi0
    Status                Available
    LUN                   0x8100000000000000
    Backing device        hdisk230
    Physloc               U5877.001.0080249-P1-C6-T1-W5005076801305A2F-L7A000000000000
    Mirrored              false
    
  3. Prepare the installation media, as a virtual target optical (VTOPT) device on one of the VIO servers:

    root@vios1-p730-342:/$ /usr/ios/cli/ioscli mkrep -sp rootvg -size 2G
    root@vios1-p730-342:/$ df -g /var/vio/VMLibrary
    
    Filesystem    GB blocks      Free %Used    Iused %Iused Mounted on
    /dev/VMLibrary      2.00      1.24   39%        7     1% /var/vio/VMLibrary
    
    source-system$ scp debian-testing-powerpc-netinst_20131214.iso root@vios1-p730-342:/var/vio/VMLibrary/
    root@vios1-p730-342:/$ /usr/ios/cli/ioscli lsrep
    
    Size(mb) Free(mb) Parent Pool         Parent Size      Parent Free
        2041     1269 rootvg                    32256             5120
    
    Name                                                  File Size Optical         Access
    debian-7.2.0-powerpc-netinst.iso                            258 None            rw
    
    root@vios1-p730-342:/$ /usr/ios/cli/ioscli mkvdev -vadapter vhost0 -fbo
    root@vios1-p730-342:/$ /usr/ios/cli/ioscli loadopt -vtd vtopt0 -disk debian-7.2.0-powerpc-netinst.iso
    root@vios1-p730-342:/$ /usr/ios/cli/ioscli lsmap -all | less
    
    SVSA            Physloc                                      Client Partition ID
    --------------- -------------------------------------------- ------------------
    vhost0          U8231.E2D.06AB34T-V1-C2                      0x00000004
    
    VTD                   vtopt0
    Status                Available
    LUN                   0x8200000000000000
    Backing device        /var/vio/VMLibrary/debian-7.2.0-powerpc-netinst.iso
    Physloc
    Mirrored              N/A
    
    VTD                   vtscsi0
    Status                Available
    LUN                   0x8100000000000000
    Backing device        hdisk230
    Physloc               U5877.001.0080249-P1-C9-T1-W5005076801205A2F-L7A000000000000
    Mirrored
    
  4. Boot the LPAR and enter the SMS menu. Select the “SCSI CD-ROM” device as a boot device:

    Select “5. Select Boot Options”:

     PowerPC Firmware
     Version AL770_052
     SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved.
    -------------------------------------------------------------------------------
     Main Menu
     1.   Select Language
     2.   Setup Remote IPL (Initial Program Load)
     3.   Change SCSI Settings
     4.   Select Console
     5.   Select Boot Options
    
     -------------------------------------------------------------------------------
     Navigation Keys:
    
                                                 X = eXit System Management Services
     -------------------------------------------------------------------------------
     Type menu item number and press Enter or select Navigation key: 5
    

    Select “1.Select Install/Boot Device”:

     PowerPC Firmware
     Version AL770_052
     SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved.
    -------------------------------------------------------------------------------
     Multiboot
     1.   Select Install/Boot Device
     2.   Configure Boot Device Order
     3.   Multiboot Startup <OFF>
     4.   SAN Zoning Support
    
     -------------------------------------------------------------------------------
     Navigation keys:
     M = return to Main Menu
     ESC key = return to previous screen         X = eXit System Management Services
     -------------------------------------------------------------------------------
     Type menu item number and press Enter or select Navigation key: 1
    

    Select “7. List all Devices”:

     PowerPC Firmware
     Version AL770_052
     SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved.
    -------------------------------------------------------------------------------
     Select Device Type
     1.   Diskette
     2.   Tape
     3.   CD/DVD
     4.   IDE
     5.   Hard Drive
     6.   Network
     7.   List all Devices
    
     -------------------------------------------------------------------------------
     Navigation keys:
     M = return to Main Menu
     ESC key = return to previous screen         X = eXit System Management Services
     -------------------------------------------------------------------------------
     Type menu item number and press Enter or select Navigation key: 7
    

    Select “2. SCSI CD-ROM”:

     PowerPC Firmware
     Version AL770_052
     SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved.
    -------------------------------------------------------------------------------
     Select Device
     Device  Current  Device
     Number  Position  Name
     1.        -      Interpartition Logical LAN
            ( loc=U8231.E2D.06AB34T-V4-C4-T1 )
     2.        3      SCSI CD-ROM
            ( loc=U8231.E2D.06AB34T-V4-C2-T1-L8200000000000000 )
    
     -------------------------------------------------------------------------------
     Navigation keys:
     M = return to Main Menu
     ESC key = return to previous screen         X = eXit System Management Services
     -------------------------------------------------------------------------------
     Type menu item number and press Enter or select Navigation key: 2
    

    Select “2. Normal Mode Boot”:

     PowerPC Firmware
     Version AL770_052
     SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved.
    -------------------------------------------------------------------------------
     Select Task
    
    SCSI CD-ROM
        ( loc=U8231.E2D.06AB34T-V4-C2-T1-L8200000000000000 )
    
     1.   Information
     2.   Normal Mode Boot
     3.   Service Mode Boot
    
     -------------------------------------------------------------------------------
     Navigation keys:
     M = return to Main Menu
     ESC key = return to previous screen         X = eXit System Management Services
     -------------------------------------------------------------------------------
     Type menu item number and press Enter or select Navigation key: 2
    

    Select “1. Yes”:

     PowerPC Firmware
     Version AL770_052
     SMS 1.7 (c) Copyright IBM Corp. 2000,2008 All rights reserved.
    -------------------------------------------------------------------------------
     Are you sure you want to exit System Management Services?
     1.   Yes
     2.   No
    
     -------------------------------------------------------------------------------
     Navigation Keys:
    
                                                 X = eXit System Management Services
     -------------------------------------------------------------------------------
     Type menu item number and press Enter or select Navigation key: 1
    
  5. At the Debian installer yaboot boot prompt enter “expert disk-detect/multipath/enable=true” to boot the installer image with multipath support enabled:

    IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM
    IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM
    IBM IBM IBM IBM IBM IBM                             IBM IBM IBM IBM IBM IBM
    IBM IBM IBM IBM IBM IBM     STARTING SOFTWARE       IBM IBM IBM IBM IBM IBM
    IBM IBM IBM IBM IBM IBM        PLEASE WAIT...       IBM IBM IBM IBM IBM IBM
    IBM IBM IBM IBM IBM IBM                             IBM IBM IBM IBM IBM IBM
    IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM
    IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM
    -
    Elapsed time since release of system processors: 150814 mins 23 secs
    
    Config file read, 1337 bytes
    
    Welcome to Debian GNU/Linux wheezy!
    
    This is a Debian installation CDROM,
    built on 20131012-15:01.
    
    [...]
    
    Welcome to yaboot version 1.3.16
    Enter "help" to get some basic usage information
    
    boot: expert disk-detect/multipath/enable=true
    
  6. Go through the following installer dialogs:

    Choose language
    Configure the keyboard
    Detect and mount CD-ROM

    and select the configuration items according to your needs and environment.

  7. In the installer dialog:

    Load installer components from CD

    select:

    +----------------+ [?] Load installer components from CD +----------------+
    |                                                                         |
    | All components of the installer needed to complete the install will     |
    | be loaded automatically and are not listed here. Some other             |
    | (optional) installer components are shown below. They are probably      |
    | not necessary, but may be interesting to some users.                    |
    |                                                                         |
    | Note that if you select a component that requires others, those         |
    | components will also be loaded.                                         |
    |                                                                         |
    | Installer components to load:                                           |
    |                                                                         |
    |  [*] multipath-modules-3.2.0-4-powerpc64-di: Multipath support          |
    |  [*] network-console: Continue installation remotely using SSH          |
    |  [*] openssh-client-udeb: secure shell client for the Debian installer  |
    |                                                                         |
    |     <Go Back>                                            <Continue>     |
    |                                                                         |
    +-------------------------------------------------------------------------+
    
  8. Go through the following installer dialogs:

    Detect network hardware
    Configure the network
    Continue installation remotely using SSH
    Set up users and passwords
    Configure the clock

    and configure the debian installer network and user settings according to your environment.

  9. Log into the system via SSH and replace the find-partitions binary on the installer:

    ~ $  /usr/lib/partconf/find-partitions  --flag prep
    Warning: The driver descriptor says the physical block size is 512 bytes, but
    Linux says it is 2048 bytes. A bug has been detected in GNU Parted. Refer to
    the web site of parted http://www.gnu.org/software/parted/parted.html for more
    information of what could be useful for bug submitting! Please email a bug
    report to bug-parted@gnu.org containing at least the version (2.3) and the
    following message: Assertion (disk != NULL) at ../../libparted/disk.c:1548 in
    function ped_disk_next_partition() failed. Aborted
    
    ~ $ mv -i /usr/lib/partconf/find-partitions /usr/lib/partconf/find-partitions.orig
    ~ $ scp user@source-system:~/find-partitions /usr/lib/partconf/
    ~ $ chmod 755 /usr/lib/partconf/find-partitions
    ~ $ /usr/lib/partconf/find-partitions --flag prep
    
  10. While still in the SSH login to the debian installer, manually load the multipath kernel module:

    ~ $ cd /lib/modules/3.2.0-4-powerpc64/kernel/drivers/md
    ~ $ modprobe multipath
    
  11. Return to the virtual terminal and select the installer dialog:

    Detect disks

    and acknowledge the question regarding parameters for the dm-emc kernel module.

  12. Select the installer dialog:

    Partition disks

    and select the:

    Manual

    paritioning method. Then select the entry:

    LVM VG mpatha, LV mpatha - 38.7 GB Linux device-mapper (multipath)

    Do NOT select one of the entries starting with “SCSI”! If there is no entry offering a “mpath” device path, the most likely cause is that the above step no. 10, loading the multipath kernel module, failed. In this case check the file /var/log/syslog for error messages.

    Select “Yes”“ to create a new partition table

    +-----------------------+ [!!] Partition disks +------------------------+
    |                                                                       |
    | You have selected an entire device to partition. If you proceed with  |
    | creating a new partition table on the device, then all current        |
    | partitions will be removed.                                           |
    |                                                                       |
    | Note that you will be able to undo this operation later if you wish.  |
    |                                                                       |
    | Create new empty partition table on this device?                      |
    |                                                                       |
    |     <Go Back>                                       <Yes>    <No>     |
    |                                                                       |
    +-----------------------------------------------------------------------+
    

    Select ”msdos“ as a partition table type:

    +-------------+ [.] Partition disks +--------------+
    |                                                  |
    | Select the type of partition table to use.       |
    |                                                  |
    | Partition table type:                            |
    |                                                  |
    |                     aix                          |
    |                     amiga                        |
    |                     bsd                          |
    |                     dvh                          |
    |                     gpt                          |
    |                     mac                          |
    |                     msdos                        |
    |                     sun                          |
    |                     loop                         |
    |                                                  |
    |     <Go Back>                                    |
    |                                                  |
    +--------------------------------------------------+
    

    Select the ”FREE SPACE“ entry on the mpath device to create new partitions:

    +------------------------+ [!!] Partition disks +-------------------------+
    |                                                                         |
    | This is an overview of your currently configured partitions and mount   |
    | points. Select a partition to modify its settings (file system, mount   |
    | point, etc.), a free space to create partitions, or a device to         |
    | initialize its partition table.                                         |
    |                                                                         |
    |  Guided partitioning                                                    |
    |  Configure software RAID                                                |
    |  Configure the Logical Volume Manager                                   |
    |  Configure encrypted volumes                                            |
    |                                                                         |
    |  LVM VG mpatha, LV mpatha - 38.7 GB Linux device-mapper (multipath      |
    |          pri/log  38.7 GB      FREE SPACE                               |
    |  SCSI1 (0,1,0) (sda) - 38.7 GB AIX VDASD                                |
    |  SCSI2 (0,1,0) (sdb) - 38.7 GB AIX VDASD                                |
    |                                                                         |
    |                                                                         |
    |     <Go Back>                                                           |
    |                                                                         |
    +-------------------------------------------------------------------------+
    

    Select ”Create a new partition“:

    +-----------+ [!!] Partition disks +-----------+
    |                                              |
    | How to use this free space:                  |
    |                                              |
    |  Create a new partition                      |
    |  Automatically partition the free space      |
    |  Show Cylinder/Head/Sector information       |
    |                                              |
    |     <Go Back>                                |
    |                                              |
    +----------------------------------------------+
    

    Enter “8” to create a parition of about 7MB size:

    +-----------------------+ [!!] Partition disks +------------------------+
    |                                                                       |
    | The maximum size for this partition is 38.7 GB.                       |
    |                                                                       |
    | Hint: "max" can be used as a shortcut to specify the maximum size, or |
    | enter a percentage (e.g. "20%") to use that percentage of the maximum |
    | size.                                                                 |
    |                                                                       |
    | New partition size:                                                   |
    |                                                                       |
    | 8____________________________________________________________________ |
    |                                                                       |
    |     <Go Back>                                          <Continue>     |
    |                                                                       |
    +-----------------------------------------------------------------------+
    

    Select ”Primary“ as a partition type:

    +-----+ [!!] Partition disks +------+
    |                                   |
    | Type for the new partition:       |
    |                                   |
    |            Primary                |
    |            Logical                |
    |                                   |
    |     <Go Back>                     |
    |                                   |
    +-----------------------------------+
    

    Select ”Beginning“ as the location for the new parition:

    +------------------------+ [!!] Partition disks +-------------------------+
    |                                                                         |
    | Please choose whether you want the new partition to be created at the   |
    | beginning or at the end of the available space.                         |
    |                                                                         |
    | Location for the new partition:                                         |
    |                                                                         |
    |                              Beginning                                  |
    |                              End                                        |
    |                                                                         |
    |     <Go Back>                                                           |
    |                                                                         |
    +-------------------------------------------------------------------------+
    

    Select ”PowerPC PReP boot partition“ as the parition usage and set the ”Bootable flag“ to ”on“:

    +------------------------+ [!!] Partition disks +-------------------------+
    |                                                                         |
    | You are editing partition #1 of LVM VG mpatha, LV mpatha. No existing   |
    | file system was detected in this partition.                             |
    |                                                                         |
    | Partition settings:                                                     |
    |                                                                         |
    |             Use as:         PowerPC PReP boot partition                 |
    |                                                                         |
    |             Bootable flag:  on                                          |
    |                                                                         |
    |             Copy data from another partition                            |
    |             Delete the partition                                        |
    |             Done setting up the partition                               |
    |                                                                         |
    |     <Go Back>                                                           |
    |                                                                         |
    +-------------------------------------------------------------------------+
    

    Repeat the above steps two more times and create the following parition layout. Then select ”Configure the Logical Volume Manager“ and acknowledge the partition scheme to be written to disk:

    +------------------------+ [!!] Partition disks +-------------------------+
    |                                                                         |
    | This is an overview of your currently configured partitions and mount   |
    | points. Select a partition to modify its settings (file system, mount   |
    | point, etc.), a free space to create partitions, or a device to         |
    | initialize its partition table.                                         |
    |                                                                         |
    |  Guided partitioning                                                    |
    |  Configure software RAID                                                |
    |  Configure the Logical Volume Manager                                   |
    |  Configure encrypted volumes                                            |
    |                                                                         |
    |  LVM VG mpatha, LV mpatha - 38.7 GB Linux device-mapper (multipath      |
    |  >     #1  primary    7.3 MB  B  K                                      |
    |  >     #2  primary  511.7 MB     f  ext4    /boot                       |
    |  >     #5  logical   38.1 GB     K  lvm                                 |
    |  SCSI1 (0,1,0) (sda) - 38.7 GB AIX VDASD                                |
    |                                                                         |
    |     <Go Back>                                                           |
    |                                                                         |
    +-------------------------------------------------------------------------+
    

    Select ”Create volume group“:

    +----------+ [!!] Partition disks +-----------+
    |                                             |
    | Summary of current LVM configuration:       |
    |                                             |
    |  Free Physical Volumes:  1                  |
    |  Used Physical Volumes:  0                  |
    |  Volume Groups:          0                  |
    |  Logical Volumes:        0                  |
    |                                             |
    | LVM configuration action:                   |
    |                                             |
    |      Display configuration details          |
    |      Create volume group                    |
    |      Finish                                 |
    |                                             |
    |     <Go Back>                               |
    |                                             |
    +---------------------------------------------+
    

    Enter a volume group name, here ”vg00“:

    +-----------------------+ [!!] Partition disks +------------------------+
    |                                                                       |
    | Please enter the name you would like to use for the new volume group. |
    |                                                                       |
    | Volume group name:                                                    |
    |                                                                       |
    | vg00_________________________________________________________________ |
    |                                                                       |
    |     <Go Back>                                          <Continue>     |
    |                                                                       |
    +-----------------------------------------------------------------------+
    

    Select the device ”/dev/mapper/mpathap5“ created above as a physical volume for the volume group:

    +-----------------+ [!!] Partition disks +------------------+
    |                                                           |
    | Please select the devices for the new volume group.       |
    |                                                           |
    | You can select one or more devices.                       |
    |                                                           |
    | Devices for the new volume group:                         |
    |                                                           |
    |      [ ] /dev/mapper/mpathap1           (7MB)             |
    |      [ ] /dev/mapper/mpathap2           (511MB; ext4)     |
    |      [*] /dev/mapper/mpathap5           (38133MB)         |
    |      [ ] (38133MB; lvm)                                   |
    |                                                           |
    |     <Go Back>                              <Continue>     |
    |                                                           |
    +-----------------------------------------------------------+
    

    Create logical volumes according to your needs and environment. See the following list as an example. Note that the logical volume for the root filesystem resides inside the volume group:

    +----------------------+ [!!] Partition disks +----------------------+
    |                                                                    |
    |                     Current LVM configuration:                     |
    | Unallocated physical volumes:                                      |
    |   * none                                                           |
    |                                                                    |
    | Volume groups:                                                     |
    |   * vg00                                                 (38130MB) |
    |     - Uses physical volume:         /dev/mapper/mpathap5 (38130MB) |
    |     - Provides logical volume:      home                 (1023MB)  |
    |     - Provides logical volume:      root                 (1023MB)  |
    |     - Provides logical volume:      srv                  (28408MB) |
    |     - Provides logical volume:      swap                 (511MB)   |
    |     - Provides logical volume:      tmp                  (1023MB)  |
    |     - Provides logical volume:      usr                  (4093MB)  |
    |     - Provides logical volume:      var                  (2046MB)  |
    |                                                                    |
    |                             <Continue>                             |
    |                                                                    |
    +--------------------------------------------------------------------+
    

    Assign filesystems to each of the logical volumes created above. See the following list as an example. Note that only the “PowerPC PReP boot partition” and the partition for the ”/boot“ filesystem resides outside the volume group:

    +------------------------+ [!!] Partition disks +-------------------------+
    |                                                                         |
    | This is an overview of your currently configured partitions and mount   |
    | points. Select a partition to modify its settings (file system, mount   |
    | point, etc.), a free space to create partitions, or a device to         |
    | initialize its partition table.                                         |
    |                                                                         |
    |  Guided partitioning                                                    |
    |  Configure software RAID                                                |
    |  Configure the Logical Volume Manager                                   |
    |  Configure encrypted volumes                                            |
    |                                                                         |
    |  LVM VG mpatha, LV mpatha - 38.7 GB Linux device-mapper (multipath      |
    |  >     #1  primary    7.3 MB  B  K                                      |
    |  >     #2  primary  511.7 MB        ext4                                |
    |  >     #5  logical   38.1 GB     K  lvm                                 |
    |  LVM VG mpathap1, LV mpathap1 - 7.3 MB Linux device-mapper (linear      |
    |  LVM VG mpathap2, LV mpathap2 - 511.7 MB Linux device-mapper (line      |
    |  >     #1           511.7 MB     K  ext4    /boot                       |
    |  LVM VG mpathap5, LV mpathap5 - 38.1 GB Linux device-mapper (linea      |
    |  LVM VG vg00, LV home - 1.0 GB Linux device-mapper (linear)             |
    |  >     #1             1.0 GB     f  ext4    /home                       |
    |  LVM VG vg00, LV root - 1.0 GB Linux device-mapper (linear)             |
    |  >     #1             1.0 GB     f  ext4    /                           |
    |  LVM VG vg00, LV srv - 28.4 GB Linux device-mapper (linear)             |
    |  >     #1            28.4 GB     f  ext4    /srv                        |
    |  LVM VG vg00, LV swap - 511.7 MB Linux device-mapper (linear)           |
    |  >     #1           511.7 MB     f  swap    swap                        |
    |  LVM VG vg00, LV tmp - 1.0 GB Linux device-mapper (linear)              |
    |  >     #1             1.0 GB     f  ext4    /tmp                        |
    |  LVM VG vg00, LV usr - 4.1 GB Linux device-mapper (linear)              |
    |  >     #1             4.1 GB     f  ext4    /usr                        |
    |  LVM VG vg00, LV var - 2.0 GB Linux device-mapper (linear)              |
    |  >     #1             2.0 GB     f  ext4    /var                        |
    |  SCSI1 (0,1,0) (sda) - 38.7 GB AIX VDASD                                |
    |  SCSI2 (0,1,0) (sdb) - 38.7 GB AIX VDASD                                |
    |                                                                         |
    |  Undo changes to partitions                                             |
    |  Finish partitioning and write changes to disk                          |
    |                                                                         |
    |     <Go Back>                                                           |
    |                                                                         |
    +-------------------------------------------------------------------------+
    

    Select ”Finish partitioning and write changes to disk“.

  13. Select the installer dialog:

    Install the base system

    Choose a Linux kernel to be installed and select a “targeted” initrd to be created.

  14. Log into the system via SSH again and manually install the necessary multipath packages in the target system:

    ~ $ chroot /target /bin/bash
    root@ststdebian01:/$ mount /sys
    root@ststdebian01:/$ mount /proc
    root@ststdebian01:/$ apt-get install multipath-tools multipath-tools-boot
    
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    The following extra packages will be installed:
      kpartx libaio1
    The following NEW packages will be installed:
      kpartx libaio1 multipath-tools multipath-tools-boot
    0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
    Need to get 0 B/258 kB of archives.
    After this operation, 853 kB of additional disk space will be used.
    Do you want to continue [Y/n]? y
    Preconfiguring packages ...
    Can not write log, openpty() failed (/dev/pts not mounted?)
    Selecting previously unselected package libaio1:powerpc.
    (Reading database ... 14537 files and directories currently installed.)
    Unpacking libaio1:powerpc (from .../libaio1_0.3.109-3_powerpc.deb) ...
    Selecting previously unselected package kpartx.
    Unpacking kpartx (from .../kpartx_0.4.9+git0.4dfdaf2b-7~deb7u1_powerpc.deb) ...
    Selecting previously unselected package multipath-tools.
    Unpacking multipath-tools (from .../multipath-tools_0.4.9+git0.4dfdaf2b-7~deb7u1_powerpc.deb) ...
    Selecting previously unselected package multipath-tools-boot.
    Unpacking multipath-tools-boot (from .../multipath-tools-boot_0.4.9+git0.4dfdaf2b-7~deb7u1_all.deb) ...
    Processing triggers for man-db ...
    Can not write log, openpty() failed (/dev/pts not mounted?)
    Setting up libaio1:powerpc (0.3.109-3) ...
    Setting up kpartx (0.4.9+git0.4dfdaf2b-7~deb7u1) ...
    Setting up multipath-tools (0.4.9+git0.4dfdaf2b-7~deb7u1) ...
    [ ok ] Starting multipath daemon: multipathd.
    Setting up multipath-tools-boot (0.4.9+git0.4dfdaf2b-7~deb7u1) ...
    update-initramfs: deferring update (trigger activated)
    Processing triggers for initramfs-tools ...
    update-initramfs: Generating /boot/initrd.img-3.2.0-4-powerpc64
    cat: /sys/devices/vio/modalias: No such device
    
    root@ststdebian01:/$ vi /etc/multipath.conf
    
    defaults {
        getuid_callout  "/lib/udev/scsi_id -g -u -d /dev/%n"
        no_path_retry  10
        user_friendly_names yes
    }
    blacklist {
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^hd[a-z][[0-9]*]"
        devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
    }
    multipaths {
        multipath {
            wwid    360050768019181279800000000000329
        }
    }
    

    Adjust the configuration items in /etc/multipath.conf according to your needs and environment, especially the disk UUID value in the wwid field.

  15. Return to the virtual terminal and go through the following installer dialogs:

    Configure the package manager
    Select and install software

    and select the configuration items according to your needs and environment.

  16. Select the installer dialog:

    Install yaboot on a hard disk

    and select /dev/mapper/mpathap1 as the device where the yaboot boot loader should be installed:

    +-----------------+ [!!] Install yaboot on a hard disk +------------------+
    |                                                                         |
    | Yaboot (the Linux boot loader) needs to be installed on a hard disk     |
    | partition in order for your system to be bootable.  Please choose the   |
    | destination partition from among these partitions that have the         |
    | bootable flag set.                                                      |
    |                                                                         |
    | Warning: this will erase all data on the selected partition!            |
    |                                                                         |
    | Device for boot loader installation:                                    |
    |                                                                         |
    |                         /dev/mapper/mpathap1                            |
    |                                                                         |
    |     <Go Back>                                                           |
    |                                                                         |
    +-------------------------------------------------------------------------+
    
  17. After the yaboot boot loader has been successfully installed, log into the system via SSH again. Change the device path for the boot partition, to reflect the naming scheme that will be present on the target system once it is running:

    ~ $ chroot /target /bin/bash
    root@ststdebian01:/$ vi /etc/fstab
    
    #/dev/mapper/mpathap2 /boot           ext4    defaults        0       2
    /dev/mapper/mpatha-part2 /boot           ext4    defaults        0       2
    

    Check the initrd for the inclusion of the multipath tools installed in the previous step:

    root@ststdebian01:/$ gzip -dc /boot/initrd.img-3.2.0-4-powerpc64 | cpio -itvf - | grep -i multipath
    
    -rwxr-xr-x   1 root     root        18212 Oct 15 05:16 sbin/multipath
    -rw-r--r--   1 root     root          392 Dec 16 18:03 etc/multipath.conf
    -rw-r--r--   1 root     root          328 Oct 15 05:16 lib/udev/rules.d/60-multipath.rules
    drwxr-xr-x   2 root     root            0 Dec 16 18:03 lib/multipath
    -rw-r--r--   1 root     root         5516 Oct 15 05:16 lib/multipath/libpriohp_sw.so
    -rw-r--r--   1 root     root         9620 Oct 15 05:16 lib/multipath/libcheckrdac.so
    -rw-r--r--   1 root     root         9632 Oct 15 05:16 lib/multipath/libpriodatacore.so
    -rw-r--r--   1 root     root        13816 Oct 15 05:16 lib/multipath/libchecktur.so
    -rw-r--r--   1 root     root         9664 Oct 15 05:16 lib/multipath/libcheckdirectio.so
    -rw-r--r--   1 root     root         5516 Oct 15 05:16 lib/multipath/libprioemc.so
    -rw-r--r--   1 root     root         9576 Oct 15 05:16 lib/multipath/libcheckhp_sw.so
    -rw-r--r--   1 root     root         9616 Oct 15 05:16 lib/multipath/libpriohds.so
    -rw-r--r--   1 root     root         9672 Oct 15 05:16 lib/multipath/libprioiet.so
    -rw-r--r--   1 root     root         5404 Oct 15 05:16 lib/multipath/libprioconst.so
    -rw-r--r--   1 root     root         9608 Oct 15 05:16 lib/multipath/libcheckemc_clariion.so
    -rw-r--r--   1 root     root         5516 Oct 15 05:16 lib/multipath/libpriordac.so
    -rw-r--r--   1 root     root         5468 Oct 15 05:16 lib/multipath/libpriorandom.so
    -rw-r--r--   1 root     root         9620 Oct 15 05:16 lib/multipath/libprioontap.so
    -rw-r--r--   1 root     root         5488 Oct 15 05:16 lib/multipath/libcheckreadsector0.so
    -rw-r--r--   1 root     root         9656 Oct 15 05:16 lib/multipath/libprioweightedpath.so
    -rw-r--r--   1 root     root         9692 Oct 15 05:16 lib/multipath/libprioalua.so
    -rw-r--r--   1 root     root         9592 Oct 15 05:16 lib/multipath/libcheckcciss_tur.so
    -rw-r--r--   1 root     root        43552 Sep 20 06:13 lib/modules/3.2.0-4-powerpc64/kernel/drivers/md/dm-multipath.ko
    -rw-r--r--   1 root     root       267620 Oct 15 05:16 lib/libmultipath.so.0
    -rwxr-xr-x   1 root     root          984 Oct 14 22:25 scripts/local-top/multipath
    -rwxr-xr-x   1 root     root          624 Oct 14 22:25 scripts/init-top/multipath
    

    If the multipath tools, which were installed in the previous step, are missing, update the initrd to include them:

    root@ststdebian01:/$ update-initramfs -u -k all
    

    Check the validity of the yaboot configuration, especially the configuration items marked with ”⇐= !!!“:

    root@ststdebian01:/$ vi /etc/yaboot.conf
    
    boot="/dev/mapper/mpathap1"         <== !!!
    partition=2
    root="/dev/mapper/vg00-root"        <== !!!
    timeout=50
    install=/usr/lib/yaboot/yaboot
    enablecdboot
    
    image=/vmlinux                      <== !!!
            label=Linux
            read-only
            initrd=/initrd.img          <== !!!
    
    image=/vmlinux.old                  <== !!!
            label=old
            read-only
            initrd=/initrd.img.old      <== !!!
    

    If changes were made to the yaboot configuration, install the boot loader again:

    root@ststdebian01:/$ ybin -v
    

    In order to have the root filesystem on LVM and yaboot still be able to access its necessary configuration, the yaboot.conf has to be placed in a subdirectory ”/etc/“ on a non-LVM device. See HOWTO-Booting with Yaboot on PowerPC for a detailed explaination which is also true for the yaboot.conf:

    It's worth noting that yaboot locates the kernel image within a partition's filesystem without regard to where that partition will eventually be mounted within the Linux root filesystem. So, for example, if you've placed a kernel image or symlink at /boot/vmlinux, but /boot is actually a separate partition on your system, then the image path for yaboot will just be image=/vmlinux.

    In this case the kernel and initrd images reside on the ”/boot“ filesystem, so the path ”/etc/yaboot.conf“ has to be placed below ”/boot“. A symlink pointing from ”/etc/yaboot.conf“ to ”/boot/etc/yaboot.conf“ is added for the convenience of tools that expect it still to be in ”/etc“:

    root@ststdebian01:/$ mkdir /boot/etc
    root@ststdebian01:/$ mv -i /etc/yaboot.conf /boot/etc/
    root@ststdebian01:/$ ln -s /boot/etc/yaboot.conf /etc/yaboot.conf
    root@ststdebian01:/$ ls -al /boot/etc/yaboot.conf /etc/yaboot.conf
    
    -rw-r--r-- 1 root root 547 Dec 16 18:20 /boot/etc/yaboot.conf
    lrwxrwxrwx 1 root root  21 Dec 16 18:48 /etc/yaboot.conf -> /boot/etc/yaboot.conf
    
  18. Select the installer dialog:

    Finish the installation
  19. After a successful reboot, change the yaboot ”boot=“ configuration directive (marked with ”⇐= !!!“) to use the device path name present in the now running system:

    root@ststdebian01:/$ vi /etc/yaboot.conf
    
    boot="/dev/mapper/mpatha-part1"     <== !!!
    partition=2
    root="/dev/mapper/vg00-root"
    timeout=50
    install=/usr/lib/yaboot/yaboot
    enablecdboot
    
    image=/vmlinux
            label=Linux
            read-only
            initrd=/initrd.img
    
    image=/vmlinux.old
            label=old
            read-only
            initrd=/initrd.img.old
    
    root@ststdebian01:/$ ybin -v
    
  20. Check the multipath status of the newly installed, now running system:

    root@ststdebian01:~$ multipath -ll
    
    mpatha (360050768019181279800000000000329) dm-0 AIX,VDASD
    size=36G features='1 queue_if_no_path' hwhandler='0' wp=rw
    `-+- policy='round-robin 0' prio=0 status=active
      |- 0:0:1:0 sda 8:0  active ready running
      `- 1:0:1:0 sdb 8:16 active ready running
    
    root@ststdebian01:~$ df -h
    
    Filesystem                Size  Used Avail Use% Mounted on
    rootfs                    961M  168M  745M  19% /
    udev                       10M     0   10M   0% /dev
    tmpfs                     401M  188K  401M   1% /run
    /dev/mapper/vg00-root     961M  168M  745M  19% /
    tmpfs                     5.0M     0  5.0M   0% /run/lock
    tmpfs                     801M     0  801M   0% /run/shm
    /dev/mapper/mpatha-part2  473M   28M  421M   7% /boot
    /dev/mapper/vg00-home     961M   18M  895M   2% /home
    /dev/mapper/vg00-srv       27G  172M   25G   1% /srv
    /dev/mapper/vg00-tmp      961M   18M  895M   2% /tmp
    /dev/mapper/vg00-usr      3.8G  380M  3.2G  11% /usr
    /dev/mapper/vg00-var      1.9G  191M  1.6G  11% /var
    
    root@ststdebian01:~$ mount | grep mapper
    
    /dev/mapper/vg00-root on / type ext4 (rw,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)
    /dev/mapper/mpatha-part2 on /boot type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
    /dev/mapper/vg00-home on /home type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
    /dev/mapper/vg00-srv on /srv type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
    /dev/mapper/vg00-tmp on /tmp type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
    /dev/mapper/vg00-usr on /usr type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
    /dev/mapper/vg00-var on /var type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
    
    root@ststdebian01:~$ pvdisplay
    
      --- Physical volume ---
      PV Name               /dev/mapper/mpatha-part5
      VG Name               vg00
      PV Size               35.51 GiB / not usable 3.00 MiB
      Allocatable           yes (but full)
      PE Size               4.00 MiB
      Total PE              9091
      Free PE               0
      Allocated PE          9091
      PV UUID               gO0PTK-xi9b-H4om-BRgj-rEy2-NSzC-UokjEw
    

Although it is a rather manual and error prone process, it is nonetheless possible to install Debian Wheezy with multipathing enabled from the start. Another and probably easier method would be to map the disk through only one VIO server during the installation phase and add the second mapping as well as the necessary multipath configuration after the installed system has been successfully brought up.

Some of the manual steps above should IMHO be handled automatically by the Debian installer. For example, why the missing packages multipath-tools and multipath-tools-boot aren't installed automatically eludes me. So there seems to be at least some room for improvement.

First tests with the Debian installer of the upcoming “Jessie” release were promising, but the completion of an install fails at the moment due to unresolved package dependencies. But that'll be the subject for future posts … ;-)

Comments

andy
No. 1 @ 2014/04/21 21:04

Thank you very much for your explanation. Very useful for me.

Leave a comment…




E H Q E Q
  • E-Mail address will not be published.
  • Formatting:
    //italic//  __underlined__
    **bold**  ''preformatted''
  • Links:
    [[http://example.com]]
    [[http://example.com|Link Text]]
  • Quotation:
    > This is a quote. Don't forget the space in front of the text: "> "
  • Code:
    <code>This is unspecific source code</code>
    <code [lang]>This is specifc [lang] code</code>
    <code php><?php echo 'example'; ?></code>
    Available: html, css, javascript, bash, cpp, …
  • Lists:
    Indent your text by two spaces and use a * for
    each unordered list item or a - for ordered ones.
This website uses cookies for visitor traffic analysis. By using the website, you agree with storing the cookies on your computer.More information