Skip to main content

Multipathd device enumeration

Testing my new kickstart file to install Linux on my old Poweredge 2950 dell servers is fine, but multipath presents me with a new problem.

We boot from SAN, even though the servers have local disks available. Eventually we will move towards Blades, so the experience is useful. But each server has a different number of LUNs available to it. The boot LUN is always LUN id 0. Without multipathd, this gave me /dev/sdb as the boot device. With multipathd, this gives me /dev/mapper/mpathX, where X is the number of LUNs I have. You may see the problem:
The kickstart file must list the boot drive order and designate a place to put the MBR. However, I can't think of a way to tell kickstart to use the last mpath device... I wish there'd be a way to make /dev/mapper/mpath0 the boot LUN...

Update: Well, it seems to be a limitation of multipath support in anaconda. I've booted the OEL 5.3 DVD ISO on 3 different bare metals and each server has a different number of LUNs associated with it. Other than that, they are identical.
On a machine with one boot LUN only, the device name is /dev/mapper/mapth0 and using that in kickstart works. On the second machine with 5 LUNs, the boot LUN becomes mapper/mpath4 even though the LUNs are numbered starting at zero for the boot LUN. Another machine with various sizes of LUNs, in total 7, has the same issue: the boot LUN - irregardless of its LUN id - becomes mapper/mpath7. Hence, the name of the multipathed boot device is dependent on the number of LUNs associated at boot time and not related to the LUN Id numbering set on the SAN-side (EMC DMX-3 in our case). This means you can't specify it in a general kickstart script but must make one for each machine, or only deploy onto machines with only a single boot LUN and add additional LUNs later on.

Comments

Konrad Rzeszutek Wilk said…
To make the /dev/mpath/ the same across all your machines you can disable the 'user_friendly_names' option.

1). Modify multipath.conf to change 'user_friendly_names' to 'user_friendly_names yes'.

2). Kick multipathd to re-read: multipathd -k'reconfigure'.

And it ought to recreate the /dev/mapper/. I say 'ought' to as I don't remember if the daemon is smart enough to delete the old ones (mpath1, etc) and create the new ones.

The WWID is unique as it is set by the SAN vendor.
Dreams said…
Thanks for the help, I appreciate it!
But in 1) you say 'disable' but set user_friendly_names to 'yes' whereas I think it should be 'no' in your case.
Also, I know the SAN determines the WWID of a LUN. My problem is not that I get a different mpath# device each reboot, but that the LUN with the lowest LUN id (i.e. '0' for a boot LUN) is not the mpath device with the lowest enumerator, e.g. mpath0. Quite the opposite, my boot LUNs seem to end up as the last mpath device.
I would expect:
LUN id 1 (17GB) -> /dev/mpath0
LUN id 2 (34GB) -> /dev/mpath1
LUN id 13 (500GB) -> /dev/mpath2
LUN id 64 (1700GB) -> /dev/mpath3

Popular posts from this blog

Tuning the nscd name cache daemon

I've been playing a bit with the nscd now and want to share some tips related to tuning the nscd.conf file. To see how the DNS cache is doing, use nscd -g. nscd configuration: 0 server debug level 26m 57s server runtime 5 current number of threads 32 maximum number of threads 0 number of times clients had to wait yes paranoia mode enabled 3600 restart internal passwd cache: no cache is enabled [other zero output removed] group cache: no cache is enabled [other zero output removed] hosts cache: yes cache is enabled yes cache is persistent yes cache is shared 211 suggested size <==== 216064 total data pool size 1144 used data pool size 3600 seconds time to live for positive entries <==== 20 seconds time to live for negative entries

Preventing PuTTY timeouts

Just found a great tip to prevent timeouts of PuTTY sessions. I'm fine with timeouts by the host, but in our case the firewall kills sessions after 30 minutes of inactivity... When using PuTTY to ssh to your Linux/Unix servers, be sure to use the feature to send NULL packets to prevent a timeout. I've set it to once every 900 seconds, i.e. 15 minutes... See screenshot on the right.

Setting up SR-IOV in RHEL6 on PowerEdge servers

Dell Community : "RHEL 6 provides SR-IOV functionality on supported hardware which provides near native performance for virtualized guests. Single-Root I/O Virtualization (SR-IOV) specification, introduced by PCI-SIG details how a single PCIe device can be shared between various virtualization guests. Devices capable of SR-IOV functionality support multiple virtual functions on top of the physical function. Virtual Function is enabled in hardware as a light weight PCIe function. Operating System cannot discover this function as it does not respond to the PCI bus scan and requires support in the host’s driver. As in PCIe pass-through, a Virtual function of a SR-IOV capable card can be directly assigned to the guest operating system. A virtual function driver running in the guest manages this device."