Map Disk Block devices on a Linux host back to the SAN Lun

There was a question posted in Oracle-L - How can one map Disk Devices (On Linux) back to the SAN LUNs? This is a no brainer on Solaris virtue of persistent binding capabilities on the HBA along with hard-coding the luns in sd.conf, but Linux is a different ball game.

Most of you would wonder – why do I care to know about this mapping? You need to - because as a DBA if you are seeing high latencies for a specific data file(s), you would need to know to which LUN , the block device (containing the data file) ties back to on the array. SAN admins work with LUNs, not with system block devices.

In Linux, the block devices are named starting with "sd" with little information as to

1. Specific Adaptor/Port(HBA) through which it is mapped.

2. Array on which the LUN is carved out.

3. The specific LUN number on the Array

While it is not impossible to find out the above information, there is no standard interface which can help you determine the above information (I may be mistaken - there may be a tool out there which I am not aware of).

On Redhat 4.x and higher versions using the Kernel 2.6.x and higher, the sysfs implementation is standard. The /sys filesystem holds the key to identifying the above information.

Unfortunately, the /sys filesystem is not well documented and is full of shortcuts pointing back
and forth. In short it is a maze. But it is Linux and so we are not surprised. So we have to devise our own methods to overcome these shortcomings.

Let us use an example to illustrate how it can be done. Let us assume that our database is showing high latencies for datafiles residing on the block device sdf.

First you need to identify how many HBAs you have on the system and how many ports/HBA
(HBAs come in single port or dual port configurations).

You can identify this in multiple ways - I prefer to use lspci

$ /sbin/lspci |grep Fibre


0f:00.0 Fibre Channel: Emulex Corporation Zephyr-X LightPulse Fibre Channel Host Adapter (rev 02) -------------- > HBA 1 Port 1

0f:00.1 Fibre Channel: Emulex Corporation Zephyr-X LightPulse Fibre Channel Host Adapter (rev 02)-------------- > HBA 1 Port 2

11:00.0 Fibre Channel: Emulex Corporation Zephyr-X LightPulse Fibre Channel Host Adapter (rev 02) -------------- > HBA 2 Port 1

11:00.1 Fibre Channel: Emulex Corporation Zephyr-X LightPulse Fibre Channel Host Adapter (rev 02) -------------- > HBA 2 Port 2

As you can see, it shows that there are 2 Dual Port Emulex HBAs installed in the system. LUNs
can be visible to Linux through any or all of the 4 available ports.

So the block device sdf as seen by the OS is visible via one of these 4 available ports (paths). Next, we need to identify the specific port through which this LUN sdf is visible to the OS.

For that, we change directories to /sys/block and run a find.

$ cd /sys/block

$ find . -name device -exec ls -l {} \; |grep sdf

lrwxrwxrwx 1 root root 0 Jul 21 17:25 ./sdf/device -> ../../devices/pci0000:00/0000:00:04.0/0000:0f:00.0/host1/target1:0:0/1:0:0:4

or another way is to

$ cd /sys/block/sdf

$ ls -l device
lrwxrwxrwx 1 root root 0 Jul 21 17:25 device ->
../../devices/pci0000:00/0000:00:04.0/0000:0f:00.0/host1/target1:0:0/1:0:0:4

As you can see from the above, we can identify that sdf is visible to the OS via the HBA Port 0f:00.0 to the OS. 0f:00.0 as we have seen above is HBA 1 Port 1. This output has several more interesting tid-bits of information.

lrwxrwxrwx 1 root root 0 Jul 21 17:25 device -> ../../devices/pci0000:00/0000:00:04.0/0000:0f:00.0/host1/target1:0:0/1:0:0:4

The target is 0 from 1:0:0:4 or from target1:0:0. We can also learn from the above output that this is LUN 4 from 1:0:0:4.

All the remains now is to identify the array which is presenting LUN 4 to the OS. This
information is available from the /proc interface.

$ cd /proc/scsi/lpfc

$ ls
1 2 3 4

Here the numbers 1-4 correspond to the specific HBA ports 1 - 4. We have identified that LUN sdf is on HBA 1 Port 1 and so we look into the contents of "1". We also know it is Target 0.

$ more 1
lpfc0t01 DID 610413 WWPN 50:06:0e:80:00:c3:bd:40 WWNN 50:06:0e:80:00:c3:bd:40
lpfc0t02 DID 610a13 WWPN 50:06:0e:80:10:09:d0:02 WWNN 50:06:0e:80:10:09:d0:02
lpfc0t00 DID 612413 WWPN 50:06:0e:80:10:09:d0:07 WWNN 50:06:0e:80:10:09:d0:07

We see that Target 0 is mapped to WWPN ending in 07. This is the Array WWPN from which the Storage Admin can identify the specific Array which is presenting LUN 4 (sdf) to the host.

So we have identified that the block device sdf is

1. Visible to the OS via HBA 1 Port 1.

2. Target 0 for the HBA.

3. LUN 4 as presented to the OS by the Array.

4. The array that is presenting LUN 4 to the OS is 50:06:0e:80:10:09:d0:07.

With this above information, your Storage Admin will get a headstart on fixing the problem for
you.