Scst and scstadmin on Ubuntu 11.04

Submitted by Tom L'Ecluse on Fri, 02/09/2011 - 18:25

After searching a lot for documentation on how to get fiber channel running on ubuntu I've found some snippets which has let to this how to. I must warn people this is not for the ubuntu newbies! Yes you can follow the guide to get it working but if anything is different from my setup then this guide won't help that much.

Install necessairy packages

First we need to install some packages that we will need to compile our kernel and other packages for getting scst & scstadmin

sudo apt-get install fakeroot kernel-wedge build-essential makedumpfile kernel-package libncurses5 libncurses5-dev

Then run the following command:

sudo apt-get build-dep --no-install-recommends linux-image-$(uname -r)

Get the current kernel source

We need to fetch the kernel that we want to be modified so it has the qla2xxx target driver.

mkdir /usr/src
cd /usr/src
apt-get source linux-image-$(uname -r)
ln -s /usr/src/linux-2.6.38 linux
cd linux-2.6.38

At the time of writing 2.6.38 was the current kernel source. The ln command is just to make things a little easier for the guide. It isn't required if you know what you are doing.

It is a good idea to start with the same .config as the currently running kernel, so type the following,

cp -vi /boot/config-`uname -r` .config

Modify the kernel for scst

First we need to get the scst & scstadmin source:

svn co https://scst.svn.sourceforge.net/svnroot/scst

Note that this package is under my current users his how directory!

Next we patch the kernel

cp ~/scst/trunk/scst/kernel/scst_exec_req_fifo-2.6.38.patch /usr/src/linux/
cd /usr/src/linux/
patch -p1 < scst_exec_req_fifo-2.6.38.patch

Thanks to the ln -s /usr/src/linux-2.6.38 linux command, I can now use /usr/src/linux in the previous command instead of cp ~/scst/trunk/scst/kernel/scst_exec_req_fifo-2.6.38.patch /usr/src/linux-2.6.38.

Notice that there are more patch files. Every kernel has a specific patch file. So if you are using a newer / older kernel. Use a different patch file which matches your kernel!

Replace the QLogic FC driver code in the kernel source tree with the modified version of QLogic FC driver code from scst code base.

mv /usr/src/linux/drivers/scsi/qla2xxx /usr/src/linux/drivers/scsi/qla2xxx_orig
ln -s ~/scst/trunk/qla2x00t /usr/src/linux/drivers/scsi/qla2xxx

Build the kernel

enable "Device Drivers->SCSI device support->SCSI low level drivers->Qlogic 2xxx target mode support"

make menuconfig

Navigate through the menu to enable the qlogic 2xxx target mode support. Then just exit and awnser "yes" to the question to save the config.
It is now time to compile. However to speed up the build if you have a quad core processor type,
 

export CONCURRENCY_LEVEL=5

General rule, concurrency level = number of processor cores + 1

make-kpkg clean
fakeroot make-kpkg --initrd --append-to-version=-some-string-here kernel-image kernel-headers

Remember to substitute the writing in green for something else, for example -scst

After a few minutes or hours your kernel compile will be complete.

Install the new kernel

Next we need to install the new kernel:

cd /usr/src/
dpkg -i linux-image-2.6.38.2-scst_2.6.38.2-scst-10.00.Custom_amd64.deb
dpkg -i linux-headers-2.6.38.2-scst_2.6.38.2-scst-10.00.Custom_amd64.deb

Replace scst by the string from the previous step. Also you can find the exact string by doing:

ls /usr/src/

Reboot the server

sudo reboot


Install scstadmin

After that we have compiled the kernel we need scstadmin tool to configure our fiber channel.

cd ~/scst/trunk/scstadmin
make
make instal

Configure scstadmin

Find your WWN from your openfilers fiber card using the command:

cat /sys/class/fc_host/host*/port_name


And write them down. I will use them as X in the following commands. If you have more than 1 fiber channel, like me (QLA 2462) card then you must use the same command multiple times with a different WWN.

Enable target mode for each adapter or interface you want to use for target service with the following command (where X is the WWN seperated by colons for example "21:01:00:1b:32:13bd:32":

scstadmin -enable_target X -driver Y

In my case it was:

scstadmin -enable_target 21:01:00:1b:32:13bd:32 -driver qla2x00t

Create a security group for the devices that will be using your target. For example, I will name my security group esxi because my esxi servers will be connecting to openfiler for their datastores. The groupname will be used as Z in the next commands. Use the command:

scstadmin -add_group Z -driver Y -target X

In my case it was:

scstadmin -add_group esxi -driver qla2x00t -target 21:01:00:1b:32:13bd:32

Assign servers to the security group. You need to find the wwn of every server that you want to connect to the openfiler server using fiber channel (Will use these WWN as W). On Esxi you can easily find this by going to Configuration > Storage Adapters and then selecting the storage adapter where you want the WWN from. See screenshot bellow.

with the following command:

scstadmin -add_init W -driver y -target x -group Z

In my case it was:

scstadmin -add_init 50:01:43:80:03:bc:3a:9d -driver qla2x00t -target 21:01:00:1b:32:13bd:32 -group esxi
scstadmin -add_init 50:01:43:80:03:bc:3a:9c -driver qla2x00t -target 21:01:00:1b:32:13bd:32 -group esxi

Next create a logical volume and select iscsi / fiber channel as filesystem. I've created a volume named vol_ssd in the volume groud vg_ssd.

14. Create scst device.

Now it's time to create a fiber channel device that points at this logical volume. Use the following command replace V with a label that you assign and U with the full path to the logical volume created in the previous step. T is how you will be accessing your storage. You can see all the options by doing the command:

scstadmin -list_handler

Then run the following command to create the device:

scstadmin -open_dev V -handler T -attributes filename=U

In my case:

scstadmin -open_dev vol_ssd -handler vdisk_fileio -attributes filename=/dev/vg_ssd/vol_ssd

Now it's time to assign the virtual disk to a security group. Use the following command where S is the LUN number starting with 0. 0 Is required before any other! Use the following command:

scstadmin -add_lun S -driver Y -target W -group Z -device V

In my case:

scstadmin -add_lun 0 -driver qla2x00t -target 21:01:00:1b:32:13bd:32 -group esxi -device vol_ssd

This completes the configuration on the openfiler system. In order for this configuration to persist across rebots, enter the following command:

scstadmin -write_config /etc/scst.conf

Make sure that this command is runned each time that a change is made to the openfiler system relating to the fiber channel sub system.

Wrap up.

I use Point to point fiber channel connections for my server so no switch is involved. If you have a switch then you should add the necessairy zones but I recommend to use p2p first and add the switch in the mix is p2p works flawlessly. It is easier to debug without the switch.

The different requirement for

The different requirement for the installation of Scst and scst admin on Ubuntu 11.04 has nicely explained in the blog in an easy and clear manner. I appreciate the time and effort took by the author for demonstrating this for the readers. Thank you!

Go back to the basic.

Best thing to do is to test 1 on 1.

So use 1 fiber card to 1 server. If it works copy the config and change the vars.

Is my /etc/scst.conf set for Masking?

Not sure if my LUN’s are masking to hosts. Have two LUN’s 0 and 1. Each I believe is masked to all of my initiators:

INITIATOR 20:00:00:25:B5:10:B8:1F
INITIATOR 20:00:00:25:B5:10:A8:1F
INITIATOR 20:00:00:25:B5:10:A8:0F
INITIATOR 20:00:00:25:B5:10:B8:0F

Each host is dual homed. Even though zoning only allows each host HBA to see one target on the openfiler, I masked all anyhow. Kind of in debug stage here.

Is my configuration missing the actual masking? I seem my servers are not seeing the LUN’s. I know the SAN Zoning etc, is fine.

At SCST Configurator v2.0.0

cat /etc/scst.conf

# Automatically generated by SCST Configurator v2.0.0.

HANDLER vdisk_blockio {
DEVICE craigsfcvolume {
t10_dev_id "craigsfcvolume c27d97b3"
usn c27d97b3

filename /dev/dasvolume/craigsfcvolume
}

DEVICE v-crsimp01 {
t10_dev_id "v-crsimp01 e951d814"
usn e951d814

filename /dev/dasvolume/v-crsimp01
}
}

TARGET_DRIVER qla2x00t {
TARGET 21:00:00:1b:32:01:2b:04 {
rel_tgt_id 2
enabled 1

GROUP DASBOOT {
LUN 0 craigsfcvolume
LUN 1 v-crsimp01

INITIATOR 20:00:00:25:B5:10:B8:1F

INITIATOR 20:00:00:25:B5:10:A8:1F

INITIATOR 20:00:00:25:B5:10:A8:0F

INITIATOR 20:00:00:25:B5:10:B8:0F
}
}

TARGET 21:01:00:1b:32:21:2b:04 {
rel_tgt_id 1
enabled 1

GROUP DASBOOT {
LUN 0 craigsfcvolume
LUN 1 v-crsimp01

INITIATOR 20:00:00:25:B5:10:B8:1F

INITIATOR 20:00:00:25:B5:10:A8:1F

INITIATOR 20:00:00:25:B5:10:A8:0F

INITIATOR 20:00:00:25:B5:10:B8:0F
}
}
}

With one server booted I see

With one server booted I see the WWPN's logged in at filer

Collecting current configuration: done.

Driver/Target: qla2x00t/21:00:00:1b:32:01:2b:04

Session: 20:00:00:25:b5:10:a8:0f

Attribute Value Writable KEY
-----------------------------------------------------------------
active_commands 0 Yes No
commands 0 Yes No
initiator_name 20:00:00:25:b5:10:a8:0f Yes No

Session: 21:fd:00:05:1e:90:17:0c

Attribute Value Writable KEY
-----------------------------------------------------------------
active_commands 0 Yes No
commands 0 Yes No
initiator_name 21:fd:00:05:1e:90:17:0c Yes No

Driver/Target: qla2x00t/21:01:00:1b:32:21:2b:04

Session: 20:00:00:25:b5:10:b8:0f

Attribute Value Writable KEY
-----------------------------------------------------------------
active_commands 0 Yes No
commands 0 Yes No
initiator_name 20:00:00:25:b5:10:b8:0f Yes No

Session: 21:fd:00:05:1e:90:17:0c

Attribute Value Writable KEY
-----------------------------------------------------------------
active_commands 0 Yes No
commands 0 Yes No
initiator_name 21:fd:00:05:1e:90:17:0c Yes No

All done.

Post new comment

The content of this field is kept private and will not be shown publicly.
To prevent automated spam submissions leave this field empty.
By submitting this form, you accept the Mollom privacy policy.