Diskless / remote boot with Open-iSCSI

From WPKG | Open Source Software Deployment and Distribution
Jump to: navigation, search

Background[edit]

It is possible to start a diskless machine remotely using Open-iSCSI on the client side. You have to take care of these things:

  • create an initrd which will connect the iSCSI node
  • configure your tftp server, so that your machine starts properly via PXE/tftp

This HOWTO explains how to set up your system so that it boots a diskless station using PXE/tftp and Open-iSCSI.

Creating initrd[edit]

Below some theory; for practical examples, see downloads section.

Your initrd has to do the following things:

  • load all needed modules, like:
    • network card modules (e1000, tg3, mii, 8139too, e100, depending on the card you use etc.)
    • filesystem modules (jbd, ext3, depending on the filesystem you use)
    • SCSI modules (scsi_mod, sd_mod)
    • Open-iSCSI modules (scsi_transport_iscsi, libiscsi, iscsi_tcp)
  • mount /proc and /sys
  • bring a network interface up
  • initiate a iSCSI session with iscsistart
  • mount your filesystem
  • switch to a new root
  • execute /sbin/init from a new root


Preparing initrd[edit]

Note that the below instruction use common tools from your favourite distribution. You may create a similar initrd with busybox/uClibc.

  • create a directory for your Open-ISCSI-aware initrd:
mkdir initrd
  • create all needed directories:
cd initrd
mkdir -p bin dev etc lib/modules proc sys sysroot
ln -s bin sbin
  • copy the following binaries to bin/:
bash  cat  chroot  echo  ifconfig  insmod  iscsistart  mount  sleep

You can do it with a following command:

which bash cat chroot echo ifconfig insmod iscsistart mount sleep | xargs -i{} cp {} bin/

If your distribution doesn't ship iscsistart (most distros don't), you have to compile open-iscsi yourself - you'll find there iscsistart.


  • copy shared libraries to lib/; you will find the shared libraries with ldd tool, for example:
# ldd bash
       linux-gate.so.1 =>  (0xb7fa6000)
       libtermcap.so.2 => /lib/libtermcap.so.2 (0xb7f9d000)
       libdl.so.2 => /lib/libdl.so.2 (0xb7f99000)
       libc.so.6 => /lib/i686/libc.so.6 (0xb7e6a000)
       /lib/ld-linux.so.2 (0xb7fa7000)


  • after all binaries and libraries are copied to correct places, verify if you didn't forget anything with chroot command - you may get a similar "no name" prompt (it's OK, since there are no users in that initrd), and all binaries should be able to start:
# cd initrd
# chroot .
[I have no name!@localhost /]# mount --help     
(...)
  • create needed nodes in dev/ (with mknod tool, or just copy them from your /dev):
console  sda  sda1  sda2


  • create the init file (don't forget to chmod 755 init); place it in the root of your initrd directory; note we load some modules here - you have to copy the modules to /lib/modules/ in your initrd; here's the example init file which detects kernel command line parameters like initiator name, target IP etc.:
#!/bin/bash
 
# Load modules
# You may only need to load the network card modules you will use
# To find module dependencies, you can use "modinfo" tool
 
echo "Loading jbd.ko module"
insmod /lib/modules/jbd.ko
# On some kernels, you will need mbcache module if it is not in the kernel
echo "Loading mbcache.ko module"
insmod /lib/modules/mbcache.ko
echo "Loading ext3.ko module"
insmod /lib/modules/ext3.ko
 
echo "Loading libcrc32c.ko module"
insmod /lib/modules/libcrc32c.ko
echo "Loading crc32c.ko module"
insmod /lib/modules/crc32c.ko
 
# Network card modules - start
echo "Loading mii.ko module"
insmod /lib/modules/mii.ko
echo "Loading 8139too.ko module"
insmod /lib/modules/8139too.ko
 
echo "Loading e100.ko module"
insmod /lib/modules/e100.ko
 
echo "Loading e1000.ko module"
insmod /lib/modules/e1000.ko
 
echo "Loading tulip.ko module"
insmod /lib/modules/tulip.ko
 
echo "Loading tg3.ko module"
insmod /lib/modules/tg3.ko
# Network card modules - end
 
echo "Loading scsi_mod.ko module"
insmod /lib/modules/scsi_mod.ko
echo "Loading sd_mod.ko module"
insmod /lib/modules/sd_mod.ko
 
echo "Loading scsi_transport_iscsi.ko module"
insmod /lib/modules/scsi_transport_iscsi.ko
echo "Loading libiscsi.ko module"
insmod /lib/modules/libiscsi.ko
echo "Loading iscsi_tcp.ko module"
insmod /lib/modules/iscsi_tcp.ko
 
 
# Mount /proc and /sys
echo Mounting /proc filesystem
mount -t proc /proc /proc
 
echo Mounting sysfs
mount -t sysfs /sys /sys
 
# We need to extract four command line parameters form /proc/cmdline:
# iscsi_i_ip - local initiator IP address/netmask (i.e. 192.168.111.168/255.255.255.192)
# iscsi_i    - local initiator name (InitiatorName)
# iscsi_t    - remote target name (TargetName)
# iscsi_a    - iSCSI target IP address
 
# Read the kernel cmdline
CMDLINE=$(cat /proc/cmdline)
 
# Find out IP/NETMASK first
TEMPVAR=${CMDLINE#*iscsi_i_ip=}
ISCSI_I_IP=${TEMPVAR%% *}
 
# Find InitiatorName
TEMPVAR=${CMDLINE#*iscsi_i=}
ISCSI_I=${TEMPVAR%% *}
 
# Find TargetName
TEMPVAR=${CMDLINE#*iscsi_t=}
ISCSI_T=${TEMPVAR%% *}
 
# Find iSCSI target IP address
TEMPVAR=${CMDLINE#*iscsi_a=}
ISCSI_A=${TEMPVAR%% *}
 
 
# Bring the network interface up
ifconfig eth0 ${ISCSI_I_IP%/*} netmask ${ISCSI_I_IP#*/}
 
# Connect the iSCSI drive
iscsistart -i "$ISCSI_I" -t "$ISCSI_T" -g 1 -a $ISCSI_A
 
mount -o ro /dev/sda2 /sysroot
 
echo Switching to new root
cd /sysroot
 
# Uncomment "bash" below, and comment out "exec ..." if you are having booting problems.
# It will give you a bash shell with limited tools.
exec chroot . sh -c 'exec /sbin/init'
#bash

Creating initrd with cpio[edit]

Now that you have all the files ready, create initrd image:

find ./ | cpio -H newc -o > ../initrd.cpio
cd ..
gzip -9 initrd.cpio
mv initrd.cpio.gz initrd.img


Next, copy initrd.img to your tftp server.

Configuring tftp server[edit]

Add something like that to a pxelinux.cfg/default file to initiate a remote boot by hand (not recommended):

LABEL remote1
   KERNEL remote/vmlinuz-2.6.17-5mdv
   APPEND initrd=remote/initrd.img iscsi_i=iqn.2007-01.com.example:server.remote1 iscsi_i_ip=10.1.1.165/255.255.255.192 iscsi_t=iqn.2007-01.com.example:storage.remote1 iscsi_a=10.1.1.180

You have to add these parameters to APPEND line appropriately:

  • initrd - initrd image, for example, initrd=remote/initrd.img
  • iscsi_i_ip - local initiator IP address/netmask (i.e. 192.168.111.168/255.255.255.192), for example, iscsi_i_ip=10.1.1.165/255.255.255.192
  • iscsi_i - local initiator name (InitiatorName), for example, iscsi_i=iqn.2007-01.com.example:server.remote1
  • iscsi_t - remote target name (TargetName), for example, iscsi_t=iqn.2007-01.com.example:storage.remote1
  • iscsi_a - iSCSI target IP address, for example, iscsi_a=10.1.1.180


It is recommended that you add the above entry to a pxelinux.cfg/aa-bb-cc-dd-ee-ff file; aa-bb-cc-dd-ee-ff being the MAC address of the PC you want to boot remotely with Open-iSCSI / PXE.


Downloads[edit]

Go to downloads page.


Additional info / troubleshooting[edit]

See diskless iSCSI troubleshooting page.


Contact[edit]

For general Open-iSCSI info, please ask at open-iscsi mailing list.
If you wish to contact me directly, write me an email: Tomasz Chmielewski <tch (at) wpkg (dot) org>.


Keywords: Open-iSCSI remote boot, diskless, iSCSI, Xen, tftp, PXE, network boot