RHEL 6 : ISCSI Administration Series : Configuring Shared ISCSI Storage

iscsi-title-1

 

This is a continuation post to our ISCSI administration series in RHEL6 ( here are the links to my previous posts  RHEL 6 – ISCSI Administration Series – Configuring ISCSI Server and Client &  RHEL 6 – ISCSI Administration Series – ISCSI Lun resize  ). This post is an hands-on demonstration of the ISCSI shared storage configuration in RedHat cluster environment

 

Purpose of the demonstration:

We want to create two shared luns from the ISCSI server and have them recognized at two cluster nodes simultaneously. And verify that the changes we are make to one of the shared lun from one node is visible to other lun directly, without making any changes to the iscsi connections.

 

For the demonstration purpose, I have made test setup using the KVM hosts as giving below.  I have total three KVM VMs  named as 

  • VM 1. gurkulsan-iscsi  – with two IPs   –    192.168.1.49   ( For external connection)      and  172.16.0.49 ( for iscsi network)
  • VM 2 . gurkulclu-node1 – with two IPs  – 192.168.1.52  ( for external connection ) and 172.16.0.52 ( for iscsi network)
  • VM 2 . gurkulclu-node2 – with two IPs  – 192.168.1.53  ( for external connection ) and 172.16.0.53 ( for iscsi network)

 

And Pictorial representation of my setup was given below

 

KVM-Setup

 

 

 Note:  Somehow , wordpress auto proofread changing my syntax wherever there is  double dash  (  – – )  to single ( ), I tried a lot to fix  a lot that but somehow I couldn’t fix it. Just as an work around I have  added a space in middle  ” – – “

ISCSI SERVER Configuration

 

Step1 : Install the SCSI Target Utilities required to configure the ISCSI Targets

 

[root@gurkulsan-iscsi ~]# yum install scsi-target-utils
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: centos.ipserverone.com
* extras: centos.ipserverone.com
* updates: centos.ipserverone.com
Setting up Install Process
Resolving Dependencies
– -> Running transaction check
– –> Package scsi-target-utils.x86_64 0:1.0.24-3.el6_4 will be installed
– -> Processing Dependency: sg3_utils for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: perl(strict) for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: perl(Getopt::Long) for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: perl(Fcntl) for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: perl(Errno) for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: perl(Config::General) for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: lsof for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: librdmacm.so.1(RDMACM_1.0)(64bit) for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: libibverbs.so.1(IBVERBS_1.1)(64bit) for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: libibverbs.so.1(IBVERBS_1.0)(64bit) for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: /usr/bin/perl for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: librdmacm.so.1()(64bit) for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Processing Dependency: libibverbs.so.1()(64bit) for package: scsi-target-utils-1.0.24-3.el6_4.x86_64
– -> Running transaction check
– –> Package libibverbs.x86_64 0:1.1.6-5.el6 will be installed
– –> Package librdmacm.x86_64 0:1.0.17-0.git4b5c1aa.el6 will be installed
– –> Package lsof.x86_64 0:4.82-4.el6 will be installed
– –> Package perl.x86_64 4:5.10.1-131.el6_4 will be installed
– -> Processing Dependency: perl-libs = 4:5.10.1-131.el6_4 for package: 4:perl-5.10.1-131.el6_4.x86_64
– -> Processing Dependency: perl-libs for package: 4:perl-5.10.1-131.el6_4.x86_64
– -> Processing Dependency: perl(version) for package: 4:perl-5.10.1-131.el6_4.x86_64
– -> Processing Dependency: perl(Pod::Simple) for package: 4:perl-5.10.1-131.el6_4.x86_64
– -> Processing Dependency: perl(Module::Pluggable) for package: 4:perl-5.10.1-131.el6_4.x86_64
– -> Processing Dependency: libperl.so()(64bit) for package: 4:perl-5.10.1-131.el6_4.x86_64
– –> Package perl-Config-General.noarch 0:2.44-1.el6 will be installed
– –> Package sg3_utils.x86_64 0:1.28-4.el6 will be installed
– -> Processing Dependency: sg3_utils-libs = 1.28-4.el6 for package: sg3_utils-1.28-4.el6.x86_64
– -> Processing Dependency: libsgutils2.so.2()(64bit) for package: sg3_utils-1.28-4.el6.x86_64
– -> Running transaction check
– –> Package perl-Module-Pluggable.x86_64 1:3.90-131.el6_4 will be installed
– –> Package perl-Pod-Simple.x86_64 1:3.13-131.el6_4 will be installed
– -> Processing Dependency: perl(Pod::Escapes) >= 1.04 for package: 1:perl-Pod-Simple-3.13-131.el6_4.x86_64
– –> Package perl-libs.x86_64 4:5.10.1-131.el6_4 will be installed
– –> Package perl-version.x86_64 3:0.77-131.el6_4 will be installed
– –> Package sg3_utils-libs.x86_64 0:1.28-4.el6 will be installed
– -> Running transaction check
– –> Package perl-Pod-Escapes.x86_64 1:1.04-131.el6_4 will be installed
– -> Finished Dependency Resolution

Dependencies Resolved

===========================================================================
Package Arch Version Repository Size
===========================================================================
Installing:
scsi-target-utils x86_64 1.0.24-3.el6_4 updates 172 k
Installing for dependencies:
libibverbs x86_64 1.1.6-5.el6 base 45 k
librdmacm x86_64 1.0.17-0.git4b5c1aa.el6 base 47 k
lsof x86_64 4.82-4.el6 base 319 k
perl x86_64 4:5.10.1-131.el6_4 updates 10 M
perl-Config-General noarch 2.44-1.el6 base 68 k
perl-Module-Pluggable x86_64 1:3.90-131.el6_4 updates 39 k
perl-Pod-Escapes x86_64 1:1.04-131.el6_4 updates 31 k
perl-Pod-Simple x86_64 1:3.13-131.el6_4 updates 211 k
perl-libs x86_64 4:5.10.1-131.el6_4 updates 577 k
perl-version x86_64 3:0.77-131.el6_4 updates 50 k
sg3_utils x86_64 1.28-4.el6 base 471 k
sg3_utils-libs x86_64 1.28-4.el6 base 51 k

Transaction Summary
========================================================================
Install 13 Package(s)

Total download size: 12 M
Installed size: 39 M
Is this ok [y/N]: y
Downloading Packages:
(1/13): libibverbs-1.1.6-5.el6.x86_64.rpm | 45 kB 00:00
(2/13): librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64.rpm | 47 kB 00:00
(3/13): lsof-4.82-4.el6.x86_64.rpm | 319 kB 00:00
(4/13): perl-5.10.1-131.el6_4.x86_64.rpm | 10 MB 00:13
(5/13): perl-Config-General-2.44-1.el6.noarch.rpm | 68 kB 00:00
(6/13): perl-Module-Pluggable-3.90-131.el6_4.x86_64.rpm | 39 kB 00:00
(7/13): perl-Pod-Escapes-1.04-131.el6_4.x86_64.rpm | 31 kB 00:00
(8/13): perl-Pod-Simple-3.13-131.el6_4.x86_64.rpm | 211 kB 00:00
(9/13): perl-libs-5.10.1-131.el6_4.x86_64.rpm | 577 kB 00:00
(10/13): perl-version-0.77-131.el6_4.x86_64.rpm | 50 kB 00:00
(11/13): scsi-target-utils-1.0.24-3.el6_4.x86_64.rpm | 172 kB 00:00
(12/13): sg3_utils-1.28-4.el6.x86_64.rpm | 471 kB 00:00
(13/13): sg3_utils-libs-1.28-4.el6.x86_64.rpm | 51 kB 00:00
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
Total 714 kB/s | 12 MB 00:17
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : libibverbs-1.1.6-5.el6.x86_64 1/13
Installing : librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64 2/13
Installing : 1:perl-Pod-Escapes-1.04-131.el6_4.x86_64 3/13
Installing : 1:perl-Module-Pluggable-3.90-131.el6_4.x86_64 4/13
Installing : 4:perl-libs-5.10.1-131.el6_4.x86_64 5/13
Installing : 3:perl-version-0.77-131.el6_4.x86_64 6/13
Installing : 1:perl-Pod-Simple-3.13-131.el6_4.x86_64 7/13
Installing : 4:perl-5.10.1-131.el6_4.x86_64 8/13
Installing : perl-Config-General-2.44-1.el6.noarch 9/13
Installing : lsof-4.82-4.el6.x86_64 10/13
Installing : sg3_utils-libs-1.28-4.el6.x86_64 11/13
Installing : sg3_utils-1.28-4.el6.x86_64 12/13
Installing : scsi-target-utils-1.0.24-3.el6_4.x86_64 13/13
Verifying : 1:perl-Pod-Escapes-1.04-131.el6_4.x86_64 1/13
Verifying : sg3_utils-libs-1.28-4.el6.x86_64 2/13
Verifying : libibverbs-1.1.6-5.el6.x86_64 3/13
Verifying : 1:perl-Pod-Simple-3.13-131.el6_4.x86_64 4/13
Verifying : 1:perl-Module-Pluggable-3.90-131.el6_4.x86_64 5/13
Verifying : perl-Config-General-2.44-1.el6.noarch 6/13
Verifying : 4:perl-5.10.1-131.el6_4.x86_64 7/13
Verifying : 4:perl-libs-5.10.1-131.el6_4.x86_64 8/13
Verifying : 3:perl-version-0.77-131.el6_4.x86_64 9/13
Verifying : scsi-target-utils-1.0.24-3.el6_4.x86_64 10/13
Verifying : sg3_utils-1.28-4.el6.x86_64 11/13
Verifying : lsof-4.82-4.el6.x86_64 12/13
Verifying : librdmacm-1.0.17-0.git4b5c1aa.el6.x86_64 13/13

Installed:
scsi-target-utils.x86_64 0:1.0.24-3.el6_4

Dependency Installed:
libibverbs.x86_64 0:1.1.6-5.el6 librdmacm.x86_64 0:1.0.17-0.git4b5c1aa.el6 lsof.x86_64 0:4.82-4.el6
perl.x86_64 4:5.10.1-131.el6_4 perl-Config-General.noarch 0:2.44-1.el6 perl-Module-Pluggable.x86_64 1:3.90-131.el6_4
perl-Pod-Escapes.x86_64 1:1.04-131.el6_4 perl-Pod-Simple.x86_64 1:3.13-131.el6_4 perl-libs.x86_64 4:5.10.1-131.el6_4
perl-version.x86_64 3:0.77-131.el6_4 sg3_utils.x86_64 0:1.28-4.el6 sg3_utils-libs.x86_64 0:1.28-4.el6

Complete!
[root@gurkulsan-iscsi ~]#

 

Step 2 : Check the Files Installed along with the SCSI-TARGET-UTILS Packages 

 

[root@gurkulsan-iscsi ~]# rpm -ql scsi-target-utils
/etc/rc.d/init.d/tgtd
/etc/sysconfig/tgtd
/etc/tgt/targets.conf
/usr/sbin/tgt-admin
/usr/sbin/tgt-setup-lun
/usr/sbin/tgtadm
/usr/sbin/tgtd
/usr/sbin/tgtimg
/usr/share/doc/scsi-target-utils-1.0.24
/usr/share/doc/scsi-target-utils-1.0.24/README
/usr/share/doc/scsi-target-utils-1.0.24/README.iscsi
/usr/share/doc/scsi-target-utils-1.0.24/README.iser
/usr/share/doc/scsi-target-utils-1.0.24/README.lu_configuration
/usr/share/doc/scsi-target-utils-1.0.24/README.mmc
/usr/share/man/man5/targets.conf.5.gz
/usr/share/man/man8/tgt-admin.8.gz
/usr/share/man/man8/tgt-setup-lun.8.gz
/usr/share/man/man8/tgtadm.8.gz

 

Step 3: Start the SCSI target Daemon

[root@gurkulsan-iscsi ~]# /etc/init.d/tgtd start
Starting SCSI target daemon: [ OK ]
[root@gurkulsan-iscsi ~]# chkconfig tgtd on

Step 4: Configure the first target with target ID – 101

[root@gurkulsan-iscsi ~]# tgtadm – -lld iscsi – -mode target – -op new – -tid 101 – -targetname iqn.2013-04.com.unixadminshool:target1

[root@gurkulsan-iscsi ~]# tgtadm – -lld iscsi – -mode target – -op show

Target 101: iqn.2013-04.com.unixadminshool:target1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00650000
SCSI SN: beaf1010
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: null
Backing store path: None
Backing store flags:
Account information:
ACL information:
[root@gurkulsan-iscsi ~]#

 

Step 5: Check the Current Disks Available to Attach to ISCSI target as a Shared Storage

 

[root@gurkulsan-iscsi ~]# fdisk -l |grep -i Disk
Disk /dev/sda: 2147 MB, 2147483648 bytes <== Disk for Shared Storage
Disk identifier: 0x0006ae60
Disk /dev/sdb: 2147 MB, 2147483648 bytes <== Disk for Shared Storage
Disk identifier: 0x000c410f
Disk /dev/vda: 8589 MB, 8589934592 bytes
Disk identifier: 0x000247fc
Disk /dev/mapper/vg_gurkulsaniscsi-lv_root: 3900 MB, 3900702720 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vg_gurkulsaniscsi-lv_swap: 4160 MB, 4160749568 bytes
Disk identifier: 0x00000000
[root@gurkulsan-iscsi ~]#

Step 4: Make sure no partitions and data exists on /dev/sdb and /dev/sdc

[root@gurkulsan-iscsi ~]# fdisk -l /dev/sda

Disk /dev/sda: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006ae60

Device Boot Start End Blocks Id System

[root@gurkulsan-iscsi ~]# fdisk -l /dev/sdb

Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c410f

Device Boot Start End Blocks Id System

Step 6: Create LUNS using the disks /dev/sda ( i.e. lun-1 ) and /dev/sdb ( i.e lun-2 )as backing-stores

 

[root@gurkulsan-iscsi iscsi]# tgtadm – -lld iscsi – -mode target – -op new – -tid 101 – -targetname iqn.2013-04.com.unixadminshool:target1
[root@gurkulsan-iscsi iscsi]# tgtadm – -lld iscsi – -mode target – -op show
Target 101: iqn.2013-04.com.unixadminshool:target1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00650000
SCSI SN: beaf1010
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: null
Backing store path: None
Backing store flags:
Account information:
ACL information:

[root@gurkulsan-iscsi iscsi]# tgtadm – -lld iscsi – -mode logicalunit – -op new – -tid 101 – -lun 1 – -backing-store /dev/sda
[root@gurkulsan-iscsi iscsi]# tgtadm – -lld iscsi – -mode logicalunit – -op new – -tid 101 – -lun 2 – -backing-store /dev/sdb

 

Step 7: Configure the Access to the ISCSI Storage to the Specicific Client IP addresses i,e. 172.16.0. 52 & 53

 

[root@gurkulsan-iscsi iscsi]# tgtadm – -lld iscsi – -mode target – -op bind – -tid 1 – -initiator-address 172.16.0.52
[root@gurkulsan-iscsi iscsi]# tgtadm – -lld iscsi – -mode target – -op bind – -tid 1 – -initiator-address 172.16.0.53
[root@gurkulsan-iscsi iscsi]# tgtadm – -lld iscsi – -mode target – -op show
Target 1: iqn.2013-04.com.unixadminshool:target1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 2147 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: rdwr
Backing store path: /dev/sda
Backing store flags:
LUN: 2
Type: disk
SCSI ID: IET 00010002
SCSI SN: beaf12
Size: 2147 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: rdwr
Backing store path: /dev/sdb
Backing store flags:
Account information:
ACL information:
ALL
172.16.0.52
172.16.0.53
[root@gurkulsan-iscsi iscsi]#

Step8 : Save the Target Configuration so that we don’t lose after the reboot

[root@gurkulsan-iscsi iscsi]# cp /etc/tgt/targets.conf /etc/tgt/targets.conf.orig
cp: overwrite `/etc/tgt/targets.conf.orig’? y
[root@gurkulsan-iscsi iscsi]# tgt-admin – -dump
default-driver iscsi

backing-store /dev/sda
backing-store /dev/sdb

[root@gurkulsan-iscsi iscsi]# tgt-admin – -dump >> /etc/tgt/targets.conf
[root@gurkulsan-iscsi iscsi]# sdiff -s /etc/tgt/targets.conf /etc/tgt/targets.conf.orig
default-driver iscsi <
<
<
backing-store /dev/sda <
backing-store /dev/sdb <
<
<
[root@gurkulsan-iscsi iscsi]# service tgtd restart
Stopping SCSI target daemon: [ OK ]
Starting SCSI target daemon: [ OK ]
Multiple default-driver definitions are not allowed!
Check your config file for errors.
[root@gurkulsan-iscsi iscsi]# vi /etc/tgt/targets.conf
[root@gurkulsan-iscsi iscsi]# service tgtd restart
Stopping SCSI target daemon: [ OK ]
Starting SCSI target daemon: [ OK ]
[root@gurkulsan-iscsi iscsi]# tgtadm – -lld iscsi – -mode target – -op show
Target 1: iqn.2013-04.com.unixadminshool:target1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 2147 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: rdwr
Backing store path: /dev/sda
Backing store flags:
LUN: 2
Type: disk
SCSI ID: IET 00010002
SCSI SN: beaf12
Size: 2147 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: rdwr
Backing store path: /dev/sdb
Backing store flags:
Account information:
ACL information:
ALL

172.16.0.52
172.16.0.53

Step 9 : Disable IPfilters so that ISCSI server can accept the connection from the ISCSI clients

 

[root@gurkulsan-iscsi iscsi]#

 

[root@gurkulsan-iscsi ~]# service iptables stop
iptables: Flushing firewall rules: [ OK ]
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Unloading modules: [ OK ]
[root@gurkulsan-iscsi ~]#

 

 

 

Configuration steps at gurkulclu-node1 and gurkulclu-node2

 

Step 1:  Configure Initiator name to something Unique to the Node

[root@gurkulclu-node1 ~]# cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.2013-08.com.unixadminschool:gukrulclu-node1
[root@gurkulclu-node1 ~]#

 [root@gurkulclu-node2 ~]# cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.2013-08.com.unixadminschool:gukrulclu-node2
[root@gurkulclu-node2 ~]#

Step 2:  Stop the ISCSI Daemon

 

[root@gurkulclu-node1 ~]# service iscsid status
iscsid is stopped

 

[root@gurkulclu-node2 ~]# service iscsid status
iscsid is stopped

Step 3: Force-restart the ISCSI Daemon to Re-read the Initiator name

[root@gurkulclu-node1 ~]# /etc/init.d/iscsid force-start
Starting iscsid: [ OK ]
[root@gurkulclu-node1 ~]#

[root@gurkulclu-node2 ~]# /etc/init.d/iscsid force-start
Starting iscsid: [ OK ]
[root@gurkulclu-node2 ~]#

step 4:  Stop the Iptables to allow connections to ISCSI server

[root@gurkulclu-node1 ~]# service iptables stop
iptables: Flushing firewall rules: [ OK ]
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Unloading modules: [ OK ]
[root@gurkulclu-node1 ~]#

 [root@gurkulclu-node2 ~]# service iptables stop
iptables: Flushing firewall rules: [ OK ]
iptables: Setting chains to policy ACCEPT: filter [ OK ]
iptables: Unloading modules: [ OK ]
[root@gurkulclu-node2 ~]#

Step 5 : Discover the Targets available at the ISCSI server ( i.e. PORTAL) using the ISCSI server’s IP address

 

[root@gurkulclu-node1 ~]# iscsiadm -m discoverydb – -type st – -portal 172.16.0.49 – -discover
172.16.0.49:3260,1 iqn.2013-04.com.unixadminshool:target1
[root@gurkulclu-node1 ~]#

 

[root@gurkulclu-node2 ~]# iscsiadm -m discoverydb – -type st – -portal 172.16.0.49 – -discover
172.16.0.49:3260,1 iqn.2013-04.com.unixadminshool:target1
[root@gurkulclu-node2 ~]#

 

Step 6: Once Target discovered, login to the portal to access the LUNs attached to the target

[root@gurkulclu-node1 ~]# iscsiadm -m node -T iqn.2013-04.com.unixadminshool:target1 -l

Logging in to [iface: default, target: iqn.2013-04.com.unixadminshool:target1, portal: 172.16.0.49,3260] (multiple)
Login to [iface: default, target: iqn.2013-04.com.unixadminshool:target1, portal: 172.16.0.49,3260] successful.
[root@gurkulclu-node1 ~]#

[root@gurkulclu-node2 ~]# iscsiadm -m node -T iqn.2013-04.com.unixadminshool:target1 -l
Logging in to [iface: default, target: iqn.2013-04.com.unixadminshool:target1, portal: 172.16.0.49,3260] (multiple)
Login to [iface: default, target: iqn.2013-04.com.unixadminshool:target1, portal: 172.16.0.49,3260] successful.
[root@gurkulclu-node2 ~]#

 

 

Step 7:  Checks the disks discovered from the ISCSI server using FDISK

 

[root@gurkulclu-node1 ~]# fdisk -l|grep -i Disk
Disk /dev/vda: 8589 MB, 8589934592 bytes
Disk identifier: 0x0001d529
Disk /dev/mapper/vg_gurkulclunode1-lv_root: 3934 MB, 3934257152 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vg_gurkulclunode1-lv_swap: 4127 MB, 4127195136 bytes
Disk identifier: 0x00000000
Disk /dev/sda: 2147 MB, 2147483648 bytes   <== LUN from ISCSI server
Disk identifier: 0x0006ae60
Disk /dev/sdb: 2147 MB, 2147483648 bytes   <== LUN from ISCSI server
Disk identifier: 0x000c410f
[root@gurkulclu-node1 ~]#

[root@gurkulclu-node2 ~]# fdisk -l|grep -i Disk
Disk /dev/vda: 8589 MB, 8589934592 bytes
Disk identifier: 0x00021019
Disk /dev/mapper/vg_gurkulclunode2-lv_root: 3934 MB, 3934257152 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/vg_gurkulclunode2-lv_swap: 4127 MB, 4127195136 bytes
Disk identifier: 0x00000000
Disk /dev/sda: 2147 MB, 2147483648 bytes
Disk identifier: 0x0006ae60
Disk /dev/sdb: 2147 MB, 2147483648 bytes
Disk identifier: 0x000c410f
[root@gurkulclu-node2 ~]#

 

Step 8 : Configure the Multipathing for ISCSI disks 

Note : In this lab , i am using only one path, but still the configuration is still same as using dual path

 

Step 8a:  Install Multipath Package

[root@gurkulclu-node1 ~]# yum install device-mapper-multipath
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: centos.ipserverone.com
* extras: centos.ipserverone.com
* updates: centos.ipserverone.com
Setting up Install Process
Package device-mapper-multipath-0.4.9-64.el6_4.2.x86_64 already installed and latest version
Nothing to do
[root@gurkulclu-node1 ~]#

Step 8b : ConfigCopy Multipath Sample file /etc/multipath.conf

[root@gurkulclu-node1 ~]# cp /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf /etc/

 

Step 8c : Modify the /etc/multipath.conf to configure the multipathing

Modifications to be made to the file :

1. Uncomment the defaults Sections

2. Un comment the blacklist section and add any filesystems that you dont want to use multipath.

Below is the sdiff oupput from after ( left side) and before ( right side) changes of the file. for your reference.

 

[root@gurkulclu-node1 ~]# sdiff -s /etc/multipath.conf /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf
defaults {                                          | #defaults {
udev_dir /dev                                       | # udev_dir /dev
polling_interval 10                                 | # polling_interval 10
path_selector “round-robin 0”                       | # path_selector “round-robin 0”
path_grouping_policy multibus                       | # path_grouping_policy multibus
getuid_callout “/lib/udev/scsi_id – -replace-        | # getuid_callout “/lib/udev/scsi_id – -whitelis
prio const                                          | # prio alua
path_checker readsector0                            | # path_checker readsector0
rr_min_io 100                                       | # rr_min_io 100
max_fds 8192                                        | # max_fds 8192
rr_weight priorities                                | # rr_weight priorities
failback immediate                                  | # failback immediate
no_path_retry fail                                  | # no_path_retry fail
user_friendly_names no                              | # user_friendly_names yes
}                                                   | #}
blacklist {                                         | #blacklist {
devnode “^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*” | # devnode “^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*”
devnode “^hd[a-z]”                                  | # devnode “^hd[a-z]”
devnode “^vd[a-z]”                                  | #}
}                                                   <

 

Step 8d:  Enable the multipathd service to autostartup during boot

<== LUN from ISCSI server[root@gurkulclu-node1 ~]# chkconfig multipathd on

 

Step 8e: multipathd Service

[root@gurkulclu-node1 ~]# service multipathd start
Starting multipathd daemon: [ OK ]

 

Step 8f: current multpath devices

 

[root@gurkulclu-node1 ~]# multipath -ll
Sep 07 05:23:51 | multipath.conf line 32, duplicate keyword: defaults
1IET_00010001 dm-2 IET,VIRTUAL-DISK
size=2.0G features=’0′ hwhandler=’0′ wp=rw
`-+- policy=’round-robin 0′ prio=1 status=active
`- 2:0:0:1 sda 8:0 active ready running
1IET_00010002 dm-3 IET,VIRTUAL-DISK
size=2.0G features=’0′ hwhandler=’0′ wp=rw
`-+- policy=’round-robin 0′ prio=1 status=active
`- 2:0:0:2 sdb 8:16 active ready running
[root@gurkulclu-node1 ~]#

Step 8g: Copy the same multipath configuration to the other node

root@gurkulclu-node1 ~]# scp /etc/multipath.conf gurkulclu-node2:/etc/multipath.conf
The authenticity of host ‘gurkulclu-node2 (192.168.1.52)’ can’t be established.
RSA key fingerprint is 9d:b2:76:35:f3:2a:3b:fd:d0:86:bf:ed:fb:d8:b7:7f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘gurkulclu-node2,192.168.1.52’ (RSA) to the list of known hosts.
root@gurkulclu-node2’s password:
multipath.conf 100% 2761 2.7KB/s 00:00
[root@gurkulclu-node1 ~]#

 

 

Step 8h : Multipath Configuration Steps at Gurkulclu-node2

 

[root@gurkulclu-node2 ~]# yum install device-mapper-multipath
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: centos.ipserverone.com
* extras: centos.ipserverone.com
* updates: centos.ipserverone.com
Setting up Install Process
Package device-mapper-multipath-0.4.9-64.el6_4.2.x86_64 already installed and latest version
Nothing to do
[root@gurkulclu-node2 ~]#

[root@gurkulclu-node2 ~]# chkconfig multipathd on

[root@gurkulclu-node2 etc]# service multipathd restart
ok
Stopping multipathd daemon: [ OK ]
Starting multipathd daemon: [ OK ]
[root@gurkulclu-node2 etc]# multipath -ll
Sep 06 18:33:22 | multipath.conf line 32, duplicate keyword: defaults
1IET_00010001 dm-2 IET,VIRTUAL-DISK
size=2.0G features=’0′ hwhandler=’0′ wp=rw
`-+- policy=’round-robin 0′ prio=1 status=active
`- 2:0:0:1 sda 8:0 active ready running
1IET_00010002 dm-3 IET,VIRTUAL-DISK
size=2.0G features=’0′ hwhandler=’0′ wp=rw
`-+- policy=’round-robin 0′ prio=1 status=active
`- 2:0:0:2 sdb 8:16 active ready running
[root@gurkulclu-node2 etc]#

 

 

Shared Storage Verification Between Two Nodes

 

step 1: On node-1 we will create a partition using fdisk

[root@gurkulclu-node1 ~]# fdisk /dev/mapper/1IET_00010001

WARNING: DOS-compatible mode is deprecated. It’s strongly recommended to
switch off the mode (command ‘c’) and change display units to
sectors (command ‘u’).

Command (m for help): p

Disk /dev/mapper/1IET_00010001: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006ae60

Device Boot Start End Blocks Id System

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-261, default 1): 1
Last cylinder, +cylinders or +size{K,M,G} (1-261, default 261): +500m
Unsupported suffix: ‘m’.
Supported: 10^N: KB (KiloByte), MB (MegaByte), GB (GigaByte)
2^N: K (KibiByte), M (MebiByte), G (GibiByte)
Last cylinder, +cylinders or +size{K,M,G} (1-261, default 261):
Using default value 261

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 22: Invalid argument.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.

Step 2 : Check for the new device path for the partition we created, and it won’t exists immediatately

[root@gurkulclu-node1 ~]# ls -l /dev/mapper
total 0
lrwxrwxrwx. 1 root root 7 Sep 7 05:23 1IET_00010001 -> ../dm-2
lrwxrwxrwx. 1 root root 7 Sep 7 05:23 1IET_00010002 -> ../dm-3
crw-rw – – – -. 1 root root 10, 58 Sep 5 14:34 control
lrwxrwxrwx. 1 root root 7 Sep 5 14:34 vg_gurkulclunode1-lv_root -> ../dm-0
lrwxrwxrwx. 1 root root 7 Sep 5 14:34 vg_gurkulclunode1-lv_swap -> ../dm-1

and we don’t see the partition 1IET_00010001p1

Step 2a: Now  probe the disk for new partition table using the partprobe utility

[root@gurkulclu-node1 ~]# partprobe /dev/mapper/1IET_00010001
[root@gurkulclu-node1 ~]# ls -l /dev/mapper
total 0
lrwxrwxrwx. 1 root root 7 Sep 7 05:23 1IET_00010001 -> ../dm-2
lrwxrwxrwx. 1 root root 7 Sep 7 05:38 1IET_00010001p1 -> ../dm-4 <== new partition visible now
lrwxrwxrwx. 1 root root 7 Sep 7 05:23 1IET_00010002 -> ../dm-3
crw-rw – – – -. 1 root root 10, 58 Sep 5 14:34 control
lrwxrwxrwx. 1 root root 7 Sep 5 14:34 vg_gurkulclunode1-lv_root -> ../dm-0
lrwxrwxrwx. 1 root root 7 Sep 5 14:34 vg_gurkulclunode1-lv_swap -> ../dm-1
[root@gurkulclu-node1 ~]#

Let’s check the new partition the new partition on the other node, without creating it.

Task to be done once the Disk partition created on the first node

Step 3: check if you can find the new partition on the second node directly

[root@gurkulclu-node2 etc]# ls -l /dev/mapper
total 0
lrwxrwxrwx. 1 root root 7 Sep 6 18:33 1IET_00010001 -> ../dm-2
lrwxrwxrwx. 1 root root 7 Sep 6 18:33 1IET_00010002 -> ../dm-3
crw-rw – – – -. 1 root root 10, 58 Sep 5 14:31 control
lrwxrwxrwx. 1 root root 7 Sep 5 14:31 vg_gurkulclunode2-lv_root -> ../dm-0
lrwxrwxrwx. 1 root root 7 Sep 5 14:31 vg_gurkulclunode2-lv_swap -> ../dm-1
[root@gurkulclu-node2 etc]#

and we don’t see the partition 1IET_00010001p1

Step 3a : probe the partition table using the part probe and see of the new partition table appears

[root@gurkulclu-node2 etc]# partprobe /dev/mapper/1IET_00010001
[root@gurkulclu-node2 etc]# ls -l /dev/mapper
total 0
lrwxrwxrwx. 1 root root 7 Sep 6 18:33 1IET_00010001 -> ../dm-2
lrwxrwxrwx. 1 root root 7 Sep 6 18:42 1IET_00010001p1 -> ../dm-4
lrwxrwxrwx. 1 root root 7 Sep 6 18:33 1IET_00010002 -> ../dm-3
crw-rw – – – -. 1 root root 10, 58 Sep 5 14:31 control
lrwxrwxrwx. 1 root root 7 Sep 5 14:31 vg_gurkulclunode2-lv_root -> ../dm-0
lrwxrwxrwx. 1 root root 7 Sep 5 14:31 vg_gurkulclunode2-lv_swap -> ../dm-1
[root@gurkulclu-node2 etc]#

Now we can see the device ” 1IET_00010001p1 -> ../dm-4″ which was actually created on the first node.

 

Please note that, here i am not creating any filesystem on this shared storage because redhat cluster environment uses GFS2 ( Global Filesystem) , CLVM ( Cluster Volume Manager) and DLM ( Distributed lock management ) concepts to effectively manage the shared storage, which I am expecting to cover in another post in this series. please stay connected  with me. 

 

Ramdev

Ramdev

I have started unixadminschool.com ( aka gurkulindia.com) in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' unixadminschool.com. You can connect me at - https://www.linkedin.com/in/unixadminschool/

You may also like...

3 Responses

  1. vincent rod says:

    Just I did it and works properly !

  2. Ramdev Ramdev says:

    Vincent, Thanks for confirming the configuration :)

  1. September 17, 2015

    […] Read –  Configuring Shared ISCSI Storage in RHEL 6 […]

What is in your mind, about this post ? Leave a Reply

Close
  Our next learning article is ready, subscribe it in your email

What is your Learning Goal for Next Six Months ? Talk to us