VCS 6.1 : How to manage SCSI-3 disk registrations and reservations for I/O fencing using vxfenadm and vxfenclearpre?

 vcs-how-to
vxfenadm: The vxfenadm utility enables the vxfen driver to manage SCSI-3 registrations and reservations on disks for purposes of I/O fencing. In the event of a split-brain condition, I/O fencing allows only a system with a valid registration to write to a disk. Thus, I/O fencing prevents data corruption on the disks.  System administrators can use the -a, -m, -n, -x, -p, -s, and -r flags to validate the accessibility of the disks and to perform troubleshooting tasks; these options do not actually implement fencing
Note:  While using -x and -p options, make sure I/O fencing is not configured; these options will overwrite valid keys used by VxVM.
A disk_filename contains the list of all disks and each path (in the event of multiple paths) in a disk group.
If administrators use the -p, -a, -m, -n, or -x options, the key is 8-bytes whatever the user supplies. Administrators can use the utility to display state information, registration and reservation information, and key definitions. The vxfenadm utility supports both character and numeric key formats.
-k key    :    key is a character string. If key is less than 8 bytes in length, then vxfenadm will append it with – (hyphen). For keys shorter than 8 bytes, use -K KEY where KEY is in the numeric (ascii value) format.
-K KEY    :   KEY is in numeric (ascii value) format. For example, the equivalent numeric format for the key APGR000 is 65,80,71,82,48,48,48,0. Because the key is seven bytes long, 0 is used for the eighth byte.
vxfenclearpre :  The vxfenclearpre utility provides a user with a way to easily clear all SCSI3 registration and reservation keys from the set of coordinator disks as well as the set of shared data disks. This applies to CVM Disk Groups only, as Private Disk Groups have their keys cleared when the disk groups are imported.
Warning: Please exercise caution and make sure that there are no other active nodes in the cluster when running this. If there are other nodes present and accessing shared storage, data corruption will result!
If you enter the following command to register a key AVCS:
# vxfenadm -m -k AVCS -f disk_filename
-k AVCS will convert the key to AVCS —- before the SCSI-3 command is issued to the device.  AVCS will register as AVCS—- on the disk.
The same holds for other SCSI-3 commands. If you try to unregister a key with character format that is less than 8 bytes, the unregistration fails. 

::: To  Read the keys registered on a disk using the vxfenadm

# vxfenadm -g all -f disk_filename
Device Name: <diskpath>
Total Number Of Keys: 2
key[0]:
     Key Value [Numeric Format]: 65,80,71,82,48,48,48,0
     Key Value [Character Format]: APGR000
If you unregister the key using the following command, the unregistration fails:
# vxfenadm -x -k APGR000 -f disk_filename
VCS FEN vxfenadm ERROR V-11-2-1204 Command 0 FAILED for:diskpath
VCS FEN vxfenadm ERROR V-11-2-1120 Unregistration FAILED for:diskpath
VCS FEN vxfenadm ERROR V-11-2-1133 Error returned
In this case, the key registered on the disks is APGR000. But the vxfenadm command implicitly converted the key to APGR000-. Hence the unregistration failed. The unregistration will succeed with the numeric format.

::: To successfully unregister the key, enter the following command:

# vxfenadm -x -K 65,80,71,82,48,48,48,0 -f disk_filename
VCS FEN vxfenadm NOTICE Deleted the key:[APGR000] from device diskpath

::: To clear all reservations on a disk, enter the following command at a node having reservation on the disk:

# vxfenadm -c -k VF000001 -f disk_filename
DMP Clear completed for disk path: disk_path

Scenario 2: Remove SCSI3 registrations and reservations from disks.

Consider a setup with 3 coordinator disks and 2 data disks.
coordinator disks : disk1 disk2 disk3
data disks : disk4 disk5
Coordinator disks are disks listed in the file /etc/vxfentab and the data disks are disks which are part of online shared disk groups.
Suppose the coordinator disks have keys as follows
disk1: VFDEED01 VFBEAD01
disk2: VFDEED01 VFBEAD01
disk3: VFDEED01 ABCDEFGH
 
and the data disks have keys as follows
disk4: APGR0001 BPGR0001
disk5: ZZZZZZZZ BPGR0001
1. vxfenclearpre -t cluster -c 56079 will clear the following keys
disk1: VFDEED01 (Cluster ID: 57069 Node ID: 1)
disk2: VFDEED01 (Cluster ID: 57069 Node ID: 1)
disk3: VFDEED01 (Cluster ID: 57069 Node ID: 1)
 
2. vxfenclearpre -t vxfen will clear the following keys
disk1: VFDEED01 VFBEAD01
disk2: VFDEED01 VFBEAD01
disk3: VFDEED01
3. vxfenclearpre -t all will clear the following keys
disk1: VFDEED01 VFBEAD01
disk2: VFDEED01 VFBEAD01
disk3: VFDEED01 ABCDEFGH
4. vxfenclearpre will clear the following keys
disk1: VFDEED01 VFBEAD01
disk2: VFDEED01 VFBEAD01
disk3: VFDEED01 ABCDEFGH
disk4: APGR0001 BPGR0001
disk5: ZZZZZZZZ BPGR0001
Ramdev

Ramdev

I have started unixadminschool.com ( aka gurkulindia.com) in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' unixadminschool.com. You can connect me at - https://www.linkedin.com/in/unixadminschool/

1 Response

  1. September 18, 2015

    […] Read – How to manage SCSI-3 disk registrations and reservations for I/O fencing using vxfenadm and vxfenclearpre? […]

What is in your mind, about this post ? Leave a Reply

Close
  Our next learning article is ready, subscribe it in your email

What is your Learning Goal for Next Six Months ? Talk to us