VxVM Learning – Resize veritas volume without using vxresize

The most common Veritas Volume Manager Operation that we do in everyday sysadmin job is, resizing Veritas volumes. In this below post we are discussing some of the scenarios to resize different types of veritas volumes.

Increase Concat Volume Size

Once the extra space is available in the diskgroup we can grow the volume and then it’s file system. Initially find out how much space, in sectors, is now available to use.

In this example we are using disks c4t0d0 & c9t0d0 to grow appsvol02:

# vxassist -g appsdg maxgrow appsvol02 c4t0d0 c9t0d0
Volume appsvol02 can be extended by 35250176 to: 212047494 (103538Mb+1670 sectors)

Now we know the amount of sectors we can growby this amount.

we can use below syntax to grow the volume

vxassist -g growby
or
vxassist -g growto

# vxassist -g appsdg growby appsvol02 35250176 c4t0d0 c9t0d0 &
Note: we are adding “&” at the end of the command just to run the command in background).
Once you grow the volume we should increase the filesystem created on top of volume, using the below fsadm command.

syntax: fsadm [-F vxfs] [-b newsize] [-r rawdev] mount_point

#/opt/VRTSvxfs/sbin/fsadm -F vxfs -b 212047494 /mnt_point

Increase Striped Volume Size

To grow a “n – way” Striped volume, we need “n” no. of new disks to be available in the diskgroup with the required space.

Example for wrong way of growing a stripped volume:

Diskgroup striedg having “gukrul_vol”, a 2way striped, volume. And we are trying to increase it’s size by adding another disk “striped03” to the diskgroup.

dg stripedg default default 125000 1006935392.1115.sptsunvm5

dm striped01 c1t1d0s2 sliced 2159 8378640 –
dm striped02 c1t3d0s2 sliced 2159 8378640 –
dm striped03 c1t4d0s2 sliced 3590 17678493 –

v gurkul_vol – ENABLED ACTIVE 16756736 SELECT gurkul_vol-01 fsgen
pl gurkul_vol-01 gurkul_vol ENABLED ACTIVE 16757392 STRIPE 2/128 RW
sd striped01-01 gurkul_vol-01 striped01 0 8378640 0/0 c1t1d0 ENA
sd striped02-01 gurkul_vol-01 striped02 0 8378640 1/0 c1t3d0 ENA

When we tried to check the maximum available size to grow, we face below error.

# vxassist -g stripedg maxgrow gurkul_vol
vxvm:vxassist: ERROR: Volume gurkul_vol cannot be extended within the given constraints

Correct method to increase striped volume size:

add two newdisks to the stripedg to increae the 2way striped volume i.e gurkul_vol. and the configuration should look like below before growing the volume

dg stripedg default default 125000 1006935392.1115.sptsunvm5

dm striped01 c1t1d0s2 sliced 2159 8378640 –
dm striped02 c1t3d0s2 sliced 2159 8378640 –
dm striped03 c1t4d0s2 sliced 3590 17678493 -dm striped04 c1t5d0s2 sliced 2159 8378640 –

v gurkul_vol – ENABLED ACTIVE 16756736 SELECT gurkul_vol-01 fsgen
pl gurkul_vol-01 gurkul_vol ENABLED ACTIVE 16757392 STRIPE 2/128 RW
sd striped01-01 gurkul_vol-01 striped01 0 8378640 0/0 c1t1d0 ENA
sd striped02-01 gurkul_vol-01 striped02 0 8378640 1/0 c1t3d0 ENA

now the maximum possible size to grow the volume ~16GB

# vxassist -g stripedg maxgrow gurkul_vol
Volume gurkul_vol can be extended from 16756736 to 33513472 (16364Mb)

What is the alternative method if we have only one disk with the required additional size?

dg stripedg default default 125000 1006935392.1115.sptsunvm5

dm striped01 c1t1d0s2 sliced 2159 8378640 –
dm striped02 c1t3d0s2 sliced 2159 8378640 –
dm striped03 c1t4d0s2 sliced 3590 17678493 –

v gurkul_vol – ENABLED ACTIVE 16756736 SELECT gurkul_vol-01 fsgen
pl gurkul_vol-01 gurkul_vol ENABLED ACTIVE 16757392 STRIPE 2/128 RW
sd striped01-01 gurkul_vol-01 striped01 0 8378640 0/0 c1t1d0 ENA
sd striped02-01 gurkul_vol-01 striped02 0 8378640 1/0 c1t3d0 ENA

Answer is you shouldn’t use vxresize command to grow the volume in this case, you have to relayout the volume using the new additional disk as mentioned below

# vxassist -g stripedg relayout gurkul_vol layout=stripe,nolog ncol=+1 striped03
or
# vxassist -g stripedg relayout gurkul_vol ncol=3 striped01 striped02 striped03

Important Note : to use the above command to work you must have at least 10% of additional free disk space in the same volume. Otherwise the above command fail as below

# vxassist -g stripedg relayout gurkul_vol ncol=3 striped01 striped02 striped03
vxvm:vxassist: WARNING: dm:striped01: No disk space matches specification
vxvm:vxassist: WARNING: dm:striped02: No disk space matches specification
vxvm:vxassist: ERROR: Cannot allocate space for 1675008 block volume
vxvm:vxassist: ERROR: Relayout operation aborted. (7)

Once you grow the volume we should increase the filesystem created on top of volume, using the below fsadm command.

syntax: fsadm [-F vxfs] [-b newsize] [-r rawdev] mount_point

#/opt/VRTSvxfs/sbin/fsadm -F vxfs -b 33514460 /mnt_point

 

Complex Scenario for Veritas Volume Manager Operations

In Disaster Recovery setup, it is common to have applications configured on the clusters where each cluster node is located in different site ( data center).  Each site is having it’s own storage device ( like EMC DMX ) which is configured to replicated the information from the other Storage device of other site.

There are many vendor side tools to replicate the data from one storage device to another device without bothering the cluster nodes for replication, but they are expensive. If the organisation chose to go for manual replication of storage data at the cluster node level, it will create lot more additional overhead to system administrator to maintain veritas volumes.

Here I  am presenting a sample server and storage architecture, to help you understand the similar setup in real time.

Below is the cluster setup where the cluster-node1 at site1 acts as production server, and the cluster-node2 at site2 acts as secondary server which is active only in disaster recovery mode.

 

 

>> Initial Vxprint configuration for above setup looks like below

Disk group: ora_dg

V  NAME         RVG/VSET/CO  KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
SV NAME         PLEX         VOLNAME  NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
SC NAME         PLEX         CACHE    DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
DC NAME         PARENTVOL    LOGVOL
SP NAME         SNAPVOL      DCO
EX NAME         ASSOC        VC                       PERMS    MODE     STATE

v  data04       –            ENABLED  ACTIVE   20480000 SELECT    –        fsgen <== volume length is “20480000”
pl data04-01    data04       ENABLED  ACTIVE   20480000 CONCAT    –        RW
sd D2740_R5_0CF6-01 data04-01 D2740_R5_0CF6 0  20480000 0         emcpower5 ENA
pl data04-02    data04       ENABLED  ACTIVE   LOGONLY  CONCAT    –        RW
sd D2740_R5_0CF6-02 data04-02 D2740_R5_0CF6 20480000 528 LOG      emcpower5 ENA
pl data04-03    data04       ENABLED  ACTIVE   20480000 CONCAT    –        RW
sd D2665_R5_0CF6-01 data04-03 D2665_R5_0CF6 0  20480000 0         emcpower28 ENA
pl data04-04    data04       ENABLED  ACTIVE   LOGONLY  CONCAT    –        RW
sd D2665_R5_0CF6-02 data04-04 D2665_R5_0CF6 20480000 528 LOG      emcpower28 ENA

 

Task : Increase the Size of the “Data04” volume in “oradg” diskgroup

Once you add disks to the disk groups , the real challenge here is increasing size of the volume without disturbing the above shown setup. In below you can see the right and wrong way of adding disks to the volume for the purpose of resize.

If we use vxassist/vxresize commands we cannot really control disks that is attaching each side of mirror component with in the volume.  That is the one reason we need to go with plex and subdisk level operation to increase the size of the volume.

 

 

Below steps describes the procedure to increase the size of the volume 

 

please note we are using two disks to increase the mirror plexes of volume, and they are named as D2665_R5_1125-01 & D2740_R5_0901-01 while adding original disks to disk group. And the naming convention helps you to understand which disk is coming from which storage ( first 5 letters talks about the EMC device ID )

>>>>>Create Veritas Subdisks using the new SAN disks

#vxmake -g ora_dg sd D2665_R5_1125-01 D2665_R5_1125,0,20898000
#vxmake -g ora_dg sd D2740_R5_0901-01 D2740_R5_0901,0,20898000

>>> associate new subdisks to both sides of mirror

#vxsd -g ora_dg assoc data04-01 D2740_R5_0901-01
#vxsd -g ora_dg assoc data04-03 D2665_R5_1125-01

>>> check the Volume configuration

# vxprint -ht -v data04
Disk group: ora_dg

V  NAME         RVG/VSET/CO  KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
SV NAME         PLEX         VOLNAME  NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
SC NAME         PLEX         CACHE    DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
DC NAME         PARENTVOL    LOGVOL
SP NAME         SNAPVOL      DCO
EX NAME         ASSOC        VC                       PERMS    MODE     STATE

v  data04       –            ENABLED  ACTIVE   20480000 SELECT    –        fsgen  <=== the overall volume size hasn’t changed and still showing 20480000
pl data04-01    data04       ENABLED  ACTIVE   41378000 CONCAT    –        RW
sd D2740_R5_0CF6-01 data04-01 D2740_R5_0CF6 0  20480000 0         emcpower5 ENA
sd D2740_R5_0901-01 data04-01 D2740_R5_0901 0  20898000 20480000  emcpower65 ENA
pl data04-02    data04       ENABLED  ACTIVE   LOGONLY  CONCAT    –        RW
sd D2740_R5_0CF6-02 data04-02 D2740_R5_0CF6 20480000 528 LOG      emcpower5 ENA
pl data04-03    data04       ENABLED  ACTIVE   41378000 CONCAT    –        RW
sd D2665_R5_0CF6-01 data04-03 D2665_R5_0CF6 0  20480000 0         emcpower28 ENA
sd D2665_R5_1125-01 data04-03 D2665_R5_1125 0  20898000 20480000  emcpower64 ENA
pl data04-04    data04       ENABLED  ACTIVE   LOGONLY  CONCAT    –        RW
sd D2665_R5_0CF6-02 data04-04 D2665_R5_0CF6 20480000 528 LOG      emcpower28 ENA

>>> Trying to expand the filesystem failed because adding the subdisks to the plex doesn’t automatically increase the volume

# /opt/VRTSvxfs/sbin/fsadm -F vxfs -b 41378000 /mnt_dir
UX:vxfs fsadm: ERROR: V-3-25811: cannot expand /dev/vx/rdsk/ora_dg/data04 more than size of the underlying device – 20480000 sectors

>>>> check the contig_len of the volume

#vxprint -g ora_dg -m data04-01 |grep contig_len
41378000

>>> set the length of the volume as per the contig_len value

#vxvol -g ora_dg set len=41378000 data04

>>>checking the volume configuration

# vxprint -ht -v data04
Disk group: ora_dg

V  NAME         RVG/VSET/CO  KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
SV NAME         PLEX         VOLNAME  NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
SC NAME         PLEX         CACHE    DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
DC NAME         PARENTVOL    LOGVOL
SP NAME         SNAPVOL      DCO
EX NAME         ASSOC        VC                       PERMS    MODE     STATE

v  data04       –            ENABLED  ACTIVE   41378000 SELECT    –        fsgen  <=== the overall volume size has changed to 41378000
pl data04-01    data04       ENABLED  ACTIVE   41378000 CONCAT    –        RW
sd D2740_R5_0CF6-01 data04-01 D2740_R5_0CF6 0  20480000 0         emcpower5 ENA
sd D2740_R5_0901-01 data04-01 D2740_R5_0901 0  20898000 20480000  emcpower65 ENA
pl data04-02    data04       ENABLED  ACTIVE   LOGONLY  CONCAT    –        RW
sd D2740_R5_0CF6-02 data04-02 D2740_R5_0CF6 20480000 528 LOG      emcpower5 ENA
pl data04-03    data04       ENABLED  ACTIVE   41378000 CONCAT    –        RW
sd D2665_R5_0CF6-01 data04-03 D2665_R5_0CF6 0  20480000 0         emcpower28 ENA
sd D2665_R5_1125-01 data04-03 D2665_R5_1125 0  20898000 20480000  emcpower64 ENA
pl data04-04    data04       ENABLED  ACTIVE   LOGONLY  CONCAT    –        RW
sd D2665_R5_0CF6-02 data04-04 D2665_R5_0CF6 20480000 528 LOG      emcpower28 ENA
ssngc420002por:root#

>>> Extend the filesystem to the level that volume increased

#/opt/VRTSvxfs/sbin/fsadm -F vxfs -b 41378000 /mnt_dir
#

Finally you can check the file system size with ”  df -k /mnt_dir  ” which reflects the new size of the volume.

 

Ramdev

Ramdev

I have started unixadminschool.com ( aka gurkulindia.com) in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' unixadminschool.com. You can connect me at - https://www.linkedin.com/in/unixadminschool/

23 Responses

  1. vignsh says:

    Hi gurukul frendz, pls post vxvm plex related issues and troubleshooting scenario.

  2. Yogesh Raheja says:

    Hi Vignsh, We will work on your request and come up with the plex realated posted soon.

  3. Ramdev says:

    Hi Vignesh,

    Please go through the below posts, and you can search the site with “vxvM” to find more related posts.

    http://gurkulindia.com/main/2011/04/vxvm-veritas-plex-state-transition/ http://gurkulindia.com/main/2011/05/vxvm-recovering-an-unstartable-volume-with-a-disabled/

  4. Sateesh says:

    it is good one.

  5. Saurabh says:

    Hi,
    What is the difference between vxresize and vxassist ?

  6. Ramdev Ramdev says:

    @Saurabh — vxassist with growby/growto + fsadm ( for filesystem expansion) = vxresize

  7. Yogesh Raheja says:

    @Saurabh, In simple terms one can say.. vxassist is only use to craete/increase “Volumes” only (best example is in case of creation/increase of RAW devices for databases) here no effects will be noticed in the output of df -h as only volume is increased. On the other hand vxresize is a combination of vxassist (increase of volume) + fsadm (expansion of Filesystem) as mentioned by Ram. But vxresize is not used in the volume creation it will only support expansion/compression.

  8. Saurabh says:

    Thanks Ram and Yogesh. I got this now. :)

  9. Yogesh Raheja says:

    @Saurabh, you are most welcome:)

  10. Anuj says:

    Could you provide a good example for growto & growby and shrinkto & shrinkby ?

  11. Ramdev Ramdev says:

    hi anuj , do you mean example with syntax or the concept?

  12. Anuj says:

    Yes Ram….just a clear cut explanation….
    I am also looking for VxVM in Linux… like Booting process, Root disk failed and system not booting up …etc…

  13. Anuj says:

    In Solaris… entries will be added to /etc/system and for Linux where does it sit ?

  14. Yogesh Raheja says:

    @Anuj, for Linux you have /etc/sysctl.conf which act as a system file. The main difference in Solaris (/etc/system) and Linux (/etc/sysctl.conf) is that the systcl.conf file can be re-read by Linux kernel while the system is up by sysctl -p command whereas if you will ammend any new entry in /etc/system you need to reboot the box to re-read the parameters for Solaris Kernel.

  15. Anuj says:

    Thxs Yogesh… 
    Does the Veritas entries will be under /etc/sysctl.conf or /etc/grub.conf or some other files?

  16. Ramdev Ramdev says:

    anuj – veritas changes will go to /etc/sysctl.conf. Grub.conf only used when you have to change boot related kernel options.

  17. Yogesh Raheja says:

    @Anuj, always remember /etc/system (Solaris) –> /etc/sysctl.conf (Linux)

  18. anuj says:

    growto & growby and shrinkto & shrinkby diffrences ?

  19. akshay says:

    Hi, I am looking 4 job change. I upload my resume on job portals. i just learned the veritas volume manager. but i never faced interview regarding VXVM. I dont know wht kind of que. ask in intrview about VXVM.
    can u pls me to sloved this issue. I just want 2 know what kind of que may be aksed in interview.
    Thanks in advance…

    • Ramdev Ramdev says:

      Hi Akshay,

      the interview purely depends on kind of environment. Below are some of the basic concepts that you should be aware of:

      – VxVM deamons and purpose.
      – how to initialize disks ( local/san) in veritas
      – how to perform disk maintenance for failed volumes from mirrored volumes ( both root and non-root)
      – how to create log files
      – how to move veritas volumes from one host to another host
      – how to perform online extension/resize for mirror volumes and raid5 volumes
      – how to configure veritas disks with using veritas disk multipath ( DMP)

  20. raj says:

    hi,

    add me in the network.
    raj

  1. September 18, 2015

    […] Read – Resize veritas volume without using vxresize […]

What is in your mind, about this post ? Leave a Reply

Close
  Our next learning article is ready, subscribe it in your email

What is your Learning Goal for Next Six Months ? Talk to us