Solaris Troubleshooting : SDS/SVM – Replace failed disk from root mirror

You will need the following info handy to proceed with the process:

        1.  ls -l /dev/dsk/c#t#d#s# of the alternate sub-mirror boot device

        2. partition table layout of the failed boot device  

 

Part I Creating an openboot prom alias to alternate boot device

1. Run metadb -i to confirm the state database replicas on boot device is non-operational. For example:

# metadb -i

flags           first blk            block count
M     p        unknown         unknown         /dev/dsk/c0t0d0s0
M     p        unknown         unknown         /dev/dsk/c0t0d0s0
a      u        16                     1034            /dev/dsk/c0t1d0s6
a      u        16                    1034            /dev/dsk/c1t2d0s3

 

o – replica active prior to last mddb configuration change
u – replica is up to date
l – locator for this replica was read successfully
c – replica’s location was in /etc/opt/SUNWmd/mddb.cf
p – replica’s location was patched in kernel
m – replica is master, this is replica selected as input
W – replica has device write errors
a – replica is active, commits are occurring to this replica
M – replica had problem with master blocks
D – replica had problem with data blocks
F – replica had format problems
S – replica is too small to hold current data base
R – replica had device read errors

 

2. The state replicas with flags in uppercase are bad. Use the metadb command to delete them.

For example:

# metadb -d c0t0d0s0

(This will delete all state replicas on /dev/dsk/c0t0d0s0.)

If you have already defined a backup boot disk alias, go to part Part II

 

3. Bring system down to ok prompt.

4. From hardcopy output of ls -l of alternate boot device. For example:

lrwxrwxrwx   1 root     root          86 Sep 15  1997 /dev/dsk/c1t3d0s0 ->

../../devices/iommu@0,10000000/sbus@0,10001000/espdma@4,8400000/esp@4,8800000/sd@3,0:a

 

The device path string beginning after “../../devices/” will be used to create the nvalias. If you do not have a ls -l hardcopy, you can use ‘show-disks’ command to guess what is the correct device path Create the alias with this command syntax:

ok > nvalias backup_root

/iommu@0,10000000/sbus@0,10001000/espdma@4,8400000/esp@4,8800000/sd@3,0:a

5. Test the nvalias by booting from the alternate boot device by:

ok boot backup_root

 

Part II: Replacing failed boot device

 

1. Gracefully power-down the system with this command:

# init 5

2. Physically replace the failed boot device.

3. Boot from alternate boot device (ok boot backup_root)

4. Repartition new boot disk in the exact configuration as specified in the hardcopy printout of the original boot device partition table layout.

 

Part III: Repairing state replica database

 

1. If you use Solaris[TM] Volume Manager on Solaris[TM] 9 or later, update the state database with the device ID for the new disk using metadevadm -u c#t#d# .

# metadevadm -u c0t0d0

2. Once new boot disk is repartitioned, add new working state replicas back into the newly replaced disk drive. For example:

# metadb -a -c 2 c0t0d0s0

(The -c #; specifies how many replicas to put into the specified partition)

 

Part IV: Resyncing the sub-mirrors

 

1. Run metastat to find all the metadevices that the failed boot device belongs to. For example:

d0: Mirror

Submirror 0: d1

State: Needs maintenance

Submirror 1: d2

State: Okay

Pass: 1

Read option: roundrobin (default)

Write option: parallel (default)

Size: 205200 blocks

 

d1: Submirror of d0

State: Needs maintenance

Size: 205200 blocks

Stripe 0:

Device              Start Block  Dbase State        Hot Spare

c0t0d0s0                   0     No    Okay

 

d2: Submirror of d0

State: Okay

Size: 205200 blocks

Stripe 0:

Device              Start Block  Dbase State        Hot Spare

c1t2d0s0                   0     No    Okay

 

 

2. Use the metareplace command to re-enable the sub-mirror. For example:

# metareplace -e d0 c0t0d0s0

(Resync operation may take about 15-20 minutes per every gigabyte of filesystem)

 

3. Repeat metareplace command to re-enable the other sub-mirrors located on the same disk:

# metareplace -e d c0t0d0s

 

4. Reboot system to have it boot from the newly repaired boot device:

Before rebooting, wait for the resync : all metadevices must be in ‘Okay‘ state, then :

# init 6

 

Ramdev

Ramdev

I have started unixadminschool.com ( aka gurkulindia.com) in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' unixadminschool.com. You can connect me at - https://www.linkedin.com/in/unixadminschool/

6 Responses

  1. Michael Michael says:

    @ RAM we need to add the hotspares to be removed also just incase if they exsists .. good one ram 

  2. Yogesh Raheja says:

    @Michael, I dont think so that we need to remove hotspares in this case if they exists.

  3. Laxxi says:

    Hi Ram,
    I tried doing rootdisk mirroring in my local host , i added a new Ide disk in my system
    and copied the prtvtoc from root disk to the newly added disk, and then when i tried to add database replicas on both the disks, it was added for boot disk but for the newly added disk i gave the error, which is like this:
     #metadb -afc 3 c0d0s7 c1d1s7
    metadb: server : /dev/rdsk/c1d1s7: No such file or address

    Whereas the c1d1s7 is visible in format o/p.

    Please help me how to proceed on this.

  4. Ramdev Ramdev says:

    hello, is that disk path correct?.. the disk names looks like cxtxdxsx…. but i see tx missing in you disk path.

  5. Laxxi says:

    Ram,
    its an Ide disk in my vmware on my local machine. both the disks are ide. 

    • Ramdev Ramdev says:

      Hi Laxxi, Can you show the format output showing the first and second disks and also the prtvtoc for first and second disks.

What is in your mind, about this post ? Leave a Reply

Close
  Our next learning article is ready, subscribe it in your email

What is your Learning Goal for Next Six Months ? Talk to us