Configuring Solaris 10 host with SAN storage ( EMC Clariion )

Ramdev

I have started unixadminschool.com ( aka gurkulindia.com) in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' unixadminschool.com. You can connect me at - https://www.linkedin.com/in/unixadminschool/

Loading Facebook Comments ...

48 Responses

  1. Sourav Chatterjee says:

    thanks ramkumar great post ……
    please post on migration:- host based and storage based

    cheers

  2. very good site ,helping me more and my colligues.thanks alot…please i need some basics of san storage…(EMC)

  3. seema says:

    Hello Ram

    i was trying to label San disks buit i am left with errors cannot label it. when i wanted to add to datapool .. i have a san disk can u please tell is initialising and formatting disk on svm opr zfs same since on veritas we can use vxdiskadm utility to initialise
    ……

  4. Yogesh Raheja says:

    Also let us know the OS version..

  5. seema says:

    OS version ” Oracle Solaris 10 9/10 SPARC
    Error: # echo | format
    Searching for disks…done

    c1t5006048ACAFDF44Cd0: configured with capacity of 44.06MB
    c2t5006048ACAFDF443d0: configured with capacity of 44.06MB

    AVAILABLE DISK SELECTIONS:
    0. c0t0d0
    /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/sd@0,0
    1. c0t1d0
    /pci@0,600000/pci@0/pci@8/pci@0/scsi@1/sd@1,0
    2. c1t5006048ACAFDF44Cd0
    /pci@1,700000/SUNW,emlxs@0/fp@0,0/ssd@w5006048acafdf44c,0
    3. c1t5006048ACAFDF44Cd258
    /pci@1,700000/SUNW,emlxs@0/fp@0,0/ssd@w5006048acafdf44c,102
    4. c2t5006048ACAFDF443d0
    /pci@1,700000/SUNW,emlxs@0,1/fp@0,0/ssd@w5006048acafdf443,0
    5. c2t5006048ACAFDF443d258
    /pci@1,700000/SUNW,emlxs@0,1/fp@0,0/ssd@w5006048acafdf443,102
    6. emcpower0a
    /pseudo/emcp@0
    Specify disk (enter its number): Specify disk (enter its number):

    partition> label
    [0] SMI Label
    [1] EFI Label
    Specify Label type[0]: 1
    Warning: This disk has an SMI label. Changing to EFI label will erase all
    current partitions.
    Continue? y
    Warning: error writing EFI. and

  6. seema says:

    Ram can you please delete above comment since it has echo| format ouput

    • Gurkulindia says:

      no problem..we need “echo|format” output anyway. But I couldn’t see which disk you are trying to label here. I could see powerpath installed on this machine. please be clear what disks you planned to use.

      And Just login to chat, and see if you can find either me / yogesh in the chat, probably we can talk to you in chat to resolve the issue.

  7. seema says:

    Sure will discuss in chat … let me know what timezone will you guys be avaiable ….

    Once again thanks …

  8. Yogesh Raheja says:

    @seema, as Ram stated you can try login into the Chat. If you find anyone of us we will discuss the same over their.
    Also from the outputs (I suppose it truncated) it is showing only one powerpath device which means there is only one path coming from the HBA. also first of all Lable below two disks and then proceed with labeling the emcpower#c. Note: you have to label the albhabet “c” not “a” i.e emcpower0c. Try and let us know the result plz.

  9. Yogesh Raheja says:

    I am available on chat now. If you are there.

  10. Yogesh Raheja says:

    Also I have checked the echo | format output of yours, it seems all devices are already labeled. Error: # echo | format
    Searching for disks…done

    c1t5006048ACAFDF44Cd0: configured with capacity of 44.06MB
    c2t5006048ACAFDF443d0: configured with capacity of 44.06MB

    “NO UNLABELLED DEVICE PRESENT” AVAILABLE DISK SELECTIONS:

  11. seema says:

    all are raw devices they are reclamed from aix OS… @yogesh please specify ur available timings…thanks

  12. Yogesh Raheja says:

    @seema, I will try my best to login tonite 00:00 hrs (as I will be coming home at midnite only).

  13. Yogesh Raheja says:

    @Seema, try it: One problem faced when issued ‘label’ command was:

    > label
    Cannot label disk when partitions are in use as described.

    And the solution for this:

    The environment variable NOINUSE_CHECK (see PSARC/2005/461), when set, can be used to turn off this new functionality:

    NOINUSE_CHECK=1
    export NOINUSE_CHECK

  14. sonu says:

    is it possible to rename the multipathed disk say “c2tXXXXXXXXXXXXXXd2s2” to c3tXXXXXXXXXXXXXXd2s2 ( changing the c2 to c3) or is it fixed and cannot be

  15. Ramdev says:

    you can play little bit on controller numbers if you know exactly how /etc/path_to_install works, but I really don’t prefer to do this. Solaris will identify the controller numbers during the boot, as per the rules mentioned in the post.

    http://gurkulindia.com/main/2011/04/solaris-how-solaris-assigns-controller-numbers/

    And if we mess it up with them, we can’t recover the server back during the crashes.

  16. sonu says:

    Thanks Ram.found a way out for this, which worked ..in case you want to change the controller number..going to /dev/cfg/ and renaming the controller number say ( #mv c2 c3)..
    and then removing all the old device link from /dev/dsk and /dev/rdsk ( rm -r c3* ).. and then running devfsadm -C …

    • Ramdev says:

      @sonu, good tweak. I am just curious why do you need to change this? And are you doing this on you work machine or personal machine?
      And I am also curious to know how your system will behave if you add more devices like EMC/External storage( which usually takes C3 , C4 ..etc ).

      In this case you actually removed the device links ( with specific major and minor numbers), that means the server no longer able to recognize the disks having those missing major and minor numbers.And you don’t realize this abnormal behavior until you add new external disks to the machine.

      Thanks for posting your experiments :)

  17. sonu says:

    sorry was stuck up with something…yes that was the requirement from the one of the team to see the same disk name in all the similar servers..

  18. Shekar says:

    By running this cmd /etc/powercf -q ,it create psuedo devices for every new LUN only or will it destroy the existing suedo names .pl advise ?

    • Ramdev says:

      Hi Sekhar, powercf -q updates emc configuration by removing PowerPath devices that were not found in the host adapter scan and by adding new PowerPath devices that were found. Saves a primary and an alternate primary path to each PowerPath device.

  19. shekar says:

    Thanks for your update Ram .The main intention i hv asked you , i want to increase the File System space .SAN team team have added 20g
    so by running cfgadm -al and decfsadm -C ,hv detected cx7ty4000…..d2 and cx7ty4000…..d2 (20g), so if i run
    1 powercf -q ( in my case i,say t detects newly created 2 psueudo names)
    or can i skip this step and continue 2 because

    in the above blog ,it says “The above PowerPath command will also DESTROY psuedo devices (emcpowerX devices only) for LUNs that were previously managed by PowerPath but have disappeared from the system” will this destroy my exising t empowerpath which is running with less space does it mean that ?

    2 powermt config
    if u run powermt display dev=all , i shud see as per the eg in step 1 psueduo devices say eg /emcpower5a /emcpower6a

    3 powermt save
    4.I beleive no reboot is required on Solaris 10 zfs FS .

    On Solaris 10 ( zfs ) to increase space

    5 format
    select the newly created device

    12 cx7ty4000…..d2
    14 cx8ty4000…..d2

    Say i hv labeled by selecting 14 ,then

    Can i run zpool add the “current poolname newl” cx8ty4000…..d2

    will this increase my FS on zfs on the fly or is there any steps missed out .pl advise .

  20. Yogesh Raheja says:

    @Sheker, Firstly powercf -q is to remove/flush/washed the storage(LUNS) which are not in use and has been removed by storage team physically. So no need to worry about existing FS/Vol. unless and untill your stoarge is attached to the server.
    Secondly in simple layman language powermt config is to configure the new SAN provide to the server so that EMC internally recongnise the new luns (consider it for scaaning purpose).
    thirdly powermt save is to save the configuration so that next time when you reboot the server your configuration wont disapperas. Suppose you have configured the SAN and forgot to save its configuration then you will loss the current configution if your system gets rebooted. and again you have to clean/configure/save the configuration for your system so every time powermt saveis must and recommended step.
    4th ZFS dont required reboot, the task can be done on the fly. 5th zpool add will add the LUN in you Pool (i.e disk group)
    5th yes you are absolutely correct that adding a disk in pool will give you the space in FS as ZFS take the space from zpool instead of individual FS space.

  21. shekar says:

    Thanks Yogesh . In case of Solaris 8 and 9 , 1 Do i have add any SAN /Emc powerpath details /kernek/drv/ in sd.conf and lpcf.conf. ? 2do i need to reebot ?

  22. Yogesh Raheja says:

    @Shekar, while expanding your LUNS nothing required, but if you are migrating any of the emc version (like powerpath/symmertix/hba firmware,driver version etc etc) then yes you have to keep copy of these files and have to put the wwn no. which storage team will provide.

  23. Yogesh Raheja says:

    @Shekar, below gurkul posts will helps you to understand storage concepts:  http://gurkulindia.com/main/2011/12/pre-planningpre-work-required-for-storage-migrations-in-unix/

    http://gurkulindia.com/main/2011/12/emcpowerpath-emcsymmetrix-upgrades-on-solaris-servers/

    http://gurkulindia.com/main/2012/01/hba-firmware-upgrades-on-solaris-servers/

  24. Santosh says:

    Hi Ram, can you let me know the limitations to a Solaris administrator who is at level 2 (L2 Support).
    i.e. what tickets/issues he will take care and what tickets/issues he will escalates to next level?

    please i need this information …

  25. Ramdev says:

    @santhosh – for a startup company / a company where you have not many teams, level2 is every thing that is related to the environment. But if you are talking about the MNC organizations there you will see clear differentiation between level2 and level3 responsibilities based on the servers/services they manage. Normally Level3 manages the core infrastructure services like NIS , DNS , LDAP, jumpstart services, automation scripts , DR / BCP setup…etc. L2 admins will be dedicated to the maintenance and configuring existing servers related to application services. And also handles the requests that required to access core infra servers(but not related to the configuration of the core infra servers) . And for some companies L2 admins also responsible for regular server installations and configurations.

  26. pitmod says:

    Thanks. really helpful post.
    cheers

  27. Ramdev says:

    @pitmod – you welcome

  28. Murali says:

    HI,
    can some one aswer the the following questions for me,

    1.) Do we really need POWERPATH to be installed on Solaris 10 server to connect to EMC clarion ?
    2.) Higer version of POWERPATH in Solaris 10 server can able to connect to the EMC Clarion having lower version of POWERPATH?

  29. Ramdev says:

    @murali – Solaris 10 supports Mpxio for multipath and you can use it  instead of power path .

    .  And basically powerpath is server side software used to configure multiple access path to storage device( clarion) to provide connection redundancy.Emc website will have compatibility matrix ( commonly called san stack ) which talks about the compatibility between the EMC device and power path version

  30. Ramachandra says:

    Hi Ram,

    I would like to clarify one doubt regarding EMCPowepath,Currently we using IBM HS22 blades and VNX EMC storage and its connected via->Brocade SAN with Emulex Lpe HBAanywhere driver.

    Currently we are running in EMCPowerpath mode and we want migrate to normal multipath and OS is SusE.

    here are the questions:;

    1.Is it is required to recreate HBA Zoning and uninstall Powerpath 1st before going for server scratch (We are going for server scratch as we also need reconfigure most of IP part and other stuff)
    2.What happens when we only re-scratch server without touching SAN and HBA part..will it automatically again install PowerPath or it will insatll Multipath software..

    Thanks in advance..

  31. Ramdev says:

    @ramachandra – i am assuming your  word ‘server scratch’ means “server reinstall”. My understanding is, as long as the HBA’s are not changing on the server there will be no  zoning and allocation again.. About the powerpath, anyway that will go away once you reinstall the OS.  You can select the native multipath products during install, and can configure after the installation.

  32. Ramachandra says:

    @Ram..

    But we faced situation after server re installation system was already come up with Emc powerpath,
    1.Actually its very huge network and So we are using almost 100servers and 3 EMC storage of 5000GB capacity,and we are using a Install server to re install other servers here…meanwhile we also updated firmware for Brocade and Emulex..with Fabric OS firmware 6.4.2b for Brocade and lpfc 0:8.2.0.106.1p… but in one location we reconfigured HBA 1st and then we re installed servers there it took native multipath..

    We are trying to debug why in other set of servers where we not reconfigured Zone,HBA and SAN switch and we just re installed servers come up with powerpath instead of native multipath ?

    And please let us how server will decide it has to go for powerpath or native multipath in re installation if possible …

    Thanks…

    • Ramdev says:

      @ramachandra – sorry for late response, I am in short vacation and away from my desk. About your issue, i would recommend to check the installation logs for both the servers to see when exactly the powerpath packages getting installed and getting activated. And also please check if you can disable SAN storage/HBAs ( from bios) before starting the installation. And then enable them later after the installation.

  33. solaris351 says:

    How to find the SAN type in the solaris server?.Please help me on this…

  34. Yogesh Raheja says:

    Hi, in simplest way you can get an idea from format command only.

  35. Mani Kishore says:

    Hi Ram Thanks for u r greate support.

  36. Ramdev says:

    Hi Mani, It’s my pleasure

  37. siva says:

    Hi Ram and Friends,

    Thanks for your valuable discussion about Host side configuration and power path configuration with Storage.

    I want to know the best process to clean up at host side once luns have been reclaimed from the host. If the devices under control of veritas or and svm also.

    And can u please provide Fcode upgradation procedure.

    Thanks in advance….

  38. Saradhi says:

    Is there any similar blog for Redhat Linux or Centos? configuring Linux host with SAN storage emc-clarion

  1. November 26, 2013
  2. September 17, 2015

    […] Read – Configuring Solaris 10 host with SAN storage ( EMC Clariion ) […]

Leave a Reply

Your email address will not be published.

[contact-form to='ramkumar.ramadevu@gmail.com' subject='New Learning Request Submitted'][contact-field label='Name' type='name' required='1'/][contact-field label='Email' type='email' required='1'/][contact-field label='Learning Request' type='textarea' required='1'/][contact-field label='Are you Looking for ' type='radio' required='1' options='Paid Training,Free Training'/][/contact-form]

What is your Learning Goal for Next Six Months ? Talk to us