VCS : Configuring Private Network for veritas cluster

VCS requires you to set up a private network between the systems that form a cluster. You can use either NICs or aggregated interfaces to set up private network.

You can use network switches instead of hubs. However, Sun systems assign the same MAC address to all interfaces by default. Thus, connecting two or more interfaces to a network switch can cause problems.

For example, consider the following case where:

    • The IP address is configured on one interface and LLT on another
    • Both interfaces are connected to a switch (assume separate VLANs)

The duplicate MAC address on the two switch ports can cause the switch to incorrectly redirect IP traffic to the LLT interface and vice versa. To avoid this issue, configure the system to assign unique MAC addresses by setting the eeprom(1M) parameter local-mac-address to true.

The following products make extensive use of the private cluster interconnects for distributed locking:

    • Veritas Storage Foundation Cluster File System (CFS)
    • Veritas Storage Foundation for Oracle RAC (SF Oracle RAC)

Symantec recommends network switches for the CFS and the SF Oracle RAC clusters due to their performance characteristics.

Below Figure shows two private networks for use with VCS.

Figure: Private network setups: two-node and four-node clusters

Symantec recommends configuring two independent networks between the cluster nodes with a network switch for each network. You can also interconnect multiple layer 2 switches for advanced failure protection. Such connections for LLT are called cross-links.

Below figure  shows a private network configuration with crossed links between the network switches.

Figure: Private network setup with crossed links

To set up the private network


1. Install the required network interface cards (NICs).

Create aggregated interfaces if you want to use these to set up private network.

 2. Connect the VCS private Ethernet controllers on each system.

 3. Use crossover Ethernet cables, switches, or independent hubs for each VCS communication network. Note that the crossover Ethernet cables are supported only on two systems.

Ensure that you meet the following requirements:

 • The power to the switches or hubs must come from separate sources.

• On each system, you must use two independent network cards to provide redundancy.

• If a network interface is part of an aggregated interface, you must not configure the network interface under LLT. However, you can configure the aggregated interface under LLT.

• When you configure Ethernet switches for LLT private interconnect, disable the spanning tree algorithm on the ports used for the interconnect.

 During the process of setting up heartbeat connections, consider a case where a failure removes all communications between the systems. Note that a chance for data corruption exists under the following conditions:

 • The systems still run, and

• The systems can access the shared storage.

 4. Configure the Ethernet devices that are used for the private network such that the autonegotiation protocol is not used. You can achieve a more stable configuration with crossover cables if the autonegotiation protocol is not used.  To achieve this stable configuration, do one of the following:

 • Edit the /etc/system file to disable autonegotiation on all Ethernet devices system-wide.

• Create a qfe.conf or bge.conf file in the /kernel/drv directory to disable autonegotiation for the individual devices that are used for private network.


5. Test the network connections. Temporarily assign network addresses and use telnet or ping to verify communications.

 LLT uses its own protocol, and does not use TCP/IP. So, you must ensure that the private network connections are used only for LLT communication and not for TCP/IP traffic. To verify this requirement, unplumb and unconfigure any temporary IP addresses that are configured on the network interfaces.

 The installvcs program configures the private network in the cluster during configuration.



I have started ( aka in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' You can connect me at -

9 Responses

  1. Ram says:

    I have one request. We have a server with zone configured already. We are planning to install and configure solaris clustering (2 node clustering – (zone clustering for the existing zone). Zone is arleady running on seperate LUN not in the rpool. Can you please let me know the steps to configure that.

    For example, we have a server A, Server B and Zone name already configured on Server A is zone-A. Now we want to setup clustering so that Zone-A can failover to Server B.
    We already have private network setup on these servers.

  2. Yogesh Raheja says:

    @Ram, for the complete steps we nned to set up the same environment. But yes this can be done and is now a days widely used by many clients having VCS. For zones to be included in the VCS you need to craete a seperate Resource Type for Zones. This resource type will instruct VCS to failover/failback the zones from one server to other one. Apart from that rest of the configuration will remain the same for VCS. Additionaly you also need to specify the zones dependency as well. Also if you are doing this first time in your environment, Symantec will help you with some pre-defined steps for how to step up zones in VCS. Hope this helps.

  3. Rashid Nihal says:

    Add me up for upcoming news and technical information on VCS

  4. kishan says:

    Grateful if someone can advise on the licensing required based the following hardware setup: there are five servers and one standby server. In the event of failure of any one the five servers, the services on the failed server should be failover automatically on the standby node. A quick response will be highly appreciated. Thank you.

  5. Sreeni says:

    Hi Ram

    After upgrade your blog almost /so many figures/diagrams are missed

    Could you please have look into this ?


  6. Ramdev Ramdev says:

    Hi Sreeni, i have fixed the images issue.

  1. September 18, 2015

    […] Read – VCS for Beginners – Configuring Private Network for veritas cluster […]

What is in your mind, about this post ? Leave a Reply

  Our next learning article is ready, subscribe it in your email

What is your Learning Goal for Next Six Months ? Talk to us