Redhat Enterprise Linux Networking Troubleshooting – Quick Reference

 

1. Network does not start due to “Device not managed by Network-Manager” error.

Generally, this error occurs when the ethernet cable does not connect to the interface well and the interface is not configured properly.

The solution for this issue is to ensure the link connectivity and configure network interface.

Also, restart ‘network’ and ‘NetworkManager’ services.
Diagnostic Steps

Ensure the status of ‘network’ and ‘NetworkManager’ services and restart them.

# service network status
# service NetworkManager status

To restart:

# service network restart
# service NetworkManager restart

Check if there any error messages while restarting these services.

If the issue does not resolve by the solutions mentioned above, please follow the instructions below.

Go to System->Preferences->Network Connections->Wired and ‘edit’ eth0 to make sure that ‘Connect automatically’ is enabled. If not, please enable it and restart ‘NetworkManager’ service.

2. Frame errors in network Interface – What does it mean?

ifconfig output shows errors related to frame field of network interface and we are noticing connectivity and performance problems.

Frame Errors: Shows the number of packets received incorrectly having a CRC error and a noninteger number of octets. On a LAN, this is usually the result of collisions or a malfunctioning Ethernet device.

Check the “frame” and “errors” field in ifconfig command output:

# ifconfig
According to output below, it’s possible to see that some interfaces had errors in the transfer of packets, as can be seen in the fields: “errors” and “frame”.

RX packets:93849080 errors:276295 dropped:0 overruns:0 frame:276295

For each interface belonging to bond, enter the command below to get NIC statistics:

# ethtool -S ethX

Look at “rx_frame_errors:”, this line will show frame errors too and you can collect more info about a lot of other data.

We need to check below things :

  • Physical devices like network cables, NICs and switch ports and configuration.
  • Switch configuration about speed and auto-negotiation are the same as in the server.

For example:
Switch was working with HALF-DUPLEX without “Autonegotiation”
In the above cases, switch configuration need to be changed to work in FULL-DUPLEX mode to get the connection established with expected performance and eliminate frame errors.

  • Server was connected to a switch which had an incompatible configuration to establish communication or you have a damaged NIC, cable or port.

3. Network restart throws the error “some other host already uses address”

Scenario :  host1 ( with 199.9.200.101 IP ) reporting the error “ifup:Error, some other host already uses address x.x.x.x”
And then the Servers won’t rejoin network after reboot. There are no other host in the network with the same IP.

Network will not start after reboot, says IP address in use

The above error message returned by the Following section from in script /etc/sysconfig/network-scripts/ifup, which is actuallu responsible to check duplicate IPS during network start

if ! arping -q -c 2 -w 3 -D -I ${REALDEVICE} ${IPADDR} ; then
echo $”Error, some other host already uses address ${IPADDR}.”
exit 1
fi

To resolve the problem, we should run a packet sniffer such as tcpdump on the system would help to know about the source MAC address of such system/device. Then we should be able to match this ‘MAC’ address with the switch MAC table and find out which system/device is connected to that port.

# tcpdump -i any -w /tmp/tcpdump.pcap

::::::  output truncated ::::::

Packet 25#  26.820528 xx:xx:xx:xx:xx:xx ARP Who has 199.9.200.101 ? Tell 0.0.0.0

Packer 26 # 26.820533 xx:xx:xx:xx:xx:xx ARP Who has 199.9.200.101 ? Tell 0.0.0.0

Above packets are arp requests sent from the system (with MAC xx:xx:xx:xx:xx:xx ) asking who is having the ip address 192.168.0.1.

Packet 27#  26.820633 Intel_yy:yy:yy ARP 199.9.200.101 is at yy:yy:yy:yy:yy:yy

Packet27 is a ARP reply from a Cisco device ( having MAC address of yy:yy:yy:yy:yy:yy ) saying 199.9.200.101 ip is at yy:yy:yy:yy:yy:yy , which is its own MAC.

4. How to map the physical network interface corresponds to which ethX device?

There are several ways of identifying network interfaces.

A Media Access Control (MAC) address, or hardware address, is unique to each interface. This can be used to identify an interface.

# ifconfig -a | grep HWaddr
eth0 Link encap:Ethernet HWaddr 00:16:3E:7B:8B:49
eth1 Link encap:Ethernet HWaddr 00:16:3E:7C:9B:49

The command ethtool’s -p option initiates adapter-specific action intended to enable an operator to easily identify the adapter by sight. Typically this involves blinking one or more LEDs on the specific Ethernet port. Note that not all network interfaces support this.

# ethtool -p eth0

The command ethtool can be used to determine if an interface is connected to an active device (it detects a link/carrier). If a link is not detected then the interface is not connected to the network.

# ethtool eth0 | grep Link
Link detected: yes

5. How do I add/delete additional Logical IP addresses to a network interface in RHEL6?

Since NetworkManager is usually disabled in server environments, either we can configure the temporary IP’s using the IP command or create them persistently by modifying the configuration files.

Adding or removing IP addresses manually:

Use the ip command provided by the iproute package.

To dislplay the existing IP addresses in a network interface:
# ip addr show eth0

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:71:98:9d brd ff:ff:ff:ff:ff:ff
inet 192.168.122.101/24 brd 192.168.122.255 scope global eth0
inet 192.168.122.12/24 scope global secondary eth0
inet 192.168.122.11/24 scope global secondary eth0
inet 192.168.122.13/24 scope global secondary eth0
inet6 fe80::5054:ff:fe71:989d/64 scope link
valid_lft forever preferred_lft forever

To delete an existing IP address:

# ip addr del 192.168.122.13/24 dev eth0
# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:71:98:9d brd ff:ff:ff:ff:ff:ff
inet 192.168.122.101/24 brd 192.168.122.255 scope global eth0
inet 192.168.122.12/24 scope global secondary eth0
inet 192.168.122.11/24 scope global secondary eth0
inet6 fe80::5054:ff:fe71:989d/64 scope link
valid_lft forever preferred_lft forever

To add an IP address:

# ip addr add 192.168.122.13/24 dev eth0
# ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:71:98:9d brd ff:ff:ff:ff:ff:ff
inet 192.168.122.101/24 brd 192.168.122.255 scope global eth0
inet 192.168.122.12/24 scope global secondary eth0
inet 192.168.122.11/24 scope global secondary eth0
inet 192.168.122.13/24 scope global secondary eth0
inet6 fe80::5054:ff:fe71:989d/64 scope link
valid_lft forever preferred_lft forever

Configuring additional IP addresses persistently:

Edit the corresponding /etc/sysconfig/network-scripts/ifcfg-eth<x> configuration file by using the following additional entries:

IPADDR<n>: the additional IP address.
PREFIX<n>: the length in bits of the netmask for the additional IP address.
NETMASK<n>: the explicit netmask value for the additional IP address.
BROADCAST<n>: the broadcast address for the additional IP address.

and add as many additional IPADDR<n> and PREFIX<n> entries as additional IP addresses are required.

For example the following configuration file:

# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
BOOTPROTO=none
NETMASK=255.255.255.0
TYPE=Ethernet
HWADDR=52:54:00:cc:de:0b
IPADDR=192.168.100.101
PREFIX=24
IPADDR2=192.168.128.101
PREFIX2=24
IPADDR3=192.168.130.101
PREFIX3=28

would give the following result:

# ip addr show eth1
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:cc:de:0b brd ff:ff:ff:ff:ff:ff
inet 192.168.100.101/24 brd 192.168.100.255 scope global eth1
inet 192.168.128.101/24 brd 192.168.128.255 scope global eth1
inet 192.168.130.101/28 brd 192.168.130.111 scope global eth1
inet6 fe80::5054:ff:fecc:de0b/64 scope link
valid_lft forever preferred_lft forever

6. Network interface does not come up after reboot; must be manually brought up with ifup

Ensure the interface config file (for example, /etc/sysconfig/network-scripts/ifcfg-ethX) contains:

:::

ONBOOT=yes

::::

Furthermore, ensure that either the network service or the NetworkManager service are enabled.

# chkconfig network on

7. How do I run a script or program immediately after my network interface goes up?

The network control scripts in /etc/sysconfig/network-scripts allows for adding scripts that will run after the device is up.
Notice the last few lines search for a file called /sbin/ifup-local. If that file is found, it is executed.

For example, looking at the /etc/sysconfig/network-scripts/ifup file, the last few lines are:

:::::::::::: snip  :::::::::::::

if [ “${IPX}” = yes ]; then
/etc/sysconfig/network-scripts/ifup-ipx ${DEVICE}
fi

exec /etc/sysconfig/network-scripts/ifup-post ${CONFIG} ${2}

A script /etc/sysconfig/network-scripts/ifup-post is the last item to be executed. Looking at that script:

if [ -x /sbin/ifup-local ]; then
/sbin/ifup-local ${DEVICE}
fi

exit 0

From the above code we can identify the file /sbin/ifup-local is the one that is available to run programs immediately after the interface is up.

For Example, create file /sbin/ifup-local and add the commands to run after the network is up

————————————-
#!/bin/bash
if [ “$1” == “eth0” ]; then
/sbin/ethtool -G $1 rx 4096 tx 4096
fi
————————————-

Make /sbin/ifup-local executable

# chmod +x /sbin/ifup-local

8. How to Configure more than one default gateway for a multi-home system ( i.e. had two or more separate Network Interface Cards)?

Scenario : Each network card goes to a separate network (ie. eth0 goes to external network, eth1 goes to internal).Each interface needs to access the network gateway to gain access to other networks.

It is not logically possible to have more than one default gateway. A default gateway is more accurately described as a “catch-all” gateway

A single desired default GATEWAY= address should be configured in the /etc/sysconfig/network file. All other references to the GATEWAY= directive should be removed from ifcfg-eth* files. Additionally, If you wish to specify which NIC device is used to carry gateway traffic, the GATEWAYDEV=ethX directive should be used.

If more than one default gateway is configured, network packets may be routed incorrectly, leading to unpredictable behaviour. Due to the way in which initscripts functions, when the GATEWAY= directive is defined in multiple ifcfg-eth* network scripts, whichever device is brought up last is the one which will define the gateway used by the system.

Alternatively, if we can create a static route from the each interface to pass the specific network related traffic.

For Example: If we have a system with 3 ethernet interfaces(eth0, eth1 and eth2), and want to configure a single default gateway as 10.0.0.1 and gatewaydev as eth1.

# cat /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=localhost
GATEWAY=10.0.0.1
GATEWAYDEV=eth1
# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=static
HWADDR=A1:B2:C3:D4:E5:F6
IPADDR=192.168.0.55
NETMASK=255.255.255.0
NETWORK=192.168.0.0
BROADCAST=192.168.0.255
ONBOOT=yes
# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
BOOTPROTO=static
HWADDR=A1:B2:C3:D4:E5:E5
IPADDR=10.0.0.66
NETMASK=255.255.255.0
NETWORK=10.0.0.0
BROADCAST=10.0.0.255
ONBOOT=yes
# cat /etc/sysconfig/network-scripts/ifcfg-eth2
DEVICE=eth2
BOOTPROTO=static
HWADDR=A1:B2:C3:D4:E5:E4
IPADDR=10.0.0.66
NETMASK=255.255.255.0
NETWORK=10.0.0.0
BROADCAST=10.0.0.255
ONBOOT=yes

Note that none of the above ifcfg-eth* files contain the GATEWAY= directive.

On all other interfaces which must use a separate gateway, create static routes to all networks requiring them.

Below shows a system, that when configured correctly as shown above, will show the gateway being used as 10.0.0.1 on device eth1.

$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
0.0.0.0 10.0.0.1 0.0.0.0 UG 0 0 0 eth1

9. What is the “ethtool” command and how can I use it to obtain information about my network devices and interfaces?

The ethtool command is a low-level interface manipulation tool with the power to directly effect firmware and driver behavior and functionality on the active interface.

Before performing any tasks it is recommend to review the network topology involved to ensure all hardware directly supports any task you are enabling, such as is the case with auto-negotiation or jumbo frames. Please note that each driver will behave differently and have different supported operations while in use, however the ethtool generic commands should be fully compatible across different driver varieties.

Most functions with ethtool can be performed live. Data collection has little to no impact on the running system, and direct manipulation of functions such as the ring buffer or the enabling or disabling of checksumming can typically be done live without any interrupt in service.

The ethtool facility, with no commands run, prints basic information about the Ethernet card for the interface in which it is run:

# ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
[…]

If you suspect packet loss or if the output of a utility such as # ifconfig shows drops, overruns, or errors, you have the ability to directly poll the card for packet statistics at boot. Please keep in mind that this data is only current since the last functioning operational state of the Ethernet controller, typically cleared at the reboot of a system.

# ethtool -S eth0
NIC statistics:
rx_packets: 2251518
tx_packets: 1410087
rx_bytes: 1295872987
tx_bytes: 288388610
rx_broadcast: 409192
tx_broadcast: 11
rx_multicast: 34163
tx_multicast: 123
rx_errors: 0
tx_errors: 0
tx_dropped: 0
multicast: 34163
[…]

Note: keep in mind these parameters are defined by the Ethernet controller, firmware, and driver that is specific to your Ethernet card, and the values contained in such may be calculated radically differently on alternative hardware. For assistance with this, please contact Red Hat Support or the driver vendor for an analysis.

Some other various data collection functions of ethtool consist of:

# ethtool -a eth*
=> shows pause parameters for interface listed*

# ethtool -g eth*
=> shows ring buffer parameters for interface listed*

# ethtool -i eth*
=> shows driver and firmware versions, as well as physical bus address for interface listed*

# ethtool -k eth*
=> shows offload information for functions such as tcp checksumming for interface listed*

The ethtool utility can also be used to make non-persistant operational changes to the working interface, typically replacing the data-collection flag with the capital letter flag of the same type. For example, to change the ring buffer on an active interface without the need to bring the interface down, we first view the available options for the buffer:

# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX: 4096 <–Maximum size for the RX(receive) ring buffer
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 256 <–Current value, which can be set to the maximum.
RX Mini: 0
RX Jumbo: 0
TX: 256

And then we can use the ethtool command to make the change interactively, without the need to take down the interface:

# ethtool -G eth0 rx 4096

Our interface should then be set to maximum value on the ring buffer. Please note this is a low level change on the interface, and as such, will not persist over reboot. To ensure any changes remain persistent, either specify them in the /etc/rc.local script which is run every time the system reboots, or in the /etc/sysconfig/network-scripts/ifcfg-eth* file specify the function you desire performed with the “ETHTOOL_OPTS” tag:

# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=”eth0″
HWADDR=”00:00:00:00:00″
NM_CONTROLLED=”no”
ONBOOT=”yes”
ETHTOOL_OPTS=”speed 100 duplex full autoneg off”

 

 10.  How do I set up VLAN (802.1q) tagging on a network interface ?.

1. Ensure that the module is loaded:

# lsmod | grep 8021q

2. If the module is not loaded, load it with the following command:

# modprobe 8021q

3. Configure your physical interface in /etc/sysconfig/network-scripts/ifcfg-ethX, where X = the interface number:

DEVICE=ethX
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes

4. Configure the VLAN interface configuration in /etc/sysconfig/network-scripts.

The configuration filename should be the physical interface plus a ‘.’ character plus the VLAN ID number.
For example, if the VLAN ID is 192, and the physical interface is eth0, then the configuration filename should be ifcfg-eth0.192:

DEVICE=ethX.192
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.1.1
NETMASK=255.255.255.0
USERCTL=no
NETWORK=192.168.1.0
VLAN=yes

If there is a need to configure second vlan (ID 193) on the same interface (eth0), add a new file with the name eth0.193 with the vlan configuration details.

5. Restart networking:

#service network restart

11. How can I route network traffic such that the packets go out via the same interface they came in?

Scenario: Example : server has 2 interfaces with IP address.

  • eth0 – > inet addr:10.66.1.51 Bcast:10.66.255.255
  • eth1 – > inet addr:10.67.1.51 Bcast:10.67.255.255

We want to forward network packets to the same network interface from where it came from. We can implement this feature by using policy-based routing (source routing). The iproute2 package provides the tools (/bin/ip) to configure this.

Setup 2 more route table with different table ids; 10.66.255.254 is the gateway for eth0 and 10.67.255.254 is the gateway for eth1.

# ip route add default via 10.66.255.254 dev eth0 table 1
# ip route add default via 10.67.255.254 dev eth1 table 2

Above command creates 2 routing table. Create rules to forward all the packets entering via a particular nic to go out the appropriate routing table.

# ip rule add iif eth0 table 1
# ip rule add iif eth1 table 2

Create rules to forward packets to go out the specific routing table.

# ip rule add from 10.66.1.51 table 1
# ip rule add from 10.67.1.51 table 2

If you want to add more routing records, following is the command format

# ip route add to 192.168.100.0 via 10.66.0.203 dev eth0 table 1

Add the above commands into scripts file and make it run on next boot.

12. Setting persistent network interfaces using udev in Red Hat Enterprise Linux 6

Scenario : When the system boots network interfaces are sometimes reordered causing the interfaces to be named differently. During a system boot, the system orders network interfaces as they come up.Once the system is booted, other network dependent tasks can fail because interfaces are not named properly.

The DEVICE= option in the ifcfg file does not determine the mapping of subchannels to network device names.

Instead, the udev rules file /etc/udev/rules.d/70-persistent-net.rules determines which network device channel gets which network device name.If network dependent services such as bonding begin to act differently, ensure interface details are matching their names by using ifconfig

To resolve the isssue, we can create Persistent network interfaces which are  bound through the creation of udev rules.

First, configure the interfaces correctly, ensuring they are in the correct order.

Then the rules can be created by running below command:

Check the current Udev rules:

# cat /etc/udev/rules.d/70-persistent-net.rules

# echo add > /sys/class/net/ethX/uevent

run this command with all the interface in use, and replace the X with the interface number.

Verify the udev rules have been added by checking the rules file:

# cat /etc/udev/rules.d/70-persistent-net.rules

Which should output something similar to:

# PCI device 0x1af4:0x1000 (virtio-pci) (custom name provided by external tool) <br />SUBSYSTEM==”net”, ACTION==”add”, DRIVERS==”?

The restart the network:

# service network restart

13. Network interface failes due to the delay in bringing up the link from switch side.

Scenario : After a server boots up, a DHCP based network interface is not working and has no IP address The following error may be found in /var/log/dmesg:

ADDRCONF(NETDEV_UP): eth0: link is not ready

Some switches do not bring up link quickly, so the network link layer is not available at the moment that the DHCP client attempts to retrieve an IP address. This can also occur in a peer-to-peer network, ie, when connecting two systems together directly via network interface. It has also been observed that some network interface cards are themselves slow to finish link negotiation during initialization post boot, leading to link not being available at the right moment.

To Workaround the problem, from the server side, we can increase the LINKDELAY value ( default is 10 seconds configured in /etc/sysconfig/network-scripts/network-functions)

Add below directive to /etc/sysconfig/network-scripts/ifcfg-ethX and tune for the most effective value:

LINKDELAY=60

The restart the interface manually

# ifdown ethX
# ifup ethX

14. How to configure network interfaces, after fresh installation.They are not available after reboot?

On the first boot the interfaces are configured to not be enabled on boot. To allow the interfaces up on boot, you need to edit the configuration bellow:

1.Identify if your interface was named with ’em’ or ‘eth’ nomenclature:

# ls /etc/sysconfig/network-scripts/

2.Edit the file:

# vim ifcfg-emX
or
# vim ifcfg-ethX

3.Change the parameter ‘ONBOOT’ from ‘no’ to ‘yes’:


ONBOOT=yes

4.Restart the network service:

# service network restart

5.Verify if the interfaces were recognized:

#ifconfig
eth0 Link encap:Ethernet HWaddr 02:00:00:00:00:01
inet addr:192.168.70.87 Bcast:192.168.70.255 Mask:255.255.255.0
inet6 addr: fe80::ff:fe00:1/64 Scope:Link
UP BROADCAST RUNNING NOARP MULTICAST MTU:1492 Metric:1
RX packets:23 errors:0 dropped:0 overruns:0 frame:0
TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:644 (644.0 b) TX bytes:264 (264.0 b)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:16204 errors:0 dropped:0 overruns:0 frame:0
TX packets:16204 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:91855053 (87.5 MiB) TX bytes:91855053 (87.5 MiB)

15. Why does network interface show “rx_errors” in ifconfig output or what is rx_crc_errors ?

Scenario : Ifconfig output of network interface ethX , shows following errors :

#ifconfig
eth0 Link encap:Ethernet HWaddr 00:26:55:7B:FB:40
inet addr:172.21.141.28 Bcast:172.21.141.255 Mask:255.255.255.0
inet6 addr: fe80::226:55ff:fe7b:fb40/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:161417251 errors:5068645 dropped:0 overruns:0 frame:5068645

From “ethtool -S eth0” output, these rx_errors are basically “rx_crc_errors: 5068645”.

$ ethtool -S eth0
….
rx_crc_errors: 5068645

Normally CRC errors indicate a problem at the layer 1, To Troubleshoot the problem :

  • Connect interface to a different port.
  • Change the cable.
  • Use a different NIC if available.
  • Set DUPLEX=FULL SPEED=1000 AUTONEG=ON as per the capability of NIC.

16. How to calculate right Prefix for the Netmask to configure in the ifcfg-eth* files, usin ipcalc?

Find the subnetmask from the prefix value:

[root@rhel5 networking]# $ ipcalc 192.168.10.0/24
Address: 192.168.10.0 11000000.10101000.00001010. 00000000
Netmask: 255.255.255.0 = 24 11111111.11111111.11111111. 00000000
Wildcard: 0.0.0.255 00000000.00000000.00000000. 11111111
Network: 192.168.10.0/24 11000000.10101000.00001010. 00000000
HostMin: 192.168.10.1 11000000.10101000.00001010. 00000001
HostMax: 192.168.10.254 11000000.10101000.00001010. 11111110
Broadcast: 192.168.10.255 11000000.10101000.00001010. 11111111
Hosts/Net: 254 Class C, Private Internet

Find the Prefix from the subnetmask and broadcast address value:

[root@rhel5 networking]# ipcalc -m -b 10.18.1.255 255.255.254.0 -n -p
NETMASK=255.255.254.0
PREFIX=23
BROADCAST=10.18.1.255
NETWORK=10.18.0.0

17. Connecting 2 network interfaces on the same subnet.

Scenario : When there are 2 interfaces on the same subnet there is no assurance as to which interface will be used to transmit traffic and the machine will accept traffic for either IP on either interface. This is because in Linux the IP address belongs to the host and is not associted with the interface.

Also, if you ping with “-I<dev>”, attempting to use a given interface, there is no guarantee the reply packet (if there even is one) will come back to the same interface.

 

/—- NIC 1 –> 10.64.208.180 –>

Linux — |                                                     – —-> Destination Hosts

—- NIC 2 –> 10.64.208.208 –> /

 

To Configure this setup, we have to add tables binding source IP address for each route and add those as default gateway for each network interface.
In this setting, we would need application having capability to handle each network devices with balanced loads by itself.

Assuming these networking enviroment :

+——————————————+
|                             Linux                              |
| eth0                                                   eth1 |
| 10.64.208.180       10.64.208.208 |
+——————————————+

 # ip addr show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:16:3e:74:8d:aa brd ff:ff:ff:ff:ff:ff
inet 10.64.208.180/24 brd 10.65.211.255 scope global eth0
inet6 fe80::216:3eff:fe74:8daa/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:16:3e:74:8d:b2 brd ff:ff:ff:ff:ff:ff
inet 10.64.208.208/24 brd 10.65.211.255 scope global eth1
inet6 fe80::216:3eff:fe74:8db2/64 scope link
valid_lft forever preferred_lft forever

Configuring Policy based IP routing:

Step 1. Add tables in /etc/iproute2/rt_tables

# vim /etc/iproute2/rt_tables
100 t1
101 t2

Step 2. Add routing table to t1, t2 :

# ip route add 10.64.208.0/24 dev eth0 src 10.64.208.180 table t1
# ip route add table t1 default via 10.64.208.254 dev eth0
# ip route show table t1
10.64.208.0 dev eth0 scope link src 10.64.208.180
default via 10.64.208.254 dev eth0

# ip route add 10.64.208.0/24 dev eth1 src 10.64.208.208 table t2
# ip route add table t2 default via 10.64.208.254 dev eth1
# ip route show table t2
10.64.208.0 dev eth1 scope link src 10.64.208.208
default via 10.64.208.254 dev eth0

To make the Changes done in Step 1) and 2) set these in the file :

Below entries in File : ifcfg-eth0

DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
NETMASK=255.0.0.0
IPADDR=10.64.208.180
GATEWAY=10.64.208.254
TYPE=Ethernet

Below entries in File : ifcfg-eth1

DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
NETMASK=255.0.0.0
IPADDR=10.64.208.208
GATEWAY=10.64.208.254
TYPE=Ethernet

Below entries in File : /etc/sysconfig/network-scripts/route-eth0

10.0.0.0/8 dev eth0 src 10.64.208.180 table t1
default via 10.64.208.254 dev eth0 table t1

Below entries in File : /etc/sysconfig/network-scripts/route-eth1

10.0.0.0/8 dev eth1 src 10.64.208.208 table t2
default via 10.64.208.254 dev eth1 table t2

Step 3. Setting table

# ip rule add table t1 from 10.64.208.180
# ip rule add table t2 from 10.64.208.208
# ip route show
10.64.208.0/24 dev eth0 proto kernel scope link src 10.64.208.180
10.64.208.0/24 dev eth1 proto kernel scope link src 10.64.208.208
169.254.0.0/16 dev eth1 scope link
default via 10.64.208.254 dev eth0

To make the Changes done in Step 1 and 2 , persistent, set these in the below files, as shown :

Below entry in File : /etc/sysconfig/network-scripts/rule-eth0

table t1 from 10.64.208.180

Below entry in File : /etc/sysconfig/network-scripts/rule-eth1

table t2 from 10.64.208.208

4. Set both of interfaces ready for receiving reply.

# sysctl net.ipv4.conf.default.arp_filter=1

To make thise Step 4 changes , persistent, add entries to the configuration files as shown below

Add below entry in the file : /etc/sysctl.conf

net.ipv4.conf.all.arp_filter = 1

5. Checking ping with ‘-I <IPADDR>’

# ping -I 10.64.208.180 <DSTADDR>

18. How to enable jumbo frames for network interfaces in Red Hat Enterprise Linux?

To enable jumbo frames, first ensure the device driver supports custom MTU sizes. Edit the network configuration script of the relevant interface .

For example, /etc/sysconfig/network-scripts/ifcfg-eth0. Add a line similar to the following which specifies the size of the frame in bytes:

MTU=9000

If the change is made while the interface is active, it will need to be restarted (brought down then up) for the new maximum frame size to take effect.

The networking stack in Red Hat Enterprise Linux 4 and above supports custom maximum frame sizes (Maximum Transmission Units, or MTUs) for Ethernet network interfaces.

It is important to realize that for large frame sizes to be effective, every intermediate network device between the sender and receiver must support large frames. If one or more devices does not support large frames then network performance can be severely adversely affected.

Another factor to consider is that the Linux kernel is significantly faster at making allocations of two pages of memory (8192 bytes) than three pages. Since three pages would be necessary to contain a frame of size 9000 bytes, better performance might be achieved in some scenarios by setting an MTU of around 8000 bytes.

19. Loopback interface lo deleted with system-config-network

The loopback interface was deleted using the system-config-network GUI.  After a reboot The loopback interface (lo) is renamed by the system with dev386 (or any other random number).

Below steps will recover the the loopback interface (lo) renamed with the correct entry on your system:

Rename dev386 in lo:

# ip link set dev386 name lo

Add in the file /etc/modprobe.conf the following line for persistents at reboot :

alias lo tg3
check if the file /etc/sysconfig/network-scripts/ifcfg-lo exist, If not please create it under this location with this content:
DEVICE=lo
IPADDR=127.0.0.1
NETMASK=255.0.0.0
BROADCAST=127.255.255.255
ONBOOT=yes
NAME=loopback

Enable the lo interface:

# ifup lo

Ramdev

Ramdev

I have started unixadminschool.com ( aka gurkulindia.com) in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' unixadminschool.com. You can connect me at - https://www.linkedin.com/in/unixadminschool/

13 Responses

  1. sajeer rehman says:

    Excellent document..covered almost everything in n/w trouble shooting .I request you
    to post a document about Linux Cluster

  2. Ramdev Ramdev says:

    Hi Sajeer, Thanks for the Comment.  Linux Cluster is a broad concept, and it might take little more time for the consolidation :)

  3. peacengell says:

    Nice tutorials

    Love this tut thanks mate

  4. Shiv says:

    I really would like to thank you for all efforts you are putting, keep up the good work.

  5. Yogesh Raheja says:

    Thanks Shiv, we will try our level best to keep it up all the times.

  6. Venkatesh says:

    Hi Ramdev, is there any wasy to test linkloop connectivity between 2 interfaces to check interface belongs to same VLAN? I am settip up bond so need to check which 2 interface are on same VLAN?

  7. Venkatesh says:

    ifconfig bond1

  8. Ramdev Ramdev says:

    normally I do tcpdump on each interface , to watch the network segment broadcast it is receiving.

  9. Venkatesh says:

    Thanks Ramdev, how can i analyze tcpdump to check VLAN connectivity? What s/w can i use to read tcpdump o/p?

    In HP i do fairly simple

    #linkloop -i but in redhat not sure?

  10. raju says:

    Really Amazing troubleshooting articles which you are providing us, Thanks much for your articles……

  11. suresh says:

    Excellent document. Thank you very much.

  1. September 16, 2015

    […] Read – Linux Networking Troubleshooting Quick Reference Guide […]

What is in your mind, about this post ? Leave a Reply

Close
  Our next learning article is ready, subscribe it in your email

What is your Learning Goal for Next Six Months ? Talk to us