Analyzing netstat

Netstat is one important utility available in Linux environment which helps in analyzing various information about the network connections. Netstat command displays various network related information such as network connections, routing tables, interface statistics, masquerade connections, multicast memberships etc. A basic usage is provided here.

In this article we will how we can analyze the output generated by the netstat command. If we run the command netstat we will see the output some thing like this,

Active Internet connections (w/o servers)

Proto Recv-Q Send-Q Local Address               Foreign Address                     State

tcp        0      0          rem.nova.com:ssh        eth0.MCS2.nova.co:48376      ESTABLISHED

tcp        0      0          local.localdomain:18923 localhost.localdomain:37458  TIME_WAIT

udp     8465   0          rem.nova:officelink2000 eth0.DCS2.nova.com:19401  CLOSE_WAIT

tcp        0      0          rem.nova.com:29010      eth0.TCS2.nova.co:40400     FIN_WAIT2

So what exactly does the Output say?

Proto: The proto column says the type of the protocol. We can see the TCP or UDP available under the proto columns. These are network protocols. TCP makes a Reliable connection but it slows down if the network quality is bad. On the other side UDP stays fast but some times it may loose packets.

Recv-Q & Send-Q: These columns tell us how much of data is in the Queue for the socket , waiting to be read or sent. if these  values are zero then every thing is going fine. If we check that above output we will see that there is a 8465 unread bytes in them.

Local and Foreign Address: The local and foreign address tell us about the host and ports that the sockets are connected. In other words we can think foreign address the remote hosts that we are connected and from the local address we can get the port that we are connected to.

In the last column we can see that local address rem.nova.com with Port 29010 was talking to eth0.TCS2.nova.co on port 40400. If you look closely at the example, you’ll see that two sockets have localhost as the Foreign Address. It means the computer is talking to itself over the network.

State: The State column is a very important column in the output. It tells us about the state of the Socket. The TCP protocol defines the state as,

ESTABLISHED: Ready for communication

CLOSE_WAIT: This state tell us that the foreign or the remote machine has already closed the connection but they are still not closed by the local machine and also if we check the same cokect with CLOSE_WAIT status has the 8465 bytes to read.

FIN_WAIT_1: This state indicates that the connection is still active but not being uses at this point

FIN_WAIT_2: This state indicates that the client just received a acknowledgement signal for the first FIN signal sent

LAST_ACK:   Indicates that the server is in the process of sending its own FIN signal

LISTENING:  Indicates that the server is ready to accept a connection

SYN_RECEIVED: Indicates that the server just received a SYN signal from the client

SYN_SEND:  Indicates that this particular connection is open and active

TIME_WAIT: Indicates that the client recognizes the connection as still active but not currently being used

If we add more arguments to the netstat command like “netstat -tulpn” we can see the PID that initiated the Connection.

Normally the CLOSE_WAIT is the one that we need to take care of the most. As sais this state indicates us that the foreign or the remote machine has already closed the connection but they are still not closed by the local machine. CLOSE_WAIT indicates that the destination has closed the connection. TIME_WAIT indicates that local endpoint ie. Source has closed the connection. The connection is being kept around so that any delayed packets can be matched to the connection and handled appropriately.

Is there any way to kill a Socket that is in a State?

A Socket in any state can be killed with out killing the parent process using,

netstat -anp | grep ':80 ' | grep CLOSE_WAIT | awk '{print $7}' | cut -d \/ -f1 | grep -oE "[[:digit:]]{1,}" | xargs kill

The above command can be used in killing all the CLOSE_WAIT sockets with the port number 80.

Now going to the next Output, we will get details about the network interface available in the system using

Dev:nove:killer-~ $ netstat -i

Kernel Interface table

Iface       MTU Met    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg

eth0       1500   0 426838981      0      0        0       453048218      0       0        0     BMRU

lo          16436  0  3646680         0      0        0       3646680          0       0       0      LRU

The above output gives us information about the network interface available. The explanations for the columns are

RX-OK: Correct packets received on this interface. 
RX-ERR: Incorrect packets received on this interface 
RX-DRP: Packets that were dropped at this interface.
RX-OVR: Packets that this interface was unable to receive. 

I will add some more analysis of the output from netstat command.A better way of analyzing the network issues can be done using the Wire shark tool here