Linux Tips - Beyond the Basics (Part 2)

Find log files in / last modified within 1 day

This is great when you are on an unfamiliar system or don’t know which log file you need to look at.

find / -name "*.log" -mtime -1

Interactively look at recent logs

Each log file modified within 1 day is constantly streamed. Simply retry the operation and watch the new log messages appear on the screen.

find / -name "*.log" -mtime -1 | xargs tail -f

Parsing Command Output

It is good to be able to quickly parse output from the command line. Fortunately with awk and grep, many operations are possible with basic knowledge.

Hypothetically, let’s say you want to find the number of network packets on the eth1 interface.

There are some easy commands to get this other commands to get this information, but, for the sake of learning, this example just uses basic parsing.

First the ifconfig command tells us this info on each interface, including the number of received packets, RX packets:

eth1      Link encap:Ethernet  HWaddr 08:00:27:B0:6A:94

         inet addr:192.168.56.101  Bcast:192.168.56.255  Mask:255.255.255.0

         inet6 addr: fe80::a00:27ff:feb0:6a94/64 Scope:Link

         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

         RX packets:1522 errors:0 dropped:0 overruns:0 frame:0

         TX packets:1461 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:1000

         RX bytes:133938 (130.7 KiB)  TX bytes:339721 (331.7 KiB)

lo        Link encap:Local Loopback

         inet addr:127.0.0.1  Mask:255.0.0.0

         inet6 addr: ::1/128 Scope:Host

         UP LOOPBACK RUNNING  MTU:65536  Metric:1

         RX packets:0 errors:0 dropped:0 overruns:0 frame:0

         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:0

         RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

But we want to only focus on the RX packets. Let's first eliminate the loopback interface. You could easily run ifconfig eth1, but it is also possible to grep for the data.

The following will focus on the "eth1" line and also display 5 lines after the eth1 match to get the RX packets line:

ifconfig | grep eth1 -A5

eth1      Link encap:Ethernet  HWaddr 08:00:27:B0:6A:94

         inet addr:192.168.56.101  Bcast:192.168.56.255  Mask:255.255.255.0

         inet6 addr: fe80::a00:27ff:feb0:6a94/64 Scope:Link

         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

         RX packets:1556 errors:0 dropped:0 overruns:0 frame:0

         TX packets:1502 errors:0 dropped:0 overruns:0 carrier:0

Next, let’s keep trying to focus on the RX packets:

ifconfig | grep eth1 -A5 | grep RX  

[root@localhost tmp]# ifconfig | grep eth1 -A5 | grep RX

         RX packets:1572 errors:0 dropped:0 overruns:0 frame:0

Now it is down to parsing this line for the number of packets, say 1572...

The awk command can be used to split a line into columns where spaces separate the fields.

Therefore, if each space in the line above represents a column we can focus on column two:

ifconfig | grep eth1 -A5 | grep RX | awk ' {  print $2 } '

packets:1572

Great, now how can we strip out the “packets:” from the raw number we are looking for?

We already know that awk is good for selecting columns from text data. As the string is now, it can also be interpreted in column format by switching the delimiter from a space to colon. Therefore, selecting column 2 using a ":" delimiter with awk will finally give us the number

[root@localhost tmp]# ifconfig | grep eth1 -A5 | grep RX | awk ' {  print $2 } ' | awk -F ":" ' { print $2 } '

1616

That's it!

What can be done with the number of packets? If you take a sample before after, say for 60 seconds one can derive a rough estimate of packets per second on the interface.

LENGTH=60

START=`ifconfig | grep eth1 -A5 | grep RX | awk ' {  print $2 } ' | awk -F ":" ' { print $2 } '`;

sleep $LENGTH

END=`ifconfig | grep eth1 -A5 | grep RX | awk ' {  print $2 } ' | awk -F ":" ' { print $2 } '`;

echo $START $END $LENGTH | awk ' { print ($2-$1)/$3 } '

Again the awk command in the last line is used to parse columns but additionally it can be used for doing simple arithmetic with the columns. For instance, packets/sec = ($END-$START)/$LENGTH

The thing to remember about this example is that this is not difficult. The solution only uses simple features of awk and grep.

This exercise can also be done without using ifconfig. As an exercise on your own try using

only the file /proc/net/dev and grep/awk commands. The same metric, RX packets, can be found here:

Inter-|   Receive                                                |  Transmit

face |bytes    packets errs drop fifo frame compressed multicast|bytes    packets errs drop fifo colls carrier compressed

 eth1:  359967    4110    0    0    0     0          0         1   786631    3985    0    0    0     0       0          0

 eth2:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0

   lo:       0