Why... Why... Why?
This blog is dedicated to documenting error resolution and other tidbits that I discover while working as a Consultant in the Oracle EPM (Hyperion) field. As much of my job revolves around issue resolution, I see an opportunity to supplement the typical troubleshooting avenues such as the Oracle Knowledgebase and Oracle Forums with more pinpointed information about specific errors as they are encountered. Beware, the information found in this blog is for informational purposes only and comes without any warranty or guarantee of accuracy.

EPMVirt: Create your own Oracle Hyperion Virtual Environment:

Thursday, April 16, 2015

Linux Tips - Beyond the Basics (Part 2)

Find log files in / last modified within 1 day
This is great when you are on an unfamiliar system or don’t know which log file you need to look at.
find / -name "*.log" -mtime -1

Interactively look at recent logs
Each log file modified within 1 day is constantly streamed. Simply retry the operation and watch the new log messages appear on the screen.
find / -name "*.log" -mtime -1 | xargs tail -f

Parsing Command Output
It is good to be able to quickly parse output from the command line. Fortunately with awk and grep, many operations are possible with basic knowledge.

Hypothetically, let’s say you want to find the number of network packets on the eth1 interface.
There are some easy commands to get this other commands to get this information, but, for the sake of learning, this example just uses basic parsing.

First the ifconfig command tells us this info on each interface, including the number of received packets, RX packets:

eth1      Link encap:Ethernet  HWaddr 08:00:27:B0:6A:94
         inet addr:  Bcast:  Mask:
         inet6 addr: fe80::a00:27ff:feb0:6a94/64 Scope:Link
         RX packets:1522 errors:0 dropped:0 overruns:0 frame:0
         TX packets:1461 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:133938 (130.7 KiB)  TX bytes:339721 (331.7 KiB)

lo        Link encap:Local Loopback
         inet addr:  Mask:
         inet6 addr: ::1/128 Scope:Host
         UP LOOPBACK RUNNING  MTU:65536  Metric:1
         RX packets:0 errors:0 dropped:0 overruns:0 frame:0
         TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:0
         RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

But we want to only focus on the RX packets. Let's first eliminate the loopback interface. You could easily run ifconfig eth1, but it is also possible to grep for the data.

The following will focus on the "eth1" line and also display 5 lines after the eth1 match to get the RX packets line:

ifconfig | grep eth1 -A5

eth1      Link encap:Ethernet  HWaddr 08:00:27:B0:6A:94
         inet addr:  Bcast:  Mask:
         inet6 addr: fe80::a00:27ff:feb0:6a94/64 Scope:Link
         RX packets:1556 errors:0 dropped:0 overruns:0 frame:0
         TX packets:1502 errors:0 dropped:0 overruns:0 carrier:0

Next, let’s keep trying to focus on the RX packets:
ifconfig | grep eth1 -A5 | grep RX  
[root@localhost tmp]# ifconfig | grep eth1 -A5 | grep RX
         RX packets:1572 errors:0 dropped:0 overruns:0 frame:0
Now it is down to parsing this line for the number of packets, say 1572...

The awk command can be used to split a line into columns where spaces separate the fields.
Therefore, if each space in the line above represents a column we can focus on column two:

ifconfig | grep eth1 -A5 | grep RX | awk ' {  print $2 } '

Great, now how can we strip out the “packets:” from the raw number we are looking for?
We already know that awk is good for selecting columns from text data. As the string is now, it can also be interpreted in column format by switching the delimiter from a space to colon. Therefore, selecting column 2 using a ":" delimiter with awk will finally give us the number
[root@localhost tmp]# ifconfig | grep eth1 -A5 | grep RX | awk ' {  print $2 } ' | awk -F ":" ' { print $2 } '

That's it!

What can be done with the number of packets? If you take a sample before after, say for 60 seconds one can derive a rough estimate of packets per second on the interface.

START=`ifconfig | grep eth1 -A5 | grep RX | awk ' {  print $2 } ' | awk -F ":" ' { print $2 } '`;
sleep $LENGTH
END=`ifconfig | grep eth1 -A5 | grep RX | awk ' {  print $2 } ' | awk -F ":" ' { print $2 } '`;
echo $START $END $LENGTH | awk ' { print ($2-$1)/$3 } '

Again the awk command in the last line is used to parse columns but additionally it can be used for doing simple arithmetic with the columns. For instance, packets/sec = ($END-$START)/$LENGTH

The thing to remember about this example is that this is not difficult. The solution only uses simple features of awk and grep.

This exercise can also be done without using ifconfig. As an exercise on your own try using
only the file /proc/net/dev and grep/awk commands. The same metric, RX packets, can be found here:
Inter-|   Receive                                                |  Transmit
face |bytes    packets errs drop fifo frame compressed multicast|bytes    packets errs drop fifo colls carrier compressed
 eth1:  359967    4110    0    0    0     0          0         1   786631    3985    0    0    0     0       0          0
 eth2:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
   lo:       0    

1 comment: