Linux Tips - Beyond the Basics (Part 1)

As a lot of recent posts have focused on Windows tips, a coworker of mine suggested writing up some Unix tips. Many day to day operations in Unix can be performed using the same old techniques and a lot of Unix users become stagnant and do not progress to help improve Unix skills beyond the basics. Here are everyday tips I still find useful from my days as a Unix admin.

There is a lot that can be covered, at this point it looks like this will be a multi-part series of articles. Feel free to comment with any suggested topics.

Simple Bash Navigation
There is no excuse for fumbling around on the command line. This is your primary interface to Unix and Hyperion. However, this all breaks down when your boss is looking over your shoulder and you start fumbling around like a novice.

(emacs key bindings)
CTRL +a beginning of line,
CTRL+ e end of line
CTRL + k - cut from current position to end of the line
CTRL + y - yank back the cut contents at the current cursor position
Learn other cool tips like swapping the position of command line arguments:
http://ss64.com/bash/syntax-keyboard.html

Ctrl+r at the Bash Shell
This is the recall command; the most useful feature in the bash shell. Maneuvering in the Unix shell often requires complex commands and long paths. Most users are familiar with the history command and the up and down arrows to access recent history. However, this is inefficient. At the command prompt press CTRL+r and type any unique part of the string containing the command that you want to recall. This powerful feature lets you pull up anything from the history by typing in a few unique characters. You may have used this before. Web browsers have copied this feature in the URL prompt. For instance, typing /em in Chrome can pull back "https://server:7002/em". After some time using this feature your brain starts to rewire and it becomes a snap to use "recall" mode to rapidly pull up commands to run or edit.

If there is more than one match, pressing CTRL+r again will go to a prior match the command history. CTRL + p and CTRL + r also cycle back and forth between matches.

Advanced users will find that the more commands you type in with full paths, the more the recall feature helps. For instance, if you run a command by changing into 6 directories with individual cd commands and then run the command with a relative path, the recall feature won't help piece this together. It is encouraged to enter the full paths to the command so it is easily run from any location at a different time.

Become Very Familiar with your Editor of Choice
The text editor is the most powerful tool on the Linux box. Vi or EMACS or Other? Either way - learn more than the basics, these are powerful editors can can save tons of time.

Emacs:
https://www.gnu.org/software/emacs/refcards/pdf/refcard.pdf

Vi:
http://www.albany.edu/faculty/hy973732/ist535/vi_editor_commands.pdf

Screen
The Unix command prompt is a fragile thing when running over a network connection via SSH. Ever run a command, have it take longer than expected and now it's time to go home? Logging out will surely kill the long running process. This is very useful when running database backups that will take a while, yet the default timeout for your bash shell is only 20 minutes. This prevents the backup terminating when your shell is killed.


Screen basics:
Before running long running command type "screen" to enter screen session.

screen -S ScreenTest

sleep 1000000

Now detach your session

CTRL+a d

[detached from 11504.ScreenTest]

Log out and go home for the day.

When you get home, check what's running:

:~$ screen -ls

There are screens on:

        11504.ScreenTest        (04/08/2015 08:44:26 PM)        (Detached)

1 Socket in /var/run/screen

reattach to the running screen session:

screen -r ScreenTest

sleep 1000000 (sleep command is still sitting there executing)

Killing
kill all processes matching the word "java"

killall java
OR
ps -wwef | grep java | awk ' { print $2 } ' | xargs kill -9

Swiftly kill everything. If you have a service account such as "hyperion" and you need to quickly make sure all processes are down, assuming you have tried all sane ways to do this, you can kill everything running under the current user id by:
kill -9 -1
Tip: Don't do this as root.

Network Testing
The old "telnet" command to test if a service is running on a port is being deprecated. The nc command, "netcat" command can be used for basic connectivity.

:~$ nc -v localhost 80
nc: connect to localhost port 80 (tcp) failed: Connection refused

Also nc can be used to startup a mock network server on a port for testing connectivity between two machines. This allows you to test firewall rules long before you have setup the environment and realize the connectivity is not working.

On server1:

nc -l 1234    (start a server on port 1234)

from another machine, try to connect:

nc -v server1 1234

typing text on the client side should appear on the server side.

Copying Files Between Servers

rsync -a ~/tmp/dir1/ oracle@epmvirt:/tmp/destination

Bash For Loops
for i in a b c; do  echo $i ; done

for server in server1 server2 server3 server4 ; do scp file $server:/tmp/; done

Disk Space Consumption

du -m /u0 | sort -n

36643   /u0/app/oracle/product

68393   /u0/app/oracle/backups

320977  /u0/app/oracle/arch

576131  /u0/app/oracle

576230  /u0/app

605054  /u0/

This shows where to find the largest disk space consumers to quickly mitigate disk space issues. In this case we could check the archive logs directory and reduce disk space.

Learning a Programming Language
Often users get sucked into writing complex shell scripts. They start off innocent enough, but quickly shell scripting will show its weaknesses. Shell scripts become unwieldy with syntax and flow control - loops all over the place, difficult command line argument handling, poor error handling... etc. I encourage having a more elegant language to write scripts in. Any will do, but python can be a quick way to get started and is proven in the real world. Even if you completely new, spending a few hours on a tutorial site and then tackling a simple program in python will pay off in the long run over using a bash script.
A tutorial can be found here:
https://docs.python.org/2/tutorial/