This isn’t your mother’s “10 Things you didn’t know Ubuntu could do” kind of list; these are low-level system commands that will prove to be invaluable as you begin to administer your system.  What follows are commands and their examples, as well as suggestions on when they should be used.

As I use Linux and BSD on a day-to-day basis, there are some commands that are simply a godsend once you learn them.  Some are common but little is known about how they work; others are usually unheard of until the day someone demonstrates its use to you, and you wonder how you’ve lived so long without using that little gold nugget of knowledge.

This is a great tool for several reasons.  First off, most people use it as a way of sending information; that is, you can use it to literally cat information across the network, like so:

cat data.txt | nc host.example.com

This will take the contents of http_data.txt and send them over the network to the specified host.  But did you know that nc can also receive data as well?  You can launch a listen-only server listening on port 8080 on localhost using the following:

# Output any incoming requests on port 8080!
nc -l 8080

If I were to telnet to port 8080, I would be able to send information over the network.  This is especially useful if you have a program sending data, but need a quick and easy way of picking up the data.  On top of all that, you can have the netcat “server” send responses from stdin–that is, if your application sends a HELO, you can respond right back.  Pretty nifty, eh?

I remember when I was younger, every time I wanted to repeatedly check something (such as, while waiting for someone to upload a file), I’d either setup a loop or repeatedly run the command.  Little did I know, there was a command that already repeated a command for you; “watch” and learn.  Say I wanted to watch a directory for new files from an interactive terminal; perhaps I’m waiting for a client to upload something, or maybe I’m waiting for a process to finished.  I could periodically type ls to see if the file is there; or I could use watch.

# Repeatedly run the ls command on /tmp
watch ls /tmp

This will, by default, run the command every two seconds.  Using -n, I can configure this time however I want.  But what if I’m watching something like memory usage, and I want to be able to look and see what’s changed between command runs?  Well, watch has that too!

# Highlight differences between runs
watch -d ls /tmp
 
# Ditto, only this time, keep the changes across ALL runs;
# So if I step away for an hour, it will show ALL new files
watch -d  --cumulative ls /tmp

Isn’t that just the best little command you never knew about?  The man page gives even more examples.

Every Linux users follows the same process.  First, they discover back ticks and use it to pipe the output of one command to another:

# If you're on 2.6.23, better hope there are less than 32k results
rm -f `find /tmp -type f`

Then, you learn about xargs, and realize the folly of your ways:

# Uhoh, better hope there aren't any spaces, quotes,
#   or other escape characters!

find /tmp -type f | xargs rm -f

Soon, you learn to use null bytes to filter results to avoid those nasty line breaks:

# Good luck trying to debug this without altering the command
find /tmp -type f -print0 | xargs -0 rm -f

And what if you try to read the output?  What followed was the result of many years of frustration with the default state of the whitespace delimiter from the command line: parallel.  As a GNU utility, its available on pretty much any *nix platform worth its salt.  It operates identically to xargs, only it uses a line-by-line algorithm to pass arguments.  That is, spaces, escape characters, etc., no longer break up your output.

find /tmp -type f | parallel rm -f

Got a quote in your filename?  No problem!  Got spaces?  Its been handled.  parallel is pretty much what most people expect when using xargs.  And, because its part of the GNU tool set (and distributed under the GPL), you have no reason to not start using it right this very moment.

Like parallel, the step-by-step learning process of killing misbehaving processes usually goes like this:

ps aux | grep java
kill

Then, you learn about killall, which means you can kill any and all processes based on their process name:

# Kill all `java' processes
killall java

But lets say you’re running several instances of Java, but you only want to kill the ones associated with tomcat6?  killall won’t work, as you’re no longer killing based on the process name.  You can go back to the first method, but this is a lot of time and overheard for something that SHOULD be automated–and easy, considering how common of a task it is.  This is where pkill comes into play; instead of a process name or a pid, pkill takes a pattern–it can be a simple line of text, or an extended regular expression–and kills any processes matching it.  That is, I can now say:

# Match anything where tomcat is in the command name;
# `java-tomcat', but NOT `java -jar tomcat6.jar'
pkill tomcat
 
# Match anything where tomcat is in the FULL command line
# `java-tomcat' <em>and</em> `java -jar tomcat6.jar'
pkill -f tomcat

pgrep serves much the same purpose, only it will return matching pids instead of sending the processes signals.

A lot of people have probably seen this command at least once in their life, but may not have given it any thought.  I know that in the times that I have had a need for it, however, I would have had no alternatives without it.  tee is a tool that allows you to copy stdout to a file or files, while not disrupting stdout itself.  That is, I can now log output while display it to the screen.  For example, lets say I want to run a find command that not only stores the logs, but allows me to look at it.  I could pipe all output to the file and then view the file, but this runs into issues where its a continuous process that may not finish immediately–and I may not want to worry about backgrounding the process. For example, lets say I have a job to restart some servers; I want to be able to check the process as its running (and maybe even give it input as it runs), but I also want the entire session to be logged.

./run_some_long_task.sh | tee output.log

Now, anytime there’s output, it will be recorded to stdout AND output.log.  Amazing, right?  I find this especially useful in Hudson jobs where I want the output to be recorded in the build log, but I also want to be able to parse the output after the command has run.  It also lends itself to cron jobs, where the output can be simultaneously mailed to a user and written to disk.

There you have it, a list of five commands you’ve probably never heard of, or never took the time to investigate!  I know I would have been at a major loss for some of these commands, and I feel that these are generally not in most users’ arsenals of commands.  If you’re a hardcore console user, there’s absolutely no reason you shouldn’t always be using these.  But for those of you who are not Linux gurus, I can promise you one thing–one day, in some random situation, you WILL be thankful for having learned these commands!