Command Line Essentials: Text and Pipeline

Table of Content, Navigating the filesystem

What I will demonstrate now is extremely powerful. This is just examples that do nothing valuable to you, but when you need to get things done on a computer, the same technique can be very productive. You need to combine your creativity and experience to be truly successful.

An example:

essentials@kvaser:~$ ls -l /  |  cat -n
     1	total 109
     2	drwxr-xr-x  2 root root  4096 Feb 18 06:51 bin
     3	drwxr-xr-x  3 root root  1024 Feb 18 07:03 boot
     4	drwxr-xr-x 13 root root  2880 Feb 18 07:06 dev
     5	drwxr-xr-x 78 root root  4096 Mar  1 20:36 etc
     6	drwxr-xr-x  8 root root  4096 Feb 28 21:39 home
     7	drwxr-xr-x 10 root root  8192 Feb 18 06:51 lib
     8	drwxr-xr-x  2 root root 49152 Feb  2 21:01 lost+found
     9	drwxr-xr-x  3 root root  4096 Feb 17 21:39 media
    10	drwxr-xr-x  2 root root  4096 Jan 16 21:45 mnt
    11	drwxr-xr-x  2 root root  4096 Feb  2 21:27 opt
    12	dr-xr-xr-x 87 root root     0 Jan  1  1970 proc
    13	drwxr-xr-x  5 root root  4096 Mar  1 18:17 root
    14	drwxr-xr-x  2 root root  4096 Feb 18 06:52 sbin
    15	drwxr-xr-x  2 root root  4096 Sep 16  2008 selinux
    16	drwxr-xr-x  2 root root  4096 Feb  2 21:27 srv
    17	drwxr-xr-x 11 root root     0 Jan  1  1970 sys
    18	drwxrwxrwt  3 root root  4096 Mar  1 20:39 tmp
    19	drwxr-xr-x 10 root root  4096 Feb  2 21:27 usr
    20	drwxr-xr-x 14 root root  4096 Feb  2 22:38 var
essentials@kvaser:~$ ls -l /  |  cat -n  |  head -n 10  |  tail -n 5
     6	drwxr-xr-x  8 root root  4096 Feb 28 21:39 home
     7	drwxr-xr-x 10 root root  8192 Feb 18 06:51 lib
     8	drwxr-xr-x  2 root root 49152 Feb  2 21:01 lost+found
     9	drwxr-xr-x  3 root root  4096 Feb 17 21:39 media
    10	drwxr-xr-x  2 root root  4096 Jan 16 21:45 mnt

What happens? We first list the contents of the root directory, and add line numbers to the output. Second, we choose line 6-10 and just output those lines.

On the command line, you can interact with a program in different ways. The most common ways are displayed above:

  1. arguments: -l, -n 5, etc (gives program instructions about what to do)
  2. stdout: the output of a program (a list of directories in text format)
  3. stdin: input to a program (in the case of cat, head and tail, connected directly to the output of the program before)

The programs themselves may not seem so powerful. But combined they can surprise you. A fundamental design principle of UNIX is:
“each program should do just one thing, but do that one thing well”
So, you might not find a command that does exactly what you want. But combining a few commands you can easily do advanced things, that the designer of the programs never even thought of. Also, those standard programs are very old, very fast and very high quality. You can trust them to do the job very very well.

Now play a little with the ls-cat-head-tail-example above, and make sure you understand exactly how it works.

If you want to know more about a command you can do (q to exit)

essentials@kvaser:~$ man head
essentials@kvaser:~$ man tail
essentials@kvaser:~$ man cat

A word of warning: the man pages are very detailed, but not very easy to read. If you are lucky you find an example in the man pages (or try Google). Tricky thing is to be aware of what programs actually exist and understand what they do – then the man pages can help with details.

More examples: (space to scroll, q to exit)

essentials@kvaser:~$ find /  |  less

less makes it possible to look at large outputs page by page.

essentials@kvaser:~$ find /  |  grep txt  |  less
find: `/var/run/exim4': Permission denied
find: `/var/run/samba/winbindd_privileged': Permission denied

grep (by default) chooses lines in the input that contains the word you search for. So, this command lists all files with txt in the filename on your filesystem (that you have permission to). Note that errors are not written to stdout but to stderr, and stderr is not (by default) send to the next command. That is why some lines are not caught by less.

If you want to find only files with the extension .txt it gets a little trickier:

essentials@kvaser:~$ find /  |  grep "\.txt$"  |  less
find: `/var/run/exim4': Permission denied
find: `/var/run/samba/winbindd_privileged': Permission denied

The period character (.) is a special character to grep, so if you really want to match “.” you need to write a backslash before it. This is called escaping. The dollar character is also a special character. It matches “the end of the line”, which is what we do want, so we dont escape the dollar character.

Actually, “\.txt$” is a regular expression (often regexp). More about those ones later, but they are super powerful.

A few more commands:

essentials@kvaser:~$ cat /etc/group  |  sort  |  head -n 5
adm:x:4:
audio:x:29:kvaser
backup:x:34:
bind:x:106:
bin:x:2:
essentials@kvaser:~$ cat /etc/group  |  sort  |  cut -d ':' -f 1  |  head -n 5
adm
audio
backup
bind
bin
essentials@kvaser:~$ cat /etc/group  | wc -l
53

So, there is a file /etc/group (I think even in Cygwin). First we sort it and output the top 5 rows. Second we only output the first column (using cut). Third, we just count the lines.

So, now use the programs I have demonstrated above, and improvise. You can use files in the /etc directory as input data.

You can also play with output from

$ ps aux
$ w
$ /sbin/ifconfig  (or ifconfig, or ipconfig)
$ dig theregister.co.uk

Use: cat, head, tail, sort, cut, wc, grep.

  1. Command Line Essentials: Introduction | TechFindings - pingback on 2011/03/01 at 21:56

Leave a Comment


NOTE - You can use these HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Time limit is exhausted. Please reload CAPTCHA.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackbacks and Pingbacks: