1.8. Background Jobs

Pipes are actually a special case of a more general feature: doing more than one thing at a time. This is a capability that many other commercial operating systems don't have, because of the rigid limits that they tend to impose upon users. UNIX, on the other hand, was developed in a research lab and meant for internal use, so it does relatively little to impose limits on the resources available to users on a computer—as usual, leaning towards uncluttered simplicity rather than overcomplexity.

"Doing more than one thing at a time" means running more than one program at the same time. You do this when you invoke a pipeline; you can also do it by logging on to a UNIX system as many times simultaneously as you wish. (If you try that on an IBM's VM/CMS system, for example, you will get an obnoxious "already logged in" message.)

The shell also lets you run more than one command at a time during a single login session. Normally, when you type a command and hit RETURN, the shell will let the command have control of your terminal until it is done; you can't type in further commands until the first one is done. But if you want to run a command that does not require user input and you want to do other things while the command is running, put an ampersand (&) after the command.

This is called running the command in the background, and a command that runs in this way is called a background job; by contrast, a job run the normal way is called a foreground job. When you start a background job, you get your shell prompt back immediately, enabling you to enter other commands.

The most obvious use for background jobs is programs that take a long time to run, such as sort or uncompress on large files. For example, assume you just got an enormous compressed file loaded into your directory from magnetic tape.^[14] Let's say the file is gcc.tar.Z, which is a compressed archive file that contains well over 10 MB of source code files.

^[14] Compressed files are created by the compress utility, which packs files into smaller amounts of space; they have names of the form filename.Z, where filename is the name of the original uncompressed file.

Type uncompress gcc.tar & (you can omit the .Z), and the system will start a job in the background that uncompresses the data "in place" and ends up with the file gcc.tar. Right after you type the command, you will see a line like this:

[1] 175

followed by your shell prompt, meaning that you can enter other commands. Those numbers give you ways of referring to your background job; Chapter 8 explains them in detail.

You can check on background jobs with the command jobs. For each background job, jobs prints a line similar to the above but with an indication of the job's status:

[1]+ Running uncompress gcc.tar &

When the job finishes, you will see a message like this right before your shell prompt:

[1]+ Done uncompress gcc.tar

The message changes if your background job terminated with an error; again, see Chapter 8 for details.

1.8.1. Background I/O

Jobs you put in the background should not do I/O to your terminal. Just think about it for a moment and you'll understand why.

By definition, a background job doesn't have control over your terminal. Among other things, this means that only the foreground process (or, if none, the shell itself) is "listening" for input from your keyboard. If a background job needs keyboard input, it will often just sit there doing nothing until you do something about it (as described in Chapter 8).

If a background job produces screen output, the output will just appear on your screen. If you are running a job in the foreground that produces output too, then the output from the two jobs will be randomly (and often annoyingly) interspersed.

If you want to run a job in the background that expects standard input or produces standard output, you usually want to redirect the I/O so that it comes from or goes to a file. Programs that produce small, one-line messages (warnings, "done" messages, etc.) are an exception to this general rule; you may not mind if these are interspersed with whatever other output you are seeing at a given time.

For example, the diff utility examines two files, whose names are given as arguments, and prints a summary of their differences on the standard output. If the files are exactly the same, diff is silent. Usually, you invoke diff expecting to see a few lines that are different.

diff, like sort and compress, can take a long time to run if the input files are very large. Suppose that you have two large files that are called warandpeace.txt and warandpeace.txt.old. The command diff warandpeace.txt warandpeace.txt.old^[15] reveals that the author decided to change the name "Ivan" to "Aleksandr" throughout the entire file—i.e., hundreds of differences, resulting in very large amounts of output.

^[15] You could use diff warandpeace* as a shorthand to save typing—as long as there are no other files with names of that form. Remember that diff doesn't see the arguments until after the shell has expanded the wildcards. Many people overlook this use of wildcards.

If you type diff warandpeace.txt warandpeace.txt.old &, then the system will spew lots and lots of output at you, which will be difficult to stop—even with the techniques explained in Chapter 7. However, if you type:

$ diff warandpeace.txt warandpeace.txt.old > txtdiff &

then the differences will be saved in the file txtdiff for you to examine later.

1.8.2. Background Jobs and Priorities

Background jobs can save you a lot of thumb-twiddling time. Just remember that such jobs eat up lots of system resources like memory and the processor (CPU). Just because you're running several jobs at once doesn't mean that they will run faster than they would if run sequentially—in fact, performance is usually slightly worse.

Every job on the system is assigned a priority, a number that tells the operating system how much priority to give the job when it doles out resources (the higher the number, the lower the priority). Commands that you enter from the shell, whether foreground or background jobs, usually have the same priority. The system administrator is able to run commands at a higher priority than normal users.

Note that if you're on a multiuser system, running lots of background jobs may eat up more than your fair share of resources, and you should consider whether having your job run as fast as possible is really more important than being a good citizen.

Speaking of good citizenship, there is also a UNIX command that lets you lower the priority of any job: the aptly named nice. If you type nice command, where command can be a complex shell command line with pipes, redirectors, etc., then the command will run at a lower priority.^[16] You can control just how much lower by giving nice a numerical argument; consult the nice manpage for details.^[17]

^[16] Complex commands following nice should be quoted.

^[17] If you are a system administrator logged in as root, then you can also use nice to raise a job's priority.