Input Output Redirection in Linux

When we open a file or start a program, Linux assigns three communication channels to it. These channels are STDIN, STDOUT, and STDERR. These channels allow it to interact with other files, programs, processes, or devices on the Linux system.

STDIN allows the opened file or process to receive data from other files, processes, and devices. STDOUT and STDERR enable it to send the processed data and errors to other files, processes, and devices. The keyboard is the default STDIN device. The monitor is the default STDOUT and STDERR device. STDIN reads from the keyboard. STDOUT and STDERR write their output to the monitor screen.

You can check the following tutorial to learn more about STDIN, STDOUT, and STDERR.

STDIN, STDOUT, and STDERR in Linux

I/O (Input / Output) Redirection

Instead of using default devices, if a process or a file wants to use other files or processes to read the input data or send the processed output data, it can use I/O redirection. I/O redirection is a shell feature that allows an opened file or process to override default devices when using communication channels. The shell assigns three symbols <, > and >> to I/O redirection. The < symbol overrides the STDIN. The > and >> symbols override the STDOUT and STDERR. The > symbol sends the output in the overwrite mode. The >> symbol sends the output in the append mode.

I/O redirection examples

Access the shell prompt and run the cat command without specifying a file name.

$cat

The cat command reads data from the specified file and displays it on the default STDOUT device. In other words, it reads the specified file and shows its contents on the monitor (default STDOUT device). If an input file is not specified, it reads data from the default STDIN device and displays it on the default STDOUT device. Since we did not specify a filename, it reads input data from the keyboard (default STDIN device) and displays it on the monitor screen (default STDOUT device).

To verify this, type anything and press the Enter key at the end of each line. It reads whatever you typed in the line and displays that as it is on the monitor. To terminate the interactive mode and return to the shell prompt, press the CTRL+D key.

The following image shows the above exercise with the output.

If we do not specify a filename, it reads input data from the keyboard and displays it on the monitor screen. What if, instead of showing the output on the screen, we want to save it in a file? In that case, we need to redirect the STDOUT to the file. For example, the following command sends the output to the file named myfile.

$cat > myfile

We can also override the default input. If we specify a file name as the argument, it uses the specified file as the input. It displays the file contents on the terminal.

$cat myfile

Since we specified the filename, it reads the input data from the specified file and displays that on the STDOUT. The following image shows the output of this command.

Append V/s Overwrite modes

As mentioned earlier, the shell can redirect the output in two modes: append and overwrite. For the append mode, it uses the >> symbol. For the overwrite mode, it uses the > symbol.

To append the existing contents, use the following command.

$cat >> [file-name]

To overwrite the existing contents, use the following command.

$cat > [file-name]

Redirecting errors

A program, process, script or command may return an error. When an error occurs, the system sends that error to STDERR. Since, by default, STDOUT and STDERR use the same device (monitor screen), errors are also displayed on the monitor screen. If you want to redirect errors to another device or save them in a file for logging and debugging purposes, use the redirection symbol. The shell uses the symbols > and >> to redirect the STDOUT. To use these symbols to redirect STDERR, we have to use file descriptors number of STDERR with these symbols.

Linux uses a unified I/O model. This model assigns an integer value to each channel called a file descriptor. File descriptor numbers for STDIN, STDOUT, and STDERR are 0, 1, and 2. We can use these numbers in our commands and scripts to refer to the STDIN, STDOUT, and STDERR. To instruct the shell to redirect STDERR instead of STDOUT, specify the output redirection symbol with the number 2 (file descriptor number of STDERR). The following exercise explains it.

$ls
$cat [an-existing-file]
$cat [an-existing-file] 2> errorlog
$cat errorlog
$cat [non-existing file]
$cat [non-existing file] 2> errorlog
$cat errorlog

In this exercise, we first displayed the contents of an existing file using the cat command. Later, we rerun the same command after adding the '2> errorlog' to the command. When executing this command, if any error occurs, the '2> error log' instructs the shell to redirect that error to the 'errorlog' file instead of displaying it on the monitor screen. Since the input file exists, no error occurred. To verify this, we displayed the contents of the errorlog file.

After that, we used the cat command to display the contents of a non-existing file. Since the file does not exist, the shell displayed the 'No such file or directory' error. Later, we executed the same command after adding the '2>errorlog' to the command. This time, since we instructed the shell to redirect errors in the 'errorlog' file, instead of displaying them on the screen, the shell sent the error to the file 'errorlog'. To verify this, we displayed the contents of the errorlog file again. The errorlog file contains the error message.

When redirecting STDOUT, we use the >> symbol to append the existing file. We use the same >> symbol to append the existing file when redirecting the STDERR. If we use the > symbol to redirect STDERR, the shell overwrites the specified file's contents.

The following exercise explains it.

$cat nofile 2> errorlog
$cat errorlog
$cat nonexistingfile 2> errorlog
$cat errorlog
$cat unknownfile 2>> errorlog
$cat errolog

In this exercise, we used the cat command to display three files that do not exist. Since files do not exist, we got an error each time. We redirected errors to the errorlog file. To redirect errors, we used the overwrite mode the first and second times, while in the third time, we used the append mode.

The following image shows this exercise with the output.

When we used the overwrite mode, the contents of the existing file were overwritten, but when we used the append mode, the contents of the existing file were appended.

Redirecting two or all three data streams together

If you want to redirect two or all three data streams together, you can do this by following the same method you use to redirect one stream. For example, the following command uses all three redirections together.

$cat < testfile > outputfile 2> errorlog

This command reads input data from the testfile and saves that data in the outputfile. In this process, if any error occurs, it writes that error in the errorlog file.

Sending errors to /dev/null

The /dev/null device discards all input data streams. This device is commonly used to discard errors and unwanted messages when executing a process, script, or command. For example, when you use the find command to search something globally, many "permission denied" errors might occur. Your genuine results may be lost in the clutter. You can redirect STDERR to the /dev/null to discard all errors.

Let us take an example.

Run the following commands and compare their outputs.

$find / -name core
$find / -name core 2> /dev/null

As you can see in the above output, we got many permission-related errors when we used the find command without redirecting STDERR. But when we used the find command by redirecting STDERR to /dev/null, we got only results. All errors were redirected to /dev/null, and all results were displayed on the screen.

I/O redirection summary

The following table summarizes I/O redirection symbols along with their meanings.

Conclusion

I/O redirection is a powerful feature that enhances interaction with files and processes. It allows users to read inputs from various sources and direct outputs to specific files instead of the default devices, enabling better data management and organization. Additionally, redirecting errors provides a means to capture and log errors separately, facilitating debugging and maintenance of scripts and commands.