Chapter 20. I/O Redirection
There are always three default files [1] open, stdin (the keyboard), stdout (the screen), and stderr (error messages output to the screen). These, and any other open files, can be redirected. Redirection simply means capturing output from a file, command, program, script, or even code block within a script (see Example 3-1 and Example 3-2) and sending it as input to another file, command, program, or script.
Each open file gets assigned a file descriptor. [2] The file descriptors for stdin, stdout, and stderr are 0, 1, and 2, respectively. For opening additional files, there remain descriptors 3 to 9. It is sometimes useful to assign one of these additional file descriptors to stdin, stdout, or stderr as a temporary duplicate link. [3] This simplifies restoration to normal after complex redirection and reshuffling (see Example 20-1).
1 COMMAND_OUTPUT > 2 # Redirect stdout to a file. 3 # Creates the file if not present, otherwise overwrites it. 4 5 ls -lR > dir-tree.list 6 # Creates a file containing a listing of the directory tree. 7 8 : > filename 9 # The > truncates file "filename" to zero length. 10 # If file not present, creates zero-length file (same effect as 'touch'). 11 # The : serves as a dummy placeholder, producing no output. 12 13 > filename 14 # The > truncates file "filename" to zero length. 15 # If file not present, creates zero-length file (same effect as 'touch'). 16 # (Same result as ": >", above, but this does not work with some shells.) 17 18 COMMAND_OUTPUT >> 19 # Redirect stdout to a file. 20 # Creates the file if not present, otherwise appends to it. 21 22 23 # Single-line redirection commands (affect only the line they are on): 24 # -------------------------------------------------------------------- 25 26 1>filename 27 # Redirect stdout to file "filename." 28 1>>filename 29 # Redirect and append stdout to file "filename." 30 2>filename 31 # Redirect stderr to file "filename." 32 2>>filename 33 # Redirect and append stderr to file "filename." 34 &>filename 35 # Redirect both stdout and stderr to file "filename." 36 # This operator is now functional, as of Bash 4, final release. 37 38 M>N 39 # "M" is a file descriptor, which defaults to 1, if not explicitly set. 40 # "N" is a filename. 41 # File descriptor "M" is redirect to file "N." 42 M>&N 43 # "M" is a file descriptor, which defaults to 1, if not set. 44 # "N" is another file descriptor. 45 46 #============================================================================== 47 48 # Redirecting stdout, one line at a time. 49 LOGFILE=script.log 50 51 echo "This statement is sent to the log file, \"$LOGFILE\"." 1>$LOGFILE 52 echo "This statement is appended to \"$LOGFILE\"." 1>>$LOGFILE 53 echo "This statement is also appended to \"$LOGFILE\"." 1>>$LOGFILE 54 echo "This statement is echoed to stdout, and will not appear in \"$LOGFILE\"." 55 # These redirection commands automatically "reset" after each line. 56 57 58 59 # Redirecting stderr, one line at a time. 60 ERRORFILE=script.errors 61 62 bad_command1 2>$ERRORFILE # Error message sent to $ERRORFILE. 63 bad_command2 2>>$ERRORFILE # Error message appended to $ERRORFILE. 64 bad_command3 # Error message echoed to stderr, 65 #+ and does not appear in $ERRORFILE. 66 # These redirection commands also automatically "reset" after each line. 67 #======================================================================= |
1 2>&1
2 # Redirects stderr to stdout.
3 # Error messages get sent to same place as standard output.
4 >>filename 2>&1
5 bad_command >>filename 2>&1
6 # Appends both stdout and stderr to the file "filename" ...
7 2>&1 | [command(s)]
8 bad_command 2>&1 | awk '{print $5}' # found
9 # Sends stderr through a pipe.
10 # |& was added to Bash 4 as an abbreviation for 2>&1 |.
11
12 i>&j
13 # Redirects file descriptor i to j.
14 # All output of file pointed to by i gets sent to file pointed to by j.
15
16 >&j
17 # Redirects, by default, file descriptor 1 (stdout) to j.
18 # All stdout gets sent to file pointed to by j. |
1 0< FILENAME 2 < FILENAME 3 # Accept input from a file. 4 # Companion command to ">", and often used in combination with it. 5 # 6 # grep search-word <filename 7 8 9 [j]<>filename 10 # Open file "filename" for reading and writing, 11 #+ and assign file descriptor "j" to it. 12 # If "filename" does not exist, create it. 13 # If file descriptor "j" is not specified, default to fd 0, stdin. 14 # 15 # An application of this is writing at a specified place in a file. 16 echo 1234567890 > File # Write string to "File". 17 exec 3<> File # Open "File" and assign fd 3 to it. 18 read -n 4 <&3 # Read only 4 characters. 19 echo -n . >&3 # Write a decimal point there. 20 exec 3>&- # Close fd 3. 21 cat File # ==> 1234.67890 22 # Random access, by golly. 23 24 25 26 | 27 # Pipe. 28 # General purpose process and command chaining tool. 29 # Similar to ">", but more general in effect. 30 # Useful for chaining commands, scripts, files, and programs together. 31 cat *.txt | sort | uniq > result-file 32 # Sorts the output of all the .txt files and deletes duplicate lines, 33 # finally saves results to "result-file". |
Multiple instances of input and output redirection and/or pipes can be combined in a single command line.
1 command < input-file > output-file 2 # Or the equivalent: 3 < input-file command > output-file # Although this is non-standard. 4 5 command1 | command2 | command3 > output-file |
Multiple output streams may be redirected to one file.
1 ls -yz >> command.log 2>&1 2 # Capture result of illegal options "yz" in file "command.log." 3 # Because stderr is redirected to the file, 4 #+ any error messages will also be there. 5 6 # Note, however, that the following does *not* give the same result. 7 ls -yz 2>&1 >> command.log 8 # Outputs an error message, but does not write to file. 9 # More precisely, the command output (in this case, null) 10 #+ writes to the file, but the error message goes only to stdout. 11 12 # If redirecting both stdout and stderr, 13 #+ the order of the commands makes a difference. |
- n<&-
Close input file descriptor n.
- 0<&-, <&-
Close stdin.
- n>&-
Close output file descriptor n.
- 1>&-, >&-
Close stdout.
Child processes inherit open file descriptors. This is why pipes work. To prevent an fd from being inherited, close it.
1 # Redirecting only stderr to a pipe. 2 3 exec 3>&1 # Save current "value" of stdout. 4 ls -l 2>&1 >&3 3>&- | grep bad 3>&- # Close fd 3 for 'grep' (but not 'ls'). 5 # ^^^^ ^^^^ 6 exec 3>&- # Now close it for the remainder of the script. 7 8 # Thanks, S.C. |
For a more detailed introduction to I/O redirection see Appendix F.
20.1. Using exec
An exec <filename command redirects stdin to a file. From that point on, all stdin comes from that file, rather than its normal source (usually keyboard input). This provides a method of reading a file line by line and possibly parsing each line of input using sed and/or awk.
Example 20-1. Redirecting stdin using exec
1 #!/bin/bash 2 # Redirecting stdin using 'exec'. 3 4 5 exec 6<&0 # Link file descriptor #6 with stdin. 6 # Saves stdin. 7 8 exec < data-file # stdin replaced by file "data-file" 9 10 read a1 # Reads first line of file "data-file". 11 read a2 # Reads second line of file "data-file." 12 13 echo 14 echo "Following lines read from file." 15 echo "-------------------------------" 16 echo $a1 17 echo $a2 18 19 echo; echo; echo 20 21 exec 0<&6 6<&- 22 # Now restore stdin from fd #6, where it had been saved, 23 #+ and close fd #6 ( 6<&- ) to free it for other processes to use. 24 # 25 # <&6 6<&- also works. 26 27 echo -n "Enter data " 28 read b1 # Now "read" functions as expected, reading from normal stdin. 29 echo "Input read from stdin." 30 echo "----------------------" 31 echo "b1 = $b1" 32 33 echo 34 35 exit 0 |
Similarly, an exec >filename command redirects stdout to a designated file. This sends all command output that would normally go to stdout to that file.
 | exec N > filename affects the entire script or current shell. Redirection in the PID of the script or shell from that point on has changed. However . . . N > filename affects only the newly-forked process, not the entire script or shell. Thank you, Ahmed Darwish, for pointing this out. |
Example 20-2. Redirecting stdout using exec
1 #!/bin/bash 2 # reassign-stdout.sh 3 4 LOGFILE=logfile.txt 5 6 exec 6>&1 # Link file descriptor #6 with stdout. 7 # Saves stdout. 8 9 exec > $LOGFILE # stdout replaced with file "logfile.txt". 10 11 # ----------------------------------------------------------- # 12 # All output from commands in this block sent to file $LOGFILE. 13 14 echo -n "Logfile: " 15 date 16 echo "-------------------------------------" 17 echo 18 19 echo "Output of \"ls -al\" command" 20 echo 21 ls -al 22 echo; echo 23 echo "Output of \"df\" command" 24 echo 25 df 26 27 # ----------------------------------------------------------- # 28 29 exec 1>&6 6>&- # Restore stdout and close file descriptor #6. 30 31 echo 32 echo "== stdout now restored to default == " 33 echo 34 ls -al 35 echo 36 37 exit 0 |
Example 20-3. Redirecting both stdin and stdout in the same script with exec
1 #!/bin/bash 2 # upperconv.sh 3 # Converts a specified input file to uppercase. 4 5 E_FILE_ACCESS=70 6 E_WRONG_ARGS=71 7 8 if [ ! -r "$1" ] # Is specified input file readable? 9 then 10 echo "Can't read from input file!" 11 echo "Usage: $0 input-file output-file" 12 exit $E_FILE_ACCESS 13 fi # Will exit with same error 14 #+ even if input file ($1) not specified (why?). 15 16 if [ -z "$2" ] 17 then 18 echo "Need to specify output file." 19 echo "Usage: $0 input-file output-file" 20 exit $E_WRONG_ARGS 21 fi 22 23 24 exec 4<&0 25 exec < $1 # Will read from input file. 26 27 exec 7>&1 28 exec > $2 # Will write to output file. 29 # Assumes output file writable (add check?). 30 31 # ----------------------------------------------- 32 cat - | tr a-z A-Z # Uppercase conversion. 33 # ^^^^^ # Reads from stdin. 34 # ^^^^^^^^^^ # Writes to stdout. 35 # However, both stdin and stdout were redirected. 36 # Note that the 'cat' can be omitted. 37 # ----------------------------------------------- 38 39 exec 1>&7 7>&- # Restore stout. 40 exec 0<&4 4<&- # Restore stdin. 41 42 # After restoration, the following line prints to stdout as expected. 43 echo "File \"$1\" written to \"$2\" as uppercase conversion." 44 45 exit 0 |
I/O redirection is a clever way of avoiding the dreaded inaccessible variables within a subshell problem.
Example 20-4. Avoiding a subshell
1 #!/bin/bash
2 # avoid-subshell.sh
3 # Suggested by Matthew Walker.
4
5 Lines=0
6
7 echo
8
9 cat myfile.txt | while read line;
10 do {
11 echo $line
12 (( Lines++ )); # Incremented values of this variable
13 #+ inaccessible outside loop.
14 # Subshell problem.
15 }
16 done
17
18 echo "Number of lines read = $Lines" # 0
19 # Wrong!
20
21 echo "------------------------"
22
23
24 exec 3<> myfile.txt
25 while read line <&3
26 do {
27 echo "$line"
28 (( Lines++ )); # Incremented values of this variable
29 #+ accessible outside loop.
30 # No subshell, no problem.
31 }
32 done
33 exec 3>&-
34
35 echo "Number of lines read = $Lines" # 8
36
37 echo
38
39 exit 0
40
41 # Lines below not seen by script.
42
43 $ cat myfile.txt
44
45 Line 1.
46 Line 2.
47 Line 3.
48 Line 4.
49 Line 5.
50 Line 6.
51 Line 7.
52 Line 8. |