for loops

for arg in [list]

This is the basic looping construct. It differs significantly from its C counterpart.

for arg in [list]
do
 command(s)...
done

During each pass through the loop, arg takes on the value of each successive variable in the list.

1 for arg in "$var1" "$var2" "$var3" ... "$varN" 2 # In pass 1 of the loop, arg = $var1 3 # In pass 2 of the loop, arg = $var2 4 # In pass 3 of the loop, arg = $var3 5 # ... 6 # In pass N of the loop, arg = $varN 7  8 # Arguments in [list] quoted to prevent possible word splitting.

The argument list may contain wild cards.

If do is on same line as for, there needs to be a semicolon after list.

for arg in [list] ; do

---

Example 11-1. Simple for loops

1 #!/bin/bash 2 # Listing the planets. 3  4 for planet in Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto 5 do 6  echo $planet # Each planet on a separate line. 7 done 8  9 echo; echo 10  11 for planet in "Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto" 12  # All planets on same line. 13  # Entire 'list' enclosed in quotes creates a single variable. 14  # Why? Whitespace incorporated into the variable. 15 do 16  echo $planet 17 done 18  19 echo; echo "Whoops! Pluto is no longer a planet!" 20  21 exit 0

---

Each [list] element may contain multiple parameters. This is useful when processing parameters in groups. In such cases, use the set command (see Example 15-16) to force parsing of each [list] element and assignment of each component to the positional parameters.

---

Example 11-2. for loop with two parameters in each [list] element

1 #!/bin/bash 2 # Planets revisited. 3  4 # Associate the name of each planet with its distance from the sun. 5  6 for planet in "Mercury 36" "Venus 67" "Earth 93" "Mars 142" "Jupiter 483" 7 do 8  set -- $planet # Parses variable "planet" 9  #+ and sets positional parameters. 10  # The "--" prevents nasty surprises if $planet is null or 11  #+ begins with a dash. 12  13  # May need to save original positional parameters, 14  #+ since they get overwritten. 15  # One way of doing this is to use an array, 16  # original_params=("$@") 17  18  echo "$1 $2,000,000 miles from the sun" 19  #-------two tabs---concatenate zeroes onto parameter $2 20 done 21  22 # (Thanks, S.C., for additional clarification.) 23  24 exit 0

---

A variable may supply the [list] in a for loop.

---

Example 11-3. Fileinfo: operating on a file list contained in a variable

1 #!/bin/bash 2 # fileinfo.sh 3  4 FILES="/usr/sbin/accept 5 /usr/sbin/pwck 6 /usr/sbin/chroot 7 /usr/bin/fakefile 8 /sbin/badblocks 9 /sbin/ypbind" # List of files you are curious about. 10  # Threw in a dummy file, /usr/bin/fakefile. 11  12 echo 13  14 for file in $FILES 15 do 16  17  if [ ! -e "$file" ] # Check if file exists. 18  then 19  echo "$file does not exist."; echo 20  continue # On to next. 21  fi 22  23  ls -l $file | awk '{ print $8 " file size: " $5 }' # Print 2 fields. 24  whatis `basename $file` # File info. 25  # Note that the whatis database needs to have been set up for this to work. 26  # To do this, as root run /usr/bin/makewhatis. 27  echo 28 done 29  30 exit 0

---

The [list] in a for loop may be parameterized.

---

Example 11-4. Operating on a parameterized file list

1 #!/bin/bash 2  3 filename="*txt" 4  5 for file in $filename 6 do 7  echo "Contents of $file" 8  echo "---" 9  cat "$file" 10  echo 11 done

---

If the [list] in a for loop contains wild cards (* and ?) used in filename expansion, then globbing takes place.

---

Example 11-5. Operating on files with a for loop

1 #!/bin/bash 2 # list-glob.sh: Generating [list] in a for-loop, using "globbing" ... 3 # Globbing = filename expansion. 4  5 echo 6  7 for file in * 8 # ^ Bash performs filename expansion 9 #+ on expressions that globbing recognizes. 10 do 11  ls -l "$file" # Lists all files in $PWD (current directory). 12  # Recall that the wild card character "*" matches every filename, 13  #+ however, in "globbing," it doesn't match dot-files. 14  15  # If the pattern matches no file, it is expanded to itself. 16  # To prevent this, set the nullglob option 17  #+ (shopt -s nullglob). 18  # Thanks, S.C. 19 done 20  21 echo; echo 22  23 for file in [jx]* 24 do 25  rm -f $file # Removes only files beginning with "j" or "x" in $PWD. 26  echo "Removed file \"$file\"". 27 done 28  29 echo 30  31 exit 0

---

Omitting the in [list] part of a for loop causes the loop to operate on $@ -- the positional parameters. A particularly clever illustration of this is Example A-15. See also Example 15-17.

---

Example 11-6. Missing in [list] in a for loop

1 #!/bin/bash 2  3 # Invoke this script both with and without arguments, 4 #+ and see what happens. 5  6 for a 7 do 8  echo -n "$a " 9 done 10  11 # The 'in list' missing, therefore the loop operates on '$@' 12 #+ (command-line argument list, including whitespace). 13  14 echo 15  16 exit 0

---

It is possible to use command substitution to generate the [list] in a for loop. See also Example 16-54, Example 11-11 and Example 16-48.

---

Example 11-7. Generating the [list] in a for loop with command substitution

1 #!/bin/bash 2 # for-loopcmd.sh: for-loop with [list] 3 #+ generated by command substitution. 4  5 NUMBERS="9 7 3 8 37.53" 6  7 for number in `echo $NUMBERS` # for number in 9 7 3 8 37.53 8 do 9  echo -n "$number " 10 done 11  12 echo 13 exit 0

---

Here is a somewhat more complex example of using command substitution to create the [list].

---

Example 11-8. A grep replacement for binary files

1 #!/bin/bash 2 # bin-grep.sh: Locates matching strings in a binary file. 3  4 # A "grep" replacement for binary files. 5 # Similar effect to "grep -a" 6  7 E_BADARGS=65 8 E_NOFILE=66 9  10 if [ $# -ne 2 ] 11 then 12  echo "Usage: `basename $0` search_string filename" 13  exit $E_BADARGS 14 fi 15  16 if [ ! -f "$2" ] 17 then 18  echo "File \"$2\" does not exist." 19  exit $E_NOFILE 20 fi 21  22  23 IFS=$'\012' # Per suggestion of Anton Filippov. 24  # was: IFS="\n" 25 for word in $( strings "$2" | grep "$1" ) 26 # The "strings" command lists strings in binary files. 27 # Output then piped to "grep", which tests for desired string. 28 do 29  echo $word 30 done 31  32 # As S.C. points out, lines 23 - 30 could be replaced with the simpler 33 # strings "$2" | grep "$1" | tr -s "$IFS" '[\n*]' 34  35  36 # Try something like "./bin-grep.sh mem /bin/ls" 37 #+ to exercise this script. 38  39 exit 0

---

More of the same.

---

Example 11-9. Listing all users on the system

1 #!/bin/bash 2 # userlist.sh 3  4 PASSWORD_FILE=/etc/passwd 5 n=1 # User number 6  7 for name in $(awk 'BEGIN{FS=":"}{print $1}' < "$PASSWORD_FILE" ) 8 # Field separator = : ^^^^^^ 9 # Print first field ^^^^^^^^ 10 # Get input from password file /etc/passwd ^^^^^^^^^^^^^^^^^ 11 do 12  echo "USER #$n = $name" 13  let "n += 1" 14 done 15  16  17 # USER #1 = root 18 # USER #2 = bin 19 # USER #3 = daemon 20 # ... 21 # USER #33 = bozo 22  23 exit $? 24  25 # Discussion: 26 # ---------- 27 # How is it that an ordinary user, or a script run by same, 28 #+ can read /etc/passwd? (Hint: Check the /etc/passwd file permissions.) 29 # Is this a security hole? Why or why not?

---

Yet another example of the [list] resulting from command substitution.

---

Example 11-10. Checking all the binaries in a directory for authorship

1 #!/bin/bash 2 # findstring.sh: 3 # Find a particular string in the binaries in a specified directory. 4  5 directory=/usr/bin/ 6 fstring="Free Software Foundation" # See which files come from the FSF. 7  8 for file in $( find $directory -type f -name '*' | sort ) 9 do 10  strings -f $file | grep "$fstring" | sed -e "s%$directory%%" 11  # In the "sed" expression, 12  #+ it is necessary to substitute for the normal "/" delimiter 13  #+ because "/" happens to be one of the characters filtered out. 14  # Failure to do so gives an error message. (Try it.) 15 done 16  17 exit $? 18  19 # Exercise (easy): 20 # --------------- 21 # Convert this script to take command-line parameters 22 #+ for $directory and $fstring.

---

A final example of [list] / command substitution, but this time the "command" is a function.

1 generate_list () 2 { 3  echo "one two three" 4 } 5  6 for word in $(generate_list) # Let "word" grab output of function. 7 do 8  echo "$word" 9 done 10  11 # one 12 # two 13 # three

The output of a for loop may be piped to a command or commands.

---

Example 11-11. Listing the symbolic links in a directory

1 #!/bin/bash 2 # symlinks.sh: Lists symbolic links in a directory. 3  4  5 directory=${1-`pwd`} 6 # Defaults to current working directory, 7 #+ if not otherwise specified. 8 # Equivalent to code block below. 9 # ---------------------------------------------------------- 10 # ARGS=1 # Expect one command-line argument. 11 # 12 # if [ $# -ne "$ARGS" ] # If not 1 arg... 13 # then 14 # directory=`pwd` # current working directory 15 # else 16 # directory=$1 17 # fi 18 # ---------------------------------------------------------- 19  20 echo "symbolic links in directory \"$directory\"" 21  22 for file in "$( find $directory -type l )" # -type l = symbolic links 23 do 24  echo "$file" 25 done | sort # Otherwise file list is unsorted. 26 # Strictly speaking, a loop isn't really necessary here, 27 #+ since the output of the "find" command is expanded into a single word. 28 # However, it's easy to understand and illustrative this way. 29  30 # As Dominik 'Aeneas' Schnitzer points out, 31 #+ failing to quote $( find $directory -type l ) 32 #+ will choke on filenames with embedded whitespace. 33 # containing whitespace. 34  35 exit 0 36  37  38 # -------------------------------------------------------- 39 # Jean Helou proposes the following alternative: 40  41 echo "symbolic links in directory \"$directory\"" 42 # Backup of the current IFS. One can never be too cautious. 43 OLDIFS=$IFS 44 IFS=: 45  46 for file in $(find $directory -type l -printf "%p$IFS") 47 do # ^^^^^^^^^^^^^^^^ 48  echo "$file" 49  done|sort 50  51 # And, James "Mike" Conley suggests modifying Helou's code thusly: 52  53 OLDIFS=$IFS 54 IFS='' # Null IFS means no word breaks 55 for file in $( find $directory -type l ) 56 do 57  echo $file 58  done | sort 59  60 # This works in the "pathological" case of a directory name having 61 #+ an embedded colon. 62 # "This also fixes the pathological case of the directory name having 63 #+ a colon (or space in earlier example) as well." 64

---

The stdout of a loop may be redirected to a file, as this slight modification to the previous example shows.

---

Example 11-12. Symbolic links in a directory, saved to a file

1 #!/bin/bash 2 # symlinks.sh: Lists symbolic links in a directory. 3  4 OUTFILE=symlinks.list # save-file 5  6 directory=${1-`pwd`} 7 # Defaults to current working directory, 8 #+ if not otherwise specified. 9  10  11 echo "symbolic links in directory \"$directory\"" > "$OUTFILE" 12 echo "---------------------------" >> "$OUTFILE" 13  14 for file in "$( find $directory -type l )" # -type l = symbolic links 15 do 16  echo "$file" 17 done | sort >> "$OUTFILE" # stdout of loop 18 # ^^^^^^^^^^^^^ redirected to save file. 19  20 # echo "Output file = $OUTFILE" 21  22 exit $?

---

There is an alternative syntax to a for loop that will look very familiar to C programmers. This requires double parentheses.

---

Example 11-13. A C-style for loop

1 #!/bin/bash 2 # Multiple ways to count up to 10. 3  4 echo 5  6 # Standard syntax. 7 for a in 1 2 3 4 5 6 7 8 9 10 8 do 9  echo -n "$a " 10 done 11  12 echo; echo 13  14 # +==========================================+ 15  16 # Using "seq" ... 17 for a in `seq 10` 18 do 19  echo -n "$a " 20 done 21  22 echo; echo 23  24 # +==========================================+ 25  26 # Using brace expansion ... 27 # Bash, version 3+. 28 for a in {1..10} 29 do 30  echo -n "$a " 31 done 32  33 echo; echo 34  35 # +==========================================+ 36  37 # Now, let's do the same, using C-like syntax. 38  39 LIMIT=10 40  41 for ((a=1; a <= LIMIT ; a++)) # Double parentheses, and naked "LIMIT" 42 do 43  echo -n "$a " 44 done # A construct borrowed from ksh93. 45  46 echo; echo 47  48 # +=========================================================================+ 49  50 # Let's use the C "comma operator" to increment two variables simultaneously. 51  52 for ((a=1, b=1; a <= LIMIT ; a++, b++)) 53 do # The comma concatenates operations. 54  echo -n "$a-$b " 55 done 56  57 echo; echo 58  59 exit 0

---

See also Example 27-16, Example 27-17, and Example A-6.

---

Now, a for loop used in a "real-life" context.

---

Example 11-14. Using efax in batch mode

1 #!/bin/bash 2 # Faxing (must have 'efax' package installed). 3  4 EXPECTED_ARGS=2 5 E_BADARGS=85 6 MODEM_PORT="/dev/ttyS2" # May be different on your machine. 7 # ^^^^^ PCMCIA modem card default port. 8  9 if [ $# -ne $EXPECTED_ARGS ] 10 # Check for proper number of command-line args. 11 then 12  echo "Usage: `basename $0` phone# text-file" 13  exit $E_BADARGS 14 fi 15  16  17 if [ ! -f "$2" ] 18 then 19  echo "File $2 is not a text file." 20  # File is not a regular file, or does not exist. 21  exit $E_BADARGS 22 fi 23  24  25 fax make $2 # Create fax-formatted files from text files. 26  27 for file in $(ls $2.0*) # Concatenate the converted files. 28  # Uses wild card (filename "globbing") 29  #+ in variable list. 30 do 31  fil="$fil $file" 32 done 33  34 efax -d "$MODEM_PORT" -t "T$1" $fil # Finally, do the work. 35 # Trying adding -o1 if above line fails. 36  37  38 # As S.C. points out, the for-loop can be eliminated with 39 # efax -d /dev/ttyS2 -o1 -t "T$1" $2.0* 40 #+ but it's not quite as instructive [grin]. 41  42 exit $? # Also, efax sends diagnostic messages to stdout.

---

The keywords do and done delineate the for-loop command block. However, these may, in certain contexts, be omitted by framing the command block within curly brackets 1 for((n=1; n<=10; n++)) 2 # No do! 3 { 4  echo -n "* $n *" 5 } 6 # No done! 7  8  9 # Outputs: 10 # * 1 ** 2 ** 3 ** 4 ** 5 ** 6 ** 7 ** 8 ** 9 ** 10 * 11 # And, echo $? returns 0, so Bash does not register an error. 12  13  14 echo 15  16  17 # But, note that in a classic for-loop: for n in [list] ... 18 #+ a terminal semicolon is required. 19  20 for n in 1 2 3 21 { echo -n "$n "; } 22 # ^ 23  24  25 # Thank you, YongYe, for pointing this out.

while

This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is true (returns a 0 exit status). In contrast to a for loop, a while loop finds use in situations where the number of loop repetitions is not known beforehand.

while [ condition ]
do
 command(s)...
done

The bracket construct in a while loop is nothing more than our old friend, the test brackets used in an if/then test. In fact, a while loop can legally use the more versatile double-brackets construct (while [[ condition ]]).

As is the case with for loops, placing the do on the same line as the condition test requires a semicolon.

while [ condition ] ; do

Note that the test brackets are not mandatory in a while loop. See, for example, the getopts construct.

---

Example 11-15. Simple while loop

1 #!/bin/bash 2  3 var0=0 4 LIMIT=10 5  6 while [ "$var0" -lt "$LIMIT" ] 7 # ^ ^ 8 # Spaces, because these are "test-brackets" . . . 9 do 10  echo -n "$var0 " # -n suppresses newline. 11  # ^ Space, to separate printed out numbers. 12  13  var0=`expr $var0 + 1` # var0=$(($var0+1)) also works. 14  # var0=$((var0 + 1)) also works. 15  # let "var0 += 1" also works. 16 done # Various other methods also work. 17  18 echo 19  20 exit 0

---

---

Example 11-16. Another while loop

1 #!/bin/bash 2  3 echo 4  # Equivalent to: 5 while [ "$var1" != "end" ] # while test "$var1" != "end" 6 do 7  echo "Input variable #1 (end to exit) " 8  read var1 # Not 'read $var1' (why?). 9  echo "variable #1 = $var1" # Need quotes because of "#" . . . 10  # If input is 'end', echoes it here. 11  # Does not test for termination condition until top of loop. 12  echo 13 done 14  15 exit 0

---

A while loop may have multiple conditions. Only the final condition determines when the loop terminates. This necessitates a slightly different loop syntax, however.

---

Example 11-17. while loop with multiple conditions

1 #!/bin/bash 2  3 var1=unset 4 previous=$var1 5  6 while echo "previous-variable = $previous" 7  echo 8  previous=$var1 9  [ "$var1" != end ] # Keeps track of what $var1 was previously. 10  # Four conditions on *while*, but only the final one controls loop. 11  # The *last* exit status is the one that counts. 12 do 13 echo "Input variable #1 (end to exit) " 14  read var1 15  echo "variable #1 = $var1" 16 done 17  18 # Try to figure out how this all works. 19 # It's a wee bit tricky. 20  21 exit 0

---

As with a for loop, a while loop may employ C-style syntax by using the double-parentheses construct (see also Example 8-5).

---

Example 11-18. C-style syntax in a while loop

1 #!/bin/bash 2 # wh-loopc.sh: Count to 10 in a "while" loop. 3  4 LIMIT=10 # 10 iterations. 5 a=1 6  7 while [ "$a" -le $LIMIT ] 8 do 9  echo -n "$a " 10  let "a+=1" 11 done # No surprises, so far. 12  13 echo; echo 14  15 # +=================================================================+ 16  17 # Now, we'll repeat with C-like syntax. 18  19 ((a = 1)) # a=1 20 # Double parentheses permit space when setting a variable, as in C. 21  22 while (( a <= LIMIT )) # Double parentheses, 23 do #+ and no "$" preceding variables. 24  echo -n "$a " 25  ((a += 1)) # let "a+=1" 26  # Yes, indeed. 27  # Double parentheses permit incrementing a variable with C-like syntax. 28 done 29  30 echo 31  32 # C and Java programmers can feel right at home in Bash. 33  34 exit 0

---

Inside its test brackets, a while loop can call a function.

1 t=0 2  3 condition () 4 { 5  ((t++)) 6  7  if [ $t -lt 5 ] 8  then 9  return 0 # true 10  else 11  return 1 # false 12  fi 13 } 14  15 while condition 16 # ^^^^^^^^^ 17 # Function call -- four loop iterations. 18 do 19  echo "Still going: t = $t" 20 done 21  22 # Still going: t = 1 23 # Still going: t = 2 24 # Still going: t = 3 25 # Still going: t = 4

Similar to the if-test construct, a while loop can omit the test brackets. 1 while condition 2 do 3  command(s) ... 4 done

By coupling the power of the read command with a while loop, we get the handy while read construct, useful for reading and parsing files.

1 cat $filename | # Supply input from a file. 2 while read line # As long as there is another line to read ... 3 do 4  ... 5 done 6  7 # =========== Snippet from "sd.sh" example script ========== # 8  9  while read value # Read one data point at a time. 10  do 11  rt=$(echo "scale=$SC; $rt + $value" | bc) 12  (( ct++ )) 13  done 14  15  am=$(echo "scale=$SC; $rt / $ct" | bc) 16  17  echo $am; return $ct # This function "returns" TWO values! 18  # Caution: This little trick will not work if $ct > 255! 19  # To handle a larger number of data points, 20  #+ simply comment out the "return $ct" above. 21 } <"$datafile" # Feed in data file.

A while loop may have its stdin redirected to a file by a < at its end. A while loop may have its stdin supplied by a pipe.

until

This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is false (opposite of while loop).

until [ condition-is-true ]
do
 command(s)...
done

Note that an until loop tests for the terminating condition at the top of the loop, differing from a similar construct in some programming languages.

As is the case with for loops, placing the do on the same line as the condition test requires a semicolon.

until [ condition-is-true ] ; do

---

Example 11-19. until loop

1 #!/bin/bash 2  3 END_CONDITION=end 4  5 until [ "$var1" = "$END_CONDITION" ] 6 # Tests condition here, at top of loop. 7 do 8  echo "Input variable #1 " 9  echo "($END_CONDITION to exit)" 10  read var1 11  echo "variable #1 = $var1" 12  echo 13 done 14  15 # --- # 16  17 # As with "for" and "while" loops, 18 #+ an "until" loop permits C-like test constructs. 19  20 LIMIT=10 21 var=0 22  23 until (( var > LIMIT )) 24 do # ^^ ^ ^ ^^ No brackets, no $ prefixing variables. 25  echo -n "$var " 26  (( var++ )) 27 done # 0 1 2 3 4 5 6 7 8 9 10 28  29  30 exit 0

---