3.2. Special characters used in shell scripts

#

Comments. Lines beginning with a # (with the exception of #!) are comments.

1 # This line is a comment.

Comments may also occur at the end of a command.

1 echo "A comment will follow." # Comment here.

Comments may also follow whitespace at the beginning of a line.

1  # A tab precedes this comment.

A command may not follow a comment on the same line. There is no method of terminating the comment, in order for "live code" to begin on the same line. Use a new line for the next command.

Of course, an escaped # in an echo statement does not begin a comment. Likewise, a # appears in certain parameter substitution constructs and in numerical constant expressions. 1 echo "The # here does not begin a comment." 2 echo 'The # here does not begin a comment.' 3 echo The \# here does not begin a comment. 4 echo The # here begins a comment. 5  6 echo ${PATH#*:} # Parameter substitution, not a comment. 7 echo $(( 2#101011 )) # Base conversion, not a comment. 8  9 # Thanks, S.C. The standard quoting and escape characters (" ' \) escape the #.

;

Command separator. Permits putting two or more commands on the same line.

1 echo hello; echo there

Note that the ";" sometimes needs to be escaped.

.

"dot" command. Equivalent to source (see Example 3-82). This is a bash builtin.

"

partial quoting. "STRING" preserves (from interpretation) most of the special characters within STRING. See also Section 3.4.

'

full quoting. 'STRING' preserves all special characters within STRING. This is a stronger form of quoting than using ". See also Section 3.4.

\

escape. \X "escapes" the character X. This has the effect of "quoting" X, equivalent to 'X'. The \ may be used to quote " and ', so they are expressed literally.

`

command substitution. `command` makes available the output of command for setting a variable. This is also known as backticks or backquotes.

:

null command. This is the shell equivalent of a "NOP" (no op, a do-nothing operation). It may be considered a synonym for the shell builtin true. Note that : is a Bash builtin, and its exit status is 0.

Endless loop:

1 while : 2 do 3  operation-1 4  operation-2 5  ... 6  operation-n 7 done 8  9 # Same as: 10 # while true 11 # do 12 # ... 13 # done

Placeholder in if/then test:

1 if condition 2 then : # Do nothing and branch ahead 3 else 4  take-some-action 5 fi

Provide a placeholder where a binary operation is expected, see Example 3-20 and default parameters.

1 : ${username=`whoami`} 2 # ${username=`whoami`} without the leading : gives an error 3 # unless "username" is a command or builtin...

Provide a placeholder where a command is expected in a here document. See Example 3-132.

Evaluate string of variables using parameter substitution, see Example 3-7:

1 : ${HOSTNAME?} ${USER?} ${MAIL?} 2 #Prints error message if one or more of essential environmental variables not set.

In combination with the > redirection operator, truncates a file to zero length, without changing its permissions. If the file did not previously exist, creates it.

1 : > data.xxx # File "data.xxx" now empty. 2  3 # Same effect as cat /dev/null >data.xxx 4 # However, this does not fork a new process, since ":" is a builtin.

See also Example 3-92.

In combination with the >> redirection operator, updates a file access/modification time (: >> new_file). If the file did not previously exist, creates it. This is equivalent to touch.

This applies to regular files, not pipes, symlinks, and certain special files.

May be used to begin a comment line, although this is not recommended. Using # for a comment turns off error checking for the remainder of that line, so almost anything may be appear in a comment. However, this is not the case with :.

1 : This is a comment that generates an error, ( if [ $x -eq 3] ).

!

reverse (or negate) the sense of a test or exit status. The ! operator inverts the exit status of the command to which it is applied (see Example 3-2). It also inverts the meaning of a test operator. This can, for example, change the sense of "equal" ( = ) to "not-equal" ( != ). The ! operator is a Bash keyword.

*

wild card. The * character serves as a "wild card" for filename expansion in globbing, as well as representing any number (or zero) characters in a regular expression.

$*, $@

positional parameters. The list of the positional parameters (command-line arguments) passed to a script.

$_

last argument. Internal shell variable set to last argument of previous command executed. See Example 3-30.

Whitespace

functions as a separator, separating commands or variables. Whitespace consists of either spaces, tabs, blank lines, or any combination thereof. In some contexts, such as variable assignment, whitespace is not permitted, and results in a syntax error.

()

command group.

1 (a=hello; echo $a)

A listing of commands within parentheses starts a subshell. Variables inside parentheses, within the subshell, are not visible to the rest of the script. The parent process, the script, cannot read variables created in the child process, the subshell. 1 a=123 2 ( a=321; ) 3  4 echo "a = $a" # a = 123 5 # "a" within parentheses acts like a local variable.

array initialization.

1 Array=(element1 element2 element3)

${}

Parameter substitution.

{xxx,yyy,zzz,...}

Brace expansion.

1 grep Linux file*.{txt,htm*} 2 # Finds all instances of the work "Linux" 3 # in the files "fileA.txt", "file2.txt", "fileR.html", "file-87.htm", etc.

A command may act upon a comma-separated list of file specs within braces. [1] Filename expansion (globbing) applies to the file specs between the braces.

No spaces allowed within the braces unless the spaces are quoted or escaped. echo {file1,file2}\ :{\ A," B",' C'} file1 : A file1 : B file1 : C file2 : A file2 : B file2 : C

{}

Block of code. Also referred to as an "inline group", this construct, in effect, creates an anonymous function. However, unlike a function, the variables in a code block remain visible to the remainder of the script.

bash$ { local a; a=123; } bash: local: can only be used in a function

1 a=123 2 { a=321; } 3 echo "a = $a" # a = 321 (value inside code block) 4  5 # Thanks, S.C.

The code block enclosed in braces may have I/O redirected to and from it.

---

Example 3-3. Code blocks and I/O redirection

1 #!/bin/bash 2 # Reading lines in /etc/fstab. 3  4 { 5 read line1 6 read line2 7 } < /etc/fstab 8  9 echo "First line in /etc/fstab is:" 10 echo "$line1" 11 echo 12 echo "Second line in /etc/fstab is:" 13 echo "$line2" 14  15 exit 0

---

---

Example 3-4. Saving the results of a code block to a file

1 #!/bin/bash 2  3 # rpm-check 4 # --------- 5 # Queries an rpm file for description, listing, and whether it can be installed. 6 # Saves output to a file. 7 # 8 # This script illustrates using a code block. 9  10 NOARGS=1 11  12 if [ -z "$1" ] 13 then 14  echo "Usage: `basename $0` rpm-file" 15  exit $NOARGS 16 fi 17  18 { 19  echo 20  echo "Archive Description:" 21  rpm -qpi $1 #Query description. 22  echo 23  echo "Archive Listing:" 24  rpm -qpl $1 #Query listing. 25  echo 26  rpm -i --test $1 #Query whether rpm file can be installed. 27  if [ ! $? ] 28  then 29  echo "$1 can be installed." 30  else 31  echo "$1 cannot be installed." 32  fi 33  echo 34 } > "$1.test" # Redirects output of everything in block to file. 35  36 echo "Results of rpm test in file $1.test" 37  38 # See rpm man page for explanation of options. 39  40 exit 0

---

Unlike a command group within (parentheses), as above, a code block enclosed by {braces} will not normally launch a subshell. [2]

{} \;

pathname. Mostly used in find constructs. This is not a shell builtin.

The ";" ends the -exec option of a find command sequence. It needs to be escaped to protect it from interpretation by the shell.

[ ]

test.

Test expression between [ ]. Note that [ is part of the shell builtin test (and a synonym for it), not a link to the external command /usr/bin/test.

[[ ]]

test.

Test expression between [[ ]] (shell keyword).

See the discussion on the [[ ... ]] construct for more details.

> >& >> <

redirection.

scriptname >filename redirects the output of scriptname to file filename. Overwrite filename if it already exists.

command >&2 redirects output of command to stderr.

scriptname >>filename appends the output of scriptname to file filename. If filename does not already exist, it will be created.

process substitution.

(command)>

<(command)

<<

redirection used in a here document.

|

pipe. Passes the output of previous command to next one, or to shell. This is a method of chaining commands together.

1 echo ls -l | sh

passes the output of "echo ls -l" to the shell, with the same result as a simple "ls -l".

1 cat *.lst | sort | uniq

sorts the output of all the .lst files and deletes duplicate lines.

The output of a command or commands may be piped to a script.

1 #!/bin/bash 2 # uppercase.sh : Changes input to uppercase. 3  4 tr '[a-z]' '[A-Z]' 5 # Letter ranges must be quoted 6 # to prevent filename generation from single-letter filenames. 7  8 exit 0

Now, let us pipe the output of ls -l to this script.

bash$ ls -l | ./uppercase.sh -RW-RW-R-- 1 BOZO BOZO 109 APR 7 19:49 1.TXT -RW-RW-R-- 1 BOZO BOZO 109 APR 14 16:48 2.TXT -RW-R--R-- 1 BOZO BOZO 725 APR 20 20:56 DATA-FILE

If one of the commands in the pipe aborts, this prematurely terminates execution of the pipe. Called a broken pipe, this condition sends a SIGPIPE signal.

>|

force redirection (even if the noclobber option is set). This will forcibly overwrite an existing file.

-

redirection from/to stdin or stdout.

1 (cd /source/directory && tar cf - . ) | (cd /dest/directory && tar xpvf -) 2 # Move entire file tree from one directory to another 3 # [courtesy Alan Cox <a.cox@swansea.ac.uk>, with a minor change] 4  5 # 1) cd /source/directory Source directory, where the files to be moved are. 6 # 2) && "And-list": if the 'cd' operation successful, then execute the next command. 7 # 3) tar cf - . The 'c' option 'tar' archiving command creates a new archive, 8 # the 'f' (file) option, followed by '-' designates the target file as stdout, 9 # and do it in current directory tree ('.'). 10 # 4) | Piped to... 11 # 5) ( ... ) a subshell 12 # 6) cd /dest/directory Change to the destination directory. 13 # 7) && "And-list", as above 14 # 8) tar xpvf - Unarchive ('x'), preserve ownership and file permissions ('p'), 15 # and send verbose messages to stdout ('v'), 16 # reading data from stdin ('f' followed by '-'). 17 # 18 # Note that 'x' is a command, and 'p', 'v', 'f' are options. 19 # Whew! 20  21  22  23 # More elegant than, but equivalent to: 24 # cd source-directory 25 # tar cf - . | (cd ../target-directory; tar xzf -) 26 # 27 # cp -a /source/directory /dest also has same effect.

1 bunzip2 linux-2.4.3.tar.bz2 | tar xvf - 2 # --uncompress tar file-- | --then pass it to "tar"-- 3 # If "tar" has not been patched to handle "bunzip2", 4 # this needs to be done in two discrete steps, using a pipe. 5 # The purpose of the exercise is to unarchive "bzipped" kernel source.

Note that in this context the "-" is not itself a Bash operator, but rather an option recognized by certain UNIX utilities that write to stdout, such as tar, cat, etc.

Where a filename is expected, - redirects output to stdout (sometimes seen with tar cf), or accepts input from stdin, rather than from a file. This is a method of using a file-oriented utility as a filter in a pipe.

bash$ file Usage: file [-bciknvzL] [-f namefile] [-m magicfiles] file...

By itself on the command line, file fails with an error message.

bash$ file - #!/bin/bash standard input: Bourne-Again shell script text executable

This time, it accepts input from stdin and filters it.

Try using diff to compare a file with a section of another.

grep bash file1 | diff file2 -

Finally, a real-world example using - with tar.

---

Example 3-5. Backup of all files changed in last day

1 #!/bin/bash 2  3 # Backs up all files in current directory 4 # modified within last 24 hours 5 # in a tarred and gzipped file. 6  7 if [ $# = 0 ] 8 then 9  echo "Usage: `basename $0` filename" 10  exit 65 11 fi 12  13 tar cvf - `find . -mtime -1 -type f -print` > $1.tar 14 gzip $1.tar 15  16  17 # Stephane Chazelas points out that the above code will fail 18 # if there are too many files found 19 # or if any filenames contain blank characters. 20  21 # He suggests the following alternatives: 22 # ------------------------------------------------------------- 23 # find . -mtime -1 -type f -print0 | xargs -0 tar rvf "$1.tar" 24 # using the GNU version of "find". 25  26 # find . -mtime -1 -type f -exec tar rvf "$1.tar" '{}' \; 27 # portable to other UNIX flavors, but much slower. 28  29  30 exit 0

---

Filenames beginning with - may cause problems when coupled with the - redirection operator. A script should check for this and pass such filenames as ./-FILENAME or $PWD/-FILENAME. If the value of a variable begins with a -, this may likewise create problems. 1 var="-n" 2 echo $var 3 # Has the effect of "echo -n", and outputs nothing.

-

previous working directory. cd - changes to previous working directory. This uses the $OLDPWD environmental variable.

This is not to be confused with the "-" redirection operator just discussed. The interpretation of the "-" depends on the context in which it appears.

~

home directory. This corresponds to the $HOME internal variable. ~bozo is bozo's home directory, and ls ~bozo lists the contents of it. ~/ is the current user's home directory, and ls ~/ lists the contents of it.

bash$ echo ~bozo /home/bozo bash$ echo ~ /home/bozo bash$ echo ~/ /home/bozo/ bash$ echo ~: /home/bozo: bash$ echo ~nonexistent-user ~nonexistent-user

Blank lines

Blank lines have no effect on the action of a script, and are therefore useful for visually separating functional sections of the script.