# Shell Scripting

# From http://inn.weizmann.ac.il/UNIXhelp/Pages/scrpt/index.html

 

Simple Example

===================

   cat display

   # This script displays the date, time, username and

   # current directory.

   echo "Date and time is:"

   date

   echo

   echo "Your username is: `whoami` \\n"

   echo "Your current directory is: \\c"

   pwd

 

The first two lines beginning with a hash (#) are comments and are not interpreted by the shell. Use comments to document your shell script; you will be surprised how easy it is to forget what your own programs do!

 

The backquotes (`) around the command whoami illustrate the use of command substitution.

 

The \\n is an option of the echo command that tells the shell to add an extra carriage return at the end of the line. The \\c tells the shell to stay on the same line. See the man page for details of other options.

 

The argument to the echo command is quoted to prevent the shell interpreting these commands as though they had been escaped with the \\ (backslash) character.

 

Command substitution

To include the output from one command within the command line for another command, enclose the command whose output is to be included within `backquotes`. For example:

 

   echo The date today is `date +%d/%m/%y`

   The date today is 17/05/95

 

The output from the command date +%d/%m/%y is substituted at the appropriate location within a command line that uses the echo command.

 

Bourne shell programming

Examples of Bourne shell scripts

   #!/bin/sh

   # usage: fsplit file1 file2
   total=0; lost=0
   while read next
   do
   total=`expr $total + 1`
   case "$next" in
   *[A-Za-z]*)  echo "$next" >> $1 ;;
   *[0-9]*)     echo "$next" >> $2 ;;
   *)           lost=`expr $lost + 1`
   esac
   done
   echo "$total lines read, $lost thrown away"

The user types the command:

   fsplit file1 file2

They then enter lines of text and issue an EOF instruction. The script then processes the lines as follows:

A line with at least one letter is appended to file1; any line with at least one digit and no letters is appended to file2. All other lines are thrown away.


To read commands from the terminal and process them:

   #!/bin/sh
   # usage: process sub-directory
   dir=`pwd`
   for i in *
   do
   if test -d $dir/$i
   then
     cd $dir/$i
     while echo "$i:" 
     read x
     do
       eval $x
     done
     cd ..
   fi
   done

The user types the command:

   process sub-directory

This script will read and process commands in the named sub-directory. The user is prompted to supply the name of the command to be read in. This command is executed using the the builtin eval function.


To create a command:

   #!/bin/sh
   flag=
   for i
   do
   case $i in
   -c)   flag=N ;;
    *)   if test -f $i
         then
           ln $i junk$$
           rm junk$$
         elif test $flag                 # true if not null
         then
           echo \'$i\' does not exist
         else
           >$i
         fi ;;
   esac
   done

This command takes filenames as its parameters. If a file exists it changes the modification date. If no file exists it creates a new one. This script is similar in action to the touch command.

The -c argument lets you specify that you only want to update a file that already exists and not to create one if it doesn't.

Passing arguments to the shell

Shell scripts can act like standard UNIX commands and take arguments from the command line.

Arguments are passed from the command line into a shell program using the positional parameters $1 through to $9. Each parameter corresponds to the position of the argument on the command line.

The positional parameter $0 refers to the command name or name of the executable file containing the shell script.

Only nine command line arguments can be accessed, but you can access more than nine using the shift command.

All the positional parameters can be referred to using the special parameter $*. This is useful when passing filenames as arguments. For example:

   cat printps
   # This script converts ASCII files to PostScript
   # and sends them to the PostScript printer ps1
   # It uses a local utility "a2ps"
   a2ps $* | lpr -Pps1
   printps elm.txt vi.ref msg

This processes the three files given as arguments to the command printps.

Examples of passing arguments to the shell

To pass several arguments from the command line to the shell:

   cat first_5args
   # This script echoes the first five arguments
   # supplied to the script
   echo The first five command line
   echo arguments are $1 $2 $3 $4 $5
   first_5args mines a pint john o.k.
   The first five command line
   arguments are mines a pint john o.k.

This passes the arguments represented by parameters $1 through $5 to the shell script.


To pass the value of each positional parameter to the shell script:

   cat printps
   # This script converts ASCII files to PostScript
   # and sends them to the PostScript printer ps1
   # It uses a local utility "a2ps"
   a2ps $* | lpr -Pps1
   printps elm.txt vi.ref msg

This processes the three files given as arguments to the command printps.


Using the shift command

Usually only nine command line arguments can be accessed using positional parameters. The shift command gives access to command line arguments greater than nine by shifting each of the arguments.

The second argument ($2) becomes the first ($1), the third ($3) becomes the second ($2) and so on. This gives you access to the tenth command line argument by making it the ninth. The first argument is no longer available.

Successive shift commands make additional arguments available. Note that there is no "unshift" command to bring back arguments that are no longer available!

Example of using the shift command

To successively shift the argument that is represented by each positional parameter:

   cat shift_demo
   #!/bin/sh
   echo "arg1=$1 arg2=$2 arg3=$3"
   shift
   echo "arg1=$1 arg2=$2 arg3=$3"
   shift
   echo "arg1=$1 arg2=$2 arg3=$3"
   shift
   echo "arg1=$1 arg2=$2 arg3=$3"
   shift_demo one two three four five six seven
   arg1=one arg2=two arg3=three
   arg1=two arg2=three arg3=four
   arg1=three arg2=four arg3=five
   arg1=four arg2=five arg3=six
   arg1=five arg2=six arg3=seven

Handling shell variables

The shell has several variables which are automatically set whenever you login.

The values of some of these variables are stored in names which collectively are called your user environment.

Any name defined in the user environment, can be accessed from within a shell script. To include the value of a shell variable into the environment you must export it.

Special shell variables

There are some variables which are set internally by the shell and which are available to the user:

 
Name          Description


 
$1 - $9       these variables are the positional parameters.
 
$0            the name of the command currently being executed.
 
$#            the number of positional arguments given to this
              invocation of the shell.
 
$?            the exit status of the last command executed is
              given as a decimal string.  When a command
              completes successfully, it returns the exit status
              of 0 (zero), otherwise it returns a non-zero exit
              status.
 
$$            the process number of this shell - useful for
              including in filenames, to make them unique.
 
$!            the process id of the last command run in
              the background.
 
$-            the current options supplied to this invocation
              of the shell.
 
$*            a string containing all the arguments to the
              shell, starting at $1.
 
$ {at}  {at}            same as above, except when quoted.
 


Notes

$* and $ {at} {at} when unquoted are identical and expand into the arguments.

"$*" is a single word, comprising all the arguments to the shell, joined together with spaces. For example '1 2' 3 becomes "1 2 3".

"$ {at} {at} " is identical to the arguments received by the shell, the resulting list of words completely match what was given to the shell. For example '1 2' 3 becomes "1 2" "3"

Evaluating shell variables

The following set of rules govern the evaluation of all shell variables.

 
Definition            Description


 
$var                  signifies the value of var or nothing,
                      if var is undefined.
 
${var}                same as above except the braces enclose
                      the name of the variable to be substituted.
 
${var-thing}          value of var if var is defined; otherwise thing.
                      $var is not set to thing.
 
${var=thing}          value of var if var is defined; otherwise thing.
                      If undefined $var is set to thing.
 
${var?message}        If defined, $var; otherwise print message
                      and exit the shell.  If the message is
                      empty, print a standard message.
 
${var+thing}          thing if $var is defined, otherwise nothing.
 

Reading user input

To read standard input into a shell script use the read command. For example:

   echo "Please enter your name:"
   read name
   echo "Welcome to Edinburgh $name"

This prompts the user for input, assigns this to the variable name and then displays the value of this variable to standard output.

If there is more than one word in the input, each word can be assigned to a different variable. Any words left over are assigned to the last named variable. For example:

   echo "Please enter your surname\n"
   echo "followed by your first name: \c"
   read name1 name2
   echo "Welcome to Glasgow $name2 $name1"

Conditional statements

Every Unix command returns a value on exit which the shell can interrogate. This value is held in the read-only shell variable $?.

A value of 0 (zero) signifies success; anything other than 0 (zero) signifies failure.

The if statement

The if statement uses the exit status of the given command and conditionally executes the statements following. The general syntax is:

   if test
   then
      commands     (if condition is true)
   else
      commands     (if condition is false)
   fi

then, else and fi are shell reserved words and as such are only recognised after a newline or ; (semicolon). Make sure that you end each if construct with a fi statement.

if statements may be nested:

   if ...
   then ...
   else if ...
       ...
     fi
   fi

The elif statement can be used as shorthand for an else if statement. For example:

   if ...
   then ...
   elif ...
     ...
   fi

Example of using an if construct

To carry out a conditional action:

   if who | grep -s keith > /dev/null
   then
   echo keith is logged in
   else
   echo keith not available
   fi

This lists who is currently logged on to the sytem and pipes the output through grep to search for the username keith.

The -s option causes grep to work silently and any error messages are directed to the file /dev/null instead of the standard output.

If the command is succesful i.e. the username keith is found in the list of users currently logged in then the message

   keith is logged on

is displayed, otherwise the second message is displayed.

The && operator

You can use the && operator to execute a command and, if it is successful, execute the next command in the list. For example:

   cmd1 && cmd2

cmd1 is executed and its exit status examined. Only if cmd1 succeeds is cmd2 executed. This is a terse notation for:

   if cmd1 
   then
     cmd2
   fi

Example of using the && operator

To notify the user about the outcome of a previous command:

   cat deliver
   #!/bin/sh
   # usage: deliver username  filename
   { cat $2 | write $1 ; } && echo done

The user types a command such as:

   deliver keith greeting

The first command { cat $2 | write $1 ; } concatenates and displays the message held in the file greeting and pipes the output through the write command whose argument is the name of the user to whom the message is to be sent.

Note the use of the positional parameters $1 and $2. The ; (semicolon) is needed to sequentially execute the preceeding pipeline.

If this command is successful, the message done is displayed on standard output.


The || operator

You can use the || operator to execute a command and, if it fails, execute the next command in the command list. For example:

   cmd1 || cmd2

cmd1 is executed and its exit status examined. If cmd1fails then cmd2 is executed. This is a terse notation for:

   cmd1
   if           test $? -ne 0
   then
     cmd2 
   fi

Example of using the || operator

To send a message to a user using the appropriate utility:

   cat writemail
   #!/bin/sh
   # usage: writemail user message
   echo "$2" |{ write "$1" || mail "$1" ;} 

The user types a command such as:

   writemail sarah 'call me'

The message entered by the user is piped through the command { write "$1" || mail "$1" ; }.

If the the message cannot be sent to the user's terminal (they are not logged on) with the command write "$1" then the message is sent to the user by mail.

Testing for files and variables with the test command

The shell uses a command called test to evaluate conditional expressions. Full details of this command can be found in the test manual page. For example:

   if test ! -f $FILE
   then
     if test "$WARN" = "yes"
     then
       echo "$FILE does not exist"
     fi
   fi

First, we test to see if the filename specified by the variable $FILE exists and is a regular file. If it does not then we test to see if the variable $WARN is assigned the value yes, and if it is a message that the filename does not exist is displayed.


The case statement

case is a flow control construct that provides for multi-way branching based on patterns.

Program flow is controlled on the basis of the wordgiven. This word is compared with each pattern in order until a match is found, at which point the associated command(s) are executed.

   case word in
   pattern1) command(s)
   ;;
   pattern2) command(s)
   ;;
   patternN) command(s)
   ;;
   esac

When all the commands are executed control is passed to the first statement after the esac. Each list of commands must end with a double semi-colon (;;).

A command can be associated with more than one pattern. Patterns can be separated from each other by a | symbol. For example:

   case word in
   pattern1|pattern2) command
   ...                                                                      ;;

Patterns are checked for a match in the order in which they appear. A command is always carried out after the first instance of a pattern.

The * character can be used to specify a default pattern as the * character is the shell wildcard character.

Example of using the case statement

To specify an action when a word matches the pattern:

   cat diary
   #!/bin/sh
   today=`date +%m/%d`          (presents the date in the format 01/31)
   case     $today  in
   07/18)   echo        "Aonoch Mhor"
   ;;
   07/21)   echo        "Ben Wyvis"
   ;;
   08/02)   echo        "Buicheille Etive Mhor"
   ;;
   08/03)   echo        "Slioch"
   ;;
   *)                         echo        "Wet..low level today"
   esac
   date +%m/%d
   07/18
   diary
   Aonoch Mhor 

The value for the word $today is generated by the date command. This is then compared with various patterns so that the appropriate commands are executed.

Note the use of the pattern *, this can be used to specify default patterns as the * character is the shell wildcard character.

The for statement

The for loop notation has the general form:

   for var in list-of-words
   do
     commands
   done

commands is a sequence of one or more commands separated by a newline or ; (semicolon).

The reserved words do and done must be preceded by a newline or ; (semicolon). Small loops can be written on a single line. For example:

   for var in list; do commands; done

Examples of using the for statement

To take each argument in turn and see if that person is logged onto the system.

   cat snooper
   #!/bin/sh
   # see if a number of people are logged in
   for i in $*
   do
     if who | grep -s $i > /dev/null
     then
       echo $i is logged in
     else
       echo $i not available
     fi
   done

For each username given as an argument an if statement is used to test if that person is logged on and an appropriate message is then displayed.


To go through each file in the current directory and compare it with the same filename in another directory:

   #!/bin/sh
   # compare files to same file in directory "old"
   for i in *
   do
     echo $i:
     cmp $i old/$i
     echo
   done

If the list-of-words is omitted, then the loop is executed once for each positional argument (i.e. assumes $* in the for statement). In this case the loop will create the empty files whose names are given as arguments.

   #!/bin/sh
   # create all named files
   for i
   do
     > $i
   done

Some examples of command substitution in for loops:

   #!/bin/sh
   # do something for all files in current
   # directory according to time modified
   for i in `ls -t`
   do
     ...
   done
   # do something for all non-fred files.
   for i in `cat filelist | grep -v fred`
   do
     ...
   done
   # do something to each sub-directory found
   for i in `for i in *
     do
       if test -d $i
       then
         echo $i
       fi
     done`
   do
     ...
   done

The while and until statements

The while statement has the general form:

   while command-list1
   do
     command-list2
   done 

The commands in command-list1 are executed; and if the exit status of the last command in that list is 0 (zero), the commands in command-list2 are executed.

The sequence is repeated as long as the exit status of command-list1 is 0 (zero).

The until statement has the general form:

   until command-list1
   do
     command-list2
   done

This is identical in function to the while command except that the loop is executed as long as the exit status of command-list1 is non-zero.

The exit status of a while/until command is the exit status of the last command executed in command-list2. If no such command list is executed, a while/until has an exit status of 0 (zero).

Examples of using the while and until statements

To wait for someone to logout:

   #!/bin/sh
   while who |grep -s $1 >/dev/null
   do
     sleep 60
   done
   echo "$1 has logged out"

This script checks to see if the username given as an argument to the script is logged on. While they are, the script waits for 60 seconds before checking again. When it is found that the user is no longer logged on a message that they have logged out is displayed.


To declare when a file has been created:

   #!/bin/sh
   until test -f $FILE
   do
     sleep 60
   done
   echo "$FILE now exists"

This tests every 60 seconds until the filename represented by the variable $FILE exists. A message is then displayed.


To watch for someone to log in:

   #!/bin/sh
   # make sure we pick up the correct commands
   PATH=/bin:/usr/bin
   # remember $# is number of positional arguments
   case $# in
   1) ;;
   *) echo 'usage: watchfor username' ; exit 1
   esac
   until who | grep -s "$1" >/dev/null
   do
     sleep 60
   done
   echo "$1 has logged in"

If more than one username is given to the command watchfor the message

   usage: watchfor username

is displayed and the command fails.


The break and continue statements

It is often necessary to handle exception conditions within loops. The statements break and continue are used for this.

The break command terminates the execution of the innermost enclosing loop, causing execution to resume after the nearest done statement.

To exit from n levels, use the command:

   break n

This will cause execution to resume after the done n levels up.

The continue command causes execution to resume at the while, until or for statement which begins the loop containing the continue command.

You can also specify an argument n|FR to continue which will cause execution to continue at the n|FRth enclosing loop up.

Example of using the break and continue statements

To prompt for commands to run:

   #!/bin/sh
   while echo "Please enter command"
   read response
   do
     case "$response" in
     'done') break           # no more commands
                       ;;
     "")     continue        # null command
                       ;;
     *)      eval $response  # do the command
                       ;;
     esac
   done

This prompts the user to enter a command. While they enter a command or null string the script continues to run. To stop the command the user enters done at the prompt.

Including text in a shell script

Text can be included in the shell script by using a here document, a special form of input redirection.

The << symbol is used to indicate that text should be read up to a given mark. For example:

   #!/bin/sh
   # this script outputs the given text before it runs
   cat << EOF
   This shellscript is currently under development, please
   report any problems to Danny (danny {at} cornflake.ed)
   EOF
   exec /usr/local/test/bin/test_version

The text is read from the script until a pattern is found which matches that after the << symbol; execution then proceeds as normal.

Forcing evaluation of commands

Another built-in function is eval which takes the arguments on the command line and executes them as a command. For example:

   #!/bin/sh
   echo "enter a command:"
   read command
   eval $command

Execute a command without creating a new process

The exec statement causes the command specified as its argument to be executed in place of the current shell without creating a new process. For example:

   exec zmail -visual

This runs just the zmail program without a shell. When you quit the application the current shell also exits.


Controlling when to exit a shell script

The exit statement will exit the current shell script. It can be given a numeric argument which is the script's exit status. If omitted the exit status of the last run command is used. 0 (zero) signifies success, non-zero signifies failure. For example:

   #!/bin/sh
   if [ $# -ne 2 ]
   # "$#" is number of parameters- here we test
   # whether it is not equal to two
   then
   echo "Usage $0 \<file1\> \<file2\>"        # not two parameters
   # so print message
   exit 2                                   # and fail ($0 is
   # name of command).
   fi
   ...<rest of script>

This script is supposed to take two positional arguments. It will exit with status 2 (error) rather than 0 (success) if it is not called with two parameters.

Trapping operating system signals

Shell procedures may use the trap command to catch or ignore Unix operating system signals. The form of the trap command is:

   trap 'command-list' signal-list

Several traps may be in effect at the same time. If multiple signals are received simultaneously, they are serviced in ascending order.

To check what traps are currently set use the trap command. For example:

   trap

Signals to be caught

The following are the signals that are usually caught with the trap command.

   0  shell exit (for any reason, including end of file EOF).
   1  hangup.
   2  interrupt (^C).
   3  quit (^\\ ; causes program to produce a core dump).
   9  kill (cannot be caught or ignored).
   15 terminate; default signal generated by kill.

trap: Handling command lists

The command list is placed between single quotes, as the command line is scanned twice, once when the shell first encounters the trap command and again when it is being executed.

   trap 'command-list' signal-list

The single quotes inhibit immediate command and variable substitution but are stripped off after the first scan, so that the commands are processed when the command is actually executed.

If command-list is not specified, then the action taken on receipt of any signal in the signal-list is reset to the default system action.

If command-list is an explicitly quoted null command (' ' or " "), then the signals in signal-list are ignored by the shell.

The command-list is treated like a subroutine call. The commands in the list are executed when the signal is trapped and control is then returned to the place at which it was interrupted.

Examples of interrupt handling

To use single quotes to inhibit command substitution:

   #!/bin/sh
   trap 'echo `pwd` >>$HOME/errdir' 2 3 15
   for i in /bin /usr/bin /usr/any/bin
   do
   cd $i
   some series of commands in the directory $i
   done

The file errdir will contain the name of the directory being worked on when the procedure is interrupted. What happens if the same procedure has double quotes around it?

   trap "echo `pwd` >errdir" 2 3 15 

The file errdir will just contain the name of the directory from which the procedure was invoked because the pwd command would be substituted on the first scan by the shell and not when it is invoked in the script.


To remove temporary files when a procedure is interrupted:

   #!/bin/sh
   temp=/tmp/file.$$
   trap 'rm $temp; exit' 0 1 2 3 15
   ls > $temp
   .....  

If any of the named signals are encountered, the command rm $temp; exit will be executed. The exit command is needed to terminate the execution of the whole procedure.


To continue processing commands after a trap command:

   #!/bin/sh
   # read and process commands
   dir=`pwd`
   for i in *
   do
     if test -d $dir/$i
     then
       cd $dir/$i
       while echo ''$i:''
       trap exit 2   # trap ^C
       read x
       do
         trap ' ' 2      # ignore interrupts
         eval $x
       done
     fi
   done

The shell continues to process commands after a trap command. The entire procedure is terminated if interrupted when waiting for input, but the interrupt is ignored while executing a command. The command list is an explicitly quoted null command and so the signal is ignored by the shell.

Debugging shell scripts

To see where a script produces an error use the command:

   sh -x script argument

The -x option to the sh command tells it to print commands and their arguments as they are executed.

You can then see what stage of the script has been reached when an error occurs.

Debugging options

You can use the following options either on the command line or with the built-in set command within a shell script to help you when debugging.

   -e in non-interactive mode, exit immediately
      if a command fails.
 
   -v print shell input lines as they are read.
 
   -n read commands but do not execute them.

Example of debugging a shell script

To print commands and their arguments as they are executed:

   cat example
   #!/bin/sh
   TEST1=result1
   TEST2=result2
   if [ $TEST1 = "result2" ]
   then
     echo $TEST1
   fi
   if [ $TEST1 = "result1" ]
   then
     echo $TEST1
   fi
   if [ $test3 = "whosit" ]
   then
     echo fail here cos it's wrong
   fi

This is a script called example which has an error in it; the variable $test3 is not set so the 3rd and last test [command will fail.

Running the script produces:

   example
   result1
   [: argument expected

The script fails and to see where the error occurred you would use the -x option like this:

   sh -x example
   TEST1=result1
   TEST2=result2
   + [ result1 = result2 ]
   + [ result1 = result1 ]
   + echo result1
   result1
   + [ = whosit ]
   example: [: argument expected

The error occurs in the command [ = whosit ] which is wrong as the variable $test3 has not been set. You can now see where to fix it.

Getting further information

You can get more information from the manual page for the Bourne shell (sh).

To get information about shell programming with another shell, read the manual page for that shell. For example:

   man ksh

This will display the manual page for the Korn shell.

There are a number of books on the different shells and how to program them. Many will be stocked in the computer section of most major booksellers. Or they may be available through your library