Shell Scripting Tips

In no particular order, here is a collection of tips to remember to help make shell scripts better. Some of these will apply more generally to other languages, especially other shells. I call them tips, because my research has been less than rigourous, particularly with regard to portability between shells/platforms.

Execution

Check that your script has valid syntax without executing it:

/bin/sh -n script.sh

Watch every command in your script as it is executed:

/bin/sh -x script.sh

$(command) is a better way to execute command than using backticks, if nothing else because it handles quoting and nesting.

Often, particularly on modern linux systems, executing with /bin/sh will get Bourne(-like?) behaviour from Bash (since /bin/sh itself is merely either a symlink to Bash or indeed the Bash executable itself, in contrast to using /bin/bash, which will get Bash semantics.

Quoting, Variables and Arguments

Always use the full syntax for referencing a variable, it's just safer:

${VAR}

Quoting the variable reference means it won't be broken by a space:

"${VAR}"

Arguments

"$@": all the arguments.
$#: the number of arguments.

Default values for variables

Use set -u to treat unset variables as an error.

Defaults are often useful. If $default is defined and not empty, set var to its value, otherwise set it to SOMEVALUE:

var=${default-SOMEVALUE}

The above is Bourne compliant, POSIXish shells like ksh and bash allow the following equivalent syntax:

var=${default:-SOMEVALUE}

getopts

Starting with the example:

while getopts "hvf:" OPTION;
do
  case "$OPTION" in
    h)
        usage
        exit
        ;;
    v)
        verbosity=$(($verbosity+1))
        ;;
    f)
        F_VARIABLE="$OPTARG"
        ;;
    *)
        die "Unrecognised Option"
        ;;
  esac
done

[[ $verbosity -gt 0 ]] && set -x
shift $((OPTIND - 1))

The : in the getopts options definition is magic, it means that the preceding option takes a value.
$OPTARG is magic, it's how getopts populates the option value into the variable that should hold it.
With the caveat that getopts in sh(1) only handles short options, use GNU getopt or a bigger scripting language if you need more than this, this handles the common use case.
set -x is a good start for verbosity, more verbose logging should probably use a log() function like die() below.
The final shift means that non-option arguments are now available in $@/$1,$2,..$N

Comparisons

-eq, -ne, gt, etc.: integer comparisons
=, !=, <, etc.: string comparisons
(()): arithmetic expressions, be careful with exit status, they are opposite from [ ... ] tests.

If you are using Bash, you probably want to prefer [[ ]] (double-bracket test operator) over [ ] (POSIX test operator, equivalent to /usb/bin/test, because you can use &&, ||, <, > and pattern matching inside [[ ]]. If you want filename expansion or word splitting, you must use the single-bracket test operator.

Reading input from terminal

Read from the terminal, putting the content into a variable, INPUT:

read -p "Input please: " INPUT
echo $INPUT

The -s flag turns off echoing to the terminal in case you want to ask for a password:

read -s -p "Password: " pw

Colouring output with tput(1)

Although in my opinion, colour output is often overused or badly used, it's sometimes helpful. Here's an example:

 NORMAL=$(tput sgr0)
 GREEN=$(tput setaf 2; tput bold)
 RED=$(tput setaf 1)

 function red() {
   $RED
   echo -e "$*"
   $NORMAL
 }

 function green() {
   $GREEN
   echo -e "$*"
   $NORMAL
}

 # To print success
 green "Successfully completed"

 # To print error
 red "FAILED"

For more/other examples, see http://code.google.com/p/bsfl/source/browse/trunk/functions.sh

Error Checking

Every command sets an exit code, which should be 0 if the commands completed successfully. The last one is in $?, so we can do basic error checking with:

[ "$?" -eq "0" ] || echo "Something wrong?" >&2

set -o errexit will cause the script (set -e) to exit immediately if a command or pipeline returns a non-true value.
set -o pipefail will cause the script (set -e) to exit immediately if any command in a pipeline returns a non-true value.

Error Reporting

This could be a lot more powerful, but it will do for simple needs:

die()
{
   msg=$1
   echo $msg >&2
   exit 1
}

An alternative:

err() {
      echo "[$(date +'%Y-%m-%dT%H:%M:%S%z')]: $@" >&2
}

A POSIX-compliant debug function:

debug() { [ "$DEBUG" ] && echo ">>> $*"; }

Logging

Here is an example of a log function and its usage that will create a log entry that resembles the usual style for unix services:

log() {  # standard logger
   local prefix="[$(date +%Y/%m/%d\ %H:%M:%S)]: "
   echo "${prefix} $@" >&2
}

log "WARNING" "something non-fatal happened"

Signal Handling with `trap`

Sometimes you'll want perform cleanup in the event of receiving SIGINT or SIGTERM (or the pseudo-signal EXIT which is triggered by the built-in called or "exit" or the failure of any command while set -e is active.

lockfiles are an example of this:

if [ ! -e $lockfile ]; then
   trap "rm -f $lockfile; exit" INT TERM EXIT
   touch $lockfile
   <important code that you want to isolate/prevent more than once simultaneously>
   rm $lockfile
   trap - INT TERM EXIT
else
   echo "should-be-unique code is already running"
fi

Portability

Some caveats concerning portability between shells/version/platforms:

most items here are likely to be sh(1) compatible (i.e. original Bourne shell).
some techniques will require bash(1) (GNU Bourne-Again shell).
I have made little attempt to note which versions of the shells support any particular feature or technique, although it is my expectation that all examples will likely work with anything even vaguely modern such GNU Bash version 3 or higher.
I have made no attempt whatsoever to try to account for different platforms (Linux distributions, BSD variants or anything more exotic), my usual environments are common linux distributions such as RHEL-derivatives, SuSE and FreeBSD.

sh (or Shell Command Language) is a programming language described by the POSIX standards, also known as the IEEE Std 1003 family. It has many implementations (ksh88, dash, ...). For example, look at The Open Group's Base Specifications Issue 6 otherwise known as IEEE 1003.1 2004 Edition.

bash can also be considered an implementation of sh. Bash started as sh-compatible, but there are many differences. It supports a POSIX mode --posix switch and tries to use POSIX compliant with either the POSIXLY_COMPLIANT environment variable or if it is invoked as sh.

Miscellaneous Goodies

if bash errors when you reference an undefined variable, set -o nounset, you'll likely catch certain logic errors

AllGoodBits.org