Scripting - bash etc

Books, articles etc

Books

Tutorials

Bash script elements

Bash built-in variables

#! /bin/sh
echo '$#' $#
echo '$@' $@
echo '$?' $?

# If you run the above script as:
# > ./test.sh 1 2 3
# You get output:
$# 3
$@ 1 2 3
$? 0

# You passed 3 parameters to your script.

$# = number of arguments. Answer is 3
$@ = what parameters were passed. Answer is 1 2 3
$? = was last command successful. Answer is 0 which means 'yes'

If-else in Bash

if TEST-COMMAND1
then
  STATEMENTS1
elif TEST-COMMAND2
then
  STATEMENTS2
else
  STATEMENTS3
fi

Comparisons

#!/bin/bash
if [[ $VAR1 -ge $VAR2 ]] && [[ $VAR1 -ge $VAR3 ]]
then
  echo "$VAR1 is the largest number."
elif [[ $VAR2 -ge $VAR1 ]] && [[ $VAR2 -ge $VAR3 ]]
then
  echo "$VAR2 is the largest number."
else
  echo "$VAR3 is the largest number."
fi

if-else operators

    -n VAR - True if the length of VAR is greater than zero.
    -z VAR - True if the VAR is empty.
    STRING1 = STRING2 - True if STRING1 and STRING2 are equal.
    STRING1 != STRING2 - True if STRING1 and STRING2 are not equal.
    INTEGER1 -eq INTEGER2 - True if INTEGER1 and INTEGER2 are equal.
    INTEGER1 -gt INTEGER2 - True if INTEGER1 is greater than INTEGER2.
    INTEGER1 -lt INTEGER2 - True if INTEGER1 is less than INTEGER2.
    INTEGER1 -ge INTEGER2 - True if INTEGER1 is equal or greater than INTEGER2.
    INTEGER1 -le INTEGER2 - True if INTEGER1 is equal or less than INTEGER2.
    -h FILE - True if the FILE exists and is a symbolic link.
    -r FILE - True if the FILE exists and is readable.
    -w FILE - True if the FILE exists and is writable.
    -x FILE - True if the FILE exists and is executable.
    -d FILE - True if the FILE exists and is a directory.
    -e FILE - True if the FILE exists and is a file, regardless of type (node, directory, socket, etc.).
    -f FILE - True if the FILE exists and is a regular file (not a directory or device).

Loops

# Listing files in a directory
for f in *; do
  echo "file name is: $f"
done

Also some nice tutorials about different kinds of loops/situations

# Example of looping over array values
values=(John Harry Jake Scott Philis)
for value in ${values[@]}; do
  echo "Current array value: ${value}"
done

Arrays

  echo ${#arr[@]}
  echo ${arr[${#arr[@]}-1]}
# NB - no spaces around "=" :)
array[$i]="abc"

SED

Streaming editor suited to modifying a per-line stream of text from a file or input string. GNU SED Manual

# pipe the string into the sed command (nb - works in ksh, bash or other shells)
# (could also use sed ... <<< "some string" but only in BASH)
# Remove the string foggy from the original string
echo 'foggy light' | sed 's/foggy//g'

# Simple example of replacing , by ;
sed 's/,/;/g' <<< "A,B,C"
sed 's/,/;/g' somefile.txt

# If you want to remove all the blocks of " -macro <WORD> "

$ sed 's/\s-macro \S*//g' <<< "cf foo -J 12345 -z -macro TEST_IFDEFINE -macro THIS -macro THIS1 -macro THIS2"  | cat -vet -
cf foo -J 12345 -z$

# \s-macro \S* matches space + -macro + space + a word.
# sed 's/something//g' removes this something as many times as it occurs in the string.

# Find numbers in form -123.45 and convert to (123.45)
sed 's/-\([0-9.]\+\)/(\1)/g' inputfile

# Find a number preceeded by a , followed by ,null] and delete it from the string (//g)
no_nulls=$(echo "$original_string" | sed 's/\(,[0-9]\+,null]\)//g' )

# Editing the line after the matching one - find a line containing FIXME and then on the next line (n;) add a prefix blah__ to the first character found.
sed -i '/FIXME.*/{n;s/\([a-z,A-Z]\)/blah__\0/;}' myfile.txt


# https://stackoverflow.com/questions/26568952/how-to-replace-multiple-patterns-at-once-with-sed
# Here what's interesting is more the "&" to output the captured match which helped where \1 wasn't good.
# This example adds a \n before each match of AND, GROUP BY etc.
sed -e 's:AND:\n&:g' -e 's:GROUP BY:\n&:g' -e 's:UNION:\n&:g' -e 's:FROM:\n&:g' file

Shell Parameter Expansions

Can be used as an alternative to calling out to sed/awk

# Remove the delimeter $del from the original_string to the end (based on the * for the pattern match)
"${original_string%%"$del"*}"

Bash script snippets

Checking logged in user

# could also check $USER variable
if [ "$(whoami)" != "username" ]; then
    echo "Script must be run as user: username"
    exit -1
fi

Using script Source directory for path to have relative paths to resource files etc

# cd into the bundle and use relative paths for resource files within the script
if [[ $BASH_SOURCE = */* ]]; then
    cd -- "${BASH_SOURCE%/*}/" || exit
fi

Here docs - ssh and then run commands from a script

Example of running a here doc with the ssh command to launch commands remotely

ssh ${connection_string} <<'ENDSSH'
 cd \~
 ls -la ENDSSH

Printing \n in scripts

echo prints literally, use printf instead

printf "hello\nworld\n"

Source vs ./

When running a script can do either "source" or ./

In most cases this is completely the same (source is a synonym of .) but there are some differences relating to source reusing the existing shell and ./ starting a sub-shell. This could have a side-effect for example if you "cd " inside the script being executed.

Stuff for the reporting script

Reporting

Proposing a list of simulations to choose from