Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases now! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Linux Shell Scripting Essentials
Linux Shell Scripting Essentials

Linux Shell Scripting Essentials: Learn shell scripting to solve complex shell-related problems and to efficiently automate your day-to-day tasks

eBook
€22.99 €32.99
Paperback
€41.99
Subscription
Free Trial
Renews at €18.99p/m

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Table of content icon View table of contents Preview book icon Preview Book

Linux Shell Scripting Essentials

Chapter 1. The Beginning of the Scripting Journey

Unix, Unix-like, or Linux-based OS provide a lot of powerful features to work upon. Among them, the most powerful and important feature is executing a wide range of commands to perform a task quickly and easily; for example, ls, cat, sort, grep, and so on. We will come to know about a subset of commands and usages throughout this book. In order to run a command, we need an interface that is widely known as shell.

Shell is a program that acts as an interface between the users (we) and the OS kernel (Linux, Unix, and so on). Understanding in terms of Windows OS, shell serves a similar purpose DOS does. Different shells are available for Unix, Unix-like, or Linux OS. Some of the popular shells are Bourne shell (sh), C shell (csh), Korn shell (ksh), Bourne Again shell (bash), and Z shell (zsh).

In this book, we will be using Linux OS and Bourne Again shell, popularly known by its acronym bash. Linux-based systems generally have bash already installed. In case bash is not installed, try installing the bash package from your distribution's package manager. In order to know which shell currently your Linux console is using, run the following command in terminal:

$ ps -p $$

The output is as follows:

  PID TTY          TIME CMD
12578 pts/4    00:00:00 bash

In the preceding ouput, we see that the CMD column has value bash. This means, we are currently using bash shell in our current console.

If your console is not using the bash shell, then you can run the following command:

$ bash

Also, your shell will be bash now. To make bash as a default login shell, run the following command:

$ chsh -s /bin/bash

The output obtained is as follows:

Changing shell for user.
Password:******
Shell changed.

We are now set with bash shell and ready to learn shell scripting in detail. Shell scripts are nothing but plain text files with a series of commands that are run by bash in a specified order. Writing shell scripts is very useful when you have to perform a series of tasks by running various commands, as bash will read each line from a script file and run it without any need of user intervention. The general file extension used for shell scripts are .sh, .bash, .zsh, .ksh, and so on. Rather than using a file extension for shell scripts, it's preferred to keep a filename without extension and let an interpreter identify the type by looking into shebang (#!). Shebang is used in scripts to indicate an interpreter for execution. It is written in the first line of a script file, for example:

#! /bin/bash

It means use the bash shell to execute a given script. To run a shell script, make sure it has execute permission. To provide execute permission to an owner of a file, run the following command:

$ chmod u+x foo

Here, foo is the shell script file. After running this command, foo will have execute permission for the owner of the file.

Now, we are ready to proceed further on learning shell scripting concepts in detail. Each topic and subtopic covered in the chapters with examples will lead us progressively towards a good shell script programmer.

In this chapter, we will talk broadly about the following topics:

  • Hello World in shell
  • Define variables of choice
  • Builtin shell variables
  • Operators
  • Shell expansions
  • Construct commands using eval
  • Make bash behave using set

Hello World in shell

Whenever we learn a new programming language, we first learn how to write the Hello World program in it. It is the best way to know and interact with a new language. This also helps in confirming that the basic environment for a program in a given language has been set up and you are good to dive deep inside this language.

Interacting with shell

We can print the output of commands in console in an interactive way. Console is also known as a standard input and output stream. To print anything in a bash console, use the echo command followed by what is to be printed:

$ echo Hello World
Hello World

Alternatively, put the text to be printed in double quotes:

$  echo "Hello World"
Hello World

You can also put the text to be printed in single quotes:

$ echo 'Hello World'
Hello World

We can also use the printf command in shell programming for printing. The printf command also supports formatted printing, similar to what we have in C programming language— the printf( ) function:

$ printf "Hello World"
Hello World$

Here, after the output, we see the command prompt ($) because printf doesn't add a default newline after execution while echo does. So, we have to explicitly add the newline (\n) in the printf statement to add a newline:

$ printf "Hello World\n"
Hello World

Similar to the C printf( ), we can specify formatted printing in bash. The syntax of bash printf is as follows:

printf FORMAT [ARGUMENTS]

FORMAT is a string that describes the format specifications and is specified within double quotes. ARGUMENTS can be the value or a variable corresponding to format specification. Format specification consists of the percentage (%) sign followed by format specifier. Format specifiers are explained in the following table:

Format specification

Description

%u

This prints an unsigned integer value

%i or %d

This prints an associated argument as a signed number

%f

This prints an associated argument as a floating point number

%o

This prints an unsigned octal value

%s

This prints a string value

%X

This prints an unsigned hexadecimal value (0 to 9 and A to F)

%x

This prints an unsigned hexadecimal value (0 to 9 and a to f)

The following examples demonstrate how to use format specification for printing different data type format in shell:

$ printf "%d mul %f = %f\n" 6 6.0 36.0
6 mul 6.000000 = 36.000000
$ printf "%s Scripting\n" Shell
Shell Scripting

We can also optionally specify a modifier in format specification to align an output to provide better formatting to the output. Format modifiers are placed between % and the format specifier character. The following table explains format modifiers:

Format Modifiers

Description

N

This is any number that specifies a minimum field width.

.

This is used together with field width. The field doesn't expand when the text is longer.

-

This is the left-bound text printing in the field.

0

This is used to fill padding with zeros (0) instead of whitespaces. By default, padding is done with whitespaces.

The following example demonstrates how to use format modifiers to improve printing formatting:

$ printf "%d mul %.2f = %.2f\n" 6 6.0 36.0
6 mul 6.00 = 36.00

Let's make it scripted

Interactive printing is good if we have to print one or two lines, but for a lot of printing, it's good and preferred to write a script file. A script file will contain all the instructions and we can run a script file to perform the needed task.

Now, we are going to create a bash script file that makes use of the echo and printf commands and print messages:

#!/bin/bash
#Filename: print.sh
#Description: print and echo

echo "Basic mathematics"
printf "%-7d %-7s %-7.2f =\t%-7.2f\n" 23 plus 5.5 28.5
printf "%-7.2f %-7s %-7d =\t%-7.2f\n" 50.50 minus 20 30.50 
printf "%-7d %-7s %-7d =\t%-7d\n" 10 mul 5 50
printf "%-7d %-7s %-7d =\t%-7.2f\n" 27 div 4 6.75

The first line in bash script represents the path of the interpreter used. The second line is a comment line telling the filename of a script file. In shell script, we use # to add a comment. Furthermore, the echo command will print strings written within double quotes. For the rest, we have used printf to print formatted output.

To run this script, we will first provide execute permission to a user/owner of this script:

$ chmod u+x print.sh

Then, run the script file in console as follows:

$ ./print.sh

The result after running this script will look as follows:

Let's make it scripted

Define variables of choice

Now we know how to write a simple hello world shell script. Next, we will be getting familiar with variables in shell and how to define and use variables in shell.

Nomenclature

A variable name can be a combination of alphanumeric and underscore. Also, the name of the variable can't start with a number. The variable names in shell script are case-sensitive. Special characters, such as *, -, +, ~, ., ^, and so on, are not used in variable names because they have a special meaning in shell. The following table illustrates the correct and incorrect ways of naming a variable:

Correct variable names

Incorrect variable names

variable

2_variable

variable1

2variable

variable_2

variable$

_variable3

variable*^

Assigning a value

We can assign a value to a variable by using an assignment (=) operator and followed by a value. While assigning a variable value, there shouldn't be any space before and after the assignment operator. Also, a variable can't be declared alone; it has to be followed by its initial value assignment:

$ book="Linux Shell Scripting"  # Stores string value$ book = "Linux Shell Scripting"  # Wrong, spaces around = operator
$ total_chapters=8    # Stores integer value
$ number_of_pages=210    # Stores integer value
$ average_pages_per_chapter=26.25    # Stores float value

So, it's quite easy to declare and assign a value to a variable in shell script. You don't have to worry about the data type of a variable on the left-hand side. Whatever value you provide on the right-hand side, the variable stores that value.

Tip

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Accessing a value

To access a variable value, use a dollar sign ($) operator followed by a variable name:

#!/bin/bash
#Filename: variables.sh
#Description: Basic variable definition and accessing them

book="Linux Shell Scripting"
total_chapters=8
number_of_pages=210
average_pages_per_chapter=26.25

echo "Book name - $book"
echo "Number of Chapters - $total_chapters"
printf "Total number of pages in book - $number_of_pages\n"
printf "Average pages in each chapter - %-.2f\n" $average_pages_per_chapter

The result of this script will look as follows:

Book name - Linux Shell Scripting
Number of Chapters - 8
Total number of pages in book - 210
Average pages in each chapter – 26.25

We can remove the value of a variable using the unset keyword in bash. Using unset to a variable deletes and resets it to null:

#!/bin/bash
#Filename: unset.sh
#Description: removing value of a variable

fruit="Apple"
quantity=6
echo "Fruit = $fruit , Quantity = $quantity"
unset fruit
echo "Fruit = $fruit , Quantity = $quantity"

The result after running this script will look as follows:

Fruit = Apple , Quantity = 6
Fruit =  , Quantity = 6

It's clear that we used unset on a fruit variable, so when we try to access a variable fruit after unsetting it in line no. 8, it prints nothing. The quantity variable still retains its value because we haven't used unset on it.

Constant variables

We can also create the constant variable in bash whose value can't be changed. The readonly keyword is used to declare a constant variable. We can also use declare -r followed by a variable name to make it constant:

#!/bin/bash
#Filename: constant.sh
#Description: constant variables in shell

readonly text="Welcome to Linux Shell Scripting"
echo $text
declare -r number=27
echo $number
text="Welcome"

The result after running this script will look as follows:

Welcome to Linux Shell Scripting
27
constant.sh: line 9: text: readonly variable

From the error message, it's clear that we can't change the value of a constant variable, and also we can't unset the value of the constant variable.

Reading variables from a user input

We can ask the user to provide input using the read shell built in command. The number of inputs to be given by a user is equivalent to the number of arguments provided to read. The value inserted by a user is stored in respective parameters passed to read. All parameters act as variables in which the corresponding user input value is stored.

The syntax of read is as follows:

read [options] var1 var2  … varN

If no variable in an argument is specified, the input value by a user will be stored in the inbuilt variable REPLY and can be accessed further using $REPLY.

We can read a user input in its input variable as follows:

$ read
    Hello World
$ echo $REPLY
    Hello World

We can read a value from user input as follows:

$ read text
    Hello
$ echo $text
    Hello

We can read multiple values from user input as follows:

$ read name usn marks
    Foo 345 78
$ echo $name $usn $marks
    Foo 345 78

We can read only the n characters and don't wait for the user to input a complete line as follows:

$ read -n 5    # option -n number takes only 5 characters from user input
    Hello$
$ echo $REPLY
    Hello

We can prompt the user a message before reading user input as follows:

$ read -p "What is your name?"    # -p allows to prompt user a message
    What is your name?Foo
$  echo $REPLY
    Foo

Hiding an input character when reading in console:

$  read -s -p "Enter your secret key:"  # -s doesn't echo input in console
Enter your secret key:$    #Pressing enter key brings command prompt $
echo $REPLY
foo

The following example shows the read command's usage:

#!/bin/bash
#Filename: read.sh
#Description: Find a file in a path entered by user

read -p "Enter filename to be searched:"
filename=$REPLY
read -p "Enter path for search:" path
echo "File $filename search matches to"
find $path -name $filename

The following is the result of running the read.sh script in bash:

Enter filename to be searched:read
Enter path for search:/usr/bin
File read search matches to
/usr/bin/read

Here, the find command has been used to search for the filename in the specified path. The detailed discussion of the command find will be done in Chapter 6, Working with Files.

Builtin shell variables

Builtin shell variables are predefined and are global variables that we can use in our script at any point of time. These are reserved shell variables and some of them may have a default value assigned by bash. Some variables' value will depend upon your current shell environment setup. The different type of shell may have a few specific reserved variables to it. All builtin shell variables' name will be in uppercase.

A few reserved shell variables available in bash shell are as follows:

Shell variables available in bash

Description

BASH

This is the absolute path of the current bash being invoked

BASH_VERSION

This is the version number of bash

BASHPID

This is the process ID of the current bash process

EUID

This is the effective user ID of the current user, which is assigned during startup

HOME

This is the current user's home directory

HOSTNAME

This is the name of the current host

PATH

This is the colon-separated list of directories where shell will look for commands

PPID

This is the process ID of the shell's parent

PWD

This is the present working directory

More shell variables can be found in man bash.

We will see what values these shell variables contain by printing its value in a shell script:

#!/bin/bash
#Filename: builtin_shell_variables.sh
#Description: Knowing about builtin shell variables

echo "My current bash path - $BASH"
echo "Bash version I am using - $BASH_VERSION"
echo "PID of bash I am running - $BASHPID"
echo "My home directory - $HOME"
echo "Where am I currently? - $PWD"
echo "My hostname - $HOSTNAME"

After running this script, the output may vary depending upon what the value of these variables is set in your system. The sample output will be as follows:

My current bash path - /bin/sh
Bash version I am using – 4.3.33(1)-release
PID of bash I am running - 4549
My home directory - /home/sinny
Where am I currently? - /home/sinny/Documents/
My hostname – localhost.localdomain

The shell variables, such as PWD, PATH, HOME, and so on, are very useful and help in getting the information quickly by just echoing a value in it. We can also add or modify the value of some of shell variables, such as PATH, in order to add a custom path in which we want shell to look for commands.

One of the use-cases of modifying the PATH variable value is: suppose, I have compiled a source code that generates a few binaries such as, foo and bar. Now, if I want shell to search in that particular directory for command as well, then add this directory path in the PATH variable and we are done. The following small shell script example shows how to do this:

#!/bin/bash
#Filename: path_variable.sh
#Description: Playing with PATH variable

echo "Current PATH variable content - $PATH"
echo "Current directory - $PWD"
echo "Content of current directory\n`ls`"
PATH=$PATH:$PWD
echo "New PATH variable content - $PATH"
# Now execute commands available in current working diectory

The output after running this script will be somewhat as follows:

Current PATH variable content - /usr/lib64/qt-3.3/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/home/sinny/go/source_code/go/bin:/home/sinny/.local/bin:/home/sinny/bin
Current directory - /home/sinny/test_project/bin
Content of current directory – foo bar
New PATH variable content - /usr/lib64/qt-/usr/lib64/qt-3.3/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/home/sinny/go/source_code/go/bin:/home/sinny/.local/bin:/home/sinny/bin: /home/sinny/test_project/bin

We see from the output that a new PATH variable has my custom path added. From the next time, whenever I run the foo or bar commands with this custom PATH variable set, the absolute path of the foo and the bar command/binary won't be required. Shell will find out these variables by looking into its PATH variable. This is true only during the current session of shell. We will see this in Chapter 5, Customizing Environment in recipe, Modifying a shell environment.

Operators

Similar to other programming languages, shell programming also supports various types of operators to perform tasks. Operators can be categorized as follows:

  • Assignment operator
  • Arithmetic operators
  • Logical operators
  • Comparison operators

The assignment operator

Equal to an operator (=) is the assignment operator that is used to initialize or change the value of a variable. This operator works on any data such as a string, integer, float, array, and so on. For example:

$ var=40           # Initializing variable var to integer value
$ var="Hello"    # Changing value of var to string value
$ var=8.9        # Changing value of var to float value

Arithmetic operators

Arithmetic operators are used for doing arithmetic operations on integers. They are as follows:

  • + (plus)
  • - (minus)
  • * (multiplication)
  • / (division)
  • ** (exponentiation)
  • % (modulo)
  • += (plus-equal)
  • -= (minus-equal)
  • *= (multiplication-equal)
  • /= (slash-equal)
  • %= (mod-equal)

To perform any arithmetic operation, we prefix the expr and let keywords before the actual arithmetic expression. The following example shows how to perform an arithmetic operation in bash:

#!/bin/bash
#Filename: arithmetic.sh
#Description: Arithmetic evaluation

num1=10 num2=5
echo "Numbers are num1 = $num1 and num2 = $num2"
echo "Addition = `expr $num1 + $num2`"`"
echo "Subtraction = `expr $num1 - $num2`"
echo "Multiplication = `expr $num1 \* $num2`"
echo "Division = `expr $num1 / $num2`"
let "exponent = $num1 ** num2"
echo "Exponentiation = $exponent" 
echo "Modulo = `expr $num1 % $num2`"
let "num1 += $num2"
echo "New num1 = $num1"
let "num1 -= $num1"
echo "New num2 = $num2"

The result after running this script will look as follows:

Numbers are num1 = 10 and num2 = 5
Addition = 15
Subtraction = 5
Multiplication = 50
Division = 2
Exponentiation = 100000
Modulo = 0
New num1 = 15
New num2 = 5

Logical operators

Logical operators are also known as Boolean operators. They are:

! (NOT), && (AND), and || (OR)

Performing a logical operation returns a Boolean value as true (1) or false (0) depending upon the values of variable(s) on which the operation is done.

One of the useful use-case is: suppose that we want to execute a command if the first command or operation returns successfully. In this case, we can use the && operator. Similarly, if we want to execute another command, irrespective of the first command that got executed or not, then we can use the || operator between two commands. We can use the ! operator to negate the true value. For example:

$ cd ~/Documents/ && ls

The cd command is used to change the current path to the specified argument. Here, the cd ~/Documents/ command will change the directory to Documents if exists. If it fails, then ls won't get executed, but if cd to Documents succeeds, the ls command will display the content of Documents directory:

$ cat ~/file.txt  || echo "Current Working directory $PWD"
cat: /home/skumari/file.txt: No such file or directory
Current Working directory /tmp/

The cat command displays the content of file.txt if it exists. Irrespective of the cat ~/file.txt command execution, later the command that is echo "Current Working directory $PWD" will be executed:

$  ! cd /tmp/foo && mkdir /tmp/foo
bash: cd: /tmp/foo: No such file or directory

By running the preceding commands, first it will try to change the directory to /tmp/foo. Here, ! cd /tmp/foo means if change directory to /tmp/foo doesn't succeed, then run the second command, which is mkdir /tmp/foo. The mkdir command is used to create a new directory. As a result of proceeding command execution, directory /tmp/foo will be created if it doesn't exist.

$ cd /tmp/foo

Since the /tmp/foo directory has been created, a successful change of the directory will occur.

Comparison operators

Comparison operators compare two variables and check whether a condition is satisfied or not. They are different for integers and strings.

Comparison operators that are valid for integer variables (consider a and b as two integer variables; for example, a=20, b=35) are as follows:

  • -eq (is equal to) - [ $a -eq $b ]
  • -ne (is not equal to) - [ $a -ne $b ]
  • -gt (is greater than) - [ $a -gt $b ]
  • -ge or >= (is greater than or equal to) - [ $a -ge $b ]
  • -lt (is less than) - [ $a -lt $b ]
  • -le (is less than or equal to) - [ $a -le $b ]
  • < (is less than) - (($a < $b))
  • <= (is less than or equal to) - (($a <= $b))
  • > (is greater than) - (($a > $b))
  • >= (is greater than or equal to) - (($a >= $b))

Comparison operators that are valid for string variables (consider a and b as two string variables; for example, a="Hello" b="World") are as follows:

  • = (is equal to); for example, [ $a = $b ]
  • != (is not equal to); for example, [ $a != $b ]
  • < (is less than); for example, [ $a \< $b ] or [[ $a \< $b ]] or (( $a \< $b ))
  • > (is greater than); for example,[ $a \> $b ] or [[ $a > $b ]] or (( $a \> $b ))
  • -n (string is non-empty); for example,[ -n $a ]
  • -z (string has zero length or null); for example,[ -z $a ]

Shell uses the < and > operators for redirection, so it should be used with an escape (\) if used under [ … ]. Double parentheses, (( ... )) or [[ … ]], doesn't need an escape sequence. Using [[ … ]] also supports pattern matching.

We will see the usage and examples of operators in more detail in Chapter 3, Effective Script Writing.

Shell expansions

While working with shell, we perform a lot of similar and repetitive tasks. For example, in the current directory, there are 100 files but we are interested only in shell script whose file extension is .sh. We can execute following command to view only shell script files in current directory:

$ ls *.sh

This will show all the files ending with .sh. An interesting take away from here is the * wildcard. It means a match list of files whose name can be anything and that ends with .sh.

Shell expands all wildcard patterns. A list of the latest wildcard patterns are as follows:

  • ~ (Tilde)
  • * (Asterisk)
  • ? (Question mark)
  • [ ] (Square brackets)
  • { } (Curly brackets)

To explain shell expansion for different wildcards, we will create a test folder in our home directory using the mkdir command containing different files mentioned as follows:

$ mkdir  ~/test && cd ~/test
$ touch a ab foo bar hello moo foo.c bar.c moo.c hello.txt foo.txt bar.sh hello.sh moo.sh

The touch command creates an empty file if it doesn't exist. If a file exists, then the file timestamp changes:

$ ls
a  ab  bar  bar.c  bar.sh  foo  foo.c  foo.txt  hello  hello.sh  hello.txt  moo  moo.c  moo.sh

Running the preceding commands will create a test directory, and inside test directory creates files given as parameter to the touch command.

~ (Tilde)

~ (Tilde) gets expanded by bash when it is present at the beginning of an unquoted string. The expansion depends upon what tilde-prefix is used. Tilde prefixes are characters until the first unquoted (/) slash. Some of the bash expansions are as follows:

  • ~: This is the user's home directory; the value is set in the $HOME variable
  • ~user_name: This is the home directory of the user's user_name
  • ~user_name/file_name: This is the file/directory file_name in the user's user_name home directory
  • ~/file_name: This is the file/directory file_name in the home directory that is $HOME/file_name
  • ~+: This is the current working directory; the value is set in the $PWD variable
  • ~-: This is the old or last working directory; the value is set in the $OLDPWD variable
  • ~+/file_name: This is the file/directory file_name in the current directory that is $PWD/file_name
  • ~-/file_name: This is the file/directory file_name in the old/last working directory that is $OLDPWD/file_name

* (Asterisk)

It matches zero or more characters. Take a test directory as an example:

  • Display all files as follows:
    $ ls *
    a  ab  bar  bar.c  bar.sh  foo  foo.c  foo.txt  hello  hello.sh  hello.txt  moo  moo.c  moo.sh
    
  • Display the C source files as follows:
    $ ls *.c
    bar.c  foo.c  moo.c
    
  • Display files that have a in its name, as follows:
    $ ls *a*
    a  ab  bar  bar.c  bar.sh
    
  • Deleting files with an extension .txt as follows:
    $ rm *.txt
    $ ls
    a  ab  bar  bar.c  bar.sh  foo  foo.c  hello  hello.sh  moo  moo.c  moo.sh
    

? (Question mark)

It matches any single character: ? (single question mark will match a single character), ?? (double question mark matches any two characters), and so on. Take a test directory as an example:

$ touch a ab foo bar hello moo foo.c bar.c moo.c hello.txt foo.txt bar.sh hello.sh moo.sh

This will recreate files that were removed during the previous example, and also update the access and modification time of the existing files:

  • Get files whose name length is irrespective of what the extension file has:
    $ ls ??
    ab
    
  • Get files whose name length is 2 or 5:
    $ ls ?? ?????
    ab  bar.c  foo.c  hello  moo.c
    
  • Delete files whose name is four characters long:
    $ rm ????
    rm: cannot remove '????': No such file or directory
    This error is because there is no file name with 4 character
    
  • Move files to the /tmp directory whose name is at least three characters long:
    $ mv ???* /tmp
    $ ls
    a ab
    

We see only two files in the test directory because the rest of the files were of the length 3 or more.

[ ] (Square brackets)

Square brackets match any character from the characters mentioned inside the square brackets. Characters can be specified as a word or range.

A range of characters can be specified using - (hyphen). For example:

  • [a-c]: This matches a, b, or c
  • [a-z]: This matches any character from a to z
  • [A-Z]: This matches any character from A to Z
  • [0-9]: This matches any character from 0 to 9

Take a test directory as an example and recreate files in a test directory:

$ touch a ab foo bar hello moo foo.c bar.c moo.c hello.txt foo.txt bar.sh hello.sh moo.sh

Get files whose name starts with a, b, c, or d with the following command:

$ ls [a-d]*
a  ab  bar  bar.c  bar.sh

Get files whose name starts with any letter and ends with a letter o or h, with the following command:

$  ls [a-zA-Z]*[oh]
foo  hello  hello.sh  moo  moo.sh

Get files that have at least the letter o twice in its name, with the following command:

$ ls *[o]*[o]*
foo  foo.c  foo.txt  moo  moo.c  moo.sh

[!characters] (Exclamation mark) is used to match a character that is not part of a charter set mentioned inside square brackets.

Get files that don't have a number in its name, with the following command:

$  ls [!0-9]*
a  ab  bar  bar.c  bar.sh  foo  foo.c  foo.txt  hello  hello.sh  hello.txt  moo  moo.c  moo.sh

{ } (Curly brackets)

It creates multiple wildcard patterns to match. A brace expression may contain either a comma-separated list of strings, a range, or a single character.

A range can be specified by using the following:

  • {a..z}: This matches all the charterer from a to z
  • {0..6}: This matches numbers 0, 1, 2, 3, 4, 5 and 6

Take a test directory as an example and recreate files in the test directory:

$ touch a ab foo bar hello moo foo.c bar.c moo.c hello.txt foo.txt bar.sh hello.sh moo.sh

Get files that have the file extension .sh or .c, with the following command:

$ ls {*.sh,*.c}
bar.c  bar.sh  foo.c  hello.sh  moo.c  moo.sh

Copy bar.c to bar.html by using the following command:

$ cp bar{.c,.cpp}  # Expands to cp bar.c bar.cpp
$ ls bar.*
bar.c  bar.cpp  bar.sh

Print the number from 1 to 50 by using the following command:

$ echo {1..50}
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

Create 10 files that start with hello and has an extension .cpp:

$ touch hello{0..9}.cpp
$ ls *.cpp
hello0.cpp  hello1.cpp  hello2.cpp  hello3.cpp  hello4.cpp  hello5.cpp  hello6.cpp  hello7.cpp  hello8.cpp  hello9.cpp

To avoid shell expansion of a wildcard, use backslash (\) or write a string within a single quote (' ').

Construct commands using eval

The eval command is a shell builtin command used to construct a command by concatenating arguments passed to eval. A concatenated command is further executed by shell and returns a result. If no arguments are given to eval, it returns 0.

The syntax of the eval command is as follows:

eval [arg …]

The following example shows the expansion of a variable to the name of another variable using eval:

$ name=foo
$ foo="Welcome to foo world"
$ echo $name
foo
$ new_name='$'$name    #new_name just stores string value $foo
$ echo $new_name
$foo
$ eval new_name='$'$name  # eval processes $foo string into variable and  prints                 # foo variable value
Welcome to foo world

Another example where eval can be useful is as follows:

$ pipe="|"
$  df $pipe wc  # Will give error because 
df: '|': No such file or directory
df: 'wc': No such file or directory
$ eval df $pipe wc  # eval executes it as shell command
12      73     705

Here, the df command shows a system disk's usage:

A shell script showing the use of eval is as follows:
#!/bin/bash
#Filename: eval.sh
#Description: Evaluating string as a command using eval

cmd="ls /usr"
echo "Output of command $cmd -"
eval $cmd   #eval will treat content of cmd as shell command and execute it
cmd1="ls /usr | wc -l"
echo "Line count of /usr -"
eval $cmd1

expression="expr 2 + 4 \* 6"
echo "Value of $expression"
eval $expression

Running the script will give you the following result:

Output of command ls /usr -
avr  bin  games  include  lib  lib64  libexec  local  sbin  share  src  tmp
Line count of /usr -
12
Value of expr 2 + 4 \* 6
26

Make bash behave using set

The set command is a shell builtin command that is used to set and unset a value of the local variables in shell.

The syntax of using set is as follows:

 set [--abBCefhHkmnpPtuvx] [-o option] [arg …]

Some of the option values are allexport, braceexpand, history, keyword, verbose, and xtrace.

Using set without any option displays the name and value of all shell variables and functions, in a format that can be reused as an input for setting and unsetting the currently set variables.

Exit on the first failure

In a shell script, by default, the next line is executed if an error occurs in the current line. Sometimes, we may want to stop running a script further after an error has been encountered. The -e option of set ensures to exit a script once any of the commands in a pipeline fails.

In the following shell script, do_not_exit_on_failure.sh doesn't use set with the option -e:

$ cat do_not_exit_on_failure.sh
#!/bin/bash
# Filename: do_not_exit_on_failure.sh
# Description: Resume script after an error

echo "Before error"
cd /root/       # Will give error
echo "After error"

After running this script, the output is as follows:

Before error
do_not_exit_on_failure.sh: line 6: cd: /root/: Permission denied
After error

We see that the command after the error gets executed as well. In order to stop the execution after an error is encountered, use set -e in the script. The following script demonstrates the same:

$ cat exit_on_failure.sh
#!/bin/bash
# Filename: exit_on_failure.sh
# Description: Exits script after an error

set -e
echo "Before error"
cd /root/       # Will give error
echo "After error"

The output after running the preceding script is as follows:

Before error
exit_on_failure.sh: line 7: cd: /root/: Permission denied

We can see that the script has been terminated after encountering an error at the line number 7.

Enabling/disabling symbolic link's resolution path

Using set with the -P option doesn't resolve symbolic links. Following example demonstrate how we can enable or disable symbolic link resolution of /bin directory which is symbolic link of /usr/bin/ directory:

$ ls -l /bin
lrwxrwxrwx. 1 root root 7 Nov 18 18:03 /bin -> usr/bin
$ set –P    # -P enable symbolic link resolution
$ cd /bin
$ pwd
/usr/bin
$ set +P   # Disable symbolic link resolution
$ pwd
/bin

Setting/unsetting variables

We can use the set command to see all local variables accessible for the current process. The local variables are not accessible in the subprocess.

We can create our own variable and set it locally as follows:

$ MYVAR="Linux Shell Scripting"
$ echo $MYVAR
 Linux Shell Scripting
$ set | grep MYVAR  # MYVAR local variable is created
MYVAR='Linux Shell Scripting'
$ bash    # Creating a new bash sub-process in current bash
$ set | grep MYVAR
$    # Blank because MYVAR is local variable

To make a variable accessible to its subprocesses as well, use the export command followed by the variable to be exported:

$ MYVARIABLE="Hello World"
$ export  MYVARIABLE
$ bash    # Creating a new bash sub-process under bash
$ echo $MYVARIABLE
Hello World

This will export the MYVARIABLE variable to any subprocess that ran from that process. To check whether MYVARIABLE has exported or not, run the following command:

$ export |grep MYVARIABLE
declare -x MYVARIABLE="Hello World"
$ export | grep MYVAR
$MYVAR variable is not present in sub-process but variable MYVARIABLE is present in sub-process.

To unset local or exported variables, use the unset command and it will reset the value of the variable to null:

$ unset MYVAR        # Unsets local variable MYVAR
$ unset  MYVARIABLE    # Unsets exported variable MYVARIABLE

Summary

After reading this chapter, you understood how to write simple shell script in bash by printing, echoing, and asking user input. You should now have a good understanding of defining and using variables in shell and what builtin shell variables exist. You are now familiar with what operators are available in shell and how they can create and evaluate their own expression. With information about wildcards available in this chapter, it makes work easier for you while you are dealing with similar kind of data or pattern. The shell builtin command set enables modifying shell variables easily.

This chapter has built a foundation for upcoming chapters. Now, in next chapter, you will get to know about standard inputs, outputs, and errors. Also, there will be a detailed coverage of how to use an output from commands and then filter/transform them to show the data according to your need.

Left arrow icon Right arrow icon

Description

Shell scripting is a quick method to prototype complex applications or problems. Shell scripts are a collection of commands to automate tasks, usually those for which the user has a repeated need, when working on Linux-based systems. Using simple commands or a combination of them in a shell can solve complex problems easily. This book starts with the basics, including essential commands that can be executed on Linux systems to perform tasks within a few nanoseconds. You’ll learn to use outputs from commands and transform them to show the data you require. Discover how to write shell scripts easily, execute script files, debug, and handle errors. Next, you’ll explore environment variables in shell programming and learn how to customize them and add a new environment. Finally, the book walks you through processes and how these interact with your shell scripts, along with how to use scripts to automate tasks and how to embed other languages and execute them.

Who is this book for?

This book is aimed at administrators and those who have a basic knowledge of shell scripting and who want to learn how to get the most out of writing shell scripts.

What you will learn

  • Write effective shell scripts easily
  • Perform search operations and manipulate large text data with a single shell command
  • Modularize reusable shell scripts by creating shell libraries
  • Redirect input, output, and errors of a command or script execution to other streams
  • Debug code with different shell debugging techniques to make your scripts bugfree
  • Manage processes, along with the environment variables needed to execute them properly
  • Execute and embed other languages in your scripts
  • Manage creation, deletion, and search operations in files
Estimated delivery fee Deliver to Malta

Premium delivery 7 - 10 business days

€32.95
(Includes tracking information)

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Nov 23, 2015
Length: 282 pages
Edition : 1st
Language : English
ISBN-13 : 9781785284441
Tools :

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR
Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Estimated delivery fee Deliver to Malta

Premium delivery 7 - 10 business days

€32.95
(Includes tracking information)

Product Details

Publication date : Nov 23, 2015
Length: 282 pages
Edition : 1st
Language : English
ISBN-13 : 9781785284441
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
€18.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
€189.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts
€264.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total 113.97
Linux Shell Scripting Essentials
€41.99
Learning Linux Shell Scripting
€41.99
Mastering Linux Shell Scripting
€29.99
Total 113.97 Stars icon

Table of Contents

9 Chapters
1. The Beginning of the Scripting Journey Chevron down icon Chevron up icon
2. Getting Hands-on with I/O, Redirection Pipes, and Filters Chevron down icon Chevron up icon
3. Effective Script Writing Chevron down icon Chevron up icon
4. Modularizing and Debugging Chevron down icon Chevron up icon
5. Customizing the Environment Chevron down icon Chevron up icon
6. Working with Files Chevron down icon Chevron up icon
7. Welcome to the Processes Chevron down icon Chevron up icon
8. Scheduling Tasks and Embedding Languages in Scripts Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Rating distribution
Full star icon Full star icon Full star icon Full star icon Half star icon 4.5
(2 Ratings)
5 star 50%
4 star 50%
3 star 0%
2 star 0%
1 star 0%
Parminder Singh Bola May 08, 2016
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I love it!!!
Amazon Verified review Amazon
skeptic Jun 24, 2017
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
A short, but well written book. cover modern bash (including some features of version 4). It mentions bashdb which is a plus. Generally the content is well thought out and the text written well (within the limitation of the size of the book, which are pretty severe)There are some errors. For example putting aliases into .bashrc file is possible but technically incorrect - aliases work only in interactive sessions. Generally they more properly belong to .bash_profile file.The book should be priced much lower, as this is a pretty short book.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is the delivery time and cost of print book? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
What is custom duty/charge? Chevron down icon Chevron up icon

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order? Chevron down icon Chevron up icon

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges? Chevron down icon Chevron up icon

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

  • If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
  • Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.
How can I cancel my order? Chevron down icon Chevron up icon

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact customercare@packt.com with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at customercare@packt.com using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy? Chevron down icon Chevron up icon

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on customercare@packt.com with the order number and issue details as explained below:

  1. If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on customercare@packt.com within one hour of placing the order and we will replace/refund you the item cost.
  2. Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on customercare@packt.com who will be able to resolve this issue for you.
  3. You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
  4. Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
  5. If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
  6. Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on customercare@packt.com within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged? Chevron down icon Chevron up icon

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use? Chevron down icon Chevron up icon

You can pay with the following card types:

  1. Visa Debit
  2. Visa Credit
  3. MasterCard
  4. PayPal
What is the delivery time and cost of print books? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela