Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Linux Shell Scripting Essentials
Linux Shell Scripting Essentials

Linux Shell Scripting Essentials: Learn shell scripting to solve complex shell-related problems and to efficiently automate your day-to-day tasks

eBook
$29.99 $43.99
Paperback
$54.99
Subscription
Free Trial
Renews at $19.99p/m

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Table of content icon View table of contents Preview book icon Preview Book

Linux Shell Scripting Essentials

Chapter 1. The Beginning of the Scripting Journey

Unix, Unix-like, or Linux-based OS provide a lot of powerful features to work upon. Among them, the most powerful and important feature is executing a wide range of commands to perform a task quickly and easily; for example, ls, cat, sort, grep, and so on. We will come to know about a subset of commands and usages throughout this book. In order to run a command, we need an interface that is widely known as shell.

Shell is a program that acts as an interface between the users (we) and the OS kernel (Linux, Unix, and so on). Understanding in terms of Windows OS, shell serves a similar purpose DOS does. Different shells are available for Unix, Unix-like, or Linux OS. Some of the popular shells are Bourne shell (sh), C shell (csh), Korn shell (ksh), Bourne Again shell (bash), and Z shell (zsh).

In this book, we will be using Linux OS and Bourne Again shell, popularly known by its acronym bash. Linux-based systems generally have bash already installed. In case bash is not installed, try installing the bash package from your distribution's package manager. In order to know which shell currently your Linux console is using, run the following command in terminal:

$ ps -p $$

The output is as follows:

  PID TTY          TIME CMD
12578 pts/4    00:00:00 bash

In the preceding ouput, we see that the CMD column has value bash. This means, we are currently using bash shell in our current console.

If your console is not using the bash shell, then you can run the following command:

$ bash

Also, your shell will be bash now. To make bash as a default login shell, run the following command:

$ chsh -s /bin/bash

The output obtained is as follows:

Changing shell for user.
Password:******
Shell changed.

We are now set with bash shell and ready to learn shell scripting in detail. Shell scripts are nothing but plain text files with a series of commands that are run by bash in a specified order. Writing shell scripts is very useful when you have to perform a series of tasks by running various commands, as bash will read each line from a script file and run it without any need of user intervention. The general file extension used for shell scripts are .sh, .bash, .zsh, .ksh, and so on. Rather than using a file extension for shell scripts, it's preferred to keep a filename without extension and let an interpreter identify the type by looking into shebang (#!). Shebang is used in scripts to indicate an interpreter for execution. It is written in the first line of a script file, for example:

#! /bin/bash

It means use the bash shell to execute a given script. To run a shell script, make sure it has execute permission. To provide execute permission to an owner of a file, run the following command:

$ chmod u+x foo

Here, foo is the shell script file. After running this command, foo will have execute permission for the owner of the file.

Now, we are ready to proceed further on learning shell scripting concepts in detail. Each topic and subtopic covered in the chapters with examples will lead us progressively towards a good shell script programmer.

In this chapter, we will talk broadly about the following topics:

  • Hello World in shell
  • Define variables of choice
  • Builtin shell variables
  • Operators
  • Shell expansions
  • Construct commands using eval
  • Make bash behave using set

Hello World in shell

Whenever we learn a new programming language, we first learn how to write the Hello World program in it. It is the best way to know and interact with a new language. This also helps in confirming that the basic environment for a program in a given language has been set up and you are good to dive deep inside this language.

Interacting with shell

We can print the output of commands in console in an interactive way. Console is also known as a standard input and output stream. To print anything in a bash console, use the echo command followed by what is to be printed:

$ echo Hello World
Hello World

Alternatively, put the text to be printed in double quotes:

$  echo "Hello World"
Hello World

You can also put the text to be printed in single quotes:

$ echo 'Hello World'
Hello World

We can also use the printf command in shell programming for printing. The printf command also supports formatted printing, similar to what we have in C programming language— the printf( ) function:

$ printf "Hello World"
Hello World$

Here, after the output, we see the command prompt ($) because printf doesn't add a default newline after execution while echo does. So, we have to explicitly add the newline (\n) in the printf statement to add a newline:

$ printf "Hello World\n"
Hello World

Similar to the C printf( ), we can specify formatted printing in bash. The syntax of bash printf is as follows:

printf FORMAT [ARGUMENTS]

FORMAT is a string that describes the format specifications and is specified within double quotes. ARGUMENTS can be the value or a variable corresponding to format specification. Format specification consists of the percentage (%) sign followed by format specifier. Format specifiers are explained in the following table:

Format specification

Description

%u

This prints an unsigned integer value

%i or %d

This prints an associated argument as a signed number

%f

This prints an associated argument as a floating point number

%o

This prints an unsigned octal value

%s

This prints a string value

%X

This prints an unsigned hexadecimal value (0 to 9 and A to F)

%x

This prints an unsigned hexadecimal value (0 to 9 and a to f)

The following examples demonstrate how to use format specification for printing different data type format in shell:

$ printf "%d mul %f = %f\n" 6 6.0 36.0
6 mul 6.000000 = 36.000000
$ printf "%s Scripting\n" Shell
Shell Scripting

We can also optionally specify a modifier in format specification to align an output to provide better formatting to the output. Format modifiers are placed between % and the format specifier character. The following table explains format modifiers:

Format Modifiers

Description

N

This is any number that specifies a minimum field width.

.

This is used together with field width. The field doesn't expand when the text is longer.

-

This is the left-bound text printing in the field.

0

This is used to fill padding with zeros (0) instead of whitespaces. By default, padding is done with whitespaces.

The following example demonstrates how to use format modifiers to improve printing formatting:

$ printf "%d mul %.2f = %.2f\n" 6 6.0 36.0
6 mul 6.00 = 36.00

Let's make it scripted

Interactive printing is good if we have to print one or two lines, but for a lot of printing, it's good and preferred to write a script file. A script file will contain all the instructions and we can run a script file to perform the needed task.

Now, we are going to create a bash script file that makes use of the echo and printf commands and print messages:

#!/bin/bash
#Filename: print.sh
#Description: print and echo

echo "Basic mathematics"
printf "%-7d %-7s %-7.2f =\t%-7.2f\n" 23 plus 5.5 28.5
printf "%-7.2f %-7s %-7d =\t%-7.2f\n" 50.50 minus 20 30.50 
printf "%-7d %-7s %-7d =\t%-7d\n" 10 mul 5 50
printf "%-7d %-7s %-7d =\t%-7.2f\n" 27 div 4 6.75

The first line in bash script represents the path of the interpreter used. The second line is a comment line telling the filename of a script file. In shell script, we use # to add a comment. Furthermore, the echo command will print strings written within double quotes. For the rest, we have used printf to print formatted output.

To run this script, we will first provide execute permission to a user/owner of this script:

$ chmod u+x print.sh

Then, run the script file in console as follows:

$ ./print.sh

The result after running this script will look as follows:

Let's make it scripted

Define variables of choice

Now we know how to write a simple hello world shell script. Next, we will be getting familiar with variables in shell and how to define and use variables in shell.

Nomenclature

A variable name can be a combination of alphanumeric and underscore. Also, the name of the variable can't start with a number. The variable names in shell script are case-sensitive. Special characters, such as *, -, +, ~, ., ^, and so on, are not used in variable names because they have a special meaning in shell. The following table illustrates the correct and incorrect ways of naming a variable:

Correct variable names

Incorrect variable names

variable

2_variable

variable1

2variable

variable_2

variable$

_variable3

variable*^

Assigning a value

We can assign a value to a variable by using an assignment (=) operator and followed by a value. While assigning a variable value, there shouldn't be any space before and after the assignment operator. Also, a variable can't be declared alone; it has to be followed by its initial value assignment:

$ book="Linux Shell Scripting"  # Stores string value$ book = "Linux Shell Scripting"  # Wrong, spaces around = operator
$ total_chapters=8    # Stores integer value
$ number_of_pages=210    # Stores integer value
$ average_pages_per_chapter=26.25    # Stores float value

So, it's quite easy to declare and assign a value to a variable in shell script. You don't have to worry about the data type of a variable on the left-hand side. Whatever value you provide on the right-hand side, the variable stores that value.

Tip

Downloading the example code

You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Accessing a value

To access a variable value, use a dollar sign ($) operator followed by a variable name:

#!/bin/bash
#Filename: variables.sh
#Description: Basic variable definition and accessing them

book="Linux Shell Scripting"
total_chapters=8
number_of_pages=210
average_pages_per_chapter=26.25

echo "Book name - $book"
echo "Number of Chapters - $total_chapters"
printf "Total number of pages in book - $number_of_pages\n"
printf "Average pages in each chapter - %-.2f\n" $average_pages_per_chapter

The result of this script will look as follows:

Book name - Linux Shell Scripting
Number of Chapters - 8
Total number of pages in book - 210
Average pages in each chapter – 26.25

We can remove the value of a variable using the unset keyword in bash. Using unset to a variable deletes and resets it to null:

#!/bin/bash
#Filename: unset.sh
#Description: removing value of a variable

fruit="Apple"
quantity=6
echo "Fruit = $fruit , Quantity = $quantity"
unset fruit
echo "Fruit = $fruit , Quantity = $quantity"

The result after running this script will look as follows:

Fruit = Apple , Quantity = 6
Fruit =  , Quantity = 6

It's clear that we used unset on a fruit variable, so when we try to access a variable fruit after unsetting it in line no. 8, it prints nothing. The quantity variable still retains its value because we haven't used unset on it.

Constant variables

We can also create the constant variable in bash whose value can't be changed. The readonly keyword is used to declare a constant variable. We can also use declare -r followed by a variable name to make it constant:

#!/bin/bash
#Filename: constant.sh
#Description: constant variables in shell

readonly text="Welcome to Linux Shell Scripting"
echo $text
declare -r number=27
echo $number
text="Welcome"

The result after running this script will look as follows:

Welcome to Linux Shell Scripting
27
constant.sh: line 9: text: readonly variable

From the error message, it's clear that we can't change the value of a constant variable, and also we can't unset the value of the constant variable.

Reading variables from a user input

We can ask the user to provide input using the read shell built in command. The number of inputs to be given by a user is equivalent to the number of arguments provided to read. The value inserted by a user is stored in respective parameters passed to read. All parameters act as variables in which the corresponding user input value is stored.

The syntax of read is as follows:

read [options] var1 var2  … varN

If no variable in an argument is specified, the input value by a user will be stored in the inbuilt variable REPLY and can be accessed further using $REPLY.

We can read a user input in its input variable as follows:

$ read
    Hello World
$ echo $REPLY
    Hello World

We can read a value from user input as follows:

$ read text
    Hello
$ echo $text
    Hello

We can read multiple values from user input as follows:

$ read name usn marks
    Foo 345 78
$ echo $name $usn $marks
    Foo 345 78

We can read only the n characters and don't wait for the user to input a complete line as follows:

$ read -n 5    # option -n number takes only 5 characters from user input
    Hello$
$ echo $REPLY
    Hello

We can prompt the user a message before reading user input as follows:

$ read -p "What is your name?"    # -p allows to prompt user a message
    What is your name?Foo
$  echo $REPLY
    Foo

Hiding an input character when reading in console:

$  read -s -p "Enter your secret key:"  # -s doesn't echo input in console
Enter your secret key:$    #Pressing enter key brings command prompt $
echo $REPLY
foo

The following example shows the read command's usage:

#!/bin/bash
#Filename: read.sh
#Description: Find a file in a path entered by user

read -p "Enter filename to be searched:"
filename=$REPLY
read -p "Enter path for search:" path
echo "File $filename search matches to"
find $path -name $filename

The following is the result of running the read.sh script in bash:

Enter filename to be searched:read
Enter path for search:/usr/bin
File read search matches to
/usr/bin/read

Here, the find command has been used to search for the filename in the specified path. The detailed discussion of the command find will be done in Chapter 6, Working with Files.

Builtin shell variables

Builtin shell variables are predefined and are global variables that we can use in our script at any point of time. These are reserved shell variables and some of them may have a default value assigned by bash. Some variables' value will depend upon your current shell environment setup. The different type of shell may have a few specific reserved variables to it. All builtin shell variables' name will be in uppercase.

A few reserved shell variables available in bash shell are as follows:

Shell variables available in bash

Description

BASH

This is the absolute path of the current bash being invoked

BASH_VERSION

This is the version number of bash

BASHPID

This is the process ID of the current bash process

EUID

This is the effective user ID of the current user, which is assigned during startup

HOME

This is the current user's home directory

HOSTNAME

This is the name of the current host

PATH

This is the colon-separated list of directories where shell will look for commands

PPID

This is the process ID of the shell's parent

PWD

This is the present working directory

More shell variables can be found in man bash.

We will see what values these shell variables contain by printing its value in a shell script:

#!/bin/bash
#Filename: builtin_shell_variables.sh
#Description: Knowing about builtin shell variables

echo "My current bash path - $BASH"
echo "Bash version I am using - $BASH_VERSION"
echo "PID of bash I am running - $BASHPID"
echo "My home directory - $HOME"
echo "Where am I currently? - $PWD"
echo "My hostname - $HOSTNAME"

After running this script, the output may vary depending upon what the value of these variables is set in your system. The sample output will be as follows:

My current bash path - /bin/sh
Bash version I am using – 4.3.33(1)-release
PID of bash I am running - 4549
My home directory - /home/sinny
Where am I currently? - /home/sinny/Documents/
My hostname – localhost.localdomain

The shell variables, such as PWD, PATH, HOME, and so on, are very useful and help in getting the information quickly by just echoing a value in it. We can also add or modify the value of some of shell variables, such as PATH, in order to add a custom path in which we want shell to look for commands.

One of the use-cases of modifying the PATH variable value is: suppose, I have compiled a source code that generates a few binaries such as, foo and bar. Now, if I want shell to search in that particular directory for command as well, then add this directory path in the PATH variable and we are done. The following small shell script example shows how to do this:

#!/bin/bash
#Filename: path_variable.sh
#Description: Playing with PATH variable

echo "Current PATH variable content - $PATH"
echo "Current directory - $PWD"
echo "Content of current directory\n`ls`"
PATH=$PATH:$PWD
echo "New PATH variable content - $PATH"
# Now execute commands available in current working diectory

The output after running this script will be somewhat as follows:

Current PATH variable content - /usr/lib64/qt-3.3/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/home/sinny/go/source_code/go/bin:/home/sinny/.local/bin:/home/sinny/bin
Current directory - /home/sinny/test_project/bin
Content of current directory – foo bar
New PATH variable content - /usr/lib64/qt-/usr/lib64/qt-3.3/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/home/sinny/go/source_code/go/bin:/home/sinny/.local/bin:/home/sinny/bin: /home/sinny/test_project/bin

We see from the output that a new PATH variable has my custom path added. From the next time, whenever I run the foo or bar commands with this custom PATH variable set, the absolute path of the foo and the bar command/binary won't be required. Shell will find out these variables by looking into its PATH variable. This is true only during the current session of shell. We will see this in Chapter 5, Customizing Environment in recipe, Modifying a shell environment.

Operators

Similar to other programming languages, shell programming also supports various types of operators to perform tasks. Operators can be categorized as follows:

  • Assignment operator
  • Arithmetic operators
  • Logical operators
  • Comparison operators

The assignment operator

Equal to an operator (=) is the assignment operator that is used to initialize or change the value of a variable. This operator works on any data such as a string, integer, float, array, and so on. For example:

$ var=40           # Initializing variable var to integer value
$ var="Hello"    # Changing value of var to string value
$ var=8.9        # Changing value of var to float value

Arithmetic operators

Arithmetic operators are used for doing arithmetic operations on integers. They are as follows:

  • + (plus)
  • - (minus)
  • * (multiplication)
  • / (division)
  • ** (exponentiation)
  • % (modulo)
  • += (plus-equal)
  • -= (minus-equal)
  • *= (multiplication-equal)
  • /= (slash-equal)
  • %= (mod-equal)

To perform any arithmetic operation, we prefix the expr and let keywords before the actual arithmetic expression. The following example shows how to perform an arithmetic operation in bash:

#!/bin/bash
#Filename: arithmetic.sh
#Description: Arithmetic evaluation

num1=10 num2=5
echo "Numbers are num1 = $num1 and num2 = $num2"
echo "Addition = `expr $num1 + $num2`"`"
echo "Subtraction = `expr $num1 - $num2`"
echo "Multiplication = `expr $num1 \* $num2`"
echo "Division = `expr $num1 / $num2`"
let "exponent = $num1 ** num2"
echo "Exponentiation = $exponent" 
echo "Modulo = `expr $num1 % $num2`"
let "num1 += $num2"
echo "New num1 = $num1"
let "num1 -= $num1"
echo "New num2 = $num2"

The result after running this script will look as follows:

Numbers are num1 = 10 and num2 = 5
Addition = 15
Subtraction = 5
Multiplication = 50
Division = 2
Exponentiation = 100000
Modulo = 0
New num1 = 15
New num2 = 5

Logical operators

Logical operators are also known as Boolean operators. They are:

! (NOT), && (AND), and || (OR)

Performing a logical operation returns a Boolean value as true (1) or false (0) depending upon the values of variable(s) on which the operation is done.

One of the useful use-case is: suppose that we want to execute a command if the first command or operation returns successfully. In this case, we can use the && operator. Similarly, if we want to execute another command, irrespective of the first command that got executed or not, then we can use the || operator between two commands. We can use the ! operator to negate the true value. For example:

$ cd ~/Documents/ && ls

The cd command is used to change the current path to the specified argument. Here, the cd ~/Documents/ command will change the directory to Documents if exists. If it fails, then ls won't get executed, but if cd to Documents succeeds, the ls command will display the content of Documents directory:

$ cat ~/file.txt  || echo "Current Working directory $PWD"
cat: /home/skumari/file.txt: No such file or directory
Current Working directory /tmp/

The cat command displays the content of file.txt if it exists. Irrespective of the cat ~/file.txt command execution, later the command that is echo "Current Working directory $PWD" will be executed:

$  ! cd /tmp/foo && mkdir /tmp/foo
bash: cd: /tmp/foo: No such file or directory

By running the preceding commands, first it will try to change the directory to /tmp/foo. Here, ! cd /tmp/foo means if change directory to /tmp/foo doesn't succeed, then run the second command, which is mkdir /tmp/foo. The mkdir command is used to create a new directory. As a result of proceeding command execution, directory /tmp/foo will be created if it doesn't exist.

$ cd /tmp/foo

Since the /tmp/foo directory has been created, a successful change of the directory will occur.

Comparison operators

Comparison operators compare two variables and check whether a condition is satisfied or not. They are different for integers and strings.

Comparison operators that are valid for integer variables (consider a and b as two integer variables; for example, a=20, b=35) are as follows:

  • -eq (is equal to) - [ $a -eq $b ]
  • -ne (is not equal to) - [ $a -ne $b ]
  • -gt (is greater than) - [ $a -gt $b ]
  • -ge or >= (is greater than or equal to) - [ $a -ge $b ]
  • -lt (is less than) - [ $a -lt $b ]
  • -le (is less than or equal to) - [ $a -le $b ]
  • < (is less than) - (($a < $b))
  • <= (is less than or equal to) - (($a <= $b))
  • > (is greater than) - (($a > $b))
  • >= (is greater than or equal to) - (($a >= $b))

Comparison operators that are valid for string variables (consider a and b as two string variables; for example, a="Hello" b="World") are as follows:

  • = (is equal to); for example, [ $a = $b ]
  • != (is not equal to); for example, [ $a != $b ]
  • < (is less than); for example, [ $a \< $b ] or [[ $a \< $b ]] or (( $a \< $b ))
  • > (is greater than); for example,[ $a \> $b ] or [[ $a > $b ]] or (( $a \> $b ))
  • -n (string is non-empty); for example,[ -n $a ]
  • -z (string has zero length or null); for example,[ -z $a ]

Shell uses the < and > operators for redirection, so it should be used with an escape (\) if used under [ … ]. Double parentheses, (( ... )) or [[ … ]], doesn't need an escape sequence. Using [[ … ]] also supports pattern matching.

We will see the usage and examples of operators in more detail in Chapter 3, Effective Script Writing.

Shell expansions

While working with shell, we perform a lot of similar and repetitive tasks. For example, in the current directory, there are 100 files but we are interested only in shell script whose file extension is .sh. We can execute following command to view only shell script files in current directory:

$ ls *.sh

This will show all the files ending with .sh. An interesting take away from here is the * wildcard. It means a match list of files whose name can be anything and that ends with .sh.

Shell expands all wildcard patterns. A list of the latest wildcard patterns are as follows:

  • ~ (Tilde)
  • * (Asterisk)
  • ? (Question mark)
  • [ ] (Square brackets)
  • { } (Curly brackets)

To explain shell expansion for different wildcards, we will create a test folder in our home directory using the mkdir command containing different files mentioned as follows:

$ mkdir  ~/test && cd ~/test
$ touch a ab foo bar hello moo foo.c bar.c moo.c hello.txt foo.txt bar.sh hello.sh moo.sh

The touch command creates an empty file if it doesn't exist. If a file exists, then the file timestamp changes:

$ ls
a  ab  bar  bar.c  bar.sh  foo  foo.c  foo.txt  hello  hello.sh  hello.txt  moo  moo.c  moo.sh

Running the preceding commands will create a test directory, and inside test directory creates files given as parameter to the touch command.

~ (Tilde)

~ (Tilde) gets expanded by bash when it is present at the beginning of an unquoted string. The expansion depends upon what tilde-prefix is used. Tilde prefixes are characters until the first unquoted (/) slash. Some of the bash expansions are as follows:

  • ~: This is the user's home directory; the value is set in the $HOME variable
  • ~user_name: This is the home directory of the user's user_name
  • ~user_name/file_name: This is the file/directory file_name in the user's user_name home directory
  • ~/file_name: This is the file/directory file_name in the home directory that is $HOME/file_name
  • ~+: This is the current working directory; the value is set in the $PWD variable
  • ~-: This is the old or last working directory; the value is set in the $OLDPWD variable
  • ~+/file_name: This is the file/directory file_name in the current directory that is $PWD/file_name
  • ~-/file_name: This is the file/directory file_name in the old/last working directory that is $OLDPWD/file_name

* (Asterisk)

It matches zero or more characters. Take a test directory as an example:

  • Display all files as follows:
    $ ls *
    a  ab  bar  bar.c  bar.sh  foo  foo.c  foo.txt  hello  hello.sh  hello.txt  moo  moo.c  moo.sh
    
  • Display the C source files as follows:
    $ ls *.c
    bar.c  foo.c  moo.c
    
  • Display files that have a in its name, as follows:
    $ ls *a*
    a  ab  bar  bar.c  bar.sh
    
  • Deleting files with an extension .txt as follows:
    $ rm *.txt
    $ ls
    a  ab  bar  bar.c  bar.sh  foo  foo.c  hello  hello.sh  moo  moo.c  moo.sh
    

? (Question mark)

It matches any single character: ? (single question mark will match a single character), ?? (double question mark matches any two characters), and so on. Take a test directory as an example:

$ touch a ab foo bar hello moo foo.c bar.c moo.c hello.txt foo.txt bar.sh hello.sh moo.sh

This will recreate files that were removed during the previous example, and also update the access and modification time of the existing files:

  • Get files whose name length is irrespective of what the extension file has:
    $ ls ??
    ab
    
  • Get files whose name length is 2 or 5:
    $ ls ?? ?????
    ab  bar.c  foo.c  hello  moo.c
    
  • Delete files whose name is four characters long:
    $ rm ????
    rm: cannot remove '????': No such file or directory
    This error is because there is no file name with 4 character
    
  • Move files to the /tmp directory whose name is at least three characters long:
    $ mv ???* /tmp
    $ ls
    a ab
    

We see only two files in the test directory because the rest of the files were of the length 3 or more.

[ ] (Square brackets)

Square brackets match any character from the characters mentioned inside the square brackets. Characters can be specified as a word or range.

A range of characters can be specified using - (hyphen). For example:

  • [a-c]: This matches a, b, or c
  • [a-z]: This matches any character from a to z
  • [A-Z]: This matches any character from A to Z
  • [0-9]: This matches any character from 0 to 9

Take a test directory as an example and recreate files in a test directory:

$ touch a ab foo bar hello moo foo.c bar.c moo.c hello.txt foo.txt bar.sh hello.sh moo.sh

Get files whose name starts with a, b, c, or d with the following command:

$ ls [a-d]*
a  ab  bar  bar.c  bar.sh

Get files whose name starts with any letter and ends with a letter o or h, with the following command:

$  ls [a-zA-Z]*[oh]
foo  hello  hello.sh  moo  moo.sh

Get files that have at least the letter o twice in its name, with the following command:

$ ls *[o]*[o]*
foo  foo.c  foo.txt  moo  moo.c  moo.sh

[!characters] (Exclamation mark) is used to match a character that is not part of a charter set mentioned inside square brackets.

Get files that don't have a number in its name, with the following command:

$  ls [!0-9]*
a  ab  bar  bar.c  bar.sh  foo  foo.c  foo.txt  hello  hello.sh  hello.txt  moo  moo.c  moo.sh

{ } (Curly brackets)

It creates multiple wildcard patterns to match. A brace expression may contain either a comma-separated list of strings, a range, or a single character.

A range can be specified by using the following:

  • {a..z}: This matches all the charterer from a to z
  • {0..6}: This matches numbers 0, 1, 2, 3, 4, 5 and 6

Take a test directory as an example and recreate files in the test directory:

$ touch a ab foo bar hello moo foo.c bar.c moo.c hello.txt foo.txt bar.sh hello.sh moo.sh

Get files that have the file extension .sh or .c, with the following command:

$ ls {*.sh,*.c}
bar.c  bar.sh  foo.c  hello.sh  moo.c  moo.sh

Copy bar.c to bar.html by using the following command:

$ cp bar{.c,.cpp}  # Expands to cp bar.c bar.cpp
$ ls bar.*
bar.c  bar.cpp  bar.sh

Print the number from 1 to 50 by using the following command:

$ echo {1..50}
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

Create 10 files that start with hello and has an extension .cpp:

$ touch hello{0..9}.cpp
$ ls *.cpp
hello0.cpp  hello1.cpp  hello2.cpp  hello3.cpp  hello4.cpp  hello5.cpp  hello6.cpp  hello7.cpp  hello8.cpp  hello9.cpp

To avoid shell expansion of a wildcard, use backslash (\) or write a string within a single quote (' ').

Construct commands using eval

The eval command is a shell builtin command used to construct a command by concatenating arguments passed to eval. A concatenated command is further executed by shell and returns a result. If no arguments are given to eval, it returns 0.

The syntax of the eval command is as follows:

eval [arg …]

The following example shows the expansion of a variable to the name of another variable using eval:

$ name=foo
$ foo="Welcome to foo world"
$ echo $name
foo
$ new_name='$'$name    #new_name just stores string value $foo
$ echo $new_name
$foo
$ eval new_name='$'$name  # eval processes $foo string into variable and  prints                 # foo variable value
Welcome to foo world

Another example where eval can be useful is as follows:

$ pipe="|"
$  df $pipe wc  # Will give error because 
df: '|': No such file or directory
df: 'wc': No such file or directory
$ eval df $pipe wc  # eval executes it as shell command
12      73     705

Here, the df command shows a system disk's usage:

A shell script showing the use of eval is as follows:
#!/bin/bash
#Filename: eval.sh
#Description: Evaluating string as a command using eval

cmd="ls /usr"
echo "Output of command $cmd -"
eval $cmd   #eval will treat content of cmd as shell command and execute it
cmd1="ls /usr | wc -l"
echo "Line count of /usr -"
eval $cmd1

expression="expr 2 + 4 \* 6"
echo "Value of $expression"
eval $expression

Running the script will give you the following result:

Output of command ls /usr -
avr  bin  games  include  lib  lib64  libexec  local  sbin  share  src  tmp
Line count of /usr -
12
Value of expr 2 + 4 \* 6
26

Make bash behave using set

The set command is a shell builtin command that is used to set and unset a value of the local variables in shell.

The syntax of using set is as follows:

 set [--abBCefhHkmnpPtuvx] [-o option] [arg …]

Some of the option values are allexport, braceexpand, history, keyword, verbose, and xtrace.

Using set without any option displays the name and value of all shell variables and functions, in a format that can be reused as an input for setting and unsetting the currently set variables.

Exit on the first failure

In a shell script, by default, the next line is executed if an error occurs in the current line. Sometimes, we may want to stop running a script further after an error has been encountered. The -e option of set ensures to exit a script once any of the commands in a pipeline fails.

In the following shell script, do_not_exit_on_failure.sh doesn't use set with the option -e:

$ cat do_not_exit_on_failure.sh
#!/bin/bash
# Filename: do_not_exit_on_failure.sh
# Description: Resume script after an error

echo "Before error"
cd /root/       # Will give error
echo "After error"

After running this script, the output is as follows:

Before error
do_not_exit_on_failure.sh: line 6: cd: /root/: Permission denied
After error

We see that the command after the error gets executed as well. In order to stop the execution after an error is encountered, use set -e in the script. The following script demonstrates the same:

$ cat exit_on_failure.sh
#!/bin/bash
# Filename: exit_on_failure.sh
# Description: Exits script after an error

set -e
echo "Before error"
cd /root/       # Will give error
echo "After error"

The output after running the preceding script is as follows:

Before error
exit_on_failure.sh: line 7: cd: /root/: Permission denied

We can see that the script has been terminated after encountering an error at the line number 7.

Enabling/disabling symbolic link's resolution path

Using set with the -P option doesn't resolve symbolic links. Following example demonstrate how we can enable or disable symbolic link resolution of /bin directory which is symbolic link of /usr/bin/ directory:

$ ls -l /bin
lrwxrwxrwx. 1 root root 7 Nov 18 18:03 /bin -> usr/bin
$ set –P    # -P enable symbolic link resolution
$ cd /bin
$ pwd
/usr/bin
$ set +P   # Disable symbolic link resolution
$ pwd
/bin

Setting/unsetting variables

We can use the set command to see all local variables accessible for the current process. The local variables are not accessible in the subprocess.

We can create our own variable and set it locally as follows:

$ MYVAR="Linux Shell Scripting"
$ echo $MYVAR
 Linux Shell Scripting
$ set | grep MYVAR  # MYVAR local variable is created
MYVAR='Linux Shell Scripting'
$ bash    # Creating a new bash sub-process in current bash
$ set | grep MYVAR
$    # Blank because MYVAR is local variable

To make a variable accessible to its subprocesses as well, use the export command followed by the variable to be exported:

$ MYVARIABLE="Hello World"
$ export  MYVARIABLE
$ bash    # Creating a new bash sub-process under bash
$ echo $MYVARIABLE
Hello World

This will export the MYVARIABLE variable to any subprocess that ran from that process. To check whether MYVARIABLE has exported or not, run the following command:

$ export |grep MYVARIABLE
declare -x MYVARIABLE="Hello World"
$ export | grep MYVAR
$MYVAR variable is not present in sub-process but variable MYVARIABLE is present in sub-process.

To unset local or exported variables, use the unset command and it will reset the value of the variable to null:

$ unset MYVAR        # Unsets local variable MYVAR
$ unset  MYVARIABLE    # Unsets exported variable MYVARIABLE

Summary

After reading this chapter, you understood how to write simple shell script in bash by printing, echoing, and asking user input. You should now have a good understanding of defining and using variables in shell and what builtin shell variables exist. You are now familiar with what operators are available in shell and how they can create and evaluate their own expression. With information about wildcards available in this chapter, it makes work easier for you while you are dealing with similar kind of data or pattern. The shell builtin command set enables modifying shell variables easily.

This chapter has built a foundation for upcoming chapters. Now, in next chapter, you will get to know about standard inputs, outputs, and errors. Also, there will be a detailed coverage of how to use an output from commands and then filter/transform them to show the data according to your need.

Left arrow icon Right arrow icon

Description

Shell scripting is a quick method to prototype complex applications or problems. Shell scripts are a collection of commands to automate tasks, usually those for which the user has a repeated need, when working on Linux-based systems. Using simple commands or a combination of them in a shell can solve complex problems easily. This book starts with the basics, including essential commands that can be executed on Linux systems to perform tasks within a few nanoseconds. You’ll learn to use outputs from commands and transform them to show the data you require. Discover how to write shell scripts easily, execute script files, debug, and handle errors. Next, you’ll explore environment variables in shell programming and learn how to customize them and add a new environment. Finally, the book walks you through processes and how these interact with your shell scripts, along with how to use scripts to automate tasks and how to embed other languages and execute them.

Who is this book for?

This book is aimed at administrators and those who have a basic knowledge of shell scripting and who want to learn how to get the most out of writing shell scripts.

What you will learn

  • Write effective shell scripts easily
  • Perform search operations and manipulate large text data with a single shell command
  • Modularize reusable shell scripts by creating shell libraries
  • Redirect input, output, and errors of a command or script execution to other streams
  • Debug code with different shell debugging techniques to make your scripts bugfree
  • Manage processes, along with the environment variables needed to execute them properly
  • Execute and embed other languages in your scripts
  • Manage creation, deletion, and search operations in files

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Nov 23, 2015
Length: 282 pages
Edition : 1st
Language : English
ISBN-13 : 9781783552375
Tools :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want

Product Details

Publication date : Nov 23, 2015
Length: 282 pages
Edition : 1st
Language : English
ISBN-13 : 9781783552375
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just $5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total $ 148.97
Linux Shell Scripting Essentials
$54.99
Learning Linux Shell Scripting
$54.99
Mastering Linux Shell Scripting
$38.99
Total $ 148.97 Stars icon

Table of Contents

9 Chapters
1. The Beginning of the Scripting Journey Chevron down icon Chevron up icon
2. Getting Hands-on with I/O, Redirection Pipes, and Filters Chevron down icon Chevron up icon
3. Effective Script Writing Chevron down icon Chevron up icon
4. Modularizing and Debugging Chevron down icon Chevron up icon
5. Customizing the Environment Chevron down icon Chevron up icon
6. Working with Files Chevron down icon Chevron up icon
7. Welcome to the Processes Chevron down icon Chevron up icon
8. Scheduling Tasks and Embedding Languages in Scripts Chevron down icon Chevron up icon
Index Chevron down icon Chevron up icon

Customer reviews

Rating distribution
Full star icon Full star icon Full star icon Full star icon Half star icon 4.5
(2 Ratings)
5 star 50%
4 star 50%
3 star 0%
2 star 0%
1 star 0%
Parminder Singh Bola May 08, 2016
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I love it!!!
Amazon Verified review Amazon
skeptic Jun 24, 2017
Full star icon Full star icon Full star icon Full star icon Empty star icon 4
A short, but well written book. cover modern bash (including some features of version 4). It mentions bashdb which is a plus. Generally the content is well thought out and the text written well (within the limitation of the size of the book, which are pretty severe)There are some errors. For example putting aliases into .bashrc file is possible but technically incorrect - aliases work only in interactive sessions. Generally they more properly belong to .bash_profile file.The book should be priced much lower, as this is a pretty short book.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.