In this article by Rejah Rehim, author of the book Mastering Python Penetration Testing, we will cover:

Setting up the scripting environment in different operating systems
Installing third-party Python libraries
Working with virtual environments
Python language basics

(For more resources related to this topic, see here.)

Python is still the leading language in the world of penetration testing (pentesting) and information security. Python-based tools include all kinds oftools used for inputting massive amounts of random data to find errors and security loop holes, proxies, and even the exploit frameworks. If you are interested in tinkering with pentesting tasks, Python is the best language to learn because of its large number of reverse engineering and exploitation libraries.

Over the years, Python has received numerous updates and upgrades. For example, Python 2 was released in 2000 and Python 3 in 2008. Unfortunately, Python 3 is not backward compatible; hence most of the programs written in Python 2 will not work in Python 3. Even though Python 3 was released in 2008, most of the libraries and programs still use Python 2. To do better penetration testing, the tester should be able to read, write, and rewrite python scripts.

As a scripting language, security experts have preferred Python as a language to develop security toolkits. Its human-readable code, modular design, and large number of libraries provide a start for security experts and researchers to create sophisticated toolswith it. Python comes with a vast library (standard library) that accommodates almost everything from simple I/O to platform-specific APIcalls. Many of the default and user-contributed libraries and modules can help us in penetration testing with building tools to achieve interesting tasks.

Setting up the scripting environment

Your scripting environment is basically the computer you use for your daily workcombined with all the tools in it that you use to write and run Python programs. The best system to learn on is the one you are using right now. This section will help you to configure the Python scripting environment on your computer so that you can create and run your own programs.

If you are using Mac OS X or Linux installation in your computer, you may have a Python Interpreter pre-installed in it. To find out if you have one, open terminal and type python. You will probably see something like this:

$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

From the preceding output, we can see that Python 2.7.6 is installed in this system. By issuing python in your terminal, you started Python interpreter in the interactive mode. Here, you can play around with Python commands; what you type will run and you'll see the outputs immediately.

You can use your favorite text editor to write your Python programs. If you do not have one, then try installing Geany or Sublime Text and it should be perfect for you. These are simple editors and offer a straightforward way to write as well as run your Python programs. In Geany, the output is shown in a separate terminal window, whereas Sublime Text uses an embedded terminal window. Sublime Text is not free, but it has a flexible trial policy that allows you to use the editor without any stricture. It is one of the few cross-platform text editors that is quite apt for beginners and has a full range of functions targeting professionals.

Setting up in Linux

Linux system is built in a way that makes it smooth for users to get started with Python programming. Most Linux distributions already have Python installed. For example, the latest versions of Ubuntu and Fedora come with Python 2.7. Also, the latest versions of Redhat Enterprise (RHEL) and CentOS come with Python 2.6. Just for the records, you might want to check it.

If it is not installed, the easiest way to install Python is to use the default package manger of your distribution, such as apt-get, yum, and so on. Install Python by issuing the following commands in the terminal.

For Debian / Ubuntu Linux / Kali Linux users:

sudo apt-get install python2

For Red Hat / RHEL / CentOS Linux user

sudo yum install python

To install Geany, leverage your distribution'spackage manger.

For Debian / Ubuntu Linux / Kali Linux users:

sudo apt-get install geany geany-common

For Red Hat / RHEL / CentOS Linux users:

sudo yum install geany

Setting up in Mac

Even though Macintosh is a good platform to learn Python, many people using Macs actually run some Linux distribution or the other on their computer or run Python within a virtual Linux machine. The latest version of Mac OS X, Yosemite, comes with Python 2.7 preinstalled. Once you verify that it is working, install Sublime Text.

For Python to run on your Mac, you have to install GCC, which can be obtained by downloading XCode, the smaller command-line tool. Also, we need to install Homebrew, a package manager.

To install Homebrew, open Terminal and run the following:

$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

After installing Homebrew, you have to insert the Homebrew directory into your PATH environment variable. You can do this by including the following line in your ~/.profile file:

export PATH=/usr/local/bin:/usr/local/sbin:$PATH

Now that we are ready to install Python 2.7, run the following command in your terminal that will do the rest:

$ brew install python

To install Sublime Text, go to Sublime Text's downloads page in http://www.sublimetext.com/3 and click on the OS X link. This will get you the Sublime Text installer for your Mac.

Setting up in Windows

Windows does not have Python preinstalled on it. To check whether it isinstalled, open a command prompt and type the word python, and press Enter. In most cases, you will get a message that says Windows does not recognize python as a command.

We have to download an installer that will set Python for Windows. Then, we have to install and configure Geany to run Python programs.

Go to Python's download page in https://www.python.org/downloads/windows/and download the Python 2.7 installer, which is compatible with your system. If you are not aware of your operating systems architecture, then download 32-bit installers, which will work on both the architectures, but 64-bit will only work on 64-bit systems.

To install Geany, go to Geany'sdownload page viahttp://www.geany.org/Download/Releases and download the full installer variant, which has a description Full Installer including GTK 2.16. By default, Geany doesn't know where Python resides on your system. So, we need to configure it manually.

For this, write a Hello world program in Geany, save it anywhere in your system as hello.py, and run it.

There are three methods you can run a python program in Geany:

Select Build | Execute.
Press F5.
Click the icon with three gears on it:
When you have a running hello.py program in Geany, go to Build | Set Build Commands.
Then, enter the python commands option withC:Python27python -m py_compile "%f"and execute command withC:Python27python "%f".
Now, you can run your Python programs while coding in Geany.

It is recommended to run a Kali Linux distribution as a virtual machine and use this as your scripting environment. Kali Linux comes with a number of tools preinstalled and is based on Debian Linux, so you'll also be able to install a wide variety of additional tools and libraries. Also, some of the libraries will not work properly on Windows systems.

Installing third-party libraries

We will be using many Python libraries and this section will help you install and use third-party libraries.

Setuptools and pip

One of the most useful pieces of third-party Python software is Setuptools. With Setuptools, you could download and install any compliant Python libraries with a single command.

The best way to install Setuptools on any system is to download the ez_setup.py file from https://bootstrap.pypa.io/ez_setup.pyand run this file with your Python installation.

In Linux, run this in terminal with the correct path to theez_setup.py script:

sudo python path/to/ez_setup.py

For Windows 8 or the older versions of Windows with PowerShell 3 installed, start Powershell with Administrative privileges and run this command in it:

> (Invoke-WebRequest https://bootstrap.pypa.io/ez_setup.py).Content | python -

For Windows systems without a PowerShell 3 installed, download the ez_setup.py file from the link provided previously using your web browser and run that file with your Python installation.

pipis a package management system used to install and manage software packages written in Python.After the successful installation of Setuptools, you can install pip by simply opening a command prompt and running the following:

$ easy_install pip

Alternatively, you could also install pip using your default distribution package managers:

On Debian, Ubuntu and Kali Linux:
```
sudo apt-get install python-pip
```
On Fedora:
```
sudo yum install python-pip
```

Now, you could run pip from the command line. Try installing a package with pip:

$ pip install packagename

Working with virtual environments

Virtual environment helps separate dependencies required for different projects; by working inside the virtual environment, it also helps to keep our global site-packages directory clean.

Using virtualenv and virtualwrapper

virtualenv is a python module which helps to create isolated Python environments for our each scripting experiments, which creates a folder with all necessary executable files and modules for a basic python project.

You can install virtual virtualenv with the following command:

sudo pip install virtualenv

To create a new virtual environment,create a folder and enter into the folder from commandline:

$ cd your_new_folder
$ virtualenv name-of-virtual-environment

This will initiate a folder with the provided name in your current working directory with all the Python executable files and pip library, which will then help install other packages in your virtual environment.

You can select a Python interpreter of your choice by providing more parameters, such as the following command:

$ virtualenv -p /usr/bin/python2.7 name-of-virtual-environment

This will create a virtual environment with Python 2.7 .We have to activate it before we start using this virtual environment:

$ source name-of-virtual-environment/bin/activate

python-scripting-essentials-img-1

Now, on the left-hand side of the command prompt, the name of the active virtual environment will appear. Any package that you install inside this prompt using pip will belong to the active virtual environment, which will be isolated from all the other virtual environments and global installation.

You can deactivate and exit from the current virtual environment using this command:

$ deactivate

virtualenvwrapper provides a better way to use virtualenv. It also organize all the virtual environments in one place.

To install, we can use pip, but let's make sure we have installed virtualenv before installing virtualwrapper.

Linux and OS X users can install it with the following method:

$ pip install virtualenvwrapper
Also,add thesethree lines inyour shell startup file like .bashrc or .profile.
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/Devel
source /usr/local/bin/virtualenvwrapper.sh

This will set theDevel folder in your home directory as the location of your virtual environment projects.

For Windows users, we can use another package virtualenvwrapper-win . This can also be installed with pip.

pip install virtualenvwrapper-win

Create a virtual environment with virtualwrapper:

$ mkvirtualenv your-project-name

This creates a folder with the provided name inside ~/Envs.

To activate this environment, we can use workon command:

$ workon your-project-name

These two commands can be combined with the single one,as follows:

$ mkproject your-project-name

We can deactivate the virtual environment with the same deactivate command in virtualenv. To delete a virtual environment, we can use the following command:

$ rmvirtualenv your-project-name

Python language essentials

In this section, we will go through the idea of variables, strings, data types, networking, and exception handling. As an experienced programmer, this section will be just a summarization of what you already know about Python.

Variables and types

Python is brilliant in case of variables—variable point to data stored in a memory location. This memory location may contain different values, such as integer, real number, Booleans, strings, lists, and dictionaries.

Python interprets and declares variables when you set some value to this variable. For example, if we set:

a = 1 andb = 2

Then, we will print the sum of these two variables with:

print (a+b)

The result will be 3 as Python will figure out both a and b are numbers.

However, if we had assigned:

a = "1" and b = "2"

Then,the output will be 12, since both a and b will be considered as strings. Here, we do not have to declare variables or their type before using them, as each variable is an object. The type() method can be used to getthe variable type.

Strings

As any other programming language, strings are one of the important things in Python. They are immutable. So, they cannot be changed once they are defined. There are many Python methods, which can modify string. They do nothing to the original one, but create a copy and return after modifications. Strings can be delimited with single quotes, double quotes, or in case of multiple lines, we can use triple quotes syntax. We can use the character to escape additional quotes, which come inside a string.

Commonly used string methods are:

string.count('x'):This returns the number of occurrences of 'x' in the string
string.find('x'):This returns the position of character 'x'in the string
string.lower():This converts the string into lowercase
string.upper():This converts the string into uppercase
string.replace('a', 'b'):This replaces alla with b in the string

Also, we can get the number of characters including white spaces in a string with the len() method.

#!/usr/bin/python
a = "Python"
b = "Pythonn"
c = "Python"

print len(a)
print len(b)
print len(c)

You can read more about the string function via https://docs.python.org/2/library/string.html.

Lists

Lists allow to store more than one variable inside it and provide a better method for sorting arrays of objects in Python. They also have methods that will help to manipulate the values inside them.

list = [1,2,3,4,5,6,7,8]
print (list[1])

This will print 2, as the Python index starts from 0. Print out the whole list:

list = [1,2,3,4,5,6,7,8]
for x in list:
print (x)

This will loop through all the elements and print them.

Useful list methods are:

.append(value):This appends an element at the end of list
.count('x'):This gets the the number of 'x' in list
.index('x'):This returns the index of 'x' in list
.insert('y','x'):This inserts 'x' at location 'y'
.pop():This returns last element and also remove it from list
.remove('x'):This removes first 'x' from list
.reverse():This reverses the elements in the list
.sort():This sorts the list alphabetically in ascending order, or numerical in ascending order

Dictionaries

A Python dictionary is a storage method for key:value pairs. In Python, dictionaries are enclosed in curly braces, {}. For example,

dictionary = {'item1': 10, 'item2': 20}
print(dictionary['item2'])

This will output 20. We cannot create multiple values with the same key. This will overwrite the previous value of the duplicate keys. Operations on dictionaries are unique. Slicing is not supported in dictionaries

We can combine two distinct dictionaries to one by using the update method. Also, the update method will merge existing elements if they conflict:

a = {'apples': 1, 'mango': 2, 'orange': 3}
b = {'orange': 4, 'lemons': 2, 'grapes ': 4}
a.update(b)

Print a

This will return:

{'mango': 2, 'apples': 1, 'lemons': 2, 'grapes ': 4, 'orange': 4}

To delete elements from a dictionary, we can use the del method:

del a['mango']
print a

This will return:

{'apples': 1, 'lemons': 2, 'grapes ': 4, 'orange': 4}

Networking

Sockets are the basic blocks behind all the network communications by a computer. All network communications go through a socket. So, sockets are the virtual endpoints of any communication channel that takes place between two applications, which may reside on the same or different computers.

The socket module in Python provides us a better way to create network connections with Python. So, to make use of this module, we will have to import this in our script:

import socket
socket.setdefaulttimeout(3)
newSocket = socket.socket()
newSocket.connect(("localhost",22))
response = newSocket.recv(1024)
print response

This script will get the response header from the server.

Handling Exceptions

Even though we wrote syntactically correct scripts, there will be some errors while executing them. So, we will have to handle the errors properly. The simplest way to handle exception in Python is try-except:

Try to divide a number with zero in your Python interpreter:

>>> 10/0
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: integer division or modulo by zero

So, we can rewrite this script with thetry-except blocks:

try:
answer = 10/0
except ZeroDivisionError, e:
answer = e
print answer

This will return the error integer division or modulo by zero.

Summary

Now, we have an idea about basic installations and configurations that we have to do before coding. Also, we have gone through the basics of Python, which may help us speed up scripting.