The Python Paramiko library

Paramiko is a Python implementation of the SSHv2 protocol. Just like the pxssh subclass of Pexpect, Paramiko simplifies the SSHv2 interaction between the host and the remote device. Unlike pxssh, Paramiko focuses only on SSHv2 with no Telnet support. It also provides both client and server operations.

Paramiko is the low-level SSH client behind the high-level automation framework Ansible for its network modules. We will cover Ansible in Chapter 4, The Python Automation Framework – Ansible Basics and Chapter 5, The Python Automation Framework – Beyond Basics. Let's take a look at the Paramiko library.

Installation of Paramiko

Installing Paramiko is pretty straightforward with Python pip. However, there is a hard dependency on the cryptography library. The library provides low-level, C-based encryption algorithms for the SSH protocol.

The installation instruction for Windows, Mac, and other flavors of Linux can be found at: https://cryptography.io/en/latest/installation/.

We will show the Paramiko installation of our Ubuntu 18.04 virtual machine in the following output. The following output shows the installation steps, as well as Paramiko successfully imported into the Python interactive prompt:

sudo apt-get install build-essential libssl-dev libffi-dev python3-dev 
pip install cryptography
pip install paramiko

Let us test the library's usage by importing it with the Python interpreter:

$ python
Python 3.6.8 (default, Aug 20 2019, 17:12:48)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import paramiko
>>> exit()

Now we are ready to take a look at Paramiko in the next section.

Paramiko overview

Let's look at a quick Paramiko example using the Python 3 interactive shell:

>>> import paramiko, time
>>> connection = paramiko.SSHClient()
>>> connection.set_missing_host_key_policy(paramiko.AutoAddPolicy())
>>> connection.connect('172.16.1.20', username='cisco', password='cisco', look_for_keys=False, allow_agent=False)
>>> new_connection = connection.invoke_shell()
>>> output = new_connection.recv(5000)
>>> print(output) b"r\n\***********************************************************************
***rn* IOSv is strictly limited to use for evaluation, demonstration and IOS *rn* education. IOSv is provided as-is and is not supported by Cisco's
*rn* Technical Advisory Center. Any use or disclosure, in whole or in part,
*rn* of the IOSv Software or Documentation to any third party for any *rn* purposes is expressly prohibited except as otherwise authorized by *rn* Cisco in writing.
*rn************************************************************************
**rniosv-1#"
>>> new_connection.send("show version | i V\n")
19
>>> time.sleep(3)
>>> output = new_connection.recv(5000)
>>> print(output)
b'show version | i VrnCisco IOS Software, IOSv Software (VIOS- ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)rnProcessor board ID 9MM4BI7B0DSWK40KV1IIRrniosv-1#'
>>> new_connection.close()
>>>

The time.sleep() function inserts a time delay to ensure that all the outputs were captured. This is particularly useful on a slower network connection or a busy device. This command is not required but is recommended depending on your situation.

Even if you are seeing the Paramiko operation for the first time, the beauty of Python and its clear syntax means that you can make a pretty good educated guess at what the program is trying to do:

>>> import paramiko
>>> connection = paramiko.SSHClient()
>>> connection.set_missing_host_key_policy(paramiko.AutoAddPolicy())
>>> connection.connect('172.16.1.20', username='cisco', password='cisco',
look_for_keys=False, allow_agent=False)

The first four lines create an instance of the SSHClient class from Paramiko. The next line sets the policy that the client should use regarding keys; in this case, iosv-1 is not present in either the system host keys or the application's keys. In our scenario, we will automatically add the key to the application's HostKeys object. At this point, if you log on to the router, you will see the additional login session from Paramiko:

iosv-1#who
Line User Host(s) Idle Location
*578 vty 0 cisco idle 00:00:00 172.16.1.1
579 vty 1 cisco idle 00:01:30 172.16.1.173
Interface User Mode Idle Peer Address
iosv-1#

The next few lines invoke a new interactive shell from the connection and a repeatable pattern of sending a command and retrieving the output. Finally, we close the connection.

Some readers who have used Paramiko before might be familiar with the exec_command() method instead of invoking a shell. Why do we need to invoke an interactive shell instead of using exec_command() directly? Unfortunately, exec_command() on Cisco IOS only allows a single command. Consider the following example with exec_command() for the connection:

>>> connection.connect('172.16.1.20', username='cisco', password='cisco', look_for_keys=False, allow_agent=False)
>>> stdin, stdout, stderr = connection.exec_command('show version | i V\n')
>>> stdout.read()
b'Cisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)rnProcessor board ID 9MM4BI7B0DSWK40KV1IIRrn'
>>>

Everything works great; however, if you look at the number of sessions on the Cisco device, you will notice that the connection is dropped by the Cisco device without you closing the connection:

iosv-1#who
Line User Host(s) Idle Location
*578 vty 0 cisco idle 00:00:00 172.16.1.1
Interface User Mode Idle Peer Address
iosv-1#

Because the SSH session is no longer active, exec_command() will return an error if you want to send more commands to the remote device:

>>> stdin, stdout, stderr = connection.exec_command('show version | i V\n') 
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/dist-packages/paramiko/client.py", line 435, in exec_command
chan = self._transport.open_session(timeout=timeout)
File "/usr/local/lib/python3.5/dist-packages/paramiko/transport.py", line 711, in open_session
timeout=timeout)
File "/usr/local/lib/python3.5/dist-packages/paramiko/transport.py", line 795, in open_channel
raise SSHException('SSH session not active') paramiko.ssh_exception.SSHException: SSH session not active
>>>

In the previous example, the new_connection.recv() command displayed what was in the buffer and implicitly cleared it out for us. What would happen if you did not clear out the received buffer? The output would just keep on filling up the buffer and would overwrite it:

>>> new_connection.send("show version | i V\n")
 19
>>> new_connection.send("show version | i V\n") 
19
>>> new_connection.send("show version | i V\n") 
19
>>> new_connection.recv(5000)
b'show version | i VrnCisco IOS Software, IOSv Software (VIOS- ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)rnProcessor
board ID 9MM4BI7B0DSWK40KV1IIRrniosv-1#show version | i VrnCisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)rnProcessor board ID 9MM4BI7B0DSWK40KV1IIRrniosv-1#show version | i VrnCisco IOS Software, IOSv Software (VIOS-ADVENTERPRISEK9-M), Version 15.6(2)T, RELEASE SOFTWARE (fc2)rnProcessor board ID 9MM4BI7B0DSWK40KV1IIRrniosv-1#'
>>>

For consistency of the deterministic output, we will retrieve the output from the buffer each time we execute a command.

Our first Paramiko program

Our first program will use the same general structure as the Pexpect program we have put together. We will loop over a list of devices and commands while using Paramiko instead of Pexpect. This will give us a good compare and contrast of the differences between Paramiko and Pexpect.

If you have not done so already, you can download the code, chapter2_3.py, from the book's GitHub repository at https://github.com/PacktPublishing/Mastering-Python-Networking-Third-Edition. I will list the notable differences here:

devices = {'iosv-1': {'ip': '172.16.1.20'}, 'iosv-2': {'ip': '172.16.1.21'}}

We no longer need to match the device prompt using Paramiko; therefore, the device dictionary can be simplified:

commands = ['show version', 'show run']

There is no sendline equivalent in Paramiko; instead, we manually include the newline break in each of the commands:

def clear_buffer(connection):
    if connection.recv_ready():
        return connection.recv(max_buffer)

We include a new method to clear the buffer for sending commands, such as terminal length 0 or enable, because we do not need the output for those commands. We simply want to clear the buffer and get to the execution prompt. This function will later be used in the loop, such as in line 25 of the script:

output = clear_buffer(new_connection)

The rest of the program should be pretty self-explanatory, similar to what we have seen in this chapter. The last thing I would like to point out is that since this is an interactive program, we place some buffer and wait for the command to be finished on the remote device before retrieving the output:

time.sleep(5)

After we clear the buffer, during the time between the execution of commands, we will wait five seconds. This will give the device adequate time to respond if it is busy.

More Paramiko features

We will look at Paramiko a bit later in Chapter 4, The Python Automation Framework – Ansible Basics, when we discuss Ansible, as Paramiko is the underlying transport for many of the network modules. In this section, we will take a look at some of the other features of Paramiko.

Paramiko for servers

Paramiko can be used to manage servers through SSHv2 as well. Let's look at an example of how we can use Paramiko to manage servers. We will use key-based authentication for the SSHv2 session.

In this example, I used another Ubuntu virtual machine on the same hypervisor as the destination server. You can also use a server on the VIRL simulator or an instance in one of the public cloud providers, such as Amazon AWS EC2.

We will generate a public-private key pair for our Paramiko host:

ssh-keygen -t rsa

This command, by default, will generate a public key named id_rsa.pub, as the public key under the user home directory ~/.ssh along with a private key named id_rsa. Treat the private key with the same attention as you would private passwords that you do not want to share with anybody else. You can think of the public key as a business card that identifies who you are. Using the private and public keys, the message will be encrypted by your private key locally and decrypted by the remote host using the public key. We should copy the public key to the remote host. In production, we can do this via out-of-band using a USB drive; in our lab, we can simply copy the public key to the remote host's ~/.ssh/authorized_keys file. Open up a Terminal window for the remote server so you can paste in the public key.

Copy the content of ~/.ssh/id_rsa.pub on your management host with Paramiko:

$ cat ~/.ssh/id_rsa.pub 
ssh-rsa <your public key>

Then, paste it to the remote host under the user directory; in this case, I am using echou for both sides:

<Remote Host>$ vim ~/.ssh/authorized_keys
ssh-rsa <your public key>

You are now ready to use Paramiko to manage the remote host. Notice in this example that we will use the private key for authentication as well as the exec_command() method for sending commands:

>>> import paramiko
>>> key = paramiko.RSAKey.from_private_key_file('/home/echou/.ssh/id_rsa')
>>> client = paramiko.SSHClient()
>>> client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
>>> client.connect('192.168.199.182', username='echou', pkey=key)
>>> stdin, stdout, stderr = client.exec_command('ls -l')
>>> stdout.read()
b'total 44ndrwxr-xr-x 2 echou echou 4096 Jan 7 10:14 Desktopndrwxr-xr-x 2
echou echou 4096 Jan 7 10:14 Documentsndrwxr-xr-x 2 echou echou 4096 Jan 7
10:14 Downloadsn-rw-r--r-- 1 echou echou 8980 Jan 7 10:03
examples.desktopndrwxr-xr-x 2 echou echou 4096 Jan 7 10:14 Musicndrwxr-xr-x
echou echou 4096 Jan 7 10:14 Picturesndrwxr-xr-x 2 echou echou 4096 Jan 7 10:14 Publicndrwxr-xr-x 2 echou echou 4096 Jan 7 10:14 Templatesndrwxr-xr-x
2 echou echou 4096 Jan 7 10:14 Videosn'
>>> stdin, stdout, stderr = client.exec_command('pwd')
>>> stdout.read()
b'/home/echoun'
>>> client.close()
>>>

Notice that in the server example, we do not need to create an interactive session to execute multiple commands. You can now turn off password-based authentication in your remote host's SSHv2 configuration for more secure key-based authentication with automation enabled. Some network devices, such as Cumulus and Vyatta switches, also support key-based authentication.

Putting things together for Paramiko

We are almost at the end of the chapter. In this last section, let's make the Paramiko program more reusable. There is one downside of our existing script: we need to open up the script every time we want to add or delete a host, or whenever we need to change the commands we want to execute on the remote host.

This is due to the fact that both the host and command information are statically entered inside of the script. Hardcoding the host and command has a higher chance of making mistakes. Besides, if you were to pass on the script to colleagues, they might not feel comfortable working in Python, Paramiko, or Linux.

By making both the host and command files be read in as parameters for the script, we can eliminate some of these concerns. Users (and a future you) can simply modify these text files when you need to make host or command changes.

We have incorporated the change in the script named chapter2_4.py.

Instead of hardcoding the commands, we broke the commands into a separate commands.txt file. Up to this point, we have been using show commands; in this example, we will make configuration changes. In particular, we will change the logging buffer size to 30000 bytes:

$ cat commands.txt 
config t
logging buffered 30000 
end
copy run start

The device's information is written into a devices.json file. We chose JSON format for the device's information because JSON data types can be easily translated into Python dictionary data types:

$ cat devices.json
{
"iosv-1": {"ip": "172.16.1.20"},
"iosv-2": {"ip": "172.16.1.21"}
}

In the script, we made the following changes:

with open('devices.json', 'r') as f:
    devices = json.load(f)
with open('commands.txt', 'r') as f:
    commands = f.readlines()

Here is an abbreviated output from the script execution:

(venv) $ python chapter2_4.py
Username: cisco
Password:
b'terminal length 0\r\niosv-1#config t\r\nEnter configuration commands, one per line.  End with CNTL/Z.\r\niosv-1(config)#'
b'logging buffered 30000\r\niosv-1(config)#'
b'end\r\niosv-1#'
<skip>

Do a quick check to make sure the change has taken place in both running-config and startup-config:

iosv-1#sh run | i logging
logging buffered 30000
iosv-1#sh start | i logging
logging buffered 30000
iosv-2#sh run | i logging
logging buffered 30000
iosv-2#sh start | i logging
logging buffered 30000

The Paramiko library is a general-purpose library that is intended for working with interactive command line programs. For network management, there is another library, Netmiko, that is a fork from Paramiko that is purpose-built for network device management. We will take a look at it in the upcoming section.