Using Python Automation to interact with network devices [Tutorial]

In this tutorial, we will learn new ways to interact with network devices using Python. We will understand how to configure network devices using configuration templates, and also write a modular code to ensure high reusability of the code to perform repetitive tasks. We will also see the benefits of parallel processing of tasks and the efficiency that can be gained through multithreading.

Python is widely used to perform network automation. With its wide set of libraries (such as Netmiko and Paramiko), there are endless possibilities for network device interactions for different vendors. Let us understand one of the most widely used libraries for network interactions. We will be using Netmiko to perform our network interactions.

Python provides a well-documented reference for each of the modules, and, for our module, the documentation can be found at pypi.org.  For installation, all we have to do is go into the folder from the command line where python.exe is installed or is present. There is a subfolder in that location called scripts. Inside the folder, we have two options that can be used for installing the easy_install.exe or pip.exe modules.

Installing the library for Python can be done in two ways:

• The syntax of easy_install is as follows:

easy_install <name of module>

For example, to install Netmiko, the following command is run:

easy_install netmiko

• The syntax of pip install is as follows:

pip install <name of module>

For example:

pip install netmiko

Here's an example of a simple script to log in to the router (an example IP is 192.168.255.249 with a username and password of cisco) and show the version:

from netmiko import ConnectHandler

device = ConnectHandler(device_type='cisco_ios', ip='192.168.255.249', username='cisco', password='cisco')
output = device.send_command("show version")
print (output)
device.disconnect()

The output of the execution of code against a router is as follows:

As we can see in the sample code, we call the ConnectHandler function from the Netmiko library, which takes four inputs (platform type, IP address of device, username, and password):

Netmiko works with a variety of vendors. Some of the supported platform types and their abbreviations to be called in Netmiko are as follows:

a10: A10SSH,
accedian: AccedianSSH,
alcatel_aos: AlcatelAosSSH,
alcatel_sros: AlcatelSrosSSH,
arista_eos: AristaSSH,
aruba_os: ArubaSSH,
avaya_ers: AvayaErsSSH,
avaya_vsp: AvayaVspSSH,
brocade_fastiron: BrocadeFastironSSH,
brocade_netiron: BrocadeNetironSSH,
brocade_nos: BrocadeNosSSH,
brocade_vdx: BrocadeNosSSH,
brocade_vyos: VyOSSSH,
checkpoint_gaia: CheckPointGaiaSSH,
ciena_saos: CienaSaosSSH,
cisco_asa: CiscoAsaSSH,
cisco_ios: CiscoIosBase,
cisco_nxos: CiscoNxosSSH,
cisco_s300: CiscoS300SSH,
cisco_tp: CiscoTpTcCeSSH,
cisco_wlc: CiscoWlcSSH,
cisco_xe: CiscoIosBase,
cisco_xr: CiscoXrSSH,
dell_force10: DellForce10SSH,
dell_powerconnect: DellPowerConnectSSH,
eltex: EltexSSH,
enterasys: EnterasysSSH,
extreme: ExtremeSSH,
extreme_wing: ExtremeWingSSH,
f5_ltm: F5LtmSSH,
fortinet: FortinetSSH,
generic_termserver: TerminalServerSSH,
hp_comware: HPComwareSSH,
hp_procurve: HPProcurveSSH,
huawei: HuaweiSSH,
juniper: JuniperSSH,
juniper_junos: JuniperSSH,
linux: LinuxSSH,
mellanox_ssh: MellanoxSSH,
mrv_optiswitch: MrvOptiswitchSSH,
ovs_linux: OvsLinuxSSH,
paloalto_panos: PaloAltoPanosSSH,
pluribus: PluribusSSH,
quanta_mesh: QuantaMeshSSH,
ubiquiti_edge: UbiquitiEdgeSSH,
vyatta_vyos: VyOSSSH,
vyos: VyOSSSH

Depending upon the selection of the platform type, Netmiko can understand the returned prompt and the correct way to SSH into the specific device. Once the connection is made, we can send commands to the device using the send_command method.

Once we get the return value, the value stored in the output variable is displayed, which is the string output of the command that we sent to the device. The last line, which uses the disconnect function, ensures that the connection is terminated cleanly once we are done with our task.

For configuration (for example, we need to provide a description to the FastEthernet 0/0 router interface), we use Netmiko, as shown in the following example:

from netmiko import ConnectHandler

print ("Before config push")
device = ConnectHandler(device_type='cisco_ios', ip='192.168.255.249', username='cisco', password='cisco')
output = device.send_command("show running-config interface fastEthernet 0/0")
print (output)

configcmds=["interface fastEthernet 0/0", "description my test"]
device.send_config_set(configcmds)

print ("After config push")
output = device.send_command("show running-config interface fastEthernet 0/0")
print (output)

device.disconnect()

The output of the execution of the preceding code is as follows:

As we can see, for config push, we do not have to perform any additional configurations but just specify the commands in the same order as we send them manually to the router in a list, and pass that list as an argument to the send_config_set function. The output in Before config push is a simple output of the FastEthernet0/0 interface, but the output under After config push now has the description that we configured using the list of commands.

In a similar way, we can pass multiple commands to the router, and Netmiko will go into configuration mode, write those commands to the router, and exit config mode.

If we want to save the configuration, we use the following command after the send_config_set command:

device.send_command("write memory")

This ensures that the router writes the newly pushed configuration in memory.

Additionally, for reference purposes across the book, we will be referring to the following GNS3 simulated network:

In this topology, we have connected four routers with an Ethernet switch. The switch is connected to the local loopback interface of the computer, which provides the SSH connectivity to all the routers.

We can simulate any type of network device and create topology based upon our specific requirements in GNS3 for testing and simulation. This also helps in creating complex simulations of any network for testing, troubleshooting, and configuration validations.

The IP address schema used is the following:

• rtr1: 192.168.20.1
• rtr2: 192.168.20.2
• rtr3: 192.168.20.3
• rtr4: 192.168.20.4
• Loopback IP of computer: 192.168.20.5

The credentials used for accessing these devices are the following:

• Username: test
• Password: test

Let us start from the first step by pinging all the routers to confirm their reachability from the computer. The code is as follows:

import socket
import os
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
for n in range(1, 5):
    server_ip="192.168.20.{0}".format(n)
    rep = os.system('ping ' + server_ip)
    if rep == 0:
        print ("server is up" ,server_ip)
    else:
        print ("server is down" ,server_ip)

The output of running the preceding code is as follows:

As we can see in the preceding code, we use the range command to iterate over the IPs 192.168.20.1-192.168.20.4. The server_ip variable in the loop is provided as an input to the ping command, which is executed for the response. The response stored in the rep variable is validated with a value of 0 stating that the router can be reached, and a value of 1 means the router is not reachable.

As a next step, to validate whether the routers can successfully respond to SSH, let us fetch the value of uptime from the show version command:

show version | in uptime

The code is as follows:

from netmiko import ConnectHandler

username = 'test'
password="test"
for n in range(1, 5):
ip="192.168.20.{0}".format(n)
device = ConnectHandler(device_type='cisco_ios', ip=ip, username='test', password='test')
output = device.send_command("show version | in uptime")
print (output)
device.disconnect()

The output of running the preceding command is as follows:

Using Netmiko, we fetched the output of the command from each of the routers and printed a return value. A return value for all the devices confirms SSH attainability, whereas failure would have returned an exception, causing the code to abruptly end for that particular router.

If we want to save the configuration, we use the following command after the send_config_set command:

device.send_command("write memory")

This ensures that the router writes the newly pushed configuration in memory.

Network device configuration using template

With all the routers reachable and accessible through SSH, let us configure a base template that sends the Syslog to a Syslog server and additionally ensures that only information logs are sent to the Syslog server. Also, after configuration, a validation needs to be performed to ensure that logs are being sent to the Syslog server.

The logging server info is as follows:

• Logging server IP: 192.168.20.5 
• Logging port: 514
• Logging protocol: TCP

Additionally, a loopback interface (loopback 30) needs to be configured with the {rtr} loopback interface description.

The code lines for the template are as follows:

logging host 192.168.20.5 transport tcp port 514
logging trap 6
interface loopback 30
description "{rtr} loopback interface"

To validate that the Syslog server is reachable and that the logs sent are informational, use the show logging command. In the event that  the output of the command contains the text:

• Trap logging: level informational: This confirms that the logs are sent as informational
• Encryption disabled, link up: This confirms that the Syslog server is reachable

The code to create the configuration, push it on to the router and perform the validation, is as follows:

from netmiko import ConnectHandler
template="""logging host 192.168.20.5 transport tcp port 514
logging trap 6
interface loopback 30
description "{rtr} loopback interface\""""

username = 'test'
password="test"

#step 1
#fetch the hostname of the router for the template
for n in range(1, 5):
ip="192.168.20.{0}".format(n)
device = ConnectHandler(device_type='cisco_ios', ip=ip, username='test', password='test')
output = device.send_command("show run | in hostname")
output=output.split(" ")
hostname=output[1]
generatedconfig=template.replace("{rtr}",hostname)

#step 2
#push the generated config on router
#create a list for generateconfig
generatedconfig=generatedconfig.split("\n")
device.send_config_set(generatedconfig)

#step 3:
#perform validations
print ("********")
print ("Performing validation for :",hostname+"\n")
output=device.send_command("show logging")
if ("encryption disabled, link up"):
print ("Syslog is configured and reachable")
else:
print ("Syslog is NOT configured and NOT reachable")
if ("Trap logging: level informational" in output):
print ("Logging set for informational logs")
else:
print ("Logging not set for informational logs")

print ("\nLoopback interface status:")
output=device.send_command("show interfaces description | in loopback interface")
print (output)
print ("************\n")

The output of running the preceding command is as follows:

Another key aspect to creating network templates is understanding the type of infrastructure device for which the template needs to be applied.

As we generate the configuration form templates, there are times when we want to save the generated configurations to file, instead of directly pushing on devices. This is needed when we want to validate the configurations or even keep a historic repository for the configurations that are to be applied on the router. Let us look at the same example, only this time, the configuration will be saved in files instead of writing back directly to routers.

The code to generate the configuration and save it as a file is as follows:

from netmiko import ConnectHandler
import os
template="""logging host 192.168.20.5 transport tcp port 514
logging trap 6
interface loopback 30
description "{rtr} loopback interface\""""

username = 'test'
password="test"

#step 1
#fetch the hostname of the router for the template
for n in range(1, 5):
ip="192.168.20.{0}".format(n)
device = ConnectHandler(device_type='cisco_ios', ip=ip, username='test', password='test')
output = device.send_command("show run | in hostname")
output=output.split(" ")
hostname=output[1]
generatedconfig=template.replace("{rtr}",hostname)

#step 2
#create different config files for each router ready to be pushed on routers.
configfile=open(hostname+"_syslog_config.txt","w")
configfile.write(generatedconfig)
configfile.close()

#step3 (Validation)
#read files for each of the router (created as routername_syslog_config.txt)
print ("Showing contents for generated config files....")
for file in os.listdir('./'):
if file.endswith(".txt"):
if ("syslog_config" in file):
hostname=file.split("_")[0]
fileconfig=open(file)
print ("\nShowing contents of "+hostname)
print (fileconfig.read())
fileconfig.close()

The output of running the preceding command is as follows:

In a similar fashion to the previous example, the configuration is now generated. However, this time, instead of being pushed directly on routers, it is stored in different files with filenames based upon router names for all the routers that were provided in input. In each case, a .txt file is created (here is a sample filename that will be generated during execution of the script: rtr1_syslog_config.txt for the rtr1 router).

As a final validation step, we read all the .txt files and print the generated configuration for each of the text files that has the naming convention containing syslog_config in the filename.

There are times when we have a multi-vendor environment, and to manually create a customized configuration is a difficult task. Let us see an example in which we leverage a library (PySNMP) to fetch details regarding the given devices in the infrastructure using Simple Network Management Protocol (SNMP).

For our test, we are using the SNMP community key mytest on the routers to fetch their model/version.

The code to get the version and model of router, is as follows:

#snmp_python.py
from pysnmp.hlapi import *
for n in range(1, 3):
server_ip="192.168.20.{0}".format(n)
errorIndication, errorStatus, errorIndex, varBinds = next(
getCmd(SnmpEngine(),
CommunityData('mytest', mpModel=0),
UdpTransportTarget((server_ip, 161)),
ContextData(),
ObjectType(ObjectIdentity('SNMPv2-MIB', 'sysDescr', 0)))
)
print ("\nFetching stats for...", server_ip)
for varBind in varBinds:
print (varBind[1])

The output of running the preceding command is as follows:

As we see in this, the SNMP query was performed on a couple of routers (192.168.20.1 and 192.168.20.2). The SNMP query was performed using the standard Management Information Base (MIB), sysDescr. The return value of the routers against this MIB request is the make and model of the router and the current OS version it is running on.

Using SNMP, we can fetch many vital statistics of the infrastructure and can generate configurations based upon the return values. This ensures that we have standard configurations even with a multi-vendor environment.

As a sample, let us use the SNMP approach to determine the number of interfaces that a particular router has and, based upon the return values, we can dynamically generate a configuration irrespective of any number of interfaces available on the device.

The code to fetch the available interfaces in a router is as follows:

#snmp_python_interfacestats.py
from pysnmp.entity.rfc3413.oneliner import cmdgen
cmdGen = cmdgen.CommandGenerator()

for n in range(1, 3):
server_ip="192.168.20.{0}".format(n)
print ("\nFetching stats for...", server_ip)
errorIndication, errorStatus, errorIndex, varBindTable = cmdGen.bulkCmd(
cmdgen.CommunityData('mytest'),
cmdgen.UdpTransportTarget((server_ip, 161)),
0,25,
'1.3.6.1.2.1.2.2.1.2'
)

for varBindTableRow in varBindTable:
for name, val in varBindTableRow:
print('%s = Interface Name: %s' % (name.prettyPrint(), val.prettyPrint()))

The output of running the preceding command is as follows:

Using the snmpbulkwalk, we query for the interfaces on the router. The result from the query is a list that is parsed to fetch the SNMP MIB ID for the interfaces, along with the description of the interface.

Multithreading

A key focus area while performing operations on multiple devices is how quickly we can perform the actions. To put this into perspective, if each router takes around 10 seconds to log in, gather the output, and log out, and we have around 30 routers that we need to get this information from, we would need 10*30 = 300 seconds for the program to complete the execution. If we are looking for more advanced or complex calculations on each output, which might take up to a minute, then it will take 30 minutes for just 30 routers.

This starts becoming very inefficient when our complexity and scalability grows. To help with this, we need to add parallelism to our programs.  Let us log in to each of the routers and fetch the show version using a parallel calling (or multithreading):

#parallel_query.py
from netmiko import ConnectHandler
from datetime import datetime
from threading import Thread
startTime = datetime.now()

threads = []
def checkparallel(ip):
device = ConnectHandler(device_type='cisco_ios', ip=ip, username='test', password='test')
output = device.send_command("show run | in hostname")
output=output.split(" ")
hostname=output[1]
print ("\nHostname for IP %s is %s" % (ip,hostname))

for n in range(1, 5):
ip="192.168.20.{0}".format(n)
t = Thread(target=checkparallel, args= (ip,))
t.start()
threads.append(t)

#wait for all threads to completed
for t in threads:
t.join()

print ("\nTotal execution time:")
print(datetime.now() - startTime)

The output of running the preceding command is as follows:

The calling to the same set of routers being done in parallel takes approximately 8 seconds to fetch the results.

Summary

In this tutorial, we learned how to interact with Network devices through Python and got familiar with an extensively used library of Python (Netmiko) for network interactions. You also learned how to interact with multiple network devices using a simulated lab in GNS3 and got to know the device interaction through SNMP. Additionally, we also touched base on multithreading, which is a key component in scalability through various examples.

To learn how to make your network robust by leveraging the power of Python, Ansible and other network automation tools, check out our book Practical Network Automation - Second Edition.


AWS announces more flexibility its Certification Exams, drops its exam prerequisites

Top 10 IT certifications for cloud and networking professionals in 2018

What matters on an engineering resume? Hacker Rank report says skills, not certifications