What is the internet? This seemingly easy question might receive different answers depending on your background. The internet means different things to different people, the young, the older, the student, the teacher, the business person, a poet, all could have a different answer to the question.

To a network engineer and systems engineer by extension, the internet is a global computer network providing a variety of information. This global computer network system is actually a web of internetwork connecting large and small networks together. Imagine your home network; it will consist of a home switch connecting your smart phone, tablet, computers, and TV together, so they can communicate with each other. Then, when it needs to communicate to the outside world, it passes the information on to the home router that connects your home network to a larger network, often appropriately called the Internet Service Provider (ISP). Your ISP often consists of edge nodes that aggregate the traffic to their core network. The core network's function is to interconnect these edge networks via a higher speed network. At special edge nodes, your ISP is connected to other ISPs to pass your traffic appropriately to your destination. The return path from your destination to your home computer, tablet, or smart phone may or may not follow the same path through all of these networks back to your screen.

Let's take a look at the components making up this web of networks.

Hosts are end nodes on the networks that communicate to other nodes. In today's world, a host can be the traditional computer, or it can be your smart phone, tablet, or TV. With the rise of Internet of Things (IoT), the broad definition of host can be expanded to include IP camera, TV set-top boxes, and the ever-increasing type of sensors that we use in agriculture, farming, automobiles, and more. With the explosion of the number of hosts connected to the internet, all of them need to be addressed, routed, and managed, and the demand for proper networking has never been greater.

Most of the time when we are on the internet, we request for services. The service can be a web page, sending or receiving emails, transferring files, and such. These services are provided by servers. As the name implies, they provide services to multiple nodes and generally have higher levels of hardware specification because of it. In a way, servers are special super nodes on the network that provide additional capabilities to its peers. We will look at servers later on in the client-server section.

If one thinks of servers and hosts as cities and towns, the network components are the roads and highways to connect them together. In fact, the term "information superhighway" comes to mind when describing the network components that transmit the ever increasing bits and bytes across the globe. In the OSI model that we will look at in a bit, these network components are the routers and switches that reside on layer two and three of the model as well as the layer on fiber optics cable, coaxial cable, twisted copper pairs, and some DWDM equipment, to name a few.

Collectively, the hosts, servers, and network components make up the internet as we know today.

In the last section, we looked at different roles that servers, hosts, and network components play in the internetwork. Because of the higher hardware capacity that the servers demand, they are often put together in a central location, so they can be managed more efficiently. We will refer to these locations as datacenters.

In a typical enterprise, the company generally has the need for internal tools such as e-mails, document storage, sales tracking, ordering, HR tools, and knowledge sharing intranet. These terms translate into file and mail servers, database servers, and web servers. Unlike user computers, these are generally high-end computers that require higher power, cooling, and network connection. A byproduct of the hardware is also the amount of noise they make. They are generally placed in a central location called the Main Distribution Frame (MDF) in the enterprise to provide the necessary power feed, power redundancy, cooling, and network speed.

To connect to the MDF, the user's traffic is generally aggregated at a location, which is sometimes called the Intermediate Distribution Frame (IDF) before they are connected to the MDF. It is not unusual for the IDF-MDF spread to follow the physical layout of the enterprise building or campus. For example, each building floor can consist of an IDF that aggregates to the MDF on another floor. If the enterprise consists of several buildings, further aggregation can be done by combining the building traffic before connecting them to the enterprise datacenter.

The enterprise datacenters generally follow the three layers of access, distribution, and core. The access layer is analogous to the ports each user connects to; the IDF can be thought of as the distribution layer, while the core layer consists of the connection to the MDF and the enterprise datacenters. This is, of course, a generalization of enterprise networks as some of them will not follow the same model.

With the rise of cloud computing and software or infrastructure as a service, we can say that the datacenters cloud providers build the cloud datacenters. Because of the number of servers they house, they generally demand a much, much higher capacity of power, cooling, network speed, and feed than any enterprise datacenter. In fact, the cloud datacenters are so big they are typically built close to power sources where they can get the cheapest rate for power, without losing too much during transportation of the power. They can also be creative when it comes to cooling down where the datacenter might be build in a generally cold climate, so they can just open the doors and windows to keep the server running at a safe temperature. Any search engine can give you some of the astounding numbers when it comes to the science of building and managing these cloud datacenters for the likes of Amazon, Microsoft, Google, and Facebook:

The services that the servers at datacenters need to provide are generally not cost efficient to be housed in any single server. They are spread among a fleet of servers, sometimes across many different racks to provide redundancy and flexibility for service owners. The latency and redundancy requirements put a tremendous amount of pressure on the network. The number of interconnection equates to an explosive growth of network equipment; this translates into the number of times these network equipment need to be racked, provisioned, and managed.

In a way, the cloud datacenter is where network automation becomes a necessity. If we follow the traditional way of managing the network devices via a Terminal and command-line interface, the number of engineering hours required would not allow the service to be available in a reasonable amount of time. This is not to mention that human repetition is error prone, inefficient, and a terrible waste of engineering talent.

The cloud datacenter is where the author started his path of network automation with Python a number of years ago, and never looked back since.

If we have sufficient computing power at the datacenter level, why keep anything anywhere else but at these datacenters? All the connections will be routed back to the server providing the service, and we can call it a day. The answer, of course, depends on the use case. The biggest limitation in routing back the request and session all the way back to the datacenter is the latency introduced in the transport. In other words, network is the bottleneck. As fast as light travels in a vacuum, the latency is still not zero. In the real world, the latency would be much higher when the packet is traversing through multiple networks and sometimes through undersea cable, slow satellite links, 3G or 4G cellular links, or Wi-Fi connections.

The solution? Reduce the number of networks the end user traverse through as much as possible by one. Be as directly connected to the user as possible; and two: place enough resources at the edge location. Imagine if you are building the next generation of video streaming service. In order to increase customer satisfaction with a smooth streaming, you would want to place the video server as close to the customer as possible, either inside or very near to the customer's ISP. Also, the upstream of my video server farm would not just be connected to one or two ISPs, but all the ISPs that I can connect to with as much bandwidth as needed. This gave rise to the peering exchanges edge datacenters of big ISP and content providers. Even when the number of network devices are not as high as the cloud datacenters, they too can benefit from network automation in terms of the increased security and visibility automation brings. We will cover security and visibility in later chapters of this book.

No network book seems to be complete without first going over the Open System Interconnection (OSI) model. The model is a conceptional model that componentizes the telecommunication functions into different layers. The model defines seven layers, and each layer sits independently on top of another one, as long as they follow defined structures and characteristics. For example, the network layer, such as IP, can sit on top of different type of data link layer such as the Ethernet or frame relay. The OSI reference model is a good way to normalize different and diverse technologies into a set of common language that people can agree on. This greatly reduces the scope for parties working on the individual layers and allows them to go in depth on the specific tasks without worrying about compatibility:

The OSI model was initially worked on in the late 1970's and later on published jointly by the International Organization for Standardization (ISO) and the now called Telecommunication Standardization Sector of the International Telecommunication Union (ITU-T). It is widely accepted and commonly referred to when introducing a new topic in telecommunication.

Around the same time period of the OSI model development, the internet was taking shape. The reference which model they used is often referred to as the TCP/IP model since the Transmission Control Protocol (TCP) and Internet Protocol (IP), because originally, this was what the protocol suites contained. It is somewhat similar to the OSI model in the sense that they divide the end-to-end data communication into abstraction layers. What is different is the model combined layers 5 to 7 in the OSI model into the Application layer while the Physical and Data link layers are combined into the Link layer:

Both the OSI and TCP/IP models are useful in providing a standard for providing an end-to-end data communication. However, for the most part, we will refer to the TCP/IP model more since that is what the internet was built on. We will specify the OSI model when needed, such as when we are discussing the web framework in the upcoming chapters.

The reference models demonstrated a standard way for data communication between two nodes. Of course by now, we all know that not all nodes are created equally. Even in its DARPA-net days, there were workstation nodes, and there were nodes with the purpose of serving content to other nodes. These units typically have higher hardware specification and are managed more closely by the engineers. Since these nodes provide resources and services to others, they are typically referred to as servers. The servers are typically sitting idle, waiting for clients to initiate requests for its resources. This model of distributed resources that are "asked for" by the client is referred to as the Client-server model.

Why is this important? If you think about it for a minute, the importance of networking is really based on this Client-server model. Without it, there is really not a lot of need for network interconnections. It is the need to transfer bits and bytes from client to server that shines a light on the importance of network engineering. Of course, we are all aware of how the biggest network of them all, the internet, is transforming the lives of all of us and continuing to do so.

How, you asked, can each node determine the time, speed, source, and destination each time they need to talk to each other? This brings us to the network protocols.

In the early days of computer networking, the protocols were proprietary and closely controlled by the company who designed the connection method. If you are using Novell's IPX/SPX protocol in your hosts, you will not able to communicate with Apple's AppleTalk hosts and vice versa. These proprietary protocol suites generally have analogous layers to the OSI reference model and follow the client-server communication method. They generally work great in Local Area Networks (LAN) that are closed without the need to communicate with the outside world. When the traffic do need to move beyond local LAN, typically an internetwork device, such as a router, is used to translate from one protocol to another such as between AppleTalk to IP. The translation is usually not perfect, but since most of the communication happens within the LAN, it is okay.

However, as the need for internetwork communication rises, the need for standardizing the network protocol suites becomes greater. These proprietary protocols eventually gave way to the standardized protocol suites of TCP, UDP, and IP that greatly enhanced the ability from one network to talk to another. The internet, the greatest network of them all, relies on these protocols to function properly. In the next few sections, we will take a look at each of the protocol suites.

The Transmission Control Protocol (TCP) is one of the main protocols used on the internet today. If you have opened a web page or have sent an e-mail, you have come across the TCP protocol. The protocol sits at layer 4 of the OSI model, and it is responsible for delivering the data segment between two nodes in a reliable and error-checked manner. The TCP consists of a 128-bit header consists of, among others, source and destination port, sequence number, acknowledgment number, control flags, and checksum:

TCP uses datagram sockets or ports to establish a host-to-host communication. The standard body called Internet Assigned Numbers Authority (IANA) designates well-known ports to indicate certain services, such as port 80 for HTTP (web) and port 25 for SMTP (mail). The server in the client-server model typically listens on one of these well-known ports in order to receive communication requests from the client. The TCP connection is managed by the operating system by the socket that represents the local endpoint for connection.

The protocol operation consist of a state machine, where the machine needs to keep track of when it is listening for incoming connection, during communication session, as well as releasing resources once the connection is closed. Each TCP connection goes through a series of states such as Listen, SYN-SENT, SYN-RECEIVED, ESTABLISHED, FIN-WAIT, CLOSE-WAIT, CLOSING, LAST-ACK, TIME-WAIT, and CLOSED.

The biggest difference between TCP and User Datagram Protocol (UDP), which is its close cousin at the same layer, is that it transmits data in an ordered and reliable fashion. The fact that the operation guarantees delivery often referred to TCP as a connection-oriented protocol. It does this by first establishing a three-way handshake to synchronize the sequence number between the transmitter and the receiver, SYN, SYN-ACK, and ACK.

The acknowledgement is used to keep track of subsequent segments in the conversation. Finally at the end of the conversation, one side will send a FIN message, the other side will ACK the FIN message as well as send a FIN message of its own. The FIN initiator will then ACK the FIN message that it received.

As many of us who have troubleshot a TCP connection can tell you, the operation can get quite complex. One can certainly appreciate that most of the time, the operation just happens silently in the background.

A whole book can be written about the TCP protocol; in fact, many excellent books have been written on the protocol.

UDP is also a core member of the internet protocol suite. Like TCP, it is on layer 4 of the OSI model that is responsible for delivering data segments between the application and the IP layer. Unlike TCP, their header is only 32-bit that only consists of source and destination port, length, and checksum. The lightweight header makes it ideal for the application to prefer a faster data delivery without setting up the session between two hosts or needing a reliable delivery of data. Perhaps it is hard to imagine in today's fast internet connections, but the extra header made a big difference in the speed of transmission in the early days of X.21 and frame relay links. Although as important as the speed save, not having to maintain various states such as TCP also saves compute resources on the two endpoints.

You might now wonder why UDP was ever used at all in the modern age; given the lack of reliable transmission, wouldn't we want all the connections to be reliable and error free? If one thinks about some of the multimedia video streaming or Skype call, those applications will benefit from a lighter header when the application just wants to deliver the datagram as fast as possible. You can also consider the DNS lookup process. When the address you type in on the browser is translated into a computer understandable address, the user will benefit from the lightweight process since this has to happen "before" even the first bit of information is delivered to you from your favorite website.

Again, this section does not do justice to the topic of UDP, and the reader is encouraged to explore the topic through various resources if he/she is interested in learning more about UDP.

As network engineers would tell you, network engineers "live" at the IP layer, which is layer 3 on the OSI model. IP has the job of addressing and routing between end nodes, among others. The addressing of IP is probably its most important job. The address space is divided into two parts: the network and the host portion. The subnet mask indicated which portion in the address consist of the network and which is the host. Both IPv4, and later, IPv6 expresses the address in the dotted notation, for example 192.168.0.1. The subnet mask can either be in a dotted notation (255.255.255.0) or use a forward slash to express the number of bits that should be considered in the network bit (/24).

The IPv6 header, next generation of IP header of IPv4, has a fixed portion and various extension headers.

The Next Header field in the fixed header section can indicate an extension header to be followed that carries additional information. The extension headers can include routing and fragment information. As much as the protocol designer would like to move from IPv4 to IPv6, the internet today is still pretty much addressed with IPv4.

Network Address Translation (NAT) is typically used for translating a range of private IPv4 addresses to publicly routable IPv4 addresses. But it can also mean a translation between IPv4 to IPv6, such as at a carrier edge when they use IPv6 inside of the network that needs to be translated to IPv4, when the packet leaves the network. Sometimes, NAT6to6 is used as well for security reasons.

Security is a continuous process that integrates all the aspects of networking, including automation and Python. This book aims at using Python to help you manage the network; security will be addressed as part of the chapter such as using SSHv2 over telnet. We will also look at how we can use Python and other tool chains to gain visibility in the network.

In my opinion, IP routing is about having the intermediate devices between the two endpoint transmit the packets between them based the IP header. For every communication on the internet, the packet will traverse through various intermediate devices. As mentioned, the intermediate devices consist of routers, switches, optical gears, and various other gears that do not examine beyond the network and transport layer. In a road trip analogy, you might travel in the United States from the city of San Diego in California to the city of Seattle in Washington. The IP source address is analogous to San Diego and the destination IP address can be thought of as Seattle. On your road trip, you will stop by many different intermediate spots, such as Los Angeles, San Francisco, and Portland; these can be thought of as the routers and switches between the source and destination.

Why was this important? In a way, this book is about managing and optimizing these intermediate devices. In the age of mega datacenters that span sizes of multiple American football fields, the need for efficient, agile, cost effective way to manage the network becomes a major point of competitive advantage for companies. In the future chapters, we will dive into how we can use Python programming to effectively manage the network.

In a nutshell, this book is about making our lives easier with Python. But what is Python and why is it the language of choice by many DevOps engineers? In the words of the Python Foundation Executive Summary (https://www.python.org/doc/essays/blurb/):

If you are somewhat new to programming, the object-oriented, dynamic semantics probably does not mean much to you. But I think we can all agree on Rapid application development, simple, and easy-to-learn syntax sounds like a good thing. Python as an interpreted language means there is no compilation process required, so the speed of write, test, and edit of the program process is greatly increased. For simple scripts, if your script fails, a few print statement is usually all you need to debug what was going on. Using the interpreter also means that Python is easily ported to different type of operating systems and Python program written on Windows can be used on Linux and Mac.

The object-oriented nature encourages code reuse by breaking into simple reusable form such modules and packages. In fact, all Python files are modules that can be reused or imported in another Python program. This makes it easy to share programs among engineers and encourage code reuse. Python also has a batteries included mantra, which means that for common tasks, you need not download any additional code. In order to achieve this without the code being too bloated, a set of standard libraries is installed when you install the Python interpreter. For common tasks such as regular expression, mathematics functions, and JSON decoding, all you need is the import statement, and the interpreter will move those functions into your program. This is what I would consider one of the killer features of the Python language.

Lastly, the fact that Python code can start in a relatively small-sized script with a few lines of code and grow into a fully production system is very handy for network engineers. As many of us know, the network typically grows organically without a master plan. A language that can grow with your network in size is invaluable. You might be surprised to see a language that was deemed as scripting language by many, which was being used for fully production systems (Organizations using Python, https://wiki.python.org/moin/OrganizationsUsingPython).

If you have ever worked in an environment where you have to switch between working on different vendor platforms, such as Cisco IOS and Juniper Junos, you know how painful it is to switch between syntax and usage when trying to achieve the same task. With Python being flexible enough for large and small programs, there is no such context switching, because it is just Python.

For the rest of the chapter, we will take a high-level tour of the Python language for a bit of a refresher. If you are already familiar with the basics, feel free to quickly scan through it or skip the rest of the chapter.

At the time of writing this book in early 2017, Python is going through a transition period of moving from Python version 2 to Python version 3. Unfortunately Python 3 is not backward compatible with Python 2. When I say transition, keep in mind that Python 3 was released back in 2008, over nine years ago with active development with the most recent release of 3.6. The latest Python 2.x release, 2.7, was released over six years ago in mid 2010. Fortunately, both version can coexist on the same machine. Personally, I use Python 2 as my default interpreter when I type in Python at the Command Prompt, and I use Python 3 when I need to use Python 3. More information is given in the next section about invoking Python interpreter, but here is an example of invoking Python 2 and Python 3 on the Ubuntu Linux machine:

    echou@pythonicNeteng:~$ python
    Python 2.7.12 (default, Nov 19 2016, 06:48:10)
    [GCC 5.4.0 20160609] on linux2
    Type "help", "copyright", "credits" or "license" for 
    more information.
    >>> exit()
    echou@pythonicNeteng:~$ python3
    Python 3.5.2 (default, Nov 17 2016, 17:05:23)
    [GCC 5.4.0 20160609] on linux
    Type "help", "copyright", "credits" or "license" for 
    more information.
    >>> exit()

With the 2.7 release being end of life on extended support only for security updates, most Python frameworks are now supporting Python 3. Python 3 also has lots of good features such as asynchronous I/O that can be taken advantage of when we need to optimize our code. This book will use Python 3 for its code examples.

If the particular library or the framework does not support Python 3, such as Ansible (they are actively working on porting to Python 3), it will be pointed out, so you can use Python 2 instead.

As mentioned, Python is cross platform. Python programs can be run on Windows, Mac, and Linux. In reality, certain care needs to be taken when you need to ensure cross-platform compatibility, such as taking care of the subtle difference backslashes in Windows filenames. Since this book is for DevOps, systems, and network engineers, Linux is the preferred platform for the intended audience, especially in production. The code in this book will be tested on the Linux Ubuntu 16.06 LTS machine. I will also try my best to make sure the code runs the same on Windows and the Mac platform.

If you are interested in the OS details, they are as follows:

    echou@pythonicNeteng:~$ uname -a
    Linux pythonicNeteng 4.4.0-31-generic #50-Ubuntu SMP 
    Wed Jul 13 00:07:12 UTC 2016 x86_64 x86_64 x86_64 
    GNU/Linux
    echou@pythonicNeteng:~$

Python programs are executed by an interpreter, which means the code is fed through this interpreter to be executed by the underlying operating system, and results are displayed . There are several different implementation of the interpreter by the Python development community, such as IronPython and Jython. In this book, we will refer to the most common Python interpreter, CPython, which is in use today.

One way you can use Python is by taking the advantage of the interactive prompt. This is useful when you want to quickly test a piece of Python code or concept without writing a whole program. This is typically done by simply typing in the Python keyword:

    Python 3.5.2 (default, Nov 17 2016, 17:05:23)
    [GCC 5.4.0 20160609] on linux
    Type "help", "copyright", "credits" or "license" for 
    more information.
    >>> print("hello world")
    hello world
    >>>

The interactive mode is one of the Python's most useful features. In the interactive shell, you can type any valid statement or sequence of statements and immediately get a result back. I typically use this to explore around a feature or library that I am not familiar with. Talk about instant gratification!

A more common way to run the Python program, however, is to save your Python file and run via the interpreter after. This will save you from typing the same statements over and over again such as in the interactive shell. Python files are just regular text files that are typically saved with the .py extension. In the *Nix world, you can also add the shebang (#!) line on top to specify the interpreter that is used to run the file. The # character can be used to specify comments that will not be executed by the interpreter. The following file, helloworld.py, has the following statements:

    # This is a comment
    print("hello world") 

This can be executed as follows:

    echou@pythonicNeteng:~/Master_Python_Networking/
    Chapter1$ python helloworld.py
    hello world
    echou@pythonicNeteng:~/Master_Python_Networking/
    Chapter1$

Python has several standard types built into the interpreter:

• None: The Null object
• Numerics: int, long, float, complex, and bool (The subclass of int with True or False value)
• Sequences: str, list, tuple, and range
• Mappings: dict
• Sets: set and frozenset

The None type denotes an object with no value. This is returned in functions that do not explicitly return anything. The None type is also used in function arguments to error out if the caller does not pass in an actual value.

Python numeric objects are basically numbers. With the exception of Boolean, the numeric types of int, long, float, and complex are all signed, meaning they can be positive or negative. Boolean is a subclass of the integer that can be one of two values: 1 for True, and 0 for False. The rest of the numeric types are differentiated by how precise they can represent the number; for example, int are whole numbers with a limited range while long are whole numbers with unlimited range. Float are numbers using the double-precision representation (64-bit) on the machine.

Sequences are ordered sets of objects with an index of non-negative integers. In this and the next few sections, we will use the interactive interpreter to illustrate the different types. Please feel free to type along on your own computer.

Sometimes it surprises people that string is actually a sequence type. But if you look closely, strings are series of characters put together. Strings are enclosed by either single, double, or triple quotes. Note in the following examples, the quotes have to match, and triple quotes allow the string to span different lines:

    >>> a = "networking is fun"
    >>> b = 'DevOps is fun too'
    >>> c = """what about coding?
    ... super fun!"""
    >>>

The other two commonly used sequence types are lists and tuples. Lists are the sequence of arbitrary objects. Lists can be created by enclosing the objects in square brackets. Just like string, lists are indexed by non-zero integers that start at zero. The values of lists are retrieved by referencing the index number:

    >>> vendors = ["Cisco", "Arista", "Juniper"]
    >>> vendors[0]
    'Cisco'
    >>> vendors[1]
    'Arista'
    >>> vendors[2]
    'Juniper'

Tuples are similar to lists, created by enclosing the values in parentheses. Like lists, the values in the tuple are retrieved by referencing its index number. Unlike list, the values cannot be modified after creation:

    >>> datacenters = ("SJC1", "LAX1", "SFO1")
    >>> datacenters[0]
    'SJC1'
    >>> datacenters[1]
    'LAX1'
    >>> datacenters[2]
    'SFO1' 

Some operations are common to all sequence types, such as returning an element by index as well as slicing:

    >>> a
    'networking is fun'
    >>> a[1]
    'e'
    >>> vendors
    ['Cisco', 'Arista', 'Juniper']
    >>> vendors[1]
    'Arista'
    >>> datacenters
    ('SJC1', 'LAX1', 'SFO1')
    >>> datacenters[1]
    'LAX1'
    >>>
    >>> a[0:2]
    'ne'
    >>> vendors[0:2]
    ['Cisco', 'Arista']
    >>> datacenters[0:2]
    ('SJC1', 'LAX1')
    >>>

There are also common functions that can be applied to sequence types, such as checking the number of elements and minimum and maximum values:

    >>> len(a)
    17
    >>> len(vendors)
    3
    >>> len(datacenters)
    3
    >>>
    >>> b = [1, 2, 3, 4, 5]
    >>> min(b)
    1
    >>> max(b)
    5

It would come as no surprise that there are various methods that apply only to strings. It is worth noting that these methods do not modify the underlying string data itself and always return a new string. If you want to use the new value, you would need to catch the return value and assign it to a different variable:

    >>> a
    'networking is fun'
    >>> a.capitalize()
    'Networking is fun'
    >>> a.upper()
    'NETWORKING IS FUN'
    >>> a
    'networking is fun'
    >>> b = a.upper()
    >>> b
    'NETWORKING IS FUN'
    >>> a.split()
    ['networking', 'is', 'fun']
    >>> a
    'networking is fun'
    >>> b = a.split()
    >>> b
    ['networking', 'is', 'fun']
    >>>

Here are some of the common methods for a list. This list is a very useful structure in terms of putting multiple items in and iterating through them. For example, we can make a list of datacenter spine switches and apply the same access list to all of them by iterating through them one by one. Since a list's value can be modified after creation (unlike tuple), we can also expand and contrast the existing list as we move along the program:

    >>> routers = ['r1', 'r2', 'r3', 'r4', 'r5']
    >>> routers.append('r6')
    >>> routers
    ['r1', 'r2', 'r3', 'r4', 'r5', 'r6']
    >>> routers.insert(2, 'r100')
    >>> routers
    ['r1', 'r2', 'r100', 'r3', 'r4', 'r5', 'r6']
    >>> routers.pop(1)
    'r2'
    >>> routers
    ['r1', 'r100', 'r3', 'r4', 'r5', 'r6']

Python provides one mapping type called dictionary. Dictionary is what I think of as a poor man's database, because it contains objects that can be indexed by keys. This is often referred to as the associated array or hashing table in other languages. If you have used any of the dictionary-like objects in other languages, you will know this is a powerful type, because you can refer to the object by a readable key. This key will make more sense for the poor guy who is trying to maintain the code, probably a few months after you wrote the code at 2 am in the morning. The object can also be another data type, such as a list. You can create a dictionary with curly braces:

    >>> datacenter1 = {'spines': ['r1', 'r2', 'r3', 'r4']}
    >>> datacenter1['leafs'] = ['l1', 'l2', 'l3', 'l4']
    >>> datacenter1
    {'leafs': ['l1', 'l2', 'l3', 'l4'], 'spines': ['r1',  
    'r2', 'r3', 'r4']}
    >>> datacenter1['spines']
    ['r1', 'r2', 'r3', 'r4']
    >>> datacenter1['leafs']
    ['l1', 'l2', 'l3', 'l4']

A set is used to contain an unordered collection of objects. Unlike lists and tuples, sets are unordered and cannot be indexed by numbers. But there is one character that makes sets standout and useful: the elements of a set are never duplicated. Imagine if you have a list of IPs that you need to put in an access list of. The only problem in this list of IPs is that they are full of duplicates. Now, think about how you would use a loop to sort it out for the unique items, the set built-in type would allow you to eliminate the duplicate entries with just one line of code. To be honest, I do not use set that much, but when I need it, I am always very thankful this exists. Once the set or sets are created, they can be compared with each other ones using the union, intersection, and differences:

    >>> a = "hello"
    >>> set(a)
    {'h', 'l', 'o', 'e'}
    >>> b = set([1, 1, 2, 2, 3, 3, 4, 4])
    >>> b
    {1, 2, 3, 4}
    >>> b.add(5)
    >>> b
    {1, 2, 3, 4, 5}
    >>> b.update(['a', 'a', 'b', 'b'])
    >>> b
    {1, 2, 3, 4, 5, 'b', 'a'}
    >>> a = set([1, 2, 3, 4, 5])
    >>> b = set([4, 5, 6, 7, 8])
    >>> a.intersection(b)
    {4, 5}
    >>> a.union(b)
    {1, 2, 3, 4, 5, 6, 7, 8}
    >>> 1 *
    {1, 2, 3}
    >>>

Python has the some numeric operators that you would expect; note that the truncating division, (//, also known as floor division) truncates the result to an integer and a floating point and return the integer value. The integer value is returned. The modulo (%) operator returns the remainder value in the division:

    >>> 1 + 2
    3
    >>> 2 - 1
    1
    >>> 1 * 5
    5
    >>> 5 / 1
    5.0
    >>> 5 // 2
    2
    >>> 5 % 2
    1

There are also comparison operators:

    >>> a = 1
    >>> b = 2
    >>> a == b
    False
    >>> a > b
    False
    >>> a < b
    True
    >>> a <= b
    True 

We will also see two of the common membership operators to see whether an object is in a sequence type:

    >>> a = 'hello world'
    >>> 'h' in a
    True
    >>> 'z' in a
    False
    >>> 'h' not in a
    False
    >>> 'z' not in a
    True

The if, else, and elif statements control conditional code execution. As one would expect, the format of the conditional statement is as follows:

    if expression:
      do something
    elif expression:
      do something if the expression meets
    elif expression:
      do something if the expression meets
    ...
    else:
      statement

Here is a simple example:

    >>> a = 10
    >>> if a > 1:
    ...   print("a is larger than 1")
    ... elif a < 1:
    ...   print("a is smaller than 1")
    ... else:
    ...   print("a is equal to 1")
    ...
    a is larger than 1
    >>>

The while loop will continue to execute until the condition is false, so be careful with this one if you don't want to continue to execute:

    while expression:
      do something
    >>> a = 10
    >>> b = 1
    >>> while b < a:
    ...   print(b)
    ...   b += 1
    ...
    1
    2
    3
    4
    5
    6
    7
    8
    9

The for loop works with any object that supports iteration; this means all the built-in sequence types such as lists, tuples, and strings can be used in a for loop. The letter i in the following for loop is an iterating variable, so you can typically pick something that make sense within the context of your code:

    for i in sequence:
      do something
    >>> a = [100, 200, 300, 400]
    >>> for number in a:
    ...   print(number)
    ...
    100
    200
    300
    400

You can also make your own object that supports the iterator protocol and be able to use the for loop for this object:

Most of the time when you find yourself reusing some pieces of code, you should break up the code into a self-contained chunk as functions. This practice allows for better modularity, is easier to maintain, and allows for code reuse. Python functions are defined using the def keyword with the function name, followed by the function parameters. The body of the function consists of Python statements that are to be executed. At the end of the function, you can choose to return a value to the function caller, or by default, it will return the None object if you do not specify a return value:

    def name(parameter1, parameter2):
      statements
      return value

We will see a lot more examples of function in the following chapters, so here is a quick example:

    >>> def subtract(a, b):
    ...   c = a - b
    ...   return c
    ...
    >>> result = subtract(10, 5)
    >>> result
    5
    >>>

Python is an Object-Oriented Programming (OOP) language. The way Python creates objects are with the class keyword. A Python object is most commonly a collection of functions (methods), variables, and attributes (properties). Once a class is defined, you can create instances of such a class. The class serves as the blueprint of the subsequent instances.

The topic of object-oriented programming is outside the scope of this chapter, so here is a simple example of a router object definition:

    >>> class router(object):
    ...   def __init__(self, name, interface_number,
     vendor):
    ...     self.name = name
    ...     self.interface_number = interface_number
    ...     self.vendor = vendor
    ...
    >>>

Once defined, you are able to create as many instances of that class as you'd like:

    >>> r1 = router("SFO1-R1", 64, "Cisco")
    >>> r1.name
    'SFO1-R1'
    >>> r1.interface_number
    64
    >>> r1.vendor
    'Cisco'
    >>>
    >>> r2 = router("LAX-R2", 32, "Juniper")
    >>> r2.name
    'LAX-R2'
    >>> r2.interface_number
    32
    >>> r2.vendor
    'Juniper'
    >>>

Of course, there is a lot more to Python objects and OOP. We will see more examples in the future chapters.

Any Python source file can be used as a module, and any functions and classes you define in that source file can be re-used. To load the code, the file referencing the module needs to use the import keyword. Three things happen when the file is imported:

1. The file creates a new namespace for the objects defined in the source file.
2. The caller executes all the code contained in the module.
3. The file creates a name within the caller that refers to the module being imported. The name matches the name of the module.

Remember the subtract() function that you defined using the interactive shell? To reuse the function, we can put it into a file named subtract.py:

    def subtract(a, b):
      c = a - b
      return c

In a file within the same directory of subtract.py, you can start the Python interpreter and import this function:

    Python 2.7.12 (default, Nov 19 2016, 06:48:10)
    [GCC 5.4.0 20160609] on linux2
    Type "help", "copyright", "credits" or "license" for  
    more information.
    >>> import subtract
    >>> result = subtract.subtract(10, 5)
    >>> result
    5

This works because, by default, Python will first search for the current directory for the available modules. If you are in a different directory, you can manually add a search path location using the sys module with sys.path. Remember the standard library that we mentioned a while back? You guessed it, those are just Python files being used as modules.

Packages allow a collection of modules to be grouped together. This further organizes the Python modules into a more namespace protection to further reusability. A package is defined by creating a directory with a name you want to use as the namespace, then yo can place the module source file under that directory. In order for Python to know it as a Python-package, just create a __init__.py file in this directory. In the same example as the subtract.py file, if you were to create a directory called math_stuff and create a __init__.py file:

    echou@pythonicNeteng:~/Master_Python_Networking/
    Chapter1$ mkdir math_stuff
    echou@pythonicNeteng:~/Master_Python_Networking/
    Chapter1$ touch math_stuff/__init__.py
    echou@pythonicNeteng:~/Master_Python_Networking/
    Chapter1$ tree .
    .
    ├── helloworld.py
    └── math_stuff
     ├── __init__.py
     └── subtract.py

    1 directory, 3 files
    echou@pythonicNeteng:~/Master_Python_Networking/
    Chapter1$

The way you will now refer to the module will need to include the package name:

    >>> from math_stuff.subtract import subtract
    >>> result = subtract(10, 5)
    >>> result
    5
    >>>

As you can see, modules and packages are great ways to organize large code files and make sharing Python code a lot easier.

In this chapter, we covered the OSI model and reviewed the network protocol suites, such as TCP, UDP, and IP. We then did a quick review of the Python language, including built-in types, operators, control flows, functions, classes, modules, and packages.

In the next chapter, we will start to look at using Python to programmatically interact with the network equipment.