Introduction to network automation

If you’re reading this book, you’ve realized you need to think differently about managing your network. And you are not alone. If you ask any network engineer, there is still not a day that goes by when they are not logging into a device via SSH and doing work manually. Over the last few decades, the most common approach to managing networks of any size, ranging from tens to thousands of devices, was connecting to the device and using the network command-line interface (CLI). The network CLI is used to gather data, troubleshoot, and make configuration changes. This remains the most common way of managing networks. However, this is changing.

Over the last 10 years, we’ve seen significant growth and improvements around the operational models for networks. The software-defined networking (SDN) era brought us controllers and APIs. Controllers provide APIs and fewer points of management. Rather than manage thousands of devices, it is possible to manage tens of controllers (or fewer in some cases). Independent of the number, the point is that the number of directly managed nodes continues to decrease. The SDN era also shined a light on the programmatic interfaces, or lack thereof, of network devices. We have evolved from SSH and SNMP to APIs – REST APIs, GraphQL, gRPC, and event-driven webhooks from controllers and devices. While SSH and SNMP are still the de facto standards across the industry – even for automation, progress is being made. For that, we need to recognize the progress and celebrate, but continue to demand more.

The progress around network automation has been driven by open source. Before network automation, there wasn’t much use of open source in the network industry. The industry is learning from its history – that is, if you solely purchase and use vertically integrated tools, there is less flexibility and you could lose control of your network. With current trends, the belief is that those that adopt even just some open source remain in control and can extend libraries and tools as needed to ensure maximum adoption of network automation in their environment. Don’t worry – we’ll cover some of the most common open source tools and technologies for network automation in the Industry trends section of this chapter.

We’ll start by exploring what network automation is, its key use cases, and the value it can provide an organization. From there, we’ll dive into SoT and Nautobot.

What is network automation?

Any advanced and hot technology always gets flak when there are formal definitions because there are always varying opinions, and that’s okay. For this book, our approach is to keep it simple. So, what is network automation? Network automation is next-generation network management. Period. We can talk about Python, Ansible, Nautobot, YAML, JSON, REST APIs, NETCONF, RESTCONF, YANG – the list can go on for pages. Here is the bottom line – all of these tools and technologies are being used to improve how networks are managed and consumed daily, which is, simply put, better network management. Network automation involves transforming operational models that can radically transform careers and technical and business operations.

One major point you should think about on your network automation journey is that it isn’t just about doing your tasks better and more efficiently. That is only the starting point. You need to be thinking about how to expose your automation to other engineers, teams, and even non-technical people, thus enabling all parties with the self-service they need to do their job functions.

Let’s assume you are automating tasks such as operating system (OS) upgrades, which involves gracefully moving traffic from one device (and circuit) to another. This is a complex workflow. Sure, this can help you when you need to upgrade a device or perform maintenance on a device, but what about exposing that automation to individual site leads? If this workflow is made more accessible, can this expand who can perform the task using your trusted automation? Does it allow you or your team to delegate a little more? How often are upgrades happening today contrasted with how often you’d like them to happen?

What about if you had automated diagnostics? What if your Network Operations Center (NOC), Security Operations Center (SOC), or service desk could go to a portal, click a button, and diagnose their most common issues? In a manual process, one person opens a ticket, and that ticket remains open and an engineer picks it up. The engineer reviews the request and sees it is a semi-common problem. Maybe they need to check with another engineer or two along the way. After a few discussions, they know where to go, which devices to log in to, and which tools to log in to. They correlate the data gathered between the devices and tools. They ensure things look good and update the ticket. Common workflows like this should be automated.

Would your leadership be astounded to learn that the countless hours needed to gather data, let alone the hours spent formatting to make it look good, can be eliminated with automation? Compliance and reporting tasks often take a lot of engineering time and effort because they involve manually gathering and processing information. Now, imagine being able to automatically create any compliance document or report you need. Documents that include pre/post change tests. Documents required for change control. Reports you need to run monthly, quarterly, or annually for compliance. Reports that verify your devices are operating as expected.

This is network automation.

Network automation use cases

We just discussed some examples of network automation to bring it to life. Now, let’s look at some of the most common use cases, including the ones that were already mentioned:

• Common config changes: Is your team performing the same types of changes day to day, week to week, or month to month? These are changes such as adding VIPs, turning up a port, adding a VLAN to a switch port, managing firewall policies (also discussed later in this chapter), turning up a new BGP peer, updating routing preferences, adding static routes, and updating zones and ACLs. These changes are ripe for automation because they happen so frequently.
• Common operational tasks: These are similar to the previous use case, but they involve performing operations tasks that do not require a configuration change. Some examples include updating SSH keys and certificates on devices, performing a config save or backing up a configuration, copying files to devices, rebooting devices, checking logs, and even performing non-network device tasks such as checking and updating tickets.
• Mass changes: While common config changes are scoped to a set of devices (this could be just a few devices), mass changes are meant to be site, campus, regional, or global. Mass changes include changes such as updating AAA, NTP, or SNMP but could also include changing the format and structure of all interface descriptions on every device. These types of changes don’t happen as frequently, but when they do, they are impactful and usually a large project.
• Data gathering and reporting: How often is someone you know logging into numerous devices or tools to perform health checks, troubleshooting, or simply to execute a request that comes in for application or network performance degradation? Automated data gathering, reporting, and documentation is not only one of the best use cases for network automation – it is a great area to start with since it is less impactful in the event there is bad automation (because it’d be read-only automation). It could also be added to nearly any other use case producing reports before and after changes or generating compliance reports specific to your team or organization.
• Configuration and operational state compliance: Compliance comes in two major flavors and can be best understood by asking the following two questions: Is the network configured as expected? and Is the network operating as expected? Configuration is easy to understand, but it does mean you’ll need to understand the intended state of the network. This is where SoT and data-driven network automation comes into play. We’ll cover this in more detail later in this chapter in the Understanding SoT section, as well as Chapter 11.
• Pre/post-change state validation: Similar to the previous compliance use case, pre/post-change state validation is more focused on a defined scope of devices. There may be automation when performing global compliance that only runs daily, but changes are happening continuously. Pre/post change state validation ensures that the network is healthy and operating as expected before and after the change.
• Firewall policy automation: How many firewall rules are you adding per day, week, or month? How do you know which firewalls need a new policy? How do you know where in the list of rules the new one should go? Do you know? Could you document this for a fellow engineer? Try. This is the start of firewall policy automation. While the last mile is configuring the actual firewall, the questions prior illustrate that a company’s firewall rule change workflow often involves many steps before the actual configuration change.
• OS upgrades: While already mentioned briefly, how often are upgrades happening today contrasted with how often you’d like them to happen? How many of your devices adhere to your software standards? How many upgrades can you currently do in a single change window? Do you find yourself watching the console of devices as you upgrade them? Do you run any automation to see if devices have the required disk space before copying the new image to the device? Do you run any automation to verify the md5 checksum of the image after it is copied to ensure it isn’t corrupt? Is your network at risk due to vulnerabilities left unpatched? Upgrading devices often happens when needed, versus having a defined cadence. It is never a priority. Automation changes that.
• Greenfield sites and devices: If you are repeating deployments, there is room for automation. It may mean adding new top-of-rack switches in the data center, it may mean adding a closet or IDF closet in a growing campus, adding a new retail location, or even a new colocation facility or point of presence (PoP). Much of the automation discussed here is around the configuration of these devices, but that is the easy part. Site planning and deployment is about data curation and management not to mention each organization’s business logic required for deployments. How do you and your team know which IP addresses, VLANs, ASNs, and overall configuration should be entered on those devices? Is it from spreadsheets or a SoT? Again, more on SoT later.
• Vendor migrations: Have you ever not moved forward with changing vendors due to the work effort of migrating configurations? With a properly defined SoT and data strategy, this becomes trivial. Your focus becomes storing the intended state of the network using data, decoupled from any vendor-specific syntax. Syntax for a given vendor is generated by running the data through a set of vendor-specific configuration templates. In a migration, you can generate the desired state configuration for a given vendor by running the data through a different set of templates and then deploying those new configurations. Beyond configuration management, you’ll also want to ensure multi-vendor operational state compliance to ensure there are no gaps in visibility during the end-to-end migration.
• Self-service: It is critical to think through how a given workflow will be triggered along with who the target user is. Self-service does not mean that it needs to be a click-button UI. It may mean an IT tool, CLI tool, pull/merge request, ChatOps, or yes, it may mean a full self-service user-friendly form. The point is that you do not need one way to expose network automation or even one way per workflow. Using an architectural and a platform approach to network automation allows you to expose the same workflow through multiple self-service interfaces. You should cater to your culture and your users. This will drive more adoption of network automation.

It is recommended to use a holistic multi-domain network automation architecture to serve as a platform to meet today’s requirements. This architecture will also serve as the foundation for tomorrow’s requirements. As you embark on the journey, be cautious about using different network automation architectures for different types of networks and domains. If so, it’ll create more issues and give your team even more tools to manage while making it harder to unify standards and processes. In Chapter 10, we will talk much more about network automation architecture to ensure a consistent approach to managing networks independent of size, domain, and location.

Why automate your network?

After covering the what, let’s take a look at the why. While many use cases are horizontal and can be used by any organization type (or verticals), the actual why, impact, and justification will differ per organization. Just to clarify, by vertical, we’re referring to companies with different business types. A few examples of different verticals include financial services, pharmaceutical, retail, telco/the cloud, manufacturing, accounting/legal professional services firms, state and federal government, K-12 education, and universities.

For some verticals, the network may be the business. It may either be a business enabler or have serious consequences if the network is down. For other verticals, other factors may be a bigger concern. For this reason, the why is going to vary widely, and we’ll cover general reasons to automate the network. Here are some common examples:

• Lower costs: Every leader in every business is always asked by their leaders or directly by finance if there is a way to lower costs. In reality, automation helps lower longer-term costs. The more a company can show how automation lowers costs, the greater the chances are that the automation projects get initial buy-in and long-term support. With some of the use cases already mentioned, costs can be significantly lowered. If a company truly documents each of the tasks required and the time to do each for a workflow (such as OS upgrades or troubleshooting) and verifies the most common incidents, they are going to see drastic savings in time and effort when using automation. Time equates to money. It doesn’t mean anyone is getting replaced. However, it does mean that there is more time for more projects, each of which adds more value to the business. Increasing velocity without needing to hire new people is a tremendous cost savings.
• Enhance security and reduce risk: In today’s world, security is top of mind for everyone; it’s integrated into all that we do. No company wants to be the headline in the local, national, or global news. Security-focused automation ranges from automated scans, firewall provisioning, VPN connects and disconnects, compliance and remediation, governance adherence and monitoring, and patch management just to name a few. Even if you are not directly on a security team, you should ask yourself if security can be improved in your domain. Can you rotate passwords more frequently? Maybe change those SNMP community strings? The list can easily go on.
• Provide greater insight and control: Data is king and that includes greater visibility into your network and automation infrastructure. Automation can be used to gather data, document data, understand patterns, and compare against known baselines. Sure, there may be tools that provide this in the user interface (UI). That’s a great start, but what about seamless workflows that open tickets, update tickets, send emails, and send chat messages in response to network data that is outside the expected range? With automation, you have the opportunity to get the insights you need to answer the questions you have and know that the answers are contained within the network. Think about that. If you are logging into a few portals, copying data into a spreadsheet, creating Excel formulas, or creating a new document to then turn into a PDF and email, there is a better way. There is an automated way.
• Increase business agility: Each business and team is always trying to go faster and also perform activities that are not possible without automation. Organizations need to work smarter and more efficiently. In some cases, it may also make sense to hire more people. However, hiring more people often slows things down because, at a certain point, people can start to get in each other’s way. In contrast, automation can reduce cost, improve performance/velocity, increase reliability, and do things that humans just cannot do. One example is automation-enabled self-service, which helps business stakeholders obtain the outcome they need sooner. Automation can also improve business-to-business connectivity, allowing organizations to either recognize revenue sooner (for those that are doing business over those connections, tunnels, or circuits) or start consuming a new service. Think about deploying a new application in a lab or test environment. If it takes weeks to get a new application and its network and security configurations deployed for each environment (dev, test, UAT, and so on), it may be an aggregate delay of months. This is either delaying employee or customer satisfaction or revenue. Using automation improves this and increases business agility.

In all that you do, keep automation top of mind, and try to understand the business and organization-level benefits for various leaders in your organization.

Persona-driven network automation

While we already looked at network automation use cases and the rationale for automation, let’s take a different spin on use cases. There is usually never one network team. There are usually teams focused on day 0 or architecture and/or engineering; day 1 or implementation; day 2 or operations. These teams may even span network domains such as LAN, WAN, WLAN, or Security, depending on the size of the network. Recognizing the work of the various teams will help structure automation projects for what’s possible within your team.

Here is a list of example projects and tasks broken down by the three types of teams often found in network organizations:

• Day 0 or architecture and/or engineering:
  • Ensure configuration standards are documented in a structured and modeled manner that is programmatically accessible
  • Ensure hardware standards are documented in a structured and modeled manner that is programmatically accessible
  • Ensure software standards are documented in a structured and modeled manner that is programmatically accessible
  • Ensure architectural and engineering tests exist within every CI pipeline – for example is there redundancy?
  • Develop automation architecture and framework used by other teams
• Day 1 or implementation:
  • Use automation to generate configurations
  • Use automation to perform configuration changes
  • Use automation for pre- and post-deployment verification
  • Use automation for continuous verification of deployment standards
• Day 2 or operations:
  • Execute network device automation for common troubleshooting tasks
  • Continuously update automation that is used for common troubleshooting tasks
  • Execute network device automation for common changes
  • Ensure automation for dynamically reading emails from ISP/NSPs for circuit notifications
  • Execute automation for gathering and collecting information from various tools and devices to aid in troubleshooting
  • Execute automation for dynamically creating, updating, and closing change management tickets

Industry trends

As we’ve already discussed, the CLI still dominates the industry. However, each year, month, week, and day brings us closer to transformative and better network management through the use of network automation. In this section, we’ll look at several of the trends that are collectively driving the industry forward to do more with less and allow for more efficient network operations.

This list is not meant to be exhaustive, but illustrative of the trends that are driving operational efficiencies and automation:

• SDN: SDN took the industry by storm in the 2010s. Most modern network architectures include controllers that simplify management and visibility and provide programmatic access with APIs. Simplified management is made possible because it allows users to manage systems versus managing devices and nodes, which allows more abstract policies to be created and applied. Because they allow for fewer points of management, SDN controllers simplify workflows and integrations using the controller (versus individual device) APIs. With SDN, you may have different controllers and solutions for campuses, WAN, data centers, and the cloud. So, if you are looking for a unified network automation strategy, there will be a bit of integration that needs to happen when it comes to data and orchestration. More on this later.
• NetDevOps: We’ve learned a lot about the DevOps industry over the last 10 years. When we talk about NetDevOps, we’re referring to doing DevOps but applied to network infrastructure, engineering, and operations. Here are a few examples that highlight trends:
  • Using Git-based version control systems (VCSs) such as GitHub, GitLab, or BitBucket. Using VCS enables collaboration while providing traceability and audibility on all software or file-based artifacts (templates, data files, scripts). VCS allows users to create owners of particular projects or sections of a project providing accountability to the respective teams.
  • Using continuous integration (CI). Organizations that use VCS will require basic CI. CI allows users to create tests that must pass before accepting or approving any changes. These tests focus on ensuring nothing is going to break in the automation or the application. CI can also be applied more directly to the network, enabling network CI.
  • Implementing network CI. If the initial CI tests pass on code and static files, users can do tests such as pre-change analysis based on models of the network (mock devices or real equipment, if you have a larger budget), running active tests on the network (does the network need to be a certain state before making the change?), perform the actual change, and then finally ensure the network is operating as expected after the change.

  While DevOps and NetDevOps can be talked about for days, the actual industry facts show that nearly every network automation project in the world includes version control, automated tests, and some level of CI. If your organization is one of the few that aren’t using these three key items, be sure to explore them as soon as you can.

• Open source: Many open source tools are used in the DevOps ecosystem. The same holds for NetDevOps. We’ll mention some of the most common tools in the Tools and technology point covered in this section. Regardless of the tools deployed, it is more important to understand the real value of open source. In the context of open source, the real value lies in its extensibility, ecosystems, and community. Extensibility and ecosystems can drastically change and improve what’s possible on your network automation journey. Keep in mind that each of these is predicated on the fact that there is a strong community at the foundation. Extensibility is what should give you confidence that no matter what decision is made for your network, you can adapt and change to account for that decision. A change may be as simple as upgrading to the latest version of software, migrating from vendor A to vendor B, or migrating from a traditional network to a controller-based network. In any of these scenarios, an organization needs to be confident that its automation can be tailored, updated, or augmented for their needs. While certain commercial tools offer extensibility, it is usually limited and extensibility features tend to be in a perpetual state of coming soon. Ecosystems built around community also play a critical role in open source software, further enhancing what is possible with particular open source projects. Ecosystems are usually fostered around extensions, adapters, apps, or add-ons that are outside of the core open source project but are powered by it. It is these ecosystems that usually incorporate the solutions required for true multi-vendor management and automation. The point is not that everything needs to be open source, but that open source software and solutions should either lead or complement any network automation strategy. If they do not, there may be a great risk to the success of the automation journey three to five years out.
• SoT: Since you’re reading this book, you’ve likely heard about SoT. In fact, the main topic of this book is Nautobot! At its core, Nautobot is a network SoT that is actively being developed specifically for network automation environments. A SoT is a growing industry trend and probably why you’re reading this book, but the short overview of a SoT is that it is the location where you can define the intended state of the network. This is the truth; it is what should be. The SoT is not what is on the device or network. That is referred to as the actual or observed state. The intended state, or SoT, can be extrapolated and used to document the intended configured state and intended operational state, or even used as the place to define the intended state for monitoring thresholds and events. Overall, it allows for greater governance of network data with a focus on what should be in a manner that is often vendor-neutral. We’ll spend much more time on SoT in the next section and throughout every other chapter in this book.
• Self-service: We covered self-service in the Network automation use cases section, but to restate it one more time, the notion of self-service is not one-sided. Those organizations that are successful on their network automation journey understand that it is about having the right mapping of workflows to people (consumers) and from those people to the right user interaction, or the right tool to execute and request that automation. If you get this wrong, there is a great chance to end up with network management systems that aren’t used, which will take us back a few decades.
• Streaming telemetry: SNMP has been around for decades, and network visibility as we know it is largely based on SNMP. Streaming telemetry is what you may expect when you think about modern network visibility. In this modern era of streaming telemetry, network devices can continuously “push” or “stream” network data to a centralized location. This allows for greater visibility, querying, and trending based on data that would have normally been lost. Wouldn’t it be great if the network device could send you the information you need when you need it? Wouldn’t it be great if you could turn on a stream of data (collection of data points) from a series of devices on particular interfaces versus getting a response from an interface poll that may kill the device if your poll frequency is too high? Wouldn’t it be great if you could build a closed-loop system that can operate in near real time? This is made possible by streaming telemetry.
• Intent-based networking (IBN): When you look at the key use cases and trends, you can start to see common components of an architecture, such as orchestration, automation, SoT, and telemetry. When these components are fully integrated, the result is an IBN. An IBN is just a comprehensive network automation architecture. It allows organizations to define intent, continuously collect network data (streaming telemetry, SNMP, show commands, and configuration data), analyze that data, ensure intent is deployed, and then react based on intent violations. The reaction to the data may be to remediate or make a change for managing capacity or minimizing the blast radius for a known issue. IBN becomes a natural progression as you start to deploy a holistic architecture for network automation.
• Artificial intelligence (AI): Our general belief is that a significant amount of automation must be implemented without AI/ML, meaning don’t let flashy new tech derail projects and outcomes that are solving today’s problems. That said, at the time of writing, we’ve seen the launch of OpenAI’s ChatGPT (https://openai.com/blog/chatgpt/), Google’s Gemini, and many more services like these. It should be obvious that AI/machine learning (ML) coupled with natural language processing (NLP) creating more digital assistants is going to have a transformative impact on where we are as an industry in 5 to 10-plus years as it gets mainstream adoption. Until then, it’ll be explored and implemented by pioneers and manufacturers who can make it consumable in a turnkey and meaningful way.
• Tools and technology: This is always one of my favorite topics since we live in a product- and tool-centric industry, but let’s look at existing tools trends for network automation. From an open source perspective, the dominant tools are Ansible, Nautobot, Batfish, and Terraform. We also see a sprinkling of Salt, but its presence is still largely seen in application and systems automation. Looking at open source from a lower-level library perspective, there is continued growth with Netmiko, NAPALM, Nornir, pyntc, ntc-templates, and scrapli. If you are using open source or building your solutions, you want to check out these projects. For example, if you need a custom Ansible module or custom Nautobot App, you’re more than likely going to consume those libraries to perform your automation. From a telemetry perspective, there is also growth in various stacks that include Prometheus, Influx, Telegraf, and Grafana. Teams that have the skills or are further on their journey can use these stacks to provide greater visibility through data aggregation, data enrichment, extremely powerful queries, and a holistic view of their networks and their IT infrastructure. From a commercial tool perspective, and exclusive of SDN products, we’re seeing the most adoption of Itential, IP Fabric, and Forward Networks.

Information

Interested in seeing a comprehensive list of all network automation projects, tools, and products? Check out Awesome Network Automation (https://github.com/networktocode/awesome-network-automation).

From a trends perspective, we thought it may be worth calling out a few things that get attention at industry events and in social circles, but aren’t gaining traction. The first is the direct use of YANG data models within automation tools. They are still mostly used by vendors to define their schema. Of course, there are outliers such as hyperscalers or a select few enterprises, but generally speaking, the actual use of YANG by network teams is not a trend. If you’re using an API that is based on a YANG schema, we do not consider that a trend for end users, but it is a trend for certain manufacturers. We’ll also call out REST APIs on network devices. While they are becoming more commonplace because the dominant majority of devices in production still don’t have APIs, and instead have two or more (different APIs per vendor and OS) ways of performing automation, the majority of device-specific automation still happens via SSH.