To follow this chapter, you need to have an active Azure subscription with preferably owner rights at a subscription level, although rights at the resource group level will also work.

You can get a trial at https://azure.microsoft.com/en-in/free/ if you do not have an Azure subscription already.

Check out the following link to see the Code in Action video:

https://bit.ly/3ggcdz8

Over the last decade, Microsoft Azure established itself as a leader in the public cloud industry. Microsoft's hybrid cloud story started back in the early days of Azure with Windows Azure Pack and progressed with Azure Stack, Azure Stack HCI, and various other products.

In November 2019, at the Ignite conference in Orlando, FL, Microsoft announced Azure Arc, which is the latest addition to its hybrid cloud capabilities. In simple words, Azure Arc lets customers run Azure services anywhere they want, that is, in their data centers or in other public clouds, and manage them through their existing Azure management capabilities. You can now leverage your favorite Azure management tools and services to host your applications wherever you want, allowing you to utilize your existing hardware investments without adding management complexities and security risks.

As Figure 1.1 illustrates, Azure Arc extends the Azure cloud beyond Microsoft's data centers. You still interact with Azure tools (the portal, CLI, PowerShell, APIs, SDKs, and even third-party deployment tools such as Terraform), but rather than using them to interact with your Azure resources, you also leverage the same tools to interact with your on-premises infrastructure and other cloud platforms, including Amazon Web Services (AWS) and Google Cloud Platform (GCP):

Figure 1.1 – Azure Arc overview

Azure Arc is an umbrella of the services comprising hybrid cloud offerings across the infrastructure and data services. At the time of writing this book, it includes the following services. It is very likely that this list will continue to expand, and we will see more scenarios being included in Microsoft's hybrid cloud story:

• Azure Arc-enabled infrastructure:

  - Azure Arc-enabled servers

  - Azure Arc-enabled Kubernetes

  - Azure Arc-enabled SQL Server

• Azure Arc-enabled services:

  - Azure Arc-enabled data services

  - Azure Arc-enabled machine learning (in private preview)

Multi-cloud architectures are an important pillar of the IT strategy for organizations of all sizes these days. With containerization and cloud-native deployments, migrating applications from one infrastructure platform to another isn't the tedious and time-consuming job it used to be years back. With Azure Arc, Microsoft is moving toward being the preferred cloud management platform for your multi-cloud architectures. You can now manage Kubernetes clusters running on AWS or GCP through the same tools you'd use to manage Azure Kubernetes Service.

With this, Azure provides a seamless management experience across on-premises data centers, edge environments, and multi-cloud architectures.

What Azure Arc isn't

Azure Arc is neither a private cloud solution nor a replacement of Azure Stack services. Azure Stack continues to grow as a go-to solution for building intelligent hybrid cloud solutions with specialized hardware.

Azure Arc lets you leverage your existing infrastructure investments, which isn't possible with Azure Stack. If you are running hundreds of Windows or Linux servers in a virtualization environment, you can bring Azure Arc in there without disrupting or rebuilding the infrastructure, which isn't the case with other hybrid cloud solutions by Microsoft.

Azure Arc isn't an orchestrator for your on-premises data centers or virtualization infrastructure. You still must manage your hardware infrastructure; however, it can let you manage and govern your infrastructure the same way you'd manage your Azure infrastructure, using the same Azure portal. Now that we know what Azure Arc is, let's see where it can be useful with the help of a few use cases in the upcoming section.

In simple words, Azure Arc lets customers bring their legacy infrastructure and still leverage modern cloud technologies to innovate their IT infrastructure, applications, and data services. You can bring your legacy hardware infrastructure running supported Window or Linux machines and manage their access control using your Azure Role-Based Access Control (RBAC) and run a managed SQL database there.

Essentially, Azure Arc services help organizations use cloud innovation wherever they need.

Azure Arc has use cases across governance, compliance, security, management, cloud-native applications, data services, and various other scenarios. Let's look at them in the next sections.

Organizing and governing across environments

In today's IT world, enterprises have enormous types of applications and data services, each having its own planning, security, and governance best practices based on its design principles and hosting architecture.

Using Azure Arc, you can organize and govern these resources consistently irrespective of their hosting location. You will be able to easily organize, manage, govern, and secure your Windows and Linux servers, SQL Server databases, and Kubernetes clusters, across data centers, edge, and multi-cloud environments. You will use familiar Azure Resource Manager (ARM) capabilities, such as ARM templates, Azure Policy, and Azure Resource Graph, to manage both your cloud and other environments, including on-premises and other cloud platforms.

In simple words, you can define your overall IT security and governance policies in one place (Azure) and apply them across all your environments along with continuous monitoring using Azure Monitor.

Building cloud-native apps at scale

Azure Arc helps you deploy your containerized apps securely and consistently across environments including Azure and non-Azure infrastructure. With Azure Arc and DevOps techniques, now you can deploy your applications to a Kubernetes cluster running anywhere in the world without leaving GitHub.

Along with app deployment, you also enable consistent monitoring and governance frameworks across the Kubernetes clusters running in Azure, on-premises, or even Elastic Kubernetes Service (EKS) or on Google Kubernetes Engine (GKE).

Running Azure data services anywhere

In the last decade, cloud databases have proven to be revolutionary and help organizations to quickly ship their products without being concerned about their database's high availability, performance, and so on.

Azure Arc allows you to run the same cloud database runtime in your own hardware. At the time of writing, it supports Azure databases for PostgreSQL and Azure SQL Database managed instances. It allows you to run a highly available, secure, and highly scalable database service close to where your compute is running.

Meeting security, compliance, and regulatory requirements

Azure Security Center and Azure Defender are hubs for security and compliance for everything in Azure. With Azure Arc, you can extend the same security and compliance capabilities to your own infrastructure and stay compliant along with meeting your regulatory requirements of hosting your data wherever you need to.

Example customer use case

Our customer, Contoso Ltd., is a financial institution based out of Europe. Over the years, Contoso has built a large IT infrastructure deployed across multiple data centers across Europe and outside, a couple of co-locations, and cloud platforms including Azure and AWS.

Security practices, guidelines, and requirements continued to change over the years depending on where the applications were hosted. With automation and DevOps practices being introduced recently, Contoso is struggling with a server sprawl situation and organizing and governing IT resources across the environments. Server sprawl defines a situation where there is an enormous number of servers being underutilized, unmanaged, poorly managed and poorly governed, or even unknown to IT teams in some situations.

Business requirements

Contoso would like to consolidate and eliminate the server sprawl situation while ensuring the governance, security, and compliance practices are met across the environment irrespective of hosting location. Let's look at some of the key requirements for Contoso Ltd., as follows:

• Apply governance and centralized management across Windows and Linux servers running as bare metal or as Virtual Machines (VMs) in data centers and public clouds.
• Apply security and configuration policies consistently, everywhere.
• Provide the ability to specify governance requirements based on applications and track the overall governance and compliance state.
• Simple visibility across environments using a single pane of glass.
• Remediate any configuration and compliance issues.

Solution with Azure Arc

Azure Arc can help Contoso overall by providing the following capabilities across their data centers, co-locations, and both the Azure and AWS cloud platforms, as follows:

• Use the Azure portal to centrally manage and govern your servers across environments.
• Consistently apply governance and compliance policies using Azure Policy and Azure Defender.
• Have a centralized compliance view across servers from different environments.
• Remediate the compliance issues through Azure Policy:

Figure 1.2 – Azure Arc use cases

In this section, we learned what Azure Arc is and where can it be useful. Let's move ahead and understand in some more detail what goes on under the hood.

Now that we know what Azure Arc is and how it helps organizations bring agility to their hybrid cloud operations and governance, let's understand the technology behind it and how Microsoft is extending its non-Azure environment.

Azure Resource Manager

ARM is the backbone of the Azure public cloud platform. All requests to Azure are received by ARM and then passed on to the backend control plane of various services. In simple words, ARM handles the deployment and management portion of your Azure environment.

There are various resource providers in Azure, such as Microsoft.Compute and Microsoft.Network. Each resource provider offers certain services and ARM is the way you interact with the resource providers. The Microsoft.Compute resource provider is responsible for resources such as VMs, VM scale sets, disks, and availability sets.

You can view the list of resource providers available in your subscription by using the following instructions. Let's take a look:

1. Navigate to the Azure portal (https://portal.azure.com).
2. Log in with your preferred Azure account.
3. In the search bar, search for Subscriptions, as seen in the following screenshot:
   

   Figure 1.3 – Azure search bar

4. Select any of your existing subscriptions and look for Resource providers on the left-hand side:
   

Figure 1.4 – Resource providers in Azure

You can see all the resource providers here; some may not be registered for your subscription. In order to use services by the resource providers, you must register them first. Typically, ARM handles this for you for common resource providers by default.

ARM, along with resource providers, builds what's called the control plane of Azure.

The Azure control plane beyond Azure – Azure Arc

Azure Arc extends the Azure control plane to non-Azure environments. Essentially, you leverage the same ARM and resource provider technologies to manage your non-Azure environment.

Azure Arc introduces new resource providers for managing non-Azure environments. At the time of writing this book, this includes Microsoft.HybridCompute and Microsoft.AzureArcData along with Microsoft.GuestConfiguration, which is responsible for providing Azure policy services across both Azure and non-Azure environments.

Exploring Azure Arc services

Azure Arc is a group of services offered to enable hybrid cloud functionality across various technologies, including computers and data. Let's dive into each service and see what they offer.

Azure Arc-enabled servers

Azure Arc-enabled servers allow you to manage and govern your Windows and Linux servers running outside Azure. You can onboard your servers running on physical servers or as VMs on your network or other public cloud platforms, to Azure. Once a server is onboarded, it is treated as a first-class citizen in Azure; that is, you will see a dedicated Azure resource for each onboarded server.

In Azure Arc terminology, each onboarded server is called a connected machine. Each connected machine has its own Azure resource ID and can be managed through the Azure portal, CLI, APIs, PowerShell, or any supported SDK and third-party automation products.

Azure Arc-enabled servers are generally available, that is, they can be used in production.

Supported scenarios

At the time of writing this book, you can perform management and governance for Arc-enabled servers limited to the following scenarios. This list will continue expanding, so be sure to check the Azure Arc-enabled server's documentation (https://docs.microsoft.com/en-in/azure/azure-arc/servers/) to stay updated on supported scenarios.

The scenarios are as follows:

• Guest configurations with Azure Policy (https://docs.microsoft.com/en-us/azure/governance/policy/overview)
• Change tracking and inventory management with Azure Automation (https://docs.microsoft.com/en-in/azure/automation/)
• Monitoring through Azure Monitor (https://docs.microsoft.com/en-in/azure/azure-monitor/insights/vminsights-overview)
• Consistent deployments with desired state configuration and custom extensions
• Update Management through Azure Automation
• Security, compliance, and threat detection with Azure Security Center (https://docs.microsoft.com/en-in/azure/security-center/security-center-introduction)

We will be discussing Azure Arc-enabled servers in detail in future chapters.

Azure Arc-enabled Kubernetes

Azure Arc-enabled Kubernetes allows you to manage and perform consistent deployment on Kubernetes clusters running outside Azure, the same way you do for Azure's native Kubernetes offering, that is, Azure Kubernetes Service.

At the time of writing, Azure Arc-enabled Kubernetes is in preview. It is not recommended to use preview services in production.

Supported scenarios

Let's look at what you can do with your Kubernetes clusters once they're in Azure. This list will continue expanding, so be sure to check the Azure Arc-enabled Kubernetes documentation (https://docs.microsoft.com/en-us/azure/azure-arc/kubernetes/overview) to stay updated on supported scenarios:

• Consistent deployment with GitOps (https://www.gitops.tech/)
• Cluster configuration management and compliance with Azure Policy
• Monitoring with Azure Monitoring for containers (https://docs.microsoft.com/en-us/azure/azure-monitor/insights/container-insights-overview)

Azure Arc-enabled data services

Azure Arc-enabled data services let you run Azure's cloud database runtime in your environment. You will need to have a supported Kubernetes cluster to deploy these services.

At the time of writing of this book, you can deploy the following data services to a supported Kubernetes cluster running anywhere:

• Azure Database for PostgreSQL (Hyperscale) (https://azure.microsoft.com/en-in/services/postgresql/)
• Azure SQL Managed Instance (https://docs.microsoft.com/en-us/azure/azure-sql/managed-instance/sql-managed-instance-paas-overview)

Supported scenarios

Let's take a look at some of the supported scenarios with Azure Arc-enabled data services. This list will continue expanding, so be sure to check the Azure Arc-enabled data services documentation (https://docs.microsoft.com/en-us/azure/azure-arc/data/overview) to stay updated on supported scenarios:

• Run PostgreSQL Hyperscale or Managed Instance databases in a non-Azure environment. It includes the features and capabilities supported by these cloud databases.
• Backup and recovery.
• Scale up and down dynamically.
• Two connectivity modes (directly connected and indirectly connected).
• Security and governance through your familiar Azure tools.
• Support for Azure Data Studio.
• Monitor with Azure Monitor.

We will be discussing the supported scenarios and limitations in the respective chapters.

Important note

The feature set of Azure Arc-enabled data services and their respective cloud database service isn't identical. Please refer to the Microsoft documentation (https://docs.microsoft.com/en-in/azure/azure-arc/) to learn more about limitations and so on.

At the time of writing, Azure Arc-enabled data services are in preview. It is not recommended to use preview services in production.

Azure Arc-enabled SQL Server

Azure Arc-enabled SQL Server lets you manage the SQL servers deployed outside Azure. Azure SQL databases have strong data protection capabilities through their advanced data security services. With Azure Arc-enabled SQL Server, you can leverage the same security capabilities for your SQL servers running outside Azure.

Azure Arc-enabled SQL Server is part of the Azure Arc-enabled servers. This service is still in preview. It is not recommended to use preview services in production. Supported scenarios

Let's look at some of the supported scenarios with Azure Arc-enabled SQL Server. This list will continue expanding, so be sure to check Azure Arc-enabled SQL Server documentation (https://docs.microsoft.com/en-us/sql/sql-server/azure-arc/overview) to stay updated on supported scenarios:

• Onboard both Windows- and Linux-based SQL servers.
• Assess your SQL servers against best practices across security, compliance, availability, business continuity, performance, and scalability.
• Protect your SQL servers with Azure Defender (https://docs.microsoft.com/en-us/azure/security-center/defender-for-sql-introduction).

As at this stage we have formed a good foundational understanding of all the offerings under the umbrella of Azure Arc and the supported scenarios, we will now move ahead and get in the real game of creating our own lab environment, on top of which we will be hosting our entire solution.

We have designed this book to be a hands-on focused book, so you will see a lot of implementation steps and example deployments. In order to prepare for that, we will need you to prepare your Azure accounts in advance.

In this section, we will create the required Azure infrastructure to simulate the on-premises environments. If you have an on-premises infrastructure, you may use that as well, rather than hosting everything in Azure.

Getting started with Azure

To start your Azure journey, you can go to https://azure.microsoft.com/free/. This takes you to the landing page of the free account offer, which looks like this:

Figure 1.5 – Signing up for a free trial

You can explore the page to learn more about the offer. You can use the services that follow on the sign-up page for free for 12 months, within the free service limits for the service. For instance, you get 750 hours of a Windows VM for free. You can spend these hours over 12 months. In addition to that, you get $200 of Azure credit for the first 30 days.

To create your free account, you need to do the following:

1. Go to https://azure.microsoft.com/free/ and click on the Start free button.
2. Log in with a Microsoft account or a GitHub account. If you don't have one yet, you can create one.
3. First, you need to verify your identity by phone. You can do that by entering your phone number and giving the verification code that you receive.
4. Next, you will have to give the details of a credit card. Don't worry, you won't be charged. By default, the Azure subscription that you create has a spending limit on it, so you can't use more than the free $200 that you receive until you remove this limit manually.
5. Fill in the personal details and click Next.
6. Finally, agree to the agreement and click Sign up. Your free Azure account will be ready in a few moments. Go to portal.azure.com and start using it.

Creating a resource group in Azure

A resource group is a container that holds related resources for an Azure solution. A resource group includes those resources that you want to manage as a group. We'll be creating three resource groups, each for its own individual lab and purpose, which will be used in their respective chapters:

• The On-prem-Server resource group will be created to host the Windows VMs considered to be on-premises servers.
• The On-prem-Kubernetes resource group will be created to host the Kubernetes cluster that will be managed by the Azure Arc management pane in Chapter 3, Azure Arc Enabled Kubernetes.
• The On-prem-Data resource group will be created in the same fashion to host the data services managed by Azure Arc.

Let's create the resource groups in Azure by following these steps:

1. Log in to the Azure portal using https://portal.azure.com with your Azure credentials.
2. Click on + Create a resource and search for resource group as you can see here:
   

   Figure 1.6 – Searching resource group in Azure

3. Click Create on the resource group page.
4. Select your subscription, as seen in Figure 1.7, and enter the resource group name as On-prem-Server.
5. Next, choose the region closest to your location:
   

   Figure 1.7 – Creating a resource group in Azure

6. Click on Review + create and then Create to start the deployment.
7. Repeat the steps to create two additional resource groups for Kubernetes and a data lab environment, named On-prem-Kubernetes and On-prem-Data respectively.

You've now created resource groups to host Azure resources.

At the time of writing this book, many of the Arc services are in preview and are offered free of charge. Additional services used by Azure Arc, such as Azure Monitor and Security Center, are billed separately.

Estimating and planning the pricing and costs for Azure Arc and related services is outside the scope of this book. Please refer to the Azure Arc pricing page (https://azure.microsoft.com/en-in/pricing/details/azure-arc/) to stay updated on Azure Arc pricing.

In this chapter, we learned about Azure Arc and the various use cases it supports. We also looked at an example customer situation where Azure Arc can be useful. We learned about various services it offers across the servers, Kubernetes, and data services areas. In the end, we prepared our Azure subscription to be ready for the hands-on learning planned in the following chapters.

With the completion of this chapter, we were able to understand, process, and apply the foundation of Azure Arc, which will be of immense help in upcoming chapters as we go through each of these offerings of Azure Arc in detail and get hands-on with all of them one by one.

Moving ahead, we will begin with a deep dive into Azure Arc-enabled servers as we commence onboarding our infrastructure and modernize VMs to be managed with the help of Azure Arc.