Azure Arc Kubernetes: Tutorial & Examples

Azure Arc is a Microsoft service that enables users to manage and govern their hybrid and multi-cloud environments from a single pane of glass. It allows organizations to bring the benefits of Azure’s management, security, and governance tools to any infrastructure, whether on-premises, in another cloud, or at the edge. It delivers a centralized view of all the resources across a hybrid environment, enabling administrators to monitor and manage them from a single location. It also includes role-based access control and compliance reporting, which help organizations ensure that their hybrid environments meet security and compliance requirements.

Azure Arc eliminates the need for organizations to manage multiple management tools and consoles for their on-premises and cloud resources. Instead, they can use a single set of tools to manage their resources regardless of location, significantly reducing the complexity and cost of operating a hybrid environment.

As of December 2022, Azure Arc offers the following services, also known as Azure Arc–enabled services:

This article focuses on Azure Arc–enabled Kubernetes because it is a prerequisite for other Arc-enabled services.

Azure Arc for Kubernetes is a service offered by Microsoft Azure that allows users to manage and govern their Kubernetes clusters across various environments, whether running on-premises, in multiple clouds, or at the edge. With Azure Arc for Kubernetes, users can manage their clusters using the same tools and APIs they use for managing Azure Kubernetes Service (AKS) clusters—such as the Azure portal, Azure Monitor, Azure Policy and Microsoft Defender —and leverage many other services:

• Inventory management, grouping, and tagging of all Kubernetes clusters
• Enabling observability and monitoring clusters using Azure Monitor
• Ensuring governance by applying policies with Azure Policies and ensuring compliance across Kubernetes clusters
• Enforcing security and threat protection using Microsoft Defender
• Leveraging Azure Active Directory for authentication and authorization
• Deploying applications and configuration centrally using GitOps.

Apart from the above-listed benefits, Azure Arc–enabled Kubernetes can bring additional Azure services to your infrastructure:

• Running Azure SQL–managed instances
• Running Azure ML
• Running Azure App Services
• Running Azure Functions
• Running Logic apps
• Running Event grid

These services depend on Azure Arc–enabled Kubernetes as one of the prerequisites to enable them.

How does Azure Arc for Kubernetes work?

All cloud services consist of servers, storage, and networks running in a physical data center, virtualized and enhanced with additional value-added services. Cloud providers take this virtualized infrastructure and assign a unique identity to each infrastructure resource, in simple terms, which users can then manage using a portal. This unique identity in Azure is managed via the Azure Resource Manager (ARM).

Azure Arc extends this functionality to non-Azure infrastructure, meaning that a Kubernetes cluster running on AWS or your premises will get an Azure Resource Manager ID (ARM ID). The Kubernetes cluster or the applications/data within the cluster are never migrated to Azure; however, because of the ARM ID, Azure considers this resource its own and enables you to use all the other Azure services for it.

The Azure Arc–enabled Kubernetes service uses several agents to onboard any CNCF-conformant Kubernetes cluster and enables Azure services for these clusters. A single command, like this one, connects and onboards your Kubernetes cluster (linked to the current kube context with default configuration) to the Azure portal:

az connectedk8s connect –name ClusterName –resource-group RGName

Internally, this command creates an “azure-arc” namespace within the Kubernetes cluster and deploys the following essential agents:

Here’s how you onboard an existing Kubernetes cluster to Azure Arc. The Kubernetes cluster can be any CNCF-conformant distribution, including EKS and GKE.

Prerequisites

1. Ensure that you have access to the kubeconfig file configured to work with your Kubernetes cluster context.
2. Have Azure CLI installed on the machine where you have the kubeconfig files for all your clusters.
3. Install, set up, and configure kubectl.
4. Install and configure Helm 3.
5. Ensure that you have access to an Azure subscription, then use the Azure CLI to log in and connect to the subscription:

   az login

6. Create an Azure service principal (SP) with a “Contributor” role:

   subscriptionId=$(az account show --query id --output tsv)
    az ad sp create-for-rbac -n "OnboardK8S" --role "Contributor" --scopes /subscriptions/$subscriptionId

You will see output similar to the following; please save this information for later.

{
  "appId": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "displayName": "OnboardK8S',
  "password": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "tenant": "XXXXXXXXXXXXXXXXXXXXXXXX
}

Onboarding

1. Enable the Azure subscription with the required resource providers for Azure Arc–enabled Kubernetes. Note that this step may take a few minutes.

   az provider register --namespace Microsoft.Kubernetes
   az provider register --namespace Microsoft.KubernetesConfiguration
   az provider register --namespace Microsoft.ExtendedLocation

2. Create a resource group in Azure in the region where you want the cluster to show up.

   az group create -l eastus -n myArcK8SCluster 

3. Set environment variables based on the Azure Service Principal created earlier.

   export appId='<Your Azure service principal name>'
   export password='<Your Azure service principal password>'
   export tenantId='<Your Azure tenant ID>'
   export resourceGroup='<Azure resource group name>'
   export arcClusterName='<preferred k8s cluster name>'

4. Install the Azure Arc for Kubernetes CLI extensions connectedk8s and k8s-configuration.

   az extension add --name connectedk8s
   az extension add --name k8s-configuration

5. Log into your Azure subscription.

   az login --service-principal --username $appId --password $password --tenant $tenantId

6. Finally, onboard or connect the Kubernetes cluster to Azure Arc.

   az connectedk8s connect --name $arcClusterName --resource-group $resourceGroup

This command might run for a while, but once it completes, you can go to your Azure portal and search for Azure Arc. In Azure Arc, Click on Kubernetes clusters under infrastructure, and you should see your connected Kubernetes cluster, like in the image below.

Once you have onboarded your on-premises or other cloud’s Kubernetes cluster to Azure, you can use Azure services centrally. A brief description of these services is provided below.

GitOps

GitOps is a method of deploying and managing applications using Git as a source of truth for declarative infrastructure and applications. It involves using Git to store declarative definitions of your infrastructure and applications and using automated tools to ensure that the actual infrastructure and applications match the desired state defined in Git.

By combining Azure Arc–enabled Kubernetes with GitOps, you can use Git as the source of truth for your Kubernetes clusters, allowing you to manage and deploy your applications consistently across different environments using a single control plane and a set of automated tools.

Azure Monitor

By using Azure Monitor with Azure Arc–enabled Kubernetes, you can gain valuable insights into the performance and availability of your clusters and applications and take proactive measures to optimize their performance and availability.

Azure Policy

Azure Policy is a service in Azure that helps you manage and enforce compliance standards across your Azure resources. You can use Azure Policy to define rules and policies that implement specific compliance standards or best practices and automatically enforce those rules and policies across your Azure resources.

You can use Azure Policy with Azure Arc–enabled Kubernetes to ensure compliance for your Kubernetes clusters and their applications. For instance, you can use Azure Policy to enforce security standards, such as requiring all images in your clusters to be scanned for vulnerabilities or to ensure that your clusters meet specific regulatory or compliance requirements.

Security: Azure Sentinel and Azure Defender

Azure Sentinel is a cloud-native security information and event management (SIEM) platform that helps you detect, investigate, and respond to threats across your organization. Azure Sentinel uses AI/ML to analyze data from multiple sources (including Azure resources, on-premises resources, and third-party solutions) to identify potential threats and anomalies.

Azure Defender is a security solution that provides advanced threat protection for Azure and non-Azure resources via Azure Arc.

You can use Azure Sentinel and Azure Defender with Azure Arc–enabled Kubernetes to help secure your Kubernetes clusters and the applications running on them. For example, you can use Azure Sentinel to monitor your clusters for potential security threats, such as unauthorized access or malicious activity, and use Azure Defender to protect against threats, such as SQL injection attacks or unauthorized access to Key Vaults.

Identity and Access Management

Azure Arc–enabled Kubernetes provides several features to help you manage identity and access for your Kubernetes clusters and their applications.

One key feature is Azure Active Directory (Azure AD) integration, which allows you to use Azure AD to manage identities and access your clusters. Azure AD can define roles and permissions for your users and groups and enforce access control policies for your clusters.

Another feature is integration with Azure Private Link, which allows you to securely access your clusters from your on-premises networks or other cloud providers using a private network connection.

Additionally, Azure Arc–enabled Kubernetes supports integration with Azure Managed Identities, allowing you to use managed identities for your applications and services rather than manually managing and rotating credentials. This reduces the risk of security breaches due to compromised credentials.

Pros

There are several benefits to using Azure Arc–enabled Kubernetes:

• Single control plane: Azure Arc–enabled Kubernetes allows you to manage and deploy your Kubernetes clusters using a single control plane, no matter where they are deployed.
• Consistent experience: Azure Arc–enabled Kubernetes provides a consistent experience across different environments, allowing you to use the same tools and processes to manage and deploy your applications.
• Integration with Azure services: Azure Arc–enabled Kubernetes integrates with Azure services such as Azure Monitor, Microsoft Defender and Azure Policy, allowing you to take advantage of Azure’s rich set of management and monitoring tools to manage and deploy your applications.
• Cross-cloud and on-premises support: Azure Arc–enabled Kubernetes supports deployment on other cloud providers and on-premises environments, providing flexibility to deploy your applications.

Cons

There are a few potential drawbacks to using Azure Arc–enabled Kubernetes:

1. Cost: Azure Arc–enabled Kubernetes may incur additional charges for using Azure services such as Azure Defender, Azure Sentinel, Azure Monitor, and Azure Policy. These costs can vary depending on your usage patterns and the specific services you use within and outside Azure. Consider using Kubecost to overcome this issue. Kubecost is a tool that helps organizations understand and optimize the cost of their Kubernetes clusters. It provides insights into the resource usage and cost of individual pods, nodes, and namespaces within a cluster and helps identify opportunities for cost optimization through resource optimization, autoscaling, and right-sizing.
2. Limited support for some features: As a new entrant, Azure Arc–enabled Kubernetes may only support some features (as listed under the “Key features” section) and capabilities of Kubernetes, depending on the specific version and configuration of your clusters. For example, it currently lacks the ability to provision additional worker nodes.

Azure Arc for Kubernetes is a valuable service for organizations looking to manage and secure their Kubernetes clusters across multiple environments. It provides a range of features that can help organizations streamline the management of their clusters and ensure that they are secure and compliant. Some of the key benefits of Azure Arc for Kubernetes include centralized management, a consistent experience, hybrid connectivity, and improved security.