Otechy - Azure Cloud Architecture, Cloud Adoption Framework, Well-Architected Frameworks and More

Avoiding CIDR Overlap in AKS

Welcome back to my Azure Kubernetes Service (AKS) learning journey! Over the past few days, we’ve covered a lot of networking concepts, from Kubenet and Azure CNI to custom BYO CNI setups.

Today, I learned about something that can completely break a cluster if you aren't careful: IP Overlapping.

Because an AKS cluster relies on multiple different IP address ranges (CIDRs) to function, planning your network before you hit the "Create" button is absolutely critical. Here is what I discovered about avoiding the dreaded CIDR overlap.

---

Four Network Spaces of an AKS Cluster

When you deploy an AKS cluster, you aren't just assigning one IP range. You are actually juggling four different network spaces:

1. Subnet CIDR: The physical IPs in your Azure Virtual Network (VNet) where your nodes live.
2. Pod CIDR: The internal IPs assigned to your individual application pods.
3. Service CIDR: The internal VIPs (Virtual IPs) used by Kubernetes Services to route traffic to your pods.
4. Docker Bridge CIDR: The network address used by the container runtime on the nodes themselves.
   
   

---

Rule #1: Do Not Overlap Within the Same Cluster

The most important rule I learned today is that none of these four CIDR ranges can overlap with each other within the same cluster. If your Pod CIDR overlaps with your Service CIDR, or your Docker Bridge overlaps with your Subnet, the cluster’s internal routing gets terribly confused. Traffic won't know where to go, and your cluster will start behaving in very strange, unpredictable ways.

Rule #2: Keep External Networks in Mind

It is not enough to just ensure your cluster's internal IPs are unique. You also have to think about the "big picture."

Your Subnet, Pod, Service, and Docker Bridge CIDRs must not overlap with:

• The broader Azure Virtual Network (VNet) your cluster sits in.
• Any other peered VNets in your Azure environment.
• Your on-premises company network (if you are connecting to Azure via a Site-to-Site VPN or ExpressRoute).

If an internal cluster IP overlaps with your company's on-premise database IP, your pods will never be able to reach that database!

Rule #3: You Can Reuse Internal IPs Across Different Clusters

Here is an interesting design pattern I picked up: while the Subnet must be unique for every cluster in your VNet, the internal ranges (Pod, Service, and Docker Bridge) can be exactly the same across multiple different clusters.

Because the Pod and Service IPs are completely internal and isolated to their specific cluster (especially when using Kubenet or CNI Overlay), Cluster A and Cluster B can both use 10.244.0.0/16 for their pods without any issues.

Best Practice: Technically, you can deploy multiple AKS clusters into the exact same Subnet, but it is highly discouraged. It is best practice to give every single cluster its own dedicated Subnet.

 

Rule #4: "Forbidden" IP Ranges

Azure reserves certain IP ranges for cloud infrastructure and internal services. During cluster creation, you must never use the following CIDRs for your Subnets, Pods, or Services:

• 169.254.0.0/16
• 172.30.0.0/16
• 172.31.0.0/16
• 192.0.2.0/24

Using any of these will cause immediate conflicts with Azure's underlying fabric.

---

Hands-On: Defining CIDR Ranges in the Azure CLI

To put this theory into practice, I looked at how we actually assign these values when creating a cluster. If you don't want to rely on Azure's default IP assignments (which is a good idea for production environments), you can explicitly declare them in the Azure CLI.

Here is an example command I put together to create a Kubenet cluster with custom, non-overlapping IP ranges:

az aks create \

    --resource-group myResourceGroup \
    --name myAKSCluster \
    --network-plugin kubenet \
    --service-cidr 10.0.0.0/16 \
    --dns-service-ip 10.0.0.10 \
    --pod-cidr 10.244.0.0/16 \
    --docker-bridge-address 172.17.0.1/16 \
    --vnet-subnet-id $SUBNET_ID

Breaking down the flags:

• --service-cidr: Sets the range for my internal Kubernetes services.
• --dns-service-ip: This is the IP address reserved for CoreDNS (the cluster's internal DNS resolver). It must be an IP address within the --service-cidr range (usually, people pick the 10th IP, hence .10).
• --pod-cidr: Sets the internal range where my pods will pull their IPs from.
• --docker-bridge-address: Sets the IP address and subnet mask for the container runtime bridge on the nodes.
• --vnet-subnet-id: Links the cluster to the specific Azure VNet subnet I created for my nodes.

Ultimate Warning: Measure Twice, Cut Once

Why is explicitly defining and checking all this IP math so important?

Because you cannot update these CIDR ranges after the cluster is created. If you make a mistake, run out of IPs, or accidentally create an overlap with an on-premises network, there is no "Edit IP Range" button. You have to completely destroy the cluster and rebuild it from scratch.

Key Takeaways

• An AKS cluster uses multiple IP ranges (Subnet, Pod, Service, Docker Bridge).
• These ranges must never overlap with each other, your VNet, or your on-premise networks.
• You can safely reuse internal Pod and Service CIDRs across different clusters.
• Give every cluster its own dedicated Subnet.
• Avoid Azure's reserved IP ranges.
• Use the Azure CLI to explicitly define your CIDR blocks to maintain full control.
• Plan ahead: CIDR ranges are permanent once the cluster is deployed!