18 Cloud Networking Concepts DevOps Engineers Must Know

18 cloud networking concepts for DevOps: CIDR planning, subnets, routes, NAT, security groups vs NACLs, load balancers, private endpoints, flow logs.

Dec 31, 2025 - 15:34
Nov 7, 2026 - 06:34
 0  101
18 Cloud Networking Concepts DevOps Engineers Must Know

The cloud networking concepts DevOps engineers use most are virtual networks and CIDR planning, public and private subnets, route tables, internet and NAT gateways, stateful and stateless firewalls, load balancers, private DNS, private endpoints, peering and transit hubs, hybrid links, and flow logs. Knowing how these fit together lets you design, automate and debug cloud environments without guessing.

This guide explains 18 concepts with the names each major provider uses, the mistakes that cause outages, and how to check each one. It is part of our networking cluster; for on-premises fundamentals, see how subnetting improves efficiency and security, and for networking inside clusters see 12 steps to debug networking in Kubernetes.

Key takeaways

  • Plan IP ranges before you build. Overlapping CIDR blocks block peering and VPN later, and renumbering a live network is painful.
  • Keep workloads in private subnets. Only load balancers and a few gateways need public addresses.
  • Security groups are stateful; AWS network ACLs are stateless. Mixing up the two is a classic cause of "it works one way only" bugs.
  • VPC peering is not transitive. Use a hub (AWS Transit Gateway, Azure Virtual WAN or hub VNet, Google Network Connectivity Center) when you have many networks.
  • Private endpoints keep traffic to managed services off the internet and often reduce NAT data charges.
  • Turn on flow logs early. They are the first thing you need when a connection silently fails.

Provider names at a glance

ConceptAWSAzureGoogle Cloud
Virtual networkVPCVirtual Network (VNet)VPC network (global)
Instance firewallSecurity groupNetwork security group (NSG)VPC firewall rules and policies
Subnet firewallNetwork ACLNSG on subnetFirewall rules (no separate ACL)
Outbound NATNAT gatewayNAT GatewayCloud NAT
Private access to servicesPrivateLink, VPC endpointsPrivate Endpoint, Private LinkPrivate Service Connect
Network hubTransit Gateway, Cloud WANVirtual WAN, hub VNetNetwork Connectivity Center
Dedicated linkDirect ConnectExpressRouteCloud Interconnect
Traffic logsVPC Flow LogsVNet flow logsVPC Flow Logs

Addressing and layout

1. Virtual networks (VPC and VNet)

A virtual network is your isolated, software-defined network in a cloud region. You choose its address range, divide it into subnets and control how traffic enters and leaves. In AWS and Azure a VPC or VNet lives in one region; in Google Cloud a VPC network is global and its subnets are regional, which changes how you design multi-region setups.

Most teams create separate networks per environment (dev, staging, prod) or per account or subscription, so a mistake in one cannot reach the others.

2. CIDR planning

Addresses are written in CIDR notation: 10.20.0.0/16 gives 65,536 addresses, /24 gives 256. Use the private ranges from RFC 1918 (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16). Each cloud also reserves some addresses in every subnet: AWS and Azure reserve five, Google Cloud reserves four in each primary range.

Write an IP plan before you build. Give every environment and region a non-overlapping block, leave room to grow, and remember that Kubernetes pods and services may need large ranges of their own. Overlaps become a blocker the day you need peering, VPN or a merger.

10.0.0.0/8     company
  10.10.0.0/16   prod  ap-south-1
  10.11.0.0/16   prod  second region
  10.20.0.0/16   staging
  10.30.0.0/16   dev
  10.100.0.0/14  Kubernetes pod ranges

3. Public and private subnets

A subnet is "public" when its route table sends 0.0.0.0/0 to an internet gateway, and "private" when it does not. Put load balancers and bastions (if you still use them) in public subnets; put application servers, databases and caches in private subnets. Spread subnets across at least two availability zones so losing one zone does not take the service down.

4. Route tables

Route tables decide where packets go next. Each subnet is associated with one. The most specific matching prefix wins. A typical private subnet table looks like this:

DestinationTargetPurpose
10.10.0.0/16localTraffic inside the VPC
10.0.0.0/8Transit gatewayOther company networks
0.0.0.0/0NAT gatewayOutbound internet

When "nothing can connect", check the route table before the firewall. A missing return route is as common as a blocked port.

5. Internet gateway and NAT

An internet gateway lets resources with public IPs talk to the internet in both directions. A NAT gateway lets private resources make outbound connections (for package updates, external APIs) without being reachable from outside. Run one NAT gateway per availability zone for resilience, and watch its data processing charges; large image pulls or backups through NAT add up quickly.

6. IPv6 and public IPv4

Public IPv4 addresses are scarce, and AWS has charged for every public IPv4 address, attached or not, since February 2024. IPv6 gives each resource a globally unique address and removes the need for NAT, though you then need an egress-only gateway (AWS) or firewall rules to block unsolicited inbound traffic. Dual-stack (IPv4 and IPv6 together) is the usual migration path. Check that your load balancers, security rules and applications all handle IPv6 before turning it on.

Security controls

7. Security groups and NSGs (stateful)

Security groups (AWS) and network security groups (Azure) filter traffic at the instance or interface. They are stateful: if you allow inbound traffic on port 443, the return traffic is allowed automatically. In AWS you can reference another security group as the source, which is cleaner than IP lists.

resource "aws_security_group_rule" "app_from_alb" {
  type                     = "ingress"
  from_port                = 8080
  to_port                  = 8080
  protocol                 = "tcp"
  security_group_id        = aws_security_group.app.id
  source_security_group_id = aws_security_group.alb.id
}

8. Network ACLs (stateless)

AWS network ACLs filter at the subnet boundary and are stateless, so you must allow return traffic explicitly, usually the ephemeral port range (1024 to 65535) for responses. Rules are evaluated in number order and the first match wins. Most teams keep NACLs simple and do the fine-grained work in security groups.

9. Zero trust and segmentation

Zero trust means no implicit trust because something is "inside" the network. Every request is authenticated and authorised, whether it comes from the internet or the next subnet. In practice that means identity-aware proxies instead of VPNs for admin access, mutual TLS between services (often through a service mesh), small segments with explicit allow rules, and short-lived credentials. NIST SP 800-207 is the standard reference.

Traffic distribution

10. Layer 4 and Layer 7 load balancers

Layer 4 load balancers (AWS Network Load Balancer, Azure Load Balancer, Google passthrough Network Load Balancer) forward TCP and UDP connections with very low overhead. Layer 7 load balancers (AWS Application Load Balancer, Azure Application Gateway, Google Application Load Balancer) understand HTTP, so they can route by host or path, terminate TLS, and attach a web application firewall.

NeedChoose
Path or host routing, TLS offload, WAFLayer 7
Non-HTTP protocols, static IPs, extreme throughputLayer 4
Expose a service privately to other accountsLayer 4 behind PrivateLink or Private Service Connect

Health checks matter more than the algorithm. A health check that only tests "port open" will keep sending traffic to an app that returns errors. Kubernetes-specific patterns are in 12 Kubernetes load balancing techniques.

11. DNS and private zones

DNS turns names into addresses, and in the cloud it also drives failover and service discovery. Learn record types (A, AAAA, CNAME, alias records, TXT, MX), TTLs, and private hosted zones that resolve only inside your networks (Route 53 private hosted zones, Azure Private DNS, Cloud DNS private zones). Hybrid setups need DNS forwarding in both directions so on-premises servers can resolve cloud names and the other way round.

dig +short api.internal.example.com
dig api.example.com +trace
nslookup db.privatelink.database.windows.net

12. Global traffic management, anycast and CDN

For users in many regions, route them to the closest healthy endpoint. DNS-based routing (latency or geolocation records) works with any backend. Anycast-based services (AWS Global Accelerator, Google's global load balancer, Azure Front Door) advertise one IP from many locations so traffic enters the provider's network near the user. A CDN caches static and some dynamic content at the edge, which cuts latency and origin load.

Connecting networks

13. VPC peering

Peering connects two virtual networks privately so they route to each other as if on one network. It is simple and has no bandwidth bottleneck, but it is not transitive: if A peers with B and B with C, A cannot reach C through B. CIDR ranges must not overlap. Peering works well for a handful of networks; beyond that the mesh of connections becomes hard to manage.

14. Transit hubs

A hub-and-spoke design connects every network to one central router: AWS Transit Gateway, Azure Virtual WAN or a hub VNet with a firewall, Google Network Connectivity Center. The hub gives you transitive routing, a central place for inspection firewalls and on-premises links, and route tables that separate prod from non-prod. Automate hub attachments in Terraform so new accounts join the network consistently.

15. Private endpoints for managed services

By default, traffic from a private subnet to a managed service (object storage, a container registry, a database API) goes out through NAT to the service's public endpoint. Private endpoints give the service a private IP inside your network instead. AWS calls these interface endpoints (PrivateLink) and gateway endpoints (for S3 and DynamoDB), Azure calls them Private Endpoints, Google calls it Private Service Connect. Benefits: no internet path, policies that restrict which network can reach a resource, and often lower NAT charges. Remember to update DNS so the service name resolves to the private IP.

16. Hybrid connectivity: VPN and dedicated links

Site-to-site IPsec VPN runs over the internet and is quick to set up; it suits low to moderate bandwidth and backup paths. Dedicated links (AWS Direct Connect, Azure ExpressRoute, Google Cloud Interconnect) give private, consistent bandwidth through a colocation or partner. Both typically use BGP to exchange routes. Plan for redundancy: two tunnels or two circuits in different locations, and test failover.

Operations

17. Flow logs and network observability

Flow logs record accepted and rejected connections (source, destination, port, protocol, action) at the subnet or interface level. They answer "is the traffic even arriving" and "which rule dropped it". Send them to your log platform, keep a retention policy, and build saved queries for rejected traffic to critical ports. Provider tools such as AWS Reachability Analyzer, Azure Network Watcher and Google Connectivity Tests simulate a path and tell you which hop blocks it.

# AWS VPC flow log record (default format, abbreviated)
2 123456789012 eni-0abc 10.10.1.25 10.10.2.40 49152 5432 6 10 840 1730966400 1730966460 REJECT OK

That record shows a TCP connection (protocol 6) to PostgreSQL on port 5432 being rejected, so look at the database security group or NACL.

18. Network as code

Every concept above should be in code: Terraform or OpenTofu modules for networks, subnets, routes and firewalls; policy checks in CI that block rules like 0.0.0.0/0 on SSH or database ports; and pull requests for every change so there is a review and a history. This makes networks reproducible across environments and stops manual drift. The DevOps tools for network automation post covers the tooling.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  name    = "prod"
  cidr    = "10.10.0.0/16"
  azs             = ["ap-south-1a", "ap-south-1b"]
  private_subnets = ["10.10.1.0/24", "10.10.2.0/24"]
  public_subnets  = ["10.10.101.0/24", "10.10.102.0/24"]
  enable_nat_gateway     = true
  one_nat_gateway_per_az = true
}

A troubleshooting order for "it cannot connect"

  1. DNS: does the name resolve to the address you expect, public or private?
  2. Routing: does the source subnet have a route to the destination, and the destination a route back?
  3. Firewalls: security group or NSG on both ends, then NACLs, then any central firewall.
  4. Listener: is the service actually listening on that port and interface (ss -tlnp)?
  5. Flow logs and reachability tools: confirm where packets stop.
  6. Application: TLS errors, wrong host headers or health check paths.

Common mistakes

  • Using the same 10.0.0.0/16 in every account, then discovering peering is impossible.
  • Opening SSH or database ports to 0.0.0.0/0 "temporarily".
  • One NAT gateway for all zones, which becomes a single point of failure.
  • Creating a private endpoint but forgetting the DNS change, so traffic still goes over NAT.
  • Load balancer health checks that pass while the application returns errors.
  • No flow logs until the day of an incident.

Learn cloud networking hands-on

VPC design, security groups, load balancing and hybrid connectivity are core topics in the AWS Solutions Architect Associate curriculum. Our AWS Solutions Architect Associate training in Pune covers them with labs, in Kharadi or live online.

Frequently asked questions

What networking should a DevOps engineer know?

IP addressing and CIDR, subnets and routing, DNS, TCP and UDP ports, TLS, NAT, firewalls, load balancers, and how cloud providers implement these as VPCs, security groups and gateways. Enough to design a network, automate it and debug failed connections.

What is the difference between a security group and a network ACL?

A security group is stateful and attached to instances or interfaces, so return traffic is allowed automatically. An AWS network ACL is stateless and attached to subnets, so you must allow return traffic explicitly, and rules are evaluated in number order.

Why is VPC peering not transitive?

Peering creates a direct route only between the two networks involved. Traffic cannot pass through a peered network to reach a third one. For many networks, use a transit hub such as AWS Transit Gateway or Azure Virtual WAN.

What is a private endpoint?

A private IP address inside your virtual network that connects to a managed cloud service, so traffic does not go over the internet. AWS PrivateLink, Azure Private Endpoint and Google Private Service Connect provide it.

When should I use a Layer 4 instead of a Layer 7 load balancer?

Use Layer 4 for non-HTTP protocols, static IPs or very high connection rates. Use Layer 7 when you need host or path routing, TLS termination, HTTP health checks or a web application firewall.

How many IP addresses does a cloud subnet reserve?

AWS and Azure reserve five addresses in every subnet. Google Cloud reserves four in each subnet's primary range. Account for these when sizing small subnets.

How do I debug a connection that times out in the cloud?

Check DNS, then route tables in both directions, then security groups and ACLs, then whether the service is listening. Use flow logs and the provider's reachability or connectivity test tool to see exactly where packets are dropped.

Do I need IPv6 in the cloud?

Not always, but it is worth learning. Public IPv4 is limited and AWS charges for every public IPv4 address. Dual-stack networks let you adopt IPv6 gradually while keeping IPv4 working.

References

What's Your Reaction?

Like Like 0
Dislike Dislike 0
Love Love 0
Funny Funny 0
Angry Angry 0
Sad Sad 0
Wow Wow 0
Mridul I am a passionate technology enthusiast with a strong focus on DevOps, Cloud Computing, and Cybersecurity. Through my blogs at DevOps Training Institute, I aim to simplify complex concepts and share practical insights for learners and professionals. My goal is to empower readers with knowledge, hands-on tips, and industry best practices to stay ahead in the ever-evolving world of DevOps.