Cloud Networking Deep Dive: VPCs, Subnets, Routing, Firewalls

VPCs are not isolated private clouds; they are just a flat, routable network segment that you’ve been given control over.

Let’s watch a packet travel from one EC2 instance to another, both within the same VPC but in different subnets.

Imagine you have two EC2 instances, i-0a1b2c3d4e5f67890 and i-0fedcba9876543210, in your VPC. i-0a1b2c3d4e5f67890 is in subnet subnet-0123456789abcdef0 (CIDR 10.0.1.0/24) and i-0fedcba9876543210 is in subnet-0fedcba987654321 (CIDR 10.0.2.0/24).

When i-0a1b2c3d4e5f67890 (IP 10.0.1.10) wants to send a packet to i-0fedcba9876543210 (IP 10.0.2.20), the following happens:

Local Routing Table Lookup: The instance’s operating system consults its own routing table. It sees that 10.0.2.0/24 is a directly connected network.
ARP Resolution: The instance’s network interface (vNIC) needs the MAC address of the destination IP (10.0.2.20). It sends an ARP request: "Who has 10.0.2.20? Tell 10.0.1.10."
VPC Network Fabric: The VPC’s network fabric intercepts this ARP request. Since 10.0.2.20 is within the VPC’s address space and is in a different subnet, the VPC fabric knows how to deliver it. It’s not sent out to the internet or an IGW. The fabric will eventually deliver the ARP reply directly to i-0a1b2c3d4e5f67890 (likely via a virtual MAC address associated with the destination instance).
Packet Forwarding: Once the MAC address is resolved, the packet is sent from i-0a1b2c3d4e5f67890’s vNIC. The VPC network fabric inspects the destination IP (10.0.2.20).
VPC Route Table: The VPC’s main route table (or the one associated with subnet-0123456789abcdef0) has a local route for 10.0.0.0/16 (which covers both 10.0.1.0/24 and 10.0.2.0/24). This route tells the VPC fabric to keep the traffic within the VPC.
Delivery: The VPC fabric routes the packet directly to the network interface of i-0fedcba9876543210.

This entire process happens at the VPC network layer, abstracted away from you, but it’s crucial to understand that the "routing" for traffic within the VPC is handled by AWS’s internal fabric, not by your instances or explicit route table entries (beyond the implicit "local" route).

Now, let’s consider traffic leaving the VPC. Suppose i-0a1b2c3d4e5f67890 (IP 10.0.1.10) wants to reach 8.8.8.8 (Google DNS).

Local Routing Table Lookup: The instance’s OS looks at its routing table. It doesn’t have a specific route for 8.8.8.8.
VPC Route Table Lookup: The route table associated with subnet-0123456789abcdef0 is consulted. It has a default route (0.0.0.0/0) pointing to an Internet Gateway (IGW) named igw-0123456789abcdef0.
Packet Forwarding to IGW: The packet is sent from i-0a1b2c3d4e5f67890’s vNIC towards the IGW.
NAT (if applicable): If i-0a1b2c3d4e5f67890 is in a private subnet and needs to access the internet, it will likely need a NAT Gateway or NAT Instance. The traffic would be routed to the NAT Gateway’s ENI, which then performs Network Address Translation, replacing the private source IP (10.0.1.10) with the NAT Gateway’s Elastic IP.
Internet Egress: The packet, now with a public IP source address (either the instance’s EIP if it has one, or the NAT Gateway’s EIP), leaves the VPC via the IGW and enters the public internet.

Key Concepts:

VPC: A virtual, isolated network in AWS. You define its IP address range (CIDR block).
Subnets: Divisions of your VPC’s IP address range. Each subnet must reside entirely within one Availability Zone. Subnets are associated with a specific route table.
Route Tables: A set of rules (routes) that determine where network traffic from your subnet is directed. Each subnet must be associated with a route table.
Internet Gateway (IGW): A horizontally scaled, redundant, and highly available VPC component that allows communication between your VPC and the internet.
Network Access Control Lists (NACLs): A stateless firewall at the subnet level. They are evaluated in order.
Security Groups: Stateful firewalls that act at the instance level. They are evaluated in order.

Example Configuration:

Let’s say your VPC has CIDR 10.0.0.0/16. You create two subnets:

subnet-public (CIDR 10.0.1.0/24) in us-east-1a
subnet-private (CIDR 10.0.2.0/24) in us-east-1b

You create a route table for public subnets:

Route 1: 10.0.0.0/16 -> local (This is implicit for all VPCs)
Route 2: 0.0.0.0/0 -> igw-xxxxxxxxxxxxxxxxx (Your Internet Gateway)

You create a route table for private subnets:

Route 1: 10.0.0.0/16 -> local
Route 2: 0.0.0.0/0 -> nat-xxxxxxxxxxxxxxxxx (Your NAT Gateway)

You associate subnet-public with the public route table and subnet-private with the private route table.

Firewall Rules (Security Group Example):

To allow SSH from anywhere to an instance in subnet-public:

Type: SSH
Protocol: TCP
Port Range: 22
Source: 0.0.0.0/0

To allow an instance in subnet-private to initiate outbound connections to the internet (e.g., for software updates):

Type: All traffic
Protocol: All
Port Range: All
Source: 10.0.2.0/24 (The CIDR of the private subnet)

The surprising truth is that NACLs are stateless, meaning you must explicitly allow inbound traffic and outbound traffic for the same protocol and port. If you allow TCP port 80 inbound, you also need to allow TCP port 1024-65535 outbound (the ephemeral port range for return traffic) for the NACL to be effective.

The next step in mastering VPC networking is understanding how VPC Peering and Transit Gateway can connect multiple VPCs together.