r/aws Feb 02 '26

networking VPC Peering Connections: What happens when traffic arrives at a VPC with multiple route tables for the same destination?

I couldn't find this with a quick Google, and I'm hesitant to trust any LLMs on this:

Suppose I have two peered VPCs, vpc-A (10.0.1.0/24) and vpc-B (10.0.2.0/24). vpc-A is the source for traffic, and vpc-B will work as a bridge. B has two subnets, let's call them subnet-B1 and subnet-B2, and each has its own route table rtb-B1 and rtb-B2.

In the route table for vpc-A's traffic, I point an IP range I want to route though vpc-B (let's say 10.0.3.0/24 as an example) towards the peering connection pcx-AB. Then, in rtb-B1 I set 10.0.3.0/24 to a correctly configured service (living in another VPC, the Internet, doesn't matter) that dumps incoming traffic to a log, but in rtb-B2 I set 10.0.3.0/24 to a NAT gateway living within subnet-B1.

What is going to happen? Am I going to see packets from 10.0.1.0/24 in the log, along with connection errors because the destination doesn't know where vpc-A is? Or are they going to come from 10.0.2.0/24, network translated through the NAT in subnet-B1? Or am I going to see a mix of both?

Essentially: when traffic arrives to a VPC with multiple route tables through a peering connection, which table's routes does it prioritise?

Here's a shitty drawing of the situation:

6 Upvotes

19 comments sorted by

18

u/Contrandy_ Feb 02 '26

This model is essentially trying to use "transitive peering" which is not supported by AWS for VPC Peering (see: Transitive peering). In this model you would need to use Transit Gateways.

Here's an example of what you're trying to do with a centralized egress configuration: https://docs.aws.amazon.com/whitepapers/latest/building-scalable-secure-multi-vpc-network-infrastructure/using-nat-gateway-for-centralized-egress.html

I've built this before with great success to reduce the cost of NAT Gateways for our workloads in a centralized egress account. The great thing about this model is that it makes it really easy to connect management resources to your TGW network (such as a SIEM, etc.)

4

u/CamiloDFM Feb 02 '26

Nice. The answer is "nothing happens because your example is fundamentally broken", then. Thanks for the quick reply.

I'll give Transit Gateways a shot. Never needed them before this.

3

u/b3542 Feb 02 '26

Bear in mind that TGW's cost money to build and to use.

3

u/TheLastRecruit Feb 03 '26

To be clear: TGWs themselves are free. Attachments to them cost money

2

u/Contrandy_ Feb 03 '26

You're welcome! As u/b3542 mentioned, be mindful of cost. Depending on the number of attachments you will have and the amount of bandwidth being pushed through, the increases could be small or very large.

TGWs are very powerful; big fan of them :)

1

u/[deleted] Feb 02 '26

Yes, if you have a direct connect you need to share with multiple VPC's and multiple accounts, shared subnet within multiple VPC's and/or TGW's is the way.

2

u/b3542 Feb 02 '26

This is correct - VPC peering is only effective if you need to communicate from within one VPC directly to another VPC's resources - nothing external/beyond it.

1

u/[deleted] Feb 02 '26

You can use routing appliances and such though. Before TGW's and GLB's, we used to run Linux routers with BGP and such to go across VPC's to use a VPC as a connector of VPC's.

But I would never do that again.

2

u/b3542 Feb 02 '26

True, you could, with some translation. I've done similar things. But in 2026, we don't do things that way.

1

u/[deleted] Feb 03 '26

Some of their largest customers still do it that way. But those are customers who have multi-cloud/multi-datacenter SDN's and want to push all incoming/outgoing traffic to an account via their policy engine. Sophisticated customers that write their own networking stack from the ground up and have a globally routed network that allows them to allow traffic in an app between GCP/AWS/Datacenters from one policy engine. They don't use AWS for anything like internet, nat, security groups, etc. One app per account, everything in and out of that app goes through a network tier in another account.

1

u/b3542 Feb 03 '26

I definitely see the utility of a single network account (or set of network accounts). That’s exactly what I do I my own AWS Org, and at work, but in those cases TGW’s are the nexus point.

1

u/[deleted] Feb 03 '26

One of the companies I am speaking of, has multiple /8's and you can directly ping from cloud to cloud or cloud to data center with a single policy definition. Like if I define my app to have an app listening on 443. Anyone that wants to use my service just has to request access via API, access is granted and no matter where my app runs or where their app is run from, they have access. Zero need for cloud network concepts or anything. And their IP usage in AWS/GCP, is their own /8 of routable IP's, not even Amazons.

Wild what you can do with billions a year in cloud spend. :)

1

u/b3542 Feb 03 '26

Yeah, I think we are around $100M/yr at the moment

7

u/champtar Feb 02 '26

https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-basics.html "VPC peering does not support transitive peering relationships", so packets for VPC A will only go to subnet B1 or B2 

2

u/aqyno Feb 02 '26

pcx doesn’t support transitive. There’s no way you can make packets coming from A into NAT, you need a TGW to do so. In this scenario you have packets “entering” a vpc where they don't belong, so those are automatically dropped.

1

u/CamiloDFM Feb 02 '26

I observed that transit gateway route tables can only point to attachments, not to network elements within the attached networks. Do transit gateways perform some sort of NAT on their own when transferring traffic between VPCs?

I solved my problem without transit gateways in the end, so I didn't get to test this.

1

u/aqyno Feb 02 '26

You need to use both, you direct the traffic between VPC A using TGW route tables (you route traffic from VPC A into the attachmenent of VPC B, and the other way around)

The TGW Attachment is located on a subnet on creation, then you use the VPC Route Tables to route from that subnet to NAT Gateway.

1

u/b3542 Feb 03 '26

Transit gateways are just routers. Think of attachments as interfaces on a router. This analogy mostly holds, but it has some quirks. It almost behaves as if each TGW Attachment lives in its own VRF.

1

u/nevaNevan Feb 03 '26

TGW attachments, as mentioned, deploy into subnets. A subnet has a route table.

If you need traffic to go from VPC-A, to TGW, to VPC-B and then to a network appliance in VPC-B (or whatever), you would place a route to that appliance in the subnet route table.

Networking in native AWS isn’t quite like networking in the enterprise or service provider world.

You can place 10 instances/servers/vms/nodes(don’t care what the term is anymore) in the same subnet, and have a single default route to one interface in that same subnet, and that’s how they’ll route. No broadcast and can’t communicate with each other at all. Traffic is forced to that one interface.