4 of the 14 questions we have for this objective, shown with the answer and the reasoning.
Design, implement, and manage Azure ExpressRoute
Kingsmead Payments runs a 2 Gbps ExpressRoute circuit with private peering through a layer 2 connectivity provider. During a recent provider fault, one of the two links lost forwarding but the BGP session stayed up for almost three minutes before traffic failed over to the healthy link, because the Microsoft edge routers detected the failure only when the BGP hold timer expired. The company wants failover in about one second and has confirmed that its Cisco edge routers support BFD.
What should you do?
- AEnable BFD on the private peering BGP sessions on the customer edge routers; BFD is already enabled on the Microsoft edge routers for all newly created peerings.Microsoft enables BFD by default on its side of ExpressRoute peerings; you only need to configure BFD on your edge routers (or ask your provider for managed layer 3) to achieve sub-second failure detection.
- BLower the BGP keepalive and hold timers on the customer edge routers to 1 second and 3 seconds.Microsoft's edge uses fixed BGP timers of 60 seconds keepalive and 180 seconds hold; aggressive BGP timers are not negotiated and BFD is the supported method for fast detection.
- COpen a support request to enable BFD on the Microsoft edge routers for the circuit.BFD has been enabled by default on Microsoft's side since 2018; existing peerings can pick it up by resetting the peering, and no support request is needed.
- DEnable FastPath on the ExpressRoute gateway connection so that failures are detected in the data path.FastPath changes the path on-premises traffic takes into the virtual network; it has nothing to do with detecting link failures on the circuit.
Why: Over ExpressRoute, BGP keepalive and hold timers default to 60 and 180 seconds, so a link that stops forwarding without going down can take up to three minutes to be detected. Bidirectional Forwarding Detection runs lightweight hello packets between the BGP neighbors (300 millisecond interval with a multiplier of 3 on the Microsoft side) and tears down the BGP session within about a second when hellos stop. BFD is enabled by default on Microsoft's edge routers for new peerings, and for older peerings you can reset the peering to activate it; the only remaining step is to enable BFD on your own routers or your provider's layer 3 devices. Tuning BGP timers is the trap because Microsoft does not change its timers.
Design, implement, and manage Azure ExpressRoute
Oakhaven Health Systems is deploying ExpressRoute Direct with a 10 Gbps port pair in a colocation facility, and its own routers will connect directly to Microsoft's edge routers. Regulatory guidance requires that all data on the physical cross-connects between the hospital's routers and Microsoft's routers be encrypted at the link layer, with encryption keys that the hospital controls and stores in its own Azure Key Vault. The design must not reduce the 10 Gbps throughput available to private peering.
What should you configure?
- AMACsec on the ExpressRoute Direct ports, with the connectivity association key (CAK) and key name (CKN) stored as secrets in Azure Key Vault and accessed by a user-assigned managed identity.MACsec is a layer 2 encryption available only on ExpressRoute Direct ports; the CAK and CKN are stored in Key Vault and referenced through a user-assigned managed identity, and encryption occurs in hardware at line rate.
- BAn IPsec site-to-site VPN over Microsoft peering using an Azure VPN gateway.IPsec over Microsoft peering encrypts end-to-end at layer 3 but requires a VPN gateway whose throughput is far below 10 Gbps and does not encrypt the physical link for private peering traffic.
- CAn IPsec VPN over private peering terminating on a VpnGw5AZ gateway in each virtual network.IPsec over private peering is a valid way to encrypt traffic end to end, but it caps throughput at the VPN gateway's limit and does not satisfy the link-layer encryption requirement for the cross-connects.
- DEnable the Encryption setting on the ExpressRoute circuit's private peering.There is no encryption toggle on a peering; ExpressRoute circuits do not encrypt traffic by default and encryption must be added via MACsec on Direct or IPsec on top.
Why: ExpressRoute does not encrypt traffic by itself, and the documentation describes two complementary approaches. MACsec is an IEEE 802.1AE link-layer encryption available only on ExpressRoute Direct ports; it protects the physical connection between your edge and Microsoft's edge, runs at line rate, and uses a CAK and CKN that you store in Azure Key Vault and reference through a user-assigned managed identity on the ExpressRoute Direct resource. IPsec gives end-to-end encryption at layer 3 over private peering (with a VPN gateway or Virtual WAN) or over Microsoft peering, but throughput is bounded by the VPN gateway. The deciding constraints are link-layer encryption, customer-controlled keys, and full 10 Gbps throughput, which only MACsec on Direct satisfies.
Design, implement, and manage Azure ExpressRoute
Ambergate Insurance reports that virtual machines in a virtual network connected to its ExpressRoute circuit cannot reach on-premises servers. In the Azure portal, the circuit shows Circuit status Enabled and Provider status Provisioned, and the private peering is listed. The connection from the ExpressRoute gateway to the circuit shows a status of Succeeded. The on-premises team says their routers have the BGP session to Azure in the Idle state.
Which diagnostic step should you perform first?
- ARetrieve the ARP table and the BGP route table for the private peering from the circuit, to confirm layer 2 reachability and whether Azure is receiving on-premises prefixes.The ARP table shows whether Microsoft's edge sees the on-premises router at layer 2, and the route table shows the BGP state and learned routes; together they isolate the fault to the layer 2 provider path or to the BGP configuration.
- BReset the ExpressRoute gateway in the virtual network.The gateway connection already shows Succeeded, and the fault is between the on-premises routers and Microsoft's edge, so a gateway reset is unlikely to help.
- CDelete and recreate the circuit so the provider reprovisions it.Recreating the circuit is disruptive and loses the service key; the provider status already shows Provisioned, so provisioning is not the problem.
- DRun IP flow verify in Network Watcher against a virtual machine in the virtual network.IP flow verify checks network security group rules for a virtual machine; it does not examine the ExpressRoute BGP session or the layer 2 path.
Why: ExpressRoute troubleshooting follows the layers: circuit provisioning state, layer 2 connectivity across the provider, then BGP at layer 3. Because the circuit is Enabled and Provisioned and the gateway connection is healthy, the next check is the peering itself. Azure exposes the ARP table (Get-AzExpressRouteCircuitARPTable) and the route table (Get-AzExpressRouteCircuitRouteTable) for each peering; a missing on-premises MAC address in the ARP table indicates a layer 2 problem with the provider or VLAN configuration, while an ARP entry with an Idle or Active BGP state points to a mismatch in ASN, peer IP, or MD5 key. Resetting the gateway, recreating the circuit, and IP flow verify are all real tools used at the wrong layer here.
Design, implement, and manage Azure ExpressRoute
Hollis Grain Trading runs a latency-sensitive trading application on virtual machines in a virtual network named vnet-trade in the UK South region. The virtual network will connect to a 10 Gbps ExpressRoute circuit. Requirements: the gateway must be zone-redundant, the on-premises-to-VM data path must bypass the gateway so that the gateway is not a throughput bottleneck, and the gateway must be deployed into an existing /27 GatewaySubnet alongside a VpnGw1AZ VPN gateway. The company wants to use fixed-capacity SKUs rather than autoscaling.
Which ExpressRoute gateway SKU should you deploy?
- AErGw1AZErGw1AZ is zone-redundant but does not support FastPath, so the gateway would remain in the data path and limit throughput to roughly 1 Gbps.
- BUltraPerformanceUltraPerformance supports FastPath and 10 Gbps, but it is not a zone-redundant SKU and does not meet the zone redundancy requirement.
- CErGw3AZErGw3AZ is the zone-redundant equivalent of UltraPerformance; it supports about 10 Gbps, is the only fixed-capacity AZ SKU that supports FastPath, and can coexist with the VPN gateway in the /27 GatewaySubnet.
- DErGw2AZErGw2AZ is zone-redundant with about 2 Gbps throughput, but FastPath requires UltraPerformance, ErGw3AZ, or ErGwScale with at least 10 scale units.
Why: ExpressRoute gateway SKUs come in a non-zone-redundant family (Standard, HighPerformance, UltraPerformance), a zone-redundant family (ErGw1AZ, ErGw2AZ, ErGw3AZ), and ErGwScale, which scales between 1 and 40 scale units of about 1 Gbps each. FastPath, which sends on-premises traffic directly to the virtual machines and bypasses the gateway, is supported only on UltraPerformance, ErGw3AZ, and ErGwScale with a minimum of 10 scale units. The deciding constraints are zone redundancy, FastPath, and a fixed-capacity SKU, which leaves ErGw3AZ as the single answer. A GatewaySubnet of /27 or larger supports VPN and ExpressRoute gateway coexistence, which the scenario already satisfies. UltraPerformance is the trap because it has the right throughput and FastPath but no zone redundancy.