top of page
  • 2 days ago
  • 23 min read

Static route tables work well for small and predictable environments. They become more difficult to operate when branch prefixes, security appliances, and network paths change regularly.

Azure Route Server introduces dynamic route exchange into an Azure virtual network. It establishes Border Gateway Protocol sessions with network virtual appliances and distributes the learned routes to Azure workloads.

In this workshop, you will build a complete Azure Route Server lab using PowerShell 7 and Azure CLI.

The lab contains:

  • one hub virtual network;

  • two spoke virtual networks;

  • one Azure Route Server;

  • two Linux-based network virtual appliances;

  • four BGP sessions per NVA pair;

  • an emulated external network prefix;

  • automatic route injection into both spokes;

  • Equal-Cost Multi-Path routing;

  • an NVA failover test;

  • optional Linux workload VMs for data-plane validation.

All configuration is performed from the command line.

No Azure portal steps, separate scripts, ARM templates, Bicep files, or Terraform files are required.

Cost warningAzure Route Server, public IP addresses, and virtual machines are billable resources.Complete the cleanup section when the workshop is finished.

Workshop overview

Item

Configuration

Level

Intermediate to advanced

Estimated duration

75–120 minutes

Deployment interface

PowerShell 7 with Azure CLI

Azure region

West Europe

Routing protocol

External BGP

Route Server ASN

Retrieved dynamically from Azure

NVA ASN

65001

NVA implementation

Ubuntu with FRRouting

High availability

Two active NVAs

Route distribution

Azure Route Server

Failover test

Deallocate one NVA

Test prefix

192.168.100.0/24

Test service IP

192.168.100.10

What you will learn

By completing this workshop, you will learn how to:

  1. Design a Route Server hub-and-spoke topology.

  2. Create the required RouteServerSubnet.

  3. Deploy Azure Route Server from Azure CLI.

  4. Configure hub-to-spoke gateway transit.

  5. Configure spoke-to-hub remote Route Server usage.

  6. Deploy two Linux router appliances.

  7. Enable NIC and operating-system IP forwarding.

  8. Install and configure FRRouting.

  9. Establish BGP sessions with both Route Server instances.

  10. Advertise an emulated external prefix.

  11. Inspect routes learned by Route Server.

  12. Inspect routes advertised to the NVAs.

  13. Validate route injection in spoke effective routes.

  14. Test the routed data path.

  15. Observe Equal-Cost Multi-Path routing.

  16. Simulate an NVA failure.

  17. Validate BGP route withdrawal and recovery.

  18. Troubleshoot common routing problems.

  19. Remove the complete environment.

Architecture

                         Emulated external network
                             192.168.100.0/24
                                      │
                              Service IP
                            192.168.100.10
                                      │
                         Advertised by both NVAs
                                      │
                      ┌───────────────┴───────────────┐
                      │                               │
              ┌───────▼────────┐              ┌───────▼────────┐
              │ NVA 01         │              │ NVA 02         │
              │ 10.0.1.4       │              │ 10.0.1.5       │
              │ ASN 65001      │              │ ASN 65001      │
              │ FRRouting      │              │ FRRouting      │
              └───────┬────────┘              └───────┬────────┘
                      │                               │
                      │ BGP to both Route Server IPs │
                      │                               │
         ┌────────────▼───────────────────────────────▼────────────┐
         │                    Hub VNet                              │
         │                    10.0.0.0/16                           │
         │                                                         │
         │  RouteServerSubnet        NvaSubnet                     │
         │  10.0.0.0/26              10.0.1.0/24                   │
         │                                                         │
         │             ┌─────────────────────────┐                 │
         │             │ Azure Route Server      │                 │
         │             │                         │                 │
         │             │ Managed BGP instances   │                 │
         │             │ ASN 65515 by default    │                 │
         │             └────────────┬────────────┘                 │
         └──────────────────────────┼──────────────────────────────┘
                                    │
                        Dynamic route injection
                                    │
                    ┌───────────────┴───────────────┐
                    │                               │
          ┌─────────▼──────────┐          ┌─────────▼──────────┐
          │ Application spoke │          │ Data spoke        │
          │ 10.10.0.0/16      │          │ 10.20.0.0/16     │
          │                    │          │                   │
          │ VM: 10.10.1.4     │          │ VM: 10.20.1.4    │
          └────────────────────┘          └───────────────────┘

Traffic flow

When the application VM sends traffic to 192.168.100.10, the following sequence occurs:

Application VM
      │
      │ Destination: 192.168.100.10
      ▼
Azure effective route table
      │
      │ BGP route: 192.168.100.0/24
      ▼
NVA 01 or NVA 02
      │
      │ Equal-cost route selection
      ▼
Loopback service IP
192.168.100.10

Azure Route Server does not forward the packets.

Its role is to exchange and distribute routing information. The actual data traffic travels directly between the workload and the selected NVA.

Address plan

Component

Address range

Hub VNet

10.0.0.0/16

Route Server subnet

10.0.0.0/26

NVA subnet

10.0.1.0/24

NVA 01

10.0.1.4

NVA 02

10.0.1.5

Application VNet

10.10.0.0/16

Application subnet

10.10.1.0/24

Application VM

10.10.1.4

Data VNet

10.20.0.0/16

Data subnet

10.20.1.0/24

Data VM

10.20.1.4

Emulated external prefix

192.168.100.0/24

Emulated service IP

192.168.100.10

All address spaces must remain non-overlapping.

Important design notes

The Route Server subnet must be named exactly:

RouteServerSubnet

This workshop uses:

10.0.0.0/26

Do not associate the following resources with RouteServerSubnet:

  • a network security group;

  • a user-defined route table;

  • a virtual machine;

  • a private endpoint;

  • another delegated service.

Each NVA must establish BGP sessions with both Route Server instance IP addresses.

Both NVAs advertise the same prefix with the same AS-path length. Azure can therefore install both next hops and use Equal-Cost Multi-Path routing.

FRRouting is used only to emulate a BGP-capable NVA. In production, use a supported network appliance appropriate for the operational and security requirements.

1. Verify the prerequisites

You need:

  • PowerShell 7;

  • Azure CLI;

  • an Azure subscription;

  • permission to create networking and compute resources;

  • quota for four small virtual machines.

Verify PowerShell:

pwsh --version

Verify Azure CLI:

az version

Confirm that both commands are available:

$RequiredCommands = @(
    "pwsh"
    "az"
)

foreach ($Command in $RequiredCommands) {
    $ResolvedCommand = Get-Command `
        -Name $Command `
        -ErrorAction SilentlyContinue

    if (-not $ResolvedCommand) {
        throw "Required command '$Command' was not found."
    }

    [pscustomobject]@{
        Command = $Command
        Path    = $ResolvedCommand.Source
    }
}

2. Sign in to Azure

Authenticate:

az login

List the available subscriptions:

az account list `
    --query "[].{Name:name,SubscriptionId:id,TenantId:tenantId,IsDefault:isDefault}" `
    --output table

Select the target subscription:

$SubscriptionId = "<your-subscription-id>"

az account set `
    --subscription $SubscriptionId

Confirm the active Azure context:

az account show `
    --query "{Subscription:name,SubscriptionId:id,TenantId:tenantId,User:user.name}" `
    --output table

Register the networking provider:

az provider register `
    --namespace Microsoft.Network `
    --wait

3. Define the workshop variables

Keep the same PowerShell session open throughout the workshop.

Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"

$SubscriptionId = (
    az account show `
        --query id `
        --output tsv
).Trim()

$TenantId = (
    az account show `
        --query tenantId `
        --output tsv
).Trim()

if ([string]::IsNullOrWhiteSpace($SubscriptionId)) {
    throw "No active Azure subscription was found."
}

$Location      = "westeurope"
$ResourceGroup = "rg-routeserver-deepdive-weu"

# Hub network

$HubVnetName             = "vnet-hub-routeserver-weu"
$HubVnetPrefix           = "10.0.0.0/16"
$RouteServerSubnetName   = "RouteServerSubnet"
$RouteServerSubnetPrefix = "10.0.0.0/26"
$NvaSubnetName           = "snet-nva"
$NvaSubnetPrefix         = "10.0.1.0/24"

# Route Server

$RouteServerName         = "rs-hub-weu"
$RouteServerPublicIpName = "pip-routeserver-weu"

# NVA pair

$NvaAsn = 65001

$Nva1Name      = "vm-nva-01"
$Nva1Nic       = "nic-nva-01"
$Nva1PublicIp  = "pip-nva-01"
$Nva1PrivateIp = "10.0.1.4"
$Nva1PeerName  = "peer-nva-01"

$Nva2Name      = "vm-nva-02"
$Nva2Nic       = "nic-nva-02"
$Nva2PublicIp  = "pip-nva-02"
$Nva2PrivateIp = "10.0.1.5"
$Nva2PeerName  = "peer-nva-02"

# Application spoke

$AppVnetName      = "vnet-spoke-app-weu"
$AppVnetPrefix    = "10.10.0.0/16"
$AppSubnetName    = "snet-app"
$AppSubnetPrefix  = "10.10.1.0/24"
$AppVmName        = "vm-spoke-app"
$AppVmNic         = "nic-spoke-app"
$AppVmPublicIp    = "pip-spoke-app"
$AppVmPrivateIp   = "10.10.1.4"

# Data spoke

$DataVnetName      = "vnet-spoke-data-weu"
$DataVnetPrefix    = "10.20.0.0/16"
$DataSubnetName    = "snet-data"
$DataSubnetPrefix  = "10.20.1.0/24"
$DataVmName        = "vm-spoke-data"
$DataVmNic         = "nic-spoke-data"
$DataVmPublicIp    = "pip-spoke-data"
$DataVmPrivateIp   = "10.20.1.4"

# VNet peerings

$HubToAppPeering  = "peer-hub-to-app"
$AppToHubPeering  = "peer-app-to-hub"
$HubToDataPeering = "peer-hub-to-data"
$DataToHubPeering = "peer-data-to-hub"

# Security

$NvaNsgName      = "nsg-routeserver-nvas"
$WorkloadNsgName = "nsg-routeserver-workloads"

# Emulated network behind the NVA pair

$EmulatedBranchPrefix    = "192.168.100.0/24"
$EmulatedBranchServiceIp = "192.168.100.10"

# Virtual machines

$VmSize          = "Standard_B1s"
$VmAdminUsername = "azureuser"

$UbuntuImage = "Canonical:0001-com-ubuntu-server-jammy:22_04-lts-gen2:latest"

az account set `
    --subscription $SubscriptionId

Review the configuration:

[pscustomobject]@{
    SubscriptionId       = $SubscriptionId
    TenantId             = $TenantId
    Location             = $Location
    ResourceGroup        = $ResourceGroup
    HubVnet              = $HubVnetName
    RouteServer          = $RouteServerName
    NvaAsn               = $NvaAsn
    AdvertisedPrefix     = $EmulatedBranchPrefix
    AdvertisedServiceIp  = $EmulatedBranchServiceIp
}

4. Check for an existing workshop deployment

$ResourceGroupExists = az group exists `
    --subscription $SubscriptionId `
    --name $ResourceGroup `
    --output tsv

if ($ResourceGroupExists -eq "true") {
    throw @"
Resource group '$ResourceGroup' already exists.

Delete the previous workshop or change the resource-group name.
"@
}

5. Create the resource group

az group create `
    --subscription $SubscriptionId `
    --name $ResourceGroup `
    --location $Location `
    --tags `
        workshop=route-server-deep-dive `
        environment=lab `
        managedBy=azure-cli `
    --only-show-errors `
    --output none

Verify it:

az group show `
    --subscription $SubscriptionId `
    --name $ResourceGroup `
    --query "{Name:name,Location:location,State:properties.provisioningState,Tags:tags}" `
    --output jsonc

6. Create the hub VNet

Create the hub with the dedicated Route Server subnet:

az network vnet create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $HubVnetName `
    --location $Location `
    --address-prefixes $HubVnetPrefix `
    --subnet-name $RouteServerSubnetName `
    --subnet-prefixes $RouteServerSubnetPrefix `
    --tags `
        workshop=route-server-deep-dive `
        role=routing-hub `
        managedBy=azure-cli `
    --only-show-errors `
    --output none

Create the NVA subnet:

az network vnet subnet create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --vnet-name $HubVnetName `
    --name $NvaSubnetName `
    --address-prefixes $NvaSubnetPrefix `
    --only-show-errors `
    --output none

Verify the hub subnets:

az network vnet subnet list `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --vnet-name $HubVnetName `
    --query "[].{Name:name,Prefix:addressPrefix,Nsg:networkSecurityGroup.id,RouteTable:routeTable.id}" `
    --output table

The RouteServerSubnet must show no NSG and no route table.

7. Create the spoke VNets

Create the application spoke:

az network vnet create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $AppVnetName `
    --location $Location `
    --address-prefixes $AppVnetPrefix `
    --subnet-name $AppSubnetName `
    --subnet-prefixes $AppSubnetPrefix `
    --tags `
        workshop=route-server-deep-dive `
        role=application-spoke `
        managedBy=azure-cli `
    --only-show-errors `
    --output none

Create the data spoke:

az network vnet create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $DataVnetName `
    --location $Location `
    --address-prefixes $DataVnetPrefix `
    --subnet-name $DataSubnetName `
    --subnet-prefixes $DataSubnetPrefix `
    --tags `
        workshop=route-server-deep-dive `
        role=data-spoke `
        managedBy=azure-cli `
    --only-show-errors `
    --output none

Display the VNet inventory:

az network vnet list `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --query "[].{Name:name,AddressSpace:addressSpace.addressPrefixes[0],SubnetCount:length(subnets),Role:tags.role}" `
    --output table

8. Create the network security groups

Create the NVA NSG:

az network nsg create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $NvaNsgName `
    --location $Location `
    --tags `
        workshop=route-server-deep-dive `
        role=nva-security `
    --only-show-errors `
    --output none

Allow BGP from the Route Server subnet:

az network nsg rule create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --nsg-name $NvaNsgName `
    --name Allow-RouteServer-BGP `
    --priority 100 `
    --direction Inbound `
    --access Allow `
    --protocol Tcp `
    --source-address-prefixes $RouteServerSubnetPrefix `
    --source-port-ranges "*" `
    --destination-address-prefixes "*" `
    --destination-port-ranges 179 `
    --description "Allow BGP from Azure Route Server." `
    --only-show-errors `
    --output none

Allow private workshop traffic to the NVAs:

az network nsg rule create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --nsg-name $NvaNsgName `
    --name Allow-Lab-Private-Traffic `
    --priority 110 `
    --direction Inbound `
    --access Allow `
    --protocol "*" `
    --source-address-prefixes `
        $HubVnetPrefix `
        $AppVnetPrefix `
        $DataVnetPrefix `
    --source-port-ranges "*" `
    --destination-address-prefixes "*" `
    --destination-port-ranges "*" `
    --description "Allow private workshop traffic to the NVAs." `
    --only-show-errors `
    --output none

Create the workload NSG:

az network nsg create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $WorkloadNsgName `
    --location $Location `
    --tags `
        workshop=route-server-deep-dive `
        role=workload-security `
    --only-show-errors `
    --output none

Allow traffic from the workshop networks:

az network nsg rule create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --nsg-name $WorkloadNsgName `
    --name Allow-Lab-Private-Traffic `
    --priority 100 `
    --direction Inbound `
    --access Allow `
    --protocol "*" `
    --source-address-prefixes `
        $HubVnetPrefix `
        $AppVnetPrefix `
        $DataVnetPrefix `
        $EmulatedBranchPrefix `
    --source-port-ranges "*" `
    --destination-address-prefixes "*" `
    --destination-port-ranges "*" `
    --description "Allow traffic from workshop private prefixes." `
    --only-show-errors `
    --output none

No inbound Internet rule is created.

9. Create the public IP addresses

The VM public IP addresses provide outbound connectivity for package installation and Azure VM Run Command.

The NSGs continue to block unsolicited inbound Internet traffic.

Define the public IP resources:

$PublicIpNames = @(
    $RouteServerPublicIpName
    $Nva1PublicIp
    $Nva2PublicIp
    $AppVmPublicIp
    $DataVmPublicIp
)

Create them:

foreach ($PublicIpName in $PublicIpNames) {
    Write-Host `
        "Creating public IP $PublicIpName..." `
        -ForegroundColor Cyan

    az network public-ip create `
        --subscription $SubscriptionId `
        --resource-group $ResourceGroup `
        --name $PublicIpName `
        --location $Location `
        --sku Standard `
        --tier Regional `
        --allocation-method Static `
        --version IPv4 `
        --tags `
            workshop=route-server-deep-dive `
            managedBy=azure-cli `
        --only-show-errors `
        --output none
}

10. Create the NVA network interfaces

Create NVA 01:

az network nic create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Nic `
    --location $Location `
    --vnet-name $HubVnetName `
    --subnet $NvaSubnetName `
    --private-ip-address $Nva1PrivateIp `
    --public-ip-address $Nva1PublicIp `
    --network-security-group $NvaNsgName `
    --ip-forwarding true `
    --tags `
        workshop=route-server-deep-dive `
        role=nva `
        instance=01 `
    --only-show-errors `
    --output none

Create NVA 02:

az network nic create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva2Nic `
    --location $Location `
    --vnet-name $HubVnetName `
    --subnet $NvaSubnetName `
    --private-ip-address $Nva2PrivateIp `
    --public-ip-address $Nva2PublicIp `
    --network-security-group $NvaNsgName `
    --ip-forwarding true `
    --tags `
        workshop=route-server-deep-dive `
        role=nva `
        instance=02 `
    --only-show-errors `
    --output none

Verify IP forwarding:

az network nic show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Nic `
    --query "{Name:name,PrivateIp:ipConfigurations[0].privateIPAddress,IpForwarding:enableIPForwarding}" `
    --output table

az network nic show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva2Nic `
    --query "{Name:name,PrivateIp:ipConfigurations[0].privateIPAddress,IpForwarding:enableIPForwarding}" `
    --output table

11. Create the workload network interfaces

Create the application VM NIC:

az network nic create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $AppVmNic `
    --location $Location `
    --vnet-name $AppVnetName `
    --subnet $AppSubnetName `
    --private-ip-address $AppVmPrivateIp `
    --public-ip-address $AppVmPublicIp `
    --network-security-group $WorkloadNsgName `
    --tags `
        workshop=route-server-deep-dive `
        role=application-workload `
    --only-show-errors `
    --output none

Create the data VM NIC:

az network nic create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $DataVmNic `
    --location $Location `
    --vnet-name $DataVnetName `
    --subnet $DataSubnetName `
    --private-ip-address $DataVmPrivateIp `
    --public-ip-address $DataVmPublicIp `
    --network-security-group $WorkloadNsgName `
    --tags `
        workshop=route-server-deep-dive `
        role=data-workload `
    --only-show-errors `
    --output none

12. Deploy the virtual machines

Define the VM inventory:

$VmDefinitions = @(
    [pscustomobject]@{
        Name = $Nva1Name
        Nic  = $Nva1Nic
        Role = "nva"
    }

    [pscustomobject]@{
        Name = $Nva2Name
        Nic  = $Nva2Nic
        Role = "nva"
    }

    [pscustomobject]@{
        Name = $AppVmName
        Nic  = $AppVmNic
        Role = "application-workload"
    }

    [pscustomobject]@{
        Name = $DataVmName
        Nic  = $DataVmNic
        Role = "data-workload"
    }
)

Create the VMs:

foreach ($Vm in $VmDefinitions) {
    Write-Host `
        "Creating $($Vm.Name)..." `
        -ForegroundColor Cyan

    az vm create `
        --subscription $SubscriptionId `
        --resource-group $ResourceGroup `
        --name $Vm.Name `
        --location $Location `
        --nics $Vm.Nic `
        --image $UbuntuImage `
        --size $VmSize `
        --admin-username $VmAdminUsername `
        --authentication-type ssh `
        --generate-ssh-keys `
        --security-type Standard `
        --os-disk-sku Standard_LRS `
        --tags `
            workshop=route-server-deep-dive `
            "role=$($Vm.Role)" `
        --no-wait `
        --only-show-errors `
        --output none
}

Wait for all four VMs:

foreach ($Vm in $VmDefinitions) {
    Write-Host `
        "Waiting for $($Vm.Name)..." `
        -ForegroundColor Yellow

    az vm wait `
        --subscription $SubscriptionId `
        --resource-group $ResourceGroup `
        --name $Vm.Name `
        --created `
        --interval 15 `
        --timeout 1800
}

Display their status:

az vm list `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --show-details `
    --query "[].{Name:name,PowerState:powerState,PrivateIps:privateIps,PublicIps:publicIps,Size:hardwareProfile.vmSize}" `
    --output table

13. Deploy Azure Route Server

Resolve the Route Server subnet ID:

$RouteServerSubnetId = (
    az network vnet subnet show `
        --subscription $SubscriptionId `
        --resource-group $ResourceGroup `
        --vnet-name $HubVnetName `
        --name $RouteServerSubnetName `
        --query id `
        --output tsv
).Trim()

Create Azure Route Server:

az network routeserver create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $RouteServerName `
    --location $Location `
    --hosted-subnet $RouteServerSubnetId `
    --public-ip-address $RouteServerPublicIpName `
    --tags `
        workshop=route-server-deep-dive `
        role=dynamic-routing `
        managedBy=azure-cli `
    --only-show-errors `
    --output none

Wait for the provisioning state:

az network routeserver wait `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $RouteServerName `
    --created `
    --interval 30 `
    --timeout 3600

Wait for the routing state:

az network routeserver wait `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $RouteServerName `
    --custom "routingState=='Provisioned'" `
    --interval 30 `
    --timeout 3600

Display the Route Server:

az network routeserver show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $RouteServerName `
    --query "{Name:name,ProvisioningState:provisioningState,RoutingState:routingState,ASN:virtualRouterAsn,RouterIPs:virtualRouterIps,RoutingPreference:hubRoutingPreference}" `
    --output jsonc

14. Configure hub-and-spoke peering

Route injection into a peered spoke requires two complementary settings:

Hub → Spoke:
Allow gateway or Route Server transit

Spoke → Hub:
Use the remote gateway or Route Server

Resolve the VNet IDs:

$HubVnetId = (
    az network vnet show `
        --subscription $SubscriptionId `
        --resource-group $ResourceGroup `
        --name $HubVnetName `
        --query id `
        --output tsv
).Trim()

$AppVnetId = (
    az network vnet show `
        --subscription $SubscriptionId `
        --resource-group $ResourceGroup `
        --name $AppVnetName `
        --query id `
        --output tsv
).Trim()

$DataVnetId = (
    az network vnet show `
        --subscription $SubscriptionId `
        --resource-group $ResourceGroup `
        --name $DataVnetName `
        --query id `
        --output tsv
).Trim()

Create the hub-to-application peering:

az network vnet peering create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $HubToAppPeering `
    --vnet-name $HubVnetName `
    --remote-vnet $AppVnetId `
    --allow-vnet-access true `
    --allow-forwarded-traffic true `
    --allow-gateway-transit true `
    --only-show-errors `
    --output none

Create the application-to-hub peering:

az network vnet peering create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $AppToHubPeering `
    --vnet-name $AppVnetName `
    --remote-vnet $HubVnetId `
    --allow-vnet-access true `
    --allow-forwarded-traffic true `
    --use-remote-gateways true `
    --only-show-errors `
    --output none

Create the hub-to-data peering:

az network vnet peering create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $HubToDataPeering `
    --vnet-name $HubVnetName `
    --remote-vnet $DataVnetId `
    --allow-vnet-access true `
    --allow-forwarded-traffic true `
    --allow-gateway-transit true `
    --only-show-errors `
    --output none

Create the data-to-hub peering:

az network vnet peering create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $DataToHubPeering `
    --vnet-name $DataVnetName `
    --remote-vnet $HubVnetId `
    --allow-vnet-access true `
    --allow-forwarded-traffic true `
    --use-remote-gateways true `
    --only-show-errors `
    --output none

Wait for every peering to report Connected:

$PeeringDefinitions = @(
    [pscustomobject]@{
        Vnet = $HubVnetName
        Name = $HubToAppPeering
    }

    [pscustomobject]@{
        Vnet = $AppVnetName
        Name = $AppToHubPeering
    }

    [pscustomobject]@{
        Vnet = $HubVnetName
        Name = $HubToDataPeering
    }

    [pscustomobject]@{
        Vnet = $DataVnetName
        Name = $DataToHubPeering
    }
)

foreach ($Peering in $PeeringDefinitions) {
    az network vnet peering wait `
        --subscription $SubscriptionId `
        --resource-group $ResourceGroup `
        --vnet-name $Peering.Vnet `
        --name $Peering.Name `
        --custom "peeringState=='Connected'" `
        --interval 15 `
        --timeout 600
}

Verify the hub peerings:

az network vnet peering list `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --vnet-name $HubVnetName `
    --query "[].{Name:name,State:peeringState,GatewayTransit:allowGatewayTransit,ForwardedTraffic:allowForwardedTraffic,RemoteVnet:remoteVirtualNetwork.id}" `
    --output table

Verify the spoke peerings:

az network vnet peering show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --vnet-name $AppVnetName `
    --name $AppToHubPeering `
    --query "{Name:name,State:peeringState,UseRemoteGateways:useRemoteGateways,ForwardedTraffic:allowForwardedTraffic}" `
    --output table

az network vnet peering show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --vnet-name $DataVnetName `
    --name $DataToHubPeering `
    --query "{Name:name,State:peeringState,UseRemoteGateways:useRemoteGateways,ForwardedTraffic:allowForwardedTraffic}" `
    --output table

15. Retrieve the Route Server BGP information

$RouteServerDetails = az network routeserver show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $RouteServerName `
    --output json |
    ConvertFrom-Json

$RouteServerAsn = [int]$RouteServerDetails.virtualRouterAsn

$RouteServerIps = @(
    $RouteServerDetails.virtualRouterIps
)

if ($RouteServerIps.Count -lt 2) {
    throw "Azure Route Server did not return two router IP addresses."
}

$RouteServerIp1 = [string]$RouteServerIps[0]
$RouteServerIp2 = [string]$RouteServerIps[1]

Display the BGP information:

[pscustomobject]@{
    RouteServer = $RouteServerName
    ASN         = $RouteServerAsn
    RouterIP1   = $RouteServerIp1
    RouterIP2   = $RouteServerIp2
}

A typical result resembles:

RouteServer    rs-hub-weu
ASN            65515
RouterIP1      10.0.0.4
RouterIP2      10.0.0.5

Use the values returned by your own deployment rather than hard-coding them.

16. Create the Route Server BGP peers

Create the NVA 01 peer:

az network routeserver peering create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --routeserver $RouteServerName `
    --name $Nva1PeerName `
    --peer-ip $Nva1PrivateIp `
    --peer-asn $NvaAsn `
    --no-wait true `
    --only-show-errors `
    --output none

Create the NVA 02 peer:

az network routeserver peering create `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --routeserver $RouteServerName `
    --name $Nva2PeerName `
    --peer-ip $Nva2PrivateIp `
    --peer-asn $NvaAsn `
    --no-wait true `
    --only-show-errors `
    --output none

Wait for the peer resources:

az network routeserver peering wait `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --routeserver $RouteServerName `
    --name $Nva1PeerName `
    --created `
    --interval 15 `
    --timeout 600

az network routeserver peering wait `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --routeserver $RouteServerName `
    --name $Nva2PeerName `
    --created `
    --interval 15 `
    --timeout 600

Display the peer configuration:

az network routeserver peering list `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --routeserver $RouteServerName `
    --query "[].{Name:name,PeerIP:peerIp,PeerASN:peerAsn,ProvisioningState:provisioningState}" `
    --output table

The provisioning state confirms that the Azure resource exists.

It does not yet confirm that the BGP session is established.

17. Configure FRRouting on both NVAs

The following function:

  • installs FRRouting;

  • enables Linux IP forwarding;

  • disables reverse-path filtering for the lab;

  • creates a persistent loopback service prefix;

  • configures both Route Server neighbors;

  • advertises 192.168.100.0/24;

  • enables multiple BGP paths;

  • restarts FRRouting.

function Set-FrrNvaConfiguration {
    param(
        [Parameter(Mandatory)]
        [string]$VmName,

        [Parameter(Mandatory)]
        [string]$Hostname,

        [Parameter(Mandatory)]
        [string]$NvaPrivateIp
    )

    $FrrConfiguration = @"
#!/usr/bin/env bash
set -euo pipefail

export DEBIAN_FRONTEND=noninteractive

apt-get update
apt-get install -y frr frr-pythontools

sed -i 's/^bgpd=no/bgpd=yes/' /etc/frr/daemons
sed -i 's/^zebra=no/zebra=yes/' /etc/frr/daemons

cat >/etc/sysctl.d/99-route-server-lab.conf <<'SYSCTL'
net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0
SYSCTL

sysctl --system >/dev/null

cat >/usr/local/sbin/configure-route-server-loopback.sh <<'LOOPBACK'
#!/usr/bin/env bash
set -e

if ! ip -4 address show dev lo | grep -q '$EmulatedBranchServiceIp/24'; then
    ip address add $EmulatedBranchServiceIp/24 dev lo
fi
LOOPBACK

chmod 0755 /usr/local/sbin/configure-route-server-loopback.sh

cat >/etc/systemd/system/route-server-loopback.service <<'SERVICE'
[Unit]
Description=Configure the Azure Route Server lab loopback
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/configure-route-server-loopback.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
SERVICE

systemctl daemon-reload
systemctl enable --now route-server-loopback.service

cat >/etc/frr/frr.conf <<'FRR'
frr defaults traditional
hostname $Hostname
log syslog informational
service integrated-vtysh-config
!
router bgp $NvaAsn
 bgp router-id $NvaPrivateIp
 no bgp ebgp-requires-policy
 neighbor $RouteServerIp1 remote-as $RouteServerAsn
 neighbor $RouteServerIp1 ebgp-multihop 2
 neighbor $RouteServerIp1 update-source $NvaPrivateIp
 neighbor $RouteServerIp2 remote-as $RouteServerAsn
 neighbor $RouteServerIp2 ebgp-multihop 2
 neighbor $RouteServerIp2 update-source $NvaPrivateIp
 !
 address-family ipv4 unicast
  network $EmulatedBranchPrefix
  neighbor $RouteServerIp1 activate
  neighbor $RouteServerIp2 activate
  maximum-paths 2
 exit-address-family
!
line vty
FRR

chown frr:frr /etc/frr/frr.conf
chmod 0640 /etc/frr/frr.conf

systemctl enable frr
systemctl restart frr

sleep 10

echo
echo "=== Loopback address ==="
ip -4 address show dev lo

echo
echo "=== IPv4 forwarding ==="
sysctl net.ipv4.ip_forward

echo
echo "=== FRR service ==="
systemctl is-active frr

echo
echo "=== BGP summary ==="
vtysh -c 'show ip bgp summary'
"@

    az vm run-command invoke `
        --subscription $SubscriptionId `
        --resource-group $ResourceGroup `
        --name $VmName `
        --command-id RunShellScript `
        --scripts $FrrConfiguration `
        --query "value[0].message" `
        --output tsv
}

Configure NVA 01:

Set-FrrNvaConfiguration `
    -VmName $Nva1Name `
    -Hostname "nva-01" `
    -NvaPrivateIp $Nva1PrivateIp

Configure NVA 02:

Set-FrrNvaConfiguration `
    -VmName $Nva2Name `
    -Hostname "nva-02" `
    -NvaPrivateIp $Nva2PrivateIp

18. Wait for BGP route exchange

Create a function that checks whether Route Server learned the emulated prefix from a specific peer:

function Test-RouteServerLearnedPrefix {
    param(
        [Parameter(Mandatory)]
        [string]$PeerName,

        [Parameter(Mandatory)]
        [string]$Prefix
    )

    $RouteOutput = az network routeserver peering list-learned-routes `
        --subscription $SubscriptionId `
        --resource-group $ResourceGroup `
        --routeserver $RouteServerName `
        --name $PeerName `
        --output json `
        2>$null

    if ($LASTEXITCODE -ne 0) {
        return $false
    }

    $RouteText = $RouteOutput -join "`n"

    return $RouteText -match [regex]::Escape($Prefix)
}

Wait until both NVAs advertise the route:

$BgpDeadline = (Get-Date).AddMinutes(20)

do {
    $Nva1RouteReady = Test-RouteServerLearnedPrefix `
        -PeerName $Nva1PeerName `
        -Prefix $EmulatedBranchPrefix

    $Nva2RouteReady = Test-RouteServerLearnedPrefix `
        -PeerName $Nva2PeerName `
        -Prefix $EmulatedBranchPrefix

    Write-Host (
        "NVA 01 route: {0} | NVA 02 route: {1}" -f `
            $Nva1RouteReady,
            $Nva2RouteReady
    )

    if (-not ($Nva1RouteReady -and $Nva2RouteReady)) {
        Start-Sleep -Seconds 30
    }
}
until (
    (
        $Nva1RouteReady -and
        $Nva2RouteReady
    ) -or
    (Get-Date) -ge $BgpDeadline
)

if (-not ($Nva1RouteReady -and $Nva2RouteReady)) {
    throw @"
Azure Route Server did not learn $EmulatedBranchPrefix from both NVAs.

Continue with the troubleshooting section.
"@
}

Write-Host `
    "Route Server learned the prefix from both NVAs." `
    -ForegroundColor Green

19. Inspect the FRRouting BGP sessions

Check NVA 01:

az vm run-command invoke `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Name `
    --command-id RunShellScript `
    --scripts "sudo vtysh -c 'show ip bgp summary'" `
    --query "value[0].message" `
    --output tsv

Check NVA 02:

az vm run-command invoke `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva2Name `
    --command-id RunShellScript `
    --scripts "sudo vtysh -c 'show ip bgp summary'" `
    --query "value[0].message" `
    --output tsv

Each NVA should show two established BGP neighbors:

Route Server instance 1
Route Server instance 2

The received-prefix count should be greater than zero.

20. Inspect the routes learned by Route Server

Display the routes learned from NVA 01:

az network routeserver peering list-learned-routes `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --routeserver $RouteServerName `
    --name $Nva1PeerName `
    --output table

Display the routes learned from NVA 02:

az network routeserver peering list-learned-routes `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --routeserver $RouteServerName `
    --name $Nva2PeerName `
    --output table

Both peer outputs should contain:

192.168.100.0/24

The next hop should correspond to the private IP address of the relevant NVA.

21. Inspect the routes advertised to the NVAs

Display routes advertised to NVA 01:

az network routeserver peering list-advertised-routes `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --routeserver $RouteServerName `
    --name $Nva1PeerName `
    --output table

Display routes advertised to NVA 02:

az network routeserver peering list-advertised-routes `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --routeserver $RouteServerName `
    --name $Nva2PeerName `
    --output table

The advertised routes should include the Azure network address spaces:

10.0.0.0/16
10.10.0.0/16
10.20.0.0/16

22. Inspect the routes inside the NVAs

Display the BGP table on NVA 01:

az vm run-command invoke `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Name `
    --command-id RunShellScript `
    --scripts @"
sudo vtysh -c 'show ip bgp'
echo
sudo vtysh -c 'show ip route bgp'
"@ `
    --query "value[0].message" `
    --output tsv

Display the BGP table on NVA 02:

az vm run-command invoke `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva2Name `
    --command-id RunShellScript `
    --scripts @"
sudo vtysh -c 'show ip bgp'
echo
sudo vtysh -c 'show ip route bgp'
"@ `
    --query "value[0].message" `
    --output tsv

This verifies bidirectional route exchange:

NVA → Route Server:
192.168.100.0/24

Route Server → NVA:
Hub and spoke VNet address spaces

23. Validate route injection into the application spoke

Display the effective routes on the application NIC:

az network nic show-effective-route-table `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $AppVmNic `
    --query "[?addressPrefix=='$EmulatedBranchPrefix']" `
    --output jsonc

The output should contain the dynamically learned route:

192.168.100.0/24

If the filtered output is empty, display the complete table:

az network nic show-effective-route-table `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $AppVmNic `
    --output table

24. Validate route injection into the data spoke

az network nic show-effective-route-table `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $DataVmNic `
    --query "[?addressPrefix=='$EmulatedBranchPrefix']" `
    --output jsonc

If required, display all effective routes:

az network nic show-effective-route-table `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $DataVmNic `
    --output table

Both spoke NICs should receive the BGP route without a manually maintained UDR.

25. Test the data plane

Create a reusable connectivity function:

function Test-EmulatedBranchService {
    param(
        [Parameter(Mandatory)]
        [string]$SourceVm
    )

    Write-Host (
        "Testing {0} -> {1}" -f `
            $SourceVm,
            $EmulatedBranchServiceIp
    ) -ForegroundColor Cyan

    $Result = az vm run-command invoke `
        --subscription $SubscriptionId `
        --resource-group $ResourceGroup `
        --name $SourceVm `
        --command-id RunShellScript `
        --scripts @"
set -x
ping -c 5 -W 3 $EmulatedBranchServiceIp
"@ `
        --query "value[0].message" `
        --output tsv

    Write-Host $Result

    if ($Result -notmatch "0% packet loss") {
        throw @"
Connectivity test failed:
$SourceVm -> $EmulatedBranchServiceIp
"@
    }

    Write-Host `
        "Connectivity test passed." `
        -ForegroundColor Green
}

Test from the application spoke:

Test-EmulatedBranchService `
    -SourceVm $AppVmName

Test from the data spoke:

Test-EmulatedBranchService `
    -SourceVm $DataVmName

A successful result demonstrates:

Spoke VM
   ↓
Route injected by Azure Route Server
   ↓
One of the two NVAs
   ↓
192.168.100.10

26. Confirm the anycast service on both NVAs

Check NVA 01:

az vm run-command invoke `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Name `
    --command-id RunShellScript `
    --scripts @"
ip -4 address show dev lo
ping -c 2 -W 2 $EmulatedBranchServiceIp
"@ `
    --query "value[0].message" `
    --output tsv

Check NVA 02:

az vm run-command invoke `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva2Name `
    --command-id RunShellScript `
    --scripts @"
ip -4 address show dev lo
ping -c 2 -W 2 $EmulatedBranchServiceIp
"@ `
    --query "value[0].message" `
    --output tsv

Both NVAs host the same loopback service IP and advertise the same prefix.

This provides a stateless anycast-style endpoint for the failover exercise.

27. Inspect the baseline before failover

Record the learned route from both peers:

[pscustomobject]@{
    Peer             = $Nva1PeerName
    AdvertisesPrefix = Test-RouteServerLearnedPrefix `
        -PeerName $Nva1PeerName `
        -Prefix $EmulatedBranchPrefix
}

[pscustomobject]@{
    Peer             = $Nva2PeerName
    AdvertisesPrefix = Test-RouteServerLearnedPrefix `
        -PeerName $Nva2PeerName `
        -Prefix $EmulatedBranchPrefix
}

Both values should be:

True

Run one final baseline test:

Test-EmulatedBranchService `
    -SourceVm $AppVmName

28. Simulate an NVA failure

Deallocate NVA 01:

az vm deallocate `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Name `
    --no-wait `
    --only-show-errors

Create a helper function for VM power-state checks:

function Wait-VmPowerState {
    param(
        [Parameter(Mandatory)]
        [string]$VmName,

        [Parameter(Mandatory)]
        [string]$DesiredState,

        [int]$TimeoutMinutes = 15
    )

    $Deadline = (Get-Date).AddMinutes($TimeoutMinutes)

    do {
        $PowerState = (
            az vm get-instance-view `
                --subscription $SubscriptionId `
                --resource-group $ResourceGroup `
                --name $VmName `
                --query "instanceView.statuses[?starts_with(code, 'PowerState/')].code | [0]" `
                --output tsv
        ).Trim()

        Write-Host `
            "$VmName power state: $PowerState"

        if ($PowerState -ne $DesiredState) {
            Start-Sleep -Seconds 15
        }
    }
    until (
        $PowerState -eq $DesiredState -or
        (Get-Date) -ge $Deadline
    )

    if ($PowerState -ne $DesiredState) {
        throw @"
VM '$VmName' did not reach state '$DesiredState'.
"@
    }
}

Wait for deallocation:

Wait-VmPowerState `
    -VmName $Nva1Name `
    -DesiredState "PowerState/deallocated"

29. Wait for BGP route withdrawal

Wait until the route from NVA 01 disappears while the route from NVA 02 remains:

$WithdrawalDeadline = (Get-Date).AddMinutes(10)

do {
    $Nva1StillAdvertises = Test-RouteServerLearnedPrefix `
        -PeerName $Nva1PeerName `
        -Prefix $EmulatedBranchPrefix

    $Nva2StillAdvertises = Test-RouteServerLearnedPrefix `
        -PeerName $Nva2PeerName `
        -Prefix $EmulatedBranchPrefix

    Write-Host (
        "NVA 01 route: {0} | NVA 02 route: {1}" -f `
            $Nva1StillAdvertises,
            $Nva2StillAdvertises
    )

    if (
        $Nva1StillAdvertises -or
        -not $Nva2StillAdvertises
    ) {
        Start-Sleep -Seconds 30
    }
}
until (
    (
        -not $Nva1StillAdvertises -and
        $Nva2StillAdvertises
    ) -or
    (Get-Date) -ge $WithdrawalDeadline
)

if (
    $Nva1StillAdvertises -or
    -not $Nva2StillAdvertises
) {
    throw "The expected BGP failover state was not reached."
}

Write-Host `
    "Route withdrawal completed. NVA 02 remains active." `
    -ForegroundColor Green

30. Test connectivity during the failure

Test from the application spoke:

Test-EmulatedBranchService `
    -SourceVm $AppVmName

Test from the data spoke:

Test-EmulatedBranchService `
    -SourceVm $DataVmName

The tests should remain successful because NVA 02 continues to advertise:

192.168.100.0/24

Inspect the application NIC route again:

az network nic show-effective-route-table `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $AppVmNic `
    --query "[?addressPrefix=='$EmulatedBranchPrefix']" `
    --output jsonc

The route remains available, but only the surviving next hop should remain after convergence.

31. Recover NVA 01

Start the failed appliance:

az vm start `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Name `
    --no-wait `
    --only-show-errors

Wait for the VM:

Wait-VmPowerState `
    -VmName $Nva1Name `
    -DesiredState "PowerState/running"

The loopback service and FRRouting configuration are persistent and start automatically.

Wait for the route to return:

$RecoveryDeadline = (Get-Date).AddMinutes(15)

do {
    $Nva1RouteRecovered = Test-RouteServerLearnedPrefix `
        -PeerName $Nva1PeerName `
        -Prefix $EmulatedBranchPrefix

    $Nva2RouteAvailable = Test-RouteServerLearnedPrefix `
        -PeerName $Nva2PeerName `
        -Prefix $EmulatedBranchPrefix

    Write-Host (
        "NVA 01 route: {0} | NVA 02 route: {1}" -f `
            $Nva1RouteRecovered,
            $Nva2RouteAvailable
    )

    if (-not ($Nva1RouteRecovered -and $Nva2RouteAvailable)) {
        Start-Sleep -Seconds 30
    }
}
until (
    (
        $Nva1RouteRecovered -and
        $Nva2RouteAvailable
    ) -or
    (Get-Date) -ge $RecoveryDeadline
)

if (-not ($Nva1RouteRecovered -and $Nva2RouteAvailable)) {
    throw "The NVA pair did not return to the redundant routing state."
}

Write-Host `
    "Both NVA routes are available again." `
    -ForegroundColor Green

Run the final test:

Test-EmulatedBranchService `
    -SourceVm $AppVmName

32. Optional route withdrawal exercise

Instead of stopping the complete VM, you can stop only FRRouting.

Stop BGP on NVA 02:

az vm run-command invoke `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva2Name `
    --command-id RunShellScript `
    --scripts "sudo systemctl stop frr" `
    --query "value[0].message" `
    --output tsv

Monitor the routes:

do {
    $Nva2RouteAvailable = Test-RouteServerLearnedPrefix `
        -PeerName $Nva2PeerName `
        -Prefix $EmulatedBranchPrefix

    Write-Host `
        "NVA 02 route available: $Nva2RouteAvailable"

    if ($Nva2RouteAvailable) {
        Start-Sleep -Seconds 30
    }
}
until (-not $Nva2RouteAvailable)

Start FRRouting again:

az vm run-command invoke `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva2Name `
    --command-id RunShellScript `
    --scripts "sudo systemctl start frr" `
    --query "value[0].message" `
    --output tsv

Wait for route recovery:

do {
    $Nva2RouteAvailable = Test-RouteServerLearnedPrefix `
        -PeerName $Nva2PeerName `
        -Prefix $EmulatedBranchPrefix

    Write-Host `
        "NVA 02 route available: $Nva2RouteAvailable"

    if (-not $Nva2RouteAvailable) {
        Start-Sleep -Seconds 30
    }
}
until ($Nva2RouteAvailable)

33. Troubleshooting

Route Server creation fails

Verify the subnet name and prefix:

az network vnet subnet show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --vnet-name $HubVnetName `
    --name $RouteServerSubnetName `
    --query "{Name:name,Prefix:addressPrefix,Nsg:networkSecurityGroup.id,RouteTable:routeTable.id}" `
    --output jsonc

Required values:

Name          RouteServerSubnet
Prefix        /26 or larger
NSG           Empty
Route table   Empty

Verify the Route Server public IP:

az network public-ip show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $RouteServerPublicIpName `
    --query "{Name:name,Sku:sku.name,Allocation:publicIPAllocationMethod,State:provisioningState}" `
    --output table

Expected values:

Sku           Standard
Allocation    Static
State         Succeeded

Route Server remains in a provisioning state

az network routeserver show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $RouteServerName `
    --query "{ProvisioningState:provisioningState,RoutingState:routingState,RouterIPs:virtualRouterIps,ASN:virtualRouterAsn}" `
    --output jsonc

Do not continue until these values are ready:

ProvisioningState    Succeeded
RoutingState         Provisioned

The peer resource exists but BGP is not established

Check the Azure peer configuration:

az network routeserver peering list `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --routeserver $RouteServerName `
    --query "[].{Name:name,PeerIP:peerIp,PeerASN:peerAsn,State:provisioningState}" `
    --output table

Check FRRouting:

az vm run-command invoke `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Name `
    --command-id RunShellScript `
    --scripts @"
sudo systemctl status frr --no-pager
echo
sudo vtysh -c 'show ip bgp summary'
echo
sudo ss -lntp | grep ':179' || true
"@ `
    --query "value[0].message" `
    --output tsv

Check the NVA NIC:

az network nic show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Nic `
    --query "{PrivateIp:ipConfigurations[0].privateIPAddress,IpForwarding:enableIPForwarding,Nsg:networkSecurityGroup.id}" `
    --output jsonc

Common causes include:

  • TCP port 179 blocked by an NSG;

  • the wrong NVA private IP configured as the Azure peer;

  • an incorrect NVA ASN;

  • a Route Server IP entered incorrectly;

  • missing ebgp-multihop;

  • FRRouting not running;

  • the NVA peered with only one Route Server instance;

  • the Route Server routing state not yet provisioned.

Route Server learns no NVA prefix

Check the loopback address:

az vm run-command invoke `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Name `
    --command-id RunShellScript `
    --scripts @"
ip -4 address show dev lo
ip -4 route show $EmulatedBranchPrefix
sudo vtysh -c 'show running-config'
sudo vtysh -c 'show ip bgp'
"@ `
    --query "value[0].message" `
    --output tsv

The NVA must contain an exact route for the BGP network statement.

Expected route:

192.168.100.0/24

Expected local address:

192.168.100.10/24

The spokes do not receive the route

Check the hub-side peering:

az network vnet peering show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --vnet-name $HubVnetName `
    --name $HubToAppPeering `
    --query "{State:peeringState,GatewayTransit:allowGatewayTransit,ForwardedTraffic:allowForwardedTraffic}" `
    --output table

Check the spoke-side peering:

az network vnet peering show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --vnet-name $AppVnetName `
    --name $AppToHubPeering `
    --query "{State:peeringState,UseRemoteGateways:useRemoteGateways,ForwardedTraffic:allowForwardedTraffic}" `
    --output table

Required settings:

Hub → Spoke
GatewayTransit       true
ForwardedTraffic     true

Spoke → Hub
UseRemoteGateways    true
ForwardedTraffic     true

Check for a spoke route table that disables BGP route propagation:

az network vnet subnet show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --vnet-name $AppVnetName `
    --name $AppSubnetName `
    --query "{RouteTable:routeTable.id}" `
    --output jsonc

If a route table is associated, inspect it:

az network route-table show `
    --subscription $SubscriptionId `
    --ids "<route-table-resource-id>" `
    --query "{Name:name,DisableBgpPropagation:disableBgpRoutePropagation}" `
    --output jsonc

BGP route propagation must not be disabled for this workshop.

The route exists but ping fails

Verify Linux IP forwarding:

az vm run-command invoke `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Name `
    --command-id RunShellScript `
    --scripts @"
sysctl net.ipv4.ip_forward
sysctl net.ipv4.conf.all.rp_filter
ip -4 address show dev lo
"@ `
    --query "value[0].message" `
    --output tsv

Expected values:

net.ipv4.ip_forward = 1
net.ipv4.conf.all.rp_filter = 0

Verify the NVA NIC IP-forwarding property:

az network nic show `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Nic `
    --query "enableIPForwarding" `
    --output tsv

Expected result:

true

Inspect the workload NSG:

az network nsg rule list `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --nsg-name $WorkloadNsgName `
    --query "[].{Name:name,Priority:priority,Access:access,Protocol:protocol,Source:sourceAddressPrefixes,Destination:destinationAddressPrefixes}" `
    --output jsonc

Failover does not occur

Confirm that the surviving peer still advertises the prefix:

az network routeserver peering list-learned-routes `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --routeserver $RouteServerName `
    --name $Nva2PeerName `
    --output table

Confirm the failed NVA state:

az vm get-instance-view `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --name $Nva1Name `
    --query "instanceView.statuses[?starts_with(code, 'PowerState/')].{Code:code,Status:displayStatus}" `
    --output table

BGP withdrawal is not necessarily immediate. Allow time for the failed session to be removed from the routing table before evaluating the final failover state.

34. Production considerations

The lab uses a stateless anycast service to make active-active routing easy to observe.

Production NVAs may require additional design for:

  • session symmetry;

  • source NAT;

  • destination NAT;

  • firewall state synchronization;

  • internal load balancers;

  • health probes;

  • active-passive routing;

  • AS-path prepending;

  • BGP communities;

  • route filtering;

  • overlapping prefix prevention;

  • maintenance procedures;

  • application-specific failover timing;

  • logging and monitoring;

  • appliance licensing;

  • throughput sizing.

For stateful firewalls, active-active ECMP can create asymmetric traffic paths.

Common alternatives include:

Active-active
Both NVAs advertise equal paths
Requires symmetry-aware design

Active-passive
Primary advertises the shortest path
Secondary advertises a longer AS path

Load-balanced next hop
Both NVAs advertise a shared internal load-balancer frontend IP

Azure Route Server remains a control-plane service in all three designs.

35. Cleanup

Review all resources in the workshop resource group:

az resource list `
    --subscription $SubscriptionId `
    --resource-group $ResourceGroup `
    --query "[].{Name:name,Type:type,Location:location}" `
    --output table

Require an explicit confirmation:

$DeleteConfirmation = Read-Host @"
Type the resource-group name to delete the workshop:
$ResourceGroup
"@

if ($DeleteConfirmation -ne $ResourceGroup) {
    throw "Deletion cancelled."
}

Delete the complete workshop:

az group delete `
    --subscription $SubscriptionId `
    --name $ResourceGroup `
    --yes `
    --no-wait `
    --only-show-errors

Monitor the deletion:

$DeletionDeadline = (Get-Date).AddHours(2)

do {
    $StillExists = az group exists `
        --subscription $SubscriptionId `
        --name $ResourceGroup `
        --output tsv

    Write-Host `
        "Resource group exists: $StillExists"

    if ($StillExists -eq "true") {
        Start-Sleep -Seconds 60
    }
}
until (
    $StillExists -eq "false" -or
    (Get-Date) -ge $DeletionDeadline
)

if ($StillExists -eq "true") {
    Write-Warning @"
Resource-group deletion is still running.

Azure Route Server deletion can take an extended period.
"@
}
else {
    Write-Host `
        "Workshop resources were deleted." `
        -ForegroundColor Green
}

Summary

This workshop created the following environment entirely from PowerShell and Azure CLI:

Azure Route Server
        │
        ├── BGP peer: NVA 01
        │       ├── Route Server instance 1
        │       └── Route Server instance 2
        │
        ├── BGP peer: NVA 02
        │       ├── Route Server instance 1
        │       └── Route Server instance 2
        │
        ├── Application spoke
        │
        └── Data spoke

Both NVAs advertised:

192.168.100.0/24

Both NVAs hosted:

192.168.100.10

Azure Route Server injected the route into the two spoke networks without requiring a manually maintained UDR.

The complete workflow was:

Create the hub and spokes
          ↓
Deploy two Linux NVAs
          ↓
Deploy Azure Route Server
          ↓
Enable Route Server transit on VNet peerings
          ↓
Configure Azure BGP peers
          ↓
Install and configure FRRouting
          ↓
Advertise the emulated external prefix
          ↓
Inspect learned and advertised routes
          ↓
Validate route injection in both spokes
          ↓
Test the routed data path
          ↓
Deallocate one NVA
          ↓
Observe route withdrawal and failover
          ↓
Recover the NVA and restore ECMP
          ↓
Delete the workshop

The central design principles are:

  1. Deploy Route Server in a dedicated /26 or larger RouteServerSubnet.

  2. Do not attach an NSG or UDR to that subnet.

  3. Peer every NVA with both Route Server instance IPs.

  4. Use an NVA ASN that differs from the Route Server ASN.

  5. Enable gateway transit on the hub-side peering.

  6. Enable remote gateway or Route Server usage on the spoke side.

  7. Allow forwarded traffic on both peering directions.

  8. Enable IP forwarding on the NVA NIC and operating system.

  9. Validate the control plane and the data plane separately.

  10. Use at least two NVAs for a resilient production design.

  11. Plan traffic symmetry carefully for stateful appliances.

  12. Remove the billable lab resources after testing.

 
 
 

Comments

bottom of page