CV Pathfinder¶
Introduction¶
Important
- CVaaS is required to run this example. Without it, only configuration generation is possible.
- Minimum EOS version 4.32.2F is required.
- The devices must be able to reach CVaaS via their Management Interface.
- Proper licenses are required for all nodes (IPSec and throughput).
This example aims to present the basic configuration blocks required to deploy CV Pathfinder using AVD but does not cover all CV Pathfinder features. In particular, it does not cover:
- Internet Exits
- WAN routers behind NAT
- Multiple inventories
The WAN how-to document provides more information on how these features are supported in AVD.
This example will go over the following:
- Build the intended configurations and documentation
- Deploy the configuration via CloudVision as a Service (CVaaS)
Warning
- Site4 is using EVPN GW which is under preview.
Installation¶
Requirements to use this example:
- Follow the installation guide for AVD
- Run the following playbook to copy the AVD examples to your current working directory, for example
ansible-avd-examples:
This will show the following:
~/ansible-avd-examples# ansible-playbook arista.avd.install_examples
PLAY [Install Examples]**********************************************************************************************
TASK [Copy all examples to ~/ansible-avd-examples]*******************************************************************
changed: [localhost]
PLAY RECAP
*********************************************************************************************************************
localhost : ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Info
If the content of any file is modified and the playbook is rerun, the file will not be overwritten. However, if any file in the example is deleted and the playbook is rerun, Ansible will re-create the file.
After the playbook has run successfully, the directory structure of the example should look like below, the contents of which will be covered in later sections:
ansible-avd-examples/ (or wherever the playbook was run)
├── cv-pathfinder
├── ansible.cfg
├── build.yml
├── deploy.yml
├── documentation
├── group_vars
├── host_vars
├── images
├── intended
├── inventory.yml
├── README.md
└── switch-basic-configurations
Overall design overview¶
Physical topology¶
The target topology comprises two Pathfinder nodes and four sites distributed in two regions.
The drawing below shows the physical topology used in this example.
- The example considers two path groups:
MPLSandINTERNET - Pathfinders
pf1andpf2are connected to both to theINTERNETand theMPLSpath groups. inet-cloudandmpls-cloudare used to mimic Service Providers.
The following table describes the characteristics of each site:
| Site Name | Region | Role (Transit/Edge) | Number of routers | Path groups | HA configuration | LAN configuration |
|---|---|---|---|---|---|---|
| Site 1 | Region 1 | Transt | 2 routers | INTERNET and MPLS on both routers |
Via the LAN | eBGP |
| Site 2 | Region 2 | Transit | 2 routers | MPLS on router 1 and INTERNET on router 2 |
Direct HA | eBGP |
| Site 3 | Region 2 | Edge | 1 router | INTERNET |
- | Subinterfaces facing L2Leaf |
| Site 4 | Region 1 | Transit | 1 router | MPLS and INTERNET |
- | EVPN Gateway using Next-Hop-Self |
IP ranges used¶
Out-of-band management IP allocation¶
Subnet: 192.168.17.0/24
| Description | IP Address |
|---|---|
| Default gateway | 192.168.17.1 |
| Pathfinders | |
| pf1 | 192.168.17.10 |
| pf2 | 192.168.17.11 |
| Site 1 | |
| site1-wan1 | 192.168.17.12 |
| site1-wan2 | 192.168.17.13 |
| site1-border1 | 192.168.17.14 |
| site1-border2 | 192.168.17.15 |
| Site 2 | |
| site2-wan1 | 192.168.17.16 |
| site2-wan2 | 192.168.17.17 |
| site2-leaf1 | 192.168.17.18 |
| site2-leaf2 | 192.168.17.19 |
| Site 3 | |
| site3-wan1 | 192.168.17.20 |
| site3-leaf1 | 192.168.17.21 |
| Site 4 | |
| site4-wan1 | 192.168.17.22 |
| site4-border1 | 192.168.17.23 |
| site4-border2 | 192.168.17.24 |
| Clouds | |
| mpls-cloud | 192.168.17.30 |
| inet-cloud | 192.168.17.31 |
Other subnet IP allocations¶
| Description | Subnet |
|---|---|
| Loopback 0 interfaces | 192.168.255.0/24 |
| DPS/VTEP interfaces | 192.168.42.0/24 |
| MLAG peer-link (interface VLAN 4094) | 10.255.252.0/24 |
| MLAG iBGP peering (interface VLAN 4093) | 10.255.251.0/24 |
| Site1 uplink between WANs and border | 10.0.1.0/24 |
| Site2 uplink between WANs and leafs | 10.0.2.0/24 |
| Site4 uplink between WANs and borders | 10.0.4.0/24 |
| pf1 to mpls-cloud | 172.18.100.0/24 |
| pf1 to inet-cloud | 100.64.100.0/24 |
| pf2 to mpls-cloud | 172.18.200.0/24 |
| pf2 to inet-cloud | 100.64.200.0/24 |
| site1-wan1 to mpls-cloud | 172.18.10.0/24 |
| site1-wan1 to inet-cloud | 100.64.10.0/24 |
| site1-wan2 to mpls-cloud | 172.18.11.0/24 |
| site1-wan2 to inet-cloud | 100.64.11.0/24 |
| site2-wan1 to mpls-cloud | 172.18.20.0/24 |
| site2-wan2 to inet-cloud | 100.64.21.0/24 |
| site3-wan1 to inet-cloud | 172.18.30.0/24 |
| site4-wan1 to mpls-cloud | 172.18.40.0/24 |
| site4-wan1 to inet-cloud | 100.64.40.0/24 |
For every connection to inet-cloud or mpls-cloud, the cloud router is allocated .1 and the site / pf router is allocated .2.
VRFs/SVIs used on border routers and leafs for testing¶
| Site | Router | VRF | IP address |
|---|---|---|---|
| Site 1 | site1-border1 | BLUE | 10.66.10.1/24 |
| Site 1 | site1-border1 | RED | 10.42.10.1/24 |
| Site 1 | site1-border2 | BLUE | 10.66.11.1/24 |
| Site 1 | site1-border2 | RED | 10.42.11.1/24 |
| Site 2 | site1-leaf1 | BLUE | 10.66.20.1/24 |
| Site 2 | site1-leaf1 | RED | 10.42.20.1/24 |
| Site 2 | site1-leaf2 | BLUE | 10.66.21.1/24 |
| Site 2 | site1-leaf2 | RED | 10.42.21.1/24 |
| Site 3 | site1-wan3 | BLUE | 10.66.30.1/24 |
| Site 3 | site1-wan3 | RED | 10.42.30.1/24 |
| Site 4 | site4-border1 | BLUE | 10.66.40.1/24 |
| Site 4 | site4-border1 | RED | 10.42.40.1/24 |
| Site 4 | site4-border2 | BLUE | 10.66.44.1/24 |
| Site 4 | site4-border2 | RED | 10.42.44.1/24 |
Note
For site 3, the IP addresses are configured on the WAN router as site3-leaf1 is an l2leaf.
Ansible inventory, group vars, and naming scheme¶
The following drawing shows a graphic overview of the Ansible inventory, group variables, and naming scheme used in this example:
The following pattern is used:
- Group names use uppercase and underscore
- All hostnames use lowercase and dashes
The drawing also shows the relationships between groups and their children. Be aware that all declarations on a higher level are inherited by children automatically.
Content of the inventory.yml file¶
This section describes the entire ansible-avd-examples/cv-pathfinder/inventory.yml file used to represent the above topology.
In this example, we consider that no DNS entry is available to reach the devices and define the IPs the Ansible host has to reach per device.
CVaaS configuration
- The example is targeting cv-staging. Please adjust to the correct CVaaS region as described in the
cv_deployrole documentation - Additionally follow the guide to create the
cv_token - the
cv_tokenshould then be loaded as an ENV variableCV_TOKENusingexport CV_TOKEN=<token>for thedeploy.ymlplaybook to work toward CVaaS.
---
all:
hosts:
cloudvision:
ansible_host: apiserver.cv-staging.corp.arista.io
# cv_token is coming from an ENV variable
ansible_password: "{{ lookup('ansible.builtin.env', 'CV_TOKEN') }}"
children:
WAN:
children:
PATHFINDERS:
hosts:
pf1:
ansible_host: 192.168.17.10
pf2:
ansible_host: 192.168.17.11
SITE1:
hosts:
site1-wan1:
ansible_host: 192.168.17.12
site1-wan2:
ansible_host: 192.168.17.13
site1-border1:
ansible_host: 192.168.17.14
site1-border2:
ansible_host: 192.168.17.15
SITE2:
hosts:
site2-wan1:
ansible_host: 192.168.17.16
site2-wan2:
ansible_host: 192.168.17.17
site2-leaf1:
ansible_host: 192.168.17.18
site2-leaf2:
ansible_host: 192.168.17.19
SITE3:
hosts:
site3-wan1:
ansible_host: 192.168.17.20
site3-leaf1:
ansible_host: 192.168.17.21
SITE4:
hosts:
site4-wan1:
ansible_host: 192.168.17.22
site4-border1:
ansible_host: 192.168.17.23
site4-border2:
ansible_host: 192.168.17.24
TRANSPORTS:
hosts:
mpls-cloud:
ansible_host: 192.168.17.30
inet-cloud:
ansible_host: 192.168.17.31
This example demonstrates the use of Ansible Vault to keep variables secure.
ansible.cfg is configured to use a given file (.vault) as the vault password
when required to decrypt files or inline variables.
[defaults]
inventory=inventory.yml
jinja2_extensions = jinja2.ext.loopcontrols,jinja2.ext.do,jinja2.ext.i18n
# vault usage example
# In production, the .vault file *must* be excluded from your git repository
vault_password_file=.vault
Danger
The .vault file is included in the example in order to be able to run it.
It must never be pushed to any public repository as it allows anyone
with read access to decrypt all the secrets.
Basic EOS config¶
As discussed in the single DC example, basic connectivity between the Ansible host and the switches must be established before Ansible can be used to push configurations. Remember, you must configure the following on all switches:
- A hostname configured purely for ease of understanding.
- An IP enabled interface - in this example, the dedicated out-of-band management interface is used.
- A username and password with the proper access privileges.
! Basic EOS config
!
! Hostname of the device
hostname site1-border1
!
! Configures username and password for the ansible user
username arista privilege 15 role network-admin secret sha512 $6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1.
!
! Defines the VRF for MGMT
vrf instance MGMT
!
! Defines the settings for the Management1 interface through which Ansible reaches the device
interface Management1
description OOB_MANAGEMENT
no shutdown
vrf MGMT
! IP address - must be set uniquely per device
ip address 192.168.17.14/24
!
! Static default route for VRF MGMT
ip route vrf MGMT 0.0.0.0/0 192.168.17.1
!
! Enables API access in VRF MGMT
management api http-commands
protocol https
no shutdown
!
vrf MGMT
no shutdown
!
end
!
! Save configuration to flash
copy running-config startup-config
Note
The folder cv-pathfinder/switch-basic-configurations/ contains a file per device for the initial configurations.
Defining device types¶
To define device types, required by AVD, this example leverages the default_node_types key:
bgp_update_wait_install: false
# define default node types based on hostnames
default_node_types:
- node_type: wan_rr
match_hostnames:
- pf.*
- node_type: wan_router
match_hostnames:
- site.*-wan.*
- node_type: l2leaf
match_hostnames:
- site3-leaf.*
- node_type: l3leaf
match_hostnames:
- site.*-border.*
- site.*-leaf.*
# Transport routers
- node_type: spine
match_hostnames:
- .*-cloud
- Using node type
spinefor transport routers.
Pathfinder nodes use the node_type wan_rr, and all WAN routers (edge and transit) use the node_type wan_router.
Global settings for WAN¶
The following table lists the eos_designs top-level keys used for WAN and how they should be set:
| Key | Must be the same for all the WAN routers | Comment |
|---|---|---|
wan_mode |
✅ | Two possible modes, autovpn and cv-pathfinder (default). |
cv_pathfinder_regions |
✅ | Defines the Region/Zone/Site hierarchy, not required for AutoVPN. |
wan_route_servers |
✘ | Indicate to which WAN route servers the WAN router should connect to. This key is also used to tell every WAN Route Reflectors with which other RRs it should peer with. |
wan_ipsec_profiles |
✅ | Defines the shared key for the Control Plane and Data Plane IPSec profiles. |
wan_stun_dtls_disable |
✅ | Disable dTLS for STUN, for instance, for the lab. (NOT recommended in production). |
wan_carriers |
✅ | Defines the list of carriers in the network; each carrier is assigned to a path-group. |
wan_path_groups |
✅ | Defines the list of path-groups in the network. |
wan_virtual_topologies |
✅ | Defines the Policies and the VRF to policy mappings. |
tenants |
✅ | The default tenant key from network_services or any other key for tenant that would hold some WAN VRF information. |
application_classification |
✅ | Defines the specific traffic classification required for the WAN, if any. |
ipv4_acls |
✘ | List of IPv4 access-lists to be assigned to WAN interfaces. |
bgp_peer_groups.wan_overlay_peers.listen_range_prefixes |
✘ | Must be set for the pathfinders for the connectivity to work. Sets the ranges of IP addresses from which to expect BGP peerings for the WAN. Include the VTEP ranges for all WAN routers connecting to this pathfinder. |
In this example, the settings are set under the group WAN. To help logically separate the variables in meaningful categories, the group variables for WAN are created in several YAML files under ansible-avd-examples/cv-pathfinder/group_vars/WAN/:
cv-pathfinder/group_vars/WAN
├── cv_pathfinder_settings.yml
├── l3_interface_profiles.yml
├── management.yml
└── tenants.yml
Management¶
The management.yml file contains the configuration for:
- The management gateway
- NTP
- Terminattr to configure the connection to CVaaS.
- Local users (ansible/ansible, arista/arista and cvpadmin/cvpadmin)
ipv4_acls: a list of ACLs used for Internet-facing WAN interfaces- DNS
- AAA
- Disabling LLDP on management interface - Only needed for lab environments.
---
# WAN Management settings
mgmt_gateway: 192.168.17.1
# NTP
ntp_settings:
servers:
- name: 0.pool.ntp.org
# CloudVision/TerminAttr
cv_settings:
cvaas:
enabled: true
# For each local user in this example, password == username
aaa_settings:
local_users:
# Username with no password configured
- name: admin
privilege: 15
role: network-admin
no_password: true
# Username with a password
- name: arista
privilege: 15
role: network-admin
sha512_password: "$6$Enl0WfE32FthwyiJ$yTyGaEJ2uPKLU.F7314YtB7J1jrzrMi7ogXIRTEHQfLdLgKWWmr1UvNlZLN6AyuxET7G5aH3AI9OYRzxVTkB1."
- name: cvpadmin
privilege: 15
role: network-admin
sha512_password: $6$a7LdQWHxWzYHpvVt$n62q.1mbm4kzQ5oBr0lhXCE9ntnTn.SNa16DovZHahFQLH.iPcPMZa5JUSFtncrDW4EDQ3oSWgP8G0S4FtOFx1
# Internet ACL
ipv4_acls:
- name: ACL-PF-INTERNET-IN
entries:
- sequence: 1
remark: "Not for PRODUCTION: This ACL is built this way because the lab has an out-of-band interface"
- sequence: 10
# IPSec traffic from anywhere
action: permit
protocol: udp
source: any
destination: interface_ip
destination_ports_match: eq
destination_ports: [isakmp, non500-isakmp]
- sequence: 20
# STUN traffic from anywhere
action: permit
protocol: udp
source: any
destination: interface_ip
destination_ports_match: eq
destination_ports: [3478]
- sequence: 30
# Troubleshooting
action: permit
protocol: icmp
source: any
destination: interface_ip
- action: deny
protocol: ip
source: any
destination: any
- name: ACL-INTERNET-IN
entries:
- sequence: 1
remark: "Not for PRODUCTION: This ACL is built this way because the lab has an out-of-band interface"
- sequence: 10
# IPSec traffic from anywhere
action: permit
protocol: udp
source: any
destination: interface_ip
destination_ports_match: eq
destination_ports: [isakmp, non500-isakmp]
- sequence: 30
# Troubleshooting
action: permit
protocol: icmp
source: any
destination: interface_ip
- action: deny
protocol: ip
source: any
destination: any
# DNS
dns_settings:
domain: wan.example.local
servers:
- ip_address: 192.168.17.1
# AAA
custom_structured_configuration_aaa_authorization:
exec:
default: local
# disabling LLDP on management interface
custom_structured_configuration_management_interfaces:
- name: Management1
lldp:
transmit: false
receive: false
CV Pathinfder settings¶
The ansible-avd-examples/cv-pathfinder/group_vars/WAN/cv_pathfinder_settings.yml file defines the global WAN settings for all the hosts of the WAN group in the inventory.
---
# WAN CV Pathfinder settings
# cv-pathfinder is default
wan_mode: cv-pathfinder
# WAN hierarchy
cv_pathfinder_regions: # (1)!
- name: REGION1
id: 1
sites:
- name: SITE1
id: 101
location: Copenhagen, Denmark
- name: SITE4
id: 102
location: Stockholm, Sweden
- name: REGION2
id: 2
sites:
- name: SITE2
id: 202
location: Ottawa, Canada
- name: SITE3
id: 203
location: Milan, Italy
cv_pathfinder_global_sites: # (2)!
- name: PF1-GLOBAL
location: Santa Clara, CA, USA
- name: PF2-GLOBAL
location: Coulomiers, France
wan_route_servers: # (3)!
- hostname: pf1
- hostname: pf2
# IPSec configuration
wan_ipsec_profiles: # (4)!
control_plane:
shared_key: 045A190F1C354D
data_plane:
shared_key: 141600021F102B
# DTLS for STUN is enabled by default
# It requires certificates to be generated and distributed on the participating
# devices. To disable it for a lab, uncomment the following line.
# wan_stun_dtls_disable: true
# Fabric Flow tracking - by default only enabled for the Dps1 interface
fabric_flow_tracking: # (5)!
uplinks:
enabled: true
downlinks:
enabled: true
# Overwrite the default settings
flow_tracking_settings:
trackers:
- name: FLOW-TRACKER
record_export:
on_inactive_timeout: 70000
# Small export interval for example
on_interval: 5000
exporters:
- name: CV-TELEMETRY
collectors:
- host: 127.0.0.1
local_interface: Loopback0
# Vaulting bgp_password variable as an example. It is encrypted with the
# password in .vault
bgp_password: !vault |
$ANSIBLE_VAULT;1.1;AES256
33383161313537343432366430633733356331616665323132303263316531373135383439383730
3539343961636531643631343561356337656439366433320a366364653135376338313731323035
30633238643661626239656365613535393066666436633535636436633633303962393363343738
6262343237396166620a386465373366376235356438356139323832383131366462646263303037
6430
# BGP settings
bgp_peer_groups: # (6)!
wan_overlay_peers:
password: "{{ bgp_password | arista.avd.encrypt(passwd_type='bgp', key='WAN-OVERLAY-PEERS') }}"
listen_range_prefixes:
- 192.168.42.0/24
wan_rr_overlay_peers:
password: "{{ bgp_password | arista.avd.encrypt(passwd_type='bgp', key='WAN-RR-OVERLAY-PEERS') }}"
# WAN path groups
wan_carriers: # (7)!
- name: ACME-MPLS-INC
path_group: MPLS
trusted: true
- name: GLOBAL-INTERNET-LIMITED
path_group: INTERNET
- name: REGION1-INTERNET-CORP
path_group: INTERNET
- name: REGION2-INTERNET-CORP
path_group: INTERNET
wan_path_groups: # (8)!
- name: MPLS
id: 101
- name: INTERNET
id: 102
# WAN virtual topologies
wan_virtual_topologies: # (9)!
vrfs:
- name: BLUE
policy: BLUE-POLICY
wan_vni: 100
- name: RED
policy: RED-POLICY
wan_vni: 101
policies:
- name: BLUE-POLICY
application_virtual_topologies:
- application_profile: VIDEO
id: 2
path_groups:
- names: [INTERNET]
preference: preferred
- names: [MPLS] # (10)!
preference: alternate
- application_profile: VOICE
id: 3
dscp: 46
lowest_hop_count: true
constraints:
jitter: 30
latency: 150
loss_rate: 1
path_groups:
- names: [MPLS]
preference: preferred
- names: [INTERNET]
preference: alternate
default_virtual_topology:
path_groups:
- names: [INTERNET, MPLS]
preference: preferred
- name: RED-POLICY
application_virtual_topologies:
- application_profile: CRITICAL-SECRET-DATA
id: 2
path_groups:
- names: [MPLS]
preference: preferred
- application_profile: NORMAL-DATA
id: 3
path_groups:
- names: [INTERNET]
preference: preferred
- names: [MPLS]
preference: alternate
- application_profile: NOT-SO-IMPORTANT-DATA
id: 4
path_groups:
- names: [INTERNET]
preference: preferred
default_virtual_topology:
# dropping unmatched traffic in VRF RED
drop_unmatched: true
# Traffic classification
application_classification:
field_sets:
l4_ports:
- name: VOICE-PORTS
port_values:
- 666-667
- name: VIDEO-PORTS
port_values:
- 4242-4244
applications:
ipv4_applications:
- name: VOICE-APP
protocols:
- tcp
tcp_dest_port_set_name: VOICE-PORTS
- name: VIDEO-APP
protocols:
- tcp
- udp
tcp_dest_port_set_name: VIDEO-PORTS
udp_dest_port_set_name: VIDEO-PORTS
- name: CRITICAL-SECRET-DATA-APP
dscp_ranges: [46]
- name: NORMAL-DATA-APP
dscp_ranges: [af23]
- name: NOT-SO-IMPORTANT-DATA-APP
dscp_ranges: [0]
application_profiles:
- name: VOICE
applications:
- name: VOICE-APP
- name: VIDEO
applications:
- name: VIDEO-APP
- name: CRITICAL-SECRET-DATA
applications:
- name: CRITICAL-SECRET-DATA-APP
- name: NORMAL-DATA
applications:
- name: NORMAL-DATA-APP
- name: NOT-SO-IMPORTANT-DATA
applications:
- name: NOT-SO-IMPORTANT-DATA-APP
cv_pathfinder_regionsis used to declare the Regions, Sites, and their location in the WAN networkcv_pathfinder_global_sitesis used to add location for “Global” Pathfinders.wan_route_serversdefines the Pathfinders each router should connect to. This variable can also be used to have per-region pathfinders, for instance, at a group level.wan_ipsec_profilesis used to control IP Security configuration; these settings are required.- For lab purposes, Flow tracking is enabled on uplinks and downlinks. Make sure to verify the scalability of CloudVision depending on your network in Production.
bgp_peer_groupsuses listen range to establish sessions between WAN routers and the Pathfinders.wan_rr_overlay_peerscontrols the connection between Pathfinders, which is a full mesh.- When a carrier is not trusted (like the Internet ones), AVD requires an ingress ACL on the WAN interfaces. Each carrier is tied to a path-group.
- More on virtual topologies after the snippet.
- Application classification is used to configure
application_profilesthat match traffic categories in the policies and apply them to a virtual topology. - When policies are defined but no path exists (in this case MPLS for site 3) the path groups configuration will not be included in the final rendering.
Virtual topologies¶
The cornerstone of the CV Pathfinder solution is the Virtual Topologies.
The Virtual Topologies are grouped into Policies. A policy is a list of match
statements, each matching a specific application profile and applying a
profile (the Virtual Topology) to this traffic. A policy may or may not have
any default match. If no default match is configured, unmatched traffic is
dropped.
A profile is used to apply a load balancing policy and potentially an internet-exit policy (not in this example). The load balancing policy defines which path-groups can be used for the traffic.
As explained above, the wan_virtual_topologies variable must be global. AVD
decides how to configure the policies for each device based on the locally present
path groups. For example, if some traffic being matched is configured to only be sent over
the MPLS path group but there is no local WAN interface connected to MPLS, then
the match statement, profile, and load-balance policy are not generated by AVD on the
device.
For this example, we define two policies, one for each VRF BLUE and RED. The figure below is a graphical representation of a policy for video-based applications. Within our application virtual topologies, we specify a video application profile. This application profile can utilize two path groups. In this case, the internet provider is the preferred path, with MPLS acting as our alternative.
Applications¶
The figure below shows a graphical view of how we define applications. We have a high-level field set that allows us to define our application’s attributes. These can be either IPs or ports. In this case, we use the port range of 4242-4244.
We can leverage field set definitions within our application definitions. For this example, we have an application called VIDEO-APP with additional attributes like the protocol type in use and the destination ports. Please note that we are leveraging the VIDEO-PORTS definition from our field sets.
We can then define our application profiles. The application profiles can be a list of applications if they share the same policies. In this example, we have a fairly one-to-one mapping of applications and policies, but that is not a requirement. Within our VIDEO application profile, we leverage the VIDEO-APP application definition that was created previously.
L3 interfaces and L3 interface profiles¶
In AVD, an L3 interface configured under a node is considered a WAN interface
when the wan_carrier setting is set. The wan_carrier allows AVD to know
which path-group to use for the interface based on the Carrier to Path-Group mapping
in the top level wan_carriers key.
wan_router:
node_groups:
- group: SITE1
cv_pathfinder_region: REGION1
cv_pathfinder_site: SITE1
# Making this site a transit site
cv_pathfinder_transit_mode: region
wan_ha:
enabled: true
nodes:
- name: site1-wan1
id: 3
l3_interfaces:
- name: Ethernet3 # (1)!
profile: MPLS-WAN-INTERFACE
peer_interface: Ethernet5
peer_ip: 172.18.10.1
ip_address: 172.18.10.2/24
wan_carrier: ACME-MPLS-INC
wan_circuit_id: mpls-site1-wan1
- name: Ethernet42 # (2)!
ip_address: 100.64.10.2/24
- Ethernet3 is a WAN interface because
wan_carrieris defined - Ethernet42 is not a WAN interface because
wan_carrieris not defined
L3 interface profiles are defined globally and reused on the site WAN routers to apply common configurations.
---
l3_interface_profiles:
- profile: MPLS-WAN-INTERFACE
peer: mpls-cloud
# The static_routes uses the peer_ip as next-hop
static_routes:
- prefix: 172.18.0.0/16
# Enabling flow tracking on WAN interfaces
flow_tracking:
enabled: true
- profile: INTERNET-WAN-INTERFACE
peer: inet-cloud
ip_address: dhcp
dhcp_accept_default_route: true
ipv4_acl_in: ACL-INTERNET-IN
# Enabling flow tracking on WAN interfaces
flow_tracking:
enabled: true
Tenants¶
The tenants are defined globally. In this example, we create SVIs in the BLUE and RED VRFs for testing purposes.
Notice that the vrf_vni is configured for the BLUE and RED VRFs. This
potentially differs from the wan_vni configured under
wan_virtual_topologies.vrfs. The former is used for EVPN, while the latter is
used for DPS.
---
# WAN Tenants network services
tenants:
- name: WAN-EXAMPLE-TENANT
mac_vrf_vni_base: 10000
vrfs:
- name: BLUE
vrf_vni: 100
svis:
- id: 666
name: BLUE-TEST
enabled: true
nodes:
- node: site1-border1
ip_address: 10.66.1.1/24
- node: site1-border2
ip_address: 10.66.11.1/24
- node: site2-leaf1
ip_address: 10.66.2.1/24
- node: site2-leaf2
ip_address: 10.66.22.1/24
- node: site3-wan1
ip_address: 10.66.3.1/24
- node: site4-border1
ip_address: 10.66.4.1/24
- node: site4-border2
ip_address: 10.66.44.1/24
- name: RED
vrf_vni: 101
svis:
- id: 42
name: RED-TEST
enabled: true
nodes:
- node: site1-border1
ip_address: 10.42.1.1/24
- node: site1-border2
ip_address: 10.42.11.1/24
- node: site2-leaf1
ip_address: 10.42.2.1/24
- node: site2-leaf2
ip_address: 10.42.22.1/24
- node: site3-wan1
ip_address: 10.42.3.1/24
- node: site4-border1
ip_address: 10.42.4.1/24
- node: site4-border2
ip_address: 10.42.44.1/24
Setting pathfinders specific configuration parameters¶
The pathfinder configuration can be found in group_vars/PATHFINDERS.yml:
---
# WAN Pathfinder nodes variables
wan_rr:
defaults:
platform: CloudEOS
data_plane_cpu_allocation_max: 1 # (1)!
loopback_ipv4_pool: 192.168.255.0/24
vtep_loopback_ipv4_pool: 192.168.42.0/24
bgp_as: 65000
nodes:
- name: pf1
id: 1
mgmt_ip: 192.168.17.10/24
cv_pathfinder_site: PF1-GLOBAL # (2)!
# WAN interfaces
l3_interfaces:
- name: Ethernet1
peer: mpls-cloud
peer_interface: Ethernet1
peer_ip: 172.18.100.1
ip_address: 172.18.100.2/24
wan_carrier: ACME-MPLS-INC
wan_circuit_id: mpls-pf1
# The static_routes uses the peer_ip as next-hop
static_routes:
- prefix: 172.18.0.0/16
- name: Ethernet2
peer: inet-cloud
peer_interface: Ethernet1
peer_ip: 100.64.100.1
ip_address: 100.64.100.2/24
wan_carrier: GLOBAL-INTERNET-LIMITED
wan_circuit_id: inet-pf1
ipv4_acl_in: ACL-PF-INTERNET-IN
static_routes:
- prefix: 0.0.0.0/0
- name: pf2
id: 2
mgmt_ip: 192.168.17.11/24
cv_pathfinder_site: PF2-GLOBAL
# WAN interfaces
l3_interfaces:
- name: Ethernet1
peer: mpls-cloud
peer_interface: Ethernet2
peer_ip: 172.18.200.1
ip_address: 172.18.200.2/24
wan_carrier: ACME-MPLS-INC
wan_circuit_id: mpls-pf2
# The static_routes uses the peer_ip as next-hop
static_routes:
- prefix: 172.18.0.0/16
- name: Ethernet2
peer: inet-cloud
peer_interface: Ethernet2
peer_ip: 100.64.200.1
ip_address: 100.64.200.2/24
wan_carrier: GLOBAL-INTERNET-LIMITED
wan_circuit_id: inet-pf2
ipv4_acl_in: ACL-PF-INTERNET-IN
static_routes:
- prefix: 0.0.0.0/0
Setting site specific configuration parameters¶
Note
For convenience in the example, both the WAN routers and the LAN devices (borders, leaf, etc.) are defined in the same group_vars file. It would be possible to separate them.
As general principles:
- WAN routers should configure the LAN devices as
uplink_switches. - Each WAN router is assigned to a site. When two routers are part of a node_group, HA will be configured.
Site 1¶
The following diagrams describe the connectivity of Site 1’s physical, LAN, and HA tunnels.
Site 1 borders are using BGP AS 65101
By default, AVD uses all uplink interfaces for the LAN_HA path groups. EOS then establishes an IPsec tunnel between all pairs of local and remote connections. In this scenario, four tunnels are created for the LAN_HA path group.
---
# SITE1 variables
# Use eBGP as underlay routing protocol on SITE1, the default is 'none' for WAN routers.
underlay_routing_protocol: ebgp
l3leaf:
defaults:
platform: vEOS-lab
loopback_ipv4_pool: 192.168.255.0/24
vtep_loopback_ipv4_pool: 192.168.42.0/24
filter:
always_include_vrfs_in_tenants: [all]
mlag_interfaces: [Ethernet5, Ethernet6]
mlag_peer_l3_ipv4_pool: 10.255.251.0/24
mlag_peer_ipv4_pool: 10.255.252.0/24
node_groups:
- group: SITE1
bgp_as: 65101
nodes:
- name: site1-border1
mgmt_ip: 192.168.17.14/24
id: 5
- name: site1-border2
mgmt_ip: 192.168.17.15/24
id: 6
wan_router:
defaults:
platform: CloudEOS
loopback_ipv4_pool: 192.168.255.0/24
vtep_loopback_ipv4_pool: 192.168.42.0/24
uplink_ipv4_pool: 10.0.1.0/24
filter:
always_include_vrfs_in_tenants: [all]
uplink_interfaces: [Ethernet1, Ethernet2]
uplink_switches: [site1-border1, site1-border2]
uplink_type: p2p-vrfs # (1)!
bgp_as: 65000
node_groups:
- group: SITE1
cv_pathfinder_region: REGION1
cv_pathfinder_site: SITE1
# Making this site a transit site
cv_pathfinder_transit_mode: region # (2)!
wan_ha:
enabled: true
# Use this knob to disable IPSec on WAN HA tunnel(s)
# ipsec: false
nodes:
- name: site1-wan1
id: 3
mgmt_ip: 192.168.17.12/24
uplink_switch_interfaces: [Ethernet3, Ethernet3]
l3_interfaces:
- name: Ethernet3
profile: MPLS-WAN-INTERFACE
peer_interface: Ethernet5
peer_ip: 172.18.10.1
ip_address: 172.18.10.2/24
wan_carrier: ACME-MPLS-INC
wan_circuit_id: mpls-site1-wan1
- name: Ethernet4
peer_interface: Ethernet5
peer_ip: 100.64.10.1
ip_address: 100.64.10.2/24
profile: INTERNET-WAN-INTERFACE
wan_carrier: REGION1-INTERNET-CORP
wan_circuit_id: inet-site1-wan1
static_routes:
- prefix: 0.0.0.0/0
- name: site1-wan2
id: 4
mgmt_ip: 192.168.17.13/24
uplink_switch_interfaces: [Ethernet4, Ethernet4]
l3_interfaces:
- name: Ethernet3
profile: MPLS-WAN-INTERFACE
peer_interface: Ethernet6
peer_ip: 172.18.11.1
ip_address: 172.18.11.2/24
wan_carrier: ACME-MPLS-INC
wan_circuit_id: mpls-site1-wan2
- name: Ethernet4
peer_interface: Ethernet6
dhcp_ip: 100.64.11.2
profile: INTERNET-WAN-INTERFACE
wan_carrier: REGION1-INTERNET-CORP
wan_circuit_id: inet-site1-wan2
- The uplink type
p2p-vrfsis used to connect to the LAN switches. cv_pathfinder_transit_modedefaults tononeand the site is considered anedgesite. To use it as transit, the variable must be set toregionas it is the case here.
Site 2¶
The following diagrams describe the Site2 physical, LAN and HA tunnels connectivity.
Site 2 borders are using BGP AS 65101
---
# SITE2 variables
# Use eBGP as underlay routing protocol on SITE2, the default is 'none' for WAN routers.
underlay_routing_protocol: ebgp
ipv4_prefix_list_catalog:
- name: ALLOW-DEFAULT
sequence_numbers:
- sequence: 10
action: permit 0.0.0.0/0
# Internet ACL
ipv4_acls:
- name: ACL-INTERNET-IN
entries:
- sequence: 1
remark: "Not for PRODUCTION: This ACL is built this way because the lab has an out-of-band interface"
- sequence: 10
# IPSec traffic from anywhere
action: permit
protocol: udp
source: any
destination: interface_ip
destination_ports_match: eq
destination_ports: [isakmp, non500-isakmp]
- sequence: 20
# Permit BGP
action: permit
protocol: tcp
source: any
destination: interface_ip
destination_ports_match: eq
destination_ports: [bgp]
- sequence: 30
# Troubleshooting
action: permit
protocol: icmp
source: any
destination: interface_ip
- action: deny
protocol: ip
source: any
destination: any
l3leaf:
defaults:
platform: vEOS-lab
loopback_ipv4_pool: 192.168.255.0/24
vtep_loopback_ipv4_pool: 192.168.42.0/24
filter:
always_include_vrfs_in_tenants: [all]
mlag_interfaces: [Ethernet5, Ethernet6]
mlag_peer_l3_ipv4_pool: 10.255.251.0/24
mlag_peer_ipv4_pool: 10.255.252.0/24
node_groups:
- group: SITE2
bgp_as: 65102
nodes:
- name: site2-leaf1
id: 9
mgmt_ip: 192.168.17.18/24
- name: site2-leaf2
id: 10
mgmt_ip: 192.168.17.19/24
wan_router:
defaults:
platform: CloudEOS
loopback_ipv4_pool: 192.168.255.0/24
vtep_loopback_ipv4_pool: 192.168.42.0/24
filter:
always_include_vrfs_in_tenants: [all]
uplink_type: p2p-vrfs
uplink_interfaces: [Ethernet1]
uplink_switch_interfaces: [Ethernet3]
uplink_ipv4_pool: 10.0.2.0/24
bgp_as: 65000
node_groups:
- group: SITE2
cv_pathfinder_region: REGION2
cv_pathfinder_site: SITE2
# Making this site a transit site
cv_pathfinder_transit_mode: region
wan_ha:
enabled: true
# Direct HA
ha_interfaces: [Ethernet5]
ha_ipv4_pool: 10.42.0.0/24
use_port_channel_for_direct_ha: false
# Use this knob to disable IPSec on WAN HA tunnel(s)
# ipsec: false
nodes:
- name: site2-wan1
id: 7
mgmt_ip: 192.168.17.16/24
uplink_switches: [site2-leaf1]
l3_interfaces:
- name: Ethernet3
profile: MPLS-WAN-INTERFACE
peer_interface: Ethernet7
peer_ip: 172.18.20.1
ip_address: 172.18.20.2/24
wan_carrier: ACME-MPLS-INC
wan_circuit_id: mpls-site2-wan1
- name: site2-wan2
id: 8
mgmt_ip: 192.168.17.17/24
uplink_switches: [site2-leaf2]
l3_interfaces:
- name: Ethernet4
peer_interface: Ethernet7
peer_ip: 100.64.21.1
ip_address: 100.64.21.2/24
profile: INTERNET-WAN-INTERFACE
wan_carrier: REGION2-INTERNET-CORP
wan_circuit_id: inet-site2-wan2
# Configure a BGP peering with the Service Provider.
bgp:
peer_as: 65666
ipv4_prefix_list_in: ALLOW-DEFAULT
Site 3¶
The following diagrams describe the Site3 physical and LAN connectivity.
---
# SITE3 variables
l2leaf:
defaults:
platform: vEOS-lab
nodes:
- name: site3-leaf1
mgmt_ip: 192.168.17.21/24
id: 12
wan_router:
defaults:
platform: CloudEOS
loopback_ipv4_pool: 192.168.255.0/24
vtep_loopback_ipv4_pool: 192.168.42.0/24
uplink_ipv4_pool: 10.0.3.0/24
uplink_interfaces: [Ethernet1]
uplink_switch_interfaces: [Ethernet1]
uplink_type: lan
bgp_as: 65000
nodes:
- name: site3-wan1
cv_pathfinder_region: REGION2
cv_pathfinder_site: SITE3
id: 11
mgmt_ip: 192.168.17.20/24
uplink_switches: [site3-leaf1]
l3_port_channels:
- name: Port-Channel4
mode: active
member_interfaces:
- name: Ethernet4
peer_interface: Ethernet8
- name: Ethernet5
peer_interface: Ethernet9
peer: inet-cloud
ip_address: dhcp
dhcp_ip: 100.64.30.2
dhcp_accept_default_route: true
ipv4_acl_in: ACL-INTERNET-IN
peer_port_channel: Port-Channel8
wan_carrier: REGION2-INTERNET-CORP
wan_circuit_id: inet-site3-wan1
flow_tracking:
enabled: true
Site 4¶
Warning
- Site4 is using EVPN GW which is under preview.
The following diagrams describe the Site4 physical and LAN connectivity.
---
# SITE4 variables
# Use eBGP as underlay routing protocol on SITE4, the default is 'none' for WAN routers.
underlay_routing_protocol: ebgp
overlay_routing_protocol: ebgp # (1)!
l3leaf:
defaults:
platform: vEOS-lab
loopback_ipv4_pool: 192.168.255.0/24
vtep_loopback_ipv4_pool: 192.168.42.0/24
filter:
always_include_vrfs_in_tenants: [all]
mlag_interfaces: [Ethernet5, Ethernet6]
mlag_peer_l3_ipv4_pool: 10.255.251.0/24
mlag_peer_ipv4_pool: 10.255.252.0/24
node_groups:
- group: SITE4
bgp_as: 65104
nodes:
- name: site4-border1
mgmt_ip: 192.168.17.23/24
id: 13
evpn_role: server
- name: site4-border2
mgmt_ip: 192.168.17.24/24
id: 14
wan_router:
defaults:
platform: CloudEOS
loopback_ipv4_pool: 192.168.255.0/24
vtep_loopback_ipv4_pool: 192.168.42.0/24
uplink_ipv4_pool: 10.0.4.0/24
filter:
always_include_vrfs_in_tenants: [all]
uplink_interfaces: [Ethernet1, Ethernet2]
uplink_switches: [site4-border1, site4-border2]
uplink_type: p2p # (2)!
evpn_role: client # (3)!
evpn_route_servers: [site4-border1] # (4)!
bgp_as: 65000
node_groups:
- group: SITE4
cv_pathfinder_region: REGION1
cv_pathfinder_site: SITE4
cv_pathfinder_transit_mode: region
wan_ha:
enabled: false
nodes:
- name: site4-wan1
id: 15
evpn_role: client
mgmt_ip: 192.168.17.22/24
uplink_switch_interfaces: [Ethernet3, Ethernet3]
l3_interfaces:
- name: Ethernet3
profile: MPLS-WAN-INTERFACE
peer_interface: Ethernet8
peer_ip: 172.18.40.1
ip_address: 172.18.40.2/24
wan_carrier: ACME-MPLS-INC
wan_circuit_id: mpls-site4-wan1
- name: Ethernet4
peer_interface: Ethernet10
peer_ip: 100.64.40.1
ip_address: 100.64.40.2/24
static_routes:
- prefix: 100.64.0.0/16
profile: INTERNET-WAN-INTERFACE
wan_carrier: REGION1-INTERNET-CORP
wan_circuit_id: inet-site4-wan1
- Setting the overlay routing protocol to eBGP.
- The uplink type
p2pis used to connect to the LAN switches using EVPN. - Set the
evpn_roleto client to enable EVPN on the LAN side. - Declares
site4-border1as the EVPN server to pair with to receive LAN side EVPN routes.
The playbooks¶
The playbook is also the same, as the actions to execute in the fabric are the same. It is important to validate the hosts variable in the playbook to include all Ansible groups.
Testing AVD output without a lab¶
Example of using this playbook without devices (local tasks):
---
# build.yml
- name: Build Configs
hosts: WAN
gather_facts: false
tasks:
- name: Generate AVD Structured Configurations and Fabric Documentation
ansible.builtin.import_role:
name: arista.avd.eos_designs
- name: Generate Device Configurations and Documentation
ansible.builtin.import_role:
name: arista.avd.eos_cli_config_gen
Please look through the folders and files described above to learn more about the output generated by AVD.
The metadata section in the structured_config¶
The CV Pathfinder visualization in CloudVision relies on the inputs of a hidden metadata studio to work.
When using AVD to generate the CV Pathfinder configurations, AVD also generates
a metadata section with all the information required to be pushed to CVaaS to
populate the studio.
The arista.avd.cv_deploy role will then pick up the metadata section and
populate the metadata studio as part of the same workspace, thus keeping the
configurations and metadata in sync.
Playbook Run¶
To build and deploy the configurations to CVaaS, run first the build.yml playbook and then the deploy.yml playbook.
Note
In this example, the deploy.yml playbook is configured to auto approve and
execute the Change Control created by the Workspace using cv_run_change_control: true.
---
# deploy.yml
- name: Deploy Configurations via CVaaS
hosts: WAN
gather_facts: false
tasks:
- name: Deploy Configurations to Devices
vars:
# Automatically execute the Change Control
cv_run_change_control: true
ansible.builtin.import_role:
# By default this will use ansible_host and ansible_password
# from the 'cloudvision' host in the inventory
name: arista.avd.cv_deploy
### Build the configurations
ansible-playbook build.yml
### Deploy Configurations to CVaaS
ansible-playbook deploy.yml
user@ubuntu:~/cv-pathfinder$ ansible-playbook build.yml
PLAY [Build Configs] ***************************************************************************
TASK [arista.avd.eos_designs : Verify Requirements] ********************************************
AVD version 5.0.0
ok: [pf1 -> localhost]
TASK [arista.avd.eos_designs : Create required output directories if not present] **************
ok: [pf1 -> localhost] => (item=//home/user/Documents/git_projects/ansible-avd-examples/cv-pathfinder/intended/structured_configs)
ok: [pf1 -> localhost] => (item=/home/user/Documents/git_projects/ansible-avd-examples/cv-pathfinder/documentation/fabric)
(...)
Troubleshooting¶
For any question reach out to AVD maintainer over Github discussions. Please share the relevant logs as well as the following information:
- AVD version
- Python version
- EOS version in your lab (if relevant)
- Whether you are using CVaaS or CV on-premises (and, in this case, the version)