Goal
Goal is to build a multi-tenancy compatible EVPN setup with the ability to route between L2 domains and the possibility of external communications.
Assumptions
-
Linux-1, Linux-2, Linux-3 are in the same broadcast domain, VNI 10000
-
Linux-4, Linux-5 are in the same broadcast domain, VNI 20000
-
Linux-1, Linux-2, Linux-3, Linux-4, Linux-5 belong to the same tenant 'tenant-1'
-
L3 routing is allowed within same tenant
-
Connectivity to external nentworks is provided
Prerequirements
Cisco Nexus router have to have some feutures enabled:
nv overlay evpn feature ospf feature bgp feature interface-vlan feature vn-segment-vlan-based feature nv overlay
Underlay
OSPFv2 is used for the underlay, all routers are in the area 0.0.0.0 I.e.:
router ospf main_ospf router-id 192.168.55.3 passive-interface default
interfaces:
interface loopback0 ip address 192.168.55.3/32 ip router ospf main_ospf area 0.0.0.0 icam monitor scale interface Ethernet1/1 description to_r1 no switchport mtu 9202 ip address 10.0.13.2/24 no ip ospf passive-interface ip router ospf main_ospf area 0.0.0.0 no shutdown interface Ethernet1/2 description to_r2 no switchport mtu 9202 ip address 10.0.23.2/24 no ip ospf passive-interface ip router ospf main_ospf area 0.0.0.0 no shutdown
Overlay
BGP overlay uses iBGP. BGP sessions are established between loopbacks. Spines r1 and r2 are route-reflectors. Leaf sw1
router bgp 65000 router-id 192.168.55.1 address-family l2vpn evpn neighbor 192.168.99.1 remote-as 65000 update-source loopback0 address-family l2vpn evpn send-community send-community extended neighbor 192.168.99.2 remote-as 65000 update-source loopback0 address-family l2vpn evpn send-community send-community extended
Spine r1
router bgp 65000 bgp router-id 192.168.99.1 address-family ipv4 unicast ! address-family ipv6 unicast ! address-family l2vpn evpn ! neighbor-group evpn-group remote-as 65000 update-source Loopback0 ! neighbor 192.168.55.0/24 use neighbor-group evpn-group cluster-id 192.168.99.1 address-family l2vpn evpn route-reflector-client ! ! neighbor 192.168.99.2 remote-as 65000 update-source Loopback0 address-family l2vpn evpn ! ! ! end
VNIs and VLANs
Leaf sw2
vlan 1,10,20,999 vlan 10 vn-segment 10000 vlan 20 vn-segment 20000 vlan 999 vn-segment 999000
Linux-2 is in the VLAN 10, Linux-4 is in the VLAN 20. VLAN 999 is needed for inter VLAN communications.
Interfaces:
vrf context tenant1 vni 999000 rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn route-target import 65000:555 route-target import 65000:555 evpn address-family ipv6 unicast route-target both auto route-target both auto evpn route-target import 65000:555 route-target import 65000:555 evpn interface Vlan10 no shutdown vrf member tenant1 ip address 10.10.10.1/24 fabric forwarding mode anycast-gateway interface Vlan20 no shutdown vrf member tenant1 ip address 10.10.20.1/24 fabric forwarding mode anycast-gateway interface Vlan999 no shutdown vrf member tenant1 ip forward ipv6 forward interface nve1 no shutdown host-reachability protocol bgp source-interface loopback0 member vni 10000 ingress-replication protocol bgp member vni 20000 ingress-replication protocol bgp member vni 999000 associate-vrf interface Ethernet1/8 description to_Linux-4 switchport access vlan 20 interface Ethernet1/9 description to_Linux-2 switchport access vlan 10
vrf context tenant1
is used to route traffic between different broadcast domains. It should be present on every leaf carryin at least one VLAN of the tenant tenant1
. anycast-gateway
is used to route traffic, thus all leafs should have the same
fabric forwarding anycast-gateway-mac 0010.0010.0010
EVPN vni for L2 domains:
evpn vni 10000 l2 route-target import auto route-target export auto vni 20000 l2 route-target import auto route-target export auto
route-target import auto
and route-target export auto
are working since leafs are in the same ASN, for our case RT:65000:10000
and RT:65000:20000.
Community 65000:555 is used to import external routes as type 5
.
Routing
Routing between L2 domains is done through type 2
EVPN routes, routing to the external resourses through type 5
EVPN routes.
VNI 999000 and local VLAN 999 are used to route traffic between VNI 10000 and VNI 20000, type 2
EVPN routes are used.
VNI 555 and VLAN 555 are defined on the leaf sw3.
More on external routing
External routing uses VLAN 555 and VNI 555 defined on leat sw3.
Thereafter those routes are imported to L3 VNI on eny leaf where it’s needed through route-target import 65000:555 evpn
vlan 555 vn-segment 555
interface Vlan555 no shutdown vrf member external-shared ip forward ipv6 forward
vrf context external-shared vni 555 rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn route-target import 65000:999000 route-target import 65000:999000 evpn address-family ipv6 unicast route-target both auto route-target both auto evpn route-target import 65000:999000 route-target import 65000:999000 evpn
BGP on leaf sw3 is a bit longer.
router bgp 65000 address-family l2vpn evpn neighbor 192.168.99.1 remote-as 65000 update-source loopback0 address-family l2vpn evpn send-community send-community extended neighbor 192.168.99.2 remote-as 65000 update-source loopback0 address-family l2vpn evpn send-community send-community extended vrf external-shared address-family ipv4 unicast address-family ipv6 unicast neighbor 2001:: remote-as 42 address-family ipv6 unicast neighbor 100.64.64.0 remote-as 42 address-family ipv4 unicast route-map ADD-RT in vrf tenant1 address-family ipv4 unicast redistribute direct route-map ALL
vrf external-shared
is to import external routes to the corresponding vrf, and thereafte to vrf context tenant1
or any other vrf where needed.
However for sw3, routes in VLAN10 and VLAN20 are local and thus won’t get to the vrf context external-shared
and later on to ISP1 router throut route-target import 65000:999000
means. thus to workaroung that
we need to add vrf tenant1
into the BGP process and redistribute direct.
Compare two outputs:
With vrf tenant1
sw3# show bgp l2vpn evpn BGP routing table information for VRF default, address family L2VPN EVPN BGP table version is 93, Local Router ID is 192.168.55.3 Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-i njected Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - b est2 Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 192.168.55.1:32777 * i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[0]:[0.0.0.0]/216 192.168.55.1 100 0 i *>i 192.168.55.1 100 0 i * i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[32]:[10.10.10.111]/272 192.168.55.1 100 0 i *>i 192.168.55.1 100 0 i *>i[3]:[0]:[32]:[192.168.55.1]/88 192.168.55.1 100 0 i * i 192.168.55.1 100 0 i Route Distinguisher: 192.168.55.1:32787 *>i[3]:[0]:[32]:[192.168.55.1]/88 192.168.55.1 100 0 i * i 192.168.55.1 100 0 i Route Distinguisher: 192.168.55.2:32777 *>i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[0]:[0.0.0.0]/216 192.168.55.2 100 0 i * i 192.168.55.2 100 0 i * i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[32]:[10.10.10.2]/272 192.168.55.2 100 0 i *>i 192.168.55.2 100 0 i *>i[3]:[0]:[32]:[192.168.55.2]/88 192.168.55.2 100 0 i * i 192.168.55.2 100 0 i Route Distinguisher: 192.168.55.2:32787 * i[2]:[0]:[0]:[48]:[6609.b7b5.b7c4]:[0]:[0.0.0.0]/216 192.168.55.2 100 0 i *>i 192.168.55.2 100 0 i *>i[3]:[0]:[32]:[192.168.55.2]/88 192.168.55.2 100 0 i * i 192.168.55.2 100 0 i Route Distinguisher: 192.168.55.3:32777 (L2VNI 10000) *>i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[0]:[0.0.0.0]/216 192.168.55.2 100 0 i *>l[2]:[0]:[0]:[48]:[5e4f.b35e.9a0c]:[0]:[0.0.0.0]/216 192.168.55.3 100 32768 i *>i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[0]:[0.0.0.0]/216 192.168.55.1 100 0 i *>i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[32]:[10.10.10.2]/272 192.168.55.2 100 0 i *>l[2]:[0]:[0]:[48]:[5e4f.b35e.9a0c]:[32]:[10.10.10.3]/272 192.168.55.3 100 32768 i *>i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[32]:[10.10.10.111]/272 192.168.55.1 100 0 i *>i[3]:[0]:[32]:[192.168.55.1]/88 192.168.55.1 100 0 i *>i[3]:[0]:[32]:[192.168.55.2]/88 192.168.55.2 100 0 i *>l[3]:[0]:[32]:[192.168.55.3]/88 192.168.55.3 100 32768 i Route Distinguisher: 192.168.55.3:32787 (L2VNI 20000) *>l[2]:[0]:[0]:[48]:[523f.4dd8.d542]:[0]:[0.0.0.0]/216 192.168.55.3 100 32768 i *>i[2]:[0]:[0]:[48]:[6609.b7b5.b7c4]:[0]:[0.0.0.0]/216 192.168.55.2 100 0 i *>i[3]:[0]:[32]:[192.168.55.1]/88 192.168.55.1 100 0 i *>i[3]:[0]:[32]:[192.168.55.2]/88 192.168.55.2 100 0 i *>l[3]:[0]:[32]:[192.168.55.3]/88 192.168.55.3 100 32768 i Route Distinguisher: 192.168.55.3:3 (L3VNI 555) *>i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[32]:[10.10.10.2]/272 192.168.55.2 100 0 i *>i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[32]:[10.10.10.111]/272 192.168.55.1 100 0 i *>l[5]:[0]:[0]:[8]:[99.0.0.0]/224 192.168.55.3 0 42 i *>l[5]:[0]:[0]:[22]:[99.0.0.0]/224 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[24]:[99.0.1.0]/224 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[24]:[99.0.2.0]/224 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[24]:[99.0.3.0]/224 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[32]:[2006::]/416 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[48]:[2001:1::]/416 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[48]:[2001:2::]/416 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[48]:[2001:3::]/416 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[127]:[2001::]/416 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[128]:[2006::2006]/416 192.168.55.3 0 0 42 ? Route Distinguisher: 192.168.55.3:4 (L3VNI 999000) *>i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[32]:[10.10.10.2]/272 192.168.55.2 100 0 i *>i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[32]:[10.10.10.111]/272 192.168.55.1 100 0 i *>l[5]:[0]:[0]:[24]:[10.10.10.0]/224 192.168.55.3 0 100 32768 ? *>l[5]:[0]:[0]:[24]:[10.10.20.0]/224 192.168.55.3 0 100 32768 ?
RP/0/RP0/CPU0:isp1#show route Fri Nov 10 00:01:48.658 UTC Codes: C - connected, S - static, R - RIP, B - BGP, (>) - Diversion path D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP i - ISIS, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, su - IS-IS summary null, * - candidate default U - per-user static route, o - ODR, L - local, G - DAGR, l - LISP A - access/subscriber, a - Application route M - mobile route, r - RPL, t - Traffic Engineering, (!) - FRR Backup path Gateway of last resort is not set S 10.0.0.0/8 [1/0] via 100.64.64.1, 01:38:48 B 10.10.10.0/24 [20/0] via 100.64.64.1, 00:00:06 B 10.10.10.2/32 [20/0] via 100.64.64.1, 00:46:23 B 10.10.10.111/32 [20/0] via 100.64.64.1, 00:52:30 B 10.10.20.0/24 [20/0] via 100.64.64.1, 00:00:06 B 99.0.0.0/8 [200/0] via 0.0.0.0, 01:38:44, Null0 S 99.0.0.0/22 is directly connected, 01:39:30, Null0 C 99.0.1.0/24 is directly connected, 01:39:29, Loopback0 L 99.0.1.1/32 is directly connected, 01:39:29, Loopback0 C 99.0.2.0/24 is directly connected, 01:39:29, Loopback0 L 99.0.2.1/32 is directly connected, 01:39:29, Loopback0 C 99.0.3.0/24 is directly connected, 01:39:29, Loopback0 L 99.0.3.1/32 is directly connected, 01:39:29, Loopback0 C 100.64.64.0/31 is directly connected, 01:38:48, GigabitEthernet0/0/0/0 L 100.64.64.0/32 is directly connected, 01:38:48, GigabitEthernet0/0/0/0
and whout it
sw3# show bgp l2vpn evpn BGP routing table information for VRF default, address family L2VPN EVPN BGP table version is 95, Local Router ID is 192.168.55.3 Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-i njected Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - b est2 Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 192.168.55.1:32777 * i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[0]:[0.0.0.0]/216 192.168.55.1 100 0 i *>i 192.168.55.1 100 0 i * i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[32]:[10.10.10.111]/272 192.168.55.1 100 0 i *>i 192.168.55.1 100 0 i *>i[3]:[0]:[32]:[192.168.55.1]/88 192.168.55.1 100 0 i * i 192.168.55.1 100 0 i Route Distinguisher: 192.168.55.1:32787 *>i[3]:[0]:[32]:[192.168.55.1]/88 192.168.55.1 100 0 i * i 192.168.55.1 100 0 i Route Distinguisher: 192.168.55.2:32777 *>i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[0]:[0.0.0.0]/216 192.168.55.2 100 0 i * i 192.168.55.2 100 0 i * i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[32]:[10.10.10.2]/272 192.168.55.2 100 0 i *>i 192.168.55.2 100 0 i *>i[3]:[0]:[32]:[192.168.55.2]/88 192.168.55.2 100 0 i * i 192.168.55.2 100 0 i Route Distinguisher: 192.168.55.2:32787 * i[2]:[0]:[0]:[48]:[6609.b7b5.b7c4]:[0]:[0.0.0.0]/216 192.168.55.2 100 0 i *>i 192.168.55.2 100 0 i *>i[3]:[0]:[32]:[192.168.55.2]/88 192.168.55.2 100 0 i * i 192.168.55.2 100 0 i Route Distinguisher: 192.168.55.3:32777 (L2VNI 10000) *>i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[0]:[0.0.0.0]/216 192.168.55.2 100 0 i *>l[2]:[0]:[0]:[48]:[5e4f.b35e.9a0c]:[0]:[0.0.0.0]/216 192.168.55.3 100 32768 i *>i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[0]:[0.0.0.0]/216 192.168.55.1 100 0 i *>i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[32]:[10.10.10.2]/272 192.168.55.2 100 0 i *>l[2]:[0]:[0]:[48]:[5e4f.b35e.9a0c]:[32]:[10.10.10.3]/272 192.168.55.3 100 32768 i *>i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[32]:[10.10.10.111]/272 192.168.55.1 100 0 i *>i[3]:[0]:[32]:[192.168.55.1]/88 192.168.55.1 100 0 i *>i[3]:[0]:[32]:[192.168.55.2]/88 192.168.55.2 100 0 i *>l[3]:[0]:[32]:[192.168.55.3]/88 192.168.55.3 100 32768 i Route Distinguisher: 192.168.55.3:32787 (L2VNI 20000) *>l[2]:[0]:[0]:[48]:[523f.4dd8.d542]:[0]:[0.0.0.0]/216 192.168.55.3 100 32768 i *>i[2]:[0]:[0]:[48]:[6609.b7b5.b7c4]:[0]:[0.0.0.0]/216 192.168.55.2 100 0 i *>i[3]:[0]:[32]:[192.168.55.1]/88 192.168.55.1 100 0 i *>i[3]:[0]:[32]:[192.168.55.2]/88 192.168.55.2 100 0 i *>l[3]:[0]:[32]:[192.168.55.3]/88 192.168.55.3 100 32768 i Route Distinguisher: 192.168.55.3:3 (L3VNI 555) *>i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[32]:[10.10.10.2]/272 192.168.55.2 100 0 i *>i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[32]:[10.10.10.111]/272 192.168.55.1 100 0 i *>l[5]:[0]:[0]:[8]:[99.0.0.0]/224 192.168.55.3 0 42 i *>l[5]:[0]:[0]:[22]:[99.0.0.0]/224 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[24]:[99.0.1.0]/224 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[24]:[99.0.2.0]/224 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[24]:[99.0.3.0]/224 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[32]:[2006::]/416 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[48]:[2001:1::]/416 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[48]:[2001:2::]/416 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[48]:[2001:3::]/416 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[127]:[2001::]/416 192.168.55.3 0 0 42 ? *>l[5]:[0]:[0]:[128]:[2006::2006]/416 192.168.55.3 0 0 42 ? Route Distinguisher: 192.168.55.3:4 (L3VNI 999000) *>i[2]:[0]:[0]:[48]:[4e33.5b83.835b]:[32]:[10.10.10.2]/272 192.168.55.2 100 0 i *>i[2]:[0]:[0]:[48]:[6e43.b74d.3712]:[32]:[10.10.10.111]/272 192.168.55.1 100 0 i
RP/0/RP0/CPU0:isp1#show route Fri Nov 10 00:01:00.516 UTC Codes: C - connected, S - static, R - RIP, B - BGP, (>) - Diversion path D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP i - ISIS, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, su - IS-IS summary null, * - candidate default U - per-user static route, o - ODR, L - local, G - DAGR, l - LISP A - access/subscriber, a - Application route M - mobile route, r - RPL, t - Traffic Engineering, (!) - FRR Backup path Gateway of last resort is not set S 10.0.0.0/8 [1/0] via 100.64.64.1, 01:38:00 B 10.10.10.2/32 [20/0] via 100.64.64.1, 00:45:35 B 10.10.10.111/32 [20/0] via 100.64.64.1, 00:51:42 B 99.0.0.0/8 [200/0] via 0.0.0.0, 01:37:56, Null0 S 99.0.0.0/22 is directly connected, 01:38:42, Null0 C 99.0.1.0/24 is directly connected, 01:38:41, Loopback0 L 99.0.1.1/32 is directly connected, 01:38:41, Loopback0 C 99.0.2.0/24 is directly connected, 01:38:41, Loopback0 L 99.0.2.1/32 is directly connected, 01:38:41, Loopback0 C 99.0.3.0/24 is directly connected, 01:38:41, Loopback0 L 99.0.3.1/32 is directly connected, 01:38:41, Loopback0 C 100.64.64.0/31 is directly connected, 01:38:00, GigabitEthernet0/0/0/0 L 100.64.64.0/32 is directly connected, 01:38:00, GigabitEthernet0/0/0/0
Basically vrf tenant1
advetises whole subnet towards ISP1 and not individual type 2
IPs like for leafs sw1 and sw2, but that the acceptable workaroung.
Verification
Packet capture showing inter VLAN/VNI communication through VNI 999000
Frame 44587: 148 bytes on wire (1184 bits), 148 bytes captured (1184 bits) on interface -, id 0 Ethernet II, Src: 0c:c7:00:00:1b:08 (0c:c7:00:00:1b:08), Dst: 0c:3a:4f:a4:00:03 (0c:3a:4f:a4:00:03) Internet Protocol Version 4, Src: 192.168.55.1, Dst: 192.168.55.3 User Datagram Protocol, Src Port: 49872, Dst Port: 4789 Virtual eXtensible Local Area Network Flags: 0x0800, VXLAN Network ID (VNI) Group Policy ID: 0 VXLAN Network Identifier (VNI): 999000 Reserved: 0 Ethernet II, Src: 0c:c7:00:00:1b:08 (0c:c7:00:00:1b:08), Dst: 0c:36:00:00:1b:08 (0c:36:00:00:1b:08) Destination: 0c:36:00:00:1b:08 (0c:36:00:00:1b:08) Source: 0c:c7:00:00:1b:08 (0c:c7:00:00:1b:08) Type: IPv4 (0x0800) Internet Protocol Version 4, Src: 10.10.10.111, Dst: 10.10.20.5 0100 .... = Version: 4 .... 0101 = Header Length: 20 bytes (5) Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT) Total Length: 84 Identification: 0xd3b1 (54193) 000. .... = Flags: 0x0 ...0 0000 0000 0000 = Fragment Offset: 0 Time to Live: 63 Protocol: ICMP (1) Header Checksum: 0x7570 [validation disabled] [Header checksum status: Unverified] Source Address: 10.10.10.111 Destination Address: 10.10.20.5 Internet Control Message Protocol
Full configs
ISP1
RP/0/RP0/CPU0:isp1#show running-config Fri Nov 10 00:07:46.391 UTC Building configuration... !! IOS XR Configuration 7.3.1 !! Last configuration change at Mon Nov 6 21:28:18 2023 by admin ! hostname isp1 username admin group root-lr group cisco-support secret 10 $6$VftYn/OCtQHC5n/.$2amSbAY5JnckN37abVwr6Xy.e2OCXQ/6kUUIiLYRm/gU14zSpBtXczVR2ya3JaAk18NnWilLpu0ifFocbjCHN/ ! call-home service active contact smart-licensing profile CiscoTAC-1 active destination transport-method http ! ! interface Loopback0 ipv4 address 99.0.1.1 255.255.255.0 ipv4 address 99.0.2.1 255.255.255.0 secondary ipv4 address 99.0.3.1 255.255.255.0 secondary ipv6 address 2001:1::/48 ipv6 address 2001:2::/48 ipv6 address 2001:3::/48 ipv6 address 2006::2006/128 ! interface MgmtEth0/RP0/CPU0/0 shutdown ! interface GigabitEthernet0/0/0/0 description to_sw1 ipv4 address 100.64.64.0 255.255.255.254 ipv6 address 2001::/127 ! interface GigabitEthernet0/0/0/1 shutdown ! interface GigabitEthernet0/0/0/2 shutdown ! interface GigabitEthernet0/0/0/3 shutdown ! route-policy IBGP-FILTER-V4-V6 pass end-policy ! router static address-family ipv4 unicast 10.0.0.0/8 100.64.64.1 99.0.0.0/22 Null0 254 tag 9999 ! address-family ipv6 unicast 2006::/32 Null0 254 tag 9999 ! ! router bgp 42 address-family ipv4 unicast network 99.0.0.0/8 aggregate-address 99.0.0.0/8 redistribute connected redistribute static ! address-family ipv6 unicast network 2000::/3 network 2001::/16 aggregate-address 2006::/32 redistribute connected redistribute static ! neighbor 2001::1 remote-as 65000 address-family ipv6 unicast route-policy IBGP-FILTER-V4-V6 in route-policy IBGP-FILTER-V4-V6 out ! ! neighbor 100.64.64.1 remote-as 65000 address-family ipv4 unicast route-policy IBGP-FILTER-V4-V6 in route-policy IBGP-FILTER-V4-V6 out ! ! ! end
Spine r1
RP/0/RP0/CPU0:r1#show running-config Fri Nov 10 00:10:33.807 UTC Building configuration... !! IOS XR Configuration 7.3.1 !! Last configuration change at Fri Nov 3 20:13:23 2023 by admin ! hostname r1 username admin group root-lr group cisco-support secret 10 $6$duKZI/iJ7JJG3I/.$lFoQpquJREMaXysffr5BfK54RdFNjNvt8YCCj3oPqUSFYJTw0ePQ7RTKws7LleeTESJosd3l5Rd7tZzv.jWv4/ ! call-home service active contact smart-licensing profile CiscoTAC-1 active destination transport-method http ! ! interface Loopback0 ipv4 address 192.168.99.1 255.255.255.255 ! interface MgmtEth0/RP0/CPU0/0 shutdown ! interface GigabitEthernet0/0/0/0 description to_sw1 mtu 9216 ipv4 address 10.0.11.1 255.255.255.0 ! interface GigabitEthernet0/0/0/1 description to_sw2 mtu 9216 ipv4 address 10.0.12.1 255.255.255.0 ! interface GigabitEthernet0/0/0/2 description to_sw3 mtu 9216 ipv4 address 10.0.13.1 255.255.255.0 ! interface GigabitEthernet0/0/0/3 shutdown ! router ospf main_osfp router-id 192.168.99.1 area 0.0.0.0 interface Loopback0 passive enable ! interface GigabitEthernet0/0/0/0 ! interface GigabitEthernet0/0/0/1 ! interface GigabitEthernet0/0/0/2 ! ! ! router bgp 65000 bgp router-id 192.168.99.1 address-family ipv4 unicast ! address-family ipv6 unicast ! address-family l2vpn evpn ! neighbor-group evpn-group remote-as 65000 update-source Loopback0 ! neighbor 192.168.55.0/24 use neighbor-group evpn-group cluster-id 192.168.99.1 address-family l2vpn evpn route-reflector-client ! ! neighbor 192.168.99.2 remote-as 65000 update-source Loopback0 address-family l2vpn evpn ! ! ! end
Spine r2
RP/0/RP0/CPU0:r2#show running-config Fri Nov 10 00:13:34.104 UTC Building configuration... !! IOS XR Configuration 7.3.1 !! Last configuration change at Fri Nov 3 20:13:51 2023 by admin ! hostname r2 username admin group root-lr group cisco-support secret 10 $6$Kq/iWRUf6s80W...$eYGtToQGVNrq3GHeqkIlVyBCCZIu2z7A.DOx8qntM5bCRz9t3RSRhF/MHzYHoHrl72SC3cUPdFKqZhWduuTgd. ! address-family ipv4 unicast ! address-family ipv6 unicast ! call-home service active contact smart-licensing profile CiscoTAC-1 active destination transport-method http ! ! interface Loopback0 ipv4 address 192.168.99.2 255.255.255.255 ! interface MgmtEth0/RP0/CPU0/0 shutdown ! interface GigabitEthernet0/0/0/0 description to_r1 mtu 9216 ipv4 address 10.0.21.1 255.255.255.0 ! interface GigabitEthernet0/0/0/1 description to_r2 mtu 9216 ipv4 address 10.0.22.1 255.255.255.0 ! interface GigabitEthernet0/0/0/2 description to_sw3 mtu 9216 ipv4 address 10.0.23.1 255.255.255.0 ! interface GigabitEthernet0/0/0/3 shutdown ! router ospf main_osfp router-id 192.168.99.2 area 0.0.0.0 interface Loopback0 passive enable ! interface GigabitEthernet0/0/0/0 ! interface GigabitEthernet0/0/0/1 ! interface GigabitEthernet0/0/0/2 ! ! ! router bgp 65000 bgp router-id 192.168.99.2 address-family ipv4 unicast ! address-family ipv6 unicast ! address-family l2vpn evpn ! neighbor-group evpn-group remote-as 65000 update-source Loopback0 ! neighbor 192.168.55.0/24 use neighbor-group evpn-group cluster-id 192.168.99.2 address-family l2vpn evpn route-reflector-client ! ! neighbor 192.168.99.1 remote-as 65000 update-source Loopback0 address-family l2vpn evpn ! ! ! end
Leaf sw1
sw1# show running-config !Command: show running-config !Running configuration last done at: Thu Nov 9 23:35:06 2023 !Time: Fri Nov 10 00:11:14 2023 version 10.2(5) Bios:version switchname sw1 vdc sw1 id 1 limit-resource vlan minimum 16 maximum 4094 limit-resource vrf minimum 2 maximum 4096 limit-resource port-channel minimum 0 maximum 511 limit-resource m4route-mem minimum 58 maximum 58 limit-resource m6route-mem minimum 8 maximum 8 nv overlay evpn feature ospf feature bgp feature interface-vlan feature vn-segment-vlan-based feature nv overlay username admin password 5 $5$EPLIOB$VcXBBD0zdD4Pt3YXX2yqBUTM6M3fJwBdJxwrYUfM6D/ role network-admin ip domain-lookup copp profile strict snmp-server user admin network-admin auth md5 3209EFD2820344D265E19A4B9A10ED749A CB priv aes-128 0040F182CD18429038CCD85C9F56E910C88D localizedV2key rmon event 1 log trap public description FATAL(1) owner PMON@FATAL rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL rmon event 3 log trap public description ERROR(3) owner PMON@ERROR rmon event 4 log trap public description WARNING(4) owner PMON@WARNING rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO system default switchport fabric forwarding anycast-gateway-mac 0010.0010.0010 vlan 1,10,20,999 vlan 10 vn-segment 10000 vlan 20 vn-segment 20000 vlan 999 vn-segment 999000 vrf context management vrf context tenant1 vni 999000 rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn route-target import 65000:555 route-target import 65000:555 evpn address-family ipv6 unicast route-target both auto route-target both auto evpn route-target import 65000:555 route-target import 65000:555 evpn interface Vlan1 interface Vlan10 no shutdown vrf member tenant1 ip address 10.10.10.1/24 fabric forwarding mode anycast-gateway interface Vlan20 no shutdown vrf member tenant1 ip address 10.10.20.1/24 fabric forwarding mode anycast-gateway interface Vlan999 no shutdown vrf member tenant1 ip forward ipv6 forward interface nve1 no shutdown host-reachability protocol bgp source-interface loopback0 member vni 10000 ingress-replication protocol bgp member vni 20000 ingress-replication protocol bgp member vni 999000 associate-vrf interface Ethernet1/1 description to_r1 no switchport mtu 9202 ip address 10.0.11.2/24 no ip ospf passive-interface ip router ospf main_ospf area 0.0.0.0 no shutdown interface Ethernet1/2 description to_r2 no switchport mtu 9202 ip address 10.0.21.2/24 no ip ospf passive-interface ip router ospf main_ospf area 0.0.0.0 no shutdown interface Ethernet1/3 interface Ethernet1/4 interface Ethernet1/5 interface Ethernet1/6 interface Ethernet1/7 interface Ethernet1/8 interface Ethernet1/9 switchport access vlan 10 interface Ethernet1/10 interface Ethernet1/11 interface Ethernet1/12 interface Ethernet1/13 interface Ethernet1/14 interface Ethernet1/15 interface Ethernet1/16 interface Ethernet1/17 interface Ethernet1/18 interface Ethernet1/19 interface Ethernet1/20 interface Ethernet1/21 interface Ethernet1/22 interface Ethernet1/23 interface Ethernet1/24 interface Ethernet1/25 interface Ethernet1/26 interface Ethernet1/27 interface Ethernet1/28 interface Ethernet1/29 interface Ethernet1/30 interface Ethernet1/31 interface Ethernet1/32 interface Ethernet1/33 interface Ethernet1/34 interface Ethernet1/35 interface Ethernet1/36 interface Ethernet1/37 interface Ethernet1/38 interface Ethernet1/39 interface Ethernet1/40 interface Ethernet1/41 interface Ethernet1/42 interface Ethernet1/43 interface Ethernet1/44 interface Ethernet1/45 interface Ethernet1/46 interface Ethernet1/47 interface Ethernet1/48 interface Ethernet1/49 interface Ethernet1/50 interface Ethernet1/51 interface Ethernet1/52 interface Ethernet1/53 interface Ethernet1/54 interface Ethernet1/55 interface Ethernet1/56 interface Ethernet1/57 interface Ethernet1/58 interface Ethernet1/59 interface Ethernet1/60 interface Ethernet1/61 interface Ethernet1/62 interface Ethernet1/63 interface Ethernet1/64 interface mgmt0 vrf member management interface loopback0 ip address 192.168.55.1/32 ip router ospf main_ospf area 0.0.0.0 icam monitor scale line console line vty boot nxos bootflash:/nxos64-cs.10.2.5.M.bin router ospf main_ospf router-id 192.168.55.1 passive-interface default router bgp 65000 router-id 192.168.55.1 address-family l2vpn evpn neighbor 192.168.99.1 remote-as 65000 update-source loopback0 address-family l2vpn evpn send-community send-community extended neighbor 192.168.99.2 remote-as 65000 update-source loopback0 address-family l2vpn evpn send-community send-community extended evpn vni 10000 l2 route-target import auto route-target export auto vni 20000 l2 route-target import auto route-target export auto no system default switchport shutdown
Leaf sw2
sw2# show running-config !Command: show running-config !No configuration change since last restart !Time: Fri Nov 10 00:12:16 2023 version 10.2(5) Bios:version switchname sw2 vdc sw2 id 1 limit-resource vlan minimum 16 maximum 4094 limit-resource vrf minimum 2 maximum 4096 limit-resource port-channel minimum 0 maximum 511 limit-resource m4route-mem minimum 58 maximum 58 limit-resource m6route-mem minimum 8 maximum 8 nv overlay evpn feature ospf feature bgp feature interface-vlan feature vn-segment-vlan-based feature nv overlay username admin password 5 $5$MABCHN$gUqF9QuVnSFIDj/pFMmyAP50tJidjCcvsT2DZ84N406 role network-admin ip domain-lookup copp profile strict snmp-server user admin network-admin auth md5 01461793D54A024F0402260F6F6A68FE17 E7 priv aes-128 494278ACA1CCD181E9D9AECAB2DAF73B8D35 localizedV2key rmon event 1 log trap public description FATAL(1) owner PMON@FATAL rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL rmon event 3 log trap public description ERROR(3) owner PMON@ERROR rmon event 4 log trap public description WARNING(4) owner PMON@WARNING rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO system default switchport fabric forwarding anycast-gateway-mac 0010.0010.0010 vlan 1,10,20,999 vlan 10 vn-segment 10000 vlan 20 vn-segment 20000 vlan 999 vn-segment 999000 vrf context management vrf context tenant1 vni 999000 rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn address-family ipv6 unicast route-target both auto route-target both auto evpn interface Vlan1 interface Vlan10 no shutdown vrf member tenant1 ip address 10.10.10.1/24 fabric forwarding mode anycast-gateway interface Vlan20 no shutdown vrf member tenant1 ip address 10.10.20.1/24 fabric forwarding mode anycast-gateway interface Vlan999 no shutdown vrf member tenant1 ip forward ipv6 forward interface nve1 no shutdown host-reachability protocol bgp source-interface loopback0 member vni 10000 ingress-replication protocol bgp member vni 20000 ingress-replication protocol bgp member vni 999000 associate-vrf interface Ethernet1/1 description to_r1 no switchport mtu 9202 ip address 10.0.12.2/24 no ip ospf passive-interface ip router ospf main_ospf area 0.0.0.0 no shutdown interface Ethernet1/2 description to_r2 no switchport mtu 9202 ip address 10.0.22.2/24 no ip ospf passive-interface ip router ospf main_ospf area 0.0.0.0 no shutdown interface Ethernet1/3 interface Ethernet1/4 interface Ethernet1/5 interface Ethernet1/6 interface Ethernet1/7 interface Ethernet1/8 description to_Linux-4 switchport access vlan 20 interface Ethernet1/9 description to_Linux-2 switchport access vlan 10 interface Ethernet1/10 interface Ethernet1/11 interface Ethernet1/12 interface Ethernet1/13 interface Ethernet1/14 interface Ethernet1/15 interface Ethernet1/16 interface Ethernet1/17 interface Ethernet1/18 interface Ethernet1/19 interface Ethernet1/20 interface Ethernet1/21 interface Ethernet1/22 interface Ethernet1/23 interface Ethernet1/24 interface Ethernet1/25 interface Ethernet1/26 interface Ethernet1/27 interface Ethernet1/28 interface Ethernet1/29 interface Ethernet1/30 interface Ethernet1/31 interface Ethernet1/32 interface Ethernet1/33 interface Ethernet1/34 interface Ethernet1/35 interface Ethernet1/36 interface Ethernet1/37 interface Ethernet1/38 interface Ethernet1/39 interface Ethernet1/40 interface Ethernet1/41 interface Ethernet1/42 interface Ethernet1/43 interface Ethernet1/44 interface Ethernet1/45 interface Ethernet1/46 interface Ethernet1/47 interface Ethernet1/48 interface Ethernet1/49 interface Ethernet1/50 interface Ethernet1/51 interface Ethernet1/52 interface Ethernet1/53 interface Ethernet1/54 interface Ethernet1/55 interface Ethernet1/56 interface Ethernet1/57 interface Ethernet1/58 interface Ethernet1/59 interface Ethernet1/60 interface Ethernet1/61 interface Ethernet1/62 interface Ethernet1/63 interface Ethernet1/64 interface mgmt0 vrf member management ip address 10.0.0.2/24 interface loopback0 ip address 192.168.55.2/32 ip router ospf main_ospf area 0.0.0.0 icam monitor scale line console line vty boot nxos bootflash:/nxos64-cs.10.2.5.M.bin router ospf main_ospf router-id 192.168.55.2 passive-interface default router bgp 65000 address-family l2vpn evpn neighbor 192.168.99.1 remote-as 65000 update-source loopback0 address-family l2vpn evpn send-community send-community extended neighbor 192.168.99.2 remote-as 65000 update-source loopback0 address-family l2vpn evpn send-community send-community extended evpn vni 10000 l2 route-target import auto route-target export auto vni 20000 l2 route-target import auto route-target export auto no system default switchport shutdown
Leaf sw3
sw3# show running-config | no-more !Command: show running-config !Running configuration last done at: Fri Nov 10 00:01:22 2023 !Time: Fri Nov 10 00:09:15 2023 version 10.2(5) Bios:version switchname sw3 vdc sw3 id 1 limit-resource vlan minimum 16 maximum 4094 limit-resource vrf minimum 2 maximum 4096 limit-resource port-channel minimum 0 maximum 511 limit-resource m4route-mem minimum 58 maximum 58 limit-resource m6route-mem minimum 8 maximum 8 nv overlay evpn feature ospf feature bgp feature interface-vlan feature vn-segment-vlan-based feature nv overlay username admin password 5 $5$EHHCCC$XE.MliFvH6EEc5GGywgWyb4Qx6NysPdtudqLMBFSQXB role network-admin ip domain-lookup copp profile strict snmp-server user admin network-admin auth md5 204F143CE4DF43D72D285A335226CE7CCE28 priv aes-128 480A6B1FD1E5408864374037136D9D239E0C localizedV2key rmon event 1 log trap public description FATAL(1) owner PMON@FATAL rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL rmon event 3 log trap public description ERROR(3) owner PMON@ERROR rmon event 4 log trap public description WARNING(4) owner PMON@WARNING rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO system default switchport fabric forwarding anycast-gateway-mac 0010.0010.0010 vlan 1,10,20,555,999 vlan 10 vn-segment 10000 vlan 20 vn-segment 20000 vlan 555 vn-segment 555 vlan 999 vn-segment 999000 ip prefix-list NETWORK seq 5 permit 99.0.0.0/8 le 32 route-map ADD-RT permit 10 match ip address prefix-list NETWORK set community 65534:65534 route-map ALL permit 10 vrf context external-shared vni 555 rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn route-target import 65000:999000 route-target import 65000:999000 evpn address-family ipv6 unicast route-target both auto route-target both auto evpn route-target import 65000:999000 route-target import 65000:999000 evpn vrf context management vrf context tenant1 vni 999000 rd auto address-family ipv4 unicast route-target both auto route-target both auto evpn route-target import 65000:555 route-target import 65000:555 evpn address-family ipv6 unicast route-target both auto route-target both auto evpn route-target import 65000:555 route-target import 65000:555 evpn interface Vlan1 interface Vlan10 no shutdown vrf member tenant1 ip address 10.10.10.1/24 fabric forwarding mode anycast-gateway interface Vlan20 no shutdown vrf member tenant1 ip address 10.10.20.1/24 fabric forwarding mode anycast-gateway interface Vlan555 no shutdown vrf member external-shared ip forward ipv6 forward interface Vlan999 no shutdown vrf member tenant1 ip forward ipv6 forward interface nve1 no shutdown host-reachability protocol bgp source-interface loopback0 member vni 555 associate-vrf member vni 10000 ingress-replication protocol bgp member vni 20000 ingress-replication protocol bgp member vni 999000 associate-vrf interface Ethernet1/1 description to_r1 no switchport mtu 9202 ip address 10.0.13.2/24 no ip ospf passive-interface ip router ospf main_ospf area 0.0.0.0 no shutdown interface Ethernet1/2 description to_r2 no switchport mtu 9202 ip address 10.0.23.2/24 no ip ospf passive-interface ip router ospf main_ospf area 0.0.0.0 no shutdown interface Ethernet1/3 interface Ethernet1/4 interface Ethernet1/5 no switchport vrf member external-shared ip address 100.64.64.1/31 ipv6 address 2001::1/127 no shutdown interface Ethernet1/6 interface Ethernet1/7 interface Ethernet1/8 description to_Linux-5 switchport access vlan 20 interface Ethernet1/9 description to_Linux-3 switchport access vlan 10 interface Ethernet1/10 interface Ethernet1/11 interface Ethernet1/12 interface Ethernet1/13 interface Ethernet1/14 interface Ethernet1/15 interface Ethernet1/16 interface Ethernet1/17 interface Ethernet1/18 interface Ethernet1/19 interface Ethernet1/20 interface Ethernet1/21 interface Ethernet1/22 interface Ethernet1/23 interface Ethernet1/24 interface Ethernet1/25 interface Ethernet1/26 interface Ethernet1/27 interface Ethernet1/28 interface Ethernet1/29 interface Ethernet1/30 interface Ethernet1/31 interface Ethernet1/32 interface Ethernet1/33 interface Ethernet1/34 interface Ethernet1/35 interface Ethernet1/36 interface Ethernet1/37 interface Ethernet1/38 interface Ethernet1/39 interface Ethernet1/40 interface Ethernet1/41 interface Ethernet1/42 interface Ethernet1/43 interface Ethernet1/44 interface Ethernet1/45 interface Ethernet1/46 interface Ethernet1/47 interface Ethernet1/48 interface Ethernet1/49 interface Ethernet1/50 interface Ethernet1/51 interface Ethernet1/52 interface Ethernet1/53 interface Ethernet1/54 interface Ethernet1/55 interface Ethernet1/56 interface Ethernet1/57 interface Ethernet1/58 interface Ethernet1/59 interface Ethernet1/60 interface Ethernet1/61 interface Ethernet1/62 interface Ethernet1/63 interface Ethernet1/64 interface mgmt0 vrf member management ip address 10.0.0.2/24 interface loopback0 ip address 192.168.55.3/32 ip router ospf main_ospf area 0.0.0.0 icam monitor scale line console line vty boot nxos bootflash:/nxos64-cs.10.2.5.M.bin router ospf main_ospf router-id 192.168.55.3 passive-interface default router bgp 65000 address-family l2vpn evpn neighbor 192.168.99.1 remote-as 65000 update-source loopback0 address-family l2vpn evpn send-community send-community extended neighbor 192.168.99.2 remote-as 65000 update-source loopback0 address-family l2vpn evpn send-community send-community extended vrf external-shared address-family ipv4 unicast address-family ipv6 unicast neighbor 2001:: remote-as 42 address-family ipv6 unicast neighbor 100.64.64.0 remote-as 42 address-family ipv4 unicast route-map ADD-RT in vrf tenant1 address-family ipv4 unicast redistribute direct route-map ALL evpn vni 10000 l2 route-target import auto route-target export auto vni 20000 l2 route-target import auto route-target export auto no system default switchport shutdown