Phase 2 – Spoke to Spoke Design
DMVPN Phase 2 design introduced the ability for dynamic spoke-to-spoke tunnels without having the traffic go through the hub. Phase 2 improved on Phase 1 by allowing spokes to build a spoke-to-spoke tunnel on demand with these restrictions:
- Spokes must use mGRE (Multipoint GRE) tunnels
- Spokes must receive specific routes for all remote spoke subnets
- The next hop of the entry in the routing table must list the remote spoke as the next hop
Check these links for
- DMVPN basics – http://www.amolak.net/dmvpn-basics/
- DMVPN Phase 1 Configuration – http://www.amolak.net/dmvpn-phase-1/
Here is the network topology for DMVPN phase 2 discussion and configuration.
DMVPN Phase 2 configuration with EIGRP
EIGRP Split Horizon Rule – The split horizon rule prohibits a router from advertising a route through an interface that the router itself uses to reach the destination. In DMVPN, Hub router learn route(s) from one spoke via Tunnel0 interface and have to advertise it to other spoke via same Tunnel0 interface. Thus, in order for DMVPN to work in Phase 2 with EIGRP, split horizon must be disabled on the tunnel interface using the “no ip split-horizon eigrp <asn>” command.
Next-hop self in EIGRP – The next hop for all of the routes must point to the remote spoke. This is the key to triggering the generation of a spoke-to-spoke tunnel. To instruct EIGRP to use the received next hop rather than itself, use the “no ip next-hop-self eigrp <asn>” command.
Configuration:
---------- R1 - Hub: ---------- hostname R1 ! ip cef ! crypto isakmp policy 10 encr aes authentication pre-share group 2 crypto isakmp key DmVpNpR3$h@r3dK3Y address 0.0.0.0 ! ! crypto ipsec transform-set TRANSFORM-SET esp-aes esp-sha-hmac mode transport ! crypto ipsec profile PROTECT-GRE set transform-set TRANSFORM-SET ! interface Tunnel0 bandwidth 4096 ip address 192.168.1.1 255.255.255.0 no ip redirects ip mtu 1400 no ip next-hop-self eigrp 100 no ip split-horizon eigrp 100 ip nhrp authentication NhRp@UtH ip nhrp map multicast dynamic ip nhrp network-id 100 ip tcp adjust-mss 1360 tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 123 tunnel protection ipsec profile PROTECT-GRE ! interface FastEthernet0/0 ip address 1.1.14.1 255.255.255.0 ! interface FastEthernet0/1 ip address 10.10.1.1 255.255.255.0 ! router eigrp 100 network 10.10.1.1 0.0.0.0 network 192.168.1.1 0.0.0.0 ! ip route 0.0.0.0 0.0.0.0 1.1.14.4 ------------ R2 - Spoke: ------------ hostname R2 ! ip cef ! crypto isakmp policy 10 encr aes authentication pre-share group 2 crypto isakmp key DmVpNpR3$h@r3dK3Y address 0.0.0.0 ! ! crypto ipsec transform-set TRANSFORM-SET esp-aes esp-sha-hmac mode transport ! crypto ipsec profile PROTECT-GRE set transform-set TRANSFORM-SET ! interface Tunnel0 bandwidth 4096 ip address 192.168.1.2 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication NhRp@UtH ip nhrp map multicast 1.1.14.1 ip nhrp map 192.168.1.1 1.1.14.1 ip nhrp network-id 100 ip nhrp nhs 192.168.1.1 ip tcp adjust-mss 1360 tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 123 tunnel protection ipsec profile PROTECT-GRE ! interface FastEthernet0/0 ip address 1.1.24.2 255.255.255.0 ! interface FastEthernet0/1 ip address 10.10.2.2 255.255.255.0 ! router eigrp 100 network 10.10.2.2 0.0.0.0 network 192.168.1.2 0.0.0.0 ! ip route 0.0.0.0 0.0.0.0 1.1.24.4 ------------ R3 - Spoke: ------------ hostname R3 ! ip cef ! crypto isakmp policy 10 encr aes authentication pre-share group 2 crypto isakmp key DmVpNpR3$h@r3dK3Y address 0.0.0.0 ! ! crypto ipsec transform-set TRANSFORM-SET esp-aes esp-sha-hmac mode transport ! crypto ipsec profile PROTECT-GRE set transform-set TRANSFORM-SET ! interface Tunnel0 bandwidth 4096 ip address 192.168.1.3 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication NhRp@UtH ip nhrp map multicast 1.1.14.1 ip nhrp map 192.168.1.1 1.1.14.1 ip nhrp network-id 100 ip nhrp nhs 192.168.1.1 ip tcp adjust-mss 1360 tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 123 tunnel protection ipsec profile PROTECT-GRE ! interface FastEthernet0/0 ip address 1.1.34.3 255.255.255.0 ! interface FastEthernet0/1 ip address 10.10.3.3 255.255.255.0 ! router eigrp 100 network 10.10.3.3 0.0.0.0 network 192.168.1.3 0.0.0.0 ! ip route 0.0.0.0 0.0.0.0 1.1.34.4 --------------- R4 - Internet: --------------- hostname R4 ! interface FastEthernet0/0 ip address 1.1.14.4 255.255.255.0 ! interface FastEthernet0/1 ip address 1.1.24.4 255.255.255.0 ! interface FastEthernet1/0 ip address 1.1.34.4 255.255.255.0 !
Verification:
- Hub and Spokes routers are using mGRE tunnels R1#show int t0 Tunnel0 is up, line protocol is up Hardware is Tunnel Internet address is 192.168.1.1/24 MTU 17870 bytes, BW 4096 Kbit/sec, DLY 50000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation TUNNEL, loopback not set Keepalive not set Tunnel source 1.1.14.1 (FastEthernet0/0) Tunnel Subblocks: src-track: Tunnel0 source tracking subblock associated with FastEthernet0/0 Set of tunnels with source FastEthernet0/0, 1 member (includes iterators), on interface <OK> Tunnel protocol/transport multi-GRE/IP R2#show int t0 Tunnel0 is up, line protocol is up Hardware is Tunnel Internet address is 192.168.1.2/24 MTU 17870 bytes, BW 4096 Kbit/sec, DLY 50000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation TUNNEL, loopback not set Keepalive not set Tunnel source 1.1.24.2 (FastEthernet0/0) Tunnel Subblocks: src-track: Tunnel0 source tracking subblock associated with FastEthernet0/0 Set of tunnels with source FastEthernet0/0, 1 member (includes iterators), on interface <OK> Tunnel protocol/transport multi-GRE/IP R3#show int t0 Tunnel0 is up, line protocol is up Hardware is Tunnel Internet address is 192.168.1.3/24 MTU 17870 bytes, BW 4096 Kbit/sec, DLY 50000 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation TUNNEL, loopback not set Keepalive not set Tunnel source 1.1.34.3 (FastEthernet0/0) Tunnel Subblocks: src-track: Tunnel0 source tracking subblock associated with FastEthernet0/0 Set of tunnels with source FastEthernet0/0, 1 member (includes iterators), on interface <OK> Tunnel protocol/transport multi-GRE/IP - Routing adjacency (EIGRP neighborship) is between Hub and Spokes only R1#show ip eigrp neighbors EIGRP-IPv4 Neighbors for AS(100) H Address Interface Hold Uptime SRTT RTO Q Seq (sec) (ms) Cnt Num 1 192.168.1.2 Tu0 13 05:09:22 201 1206 0 3 0 192.168.1.3 Tu0 13 05:09:32 167 1002 0 4 R2#show ip eigrp neighbors EIGRP-IPv4 Neighbors for AS(100) H Address Interface Hold Uptime SRTT RTO Q Seq (sec) (ms) Cnt Num 0 192.168.1.1 Tu0 13 05:09:37 140 840 0 6 R3#show ip eigrp neighbors EIGRP-IPv4 Neighbors for AS(100) H Address Interface Hold Uptime SRTT RTO Q Seq (sec) (ms) Cnt Num 0 192.168.1.1 Tu0 11 05:09:49 156 936 0 6 - Routing Table (see next-hop IP on spoke routers to reach other spoke subnets) R1#show ip route eigrp | beg Gate Gateway of last resort is 1.1.14.4 to network 0.0.0.0 10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks D 10.10.2.0/24 [90/1907456] via 192.168.1.2, 05:11:20, Tunnel0 D 10.10.3.0/24 [90/1907456] via 192.168.1.3, 05:11:30, Tunnel0 R2#show ip route eigrp | beg Gate Gateway of last resort is 1.1.24.4 to network 0.0.0.0 10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks D 10.10.1.0/24 [90/1907456] via 192.168.1.1, 05:11:31, Tunnel0 D 10.10.3.0/24 [90/3187456] via 192.168.1.3, 05:11:31, Tunnel0 R3#show ip route eigrp | beg Gate Gateway of last resort is 1.1.34.4 to network 0.0.0.0 10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks D 10.10.1.0/24 [90/1907456] via 192.168.1.1, 05:11:42, Tunnel0 D 10.10.2.0/24 [90/3187456] via 192.168.1.2, 05:11:32, Tunnel0 - DMVPN Table - Hub shows dynamic entries, spokes registered themselves with hub - Spokes show static entries, static entry defined for hub R1#show dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting UpDn Time --> Up or Down Time for a Tunnel ====================================================================== Interface: Tunnel0, IPv4 NHRP Details Type:Hub, NHRP Peers:2, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 1.1.24.2 192.168.1.2 UP 00:51:08 D 1 1.1.34.3 192.168.1.3 UP 00:51:08 D R2#show dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting UpDn Time --> Up or Down Time for a Tunnel ====================================================================== Interface: Tunnel0, IPv4 NHRP Details Type:Spoke, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 1.1.14.1 192.168.1.1 UP 00:51:28 S R3#show dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting UpDn Time --> Up or Down Time for a Tunnel ====================================================================== Interface: Tunnel0, IPv4 NHRP Details Type:Spoke, NHRP Peers:1, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 1.1.14.1 192.168.1.1 UP 00:51:30 S - Traffic between Spoke R2 and Spoke R3 Routing entry on R2 shows R3 subnet (10.10.3.0/24) is reachable via next-hop IP 192.168.1.3 (R3 Tunnel IP) D 10.10.3.0/24 [90/3187456] via 192.168.1.3, 00:05:12, Tunnel0 Lets check CEF adjacency for next-hop IP 192.168.1.3 R2#show adjacency 192.168.1.3 Protocol Interface Address IP Tunnel0 192.168.1.3(5) (incomplete) The adjacency is showing incomplete. The incomplete adjacency triggers a CEF punt to the CPU for further processing (to resolve the address). R2#show ip cef 192.168.1.3 internal 192.168.1.0/24, epoch 0, flags attached, connected, cover dependents, need deagg, RIB[C], refcount 5, per-destination sharing sources: RIB feature space: IPRM: 0x0003800C subblocks: gsb Connected chain head(1): 0x6A36ADFC Covered dependent prefixes: 3 need deagg: 2 notify cover updated: 1 ifnums: Tunnel0(6) path 695C0994, path list 6A90BBB8, share 1/1, type connected prefix, for IPv4 connected to Tunnel0, adjacency punt output chain: punt This causes R2(Spoke) to send a resolution request to R1(Hub) for Spoke-R3’s NBMA address. The request gets forwarded from R1(Hub) to Spoke-R3. Spoke-R3 replies directly to Spoke-R2 with its mapping information. During this process, R2(Spoke) will send the actual data packet to R1(Hub) to be delivered to R3(Spoke) as a last-ditch effort to not drop the traffic. The first traceroute will look as below, the traffic traverse via hub. R2#traceroute 10.10.3.3 so 10.10.2.2 Type escape sequence to abort. Tracing the route to 10.10.3.3 VRF info: (vrf in name/id, vrf out name/id) 1 192.168.1.1 180 msec 128 msec 152 msec << Hub R1 Tunnel IP 2 192.168.1.3 252 msec - Debug nhrp packet shows following events: Spoke-R2 sends resolution request via Tunnel0, it will reach to Hub-R1 as this is the only active tunnel on R2 at this moment. R2# *Mar 1 00:29:49.567: NHRP: Send Resolution Request via Tunnel0 vrf 0, packet size: 88 *Mar 1 00:29:49.575: src: 192.168.1.2, dst: 192.168.1.3 *Mar 1 00:29:49.583: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1 *Mar 1 00:29:49.583: shtl: 4(NSAP), sstl: 0(NSAP) *Mar 1 00:29:49.587: pktsz: 88 extoff: 52 *Mar 1 00:29:49.591: (M) flags: "router auth src-stable nat ", reqid: 2 *Mar 1 00:29:49.595: src NBMA: 1.1.24.2 *Mar 1 00:29:49.595: src protocol: 192.168.1.2, dst protocol: 192.168.1.3 *Mar 1 00:29:49.603: (C-1) code: no error(0) *Mar 1 00:29:49.603: prefix: 32, mtu: 17870, hd_time: 7200 *Mar 1 00:29:49.603: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0 This request arrives at Hub-R1. Hub-R1 finds it is destined for destination IP 192.168.1.3 and Hub-R1 will forward it to Spoke-R3. R1# *Mar 1 00:29:49.683: NHRP: Receive Resolution Request via Tunnel0 vrf 0, packet size: 88 *Mar 1 00:29:49.687: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1 *Mar 1 00:29:49.691: shtl: 4(NSAP), sstl: 0(NSAP) *Mar 1 00:29:49.695: pktsz: 88 extoff: 52 *Mar 1 00:29:49.695: (M) flags: "router auth src-stable nat ", reqid: 2 *Mar 1 00:29:49.699: src NBMA: 1.1.24.2 *Mar 1 00:29:49.703: src protocol: 192.168.1.2, dst protocol: 192.168.1.3 *Mar 1 00:29:49.707: (C-1) code: no error(0) *Mar 1 00:29:49.707: prefix: 32, mtu: 17870, hd_time: 7200 *Mar 1 00:29:49.707: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0 *Mar 1 00:29:49.715: NHRP: Forwarding Resolution Request via Tunnel0 vrf 0, packet size: 108 *Mar 1 00:29:49.719: src: 192.168.1.1, dst: 192.168.1.3 *Mar 1 00:29:49.723: (F) afn: AF_IP(1), type: IP(800), hop: 254, ver: 1 *Mar 1 00:29:49.723: shtl: 4(NSAP), sstl: 0(NSAP) *Mar 1 00:29:49.723: R1# pktsz: 108 extoff: 52 *Mar 1 00:29:49.723: (M) flags: "router auth src-stable nat ", reqid: 2 *Mar 1 00:29:49.723: src NBMA: 1.1.24.2 *Mar 1 00:29:49.723: src protocol: 192.168.1.2, dst protocol: 192.168.1.3 *Mar 1 00:29:49.723: (C-1) code: no error(0) *Mar 1 00:29:49.723: prefix: 32, mtu: 17870, hd_time: 7200 *Mar 1 00:29:49.723: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0 Then Spoke-R3 receives NHRP resolution request. This request contains Spoke-R2's details including NBMA and Tunnel IP of Spoke-R2. Spoke-R3 would initiate a direct DMVPN tunnel to R2 and send reply to R2. R3# *Mar 1 00:29:49.671: NHRP: Receive Resolution Request via Tunnel0 vrf 0, packet size: 108 *Mar 1 00:29:49.675: (F) afn: AF_IP(1), type: IP(800), hop: 254, ver: 1 *Mar 1 00:29:49.675: shtl: 4(NSAP), sstl: 0(NSAP) *Mar 1 00:29:49.675: pktsz: 108 extoff: 52 *Mar 1 00:29:49.675: (M) flags: "router auth src-stable nat ", reqid: 2 *Mar 1 00:29:49.675: src NBMA: 1.1.24.2 *Mar 1 00:29:49.675: src protocol: 192.168.1.2, dst protocol: 192.168.1.3 *Mar 1 00:29:49.675: (C-1) code: no error(0) *Mar 1 00:29:49.675: prefix: 32, mtu: 17870, hd_time: 7200 *Mar 1 00:29:49.675: addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 0 *Mar 1 00:29:50.487: NHRP: Send Resolution Reply via Tunnel0 vrf 0, packet size: 136 *Mar 1 00:29:50.487: src: 192.168.1.3, dst: 192.168.1.2 *Mar 1 00:29:50.487: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1 *Mar 1 00:29:50.487: shtl: 4(NSAP), sstl: 0(NSAP) *Mar 1 00:29:50.487: pktsz R3#: 136 extoff: 60 *Mar 1 00:29:50.487: (M) flags: "router auth dst-stable unique src-stable nat ", reqid: 2 *Mar 1 00:29:50.487: src NBMA: 1.1.24.2 *Mar 1 00:29:50.487: src protocol: 192.168.1.2, dst protocol: 192.168.1.3 *Mar 1 00:29:50.487: (C-1) code: no error(0) *Mar 1 00:29:50.487: prefix: 32, mtu: 17870, hd_time: 7200 *Mar 1 00:29:50.487: addr_len: 4(NSAP), subaddr_len: 0(NSAP), proto_len: 4, pref: 0 *Mar 1 00:29:50.487: client NBMA: 1.1.34.3 *Mar 1 00:29:50.487: client protocol: 192.168.1.3 R2 receives reply and at this moment both R2 and R3 knows each other's NBMA IP and have direct dmvpn tunnel. R2# *Mar 1 00:29:50.635: NHRP: Receive Resolution Reply via Tunnel0 vrf 0, packet size: 136 *Mar 1 00:29:50.643: (F) afn: AF_IP(1), type: IP(800), hop: 255, ver: 1 *Mar 1 00:29:50.647: shtl: 4(NSAP), sstl: 0(NSAP) *Mar 1 00:29:50.647: pktsz: 136 extoff: 60 *Mar 1 00:29:50.651: (M) flags: "router auth dst-stable unique src-stable nat ", reqid: 2 *Mar 1 00:29:50.655: src NBMA: 1.1.24.2 *Mar 1 00:29:50.655: src protocol: 192.168.1.2, dst protocol: 192.168.1.3 *Mar 1 00:29:50.663: (C-1) code: no error(0) *Mar 1 00:29:50.663: prefix: 32, mtu: 17870, hd_time: 7200 *Mar 1 00:29:50.663: addr_len: 4(NSAP), subaddr_len: 0(NSAP), proto_len: 4, pref: 0 *Mar 1 00:29:50.663: client NBMA: 1.1.34.3 *Mar 1 00:29:50.663: client protocol: 192.168.1.3 * 140 msec - Check DMVPN status now R1#show dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting UpDn Time --> Up or Down Time for a Tunnel ===================================================================== Interface: Tunnel0, IPv4 NHRP Details Type:Hub, NHRP Peers:2, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 1.1.24.2 192.168.1.2 UP 05:15:09 D 1 1.1.34.3 192.168.1.3 UP 05:15:19 D R2#show dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting UpDn Time --> Up or Down Time for a Tunnel ====================================================================== Interface: Tunnel0, IPv4 NHRP Details Type:Spoke, NHRP Peers:2, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 1.1.14.1 192.168.1.1 UP 05:15:26 S 1 1.1.34.3 192.168.1.3 UP 00:34:02 D R3#show dmvpn Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete N - NATed, L - Local, X - No Socket # Ent --> Number of NHRP entries with same NBMA peer NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting UpDn Time --> Up or Down Time for a Tunnel ====================================================================== Interface: Tunnel0, IPv4 NHRP Details Type:Spoke, NHRP Peers:2, # Ent Peer NBMA Addr Peer Tunnel Add State UpDn Tm Attrb ----- --------------- --------------- ----- -------- ----- 1 1.1.14.1 192.168.1.1 UP 05:15:51 S 1 1.1.24.2 192.168.1.2 UP 00:34:17 D - NHRP State R1#show ip nhrp 192.168.1.2/32 via 192.168.1.2 Tunnel0 created 05:16:06, expire 01:23:19 Type: dynamic, Flags: unique registered used NBMA address: 1.1.24.2 192.168.1.3/32 via 192.168.1.3 Tunnel0 created 05:16:16, expire 01:23:19 Type: dynamic, Flags: unique registered used NBMA address: 1.1.34.3 R2#show ip nhrp 192.168.1.1/32 via 192.168.1.1 Tunnel0 created 05:17:28, never expire Type: static, Flags: used NBMA address: 1.1.14.1 192.168.1.2/32 via 192.168.1.2 Tunnel0 created 00:35:19, expire 01:24:40 Type: dynamic, Flags: router unique local NBMA address: 1.1.24.2 (no-socket) 192.168.1.3/32 via 192.168.1.3 Tunnel0 created 00:35:20, expire 01:24:40 Type: dynamic, Flags: router used NBMA address: 1.1.34.3 R3#show ip nhrp 192.168.1.1/32 via 192.168.1.1 Tunnel0 created 05:17:14, never expire Type: static, Flags: used NBMA address: 1.1.14.1 192.168.1.2/32 via 192.168.1.2 Tunnel0 created 00:35:06, expire 01:24:34 Type: dynamic, Flags: router used NBMA address: 1.1.24.2 192.168.1.3/32 via 192.168.1.3 Tunnel0 created 00:35:05, expire 01:24:34 Type: dynamic, Flags: router unique local NBMA address: 1.1.34.3 (no-socket) - Ping and traceroute between Spoke-R2 and Spoke-R3 - Now traffic is going through direct spoke-to-spoke tunnel R2#ping 10.10.3.3 so 10.10.2.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.10.3.3, timeout is 2 seconds: Packet sent with a source address of 10.10.2.2 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 116/128/136 ms R2#trace 10.10.3.3 so 10.10.2.2 Type escape sequence to abort. Tracing the route to 10.10.3.3 VRF info: (vrf in name/id, vrf out name/id) 1 192.168.1.3 156 msec * 144 msec << R3 Tunnel0 IP - CEF would look as below after direct spoke-to-spoke DMVPN tunnel creation R2#show adjacency 192.168.1.3 Protocol Interface Address IP Tunnel0 192.168.1.3(11) R2#show ip cef 192.168.1.3 internal 192.168.1.3/32, epoch 0, flags attached, refcount 5, per-destination sharing sources: Adj subblocks: Adj source: IP midchain out of Tunnel0, addr 192.168.1.3 6AFD37A0 Dependent covered prefix type adjfib, cover 192.168.1.0/24 ifnums: Tunnel0(6): 192.168.1.3 path 6AFD6068, path list 6AFD78A0, share 1/1, type adjacency prefix, for IPv4 attached to Tunnel0, adjacency IP midchain out of Tunnel0, addr 192.168.1.3 6AFD37A0 output chain: IP midchain out of Tunnel0, addr 192.168.1.3 6AFD37A0 IP adj out of FastEthernet0/0, addr 1.1.24.4 69237240 R2#
DMVPN Phase 2 configuration with OSPF
- Configure Hub router as DR. If there is 2nd hub for redundancy, 2nd hub will be BDR.
- Configure Spokes routers with ospf priority as ZERO so that they should not participate in DR election and always act as DROTHER.
- All spoke routers will have OSPF adjacencies with DR and BDR only.
Configuration:
--------- R1 - Hub: --------- hostname R1 ! ip cef ! crypto isakmp policy 10 encr aes authentication pre-share group 2 crypto isakmp key DmVpNpR3$h@r3dK3Y address 0.0.0.0 ! ! crypto ipsec transform-set TRANSFORM-SET esp-aes esp-sha-hmac mode transport ! crypto ipsec profile PROTECT-GRE set transform-set TRANSFORM-SET ! interface Tunnel0 bandwidth 4096 ip address 192.168.1.1 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication NhRp@UtH ip nhrp map multicast dynamic ip nhrp network-id 100 ip tcp adjust-mss 1360 ip ospf network broadcast ip ospf priority 255 tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 123 tunnel protection ipsec profile PROTECT-GRE ! interface FastEthernet0/0 ip address 1.1.14.1 255.255.255.0 ! interface FastEthernet0/1 ip address 10.10.1.1 255.255.255.0 ! router ospf 1 network 10.10.1.1 0.0.0.0 area 0 network 192.168.1.1 0.0.0.0 area 0 ! ip route 0.0.0.0 0.0.0.0 1.1.14.4 ------------ R2 - Spoke: ------------ hostname R2 ! ip cef ! crypto isakmp policy 10 encr aes authentication pre-share group 2 crypto isakmp key DmVpNpR3$h@r3dK3Y address 0.0.0.0 ! ! crypto ipsec transform-set TRANSFORM-SET esp-aes esp-sha-hmac mode transport ! crypto ipsec profile PROTECT-GRE set transform-set TRANSFORM-SET ! interface Tunnel0 bandwidth 4096 ip address 192.168.1.2 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication NhRp@UtH ip nhrp map multicast 1.1.14.1 ip nhrp map 192.168.1.1 1.1.14.1 ip nhrp network-id 100 ip nhrp nhs 192.168.1.1 ip tcp adjust-mss 1360 ip ospf network broadcast ip ospf priority 0 tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 123 tunnel protection ipsec profile PROTECT-GRE ! interface FastEthernet0/0 ip address 1.1.24.2 255.255.255.0 ! interface FastEthernet0/1 ip address 10.10.2.2 255.255.255.0 ! router ospf 1 network 10.10.2.2 0.0.0.0 area 0 network 192.168.1.2 0.0.0.0 area 0 ! ip route 0.0.0.0 0.0.0.0 1.1.24.4 ------------ R3 - Spoke: ------------ hostname R3 ! ip cef ! crypto isakmp policy 10 encr aes authentication pre-share group 2 crypto isakmp key DmVpNpR3$h@r3dK3Y address 0.0.0.0 ! ! crypto ipsec transform-set TRANSFORM-SET esp-aes esp-sha-hmac mode transport ! crypto ipsec profile PROTECT-GRE set transform-set TRANSFORM-SET ! interface Tunnel0 bandwidth 4096 ip address 192.168.1.3 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication NhRp@UtH ip nhrp map multicast 1.1.14.1 ip nhrp map 192.168.1.1 1.1.14.1 ip nhrp network-id 100 ip nhrp nhs 192.168.1.1 ip tcp adjust-mss 1360 ip ospf network broadcast ip ospf priority 0 tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 123 tunnel protection ipsec profile PROTECT-GRE ! interface FastEthernet0/0 ip address 1.1.34.3 255.255.255.0 ! interface FastEthernet0/1 ip address 10.10.3.3 255.255.255.0 ! router ospf 1 network 10.10.3.3 0.0.0.0 area 0 network 192.168.1.3 0.0.0.0 area 0 ! ip route 0.0.0.0 0.0.0.0 1.1.34.4 !
Verification:
- OSPF Tunnel Network Type R1#show ip ospf int t0 Tunnel0 is up, line protocol is up Internet Address 192.168.1.1/24, Area 0, Attached via Network Statement Process ID 1, Router ID 192.168.1.1, Network Type BROADCAST, Cost: 24 Topology-MTID Cost Disabled Shutdown Topology Name 0 24 no no Base Transmit Delay is 1 sec, State DR, Priority 255 Designated Router (ID) 192.168.1.1, Interface address 192.168.1.1 No backup designated router on this network Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5 oob-resync timeout 40 Hello due in 00:00:04 Supports Link-local Signaling (LLS) Cisco NSF helper support enabled IETF NSF helper support enabled Index 2/2, flood queue length 0 Next 0x0(0)/0x0(0) Last flood scan length is 1, maximum is 1 Last flood scan time is 4 msec, maximum is 4 msec Neighbor Count is 2, Adjacent neighbor count is 2 Adjacent with neighbor 192.168.1.2 Adjacent with neighbor 192.168.1.3 Suppress hello for 0 neighbor(s) R2#show ip ospf int t0 Tunnel0 is up, line protocol is up Internet Address 192.168.1.2/24, Area 0, Attached via Network Statement Process ID 1, Router ID 192.168.1.2, Network Type BROADCAST, Cost: 24 Topology-MTID Cost Disabled Shutdown Topology Name 0 24 no no Base Transmit Delay is 1 sec, State DROTHER, Priority 0 Designated Router (ID) 192.168.1.1, Interface address 192.168.1.1 No backup designated router on this network Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5 oob-resync timeout 40 Hello due in 00:00:01 Supports Link-local Signaling (LLS) Cisco NSF helper support enabled IETF NSF helper support enabled Index 2/2, flood queue length 0 Next 0x0(0)/0x0(0) Last flood scan length is 1, maximum is 1 Last flood scan time is 4 msec, maximum is 4 msec Neighbor Count is 1, Adjacent neighbor count is 1 Adjacent with neighbor 192.168.1.1 (Designated Router) Suppress hello for 0 neighbor(s) R3#show ip ospf int t0 Tunnel0 is up, line protocol is up Internet Address 192.168.1.3/24, Area 0, Attached via Network Statement Process ID 1, Router ID 192.168.1.3, Network Type BROADCAST, Cost: 24 Topology-MTID Cost Disabled Shutdown Topology Name 0 24 no no Base Transmit Delay is 1 sec, State DROTHER, Priority 0 Designated Router (ID) 192.168.1.1, Interface address 192.168.1.1 No backup designated router on this network Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5 oob-resync timeout 40 Hello due in 00:00:07 Supports Link-local Signaling (LLS) Cisco NSF helper support enabled IETF NSF helper support enabled Index 2/2, flood queue length 0 Next 0x0(0)/0x0(0) Last flood scan length is 1, maximum is 1 Last flood scan time is 4 msec, maximum is 4 msec Neighbor Count is 1, Adjacent neighbor count is 1 Adjacent with neighbor 192.168.1.1 (Designated Router) Suppress hello for 0 neighbor(s) - OSPF Adjacencies R1#show ip ospf neighbor Neighbor ID Pri State Dead Time Address Interface 192.168.1.2 0 FULL/DROTHER 00:00:32 192.168.1.2 Tunnel0 192.168.1.3 0 FULL/DROTHER 00:00:32 192.168.1.3 Tunnel0 R2#show ip ospf neighbor Neighbor ID Pri State Dead Time Address Interface 192.168.1.1 255 FULL/DR 00:00:31 192.168.1.1 Tunnel0 R3#show ip ospf neighbor Neighbor ID Pri State Dead Time Address Interface 192.168.1.1 255 FULL/DR 00:00:39 192.168.1.1 Tunnel0 - Routing Table R1#show ip route ospf | beg Gate Gateway of last resort is 1.1.14.4 to network 0.0.0.0 10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks O 10.10.2.0/24 [110/25] via 192.168.1.2, 00:25:23, Tunnel0 O 10.10.3.0/24 [110/25] via 192.168.1.3, 00:25:23, Tunnel0 R2#show ip route ospf | beg Gate Gateway of last resort is 1.1.24.4 to network 0.0.0.0 10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks O 10.10.1.0/24 [110/25] via 192.168.1.1, 00:25:28, Tunnel0 O 10.10.3.0/24 [110/25] via 192.168.1.3, 00:25:18, Tunnel0 R3#show ip route ospf | beg Gate Gateway of last resort is 1.1.34.4 to network 0.0.0.0 10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks O 10.10.1.0/24 [110/25] via 192.168.1.1, 00:25:30, Tunnel0 O 10.10.2.0/24 [110/25] via 192.168.1.2, 00:25:30, Tunnel0 - Traffic between Spoke R2 and Spoke R3 R2#ping 10.10.3.3 so 10.10.2.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.10.3.3, timeout is 2 seconds: Packet sent with a source address of 10.10.2.2 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 116/123/128 ms R2#trace 10.10.3.3 so 10.10.2.2 Type escape sequence to abort. Tracing the route to 10.10.3.3 VRF info: (vrf in name/id, vrf out name/id) 1 192.168.1.3 144 msec * 128 msec << R3 Tunnel IP
DMVPN Phase 2 configuration with BGP
- Hub router would be BGP Route-Reflector server
- Spoke routers would be BGP Route-Reflector clients
- All tunnels would be mGRE
- You can use eBGP also, where Hub and Spoke routers could be configured in different autonomous system number (ASN). Here we will use iBGP only.
----------- R1 - Hub: ----------- hostname R1 ! ip cef ! crypto isakmp policy 10 encr aes authentication pre-share group 2 crypto isakmp key DmVpNpR3$h@r3dK3Y address 0.0.0.0 ! ! crypto ipsec transform-set TRANSFORM-SET esp-aes esp-sha-hmac mode transport ! crypto ipsec profile PROTECT-GRE set transform-set TRANSFORM-SET ! interface Tunnel0 bandwidth 4096 ip address 192.168.1.1 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication NhRp@UtH ip nhrp map multicast dynamic ip nhrp network-id 100 ip tcp adjust-mss 1360 tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 123 tunnel protection ipsec profile PROTECT-GRE ! interface FastEthernet0/0 ip address 1.1.14.1 255.255.255.0 ! interface FastEthernet0/1 ip address 10.10.1.1 255.255.255.0 ! router bgp 65001 bgp router-id 192.168.1.1 bgp log-neighbor-changes bgp listen range 192.168.1.0/24 peer-group DMVPN-SPOKES bgp listen limit 50 network 10.10.1.0 mask 255.255.255.0 neighbor DMVPN-SPOKES peer-group neighbor DMVPN-SPOKES remote-as 65001 neighbor DMVPN-SPOKES route-reflector-client ! ip route 0.0.0.0 0.0.0.0 1.1.14.4 ------------ R2 - Spoke: ------------ hostname R2 ! ip cef ! crypto isakmp policy 10 encr aes authentication pre-share group 2 crypto isakmp key DmVpNpR3$h@r3dK3Y address 0.0.0.0 ! ! crypto ipsec transform-set TRANSFORM-SET esp-aes esp-sha-hmac mode transport ! crypto ipsec profile PROTECT-GRE set transform-set TRANSFORM-SET ! interface Tunnel0 bandwidth 4096 ip address 192.168.1.2 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication NhRp@UtH ip nhrp map multicast 1.1.14.1 ip nhrp map 192.168.1.1 1.1.14.1 ip nhrp network-id 100 ip nhrp nhs 192.168.1.1 ip tcp adjust-mss 1360 tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 123 tunnel protection ipsec profile PROTECT-GRE ! interface FastEthernet0/0 ip address 1.1.24.2 255.255.255.0 ! interface FastEthernet0/1 ip address 10.10.2.2 255.255.255.0 ! router bgp 65001 bgp router-id 192.168.1.2 bgp log-neighbor-changes network 10.10.2.0 mask 255.255.255.0 neighbor 192.168.1.1 remote-as 65001 ! ip route 0.0.0.0 0.0.0.0 1.1.24.4 ------------ R3 - Spoke: ------------ hostname R3 ! ip cef ! crypto isakmp policy 10 encr aes authentication pre-share group 2 crypto isakmp key DmVpNpR3$h@r3dK3Y address 0.0.0.0 ! ! crypto ipsec transform-set TRANSFORM-SET esp-aes esp-sha-hmac mode transport ! crypto ipsec profile PROTECT-GRE set transform-set TRANSFORM-SET ! interface Tunnel0 bandwidth 4096 ip address 192.168.1.3 255.255.255.0 no ip redirects ip mtu 1400 ip nhrp authentication NhRp@UtH ip nhrp map multicast 1.1.14.1 ip nhrp map 192.168.1.1 1.1.14.1 ip nhrp network-id 100 ip nhrp nhs 192.168.1.1 ip tcp adjust-mss 1360 tunnel source FastEthernet0/0 tunnel mode gre multipoint tunnel key 123 tunnel protection ipsec profile PROTECT-GRE ! interface FastEthernet0/0 ip address 1.1.34.3 255.255.255.0 ! interface FastEthernet0/1 ip address 10.10.3.3 255.255.255.0 ! router bgp 65001 bgp router-id 192.168.1.3 bgp log-neighbor-changes network 10.10.3.0 mask 255.255.255.0 neighbor 192.168.1.1 remote-as 65001 ! ip route 0.0.0.0 0.0.0.0 1.1.34.4
Verification:
- BGP Neighborship between Hub and Spokes only R1#show ip bgp sum BGP router identifier 192.168.1.1, local AS number 65001 BGP table version is 4, main routing table version 4 3 network entries using 444 bytes of memory 3 path entries using 192 bytes of memory 2/2 BGP path/bestpath attribute entries using 272 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 908 total bytes of memory BGP activity 3/0 prefixes, 3/0 paths, scan interval 60 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd *192.168.1.2 4 65001 11 15 4 0 0 00:06:11 1 *192.168.1.3 4 65001 12 13 4 0 0 00:06:21 1 * Dynamically created based on a listen range command Dynamically created neighbors: 2, Subnet ranges: 1 BGP peergroup DMVPN-SPOKES listen range group members: 192.168.1.0/24 Total dynamically created neighbors: 2/(50 max), Subnet ranges: 1 R2#show ip bgp sum BGP router identifier 192.168.1.2, local AS number 65001 BGP table version is 4, main routing table version 4 3 network entries using 444 bytes of memory 3 path entries using 192 bytes of memory 2/2 BGP path/bestpath attribute entries using 272 bytes of memory 1 BGP rrinfo entries using 24 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 932 total bytes of memory BGP activity 3/0 prefixes, 3/0 paths, scan interval 60 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 192.168.1.1 4 65001 15 11 4 0 0 00:06:17 2 R3#show ip bgp sum BGP router identifier 192.168.1.3, local AS number 65001 BGP table version is 4, main routing table version 4 3 network entries using 444 bytes of memory 3 path entries using 192 bytes of memory 2/2 BGP path/bestpath attribute entries using 272 bytes of memory 1 BGP rrinfo entries using 24 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 932 total bytes of memory BGP activity 3/0 prefixes, 3/0 paths, scan interval 60 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 192.168.1.1 4 65001 13 12 4 0 0 00:06:29 2 - BGP Table R1#show ip bgp BGP table version is 4, local router ID is 192.168.1.1 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path *> 10.10.1.0/24 0.0.0.0 0 32768 i *>i 10.10.2.0/24 192.168.1.2 0 100 0 i *>i 10.10.3.0/24 192.168.1.3 0 100 0 i R2#show ip bgp BGP table version is 4, local router ID is 192.168.1.2 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path *>i 10.10.1.0/24 192.168.1.1 0 100 0 i *> 10.10.2.0/24 0.0.0.0 0 32768 i *>i 10.10.3.0/24 192.168.1.3 0 100 0 i R3#show ip bgp BGP table version is 4, local router ID is 192.168.1.3 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found Network Next Hop Metric LocPrf Weight Path *>i 10.10.1.0/24 192.168.1.1 0 100 0 i *>i 10.10.2.0/24 192.168.1.2 0 100 0 i *> 10.10.3.0/24 0.0.0.0 0 32768 i - Routing Table R1#show ip route bgp | beg Gate Gateway of last resort is 1.1.14.4 to network 0.0.0.0 10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks B 10.10.2.0/24 [200/0] via 192.168.1.2, 00:07:09 B 10.10.3.0/24 [200/0] via 192.168.1.3, 00:08:11 R2#show ip route bgp | beg Gate Gateway of last resort is 1.1.24.4 to network 0.0.0.0 10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks B 10.10.1.0/24 [200/0] via 192.168.1.1, 00:07:16 B 10.10.3.0/24 [200/0] via 192.168.1.3, 00:07:16 R3#show ip route bgp | beg Gate Gateway of last resort is 1.1.34.4 to network 0.0.0.0 10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks B 10.10.1.0/24 [200/0] via 192.168.1.1, 00:08:20 B 10.10.2.0/24 [200/0] via 192.168.1.2, 00:07:18 - Traffic between Spoke R2 and Spoke R3 R2#ping 10.10.3.3 so 10.10.2.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.10.3.3, timeout is 2 seconds: Packet sent with a source address of 10.10.2.2 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 108/120/128 ms R2#trace 10.10.3.3 so 10.10.2.2 Type escape sequence to abort. Tracing the route to 10.10.3.3 VRF info: (vrf in name/id, vrf out name/id) 1 192.168.1.3 132 msec * 120 msec << R3 Tunnel IP
DMVPN Phase 2 deployment provides direct spoke-to-spoke tunnels, but one of the limitations is maintaining full routing tables on the spokes. Each route for remote spoke networks needs to be a specific route with the next hop pointing to the remote spoke’s tunnel address. This prevents the hub from being able to send down a summarized route to the spokes for a more concise routing table. This limitation is addressed in DMVPN Phase 3 design, which we will discuss later.
