Symptoms or Error
Virtual Path is showing DEAD between the Branch SDWAN devices after adding a device as Secondary MCN and pushed the configuration.
You will see the following highlighted message getting incremented in ( SDWAN_Diagnostic logs ) :
Command: drop_counts
*****************************************************************************************************************************************
Index Count File Name Function Name Line # Reason text
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 3711 forward/virtual_path_service/crypto.c decrypted_receive_packet 1258 Failed to decrypt because path check fails
2 3205053 forward/hosted/ip_host_api.c iph_icmp_ttl_exceeded_handler 933 Received ICMP TTL exceeded, but not running trace route
3 10957819 forward/virtual_path_service/path.c store_reliable 3550 Free reliable packets on dead Virtual_Path
4 20108355 forward/virtual_path_service/path.c valid_receive_index 4344 Packet rwl and lwl does not match the Virtual_Path_id and path_num of the TRP packet
In the SDWAN_Diagnostic logs, you will also see that, the same IP address X.X.X.138 is mapped on both WANLLINK 1 and WANLINK 2:
*****************************************************************************************************************************************
Command: mcn_wl_table
*****************************************************************************************************************************************
Number of entries in table: 12
Number of entries in tree: 12
* Wan Link Wan Link Proxy IP UDP
ID Name Address Port
-------------------------------------------------------------------------
1 10 WANLINK 1 X.X..X.138 4980
1 11 WANLINK 2 0.0.0.0 4980
Paths dump:
----------------------------------------------------------------
Azure-France-Central-Slovakia-2-Barca-KB 0 X.X.X.4 X.X.X.138 4980 4980 Defau 172.16.2.1 0.0.0.0 READY NOT_READY 15 10 XXXX-WL-1 WANLINK 1 50 50 aes128 TRUE 0
Azure-France-Central-Slovakia-2-Barca-KB 1 X.X.X.4 X.X.X.138 4980 4980 Defau 172.16.2.1 0.0.0.0 READY NOT_READY 15 11 XXXX-WL-1 WANLINK 2 50 50 aes128 TRUE 0
**
You will see the following highlighted message getting incremented in ( SDWAN_Diagnostic logs ) :
Command: drop_counts
*****************************************************************************************************************************************
Index Count File Name Function Name Line # Reason text
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1 3711 forward/virtual_path_service/crypto.c decrypted_receive_packet 1258 Failed to decrypt because path check fails
2 3205053 forward/hosted/ip_host_api.c iph_icmp_ttl_exceeded_handler 933 Received ICMP TTL exceeded, but not running trace route
3 10957819 forward/virtual_path_service/path.c store_reliable 3550 Free reliable packets on dead Virtual_Path
4 20108355 forward/virtual_path_service/path.c valid_receive_index 4344 Packet rwl and lwl does not match the Virtual_Path_id and path_num of the TRP packet
In the SDWAN_Diagnostic logs, you will also see that, the same IP address X.X.X.138 is mapped on both WANLLINK 1 and WANLINK 2:
*****************************************************************************************************************************************
Command: mcn_wl_table
*****************************************************************************************************************************************
Number of entries in table: 12
Number of entries in tree: 12
* Wan Link Wan Link Proxy IP UDP
ID Name Address Port
-------------------------------------------------------------------------
1 10 WANLINK 1 X.X..X.138 4980
1 11 WANLINK 2 0.0.0.0 4980
Paths dump:
----------------------------------------------------------------
Azure-France-Central-Slovakia-2-Barca-KB 0 X.X.X.4 X.X.X.138 4980 4980 Defau 172.16.2.1 0.0.0.0 READY NOT_READY 15 10 XXXX-WL-1 WANLINK 1 50 50 aes128 TRUE 0
Azure-France-Central-Slovakia-2-Barca-KB 1 X.X.X.4 X.X.X.138 4980 4980 Defau 172.16.2.1 0.0.0.0 READY NOT_READY 15 11 XXXX-WL-1 WANLINK 2 50 50 aes128 TRUE 0
**
Solution
Upgrade to 10.0.4 to pickup the fix.
Problem Cause
Its a bug on SDWAN.
Root cause:
===========
When the SDWAN was added as SECONDARY MCN, the configuration was pushed to all the Sites. At that instant, the Branches were having issues in terms of recalculating the PATH IDs. While recalculating the PATHS ID, Branch 1 has mapped the same PUBLIC IP (X.X.X .138 ) on both the WAN LINKS WAN LINK 1 and WAN LINK 2 as shown in t. Because of which the PATHS are permanently showing DEAD.
Root cause:
===========
When the SDWAN was added as SECONDARY MCN, the configuration was pushed to all the Sites. At that instant, the Branches were having issues in terms of recalculating the PATH IDs. While recalculating the PATHS ID, Branch 1 has mapped the same PUBLIC IP (X.X.X .138 ) on both the WAN LINKS WAN LINK 1 and WAN LINK 2 as shown in t. Because of which the PATHS are permanently showing DEAD.