MTU differences on platforms

Problem statement

Various routing platforms differently interpret meaning of an MTU value.

IOS XR

Interface confugured as below:

interface GigabitEthernet0/0/0/2
 description to_sw3
 mtu 9216
 ipv4 address 10.0.13.1 255.255.255.0

Interface information is stored in the shared memory database of the interface manager (IM). Resulting MTU values:

RP/0/RP0/CPU0:r1#show im database interface GigabitEthernet0/0/0/2
Wed Nov  1 21:39:35.430 UTC

View: OWN - Owner, L3P - Local 3rd Party, G3P - Global 3rd Party, LDP - Local Data Plane
      GDP - Global Data Plane, RED - Redundancy, UL - UL

Node 0/0/CPU0 (0x10c0)

Interface GigabitEthernet0/0/0/2, ifh 0x01000028 (up, 9216)
  Interface flags:          0x000000000110059f (ROOT_IS_HW|IFCONNECTOR
                            |IFINDEX|SUP_NAMED_SUB|BROADCAST|CONFIG|HW|VIS
                            |DATA|CONTROL)
  Encapsulation:            ether
  Interface type:           IFT_GETHERNET
  Control parent:           None
  Data parent:              None
  Views:                    GDP|LDP|L3P|OWN

  Protocol        Caps (state, mtu)
  --------        -----------------
  None            ether (up, 9216)
  arp             arp (up, 9202)
  ipv4            ipv4 (up, 9202)
  ether_sock      ether_sock (up, 9202)

NX-OS

Interface configured as below:

interface Ethernet1/1
  description to_r1
  no switchport
  mtu 9202
  ip address 10.0.13.2/24
  no ip ospf passive-interface
  ip router ospf main_osfp area 0.0.0.0
  no shutdown

Resulting MTU value for that L3 interface shown as 9202, please note that is L3 level MTU and thus L2 Ethernet MTU is 9216 (like in the example above for IOS XR)

sw3# show system internal ethpm info interface eth 1/1

Ethernet1/1 - if_index: 0x1A000000
Backplane MAC address: 0c:36:00:00:01:01
Router MAC address:    0c:36:00:00:1b:08

Admin Config Information:
  state(up), mode(access), speed(auto), duplex(Auto), medium_db(0), fec(auto), p
ort-type-fabric(not-fabric) port-type-external(not-external) dfe-adaptive-tuning
 (1)
  layer(L3), dce-mode(edge), description(to_r1), transmit-reset-skip(0)
  mac-up-timer(0), max-bringup-timer(0), active-jitter-mgmt(0), media-type(none)
 link loopback(off)
  auto neg(on), auto mdix(on), beacon(off), num_of_si(0)
  config auto neg(on) openflow(0) switchport_isolated(0)
  medium(broadcast), snmp trap(on), MTU(9202),
  flowcontrol rx(off) tx(off), link debounce(100),link debounce(link-up 0)
  storm-control bcast:100.00% mcast:100.00% ucast:100.00%
  storm-control bcast level1:100.00% bcast level2:100.00%  mcast level1:100.00%
mcast level2:100.00% ucast level1:100.00% uncast level2:100.00%
  storm-control pps bcast:4294967295 mcast:4294967295 ucast:4294967295
  storm-control mcast:200 ucast:200
   span mode(0 - not a span-destination)
  Flexlink mode(disabled), Active Interface(none), Backup Interface(none), cfg_i

SONiC

Interface configured as below:

interface Ethernet0
 mtu 9202
 speed 25000
 unreliable-los auto
 no shutdown
 ip address 10.0.14.101/24
 ip ospf area 0.0.0.0
 ip ospf cost 10
!

SONiC in this case treats mtu 9202 as a IP MTU.

sonic# show interface Ethernet 0 | grep MTU
IP MTU 9202 bytes

Ping verification from IOS XR to NX-OS

Run a ping command with a df-bit

RP/0/RP0/CPU0:r1#ping 10.0.13.2 size 9202 df-bit
Wed Nov  1 22:08:53.965 UTC
Type escape sequence to abort.
Sending 5, 9202-byte ICMP Echos to 10.0.13.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/2/4 ms

Resulting packet capture shows Ethernet frames of 9216 bytes

Frame 9: 9216 bytes on wire (73728 bits), 9216 bytes captured (73728 bits) on interface -, id 0
Ethernet II, Src: 0c:72:fa:dc:00:05 (0c:72:fa:dc:00:05), Dst: 0c:36:00:00:1b:08 (0c:36:00:00:1b:08)
Internet Protocol Version 4, Src: 10.0.13.1, Dst: 10.0.13.2
Internet Control Message Protocol

Ping verification from IOS XR to SONiC

Run a ping command with a df-bit

RP/0/RP0/CPU0:r1#ping 10.0.14.101 size 9202 df-bit
Fri Dec 22 22:09:03.231 UTC
Type escape sequence to abort.
Sending 5, 9202-byte ICMP Echos to 10.0.14.101, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 6/6/7 ms

Resulting packet capture shows Ethernet frames of 9216 bytes

22:09:03.084397 0c:72:fa:dc:00:06 > 0c:39:b3:ac:00:0a, ethertype IPv4 (0x0800), length 9216: (tos 0x0, ttl 255, id 4, offset 0, flags [DF], proto ICMP (1), length 9202)
    10.0.14.1 > 10.0.14.101: ICMP echo request, id 12233, seq 4, length 9182
22:09:03.087895 0c:39:b3:ac:00:0a > 0c:72:fa:dc:00:06, ethertype IPv4 (0x0800), length 9216: (tos 0x0, ttl 64, id 34473, offset 0, flags [none], proto ICMP (1), length 9202)
    10.0.14.101 > 10.0.14.1: ICMP echo reply, id 12233, seq 4, length 9182