Hi All,
Consider the following topology where h1 and h2 are end hosts, s1 and s2 are OF-enabled HPE L2 switches, and r1 is an OF-enabled HPE L3 switch/router.
h1 --- s1 ------- (port 1) - r1 - (port 2) ------- s2 --- h2
On r1 port 1, we have VLAN 2 with IP address 172.23.7.161/29. On r1 port 2, we have VLAN 3 with IP address 172.23.7.169/29. Routing is enabled.
We have found that traffic traveling from h1 to h2 (or vice versa) over the NORMAL rule will flow correctly. However, we added an OpenFlow rule with higher priority than NORMAL in r1. It matches a very specific TCP flow from h1 to h2. The actions for the rule are to rewrite the MAC addresses correctly, change the VLAN, and output on port 1 or port 2 (we have two flows to create a bidirectional path). We send test traffic from h1 to h2 that matches the flow. We observe two possible outcomes.
Outcome 1
Everything functions as normal. The MACs and VLAN are rewritten, and the packets are forwarded correctly.
Outcome 2
Packets match the rule in r1, but no packets leave the outgoing port. We verified this by monitoring the outgoing port for traffic. Thus, the packet is being dropped by the HPE switch *after* matching an OpenFlow rule instruction the packet to output on a specific port.
We discovered that outcome 2 happens whenever r1's ARP table loses its entries for h1 or h2. Our question is why? We understand that for NORMAL to work, the router must ARP for h1 or h2 in order to encapsulate the packet in a new L2 frame. However, at the OpenFlow layer, we are rewriting MAC addresses ourselves (i.e., doing what the router would do if it had static ARP entries). Shouldn't this bypass the router's default processing? Why does the router need to know about neighbors for a very specific OpenFlow rule to work?
Thanks!