BUM traffic flow in ACI

BUM traffic flow in ACI (Single Pod)

Below article details the insights on the BUM traffic flow within ACI. The article is based on the assumption that reader is aware of the ACI basic terminology. Cisco ACI as understood uses the hardware VTEP Leaf/Spine architecture.

Both Leaf and Spine are allocated the VTEP address assigned dynamically from the pool provided in the APIC configuration.

ACI works efficiently in a way to reduce the flooding within the ACI fabric.

The bridge domain is defined as the broadcast domain within the  ACI architecture. There can exist different EPGs within the same bridge domain. The EPGs within the same bridge domain can be associated with different VLANs. Thus inspite of being in different VLANs the EPGs associated in the same bridge domain are bound to experience flood traffic within the same Bridge domain unlike to the traditional architecture where every VLAN is a different broadcast domain.

ACI works on the concept of no unknowns i.e. it learns the IP address and MAC address of every host connected to the leaf and then stores the details on the respective LOCAL STATION TABLE (LST) of every Leaf. The same information is sent to the SPINE and is stored in its PROXY STATION TABLE . Thus SPINE is aware of every host connected to its leafs. All these are created and published in the COOP database that gets created on every Leaf and Spine. The remote endpoint IP addresses are published under the GLOBAL STATION TABLE that gets created on every leaf.

ACI with this concept works towards reducing the flood in its fabric.

Broadcast / ARP traffic

ARP traffic in a traditional design is usually sent as a broadcast in the network by the legacy switches. In ACI, when the leaf switches are not aware of the destination/remote switches they simply forward them in a VXLAN tunnel to the nearby Spine switch. Spine switches are aware of all the Hosts IPs and MACs in its PROXY STATION TABLE (which is built from the LOCAL STATION TABLE of every Leaf switch). Spine switches then forward the ARP request to the destination Leaf switch. The receiving leaf switch then becomes aware of Source Leaf switch and its corresponding source Host IP and MAC and uses this info to respond back to the ARP request.

Unknown Unicast

Unknown Unicast is termed when the source Host is aware of the destination IP and MAC address. This Host sends the packet to connected source Leaf along with the destination MAC. Source Leaf is unaware of the destination MAC. Thus sends it across to the Spine for PROXY STATION TABLE lookup thus avoiding the broadcast in ACI network unlikely to the way traditional network behaves where unknown unicast triggers the broadcast.

Multicast

ACI also follows the FTAG (Forwarding TAG) trees concept for multicast. There are 12 FTAG trees that can exist in an ACI network. In case of ACI, the multicast stream is sent by the in path Spine to the Leafs in line with the FTAG tree. FTAG ensures every switch is covered without looping.

FTAGs are chosen as per the hash. There are 2 modes which exists within multicast.

• Flooding Mode

Here all the leaf switches receive the multicast stream inspite of the receivers connected or not. As depicted in the diagram below the yellow arrows depict the flood mode multicast flow. Leaf 2 and Leaf 3 have receiving hosts connected only on 1 port. Thus the switches expect a multicast Join message on these switches before sending the multicast stream. So the ports on Leaf 2 and Leaf 3 which do not have the Receiver host connected will not receive the multicast stream. This behavior is not there in Leaf 4 which forwards the multicast stream in all Leaf ports by default because there is no Receiver host connected on any of its ports, thus it doesnot expect a Join message for allowing the multicast stream through its ports.

• Optimized Mode

In this mode the only difference between this mode and Flooding mode is that all the switches expect a Join message before sending the multicast stream along its ports. Thus, here the Leaf 4 will not forward the multicast stream in any of its ports because there is no Join message received.