<?xml version="1.0" encoding="US-ASCII"?> version='1.0' encoding='utf-8'?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
  <!ENTITY RFC2119 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"> nbsp    "&#160;">
  <!ENTITY RFC8174 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"> zwsp   "&#8203;">
  <!ENTITY RFC6514 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6514.xml"> nbhy   "&#8209;">
  <!ENTITY RFC7432 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7432.xml">
<!ENTITY I-D.ietf-bess-evpn-bum-procedure-updates SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml3/reference.I-D.ietf-bess-evpn-bum-procedure-updates.xml">
<!ENTITY RFC8365 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8365.xml">
<!ENTITY RFC7902 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7902.xml">
<!ENTITY RFC6513 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.6513.xml">
<!ENTITY RFC7348 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7348.xml">
<!ENTITY RFC4023 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.4023.xml">
<!ENTITY RFC7637 SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.7637.xml">
<!ENTITY I-D.ietf-bess-evpn-proxy-arp-nd SYSTEM "https://xml2rfc.ietf.org/public/rfc/bibxml3/reference.I-D.ietf-bess-evpn-proxy-arp-nd.xml"> wj     "&#8288;">
]>
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="3"?>
<?rfc tocindent="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc comments="yes"?>
<?rfc inline="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?>

<rfc xmlns:xi="http://www.w3.org/2001/XInclude" category="std" docName="draft-ietf-bess-evpn-optimized-ir-12" number="9574" ipr="trust200902" submissionType="IETF">
  <!-- Generated by id2xml 1.5.0 on 2019-12-29T20:19:06Z -->

  <?rfc strict="yes"?>

  <?rfc compact="yes"?>

  <?rfc subcompact="no"?>

  <?rfc symrefs="yes"?>

  <?rfc sortrefs="no"?>

  <?rfc text-list-symbols="-o+*"?>

  <?rfc toc="yes"?> submissionType="IETF" obsoletes="" updates="" xml:lang="en" tocInclude="true" consensus="true"  tocDepth="3" symRefs="true" sortRefs="true" version="3">

  <front>
    <title abbrev="EVPN Optimized IR">Optimized Ingress Replication Solution
    for Ethernet VPN (EVPN)</title> VPNs (EVPNs)</title>

    <seriesInfo name="RFC" value="9574"/>
    <author fullname="J. fullname="Jorge Rabadan" initials="J." role="editor" surname="Rabadan">
      <organization>Nokia</organization>
      <address>
        <postal>
          <street>777 Middlefield Road</street>
          <city>Mountain View</city>
          <region>CA</region>
          <code>94043</code>

          <country>USA</country>
          <country>United States of America</country>
        </postal>
        <email>jorge.rabadan@nokia.com</email>
      </address>
    </author>
    <author fullname="S. fullname="Senthil Sathappan" initials="S." surname="Sathappan">
      <organization>Nokia</organization>
      <address>
        <email>senthil.sathappan@nokia.com</email>
      </address>
    </author>
    <author fullname="W. fullname="Wen Lin" initials="W." surname="Lin">
      <organization>Juniper Networks</organization>
      <address>
        <email>wlin@juniper.net</email>
      </address>
    </author>
    <author fullname="M. fullname="Mukul Katiyar" initials="M." surname="Katiyar">
      <organization>Versa Networks</organization>
      <address>
        <email>mukul@versa-networks.com</email>
      </address>
    </author>
    <author fullname="A. fullname="Ali Sajassi" initials="A." surname="Sajassi">
      <organization>Cisco Systems</organization>
      <address>
        <email>sajassi@cisco.com</email>
      </address>
    </author>
    <date day="25" month="January" year="2022"/>

    <workgroup>BESS Workgroup</workgroup>  month="April" year="2024"/>
    <area>rtg</area>
    <workgroup>BESS</workgroup>

<keyword>Assisted Replication</keyword>
<keyword>AR</keyword>
<keyword>AR-Replicator</keyword>
<keyword>RNVE</keyword>
<keyword>Pruned Flood List</keyword>
<keyword>PFL</keyword>
<keyword>Pruned Flooding List</keyword>

<abstract>
      <t>Network Virtualization Overlay (NVO) networks using Ethernet VPN (EVPN) VPNs
      (EVPNs) as their control plane may use Ingress Replication trees based on ingress replication
      or PIM (Protocol Protocol Independent Multicast)-based trees Multicast (PIM) to convey the overlay Broadcast,
      Unknown unicast and Unicast, or Multicast (BUM) traffic. PIM provides an efficient
      solution to avoid that prevents sending multiple copies of the same packet over the
      same physical link, however link; however, it may not always be deployed in the Network
      Virtualization Overlay core network.
      NVO network core. Ingress Replication replication avoids the
      dependency on PIM in the Network Virtualization Overlay NVO network core.
      While Ingress Replication ingress replication provides a simple multicast transport, some
      Network Virtualization Overlay
      NVO networks with demanding multicast
      applications require a more efficient solution without PIM in the core.
      This document describes a solution to optimize the efficiency of Ingress
      Replication ingress
      replication trees.</t>
    </abstract>
  </front>
  <middle>
    <section anchor="sect-1" title="Introduction"> numbered="true" toc="default">
      <name>Introduction</name>
      <t>Ethernet Virtual Private Networks (EVPN) (EVPNs) may be used as the control
      plane for a Network Virtualization Overlay (NVO) network <xref
      target="RFC8365"/>. target="RFC8365" format="default"/>. Network Virtualization Edge (NVE) and Provider Edge
      (PE) devices that are part of the same EVPN Broadcast Domain (BD) use
      Ingress Replication (IR) or PIM-based trees to transport the tenant's
      Broadcast, Unknown unicast and Unicast, or Multicast (BUM) traffic.</t>

      <t>In
      <t> In the Ingress Replication ingress replication approach, the ingress NVE receving receiving a BUM
      frame from the Tenant System (TS) will create as many copies of the
      frame as the number of remote NVEs/PEs that are attached to the BD. Each of
      those copies will be
      encapsulated into an IP packet where the outer IP Destination Address
      (IP DA) identifies the loopback of the egress NVE/PE. The IP fabric core
      nodes (also known as Spines) spines) will simply route the IP encapsulated IP-encapsulated BUM
      frames based on the outer IP DA. If PIM-based trees are used instead of
      Ingress Replication,
      ingress replication, the NVEs/PEs attached to the same BD will join a
      PIM-based tree. The ingress NVE receiving a BUM frame will send a single
      copy of the frame, encapsulated into an IP packet where the outer IP DA
      is the multicast address that represents the PIM-based tree. The IP
      fabric core nodes are part of the PIM tree and keep multicast state for
      the multicast group, so that IP encapsulated IP-encapsulated BUM frames can be routed to
      all the NVEs/PEs that joined the tree.</t>
      <t>The two approaches are illustrated in <xref target="IR-PIM"/>. target="IR-PIM" format="default"/>. On the
      left-hand side, side of the diagram, NVE1 uses Ingress Replication ingress replication to send a BUM frame
      (originated from Tenant System TS1) to the remote nodes attached to the
      BD, i.e., NVE2, NV3, NVE3, and PE1. On the right-hand side of the diagram, side, the
      same example is depicted but using a PIM-based tree, i.e., (S1,G1),
      instead of Ingress Replication. ingress replication. While a single copy of the tunneled BUM
      frame is generated in the latter approach, all the routers in the fabric
      need to keep muticast multicast state, e.g., the Spine spine keeps a PIM multicast
      routing entry for (S1,G1) with an Incoming Interface (IIF) and three
      Outgoing Interfaces (OIFs).</t>

      <t><figure anchor="IR-PIM"
          title="Ingress
      <figure anchor="IR-PIM">
        <name>Ingress Replication vs PIM-based trees vs. PIM-Based Trees in NVO networks">
          <artwork><![CDATA[           To-WAN                           To-WAN Networks</name>
        <artwork name="" type="" align="left" alt=""><![CDATA[           To WAN                           To WAN
              ^                                ^
              |                                |
           +-----+                          +-----+
+----------| PE1 |-----------+   +----------| PE1 |-----------+
|          +--^--+           |   |          +--^--+           |
|             |    IP Fabric |   |             |    IP Fabric |
|             PE             |   |    (S1,G1)  |OIF to-G to G1     |
| +----PE->+-----+ No State  |   |      IIF +-----+ OIF to-G to G1 |
| | +---2->|Spine|------+    |   |   +------>Spine|------+    |
| | | +-3->+-----+      |    |   |   |      +-----+      |    |
| | | |       2         3    |   |   |PIM      |OIF to-G | to G1|    |
| | | |IR     |         |    |   |   |tree     |         |    |
|+-----+   +--v--+   +--v--+ |   |+-----+   +--v--+   +--v--+ |
+| NVE1|---| NVE2|---| NVE3|-+   +| NVE1|---| NVE2|---| NVE3|-+
 +--^--+   +-----+   +-----+      +--^--+   +-----+   +-----+
    |         |         |            |         |         |
    |         v         v            |         v         v
   TS1       TS2       TS3          TS1       TS2       TS3]]></artwork>
        </figure></t>
      </figure>

      <t>In Network Virtualization Overlay NVO networks where PIM-based trees
      cannot be used, Ingress Replication ingress replication is the only option. Examples of
      these situations are Network Virtualization Overlay NVO networks where the
      core nodes do not support PIM or the network operator does not want to
      run PIM in the core.</t>
      <t>In some use-cases, use cases, the amount of replication for BUM traffic is kept
      under control on the NVEs due to the following fairly common
      assumptions:</t>

      <t><list hangIndent="3" style="letters">
          <t hangText="">Broadcast

      <ol spacing="normal" type="a"><li>Broadcast traffic is greatly reduced due to the proxy ARP
          (Address
          Address Resolution Protocol) Protocol (ARP) and proxy ND (Neighbor Discovery) Neighbor Discovery (ND)
          capabilities supported by EVPN EVPNs <xref target="RFC9161" format="default"/> on the NVEs <xref
          target="I-D.ietf-bess-evpn-proxy-arp-nd"/>. NVEs. Some NVEs can even
          provide Dynamic Host Configuration Protocol (DHCP) server functions
          for the attached Tenant Systems, TSs, reducing the broadcast even
          further.</t>

          <t
          hangText="b) Unknown unicast traffic is greatly reduced in virtualized NVO">Unknown even
          further.</li>
        <li>Unknown
          unicast traffic is greatly reduced in Network Virtualization Overlay NVO
          networks where all the MAC Media Access Control (MAC) and IP addresses from the Tenant Systems TSs
          are learned in the control plane.</t>

          <t>Multicast plane.</li>
        <li>Multicast applications are not used.</t>
        </list></t> used.</li>
      </ol>
      <t>If the above assumptions are true for a given Network Virtualization
      Overlay NVO network, then Ingress Replication ingress replication provides a simple solution for
      multi-destination traffic. However, the statement c) c. above is not always
      true
      true, and multicast applications are required in many use-cases.</t> use cases.</t>

      <t>When the multicast sources are attached to NVEs residing in
      hypervisors or low-performance-replication TORs (Top Of Rack switches), Top-of-Rack (ToR) switches,
      the ingress replication of a large amount of multicast traffic to a
      significant number of remote NVEs/PEs can seriously degrade the
      performance of the NVE and impact the application.</t>
      <t>This document describes a solution that makes use of two Ingress
      Replication ingress
      replication optimizations:</t>

      <t><list style="numbers">
          <t>Assisted-Replication (AR)</t>

          <t>Pruned-Flood-Lists (PFL)</t>
        </list></t>

      <t>Assisted-Replication
      <ol spacing="normal" type="1"><li>Assisted Replication (AR)</li>
        <li>Pruned Flooding Lists (PFLs)</li>
      </ol>
      <t>Assisted Replication consists of a set of procedures that allows the
      ingress NVE/PE to send a single copy of a Broadcast broadcast or Multicast multicast frame
      received from a Tenant System TS to the Broadcast Domain, BD without the need
      for PIM in the underlay. Assisted Replication defines the roles of
      AR-REPLICATOR and AR-LEAF routers. The AR-LEAF is the ingress NVE/PE
      attached to the Tenant System. TS. The AR-LEAF sends a single copy of a
      Broadcast
      broadcast or Multicast multicast packet to a selected AR-REPLICATOR that
      replicates the packet mutiple multiple times to remote AR-LEAF or AR-REPLICATOR
      routers,
      routers and is therefore "assisting" the ingress AR-LEAF in delivering the
      Broadcast
      broadcast or Multicast multicast traffic to the remote NVEs/PEs attached to the
      same Broadcast Domain. Assisted-Replication BD. Assisted Replication can use a single
      AR-REPLICATOR or two AR-REPLICATOR routers in the path between the
      ingress AR-LEAF and the remote destination NVE/PEs. NVEs/PEs. The procedures that
      use a single AR-REPLICATOR (Non-Selective Assisted-Replication Solution) (the non-selective Assisted Replication solution)
      are specified in <xref target="sect-5"/>, target="sect-5" format="default"/>, whereas <xref
      target="sect-6"/> target="sect-6" format="default"/> describes how multi-staged multi-stage replication, i.e., two
      AR-REPLICATOR routers in the path between the ingress AR-LEAF and
      destination NVEs/PEs, is accomplished (Selective Assisted-Replication
      Solution). (the selective Assisted Replication
      solution). The Assisted-Replication procedures for Assisted Replication do not impact unknown
      unicast traffic, which follows the same forwarding procedures as known
      unicast traffic so that packet re-ordering reordering does not occur.</t>

      <t>Pruned-Flood-Lists is
      <t>PFLs provide a method for the ingress NVE/PE to prune or
      remove certain destination NVEs/PEs from a flood-list, flooding list, depending on the
      interest of those NVEs/PEs in receiving Broadcast, Multicast or Unknown
      unicast. BUM traffic. As specified in <xref target="RFC8365"/>, target="RFC8365" format="default"/>, an NVE/PE builds a
      flood-list
      flooding list for BUM traffic based on the Next-Hops next hops of the received EVPN
      Inclusive Multicast Ethernet Tag routes for the Broadcast Domain. BD. While
      <xref target="RFC8365"/> target="RFC8365" format="default"/> states that the flood-list flooding list is used for all BUM
      traffic, this document allows pruning certain Next-Hops next hops from the list.
      As an example, suppose an ingress NVE creates a flood-list flooding list with
      Next-Hops
      next hops PE1, PE2 PE2, and PE3. If PE2 and PE3 signaled no-interest did not signal any interest in
      receiving Unknown Unicast unknown unicast traffic in their Inclusive Multicast Ethernet Tag
      routes, when the ingress NVE receives an Unknown Unicast unknown unicast frame from a
      Tenant System
      TS, it will replicate it only to PE1. That is, PE2 and PE3 are
      "pruned" from the NVE's flood-list flooding list for Unknown Unicast unknown unicast traffic.
      Pruned-Flood-Lists
      PFLs can be used with Ingress Replication ingress replication or
      Assisted-Replication,
      Assisted Replication and it is are described in <xref
      target="sect-7"/>.</t> target="sect-7" format="default"/>.</t>
      <t>Both optimizations, Assisted-Replication optimizations -- Assisted Replication and Pruned-Flood-Lists, PFLs -- may
      be used together or independently so that the performance and efficiency
      of the network to transport multicast can be improved. Both solutions
      require some extensions to the BGP attributes used in <xref
      target="RFC7432"/>, and they are described in target="RFC7432" format="default"/>; see <xref
      target="sect-4"/>.</t> target="sect-4" format="default"/> for details.</t>
      <t>The Assisted-Replication Assisted Replication solution described in this document is
      focused on Network Virtualization Overlay NVO networks (hence it uses its use of IP
      tunnels) and
      tunnels). MPLS transport networks are out of scope. scope for this document. The
      Pruned-Flood-Lists
      PFLs solution MAY <bcp14>MAY</bcp14> be used in Network Virtualization
      Overlay NVO
      and MPLS transport networks.</t>
      <t><xref target="sect-3"/> target="sect-3" format="default"/> lists the requirements of the combined
      optimized Ingress Replication ingress replication solution, whereas <xref target="sect-5"/> Sections&nbsp;<xref target="sect-5" format="counter"/>
      and <xref target="sect-6"/> target="sect-6" format="counter"/> describe the Assisted-Replication Assisted Replication solution
      (for Non-Selective
      for non-selective and Selective selective procedures, respectively), and respectively. <xref
      target="sect-7"/> target="sect-7" format="default"/> provides the Pruned-Flood-Lists PFLs solution.</t>
    </section>

    <section anchor="sect-2" title="Terminology numbered="true" toc="default">
      <name>Terminology and Conventions"> Conventions</name>
      <t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
      "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", "<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>",
      "<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>", "<bcp14>SHALL
      NOT</bcp14>", "<bcp14>SHOULD</bcp14>", "<bcp14>SHOULD NOT</bcp14>",
      "<bcp14>RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>",
      "<bcp14>MAY</bcp14>", and
      "OPTIONAL" "<bcp14>OPTIONAL</bcp14>" in this document are
      to be interpreted as described in BCP 14 BCP&nbsp;14 <xref target="RFC2119"/> target="RFC2119"
      format="default"/> <xref target="RFC8174"/> target="RFC8174" format="default"/> when, and
      only when, they appear in all capitals, as shown here.</t>

      <t>The following terminology is used throughout the this document:</t>

      <t><list style="symbols">
          <t>Asisted

      <dl>
        <dt>AR-IP:</dt><dd>Assisted Replication forwarding mode: for an AR-LEAF, it means
          sending an Attachment Circuit BM packet - IP. Refers to a single AR-REPLICATOR
          with tunnel destination IP AR-IP. For an AR-REPLICATOR, it means
          sending a BM packet IP address owned by the AR-REPLICATOR and used to a selected number or all
          differentiate the overlay tunnels
          when incoming traffic that must follow the packet was previously received from an overlay tunnel.</t>

          <t>AR-LEAF: Assisted AR
          procedures. The AR-IP is also used in the Tunnel Identifier and
          Next Hop fields of the Replicator-AR route.</dd>
        <dt>AR-LEAF:</dt><dd>Assisted Replication - LEAF, refers LEAF. Refers to an NVE/PE that
          sends all the Broadcast and Multicast BM traffic to an AR-REPLICATOR
          that can replicate the traffic further on its behalf. An AR-LEAF is
          typically an NVE/PE with poor replication performance
          capabilities.</t>

          <t>AR-REPLICATOR: Assisted
          capabilities.</dd>
        <dt>AR-REPLICATOR:</dt><dd>Assisted Replication - REPLICATOR, refers REPLICATOR. Refers to an
          NVE/PE that can replicate Broadcast broadcast or Multicast multicast traffic received on
          overlay tunnels to other overlay tunnels and local Attachment
          Circuits. ACs.
          This document defines the control and data plane
          procedures that an AR-REPLICATOR needs to follow.</t>

          <t>AR-IP: IP address owned by the AR-REPLICATOR and used follow.</dd>
        <dt>AR-VNI:</dt><dd>Assisted Replication - VNI. Refers to
          differentiate the incoming traffic that must follow the AR
          procedures. The AR-IP is also used in the Tunnel a Virtual eXtensible Local Area Network (VXLAN) Network Identifier and
          Next-Hop fields of the Replicator-AR route.</t>

          <t>AR-VNI: VNI (VNI) advertised by the AR-REPLICATOR along with the
          Replicator-AR route. It is used to identify the incoming packets
          that must follow the AR procedures ONLY in the Single-IP single-IP AR-REPLICATOR
          case (see <xref target="sect-8"/>.</t>

          <t>BM traffic: Refers to target="sect-8" format="default"/>).</dd>
        <dt>Assisted Replication forwarding mode:</dt><dd>In the case of an AR-LEAF,
          sending an Attachment Circuit (AC) Broadcast and Multicast frames (excluding
          unknown unicast frames).</t>

          <t>BD: (BM) packet to a single AR-REPLICATOR
          with a tunnel destination address AR-IP. In the case of an AR-REPLICATOR, this means
          sending a BM packet to a selected number of, or all of, the overlay tunnels
          when the packet was previously received from an overlay tunnel.</dd>
        <dt>BD:</dt><dd> Broadcast Domain, as defined in <xref target="RFC7432"/>.</t>

          <t>BD label: defined target="RFC7432" format="default"/>.</dd>
        <dt>BD label:</dt><dd>Defined as the MPLS label that identifies the Broadcast
          Domain BD
          and is advertised in Regular-IR or Replicator-AR routes, when
          the encapsulation is MPLSoGRE MPLS over GRE (MPLSoGRE) or MPLSoUDP. </t>

          <t>DF MPLS over UDP (MPLSoUDP). </dd>
        <dt>BM traffic:</dt><dd>Refers to broadcast and NDF: Designated multicast frames (excluding
          unknown unicast frames).</dd>
        <dt>DF and NDF:</dt><dd>Designated Forwarder and Non-Designated Forwarder, Forwarder.
          These are roles defined in NVE/PEs NVEs/PEs attached to Multi-Homed Tenant Systems, multihomed TSs,
          as per <xref target="RFC7432"/> target="RFC7432" format="default"/> and <xref target="RFC8365"/>.</t>

          <t>ES target="RFC8365" format="default"/>.</dd>
        <dt>ES and ESI: Ethernet ESI:</dt><dd>Ethernet Segment and Ethernet Segment Identifier, as Identifier.
          EVPN Multi-Homing multihoming concepts as specified in <xref
          target="RFC7432"/>.</t>

          <t>EVI: target="RFC7432" format="default"/>.</dd>
        <dt>EVI:</dt><dd> EVPN Instance. A group of Provider Edge (PE) devices
          participating in the same EVPN service, as specified in <xref
          target="RFC7432"/>.</t>

          <t>GRE: Generic target="RFC7432" format="default"/>.</dd>
        <dt>GRE:</dt><dd>Generic Routing Encapsulation <xref target="RFC4023"/>.</t>

          <t>Ingress target="RFC4023" format="default"/>.</dd>
        <dt>Ingress Replication forwarding mode: it refers mode:</dt><dd> Refers to the Ingress
          Replication ingress
          replication behavior explained in <xref target="RFC7432"/>. It means
          sending target="RFC7432" format="default"/>. In
          this mode, an Attachment Circuit AC BM packet copy is sent to each remote PE/NVE
          in the BD BD, and sending an overlay BM packet is sent only to the Attachment
          Circuits ACs
          and not to other overlay tunnels.</t>

          <t>IR-IP: tunnels.</dd>
        <dt>IR-IP:</dt><dd>Ingress Replication - IP. Refers to the local IP address of an NVE/PE that is used for the Ingress
          Replication ingress
          replication signaling and procedures provided in <xref target="RFC7432"/>. target="RFC7432" format="default"/>.
          Encapsulated incoming traffic with an outer destination IP address matching the
          IR-IP will follow the Ingress Replication procedures for ingress replication and not the
          Assisted-Replication procedures.
          procedures for Assisted Replication. The IR-IP is also used in the
          Tunnel Identifier and Next-hop Next Hop fields of the Regular-IR route.</t>

          <t>IR-VNI: route.</dd>
        <dt>IR-VNI:</dt><dd>Ingress Replication - VNI. Refers to a VNI advertised along with the Inclusive Multicast
          Ethernet Tag route for Ingress Replication Tunnel Type.</t>

          <t>MPLS: Multi-Protocol the ingress replication tunnel type.</dd>
        <dt>MPLS:</dt><dd>Multi-Protocol Label Switching.</t>

          <t>NVE: Network Switching.</dd>
        <dt>NVE:</dt><dd>Network Virtualization Edge router, used in this document as
          in <xref target="RFC8365"/>.</t>

          <t>NVGRE: Network Virtualization target="RFC8365" format="default"/>.</dd>
        <dt>NVGRE:</dt><dd>Network virtualization using Generic Routing
          Encapsulation, as in
          Encapsulation <xref target="RFC7637"/>.</t>

          <t>PE: Provider Edge router.</t>

          <t>PMSI: P-Multicast target="RFC7637" format="default"/>.</dd>
        <dt>PE:</dt><dd>Provider Edge.</dd>
        <dt>PMSI:</dt><dd>P-Multicast Service Interface - a Interface. A conceptual interface for
          a PE to send customer multicast traffic to all or some PEs in the
          same VPN <xref target="RFC6513"/>.</t>

          <t>RD: Route Distinguisher.</t>

          <t>Regular-IR route: an target="RFC6513" format="default"/>.</dd>
        <dt>RD:</dt><dd>Route Distinguisher.</dd>
        <dt>Regular-IR route:</dt><dd>An EVPN Inclusive Multicast Ethernet Tag route
          <xref target="RFC7432"/> target="RFC7432" format="default"/> that uses Ingress Replication Tunnel
          Type.</t>

          <t>RNVE: Regular NVE, refers the ingress replication tunnel
          type.</dd>
        <dt>Replicator-AR route:</dt><dd>An EVPN Inclusive Multicast Ethernet Tag
          route that is advertised by an AR-REPLICATOR to signal its
          capabilities, as described in <xref target="sect-4" format="default"/>.</dd>
        <dt>RNVE:</dt><dd>Regular NVE. Refers to an NVE that supports the procedures
          of
          provided in <xref target="RFC8365"/> target="RFC8365" format="default"/> and does not support the procedures provided in
          this document. However, this document defines procedures to
          interoperate with RNVEs.</t>

          <t>Replicator-AR route: an EVPN Inclusive Multicast Ethernet Tag
          route that is advertised by an AR-REPLICATOR to signal its
          capabilities, as described in <xref target="sect-4"/>.</t>

          <t>TOR: Top Of Rack switch.</t>

          <t>TS RNVEs.</dd>
          <dt>ToR switch:</dt><dd>Top-of-Rack switch.</dd>
        <dt>TS and VM: Tenant VM:</dt><dd>Tenant System and Virtual Machine. In this document
          Tenant Systems document,
          TSs and Virtual Machiness VMs are the devices connected to
          the Attachment Circuits ACs of the PEs and NVEs.</t>

          <t>VNI: VXLAN NVEs.</dd>
        <dt>VNI:</dt><dd>VXLAN Network Identifier, used Identifier. Used in VXLAN tunnels.</t>

          <t>VSID: Virtual tunnels.</dd>
        <dt>VSID:</dt><dd>Virtual Segment Identifier, used Identifier. Used in NVGRE tunnels.</t>

          <t>VXLAN: Virtual Extensible LAN tunnels.</dd>
        <dt>VXLAN:</dt><dd>Virtual eXtensible Local Area Network <xref target="RFC7348"/>.</t>
        </list></t> target="RFC7348" format="default"/>.</dd>
      </dl>
    </section>

    <section anchor="sect-3" title="Solution Requirements"> numbered="true" toc="default">
      <name>Solution Requirements</name>
      <t>The Ingress Replication ingress replication optimization solution specified in this
      document meets the following requirements:</t>

      <t><list style="letters">
          <t>It
      <ol spacing="normal" type="a"><li>The solution provides an Ingress Replication ingress replication optimization for Broadcast and
          Multicast BM
          traffic without the need for PIM, PIM while preserving the
          packet order for unicast applications, i.e., unknown unicast traffic
          should follow the same path as known unicast traffic. This
          optimization is required in low-performance NVEs.</t>

          <t>It NVEs.</li>
        <li>The solution reduces the flooded traffic in Network Virtualization Overlay NVO
          networks where some NVEs do not need broadcast/multicast and/or
          unknown unicast traffic.</t> traffic.</li>
        <li>
          <t>The solution is compatible with <xref target="RFC7432"/> target="RFC7432" format="default"/> and
          <xref target="RFC8365"/> target="RFC8365" format="default"/> and has no impact on the CE Customer Edge (CE) procedures for
          BM traffic. In particular, the solution supports the following EVPN
          functions: <list style="symbols">
              <t>All-active multi-homing,
          functions:</t>
          <ul spacing="normal">
            <li>All-active multihoming, including the split-horizon and
              Designated Forwarder (DF) functions.</t>

              <t>Single-active multi-homing,
              DF functions.</li>
            <li>Single-active multihoming, including the DF function.</t>

              <t>Handling function.</li>
            <li>Handling of multi-destination traffic and processing of
              broadcast and multicast
              BM traffic as per <xref target="RFC7432"/>.</t>
            </list></t>

          <t>The target="RFC7432" format="default"/>.</li>
          </ul>
        </li>
        <li>The solution is backwards backward compatible with existing NVEs using a
          non-optimized version of Ingress Replication. ingress replication. A given BD can have
          NVEs/PEs supporting regular Ingress Replication ingress replication and optimized
          Ingress Replication.</t>

          <t>The
          ingress replication.</li>
        <li>The solution is independent of the Network Virtualization Overlay
          specific NVO-specific data plane encapsulation and the virtual identifiers being
          used, e.g.: e.g., VXLAN VNIs, NVGRE VSIDs VSIDs, or MPLS labels, as long as the
          tunnel is IP-based.</t>
        </list></t> IP based.</li>
      </ol>
    </section>
    <section anchor="sect-4"
             title="EVPN numbered="true" toc="default">
      <name>EVPN BGP Attributes for Optimized Ingress Replication">
      <t>This Replication</name>
      <t>The ingress replication optimization solution specified in this document
extends the <xref target="RFC7432"/> Inclusive
      Multicast Ethernet Tag routes and attributes described in <xref target="RFC7432" format="default"/> so that an NVE/PE can
      signal its optimized Ingress Replication ingress replication capabilities.</t>
      <t>The NLRI Network Layer Reachability Information (NLRI) of the Inclusive Multicast Ethernet Tag route as in <xref
      target="RFC7432"/> target="RFC7432" format="default"/> is shown in <xref target="imet-route"/> target="imet-route" format="default"/> and it is
      used in this document without any modifications to its format. The PMSI
      Tunnel Attribute's general format as provided in <xref target="RFC7432"/> target="RFC7432" format="default"/> (which
      takes it from <xref target="RFC6514"/>) target="RFC6514" format="default"/>) is used in this document, document; only a
      new Tunnel Type tunnel type and new flags are specified, as shown in <xref
      target="pta"/>:</t>

      <t><figure anchor="imet-route"
          title="EVPN target="pta" format="default"/>.</t>
      <figure anchor="imet-route">
        <name>EVPN Inclusive Multicast Ethernet Tag route's NLRI">
          <artwork><![CDATA[                 +---------------------------------+ Route's NLRI</name>
        <artwork name="" type="" align="left" alt=""><![CDATA[                 +------------------------------------+
                 |      RD (8 octets)                 |
                 +---------------------------------+
                 +------------------------------------+
                 |  Ethernet Tag ID (4 octets)        |
                 +---------------------------------+
                 +------------------------------------+
                 |  IP Address Length (1 octet)       |
                 +---------------------------------+
                 +------------------------------------+
                 |  Originating Router's IP Addr Address   |
                 |        (4 or 16 octets)            |
                 +---------------------------------+]]></artwork>
        </figure><figure anchor="pta" title="PMSI
                 +------------------------------------+]]></artwork>
      </figure>

      <figure anchor="pta">
        <name>PMSI Tunnel Attribute">
          <artwork><![CDATA[ Attribute</name>
        <artwork name="" type="" align="left" alt=""><![CDATA[                                        0  1  2  3  4  5  6  7
+---------------------------------+    +--+--+--+--+--+--+--+--+
|  Flags (1 octet)                | -> |x |E |x |  T  |BM|U |L |
+---------------------------------+    +--+--+--+--+--+--+--+--+
|  Tunnel Type (1 octets) octet)          |    T = Assisted-Replication Assisted Replication Type
+---------------------------------+    BM = Broadcast and Multicast
|  MPLS Label (3 octets)          |    U = Unknown unicast (unknown unicast)
+---------------------------------+    x = unassigned
|  Tunnel Identifier (variable)   |
+---------------------------------+]]></artwork>
        </figure>The
      </figure>
      <t>The Flags field in <xref target="pta"/> target="pta" format="default"/> is 8 bits long as per
      <xref target="RFC7902"/>, where the target="RFC7902" format="default"/>. The Extension flag (E) flag was allocated by <xref target="RFC7902" format="default"/>, and the Leaf
      Information Required (L) Flag are already allocated. flag was allocated by <xref target="RFC6514" format="default"/>. This document defines the use of 4 bits of this Flags field, and suggests the
      following allocation to IANA:</t>

      <t><list style="symbols">
          <t>bits field:
      </t>

      <ul spacing="normal">
        <li>Bits 3 and 4, forming which together form the Assisted-Replication Assisted Replication Type (T)
          field</t>

          <t>bit
          field</li>
        <li>Bit 5, called the Broadcast and Multicast (BM) flag</t>

          <t>bit flag</li>
        <li>Bit 6, called the Unknown (U) flag</t>
        </list>Bits flag</li>
      </ul>
      <t>Bits 5 and 6 are collectively referred to as the Pruned-Flood Pruned Flooding Lists (PFL) (PFLs) flags.</t>
      <t>The T field and Pruned-Flood-Lists PFLs flags are defined as follows:</t>

      <t><list style="symbols">
      <ul spacing="normal">
        <li>
          <t>T is the Assisted-Replication Assisted Replication Type field (2 bits) that bits), which defines
          the AR role of the advertising router:<list style="symbols">
              <t>00 router:</t>
          <ul spacing="normal">
            <li>00 (decimal 0) = RNVE (non-AR support)</t>

              <t>01 support)</li>
            <li>01 (decimal 1) = AR-REPLICATOR</t>

              <t>10 AR-REPLICATOR</li>
            <li>10 (decimal 2) = AR-LEAF</t>

              <t>11 AR-LEAF</li>
            <li>11 (decimal 3) = RESERVED</t>
            </list></t> RESERVED</li>
          </ul>
        </li>
        <li>
          <t>The Pruned-Flood-Lists PFLs flags define the desired behavior of the
          advertising router for the different types of traffic:<list
              style="symbols">
              <t>Broadcast traffic:</t>
          <ul spacing="normal">
            <li>Broadcast and Multicast (BM) flag. BM=1 BM = 1 means "prune-me" "prune me from
              the BM flooding list. BM=0 means list". BM = 0 indicates regular behavior.</t>

              <t>Unknown behavior.</li>
            <li>Unknown (U) flag. U=1 U = 1 means "prune-me" "prune me from the Unknown
              flooding list. U=0 means list". U = 0 indicates regular behavior.</t>
            </list></t>

          <t>Flag behavior.</li>
</ul>
</li>

        <li>The L is an existing flag (bit 7) is defined in <xref target="RFC6514"/>
          (L=Leaf Information Required, bit 7) target="RFC6514" format="default"/>
          and it will be used only in the
          Selective
          selective AR Solution.</t>
        </list></t> solution.</li>
      </ul>
      <t>Please refer to <xref target="sect-11"/> target="sect-11" format="default"/> for the IANA considerations
      related to the PMSI Tunnel Attribute flags.</t>
      <t>In this document, the above Inclusive Multicast Ethernet Tag route
      <xref target="imet-route"/>
      (<xref target="imet-route" format="default"/>) and PMSI Tunnel Attribute <xref
      target="pta"/> (<xref target="pta" format="default"/>) can be used in two different modes for the same BD:</t>

      <t><list style="symbols">
          <t>Regular-IR route: in

      <dl>
        <dt>Regular-IR route:</dt><dd>In this route, Originating Router's IP Address,
          Tunnel Type (0x06), MPLS Label Label, and Tunnel Identifier MUST <bcp14>MUST</bcp14> be used as
          described in <xref target="RFC7432"/> target="RFC7432" format="default"/> when Ingress Replication ingress replication is in
          use. The NVE/PE that advertises the route will set the Next-Hop Next Hop to
          an IP address that we denominate IR-IP in this document. When
          advertised by an AR-LEAF node, the Regular-IR route MUST <bcp14>MUST</bcp14> be
          advertised with type the T field set to 10 (AR-LEAF).</t>

          <t>Replicator-AR route: this (AR-LEAF).</dd>
        <dt>
          Replicator-AR route:</dt><dd><t>This route is used by the AR-REPLICATOR to
          advertise its AR capabilities, with the fields set as follows:<list
              style="symbols"> follows:</t>
          <ul spacing="normal">
            <li>

              <t>Originating Router's IP Address MUST <bcp14>MUST</bcp14> be set to an IP address
              of the advertising router that is common to all the EVIs on the
              PE (usually this is a loopback address of the PE). <list
                  style="symbols">
                  <t>The </t>
              <ul spacing="normal">
                <li>The Tunnel Identifier and Next-Hop SHOULD Next Hop fields <bcp14>SHOULD</bcp14> be set to the
                  same IP address as the Originating Router's IP address Address field when
                  the NVE/PE originates the route, route -- that is, when the NVE/PE is
                  not an ASBR as in section 10.2 of ASBR; see <xref target="RFC8365"/>. target="RFC8365" section="10.2" sectionFormat="of"/>.
                  Irrespective of the values in the Tunnel Identifier and
                  Originating Router's IP Address fields, the ingress NVE/PE
                  will process the received Replicator-AR route and will use
                  the IP Address address setting in the Next-Hop Next Hop field to create IP tunnels to
                  the AR-REPLICATOR.</t>

                  <t>The Next-Hop AR-REPLICATOR.</li>
                <li>The Next Hop address is referred to as the AR-IP and MUST <bcp14>MUST</bcp14>
                  be different from the IR-IP for a given PE/NVE, unless the
                  procedures provided in <xref target="sect-8"/> target="sect-8" format="default"/> are followed.</t>
                </list></t>

              <t>Tunnel followed.</li>
              </ul>
            </li>
            <li>Tunnel Type MUST <bcp14>MUST</bcp14> be set to Assisted-Replication Assisted Replication Tunnel. <xref
              target="sect-11"/> target="sect-11" format="default"/> provides the allocated type value.</t>

              <t>T (AR role value.</li>
            <li>T (Assisted Replication type) MUST <bcp14>MUST</bcp14> be set to 01 (AR-REPLICATOR).</t>

              <t>L (AR-REPLICATOR).</li>
            <li>L (Leaf Information Required) MUST <bcp14>MUST</bcp14> be set to 0 (for for
              non-selective AR), AR and MUST <bcp14>MUST</bcp14> be set to 1 (for for selective AR).</t>
            </list></t>
        </list></t> AR.</li>
          </ul>
        </dd>
      </dl>

      <t>An NVE/PE configured as an AR-REPLICATOR for a BD MUST <bcp14>MUST</bcp14> advertise a
      Replicator-AR route for the BD and MAY <bcp14>MAY</bcp14> advertise a Regular-IR route. The
      advertisement of the Replicator-AR route will indicate to the AR-LEAFs what which
      outer IP DA, i.e., the which AR-IP, they need to use for IP encapsulated IP-encapsulated BM
      frames that use Assisted Replication forwarding mode. The AR-REPLICATOR
      will forward an IP encapsulated IP-encapsulated BM frame in Assisted Replication
      forwarding mode if the outer IP DA matches its AR-IP, AR-IP but will forward
      in Ingress Replication forwarding mode if the outer IP DA matches its
      IR-IP.</t>
      <t>In addition, this document also uses the Leaf Auto-Discovery (Leaf
      A-D) route defined in <xref
      target="I-D.ietf-bess-evpn-bum-procedure-updates"/> target="RFC9572" format="default"/> in case cases where the
      selective AR mode is used. An AR-LEAF MAY <bcp14>MAY</bcp14> send a Leaf A-D route in
      response to reception of a Replicator-AR route whose L flag is set. The
      Leaf Auto-Discovery A-D route is only used for selective AR AR, and the fields
      of such a route are set as follows:</t>

      <t><list hangIndent="2" style="empty">
          <t><list style="symbols">
              <t>Originating
          <ul spacing="normal">
            <li>Originating Router's IP Address is set to the advertising
              router's IP address (same (the same IP address used by the AR-LEAF in regular-IR Regular-IR
              routes). The Next-Hop Next Hop address is set to the IR-IP, which SHOULD <bcp14>SHOULD</bcp14>
              be the same IP address as the advertising router's IP address,
              when the NVE/PE originates the route, i.e., when the NVE/PE is
              not an ASBR as in section 10.2 of ASBR; see <xref target="RFC8365"/>.</t>

              <t>Route target="RFC8365" sectionFormat="of" section="10.2" format="default"/>.</li>
            <li>Route Key <xref target="RFC9572" format="default"/> is the "Route Type Specific" NLRI of the
              Replicator-AR route for which this Leaf Auto-Discovery A-D route is
              generated.</t>

              <t>The
              generated.</li>
            <li>The AR-LEAF constructs an IP-address-specific route-target, Route Target,
              analogously to <xref
              target="I-D.ietf-bess-evpn-bum-procedure-updates"/>, target="RFC9572" format="default"/>, by placing
              the IP address carried in the Next-Hop Next Hop field of the received
              Replicator-AR route in the Global Administrator field of the
              Community,
              extended community, with the Local Administrator field of this Community extended community
              set to 0, and setting the Extended Communities attribute of the
              Leaf Auto-Discovery A-D route to that Community. extended community. The same
              IP-address-specific import route-target Route Target is auto-configured by
              the AR-REPLICATOR that sent the Replicator-AR route, in order to
              control the acceptance of the Leaf Auto-Discovery routes.</t>

              <t>The A-D routes.</li>
            <li>The Leaf Auto-Discovery A-D route MUST <bcp14>MUST</bcp14> include the PMSI Tunnel
              attribute
              Attribute with the Tunnel Type set to AR Assisted Replication Tunnel (<xref
              target="sect-11"/>), target="sect-11" format="default"/>), T (AR role (Assisted Replication type) set to AR-LEAF AR-LEAF, and the
              Tunnel Identifier set to the IP address of the advertising
              AR-LEAF. The PMSI Tunnel attribute MUST Attribute <bcp14>MUST</bcp14> carry a
              downstream-assigned MPLS label or VNI that is used by the
              AR-REPLICATOR to send traffic to the AR-LEAF.</t>
            </list></t>
        </list></t> AR-LEAF.</li>
          </ul>

      <t>Each AR-enabled node understands and process processes the T
      (Assisted-Replication
      (Assisted Replication type) field in the PMSI Tunnel Attribute (Flags
      field) of the routes, routes and MUST <bcp14>MUST</bcp14> signal the corresponding type
      (AR-REPLICATOR or AR-LEAF type) according to its administrative choice.
      An NVE/PE following this specification is not expected to set the
      Assisted-Replication
      Assisted Replication Type field to decimal 3 (which is a RESERVED
      value). If a route with the AR type Assisted Replication Type field set to decimal 3 is received
      by an AR-REPLICATOR or AR-LEAF, the router will process the route as a
      Regular-IR route advertised by an RNVE.</t>
      <t>Each node attached to the BD may understand and process the BM/U
      flags (Pruned-Flood-Lists (PFLs flags). Note that these BM/U flags may be used
      to optimize the delivery of multi-destination traffic and traffic; their use
      SHOULD
      <bcp14>SHOULD</bcp14> be an administrative choice, choice and independent of the AR role. When
      the Pruned-Flood-List PFL capability is enabled, the BM/U flags can be used
      with the Regular-IR, Replicator-AR Replicator-AR, and Leaf Auto-Discovery A-D routes.</t>
      <t>Non-optimized Ingress Replication ingress replication NVEs/PEs will be unaware of the new
      PMSI Tunnel Attribute flag definition as well as the new Tunnel Type tunnel type
      (AR), i.e., non-upgraded NVEs/PEs will ignore the information contained
      in the flags Flags field or an unknown Tunnel Type tunnel type (type AR in this case) for
      any Inclusive Multicast Ethernet Tag route.</t>
    </section>
    <section anchor="sect-5"
             title="Non-Selective Assisted-Replication numbered="true" toc="default">
      <name>Non-selective Assisted Replication (AR) Solution Description"> Description</name>
      <t><xref target="ure-optimized-ir-scenario"/> target="ure-optimized-ir-scenario" format="default"/> illustrates an example
      Network Virtualization Overlay
      NVO network where the non-selective AR
      function is enabled. Three different roles are defined for a given BD:
      AR-REPLICATOR, AR-LEAF AR-LEAF, and RNVE (Regular NVE). RNVE. The solution is called
      "non-selective" because the chosen AR-REPLICATOR for a given flow MUST <bcp14>MUST</bcp14>
      replicate the BM traffic to all the NVE/PEs NVEs/PEs in the BD except for the
      source NVE/PE. Network Virtualization Overlay NVO tunnels, i.e., IP tunnels,
      exist among all the PEs and NVEs in the diagram. The PEs and NVEs in the
      diagram have Tenant Systems TSs or Virtual Machines VMs connected to their
      Attachment Circuits.</t>
      ACs.</t>
      <figure anchor="ure-optimized-ir-scenario"
              title="Non-Selective anchor="ure-optimized-ir-scenario">
        <name>Non-selective AR scenario">
        <artwork><![CDATA[ Scenario</name>
        <artwork name="" type="" align="left" alt=""><![CDATA[
                        (           )
                       (_    WAN    _)
                    +---(_         _)----+
                    |     (_      _)     |
              PE1   |                PE2 |
             +------+----+          +----+------+
        TS1--+  (BD-1)   |          |  (BD-1)   +--TS2
             |REPLICATOR |          |REPLICATOR |
             +--------+--+          +--+--------+
                      |                |
                   +--+----------------+--+
                   |                      |
                   |                      |
              +----+ VXLAN/nvGRE/MPLSoGRE VXLAN/NVGRE/MPLSoGRE +----+
              |    |      IP Fabric       |    |
              |    |                      |    |
    NVE1      |    +-----------+----------+    |      NVE3
    Hypervisor|          TOR          ToR   |  NVE2         |Hypervisor
    +---------+-+        +-----+-----+       +-+---------+
    |  (BD-1)   |        |  (BD-1)   |       |  (BD-1)   |
    |    LEAF   |        |   RNVE    |       |    LEAF   |
    +--+-----+--+        +--+-----+--+       +--+-----+--+
       |     |              |     |             |     |
      VM11  VM12           TS3   TS4           VM31  VM32
]]></artwork>
      </figure>
      <t>In AR BDs BDs, such as BD-1 in the example, <xref target="ure-optimized-ir-scenario"/>, BM (Broadcast and Multicast)
      traffic between two NVEs may follow a different path than unicast
      traffic. This solution recommends the replication of BM traffic through the
      AR-REPLICATOR node, whereas unknown/known unicast traffic will be delivered
      directly from the source node to the destination node without being
      replicated by any intermediate node.</t>
      <t>Note that known unicast forwarding is not impacted by this solution,
      i.e., unknown unicast SHALL traffic <bcp14>SHALL</bcp14> follow the same path as known unicast
      traffic.</t>
      <section anchor="sect-5.1"
               title="Non-selective numbered="true" toc="default">
        <name>Non-selective AR-REPLICATOR Procedures"> Procedures</name>
        <t>An AR-REPLICATOR is defined as an NVE/PE capable of replicating
        incoming BM traffic received on an overlay tunnel to other overlay
        tunnels and local Attachment Circuits. ACs. The AR-REPLICATOR signals its
        role in the control plane and understands where the other roles
        (AR-LEAF nodes, RNVEs RNVEs, and other AR-REPLICATORs) are located. A given
        AR-enabled BD service may have zero, one one, or more AR-REPLICATORs. In
        our example in <xref target="ure-optimized-ir-scenario"/>, target="ure-optimized-ir-scenario" format="default"/>, PE1 and PE2
        are defined as AR-REPLICATORs. The following considerations apply to
        the AR-REPLICATOR role:</t>

        <t><list hangIndent="3" style="letters">
            <t hangText="">The
        <ol spacing="normal" type="a"><li>The AR-REPLICATOR role SHOULD <bcp14>SHOULD</bcp14> be an administrative
            choice in any NVE/PE that is part of an AR-enabled BD. This
            administrative option to enable AR-REPLICATOR capabilities MAY <bcp14>MAY</bcp14> be
            implemented as a system level system-level option as opposed to as a per-BD
            option.</t>

            <t hangText="">An
            option.</li>
          <li>An AR-REPLICATOR MUST <bcp14>MUST</bcp14> advertise a Replicator-AR
            route and MAY <bcp14>MAY</bcp14> advertise a Regular-IR route. The AR-REPLICATOR MUST
            NOT <bcp14>MUST
            NOT</bcp14> generate a Regular-IR route if it does not have local
            attachment circuits (AC).
            ACs. If the Regular-IR route is advertised,
            the Assisted-Replication Assisted Replication Type field of the Regular-IR route MUST <bcp14>MUST</bcp14>
            be set to zero.</t>

            <t hangText="">The 0.</li>
          <li>The Replicator-AR and Regular-IR routes are
            generated according to <xref target="sect-4"/>. target="sect-4" format="default"/>. The AR-IP and
            IR-IP are different IP addresses owned by the AR-REPLICATOR.</t>

            <t hangText="">When AR-REPLICATOR.</li>
          <li>
            <t>When a node defined as an AR-REPLICATOR receives a BM
            packet on an overlay tunnel, it will do a tunnel destination IP
            address lookup and apply the following procedures: <list
                style="symbols">
                <t>If </t>
            <ul spacing="normal">
              <li>If the destination IP address is the AR-REPLICATOR IR-IP
                Address
                address, the node will process the packet normally as discussed in <xref
                target="RFC7432"/>.</t>

                <t>If target="RFC7432" format="default"/>.</li>
              <li>If the destination IP address is the AR-REPLICATOR AR-IP
                Address
                address, the node MUST <bcp14>MUST</bcp14> replicate the packet to local Attachment
                Circuits ACs
                and overlay tunnels (excluding the overlay tunnel to
                the source of the packet). When replicating to remote
                AR-REPLICATORs
                AR-REPLICATORs, the tunnel destination IP address will be an
                IR-IP. That This will be an indication for indicate to the remote AR-REPLICATOR
                that it MUST NOT <bcp14>MUST NOT</bcp14> replicate to overlay tunnels. The tunnel
                source IP address used by the AR-REPLICATOR MUST <bcp14>MUST</bcp14> be its IR-IP
                when replicating to AR-REPLICATOR or AR-LEAF nodes.</t>
              </list></t>
          </list>An nodes.</li>
            </ul>
          </li>
        </ol>

        <t>An AR-REPLICATOR MUST <bcp14>MUST</bcp14> follow a data path implementation
        compatible with the following rules:</t>

        <t><list style="symbols">
            <t>The
        <ul spacing="normal">
          <li>The AR-REPLICATORs will build a flooding list composed of
            Attachment Circuits
            ACs and overlay tunnels to remote nodes in the BD.
            Some of those overlay tunnels MAY <bcp14>MAY</bcp14> be flagged as non-BM receivers
            based on the BM flag received from the remote nodes in the BD.</t>

            <t>When BD.</li>
          <li>When an AR-REPLICATOR receives a BM packet on an Attachment
            Circuit, AC,
            it will forward the BM packet to its flooding list
            (including local Attachment Circuits ACs and remote NVE/PEs), NVEs/PEs), skipping
            the non-BM overlay tunnels.</t> tunnels.</li>
          <li>
            <t>When an AR-REPLICATOR receives a BM packet on an overlay
            tunnel, it will check the destination IP address of the underlay
            IP header and: <list style="symbols">
                <t>If and:</t>
            <ul spacing="normal">
              <li>If the destination IP address matches its IR-IP, the
                AR-REPLICATOR will skip all the overlay tunnels from the
                flooding list, i.e. i.e., it will only replicate to local Attachment
                Circuits. ACs.
                This is the regular Ingress Replication ingress replication behavior
                described in <xref target="RFC7432"/>.</t>

                <t>If target="RFC7432" format="default"/>.</li>
              <li>If the destination IP address matches its AR-IP, the
                AR-REPLICATOR MUST <bcp14>MUST</bcp14> forward the BM packet to its flooding list
                (ACs and overlay tunnels) tunnels), excluding the non-BM overlay
                tunnels. The AR-REPLICATOR will ensure that the traffic is not sent
                back to the originating AR-LEAF.</t>

                <t>If AR-LEAF.</li>
              <li>If the encapsulation is MPLSoGRE or MPLSoUDP and the
                received BD label that the AR-REPLICATOR advertised in the
                Replicator-AR route is not at the bottom of the stack, the
                AR-REPLICATOR MUST <bcp14>MUST</bcp14> copy the all the labels below the BD label
                and propagate them when forwarding the packet to the egress
                overlay tunnels.</t>
              </list></t> tunnels.</li>
            </ul>
          </li>
          <li>
            <t>The AR-REPLICATOR/LEAF nodes will build an Unknown unknown unicast
            flood-list
            flooding list composed of Attachment Circuits ACs and overlay tunnels to
            the IR-IP Addresses addresses of the remote nodes in the BD. Some of those
            overlay tunnels MAY <bcp14>MAY</bcp14> be flagged as non-U (Unknown (unknown unicast)
            receivers based on the U flag received from the remote nodes in
            the BD.<list style="symbols">
                <t>When BD.</t>
            <ul spacing="normal">
              <li>When an AR-REPLICATOR/LEAF receives an unknown unicast
                packet on an Attachment Circuit, AC, it will forward the unknown
                unicast packet to its flood-list, flooding list, skipping the non-U overlay
                tunnels.</t>

                <t>When
                tunnels.</li>
              <li>When an AR-REPLICATOR/LEAF receives an unknown unicast
                packet on an overlay tunnel, it will forward the unknown
                unicast packet to its local Attachment Circuits ACs and never to
                an overlay tunnel. This is the regular Ingress Replication ingress replication
                behavior described in <xref target="RFC7432"/>.</t>
              </list></t>
          </list></t> target="RFC7432" format="default"/>.</li>
            </ul>
          </li>
        </ul>
      </section>
      <section anchor="sect-5.2" title="Non-Selective numbered="true" toc="default">
        <name>Non-selective AR-LEAF Procedures</name>
        <t>An AR-LEAF Procedures">
        <t>AR-LEAF is defined as an NVE/PE that - that, given its poor replication
        performance -
        performance, sends all the BM traffic to an AR-REPLICATOR that can
        replicate the traffic further on its behalf. It MAY <bcp14>MAY</bcp14> signal its AR-LEAF
        capability in the control plane and understands where the other roles
        are located (AR-REPLICATOR (AR-REPLICATORs and RNVEs). A given service can have zero,
        one
        one, or more AR-LEAF nodes. In <xref target="ure-optimized-ir-scenario"/>
        shows target="ure-optimized-ir-scenario" format="default"/>,
        NVE1 and NVE3 (both residing in hypervisors) acting act as AR-LEAF. AR-LEAF nodes.
        The following considerations apply to the AR-LEAF role:</t>

        <t><list hangIndent="3" style="letters">
            <t hangText="">The
        <ol spacing="normal" type="a"><li>The AR-LEAF role SHOULD <bcp14>SHOULD</bcp14> be an administrative choice
            in any NVE/PE that is part of an AR-enabled BD. This
            administrative option to enable AR-LEAF capabilities MAY <bcp14>MAY</bcp14> be
            implemented as a system level system-level option as opposed to as a per-BD
            option.</t>

            <t hangText="">In
            option.</li>
          <li>In this non-selective AR solution, the AR-LEAF MUST <bcp14>MUST</bcp14>
            advertise a single Regular-IR inclusive multicast Inclusive Multicast Ethernet Tag route as described in
            <xref target="RFC7432"/>. target="RFC7432" format="default"/>. The AR-LEAF SHOULD <bcp14>SHOULD</bcp14> set the
            Assisted-Replication
            Assisted Replication Type field to AR-LEAF. Note that although this
            field does not make any difference for affect the remote nodes when creating an EVPN destination
            to the AR-LEAF, this field is useful
            for an easy from the standpoint of ease of
            operation and troubleshooting of the BD.</t>

            <t hangText="">In BD.</li>
          <li>
            <t>In a BD where there are no AR-REPLICATORs due to
            the AR-REPLICATORs being down or reconfigured, the AR-LEAF MUST <bcp14>MUST</bcp14>
            use regular Ingress Replication, ingress replication based on the remote Regular-IR
            Inclusive Multicast Routes Ethernet Tag routes as described in <xref
            target="RFC7432"/>. target="RFC7432" format="default"/>. This may happen in the following cases: <list
                style="symbols">
                <t>The </t>
            <ul spacing="normal">
              <li>The AR-LEAF has a list of AR-REPLICATORs for the BD, but it
                detects that all the AR-REPLICATORs for the BD are down (via
                next-hop tracking in the IGP or any some other detection
                mechanism).</t>

                <t>The
                mechanism).</li>
              <li>The AR-LEAF receives updates from all the former
                AR-REPLICATORs containing a non-REPLICATOR AR type in the
                Inclusive Multicast Etherner Ethernet Tag routes.</t>

                <t>The routes.</li>
              <li>The AR-LEAF never discovered an AR-REPLICATOR for the
                BD.</t>
              </list></t>

            <t hangText="">In
                BD.</li>
            </ul>
          </li>
          <li>
            <t>In a service where there is are one or more
            AR-REPLICATORs (based on the received Replicator-AR routes for the
            BD), the AR-LEAF can locally select which AR-REPLICATOR it sends
            the BM traffic to: <list style="symbols">
                <t>A to:</t>
            <ul spacing="normal">
              <li>A single AR-REPLICATOR MAY <bcp14>MAY</bcp14> be selected for all the BM
                packets received on the AR-LEAF attachment circuits (ACs) ACs for
                a given BD. This selection is a local decision and it does not
                have to match other AR-LEAFs' selections within the same
                BD.</t>

                <t>An
                BD.</li>
              <li>An AR-LEAF MAY <bcp14>MAY</bcp14> select more than one AR-REPLICATOR and do
                either per-flow or per-BD load balancing.</t>

                <t>In balancing.</li>
              <li>In the case of a failure of the selected AR-REPLICATOR, another
                AR-REPLICATOR SHOULD <bcp14>SHOULD</bcp14> be selected by the AR-LEAF.</t>

                <t>When AR-LEAF.</li>
              <li>When an AR-REPLICATOR is selected for a given flow or BD,
                the AR-LEAF MUST <bcp14>MUST</bcp14> send all the BM packets targeted to that
                AR-REPLICATOR using the forwarding information given by the
                Replicator-AR route for the chosen AR-REPLICATOR, with tunnel
                type Tunnel
                Type = 0x0A (AR tunnel). The underlay destination IP address
                MUST
                <bcp14>MUST</bcp14> be the AR-IP advertised by the AR-REPLICATOR in the
                Replicator-AR route.</t>

                <t>An route.</li>
              <li>An AR-LEAF MAY <bcp14>MAY</bcp14> change the AR-REPLICATOR(s) selection
                dynamically, of AR-REPLICATOR(s)
                dynamically due to an administrative or policy configuration
                change.</t>

                <t>AR-LEAF
                change.</li>
              <li>AR-LEAF nodes SHALL <bcp14>SHALL</bcp14> send service-level BM control plane
                packets
                packets, following the procedures for regular Ingress Replication procedures. ingress replication. An
                example would be IGMP, MLD Multicast Listener Discovery (MLD), or PIM multicast
packets, and and, in
                general
                general, any packets using link-local scope multicast IPv4 or
                IPv6 packets. The AR-REPLICATORs MUST NOT <bcp14>MUST NOT</bcp14> replicate these
                control plane packets to other overlay tunnels tunnels, since they will
                use the regular IR-IP Address.</t>
              </list></t>

            <t hangText="">The address.</li>
            </ul>
          </li>
	  <li>The use of an AR-REPLICATOR-activation-timer (in
            seconds, with a default value is of 3) on the AR-LEAF nodes is RECOMMENDED. <bcp14>RECOMMENDED</bcp14>.
            Upon receiving a new Replicator-AR route where the AR-REPLICATOR
            is selected, the AR-LEAF will run a timer before programming the
            new AR-REPLICATOR. In the case of a new newly added AR-REPLICATOR, AR-REPLICATOR or in
            case the if
            an AR-REPLICATOR reboots, this timer will give the
            AR-REPLICATOR some time to program the AR-LEAF nodes before the
            AR-LEAF sends BM traffic. The AR-REPLICATOR-activation-timer
            SHOULD
            <bcp14>SHOULD</bcp14> be configurable in seconds, and its value needs to account for the
            time it takes for the AR-LEAF Regular-IR inclusive multicast Inclusive Multicast Ethernet Tag route
            to get to the AR-REPLICATOR and be programmed. While the
            AR-REPLICATOR-activation-time
            AR-REPLICATOR-activation-timer is running, the AR-LEAF node will
            use regular ingress replication.</t>

            <t>If replication.</li>
          <li>If the AR-LEAF has selected an AR-REPLICATOR, it is a matter of
            local policy whether
            or not to change to a new preferred AR-REPLICATOR for the existing BM traffic flows.</t>
          </list>An flows is a matter of local policy.</li>
        </ol>
        <t>An AR-LEAF MUST <bcp14>MUST</bcp14> follow a data path implementation compatible
        with the following rules:</t>

        <t><list style="symbols">
        <ul spacing="normal">
          <li>
            <t>The AR-LEAF nodes will build two flood-lists:<list
                style="numbers">
                <t>Flood-list #1 - composed flooding lists:</t>
	    <dl>

	      <dt>Flooding list #1:
	      </dt>
	      <dd>Composed of Attachment Circuits ACs and an AR-REPLICATOR-set of
	      overlay tunnels. The AR-REPLICATOR-set is defined as one or more
	      overlay tunnels to the AR-IP Addresses addresses of the remote
	      AR-REPLICATOR(s) in the BD. The selection of more than one
	      AR-REPLICATOR is described in point d) item d. above and
                it is a local
	      AR-LEAF decision.</t>

                <t>Flood-list #2 - composed decision.
	      </dd>

	      <dt>Flooding list #2:
	      </dt>
	      <dd>Composed of Attachment Circuits ACs and overlay tunnels to the
	      remote IR-IP Addresses.</t>
              </list></t> addresses.
	      </dd>

</dl>

</li>
          <li>
            <t>When an AR-LEAF receives a BM packet on an Attachment Circuit, AC,
            it will check the AR-REPLICATOR-set:<list style="symbols">
                <t>If AR-REPLICATOR-set:</t>
            <ul spacing="normal">
              <li>If the AR-REPLICATOR-set is empty, the AR-LEAF MUST <bcp14>MUST</bcp14> send
                the packet to flood-list #2.</t>

                <t>If flooding list #2.</li>
              <li>If the AR-REPLICATOR-set is NOT empty, the AR-LEAF MUST <bcp14>MUST</bcp14>
                send the packet to flood-list flooding list #1, where only one of the
                overlay tunnels of the AR-REPLICATOR-set is used.</t>
              </list></t>

            <t>When used.</li>
            </ul>
          </li>
          <li>When an AR-LEAF receives a BM packet on an overlay tunnel, it
            will forward the BM packet to its local Attachment Circuits ACs and
            never to an overlay tunnel. This is the regular Ingress
            Replication ingress
            replication behavior described in <xref target="RFC7432"/>.</t>

            <t>AR-LEAF target="RFC7432" format="default"/>.</li>
          <li>AR-LEAF nodes process Unknown unknown unicast traffic in the same way
            AR-REPLICATORS do, as described in <xref target="sect-5.1"/>.</t>
          </list></t> target="sect-5.1" format="default"/>.</li>
        </ul>
      </section>

      <section anchor="sect-5.3" title="RNVE Procedures">
        <t>RNVE (Regular Network Virtualization Edge node) numbered="true" toc="default">
        <name>RNVE Procedures</name>
        <t>An RNVE is defined as an
        NVE/PE without AR-REPLICATOR or AR-LEAF capabilities that does Ingress
        Replication ingress
        replication as described in <xref target="RFC7432"/>. target="RFC7432" format="default"/>. The RNVE does
        not signal any AR role and is unaware of the AR-REPLICATOR/LEAF roles
        in the BD. The RNVE will ignore the Flags flags in the Regular-IR routes and
        will ignore the Replicator-AR routes (due to an unknown tunnel type in
        the PMSI Tunnel Attribute) and the Leaf Auto-Discovery A-D routes (due to
        the IP-address-specific route-target).</t> Route Target).</t>
        <t>This role provides EVPN EVPNs with the backwards backward compatibility required
        in optimized Ingress Replication ingress replication BDs. In <xref
        target="ure-optimized-ir-scenario"/> shows target="ure-optimized-ir-scenario" format="default"/>, NVE2 acts as an RNVE.</t>
      </section>
    </section>
    <section anchor="sect-6"
             title="Selective Assisted-Replication numbered="true" toc="default">
      <name>Selective Assisted Replication (AR) Solution Description"> Description</name>
      <t><xref target="selective-ar"/> target="selective-ar" format="default"/> is used to describe the selective AR
      solution.</t>
      <figure anchor="selective-ar" title="Selective anchor="selective-ar">
        <name>Selective AR scenario">
        <artwork><![CDATA[ Scenario</name>
        <artwork name="" type="" align="left" alt=""><![CDATA[
                        (           )
                       (_    WAN    _)
                    +---(_         _)----+
                    |     (_      _)     |
              PE1   |                PE2 |
             +------+----+          +----+------+
        TS1--+  (BD-1)   |          |  (BD-1)   +--TS2
             |REPLICATOR |          |REPLICATOR |
             +--------+--+          +--+--------+
                      |                |
                   +--+----------------+--+
                   |                      |
                   |                      |
              +----+ VXLAN/nvGRE/MPLSoGRE VXLAN/NVGRE/MPLSoGRE +----+
              |    |      IP Fabric       |    |
              |    |                      |    |
    NVE1      |    +-----------+----------+    |      NVE3
    Hypervisor|          TOR          ToR   |  NVE2         |Hypervisor
    +---------+-+        +-----+-----+       +-+---------+
    |  (BD-1)   |        |  (BD-1)   |       |  (BD-1)   |
    | LEAF-set1
    |LEAF-set-1 |        |LEAF-set-1 |       |LEAF-set-2 |
    +--+-----+--+        +--+-----+--+       +--+-----+--+
       |     |              |     |             |     |
      VM11  VM12           TS3   TS4           VM31  VM32
]]></artwork>
      </figure>

      <t>The solution is called "selective" because a given AR-REPLICATOR MUST <bcp14>MUST</bcp14>
      replicate the BM traffic to only the AR-LEAFs that requested the
      replication (as opposed to all the AR-LEAF nodes) and MUST <bcp14>MUST</bcp14> replicate the
      BM traffic to the RNVEs (if there are any). The same AR roles as those defined in
      Sections&nbsp;<xref target="sect-4" format="counter"/> and <xref target="sect-4"/> target="sect-5" format="counter"/> are used here, however here; however, the procedures are
      different.</t>
      <t>The Selective selective AR procedures create multiple AR-LEAF-sets in the EVPN
      BD,
      BD and build single-hop trees among AR-LEAFs of the same set
      (AR-LEAF-&gt;AR-REPLICATOR-&gt;AR-LEAF),
      (AR-LEAF-&gt;AR-REPLICATOR-&gt;AR-LEAF) and two-hop trees among
      AR-LEAFs of different sets
      (AR-LEAF-&gt;AR-REPLICATOR-&gt;AR-REPLICATOR-&gt;AR-LEAF). Compared to
      the Selective selective solution, the Non-Selective non-selective AR method assumes that all the
      AR-LEAFs of the BD are in the same set and always creates two-hop single-hop trees
      among AR-LEAFs. While the Selective selective solution is more efficient than the
      Non-Selective
      non-selective solution in multi-stage IP fabrics, the trade-off is
      additional signaling and an additional outer source IP address
      lookup.</t>
      <t>The following sub-sections subsections describe the differences in the procedures
      of AR-REPLICATOR/LEAFs
      for AR-REPLICATORs/LEAFs compared to the non-selective AR solution. There
      is
      are no change on the changes applicable to RNVEs.</t>
      <section anchor="sect-6.1" title="Selective numbered="true" toc="default">
        <name>Selective AR-REPLICATOR Procedures"> Procedures</name>
        <t>In our example in <xref target="selective-ar"/>, target="selective-ar" format="default"/>, PE1 and PE2 are
        defined as Selective selective AR-REPLICATORs. The following considerations
        apply to the Selective selective AR-REPLICATOR role:</t>

        <t><list style="letters">
            <t>The Selective
        <ol spacing="normal" type="a"><li>The selective AR-REPLICATOR capability SHOULD role <bcp14>SHOULD</bcp14> be an
            administrative choice in any NVE/PE that is part of an
            Assisted-Replication-enabled BD, as the AR role itself.
            AR-enabled BD. This
            administrative option MAY <bcp14>MAY</bcp14> be implemented as a system level system-level option
            as opposed to as a per-BD option.</t>

            <t>Each option.</li>
          <li>Each AR-REPLICATOR will build a list of AR-REPLICATOR, AR-LEAF AR-LEAF,
            and RNVE nodes. In spite of the 'Selective' "selective" administrative option,
            an AR-REPLICATOR MUST NOT <bcp14>MUST NOT</bcp14> behave as a Selective selective AR-REPLICATOR if
            at least one of the AR-REPLICATORs has the L flag NOT set. If at
            least one AR-REPLICATOR sends a Replicator-AR route with L=0 L = 0 (in
            the BD context), the rest of the AR-REPLICATORs will fall back to
            non-selective AR mode.</t> mode.</li>
          <li>
            <t>The Selective selective AR-REPLICATOR MUST <bcp14>MUST</bcp14> follow the procedures
            described in <xref target="sect-5.1"/>, target="sect-5.1" format="default"/>, except for the following
            differences:<list style="symbols">
                <t>The Replicator-AR route MUST include L=1 (Leaf Information
                Required) in
            differences:</t>
            <ul spacing="normal">
              <li>The AR-REPLICATOR <bcp14>MUST</bcp14> have the L flag set to 1
                when advertising the Replicator-AR route. This flag is used by the
                AR-REPLICATORs to advertise their 'selective' "selective" AR-REPLICATOR
                capabilities. In addition, the AR-REPLICATOR auto-configures
                its IP-address-specific import route-target Route Target as described in
                the third bullet of the procedures for Leaf Auto-Discovery
                route A-D
                routes in <xref target="sect-4"/>.</t>

                <t>The target="sect-4" format="default"/>.</li>
              <li>The AR-REPLICATOR will build a 'selective' "selective" AR-LEAF-set with
                the list of nodes that requested replication to its own AR-IP.
                For instance, assuming that NVE1 and NVE2 advertise a Leaf
                Auto-Discovery
                A-D route with PE1's IP-address-specific
                route-target
                Route Target and NVE3 advertises a Leaf Auto-Discovery A-D route
                with PE2's IP-address-specific route-target, Route Target, PE1 will only add
                NVE1/NVE2 to its selective AR-LEAF-set for BD-1, BD-1 and exclude
                NVE3. Likewise, PE2 will only add NVE3 to its selective
                AR-LEAF-set for BD-1, BD-1 and exclude NVE1/NVE2.</t> NVE1/NVE2.</li>
              <li>
                <t>When a node defined and operating as a Selective selective
                AR-REPLICATOR receives a packet on an overlay tunnel, it will
                do a tunnel destination IP lookup lookup, and if the destination IP
                address is the AR-REPLICATOR AR-IP Address, address, the node MUST <bcp14>MUST</bcp14>
                replicate the packet to:<list style="symbols">
                    <t>local Attachment Circuits</t>

                    <t>overlay to:</t>
                <ul spacing="normal">
                  <li>Local ACs.</li>
                  <li>Overlay tunnels in the Selective selective AR-LEAF-set, excluding
                    the overlay tunnel to the source AR-LEAF.</t>

                    <t>overlay AR-LEAF.</li>
                  <li>Overlay tunnels to the RNVEs if the tunnel source IP
                    address is the IR-IP of an AR-LEAF. In any other case, the
                    AR-REPLICATOR MUST NOT <bcp14>MUST NOT</bcp14> replicate the BM traffic to remote
                    RNVEs. In other words, only the first-hop selective
                    AR-REPLICATOR will replicate to all the RNVEs.</t>

                    <t>overlay RNVEs.</li>
                  <li>Overlay tunnels to the remote Selective selective AR-REPLICATORs
                    if the tunnel source IP address (of the encapsulated
                    packet that arrived on the overlay tunnel) is an IR-IP of
                    its own AR-LEAF-set. In any other case, the AR-REPLICATOR
                    MUST NOT
                    <bcp14>MUST NOT</bcp14> replicate the BM traffic to remote
                    AR-REPLICATORs. When doing this replication, the tunnel
                    destination IP address is the AR-IP of the remote
                    Selective
                    selective AR-REPLICATOR. The tunnel destination IP address AR-IP
                    will be an indication for indicate to the remote Selective selective
                    AR-REPLICATOR that the packet needs further replication to
                    its AR-LEAFs.</t>
                  </list></t>
              </list></t>
          </list>A Selective AR-LEAFs.</li>
                </ul>
              </li>
            </ul>
          </li>
        </ol>
        <t>A selective AR-REPLICATOR data path implementation MUST <bcp14>MUST</bcp14> be
        compatible with the following rules:</t>

        <t><list style="symbols">
        <ul spacing="normal">
          <li>
            <t>The Selective selective AR-REPLICATORs will build two flood-lists:<list
                style="numbers">
                <t>Flood-list #1 - composed flooding lists:</t>

	    <dl>
	      <dt>Flooding list #1:
	      </dt>
	      <dd><t>Composed of Attachment Circuits ACs and overlay tunnels to the
	      remote nodes in the BD, always using the IR-IPs in the tunnel
	      destination IP addresses.</t>

                <t>Flood-list #2 - composed
	      </dd>

	      <dt>Flooding list #2:
	      </dt>
	      <dd><t>Composed of Attachment Circuits, ACs, a
                Selective AR-LEAF-set selective AR-LEAF-set, and
	      a Selective selective AR-REPLICATOR-set,
                where:<list style="symbols">
                    <t>The Selective where:</t>

		 <ul spacing="normal">
                  <li>The selective AR-LEAF-set is composed of the overlay
                  tunnels to the AR-LEAFs that advertise a Leaf
                    Auto-Discovery A-D
                  route for the local AR-REPLICATOR. This set is updated with
                  every Leaf Auto-Discovery A-D route received/withdrawn from a
                  new AR-LEAF.</t>

                    <t>The Selective AR-LEAF.</li>
                  <li>The selective AR-REPLICATOR-set is composed of the
                  overlay tunnels to all the AR-REPLICATORs that send a
                  Replicator-AR route with L=1. L = 1. The AR-IP addresses are used
                  as tunnel destination IP.</t>
                  </list></t>
              </list></t>

            <t>Some IP addresses.</li>
                </ul>

	      </dd>
</dl>

          </li>
          <li>Some of the overlay tunnels in the flood-lists MAY flooding lists <bcp14>MAY</bcp14> be flagged
            as non-BM receivers based on the BM flag received from the remote
            nodes in the routes.</t>

            <t>When routes.</li>
          <li>When a Selective selective AR-REPLICATOR receives a BM packet on an
            Attachment Circuit,
            AC, it MUST <bcp14>MUST</bcp14> forward the BM packet to its
            flood-list
            flooding list #1, skipping the non-BM overlay tunnels.</t> tunnels.</li>
          <li>
            <t>When a Selective selective AR-REPLICATOR receives a BM packet on an
            overlay tunnel, it will check the destination and source IPs of
            the underlay IP header and:<list style="symbols">
                <t>If and:</t>
            <ul spacing="normal">
              <li>If the destination IP address matches its AR-IP and the
                source IP address matches an IP of its own Selective selective
                AR-LEAF-set, the AR-REPLICATOR MUST <bcp14>MUST</bcp14> forward the BM packet to
                its flood-list flooding list #2, unless some AR-REPLICATOR within the BD has
                advertised L=0. L = 0. In the latter case, the node reverts back to
                non-selective mode
                Non-selective mode, and flood-list flooding list #1 MUST <bcp14>MUST</bcp14> be used. Non-BM
                overlay tunnels are skipped when sending BM packets.</t>

                <t>If packets.</li>
              <li>If the destination IP address matches its AR-IP and the
                source IP address does not match any IP address of its
                Selective
                selective AR-LEAF-set, the AR-REPLICATOR MUST <bcp14>MUST</bcp14> forward the BM
                packet to flood-list #2 but flooding list #2, skipping the AR-REPLICATOR-set.
                Non-BM overlay tunnels are skipped when sending BM
                packets.</t>

                <t>If
                packets.</li>
              <li>If the destination IP address matches its IR-IP, the
                AR-REPLICATOR MUST <bcp14>MUST</bcp14> use flood-list flooding list #1 but MUST <bcp14>MUST</bcp14> skip all the
                overlay tunnels from the flooding list, i.e. i.e., it will only
                replicate to local Attachment Circuits. ACs. This is the regular-IR regular ingress replication
                behavior described in <xref target="RFC7432"/>. target="RFC7432" format="default"/>. Non-BM overlay
                tunnels are skipped when sending BM packets.</t>
              </list></t>

            <t>In packets.</li>
            </ul>
          </li>
          <li>In any case, the AR-REPLICATOR ensures that the traffic is not sent
            back to the originating source. If the encapsulation is MPLSoGRE
            or MPLSoUDP and the received BD label (the label that the
            AR-REPLICATOR advertised in the Replicator-AR route) is not at the
            bottom of the stack, the AR-REPLICATOR MUST <bcp14>MUST</bcp14> copy the rest of the
            labels when forwarding them to the egress overlay tunnels.</t>
          </list></t> tunnels.</li>
        </ul>
      </section>
      <section anchor="sect-6.2" title="Selective numbered="true" toc="default">
        <name>Selective AR-LEAF Procedures"> Procedures</name>

        <t>A Selective selective AR-LEAF chooses a single Selective selective AR-REPLICATOR per BD
        and:</t>

        <t><list style="symbols">
            <?rfc subcompact="yes"?>

            <t>Sends
        <ul spacing="normal">
          <li>Sends all the BD's BM traffic to that AR-REPLICATOR and</t>

            <t>Expects and</li>
          <li>Expects to receive all the BM traffic for a given BD from the
            same AR-REPLICATOR (except for the BM traffic from the RNVEs,
            which comes directly from the RNVEs)</t>

            <?rfc subcompact="no"?>
          </list></t> RNVEs)</li>
        </ul>
        <t>In the example of in <xref target="selective-ar"/>, target="selective-ar" format="default"/>, we consider
        NVE1/NVE2/NVE3 as Selective selective AR-LEAFs. NVE1 selects PE1 as its
        Selective
        selective AR-REPLICATOR. If that is so, NVE1 will send all its BM
        traffic for BD-1 to PE1. If other AR-LEAF/REPLICATORs AR-LEAFs/REPLICATORs send BM traffic,
        NVE1 will receive that traffic from PE1. These are the differences in
        the behavior of a Selective A selective AR-LEAF compared to and a non-selective
        AR-LEAF:<list style="letters">
            <t>The AR-LEAF role behave differently, as follows:</t>
        <ol spacing="normal" type="a"><li>The selective capability SHOULD AR-LEAF role <bcp14>SHOULD</bcp14> be an
            administrative choice in any NVE/PE that is part of an
            Assisted-Replication-enabled
            AR-enabled BD. This administrative option to
            enable AR-LEAF capabilities MAY <bcp14>MAY</bcp14> be implemented as a system level system-level option as opposed to as a per-BD option.</t>

            <t>The option.</li>
          <li>The AR-LEAF MAY <bcp14>MAY</bcp14> advertise a Regular-IR route if there are RNVEs
            in the BD. The Selective selective AR-LEAF MUST <bcp14>MUST</bcp14> advertise a Leaf
            Auto-Discovery
            A-D route after receiving a Replicator-AR route with
            L=1.
            L = 1. It is RECOMMENDED <bcp14>RECOMMENDED</bcp14> that the Selective selective AR-LEAF waits wait for a period specified by an
            AR-LEAF-join-wait-timer (in seconds, with a default value is of 3) before
            sending the Leaf Auto-Discovery A-D route, so that the AR-LEAF can
            collect all the Replicator-AR routes for the BD before advertising
            the Leaf Auto-Discovery A-D route. If the Replicator-AR route with L=1 L = 1
            is withdrawn, the corresponding Leaf Auto-Discovery A-D route is
            withdrawn too.</t> too.</li>
          <li>
            <t>In a service where there is more than one Selective
            AR-REPLICATOR selective
            AR-REPLICATOR, the Selective selective AR-LEAF MUST <bcp14>MUST</bcp14> locally select a single
            Selective
            selective AR-REPLICATOR for the BD. Once selected: <list
                style="symbols">
                <t>The Selective </t>
            <ul spacing="normal">
              <li>The selective AR-LEAF MUST <bcp14>MUST</bcp14> send a Leaf Auto-Discovery route A-D route,
                including the Route-key route key and IP-address-specific route-target Route Target
                of the selected AR-REPLICATOR.</t>

                <t>The Selective AR-REPLICATOR.</li>
              <li>The selective AR-LEAF MUST <bcp14>MUST</bcp14> send all the BM packets received
                on the attachment circuits (ACs) ACs for a given BD to that
                AR-REPLICATOR.</t>

                <t>In
                AR-REPLICATOR.</li>
              <li>In the case of a failure on of the selected AR-REPLICATOR
                (detected when the Replicator-AR route becomes infeasible as
                the
                a result of any of the underlying BGP mechanisms), another
                AR-REPLICATOR will be selected and a new Leaf Auto-Discovery A-D
                update will be issued for the new AR-REPLICATOR. This new
                route will update the selective list in the new Selective selective
                AR-REPLICATOR. In the case of failure of the active Selective selective
                AR-REPLICATOR, it is RECOMMENDED for <bcp14>RECOMMENDED</bcp14> that the Selective selective AR-LEAF to
                revert to Ingress Replication ingress replication behavior for a timer an
                AR-REPLICATOR-activation-timer (in seconds, with a default value is of
                3) to mitigate the traffic impact. When the timer expires, the
                Selective
                selective AR-LEAF will resume its AR mode with the new
                Selective
                selective AR-REPLICATOR. The AR-REPLICATOR-activation-timer
                MAY
                <bcp14>MAY</bcp14> be the same configurable parameter as the parameter discussed in <xref
                target="sect-5.2"/>.</t>

                <t>A Selective target="sect-5.2" format="default"/>.</li>
              <li>A selective AR-LEAF MAY <bcp14>MAY</bcp14> change the AR-REPLICATOR(s) selection dynamically, of AR-REPLICATOR(s)
                dynamically due to an administrative or policy
                configuration change.</t>
              </list></t>
          </list></t> change.</li>
            </ul>
          </li>
        </ol>
        <t>All the AR-LEAFs in a BD are expected to be configured as either
        selective or non-selective. A mix of selective and non-selective
        AR-LEAFs SHOULD NOT <bcp14>SHOULD NOT</bcp14> coexist in the same BD. In case there is If a
        non-selective AR-LEAF, AR-LEAF is present, its BM traffic sent to a selective
        AR-REPLICATOR will not be replicated to other AR-LEAFs that are not in
        its Selective selective AR-LEAF-set.</t>
        <t>A Selective selective AR-LEAF MUST <bcp14>MUST</bcp14> follow a data path implementation
        compatible with the following rules:<list style="symbols"> rules:</t>
        <ul spacing="normal">
          <li>
            <t>The Selective selective AR-LEAF nodes will build two flood-lists:<list
                style="numbers">
                <t>Flood-list #1 - composed flooding lists:</t>

<dl>

<dt>Flooding list #1:
</dt>
<dd>Composed of Attachment Circuits ACs and the overlay tunnel to the selected
AR-REPLICATOR (using the AR-IP as the tunnel destination IP address).</t>

                <t>Flood-list #2 - composed address).
</dd>

<dt>Flooding list #2:
</dt>
<dd>Composed of Attachment Circuits ACs and overlay tunnels to the remote IR-IP addresses.</t>
              </list></t>

            <t>Some
addresses.
</dd>

</dl>

          </li>
          <li>Some of the overlay tunnels in the flood-lists MAY flooding lists <bcp14>MAY</bcp14> be flagged
            as non-BM receivers based on the BM flag received from the remote
            nodes in the routes.</t>

            <t>When routes.</li>
          <li>When an AR-LEAF receives a BM packet on an Attachment Circuit, AC,
            it will check to see if there an AR-REPLICATOR was selected; if one is any selected AR-REPLICATOR. If there is,
            flood-list found,
            flooding list #1 MUST <bcp14>MUST</bcp14> be used. Otherwise, flood-list flooding list #2 MUST <bcp14>MUST</bcp14> be used.
            Non-BM overlay tunnels are skipped when sending BM packets.</t>

            <t>When packets.</li>
          <li>When an AR-LEAF receives a BM packet on an overlay tunnel, it
            MUST
            <bcp14>MUST</bcp14> forward the BM packet to its local Attachment Circuits ACs and
            never to an overlay tunnel. This is the regular Ingress
            Replication ingress
            replication behavior described in <xref target="RFC7432"/>.</t>
          </list></t> target="RFC7432" format="default"/>.</li>
        </ul>
      </section>
    </section>
    <section anchor="sect-7" title="Pruned-Flood-Lists (PFL)"> numbered="true" toc="default">
      <name>Pruned Flooding Lists (PFLs)</name>
      <t>In addition to AR, the second optimization supported by this the ingress
replication optimization solution specified in this document
      is the ability for the of all the BD nodes to signal Pruned-Flood-Lists
      (PFL). PFLs. As described in <xref target="sect-4"/>, target="sect-4" format="default"/>, an EVPN node can signal
      a given value for the BM and U Pruned-Food-Lists PFLs flags in the
      Regular-IR, Replicator-AR Replicator-AR, or Leaf Auto-Discovery A-D routes, where:</t>

      <t><list style="symbols">
          <t>BM
      <ul spacing="normal">
        <li>BM is the Broadcast and Multicast flag. BM=1 BM = 1 means "prune-me" "prune me
          from the BM flood-list. BM=0 means flooding list". BM = 0 indicates regular behavior.</t>

          <t>U behavior.</li>
        <li>U is the Unknown flag. U=1 U = 1 means "prune-me" "prune me from the Unknown
          flood-list. U=0 means
          flooding list". U = 0 indicates regular behavior.</t>
        </list></t> behavior.</li>
      </ul>
      <t>The ability to signal and process these Pruned-Flood-Lists PFLs flags
      SHOULD
      <bcp14>SHOULD</bcp14> be an administrative choice. If a node is configured to process
      the Pruned-Flood-Lists PFLs flags, upon receiving a non-zero
      Pruned-Flood-Lists
      PFLs flag for a route, the an NVE/PE will add the
      corresponding flag to the created overlay tunnel in the flood-list. flooding list. When
      replicating a BM packet in the context of a flood-list, flooding list, the NVE/PE will
      skip the overlay tunnels marked with the flag BM=1, BM = 1, since the NVE/PE NVEs/PEs at
      the end of those tunnels are not expecting BM packets. Similarly, when
      replicating Unknown unknown unicast packets, the NVE/PE will skip the overlay
      tunnels marked with U=1.</t> U = 1.</t>
      <t>An NVE/PE not following this document or not configured for this
      optimization will ignore any of the received Pruned-Flood-Lists PFLs flags.
      An AR-LEAF or RNVE receiving BUM traffic on an overlay tunnel MUST <bcp14>MUST</bcp14>
      replicate the traffic to its local Attachment Circuits, ACs, regardless of
      the BM/U flags on the overlay tunnels.</t>
      <t>This optimization MAY <bcp14>MAY</bcp14> be used along with the Assisted-Replication Assisted Replication
      solution.</t>
      <section anchor="sect-7.1" title="A Pruned-Flood-List Example"> numbered="true" toc="default">
        <name>Example of a Pruned Flooding List</name>
        <t>In order to illustrate the use of the solution described in this
        document, PFLs solution, we will assume that BD-1 in <xref
        target="ure-optimized-ir-scenario"/> target="ure-optimized-ir-scenario" format="default"/> is optimized Ingress Replication ingress replication
        enabled and:</t>

        <t><list style="symbols">
            <t>PE1
        <ul spacing="normal">
          <li>PE1 and PE2 are administratively configured as AR-REPLICATORs, AR-REPLICATORs
            due to their high-performance replication capabilities. PE1 and
            PE2 will send a Replicator-AR route with BM/U flags = 00.</t> 00.</li>
          <li>
            <t>NVE1 and NVE3 are administratively configured as AR-LEAF nodes, nodes
            due to their low-performance software-based replication
            capabilities. They will advertise a Regular-IR route with type
            AR-LEAF. Assuming that both NVEs advertise all of the attached Virtual
            Machines VMs'
            MAC and IP addresses in EVPN EVPNs as soon as they come up, up and
            these NVEs do not have any Virtual Machines VMs interested in
            multicast applications, they will be configured to signal BM/U
            flags = 11 for BD-1. That is, neither NVE1 nor NVE3 are is interested
            in receiving BM or Unknown Unicast traffic since:<list
                style="symbols">
                <t>Their unknown unicast traffic, since:</t>
            <ul spacing="normal">
              <li>Their attached VMs (VM11, VM12, VM31, VM32) do not support
                multicast applications.</t>

                <t>Their applications.</li>
              <li>Their attached VMs will not receive ARP Requests. Proxy-ARP Proxy ARP
                <xref target="I-D.ietf-bess-evpn-proxy-arp-nd"/> target="RFC9161" format="default"/> on the remote
                NVE/PEs
                NVEs/PEs will reply to ARP Requests locally, and no other
                Broadcast
                broadcast traffic is expected.</t>

                <t>Their expected.</li>
              <li>Their attached VMs will not receive unknown unicast
                traffic, since the VMs' MAC and IP addresses are always
                advertised by EVPN EVPNs as long as the VMs are active.</t>
              </list></t>

            <t>NVE2 active.</li>
            </ul>
          </li>
          <li>NVE2 is optimized Ingress Replication ingress replication unaware; therefore therefore, it
            takes on the RNVE role in BD-1.</t>
          </list></t> BD-1.</li>
        </ul>
        <t>Based on the above assumptions assumptions, the following forwarding behavior
        will take place:</t>

        <t><list hangIndent="4" style="numbers">
            <t hangText="">Any
        <ol spacing="normal" type="1"><li>Any BM packets sent from VM11 will be sent to VM12
            and PE1. PE1 will then forward further the BM packets on to TS1, the WAN link,
            PE2
            PE2, and NVE2, NVE2 but not to NVE3. PE2 and NVE2 will replicate the BM
            packets to their local Attachment Circuits ACs, but we will avoid NVE3 will be prevented from
            having to replicate unnecessarily those BM packets to VM31 and
            VM32.</t>

            <t hangText="">Any
            VM32 unnecessarily.</li>
          <li>Any BM packets received on PE2 from the WAN will be
            sent to PE1 and NVE2, NVE2 but not to NVE1 and NVE3, sparing the two
            hypervisors from replicating unnecessarily to their local Virtual
            Machines. VMs.
            PE1 and NVE2 will replicate to their local Attachment
            Circuits only.</t>

            <t hangText="">Any Unknown ACs
            only.</li>
          <li>Any unknown unicast packet sent from VM31 will be
            forwarded by NVE3 to NVE2, PE1 PE1, and PE2 but not to NVE1. The solution
            avoids the
            prevents unnecessary replication to NVE1, since the destination
            of the unknown traffic cannot be at NVE1.</t>

            <t hangText="">Any Unknown NVE1.</li>
          <li>Any unknown unicast packet sent from TS1 will be
            forwarded by PE1 to the WAN link, PE2 PE2, and NVE2 but not to NVE1 and
            NVE3, since the target of the unknown traffic cannot be at those
            NVEs.</t>
          </list></t> NVE1 or
            NVE3.</li>
        </ol>

      </section>
    </section>
    <section anchor="sect-8"
             title="AR numbered="true" toc="default">
      <name>AR Procedures for Single-IP AR-REPLICATORS"> AR-REPLICATORS</name>
      <t>The procedures explained in sections <xref target="sect-5"/> Sections&nbsp;<xref target="sect-5" format="counter"/> and
      <xref target="sect-6"/> target="sect-6" format="counter"/> assume that the AR-REPLICATOR can use two local
      routable IP addresses to terminate and originate Network Virtualization
      Overlay NVO
      tunnels, i.e. i.e., IR-IP and AR-IP addresses. This is usually the
      case for PE-based AR-REPLICATOR nodes.</t>
      <t>In some cases, the AR-REPLICATOR node does not support more than one
      IP address to terminate and originate Network Virtualization Overlay NVO
      tunnels, i.e. i.e., the IR-IP and AR-IP are the same IP addresses. This may be
      the case in some software-based or low-end AR-REPLICATOR nodes. If this
      is the case, the procedures provided in sections <xref target="sect-5"/> Sections&nbsp;<xref target="sect-5" format="counter"/> and <xref target="sect-6"/> MUST target="sect-6" format="counter"/> <bcp14>MUST</bcp14> be modified in the following way:</t>

      <t><list style="symbols">
          <t>The
      <ul spacing="normal">
        <li>The Replicator-AR routes generated by the AR-REPLICATOR use an
          AR-IP that will match its IR-IP. In order to differentiate the data
          plane packets that need to use Ingress Replication ingress replication from the packets
          that must use Assisted Replication forwarding mode, the
          Replicator-AR route MUST <bcp14>MUST</bcp14> advertise a different VNI/VSID than the one
          used by the Regular-IR route. For instance, the AR-REPLICATOR will
          advertise an AR-VNI along with the Replicator-AR route and an IR-VNI along
          with the Regular-IR route. Since both routes have the same key,
          different Route Distinguishers are needed in each route.</t>

          <t>An route.</li>
        <li>An AR-REPLICATOR will perform Ingress Replication forwarding mode or Assisted
          Replication forwarding mode for the incoming Overlay overlay packets based
          on an ingress VNI lookup, lookup as opposed to the tunnel IP DA lookup.
          Note that, that when replicating to remote AR-REPLICATOR nodes, the use
          of the IR-VNI or AR-VNI advertised by the egress node will determine
          the
          whether Ingress Replication forwarding mode or Assisted Replication forwarding mode is used at
          the subsequent AR-REPLICATOR.</t>
        </list></t> AR-REPLICATOR.</li>
      </ul>
      <t>The rest of the procedures will follow what is those described in sections
      <xref target="sect-5"/>
      Sections&nbsp;<xref target="sect-5" format="counter"/> and
      <xref target="sect-6"/>.</t> target="sect-6" format="counter"/>.</t>
    </section>
    <section anchor="sect-9"
             title="AR numbered="true" toc="default">
      <name>AR Procedures and EVPN All-Active Multi-homing Split-Horizon"> Multihoming Split-Horizon</name>
      <t>This section extends the procedures for the cases where two or more
      AR-LEAF nodes are attached to the same Ethernet Segment, ES and two or more
      AR-REPLICATOR nodes are attached to the same Ethernet Segment ES in the BD.
      The mixed case, that is, case -- where an AR-LEAF node and an AR-REPLICATOR node are
      attached to the same Ethernet Segment, ES -- would require extended procedures
      and it is
      that are out of scope.</t> scope for this document.</t>
      <section anchor="sect-9.1" title="Ethernet numbered="true" toc="default">
        <name>Ethernet Segments on AR-LEAF Nodes"> Nodes</name>
        <t>If a VXLAN or NVGRE are used, is used and if the Split-horizon split-horizon is based on
        the tunnel source IP Source Address address and "Local-Bias" "local bias" as described in <xref
        target="RFC8365"/>, target="RFC8365" format="default"/>, the Split-horizon split-horizon check will not work if there is
        an Ethernet-Segment ES is shared between two AR-LEAF nodes, and the
        AR-REPLICATOR replaces the tunnel source IP Source Address address of the packets
        with its own AR-IP.</t>
        <t>In order to be compatible with the source IP Source Address address split-horizon
        check, the AR-REPLICATOR MAY <bcp14>MAY</bcp14> keep the original received tunnel source IP
        Source Address
        address when replicating packets to a remote AR-LEAF or RNVE.
        This will allow AR-LEAF nodes to apply Split-horizon split-horizon check procedures
        for BM packets, packets before sending them to the local Ethernet-Segment. ES.
        Even if the AR-LEAF's source IP Source Address address is preserved when replicating
        to AR-LEAFs or RNVEs, the AR-REPLICATOR MUST <bcp14>MUST</bcp14> always use its IR-IP as
        the source IP Source Address address when replicating to other AR-REPLICATORs.</t>
        <t>When EVPN is EVPNs are used for MPLS over GRE (or UDP), MPLSoGRE or MPLSoUDP, the ESI-label based ESI-label-based
        split-horizon procedure as provided in <xref target="RFC7432"/> target="RFC7432" format="default"/> will not work
        for multi-homed Ethernet-Segments multihomed ESs defined on AR-LEAF nodes.
        "Local-Bias"
        Local bias is recommended in this case, as it is in the case of a VXLAN or
        NVGRE as explained above. The "Local-Bias" local-bias and tunnel source IP Source Address address
        preservation mechanisms provide the required split-horizon behavior in
        non-selective or selective AR.</t>
        <t>Note that if the AR-REPLICATOR implementation keeps the received
        tunnel source IP Source Address, address, the use of uRPF (unicast unicast Reverse Path
        Forwarding)
        Forwarding (uRPF) checks in the IP fabric based on the tunnel source IP Source
        Address MUST
        address <bcp14>MUST</bcp14> be disabled.</t>
      </section>

      <section anchor="sect-9.2"
               title="Ethernet numbered="true" toc="default">
        <name>Ethernet Segments on AR-REPLICATOR nodes"> Nodes</name>
        <t>AR-REPLICATOR nodes attached to the same all-active Ethernet
        Segment ES
        will follow "Local-Bias" local-bias procedures <xref target="RFC8365"/>, target="RFC8365" format="default"/>
        as follows:</t>

        <t><list style="letters">
            <t>For
        <ol spacing="normal" type="a"><li>For BUM traffic received on a local AR-REPLICATOR's Attachment
            Circuit, "Local-Bias" AC,
            local-bias procedures as provided in <xref target="RFC8365"/>
            MUST target="RFC8365" format="default"/>
            <bcp14>MUST</bcp14> be followed.</t>

            <t>For followed.</li>
          <li>For BUM traffic received on an AR-REPLICATOR overlay tunnel
            with AR-IP as the IP Destination Address, "Local-Bias" MUST DA, local bias <bcp14>MUST</bcp14> also
            be followed. That is, traffic received with AR-IP as the IP
            Destination Address DA
            will be treated as though it had been received
            on a local Attachment Circuit AC that is part of the Ethernet Segment ES
            and will be forwarded to all local Ethernet Segments, ESs, irrespective
            of their DF or NDF state.</t>

            <t>BUM state.</li>
          <li>BUM traffic received on an AR-REPLICATOR overlay tunnel with
            IR-IP as the IP Destination Address, DA will follow regular <xref
            target="RFC8365"/> "Local-Bias" local-bias rules <xref target="RFC8365" format="default"/> and will not be forwarded to
            local Ethernet Segments ESs that are shared with the AR-LEAF or
            AR-REPLICATOR originating the traffic.</t>

            <t>In traffic.</li>
          <li>In cases where the AR-REPLICATOR supports a single IP address,
            the IR-IP and the AR-IP are the same IP address, as discussed in
            <xref target="sect-8"/>. target="sect-8" format="default"/>. The received BUM traffic will be treated
            as specified in 'b' item b above if the received VNI is the AR-VNI, AR-VNI and as specified in 'c' item c if the VNI is the IR-VNI.</t>
          </list></t> IR-VNI.</li>
        </ol>
      </section>
    </section>
    <section anchor="sect-10" title="Security Considerations"> numbered="true" toc="default">
      <name>Security Considerations</name>
      <t>The Security Considerations security considerations in <xref target="RFC7432"/> target="RFC7432" format="default"/> and <xref
      target="RFC8365"/> target="RFC8365" format="default"/> apply to this document. The Security Considerations security considerations
      related to the Leaf Auto-Discovery A-D route in <xref
      target="I-D.ietf-bess-evpn-bum-procedure-updates"/> target="RFC9572" format="default"/> apply too.</t>
      <t>In addition, the Assisted-Replication Assisted Replication method introduced by this
      document may bring introduce some new risks for that could affect the successful delivery of BM
      traffic. Unicast traffic is not affected by Assisted-Replication Assisted Replication
      (although Unknown unknown unicast traffic is affected by the Pruned-Flood-Lists
      procedures). procedures for PFLs). The forwarding of Broadcast and Multicast (BM) BM traffic is
      modified, and BM traffic from the AR-LEAF nodes will be attracted by the
      existence of drawn toward
AR-REPLICATORs in the BD. An AR-LEAF will forward BM
      traffic to its selected AR-REPLICATOR, therefore AR-REPLICATOR; therefore, an attack on the
      AR-REPLICATOR could impact the delivery of the BM traffic using that
      node. Also, an attack on the AR-REPLICATOR and any change of to the advertised
      AR type will modify the selection on selections made by the AR-LEAF nodes. If no other
      AR-REPLICATOR is selected, the AR-LEAF nodes will be forced to use
      Ingress Replication forwarding mode, which will impact on their
      performance, since the AR-LEAF nodes are usually NVEs/PEs with poor
      replication performance.</t>
      <t>This document introduces the ability for of the AR-REPLICATOR to forward
      traffic received on an overlay tunnel to another overlay tunnel. The
      reader may interpret determine that this introduces the risk of BM loops. That loops -- that is,
      an AR-LEAF receiving a BM encapsulated BM-encapsulated packet that the AR-LEAF
      originated in the first place, place due to one or two AR-REPLICATORs
      "looping" the BM traffic back to the AR-LEAF. The Following the procedures provided in this
      document will prevent these BM loops, since the AR-REPLICATOR will always
      forward the BM traffic using the correct tunnel IP Destination Address DA
      (or the correct VNI in the case of single-IP AR-REPLICATORs) that AR-REPLICATORs), which instructs the
      remote nodes regarding how to forward the traffic. This is true in for both the
      Non-Selective
      Non-selective and Selective modes defined in this document. However, a
      wrong
      incorrect implementation of the procedures provided in this document may lead to
      those unexpected BM loops.</t>
      <t>The Selective mode provides a multi-staged multi-stage replication solution,
      where a proper configuration of all the AR-REPLICATORs will avoid prevent any
      issues. A mix of mistakenly configured Selective selective and Non-Selective non-selective
      AR-REPLICATORs in the same BD could theoretically create packet
      duplication in some AR-LEAFs, however AR-LEAFs; however, this document specifies a fall
      back fallback solution -- falling back to Non-Selective Non-selective mode in case cases where the AR-REPLICATORs
      advertised an inconsistent AR Replication mode.</t>
      <t>This document allows the AR-REPLICATOR to preserve the tunnel source IP
      Source Address
      address of the AR-LEAF (as an option) when forwarding BM packets
      from an overlay tunnel to another overlay tunnel. Preserving the AR-LEAF
      source IP Source Address address makes the "Local Bias" local-bias filtering procedures possible
      for AR-LEAF nodes that are attached to the same Ethernet Segment. ES. If the
      AR-REPLICATOR does not preserve the AR-LEAF source IP Source Address, address, AR-LEAF
      nodes attached to all-active Ethernet Segments ESs will cause packet
      duplication on the multi-homed multihomed CE.</t>
      <t>The AR-REPLICATOR nodes are, by design, using more bandwidth than
      <xref target="RFC7432"/> PEs or <xref target="RFC8365"/> target="RFC7432" format="default"/> or NVEs <xref target="RFC8365" format="default"/> would use.
      Certain network events or unexpected low performance may exceed the
      AR-REPLICATOR
      AR-REPLICATOR's local bandwidth and cause service disruption.</t>
      <t>Finally, the use of PFL as in <xref target="sect-7"/>, PFLs (<xref target="sect-7" format="default"/>) should be
      handled
      used with care. An intentional Intentional or unintentional misconfiguration of
      the BDs on a given leaf node may result in the leaf not receiving the
      required BM or Unknown unknown unicast traffic.</t>
    </section>
    <section anchor="sect-11" title="IANA Considerations"> numbered="true" toc="default">
      <name>IANA Considerations</name>
      <t>IANA has allocated the following Border Gateway Protocol (BGP)
      Parameters:</t>

      <t><list hangIndent="3" style="symbols">
          <t hangText="">Allocation
      parameters:</t>
      <ul spacing="normal">
        <li>Allocation in the P-Multicast "P-Multicast Service Interface Tunnel (PMSI Tunnel) Tunnel Types registry:</t>
        </list></t>

      <figure>
        <artwork><![CDATA[
   Value     Meaning                        Reference
   0x0A      Assisted-Replication Tunnel    [This document]
]]></artwork>
      </figure>

      <t><list hangIndent="3" style="symbols">
          <t hangText="">Allocations Types" registry:</li>
      </ul>
      <table align="center">
	<thead>
	  <tr>
<th>Value</th>
<th>Meaning</th>
<th>Reference</th>
</tr>
</thead>
<tbody>
<tr>
  <td>0x0A</td>
  <td>Assisted Replication Tunnel</td>
   <td>RFC 9574</td>
</tr>
</tbody>
</table>
      <ul spacing="normal">
        <li>Allocations in the P-Multicast "P-Multicast Service Interface (PMSI) Tunnel Attribute Flags registry:</t>
        </list></t>

      <figure>
        <artwork><![CDATA[
   Value    Name                           Reference
   3-4      Assisted-Replication Flags" registry:</li>
      </ul>

      <table align="center">
<thead>
<tr>
 <th>Value</th>
<th>Name</th>
<th>Reference</th>
</tr>
</thead>
<tbody>
<tr>
  <td>3-4</td>
<td>Assisted Replication Type (T)  [This document]
   5        Broadcast (T)</td>
<td>RFC 9574</td>
</tr>
<tr>
<td>5</td>
<td>Broadcast and Multicast (BM)   [This document]
   6        Unknown (U)                    [This document]
]]></artwork>
      </figure> (BM)</td>
<td>RFC 9574</td>
</tr>
<tr>
<td>6</td>
<td>Unknown (U)</td>
<td>RFC 9574</td>
</tr>
</tbody>
</table>
    </section>

    <section title="Contributors">
      <t>In addition to the names in the front page,
  </middle>
  <back>

    <references>
      <name>References</name>
      <references>
        <name>Normative References</name>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.6514.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7432.xml"/>

<!-- draft-ietf-bess-evpn-bum-procedure-updates (RFC 9572)
   Fix title if 9572 authors approve the following co-authors
      also contributed change -->
<reference anchor="RFC9572" target="https://www.rfc-editor.org/info/rfc9572">
<front>
<title>Updates to this document:</t>

      <t><figure>
          <artwork><![CDATA[
Wim Henderickx
Nokia

Kiran Nagaraj
Nokia

Ravi Shekhar
Juniper Networks

Nischal Sheth
Juniper Networks

Aldrin Isaac
Juniper

Mudassir Tufail
Citibank
]]></artwork>
        </figure></t>
    </section> EVPN Broadcast, Unknown Unicast, or Multicast (BUM) Procedures</title>
<author initials='Z' surname='Zhang' fullname='Z. Zhang'>
  <organization/>
</author>
<author initials='W' surname='Lin' fullname='W. Lin'>
  <organization/>
</author>
<author initials='J' surname='Rabadan' fullname='J. Rabadan'>
  <organization/>
</author>
<author initials='K' surname='Patel' fullname='K. Patel'>
  <organization/>
</author>
<author initials='A' surname='Sajassi' fullname='A. Sajassi'>
  <organization/>
</author>
<date month='April' year='2024'/>
</front>
<seriesInfo name="RFC" value="9572"/>
<seriesInfo name="DOI" value="10.17487/RFC9572"/>
</reference>

	<xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7902.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.6513.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8365.xml"/>
      </references>
      <references>
        <name>Informative References</name>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7348.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.4023.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7637.xml"/>

<!-- draft-ietf-bess-evpn-proxy-arp-nd (RFC 9161; published) -->
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.9161.xml"/>

      </references>
    </references>

 <section title="Acknowledgments"> numbered="false" toc="default">
      <name>Acknowledgements</name>
      <t>The authors would like to thank Neil Hart, David Motz, Dai Truong,
      Thomas Morin, Jeffrey Zhang, Shankar Murthy and Krzysztof Szarkowicz <contact fullname="Neil Hart"/>, <contact fullname="David Motz"/>, <contact fullname="Dai Truong"/>,
      <contact fullname="Thomas Morin"/>, <contact fullname="Jeffrey Zhang"/>, <contact
fullname="Shankar Murthy"/>, and <contact fullname="Krzysztof Szarkowicz"/> for
      their valuable feedback and contributions. Also Also, thanks to John Scudder <contact fullname="John Scudder"/> for his thorough review that review, which improved the quality of the document
      significantly. </t>
    </section>
  </middle>

  <back>
    <references title="Normative References">
      &RFC2119;

      &RFC8174;

      &RFC6514;

      &RFC7432;

      &I-D.ietf-bess-evpn-bum-procedure-updates;

      &RFC7902;

      &RFC6513;

      &RFC8365;
    </references>

    <references title="Informative References">
      &RFC7348;

      &RFC4023;

      &RFC7637;

      &I-D.ietf-bess-evpn-proxy-arp-nd;
    </references>

    <section numbered="false" toc="default">
      <name>Contributors</name>
      <t>In addition to the authors listed on the front page, the following people also contributed to this document and should be considered coauthors:</t>

<contact fullname="Wim Henderickx">
<organization>Nokia</organization>
</contact>

<contact fullname="Kiran Nagaraj">
<organization>Nokia</organization>
</contact>

<contact fullname="Ravi Shekhar">
<organization>Juniper Networks</organization>
</contact>

<contact fullname="Nischal Sheth">
<organization>Juniper Networks</organization>
</contact>

<contact fullname="Aldrin Isaac">
<organization>Juniper</organization>
</contact>

<contact fullname="Mudassir Tufail">
<organization>Citibank</organization>
</contact>

    </section>
  </back>
</rfc>