Overview

Normally on a peering exchange, all connected parties will establish bilateral peering relationships with each other member port connected to the exchange. As the number of connected parties increases, it becomes increasingly more difficult to manage peering relationships with members of the exchange. A typical peering exchange full-mesh eBGP configuration might look something similar to the diagram on the left hand side.

 

[ IXP full mesh peering relationships ] [ IXP route server peering relationships ]
IXP full mesh peering relationships IXP route server peering relationships

 

The full-mesh BGP session relationship scenario requires that each BGP speaker configure and manage BGP sessions to every other BGP speaker on the exchange. In this example, a full-mesh setup requires 7 BGP sessions per member router, and this increases every time a new member connects to the exchange.

However, by using a route server for all peering relationships, the number of BGP sessions per router stays at two: one for each route server. Clearly this is a more sustainable way of maintaining IXP peering relationships with a large number of participants.


Should I use this service?

The BCIX route server cluster is aimed at:

  • small to medium sized members of the exchange who don’t have the time or resources to aggressively manage their peering relationships
  • larger members of the exchange who have an open peering policy, but where it may not be worth their while managing peering relationships with smaller members of the exchange.

As a rule of thumb: If you do not have any good reason not to use the route server cluster, you should probably use it.

The service is designed to be reliable. It operates on two physical servers, each located in a different data centre. The service is available for both IPv4 and IPv6. Each server runs a separate routing daemon per VLAN and per L3 protocol. Should a single BGP server die for some unlikely reason, no other BGP server is likely to be affected. If one of the physical servers becomes unavailable, the second server will continue to provide BGP connectivity.

BCIX has also implemented inbound prefix filtering on its route-server cluster. This uses both RPKI and internet routing registry data from the various IRR databases to allow connected members to announce only the address prefixes which they have registered publicly. If your prefix has a valid RPKI ROA, it will pass. If the RPKI ROA check result is unknown (you have not set up a ROA), we fall back to the IRRDB test.

BCIX uses BIRD2 (Bird Internet Routing Demon) running on Centos 7 for our route server cluster. BIRD is widely used at internet exchanges for route server clusters, and has been found to be reliable in production.


How do I use the service?

If enabled, the route servers are set up to accept BGP connections from your router. Once this has been done, you will need to configure a BGP peering session to the correct internet address. The IP addresses of the route servers are listed as follows:


Route Server #1 193.178.185.5 2001:7f8:19:1::3ff6:5
Route Server #2 193.178.185.6 2001:7f8:19:1::3ff6:6

Maximum Prefix Recommendations

We recommend setting the max-prefix limit of the BGP sessions to our route servers to


160000 80000

Example Configs

Cisco IOS

router bgp 99999
 no bgp enforce-first-as
 !
 ! Route Server #1
 !
 neighbor 193.178.185.5 remote-as 16374
 neighbor 193.178.185.5 description BCIX Route Server 1
 neighbor 193.178.185.5 password PASSWORD
 !
 address-family ipv4
  neighbor 193.178.185.5 activate
  neighbor 193.178.185.5 route-map BCIX-IN-v4 in
  neighbor 193.178.185.5 route-map BCIX-OUT-v4 out
  neighbor 193.178.185.5 maximum-prefix 160000
 exit-address-family
 !
 neighbor 2001:7f8:19:1::3ff6:5 remote-as 16374
 neighbor 2001:7f8:19:1::3ff6:5 description BCIX Route Server 1
 neighbor 2001:7f8:19:1::3ff6:5 password PASSWORD
 !
 address-family ipv4
  no neighbor 2001:7f8:19:1::3ff6:5 activate
 exit-address-family
 !
 address-family ipv6
  neighbor 2001:7f8:19:1::3ff6:5 activate
  neighbor 2001:7f8:19:1::3ff6:5 route-map BCIX-IN-v6 in
  neighbor 2001:7f8:19:1::3ff6:5 route-map BCIX-OUT-v6 out
  neighbor 2001:7f8:19:1::3ff6:5 maximum-prefix 80000
 exit-address-family
 !
 ! Route Server #2
 !
 neighbor 193.178.185.6 remote-as 16374
 neighbor 193.178.185.6 description BCIX Route Server 2
 neighbor 193.178.185.6 password PASSWORD
 !
 address-family ipv4
  neighbor 193.178.185.6 activate
  neighbor 193.178.185.6 route-map BCIX-IN-v4 in
  neighbor 193.178.185.6 route-map BCIX-OUT-v4 out
  neighbor 193.178.185.6 maximum-prefix 160000
 exit-address-family
 !
 neighbor 2001:7f8:19:1::3ff6:6 remote-as 16374
 neighbor 2001:7f8:19:1::3ff6:6 description BCIX Route Server 2
 neighbor 2001:7f8:19:1::3ff6:6 password PASSWORD
 !
 address-family ipv4
  no neighbor 2001:7f8:19:1::3ff6:6 activate
 exit-address-family
 !
 address-family ipv6
  neighbor 2001:7f8:19:1::3ff6:6 activate
  neighbor 2001:7f8:19:1::3ff6:6 route-map BCIX-IN-v6 in
  neighbor 2001:7f8:19:1::3ff6:6 route-map BCIX-OUT-v6 out
  neighbor 2001:7f8:19:1::3ff6:6 maximum-prefix 80000
 exit-address-family
 !
!

Cisco IOS XR

router bgp 99999
 bgp enforce-first-as disable
 !
 ! Route Server #1
 !
 neighbor 193.178.185.5
  remote-as 16374
  password encrypted PASSWORD
  description BCIX Route Server 1
  address-family ipv4 unicast
   maximum-prefix 160000 75
   route-policy BCIX-IN-v4 in
   route-policy BCIX-OUT-v4 out
  !
 !
 neighbor 2001:7f8:19:1::3ff6:5
  remote-as 16374
  password encrypted PASSWORD
  description BCIX Route Server 1
  address-family ipv6 unicast
   maximum-prefix 80000 75
   route-policy BCIX-IN-v6 in
   route-policy BCIX-OUT-v6 out
  !
 !
 ! Route Server #2
 !
 neighbor 193.178.185.6
  remote-as 16374
  password encrypted PASSWORD
  description BCIX Route Server 2
  address-family ipv4 unicast
   maximum-prefix 160000 75
   route-policy BCIX-IN-v4 in
   route-policy BCIX-OUT-v4 out
  !
 !
 neighbor 2001:7f8:19:1::3ff6:6
  remote-as 16374
  password encrypted PASSWORD
  description BCIX Route Server 2
  address-family ipv6 unicast
   maximum-prefix 80000 75
   route-policy BCIX-IN-v6 in
   route-policy BCIX-OUT-v6 out
  !
 !

Juniper JunOS

protocols {
    bgp {
        group AS16374-BCIX-RS1 {
            type external;
            description "BCIX Route Server 1";
            remove-private;
            peer-as 16374;
            neighbor 193.178.185.5 {
                import BCIX-IN-v4;
                family inet {
                    unicast {
                        prefix-limit {
                            maximum 160000;
                            teardown 75 idle-timeout 60;
                        }
                    }
                }
                authentication-key PASSWORD;
                export BCIX-OUT-v4;
            }
            neighbor 2001:7f8:19:1::3ff6:5 {
                import BCIX-IN-v6;
                family inet6 {
                    unicast {
                        prefix-limit {
                            maximum 80000;
                            teardown 75 idle-timeout 60;
                        }
                    }
                }
                authentication-key PASSWORD;
                export BCIX-OUT-v6;
            }
        }
        group AS16374-BCIX-RS2 {
            type external;
            description "BCIX Route Server 2";
            remove-private;
            peer-as 16374;
            neighbor 193.178.185.6 {
                import BCIX-IN-v4;
                family inet {
                    unicast {
                        prefix-limit {
                            maximum 160000;
                            teardown 75 idle-timeout 60;
                        }
                    }
                }
                authentication-key PASSWORD;
                export BCIX-OUT-v4;
            }
            neighbor 2001:7f8:19:1::3ff6:6 {
                import BCIX-IN-v6;
                family inet6 {
                    unicast {
                        prefix-limit {
                            maximum 80000;
                            teardown 75 idle-timeout 60;
                        }
                    }
                }
                authentication-key PASSWORD;
                export BCIX-OUT-v6;
            }
        }
    }
}

You should by all means use filtering to control incoming and outgoing prefix announcements to allow only prefixes which you intend to announce and receive.

Note that the route server system depends on information in either RPKI or the various IRR databases (typically the RIPE IRR database for BCIX members). If you have not published either a valid ROAs or correct route: and route6: objects in this database, your prefix announcements will be ignored by the route server and your peers will not route their traffic to you via the exchange. This is checked by BCIX Operations during provisioning and guidance and assistance is provided as required.


AS-SET

If you do IRRDB based filtering, have look at our AS-SET object AS-BCIX in the RIPE DB.


Community based prefix filtering

The BCIX route server system also provides well known communities to allow members to control the distribution of their prefixes. These communities are defined as follows:


Prevent announcement of a prefix to a peer 0:peer-as
Announce a route to a certain peer 16374:peer-as
Prevent announcement of a prefix to all peers 0:16374
Announce a route to all peers 16374:16374

So, for example, to instruct the route server to distribute a particular prefix only to AS64111 and AS64222, the prefix should be tagged with communities: 0:16374, 16374:64111 and 16374:64222.

Alternatively, to announce a prefix to all BCIX members, excluding AS64333, the prefix should be tagged with community 0:64333.

The BCIX route server cluster supports BGP large community prefix distribution control, using the following policy:


Prevent announcement of a prefix to a peer 16374:0:peer-as
Announce a route to a certain peer 16374:1:peer-as
Prevent announcement of a prefix to all peers 16374:0:0
Announce a route to all peers 16374:1:0

The BCIX route server cluster also supports BGP large community AS path prepending control, using the following policy:


Prepend to peer AS once 16374:101:peer-as
Prepend to peer AS twice 16374:102:peer-as
Prepend to peer AS three times 16374:103:peer-as

Looking Glass

We operate an Alice Looking Glass for you to check our view of your announcements.

If a route servers is filtering a prefix, it will show in the looking glass along with a reason why the prefix is filtered.


Filtering Policy

Our Route Server filtering policy is defined in the source code for IXP Manager on github. This is a summary of what it does.

  1. Drop small prefixes – longer than /24 for IPv4 and longer than /48 for IPv6.
  2. Drop all well-known martians and bogons.
  3. Ensure that there is at least 1 ASN and less than 64 ASNs in the AS path.
  4. Ensure that the peer AS is the same as the first AS in the AS path.
  5. Drop any prefix where the next-hop IP address is not the same as the peer IP address. This prevents prefix hijacking.
  6. Drop any prefix with a transit network ASN in the AS path.
  7. Ensure that origin AS is in set of ASNs from the client’s IRRDB AS-SET.
  8. If the prefix is evaluated as RPKI valid, accept.
  9. If the prefix is evaluated as RPKI invalid, drop.
  10. If the prefix is evaluated as RPKI unknown, revert to standard IRRDB prefix filtering.