Managing CE eBGP details & common/accepted CE-facing BGP practices

[streiner at cluebyfour.org (Justin M. Streiner)]

On Sat, 20 Dec 2008, Justin Shore wrote:

> Does anyone have any preferred ways to manage their customer-facing BGP 
> details?  I'm thinking about the customer's ASN (SP assigned private ASN or 
> RIR assigned ASN), permitted prefixes, etc?  While I'm sure this could be 
> easily stored in a spreadsheet I'm not sure if there is any merit to storing 
> some of these details outside of the configuration on the PE (assuming of 
> course that the PE's config is regularly archived).  Now if the PE's BGP 
> config was auto-generated via a script then it would make sense for all the 
> details to be stored off in a DB in the NOC.  Beyond that is there a good 
> reason to do archive it in a textual format off of the PE and if there is a 
> sound reason to do it, is therea good or preferred way to do accomplish this?

You could certainly store all of the relevant config details in a database 
of some sort, and it certainly can't hurt to do so.  Same goes for backing 
up your device configurations - always a good idea.  As far as storing 
things like ASNs, allowed prefixes, etc, you may want to look at storing 
that information in an RPSL format.  Many providers require their 
customers to register route/AS/policy objects either in one of the 
Internet Routing Registries (RADB, AltDB, etc...), or in a similar system 
that's operated by the provider.  They then use this information for 
pushing out configuraiton changes such as the list of prefixes that a 
customer is allowed to announce, etc.

> We're moving beyond our typical residential and very small SMB service to 
> larger customers over the next few months.  These areas have larger, more 
> advanced customers and I'm sure we'll run into multi-homed environments and 
> customer who will expect BGP peering options.  I would like to be prepared 
> with sound practices before we get our first customer that wants to get a 
> default route via BGP, wants full tables, or has their own ASN and is 
> bringing their own PI space with them.  Some of this of course implies 
> multiple processes to confirm that the ASN belongs to the customer in 
> question, that the PI space belongs to the customer in question, notifying 
> our upstreams to accept the customer's PI space, etc.  It's hammering out the 
> scalable and best practice config details that I'm concerned with at the 
> moment.

It's good that you're thinking about this now, particularly the "is this 
customer legitimately allowed to announce prefix ABC, or source their 
traffic from AS XYZ"?  Most larger providers put at least some limitations 
on what they will allow a customer to announce, though the level of 
investigation done to attempt to establich legitimacy for those 
announcements varies.  Some providers require customers to provide some 
sort of Letter of Authorization/Agency for the prefixes they want to 
announce.

> When assigning private ASNs to customers, are there any gotchas to be aware 
> of?  Is it possible to use the same private ASN for more than one customer on 
> the same PE?

Yes.  As long as the organizations that are using the private AS aren't 
1. trying to advertise the same space, 2. possibly connect directly to 
each other directly, or 3. expecting to be able to connect to multiple 
upstream providers, then you should be OK.  VZB (former UUNET) did 
something similar, using AS7046 (not a private AS, but the principle is 
the same), and I believe other carriers have had similar arrangements 
for customers.

> What are common and accepted CE-facing BGP practices?  MD5 AUTH, GTSM, max 
> prefix limits?  Which is preferred, route-maps or prefix-lists for 
> controlling advertised and/or received routes?  Do any SPs utilize AS-Path 
> ACLs to check that prefixes from an customer's ASN are claimed to originate 
> from there?  Are there any SPs out there offering BFD support for BGP or 
> CE-facing peering sessions?

MD5 is good, but most providers I've seen make this an opt-in feature - 
they don't force their customers to use it.  Setting a reasonable 
max-prefix limit and adjusting it as the number of prefixes a customer 
announces is always a good idea, and I'd consider it to be a best 
practice.  Prefix lists and route maps can do different things, or 
accomplish the same task in different ways.  It also depends on what 
functionality you want to offer your customers.  Do you plan to publish 
and support a consistent set of customer-settable BGP communities for 
doing things like selective AS prepends?  Do you plan to tag incoming 
advertisements with communities to identify them as customer routes, and 
pass those communities to your customers and peers?  Some providers use AS 
path ACLs, and others avoid them at all costs.

> Should we have the customer announce their PA space to us or do we advertise 
> it for them (redist a static)?  Do SPs restrict access to tcp/179 on the CE 
> from the Internet in the CE-facing ACL?  Do SPs block access to the PE-CE 
> subnet from the outside world like what was described in the Router Security 
> Strategies book (pages 189-193)?  What about dropping incoming traffic to 
> everything but the CE IP?

If the customers need to connect to multiple upstream providers, it's much 
less hassle to have them originate their prefixes from their own AS, and 
then you just need to propagate them.  If they are single-homed, you can 
announce the prefix(es) for them and then just statically route them to 
the customer.  Other things to consider are sound ingress/egress filtering 
policies, loose/strict RPF etc.  Implementing a Netflow based monitoring 
solution, and applying flow caching to PE-CE interfaces is also agood 
idea.

> While I don't predict our CE-facing BGP load to be terribly significant at 
> this point, I would like to establish sound practices now rather than down 
> the road once we're neck deep in temporarily production workarounds.

Again, it's great that you're being proactive about this.  Setting up good 
policies now will make things much easier to maintain and provision in the 
future as your network grows.  Don't forget to document those policies 
thoroughly, particularly if other people will be tasked with implementing 
it on a dat yo day basis, i.e. handling new customer turn-ups, making BGP 
prefix list modifications, etc.

> Is there any consensus on what's best practice for CE-facing BGP?  I imagine 
> most SP engineer's BGP practices could be better equated to a religious holy 
> war on par with Chevy vs Ford or Mac vs PC.  I would be interested in hearing 
> what they are though and learning from the group's expertise.

It seems like you have a pretty good handle on what you need to do.  I 
don't know that there will be much of the holy war angle to the responses. 
Different people adopt different customer routing policies for different 
reasons - some technical, some business, some political.  While there are 
different ways of accomplishing this task, most of the more scalable ways 
have already been a part of most large providers' policies for awhile.
One-offs are bad, in my opinion, so the more you do to avoid them now, the 
fewer headaches you will have down the road.

jms