In times where downtime is no option, offering applications at multiple physical locations through the same IP is an absolute condition to keep your users happy. After all, if things just are at only one physical datacenter location they are prone to hardware, power, network or any other physical imperfections. Beside of this, today our IP data should be close to our users and network system administrators should be able tweak their network to their users and should also be able to perform maintenance without downtime. Therefore the use of multiple datacenters is key for every serious IP application.
Some call it BGP network anycast, others dual datacenter IP setup and others twin datacenter colocation housing. All different naming with the same goal: using the same IP subnets on the same moment at multiple colocation facilities. By financial reasons most people use the same colocation datacenter company for their dual datacenter network setup. Others are using many datacenters round the globe. In case of a failing physical location, users will be automatically forwarded by BGP routing in miliseconds to the next nearest datacenter. No matter from what part of the world your users request want to access your IP's.
The definition of anycast routing
First of all the word "anycast" routing needs be explained because it's naming is used in many cases where it should not be used. At Serverius we say that anycast is a collection of IP services which are present using the same public IP address on the internet to offer their content to by different BGP networks to the nearest user. Data traffic from a user (source computer user) will be routed to the nearest BGP IP network. This will boost latency and improves load time, cut down bandwidth costs, improves availability and boosts the scalability. It's important to know that "topographically closer" does not inherently mean geographically closer, although this is mostly the case.
Anycast is linked with the BGP protocol which ensures that all of a router’s neighbors are aware of the networks that can be reached through that router and the topographical distance to those networks. The IP address range is advertised by BGP at multiple routers which are using multiple carriers which are at multiple phisical datacenter locations. As this propagates across the Internet, routers become aware of which of their neighbors provides the shortest topographical path to the advertised IP address.With Anycast routing, nobody knows upfront which server will receive an incoming request. In this case, all hardware which share the same IP address should have the same or similar content and should be ready to reply to all requests. So, with anycast routing client should take a care about synchronization of data between hardware. The cool thing of Serverius is the we provide our clients a transparent L2 channel between our datacenter locations. It makes data synchronization easy and fast.
Like many other things in life, "less risk will you cost more money". When people want to start using a multi datacenter IP setup they would like to go for the best possible setup. Of course that's logical, but in reality I see that allot of them cannot afford the best possible setup because the total cost of routers, IP subnets, uplinks, colocation racks, fibers between datacenters racks, time and knowledge to manage, are simply to much.
To understand the basics is sometimes a bit hard, especially in addition to their existing network infrastructure. I's complicated but with nice new technical options that helps our financial smaller budget users to get an acceptable anycast alternative. This way these dual datacenter IP setups are not only toys from large enterprises anymore. :)
Secondly there are also less important arguments what can be important:
The outcome will show you if it's a "nice to have" or if it's essential to have. If the answer is (almost) "yes" the Serverius NOC department can assists to setup a high availability network setup at both our SDC1 and SDC2 datacenters and even others. The principle "start small, expand later when you need it" is used to give a honest and affordable advice.
Let’s discuss some possible examples to create a default setup in a way how Serverius is offering it to its users by default. The 4 default configuration examples below are not all possible setups because there are much more ways to get an even higher availability and lower latency. Although the principles will still be the same: creating an optimal solution on anycast or default dual datacenter network technology.
This is the best situation for anycast routing when a client has its own BGP routers with its own ASn and at least one /24 IP range to announce to the internet (client can rent IP range from Serverius or elsewhere). In both Serverius facilities you can establish multiple BGP connection to Serverius and other IP-transit carriers. In this case, IP traffic for this IPs will be delivered to the closest datacenter near the user (the shortest traceroute possible). In case of and network outage at one location, the other locations will automatically take over all IP traffic.
You can manage IP data traffic to your requirement 100%, so specific BGP adjustments, special BGP community preferences, disable routes for maintenance, disabling carriers by partial carriers outages, etc. You are in full control to act where needed.
In most of these cases people will use only 2 or more physical datacenter facilities where they will even make the network redundant with multiple carries per location. For example, they use Serverius and the NL-IX at SDC1 and Serverius and Retn at SDC2 and in case of a 3rd facillity on or two other carriers. When using multiple carriers, a full routing table router will be needed. Otherwise a more simple layer 3 switch what can do a partial routing table will do.
Service and hardware cost:
If you do not (yet) have your own BGP routers with a private ASn, Serverius and another carrier can announce your /24 or larger IP subnets under the carrier its ASn for you. This way you are using all advantages of anycast with a limited amount of own control. IP traffic for client’s will be delivered to the closest carrier/datacenter and in case of an carrier outage at one location the data traffic will be automatically re-routed to another carrier and another datacenter facility. You can still use the Serverius facilities and others, but please be ware the not all carriers will announcing IP subnets from their ASn routers (some will force you to use your own routers and ASn).
You can't fully manage your IP data-traffic yourself. Adjustments and routing tweaks are simply not possible because you do not have access to the routers yourself. The only thing you can do during network problems is disable a carrier in total.
In most of cases people will use this setup to use anycast as cheap and simple as possible. Route optimizing is done by the carrier quality, during partial outages they disable the carrier and for support they will fully rely on fast carrier response.
If you don’t have (or don't want have) your own BGP router with own ASn to setup your BGP infrastructure (like example 1 and 2) you can also use a /24 or larger subnet to setup a fully redundant dual datacenter solution by only the Serverius BGP network. In case of an outage at one physical datacenter location the BGP will stop advertising the /24 and the other locations will automatically take over. Like this you will have almost an anycast solution, but because the BGP is managed by the Serverius routers and you do not use any other carrier.
In normal situation, your IP traffic will be delivered to the closest Serverius datacenter. For example, if someone in the south of the Netherlands, from London/America use to the IP, it will probably end up in SDC1. If it is someone from central/north Netherlands, Germany, Northern Europe or Russia, it will most likely come out in SDC2.
The nice thing here is that with this option you can use an IP address on only one location or use the same IP on the same time at SDC1 and SDC2. But the downside of this setup is that you will have a little bit less personal control. For example, when you like to make personal routing changes it's not possible or if you want to disable one /24 on one location during your maintenance, you need to contact the Serverius NOC department or disable your uplink. Also, because of the use of only 1 network carrier a general network error of Serverius will affect both datacenter locations.
This example is a good way to start your own network to upgrade to real anycast later on.
If you are using small subnets which are smaller than a /24 and you want use a dual datacenter solution, the default Serverius IP infrastructure will can offer you it. In case of an outage at one physical datacenter location the BGP will stop advertising the /24 where your smaller subnets are part of and the other locations will automatically take over. But in case of a partial Serverius network outage on one location, data traffic at the affected facility could not be automatically forwarded to your rack(s) at the other location. It really depends on the type of outage, in sometimes it does flap over, sometimes not. Therefore this is not an real redundant or anycast solution and in this case you will take the risk of an partial outage during Serverius network outages at one location.
In normal situation, your IP traffic will be delivered to the closest Serverius datacenter. For example, if someone in the south of the Netherlands, from London/America use to the IP, it will probably end up in SDC1. If it is someone from central/northern Netherlands, Germany, Northern Europe or Russia, it will most likely come out in SDC2. You need to create one VLAN between all your racks at both datacenter facilities because when your uplink is up, data traffic for your IP's are forward to you, even when the IP is on a hardware device at the other datacenter facility. Therefore you have to forward your traffic to the other location by the VLAN. This "transport VLAN" is offered by a extra physical fiber/copper cable (from another hardware infrastructure) and is offered by Serverius in most cases for free.
Also with this option you can use an IP address on only one location or use the same IP on the same time at SDC1 and SDC2. But the downside of this setup is that you will have a no personal control. For example, when you like to disable your /27 on one location during your maintenance, you need to contact the Serverius NOC department or disable your uplink. Also, because of the use of only 1 network carrier a general network error of Serverius will affect both datacenter locations.
This example is a good way to expand your colocation racks and use it as a whole. Upgrading to real anycast later on is not possible.
Service and hardware cost:
The Serverius NOC department assists its users to setup a high availability network setup at both our own datacenters and also external datacenters. We can advice, buy, finance, build, monitor and maintain it by a Noc as a Service (NOCaaS). In the past years we did this for more than a few clients, where the combination with the client and the NOC is perfectly arranged. Sometimes people like us to perform the whole network and hardware setup, others only want to use the NOC to fall back on if needed.
FYI: The options above are default configuration examples. Keep in mind these are not all possible setups because there are even more advanced versions for even higher availability and lower latency. Although the principles will still be the same, creating an optimal solution on anycast or default dual datacenter network technology.