26 May

Why having cloud load-balancer for on-premises services is not a good idea?

We were thinking about redundancy options for CCIE.PL today. There are few restrictions we have there, both came either from policy or our personal thoughts about several aspects of paid services and sharing admin access. But simply we are thinking how to automate failover in case our primary server or database have problems. Easiest solution would be to use Cloudflare free tier service but let’s say we don’t want to do this now. So we were looking on the other options and there was an idea that maybe we can use cloud load-balancer for on-premise services. First thought – it’s brilliant. On second thought – definitely that idea was wrong. Let me show you why.

Why there is no such service as cloud load-balancer for on-premises services?

Our idea was simple – load balancer service in any public cloud that will probe our internet-facing servers in on-premises data centers and will provide failover functionality in case our primary servers is down. Sounds easy. But it’s not possible to do that. Here are the reasons:

  • Reason 0: This is the only reason that have it’s weight here, others are in pretty much random order – Neither Amazon Web Service, Microsoft Azure or e24cloud provide service where pool can contain external servers. So you can create load balancer service but it can only distribute between servers your service inside public cloud. Why? For same reasons why our idea was not that brilliant
  • Reason 1: Health probes might not give back correct result. Let’s say our servers are in Poland but closest load balancer can be set up in UK. So we face here typical “traditional network” problems:
    • RTT (round-trip time) may be quite long so probes may return false positive alerts
    • Jitter (variance of RTT) may cause problems for health checks because path may change quite often in Internet
    • Servers in Poland may be temporarily unaccessible from UK even if they are still accessible from Poland. Well, that’s how internet works after all
  • Reason 2: We increase latency for users and slow down our users – most of people visiting ccie.pl are right now from Poland (try to check it, there is English section as well!). So if we set up load balancer in UK their request will goes from Poland to UK then back to Poland. That means higher latency and bad user experience due to longer than expected time required for content to load
  • Reason 3: Routing asymmetry – AWS and Azure load balancers don’t apply SNAT for incoming traffic (I’m not sure about e24cloud buy I assume it’s the same). That means we will send questy to load balancer IP while response will be received from public IP address of the real server Asymmetry in routing is generally thing that cause problems. On network, system and application level.
  • Reason 4: Cost – common practice in public clouds is that you pay for service and traffic outgoing from the cloud. With asymmetry they can earn only for the service. It there would be SNAT they can charge us for traffic but well…. imagine DDoS attack (I know they have protection in place) or large download of data from our server. Yeah that may cost us a lot of money.

There are lot more, but in my opinion those are the major ones that makes this service not a good idea.

What we can do if cloud load-balancer for on-premises services is a bad idea? There are other options like Cloudflare or our on data center load balancer, even virtual. And we consider other options as well 🙂


7 thoughts on “Why having cloud load-balancer for on-premises services is not a good idea?

  1. How about deploying haproxy or NGINX in reverse proxy mode on a VM in one of Polish Cloud Service Providers? Sure, it is not ready to consume cloud service, but if you want to limit system installation and maintenance overhead you can use some cloud-optimized linux distro (like CoreOS Cloud Linux) with built-in docker engine + official haproxy docker container from docker hub repository... Setting up such a system should take no more than half an hour including writing haproxy config. And of course you can hide multiple instances behind CSP load-balancer or just use DNS round-robin for increased availability.
    • Sure, that's an option but what I meant in this article is using purely public cloud load balancing solution, not the service implemented on VM in the public cloud. People do that usually if they want to load balance between public cloud instances and on-premise installation. But it is not solving one problem which is the cost of outgoing traffic from the cloud. And you still have server maintenance overhead. But in some scenarios, of course, your proposal will fit :)
  2. I am late to the party, but why limit options to AWS,Azure, E24? What about cloud load balancer as a service - like https://www.rackspace.com/en-gb/cloud/load-balancing ? Servers are on-premise, LB is in the cloud, assuming it is all in same region (UK in this case) healthchecks should work OK. It supports SNAT as far as I know, so another point no longer valid. Am I missing something here?
    • Yes, there are some services that allow load balancing. But in this internal discussion, we considered AWS as an option. It is always a matter of reliability, available features, and cost of the service.
  3. AWS's new application load balancer now supports hybrid load balancing between both on-premises and cloud targets: https://aws.amazon.com/blogs/aws/new-application-load-balancing-via-ip-address-to-aws-on-premises-resources/
    • But you still have to pay for the traffic! The more your users (or DDoS attack) generate, ​the more you pay.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: