Architecture which Never Fails but Scales
In today’s age, no one can imagine if he is unable to book flight or shop online at any given time due to system failure. Though technology brings everything to your fingertips but it’s highly volatile as well at the same time because software must run on a hardware which is built using tens of thousands of small electronic circuits. Probability of failure is high because if probability of one circuit’s failure is 0.0001 and if there are 10K circuits, probability of failure is much higher (I believe there is no need to prove 😊).
Redundancy is KEY for any computer architecture whether it’s deployed in house or Cloud. Redundancy means though you need two items to fulfill the need, but you keep additional one(s) in case any of these fails.
When browser request to open a webpage, request reaches to the DNS Server and eventually reach to the web server to process the request but if due to hardware failure, server is down then request will not be served. To avoid this situation and remain “ALWAYS ON”, there should be additional server(s) so that if one is down, request could be served by other servers. Well, how DNS server knows that existing mapping is not valid because server is down, and it should start routing to the new server. But who will inform DNS server(s) that there is new mapping? May be someone update its Domain to IP mapping manually or automated way, but DNS Servers are scattered across world, it will take some time before all DNS servers update its mapping which means users trying to access your website will not be able to access for undefined duration.
Is there a way that we don’t need to update DNS Mapping rather it remains static? Yes, that’s where Load Balancer (NLB) come into picture (NLB could be hardware or software. F5 is one of the famous Load Balancer). Load Balancer is a component which has intelligence to route traffic to different servers while retaining single IP Address. So, we define our NLB’s IP in DNS and NLB decides where it should route traffic. How NLB knows if any of the server is down? There are different mechanisms but easiest is heart beat between NLB and servers. NLB keeps pushing a message to each server and if it doesn’t hear back acknowledgement then it means that server is down. So, we resolved the problem. Correct? No, what if NLB itself fails. More NLB(s)? Again issue of DNS update? Think… there is a way
Single server has limited capacity and it can handle limited number of requests from customers. we need more computing power so, let’s increase the power of server itself with higher CPU & RAM. It works and now I’m able to handle more customer requests. It’s called VERTICAL SCALLING when you add additional power to the server. But, there is limit of the RAM & CPU plugged to a server and as we go for higher specs it keeps on getting more and more costly. Just imaging 2xRAM of 64 GB are cheaper than 1x128GB. Not a feasible option because you are bound due to hardware limitation & cost how many customers system can serve at a given time. Just imagine Uber, Ali Baba, Amazon handling millions of concurrent requests how it’s possible.
We need to have more servers which could serve more requests at given time and NLB should distribute the load to all the active servers. As customer base grows, I keep on adding new cheap servers rather than expensive servers. This is called HORIZONTAL SCALLING when you add more nodes (servers) and it proportionally increase the capacity.
To be continued……