What Backend Engineers Need to Know for Interviews — Load Balancer vs. Reverse Proxy vs. API Gateway vs. HAProxy

Posted on  Sep 11, 2024  in  Backend Interview Preparation  by  Amo Chen  ‐ 6 min read

Backend engineers often encounter the following technologies and tools in their daily work or during interviews:

  • Load Balancer
  • Reverse Proxy
  • API Gateway
  • HAProxy

The commonality between these technologies is that they are all on the front line of handling traffic, but their uses differ, which can lead to confusion.

This article will clarify the differences between these technologies and tools to understand their respective functions and application scenarios, avoiding common misunderstandings.

Load Balancer

As the name suggests, a Load Balancer is responsible for distributing incoming network requests to multiple servers. It effectively spreads out the stress of handling requests, preventing any single server from becoming overloaded and provides flexibility for backend server clusters, enhancing the system’s scalability.

Load Balancer functionality can be provided by hardware devices known as Hardware Load Balancer Devices (HLD) or by software, such as Nginx.

A common operational principle for load balancers is that requests are initially sent to the load balancer, which then forwards the requests to backend servers and receives responses from them, finally sending the responses back to the users. Since users constantly interact with the load balancer, we can flexibly expand the number of backend servers by simply adjusting the load balancer settings to boost the system’s request-handling capability.

load-balancer.png

One may wonder, “In this architecture, if the load balancer becomes overwhelmed with a large number of requests and responses, wouldn’t it affect the system’s availability and stability?”

That’s correct, which is why there’s a technique called Direct Routing (DR), also known as Direct Server Return. It allows backend servers to respond to users directly. The principle is that the load balancer modifies network packets to forward requests directly to the backend servers, enabling them to respond directly to users. This mode is well-suited for situations involving media streaming, although the network setup is more complex.

direct-routing.png

Aside from the operational principle mentioned above, there are load balancers using different methods:

  • DNS (Domain Name System) based load balancing, which mainly responds with different IP addresses to users during the DNS resolution stage for load distribution. The downside is the long update time required for DNS, making it less flexible.
  • HTTP redirects, using HTTP redirection to distribute load. The downside here is that users make two requests; the first receive a redirection response, and the second gets the actual response, resulting in slower overall response speed.

Additionally, understanding the distribution strategy is often necessary when discussing Load Balancers. Common strategies include:

  • Round Robin, where requests are distributed sequentially among servers.
  • Least Connection, which assigns requests to the server with the fewest current connections.
  • Least Response Time, directing requests to the server with the fastest response speed.
  • IP Hash, hashing the IP address to allocate it to a specific server, allowing the same IP to be serviced by the same server.
  • Random allocation.

load-balancing-algorithms.gif

ref: DesignGurus

Each strategy has its advantages, disadvantages, and suitable scenarios. For more in-depth explanations, refer to the article Load Balancing Algorithms.

Load Balancer Configuration Example

Here is an example configuration using Nginx as an HTTP load balancer with the Least Connection strategy:

http {
    upstream backend {
        least_conn;
        server backend1.example.com;
        server backend2.example.com;
    }

    server {
        location / {
            proxy_pass http://backend;
        }
    }
}

The above configuration should give a clearer view of the load balancer’s operation.

Reverse Proxy

A Reverse Proxy is easily confused with a load balancer.

The reason is that a reverse proxy often plays the role of a load balancer at the same time; for example, Nginx offers both reverse proxy and load balancer functionalities, and they have similar positions in backend system architecture.

reverse-proxy.png

However, while a load balancer focuses on distributing requests among multiple servers, a reverse proxy focuses on receiving requests and, based on their characteristics (e.g., the path of the HTTP request), forwarding them to different servers for processing. The following Nginx configuration demonstrates this difference, as a load balancer does not consider the location differences:

http {
    upstream backendv1 {
        least_conn;
        server backend1.example.com;
        server backend2.example.com;
    }

    server {
        location /api/v1/ {
            proxy_pass http://backendv1;
        }

        location /api/v2/ {
            proxy_pass http://backend3.example.com:8000;
        }
    }
}

Illustrated graphically:

reverse-proxy-and-load-balancer.png

Additionally, a reverse proxy also has the capability to cache responses, reducing the load on backend servers and enhancing response speed, as shown in the Nginx configuration below:

http {
    proxy_cache_path  /var/www/cache levels=1:2 keys_zone=one:10m;
    proxy_temp_path /var/www/cache/tmp;

    upstream backendv1 {
        least_conn;
        server backend1.example.com;
        server backend2.example.com;
    }

    server {
        location /api/v1/ {
            proxy_pass http://backendv1;
        }

        location /api/v2/ {
            proxy_pass http://backend3.example.com:8000;
        }

        location /static/ {
           proxy_pass http://static.example.com;
           proxy_cache one;
           proxy_cache_valid  200 302  10m;
           proxy_cache_valid  404      1m;
        }
    }
}

Common Issue: Forward Proxy vs. Reverse Proxy

A Forward Proxy is a proxy server that users can configure and is usually near the user side, whereas a Reverse Proxy is something users cannot configure and is a part of the backend service system.

From a user’s perspective, users make requests to a reverse proxy server, which then forwards the requests to backend servers, receives their responses, and returns them to the users.

For more details, see What is a reverse proxy? | Proxy servers explained.

API Gateway

API Gateway, similar to load balancer and reverse proxy, sits at the forefront of traffic management.

However, an API Gateway mainly deals with accepting various API requests and forwarding these requests to the defined servers based on predefined rules. API Gateways are often employed in microservices architecture, acting as a unified interface for multiple services working together to provide API services, thus introducing flexibility to backend systems.

The diagram below illustrates an API Gateway’s functionality well and helps understand its role:

api-gateway.svg

ref: What Is an API Gateway? A Quick Learn Guide

Besides processing various API requests, an API Gateway may provide the following features (depending on each service provider):

  • Security management mechanisms, including authentication, authorization, access control, encryption, etc.
  • Monitoring mechanisms, such as tracking latency, error rates, and call statistics.
  • Resource management like rate limiting.
  • Load balancing.

Common API Gateway solutions include NGINX Ingress Controller (used with K8s), Amazon API Gateway, among others.

For more information on API Gateways, see What Is an API Gateway? A Quick Learn Guide.

HAProxy

HAProxy is a widely-used open-source load balancer and reverse proxy server, known for its high availability and performance, and it provides proxy capabilities for both TCP and HTTP. Thus, HAProxy is not a type of technology but an open-source software.

Conclusion

This article sorts out the questions I encountered while preparing for system design. These technologies and tools indeed have overlapping functions, which is a common source of confusion. However, by clarifying their differences and application scenarios, you can design and explain more reasonably in interviews.

That’s all!

Enjoy!

References

Load Balancing Algorithms | Design Guru

8 Load Balancing Algorithms for Distributed Systems

What is DNS-based load balancing? | DNS load balancing

HTTP redirects

Direct Server Return

What is a reverse proxy? | Proxy servers explained

What Is an API Gateway? A Quick Learn Guide