The API Gateway Pattern
What an API gateway does in front of many services: routing, authentication, rate limiting, aggregation. Three 50 ms backend calls took 151 ms in sequence and 50 ms in parallel.
An API gateway is a single entry point between clients and backend services that routes requests, verifies authentication once, applies rate limits and can aggregate several backend calls into one response. Interviewers expect you to note that it is a potential single point of failure, so it should be stateless, run as several instances and stay free of business logic.
An API gateway is the single entry point that sits between clients and a set of backend services. It routes requests, authenticates callers, applies rate limits, and can combine several backend calls into one response. In a test, three 50 ms calls cost 151 ms one after another and 50 ms when made in parallel.
- An API gateway is a single entry point in front of backend services that handles routing, authentication, rate limiting, TLS termination and tracing.
- When an API gateway aggregates three independent 50 ms backend calls, making them in sequence took 151 ms while making them in parallel took 50 ms.
- With parallel aggregation in an API gateway, total latency becomes the slowest backend call rather than the sum of all calls.
- The Backend for Frontend (BFF) pattern uses one gateway per client type, such as phone, web and partner API, each owned by the team that builds that client.
- A load balancer spreads traffic across identical instances of one service, while an API gateway adds API-aware policies such as per-client limits and routing by path and version.
01The problem it solves
Suppose a mobile app needs a product page: product details from one service, prices from another, reviews from a third. Without a gateway, the app calls three services directly. It must know their addresses, handle authentication three times, make three network round trips over a mobile connection, and change whenever the backend is reorganised.
An API gateway is a server in front of the services that provides one public entry point. Clients send one request to api.example.com. The gateway decides where it goes, checks who is calling, and shapes the response. Internal services can then be split, merged or moved without clients noticing.
02Core responsibilities
Routing: map paths and methods to backend services (/orders/* goes to the order service). Authentication and authorisation: verify tokens once at the edge and pass the caller's identity downstream, so every service need not implement it. Rate limiting and quotas: protect backends and enforce plans (see the rate limiting topic). TLS termination: handle HTTPS at the edge. Request and response transformation: translate protocols (public REST to internal gRPC), rename fields, or trim data. Logging, metrics and tracing: one place that sees all traffic and attaches a request id so a call can be followed across services.
Also common: caching of idempotent responses, retries and timeouts for backend calls, circuit breaking to stop calling a failing service, and canary routing that sends a small share of traffic to a new version.
03Aggregation: fewer round trips for the client
For the product page, the gateway can call the three services itself, over a fast internal network, and return one combined response. The important detail is to call them concurrently when they do not depend on each other.
import asyncio
async def call(n):
await asyncio.sleep(0.05) # a 50 ms backend call
return n
async def sequential():
return [await call(i) for i in range(3)]
async def parallel():
return await asyncio.gather(*[call(i) for i in range(3)])Timing both, the sequential version took 151 ms and the parallel one 50 ms. Total latency became the slowest call, not the sum. A real aggregator also needs a timeout for each backend and a rule for partial failure: if the reviews service is slow, return the page without reviews instead of failing everything.
04The Backend for Frontend variant
A phone app, a web app and a partner API want different payloads: the phone wants small responses, the partner wants stable versioned fields. A single general gateway ends up serving all of them awkwardly. Backend for Frontend (BFF) uses one gateway per client type, each owned by the team that builds that client. It keeps client-specific logic out of the core services, at the cost of several gateways to maintain.
05Costs and risks
Every request now passes through an extra hop, which adds a little latency and a component that must be highly available. Run several instances behind a load balancer, keep the gateway stateless, and monitor it closely, since if it fails, everything fails.
The bigger risk is design drift. Business logic creeps into the gateway because it is convenient, until it is a monolith that every team must change for every feature. Keep it to cross-cutting concerns and light aggregation. Deep business rules belong in services. Ready-made gateways (Kong, NGINX, Envoy, cloud API gateways) cover the standard features, so build custom logic only where you must.
06Gateway, load balancer, reverse proxy
A reverse proxy forwards client requests to servers and can cache and terminate TLS. A load balancer spreads traffic across instances of the same service. An API gateway adds API-aware features on top: authentication, per-client limits, routing by path and version, and aggregation. In practice one product (NGINX, Envoy) can play several roles, and a typical stack has a load balancer in front of a gateway cluster, with load balancing again between the gateway and each service.
07Common interview questions
Q: What is an API gateway?
A: A single entry point in front of backend services that handles routing, authentication, rate limiting and other cross-cutting concerns.
Q: Why aggregate in the gateway?
A: It cuts client round trips, and calling backends in parallel makes latency the slowest call rather than the sum.
Q: What is the Backend for Frontend pattern?
A: A separate gateway per client type, tailored to its needs.
Q: How does a gateway fail, and what do you do about it?
A: It can be a single point of failure and a bottleneck. Run it stateless in multiple instances and monitor it.
Q: Gateway versus load balancer?
A: A load balancer distributes traffic among identical instances. A gateway understands the API and applies policies and routing.
Q: What should not go in a gateway?
A: Business logic that belongs to individual services.