Quick take
Your gateway should be boring. Pick one pattern that matches your clients, keep it thin (auth, rate limits, routing), and treat config changes like code deploys. Most gateway disasters come from stuffing business logic where it doesn’t belong.
I’ve built or inherited API gateways at every company I’ve worked with. At Decloud we ran a single edge gateway in front of about a dozen services. At a large telecom, I walked into a gateway that had become a second application server – hundreds of Kong plugins, custom Lua transformations, business rules nobody could trace back to a ticket. It took three engineers two months to untangle.
The gateway itself was fine. The pattern choice was the problem.
When you actually need one
Not every system needs an API gateway. If you have one service and one client, a gateway is overhead. You need a gateway when you have multiple services exposed externally, or when you have distinct client types (web, mobile, partner APIs) that need different response shapes.
At Decloud we hit that threshold around service number four. Before that, we just had nginx doing TLS termination and basic routing. That was fine. The moment we needed per-client rate limits and centralized auth token validation, we added a proper gateway. Not before.
The patterns worth knowing
Edge gateway
Single entry point. All external traffic flows through it. This is where most teams should start and where many teams should stop.
You get centralized auth, rate limiting, and routing. Your services stay behind a private network. The config is one place, the logs are one place, the failure mode is one place. I like boring. Boring is debuggable.
At Decloud, our edge gateway handled TLS termination, JWT validation, rate limits, and request ID injection. That’s it. We resisted every request to add “just one more transformation.” Every single time someone wanted to add response shaping or field filtering at the gateway layer, the answer was no. Put it in a service.
Backend for Frontend (BFF)
A separate gateway per client type. Your mobile app gets one gateway that returns compact payloads. Your web app gets another. Your partner API gets a third.
I ended up recommending this at a telecom where the mobile team and web team were in a constant fight over the shape of API responses. The mobile team needed tiny payloads. The web team wanted rich nested objects. One gateway couldn’t make both happy without turning into a mess of conditional logic.
BFFs solve this cleanly. Each team owns their gateway, shapes responses for their client, and deploys independently. The tradeoff is more gateways to operate. Worth it when the alternative is a god-gateway that tries to please everyone.
Service mesh ingress
If you’re already running Istio or Linkerd inside your cluster, your mesh’s ingress gateway can handle the edge. You get mTLS, traffic shifting, and telemetry for free because the mesh is already doing that internally.
I’d only go this route if the mesh is already in place and working. Don’t adopt a service mesh just to get a gateway. That’s backwards.
What belongs in the gateway
Keep this list short. I mean it.
- TLS termination
- Auth token validation (not the auth logic itself, just “is this token valid”)
- Rate limiting
- Request ID injection
- Consistent error formatting
- Basic routing
A typical route config should be dead simple:
routes:
- path: /api/orders
upstream: orders-service
timeout_ms: 3000
rate_limit_per_minute: 120
auth: required
add_headers:
X-Request-ID: $request_id
If your gateway config requires a senior engineer to review, it’s too complex.
What doesn’t belong in the gateway
This is the part people get wrong.
- Business validation. (“Is this order amount valid?” No. That’s a service concern.)
- Response aggregation across multiple services. (That’s a BFF or a dedicated orchestration service.)
- Data transformation beyond trivial header manipulation.
- Anything that requires state.
I watched a team spend six months debugging production issues that all traced back to a Kong plugin doing field-level authorization. The plugin had its own config, its own cache, its own failure modes. It was invisible to application-level monitoring. When it failed, the errors looked like they came from downstream services.
Don’t do this.
Operational discipline
A gateway is in every request path. Treat it accordingly.
Timeouts aren’t optional. At one company, I found a gateway with no timeout configuration. A slow downstream service caused requests to pile up at the gateway until it ran out of connections. The entire platform went down because one service was having a bad day. Set timeouts. Set them tight. 3-5 seconds for most APIs. If an operation takes longer, it probably shouldn’t be synchronous.
Retries need limits. One retry with jitter is usually fine. Aggressive retry policies create thundering herds. I’ve seen a gateway with 3 retries and no backoff take a struggling service from “slow” to “completely dead” in under a minute.
Config changes are deploys. Version your gateway config. Review it. Roll it out gradually. A bad config change at the gateway layer is an outage for every service behind it.
Choosing a technology
In 2021, I see three real options:
Kong if you want plugin-driven API management and fast setup. Good ecosystem, reasonable operational story. Watch out for plugin sprawl.
Envoy if you want deep traffic control and high performance. Steeper learning curve. Better fit for teams that already think in terms of proxies and traffic policy.
AWS API Gateway if you’re all-in on AWS and want managed infrastructure. Less flexibility, but less operational burden. Fine for most REST APIs.
I’ve used all three. For most teams, Kong or Envoy behind a load balancer covers everything you need. If you’re on AWS and don’t want to operate infrastructure, the managed option is perfectly reasonable.
The real lesson
The best gateways I’ve seen are the ones nobody talks about. They’re boring. They do five things well. They have clean configs, good logs, and predictable failure modes.
The worst gateways are the ones that became “the platform.” Business logic, custom transformations, complex routing rules that only one person understands. Once a gateway reaches that point, you’re not operating a gateway anymore. You’re operating a distributed monolith with extra latency.
Keep it thin. Keep it boring. Your future self will thank you.