Rate Limiter is used to control the rate of traffic sent by a client or a service. It limits the number of client requests allowed to be sent over a specified period.
Benefits:
- It prevents resource starvation caused by Denial of Service (DoS) attack
- Reduce cost
- Prevent users from being overloaded
API Gateaway is a middleware that supports:
- Rate Limiting
- SSL Termination
- Authentication
- IP whitelisting
- Service Static Content
We can place the rate limiter on:
- CLIENT-SIDE: Not recommended since we don’t have so much control here
- SERVE-SIDE: API Gateaway
Algorithms for rate limiting:
- Token Bucket: Container with pre-defined capacity. If there are not enough tokens, the request is dropped.
- Leaking Bucket: when a request arrives the system, it checks if the queue is not full, the request is added to the queue
- Fixed Window Counter algorithm: Only X number of requests are allowed per certain time
- Sliding Window Log Algorithm: Similar to Fixed Window Counter algorithm, but it also includes timestamp
- Sliding Window Counter Algorithm: Hybrid between fixed and sliding window log algorithm
High Level Design:
- In-memory cache – Redis to keep track of the counter
Detailed Design
Issues:
- Race Condition:
- Issue: Two requests at the same time increment the counter with the same number
- Solution: Lua Script and Sorted Sets Data Structured
- Synchronization:
- Issue: If you have more than 1 rate limiter, these are not synchronized
- Solution: Centralized REDIS
Performance Optimization:
- Multi Data Center Setup
- Monitoring
Daniela Fernandez 