Rate Limiter is used to control the rate of traffic sent by a client or a service. It limits the number of client requests allowed to be sent over a specified period.
Benefits:
- It prevents resource starvation caused by Denial of Service (DoS) attack
- Reduce cost
- Prevent users from being overloaded
API Gateaway is a middleware that supports:
- Rate Limiting
- SSL Termination
- Authentication
- IP whitelisting
- Service Static Content
We can place the rate limiter on:
- CLIENT-SIDE: Not recommended since we don’t have so much control here
- SERVE-SIDE: API Gateaway
Algorithms for rate limiting:
- Token Bucket: Container with pre-defined capacity. If there are not enough tokens, the request is dropped.
- Leaking Bucket: when a request arrives the system, it checks if the queue is not full, the request is added to the queue
- Fixed Window Counter algorithm: Only X number of requests are allowed per certain time
- Sliding Window Log Algorithm: Similar to Fixed Window Counter algorithm, but it also includes timestamp
- Sliding Window Counter Algorithm: Hybrid between fixed and sliding window log algorithm
High Level Design:
- In-memory cache – Redis to keep track of the counter
Detailed Design
Issues:
- Race Condition:
- Issue: Two requests at the same time increment the counter with the same number
- Solution: Lua Script and Sorted Sets Data Structured
- Synchronization:
- Issue: If you have more than 1 rate limiter, these are not synchronized
- Solution: Centralized REDIS
Performance Optimization:
- Multi Data Center Setup is crucial for a rate limiter because latency is high for users located far away from the data center
- Synchronize data with an eventual consistency model.
Monitoring
After the rate limiter is put in place, it is important to gather analytics data to check whether the rate limiter is effective. Primarily, we want to make sure:
- The rate limiting algorithm is effective.
- The rate limiting rules are effective.
Daniela Fernandez 