Design a Rate Limiter

Question 1

What is a client-side rate limiter ?

Answer 1

A client-side rate limiter is a mechanism implemented on the client (browser, mobile app, or desktop app) to control the frequency of requests sent to a server. It helps prevent excessive API calls, improves performance, and ensures compliance with API rate limits.

Why Use a Client-Side Rate Limiter ?

• Prevent excessive API calls: Avoid sending too many requests in a short time.
• Reduce server load: Distribute requests over time to prevent server overload.
• Avoid hitting API rate limits: Many services (e.g., AWS, Stripe, OpenAI) enforce request limits.
• Improve user experience: Prevent UI slowdowns caused by frequent API requests.
• Lower costs: Avoid unnecessary API usage, especially for paid APIs.

Question 2

Why Use a Client-Side Rate Limiter?

Answer 2

1. Prevent excessive API calls: Avoid sending too many requests in a short time.
2. Reduce server load: Distribute requests over time to prevent server overload.
3. Avoid hitting API rate limits: Many services (e.g., AWS, Stripe, OpenAI) enforce request limits.
4. Improve user experience: Prevent UI slowdowns caused by frequent API requests.
5. Lower costs: Avoid unnecessary API usage, especially for paid APIs.

Question 3

What are the Common Client-Side Rate Limiting Strategies ?

Answer 3

1️⃣ Debouncing (Delay Rapid Requests)
2️⃣ Throttling (Limit Execution Rate)
3️⃣ Token Bucket Algorithm (Leaky Bucket)

See source code in algo project

Question 4

What is a server-side rate limiter ?

Answer 4

A server-side rate limiter is a mechanism used to control the number of requests a user or client can make to a server within a certain time period.
The goal of rate limiting is:
1. to prevent abuse
2. maintain the performance of the server
3. protect against potential attacks like Denial-of-Service (DoS) or brute force attacks.

In a typical web application, a rate limiter helps ensure that users or clients cannot overwhelm the server with too many requests, either intentionally or unintentionally. It does this by keeping track of the number of requests a client makes and limiting them once a certain threshold is exceeded.

Question 5

What are the requirements for a server-side rate limiter?

Answer 5

• Accurately limit excessive requests.
• Low latency. The rate limiter should not slow down HTTP response time.
• Use as little memory as possible.
• Distributed rate limiting. The rate limiter can be shared across multiple servers or processes.
• Exception handling. Show clear exceptions to users when their requests are throttled.
• High fault tolerance. If there are any problems with the rate limiter (for example, a cache server goes offline), it does not affect the entire system.

Question 6

What are some algorithms for rate limiting ?

Answer 6

• Token bucket
• Leaking bucket
• Fixed window counter
• Sliding window log
• Sliding window counter

Question 7

What are the problems to solve in a distributed rate-limiter ?

Answer 7

• Race condition: Lua script [13] and sorted sets data structure in Redis [8].
• Synchronization issue: sticky sessions or centralized data store like redis

Question 8

Name two performance optimizations

Answer 8

• multi-data center setup is crucial for a rate limiter because latency is high for users located far away from the data center
• synchronize data with an eventual consistency model

Question 9

Hard vs soft rate limiting.

Answer 9

• Hard: The number of requests cannot exceed the threshold.
• Soft: Requests can exceed the threshold for a short period.