Building a Distributed Token Bucket Rate Limiter With Redis and Lua
The first rate limiter I shipped was an in-memory Map counting requests per IP, and it worked fine right up until we ran a second instance of the API behind a load balancer, at which point every client effectively got double the intended limit, since each instance kept its own counter with no idea the other existed. Moving the state into Redis and getting the check-and-decrement step to be genuinely atomic across concurrent requests is the actual engineering problem, and it's worth walking through properly rather than reaching for a library without understanding what it's doing underneath.
Why Token Bucket, Specifically
A fixed window counter (count requests in each clock-aligned minute, reset to zero) is simple but allows a burst of 2x the limit right at a window boundary, a client can send the full limit in the last second of one window and the full limit again in the first second of the next. Token bucket avoids this: tokens refill continuously at a steady rate up to a capacity ceiling, so the maximum burst is bounded by the bucket's capacity regardless of timing, while still averaging out to the intended steady-state rate over time.
Why the Logic Has to Live in a Lua Script, Not Application Code
The obvious-looking approach, read the current token count with GET, check if there are enough tokens, then decrement with DECRBY, has a race condition: two concurrent requests can both read the same count before either writes back, and both proceed as if they alone consumed a token. Redis executes Lua scripts atomically, the entire script runs as one uninterruptible operation from Redis's perspective, so the read-check-write cycle for the bucket state can't be split by a concurrent request landing in the middle of it. This is the actual reason to reach for EVAL instead of a sequence of separate commands, not just a performance optimization.
The Lua Script Itself
The bucket's state, current token count and the timestamp it was last refilled, is stored as two fields on a Redis hash keyed by the client identifier. On each call, the script computes how many tokens should have accumulated since the last refill based on elapsed time and the configured rate, caps that at the bucket's capacity, then checks whether enough tokens exist to satisfy the request.
-- KEYS[1] = bucket key, e.g. "ratelimit:user:4821"
-- ARGV[1] = capacity, ARGV[2] = refill_rate (tokens/sec)
-- ARGV[3] = tokens_requested, ARGV[4] = now (ms)
local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local refill_rate = tonumber(ARGV[2])
local requested = tonumber(ARGV[3])
local now = tonumber(ARGV[4])
local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
local tokens = tonumber(bucket[1])
local last_refill = tonumber(bucket[2])
if tokens == nil then
tokens = capacity
last_refill = now
end
local elapsed = math.max(0, now - last_refill) / 1000
local refilled = math.min(capacity, tokens + elapsed * refill_rate)
local allowed = 0
if refilled >= requested then
allowed = 1
refilled = refilled - requested
end
redis.call('HMSET', key, 'tokens', refilled, 'last_refill', now)
redis.call('EXPIRE', key, math.ceil(capacity / refill_rate) * 2)
return { allowed, math.floor(refilled) }Calling It From Node with ioredis, Correctly Cached
Sending the full Lua source with every EVAL call works but wastes bandwidth and parse time on every request. Loading the script once with SCRIPT LOAD and calling it by its SHA1 hash via EVALSHA is faster in steady state, but Redis can evict cached scripts (a restart, a FLUSHALL, a failover to a replica that never saw the load), so production code needs a fallback: catch the NOSCRIPT error and retry with a plain EVAL, which also reloads the script into cache for next time.
const Redis = require('ioredis');
const redis = new Redis(process.env.REDIS_URL);
const LUA_SCRIPT = `...`; // the script above
let scriptSha = null;
async function checkRateLimit(key, capacity, refillRate, tokensRequested = 1) {
const now = Date.now();
const args = [key, capacity, refillRate, tokensRequested, now];
try {
if (!scriptSha) {
scriptSha = await redis.script('LOAD', LUA_SCRIPT);
}
const [allowed, remaining] = await redis.evalsha(scriptSha, 1, ...args);
return { allowed: allowed === 1, remaining };
} catch (err) {
if (err.message.includes('NOSCRIPT')) {
const [allowed, remaining] = await redis.eval(LUA_SCRIPT, 1, ...args);
scriptSha = await redis.script('LOAD', LUA_SCRIPT);
return { allowed: allowed === 1, remaining };
}
throw err;
}
}Express Middleware That Fails Open, Deliberately
Wrapping the check in Express middleware raised an operational question I hadn't thought through initially: what happens to every request if Redis itself is unreachable? Failing closed, rejecting all traffic because the rate limiter is down, turns a Redis blip into a full outage of the actual API. I fail open instead, catching a Redis connection error specifically and allowing the request through with a logged warning, treating rate limiting as a protective layer that degrades gracefully rather than a hard dependency the whole API goes down without.
function rateLimitMiddleware(capacity, refillRate) {
return async (req, res, next) => {
const key = `ratelimit:${req.user?.id || req.ip}`;
try {
const { allowed, remaining } = await checkRateLimit(key, capacity, refillRate);
res.set('X-RateLimit-Remaining', remaining);
if (!allowed) return res.status(429).json({ error: 'rate limit exceeded' });
next();
} catch (err) {
console.error('rate limiter unavailable, failing open:', err.message);
next();
}
};
}Setting Capacity and Refill Rate for Real Traffic Patterns
The numbers matter more than the algorithm. A capacity of 20 with a refill rate of 5 tokens per second allows a legitimate user to burst 20 requests immediately, useful for a page load firing several requests at once, then settles to a steady 5 requests per second afterward. Setting capacity too close to the steady-state rate defeats the point of using token bucket over a simpler fixed window in the first place; the bucket needs real headroom above the average rate to actually absorb bursts.
Final Verdict
The Lua script is maybe thirty lines, but it's the thirty lines that make the difference between a rate limiter that's actually correct under concurrent load and one that silently allows double the intended limit under real production traffic. If you're currently rate limiting with an in-memory counter behind more than one instance, this is the fix, and it's worth understanding the atomicity argument specifically, not just copying the script.
