Token-Bucket Rate Limiting Pack

A fixed window can honestly report that every minute stayed under the limit while allowing twice the intended traffic in one second.

Sixty requests at 12:00:59 and sixty more at 12:01:00 fit into separate minute buckets. Your dashboard looks clean. The backend receives 120 requests across the seam.

The token-bucket model

The bucket has:

a maximum capacity, which is the allowed burst;
a refill rate, such as one token per second;
the last refill time;
the current token balance.

For each request, calculate tokens earned since the previous update, cap the result at capacity, then consume the request cost if enough tokens remain. A quiet client accumulates burst room. After the burst, it is held to the refill rate.

In a distributed system, refill and consume must be one atomic operation, commonly a server-side script or transactional command in the shared store. A read followed by a write can over-admit when several instances race.

Choose the identity and failure rule

Key the limiter by the real boundary: tenant, user, API key, or endpoint plus tenant. A proxy IP can group thousands of unrelated users. A user-only key can let one expensive endpoint consume the budget intended for everything.

Return enough information for the caller to behave well: limit, remaining capacity, and a retry time based on the next available token. Use a server-controlled or monotonic clock where possible.

Write down what happens when the limiter store is unavailable. A login or expensive AI endpoint may need to fail closed or use a small local emergency budget. A low-risk read may accept bounded fail-open behaviour. Do not let a timeout make the choice.

Verification checklist

Send traffic on both sides of a fixed-window boundary. The token bucket never grants an accidental double burst.
Empty the bucket and prove requests resume only at the refill rate.
Release concurrent requests from several instances against one key. Atomic accounting holds.
Run two tenant keys and prove neither consumes the other's tokens.
Simulate clock movement and limiter-store failure.
Check response headers and retry timing against the actual next token.
Load-test the maximum documented burst plus sustained traffic.

The starter accepts the policy, identity rules, capacity, refill count, distributed-store model, and request events. It returns the algorithm, atomic operation, header contract, retry timing, burst tests, and observability plan. Dry-run sends no traffic and changes no limiter state.

Download the runnable pack

Comment BUCKET for the pack. You are metering traffic, not counting calendar boxes.