AI API rate limits explained

A rate limit constrains how an account or model can accept work over time. Identify the metric and scope before deciding whether a failed request should wait, change or stop.

Definition with a live example

Requests per minute describes a request-count allowance, while tokens per minute describes a token workload allowance. Concurrency describes active requests at the same time. These metrics answer different questions. A burst of small calls and a set of long document calls can stress different boundaries even when they complete the same number of business tasks.

Claude’s rate guide documents multiple capacity metrics and scopes; the applicable model and account conditions matter to the request. Official documentation.

The live table illustrates why scope belongs beside every value. An organization allowance is not multiplied by creating more keys, and a project may have a narrower control of its own. Keep the account hierarchy with the worker configuration so separate services do not assume they each own the full parent capacity.

Throughput limits are distinct from cumulative spending controls. A request can be affordable but too bursty, or well paced but blocked by an exhausted credit balance. Preserve the exact error code and account condition before changing the worker count or adding funds.

Latency is another measurement. Time to first output, time to final result and time waiting in your own queue can differ. Rate limiting governs accepted work; it does not provide a universal statement about how quickly every accepted response will finish.

How it affects cost / limits / results

Start by mapping all producers using the account. Include scheduled jobs, evaluation harnesses and interactive features. Independent local retry loops can collectively create a burst that no single service owner recognizes. Coordinate the shared resource with a queue or limiter whose scope matches the provider’s actual boundary.

Keep task deadlines and priorities explicit. An offline summary can wait differently from an interactive answer. A queued request should expose its state, and expired work should stop rather than consume capacity for a result nobody needs. Useful throughput counts accepted completed tasks, not merely attempts sent.

Retries need an owner. Decide whether the SDK, worker or another layer controls them, then set an attempt cap and deadline. Honor response timing when supplied and introduce jitter for temporary pressure. Do not retry invalid input, missing permissions or account funding conditions as if waiting could repair them.

A model fallback is a product choice. It can improve completion when a candidate is unavailable, but every permitted replacement needs the same mandatory capabilities, output checks and operating conditions. Preserve the actual model used and validate its result before calling the task recovered.

Use the workload calculator to inspect the cost of the traffic plan, including retries and additional stages. A higher capacity allowance can enable more spending, so keep throughput and budget controls coordinated.

For a capacity request, prepare evidence of the real workload: ordinary demand, burst shape, token distribution, acceptable waiting and current rejection pattern. A precise request is easier to evaluate than a demand for the largest available quota. After approval, ramp gradually and observe the rest of the application.

Provider differences

OpenAI limits use organization and project context with model-specific capacity. Follow the official rate guide and distinguish spend conditions from request throttling.

Gemini limits apply to project-scoped usage. Inspect Google’s current limits guide instead of assuming each new API key creates a separate pool.

Alibaba Model Studio limits aggregate usage under the cloud account. The official source distinguishes model policies and account-wide producers.

The examples show why a generic retry recipe cannot replace account mapping. Similar status codes can represent different conditions, and some models use dynamic capacity rather than a fixed published quota. Preserve the source’s uncertainty instead of filling the gap with a guessed allowance.

Asynchronous paths also differ. A provider may give batch work a separate queue or treatment, but that does not make it unlimited. Check model eligibility, submission constraints and result reconciliation before moving offline work to the batch interface.

Common mistakes

Do not create extra keys as a strategy to evade a shared limit. Keys identify access; the provider’s documented scope determines capacity. Coordinate legitimate workloads within that scope.

Do not react to every throttle by adding funds. First inspect whether the response describes credit, a configured cap, request pace or service overload. Those require different actions.

Do not count a partial stream as a completed task. Keep the final state and avoid repeating downstream side effects after retry or fallback. A recovery policy must reconcile work, not only obtain another response.

Do not judge a concurrency increase by attempted traffic alone. Watch accepted completion rate, waiting time, error rate and application validation together. More parallel workers can produce more contention without more useful output.

Do not assume an approved temporary increase lasts forever. Record its source and validity, then plan a graceful response if the account returns to its prior capacity.

Last verified 12 Sep 2026 · Source ↗

Frequently asked questions

Sources

• Claude rate limits ↗
• OpenAI rate limits ↗
• Gemini rate limits ↗
• Model Studio rate limits ↗

Last verified 12 Sep 2026 · Source ↗