Rate limits

Each account can have up to 200 requests in flight at the same time for each model. Limits are per model, so you can run 200 concurrent requests on one model and 200 on another.

There is no separate requests-per-minute or tokens-per-minute limit. How many requests per minute you can send depends on how long each one takes:

Average request time	Requests per minute at 200 concurrent
1 second	about 12,000
10 seconds	about 1,200
60 seconds	about 200

When you hit the limit

Requests above the limit return 429. You can also get 429 below the limit when a model is under unusually heavy load, while capacity scales up. In both cases, wait a moment and retry with exponential backoff.

For large batch jobs, cap your own concurrency below 200 with a queue or semaphore instead of relying on 429 responses.

Higher limits

Email [email protected] with the model, the concurrency you need and a short description of the workload.