OpenRouter Rate Limited Upstream: Meaning and Fixes

What the message actually says

If you searched for openrouter rate limited upstream, you probably saw a 429 whose text blames an upstream provider. A router such as OpenRouter does not run every model itself. It forwards your request to whichever provider hosts the model, and that provider has its own quotas. When the provider is saturated or your traffic exceeds what it allows for that route, it answers with a rate-limit response, and the router passes it back to you.

The practical consequence is that the limit is often not tied to your key at all. Two users on identical accounts can see different results at the same minute, because the shared provider path is busy for one model and idle for another. Treat the error as a temporary condition of the route, not as a verdict on your account.

Check the exact wording and any headers in the router's current documentation, since the details can change. What stays constant is the handling: wait, retry, and spread the load.

Tell the kinds of 429 apart

Not every 429 deserves the same reaction. Before you write retry logic, sort the responses you receive into three buckets.

• Your own request rate is too high. You sent more requests per minute than your key allows. Slow down and the problem disappears.
• Too many requests at once. Your concurrency is the issue, not the per-minute count. Long streamed responses make this easy to hit.
• The provider is overloaded. Nothing you did caused it. Waiting a few seconds, or switching route, is the only cure.

A direct API makes this sorting easy because the error code tells you which case you are in. On our endpoint, going over the per-minute limit returns HTTP 429 with the code rate_limited, too many parallel requests returns 429 with concurrency, and temporary overload returns 429 with a Retry-After: 5 header. Read the code from the JSON body, then pick the response that fits. Behind a router the same sorting is harder, which is why logging the full error body is worth the few lines it costs.

Retry with exponential backoff and jitter

The standard fix is to wait, double the wait after each failure, and add a random fraction so that many clients do not all return at the same instant. Without jitter, ten workers that failed together will retry together and fail together again.

The loop below uses the OpenAI Python SDK against our base URL. It turns off the SDK's built-in retries so the behaviour is visible, honours Retry-After when the server sends it, and falls back to a doubling delay capped at 30 seconds.

Copyimport random
import time

from openai import OpenAI, RateLimitError

client = OpenAI(
    base_url="https://api.openrouterratelimit.com/v1",
    api_key="YOUR_KEY",
    max_retries=0,          # we handle retries ourselves
)

def chat(messages, attempts=6):
    delay = 1.0
    for attempt in range(attempts):
        try:
            return client.chat.completions.create(
                model="uncensored",
                messages=messages,
            )
        except RateLimitError as err:
            if attempt == attempts - 1:
                raise
            retry_after = err.response.headers.get("Retry-After")
            if retry_after and retry_after.isdigit():
                wait = float(retry_after)
            else:
                wait = delay
                delay = min(delay * 2, 30)
            wait += random.uniform(0, 0.5 * wait)   # jitter
            print(f"429, waiting {wait:.1f}s (attempt {attempt + 1})")
            time.sleep(wait)

reply = chat([{"role": "user", "content": "Summarise exponential backoff in two sentences."}])
print(reply.choices[0].message.content)

Keep the attempt count small. Six tries with a cap of 30 seconds means a request gives up after roughly a minute, which is long enough to ride out a short spike and short enough that your own users are not left staring at a spinner. Retry only 429 and 5xx responses. A 400 or 401 will fail identically every time.

Cap concurrency before the server does it for you

Backoff reacts to a problem after it happened. A concurrency cap prevents it. If your batch job fires 200 requests with asyncio.gather, the first few dozen will succeed and the rest will bounce. A semaphore smaller than the limit turns that spike into a steady stream.

Copyimport asyncio
from openai import AsyncOpenAI

client = AsyncOpenAI(base_url="https://api.openrouterratelimit.com/v1", api_key="YOUR_KEY")
gate = asyncio.Semaphore(6)   # stay under the 8-slot ceiling of a full key

async def ask(prompt):
    async with gate:
        resp = await client.chat.completions.create(
            model="uncensored",
            messages=[{"role": "user", "content": prompt}],
            max_tokens=300,
        )
        return resp.choices[0].message.content

async def main():
    prompts = [f"Write a one-line tagline, variant {i}" for i in range(40)]
    results = await asyncio.gather(*(ask(p) for p in prompts))
    print(len(results), "done")

asyncio.run(main())

A full key on our API allows 8 concurrent requests and 300 requests per minute, so a semaphore of 6 leaves headroom. A trial key allows 2 concurrent and 60 per minute, so use 1 or 2 there. Remember that a streamed request holds its slot until the last chunk arrives, so a slow stream counts against you for its whole duration.

Shape requests so fewer of them get limited

Retries are a safety net. The better habit is to send fewer and lighter requests.

• Set max_tokens on tasks with short answers. It keeps each call brief and frees the slot sooner.
• Merge small calls. One prompt that returns a JSON list of ten items beats ten separate calls. Use JSON mode (response_format with json_object) to keep the output parseable.
• Queue instead of firing. A simple worker pool with a fixed size is easier to reason about than ad-hoc retries scattered through the code.
• Cache identical prompts. Test suites and demo pages repeat themselves more than you expect.
• Log the request ID. Every response carries an X-Request-Id header, which makes a support question answerable.

You can probe the headers of any response with a single call:

Copycurl -i https://api.openrouterratelimit.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"uncensored","messages":[{"role":"user","content":"ping"}],"max_tokens":5}'

The response includes X-RateLimit-Limit-Requests and X-RateLimit-Limit-Concurrency, which tell you the ceilings that apply to the key you used. Read them once at start-up and size your pool from the numbers rather than guessing.

Read your logs before you tune anything

Retry code hides problems if you never look at how often it fires. Log one line per 429 with the timestamp, the model, the error code, the request ID and how long you waited. After a day of traffic those lines answer the questions that matter.

• Is it bursty or sustained? A cluster of errors within a few seconds points to a spike in your own traffic. Errors spread evenly across the hour point to an average rate that is simply too high.
• Does it follow the clock? If failures bunch up at the same hours every day, the provider side is busiest then, and scheduling batch work for quiet hours is cheaper than any retry logic.
• Is it one model? If only one model route produces the errors, the shared capacity for that route is the cause and your other routes are fine.
• Do retries succeed on the first try? If most requests need three or four attempts, the delay is too short or the load too high.

Add a counter for the share of requests that needed at least one retry, and alert when it passes a few percent. A rising line is an early warning that you are close to a ceiling, long before users complain.

A circuit breaker for long outages

Backoff handles brief trouble. If the errors keep coming for minutes, stop sending for a short period instead of keeping every worker busy retrying. A circuit breaker counts consecutive failures, opens after a threshold such as ten, rejects new work for 30 seconds, then lets a single probe request through. If the probe succeeds the breaker closes and traffic resumes. This protects both sides: the provider is spared a retry storm, and your queue does not fill with requests that were never going to succeed.

Pair the breaker with a clear message to the end user, such as asking them to try again in a minute, rather than a spinner that never ends. People tolerate a short honest delay far better than an unexplained hang.

When a direct endpoint makes the problem simpler

A router adds flexibility, and with it an extra layer where limits can come from. If your workload needs one kind of model, a direct endpoint with stated limits is easier to plan around. You know the per-minute ceiling, the concurrency ceiling and the error code for each case, so there is no second party whose quotas you cannot see.

Our API is OpenAI-compatible with the model ID uncensored, priced at $0.15 per 1M input tokens and $0.50 per 1M output tokens from prepaid credit. Failed requests are not charged, so a request that ends in a 429 costs nothing. The docs list every endpoint and error code, and pricing shows the top-up options. New accounts can claim $0.50 of trial credit from the API key page to test their retry logic against real limits.

For background on how this endpoint differs from a router, see OpenRouter API: myths vs facts.

More guides

• Does OpenRouter Have Rate Limits? What to Ask Any API
• OpenRouter API: Myths vs Facts