Skip to main content

Rate Limiting

SASHA limits how many requests each partner sends to the Partner API per minute. The limits are enforced at SASHA's edge, before a request reaches the API, and a request over a limit is rejected, not queued.

The limits cover every request to partner.api.sasha.eu, REST and gRPC alike. Each operation, such as submitting an embed job or reading a job, has its own limit, and the edge counts a request against the partner its access token belongs to:

  • Every credential and every server of one partner shares the operation's budget, whichever IP address the requests come from.
  • One operation reaching its limit does not affect the others. Polling jobs heavily does not use up the budget for submitting them.
  • Partners do not share a budget, even when they send from the same address.

A request that carries no valid access token, such as the token request itself or a request with an expired token, has no partner to count against. The edge counts it by the IP address it arrives from instead, so several servers behind one public address, such as a NAT gateway, share that budget.

If a workload needs a higher limit for an operation, contact SASHA.

A rejected request​

RESTHTTP 429 Too Many Requests, with an HTML body
gRPCUNAVAILABLE, with an HTML page as the status message

On gRPC, UNAVAILABLE is the same code a temporary outage returns, so a client handles both the same way.

Because the edge answers before the API is reached, the response carries no SASHA-Request-ID and no JSON ErrorResponse body.

A rejected request does no work. No job is created, no image is read, and no Record is changed.

A 429 with a JSON ErrorResponse body is one of the API's own status codes, not the rate limit: a video submit gets one when the partner already has as many video jobs in progress as it is allowed. On gRPC that refusal is RESOURCE_EXHAUSTED, not UNAVAILABLE.

Recover from a rejection​

Retry a rejected request only when the operation is safe to repeat, and only with the backoff in Retrying a request. That backoff waits less than a second in total, so it does not wait out the rate limit. It is there for a short outage: a REST 503, or on gRPC the same UNAVAILABLE.

While a partner sends an operation faster than its limit, the requests over the limit are rejected, and each retry counts against the same limit as the request it repeats. Lower the request rate rather than retrying harder.

Stay under the limits​

  • Cache the access token and reuse it until it expires, rather than requesting a new one per call. The token request is counted by IP address, and a request sent with an expired token is too. See Handle Token Expiration.
  • Replace status polling with callbacks. Polling one job every second spends 60 requests a minute of the job-read budget on that job alone.
  • Spread a large batch of submits over time instead of sending it in one burst. The submit limit counts every server of the partner together, so sending from more servers does not raise it.