BigQuery AI Token Quotas Are GA: 200B In, 20B Out, Per Project Per Day

What became generally available

The feature is a set of daily quotas on the number of input and output tokens consumed by generative AI functions in BigQuery, managed from IAM & Admin under Quotas & System Limits. You can raise them if your workload needs more capacity, or create an override to cap usage more tightly than the default. Both directions matter: the default protects you from a runaway query, and a stricter override protects a shared project from one team's experiment.

The coverage list is specific. AI.GENERATE, AI.GENERATE_TABLE, AI.GENERATE_TEXT, AI.CLASSIFY, AI.IF, AI.AGG and AI.SCORE are subject to the quotas. Embedding generation and semantic search functions such as AI.EMBED and ML.GENERATE_EMBEDDING are not, because they run on different, cheaper models with a different billing structure.

-- Estimate before you spend. AI.COUNT_TOKENS prices a dry run instead of a surprise.
SELECT
  COUNT(*)                    AS rows_to_process,
  SUM(tok.total_tokens)       AS estimated_tokens,
  SUM(tok.prompt_tokens)      AS estimated_input,
  SUM(tok.candidates_tokens)  AS estimated_output
FROM (
  SELECT
    review_id,
    AI.COUNT_TOKENS(
      'gemini-2.5-flash',
      CONCAT('Classify this review as positive, negative or mixed:\n', review_text)
    ) AS tok
  FROM `project.dataset.reviews`
  WHERE processed_at IS NULL
  LIMIT 1000
)
-- Multiply the per-1000-row average by your real row count, then compare against
-- the quota table: 200,000,000,000 input and 20,000,000,000 output tokens per day per project.

The default numbers, in units you can reason about

Input tokens: 200 billion per project per day, 150 billion per project per user per day. Output tokens: 20 billion per project per day, 15 billion per project per user per day. Output includes thought tokens generated during internal reasoning, which is the detail that breaks naive estimates, because reasoning-heavy prompts can produce far more billable output than the visible answer suggests.

Two more details worth knowing. Cached tokens do not count toward the quotas. And the quotas are tracked in increments of millions of tokens, so while you can set precise values, anything below a few million may not be reflected with perfect accuracy. The quotas are global across all regions for the project, which is what you want from a cost control, and they reset at midnight Pacific time.

# Budget arithmetic in Python, because the quota units are easy to misread.
QUOTAS = {
    "input_per_project_per_day":  200_000_000_000,
    "input_per_user_per_day":     150_000_000_000,
    "output_per_project_per_day":  20_000_000_000,
    "output_per_user_per_day":     15_000_000_000,
}

def plan(rows, avg_input, avg_output, users=10, batch_size=10_000):
    in_total = rows * avg_input
    out_total = rows * avg_output
    batches = -(-rows // batch_size)
    return {
        "rows": rows,
        "input_tokens": f"{in_total:,}",
        "output_tokens": f"{out_total:,}",
        "input_pct_of_project_quota": f"{in_total / QUOTAS['input_per_project_per_day']:.1%}",
        "output_pct_of_project_quota": f"{out_total / QUOTAS['output_per_project_per_day']:.1%}",
        "output_pct_of_user_quota": f"{out_total / users / QUOTAS['output_per_user_per_day']:.1%}",
        "batches": batches,
    }

# 5 million rows, 700 input tokens and 90 output tokens each
print(plan(5_000_000, 700, 90))
# Output is usually the binding constraint: 450M output tokens is 2.25% of the project cap
# but only 15M of headroom per user on a shared service account is a different story.

Enforcement is where the surprises live

BigQuery checks quota before execution. If the relevant quota is already exhausted, the query is rejected with a QuotaExceeded error and nothing runs. That is the good case.

The harder case is exhaustion mid-query. When a running query consumes enough tokens to exhaust a configured quota, new LLM calls within that query are rejected, and remaining rows that depend on those calls hit a quota exhaustion error. For AI.IF, if you have used max_error_ratio, partial results can still be returned as long as the error ratio stays inside the limit; otherwise the whole query fails. For AI.GENERATE, the job can complete, but rows that could not be processed come back with no generated output and an error message in their status field.

That last behaviour is the dangerous one in production, because it fails softly. A downstream query that sums a classification column will happily aggregate the rows that succeeded and silently under-count the rest. Subsequent generative AI queries in the same project fail with QuotaExceeded until the daily reset, so a bad afternoon can turn into a bad day.

# Mid-query exhaustion is not an all-or-nothing failure. Handle the partial result.
from google.cloud import bigquery

QUERY = '''
SELECT
  review_id,
  AI.GENERATE(
    ('Classify the sentiment', review_text),
    connection_id => 'us.conn',
    endpoint => 'gemini-2.5-flash',
    output_schema => 'sentiment STRING, confidence FLOAT64'
  ) AS result
FROM `project.dataset.reviews`
WHERE processed_at IS NULL
'''

def run_with_partial_handling(client, query):
    job = client.query(query)          # rejected up front with QuotaExceeded if exhausted
    rows = list(job.result())
    ok = [r for r in rows if r["result"] and not r["result"].get("error")]
    failed = [r for r in rows if not (r["result"] and not r["result"].get("error"))]
    return {"ok": len(ok), "failed": len(failed), "job_id": job.job_id}

# AI.IF can return partial results when max_error_ratio is used; AI.GENERATE completes
# the job but leaves an error message on the rows it could not process.

Output quotas are two orders of magnitude smaller than input quotas. Guess which one binds.

Estimate before you run, not after

The cheapest habit is a dry run on a sample. AI.COUNT_TOKENS gives you prompt and candidate token counts for a given model and prompt, so you can price a thousand rows, multiply, and compare that against the quota table before you touch the full table. The same function is useful for prompt optimisation: trim a prompt by 120 tokens and you cut input cost by 17% at 700 tokens per row.

In practice, output is the binding constraint far more often than input, because output quotas are two orders of magnitude smaller by default. If your workload is classification, that is fine, since outputs are short. If it involves reasoning models writing structured JSON with explanations, you can burn 20 billion output tokens faster than you expect.

# Quotas can be raised, lowered or overridden. Do it as code so the value is reviewable.
import subprocess, json

def current_quota(project, quota_id="GenAiInputTokensPerDay"):
    out = subprocess.run(
        ["gcloud", "beta", "quotas", "info", "list",
         "--service=bigquery.googleapis.com", "--project", project, "--format=json"],
        capture_output=True, text=True, check=True,
    ).stdout
    for q in json.loads(out):
        if quota_id and quota_id in q.get("name", ""):
            return q
    return None

def desired_limits(monthly_rows, avg_input, avg_output, working_days=21):
    per_day_in = monthly_rows / working_days * avg_input
    per_day_out = monthly_rows / working_days * avg_output
    return {
        "GenAiInputTokensPerDay": int(per_day_in * 1.5),
        "GenAiOutputTokensPerDay": int(per_day_out * 1.5),
    }

print(desired_limits(3_000_000, 700, 90))     # headroom of 50% over expected daily usage
# Apply through the Cloud Quotas API, gcloud, or Terraform so the override is in git.

Treat quotas as configuration, not as a console setting

A quota value that only exists in a console is a number nobody reviews. The Cloud Quotas API, gcloud and Terraform all let you manage these values as code, which means the change lands in a pull request with a reason attached. That is worth the small amount of wiring, because quota changes tend to happen at the worst possible moment, usually during an incident.

One caveat from the documentation: if you are using Gemini Enterprise Agent Platform models with provisioned throughput, billing is not based on token usage at all. Set the BigQuery token quotas high in that case, otherwise a quota designed for pay-per-token workloads will block queries you have already paid for.

# Fail loudly when a job is about to sit on the quota instead of quietly burning it.
from dataclasses import dataclass

@dataclass
class Guardrail:
    name: str
    limit: int
    used: int

    @property
    def ratio(self):
        return self.used / self.limit if self.limit else 0.0

def preflight(guardrails, warn=0.7, block=0.95):
    findings = []
    for g in guardrails:
        status = "block" if g.ratio >= block else "warn" if g.ratio >= warn else "ok"
        findings.append({"guardrail": g.name, "used_pct": round(g.ratio * 100, 1), "status": status})
    return findings

print(preflight([
    Guardrail("input_per_project", 200_000_000_000, 46_000_000_000),
    Guardrail("output_per_project", 20_000_000_000, 19_200_000_000),
    Guardrail("output_per_user", 15_000_000_000, 4_100_000_000),
]))
# [(input_per_project, 23.0, ok), (output_per_project, 96.0, block), (output_per_user, 27.3, ok)]
# Output is the one that runs out first in almost every pipeline I have seen.

Quotas are global per project and reset at midnight Pacific time.

The guardrails I would actually put in place

First, a preflight check that fails the job when a guardrail is above 95% rather than letting it try and produce partial output. Second, a per-user or per-service-account cap as blast radius control, so one notebook cannot consume a shared project's entire daily output budget. Third, a post-run assertion that counts unprocessed rows and refuses to write results downstream when that count is non-zero.

None of this is exotic. It is the same discipline you would apply to a rate-limited third-party API, and it is missing most often in the tools that feel like SQL. SQL feels like it either works or errors. Generative AI functions in SQL are the first thing in that environment that can half-work, which is why the accounting has to be explicit.