Grounding Agents in Live Web Search Through an AI Gateway

There is a specific failure pattern behind this launch: when an agent needs a live web page, it usually guesses the URL and then calls something to fetch it. The guess is often wrong, which is why you see 404s showing up in agent traces as if they were model mistakes. Cloudflare's answer is to give the agent a search engine instead of a random URL generator, and to route it through AI Gateway so the calls land in the same logs, credits and access controls as model inference. The launch partners are Ceramic.ai, Exa and Linkup.

A retrieval layer connects the model to the world after its training cutoff

What actually launched, and why it matters

Cloudflare's Web Search API integrates partner search providers into AI Gateway. Your application gets fresh structured snippets from the web injected straight into context, which addresses a structural limit rather than a temporary one: models are trained and then frozen, so anything about recent events, changing APIs or fast-moving releases is outside what they know. The stated motivation is exactly that — models are only as good as the context you feed them. Instead of the agent guessing a URL or hallucinating a document, the retrieval step becomes a real query with real results, and the model sees a citation-ready snippet rather than a guess.

# REST path: one POST, no SDK required.
import json, os, urllib.request

ACCOUNT = os.environ['CLOUDFLARE_ACCOUNT_ID']
TOKEN = os.environ['CLOUDFLARE_API_TOKEN']
URL = 'https://api.cloudflare.com/client/v4/accounts/{}/ai/websearch/'.format(ACCOUNT)

payload = {
    'query': 'Cloudflare Workers Durable Objects long-running video pipelines',
    'provider': 'ceramic',
    'limit': 5,
    'options': {'gateway': {'id': 'default'}},
}

req = urllib.request.Request(URL, data=json.dumps(payload).encode(),
                             headers={'Authorization': 'Bearer ' + TOKEN,
                                      'Content-Type': 'application/json'})
with urllib.request.urlopen(req, timeout=30) as resp:
    data = json.loads(resp.read().decode())

# Keep the source link. A snippet without a URL cannot be cited or audited.
for row in data.get('result', []):
    print(row.get('title'), row.get('url'))

The crawler standards clause is the more interesting half

Cloudflare states that its search partners committed to its bot crawling standards: the crawler must comply with its publicly stated requirements for Verified bots, and web search responses must include a link to the location of crawled content. Read that as a product feature with a governance side effect. Attribution is not a courtesy bolted on at the end, it is part of the response contract, which means your pipeline can rely on getting a URL it can show a reader. It also means site owners get a defined interaction: identify your crawler, respect robots.txt, and hand back the source. If you are building anything that consumes this, keep the link attached to the snippet all the way through your own stack, because the moment you strip it you lose both the audit trail and the reader's ability to check the claim.

# Workers binding: one line in the request path, provider as a knob.
PROVIDERS = ['ceramic', 'exa', 'linkup']

export default {
  async fetch(request, env) {
    const q = new URL(request.url).searchParams.get('q') || 'cloudflare ai gateway';
    for (const provider of PROVIDERS) {
      try {
        const res = await env.AI.websearch({ gatewayId: 'default', query: q, provider, limit: 5 });
        const body = await res.json();
        if (body && body.result && body.result.length) {
          return Response.json({ provider, results: body.result });
        }
      } catch (err) {
        // Fall through to the next provider rather than failing the request.
      }
    }
    return Response.json({ error: 'no provider returned results' }, { status: 502 });
  },
};

Two integration paths: a REST call and a Workers binding

Wiring it up without an SDK

There are two paths. From an existing backend, mobile app or service, POST to the accounts-scoped AI Web Search endpoint with your Cloudflare API token, a query, a provider name, a limit and the gateway id in the options object. From Cloudflare Workers, it is a single binding call: pass the gateway id, query, provider and limit to env.AI.websearch, then read the JSON. Both accept bring-your-own-key with the search providers, the same pattern AI Gateway already supports for inference providers, so an organisation with existing contracts with one of the partners can point at its own credentials. Results are billed at the providers' list API pricing with no additional markup, and show up in the normal AI Gateway observability logs, which means a search call is queryable next to the inference call it fed.

# Grounding is a pipeline, not a call. Retrieve, filter, attribute, then answer.
def build_grounded_prompt(question, search_results, max_snippets=4):
    kept, seen = [], set()
    for row in search_results:
        url = row.get('url')
        text = (row.get('snippet') or '').strip()
        if not url or len(text) < 60 or url in seen:
            continue
        seen.add(url)
        kept.append({'url': url, 'title': row.get('title'), 'text': text})
        if len(kept) >= max_snippets:
            break

    if not kept:
        return {'prompt': None, 'action': 'answer from priors and say so explicitly'}

    context = '\n\n'.join('[{}] {}\n{}'.format(i + 1, r['title'], r['text'])
                          for i, r in enumerate(kept))
    return {'prompt': context + '\n\nQuestion: ' + question,
            'sources': [r['url'] for r in kept],
            'action': 'answer with inline citations'}

# A retrieval step that can silently return nothing is worse than no retrieval step.

Grounding is a pipeline, so build it like one

A single search call with a prompt template is not grounding, it is a demo. The pipeline has four steps and the middle two are where the value is. Retrieve with a real query. Filter aggressively, because a snippet that is too short, duplicated or missing a URL is worse than nothing — it consumes context and looks authoritative. Attribute, keeping the source URL bound to each snippet so the model can cite it and your logs can reconstruct where a claim came from. Then answer, and make the empty case explicit: if retrieval returns nothing usable, the correct behaviour is to answer from priors while saying so, not to quietly proceed as if retrieval had succeeded. Silent retrieval failure is the failure mode that costs you trust.

# Not every provider for every caller, and not every provider for every data class.
POLICY = {
    'public-docs-agent': {'providers': ['ceramic', 'exa', 'linkup'], 'zdr_required': False},
    'customer-support':  {'providers': ['exa'],                     'zdr_required': True},
    'internal-research': {'providers': ['ceramic', 'linkup'],       'zdr_required': True},
}

CAPABILITIES = {
    'ceramic': {'zdr': False},
    'exa':     {'zdr': True},
    'linkup':  {'zdr': True},
}

def allowed_providers(agent, requested):
    rule = POLICY.get(agent)
    if rule is None:
        return {'allowed': [], 'reason': 'agent not enrolled'}
    ok = [p for p in requested if p in rule['providers']]
    if rule['zdr_required']:
        ok = [p for p in ok if CAPABILITIES[p]['zdr']]
    return {'allowed': ok, 'reason': 'filtered by policy'}

print(allowed_providers('customer-support', ['ceramic', 'exa']))

Search cost and inference cost belong on the same ledger

Governance: credits, logs, retention and provider choice

Because the calls run through AI Gateway, the operational story is the one you already built for inference. Web search consumes AI Gateway credits, so spend is visible in one place instead of split across vendor invoices. Requests land in observability logs, so you can join a search call to the model call it fed and to the latency and error fields around it. Cloudflare says it will identify partners that support Zero Data Retention so you know when data is not retained, which matters if your queries carry anything sensitive. And it lets you control who gets access to which search provider, which is the part most teams under-specify: a public documentation agent and a customer support agent should not automatically have the same providers, and a retention-sensitive workflow should be restricted to a provider whose retention posture you have actually confirmed.

# Web search draws on the same AI Gateway credits as inference. Meter it.
PRICE_PER_CALL = 0.0  # list pricing from the provider, no Cloudflare markup

def search_budget(task, calls=1, tokens_in=0, tokens_out=0, usd_per_m_in=2.0, usd_per_m_out=10.0):
    inference = (tokens_in / 1e6) * usd_per_m_in + (tokens_out / 1e6) * usd_per_m_out
    return {'task': task,
            'search_calls': calls,
            'search_cost': round(calls * PRICE_PER_CALL, 6),
            'inference_cost': round(inference, 4),
            'total': round(calls * PRICE_PER_CALL + inference, 4)}

print(search_budget('docs-question', calls=2, tokens_in=1800, tokens_out=400))

# Twenty searches per request looks cheap until you multiply it by your traffic.

What to do before server tools arrive

Cloudflare says native Server Tools are coming into AI Gateway, with web search among the first, so you will eventually not have to define the tool yourself. Do not wait for that to fix your retrieval layer. Model the search step as an interface with a provider-agnostic request and a normalised result shape, so swapping Ceramic.ai, Exa or Linkup is a configuration change rather than a rewrite, and so a future server tool can slot underneath the same interface. Build the failure path second and the happy path third. And meter search calls in the same budget as tokens, because the expensive habit is not one big search, it is a loop that searches on every turn and never notices.