What is Web Bot Auth, and should your AI agent sign requests?

Web Bot Auth is a way for an automated client, like an AI agent, to prove who it is on every HTTP request using a cryptographic signature. instead of a website guessing from your user agent string or IP address, it checks a signature against a public key you have published. if it matches, the site knows the request really came from you.

I run browser and phone agents out of Singapore, and the question I keep hearing from other operators is simple: do I need this, and what happens if I skip it? short answer, you probably don’t need it today for a small private agent, but if you run an agent that visits sites you don’t own, it is worth understanding now. this piece covers what it is, how it works, and where it falls short. details here are as of October 2026, and the standards behind it are still moving, so check the linked docs before building anything.

what it is

Web Bot Auth is not one product. it is a pattern built on an existing internet standard, HTTP Message Signatures, defined in RFC 9421. that RFC describes how to sign parts of an HTTP request so the receiver can check they were not altered and came from the holder of a particular key.

Cloudflare popularised the “Web Bot Auth” name in 2025 when it proposed using those signatures to identify bots and agents, as an alternative to relying on IP lists and user agent strings. an IETF working group named webbotauth now exists to standardise the details, and you can follow its drafts on the IETF datatracker. as far as I can tell, those documents are still drafts as of October 2026, not finished RFCs.

In plain terms, Web Bot Auth gives your agent a verifiable name tag. the tag is hard to forge because it relies on a private key only you hold.

how it works

The flow has four parts. I’ll describe the version in the drafts and in Cloudflare’s Web Bot Auth documentation. names of headers and fields can change as the drafts change, so treat this as the shape, not the spec.

you generate a keypair. the drafts use Ed25519, a common signature algorithm. the private key stays with your agent. the public key is what you share.
you publish the public key. you host it as a JSON Web Key Set on a URL you control, in a “key directory”. the drafts suggest serving it from a well-known path on your domain, /.well-known/http-message-signatures-directory.
your agent signs each request. it adds a Signature-Agent header pointing at your key directory, plus Signature-Input and Signature headers from RFC 9421. the signed parts typically include the target host, so a signature made for one site can’t simply be replayed against another. the input also carries timestamps for creation and expiry, and a tag value that marks it as a web-bot-auth signature.
the site verifies. it fetches your public key from the directory, rebuilds the signed string from the request it received, and checks the signature. if it passes, the site knows the request came from whoever controls that key directory.

A few things follow from this design:

the identity is tied to a domain you control, because the key directory lives on your domain.
nothing about the request body or your agent’s behaviour is proven. only the origin is.
the site still decides what to do with a verified identity. it can allow, rate limit, charge, or block you.

On the receiving side, Cloudflare’s docs describe a registration step for bot operators who want to be recognised as a verified bot by their network. other sites or CDNs may build their own checks from the same drafts. I would not assume any one vendor’s process applies everywhere.

why it matters

Here are the reasons I think this is worth your attention, in rough order of how much they affect operators day to day.

IP addresses are a weak identity. an IP can be shared, rotated, or rented, and cloud ranges get lumped together. a signature identifies the operator rather than the network the request happened to leave from.
it gives sites a way to say yes. most bot defences are built to say no. a signed, identifiable agent gives a site owner something they can reasonably allow, which matters if your agent acts for a consenting user on a site’s own terms.
accountability runs both ways. if your agent misbehaves, the site can see which operator it was and contact you. that is a cost for bad actors and a feature for honest ones.
it separates “which agent” from “which user”. one operator’s key can sit behind many users’ tasks, so the signature says who is running the agent, not who asked it to run. keep that distinction in mind when you write your own logs.

common misconceptions

I see the same wrong takes repeated, so here are the ones worth correcting.

“signing makes my agent trusted.” no. a signature proves identity, not good behaviour. a site can verify you and still rate limit or refuse you based on its own policy. identity is the entry ticket, not a free pass.
“it replaces proxies and IP choices.” no. signing sits at the application layer and does nothing to your network path. whether you pick residential or mobile proxies for a browser agent is a separate decision about where requests come from and what the site expects from that location. the two can work together, since a stable signed identity does not need a rotating IP.
“it stops prompt injection or protects my agent.” no. Web Bot Auth tells a site who you are. it does nothing about hostile content a page feeds your agent. for that, read prompt injection for browser agents explained and think about isolation, for example with a sandbox for a computer-use agent.
“every site checks it now.” no. adoption is uneven. some CDNs and platforms can verify signatures, many sites ignore them, and a missing signature is not automatically a reason for a block. I would not build a workflow that depends on a site honouring your signature today.

There is one more mistake I’d flag, which is treating a leaked key lightly. if someone gets your private key, they can sign requests as you. the drafts include expiry times and key rotation for that reason, so plan for rotation from day one and keep the private key out of logs and repos.

so should your agent sign requests?

This is my own view as an operator, not a rule, and it is not legal advice.

I would sign if:

your agent regularly visits sites you do not own, and you want a stable identity those sites can recognise and contact.
you run a public-facing crawler or assistant that sites may want to allow on purpose.
you are already getting blocked or challenged for being an unrecognised bot, and you are operating within each site’s terms.

I would wait if:

your agent only works on your own properties or on accounts and pages you are authorised to use, where an allowlist or API key is simpler.
you cannot yet host a stable key directory on a domain you control.
the target sites you care about have no signature check, so there is nothing for a signature to do.

If you do start, the practical path is small. pick a domain for the key directory, generate an Ed25519 keypair, publish the directory, then add the signing step at the point where your agent makes requests. with Playwright you would typically do this at the network layer through request interception or a proxy in front of the browser, which is one more reason to compare tools first. my notes on Playwright vs Puppeteer for AI browser agents cover how each handles request-level control. test against a site that documents verification, such as one behind Cloudflare with Web Bot Auth enabled, rather than guessing.

One caution on the signing step: the browser sends many requests per page, so signing everything adds load and complexity. for most agents I’d sign top-level navigations and API calls first, then expand if a site needs more.

where to go from here

If this was your first look at Web Bot Auth, here is where I’d read next:

the RFC 9421 text for the signing mechanics, which are less scary than the length suggests.
Cloudflare’s Web Bot Auth docs for how one large network verifies signed agents and how operators register.
prompt injection for browser agents explained, because identity is only half of running an agent safely.
the full list of guides on the blog index, including pieces on sandboxing and proxy choice.

I’ll update this when the IETF drafts settle or when more platforms publish how they verify signatures.

Written by Xavier Fok

disclosure: this article may contain affiliate links. if you buy through them we may earn a commission at no extra cost to you. verdicts are independent of payouts. last reviewed by Xavier Fok on 2026-10-07.