Why a browser agent re-discovers the same UI elements on every single run

If you run a browser agent on the same task every morning, say logging into your own dashboard, opening a report and exporting a CSV, you would expect it to get better at it. It doesn’t. On run 200 it looks at the page the way it did on run 1. It reads the page again, works out which element is the “Export” button again, and pays for that thinking again.

I run agents for a living out of Singapore, and this is one of the first things that surprises people who come from ordinary scripting. A script remembers its selectors because you wrote them down. An LLM-driven agent has nothing written down. This article explains why, what it costs, and what changes if you record stable selectors. It also answers a question that comes up on the browser-use GitHub discussions (#3856): why doesn’t the agent just remember the elements it found last time?

what it is

Element re-discovery is the work an agent does at the start of every step, and every run, to figure out what is on the page and which part of it to act on. It is not a bug; it is the default design.

A few terms, in plain language:

• element: a button, link, input or other thing on the page you can interact with
• selector: a stable way of pointing at one element, such as a CSS selector, an XPath, a test id or an accessible role plus name
• discovery: the agent working out, from the current page, which element matches the goal (“click export”)
• persistence: keeping what discovery found so the next run can skip the work

The short answer to the discussion question is this. Nothing persists between runs because, by default, the thing that finds the element is a language model reading a fresh snapshot of the page, and a model call has no memory beyond the prompt it was given. The run ends, the context is thrown away, and the knowledge “the export button is the one labelled Export in the toolbar” goes with it.

how it works

Most browser agents, including the open source browser-use project, follow the same loop. I am describing the general pattern here, not claiming details of any one release, so check the code of the version you run.

1. the agent takes a snapshot of the page. Depending on the tool this is a trimmed DOM, an accessibility tree, a screenshot, or a mix
2. it turns the interactive parts into a list the model can read, often with a temporary index number next to each one
3. the model reads the goal plus that list and picks an action, like “click item 14”
4. the tool maps item 14 back to a real element and performs the click
5. the page changes, and the loop repeats from step 1

The key detail is step 2. Those index numbers are assigned fresh for each snapshot. Item 14 on this step is not a durable name for the export button. If a banner loads above it on the next run, the same button might be item 17. The index was never meant to outlive the step, so there is nothing worth saving.

The accessibility tree is a useful input here. MDN describes it as a structure browsers build from the DOM for assistive technology, with roles, names and states. Agent tools like it because it is shorter than raw HTML and gives the model meaningful labels. The Chrome DevTools Protocol exposes it through its Accessibility domain, which is one route tools use to read it.

Now compare that with how a human-written Playwright test works. The Playwright docs recommend locators built on role, text and test ids, such as getByRole('button', { name: 'Export' }). That locator is written once, saved in a file, and reused on every run with zero model calls. The agent does the same job, finding the Export button, but with an LLM call each time and no file to save the answer in.

So there are three separate reasons nothing carries over:

• the model is stateless, so only what is in the prompt exists
• the element indexes are per-snapshot and not stable identifiers
• the agent framework usually has no step that writes “what worked” to disk and reads it back on the next run

Some setups add memory features or let you pass in saved instructions. Those help with task knowledge, like “the report is under Analytics”. As of October 2026, I have not seen a default behaviour in the agents I use that records the resolved selector for each step and replays it automatically. If your version does, good, but verify it by watching what actually happens on a second run rather than trusting the docs.

why it matters

There are four reasons I care about this when running repeat tasks.

cost

Every re-discovery step is a model call carrying a page snapshot. On a long page, that snapshot is thousands of tokens, and you send it again for every step of every run. A task that touches six pages is six snapshots, daily. I wrote about this pattern in cost engineering for computer use agents. The short version is that repeated reading of unchanged pages is often the biggest avoidable line item. It is also why a small change in how snapshots are built can move a bill a lot, as in what silently doubled a browser agent’s token bill after a routine version upgrade.

reliability

This one is less obvious. If the model has to choose the element fresh each time, it can choose differently each time. On a page with two buttons that both say “Export” (one in the header, one in a table row), run 1 might pick the header one and run 2 the row one. The page is ambiguous and nothing pinned the answer down. A recorded selector removes that coin flip for steps that worked before, because the agent would act on the element you already verified rather than re-deciding.

speed

A model call per step takes seconds. A replayed selector takes milliseconds. For a daily job that barely matters, but for a job a user is waiting on, or one that runs thousands of times, it adds up.

debuggability

When a run fails today, you often cannot tell whether the model misjudged the page or the page changed. With recorded selectors you get a clear signal: this selector worked for 40 runs and failed on run 41, so the page changed at that step. Tracing helps here too, and I list options in best tools for tracing and debugging AI agents.

what recording stable selectors would change

This is the other half of the discussion question. Suppose the agent saved, per step, the selector that resolved to the element it acted on, ideally a role-and-name locator or a test id rather than a position index. On the next run it would try the saved selector first and only call the model if that selector failed to match.

What would improve:

• repeat runs on unchanged pages would need far fewer model calls
• results would be more consistent, since the same element gets hit each time
• failures would be easier to localise, since a broken selector points at the exact step

What would not improve, and what I’d warn about:

• pages that change often would invalidate selectors, so you still need the model as a fallback
• a selector that matches the wrong element silently is worse than one that fails loudly, so replay needs a check, such as confirming the expected text or URL afterwards
• steps that change state, like submitting a payment or sending a message, need care when replayed or retried. That is an idempotency problem, and a fast replay makes double-submits easier to cause, not harder

You can approximate this today by hand. Run the task, read the trace to see which elements were used, then write those as Playwright locators in a normal script, and keep the agent only for the steps that vary.

common misconceptions

“The agent learns from repeated runs.” It doesn’t. Running a task 500 times does not train the model or update anything. Each run starts from the same place unless you built storage around it.

“Element indexes are IDs I can reuse.” They aren’t. They are labels for one snapshot. Saving “click 14” and replaying it later will click whatever is at position 14 that day.

“If it worked yesterday it will work today.” Only if the page is the same. Sites ship changes, and a well-behaved agent adapts, which is exactly why it re-reads the page. Re-discovery is the cost of that flexibility.

“Persistent browser sessions fix this.” They fix a different problem. Keeping cookies and logins, for example by attaching to an existing Chrome session instead of launching a fresh browser, saves you from logging in again. The agent still has to find the buttons after you are logged in.

where to go from here

If you want to dig further, these are the follow-ups I’d read in order:

• cost engineering for computer use agents: how to measure what re-reading pages costs you
• best tools for tracing and debugging AI agents: how to see which elements your agent actually picked
• idempotency for AI agents explained: how to make retries and replays safe
• the blog index for the rest of the agent explainers

Then read the Playwright locators guide and try converting one daily agent task into a saved script for its stable steps.

Written by Xavier Fok

disclosure: this article may contain affiliate links. if you buy through them we may earn a commission at no extra cost to you. verdicts are independent of payouts. last reviewed by Xavier Fok on 2026-10-23.