Takeover to test steps

How manual actions during a takeover become reviewable, re-runnable steps.

Three techniques are fused, and the order matters.

Resolve at click time

On each human click the broker asks the browser which node is under the cursor, describes it, reads the relevant part of the accessibility tree and generates a locator through the cascade: role and accessible name, label, test id, placeholder or text, then CSS. Never XPath. This happens before the mouse press is dispatched, because after the click the page may have navigated and the element may be gone.

Two locators are stored per step, a primary and a secondary, so the healer has a second anchor later. Keystrokes are attributed to the focused element; password-shaped fields are masked at capture and stored as {{secret:<field>}}.

Diff the accessibility tree

The tree is snapshotted before the takeover and after hand-back. The diff describes what your actions accomplished (a URL change, a new heading, a changed value) and proposes assertions from it. When the recorded actions do not explain the observed change, that is surfaced too: it usually means an action happened in a cross-origin frame or a native dialog the recorder could not see.

Compile, then confirm

The recording compiles deterministically into named steps with parameters. A bound model can refine names, but the compilation never depends on one. The proposal is shown on the run page as a real review step, and nothing is persisted until you accept it.

After hand-back the agent re-plans

The agent discards its snapshot and plan, re-reads the frame tree (you may have navigated or opened a tab), and asks the model for a new plan given the original goal, the steps completed before the pause, a summary of what you did, and the fresh observation. One extra model call is cheaper than a wrong click on a live system.