More information about frames

[christian-bromann]

Expected Behavior

The Chrome Recorder should allow extensions like the WebdriverIO Recorder to make good choices selecting the frame the user interacts with.

Actual Behavior

Currently the recorder only returns a number, e.g.:

{
    "title": "Recording 10.1.2025 at 07:08:16",
    "steps": [
        {
            "type": "setViewport",
            // omitted
        },
        {
            "type": "navigate",
            // omitted
        },
        {
            "type": "click",
            "target": "main",
            "selectors": [],
            "offsetY": 84,
            "offsetX": 158,
            "frame": [
                0
            ]
        },
        {
            "type": "click",
            "target": "main",
            "selectors": [],
            "offsetY": 82,
            "offsetX": 111,
            "frame": [
                1
            ]
        },
        {
            "type": "click",
            "target": "main",
            "selectors": [],
            "offsetY": 90,
            "offsetX": 94,
            "frame": [
                2
            ]
        },
        {
            "type": "click",
            "target": "main",
            "selectors": [
                [
                    "body"
                ]
            ],
            "offsetY": 132,
            "offsetX": 998
        }
    ]
}

It is unclear to what this number refers to. It is certainly difficult for a WebDriver based framework to identify the frame the user interacts with.

See the reference issue from the WebdriverIO recorder: webdriverio/chrome-recorder#150

Proposal

Can we add additional information about the frame, e.g. a generated xPath or other forms of identification?

[OrKoN]

The number refers to the index of the frame in the frame tree level, each array item is a level, so the first number N is Nth frame in the main frame. Computing selectors wouldn’t be supported any time soon due to difficulty of computing exact selectors at the time of event in multi process architecture of the browser.

[OrKoN]

In hindsight, we probably should not have exposed both target and frame in recordings as those are very specific to the Chrome implementation and used something else but I think short-term and mid-term it’s not going to change.