"Get BoundingBox" fails if element is not visible, wo proper error.

[Andre-A-AtGithub]

Describe the bug
Using FOR loop to iterate "Elements" "Get BoundingBox" gives error.

This worked before when using "older" versions. After updating to latest version of Python, Robot framework, Browser, etc this issue arose. See below overview of exact "old" and "new" versions:

Python: 3.9 → 3.12
Robotframework: 4.1.1 → 7.0
Robotframework-browser: 8.0.2 → 18.5.1

For more information see Robot framework Slack: https://robotframework.slack.com/archives/C015KB1QSDN/p1717657032495999

Print screen of debug log

To Reproduce
Environment:
This mainly happens in the test auto test framework I use, it is for me not certain if this also occurs in other setups.

Steps to reproduce the behavior:

1. Run "Get Elements" in a way it gives more than 1 results
2. Use FOR loop to iterate the elements to find the boundingbox of the elements
3. When the list of elements contains more than 1 element in the list than there is a error: TypeError: 'NoneType' object is not iterable". I also added a image of the error.

Relevant code in our test framework:
${elements}= Get Elements ${selector}
FOR ${element} IN @{elements}
${box}= Get BoundingBox ${element}
IF ${box}
Append To List ${bounded_elements} ${element}
END
END
RETURN ${bounded_elements}

Expected behavior
That it iterates all elements that are in the list

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Windows 10/11 (both give same issue)
• Browser: Chrome
• Version: 125.0.6422.26 (developer build) (64-bits)

Additional context
Add any other context about the problem here.

[aaltat]

@allcontributors please add @Andre-A-AtGithub for bugs

[allcontributors]

@aaltat

I've put up a pull request to add @Andre-A-AtGithub! 🎉

[robinmackaij]

I'm assisting Andre with the analysis of this issue and this is my current understanding of what's happening:

In the past (when this framework / these keywords were created), Get Elements returned a list of Playwright ElementHandles (https://playwright.dev/docs/api/class-elementhandle). This has changed; now, Get Elements returns a list of Playwright Locators (https://playwright.dev/docs/api/class-locator).

This change seems to be (partially?) incompatible with the cascaded locator syntax (>>), especially with the * prefix.

What we see happening now is that where in the past we got something like <some_guid> >> text=some_text we now get <the locator that was used for Get Elements >> nth=# >> text=some_text.

Note the nth=# that Playwright injects.

It really goes off-track when * is involved. Here, ${element} is an element from an Get Elements result where the locator was something like *css=.Pane >> text=some_text
${element} >> css=.SubPane now results in css=.Pane >> text=some_text >> css=.SubPane >> nth=#
So instead of trying to get the nth of the matched .Pane elements, it now tries to get the nth of the .SubPane.

I hope this makes sense, it's hard to provide an example showing the issue, especially since such a sample would require different Playwright / Browser versions.

If the problem is clear, would there be any suggestions how we could repair the framework? Note there's quite a bit of use of the * option and mixed locator strategies. From the Playwright documentation I gather that getting an ElementHandle is still possible. Could this be made possible with the Browser keywords?

[aaltat]

@Snooz82 I think you did drive this change and I don’t anymore remember why it was done. Could you explain and even better if you have suggestions how to fix it.

[Snooz82]

the real origin was that back in the days playwright did actually keep a handle on the real element. similar to Selenium. BUT you may know the problems with Seleniums „Element Stale Exception“ that if that element changes, it may not be accessible.

so they changed it so, that the Locator elements basically just store the selector.

so if you do in playwright page.getByRole('button', { name='Login' }) you would also just get an object that contains the selector. not the element itself.

Playwright docs:

The method returns an element locator that can be used to perform actions on this page / frame. Locator is resolved to the element immediately before performing an action, so a series of actions on the same locator can in fact be performed on different DOM elements. That would happen if the DOM structure between those actions has changed

so when we do Get Elements we actually just do Get Element Count and then return a list of the original selector with the appendix >> nth=i while i is the index of the element.

[Snooz82]

the * selection does not work with the cascading selectors! that is independent if you get the guid or not, because this guid thing was still just resolved to a selector. we could try if we could do Get Element with a parent and do that with playwright API to see if that solves the issue with the *

[Snooz82]

So this Playwright code:

await page.locator('*css=.box >> "ABS"').locator('input')._selector

gives the following selector:

*css=.box >> "ABS" >> input

So it will not be the input element that is a child of the .box element, but the .box element itself.

SO:
We can not do anything about the * thing not working.
In these cases you have to use xpath ../.. and so...

[Snooz82]

And i think the overall problem here is, that you have an element that is not visible and therefore Get Bounding Box does fail with this error.

I can without any problems iterate over many elements like this:

${e}    Get Elements    input
FOR     ${el}     IN     @{e}
    ${t}    Get Attribute    ${el}     attribute=type
    Log To Console     ${t}
    ${b}    Get Element States    ${el}
    Log To Console     ${b}
END

As long as all of them are visible.
But once one element is hidden Get Bounding Box does fail.
I think we can fix that, that you get a proper Error Message instead of this.

[Snooz82]

Get BoundingBox does fail with weird error:

async function getBoundingBox(request2, state2) {
  const selector = request2.getSelector();
  const strictMode = request2.getStrict();
  const locator = await findLocator(state2, selector, strictMode, void 0, true);
  const boundingBox = await locator.boundingBox();
  return jsonResponse(JSON.stringify(boundingBox), "Got bounding box successfully.");
}

So locator.boundingBox(); does return null when element is not visible.
Thats why it fails.

If you are ok with it, i would change that issue to fix the keyword.

[robinmackaij]

Get BoundingBox does fail with weird error:

async function getBoundingBox(request2, state2) {
  const selector = request2.getSelector();
  const strictMode = request2.getStrict();
  const locator = await findLocator(state2, selector, strictMode, void 0, true);
  const boundingBox = await locator.boundingBox();
  return jsonResponse(JSON.stringify(boundingBox), "Got bounding box successfully.");
}

So locator.boundingBox(); does return null when element is not visible. Thats why it fails.

If you are ok with it, i would change that issue to fix the keyword.

If instead of raising an error, it would return None, that'd resolve the original issue.

I only have access to the codebase if I'm in office with Andre, I'll check the use of * in combination with loops; I wrote these keywords a bit over 3 years ago and they worked back then. Andre called on me since things broke after recent Playwright / Browser updates. I don't have all the details yet, I just know it did work back then. The information you provided does give me a better idea of what to look for though.

[Andre-A-AtGithub]

First of all I really appreciate the effort you all put in this. With the new info I am now able to reproduce this issue fully, I can confirm it is specifically with a element that is hidden. I used this test to reproduce it, it uses a dutch webshop.

*** Test Cases ***
Open browser and do stuff
@{elements} Create List
@{bounded_elements} Create List
New Browser chromium headless=No
New Page https://www.megekko.nl
Click css=#CybotCookiebotDialogBodyLevelButtonLevelOptinAllowAll
${elements}= Get Elements css=.hdr_catMenuMobile
FOR ${element} IN @{elements}
${box}= Get BoundingBox ${element}
IF ${box}
Append To List ${bounded_elements} ${element}
Log ${box}
END
END
Log ${bounded_elements}