User agent string cannot be set when using command line headless option

[chingc]

Hi, I'm using pyleniumio 1.12.2 and it seems that the user agent string override option in Chrome isn't respected if the headless option is invoked on the command line.

This was tested on

[WDM] - ====== WebDriver manager ======
[WDM] - Current google-chrome version is 89.0.4389
[WDM] - Get LATEST driver version for 89.0.4389

The test:

def test_user_agent(py):
    assert py.execute_script('return navigator.userAgent') == 'HelloWorld'

In this example, pylenium.json with headless option and user-agent override works as expected.

pylenium.json
-------------
{
    "driver": {
        "browser": "chrome",
        "options": ["headless", "user-agent=HelloWorld"]
    }
}

command
-------
pytest -k test_user_agent

result
------
pass  # user agent string was: HelloWorld

However, if the headless option is invoked on the command line, the user-agent override is not respected.

pylenium.json
-------------
{
    "driver": {
        "browser": "chrome",
        "options": ["user-agent=HelloWorld"]
    }
}

command
-------
pytest -k test_user_agent --options="headless"

result
------
fail  # user agent string was: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_3) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/89.0.4389.114 Safari/537.36

[ElSnoMan]

Hi @chingc,

First off, thanks for such a clean and readable Issue! I really appreciate the details and structure because that made it super easy to understand what's going on.

Ok, now to the good stuff. I changed this from bug to discussion because this is expected behavior, but I'm always open to discuss other solutions! Let's break down the order of operations with the PyleniumConfig object:

1. Instance of PyleniumConfig is created from pylenium.json
2. Fields are overriden (not appended) by CLI arguments
3. Fields can be further overriden or appended within custom fixtures or tests (I'll provide an example below)
4. Pylenium is then instantiated with the instance of PyleniumConfig

This was intentional because it provides pipelines and automation the ability to control the framework rather than having to rely on too many layers. In other words, you can get a lot of the information about the pipeline and its steps by looking at the scripts rather than having to look in multiple places (ie the scripts and pylenium.json), but it's still really flexible.

However, I could also see the use case for CLI args appending the options. Something like this which would combine them:

cli_browser_options = request.config.getoption('--options')
    if cli_browser_options: # if at least one option is passed via CLI
        for opt in [option.strip() for option in cli_browser_options.split(',')]:
            if opt not in config.driver.options: # if that option is not already in config.driver.options
                config.driver.options.append(opt) # then add it

Examples

1. I want to see the browser during local development, but --headless in the pipelines

pylenium.json

"driver": {
    ...
    "options": []
}

command

pytest --options="headless"

# ---or---

pytest --options="--headless"

---

2. I want to see the browser during local development, but --headless in the pipelines. However, local and cloud should always be in incognito mode

pylenium.json

"driver": {
    ...
    "options": ["incognito"]
}

command

pytest --options="headless, incognito"

# ---or---

pytest --options="--headless, --incognito"

May seem redundant to have "incognito" twice, but it makes it very clear to anyone looking at the pipeline or scripts what the options were for the Test Run

---

3. I want local and cloud to always be headless

"driver": {
    ...
    "options": ["headless"]
}

command

# nothing to add since pylenium.json handles it
pytest 

---

4. I want an option and target for local development, but I want the same option with a different target in pipelines

"driver": {
    ...
    "options": ["user-data-dir=/path/to/your/local/profile"]
}

command

pytest --options="user-data-dir=/path/to/your/cloud/profile"

---

5. I want options for local development, but remove them in the pipelines

"driver": {
    ...
    "options": ["incognito", "start-maximized]
}

command

# an underscore or any other character will work
pytest --options="_"

---

I hope that clarifies how things work! Thanks again for taking the time to submit an issue. Let me know what you think 😄

[chingc]

@ElSnoMan Thanks for the quick reply. I understand what's going on now, and the examples were especially helpful!