Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebDriver failure with latest Selenium Server #20

Open
asifhg opened this issue Jun 6, 2023 · 8 comments
Open

WebDriver failure with latest Selenium Server #20

asifhg opened this issue Jun 6, 2023 · 8 comments

Comments

@asifhg
Copy link

asifhg commented Jun 6, 2023

I might be missing something in the execution. Any help would be greatly appreciated!

Install: Caius

https://github.com/tobijk/caius.git

Download: Selenium Server

https://github.com/SeleniumHQ/selenium/releases/download/selenium-4.9.0/selenium-server-4.9.1.jar

Execute: Server Start

java -jar selenium-server-4.9.1.jar standalone --host "127.0.0.1" --port 4445 --http-logs true --log-level "FINE" ;

Execute: Client Start

package require WebDriver ;
set seleniumServer "http://127.0.0.1:4445" ;
set caps [namespace which [WebDriver::Capabilities #auto -browser_name firefox]] ;
set session [WebDriver::Session #auto $seleniumServer/wd/hub $caps] ;
# The previous cmd does not return. Several minutes later we get the following,
# and the server is still alive.
::unknownError0

Log: Server

23:25:03.365 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
23:25:03.371 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
23:25:03.459 DEBUG [AutoConfiguredOpenTelemetrySdkBuilder.build] - Global OpenTelemetry set to OpenTelemetrySdk{tracerProvider=SdkTracerProvider{clock=SystemClock{}, idGenerator=RandomIdGenerator{}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.19.0, attributes={service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.25.0"}}, spanLimitsSupplier=SpanLimitsValue{maxNumberOfAttributes=128, maxNumberOfEvents=128, maxNumberOfLinks=128, maxNumberOfAttributesPerEvent=128, maxNumberOfAttributesPerLink=128, maxAttributeValueLength=2147483647}, sampler=ParentBased{root:AlwaysOnSampler,remoteParentSampled:AlwaysOnSampler,remoteParentNotSampled:AlwaysOffSampler,localParentSampled:AlwaysOnSampler,localParentNotSampled:AlwaysOffSampler}, spanProcessor=SimpleSpanProcessor{spanExporter=org.openqa.selenium.remote.tracing.opentelemetry.SeleniumSpanExporter$1@11c20519}}, meterProvider=SdkMeterProvider{clock=SystemClock{}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.19.0, attributes={service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.25.0"}}, metricReaders=[], views=[]}, loggerProvider=SdkLoggerProvider{clock=SystemClock{}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.19.0, attributes={service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.25.0"}}, logLimits=LogLimits{maxNumberOfAttributes=128, maxAttributeValueLength=2147483647}, logRecordProcessor=io.opentelemetry.sdk.logs.NoopLogRecordProcessor@701fc37a}, propagators=DefaultContextPropagators{textMapPropagator=MultiTextMapPropagator{textMapPropagators=[W3CTraceContextPropagator, W3CBaggagePropagator]}}} by autoconfiguration
23:25:04.101 INFO [NodeOptions.getSessionFactories] - Detected 4 available processors
23:25:04.115 INFO [NodeOptions.discoverDrivers] - Driver(s) already present on the host: 1
23:25:04.145 INFO [NodeOptions.report] - Adding Firefox for {"browserName": "firefox","platformName": "linux"} 4 times (Host)
23:25:04.166 INFO [Node.<init>] - Binding additional locator mechanisms: relative
23:25:04.203 DEBUG [GridModel.add] - Adding node with id fd1e12ec-cf8a-43c5-8681-4b88a67c1e3e and URI http://127.0.0.1:4445
23:25:04.204 DEBUG [LocalDistributor.updateNodeAvailability] - Health check result for http://127.0.0.1:4445 was UP
23:25:04.205 INFO [GridModel.setAvailability] - Switching Node fd1e12ec-cf8a-43c5-8681-4b88a67c1e3e (uri: http://127.0.0.1:4445) from DOWN to UP
23:25:04.206 INFO [LocalDistributor.add] - Added node fd1e12ec-cf8a-43c5-8681-4b88a67c1e3e at http://127.0.0.1:4445. Health check every 120s
23:25:04.226 DEBUG [MultithreadEventLoopGroup.<clinit>] - -Dio.netty.eventLoopThreads: 8
23:25:04.234 DEBUG [GlobalEventExecutor.<clinit>] - -Dio.netty.globalEventExecutor.quietPeriodSeconds: 1
23:25:04.244 DEBUG [InternalThreadLocalMap.<clinit>] - -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024
23:25:04.244 DEBUG [InternalThreadLocalMap.<clinit>] - -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096
23:25:04.266 DEBUG [PlatformDependent0.explicitNoUnsafeCause0] - -Dio.netty.noUnsafe: false
23:25:04.267 DEBUG [PlatformDependent0.javaVersion0] - Java version: 11
23:25:04.267 DEBUG [PlatformDependent0.<clinit>] - sun.misc.Unsafe.theUnsafe: available
23:25:04.268 DEBUG [PlatformDependent0.<clinit>] - sun.misc.Unsafe.copyMemory: available
23:25:04.268 DEBUG [PlatformDependent0.<clinit>] - sun.misc.Unsafe.storeFence: available
23:25:04.268 DEBUG [PlatformDependent0.<clinit>] - java.nio.Buffer.address: available
23:25:04.268 DEBUG [PlatformDependent0.<clinit>] - direct buffer constructor: unavailable: Reflective setAccessible(true) disabled
23:25:04.269 DEBUG [PlatformDependent0.<clinit>] - java.nio.Bits.unaligned: available, true
23:25:04.270 DEBUG [PlatformDependent0.<clinit>] - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable: class io.netty.util.internal.PlatformDependent0$7 cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @c05fddc
23:25:04.271 DEBUG [PlatformDependent0.<clinit>] - java.nio.DirectByteBuffer.<init>(long, int): unavailable
23:25:04.273 DEBUG [PlatformDependent.unsafeUnavailabilityCause0] - sun.misc.Unsafe: available
23:25:04.274 DEBUG [PlatformDependent.estimateMaxDirectMemory] - maxDirectMemory: 4167041024 bytes (maybe)
23:25:04.274 DEBUG [PlatformDependent.tmpdir0] - -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
23:25:04.274 DEBUG [PlatformDependent.bitMode0] - -Dio.netty.bitMode: 64 (sun.arch.data.model)
23:25:04.275 DEBUG [PlatformDependent.<clinit>] - -Dio.netty.maxDirectMemory: -1 bytes
23:25:04.275 DEBUG [PlatformDependent.<clinit>] - -Dio.netty.uninitializedArrayAllocationThreshold: -1
23:25:04.276 DEBUG [CleanerJava9.<clinit>] - java.nio.ByteBuffer.cleaner(): available
23:25:04.276 DEBUG [PlatformDependent.<clinit>] - -Dio.netty.noPreferDirect: false
23:25:04.278 DEBUG [NioEventLoop.<clinit>] - -Dio.netty.noKeySetOptimization: false
23:25:04.278 DEBUG [NioEventLoop.<clinit>] - -Dio.netty.selectorAutoRebuildThreshold: 512
23:25:04.283 DEBUG [PlatformDependent$Mpsc.<clinit>] - org.jctools-core.MpscChunkedArrayQueue: available
23:25:04.305 DEBUG [DefaultChannelId.<clinit>] - -Dio.netty.processId: 190894 (auto-detected)
23:25:04.308 DEBUG [NetUtil.<clinit>] - -Djava.net.preferIPv4Stack: false
23:25:04.309 DEBUG [NetUtil.<clinit>] - -Djava.net.preferIPv6Addresses: false
23:25:04.310 DEBUG [NetUtilInitializations.determineLoopback] - Loopback interface: lo (lo, 0:0:0:0:0:0:0:1%lo)
23:25:04.311 DEBUG [NetUtil$SoMaxConnAction.run] - /proc/sys/net/core/somaxconn: 4096
23:25:04.312 DEBUG [DefaultChannelId.<clinit>] - -Dio.netty.machineId: 94:c6:91:ff:fe:11:df:a9 (auto-detected)
23:25:04.319 DEBUG [ResourceLeakDetector.<clinit>] - -Dio.netty.leakDetection.level: simple
23:25:04.319 DEBUG [ResourceLeakDetector.<clinit>] - -Dio.netty.leakDetection.targetRecords: 4
23:25:04.341 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.numHeapArenas: 8
23:25:04.341 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.numDirectArenas: 8
23:25:04.341 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.pageSize: 8192
23:25:04.341 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.maxOrder: 9
23:25:04.342 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.chunkSize: 4194304
23:25:04.342 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.smallCacheSize: 256
23:25:04.342 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.normalCacheSize: 64
23:25:04.342 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.maxCachedBufferCapacity: 32768
23:25:04.342 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.cacheTrimInterval: 8192
23:25:04.342 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.cacheTrimIntervalMillis: 0
23:25:04.342 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.useCacheForAllThreads: false
23:25:04.343 DEBUG [PooledByteBufAllocator.<clinit>] - -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
23:25:04.350 DEBUG [ByteBufUtil.<clinit>] - -Dio.netty.allocator.type: pooled
23:25:04.350 DEBUG [ByteBufUtil.<clinit>] - -Dio.netty.threadLocalDirectBufferSize: 0
23:25:04.350 DEBUG [ByteBufUtil.<clinit>] - -Dio.netty.maxThreadLocalCharBufferSize: 16384
23:25:04.369 DEBUG [LoggingHandler.channelRegistered] - [id: 0xb3aa8f7e] REGISTERED
23:25:04.373 DEBUG [LoggingHandler.bind] - [id: 0xb3aa8f7e] BIND: /127.0.0.1:4445
23:25:04.376 INFO [Standalone.execute] - Started Selenium Standalone 4.9.1 (revision eb2032df7f): http://127.0.0.1:4445
23:25:04.376 DEBUG [LoggingHandler.channelActive] - [id: 0xb3aa8f7e, L:/127.0.0.1:4445] ACTIVE
################################################################################
# WebDriver::Session
# set session [WebDriver::Session #auto $seleniumServer/wd/hub $caps] ;
################################################################################
23:26:05.668 DEBUG [LoggingHandler.channelRead] - [id: 0xb3aa8f7e, L:/127.0.0.1:4445] READ: [id: 0x1cb980c8, L:/127.0.0.1:4445 - R:/127.0.0.1:33345]
23:26:05.669 DEBUG [LoggingHandler.channelReadComplete] - [id: 0xb3aa8f7e, L:/127.0.0.1:4445] READ COMPLETE
23:26:05.690 DEBUG [AbstractByteBuf.<clinit>] - -Dio.netty.buffer.checkAccessible: true
23:26:05.690 DEBUG [AbstractByteBuf.<clinit>] - -Dio.netty.buffer.checkBounds: true
23:26:05.691 DEBUG [ResourceLeakDetectorFactory$DefaultResourceLeakDetectorFactory.newResourceLeakDetector] - Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@57973a03
23:26:05.735 DEBUG [ZlibCodecFactory.<clinit>] - -Dio.netty.noJdkZlibDecoder: false
23:26:05.736 DEBUG [ZlibCodecFactory.<clinit>] - -Dio.netty.noJdkZlibEncoder: false
23:26:05.747 DEBUG [Recycler.<clinit>] - -Dio.netty.recycler.maxCapacityPerThread: 4096
23:26:05.747 DEBUG [Recycler.<clinit>] - -Dio.netty.recycler.ratio: 8
23:26:05.747 DEBUG [Recycler.<clinit>] - -Dio.netty.recycler.chunkSize: 32
23:26:05.747 DEBUG [Recycler.<clinit>] - -Dio.netty.recycler.blocking: false
23:26:05.747 DEBUG [Recycler.<clinit>] - -Dio.netty.recycler.batchFastThreadLocalOnly: true
23:26:05.774 DEBUG [RequestConverter.channelRead0] - Incoming message: DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
POST /wd/hub/session HTTP/1.1
Host: 127.0.0.1:4445
User-Agent: Caius FT Framework
Connection: close
Accept: application/json;charset=utf-8
Accept-Encoding: gzip,deflate,compress
Content-Type: application/json;charset=utf-8
Content-Length: 487
23:26:05.774 DEBUG [RequestConverter.channelRead0] - Start of http request: DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
POST /wd/hub/session HTTP/1.1
Host: 127.0.0.1:4445
User-Agent: Caius FT Framework
Connection: close
Accept: application/json;charset=utf-8
Accept-Encoding: gzip,deflate,compress
Content-Type: application/json;charset=utf-8
Content-Length: 487
23:26:05.781 DEBUG [RequestConverter.channelRead0] - Incoming message: DefaultLastHttpContent(data: PooledSlicedByteBuf(ridx: 0, widx: 487, cap: 487/487, unwrapped: PooledUnsafeDirectByteBuf(ridx: 740, widx: 740, cap: 2048)), decoderResult: success)
23:26:05.785 DEBUG [RequestConverter.channelRead0] - End of http request: DefaultLastHttpContent(data: PooledSlicedByteBuf(ridx: 487, widx: 487, cap: 487/487, unwrapped: PooledUnsafeDirectByteBuf(ridx: 740, widx: 740, cap: 2048)), decoderResult: success)
23:26:05.837 DEBUG [SpanWrappedHttpHandler.execute] - Wrapping request. Before OpenTelemetryContext{tracer=io.opentelemetry.sdk.trace.SdkTracer@120d6fe6, context={}, span id=0000000000000000, trace id=00000000000000000000000000000000} and after OpenTelemetryContext{tracer=io.opentelemetry.sdk.trace.SdkTracer@120d6fe6, context={opentelemetry-trace-span-key=SdkSpan{traceId=0a4d56167357df63bfdb9deb447bebb5, spanId=60a7f44bf24dca0c, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, name=session_queue, kind=INTERNAL, attributes=AttributesMap{data={random.key=c66470e0-6cf5-4a7a-beba-2b71564e43fc}, capacity=128, totalAddedValues=1}, status=ImmutableStatusData{statusCode=UNSET, description=}, totalRecordedEvents=0, totalRecordedLinks=0, startEpochNanos=1686032765835000000, endEpochNanos=0}}, span id=60a7f44bf24dca0c, trace id=0a4d56167357df63bfdb9deb447bebb5}
23:26:05.840 DEBUG [HttpTracing.inject] - Injecting (POST) /session into OpenTelemetrySpan{traceId=0a4d56167357df63bfdb9deb447bebb5,spanId=60a7f44bf24dca0c} at org.openqa.selenium.remote.tracing.SpanWrappedHttpHandler:84
23:27:03.698 DEBUG [LocalDistributor.lambda$asRunnableHealthCheck$9] - Running healthcheck for Node http://127.0.0.1:4445
23:27:03.699 DEBUG [LocalDistributor.updateNodeAvailability] - Health check result for http://127.0.0.1:4445 was UP
23:29:03.698 DEBUG [LocalDistributor.lambda$asRunnableHealthCheck$9] - Running healthcheck for Node http://127.0.0.1:4445
23:29:03.699 DEBUG [LocalDistributor.updateNodeAvailability] - Health check result for http://127.0.0.1:4445 was UP
@asifhg
Copy link
Author

asifhg commented Jun 6, 2023

Your tutorial documentation uses:
selenium-server-standalone-2.38.0.jar
I want to try using WebDriver with selenium-server-standalone-2.38.0.jar.
But I'm unable to find this version of the jar file:
https://github.com/SeleniumHQ/selenium/tags?after=selenium-2.40.0
I will have to download the src tarball and build it, which I do not want to do.

@asifhg
Copy link
Author

asifhg commented Jun 7, 2023

I tried each of the Selenium-Server versions backwards and found that version 4.8.3 works.
The tcl instruction:
set session [WebDriver::Session #auto $seleniumServer/wd/hub $caps] ;
starts an instance of the Firefox browser.

@tobijk
Copy link
Owner

tobijk commented Jun 7, 2023

I wonder if this is simply an incompatibility between the browser and the latest selenium server. Are you using a recent version of Firefox or some LTS version maybe?

@tobijk
Copy link
Owner

tobijk commented Jun 7, 2023

Hm, no, it also doesn't work with chromedriver.

@tobijk
Copy link
Owner

tobijk commented Jun 7, 2023

Then probably some backwards-compatibility was removed. This Tcl module was written years ago, even before WebDriver became a W3C standard, so there may be things in there that simply aren't compliant with the standard.

@tobijk
Copy link
Owner

tobijk commented Jun 7, 2023

It's probably about how the session is being requested. I see things have changed. There are no desiredCapabilites and requiredCapabilities anymore. Instead there is alwaysMatch and firstMatch.

@asifhg
Copy link
Author

asifhg commented Jun 8, 2023

Firefox Version
Whatever the latest for Ubuntu 22.04 is: 113.0.2 (64-bit)

Would be useful to get the minimal tcl code to make Selenium-Webdriver start chrome instead of firefox. I tried the following to no avail:
java -jar selenium-server/selenium-server-4.8.3.jar standalone --host 127.0.0.1 --port 4445 --http-logs true --log-level INFO --service-configuration stereotype='{"browserName": "chrome"}' ;

But then the same cmd in the following for firefox fails:
java -jar selenium-server/selenium-server-4.8.3.jar standalone --host 127.0.0.1 --port 4445 --http-logs true --log-level INFO --service-configuration stereotype='{"browserName": "firefox"}' ;

I am able to do the following successfully.

  • Open the browser.
  • Make the browser navigate to https://duckduckgo.com.
  • Populate the search string filed on the webpage.
  • Submit the search.

Server

java -jar selenium-server/selenium-server-4.8.3.jar standalone --host 127.0.0.1 --port 4445 --http-logs true --log-level INFO ;

Client

/usr/bin/tclsh ;
package require WebDriver ;

set caps [ namespace which [ WebDriver::Capabilities #auto -browser_name firefox ]] ;
set session [ WebDriver::Session #auto http://127.0.0.1:4445/wd/hub $caps ] ;
# Now we have a running instance of the firefox browser.

set window [$session active_window] ;
set url "https://duckduckgo.com" ;
$window set_url $url ;
# Now the browser has opened the website.

set elements_name [ $window elements by_xpath {//*[@name]} ] ;
set e_search_field [ lindex $elements_name end-1 ] ;
set e_search_button [ lindex $elements_name end ] ;

$e_search_field click ;
$e_search_field clear ;
$e_search_field send_keys "google" ;
$e_search_button submit ;

I'm no Itcl expert, so debugging this is a challenge.
But then again, there's nothing to debug for the above simple experiment.
I'm going to keep going further with my experimentations such as loging into a legit a/c and report back and take it further.

@asifhg
Copy link
Author

asifhg commented Jun 21, 2023

I want to use Caius to be able to launch the Firefox browser with a specific user profile. I tried to look through the source code, but I'm not sure whether passing cmdline args to Firefox is supported. Any help would be much appreciated.
To illustrate, here is a simple example in Python3 Selenium.

  1. Init user profile in cmdline options passed to Firefox.
  2. Launch Firefox.
  3. Load URL: about:profiles
  4. Exit

OS Platform

lsb_release -a ;
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.2 LTS
Release:	22.04
Codename:	jammy

Installation

sudo apt-get install python3 python3-pip ;
pip3 install selenium webdriver-manager ;
pip3 install -U selenium ;
pip3 install -U webdriver-manager ;

Prequisites

# Create Firefox user profile and note the location of the user profile directory.
firefox -ProfileManager ;

web_user_profile.py

import sys
import time
import os

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys

from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.service import Service
from webdriver_manager.firefox import GeckoDriverManager

options = Options()
options.page_load_strategy = 'normal'

profile_arg_param = '-profile'
# This is usually where the Firefox user profile is created.
profile_arg_value = os.getenv('HOME') + '/snap/firefox/common/.mozilla/firefox/WHATEVER_NAME_OF_THE_USER_PROFILE'

options.add_argument(profile_arg_param)
options.add_argument(profile_arg_value)

# Start an instance of the Firefox browser.
browser = webdriver.Firefox(service=Service(GeckoDriverManager().install()), options=options)

# Load URL
url = "about:profiles"
browser.get(url)

# Sleep, close browser and exit.
time.sleep(10) ;# This should give you enough time to look at the user profile used by the browser.
browser.close()
exit()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants