LEGO boost

[ratschan]

I am trying to connect to LEGO boost from Ubuntu/Firefox/Scratch via a BLE dongle using pyscrlink. Installation (following the instructions at https://github.com/kawasaki/pyscrlink) works fine. But trying to connect from Scratch, as described in the instructions, fails, the LEGO boost extension saying "no devices found". Doing so, scratch_link -d outputs the following:

Print debug messages
2023-02-25 15:52:14,387 set scan_seconds: 10.0
2023-02-25 15:52:14,412 Certificate is ready in FireFox NSS DB: /home/ratschan/.mozilla/firefox/wx5xyc5d.default-release
2023-02-25 15:52:15,271 Certificate is ready in FireFox NSS DB: /home/ratschan/.mozilla/firefox/dj0c5tt0.default
2023-02-25 15:52:15,301 Certificate is ready for Chrome
2023-02-25 15:52:15,356 Started scratch-link
2023-02-25 15:54:57,652 Start session for web socket path: /scratch/ble
2023-02-25 15:54:57,652 start session handler
2023-02-25 15:54:57,652 start recv_request
2023-02-25 15:54:57,754 start recv_request
2023-02-25 15:54:57,754 request: {"jsonrpc":"2.0","method":"discover","params":{"filters":[{"services":["00001623-1212-efde-1623-785feabcd123"],"manufacturerData":{"919":{"dataPrefix":[0,64],"mask":[0,255]}}}],"optionalServices":[]},"id":0}
2023-02-25 15:54:57,754 handle request to BLE device
2023-02-25 15:54:57,754 discover
2023-02-25 15:54:57,754 {'filters': [{'services': ['00001623-1212-efde-1623-785feabcd123'], 'manufacturerData': {'919': {'dataPrefix': [0, 64], 'mask': [0, 255]}}}], 'optionalServices': []}
2023-02-25 15:54:57,763 start BLE scan: 10.0 seconds
2023-02-25 15:55:07,788 Failure in session for web socket path: /scratch/ble
2023-02-25 15:55:07,788 BTLEDisconnectError: Device disconnected

I also tried to follow the steps "at In Case You Fail to Connect" but could not really make sense of the instructions. Opening "https://device-manager.scratch.mit.edu:20110/" in the browser, I get "Failed to open a WebSocket connection: invalid Connection header: keep-alive. You cannot access a WebSocket server directly with a browser. You need a WebSocket client.". But the instructions say "This is expected". Now I do not understand what to conclude from this ...

[kawasaki]

@ratschan Thanks for this report. Today, I tried to dig into this, but still not sure what is happening. Your report with -d command just says the device was disconnected, but it has no clue of the disconnection cause.

I tried to recreate this issue with my environment and my micro:bit. I once succeeded to recreate very similar issue. The connection to my micro:bit failed with the message below:

$ scratch_link -d
Print debug messages
2023-03-05 17:58:24,476 set scan_seconds: 10.0
2023-03-05 17:58:24,488 Certificate is ready in FireFox NSS DB: /home/shin/.mozilla/firefox/nnysaloj.default
2023-03-05 17:58:24,597 Certificate is ready in FireFox NSS DB: /home/shin/.mozilla/firefox/nnysaloj.default/chrome_debugger_profile
2023-03-05 17:58:24,610 Certificate is ready for Chrome
2023-03-05 17:58:24,651 Started scratch-link
2023-03-05 17:58:27,272 Start session for web socket path: /scratch/ble
2023-03-05 17:58:27,272 start session handler
2023-03-05 17:58:27,272 start recv_request
2023-03-05 17:58:27,373 start recv_request
2023-03-05 17:58:27,374 request: {"jsonrpc":"2.0","method":"discover","params":{"filters":[{"services":[61445]}]},"id":0}
2023-03-05 17:58:27,374 handle request to BLE device
2023-03-05 17:58:27,374 discover
2023-03-05 17:58:27,374 {'filters': [{'services': [61445]}]}
2023-03-05 17:58:27,375 start BLE scan: 10.0 seconds
2023-03-05 17:58:37,398 Failure in session for web socket path: /scratch/ble
2023-03-05 17:58:37,398 BTLEDisconnectError: Device disconnected

When I repeat it, the failure was recreated a few times (sometimes succeeded, and sometimes failed).

As a next step, I tried to run scratch_link.py in place of the scratch_link command.

$ python ~/.local/lib/python3.10/site-packages/pyscrlink/scratch_link.py

This command did not recreate the failure. I'm not sure why, because the scratch_link command is just running the scrach_link.py. After that, I tried to recreate the failure with the scratch_link command, I can no longer recreate the failure...

This is sad, since if the failure would be recreated on my system, it was easier for me to chase it down. And I'm not sure why the failure disappeared.

Today, I stop here. If you have time, please try to run the scratch_link.py instead of scratch_link command and see if the failure is recreated or not.

[ratschan]

Thanks a lot for taking the time to look into this and for your suggestion. Unfortunately, the way of starting scratch_link does not make a difference. I will try to read and understand your code in detail, maybe this will allow me to find the problem. Anyway, would you maybe have an output of scratch_link -d available that shows how it would look like if everything works correctly? This would help me to understand what goes wrong ...

[kawasaki]

FYI, here's the scratch_link -d outputs with my system and my micro:bit. If you see the word "match..." the scratch_link finds out the devices to list in Scratch dialog.

2023-03-08 15:27:46,008 set scan_seconds: 10.0
2023-03-08 15:27:46,021 Certificate is ready in FireFox NSS DB: /home/shin/.mozilla/firefox/nnysaloj.default
2023-03-08 15:27:46,127 Certificate is ready in FireFox NSS DB: /home/shin/.mozilla/firefox/nnysaloj.default/chrome_debugger_profile
2023-03-08 15:27:46,140 Certificate is ready for Chrome
2023-03-08 15:27:46,173 Started scratch-link
2023-03-08 15:27:56,430 Start session for web socket path: /scratch/ble
2023-03-08 15:27:56,430 start session handler
2023-03-08 15:27:56,430 start recv_request
2023-03-08 15:27:56,532 start recv_request
2023-03-08 15:27:56,532 request: {"jsonrpc":"2.0","method":"discover","params":{"filters":[{"services":[61445]}]},"id":0}
2023-03-08 15:27:56,532 handle request to BLE device
2023-03-08 15:27:56,532 discover
2023-03-08 15:27:56,532 {'filters': [{'services': [61445]}]}
2023-03-08 15:27:56,533 start BLE scan: 10.0 seconds
2023-03-08 15:28:06,550 in matches db:1e:33:07:5a:da [{'services': [61445]}]
2023-03-08 15:28:06,550 service to check: 61445
2023-03-08 15:28:06,550 given UUID: 61445 hash=478346013097353740
2023-03-08 15:28:06,550 service class uuid for adtype complete 16b/3: 0000f005-0000-1000-8000-00805f9b34fb
2023-03-08 15:28:06,550 dev UUID: 0000f005-0000-1000-8000-00805f9b34fb hash=478346013097353740
2023-03-08 15:28:06,550 True
2023-03-08 15:28:06,550 match...
2023-03-08 15:28:06,550 BLE device found with iface #0
2023-03-08 15:28:06,550 in matches 75:0c:d0:21:a2:0b [{'services': [61445]}]
2023-03-08 15:28:06,550 service to check: 61445
2023-03-08 15:28:06,550 given UUID: 61445 hash=478346013097353740
2023-03-08 15:28:06,550 service class uuid for adtype complete 16b/3: 0000fe9f-0000-1000-8000-00805f9b34fb
2023-03-08 15:28:06,550 dev UUID: 0000fe9f-0000-1000-8000-00805f9b34fb hash=4341693373872439595
2023-03-08 15:28:06,550 False
2023-03-08 15:28:06,550 in matches 4d:e4:30:bd:24:84 [{'services': [61445]}]
2023-03-08 15:28:06,550 service to check: 61445
2023-03-08 15:28:06,550 given UUID: 61445 hash=478346013097353740
2023-03-08 15:28:06,550 service class uuid for adtype complete 16b/3: 0000fe9f-0000-1000-8000-00805f9b34fb
2023-03-08 15:28:06,550 dev UUID: 0000fe9f-0000-1000-8000-00805f9b34fb hash=4341693373872439595
2023-03-08 15:28:06,551 False
2023-03-08 15:28:06,551 in matches 4c:5a:b1:97:d3:a4 [{'services': [61445]}]
2023-03-08 15:28:06,551 service to check: 61445
2023-03-08 15:28:06,551 given UUID: 61445 hash=478346013097353740
2023-03-08 15:28:06,551 service class uuid for adtype complete 16b/3: 0000fe9f-0000-1000-8000-00805f9b34fb
2023-03-08 15:28:06,551 dev UUID: 0000fe9f-0000-1000-8000-00805f9b34fb hash=4341693373872439595
2023-03-08 15:28:06,551 False
2023-03-08 15:28:06,551 in matches 43:33:f5:a9:d3:44 [{'services': [61445]}]
2023-03-08 15:28:06,551 service to check: 61445
2023-03-08 15:28:06,551 given UUID: 61445 hash=478346013097353740
2023-03-08 15:28:06,551 in matches c8:07:9b:d6:4e:95 [{'services': [61445]}]
2023-03-08 15:28:06,551 service to check: 61445
2023-03-08 15:28:06,551 given UUID: 61445 hash=478346013097353740
2023-03-08 15:28:06,551 start BLE scan: 10.0 seconds
2023-03-08 15:28:06,554 BLE iface #1: Failed to execute management command 'le on' (code: 17, error: Invalid Index)
2023-03-08 15:28:06,554 start BLE scan: 10.0 seconds
2023-03-08 15:28:06,559 BLE iface #2: Failed to execute management command 'le on' (code: 17, error: Invalid Index)
2023-03-08 15:28:06,560 loop in BLE thread
2023-03-08 15:28:06,560 {'jsonrpc': '2.0', 'result': None}
2023-03-08 15:28:06,560 response: {"jsonrpc": "2.0", "result": null, "id": 0}
2023-03-08 15:28:06,561 end_request of BLESession
2023-03-08 15:28:06,561 start to notify
2023-03-08 15:28:06,561 in handle loop
2023-03-08 15:28:06,561 start recv_request
2023-03-08 15:28:06,561 start to notify
2023-03-08 15:28:06,561 in handle loop
2023-03-08 15:28:06,561 start recv_request
2023-03-08 15:28:06,560 send out found devices
2023-03-08 15:28:06,563 start to notify
2023-03-08 15:28:06,563 notification: {"jsonrpc": "2.0", "method": "didDiscoverPeripheral", "params": {"rssi": -60, "peripheralId": 0, "name": "BBC micro:bit [zogev]"}}
2023-03-08 15:28:06,563 in handle loop
2023-03-08 15:28:06,563 start recv_request
...

My current guess is that the failure happened in _scan_devices(), at the line to call scanner.scan(). This scan() is bluepy API. So I guess we need to dig into bluepy layer, and which is a rather hard work for me.

[ratschan]

Thanks a lot! I works now by simply executing "hciconfig hci0 down" before starting scratch_link.

Btw., on another machine with a fresh Ubuntu 22.04 installation I had to do a change to gencert.py to make it work. The reason is that firefix directory is not ~/.mozilla any more, but ~/snap/firefox/common/.mozilla.

Anyway, thanks a lot for your work! It is really great to have this available!

[kawasaki]

@ratschan Thanks for the update. Though I still wonder why"hciconfig hci0 down" does the trick, it's great to hear that now it works for you. Also, thanks for your finding bout the gencert.py. I guess it explains the issue #35.

[kawasaki]

As for the NSS DB path difference of the Snap package, I created a fix commit at "snap" branch. Will do some more evaluation before merging to the master branch.

[kawasaki]

An update on my follow-up work: I've merged the fix for Snap packages.

Today, I spent some more time using my old laptop with Ubuntu 22.04 LTS. It recreates the BTLEDisconnectError in stable manner. I still do not understand the cause, but I confirmed that the error happens at scanner.scan(). I also found that if the scanner.scan() calls is repeated, the error can be worked around.

As a temporary work around solution, I implemented "-r" option of scratch_link. With "-r 2" option, I see the error can be avoided. Using -s command together, "scratch_link -r 2 -s 3" command works stable in my environments.

With these changes, I released pyscrlink version 0.2.7, even though I'm not sure the -r option help you or not.

[shmoun]

I can confirm that

scratch_link -r 2 -s 3

makes the difference in connecting successfully to the LEGO BOOST hub.

Thank you for your work!