The Fetchbot interface allows users to connect to and control a Raspberry Pi based robot "Fetchbot", without internet connection. The connection between a computer and the robot is established through Wifi.
Please refer to the last section for interface description & editing.
This project has been tested on a Windows 10 64-bit machine and a Raspberry Pi 4.
Presentation & installation video:
-
Install Anaconda Python (Installer in the Google drive: https://drive.google.com/drive/folders/1O0ZvvSjgypUunFu7uR9pzGn0pdYZAEDJ?usp=share_link)
-
Navigate to https://github.com/lascientotheque/fetchbot-blockly and click on code>Download ZIP. Extract the files once the download is complete.
In the command prompt, enter the following commands:
cd "path of the extracted fetchbot repository"
pip install -r requirements.txt
On your Windows PC, go to Wifi networks and connect to the raspberry_pi_xx access point (where 'xx' is the ID of the Raspberry Pi).
The password is not the PIN code, but the security key. Click on 'Se connecter à l'aide d'une clé de sécurité', and use default password. If asked for allowing discovery of the computer, select 'Oui'.
The IP address of the Raspberry Pi is 10.42.0.1. You may use VNC viewer to connect to it (loging: pi, password: raspberry).
In your browser, go to 10.42.0.1:2204
- If the camera image is not displayed, try reloading the page with 'CTRL+R'
- If the motors do not work, check that the battery for the motos is 'on'.
The program can be executed from the command prompt with:
cd "path of the extracted fetchbot repository"
python main.py
The program starts automatically on startup (with 'cron').
Make sure that the Wifi connection is active between the Raspberry Pi and the Windows PC before launching the program on the Windows PC.
The index.html file holds 3 iframes:
- Blockly Interface (JS)
- Video & test box (Python Flask)
- AI Classifier (Python Flask)
Code for the Blockly Interface are found in blockly/demos/code/.
The html files for the video & text box and the AI classifier are found in templates/.
Main.py or master program runs a flask server which serves the video & test box and the AI classifier. When the "start" button is pressed on the blockly interface, a POST request with the python translated code is sent to the master program. The master program creates a new temp.py python file based on this translated code, which is then executed as subprocess.
The temp.py file uses the fetchbot.py library in src/ to send mouvement and text commands by POST requests to the master program, the latter then sends those commands to the robot via serial-over-bluetooth (as only one serial connection can exist at a time).
The image feed is sent from the robot to the master program via the serial-over-bluetooth connection.
The AI classifier allows the user to take image stills from the video feed and save them to a created class. The created classes and images are found in classes.
A new block can be created using the Blockly block factory tool in blockly/demos/blockfactory/index.html.
New blocks are created by defining their inputs, fields, types and colours using the drag and drop interface on the left.
The Block Definition specifies what the block looks like and the Generator stub specifies what the code does (the python line of code associated to it).
Once a new block is created, copy the Block Definition (select JavaScript) and the Generator stub (select Python), and paste it at the end of blockly/demos/code/fetchbot.js.
Blockly.Blocks['print_hello'] = {
init: function() {
this.appendDummyInput()
.appendField("this block prints hello");
this.setPreviousStatement(true, null);
this.setNextStatement(true, null);
this.setColour(230);
this.setTooltip("");
this.setHelpUrl("");
}
};
Blockly.Python['print_hello'] = function(block) {
// TODO: Assemble Python into code variable.
var code = 'print("hello")\n';
return code;
};
To display this block on the blockly interface, add the following line between the <category name="Fetchbot" colour="#a83236"></category> tags in blockly/demos/code/index.html:
<block type="print_hello"></block>