Help to create a new braille device

[discapacidad5]

Hi, I don't speak English, I'm using the google translator, I hope they understand me, my language is Spanish

I am developing a braille line with the ardiuno IDE and esp32 I have all the hardware but my problem is precisely that I do not fear how to communicate with the esp32
I have tried to connect the serial monitor of ardiuno and connect the esp32 by bluetooth and in the output of the serial port I get pure question marks and unreadable characters
I would like help to develop the software that would go in the esp32 that exchange data with the devices

my project consists of a very low cost braille display probably anyone can manufacture with less than $ 50 the idea is to create everything and make it available so that anyone can print their own braille display on a 3d printer and added the esp32 card and the driver can make your own braille line I hope someone can help me

[Adriani90]

cc: @FelixGruetzmacher

[seanbudd]

Thanks for reaching out. This is an exciting project, which we hope is successful.

Once your device has been proven to display braille correctly, e.g. from an internal source, we can help ensure that NVDA supports it with a braille driver, such as braille HID.

We have notes on using braille HID with NVDA, which are not translated. The NVDA Spanish community might be of help here - https://nvda.es/.

Let us know where you're currently at with the project, and what you are stuck with. Sharing a public GitHub repository may allow others to review your designs and software.

[EngineeringMario]

Very good initiative. It might be an idea to find involvement from people that can draw 3d housing designs for producing a housing with 3d printer.
I'm curious about the cells that will be used.

[discapacidad5]

Como no hablo inglés y no hay nadie aquí que me responda en español, es difícil para mí explicar los conceptos, así que en realidad le estaba pidiendo a alguien que habla español que abra la publicación en español primero para que sirva de intérprete.

As I do not speak English and there is no one here to answer me in Spanish, it is difficult for me to explain the concepts, so I was actually asking someone who speaks Spanish to open the publication in Spanish first to serve as an interpreter.

Pero en vista de que no ha sido posible tengo que terminar un primer prototipo para grabar un video y mostrar en video cómo funciona creo que será la única forma de hacerle entender el concepto de mi diseño braile a falta de un intérprete

But since it has not been possible, I have to finish a first prototype to record a video and show on video how it works, I think it will be the only way to make him understand the concept of my braille design in the absence of an interpreter

[seanbudd]

To those interested in working on this, now that NVDA supports Braille HID, we won't be accepting any new 'legacy braille drivers' into the codebase. However the 'legacy approach' may be helpful to aid development of the display, such as creating a braille display driver add-on.

[seanbudd]

@discapacidad5 - we've put out a request for anyone in the community who can help with development. The conversation is here.

[discapacidad5]

Thank you very much right now I get in touch I hope to get more people who can support El jue, 25 nov 2021 a las 2:55, Sean Budd ***@***.***>) escribió:

…

@discapacidad5 <https://github.com/discapacidad5> - we've put out a request for anyone in the community who can help with development. The conversation is here <https://groups.io/g/nvda-devel/topic/87275635#45915> . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13070 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACL5OJSHE2UFJGSJBZNAYULUNXMXFANCNFSM5IKSHLDQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[discapacidad5]

I share my repository is the following
https://github.com/brailletouch/Brailletouch

I think what I need is HID support for esp32, especially HID braille, they could help with that and I have seen they make keyboards with ESP32 and with Arduino and I guess HID braille is something similar but I don't know how to do it

[discapacidad5]

This is a person who gave me this answer but the truth is I don't understand how to do that, what does he explain to me there Dear @discapacidad5 , very interesting approach with one real braille cell and the virtual cells! Regarding the Bluetooth HID implementation: unfortunately, we don't need a Braille implementation in this repository (maybe my colleagues will find a use case, but I can't). Nevertheless, if you create a new repository for the ESP32 firmware (I would recommend to split the firmware & case designs) I can contribute. Thanks for the USB reference document, I think there is everything you need: To change the HID functionality you need following steps (mostly in the file hid_device_le_prf.c): Change the HID descriptor: I would simply use the example from the referenced document. It seems to include left/right control buttons, braille input and some buttons (look for collections which end with an input tag) Change the report structure: On the first read, there seems to be only one input and one output report (no report IDs). Combining all report size/report count tags in the HID descriptor you will get the bit/byte positions in your report where data should be sent / read. Hope I could help, I like your projects! Greetings OK, I would get started with the Espressif example: https://github.com/espressif/esp-idf/tree/master/examples/bluetooth/bluedroid/ble/ble_hid_device_demo There is everything you need to get started based on a mouse/keyboard project (exactly as we did here). Try to compile and flash the example, to ensure a correct development setup. (getting started for the ESP32: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/index.html ) And then start to change the HID descriptor to the example from your document. El jue., 25 de noviembre de 2021 7:47 a. m., discapacidad5 < ***@***.***> escribió:

…

Thank you very much right now I get in touch I hope to get more people who can support El jue, 25 nov 2021 a las 2:55, Sean Budd ***@***.***>) escribió: > @discapacidad5 <https://github.com/discapacidad5> - we've put out a > request for anyone in the community who can help with development. The > conversation is here > <https://groups.io/g/nvda-devel/topic/87275635#45915>. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#13070 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ACL5OJSHE2UFJGSJBZNAYULUNXMXFANCNFSM5IKSHLDQ> > . > Triage notifications on the go with GitHub Mobile for iOS > <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> > or Android > <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. > >

[discapacidad5]

I found this library for esp32 on arduino https://github.com/T-vK/ESP32-BLE-Keyboard/blob/master/BleKeyboard.cpp
And this other https://github.com/T-vK/ESP32-BLE-Mouse/blob/master/BleMouse.cpp I think that modifying it could adapt to the HID braille it is in charge of automatically making the description table for the controller you just have to load the list of the hid braille then one in the code adds the keys or functions that you need to use

the libraries that are called to do all the work are "BLEHIDDevice" y "HIDTypes.h" en https://github.com/espressif/arduino-esp32/blob/master/libraries/BLE/src/

// BleKeyboard.cpp

#include "BleKeyboard.h"     // modify by Blebraille.h

static const uint8_t _hidReportDescriptor[] = {
  USAGE_PAGE(1),      0x01,   // USAGE_PAGE (Generic Desktop Ctrls)  // modify by HID braille   USAGE_PAGE(1),      0x41, 
  USAGE(1),           0x06,          // USAGE (Keyboard)  // modify by   USAGE(1),           0x01,  Braille Display
  COLLECTION(1),      0x01,          // COLLECTION (Application)
  
};
 ..............

and

// BleMouse.cpp

static const uint8_t _hidReportDescriptor[] = {
  USAGE_PAGE(1),       0x01, // USAGE_PAGE (Generic Desktop)   //  modify by HID braille     USAGE_PAGE(1),      0x41,
  USAGE(1),            0x02, // USAGE (Mouse)  // modify by  el HID  USAGE(1),           0x01,  Braille Display
  COLLECTION(1),       0x01, // COLLECTION (Application)
  USAGE(1),            0x01, //   USAGE (Pointer)
  COLLECTION(1),       0x00, //   COLLECTION (Physical)
  // ------------------------------------------------- Buttons (Left, Right, Middle, Back, Forward)
  // USAGE_PAGE(1),       0x09, //     USAGE_PAGE (Button)
  // USAGE_MINIMUM(1),    0x01, //     USAGE_MINIMUM (Button 1)
  // USAGE_MAXIMUM(1),    0x05, //     USAGE_MAXIMUM (Button 5)
  // LOGICAL_MINIMUM(1),  0x00, //     LOGICAL_MINIMUM (0)
  // LOGICAL_MAXIMUM(1),  0x01, //     LOGICAL_MAXIMUM (1)
  // REPORT_SIZE(1),      0x01, //     REPORT_SIZE (1)
  // REPORT_COUNT(1),     0x05, //     REPORT_COUNT (5)
  // HIDINPUT(1),         0x02, //     INPUT (Data, Variable, Absolute) ;5 button bits
  // ------------------------------------------------- Padding
  // REPORT_SIZE(1),      0x03, //     REPORT_SIZE (3)
  // REPORT_COUNT(1),     0x01, //     REPORT_COUNT (1)
  // HIDINPUT(1),         0x03, //     INPUT (Constant, Variable, Absolute) ;3 bit padding
  // ------------------------------------------------- X/Y position, Wheel
  // USAGE_PAGE(1),       0x01, //     USAGE_PAGE (Generic Desktop)
  // USAGE(1),            0x30, //     USAGE (X)
  // USAGE(1),            0x31, //     USAGE (Y)
 // USAGE(1),            0x38, //     USAGE (Wheel)
 // LOGICAL_MINIMUM(1),  0x81, //     LOGICAL_MINIMUM (-127)
  // LOGICAL_MAXIMUM(1),  0x7f, //     LOGICAL_MAXIMUM (127)
 // REPORT_SIZE(1),      0x08, //     REPORT_SIZE (8)
 // REPORT_COUNT(1),     0x03, //     REPORT_COUNT (3)
 // HIDINPUT(1),         0x06, //     INPUT (Data, Variable, Relative) ;3 bytes (X,Y,Wheel)
  // ------------------------------------------------- Horizontal wheel
  // USAGE_PAGE(1),       0x0c, //     USAGE PAGE (Consumer Devices)
 //  USAGE(2),      0x38, 0x02, //     USAGE (AC Pan)
  // LOGICAL_MINIMUM(1),  0x81, //     LOGICAL_MINIMUM (-127)
  // LOGICAL_MAXIMUM(1),  0x7f, //     LOGICAL_MAXIMUM (127)
  // REPORT_SIZE(1),      0x08, //     REPORT_SIZE (8)
  // REPORT_COUNT(1),     0x01, //     REPORT_COUNT (1)
  // HIDINPUT(1),         0x06, //     INPUT (Data, Var, Rel)
  // END_COLLECTION(0),         //   END_COLLECTION
  // END_COLLECTION(0)          // END_COLLECTION
// };
..................

Those examples could be replaced by the HID braille example

//	Braille Display Page (0x41)

// Sample Report Descriptor ‐ Braille Display
‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
    0x05, 0x41,        // Usage Page (Braille)
    0x09, 0x01,        // USAGE (Braille Display)
    0xA1, 0x01,        // Collection (Application)
    0x1A, 0x01, 0x02,  //   Usage Minimum (Braille Keyboard Dot 1)
    0x2A, 0x08, 0x02,  //   Usage Maximum (Braille Keyboard Dot 8)
    0x75, 0x01,        //   Report Size (1)
    0x95, 0x08,        //   Report Count (8)
    0x15, 0x00,        //   Logical Minimum (0)
    0x25, 0x01,        //   Logical Maximum (1)
    0x81, 0x02,        //   Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0x05, 0x41,        //   Usage Page (Braille)
    0x0A, 0x0A, 0x02,  //   Usage (Braille Keyboard Left Space)
    0x0A, 0x0B, 0x02,  //   Usage (Braille Keyboard Right Space)
    0x0A, 0x10, 0x02,  //   Usage (Braille Joystick Center)
    0x0A, 0x11, 0x02,  //   Usage (Braille Joystick Up)
    0x0A, 0x12, 0x02,  //   Usage (Braille Joystick Down)
    0x0A, 0x13, 0x02,  //   Usage (Braille Joystick Left)
    0x0A, 0x14, 0x02,  //   Usage (Braille Joystick Right)
    0x75, 0x01,        //   Report Size (1)
    0x95, 0x07,        //   Report Count (7)
    0x15, 0x00,        //   Logical Minimum (0)
    0x25, 0x01,        //   Logical Maximum (1)
    0x81, 0x02,        //   Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0x75, 0x01,        //   Report Size (1)
    0x95, 0x01,        //   Report Count (1)
    0x81, 0x03,        //   Input (Const,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0x0A, 0x0D, 0x02,  //   Usage (Braille Left Controls)
    0xA1, 0x02,        //   Collection (Logical)
    0x05, 0x09,        //     Usage Page (Button)
    0x19, 0x01,        //     Usage Minimum (Button 1)
    0x29, 0x03,        //     Usage Maximum (Button 3)
    0x75, 0x01,        //     Report Size (1)
    0x95, 0x03,        //     Report Count (3)
    0x15, 0x00,        //     Logical Minimum (0)
    0x25, 0x01,        //     Logical Maximum (1)
    0x81, 0x02,        //     Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0xC0,              //   End Collection
    0x05, 0x41,        //   Usage Page (Braille)
    0x0A, 0x0E, 0x02,  //   Usage (Braille Right Controls)
    0xA1, 0x02,        //   Collection (Logical)
    0x05, 0x09,        //     Usage Page (Button)
    0x19, 0x01,        //     Usage Minimum (Button 1)
    0x29, 0x03,        //     Usage Maximum (Button 3)
    0x75, 0x01,        //     Report Size (1)
    0x95, 0x03,        //     Report Count (3)
    0x15, 0x00,        //     Logical Minimum (0)
    0x25, 0x01,        //     Logical Maximum (1)
    0x81, 0x02,        //     Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0xC0,              //   End Collection
    0x75, 0x02,        //     Report Size (2)
    0x95, 0x01,        //     Report Count (1)
    0x81, 0x03,        //     Input (Const,Var,Abs,No Wrap,Linear,Preferred
State,No Null Position) //2 bit pad
    0x05, 0x41,        //   Usage Page (Braille)
    0x0A, 0x0C, 0x02,  //   Usage (Braille Face Controls)
    0xA1, 0x02,        //   Collection (Logical)
    0x05, 0x09,        //     Usage Page (Button)
    0x19, 0x01,        //     Usage Minimum (Button 1)
    0x29, 0x03,        //     Usage Maximum (Button 4)
    0x75, 0x01,        //     Report Size (1)
    0x95, 0x04,        //     Report Count (4)
    0x15, 0x00,        //     Logical Minimum (0)
    0x25, 0x01,        //     Logical Maximum (1)
    0x81, 0x02,        //     Input (Data,Var,Abs,No Wrap,Linear,Preferred State,No
Null Position)
    0x75, 0x04,        //     Report Size (4)
    0x95, 0x01,        //     Report Count (1)
    0x81, 0x03,        //     Input (Const,Var,Abs,No Wrap,Linear,Preferred
State,No Null Position)
    0xC0,              //   End Collection
    
    0x05, 0x41,        //   Usage Page (Braille)
    0x09, 0x02,        //   USAGE (Braille Row)
    0xA1, 0x02,        //   Collection (Logical)
    0x09, 0x02,        //     Usage (8 Dot Braille Cell)
    0x15, 0x00,        //     Logical Minimum (0)
    0x26, 0xFF, 0x00,  //     Logical Maximum (255)
    0x75, 0x08,        //     Report Size (8)
    0x95, 0x14,        //     Report Count (20)
    0x91, 0x03,        //     Output (Const,Var,Abs,No Wrap,Linear,Preferred
State,No Null Position,Non‐volatile)
    
    0x09, 0xFA,        //     USAGE (Router Set 1)
    0xA1, 0x02,        //     Collection (Logical)
    0x0A, 0x00, 0x01,  //       Usage (Router Key)
    0x15, 0x00,        //       Logical Minimum (0)
    0x25, 0x01,        //       Logical Maximum (1)
    0x75, 0x01,        //       Report Size (1)
    0x95, 0x14,        //       Report Count (20)
    0x81, 0x02,        //       Input (Data,Var,Abs,No Wrap,Linear,Preferred
State,No Null Position)
    0x75, 0x04,        //       Report Size (4)
    0x95, 0x01,        //       Report Count (1)
    0x81, 0x03,        //       Input (Const,Var,Abs,No Wrap,Linear,Preferred
State,No Null Position) //4‐bit pad
    0xC0,              //     End Collection
    0xC0,              //   End Collection
    0xC0,              // End Collection

It is only to order it as a library so that later in the code you can call the library and pass the parameters that are needed such as the number of braille cells, buttons, etc.

something like that

#include < Blebraille..h>

 Blebraille  Blebraille(40); // para definir una pantalla de 40 caracteres

void setup() {
  Serial.begin(115200);
  Serial.println("Starting BLE work!");
  Blebraille.begin();
}

void loop() {
  if(Blebraille.isConnected()) {
    Serial.println("Sending 'Hello world'...");
Blebraille.print("Hello world");
if (Blebraille.available()) { // Si hay dato en el enviado por el lector de pantalla
             dato = Blebraille.read(); // Lo lee HID ble
              }


    delay(1000);

    Serial.println("Sending Enter key...");
    Blebraille.write(BRAILLE_KEYBOARD_DOT_1);

    delay(1000);

   
  }

  Serial.println("Waiting 5 seconds...");
  delay(5000);
}

	

This is what I have been able to understand so far

I understand that with 0x05 I must pass how many braille cells my screen has but I don't know how to do that

according to the hid code nvda source brailleDisplayDrivers hid.py is defined like this

class BraillePageUsageID(enum.IntEnum):
	UNDEFINED = 0
	BRAILLE_DISPLAY = 0x1
	BRAILLE_ROW = 0x2
	EIGHT_DOT_BRAILLE_CELL = 0x3
	SIX_DOT_BRAILLE_CELL = 0x4
	NUMBER_OF_BRAILLE_CELLS = 0x5
	SCREEN_READER_CONTROL = 0x6
	SCREEN_READER_IDENTIFIER = 0x7
	ROUTER_SET_1 = 0xFA
	ROUTER_SET_2 = 0xFB
	ROUTER_SET_3 = 0xFC
	ROUTER_KEY = 0x100
	ROW_ROUTER_KEY = 0x101
	BRAILLE_BUTTONS = 0x200
	BRAILLE_KEYBOARD_DOT_1 = 0x201
	BRAILLE_KEYBOARD_DOT_2 = 0x202
	BRAILLE_KEYBOARD_DOT_3 = 0x203
	BRAILLE_KEYBOARD_DOT_4 = 0x204
	BRAILLE_KEYBOARD_DOT_5 = 0x205
	BRAILLE_KEYBOARD_DOT_6 = 0x206
	BRAILLE_KEYBOARD_DOT_7 = 0x207
	BRAILLE_KEYBOARD_DOT_8 = 0x208
	BRAILLE_KEYBOARD_SPACE = 0x209
	BRAILLE_KEYBOARD_LEFT_SPACE = 0x20A
	BRAILLE_KEYBOARD_RIGHT_SPACE = 0x20B
	BRAILLE_FACE_CONTROLS = 0x20C
	BRAILLE_LEFT_CONTROLS = 0x20D
	BRAILLE_RIGHT_CONTROLS = 0x20E
	BRAILLE_TOP_CONTROLS = 0x20F
	BRAILLE_JOYSTICK_CENTER = 0x210
	BRAILLE_JOYSTICK_UP = 0x211
	BRAILLE_JOYSTICK_DOWN = 0x212
	BRAILLE_JOYSTICK_LEFT = 0x213
	BRAILLE_JOYSTICK_RIGHT = 0x214
	BRAILLE_DPAD_CENTER = 0x215
	BRAILLE_DPAD_UP = 0x216
	BRAILLE_DPAD_DOWN = 0x217
	BRAILLE_DPAD_LEFT = 0x218
	BRAILLE_DPAD_RIGHT = 0x219
	BRAILLE_PAN_LEFT = 0x21A
	BRAILLE_PAN_RIGHT = 0x21B
	BRAILLE_ROCKER_UP = 0x21C
	BRAILLE_ROCKER_DOWN = 0x21D
	BRAILLE_ROCKER_PRESS = 0x21E

I don't know if I'm on the right track or I'm totally lost, but if someone understands and knows how I can continue this structure, it would be good to help.

[Limiter1980]

Hi everyone,

I try to run Braille HID via Bluetooth with the Silabs BT121/BT122.
I can connect to VoiceOver, but NVDA under Windows won't find any standard HID device.
Windows tells me that no driver is avalaible.
May somebody can help me here with this problem? Or should I start a new issue?

Thanks for your help and best regards
Limiter

[discapacidad5]

Hola a todos,

Intento ejecutar Braille HID a través de Bluetooth con Silabs BT121/BT122. Puedo conectarme a VoiceOver, pero NVDA bajo Windows no encontrará ningún dispositivo HID estándar. Windows me dice que no hay ningún controlador disponible. ¿Alguien puede ayudarme aquí con este problema? ¿O debería empezar un nuevo número?

Gracias por tu ayuda y un saludo Limiter

Could I see your code? Could you share it so I can take a look?

[ruifontes]

Sorry, but you have to face the problem in, at least, two steps: 1 - HID protocol is a piece of software to allow the communication between Windows and a device. It allows Windows to install automatically the device without needing to install a proprietary driver. This part is accomplished wit your device? 2 - Braille HID is an extension of HID protocol and I do not know how to implement it... But making a search in the net should gave you something... 3 - Having the first part accomplished you can make a NVDA driver instead using Braille HID protocol... Rui Fontes 2 - Às 06:07 de 16/03/2023, discapacidad5 escreveu:

…

Hola a todos, Intento ejecutar Braille HID a través de Bluetooth con Silabs BT121/BT122. Puedo conectarme a VoiceOver, pero NVDA bajo Windows no encontrará ningún dispositivo HID estándar. Windows me dice que no hay ningún controlador disponible. ¿Alguien puede ayudarme aquí con este problema? ¿O debería empezar un nuevo número? Gracias por tu ayuda y un saludo Limiter Could I see your code? Could you share it so I can take a look? — Reply to this email directly, view it on GitHub <#13070 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADZAPRUPA53CVHPLPKQZFZ3W4KU2TANCNFSM5IKSHLDQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Limiter1980]

Hola discapacidad,

I used the bt121 classic hid sample as a starting point.
Attached you will find the 6 files for the project.
I had to rename the .xml files to .txt to upload them.

Utilicé la muestra bt121 classic hid como punto de partida.
Adjunto encontrará los 6 archivos para el proyecto.
Tuve que cambiar el nombre de los archivos .xml a .txt para cargarlos.

hid_Braille.txt
hid_Braille.xml.txt
project.xml.txt
did.xml.txt
gatt_db.xml.txt
hardware.xml.txt

[Limiter1980]

Hi Rui,

I am aware of the basics of HID. It works via USB with NVDA as a standard HID device.
And it works via bluetooth with VoiceOver, so it can't be completely wrong.
But by using it via Bluetooth I get a driver error and NVDA doesn't accept it as standard HID device.

Best regards
Limiter

[ruifontes]

As your Braille display is not yet included in the automatic Braille display detection, you will need to make an Braille driver add-on to include it in that function.

See in NVDA sources the bdDetect.py file...