Volume command is doing nothing in esp_hid_device on esp-idf V4.4 (IDFGH-11233) #12397

josh-gc · 2023-10-12T18:25:52Z

Answers checklist.

I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v4.4.5-249-gcc71308e2f (also tried on v4.4.6-35-gcdd72ac1d4)

Espressif SoC revision.

ESP32-PICO-V3-02 (revision v3.0)

Operating System used.

macOS

How did you build your project?

VS Code IDE

If you are using Windows, please specify command line type.

None

Development Kit.

Custom PCB

Power Supply used.

External 3.3V

What is the expected behavior?

When the code sends the volume up and down commands this would be reflected on the connected device.

What is the actual behavior?

Nothing... The volume was not being adjusted by the esp32.

Steps to reproduce.

Flash the ESP_HID_DEVICE example from the IDF to an ESP32 board
Connect to iPhone*
Code should connect fine
Console will show "HID_DEV_DEMO: Send the volume"
No volume adjustment happens on the phone

If using a Pixel7 it will connect using BT Classic which has a different demo code (which worked). However when I modified the code to send the volume commands it did not work. Also connected to a MacBook Pro and that also did not change the volume

Debug Logs.

No response

More Information.

No response

josh-gc · 2023-10-12T20:55:30Z

Update:
Sorry if this is not the best way to document but I made some major changes to the code so the volume up/down was from user input and not the timed xTask. This now works on the iPhone13 (on ble) but still nothing else.

Also, I am getting an error on the esp_hidd_dev_input_set command when connected as a BT Classic device (Pixel 7).

Code:

if(hid_connection_status == HID_IS_BLE)
{
	esp_hidd_dev_input_set(s_ble_hid_param.hid_dev, 1, HID_RPT_ID_CC_IN, buffer, HID_CC_IN_RPT_LEN);
}
else if(hid_connection_status == HID_IS_BT)
{
	esp_hidd_dev_input_set(s_bt_hid_param.hid_dev, 1, HID_RPT_ID_CC_IN, buffer, HID_CC_IN_RPT_LEN);
}
else
{
	ESP_LOGE(TAG, "HID not connected!");
}

Error:

I (0) cpu_start: Starting scheduler on APP CPU.
I (566) app_main: app_main
I (606) esp_hid_gap_init: esp_hid_gap_init
I (606) init_low_level: init_low_level
I (606) BTDM_INIT: BT controller compile version [946b762]
I (606) system_api: Base MAC address is not set
I (616) system_api: read default base MAC address from EFUSE
I (616) BTDM_INIT: Bluetooth MAC: 4c:75:25:f1:94:4a
I (626) phy_init: phy_version 4670,719f9f6,Feb 18 2021,17:07:07
I (1346) init_bt_gap: init_bt_gap
I (1356) init_ble_gap: init_ble_gap
I (1356) esp_hid_ble_gap_adv_init: esp_hid_ble_gap_adv_init
W (1366) BT_BTM: BTM_BleWriteAdvData, Partial data write into ADV
I (1386) ble_hidd_event_callback: Setting to connectable, discoverable
I (2396) bt_hidd_event_callback: Setting to connectable, discoverable
I (2396) esp_hid_device_main: run_example
##############################
ESP HID DEVICE DEMO COMMANDS:
You can input these values
s -- Start pairing
e -- End pairing/disconnect
u -- Send volume up
d -- Send volume down
##############################

I (68116) esp_hid_device_main: send_volume_up
E (68116) esp_hid_device_main: HID not connected!
I (72296) esp_hid_device_main: send_volume_down
E (72296) esp_hid_device_main: HID not connected!
W (524716) BT_HCI: hcif conn complete: hdl 0x81, st 0x0
W (524766) BT_HCI: hcif link supv_to changed: hdl 0x81, supv_to 8000
W (524766) BT_HCI: hcif link supv_to changed: hdl 0x81, supv_to 8000
W (526536) BT_APPL: new conn_srvc id:20, app_id:1
W (526546) BT_APPL: bta_dm_pm_ssr conn_srvc id:20, app_id:1
W (526546) BT_APPL: bta_dm_pm_ssr:3, lat:360
I (526546) BT_HIDD: Connected to c8:2a:dd:82:0f:0d
I (526556) bt_hidd_event_callback: CONNECTED
W (531746) BT_HCI: hci cmd send: sniff: hdl 0x81, intv(10 18)
W (531756) BT_HCI: hcif mode change: hdl 0x81, mode 2, intv 18, status 0x0
I (531756) ESP_HID_GAP: BT GAP MODE_CHG_EVT mode:2
W (531816) BT_HCI: hcif ssr evt: st 0x0, hdl 0x81, tx_lat 360 rx_lat 720
I (580116) esp_hid_device_main: send_volume_up
E (580116) BT_HIDD: index out of range[0-0]
E (580116) BT_HIDD: HID device not connected!
E (580216) BT_HIDD: index out of range[0-0]
E (580216) BT_HIDD: HID device not connected!

I made a repo for the full code here: https://github.com/josh-gc/esp_selfie
Any help would be greatly appreciated!

josh-gc · 2023-10-12T21:05:09Z

Update again:
The iPhone works on the first connection after reboot but not if the device is disconnected and then reconnected...

esp-zhp · 2023-10-17T04:14:12Z

@josh-gc
Hello, please apply the following patch and test with iPhone 13 again.
(esp-idf/examples/bluetooth/esp_hid_device)

0003-fix-bt-bluedroid-Fix-bugs-in-ble-service-change-char.patch

If you continue to encounter issues, please let me know.

thanks.

esp-zhp · 2023-10-17T04:24:51Z

@josh-gc
I tested it using an iPhone 13（IOS 17.0.3）, and it appears to be working correctly. Therefore, I need your help in reproducing the issue. Here is a summary of the my testing results:

After connecting to the esp32 (esp_hid_device), the volume on the iPhone 13 starts updating. When I disconnect the connection and open Bluetooth again, it automatically reconnects, and the volume continues to update.

test.esp_hid_device.with.Iphone.13.mp4

josh-gc · 2023-10-17T19:58:44Z

@zhp0406 Thank you for responding. Sorry my bug was not clear, here is the process to reproduce with the patch:

"Forget" the connection and reset the ESP32
Connect to phone -> Volume works
"Forget" the device
Reconnect the device -> Volume does not work
Reset the ESP32
Reconnect the device -> Volume works again

Here is the process to reproduce before the patch:

"Forget" the connection and reset the ESP32
Connect to phone -> Volume does not work
Reset the ESP32
Reconnect the device -> Volume works again

Please let me know if that is not clear and I can make a video.

I had the same experience with my code (esp_selfie) and the esp_hid_device example. I made another thread in the espressif forum to track the issue with bt/Android commands in my code.

josh-gc · 2023-10-18T08:56:47Z

Other thread here: https://www.esp32.com/viewtopic.php?f=13&t=36330

esp-zhp · 2023-10-18T09:59:50Z

@josh-gc
Hi,
I was able to reproduce the issue without applying any patches. Here is the environment I used:

iPhone 13
esp-idf/examples/bluetooth/esp_hid_device
IDF commit: cc71308
I observed a phenomenon: when the iOS pairing window pops up, if I quickly click on pairing, the subsequent operations work fine. However, if I don't click quickly (e.g., waiting for 5 seconds before clicking on pairing), an issue occurs.

Is this observation similar to what you've experienced?
Could you please test whether it works correctly after quickly clicking on pairing to make sure we are experiencing the same issue?

josh-gc · 2023-10-18T13:05:12Z

@zhp0406 I don't think that the timing is the issue. Here is a screen capture from the iPhone where I:

Open BLE
Connect to device
Volume commands fails
Reset the ESP32
Reconnect
Volume commands work
Turn off / on bluetooth
Reconnects
Volume commands work
Forget device
Reconnect device
Volume commands fails
Reset device
Reconnect
Volume commands work

Also, when I tried to do the screen capture over a cable from my computer I could not get it to work at all.

Also, after about a minute I started getting these error messages:
E (56718) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (56718) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (58328) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (58328) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (63398) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (63398) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (74978) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (74978) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (120128) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (120128) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (209268) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (209268) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (218378) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (218378) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT

(This is on my code, not the esp_hid_device but I saw the same issues on both)

BLE_HID_TEST.mov

esp-zhp · 2023-10-19T12:52:50Z

@josh-gc
Based on the current analysis, this should be a compatibility issue. If encryption is not successful, Apple devices will not recognize the ESP32 as an HID device. To verify my theory, please kindly test with the following patch. Thank you.
0001-fix-bt-bluedroid-Fixed-BLE-hid-example-bugs.patch

esp-zhp · 2023-10-19T12:54:37Z

E (63398) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
The above log indicates that writing to this characteristic requires link encryption, but the link is not currently encrypted.

josh-gc · 2023-10-19T17:13:22Z

@zhp0406
add_db_record(_last_db, index++, (uint8_t *)&s_character_client_config_uuid, ESP_GATT_PERM_READ | ESP_GATT_PERM_WRITE_ENCRYPTED, 2, 0, NULL);
The above change you made looks to have solved the re-connection after "forget"!

I am still getting the 'E (94802) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT' error though. It only happens when the iPhone goes to sleep so I had to modify the code to pause long enough to let it sleep. Here is my modified ble_hid_demo_task:

void ble_hid_demo_task(void *pvParameters)
{
	static bool send_volum_up = false;
	static int	count = 0;
	while (1) {
		ESP_LOGI(TAG, "Send the volume");
		if (send_volum_up) {
			esp_hidd_send_consumer_value(HID_CONSUMER_VOLUME_UP, true);
			vTaskDelay(100 / portTICK_PERIOD_MS);
			esp_hidd_send_consumer_value(HID_CONSUMER_VOLUME_UP, false);
		} else {
			esp_hidd_send_consumer_value(HID_CONSUMER_VOLUME_DOWN, true);
			vTaskDelay(100 / portTICK_PERIOD_MS);
			esp_hidd_send_consumer_value(HID_CONSUMER_VOLUME_DOWN, false);
		}
		send_volum_up = !send_volum_up;
		vTaskDelay(2000 / portTICK_PERIOD_MS);

		count ++;	// Increment count

		// Delay for 60 seconds to let the phone sleep
		if(count > 4)
		{
			printf("pause \n");
			vTaskDelay(60000 / portTICK_PERIOD_MS);
			count = 0;
		}
	}
}

and here is the error logs:

I (2402) HID_DEV_BT: Setting to connectable, discoverable
I (5522) HID_DEV_BLE: CONNECT
E (6072) BT_BTM: btm_ble_remove_resolving_list_entry_complete remove resolving list error 0x2
E (6332) BT_SMP: Value for numeric comparison = 995800
I (6332) ESP_HID_GAP: BLE GAP NC_REQ passkey:995800
E (7592) BT_GATT: gatts_write_attr_perm_check - GATT_INSUF_ENCRYPTION
W (10292) BT_SMP: FOR LE SC LTK IS USED INSTEAD OF STK
I (10412) ESP_HID_GAP: BLE GAP KEY type = ESP_LE_KEY_LENC
I (10412) ESP_HID_GAP: BLE GAP KEY type = ESP_LE_KEY_PENC
I (10412) ESP_HID_GAP: BLE GAP KEY type = ESP_LE_KEY_LID
I (10542) ESP_HID_GAP: BLE GAP KEY type = ESP_LE_KEY_PID
I (10692) ESP_HID_GAP: BLE GAP AUTH SUCCESS
I (10692) HID_DEV_DEMO: Send the volume
I (12792) HID_DEV_DEMO: Send the volume
I (14892) HID_DEV_DEMO: Send the volume
I (16992) HID_DEV_DEMO: Send the volume
I (19092) HID_DEV_DEMO: Send the volume
pause 
E (59612) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (59612) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (61392) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (61402) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (69632) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (69632) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
I (81192) HID_DEV_DEMO: Send the volume
E (81592) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
E (81592) BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT
I (83292) HID_DEV_DEMO: Send the volume
I (85392) HID_DEV_DEMO: Send the volume

When I wake up the phone it works fine but the volume command does not wake it up on it's own.

esp-zhp · 2023-10-23T08:06:29Z

@josh-gc
The first occurrence of "BT_GATT: gatts_write_attr_perm_check - GATT_WRITE_NOT_PERMIT" is normal because the connection is not yet encrypted at this point. However, there were several subsequent occurrences, which indicate a bug. I have fixed it in the patch provided below.You can directly apply this patch.
0001-fix-bt-bluedroid-Fixed-BLE-hid-example-bugs_2.patch

Furthermore, I have also addressed another compatibility issue with newer Android devices. If your product is intended for use on higher versions of Android devices, it is recommended to apply this patch as well.

0002-fix-bt-bluedroid-Optimize-compatibility-with-Android.patch

If you encounter any other issues during usage, I will promptly resolve them for you.

esp-zhp · 2023-10-23T13:10:05Z

@josh-gc
Please test it with the patch, and I look forward to your response. Thank you!

josh-gc · 2023-10-24T16:19:09Z

@zhp0406 I was having issues recreating the GATT_WRITE_NOT_PERMIT issue so I reset everything and went through all the steps. Here are my results:

Update the IDF/submodules to: v4.4.6-98-g5f257494c5

Tried to apply patch 0003-fix-bt-bluedroid-Fix-bugs-in-ble-service-change-char but this was already there.
Confirmed the volume commands do not work on iPhone after "Forget Device"
Applied 0001-fix-bt-bluedroid-Fixed-BLE-hid-example-bugs
Confirmed that this patch solved the above issue
I added the delay to test if the phone sleeping caused an issue and was not able to recreate this bug. Sorry for the false report. I skipped 0001-fix-bt-bluedroid-Fixed-BLE-hid-example-bugs_2 since this was no longer needed.
I updated the config to be BLE only:
a. Component config → Bluetooth → Controller Options → Bluetooth controller mode (BR/EDR/BLE/DUALMODE) (BLE Only)
b. Component config → Bluetooth → Bluedroid Options → Classic Bluetooth(Disabled)
Tested on Android (Pixel 7) and the volume buttons did not work
Applied patch 0002-fix-bt-bluedroid-Optimize-compatibility-with-Android
This now works great on my phone!

All of my issues are solved. Thank you for your help!
Will this task stay open until these two patches are in the main branch or is there another way I can keep up with this?

esp-zhp · 2023-10-25T08:51:32Z

@josh-gc
All are okay. I suggest closing the current issue for now. If you encounter other issues, then create a new issue.
At the same time, I will merge the patch I gave you into the master branch(and release branch) after testing it.
Thank you very much for reporting the issue.

esp-zhp · 2023-10-25T08:52:59Z

@josh-gc
I will report back to you after merging into the master branch. However, this might take some time as there are many tests to run. In the meantime, please go ahead and use the patch I gave you.

josh-gc · 2023-10-25T12:03:48Z

Closing issue (still waiting on merge to main branch)

esp-zhp · 2023-11-09T04:28:23Z

@josh-gc
We have fixed the issue with the BLE HID device. Please use the latest branch.

Alternatively, you can also use the following commits:

Commit c73d7c0 improves the compatibility of the HID device on the Windows platform.
Commit 5727802 enhances compatibility with iOS and macOS.
Commit b90dc6a adds support for the HID device control point feature.
Commit f7e174e improves compatibility with Android versions after 10.

josh-gc added the Type: Bug bugs in IDF label Oct 12, 2023

espressif-bot added the Status: Opened Issue is new label Oct 12, 2023

github-actions bot changed the title ~~Volume command is doing nothing in esp_hid_device on esp-idf V4.4~~ Volume command is doing nothing in esp_hid_device on esp-idf V4.4 (IDFGH-11233) Oct 12, 2023

espressif-bot assigned esp-zhp Oct 17, 2023

espressif-bot added Status: In Progress Work is in progress and removed Status: Opened Issue is new labels Oct 17, 2023

josh-gc closed this as completed Oct 25, 2023

espressif-bot added Status: Done Issue is done internally Resolution: Done Issue is done internally and removed Status: In Progress Work is in progress labels Oct 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Volume command is doing nothing in esp_hid_device on esp-idf V4.4 (IDFGH-11233) #12397

Volume command is doing nothing in esp_hid_device on esp-idf V4.4 (IDFGH-11233) #12397

josh-gc commented Oct 12, 2023

josh-gc commented Oct 12, 2023

josh-gc commented Oct 12, 2023

esp-zhp commented Oct 17, 2023

esp-zhp commented Oct 17, 2023

josh-gc commented Oct 17, 2023 •

edited

Loading

josh-gc commented Oct 18, 2023

esp-zhp commented Oct 18, 2023 •

edited

Loading

josh-gc commented Oct 18, 2023 •

edited

Loading

esp-zhp commented Oct 19, 2023

esp-zhp commented Oct 19, 2023 •

edited

Loading

josh-gc commented Oct 19, 2023 •

edited

Loading

esp-zhp commented Oct 23, 2023

esp-zhp commented Oct 23, 2023

josh-gc commented Oct 24, 2023 •

edited

Loading

esp-zhp commented Oct 25, 2023

esp-zhp commented Oct 25, 2023

josh-gc commented Oct 25, 2023

esp-zhp commented Nov 9, 2023

Volume command is doing nothing in esp_hid_device on esp-idf V4.4 (IDFGH-11233) #12397

Volume command is doing nothing in esp_hid_device on esp-idf V4.4 (IDFGH-11233) #12397

Comments

josh-gc commented Oct 12, 2023

Answers checklist.

IDF version.

Espressif SoC revision.

Operating System used.

How did you build your project?

If you are using Windows, please specify command line type.

Development Kit.

Power Supply used.

What is the expected behavior?

What is the actual behavior?

Steps to reproduce.

Debug Logs.

More Information.

josh-gc commented Oct 12, 2023

Code:

Error:

josh-gc commented Oct 12, 2023

esp-zhp commented Oct 17, 2023

esp-zhp commented Oct 17, 2023

josh-gc commented Oct 17, 2023 • edited Loading

josh-gc commented Oct 18, 2023

esp-zhp commented Oct 18, 2023 • edited Loading

josh-gc commented Oct 18, 2023 • edited Loading

esp-zhp commented Oct 19, 2023

esp-zhp commented Oct 19, 2023 • edited Loading

josh-gc commented Oct 19, 2023 • edited Loading

esp-zhp commented Oct 23, 2023

esp-zhp commented Oct 23, 2023

josh-gc commented Oct 24, 2023 • edited Loading

esp-zhp commented Oct 25, 2023

esp-zhp commented Oct 25, 2023

josh-gc commented Oct 25, 2023

esp-zhp commented Nov 9, 2023

josh-gc commented Oct 17, 2023 •

edited

Loading

esp-zhp commented Oct 18, 2023 •

edited

Loading

josh-gc commented Oct 18, 2023 •

edited

Loading

esp-zhp commented Oct 19, 2023 •

edited

Loading

josh-gc commented Oct 19, 2023 •

edited

Loading

josh-gc commented Oct 24, 2023 •

edited

Loading