diff --git a/CHANGELOG.md b/CHANGELOG.md index 7a1df83..ec489a8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,30 @@ ## ChangeLog +### v1.0b1r7 (Beta) - 2018-01-08 + +**Enhancements** + +* Support for Google Voice Assistant (GVA) and Google Dialogflow (Alpha release). Please check README of each voice assistant to get started. +* Support to play custom tones from application. +* nvs-set CLI supports setting and getting int8_t variable type. + +**API Changes** + +* Example directories are renamed. +* Many structures, functions and header files with "alexa" prefix are renamed with "va" prefix. Please follow README-FIRST.md if you want to compile and run older examples with new SDK. + +**Bug Fixes** + +* Background noise in audio with lower sampling rates. +* In case of incorrect provisioning parameters, device needed to be reset via command line (via esptool.py on host or nvs-erase on device). Now reset-to-factory push button (Mode) could also be used. +* Capabilities are announced automatically when an updated firmware with new capabilities is flashed onto the device. +* More SNTP servers are added for faster time synchronization on device boot-up. + +**Known Issues/Improvements** + +* Only limited TuneIn radio stations are supported +* It is largely tested with internet and WiFi connectivity intact throughout its operation. Some issues are seen when device loses connectivity. + ### v1.0b1r6 (Beta) - 2018-12-24 **Enhancements** diff --git a/README-Alexa.md b/README-Alexa.md new file mode 100644 index 0000000..7f83641 --- /dev/null +++ b/README-Alexa.md @@ -0,0 +1,36 @@ +# Introduction +Alexa is Amazon's personal virtual assistant which listens to user's voice commands and responds with appropriate answers. Apart from conversing with the user, Alexa lets you play music from a variety of music streaming services. Alexa also helps you manage to-do lists and allows for voice-assisted shopping from Amazon. + +# Device Configuration +* Before proceeding with device configuration, make sure you have read and followed `README-Getting-Started.md`. +* By default, when the firmware comes up it is un-provisioned. The device console should display the following lines in this mode: +``` +I (xxx) conn_mgr_prov: Provisioning started with : + service name = ESP-Alexa-xxxx + service key = +``` +You can use either the Android or iOS apps (links below) to provision your device to the desired Wi-Fi Access Point and associate the user's Amazon account with the device. +* [Android](https://github.com/espressif/esp-avs-sdk/releases) +* [iOS](https://github.com/espressif/esp-idf-provisioning-ios/tree/versions/avs) + +# Demo +* Once the board boots up and successfully connects to the Wi-Fi network after provisioning, you will see a print "Alexa is ready", after which you can use either use "Rec" button on the board or say "Alexa" to start a conversation. For Tap-to-Talk, press and release the button and speak. The green LED glows when the microphone is active. +* You can connect any external speaker/headphone with 3.5mm connector to PHONE JACK to listen to responses. +* you can now ask any command like: + * tell me a joke + * how is the weather? + * will it rain today? + * Sing a song + * Play TuneIn radio + * Set volume level to 7 +* Press and Hold "Mode" button for 3 seconds to reset the board to factory settings. + +# Supported music streaming services +* Amazon Music +* Saavn (India) +* Pandora +* TuneIn Radio +* iHeart Radio + +# Production notes +* In order to create Alexa-enabled commercial products, Amazon certified acoustic front-end has to be used. Please reach out to Espressif if you are looking to go to production. diff --git a/README-Dialogflow.md b/README-Dialogflow.md new file mode 100644 index 0000000..ff76ee9 --- /dev/null +++ b/README-Dialogflow.md @@ -0,0 +1,65 @@ +# Introduction +Dialogflow (previously known as API.AI) is a voice enabled conversational interface from Google. +It enables IoT users to include natural language user interface in their applications, services and devices. + +The advantages of Dialogflow wrt voice assistants are less complexity, pay as you go pricing, custom wakeword allowed and no certification hassles. + +Unlike voice-assistants, Dialogflow let's you configure every step of the conversation, and it won't answer other trivia/questions like voice-assistants typically do. For e.g. A Dialogflow agent for Laundry project will provide information only about the configurable parameters of the laundry (like state, temperature, wash cycle etc.) + +This release facilitates the audio communication of ESP32 with a Google Dialogflow agent using its v2beta1 gRPC APIs + +# Dialogflow Agent Setup +You will have to create a Dialogflow account and setup a Dialogflow agent in the cloud. This agent configuration is where you will specify what conversations will you be supporting. +* Follow this [link](https://dialogflow.com/docs/getting-started) to get started. +* Create your own agent + * You can add intents, entities, actions and parameters as per your agent's requirements. + * Build, test your agent and validate the responses using the console on Dialogflow. +* Optionally, you can add an existing sample agent from [here](https://dialogflow.com/docs/samples) to your Dialogflow account and use the same. +Note: +> Make sure that "Set this intent as end of conversation" is enabled under "Responses" tab in each intent of your Dialogflow agent so that the device can use this information to close the interaction with user + +# Device Configuration +* Your device needs to be configured with the correct credentials to talk to the above project. +* Navigate to this [link](https://console.cloud.google.com/apis/dashboard) + * Select the agent created above as the project + * Go to Credentials section (on the left) + * Credentials -> Create credentials -> OAuth client ID -> Other + * OAuth consent screen -> Fill in the Support email (and other details as required) -> Save + * Credentials -> OAuth 2.0 client IDs -> Download the created OAuth client ID file (`client_secret_.json`) +* Follow the steps specified in this [link](https://developers.google.com/assistant/sdk/guides/library/python/embed/install-sample#generate_credentials) + * While using this step, use the following command instead of the one specified: +``` +google-oauthlib-tool --scope https://www.googleapis.com/auth/cloud-platform \ + --save --headless --client-secrets /path/to/client_secret_.json +``` + +* Modify the example application provided in this SDK, to add the project ID (from `client_secret_.json` file) in the `project_name` member of `device_config` before making a call to `dialogflow_init()` +* Once you download credentials.json, you can use the following commands on the device console to set the client ID, client secret and refresh token on the device. +``` +[Enter] +>> nvs-set avs refreshToken string +>> nvs-set avs clientId string +>> nvs-set avs clientSecret string +``` +* Use below CLI command to configure device's station interface +``` +[Enter] +>> wifi-set +``` + +# Demo +* Once the board successfully connects to the Wi-Fi network, you can either use "Rec" button on the board or say "Alexa" to start a conversation. (The current example only supports the _Alexa_ wakeword. Support for other wakeword will be available soon.) For Tap-to-Talk, press and release the button and speak. +* For an example Laundry project, one can set below configurable parameters while creating the project as described above: + * State: On/Start or Off/Stop + * Temperature: Valid temperature values + * Wash Cycle: Heavy, Medium or Light +* Now you can wake the device using either its wakeword ("Alexa") or pressing "Rec" button and say command like: + * Start the laundry with temperature 68 and heavy wash cycle +* You can also initiate multi-turn conversations like: + * Turn on the laundry + * (Dialogflow: At what temperature) 75 + * (Dialogflow: What is the wash cycle) Light wash cycle +* The Assistant's language can be changed by setting an appropriate code string va_cfg->device_config.device_language in app_main.c. List of valid code strings can be found [here](https://dialogflow.com/docs/reference/language). + +NOTE: +> Once a multi-turn conversation is in-progress you do not need to press "Rec" button or say wakeword for every turn. diff --git a/README-FIRST.md b/README-FIRST.md index 980e56c..e2f9fc9 100644 --- a/README-FIRST.md +++ b/README-FIRST.md @@ -1,3 +1,12 @@ +## Upgrading to v1.0b1r7 (Beta) + +* Most API names have changed from alexa\_ to va\_. Please follow these steps to upgrade. +* Download script v1_0r6Tov1_0r7_update_script.sh available under "releases" tab, and copy it in SDK's root directory (esp-voice-assistants). +* Copy your example in examples directory. +* Run the script using below commands: + * cd /path/to/esp-voice-assistants + * ./v1_0r6Tov1_0r7_update_script.sh + ## Upgrading to v1.0b1r6 (Beta) * Release v1.0b1r6 changes the way WiFi credentials are stored in NVS. Hence when you upgrade your device to latest SDK, your device would go into provisioning mode again, and needs to be re-provisioned via provisioning app. diff --git a/README-GVA.md b/README-GVA.md new file mode 100644 index 0000000..f733bae --- /dev/null +++ b/README-GVA.md @@ -0,0 +1,37 @@ +# Introduction +Google Voice Assistant(GVA) is Google's version of a personal voice assistant. GVA is multilingual and allows users to converse in their preferred language. Apart from general queries, it allows users to check on the traffic conditions, emails, weather conditions and much more. + +# Project Setup +* Before proceeding with this section, make sure you have read and followed `README-Getting-Started.md`. +* Follow steps specified in this [link](https://developers.google.com/assistant/sdk/guides/library/python/embed/config-dev-project-and-account) and execute the following sections: + * Configure an Actions Console project + * Set activity controls for your account + * Register the Device Model using the registration UI + * Generate credentials + +# Device Configuration +* In the project setup steps above, you would also have generated credentials to be configured in the device. +* Once you download credentials.json, you can use below commands on device console to set client ID, client secret and refresh token on the device. +``` +[Enter] +>> nvs-set avs refreshToken string +>> nvs-set avs clientId string +>> nvs-set avs clientSecret string +``` +* Use below CLI command to configure device's station interface +``` +[Enter] +>> wifi-set +``` + +# Demo +* Once the board successfully connects to the Wi-Fi network, you can use either use "Rec" button on the board or say "Alexa" to start a conversation. (The current example only supports the _Alexa_ wakeword. Support for other wakeword will be available soon.) For Tap-to-Talk, press and release the button and speak. +* You can connect any external speaker/headphone with 3.5mm connector to PHONE JACK to listen to responses. +* you can now ask any command like: + * tell me a joke + * how is the weather? + * will it rain today? + * Sing a song + * Set volume level to 7 +* Press and Hold "Mode" button for 3 seconds to reset the board to factory settings +* Assistant's language can be changed by setting appropriate code string va_cfg->device_config.device_language in app_main.c. List of valid codes strings can be found [here](https://developers.google.com/actions/localization/languages-locales). diff --git a/README-Getting-Started.md b/README-Getting-Started.md new file mode 100644 index 0000000..97ff32b --- /dev/null +++ b/README-Getting-Started.md @@ -0,0 +1,39 @@ +# Prerequisites +Please prepare your host system with the toolchain. Please see http://esp-idf.readthedocs.io/en/latest/get-started/index.html for the host setup. + +# Supported Hardware +lyrat and lyrat_sr application supports ESP32 based LyraT v4.1, LyraT v4.2 and LyraT v4.3 +lyratd_msc_sr application supports ESP32 based LyraTD_MSC v2.0, LyraTD_MSC v2.1 + +# Prepare Images + +## Clone all the repositories +``` +$ git clone --recursive https://github.com/espressif/esp-idf.git + +$ cd esp-idf; git checkout release/v3.1; cd .. + +$ git clone https://github.com/espressif/esp-avs-sdk.git +``` + +## Apply patches on esp-idf +``` +$ cd esp-idf + +$ git apply ../esp-avs-sdk/esp-idf-patches/memset-i2s-dma-buffers-zero.patch + +$ git apply ../esp-avs-sdk/esp-idf-patches/esp-tls-Add-support-for-global-CA-store.-All-mbedtls.patch +``` + +## Build and flash the project +``` +$ cd esp-avs-sdk/examples/ + +$ export IDF_PATH=/path/to/esp-idf + +$ export ESPPORT=/dev/cu.SLAB_USBtoUART (or /dev/ttyUSB0 or /dev/ttyUSB1 on Linux or COMxx on MinGW) + +$ make -j 8 flash VOICE_ASSISTANT= monitor +``` +NOTE: +> lyrat app only supports Tap-to-talk whereas lyrat_sr and lyratd_msc_sr apps support both, "Alexa" wakeword and tap-to-talk. diff --git a/README.md b/README.md index 24c666f..58adf74 100644 --- a/README.md +++ b/README.md @@ -1,29 +1,46 @@ ## Overview -The ESP-Alexa SDK provides an implementation of Amazon's Alexa Voice Service endpoint for ESP32 microcontroller. This facilitates the developers to evaluate ESP32 based Alexa integrated devices like speakers and IoT devices. Please refer to [Changelog](CHANGELOG.md) to track release changes and known-issues. +The ESP-Voice-Assistant SDK provides an implementation of Amazon's Alexa Voice Service, Google Voice Assistant and Google's conversational interface (aka Dialogflow) for ESP32 microcontroller. This facilitates the developers to evaluate ESP32 based voice assistant/s integrated devices like speakers and IoT devices. Please refer to [Changelog](CHANGELOG.md) to track release changes and known-issues. ### About SDK -The SDK contains pre-built library of Alexa SDK along with sources of some of the utility components such as audio pipeline and connection manager. The SDK supports all major features of Alexa such as: -* Basic Alexa conversation -* Alexa dialogues and multi-turn -* Audio Streaming and Playback: Saavn, Amazon music, TuneIn (Only limited stations are supported as of now) -* Audio Book Support: Kindle, Audible -* Volume control via Alexa command -* Seek support for Audible -* Alerts/Timers, Reminders, Notifications - -For now, Tap-To-Talk is the only interaction mode supported on LyraT. +The SDK contains pre-built libraries for Alexa, GVA and Dialogflow along with sources of some of the utility components such as audio pipeline and connection manager. Below are the list of features supported for each voice assistant: +* **Alexa**: + * Basic Alexa conversation + * Alexa dialogues and multi-turn + * Audio Streaming and Playback: Saavn, Amazon music, TuneIn (Only limited stations are supported as of now) + * Audio Book Support: Kindle, Audible + * Volume control via Alexa command + * Seek support for Audible + * Alerts/Timers, Reminders, Notifications + +* **Google Voice Assistant**: + * Basic conversation + * Multi-turn conversations + * Getting weather reports + * Multiple language support + +* **Google Dialogflow**: + * Basic conversation + * Multi-turn conversations + * Configure and control connected devices via voice, e.g "Turn the light on" + * Multiple language support ## Supported Hardware -Release supports following hardware platforms: +The SDK supports the following hardware platforms: * [ESP32-LyraT](https://www.espressif.com/en/products/hardware/esp32-lyrat) * [ESP32-LyraTD-MSC](https://www.espressif.com/en/products/hardware/esp32-lyratd-msc) -The SDK can easily be extended to other ESP32 based audio platforms that have SPIRAM availability. +The following list of acoustic front-ends is also supported. Please contact Espressif to enable acccess to these solutions. +* DSPG DBMD5 +* Intel s1000 +* Synaptics CX20921 ## Getting started -* When flashing the SDK for the first time, it is recommended to do `make erase_flash` to wipe out entire flash and start out fresh. -* Please refer to example READMEs to get started with flashing, provisioning and Alexa interactions. +* Follow the `README-Getting-Started.md` to clone the required repositories and to compile and flash the firmware. + * When flashing the SDK for the first time, it is recommended to do `make erase_flash` to wipe out entire flash and start out fresh. +* Go through `README-.md` to know how to provision the device and to get authentication tokens from respective authorization server and flash them onto the device. +* Check example application's README for board or example specific changes that might be required. +* If you are updating from previous release, please check `README-FIRST.md` to know about any specific actions that needs to be taken while upgrading. diff --git a/components/codecs/include/audio_codec.h b/components/codecs/include/audio_codec.h index fe43ad4..c2aee78 100644 --- a/components/codecs/include/audio_codec.h +++ b/components/codecs/include/audio_codec.h @@ -41,9 +41,22 @@ typedef enum { CODEC_TYPE_MP3 = 1, CODEC_TYPE_AAC, CODEC_TYPE_MP4, - CODEC_TYPE_OPUS + CODEC_TYPE_OPUS, + CODEC_TYPE_FLAC, + CODEC_TYPE_AMR } audio_codec_identifier_t; +/* Audio type */ +typedef enum { + AUDIO_TYPE_UNKNOWN, + AUDIO_TYPE_WAV, + AUDIO_TYPE_AMRNB, + AUDIO_TYPE_AMRWB, + AUDIO_TYPE_M4A, + AUDIO_TYPE_AAC, + AUDIO_TYPE_TSAAC +} audio_type_t; + typedef enum { CODEC_STATE_INIT = 1, CODEC_STATE_RUNNING, @@ -52,6 +65,12 @@ typedef enum { CODEC_STATE_DESTROYED, } audio_codec_state_t; +enum { + CODEC_FAIL = 0, + CODEC_DONE = 1, + CODEC_OK +}; + typedef struct audio_codec_audio_info { int sampling_freq; int channels; diff --git a/components/codecs/include/resampling.h b/components/codecs/include/resampling.h index 366baf3..f76bee7 100644 --- a/components/codecs/include/resampling.h +++ b/components/codecs/include/resampling.h @@ -38,7 +38,8 @@ typedef struct { short inpcm[INPCM_DELAY_SIZE * 2]; ///the pcm value of last time calling. maximum should be 6: 48000/8000; int innum; /// the total input pcm number int outnum; /// the total outnum pcm number - float hp_mem[4]; ///for filter, the first two is for first channel, the last two is for second channel + float hp_mem[4]; /// for filter, the first two is for first channel, the last two is for second channel + int resample_ratio; /// (out_freq * 512)/in_freq. if prev resample ratio is not same as current, reset whole structure. } audio_resample_config_t; /** diff --git a/components/codecs/lib/libcodecs.a b/components/codecs/lib/libcodecs.a index e84cbae..96662c0 100644 Binary files a/components/codecs/lib/libcodecs.a and b/components/codecs/lib/libcodecs.a differ diff --git a/components/esp-alexa/component.mk b/components/esp-alexa/component.mk deleted file mode 100644 index 5e98cf0..0000000 --- a/components/esp-alexa/component.mk +++ /dev/null @@ -1,16 +0,0 @@ -# -# Component Makefile -# - -COMPONENT_ADD_INCLUDEDIRS := include - -#LIBS := alexa - -#COMPONENT_ADD_LDFLAGS += -L$(COMPONENT_PATH)/alexa/lib \ - $(addprefix -l,$(LIBS)) \ - -#ALL_LIB_FILES += $(patsubst %,$(COMPONENT_PATH)/%/lib/lib%.a,$(LIBS)) - -ALEXA_LIB_PATH := $(COMPONENT_PATH)/lib/libalexa.a -COMPONENT_ADD_LDFLAGS += $(ALEXA_LIB_PATH) -COMPONENT_ADD_LINKER_DEPS += $(ALEXA_LIB_PATH) diff --git a/components/esp-alexa/include/tone.h b/components/esp-alexa/include/tone.h deleted file mode 100644 index fb07e77..0000000 --- a/components/esp-alexa/include/tone.h +++ /dev/null @@ -1,22 +0,0 @@ -// Copyright 2018 Espressif Systems (Shanghai) PTE LTD -// All rights reserved. - -#pragma once - -#include - -typedef enum { - TONE_WAKE, - TONE_WAKE_TOUCH, - TONE_ENDPOINT, - TONE_PRIVACY_ON, - TONE_PRIVACY_OFF, - TONE_ALARM_SHORT, - TONE_TIMER_SHORT, - TONE_MAX, -} tone_type_t; - -int tone_play(tone_type_t type); -/** This API will enable a tone which would be played if dialog is in progress and timer/alarm goes off. - * Enabling this tone would increase size of binary by around 768 KB */ -void tone_enable_larger_tones(); diff --git a/components/esp-voice-assistant/component.mk b/components/esp-voice-assistant/component.mk new file mode 100644 index 0000000..e1a97cb --- /dev/null +++ b/components/esp-voice-assistant/component.mk @@ -0,0 +1,16 @@ +# +# Component Makefile +# + +COMPONENT_ADD_INCLUDEDIRS := include + +ifeq ("$(VOICE_ASSISTANT)", "gva") +VA_LIB_PATH := $(COMPONENT_PATH)/lib/libgva.a +else ifeq ("$(VOICE_ASSISTANT)", "dialogflow") +VA_LIB_PATH := $(COMPONENT_PATH)/lib/libdialogflow.a +else +VA_LIB_PATH := $(COMPONENT_PATH)/lib/libalexa.a +endif + +COMPONENT_ADD_LDFLAGS += $(VA_LIB_PATH) +COMPONENT_ADD_LINKER_DEPS += $(VA_LIB_PATH) diff --git a/components/esp-alexa/include/alerts.h b/components/esp-voice-assistant/include/alerts.h similarity index 100% rename from components/esp-alexa/include/alerts.h rename to components/esp-voice-assistant/include/alerts.h diff --git a/components/esp-voice-assistant/include/alexa.h b/components/esp-voice-assistant/include/alexa.h new file mode 100644 index 0000000..86287fe --- /dev/null +++ b/components/esp-voice-assistant/include/alexa.h @@ -0,0 +1,28 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + +#ifndef _ALEXA_H_ +#define _ALEXA_H_ + +#include "voice_assistant.h" + +/** The Alexa Configuration Structure + */ +typedef struct { + /** Configuration for the Auth Delegate */ + auth_delegate_config_t auth_delegate; + va_playback_config_t va_playback; +} alexa_config_t; + +/** Initialize Alexa + * + * This call must be made after the Wi-Fi connection has been established with the configured Access Point. + * + * \param[in] cfg The Alexa Configuration + * + * \return + * - 0 on Success + * - an error code otherwise + */ +int alexa_init(alexa_config_t *cfg); +#endif /*_ALEXA_H_ */ diff --git a/components/esp-alexa/include/audio_player.h b/components/esp-voice-assistant/include/audio_player.h similarity index 100% rename from components/esp-alexa/include/audio_player.h rename to components/esp-voice-assistant/include/audio_player.h diff --git a/components/esp-alexa/include/avs_config.h b/components/esp-voice-assistant/include/avs_config.h similarity index 100% rename from components/esp-alexa/include/avs_config.h rename to components/esp-voice-assistant/include/avs_config.h diff --git a/components/esp-voice-assistant/include/dialogflow.h b/components/esp-voice-assistant/include/dialogflow.h new file mode 100644 index 0000000..3732a2c --- /dev/null +++ b/components/esp-voice-assistant/include/dialogflow.h @@ -0,0 +1,61 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + +#ifndef _DIALOGFLOW_H_ +#define _DIALOGFLOW_H_ + +#include "voice_assistant.h" +#include "session.pb-c.h" + +/** Device specific configuration */ +typedef struct { + /** The registered name of the project */ + char *project_name; + /** The supported device language (default: "en-US") */ + char *device_language; +} dialogflow_device_config_t; + +/** The Dialogflow Configuration Structure */ +typedef struct { + /** Configuration for the Auth Delegate */ + auth_delegate_config_t auth_delegate; + va_playback_config_t va_playback; + /** Configurations for the device */ + dialogflow_device_config_t device_config; +} dialogflow_config_t; + +/** Initialize Dialogflow + * + * This call must be made after the Wi-Fi connection has been established with the configured Access Point. + * + * \param[in] cfg The Dialogflow configuration of type dialogflow_config_t + * + * \return + * - 0 on Success + * - an error code otherwise + */ +int dialogflow_init(dialogflow_config_t *cfg); + +/** Audio request transcript + * + * This callback is executed each time the text transcript of the input audio + * is received from the Dialogflow server. Multiple transcripts can be received + * during the course of a single input audio request. The final one will have the is_final bool set to true + * + * \param[in] text The current transcript of audio request as interpreted by the Dialogflow server + * \param[in] is_final If false, it represents an interim result that may change + * If true, the recognizer will not return any further hypotheses about this piece of audio + */ +void dialogflow_app_query_text_transcript(char *text, bool is_final); + +/** Dialogflow response data + * + * This callback is executed each time the query result is populated by the Dialogflow server + * and sent back to the device. It is the responsibility of the application to check if all + * the required entities of an intent are received and then take the necessary action. + * + * \param[in] response The response from Dialogflow server of type Google__Cloud__Dialogflow__V2beta1__DetectIntentResponse + */ +void dialogflow_app_response_data(Google__Cloud__Dialogflow__V2beta1__StreamingDetectIntentResponse *response); + +#endif /*_DIALOGFLOW_H_ */ diff --git a/components/esp-alexa/include/equalizer_controller.h b/components/esp-voice-assistant/include/equalizer_controller.h similarity index 100% rename from components/esp-alexa/include/equalizer_controller.h rename to components/esp-voice-assistant/include/equalizer_controller.h diff --git a/components/esp-voice-assistant/include/gva.h b/components/esp-voice-assistant/include/gva.h new file mode 100644 index 0000000..30dffcd --- /dev/null +++ b/components/esp-voice-assistant/include/gva.h @@ -0,0 +1,30 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + +#ifndef _GVA_H_ +#define _GVA_H_ + +#include "voice_assistant.h" + +/** Device specific configuration */ +typedef struct { + /** The registered device model */ + char *device_model; + /** A unique device id */ + char *device_id; + /** The supported device language (default: "en-IN") */ + char *device_language; +} gva_device_config_t; + +/** The GVA Configuration Structure */ +typedef struct { + /** Configuration for the Auth Delegate */ + auth_delegate_config_t auth_delegate; + va_playback_config_t va_playback; + /** Configurations for the device */ + gva_device_config_t device_config; +} gva_config_t; + +int gva_init(gva_config_t *gva_cfg); + +#endif /*_GVA_H_ */ diff --git a/components/esp-alexa/include/playback_controller.h b/components/esp-voice-assistant/include/playback_controller.h similarity index 100% rename from components/esp-alexa/include/playback_controller.h rename to components/esp-voice-assistant/include/playback_controller.h diff --git a/components/esp-alexa/include/speaker.h b/components/esp-voice-assistant/include/speaker.h similarity index 54% rename from components/esp-alexa/include/speaker.h rename to components/esp-voice-assistant/include/speaker.h index 38c29da..96571a7 100644 --- a/components/esp-alexa/include/speaker.h +++ b/components/esp-voice-assistant/include/speaker.h @@ -6,11 +6,11 @@ /** Notify Speaker Volume Change * - * This API should be called by the application to notify Alexa that a + * This API should be called by the application to notify voice assistant that a * user has changed the speaker volume using on-device buttons. * * Typically, a user presses a button multiple times to change the - * volumes. In order to avoid sending multiple requests to Alexa, the + * volumes. In order to avoid sending multiple requests to the voice assistant, the * application must take care that the multiple button presses are * absorbed by the application and a call with the final volume * setting be made. @@ -23,7 +23,8 @@ int speaker_notify_vol_changed(int vol); /** Change the speaker volume * * This API should be called by the application to change the volume of the speaker. - * The volume is changed instantly. Also an event for volume changed is sent to Alexa. + * The volume is changed instantly. Also a callback event for volume changed is sent to + * the voice assistant. * * \param[in] vol Volume represented in percentage. Allowed range is 0-100. * @@ -31,11 +32,30 @@ int speaker_notify_vol_changed(int vol); int speaker_set_vol(int vol); /** Get the speaker volume - * + * * This API returns the currrent speaker volume. The value is in * percentage and its range is 0-100. * */ int speaker_get_vol(); +/** Change the speaker mute state + * + * This API should be called by the application to change the mute state of the speaker. + * The mute is enabled/disabled instantly. Also a callback event for mute changed is sent * to the voice assistant. + * the call. + * + * \param[in] vol Volume represented in percentage. Allowed range is 0-100. + * + */ +int speaker_set_mute(bool mute); + +/** Get the mute state + * + * This API returns the currrent mute state. The value is true is mute is enabled and + * false if mute is disabled. + * + */ +bool speaker_get_mute(); + #endif /* _SPEAKER_H_ */ diff --git a/components/esp-alexa/include/speech_recognizer.h b/components/esp-voice-assistant/include/speech_recognizer.h similarity index 67% rename from components/esp-alexa/include/speech_recognizer.h rename to components/esp-voice-assistant/include/speech_recognizer.h index 7577c4e..27b2373 100644 --- a/components/esp-alexa/include/speech_recognizer.h +++ b/components/esp-voice-assistant/include/speech_recognizer.h @@ -12,13 +12,13 @@ enum initiator { TAP, /** This is not yet supported */ HOLD_AND_TALK, - /** Communication was initatied by Alexa due to ExpectSpeech. The application will never send this value */ + /** Communication was initatied by voice assistant due to ExpectSpeech. The application will never send this value */ EXPECT_SPEECH }; -/** Stream data to Alexa +/** Stream data to the voice assistant * - * Once an application indicates to Alexa of a user initiated + * Once an application indicates to the voice assistant of a user initiated * communication, this API must be called to stream the audio data * captured by the microphone. This data must be 16-byte 16KHz sampled * audio data. @@ -28,16 +28,16 @@ enum initiator { */ int speech_recognizer_record(void *data, int len); -/** Signal end of data to Alexa +/** Signal end of data to the voice assistant * -* This API must be called when all the data has been sent to Alexa. +* This API must be called when all the data has been sent to the voice assistant. */ int speech_recognizer_record_stop(); -/** Notify Alexa of a recognize event +/** Notify the voice assistant of a recognize event * * The application should call this function when the user initiates - * communication with Alexa either through a wakeword or through + * communication with the voice assistant either through a wakeword or through * Tap-to-talk. * * \param[in] ww_length The length of the wakeword, if this is a wakeword driven interaction diff --git a/components/esp-voice-assistant/include/tone.h b/components/esp-voice-assistant/include/tone.h new file mode 100644 index 0000000..ccf30dd --- /dev/null +++ b/components/esp-voice-assistant/include/tone.h @@ -0,0 +1,34 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + +#pragma once + +#include +#include + +typedef enum { + TONE_WAKE, + TONE_WAKE_TOUCH, + TONE_ENDPOINT, + TONE_PRIVACY_ON, + TONE_PRIVACY_OFF, + TONE_ALARM_SHORT, + TONE_TIMER_SHORT, + TONE_MAX, +} tone_type_t; + +/** This API will play custom tones provided by the application (should be the part of firmware itself) + * + * Please check "Build System" section of ESP-IDF Programming guide to know how to make changes in Makefile or component.mk to embed binary data in firmware and get start and end pointers of the binary. + * Supported files types: wav, mp3 + * + * \param[in] start Pointer to start of tone in flash + * \param[in] end Pointer to end of tone in flash + * \param[in] playback_param Pointer to playback parameters. Please note this needs to point to valid location while the tone is being played. It's recommeded that it is declared as either global or static variable. + * \return 0 on success, -1 otherwise + */ +int tone_play_custom(const uint8_t *start, const uint8_t *end, va_resample_param_t *playback_param); +int tone_play(tone_type_t type); +/** This API will enable a tone which would be played if dialog is in progress and timer/alarm goes off. + * Enabling this tone would increase size of binary by around 768 KB */ +void tone_enable_larger_tones(); diff --git a/components/esp-alexa/include/alexa.h b/components/esp-voice-assistant/include/voice_assistant.h similarity index 64% rename from components/esp-alexa/include/alexa.h rename to components/esp-voice-assistant/include/voice_assistant.h index 6aeb6b1..f6f6022 100644 --- a/components/esp-alexa/include/alexa.h +++ b/components/esp-voice-assistant/include/voice_assistant.h @@ -1,15 +1,11 @@ // Copyright 2018 Espressif Systems (Shanghai) PTE LTD // All rights reserved. -#ifndef _ALEXA_H_ -#define _ALEXA_H_ +#ifndef _VOICE_ASSISTANT_H_ +#define _VOICE_ASSISTANT_H_ #include -#ifdef __cplusplus -extern "C" { -#endif - typedef struct { const char *auth_code; const char *client_id; @@ -25,14 +21,14 @@ typedef struct { /** Auth Delegate Configuration Type * - * Please refer to the documentation for \ref alexa_auth_type_t for more details. + * Please refer to the documentation for \ref va_auth_type_t for more details. */ typedef enum { /** Subsequent authentication */ auth_type_subsequent = 1, /** Authentication tokens from companion app */ auth_type_comp_app, -} alexa_auth_type_t; +} va_auth_type_t; /** Auth Delegate Configuration * @@ -41,7 +37,7 @@ typedef enum { * - subsequent: We have the real access tokens in NVS, and we should use that for authentication. This step typically happens after the comp_app authentication has been used once. */ typedef struct { - alexa_auth_type_t type; + va_auth_type_t type; union { comp_app_config_t comp_app; subsequent_auth_config_t subsequent; @@ -56,36 +52,14 @@ typedef struct { size_t stack_size; //Default task stack size is 5000 int task_priority; //Default priority is 5 size_t buf_size; //Default buffer size is 512 bytes -} alexa_playback_config_t; +} va_playback_config_t; -/** The Alexa Configuration Structure - */ -typedef struct { - /** Configuration for the Auth Delegate */ - auth_delegate_config_t auth_delegate; - alexa_playback_config_t alexa_playback; -} alexa_config_t; -/** Initialize Alexa - * - * This call must be made after the Wi-Fi connection has been established with the configured Access Point. - * - * \param[in] cfg The Alexa Configuration - * - * \return - * - 0 on Success - * - an error code otherwise - */ -int alexa_init(alexa_config_t *cfg); -/** Get current Alexa SDK version +/** Get current Voice Assistant SDK version * * \return Pointer to version string */ -const char *alexa_get_sdk_version(); - -#ifdef __cplusplus -} -#endif +const char *va_get_sdk_version(); -#endif /*_ALEXA_H_ */ +#endif /*_VOICE_ASSISTANT_H_ */ diff --git a/components/esp-alexa/include/alexa_app_cb.h b/components/esp-voice-assistant/include/voice_assistant_app_cb.h similarity index 52% rename from components/esp-alexa/include/alexa_app_cb.h rename to components/esp-voice-assistant/include/voice_assistant_app_cb.h index 0c0315c..b725035 100644 --- a/components/esp-alexa/include/alexa_app_cb.h +++ b/components/esp-voice-assistant/include/voice_assistant_app_cb.h @@ -1,59 +1,55 @@ // Copyright 2018 Espressif Systems (Shanghai) PTE LTD // All rights reserved. -#ifndef _ALEXA_APP_CB_H_ -#define _ALEXA_APP_CB_H_ +#ifndef _VOICE_ASSISTANT_APP_CB_H_ +#define _VOICE_ASSISTANT_APP_CB_H_ #include -#ifdef __cplusplus -extern "C" { -#endif - -/** Alexa Dialog States +/** Voice Assistant Dialog States */ typedef enum { - /** Alexa is Thinking */ - ALEXA_THINKING = 1 << 0, - /** Alexa is Speaking */ - ALEXA_SPEAKING = 1 << 1, - /** Alexa is Listening */ - ALEXA_LISTENING = 1 << 2, - /** Alexa is Idle */ - ALEXA_IDLE = 1 << 3, + /** Voice Assistant is Thinking */ + VA_THINKING = 1 << 0, + /** Voice Assistant is Speaking */ + VA_SPEAKING = 1 << 1, + /** Voice Assistant is Listening */ + VA_LISTENING = 1 << 2, + /** Voice Assistant is Idle */ + VA_IDLE = 1 << 3, /** End of current states markers */ - ALEXA_END_STATES = 1 << 4, - /** Alexa diaglog state Max*/ - ALEXA_DIALOG_STATES_MAX, -} alexa_dialog_states_t; + VA_END_STATES = 1 << 4, + /** Voice Assistant diaglog state Max*/ + VA_DIALOG_STATES_MAX, +} va_dialog_states_t; -/** Alexa Mute States +/** Voice Assistant Mute States */ typedef enum { /** Mute is disabled */ - ALEXA_MUTE_DISABLE = (ALEXA_DIALOG_STATES_MAX + 1), + VA_MUTE_DISABLE = (VA_DIALOG_STATES_MAX + 1), /** Mute is enabled */ - ALEXA_MUTE_ENABLE, -} alexa_mute_state_t; + VA_MUTE_ENABLE, +} va_mute_state_t; -/** Speaker Alexa Mute States +/** Speaker Voice Assistant Mute States */ typedef enum { /** Speaker Mute is disabled */ - ALEXA_SPEAKER_MUTE_ENABLE = (ALEXA_MUTE_ENABLE + 1), + VA_SPEAKER_MUTE_ENABLE = (VA_MUTE_ENABLE + 1), /** Speaker Mute is enabled */ - ALEXA_SPEAKER_MUTE_DISABLE, -} alexa_speaker_mute_state_t; + VA_SPEAKER_MUTE_DISABLE, +} va_speaker_mute_state_t; /** Set volume */ typedef enum { /** Set volume */ - ALEXA_SET_VOLUME = (ALEXA_SPEAKER_MUTE_DISABLE + 1), -} alexa_set_volume_t; + VA_SET_VOLUME = (VA_SPEAKER_MUTE_DISABLE + 1), +} va_set_volume_t; /** Alexa Alert Types */ @@ -78,59 +74,59 @@ typedef enum { } alexa_alert_state_t; /* - * Alexa resample i2s parameters + * Voice Assistant resample i2s parameters */ typedef struct { - int alexa_resample_freq; - int alexa_resample_ch; -} alexa_resample_param_t; + int va_resample_freq; + int va_resample_ch; +} va_resample_param_t; /** Dialog State callback * * This callback is executed when the dialog state changes. Please - * refer to \ref alexa_dialog_state_t for the various dialog + * refer to \ref va_dialog_state_t for the various dialog * states. The application is expected to show any LED indicators as - * desired for this Alexa state. + * desired for this Voice Assistant state. * * \note The callback may be called from different thread contexts - * from with the Alexa submodule. It is advised that this callback + * from with the Voice Assistant submodule. It is advised that this callback * doesn't take much stack or time during execution. * - * \param[in] alexa_states The current dialog state + * \param[in] va_states The current dialog state */ -void alexa_app_dialog_states(alexa_dialog_states_t alexa_states); +void va_app_dialog_states(va_dialog_states_t va_states); /** Stop Speech callback * - * This callback is executed when Alexa wishes to stop the microphone + * This callback is executed when the voice assistant wishes to stop the microphone * capture. The application is expected to stop the microphone and * stop streaming microphone data to AVS. */ -int alexa_app_speech_stop(); +int va_app_speech_stop(); /** Start Speech callback * - * This callback is executed when Alexa wishes to start the microphone + * This callback is executed when the voice assistant wishes to start the microphone * capture. The application is expected to start the microphone and * start streaming micro-phone data to AVS. This call is typically - * called in response to an ExpectSpeech directive, where Alexa queries + * called in response to an ExpectSpeech directive, where the voice assistant queries * the user for additional information. */ -int alexa_app_speech_start(); +int va_app_speech_start(); /** Set Volume callback * - * This callback is executed when Alexa wishes to change the output + * This callback is executed when the voice assistant wishes to change the output * volume of the speaker. */ -int alexa_app_set_volume(int vol); +int va_app_set_volume(int vol); /** Set Mute callback * - * This callback is executed when Alexa wishes to toggle the mute + * This callback is executed when the voice assistant wishes to toggle the mute * state of the speaker. */ -int alexa_app_set_mute(alexa_mute_state_t alexa_mute_state); +int va_app_set_mute(va_mute_state_t va_mute_state); /** Set Alert callback * @@ -143,11 +139,6 @@ int alexa_app_raise_alert(alexa_alert_types_t alexa_alert_type, alexa_alert_stat *This is a callback function to resample the playback signal which is coming from sys_playback *The resampled signal is then passed to the codec/dsp/amplifier */ -int alexa_app_playback_data(alexa_resample_param_t *alexa_resample_param, void *buf, ssize_t len); - - -#ifdef __cplusplus -} -#endif +int va_app_playback_data(va_resample_param_t *va_resample_param, void *buf, ssize_t len); -#endif /* _ALEXA_APP_CB_H_ */ +#endif /* _VOICE_ASSISTANT_APP_CB_H_ */ diff --git a/components/esp-alexa/lib/libalexa.a b/components/esp-voice-assistant/lib/libalexa.a similarity index 89% rename from components/esp-alexa/lib/libalexa.a rename to components/esp-voice-assistant/lib/libalexa.a index 32726dd..b1bf3b5 100644 Binary files a/components/esp-alexa/lib/libalexa.a and b/components/esp-voice-assistant/lib/libalexa.a differ diff --git a/components/esp-voice-assistant/lib/libdialogflow.a b/components/esp-voice-assistant/lib/libdialogflow.a new file mode 100644 index 0000000..e50337c Binary files /dev/null and b/components/esp-voice-assistant/lib/libdialogflow.a differ diff --git a/components/esp-voice-assistant/lib/libgva.a b/components/esp-voice-assistant/lib/libgva.a new file mode 100644 index 0000000..6d575e9 Binary files /dev/null and b/components/esp-voice-assistant/lib/libgva.a differ diff --git a/components/esp_httpd_ota/src/esp_httpd_ota.c b/components/esp_httpd_ota/src/esp_httpd_ota.c index 66638e7..7b19458 100644 --- a/components/esp_httpd_ota/src/esp_httpd_ota.c +++ b/components/esp_httpd_ota/src/esp_httpd_ota.c @@ -1,6 +1,5 @@ #include #include -#include #include #include #include diff --git a/components/misc/avs_nvs_utils.c b/components/misc/avs_nvs_utils.c index a52d80d..a72f4c3 100644 --- a/components/misc/avs_nvs_utils.c +++ b/components/misc/avs_nvs_utils.c @@ -50,6 +50,7 @@ struct nvs_ops_params { ERASE, } op; enum { + I8, STR, BLOB, INVALID, @@ -83,6 +84,8 @@ static void nvs_task(void *arg) err = nvs_set_blob(handle, params->key, params->val_buf, *(params->buf_size)); } else if (params->type == STR) { err = nvs_set_str(handle, params->key, (char *)params->val_buf); + } else if (params->type == I8) { + err = nvs_set_i8(handle, params->key, *(params->val_buf)); } if (err != ESP_OK) { ESP_LOGE(TAG, "Error setting value: %s", params->key); @@ -93,6 +96,8 @@ static void nvs_task(void *arg) err = nvs_get_blob(handle, params->key, params->val_buf, params->buf_size); } else if (params->type == STR) { err = nvs_get_str(handle, params->key, (char *)params->val_buf, params->buf_size); + } else if (params->type == I8) { + err = nvs_get_i8(handle, params->key, (int8_t *)params->val_buf); } if (err != ESP_OK) { ESP_LOGI(TAG, "No value set for: %s", params->key); @@ -171,6 +176,38 @@ esp_err_t avs_nvs_get_str(const char *ns, const char *key, char *val_buf, size_t } } +esp_err_t avs_nvs_set_i8(const char *ns, const char *key, uint8_t val_buf) +{ + struct nvs_ops_params tp = {ns, key, &val_buf, NULL, SET, I8, xTaskGetCurrentTaskHandle()}; + /* This will create a thread, do NVS operation and complete the thread */ + TaskHandle_t nvs_task_handle; + xTaskCreate(nvs_task, "nvs_task", NVS_TASK_STACK_SIZE, &tp, CONFIG_ESP32_PTHREAD_TASK_PRIO_DEFAULT, &nvs_task_handle); + uint32_t result; + /* Wait for operation to complete */ + xTaskNotifyWait(0, 0, &result, portMAX_DELAY); + if ((int)result != ESP_OK) { + return ESP_FAIL; + } else { + return ESP_OK; + } +} + +esp_err_t avs_nvs_get_i8(const char *ns, const char *key, uint8_t *val_buf) +{ + struct nvs_ops_params tp = {ns, key, (uint8_t *)val_buf, NULL, GET, I8, xTaskGetCurrentTaskHandle()}; + /* This will create a thread, do NVS operation and complete the thread */ + TaskHandle_t nvs_task_handle; + xTaskCreate(nvs_task, "nvs_task", NVS_TASK_STACK_SIZE, &tp, CONFIG_ESP32_PTHREAD_TASK_PRIO_DEFAULT, &nvs_task_handle); + uint32_t result; + /* Wait for operation to complete */ + xTaskNotifyWait(0, 0, &result, portMAX_DELAY); + if ((int)result != ESP_OK) { + return ESP_FAIL; + } else { + return ESP_OK; + } +} + esp_err_t avs_nvs_flash_erase() { struct nvs_ops_params tp = {NULL, NULL, NULL, 0, ERASE, INVALID, xTaskGetCurrentTaskHandle()}; diff --git a/components/misc/avs_nvs_utils.h b/components/misc/avs_nvs_utils.h index 84364e4..a8ab473 100644 --- a/components/misc/avs_nvs_utils.h +++ b/components/misc/avs_nvs_utils.h @@ -29,4 +29,6 @@ esp_err_t avs_nvs_get_blob(const char *ns, const char *key, uint8_t *val_buf, si esp_err_t avs_nvs_set_blob(const char *ns, const char *key, uint8_t *val_buf, size_t buf_size); esp_err_t avs_nvs_get_str(const char *ns, const char *key, char *val_buf, size_t *buf_size); esp_err_t avs_nvs_set_str(const char *ns, const char *key, char *val_buf); +esp_err_t avs_nvs_get_i8(const char *ns, const char *key, uint8_t *val_buf); +esp_err_t avs_nvs_set_i8(const char *ns, const char *key, uint8_t val_buf); esp_err_t avs_nvs_flash_erase(); diff --git a/components/misc/diag_cli.c b/components/misc/diag_cli.c deleted file mode 100644 index 90ae063..0000000 --- a/components/misc/diag_cli.c +++ /dev/null @@ -1,341 +0,0 @@ -/* - * ESPRESSIF MIT License - * - * Copyright (c) 2018 - * - * Permission is hereby granted for use on all ESPRESSIF SYSTEMS products, in which case, - * it is free of charge, to any person obtaining a copy of this software and associated - * documentation files (the "Software"), to deal in the Software without restriction, including - * without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, - * and/or sell copies of the Software, and to permit persons to whom the Software is furnished - * to do so, subject to the following conditions: - * - * The above copyright notice and this permission notice shall be included in all copies or - * substantial portions of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS - * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR - * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER - * IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN - * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - * - */ -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -static const char *TAG = "diag"; -static int task_dump_cli_handler(int argc, char *argv[]) -{ - int num_of_tasks = uxTaskGetNumberOfTasks(); - TaskStatus_t *task_array = va_mem_alloc(num_of_tasks * sizeof( TaskStatus_t ), VA_MEM_EXTERNAL); - if (!task_array) { - ESP_LOGE(TAG, "Memory not allocated for task list."); - } - num_of_tasks = uxTaskGetSystemState(task_array, num_of_tasks, NULL); - printf("Name\tNumber\tPriority\tStackWaterMark\n"); - for (int i = 0; i < num_of_tasks; i++) { - printf("%16s\t%d\t%d\t%d\n", - task_array[i].pcTaskName, - task_array[i].xTaskNumber, - task_array[i].uxCurrentPriority, - task_array[i].usStackHighWaterMark); - } - va_mem_free(task_array); - return 0; -} - -static int va_mem_dump_cli_handler(int argc, char *argv[]) -{ - printf("Description\tInternal\tSPIRAM\n"); - printf("Current Free Memory\t%d\t%d\n", - heap_caps_get_free_size(MALLOC_CAP_8BIT) - heap_caps_get_free_size(MALLOC_CAP_SPIRAM), - heap_caps_get_free_size(MALLOC_CAP_SPIRAM)); - printf("Largest Free Block\t%d\t%d\n", - heap_caps_get_largest_free_block(MALLOC_CAP_8BIT | MALLOC_CAP_INTERNAL), - heap_caps_get_largest_free_block(MALLOC_CAP_SPIRAM)); - printf("Min. Ever Free Size\t%d\t%d\n", - heap_caps_get_minimum_free_size(MALLOC_CAP_8BIT | MALLOC_CAP_INTERNAL), - heap_caps_get_minimum_free_size(MALLOC_CAP_SPIRAM)); - return 0; -} - -static void hex_dump(uint8_t *data, int length) -{ - for (int i = 0; i < length; i++) { - printf("0x%02x ", data[i]); - if (i && ((i % 16) == 0)) { - printf("\n"); - } - } - printf("\n"); -} - -static int nvs_get_cli_handler(int argc, char *argv[]) -{ - const char *namespace = argv[1]; - const char *variable = argv[2]; - const char *type = argv[3]; - size_t val_length = 0; - if (argc != 4) { - printf("Incorrect arguments\n"); - return 0; - } - if (strcmp(type, "string") == 0) { - avs_nvs_get_str(namespace, variable, NULL, &val_length); - if (val_length == 0) { - printf("Variable with 0 length\n"); - return 0; - } else { - char *value = va_mem_alloc(val_length, VA_MEM_EXTERNAL); - if (value == NULL) { - printf("Memory allocation failed"); - return 0; - } - avs_nvs_get_str(namespace, variable, value, &val_length); - printf("%s\n", value); - va_mem_free(value); - } - } else if (strcmp(type, "blob") == 0) { - avs_nvs_get_blob(namespace, variable, NULL, &val_length); - if (val_length == 0) { - printf("Variable with 0 length\n"); - return 0; - } else { - uint8_t *value = va_mem_alloc(val_length, VA_MEM_EXTERNAL); - if (value == NULL) { - printf("Memory allocation failed"); - return 0; - } - avs_nvs_get_blob(namespace, variable, value, &val_length); - hex_dump(value, val_length); - va_mem_free(value); - } - } else { - printf("Incorrect argument\n"); - } - return 0; -} - -static int nvs_set_cli_handler(int argc, char *argv[]) -{ - const char *namespace = argv[1]; - const char *variable = argv[2]; - const char *type = argv[3]; - char *value = argv[4]; - - if (argc != 5) { - printf("Incorrect arguments\n"); - return 0; - } - if (strcmp(type, "string") == 0) { - avs_nvs_set_str(namespace, variable, value); - } else if (strcmp(type, "blob") == 0) { - printf("Not yet supported\n"); - } else { - printf("Incorrect argument\n"); - } - return 0; -} - -static int nvs_erase_cli_handler(int argc, char *argv[]) -{ - avs_nvs_flash_erase(); - return 0; -} - -static int reboot_cli_handler(int argc, char *argv[]) -{ - esp_restart(); - return 0; -} - -static int heap_trace_records; -static heap_trace_record_t *heap_trace_records_buf; -static void cli_heap_trace_start() -{ - if (!heap_trace_records_buf) { - heap_trace_records_buf = va_mem_alloc(heap_trace_records * sizeof(heap_trace_record_t), VA_MEM_INTERNAL); - if (!heap_trace_records_buf) { - printf("Failed to allocate records buffer\n"); - return; - } - if (heap_trace_init_standalone(heap_trace_records_buf, heap_trace_records) != ESP_OK) { - printf("Failed to initialise tracing\n"); - goto error1; - } - } - if (heap_trace_start(HEAP_TRACE_LEAKS) != ESP_OK) { - printf("Failed to start heap trace\n"); - goto error2; - } - return; - error2: - heap_trace_init_standalone(NULL, 0); - error1: - va_mem_free(heap_trace_records_buf); - heap_trace_records_buf = NULL; - return; -} - -static void cli_heap_trace_stop() -{ - if (!heap_trace_records_buf) { - printf("Tracing not started?\n"); - return; - } - heap_trace_stop(); - heap_trace_dump(); - heap_trace_init_standalone(NULL, 0); - va_mem_free(heap_trace_records_buf); - heap_trace_records_buf = NULL; -} - -static int heap_trace_cli_handler(int argc, char *argv[]) -{ -#ifndef CONFIG_HEAP_TRACING - printf("To use this utility enable: Component config --> Heap memory debugging --> Enable heap tracing\n"); - return 0; -#endif - if (argc < 2) { - printf("Incorrect arguments\n"); - return 0; - } - if (strcmp(argv[1], "start") == 0) { -#define DEFAULT_HEAP_TRACE_RECORDS 100 - if (argc != 3) { - heap_trace_records = DEFAULT_HEAP_TRACE_RECORDS; - } else { - heap_trace_records = atoi(argv[2]); - } - printf("Using a buffer to trace %d records\n", heap_trace_records); - cli_heap_trace_start(); - } else if (strcmp(argv[1], "stop") == 0) { - cli_heap_trace_stop(); - } else { - printf("Invalid argument:%s:\n", argv[1]); - } - return 0; -} - -static int button_pressed_handler(int argc, char *argv[]) -{ - enum playback_controller_feature feature; - enum playback_controller_action action; - - if (argc < 2) { - printf("Incorrect arguments\n"); - return 0; - } - - if (strcmp(argv[1], "play") == 0) { - playback_controller_notify_play(); - } else if (strcmp(argv[1], "pause") == 0) { - playback_controller_notify_pause(); - } else if (strcmp(argv[1], "next") == 0) { - playback_controller_notify_next(); - } else if (strcmp(argv[1], "previous") == 0) { - playback_controller_notify_previous(); - } else if (strcmp(argv[1], "forward") == 0) { - playback_controller_notify_skip_forward(); - } else if (strcmp(argv[1], "backward") == 0) { - playback_controller_notify_skip_backward(); - } else if (strcmp(argv[1], "feature") == 0) { - if (argc < 4) { - printf("Incorrect arguments\n"); - return 0; - } - if (strcmp(argv[2], "shuffle") == 0) { - feature = PLAYBACK_CONTROLLER_SHUFFLE; - } else if (strcmp(argv[2], "loop") == 0) { - feature = PLAYBACK_CONTROLLER_LOOP; - } else if (strcmp(argv[2], "repeat") == 0) { - feature = PLAYBACK_CONTROLLER_REPEAT; - } else if (strcmp(argv[2], "thumbsup") == 0) { - feature = PLAYBACK_CONTROLLER_THUMBSUP; - } else if (strcmp(argv[2], "thumbsdown") == 0) { - feature = PLAYBACK_CONTROLLER_THUMBSDOWN; - } else { - printf("Incorrect feature\n"); - return 0; - } - if (strcmp(argv[3], "select") == 0) { - action = PLAYBACK_CONTROLLER_SELECT; - } else if (strcmp(argv[3], "deselect") == 0) { - action = PLAYBACK_CONTROLLER_DESELECT; - } else { - printf("Incorrect action\n"); - return 0; - } - playback_controller_notify_feature(feature, action); - } else { - printf("Incorrect arguments\n"); - return 0; - } - return 0; -} - -static esp_console_cmd_t diag_cmds[] = { - { - .command = "mem-dump", - .help = "", - .func = va_mem_dump_cli_handler, - }, - { - .command = "task-dump", - .help = "", - .func = task_dump_cli_handler, - }, - { - .command = "nvs-get", - .help = " ", - .func = nvs_get_cli_handler, - }, - { - .command = "nvs-set", - .help = " ", - .func = nvs_set_cli_handler, - }, - { - .command = "nvs-erase", - .help = " ", - .func = nvs_erase_cli_handler, - }, - { - .command = "reboot", - .help = " ", - .func = reboot_cli_handler, - }, - { - .command = "heap-trace", - .help = " [trace-buf-size]", - .func = heap_trace_cli_handler, - }, - { - .command = "button", - .help = " ", - .func = button_pressed_handler, - }, -}; - -int diag_register_cli() -{ - int cmds_num = sizeof(diag_cmds) / sizeof(esp_console_cmd_t); - int i; - for (i = 0; i < cmds_num; i++) { - printf("Registering command: %s \n", diag_cmds[i].command); - esp_console_cmd_register(&diag_cmds[i]); - } - return 0; -} - diff --git a/components/misc/str_utils.c b/components/misc/str_utils.c index a317539..b7532f0 100644 --- a/components/misc/str_utils.c +++ b/components/misc/str_utils.c @@ -29,13 +29,11 @@ static const char *TAG = "[str_utils]"; -void str_create_or_append(char **current_data, const char *data, int size) +void blob_create_or_append(char **current_data, size_t current_size, const char *data, int size) { - int current_size = 0; if (*current_data == NULL) { *current_data = (char *) va_mem_alloc(size + 1, VA_MEM_EXTERNAL); } else { - current_size = strlen(*current_data); *current_data = (char *) va_mem_realloc(*current_data, current_size + size + 1, VA_MEM_EXTERNAL); } if (*current_data) { @@ -103,3 +101,8 @@ void estr_delete(estr_t *estr) va_mem_free(estr->buf); va_mem_free(estr); } + +char *estr_get_buf_ptr(estr_t *estr) +{ + return estr->buf; +} diff --git a/components/misc/str_utils.h b/components/misc/str_utils.h index 7e8e9ae..cbebe5a 100644 --- a/components/misc/str_utils.h +++ b/components/misc/str_utils.h @@ -25,6 +25,7 @@ #define _STR_UTILS_H_ #include +#include #include #define DEFAULT_REALLOC_BLOCK_SIZE 2000 @@ -38,6 +39,15 @@ typedef struct { estr_t *estr_new(size_t size, size_t realloc_block_size); void estr_delete(estr_t *estr); int estr_append(estr_t *estr, const char *str, ...); -void str_create_or_append(char **current_data, const char *data, int size); +void blob_create_or_append(char **current_data, size_t current_len, const char *data, int size); +char *estr_get_buf_ptr(estr_t *estr); +static inline void str_create_or_append(char **current_data, const char *data, int size) +{ + size_t current_len = 0; + if (*current_data) { + current_len = strlen(*current_data); + } + return blob_create_or_append(current_data, current_len, data, size); +} #endif /* _STR_UTILS_H_ */ diff --git a/components/misc/va_diag_cli.c b/components/misc/va_diag_cli.c new file mode 100644 index 0000000..8c85005 --- /dev/null +++ b/components/misc/va_diag_cli.c @@ -0,0 +1,175 @@ +/* + * ESPRESSIF MIT License + * + * Copyright (c) 2018 + * + * Permission is hereby granted for use on all ESPRESSIF SYSTEMS products, in which case, + * it is free of charge, to any person obtaining a copy of this software and associated + * documentation files (the "Software"), to deal in the Software without restriction, including + * without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the Software is furnished + * to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all copies or + * substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER + * IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + * + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static const char *TAG = "diag"; + +static void hex_dump(uint8_t *data, int length) +{ + for (int i = 0; i < length; i++) { + printf("0x%02x ", data[i]); + if (i && ((i % 16) == 0)) { + printf("\n"); + } + } + printf("\n"); +} + +static int nvs_get_cli_handler(int argc, char *argv[]) +{ + const char *namespace = argv[1]; + const char *variable = argv[2]; + const char *type = argv[3]; + size_t val_length = 0; + if (argc != 4) { + printf("Incorrect arguments\n"); + return 0; + } + if (strcmp(type, "string") == 0) { + avs_nvs_get_str(namespace, variable, NULL, &val_length); + if (val_length == 0) { + printf("Variable with 0 length\n"); + return 0; + } else { + char *value = va_mem_alloc(val_length, VA_MEM_EXTERNAL); + if (value == NULL) { + printf("Memory allocation failed"); + return 0; + } + avs_nvs_get_str(namespace, variable, value, &val_length); + printf("%s\n", value); + va_mem_free(value); + } + } else if (strcmp(type, "blob") == 0) { + avs_nvs_get_blob(namespace, variable, NULL, &val_length); + if (val_length == 0) { + printf("Variable with 0 length\n"); + return 0; + } else { + uint8_t *value = va_mem_alloc(val_length, VA_MEM_EXTERNAL); + if (value == NULL) { + printf("Memory allocation failed"); + return 0; + } + avs_nvs_get_blob(namespace, variable, value, &val_length); + hex_dump(value, val_length); + va_mem_free(value); + } + } else if (strcmp(type, "i8") == 0) { + int8_t value; + avs_nvs_get_i8(namespace, variable, (uint8_t *)&value); + printf("%d\n", value); + return 0; + } else { + printf("Incorrect argument\n"); + } + return 0; +} + +static int nvs_set_cli_handler(int argc, char *argv[]) +{ + const char *namespace = argv[1]; + const char *variable = argv[2]; + const char *type = argv[3]; + char *value = argv[4]; + + if (argc != 5) { + printf("Incorrect arguments\n"); + return 0; + } + if (strcmp(type, "string") == 0) { + avs_nvs_set_str(namespace, variable, value); + } else if (strcmp(type, "i8") == 0) { + uint8_t val; + val = atoi((const char *)value); + if (val == 0 && *value != 0x30) { + ESP_LOGE(TAG, "Invalid value"); + return -1; + } + avs_nvs_set_i8(namespace, variable, val); + } else if (strcmp(type, "blob") == 0) { + printf("Not yet supported\n"); + } else { + printf("Incorrect argument\n"); + } + return 0; +} + +static int nvs_erase_cli_handler(int argc, char *argv[]) +{ + avs_nvs_flash_erase(); + return 0; +} + +static int reboot_cli_handler(int argc, char *argv[]) +{ + esp_restart(); + return 0; +} + +static esp_console_cmd_t diag_cmds[] = { + { + .command = "nvs-get", + .help = " ", + .func = nvs_get_cli_handler, + }, + { + .command = "nvs-set", + .help = " ", + .func = nvs_set_cli_handler, + }, + { + .command = "nvs-erase", + .help = " ", + .func = nvs_erase_cli_handler, + }, + { + .command = "reboot", + .help = " ", + .func = reboot_cli_handler, + } +}; + +int va_diag_register_cli() +{ + diag_register_cli(); + int cmds_num = sizeof(diag_cmds) / sizeof(esp_console_cmd_t); + int i; + for (i = 0; i < cmds_num; i++) { + printf("Registering command: %s \n", diag_cmds[i].command); + esp_console_cmd_register(&diag_cmds[i]); + } + return 0; +} + diff --git a/components/misc/va_diag_cli.h b/components/misc/va_diag_cli.h new file mode 100644 index 0000000..f3d4a98 --- /dev/null +++ b/components/misc/va_diag_cli.h @@ -0,0 +1,26 @@ +/* + * ESPRESSIF MIT License + * + * Copyright (c) 2018 + * + * Permission is hereby granted for use on all ESPRESSIF SYSTEMS products, in which case, + * it is free of charge, to any person obtaining a copy of this software and associated + * documentation files (the "Software"), to deal in the Software without restriction, including + * without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the Software is furnished + * to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all copies or + * substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER + * IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + * + */ +#pragma once + +int va_diag_register_cli(); diff --git a/components/speech_recog/lib/libc_speech_features.a b/components/speech_recog/lib/libc_speech_features.a index 501a2ea..0c823d0 100644 Binary files a/components/speech_recog/lib/libc_speech_features.a and b/components/speech_recog/lib/libc_speech_features.a differ diff --git a/components/speech_recog/lib/libdl_lib.a b/components/speech_recog/lib/libdl_lib.a index 7f4fd83..d08671e 100644 Binary files a/components/speech_recog/lib/libdl_lib.a and b/components/speech_recog/lib/libdl_lib.a differ diff --git a/components/speech_recog/lib/libesp_wwe.a b/components/speech_recog/lib/libesp_wwe.a index be6bb73..5044ae8 100644 Binary files a/components/speech_recog/lib/libesp_wwe.a and b/components/speech_recog/lib/libesp_wwe.a differ diff --git a/components/speech_recog/lib/libnn_model.a b/components/speech_recog/lib/libnn_model.a index 854c5b8..e262588 100644 Binary files a/components/speech_recog/lib/libnn_model.a and b/components/speech_recog/lib/libnn_model.a differ diff --git a/components/speech_recog/lib/libspeech_recog.a b/components/speech_recog/lib/libspeech_recog.a index 528fd71..2249b15 100644 Binary files a/components/speech_recog/lib/libspeech_recog.a and b/components/speech_recog/lib/libspeech_recog.a differ diff --git a/components/misc/diag_cli.h b/components/utils/include/diag_cli.h similarity index 100% rename from components/misc/diag_cli.h rename to components/utils/include/diag_cli.h diff --git a/components/misc/scli.h b/components/utils/include/scli.h similarity index 100% rename from components/misc/scli.h rename to components/utils/include/scli.h diff --git a/components/utils/src/diag_cli.c b/components/utils/src/diag_cli.c new file mode 100644 index 0000000..5e89d33 --- /dev/null +++ b/components/utils/src/diag_cli.c @@ -0,0 +1,187 @@ +/* + * ESPRESSIF MIT License + * + * Copyright (c) 2018 + * + * Permission is hereby granted for use on all ESPRESSIF SYSTEMS products, in which case, + * it is free of charge, to any person obtaining a copy of this software and associated + * documentation files (the "Software"), to deal in the Software without restriction, including + * without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the Software is furnished + * to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all copies or + * substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS + * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR + * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER + * IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + * + */ +#include +#include +#include +#ifdef CONFIG_HEAP_TRACING +#include +#endif +#include +#include +#include +#include + +static const char *TAG = "diag"; +static int task_dump_cli_handler(int argc, char *argv[]) +{ + int num_of_tasks = uxTaskGetNumberOfTasks(); + TaskStatus_t *task_array = esp_audio_mem_calloc(1, num_of_tasks * sizeof(TaskStatus_t)); + if (!task_array) { + ESP_LOGE(TAG, "Memory not allocated for task list."); + return 0; + } + num_of_tasks = uxTaskGetSystemState(task_array, num_of_tasks, NULL); + printf("\tName\tNumber\tPriority\tStackWaterMark\n"); + for (int i = 0; i < num_of_tasks; i++) { + printf("%16s\t%d\t%d\t%d\n", + task_array[i].pcTaskName, + task_array[i].xTaskNumber, + task_array[i].uxCurrentPriority, + task_array[i].usStackHighWaterMark); + } + esp_audio_mem_free(task_array); + return 0; +} + +static int cpu_dump_cli_handler(int argc, char *argv[]) +{ +#ifndef CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS + printf("To use this utility enable: Component config --> FreeRTOS --> Enable FreeRTOS to collect run time stats\n"); +#else + char *buf = esp_audio_mem_calloc(1, 2 * 1024); + vTaskGetRunTimeStats(buf); + printf("Run Time Stats:\n%s\n", buf); + esp_audio_mem_free(buf); +#endif + return 0; +} + +static int mem_dump_cli_handler(int argc, char *argv[]) +{ + printf("\tDescription\tInternal\tSPIRAM\n"); + printf("Current Free Memory\t%d\t\t%d\n", + heap_caps_get_free_size(MALLOC_CAP_8BIT) - heap_caps_get_free_size(MALLOC_CAP_SPIRAM), + heap_caps_get_free_size(MALLOC_CAP_SPIRAM)); + printf("Largest Free Block\t%d\t\t%d\n", + heap_caps_get_largest_free_block(MALLOC_CAP_8BIT | MALLOC_CAP_INTERNAL), + heap_caps_get_largest_free_block(MALLOC_CAP_SPIRAM)); + printf("Min. Ever Free Size\t%d\t\t%d\n", + heap_caps_get_minimum_free_size(MALLOC_CAP_8BIT | MALLOC_CAP_INTERNAL), + heap_caps_get_minimum_free_size(MALLOC_CAP_SPIRAM)); + return 0; +} + +#ifdef CONFIG_HEAP_TRACING +static int heap_trace_records; +static heap_trace_record_t *heap_trace_records_buf; +static void cli_heap_trace_start() +{ + if (!heap_trace_records_buf) { + heap_trace_records_buf = malloc(heap_trace_records * sizeof(heap_trace_record_t)); + if (!heap_trace_records_buf) { + printf("Failed to allocate records buffer\n"); + return; + } + if (heap_trace_init_standalone(heap_trace_records_buf, heap_trace_records) != ESP_OK) { + printf("Failed to initialise tracing\n"); + goto error1; + } + } + if (heap_trace_start(HEAP_TRACE_LEAKS) != ESP_OK) { + printf("Failed to start heap trace\n"); + goto error2; + } + return; +error2: + heap_trace_init_standalone(NULL, 0); +error1: + free(heap_trace_records_buf); + heap_trace_records_buf = NULL; + return; +} + +static void cli_heap_trace_stop() +{ + if (!heap_trace_records_buf) { + printf("Tracing not started?\n"); + return; + } + heap_trace_stop(); + heap_trace_dump(); + heap_trace_init_standalone(NULL, 0); + free(heap_trace_records_buf); + heap_trace_records_buf = NULL; +} +#endif + +static int heap_trace_cli_handler(int argc, char *argv[]) +{ +#ifndef CONFIG_HEAP_TRACING + printf("To use this utility enable: Component config --> Heap memory debugging --> Enable heap tracing\n"); +#else + if (argc < 2) { + printf("Incorrect arguments\n"); + return 0; + } + if (strcmp(argv[1], "start") == 0) { +#define DEFAULT_HEAP_TRACE_RECORDS 200 + if (argc != 3) { + heap_trace_records = DEFAULT_HEAP_TRACE_RECORDS; + } else { + heap_trace_records = atoi(argv[2]); + } + printf("Using a buffer to trace %d records\n", heap_trace_records); + cli_heap_trace_start(); + } else if (strcmp(argv[1], "stop") == 0) { + cli_heap_trace_stop(); + } else { + printf("Invalid argument:%s:\n", argv[1]); + } +#endif + return 0; +} + +static esp_console_cmd_t diag_cmds[] = { + { + .command = "mem-dump", + .help = "", + .func = mem_dump_cli_handler, + }, + { + .command = "task-dump", + .help = "", + .func = task_dump_cli_handler, + }, + { + .command = "cpu-dump", + .help = "", + .func = cpu_dump_cli_handler, + }, + { + .command = "heap-trace", + .help = " [trace-buf-size]", + .func = heap_trace_cli_handler, + }, +}; + +int diag_register_cli() +{ + int cmds_num = sizeof(diag_cmds) / sizeof(esp_console_cmd_t); + int i; + for (i = 0; i < cmds_num; i++) { + printf("Registering command: %s \n", diag_cmds[i].command); + esp_console_cmd_register(&diag_cmds[i]); + } + return 0; +} diff --git a/components/misc/scli.c b/components/utils/src/scli.c similarity index 89% rename from components/misc/scli.c rename to components/utils/src/scli.c index 43be5bb..667f070 100644 --- a/components/misc/scli.c +++ b/components/utils/src/scli.c @@ -30,7 +30,7 @@ #include #include -#include +#include static TaskHandle_t cli_task; static int stop; @@ -96,25 +96,6 @@ static void scli_task(void *arg) vTaskDelete(NULL); } -static int test_cmd_handler(int argc, char** argv) -{ - printf("Hi There!\n"); - return 0; -} - -int test_cli_register() -{ - esp_console_cmd_t cmd; - memset(&cmd, 0, sizeof(cmd)); - - cmd.command = "test"; - cmd.help = ""; - cmd.func = test_cmd_handler; - - esp_console_cmd_register(&cmd); - return 0; -} - int scli_init() { static int cli_started; @@ -122,7 +103,7 @@ int scli_init() return 0; } #define SCLI_STACK_SIZE 7000 - StackType_t *task_stack = (StackType_t *)va_mem_alloc(SCLI_STACK_SIZE, VA_MEM_EXTERNAL); + StackType_t *task_stack = (StackType_t *)esp_audio_mem_calloc(1, SCLI_STACK_SIZE); static StaticTask_t task_buf; cli_task = xTaskCreateStatic(scli_task, "scli_cli", SCLI_STACK_SIZE, (void *) 0, 3, task_stack, &task_buf); if (cli_task == NULL) { diff --git a/examples/lyrat_alexa/Makefile b/examples/lyrat/Makefile similarity index 53% rename from examples/lyrat_alexa/Makefile rename to examples/lyrat/Makefile index cad1e50..abcb461 100644 --- a/examples/lyrat_alexa/Makefile +++ b/examples/lyrat/Makefile @@ -3,7 +3,16 @@ # project subdirectory. # +ifeq ("$(VOICE_ASSISTANT)","gva") +PROJECT_NAME := gva +CPPFLAGS += -DGVA +else ifeq ("$(VOICE_ASSISTANT)","dialogflow") +PROJECT_NAME := dialogflow +CPPFLAGS += -DDIALOGFLOW +else PROJECT_NAME := alexa +CPPFLAGS += -DALEXA +endif EXTRA_COMPONENT_DIRS += $(PROJECT_PATH)/../../components diff --git a/examples/lyrat_alexa/components/audio_board_lyrat/audio_board_lyrat.c b/examples/lyrat/components/audio_board_lyrat/audio_board_lyrat.c similarity index 100% rename from examples/lyrat_alexa/components/audio_board_lyrat/audio_board_lyrat.c rename to examples/lyrat/components/audio_board_lyrat/audio_board_lyrat.c diff --git a/examples/lyrat_alexa/components/audio_board_lyrat/component.mk b/examples/lyrat/components/audio_board_lyrat/component.mk similarity index 100% rename from examples/lyrat_alexa/components/audio_board_lyrat/component.mk rename to examples/lyrat/components/audio_board_lyrat/component.mk diff --git a/examples/lyrat_alexa/components/codec_es8388/component.mk b/examples/lyrat/components/codec_es8388/component.mk similarity index 100% rename from examples/lyrat_alexa/components/codec_es8388/component.mk rename to examples/lyrat/components/codec_es8388/component.mk diff --git a/examples/lyrat_alexa/components/codec_es8388/es8388.c b/examples/lyrat/components/codec_es8388/es8388.c similarity index 100% rename from examples/lyrat_alexa/components/codec_es8388/es8388.c rename to examples/lyrat/components/codec_es8388/es8388.c diff --git a/examples/lyrat_alexa/components/codec_es8388/es8388.h b/examples/lyrat/components/codec_es8388/es8388.h similarity index 100% rename from examples/lyrat_alexa/components/codec_es8388/es8388.h rename to examples/lyrat/components/codec_es8388/es8388.h diff --git a/examples/lyrat_alexa/components/codec_es8388/media_hal.c b/examples/lyrat/components/codec_es8388/media_hal.c similarity index 100% rename from examples/lyrat_alexa/components/codec_es8388/media_hal.c rename to examples/lyrat/components/codec_es8388/media_hal.c diff --git a/examples/lyrat_alexa/main/Kconfig.projbuild b/examples/lyrat/main/Kconfig.projbuild similarity index 100% rename from examples/lyrat_alexa/main/Kconfig.projbuild rename to examples/lyrat/main/Kconfig.projbuild diff --git a/examples/lyrat_alexa/main/app_dsp.c b/examples/lyrat/main/app_dsp.c similarity index 94% rename from examples/lyrat_alexa/main/app_dsp.c rename to examples/lyrat/main/app_dsp.c index 3e0c546..e9e063b 100644 --- a/examples/lyrat_alexa/main/app_dsp.c +++ b/examples/lyrat/main/app_dsp.c @@ -1,19 +1,22 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + #include #include #include +#include #include #include #include -#include "app_dsp.h" -#include -#include #include #include #include #include #include +#include "app_dsp.h" + #define DETECT_SAMP_RATE 16000UL #define SAMP_RATE 48000UL #define SAMP_BITS 16 @@ -29,6 +32,7 @@ static struct dsp_data { bool write_to_store; char pcm_store[PCM_SIZE]; int16_t data_buf[SAMPLE_SZ]; + bool dsp_inited; } dd; static media_hal_config_t media_hal_conf = { @@ -77,7 +81,7 @@ static ssize_t dsp_write_cb(void *h, void *data, int len, uint32_t wait) return sent_len; } -int alexa_app_speech_stop() +int va_app_speech_stop() { ESP_LOGI(TAG, "Sending stop command"); if (audio_stream_stop(&dd.read_i2s_stream->base) != 0) { @@ -88,8 +92,11 @@ int alexa_app_speech_stop() return ESP_OK; } -int alexa_app_speech_start() +int va_app_speech_start() { + if (!dd.dsp_inited) + return ESP_FAIL; + ESP_LOGI(TAG, "Sending start command"); if (audio_stream_start(&dd.read_i2s_stream->base) != 0) { ESP_LOGE(TAG, "Failed to start I2S audio stream"); @@ -100,6 +107,9 @@ int alexa_app_speech_start() void app_dsp_send_recognize() { + if (!dd.dsp_inited) + return; + ESP_LOGI(TAG, "Sending start command"); if (audio_stream_start(&dd.read_i2s_stream->base) != 0) { ESP_LOGE(TAG, "Failed to start I2S audio stream"); @@ -115,9 +125,6 @@ void app_dsp_reset() void app_dsp_init(void) { - ui_led_init(); - ui_button_init(); - i2s_stream_config_t i2s_cfg; memset(&i2s_cfg, 0, sizeof(i2s_cfg)); i2s_cfg.i2s_num = 0; @@ -148,4 +155,5 @@ void app_dsp_init(void) vTaskDelay(10/portTICK_RATE_MS); audio_stream_stop(&dd.read_i2s_stream->base); i2s_set_clk(I2S_NUM_0, SAMP_RATE, SAMP_BITS, I2S_CHANNEL_STEREO); + dd.dsp_inited = true; } diff --git a/examples/lyrat_alexa/main/app_dsp.h b/examples/lyrat/main/app_dsp.h similarity index 82% rename from examples/lyrat_alexa/main/app_dsp.h rename to examples/lyrat/main/app_dsp.h index 80d6ff7..221ce6b 100644 --- a/examples/lyrat_alexa/main/app_dsp.h +++ b/examples/lyrat/main/app_dsp.h @@ -4,14 +4,14 @@ #ifndef _APP_DSP_H_ #define _APP_DSP_H_ -#include +#include #ifdef __cplusplus extern "C" { #endif typedef enum dsp_called_states { - ALEXA_CAN_START = (ALEXA_END_STATES + 1), + VA_CAN_START = (VA_END_STATES + 1), } dsp_called_states_t; void app_dsp_init(void); diff --git a/examples/lyrat_alexa/main/app_lyrat_cb.c b/examples/lyrat/main/app_lyrat_cb.c similarity index 53% rename from examples/lyrat_alexa/main/app_lyrat_cb.c rename to examples/lyrat/main/app_lyrat_cb.c index 8a7ece3..3378300 100644 --- a/examples/lyrat_alexa/main/app_lyrat_cb.c +++ b/examples/lyrat/main/app_lyrat_cb.c @@ -4,11 +4,15 @@ #include #include #include -#include +#include #include #include #include +#if DIALOGFLOW +#include +#endif + #define I2S_PORT_NUM I2S_NUM_0 #define DES_BITS_PER_SAM 16 #define SRC_BITS_PER_SAM 16 @@ -26,30 +30,30 @@ static inline int heap_caps_get_free_size_sram() return heap_caps_get_free_size(MALLOC_CAP_8BIT) - heap_caps_get_free_size(MALLOC_CAP_SPIRAM); } -void alexa_app_dialog_states(alexa_dialog_states_t alexa_states) +void va_app_dialog_states(va_dialog_states_t va_state) { - ui_led_set(alexa_states); - ESP_LOGI(TAG, "Current mode is: %d", alexa_states); + ui_led_set(va_state); + ESP_LOGI(TAG, "Current mode is: %d", va_state); } -int alexa_app_set_volume(int vol) +int va_app_set_volume(int vol) { es8388_control_volume(vol); ESP_LOGI(TAG, "Volume changed to %d", vol); return 0; } -int alexa_app_set_mute(alexa_mute_state_t alexa_mute_state) +int va_app_set_mute(va_mute_state_t va_mute_state) { uint8_t current_vol; - if (alexa_mute_state == ALEXA_MUTE_ENABLE) { + if (va_mute_state == VA_MUTE_ENABLE) { es8388_set_mute(1); } else { es8388_set_mute(0); es8388_get_volume(¤t_vol); es8388_control_volume(current_vol); } - ESP_LOGI(TAG, "Mute: %d", alexa_mute_state); + ESP_LOGI(TAG, "Mute: %d", va_mute_state); return 0; } @@ -58,7 +62,7 @@ int alexa_app_raise_alert(alexa_alert_types_t alexa_alert_type, alexa_alert_stat switch (alexa_alert_type) { case ALEXA_ALERT_NOTIFICATION: //ui_led_set_alert(alexa_alert_type, alexa_alert_state); - //ui_led_set(ALEXA_IDLE); + //ui_led_set(VA_IDLE); break; default: @@ -67,18 +71,18 @@ int alexa_app_raise_alert(alexa_alert_types_t alexa_alert_type, alexa_alert_stat return 0; } -int alexa_app_playback_data(alexa_resample_param_t *alexa_resample_param, void *buf, ssize_t len) +int va_app_playback_data(va_resample_param_t *va_resample_param, void *buf, ssize_t len) { //printf("[resample-cb] %d spiram %d\n", heap_caps_get_free_size_sram(), heap_caps_get_free_size(MALLOC_CAP_SPIRAM)); memset(convert_buf, 0, BUF_SZ); - audio_resample_config_t resample = {0}; + static audio_resample_config_t resample = {0}; int current_convert_block_len; int convert_block_len = 0; int send_offset = 0; size_t sent_len = 0; int conv_len = 0; - if (alexa_resample_param->alexa_resample_ch == 1) { + if (va_resample_param->va_resample_ch == 1) { /* If mono recording, we need to up-sample, so need half the buffer empty, also uint16_t data*/ convert_block_len = CONVERT_BUF_SIZE / 4; } else { @@ -91,9 +95,9 @@ int alexa_app_playback_data(alexa_resample_param_t *alexa_resample_param, void * if (current_convert_block_len & 1) { printf("Odd bytes in up sampling data, this should be backed up\n"); } - conv_len = audio_resample((short *)((char *)buf + send_offset), (short *)convert_buf, alexa_resample_param->alexa_resample_freq, SAMPLING_RATE, - current_convert_block_len / 2, BUF_SZ, alexa_resample_param->alexa_resample_ch, &resample); - if (alexa_resample_param->alexa_resample_ch == 1) { + conv_len = audio_resample((short *)((char *)buf + send_offset), (short *)convert_buf, va_resample_param->va_resample_freq, SAMPLING_RATE, + current_convert_block_len / 2, BUF_SZ, va_resample_param->va_resample_ch, &resample); + if (va_resample_param->va_resample_ch == 1) { conv_len = audio_resample_up_channel((short *)convert_buf, (short *)convert_buf, SAMPLING_RATE, SAMPLING_RATE, conv_len, BUF_SZ, &resample); } len -= current_convert_block_len; @@ -133,3 +137,64 @@ int i2s_playback_init() ret = i2s_zero_dma_buffer(I2S_PORT_NUM); return ret; } + +#if DIALOGFLOW +void dialogflow_app_response_data(Google__Cloud__Dialogflow__V2beta1__StreamingDetectIntentResponse *response) +{ + int i; + bool end_conversation = false; + printf("The query text is: %s\n", response->query_result->query_text); + printf("The fulfillment text is: %s\n", response->query_result->fulfillment_text); + + Google__Protobuf__Struct *entities = response->query_result->parameters; + if (!entities) { + return; + } + + /* Diagnostics information can be used to mark the end of a conversation. Make sure that "Set this intent as end + * of conversation" is enabled under "Responses" tab in each intent of your Dialogflow agent + */ + Google__Protobuf__Struct *info = response->query_result->diagnostic_info; + if (info) { + for (i = 0; i < info->n_fields; i++) { + switch (info->fields[i]->value->kind_case) { + case GOOGLE__PROTOBUF__VALUE__KIND_BOOL_VALUE: + if (!strncmp(info->fields[i]->key, "end_conversation", strlen(info->fields[i]->key)) && + info->fields[i]->value->bool_value) { + end_conversation = true; + } + break; + default: + break; + } + } + } + + if (end_conversation) { + printf("Dialogflow conversation complete:\nNumber of entities: %d\n", entities->n_fields); + for (i = 0; i < entities->n_fields; i++) { + switch (entities->fields[i]->value->kind_case) { + case GOOGLE__PROTOBUF__VALUE__KIND_NUMBER_VALUE: + printf("[%d] %s : %f\n", i, entities->fields[i]->key, entities->fields[i]->value->number_value); + break; + case GOOGLE__PROTOBUF__VALUE__KIND_STRING_VALUE: + printf("[%d] %s : %s\n", i, entities->fields[i]->key, entities->fields[i]->value->string_value); + break; + case GOOGLE__PROTOBUF__VALUE__KIND_BOOL_VALUE: + printf("[%d] %s : %s\n", i, entities->fields[i]->key, entities->fields[i]->value->bool_value ? "true" : "false"); + break; + default: + printf("Unknown type %d: Please update this debug function\n", entities->fields[i]->value->kind_case); + break; + } + } + } else { + printf("Dialogflow conversation continued\n"); + } +} + +void dialogflow_app_query_text_transcript(char *text, bool is_final) +{ + printf("%s transcript is: %s\n", (is_final != 0 ? "Final" : "Interim"),text); +} +#endif diff --git a/examples/lyrat_alexa/main/app_main.c b/examples/lyrat/main/app_main.c similarity index 70% rename from examples/lyrat_alexa/main/app_main.c rename to examples/lyrat/main/app_main.c index 7623e16..6e150ba 100644 --- a/examples/lyrat_alexa/main/app_main.c +++ b/examples/lyrat/main/app_main.c @@ -1,7 +1,7 @@ // Copyright 2018 Espressif Systems (Shanghai) PTE LTD // All rights reserved. -/* Alexa Example +/* Voice Assistant Example This example code is in the Public Domain (or CC0 licensed, at your option.) @@ -21,25 +21,32 @@ #include #include +#include +#ifdef ALEXA +#include +#elif GVA +#include +#elif DIALOGFLOW +#include +#endif + #include #include -#include +#include #include -#include #include #include "app_dsp.h" #include "ui_led.h" +#include "ui_button.h" #define SOFTAP_SSID_PREFIX "ESP-Alexa-" -static const char *TAG = "alexa"; +static const char *TAG = "app_main"; static EventGroupHandle_t cm_event_group; const int CONNECTED_BIT = BIT0; -static alexa_config_t *alexa_cfg; - static esp_err_t event_handler(void *ctx, system_event_t *event) { conn_mgr_prov_event_handler(ctx, event); @@ -74,14 +81,21 @@ extern int i2s_playback_init(); int app_main() { - ESP_LOGI(TAG, "==== Alexa SDK version: %s ====", alexa_get_sdk_version()); - - alexa_cfg = va_mem_alloc(sizeof(alexa_config_t), VA_MEM_EXTERNAL); - if (!alexa_cfg) { - ESP_LOGE(TAG, "Failed to alloc alexa config"); + ESP_LOGI(TAG, "==== Voice Assistant SDK version: %s ====", va_get_sdk_version()); + +#ifdef ALEXA + alexa_config_t *va_cfg = va_mem_alloc(sizeof(alexa_config_t), VA_MEM_EXTERNAL); +#elif GVA + gva_config_t *va_cfg = va_mem_alloc(sizeof(gva_config_t), VA_MEM_EXTERNAL); +#elif DIALOGFLOW + dialogflow_config_t *va_cfg = va_mem_alloc(sizeof(dialogflow_config_t), VA_MEM_EXTERNAL); +#endif + + if (!va_cfg) { + ESP_LOGE(TAG, "Failed to alloc voice assistant config"); abort(); } - alexa_cfg->auth_delegate.type = auth_type_subsequent; + va_cfg->auth_delegate.type = auth_type_subsequent; // Initialize NVS esp_err_t ret = nvs_flash_init(); @@ -91,12 +105,14 @@ int app_main() } ESP_ERROR_CHECK( ret ); + ui_led_init(); + ui_button_init(); scli_init(); - diag_register_cli(); + va_diag_register_cli(); wifi_register_cli(); cm_event_group = xEventGroupCreate(); - tcpip_adapter_init(); + ESP_ERROR_CHECK(esp_event_loop_init(event_handler, NULL) ); bool provisioned = false; @@ -113,7 +129,7 @@ int app_main() conn_mgr_prov_t prov_type = conn_mgr_prov_mode_softap; prov_type.event_cb = alexa_conn_mgr_prov_cb; - prov_type.cb_user_data = (void *)alexa_cfg; + prov_type.cb_user_data = (void *)va_cfg; int security = 1; const char *pop = "abcd1234"; const char *service_key = ""; @@ -128,9 +144,22 @@ int app_main() i2s_playback_init(); app_dsp_init(); - if (alexa_init(alexa_cfg) != ESP_OK) { + +#ifdef ALEXA + ret = alexa_init(va_cfg); +#elif GVA + va_cfg->device_config.device_model = "device-model-default"; // Enter your model id (name) here + va_cfg->device_config.device_id = "device-id-default"; // Enter your (unique) device id here + ret = gva_init(va_cfg); +#elif DIALOGFLOW + va_cfg->device_config.project_name = "project-name-default"; // Enter your dialogflow project name here. + ret = dialogflow_init(va_cfg); +#endif + + if (ret != ESP_OK) { while(1) vTaskDelay(2); } - va_mem_free(alexa_cfg); + + va_mem_free(va_cfg); return ESP_OK; } diff --git a/examples/lyrat_alexa/main/component.mk b/examples/lyrat/main/component.mk similarity index 100% rename from examples/lyrat_alexa/main/component.mk rename to examples/lyrat/main/component.mk diff --git a/examples/lyrat_alexa/main/ui_button.c b/examples/lyrat/main/ui_button.c similarity index 99% rename from examples/lyrat_alexa/main/ui_button.c rename to examples/lyrat/main/ui_button.c index b35c331..c3f2129 100644 --- a/examples/lyrat_alexa/main/ui_button.c +++ b/examples/lyrat/main/ui_button.c @@ -8,7 +8,7 @@ #include #include #include -#include +#include #include #include "ui_button.h" #include diff --git a/examples/lyrat_alexa/main/ui_button.h b/examples/lyrat/main/ui_button.h similarity index 100% rename from examples/lyrat_alexa/main/ui_button.h rename to examples/lyrat/main/ui_button.h diff --git a/examples/lyrat_alexa/main/ui_led.c b/examples/lyrat/main/ui_led.c similarity index 83% rename from examples/lyrat_alexa/main/ui_led.c rename to examples/lyrat/main/ui_led.c index c6fa027..168ecba 100644 --- a/examples/lyrat_alexa/main/ui_led.c +++ b/examples/lyrat/main/ui_led.c @@ -6,17 +6,17 @@ #include #include #include -#include +#include static const char *UI_LED_TAG = "UI_LED"; -esp_err_t ui_led_set(int alexa_state) +esp_err_t ui_led_set(int va_state) { - if (alexa_state == ALEXA_LISTENING) { + if (va_state == VA_LISTENING) { gpio_set_level(LED_GPIO_PIN, 1); - } else if (alexa_state == LED_RESET){ + } else if (va_state == LED_RESET){ gpio_set_level(LED_BOOT_PIN, 1); - } else if (alexa_state == LED_OFF) { + } else if (va_state == LED_OFF) { gpio_set_level(LED_GPIO_PIN, 0); gpio_set_level(LED_BOOT_PIN, 0); } else { diff --git a/examples/lyrat_alexa/main/ui_led.h b/examples/lyrat/main/ui_led.h similarity index 91% rename from examples/lyrat_alexa/main/ui_led.h rename to examples/lyrat/main/ui_led.h index e52c142..518f3e2 100644 --- a/examples/lyrat_alexa/main/ui_led.h +++ b/examples/lyrat/main/ui_led.h @@ -1,7 +1,6 @@ // Copyright 2018 Espressif Systems (Shanghai) PTE LTD // All rights reserved. - #ifndef _UI_LED_H_ #define _UI_LED_H_ @@ -23,7 +22,7 @@ enum { LED_OFF, }; -esp_err_t ui_led_set(int alexa_state); +esp_err_t ui_led_set(int va_state); esp_err_t ui_led_init(); diff --git a/examples/lyrat_alexa/partitions.csv b/examples/lyrat/partitions.csv similarity index 100% rename from examples/lyrat_alexa/partitions.csv rename to examples/lyrat/partitions.csv diff --git a/examples/lyrat_alexa/sdkconfig.defaults b/examples/lyrat/sdkconfig.defaults similarity index 98% rename from examples/lyrat_alexa/sdkconfig.defaults rename to examples/lyrat/sdkconfig.defaults index 5cbf533..c9c809d 100644 --- a/examples/lyrat_alexa/sdkconfig.defaults +++ b/examples/lyrat/sdkconfig.defaults @@ -75,6 +75,7 @@ CONFIG_FREERTOS_USE_TRACE_FACILITY=y CONFIG_LWIP_MAX_SOCKETS=4 CONFIG_USE_ONLY_LWIP_SELECT=y CONFIG_LWIP_ETHARP_TRUST_IP_MAC=y +CONFIG_LWIP_DHCP_MAX_NTP_SERVERS=3 # # TCP diff --git a/examples/lyrat_alexa/README.md b/examples/lyrat_alexa/README.md deleted file mode 100644 index c07499f..0000000 --- a/examples/lyrat_alexa/README.md +++ /dev/null @@ -1,54 +0,0 @@ -# Prerequisites -Please prepare your host system with the toolchain. Please see http://esp-idf.readthedocs.io/en/latest/get-started/index.html for the host setup. - -# Supported Hardware -The current application supports ESP32 based LyraT v4.1, LyraT v4.2 and LyraT v4.3 - -# Prepare Images -If you already have the pre-compiled images flashed on the board, please jump to the next section (Device Configuration). - -## Clone all the repositories -``` -$ git clone --recursive https://github.com/espressif/esp-idf.git - -$ cd esp-idf; git checkout release/v3.1; cd .. - -$ git clone https://github.com/espressif/esp-avs-sdk.git -``` - -## Applying extraneous patches on esp-idf -``` -$ cd esp-idf - -$ git apply ../esp-avs-sdk/esp-idf-patches/memset-i2s-dma-buffers-zero.patch - -$ git apply ../esp-avs-sdk/esp-idf-patches/esp-tls-Add-support-for-global-CA-store.-All-mbedtls.patch -``` - -## Build and flash the project -``` -$ cd esp-avs-sdk/examples/lyrat_alexa - -$ export IDF_PATH=/path/to/esp-idf - -$ export ESPPORT=/dev/cu.SLAB_USBtoUART (or /dev/ttyUSB0 or /dev/ttyUSB1 on Linux or COMxx on MinGW) - -$ make -j 8 flash monitor -``` - -# Device Configuration -Android and iOS apps are available to configure WiFi credentials and associate user's Amazon account with the device. -* [Android](https://github.com/espressif/esp-avs-sdk/releases) -* [iOS](https://github.com/espressif/esp-idf-provisioning-ios/tree/versions/avs) - -# Demo -* Once the board boots up and successfully connects to the Wi-Fi network after provisioning, you will see a print "Alexa is ready", after which you can use "Rec" button on LyraT board for conversation. For Tap-to-Talk, press and release the button and speak. The green LED glows when the microphone is active. -* You can connect any external speaker/headphone with 3.5mm connector to PHONE JACK to listen to responses. -* you can now ask any command like: - * tell me a joke - * how is the weather? - * will it rain today? - * Sing a song - * Play TuneIn radio - * Set volume level to 7 -* Press and Hold "Mode" button for 3 seconds to reset the board to factory settings diff --git a/examples/lyrat_alexa_local_player_with_equalizer/README.md b/examples/lyrat_alexa_local_player_with_equalizer/README.md index c07499f..e8a1c4b 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/README.md +++ b/examples/lyrat_alexa_local_player_with_equalizer/README.md @@ -27,7 +27,7 @@ $ git apply ../esp-avs-sdk/esp-idf-patches/esp-tls-Add-support-for-global-CA-sto ## Build and flash the project ``` -$ cd esp-avs-sdk/examples/lyrat_alexa +$ cd esp-avs-sdk/examples/test/lyrat_alexa_local_player_with_equalizer $ export IDF_PATH=/path/to/esp-idf @@ -42,8 +42,12 @@ Android and iOS apps are available to configure WiFi credentials and associate u * [iOS](https://github.com/espressif/esp-idf-provisioning-ios/tree/versions/avs) # Demo -* Once the board boots up and successfully connects to the Wi-Fi network after provisioning, you will see a print "Alexa is ready", after which you can use "Rec" button on LyraT board for conversation. For Tap-to-Talk, press and release the button and speak. The green LED glows when the microphone is active. -* You can connect any external speaker/headphone with 3.5mm connector to PHONE JACK to listen to responses. +* Once the board boots up and successfully connects to the Wi-Fi network after provisioning, device will initialize Alexa and will start playing local audio from SD card. Below is the sequence of local playback: + * Play 1.mp3 for 10 seconds + * Play 2.mp3 (This will interrupt 1.mp3 and start playing 2.mp3) + * Enqueue 3.mp3 and 1.mp3 (This will be played after 2.mp3 completes playback) +* You can connect any external speaker/headphone with 3.5mm connector to PHONE JACK to listen to playback/Alexa responses. +* Local playback can be interrupted to talk to Alexa. You can press and release Tap-to-talk button and speak. The green LED glows when the microphone is active. * you can now ask any command like: * tell me a joke * how is the weather? @@ -51,4 +55,5 @@ Android and iOS apps are available to configure WiFi credentials and associate u * Sing a song * Play TuneIn radio * Set volume level to 7 +* After the response is completely played, device will resume local playback. * Press and Hold "Mode" button for 3 seconds to reset the board to factory settings diff --git a/examples/lyrat_alexa_local_player_with_equalizer/main/app_dsp.c b/examples/lyrat_alexa_local_player_with_equalizer/main/app_dsp.c index 3e0c546..889271c 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/main/app_dsp.c +++ b/examples/lyrat_alexa_local_player_with_equalizer/main/app_dsp.c @@ -1,3 +1,6 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + #include #include #include @@ -77,7 +80,7 @@ static ssize_t dsp_write_cb(void *h, void *data, int len, uint32_t wait) return sent_len; } -int alexa_app_speech_stop() +int va_app_speech_stop() { ESP_LOGI(TAG, "Sending stop command"); if (audio_stream_stop(&dd.read_i2s_stream->base) != 0) { @@ -88,7 +91,7 @@ int alexa_app_speech_stop() return ESP_OK; } -int alexa_app_speech_start() +int va_app_speech_start() { ESP_LOGI(TAG, "Sending start command"); if (audio_stream_start(&dd.read_i2s_stream->base) != 0) { diff --git a/examples/lyrat_alexa_local_player_with_equalizer/main/app_dsp.h b/examples/lyrat_alexa_local_player_with_equalizer/main/app_dsp.h index 80d6ff7..221ce6b 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/main/app_dsp.h +++ b/examples/lyrat_alexa_local_player_with_equalizer/main/app_dsp.h @@ -4,14 +4,14 @@ #ifndef _APP_DSP_H_ #define _APP_DSP_H_ -#include +#include #ifdef __cplusplus extern "C" { #endif typedef enum dsp_called_states { - ALEXA_CAN_START = (ALEXA_END_STATES + 1), + VA_CAN_START = (VA_END_STATES + 1), } dsp_called_states_t; void app_dsp_init(void); diff --git a/examples/lyrat_alexa_local_player_with_equalizer/main/app_lyrat_cb.c b/examples/lyrat_alexa_local_player_with_equalizer/main/app_lyrat_cb.c index 8a7ece3..ffe917b 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/main/app_lyrat_cb.c +++ b/examples/lyrat_alexa_local_player_with_equalizer/main/app_lyrat_cb.c @@ -4,7 +4,7 @@ #include #include #include -#include +#include #include #include #include @@ -26,30 +26,30 @@ static inline int heap_caps_get_free_size_sram() return heap_caps_get_free_size(MALLOC_CAP_8BIT) - heap_caps_get_free_size(MALLOC_CAP_SPIRAM); } -void alexa_app_dialog_states(alexa_dialog_states_t alexa_states) +void va_app_dialog_states(va_dialog_states_t va_state) { - ui_led_set(alexa_states); - ESP_LOGI(TAG, "Current mode is: %d", alexa_states); + ui_led_set(va_state); + ESP_LOGI(TAG, "Current mode is: %d", va_state); } -int alexa_app_set_volume(int vol) +int va_app_set_volume(int vol) { es8388_control_volume(vol); ESP_LOGI(TAG, "Volume changed to %d", vol); return 0; } -int alexa_app_set_mute(alexa_mute_state_t alexa_mute_state) +int va_app_set_mute(va_mute_state_t va_mute_state) { uint8_t current_vol; - if (alexa_mute_state == ALEXA_MUTE_ENABLE) { + if (va_mute_state == VA_MUTE_ENABLE) { es8388_set_mute(1); } else { es8388_set_mute(0); es8388_get_volume(¤t_vol); es8388_control_volume(current_vol); } - ESP_LOGI(TAG, "Mute: %d", alexa_mute_state); + ESP_LOGI(TAG, "Mute: %d", va_mute_state); return 0; } @@ -58,7 +58,7 @@ int alexa_app_raise_alert(alexa_alert_types_t alexa_alert_type, alexa_alert_stat switch (alexa_alert_type) { case ALEXA_ALERT_NOTIFICATION: //ui_led_set_alert(alexa_alert_type, alexa_alert_state); - //ui_led_set(ALEXA_IDLE); + //ui_led_set(VA_IDLE); break; default: @@ -67,7 +67,7 @@ int alexa_app_raise_alert(alexa_alert_types_t alexa_alert_type, alexa_alert_stat return 0; } -int alexa_app_playback_data(alexa_resample_param_t *alexa_resample_param, void *buf, ssize_t len) +int va_app_playback_data(va_resample_param_t *va_resample_param, void *buf, ssize_t len) { //printf("[resample-cb] %d spiram %d\n", heap_caps_get_free_size_sram(), heap_caps_get_free_size(MALLOC_CAP_SPIRAM)); memset(convert_buf, 0, BUF_SZ); @@ -78,7 +78,7 @@ int alexa_app_playback_data(alexa_resample_param_t *alexa_resample_param, void * size_t sent_len = 0; int conv_len = 0; - if (alexa_resample_param->alexa_resample_ch == 1) { + if (va_resample_param->va_resample_ch == 1) { /* If mono recording, we need to up-sample, so need half the buffer empty, also uint16_t data*/ convert_block_len = CONVERT_BUF_SIZE / 4; } else { @@ -91,9 +91,9 @@ int alexa_app_playback_data(alexa_resample_param_t *alexa_resample_param, void * if (current_convert_block_len & 1) { printf("Odd bytes in up sampling data, this should be backed up\n"); } - conv_len = audio_resample((short *)((char *)buf + send_offset), (short *)convert_buf, alexa_resample_param->alexa_resample_freq, SAMPLING_RATE, - current_convert_block_len / 2, BUF_SZ, alexa_resample_param->alexa_resample_ch, &resample); - if (alexa_resample_param->alexa_resample_ch == 1) { + conv_len = audio_resample((short *)((char *)buf + send_offset), (short *)convert_buf, va_resample_param->va_resample_freq, SAMPLING_RATE, + current_convert_block_len / 2, BUF_SZ, va_resample_param->va_resample_ch, &resample); + if (va_resample_param->va_resample_ch == 1) { conv_len = audio_resample_up_channel((short *)convert_buf, (short *)convert_buf, SAMPLING_RATE, SAMPLING_RATE, conv_len, BUF_SZ, &resample); } len -= current_convert_block_len; diff --git a/examples/lyrat_alexa_local_player_with_equalizer/main/app_main.c b/examples/lyrat_alexa_local_player_with_equalizer/main/app_main.c index 2705a01..08c9440 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/main/app_main.c +++ b/examples/lyrat_alexa_local_player_with_equalizer/main/app_main.c @@ -21,10 +21,12 @@ #include #include +#include +#include + #include #include -#include -#include +#include #include #include "app_dsp.h" @@ -34,13 +36,11 @@ #define SOFTAP_SSID_PREFIX "ESP-Alexa-" -static const char *TAG = "alexa"; +static const char *TAG = "app_main"; static EventGroupHandle_t cm_event_group; const int CONNECTED_BIT = BIT0; -static alexa_config_t *alexa_cfg; - static esp_err_t event_handler(void *ctx, system_event_t *event) { conn_mgr_prov_event_handler(ctx, event); @@ -75,14 +75,15 @@ extern int i2s_playback_init(); int app_main() { - ESP_LOGI(TAG, "==== Alexa SDK version: %s ====", alexa_get_sdk_version()); + ESP_LOGI(TAG, "==== Voice Assistant SDK version: %s ====", va_get_sdk_version()); - alexa_cfg = va_mem_alloc(sizeof(alexa_config_t), VA_MEM_EXTERNAL); - if (!alexa_cfg) { - ESP_LOGE(TAG, "Failed to alloc alexa config"); + alexa_config_t *va_cfg = va_mem_alloc(sizeof(alexa_config_t), VA_MEM_EXTERNAL); + + if (!va_cfg) { + ESP_LOGE(TAG, "Failed to alloc voice assistant config"); abort(); } - alexa_cfg->auth_delegate.type = auth_type_subsequent; + va_cfg->auth_delegate.type = auth_type_subsequent; // Initialize NVS esp_err_t ret = nvs_flash_init(); @@ -93,7 +94,7 @@ int app_main() ESP_ERROR_CHECK( ret ); scli_init(); - diag_register_cli(); + va_diag_register_cli(); cm_event_group = xEventGroupCreate(); tcpip_adapter_init(); @@ -113,7 +114,7 @@ int app_main() conn_mgr_prov_t prov_type = conn_mgr_prov_mode_softap; prov_type.event_cb = alexa_conn_mgr_prov_cb; - prov_type.cb_user_data = (void *)alexa_cfg; + prov_type.cb_user_data = (void *)va_cfg; int security = 1; const char *pop = "abcd1234"; const char *service_key = ""; @@ -128,13 +129,16 @@ int app_main() i2s_playback_init(); app_dsp_init(); - if (alexa_init(alexa_cfg) != ESP_OK) { + + ret = alexa_init(va_cfg); + + if (ret != ESP_OK) { while(1) vTaskDelay(2); } - va_mem_free(alexa_cfg); local_player_init(); equalizer_init(); + va_mem_free(va_cfg); return ESP_OK; } diff --git a/examples/lyrat_alexa_local_player_with_equalizer/main/equalizer.c b/examples/lyrat_alexa_local_player_with_equalizer/main/equalizer.c index e1719ad..a34ddd3 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/main/equalizer.c +++ b/examples/lyrat_alexa_local_player_with_equalizer/main/equalizer.c @@ -1,3 +1,6 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + #include #include diff --git a/examples/lyrat_alexa_local_player_with_equalizer/main/local_player.c b/examples/lyrat_alexa_local_player_with_equalizer/main/local_player.c index 04c888a..801cfd5 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/main/local_player.c +++ b/examples/lyrat_alexa_local_player_with_equalizer/main/local_player.c @@ -1,3 +1,6 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + #include #include @@ -93,18 +96,19 @@ void local_player_task() char *url = NULL; while(1) { vTaskDelay(10000 / portTICK_PERIOD_MS); - url = va_mem_strdup("file:/sdcard/Ted.mp3", VA_MEM_EXTERNAL); + url = va_mem_strdup("file:///sdcard/1.mp3", VA_MEM_EXTERNAL); local_player_play(url, REPLACE_ALL); vTaskDelay(10000 / portTICK_PERIOD_MS); - url = va_mem_strdup("file:/sdcard/Saavn.mp3", VA_MEM_EXTERNAL); + url = va_mem_strdup("file:///sdcard/2.mp3", VA_MEM_EXTERNAL); local_player_play(url, REPLACE_ALL); vTaskDelay(10000 / portTICK_PERIOD_MS); - url = va_mem_strdup("file:/sdcard/Saavn.mp3", VA_MEM_EXTERNAL); + url = va_mem_strdup("file:///sdcard/3.mp3", VA_MEM_EXTERNAL); local_player_play(url, ENQUEUE); vTaskDelay(10000 / portTICK_PERIOD_MS); - url = va_mem_strdup("file:/sdcard/Ted.mp3", VA_MEM_EXTERNAL); + url = va_mem_strdup("file:///sdcard/1.mp3", VA_MEM_EXTERNAL); local_player_play(url, ENQUEUE); - vTaskDelay(1000000 / portTICK_PERIOD_MS); + while (1) + vTaskDelay(portMAX_DELAY); } } diff --git a/examples/lyrat_alexa_local_player_with_equalizer/main/ui_button.c b/examples/lyrat_alexa_local_player_with_equalizer/main/ui_button.c index b35c331..c3f2129 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/main/ui_button.c +++ b/examples/lyrat_alexa_local_player_with_equalizer/main/ui_button.c @@ -8,7 +8,7 @@ #include #include #include -#include +#include #include #include "ui_button.h" #include diff --git a/examples/lyrat_alexa_local_player_with_equalizer/main/ui_button.h b/examples/lyrat_alexa_local_player_with_equalizer/main/ui_button.h index 93ce16d..0631293 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/main/ui_button.h +++ b/examples/lyrat_alexa_local_player_with_equalizer/main/ui_button.h @@ -1,7 +1,6 @@ // Copyright 2018 Espressif Systems (Shanghai) PTE LTD // All rights reserved. - #ifndef _UI_BUTTON_H_ #define _UI_BUTTON_H_ #include diff --git a/examples/lyrat_alexa_local_player_with_equalizer/main/ui_led.c b/examples/lyrat_alexa_local_player_with_equalizer/main/ui_led.c index c6fa027..168ecba 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/main/ui_led.c +++ b/examples/lyrat_alexa_local_player_with_equalizer/main/ui_led.c @@ -6,17 +6,17 @@ #include #include #include -#include +#include static const char *UI_LED_TAG = "UI_LED"; -esp_err_t ui_led_set(int alexa_state) +esp_err_t ui_led_set(int va_state) { - if (alexa_state == ALEXA_LISTENING) { + if (va_state == VA_LISTENING) { gpio_set_level(LED_GPIO_PIN, 1); - } else if (alexa_state == LED_RESET){ + } else if (va_state == LED_RESET){ gpio_set_level(LED_BOOT_PIN, 1); - } else if (alexa_state == LED_OFF) { + } else if (va_state == LED_OFF) { gpio_set_level(LED_GPIO_PIN, 0); gpio_set_level(LED_BOOT_PIN, 0); } else { diff --git a/examples/lyrat_alexa_local_player_with_equalizer/main/ui_led.h b/examples/lyrat_alexa_local_player_with_equalizer/main/ui_led.h index e52c142..518f3e2 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/main/ui_led.h +++ b/examples/lyrat_alexa_local_player_with_equalizer/main/ui_led.h @@ -1,7 +1,6 @@ // Copyright 2018 Espressif Systems (Shanghai) PTE LTD // All rights reserved. - #ifndef _UI_LED_H_ #define _UI_LED_H_ @@ -23,7 +22,7 @@ enum { LED_OFF, }; -esp_err_t ui_led_set(int alexa_state); +esp_err_t ui_led_set(int va_state); esp_err_t ui_led_init(); diff --git a/examples/lyrat_alexa_local_player_with_equalizer/sdkconfig.defaults b/examples/lyrat_alexa_local_player_with_equalizer/sdkconfig.defaults index 5cbf533..c9c809d 100644 --- a/examples/lyrat_alexa_local_player_with_equalizer/sdkconfig.defaults +++ b/examples/lyrat_alexa_local_player_with_equalizer/sdkconfig.defaults @@ -75,6 +75,7 @@ CONFIG_FREERTOS_USE_TRACE_FACILITY=y CONFIG_LWIP_MAX_SOCKETS=4 CONFIG_USE_ONLY_LWIP_SELECT=y CONFIG_LWIP_ETHARP_TRUST_IP_MAC=y +CONFIG_LWIP_DHCP_MAX_NTP_SERVERS=3 # # TCP diff --git a/examples/lyrat_alexa_sr/README.md b/examples/lyrat_alexa_sr/README.md deleted file mode 100644 index 872bbfe..0000000 --- a/examples/lyrat_alexa_sr/README.md +++ /dev/null @@ -1,87 +0,0 @@ -# Prerequisites -Please prepare your host system with the toolchain. Please see http://esp-idf.readthedocs.io/en/latest/get-started/index.html for the host setup. - -# Supported Hardware -The current application supports ESP32 based LyraT v4.1, LyraT v4.2 and LyraT v4.3 - -# Prepare Images -If you already have the pre-compiled images flashed on the board, please jump to the next section (Device Configuration). - -## Clone all the repositories -``` -$ git clone --recursive https://github.com/espressif/esp-idf.git - -$ cd esp-idf; git checkout release/v3.1; cd .. - -$ git clone https://github.com/espressif/esp-avs-sdk.git -``` - -## Applying extraneous patches on esp-idf -``` -$ cd esp-idf - -$ git apply ../esp-avs-sdk/esp-idf-patches/memset-i2s-dma-buffers-zero.patch - -$ git apply ../esp-avs-sdk/esp-idf-patches/esp-tls-Add-support-for-global-CA-store.-All-mbedtls.patch -``` - -## Build and flash the project -``` -$ cd esp-avs-sdk/examples/lyrat_alexa_sr - -$ export IDF_PATH=/path/to/esp-idf - -$ export ESPPORT=/dev/cu.SLAB_USBtoUART (or /dev/ttyUSB0 or /dev/ttyUSB1 on Linux or COMxx on MinGW) - -$ make -j 8 flash monitor -``` - -# Device Configuration -Android and iOS apps are available to configure WiFi credentials and associate user's Amazon account with the device. -* [Android](https://github.com/espressif/esp-avs-sdk/releases) -* [iOS](https://github.com/espressif/esp-idf-provisioning-ios/tree/versions/avs) - -# Enabling AWS [Optional] -To Enable AWS IoT platform on ESP32 follow below steps: - -* Enable Amazon Web Services IoT Platform -``` -make menuconfig -> Component config -> Amazon Web Services IoT Platform -``` -* Enter AWS IoT Endpoint Hostname -``` -make menuconfig -> Component config -> Amazon Web Services IoT Platform -> AWS IoT Endpoint Hostname -> Enter the hostname -``` - -* Copy certificate file (certificate.pem.cert and private.pem.key) to examples/lyrat_alexa_sr/main/certs/ folder appropriately to communicate with AWS endpoint. - -* Subscribe to "test_topic/esp32" on AWS Console to see the messages. - -### Locally Check The Root Certificate [Optional] -The Root CA certificate provides a root-of-trust when the ESP32 connects to AWS IoT. We have supplied the root CA certificate already (in PEM format) in the file `main/certs/aws-root-ca.pem`. - -If connection to server fails, you might want to locally verify that this Root CA certificate hasn't changed, you can run the following command against your AWS MQTT Host: - -``` -openssl s_client -showcerts -connect hostname:8883 < /dev/null -``` - -(Replace hostname with your AWS MQTT endpoint host.) The Root CA certificate is the last certificate in the list of certificates printed. You can copy-paste this in place of the existing `aws-root-ca.pem` file. - -# Known Issues -* While playing from Amazon Music there are few initial glitches before the song starts playing smoothly. -* When AWS is enabled, after provisioning the device takes time(15-20 sec) to connect to AWS. - -# Demo -* Once the board boots up and successfully connects to the Wi-Fi network after provisioning, you will see a print "Alexa is ready", after which you can say ALEXA and speak or use "Rec" button on LyraT board for conversation. For Tap-to-Talk, press and release the button and speak. The green LED glows when the microphone is active. -* You can connect any external speaker/headphone with 3.5mm connector to PHONE JACK to listen to responses. -* you can now ask any command like: - * tell me a joke - * how is the weather? - * will it rain today? - * Sing a song - * Play TuneIn radio - * Set volume level to 7 -* Each time Alexa is detected the WakeWord count is incremented on AWS IoT console -* Press and Hold "Mode" button for 3 seconds to reset the board to factory settings - diff --git a/examples/lyrat_alexa_sr/main/app_dsp.h b/examples/lyrat_alexa_sr/main/app_dsp.h deleted file mode 100644 index 1513a56..0000000 --- a/examples/lyrat_alexa_sr/main/app_dsp.h +++ /dev/null @@ -1,46 +0,0 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -#ifndef _APP_DSP_H_ -#define _APP_DSP_H_ - -#include - -#ifdef __cplusplus -extern "C" { -#endif - -typedef enum dsp_called_states { - ALEXA_CAN_START = (ALEXA_END_STATES + 1), -} dsp_called_states_t; - -void app_dsp_init(void); - -void app_dsp_send_recognize(); - -void app_dsp_reset(void); - -#ifdef __cplusplus -} -#endif - -#endif /* _APP_DSP_H_ */ diff --git a/examples/lyrat_alexa_sr/main/ui_button.h b/examples/lyrat_alexa_sr/main/ui_button.h deleted file mode 100644 index de6fa38..0000000 --- a/examples/lyrat_alexa_sr/main/ui_button.h +++ /dev/null @@ -1,54 +0,0 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -#ifndef _UI_BUTTON_H_ -#define _UI_BUTTON_H_ -#include -#include -#include -#include "freertos/FreeRTOS.h" -#include "freertos/task.h" -#include "esp_log.h" -#include "esp_system.h" -#include "driver/gpio.h" - - -#ifdef __cplusplus -extern "C" { -#endif - -#define UI_BUTTON_TASK_BUFFER_SZ (8 * 1024) - -#define GPIO_RECORD_BUTTON 36 -#define GPIO_MODE_BUTTON 39 -#define ESP_INTR_FLAG_DEFAULT 0 - -/** - * @brief initialize button service - */ -esp_err_t ui_button_init(); - -#ifdef __cplusplus -} -#endif - -#endif /* _UI_BUTTON_H_ */ diff --git a/examples/lyrat_alexa_sr/main/ui_led.c b/examples/lyrat_alexa_sr/main/ui_led.c deleted file mode 100644 index 10fb982..0000000 --- a/examples/lyrat_alexa_sr/main/ui_led.c +++ /dev/null @@ -1,62 +0,0 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -#include -#include -#include -#include -#include -#include - -static const char *UI_LED_TAG = "UI_LED"; - -esp_err_t ui_led_set(int alexa_state) -{ - if (alexa_state == ALEXA_LISTENING) { - gpio_set_level(LED_GPIO_PIN, 1); - } else if (alexa_state == LED_RESET){ - gpio_set_level(LED_BOOT_PIN, 1); - } else if (alexa_state == LED_OFF) { - gpio_set_level(LED_GPIO_PIN, 0); - gpio_set_level(LED_BOOT_PIN, 0); - } else { - gpio_set_level(LED_GPIO_PIN, 0); - } - return ESP_OK; -} - -esp_err_t ui_led_init() -{ - esp_err_t ret; - gpio_config_t io_conf; - io_conf.intr_type = GPIO_INTR_DISABLE; - io_conf.pin_bit_mask = BIT(LED_GPIO_PIN); - io_conf.mode = GPIO_MODE_OUTPUT; - ret = gpio_config(&io_conf); - gpio_set_level(LED_GPIO_PIN, 0); - - io_conf.pin_bit_mask = BIT(LED_BOOT_PIN); - ret |= gpio_config(&io_conf); - gpio_set_level(LED_BOOT_PIN, 0); - ESP_LOGI(UI_LED_TAG, "LED initialized"); - return ret; -} diff --git a/examples/lyrat_alexa_sr/main/ui_led.h b/examples/lyrat_alexa_sr/main/ui_led.h deleted file mode 100644 index c85c596..0000000 --- a/examples/lyrat_alexa_sr/main/ui_led.h +++ /dev/null @@ -1,52 +0,0 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -#ifndef _UI_LED_H_ -#define _UI_LED_H_ - -#include -#include -#include -#include - -#ifdef __cplusplus -extern "C" { -#endif - - -#define LED_GPIO_PIN 22 -#define LED_BOOT_PIN 19 - -enum { - LED_RESET = 100, - LED_OFF, -}; - -esp_err_t ui_led_set(int alexa_state); - -esp_err_t ui_led_init(); - -#ifdef __cplusplus -} -#endif - -#endif /* _UI_LED_H_ */ diff --git a/examples/lyrat_alexa_sr/Makefile b/examples/lyrat_sr/Makefile similarity index 58% rename from examples/lyrat_alexa_sr/Makefile rename to examples/lyrat_sr/Makefile index 7644270..202509b 100644 --- a/examples/lyrat_alexa_sr/Makefile +++ b/examples/lyrat_sr/Makefile @@ -3,7 +3,16 @@ # project subdirectory. # +ifeq ("$(VOICE_ASSISTANT)","gva") +PROJECT_NAME := gva +CPPFLAGS += -DGVA +else ifeq ("$(VOICE_ASSISTANT)","dialogflow") +PROJECT_NAME := dialogflow +CPPFLAGS += -DDIALOGFLOW +else PROJECT_NAME := alexa +CPPFLAGS += -DALEXA +endif EXTRA_COMPONENT_DIRS += $(PROJECT_PATH)/../../components $(PROJECT_PATH)/../../components/speech_recog diff --git a/examples/lyrat_sr/README-AWS-IoT.md b/examples/lyrat_sr/README-AWS-IoT.md new file mode 100644 index 0000000..d134f81 --- /dev/null +++ b/examples/lyrat_sr/README-AWS-IoT.md @@ -0,0 +1,31 @@ +# Enabling AWS [Optional] +To Enable AWS IoT platform on ESP32 follow below steps: + +* Enable Amazon Web Services IoT Platform +``` +make menuconfig -> Component config -> Amazon Web Services IoT Platform +``` +* Enter AWS IoT Endpoint Hostname +``` +make menuconfig -> Component config -> Amazon Web Services IoT Platform -> AWS IoT Endpoint Hostname -> Enter the hostname +``` + +* Copy certificate file (certificate.pem.cert and private.pem.key) to examples/lyrat_alexa_sr/main/certs/ folder appropriately to communicate with AWS endpoint. + +* Subscribe to "test_topic/esp32" on AWS Console to see the messages. + +### Locally Check The Root Certificate [Optional] +The Root CA certificate provides a root-of-trust when the ESP32 connects to AWS IoT. We have supplied the root CA certificate already (in PEM format) in the file `main/certs/aws-root-ca.pem`. + +If connection to server fails, you might want to locally verify that this Root CA certificate hasn't changed, you can run the following command against your AWS MQTT Host: + +``` +openssl s_client -showcerts -connect hostname:8883 < /dev/null +``` + +(Replace hostname with your AWS MQTT endpoint host.) The Root CA certificate is the last certificate in the list of certificates printed. You can copy-paste this in place of the existing `aws-root-ca.pem` file. + +# Demo +* App supports usual demo scenario for each voice assistant as described in the respective README in the SDK root directory. +* When AWS-IoT is enabled, each time Alexa wakeword is detected the WakeWord count is incremented on AWS IoT console. +* Board only support "Alexa" wakeword as of now. Hence even to use Google Voice Assistant or Dialogflow one must say "Alexa" to wake the device. diff --git a/examples/lyrat_alexa_sr/components/audio_board_lyrat/audio_board_lyrat.c b/examples/lyrat_sr/components/audio_board_lyrat/audio_board_lyrat.c similarity index 100% rename from examples/lyrat_alexa_sr/components/audio_board_lyrat/audio_board_lyrat.c rename to examples/lyrat_sr/components/audio_board_lyrat/audio_board_lyrat.c diff --git a/examples/lyrat_alexa_sr/components/audio_board_lyrat/component.mk b/examples/lyrat_sr/components/audio_board_lyrat/component.mk similarity index 100% rename from examples/lyrat_alexa_sr/components/audio_board_lyrat/component.mk rename to examples/lyrat_sr/components/audio_board_lyrat/component.mk diff --git a/examples/lyrat_alexa_sr/components/codec_es8388/component.mk b/examples/lyrat_sr/components/codec_es8388/component.mk similarity index 100% rename from examples/lyrat_alexa_sr/components/codec_es8388/component.mk rename to examples/lyrat_sr/components/codec_es8388/component.mk diff --git a/examples/lyrat_alexa_sr/components/codec_es8388/es8388.c b/examples/lyrat_sr/components/codec_es8388/es8388.c similarity index 100% rename from examples/lyrat_alexa_sr/components/codec_es8388/es8388.c rename to examples/lyrat_sr/components/codec_es8388/es8388.c diff --git a/examples/lyrat_alexa_sr/components/codec_es8388/es8388.h b/examples/lyrat_sr/components/codec_es8388/es8388.h similarity index 100% rename from examples/lyrat_alexa_sr/components/codec_es8388/es8388.h rename to examples/lyrat_sr/components/codec_es8388/es8388.h diff --git a/examples/lyrat_alexa_sr/components/codec_es8388/media_hal.c b/examples/lyrat_sr/components/codec_es8388/media_hal.c similarity index 100% rename from examples/lyrat_alexa_sr/components/codec_es8388/media_hal.c rename to examples/lyrat_sr/components/codec_es8388/media_hal.c diff --git a/examples/lyrat_alexa_sr/main/Kconfig.projbuild b/examples/lyrat_sr/main/Kconfig.projbuild similarity index 100% rename from examples/lyrat_alexa_sr/main/Kconfig.projbuild rename to examples/lyrat_sr/main/Kconfig.projbuild diff --git a/examples/lyrat_alexa_sr/main/app_dsp.c b/examples/lyrat_sr/main/app_dsp.c similarity index 84% rename from examples/lyrat_alexa_sr/main/app_dsp.c rename to examples/lyrat_sr/main/app_dsp.c index 097fe98..6065ddc 100644 --- a/examples/lyrat_alexa_sr/main/app_dsp.c +++ b/examples/lyrat_sr/main/app_dsp.c @@ -1,24 +1,5 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. #include #include @@ -29,19 +10,17 @@ #include #include -#include -#include -#include "app_dsp.h" -#include -#include +#include #include #include #include -#include #include +#include +#include + +#include "app_dsp.h" #include "resampling.h" -//Speech recognition headers -#include +#include "ui_led.h" #define WWE_TASK_STACK (8 * 1024) #define DETECT_SAMP_RATE 16000UL @@ -71,6 +50,7 @@ static struct dsp_data { bool write_to_store; int16_t data_buf[SAMPLE_SZ]; char pcm_store[PCM_SIZE]; + bool dsp_inited; } dd; static media_hal_config_t media_hal_conf = { @@ -99,7 +79,7 @@ static ssize_t dsp_write_cb(void *h, void *data, int len, uint32_t wait) return sent_len; } -int alexa_app_speech_stop() +int va_app_speech_stop() { ESP_LOGI(TAG, "Sending stop command"); dd.speech_recog_en = false; @@ -109,7 +89,7 @@ int alexa_app_speech_stop() return ESP_OK; } -int alexa_app_speech_start() +int va_app_speech_start() { ESP_LOGI(TAG, "Sending start command"); dd.detect_wakeword = false; @@ -125,6 +105,9 @@ void app_dsp_reset() void app_dsp_send_recognize() { + if (!dd.dsp_inited) + return; + ESP_LOGI(TAG, "Sending start command"); dd.detect_wakeword = false; ESP_LOGI(TAG, "Starting I2S audio stream"); @@ -202,9 +185,6 @@ void nn_task(void *arg) void app_dsp_init(void) { dd.temp_rb = rb_init("nn-recog", 4 * 1024); - ui_led_init(); - ui_button_init(); - i2s_stream_config_t i2s_cfg; memset(&i2s_cfg, 0, sizeof(i2s_cfg)); i2s_cfg.i2s_num = 0; @@ -250,4 +230,5 @@ void app_dsp_init(void) vTaskDelay(10/portTICK_RATE_MS); audio_stream_start(&dd.read_i2s_stream->base); dd.detect_wakeword = true; + dd.dsp_inited = true; } diff --git a/examples/lyrat_sr/main/app_dsp.h b/examples/lyrat_sr/main/app_dsp.h new file mode 100644 index 0000000..221ce6b --- /dev/null +++ b/examples/lyrat_sr/main/app_dsp.h @@ -0,0 +1,27 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + +#ifndef _APP_DSP_H_ +#define _APP_DSP_H_ + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +typedef enum dsp_called_states { + VA_CAN_START = (VA_END_STATES + 1), +} dsp_called_states_t; + +void app_dsp_init(void); + +void app_dsp_send_recognize(); + +void app_dsp_reset(void); + +#ifdef __cplusplus +} +#endif + +#endif /* _APP_DSP_H_ */ diff --git a/examples/lyrat_alexa_sr/main/app_lyrat_cb.c b/examples/lyrat_sr/main/app_lyrat_cb.c similarity index 50% rename from examples/lyrat_alexa_sr/main/app_lyrat_cb.c rename to examples/lyrat_sr/main/app_lyrat_cb.c index 0a01313..c1d19d4 100644 --- a/examples/lyrat_alexa_sr/main/app_lyrat_cb.c +++ b/examples/lyrat_sr/main/app_lyrat_cb.c @@ -1,33 +1,18 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. #include #include -#include +#include #include #include #include #include +#if DIALOGFLOW +#include +#endif + #define I2S_PORT_NUM I2S_NUM_0 #define DES_BITS_PER_SAM 16 #define SRC_BITS_PER_SAM 16 @@ -45,30 +30,30 @@ static inline int heap_caps_get_free_size_sram() return heap_caps_get_free_size(MALLOC_CAP_8BIT) - heap_caps_get_free_size(MALLOC_CAP_SPIRAM); } -void alexa_app_dialog_states(alexa_dialog_states_t alexa_states) +void va_app_dialog_states(va_dialog_states_t va_state) { - ui_led_set(alexa_states); - ESP_LOGI(TAG, "Current mode is: %d", alexa_states); + ui_led_set(va_state); + ESP_LOGI(TAG, "Current mode is: %d", va_state); } -int alexa_app_set_volume(int vol) +int va_app_set_volume(int vol) { es8388_control_volume(vol); ESP_LOGI(TAG, "Volume changed to %d", vol); return 0; } -int alexa_app_set_mute(alexa_mute_state_t alexa_mute_state) +int va_app_set_mute(va_mute_state_t va_mute_state) { uint8_t current_vol; - if (alexa_mute_state == ALEXA_MUTE_ENABLE) { + if (va_mute_state == VA_MUTE_ENABLE) { es8388_set_mute(1); } else { es8388_set_mute(0); es8388_get_volume(¤t_vol); es8388_control_volume(current_vol); } - ESP_LOGI(TAG, "Mute: %d", alexa_mute_state); + ESP_LOGI(TAG, "Mute: %d", va_mute_state); return 0; } @@ -77,17 +62,17 @@ int alexa_app_raise_alert(alexa_alert_types_t alexa_alert_type, alexa_alert_stat return 0; } -int alexa_app_playback_data(alexa_resample_param_t *alexa_resample_param, void *buf, ssize_t len) +int va_app_playback_data(va_resample_param_t *va_resample_param, void *buf, ssize_t len) { memset(convert_buf, 0, BUF_SZ); - audio_resample_config_t resample = {0}; + static audio_resample_config_t resample = {0}; int current_convert_block_len; int convert_block_len = 0; int send_offset = 0; size_t sent_len = 0; int conv_len = 0; - if (alexa_resample_param->alexa_resample_ch == 1) { + if (va_resample_param->va_resample_ch == 1) { /* If mono recording, we need to up-sample, so need half the buffer empty, also uint16_t data*/ convert_block_len = CONVERT_BUF_SIZE / 4; } else { @@ -100,9 +85,9 @@ int alexa_app_playback_data(alexa_resample_param_t *alexa_resample_param, void * if (current_convert_block_len & 1) { printf("Odd bytes in up sampling data, this should be backed up\n"); } - conv_len = audio_resample((short *)((char *)buf + send_offset), (short *)convert_buf, alexa_resample_param->alexa_resample_freq, SAMPLING_RATE, - current_convert_block_len / 2, BUF_SZ, alexa_resample_param->alexa_resample_ch, &resample); - if (alexa_resample_param->alexa_resample_ch == 1) { + conv_len = audio_resample((short *)((char *)buf + send_offset), (short *)convert_buf, va_resample_param->va_resample_freq, SAMPLING_RATE, + current_convert_block_len / 2, BUF_SZ, va_resample_param->va_resample_ch, &resample); + if (va_resample_param->va_resample_ch == 1) { conv_len = audio_resample_up_channel((short *)convert_buf, (short *)convert_buf, SAMPLING_RATE, SAMPLING_RATE, conv_len, BUF_SZ, &resample); } len -= current_convert_block_len; @@ -142,3 +127,64 @@ int i2s_playback_init() ret = i2s_zero_dma_buffer(I2S_PORT_NUM); return ret; } + +#if DIALOGFLOW +void dialogflow_app_response_data(Google__Cloud__Dialogflow__V2beta1__StreamingDetectIntentResponse *response) +{ + int i; + bool end_conversation = false; + printf("The query text is: %s\n", response->query_result->query_text); + printf("The fulfillment text is: %s\n", response->query_result->fulfillment_text); + + Google__Protobuf__Struct *entities = response->query_result->parameters; + if (!entities) { + return; + } + + /* Diagnostics information can be used to mark the end of a conversation. Make sure that "Set this intent as end + * of conversation" is enabled under "Responses" tab in each intent of your Dialogflow agent + */ + Google__Protobuf__Struct *info = response->query_result->diagnostic_info; + if (info) { + for (i = 0; i < info->n_fields; i++) { + switch (info->fields[i]->value->kind_case) { + case GOOGLE__PROTOBUF__VALUE__KIND_BOOL_VALUE: + if (!strncmp(info->fields[i]->key, "end_conversation", strlen(info->fields[i]->key)) && + info->fields[i]->value->bool_value) { + end_conversation = true; + } + break; + default: + break; + } + } + } + + if (end_conversation) { + printf("Dialogflow conversation complete:\nNumber of entities: %d\n", entities->n_fields); + for (i = 0; i < entities->n_fields; i++) { + switch (entities->fields[i]->value->kind_case) { + case GOOGLE__PROTOBUF__VALUE__KIND_NUMBER_VALUE: + printf("[%d] %s : %f\n", i, entities->fields[i]->key, entities->fields[i]->value->number_value); + break; + case GOOGLE__PROTOBUF__VALUE__KIND_STRING_VALUE: + printf("[%d] %s : %s\n", i, entities->fields[i]->key, entities->fields[i]->value->string_value); + break; + case GOOGLE__PROTOBUF__VALUE__KIND_BOOL_VALUE: + printf("[%d] %s : %s\n", i, entities->fields[i]->key, entities->fields[i]->value->bool_value ? "true" : "false"); + break; + default: + printf("Unknown type %d: Please update this debug function\n", entities->fields[i]->value->kind_case); + break; + } + } + } else { + printf("Dialogflow conversation continued\n"); + } +} + +void dialogflow_app_query_text_transcript(char *text, bool is_final) +{ + printf("%s transcript is: %s\n", (is_final != 0 ? "Final" : "Interim"),text); +} +#endif diff --git a/examples/lyrat_alexa_sr/main/app_main.c b/examples/lyrat_sr/main/app_main.c similarity index 72% rename from examples/lyrat_alexa_sr/main/app_main.c rename to examples/lyrat_sr/main/app_main.c index 539cad4..ae57c3f 100644 --- a/examples/lyrat_alexa_sr/main/app_main.c +++ b/examples/lyrat_sr/main/app_main.c @@ -1,7 +1,7 @@ // Copyright 2018 Espressif Systems (Shanghai) PTE LTD // All rights reserved. -/* Alexa + AWS IoT Example +/* Voice Assistant Example (Alexa is with AWS IoT) This example code is in the Public Domain (or CC0 licensed, at your option.) @@ -21,7 +21,14 @@ #include #include +#ifdef ALEXA #include +#elif GVA +#include +#elif DIALOGFLOW +#include +#endif + #include #include #include @@ -30,6 +37,7 @@ #include "app_dsp.h" #include "ui_led.h" +#include "ui_button.h" #ifdef CONFIG_AWS_IOT_SDK extern void aws_iot_init(); @@ -37,13 +45,11 @@ extern void aws_iot_init(); #define SOFTAP_SSID_PREFIX "ESP-Alexa-" -static const char *TAG = "alexa"; +static const char *TAG = "app_main"; static EventGroupHandle_t cm_event_group; const int CONNECTED_BIT = BIT0; -static alexa_config_t *alexa_cfg; - static esp_err_t event_handler(void *ctx, system_event_t *event) { conn_mgr_prov_event_handler(ctx, event); @@ -78,15 +84,21 @@ extern int i2s_playback_init(); int app_main() { - ESP_LOGI(TAG, "==== Alexa SDK version: %s ====", alexa_get_sdk_version()); + ESP_LOGI(TAG, "==== Voice Assistant SDK version: %s ====", va_get_sdk_version()); + +#ifdef ALEXA + alexa_config_t *va_cfg = va_mem_alloc(sizeof(alexa_config_t), VA_MEM_EXTERNAL); +#elif GVA + gva_config_t *va_cfg = va_mem_alloc(sizeof(gva_config_t), VA_MEM_EXTERNAL); +#elif DIALOGFLOW + dialogflow_config_t *va_cfg = va_mem_alloc(sizeof(dialogflow_config_t), VA_MEM_EXTERNAL); +#endif - alexa_cfg = va_mem_alloc(sizeof(alexa_config_t), VA_MEM_EXTERNAL); - if (!alexa_cfg) { - ESP_LOGE(TAG, "Failed to alloc alexa config"); + if (!va_cfg) { + ESP_LOGE(TAG, "Failed to alloc voice assistant config"); abort(); } - - alexa_cfg->auth_delegate.type = auth_type_subsequent; + va_cfg->auth_delegate.type = auth_type_subsequent; // Initialize NVS esp_err_t ret = nvs_flash_init(); @@ -96,12 +108,14 @@ int app_main() } ESP_ERROR_CHECK( ret ); + ui_led_init(); + ui_button_init(); scli_init(); diag_register_cli(); wifi_register_cli(); cm_event_group = xEventGroupCreate(); - tcpip_adapter_init(); + ESP_ERROR_CHECK(esp_event_loop_init(event_handler, NULL) ); bool provisioned = false; @@ -118,7 +132,7 @@ int app_main() conn_mgr_prov_t prov_type = conn_mgr_prov_mode_softap; prov_type.event_cb = alexa_conn_mgr_prov_cb; - prov_type.cb_user_data = (void *)alexa_cfg; + prov_type.cb_user_data = (void *)va_cfg; int security = 1; const char *pop = "abcd1234"; const char *service_key = ""; @@ -133,12 +147,25 @@ int app_main() i2s_playback_init(); app_dsp_init(); - if (alexa_init(alexa_cfg) != ESP_OK) { - while(1) vTaskDelay(2); - } + +#ifdef ALEXA + ret = alexa_init(va_cfg); #ifdef CONFIG_AWS_IOT_SDK aws_iot_init(); #endif - va_mem_free(alexa_cfg); + +#elif GVA + va_cfg->device_config.device_model = "device-model-default"; // Enter your model id (name) here + va_cfg->device_config.device_id = "device-id-default"; // Enter your (unique) device id here + ret = gva_init(va_cfg); +#elif DIALOGFLOW + va_cfg->device_config.project_name = "project-name-default"; // Enter your dialogflow project name here. + ret = dialogflow_init(va_cfg); +#endif + + if (ret != ESP_OK) { + while(1) vTaskDelay(2); + } + va_mem_free(va_cfg); return ESP_OK; } diff --git a/examples/lyrat_alexa_sr/main/certs/README.md b/examples/lyrat_sr/main/certs/README.md similarity index 100% rename from examples/lyrat_alexa_sr/main/certs/README.md rename to examples/lyrat_sr/main/certs/README.md diff --git a/examples/lyrat_alexa_sr/main/certs/aws-root-ca.pem b/examples/lyrat_sr/main/certs/aws-root-ca.pem similarity index 100% rename from examples/lyrat_alexa_sr/main/certs/aws-root-ca.pem rename to examples/lyrat_sr/main/certs/aws-root-ca.pem diff --git a/examples/lyrat_alexa_sr/main/component.mk b/examples/lyrat_sr/main/component.mk similarity index 100% rename from examples/lyrat_alexa_sr/main/component.mk rename to examples/lyrat_sr/main/component.mk diff --git a/examples/lyrat_alexa_sr/main/subscribe_publish_sample.c b/examples/lyrat_sr/main/subscribe_publish_sample.c similarity index 100% rename from examples/lyrat_alexa_sr/main/subscribe_publish_sample.c rename to examples/lyrat_sr/main/subscribe_publish_sample.c diff --git a/examples/lyrat_alexa_sr/main/ui_button.c b/examples/lyrat_sr/main/ui_button.c similarity index 78% rename from examples/lyrat_alexa_sr/main/ui_button.c rename to examples/lyrat_sr/main/ui_button.c index 191ce71..c3f2129 100644 --- a/examples/lyrat_alexa_sr/main/ui_button.c +++ b/examples/lyrat_sr/main/ui_button.c @@ -1,24 +1,5 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. #include #include @@ -27,7 +8,7 @@ #include #include #include -#include +#include #include #include "ui_button.h" #include diff --git a/examples/lyrat_sr/main/ui_button.h b/examples/lyrat_sr/main/ui_button.h new file mode 100644 index 0000000..b7db1f8 --- /dev/null +++ b/examples/lyrat_sr/main/ui_button.h @@ -0,0 +1,35 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + +#ifndef _UI_BUTTON_H_ +#define _UI_BUTTON_H_ +#include +#include +#include +#include "freertos/FreeRTOS.h" +#include "freertos/task.h" +#include "esp_log.h" +#include "esp_system.h" +#include "driver/gpio.h" + + +#ifdef __cplusplus +extern "C" { +#endif + +#define UI_BUTTON_TASK_BUFFER_SZ (8 * 1024) + +#define GPIO_RECORD_BUTTON 36 +#define GPIO_MODE_BUTTON 39 +#define ESP_INTR_FLAG_DEFAULT 0 + +/** + * @brief initialize button service + */ +esp_err_t ui_button_init(); + +#ifdef __cplusplus +} +#endif + +#endif /* _UI_BUTTON_H_ */ diff --git a/examples/lyrat_sr/main/ui_led.c b/examples/lyrat_sr/main/ui_led.c new file mode 100644 index 0000000..168ecba --- /dev/null +++ b/examples/lyrat_sr/main/ui_led.c @@ -0,0 +1,43 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + +#include +#include +#include +#include +#include +#include + +static const char *UI_LED_TAG = "UI_LED"; + +esp_err_t ui_led_set(int va_state) +{ + if (va_state == VA_LISTENING) { + gpio_set_level(LED_GPIO_PIN, 1); + } else if (va_state == LED_RESET){ + gpio_set_level(LED_BOOT_PIN, 1); + } else if (va_state == LED_OFF) { + gpio_set_level(LED_GPIO_PIN, 0); + gpio_set_level(LED_BOOT_PIN, 0); + } else { + gpio_set_level(LED_GPIO_PIN, 0); + } + return ESP_OK; +} + +esp_err_t ui_led_init() +{ + esp_err_t ret; + gpio_config_t io_conf; + io_conf.intr_type = GPIO_INTR_DISABLE; + io_conf.pin_bit_mask = BIT(LED_GPIO_PIN); + io_conf.mode = GPIO_MODE_OUTPUT; + ret = gpio_config(&io_conf); + gpio_set_level(LED_GPIO_PIN, 0); + + io_conf.pin_bit_mask = BIT(LED_BOOT_PIN); + ret |= gpio_config(&io_conf); + gpio_set_level(LED_BOOT_PIN, 0); + ESP_LOGI(UI_LED_TAG, "LED initialized"); + return ret; +} diff --git a/examples/lyrat_sr/main/ui_led.h b/examples/lyrat_sr/main/ui_led.h new file mode 100644 index 0000000..518f3e2 --- /dev/null +++ b/examples/lyrat_sr/main/ui_led.h @@ -0,0 +1,33 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + +#ifndef _UI_LED_H_ +#define _UI_LED_H_ + +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + + +#define LED_GPIO_PIN 22 +#define LED_BOOT_PIN 19 + +enum { + LED_RESET = 100, + LED_OFF, +}; + +esp_err_t ui_led_set(int va_state); + +esp_err_t ui_led_init(); + +#ifdef __cplusplus +} +#endif + +#endif /* _UI_LED_H_ */ diff --git a/examples/lyrat_alexa_sr/partitions.csv b/examples/lyrat_sr/partitions.csv similarity index 100% rename from examples/lyrat_alexa_sr/partitions.csv rename to examples/lyrat_sr/partitions.csv diff --git a/examples/lyratd_msc_alexa_sr/sdkconfig.defaults b/examples/lyrat_sr/sdkconfig.defaults similarity index 98% rename from examples/lyratd_msc_alexa_sr/sdkconfig.defaults rename to examples/lyrat_sr/sdkconfig.defaults index 3614629..966d92e 100644 --- a/examples/lyratd_msc_alexa_sr/sdkconfig.defaults +++ b/examples/lyrat_sr/sdkconfig.defaults @@ -76,6 +76,7 @@ CONFIG_FREERTOS_USE_TRACE_FACILITY=y CONFIG_LWIP_MAX_SOCKETS=4 CONFIG_USE_ONLY_LWIP_SELECT=y CONFIG_LWIP_ETHARP_TRUST_IP_MAC=y +CONFIG_LWIP_DHCP_MAX_NTP_SERVERS=3 # # TCP diff --git a/examples/lyratd_msc_alexa_sr/README.md b/examples/lyratd_msc_alexa_sr/README.md deleted file mode 100644 index 572c88f..0000000 --- a/examples/lyratd_msc_alexa_sr/README.md +++ /dev/null @@ -1,57 +0,0 @@ -# Prerequisites -Please prepare your host system with the toolchain. Please see http://esp-idf.readthedocs.io/en/latest/get-started/index.html for the host setup. - -# Supported Hardware -The current application supports ESP32 based LyraTD_MSC v2.0, LyraTD_MSC v2.1 - -# Prepare Images -If you already have the pre-compiled images flashed on the board, please jump to the next section (Device Configuration). - -## Clone all the repositories -``` -$ git clone --recursive https://github.com/espressif/esp-idf.git - -$ cd esp-idf; git checkout release/v3.1; cd .. - -$ git clone https://github.com/espressif/esp-avs-sdk.git -``` - -## Applying extraneous patches on esp-idf -``` -$ cd esp-idf - -$ git apply ../esp-avs-sdk/esp-idf-patches/memset-i2s-dma-buffers-zero.patch - -$ git apply ../esp-avs-sdk/esp-idf-patches/esp-tls-Add-support-for-global-CA-store.-All-mbedtls.patch -``` - -## Build and flash the project -``` -$ cd esp-avs-sdk/examples/lyratd_msc_alexa_sr - -$ export IDF_PATH=/path/to/esp-idf - -$ export ESPPORT=/dev/cu.SLAB_USBtoUART (or /dev/ttyUSB0 or /dev/ttyUSB1 on Linux or COMxx on MinGW) - -$ make -j 8 flash monitor -``` - -# Device Configuration -Android and iOS apps are available to configure WiFi credentials and associate user's Amazon account with the device. -* [Android](https://github.com/espressif/esp-avs-sdk/releases) -* [iOS](https://github.com/espressif/esp-idf-provisioning-ios/tree/versions/avs) - -# Known Issues -* While playing from Amazon Music there are few initial glitches before the song starts playing smoothly. - -# Demo -* Once the board boots up and successfully connects to the Wi-Fi network after provisioning, you will see a print "Alexa is ready", after which you can say ALEXA and speak or use "Rec" button on LyraTD-MSC board for conversation. For Tap-to-Talk, press and release the button and speak. The green LED glows when the microphone is active. -* You can connect any external speaker/headphone with 3.5mm connector to PHONE JACK to listen to responses. -* you can now ask any command like: - * tell me a joke - * how is the weather? - * will it rain today? - * Sing a song - * Play TuneIn radio - * Set volume level to 7 -* Press and Hold "Mode" button for 3 seconds to reset the board to factory settings diff --git a/examples/lyratd_msc_alexa_sr/main/app_dsp.h b/examples/lyratd_msc_alexa_sr/main/app_dsp.h deleted file mode 100644 index 1513a56..0000000 --- a/examples/lyratd_msc_alexa_sr/main/app_dsp.h +++ /dev/null @@ -1,46 +0,0 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -#ifndef _APP_DSP_H_ -#define _APP_DSP_H_ - -#include - -#ifdef __cplusplus -extern "C" { -#endif - -typedef enum dsp_called_states { - ALEXA_CAN_START = (ALEXA_END_STATES + 1), -} dsp_called_states_t; - -void app_dsp_init(void); - -void app_dsp_send_recognize(); - -void app_dsp_reset(void); - -#ifdef __cplusplus -} -#endif - -#endif /* _APP_DSP_H_ */ diff --git a/examples/lyratd_msc_alexa_sr/Makefile b/examples/lyratd_msc_sr/Makefile similarity index 58% rename from examples/lyratd_msc_alexa_sr/Makefile rename to examples/lyratd_msc_sr/Makefile index 7644270..202509b 100644 --- a/examples/lyratd_msc_alexa_sr/Makefile +++ b/examples/lyratd_msc_sr/Makefile @@ -3,7 +3,16 @@ # project subdirectory. # +ifeq ("$(VOICE_ASSISTANT)","gva") +PROJECT_NAME := gva +CPPFLAGS += -DGVA +else ifeq ("$(VOICE_ASSISTANT)","dialogflow") +PROJECT_NAME := dialogflow +CPPFLAGS += -DDIALOGFLOW +else PROJECT_NAME := alexa +CPPFLAGS += -DALEXA +endif EXTRA_COMPONENT_DIRS += $(PROJECT_PATH)/../../components $(PROJECT_PATH)/../../components/speech_recog diff --git a/examples/lyratd_msc_alexa_sr/components/LICENSE.txt b/examples/lyratd_msc_sr/components/LICENSE.txt similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/LICENSE.txt rename to examples/lyratd_msc_sr/components/LICENSE.txt diff --git a/examples/lyratd_msc_alexa_sr/components/Readme_twolf.txt b/examples/lyratd_msc_sr/components/Readme_twolf.txt similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/Readme_twolf.txt rename to examples/lyratd_msc_sr/components/Readme_twolf.txt diff --git a/examples/lyratd_msc_alexa_sr/components/audio_board_lyratd_msc/audio_board_lyratd_ms.c b/examples/lyratd_msc_sr/components/audio_board_lyratd_msc/audio_board_lyratd_ms.c similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/audio_board_lyratd_msc/audio_board_lyratd_ms.c rename to examples/lyratd_msc_sr/components/audio_board_lyratd_msc/audio_board_lyratd_ms.c diff --git a/examples/lyratd_msc_alexa_sr/components/audio_board_lyratd_msc/component.mk b/examples/lyratd_msc_sr/components/audio_board_lyratd_msc/component.mk similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/audio_board_lyratd_msc/component.mk rename to examples/lyratd_msc_sr/components/audio_board_lyratd_msc/component.mk diff --git a/examples/lyratd_msc_alexa_sr/components/codec_zl38063/component.mk b/examples/lyratd_msc_sr/components/codec_zl38063/component.mk similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/codec_zl38063/component.mk rename to examples/lyratd_msc_sr/components/codec_zl38063/component.mk diff --git a/examples/lyratd_msc_alexa_sr/components/codec_zl38063/media_hal.c b/examples/lyratd_msc_sr/components/codec_zl38063/media_hal.c similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/codec_zl38063/media_hal.c rename to examples/lyratd_msc_sr/components/codec_zl38063/media_hal.c diff --git a/examples/lyratd_msc_alexa_sr/components/codec_zl38063/zl38063.c b/examples/lyratd_msc_sr/components/codec_zl38063/zl38063.c similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/codec_zl38063/zl38063.c rename to examples/lyratd_msc_sr/components/codec_zl38063/zl38063.c diff --git a/examples/lyratd_msc_alexa_sr/components/codec_zl38063/zl38063.h b/examples/lyratd_msc_sr/components/codec_zl38063/zl38063.h similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/codec_zl38063/zl38063.h rename to examples/lyratd_msc_sr/components/codec_zl38063/zl38063.h diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/component.mk b/examples/lyratd_msc_sr/components/dsp_zl38063/component.mk similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/component.mk rename to examples/lyratd_msc_sr/components/dsp_zl38063/component.mk diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_hal_verify.c b/examples/lyratd_msc_sr/components/dsp_zl38063/tw_hal_verify.c similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_hal_verify.c rename to examples/lyratd_msc_sr/components/dsp_zl38063/tw_hal_verify.c diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_ldcfg.c b/examples/lyratd_msc_sr/components/dsp_zl38063/tw_ldcfg.c similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_ldcfg.c rename to examples/lyratd_msc_sr/components/dsp_zl38063/tw_ldcfg.c diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_ldfw.c b/examples/lyratd_msc_sr/components/dsp_zl38063/tw_ldfw.c similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_ldfw.c rename to examples/lyratd_msc_sr/components/dsp_zl38063/tw_ldfw.c diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_ldfwcfg.c b/examples/lyratd_msc_sr/components/dsp_zl38063/tw_ldfwcfg.c similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_ldfwcfg.c rename to examples/lyratd_msc_sr/components/dsp_zl38063/tw_ldfwcfg.c diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_spi_access.c b/examples/lyratd_msc_sr/components/dsp_zl38063/tw_spi_access.c similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_spi_access.c rename to examples/lyratd_msc_sr/components/dsp_zl38063/tw_spi_access.c diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_spi_access.h b/examples/lyratd_msc_sr/components/dsp_zl38063/tw_spi_access.h similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/tw_spi_access.h rename to examples/lyratd_msc_sr/components/dsp_zl38063/tw_spi_access.h diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/vprocTwolf_access.c b/examples/lyratd_msc_sr/components/dsp_zl38063/vprocTwolf_access.c similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/vprocTwolf_access.c rename to examples/lyratd_msc_sr/components/dsp_zl38063/vprocTwolf_access.c diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/vprocTwolf_access.h b/examples/lyratd_msc_sr/components/dsp_zl38063/vprocTwolf_access.h similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/vprocTwolf_access.h rename to examples/lyratd_msc_sr/components/dsp_zl38063/vprocTwolf_access.h diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/vproc_common.c b/examples/lyratd_msc_sr/components/dsp_zl38063/vproc_common.c similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/vproc_common.c rename to examples/lyratd_msc_sr/components/dsp_zl38063/vproc_common.c diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/vproc_common.h b/examples/lyratd_msc_sr/components/dsp_zl38063/vproc_common.h similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/vproc_common.h rename to examples/lyratd_msc_sr/components/dsp_zl38063/vproc_common.h diff --git a/examples/lyratd_msc_alexa_sr/components/dsp_zl38063/vproc_data_types.h b/examples/lyratd_msc_sr/components/dsp_zl38063/vproc_data_types.h similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/dsp_zl38063/vproc_data_types.h rename to examples/lyratd_msc_sr/components/dsp_zl38063/vproc_data_types.h diff --git a/examples/lyratd_msc_alexa_sr/components/firmware_zl38063/component.mk b/examples/lyratd_msc_sr/components/firmware_zl38063/component.mk similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/firmware_zl38063/component.mk rename to examples/lyratd_msc_sr/components/firmware_zl38063/component.mk diff --git a/examples/lyratd_msc_alexa_sr/components/firmware_zl38063/include/zl38063_config.h b/examples/lyratd_msc_sr/components/firmware_zl38063/include/zl38063_config.h similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/firmware_zl38063/include/zl38063_config.h rename to examples/lyratd_msc_sr/components/firmware_zl38063/include/zl38063_config.h diff --git a/examples/lyratd_msc_alexa_sr/components/firmware_zl38063/include/zl38063_firmware.h b/examples/lyratd_msc_sr/components/firmware_zl38063/include/zl38063_firmware.h similarity index 100% rename from examples/lyratd_msc_alexa_sr/components/firmware_zl38063/include/zl38063_firmware.h rename to examples/lyratd_msc_sr/components/firmware_zl38063/include/zl38063_firmware.h diff --git a/examples/lyratd_msc_alexa_sr/components/firmware_zl38063/lib/libfirmware_zl38063.a b/examples/lyratd_msc_sr/components/firmware_zl38063/lib/libfirmware_zl38063.a similarity index 99% rename from examples/lyratd_msc_alexa_sr/components/firmware_zl38063/lib/libfirmware_zl38063.a rename to examples/lyratd_msc_sr/components/firmware_zl38063/lib/libfirmware_zl38063.a index ab6a756..b4093fc 100644 Binary files a/examples/lyratd_msc_alexa_sr/components/firmware_zl38063/lib/libfirmware_zl38063.a and b/examples/lyratd_msc_sr/components/firmware_zl38063/lib/libfirmware_zl38063.a differ diff --git a/examples/lyratd_msc_alexa_sr/main/Kconfig.projbuild b/examples/lyratd_msc_sr/main/Kconfig.projbuild similarity index 100% rename from examples/lyratd_msc_alexa_sr/main/Kconfig.projbuild rename to examples/lyratd_msc_sr/main/Kconfig.projbuild diff --git a/examples/lyratd_msc_alexa_sr/main/app_dsp.c b/examples/lyratd_msc_sr/main/app_dsp.c similarity index 84% rename from examples/lyratd_msc_alexa_sr/main/app_dsp.c rename to examples/lyratd_msc_sr/main/app_dsp.c index 8fc96ea..7cd11ef 100644 --- a/examples/lyratd_msc_alexa_sr/main/app_dsp.c +++ b/examples/lyratd_msc_sr/main/app_dsp.c @@ -1,24 +1,5 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. #include #include @@ -29,19 +10,17 @@ #include #include -#include -#include -#include "app_dsp.h" -#include -#include +#include #include #include #include -#include #include +#include +#include + #include "resampling.h" -//Speech recognition headers -#include +#include "app_dsp.h" +#include "ui_led.h" #define WWE_TASK_STACK (8 * 1024) #define DETECT_SAMP_RATE 16000UL @@ -71,6 +50,7 @@ static struct dsp_data { bool write_to_store; int16_t data_buf[SAMPLE_SZ]; char pcm_store[PCM_SIZE]; + bool dsp_inited; } dd; static media_hal_config_t media_hal_conf = { @@ -99,7 +79,7 @@ static ssize_t dsp_write_cb(void *h, void *data, int len, uint32_t wait) return sent_len; } -int alexa_app_speech_stop() +int va_app_speech_stop() { ESP_LOGI(TAG, "Sending stop command"); dd.speech_recog_en = false; @@ -109,7 +89,7 @@ int alexa_app_speech_stop() return ESP_OK; } -int alexa_app_speech_start() +int va_app_speech_start() { ESP_LOGI(TAG, "Sending start command"); dd.detect_wakeword = false; @@ -125,6 +105,9 @@ void app_dsp_reset() void app_dsp_send_recognize() { + if (!dd.dsp_inited) + return; + ESP_LOGI(TAG, "Sending start command"); dd.detect_wakeword = false; ESP_LOGI(TAG, "Starting I2S audio stream"); @@ -202,8 +185,6 @@ void nn_task(void *arg) void app_dsp_init(void) { dd.temp_rb = rb_init("nn-recog", 4 * 1024); - ui_led_init(); - ui_button_init(); i2s_stream_config_t i2s_cfg; memset(&i2s_cfg, 0, sizeof(i2s_cfg)); @@ -250,4 +231,5 @@ void app_dsp_init(void) vTaskDelay(10/portTICK_RATE_MS); audio_stream_start(&dd.read_i2s_stream->base); dd.detect_wakeword = true; + dd.dsp_inited = true; } diff --git a/examples/lyratd_msc_sr/main/app_dsp.h b/examples/lyratd_msc_sr/main/app_dsp.h new file mode 100644 index 0000000..221ce6b --- /dev/null +++ b/examples/lyratd_msc_sr/main/app_dsp.h @@ -0,0 +1,27 @@ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. + +#ifndef _APP_DSP_H_ +#define _APP_DSP_H_ + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +typedef enum dsp_called_states { + VA_CAN_START = (VA_END_STATES + 1), +} dsp_called_states_t; + +void app_dsp_init(void); + +void app_dsp_send_recognize(); + +void app_dsp_reset(void); + +#ifdef __cplusplus +} +#endif + +#endif /* _APP_DSP_H_ */ diff --git a/examples/lyratd_msc_alexa_sr/main/app_lyratd_msc_cb.c b/examples/lyratd_msc_sr/main/app_lyratd_msc_cb.c similarity index 50% rename from examples/lyratd_msc_alexa_sr/main/app_lyratd_msc_cb.c rename to examples/lyratd_msc_sr/main/app_lyratd_msc_cb.c index 51d1b03..62c8de7 100644 --- a/examples/lyratd_msc_alexa_sr/main/app_lyratd_msc_cb.c +++ b/examples/lyratd_msc_sr/main/app_lyratd_msc_cb.c @@ -1,33 +1,18 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. #include #include -#include +#include #include #include #include #include +#if DIALOGFLOW +#include +#endif + #define I2S_PORT_NUM I2S_NUM_0 #define DES_BITS_PER_SAM 16 #define SRC_BITS_PER_SAM 16 @@ -45,30 +30,30 @@ static inline int heap_caps_get_free_size_sram() return heap_caps_get_free_size(MALLOC_CAP_8BIT) - heap_caps_get_free_size(MALLOC_CAP_SPIRAM); } -void alexa_app_dialog_states(alexa_dialog_states_t alexa_states) +void va_app_dialog_states(va_dialog_states_t va_state) { - ui_led_set(alexa_states); - ESP_LOGI(TAG, "Current mode is: %d", alexa_states); + ui_led_set(va_state); + ESP_LOGI(TAG, "Current mode is: %d", va_state); } -int alexa_app_set_volume(int vol) +int va_app_set_volume(int vol) { zl38063_control_volume(vol); ESP_LOGI(TAG, "Volume changed to %d", vol); return 0; } -int alexa_app_set_mute(alexa_mute_state_t alexa_mute_state) +int va_app_set_mute(va_mute_state_t va_mute_state) { uint8_t current_vol; - if (alexa_mute_state == ALEXA_MUTE_ENABLE) { + if (va_mute_state == VA_MUTE_ENABLE) { zl38063_set_mute(1); } else { zl38063_set_mute(0); zl38063_get_volume(¤t_vol); zl38063_control_volume(current_vol); } - ESP_LOGI(TAG, "Mute: %d", alexa_mute_state); + ESP_LOGI(TAG, "Mute: %d", va_mute_state); return 0; } @@ -77,17 +62,17 @@ int alexa_app_raise_alert(alexa_alert_types_t alexa_alert_type, alexa_alert_stat return 0; } -int alexa_app_playback_data(alexa_resample_param_t *alexa_resample_param, void *buf, ssize_t len) +int va_app_playback_data(va_resample_param_t *va_resample_param, void *buf, ssize_t len) { memset(convert_buf, 0, BUF_SZ); - audio_resample_config_t resample = {0}; + static audio_resample_config_t resample = {0}; int current_convert_block_len; int convert_block_len = 0; int send_offset = 0; size_t sent_len = 0; int conv_len = 0; - if (alexa_resample_param->alexa_resample_ch == 1) { + if (va_resample_param->va_resample_ch == 1) { /* If mono recording, we need to up-sample, so need half the buffer empty, also uint16_t data*/ convert_block_len = CONVERT_BUF_SIZE / 4; } else { @@ -100,9 +85,9 @@ int alexa_app_playback_data(alexa_resample_param_t *alexa_resample_param, void * if (current_convert_block_len & 1) { printf("Odd bytes in up sampling data, this should be backed up\n"); } - conv_len = audio_resample((short *)((char *)buf + send_offset), (short *)convert_buf, alexa_resample_param->alexa_resample_freq, SAMPLING_RATE, - current_convert_block_len / 2, BUF_SZ, alexa_resample_param->alexa_resample_ch, &resample); - if (alexa_resample_param->alexa_resample_ch == 1) { + conv_len = audio_resample((short *)((char *)buf + send_offset), (short *)convert_buf, va_resample_param->va_resample_freq, SAMPLING_RATE, + current_convert_block_len / 2, BUF_SZ, va_resample_param->va_resample_ch, &resample); + if (va_resample_param->va_resample_ch == 1) { conv_len = audio_resample_up_channel((short *)convert_buf, (short *)convert_buf, SAMPLING_RATE, SAMPLING_RATE, conv_len, BUF_SZ, &resample); } len -= current_convert_block_len; @@ -142,3 +127,64 @@ int i2s_playback_init() ret = i2s_zero_dma_buffer(I2S_PORT_NUM); return ret; } + +#if DIALOGFLOW +void dialogflow_app_response_data(Google__Cloud__Dialogflow__V2beta1__StreamingDetectIntentResponse *response) +{ + int i; + bool end_conversation = false; + printf("The query text is: %s\n", response->query_result->query_text); + printf("The fulfillment text is: %s\n", response->query_result->fulfillment_text); + + Google__Protobuf__Struct *entities = response->query_result->parameters; + if (!entities) { + return; + } + + /* Diagnostics information can be used to mark the end of a conversation. Make sure that "Set this intent as end + * of conversation" is enabled under "Responses" tab in each intent of your Dialogflow agent + */ + Google__Protobuf__Struct *info = response->query_result->diagnostic_info; + if (info) { + for (i = 0; i < info->n_fields; i++) { + switch (info->fields[i]->value->kind_case) { + case GOOGLE__PROTOBUF__VALUE__KIND_BOOL_VALUE: + if (!strncmp(info->fields[i]->key, "end_conversation", strlen(info->fields[i]->key)) && + info->fields[i]->value->bool_value) { + end_conversation = true; + } + break; + default: + break; + } + } + } + + if (end_conversation) { + printf("Dialogflow conversation complete:\nNumber of entities: %d\n", entities->n_fields); + for (i = 0; i < entities->n_fields; i++) { + switch (entities->fields[i]->value->kind_case) { + case GOOGLE__PROTOBUF__VALUE__KIND_NUMBER_VALUE: + printf("[%d] %s : %f\n", i, entities->fields[i]->key, entities->fields[i]->value->number_value); + break; + case GOOGLE__PROTOBUF__VALUE__KIND_STRING_VALUE: + printf("[%d] %s : %s\n", i, entities->fields[i]->key, entities->fields[i]->value->string_value); + break; + case GOOGLE__PROTOBUF__VALUE__KIND_BOOL_VALUE: + printf("[%d] %s : %s\n", i, entities->fields[i]->key, entities->fields[i]->value->bool_value ? "true" : "false"); + break; + default: + printf("Unknown type %d: Please update this debug function\n", entities->fields[i]->value->kind_case); + break; + } + } + } else { + printf("Dialogflow conversation continued\n"); + } +} + +void dialogflow_app_query_text_transcript(char *text, bool is_final) +{ + printf("%s transcript is: %s\n", (is_final != 0 ? "Final" : "Interim"),text); +} +#endif diff --git a/examples/lyratd_msc_alexa_sr/main/app_main.c b/examples/lyratd_msc_sr/main/app_main.c similarity index 71% rename from examples/lyratd_msc_alexa_sr/main/app_main.c rename to examples/lyratd_msc_sr/main/app_main.c index 9f7c65b..78f47c2 100644 --- a/examples/lyratd_msc_alexa_sr/main/app_main.c +++ b/examples/lyratd_msc_sr/main/app_main.c @@ -21,25 +21,32 @@ #include #include +#include +#ifdef ALEXA #include +#elif GVA +#include +#elif DIALOGFLOW +#include +#endif + #include #include -#include +#include #include #include #include "app_dsp.h" #include "ui_led.h" +#include "ui_button.h" #define SOFTAP_SSID_PREFIX "ESP-Alexa-" -static const char *TAG = "alexa"; +static const char *TAG = "app_main"; static EventGroupHandle_t cm_event_group; const int CONNECTED_BIT = BIT0; -static alexa_config_t *alexa_cfg; - static esp_err_t event_handler(void *ctx, system_event_t *event) { conn_mgr_prov_event_handler(ctx, event); @@ -74,15 +81,21 @@ extern int i2s_playback_init(); int app_main() { - ESP_LOGI(TAG, "==== Alexa SDK version: %s ====", alexa_get_sdk_version()); - - alexa_cfg = va_mem_alloc(sizeof(alexa_config_t), VA_MEM_EXTERNAL); - if (!alexa_cfg) { - ESP_LOGE(TAG, "Failed to alloc alexa config"); + ESP_LOGI(TAG, "==== Voice Assistant SDK version: %s ====", va_get_sdk_version()); + +#ifdef ALEXA + alexa_config_t *va_cfg = va_mem_alloc(sizeof(alexa_config_t), VA_MEM_EXTERNAL); +#elif GVA + gva_config_t *va_cfg = va_mem_alloc(sizeof(gva_config_t), VA_MEM_EXTERNAL); +#elif DIALOGFLOW + dialogflow_config_t *va_cfg = va_mem_alloc(sizeof(dialogflow_config_t), VA_MEM_EXTERNAL); +#endif + + if (!va_cfg) { + ESP_LOGE(TAG, "Failed to alloc voice assistant config"); abort(); } - - alexa_cfg->auth_delegate.type = auth_type_subsequent; + va_cfg->auth_delegate.type = auth_type_subsequent; // Initialize NVS esp_err_t ret = nvs_flash_init(); @@ -93,11 +106,13 @@ int app_main() ESP_ERROR_CHECK( ret ); scli_init(); - diag_register_cli(); + va_diag_register_cli(); wifi_register_cli(); + ui_led_init(); + ui_button_init(); cm_event_group = xEventGroupCreate(); - tcpip_adapter_init(); + ESP_ERROR_CHECK(esp_event_loop_init(event_handler, NULL) ); bool provisioned = false; @@ -114,7 +129,7 @@ int app_main() conn_mgr_prov_t prov_type = conn_mgr_prov_mode_softap; prov_type.event_cb = alexa_conn_mgr_prov_cb; - prov_type.cb_user_data = (void *)alexa_cfg; + prov_type.cb_user_data = (void *)va_cfg; int security = 1; const char *pop = "abcd1234"; const char *service_key = ""; @@ -129,11 +144,24 @@ int app_main() i2s_playback_init(); app_dsp_init(); - if (alexa_init(alexa_cfg) != ESP_OK) { + +#ifdef ALEXA + ret = alexa_init(va_cfg); +#elif GVA + va_cfg->device_config.device_model = "device-model-default"; // Enter your model id (name) here + va_cfg->device_config.device_id = "device-id-default"; // Enter your (unique) device id here + ret = gva_init(va_cfg); +#elif DIALOGFLOW + va_cfg->device_config.project_name = "project-name-default"; // Enter your dialogflow project name here. + ret = dialogflow_init(va_cfg); +#endif + + if (ret != ESP_OK) { while(1) vTaskDelay(2); } - va_mem_free(alexa_cfg); - ui_led_set(ALEXA_IDLE); + + va_mem_free(va_cfg); + ui_led_set(VA_IDLE); return ESP_OK; } diff --git a/examples/lyratd_msc_alexa_sr/main/component.mk b/examples/lyratd_msc_sr/main/component.mk similarity index 100% rename from examples/lyratd_msc_alexa_sr/main/component.mk rename to examples/lyratd_msc_sr/main/component.mk diff --git a/examples/lyratd_msc_alexa_sr/main/ui_button.c b/examples/lyratd_msc_sr/main/ui_button.c similarity index 84% rename from examples/lyratd_msc_alexa_sr/main/ui_button.c rename to examples/lyratd_msc_sr/main/ui_button.c index 847c93a..6f2cf6e 100644 --- a/examples/lyratd_msc_alexa_sr/main/ui_button.c +++ b/examples/lyratd_msc_sr/main/ui_button.c @@ -1,30 +1,11 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. #include #include - #include #include + #include #include #include @@ -32,8 +13,14 @@ #include #include #include -#include -#include +#include + +#include "speaker.h" +#include "app_dsp.h" +#include "avs_nvs_utils.h" +#include "zl38063.h" +#include "ui_button.h" +#include "ui_led.h" #define BUTTON_TAG "UI_BUTTON" @@ -51,14 +38,6 @@ typedef struct { uint16_t but_high_val; //number of data } button_val_range_t; -#include "ui_button.h" -#include -#include "speaker.h" -#include "app_dsp.h" -#include "avs_nvs_utils.h" -#include "ui_led.h" -#include "zl38063.h" - //#define EN_AUDIO_JACK #define VOLUME_STEP 5 diff --git a/examples/lyratd_msc_alexa_sr/main/ui_button.h b/examples/lyratd_msc_sr/main/ui_button.h similarity index 54% rename from examples/lyratd_msc_alexa_sr/main/ui_button.h rename to examples/lyratd_msc_sr/main/ui_button.h index b06d4eb..0ba2e06 100644 --- a/examples/lyratd_msc_alexa_sr/main/ui_button.h +++ b/examples/lyratd_msc_sr/main/ui_button.h @@ -1,24 +1,5 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. #ifndef _UI_BUTTON_H_ #define _UI_BUTTON_H_ diff --git a/examples/lyratd_msc_alexa_sr/main/ui_led.c b/examples/lyratd_msc_sr/main/ui_led.c similarity index 85% rename from examples/lyratd_msc_alexa_sr/main/ui_led.c rename to examples/lyratd_msc_sr/main/ui_led.c index d23e5ac..e2c973d 100644 --- a/examples/lyratd_msc_alexa_sr/main/ui_led.c +++ b/examples/lyratd_msc_sr/main/ui_led.c @@ -1,24 +1,5 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. #include #include @@ -234,16 +215,16 @@ static esp_err_t ui_led_ring() return res; } -esp_err_t ui_led_set(int alexa_states) +esp_err_t ui_led_set(int va_state) { - if (alexa_states != ALEXA_CAN_START) { + if (va_state != VA_CAN_START) { led_st.ui_led_init_bool = false; } #ifdef EN_STACK_MEASUREMENT ESP_LOGI("UI_LED_TAG", "Free Task Stack is: %s %u\n\n\n", __func__, uxTaskGetStackHighWaterMark(led_st.ui_led_task_handle)); #endif - if (xQueueSend(led_st.ui_led_queue, &alexa_states, 0) != pdTRUE) { - ESP_LOGE(UI_LED_TAG, "Cannot send alexa state"); + if (xQueueSend(led_st.ui_led_queue, &va_state, 0) != pdTRUE) { + ESP_LOGE(UI_LED_TAG, "Cannot send assistant state"); return ESP_FAIL; } return ESP_OK; @@ -251,13 +232,13 @@ esp_err_t ui_led_set(int alexa_states) static void ui_led_task(void *arg) { - int alexa_states; + int va_state; while (1) { - if (xQueueReceive(led_st.ui_led_queue, &alexa_states, portMAX_DELAY) != pdTRUE) { + if (xQueueReceive(led_st.ui_led_queue, &va_state, portMAX_DELAY) != pdTRUE) { ESP_LOGE(UI_LED_TAG, "Failed to receive from led queue"); } - switch (alexa_states) { - case ALEXA_CAN_START : + switch (va_state) { + case VA_CAN_START : while (led_st.ui_led_init_bool == true) { ui_led_solid(100); vTaskDelay(333/portTICK_RATE_MS); @@ -265,17 +246,17 @@ static void ui_led_task(void *arg) vTaskDelay(333/portTICK_RATE_MS); } break; - case ALEXA_IDLE : + case VA_IDLE : //Do nothing ui_led_solid(0); break; - case ALEXA_LISTENING : + case VA_LISTENING : ui_led_solid(100); break; - case ALEXA_THINKING : + case VA_THINKING : ui_led_ring(); break; - case ALEXA_SPEAKING : + case VA_SPEAKING : //Do nothing ui_led_solid(0); break; @@ -319,6 +300,6 @@ esp_err_t ui_led_init() ESP_LOGE(UI_LED_TAG, "Could not creake ui led task"); return ESP_FAIL; } - ui_led_set(ALEXA_CAN_START); + ui_led_set(VA_CAN_START); return ret; } diff --git a/examples/lyratd_msc_alexa_sr/main/ui_led.h b/examples/lyratd_msc_sr/main/ui_led.h similarity index 83% rename from examples/lyratd_msc_alexa_sr/main/ui_led.h rename to examples/lyratd_msc_sr/main/ui_led.h index a0865e6..d7d393e 100644 --- a/examples/lyratd_msc_alexa_sr/main/ui_led.h +++ b/examples/lyratd_msc_sr/main/ui_led.h @@ -1,24 +1,5 @@ -/* - * Copyright 2018, Espressif Systems (Shanghai) Pte Ltd. - * All rights regarding this code and its modifications reserved. - * - * This code contains confidential information of Espressif Systems - * (Shanghai) Pte Ltd. No licenses or other rights express or implied, - * by estoppel or otherwise are granted herein. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - */ +// Copyright 2018 Espressif Systems (Shanghai) PTE LTD +// All rights reserved. #ifndef _UI_LED_H_ #define _UI_LED_H_ @@ -31,7 +12,7 @@ #include "driver/i2c.h" #include "esp_log.h" #include "esp_system.h" -#include +#include #ifdef __cplusplus extern "C" { @@ -203,7 +184,7 @@ enum { LED_OFF, }; -esp_err_t ui_led_set(int alexa_states); +esp_err_t ui_led_set(int va_state); void ui_led_set_colour(ui_led_set_colour_t ui_led_colour); diff --git a/examples/lyratd_msc_alexa_sr/partitions.csv b/examples/lyratd_msc_sr/partitions.csv similarity index 100% rename from examples/lyratd_msc_alexa_sr/partitions.csv rename to examples/lyratd_msc_sr/partitions.csv diff --git a/examples/lyrat_alexa_sr/sdkconfig.defaults b/examples/lyratd_msc_sr/sdkconfig.defaults similarity index 98% rename from examples/lyrat_alexa_sr/sdkconfig.defaults rename to examples/lyratd_msc_sr/sdkconfig.defaults index 3614629..966d92e 100644 --- a/examples/lyrat_alexa_sr/sdkconfig.defaults +++ b/examples/lyratd_msc_sr/sdkconfig.defaults @@ -76,6 +76,7 @@ CONFIG_FREERTOS_USE_TRACE_FACILITY=y CONFIG_LWIP_MAX_SOCKETS=4 CONFIG_USE_ONLY_LWIP_SELECT=y CONFIG_LWIP_ETHARP_TRUST_IP_MAC=y +CONFIG_LWIP_DHCP_MAX_NTP_SERVERS=3 # # TCP