ESP-ADF Pre-release v2.0-beta2

Below is the release information of ESP-ADF v2.0-beta2 and its documentation is available here.

Known Issue

• esp_audio loses duration during seek (by byte) operation.

Known issue bugfix is planned before the next v2.0 pre-release.

ADF Framework

New Features

• Added audio_pipeline_get_el_once API
• Added audio_pipeline_get_el_once API
• Added audio_pipeline_reset_elements API
• Added audio_element_report_pos API
• Added audio_element_change_cmd API
• Added audio_element_process_init API
• Added audio_element_process_deinit API
• Added support to configure the number of element's multi-input/output
• Added support for CMake build system
• Added unit test for most of components
• Added playlist function

Bugfix

• Fixed the issue that element fails to stop after multiple pause/resume operations

Services

Bugfix

• Fixed the issue that component wifi_service loses connection during the notification process

Audio Hal

New Features

• Added audio_hal_set_mute API

Audio Stream

New Features

• Added spiffs stream
• Added tone stream
• Added algorithm-related streams

Bugfix

• Fixed the issue that I2S stream pops noise in DAC mode

ESP Codec and Filter Libraries

You can find the released codec and filter libraries here.

New Features

• Downmix support a maximum of 8 input streams
• Added an audio_forge filter
• Modified Downmix to support "stereo-to-mono" via mix
• Supported markup language format of wav

Bugfix

• Fixed the issue that MP3 fails to open because the idle task does not have enough time to free memory
• Optimized speed for OPUS
• Optimized speed for AAC

ESP Recorder

New Features

• Supported esp_sr

ESP Audio

New Features

• Added set_time and time_pos parameters for esp_audio info

Bugfix

• Fixed the issue of esp_audio_destroy memory leakage

Audio Protocol

New Features

• Added a DTMF event
• Added local certificate config for TLS
• Added sip unregister during sip destroy
• Added support to keep the NAT port active for longer periods

Bugfix

• Optimized sip rtp memory
• Fixed the issue that server could fail during sip re-register
• Handled 401 unauthorized request during the invitation process

Docs

• Provided Chinese translation for ESP32-LyraT-Mini Getting Started Guide.

New Examples of ESP-ADF

Below are the added examples that demonstrate the usage of ESP-ADF components:

• examples/recorder/element_wav_amr_sdcard
• examples/recorder/pipeline_wav_amr_sdcard
• examples/get-started/pipeline_a2dp_sink_and_hfp
• examples/audio_processing/pipeline_audio_forge
• examples/player/pipeline_spiffs_mp3
• examples/player/pipeline_flash_tone

Get This Release

Using git commands below to obtain ESP-ADF v2.0-beta2 is highly recommended. Note that the submodules should also be updated (using git submodule update --init --recursive). Otherwise, the source files may be not working properly.

git clone https://github.com/espressif/esp-adf.git esp-adf-v2.0-beta2
cd esp-adf-v2.0-beta2/
git checkout v2.0-beta2
git submodule update --init --recursive

To get the programming guide for Audio Development Framework, please check here.

ESP-ADF Pre-release v2.0-beta1

Below is the release information of ESP-ADF v2.0-beta1 and its documentation is available here.

Known Issue

• Some TS formats are not supported.

Major New Features

ESP Dispatcher

ESP_Dispatcher provides a design structure that improves function reusability of existing products and shortens development cycle of new products.
New features of ESP Dispatcher Framework:

• Separate application events and actions
• Support ESP dispatcher
• Support audio service class
• Support peripheral service class
• Support independent Display, DuerOS, Player, Recorder and Wi-Fi actions

Service Framework

New Features

• Input key service
  • Support ADC, GPIO and Touch key
• Display service
  • Support LEDC display pattern
  • Support LED array display pattern
• DuerOS service
  • Support LightDuer v3.0.7
• Wi-Fi service
  • Support Wi-Fi setting by Smartconfig
  • Support Wi-Fi setting by AirKiss

ESP Peripherals

New Features

• esp_peripherals is divided into two parts: esp_periph_set and peripheral handle
• Add GPIO ISR to peripherals libraries
• Add ADC button to peripherals libraries

Peripherals

New Features

• LEDC
  • Add a new display pattern
• LED Bar
  • Add a new display pattern

Pipeline Framework

New Features

• Support event callback
• Add audio_element_set_output_ringbuf_size API
• Add audio_element_wait_for_stop_ms API
• Add audio_element_set_event_callback API
• Add a new element state: AEL_STATUS_STATE_FINISHED
• Add audio_pipeline_change_state API
• Add audio_pipeline_relink_more API
• Add rb_unblock_reader API
• Adjust audio_hal

Bugfix

• Fixed pipeline relink failure
• Fixed ringbuffer blocking issue occurred when read size or write size is larger than the size of ringbuffer

Audio Stream

New Features

• Support ALC in I2S stream
• Add more audio_type checks in HTTP stream
• Support two-level m3u8 parsing in HTTP stream

Bugfix

• Fixed HTTP stream parsing and insertion issue in relative URIs
• Changed HTTP stream from strdup to audio_strdup to optimize RAM
• Changed RAW API from raw_stream_read(char *buffer, int buf_size) to raw_stream_read(audio_element_handle_t pipeline, char *buffer, int buf_size)

ESP Speech Recognition (SR) Libraries

New Feature

• Support WakeNet5

ESP Codec and Filter Libraries

You can find the released codec and filter libraries here.

New Features

• Add ALC libraries to adjust volume
• Optimize OPUS encoder on mono
• Optimize HC_AAC speed
• Add setting APIs for downmix and equalizer

Bugfix

• Fixed the crash of WAV stream (its bit width is not 16)
• Corrected the complexity level of resample to 5

ESP Recorder

New Features

• Select specific functionality to lower binary size
• Support WWE disable
• Support customized task priority

ESP Audio

New Features

• Add media_source_type API
• Add playing info get/set API
• Add esp_audio_sync_play API
• Support memory-preferred mode and speed-preferred mode
• Support events callback
• Improve playback response time
• Remove HAL handle, replaced by callback function API

Bugfix

• Fixed raw URI failure

Audio Protocol

• Support 2.0 SIP protocol
• Support 2.0 RTP protocol
• Support RFC4566-based SDP
• Support servers: FreePBX, Yate Server and Freeswitch

Codec IC

ESP-ADF newly provides support for the following Codec ICs:

• ES8311
• ES7243

Support New Development Board

• Support ESP32-LyraT mini v1.1

New Examples of ESP-ADF

Below are the added examples that demonstrate the usage of ESP-ADF components:

• examples/audio_processing/pipeline_alc
• examples/recorder/element_cb_sdcard_amr
• examples/recorder/pipeline_opus_sdcard
• examples/advanced_examples/esp_dispatcher_dueros
• examples/advanced_examples/record_while_play
• examples/advanced_examples/voip

Get This Release

We use git submodules, therefore, the source files attached to this version will not work.
Using git commands here to obtain ESP-ADF v2.0-beta1 is highly recommended.

git clone https://github.com/espressif/esp-adf.git esp-adf-v2.0-beta1
cd esp-adf-v2.0-beta1/
git checkout v2.0-beta1
git submodule update --init --recursive

To get the programming guide for Audio Development Framework, please tap here.

ESP-ADF Release v1.0

Below is the release information of ESP-ADF v1.0. and its documentation is available here.

Known Issues

• Some TS formats are not supported.
• Music stutter may occur in touch driver that takes a lot of CPU usage.

New Functionalities

Pipeline Framework

Below are the open-source parts of ESP-ADF:

• Audio Element

  • Flexible and fully-functional APIs
  • Non-task oriented
  • Support callback input
  • Support customized tasks
  • Support multiple inputs and outputs

• Audio Pipeline

  • Flexible and fully-functional APIs
  • Non-task oriented
  • Support flexible replacement of pipeline elements

• Audio Event Interface

• RingBuffer

ESP Speech Recognition (SR) Libraries

Espressif speech recognition libraries inlcude:

• ESP VAD
  • Multiple sampling rates
  • Multiple frame sizes
• ESP SR
  • Support single wakeup word
  • Support multiple wakeup words

ESP Codec Libraries

You can find the released codec libraries here.

Decoder

• MP3-LayerIII
• WAV
• AAC-LC, AAC-HE
• TS
• AMR-NB
• AMR-WB
• OGG
• FLAC
• OPUS
• Auto-Codec

Encoder

• WAV
• AMR-WB
• AMR-NB

Filter

• Resample

  • Support
    • Single-channel to Single-channel;
    • Dual-channel to Dual-channel;
    • Single-channel to Dual-channel;
    • Dual-channel to Single-channel.
  • Up-sample
  • Down-sample
  • Support high quality mode

• EQ

  • Support 10-band equalizer

• Downmixer

  • Support 2-channel input downmixer

• Sonic (voice change)

  • Support change of voice speed
  • Support change of voice pitch

Others

• ESP Recorder

  • Triggered by APIs
  • Support stop of VAD (Voice Activity Detection) recording

• ESP Player

  • Use simple APIs
  • Played by URI
  • Recorded by URI

• DLNA protocol

  • SSDP
  • UPNP
  • DLNA-Renderer

Codec IC

ESP-ADF now supports the following Codec ICs:

• ES8388
• ES8374
• ZL38063

Supported Development Boards

The series of LyraT development boards are designed for the solutions with buttons, and the LyraTD series are designed for the far-field solutions. Currently, ESP-ADF supports the following boards:

• LyraT v4.2
• LyraT v4.3
• LyraTD-MSC v2.1
• LyraTD-MSC v2.2

Peripherals

• Wi-Fi
  • Setting Wi-Fi by Smartconfig
  • Wi-Fi event notification
• BLE
  • Setting Wi-Fi by BLE
• Touch
  • Short touch
  • Touch and hold
• Button
  • Short press
  • Press and hold
• Console
• SD Card
• LED
  • LED blink
  • LED fade blink
• LED Bar
  • LED blink
  • LED audio bar
• ADC-Button
  • Support multiple buttons
  • Support multiple ADC channels

Cloud Platform

The cloud platforms supported by ESP-ADF as planned are shown here. Currently the supported cloud platform is:

• DuerOS DCS 3.0.7

Audio Stream

ESP-ADF now supports the following audio streams:

• I2S Stream
• HTTP Stream
• FatFs Stream
• Raw Stream
• DAC Stream
• SPIFFS Stream

New Examples of ESP-ADF

Below are the added examples that demonstrates the usage of ESP-ADF components:

• ESP ASR Demo here.
• AWS Polly Demo here.
• Baidu Speech Demo here.
• Google Translate Demo here.
• DuerOS Solution here.
• Resample Demo here.
• Get-started Demos here.
• Pipeline BT Sink here.
• BT Source Demo here.
• Play HTTP Server MP3 Music here.
• Play SD Card MP3 and Controlled by Touch here.
• Recording WAV to SD Card Demo here.
• Recording WAV to HTTP Server Demo here.
• Recording AMR to SD Card Demo here.
• Equalizer Demo here.
• Voice Change Demo here.
• Elements Multi-input Demo here.
• Elements Multi-output Demo here.
• Command-line Interface Demo here.
• Flexible Replacement Of Pipeline Elements Demo here.
• HTTP Living Stream Demo here.
• DLNA Service Demo here.

Get This Release

We use git submodules, therefore, the source files attached to this version will not work.
Using git commands here to obtain ESP-ADF v1.0 is highly recommended.

git clone https://github.com/espressif/esp-adf.git esp-adf-v1.0
cd esp-adf-v1.0/
git checkout v1.0
git submodule update --init --recursive

To get the programming guide for Audio Development Framework, please tap here.

ESP-ADF Pre-release v1.0-beta1

Documentation for release v1.0-beta1 is available at: https://esp-adf.readthedocs.io/en/latest/?badge=latest

The following contains the release information for the first release of ESP-ADF (v1.0-beta1).

Known Issues

• Some TS formats are not supported

New Functionalities

Pipeline Framework

Below are the open-source parts of ESP-ADF:

• Audio Element

  • Element: flexible and fully-functional APIs
  • Element: non-task oriented
  • Support callback input
  • Support customized tasks

• Audio Pipeline

  • Flexible and fully-functional APIs
  • Non-task oriented implementation

• Audio Event Interface

• RingBuffer

ESP ADF Libraries

The planned ADF libraries are shown as below:https://github.com/espressif/esp-adf#espressif-audio-development-framework. Currently, the released libraries include:

Decoder

• MP3-LayerIII
• WAV
• AAC-LC, AAC-HE
• TS

Encoder

• WAV

Filter

• Resample
  • Support for
    • Single-channel to Single-channel;
    • Dual-channel to Dual-channel;
    • Single-channel to Dual-channel;
    • Dual-channel to Single-channel.
  • Up-sample
  • Down-sample

Others

• ESP Recorder

  • Triggered by APIs
  • Supports stop of VAD (Voice Activity Detection) recording

• ESP Player

  • Simple APIs
  • Playing by URI
  • Recording by URI

• ESP VAD

  • Multiple sampling rate
  • Multi frame size

CODEC IC

Currently the supported codec ICs include:

• ES8388
• ES8374

Supported Development Boards

The series of LyraT development boards are designed for the solutions with push buttons, and the LyraTD series are designed for the far-field solutions. Currently the supported boards are as follows:

• LyraT-V4.2
• LyraT-V4.3

Peripherals

• Wi-Fi
  • Setting Wi-Fi by Smartconfig
  • Wi-Fi event notification
• Touch
  • Short touch
  • Touch and hold
• Button
  • Short press
  • Press and hold
• Console
• SD Card
• LED
  • LED blink
  • LED fade blink

Cloud Platform

The cloud platforms supported by ADF as planned are shown as below https://github.com/espressif/esp-adf#espressif-audio-development-framework. Currently the supported cloud platform is:

• DuerOS DCS 3.0.2

Audio Stream

Currently the supported audio streams include:

• I2S Stream
• HTTP Stream
• FatFs Stream
• Raw Stream
• DAC Stream
• HTTP Live Stream

New Examples of ADF

Examples to demonstrate the usage of ADF components are as follows:

• CLI
• AWS Polly Demo
• Google Translate Demo
• Baidu Speech Demo
• DuerOS Solution
• Resample Demo
• Get-started Demos
• Pipeline BT Sink
• Play http server mp3 music
• Play SD Card MP3 and controlled by touch
• Recording WAV to SD Card Demo

Get This Release

Because we use the git submodules, the source files attached to this version will not work.

It is recommended using "git" to obtain ESP-ADF (v1.0-beta1):

git clone https://github.com/espressif/esp-adf.git esp-adf-v1.0-beta1
cd esp-adf-v1.0-beta1/
git checkout v1.0-beta1
git submodule update --init --recursive

To get the programming guide for Audio Development Framework, please refer to https://esp-adf.readthedocs.io/en/latest/?badge=latest.