ESP32 port #9426

gschorcht · 2018-06-27T11:03:34Z

Description

This is the first working version of the RIOT-OS port for ESP32. Even though it is the first commit, most of RIOT's features are implemented and working, see the current status below. All related tests I executed from directory tests passed. Networking is realized using either

external IEEE 802.15.4 radio modules.
internal WiFi with AP-less ESPNOW
internal Ethernet interface

Boards

The inital commit includes 5 board definitions:

ESP32 DevKit and clones (generic)
ESP-WROVER-KIT
MH-ET Live MiniKit (pin compatible with Wemos D1 mini)
Olimex ESP32-EVB
Wemos Lolin D32

Current status (2018-07-25)

context saving and restoring (implementing the core's thread inteface) is implemented and tested
interrupt handling (the core's irq interface) is implemented and tested
GPIO module periph_gpio is implemented and tested
Hardware random number generator periph_hwrng implemented and tested
CPU id module periph_cpuid implemented and tested
timer module perihph_timer implemented and tested
UART module periph_uart implemented and tested
SPI module periph_spi implemented and tested
I2C module periph_i2c implemented and tested
PWM module periph_pwm implemented and tested
ADC module periph_adc implemented and tested
DAC module periph_dac implemented and tested
RTC module periph_rtc implemented and tested
MTD device implemented for flash and can be used with SPIFFS and VFS
SPI RAM implemented and tested
shell and a lot of shell-based test applications are working
networking using external IEEE 802.15.4 transceiver modules (MRF24J40) implemented
netdev using built-in WiFi module with the AP less ESP-NOW technology implemented (compatible to ESP8266), module esp_now
netdev using built-in Ethernet interface (EMAC) implemented, module esp_eth
CAN device driver implemented, module esp_can

Limitations

The implementation uses only one core (the PRO CPU) because RIOT does not support running multiple threads simultaneously.
WiFi works only in AP-less mode with Espressif's ESPNOW technology. Even though it is possible to connect to an AP, at the moment there are no network or transport layer functions implemented.
Bluetooth cannot be used at the moment.
Flash encryption is not yet supported.

Current work

extend built-in WiFi based networking using an AP

Next steps

implement deep sleep mechanisms
Bluetooth based networking (don't know whether it is really possible without ESP-IDF)

Issues / PRs

depends on #10145

gschorcht · 2018-06-27T15:14:48Z

Just splitted the commit in several packages.

kYc0o · 2018-06-29T14:14:54Z

why closed?

gschorcht · 2018-06-29T14:21:34Z

@kYc0o Just by mistake :-( I just wanted to recommit the port to add some minor changes and the new periph_rtc module. So I resetted by branch to the master, added and committed all the files again. After forced push (at least I thought it was a push), the PR was closed and the branch is empty in origin.

kYc0o · 2018-06-29T14:24:59Z

wow! Ok, I hope you can restablish everything. I'm willing to review and test. However I only have a sha2017 badge which embeds a ESP-WROOM32. I hope with few modifications I can make it work with this PR.

gschorcht · 2018-06-29T14:41:18Z

@kYc0o I had a local version of the repository. So I could uploaded the complete commit.

Regarding you board, I can take a look what it is. The port should work with all WROVER and WROOM boards with some modifications in board definition.

gschorcht · 2018-06-29T14:44:55Z

@kYc0o SHA2017 Badge seems to use the touch sensors. Touch sensor inputs are not supported at the moment since RIOT has no low level interface for touch sensors. The according GPIOs can be used as ADC channels or digial I/O.

kYc0o · 2018-06-29T16:13:23Z

Yes, no hurries about it, the most important thing is to have the basic stuff merged, the rest can be added in follow-up PRs.

jia200x · 2018-10-15T09:23:56Z

@jia200x Thanks. So the CI: ready for build label could be set again?

Nein :( The workers need to be restarted. We will arrange that.

gschorcht · 2018-10-15T10:43:22Z

@jia200x How long will this take. Even though I have compiled all the tests with all the boards and I do not expect any surprises, I'm a little curious if murdock gives the same results.

smlng · 2018-10-15T13:21:25Z

I just updated all murdock workers, so it should in theory work now 🤞

gschorcht · 2018-10-15T14:04:37Z

@smlng compilations on haw-icc always fail with: Compiler xtensa-esp32-elf-gcc is required but not found in PATH. Does it take a while before the update of the workers become effective.

gschorcht · 2018-10-15T15:17:20Z

@smlng @miri64 Hm, it seems that murdock failed due to compilation errors that are not related to this PR 😟 Who should take a look?

compile/tests/cpp11_mutex/mulle:llvm
compile/tests/cpp11_mutex/iotlab-m3:llvm
compile/tests/cpp11_mutex/nrf52dk:llvm
compile/tests/cpp11_mutex/samr21-xpro:llvm
compile/tests/cpp11_mutex/nucleo-f401re:llvm
compile/tests/cpp11_mutex/slstk3402a:llvm
compile/tests/cpp11_condition_variable/nrf52dk:llvm
compile/tests/cpp11_condition_variable/nucleo-f401re:llvm
compile/tests/cpp11_condition_variable/samr21-xpro:llvm
compile/tests/cpp11_condition_variable/iotlab-m3:llvm
compile/tests/cpp11_condition_variable/mulle:llvm compile/tests/cpp11_condition_variable/slstk3402a:llvm
compile/tests/cpp11_thread/nrf52dk:llvm
compile/tests/cpp11_thread/mulle:llvm
compile/tests/cpp11_thread/nucleo-f401re:llvm
compile/tests/cpp11_thread/iotlab-m3:llvm
compile/tests/cpp11_thread/samr21-xpro:llvm
compile/examples/riot_and_cpp/iotlab-m3:llvm
compile/examples/riot_and_cpp/nucleo-f401re:llvm

jia200x · 2018-10-15T15:54:55Z

Who should take a look?

I'm already taking care of this.

The good news are: it seems there are no ESP32 issues 🎉

jia200x · 2018-10-16T12:17:03Z

@gschorcht btw thank you so much for the efforts you put in documentation

jia200x · 2018-10-16T13:19:02Z

Murdock is green!

jia200x

Ok, everything seems to be OK. Code looks good and the platform passed some extensive tests (thanks @MrKevinWeiss). Thanks for all the efforts to @gschorcht and all people involved.

Let's get this in the Release 2018.10. ACK

miri64 · 2018-10-16T13:43:56Z

🎉

waehlisch · 2018-10-16T14:11:54Z

super cool!

gschorcht · 2018-10-17T08:35:24Z

Thank you all for your support, especially @MrKevinWeiss for the intensive testing of the port.

emmanuelsearch · 2018-10-17T09:34:07Z

@gschorcht and everyone else involved: congrats!

miri64 · 2018-12-06T11:35:04Z

cpu/esp32/esp-now/esp_now_netdev.c

+        #ifdef MODULE_GNRC_IPV6_NIB
+        /* for other addresses, try to find an entry in NIB cache */
+        gnrc_ipv6_nib_nc_t nce;
+        int ret = gnrc_ipv6_nib_get_next_hop_l2addr (&ipv6_hdr->dst, dev->netif,


What is IPv6 stuff doing here oO

Good question, it is long time ago 😟

As I remember, the first packet snippet the netdev driver gets in the _send function is the IPv6 header. If the IPv6 destination address is not a link local address, the destination MAC address cannot be derived from the IPv6 destination address. Of course, I would have the same problem when privacy extensions are used. So I thought, that nib is the right way to get the mapping from an IPv6 destination address to the next hop MAC address. Maybe, I was wrong. In that case the question is how it should be.

From my understanding, IPv6 is responsible to find the next hop IPv6 address from the routing table and to use the neighbor discovery protocol to determine the next hop MAC address. This MAC address is then given as destination parameter of the MA_UNIDATA.request primitive to the L2 interface.

I couldn't figure out how this is exactly realized in GNRC.

This is not supposed to go in the network device. GNRC's IPv6 module should take care of all of that. If the link-layer is not multicast/broadcast capable, special care needs to be taken anyway.

This is what I feared. ESP-NOW is unfortunately a peer-to-peer network and not a meshed network 😟 That is, each message would have to be sent to each known peer as MAC unicast to simulate the broadcast. I thought I could do this hack with nib to save time by trying to figure out the peer based on IPv6 address and then only to send the message as a single unicast message. It doesn't matter if we have only 2 or 3 peers. But when the number of peers increases ...

@miri64
You said that the whole file looks like a hack. That might be, but the ESP32 port was a big project in parallel to my regular business and I was happy to get it working with WiFi at all even if it is not yet perfect. Now, where I will have some time over the comming holiday season, I would like to improve or correct it.

Therefore, I would like to ask you to give me a short guide which modules have to be used and how to get a MAC header as the first packet snippet in the _send function of the netdev driver, especially if I have Ethernet addresses, but none MAC according to IEEE 805.14 or IEEE 802.3.

I tried again to figure out how GNRC IPv6 works with netdev_eth and enc28j60, but I'm still not sure. Do I understand it correctly that the _send function of enc28j60 receives a preformatted IEEE802.3 or Ethernet II MAC header as the first packet snippet in the parameter iolist? If so, would it be an appropriate way to change the esp_now netdev driver to be rather an adoption layer that emulates an Ethernet MAC layer over ESP-NOW and simply to use it like an netdev_eth device?

You are right about that. The raw netif explicitly strips that header away right here: https://github.com/RIOT-OS/RIOT/blob/master/sys/net/gnrc/netif/gnrc_netif_raw.c#L97

I got an initial (but yet fully untested beyond compiling it) version of the rework of this on https://github.com/TimoRoth/RIOT

Am I right that the reason why ESP-NOW gets the IPv6 header as the first header instead of the netif header with the required L2 information because it creates a netif thread of type raw with function gnrc_netif_raw_create?

Yepp. (also as a side-note: the current gnrc_netif concept is a bit different and younger than the state of my master thesis)

But to restate my comment from #10531 (comment):

The gnrc_netif_raw implementation was intended for link-layers without any addresses (e.g. SLIP) or were access to the link-layer address is not necessary.

I got an initial (but yet fully untested beyond compiling it) version of the rework of this on https://github.com/TimoRoth/RIOT

Now provided as PR to discuss on #10581.

miri64 · 2018-12-10T13:02:16Z

Just as a hint for future reference: the esp-now and esp-wifi part could have been stripped out of this PR and put into a separate one. Discussions like the one above could have been avoided otherwise.

gschorcht · 2018-12-10T13:26:36Z

@miri64 Thanks 😄

gschorcht force-pushed the esp32 branch from 2425bad to 3bd6f23 Compare June 27, 2018 12:06

kaspar030 added Type: new feature The issue requests / The PR implemements a new feature for RIOT CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Jun 27, 2018

gschorcht force-pushed the esp32 branch from 3bd6f23 to 2c7b2d6 Compare June 27, 2018 15:13

tcschmidt requested review from jia200x, MrKevinWeiss and kYc0o June 28, 2018 11:36

gschorcht force-pushed the esp32 branch from 2c7b2d6 to 7dac5fd Compare June 29, 2018 13:43

gschorcht closed this Jun 29, 2018

gschorcht force-pushed the esp32 branch from 7dac5fd to 6e6716a Compare June 29, 2018 14:07

gschorcht reopened this Jun 29, 2018

gschorcht force-pushed the esp32 branch 10 times, most recently from 4cd71cf to a74896f Compare July 29, 2018 21:39

miri64 added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Jul 30, 2018

MrKevinWeiss added the Reviewed: 3-testing The PR was tested according to the maintainer guidelines label Oct 15, 2018

smlng added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Oct 15, 2018

boards/esp32: fix of Arduino pinmap

dc6b193

smlng added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Oct 15, 2018

MrKevinWeiss added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Oct 16, 2018

jia200x added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Oct 16, 2018

jia200x added Reviewed: 5-documentation The documentation details of the PR were reviewed according to the maintainer guidelines Reviewed: 2-code-design The code design of the PR was reviewed according to the maintainer guidelines labels Oct 16, 2018

jia200x approved these changes Oct 16, 2018

View reviewed changes

MrKevinWeiss merged commit 7187bbf into RIOT-OS:master Oct 16, 2018

gschorcht deleted the esp32 branch October 17, 2018 08:48

gschorcht mentioned this pull request Nov 21, 2018

ESP32 toolchain linked to in doc.txt is broken #10432

Closed

miri64 reviewed Dec 6, 2018

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ESP32 port #9426

ESP32 port #9426

gschorcht commented Jun 27, 2018 •

edited

Loading

gschorcht commented Jun 27, 2018

kYc0o commented Jun 29, 2018

gschorcht commented Jun 29, 2018

kYc0o commented Jun 29, 2018

gschorcht commented Jun 29, 2018 •

edited

Loading

gschorcht commented Jun 29, 2018

kYc0o commented Jun 29, 2018

jia200x commented Oct 15, 2018

gschorcht commented Oct 15, 2018

smlng commented Oct 15, 2018

gschorcht commented Oct 15, 2018

gschorcht commented Oct 15, 2018

jia200x commented Oct 15, 2018

jia200x commented Oct 16, 2018

jia200x commented Oct 16, 2018

jia200x left a comment

miri64 commented Oct 16, 2018 •

edited

Loading

waehlisch commented Oct 16, 2018

gschorcht commented Oct 17, 2018

emmanuelsearch commented Oct 17, 2018

miri64 Dec 6, 2018

gschorcht Dec 6, 2018

miri64 Dec 6, 2018

gschorcht Dec 6, 2018 •

edited

Loading

gschorcht Dec 7, 2018

TimoRoth Dec 9, 2018

TimoRoth Dec 9, 2018

miri64 Dec 10, 2018

miri64 Dec 10, 2018

miri64 Dec 10, 2018

miri64 commented Dec 10, 2018

gschorcht commented Dec 10, 2018

ESP32 port #9426

ESP32 port #9426

Conversation

gschorcht commented Jun 27, 2018 • edited Loading

Description

Boards

Current status (2018-07-25)

Limitations

Current work

Next steps

Issues / PRs

gschorcht commented Jun 27, 2018

kYc0o commented Jun 29, 2018

gschorcht commented Jun 29, 2018

kYc0o commented Jun 29, 2018

gschorcht commented Jun 29, 2018 • edited Loading

gschorcht commented Jun 29, 2018

kYc0o commented Jun 29, 2018

jia200x commented Oct 15, 2018

gschorcht commented Oct 15, 2018

smlng commented Oct 15, 2018

gschorcht commented Oct 15, 2018

gschorcht commented Oct 15, 2018

jia200x commented Oct 15, 2018

jia200x commented Oct 16, 2018

jia200x commented Oct 16, 2018

jia200x left a comment

Choose a reason for hiding this comment

miri64 commented Oct 16, 2018 • edited Loading

waehlisch commented Oct 16, 2018

gschorcht commented Oct 17, 2018

emmanuelsearch commented Oct 17, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gschorcht Dec 6, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

miri64 commented Dec 10, 2018

gschorcht commented Dec 10, 2018

gschorcht commented Jun 27, 2018 •

edited

Loading

gschorcht commented Jun 29, 2018 •

edited

Loading

miri64 commented Oct 16, 2018 •

edited

Loading

gschorcht Dec 6, 2018 •

edited

Loading