Add ESP32 to supported devices with basic type hints

[Gor-Ren]

Relates to #44

Changes

Made ESP32 a supported device, appearing as a device in the MicroPython settings list with auto-detect support.

• Added Esp32DeviceProvider class, largely a copy of the Esp8266 one
  • Tested manually using ./gradlew clean runIde
  • Confirmed auto-detect is working, at least for my ESP32-WROOM-32 MCU
• Add /typehints/esp32/ dir, initially populated with the same files as ESP8266 - to be expanded in later PRs

Out of Scope (but happy to discuss if you think it should be in this PR 😄)

• expanding ESP32 typehints
  • I think this will require a change to how typehints are aggregated, because some devices will need to "override" common library stuff with extra methods. I have some thoughts on how to do this that could be discussed separately.

CLA
I've signed it.

[Gor-Ren]

These are copied from the ESP8266.

To find the values for an ESP32, I connected MCUs by USB and ran $ cat /sys/bus/usb-serial/devices/ttyUSB0/../uevent. I tested with an ESP8266 and an ESP32 and both output PRODUCT=10c4/ea60/100. I was a bit surprised - I expected them to be different.

PRODUCT=10c4/ea60/100 corresponds to the second element in this list. Auto-detect of my ESP32 subsequently worked. Shall we keep all the values copied from ESP8266 the same, then?

[Gor-Ren]

expanding ESP32 typehints
I think this will require a change to how typehints are aggregated, because some devices will need to "override" common library stuff with extra methods.

To expand on this, one example would be the machine.ADC class, which has a simple generic interface for any board (docs) but has more methods available specifically on the ESP32 (docs).

This presents a problem: if we have a generic /micropython/machine::ADC and a more specific /esp32/machine::ADC we have a name clash... from testing, it seems that the type machinery picks the micropython module's ADC implementation and we get red squiggles trying to use the ESP32-specific stuff (cannot find reference 'xyz' in 'machine.pyi').

There's a similar issue in the network module, where Micropython provides a universal network package containing an abstract NIC class and some constants, and then various board-specific implementations are available. How do we merge the universal network and board-specific network modules?

I was thinking that board-specific type hint directories could maybe have an __init__.py which could co-ordinate what gets imported from the generic micropython directory or overriden. Will need to test if this is actually feasible.

[jsonpoindexter]

@Gor-Ren thank you for taking this on. I would like to test your added support for ESP32 but I am running into issue.
I have cloned your https://github.com/Gor-Ren/intellij-micropython/tree/support-esp32, branch opened the project in IntelliJ IDEA Community2020.2 Beta, and ran runIde from Gradel. It builds the plugin and launched PyCharm 2020.1 with the micropython plugin prebuilt in it. When I go to Settings -> Languages & Fameworks -> MicroPython I do not see ESP32 under the Device Type dropdown. Am I doing something wrong? Thank you.

[Gor-Ren]

@jsonpoindexter hrm, I can't seem to reproduce

• starting from IntelliJ 2020.2 with the latest MicroPython 1.1.1 and the support-esp32 branch
• (venv) gor@gor-desktop:~/dev/intellij-micropython$ ./gradlew clean runIde
• PyCharm dev window opens
• File -> Settings -> Languages & Frameworks -> MicroPython -> "Device type" dropdown -> ESP32 is in the list

maybe make sure you are running ./gradlew clean runIde with the clean - without it, it might be loading a previously deployed version I guess.

As a sanity check you could try loading the SNAPSHOT build into your IDE:

• (venv) gor@gor-desktop:~/dev/intellij-micropython$ ./gradlew clean buildPlugin
• you will now have a dev plugin ready at .../intellij-micropython/build/distributions/intellij-micropython-SNAPSHOT.zip
• File -> Settings -> Plugins -> find "MicroPython"
• click settings cog for MicroPython at top -> "Install plugin from disk" -> select intellij-micropython-SNAPSHOT.zip you built -> Apply -> Restart IDE
• on restart you should see MicroPython is on version "SNAPSHOT" and have access to the ESP32 in the dropdown
  • if on IntelliJ, the settings are in File -> Project Settings -> Facets -> MicroPython

[kdschlosser]

Question on this concerning the stub files. Why can't you change the library root to a directory specific to a type of board and that directory would contain the stub files needed?

[gordon-rennie]

Question on this concerning the stub files. Why can't you change the library root to a directory specific to a type of board and that directory would contain the stub files needed?

Many of the stubs describe common code to all boards. Therefore if possible it's desirable to consolidate it in one place, not to duplicate them across every board, which would be an onerous maintenance burden.

Of course it starts to break down if the "common" code turns out to be a lot more custom for each board (which we are seeing with some methods in Machine for example) and maybe duplication is unavoidable. But I would like some feedback from @vlasovskikh before proposing such a change. Also, there may be ways in Python to cherrypick import code into packages, perhaps using the package's __init__.py --- I'm not knowledgeable on the subject, though.

[kdschlosser]

I can understand the duplicate code thing. I know it would not be ideal but it is a solution.
Another question..
Does the order in which the library directories being added have anything to do with what stub file gets used first?

T do not know how much manipulation is allowed to be done with PyCharm I am thinking that if you can make the directory for the micropython stubs as a sub directory of a boards stub files then you would be able to add the board stub file folder as the library root and your problem would be solved. The directory for the micropython stub files being a sub directory of the board would only exist in PyCharm and not on the filesystem. Tho you could use symbolic links to accomplish the same thing as well.

I am going to run some tests to try and manipulate the library root. maybe it can be made to behave the way that is needed.

[kdschlosser]

Stub files are there only for the purposes of having an IDE work properly. I do not think they are there for any other purpose.

I am the administrator of a project called EventGhost and this application is written mostly in Python. It is fairly large at around 200,000 lines of code. Some of the modules in it need to be loaded in a very specific order and we also wanted to keep memory management and the resources under control. We do this by only importing a modules when it is actually needed. This is done by wrapping a class around the main module and using __getattr__. The problem with this is an IDE is not able to offer code completion and type hints for those moduels because there is no hard coded import of the modules.

This is what I did to make everything work correctly.

import os

if 'THIS_CODE_NEVER_REALLY_RUNS' in os.environ:
    from Classes import SomeModule as _SomeModule
    SomeModule = _SomeModule

Because PyCharm thinks that the code has a possibility of running it then scans the modules and allows all of the goodies in an IDE to work properly. This kind of manipulation can also be done as well in a pyi file. You would need to add in the imports of the board specific pyi files at the end the question now becomes how can this be done without the user being able to see the board specific files in the library root.

[amhendley]

Looks like a missed comment update. Still referring to ESP8266. Not ESP32.