ESP32 with W5500 Chip

[Yoshibeatsu]

Hi, once again :D

I hope that I dont bother you too much...

Sorry for the stupid question but is it possible to get it working on Ethernet with an esp32 and w5500 chip?

We are quite fixed on our hardware/chip choices because of this beauty.

https://www.industrialshields.com/shop/product/034001000200-esp32-plc-21-2801#attr=165,491,2238,2304,2305

We are trying for days to get it working with our esp32 and Ethernet, but but it doesn't seem to work. We tried a ntp server script only and it worked fine.

Any ideas or suggestions? Maybe some common issues if you will.

Thank you in advance

[mobizt]

There are examples using W5500 module.

This is example for ESP32 + W5500 using Ethernet client (basic client or non-secure client). This is suitable for ESP32 and ESP8266 that use the core SSL/TLS engine library.

This is example for ESP32 + W5500 using Ethernet client + external SSL client. This is for Arduino devices that this library does not support natively but through external SSL client.

I recommend the first one as it's not required external SSL client library.

[mobizt]

You need to read comments in the examples carefully.

[Yoshibeatsu]

The problem is that we can not find any information about the pin out of the SPI BUS on this product. All examples provided by the manufacturer are using codes without defined SPI Pins.

[mobizt]

That's debug printing enable when using smtp.debug(1);

All debug print means all printing information you see in the Serial monitor because you post only last line E: connection closed

[Yoshibeatsu]

Thank you for your quick answer but there is no wiring for the w5550. Its embedded in the system, there is no wiring to be changed.

I did try another example with ethernet and a ntp server and its working.

That is all I got in the serial monitor... it didnt print more

[mobizt]

This is the printing message you should see from Serial

Starting Ethernet connection...
Connected with IP xx.xx.xx.xx

Connecting to SMTP server...
> C: ESP Mail Client v2.7.10
> C: Wait for NTP server time synching

Instead of this

Connected with IP 192.168.178.75
Error, connection closed

E: connection closed

Are you sure this 192.168.178.75 is the IP of your device connected to the router directly via ethernet port?

[mobizt]

If you are using old version of library, you should update to v2.7.10 which is only available in Github repos source file only.

In Library Manager, the latest library version is 2.7.9.

[mobizt]

W5500 reset pin is not known in your device then you can remove the ResetEthernet() function and all GPIO definitions from the code which are not needed any more as I guess it is already connected to W5500 via VSPI already.

And then the networkConnection function will be

void networkConnection()
{

    Ethernet.init(5); // Assign GPIO 5 for CS.

    Serial.println("Starting Ethernet connection...");
    Ethernet.begin(Eth_MAC, Eth_IP);

    unsigned long to = millis();

    while (Ethernet.linkStatus() == LinkOFF || millis() - to < 2000)
    {
        delay(100);
    }

    if (Ethernet.linkStatus() == LinkON)
    {
        Serial.print("Connected with IP ");
        Serial.println(Ethernet.localIP());
    }
    else
    {
        Serial.println("Can't connected");
    }
}

It should work now.

[mobizt]

If provided IP doesn't work, call begin with MAC address only.
Ethernet.begin(Eth_MAC);

[Yoshibeatsu]

We tried all the mentioned points

-removed the pinouts

-im pretty sure about the ip adress because i checked it in the router

-we tried you latest libary downloaded it from here and mounted the zip

-and we also call begin with mac adress only

still the same issue

same message in the serial printer

[Yoshibeatsu]

I tried now to upload on a normal "ESP32 DOIT DevKit v1" with a W5500 Ethernet shield.

Same results as before.

and the NTP client is working also on the ESP32 DOIT DevKit v1 with W5500 Ethernet shield

[mobizt]

Please make sure, you can run this example.

The above example uses WiFIClient as a basic client which is similar to this example that you currently try that uses Ethernet client as basic client unless the network client.

If it works, you have a problem to config internal W550 chip to work with its ESP32. It's not matter what you mention that the NTP example using Ethernet works because it connects to server via non-secure UDP but if you can't use your device to work with this library ethernet example, it means W5500 chip on your device may not have enough memory buffer or it's not config properly in Ethernet library.

You should make sure you install Ethernet library v2 aka. Ethernet2 from Arduino. The rx buffer size of Ethernet module should set to 1k or more through the Ethernet library setting. This minimum buffer size of on-chip buffer is required to handle the encrypted SSL data which quite larger than non-secure data or plain text.

This may relate to W5500 firmware, chip revision, and Ethernet library setting.

[Yoshibeatsu]

Wifi script is working fine.

Ok i will research the Ethernet2 libary now

[mobizt]

Normally the W5500 module can work with non-secure connection as it not required large memory buffer as in secure connection case.

In secure connection, the ethernet chip should have large memory buffer to handle SSL cipher suite and data to be sent and received to/from SSL Client that used BearSSL (ESP8266) and mbedTLS (ESP32) as in this library.

[mobizt]

This is the issue that you can found on the internet from many users that are trying to use W5500 with other microcontrollers, especially ESP32 and ESP8266 for HTTPS web client which may fail for the first time.

If you can use these SSL client libraries to send the HTTP requests to secure server via port 443 using ESP32 and W5500 module, you can use this Email library too. Please see the example of these libraries and adapt it for your hardware.

https://github.com/mobizt/ESP_SSLClient
https://github.com/mobizt/SSLClient
https://github.com/OPEnSLab-OSU/SSLClient
https://github.com/govorox/SSLClient

[Yoshibeatsu]

I tried now https://github.com/OPEnSLab-OSU/EthernetLarge with #include <EthernetLarge.h> with the same result. It is also very strange that the serial monitor is outputting so less.....

I will read your mentioned information now

[mobizt]

You should try this SSL client library with this simple example.
https://github.com/mobizt/ESP_SSLClient#basic-usage

If it's not work, you have problem with Ethernet library or W5500 memory buffer.

[Yoshibeatsu]

Here is what I tried but I only get

Resetting WIZnet W5500 Ethernet Board...
Starting Ethernet connection...
Connected with IP 192.168.1.104
Connecting to server... failed

#include <Arduino.h>
  #if defined(ESP32) || defined(PICO_RP2040)
  #include <WiFi.h>
  #elif defined(ESP8266)
  #include <ESP8266WiFi.h>
  #endif
  #include <ESP_SSLClient.h>

#include <SPI.h>
//#include <Ethernet.h>
#include <EthernetLarge.h>
#include <ESP_SSLClient.h>


  ESP_SSLClient ssl_client;
  EthernetClient basic_client;
  

#define WIZNET_RESET_PIN 26 // Connect W5500 Reset pin to GPIO 26 of ESP32
#define WIZNET_CS_PIN 5     // Connect W5500 CS pin to GPIO 5 of ESP32
#define WIZNET_MISO_PIN 19  // Connect W5500 MISO pin to GPIO 19 of ESP32
#define WIZNET_MOSI_PIN 23  // Connect W5500 MOSI pin to GPIO 23 of ESP32
#define WIZNET_SCLK_PIN 18  // Connect W5500 SCLK pin to GPIO 18 of ESP32

unsigned long timestamp = 0;

unsigned long sentMillis = 0;


uint8_t Eth_MAC[] = {0x02, 0xF0, 0x0D, 0xBE, 0xEF, 0x01};

IPAddress Eth_IP(192, 168, 1, 104);

void ResetEthernet()
{
    Serial.println("Resetting WIZnet W5500 Ethernet Board...  ");
    pinMode(WIZNET_RESET_PIN, OUTPUT);
    digitalWrite(WIZNET_RESET_PIN, HIGH);
    delay(200);
    digitalWrite(WIZNET_RESET_PIN, LOW);
    delay(50);
    digitalWrite(WIZNET_RESET_PIN, HIGH);
    delay(200);
}


void networkConnection()
{

    Ethernet.init(WIZNET_CS_PIN);

    ResetEthernet();

    Serial.println("Starting Ethernet connection...");
    Ethernet.begin(Eth_MAC, Eth_IP);

    unsigned long to = millis();

    while (Ethernet.linkStatus() == LinkOFF || millis() - to < 2000)
    {
        delay(100);
    }

    if (Ethernet.linkStatus() == LinkON)
    {
        Serial.print("Connected with IP ");
        Serial.println(Ethernet.localIP());
    }
    else
    {
        Serial.println("Can't connected");
    }
}

// Define the callback function to handle server status acknowledgement



void setup ()
{ 
   Serial.begin(115200);

    Serial.println();

    networkConnection();
  // ignore server ssl certificate verification
  ssl_client.setInsecure();
  
  // assign the basic client
  ssl_client.setClient(&basic_client);

  Serial.print("Connecting to server...");

  String payload = "{\"title\":\"hello\"}";

  if (ssl_client.connect("reqres.in", 443))
  {
    Serial.println(" ok");
    Serial.println("Send POST request...");
    ssl_client.print("POST /api/users HTTP/1.1\n");
    ssl_client.print("Host: reqres.in\n");
    ssl_client.print("Content-Type: application/json\n");
    ssl_client.print("Content-Length: ");
    ssl_client.print(payload.length());
    ssl_client.print("\n\n");
    ssl_client.print(payload);

    Serial.print("Read response...");

    unsigned long ms = millis();
    while (!ssl_client.available() && millis() - ms < 3000)
    {
      delay(0);
    }
    
    Serial.println();
    while (ssl_client.available())
    {
      Serial.print((char)ssl_client.read());
    }
    Serial.println();
  }
  else
    Serial.println(" failed\n");

  ssl_client.stop();
}

void loop()
{

}

[Yoshibeatsu]

I really don't understand why it is not working, even without the SSL encryption

[mobizt]

It works for plain and SSL, you should see the library examples.

SSLClient works for secure connection only except for my fork version of OPEnSLab SSLClient and ESP_SSLClient.

Original OPEnSLab SSLClient cannot use for non SSL connection.

With the OPEnSLab SSL Client (fork version), here the second example I provided from my first reply.

And I post again here.

https://github.com/mobizt/ESP-Mail-Client/tree/master/examples%2FSMTP%2FExternal_Client%2FSSLClient%2FEthernet_SSLClient

The different is OPEnSLab SSL client uses BearSSL engine for all microcontrollers.

ESP_SSLClient uses BearSSL engine for ESP8266 and uses mbedTLS for ESP32.

And govorox' SSLClient uses mbedTLS engine for ESP32 only

All SSLClient libraries work with ESP32 and W5500 which I test it all already.

[mobizt]

OPEnSLab SSLClient cannot use with ESP8266 due to stack overflow.

[Yoshibeatsu]

I always get the same fault message during connection, even on without SSL examples like this one. I don't think that the SSL connection is the issue, but I don't think that the library is the issue as well, because other examples from other sources are working fine with the library

https://github.com/mobizt/ESP-Mail-Client/blob/master/examples/SMTP/External_Client/BasicClient/Ethernet_BasicClient/Ethernet_BasicClient.ino

[mobizt]

As I replied previously that the SSL engines are different.

The mbedTLS engine may required memory buffer differently (larger than required by BearSSL engine).

The mededTLS engine's SSLClient works with my ESP32 and W5500 Ethernet module and Ethernet.h library.

[mobizt]

This is the hardware buffer memory issue as I know in WIZNet W5500 chip which works differently.

[Yoshibeatsu]

[mobizt]

Is your ESP32 + W5500 works in this example?
https://github.com/mobizt/ESP-Mail-Client/tree/master/examples%2FSMTP%2FExternal_Client%2FSSLClient%2FEthernet_SSLClient

The things that OPEnSLab SSLClient can bother the user are it required trust anchors of mail server you want to connect to and does not work in ESP8266.

There is no insecure mode to skip this as in other SSL clients.

[mobizt]

Yes, call that function to get time in loop until you get timestamp more than 0

if (timestamp == 0)
{
        timestamp = ntpClient.getTime(2000 /* wait 2000 ms */);

        if (timestamp > 0)
            smtp.setSystemTime(timestamp);
}

[Yoshibeatsu]

Its the same issue.... I hate this stuff...

I changed now the void setup to this and I get the same fault in the serial monitor

void setup()
{

    Serial.begin(115200);

    Serial.println();

    networkConnection();
 
 // Begin NTP client
    ntpClient.begin();

    if (timestamp == 0)
    {
        timestamp = ntpClient.getTime(2000 /* wait 2000 ms */);

        if (timestamp > 0)
            smtp.setSystemTime(timestamp);
    }

    /** Enable the debug via Serial port
     * 0 for no debugging
     * 1 for basic level debugging
     *
     * Debug port can be changed via ESP_MAIL_DEFAULT_DEBUG_PORT in ESP_Mail_FS.h
     */
    smtp.debug(1);

    /* Set the callback function to get the sending results */
    smtp.callback(smtpCallback);

   
}

[mobizt]

For the reason Error, connection closed I will investigate what's wrong in your code. Actually, it checks for network status first by calling this callback function.

ESP-Mail-Client/examples/SMTP/External_Client/BasicClient/Ethernet_BasicClient/Ethernet_BasicClient.ino

Lines 132 to 136 in 7ffa7e8

	void networkStatusRequestCallback()
	{
	// Set the network status
	smtp.setNetworkStatus(Ethernet.linkStatus() == LinkON);
	}

I think you should try to wait until time was set before calling smtp.connect.

[mobizt]

No, you should run ntpClient.getTime in loop.

[Yoshibeatsu]

Maybe its not the time my friend... I get the same issue

I changed the loop to this. I hope this is not wrong ...

void loop()
{
      if (timestamp == 0)
    {
        timestamp = ntpClient.getTime(2000 /* wait 2000 ms */);

        if (timestamp > 0)
            smtp.setSystemTime(timestamp);
    }


    if(timestamp != 0)
    {
      if (millis() - sentMillis > 120000 || sentMillis == 0)
      {
          sentMillis = millis();
          sendEmail();
      }
    }
}

[mobizt]

I think you should remove the library and install the latest version via Library Manager (v2.7.9).

This is because, the strange thing was detected.

If you define this macro ENABLE_CUSTOM_CLIENT in ESP_Mail_FS.h and all calbacks assigned

    smtp.connectionRequestCallback(connectionRequestCallback);

    smtp.connectionUpgradeRequestCallback(connectionUpgradeRequestCallback);

    smtp.networkConnectionRequestCallback(networkConnection);

    smtp.networkStatusRequestCallback(networkStatusRequestCallback);

you should get debug message like this

Connected with IP: xx.xx.xx.xx

> I: The Connection Request Callback is now optional.

Connecting to SMTP server...
> C: ESP Mail Client v2.7.9
> C: Wait for NTP server time synching
> W: UDP client is required for NTP server time synching based on your network type 
e.g. WiFiUDP or EthernetUDP. Please call MailClient.setUDPClient(&udpClient, gmtOffset); to assign the UDP client.
Error, NTP server time synching timed out
> E: NTP server time synching timed out
> C: Connect to SMTP server
> C: Host > smtp.gmail.com
> C: Port > 465
> U: Connecting to server via custom Client... success.
> C: SMTP server connected

SMTP server connected, wait for greeting...

[Yoshibeatsu]

We got the same fault message with the version 2.7.9, thats why I updateted to 2.7.10 under your permission

I get all the Messages in the serial print with wifi. But not with ethernet.

But I can try with 2.7.9 again if you want

[mobizt]

This is the old example from version 2.7.2 which uses WiFiClientSecure as external client.
https://github.com/mobizt/ESP-Mail-Client/blob/v2.7.2/examples/SMTP/External_Client/SSLClient/WiFi_SSLClient/WiFi_SSLClient.ino
You should test it.

Why I want you to test with new library installation? It is because something is not properly set or config in the macro definition or the source code that used the macro.

With this example, the same code flow as Ethernet client and you should get the same message on serial as my previous reply.

This message should print on seral if it properly configures with the fresh installation library and define macro ENABLE_CUSTOM_CLIENT in ESP_Mail_FS.h

> U: Connecting to server via custom Client... success.

If you don't see that message, this means the macro ENABLE_CUSTOM_CLIENT was not yet defined in ESP_Mail_FS.h

[mobizt]

The error "not connect" is because the network is not ready or disconnected.

With external client, the callback function networkStatusRequestCallback was called and in that function it should call smtp.setNetworkStatus or imap.setNetworkStatus to let the library knows the network status of your network interface.

In the function networkStatusRequestCallback, you should print something on serial to check that this function was called.

void networkStatusRequestCallback() 
 { 
     Serial.println("Check for network status");
    
     // for Ethernet
     // smtp.setNetworkStatus(Ethernet.linkStatus() == LinkON); 

    // for WiFi
    smtp.setNetworkStatus(WiFi.status() == WL_CONNECTED);
 } 

I suspected that the networkStatusRequestCallback() was not called in your case because something was wrong with library installation or macros definition.

[Yoshibeatsu]

Thank you so much for your support!! I will start tomorrow morning (german time) again to check further all your suggestions!!

[Yoshibeatsu]

I don't think that its the installation because my colleagues had the same issue on his pc with the same microcontroller, he installed the library in the library manager. But I will make a fresh install of the library tomorrow and check everything further more

[mobizt]

I create a simple test sketch for ESP32 Gmail SMTP server connection testing using WiFiClient and EthernetClient (via W5500) with OPEnSLab's SSL client or built-in SSL client.

This test will show what' s wrong with your library installation, settings and network interfaces.

You can test it and post all Serial messages here.

Before test, please change WIFI_SSID, WIFI_PASSWORD, TEST_MODE and SSLCLIENT_LIB on the top of sketch.

TEST_MODE can be 0 for WiFi or 1 for Ethernet test
SSLCLIENT_LIB can be 0 for using OPEnSLab's SSLClient or 1 for using Mobizt's ESP_SSLClient or 2 for using built-in SSL client.

simple_test.zip

[Yoshibeatsu]

So... I deleted the library folder and downloaded 2.7.9 from the library manager...

Here are the results. It's quite interesting. I think we might found the issue

There are issues on TEST 1-1 and TEST 1-2

Looks like there might be an issue on ESP_SSLClient and the Built-in SSL client on Ethernet connection

TEST 0-0

10:24:56.621 -> > I: Test started using OPEnSLab's SSL Client and WiFiClient
10:24:56.621 -> 
10:24:56.621 -> > I: Connecting to AP.
10:24:56.935 -> > I: Connected with IP 192.168.178.37
10:24:56.935 -> 
10:24:56.935 -> > I: The Connection Request Callback is now optional.
10:24:56.935 -> 
10:24:56.935 -> > C: ESP Mail Client v2.7.9
10:24:56.935 -> > C: Wait for NTP server time synching
10:24:59.622 -> > C: Connect to SMTP server
10:24:59.622 -> > C: Host > smtp.gmail.com
10:24:59.622 -> > C: Port > 465
10:24:59.622 -> > U: Connecting to server via custom Client... (SSLClient)(SSL_WARN)(m_run_until): Discarded unread data to favor a write operation
10:25:00.547 -> success.
10:25:00.547 -> > C: SMTP server connected
10:25:30.563 -> > E: session timed out
10:25:30.563 -> 
10:25:30.563 -> > I: Test done!


TEST 0-1

11:08:11.471 -> > I: Test started using Mobizt's ESP_SSLClient and WiFiClient
11:08:11.518 -> 
11:08:11.518 -> > I: Connecting to AP.
11:08:11.800 -> > I: Connected with IP 192.168.178.37
11:08:11.800 -> 
11:08:11.800 -> > I: The Connection Request Callback is now optional.
11:08:11.800 -> 
11:08:11.800 -> > C: ESP Mail Client v2.7.9
11:08:11.846 -> > C: Wait for NTP server time synching
11:08:14.209 -> > C: Connect to SMTP server
11:08:14.209 -> > C: Host > smtp.gmail.com
11:08:14.209 -> > C: Port > 465
11:08:14.209 -> > U: Connecting to server via custom Client... success.
11:08:15.331 -> > C: SMTP server connected
11:08:15.378 -> < S: 220 smtp.gmail.com ESMTP g20-20020a7bc4d4000000b003a3442f1229sm10993957wmk.29 - gsmtp
11:08:15.378 -> > C: Send SMTP command, HELO
11:08:15.409 -> < S: 250-smtp.gmail.com at your service, [213.196.213.170]
11:08:15.409 -> < S: 250-SIZE 35882577
11:08:15.409 -> < S: 250-8BITMIME
11:08:15.409 -> < S: 250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
11:08:15.409 -> < S: 250-ENHANCEDSTATUSCODES
11:08:15.409 -> < S: 250-PIPELINING
11:08:15.409 -> < S: 250-CHUNKING
11:08:15.455 -> < S: 250 SMTPUTF8
11:08:15.455 -> 
11:08:15.455 -> > I: Test done!


TEST 0-2

11:12:26.240 -> > I: Test started using Built-in SSL Client and WiFiClient
11:12:26.240 -> 
11:12:26.240 -> > I: Connecting to AP.
11:12:26.553 -> > I: Connected with IP 192.168.178.37
11:12:26.601 -> 
11:12:26.601 -> > C: ESP Mail Client v2.7.9
11:12:26.601 -> > C: Wait for NTP server time synching
11:12:34.492 -> > C: Connect to SMTP server
11:12:34.492 -> > C: Host > smtp.gmail.com
11:12:34.492 -> > C: Port > 465
11:12:34.529 -> > C: seeding the random number generator
11:12:34.529 -> > C: setting up the SSL/TLS structure
11:12:34.529 -> ! W: Skipping SSL Verification. INSECURE!
11:12:34.529 -> > C: setting hostname for TLS session
11:12:34.529 -> > C: perform the SSL/TLS handshake
11:12:35.603 -> > C: verifying peer X.509 certificate
11:12:35.603 -> > C: SMTP server connected
11:12:35.650 -> < S: 220 smtp.gmail.com ESMTP z2-20020a5d4c82000000b002c4064f087esm5704090wrs.48 - gsmtp
11:12:35.650 -> > C: Send SMTP command, HELO
11:12:35.682 -> < S: 250-smtp.gmail.com at your service, [213.196.213.170]
11:12:35.682 -> < S: 250-SIZE 35882577
11:12:35.682 -> < S: 250-8BITMIME
11:12:35.682 -> < S: 250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
11:12:35.682 -> < S: 250-ENHANCEDSTATUSCODES
11:12:35.728 -> < S: 250-PIPELINING
11:12:35.728 -> < S: 250-CHUNKING
11:12:35.728 -> < S: 250 SMTPUTF8
11:12:35.728 -> 
11:12:35.728 -> > I: Test done!


TEST 1-0

11:03:52.081 -> > I: Test started using OPEnSLab's SSL Client and EthernetClient
11:03:52.128 -> 
11:03:52.128 -> > I: Resetting Ethernet...  
11:03:52.569 -> > I: Starting Ethernet connection...
11:03:59.460 -> > I: Connected with IP 192.168.178.43
11:03:59.460 -> > I: The Connection Request Callback is now optional.
11:03:59.506 -> 
11:03:59.506 -> > C: ESP Mail Client v2.7.9
11:03:59.506 -> > C: Wait for NTP server time synching
11:04:01.558 -> > C: Connect to SMTP server
11:04:01.558 -> > C: Host > smtp.gmail.com
11:04:01.558 -> > C: Port > 465
11:04:01.558 -> > U: Connecting to server via custom Client... (SSLClient)(SSL_WARN)(m_run_until): Discarded unread data to favor a write operation
11:04:02.533 -> success.
11:04:02.533 -> > C: SMTP server connected
11:04:32.551 -> > E: session timed out
11:04:32.551 -> 
11:04:32.551 -> > I: Test done!


TEST 1-1

11:18:55.826 -> > I: Test started using Mobizt's ESP_SSLClient and EthernetClient
11:18:55.826 -> 
11:18:55.826 -> > I: Resetting Ethernet...  
11:18:56.298 -> > I: Starting Ethernet connection...
11:19:03.119 -> > I: Connected with IP 192.168.178.43
11:19:03.120 -> > I: The Connection Request Callback is now optional.
11:19:03.120 -> 
11:19:03.120 -> > C: ESP Mail Client v2.7.9
11:19:03.120 -> > C: Wait for NTP server time synching
11:19:05.189 -> > C: Connect to SMTP server
11:19:05.189 -> > C: Host > smtp.gmail.com
11:19:05.189 -> > C: Port > 465
11:19:05.189 -> > U: Connecting to server via custom Client... ERROR - Generic error
11:19:06.242 -> failed.
11:19:06.242 -> > E: unable to connect to server
11:19:06.242 -> 
11:19:06.242 -> > I: Test done!



TEST 1-2

11:14:34.542 -> > I: Test started using Built-in SSL Client and EthernetClient
11:14:34.542 -> 
11:14:34.542 -> > I: Resetting Ethernet...  
11:14:34.951 -> > I: Starting Ethernet connection...
11:14:41.815 -> > I: Connected with IP 192.168.178.43
11:14:41.815 -> > E: connection closed
11:14:41.815 -> 
11:14:41.815 -> > I: Test done!

[mobizt]

The callback functions required as it related to the external Client and network interfaces that defined by user and are currently using.

Library will not know the type of network interface and client. The callback function assign to networkStatusRequestCallback called when library wants to check network status.

The callback function assigned to networkConnectionRequestCallback is for network resume in case network disconnection occurred.

The connectionRequestCallback assigned function is for server connection which is not necessary (as it can do internally) in the new version and keep it for compatibility.

I can summarize that your ethernet (device or network) is not stable due to hardware issue as it disconnected before we open the connection to server.

[Yoshibeatsu]

But only on test 1-1 and test 1-2. test 1-0 is working. Why is the first one working when it’s hardware issue ?

[mobizt]

In TEST 1-2, the networkStatusRequestCallback was called and Ethernet.linkStatus() returns down then the status was set back to the library via this line in the callback function smtp.setNetworkStatus(WiFi.status() == WL_CONNECTED || Ethernet.linkStatus() == LinkON);

Nothing can do when library sees the network status was disconnected then it exits immediately with error E: connection closed

[mobizt]

For the hardware issue I don't know.

I never seen such hardware issues on my test.

As I said before about this W5500 module which some users can make it works, some user can't.

[mobizt]

The first priority code for all my network related libraries is to check network status because to prevent the wdt reset or unexpected error in TCP client in ESP MCUs.

[Yoshibeatsu]

Solution might be to use the SSL Client from the test 1-0

[mobizt]

Test with Ethernet failed for external SSL client as incomplete SSL data received from W5500 buffer and SSL (BearSSL and mbedTLS) engine cannot decrypt it.

The last test is because network disconnected before open the connection and sending the request.

[Yoshibeatsu]

But test 1-0 with OPEnSLab's SSL Client was ok right?

[Yoshibeatsu]

I will try later Test 1-1 and 1-2 with Ethernetlarge.h. Maybe it will resolve our buffer issue

[mobizt]

This is example W5500 module type I used from unknown manufacturer with small footprint and lower noise as its footprint.
This is for reference only. I buy from other local stores.
https://www.makerfabs.com/mini-ethernet-board-w5500.html

[mobizt]

Good news.

For the TEST 0-0 and TEST 1-0, which OPEnSLab' SSLClient shows error
(SSLClient)(SSL_WARN)(m_run_until): Discarded unread data to favor a write operation,
you can use my fork library https://github.com/mobizt/SSLClient instead which this error is already fixed.

Actually, in TEST 1-2, it still needs ENABLE_CUSTOM_CLIENT macro in ESP_Mail_FS.h because external basic client was used with the internal SSL engine.

And these lines in the test sketch should be deleted.

Serial.println("> E: Please remove ENABLE_CUSTOM_CLIENT Macro from src/ESP_Mail_FS.h");
goto exit;

And TEST 1-2 should be ok now with error ERROR - Generic error as in TEST 1-1 as they use the same SSL engine library.

[Yoshibeatsu]

I think I used your fork library during the tests. Ok I will double check this on test 1-2

[mobizt]

I just update.

[Yoshibeatsu]

Hello my friend,

I modified your test code to send an email for testing. Problem is, I can not make it work on wifi with the first SSL Client. Its working on ethernet.

This is what I get on Wifi

11:48:50.107 -> > I: Connecting to AP.
11:48:50.434 -> > I: Connected with IP 192.168.178.37
11:48:50.434 -> 
11:48:50.434 -> > I: The Connection Request Callback is now optional.
11:48:50.434 -> 
11:48:50.434 -> Connecting to SMTP server...
11:48:50.434 -> > C: ESP Mail Client v2.7.9
11:48:50.434 -> > C: Wait for NTP server time synching
11:48:52.448 -> > C: Connect to SMTP server
11:48:52.448 -> > C: Host > smtp.gmail.com
11:48:52.448 -> > C: Port > 465
11:48:52.448 -> > U: Connecting to server via custom Client... (SSLClient)(SSL_WARN)(m_run_until): Discarded unread data to favor a write operation
11:48:53.378 -> success.
11:48:53.378 -> > C: SMTP server connected
11:48:53.378 -> 
11:48:53.378 -> SMTP server connected, wait for greeting...
11:49:23.392 -> Error, session timed out
11:49:23.392 -> > E: session timed out

Sometimes I get this on Ethernet as well very strange. Internet seems to work well, I did a speedtest on https://www.speedtest.net/de

Here is the code

// Change the folowing credentials, TEST_MODE and SSLCLIENT_LIB
// ############################################################
#define WIFI_SSID "Mohnzimmer"
#define WIFI_PASSWORD "SECRET"


// Test mode
// 0 - WiFi test
// 1 - Ethernet test
#define TEST_MODE 0

// SSLClient library used for test
// 0 - OPEnSLab's SSLClient // https://github.com/OPEnSLab-OSU/SSLClient or https://github.com/mobizt/SSLClient
// 1 - Mobizt's MB_SSLClient // https://github.com/mobizt/ESP_SSLClient
// 2 - Built-in SSL client
#define SSLCLIENT_LIB 0
// ############################################################

#include <Arduino.h>
#include <ESP_Mail_Client.h>
//#include <Ethernet.h>
#include "EthernetLarge.h"
#include <EthernetUdp.h>
#if SSLCLIENT_LIB == 0
#include <SSLClient.h>
#include "trust_anchors.h"
const int analog_pin = 34;
#elif SSLCLIENT_LIB == 1
#include <ESP_SSLClient.h>
#endif

#define SMTP_HOST "smtp.gmail.com"
#define SMTP_PORT 465

#if TEST_MODE == 0
WiFiUDP udpClient;
#elif TEST_MODE == 1

#define WIZNET_RESET_PIN 26
#define WIZNET_CS_PIN 5

uint8_t Eth_MAC[] = {0x02, 0xF0, 0x0D, 0xBE, 0xEF, 0x01};
IPAddress Eth_IP(192, 168, 178, 104);

EthernetUDP udpClient;
#endif

#if TEST_MODE == 0
WiFiClient basic_client;
#elif TEST_MODE == 1
EthernetClient basic_client;
#endif

#if SSLCLIENT_LIB == 0
SSLClient ssl_client(basic_client, TAs, (size_t)TAs_NUM, analog_pin);
#elif SSLCLIENT_LIB == 1
ESP_SSLClient ssl_client;
#endif

#if SSLCLIENT_LIB < 2
SMTPSession smtp(&ssl_client, esp_mail_external_client_type_ssl);
#define WORK_CLIENT ssl_client
#else
SMTPSession smtp(&basic_client, esp_mail_external_client_type_basic);
#define WORK_CLIENT basic_client
#endif

void smtpCallback(SMTP_Status status);



/* The sign in credentials */
#define AUTHOR_EMAIL "[email protected]"
#define AUTHOR_PASSWORD "SECRET"

/* Recipient's email*/
#define RECIPIENT_EMAIL "[email protected]"

/* The SMTP Session object used for Email sending */

SMTP_Message message;
ESP_Mail_Session session;

/* Callback function to get the Email sending status */
bool last_status = 0 ;



void networkConnection()
{

#if TEST_MODE == 0

  Serial.print("> I: Connecting to AP");

  WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
  while (WiFi.status() != WL_CONNECTED)
  {
    Serial.print(".");
    delay(200);
  }

  Serial.println("");
  Serial.print("> I: Connected with IP ");
  Serial.println(WiFi.localIP());
  Serial.println();

#elif TEST_MODE == 1

  Ethernet.init(WIZNET_CS_PIN);
  Serial.println("> I: Resetting Ethernet...  ");
  pinMode(WIZNET_RESET_PIN, OUTPUT);
  digitalWrite(WIZNET_RESET_PIN, HIGH);
  delay(200);
  digitalWrite(WIZNET_RESET_PIN, LOW);
  delay(50);
  digitalWrite(WIZNET_RESET_PIN, HIGH);
  delay(200);

  Serial.println("> I: Starting Ethernet connection...");
  Ethernet.begin(Eth_MAC/*, Eth_IP*/);

  unsigned long to = millis();

  while (Ethernet.linkStatus() == LinkOFF || millis() - to < 2000)
  {
    delay(100);
  }

  if (Ethernet.linkStatus() == LinkON)
  {
    Serial.print("> I: Connected with IP ");
    Serial.println(Ethernet.localIP());
  }

#endif
}

void networkStatusRequestCallback()
{
  smtp.setNetworkStatus(WiFi.status() == WL_CONNECTED || Ethernet.linkStatus() == LinkON);
}

void connectionRequestCallback(const char *host, int port)
{

  Serial.print("> U: Connecting to server via custom Client... ");
  if (!WORK_CLIENT.connect(host, port))
  {
    Serial.println("failed.");
    return;
  }
  Serial.println("success.");
}

void setup()
{


  Serial.begin(115200);

  Serial.println();

  delay(5000);

#if SSLCLIENT_LIB == 0
  Serial.print("> I: Test started using OPEnSLab's SSL Client and ");
#elif SSLCLIENT_LIB == 1
  Serial.print("> I: Test started using Mobizt's ESP_SSLClient and ");
#elif SSLCLIENT_LIB == 2
  Serial.print("> I: Test started using Built-in SSL Client and ");
#endif

#if TEST_MODE == 0
  Serial.println("WiFiClient\n");
#elif TEST_MODE == 1
  Serial.println("EthernetClient\n");
#endif

#if (SSLCLIENT_LIB < 2) && !defined(ENABLE_CUSTOM_CLIENT)
  Serial.println("> E: ENABLE_CUSTOM_CLIENT Macro was not detected.\n> I: Please add #define ENABLE_CUSTOM_CLIENT in src/ESP_Mail_FS.h");
  //goto exit;
#elif SSLCLIENT_LIB == 2 && defined(ENABLE_CUSTOM_CLIENT)
  Serial.println("> E: Please remove ENABLE_CUSTOM_CLIENT Macro from src/ESP_Mail_FS.h");
  //goto exit;
#endif

  networkConnection();

  if (WiFi.status() != WL_CONNECTED && Ethernet.linkStatus() != LinkON)
  {
    Serial.println("> E: network connection failed.");
    //goto exit;
  }

#if SSLCLIENT_LIB == 1
  WORK_CLIENT.setClient(&basic_client);
  WORK_CLIENT.setInsecure();
#endif

  MailClient.networkReconnect(true);

  MailClient.setUDPClient(&udpClient, 0);

 smtp.debug(1);
  smtp.callback(smtpCallback);

  session.server.host_name = SMTP_HOST;
  session.server.port = SMTP_PORT;
  session.time.ntp_server = F("pool.ntp.org,time.nist.gov");
  session.login.email = AUTHOR_EMAIL;
  session.login.password = AUTHOR_PASSWORD;
  session.login.user_domain = "";

  SMTP_Message message;

  message.sender.name = "Lackiermaschine";
  message.sender.email = AUTHOR_EMAIL;
  message.subject = "TEST";
  message.addRecipient("Mail Verteiler", "[email protected]");

   String textMsg = "Message";

  message.text.content = textMsg.c_str();
  message.text.charSet = "us-ascii";
  message.text.transfer_encoding = Content_Transfer_Encoding::enc_7bit;
  message.priority = esp_mail_smtp_priority::esp_mail_smtp_priority_low;
  message.response.notify = esp_mail_smtp_notify_success | esp_mail_smtp_notify_failure | esp_mail_smtp_notify_delay;

 

  smtp.connectionRequestCallback(connectionRequestCallback);
  smtp.networkConnectionRequestCallback(networkConnection);
  smtp.networkStatusRequestCallback(networkStatusRequestCallback);

  if (!smtp.connect(&session))
  {
    return;
  }

  /* Start sending Email and close the session */

  if (!MailClient.sendMail(&smtp, &message))
  {
   Serial.println("Error sending Email, " + smtp.errorReason());
  }
  


//exit:

  Serial.println("\n> I: Test done!");
  
}

void loop()
{
}

void smtpCallback(SMTP_Status status)
{
  Serial.println(status.info());
}

[mobizt]

What's the error. It works when I test.

[Yoshibeatsu]

Please see edit I send it to early before I was done with writing hahaha

[mobizt]

That error message
(SSLClient)(SSL_WARN)(m_run_until): Discarded unread data to favor a write operation
will not exist anymore because I remove it already.

This means you are using original OPEnSLab's SSLClient instead of my fork version.

You need to delete SSLClient library and reinstall.

[mobizt]

https://github.com/mobizt/SSLClient/blob/3848587093058cbe7bec5302cc934e70eeb0a938/src/SSLClient.cpp#L615-L618

[mobizt]

You can check beween my fork version SSLClient vs original SSLClient with macro SSLCLIENT_CONNECTION_UPGRADABLE.

#if defined(SSLCLIENT_CONNECTION_UPGRADABLE)
// SSLClient fork version
#else 
// SSLClient original version
#endif

[Yoshibeatsu]

Do you have any idea what this fault message on IMAP call could be? SMTP is working perfectly
Its on Wifi, your fork SSL and 2.7.9

20:05:47.611 -> Connecting to IMAP server...
20:05:47.611 -> > C: ESP Mail Client v2.7.9
20:05:47.611 -> > C: Connect to IMAP server
20:05:47.611 -> > C: Host > imap.gmail.com
20:05:47.611 -> > C: Port > 993
20:05:47.611 -> > U: Connecting to server via custom Client... success.
20:05:48.538 -> IMAP server connected
20:05:48.538 -> > C: Server connected
20:05:48.538 -> 
20:05:48.538 -> Check the capability...
20:05:48.538 -> > C: Check the capability
20:05:48.538 -> Error, connection closed
20:05:48.538 -> > E: connection closed

My code is quite long, so if you need I can provide

[mobizt]

I test it already and I don't have any issue.

You should update library and test.

[mobizt]

If you provide connection request callback you should see what's going on.

In the new library examples, see the IMAP and SMTP external client examples and read the comments in the examples about what you should know before use.

[Yoshibeatsu]

I updated now to 2.7.10 and get the same result. Its stuck on Check capability and gets an Error, connection closed.

23:51:46.383 -> Connecting to IMAP server...
23:51:46.383 -> > C: ESP Mail Client v2.7.10
23:51:46.383 -> > C: Connect to IMAP server
23:51:46.383 -> > C: Host > imap.gmail.com
23:51:46.383 -> > C: Port > 993
23:51:46.383 -> > U: Connecting to server via custom Client... success.
23:51:47.309 -> IMAP server connected
23:51:47.309 -> > C: Server connected
23:51:47.309 -> 
23:51:47.309 -> Check the capability...
23:51:47.309 -> > C: Check the capability
23:51:47.309 -> Error, connection closed
23:51:47.309 -> > E: connection closed

I already implemented three callbacks as you teach me before, but get the same results. Its connecting but doesn't pass the capability.

imap.connectionRequestCallback(connectionRequestCallback);
    imap.networkConnectionRequestCallback(networkConnection);
    imap.networkStatusRequestCallback(networkStatusRequestCallback);
  

    if (!imap.connect(&session /* session credentials */, &config /* operating options and its result */))
        return 0;

    printAllMailboxesInfo(imap);
    if (!imap.selectFolder(F("INBOX")))
        return 0;

SMTP works fine

[mobizt]

Did it fail with Wi-Fi or Ethernet or both?

[Yoshibeatsu]

Both

[mobizt]

Strange because it works for me. At least it should work with WiFi + original SSL Client and there is something wrongs in your code.

The error Error, connection closed is only because the network status set in networkStatusRequestCallback is false;

Check this, is it similar to your code?

void networkStatusRequestCallback()
{
    // Set the network status by calling imap method.
    imap.setNetworkStatus(WiFi.status() == WL_CONNECTED || Ethernet.linkStatus() == LinkON);
}

[Yoshibeatsu]

Oh my god... Im stupid. Imap callback was outcommented...

void networkStatusRequestCallback()
{
  smtp.setNetworkStatus(WiFi.status() == WL_CONNECTED || Ethernet.linkStatus() == LinkON);
  //imap.setNetworkStatus(WiFi.status() == WL_CONNECTED || Ethernet.linkStatus() == LinkON);
}

[mobizt]

There is no global method which make it easier to use like this.
MailClient.setNetworkStatus

This is because smtp.setNetworkStatus and imap.setNetworkStatus set the network status flag directly to the internal TCP client that uses in each smtp and imap objects.

[Yoshibeatsu]

Everything is working fine for now. Thank you so much for your support. Me and my team, we will donate you something at the end of our project !!

[nada0302]

I refered this discussion and leaned a lot. Thank you !
I successed Test 1-0 and success to send email through W5500.
implemented with

For the TEST 0-0 and TEST 1-0, which OPEnSLab' SSLClient shows error
(SSLClient)(SSL_WARN)(m_run_until): Discarded unread data to favor a write operation,
you can use my fork library https://github.com/mobizt/SSLClient instead which this error is already fixed.

in Test 1-1, I faced error msg as below,
is there anything wrong ?
appreciate your suggestion.
Thank you!

I: Test started using Mobizt's ESP_SSLClient and EthernetClient

I: Resetting Ethernet...
I: Starting Ethernet connection...
I: Connected with IP 192.168.10.9
I: The Connection Request Callback is now optional.

C: ESP Mail Client v3.1.1
C: wait for NTP server time synching
C: connecting to SMTP server
C: Host > smtp.gmail.com
C: Port > 465
U: Connecting to server via custom Client... ERROR - Generic error
[ 12966][E][MB_ESP32_SSL_Client.cpp:63] _esp32_ssl_handle_error(): [connect_ssl():354]: (-1) ERROR - Generic error
[ 12967][E][MB_ESP32_TCPClient.cpp:273] connectSSL(): MB_ESP32_TCPClient Error: SSL connection, -1
failed.
! E: unable to connect to server

I: Test done!

ini file in platform.io is here

[env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
lib_deps =
;openslab-osu/SSLClient@^1.6.11
https://github.com/mobizt/SSLClient
mobizt/ESP Mail Client@^3.1.1
arduino-libraries/Ethernet@^2.0.1
;https://github.com/khoih-prog/EthernetLarge.git
https://github.com/mobizt/ESP_SSLClient

[mobizt]

That is the error due to SSL engine library ued in ESP32 core sdk i.e., mbedTLS which does not work when use with W5500 via client class.

This may relate to the buffer size of mbedTLS which is always be large as 16k and may not work with smaller buffer that provided by the W5500 but this assumption may not true if the socket was controlled directly by mcu instead of through the client class as in esp-idf framework that supports W5500 and works normally.

Only SSL Client library e.g., only OPEnSLab' SSLClient that uses BearSSL SSL engine library can work with W5500 for now.

[nada0302]

Thanks for your reply in quick !
well noted & thanks again !