TinyGSM on ESP32 only working with #define TINY_GSM_RX_BUFFER 2

[fbeek]

TinyGSM version: 0.7.4
Hardware: ESP32 Dev Kit V1 with Framework 1.0.2 (1.0.1 also tested), Sim800L EVB
HardwareSerial 1 for AT commands.

Scenario, steps to reproduce

Iam trying to get the diagnostic and mqtt sketch to work. Lab Power Supply with 5V 4A is working nicely, so no VCC drops or resets.

Expected result

Passes http transfer test with 121 Bytes. And connect to mqtt

Actual result

I can only receive the complete data at the http test when I set #define TINY_GSM_RX_BUFFER 2. With larger buffer values I can only get a part of the 121 Bytes of the TINYGSM test logo file. I thought the ESP32 should have enough power and ram?

With this settings the MQTT Sketch is also not working.

In the command log there where no errors until the end. So i think it must be something in the ESP32 context.

Does Anyone have this lib working on an esp32

[SRGDamia1]

You're not getting more than two bytes at a time? That seems pretty unreasonable.

What errors did you get at the end? Please post the AT/errors.

[hardmax]

Hi,

I have the same problem with the similar hardware (ESP32 with SIM800L). When is conecting to MQTT and I view in server side, conection petition, but this is not finished, and conecction is dropped.

Change library to 0.3.5 and apply this workarround #269 to compile, and works all againg....

The problem is with 0.6.2 and 0.7.4. Return code MQTT is 1 (Bad protocol)

[SRGDamia1]

AT log? Debug prints?

[fbeek]

Hello together,

here is my full log with a Buffer Size of 256:

Initializing modem...[4031] ### TinyGSM Version: 0.7.4
AT

OK
AT&FZ

OK
ATE0

OK
[4066] ### Modem: SIMCom SIM800
AT+CPIN?
ATE0

OK

+CPIN: READY

OK
[OK]
ATI

SIM800 R14.18

OK
Modem: SIM800 R14.18
Waiting for network...AT+CREG?

+CREG: 0,5

OK
[OK]
Connecting to pinternet.interkom.deAT+CIPSHUT

SHUT OK
AT+CGATT=0

OK
AT+SAPBR=3,1,"Contype","GPRS"

OK
AT+SAPBR=3,1,"APN","pinternet.interkom.de"

OK
AT+CGDCONT=1,"IP","pinternet.interkom.de"

OK
AT+CGACT=1,1

OK
AT+SAPBR=1,1

OK
AT+SAPBR=2,1

+SAPBR: 1,1,"10.177.109.126"

OK
AT+CGATT=1

OK
AT+CIPMUX=1

OK
AT+CIPQSEND=1

OK
AT+CIPRXGET=1

OK
AT+CSTT="pinternet.interkom.de","",""

OK
AT+CIICR

OK
AT+CIFSR;E0

10.177.109.126

OK
AT+CDNSCFG="8.8.8.8","8.8.4.4"

OK
[OK]
AT+CIFSR;E0

10.177.109.126

OK
Local IP: 10.177.109.126
Connecting to vsh.pp.uaAT+CIPCLOSE=1,1

ERROR
AT+CIPSSL=0

OK
AT+CIPSTART=1,"TCP","vsh.pp.ua",80

OK

1, CONNECT OK
[OK]
AT+CIPSEND=1,32

GET /TinyGSM/logo.txt HTTP/1.0

DATA ACCEPT:1,32
AT+CIPSEND=1,17

Host: vsh.pp.ua

DATA ACCEPT:1,17
AT+CIPSEND=1,21

Connection: close

DATA ACCEPT:1,21
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,0

OK
AT+CIPSTATUS=1

+CIPSTATUS: 1,0,"TCP","212.237.59.104","80","CONNECTED"

OK
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,0

OK
AT+CIPSTATUS=1

+CIPSTATUS: 1,0,"TCP","212.237.59.104","80","CONNECTED"

OK
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,633

OK

AT+CIPRXGET=2,1,255

+CIPRXGET: 2,1,255,378
HTTP/1.1 200 OK
Server: nginx/1.10.3 (Ubuntu)
Date: Thu, 23 May 2019 07:48:43 GMT
Content-Type: text/plain; charset=UTF-8
Content-Length: 121
Connection: close
X-DNS-Prefetch-Control: off
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-
OK
[11256] ### READ: 255 from 1
AT+CIPRXGET=2,1,255

+CIPRXGET: 2,1,255,123
[11334] 132 fewer bytes confirmed than requested!
age=15552000; includeSubDomains
X-Download-Options: noopen
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Accept-Ranges: bytes
Cache-Control: public, max-age=0
Last-Modified: Wed, 27 Sep 2017 09:03:12 GMT
ETag: W/"79-15ec2936080"

OK
[11427] ### READ: 123 from 1
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,123

OK
AT+CIPRXGET=2,1,123

+CIPRXGET: 2,1,123,0
[11532] 123 fewer bytes confirmed than requested!

---

|  | |\ | \_/ |  ___ |_____ |  |  |
|  | | \|  |  |_____| _____||  |  |

OK
[11555] ### READ: 0 from 1
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,0

OK
AT+CIPSTATUS=1

+CIPSTATUS: 1,0,"TCP","212.237.59.104","80","REMOTE CLOSING"

OK
[12714] ### Unhandled: +CIPSTATUS: 1,0,"TCP","212.237.59.104","80","REMOTE CLOSING"

OK
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,0

OK
AT+CIPSTATUS=1

+CIPSTATUS: 1,0,"TCP","212.237.59.104","80","REMOTE CLOSING"

OK
[14749] ### Unhandled: +CIPSTATUS: 1,0,"TCP","212.237.59.104","80","REMOTE CLOSING"

OK
AT+CIPCLOSE=1,1

1, CLOSE OK
Server disconnected
AT+CIPSHUT

SHUT OK
AT+CGATT=0

+SAPBR 1: DEACT

OK
GPRS disconnected

---

Received: 0 bytes
Test: FAILED

---

Here is the last part of the log for a buffer size of 2:

[28848] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,79

OK
[28878] ### READ: 1 from 1
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,79

OK
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,78

OK
[28936] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,77

OK
[28967] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,76
|
OK
[28997] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,75

OK
[29028] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,74

OK
[29058] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,73
|
OK
[29088] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,72

OK
[29119] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,71
|
OK
[29149] ### READ: 1 from 1
|AT+CIPRXGET=4,1

+CIPRXGET: 4,1,71

OK
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,70

OK
[29207] ### READ: 1 from 1
\AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,69

OK
[29238] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,68
|
OK
[29268] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,67

OK
[29298] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,66

OK
[29329] ### READ: 1 from 1
\AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,65
_
OK
[29359] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,64
/
OK
[29390] ### READ: 1 from 1
/AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,63

OK
[29420] ### READ: 1 from 1
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,63

OK
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,62
|
OK
[29478] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,61

OK
[29508] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,60

OK
[29539] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,59
_
OK
[29569] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,58
_
OK
[29600] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,57
_
OK
[29630] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,56

OK
[29660] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,55
|
OK
[29691] ### READ: 1 from 1
|AT+CIPRXGET=4,1

+CIPRXGET: 4,1,55

OK
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,54
_
OK
[29749] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,53
_
OK
[29780] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,52
_
OK
[29810] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,51
_
OK
[29840] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,50
_
OK
[29871] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,49

OK
[29901] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,48
|
OK
[29932] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,47

OK
[29962] ### READ: 1 from 1
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,47

OK
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,46

OK
[30020] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,45
|
OK
[30050] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,44

OK
[30081] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,43

OK
[30111] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,42
|
OK
[30142] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,41

OK
[30172] ### READ: 1 from 1

AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,40

OK
[30203] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,39

OK
[30233] ### READ: 1 from 1
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,39

OK
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,38

OK
[30291] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,37

OK
[30322] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,36
|
OK
[30352] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,35

OK
[30382] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,34

OK
[30413] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,33
|
OK
[30443] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,32

OK
[30474] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,31
|
OK
[30504] ### READ: 1 from 1
|AT+CIPRXGET=4,1

+CIPRXGET: 4,1,31

OK
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,30

OK
[30562] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,29

OK
[30592] ### READ: 1 from 1
\AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,28
|
OK
[30623] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,27

OK
[30654] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,26

OK
[30684] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,25
|
OK
[30714] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,24

OK
[30745] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,23

OK
[30775] ### READ: 1 from 1
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,23

OK
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,22
|
OK
[30833] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,21
_
OK
[30864] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,20
_
OK
[30894] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,19
_
OK
[30925] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,18
_
OK
[30955] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,17
_
OK
[30986] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,16
|
OK
[31016] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,15

OK
[31046] ### READ: 1 from 1
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,15

OK
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,14
_
OK
[31106] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,13
_
OK
[31137] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,12
_
OK
[31168] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,11
_
OK
[31199] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,10
_
OK
[31230] ### READ: 1 from 1
_AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,9
|
OK
[31260] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,8
|
OK
[31290] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,7

OK
[31320] ### READ: 1 from 1
AT+CIPRXGET=4,1

+CIPRXGET: 4,1,7

OK
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,6

OK
[31377] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,5
|
OK
[31407] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,4

OK
[31437] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,3

OK
[31467] ### READ: 1 from 1
AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,2
|
OK
[31497] ### READ: 1 from 1
|AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,1

OK
[31527] ### READ: 1 from 1

AT+CIPRXGET=2,1,1

+CIPRXGET: 2,1,1,0
[31557] 1 fewer bytes confirmed than requested!

OK
[31558] ### READ: 0 from 1
⸮AT+CIPRXGET=4,1

+CIPRXGET: 4,1,0

OK
AT+CIPSTATUS=1

+CIPSTATUS: 1,0,"TCP","212.237.59.104","80","REMOTE CLOSING"

OK
[32606] ### Unhandled: +CIPSTATUS: 1,0,"TCP","212.237.59.104","80","REMOTE CLOSING"

OK
AT+CIPCLOSE=1,1

1, CLOSE OK
Server disconnected
AT+CIPSHUT

SHUT OK
AT+CGATT=0

+SAPBR 1: DEACT

OK
GPRS disconnected

---

Received: 121 bytes
Test: PASSED

---

[fbeek]

I tried the solution from @hardmax and it works, but this cannot be the solution.

I think the newer version has some nice fixes and additions.

[SRGDamia1]

I think this issue and #284 are different faces of the same problem. I don't think it's something related to the ESP32.

I've modified the read to de-increment the sock_available as we read out, which should reduce/eliminate the number of fewer bytes confirmed than requested! problems. You can test it from https://github.com/EnviroDIY/TinyGSM.

I'd changed the read function to not attempt to read more characters than the response confirmed were available, but it seems the SIM800 occasionally lies about how much is available. I still need to think about how to deal with it. That didn't happen when I did some quick tests with the SIM800 before pushing the update. I'll try and test again somewhere with 2G service.

[tkralj]

Hello, i tried different approaches and all of them resulted in different output for the same code. I changed only TINY_GSM_RX_BUFFER from 1, 2, 16 and 1024 and each value produced different output from screen full of question marks to nothing at all . TinyGSM 0.7.4 ESP32 core 1.0.1. i used WebClient example for testing purposes.

this is the output if i set buffer to 1024:
Initializing modem...
Modem: SIM800 R14.18
Waiting for network... OK
Network connected
Connecting to wm
OK
Connecting to iot-test.softlab.hr
OK
Performing HTTP GET request...
⸮
Server disconnected
GPRS disconnected

this is the output if i set buffer to 2:
Initializing modem...
Modem: SIM800 R14.18
Waiting for network... OK
Network connected
Connecting to wm
OK
Connecting to iot-test.softlab.hr
OK
Performing HTTP GET request...
HTTP/1.1 200 OK
Content-Type: text/plain
Last-Modified: Sun, 19 May 2019 07:12:31 GMT
Accept-Ranges: bytes
ETag: "203acf3912ed51:0"
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Fri, 24 May 2019 11:20:33 GMT
Connection: close
Content-Length: 5

hell⸮
Server disconnected
GPRS disconnected

(almost perfect, but misses "o" as in hello)

This is the output if i set buffer to 16:

Initializing modem...
Modem: SIM800 R14.18
Waiting for network... OK
Network connected
Connecting to wm
OK
Connecting to iot-test.softlab.hr
OK
Performing HTTP GET request...
HTTP/1.1 200 OK
Content-Type: text/plain
Last-Modified: Sun, 19 May 2019 07:12:31 GMT
Accept-Ranges: bytes
ETag: "203acf3912ed51:0"
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Fri, 24 May 2019 11:22:32 GMT
Connection: close
Content-Len⸮
Server disconnected
GPRS disconnected

[SRGDamia1]

I'm sorry, guys. I've (hopefully) caught the issues now. I think it was all in the SIM800 and probably not on the ESP32 side. Can you try again with 0.7.7 0.7.9 and let me know how you fare?

[fbeek]

Good Morning,

on monday i am back at home and will test the new version.

Thanks for your help.

Regards

[hardmax]

Its Working........
Nice job...Thanks

Reagards

[SRGDamia1]

@fbeek - can I close this?

[fbeek]

Hello,

sorry for my late response.
You can close this issue, its working now.

Many thanks to you for your work.

Regards
fbeek

[einmalpizzafunghibitte]

Hi @SRGDamia1 ,
I am trying the very same thing with a Wemos D1 Mini (esp8266) + SIM7600E-H right now: Same issue. If i set TINY_GSM_RX_BUFFER to 4096 or anything (I have tried many, many values 1024, 2048,8192,16384). It will just freeze in the middle of processing the "CIPRXGET"

See:
problem.txt

AT+CIPRXGET=2,0,568
AT+CIPRXGET=2,0,568
+CIPRXGET: 2,0,568,0
HTTP/1.1 200 OK
Servea��

This is where is get's stuck.
If is set TINY_GSM_RX_BUFFER to 2 it works fine. In fact it works for quite a lot of "small buffers", e.g. 8, 16. Weirdly it got a lot slower with TINY_GSM_RX_BUFFER=32 than TINY_GSM_RX_BUFFER=2.
Maybe there is some fix missing to be ported from SIM800 to SIM7600?

The example that I am trying (HttpClient.ino) explicitly states that I might have to increase the buffer for things to work with esp8266. Maybe you could change the recommendation. I know to little of how this works to know whether that is correct or better to change something in code and keep the recommendation.

Thanks in advance!