ESP-IDF 4.4.5: I2C crash during flash operations (IDFGH-11007)

[kriegste]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

4.4.5

Operating System used.

Windows

How did you build your project?

VS Code IDE

If you are using Windows, please specify command line type.

None

Development Kit.

TTGO

Power Supply used.

USB

What is the expected behavior?

#10079 and 9a6a287
were implemented in ESP-IDF 4.4.5 and I thought I2C was stable now.

What is the actual behavior?

I still get a an occasional

Guru Meditation Error: Core  0 panic'ed (Cache disabled but cached memory region accessed). 

Core  0 register dump:
PC      : 0x40087b1c  PS      : 0x00060034  A0      : 0x80087bf9  A1      : 0x3ffb09f0  
A2      : 0x00000000  A3      : 0x00000001  A4      : 0x3ffbf980  A5      : 0x00000002  
A6      : 0x3f419e8c  A7      : 0x6001301c  A8      : 0x3ffbb624  A9      : 0x3ff53000  
A10     : 0x00000900  A11     : 0x000000ad  A12     : 0x00000002  A13     : 0x00000002  
A14     : 0x00000000  A15     : 0x3ffb12a0  SAR     : 0x00000004  EXCCAUSE: 0x00000007  
EXCVADDR: 0x00000000  LBEG    : 0x00000000  LEND    : 0x00000000  LCOUNT  : 0x00000000  

Backtrace: 0x40087b19:0x3ffb09f0 0x40087bf6:0x3ffb0a30 0x400834ed:0x3ffb0a70 0x40089da8:0x3ffaf4e0 0x400876d1:0x3ffaf500

0x40087b19: i2c_ll_write_txfifo at C:/**/esp-idf/components/hal/esp32/include/hal/i2c_ll.h:565
 (inlined by) i2c_master_cmd_begin_static at C:/**/esp-idf/components/driver/i2c.c:1364
0x40087bf6: i2c_isr_handler_default at C:/**/esp-idf/components/driver/i2c.c:517
0x400834ed: _xt_lowint1 at C:/**/esp-idf/components/freertos/port/xtensa/xtensa_vectors.S:1114
0x40089da8: spi_flash_op_block_func at C:/**/esp-idf/components/spi_flash/cache_utils.c:124 (discriminator 1)
0x400876d1: ipc_task at C:/**/esp-idf/components/esp_ipc/src/esp_ipc.c:77

Steps to reproduce.

The crashes are rare and seem to happen when accessing the flash while another task is busy with I2C. The I2C driver is installed with flag ESP_INTR_FLAG_IRAM. I wasn't able to create a minimum code example, though...

Maybe someone has an idea anyway.

Debug Logs.

No response

More Information.

No response

[kriegste]

https://github.com/espressif/esp-idf/blob/release/v4.4/components/hal/esp32/include/hal/i2c_ll.h#L565
This is the line where the crash supposedly happens. Strange.

[mythbuster5]

i2c.txt
could you please try with this patch?

[kriegste]

error: 'i2c_dev_t' {aka 'volatile struct i2c_dev_s'} has no member named 'data'

[mythbuster5]

Oh.. data->fifo_data

[kriegste]

HAL_FORCE_MODIFY_U32_REG_FIELD(hw->fifo_data, data, ptr[i]);

compiles. I'm testing.....

I2C is still working......

No crash so far..... I'll report back in a few hours.

[kriegste]

Unfortunately...

Guru Meditation Error: Core  0 panic'ed (Cache disabled but cached memory region accessed). 

Core  0 register dump:
PC      : 0x40087b27  PS      : 0x00060034  A0      : 0x80087c0d  A1      : 0x3ffb09f0  
A2      : 0x00000000  A3      : 0x3f419ed8  A4      : 0x3ffbf980  A5      : 0x00000002  
A6      : 0xffffff00  A7      : 0x000000ad  A8      : 0x00000000  A9      : 0x3ff53000  
A10     : 0x3ffbb684  A11     : 0x00000900  A12     : 0x00000001  A13     : 0x00000002  
A14     : 0x00000000  A15     : 0x00000000  SAR     : 0x00000004  EXCCAUSE: 0x00000007  
EXCVADDR: 0x00000000  LBEG    : 0x00000000  LEND    : 0x00000000  LCOUNT  : 0x00000000  

Backtrace: 0x40087b24:0x3ffb09f0 0x40087c0a:0x3ffb0a30 0x400834e1:0x3ffb0a70 0x40089dce:0x3ffaf4e0 0x400876c5:0x3ffaf500

0x40087b24: i2c_ll_write_txfifo at C:/**/esp-idf/components/hal/esp32/include/hal/i2c_ll.h:564
 (inlined by) i2c_master_cmd_begin_static at C:/**/esp-idf/components/driver/i2c.c:1364
0x40087c0a: i2c_isr_handler_default at C:/**/esp-idf/components/driver/i2c.c:517
0x400834e1: _xt_lowint1 at C:/**/esp-idf/components/freertos/port/xtensa/xtensa_vectors.S:1114
0x40089dce: spi_flash_op_block_func at C:/**/esp-idf/components/spi_flash/cache_utils.c:123
0x400876c5: ipc_task at C:/**/esp-idf/components/esp_ipc/src/esp_ipc.c:77

[kriegste]

Here we go. I finally was able to create a minimum code example. I added the sdkconfig as well as it seems to be important for the crash. For example, running at 240 MHz is crucial... but I have no clue why.

i2c_crash.zip

[kriegste]

Can you reproduce the crash with my example?

[kriegste]

FYI, a freshly installed 4.4.6 crashes reliably, just like the previous version.

[kriegste]

Any news here?

[kriegste]

@igrr maybe you could have a look at this, too? 5.1 is affected as well...

[mythbuster5]

Seems your CMD is not in RAM?

[mythbuster5]

Oh no, this time I see your code I find that this issue has no relationship with this #10079 . That means the I2C Cache disabled but cached memory region accessed only in ISR. But I see your code, that you want all I2C driver can work when flash is working. This is a risk (although currently bugs are reported in ISR.)

But what I can see now is some parameter like A3:0x3f419ed8 so I guess something is still in flash... Don't know how much time I need to reproduce the issue. Looks fine so far.

[kriegste]

Oh, yes, you have found the cause.

A pre-defined I2C command sequence like (from the example)

static const uint8_t CMD[] = {
	0x00, 0x00
};

ends up in flash. Then, in i2c_master_cmd_begin_static (in i2c.c) the pointer to this sequence is handed down to i2c_ll_write_txfifo (in i2c_ll.h). Both functions are in IRAM, but the data is not. So, the correct way to define a command sequence would be

static const DRAM_ATTR uint8_t CMD[] = {
	0x00, 0x00
};

I tried it, and it does not crash with the same error any more. In my example, the task wdt is triggered, though. I don't know, if this is normal.

I think, there should be a warning. Often it is difficult to find out what the compiler does with variables (even without "const").

[kriegste]

A check could be implemented, like

ESP_RETURN_ON_FALSE(esp_ptr_in_dram(data), ESP_ERR_INVALID_ARG, I2C_TAG, "data not in DRAM");

in i2c_master_write. However, it would be best to have this at compile time...

[mythbuster5]

4.4 is EOL, so I will close this issue.