Problems porting Reliance Edge File System to ESP32 - No Persistence

[Tuxera-JamieF]

[Originally asked and responded to in the Forum from 2019-08-04 – 2019-08-20]

[Customer]
I’m trying use the url=https://github.com/datalightinc/reliance-edge/Reliance Edge library to a project I’m developing with ESP32, which is a fail-safe file system for SD cards. I have found a https://github.com/lllucius/esp32_redflash GitHub project that ports the library to ESP32, but I am having some issues trying to persist a written file. My application is just initializing the port library, which initializes the file system. After that, I’m writing a .txt file and reading it right afterwards. The problem is that when I reprogram the board to just read those test files, I find out that they doesn’t persist in the SD card when I try to just read the file written last time. A section of my code is presented below:

test_setup(OPENFILES);
char *text = “Hello World!\r\n”;
write_test(MOUNT_POINT “/hello.txt”,text);
char *resp = read_test(MOUNT_POINT “/hello.txt”) /code

Some more information about the code above: OPENFILES is the max files I can open, defined as 4
MOUNT_POINT is another define (/redflash)
test_setup() doesn’t format the SD card at any moment
write_test() and read_test() are basically just fwrite() and fread() with appropriate fclose()

When I reprogram ESP, I comment lines 2 and 3 leaving just the reading section. The error occurs right after it tries to fread() a FILE pointer that is NULL by fopen(). I can provide more code and documentation if someone has any idea of what might be happening. Thanks in advance!

[Tuxera-JamieF]

[Tuxera]
It would be helpful to see the actual code—test_setup(), write_test(), read_test()—in order to determine if there is an application error. A few other thoughts:

• Is a transaction point occurring? If there is no transaction point, the file won’t persist. By default, creating a file will result in a transaction point, but that can be disabled.

• If you have access to a Linux system, can you build the FUSE driver? (Copy the redconf.h/redconf.c into projects/linux, cd into projects/linux/host and make redfuse.) If so, do the files created on the target show up via FUSE? Do files created via FUSE show up on the target?

• The problem might be that the SD/MMC sector writes aren’t working correctly.

If this doesn't answer your question, you can send the requested code to [email protected]?

[Tuxera-JamieF]

[Customer]
I am aware of the fact that a transaction point is necessary, and also that it is activated after writing files. It would be great if someone indicated me which file that this configuration could be changed. Is it related to those redconf.h and redconf.c files? About the FUSE driver, I am actually using Windows, so I would have to compile the Image Builder tool, but even following the steps from the developer's guide I cannot make the executable. Any help would be great. About the code, my VFS is already configured to use all the Reliance Edge functions such as red_open() and red_write(), so my application calls fopen() and fwrite() inside those functions write_test() and read_test() that I created. I would post the test_setup() function here if the forum fomatting worked better, because it is a bit long. I will probably send it to the Support email. Thanks!

[Tuxera-JamieF]

]I am aware of the fact that a transaction point is necessary, and also that it is activated after writing files. It would be great if someone indicated me which file that this configuration could be changed. Is it related to those redconf.h and redconf.c files?

[Tuxera] The default transaction mask doesn’t include an automatic transaction for red_write(). It does, however, include automatic transactions for red_close() and red_fsync(). The transaction mask can be found in redconf.h: it’s the REDCONF_TRANSACT_DEFAULT macro. You can edit it by hand or use the Reliance Edge Configuration Utility. In addition, the transaction mask can be changed at run-time via the red_settransmask() API (although that might not be accessible via the VFS). I wouldn’t usually recommend enabling RED_TRANSACT_WRITE, because it hurts performance. Transacting on close and fsync is typically more appropriate. However, since you are accessing Reliance Edge indirectly, via the VFS, you might have to investigate how the high-level options (like fwrite() and fclose()) are implemented in order to determine when an automatic transaction point can be expected.

] About the FUSE driver, I am actually using Windows, so I would have to compile the Image Builder tool, but even following the steps from the developers guide I cannot make the executable. Any help would be great.

[Tuxera] We will need more details in order to assist you: the OS, Visual Studio version, build log. Usually, it’s as simple as installing a version of Visual Studio (the Community Edition will work), installing GNU make, and then running “make” from projects/win32/host.

[Tuxera-JamieF]

[Customer]
Hi, thanks for the reply. I understand that using an API with Reliance Edge is a bit more complicated than accessing the library directly, but I am certain that my application is calling the right functions, such as red_open() and red_close(). I am logging my results through a serial port, and whenever I call the standard functions like fopen() and fclose(), the API does its job and calls the Reliance Edge functions. About the REDCONF_TRANSACT_DEFAULT macro, thanks for the indication, however I am calling fclose() when I write and read a file, and its transaction point is already enabled. Exploring a bit more, I have been analyzing a few example codes that uses Reliance Edge, and every one of them formats the partition before mounting it in the beginning. Is it some kind of standard? Another difficulty I am facing is related to Image Builder commands and its syntax (host tools successfully compiled). I am getting an "error number 22 formatting volume", what does it mean?

[Tuxera-JamieF]

[Tuxera]
The volume does need to be formatted before it can be mounted for the first time, but assuming the storage medium is persistent, subsequent mounts should succeed without a format. If that isn’t working, there are probably storage device I/O issues.

Error 22 is RED_EINVAL. Usually that means the configuration is invalid. Try copying the redconf.h and redconf.c from the target into projects/win32 (not projects/win32/host: it has special redconf files that make use of the configuration files in the parent directory).