Improve OTA process, reduce LittleFS partition

[dbeinder]

nvs,data,nvs,0x9000,20K,
otadata,data,ota,0xe000,8K,
app0,app,ota_0,0x10000,1280K,
app1,app,ota_1,0x150000,1280K,
spiffs,data,spiffs,0x290000,1408K,
coredump,data,coredump,0x3f0000,64K,

With two app partitions, the LittleFS partition doesn't need to be as large as it is right now. To make space for future features and address the C3 issue (#786), it would be nice to reduce the FS to 256k or so and grow the app partitions. Then write directly into the inactive one during update and swap the booting partition after verifying. This would give ~1.8MB app space on 4MB devices.

[gskjold]

This is a good idea, which I have also though about in the past, but as long as the manual upload is temporarily stored on littlefs, this will not get high priority from me. Other priorities have kept me from rewriting the upload feature.

Also (as far as I know) there is no way to re-partition existing installations through a simple firmware upgrade, making this useful only for new devices, leaving existing devices stuck on the last version that fit the current partitioning scheme. This is a deal breaker for us, as we have some thousands out there already.

I must also admit I have not done too much research on any of these aspects, so the solution may be simple. I am open for hints and discussions on this matter.

[dbeinder]

Re-partitioning should be as simple as writing a new partitions.bin to the correct flash offset and rebooting. Since the start offset of app0 doesn't change, we'd just have to make sure app0 is the currently active partition. So upgrading may involve copying app1 to app0, swapping and rebooting before re-partitioning. Garbage at the end of app0 that was previously the start of app1 shouldn't be an issue, since you aren't using signed updates. The other thing would be changing the update URL, to make sure users don't skip the version that does the re-partition.

Edit: Moving LittleFS might be tricky, if its contents don't fit into RAM. But it can also be done by creating a temporary LittleFS on app1 before repartitioning, copying its contents there. Then creating the new LittleFS at correct offset of the future spiffs partition and copying everything from the temp filsystem to its final destination.

[gskjold]

Thank you for the valuable input, really appreciate it. I will slot out some time to look closer on this, as it sounds like the best way forward

[gskjold]

Just as an update, I have a working solution locally, but there are some details I need to sort out still which I have not yet found time for.

[gskjold]

This feature is now out for testing in #879