Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LFS_F_WRUNCHECKED option to avoid validation and hence reading blocks when writing #1056

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

andyp1per
Copy link

In ArduPilot we predominantly stream logs to disk while flying. In our current block logger we do this without validation and this PR adds an option to skip validation while writing to streamline the writing process.

Also fixes a build issue with the file backend when compiling without debug.

@geky-bot
Copy link
Collaborator

Tests passed ✓, Code: 17136 B (+0.0%), Stack: 1440 B (+0.0%), Structs: 812 B (+0.0%)
Code Stack Structs Coverage
Default 17136 B (+0.0%) 1440 B (+0.0%) 812 B (+0.0%) Lines 2400/2572 lines (-0.0%)
Readonly 6250 B (+0.0%) 448 B (+0.0%) 812 B (+0.0%) Branches 1283/1616 branches (+0.0%)
Threadsafe 17984 B (+0.0%) 1440 B (+0.0%) 820 B (+0.0%) Benchmarks
Multiversion 17208 B (+0.0%) 1440 B (+0.0%) 816 B (+0.0%) Readed 29369693876 B (+0.0%)
Migrate 18824 B (+0.0%) 1736 B (+0.0%) 816 B (+0.0%) Proged 1482874766 B (+0.0%)
Error-asserts 17900 B (+0.0%) 1432 B (+0.0%) 812 B (+0.0%) Erased 1568888832 B (+0.0%)

@geky geky added enhancement needs minor version new functionality only allowed in minor versions labels Dec 19, 2024
@geky
Copy link
Member

geky commented Dec 19, 2024

Hi @andyp1per, thanks for creating a PR.

Have you considered making this a filesystem-wide ifdef, say, LFS_NO_CKPROGS? That way you could also avoid the reads to validate metadata commits (here). I have a similar thing on a fork (albeit via mount flags).

I'm curious, did this impact performance noticeably? I've been viewing this as low priority as I assumed the read overhead for validation would always be overshadowed by the related prog+erase overhead.

@andyp1per
Copy link
Author

Hi @andyp1per, thanks for creating a PR.

Have you considered making this a filesystem-wide ifdef, say, LFS_NO_CKPROGS? That way you could also avoid the reads to validate metadata commits (here). I have a similar thing on a fork (albeit via mount flags).

I'm curious, did this impact performance noticeably? I've been viewing this as low priority as I assumed the read overhead for validation would always be overshadowed by the related prog+erase overhead.

I am interested in other types of file being checked, that's why I went for a per-file option. It might also be possible to set it temporarily - so for instance our logs are self-describing, we probably want the description to be stored reliably but the data not so much.

read is about 10x faster than write and it also chews up the SPI bus, so not nothing - in my testing so far this plus avoiding checking the file size seems to get me where I need to, but I need to do another full flight test to fully check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement needs minor version new functionality only allowed in minor versions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants