Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dlt-system uses a lot of page cache during file transfer #35

Open
jeremiah opened this issue Jan 23, 2018 · 2 comments
Open

dlt-system uses a lot of page cache during file transfer #35

jeremiah opened this issue Jan 23, 2018 · 2 comments
Assignees
Labels

Comments

@jeremiah
Copy link
Contributor

dlt-system reads file content block by block only once and sends it to the dlt library. The file content is never read again, thus file access could be done in O_DIRECT mode without consuming as much memory resources.

@anoojgopi
Copy link

May be we shall change the subject of the ticket as "dlt-system uses a lot of page cache during file transfer", which make more sense.

@jeremiah jeremiah changed the title dlt-system uses a lot of heap during file transfer dlt-system uses a lot of page cache during file transfer Jan 24, 2018
@ssugiura ssugiura assigned ssugiura and unassigned clipka May 30, 2019
@mfukar
Copy link

mfukar commented Jul 12, 2019

posix_fadvise with an appropriate hint (e.g. POSIX_FADV_SEQUENTIAL) is best, O_DIRECT has other implications which are orthogonal to the problem described.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants