Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make open/write defer opening an object-stream (PUT), like open/read does (with GET) #112

Closed
jti-lanl opened this issue Mar 15, 2016 · 5 comments
Assignees

Comments

@jti-lanl
Copy link
Contributor

This was item "(3)" in issue #83. Moving here, so we can close that.

This comes up for the MD scaling demo. Otherwise, we don't particularly care about zero-length files.

Refactor MarFS write() to work like read()
-- don't start the PUT in open(), wait until the write().
-- zero-length files might have no object? (Maybe that's good.)
-- tweak code to handle zero-length files?

The point of this is to make it easier/cheaper to write zero-length files. Currently, we'd pay the cost of opening/closing a PUT for each zero-length file.

But, zero-length files invite questions, which I've recorded in item #111.

@jti-lanl jti-lanl changed the title Make open/write defer streams, like open/read does Make open/write defer opening and object-stream (PUT), like open/read does (with GET) Mar 15, 2016
@jti-lanl jti-lanl changed the title Make open/write defer opening and object-stream (PUT), like open/read does (with GET) Make open/write defer opening an object-stream (PUT), like open/read does (with GET) Mar 15, 2016
@jti-lanl
Copy link
Contributor Author

The suggestion in #111 to use a DIRECT repo, for size=0, would make this problem go away (?)

@brettkettering
Copy link
Contributor

Related to #96.

@brettkettering
Copy link
Contributor

A good candidate for the DAL (Data Abstraction Layer) work.

wfvining added a commit that referenced this issue Oct 4, 2016
…led.

This is functionality we have wanted for a while (issue #112), and is
potentialy needed as part of the fix for issue #163. Additionally this
should give us better throughput when overwriting large directory trees.
@wfvining
Copy link
Contributor

Jeff and I decided we should do this in MarFS itself, rather than leaving it to the various DAL implementations. I have a working implementation in the lazy-open branch which needs further integration testing against both the OBJ and MC DALs, but should be just about ready to merge.

@wfvining wfvining self-assigned this Dec 19, 2016
@brettkettering brettkettering removed this from the Program Development Demo milestone Feb 1, 2017
@wfvining
Copy link
Contributor

This has been done as part of implementing marfs_flush().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants