Add support for recursive delete in EncryptedFileSystem #118
Conversation
| boolean deleteSuccessful = delete(child.getPath(), true); | ||
|
|
||
| if (!deleteSuccessful) { | ||
| return false; |
There was a problem hiding this comment.
The definition of delete in the Hadoop documentation does not specify whether we should stop immediately here, or continue with the rest of the delete. If we're modeling this after rm -rf then I'd expect it to continue deleting what it can though, right?
Not sure if there's a reference impl elsewhere we can look at.
There was a problem hiding this comment.
Easy to do either, but yeah, I'm unsure what behavior is preferable. RawLocalFileSystem continues and eventually returns false if anything failed. FTPFileSystem continues and ignores any intermediate issues (maybe a bug). DistributedFileSystem is hard to follow, but seems to atomically destroy everything, making this is a non-issue. S3FileSystem matches the behavior of this PR, returning early if anything fails.
There was a problem hiding this comment.
I'd really prefer to avoid supporting recursive deletes within EncryptedFileSystem until we can do so efficiently. I'd propose that you do:
RemoteIterator<LocatedFileStatus> files = efs.listFiles(path, true);
while (files.hasNext()) {
efs.delete(files.next().getPath(), false);
}
as a workaround for the time being.
Fixes #117
We want to be able to delete folders of many items. CC @eraboin
If this change seems generally reasonable, I can add more tests