[Bug] OrphanFilesClean has a chance to delete normal files

[wxplovecc]

Search before asking

• I searched in the issues and found nothing similar.

Paimon version

master

Compute Engine

flink

Minimal reproduce step

1、Start a streaming write job with ckp (5 second)
2、Sleep some time between
Set<String> usedFiles = getUsedFiles(); Thread.sleep(20000) Map<String, Path> candidates = getCandidateDeletingFiles();
3、Do clean orphanFiles
4、

What doesn't meet your expectations?

Should not delete normal file

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[yuzelin]

In previous design, getCandidateDeletingFiles is called after getUsedFiles, so it may read newly committed files, and the newly committed files are not in usedFiles, so these files will be deleted incorrectly, am I right?
But we use oldEnough to protect the new files, so I don't know why the files are deleted. Can you explain more details?

[yuzelin]

Anyway, put getUsedFiles() after getCandidateDeletingFiles() is correct. I think your change is right.