[Q] Parallel re-upload

[svasan]

Hi,

For our use case, we may sometimes upload a file more than once. (The previous version is still live.) This is not the usual behaviour, but it's okay when it happens. This works for small files but for large files that use parallel upload and compose, uploading again fails with a "Precondition Failed" error:

Permanent error in ComposeObject: {\n "error": {\n "code": 412,\n "message": "Precondition Failed",\n "errors": [\n {\n "message": "Precondition Failed",\n "domain": "global",\n "reason": "conditionNotMet",\n "locationType": "header",\n "location": "If-Match"\n }\n ]\n }\n}\n [FAILED_PRECONDITION]'

That seems to be caused by this IfGenerationMatch(0) header in the last operation in ComposeMany(...). With that removed, uploading large files again works as expected.

If overwriting the older file is okay in these cases, are there any issues in removing that IfGenerationMatch(0) and letting compose do that? Is there a particular reason that check is enforced for the compose operation?

Thanks.

[coryan]

That looks like a bug to me. The final compose is the one that should not be using IfGenerationMatch(0), as this is creating the object defined by the application.

Previous compose operations may want to use IfGenerationMatch(0) because that prevents erasing customer data if there is a clash with the temporary object names. I say "may" because we also need to deal with upload operations that fail and are restarted, using the same temporary objects.

In any case, thanks for pointing this out, I will open a bug from this question.

[coryan]

I just sent a PR (#6908) to fix this problem, thanks for the detailed analysis!

[svasan]

Great, Thanks!