410 errors periodically stopping the backup

[cjenow]

After about 2000 successful photo downloads, I'm getting a 410 error upon each and every file download/write attempt. I was able to get the backup to continue by simply re-running the script, but this involved a very long delay of enumerating all ~34,000 of my photos again.

I'm locally testing a change that, upon a 410 error, automatically re-auths & re-requests/generates the album object.

I'm curious if anyone knows why the 410 download errors are happening and how best to re-design the script, ideally to not hit them, or at least to be resilient if they occur.

[cjenow]

Hi @pablogzalez - for your info:

1. I did get the full backup (of about 150 GB of photos and videos) to complete after I added logic to handle the 410 error by re-starting the whole iCloud photos enumeration and backup process--not an ideal or efficient solution, but it worked.

except PyiCloudAPIResponseException as descargar_error:
            print(colored(f"\n🔴 Error al descargar {photo_id_with_filename}: {descargar_error}. Omitiendo...", "red"))
            if "Gone" in str(descargar_error): 
                print(colored(f"\n🔴 Detalles: Error 410 al descargar {photo_id_with_filename}: {descargar_error}. Re-authing, then recursing...", "red"))
                api = authenticate(username, password)  # Re-authenticate
                backup_photos_to_local(api, album_name, dest_directory)

2. It's appearing to me that the 410 errors (when initially, as with the code in the main branch, not handled by re-starting the backup process) are what initially caused the zero-byte files to be written (because each file write began, creating a zero byte file, but then each download failed, as soon as the 410 errors began; as soon as one 410 error occurred, it would continue occurring for every next photo download attempt too, thus creating many, many zero-byte files.)