Only upload file to device if remote is different #107
Conversation
Gets a hash of the remote file and compares it to a hash of the local file. If they are different it uploads. If not it skips over.
|
Found a few bugs with this PR so I'll fix them when I have a chance. |
|
@jdjjm Thanks for working on this! It's an interesting idea. Let's see if it brings the benefits of faster uploading without adding bugs or too much complexity to the code base. I'll mark this PR as draft while you're working on it. Feel free to convert it back and/or ask me to review it. |
@jdjjm You can improve it further. If you check if the files are of different size, then it makes no sense to continue with the hash, since you already know it differs. So even if you upload a single file (that has been edited and changed its size), you can skip calculating the hash. |
|
I hadn't thought of that but that is a good idea. I think os.stat might be implemented in uos so that could be an easy way to get the file size. |
Fixed windows path issue when finding the remote path. Used ubinascii to get hexdigest as string when finding hash to fix bug when hashing certain hex values.
Removed print commands used for development
|
@vlasovskikh PR is ready for testing. Let me know if you have any issues and remember you need to set the --different flag to enable it. |
Removed some debug print statements I forgot to remove.
|
added support for esp32 seguindo seu conselho, estou tentando aprender kotlin. Edit by @vlasovskikh: I've removed the link to the *.zip file since this file is not the official version of intellij-micropython. |
|
@marcelocrispim If you have changes you have worked on please create a new pull request for it to be reviewed. This pull request is only for discussing what the title outlines. |
|
@marcelocrispim Could you please send your changes as a separate PR? Also I would like to kindly ask you not to publish non-official builds of the intellij-micropython plugin with the name of the official plugin and select a different name instead. "intellij-micropython-1.1.2" sends a wrong message that it may be the next official release of intellij-micropython while it isn't. |
|
@vlasovskikh sorry, I will make the pull request over the weekend best Regards |
| remote_file_size = files.get_size(remote_path) | ||
| if local_file_size == remote_file_size: | ||
| remote_file_hash = files.get_hash(remote_path) | ||
| if remote_file_hash == local_file_hash: | ||
| print("File Identical... Skipping upload", file=sys.stderr, flush=True) | ||
| else: | ||
| wait_for_board() | ||
| files.put(remote_path, raw) | ||
| else: | ||
| wait_for_board() | ||
| files.put(remote_path, raw) |
There was a problem hiding this comment.
You can simplify to something like,
| remote_file_size = files.get_size(remote_path) | |
| if local_file_size == remote_file_size: | |
| remote_file_hash = files.get_hash(remote_path) | |
| if remote_file_hash == local_file_hash: | |
| print("File Identical... Skipping upload", file=sys.stderr, flush=True) | |
| else: | |
| wait_for_board() | |
| files.put(remote_path, raw) | |
| else: | |
| wait_for_board() | |
| files.put(remote_path, raw) | |
| if local_file_size == files.get_size(remote_path) and local_file_hash == files.get_hash(remote_path): | |
| print("File Identical... Skipping upload", file=sys.stderr, flush=True) | |
| else: | |
| wait_for_board() | |
| files.put(remote_path, raw) | |
| else: | |
| files.put(remote_path, fd.read()) |
The interpreter will short-circuit if the size check local_file_size == files.get_size(remote_path) evaluates false, and not calculate the hash too.
It sounds like a good enhancement 😄 I have some big SSL certificates that take a while to re-upload, so I find myself cherrypicking only files that have changed to flash. This would automate that. |
This PR adds the ability to check if the remote file is different to the local file on upload. If its the same it can skip uploading to save time.
For a single file upload this will increase the time to upload it takes a bit under a second to check if they are different plus the time to upload. However, when working on larger projects with multiple files, if you are only modifying a couple at a time it can speed up the upload process as it will only upload the files that need to be uploaded. It also means you don't need to upload each file individually.
You have to add the
--differentoption to the script call otherwise it will use the default to uploading all files.Just wanted everyone's opinion on whether they see themselves using this or just prefer the old way.
If it was to be implemented I would recommend the run configuration to have a checkbox on whether it is enabled or not. I would also recommend disabling it for single file uploads and save it for project level or other multi-file uploads.