urlopen error retrieval incomplete in modules/autoencoder_mednist.ipynb

[KumoLiu]

[2024-09-07T20:03:50.937Z] Running ./modules/autoencoder_mednist.ipynb
[2024-09-07T20:03:50.937Z] Checking PEP8 compliance...
[2024-09-07T20:03:50.937Z] Running notebook...
[2024-09-07T20:03:50.937Z] Before:
[2024-09-07T20:03:50.937Z]     "def train(dict_key_for_training, max_epochs=10, learning_rate=1e-3):\n",
[2024-09-07T20:03:50.937Z]     "max_epochs = 50\n",
[2024-09-07T20:03:50.937Z] After:
[2024-09-07T20:03:50.937Z]     "def train(dict_key_for_training, max_epochs = 1, learning_rate=1e-3):\n",
[2024-09-07T20:03:50.937Z]     "max_epochs = 1\n",
[2024-09-07T20:03:53.451Z] WARNING:torch_tensorrt.dynamo.conversion.aten_ops_converters:Unable to import quantization op. Please install modelopt library (https://github.com/NVIDIA/TensorRT-Model-Optimizer?tab=readme-ov-file#installation) to add support for compiling quantized models
[2024-09-07T20:03:56.713Z] MONAI version: 1.4.0rc6+8.gd02ba11d
[2024-09-07T20:03:56.713Z] Numpy version: 1.24.4
[2024-09-07T20:03:56.713Z] Pytorch version: 2.5.0a0+872d972e41.nv24.08
[2024-09-07T20:03:56.713Z] MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
[2024-09-07T20:03:56.713Z] MONAI rev id: d02ba11d8069870d71316a616f047c499627c71c
[2024-09-07T20:03:56.713Z] MONAI __file__: /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/__init__.py
[2024-09-07T20:03:56.713Z] 
[2024-09-07T20:03:56.713Z] Optional dependencies:
[2024-09-07T20:03:56.713Z] Pytorch Ignite version: 0.4.11
[2024-09-07T20:03:56.713Z] ITK version: 5.4.0
[2024-09-07T20:03:56.713Z] Nibabel version: 5.2.1
[2024-09-07T20:03:56.713Z] scikit-image version: 0.19.3
[2024-09-07T20:03:56.713Z] scipy version: 1.14.0
[2024-09-07T20:03:56.713Z] Pillow version: 7.0.0
[2024-09-07T20:03:56.713Z] Tensorboard version: 2.16.2
[2024-09-07T20:03:56.713Z] gdown version: 5.2.0
[2024-09-07T20:03:56.713Z] TorchVision version: 0.20.0a0
[2024-09-07T20:03:56.713Z] tqdm version: 4.66.5
[2024-09-07T20:03:56.713Z] lmdb version: 1.5.1
[2024-09-07T20:03:56.713Z] psutil version: 6.0.0
[2024-09-07T20:03:56.713Z] pandas version: 2.2.2
[2024-09-07T20:03:56.713Z] einops version: 0.7.0
[2024-09-07T20:03:56.713Z] transformers version: 4.40.2
[2024-09-07T20:03:56.713Z] mlflow version: 2.16.0
[2024-09-07T20:03:56.713Z] pynrrd version: 1.0.0
[2024-09-07T20:03:56.713Z] clearml version: 1.16.4
[2024-09-07T20:03:56.713Z] 
[2024-09-07T20:03:56.713Z] For details about installing the optional dependencies, please visit:
[2024-09-07T20:03:56.713Z]     https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
[2024-09-07T20:03:56.713Z] 
[2024-09-07T20:03:58.599Z] papermill  --progress-bar --log-output -k python3
[2024-09-07T20:03:59.157Z] /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:149: UserWarning: the file is not specified with any extension : -
[2024-09-07T20:03:59.157Z]   warnings.warn(f"the file is not specified with any extension : {os.path.basename(path)}")
[2024-09-07T20:04:22.500Z] 
Executing:   0%|          | 0/25 [00:00<?, ?cell/s]
Executing:   4%|▍         | 1/25 [00:00<00:22,  1.06cell/s]
Executing:  16%|█▌        | 4/25 [00:10<01:00,  2.88s/cell]
Executing:  24%|██▍       | 6/25 [00:19<01:04,  3.40s/cell]
Executing:  44%|████▍     | 11/25 [00:20<00:21,  1.53s/cell]
Executing:  44%|████▍     | 11/25 [00:23<00:29,  2.11s/cell]
[2024-09-07T20:04:22.500Z] /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:149: UserWarning: the file is not specified with any extension : -
[2024-09-07T20:04:22.500Z]   warnings.warn(f"the file is not specified with any extension : {os.path.basename(path)}")
[2024-09-07T20:04:22.500Z] Traceback (most recent call last):
[2024-09-07T20:04:22.500Z]   File "/usr/local/bin/papermill", line 8, in <module>
[2024-09-07T20:04:22.500Z]     sys.exit(papermill())
[2024-09-07T20:04:22.500Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
[2024-09-07T20:04:22.500Z]     return self.main(*args, **kwargs)
[2024-09-07T20:04:22.500Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
[2024-09-07T20:04:22.500Z]     rv = self.invoke(ctx)
[2024-09-07T20:04:22.500Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
[2024-09-07T20:04:22.500Z]     return ctx.invoke(self.callback, **ctx.params)
[2024-09-07T20:04:22.501Z]   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
[2024-09-07T20:04:22.501Z]     return __callback(*args, **kwargs)
[2024-09-07T20:04:22.501Z]   File "/usr/local/lib/python3.10/dist-packages/click/decorators.py", line 33, in new_func
[2024-09-07T20:04:22.501Z]     return f(get_current_context(), *args, **kwargs)
[2024-09-07T20:04:22.501Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/cli.py", line 235, in papermill
[2024-09-07T20:04:22.501Z]     execute_notebook(
[2024-09-07T20:04:22.501Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 131, in execute_notebook
[2024-09-07T20:04:22.501Z]     raise_for_execution_errors(nb, output_path)
[2024-09-07T20:04:22.501Z]   File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 251, in raise_for_execution_errors
[2024-09-07T20:04:22.501Z]     raise error
[2024-09-07T20:04:22.501Z] papermill.exceptions.PapermillExecutionError: 
[2024-09-07T20:04:22.501Z] ---------------------------------------------------------------------------
[2024-09-07T20:04:22.501Z] Exception encountered at "In [6]":
[2024-09-07T20:04:22.501Z] ---------------------------------------------------------------------------
[2024-09-07T20:04:22.501Z] ContentTooShortError                      Traceback (most recent call last)
[2024-09-07T20:04:22.501Z] Cell In[6], line 7
[2024-09-07T20:04:22.501Z]       5 data_dir = os.path.join(root_dir, "MedNIST")
[2024-09-07T20:04:22.501Z]       6 if not os.path.exists(data_dir):
[2024-09-07T20:04:22.501Z] ----> 7     download_and_extract(resource, compressed_file, root_dir, md5)
[2024-09-07T20:04:22.501Z] 
[2024-09-07T20:04:22.501Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:335, in download_and_extract(url, filepath, output_dir, hash_val, hash_type, file_type, has_base, progress)
[2024-09-07T20:04:22.501Z]     333 with tempfile.TemporaryDirectory() as tmp_dir:
[2024-09-07T20:04:22.501Z]     334     filename = filepath or Path(tmp_dir, _basename(url)).resolve()
[2024-09-07T20:04:22.501Z] --> 335     download_url(url=url, filepath=filename, hash_val=hash_val, hash_type=hash_type, progress=progress)
[2024-09-07T20:04:22.501Z]     336     extractall(filepath=filename, output_dir=output_dir, file_type=file_type, has_base=has_base)
[2024-09-07T20:04:22.501Z] 
[2024-09-07T20:04:22.501Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:226, in download_url(url, filepath, hash_val, hash_type, progress, **gdown_kwargs)
[2024-09-07T20:04:22.501Z]     221             raise RuntimeError(
[2024-09-07T20:04:22.501Z]     222                 f"Download of file from {download_url}, received from {url} "
[2024-09-07T20:04:22.501Z]     223                 + f" to {filepath} failed due to network issue or denied permission."
[2024-09-07T20:04:22.501Z]     224             )
[2024-09-07T20:04:22.501Z]     225 else:
[2024-09-07T20:04:22.501Z] --> 226     _download_with_progress(url, tmp_name, progress=progress)
[2024-09-07T20:04:22.501Z]     227 if not tmp_name.exists():
[2024-09-07T20:04:22.501Z]     228     raise RuntimeError(
[2024-09-07T20:04:22.501Z]     229         f"Download of file from {url} to {filepath} failed due to network issue or denied permission."
[2024-09-07T20:04:22.501Z]     230     )
[2024-09-07T20:04:22.501Z] 
[2024-09-07T20:04:22.501Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:119, in _download_with_progress(url, filepath, progress)
[2024-09-07T20:04:22.501Z]     117 except (URLError, HTTPError, ContentTooShortError, OSError) as e:
[2024-09-07T20:04:22.501Z]     118     logger.error(f"Download failed from {url} to {filepath}.")
[2024-09-07T20:04:22.501Z] --> 119     raise e
[2024-09-07T20:04:22.501Z] 
[2024-09-07T20:04:22.501Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:112, in _download_with_progress(url, filepath, progress)
[2024-09-07T20:04:22.501Z]     109             self.update(b * bsize - self.n)  # will also set self.n = b * bsize
[2024-09-07T20:04:22.501Z]     111     with TqdmUpTo(unit="B", unit_scale=True, unit_divisor=1024, miniters=1, desc=_basename(filepath)) as t:
[2024-09-07T20:04:22.501Z] --> 112         urlretrieve(url, filepath, reporthook=t.update_to)
[2024-09-07T20:04:22.501Z]     113 else:
[2024-09-07T20:04:22.501Z]     114     if not has_tqdm and progress:
[2024-09-07T20:04:22.501Z] 
[2024-09-07T20:04:22.501Z] File /usr/lib/python3.10/urllib/request.py:280, in urlretrieve(url, filename, reporthook, data)
[2024-09-07T20:04:22.501Z]     277                 reporthook(blocknum, bs, size)
[2024-09-07T20:04:22.501Z]     279 if size >= 0 and read < size:
[2024-09-07T20:04:22.501Z] --> 280     raise ContentTooShortError(
[2024-09-07T20:04:22.501Z]     281         "retrieval incomplete: got only %i out of %i bytes"
[2024-09-07T20:04:22.501Z]     282         % (read, size), result)
[2024-09-07T20:04:22.501Z]     284 return result
[2024-09-07T20:04:22.501Z] 
[2024-09-07T20:04:22.501Z] ContentTooShortError: <urlopen error retrieval incomplete: got only 41943040 out of 61834679 bytes>