代码问题

[wzl0422]

尊敬的作者您好！
我在运行bash scripts/pretrain_fb15k-237-image.sh时有以下警告：
1、/root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.)
tensor = as_tensor(value)
2、另外，每一个epoch结束后，是进行了100%，但是为什么终端显示的都是50%
如下：
Epoch 2: 50%|██████████████████████████████████████████████████████████████████████████▌ | 234/468 [02:28<02:28, 1.58it/s, loss=8.93, v_num=1, hits10=0.232, hits1=0.141]
/root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.)
tensor = as_tensor(value)
/root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.)
tensor = as_tensor(value)
/root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.)
tensor = as_tensor(value)
/root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.)
tensor = as_tensor(value)

3、还提示我数据集中有损坏的图片，我不明白这是什么意思，我直接从所给出的google链接下载的数据集。
我想知道有问题的图片是说的下图所示这种吗？还是什么意思？

还请作者解答疑惑！感激不尽！

[wzl0422]

这个警告信息表明，当前代码在将 numpy.ndarray 列表转换为 PyTorch 张量时速度非常慢。具体原因是，transformers 库中的 feature_extraction_utils.py 正在逐个元素地将列表转换为张量，这种方法会导致较高的计算开销

[njcx-ai]

您好，非常感谢您对我们工作的关注。我们将尽快排查下具体是啥问题，明晚之前给您回复。

[flow3rdown]

尊敬的作者您好！ 我在运行bash scripts/pretrain_fb15k-237-image.sh时有以下警告： 1、/root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.) tensor = as_tensor(value) 2、另外，每一个epoch结束后，是进行了100%，但是为什么终端显示的都是50% 如下： Epoch 2: 50%|██████████████████████████████████████████████████████████████████████████▌ | 234/468 [02:28<02:28, 1.58it/s, loss=8.93, v_num=1, hits10=0.232, hits1=0.141] /root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.) tensor = as_tensor(value) /root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.) tensor = as_tensor(value) /root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.) tensor = as_tensor(value) /root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.) tensor = as_tensor(value)

3、还提示我数据集中有损坏的图片，我不明白这是什么意思，我直接从所给出的google链接下载的数据集。 我想知道有问题的图片是说的下图所示这种吗？还是什么意思？ 还请作者解答疑惑！感激不尽！

1. 最好是确认一下哪一行导致的这个警告，目测是因为clip process导致的，可能是跟transformers的版本有关，但如果不影响模型的结果的话可以忽略；
2. 这个跟你的系统和终端环境有关系；
3. 如果本地无法打开这个图片的话说明这个图片是有损坏的。

[wzl0422]

尊敬的作者您好！ 我在运行bash scripts/pretrain_fb15k-237-image.sh时有以下警告： 1、/root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.) tensor = as_tensor(value) 2、另外，每一个epoch结束后，是进行了100%，但是为什么终端显示的都是50% 如下： Epoch 2: 50%|██████████████████████████████████████████████████████████████████████████▌ | 234/468 [02:28<02:28, 1.58it/s, loss=8.93, v_num=1, hits10=0.232, hits1=0.141] /root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.) tensor = as_tensor(value) /root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.) tensor = as_tensor(value) /root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.) tensor = as_tensor(value) /root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.) tensor = as_tensor(value)
3、还提示我数据集中有损坏的图片，我不明白这是什么意思，我直接从所给出的google链接下载的数据集。 我想知道有问题的图片是说的下图所示这种吗？还是什么意思？ 还请作者解答疑惑！感激不尽！

1. 最好是确认一下哪一行导致的这个警告，目测是因为clip process导致的，可能是跟transformers的版本有关，但如果不影响模型的结果的话可以忽略；
2. 这个跟你的系统和终端环境有关系；
3. 如果本地无法打开这个图片的话说明这个图片是有损坏的。

1、那这个图片需要手动删除吗？还是不需要管，让他保留？
2、另外话有一个问题是：在我运行该脚本内容时bash scripts/fb15k-237-image.sh，运行时长特别久，一个epoch就要1小时，我在此用的时3090的显卡，24GB的内存。我想知道是什么原因造成的呢？是因为显卡太差了还是因为上面提示的错误(这个警告信息表明，当前代码在将 numpy.ndarray 列表转换为 PyTorch 张量时速度非常慢。具体原因是，transformers 库中的 feature_extraction_utils.py 正在逐个元素地将列表转换为张量，这种方法会导致较高的计算开销)造成的？那个错误并没有具体表明哪句话，而是：/root/miniconda3/envs/MKGformer/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:210.)
tensor = as_tensor(value)