forked from artidoro/qlora
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathgenerate_v2-51388579.err
53 lines (52 loc) · 4.62 KB
/
generate_v2-51388579.err
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
Running command git clone --quiet https://github.com/huggingface/transformers.git /tmp/pip-install-dpe7foqj/transformers_b2d718b4d3054bc28ff90dd4fb44c04b
Running command git clone --quiet https://github.com/huggingface/peft.git /tmp/pip-install-dpe7foqj/peft_a7f0faac2d4b4124b927504a46ee9e92
Running command git clone --quiet https://github.com/huggingface/accelerate.git /tmp/pip-install-dpe7foqj/accelerate_eec8dde8d3c345a9b42d9946ec933726
/home/brc4cb/.conda/envs/falcon_40B/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/home/brc4cb/.conda/envs/falcon_40B/lib/libcudart.so'), PosixPath('/home/brc4cb/.conda/envs/falcon_40B/lib/libcudart.so.11.0')}.. We'll flip a coin and try one of these, in order to fail forward.
Either way, this might cause trouble in the future:
If you get `CUDA error: invalid device function` errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.
warn(msg)
Loading checkpoint shards: 0%| | 0/9 [00:00<?, ?it/s]Loading checkpoint shards: 11%|█ | 1/9 [00:06<00:49, 6.17s/it]Loading checkpoint shards: 22%|██▏ | 2/9 [00:11<00:38, 5.53s/it]Loading checkpoint shards: 33%|███▎ | 3/9 [00:16<00:32, 5.47s/it]Loading checkpoint shards: 44%|████▍ | 4/9 [00:22<00:27, 5.46s/it]Loading checkpoint shards: 56%|█████▌ | 5/9 [00:27<00:21, 5.36s/it]Loading checkpoint shards: 67%|██████▋ | 6/9 [00:32<00:16, 5.46s/it]Loading checkpoint shards: 78%|███████▊ | 7/9 [00:38<00:10, 5.45s/it]Loading checkpoint shards: 89%|████████▉ | 8/9 [00:43<00:05, 5.43s/it]Loading checkpoint shards: 100%|██████████| 9/9 [00:48<00:00, 5.14s/it]Loading checkpoint shards: 100%|██████████| 9/9 [00:48<00:00, 5.36s/it]
/home/brc4cb/.conda/envs/falcon_40B/lib/python3.9/site-packages/transformers/generation/utils.py:1261: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation )
warnings.warn(
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
Setting `pad_token_id` to `eos_token_id`:11 for open-end generation.
slurmstepd: error: *** JOB 51388579 ON udc-an37-1 CANCELLED AT 2023-07-03T12:17:32 ***