Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CUDA Setup failed despite GPU being available. Please run the following command to get more information: #290

Open
WuRui-Ella opened this issue Jul 4, 2024 · 1 comment

Comments

@WuRui-Ella
Copy link

WuRui-Ella commented Jul 4, 2024

配置:
系统:CentOS Linux release 7.9.2009
CUDA版本: 12.4
python版本: 3.11
NVIDIA-SMI :550.54.14
我在使用hello_world推理时遇到了这个问题,已经提示重新编译了bitsandbytes,但还是出现了这个问题。

问题如下:
CUDA SETUP: Something unexpected happened. Please compile from source:
git clone [email protected]:TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=124
python setup.py install
CUDA SETUP: Setup Failed!
CUDA SETUP: Something unexpected happened. Please compile from source:
git clone [email protected]:TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=124
python setup.py install
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/ToolBench-master/toolbench/inference/qa_pipeline.py:13 in │
│ │
│ 10 os.environ['CUDA_VERSION'] = "12.4" │
│ 11 os.environ['NVCC'] = '/usr/local/cuda-12.4/bin/nvcc' │
│ 12 │
│ ❱ 13 from toolbench.inference.Downstream_tasks.rapidapi import pipeline_runn │
│ 14 │
│ 15 │
│ 16 if name == "main": │
│ │
│ /home/ToolBench-master/toolbench/inference/Downstream_tasks/rapidapi.py:1 │
│ 0 in │
│ │
│ 7 from termcolor import colored │
│ 8 import random │
│ 9 from toolbench.inference.LLM.chatgpt_function_model import ChatGPTFunc │
│ ❱ 10 from toolbench.inference.LLM.davinci_model import Davinci │
│ 11 from toolbench.inference.LLM.tool_llama_lora_model import ToolLLaMALoR │
│ 12 from toolbench.inference.LLM.tool_llama_model import ToolLLaMA │
│ 13 from toolbench.inference.LLM.retriever import ToolRetriever │
│ │
│ /home/ToolBench-master/toolbench/inference/LLM/davinci_model.py:9 in │
│ │
│ │
│ 6 import random │
│ 7 import openai │
│ 8 from typing import Optional │
│ ❱ 9 from toolbench.model.model_adapter import get_conversation_template │
│ 10 from toolbench.inference.utils import SimpleChatIO, react_parser │
│ 11 from toolbench.inference.Prompts.ReAct_prompts import FORMAT_INSTRUCTI │
│ 12 │
│ │
│ /home/ToolBench-master/toolbench/model/init.py:1 in │
│ │
│ ❱ 1 from toolbench.model.model_adapter import ( │
│ 2 │ load_model, │
│ 3 │ get_conversation_template, │
│ 4 │ add_model_args, │
│ │
│ /home/ToolBench-master/toolbench/model/model_adapter.py:21 in │
│ │
│ 18 │ AutoTokenizer, │
│ 19 │ LlamaForCausalLM, │
│ 20 ) │
│ ❱ 21 from peft import PeftModel │
│ 22 │
│ 23 from toolbench.tool_conversation import Conversation, get_conv_templat │
│ 24 from toolbench.model.compression import load_compress_model │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/init.py:2 │
│ 2 in │
│ │
│ 19 │
│ 20 version = "0.3.0" │
│ 21 │
│ ❱ 22 from .mapping import MODEL_TYPE_TO_PEFT_MODEL_MAPPING, PEFT_TYPE_TO_CON │
│ 23 from .peft_model import ( │
│ 24 │ PeftModel, │
│ 25 │ PeftModelForCausalLM, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/mapping.py:16 │
│ in │
│ │
│ 13 # See the License for the specific language governing permissions and │
│ 14 # limitations under the License. │
│ 15 │
│ ❱ 16 from .peft_model import ( │
│ 17 │ PeftModel, │
│ 18 │ PeftModelForCausalLM, │
│ 19 │ PeftModelForSeq2SeqLM, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/peft_model.py │
│ :31 in │
│ │
│ 28 from transformers.modeling_outputs import SequenceClassifierOutput, T │
│ 29 from transformers.utils import PushToHubMixin │
│ 30 │
│ ❱ 31 from .tuners import ( │
│ 32 │ AdaLoraModel, │
│ 33 │ AdaptionPromptModel, │
│ 34 │ LoraModel, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/tuners/__init │
│ __.py:21 in │
│ │
│ 18 # limitations under the License. │
│ 19 │
│ 20 from .adaption_prompt import AdaptionPromptConfig, AdaptionPromptModel │
│ ❱ 21 from .lora import LoraConfig, LoraModel │
│ 22 from .adalora import AdaLoraConfig, AdaLoraModel │
│ 23 from .p_tuning import PromptEncoder, PromptEncoderConfig, PromptEncoder │
│ 24 from .prefix_tuning import PrefixEncoder, PrefixTuningConfig │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/peft/tuners/lora.p │
│ y:40 in │
│ │
│ 37 │
│ 38 │
│ 39 if is_bnb_available(): │
│ ❱ 40 │ import bitsandbytes as bnb │
│ 41 │
│ 42 │
│ 43 @DataClass
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/init.py:7 in │
│ │
│ 4 # LICENSE file in the root directory of this source tree. │
│ 5 │
│ 6 from . import cuda_setup, utils │
│ ❱ 7 from .autograd._functions import ( │
│ 8 │ MatmulLtState, │
│ 9 │ bmm_cublas, │
│ 10 │ matmul, │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/autograd/init.py:1 in │
│ │
│ ❱ 1 from ._functions import undo_layout, get_inverse_transform_indices │
│ 2 │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/autograd/_functions.py:9 in │
│ │
│ 6 │
│ 7 import torch │
│ 8 │
│ ❱ 9 import bitsandbytes.functional as F │
│ 10 │
│ 11 │
│ 12 # math.prod not compatible with python < 3.8 │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/functional.py:17 in │
│ │
│ 14 from typing import Tuple │
│ 15 from torch import Tensor │
│ 16 │
│ ❱ 17 from .cextension import COMPILED_WITH_CUDA, lib │
│ 18 │
│ 19 │
│ 20 # math.prod not compatible with python < 3.8 │
│ │
│ /home/anaconda3/envs/toolllm/lib/python3.11/site-packages/bitsandbytes-0.38. │
│ 0.post2-py3.11.egg/bitsandbytes/cextension.py:20 in │
│ │
│ 17 │ if lib is None and torch.cuda.is_available(): │
│ 18 │ │ CUDASetup.get_instance().generate_instructions() │
│ 19 │ │ CUDASetup.get_instance().print_log_stack() │
│ ❱ 20 │ │ raise RuntimeError(''' │
│ 21 │ │ CUDA Setup failed despite GPU being available. Please run the f │
│ 22 │ │ │
│ 23 │ │ python -m bitsandbytes │
╰──────────────────────────────────────────────────────────────────────────────╯
RuntimeError:
CUDA Setup failed despite GPU being available. Please run the following
command to get more information:

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA

libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the
information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues
./scripts/inference_toolllama_helloworld_demo.sh:行3: --tool_root_dir: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行4: --backbone_model: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行5: --model_path: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行6: --max_observation_length: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行7: --observ_compress_method: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行8: --method: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行9: --input_query_file: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行10: --output_answer_file: 未找到命令
./scripts/inference_toolllama_helloworld_demo.sh:行11: --api_customization: 未找到命令

终端输入nvcc -V
微信图片_20240704153837
能看出来cuda是正确安装的,

找不到解决办法了55555555,特来问问大家,有知道怎么解决的嘛

@Cgf601
Copy link

Cgf601 commented Sep 5, 2024

你好 请问你解决了吗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants