Jan doesnt acknowledge my Nvidia Card - I've installed CUDA but still got nothing.... HELP!

[imtuyethan]

https://discord.com/channels/1107178041848909847/1279468088311414880

[imtuyethan]

You may consider the settings of "n_gl" or "n_gpu_layers" on right panel:

What does this mean?

"n_gl" or "n_gpu_layers" is a setting that controls how many layers of the AI model are loaded into the GPU memory. It's basically a way to balance between speed and memory usage:

• Higher n_gpu_layers = more GPU memory used, faster processing
• Lower n_gpu_layers = less GPU memory used, potentially slower processing

Suggestions

• For 7B models: Try starting with 32 layers (n_gpu_layers=32)
• For 13B models: Start with around 20-24 layers
• For larger models: You may need to go even lower, perhaps 16 or fewer layers