Failing convert_llama_hf_to_nemo.py

[yonas-g]

Failing convert_llama_hf_to_nemo.py

Running convert_llama_hf_to_nemo.py fails with an error TypeError: expected string or bytes-like object when trying to execute python /home/ubuntu/NeMo/scripts/checkpoint_converters/convert_llama_hf_to_nemo.py --input_name_or_path meta-llama/Llama-3.2-1B-Instruct --output_path Llama-3.2-1B.nemo

Steps/Code to reproduce bug

When running python /home/ubuntu/NeMo/scripts/checkpoint_converters/convert_llama_hf_to_nemo.py --input_name_or_path meta-llama/Llama-3.2-1B-Instruct --output_path Llama-3.2-1B.nemo I get the following error.

Traceback (most recent call last):
  File "/home/ubuntu/NeMo/scripts/checkpoint_converters/convert_llama_hf_to_nemo.py", line 344, in <module>
    convert(args)
  File "/home/ubuntu/NeMo/scripts/checkpoint_converters/convert_llama_hf_to_nemo.py", line 310, in convert
    model = load_state_dict_helper(MegatronGPTModel, nemo_config, trainer, checkpoint['state_dict'])
  File "/home/ubuntu/miniconda3/envs/nemo/lib/python3.10/site-packages/nemo/collections/nlp/parts/utils_funcs.py", line 220, in load_state_dict_helper
    model = cls(cfg, trainer)
  File "/home/ubuntu/miniconda3/envs/nemo/lib/python3.10/site-packages/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py", line 331, in __init__
    super().__init__(cfg, trainer=trainer, no_lm_init=True)
  File "/home/ubuntu/miniconda3/envs/nemo/lib/python3.10/site-packages/nemo/collections/nlp/models/language_modeling/megatron_base_model.py", line 153, in __init__
    self._enable_nvidia_optimizations()
  File "/home/ubuntu/miniconda3/envs/nemo/lib/python3.10/site-packages/nemo/collections/nlp/models/language_modeling/megatron_base_model.py", line 393, in _enable_nvidia_optimizations
    if not is_official_release_version(nvidia_torch_version):
  File "/home/ubuntu/miniconda3/envs/nemo/lib/python3.10/site-packages/nemo/collections/nlp/models/language_modeling/megatron_base_model.py", line 390, in is_official_release_version
    return re.fullmatch("[0-9][0-9]\.[0-9][0-9].*", nvidia_torch_version)  # "YY.MM.*"
  File "/home/ubuntu/miniconda3/envs/nemo/lib/python3.10/re.py", line 195, in fullmatch
    return _compile(pattern, flags).fullmatch(string)
TypeError: expected string or bytes-like object

**Environment overview **

• Environment location: Cloud(Lambdalabs)
• Method of NeMo install: from srouce using python -m pip install git+https://github.com/NVIDIA/[email protected]#egg=nemo_toolkit[all]

Environment details

• OS version: Debian
• PyTorch version: 2.4.1+cu124
• Python version: Python 3.10

Additional context
GPU model: A100