Error importing third party model: dunzhang/stella_en_400M_v5

[ivssh]

I am trying to import the aforesaid model using the following command:

eland_import_hub_model  --url <url>  --hub-model-id dunzhang/stella_en_400M_v5 \
--task-type text_embedding       --es-username elastic --es-password <password> --es-model-id stella-en-400m-v5 --insecure

I am getting the below error, I suspect it has something to do with the version of transformers package being used. Do I need to upgrade?

Traceback (most recent call last):
  File "/home/abhilash/.local/bin/./eland_import_hub_model", line 8, in <module>
    sys.exit(main())
  File "/home/abhilash/.local/lib/python3.10/site-packages/eland/cli/eland_import_hub_model.py", line 279, in main
    tm = TransformerModel(
  File "/home/abhilash/.local/lib/python3.10/site-packages/eland/ml/pytorch/transformers.py", line 665, in __init__
    self._traceable_model = self._create_traceable_model()
  File "/home/abhilash/.local/lib/python3.10/site-packages/eland/ml/pytorch/transformers.py", line 1000, in _create_traceable_model
    model = _SentenceTransformerWrapperModule.from_pretrained(
  File "/home/abhilash/.local/lib/python3.10/site-packages/eland/ml/pytorch/transformers.py", line 313, in from_pretrained
    model = AutoModel.from_pretrained(model_id, token=token, torchscript=True)
  File "/home/abhilash/.local/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 561, in from_pretrained
    return model_class.from_pretrained(
  File "/home/abhilash/.local/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3236, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "/home/abhilash/.cache/huggingface/modules/transformers_modules/dunzhang/stella_en_400M_v5/1bb50bc7bb726810eac2140e62155b88b0df198f/modeling.py", line 823, in __init__
    self.encoder = NewEncoder(config)
  File "/home/abhilash/.cache/huggingface/modules/transformers_modules/dunzhang/stella_en_400M_v5/1bb50bc7bb726810eac2140e62155b88b0df198f/modeling.py", line 696, in __init__
    self.layer = nn.ModuleList([NewLayer(config) for _ in range(config.num_hidden_layers)])
  File "/home/abhilash/.cache/huggingface/modules/transformers_modules/dunzhang/stella_en_400M_v5/1bb50bc7bb726810eac2140e62155b88b0df198f/modeling.py", line 696, in <listcomp>
    self.layer = nn.ModuleList([NewLayer(config) for _ in range(config.num_hidden_layers)])
  File "/home/abhilash/.cache/huggingface/modules/transformers_modules/dunzhang/stella_en_400M_v5/1bb50bc7bb726810eac2140e62155b88b0df198f/modeling.py", line 627, in __init__
    attn_implementation = config._attn_implementation
  File "/home/abhilash/.local/lib/python3.10/site-packages/transformers/configuration_utils.py", line 262, in __getattribute__
    return super().__getattribute__(key)
AttributeError: 'NewConfig' object has no attribute '_attn_implementation'

[allamiro]

It seems there is incompatibility between the version of the transformers package and the model you're trying to import (dunzhang/stella_en_400M_v5).

Specifically, it seems that the model's configuration is expecting an attribute (_attn_implementation) that your current version of the transformers library doesn't support.

Sometimes custom models may have unique configurations that aren't fully supported by certain versions of transformers may be attempt to upgrade or update and see if that fixes the problem.

[ivssh]

I think newer versions of the transformers package aren't compatible with eland, Eland needs to bump up the versions so that the above may work.