xpu: set of not implemented aten ops affecting huggingface tests

[dvrogozh]

With the xpu support landed in huggingface (use huggingface/accelerate@3b5a00e and huggingface/transformers@eed9ed6 or later), there are few aten ops not implemented in xpu which affect common huggingface tests. Please, implement:

• tests/trainer/test_trainer.py:

  • aten::addcdiv.out, Add aten::addcdiv and its variants intel/torch-xpu-ops#486
  • aten::addcmul.out, Add aten::addcmul intel/torch-xpu-ops#418
  • aten::equal, Add aten::equal intel/torch-xpu-ops#550
  • aten::lerp.Scalar_out, Add aten::lerp_Tensor/lerp_Scalar and their variants intel/torch-xpu-ops#449
  • aten::linalg_vector_norm.out, Add aten::linalg_vector_norm intel/torch-xpu-ops#460
  • aten::mse_loss_backward, Add aten::mse_loss/mse_loss_backward and their variants intel/torch-xpu-ops#438
  • aten::mse_loss.out, Add aten::mse_loss/mse_loss_backward and their variants intel/torch-xpu-ops#438
  • aten::norm.out, Add aten::norm and its variants intel/torch-xpu-ops#556

• tests/test_modeling_utils.py:

  • aten::logical_or.out, Add aten::logical_and/or/xor and variant operators intel/torch-xpu-ops#529

• tests/pipelines/*:

  • aten::isin.Tensor_Tensor_out, Add aten::isin and its variant operators intel/torch-xpu-ops#548
  • aten::multinomial, Add aten::multinomial intel/torch-xpu-ops#520
  • aten::topk.values, Add aten::topk and its variants intel/torch-xpu-ops#547

CC: @gujinghui @EikanWang @fengyuan14 @guangyey @jgong5

cc @gujinghui @EikanWang @fengyuan14 @guangyey

[fengyuan14]

Except for equal and isin, others are included in PT2.5 support plan. We will evaluate the efforts to see whether to add them.

[dvrogozh]

First group of ops landed after #129353 got merged. See updated issue description.

[fengyuan14]

Except for equal and isin, others are included in PT2.5 support plan. We will evaluate the efforts to see whether to add them.

@dvrogozh These two operators has been added to PT2.5 support list.

[dvrogozh]

@fengyuan14 : fyi, not implemented ops which I reported before after running tests/pipelines/* also affect running models from HF model cards. For example, this happens running aten::isin.Tensor_Tensor_out, intel/torch-xpu-ops#548. I believe we already target them PT2.5, let's make sure we don't miss this. Ops are:

• aten::isin.Tensor_Tensor_out, Add aten::isin and its variant operators intel/torch-xpu-ops#548
• aten::multinomial, Add aten::multinomial intel/torch-xpu-ops#520
• aten::topk.values, Add aten::topk and its variants intel/torch-xpu-ops#547

Ran aten::isin.Tensor_Tensor_out, intel/torch-xpu-ops#548 using this script

import torch
import transformers
from transformers import AutoTokenizer
from transformers import pipeline

name = 'mosaicml/mpt-7b'

model = transformers.AutoModelForCausalLM.from_pretrained(
  name,
  torch_dtype=torch.bfloat16, # Load model weights in bfloat16
  trust_remote_code=True)

tokenizer = AutoTokenizer.from_pretrained('EleutherAI/gpt-neox-20b')

pipe = pipeline('text-generation', model=model, tokenizer=tokenizer, device='xpu:0')

with torch.autocast('xpu', dtype=torch.bfloat16):
    print(
        pipe('Here is a recipe for vegan banana bread:\n',
            max_new_tokens=100,
            do_sample=True,
            use_cache=True))

With:

• 3477ee3
• pytorch/vision@8f9d810
• intel/torch-xpu-ops@4db0b0c
• huggingface/accelerate@12a007d
• huggingface/transformers@3345ae7

[dvrogozh]

Few more ops are now available after #131015 merged. Remainder:

• aten::norm.out, Add aten::norm and its variants intel/torch-xpu-ops#556
• aten::topk.values, Add aten::topk and its variants intel/torch-xpu-ops#547

[dvrogozh]

All tasks done after #131643 merged.