Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xlnet doesn't suport use_biaffine #34

Open
yonghuiwuf opened this issue Mar 31, 2023 · 1 comment
Open

Xlnet doesn't suport use_biaffine #34

yonghuiwuf opened this issue Mar 31, 2023 · 1 comment

Comments

@yonghuiwuf
Copy link
Collaborator

python src/run_transformer_ner.py
--model_type xlnet
--pretrained_model xlnet-base-cased
--data_dir ./test_data/conll-2003
--new_model_dir ./new_bert_ner_model
--overwrite_model_dir
--predict_output_file ./bert_pred.txt
--max_seq_length 256
--save_model_core
--do_train
--do_predict
--model_selection_scoring strict-f_score-1
--do_lower_case
--train_batch_size 8
--eval_batch_size 8
--train_steps 500
--learning_rate 1e-5
--num_train_epochs 1
--gradient_accumulation_steps 1
--do_warmup
--seed 13
--warmup_ratio 0.1
--max_num_checkpoints 3
--log_file ./log.txt
--progress_bar
--early_stop 3

Traceback (most recent call last):
File "/data/datasets/yonghui/project/ClinicalTransformerNER/src/run_transformer_ner.py", line 169, in main
run_task(global_args)
File "/data/datasets/yonghui/project/ClinicalTransformerNER/src/transformer_ner/task.py", line 604, in run_task
model = model_model.from_pretrained(args.pretrained_model, config=config)
File "/home/yonghui.wu/.pyenv/versions/anaconda3-2021.11/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2024, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
File "/data/datasets/yonghui/project/ClinicalTransformerNER/src/transformer_ner/model.py", line 308, in init
if config.use_biaffine:
File "/home/yonghui.wu/.pyenv/versions/anaconda3-2021.11/lib/python3.9/site-packages/transformers/configuration_utils.py", line 253, in getattribute
return super().getattribute(key)
AttributeError: 'XLNetConfig' object has no attribute 'use_biaffine'

@bugface
Copy link
Contributor

bugface commented Mar 31, 2023

we need to revisit XLNet, the implementation has problems where the performances are very low consistently cross different datasets. So the new features are not integrated with XLNet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants