dbmdz/bert-base-german-cased

[maikshift]

Hello,

I have 2 short questions:

1. is it correct that the model 'distilbert-base-german-cased' (https://huggingface.co/distilbert-base-german-cased) was distilled from the model 'dbmdz/bert-base-german-cased' (https://huggingface.co/dbmdz/bert-base-german-cased)?

2. is there a paper on the 'dbmdz/bert-base-german-cased' and / or the 'distilbert-base-german-cased' (which can also be used for citation purposes)?

Thanks in advance!

[stefan-it]

Hi @maikshift ,

to answer your questions:

1.) yes, the distilbert-base-german-cased was distilled with the dbmdz/bert-base-german-cased model!
2.) unfortunately, there are no papers out there for these models. But there's a paper for (better and larger) German language models (GBERT and GELECTRA): https://aclanthology.org/2020.coling-main.598/

[maikshift]

Thank you very much! :)

[stefan-it]

Oh, and you can find a bit more information about the German DistilBERT in this PR:

huggingface/transformers#1873