Skip to content

Commit

Permalink
Update CHARACTER_EMBEDDINGS.md
Browse files Browse the repository at this point in the history
  • Loading branch information
Alan Akbik authored Aug 26, 2019
1 parent 6879a70 commit f9f92ac
Showing 1 changed file with 2 additions and 5 deletions.
7 changes: 2 additions & 5 deletions resources/docs/embeddings/CHARACTER_EMBEDDINGS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,7 @@
are randomly initialized when you initialize the class, so they are not meaningful unless you train them on a specific
downstream task.

For instance, the standard sequence labeling architecture used by Lample et al. is a combination of classic word embeddings
with task-trained character features. Normally this would require you to implement a
[hierarchical embedding architecture](http://neuroner.com/NeuroNERengine_with_caption_no_figure.png) in which character-level
embeddings for each word are computed using an RNN and then concatenated with word embeddings.
For instance, the standard sequence labeling architecture used by [Lample et al. (2016)](https://www.aclweb.org/anthology/N16-1030) is a combination of classic word embeddings with task-trained character features. Normally this would require you to implement a [hierarchical embedding architecture](http://neuroner.com/NeuroNERengine_with_caption_no_figure.png) in which character-level embeddings for each word are computed using an RNN and then concatenated with word embeddings.

In Flair, we simplify this by treating `CharacterEmbeddings` just like any other embedding class. To reproduce the
Lample architecture, you need only combine them with standard `WordEmbeddings` in an embedding stack:
Expand All @@ -27,4 +24,4 @@ embedding = StackedEmbeddings(
```

If you pass this stacked embedding to a train method, the character-level features will now automatically be trained
for your downstream task.
for your downstream task.

0 comments on commit f9f92ac

Please sign in to comment.