Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve DyNet usage #23

Open
LoicGrobol opened this issue Mar 27, 2020 · 3 comments
Open

Improve DyNet usage #23

LoicGrobol opened this issue Mar 27, 2020 · 3 comments
Assignees

Comments

@LoicGrobol
Copy link
Collaborator

LoicGrobol commented Mar 27, 2020

We can take better advantage of what DyNet offers

  • Activating autobatching (easy to do and seems to improve speed in at least my anecdotal benchmark :-)
  • Remove noop dy.parameter calls
@LoicGrobol LoicGrobol self-assigned this Mar 27, 2020
@mdelhoneux
Copy link
Member

Just saw this, regarding autobatching, I tried to get it to work a while ago (2 years?) and accuracy took a hit :/ talked to Yoav Goldberg about it who did not seem surprised

@LoicGrobol
Copy link
Collaborator Author

Ah, too bad. Did he have an idea why?

@mdelhoneux
Copy link
Member

Well batching is good for efficiency not accuracy so with a larger batch size, you need to train for more epochs if you want to reach the same result as with no batch at all. I didn't experiment so much so it might still be worth doing, especially if you want to train models on multiple treebanks, but you probably will need to train for more epochs and might not gain so much in training speed in the end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants