Initial values for the hidden/cell state for LSTM and GRU models in Pytorch #1120
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR addresses #1074 and implements the passing of initial values for the hidden and cell states in GRU and LSTM models, which is supported in pytorch. This first version implements this only for the pytorch parser, but it should be able to be extended it for keras and other parsers.
I have tested this for Vivado, Vitis, and Quartus. I don't have access to Catapult or oneAPI at the moment, so I haven't implemented this fort those backends.
Note that this currently only works in
io_parallel
. Inio_stream
I was having some conceptual issues and was unsure if I should treat these initial states are streamed inputs or not. Might be good enough for now and I can revisitio_stream
if there are any suggestions how to tackle that.Type of change
Tests
Tested in both standalone scripts and also the pytests to ensure that model parsing and evaluation work with and without passing these optional tensors.
Checklist
pre-commit
on the files I edited or added.