Improve Batched GEMV Speed #113
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Shaun Ren <[email protected]>
In addition, run GEMV multiple times to improve the accuracy of the result. Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
nengo_ocl/raggedarray.py
Outdated
| def __init__(self, arrays, names=None, dtype=None, align=False): | ||
| arrays = [np.asarray(a) for a in arrays] | ||
| def __init__(self, arrays, names=None, dtype=None, align=False, order='C'): | ||
| assert order in 'CF' |
There was a problem hiding this comment.
I think assert order in ('C', 'F') would be better, because 'CF' would also be accepted with what you have above, right?
There was a problem hiding this comment.
You're right, I overlooked this bug. I'll change it in another commit.
|
Very cool! I'm looking forward to trying this out when I get back. What still needs to be done before this is ready for full review/merge? |
Signed-off-by: Shaun Ren <[email protected]>
|
There are still some kernels that need to be modified to work with column-major; and an autotuner should be added. |
Signed-off-by: Shaun Ren <[email protected]>
|
The autotuner is a bigger job, and I think it should be a separate PR. Getting the other kernels work with column-major sounds like the main work needed for this PR. |
This commit implements the more accurate LIF model, implemented in Nengo 2.1.1, in OpenCL. A new boolean argument `fastlif' is also added in plan_lif, which defaults to False. See <nengo/nengo#975> for details regarding the new LIF model. Signed-off-by: Shaun Ren <[email protected]>
shaunren
force-pushed
the
fastgemv
branch
2 times, most recently
from
3295671
to
41e9e9d
Compare
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
Signed-off-by: Shaun Ren <[email protected]>
|
This patchset is now ready for review/merge. |
Open
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using column-major instead of row-major matrices is found to improve the speed of GEMV in nearly all cases, with improvements in some practical scenarios, such as Spaun 2.0.
Also, the LIF kernel is updated to use the more accurate LIF model in Nengo 2.1.1. This is found to have negligible performance penalties.
The current, non autotuned colmun-major kernel is able to achieve a total of 31 s of Sim time speedup on Spaun 2.0 (202s -> 171s) on a GTX 970.