Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dynamic MVAU integration #1242

Draft
wants to merge 205 commits into
base: dev
Choose a base branch
from
Draft

Dynamic MVAU integration #1242

wants to merge 205 commits into from

Conversation

azizb-xlnx
Copy link
Collaborator

@azizb-xlnx azizb-xlnx commented Nov 29, 2024 •
edited
Loading

To minimize changes, we overload the weight input of the existing MVAU superclass to serve as the second input for a dynamic MVAU. The specialized layer transform will instantiate the appropriate RTL subclass based on the presence of a weight initializer.

auphelia and others added 30 commits June 10, 2024 16:19
@auphelia
[Transform] Skip broadcasting of thresholds in onnx conversion and ex…
4c7288d 
…tend conversion to hw layers
@auphelia
Merge branch 'dev' into custom/bert
85f98fd
@auphelia
Merge branch 'dev' into custom/bert
1906498
@auphelia
Merge branch 'feature/per_tensor_rtl_thresh' into custom/bert
7f296bd
@auphelia
[MVAU] Initial draft of pumped compute for RTL MVU
933b4eb
@auphelia
[Transform] Add commands to ip stitching for second clk
89e5c82
@auphelia
[Deps] Change pyverilator commit to enable second clock simulation
c54a8b4
@auphelia
[Util] Add v80 to versal list
0ec5e58
@auphelia
[MVAU] Comment out assert for uram mem mode
1a24b36
@auphelia
[RTL MVAU] Allow for 4bit compute with dsp48 for versal devices
0f7706f
@auphelia
Merge branch 'dev' into feature/double_pumping_mem
550499e
@auphelia
[CustomOp] Enable double pumped memstream for MVAU
69de66c
@maltanar
[Deps] add pyxsi fork and its deps
42cbf47
@maltanar
[Util] add util to fetch Vivado path
b6dba8a
@maltanar
[Sim] add a first draft of pyxsi-based simulation, untested
1468dbe
@maltanar
[Sim] try enabling pyxsi logging and tracing
6907167
@maltanar
[Op] experiment with enabling rtlsim with pyxsi
5a5461c
@maltanar
[Deps] update pyxsi
a50cdb6
@maltanar
[Sim] update pyxsi API to include verilog spec, assume precompiled
77680dd
@maltanar
[Deps] ensure pyxsi is compiled + set envvars for its use
1804c47
@maltanar
[Sim] remove LD_LIBRARY_PATH setting from pyxsi util, doesn't work
64e9fc1
@maltanar
[Deps] update pyxsi
d480e49
@maltanar
[AddStreams] set rtlsim_cycles for pyxsi prototyping
3560933
@maltanar
[Infra] only trigger pyxsi compile if Vivado detected
212927c
@maltanar
[Sim] move pyxsiutils to own repo, update deps, fix entrypoint chk
4fd5610
@auphelia
Merge dev into feature/double_pumping_mem
edb6991
@maltanar
[pyxsi] trialling RPC server as LD_LIBRARY_PATH workaround
6e4d085
@maltanar
[Infra] remove global LD_LIBRARY_PATH setting from entrypoint
4558314
@maltanar
[pyxsi] add license text to rpcserver&client
aff109f
@maltanar
[Infra] start pyxsi rpc server in entrypoint
39ac22e
aziz bahri and others added 30 commits December 12, 2024 13:38
clean up test
b14474e
dynMM Test: remove trace
76e2ab7
@auphelia
[DuplicateStreams] Relax constraints on datatype for hw conversion fu…
6de05a0 
…nction
dynMM Test: split up the tests
8ec1e58
@auphelia
[DuplicateStreams] Remove leftover check for integer input
706952a
@auphelia
Merge pull request #1252 from Xilinx/custom/transformer_fp_ds
69255a7 
Floating point support for DuplicateStreams component
@auphelia
[Tests] Add streamlining test to move scalar mul past dynamic matmul
073a5de
white space
7a8880d 
Signed-off-by: aziz bahri <[email protected]>
@auphelia
[Thresholding] Fix threshold data type minimization for float inputs
4a671f4
@auphelia
[qonnx2finn] Remove broadcasting of thresholds for relu/selu multithr…
1b03f6b 
…esholds
dynMM: fix class hiearchy
bcb80a9
Merge remote-tracking branch 'origin/custom/transformer' into feature…
01c4628 
…/mvau_dyn
@auphelia
Merge branch 'dev' into custom/transformer
cdf705c
@STFleming
Add unique names as part of the specialize_layers step
684d16e
dynMM: set rtlsim backend
0816e62
dynMM: stub weight minimize transforms
a4fac05
Merge remote-tracking branch 'origin/custom/transformer' into feature…
4cffe7e 
…/mvau_dyn
DynMM: use inner dim as MW and MH
251d595 
Signed-off-by: aziz bahri <[email protected]>
dynMM test: test with 4 dim inputs
6ed4cc1 
Signed-off-by: aziz bahri <[email protected]>
dynMM: set folded input shape for Matb correctly
f9a3f83
dynMM: Linting
ed79c0a 
Signed-off-by: aziz bahri <[email protected]>
dynMM: Linting
5cdbbec 
Signed-off-by: aziz bahri <[email protected]>
dynMM: add fifodepth attribute
4d06f2a 
Signed-off-by: aziz bahri <[email protected]>
dynMM: Remove comment
69b9298 
Signed-off-by: aziz bahri <[email protected]>
dynMM Test: remove redundant test
6868f6c 
Signed-off-by: aziz bahri <[email protected]>
dynMM: optimise execute node loop
055d554 
Signed-off-by: aziz bahri <[email protected]>
dynMM: Remove N_VECTORS attribute
db69a6f
dynMM: Clean up test
6508b4b
dynMM: rename to dynMVU_rtl
af849a1
@STFleming
Minor fixes for DynMatMul mem_mode and inFIFODepths for getting throu…
6479daa 
…gh endtoend bert flow for EoY goal
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@auphelia auphelia Awaiting requested review from auphelia

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@azizb-xlnx @auphelia @maltanar @STFleming @alexhornburg-xlnx @preusser