Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIFO depth optimizer for Vitis backend #1037

Open
wants to merge 47 commits into
base: main
Choose a base branch
from

Conversation

steltze
Copy link

@steltze steltze commented Jul 18, 2024

FIFO depth optimizer for the Vitis backend using the built-in cosim FIFO profiling feature of Vitis HLS

  • Inserted the new optimizer to the Vitis backend flows
  • Implemented the optimizer. The steps are similar to the Vivado optimizer, only the profiling and FIFO depth parsing parts change
  • Updated the Vitis Writer to overwrite the project.tcl file with Vitis attributes
  • Updated the build_prj.tcl file to skip the add_vcd_instructions_tcl function in the case of Vitis backend
  • Added a pytest that checks if the depths decreased after the optimization and the cosim with the optimized depths passes, without detecting any deadlocks

Type of change

For a new feature or function, please create an issue first to discuss it
with us before submitting a pull request.

Note: Please delete options that are not relevant.

  • Documentation update
  • New feature (non-breaking change which adds functionality)

Tests

  • Used Vitis HLS 2023.2
  • Dummy dense model example. However, optimized depths are the same as the initial ones set by hls4ml
import hls4ml
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Sequential
model = Sequential()
model.add(Dense(64, input_shape=(16,), name='fc1', activation='relu'))
model.add(Dense(32, name='fc2', activation='relu'))
model.add(Dense(32, name='fc3', activation='relu'))
model.add(Dense(5, name='fc3', activation='softmax'))
model.add(Dense(5, name='fc4', activation='softmax'))

config = hls4ml.utils.config_from_keras_model(model, default_precision='ap_fixed<8, 4>')
config['Flows'] = ['vitis:fifo_depth_optimization']
hls4ml.model.optimizer.get_optimizer('vitis:fifo_depth_optimization').configure(profiling_fifo_depth=200_000)

output_dir = 'hls4mlprj_fifo_depth_opt_vitis'
hls_model = hls4ml.converters.convert_from_keras_model(model,
                                                       io_type='io_stream',
                                                       hls_config=config,
                                                       output_dir=output_dir,
                                                       part='xc7z020clg400-1',
                                                       backend='Vitis')

hls_model.build(reset=False, csim=False, synth=True, cosim=True)
hls4ml.report.read_vivado_report(output_dir)
  • Tested the model above with Vivado backend to ensure nothing broke
  • 10k param U-Net model. After running cosim with the optimized depths, no deadlocks are detected by Vitis

Test Configuration:

Checklist

  • I have read the guidelines for contributing.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings. There is a warning on dense_resource, " 'Resource pragma' is deprecated, use 'bind_op/bind_storage pragma' "
  • I have installed and run pre-commit on the files I edited or added.
  • I have added tests that prove my fix is effective or that my feature works. Fixed
    There is a small problem with the branched model and adds a linear layer in the input causing a very large FIFO between that layer and the depthwise convolution.

@steltze steltze changed the title Fifo depth opt vitis FIFO depth optimizer for Vitis backend Jul 18, 2024
@steltze
Copy link
Author

steltze commented Jul 25, 2024

hey @vloncar @jmitrevs do you think a unit test is required? To avoid delaying the CI pipeline, I could add a dummy model that verifies synthesis and that the flow is executed. Otherwise, I can add a complete unit test that also checks the FIFO depth results

@vloncar
Copy link
Contributor

vloncar commented Jul 25, 2024

I think it would be good to have a full test, but until we improve the synthesis testing pipeline, maybe mark it with @pytest.mark.skip(reason='Skipping synthesis tests for now'). This way we could run it locally when we review your code and it won't cause issues in the CI.

@steltze steltze marked this pull request as draft July 30, 2024 13:22
@steltze steltze marked this pull request as ready for review July 31, 2024 12:36
@steltze
Copy link
Author

steltze commented Aug 1, 2024

To the reviewers: when we agree on how we want to test this, I can open an issue to also include the Vivado/VivadoAccelerator FIFO optimization in the tests, as a future PR, as some changes in the initial implementation of those backends will be needed

@steltze
Copy link
Author

steltze commented Sep 23, 2024

After discussing, some more todos:

  • Update the hls4ml documentation by including an example of this feature with the Vitis backend
  • Include a UNet model on the pytest
  • There was a discussion about using Automated FIFO Sizing instead of Global FIFO Sizing source. @jmitrevs

@steltze steltze mentioned this pull request Nov 20, 2024
7 tasks
@steltze steltze force-pushed the fifo_depth_opt_vitis branch from 1240d23 to e7b4caa Compare November 27, 2024 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants