partitioned elements number (10000) has exeeded the threshold (4096) - using HLS4ML_tutorial

[Stefanvanaardt]

Good day, all,
I am new to hls4ml, so please excuse my lack of knowledge. I have read almost any discussion and forum I can find on this problem and this is my last resort.
I have successfully tested very small autoencoder networks (synthesis and building in Vivado), however, I run into problems when implementing anything of a larger size. To see if the problem was on my side, I downloaded and ran the docker (with Vivado) as per the tutorial and cloned the stipulated repository by git clone https://github.com/fastmachinelearning/hls4ml-tutorial. The version of hls4ml used is 0.7.1.

I used the following test model KERAS_dense_16x100x100x100x100x100x5.json, and ran the following in Jupyter Notebook:

import hls4ml
config = hls4ml.utils.fetch_example_model('KERAS_dense_16x100x100x100x100x100x5.json')
print(config)
Downloading example model files ...
{'OutputDir': 'my-hls-test', 'ProjectName': 'myproject', 'Backend': 'Vivado', 'Part': 'xcku115-flvb2104-2-i', 'ClockPeriod': 5, 'IOType': 'io_parallel', 'HLSConfig': {'Model': {'Precision': 'ap_fixed<16,6>', 'ReuseFactor': 1}}, 'KerasJson': 'KERAS_dense_16x100x100x100x100x100x5.json', 'KerasH5': 'KERAS_dense_16x100x100x100x100x100x5_weights.h5'}
Interpreting Model
Topology:
Layer name: input_1, layer type: InputLayer, input shapes: [[None, 16]], output shape: [None, 16]
Layer name: fc1_relu, layer type: Dense, input shapes: [[None, 16]], output shape: [None, 100]
Layer name: fc2_relu, layer type: Dense, input shapes: [[None, 100]], output shape: [None, 100]
Layer name: fc3_relu, layer type: Dense, input shapes: [[None, 100]], output shape: [None, 100]
Layer name: fc4_relu, layer type: Dense, input shapes: [[None, 100]], output shape: [None, 100]
Layer name: fc5_relu, layer type: Dense, input shapes: [[None, 100]], output shape: [None, 100]
Layer name: output_softmax, layer type: Dense, input shapes: [[None, 100]], output shape: [None, 5]
Creating HLS model
hls_model = hls4ml.converters.keras_to_hls(config)
hls_model.compile()
Writing HLS project
Done    

I then modified the hls4ml_config.yml (and part number in project.tcl) file as per the below by modifying the strategy, the IOtype to both stream and parallel, and the reuse factor to 2000, additionally i changed the FPGA part number to one installed):

Backend: Vivado
ClockPeriod: 5
HLSConfig:
  Model:
    Precision: ap_fixed<16,6>
    ReuseFactor: 2000
    Strategy: Resource
IOType: io_stream
KerasH5: KERAS_dense_16x100x100x100x100x100x5_weights.h5
KerasJson: KERAS_dense_16x100x100x100x100x100x5.json
OutputDir: my-hls-test
Part: xcu250-figd2104-2L-e
ProjectName: myproject
Stamp: 85e47f44

as well as:

Backend: Vivado
ClockPeriod: 5
HLSConfig:
  Model:
    Precision: ap_fixed<16,6>
    ReuseFactor: 2000
    Strategy: Resource
IOType: io_parallel
KerasH5: KERAS_dense_16x100x100x100x100x100x5_weights.h5
KerasJson: KERAS_dense_16x100x100x100x100x100x5.json
OutputDir: my-hls-test
Part: xcu250-figd2104-2L-e
ProjectName: myproject
Stamp: 85e47f44

following this, building the project results in the following errors in all cases:
hls_model.build()

ERROR: [XFORM 203-103] Array 'mult.V' (firmware/nnet_utils/nnet_dense_latency.h:17): partitioned elements number (10000) has exeeded the threshold (4096), which may cause long run-time.
ERROR: [HLS 200-70] Pre-synthesis failed.
command 'ap_source' returned error code
    while executing
"source build_prj.tcl"
    ("uplevel" body line 1)
    invoked from within
"uplevel \#0 [list source $arg] "

INFO: [Common 17-206] Exiting vivado_hls at Mon Aug 28 02:45:59 2023...
CSynthesis report not found.
Vivado synthesis report not found.
Cosim report not found.
Timing report not found.
Found 1 solution(s) in my-hls-test/myproject_prj.
Reports for solution "solution1":

The same also happens when I use the tutorial examples and manually increase the network size.
Thank you for any assistance in advance.
Stefan

[bo3z]

The reason for this error is that the number of elements in a (sub)array stored in registers exceeds 4,096. This typically happens in Latency strategy, when all the weights are stored in registers.

Looking at the logs, there is an error in dense_latency even though the strategy is meant to be Resource.

The reason for this is the way the strategy is set. In hls4ml, the config must be modified before writing and compiling the model, as the config is used to write all the C++ header files. To verify this, have a look at firmware/parameters.h in your output directory - for all of your layers, the strategy will be set to Latency.

To fix this, update the config dictionary before calling convert, with the following line: config['Model']['Strategy'] = 'Resource'

(As far as I am aware) The hls4ml_config.yaml is only stored for completness sake after the model is written - all the necessary properties were extracted from the config before the model was written and compiled, and stored in the relevant C++ header files and .tcl scripts.

[Stefanvanaardt]

Thank you kindly for the feedback. I understand the problem better. I note the correct method of modifying the config correctly by the mentioned above. I was unaware that the compiling had to be done again and that the .yaml file is for completeness as opposed to modifying for the compiling/building.

I have subsequently made the following changes:

config['Model']['Strategy']= 'Resource'
config['Model']['ReuseFactor']= 3000
hls_model.compile()
writing HLS project
Done
print(config)

{'OutputDir': 'my-hls-test', 'ProjectName': 'myproject', 'Backend': 'Vivado', 'Part': 'xcku115-flvb2104-2-i', 'ClockPeriod': 5, 'IOType': 'io_parallel', 'HLSConfig': {'Model': {'Precision': 'ap_fixed<16,6>', 'ReuseFactor': 1}}, 'KerasJson': 'KERAS_dense_16x100x100x100x100x100x5.json', 'KerasH5': 'KERAS_dense_16x100x100x100x100x100x5_weights.h5', 'Stamp': '85e47f44', 'Model': {'Strategy': 'Resource', 'ReuseFactor': 3000}}
which resulted in the same error. I have now adjusted the config to be aligned with the hls4ml_config.yaml file HLSConfig -> Model -> Strategy:

config['HLSConfig']['Model']['Strategy']= 'Resource'
config['HLSConfig']['Model']['ReuseFactor']= 3000
hls_model.compile()
writing HLS project
Done

print(config)
{'OutputDir': 'my-hls-test', 'ProjectName': 'myproject', 'Backend': 'Vivado', 'Part': 'xcku115-flvb2104-2-i', 'ClockPeriod': 5, 'IOType': 'io_parallel', 'HLSConfig': {'Model': {'Precision': 'ap_fixed<16,6>', 'ReuseFactor': 3000, 'Strategy': 'Resource'}}, 'KerasJson': 'KERAS_dense_16x100x100x100x100x100x5.json', 'KerasH5': 'KERAS_dense_16x100x100x100x100x100x5_weights.h5', 'Stamp': 'Ce7746eF', 'Model': {'Strategy': 'Resource', 'ReuseFactor': 3000}}hls_model.build()

I do take note that the latest parameters.h file does indeed still have the strategy set as Latency and the reuse factor as 1, not the adjusted values.

I am not sure as to why the intended changes are not being implemented by the above attempts. In my own created test network, I modified the config['Model']['Strategy']= 'Resource' has indeed resulted in the parameters.h file correctly displaying the strategy as nnet::resource in addition to the correct ReuseFactor.

At least my own network has managed to build further than possible before which is great.

Again, I greatly appreciate your time and any feedback on this.

[bo3z]

Make the config changes before calling hls4ml.converters.keras_to_hls(config). The converters write (generate) the C++ files; the compile() function runs C++ compilation with GCC on the generated files.

As a side note - In your example, I think the correct way is the second (aligned with hls4ml_config.yml). But, this depends how the config was generated - from scratch or from some example.

[Stefanvanaardt]

I have yet to correctly adjust the strategy for the predefined model correctly with the above, but it now works perfectly for multiple custom models, so I am overjoyed. I will try again over the weekend and revert back on the KERAS_dense_16x100x100x100x100x100x5 model.