scripts: No config rebuild

[IstvanZsSzekely]

• Build the project first using make
• Run the simulation only with make RUN=sim after the build is created

This is intended to be used mainly for repeated simulations, where the project or simulation files are finalized and we want to run the tests repeatedly one after another.

[gastmaier]

Even better than a RUN=sim flag is to use what we are implementing analogdevicesinc/hdl#1202 here.
Would change the library target to an array of component.xml paths, so the Makefile can properly manage the lib dependencies, re-running cfg build only when necessary.

However, resolving my review comments, I'm OK with merging this as is for now.

[gastmaier]

Why touch system_project.log, and not only:

ifneq ($(RUN), sim)
	-rm -rf $(addprefix runs/,$(1))
	(...)
endif

[IstvanZsSzekely]

I haven't thought about that approach to be honest, and yes, it's better.

[gastmaier]

$(ENV_DEPS) has no effect, since the library target is not managed (no timestamp), always triggering a rebuild.

[IstvanZsSzekely]

I haven't dug deep into what is going on in the make dependencies beyond the base requirements that were set in the Makefile. When the PR was created, I found that the the library is always "remade" and it showed the make script as it was changed, forcing a rebuild. I haven't tried to find a fix for the library scripting so the configs are not rebuilt in the testbenches.

[IstvanZsSzekely]

Seeing that there is an ongoing PR in the HDL repository regarding the parallel build, and potentially fixing the library targets as you mentioned, I'll put this PR on hold for the time being. Once it gets merged with the main branch, I'll do the tests with the main Testbenches branch and check if this PR is even needed at that point.

[IstvanZsSzekely]

Removed the option for RUN=sim
Simulation always runs
Configuration is rebuilt only if configuration or library related files change

[gastmaier]

Doesn't work in parallel make -j4, I will add to my list of TODOs to examine more carefully.
Ignore this comment, I improperly tested it.

[gastmaier]

Test condition:
Rebased to main

git rebase origin/main

Clean-all then build, with 5 processes.
Builds libraries in parallel then runs a single tb.

pulsar_adc_pmdz$ (cd ../../library/ ; make clean-all) ; make CFG=cfg1 TST=test_sleep_delay -j5

Builds libraries in parallel then runs multiple tbs in parallel.

jesd_loopback$ make  -j5

Build without cleaning.
Runs project without rebuilding cfg

pulsar_adc_pmdz$ make CFG=cfg1 TST=test_sleep_delay -j5

Touch library then build.
Rebuilds touched libraries, rebuilds cfg then runs tb.

pulsar_adc_pmdz$ touch cd ../../library/spi_engine/axi_spi_engine/axi_spi_engine.v ; \
touch cd ../../library/spi_engine/spi_engine_execution/spi_engine_execution.v ; \
make CFG=cfg1 TST=test_sleep_delay -j5

Results: The libraries are properly managed and built in parallel,
however, different tbs sharing the same cfg cannot be run in parallel as is since they share the same disk files, which vivados read/writes/locks, causing the tbs to crash.
I believe the only affected project it the pulsar, due to #53, that could be reverted.

Another PR should improve the Running test_program test print since it seems to be missing linebreaks, e.g.,

Building cfg2 env [/home/me/Documents/adi/hdl/testbenches/jesd_loopback/runs/cfg2/system_project.log] ...Building cfg2_np12 env [/home/me/Documents/adi/hdl/testbenches/jesd_loopback/runs/cfg2_np12/system_project.log] ...Building cfg1 env [/home/me/Documents/adi/hdl/testbenches/jesd_loopback/runs/cfg1/system_project.log] ...Building cfg4_F8 env [/home/me/Documents/adi/hdl/testbenches/jesd_loopback/runs/cfg4_F8/system_project.log] ...Building cfg3_np12_L2M8 env [/home/me/Documents/adi/hdl/testbenches/jesd_loopback/runs/cfg3_np12_L2M8/system_project.log] ... OK
 OK
 OK
 OK
 OK

and resolve the issue described before.

Unrelated: the jesd_loopback cfg6_F64 is failing with error:

[ERROR] 89535 ns   Address : 0000000044a90314; Data mismatch. Read data is : 00000000; expected is 1f3f8000

[IstvanZsSzekely]

Results: The libraries are properly managed and built in parallel, however, different tbs sharing the same cfg cannot be run in parallel as is since they share the same disk files, which vivados read/writes/locks, causing the tbs to crash. I believe the only affected project it the pulsar, due to #53, that could be reverted.

I did not try to run any test program in parallel, but I think that this is acceptable for now, up until we end up having a lot more test programs in one project that share configuration files and require a significant amount of time to complete. This will more than likely change the structure of the code and the way the project and results are generated.

Another PR should improve the Running test_program test print since it seems to be missing linebreaks, e.g.,

Building cfg2 env [/home/me/Documents/adi/hdl/testbenches/jesd_loopback/runs/cfg2/system_project.log] ...Building cfg2_np12 env [/home/me/Documents/adi/hdl/testbenches/jesd_loopback/runs/cfg2_np12/system_project.log] ...Building cfg1 env [/home/me/Documents/adi/hdl/testbenches/jesd_loopback/runs/cfg1/system_project.log] ...Building cfg4_F8 env [/home/me/Documents/adi/hdl/testbenches/jesd_loopback/runs/cfg4_F8/system_project.log] ...Building cfg3_np12_L2M8 env [/home/me/Documents/adi/hdl/testbenches/jesd_loopback/runs/cfg3_np12_L2M8/system_project.log] ... OK
 OK
 OK
 OK
 OK

and resolve the issue described before.

I didn't experience this issue. Is it maybe because of parallel jobs? If so, as stated before, this can be left alone for now, but this PR will be noted when a change is required.

Unrelated: the jesd_loopback cfg6_F64 is failing with error:

[ERROR] 89535 ns   Address : 0000000044a90314; Data mismatch. Read data is : 00000000; expected is 1f3f8000

This is a known issue, someone who knows the JESD infrastructure will have to take a look at it and try to figure out what changed since the last known working version.