Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

All targets/RISC-V/spike32.py tests failing #1098

Closed
TommyMurphyTM1234 opened this issue Jul 2, 2024 · 6 comments
Closed

All targets/RISC-V/spike32.py tests failing #1098

TommyMurphyTM1234 opened this issue Jul 2, 2024 · 6 comments

Comments

@TommyMurphyTM1234
Copy link
Collaborator

I am using the following:

and when I run the Spike32 tests:

./gdbserver.py ./targets/RISC-V/spike32.py --isolate --print-failures --remotelogfile-enable --server_cmd "openocd -d3" 2>&1 | tee test-spike32.log

all tests fail:

/gdbserver.py ./targets/RISC-V/spike32.py --isolate --print-failures --remotelogfile-enable --server_cmd "openocd -d3" 2>&1 | tee test-spike32.log
Using $misa from hart definition: 0x4034112d
PRNG seed for spike32 is generated automatically
PRNG seed for spike32 is 1719915165
[CeaseStepiTest] Starting > logs/20240702-111245-spike32-CeaseStepiTest.log
[CeaseStepiTest] not_applicable in 0.00s
[CheckMisa] Starting > logs/20240702-111245-spike32-CheckMisa.log
[CheckMisa] exception in 3.57s
Test: CheckMisa
Target: spike32
---------------------------------[ Compile ]----------------------------------
+ riscv64-unknown-elf-gcc -g programs/checksum.c programs/tiny-malloc.c programs/infinite_loop.S -DDEFINE_MALLOC -DDEFINE_FREE -DCLINT=33554432 programs/entry.S programs/init.c -DNHARTS=1 -I ../env -T targets/RISC-V/spike32.lds nostartfiles -mcmodel=medany -DXLEN=32 -o /tmp/spike32_checksum-4034112d_h3qffwsl -march=rv32imafdcv_zicsr -mabi=ilp32
-------------------------[ /tmp/spike-bti84wl5.log ]--------------------------
+ spike -p1 --isa RV32IMAFDCV_Zvl128b_Zve64d --dm-auth --dmi-rti 4 --dm-no-abstract-fpr --dm-no-halt-groups -m0x10100000:0x10000000 --rbb-port 0 /tmp/spike32_checksum-4034112d_h3qffwsl
error: bad --isa option 'RV32IMAFDCV_Zvl128b_Zve64d'. unsupported extension: zvl128b

--------------------------------[ Traceback ]---------------------------------
Traceback (most recent call last):
  File "/home/user/Downloads/openocd-test/riscv-tests/debug/testlib.py", line 1379, in run
    self.classSetup()
  File "/home/user/Downloads/openocd-test/riscv-tests/debug/testlib.py", line 1435, in classSetup
    BaseTest.classSetup(self)
  File "/home/user/Downloads/openocd-test/riscv-tests/debug/testlib.py", line 1342, in classSetup
    self.target_process = self.target.create()
  File "/home/user/Downloads/openocd-test/riscv-tests/debug/./targets/RISC-V/spike32.py", line 24, in create
    return testlib.Spike(self, isa="RV32IMAFDCV", dmi_rti=4,
  File "/home/user/Downloads/openocd-test/riscv-tests/debug/testlib.py", line 120, in __init__
    raise TestLibError("Didn't get spike message about bitbang "
testlib.TestLibError: Didn't get spike message about bitbang connection
-------------------------------[ End of logs ]--------------------------------
Result: exception
Logfile: logs/20240702-111245-spike32-CheckMisa.log
Reproduce: ./gdbserver.py ./targets/RISC-V/spike32.py CheckMisa
Time elapsed: 3.57s
[CustomRegisterTest] Starting > logs/20240702-111249-spike32-CustomRegisterTest.log
[CustomRegisterTest] exception in 3.59s
Test: CustomRegisterTest
Target: spike32
---------------------------------[ Compile ]----------------------------------
+ riscv64-unknown-elf-gcc -g programs/checksum.c programs/tiny-malloc.c programs/infinite_loop.S -DDEFINE_MALLOC -DDEFINE_FREE -DCLINT=33554432 programs/entry.S programs/init.c -DNHARTS=1 -I ../env -T targets/RISC-V/spike32.lds nostartfiles -mcmodel=medany -DXLEN=32 -o /tmp/spike32_checksum-4034112d_dgmrsm6l -march=rv32imafdcv_zicsr -mabi=ilp32
-------------------------[ /tmp/spike-lhs71q6e.log ]--------------------------
+ spike -p1 --isa RV32IMAFDCV_Zvl128b_Zve64d --dm-auth --dmi-rti 4 --dm-no-abstract-fpr --dm-no-halt-groups -m0x10100000:0x10000000 --rbb-port 0 /tmp/spike32_checksum-4034112d_dgmrsm6l
error: bad --isa option 'RV32IMAFDCV_Zvl128b_Zve64d'. unsupported extension: zvl128b

--------------------------------[ Traceback ]---------------------------------
Traceback (most recent call last):
  File "/home/user/Downloads/openocd-test/riscv-tests/debug/testlib.py", line 1379, in run
    self.classSetup()
  File "/home/user/Downloads/openocd-test/riscv-tests/debug/testlib.py", line 1435, in classSetup
    BaseTest.classSetup(self)
  File "/home/user/Downloads/openocd-test/riscv-tests/debug/testlib.py", line 1342, in classSetup
    self.target_process = self.target.create()
  File "/home/user/Downloads/openocd-test/riscv-tests/debug/./targets/RISC-V/spike32.py", line 24, in create
    return testlib.Spike(self, isa="RV32IMAFDCV", dmi_rti=4,
  File "/home/user/Downloads/openocd-test/riscv-tests/debug/testlib.py", line 120, in __init__
    raise TestLibError("Didn't get spike message about bitbang "
testlib.TestLibError: Didn't get spike message about bitbang connection
-------------------------------[ End of logs ]--------------------------------
Result: exception
Logfile: logs/20240702-111249-spike32-CustomRegisterTest.log
Reproduce: ./gdbserver.py ./targets/RISC-V/spike32.py CustomRegisterTest
Time elapsed: 3.59s

...

:::::::::::::::::::::::::::[ ran 73 tests in 222s ]:::::::::::::::::::::::::::
12 tests returned not_applicable
61 tests returned exception
   CheckMisa > logs/20240702-111245-spike32-CheckMisa.log
   CustomRegisterTest > logs/20240702-111249-spike32-CustomRegisterTest.log
   DebugBreakpoint > logs/20240702-111253-spike32-DebugBreakpoint.log
   DebugChangeString > logs/20240702-111256-spike32-DebugChangeString.log
   DebugCompareSections > logs/20240702-111300-spike32-DebugCompareSections.log
   DebugExit > logs/20240702-111304-spike32-DebugExit.log
   DebugFunctionCall > logs/20240702-111307-spike32-DebugFunctionCall.log
   DebugSymbols > logs/20240702-111311-spike32-DebugSymbols.log
   DebugTurbostep > logs/20240702-111314-spike32-DebugTurbostep.log
   DisconnectTest > logs/20240702-111318-spike32-DisconnectTest.log
   DownloadTest > logs/20240702-111321-spike32-DownloadTest.log
   EbreakTest > logs/20240702-111325-spike32-EbreakTest.log
   EtriggerTest > logs/20240702-111329-spike32-EtriggerTest.log
   FreeRtosTest > logs/20240702-111332-spike32-FreeRtosTest.log
   Hwbp1 > logs/20240702-111336-spike32-Hwbp1.log
   Hwbp2 > logs/20240702-111340-spike32-Hwbp2.log
   HwbpManual > logs/20240702-111343-spike32-HwbpManual.log
   IcountTest > logs/20240702-111347-spike32-IcountTest.log
   InfoTest > logs/20240702-111351-spike32-InfoTest.log
   InstantChangePc > logs/20240702-111354-spike32-InstantChangePc.log
   InstantHaltTest > logs/20240702-111358-spike32-InstantHaltTest.log
   InterruptTest > logs/20240702-111402-spike32-InterruptTest.log
   ItriggerTest > logs/20240702-111405-spike32-ItriggerTest.log
   JumpHbreak > logs/20240702-111409-spike32-JumpHbreak.log
   MemTest16 > logs/20240702-111413-spike32-MemTest16.log
   MemTest32 > logs/20240702-111416-spike32-MemTest32.log
   MemTest64 > logs/20240702-111420-spike32-MemTest64.log
   MemTest8 > logs/20240702-111423-spike32-MemTest8.log
   MemTestBlock0 > logs/20240702-111427-spike32-MemTestBlock0.log
   MemTestBlock1 > logs/20240702-111431-spike32-MemTestBlock1.log
   MemTestBlock2 > logs/20240702-111434-spike32-MemTestBlock2.log
   MemTestReadInvalid > logs/20240702-111438-spike32-MemTestReadInvalid.log
   PrivChange > logs/20240702-111442-spike32-PrivChange.log
   PrivRw > logs/20240702-111445-spike32-PrivRw.log
   ProgramHwWatchpoint > logs/20240702-111449-spike32-ProgramHwWatchpoint.log
   ProgramSwWatchpoint > logs/20240702-111453-spike32-ProgramSwWatchpoint.log
   Registers > logs/20240702-111456-spike32-Registers.log
   RepeatReadTest > logs/20240702-111500-spike32-RepeatReadTest.log
   Semihosting > logs/20240702-111503-spike32-Semihosting.log
   SemihostingFileio > logs/20240702-111507-spike32-SemihostingFileio.log
   SimpleF18Test > logs/20240702-111511-spike32-SimpleF18Test.log
   SimpleNoExistTest > logs/20240702-111514-spike32-SimpleNoExistTest.log
   SimpleS0Test > logs/20240702-111518-spike32-SimpleS0Test.log
   SimpleS1Test > logs/20240702-111522-spike32-SimpleS1Test.log
   SimpleT0Test > logs/20240702-111525-spike32-SimpleT0Test.log
   SimpleT1Test > logs/20240702-111529-spike32-SimpleT1Test.log
   SimpleV13Test > logs/20240702-111532-spike32-SimpleV13Test.log
   StepTest > logs/20240702-111536-spike32-StepTest.log
   Sv32Test > logs/20240702-111540-spike32-Sv32Test.log
   TooManyHwbp > logs/20240702-111544-spike32-TooManyHwbp.log
   TriggerDmode > logs/20240702-111547-spike32-TriggerDmode.log
   TriggerExecuteInstant > logs/20240702-111551-spike32-TriggerExecuteInstant.log
   TriggerLoadAddressInstant > logs/20240702-111554-spike32-TriggerLoadAddressInstant.log
   TriggerStoreAddressInstant > logs/20240702-111558-spike32-TriggerStoreAddressInstant.log
   UnavailableCycleTest > logs/20240702-111602-spike32-UnavailableCycleTest.log
   UnavailableHaltedTest > logs/20240702-111605-spike32-UnavailableHaltedTest.log
   UnavailableRunTest > logs/20240702-111609-spike32-UnavailableRunTest.log
   UserInterrupt > logs/20240702-111613-spike32-UserInterrupt.log
   VectorTest > logs/20240702-111616-spike32-VectorTest.log
   WriteCsrs > logs/20240702-111620-spike32-WriteCsrs.log
   WriteGprs > logs/20240702-111624-spike32-WriteGprs.log

The problem seems to be this:

+ spike -p1 --isa RV32IMAFDCV_Zvl128b_Zve64d --dm-auth --dmi-rti 4 --dm-no-abstract-fpr --dm-no-halt-groups -m0x10100000:0x10000000 --rbb-port 0 /tmp/spike32_checksum-4034112d_h3qffwsl
error: bad --isa option 'RV32IMAFDCV_Zvl128b_Zve64d'. unsupported extension: zvl128b

But...

  1. I don't know what triggered this?
  2. I don't know why the spike32.py uses a different ISA specification (including V) to other targets and why there isn't more consistency here?
  3. I don't know why the CI runs don't seem to be affected (yet?)?
This was referenced Jul 2, 2024
Open
Merged
@aap-sc
Copy link
Collaborator

aap-sc commented Jul 2, 2024 

I don't know what triggered this?

This is like related to changes in upstream spike. If you look at the current changelog there is lot's of changes related to how RVV configuration is handled.

I don't know why the spike32.py uses a different ISA specification (including V) to other targets and why there isn't more consistency here?

I think this we should discuss this in riscv-software-src/riscv-tests#568

I don't know why the [CI runs](https://github.com/riscv-collab/riscv-openocd/actions) don't seem to be affected (yet?)?

The current TOT in spike is 1b1a33376 . Which correspond to the latest pipeline we have in OpenOCD. It seems that you spike version is deprecated.

@aap-sc
Copy link
Collaborator

aap-sc commented Jul 2, 2024

@TommyMurphyTM1234

This is like related to changes in upstream spike. If you look at the current changelog there is lot's of changes related to how RVV configuration is handled.

In addition to my point above you can also see here: riscv-software-src/riscv-tests@e06a435 . So yes, this is changes in upstream spike

@TommyMurphyTM1234
Copy link
Collaborator Author

Thanks @aap-sc.

The current TOT in spike is 1b1a33376 . Which correspond to the latest pipeline we have in OpenOCD. It seems that you spike version is deprecated.

I just used the version that the riscv-gnu-toolchain repo uses as a submodule. So maybe that needs to be bumped over there?

@aap-sc
Copy link
Collaborator

aap-sc commented Jul 2, 2024 

I just used the version that the riscv-gnu-toolchain repo uses as a submodule. So maybe that needs to be bumped over there?

Have no opinion on the matter since we don't use riscv-gnu-toolchain :(

@TommyMurphyTM1234
Copy link
Collaborator Author

As mentioned above, riscv-gnu-toolchain currently uses riscv-software-src/riscv-isa-sim@20a2b6d for its Spike submodule. This is about three months old and predates the changes related to V extension handling.

I did a local build of Spike at the current top of tree commit - riscv-software-src/riscv-isa-sim@f7d0dba - and with this version I no longer have the aforementioned spike32.py test failures.

So the issue is indeed that riscv-gnu-toolchain uses a Spike that is not up to date with the requirements of the OpenOCD tests.

I will see if I can safely bump riscv-gnu-tolchain's Spike submodule to use the latest Spike commit. I need to run the tests suites for the bare-metal/Newlib and Linux toolchains just to check that there is no adverse impact from doing this.

@TommyMurphyTM1234 TommyMurphyTM1234 mentioned this issue Jul 12, 2024
Closed
@TommyMurphyTM1234
Copy link
Collaborator Author

Addressed by this PR:

which is also discussed in this issue:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@TommyMurphyTM1234 @aap-sc