Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The network boot test suite shall pass under QEMU #544

Closed
Tracked by #537
pietrushnic opened this issue Oct 23, 2023 · 11 comments
Closed
Tracked by #537

The network boot test suite shall pass under QEMU #544

pietrushnic opened this issue Oct 23, 2023 · 11 comments
Labels
qemu_q35 QEMU Q35

Comments

@pietrushnic
Copy link

No description provided.

@pietrushnic pietrushnic mentioned this issue Oct 23, 2023
Closed
2 tasks
@pietrushnic
Copy link
Author

First, run with Dasharo/edk2#101

(venv) [23:13:08] pietrushnic:open-source-firmware-validation git:(qemu_q35_v0.1.0-rc1*) $ robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no dasharo-compatibility/network-boot.robot
==============================================================================
Network-Boot                                                                  
==============================================================================
PXE001.001 Dasharo Network Boot is available :: This test aims to ... | SKIP |
PXE001.001 not supported
------------------------------------------------------------------------------
PXE002.001 Dasharo network boot menu boot options order is correct... | SKIP |
PXE002.001 not supported
------------------------------------------------------------------------------
PXE003.001 Autoboot option is available and works correctly :: Thi... | SKIP |
PXE003.001 not supported
------------------------------------------------------------------------------
PXE004.001 DTS option is available and works correctly :: This tes... | SKIP |
PXE004.001 not supported
------------------------------------------------------------------------------
PXE005.001 OS installation option is available and works correctly... | SKIP |
PXE005.001 not supported
------------------------------------------------------------------------------
PXE006.001 iPXE shell option is available and works correctly :: T... | SKIP |
PXE006.001 not supported
------------------------------------------------------------------------------
Network-Boot                                                          | SKIP |
6 tests, 0 passed, 0 failed, 6 skipped
==============================================================================
Output:  /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/output.xml
Log:     /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/log.html
Report:  /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/report.html

@pietrushnic
Copy link
Author

Before this task can be completed, tests for enabling/disabling network boot have to work and the above suite should somehow depend on enabling network boot otherwise, those tests cannot pass.

@pietrushnic
Copy link
Author 

(venv) [0:14:04] pietrushnic:open-source-firmware-validation git:(qemu_q35_v0.1.0-rc1*) $ robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no dasharo-security/network-stack.robot
==============================================================================
Network-Stack                                                                 
==============================================================================
NBA001.001 Enable Network Boot (firmware) :: This test aims to ver... | FAIL |
Variable '${end}' not found.
------------------------------------------------------------------------------
NBA002.001 Disable Network Boot (firmware) :: This test aims to ve... | PASS |
------------------------------------------------------------------------------
Network-Stack                                                         | FAIL |
2 tests, 1 passed, 1 failed
==============================================================================
Output:  /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/output.xml
Log:     /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/log.html
Report:  /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/report.html

@pietrushnic
Copy link
Author

pietrushnic commented Oct 23, 2023 •
edited
Loading

Using:

I'm able to pass dasharo-security/network-stack:

(venv) [0:55:01] pietrushnic:open-source-firmware-validation git:(qemu_q35_v0.1.0-rc1) $ robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no dasharo-security/network-stack.robot
==============================================================================
Network-Stack                                                                 
==============================================================================
NBA001.001 Enable Network Boot (firmware) :: This test aims to ver... | PASS |
------------------------------------------------------------------------------
NBA002.001 Disable Network Boot (firmware) :: This test aims to ve... | PASS |
------------------------------------------------------------------------------
Network-Stack                                                         | PASS |
2 tests, 2 passed, 0 failed
==============================================================================
Output:  /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/output.xml
Log:     /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/log.html
Report:  /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/report.html

@pietrushnic pietrushnic added the qemu_q35 QEMU Q35 label Oct 24, 2023
@pietrushnic
Copy link
Author

With recent version of code I'm able to achieve:

(venv) [17:27:01] pietrushnic:open-source-firmware-validation git:(qemu_q35_v0.1.0-rc1*) $ robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no dasharo-compatibility/network-boot.robot
==============================================================================
Network-Boot
==============================================================================
PXE001.001 Dasharo Network Boot is available :: This test aims to ... | PASS |
------------------------------------------------------------------------------
PXE002.001 Dasharo network boot menu boot options order is correct... | FAIL |
'Autoboot (DHCP) (3)' does not contain 'Dasharo Network Boot Menu'
------------------------------------------------------------------------------
PXE003.001 Autoboot option is available and works correctly :: Thi... | PASS |
------------------------------------------------------------------------------
PXE004.001 DTS option is available and works correctly :: This tes... | FAIL |
No match found for 'Enter an option' in 5 minutes. Output:






                      /---------------------------------\
                      |   Please select boot device:    |
                      |---------------------------------|
                      |  Setup                          |
                      |  PXEv4 (MAC:525400123456)       |
                      |  UEFI Shell                     |
                      |> iPXE Network Boot              |
                      |---------------------------------|
                      |    ^ and v to move selection    |
                      |   ENTER to select boot device   |
                      |           ESC to exit           |
                      \---------------------------------/
------------------------------------------------------------------------------
PXE005.001 OS installation option is available and works correctly... | PASS |
------------------------------------------------------------------------------
PXE006.001 iPXE shell option is available and works correctly :: T... | PASS |
------------------------------------------------------------------------------
Network-Boot                                                          | FAIL |
6 tests, 4 passed, 2 failed
==============================================================================
Output:  /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/output.xml
Log:     /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/log.html
Report:  /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/report.html

Although there may be some questionable design choices (e.g. network enabling is in first test in suite).

@pietrushnic
Copy link
Author

Most recent version from above PRs after running test:

robot -t "NBA001.001*" -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no dasharo-security/network-stack.robot

Give pass on all test in network boot suite:

(venv) [22:33:24] pietrushnic:open-source-firmware-validation git:(qemu_q35_v0.1.0-rc1*) $ robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no dasharo-compatibility/network-boot.robot
==============================================================================
Network-Boot
==============================================================================
PXE001.001 Dasharo Network Boot is available :: This test aims to ... | PASS |
------------------------------------------------------------------------------
PXE002.001 Dasharo network boot menu boot options order is correct... | PASS |
------------------------------------------------------------------------------
PXE003.001 Autoboot option is available and works correctly :: Thi... | PASS |
------------------------------------------------------------------------------
PXE004.001 DTS option is available and works correctly :: This tes... | PASS |
------------------------------------------------------------------------------
PXE005.001 OS installation option is available and works correctly... | PASS |
------------------------------------------------------------------------------
PXE006.001 iPXE shell option is available and works correctly :: T... | PASS |
------------------------------------------------------------------------------
Network-Boot                                                          | PASS |
6 tests, 6 passed, 0 failed
==============================================================================
Output:  /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/output.xml
Log:     /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/log.html
Report:  /home/pietrushnic/src/3mdeb/dasharo/open-source-firmware-validation/report.html

@macpijan
Copy link
Contributor

I tried again and indeed it worked for me on the: Dasharo/open-source-firmware-validation@91ee40b revision.

@macpijan
Copy link
Contributor

The changes in test environment has been merged. See: Dasharo/open-source-firmware-validation#70

@pietrushnic
Copy link
Author

@macpijan thanks.

@pietrushnic
Copy link
Author

pietrushnic commented Oct 25, 2023 •
edited
Loading

@macpijan what do we need to close this issue?

@macpijan
Copy link
Contributor

macpijan commented Nov 8, 2023

This suite has passed for me many times already, so can close that one.

@macpijan macpijan closed this as completed Nov 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
qemu_q35 QEMU Q35
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pietrushnic @macpijan