Fix linuxboot builds #845

NHellFire · 2020-09-26T23:50:53Z

Tested builds of qemu-linuxboot and qemu-coreboot, both succeeded with gcc 7. With gcc 8+, linuxboot will fail on edk2 (linuxboot/linuxboot#43).

tlaurion · 2020-09-28T14:06:40Z

@NHellFire Ci fails because #842 not being merged. This will need to be rebased ton top of master once merged in.

MrChromebox

wait, why is coreboot being included as a module for a Linuxboot build anyway? Seems like that's the issue that needs to be fixed

NHellFire · 2020-09-29T05:38:07Z

wait, why is coreboot being included as a module for a Linuxboot build anyway? Seems like that's the issue that needs to be fixed

Top Makefile includes all modules (00559de, 1c64e4c). It was fine before the version check was added as it only contained variables.

tlaurion · 2020-09-29T17:29:56Z

@MrChromebox I like the fix. Rebasing on master with linixbootboarf added in CI should show the fix working.

tlaurion · 2020-09-29T17:30:31Z

@NHellFire can you add the board in CircleCI and rebase on master?

…linuxboot#841) Signed-off-by: Nathan Rennie-Waldock <[email protected]>

Signed-off-by: Nathan Rennie-Waldock <[email protected]>

NHellFire · 2020-09-29T21:27:25Z

@NHellFire can you add the board in CircleCI and rebase on master?

I've rebased (and fixed showing the error).
Currently, readding linuxboot back to CI will just fail as Debian 10 uses gcc 8.3 (linuxboot/edk2#1).

tlaurion · 2020-09-30T20:21:56Z

@NHellFire So if I understand well, this will work once linuxboot has your PR merged upstream, while linuxboot will not offer reproducibility since not bound to particular commit ID as other modules. Actually, it seems that linuxboot is the only module still not being bound to a commit id

EDIT: confirmed:

user@x230-master:~/heads$ grep "_version := git" modules/*
modules/coreboot:#coreboot_version := git
modules/flashrom:#flashrom_version := git
modules/linuxboot:linuxboot_version := git
modules/msrtools:#msrtools_version := git
modules/pciutils:#pciutils_version := git
modules/tpmtotp:#tpmtotp_version := git

Consequence, it will be impossible to reproduce Head's linuxboot based ROM based on a signle Head's commit ID in the future. Might need another issue and PR, or added to this PR, since linuxboot cannot be build for more then a month now (coreboot 4.8.1 version specification inside of coreboot module), and CircleCI's builds are not built since a while.

It seems that linuxboot builds are not often built for the following platforms:

Leopard OCP node: @osresearch others?
Winterfell OCP node: @osresearch others?
Intel S2600wf : @osresearch others?
r630: @osresearch others?

Signed-off-by: Nathan Rennie-Waldock <[email protected]>

NHellFire · 2020-10-01T03:00:56Z

CI should pass now. Local build passed with gcc 9.3.0 and binutils 2.34 (Ubuntu 20.04.1).

tlaurion · 2020-10-02T19:37:48Z

@NHellFire not sure about the hack implemented in the linuxboot module to apply patches outside of the root Makefile logic.
Why are those patch not directly applied into linuxboot upstream, or for a specific linuxboot git commit as other modules?

if we look at musl-cross-make module, being dependend on a specific commit for reproducible builds in the future on a specific Heads commit being rebuild, the following module specifies what to download as opposed to downloading master which cannot be replicated in the future if master changes.

As a result for that specific musl-cross-make git version, no patch are currently applied (no patch exists for that specific module-hash), since no patches relative to that specific commit id exits (Duh. there is a past artifact for musl-cross lying around, will fix that).

Does that make sense?

In the present case, linuxboot should be fixed to commit id linuxboot/linuxboot@10be0dc into module, and your linuxboot-ed2k patches should be applyable directly inside of the decompressed archive for that specific commit ID, where the patch would be in a file patches/linuxboot-10be0dc2652e1fc8c11d419ca2c4ff93920d4165.patch or in seperate files under patches/linuxboot-https://github.com/linuxboot-10be0dc2652e1fc8c11d419ca2c4ff93920d4165/1.patch etc following root Makefile patch application rule.

tlaurion · 2020-10-16T20:00:06Z

@NHellFire ?

NHellFire · 2020-10-17T19:49:34Z

I'll see if I can get to this in the next few days. I've been away from home and a bit busy.

…

On Fri, 16 Oct 2020, 21:00 tlaurion, ***@***.***> wrote: @NHellFire <https://github.com/NHellFire> ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#845 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIYA243XRIHP7XTQVGP633SLCQ5NANCNFSM4R3GFE3A> .

tlaurion · 2020-10-18T14:22:04Z

@NHellFire please tag me for followups

* add linuxboot-edk2 module (if not building latest git) * patch linuxboot-edk2 Makefile to not conflict with ours * update linuxboot build dir in circleci and boards/winterfell Signed-off-by: Nathan Rennie-Waldock <[email protected]>

…gcc 8 Signed-off-by: Nathan Rennie-Waldock <[email protected]>

NHellFire · 2020-10-20T19:39:59Z

@tlaurion Can you check the latest revision?
Thanks

tlaurion · 2020-10-20T21:47:11Z

boards/winterfell/winterfell.config

@@ -52,7 +52,7 @@ dxe_offset := 860000
 dxe_size := 6a0000
 flash-dxe: $(build)/$(BOARD)/linuxboot.rom
 	( echo u$(dxe_offset) $(dxe_size) ; \
-	pv $(build)/linuxboot-git/build/$(BOARD)/dxe.vol \
+	pv $(build)/linuxboot-b5376a441e8e85cbf722e943bb8294958e87c784/build/$(BOARD)/dxe.vol \


Ok, I get what you are doing here.
You can get the build directory by reusing Makefile included module.

In this case: $(linuxboot_base_dir)

tlaurion · 2020-10-20T21:47:44Z

.circleci/config.yml

+          command: |
+            make \
+                BOARD=qemu-linuxboot \
+                `/bin/pwd`/build/linuxboot-b5376a441e8e85cbf722e943bb8294958e87c784/build/qemu/.configured \


That will break in future linuxboot version bump. Why static?
$(linuxboot_base_dir)/build/qemu/.configured ?

tlaurion · 2020-10-20T21:48:05Z

modules/coreboot

@@ -1,5 +1,6 @@
 modules-$(CONFIG_COREBOOT) += coreboot

+ifeq "$(CONFIG_COREBOOT)" "y"
 ifeq "$(CONFIG_COREBOOT_VERSION)" "4.8.1"


Indentation should shift

tlaurion · 2020-10-20T21:49:09Z

modules/linuxboot

+
+linuxboot-edk2_tar := linuxboot-edk2-$(linuxboot-edk2_version).tar.gz
+linuxboot-edk2_url := https://github.com/linuxboot/edk2/archive/$(linuxboot-edk2_version).tar.gz
+linuxboot-edk2_dir := $(linuxboot_base_dir)/edk2


so that is the env variable to reuse $linuxboot-edk2_dir

tlaurion · 2020-11-04T19:25:50Z

@NHellFire I see that requests for change in this PR are still valid?

synackd · 2020-12-23T23:42:39Z

modules/linuxboot

@@ -21,7 +27,7 @@ linuxboot_configure := \
 	if [ "$(linuxboot_board)" = "qemu" ]; then \
 		echo >&2 "Pre-building edk2 OVMF" ; \
 		( cd $(build)/$(linuxboot_base_dir)/edk2/OvmfPkg ; \
-		./build.sh -n $$CPUS \
+		./build.sh -n $(CPUS) \


@tlaurion What do you say to making this change a separate PR and merging it in? I was about to make one addressing this before seeing this change in this PR.

@synackd please do. Same should be made under coreboot module

In hindsight, I probably could have combined #940 and #941 into one PR, though it does categorically separate the linuxboot and coreboot modules...

~~Have you tested it? The idea here is to pass CPUS to command line~~
~~make BOARD= x230 CPUS=2~~

~~There is no such invocation as CPUS as a command.~~
@synackd?
EDIT: nevermind! Tested.

These changes can be retracted since they are already merged as of #940.

tlaurion · 2021-01-17T17:46:57Z

@NHellFire @synackd ping into cleaning this PR?

tlaurion · 2021-01-17T17:50:19Z

@NHellFire not sure about the hack implemented in the linuxboot module to apply patches outside of the root Makefile logic.
Why are those patch not directly applied into linuxboot upstream, or for a specific linuxboot git commit as other modules?

@NHellFire patches are under patches/module-version and applied automatically, if modules respect module statements.
I cannot merge modules pinning to specific version. See requested changes above.

tlaurion · 2021-02-14T14:44:25Z

Linked with #971 (comment)

MrChromebox reviewed Sep 28, 2020

View reviewed changes

modules/coreboot: Only check version if we're building coreboot (fixes …

aaa1912

…linuxboot#841) Signed-off-by: Nathan Rennie-Waldock <[email protected]>

NHellFire force-pushed the fix-coreboot-version-check branch from 6bb5b59 to 7e2d39d Compare September 29, 2020 20:42

modules/coreboot: Unindent error, we're not in a recipe

c20c890

Signed-off-by: Nathan Rennie-Waldock <[email protected]>

NHellFire force-pushed the fix-coreboot-version-check branch from 7e2d39d to c20c890 Compare September 29, 2020 20:42

CircleCI: Readd linuxboot

bb3df89

Signed-off-by: Nathan Rennie-Waldock <[email protected]>

NHellFire changed the title ~~modules/coreboot: Only do coreboot version check if we're building boot (fixes #841)~~ Fix linuxboot builds Oct 1, 2020

NHellFire added 3 commits October 18, 2020 21:59

modules/linuxboot: Use a specific commit

5b7eb07

* add linuxboot-edk2 module (if not building latest git) * patch linuxboot-edk2 Makefile to not conflict with ours * update linuxboot build dir in circleci and boards/winterfell Signed-off-by: Nathan Rennie-Waldock <[email protected]>

patches/linuxboot-edk2: Import upstream patches to fix building with …

8a1eb75

…gcc 8 Signed-off-by: Nathan Rennie-Waldock <[email protected]>

NHellFire force-pushed the fix-coreboot-version-check branch from f48a5db to 8a1eb75 Compare October 20, 2020 16:56

tlaurion requested changes Oct 20, 2020

View reviewed changes

tlaurion mentioned this pull request Nov 2, 2020

Supermicro x11ssh coreboot configuration #601

Draft

synackd reviewed Dec 23, 2020

View reviewed changes

This was referenced Dec 24, 2020

qemu-linuxboot: x86_64-linux-musl-gcc cannot find Linux headers when building #939

Open

linuxboot: Pass $CPUS to edk2/OvmfPkg/build.sh unerroneously #940

Merged

tlaurion mentioned this pull request Feb 4, 2021

qemu-linuxboot: ASSERT error when running linuxboot.rom in QEMU #971

Open

tlaurion mentioned this pull request Feb 25, 2021

Linuxboot - Won't build on Ubuntu 20.04 #978

Open

synackd mentioned this pull request Oct 27, 2021

WIP: Pin Linuxboot to Commit and Separate EDK2 into a Separate Module #1047

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix linuxboot builds #845

Fix linuxboot builds #845

NHellFire commented Sep 26, 2020 •

edited

Loading

tlaurion commented Sep 28, 2020

MrChromebox left a comment

NHellFire commented Sep 29, 2020

tlaurion commented Sep 29, 2020

tlaurion commented Sep 29, 2020

NHellFire commented Sep 29, 2020

tlaurion commented Sep 30, 2020 •

edited

Loading

NHellFire commented Oct 1, 2020 •

edited

Loading

tlaurion commented Oct 2, 2020

tlaurion commented Oct 16, 2020

NHellFire commented Oct 17, 2020 via email

tlaurion commented Oct 18, 2020

NHellFire commented Oct 20, 2020

tlaurion Oct 20, 2020

tlaurion Oct 20, 2020

tlaurion Oct 20, 2020

tlaurion Oct 20, 2020

tlaurion commented Nov 4, 2020

synackd Dec 23, 2020 •

edited

Loading

tlaurion Dec 26, 2020

synackd Dec 26, 2020

tlaurion Dec 27, 2020 •

edited

Loading

synackd Jan 22, 2021

tlaurion commented Jan 17, 2021

tlaurion commented Jan 17, 2021

tlaurion commented Feb 14, 2021

Fix linuxboot builds #845

Are you sure you want to change the base?

Fix linuxboot builds #845

Conversation

NHellFire commented Sep 26, 2020 • edited Loading

tlaurion commented Sep 28, 2020

MrChromebox left a comment

Choose a reason for hiding this comment

NHellFire commented Sep 29, 2020

tlaurion commented Sep 29, 2020

tlaurion commented Sep 29, 2020

NHellFire commented Sep 29, 2020

tlaurion commented Sep 30, 2020 • edited Loading

NHellFire commented Oct 1, 2020 • edited Loading

tlaurion commented Oct 2, 2020

tlaurion commented Oct 16, 2020

NHellFire commented Oct 17, 2020 via email

tlaurion commented Oct 18, 2020

NHellFire commented Oct 20, 2020

tlaurion Oct 20, 2020

Choose a reason for hiding this comment

tlaurion Oct 20, 2020

Choose a reason for hiding this comment

tlaurion Oct 20, 2020

Choose a reason for hiding this comment

tlaurion Oct 20, 2020

Choose a reason for hiding this comment

tlaurion commented Nov 4, 2020

synackd Dec 23, 2020 • edited Loading

Choose a reason for hiding this comment

tlaurion Dec 26, 2020

Choose a reason for hiding this comment

synackd Dec 26, 2020

Choose a reason for hiding this comment

tlaurion Dec 27, 2020 • edited Loading

Choose a reason for hiding this comment

synackd Jan 22, 2021

Choose a reason for hiding this comment

tlaurion commented Jan 17, 2021

tlaurion commented Jan 17, 2021

tlaurion commented Feb 14, 2021

NHellFire commented Sep 26, 2020 •

edited

Loading

tlaurion commented Sep 30, 2020 •

edited

Loading

NHellFire commented Oct 1, 2020 •

edited

Loading

synackd Dec 23, 2020 •

edited

Loading

tlaurion Dec 27, 2020 •

edited

Loading