Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

coreboot version bump from 4.22.01 to 24.02.01 (Q35/ivy/sandy/haswell/librems) #1715

Closed
wants to merge 11 commits into from

Conversation

tlaurion
Copy link
Collaborator

@tlaurion tlaurion commented Jul 10, 2024

Coreboot version bump to 24.02.01 for all platforms that depended on coreboot 4.20.01.

Testable roms available under green mark, select board, then go under artifacts and select zip file to internally upgrade. Do backup first from recovery shell and be ready to externally flash backup if bricked.

EDIT: 8d3c5cf inclues @JonathonHall-Purism patches (that can be seen under patches/coreboot-24.02.01 of this PR):

Tested:

  • qemu tpm1/tpm2: ok!
  • x230

Old (prior of 8d3c5cf)

Tested


All boards untested will be prepended UNTESTED, then all boards UNTESTED will go UNMAINTAINED.

grep -R 24.02.01 boards/ | awk -F "/" {'print $2'} | while read board; do sudo make BOARD=$board coreboot.save_in_oldconfig_format_in_place; done

Signed-off-by: Thierry Laurion <[email protected]>
@tlaurion tlaurion self-assigned this Jul 10, 2024
@tlaurion tlaurion marked this pull request as draft July 10, 2024 16:28
@tlaurion tlaurion changed the title coreboot version bump build attempt to 24.02.01 for ivy/sandy/haswell coreboot version bump build attempt to 24.02.01 for Q35/ivy/sandy/haswell Jul 10, 2024
…712"

This reverts commit c43b6fc, reversing
changes made to fb9c558.

Signed-off-by: Thierry Laurion <[email protected]>
@tlaurion
Copy link
Collaborator Author

tlaurion commented Jul 10, 2024

@tlaurion tlaurion marked this pull request as ready for review July 11, 2024 12:54
@tlaurion tlaurion changed the title coreboot version bump build attempt to 24.02.01 for Q35/ivy/sandy/haswell coreboot version bump from 4.20.01 to 24.02.01 (Q35/ivy/sandy/haswell/librems) Jul 11, 2024
@tlaurion tlaurion changed the title coreboot version bump from 4.20.01 to 24.02.01 (Q35/ivy/sandy/haswell/librems) coreboot version bump from 4.22.01 to 24.02.01 (Q35/ivy/sandy/haswell/librems) Jul 11, 2024
@tlaurion
Copy link
Collaborator Author

@aluciani
Copy link

Tested https://output.circle-artifacts.com/output/job/a3a29799-197a-43a6-ad76-ec3b0e1b238d/artifacts/0/build/x86/x230-maximized/heads-x230-maximized-v0.2.0-2215-g21ab781.zip working (not causing brick).

BOOTSPLASH IS GONE

I was going to report
The t430 also miss the bootsplash, but booting to debian 12 is working, didn t really did extensive test ...

@tlaurion tlaurion marked this pull request as draft July 11, 2024 13:15
@tlaurion
Copy link
Collaborator Author

tlaurion commented Jul 11, 2024

Putting back as draft: cbmem -c | grep -ie error -ie warning shows errors with memalign and out of memory errros when trying to map bootsplash.

Also shows PNP: missing read resources errors.

@tlaurion
Copy link
Collaborator Author

tlaurion commented Jul 11, 2024

Testing local builds pointing to v0.2.0 docker image which contains libnss3-dev new coreboot requirement from built docker image from 21ab781 which was #1713

This is done locally by calling
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=x230-maximized

(latest currently points to v0.1.9, not v0.2.0 to reflect master's currently used v0.1.9 docker image)

@tlaurion
Copy link
Collaborator Author

docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=qemu-coreboot-whiptail-tpm1

Replicates the no bootsplash issue. So can be tested with qemu, which will be more easy to report to coreboot and involve coreboot to fix this without real hardware.

@tlaurion
Copy link
Collaborator Author

docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=qemu-coreboot-whiptail-tpm1

Replicates the no bootsplash issue. So can be tested with qemu, which will be more easy to report to coreboot and involve coreboot to fix this without real hardware.

~ # cbmem -c | grep -ie error -ie warning
[ERROR]  memalign(boundary=8, size=1179648): failed: Tried to round up free_mem_ptr 0x7fece6c8 to 0x7ffee6c8
[ERROR]  but free_mem_end_ptr is 0x7ffcc7d0
[ERROR]  Error! memalign: Out of memory (free_mem_ptr >= free_mem_end_ptr)Bootsplash could not be decoded. jpeg_decode returned 1.

@JonathonHall-Purism I guess you get the same on librems?

@tlaurion
Copy link
Collaborator Author

tlaurion commented Jul 11, 2024

Attempting to build for coreboot 2405 fils with coreboot buildstack dependencies pointing to unavailable tarballs, as can be seen with this CircleCI build attempt at https://app.circleci.com/pipelines/github/tlaurion/heads/2642/workflows/92c0c23d-340a-4bd5-8cd7-9b5b479b00bf for testing branch https://github.com/tlaurion/heads/tree/xxx0_coreboot2405 testing commit tlaurion@481c173 with commit log stating error that will show in build logs there....

coreboot regressions all around it seems.

TLDR: coreboot 2405 depends on https://distfiles.macports.org/acpica/acpica-unix-20230628.tar.gz which doesn't exist.
See: https://app.circleci.com/pipelines/github/tlaurion/heads/2642/workflows/92c0c23d-340a-4bd5-8cd7-9b5b479b00bf/jobs/48310/parallel-runs/0/steps/0-102

@tlaurion
Copy link
Collaborator Author

Attempting to build for coreboot 2405 fils with coreboot buildstack dependencies pointing to unavailable tarballs, as can be seen with this CircleCI build attempt at https://app.circleci.com/pipelines/github/tlaurion/heads/2642/workflows/92c0c23d-340a-4bd5-8cd7-9b5b479b00bf for testing branch https://github.com/tlaurion/heads/tree/xxx0_coreboot2405 testing commit tlaurion@481c173 with commit log stating error that will show in build logs there....

coreboot regressions all around it seems.

TLDR: coreboot 2405 depends on https://distfiles.macports.org/acpica/acpica-unix-20230628.tar.gz which doesn't exist. See: https://app.circleci.com/pipelines/github/tlaurion/heads/2642/workflows/92c0c23d-340a-4bd5-8cd7-9b5b479b00bf/jobs/48310/parallel-runs/0/steps/0-102

@JonathonHall-Purism can you cache https://downloadmirror.intel.com/783534/acpica-unix-20230628.tar.gz under purism mirror please?

@tlaurion
Copy link
Collaborator Author

Attempting to build for coreboot 2405 fils with coreboot buildstack dependencies pointing to unavailable tarballs, as can be seen with this CircleCI build attempt at https://app.circleci.com/pipelines/github/tlaurion/heads/2642/workflows/92c0c23d-340a-4bd5-8cd7-9b5b479b00bf for testing branch https://github.com/tlaurion/heads/tree/xxx0_coreboot2405 testing commit tlaurion@481c173 with commit log stating error that will show in build logs there....
coreboot regressions all around it seems.
TLDR: coreboot 2405 depends on https://distfiles.macports.org/acpica/acpica-unix-20230628.tar.gz which doesn't exist. See: https://app.circleci.com/pipelines/github/tlaurion/heads/2642/workflows/92c0c23d-340a-4bd5-8cd7-9b5b479b00bf/jobs/48310/parallel-runs/0/steps/0-102

@JonathonHall-Purism can you cache https://downloadmirror.intel.com/783534/acpica-unix-20230628.tar.gz under purism mirror please?

@JonathonHall-Purism I just came to realize that acpica-unix is not cached under packages/x86/...

@tlaurion
Copy link
Collaborator Author

Attempting to build for coreboot 2405 fils with coreboot buildstack dependencies pointing to unavailable tarballs, as can be seen with this CircleCI build attempt at https://app.circleci.com/pipelines/github/tlaurion/heads/2642/workflows/92c0c23d-340a-4bd5-8cd7-9b5b479b00bf for testing branch https://github.com/tlaurion/heads/tree/xxx0_coreboot2405 testing commit tlaurion@481c173 with commit log stating error that will show in build logs there....
coreboot regressions all around it seems.
TLDR: coreboot 2405 depends on https://distfiles.macports.org/acpica/acpica-unix-20230628.tar.gz which doesn't exist. See: https://app.circleci.com/pipelines/github/tlaurion/heads/2642/workflows/92c0c23d-340a-4bd5-8cd7-9b5b479b00bf/jobs/48310/parallel-runs/0/steps/0-102

@JonathonHall-Purism can you cache https://downloadmirror.intel.com/783534/acpica-unix-20230628.tar.gz under purism mirror please?

@JonathonHall-Purism I just came to realize that acpica-unix is not cached under packages/x86/...

@JonathonHall-Purism that was our fault, we override the acpica-unix url defined under our mirroring using script which I hacked now as of tlaurion@71a6035

@fhvyhjriur
Copy link
Contributor

fhvyhjriur commented Jul 14, 2024

Would you like to send link to maximized T530, T420(dgpu - if you like to also test dgpu), and T440p(dgpu - if you like to also test dgpu) versions? I would take the risk on me to have to spend hours on each device externally reflashing it.

@fhvyhjriur : We know bootsplash is not working, since coreboot changed its memory manager for jpg processing. This will require to change all Heads bootsplashes to be smaller.

As for testing roms from this PR, the same applies as documented at https://osresearch.net/Downloading#downloading-heads-from-circleci

If something missing there (like same applies to PR) please provide a PR to change heads-wiki there following community guidelines for contributions.

@fhvyhjriur
Copy link
Contributor

I think i have find out the files for this PR. I expected same job-id in the link, but it seems like the job-id should differ from thinkpad model to other thinkpad model. When i am correct, this should be the image for T530: https://output.circle-artifacts.com/output/job/dc335a65-ce66-4455-ade6-298db6863f2a/artifacts/0/build/x86/UNMAINTAINED_t530-maximized/heads-UNMAINTAINED_t530-maximized-v0.2.0-2215-g21ab781.rom

Should i wait for the new release with the updated bootsplash jpg file and report if its working fine with the different screen resolutions? They differ from thinkpad to thinkpad and this would probably be a good test.

@tlaurion
Copy link
Collaborator Author

@JonathonHall-Purism
Copy link
Collaborator

@JonathonHall-Purism can you cache https://downloadmirror.intel.com/783534/acpica-unix-20230628.tar.gz under purism mirror please?

Mirror is updated with deps for 24.02.01 (this PR) and 24.05 (#1716). Thanks for the reminder.

…ailure on bootsplash and defaulting to 4mb of heap when bootsplash

Add upstreeam patches to downstream:
git fetch https://review.coreboot.org/coreboot refs/changes/75/83475/1 && git format-patch -1 --stdout FETCH_HEAD > 0001-prevent_bootsplash_heap_allocation_failure_to_booting.patch
git fetch https://review.coreboot.org/coreboot refs/changes/76/83476/2 && git format-patch -1 --stdout FETCH_HEAD > 0002-increase_heap_from1mb_to4mb_when_bootpslash_enabled.patch

Rebuilding coreboot version with patches applied:
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=qemu-coreboot-whiptail-tpm1 real.remove_canary_files-extract_patch_rebuild_what_changed
Remove file added by patch when patching fails:
sudo rm build/x86/libgpg-error-1.46/src/syscfg/lock-obj-pub.powerpc64le-unknown-linux-musl.h

Applying patches touching coreboot version 24.02.01 defiend in board config to all boards:
grep -R 24.02.01 boards/ | awk -F "/" {'print $2'} | while read board; do sudo make BOARD=$board coreboot.save_in_oldconfig_format_in_place; done

Adding added/modified files to git tracked files for commit:
git status | grep -e modified -e added | awk -F ":" {'print $2'}| xargs git add

Signed-off-by: Thierry Laurion <[email protected]>
…s from default vs master

Signed-off-by: Thierry Laurion <[email protected]>
…iptail-tpm1-hotp.config, remove unneeded diff from defconfig

TODO:
- all xx20/xx30/xx4x have wifi support, top down allocation disabled etc: create issue to track what is needed/unneeded

Signed-off-by: Thierry Laurion <[email protected]>
-CONFIG_BOOTSPLASH_CONVERT=y
-CONFIG_BOOTSPLASH_CONVERT_QUALITY=70

Signed-off-by: Thierry Laurion <[email protected]>
grep -R 24.02.01 boards/ | awk -F "/" {'print $2'} | while read board; do sudo make BOARD=$board coreboot.save_in_oldconfig_format_in_place; done

Signed-off-by: Thierry Laurion <[email protected]>
@tlaurion
Copy link
Collaborator Author

@JonathonHall-Purism I added your fixes, tested on qemu and works.

This invalidates #1720 which I will close.
#1716 : not from now and will be based on this PR, so closing as well.

@tlaurion
Copy link
Collaborator Author

tlaurion commented Jul 16, 2024

@JonathonHall-Purism posted PR to oreboot version bump to pr pointing to their coreboot fork with fixes included at #1721

@JonathonHall-Purism : another PR will need to change CircleCI board cache dependencies: all board will be able to share 24.02.01 caches once both are merged.

@tlaurion tlaurion marked this pull request as ready for review July 16, 2024 14:41
@tlaurion
Copy link
Collaborator Author

I think i have find out the files for this PR. I expected same job-id in the link, but it seems like the job-id should differ from thinkpad model to other thinkpad model. When i am correct, this should be the image for T530: https://output.circle-artifacts.com/output/job/dc335a65-ce66-4455-ade6-298db6863f2a/artifacts/0/build/x86/UNMAINTAINED_t530-maximized/heads-UNMAINTAINED_t530-maximized-v0.2.0-2215-g21ab781.rom

Should i wait for the new release with the updated bootsplash jpg file and report if its working fine with the different screen resolutions? They differ from thinkpad to thinkpad and this would probably be a good test.

@fhvyhjriur once 8d3c5cf CircleCI builds are successful, the roms produces for xx20/xx30/xx4x will need testing.

@JonathonHall-Purism Thanks for your precious contribution upstream!

@tlaurion
Copy link
Collaborator Author

@JonathonHall-Purism since PR title is also for Librems, I could merge your PR here so that another commit adds CircleCI boards dependencies to reduce time for ROM production if all boards sharing 24.02.01 could be built from same caches?

@tlaurion
Copy link
Collaborator Author

7454d58 is merging changes from #1700 (merged to master)

@tlaurion
Copy link
Collaborator Author

@JonathonHall-Purism testing gains of shared coreboot 24.02.01 buildstack at https://github.com/tlaurion/heads/tree/test_240201_shared_coreboot_buildstack building at https://app.circleci.com/pipelines/github/tlaurion/heads/2667/workflows/5086fa75-30a6-44ac-aa72-69bf140de9c7

If works/useful: will add that commit here.
Testing on x230 and if successful, call for testing for coreboot version bump.

@fhvyhjriur
Copy link
Contributor

fhvyhjriur commented Jul 17, 2024

@fhvyhjriur once 8d3c5cf CircleCI builds are successful, the roms produces for xx20/xx30/xx4x will need testing.

Build failed for some boards. The build for the boards i have did not fail. For example this looks to be the build for T530: https://output.circle-artifacts.com/output/job/cf336210-0ac2-4458-82a0-a4bd8db904fb/artifacts/0/build/x86/UNMAINTAINED_t530-maximized/heads-UNMAINTAINED_t530-maximized-v0.2.0-2225-g9e9ab76.rom
Should i test 8d3c5cf and ignore the failed build for some systems?

@tlaurion
Copy link
Collaborator Author

@JonathonHall-Purism testing gains of shared coreboot 24.02.01 buildstack at https://github.com/tlaurion/heads/tree/test_240201_shared_coreboot_buildstack building at https://app.circleci.com/pipelines/github/tlaurion/heads/2667/workflows/5086fa75-30a6-44ac-aa72-69bf140de9c7

If works/useful: will add that commit here. Testing on x230 and if successful, call for testing for coreboot version bump.

Fails since coreboot forks and coreboot releases tarballs cannot be mixed to cross use their buildstacks. This will be postponed.
Anyway: purism uses a newer kernel than xx20/xx30/xx4x/qemu, so community supported platforms (no funding, not dev time paid) needs to follow paid development fork to economize build time. In this case: purism uses a newer kerne then heads community supported boards, so gains of caching build artifacts would be lesser than ideal (cache needs to have prebuilt coreboot build stack and kernel artifacts to be fully efficient.

Will open issue and unrelated to this coreboot 24.02.01 being common. Will open issue to track and fix.

@tlaurion
Copy link
Collaborator Author

tlaurion commented Jul 17, 2024

@fhvyhjriur once 8d3c5cf CircleCI builds are successful, the roms produces for xx20/xx30/xx4x will need testing.

Build failed for some boards. The build for the boards i have did not fail. For example this looks to be the build for T530: https://output.circle-artifacts.com/output/job/cf336210-0ac2-4458-82a0-a4bd8db904fb/artifacts/0/build/x86/UNMAINTAINED_t530-maximized/heads-UNMAINTAINED_t530-maximized-v0.2.0-2225-g9e9ab76.rom Should i test 8d3c5cf and ignore the failed build for some systems?

@aluciani there is no gain outside of sharing coreboot buildstack (crossgcc) and linux built artifacts to speed up CircleCI as said in previous comment, and as you witnessed, ti doesn't work out of the box today under CircleCI building

@JonathonHall-Purism testing gains of shared coreboot 24.02.01 buildstack at https://github.com/tlaurion/heads/tree/test_240201_shared_coreboot_buildstack building at https://app.circleci.com/pipelines/github/tlaurion/heads/2667/workflows/5086fa75-30a6-44ac-aa72-69bf140de9c7

If works/useful: will add that commit here. Testing on x230 and if successful, call for testing for coreboot version bump.

When it comes to testing PRs, when you scroll down, you will see all tests and links to CirlceCI to download roms.
Note again that for internal flashing, it is preferred to download zip files then rom files, since the zip file contains a sha256sum of the rom file and would prevent USB thumb drive flash corruption from causing a brick, so please use zip file for internal flashing instead of flashing rom files directly.

Note that this PR final commit (unless bugs are found under specific platforms) is 7454d58 which is atop of master, and completed producing rom/zipfiles artifacts for all boards currently defined under CircleCI under master.

Note: 8d3c5cf was commit prior of 7454d58. CircleCI interrupted building the prior because I commited the latter and CircleCI interrupts builds for the same branch when newer commti exists. In PR, we can dismiss commits with red X if a green checkmark is provided by CircleCI: this means that the latest commit under a PR successfully produced roms/zip files for all boards contained under .circleci/config.yaml and that those roms are ready to test for willing board testers for coreboot/linux version bumps, which are the riskiest thing to test and requires an external programmer in case the roms produce bricks for tester to revert to master and report results in testing PR

@tlaurion tlaurion marked this pull request as draft July 17, 2024 12:33
@tlaurion
Copy link
Collaborator Author

I have put this PR as draft because i'm concerned in differences between defconfig changes between 4.22.01 and 24.02.01, and since doing a call for testing from all #692 was historically long, I would like this PR to contain proper coreboot changes to be tested by all community testers to fasten this merge as much as possible.

@tlaurion
Copy link
Collaborator Author

Something is corrupting heads repo from coreboot forks branch.

Traces of what I try to accomplish here.

Try to generate all coreboot defconfig to see deviation from coreboot specific version:
grep -R CONFIG_COREBOOT_VERSION boards/ | awk -F "/" {'print $2'} | while read board; do if ! sudo make BOARD=$board coreboot.save_in_defconfig_format_in_place > /dev/null 2>&1; then echo $board failed;fi; done

Then from failed boards, pick one board per shared coreboot version and build:

docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=nitropad-nv41
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=nitropad-nv41
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=qemu-coreboot-whiptail-tpm1
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=librem_l1um
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make -d BOARD=talos-2

Then regenerate coreboot configs in defconfig:
grep -R CONFIG_COREBOOT_VERSION boards/ | awk -F "/" {'print $2'} | while read board; do if ! sudo make BOARD=$board coreboot.save_in_defconfig_format_in_place > /dev/null 2>&1; then echo $board failed;fi; done


Unfortunately, heads repo gets replaced by Dasharo when doing some randomly. Will revert to backup and snapshot qube in between to figure out what is happening with make -d, something seems wrong under modules/coreboot: picking wrong git root and replacing Heads instead of coreboot local dir which should be under build/ARCH/coreboot-VER

@tlaurion
Copy link
Collaborator Author

Something is corrupting heads repo from coreboot forks branch.

Traces of what I try to accomplish here.

Try to generate all coreboot defconfig to see deviation from coreboot specific version:
grep -R CONFIG_COREBOOT_VERSION boards/ | awk -F "/" {'print $2'} | while read board; do if ! sudo make BOARD=$board coreboot.save_in_defconfig_format_in_place > /dev/null 2>&1; then echo $board failed;fi; done

Then from failed boards, pick one board per shared coreboot version and build:

docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=nitropad-nv41
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=nitropad-nv41
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=qemu-coreboot-whiptail-tpm1
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make BOARD=librem_l1um
docker run -e DISPLAY=$DISPLAY --network host --rm -ti -v $(pwd):$(pwd) -w $(pwd) tlaurion/heads-dev-env:v0.2.0 -- make -d BOARD=talos-2

Then regenerate coreboot configs in defconfig:
grep -R CONFIG_COREBOOT_VERSION boards/ | awk -F "/" {'print $2'} | while read board; do if ! sudo make BOARD=$board coreboot.save_in_defconfig_format_in_place > /dev/null 2>&1; then echo $board failed;fi; done


Unfortunately, heads repo gets replaced by Dasharo when doing some randomly. Will revert to backup and snapshot qube in between to figure out what is happening with make -d, something seems wrong under modules/coreboot: picking wrong git root and replacing Heads instead of coreboot local dir which should be under build/ARCH/coreboot-VER

This was me calling docker in parallel for multiple boards.

@tlaurion
Copy link
Collaborator Author

Deprecated in favor of #1723

@tlaurion tlaurion closed this Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants