Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression: Some peripherals are not discovered since 4.22.01 -> 24.02.01 Coreboot Version bump (upstream: bogus top down allocator) #1740

Closed
13 of 50 tasks
Oessel opened this issue Jul 31, 2024 · 16 comments · Fixed by #1743

Comments

@Oessel
Copy link

Oessel commented Jul 31, 2024

Please identify some basic details to help process the report

A. Provide Hardware Details

  1. What board are you using? (Choose from the list of boards here)
    Lenovo Thinkpad T430

  2. Does your computer have a dGPU or is it iGPU-only?

    • dGPU (Distinct GPU other then internal GPU)
    • iGPU-only (Internal GPU, normally Intel GPU)
  3. Who installed Heads on this computer?

  4. What PGP key is being used?

    • Librem Key (Nitrokey Pro 2 rebranded)
    • Nitrokey Pro
    • Nitrokey Pro 2
    • Nitrokey 3 NFC
    • Nitrokey 3 NFC Mini
    • Nitrokey Storage
    • Nitrokey Storage 2
    • Yubikey
    • Other
  5. Are you using the PGP key to provide HOTP verification?

    • Yes
    • No
    • I don't know

B. Identify how the board was flashed

  1. Is this problem related to updating heads or flashing it for the first time?

    • First-time flash
    • Updating heads
  2. If the problem is related to an update, how did you attempt to apply the update?

    • Using the Heads menus
    • Flashrom via the Recovery Shell
    • External flashing
  3. How was Heads initially flashed?

    • External flashing
    • Internal-only / 1vyprep+1vyrain / skulls
    • Don't know
  4. Was the board flashed with a maximized or non-maximized/legacy rom?

    • Maximized
    • Non-maximized / legacy
    • I don't know
  5. If Heads was externally flashed, was IFD unlocked?

    • Yes
    • No
    • Don't know

C. Identify the rom related to this bug report

V0.2.0-2254-g27d09d4

  1. Did you download or build the rom at issue in this bug report?

    • I downloaded it
    • I built it
  2. If you downloaded your rom, where did you get it from?

    • Heads CircleCi
    • Purism
    • Nitrokey
    • Dasharo DTS (Novacustom)
    • Somewhere else (please identify)

    Please provide the release number or otherwise identify the rom downloaded

  3. If you built your rom, which repository:branch did you use?

    • Heads:Master
    • Other (please identify)
  4. What version of coreboot did you use in building?
    { You can find this information from github commit ID or once flashed, by giving the complete version from Sytem Information under Options --> menu}

  5. In building the rom, where did you get the blobs?

    • No blobs required
    • Provided by the company that installed Heads on the device
    • Extracted from a backup rom taken from this device(MAC-Adress in gbe.bin)
    • Extracted from another backup rom taken from another device (please identify the board model)
    • Extracted from the online bios using the automated tools provided in Heads(me.bin)
    • I don't know

Please describe the problem

Describe the bug
A clear and concise description of what the bug is.

The two USB-Ports close to the Power-Button don't work after heads update. (On two devices, t430-maximized with a clean checkout and t430-hotp-maximized from dirty local repository)

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

@tlaurion
Copy link
Collaborator

tlaurion commented Aug 1, 2024

Per https://github.com/linuxboot/heads/blob/master/BOARD_TESTERS.md

@nestire(t430-legacy, t430-maximized) @Thrilleratplay @alexmaloteaux @lsafd @bwachter(iGPU maximized) @shamen123 @eganonoa(iGPU) @nitrosimon @jans23 @icequbes1 (iGPU) @weyounsix (t430-dgpu): Can someone reproduce?

Master merged recently 24.02.01 coreboot version bump was t430 was reported working per @srgrint at #1723 (comment)

Which included non-related changes https://github.com/linuxboot/heads/pull/1723/files#diff-17c0a4050b64b2c80df9b9f451dba6a5e9307f56eb84c3e4696e532fa384b6c4 :

  • Top down resource allocation
  • heap size for bootsplash jpg rendering change
  • wifi init under coreboot

I would need someone reproducing this prior of investing time investigating the issue.

@Oessel, others: from recovery shell, can you please:

  • bash
  • source /etc/functions
  • enable_usb
  • plug usb device on either usb ports reported non-functional
  • lsusb
  • take screenshot
  • dmesg | grep -i -e usb
  • take screenshot

@Oessel
Copy link
Author

Oessel commented Aug 1, 2024

IMG_20240801_135956
IMG_20240801_135845

The ports are working under heads, but after booting qubes they aren't recognized. I also tested commit ID 2243 which is also affected and commit id 2221 is working fine

@tlaurion
Copy link
Collaborator

tlaurion commented Aug 1, 2024

@Oessel to my eyes everything works.

What does "USB ports don't work" mean explicitely? Where, when?

Under Heads, we see here EHCI driver enabling everything but USB1 controllers, then xhci enabling USB1 controller.
Hence not sure what "USB ports don't work" because not working is not saying anything to be worked on uless more context is given.

Please change title and give more info.From last reply, this is QubesOS/sys-usb related, where non-QubesOS users won't be affected from what I understand.

Please porive OS logs then from sys-usb. Maybe controllers for some reason have changed names? Check under sys-usb being turned off what PCI controller are affected?

@tlaurion
Copy link
Collaborator

tlaurion commented Aug 1, 2024

The ports are working under heads, but after booting qubes they aren't recognized. I also tested commit ID 2243 which is also affected and commit id 2221 is working fine

Ok, so related to top down resource allocation maybe as of now

@Oessel Oessel changed the title T430 side USB Ports don't work after heads update T430 side USB Ports don't work under QubesOS after heads update Aug 1, 2024
@tlaurion
Copy link
Collaborator

tlaurion commented Aug 1, 2024

The ports are working under heads, but after booting qubes they aren't recognized. I also tested commit ID 2243 which is also affected and commit id 2221 is working fine

Ok, so related to top down resource allocation maybe as of now

@Oessel note that of now, this is not considered Heads issue and reassigning pci devices under sys-usb qube settings devices tab will most probably fix this and may need to open issue under qubesos.

@marmarek : won't fix on your side too?

@marmarek
Copy link
Contributor

marmarek commented Aug 2, 2024

I guess PCI BDF of USB controllers changed after update? If that's the case, I think the firmware update doc (especially EDK-II -> Heads) should have a step to check/update devices assigned to sys-usb and sys-net.

In R4.3 we will have a bit smarter device identification, so at least there will be a clear message what happened (but hopefully it will deal with such changes automatically, at least in some cases).

@tlaurion
Copy link
Collaborator

tlaurion commented Aug 3, 2024

@Oessel ping

@Oessel
Copy link
Author

Oessel commented Aug 3, 2024

I have three USB controllers listed in the device tab of sys-usb, but with heads v0.2.0-2221 they all work fine, with heads v0.2.0-2243 and 2254 there are the same three controllers but the webcam, the internal smartcard reader and two usb ports are not recognized anymore, thats why i thought it could be heads-related.

@tlaurion
Copy link
Collaborator

tlaurion commented Aug 3, 2024

Ping @nestire(t430-legacy, t430-maximized) @Thrilleratplay @alexmaloteaux @lsafd @bwachter(iGPU maximized) @shamen123 @eganonoa(iGPU) @nitrosimon @jans23 @icequbes1 (iGPU) @weyounsix (t430-dgpu):

Can someone reproduce and give traces of what happen under OS/dom0, this is coreboot bug here.

@Oessel Oessel changed the title T430 side USB Ports don't work under QubesOS after heads update Some USB devices are not recognized on Thinkpad T430 under QubesOS after Coreboot Version bump Aug 3, 2024
tlaurion added a commit to tlaurion/heads that referenced this issue Aug 4, 2024
…SOURCE_ALLOCATION_TOP_DOWN is not set

Repro:
sudo sed -i 's/CONFIG_RESOURCE_ALLOCATION_TOP_DOWN=y/# CONFIG_RESOURCE_ALLOCATION_TOP_DOWN is not set/g' config/coreboot-*.config
grep -R CONFIG_COREBOOT_VERSION boards/ | awk -F "/" {'print $2'} | while read board; do if ! sudo make BOARD=$board coreboot.save_in_oldconfig_format_in_place  > /dev/null 2>&1; then echo $board failed;fi; done
git status | grep modified | awk -F ":" {'print $2'}| xargs sudo git add
git commmit --signoff -m "coreboot configs : CONFIG_RESOURCE_ALLOCATION_TOP_DOWN=y-># CONFIG_RESOURCE_ALLOCATION_TOP_DOWN is not set"

Fixes issue linuxboot#1740, reproduced on x230 (camera was not found, all USB controllers not changed under QubesOS, only cam missing)

Signed-off-by: Thierry Laurion <[email protected]>
@tlaurion tlaurion changed the title Some USB devices are not recognized on Thinkpad T430 under QubesOS after Coreboot Version bump Regression: Some devices are not discovered since 4.22.01 -> 24.02.04 Coreboot Version bump (upstream: bogus top down allocator) Aug 4, 2024
@tlaurion
Copy link
Collaborator

tlaurion commented Aug 4, 2024

@Oessel I replicated on x230: second msata m.2 and camera were not discovered: USB controllers IDs didn't change.
This is coreboot regression.

#1742 fixed the regression on my tests on x230. Please validate.


@JonathonHall-Purism no regression on librems with master?

Need of unit tests begins to feel.

@Oessel
Copy link
Author

Oessel commented Aug 4, 2024

Regression is fixed on t430 too!

@tlaurion tlaurion changed the title Regression: Some devices are not discovered since 4.22.01 -> 24.02.04 Coreboot Version bump (upstream: bogus top down allocator) Regression: Some peripherals are not discovered since 4.22.01 -> 24.02.04 Coreboot Version bump (upstream: bogus top down allocator) Aug 4, 2024
@Oessel Oessel closed this as completed Aug 4, 2024
@tlaurion tlaurion reopened this Aug 4, 2024
@tlaurion
Copy link
Collaborator

tlaurion commented Aug 5, 2024

This TOP DOWN allocator was introduced here by Nico Huber: https://review.coreboot.org/c/coreboot/+/41957

coreboot/coreboot@526c642

@i-c-o-n: Was that tested across all boards?

No, nothing in coreboot is.

Seems to cause regressions still today so not sure why its defconfig?

Because we prefer fixing bugs over working around them. The allocator code has proven to be sound so far. The problems are all around it (in chipset specific code, or even outside coreboot).

Not all fixes had been merged, though (see below).

@i-c-o-n I can provide logs if needed, let me know what you need

Please test with https://review.coreboot.org/c/coreboot/+/80207 applied. If that doesn't help, cbmem and dmesg logs for a start.

Originally posted by @i-c-o-n in #1742 (comment)

Alternative PR to not revert to bottom up allocator is pending and will superseed #1742.

@tlaurion
Copy link
Collaborator

tlaurion commented Aug 5, 2024

Instead of reverting regression as mitigation(#1742), apply fix and push testing of fixes applied upstream (#1743)

@i-c-o-n: works on x230.

@tlaurion
Copy link
Collaborator

tlaurion commented Aug 5, 2024

user@localhost:~/heads$ find config/coreboot-*| while read config; do mmconf=$(sudo grep CONFIG_HWBASE_DEFAULT_MMCONF $config | awk -F "=" {'print $2'}); bitlimit=$(sudo grep CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT $config | awk -F "=" {'print $2'}); echo; echo $config; if [ "$mmconf" == "$bitlimit" ]; then echo "CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT == CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT"; else echo REVIEW_CONFIG;fi;done | grep -B1 REVIEW_CONFIG
config/coreboot-librem_11.config
REVIEW_CONFIG
--
config/coreboot-librem_l1um_v2.config
REVIEW_CONFIG
--
config/coreboot-nitropad-ns50.config
REVIEW_CONFIG
--
config/coreboot-nitropad-nv41.config
REVIEW_CONFIG
--
config/coreboot-p8z77-m_pro-tpm1.config
REVIEW_CONFIG
--
config/coreboot-qemu-tpm1.config
REVIEW_CONFIG
--
config/coreboot-qemu-tpm2.config
REVIEW_CONFIG
--
config/coreboot-t420.config
REVIEW_CONFIG
--
config/coreboot-t430-legacy.config
REVIEW_CONFIG
--
config/coreboot-t430-legacy-flash.config
REVIEW_CONFIG
--
config/coreboot-t520-maximized.config
REVIEW_CONFIG
--
config/coreboot-t530-dgpu-maximized.config
REVIEW_CONFIG
--
config/coreboot-w530-dgpu-K1000m-maximized.config
REVIEW_CONFIG
--
config/coreboot-w530-dgpu-K2000m-maximized.config
REVIEW_CONFIG
--
config/coreboot-x220.config
REVIEW_CONFIG

Analysis:

  • Most of coreboot configs reported above are used by unmaintained_boards (not maintained for the moment)

To be edited after another round of review

@tlaurion
Copy link
Collaborator

tlaurion commented Aug 5, 2024

@i-c-o-n: q35/nv41/ns50 might miss a similar same patch?

user@localhost:~/heads$ sudo grep -Rn -e CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT -e  CONFIG_HWBASE_DEFAULT_MMCONF -e TOP_DOWN config/coreboot-qemu-tpm*.config 
config/coreboot-qemu-tpm1.config:213:CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT=0xfe000000
config/coreboot-qemu-tpm1.config:352:CONFIG_RESOURCE_ALLOCATION_TOP_DOWN=y
config/coreboot-qemu-tpm2.config:210:CONFIG_DOMAIN_RESOURCE_32BIT_LIMIT=0xfe000000
config/coreboot-qemu-tpm2.config:346:CONFIG_RESOURCE_ALLOCATION_TOP_DOWN=y

note: @macpijan nv41/ns50 have TOP_DOWN ress allocator dsabled on heads master.

@JonathonHall-Purism
Copy link
Collaborator

@JonathonHall-Purism no regression on librems with master?

Nope, no problems on Librems in master.

@tlaurion tlaurion changed the title Regression: Some peripherals are not discovered since 4.22.01 -> 24.02.04 Coreboot Version bump (upstream: bogus top down allocator) Regression: Some peripherals are not discovered since 4.22.01 -> 24.02.01 Coreboot Version bump (upstream: bogus top down allocator) Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants