Bring gnat6 back #225191
Bring gnat6 back #225191
Conversation
|
@vcunat for you to 👁️ too. Thanks for your pointer in matrix! |
|
Adding some background notes on why this is so badly needed. Gnat requires to be built from older versions. Gnat is required to build coreboot's libgfxinit, that is, native graphic initialization, providing firmware based text console or framebuffer. Coreboot buildstack is cross-built, per coreboot version, from host's buildstack to provide per version reproducible bootblock, ramstage, romstage etc. It is possible to build directly from any toolchain (corrboot config option exists) but that is far from being recommended and is really fragile. For one to be able to build coreboot's buikdstack dependable gnat, host buildstack needs to provide older version of gnat to be built. Under Heads, the host buildstack is also used to build musl-cross-make, which is then used to build all other linux libraires and binaries (Heads modules). So without gnat6, coreboot 4.11 platforms for which a configuration requires gnat (libgfxinit) will fail to produce rom. Newer coreboot crossbuilt buildstack would succeed being built from NixOS today, but older ones won't build. Older buildstacks are only needed to build newer cross-compiled buildstacks. NixOS already builds coreboot buildstack for newer coreboot versions (4.16+ if I recall well) but older and freer platforms having been dropped by coreboot (kgpe-d16, kcma-d8) under 4.11 won't cross compile on NixOS without gnat6. Can't wait to see this merged and starting to collaborate on building reproducible docker images and recipes to build on top of other Linux OS, dodging other reproducibility issues linked to having so many Linux host tools being moving targets and have the same ROMs built for same Heads commit 10 years from now. |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/prs-ready-for-review/3032/2059 |
|
|
||
| if [[ ${majorVersion} == 4 ]]; then | ||
| mv $out/bin/gnatgcc_2wrap $out/bin/gnatgcc | ||
| ln -s $out/bin/gnatgcc $out/bin/gcc | ||
| fi | ||
| '' |
There was a problem hiding this comment.
| if [[ ${majorVersion} == 4 ]]; then | |
| mv $out/bin/gnatgcc_2wrap $out/bin/gnatgcc | |
| ln -s $out/bin/gnatgcc $out/bin/gcc | |
| fi | |
| '' | |
| '' + '' lib.optionalString (majorVersion == 4) '' | |
| mv $out/bin/gnatgcc_2wrap $out/bin/gnatgcc | |
| ln -s $out/bin/gnatgcc $out/bin/gcc | |
| '' |
if we are already using nix variables, we can make this a little cleaner
| version = | ||
| if majorVersion == "4" then | ||
| gccVersion | ||
| else | ||
| "${gccVersion}-${alireRevision}"; | ||
| src = | ||
| if majorVersion == "4" then | ||
| fetchurl { | ||
| inherit url hash; | ||
| } | ||
| else | ||
| fetchzip { | ||
| url = "https://github.com/alire-project/GNAT-FSF-builds/releases/download/gnat-${version}/gnat-${stdenv.hostPlatform.system}-${version}.tar.gz"; | ||
| inherit hash; | ||
| }; |
There was a problem hiding this comment.
Maybe we can move this into versionMap, to reduce the branching and make the code a bit cleaner.
There was a problem hiding this comment.
I tried this there too originally, the version is easily moved up and I'll add that in. On the other hand src is quite a pain. I'm not sure how I'd avoid repeating the alire-project releases url for each version and architecture, it doesn't seem to end up being cleaner at all for src.
There was a problem hiding this comment.
Actually that doesn't work either, not sure why. When I tried building with cb798bcae40 nix-build errored with:
nix-build -A gnat-bootstrap4
error: undefined variable 'gccVersion'
at /home/manny/cloned/github.com/NixOS/nixpkgs/pkgs/development/compilers/gnat-bootstrap/default.nix:23:20:
22| alireRevision = "4";
23| version = "${gccVersion}-${alireRevision}";
| ^
24| } // {
So going to go back.
|
I requested reviews from @felixsinger and @blitz as I know you have interest or have been involved with coreboot ecosystem in the past. Apologies if this was not desired. |
|
This is an awesome PR as I use coreboot on my X230 with NixOS and I have been looking into adding a coreboot NixOS module to manage end-to-end everything in NixOS. :) Some classical points: please read the CONTRIBUTING guide, rename your commits accordingly |
May I ask why you are not building a more recent of coreboot? |
|
@RaitoBezarius Thanks for mentioning :) Will try to have a look later in the evening. |
This is mentioned in the second message:
|
Are these older coreboot versions being maintained? What would it take to update their required gnat version? I'm sympathetic, but I'm not sure we want to commit to indefinitely maintaining every package that happens to be required by an old, static version of Coreboot. |
|
+1, I would be much happier if we could figure out how to build coreboot for your platform without keeping around the older compiler(s) |
mmlb
force-pushed
the
gnat6
branch
2 times, most recently
from
5e735ed
to
7bbaebb
Compare
|
I can't really say whether we should have this in nixpkgs or not. I suspect it would be very welcome by anyone wanting to build coreboot for their platform. I suspect the maintenance burden to be low. I'd also like to try and use gnat6 as the bootstrap compiler for the other gnats. It would be nice to minimize external dependencies imo. |
|
@amjoseph-nixpkgs, I've rebased on top of #226179 to see what that looks like for this PR. |
|
@amjoseph-nixpkgs ping :) |
|
Ada might still be required. Otherwise some platforms stay in the dark until DRM+GPU kernel drivers take ownership of fb. With coreboot libgfxinit (which requires ada/gnat) a bootsplash can be showed within milliseconds until DRM+GPU takes over fb to provide proper fbwhiptail support. Otherwise leaving users in the dark for now around 8 to 10 seconds, depending of much stuff is inside of the spi chip and the speed in which that content can be read from spi (reading payload from spi up to jumping into it takes 6-8 seconds deonding if its size on older platforms at 50mhz) to a full fledged recovery environment userland (2 seconds). Alternatives are evaluated, but gnat support should not be excluded, at least in the transition. Details linuxboot/heads#1381 (comment) |
|
I feel like I'm still missing answers to some important questions here.
|
No, this is still using the same gnat4 binary originally from gentoo. If binary free bootstrapping is wanted I'm happy to look into it for gnat. |
@alyssais Sorry for the delay answering this. As said in tickets referred here, libgfxinit is not "required" anymore on the only platform having GMA (which libgfxinit supports) while kgpe-d16 implements native graphic init and doesn't need libgfxinit at all. The real problem here, if distributions wants to use Nix/NixOS as a buildstack for coreboot branches, is that coreboot by itself, outside of its possible configurations, will not build unless gnat is available for libgfxinit. So for example here, libreboot being a distribution of coreboot and having decided to pack seabios/grub as a payload will depend on seabios+vbios blob or use seabios graphic init. But that might change. Skulls depends on seabios the same way. linuxboot tries to provide the most generic linux configuration possible (now directly under coreboot since 4.18 if I recall well) and therefore the same problem arises on buildstack dependencies to gnat there, and yet again, if some boards are to depend on libgfxinit+linux to provide VESA, things might fail requiring Nix users to tackle this problem again. libreml1um, and 4.11 coreboot, could definitely be patched directly under Heads though, if for whatever reason they would want to reenable libgfxinit. Note that libgfxinit offers interesting properties, including drawing a bootsplash directly in ramstage, so that slowing platforms reading from SPI are not left in the dark until linux kernel owns the graphic card for the first time in absence of libgfxinit. The reason of all of this is purely generic. Ideally, older platforms supported by coreboot 4.11 releaseshould be able to be built if provided with the proper dependencies. And abusing Nix for thatwould simply be amazing. But to be honest, the librem1um being the only known case for libgfxinit potentially being needed might not justify the effort today. The platform is fast enough to not leave users in the dark until i915+drm drivers to own and draw in a functional framebuffer. The ecosystem moving forward and away of fb, preferring KMS enabled drm drivers are desired and being pushed forward. So ideally, firwares would present a fb (native init or libgfxinit or efifb) so that the linux kernel can use it without overhead. Ideally, this would mean a single drm driver (simpledrm) with a single fb driver to fallback to (simplefb) which works out of the box without having to pack in the kernel anything big or complicated to get a console output on screen. When those combined (sandy/ivy bridge platformsused as example here) libgfxinit can draw a bootsplash within millisecond, after which slow spi access continues reading kernel initrd (8 seconds showing bootsplash) and then the kernel kicks in and can use sysfb exposed fb through simplefb. When kexec'ing into another OS, that framebuffer (simplefb) can be exposed through kexec and be reused again through simplefb+simpledrm and then more complex drm drivers can take ownership of the fb again for i915drm. Without libgfxinit, the kernel needs to pack i915drm which even if compressed takes 500kb of precious SPI space. And leave the user in the dark untilthat i915drm driver can draw in the console. This is now 10 seconds on the x230. No problem here, the x230 is currently maintained under coreboot. But the librem1um was supported 2 years ago and now isnt. And the ecosystem is moving leaving that platform not being able to be built with Nix today. Now, on what really matters here for Nix, ideally, if any of those platforms are deprecated into coreboot tomorrow, the desire is to be able to pin a package commit that would permit that board to continue to be able to build coreboot without any patches needed on top of coreboot. A downstream project using Nix to build a reproducible ROM should be able to do so in 10 years from now. Sorry for the long reply and hoping this explains why those old perceived tools are needed otherwise patching things are needed and therefore repruducible builds cannot be obtained (We cannot build librem1um with libgfxinit enabled in coreboot today with Nix). |
mweinelt
added
the
12.approvals: 2
|
@ofborg eval |
ghost
marked this pull request as draft
|
Rebased. Apparently github can't see that this is a nonconflicting rebase. |
ghost
force-pushed
the
gnat6
branch
2 times, most recently
from
bef646f
to
ac68fed
Compare
ghost
marked this pull request as ready for review
|
My assumptions:
If you want to build coreboot 4.11 from 2019, why don't you use a nixpkgs checkout from 2019, which has the gnat6 you need (more specifically Otherwise what is the expectation; until when do we keep this around? |
This is an incorrect assumption. Coreboot 4.12 dropped support for all the IME/PSP-free hardware. It was a very contentious decision. A lot of people are still very bitter about it. Please don't reopen this old wound.
Until
The removal had more to do with nobody being motivated to fix it. As long as @mmlb is willing to maintain it with high-quality PRs like this one I think we should merge them. If it breaks again and @mmlb goes AWOL we can remove it again. |
|
@mmlb, do you have an alternative source for |
I do not have one up any where atm but would be fine with a quick GH repo with a release that contains the tarball. |
Not really, we're still using a binary build of gnat here.
Very far it seems. I've recently learned that gnat has never been bootstrappable, even the first release required a pre-existing gnat compiler 🤯 . There's a project at the very earliest of stages to try and bootstrap gnat (https://fossil.irvise.xyz/gnat-bootstrap/forum) |
wegank
removed
the
12.approvals: 2
Mostly a partial revert of 6e64369 but massaged to make it work with the current gnat-bootstrap/default.nix setup.
Use gnatboot4 to compile gnat6. Bringing this back to support building coreboot 4.11 outside of nixpkgs.
|
I've uploaded the gnat4.1 tarball to a repo I control and have updated the PR to use it instead of mirror based off from gentoo. I'd really like to see the repo under nixos' umbrella instead though, thoughts on doing so? |
Interestingly, it happens I went into the rabbit hole with OwnerBoot. I am using the coreboot-toolchain 4.16 with coreboot 4.9 (worse than 4.11 arguably) as this is the ownerboot tree (https://git.sr.ht/~amjoseph/ownerboot/). Basically, for my X230, I encountered many toolchain issues indeed and it was not super hard to fix but quite annoying to use intuition to guess "oh this is a new warning, let's silence it here, let's do that, etc." or read next commits to find out about a new GCC breakage for GNAT (or even GCC!). So I can confirm it is possible to get: Of course, it fails for unrelated reason I believe: Maybe, I will get somewhere on a future nerdsnipe in some oceansprint.org with some fellows who are interested into better coreboot in nixpkgs. As a matter of fact, I believe it is very possible to use libgfxinit + libhwbase on older coreboot with newer coreboot-toolchain (4.16), I think we can go further, but I will let this to the next one, the only reason I went for 4.16 is that glibc changed the fact that minimum stack size was a constant, and it created serious issues with GCC 8.3.0. |
wegank
added
the
2.status: stale
|
no longer needing this, so just going to go ahead on close |
Description of changes
Re-enables gnat6. This is mostly so we can build coreboot-4.11 without having to override its own build-tools scripts. coreboot seems to be very picky about its compilers and I'd like to let it do its thing. This will be helpful for linuxboot/heads#1269 for example.
I've also renamed gnatboot to gnat-bootstrap. I did this primarily for 2 reasons.
gnatbootwas the name of an actual binary, not a bootstrapping compiler. The new names makes it much more obvious to others like me.Things done
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)