Parse the memtest output

[MeTalOneOEight]

Thanks for your effort on this.
I saw your reply on one of my posts on macuser.de.

A feature you may want to add in the future:
Memtest can output a file with the faulty mem locations.
The bad ram patch can use it, so maybe you could look for the file parsing code there.

Another thing, don't call this a script, it's a C program that needs compiling.
I think it confuses people.

[0nelight]

Hi! Thanks for pointing out :)

Let's call it a "Utility" from now on :)

Actually I updated the Utility today so that it can take the lowest Error Address and highest Error Address provided by the memtest86+ in the "Error Summary" mode.

You can find it under Release Page.

I also thought about supporting the badram-pattern but given the - in my eyes - often unknown behaivour of the non-public Apple EFI 1.1, the limited avaliablity of ready-made parsing code for EFI 1.1 and my limited expirience in C I have currently no plans implementing it - the current Version with providing the lowest and highest error address via the rEFInd command line is already a Milestone for me. 😊

[MeTalOneOEight]

Learning by doing.
I don't know what the EFI API provides and how difficult it would be to implement.
Another milestone may be the integration into Open Core Legacy Patcher.
But I guess that would a project for a long winter night.
Maybe create a Pull request or something, so they notice and maybe integrate it and do some improvements on the code.

[adrian77]

I also used MemTest output to identify the memory range. In my case it was 32 MB big. But later when I processed the memtest log. I realised it was actually two smaller ranges of 2.5kb with alost 32 MB space between.

Logic to find these:

1. Scan the log for faulty addresses.
2. Cluster the values and identify the buckets.
3. Add these buckets or individual addresses into reserve-ramtxt
4. Make support for disable-ram.efi to support file argument to load the information from. (Note: that will also perhaps avoid need of nsh script and full_efi shell if argument less design would load the file as default)

[0nelight]

Current way to do this would be to run the Utility multiple times to get all areas covered - fe. with an nsh file.

Regarding loading the ram-areas from a txt file - sure it can be done and would be a nice project for someone who has the passion to implement this. Everybody is free to fork the repo. 😊

It is possible that the defective areas don't stay the same position and size between reboots depending on the exact reason for the RAM-Defect. It can be on multiple levels. So disabling a larger area could be benefitial too.

Always nice to see people exploring this low level stuff. Nice work and cool Diagram 😊

[BrianCArnold]

I actually need to block several areas of memory on a friend's MBP 2019, and I am a C developer. Well, mostly a C# dev, but I still know enough C to read a file in and parse it.

And I have the rest of the week off to work on this.

[MeTalOneOEight]

Maybe using edk2 would make it easier, since it comes with more helper functions for EFI development.
It seems to have a steep learning curve though and the documentation was a bunch of HTML files.

I was thinking of general improvements to make this more user friendly, but I guess this would end up in a complete rewrite.

The parsing of the memtest log could be moved to an installer, since memtest saves it on the test USB stick.
A config file is needed then though.

I guess it also would be better to change the loading logic.
Let this tool chainload the boot loader, there's sample code how to do that.
Saves dealing with the nsh and rEFInd configs and makes it easy to integrate into OCLP.

[BrianCArnold]

Yeah, my current plan is to finish creating a file parser, and then chainload a configured EFI executable, whether MacOS, rEFInd, or ZBM.

Note: also rewrite the README.md to reference the fact that this is actually not a Mac specific tool at all, but a universal x86_64 UEFI application that can be used on any EFI system that needs to restrict which pages should be used by an OS.

Obviously, we need instructions on how to use it on an Intel Mac, but its no more specific to Macs than Thunderbolt or USB-C are.

[0nelight]

Apple has been implementing UEFI 2.0 features ever since thex went x64, though never really complied to either EFI or UEFI spec to full extent. Source

So this repo is meant only for the - Apple EFI (allegedly EFI 1.1) - devices. Feel free to make you own repo regarding support for more platforms. Happy to support you there! :-)

[adrian77]

If the program is on EFI 1.1 level, it targets vast of HW right? If future generations only support 2.0 and drops 1.1 that might be a problem if program is using stuff that has been dropped in 2.0.

So do you know guys if program works on UEFI 2.0 only systems? If not, can one create two files for targeting different generations? Or make one file autodetect the generation?
Unless it already something at compile / SDK level.

@0nelight would it be ok to use your repo for such support and contributing code, or you still feel a separate repo for catering the variations?
Basically we read you that you don't want to spend much more work on this :)

[0nelight]

I would like to keep this Repo for Apple MacOs Support only.

Because we don't know what (parts of) EFI and what Versions of EFI Apple uses (and will use in the future) and even how exactly their EFI implementation looks like on their devices - so that is enough complexity to maintain for me.

I am happy to contribute with my comments or otherwise to a fork which supports a wider range of systems - maintained by one of you two guys - or somebody else. 😊

[MeTalOneOEight]

I don't think it uses any of the deprecated functions and drivers listed in the UEFI 2.x specification under chapter
L.1 Protocol and GUID Name Changes from EFI 1.10
after a quick check.

Some people consider gnu-efi more a problem since it's binary hack to produce the .efi binary.

Apple dropped UEFI anyway for the ARM Macs and I don't know how many other companies use soldered RAM on their machines.
Windows, FreeBSD and Linux already have a Bad RAM function integrated.
No need to use this tool for them.

[0nelight]

Why do you think Apple dropped EFI on their ARM Macs? Have they?

[adrian77]

@MeTalOneOEight I think the target is both soldered RAM, but also generally non ECC RAM, which is laptops. Most of desktop and servers have ECC RAM. But if windows and linux already have means to achieve it by config, then it becomes rather Mac specific.

Now I am concerned about your statement ARM Macs dropping UEFI, because I also know the M3/M4 will be non ECC soldered ram.
And I am planning to upgrade from my intel to M4, but I got really upset having 0,001% ram problems after already 4 years costing me 1000$ to repair if I didn't had this utility that could defer my repair cost or new purchase to future.

[MeTalOneOEight]

@0nelight
Since ARM Macs are basically giant iPhones hardware wise it's only natural Apple uses the same boot mechanism for both.
Maybe also more in-house control and less licensing from other companies.

@adrian77
You may invest in AppleCare+ then, if you want to factor in future repair costs before purchase for Total Cost of Ownership.
Beside the possible RAM failure, you need to consider that the ARM Macs won't boot anymore if the internal SSD fails.
No external booting without it.
Though SSD are durable these days with high MTBF.

[adrian77]

Yes, AppleCare+ can help, but only up to 3 years. 4+ years down the road and you bump into something like I did 1 bit flipping on specific bit on few addresses one is really doomed.

With Apple silicon based chip it seems they use complete different loader which is not open. Like iBoot with more secure components involved to prevent HW tampering. They seem to support also booting other OSes, like linux. But windows will not longer work due to arm arch. I also understand that non ECC will not be supported in near future, due to i) manufacturing good quality ii) micro soldering teqnique iii) energy consumption iv) decreased performance.

This is bit off the topic, but interesting.

[0nelight]

@MeTalOneOEight But I think think your statement that they are not using EFI anymore is speculation on your side? Do you have any source for your claim?

@adrian77 .."it seems they use" - is it also speculation on your side?

[MeTalOneOEight]

@0nelight
No guessing, for example look at the reverse engineering Asahi Linux has done and documented:
https://github.com/AsahiLinux/docs/wiki/Open-OS-Ecosystem-on-Apple-Silicon-Macs

They created their bootloader U-Boot to create an EFI environment.
But that doesn't help here in this case.

[adrian77]

@0nelight I consulted chatGPT on the matter.

[adrian77]

@0nelight I was just imaging that we would ironically discuss again same issue around 4 years from now at the time being obsolete apple silicon based laptop and how to fix 'em. Like: here we go again....

Now, again off topic. My intentions with upgrade to M3/4 is to use it for LLM models.
Basically buying 64/128 GB laptop and run local AIs (ollama/huggingface models). But after this experience and the nature of LLM occupying major areas of memory , knowing its non ECC I start to think very differently what a laptop should be used for.

I mean using a laptop with some faulty memory segments after 4+ years might be ok. But not for running huge programs, dealing with huge files, even from development point of view. I moved away from desktops long time ago. But the AI/LLM use cases brings me back on desktop computer thoughts, or datacenter approach off course.

[0nelight]

@0nelight , this also brings me slightly back to the other discussion related to verification of patch applicability . That what happens if the EFI program is sitting in the bad area? And reserving memory simply fails. In term of automation, have you thought about it? Like perhaps reboot computer again?

So the RAM Area the EFI Program is using is microscopic compared to the whole System-RAM of >8GB. So you have to have pretty dramatic bad luck to expirience such a case. First find a MacBook which has a faulty RAM - which is very rare I think. Then you have to have this error occuring in the first few kbytes of your RAM - again very, very, very rare. (Off)-Topic: Nothing in life is 100% certain.

@MeTalOneOEight: Asahi did a lot of work understanding the Boot Process of Apple Silicon, so your comment is pretty useless. Apple Silicon has to allocate their memory also in "some" way. So alternative Boot Managers will arise and will make it possible to preload something before booting MacOs - of course breaking the security of the System in "some" - but not necessarily - in a relevant way.

[MeTalOneOEight]

@0nelight well why did you link the m1n1 docs first (on which I replied) and then edited it to the introduction link?
Just to tell me my comment is useless?

[0nelight]

I found the introduction link more relevant to the discussion here - that is the reason I changed it. At that time I wasn't even aware of you high-value comment. Seems you made your concolusion too fast. 😄

[0nelight]

@adrian77 Would be interesting what an Apple Silicon Mac outputs for "sysctl -a | grep memmap".

[adrian77]

@0nelight true, also considering the apple M chips uses unified memory which is shared between CPU, GPU and Neural Engine.

This memory is also part of SoC (chip) it self instead of external to CPU/GPU as for Intel Macs.
Now of course one wonders if tear and wear issues becomes worse or better considering that data between CPU and GPU never need to be shuffled between the different memory spaces.

After few reasonings with ChatGPT I lean towards its might be better for the wear and tear over time.

Sharing some conclusions, but there are no really proofs of that.