Unable to install; Large overhead without any snapshots

[jasonw22]

Are there other things that can cause APFS overhead besides time machine snapshots?

APFS [Macintosh HD] (2.00 TB, 6 volumes)
  Total size: 2.00 TB
  Free space: 418.62 GB
  Available space: 301.78 GB
  Overhead: 78.85 GB
  Minimum new size: 1.69 TB (84.88%)

  Warning: The selected partition has a large amount of overhead space.
  This prevents you from resizing the partition to a smaller size, even
  though macOS reports that space as free.

  This is usually caused by APFS snapshots used by Time Machine, which
  use up free disk space and block resizing the partition to a smaller
  size. It can also be caused by having a pending macOS upgrade.

  If you want to resize your partition to a smaller size, please complete
  any pending macOS upgrades and visit this link to learn how to manually
  delete Time Machine snapshots:

    https://alx.sh/tmcleanup

» Continue anyway? (y/N):
(base) jason@Jasons-MacBook-Air ~ % tmutil listlocalsnapshots /
Snapshots for disk /:
(base) jason@Jasons-MacBook-Air ~ %

[PaulCharlton]

see:

#243 (comment)

[PaulCharlton]

heuristic need updating:

asahi-installer/src/main.py

Line 718 in 5f0814b

if overhead > 16_000_000_000:

---

the volume in question above has about 78GB overhead on a 2 TB partition ... we should likely use a percentage, or some multiple of a snapshot's increase in space used instead of a raw 16GB criteria.

16GB overhead is a drop in the water from a 2TB partition. even 78GB is reasonable under the circumstances.

see also #243

[PaulCharlton]

regarding the heuristic here:

asahi-installer/src/main.py

Line 718 in 5f0814b

if overhead > 16_000_000_000:

it may be reasonable to set the minimum as "overhead + 16GB" instead of "16GB" ... ultimately a judgment call by the macOS user and how much disk space they want to set aside for macOS if they are actually using Linux.

[marcan]

The real question here is what causes overhead and how we can reduce it. The heuristic is just random. There is no evidence that a percentage approach is more correct. If you want to help this issue, please research what causes purgeable data in macOS so we can actually understand what is going on and where the overhead comes from, and whether we can clear it somehow.

@jasonw22 the installer clearly lets you continue there, so "Unable to install" is not correct. It's just a warning.

[jasonw22]

Thanks @marcan . It wasn't clear whether there would be negative consequences from continuing in this state, so I paused there. But given the above discussion, I agree it seems safe enough to continue. If I run into any problems I'll re-open this.

[PaulCharlton]

@marcan did a bit of a deep dive on what creates "purgeable" overhead -- as far as I can see, there are a number of main contributors:

1. Time Machine snapshots, can be managed with tmutil ...
2. APFS snapshots (other than TimeMachine), can be managed with diskutil apfs ...
3. APFS volumes (other than the current volume) in the same APFS volume group and container (sharing the underlying APFS container), can be managed with diskutil apfs .... The Update volume appears to be one of these to hold pending downloads which have not yet been applied.
4. mobile device backups ( located at ~/Library/Application Support/MobileSync/Backup/)
5. Spotlight indices - manage with mdutil -I off /VolumeMount and mdutil -E /VolumeMount
6. large file deletes on APFS are chunked into smaller transactions -- if the system is shutdown and those transactions are not done posting, the space from the large file is still (partially) marked as "in use". AFAIK the only way to clear this is with the first aid utility which will find orphaned blocks and mark them as free/available space in that APFS container.

of note, 1-3 do not show up with utilities such as DaisyDisk or GrandPerspective