Implement PEP 639, Metadata 2.4

[ewdurbin]

Depends on merge/release of pypa/packaging#828

Closes #16620

TODO: Move to release of packaging with license-expression parser

[miketheman]

Looking cool, and coming along nicely! I commented a bit inside.

One thing I'm curious about - at what point should we be looking to consolidate any file operations where we pop open the file and read stuff from it, so we only have to open/iterate once?

[ewdurbin]

One thing I'm curious about - at what point should we be looking to consolidate any file operations where we pop open the file and read stuff from it, so we only have to open/iterate once?

I thought about this a little and can imagine an approach of stashing a list of filenames when we iterate through to in _is_valid_dist_file, and fetching any files we actually need to read... at the moment that would only be .dist-info/METADATA. I wonder if a simple utility class wrapper that does this on __init__ would allow for us to make calls to get_wheel_metadata and contains or something would work... will poke a bit.

Edit: On second consideration... nearly all file uploads are dominated by either the advisory lock on journals or s3.putobject. Not sure this optimization is worth the effort at this point.

[ewdurbin]

@befeleme would you be able to take a look at this to review the implementation details?