Gathering data from package managers

[mcutshaw]

Is this something we want to do? Is this something that fits into the existent schema? Its potentially possible to pull from apt/dpkg, a variety of linux package managers, I'm sure its doable on mach-os/windows as well.

[nightlark]

Yes, it is something that I think we should do -- but it isn't really clear how it fits into the existing schema, other than as "systems", some metadata attached to software entries for files, or "fake" software entries with no actual file/hash (at least one hash is required though, so this wouldn't work...)

Being able to get PURLs/CPEs for packages present on a system would be useful for finding CVEs that may apply, and mapping the files we see to what common package they are part of would be helpful for future analysis.

I was wondering if it would be useful to have pre/post file id/info extraction hooks that run for each top-level directory (/context entry) for this type of information gathering. It seems like the type of thing that could benefit from looking at the files present as a whole, and perhaps running before the individual file info extractors (so it may be the thing that creates software entries for files it finds that are associated with packages).

[nightlark]

I think we may also need to figure out how to handle "folder"-based software/applications, which could be relevant for tying package manager info to files. A macOS application is an example of this -- it's a folder with a name that ends in .app that has a particular directory/file structure within it; for the purposes of a creating an SBOM, some of these files contain useful metadata (Contents/Info.plist) and indicate how other files within the Application relate to one another (e.g. which one is the main binary).

Similarly, the Python packaging ecosystem has .dist-info folders within the site-packages folder for a Python install, which contain a variety of files with package metadata and what files installed on the system make up that package (assuming multiple packages don't have files that conflict with one another). Some questions that need to be considered for this particularly case, but that are likely relevant to other package managers and illustrate a bit of how the "entire folder structure" needs to be taken into account include:

• Does a dist-info folder mean anything to Python if it isn't the immeditate child of a site-packages folder?
• Is it the dist-info folder name ending that has meaning for Python, or is it the set of files contained within the folder that Python picks up on?
• How do you determine if a folder is a site-packages folder for Python?
• Can a Python site-packages folder ever have a different name? What about for different interpreters (CPython vs PyPI vs IronPython)?

Maybe support for this could be an "id_special_directory"/"extract_directory_info" hook that looks at a directory and tries to determine if it is special for a particular ecosystem? There are some questions that come up though for when it should run... on one hand, it seems like it could make sense as soon as a directory is encountered -- but if it encounters information that should update the info for a file within the folder, then the software entry won't exist yet.

• Should it need to worry about creating/locating an existing software entry and adding info to them?
• Should the information gathered be attached to the "folder" and then processed as part of the analysis/relationship step later?
• In some cases the directory should be included as a software entry in the SBOM; in other cases it should be a "virtual" package concept that itself doesn't live on disk, but contains files that do. How do we determine the right thing to do? Building off this a bit more... will all files within a "folder" package be "Contained" within it, or are there cases where some files within the folder aren't part of the package (data/config/plugins that were installed later)?

[nightlark]

Adding a Grype plugin (#196) could be an example of a plugin that would benefit from having a hook to run on top-level directory entries.