Should a Gazelle plugin pre-compute and store information about external dependencies?

[cgrindel]

Related to #1591.

Overview

Some Gazelle plugins (e.g., Go), process information about external dependencies during the loading phase. Other Gazelle plugins (e.g. rules_swift_package_manager), pre-process information about external dependencies storing the result in a file (e.g. swift_deps_index.json) that is read during the loading phase.

Should Gazelle plugins pre-process external dependencies? If so, what information is appropriate to be stored?

Background

The rules_swift_package_manager Gazelle plugin supports update-repos. When a new external dependency is added to the Package.swift file, the client runs update-repos to recompute the external dependency information. One of the update-repos command outputs is a swift_deps_index.json file (example). This file contains the following information:

• Direct dependencies for the workspace. This is used for visibility and package-module-label resolution.
• Swift package information (e.g., download info).
• Swift module information (e.g., Bazel target)
• Swift product information (e.g., Bazel targets, package mapping)

The swift_deps_index.json is then loaded by the swift_deps extension when bzlmod is enabled.

Thoughts on Pre-processing External Dependencies

Here are some benefits of pre-processing external dependencies:

• Reduce the Frequency of Compute Heavy External Dependency Resolution: The heavy lifting (e.g., resolving transitive dependencies, map/index creation) only happens when external dependencies are added or updated.
• Simplified loading phase logic: The loading phase logic could be as simple as downloading the dependency and loading the pre-computed information.
• Transparency/Easier Debugging: The serialized output can be very helpful in debugging language import to Bazel target resolution.

Some negatives of this approach:

• Need to Execute a Command When Adding a Dependency: This is true. However, most languages require this anyway.
• Yet Another File in Source Control: Today, each language that pre-computes external dependencies needs to store the information in its own file. However, I could imagine this could be included in the Bazel lock file if sections were set aside for language-specific information and Bazel became aware of when new dependencies were added.

I am interested to understand what the community thinks about this approach.