Optimize copy_file

* `ctx.actions.symlink` can be used on all platforms and falls back to a copy if a symlink is unsupported. It is also heavily optimized, avoiding the need to hash the output file again.
* Bazel never guarantees that an input to an action is staged as a non-symlink, so whether an output is a symlink or a hard copy only matters for top-level outputs consumed outside Bazel and handled by tools that don't follow symlinks by default, which should be extremely rare and could be worked around by explicitly setting `allow_symlink` to `False` in user-controlled code.
* For file copy actions that do not go through `ctx.actions.symlink`, caching is extremely cheap since the CAS entry of the input will be reused as the CAS entry of the output. Allowing remote execution and caching enables BwoB for the copy, which can avoid downloads of both the input and the output file.

The same changes are not applied to `copy_directory` as source directories are not officially supported by Bazel and any kind of change could cause subtle incorrectness.

Author: fmeum

docs/copy_file_doc.md

@@ -34,7 +34,7 @@ This rule uses a Bash command on Linux/macOS/non-Windows, and a cmd.exe command
 | <a id="copy_file-src"></a>src |  A Label. The file to make a copy of. (Can also be the label of a rule that generates a file.)   |  none |
 | <a id="copy_file-out"></a>out |  Path of the output file, relative to this package.   |  none |
 | <a id="copy_file-is_executable"></a>is_executable |  A boolean. Whether to make the output file executable. When True, the rule's output can be executed using `bazel run` and can be in the srcs of binary and test rules that require executable sources. WARNING: If `allow_symlink` is True, `src` must also be executable.   |  `False` |
-| <a id="copy_file-allow_symlink"></a>allow_symlink |  A boolean. Whether to allow symlinking instead of copying. When False, the output is always a hard copy. When True, the output *can* be a symlink, but there is no guarantee that a symlink is created (i.e., at the time of writing, we don't create symlinks on Windows). Set this to True if you need fast copying and your tools can handle symlinks (which most UNIX tools can).   |  `False` |
+| <a id="copy_file-allow_symlink"></a>allow_symlink |  A boolean. Whether to allow symlinking instead of copying. When False, the output is always a hard copy, but actions consuming that output as an input may still see a symlink (e.g. when using sandboxed excution). When True, the output *can* be a symlink, but there is no guarantee that a symlink is created (i.e., at the time of writing, we don't create symlinks on Windows by default). This defaults to True if `is_executable` is False, and False otherwise.   |  `None` |
 | <a id="copy_file-kwargs"></a>kwargs |  further keyword arguments, e.g. `visibility`   |  none |
 
 

rules/private/copy_file_private.bzl

@@ -19,8 +19,6 @@ cmd.exe (on Windows). '_copy_xfile' marks the resulting file executable,
 '_copy_file' does not.
 """
 
-load(":copy_common.bzl", "COPY_EXECUTION_REQUIREMENTS")
-
 def copy_cmd(ctx, src, dst):
     # Most Windows binaries built with MSVC use a certain argument quoting
     # scheme. Bazel uses that scheme too to quote arguments. However,
@@ -46,7 +44,6 @@ def copy_cmd(ctx, src, dst):
         mnemonic = "CopyFile",
         progress_message = "Copying files",
         use_default_shell_env = True,
-        execution_requirements = COPY_EXECUTION_REQUIREMENTS,
     )
 
 def copy_bash(ctx, src, dst):
@@ -58,7 +55,6 @@ def copy_bash(ctx, src, dst):
         mnemonic = "CopyFile",
         progress_message = "Copying files",
         use_default_shell_env = True,
-        execution_requirements = COPY_EXECUTION_REQUIREMENTS,
     )
 
 def _copy_file_impl(ctx):
@@ -104,7 +100,7 @@ _copy_xfile = rule(
     attrs = _ATTRS,
 )
 
-def copy_file(name, src, out, is_executable = False, allow_symlink = False, **kwargs):
+def copy_file(name, src, out, is_executable = False, allow_symlink = None, **kwargs):
     """Copies a file to another location.
 
     `native.genrule()` is sometimes used to copy files (often wishing to rename them). The 'copy_file' rule does this with a simpler interface than genrule.
@@ -121,11 +117,12 @@ def copy_file(name, src, out, is_executable = False, allow_symlink = False, **kw
           in the srcs of binary and test rules that require executable sources.
           WARNING: If `allow_symlink` is True, `src` must also be executable.
       allow_symlink: A boolean. Whether to allow symlinking instead of copying.
-          When False, the output is always a hard copy. When True, the output
-          *can* be a symlink, but there is no guarantee that a symlink is
-          created (i.e., at the time of writing, we don't create symlinks on
-          Windows). Set this to True if you need fast copying and your tools can
-          handle symlinks (which most UNIX tools can).
+          When False, the output is always a hard copy, but actions consuming
+          that output as an input may still see a symlink (e.g. when using
+          sandboxed excution). When True, the output *can* be a symlink, but
+          there is no guarantee that a symlink is created (i.e., at the time of
+          writing, we don't create symlinks on Windows by default). This
+          defaults to True if `is_executable` is False, and False otherwise.
       **kwargs: further keyword arguments, e.g. `visibility`
     """
 
@@ -142,6 +139,9 @@ def copy_file(name, src, out, is_executable = False, allow_symlink = False, **kw
             "//conditions:default": False,
         }),
         is_executable = is_executable,
-        allow_symlink = allow_symlink,
+        # Default to True if is_executable is False since symlinking avoids
+        # running a full action to copy the file. If the output needs to be
+        # executable, a copy may be required if the input isn't.
+        allow_symlink = allow_symlink if allow_symlink != None else not is_executable,
         **kwargs
     )