`local_runtime_repo` fails on macOS/Xcode Python with watch_tree() error against missing directory

[erikkerber]

🐞 bug report

Affected Rule

local_runtime_repo

Is this a regression?

No

Description

The metadata for /usr/bin/python3 on macOS will return a path that does not exist (by default):

DEBUG: local_runtime_repo.bzl:101:10: {"major": 3, "minor": 9, "micro": 6, "include": "/Library/Python/3.9/include", "implementation_name": "cpython", "base_executable": "/Applications/Xcode-16.4.0.app/Contents/Developer/usr/bin/python3", "LDLIBRARY": "Python3.framework/Versions/3.9/Python3", "LIBDIR": "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib", "INSTSONAME": "Python3.framework/Versions/3.9/Python3", "PY3LIBRARY": "", "SHLIB_SUFFIX": ".so"}

This is pointing to a stock Python3 on macOS. The reported include directory does not exist, and will result in the error:

Error in watch_tree: can't call watch_tree() on non-directory /Library/Python/3.9/include

🔬 Minimal Reproduction

Cribbing heavily off of the local toolchain setup in the docs, on macOS 15.5:

python.toolchain(python_version = "3.9")

local_runtime_repo = use_repo_rule(
    "@rules_python//python/local_toolchains:repos.bzl",
    "local_runtime_repo",
)

local_runtime_toolchains_repo = use_repo_rule(
    "@rules_python//python/local_toolchains:repos.bzl",
    "local_runtime_toolchains_repo",
)

local_runtime_repo(
    name = "local_python3",
    interpreter_path = "/usr/bin/python3",
    on_failure = "warn"
)

local_runtime_toolchains_repo(
    name = "local_toolchains",
    runtimes = ["local_python3"],
)

# Step 3: Register the toolchains
register_toolchains("@local_toolchains//:all", dev_dependency = True)

🔥 Exception or Error

2025-07-01 10:07:16 CDT | : in 
-- | --
  | 2025-07-01 10:07:16 CDT | Repository rule local_runtime_repo defined at:
  | 2025-07-01 10:07:16 CDT | /Users/buildkite/.cache/bazel/a876e18133e23b0f96cd6798b2a722d4/external/rules_python+/python/private/local_runtime_repo.bzl:149:37: in 
  | 2025-07-01 10:07:16 CDT | ERROR: /Users/buildkite/.cache/bazel/a876e18133e23b0f96cd6798b2a722d4/external/rules_python+/python/private/repo_utils.bzl:448:25: An error occurred during the fetch of repository '+_repo_rules+local_python3':
  | 2025-07-01 10:07:16 CDT | Traceback (most recent call last):
  | 2025-07-01 10:07:16 CDT | File "/Users/buildkite/.cache/bazel/a876e18133e23b0f96cd6798b2a722d4/external/rules_python+/python/private/local_runtime_repo.bzl", line 102, column 26, in _local_runtime_repo_impl
  | 2025-07-01 10:07:16 CDT | repo_utils.watch_tree(rctx, rctx.path(info["include"]))
  | 2025-07-01 10:07:16 CDT | File "/Users/buildkite/.cache/bazel/a876e18133e23b0f96cd6798b2a722d4/external/rules_python+/python/private/repo_utils.bzl", line 448, column 25, in _watch_tree
  | 2025-07-01 10:07:16 CDT | mrctx.watch_tree(*args, **kwargs)
  | 2025-07-01 10:07:16 CDT | Error in watch_tree: can't call watch_tree() on non-directory /Library/Python/3.9/include
  | 2025-07-01 10:07:16 CDT | ERROR: no such package '@@+_repo_rules+local_python3//': can't call watch_tree() on non-directory /Library/Python/3.9/include
  | 2025-07-01 10:07:16 CDT | ERROR: /Users/buildkite/.cache/bazel/a876e18133e23b0f96cd6798b2a722d4/external/+_repo_rules2+local_toolchains/BUILD.bazel:6:30: @@+_repo_rules2+local_toolchains//:0001_default_py_cc_toolchain depends on @@+_repo_rules+local_python3//:os in repository @@+_repo_rules+local_python3 which failed to fetch. no such package '@@+_repo_rules+local_python3//': can't call watch_tree() on non-directory /Library/Python/3.9/include
  | 2025-07-01 10:07:16 CDT | ERROR: /Users/buildkite/builds/mbr-buildkite-agent-ios-prod-polite-carp-p6d4-1/slack/ios-pull-request/tools/pipeline_generator/BUILD.bazel:16:10: Analysis failed
  | 2025-07-01 10:07:16 CDT | ERROR: Analysis of target '//tools/pipeline_generator:main' failed; build aborted

🌍 Your Environment

Operating System: macOS 15.5

Output of bazel version:

  
Bazelisk version: 1.26.0
INFO: Invocation ID: 6e64a830-cfe4-4daf-bc8e-28424507a9c3
Build label: 8.2.1
Build target: @@//src/main/java/com/google/devtools/build/lib/bazel:BazelServer
Build time: Thu Apr 17 18:31:45 2025 (1744914705)
Build timestamp: 1744914705
Build timestamp as int: 1744914705
  

Rules_python version:

  
1.5.0
  

Anything else relevant?

This can be worked around by creating the include directory:

sudo  mkdir -p /Library/Python/3.9/include

[rickeylev]

In this case, what is the correct directory for includes?

Finding that directory is done by this script: https://github.com/bazel-contrib/rules_python/blob/main/python/private/get_local_runtime_info.py

It's probably somewhere in something. I don't have a mac, though, so its hard for me to test mac behavior.

[rickeylev]

Also, lets skip watch() if the directory doesn't exist. Includes missing isn't ideal, but they're only needed if C extensions are built.

[rickeylev]

I searched around a bit trying to see if there's some other way to get the include directory, but didn't come up with much. sysconfig.get_path("platinclude") might be an alternative value, but it's unclear.

Running python-config --includes should print the include paths as consumed by e.g. gcc. This appears to be a thin wrapper calling sysconfig.get_path("include"), though, so it'll probably return the same bad path.

Can you post the python -m sysconfig output? (link to output or file, please, the output will be very large).

I tried looking into how setup.py and setuptools builds and extension, but couldn't figure out how it locates the python headers.

Unfortunately, I don't have a mac, so can't easily debug this.

[erikkerber]

In this case, what is the correct directory for includes?

Python is in a supporting role in our practice so I can't say I'm necessarily an expert, but my assumption was that there just aren't any includes used in an Xcode-based Python toolchain. That it returns a value despite not having that folder seems like a appercut to anything that fails if including a existent directory.

Thanks for the quick turnaround 🙇🏼