Feature/file based filter todo1

[xinzhengzhang]

This is a pr that solves part of todo.

Fix timeout not working due to scope issue

Normalize input file to relative path

The following conditions were manually tested

Source file

srcs/app/xx/x.h
bazel info execution_root/srcs/app/xx/x.h
bazel info workspace/srcs/app/xx/x.h

external/zlib1.2.13/crc32.h
bazel info output_base/external/zlib1.2.13/crc32.h
bazel info workspace/external/zlib~1.2.13/crc32.h

Generated file

bazel-out//bin/external/_mainrepo_deps_extbapis/xx/Ad.blrpc.h
bazel info execution_rootbazel-out//bin/external/_mainrepo_deps_extbapis/xx/Ad.blrpc.h

bazel-out//bin/srcs/app/xx/x_generate.h
bazel info execution_root/bazel-out//bin/srcs/app/xx/x_generate.h

System file

/Applications/Xcode.app/Contents//Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk/System/Library/Frameworks/UIKit.framework/Headers/UIKit.h

Handling cases where input files are intermediates

Looks like there's a bug in bazel aquery. Please help to see if there is a problem with the way I use it. If it really exists, I may only be able to query the result once, and then merge it and pass it to aquery.

# Step to reproduce

# Both somepath and allpath are the same
query_statment = "somepath(target, subtarget)"
bazel query "$query_statment"
bazel aquery "$query_statment" // The log says found 2 targets, but the query just now is much larger than 2, and no actions to be printed

Ex.
https://github.com/bazelbuild/rules_apple

bazel aquery "allpaths(//examples/ios/HelloWorld:HelloWorld, //examples/ios/HelloWorld:Sources)"

---
# This is work

bazel aquery "//examples/ios/HelloWorld:HelloWorld+//examples/ios/HelloWorld:Sources"

[cpsauer]

Thanks again for moving this forward! Quick inline comments below.

On aquery: I'm afraid I'm either not understanding or not seeing the bug you're talking about. As an example, I was running bazel aquery "mnemonic('(Objc|Cpp)Compile', allpaths(//API/C:iOS, let v = deps(//API/C:iOS) in attr(hdrs, 'DoubleBuffer.hpp', \$v) + attr(srcs, 'DoubleBuffer.hpp', \$v)))" --noinclude_aspects and was getting the actions I'd expect

[cpsauer]

Really appreciate your tackling. But on a quick read, I think this part could use some polish along the lines we'd discussed.

A couple things that jump out:

We should probably still have an error for the case that's in-output-base but not external or execroot.
...
(Remember the backwards compatible relative_to helper, as above)
(I'm sure there's more I missed on a quick pass; I'm counting on you to experiment and track them down, and give it some solid manual testing.)

Probably also worth using subprocess.run instead of subprocess.check_output, just because of deprecation and to match the others? (Also, there's the encoding issue.)

[xinzhengzhang]

I would like to ask if there are any other problems with the processing of paths besides subprocess.check_output? At present, I have modified these 3 calls, but I have not changed other places in the file

[xinzhengzhang]

Thanks again for moving this forward! Quick inline comments below.

On aquery: I'm afraid I'm either not understanding or not seeing the bug you're talking about. As an example, I was running bazel aquery "mnemonic('(Objc|Cpp)Compile', allpaths(//API/C:iOS, let v = deps(//API/C:iOS) in attr(hdrs, 'DoubleBuffer.hpp', \$v) + attr(srcs, 'DoubleBuffer.hpp', \$v)))" --noinclude_aspects and was getting the actions I'd expect

There is no problem to aquery like this. I went through a separate aquery to get the target who produced the input file.
Then get the dependency tree through allpath to get the compile commands.

Reproduce

git clone [email protected]:bazelbuild/rules_apple.git && cd rules_apple

# //examples/ios/HelloWorld:Sources was queried by `bazel query "let v = deps(//examples/ios/HelloWorld:HelloWorld) in attr(srcs, 'AppDelegate.m', \$v)"`

bazel aquery "allpaths(//examples/ios/HelloWorld:HelloWorld, '//examples/ios/HelloWorld:Sources')"
    INFO: Analyzed 2 targets (0 packages loaded, 0 targets configured).
    INFO: Found 2 targets...
    INFO: Empty query results
    INFO: Elapsed time: 0.052s, Critical Path: 0.00s
    INFO: 0 processes.
    INFO: Build completed successfully, 0 total actions

# getting the actions expect
# bazel aquery "allpaths(//examples/ios/HelloWorld:HelloWorld, let v = deps(//examples/ios/HelloWorld:HelloWorld) in attr(srcs, 'AppDelegate.m', \$v))"

[cpsauer]

Minor, but as in the TODO, we should really be quoting targets everywhere!

[xinzhengzhang]

I also quote in other queries. But I found another problem in bazel query.
Because of the bug mentioned above I tried to use another query statement to return the middle_target_candidate.
f"filter({re.escape(candidate}$)" but when the label starts with @ it will return empty realist... so I just remove the first @ symbol for workaround 6b098e8#diff-a1d7061df7c566a1f7656624ec608ad53dd3aff7a7789b9b6e4866a3b1616042R1016

[cpsauer]

Re aquery: Huh! Really not obvious to me what's going on there. Would you be down to report as an issue on bazelbuild/bazel and see if they help or confirm that it's a bug?
(I played around with it for a little bit, but couldn't figure out what was wrong. My initial hypothesis was that it might not be working because of the configuring transitions in the HelloWorld's ios_application, but the cquery results were non-empty, and --universe_scope=//examples/ios/HelloWorld:HelloWorld didn't help.

By the way: Are you also seeing super slow GitHub downloads these past couple weeks?

[xinzhengzhang]

By the way: Are you also seeing super slow GitHub downloads these past couple weeks?

In my country like github, these are all blocked by default so I use vpn. But I did find that github has been hung up a few times recently and in most cases, it can be restored in half an hour.(I can't even tell if it's a github problem or my ladder problem) In terms of speed, because it is very slow, we have built mirrors for enterprise, so we can't perceive it

[cpsauer]

Interesting! Thanks. Usually it's blazingly fast here--just not these last couple weeks.