Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Absolute path exec handling #10857

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

Athishpranav2003
Copy link

Addressing #10536

@Athishpranav2003
Copy link
Author

Athishpranav2003 commented Aug 28, 2024

@Khady @rgrinberg
please check the PR. I have basically stripped the common prefix in the abs path to figure the correct path relatively

Tho this doesnt work indeed if its run from a different dune project(since we cannot resolve the existing builds from different dune project)

@Athishpranav2003 Athishpranav2003 changed the title Handled Abs path exec handling Absolute path exec handling Aug 28, 2024
@Athishpranav2003
Copy link
Author

For now i have made the code to preserve the behavior in case of absolute path(where build is not invoked on exec). But i feel it should be same for both relative and absolute. Open for suggestions(the change is just 2 line change only)

otherlibs/stdune/src/path.ml Outdated Show resolved Hide resolved
bin/exec.ml Outdated
Comment on lines 235 to 246
Build_system.file_exists path
>>= (function
| true -> Memo.return (Some path)
| false ->
if not (Filename.check_suffix prog ".exe")
then Memo.return None
else (
let path = Path.extend_basename path ~suffix:".exe" in
Build_system.file_exists path
>>| function
| true -> Some path
| false -> None))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what this code is doing exactly, but this might need to support the .bc extension too?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't cause any regression so this would a very new thing. Current code has no regression from previous one

@Khady
Copy link
Contributor

Khady commented Aug 28, 2024

thanks for helping with this issue. I'll try to compile and test

@Athishpranav2003
Copy link
Author

@Khady do you need help in testing this?

@Khady
Copy link
Contributor

Khady commented Aug 30, 2024

looks like it works for a program in the current directory or a child directory, but not pointing to a parent directory

/home/me/Code/github/dune/_build/install/default/bin/dune is dune built from your branch

dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune build
dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec ./demo.exe
hello world                      
dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec demo
hello world                      
dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $PWD/./demo.exe
hello world                       
dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $PWD/demo.exe
hello world                       
dune-exec-abs-path$ cd lib
dune-exec-abs-path/lib$ /home/me/Code/github/dune/_build/install/default/bin/dune exec ../demo.exe
Entering directory '/home/me/Code/github/dune-exec-abs-path'
Leaving directory '/home/me/Code/github/dune-exec-abs-path'
hello world                      
dune-exec-abs-path/lib$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $PWD/../demo.exe
Entering directory '/home/me/Code/github/dune-exec-abs-path'
Error: Program '/home/me/Code/github/dune-exec-abs-path/lib/../demo.exe' not
found!
dune-exec-abs-path/lib$ realpath /home/me/Code/github/dune-exec-abs-path/lib/../demo.exe
/home/me/Code/github/dune-exec-abs-path/demo.exe
dune-exec-abs-path/lib$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $(realpath $PWD/../demo.exe)
Entering directory '/home/me/Code/github/dune-exec-abs-path'
Error: Program '/home/me/Code/github/dune-exec-abs-path/demo.exe' not found!

@Athishpranav2003
Copy link
Author

Athishpranav2003 commented Aug 30, 2024

looks like it works for a program in the current directory or a child directory, but not pointing to a parent directory

/home/me/Code/github/dune/_build/install/default/bin/dune is dune built from your branch

dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune build
dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec ./demo.exe
hello world                      
dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec demo
hello world                      
dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $PWD/./demo.exe
hello world                       
dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $PWD/demo.exe
hello world                       
dune-exec-abs-path$ cd lib
dune-exec-abs-path/lib$ /home/me/Code/github/dune/_build/install/default/bin/dune exec ../demo.exe
Entering directory '/home/me/Code/github/dune-exec-abs-path'
Leaving directory '/home/me/Code/github/dune-exec-abs-path'
hello world                      
dune-exec-abs-path/lib$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $PWD/../demo.exe
Entering directory '/home/me/Code/github/dune-exec-abs-path'
Error: Program '/home/me/Code/github/dune-exec-abs-path/lib/../demo.exe' not
found!
dune-exec-abs-path/lib$ realpath /home/me/Code/github/dune-exec-abs-path/lib/../demo.exe
/home/me/Code/github/dune-exec-abs-path/demo.exe
dune-exec-abs-path/lib$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $(realpath $PWD/../demo.exe)
Entering directory '/home/me/Code/github/dune-exec-abs-path'
Error: Program '/home/me/Code/github/dune-exec-abs-path/demo.exe' not found!

@Khady check this comment #10857 (comment)

Is the parent directory also in same project?
If thats the case then its a edgecase i didnt handle(just require minor changes)

@Khady
Copy link
Contributor

Khady commented Aug 30, 2024

yes it is in the same project. I'm just going down with cd lib in https://github.com/Khady/dune-exec-abs-path/tree/main/lib

@Athishpranav2003
Copy link
Author

@Khady i will make the changes and push it tonight. Will ping once done

@Athishpranav2003
Copy link
Author

Athishpranav2003 commented Aug 30, 2024

@Khady can you check it now?
Still following wont work

(° ͜ʖ͡°)╭∩╮=>../../dune/dune.exe exec /home/aggressive_racer1/projects/dune-exec-abs-path/demo.exe     Fri 30 Aug 2024 11:43:56 PM IST
Entering directory '/home/aggressive_racer1/projects/dune-exec-abs-path'
Error: Program '/home/aggressive_racer1/projects/dune-exec-abs-path/demo.exe'
not found!

The reason is its dependent on the design of dune in itself.

@Khady
Copy link
Contributor

Khady commented Aug 30, 2024

Now I'm facing a problem where if I use exec of absolute path it does nothing (see line 2) unless I've built first (see line 20 and 22). The behavior of the last 2 commands is also surprising, something different happens depending on if the path is resolved or not.

🌿main *|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path$ dune clean
🌿main *|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $PWD/demo.exe
🌿main *|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path$ cat _build/log
# /home/me/Code/github/dune/_build/install/default/bin/dune exec /home/me/Code/github/dune-exec-abs-path/demo.exe
# OCAMLPARAM: unset
# Shared cache: enabled
# Shared cache location: /home/me/.cache/dune/db
# Workspace root: /home/me/Code/github/dune-exec-abs-path
# Auto-detected concurrency: 256
# Dune context:
#  { name = "default"
#  ; kind = "default"
#  ; profile = Dev
#  ; merlin = true
#  ; fdo_target_exe = None
#  ; build_dir = In_build_dir "default"
#  ; instrument_with = []
#  }
$ /home/me/.opam/5.2.0/bin/ocamlc.opt -config > /tmp/dune_71fa9a_output
🌿main *|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec ./demo.exe
hello world?                     
🌿main *|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $PWD/demo.exe
hello world?                      
🌿main *|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path$ cd lib
🌿main *|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path/lib$ dune clean
Entering directory '/home/me/Code/github/dune-exec-abs-path'
🌿main *|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path/lib$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $PWD/../demo.exe
Entering directory '/home/me/Code/github/dune-exec-abs-path'
Leaving directory '/home/me/Code/github/dune-exec-abs-path'
🌿main *|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path/lib$ /home/me/Code/github/dune/_build/install/default/bin/dune exec $(realpath $PWD/../demo.exe)
Entering directory '/home/me/Code/github/dune-exec-abs-path'
Error: Program '/home/me/Code/github/dune-exec-abs-path/demo.exe' not found!

About the code, it seems that you don't have auto formatting setup in your workflow. So you'll need to run make fmt (after maybe make install-ocamlformat) to get the desired style.

@Athishpranav2003
Copy link
Author

@Khady so in the previous version as well there is no build when trying to use absolute path. And for the last 2 commands its not supported since for the executable to be found we need to change to root build dir which is what happening when we include a ".." in the code. Manually it finds and change to the parent build dir

@Khady
Copy link
Contributor

Khady commented Aug 31, 2024

I feel like it should be possible to compute the resolved real path in ocaml and make it to work. But I don’t have time to investigate that for now. Will see if I can do it in the following weeks.

@Athishpranav2003
Copy link
Author

Athishpranav2003 commented Aug 31, 2024

@Khady sure
i will also try to look at the program workflow. As far as i know we are not looking at the right function since the problem is in a higher level.

Only thing we are missing now is using the project info from paths excluding basename ryt?

@Athishpranav2003
Copy link
Author

Athishpranav2003 commented Sep 3, 2024

@Khady i guess i resolved the problem you were mentioning. The current approach changes the root directory to directory mentioned in path

 ~/p/dune-exec-abs-path   …  
(° ͜ʖ͡°)╭∩╮=>/home/aggressive_racer1/projects/dune/dune.exe exec ./demo.exe                            Tue 03 Sep 2024 02:12:31 PM IST
hello world                      
 ~/p/dune-exec-abs-path   …  
(° ͜ʖ͡°)╭∩╮=>/home/aggressive_racer1/projects/dune/dune.exe exec /home/aggressive_racer1/projects/dune-exec-abs-path/demo.exe
hello world                     
 ~/p/dune-exec-abs-path   …  
(° ͜ʖ͡°)╭∩╮=>/home/aggressive_racer1/projects/dune/dune.exe exec /home/aggressive_racer1/projects/dune-exec-abs-path/lib/../demo.exe
hello world                     
 ~/p/dune-exec-abs-path   …  
(° ͜ʖ͡°)╭∩╮=>cd lib/                                                                                   Tue 03 Sep 2024 02:12:40 PM IST
 ~/p/dune-exec-abs-path   …  lib  
(° ͜ʖ͡°)╭∩╮=>/home/aggressive_racer1/projects/dune/dune.exe exec ../demo.exe                           Tue 03 Sep 2024 02:12:42 PM IST
Entering directory '/home/aggressive_racer1/projects/dune-exec-abs-path'
Leaving directory '/home/aggressive_racer1/projects/dune-exec-abs-path'
hello world                      
 ~/p/dune-exec-abs-path   …  lib  
(° ͜ʖ͡°)╭∩╮=>/home/aggressive_racer1/projects/dune/dune.exe exec $PWD/../demo.exe                      Tue 03 Sep 2024 02:12:48 PM IST
Entering directory '/home/aggressive_racer1/projects/dune-exec-abs-path'
Leaving directory '/home/aggressive_racer1/projects/dune-exec-abs-path'
hello world            

Could you please test this updated one?

@Athishpranav2003 Athishpranav2003 force-pushed the handle-absolute-path branch 3 times, most recently from 3dd412b to 6640a06 Compare September 3, 2024 08:52
Signed-off-by: Athish Pranav D <[email protected]>
Signed-off-by: Athish Pranav D <[email protected]>
Signed-off-by: Athish Pranav D <[email protected]>
@Athishpranav2003 Athishpranav2003 force-pushed the handle-absolute-path branch 3 times, most recently from 5ad1169 to b6c7da6 Compare September 3, 2024 12:53
@Athishpranav2003
Copy link
Author

@Khady can you trigger the CI builds?

@Athishpranav2003 Athishpranav2003 force-pushed the handle-absolute-path branch 4 times, most recently from 31957c1 to ed1866f Compare September 3, 2024 13:01
@Khady
Copy link
Contributor

Khady commented Sep 3, 2024

@Khady can you trigger the CI builds?

I am only a contributor to dune, not a maintainer, I don't have the rights to do so (nor to accept and merge your work).

@Athishpranav2003
Copy link
Author

Athishpranav2003 commented Sep 3, 2024

Oh I see
Fine

Can you please pull it and check locally?
Once you confirm I will ping the maintainers

@Athishpranav2003
Copy link
Author

Athishpranav2003 commented Sep 4, 2024

@Khady i want to debug the failing builds but running make test locally
Error log

https://gist.github.com/Athishpranav2003/0dce8e130add559644135913220f1ef3

@Athishpranav2003
Copy link
Author

@rgrinberg could you please help me with the make issue?

@Khady
Copy link
Contributor

Khady commented Sep 4, 2024

I think that you need to install the test dependencies

opam install . --deps-only --with-test --with-dev-setup

I don't know if strace is the binary to trace system calls. If it is then it should be installed with a package manager on linux. Same for the node binary (nodejs).

@Athishpranav2003
Copy link
Author

@Khady other errors are resolved except ppx_expect.common . Any idea for that alone?

@Khady
Copy link
Contributor

Khady commented Sep 5, 2024

This one would be handled by the ppx_expect opam package I suppose

dune/dune.opam

Line 58 in d9b0e2e

"ppx_expect" { with-dev-setup & >= "v0.17" & os != "win32" }

opam install ppx_expect

@Athishpranav2003
Copy link
Author

I have it installed but still the error persists

@Khady
Copy link
Contributor

Khady commented Sep 5, 2024

Is it still the same error as you shared before?

This is what I have installed

opam-version: "2.0"
compiler: [
  "base-bigarray.base"
  "base-threads.base"
  "base-unix.base"
  "host-arch-x86_64.1"
  "host-system-other.1"
  "ocaml.4.14.1"
  "ocaml-base-compiler.4.14.1"
  "ocaml-config.2"
  "ocaml-options-vanilla.1"
]
roots: [
  "ocaml-base-compiler.4.14.1" "ocaml-lsp-server.1.18.0" "ocamlformat.0.26.2"
]
installed: [
  "astring.0.8.5"
  "base.v0.16.3"
  "base-bigarray.base"
  "base-bytes.base"
  "base-threads.base"
  "base-unix.base"
  "bigarray-compat.1.1.0"
  "camlp-streams.5.0.1"
  "chrome-trace.3.16.0"
  "cinaps.v0.15.1"
  "cmdliner.1.3.0"
  "conf-bash.1"
  "cppo.1.6.9"
  "crunch.3.3.1"
  "csexp.1.5.2"
  "ctypes.0.23.0"
  "dune.3.16.0"
  "dune-build-info.3.16.0"
  "dune-configurator.3.16.0"
  "dune-rpc.3.16.0"
  "dyn.3.16.0"
  "either.1.0.0"
  "fiber.3.7.0"
  "fix.20230505"
  "fmt.0.9.0"
  "fpath.0.7.3"
  "gen.1.1"
  "host-arch-x86_64.1"
  "host-system-other.1"
  "integers.0.7.0"
  "jane-street-headers.v0.16.0"
  "js_of_ocaml.5.8.2"
  "js_of_ocaml-compiler.5.8.2"
  "jsonrpc.1.18.0"
  "jst-config.v0.16.0"
  "lambda-term.3.3.2"
  "logs.0.7.0"
  "lsp.1.18.0"
  "lwt.5.7.0"
  "lwt_react.1.2.0"
  "mdx.2.4.1"
  "melange.4.0.0-414"
  "menhir.20240715"
  "menhirCST.20240715"
  "menhirLib.20240715"
  "menhirSdk.20240715"
  "merlin-lib.4.16-414"
  "mew.0.1.0"
  "mew_vi.0.5.0"
  "ocaml.4.14.1"
  "ocaml-base-compiler.4.14.1"
  "ocaml-compiler-libs.v0.12.4"
  "ocaml-config.2"
  "ocaml-lsp-server.1.18.0"
  "ocaml-options-vanilla.1"
  "ocaml-version.3.6.8"
  "ocamlbuild.0.15.0"
  "ocamlc-loc.3.16.0"
  "ocamlfind.1.9.6"
  "ocamlformat.0.26.2"
  "ocamlformat-lib.0.26.2"
  "ocamlformat-rpc-lib.0.26.2"
  "ocp-indent.1.8.1"
  "ocplib-endian.1.2"
  "odoc.2.4.2"
  "odoc-parser.2.4.2"
  "ordering.3.16.0"
  "pp.1.2.0"
  "ppx_assert.v0.16.0"
  "ppx_base.v0.16.0"
  "ppx_cold.v0.16.0"
  "ppx_compare.v0.16.0"
  "ppx_derivers.1.2.1"
  "ppx_enumerate.v0.16.0"
  "ppx_expect.v0.16.0"
  "ppx_globalize.v0.16.0"
  "ppx_hash.v0.16.0"
  "ppx_here.v0.16.0"
  "ppx_inline_test.v0.16.1"
  "ppx_optcomp.v0.16.0"
  "ppx_sexp_conv.v0.16.0"
  "ppx_yojson_conv_lib.v0.16.0"
  "ppxlib.0.33.0"
  "ptime.1.1.0"
  "re.1.11.0"
  "react.1.2.2"
  "result.1.5"
  "sedlex.3.2"
  "seq.base"
  "sexplib0.v0.16.0"
  "spawn.v0.15.1"
  "stdio.v0.16.0"
  "stdlib-shims.0.3.0"
  "stdune.3.16.0"
  "time_now.v0.16.0"
  "topkg.1.0.7"
  "trie.1.0.0"
  "tyxml.4.6.0"
  "uchar.0.0.2"
  "utop.2.14.0"
  "uucp.15.1.0"
  "uuseg.15.1.0"
  "uutf.1.0.3"
  "xdg.3.16.0"
  "yojson.2.2.2"
  "zed.3.2.3"
]

@Athishpranav2003
Copy link
Author

@Khady

(° ͜ʖ͡°)╭∩╮=>opam install ppx_expect                                                          4271ms  Thu 05 Sep 2024 07:42:20 PM IST
[NOTE] Package ppx_expect is already installed (current version is v0.17.0).
 ~/p/dune   handle-absolute-path $  
(° ͜ʖ͡°)╭∩╮=>make test                                                                        1954ms  Thu 05 Sep 2024 07:42:41 PM IST
./_boot/dune.exe runtest
File "otherlibs/stdune/src/format.mli", line 1, characters 0-21:
1 | val pp_infinity : int
    ^^^^^^^^^^^^^^^^^^^^^
Error (warning 32 [unused-value-declaration]): unused value pp_infinity.
File "test/expect-tests/dune", line 72, characters 2-19:
72 |   ppx_expect.common
       ^^^^^^^^^^^^^^^^^
Error: Library "ppx_expect.common" not found.
-> required by library "persistent_tests" in _build/default/test/expect-tests
-> required by
   _build/default/test/expect-tests/.persistent_tests.inline-tests/inline_test_runner_persistent_tests.exe
-> required by alias test/expect-tests/runtest in test/expect-tests/dune:58

@Athishpranav2003
Copy link
Author

@Khady did you try running the PR locally? Was it working ?
(The problem with CI build is different, i will check it later but need to resolve problem with my own local testing env)

@Khady
Copy link
Contributor

Khady commented Sep 16, 2024

Looks like there is still a problem. The execution with an absolute path only works if it was done with a relative path first.

🌿main|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune clean
🌿main|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec /home/me/Code/github/dune-exec-abs-path/demo.exe
Error: Program '/home/me/Code/github/dune-exec-abs-path/demo.exe' not found!
🌿main|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec ./demo.exe
hello world                       
🌿main|u= 🐫5.2.0 sg:~/Code/github/dune-exec-abs-path$ /home/me/Code/github/dune/_build/install/default/bin/dune exec /home/me/Code/github/dune-exec-abs-path/demo.exe
hello world     

The other tests looked fine, but I didn't spend too much time trying.

@Athishpranav2003
Copy link
Author

Athishpranav2003 commented Sep 16, 2024

@Khady so basically it needs to build before run and the build option was not present before in abs path(i also followed same to preserve behaviour but will change it now). I can add that as well now and push it. Give me an hour

Signed-off-by: Athish Pranav D <[email protected]>
@Athishpranav2003
Copy link
Author

@Khady now i have changed the program to run build even when passing absolute path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants