Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement carton test command #42

Closed
2 of 5 tasks
MaxDesiatov opened this issue Jul 13, 2020 · 2 comments · Fixed by #173
Closed
2 of 5 tasks

Implement carton test command #42

MaxDesiatov opened this issue Jul 13, 2020 · 2 comments · Fixed by #173
Labels
enhancement New feature or request

Comments

@MaxDesiatov
Copy link
Collaborator

MaxDesiatov commented Jul 13, 2020

This should build tests, passing the resulting binary to a test runner after that. This could be implemented in a few separate steps:

  • use wasmer as a basic CLI test runner;
  • use the --enable-test-discovery flag to inject the JSON test observer so that test reports can be parsed (blocked by the lack of libIndexStore.dylib in recent snapshots);
  • consider using Node.js as another CLI runner option? Tracked separately in Node.js support as a destination environment #174
  • host the test bundle via HTTP on localhost with a browser loading an appropriate entrypoint, running the tests and reporting test results back to the HTTP server;
  • make wasmer an optional dependency. Tracked separately in Make Wasmer an optional dependency? #177

There's a question of how the target environment is selected. The wasmer option is what will be implemented first, and probably will be the default, but when browser environments are supported, should carton test launch tests in parallel in all possible environments? What should the appropriate option name be for selecting a specific test runner? --target (as in "the target environment"), --host (as in "the webassembly host"), --environment (probably the most unambiguous), --runner? I'm currently thinking of --environment, but all feedback is welcome as usual.

@MaxDesiatov MaxDesiatov added the enhancement New feature or request label Jul 13, 2020
MaxDesiatov added a commit that referenced this issue Jul 15, 2020
MaxDesiatov added a commit that referenced this issue Jul 15, 2020
Refines the `carton test` command as a continuation of #42 work.
MaxDesiatov added a commit that referenced this issue Oct 20, 2020
In the latest 5.3 toolchain all known issues with IndexStoreDB and SwiftPM support were resolved.

Resolves the second point of #42.

Thanks to this you no longer need to maintain `LinuxMain.swift` and `XCTestManifests.swift` files in your test suites, so these are deleted from the test app `Tests` directory.
MaxDesiatov added a commit that referenced this issue Dec 1, 2020
Resolves #42.

Also updates JavaScriptKit dependency to 0.9.0.

* Demangle and print Firefox stacktraces in terminal

* Update the entrypoints archive URL, rename file

* Add missing newline to `ProcessRunner.swift`

* Remove redundant `console.log` call from `dev.js`

* Detect destination env from `User-Agent` header

* Silence linter in tests where it can't be avoided

* Add support for basic testing in browsers

* Add a browser message on finish, shut down server

* Update JavaScriptKit to 0.9.0
@thecb4
Copy link
Contributor

thecb4 commented May 18, 2021

@MaxDesiatov a couple of questions on this. When we test. We're running a JS Script that tests against the wasm file. Is there any way to mimic UITests and reference "IDs" or names generated in the HTML? If there is a way for introspection and use by the test harness, then we shouldn't care about a browser. We could use pure swift, and test browsers as a secondary effort.

@MaxDesiatov
Copy link
Collaborator Author

Unfortunately, I don't think that's possible without parsing that HTML and maybe even having a complete DOM environment with WebAssembly support. I reckon that in some way it would be an effort comparable to maintaining our own small browser engine. I may be wrong and I'm happy to consider any viable options though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants