Is it possible to run the test in Travis?

[dmitshur]

If so, it'd be nice to do that and add a build status badge.

[mjohnson9]

I'm not sure. The current test uses QUnit, and I have no experience with QUnit and Travis-CI.

[dmitshur]

Can you please describe how one can run the tests? I've never run them myself because I don't know how.

[mjohnson9]

Of course!

• In the test directory, use Go to build server.go and run it.
• In the test/test directory, use GopherJS to build index.go -> index.js
• In a browser, navigate to localhost:3000

Your browser should show the QUnit test page and whether or not each test has passed.

I haven't messed with the tests in quite a while and I know that the dom package has undergone updates. It may not be compatible anymore.

[dmitshur]

Thanks. I was able to run the tests.

I had to fix up github.com/rusco/qunit, and I've also made improvements to the tests here. 2 PRs coming.

[dmitshur]

For posterity. Due to a change in GopherJS (done a while ago), it's not enough to do gopherjs build in test/test dir, need to do gopherjs build -o index.js or gopherjs build && mv test.js index.js.

It'd be really nice to have these run in CI because it's not as easy as doing go test $(go list ./... | grep -v /vendor/) in the terminal.

[mjohnson9]

I concur. But, don't we need a browser in order to test this? I don't believe Node has a standards compliant WebSocket implementation built-in.

Perhaps PhantomJS?

[flimzy]

I spent some time a couple months ago trying to get GopherJS tests to run in PhantomJS, and ran into several difficulties.

The TLDR version is basically: PhantomJS's JS implementation isn't entirely standard. Its error handling isn't consistent with V8, as one example, which means certain assumptions GopherJS makes simply don't hold true. As I recall, the feeling of the project at the time I brought it up was that it wasn't the responsibility of GopherJS to provide that sort of compatibility layer. I did have limited success in wrapping GopherJS's output with my own PhantomJS compatibility shim, but I never produced a complete working example, as I hit some other unresolved barriers.

One such issue is documented/discussed here: gopherjs/gopherjs#383

If there are others interested in running GopherJS tests in the browser (via PhantomJS, or some other means) I would love to see this happen, and would be happy to contribute some time toward the effort.

At the time I was trying, it seemed I felt I there was no other interest, and I wasn't willing to tackle all the complexities on my own.

[mjohnson9]

Came across this while browsing around for another project: http://swizec.com/blog/how-to-run-javascript-tests-in-chrome-on-travis/swizec/6647

[bep]

This runs for me in Travis:

https://github.com/bep/gr/blob/master/.travis.yml

[dmitshur]

Thanks @bep. Having a reference for how to install gopherjs in Travis is helpful.

However, it's not enough to resolve the testing for this package, because these tests require the use of a browser. Just gopherjs test in a terminal isn't enough.

[bep]

Just gopherjs test in a terminal isn't enough.

I understand that, but "open source" is built on small baby steps like these ... I don't know how to run a headless browser on Travis, but getting gopherjs + Node running seems like a good first step. And the goal for a good testing experience should be "go test ./... --with-browser" (or something) (the go tools should merge) and not some main methods wired up with qunit or whatever.

[bep]

I see these guys use chromium on Travis:

https://github.com/bep/react-router/blob/master/.travis.yml

[mjohnson9]

I had thought about this the other day and came across Selenium. Travis CI has a guide to using Sauce Labs for Selenium testing. Sauce Labs provides free services for OSS.

We could also do code quality checking in Travis.

https://docs.travis-ci.com/user/gui-and-headless-browsers/

Thoughts?

[dmitshur]

I'm not opposed, it would be very helpful. If you're able to set it up, I can help with review.

For code quality checking, do you mean running gofmt -d -s and go tool vet . (like I normally do in all my Go repositories) to make sure they don't spot any issues?

[mjohnson9]

For code quality checking, do you mean running gofmt -d -s and go tool vet . (like I normally do in all my Go repositories) to make sure they don't spot any issues?

Indeed. In fact, if you don't mind, I'll use your travis.yml as a starting place.

[dmitshur]

Please feel free, I'd be happy if it's a helpful resource!

[mjohnson9]

Should we have one Sauce Labs team for all of the GopherJS projects that need browser tests?

Should we write a more idiomatic library for running GopherJS tests on Sauce Labs?

I worry that I'm setting precedent with our hacky tests.

[mjohnson9]

Should we write a more idiomatic library for running GopherJS tests on Sauce Labs?

I just found gopherjs test -c, so all we'd need is a little bit of glue to get it to push its results back to the Sauce Labs system. From what I've seen so far, that bit will be simple.

[mjohnson9]

I'm thinking of writing a tool for running idiomatic Go tests in the browser. My plan so far is that it would do what "gopherjs test" does, but add in some generated code for parsing test results and displaying them in the browser.

So far, it looks like everyone parses stdout from the testing library to get the results. It wouldn't be too difficult to replace os.Stdout with a mock and parse it.

Of course, it would also provide the results in the format that Sauce Labs expects.

[dmitshur]

Thanks for doing the work to improve the status quo here, @nightexcessive. I don't have much bandwidth to look into Sauce Labs and all the details, but what you're describing at a high level sounds good to me so far. 👍 Feel free to take lead on this, I trust your judgement. I will try to look it over, but I don't want you to be blocked on me. Thanks!

[flimzy]

I'm thinking of writing a tool for running idiomatic Go tests in the browser. My plan so far is that it would do what "gopherjs test" does, but add in some generated code for parsing test results and displaying them in the browser.

This is a great idea, and something I spent several weeks working on about a year ago.The blocking issue I ran into was gopherjs/gopherjs#383 .

In short, it's easy to run tests in the browser. It's also easy to capture their output and put them in a web page.

What's not currently possible is detect when a test has finished. This makes it effectively impossible to run an entire test suite in any automated fashion in the browser. Even using PhantomJS, or some other headless solution, there's no way to know when a test has finished.

If you can devise a work-around, I think that would be amazing. But I suspect it will require changes to the GopherJS runtime. And so far, I've detected very little traction when suggesting this. :)

[johanbrandhorst]

Just discovered this discussion, I'm currently running automated tests using chrome and firefox for my protobuf project (github.com/johanbrandhorst/protobuf). I've detailed the work in https://jbrandhorst.com/post/gopherjs-integration-tests/ and https://jbrandhorst.com/post/circleci-docker-compose/. The tests were actually based on the tests in this repo originally. Basically I'm letting Qunit output the tests results then I use Agouti to parse the DOM to find any failures.