-
Notifications
You must be signed in to change notification settings - Fork 9
Update testing commands #97
Changes from 2 commits
227ec7a
5df631f
ffa1b47
61165f1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,45 +13,40 @@ function run(type) { | |
server.start(); | ||
} | ||
|
||
function getConfig(type) { | ||
switch (type) { | ||
case 'headless': | ||
case 'hl': | ||
case 'h': | ||
return getHeadlessConfig(); | ||
case 'all': | ||
case 'a': | ||
return getAllConfig(); | ||
case 'watch': | ||
case 'w': | ||
return getWatchConfig(); | ||
default: | ||
return getSingleConfig(); | ||
} | ||
} | ||
|
||
function getSingleConfig() { | ||
let config = getHeadlessConfig(); | ||
|
||
config.singleRun = true; | ||
function getConfig(options) { | ||
let config = getAllConfig(options.watch); | ||
config.browsers = options.browsers; | ||
config.singleRun = !options.watch; | ||
|
||
return config; | ||
return config; | ||
} | ||
|
||
function getHeadlessConfig() { | ||
let config = getAllConfig(); | ||
|
||
config.browsers = ['PhantomJS']; | ||
|
||
return config; | ||
} | ||
|
||
function getWatchConfig() { | ||
let config = getAllConfig(true); | ||
|
||
config.browsers = ['Chrome']; | ||
|
||
return config; | ||
function parseOptions(args){ | ||
let options = {}; | ||
|
||
args.forEach((arg, index)=>{ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The changes that would help here are to just follow the coding conventions/style I've been using.
Admittedly, I need to do a better job of managing the styling via ESLint or Prettier, but I never got around to it |
||
let value = arg.match(/^(\w+)/)[1]; | ||
switch(value){ //there are probably libraries that parse cmd line arguments... | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. there are, but when I started this, the intent was to keep it as simple & lightweight as possible, as you might see, there aren't a ton of external includes from Librarian code There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wanted to mention, Librarian has its own focused utility for handling options: Here's an example of how its used: Again, totally understand there are libraries that do this, but when I started Librarian I was just having fun and rolled my own--the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh! I didn't realize you had code to handle that. The original code wasn't using it either and is the only reason I suggested finding a library. I'll see what I can do. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok now I've tried to incorporate options.js and my inexperience with writing developer tools may be biting me in the butt. This file (test.js) gets moved to the host library while options.js stays in the angular-librarian folder. This gives it two different requires paths: one to allow the library to build ('../../../../tools/utilities') and one from the host library ('angular-librarian/tools/utilities'). Is there something in the initialization that will resolve this automatically or am I supposed to pretend that test.js is in a host library and use that path? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This file, There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Now I've attempted to use the |
||
case "headless": | ||
case "hl": | ||
case "h": | ||
options.browsers = ["PhantomJS"]; | ||
break; | ||
case "watch": | ||
case "w": | ||
options.watch = true; | ||
break; | ||
case "browsers": | ||
case "b": | ||
let browsers = arg.match(/\w+=([\w,]*)/i); | ||
options.browsers = (browsers && !options.browsers) ? browsers[1].split(',') : options.browsers; | ||
break; | ||
//the 'all' option did not modify the browser options and it did not change the watch option. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. good point on this one |
||
//therefore removing it will not break current setups. Unless the developer removed all browsers | ||
//from the base karma.config.js file | ||
} | ||
}); | ||
return options; | ||
} | ||
|
||
const getAllConfig = (watch) => ({ | ||
|
@@ -62,5 +57,7 @@ const getAllConfig = (watch) => ({ | |
module.exports = run; | ||
|
||
if (!module.parent) { | ||
run(process.argv[2]); | ||
//skip the first two args (exe and script) and grab all options that start with a 'word' | ||
let optionArgs = process.argv.filter((value, index) => index > 1 && value.match(/^\w+/)); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You might be able to combine this with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure if you saw my comments about |
||
run(parseOptions(optionArgs)); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The default option needs to remain headless, as the
publish
command leveragesnpm test
to run. Part of the reason for the defaults, also is to provide convention to developers over configuration. By providing the defaults, we keep users from having to extend karma if they don't want to.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm starting to see a conflict that I'm not sure how to reconcile. So I'll present that and we can decide how to proceed.
When I began trying to understand how client-side testing should should work, PhantomJS was there to help fill that gap needed for headless testing. But because it's not an actual browser, some implementations didn't work as expected in real browsers. Since then, browser makers have added facilities to allow developers to use actual browsers during tests which also gave rise to Selenium Grid providers like BrowserStack, SauceLabs and others. Most, if not all offer a free version of their service of some kind. This further pushes the need to use PhantomJS in the background.
This is where I was when I suggested these changes. I actually use SauceLabs for my project and don't want to use PhantomJS at all on my CI server in case something PhantomJS doesn't support well fails. (In my case PhantomJS is fine but I was thinking of the broader community). At this time, there isn't a way to only use the browsers I configure in the Karma config. We're forced to use PhantomJS if we don't have any parameters or if we specify headless.
Now, when angular-librarian creates a new project for a developer, it sets the browsers property to Chrome and PhantomJS. Default as I've written will run these two browsers if the developers doesn't do anything which fulfills the convention over configuration issue you presented but exposes a different problem with the publish.
I also use TravisCI since it was free with GitHub useage. CI systems usually have a few phases like, checkout, build, test, and "post testing". Typically, after testing succeeds, you may want to publish your library and that's where the "post testing" phase comes in. The current setup will cause another test to occur just because I'm publishing. It seems like some of the decisions made for the angular-librarian workflow were based on a developers doing everything from their computer. I'm not actually saying that was how it was done but I had to heavily modify some of the files angular-librarian generated to get it to allow the CI server to work as I expected it to.
angular-librarian should ideally make it easy to handle both situations. Since I'm familiar with the CI half, I've been trying to add that functionality to it. Maybe we should look at these changes more holistically to see how it fits in overall.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I think, perhaps, we can make it so that if no browsers are provided, either via the command line or by a config file, we fallback to the defaults of PhantomJS or Chrome. Does that sound ok to you?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok just so I'm not making assumptions about what you're saying I'm going to paraphrase what I believe you are saying. angular-librarian will still by default place Chrome and PhantomJS in the browsers property. The default testing option of not specifying any parameters will provide the new functionality of this PR, which is to run the browsers specified in the karma file. But if the developer has purposefully removed all browsers from the karma file, the tool will attempt to at the minimum use PhantomJS or Chrome. Is my assessment correct or did I miss anything? If this is correct, I'd like to suggest just using PhantomJS just in case a CI server is involved, since CI servers generally don't have browsers installed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Glad you asked to clarify! Here's the scenarios I'm thinking of...(note that
ngl test
is the same as runningnpm test
)User runs the following:
npm test
Since no browser overrides have been provided, it falls back to PhantomJS
User provides a command-line argument for
ChromeHeadless
:npm test --browsers=ChromeHeadless
Since the custom config provides an override, tests are run with
ChromeHeadless
User provides a command-line argument for Chrome & Firefox for a watch
npm test watch --browsers=Chrome,Firefox
Both Chrome & Firefox are opened by Karma and files are watched
However, does this cover your use case for CI testing? I'm wondering if there needs to be a way of specifying browsers to fallback to as either a custom config or on project init.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, I really want to thank you for the effort you've already put in between discussions and actual code!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, no problem but I have to admit helping is slightly selfish on my part. I'm hoping that the library can get to the point where I won't have to do a lot of work every time there's an update. :) lol.
Anyway, I think you're over thinking this a bit. The karma file should be the source of truth for configuration. If a developer is running Karma tests, there's supposed to be browsers there for the tests to use. The override is for when you want to use a sub or super set of those browsers. In my specific case, my karma file has 10 browsers for SauceLabs testing. When I run the testing, I want those browsers to be used and not added to or overridden with PhantomJS.
On the other hand, if the karma file itself specifies no browsers by the time the library is ready to start Karma THEN we fallback to PhantomJS because a Karma test with no browser is completely useless.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I'm ok with the proposal if there's no browsers specified always falling back to PhantomJS.