Parallel options

[grawk]

If opts.iterations is defined, mochaStream will pass along all test files as an array, and execute spawnMocha.add for each iteration array element.

This allows parallel behavior where each spawned process operates over the same file/fileset, but each process will differ based on the iteration level "opts" overrides to the top level "opts" object.

Only for those opts properties that it makes sense to override on a per-process basis is the iteration level override used.

[sebv]

Could you not mix code formatting and functional commits? It makes it makes it very had to read the changes.

[grawk]

Hi @sebv apologies for that. I have a habit of auto-formatting as I go. I tried to figure out how to change my editor rules to match the style you'd originally used, but couldn't. And I was interested to get your opinion on the substantive changes. When I can sit at my computer for a length of time I'll go carefully through and roll back the stylistic changes.

If you're inclined you can use w to evaluate the non-whitespace changes:
https://github.com/sebv/spawn-mocha-parallel/pull/2/files?w=1

[sebv]

This files related logic is a stream you need to do in your own customs stream, it affects concurrency. To optimize concurrency, you want the default to be splitting the jobs file by file.

[grawk]

This PR is to base concurrency on the iterations Array elements as an alternative to file concurrency. Are you concerned that waiting for the gulp.src stream to end before spawning the mocha processes is going to be a performance drag?

[sebv]

No I am concerned that your code is reverting what gulp.src does. If someone is using gulp.src, it is because they want file by file effect.

It'll be better not to use stream at all in this case, just use node-glob, promises and the spawnMocha object. (That also shows that in my current code, the error handling is at the wrong place.)

[grawk]

Thanks for taking the time to look at this. I apologize that I'm learning as I go here. I'll give
what you've said some thought and try the promises approach.

[grawk]

I added a new exported method to cover this use case. Instead of using promises I used the callback task pattern as mentioned in the gulp docs: https://github.com/gulpjs/gulp/blob/master/docs/API.md#accept-a-callback

There is duplicated code between mochaStream and mochaIteration. That can be cleaned up by extracting the error and end handlers from both and generalizing. But, I wanted to run this approach by you first.

[sebv]

Try this pattern, it's better for promise:

var Q = require('q'),
      exec =  Q.denodeify(require('child_process').exec),
      glob = Q.denodeify(require('glob;));

gulp.task('a task', function() {
   return exec('ls').spread(function(stdout, stderr) {
        ....
   });
})

gulp.task('a task', function() {
   return glob('**/*-specs').then(function(files) {
      ....
   });
})

Edit: Sorry might not be clear, I try to stay away from callbacks, they don't really scale in complexity, so I keep them to very low level stuff, and wrap them immediately and systematically in promises.

[grawk]

What do you think of SpawnMocha being its own module? If it is, and has the small changes for iterations then it can be consumed in any number of styles of module (gulp task, grunt task, promise API, callback API, stream API).

[sebv]

It already is, there are no dependencies on gulp.

[grawk]

Yes of course that's true. I've pared down the PR to be the essence of what is necessary to override the opts for individual spawned processes. The callback example remains in the gulpfile as an example of how to use overrides.