[cypress] test:e2e doesn't run in headless mode if project uses cli-plugin-typescript

[cexbrayat]

Version

3.1.1

Reproduction link

https://github.com/cexbrayat/vue-cypress-e2e-issue

Node and OS info

Node 8.11, npm 6.4, macOS 10.14

Steps to reproduce

npx @vue/[email protected] create vue-cypress-e2e-issue --inlinePreset '{"useConfigFiles": true,"plugins": {"@vue/cli-plugin-typescript": {"classComponent": true,"tsLint": true,"lintOn": ["save"]},"@vue/cli-plugin-unit-jest": {},"@vue/cli-plugin-e2e-cypress": {}}}'

# or git clone https://github.com/cexbrayat/vue-cypress-e2e-issue

cd vue-cypress-e2e-issue
npm run test:e2e -- --headless

What is expected?

When running npm run test:e2e -- --headless, the test should complete.

What is actually happening?

The task hangs and never runs the test nor fails or completes.

It logs:

vue-cypress-e2e-issue git:master ❯ npm run test:e2e -- --headless

> [email protected] test:e2e /Users/ced-pro/Code/test/vue/vue-cypress-e2e-issue
> vue-cli-service test:e2e "--headless"

 INFO  Starting e2e tests...
 INFO  Starting development server...
Starting type checking and linting service...
Using 1 worker with 2048MB memory limit


 DONE  Compiled successfully in 2012ms                                                                                            11:45:27 AM

Type checking and linting in progress...

  App running at:
  - Local:   http://localhost:8080/
  - Network: http://192.168.150.187:8080/

  App is served in production mode.
  Note this is for preview or E2E testing only.

No type errors found
No lint errors found
Version: typescript 3.1.6, tslint 5.11.0
Time: 2246ms

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:    3.1.1                                                                              │
  │ Browser:    Electron 59 (headless)                                                             │
  │ Specs:      1 found (test.js)                                                                  │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────

  Running: test.js...                                                                      (1 of 1)
Starting type checking and linting service...
Using 1 worker with 2048MB memory limit


 DONE  Compiled successfully in 2994ms

And hangs there indefinitely.

Note that it works well if not in headless mode.

---

The project uses cli-plugin-typescript. The e2e tests run fine if not in headless mode.

If TypeScript is not used, the e2e tests are fine in headless mode or not.

Workaround

Commenting:

module.exports = (on, config) => {
  on('file:preprocessor', webpack({
    // webpackOptions: require('@vue/cli-service/webpack.config'),
    watchOptions: {}
  }))

in tests/e2e/plugins/index.js fixes the issue.
The task then run fine in headless mode as well.

Might be related to #2783 but in this case I never have a complete run not even the first one.

[LinusBorg]

@sodatea This is another issue that the webpack preprocessor introduces

And as far as I understand it, we only added it to support aliases (that was the feature request mentioned in the commit, at least).

So I think we should consider dropping this thing if that's the only real selling point (or go in deep and figure out how to make this work reliably)

People who want to use it can add those 2-3 lines themselves, we could even keep them in the cypress config file commented out, with a hint to install the preprocessor package to make it work.

But right now, this preprocessor seems to cause more problems than it adds value

[cexbrayat]

@LinusBorg 👍 do you want me to submit a PR that removes the file:preprocessor from https://github.com/vuejs/vue-cli/blob/dev/packages/@vue/cli-plugin-e2e-cypress/generator/template/tests/e2e/plugins/index.js ? Should the webpack prepropcessor also be removed from https://github.com/vuejs/vue-cli/blob/dev/packages/%40vue/cli-plugin-e2e-cypress/generator/index.js or should we keep it?

[LinusBorg]

I'd like to get @sodatea's view on this first. If he's on the same page, you can submit a PR, sure.

[haoqunjiang]

@LinusBorg Yeah I think so. There are so many bug in the webpack preprocessor that I think we'd better make it opt in.

@cexbrayat Go ahead please. (Note: the lines in package.json should be removed as well)

IMO commenting them out could be a semver minor now that they only present in the generator folder. Any thoughts on this?

[LinusBorg]

IMO commenting them out could be a semver minor now that they only present in the generator folder. Any thoughts on this?

Hm, not sure.

Existing projects that currently have this preprocessor configured would have their builds fail after updating the cypress package, since fresh npm install calls wouldn't install the preprocessor dependency anymore.

We could keep the dependency so existing projects don't break, and remove it in the next major?

Or we take the position that removing this dependency is necessary to fix some serious bugs, and that solving the problem this change creates for existing projects (could not resolve module '@cypress/webpack-preprocessor') would be easy enough to fix to accept it as a side-effect of this bugfix.

IMHO it's not easy to decide what's a semver breaking change for a project like vue-cli

[cexbrayat]

@sodatea @LinusBorg Ok done in #2904

I kept the dependency for now. This is my first contribution to vue-cli but I contribute quite often to Angular CLI and this would be a breaking change to remove the dep, and would only be done in the next major (they have a 6 months cycle per major release).

Let me know how the PR can be improved (I also added a test for cypress and TS).

[haoqunjiang]

Existing projects that currently have this preprocessor configured would have their builds fail after updating the cypress package, since fresh npm install calls wouldn't install the preprocessor dependency anymore.

Actually this dependency is injected into package.json once the generator gets called so no existing project will break.

vue-cli/packages/@vue/cli-plugin-e2e-cypress/generator/index.js

Lines 7 to 10 in 662b1d3

	api.extendPackage({
	devDependencies: {
	'@cypress/webpack-preprocessor': '^3.0.0'
	},

[cexbrayat]

@sodatea 👌 I'm going to remove the dependency in the PR then.

[DorianGrey]

Correct if I'm wrong, but isn't the preprocessor essentially required for every test file that has to be transpiled in some way to work in a browser, regardless of babel or typescript doing so?

Personally, I've had way less issues with that preprocessor when using a very minimal webpack config instead of the full-featured one provided by default. It seems more like some webpack config options and resp. or plugins are causing those issues.

[haoqunjiang]

@DorianGrey
Cypress does preprocess files by default: https://docs.cypress.io/guides/guides/plugins-guide.html#Preprocessors
So a minimal webpack config may be unnecessary if you don't need alias support or some latest ES features…

[DorianGrey]

This doc entry mentions that:

[...] By default Cypress handles CoffeeScript and ES6 using babel [...] 

So typescript still requires manual configuration (and still has some additional issues trying to use it in this context, see e.g. this issue: #1350).
Thus, it might be favorable to retain that plugin, and only provide a minimal config for it - at least when using the cli-plugin-typescript as well.

[haoqunjiang]

@DorianGrey
Now it seems that @cypress/webpack-preprocessor is incompatible with our default typescript config, so it requires additional setup all the way along.

Considering there're so many issues caused by this plugin, we don't have the capacity to fix all the upstream issues or to maintain a separate config for cypress in parallel at the moment, I think it's better to make this plugin opt-in.

[DorianGrey]

Note that we're talking about a really small webpack configuration, like:

{
  resolve: {
    extensions: [".ts", ".tsx",]
  },
  module: {
    rules: [
     {
       test: /\.tsx?$/,
       loader: "ts-loader",
          options: { transpileOnly: false }
       }
   ]
}

(Had a small testing setup for this, which only has some problems with linting left: DorianGrey/vue-ts-playground#12)

Yet - I understand that it causes more problems than it solves, esp. since its directy dependencies are using fixed version, which may conflict with vue-cli itself or one of its other plugins. So it sounds definitely legit to remove it from the default setup.
Considering that - it might be easier to try the browserify preprocessor for this purpose (even though that means: more dependencies), or just use tsc directly (maybe with a small helper script to handle transpile & cleanup). At least both should cause less conflicts.

[LinusBorg]

@DorianGrey Would you mind opening a new issue as a feature request with these suggestions?

We will soon close this issue here once the fix is merged/published. So we should track the improvement of typescript support for Cypress in a feature issue.

[DorianGrey]

Sure - #2913.

[sava-vidakovic]

How can I run headless cypress with typescript?
The only solution that I see now is to have a separate config for cypress... 🤔

[Elif5757]

How can I run headless cypress with typescript?
The only solution that I see now is to have a separate config for cypress... 🤔

@sava-vidakovic I have the same Issue do you have the config ?