Skip to content
This repository has been archived by the owner on Dec 7, 2019. It is now read-only.

Make sure Composer marks test run as failed if test runner throws error before actual test run. #89

Open
artem-zinnatullin opened this issue Jul 27, 2017 · 0 comments
Assignees
Milestone

Comments

@artem-zinnatullin
Copy link
Collaborator

AndroidJUnitRunner from test support library v1.0 fails test run if app or test apk references non-existing classes. See https://issuetracker.google.com/u/1/issues/64094195

Runner v0.5 printed that as instrumentation problem but continued to run tests (that was not perfect, but better behavior), Runner v1.0 fails test run.

Firebase Test Cloud happily marks test run as successful in both cases even though in fact in second case none of the tests did actually run.

That's why I'm creating this issue against Composer to make sure (add test) that following instrumentation output will cause error in Composer and test run won't be marked as successful.

INSTRUMENTATION_RESULT: stream=

Time: 0
There was 1 failure:
1) Fatal exception when running tests
java.lang.NoClassDefFoundError: Failed resolution of: Ljavax/annotation/processing/AbstractProcessor;
    at java.lang.Class.classForName(Native Method)
    at java.lang.Class.forName(Class.java:400)
    at android.support.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:76)
    at android.support.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:104)
    at android.support.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:808)
    at android.support.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:481)
    at android.support.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:367)
    at com.app.test.InstrumentationRunner.onStart(InstrumentationRunner.java:26)
    at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1962)
Caused by: java.lang.ClassNotFoundException: Didn't find class "javax.annotation.processing.AbstractProcessor" on path: DexPathList[[zip file "/system/framework/android.test.runner.jar", zip file "/data/app/com.app.test-1/base.apk", zip file "/data/app/com.app-1/base.apk"],nativeLibraryDirectories=[/data/app/com.app.test-1/lib/arm64, /data/app/com.app-1/lib/arm, /system/fake-libs, /data/app/com.app.test-1/base.apk!/lib/armeabi-v7a, /data/app/com.app-1/base.apk!/lib/armeabi-v7a, /system/lib, /vendor/lib]]
    at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:380)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:312)
    ... 9 more

FAILURES!!!
Tests run: 0,  Failures: 1


INSTRUMENTATION_CODE: -1

Composer already fails test run if 0 tests were run, but I just want make code even more robust against such stupid errors and be better than Firebase at parsing Google's crazy instrumentation output format.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant