System.Text.Json.SourceGeneration.Tests are disabled in WASM #59228

layomia · 2021-09-16T21:33:17Z

The System.Text.Json.SourceGeneration.Roslyn.x.tests appear to be timing out during the AOT compilation phase. From PR #59042:

https://helix.dot.net/api/2019-06-17/jobs/770ad140-31bb-4858-9396-d50162076efd/workitems/System.Text.Json.SourceGeneration.Roslyn3.11.Tests/console

  [39/47] xunit.runner.utility.netcoreapp10.dll.bc -> xunit.runner.utility.netcoreapp10.dll.o [took 20.326s]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : Failed to compile /datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/wasm_build/obj/wasm/System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.bc -> /datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/wasm_build/obj/wasm/System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.o [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error :  "/datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/bin/clang++" -target wasm32-unknown-emscripten -DEMSCRIPTEN -fno-inline-functions -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=23 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -Oz -g -v -c /datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/wasm_build/obj/wasm/System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.bc -o /tmp/tmpkKeaiw.tmp [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : clang version 13.0.0 (/b/s/w/ir/cache/git/chromium.googlesource.com-external-github.com-llvm-llvm--project 5852582532b3eb3ea8da51a1e272d8d017bd36c9) [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : Target: wasm32-unknown-emscripten [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : Thread model: posix [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : InstalledDir: /datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/bin [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error :  (in-process) [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error :  "/datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/bin/clang++" -cc1 -triple wasm32-unknown-emscripten -emit-obj --mrelax-relocations -disable-free -main-file-name System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.bc -mrelocation-model static -mframe-pointer=none -fno-rounding-math -mconstructor-aliases -target-cpu generic -fvisibility hidden -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -v -fcoverage-compilation-dir=/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish -resource-dir /datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/lib/clang/13.0.0 -Oz -Werror=implicit-function-declaration -fdebug-compilation-dir=/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish -ferror-limit 19 -fgnuc-version=4.2.1 -fno-inline-functions -vectorize-slp -iwithsysroot/include/SDL -iwithsysroot/include/compat -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -o /tmp/tmpkKeaiw.tmp -x ir /datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/wasm_build/obj/wasm/System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.bc [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : clang -cc1 version 13.0.0 based upon LLVM 13.0.0git default target x86_64-unknown-linux-gnu [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : emcc: error: '/datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/bin/clang++ -target wasm32-unknown-emscripten -DEMSCRIPTEN -fno-inline-functions -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=23 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -Oz -g -v -c /datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/wasm_build/obj/wasm/System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.bc -o /tmp/tmpkKeaiw.tmp' failed (received SIGKILL (-9)) [took 433.767s] [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
XHarness artifacts: /datadisks/disk1/work/9D6C08B7/w/AA0A0963/uploads/xharness-output

In that same PR I'm disabling the tests to unblock green CI.

cc @eerhardt @lewing @radical @SamMonoRT @vargaz

The text was updated successfully, but these errors were encountered:

ghost · 2021-09-16T21:33:20Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

The System.Text.Json.SourceGeneration.Roslyn.x.tests appear to be timing out during the AOT compilation phase. From PR #59042:

https://helix.dot.net/api/2019-06-17/jobs/770ad140-31bb-4858-9396-d50162076efd/workitems/System.Text.Json.SourceGeneration.Roslyn3.11.Tests/console

  [39/47] xunit.runner.utility.netcoreapp10.dll.bc -> xunit.runner.utility.netcoreapp10.dll.o [took 20.326s]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : Failed to compile /datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/wasm_build/obj/wasm/System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.bc -> /datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/wasm_build/obj/wasm/System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.o [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error :  "/datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/bin/clang++" -target wasm32-unknown-emscripten -DEMSCRIPTEN -fno-inline-functions -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=23 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -Oz -g -v -c /datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/wasm_build/obj/wasm/System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.bc -o /tmp/tmpkKeaiw.tmp [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : clang version 13.0.0 (/b/s/w/ir/cache/git/chromium.googlesource.com-external-github.com-llvm-llvm--project 5852582532b3eb3ea8da51a1e272d8d017bd36c9) [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : Target: wasm32-unknown-emscripten [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : Thread model: posix [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : InstalledDir: /datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/bin [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error :  (in-process) [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error :  "/datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/bin/clang++" -cc1 -triple wasm32-unknown-emscripten -emit-obj --mrelax-relocations -disable-free -main-file-name System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.bc -mrelocation-model static -mframe-pointer=none -fno-rounding-math -mconstructor-aliases -target-cpu generic -fvisibility hidden -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -v -fcoverage-compilation-dir=/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish -resource-dir /datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/lib/clang/13.0.0 -Oz -Werror=implicit-function-declaration -fdebug-compilation-dir=/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish -ferror-limit 19 -fgnuc-version=4.2.1 -fno-inline-functions -vectorize-slp -iwithsysroot/include/SDL -iwithsysroot/include/compat -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -o /tmp/tmpkKeaiw.tmp -x ir /datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/wasm_build/obj/wasm/System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.bc [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : clang -cc1 version 13.0.0 based upon LLVM 13.0.0git default target x86_64-unknown-linux-gnu [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
/datadisks/disk1/work/9D6C08B7/p/build/wasm/WasmApp.Native.targets(293,5): error : emcc: error: '/datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/bin/clang++ -target wasm32-unknown-emscripten -DEMSCRIPTEN -fno-inline-functions -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-cxx-exceptions -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=23 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/datadisks/disk1/work/9D6C08B7/p/build/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -Oz -g -v -c /datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/wasm_build/obj/wasm/System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll.bc -o /tmp/tmpkKeaiw.tmp' failed (received SIGKILL (-9)) [took 433.767s] [/datadisks/disk1/work/9D6C08B7/w/AA0A0963/e/publish/ProxyProjectForAOTOnHelix.proj]
XHarness artifacts: /datadisks/disk1/work/9D6C08B7/w/AA0A0963/uploads/xharness-output

In that same PR I'm disabling the tests to unblock green CI.

cc @eerhardt @lewing @radical @SamMonoRT @vargaz

Author:	layomia
Assignees:	-
Labels:	`arch-wasm`, `area-System.Text.Json`, `disabled-test`
Milestone:	7.0.0

danmoseley · 2021-09-17T02:20:56Z

@steveisok is already tracking this one. Curious how it got in actually.

lewing · 2021-09-17T02:26:58Z

@danmoseley pretty sure this is the host killing the aot compiler for some reason, probably OOM in this case.

@radical @steveisok we can try turning down the parallelism on the json tests as a first step.

danmoseley · 2021-09-17T02:48:09Z

Oops, mixed this up with the unrelated Android issue triggered by the digit in the name Roslyn3.11.Tests

radical · 2021-09-20T18:19:13Z

@radical @steveisok we can try turning down the parallelism on the json tests as a first step.

Running it serially didn't help. It is still getting killed in the same way.

radical · 2021-09-20T21:43:05Z

OOM:

python3 invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0

...
[  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[  24770]  1000 24770    14474     2510   155648        0             0 python3
[  24775]  1000 24775  1906650  1837676 15310848        0             0 clang++

...

out of memory: Killed process 24775 (clang++) total-vm:7626600kB, anon-rss:7350704kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:14952kB oom_score_adj:0

https://gist.github.com/radical/3c45276841d3c4e2d2cc91d5069a4d7a

danmoseley · 2021-09-20T22:18:01Z

It would be invaluable if there was logging such that we knew what tests were in flight. There is likely one or two tests that are memory heavy and just need disabling for WASM.

I'm guessing there is no test results file already written when it's killed?

Cc @dotnet/area-system-text-json

radical · 2021-09-20T23:37:56Z

The tests haven't run at this point. We do the AOT part of the build on helix, because it's very slow. And this is getting killed while compiling one of the assemblies (System.Text.Json.SourceGeneration.Roslyn3.11.Tests.dll).

vargaz · 2021-09-21T00:23:09Z

This might be fixed by:
#59352
which causes input bitcode to be smaller, so emcc runs faster/uses less memory.

danmoseley · 2021-09-21T00:44:57Z

@radical ah, I missed that.

Fixed by dotnet#59352 . Fixes dotnet#59228 .

layomia added arch-wasm WebAssembly architecture area-System.Text.Json disabled-test The test is disabled in source code against the issue labels Sep 16, 2021

layomia added this to the 7.0.0 milestone Sep 16, 2021

ghost added the untriaged New issue has not been triaged by the area owner label Sep 16, 2021

layomia mentioned this issue Sep 16, 2021

Update source-gen APIs according to review #59042

Merged

danmoseley modified the milestones: 7.0.0, 6.0.0 Sep 17, 2021

lewing added the area-Codegen-AOT-mono label Sep 17, 2021

danmoseley removed the area-System.Text.Json label Sep 17, 2021

lewing removed the untriaged New issue has not been triaged by the area owner label Sep 17, 2021

lewing modified the milestones: 6.0.0, 7.0.0 Sep 17, 2021

radical added a commit to radical/runtime that referenced this issue Sep 21, 2021

[wasm] Enable System.Text.Json.SourceGeneration.Tests

b4f830b

Fixed by dotnet#59352 . Fixes dotnet#59228 .

radical mentioned this issue Sep 21, 2021

[wasm] Enable System.Text.Json.SourceGeneration.Tests #59396

Merged

ghost added the in-pr There is an active PR which will close this issue when it is merged label Sep 21, 2021

radical closed this as completed in #59396 Sep 22, 2021

ghost removed the in-pr There is an active PR which will close this issue when it is merged label Sep 22, 2021

ghost locked as resolved and limited conversation to collaborators Nov 3, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

System.Text.Json.SourceGeneration.Tests are disabled in WASM #59228

System.Text.Json.SourceGeneration.Tests are disabled in WASM #59228

layomia commented Sep 16, 2021

ghost commented Sep 16, 2021

danmoseley commented Sep 17, 2021

lewing commented Sep 17, 2021

danmoseley commented Sep 17, 2021

radical commented Sep 20, 2021

radical commented Sep 20, 2021 •

edited

Loading

danmoseley commented Sep 20, 2021

radical commented Sep 20, 2021

vargaz commented Sep 21, 2021

danmoseley commented Sep 21, 2021

System.Text.Json.SourceGeneration.Tests are disabled in WASM #59228

System.Text.Json.SourceGeneration.Tests are disabled in WASM #59228

Comments

layomia commented Sep 16, 2021

ghost commented Sep 16, 2021

danmoseley commented Sep 17, 2021

lewing commented Sep 17, 2021

danmoseley commented Sep 17, 2021

radical commented Sep 20, 2021

radical commented Sep 20, 2021 • edited Loading

danmoseley commented Sep 20, 2021

radical commented Sep 20, 2021

vargaz commented Sep 21, 2021

danmoseley commented Sep 21, 2021

radical commented Sep 20, 2021 •

edited

Loading