There doesn't seem to be a way to get CFLite to upload files crashing fuzz targets if they crash the latest build

[evverx]

I've been using CFLite to test various patches and when they introduce new issues CFLite uploads files to the "summary" tab as expected. But if CFLite thinks that bugs aren't introduced in PRs it just skips that step. I think it makes sense in general but in some cases bugs can collide so to speak and crash PRs and the latest build differently. It would be great if I could flip a flag named, say, 'always_upload_crashes` to avoid running something like

./infra/helper.py build_fuzzers --external ~/elfutils
./infra/helper.py run_fuzzer --external ~/elfutils fuzz-dwfl-core

in my VMs and copying files to machines that have access to bug trackers to just attach those files to bug reports.

[jonathanmetzman]

How about a switch to shut off crash novelty checking instead?

[evverx]

It would make it harder to tell really new bugs and bugs that just happen to collide apart, which would probably mean that I wouldn't even try to turn it off for projects that have just started fixing bugs found by fuzz targets.

[jonathanmetzman]

Sorry, I think I missed something in your original request.
Do you want uploading of fuzz targets and crashes that are not novel?

[jonathanmetzman]

Should we reopen this and close google/oss-fuzz#6984 instead?

[evverx]

Should we reopen this and close google/oss-fuzz#6984 instead?

I thought I was doing exactly that :-) Looks like I lost track of all the issues I've opened :-)

Another feature that I think would be useful if CFLite could show backtraces of the last builds as well. I've just discovered an issue that I haven't seen before but

2021-12-08 23:31:17,581 - root - INFO - Fuzzer: fuzz-dwfl-core. Detected bug.
2021-12-08 23:31:17,582 - root - INFO - Trying to reproduce crash using: /tmp/tmp7n4idjys/oom-e7fed1fe8da16c1482845db8b8d1fac479a425b5.
2021-12-08 23:31:18,452 - root - INFO - Reproduce command returned: 71. Reproducible on /github/workspace/build-out/fuzz-dwfl-core.
2021-12-08 23:31:18,452 - root - INFO - Crash is reproducible.
2021-12-08 23:31:18,462 - root - INFO - Trying to downloading previous build 51106acb0d7f01845ebab85a3ce4e8b46ae95ab5.
2021-12-08 23:31:20,151 - root - INFO - Done downloading previous build.
2021-12-08 23:31:20,152 - root - INFO - Trying to reproduce crash using: /tmp/tmp7n4idjys/oom-e7fed1fe8da16c1482845db8b8d1fac479a425b5.
2021-12-08 23:31:20,982 - root - INFO - Reproduce command returned: 71. Reproducible on /github/workspace/cifuzz-prev-build/fuzz-dwfl-core

so I have a patch and a new backtrace and to figure out what is going there it seems I have to build and run the fuzz target locally to get the file triggering it, then I have to revert that patch and build the fuzz target once again to pass the testcase to it to get the backtrace. It wouldn't be necessary if CFLite uploaded files crashing fuzz targets unconditionally and showed "old" backtraces.

[jonathanmetzman]

Should we reopen this and close google/oss-fuzz#6984 instead?

I thought I was doing exactly that :-) Looks like I lost track of all the issues I've opened :-)

Another feature that I think would be useful if CFLite could show backtraces of the last builds as well. I've just discovered an issue that I haven't seen before but

2021-12-08 23:31:17,581 - root - INFO - Fuzzer: fuzz-dwfl-core. Detected bug.
2021-12-08 23:31:17,582 - root - INFO - Trying to reproduce crash using: /tmp/tmp7n4idjys/oom-e7fed1fe8da16c1482845db8b8d1fac479a425b5.
2021-12-08 23:31:18,452 - root - INFO - Reproduce command returned: 71. Reproducible on /github/workspace/build-out/fuzz-dwfl-core.
2021-12-08 23:31:18,452 - root - INFO - Crash is reproducible.
2021-12-08 23:31:18,462 - root - INFO - Trying to downloading previous build 51106acb0d7f01845ebab85a3ce4e8b46ae95ab5.
2021-12-08 23:31:20,151 - root - INFO - Done downloading previous build.
2021-12-08 23:31:20,152 - root - INFO - Trying to reproduce crash using: /tmp/tmp7n4idjys/oom-e7fed1fe8da16c1482845db8b8d1fac479a425b5.
2021-12-08 23:31:20,982 - root - INFO - Reproduce command returned: 71. Reproducible on /github/workspace/cifuzz-prev-build/fuzz-dwfl-core

so I have a patch and a new backtrace and to figure out what is going there it seems I have to build and run the fuzz target locally to get the file triggering it, then I have to revert that patch and build the fuzz target once again to pass the testcase to it to get the backtrace. It wouldn't be necessary if CFLite uploaded files crashing fuzz targets unconditionally and showed "old" backtraces.

Sorry, I'm having trouble understanding this. We already show the stacktrace of the crash on the newer binary, why do we need to show the crash from the older binary?

[evverx]

why do we need to show the crash from the older binary?

Because the backtrace can be different if PRs introduce new bugs on top of old bugs.