[Git HEAD] 3 of 37 tests are failing (in Ubuntu 24.04.1 LTS of GitHub Actions)

[hartwork]

This is with Ubuntu 24.04 LTS:

+ make check
Creating coverage files (2 tests, 5 source files)
  Source tree ......... done (950 lines, 50 functions, 3428 branches)
  Full coverage ....... done
  Target coverage ..... done
  Partial coverage .... done
  Zero coverage ....... done
Starting tests
genhtml/full.sh ................... [pass] (time 0.4s, mem 47.0MB)
genhtml/zero.sh ................... [pass] (time 0.4s, mem 46.7MB)
genhtml/demangle.sh ............... [pass] (time 1.3s, mem 38.8MB)
genhtml/relative/relative.sh ...... [pass] (time 0.3s, mem 39.3MB)
genhtml/lambda/lambda.sh .......... [pass] (time 3.7s, mem 126.3MB)
lcov/add/zero.sh .................. [pass] (time 0.3s, mem 37.5MB)
lcov/add/zero2.sh ................. [pass] (time 0.5s, mem 42.7MB)
lcov/add/full.sh .................. [pass] (time 0.3s, mem 37.5MB)
lcov/add/full2.sh ................. [pass] (time 0.5s, mem 42.7MB)
lcov/add/prune.sh ................. [pass] (time 0.7s, mem 33.7MB)
lcov/add/track.sh ................. [pass] (time 0.2s, mem 33.3MB)
lcov/misc/help.sh ................. [pass] (time 0.2s, mem 32.5MB)
lcov/misc/version.sh .............. [pass] (time 0.2s, mem 32.5MB)
lcov/summary/zero.sh .............. [pass] (time 0.3s, mem 37.5MB)
lcov/summary/full.sh .............. [pass] (time 0.3s, mem 37.6MB)
lcov/extract/extract.sh ........... [pass] (time 14.4s, mem 61.8MB)
lcov/demangle/demangle.sh ......... [pass] (time 2.6s, mem 61.3MB)
lcov/exception/exception.sh ....... [pass] (time 3.1s, mem 48.6MB)
lcov/gcov-tool/path.sh ............ [pass] (time 4.4s, mem 33.8MB)
lcov/branch/branch.sh ............. [pass] (time 1.4s, mem 61.5MB)
lcov/merge/merge.sh ............... [fail] (time 1.6s, mem 33.0MB)
    Skipping 117 previous lines (see /home/runner/work/lcov-fork/lcov-fork/tests//test.log)
    ...
    	(use "lcov --ignore-errors inconsistent,inconsistent ..." to suppress this warning)
    DA:6,0
    LF:8
    LH:2
    Found 4 cores.
    Combining tracefiles.
    lcov: ERROR: (missing) 'functionBug_1.dat' found from pattern 'functionBug_1.dat' is not a readable file
    	(use "lcov --ignore-errors missing ..." to bypass this error)
    Error:  function merge failed
  EXITCODE ...: 1
lcov/format/format.sh ............. [pass] (time 1.4s, mem 33.0MB)
lcov/errs/errs.sh ................. [pass] (time 6.9s, mem 33.0MB)
lcov/multiple/multiple.sh ......... [pass] (time 1.2s, mem 33.7MB)
lcov/follow/follow.sh ............. [pass] (time 1.2s, mem 33.7MB)
lcov/initializer/initializer.sh ... [pass] (time 1.0s, mem 84.9MB)
lcov/lambda/lambda.sh ............. [pass] (time 0.3s, mem 36.3MB)
lcov/mcdc/mcdc.sh ................. [pass] (time 7.3s, mem 86.2MB)
gendiffcov/simple/script.sh ....... [fail] (time 2.3s, mem 65.4MB)
    Skipping 131 previous lines (see /home/runner/work/lcov-fork/lcov-fork/tests//test.log)
    ...
    Writing data to filt2.info
    Summary coverage rate:
      source files: 1
      lines.......: 62.5% (10 of 16 lines)
      functions...: 100.0% (1 of 1 function)
      branches....: 33.3% (2 of 6 branches)
    Message summary:
      no messages were reported
    ERROR: filter with mismatched version did not fail
  EXITCODE ...: 1
gendiffcov/filter/filter.pl ....... [pass] (time 0.1s, mem 31.8MB)
gendiffcov/function/function.sh ... [pass] (time 7.1s, mem 66.9MB)
...fcov/insensitive/insensitive.sh  [pass] (time 3.7s, mem 65.4MB)
...iffcov/synthesize/synthesize.sh  [pass] (time 3.6s, mem 65.2MB)
gendiffcov/errs/msgtest.sh ........ [pass] (time 18.3s, mem 65.3MB)
py2lcov/py2lcov.sh ................ [fail] (time 0.3s, mem 31.1MB)
    Skipping 19 previous lines (see /home/runner/work/lcov-fork/lcov-fork/tests//test.log)
    ...
        main()
      File "/home/runner/work/lcov-fork/lcov-fork/bin/py2lcov", line 189, in main
        p.process_xml_file(xml)
      File "/home/runner/work/lcov-fork/lcov-fork/bin/xml2lcovutil.py", line 183, in process_xml_file
        path = os.path.join(s[0], name)
               ^^^^^^^^^^^^^^^^^^^^^^^^
      File "<frozen posixpath>", line 76, in join
    TypeError: expected str, bytes or os.PathLike object, not NoneType
    py2lcov failed function example
  EXITCODE ...: 1
perl2lcov/perltest1.sh ............ [pass] (time 8.3s, mem 71.1MB)
xml2lcov/xml2lcov.sh .............. [pass] (time 1.1s, mem 33.7MB)
37 tests executed, 34 passed, 3 failed, 0 skipped (time 102.9s, mem 1803.5MB)

[hartwork]

@henry2cox it turns out that vanilla Ubuntu 24.04 does not show the same symptoms, so it must be something in the GitHub Actions images that GitHub provides 🤔

Here's the Dockerfile that I used to assess status quo:

FROM ubuntu:24.04
RUN apt-get update \
        && \
    apt-get install --no-install-recommends --yes -V \
            libcapture-tiny-perl \
            libdatetime-perl \
            libdevel-cover-perl \
            libdigest-md5-file-perl \
            libfile-spec-native-perl \
            libjson-xs-perl \
            libscalar-list-utils-perl \
            llvm \
            python3-coverage \
            python3-xlsxwriter \
            libgd-perl \
            git \
            make
RUN perl -MCPAN -e 'install(Memory::Process)'  # no package in Debian/Ubuntu
RUN ln -s python3-coverage /usr/bin/coverage  # until issue #347 is fixed
RUN git clone --depth 1 https://github.com/linux-test-project/lcov
WORKDIR /lcov/
RUN make check

[hartwork]

@henry2cox PS: Could it be the version of GCC used? GitHub Actions seems to have GCC 13.2.0 — do you see anything at https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md that already looks like trouble?

[hartwork]

I tried with image gcc:13.2.0 and also image gcc:13 based on Debian bookworm and GCC 13.2.0/13.3.0 and all tests pass there. So it's probably not "just GCC 13".

[hartwork]

Thinking aloud: could be related to #341

[henry2cox]

@henry2cox PS: Could it be the version of GCC used? GitHub Actions seems to have GCC 13.2.0 — do you see anything at https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md that already looks like trouble?

Possibly, I lost the plot...which issue do you think might be version related?

I think that the 'simple' case just needs another portability fix (related to one of the shortcuts I took, when I wrote the test).
Not sure what the issue is with the py2lcov test. Need more shrapnel, I fear.

[hartwork]

which issue do you think might be version related?

@henry2cox none in particular and all of them at the same time.

[hartwork]

Update for the record for anyone else reading and interested: With #353 merged now, the failures are down to 2 and the output is:

Creating coverage files (2 tests, 5 source files)
  Source tree ......... done (950 lines, 50 functions, 3428 branches)
  Full coverage ....... done
  Target coverage ..... done
  Partial coverage .... done
  Zero coverage ....... done
Starting tests
genhtml/full.sh ................... [pass] (time 0.4s, mem 47.2MB)
genhtml/zero.sh ................... [pass] (time 0.4s, mem 46.8MB)
genhtml/demangle.sh ............... [pass] (time 1.2s, mem 39.0MB)
genhtml/relative/relative.sh ...... [pass] (time 0.3s, mem 39.0MB)
genhtml/lambda/lambda.sh .......... [pass] (time 1.8s, mem 124.0MB)
lcov/add/zero.sh .................. [pass] (time 0.3s, mem 37.7MB)
lcov/add/zero2.sh ................. [pass] (time 0.5s, mem 42.7MB)
lcov/add/full.sh .................. [pass] (time 0.3s, mem 37.5MB)
lcov/add/full2.sh ................. [pass] (time 0.5s, mem 42.5MB)
lcov/add/prune.sh ................. [pass] (time 0.7s, mem 33.7MB)
lcov/add/track.sh ................. [pass] (time 0.2s, mem 33.1MB)
lcov/misc/help.sh ................. [pass] (time 0.1s, mem 32.5MB)
lcov/misc/version.sh .............. [pass] (time 0.1s, mem 32.3MB)
lcov/summary/zero.sh .............. [pass] (time 0.3s, mem 37.5MB)
lcov/summary/full.sh .............. [pass] (time 0.3s, mem 37.5MB)
lcov/extract/extract.sh ........... [pass] (time 14.0s, mem 59.3MB)
lcov/demangle/demangle.sh ......... [pass] (time 2.5s, mem 59.6MB)
lcov/exception/exception.sh ....... [pass] (time 3.1s, mem 45.9MB)
lcov/gcov-tool/path.sh ............ [pass] (time 4.3s, mem 33.8MB)
lcov/branch/branch.sh ............. [pass] (time 1.3s, mem 57.9MB)
lcov/merge/merge.sh ............... [pass] (time 1.9s, mem 33.0MB)
lcov/format/format.sh ............. [pass] (time 1.4s, mem 32.8MB)
lcov/errs/errs.sh ................. [pass] (time 6.8s, mem 33.0MB)
lcov/multiple/multiple.sh ......... [pass] (time 1.1s, mem 33.8MB)
lcov/follow/follow.sh ............. [pass] (time 1.1s, mem 33.8MB)
lcov/initializer/initializer.sh ... [pass] (time 0.9s, mem 82.2MB)
lcov/lambda/lambda.sh ............. [pass] (time 0.3s, mem 36.3MB)
lcov/mcdc/mcdc.sh ................. [pass] (time 4.9s, mem 86.1MB)
gendiffcov/simple/script.sh ....... [fail] (time 8.9s, mem 62.1MB)
    Skipping 484 previous lines (see /home/runner/work/lcov/lcov/tests//test.log)
    ...
    Found 4 cores.
    Reading tracefile ./current.info.gz.
    Found 1 entries.
    Reading diff file diff.txt
    Reading tracefile ./baseline.info.gz.
    Found 1 baseline entries.
    genhtml: ERROR: (inconsistent) File "/home/runner/work/lcov/lcov/tests/gendiffcov/simple/test.cpp" appears in 'diff' data file 'diff.txt' but 'baseline' and 'current' versions 'SHA b70a9f2ba07dc0a119272b2854acab71e88cf139' match
    	(use "genhtml --ignore-errors inconsistent ..." to bypass this error)
    ERROR: genhtml  failed (1)
  EXITCODE ...: 1
gendiffcov/filter/filter.pl ....... [pass] (time 0.1s, mem 31.6MB)
gendiffcov/function/function.sh ... [pass] (time 6.8s, mem 63.8MB)
...fcov/insensitive/insensitive.sh  [pass] (time 3.5s, mem 62.0MB)
...iffcov/synthesize/synthesize.sh  [pass] (time 3.5s, mem 62.0MB)
gendiffcov/errs/msgtest.sh ........ [pass] (time 18.0s, mem 62.1MB)
py2lcov/py2lcov.sh ................ [fail] (time 0.3s, mem 31.1MB)
    Skipping 19 previous lines (see /home/runner/work/lcov/lcov/tests//test.log)
    ...
        main()
      File "/home/runner/work/lcov/lcov/bin/py2lcov", line 204, in main
        p.process_xml_file(xml)
      File "/home/runner/work/lcov/lcov/bin/xml2lcovutil.py", line 183, in process_xml_file
        path = os.path.join(s[0], name)
               ^^^^^^^^^^^^^^^^^^^^^^^^
      File "<frozen posixpath>", line 76, in join
    TypeError: expected str, bytes or os.PathLike object, not NoneType
    py2lcov failed function example
  EXITCODE ...: 1
perl2lcov/perltest1.sh ............ [pass] (time 8.1s, mem 71.0MB)
xml2lcov/xml2lcov.sh .............. [pass] (time 1.1s, mem 33.8MB)
37 tests executed, 35 passed, 2 failed, 0 skipped (time 103.1s, mem 1771.4MB)
Result log stored in /home/runner/work/lcov/lcov/tests/test.log

[hartwork]

Update: With the latest fixes from #354 this single one remains left:

Creating coverage files (2 tests, 5 source files)
  Source tree ......... done (950 lines, 50 functions, 3428 branches)
  Full coverage ....... done
  Target coverage ..... done
  Partial coverage .... done
  Zero coverage ....... done
Starting tests
genhtml/full.sh ................... [pass] (time 0.4s, mem 47.0MB)
genhtml/zero.sh ................... [pass] (time 0.4s, mem 46.8MB)
genhtml/demangle.sh ............... [pass] (time 1.2s, mem 38.8MB)
genhtml/relative/relative.sh ...... [pass] (time 0.3s, mem 39.2MB)
genhtml/lambda/lambda.sh .......... [pass] (time 1.8s, mem 124.2MB)
lcov/add/zero.sh .................. [pass] (time 0.3s, mem 37.5MB)
lcov/add/zero2.sh ................. [pass] (time 0.5s, mem 42.7MB)
lcov/add/full.sh .................. [pass] (time 0.3s, mem 37.7MB)
lcov/add/full2.sh ................. [pass] (time 0.5s, mem 42.8MB)
lcov/add/prune.sh ................. [pass] (time 0.7s, mem 33.8MB)
lcov/add/track.sh ................. [pass] (time 0.2s, mem 33.1MB)
lcov/misc/help.sh ................. [pass] (time 0.1s, mem 32.3MB)
lcov/misc/version.sh .............. [pass] (time 0.1s, mem 32.3MB)
lcov/summary/zero.sh .............. [pass] (time 0.3s, mem 37.5MB)
lcov/summary/full.sh .............. [pass] (time 0.3s, mem 37.6MB)
lcov/extract/extract.sh ........... [pass] (time 14.0s, mem 59.2MB)
lcov/demangle/demangle.sh ......... [pass] (time 2.5s, mem 59.4MB)
lcov/exception/exception.sh ....... [pass] (time 3.1s, mem 45.10MB)
lcov/gcov-tool/path.sh ............ [pass] (time 4.4s, mem 33.8MB)
lcov/branch/branch.sh ............. [pass] (time 1.3s, mem 57.9MB)
lcov/merge/merge.sh ............... [pass] (time 1.8s, mem 32.8MB)
lcov/format/format.sh ............. [pass] (time 1.4s, mem 32.7MB)
lcov/errs/errs.sh ................. [pass] (time 6.8s, mem 33.0MB)
lcov/multiple/multiple.sh ......... [pass] (time 1.1s, mem 33.7MB)
lcov/follow/follow.sh ............. [pass] (time 1.1s, mem 33.8MB)
lcov/initializer/initializer.sh ... [pass] (time 0.9s, mem 82.4MB)
lcov/lambda/lambda.sh ............. [pass] (time 0.3s, mem 36.4MB)
lcov/mcdc/mcdc.sh ................. [pass] (time 4.4s, mem 86.2MB)
gendiffcov/simple/script.sh ....... [pass] (time 28.2s, mem 62.1MB)
gendiffcov/filter/filter.pl ....... [pass] (time 0.1s, mem 31.6MB)
gendiffcov/function/function.sh ... [pass] (time 6.8s, mem 63.9MB)
...fcov/insensitive/insensitive.sh  [pass] (time 3.5s, mem 62.1MB)
...iffcov/synthesize/synthesize.sh  [pass] (time 3.5s, mem 61.9MB)
gendiffcov/errs/msgtest.sh ........ [pass] (time 18.0s, mem 62.0MB)
py2lcov/py2lcov.sh ................ [fail] (time 0.3s, mem 31.1MB)
    Skipping 19 previous lines (see /home/runner/work/lcov/lcov/tests//test.log)
    ...
        main()
      File "/home/runner/work/lcov/lcov/bin/py2lcov", line 204, in main
        p.process_xml_file(xml)
      File "/home/runner/work/lcov/lcov/bin/xml2lcovutil.py", line 183, in process_xml_file
        path = os.path.join(s[0], name)
               ^^^^^^^^^^^^^^^^^^^^^^^^
      File "<frozen posixpath>", line 76, in join
    TypeError: expected str, bytes or os.PathLike object, not NoneType
    py2lcov failed function example
  EXITCODE ...: 1
perl2lcov/perltest1.sh ............ [pass] (time 8.3s, mem 71.2MB)
xml2lcov/xml2lcov.sh .............. [pass] (time 1.1s, mem 34.0MB)
37 tests executed, 36 passed, 1 failed, 0 skipped (time 122.1s, mem 1772.1MB)
Result log stored in /home/runner/work/lcov/lcov/tests/test.log

[hartwork]

@henry2cox the remaining issue seems to have been fixed by 193cc76 . Can this issue be closed as fixed in your eyes?

[henry2cox]

Yes...I think that this one and #348 can both be closed -
(But I prefer bugs to be closed by the person who opened them - as leads to less confusion where I think I fixed something and either the fix didn't work or was entirely unrelated to the actual problem.)

[hartwork]

@henry2cox excellent! Closing as fixed…