Test Coverage Improvements needed (November 2024)

[huitema]

The code coverage of the test of the main picoquic libraries as of 2024/11/20 stands as:

	Exec	Total	Coverage
lines	27444	32479	84.5%
functions	1598	1785	89.5%
branches	11653	16066	72.5%

It should be improved by pruning obsolete code (e.g., the "siduck" test of the datagram function). and by targeted improvements in test code.

[huitema]

After removing Siduck code (PR #1780):

	Exec	Total	Coverage
lines	27442	32354	84.8%
functions	1598	1779	89.8%
branches	11650	15993	72.8%

[huitema]

After adding tests and removing dead code from quicctx.c (PR #1784):

	Exec	Total	Coverage
lines	27625	32339	85.4%
functions	1627	1777	91.6%
branches	11736	15980	73.4%

[huitema]

After adding tests of the frames parsing and formatting functions (PR #1785):

	Exec	Total	Coverage
lines	27936	32406	86.2%
functions	1634	1776	92.0%
branches	11905	16018	74.3%

[huitema]

Fixing the spinbit tests in PR #1786 includes better design of the Spinbit API, and brings the test coverage of the spincode to 100%. However, that code is tiny, and there is hardly any effect on global coverage.

	Exec	Total	Coverage
lines	27955	32419	86.2%
functions	1637	1777	92.1%
branches	11908	16022	74.3%

[huitema]

The BBRv1 code is kept in the build for research purposes, such as measuring the difference between old and new versions. There was just one scenario tested, and the test coverage of that module was less than 60%. After adding a few tests in PR #1787, the coverage of that module increases to 81.7% of code lines. That's not great, but probably OK given the expectations. Increasing that would probably require working on the details of the BBRv1 implementation, which is low priority. In any case, after these improvements, the coverage becomes:

	Exec	Total	Coverage
lines	28137	32419	86.8%
functions	1647	1777	92.7%
branches	12000	16022	74.9%

[huitema]

Before PR #1790, the coverage of the module tls_api.c stood at 83.4% of lines and 66.9% of paths. There were a number of functions that were not tested, mostly because they were present in the API but not exercised by the existing code. There were also a number of error paths that could only be triggered by an error in an external component like Picotls or OpenSSL. The PR includes a mix of additional tests, removal of unused code, and some rewriting of dubious error handling. After the PR, the coverage becomes:

	Exec	Total	Coverage
lines	28216	32321	87.3%
functions	1653	1770	93.4%
branches	12030	15987	75.2%

[huitema]

The h3zero_common.c code implements a large part of the HTTP3 protocol, but it was not fully tested. There are a number of branches for handling malloc failures. The existing tests did not cover the "client" part of HTTP3 in h3zero_common.c, and also did not cover well the handling of datagrams. Before PR #1791, the coverage of the module stood at 77.5% of lines and 64% of paths. PR #1791 adds an extensive testing of the "client data" handling, bringing the module coverage to 84.5% of lines and 71.6% of paths. The expectation is that the coverage will be extended by better testing of the datagram code, as part of Web Transport testing. After PR #1791, the total coverage becomes:

	Exec	Total	Coverage
lines	28307	32311	87.6%
functions	1656	1769	93.6%
branches	12086	15978	75.6%

[huitema]

Prior to PR #1792, the test coverage of transport.c was 86.5% lines and 76.0% branches. The number were pulled down by 70 lines of dead code, used to support encoding of transport parameters in old draft versions. After removing these lines, the coverage of transport.c climbs to 93% of lines and 78.2% of branches. The global coverage becomes:

	Exec	Total	Coverage
lines	28306	32263	87.7%
functions	1656	1766	93.8%
branches	12086	15966	75.7%

[huitema]

PR #1793 adds more tests for the "demo client" code. Prior to that, the test coverage of democlient.c was 74.2% lines and 64.3% branches. After the PR, we get 91.5% lines and 77.8% branches. The global coverage becomes:

	Exec	Total	Coverage
lines	28453	32263	88.2%
functions	1658	1766	93.9%
branches	12168	15966	76.2%

[huitema]

The memory log code was untested, and so was the "svg" logging mode available in picolog. There were also some important testing holes in the qlog code, and in the autoqlog.c module. And there were of course a few bugs hiding in these holes. This is fixed in PR #1797. The global coverage becomes:

	Exec	Total	Coverage
lines	28934	32281	89.6%
functions	1690	1766	95.7%
branches	12283	15972	76.9%

[huitema]

The test coverage of the web transport implementation was limited. It is significantly improved in PR #1798. The global coverage becomes:

	Exec	Total	Coverage
lines	28977	32263	89.8%
functions	1691	1766	95.8%
branches	12302	15972	77.0%

[huitema]

After removing unused code from packet.c in PR #1799, the global coverage becomes:

	Exec	Total	Coverage
lines	28963	32206	89.9%
functions	1690	1765	95.8%
branches	12295	15934	77.2%

[huitema]

After removing dead code and add tests for the functions in util.c in PR #1800, the global coverage becomes:

	Exec	Total	Coverage
lines	28945	32131	90.1%
functions	1690	1759	96.1%
branches	12287	15888	77.3%

Add this point, we have met the goals set for the effort: global coverage over 90% for lines, over 75% for branches. Further efforts will be done when adding new functions.