Bump smtml to 0.3.1

[filipeom]

Bumps smtml version to 0.3.1 (latest) which now supports Alt-Ergo and includes the fix for Bitwuzla.

I ran Owi on testcomp with Z3, 4 workers, and 5s timeout and compared the latest (0.3.1) version of smtml to the previous (0.2.5). Results appear the same, only a slight variation in two benchmarks which is expected with a low timeout. Results are summarized below:

w4_O3_sZ3 smtml.0.3.1 (latest)

Results:

Nothing	Reached	Timeout	Other	Killed	Total
0	578	637	0	0	1215

Time stats (in seconds):

Total	Mean	Median	Min	Max
3504.09	2.88	5.00	0.09	5.00

w4_O3_sZ3 smtml.0.2.5 (previous)

Results:

Nothing	Reached	Timeout	Other	Killed	Total
0	580	635	0	0	1215

Time stats (in seconds):

Total	Mean	Median	Min	Max
3496.49	2.88	5.00	0.09	5.00

I also ran bitwuzla with the latest version. Results are better and there are no segfaults. There are 6 benchmarks which crash due to a missing operator in Bitwuzla, but this is expected. There are also 15 which are killed but I'm not quite sure why. I'll try to investigate when I get some time.

w4_O3_sBitwuzla smtml.0.3.1 (latest)

Results:

Nothing	Reached	Timeout	Other	Killed	Total
0	522	672	6	15	1215

Time stats (in seconds):

Total	Mean	Median	Min	Max
3658.69	3.01	5.00	0.08	5.00

[zapashcanon]

Thanks!

Out of curiosity, did you try to run Test-Comp with alt-ergo?

[filipeom]

Yes I did! But the results are still not very good.
But these are probably issues on our side

Using:

• Tool: owi_w4_O3_sAlt-Ergo
• Timeout: 5.
• Output dir: results-testcomp-owi_w4_O3_sAlt-Ergo-2024-11-07_16h50m23s/

Results:

Nothing	Reached	Timeout	Other	Killed	Total
2	198	547	468	0	1215

Time stats (in seconds):

Total	Mean	Median	Min	Max
3117.67	2.57	2.23	0.09	5.00

[zapashcanon]

OK interesting, it's already a good first step. :)