Random class and method mutant selection strategies #102
Conversation
…tract superclass random grouped selection & tests
| MTRandomClassMutantSelectionStrategyTest >> classToTest [ | ||
|
|
||
| ^ MTRandomClassMutantSelectionStrategy | ||
| ] |
There was a problem hiding this comment.
This class do not define new tests?
There was a problem hiding this comment.
I agree, but I struggle a bit to see what kind of tests I could do, given the random nature of the selection.
I guess I could create an auxiliar class for each one, like a class with let's say 4 methods, which produce enough mutants to be able to have around 25% of mutants from each method with the random method mutant selection.
There was a problem hiding this comment.
Or else I could create a set of auxiliar classes for all random selections, and given a seed (or maybe even not), I check that I get a different mutant selection with each strategy
There was a problem hiding this comment.
Additionally, we can see if we can parametrise the random selector with a seed, and use a same fixed seed always in the test, to guarantee we have the same results.
There was a problem hiding this comment.
I would combine both your ideas: (1) a class with a list of selectors that we control easily, and (2) the seed. :)
|
I see, the tests are failing in previous versions of Pharo because the seed doesn't seem to produce the same things as in Pharo 12. |
| precision: 0.1 ]. | ||
|
|
||
| self assert: | ||
| (mutationsDic values fold: [ :bool1 :bool2 | bool1 and: bool2 ]) |
There was a problem hiding this comment.
So you are testing that the results are evenly distributed by the grouping condition?
It's a fun way to test it ^^.
I understand you did this because the default random generator changed in Pharo11 or Pharo12, making tests not compatible, right?
Alternative solution: have a set of tests for Pharo<11 and a set of tests for Pharo>=11
Also, I see that Pharo9 is breaking because truncateTo: is not defined in collections.
Maybe we can (should?) drop Pharo9 support?
There was a problem hiding this comment.
At first I checked that the results matched exactly the expected distribution (like #( 30 31 39 ) ), but yes in previous pharo versions the expected distributions are not the same because, even with the same seed, the random generator don't give the same results.
Yes I guess it could be a good (maybe even better) solution to have a set for pharo >= 12 and a set for pharo <= 11.
Given the fact that pharo 9 is in deprecated distributions in the pharo launcher, I think it's acceptable to drop pharo 9 support.
There was a problem hiding this comment.
Just, I don't know how to make tests for specific versions of Pharo :') But I guess it's not that hard
Fixes #44, fixes #45