Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Refactor random testing framework (#3130)
This PR offers a substantial rewrite of the current framework for random integration testing. The most significant change is that now every "attack method" is derived from `RandomAttackMethod` base class, and implements basic methods such as `log_to_console()`, `apply_to_dtframe()` and `apply_to_pyframe()`. Notably, `MetaFrame` is no longer responsible for performing operations on itself, instead, this functionality has been merged into each attack method. These changes make adding new random tests easier: they no longer need to live in 2 separate files and make those files unbearably large. Now, each test method lives in a separate file, and conforms to a strict API, which makes the code more consistent. Additional changes include: - An attack method can be skipped using `self.skipped = True`; - An attack method can declare that it produces an exception: `self.raises = ValueError`, and the framework will catch that exception and verify that it has the correct type/message; - Evaluation context now has the ability to add deferred checks, which will be automatically run at the end of a testing session.
- Loading branch information