[RFC]: Continuous Integration Design

[Luni-4]

This issue is an extension of #428 and it is an RFC to verify if there is a consensus with this new design

Feature description

A new design for continuous integration might be the following one:

• One job for code formatting and linting issues
• One job that runs all std tests on Linux
• One job that runs all std tests on MacOs (@dcvz)
• One job that runs all std tests on Windows (@syl20bnr)
• One job that runs all no-std tests

Let's try to tackle each point one-by-one

• (1) Some GitHub actions allow to visualize clippy lints and bad formatting as messages in a PR or in a report. This approach reduces the time spent in detecting these problems because they are immediately viewable. A contributor, especially a new one, does not have to scrape through CI logs in order to find them. This job could run only on a Linux operating system. Running this job on more operating system could be a waste of resources.

• (2, 3, 4) Run all current checks and tests on the specific operating system using a script. The run-checks.sh script could be rewritten in Rust language in order to be cross-platform and less error-prone. Collect code coverage data through Rust source code coverage and then aggregate the results using grcov, finally publish this data on codecov in order to immediately visualize which lines of code are not covered by tests. (thanks to @gonzalesMK for the help)

• (5) Maintain the same checks which are already performed. Even this job will benefit from run-checks.sh rewriting in Rust.

Feature motivation

Having a Continuous Integration that implements the points described above, in my opinion, reduces the effort in code maintainability, making the development more interesting, also decreases the possibility to have problems in the future.

[Luni-4]

What we still have to implement :

• Use GitHub Actions filters to avoid running CI for nothing
• Implement code coverage checks ONLY on CI (grcov, codecov)
• Try another strategy to run clippy on CI
• Add MacOS and Windows configurations through matrix
• Use more https://github.com/Swatinem/rust-cache#example-usage parameters to cache more things and speed up CI
• Use binaries for typos-cli instead of cargo install
• Ideas to speed up CI https://xxchan.me/cs/2023/02/17/optimize-rust-comptime-en.html#step-1-compilation-cache

[gonzalesMK]

@Luni-4 I have 2 points to consider:

1. Should we run Rust Doc just once?
2. Can we minimize running coverage? Maybe run coverage in Linux jobs and then run coverage for specific packages on other OS?

[Luni-4]

@gonzalesMK

1. What do you mean with just once? Right now we run cargo doc command for std tests on Linux
2. Yep, that would be fantastic! The code coverage for each crates on Linux and some coverage tests for cross-platform crates. That would be the ideal solution in my opinion. A tradeoff between the usage of heavy resources and fine-grained tests

[gonzalesMK]

1. Yeah, I just meant to keep like that. I think I misread the first post because I thought the doc was going to run for windows/mac/linux. 👍

Besides, should I close this PR #797 , so to implement this RFC in a new PR ?

[Luni-4]

@gonzalesMK

Perhaps we can transform each bullet point into a PR, in this way it would be simpler to review. What do you think?

[nathanielsimard]

@Luni-4 I think the current project has a quite fast compilation time, relatively speaking, compared to other Rust projects. The main advantage is splitting it into multiple crates that can easily be cached. I think we could greatly benefit from a faster linker though, maybe something like mold.

[Luni-4]

@nathanielsimard

Yep, the mold idea is contained in the link provided by @gonzalesMK

[gonzalesMK]

Perhaps we can transform each bullet point into a PR, in this way it would be simpler to review. What do you think?

Sounds good to me.

[Luni-4]

Thanks to @gonzalesMK, @dcvz, and @syl20bnr this issue can be closed! Congrats to everyone for your work and help!