Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: enhance pipeline #236

Merged
merged 30 commits into from
Nov 16, 2024
Merged

Conversation

MahmoudMabrok
Copy link
Contributor

@MahmoudMabrok MahmoudMabrok commented Nov 7, 2024

Contributor checklist


Description

  • make single workflow instead of three ones
  • fix issue of not writing coverage report into PR Add code coverage for unit tests to Scribe Android #196 .
  • add new caching which make build step from 3 minutes into 12 seconds.
  • add build app step and upload it as artifacts so can be downloaded and tried on real devices if needed

Related issue

Copy link

github-actions bot commented Nov 7, 2024

Thank you for the pull request!

The Scribe team will do our best to address your contribution as soon as we can. The following is a checklist for maintainers to make sure this process goes as well as possible. Feel free to address the points below yourself in further commits if you realize that actions are needed :)

If you're not already a member of our public Matrix community, please consider joining! We'd suggest using Element as your Matrix client, and definitely join the General and Android rooms once you're in. Also consider joining our bi-weekly Saturday dev syncs. It'd be great to have you!

Maintainer checklist

  • The linting and formatting workflows within the PR checks do not indicate new errors in the files changed

  • The CHANGELOG has been updated with a description of the changes for the upcoming release and the corresponding issue (if necessary)

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First PR Commit Check

  • The commit messages for the remote branch should be checked to make sure the contributor's email is set up correctly so that they receive credit for their contribution
    - The contributor's name and icon in remote commits should be the same as what appears in the PR
    - If there's a mismatch, the contributor needs to make sure that the email they use for GitHub matches what they have for git config user.email in their local Scribe-Android repo

@andrewtavis
Copy link
Member

Thanks for this, @MahmoudMabrok! Let me know when we're all ready for a review 😊

@MahmoudMabrok
Copy link
Contributor Author

It is ready but step of jacoco keep failing.
Once fix will make it ready for review.

@MahmoudMabrok MahmoudMabrok marked this pull request as ready for review November 10, 2024 22:58
@andrewtavis
Copy link
Member

Looping in @angrezichatterbox for the review here :) Thanks for the work here, @MahmoudMabrok!

Copy link
Member

@angrezichatterbox angrezichatterbox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, @MahmoudMabrok—everything looks great! I do have one suggestion: would it be better to handle caching in a separate job alongside the JDK environment setup, rather than configuring it in both jobs?

@andrewtavis
Copy link
Member

One thing that I'm wondering as well, do we want all of this in one job? Would it not be a bit easier for people to respond to the issues if they could compartmentalize the errors that they're getting into linting/detekt in one as well as other types in others? Might be a bit overwhelming if it's all in one, whereas the other we could say "Please start with the linting errors here and then we'll move to the others".

Feedback on this is welcome!

@MahmoudMabrok
Copy link
Contributor Author

Thanks for the PR, @MahmoudMabrok—everything looks great! I do have one suggestion: would it be better to handle caching in a separate job alongside the JDK environment setup, rather than configuring it in both jobs?

We need to br done in same job as lint, detect generate files that needs to be cached for faster run so if we split it we will lose this files.

Or you mean to make it as composite action so we reuse it inside each job if this so we can handle it on separate PR.

@MahmoudMabrok
Copy link
Contributor Author

One thing that I'm wondering as well, do we want all of this in one job? Would it not be a bit easier for people to respond to the issues if they could compartmentalize the errors that they're getting into linting/detekt in one as well as other types in others? Might be a bit overwhelming if it's all in one, whereas the other we could say "Please start with the linting errors here and then we'll move to the others".

Feedback on this is welcome!

Let me try to explain my point of view might things be clearer.

Using it in single job helps in lower time as overall as we do not do checkout, install java, Gradle and some initialization also some steps are faster when run in sequence.

For PR owner, when they got issue in lint so they will get lint step as error so they will fix it, then they will push again so lint will run again and if pass will check other steps, what if we use separate job, it has advantage of it will show all stages that passed or failed but let's say PR has issue of lint, so lint will fail and other will success then when PR owner will push again which will run all jobs again even there were passed.

@MahmoudMabrok
Copy link
Contributor Author

@andrewtavis @angrezichatterbox is it clear now.

@angrezichatterbox
Copy link
Member

Thanks for the PR, @MahmoudMabrok—everything looks great! I do have one suggestion: would it be better to handle caching in a separate job alongside the JDK environment setup, rather than configuring it in both jobs?

We need to br done in same job as lint, detect generate files that needs to be cached for faster run so if we split it we will lose this files.

Or you mean to make it as composite action so we reuse it inside each job if this so we can handle it on separate PR.

I meann't we could have a job that would do the JDK install and caching? And let the other jobs depend on it. Could this be possible @MahmoudMabrok

@MahmoudMabrok
Copy link
Contributor Author

Thanks for the PR, @MahmoudMabrok—everything looks great! I do have one suggestion: would it be better to handle caching in a separate job alongside the JDK environment setup, rather than configuring it in both jobs?

We need to br done in same job as lint, detect generate files that needs to be cached for faster run so if we split it we will lose this files.
Or you mean to make it as composite action so we reuse it inside each job if this so we can handle it on separate PR.

I meann't we could have a job that would do the JDK install and caching? And let the other jobs depend on it. Could this be possible @MahmoudMabrok

We could do but it will cache only Gradle which won't add benefits for us.

But we can make it as composite action and re use it.

@MahmoudMabrok
Copy link
Contributor Author

Thanks for the PR, @MahmoudMabrok—everything looks great! I do have one suggestion: would it be better to handle caching in a separate job alongside the JDK environment setup, rather than configuring it in both jobs?

We need to br done in same job as lint, detect generate files that needs to be cached for faster run so if we split it we will lose this files.
Or you mean to make it as composite action so we reuse it inside each job if this so we can handle it on separate PR.

I meann't we could have a job that would do the JDK install and caching? And let the other jobs depend on it. Could this be possible @MahmoudMabrok

The issue that jobs not share data with each other's.

@angrezichatterbox
Copy link
Member

For PR owner, when they got issue in lint so they will get lint step as error so they will fix it, then they will push again so lint will run again and if pass will check other steps, what if we use separate job, it has advantage of it will show all stages that passed or failed but let's say PR has issue of lint, so lint will fail and other will success then when PR owner will push again which will run all jobs again even there were passed.

So, as I understand it, even in this setup, the entire job (or each individual job) re-runs from the beginning when a failure occurs and a new change is pushed by the user. Am I correct?

@MahmoudMabrok
Copy link
Contributor Author

For PR owner, when they got issue in lint so they will get lint step as error so they will fix it, then they will push again so lint will run again and if pass will check other steps, what if we use separate job, it has advantage of it will show all stages that passed or failed but let's say PR has issue of lint, so lint will fail and other will success then when PR owner will push again which will run all jobs again even there were passed.

So, as I understand it, even in this setup, the entire job (or each individual job) re-runs from the beginning when a failure occurs and a new change is pushed by the user. Am I correct?

it will re-run, but it will be caching generated steps so subsequent builds will be less in time.

as you see here the pipeline checks runs in only 1m , here as old run is 2m and so on.

Copy link
Member

@angrezichatterbox angrezichatterbox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good to me. Having them in a single job has its advantages. @andrewtavis, the decision is yours.

@andrewtavis
Copy link
Member

For PR owner, when they got issue in lint so they will get lint step as error so they will fix it, then they will push again so lint will run again and if pass will check other steps, what if we use separate job, it has advantage of it will show all stages that passed or failed but let's say PR has issue of lint, so lint will fail and other will success then when PR owner will push again which will run all jobs again even there were passed.

I'm ultimately ok with the latter version of this and would like the jobs split. In thinking of developer experience, especially in a project that needs to be welcoming to early stage developers, pushing to a repo and getting errors that you fix only to push and get more errors isn't something that will engender confidence in potential contributors. Yes the jobs individually will take longer, but the goal is also that they will be run fewer times as all changes can be fixed in one go. This also means less notifications for maintainers in the hopes that changes will be fixed in fewer commits. It also makes the conversation with a contributor much easier as we would be able to direct them to specific errors in a way that we can confidently say that fixes of say ktlint or detekt errors are all that's needed for the PR to be merged.

Appreciate the conversation here, and hope this makes sense!

@MahmoudMabrok
Copy link
Contributor Author

I got your point, @andrewtavis .

I did changes, but pipeline failed duo to #243 which is not related to current changes.

@MahmoudMabrok
Copy link
Contributor Author

MahmoudMabrok commented Nov 16, 2024

@andrewtavis could you check workflow, so if there is changes I would work on it.

they splitted into different jobs now, also we did a composite action to be reusable.

Copy link
Member

@andrewtavis andrewtavis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all really well done, @MahmoudMabrok :) Really appreciate the discussion here and the work towards a great result that is very much in line with the org's values 😊

@andrewtavis andrewtavis merged commit 39ce3ad into scribe-org:main Nov 16, 2024
4 checks passed
@MahmoudMabrok MahmoudMabrok deleted the ci/enhance-pipeline branch November 16, 2024 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants