Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix unstable SemaphoreCI tests #2639

Merged
merged 16 commits into from
Nov 2, 2024
Merged

Conversation

ioanmo226
Copy link
Collaborator

@ioanmo226 ioanmo226 commented Oct 23, 2024

This PR fixed unstable SemaphoreCI tests.

close #2638 // if this PR closes an issue


Tests (delete all except exactly one):

  • Tests added or updated

To be filled by reviewers

I have reviewed that this PR... (tick whichever items you personally focused on during this review):

  • addresses the issue it closes (if any)
  • code is readable and understandable
  • is accompanied with tests, or tests are not needed
  • is free of vulnerabilities
  • is documented clearly and usefully, or doesn't need documentation

@ioanmo226 ioanmo226 marked this pull request as ready for review November 1, 2024 09:48
await KeysScreen.checkKeysScreen([ekmKeySamples.key0]);
// TODO: Temporarily disable the logic below as the cancel button on the refresh screen
// is not functioning correctly in Semaphore CI for some reason.
// https://flowcrypt.semaphoreci.com/workflows/7310b5b7-d544-428c-82ce-7f7f060e7e60?pipeline_id=186f2ad1-5d22-44d3-ba07-76deaf3ea49f
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sosnovsky Have any idea why this test fails (can't click button even though elements are visible on the screen)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be caused by some changes in iOS behavior which were adapted in the newer Appium version.
After upgrading to the latest Appium we can re-check if it works there.

@ioanmo226
Copy link
Collaborator Author

@sosnovsky I’ve fixed most of the unstable tests, but in a few cases, they still occasionally fail on SemaphoreCI.

The main issue seems to be a 2-hour timeout limit we set in SemaphoreCI config, where tests get stuck on the home screen (with a list of apps in a closed state) or on the loading progress HUD.
Looks like we need to live with this until SemaphoreCI upgrades to M2 or M3 Macs.

Let me know your thoughts

Copy link
Collaborator

@sosnovsky sosnovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for checking out, hope it'll help to make tests passing in the new PRs 👍

@sosnovsky sosnovsky merged commit e03d802 into master Nov 2, 2024
11 checks passed
@sosnovsky sosnovsky deleted the 2638-unstable-semaphoreci-tests branch November 2, 2024 19:52
@sosnovsky
Copy link
Collaborator

The main issue seems to be a 2-hour timeout limit we set in SemaphoreCI config, where tests get stuck on the home screen (with a list of apps in a closed state) or on the loading progress HUD.
Looks like we need to live with this until SemaphoreCI upgrades to M2 or M3 Macs.

It should be ok for now, as previously it was almost impossible to make all tests pass.
Let's hope Semaphore will start using M3 machines soon, which should greatly improve tests performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unstable SemaphoreCI tests
2 participants