Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update tutorial execution to eliminate SIGTERM #1830

Closed
wants to merge 12 commits into from
Closed

Conversation

saitcakmak
Copy link
Contributor

@saitcakmak saitcakmak commented Sep 8, 2023

The current tutorial script hangs for a while and produces

*** SIGTERM received at time=1694205109 on cpu 0 ***
PC: @     0x7f5698ffcfde  (unknown)  epoll_wait
    @     0x7f5698f19520  (unknown)  (unknown)
[2023-09-08 20:31:49,723 E 2808 2808] logging.cc:361: *** SIGTERM received at time=1694205109 on cpu 0 ***
[2023-09-08 20:31:49,723 E 2808 2808] logging.cc:361: PC: @     0x7f5698ffcfde  (unknown)  epoll_wait
[2023-09-08 20:31:49,723 E 2808 2808] logging.cc:361:     @     0x7f5698f19520  (unknown)  (unknown)

before moving onto executing the tutorials. Identified ExecutePreprocessor as the source of this and replaced it with papermill (code borrowed from BoTorch) to eliminate the issue.

Example workflow before this diff: https://github.com/facebook/Ax/actions/runs/6125894824/job/16628813894
Workflow with this diff: https://github.com/facebook/Ax/actions/runs/6126463133/job/16630514050
The new one ran significantly faster!

@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Sep 8, 2023
@facebook-github-bot
Copy link
Contributor

@saitcakmak has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@saitcakmak has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

1 similar comment
@facebook-github-bot
Copy link
Contributor

@saitcakmak has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@saitcakmak has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

1 similar comment
@facebook-github-bot
Copy link
Contributor

@saitcakmak has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@codecov-commenter
Copy link

codecov-commenter commented Sep 8, 2023

Codecov Report

Patch and project coverage have no change.

Comparison is base (5da3e55) 94.57% compared to head (0891ebb) 94.57%.
Report is 1 commits behind head on main.

❗ Current head 0891ebb differs from pull request most recent head b1c6993. Consider uploading reports for the commit b1c6993 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1830   +/-   ##
=======================================
  Coverage   94.57%   94.57%           
=======================================
  Files         456      456           
  Lines       43010    43010           
=======================================
  Hits        40677    40677           
  Misses       2333     2333           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@facebook-github-bot
Copy link
Contributor

@saitcakmak merged this pull request in eea5c03.

@Balandat Balandat deleted the debug-sigterm branch September 9, 2023 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Do not delete this pull request or issue due to inactivity. Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants