ROBOT tutorial notebook #2883
ROBOT tutorial notebook #2883
Conversation
facebook-github-bot
added
the
CLA Signed
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2883 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 211 211
Lines 19512 19512
=========================================
Hits 19512 19512 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Nice, thank you for submitting this! About how long does this notebook take to run? I think this probably belongs in notebooks_community since I'm not confident that the BoTorch team can promise to maintain this or run it regularly in the CI.
Edit: But feel free to keep it where it is for now so that we can keep getting CI signals on this -- seems like it's not running successfully now.
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "def delta(x1, x2 ):\n", |
There was a problem hiding this comment.
type annotations would be helpful for readability
There was a problem hiding this comment.
Agreed, thanks for adding those, I've committed your suggested changes.
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ |
There was a problem hiding this comment.
Can we print less frequently?
There was a problem hiding this comment.
Yeah we could for example only print when the average reward improves, although I think can also be helpful for people to see an update on every iteration? I.e. in other tutorials such as turbo tutorial (https://github.com/pytorch/botorch/blob/main/tutorials/turbo_1/turbo_1.ipynb) there is a progress update printed on every iteration.
There was a problem hiding this comment.
I don't feel too strongly about it -- I think if you're reading the tutorial it's nice for the output to be more minimal, but if you're running this yourself on your own data, it's nice to see verbose output
There was a problem hiding this comment.
As a compromise, I modified to print every 10 iterations instead of every iteration 👍
Co-authored-by: Elizabeth Santorella <[email protected]>
… into robot-tutorial
Co-authored-by: Elizabeth Santorella <[email protected]>
Co-authored-by: Elizabeth Santorella <[email protected]>
Co-authored-by: Elizabeth Santorella <[email protected]>
Co-authored-by: Elizabeth Santorella <[email protected]>
Co-authored-by: Elizabeth Santorella <[email protected]>
|
@esantorella, thank you for reviewing this! Regarding your concern about it taking too long to run, I agree and added a SMOKE_TEST option so that in SMOKE_TEST mode it only iterates a couple of times and completes much faster. Given that update I think it should be okay to keep this in tutorials/ ? |
Sounds reasonable to me since it isn't using any functionality from botorch_community. |
|
@esantorella has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Motivation
This PR adds a tutorial notebook for the Rank Ordered Bayesian Optimization with Trust Regions (ROBOT) method proposed by the AISTATS 2023 paper titled Discovering Many Diverse Solutions with Bayesian Optimization. The motivation for the notebook is to provide a simple clean example of how the ROBOT method can be applied in practice to optimize a set of diverse solutions to a black-box objective function.
Have you read the Contributing Guidelines on pull requests?
Yes.
Test Plan
The only change made is to add an isolated Jupyter notebook (robot.ipynb) to the tutorials folder. I checked that the notebook runs as it should and produces desired outputs. Since none of the BoTorch code was actually changed at all (all I did was add this notebook), no other tests are required as far as I am aware.
Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/pytorch/botorch, and link to your PR here.)