Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Adding
auxiliary_experiments_by_purpose
to COPY_DB_IDS_ATTRS_TO_SKIP (
#2934) Summary: Pull Request resolved: #2934 Adding `auxiliary_experiments_by_purpose` to COPY_DB_IDS_ATTRS_TO_SKIP. ## Context: In the follow-up diff D64281332, I'm making changes to make [play_preference_game](https://www.internalfb.com/code/fbsource/[e706ec5635b4cb836d54c8ade1dfddf297806aab]/fbcode/ax/fb/utils/preference/preference.py?lines=499) use [get_preference_exploration_gs](https://www.internalfb.com/code/fbsource/[d119de48295da9f1682c1b1bc1a632abd0f3605c]/fbcode/ax/fb/modelbridge/generation_strategies.py?lines=1558) instead of its own custom logic. The error this diff is trying to fix happens *after* the preference game is finished and [when we try to save the PE experiment](https://www.internalfb.com/code/fbsource/[b8af5cabad15579a0e81776a78b43b144976b420]/fbcode/ax/fb/utils/preference/preference.py?lines=393)(the link is based on v0.5 of D64281332). The logic inside `interactive_optimize` used there might be a bit convoluted with local async candidate generation, but the (synchrounous version) logic's pseudo code looks like this: ``` pe_experiment["bo_aux_exp"] = [bo_exp] gs = get_preference_exploration_gs(pe_experiment) for i in (nums_pref_questions): pref_game_arms = gs.gen(n=2) user_response = elicit_user_preference(pref_game_arms) pe_experiment.new_batch_trial(user_response) save_experiment(pe_experiment) save_experiment(pe_experiment) # <- the error happens ``` The error we see is (stack trace pasted at: P1656339174): >SQADecodeError: Error encountered while traversing source ['auxiliary_experiments_by_purpose', <AuxiliaryExperimentPurposeFB.BO_EXPERIMENT: 'bo_experiment'>, '0', 'experiment', 'runner', 'None'] and target ['auxiliary_experiments_by_purpose', <AuxiliaryExperimentPurposeFB.BO_EXPERIMENT: 'bo_experiment'>, '0', 'experiment', 'runner', '<ax.fb.runners.json.JSONRunner object at 0x7f17a6997f10>']: Encountered two objects of different types: <class 'NoneType'> and <class 'ax.fb.runners.json.JSONRunner'>.Consider adding to COPY_DB_IDS_ATTRS_TO_SKIP if appropriate. I'm not entirely sure where the runner in the new object is coming from as I have never explicitly set the runner of the BO experiment, and the [trial (of the PE experiment) is marked running with `no_runner_required=True`](https://www.internalfb.com/code/fbsource/[b8af5cabad15579a0e81776a78b43b144976b420]/fbcode/ax/fb/utils/preference/preference.py?lines=185) (the link is based on v0.5 of D64281332). Note that the aux experiment is the BO experiment, which is not expected to change at all during the course of the preference game. There are other related errors that happened in `copy_db_ids` that I was able to get rid of (e.g., by manually dropping the [place-holder optimization_config that we are not using](https://www.internalfb.com/diff/D63794133) in the [generated GR](https://www.internalfb.com/code/fbsource/[b8af5cabad15579a0e81776a78b43b144976b420]/fbcode/ax/fb/modelbridge/preference.py?lines=309)), but it seems likt to fix this one, the more appropriate solution is to add `auxiliary_experiments_by_purpose` to `COPY_DB_IDS_ATTRS_TO_SKIP`. The alternative would be to figure out where the runner is coming from and manually remove that before saving. However, for a real experiment, we probably don't wanna remove the runner of the BO experiment since we'd need that to deploy the QE experiment, for example. Reviewed By: lena-kashtelyan Differential Revision: D64520734 fbshipit-source-id: 7c5675c633615047391afa482cd70610b299850a
- Loading branch information