Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the ability to warm start models using data from previous experiments. #135

Open
crasanders opened this issue Aug 26, 2022 · 1 comment

Comments

@crasanders
Copy link
Contributor

While data from multiple experiments can be stored in the same database, data from different experiments is currently modeled independently. Accurate models could potentially be fitted using fewer trials by seeding it with data collected from previous experiments. Users should be able to specify in either configs or messages to the server that they would like to warm start from previous data. The first iteration of this would just treat any data from previous experiments the same as data collected from the current experiment, but future iterations would implement more sophisticated hierarchical/group models, so this would need to be implemented in a generalizable way.

@mshvartsman
Copy link
Contributor

A bit more detail:

  • Once Make an AEPsych utility executable #192 lands, the remaining server executable should take a --group argument.
  • If the group argument is passed, then after a strategy is created (https://github.com/facebookresearch/aepsych/blob/main/aepsych/server/server.py#L723) the server should also call strat.add_group_data(configs, outcomes, subj_ids), where configs and outcomes are tensors after processing the raw data using config_to_tensor and whatever new functionality Support named outcomes #187 introduces for outcomes, and subj_ids is a tensor with 1 column and as many rows as there are total datapoints (i.e. same number of rows as outcomes and configs) indicating which datapoint comes from which participant id. subj_ids is 0-indexed, it does not use participant UUIDs.
  • SequentialStrategy should have an add_group_data method added. For now this method should just raise since no group models have been landed, but eventually it will instantiate the group model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants