Do not support serializing BenchmarkRunners #2889
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D64272347 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Oct 16, 2024
Summary: Context: As of D64263610, serializing `BenchmarkProblem`s and `BenchmarkRunner`s is not needed, and in the future, we will make changes that are difficult to make work well with `SerializationMixin`. Therefore, this diff revokes support for serializating `BenchmarkProblem`s and `BenchmarkRunner`s. *But what if we want to serialize these?* IMO, we should use the benchmark problem registry to reconstruct these, which will be much more ergonomic. You might worry that it's expensive to reconstruct problems from scratch, but there was never any computational efficiency benefit from serialization rather than rebuilding from scratch, because the most expensive components (surrogates and data) could not be serialized anyway. This diff: * Strips the `BenchmarkRunner` from an `Experiment` before saving the experiment -- that probably should not have been saved anyway * Makes all `BenchmarkRunners` throw exceptions if their `serialize_int_args` or `deserialize_init_args` methods are used * Removes `SyntheticProblemRunner`'s serialization methods * Does the same for `SurrogateRunner` (which only had fake serialization anyway) Reviewed By: Balandat Differential Revision: D64272347
esantorella
force-pushed
the
export-D64272347
branch
from
8acafc9
to
4fb12b9
Compare
|
This pull request was exported from Phabricator. Differential Revision: D64272347 |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2889 +/- ##
=======================================
Coverage 95.78% 95.79%
=======================================
Files 504 504
Lines 49829 49799 -30
=======================================
- Hits 47730 47703 -27
+ Misses 2099 2096 -3 ☔ View full report in Codecov by Sentry. |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Oct 16, 2024
Summary: Context: As of D64263610, serializing `BenchmarkProblem`s and `BenchmarkRunner`s is not needed, and in the future, we will make changes that are difficult to make work well with `SerializationMixin`. Therefore, this diff revokes support for serializating `BenchmarkProblem`s and `BenchmarkRunner`s. *But what if we want to serialize these?* IMO, we should use the benchmark problem registry to reconstruct these, which will be much more ergonomic. You might worry that it's expensive to reconstruct problems from scratch, but there was never any computational efficiency benefit from serialization rather than rebuilding from scratch, because the most expensive components (surrogates and data) could not be serialized anyway. This diff: * Strips the `BenchmarkRunner` from an `Experiment` before saving the experiment -- that probably should not have been saved anyway * Makes all `BenchmarkRunners` throw exceptions if their `serialize_int_args` or `deserialize_init_args` methods are used * Removes `SyntheticProblemRunner`'s serialization methods * Does the same for `SurrogateRunner` (which only had fake serialization anyway) Reviewed By: Balandat Differential Revision: D64272347
esantorella
force-pushed
the
export-D64272347
branch
from
4fb12b9
to
44631a4
Compare
|
This pull request was exported from Phabricator. Differential Revision: D64272347 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Oct 16, 2024
Summary: Context: As of D64263610, serializing `BenchmarkProblem`s and `BenchmarkRunner`s is not needed, and in the future, we will make changes that are difficult to make work well with `SerializationMixin`. Therefore, this diff revokes support for serializating `BenchmarkProblem`s and `BenchmarkRunner`s. *But what if we want to serialize these?* IMO, we should use the benchmark problem registry to reconstruct these, which will be much more ergonomic. You might worry that it's expensive to reconstruct problems from scratch, but there was never any computational efficiency benefit from serialization rather than rebuilding from scratch, because the most expensive components (surrogates and data) could not be serialized anyway. This diff: * Strips the `BenchmarkRunner` from an `Experiment` before saving the experiment -- that probably should not have been saved anyway * Makes all `BenchmarkRunners` throw exceptions if their `serialize_int_args` or `deserialize_init_args` methods are used * Removes `SyntheticProblemRunner`'s serialization methods * Does the same for `SurrogateRunner` (which only had fake serialization anyway) Reviewed By: Balandat Differential Revision: D64272347
esantorella
force-pushed
the
export-D64272347
branch
from
44631a4
to
2c455e3
Compare
|
This pull request was exported from Phabricator. Differential Revision: D64272347 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Oct 16, 2024
Summary: Context: As of D64263610, serializing `BenchmarkProblem`s and `BenchmarkRunner`s is not needed, and in the future, we will make changes that are difficult to make work well with `SerializationMixin`. Therefore, this diff revokes support for serializating `BenchmarkProblem`s and `BenchmarkRunner`s. *But what if we want to serialize these?* IMO, we should use the benchmark problem registry to reconstruct these, which will be much more ergonomic. You might worry that it's expensive to reconstruct problems from scratch, but there was never any computational efficiency benefit from serialization rather than rebuilding from scratch, because the most expensive components (surrogates and data) could not be serialized anyway. This diff: * Strips the `BenchmarkRunner` from an `Experiment` before saving the experiment -- that probably should not have been saved anyway * Makes all `BenchmarkRunners` throw exceptions if their `serialize_int_args` or `deserialize_init_args` methods are used * Removes `SyntheticProblemRunner`'s serialization methods * Does the same for `SurrogateRunner` (which only had fake serialization anyway) Reviewed By: Balandat Differential Revision: D64272347
esantorella
force-pushed
the
export-D64272347
branch
from
2c455e3
to
a1fcc6a
Compare
|
This pull request was exported from Phabricator. Differential Revision: D64272347 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Oct 16, 2024
Summary: Context: As of D64263610, serializing `BenchmarkProblem`s and `BenchmarkRunner`s is not needed, and in the future, we will make changes that are difficult to make work well with `SerializationMixin`. Therefore, this diff revokes support for serializating `BenchmarkProblem`s and `BenchmarkRunner`s. *But what if we want to serialize these?* IMO, we should use the benchmark problem registry to reconstruct these, which will be much more ergonomic. You might worry that it's expensive to reconstruct problems from scratch, but there was never any computational efficiency benefit from serialization rather than rebuilding from scratch, because the most expensive components (surrogates and data) could not be serialized anyway. This diff: * Strips the `BenchmarkRunner` from an `Experiment` before saving the experiment -- that probably should not have been saved anyway * Makes all `BenchmarkRunners` throw exceptions if their `serialize_int_args` or `deserialize_init_args` methods are used * Removes `SyntheticProblemRunner`'s serialization methods * Does the same for `SurrogateRunner` (which only had fake serialization anyway) Reviewed By: Balandat Differential Revision: D64272347
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Oct 16, 2024
Summary: Context: As of D64263610, serializing `BenchmarkProblem`s and `BenchmarkRunner`s is not needed, and in the future, we will make changes that are difficult to make work well with `SerializationMixin`. Therefore, this diff revokes support for serializating `BenchmarkProblem`s and `BenchmarkRunner`s. *But what if we want to serialize these?* IMO, we should use the benchmark problem registry to reconstruct these, which will be much more ergonomic. You might worry that it's expensive to reconstruct problems from scratch, but there was never any computational efficiency benefit from serialization rather than rebuilding from scratch, because the most expensive components (surrogates and data) could not be serialized anyway. This diff: * Strips the `BenchmarkRunner` from an `Experiment` before saving the experiment -- that probably should not have been saved anyway * Makes all `BenchmarkRunners` throw exceptions if their `serialize_int_args` or `deserialize_init_args` methods are used * Removes `SyntheticProblemRunner`'s serialization methods * Does the same for `SurrogateRunner` (which only had fake serialization anyway) Reviewed By: Balandat Differential Revision: D64272347
Summary: Context: As of D64263610, serializing `BenchmarkProblem`s and `BenchmarkRunner`s is not needed, and in the future, we will make changes that are difficult to make work well with `SerializationMixin`. Therefore, this diff revokes support for serializating `BenchmarkProblem`s and `BenchmarkRunner`s. *But what if we want to serialize these?* IMO, we should use the benchmark problem registry to reconstruct these, which will be much more ergonomic. You might worry that it's expensive to reconstruct problems from scratch, but there was never any computational efficiency benefit from serialization rather than rebuilding from scratch, because the most expensive components (surrogates and data) could not be serialized anyway. This diff: * Strips the `BenchmarkRunner` from an `Experiment` before saving the experiment -- that probably should not have been saved anyway * Makes all `BenchmarkRunners` throw exceptions if their `serialize_int_args` or `deserialize_init_args` methods are used * Removes `SyntheticProblemRunner`'s serialization methods * Does the same for `SurrogateRunner` (which only had fake serialization anyway) Reviewed By: Balandat Differential Revision: D64272347
esantorella
force-pushed
the
export-D64272347
branch
from
a1fcc6a
to
2aac283
Compare
|
This pull request was exported from Phabricator. Differential Revision: D64272347 |
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Oct 17, 2024
Summary: Context: As of D64263610, serializing `BenchmarkProblem`s and `BenchmarkRunner`s is not needed, and in the future, we will make changes that are difficult to make work well with `SerializationMixin`. Therefore, this diff revokes support for serializating `BenchmarkProblem`s and `BenchmarkRunner`s. *But what if we want to serialize these?* IMO, we should use the benchmark problem registry to reconstruct these, which will be much more ergonomic. You might worry that it's expensive to reconstruct problems from scratch, but there was never any computational efficiency benefit from serialization rather than rebuilding from scratch, because the most expensive components (surrogates and data) could not be serialized anyway. This diff: * Strips the `BenchmarkRunner` from an `Experiment` before saving the experiment -- that probably should not have been saved anyway * Makes all `BenchmarkRunners` throw exceptions if their `serialize_int_args` or `deserialize_init_args` methods are used * Removes `SyntheticProblemRunner`'s serialization methods * Does the same for `SurrogateRunner` (which only had fake serialization anyway) Reviewed By: Balandat Differential Revision: D64272347
esantorella
added a commit
to esantorella/Ax
that referenced
this pull request
Oct 17, 2024
Summary: Context: As of D64263610, serializing `BenchmarkProblem`s and `BenchmarkRunner`s is not needed, and in the future, we will make changes that are difficult to make work well with `SerializationMixin`. Therefore, this diff revokes support for serializating `BenchmarkProblem`s and `BenchmarkRunner`s. *But what if we want to serialize these?* IMO, we should use the benchmark problem registry to reconstruct these, which will be much more ergonomic. You might worry that it's expensive to reconstruct problems from scratch, but there was never any computational efficiency benefit from serialization rather than rebuilding from scratch, because the most expensive components (surrogates and data) could not be serialized anyway. This diff: * Strips the `BenchmarkRunner` from an `Experiment` before saving the experiment -- that probably should not have been saved anyway * Makes all `BenchmarkRunners` throw exceptions if their `serialize_int_args` or `deserialize_init_args` methods are used * Removes `SyntheticProblemRunner`'s serialization methods * Does the same for `SurrogateRunner` (which only had fake serialization anyway) Reviewed By: Balandat Differential Revision: D64272347
|
This pull request has been merged in 37e6f54. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Context: As of D64263610, serializing
BenchmarkProblems andBenchmarkRunners is not needed, and in the future, we will make changes that are difficult to make work well withSerializationMixin. Therefore, this diff revokes support for serializatingBenchmarkProblems andBenchmarkRunners.But what if we want to serialize these? IMO, we should use the benchmark problem registry to reconstruct these, which will be much more ergonomic. You might worry that it's expensive to reconstruct problems from scratch, but there was never any computational efficiency benefit from serialization rather than rebuilding from scratch, because the most expensive components (surrogates and data) could not be serialized anyway.
This diff:
BenchmarkRunnerfrom anExperimentbefore saving the experiment -- that probably should not have been saved anywayBenchmarkRunnersthrow exceptions if theirserialize_int_argsordeserialize_init_argsmethods are usedSyntheticProblemRunner's serialization methodsSurrogateRunner(which only had fake serialization anyway)Reviewed By: Balandat
Differential Revision: D64272347