Includes all deferred conversion costs in benchmarks #34
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…fully evaluate so benchmark includes all costs.
rlratzel
added
improvement
eriknw
approved these changes
Nov 20, 2024
| @@ -489,7 +502,7 @@ def bench_shortest_path(benchmark, graph_obj, backend_wrapper): | |||
| node = get_highest_degree_node(graph_obj) | |||
|
|
|||
| result = benchmark.pedantic( | |||
| target=backend_wrapper(nx.shortest_path), | |||
| target=backend_wrapper(nx.shortest_path, force_unlazy_eval=True), | |||
There was a problem hiding this comment.
I think it would be nice to be able to run this benchmark with both force_unlazy_eval=True and force_unlazy_eval=False.
There was a problem hiding this comment.
I think it would be nice to be able to run this benchmark with both
force_unlazy_eval=Trueandforce_unlazy_eval=False.
Yes I agree that's useful. I'll add a FIXME/issue to parameterize that. FWIW, here's the results with/without when I ran locally:
Fortunately run times don't have a noticeable change until the hollywood (57M edges) or larger datasets.
nv-rliu
approved these changes
Nov 20, 2024
…2-benchmarks_update
…lratzel/nx-cugraph into branch-24.12-benchmarks_update
|
/merge |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This refactors the helper that returns the callable to benchmark to fully expand Mapping return types. This is needed in case those return types lazily convert values from device to host, so the complete cost can be included in the measured/benchmarked run. This is similar to exhausting returned generators that deferred computations in NetworkX, and doing this keeps us consistent with benchmarking the full compute and/or conversion costs for both libraries.
I changed the option name from
exhaust_returned_iteratortoforce_unlazy_evalsince this is no longer being used only for iterators, but I'm taking suggestions for a better name.