Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests: Timings include initialization #33

Open
jllanfranchi opened this issue Jan 29, 2017 · 2 comments
Open

Tests: Timings include initialization #33

jllanfranchi opened this issue Jan 29, 2017 · 2 comments
Milestone

Comments

@jllanfranchi
Copy link

The initialization of the gpu histogrammer with a given ftype is included in the timings; while this is interesting on some level, it is also interesting (and moreso for PISA) seeing the timing of solely the histogramming operation itself. (We will instantiate the gpu histogrammer just once at the start of invoking PISA, probably the first time a PISA module is imported; from there on out, the same histogrammer will be used. So the init time penalty hits us just once, compared to countless uses of the histogrammer there on out.) Note that I made this change in PR #27, to only timing the histogramming and not the init.

One could also time both the init and the histogramming separately and report both. That way the number of trials won't impact the results (where the impact of init diminishes with # of trials if you include both). And then the user can make a fully informed judgement of whether it's worthwhile to use this for his/her application (and how to go about using it in a sufficiently efficient way).

@mhieronymus
Copy link
Owner

mhieronymus commented Jan 30, 2017

For every configuration (number of bins, dimensions and bins, single or double precision) the histogram object gets initialized once and several histograms are computed (tests = 10 at the moment). It would be better to create tests, where the init and the histogramming are timed separately.

Edit:
I merged your PR #27 into another branch merge_conflict. Once I get pandas working with the mess I call plot_timings we can see more detailed information. Unfortunately it will be this weekend until I have time to work on it again.

@mhieronymus mhieronymus added this to the Version 1.0 milestone Jan 30, 2017
@mhieronymus
Copy link
Owner

mhieronymus commented Feb 6, 2017

The master branch now has your optimizations. I did not include the init in the test but I might do this later. Making a test is now very easy:
python main.py --test --outdir dir
You may take a look at the updated wikipage: creating benchmarks (the image used there is still old).

Edit:
The wiki just got updated benchmarks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants