Skip to content

add 1-arg count, sum, prod, etc. methods; with tests #243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 20, 2025

Conversation

bjarthur
Copy link
Member

@bjarthur
Copy link
Member Author

tests pass. ready for review

@bjarthur
Copy link
Member Author

bjarthur commented Mar 17, 2025

hmm... should probably handle the dims and init kwargs while we're at it.

@bjarthur bjarthur changed the title add 2-arg count method, with tests add 1-arg count, sum, prod, etc. methods; with tests Mar 17, 2025
@bjarthur
Copy link
Member Author

ok, so, adding only the init kwarg to count caused the tests to fail here. must be some sort of dispatch problem since that line tries to pass in a dims kwarg.

and adding a dims kwarg is more than i can chew a.t.m, particularly given that one can simply use the dims and init kwargs of mapreduce instead.

so i reverted the init kwarg commit, and added one-arg (hah, not 2, misspoke in the original title to this PR) methods to sum, prod, etc. just above.

@bjarthur
Copy link
Member Author

tests aren't the greatest. i tried using AccessCountDiskArray but the counts were the same w/ and w/o the PR, despite @time reporting it being 10x faster and allocating 10x less memory.

@rafaqz
Copy link
Collaborator

rafaqz commented Mar 18, 2025

Yeah, dims keywords are HARD!

We could add methods for sum mean maximim etc but not for anonymous funcions or median.

@bjarthur
Copy link
Member Author

We could add methods for sum mean maximim etc but not for anonymous funcions or median.

can you please clarify? i don't understand.

@bjarthur
Copy link
Member Author

access counts are the same i now understand because without this PR the fallback is to the Generator.

@rafaqz
Copy link
Collaborator

rafaqz commented Mar 18, 2025

We could add methods for sum mean maximim etc but not for anonymous functions or median

I mean we know order doesn't matter with these methods, so we can allocate the final array size then reduce each chunk into it in the right places.

But generally we don't know that so it would be specialised.

@bjarthur
Copy link
Member Author

i figured out a way to test that the new methods do not use the generator. involves tracing the called functions, and requires two extra test dependancies. let me know what you think.

@rafaqz
Copy link
Collaborator

rafaqz commented Mar 20, 2025

That could be generally useful to have for other tests too. @meggart what do you think about test deps

@meggart
Copy link
Collaborator

meggart commented Mar 20, 2025

I have no problem to add these as test dependencies and always curious to learn, looks useful and indeed we should use this in other test contexts as well.

@rafaqz rafaqz merged commit 6b2fa78 into JuliaIO:main Mar 20, 2025
12 checks passed
@bjarthur bjarthur deleted the bja/count branch March 21, 2025 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants