Histogram advice

[jmh530]

@9il I've got a basic implementation of a histogram function, available here. However, I'm not entirely happy with how I am handling the types.

For the HistogramAccumulator, I took a similar approach as in that MovingAverage UT in using T[] and U[]. This requires converting the inputs in histogram to be rcarrays and then slicing them. However, then the output type is a U[] for the histogram function, and I would rather it be a slice of some kind. I'm not really sure on the best way to accomplish this and would appreciate some advice if you can think of a better way to structure it.

[9il]

bool isRightClosed, and bool includeEdgea doesn't look like required params. They can be removed though. nextUp/nextDown can be used instead by user.

We don't need an accumulator struct here. Just a function that accepts iterable or a single element.

We can add two kinds of historgram API. Functions that accept slices to fill into. And functions that gc-allocates zero slices and use the previous one functions as a low level implementation.

Two algorithms can be provided:

1. With breaks. Complexity is O(log(binCount)) for one input element
2. Without breaks (but with lower/upper/length). Complexity is O(1) for one input element

[jmh530]

Thanks!

The isRightClosed and includeEdge were influenced by R's implementation (here)

I'm not sure what you mean by nextUp/nextDown.

My intention was to first get it working with user-provided breaks and then to create functions that create breaks and integrate them somehow. The part I wasn't sure about was whether to gc-allocate the result. So I think you've made that a little more clear.

I'm not sure I completely understand your second point about an algorithm without breaks. You mean that this would take one input element and tell you if it is within lower/upper? But what is length for? To tell you where lower is within breaks?

[jmh530]

@9il I've spent some time looking at Boost Histogram. It makes some interesting design decisions. I think your #2 on the algorithms could correspond to their idea of an axis, which provides the mapping from the input to the location in the bin.

[9il]

I'm not sure what you mean by nextUp/nextDown.

http://mir-core.libmir.org/mir_math_ieee.html#.nextDown

[9il]

I'm not sure I completely understand your second point about an algorithm without breaks. You mean that this would take one input element and tell you if it is within lower/upper? But what is length for? To tell you where lower is within breaks?

I mean a uniform grid. If you have lower, upper bounds, and length, don't need to have the breaks then.

[jmh530]

Ah, Boost's Histogram has `axis::integer<>` and `axis::regular<>` can both implement a uniform grid. I have implemented the concept on my local copy (and reorganized the code significantly around this concept), though I have more to do. One nice thing about regular axis is that it can handle transformations, such as logs. It was pretty easy to set up (integer axis was even simpler).

…

On Thu, Jul 30, 2020 at 8:44 PM Ilya Yaroshenko ***@***.***> wrote: I'm not sure I completely understand your second point about an algorithm without breaks. You mean that this would take one input element and tell you if it is within lower/upper? But what is length for? To tell you where lower is within breaks? I mean a uniform grid. If you have lower, upper bounds, and length, don't need to have the breaks then. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADBFNSQN6D2PXZBPSVUZGFLR6IHV3ANCNFSM4PFTNNQA> .

[jmh530]

Worth it to consider adding functionality for this:
https://statmodeling.stat.columbia.edu/2023/04/13/the-percentogram-a-histogram-binned-by-percentages-of-the-cumulative-distribution-rather-than-using-fixed-bin-widths/