default_capacity as a compile time define?

[mvphilip]

Feature category

• enhancement - indicators
• enhancement - data structures
• enhancement - bindings
• enhancement - build system
• enhancement - documentation

The problem
Hello Alan. This is a great library. Thank you for releasing it.
Can you please make the default_capacity of 1000 as a compile time
define?

static constexpr size_t default_capacity =
number_of_compile_dimensions == 0 ? 1000
: number_of_compile_dimensions <= 10
? static_cast(50)
<< (number_of_compile_dimensions - 1)
: 100000;

I ran into an issue where I exceeded the 1000 limit and end up asserting when _size != total_front_size().
It took some tracing to see why there was an assert as I didn't know of the 1000 limit.

The solution I'd like
#ifndef DEFAULT_CAPACITY
#define DEFAULT_CAPACITY 1000
#endif

static constexpr size_t default_capacity =
number_of_compile_dimensions == 0 ? DEFAULT_CAPACITY
: number_of_compile_dimensions <= 10
? static_cast(50)
<< (number_of_compile_dimensions - 1)
: 100000;

Alternatives I've considered
None

Additional context

[alandefreitas]

Hi @mvphilip

Thanks for bringing this up. Yes. It's completely fine to create this macro. But I'm afraid it might not solve your problem yet.

This define is only used when the dimension is set at runtime. This is represented with the sentinel number_of_compile_dimensions == 0 and the default capacity is 1000, since we don't know how many dimensions the user is going to define later.

If you exceeded the capacity with the default capacity, then are probably defining the dimension at compile-time with C++, in which case your default capacity is being set to static_cast(50) << (number_of_compile_dimensions - 1), or $50^m$.

Indeed, a default capacity of max(50^m, 10000) was a stupid decision, as 50^9 = 1.95*10^15.
I made that decision quickly and it ended up not influencing the experiments so I forgot about it.

The intuition is reasonable though. The search space increases exponentially, so you probably want to be able to represent an exponentially larger number of solutions, up to a limit.

A better default would be $75*2^m$. So we have the common 300 elements for two dimensions and let it grow more reasonably up to the limit of 100000.

number_of_compile_dimensions <= 10 ? static_cast(75) * 2 << (number_of_compile_dimensions - 1) : 100000;

and then you can later adjust that with the capacity() function.

Let me know if that sounds ok? This change is quite simple, so also feel free to open a PR if you want.

[mvphilip]

Hello Alan, Thank you for the prompt reply. I am indeed using the archive container in a runtime fashion. It's part of reporting system that cannot know the dimension of the backend data until it actually loads the data.

I will take a stab at a PR and get back to you.

thanks,

-mp

[alandefreitas]

If you set number_of_compile_dimensions to 0, then you shouldn't have any problem with the default capacity 1000. You might have a better look at that to make sure it solves your problem.