You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
lukaseder
changed the title
Avoid associative aggregations over the complete partition for [Long.MIN_VALUE .. 0] frames
Avoid associative (and identity) aggregations over the complete partition
Jan 9, 2016
Associative and identity aggregations could be calculated based on the result of the aggregation of the previous value:
As the partition is already sorted correctly, we can simply access the (cached) calculated value from
lag()
Associative and identity operations currently supported are:
count()
(optimisation not necessary, ascount()
is alreadyO(1)
)sum()
(and all derived aggregations)avg()
(and all derived aggregations - not strictly associative, but can depend onsum()
andcount()
)toList()
(a potential optimisation would require an immutable linked list)toString()
More limited optimisation is possible for associative-only aggregations, when using a
Long.MIN_VALUE .. 0
frame. These include:min()
max()
median()
(would depend on cachedtoList()
)percentile()
(would depend on cachedtoList()
)allMatch()
anyMatch()
noneMatch()
The text was updated successfully, but these errors were encountered: