You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Background
Data compactions make up a significant part of the background computational (and therefore financial) cost of Sleeper. The native implementation in Java may be accelerated by writing the compaction algorithm in a different language. We want to experiment with alternative accelerated based compaction to test this. This maybe acceleration through HW such as GPU support or through alternative software implementations.
The compactions must be compatible with the Java version, e.g. updating the state stores and file statuses of compacted files, producing quantiles sketches of the data etc.
Description
Have a configurable table property which allows for compactions to use a different method for compaction.
The text was updated successfully, but these errors were encountered:
Background
Data compactions make up a significant part of the background computational (and therefore financial) cost of Sleeper. The native implementation in Java may be accelerated by writing the compaction algorithm in a different language. We want to experiment with alternative accelerated based compaction to test this. This maybe acceleration through HW such as GPU support or through alternative software implementations.
The compactions must be compatible with the Java version, e.g. updating the state stores and file statuses of compacted files, producing quantiles sketches of the data etc.
Description
Have a configurable table property which allows for compactions to use a different method for compaction.
The text was updated successfully, but these errors were encountered: