Skip to content

Commit

Permalink
PERF-#6388: avoid masking in __getitem__ when the number of rows to b…
Browse files Browse the repository at this point in the history
…e taken > 90%

Signed-off-by: Anatoly Myachev <[email protected]>
  • Loading branch information
anmyachev committed Jul 27, 2023
1 parent 532d66e commit 15b959e
Showing 1 changed file with 13 additions and 0 deletions.
13 changes: 13 additions & 0 deletions modin/core/dataframe/pandas/dataframe/dataframe.py
Original file line number Diff line number Diff line change
Expand Up @@ -990,6 +990,19 @@ def _take_2d_positional(
if col_positions is None and row_positions is None:
return self.copy()

if col_positions is None and row_positions is not None:
# fast path
all_rows = None
if self.has_materialized_index:
all_rows = len(self.index)
elif self._row_lengths_cache:
all_rows = sum(self._row_lengths_cache)

Check warning on line 999 in modin/core/dataframe/pandas/dataframe/dataframe.py

View check run for this annotation

Codecov / codecov/patch

modin/core/dataframe/pandas/dataframe/dataframe.py#L998-L999

Added lines #L998 - L999 were not covered by tests
if all_rows:
if len(row_positions) > int(0.9 * all_rows):
return self._reorder_labels(
row_positions=row_positions, col_positions=col_positions
)

sorted_row_positions = sorted_col_positions = None

if row_positions is not None:
Expand Down

0 comments on commit 15b959e

Please sign in to comment.