Skip to content

Commit

Permalink
Minor: Improve ExecutionMode documentation (#12214)
Browse files Browse the repository at this point in the history
* Minor: Improve ExecutionMode documentation

* Update datafusion/physical-plan/src/execution_plan.rs

Co-authored-by: Mehmet Ozan Kabak <[email protected]>

* Update datafusion/physical-plan/src/execution_plan.rs

Co-authored-by: Mehmet Ozan Kabak <[email protected]>

---------

Co-authored-by: Mehmet Ozan Kabak <[email protected]>
  • Loading branch information
alamb and ozankabak authored Sep 1, 2024
1 parent ff7d76d commit 780cccb
Showing 1 changed file with 32 additions and 13 deletions.
45 changes: 32 additions & 13 deletions datafusion/physical-plan/src/execution_plan.rs
Original file line number Diff line number Diff line change
Expand Up @@ -482,24 +482,43 @@ impl ExecutionPlanProperties for &dyn ExecutionPlan {
}
}

/// Describes the execution mode of an operator's resulting stream with respect
/// to its size and behavior. There are three possible execution modes: `Bounded`,
/// `Unbounded` and `PipelineBreaking`.
/// Describes the execution mode of the result of calling
/// [`ExecutionPlan::execute`] with respect to its size and behavior.
///
/// The mode of the execution plan is determined by the mode of its input
/// execution plans and the details of the operator itself. For example, a
/// `FilterExec` operator will have the same execution mode as its input, but a
/// `SortExec` operator may have a different execution mode than its input,
/// depending on how the input stream is sorted.
///
/// There are three possible execution modes: `Bounded`, `Unbounded` and
/// `PipelineBreaking`.
#[derive(Clone, Copy, PartialEq, Debug)]
pub enum ExecutionMode {
/// Represents the mode where generated stream is bounded, e.g. finite.
/// The stream is bounded / finite.
///
/// In this case the stream will eventually return `None` to indicate that
/// there are no more records to process.
Bounded,
/// Represents the mode where generated stream is unbounded, e.g. infinite.
/// Even though the operator generates an unbounded stream of results, it
/// works with bounded memory and execution can still continue successfully.
/// The stream is unbounded / infinite.
///
/// The stream that results from calling `execute` on an `ExecutionPlan` that is `Unbounded`
/// will never be done (return `None`), except in case of error.
/// In this case, the stream will never be done (never return `None`),
/// except in case of error.
///
/// This mode is often used in "Steaming" use cases where data is
/// incrementally processed as it arrives.
///
/// Note that even though the operator generates an unbounded stream of
/// results, it can execute with bounded memory and incrementally produces
/// output.
Unbounded,
/// Represents the mode where some of the operator's input stream(s) are
/// unbounded; however, the operator cannot generate streaming results from
/// these streaming inputs. In this case, the execution mode will be pipeline
/// breaking, e.g. the operator requires unbounded memory to generate results.
/// Some of the operator's input stream(s) are unbounded, but the operator
/// cannot generate streaming results from these streaming inputs.
///
/// In this case, the execution mode will be pipeline breaking, e.g. the
/// operator requires unbounded memory to generate results. This
/// information is used by the planner when performing sanity checks
/// on plans processings unbounded data sources.
PipelineBreaking,
}

Expand Down

0 comments on commit 780cccb

Please sign in to comment.