Skip to content

Commit

Permalink
Add additional info about memory reservation to the doc of MemoryPool (
Browse files Browse the repository at this point in the history
  • Loading branch information
Kontinuation authored Feb 20, 2025
1 parent 568a2b1 commit 84232d8
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion datafusion/execution/src/memory_pool/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,10 @@ pub use pool::*;
/// "large" amounts of memory (proportional to number of input rows), such as
/// `GroupByHashExec`. It does NOT track and limit memory used internally by
/// other operators such as `DataSourceExec` or the `RecordBatch`es that flow
/// between operators.
/// between operators. Furthermore, operators should not reserve memory for the
/// batches they produce. Instead, if a parent operator needs to hold batches
/// from its children in memory for an extended period, it is the parent
/// operator's responsibility to reserve the necessary memory for those batches.
///
/// In order to avoid allocating memory until the OS or the container system
/// kills the process, DataFusion `ExecutionPlan`s (operators) that consume
Expand Down

0 comments on commit 84232d8

Please sign in to comment.