org.apache.spark.sql.execution

Class Limit

Object

org.apache.spark.sql.catalyst.trees.TreeNode<PlanType>

org.apache.spark.sql.catalyst.plans.QueryPlan<SparkPlan>

org.apache.spark.sql.execution.SparkPlan

org.apache.spark.sql.execution.Limit

All Implemented Interfaces:

com.typesafe.scalalogging.slf4j.Logging, java.io.Serializable, scala.Equals, scala.Product

public class Limit
extends SparkPlan
implements scala.Product, scala.Serializable

:: DeveloperApi :: Take the first limit elements. Note that the implementation is different depending on whether this is a terminal operator or not. If it is terminal and is invoked using executeCollect, this operator uses something similar to Spark's take method on the Spark driver. If it is not terminal or is invoked using execute, we first take the limit on each partition, and then repartition all the data to a single partition to compute the global limit.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
Limit(int limit, SparkPlan child, SQLContext sqlContext)

Method Summary

Methods

Modifier and Type	Method and Description
SparkPlan	child()
RDD<org.apache.spark.sql.catalyst.expressions.Row>	execute() Runs this query returning the result as an RDD.
org.apache.spark.sql.catalyst.expressions.Row[]	executeCollect() A custom implementation modeled after the take function on RDDs but which never runs any job locally.
int	limit()
scala.collection.immutable.List<SQLContext>	otherCopyArgs()
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute>	output()
org.apache.spark.sql.catalyst.plans.physical.Partitioning	outputPartitioning()

Methods inherited from class org.apache.spark.sql.execution.SparkPlan

outputPartitioning, requiredChildDistribution

Methods inherited from class org.apache.spark.sql.catalyst.plans.QueryPlan

expressions, generateSchemaString, generateSchemaString, org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionDown$1, org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionUp$1, outputSet, printSchema, schemaString, transformAllExpressions, transformExpressions, transformExpressionsDown, transformExpressionsUp

Methods inherited from class org.apache.spark.sql.catalyst.trees.TreeNode

apply, argString, asCode, children, collect, fastEquals, flatMap, foreach, generateTreeString, getNodeNumbered, id, makeCopy, map, mapChildren, nextId, nodeName, numberedTreeString, sameInstance, simpleString, stringArgs, toString, transform, transformChildrenDown, transformChildrenUp, transformDown, transformUp, treeString, withNewChildren

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.catalyst.trees.UnaryNode

children

Methods inherited from interface scala.Product

productArity, productElement, productIterator, productPrefix

Methods inherited from interface scala.Equals

canEqual, equals

Methods inherited from interface com.typesafe.scalalogging.slf4j.Logging

logger

Constructor Detail

Limit

public Limit(int limit,
     SparkPlan child,
     SQLContext sqlContext)

Method Detail

limit

public int limit()

child

public SparkPlan child()

otherCopyArgs

public scala.collection.immutable.List<SQLContext> otherCopyArgs()

output

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> output()

Specified by:

output in class org.apache.spark.sql.catalyst.plans.QueryPlan<SparkPlan>

executeCollect

public org.apache.spark.sql.catalyst.expressions.Row[] executeCollect()

A custom implementation modeled after the take function on RDDs but which never runs any job locally. This is to avoid shipping an entire partition of data in order to retrieve only a few rows.

Overrides:

executeCollect in class SparkPlan

execute

public RDD<org.apache.spark.sql.catalyst.expressions.Row> execute()

Description copied from class: SparkPlan

Runs this query returning the result as an RDD.

Specified by:

execute in class SparkPlan

outputPartitioning

public org.apache.spark.sql.catalyst.plans.physical.Partitioning outputPartitioning()