org.apache.spark.sql.parquet

Class ParquetTableScan

Object

org.apache.spark.sql.catalyst.trees.TreeNode<PlanType>

org.apache.spark.sql.catalyst.plans.QueryPlan<SparkPlan>

org.apache.spark.sql.execution.SparkPlan

org.apache.spark.sql.parquet.ParquetTableScan

All Implemented Interfaces:

com.typesafe.scalalogging.slf4j.Logging, java.io.Serializable, scala.Equals, scala.Product

public class ParquetTableScan
extends SparkPlan
implements scala.Product, scala.Serializable

Parquet table scan operator. Imports the file that backs the given ParquetRelation as a RDD[Row].

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ParquetTableScan(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> output, org.apache.spark.sql.parquet.ParquetRelation relation, scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> columnPruningPred, SQLContext sqlContext)

Method Summary

Methods

Modifier and Type	Method and Description
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression>	columnPruningPred()
RDD<org.apache.spark.sql.catalyst.expressions.Row>	execute() Runs this query returning the result as an RDD.
scala.collection.immutable.List<SQLContext>	otherCopyArgs()
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute>	output()
ParquetTableScan	pruneColumns(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> prunedAttributes) Applies a (candidate) projection.
org.apache.spark.sql.parquet.ParquetRelation	relation()
SQLContext	sqlContext()

Methods inherited from class org.apache.spark.sql.execution.SparkPlan

executeCollect, outputPartitioning, requiredChildDistribution

Methods inherited from class org.apache.spark.sql.catalyst.plans.QueryPlan

expressions, generateSchemaString, generateSchemaString, org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionDown$1, org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionUp$1, outputSet, printSchema, schemaString, transformAllExpressions, transformExpressions, transformExpressionsDown, transformExpressionsUp

Methods inherited from class org.apache.spark.sql.catalyst.trees.TreeNode

apply, argString, asCode, children, collect, fastEquals, flatMap, foreach, generateTreeString, getNodeNumbered, id, makeCopy, map, mapChildren, nextId, nodeName, numberedTreeString, sameInstance, simpleString, stringArgs, toString, transform, transformChildrenDown, transformChildrenUp, transformDown, transformUp, treeString, withNewChildren

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface scala.Product

productArity, productElement, productIterator, productPrefix

Methods inherited from interface scala.Equals

canEqual, equals

Methods inherited from interface com.typesafe.scalalogging.slf4j.Logging

logger

Constructor Detail

ParquetTableScan

public ParquetTableScan(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> output,
                org.apache.spark.sql.parquet.ParquetRelation relation,
                scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> columnPruningPred,
                SQLContext sqlContext)

Method Detail

output

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> output()

Specified by:

output in class org.apache.spark.sql.catalyst.plans.QueryPlan<SparkPlan>

relation

public org.apache.spark.sql.parquet.ParquetRelation relation()

columnPruningPred

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> columnPruningPred()

sqlContext

public SQLContext sqlContext()

execute

public RDD<org.apache.spark.sql.catalyst.expressions.Row> execute()

Description copied from class: SparkPlan

Runs this query returning the result as an RDD.

Specified by:

execute in class SparkPlan

otherCopyArgs

public scala.collection.immutable.List<SQLContext> otherCopyArgs()

pruneColumns

public ParquetTableScan pruneColumns(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> prunedAttributes)

Applies a (candidate) projection.

Parameters:

prunedAttributes - The list of attributes to be used in the projection.

Returns:

Pruned TableScan.