org.apache.spark

Class CacheManager

Object

org.apache.spark.CacheManager

All Implemented Interfaces:

Logging

public class CacheManager
extends Object
implements Logging

Spark class responsible for passing RDDs partition contents to the BlockManager and making sure a node doesn't load two copies of an RDD at once.

Constructor Summary

Constructors

Constructor and Description
CacheManager(BlockManager blockManager)

Method Summary

Methods

Modifier and Type	Method and Description
<T> scala.collection.Iterator<T>	getOrCompute(RDD<T> rdd, Partition partition, TaskContext context, StorageLevel storageLevel) Gets or computes an RDD partition.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

CacheManager

public CacheManager(BlockManager blockManager)

Method Detail

getOrCompute

public <T> scala.collection.Iterator<T> getOrCompute(RDD<T> rdd,
                                            Partition partition,
                                            TaskContext context,
                                            StorageLevel storageLevel)

Gets or computes an RDD partition. Used by RDD.iterator() when an RDD is cached.