org.apache.spark.broadcast

Class TorrentBroadcastFactory

Object

org.apache.spark.broadcast.TorrentBroadcastFactory

All Implemented Interfaces:

BroadcastFactory

public class TorrentBroadcastFactory
extends Object
implements BroadcastFactory

A Broadcast implementation that uses a BitTorrent-like protocol to do a distributed transfer of the broadcasted data to the executors. Refer to TorrentBroadcast for more details.

Constructor Summary

Constructors

Constructor and Description
TorrentBroadcastFactory()

Method Summary

Methods

Modifier and Type	Method and Description
void	initialize(boolean isDriver, SparkConf conf, SecurityManager securityMgr)
<T> TorrentBroadcast<T>	newBroadcast(T value_, boolean isLocal, long id, scala.reflect.ClassTag<T> evidence$1) Creates a new broadcast variable.
void	stop()
void	unbroadcast(long id, boolean removeFromDriver, boolean blocking) Remove all persisted state associated with the torrent broadcast with the given ID.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TorrentBroadcastFactory

public TorrentBroadcastFactory()

Method Detail

initialize

public void initialize(boolean isDriver,
              SparkConf conf,
              SecurityManager securityMgr)

Specified by:

initialize in interface BroadcastFactory

newBroadcast

public <T> TorrentBroadcast<T> newBroadcast(T value_,
                                   boolean isLocal,
                                   long id,
                                   scala.reflect.ClassTag<T> evidence$1)

Description copied from interface: BroadcastFactory

Creates a new broadcast variable.

Specified by:

newBroadcast in interface BroadcastFactory

Parameters:

value_ - value to broadcast

isLocal - whether we are in local mode (single JVM process)

id - unique id representing this broadcast variable

stop

public void stop()

Specified by:

stop in interface BroadcastFactory

unbroadcast

public void unbroadcast(long id,
               boolean removeFromDriver,
               boolean blocking)

Remove all persisted state associated with the torrent broadcast with the given ID.

Specified by:

unbroadcast in interface BroadcastFactory

Parameters:

removeFromDriver - Whether to remove state from the driver.

blocking - Whether to block until unbroadcasted