org.apache.spark.streaming.dstream

Class RawInputDStream<T>

Object

org.apache.spark.streaming.dstream.DStream<T>

org.apache.spark.streaming.dstream.InputDStream<T>

org.apache.spark.streaming.dstream.ReceiverInputDStream<T>

org.apache.spark.streaming.dstream.RawInputDStream<T>

All Implemented Interfaces:

java.io.Serializable, Logging

public class RawInputDStream<T>
extends ReceiverInputDStream<T>
implements Logging

An input stream that reads blocks of serialized objects from a given network address. The blocks will be inserted directly into the block store. This is the fastest way to get data into Spark Streaming, though it requires the sender to batch data and serialize it in the format that the system is configured with.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
RawInputDStream(StreamingContext ssc_, String host, int port, StorageLevel storageLevel, scala.reflect.ClassTag<T> evidence$1)

Method Summary

Methods

Modifier and Type	Method and Description
Receiver<T>	getReceiver() Gets the receiver object that will be sent to the worker nodes to receive data.

Methods inherited from class org.apache.spark.streaming.dstream.ReceiverInputDStream

compute, id, start, stop

Methods inherited from class org.apache.spark.streaming.dstream.InputDStream

dependencies, isTimeValid, lastValidTime, slideDuration

Methods inherited from class org.apache.spark.streaming.dstream.DStream

cache, checkpoint, checkpointData, checkpointDuration, clearCheckpointData, clearMetadata, context, count, countByValue, countByValueAndWindow, countByWindow, creationSite, filter, flatMap, foreach, foreach, foreachRDD, foreachRDD, generatedRDDs, generateJob, getCreationSite, getOrCompute, glom, graph, initialize, isInitialized, map, mapPartitions, mustCheckpoint, parentRememberDuration, persist, persist, print, print, reduce, reduceByWindow, reduceByWindow, register, remember, rememberDuration, repartition, restoreCheckpointData, saveAsObjectFiles, saveAsTextFiles, setContext, setGraph, slice, slice, ssc, storageLevel, toPairDStreamFunctions, transform, transform, transformWith, transformWith, union, updateCheckpointData, validate, window, window, zeroTime

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

RawInputDStream

public RawInputDStream(StreamingContext ssc_,
               String host,
               int port,
               StorageLevel storageLevel,
               scala.reflect.ClassTag<T> evidence$1)

Method Detail

getReceiver

public Receiver<T> getReceiver()

Description copied from class: ReceiverInputDStream

Gets the receiver object that will be sent to the worker nodes to receive data. This method needs to defined by any specific implementation of a ReceiverInputDStream.

Specified by:

getReceiver in class ReceiverInputDStream<T>