ReceiverInputDStream

abstract def getReceiver(): Receiver[T]

Gets the receiver object that will be sent to the worker nodes to receive data.

final def !=(arg0: AnyRef): Boolean

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: AnyRef): Boolean

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

val baseScope: Option[String]

The base scope associated with the operation that created this DStream.

def cache(): DStream[T]

Persist RDDs of this DStream with the default storage level (MEMORY_ONLY_SER)

def checkpoint(interval: Duration): DStream[T]

Enable periodic checkpointing of RDDs of this DStream

def clone(): AnyRef

def compute(validTime: Time): Option[RDD[T]]

Generates RDDs with blocks received by the receiver of this stream.

def context: StreamingContext

Return the StreamingContext associated with this DStream

def count(): DStream[Long]

Return a new DStream in which each RDD has a single element generated by counting each RDD of this DStream.

def countByValue(numPartitions: Int = ssc.sc.defaultParallelism)(implicit ord: Ordering[T] = null): DStream[(T, Long)]

Return a new DStream in which each RDD contains the counts of each distinct value in each RDD of this DStream.

def countByValueAndWindow(windowDuration: Duration, slideDuration: Duration, numPartitions: Int = ssc.sc.defaultParallelism)(implicit ord: Ordering[T] = null): DStream[(T, Long)]

Return a new DStream in which each RDD contains the count of distinct elements in RDDs in a sliding window over this DStream.

def countByWindow(windowDuration: Duration, slideDuration: Duration): DStream[Long]

Return a new DStream in which each RDD has a single element generated by counting the number of elements in a sliding window over this DStream.

def createRDDWithLocalProperties[U](time: Time)(body: ⇒ U): U

Wrap a body of code such that the call site and operation scope information are passed to the RDDs created in this body properly.

def dependencies: List[DStream[_]]

List of parent DStreams on which this DStream depends on

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def filter(filterFunc: (T) ⇒ Boolean): DStream[T]

Return a new DStream containing only the elements that satisfy a predicate.

def finalize(): Unit

def flatMap[U](flatMapFunc: (T) ⇒ Traversable[U])(implicit arg0: ClassTag[U]): DStream[U]

Return a new DStream by applying a function to all elements of this DStream, and then flattening the results

def foreachRDD(foreachFunc: (RDD[T], Time) ⇒ Unit): Unit

Apply a function to each RDD in this DStream.

def foreachRDD(foreachFunc: (RDD[T]) ⇒ Unit): Unit

Apply a function to each RDD in this DStream.

final def getClass(): Class[_]

def glom(): DStream[Array[T]]

Return a new DStream in which each RDD is generated by applying glom() to each RDD of this DStream.

def hashCode(): Int

val id: Int

This is an unique identifier for the input stream.

final def isInstanceOf[T0]: Boolean

def isTraceEnabled(): Boolean

def log: Logger

def logDebug(msg: ⇒ String, throwable: Throwable): Unit

def logDebug(msg: ⇒ String): Unit

def logError(msg: ⇒ String, throwable: Throwable): Unit

def logError(msg: ⇒ String): Unit

def logInfo(msg: ⇒ String, throwable: Throwable): Unit

def logInfo(msg: ⇒ String): Unit

def logName: String

def logTrace(msg: ⇒ String, throwable: Throwable): Unit

def logTrace(msg: ⇒ String): Unit

def logWarning(msg: ⇒ String, throwable: Throwable): Unit

def logWarning(msg: ⇒ String): Unit

def map[U](mapFunc: (T) ⇒ U)(implicit arg0: ClassTag[U]): DStream[U]

Return a new DStream by applying a function to all elements of this DStream.

def mapPartitions[U](mapPartFunc: (Iterator[T]) ⇒ Iterator[U], preservePartitioning: Boolean = false)(implicit arg0: ClassTag[U]): DStream[U]

Return a new DStream in which each RDD is generated by applying mapPartitions() to each RDDs of this DStream.

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def persist(): DStream[T]

Persist RDDs of this DStream with the default storage level (MEMORY_ONLY_SER)

def persist(level: StorageLevel): DStream[T]

Persist the RDDs of this DStream with the given storage level

def print(num: Int): Unit

Print the first num elements of each RDD generated in this DStream.

def print(): Unit

Print the first ten elements of each RDD generated in this DStream.

val rateController: Option[RateController]

Asynchronously maintains & sends new rate limits to the receiver through the receiver tracker.

def reduce(reduceFunc: (T, T) ⇒ T): DStream[T]

Return a new DStream in which each RDD has a single element generated by reducing each RDD of this DStream.

def reduceByWindow(reduceFunc: (T, T) ⇒ T, invReduceFunc: (T, T) ⇒ T, windowDuration: Duration, slideDuration: Duration): DStream[T]

Return a new DStream in which each RDD has a single element generated by reducing all elements in a sliding window over this DStream.

def reduceByWindow(reduceFunc: (T, T) ⇒ T, windowDuration: Duration, slideDuration: Duration): DStream[T]

Return a new DStream in which each RDD has a single element generated by reducing all elements in a sliding window over this DStream.

def repartition(numPartitions: Int): DStream[T]

Return a new DStream with an increased or decreased level of parallelism.

def saveAsObjectFiles(prefix: String, suffix: String = ""): Unit

Save each RDD in this DStream as a Sequence file of serialized objects.

def saveAsTextFiles(prefix: String, suffix: String = ""): Unit

Save each RDD in this DStream as at text file, using string representation of elements.

def slice(fromTime: Time, toTime: Time): Seq[RDD[T]]

Return all the RDDs between 'fromTime' to 'toTime' (both included)

def slice(interval: Interval): Seq[RDD[T]]

Return all the RDDs defined by the Interval object (both end times included)

def slideDuration: Duration

Time interval after which the DStream generates a RDD

def start(): Unit

Method called to start receiving data.

def stop(): Unit

Method called to stop receiving data.

final def synchronized[T0](arg0: ⇒ T0): T0

def toString(): String

def transform[U](transformFunc: (RDD[T], Time) ⇒ RDD[U])(implicit arg0: ClassTag[U]): DStream[U]

Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream.

def transform[U](transformFunc: (RDD[T]) ⇒ RDD[U])(implicit arg0: ClassTag[U]): DStream[U]

Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream.

def transformWith[U, V](other: DStream[U], transformFunc: (RDD[T], RDD[U], Time) ⇒ RDD[V])(implicit arg0: ClassTag[U], arg1: ClassTag[V]): DStream[V]

Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream and 'other' DStream.

def transformWith[U, V](other: DStream[U], transformFunc: (RDD[T], RDD[U]) ⇒ RDD[V])(implicit arg0: ClassTag[U], arg1: ClassTag[V]): DStream[V]

Return a new DStream in which each RDD is generated by applying a function on each RDD of 'this' DStream and 'other' DStream.

def union(that: DStream[T]): DStream[T]

Return a new DStream by unifying data of another DStream with this DStream.

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

def window(windowDuration: Duration, slideDuration: Duration): DStream[T]

Return a new DStream in which each RDD contains all the elements in seen in a sliding window of time over this DStream.

def window(windowDuration: Duration): DStream[T]

Return a new DStream in which each RDD contains all the elements in seen in a sliding window of time over this DStream.

def foreach(foreachFunc: (RDD[T], Time) ⇒ Unit): Unit

Apply a function to each RDD in this DStream.

def foreach(foreachFunc: (RDD[T]) ⇒ Unit): Unit

Apply a function to each RDD in this DStream.