IndexedRowMatrix

Instance Constructors

new IndexedRowMatrix(rows: RDD[IndexedRow])

Alternative constructor leaving matrix dimensions to be determined automatically.
Alternative constructor leaving matrix dimensions to be determined automatically.

Annotations
@Since( "1.0.0" )
new IndexedRowMatrix(rows: RDD[IndexedRow], nRows: Long, nCols: Int)

rows
indexed rows of this matrix
nRows
number of rows. A non-positive value means unknown, and then the number of rows will be determined by the max row index plus one.
nCols
number of columns. A non-positive value means unknown, and then the number of columns will be determined by the size of the first row.

Annotations
@Since( "1.0.0" )

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
def columnSimilarities(): CoordinateMatrix

Compute all cosine similarities between columns of this matrix using the brute-force approach of computing normalized dot products.
Compute all cosine similarities between columns of this matrix using the brute-force approach of computing normalized dot products.
returns
An n x n sparse upper-triangular matrix of cosine similarities between columns of this matrix.

Annotations
@Since( "1.6.0" )
def computeGramianMatrix(): Matrix

Computes the Gramian matrix A^T A.
Computes the Gramian matrix A^T A.

Annotations
@Since( "1.0.0" )
Note
This cannot be computed on matrices with more than 65535 columns.
def computeSVD(k: Int, computeU: Boolean = false, rCond: Double = 1e-9): SingularValueDecomposition[IndexedRowMatrix, Matrix]

Computes the singular value decomposition of this IndexedRowMatrix.
Computes the singular value decomposition of this IndexedRowMatrix. Denote this matrix by A (m x n), this will compute matrices U, S, V such that A = U * S * V'.
The cost and implementation of this method is identical to that in org.apache.spark.mllib.linalg.distributed.RowMatrix With the addition of indices.
At most k largest non-zero singular values and associated vectors are returned. If there are k such values, then the dimensions of the return will be:
U is an org.apache.spark.mllib.linalg.distributed.IndexedRowMatrix of size m x k that satisfies U'U = eye(k), s is a Vector of size k, holding the singular values in descending order, and V is a local Matrix of size n x k that satisfies V'V = eye(k).
k
number of singular values to keep. We might return less than k if there are numerically zero singular values. See rCond.
computeU
whether to compute U
rCond
the reciprocal condition number. All singular values smaller than rCond * sigma(0) are treated as zero, where sigma(0) is the largest singular value.
returns
SingularValueDecomposition(U, s, V)

Annotations
@Since( "1.0.0" )
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def hashCode(): Int

Definition Classes
AnyRef → Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def multiply(B: Matrix): IndexedRowMatrix

Multiply this matrix by a local matrix on the right.
Multiply this matrix by a local matrix on the right.
B
a local matrix whose number of rows must match the number of columns of this matrix
returns
an IndexedRowMatrix representing the product, which preserves partitioning

Annotations
@Since( "1.0.0" )
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
def numCols(): Long

Gets or computes the number of columns.
Gets or computes the number of columns.

Definition Classes
IndexedRowMatrix → DistributedMatrix
Annotations
@Since( "1.0.0" )
def numRows(): Long

Gets or computes the number of rows.
Gets or computes the number of rows.

Definition Classes
IndexedRowMatrix → DistributedMatrix
Annotations
@Since( "1.0.0" )
val rows: RDD[IndexedRow]

indexed rows of this matrix
indexed rows of this matrix

Annotations
@Since( "1.0.0" )
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toBlockMatrix(rowsPerBlock: Int, colsPerBlock: Int): BlockMatrix

Converts to BlockMatrix.
Converts to BlockMatrix. Creates blocks of SparseMatrix.
rowsPerBlock
The number of rows of each block. The blocks at the bottom edge may have a smaller value. Must be an integer value greater than 0.
colsPerBlock
The number of columns of each block. The blocks at the right edge may have a smaller value. Must be an integer value greater than 0.
returns
a BlockMatrix

Annotations
@Since( "1.3.0" )
def toBlockMatrix(): BlockMatrix

Converts to BlockMatrix.
Converts to BlockMatrix. Creates blocks of SparseMatrix with size 1024 x 1024.

Annotations
@Since( "1.3.0" )
def toCoordinateMatrix(): CoordinateMatrix

Converts this matrix to a org.apache.spark.mllib.linalg.distributed.CoordinateMatrix.
Converts this matrix to a org.apache.spark.mllib.linalg.distributed.CoordinateMatrix.

Annotations
@Since( "1.3.0" )
def toRowMatrix(): RowMatrix

Drops row indices and converts this matrix to a org.apache.spark.mllib.linalg.distributed.RowMatrix.
Drops row indices and converts this matrix to a org.apache.spark.mllib.linalg.distributed.RowMatrix.

Annotations
@Since( "1.0.0" )
def toString(): String

Definition Classes
AnyRef → Any
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Doc: package distributed

class IndexedRowMatrix extends DistributedMatrix

Instance Constructors

new IndexedRowMatrix(rows: RDD[IndexedRow])

new IndexedRowMatrix(rows: RDD[IndexedRow], nRows: Long, nCols: Int)

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def clone(): AnyRef

def columnSimilarities(): CoordinateMatrix

def computeGramianMatrix(): Matrix

def computeSVD(k: Int, computeU: Boolean = false, rCond: Double = 1e-9): SingularValueDecomposition[IndexedRowMatrix, Matrix]

final def eq(arg0: AnyRef): Boolean

def equals(arg0: Any): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def hashCode(): Int

final def isInstanceOf[T0]: Boolean

def multiply(B: Matrix): IndexedRowMatrix

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

def numCols(): Long

def numRows(): Long

val rows: RDD[IndexedRow]

final def synchronized[T0](arg0: ⇒ T0): T0

def toBlockMatrix(rowsPerBlock: Int, colsPerBlock: Int): BlockMatrix

def toBlockMatrix(): BlockMatrix

def toCoordinateMatrix(): CoordinateMatrix

def toRowMatrix(): RowMatrix

def toString(): String

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from DistributedMatrix

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped