org.apache.spark.ml.clustering

Class PowerIterationClustering

Object

org.apache.spark.ml.clustering.PowerIterationClustering

All Implemented Interfaces:

java.io.Serializable, PowerIterationClusteringParams, Params, HasMaxIter, HasWeightCol, DefaultParamsWritable, Identifiable, MLWritable

public class PowerIterationClustering
extends Object
implements PowerIterationClusteringParams, DefaultParamsWritable

:: Experimental :: Power Iteration Clustering (PIC), a scalable graph clustering algorithm developed by Lin and Cohen. From the abstract: PIC finds a very low-dimensional embedding of a dataset using truncated power iteration on a normalized pair-wise similarity matrix of the data.

This class is not yet an Estimator/Transformer, use assignClusters method to run the PowerIterationClustering algorithm.

See Also:

Spectral clustering (Wikipedia), Serialized Form

Constructor Summary

Constructors

Constructor and Description
PowerIterationClustering()

Method Summary

Modifier and Type	Method and Description
Dataset<Row>	assignClusters(Dataset<?> dataset) Run the PIC algorithm and returns a cluster assignment for each input vertex.
PowerIterationClustering	copy(ParamMap extra) Creates a copy of this instance with the same UID and some extra params.
static PowerIterationClustering	load(String path)
static MLReader<T>	read()
PowerIterationClustering	setDstCol(String value)
PowerIterationClustering	setInitMode(String value)
PowerIterationClustering	setK(int value)
PowerIterationClustering	setMaxIter(int value)
PowerIterationClustering	setSrcCol(String value)
PowerIterationClustering	setWeightCol(String value)
String	uid() An immutable unique ID for the object and its derivatives.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.ml.clustering.PowerIterationClusteringParams

dstCol, getDstCol, getInitMode, getK, getSrcCol, initMode, k, srcCol

Methods inherited from interface org.apache.spark.ml.param.shared.HasMaxIter

getMaxIter, maxIter

Methods inherited from interface org.apache.spark.ml.param.shared.HasWeightCol

getWeightCol, weightCol

Methods inherited from interface org.apache.spark.ml.param.Params

clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn

Methods inherited from interface org.apache.spark.ml.util.Identifiable

toString

Methods inherited from interface org.apache.spark.ml.util.DefaultParamsWritable

write

Methods inherited from interface org.apache.spark.ml.util.MLWritable

save

Constructor Detail

PowerIterationClustering

public PowerIterationClustering()

Method Detail

load

public static PowerIterationClustering load(String path)

read

public static MLReader<T> read()

uid

public String uid()

Description copied from interface: Identifiable

An immutable unique ID for the object and its derivatives.

Specified by:

uid in interface Identifiable

Returns:

(undocumented)

setK

public PowerIterationClustering setK(int value)

setInitMode

public PowerIterationClustering setInitMode(String value)

setMaxIter

public PowerIterationClustering setMaxIter(int value)

setSrcCol

public PowerIterationClustering setSrcCol(String value)

setDstCol

public PowerIterationClustering setDstCol(String value)

setWeightCol

public PowerIterationClustering setWeightCol(String value)

assignClusters

public Dataset<Row> assignClusters(Dataset<?> dataset)

Run the PIC algorithm and returns a cluster assignment for each input vertex.

Parameters:

dataset - A dataset with columns src, dst, weight representing the affinity matrix, which is the matrix A in the PIC paper. Suppose the src column value is i, the dst column value is j, the weight column value is similarity s,,ij,, which must be nonnegative. This is a symmetric matrix and hence s,,ij,, = s,,ji,,. For any (i, j) with nonzero similarity, there should be either (i, j, s,,ij,,) or (j, i, s,,ji,,) in the input. Rows with i = j are ignored, because we assume s,,ij,, = 0.0.

Returns:

A dataset that contains columns of vertex id and the corresponding cluster for the id. The schema of it will be: - id: Long - cluster: Int

copy

public PowerIterationClustering copy(ParamMap extra)

Description copied from interface: Params

Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().

Specified by:

copy in interface Params

Parameters:

extra - (undocumented)

Returns:

(undocumented)