public final class ChiSqSelector extends Estimator<ChiSqSelectorModel> implements ChiSqSelectorParams, DefaultParamsWritable
numTopFeatures, percentile, fpr,
fdr, fwe.
- numTopFeatures chooses a fixed number of top features according to a chi-squared test.
- percentile is similar but chooses a fraction of all features instead of a fixed number.
- fpr chooses all features whose p-value are below a threshold, thus controlling the false
positive rate of selection.
- fdr uses the [Benjamini-Hochberg procedure]
(https://en.wikipedia.org/wiki/False_discovery_rate#Benjamini.E2.80.93Hochberg_procedure)
to choose all features whose false discovery rate is below a threshold.
- fwe chooses all features whose p-values are below a threshold. The threshold is scaled by
1/numFeatures, thus controlling the family-wise error rate of selection.
By default, the selection method is numTopFeatures, with the default number of top features
set to 50.| Constructor and Description |
|---|
ChiSqSelector() |
ChiSqSelector(String uid) |
| Modifier and Type | Method and Description |
|---|---|
ChiSqSelector |
copy(ParamMap extra)
Creates a copy of this instance with the same UID and some extra params.
|
ChiSqSelectorModel |
fit(Dataset<?> dataset)
Fits a model to the input data.
|
static ChiSqSelector |
load(String path) |
static MLReader<T> |
read() |
ChiSqSelector |
setFdr(double value) |
ChiSqSelector |
setFeaturesCol(String value) |
ChiSqSelector |
setFpr(double value) |
ChiSqSelector |
setFwe(double value) |
ChiSqSelector |
setLabelCol(String value) |
ChiSqSelector |
setNumTopFeatures(int value) |
ChiSqSelector |
setOutputCol(String value) |
ChiSqSelector |
setPercentile(double value) |
ChiSqSelector |
setSelectorType(String value) |
StructType |
transformSchema(StructType schema)
:: DeveloperApi ::
|
String |
uid()
An immutable unique ID for the object and its derivatives.
|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitfdr, fpr, fwe, getFdr, getFpr, getFwe, getNumTopFeatures, getPercentile, getSelectorType, numTopFeatures, percentile, selectorTypefeaturesCol, getFeaturesColgetOutputCol, outputColgetLabelCol, labelColclear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, paramMap, params, set, set, set, setDefault, setDefault, shouldOwntoStringwritesaveinitializeLogging, initializeLogIfNecessary, initializeLogIfNecessary, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarningpublic ChiSqSelector(String uid)
public ChiSqSelector()
public static ChiSqSelector load(String path)
public static MLReader<T> read()
public String uid()
Identifiableuid in interface Identifiablepublic ChiSqSelector setNumTopFeatures(int value)
public ChiSqSelector setPercentile(double value)
public ChiSqSelector setFpr(double value)
public ChiSqSelector setFdr(double value)
public ChiSqSelector setFwe(double value)
public ChiSqSelector setSelectorType(String value)
public ChiSqSelector setFeaturesCol(String value)
public ChiSqSelector setOutputCol(String value)
public ChiSqSelector setLabelCol(String value)
public ChiSqSelectorModel fit(Dataset<?> dataset)
Estimatorfit in class Estimator<ChiSqSelectorModel>dataset - (undocumented)public StructType transformSchema(StructType schema)
PipelineStageCheck transform validity and derive the output schema from the input schema.
We check validity for interactions between parameters during transformSchema and
raise an exception if any parameter value is invalid. Parameter value checks which
do not depend on other parameters are handled by Param.validate().
Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.
transformSchema in class PipelineStageschema - (undocumented)public ChiSqSelector copy(ParamMap extra)
ParamsdefaultCopy().copy in interface Paramscopy in class Estimator<ChiSqSelectorModel>extra - (undocumented)