spark技术分享ClusteringEvaluator — Evaluator of Clustering Models
ClusteringEvaluator is an Evaluator of clustering models (e.g. FPGrowth, GaussianMixture, ALS, KMeans, LinearSVC, RandomForestRegressor, GeneralizedLinearRegression, LinearRegression, GBTRegressor, DecisionTreeRegressor, NaiveBayes)
|
Note
|
ClusteringEvaluator is available since Spark 2.3.0.
|
ClusteringEvaluator finds the best model by maximizing the model evaluation metric (i.e. isLargerBetter is always turned on).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
import org.apache.spark.ml.evaluation.ClusteringEvaluator val cluEval = new ClusteringEvaluator(). setPredictionCol("prediction"). setFeaturesCol("features"). setMetricName("silhouette") scala> cluEval.isLargerBetter res0: Boolean = true scala> println(cluEval.explainParams) featuresCol: features column name (default: features, current: features) metricName: metric name in evaluation (silhouette) (default: silhouette, current: silhouette) predictionCol: prediction column name (default: prediction, current: prediction) |
| Parameter | Default Value | Description | ||
|---|---|---|---|---|
|
|
Name of the column with features (of type |
|||
|
|
Name of the classification metric for evaluation
|
|||
|
|
Name of the column with prediction (of type |
Evaluating Model Output — evaluate Method
|
1 2 3 4 5 |
evaluate(dataset: Dataset[_]): Double |
|
Note
|
evaluate is part of Evaluator Contract.
|
evaluate…FIXME
