spark技术分享DataFrameStatFunctions — Working With Statistic Functions
DataFrameStatFunctions is used to work with statistic functions in a structured query (a DataFrame).
| Method | Description | ||
|---|---|---|---|
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
DataFrameStatFunctions is available using stat untyped transformation.
|
1 2 3 4 5 6 |
val q: DataFrame = ... q.stat |
approxQuantile Method
|
1 2 3 4 5 6 7 8 9 10 11 12 |
approxQuantile( cols: Array[String], probabilities: Array[Double], relativeError: Double): Array[Array[Double]] approxQuantile( col: String, probabilities: Array[Double], relativeError: Double): Array[Double] |
approxQuantile…FIXME
bloomFilter Method
|
1 2 3 4 5 6 7 8 |
bloomFilter(col: Column, expectedNumItems: Long, fpp: Double): BloomFilter bloomFilter(col: Column, expectedNumItems: Long, numBits: Long): BloomFilter bloomFilter(colName: String, expectedNumItems: Long, fpp: Double): BloomFilter bloomFilter(colName: String, expectedNumItems: Long, numBits: Long): BloomFilter |
bloomFilter…FIXME
buildBloomFilter Internal Method
|
1 2 3 4 5 |
buildBloomFilter(col: Column, zero: BloomFilter): BloomFilter |
buildBloomFilter…FIXME
|
Note
|
convertToDouble is used when…FIXME
|
corr Method
|
1 2 3 4 5 6 |
corr(col1: String, col2: String): Double corr(col1: String, col2: String, method: String): Double |
corr…FIXME
countMinSketch Method
|
1 2 3 4 5 6 7 8 9 10 |
countMinSketch(col: Column, eps: Double, confidence: Double, seed: Int): CountMinSketch countMinSketch(col: Column, depth: Int, width: Int, seed: Int): CountMinSketch countMinSketch(colName: String, eps: Double, confidence: Double, seed: Int): CountMinSketch countMinSketch(colName: String, depth: Int, width: Int, seed: Int): CountMinSketch // PRIVATE API countMinSketch(col: Column, zero: CountMinSketch): CountMinSketch |
countMinSketch…FIXME
