DataFrameNaFunctions — Working With Missing Data-spark技术分享

DataFrameNaFunctions — Working With Missing Data

DataFrameNaFunctions is used to work with missing data in a structured query (a DataFrame).

Table 1. DataFrameNaFunctions API

Method

Description



drop(): DataFrame
drop(cols: Array[String]): DataFrame
drop(minNonNulls: Int): DataFrame
drop(minNonNulls: Int, cols: Array[String]): DataFrame
drop(minNonNulls: Int, cols: Seq[String]): DataFrame
drop(cols: Seq[String]): DataFrame
drop(how: String): DataFrame
drop(how: String, cols: Array[String]): DataFrame
drop(how: String, cols: Seq[String]): DataFrame

drop(): DataFrame

drop(cols: Array[String]): DataFrame

drop(minNonNulls: Int): DataFrame

drop(minNonNulls: Int, cols: Array[String]): DataFrame

drop(minNonNulls: Int, cols: Seq[String]): DataFrame

drop(cols: Seq[String]): DataFrame

drop(how: String): DataFrame

drop(how: String, cols: Array[String]): DataFrame

drop(how: String, cols: Seq[String]): DataFrame

fill



fill(value: Boolean): DataFrame
fill(value: Boolean, cols: Array[String]): DataFrame
fill(value: Boolean, cols: Seq[String]): DataFrame
fill(value: Double): DataFrame
fill(value: Double, cols: Array[String]): DataFrame
fill(value: Double, cols: Seq[String]): DataFrame
fill(value: Long): DataFrame
fill(value: Long, cols: Array[String]): DataFrame
fill(value: Long, cols: Seq[String]): DataFrame
fill(valueMap: Map[String, Any]): DataFrame
fill(value: String): DataFrame
fill(value: String, cols: Array[String]): DataFrame
fill(value: String, cols: Seq[String]): DataFrame

fill(value: Boolean): DataFrame

fill(value: Boolean, cols: Array[String]): DataFrame

fill(value: Boolean, cols: Seq[String]): DataFrame

fill(value: Double): DataFrame

fill(value: Double, cols: Array[String]): DataFrame

fill(value: Double, cols: Seq[String]): DataFrame

fill(value: Long): DataFrame

fill(value: Long, cols: Array[String]): DataFrame

fill(value: Long, cols: Seq[String]): DataFrame

fill(valueMap: Map[String, Any]): DataFrame

fill(value: String): DataFrame

fill(value: String, cols: Array[String]): DataFrame

fill(value: String, cols: Seq[String]): DataFrame

replace



replace[T](cols: Seq[String], replacement: Map[T, T]): DataFrame
replace[T](col: String, replacement: Map[T, T]): DataFrame

replace[T](cols: Seq[String], replacement: Map[T, T]): DataFrame

replace[T](col: String, replacement: Map[T, T]): DataFrame

DataFrameNaFunctions is available using na untyped transformation.



val q: DataFrame = ...
q.na

val q: DataFrame = ...

q.na

`convertToDouble` Internal Method



convertToDouble(v: Any): Double

convertToDouble(v: Any): Double

convertToDouble…FIXME

Note	`convertToDouble` is used when…FIXME

`drop` Method



drop(): DataFrame
drop(cols: Array[String]): DataFrame
drop(minNonNulls: Int): DataFrame
drop(minNonNulls: Int, cols: Array[String]): DataFrame
drop(minNonNulls: Int, cols: Seq[String]): DataFrame
drop(cols: Seq[String]): DataFrame
drop(how: String): DataFrame
drop(how: String, cols: Array[String]): DataFrame
drop(how: String, cols: Seq[String]): DataFrame

drop(): DataFrame

drop(cols: Array[String]): DataFrame

drop(minNonNulls: Int): DataFrame

drop(minNonNulls: Int, cols: Array[String]): DataFrame

drop(minNonNulls: Int, cols: Seq[String]): DataFrame

drop(cols: Seq[String]): DataFrame

drop(how: String): DataFrame

drop(how: String, cols: Array[String]): DataFrame

drop(how: String, cols: Seq[String]): DataFrame

drop…FIXME

`fill` Method



fill(value: Boolean): DataFrame
fill(value: Boolean, cols: Array[String]): DataFrame
fill(value: Boolean, cols: Seq[String]): DataFrame
fill(value: Double): DataFrame
fill(value: Double, cols: Array[String]): DataFrame
fill(value: Double, cols: Seq[String]): DataFrame
fill(value: Long): DataFrame
fill(value: Long, cols: Array[String]): DataFrame
fill(value: Long, cols: Seq[String]): DataFrame
fill(valueMap: Map[String, Any]): DataFrame
fill(value: String): DataFrame
fill(value: String, cols: Array[String]): DataFrame
fill(value: String, cols: Seq[String]): DataFrame

fill(value: Boolean): DataFrame

fill(value: Boolean, cols: Array[String]): DataFrame

fill(value: Boolean, cols: Seq[String]): DataFrame

fill(value: Double): DataFrame

fill(value: Double, cols: Array[String]): DataFrame

fill(value: Double, cols: Seq[String]): DataFrame

fill(value: Long): DataFrame

fill(value: Long, cols: Array[String]): DataFrame

fill(value: Long, cols: Seq[String]): DataFrame

fill(valueMap: Map[String, Any]): DataFrame

fill(value: String): DataFrame

fill(value: String, cols: Array[String]): DataFrame

fill(value: String, cols: Seq[String]): DataFrame

fill…FIXME

`fillCol` Internal Method



fillCol[T](col: StructField, replacement: T): Column

fillCol[T](col: StructField, replacement: T): Column

fillCol…FIXME

Note	`fillCol` is used when…FIXME

`fillMap` Internal Method



fillMap(values: Seq[(String, Any)]): DataFrame

fillMap(values: Seq[(String, Any)]): DataFrame

fillMap…FIXME

Note	`fillMap` is used when…FIXME

`fillValue` Internal Method



fillValue[T](value: T, cols: Seq[String]): DataFrame

fillValue[T](value: T, cols: Seq[String]): DataFrame

fillValue…FIXME

Note	`fillValue` is used when…FIXME

`replace0` Internal Method



replace0[T](cols: Seq[String], replacement: Map[T, T]): DataFrame

replace0[T](cols: Seq[String], replacement: Map[T, T]): DataFrame

replace0…FIXME

Note	`replace0` is used when…FIXME

`replace` Method



replace[T](cols: Seq[String], replacement: Map[T, T]): DataFrame
replace[T](col: String, replacement: Map[T, T]): DataFrame

replace[T](cols: Seq[String], replacement: Map[T, T]): DataFrame

replace[T](col: String, replacement: Map[T, T]): DataFrame

replace…FIXME

`replaceCol` Internal Method



replaceCol(col: StructField, replacementMap: Map[_, _]): Column

replaceCol(col: StructField, replacementMap: Map[_, _]): Column

replaceCol…FIXME

Note	`replaceCol` is used when…FIXME

DataFrameNaFunctions — Working With Missing Data