spark技术分享Fundamentals of Spark SQL Application Development
Development of a Spark SQL application requires the following steps:
-
Setting up Development Environment (IntelliJ IDEA, Scala and sbt)
-
Specifying Library Dependencies
-
Creating SparkSession
-
Loading Data from Data Sources
-
Processing Data Using Dataset API
-
Saving Data to Persistent Storage
-
Deploying Spark Application to Cluster (using
spark-submit)
