How to persistent database/table created in sparkSession

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
163
Reply | Threaded
Open this post in threaded view
|

How to persistent database/table created in sparkSession

163
Hi, 
    How can I persistent database/table created in spark application?

object TestPersistentDB {
        def main(args:Array[String]): Unit = {
                val spark = SparkSession.builder()
                        .appName("Create persistent table")
                        .config("spark.master”,"local")
                        .getOrCreate()
                import spark.implicits._
                spark.sql("create database testdb location \"<a href="hdfs://node1:8020/testdb\" class="">hdfs://node1:8020/testdb\")

        }
}

  When I use spark.sql(“create database”) in sparkSession, and close this sparkSession. 
  The created database is not persisted to metadata, So I cannot find it in spark-sql: show databases.



regards
wendy