Problem of how to retrieve file from HDFS

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Problem of how to retrieve file from HDFS

Ashish Mittal
I am trying to store and retrieve csv file from HDFS.but i have successfully store csv file in HDFS using LinearRegressionModel in spark using Java.but not retrieve csv file from HDFS. how to retrieve csv file from HDFS.
SparkSession sparkSession = SparkSession.builder().appName("JavaSparkModelWithHadoopHDFSExample").master("local[2]").getOrCreate();
        SQLContext sqlContext = new SQLContext(sparkSession);

        VectorAssembler assembler = new VectorAssembler();
        assembler.setInputCols(new String[] { "MONTH_1", "MONTH_2", "MONTH_3", "MONTH_4", "MONTH_5", "MONTH_6" })

        Dataset<Row> rowDataSet ="csv").option("header", "true").option("inferSchema", "true")

        Dataset<Row> vectorDataSet = assembler.transform(rowDataSet).drop("CUST_ID");;

        LinearRegression lr = new LinearRegression().setMaxIter(10).setRegParam(0.3).setElasticNetParam(0.8)

        LinearRegressionModel lrModel =;

This code is successfully store csv file. but i don't know how to retrieve csv file from hdfs. Please help me.

Thanks & Regards,
Ashish Mittal