What is the best way for Spark to read HDF5@scale?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

What is the best way for Spark to read HDF5@scale?

kathleen li
Hi,
Any Spark-connector for HDF5? 

The following link does not work anymore?

Thanks,

Kathleen

Reply | Threaded
Open this post in threaded view
|

Re: What is the best way for Spark to read HDF5@scale?

Saurav Sinha

Here is the solution

sc.textFile("hdfs://nn1home:8020/input/war-and-peace.txt")

How did I find out nn1home:8020?

Just search for the file core-site.xml and look for xml element fs.defaultFS


On Fri, Sep 14, 2018 at 7:57 PM kathleen li <[hidden email]> wrote:
Hi,
Any Spark-connector for HDF5? 

The following link does not work anymore?

Thanks,

Kathleen



--
Thanks and Regards,

Saurav Sinha

Contact: 9742879062