How to get the data url

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

How to get the data url

onoke
Hi,

I am searching a useful API for getting a data URL that is accessed by a
application on Spark.
For example, when this URL is in a application

       new
URL("https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv")

How to get this url from using Spark API?
I looked in org.apach.api.java and org.apache.spark.status.api.v1, but they
do not provide any URL info.

Any advice are welcome.

-Keiji



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to get the data url

Jean Georges Perrin
I am a little confused by your question… Are you trying to ingest a file from S3?

If so… look for net.jgp.labs.spark on GitHub and look for net.jgp.labs.spark.l000_ingestion.l001_csv_in_progress.S3CsvToDataset 

You can modify the file as the keys are yours…

If you want to download first: look at net.jgp.labs.spark.l900_analytics.ListNCSchoolDistricts

jgp

On Oct 29, 2017, at 22:20, onoke <[hidden email]> wrote:

Hi,

I am searching a useful API for getting a data URL that is accessed by a
application on Spark.
For example, when this URL is in a application

      new
URL("https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv")

How to get this url from using Spark API?
I looked in org.apach.api.java and org.apache.spark.status.api.v1, but they
do not provide any URL info.

Any advice are welcome.

-Keiji



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: How to get the data url

onoke
Thank you for your reply, jgp.
The URL in my question was just a sample. Whatever it is.

I mean, let's image multi user environment spark.
This is just a case model.
 -I am a watcher on a spark system.
 -Some users run their applications on my spark and ,I need to know what URL running on my spark.
 -Maybe You say I can see the URL in a log file, but it is troublesome.
 -If there is a API that get a access url string, I could make a program to gather all running URL.

I believe there is the best API in it, but just i do not know yet.

Thx.

-Keiji

2017-11-03 19:48 GMT+09:00 Jean Georges Perrin <[hidden email]>:
I am a little confused by your question… Are you trying to ingest a file from S3?

If so… look for net.jgp.labs.spark on GitHub and look for net.jgp.labs.spark.l000_ingestion.l001_csv_in_progress.S3CsvToDataset 

You can modify the file as the keys are yours…

If you want to download first: look at net.jgp.labs.spark.l900_analytics.ListNCSchoolDistricts

jgp

On Oct 29, 2017, at 22:20, onoke <[hidden email]> wrote:

Hi,

I am searching a useful API for getting a data URL that is accessed by a
application on Spark.
For example, when this URL is in a application

      new
URL("https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv")

How to get this url from using Spark API?
I looked in org.apach.api.java and org.apache.spark.status.api.v1, but they
do not provide any URL info.

Any advice are welcome.

-Keiji



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]