spark-stream memory table global?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

spark-stream memory table global?

Imran Rajjad
Hi,

Does the memory table in which spark-structured streaming results are sinked into, is available to other spark applications on the cluster? Is it by default global or will only be available to context where streaming is being done

thanks
Imran

--
I.R
Reply | Threaded
Open this post in threaded view
|

Re: spark-stream memory table global?

Shixiong(Ryan) Zhu
It must be accessed under the same SparkSession. We can also add an option to make it be a global temp view. Feel free to open a PR to improve it.

On Fri, Nov 10, 2017 at 4:56 AM, Imran Rajjad <[hidden email]> wrote:
Hi,

Does the memory table in which spark-structured streaming results are sinked into, is available to other spark applications on the cluster? Is it by default global or will only be available to context where streaming is being done

thanks
Imran

--
I.R