running updates using SPARK

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

running updates using SPARK

Gourav Sengupta
Hi,

Is there any time soon that SPARK will support UPDATES?  Databricks does provide Delta which supports UPDATE, but I think that the open source SPARK does not have the UPDATE option. 

HIVE has been supporting UPDATES for a very very long time now, and I was thinking when would that become available to SPARK users? With GDPR that becomes kind of important.


Regards,
Gourav
Reply | Threaded
Open this post in threaded view
|

Re: running updates using SPARK

Jiaan Geng
I think Spark is a Calculation engine design for OLAP or Ad-hoc.Spark is not
a traditional relational database,UPDATE need some mandatory constraint like
transaction and lock.



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re:running updates using SPARK

Jiaan Geng
In reply to this post by Gourav Sengupta
I think Spark is a Calculation engine design for OLAP or Ad-hoc.Spark is not 
a traditional relational database,UPDATE need some mandatory constraint like 
transaction and lock. 





At 2018-12-21 06:05:54, "Gourav Sengupta" <[hidden email]> wrote:
Hi,

Is there any time soon that SPARK will support UPDATES?  Databricks does provide Delta which supports UPDATE, but I think that the open source SPARK does not have the UPDATE option. 

HIVE has been supporting UPDATES for a very very long time now, and I was thinking when would that become available to SPARK users? With GDPR that becomes kind of important.


Regards,
Gourav


 

Reply | Threaded
Open this post in threaded view
|

Re: running updates using SPARK

Gourav Sengupta
In reply to this post by Jiaan Geng
Hi Jiaan,

Spark does support UPDATES but in the version that Databricks has. The question to the community was asking when are they going to support it.

Regards,
Gourav

On Fri, 21 Dec 2018, 03:36 Jiaan Geng <[hidden email] wrote:
I think Spark is a Calculation engine design for OLAP or Ad-hoc.Spark is not
a traditional relational database,UPDATE need some mandatory constraint like
transaction and lock.



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re:Re: running updates using SPARK

Jiaan Geng
Hi Gourav Sengupta,
Thank you provide the infomation of Databricks.





At 2018-12-21 16:55:52, "Gourav Sengupta" <[hidden email]> wrote:
Hi Jiaan,

Spark does support UPDATES but in the version that Databricks has. The question to the community was asking when are they going to support it.

Regards,
Gourav

On Fri, 21 Dec 2018, 03:36 Jiaan Geng <[hidden email] wrote:
I think Spark is a Calculation engine design for OLAP or Ad-hoc.Spark is not
a traditional relational database,UPDATE need some mandatory constraint like
transaction and lock.



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]