Re: [ANNOUNCE] Announcing Apache Spark 2.4.0

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] Announcing Apache Spark 2.4.0

Marcelo Vanzin-2
+user@

>> ---------- Forwarded message ---------
>> From: Wenchen Fan <[hidden email]>
>> Date: Thu, Nov 8, 2018 at 10:55 PM
>> Subject: [ANNOUNCE] Announcing Apache Spark 2.4.0
>> To: Spark dev list <[hidden email]>
>>
>>
>> Hi all,
>>
>> Apache Spark 2.4.0 is the fifth release in the 2.x line. This release adds Barrier Execution Mode for better integration with deep learning frameworks, introduces 30+ built-in and higher-order functions to deal with complex data type easier, improves the K8s integration, along with experimental Scala 2.12 support. Other major updates include the built-in Avro data source, Image data source, flexible streaming sinks, elimination of the 2GB block size limitation during transfer, Pandas UDF improvements. In addition, this release continues to focus on usability, stability, and polish while resolving around 1100 tickets.
>>
>> We'd like to thank our contributors and users for their contributions and early feedback to this release. This release would not have been possible without you.
>>
>> To download Spark 2.4.0, head over to the download page: http://spark.apache.org/downloads.html
>>
>> To view the release notes: https://spark.apache.org/releases/spark-release-2-4-0.html
>>
>> Thanks,
>> Wenchen
>>
>> PS: If you see any issues with the release notes, webpage or published artifacts, please contact me directly off-list.



--
Marcelo

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] Announcing Apache Spark 2.4.0

Swapnil Shinde
Great news.. thank you very much!

On Thu, Nov 8, 2018, 5:19 PM Stavros Kontopoulos <[hidden email] wrote:
Awesome!

On Thu, Nov 8, 2018 at 9:36 PM, Jules Damji <[hidden email]> wrote:
Indeed! 

Sent from my iPhone
Pardon the dumb thumb typos :)

On Nov 8, 2018, at 11:31 AM, Dongjoon Hyun <[hidden email]> wrote:

Finally, thank you all. Especially, thanks to the release manager, Wenchen!

Bests,
Dongjoon.


On Thu, Nov 8, 2018 at 11:24 AM Wenchen Fan <[hidden email]> wrote:
+ user list

On Fri, Nov 9, 2018 at 2:20 AM Wenchen Fan <[hidden email]> wrote:
resend

On Thu, Nov 8, 2018 at 11:02 PM Wenchen Fan <[hidden email]> wrote:


---------- Forwarded message ---------
From: Wenchen Fan <[hidden email]>
Date: Thu, Nov 8, 2018 at 10:55 PM
Subject: [ANNOUNCE] Announcing Apache Spark 2.4.0
To: Spark dev list <[hidden email]>


Hi all,

Apache Spark 2.4.0 is the fifth release in the 2.x line. This release adds Barrier Execution Mode for better integration with deep learning frameworks, introduces 30+ built-in and higher-order functions to deal with complex data type easier, improves the K8s integration, along with experimental Scala 2.12 support. Other major updates include the built-in Avro data source, Image data source, flexible streaming sinks, elimination of the 2GB block size limitation during transfer, Pandas UDF improvements. In addition, this release continues to focus on usability, stability, and polish while resolving around 1100 tickets.

We'd like to thank our contributors and users for their contributions and early feedback to this release. This release would not have been possible without you.

To download Spark 2.4.0, head over to the download page: http://spark.apache.org/downloads.html


Thanks,
Wenchen

PS: If you see any issues with the release notes, webpage or published artifacts, please contact me directly off-list.




Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] Announcing Apache Spark 2.4.0

Li Gao-2
this is wonderful !
I noticed the official spark download site does not have 2.4 download links yet.

On Thu, Nov 8, 2018, 4:11 PM Swapnil Shinde <[hidden email] wrote:
Great news.. thank you very much!

On Thu, Nov 8, 2018, 5:19 PM Stavros Kontopoulos <[hidden email] wrote:
Awesome!

On Thu, Nov 8, 2018 at 9:36 PM, Jules Damji <[hidden email]> wrote:
Indeed! 

Sent from my iPhone
Pardon the dumb thumb typos :)

On Nov 8, 2018, at 11:31 AM, Dongjoon Hyun <[hidden email]> wrote:

Finally, thank you all. Especially, thanks to the release manager, Wenchen!

Bests,
Dongjoon.


On Thu, Nov 8, 2018 at 11:24 AM Wenchen Fan <[hidden email]> wrote:
+ user list

On Fri, Nov 9, 2018 at 2:20 AM Wenchen Fan <[hidden email]> wrote:
resend

On Thu, Nov 8, 2018 at 11:02 PM Wenchen Fan <[hidden email]> wrote:


---------- Forwarded message ---------
From: Wenchen Fan <[hidden email]>
Date: Thu, Nov 8, 2018 at 10:55 PM
Subject: [ANNOUNCE] Announcing Apache Spark 2.4.0
To: Spark dev list <[hidden email]>


Hi all,

Apache Spark 2.4.0 is the fifth release in the 2.x line. This release adds Barrier Execution Mode for better integration with deep learning frameworks, introduces 30+ built-in and higher-order functions to deal with complex data type easier, improves the K8s integration, along with experimental Scala 2.12 support. Other major updates include the built-in Avro data source, Image data source, flexible streaming sinks, elimination of the 2GB block size limitation during transfer, Pandas UDF improvements. In addition, this release continues to focus on usability, stability, and polish while resolving around 1100 tickets.

We'd like to thank our contributors and users for their contributions and early feedback to this release. This release would not have been possible without you.

To download Spark 2.4.0, head over to the download page: http://spark.apache.org/downloads.html


Thanks,
Wenchen

PS: If you see any issues with the release notes, webpage or published artifacts, please contact me directly off-list.




Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] Announcing Apache Spark 2.4.0

rxin
Do you have a cached copy? I see it here 




On Thu, Nov 8, 2018 at 4:12 PM Li Gao <[hidden email]> wrote:
this is wonderful !
I noticed the official spark download site does not have 2.4 download links yet.

On Thu, Nov 8, 2018, 4:11 PM Swapnil Shinde <[hidden email] wrote:
Great news.. thank you very much!

On Thu, Nov 8, 2018, 5:19 PM Stavros Kontopoulos <[hidden email] wrote:
Awesome!

On Thu, Nov 8, 2018 at 9:36 PM, Jules Damji <[hidden email]> wrote:
Indeed! 

Sent from my iPhone
Pardon the dumb thumb typos :)

On Nov 8, 2018, at 11:31 AM, Dongjoon Hyun <[hidden email]> wrote:

Finally, thank you all. Especially, thanks to the release manager, Wenchen!

Bests,
Dongjoon.


On Thu, Nov 8, 2018 at 11:24 AM Wenchen Fan <[hidden email]> wrote:
+ user list

On Fri, Nov 9, 2018 at 2:20 AM Wenchen Fan <[hidden email]> wrote:
resend

On Thu, Nov 8, 2018 at 11:02 PM Wenchen Fan <[hidden email]> wrote:


---------- Forwarded message ---------
From: Wenchen Fan <[hidden email]>
Date: Thu, Nov 8, 2018 at 10:55 PM
Subject: [ANNOUNCE] Announcing Apache Spark 2.4.0
To: Spark dev list <[hidden email]>


Hi all,

Apache Spark 2.4.0 is the fifth release in the 2.x line. This release adds Barrier Execution Mode for better integration with deep learning frameworks, introduces 30+ built-in and higher-order functions to deal with complex data type easier, improves the K8s integration, along with experimental Scala 2.12 support. Other major updates include the built-in Avro data source, Image data source, flexible streaming sinks, elimination of the 2GB block size limitation during transfer, Pandas UDF improvements. In addition, this release continues to focus on usability, stability, and polish while resolving around 1100 tickets.

We'd like to thank our contributors and users for their contributions and early feedback to this release. This release would not have been possible without you.

To download Spark 2.4.0, head over to the download page: http://spark.apache.org/downloads.html


Thanks,
Wenchen

PS: If you see any issues with the release notes, webpage or published artifacts, please contact me directly off-list.




Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] Announcing Apache Spark 2.4.0

Xiao Li-2
Try to clear your browsing data or use a different web browser.  

Enjoy it, 

Xiao

On Thu, Nov 8, 2018 at 4:15 PM Reynold Xin <[hidden email]> wrote:
Do you have a cached copy? I see it here 




On Thu, Nov 8, 2018 at 4:12 PM Li Gao <[hidden email]> wrote:
this is wonderful !
I noticed the official spark download site does not have 2.4 download links yet.

On Thu, Nov 8, 2018, 4:11 PM Swapnil Shinde <[hidden email] wrote:
Great news.. thank you very much!

On Thu, Nov 8, 2018, 5:19 PM Stavros Kontopoulos <[hidden email] wrote:
Awesome!

On Thu, Nov 8, 2018 at 9:36 PM, Jules Damji <[hidden email]> wrote:
Indeed! 

Sent from my iPhone
Pardon the dumb thumb typos :)

On Nov 8, 2018, at 11:31 AM, Dongjoon Hyun <[hidden email]> wrote:

Finally, thank you all. Especially, thanks to the release manager, Wenchen!

Bests,
Dongjoon.


On Thu, Nov 8, 2018 at 11:24 AM Wenchen Fan <[hidden email]> wrote:
+ user list

On Fri, Nov 9, 2018 at 2:20 AM Wenchen Fan <[hidden email]> wrote:
resend

On Thu, Nov 8, 2018 at 11:02 PM Wenchen Fan <[hidden email]> wrote:


---------- Forwarded message ---------
From: Wenchen Fan <[hidden email]>
Date: Thu, Nov 8, 2018 at 10:55 PM
Subject: [ANNOUNCE] Announcing Apache Spark 2.4.0
To: Spark dev list <[hidden email]>


Hi all,

Apache Spark 2.4.0 is the fifth release in the 2.x line. This release adds Barrier Execution Mode for better integration with deep learning frameworks, introduces 30+ built-in and higher-order functions to deal with complex data type easier, improves the K8s integration, along with experimental Scala 2.12 support. Other major updates include the built-in Avro data source, Image data source, flexible streaming sinks, elimination of the 2GB block size limitation during transfer, Pandas UDF improvements. In addition, this release continues to focus on usability, stability, and polish while resolving around 1100 tickets.

We'd like to thank our contributors and users for their contributions and early feedback to this release. This release would not have been possible without you.

To download Spark 2.4.0, head over to the download page: http://spark.apache.org/downloads.html


Thanks,
Wenchen

PS: If you see any issues with the release notes, webpage or published artifacts, please contact me directly off-list.






--
Spark+AI Summit North America 2019
Reply | Threaded
Open this post in threaded view
|

Re: [ANNOUNCE] Announcing Apache Spark 2.4.0

purna pradeep
In reply to this post by Marcelo Vanzin-2
Thanks this is a great news

Can you please lemme if dynamic resource allocation is available in spark 2.4?

I’m using spark 2.3.2 on Kubernetes, do I still need to provide executor memory options as part of spark submit command or spark will manage required executor memory based on the spark job size ?

On Thu, Nov 8, 2018 at 2:18 PM Marcelo Vanzin <[hidden email]> wrote:
+user@

>> ---------- Forwarded message ---------
>> From: Wenchen Fan <[hidden email]>
>> Date: Thu, Nov 8, 2018 at 10:55 PM
>> Subject: [ANNOUNCE] Announcing Apache Spark 2.4.0
>> To: Spark dev list <[hidden email]>
>>
>>
>> Hi all,
>>
>> Apache Spark 2.4.0 is the fifth release in the 2.x line. This release adds Barrier Execution Mode for better integration with deep learning frameworks, introduces 30+ built-in and higher-order functions to deal with complex data type easier, improves the K8s integration, along with experimental Scala 2.12 support. Other major updates include the built-in Avro data source, Image data source, flexible streaming sinks, elimination of the 2GB block size limitation during transfer, Pandas UDF improvements. In addition, this release continues to focus on usability, stability, and polish while resolving around 1100 tickets.
>>
>> We'd like to thank our contributors and users for their contributions and early feedback to this release. This release would not have been possible without you.
>>
>> To download Spark 2.4.0, head over to the download page: http://spark.apache.org/downloads.html
>>
>> To view the release notes: https://spark.apache.org/releases/spark-release-2-4-0.html
>>
>> Thanks,
>> Wenchen
>>
>> PS: If you see any issues with the release notes, webpage or published artifacts, please contact me directly off-list.



--
Marcelo

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]