Announcing .NET for Apache Spark™ 0.12

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Announcing .NET for Apache Spark™ 0.12

imback82
We are happy to announce that .NET for Apache Spark™ v0.12 has been released! Thanks to the community for the great feedback. The release note includes the full list of features/improvements of this release.

Here are the some of the highlights:
  • Ability to write UDFs using complex types such as Row, Array, Map, Date, Timestamp, etc.
  • Ability to write UDFs using .NET DataFrame (backed by Apache Arrow)
  • Enhanced structured streaming support with ForeachBatch/Foreach APIs
  • .NET binding for Delta Lake v0.6 and Hyperspace v0.1
  • Support for Apache Spark™ 2.4.6 (3.0 support is on the way!)
  • SparkSession.CreateDataFrame, Broadcast variable
  • Preliminary support for MLLib (TF-IDF, Word2Vec, Bucketizer, etc.)
  • Support for .NET Core 3.1
We would like to thank all those who contributed to this release.
 
Thanks,
Terry Kim on behalf of the .NET for Apache Spark™ team