Announcing Delta Lake 0.3.0

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Announcing Delta Lake 0.3.0

tdas@databricks.com
Hello everyone, 

We are excited to announce the availability of Delta Lake 0.3.0 which introduces new programmatic APIs for manipulating and managing data in Delta Lake tables.

Here are the main features: 


  • Scala/Java APIs for DML commands - You can now modify data in Delta Lake tables using programmatic APIs for Delete, Update and Merge. These APIs mirror the syntax and semantics of their corresponding SQL commands and are great for many workloads, e.g., Slowly Changing Dimension (SCD) operations, merging change data for replication, and upserts from streaming queries. See the documentation for more details.


  • Scala/Java APIs for query commit history - You can now query a table’s commit history to see what operations modified the table. This enables you to audit data changes, time travel queries on specific versions, debug and recover data from accidental deletions, etc. See the documentation for more details.


  • Scala/Java APIs for vacuuming old files - Delta Lake uses MVCC to enable snapshot isolation and time travel. However, keeping all versions of a table forever can be prohibitively expensive. Stale snapshots (as well as other uncommitted files from aborted transactions) can be garbage collected by vacuuming the table. See the documentation for more details.


To try out Delta Lake 0.3.0, please follow the Delta Lake Quickstart: https://docs.delta.io/0.3.0/quick-start.html

To view the release notes:

We would like to thank all the community members for contributing to this release.

TD
Reply | Threaded
Open this post in threaded view
|

Re: Announcing Delta Lake 0.3.0

Gourav Sengupta
Yahoooo!!!!!

celebrations, wine, cocktails, parties, dances tonight :) 

Regards,
Gourav

On Fri, Aug 2, 2019 at 2:44 AM Tathagata Das <[hidden email]> wrote:
Hello everyone, 

We are excited to announce the availability of Delta Lake 0.3.0 which introduces new programmatic APIs for manipulating and managing data in Delta Lake tables.

Here are the main features: 


  • Scala/Java APIs for DML commands - You can now modify data in Delta Lake tables using programmatic APIs for Delete, Update and Merge. These APIs mirror the syntax and semantics of their corresponding SQL commands and are great for many workloads, e.g., Slowly Changing Dimension (SCD) operations, merging change data for replication, and upserts from streaming queries. See the documentation for more details.


  • Scala/Java APIs for query commit history - You can now query a table’s commit history to see what operations modified the table. This enables you to audit data changes, time travel queries on specific versions, debug and recover data from accidental deletions, etc. See the documentation for more details.


  • Scala/Java APIs for vacuuming old files - Delta Lake uses MVCC to enable snapshot isolation and time travel. However, keeping all versions of a table forever can be prohibitively expensive. Stale snapshots (as well as other uncommitted files from aborted transactions) can be garbage collected by vacuuming the table. See the documentation for more details.


To try out Delta Lake 0.3.0, please follow the Delta Lake Quickstart: https://docs.delta.io/0.3.0/quick-start.html

To view the release notes:

We would like to thank all the community members for contributing to this release.

TD

--
You received this message because you are subscribed to the Google Groups "Delta Lake Users and Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/delta-users/CA%2BAHuKmAhUar%3D7GZ9bUwJKmh%3Diu67%3DTVzH%2BhiwTpC0v33A_MQQ%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Announcing Delta Lake 0.3.0

Nicolas Paris-2
In reply to this post by tdas@databricks.com
>   • Scala/Java APIs for DML commands - You can now modify data in Delta Lake
>     tables using programmatic APIs for Delete, Update and Merge. These APIs
>     mirror the syntax and semantics of their corresponding SQL commands and are
>     great for many workloads, e.g., Slowly Changing Dimension (SCD) operations,
>     merging change data for replication, and upserts from streaming queries.
>     See the documentation for more details.

just tested the merge feature on a large table: awesome
- fast to build
- fast to query afterward
- robust (version history is an incredible feature)


thanks


On Thu, Aug 01, 2019 at 06:44:30PM -0700, Tathagata Das wrote:

> Hello everyone, 
>
> We are excited to announce the availability of Delta Lake 0.3.0 which
> introduces new programmatic APIs for manipulating and managing data in Delta
> Lake tables.
>
>
> Here are the main features: 
>
>
>   • Scala/Java APIs for DML commands - You can now modify data in Delta Lake
>     tables using programmatic APIs for Delete, Update and Merge. These APIs
>     mirror the syntax and semantics of their corresponding SQL commands and are
>     great for many workloads, e.g., Slowly Changing Dimension (SCD) operations,
>     merging change data for replication, and upserts from streaming queries.
>     See the documentation for more details.
>
>
>   • Scala/Java APIs for query commit history - You can now query a table’s
>     commit history to see what operations modified the table. This enables you
>     to audit data changes, time travel queries on specific versions, debug and
>     recover data from accidental deletions, etc. See the documentation for more
>     details.
>
>
>   • Scala/Java APIs for vacuuming old files - Delta Lake uses MVCC to enable
>     snapshot isolation and time travel. However, keeping all versions of a
>     table forever can be prohibitively expensive. Stale snapshots (as well as
>     other uncommitted files from aborted transactions) can be garbage collected
>     by vacuuming the table. See the documentation for more details.
>
>
> To try out Delta Lake 0.3.0, please follow the Delta Lake Quickstart: https://
> docs.delta.io/0.3.0/quick-start.html
>
> To view the release notes:
> https://github.com/delta-io/delta/releases/tag/v0.3.0
>
> We would like to thank all the community members for contributing to this
> release.
>
> TD

--
nicolas

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]