Operators supported by Spark Structured Streaming

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Operators supported by Spark Structured Streaming

shicheng31604@gmail.com
Hi:
    Spark Structured Streaming uses the DataFrame API. When programming, there are no compilation errors, but when running, it will report various unsupported conditions. The official website does not seem to have a document to list the unsupported operators. This will Inconvenient when developing. How did you solve this problem?


Reply | Threaded
Open this post in threaded view
|

Re: Operators supported by Spark Structured Streaming

Jungtaek Lim-2
Hi,

Unsupported operations in Structured Streaming is explained in the guide doc.

Thanks,
Jungtaek Lim (HeartSaVioR)


On Fri, Nov 29, 2019 at 2:08 PM [hidden email] <[hidden email]> wrote:
Hi:
    Spark Structured Streaming uses the DataFrame API. When programming, there are no compilation errors, but when running, it will report various unsupported conditions. The official website does not seem to have a document to list the unsupported operators. This will Inconvenient when developing. How did you solve this problem?


Reply | Threaded
Open this post in threaded view
|

Re: Operators supported by Spark Structured Streaming

shicheng31604@gmail.com
It is still not complete. For example, the supported operators are different under different outputmodes. For developers, only try.

Jungtaek Lim <[hidden email]> 于2019年11月29日周五 下午1:36写道:
Hi,

Unsupported operations in Structured Streaming is explained in the guide doc.

Thanks,
Jungtaek Lim (HeartSaVioR)


On Fri, Nov 29, 2019 at 2:08 PM [hidden email] <[hidden email]> wrote:
Hi:
    Spark Structured Streaming uses the DataFrame API. When programming, there are no compilation errors, but when running, it will report various unsupported conditions. The official website does not seem to have a document to list the unsupported operators. This will Inconvenient when developing. How did you solve this problem?