Counting streaks in stateful structured streaming

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Counting streaks in stateful structured streaming

nimrod
Hi all,

I have a stream with a list of values and I want to count consecutive values
a period of 1 minute (even if it is across samples). Can it be done at all?

Nimrod



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Counting streaks in stateful structured streaming

Zahid Rahman
From this paragraph it appears the answer to your query is YES.

page 334 Spark the definitive guide states :

"Stream processing is the act of continuously incorporating new data to compute a result. In stream processing,
the input data is unbounded and has no predetermined beginning  or end.  It simply forms a series of events that arrive at the stream processing system (e.g. credit card transactions, clicks on a website , or sensor readings ..)
User applications can then compute various queries over this stream of events (e..g. tracking a running count of each type of event or aggregating them into in hourly windows). The application will output multiple versions of the results as it runs or perhaps keep it up to date in an  external "sink" system such as a key-value store"

¯\_(ツ)_/¯
♡۶Java♡۶RMI ♡۶
Make Use Method {MUM}


On Sun, 1 Mar 2020 at 08:16, nimrod <[hidden email]> wrote:
Hi all,

I have a stream with a list of values and I want to count consecutive values
a period of 1 minute (even if it is across samples). Can it be done at all?

Nimrod



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]