setting initial state for mapGroupsWithState

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

setting initial state for mapGroupsWithState

dpristin
Hi,

I'm in the process of migrating our DStream jobs to structured streaming and
I'm looking for an advise on how to provide initial state for the
mapGroupsWithState, similarly to what DStream's mapWithState does:

stream.mapWithState(StateSpec.function(updateState
_).*initialState*(historicalState)

We store our state in Sql Server and there is a good amount of historical
aggregated data that we use in our stateful operations. Would very much
appreciate your insight.

Thanks!



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]