Hi Rishi,

Spark and Flint are useful during the data engineering phase, but
you'd need to look elsewhere after that. I'm not aware of any
active Spark-native project to do ML/forecast on time series data.

If the data that you want to train the model on can fit in one
node's memory, you can use libs and models like ARIMA, Prophet, or
LSTM-based NN to train a model and use them for forecasting. You
can then use Spark to parallelize the grid search over the space
of hyperparameters to get the optimal model faster, as the grid
search would be a perfectly-parallel job (a.k.a, embarrassingly
parallel). I gave a talk on this which you may find useful: https://www.analytical.works/Talk-spark-ml.html

Masood

__________________
Masood Krohy, Ph.D.
Data Science Advisor|Platform Architect
https://www.analytical.works

On 12/29/19 11:30 AM, Rishi Shah wrote:

Hi All,

Checking in to see if anyone had input around time series
libraries using Spark. I in interested in financial
forecasting model & regression mainly at this point.
Input is a bunch of pricing data points.

I have read a lot of spark-timeseries and flint libraries
but I am not sure of the best way/use cases to use these
libraries for or if there's any other preferred way of
tackling time series problems at scale.

Thanks,

-Shraddha

Thanks Jorn. I am interested in timeseries
forecasting for now but in general I was unable to find a
good way to work with different time series methods using
spark..

Time
series can mean a lot of different things and algorithms.
Can you describe more what you mean by time series use
case, ie what is the input, what do you like to do with
the input and what is the output?

> Am 14.06.2019 um 06:01 schrieb Rishi Shah <[hidden email]>:

>

> Hi All,

>

> I have a time series use case which I would like to
implement in Spark... What would be the best way to do so?
Any built in libraries?

>

> --

> Regards,

>

> Rishi Shah

--

--