Time-Series Forecasting

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Time-Series Forecasting

Mina Aslani
Hi,
I have a question for you. Do we have any Time-Series Forecasting library in Spark? 

Best regards,
Mina
Reply | Threaded
Open this post in threaded view
|

Re: Time-Series Forecasting

Mina Aslani
Hi,
I saw spark-ts, however, looks like it's not under active development any more. I really appreciate to get your insight.

Kindest regards,
Mina

On Wed, Sep 19, 2018 at 12:01 PM Mina Aslani <[hidden email]> wrote:
Hi,
I have a question for you. Do we have any Time-Series Forecasting library in Spark? 

Best regards,
Mina
Reply | Threaded
Open this post in threaded view
|

Re: Time-Series Forecasting

Jörn Franke
In reply to this post by Mina Aslani
What functionality do you need ? Ie which methods?

> On 19. Sep 2018, at 18:01, Mina Aslani <[hidden email]> wrote:
>
> Hi,
> I have a question for you. Do we have any Time-Series Forecasting library in Spark?
>
> Best regards,
> Mina

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Time-Series Forecasting

chris-2
There’s also flint: https://github.com/twosigma/flint

On 19 Sep 2018, at 17:55, Jörn Franke <[hidden email]> wrote:

What functionality do you need ? Ie which methods?

On 19. Sep 2018, at 18:01, Mina Aslani <[hidden email]> wrote:

Hi,
I have a question for you. Do we have any Time-Series Forecasting library in Spark?

Best regards,
Mina

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Time-Series Forecasting

Mina Aslani
In reply to this post by Jörn Franke

Hi,

Thank you for your quick response, really appreciate it.

I just started learning TimeSeries forecasting, and I may try different methods and observe their predictions/forecasting.

However, my understanding is that below methods are needed:


- Smoothing

- Decomposing(e.g. remove/separate trend/seasonality)

- AR Model/MA Model/Combined Model (e.g. ARMA, ARIMA)

- ACF (Autocorrelation Function)/PACF (Partial Autocorrelation Function)

- Recurrent Neural Network (LSTM: Long Short Term Memory)


Kindest regards,
Mina



On Wed, Sep 19, 2018 at 12:55 PM Jörn Franke <[hidden email]> wrote:
What functionality do you need ? Ie which methods?

> On 19. Sep 2018, at 18:01, Mina Aslani <[hidden email]> wrote:
>
> Hi,
> I have a question for you. Do we have any Time-Series Forecasting library in Spark?
>
> Best regards,
> Mina
Reply | Threaded
Open this post in threaded view
|

Re: Time-Series Forecasting

ayan guha
Hi

I work mostly in data engineering and trying to promote use of sparkR within the company I recently joined. Some of the users are working around forecasting a bunch of things and want to use SparklyR as they found time series implementation is better than SparkR. 

Does anyone have a point of view regarding this? Is SparklyR is better than SparkR in certain use cases?

On Thu, Sep 20, 2018 at 4:07 AM, Mina Aslani <[hidden email]> wrote:

Hi,

Thank you for your quick response, really appreciate it.

I just started learning TimeSeries forecasting, and I may try different methods and observe their predictions/forecasting.

However, my understanding is that below methods are needed:


- Smoothing

- Decomposing(e.g. remove/separate trend/seasonality)

- AR Model/MA Model/Combined Model (e.g. ARMA, ARIMA)

- ACF (Autocorrelation Function)/PACF (Partial Autocorrelation Function)

- Recurrent Neural Network (LSTM: Long Short Term Memory)


Kindest regards,
Mina



On Wed, Sep 19, 2018 at 12:55 PM Jörn Franke <[hidden email]> wrote:
What functionality do you need ? Ie which methods?

> On 19. Sep 2018, at 18:01, Mina Aslani <[hidden email]> wrote:
>
> Hi,
> I have a question for you. Do we have any Time-Series Forecasting library in Spark?
>
> Best regards,
> Mina



--
Best Regards,
Ayan Guha
Reply | Threaded
Open this post in threaded view
|

Re: Time-Series Forecasting

Aakash Basu-2
Hey,

Even though I'm more of a Data Engineer than Data Scientist, but still, I work closely with the DS guys extensively on Spark ML, it is something which they're still working on following the scikit-learn trend, but, I never saw Spark handling Time-Series problems. Talking about both Scala-Spark and PySpark.

So, in short, I think it is yet to be added in the future releases of Spark, that too, Scala-Spark will get the first release and then they'll come to other language APIs in future minor releases as per need, usage and importance.

Best,
AB.

On Thu 20 Sep, 2018, 4:43 AM ayan guha, <[hidden email]> wrote:
Hi

I work mostly in data engineering and trying to promote use of sparkR within the company I recently joined. Some of the users are working around forecasting a bunch of things and want to use SparklyR as they found time series implementation is better than SparkR. 

Does anyone have a point of view regarding this? Is SparklyR is better than SparkR in certain use cases?

On Thu, Sep 20, 2018 at 4:07 AM, Mina Aslani <[hidden email]> wrote:

Hi,

Thank you for your quick response, really appreciate it.

I just started learning TimeSeries forecasting, and I may try different methods and observe their predictions/forecasting.

However, my understanding is that below methods are needed:


- Smoothing

- Decomposing(e.g. remove/separate trend/seasonality)

- AR Model/MA Model/Combined Model (e.g. ARMA, ARIMA)

- ACF (Autocorrelation Function)/PACF (Partial Autocorrelation Function)

- Recurrent Neural Network (LSTM: Long Short Term Memory)


Kindest regards,
Mina



On Wed, Sep 19, 2018 at 12:55 PM Jörn Franke <[hidden email]> wrote:
What functionality do you need ? Ie which methods?

> On 19. Sep 2018, at 18:01, Mina Aslani <[hidden email]> wrote:
>
> Hi,
> I have a question for you. Do we have any Time-Series Forecasting library in Spark?
>
> Best regards,
> Mina



--
Best Regards,
Ayan Guha
Reply | Threaded
Open this post in threaded view
|

Re: Time-Series Forecasting

Akash Mishra
We are using Yahoo Egads for our Anomaly Detection system on time series data. If has good forecasting and Anomaly Detection modules. 



On Thu, Sep 20, 2018 at 5:22 AM Aakash Basu <[hidden email]> wrote:
Hey,

Even though I'm more of a Data Engineer than Data Scientist, but still, I work closely with the DS guys extensively on Spark ML, it is something which they're still working on following the scikit-learn trend, but, I never saw Spark handling Time-Series problems. Talking about both Scala-Spark and PySpark.

So, in short, I think it is yet to be added in the future releases of Spark, that too, Scala-Spark will get the first release and then they'll come to other language APIs in future minor releases as per need, usage and importance.

Best,
AB.

On Thu 20 Sep, 2018, 4:43 AM ayan guha, <[hidden email]> wrote:
Hi

I work mostly in data engineering and trying to promote use of sparkR within the company I recently joined. Some of the users are working around forecasting a bunch of things and want to use SparklyR as they found time series implementation is better than SparkR. 

Does anyone have a point of view regarding this? Is SparklyR is better than SparkR in certain use cases?

On Thu, Sep 20, 2018 at 4:07 AM, Mina Aslani <[hidden email]> wrote:

Hi,

Thank you for your quick response, really appreciate it.

I just started learning TimeSeries forecasting, and I may try different methods and observe their predictions/forecasting.

However, my understanding is that below methods are needed:


- Smoothing

- Decomposing(e.g. remove/separate trend/seasonality)

- AR Model/MA Model/Combined Model (e.g. ARMA, ARIMA)

- ACF (Autocorrelation Function)/PACF (Partial Autocorrelation Function)

- Recurrent Neural Network (LSTM: Long Short Term Memory)


Kindest regards,
Mina



On Wed, Sep 19, 2018 at 12:55 PM Jörn Franke <[hidden email]> wrote:
What functionality do you need ? Ie which methods?

> On 19. Sep 2018, at 18:01, Mina Aslani <[hidden email]> wrote:
>
> Hi,
> I have a question for you. Do we have any Time-Series Forecasting library in Spark?
>
> Best regards,
> Mina



--
Best Regards,
Ayan Guha


--

Regards,
Akash Mishra.


"It's not our abilities that make us, but our decisions."--Albus Dumbledore

Reply | Threaded
Open this post in threaded view
|

Re: Time-Series Forecasting

Gourav Sengupta
In reply to this post by Mina Aslani
Hi,

If you are following the time series forecasting with the mathematical rigour and tractability then I think that using R is the best option. I do think that people tend to claim quite a lot these days that SPARK ML and other Python libraries are better, but just pick up a classical text book on time series forecasting and start asking fundamental mathematical questions and compare for yourself.


Regards,
Gourav Sengupta

On Wed, Sep 19, 2018 at 5:02 PM Mina Aslani <[hidden email]> wrote:
Hi,
I have a question for you. Do we have any Time-Series Forecasting library in Spark? 

Best regards,
Mina
Reply | Threaded
Open this post in threaded view
|

Re: Time-Series Forecasting

Peyman Mohajerian
I don't have an opinion about it, just that Flint was mentioned earlier.

On Thu, Sep 20, 2018 at 2:12 AM, Gourav Sengupta <[hidden email]> wrote:
Hi,

If you are following the time series forecasting with the mathematical rigour and tractability then I think that using R is the best option. I do think that people tend to claim quite a lot these days that SPARK ML and other Python libraries are better, but just pick up a classical text book on time series forecasting and start asking fundamental mathematical questions and compare for yourself.


Regards,
Gourav Sengupta

On Wed, Sep 19, 2018 at 5:02 PM Mina Aslani <[hidden email]> wrote:
Hi,
I have a question for you. Do we have any Time-Series Forecasting library in Spark? 

Best regards,
Mina

Reply | Threaded
Open this post in threaded view
|

Re: Time-Series Forecasting

Mina Aslani
Thank you very much, really appreciate the information.

Kindest regards,
Mina

On Sat, Sep 29, 2018 at 9:42 PM Peyman Mohajerian <[hidden email]> wrote:
I don't have an opinion about it, just that Flint was mentioned earlier.

On Thu, Sep 20, 2018 at 2:12 AM, Gourav Sengupta <[hidden email]> wrote:
Hi,

If you are following the time series forecasting with the mathematical rigour and tractability then I think that using R is the best option. I do think that people tend to claim quite a lot these days that SPARK ML and other Python libraries are better, but just pick up a classical text book on time series forecasting and start asking fundamental mathematical questions and compare for yourself.


Regards,
Gourav Sengupta

On Wed, Sep 19, 2018 at 5:02 PM Mina Aslani <[hidden email]> wrote:
Hi,
I have a question for you. Do we have any Time-Series Forecasting library in Spark? 

Best regards,
Mina