spark streaming 2.9.3 vs spark core 2.10

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

spark streaming 2.9.3 vs spark core 2.10

amoc

Hi

I’m wondering if spark streaming 2.9.3 if the previous version of spark core 2.10 or if these are altogether different libraries, one for streaming and one for other things (batch?). These 2 versions don’t work together (at least with my build file) too well.

 

I can find StreamingContext only in 2.9.3 and not in spark core 2.10? Is there such a class in 2.10?

 

Thanks

-A

 

Reply | Threaded
Open this post in threaded view
|

Re: spark streaming 2.9.3 vs spark core 2.10

Prashant Sharma
The ending suffix indicates "scala version" of that released binary and not the version of that artifact.  Obviously they are not supposed to work together. 


On Wed, Feb 12, 2014 at 8:48 PM, Adrian Mocanu <[hidden email]> wrote:

Hi

I’m wondering if spark streaming 2.9.3 if the previous version of spark core 2.10 or if these are altogether different libraries, one for streaming and one for other things (batch?). These 2 versions don’t work together (at least with my build file) too well.

 

I can find StreamingContext only in 2.9.3 and not in spark core 2.10? Is there such a class in 2.10?

 

Thanks

-A

 




--
Prashant
Reply | Threaded
Open this post in threaded view
|

Re: spark streaming 2.9.3 vs spark core 2.10

Tathagata Das
The spark-core_XYZ contains all and only the core Spark batch functionality, and spark-streaming_XYZ contains all the streaming functionality. spark-streaming_XYZ depends on the spark-core_XYZ of the same XYZ, so it is not a good idea to mix and match different versions of spark-core and spark-streaming.

TD


On Wed, Feb 12, 2014 at 9:13 PM, Prashant Sharma <[hidden email]> wrote:
The ending suffix indicates "scala version" of that released binary and not the version of that artifact.  Obviously they are not supposed to work together. 


On Wed, Feb 12, 2014 at 8:48 PM, Adrian Mocanu <[hidden email]> wrote:

Hi

I’m wondering if spark streaming 2.9.3 if the previous version of spark core 2.10 or if these are altogether different libraries, one for streaming and one for other things (batch?). These 2 versions don’t work together (at least with my build file) too well.

 

I can find StreamingContext only in 2.9.3 and not in spark core 2.10? Is there such a class in 2.10?

 

Thanks

-A

 




--
Prashant