conflicting version question

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

conflicting version question

Nathan Kronenfeld-2
Our code is currently using Gson 2.8.5.  Spark, through Hadoop-API, pulls in Gson 2.2.4.

At the moment, we just get "method X not found" exceptions because of this - because when we run in Spark, 2.2.4 is what gets loaded.

Is there any way to have both versions exist simultaneously? To load 2.8.5 so that our code uses it, without messing up spark?

Thanks,
                  -Nathan Kronenfeld
Reply | Threaded
Open this post in threaded view
|

Re: conflicting version question

Anastasios Zouzias
Hi Nathan,

You can try to shade the dependency version that you want to use. That said, shading is a tricky technique. Good luck.



See also elasticsearch's discussion on shading


Best,
Anastasios


On Fri, 26 Oct 2018, 15:45 Nathan Kronenfeld, <[hidden email]> wrote:
Our code is currently using Gson 2.8.5.  Spark, through Hadoop-API, pulls in Gson 2.2.4.

At the moment, we just get "method X not found" exceptions because of this - because when we run in Spark, 2.2.4 is what gets loaded.

Is there any way to have both versions exist simultaneously? To load 2.8.5 so that our code uses it, without messing up spark?

Thanks,
                  -Nathan Kronenfeld
Reply | Threaded
Open this post in threaded view
|

Re: conflicting version question

Nathan Kronenfeld-2
Thanks for the suggestion.

Ouch.  That looks painful.

On Fri, Oct 26, 2018 at 1:28 PM Anastasios Zouzias <[hidden email]> wrote:
Hi Nathan,

You can try to shade the dependency version that you want to use. That said, shading is a tricky technique. Good luck.



See also elasticsearch's discussion on shading


Best,
Anastasios


On Fri, 26 Oct 2018, 15:45 Nathan Kronenfeld, <[hidden email]> wrote:
Our code is currently using Gson 2.8.5.  Spark, through Hadoop-API, pulls in Gson 2.2.4.

At the moment, we just get "method X not found" exceptions because of this - because when we run in Spark, 2.2.4 is what gets loaded.

Is there any way to have both versions exist simultaneously? To load 2.8.5 so that our code uses it, without messing up spark?

Thanks,
                  -Nathan Kronenfeld