missing method in my slf4j after excluding Spark ZK log4j

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

missing method in my slf4j after excluding Spark ZK log4j

amoc

Hey guys,

I've asked before, in Spark 0.9 - I now use 0.9.1, about removing log4j dependency and was told that it was gone. However I still find it part of zookeeper imports. This is fine since I exclude it myself in the sbt file, but another issue arises.

I wonder if anyone else has run into this.

 

Spark uses log4j v1.2.17 and slf4j-log4j12:1.7.2

I use slf4j 1.7.5, logback 1.0.13, and log4joverslf4j v 1.7.5

 

I think my slf4j 1.7.5 doesn't agree with what zookeeper expects in its log4j v 1.2.17 because I get missing method error:

java.lang.NoSuchMethodError: org.apache.log4j.Logger.setLevel(Lorg/apache/log4j/Level;)V

                at org.apache.spark.util.AkkaUtils$$anonfun$createActorSystem$1.apply(AkkaUtils.scala:58)

                at org.apache.spark.util.AkkaUtils$$anonfun$createActorSystem$1.apply(AkkaUtils.scala:58)

                at scala.Option.map(Option.scala:145)

                at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:58)

                at org.apache.spark.SparkEnv$.create(SparkEnv.scala:126)

                at org.apache.spark.SparkContext.<init>(SparkContext.scala:139)

                at org.apache.spark.streaming.StreamingContext$.createNewSparkContext(StreamingContext.scala:500)

                at org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:76)

...

 

Is there a way to find out what versions of slf4j I need to make it work with log4j 1.2.17?

 

-Adrian

 

Reply | Threaded
Open this post in threaded view
|

Re: missing method in my slf4j after excluding Spark ZK log4j

sowen
It sounds like you are doing everything right.

NoSuchMethodError suggests it's finding log4j, just not the right
version. That method is definitely in 1.2; it might have been removed
in 2.x? (http://logging.apache.org/log4j/2.x/manual/migration.html)

So I wonder if something is sneaking in log4j 2.x in your app? that's
a first guess.

I'd say consult mvn dependency:tree, but you're on sbt and I don't
know the equivalent.

On Mon, May 12, 2014 at 3:51 PM, Adrian Mocanu
<[hidden email]> wrote:

> Hey guys,
>
> I've asked before, in Spark 0.9 - I now use 0.9.1, about removing log4j
> dependency and was told that it was gone. However I still find it part of
> zookeeper imports. This is fine since I exclude it myself in the sbt file,
> but another issue arises.
>
> I wonder if anyone else has run into this.
>
>
>
> Spark uses log4j v1.2.17 and slf4j-log4j12:1.7.2
>
> I use slf4j 1.7.5, logback 1.0.13, and log4joverslf4j v 1.7.5
>
>
>
> I think my slf4j 1.7.5 doesn't agree with what zookeeper expects in its
> log4j v 1.2.17 because I get missing method error:
>
> java.lang.NoSuchMethodError:
> org.apache.log4j.Logger.setLevel(Lorg/apache/log4j/Level;)V
>
>                 at
> org.apache.spark.util.AkkaUtils$$anonfun$createActorSystem$1.apply(AkkaUtils.scala:58)
>
>                 at
> org.apache.spark.util.AkkaUtils$$anonfun$createActorSystem$1.apply(AkkaUtils.scala:58)
>
>                 at scala.Option.map(Option.scala:145)
>
>                 at
> org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:58)
>
>                 at org.apache.spark.SparkEnv$.create(SparkEnv.scala:126)
>
>                 at
> org.apache.spark.SparkContext.<init>(SparkContext.scala:139)
>
>                 at
> org.apache.spark.streaming.StreamingContext$.createNewSparkContext(StreamingContext.scala:500)
>
>                 at
> org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:76)
>
> ...
>
>
>
> Is there a way to find out what versions of slf4j I need to make it work
> with log4j 1.2.17?
>
>
>
> -Adrian
>
>
Reply | Threaded
Open this post in threaded view
|

Re: missing method in my slf4j after excluding Spark ZK log4j

Tathagata Das
This gives dependency tree in SBT (spark uses this). 

TD


On Mon, May 12, 2014 at 4:55 PM, Sean Owen <[hidden email]> wrote:
It sounds like you are doing everything right.

NoSuchMethodError suggests it's finding log4j, just not the right
version. That method is definitely in 1.2; it might have been removed
in 2.x? (http://logging.apache.org/log4j/2.x/manual/migration.html)

So I wonder if something is sneaking in log4j 2.x in your app? that's
a first guess.

I'd say consult mvn dependency:tree, but you're on sbt and I don't
know the equivalent.

On Mon, May 12, 2014 at 3:51 PM, Adrian Mocanu
<[hidden email]> wrote:
> Hey guys,
>
> I've asked before, in Spark 0.9 - I now use 0.9.1, about removing log4j
> dependency and was told that it was gone. However I still find it part of
> zookeeper imports. This is fine since I exclude it myself in the sbt file,
> but another issue arises.
>
> I wonder if anyone else has run into this.
>
>
>
> Spark uses log4j v1.2.17 and slf4j-log4j12:1.7.2
>
> I use slf4j 1.7.5, logback 1.0.13, and log4joverslf4j v 1.7.5
>
>
>
> I think my slf4j 1.7.5 doesn't agree with what zookeeper expects in its
> log4j v 1.2.17 because I get missing method error:
>
> java.lang.NoSuchMethodError:
> org.apache.log4j.Logger.setLevel(Lorg/apache/log4j/Level;)V
>
>                 at
> org.apache.spark.util.AkkaUtils$$anonfun$createActorSystem$1.apply(AkkaUtils.scala:58)
>
>                 at
> org.apache.spark.util.AkkaUtils$$anonfun$createActorSystem$1.apply(AkkaUtils.scala:58)
>
>                 at scala.Option.map(Option.scala:145)
>
>                 at
> org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:58)
>
>                 at org.apache.spark.SparkEnv$.create(SparkEnv.scala:126)
>
>                 at
> org.apache.spark.SparkContext.<init>(SparkContext.scala:139)
>
>                 at
> org.apache.spark.streaming.StreamingContext$.createNewSparkContext(StreamingContext.scala:500)
>
>                 at
> org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:76)
>
> ...
>
>
>
> Is there a way to find out what versions of slf4j I need to make it work
> with log4j 1.2.17?
>
>
>
> -Adrian
>
>

Reply | Threaded
Open this post in threaded view
|

Re: missing method in my slf4j after excluding Spark ZK log4j

Paul Brown
In reply to this post by amoc

Hi, Adrian --

If my memory serves, you need 1.7.7 of the various slf4j modules to avoid that issue.

Best.
-- Paul


[hidden email] | Multifarious, Inc. | http://mult.ifario.us/


On Mon, May 12, 2014 at 7:51 AM, Adrian Mocanu <[hidden email]> wrote:

Hey guys,

I've asked before, in Spark 0.9 - I now use 0.9.1, about removing log4j dependency and was told that it was gone. However I still find it part of zookeeper imports. This is fine since I exclude it myself in the sbt file, but another issue arises.

I wonder if anyone else has run into this.

 

Spark uses log4j v1.2.17 and slf4j-log4j12:1.7.2

I use slf4j 1.7.5, logback 1.0.13, and log4joverslf4j v 1.7.5

 

I think my slf4j 1.7.5 doesn't agree with what zookeeper expects in its log4j v 1.2.17 because I get missing method error:

java.lang.NoSuchMethodError: org.apache.log4j.Logger.setLevel(Lorg/apache/log4j/Level;)V

                at org.apache.spark.util.AkkaUtils$$anonfun$createActorSystem$1.apply(AkkaUtils.scala:58)

                at org.apache.spark.util.AkkaUtils$$anonfun$createActorSystem$1.apply(AkkaUtils.scala:58)

                at scala.Option.map(Option.scala:145)

                at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:58)

                at org.apache.spark.SparkEnv$.create(SparkEnv.scala:126)

                at org.apache.spark.SparkContext.<init>(SparkContext.scala:139)

                at org.apache.spark.streaming.StreamingContext$.createNewSparkContext(StreamingContext.scala:500)

                at org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:76)

...

 

Is there a way to find out what versions of slf4j I need to make it work with log4j 1.2.17?

 

-Adrian

 


Reply | Threaded
Open this post in threaded view
|

RE: missing method in my slf4j after excluding Spark ZK log4j

amoc
In reply to this post by sowen
Yea, had to change the versions of  slf4j and log4joversslf4j to1.7.6

-----Original Message-----
From: Sean Owen [mailto:[hidden email]]
Sent: May-12-14 7:55 PM
To: [hidden email]
Subject: Re: missing method in my slf4j after excluding Spark ZK log4j

It sounds like you are doing everything right.

NoSuchMethodError suggests it's finding log4j, just not the right version. That method is definitely in 1.2; it might have been removed in 2.x? (http://logging.apache.org/log4j/2.x/manual/migration.html)

So I wonder if something is sneaking in log4j 2.x in your app? that's a first guess.

I'd say consult mvn dependency:tree, but you're on sbt and I don't know the equivalent.

On Mon, May 12, 2014 at 3:51 PM, Adrian Mocanu <[hidden email]> wrote:

> Hey guys,
>
> I've asked before, in Spark 0.9 - I now use 0.9.1, about removing
> log4j dependency and was told that it was gone. However I still find
> it part of zookeeper imports. This is fine since I exclude it myself
> in the sbt file, but another issue arises.
>
> I wonder if anyone else has run into this.
>
>
>
> Spark uses log4j v1.2.17 and slf4j-log4j12:1.7.2
>
> I use slf4j 1.7.5, logback 1.0.13, and log4joverslf4j v 1.7.5
>
>
>
> I think my slf4j 1.7.5 doesn't agree with what zookeeper expects in
> its log4j v 1.2.17 because I get missing method error:
>
> java.lang.NoSuchMethodError:
> org.apache.log4j.Logger.setLevel(Lorg/apache/log4j/Level;)V
>
>                 at
> org.apache.spark.util.AkkaUtils$$anonfun$createActorSystem$1.apply(Akk
> aUtils.scala:58)
>
>                 at
> org.apache.spark.util.AkkaUtils$$anonfun$createActorSystem$1.apply(Akk
> aUtils.scala:58)
>
>                 at scala.Option.map(Option.scala:145)
>
>                 at
> org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:58)
>
>                 at
> org.apache.spark.SparkEnv$.create(SparkEnv.scala:126)
>
>                 at
> org.apache.spark.SparkContext.<init>(SparkContext.scala:139)
>
>                 at
> org.apache.spark.streaming.StreamingContext$.createNewSparkContext(Str
> eamingContext.scala:500)
>
>                 at
> org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.sc
> ala:76)
>
> ...
>
>
>
> Is there a way to find out what versions of slf4j I need to make it
> work with log4j 1.2.17?
>
>
>
> -Adrian
>
>