Facing MetricsSystem error on Running Spark applications

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Facing MetricsSystem error on Running Spark applications

Vibhor Banga
Hi,

I am facing following error on running spark applications. What could be missing which is causing this issue.

org.eclipse.jetty.server.AbstractConnector  - Started SocketConnector@0.0.0.0:55046
3574 [main] ERROR org.apache.spark.metrics.MetricsSystem  - Sink class org.apache.spark.metrics.sink.MetricsServlet cannot be instantialized
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:134)
at org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:129)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
at org.apache.spark.metrics.MetricsSystem.registerSinks(MetricsSystem.scala:129)
at org.apache.spark.metrics.MetricsSystem.<init>(MetricsSystem.scala:83)
at org.apache.spark.metrics.MetricsSystem$.createMetricsSystem(MetricsSystem.scala:163)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:198)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:139)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:100)
at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:81)
at com.flipkart.sniper.common.ep.SparkResult.processTable(SparkResult.java:50)
at com.flipkart.sniper.common.ep.LocalEventProcessorRunner.processTable(LocalEventProcessorRunner.java:95)
at com.flipkart.sniper.common.ep.LocalEventProcessorRunner.call(LocalEventProcessorRunner.java:73)
at com.flipkart.sniper.common.job.JobRunner.runJob(JobRunner.java:64)
at com.flipkart.sniper.common.job.JobRunner.main(JobRunner.java:166)
Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.module.SimpleSerializers.<init>(Ljava/util/List;)V
at com.codahale.metrics.json.MetricsModule.setupModule(MetricsModule.java:213)
at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:469)
at org.apache.spark.metrics.sink.MetricsServlet.<init>(MetricsServlet.scala:44)
... 23 more

Thanks,
-Vibhor
Reply | Threaded
Open this post in threaded view
|

Re: Facing MetricsSystem error on Running Spark applications

sowen
You've got a conflict in the version of Jackson that is being used:

Caused by: java.lang.NoSuchMethodError:
com.fasterxml.jackson.databind.module.SimpleSerializers.<init>(Ljava/util/List;)V

Looks like you are using Jackson 2.x somewhere, but AFAIK all of the
Hadoop/Spark libs are still on 1.x.

That's roughly the proximate problem, but how to resolve it will
depend a bit more on what your app is doing.

On Wed, Jun 4, 2014 at 1:31 PM, Vibhor Banga <[hidden email]> wrote:

> Hi,
>
> I am facing following error on running spark applications. What could be
> missing which is causing this issue.
>
> org.eclipse.jetty.server.AbstractConnector  - Started
> SocketConnector@0.0.0.0:55046
> 3574 [main] ERROR org.apache.spark.metrics.MetricsSystem  - Sink class
> org.apache.spark.metrics.sink.MetricsServlet cannot be instantialized
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:134)
> at
> org.apache.spark.metrics.MetricsSystem$$anonfun$registerSinks$1.apply(MetricsSystem.scala:129)
> at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
> at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
> at
> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
> at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
> at
> org.apache.spark.metrics.MetricsSystem.registerSinks(MetricsSystem.scala:129)
> at org.apache.spark.metrics.MetricsSystem.<init>(MetricsSystem.scala:83)
> at
> org.apache.spark.metrics.MetricsSystem$.createMetricsSystem(MetricsSystem.scala:163)
> at org.apache.spark.SparkEnv$.create(SparkEnv.scala:198)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:139)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:100)
> at
> org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:81)
> at
> com.flipkart.sniper.common.ep.SparkResult.processTable(SparkResult.java:50)
> at
> com.flipkart.sniper.common.ep.LocalEventProcessorRunner.processTable(LocalEventProcessorRunner.java:95)
> at
> com.flipkart.sniper.common.ep.LocalEventProcessorRunner.call(LocalEventProcessorRunner.java:73)
> at com.flipkart.sniper.common.job.JobRunner.runJob(JobRunner.java:64)
> at com.flipkart.sniper.common.job.JobRunner.main(JobRunner.java:166)
> Caused by: java.lang.NoSuchMethodError:
> com.fasterxml.jackson.databind.module.SimpleSerializers.<init>(Ljava/util/List;)V
> at
> com.codahale.metrics.json.MetricsModule.setupModule(MetricsModule.java:213)
> at
> com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:469)
> at
> org.apache.spark.metrics.sink.MetricsServlet.<init>(MetricsServlet.scala:44)
> ... 23 more
>
> Thanks,
> -Vibhor