com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.9.6 requires Jackson Databind version >= 2.9.0 and < 2.10.0

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.9.6 requires Jackson Databind version >= 2.9.0 and < 2.10.0

Julian Jiang

when I run it in my idea ,it works well.but when I submit to cluster ,it appear this problem.thanks for help me .

  My version is as follow:

       <scala.version>2.11.8</scala.version>

        <hadoop.version>3.1.1</hadoop.version>

        <spark.version>2.3.2</spark.version>

        <clickhouse-jdbc.version>0.2.4</clickhouse-jdbc.version>

My code is as follow :

val spark:SparkSession = SparkSession
  .builder()
  .appName(
"CkConnect")
  .master(
"local[2]")
  .getOrCreate()
val properties = new Properties()
//设置用户名和密码
properties.setProperty("user","*")
properties.setProperty(
"password","*")
val dataFrame: DataFrame = spark.read.jdbc("jdbc:clickhouse://*","stu",properties)

when read jdbc ,it don’t’t work..

 

Exception in thread "main" java.lang.ExceptionInInitializerError

       at org.apache.spark.scheduler.EventLoggingListener$.initEventLog(EventLoggingListener.scala:303)

       at org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:128)

       at org.apache.spark.SparkContext.<init>(SparkContext.scala:522)

       at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2493)

       at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:934)

       at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:925)

       at scala.Option.getOrElse(Option.scala:121)

       at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:925)

       at org.synnex.WordCount.main(WordCount.java:25)

       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

       at java.lang.reflect.Method.invoke(Method.java:498)

       at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)

       at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:904)

       at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)

       at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)

       at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)

       at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Caused by: com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.9.6 requires Jackson Databind version >= 2.9.0 and < 2.10.0

       at com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:61)

       at com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:18)

       at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:722)

       at org.apache.spark.util.JsonProtocol$.<init>(JsonProtocol.scala:59)

       at org.apache.spark.util.JsonProtocol$.<clinit>(JsonProtocol.scala)

       ... 19 more

Reply | Threaded
Open this post in threaded view
|

Re: com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.9.6 requires Jackson Databind version >= 2.9.0 and < 2.10.0

srowen
You have a Jackson version conflict somewhere. It might be from other
libraries you include in your application.
I am not sure Spark 2.3 works with Hadoop 3.1, so this may be the
issue. Make sure you match these to Spark, and/or use the latest
versions.

On Thu, Jul 9, 2020 at 8:23 AM Julian Jiang <[hidden email]> wrote:

>
> when I run it in my idea ,it works well.but when I submit to cluster ,it appear this problem.。thanks for help me .
>
>   My version is as follow:
>
>        <scala.version>2.11.8</scala.version>
>
>         <hadoop.version>3.1.1</hadoop.version>
>
>         <spark.version>2.3.2</spark.version>
>
>         <clickhouse-jdbc.version>0.2.4</clickhouse-jdbc.version>
>
> My code is as follow :
>
> val spark:SparkSession = SparkSession
>   .builder()
>   .appName("CkConnect")
>   .master("local[2]")
>   .getOrCreate()
> val properties = new Properties()
> //设置用户名和密码
> properties.setProperty("user","*")
> properties.setProperty("password","*")
> val dataFrame: DataFrame = spark.read.jdbc("jdbc:clickhouse://*","stu",properties)
>
> when read jdbc ,it don’t’t work..
>
>
>
> Exception in thread "main" java.lang.ExceptionInInitializerError
>
>        at org.apache.spark.scheduler.EventLoggingListener$.initEventLog(EventLoggingListener.scala:303)
>
>        at org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:128)
>
>        at org.apache.spark.SparkContext.<init>(SparkContext.scala:522)
>
>        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2493)
>
>        at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:934)
>
>        at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:925)
>
>        at scala.Option.getOrElse(Option.scala:121)
>
>        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:925)
>
>        at org.synnex.WordCount.main(WordCount.java:25)
>
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
>        at java.lang.reflect.Method.invoke(Method.java:498)
>
>        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
>
>        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:904)
>
>        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
>
>        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
>
>        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
>
>        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>
> Caused by: com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.9.6 requires Jackson Databind version >= 2.9.0 and < 2.10.0
>
>        at com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:61)
>
>        at com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:18)
>
>        at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:722)
>
>        at org.apache.spark.util.JsonProtocol$.<init>(JsonProtocol.scala:59)
>
>        at org.apache.spark.util.JsonProtocol$.<clinit>(JsonProtocol.scala)
>
>        ... 19 more

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]