I have spark job that is failing after the upgrade of the cdh from 5.5.4 which had spark 1.5.0 to cdh 5.13.0 which has spark 1.6.0
The job is running with the new spark dependancies but i see strange behaviour for one spark job that:
1) sometimes it's oozie launcher marked as success and other as killed,
You can see below snapshots and the log for the 2 oozies attached.
2) also for the spark job itself i see that is failing on the jackson databind.
2018-01-05 19:07:17,672 [Driver] ERROR org.apache.spark.deploy.yarn.ApplicationMaster - User class threw exception: java.lang.VerifyError: Bad type on operand stack
org/apache/spark/metrics/sink/MetricsServlet.<init>(Ljava/util/Properties;Lcom/codahale/metrics/MetricRegistry;Lorg/apache/spark/SecurityManager;)V @116: invokevirtual
Type 'com/codahale/metrics/json/MetricsModule' (current frame, stack) is not assignable to 'com/fasterxml/jackson/databind/Module'
attached the log for the 2 jobs one which is success the other failed.