Spark internal class is not registered by Kryo

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Spark internal class is not registered by Kryo

曹礼俊
Hi developers:

I have set spark.kryo.registrationRequired=true, but an exception occured: java.lang.IllegalArgumentException: Class is not registered: org.apache.spark.internal.io.FileCommitProtocol$TaskCommitMessage when I run the program. 

I tried to register it manually by kryo.register() and Sparkconf.registerKryoClasses, but it still not work.

The related JIRA is SPARK-21569. And SPARK-10251, SPARK-6497 are may the same problem I think. 

Be grateful if someone can help me..

Best Regards,
Lijun Cao