submitting dependencies

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

submitting dependencies

amin mohebbi
Could you please help me to understand how I should submit my spark application ?

I have used this connector (pygmalios/reactiveinflux-spark) to connect spark with influxdb. 



The code is working fine, but when I submit the jar file through spark-submit I will face this error 

"Exception in thread "main" java.lang.NoClassDefFoundError: com/pygmalios/reactiveinflux/ReactiveInfluxDbName"

I submit like this : 
spark-submit --jars /home/sshuser/reactiveinflux-spark_2.10-1.4.0.10.0.5.1.jar sapn_2.11-1.0.jar

Can you help to solve this issue? 


Best Regards ....................................................... Amin Mohebbi PhD candidate in Software Engineering   at university of Malaysia   Tel : +60 18 2040 017 E-Mail : [hidden email]               [hidden email]
jgp
Reply | Threaded
Open this post in threaded view
|

Re: submitting dependencies

jgp
Have you tried to build a uber jar to bundle all your classes together?

On Jun 27, 2018, at 01:27, amin mohebbi <[hidden email]> wrote:

Could you please help me to understand how I should submit my spark application ?

I have used this connector (pygmalios/reactiveinflux-spark) to connect spark with influxdb. 



The code is working fine, but when I submit the jar file through spark-submit I will face this error 

"Exception in thread "main" java.lang.NoClassDefFoundError: com/pygmalios/reactiveinflux/ReactiveInfluxDbName"

I submit like this : 
spark-submit --jars /home/sshuser/reactiveinflux-spark_2.10-1.4.0.10.0.5.1.jar sapn_2.11-1.0.jar

Can you help to solve this issue? 


Best Regards ....................................................... Amin Mohebbi PhD candidate in Software Engineering   at university of Malaysia   Tel : +60 18 2040 017 E-Mail : [hidden email]               [hidden email]