Spark Streaming Maven Build

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Spark Streaming Maven Build

bin wang
Hi there, 

I tried the Kafka WordCount example and it works perfect and the code is pretty straightforward to understand. 

Can anyone show to me how to start your own maven project with the KafkaWordCount example using minimum-effort. 

1. How the pom file should look like (including jar-plugin? assembly-plugin?..etc)
2. mvn install or mvn clean install or mvn install compile assembly:single?
3. after you have a jar file, then how do you execute the jar file instead of using bin/run-example...

To answer those people who might ask what you have done
(Here is a derivative from the KafkaWordCount example that I have written to do IP count example where the input data from Kafka is actually JSON string. 
I had such a bad lucky got it to working. So if anyone can copy the code of WordCountExample and build it using maven and got it working.. if you can share your pom and those maven commands, I will be so appreciated!)

Best regards and let me know if you need further info.