Spark Job Server application compilation issue

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Spark Job Server application compilation issue

cyberjog

Input is a json request, which would be decoded in myJob() & processed further.

Not sure what is wrong with below code, it emits errors as unimplemented methods (runJob/validate), 
any pointers on this would be helpful, 

jobserver-0.8.0 

object MyJobServer extends SparkSessionJob {

  type JobData = String
  type JobOutput = Seq[String]

  def myJob(a : String)  = {
}

  def runJob(sc: SparkContext, runtime: JobEnvironment, data: JobData): JobOutput = {
           myJob(a)
   }

 def validate(sc: SparkContext, runtime: JobEnvironment, config: Config): JobData Or Every[ValidationProblem] = {
   Good(config.root().render())
 }

Reply | Threaded
Open this post in threaded view
|

Re: Spark Job Server application compilation issue

Vadim Semenov-2
This question should be directed to the `spark-jobserver` group: https://github.com/spark-jobserver/spark-jobserver#contact

They also have a gitter chat.

Also include the errors you get once you're going to be asking them a question

On Wed, Mar 14, 2018 at 1:37 PM, sujeet jog <[hidden email]> wrote:

Input is a json request, which would be decoded in myJob() & processed further.

Not sure what is wrong with below code, it emits errors as unimplemented methods (runJob/validate), 
any pointers on this would be helpful, 

jobserver-0.8.0 

object MyJobServer extends SparkSessionJob {

  type JobData = String
  type JobOutput = Seq[String]

  def myJob(a : String)  = {
}

  def runJob(sc: SparkContext, runtime: JobEnvironment, data: JobData): JobOutput = {
           myJob(a)
   }

 def validate(sc: SparkContext, runtime: JobEnvironment, config: Config): JobData Or Every[ValidationProblem] = {
   Good(config.root().render())
 }




--
Sent from my iPhone
Reply | Threaded
Open this post in threaded view
|

Re: Spark Job Server application compilation issue

cyberjog
Thanks for pointing . 


On Wed, Mar 14, 2018 at 11:19 PM, Vadim Semenov <[hidden email]> wrote:
This question should be directed to the `spark-jobserver` group: https://github.com/spark-jobserver/spark-jobserver#contact

They also have a gitter chat.

Also include the errors you get once you're going to be asking them a question

On Wed, Mar 14, 2018 at 1:37 PM, sujeet jog <[hidden email]> wrote:

Input is a json request, which would be decoded in myJob() & processed further.

Not sure what is wrong with below code, it emits errors as unimplemented methods (runJob/validate), 
any pointers on this would be helpful, 

jobserver-0.8.0 

object MyJobServer extends SparkSessionJob {

  type JobData = String
  type JobOutput = Seq[String]

  def myJob(a : String)  = {
}

  def runJob(sc: SparkContext, runtime: JobEnvironment, data: JobData): JobOutput = {
           myJob(a)
   }

 def validate(sc: SparkContext, runtime: JobEnvironment, config: Config): JobData Or Every[ValidationProblem] = {
   Good(config.root().render())
 }




--
Sent from my iPhone