Testing with spark-base-test

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Testing with spark-base-test

Guillermo Ortiz

I'm using spark-unit-test and I don't get to compile the code.

  test("Testging") {
    val inputInsert = A("data2")
    val inputDelete = A("data1")
    val outputInsert = B(1)
    val outputDelete = C(1)

    val input = List(List(inputInsert), List(inputDelete))
    val output = (List(List(outputInsert)), List(List(outputDelete)))

    //Why doesn't it compile?? I have tried many things here.
    testOperation[A,(B,C)](input, service.processing _, output)
  }

My method is:

def processing(avroDstream: DStream[A]) : (DStream[B],DStream[C]) ={...}

What does the "_" means in this case?