"Failed to find Spark examples assembly..." error message

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

"Failed to find Spark examples assembly..." error message

David Swearingen
I'm running 0.9.0 and attempting to try the example described here: https://spark.incubator.apache.org/docs/0.9.0/streaming-programming-guide.html#a-quick-example 

./bin/run-example org.apache.spark.streaming.examples.JavaNetworkWordCount local[2] localhost 9999

But get the error:

./bin/run-example: line 47: [: /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10/spark-examples-assembly-0.9.0-incubating.jar: binary operator expected
Failed to find Spark examples assembly in /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target
You need to build Spark with sbt/sbt assembly before running this program

I've done the 'sbt/sbt assembly' step.  I've tried to debug the shell script but it seems like the first problem is the missing "assembly" file...

Any suggestions?

Thanks!
______________________________________________________________________
This email has been scanned by the Boundary Defense for Email Security System. For more information please visit http://www.apptix.com/email-security/antispam-virus
______________________________________________________________________
Reply | Threaded
Open this post in threaded view
|

Re: "Failed to find Spark examples assembly..." error message

Mark Hamstra
Is the assembly jar actually missing, or is the script somehow looking in the wrong place?  Check /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10.  If the examples-assembly jar is not there, then either `./sbt/sbt assembly` or `./sbt/sbt examples/assembly` should build it.  Else you need to be looking for why the script is not coming up with the right path to that file.


On Mon, Feb 10, 2014 at 8:49 AM, David Swearingen <[hidden email]> wrote:
I'm running 0.9.0 and attempting to try the example described here: https://spark.incubator.apache.org/docs/0.9.0/streaming-programming-guide.html#a-quick-example 

./bin/run-example org.apache.spark.streaming.examples.JavaNetworkWordCount local[2] localhost 9999

But get the error:

./bin/run-example: line 47: [: /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10/spark-examples-assembly-0.9.0-incubating.jar: binary operator expected
Failed to find Spark examples assembly in /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target
You need to build Spark with sbt/sbt assembly before running this program

I've done the 'sbt/sbt assembly' step.  I've tried to debug the shell script but it seems like the first problem is the missing "assembly" file...

Any suggestions?

Thanks!
______________________________________________________________________
This email has been scanned by the Boundary Defense for Email Security System. For more information please visit http://www.apptix.com/email-security/antispam-virus
______________________________________________________________________

Reply | Threaded
Open this post in threaded view
|

Re: "Failed to find Spark examples assembly..." error message

David Swearingen
The file examples-assembly is not there.

I ran 'sbt/sbt examples assembly' and that didn't change anything.

Note that the shell script is complaining about "binary operator
expected" but I'm not positive that's a cause or a result for some
other problem.

Here's what's in the directory:

ls -al examples/target/scala-2.10/
total 199648
drwxrwxr-x 4 centrifuge centrifuge      4096 Feb  9 14:51 .
drwxrwxr-x 8 centrifuge centrifuge      4096 Feb  9 14:48 ..
drwxrwxr-x 3 centrifuge centrifuge      4096 Feb  9 14:47 cache
drwxrwxr-x 3 centrifuge centrifuge      4096 Feb  9 14:50 classes
-rw-rw-r-- 1 centrifuge centrifuge 125932602 Feb  9 14:51
spark-examples-assembly-0.9.0-incubating.jar
-rw-rw-r-- 1 centrifuge centrifuge  78485738 Feb  2 19:05
spark-examples_2.10-assembly-0.9.0-incubating.jar

Thanks again.

On Mon Feb 10 12:31:31 2014, Mark Hamstra wrote:

> Is the assembly jar actually missing, or is the script somehow looking
> in the wrong place?  Check
> /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10.
>  If the examples-assembly jar is not there, then either `./sbt/sbt
> assembly` or `./sbt/sbt examples/assembly` should build it.  Else you
> need to be looking for why the script is not coming up with the right
> path to that file.
>
>
> On Mon, Feb 10, 2014 at 8:49 AM, David Swearingen
> <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     I'm running 0.9.0 and attempting to try the example described
>     here:
>     https://spark.incubator.apache.org/docs/0.9.0/streaming-programming-guide.html#a-quick-example
>
>
>     ./bin/run-example
>     org.apache.spark.streaming.examples.JavaNetworkWordCount local[2]
>     localhost 9999
>
>     But get the error:
>
>     ./bin/run-example: line 47: [:
>     /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10/spark-examples-assembly-0.9.0-incubating.jar:
>     binary operator expected
>     Failed to find Spark examples assembly in
>     /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target
>     You need to build Spark with sbt/sbt assembly before running this
>     program
>
>     I've done the 'sbt/sbt assembly' step.  I've tried to debug the
>     shell script but it seems like the first problem is the missing
>     "assembly" file...
>
>     Any suggestions?
>
>     Thanks!
>     ______________________________________________________________________
>     This email has been scanned by the Boundary Defense for Email
>     Security System. For more information please visit
>     http://www.apptix.com/email-security/antispam-virus
>     ______________________________________________________________________
>
>
>
> ______________________________________________________________________
> This email has been scanned by the Boundary Defense for Email Security
> System. For more information please visit
> http://www.apptix.com/email-security/antispam-virus
> ______________________________________________________________________

--
----------------------------------------------
Centrifuge Systems
Director, Professional Services
http://www.centrifugesystems.com
7926 Jones Branch Drive  McLean, VA  22102
703-517-6910 (m)

______________________________________________________________________
This email has been scanned by the Boundary Defense for Email Security System. For more information please visit http://www.apptix.com/email-security/antispam-virus
______________________________________________________________________
Reply | Threaded
Open this post in threaded view
|

Re: "Failed to find Spark examples assembly..." error message

Mark Hamstra
Blow away the examples/target directory, then do `./sbt/sbt examples/assembly` -- you've got two assembly jars where there should be only one in target/scala-2.10:

spark-examples-assembly-0.9.0-incubating.jar



On Mon, Feb 10, 2014 at 10:06 AM, David Swearingen <[hidden email]> wrote:
The file examples-assembly is not there.

I ran 'sbt/sbt examples assembly' and that didn't change anything.

Note that the shell script is complaining about "binary operator expected" but I'm not positive that's a cause or a result for some other problem.

Here's what's in the directory:

ls -al examples/target/scala-2.10/
total 199648
drwxrwxr-x 4 centrifuge centrifuge      4096 Feb  9 14:51 .
drwxrwxr-x 8 centrifuge centrifuge      4096 Feb  9 14:48 ..
drwxrwxr-x 3 centrifuge centrifuge      4096 Feb  9 14:47 cache
drwxrwxr-x 3 centrifuge centrifuge      4096 Feb  9 14:50 classes
-rw-rw-r-- 1 centrifuge centrifuge 125932602 Feb  9 14:51 spark-examples-assembly-0.9.0-incubating.jar
-rw-rw-r-- 1 centrifuge centrifuge  78485738 Feb  2 19:05 spark-examples_2.10-assembly-0.9.0-incubating.jar

Thanks again.


On Mon Feb 10 12:31:31 2014, Mark Hamstra wrote:
Is the assembly jar actually missing, or is the script somehow looking
in the wrong place?  Check
/home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10.
 If the examples-assembly jar is not there, then either `./sbt/sbt
assembly` or `./sbt/sbt examples/assembly` should build it.  Else you
need to be looking for why the script is not coming up with the right
path to that file.


On Mon, Feb 10, 2014 at 8:49 AM, David Swearingen
<[hidden email]
<mailto:[hidden email]>> wrote:

    I'm running 0.9.0 and attempting to try the example described
    here:
    https://spark.incubator.apache.org/docs/0.9.0/streaming-programming-guide.html#a-quick-example


    ./bin/run-example
    org.apache.spark.streaming.examples.JavaNetworkWordCount local[2]
    localhost 9999

    But get the error:

    ./bin/run-example: line 47: [:
    /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10/spark-examples-assembly-0.9.0-incubating.jar:
    binary operator expected
    Failed to find Spark examples assembly in
    /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target
    You need to build Spark with sbt/sbt assembly before running this
    program

    I've done the 'sbt/sbt assembly' step.  I've tried to debug the
    shell script but it seems like the first problem is the missing
    "assembly" file...

    Any suggestions?

    Thanks!
    ______________________________________________________________________
    This email has been scanned by the Boundary Defense for Email
    Security System. For more information please visit
    http://www.apptix.com/email-security/antispam-virus
    ______________________________________________________________________



______________________________________________________________________
This email has been scanned by the Boundary Defense for Email Security
System. For more information please visit
http://www.apptix.com/email-security/antispam-virus
______________________________________________________________________

--
----------------------------------------------
Centrifuge Systems
Director, Professional Services
http://www.centrifugesystems.com
7926 Jones Branch Drive  McLean, VA  22102
<a href="tel:703-517-6910" value="+17035176910" target="_blank">703-517-6910 (m)


______________________________________________________________________
This email has been scanned by the Boundary Defense for Email Security System. For more information please visit http://www.apptix.com/email-security/antispam-virus
______________________________________________________________________