ZeroMQ Stream -> stack guard problem and no data

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

ZeroMQ Stream -> stack guard problem and no data

Tobias Pfeiffer
Hi,

I am trying to use Spark Streaming (1.0.0) with ZeroMQ, i.e. I say

    def bytesToStringIterator(x: Seq[ByteString]) =
(x.map(_.utf8String)).iterator
    val lines: DStream[String] =
      ZeroMQUtils.createStream(ssc,
        "tcp://localhost:5556",
        Subscribe("mytopic"),
        bytesToStringIterator _)
    lines.print()

but when I start this program (in local mode), I get

OpenJDK 64-Bit Server VM warning: You have loaded library
/tmp/jna2713405829859698528.tmp which might have disabled stack guard.
The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c
<libfile>', or link it with '-z noexecstack'.

and no data is received. The ZeroMQ setup should be ok, though; the Python code

    context = zmq.Context()
    socket = context.socket(zmq.SUB)
    socket.setsockopt(zmq.SUBSCRIBE, "mytopic")
    socket.connect("tcp://localhost:5556")
    while True:
        msg = socket.recv()
        print msg
        time.sleep(1)

works fine and prints the messages issued by the publisher.

Any suggestions on what is going wrong here?

Thanks
Tobias
Reply | Threaded
Open this post in threaded view
|

Re: ZeroMQ Stream -> stack guard problem and no data

Prashant Sharma
Hi,

What is your Zeromq version ? It is known to work well with 2.2

an output of `sudo ldconfig -v | grep zmq` would helpful in this regard.

Thanks

Prashant Sharma


On Wed, Jun 4, 2014 at 11:40 AM, Tobias Pfeiffer <[hidden email]> wrote:
Hi,

I am trying to use Spark Streaming (1.0.0) with ZeroMQ, i.e. I say

    def bytesToStringIterator(x: Seq[ByteString]) =
(x.map(_.utf8String)).iterator
    val lines: DStream[String] =
      ZeroMQUtils.createStream(ssc,
        "tcp://localhost:5556",
        Subscribe("mytopic"),
        bytesToStringIterator _)
    lines.print()

but when I start this program (in local mode), I get

OpenJDK 64-Bit Server VM warning: You have loaded library
/tmp/jna2713405829859698528.tmp which might have disabled stack guard.
The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c
<libfile>', or link it with '-z noexecstack'.

and no data is received. The ZeroMQ setup should be ok, though; the Python code

    context = zmq.Context()
    socket = context.socket(zmq.SUB)
    socket.setsockopt(zmq.SUBSCRIBE, "mytopic")
    socket.connect("tcp://localhost:5556")
    while True:
        msg = socket.recv()
        print msg
        time.sleep(1)

works fine and prints the messages issued by the publisher.

Any suggestions on what is going wrong here?

Thanks
Tobias

Reply | Threaded
Open this post in threaded view
|

Re: ZeroMQ Stream -> stack guard problem and no data

sowen
In reply to this post by Tobias Pfeiffer
It's complaining about the native library shipped with ZeroMQ, right?
That message is the JVM complaining about how it was compiled. If so,
I think it's a question for ZeroMQ?

On Wed, Jun 4, 2014 at 7:10 AM, Tobias Pfeiffer <[hidden email]> wrote:

> Hi,
>
> I am trying to use Spark Streaming (1.0.0) with ZeroMQ, i.e. I say
>
>     def bytesToStringIterator(x: Seq[ByteString]) =
> (x.map(_.utf8String)).iterator
>     val lines: DStream[String] =
>       ZeroMQUtils.createStream(ssc,
>         "tcp://localhost:5556",
>         Subscribe("mytopic"),
>         bytesToStringIterator _)
>     lines.print()
>
> but when I start this program (in local mode), I get
>
> OpenJDK 64-Bit Server VM warning: You have loaded library
> /tmp/jna2713405829859698528.tmp which might have disabled stack guard.
> The VM will try to fix the stack guard now.
> It's highly recommended that you fix the library with 'execstack -c
> <libfile>', or link it with '-z noexecstack'.
>
> and no data is received. The ZeroMQ setup should be ok, though; the Python code
>
>     context = zmq.Context()
>     socket = context.socket(zmq.SUB)
>     socket.setsockopt(zmq.SUBSCRIBE, "mytopic")
>     socket.connect("tcp://localhost:5556")
>     while True:
>         msg = socket.recv()
>         print msg
>         time.sleep(1)
>
> works fine and prints the messages issued by the publisher.
>
> Any suggestions on what is going wrong here?
>
> Thanks
> Tobias
Reply | Threaded
Open this post in threaded view
|

Re: ZeroMQ Stream -> stack guard problem and no data

Tobias Pfeiffer
Hi,

thanks for your messages. I'm not pursuing this further, though.
People in #zeromq IRC advised strongly against using libraries based
on version 2 of libzmq (for example, the Akka ZeroMQ library) due to a
number issues in that version. In fact,
https://github.com/zeromq/jeromq seems the way to go for ZeroMQ use in
Java/Scala, as it seems...

Thanks
Tobias