storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nathan Leung <ncle...@gmail.com>
Subject Re: Segfault in worker when submitting topology
Date Fri, 03 Jan 2014 20:06:13 GMT
can you try storm 0.9.0.1 with zmq 2.1.7 and frozen jzmq?  assuming that it
doesn't work, can you try with netty as your transport?  It would seem to
me that something is wrong with your zmq library,


On Fri, Jan 3, 2014 at 2:58 PM, Brian C. Huffman <
bhuffman@etinternational.com> wrote:

>  Yes - that log was from the zmq 4 run.
>
> Here's the hs_err_pid file from the 2.1.7 run:
> # Problematic frame:
> # C  [libzmq.so.1+0x19100]  zmq::i_inout::~i_inout()+0x10
>
> <snip>
>
> Stack: [0x00007f3a337d2000,0x00007f3a338d3000],  sp=0x00007f3a338d0f08,
> free space=1019k
>
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> C  [libzmq.so.1+0x19100]  zmq::i_inout::~i_inout()+0x10
>
>
> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
> j  org.zeromq.ZMQ$Socket.setLongSockopt(IJ)V+0
> j  org.zeromq.ZMQ$Socket.setLinger(J)V+17
>
> Thanks,
> Brian
>
>
> On 01/03/2014 02:46 PM, Nathan Leung wrote:
>
> You shouldn't use a newer version of zmq, the API changed.  If you try a
> newer version of storm you should try it with zmq 2.1.7 and the frozen
> jzmq.  Is the log you posted from the zmq 4 run?  What does it look like
> with zmq 2.1.7?
>
>
> On Fri, Jan 3, 2014 at 1:56 PM, Brian C. Huffman <
> bhuffman@etinternational.com> wrote:
>
>> All,
>>
>> I'm having trouble getting storm set up and submitting a topology. I've
>> tried two different installs of storm and associated dependencies:  One
>> using the older storm (0.8.1), zeromq (2.1.7) and jzmq (frozen for storm).
>>  And then another using the newest of each: storm(0.9.0.1), zeromq (4.0.3),
>> and latest jzmq master.
>>
>> Then I'm using the storm-starter package with the WordCountTopology
>>
>> Storm *appears* to be running fine.  I have one nimbus node and 3
>> supervisors.  And the UI looks good.
>>
>> The topology submits fine:
>> [storm@thor4 bin]$ ./storm jar
>> /home/storm/storm-starter/target/storm-starter-0.0.1-SNAPSHOT.jar
>> storm.starter.WordCountTopology wc5
>> Running: java -client -Dstorm.options=
>> -Dstorm.home=/share/storm/storm-0.9.0.1
>> -Djava.library.path=/share/storm/zeromq-4.0.3/lib:/share/storm/jzmq/lib:/usr/lib
>> -Dstorm.conf.file= -cp
>> /share/storm/storm-0.9.0.1/storm-netty-0.9.0.1.jar:/share/storm/storm-0.9.0.1/storm-console-logging-0.9.0.1.jar:/share/storm/storm-0.9.0.1/storm-core-0.9.0.1.jar:/share/storm/storm-0.9.0.1/lib/disruptor-2.10.1.jar:/share/storm/storm-0.9.0.1/lib/commons-exec-1.1.jar:/share/storm/storm-0.9.0.1/lib/jetty-6.1.26.jar:/share/storm/storm-
>> 0.9.0.1/lib/math.numeric-
>> tower-0.0.1.jar:/share/storm/storm-0.9.0.1/lib/logback-core-1.0.6.jar:/share/storm/storm-0.9.0.1/lib/curator-framework-1.0.1.jar:/share/storm/storm-0.9.0.1/lib/joda-time-2.0.jar:/share/storm/storm-0.9.0.1/lib/ring-jetty-adapter-0.3.11.jar:/share/storm/storm-0.9.0.1/lib/commons-io-1.4.jar:/share/storm/storm-0.9.0.1/lib/logback-classic-1.0.6.jar:/share/storm/storm-0.9.0.1/lib/asm-4.0.jar:/share/storm/storm-0.9.0.1/lib/tools.cli-0.2.2.jar:/share/storm/storm-0.9.0.1/lib/reflectasm-1.07-shaded.jar:/share/storm/storm-
>> 0.9.0.1/lib/core.incubator-0
>> .1.0.jar:/share/storm/storm-0.9.0.1/lib/clj-time-0.4.1.jar:/share/storm/storm-0.9.0.1/lib/minlog-1.2.jar:/share/storm/storm-0.9.0.1/lib/kryo-2.17.jar:/share/storm/storm-0.9.0.1/lib/log4j-over-slf4j-1.6.6.jar:/share/storm/storm-0.9.0.1/lib/netty-3.6.3.Final.jar:/share/storm/storm-0.9.0.1/lib/commons-codec-1.4.jar:/share/storm/storm-0.9.0.1/lib/clout-1.0.1.jar:/share/storm/storm-
>> 0.9.0.1/lib/objenesis-1.2.jar
>> :/share/storm/storm-0.9.0.1/lib/curator-client-1.0.1.jar:/share/storm/storm-0.9.0.1/lib/mockito-all-1.9.5.jar:/share/storm/storm-0.9.0.1/lib/servlet-api-2.5.jar:/share/storm/storm-0.9.0.1/lib/jzmq-2.1.0.jar:/share/storm/storm-
>> 0.9.0.1/lib/json-simple-1.1
>> .jar:/share/storm/storm-0.9.0.1/lib/guava-13.0.jar:/share/storm/storm-0.9.0.1/lib/libthrift7-0.7.0-2.jar:/share/storm/storm-0.9.0.1/lib/slf4j-api-1.6.5.jar:/share/storm/storm-0.9.0.1/lib/hiccup-0.3.6.jar:/share/storm/storm-0.9.0.1/lib/snakeyaml-1.11.jar:/share/storm/storm-0.9.0.1/lib/jetty-util-6.1.26.jar:/share/storm/storm-0.9.0.1/lib/ring-core-1.1.5.jar:/share/storm/storm-0.9.0.1/lib/ring-devel-0.3.11.jar:/share/storm/storm-
>> 0.9.0.1/lib/servlet-api-2.5-
>> 20081211.jar:/share/storm/storm-0.9.0.1/lib/jline-0.9.94.jar:/share/storm/storm-0.9.0.1/lib/commons-fileupload-1.2.1.jar:/share/storm/storm-0.9.0.1/lib/clojure-1.4.0.jar:/share/storm/storm-0.9.0.1/lib/tools.macro-0.1.0.jar:/share/storm/storm-0.9.0.1/lib/clj-stacktrace-0.2.2.jar:/share/storm/storm-0.9.0.1/lib/jgrapht-0.8.3.jar:/share/storm/storm-0.9.0.1/lib/ring-servlet-0.3.11.jar:/share/storm/storm-0.9.0.1/lib/compojure-1.1.3.jar:/share/storm/storm-0.9.0.1/lib/clojure-complete-0.2.3.jar:/share/storm/storm-0.9.0.1/lib/tools.logging-0.2.3.jar:/share/storm/storm-0.9.0.1/lib/commons-lang-2.5.jar:/share/storm/storm-0.9.0.1/lib/tools.nrepl-0.2.3.jar:/share/storm/storm-0.9.0.1/lib/httpclient-4.1.1.jar:/share/storm/storm-
>> 0.9.0.1/lib/httpcore-4.1.jar:/share/storm/storm-0.9.0.1/lib/zookeeper-3.3.3.jar:/share/storm/storm-0.9.0.1/lib/carbonite-1.5.0.jar:/share/storm/storm-0.9.0.1/lib/commons-logging-1.1.1.jar:/share/storm/storm-0.9.0.1/lib/junit-3.8.1.jar:/home/storm/storm-starter/target/storm-starter-0.0.1-SNAPSHOT.jar:/share/storm/storm-0.9.0.1/conf:/share/storm/storm-0.9.0.1/bin
>> -Dstorm.jar=/home/storm/storm-starter/target/storm-starter-0.0.1-SNAPSHOT.jar
>> storm.starter.WordCountTopology wc5
>> 375  [main] INFO  backtype.storm.StormSubmitter - Jar not uploaded to
>> master yet. Submitting jar...
>> 389  [main] INFO  backtype.storm.StormSubmitter - Uploading topology jar
>> /home/storm/storm-starter/target/storm-starter-0.0.1-SNAPSHOT.jar to
>> assigned location:
>> /usr/local/var/storm/nimbus/inbox/stormjar-e10a559a-59e8-4411-8cd3-c3b6e6c16bc9.jar
>> 410  [main] INFO  backtype.storm.StormSubmitter - Successfully uploaded
>> topology jar to assigned location:
>> /usr/local/var/storm/nimbus/inbox/stormjar-e10a559a-59e8-4411-8cd3-c3b6e6c16bc9.jar
>> 411  [main] INFO  backtype.storm.StormSubmitter - Submitting topology wc5
>> in distributed mode with conf {"topology.workers":3,"topology.debug":true}
>> 846  [main] INFO  backtype.storm.StormSubmitter - Finished submitting
>> topology: wc5
>>
>>
>> However, it's segfaulting.  I see many hs_err_pid files with the
>> following content:
>> # Problematic frame:
>> # C  [libzmq.so.3+0x28510]  zmq::signaler_t::signaler_t()+0x30
>>
>> <snip>
>>
>> Stack: [0x00007f945ac1c000,0x00007f945ad1d000], sp=0x00007f945ad1aed8,
>>  free space=1019k
>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
>> code)
>> C  [libzmq.so.3+0x28510]  zmq::signaler_t::signaler_t()+0x30
>>
>> Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
>> j  org.zeromq.ZMQ$Socket.setLongSockopt(IJ)V+0
>> j  org.zeromq.ZMQ$Socket.setLinger(J)V+17
>>
>> I've spent a few hours troubleshooting but just can't find a solution.
>>  Can any one make suggestions on what might be wrong?
>>
>> Thanks,
>> Brian
>>
>>
>>
>>
>
>

Mime
View raw message