Return-Path: X-Original-To: apmail-storm-user-archive@minotaur.apache.org Delivered-To: apmail-storm-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CD36310FCA for ; Sun, 6 Apr 2014 13:49:47 +0000 (UTC) Received: (qmail 86635 invoked by uid 500); 6 Apr 2014 13:49:47 -0000 Delivered-To: apmail-storm-user-archive@storm.apache.org Received: (qmail 86016 invoked by uid 500); 6 Apr 2014 13:49:46 -0000 Mailing-List: contact user-help@storm.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@storm.incubator.apache.org Delivered-To: mailing list user@storm.incubator.apache.org Received: (qmail 86008 invoked by uid 99); 6 Apr 2014 13:49:44 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 06 Apr 2014 13:49:44 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of svend.vanderveken@gmail.com designates 209.85.192.43 as permitted sender) Received: from [209.85.192.43] (HELO mail-qg0-f43.google.com) (209.85.192.43) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 06 Apr 2014 13:49:39 +0000 Received: by mail-qg0-f43.google.com with SMTP id f51so5160186qge.2 for ; Sun, 06 Apr 2014 06:49:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=ENkhUAT/Drp2HPXBkH7HtCEplxfGU69w0WkMFk7bne0=; b=LMKvqqq4CT5gL+HcS+vKg5BF22+Nz4gdvXJHacxmHtzT2VWaVheTfFkFPH6US53bNT X4LY9daWOq0WTTlvxXuoZBNSGnliMWQb5YKTZDFFpjcXRDQCJumgUW1pSQiKUXZXKmpf YkBrKFEbEOvVvw6K7NeNZb7VnKeEihwGFPVPzYPdvQV5CsYYGAwacmP6cWG0O7Q+MzUt duERgiC7FggNJXK85wZU9d5+IBPSAk0Fm0z62jE3gvFBVQLP+IhfcTuH21MusOkuiMU5 1TlFDy0EGXDfMkys2S7dyVm/lVH8jXOWvmCUcQ363/cMzjEZQXmNpwM2HoL6BmezDBnG FXIw== MIME-Version: 1.0 X-Received: by 10.224.45.67 with SMTP id d3mr27187295qaf.33.1396792157086; Sun, 06 Apr 2014 06:49:17 -0700 (PDT) Received: by 10.140.44.74 with HTTP; Sun, 6 Apr 2014 06:49:17 -0700 (PDT) In-Reply-To: References: Date: Sun, 6 Apr 2014 09:49:17 -0400 Message-ID: Subject: Re: Replace Netty with ZeroMQ in Storm 0.9.1 From: Svend Vanderveken To: user@storm.incubator.apache.org Content-Type: multipart/alternative; boundary=001a11c2926459292404f66004a2 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c2926459292404f66004a2 Content-Type: text/plain; charset=ISO-8859-1 Daria, Here are the old installation instructions of Storm (pre Netty), that describe how to install a previous version, I think this was written with 0.8.2 in mind, together with its 0mq and jzmq dependencies, hopefully this should help you get 0mz working with the latest version of Storm : https://github.com/nathanmarz/storm/wiki/Setting-up-a-Storm-cluster Cheers, S On Fri, Apr 4, 2014 at 9:34 AM, Daria Mayorova wrote: > Thanks for the info, Jon. > > You're right, I've found this repository - > https://github.com/ptgoetz/storm-0mq. I guess this is what you meant. > I'll give it a try, let's see if I can make it work. > > > On Fri, Apr 4, 2014 at 2:58 PM, Jon Logan wrote: > >> You need to include the zeromq Storm binding in your storm lib folder >> probably...it's no longer packaged together due to GPL licensing issues. >> I've never done this, but I think it was broken off into a separate github >> repository. >> >> >> On Fri, Apr 4, 2014 at 7:01 AM, Daria Mayorova wrote: >> >>> Thank you Susheel, >>> >>> We've tried to change this parameter, but the topology isn't >>> initializing. The following exception is returned: >>> >>> 2014-04-04 10:35:28 b.s.d.worker [ERROR] Error on initialization of >>> server mk-worker >>> java.lang.RuntimeException: Fail to construct messaging plugin from >>> plugin backtype.storm.messaging.zmq >>> at >>> backtype.storm.messaging.TransportFactory.makeContext(TransportFactory.java:53) >>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] >>> at >>> backtype.storm.daemon.worker$worker_data$fn__4263.invoke(worker.clj:185) >>> ~[na:0.9.1-incubating] >>> at backtype.storm.util$assoc_apply_self.invoke(util.clj:792) >>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] >>> at >>> backtype.storm.daemon.worker$worker_data.invoke(worker.clj:181) >>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] >>> at >>> backtype.storm.daemon.worker$eval4413$exec_fn__1102__auto____4414.invoke(worker.clj:353) >>> ~[na:0.9.1-incubating] >>> at clojure.lang.AFn.applyToHelper(AFn.java:185) >>> [clojure-1.4.0.jar:na] >>> at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na] >>> at clojure.core$apply.invoke(core.clj:601) >>> ~[clojure-1.4.0.jar:na] >>> at >>> backtype.storm.daemon.worker$eval4413$mk_worker__4469.doInvoke(worker.clj:344) >>> [na:0.9.1-incubating] >>> at clojure.lang.RestFn.invoke(RestFn.java:512) >>> [clojure-1.4.0.jar:na] >>> at backtype.storm.daemon.worker$_main.invoke(worker.clj:454) >>> [na:0.9.1-incubating] >>> at clojure.lang.AFn.applyToHelper(AFn.java:172) >>> [clojure-1.4.0.jar:na] >>> at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na] >>> at backtype.storm.daemon.worker.main(Unknown Source) >>> [storm-core-0.9.1-incubating.jar:0.9.1-incubating] >>> Caused by: java.lang.ClassNotFoundException: backtype.storm.messaging.zmq >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:217) >>> ~[na:1.6.0_27] >>> at java.security.AccessController.doPrivileged(Native Method) >>> ~[na:1.6.0_27] >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:205) >>> ~[na:1.6.0_27] >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:321) >>> ~[na:1.6.0_27] >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) >>> ~[na:1.6.0_27] >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:266) >>> ~[na:1.6.0_27] >>> at java.lang.Class.forName0(Native Method) ~[na:1.6.0_27] >>> at java.lang.Class.forName(Class.java:188) ~[na:1.6.0_27] >>> at >>> backtype.storm.messaging.TransportFactory.makeContext(TransportFactory.java:38) >>> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating] >>> ... 13 common frames omitted >>> 2014-04-04 10:35:28 b.s.util [INFO] Halting process: ("Error on >>> initialization") >>> >>> 0mq and jzmq are installed correctly on the machine, as before Storm >>> 0.8.1 was running on it using 0mq. >>> >>> Any other thing that should be done to make this work? >>> >>> >>> On Fri, Apr 4, 2014 at 10:52 AM, Susheel Kumar Gadalay < >>> skgadalay@gmail.com> wrote: >>> >>>> Set this parameter in storm.yaml >>>> >>>> storm.messaging.transport: "backtype.storm.messaging.zmq" >>>> >>>> On 4/4/14, Daria Mayorova wrote: >>>> > Hi everyone, >>>> > >>>> > I am wondering whether it is possible to go back to using 0mq as >>>> transport >>>> > instead of Netty? I am pretty sure I saw some messages about >>>> switching back >>>> > to 0mq in Storm 0.9.1 at the old storm-user google group. >>>> > There is an issue related to this on Jira - >>>> > https://issues.apache.org/jira/browse/STORM-215 - but it seems to be >>>> > abandoned. >>>> > >>>> > Can anybody give me an idea of how this can be achieved? >>>> > >>>> > Thanks. >>>> > >>>> > Best regards, >>>> > Daria >>>> > >>>> >>> >>> >> > --001a11c2926459292404f66004a2 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Daria, 

Here are the old installat= ion instructions of Storm (pre Netty), that describe how to install a previ= ous version, I think this was written with 0.8.2 in mind, together with its= 0mq and jzmq dependencies, hopefully this should help you get 0mz working = with the latest version of Storm : 


Cheers, 
=
S






On F= ri, Apr 4, 2014 at 9:34 AM, Daria Mayorova <d.mayorova@gmail.com>= ; wrote:
Thanks for the info, Jon.
You're right, I've found this repository –&= nbsp;htt= ps://github.com/ptgoetz/storm-0mq. I guess this is what you meant. I= 9;ll give it a try, let's see if I can make it work.

On Fri, Apr 4, 2014 at 2:58 PM, Jon Logan = <jmlogan@buffalo.edu> wrote:
You need to include the zer= omq Storm binding in your storm lib folder probably...it's no longer pa= ckaged together due to GPL licensing issues. I've never done this, but = I think it was broken off into a separate github repository.


On Fri, Apr 4= , 2014 at 7:01 AM, Daria Mayorova <d.mayorova@gmail.com> = wrote:
Thank you Susheel,

=
We've tried to change this parameter, but the topology isn&#= 39;t initializing. The following exception is returned:

2014-04-04 10:35:28 b.s.d.worker [ERROR] Error on = initialization of server mk-worker
java.lang.RuntimeException: Fail to construct messaging plugin from pl= ugin backtype.storm.messaging.zmq
        at = backtype.storm.messaging.TransportFactory.makeContext(TransportFactory.java= :53) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
        at backtype.storm.daemon.worker$worker_dat= a$fn__4263.invoke(worker.clj:185) ~[na:0.9.1-incubating]
  &= nbsp;     at backtype.storm.util$assoc_apply_self.invoke(util.clj= :792) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
        at backtype.storm.daemon.worker$worker_dat= a.invoke(worker.clj:181) ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating= ]
        at backtype.storm.daemon.worker$eva= l4413$exec_fn__1102__auto____4414.invoke(worker.clj:353) ~[na:0.9.1-incubat= ing]
        at clojure.lang.AFn.applyToHelper(AFn.java= :185) [clojure-1.4.0.jar:na]
        at cloju= re.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na]
  &= nbsp;     at clojure.core$apply.invoke(core.clj:601) ~[clojure-1.= 4.0.jar:na]
        at backtype.storm.daemon.worker$eval4413$m= k_worker__4469.doInvoke(worker.clj:344) [na:0.9.1-incubating]
&nb= sp;       at clojure.lang.RestFn.invoke(RestFn.java:512) [cl= ojure-1.4.0.jar:na]
        at backtype.storm= .daemon.worker$_main.invoke(worker.clj:454) [na:0.9.1-incubating]
        at clojure.lang.AFn.applyToHelper(AFn.java= :172) [clojure-1.4.0.jar:na]
        at cloju= re.lang.AFn.applyTo(AFn.java:151) [clojure-1.4.0.jar:na]
  &= nbsp;     at backtype.storm.daemon.worker.main(Unknown Source) [s= torm-core-0.9.1-incubating.jar:0.9.1-incubating]
Caused by: java.lang.ClassNotFoundException: backtype.storm.messaging.= zmq
        at java.net.URLClassLoader$1.run(= URLClassLoader.java:217) ~[na:1.6.0_27]
      &nbs= p; at java.security.AccessController.doPrivileged(Native Method) ~[na:1.6.0= _27]
        at java.net.URLClassLoader.findClass(URLCl= assLoader.java:205) ~[na:1.6.0_27]
        at= java.lang.ClassLoader.loadClass(ClassLoader.java:321) ~[na:1.6.0_27]
=
        at sun.misc.Launcher$AppClassLoader.loadCl= ass(Launcher.java:294) ~[na:1.6.0_27]
        at java.lang.ClassLoader.loadClass(ClassLo= ader.java:266) ~[na:1.6.0_27]
        at java= .lang.Class.forName0(Native Method) ~[na:1.6.0_27]
    =     at java.lang.Class.forName(Class.java:188) ~[na:1.6.0_27]
        at backtype.storm.messaging.TransportFacto= ry.makeContext(TransportFactory.java:38) ~[storm-core-0.9.1-incubating.jar:= 0.9.1-incubating]
        ... 13 common frame= s omitted
2014-04-04 10:35:28 b.s.util [INFO] Halting process: (&= quot;Error on initialization")

0mq and jzmq are installed correctly on the machi= ne, as before Storm 0.8.1 was running on it using 0mq.

=
Any other thing that should be done to make this work?


On Fri, Apr 4= , 2014 at 10:52 AM, Susheel Kumar Gadalay <skgadalay@gmail.com> wrote:
Set this parameter in storm.yaml

storm.messaging.transport: "backtype.storm.messaging.zmq"

On 4/4/14, Daria Mayorova <d.mayorova@gmail.com> wrote:
> Hi everyone,
>
> I am wondering whether it is possible to go back to using 0mq as trans= port
> instead of Netty? I am pretty sure I saw some messages about switching= back
> to 0mq in Storm 0.9.1 at the old storm-user google group.
> There is an issue related to this on Jira -
> https://issues.apache.org/jira/browse/STORM-215 - but= it seems to be
> abandoned.
>
> Can anybody give me an idea of how this can be achieved?
>
> Thanks.
>
> Best regards,
> Daria
>




--001a11c2926459292404f66004a2--