Return-Path: X-Original-To: apmail-storm-dev-archive@minotaur.apache.org Delivered-To: apmail-storm-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5A094101A3 for ; Sat, 22 Feb 2014 09:03:44 +0000 (UTC) Received: (qmail 91829 invoked by uid 500); 22 Feb 2014 09:03:43 -0000 Delivered-To: apmail-storm-dev-archive@storm.apache.org Received: (qmail 91799 invoked by uid 500); 22 Feb 2014 09:03:43 -0000 Mailing-List: contact dev-help@storm.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@storm.incubator.apache.org Delivered-To: mailing list dev@storm.incubator.apache.org Received: (qmail 91788 invoked by uid 99); 22 Feb 2014 09:03:42 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 22 Feb 2014 09:03:42 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of ramtrg@gmail.com designates 209.85.213.54 as permitted sender) Received: from [209.85.213.54] (HELO mail-yh0-f54.google.com) (209.85.213.54) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 22 Feb 2014 09:03:34 +0000 Received: by mail-yh0-f54.google.com with SMTP id z6so3434092yhz.13 for ; Sat, 22 Feb 2014 01:03:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=vto/tqoUsTq8XxyCMquerU0MKNcURfREVYeA6IQuwoY=; b=KC5saA95OTCousNl3kAleZ2bongg3GFKVnLmqVcupwFv9Ez9fCWQc1SJ2N8cf6Z4b0 9WMnlLHkywieQKLYaSMe0bbsW2UMncgEvsRrtX+5jOGnzw8zTnwX9QPqNMLcyxOjYqRU ThKxsdRBiDB83NAl+XQHf7cbkYLUf6JfBr7Id5g+BT4wS1gI2DoMZDptKFL6X8uIymal G36m2l7KjqYM0EGBlzEUNe2zFoTXnWBNB224UPyoxzh+X8pGLbPGqYRCb5QINdVxQdHV 2EZBGU+wnSfKptvqDht97Jy6tMB80rfLGXLJZ235mOaZwHXTFj9o18/vk20vvsNOSF+y Wcug== MIME-Version: 1.0 X-Received: by 10.236.128.170 with SMTP id f30mr16827597yhi.89.1393059793571; Sat, 22 Feb 2014 01:03:13 -0800 (PST) Received: by 10.170.166.4 with HTTP; Sat, 22 Feb 2014 01:03:13 -0800 (PST) In-Reply-To: References: Date: Sat, 22 Feb 2014 14:33:13 +0530 Message-ID: Subject: =?UTF-8?B?UmU6IOWbnuWkje+8miBVbmRlcnN0YW5kIHRoZSBwcm9jZXNzaW5nIGZsb3cgb2Ygc3Rvcg==?= =?UTF-8?B?bQ==?= From: Ramprasad Ranganathan To: dev@storm.incubator.apache.org Content-Type: multipart/related; boundary=20cf3010e99125a9c704f2fb02b6 X-Virus-Checked: Checked by ClamAV on apache.org --20cf3010e99125a9c704f2fb02b6 Content-Type: multipart/alternative; boundary=20cf3010e99125a9c204f2fb02b5 --20cf3010e99125a9c204f2fb02b5 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi,Kang Xiao Thank you. From your answer I understood, After submitting topology to Nimbus node it does only distributing code and other management works.Then the spouts and bolts are distributed evenly to worker nodes. I have some other questions, - If I have only 3 machines,how 1 spout and 3 bolts will distributed to nodes? - Storm is for real time processing, If all data in the queue is processed, spouts and bolts will go to "Waiting" state or what will happ= en? - In following Image, If 3 bolts are distributed to each 3 nodes and processing parallel,how Bolt 2 will process without the result of Bolt 1= ? is it go to waiting state to receive bolt 1 result tuple? - Bolts process tuples synchronously or asynchronously? If 1 million tuples emitted by spout, each tuple process one by one? - If 2 tuples emitted by spout,1st bolt on node 1 process and send to 2nd bolt on Bolt. Same way tuple 2 process on 1 bolt on 1 node and send = to 2 bolt on 2 node. Is my understanding is correct or not? [image: Inline image 1] On Sat, Feb 22, 2014 at 1:21 PM, Kang Xiao wrote: > Hi R.RAMPRASAD > > I try to answer your questions and hope that will help you. > > > 1. I have a topology(1 spout and 3 bolts) and 3 machines(1 nimbus and 2 > > worker nodes).Whether I want to run my topology on all 3 nodes or run o= n > > nimbus is enough? Is it nimbus will take care of distributing the code = to > > other nodes? > > > > Just submit your topology to nimbus and nimbus will take care of > distributing the code to worker nodes. > > > > > 2. My spout will run on nimbus or any of the worker nodes? > > Spouts and Bolts will all run on worker nodes. > > > > > 3. The 3 bolts are run on 3 separate nodes or run on same node? is it > will > > take care by nimbus? > > Spouts and Bolts will run in workers. Workers will distribute evenly on > worker nodes. If you specify enough workers eg. 4 workers in your case, > they will run on separate nodes. > > > > > 4. How do we track processing of bolt in nodes? > Storm UI keep track of tuple emit/ack/fail count of spout/bolt. For more > detail information you can turn on topology debug mode and worker will lo= g > every tuple. > > > > > 5. Is it any documentation available to understand the complete flow of > > processing a message in Storm? > > > Your can refer to the storm documentation site: > http://storm.incubator.apache.org/documentation/Implementation-docs.html > > > -- > Best Regards! > > =E8=82=96=E5=BA=B7(Kang Xiao,) > Distributed Software Engineer > > =E5=9C=A8 2014=E5=B9=B42=E6=9C=8822=E6=97=A5 =E6=98=9F=E6=9C=9F=E5=85=AD= =EF=BC=8C14:51=EF=BC=8CRamprasad Ranganathan =E5=86=99=E9=81=93=EF=BC=9A > > Hi, > > > > I am beginner in Storm.I am trying to understand the processing flow of > > storm by following questions. > > > > 1. I have a topology(1 spout and 3 bolts) and 3 machines(1 nimbus and 2 > > worker nodes).Whether I want to run my topology on all 3 nodes or run o= n > > nimbus is enough? Is it nimbus will take care of distributing the code = to > > other nodes? > > > > 2. My spout will run on nimbus or any of the worker nodes? > > > > 3. The 3 bolts are run on 3 separate nodes or run on same node? is it > will > > take care by nimbus? > > > > 4. How do we track processing of bolt in nodes? > > > > 5. Is it any documentation available to understand the complete flow of > > processing a message in Storm? > > > > -- > > by > > > > *R.RAMPRASAD* > > --=20 by *R.RAMPRASAD* --20cf3010e99125a9c204f2fb02b5 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,Kang Xiao

=C2=A0 =C2=A0 =C2=A0 Thank you. From your answer I unders= tood, After submitting topology to Nimbus node it does only distributing co= de and other management works.Then the spouts and bolts are distributed eve= nly to worker nodes. I have some other questions,
  • If I have only 3 machines,how 1 spout and 3 bolts will distrib= uted to nodes?
  • Storm is for real time processing, If all data in th= e queue is processed, spouts and bolts will go to "Waiting" state= or what will happen?
  • In following Image, If 3 bolts are distributed to each 3 nodes and proc= essing parallel,how Bolt 2 will process without the result of Bolt 1? is it= go to waiting state to receive bolt 1 result tuple?
  • Bolts process = tuples synchronously or asynchronously? If 1 million tuples emitted by spou= t, each tuple process one by one?
  • If 2 tuples emitted by spout,1st bolt on node 1 process and send to 2nd= bolt on Bolt. Same way tuple 2 process on 1 bolt on 1 node and send to 2 b= olt on 2 node. Is my understanding is correct or not?
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0=C2=A03D"Inline
<= span style=3D"font-size:19px;font-family:Arial;color:rgb(0,0,0);background-= color:transparent;vertical-align:baseline;white-space:pre-wrap">
=


On Sat, Feb 22, 2014 at 1:21 PM, Kang Xiao <kxiao.tiger@gmail.co= m> wrote:
Hi R.RAMPRASAD

I try to answer your questions and hope that will help you.

> 1. I have a topology(1 spout and 3 bolts) and 3 machines(1 nimbus and = 2
> worker nodes).Whether I want to run my topology on all 3 nodes or run = on
> nimbus is enough? Is it nimbus will take care of distributing the code= to
> other nodes?
>

Just submit your topology to nimbus and nimbus will take care of dist= ributing the code to worker nodes.

>
> 2. My spout will run on nimbus or any of the worker nodes?

Spouts and Bolts will all run on worker nodes.

>
> 3. The 3 bolts are run on 3 separate nodes or run on same node? is it = will
> take care by nimbus?

Spouts and Bolts will run in workers. Workers will distribute evenly = on worker nodes. If you specify enough workers eg. 4 workers in your case, = they will run on separate nodes.

>
> 4. How do we track processing of bolt in nodes?
Storm UI keep track of tuple emit/ack/fail count of spout/bolt. For m= ore detail information you can turn on topology debug mode and worker will = log every tuple.

>
> 5. Is it any documentation available to understand the complete flow o= f
> processing a message in Storm?


Your can refer to the storm documentation site: http://storm.incubator.apache.org/documentation/Implementation-docs.= html


--
Best Regards!

=E8=82=96=E5=BA=B7(Kang Xiao,<k= xiao.tiger@gmail.com (mailto:k= xiao.tiger@gmail.com)>)
Distributed Software Engineer

=E5=9C=A8 2014=E5=B9=B42=E6=9C=8822=E6=97=A5 =E6=98=9F=E6=9C=9F=E5=85=AD=EF= =BC=8C14:51=EF=BC=8CRamprasad Ranganathan =E5=86=99=E9=81=93=EF=BC=9A
> Hi,
>
> I am beginner in Storm.I am trying to understand the processing flow o= f
> storm by following questions.
>
> 1. I have a topology(1 spout and 3 bolts) and 3 machines(1 nimbus and = 2
> worker nodes).Whether I want to run my topology on all 3 nodes or run = on
> nimbus is enough? Is it nimbus will take care of distributing the code= to
> other nodes?
>
> 2. My spout will run on nimbus or any of the worker nodes?
>
> 3. The 3 bolts are run on 3 separate nodes or run on same node? is it = will
> take care by nimbus?
>
> 4. How do we track processing of bolt in nodes?
>
> 5. Is it any documentation available to understand the complete flow o= f
> processing a message in Storm?
>
> --
> by
>
> *R.RAMPRASAD*




--
by
R.RAMPRASAD
--20cf3010e99125a9c204f2fb02b5-- --20cf3010e99125a9c704f2fb02b6--