Return-Path: X-Original-To: apmail-storm-user-archive@minotaur.apache.org Delivered-To: apmail-storm-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 45D6818CA2 for ; Sat, 13 Feb 2016 05:43:07 +0000 (UTC) Received: (qmail 91994 invoked by uid 500); 13 Feb 2016 05:43:06 -0000 Delivered-To: apmail-storm-user-archive@storm.apache.org Received: (qmail 91944 invoked by uid 500); 13 Feb 2016 05:43:06 -0000 Mailing-List: contact user-help@storm.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@storm.apache.org Delivered-To: mailing list user@storm.apache.org Received: (qmail 91933 invoked by uid 99); 13 Feb 2016 05:43:05 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 13 Feb 2016 05:43:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 869D8C0BE4 for ; Sat, 13 Feb 2016 05:43:05 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.179 X-Spam-Level: * X-Spam-Status: No, score=1.179 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx2-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 3U_21mWN7QlJ for ; Sat, 13 Feb 2016 05:43:03 +0000 (UTC) Received: from mail-vk0-f51.google.com (mail-vk0-f51.google.com [209.85.213.51]) by mx2-lw-us.apache.org (ASF Mail Server at mx2-lw-us.apache.org) with ESMTPS id F37C85FE0C for ; Sat, 13 Feb 2016 05:43:02 +0000 (UTC) Received: by mail-vk0-f51.google.com with SMTP id e185so75441727vkb.1 for ; Fri, 12 Feb 2016 21:43:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=YNksPNYNxpXvc2P1tod5P8Ev/Nf/dX2+TEha8aRngLs=; b=lWoW+Cgr4pGqQILVd/VcWwD7iXgDSRQ+QMaAvHFFXpdp34ms+HziYWbb+91aPyP3Dy FRLVb6sE3rW8wt7kGTMlzOqHUisxXvMY5DGL+te6jgPH3iMiA7itnKWOlYIZoE6Bdk6A DofMqPFaage3fYE5DHlWBy26BLg28VX7DuoIfoBY6UNYckje1N+FGEYSXwW487rDS8ph s/4Wrml4RxtRGdliM8Sq1F5ZYuNJSuiQw9RWyPvF560G+zZVM3KrhJEe+qQKAqb/W7dd lFSwZIKYYkArNoEK5aStyY97JmQRtn17buSeaVWuiiqN4EekQiY4wl4gpXPfglh11cIf RZ6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=YNksPNYNxpXvc2P1tod5P8Ev/Nf/dX2+TEha8aRngLs=; b=Smxcaf/xA7SR0g0f9vju+e8oem/N8sftZIohkRwgiFWubwNRpUYoiPKvbRrTIq4bXl 6x/wPPkY0k/mvX+kuo9oJELZtOPBfgImQ1rvEWolA/exyHC6cEkzsYXa7Ys3iC264Zwp poT5nyU6Ll7qAe62BG0cARHoRwYDkT7NSDkMT/Kgw5FRWFflCMubOiImWCe2kk0iiMl/ 9e5pBYjK6ULPU8m/AhTYOv7Z1cX1OHVs2A8R1opL3EI2M0R4q7bWLycTWe3fU7/8XTS8 m28jdnYEAFm8vJLiRMi4AWfRINhU3lhw7dI0FHTHhGvrD6mXo107MqjX3FOSkU/yRbY3 eI8Q== X-Gm-Message-State: AG10YOTQnXn3CyhwJCOtKA2ZswUKnTpof5evRL0JJpZIs7cVDYisLUFQax/RHTZ80TVrT3nGZDy1ILQ6y/giQw== MIME-Version: 1.0 X-Received: by 10.31.139.132 with SMTP id n126mr4575067vkd.78.1455342182201; Fri, 12 Feb 2016 21:43:02 -0800 (PST) Received: by 10.31.227.5 with HTTP; Fri, 12 Feb 2016 21:43:02 -0800 (PST) In-Reply-To: <56BCE552.3080601@apache.org> References: <56BCE552.3080601@apache.org> Date: Sat, 13 Feb 2016 00:43:02 -0500 Message-ID: Subject: Re: Need Help regarding topology with numWorker>1 From: Rudraneel chakraborty To: "user@storm.apache.org" Cc: mjsax@apache.org Content-Type: multipart/alternative; boundary=001a1144169ecc002d052ba041f3 --001a1144169ecc002d052ba041f3 Content-Type: text/plain; charset=UTF-8 Thanks for the reply. So after some thinking , i figured if i shorten the size of the worker incoming queue , i can create the scenario where using more than one worker might result in better performance. Any thoughts on how to do that? The property topology, receiver.buffer.size should be the the key as mentioned here http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers/ But this jira issue says this property is no longer used https://issues.apache.org/jira/plugins/servlet/mobile#issue/STORM-596 Any thoughts on how to change the incoming queue size of the workers? On Thursday, 11 February 2016, Matthias J. Sax wrote: > I am not sure what you mean: > > - number of worker slots per supervisor > or > - topology parameter "number of workers" > > Can you clarify? > > -Matthias > > On 02/11/2016 05:14 AM, anshu shukla wrote: > > Not like that.. But i have used workers equal to number of cores. Each > > vm with 8 corea. > > > > On 11 Feb 2016 9:07 am, "Rudraneel chakraborty" > > > > >> wrote: > > > > more than one worker on same node ? Did u use custom schedule r? > > because by default, the workers would be spread throughout the > cluster > > > > -- Rudraneel Chakraborty Carleton University Real Time and Distributed Systems Reserach --001a1144169ecc002d052ba041f3 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks for the reply. So after some thinking , i figured if i shorten the s= ize of the worker incoming queue , i can create the scenario where using mo= re than one worker might result in better performance. Any thoughts on how = to do that?=C2=A0

The property topology, receiver.buffer= .size should be the the key as mentioned here

But this jira issue say= s this property is no longer used=C2=A0https://issues.apache.org/jir= a/plugins/servlet/mobile#issue/STORM-596
Any thoughts on how = to change the incoming queue size of the workers?=C2=A0
On Thursday, 11 February 2016, Matthias J. Sax <mjsax@apache.org> wrote:
I am not sure what you mean:

=C2=A0 - number of worker slots per supervisor
or
=C2=A0 - topology parameter "number of workers"

Can you clarify?

-Matthias

On 02/11/2016 05:14 AM, anshu shukla wrote:
> Not=C2=A0 like that..=C2=A0 But i have used workers equal to number of= cores. Each
> vm with 8 corea.
>
> On 11 Feb 2016 9:07 am, "Rudraneel chakraborty"
> <rudraneel.chakraborty@gmail.com=
> <mailto:rudraneel.chakraborty@gmail.= com>> wrote:
>
>=C2=A0 =C2=A0 =C2=A0more than one worker on same node ? Did u use custo= m schedule r?
>=C2=A0 =C2=A0 =C2=A0because by default, the workers would be spread thr= oughout the cluster
>



--
Rudraneel Chakrabor= ty
Carleton University Real Time and Distributed Systems Rese= rach

--001a1144169ecc002d052ba041f3--