Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3D2781159C for ; Tue, 2 Sep 2014 22:16:47 +0000 (UTC) Received: (qmail 49346 invoked by uid 500); 2 Sep 2014 22:16:37 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 49223 invoked by uid 500); 2 Sep 2014 22:16:37 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 49213 invoked by uid 99); 2 Sep 2014 22:16:36 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Sep 2014 22:16:36 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of bobyangbo@gmail.com designates 209.85.212.171 as permitted sender) Received: from [209.85.212.171] (HELO mail-wi0-f171.google.com) (209.85.212.171) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Sep 2014 22:16:10 +0000 Received: by mail-wi0-f171.google.com with SMTP id hi2so15161514wib.4 for ; Tue, 02 Sep 2014 15:16:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=pfQVApUuno8SOUCJAcPW10Us4gHkVJOCivofzE7zksE=; b=ka5F+2EcxESE4EQFpIOr4qwO40SQ6QZKVeHKra0mTrf9CnMKxsjqfumPDXv/DALVIU zAfQpdTEIHM97mMnHR3HiLiapymk5ZoqQrtbdeo+pzxPCH+lRRVuAEoYQsOzNLh3mfnx 6gVjavNEwuZUUEk8uXs5IHCX1RVcOUG6v4GigplJPGkuJcpTclL7bPA2ejU1sIoKTLz/ IoP1PWV1YBJY5rAHHnQaksAFzjL84KOspvW8J/7KTxz0Seme+2KOgJymncTwBbtY9Rjl MhpG7DgdcsmkG02FeVhlDYQg4MSOkHdcsxoD2KaAlzlFLV+gbbRxTTQDsFNJFg12FLUD tPTg== MIME-Version: 1.0 X-Received: by 10.194.173.234 with SMTP id bn10mr18873780wjc.81.1409696169996; Tue, 02 Sep 2014 15:16:09 -0700 (PDT) Received: by 10.216.66.67 with HTTP; Tue, 2 Sep 2014 15:16:09 -0700 (PDT) In-Reply-To: References: Date: Tue, 2 Sep 2014 15:16:09 -0700 Message-ID: Subject: Re: Any issue with large concurrency due to single active instance of YARN Resource Manager? From: bo yang To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=089e0122f3727439cb05021c777a X-Virus-Checked: Checked by ClamAV on apache.org --089e0122f3727439cb05021c777a Content-Type: text/plain; charset=UTF-8 Hi Zhijie, Looks great that YARN is mature enough to support so large cluster. It gives me more confidence to use :) Thanks a lot! Best, Bo On Tue, Sep 2, 2014 at 2:41 PM, Zhijie Shen wrote: > Hi Bo, > > I don't have the exact number about the max concurrent job. FYI, RM is > multip-threaded, but the threads are working for different purpose. For > example, scheduler and rmstatestore has their separate thread, and RPC > calls are on individual threads as well. It's complicated to evaluate the > upper bound of concurrent apps, but I've heard of the YARN cluster > deployment on a cluster of thousands of nodes. > > Thanks, > Zhijie > > > On Tue, Sep 2, 2014 at 10:42 AM, bo yang wrote: > >> Hi Zhijie, >> >> That is great to know. Thanks! >> >> So there seems no be much limit to support large concurrency. To move >> this question further, what might be the max number of concurrent jobs >> which one Resource Manager could support? Is there any numbers from your >> experience? >> >> Thanks, >> Bo >> >> >> >> >> >> >> On Tue, Sep 2, 2014 at 12:10 AM, Zhijie Shen >> wrote: >> >>> Hi Bo, >>> >>> RM doesn't create an individual thread for each running app. The app >>> life cycle management is event driven. There's a dispatcher, which runs on >>> one thread to handle the events for all apps. >>> >>> Zhijie >>> >>> >>> On Mon, Sep 1, 2014 at 11:39 PM, bo yang wrote: >>> >>>> Hi Guys, >>>> >>>> I am thinking how many concurrent jobs a single Resource Manager might >>>> be able to manage? Following is my understanding, please correct me if I am >>>> wrong. >>>> >>>> Let's say if we have 1000 concurrent jobs running. Resource Manager >>>> will have 1000 records in memory to manage these jobs. And it will also >>>> have 1000 threads, where each thread is waiting for one job to finish. >>>> >>>> The memory part will probably be ok. For the 1000 threads, will there >>>> be any potential problem? >>>> >>>> Thanks, >>>> Bo >>>> >>> >>> >>> >>> -- >>> Zhijie Shen >>> Hortonworks Inc. >>> http://hortonworks.com/ >>> >>> CONFIDENTIALITY NOTICE >>> NOTICE: This message is intended for the use of the individual or entity >>> to which it is addressed and may contain information that is confidential, >>> privileged and exempt from disclosure under applicable law. If the reader >>> of this message is not the intended recipient, you are hereby notified that >>> any printing, copying, dissemination, distribution, disclosure or >>> forwarding of this communication is strictly prohibited. If you have >>> received this communication in error, please contact the sender immediately >>> and delete it from your system. Thank You. >> >> >> > > > -- > Zhijie Shen > Hortonworks Inc. > http://hortonworks.com/ > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. > --089e0122f3727439cb05021c777a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Zhijie,

Looks great that = YARN is mature enough to support so large cluster. It gives me more confide= nce to use :)

Thanks a lot!

Best,
Bo


On Tue, Sep 2, 2014 at 2:41 PM, Zhijie Shen= <zshen@hortonworks.com> wrote:
Hi Bo,

I= don't have the exact number about the max concurrent job. FYI, RM is m= ultip-threaded, but the threads are working for different purpose. For exam= ple, scheduler and rmstatestore has their separate thread, and RPC calls ar= e on individual threads as well. It's complicated to evaluate the upper= bound of concurrent apps, but I've heard of the YARN cluster deploymen= t on a cluster of thousands of nodes.

Thanks,
Zhijie


On Tue, Sep 2, 2014 at 10:42 AM, bo yang <bobyangbo@gmail.com> wrote:
Hi Zhijie,

= That is great to know. Thanks!

So there seems no be much limit to support large concur= rency. To move this question further, what might be the max number of concu= rrent jobs which one Resource Manager could support? Is there any numbers f= rom your experience?

Thanks,
Bo






Hi Bo,

RM doesn't create an individual thread for each running app. The a= pp life cycle management is event driven. There's a dispatcher, which r= uns on one thread to handle the events for all apps.

Zhijie
=

On Mon, Sep 1, 2014 at 11:39 PM, bo yang= <bobyangbo@gmail.com> wrote:
Hi Guys,

I am thinking= how many concurrent jobs a single=C2=A0Resource Manager might be able to m= anage? Following is my understanding, please correct me if I am wrong.=C2= =A0

Let's say if we have 1000 concurrent jobs running. Resource Manager wil= l have 1000 records in memory to manage these jobs. And it will also have 1= 000 threads, where each thread is waiting for one job to finish.

The memory part will probably be ok. For the 1000 threads, w= ill there be any potential problem?

Thanks,
<= div>Bo



<= font color=3D"#888888">--
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/=

CONFIDENTIALITY NOTICE
NOTICE: This message is = intended for the use of the individual or entity to which it is addressed a= nd may contain information that is confidential, privileged and exempt from= disclosure under applicable law. If the reader of this message is not the = intended recipient, you are hereby notified that any printing, copying, dis= semination, distribution, disclosure or forwarding of this communication is= strictly prohibited. If you have received this communication in error, ple= ase contact the sender immediately and delete it from your system. Thank Yo= u.




--
Zhijie Shen<= div>Hortonworks Inc.
CONFIDENTIALITY NOTICE
NOTICE: This message is = intended for the use of the individual or entity to which it is addressed a= nd may contain information that is confidential, privileged and exempt from= disclosure under applicable law. If the reader of this message is not the = intended recipient, you are hereby notified that any printing, copying, dis= semination, distribution, disclosure or forwarding of this communication is= strictly prohibited. If you have received this communication in error, ple= ase contact the sender immediately and delete it from your system. Thank Yo= u.
--089e0122f3727439cb05021c777a--