Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E1B61F34A for ; Tue, 7 May 2013 21:11:14 +0000 (UTC) Received: (qmail 9020 invoked by uid 500); 7 May 2013 21:11:14 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 8982 invoked by uid 500); 7 May 2013 21:11:14 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 8971 invoked by uid 99); 7 May 2013 21:11:14 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 May 2013 21:11:14 +0000 X-ASF-Spam-Status: No, hits=-10.8 required=5.0 tests=ENV_AND_HDR_SPF_MATCH,HTML_MESSAGE,RCVD_IN_DNSWL_HI,SPF_PASS,USER_IN_DEF_SPF_WL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of aramani@paypal.com designates 216.113.175.153 as permitted sender) Received: from [216.113.175.153] (HELO den-mipot-002.corp.ebay.com) (216.113.175.153) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 07 May 2013 21:11:07 +0000 DomainKey-Signature: s=paypalcorp; d=paypal.com; c=nofws; q=dns; h=X-EBay-Corp:X-IronPort-AV:Received:Received:Received: From:To:Subject:Thread-Topic:Thread-Index:Date:Message-ID: In-Reply-To:Accept-Language:Content-Language: X-MS-Has-Attach:X-MS-TNEF-Correlator:user-agent: x-originating-ip:Content-Type:MIME-Version:Return-Path: X-CFilter; b=QxW90QffmJyZ1QBaj7+cfE/fylJckS37WnwYwMkrJIuOAXA7uP/Ab897 m+1D/Wsuq2AXsnD4o5kQjHsNlVjh3gWpLa3tJjH60ChebAKljfbl0X2uW 3yz/tP4ZCYLprxqAzeGDRGahbVIpjqSOQ0N6rQiwe1s0VurIBNM00qw2M I=; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paypal.com; i=@paypal.com; q=dns/txt; s=paypalcorp; t=1367961067; x=1399497067; h=from:to:subject:date:message-id:in-reply-to:mime-version; bh=gnjCIg5sc5fnrmsuogbs3RLITifa7DMUk9jGQwqHPsk=; b=hxp3T5RplI1bsGRGhOjgv4Sp7JZ/sa5fry2VytIVSR8IOKQ0D5c7AnOQ pjkJDP1jGlvWIMyzjQupd6llfAfrluIUKxIl9Hk7cZQZC/4uNVbCPvzeo phIKt29d1Fkzsi/OvwYspgzuchPvk104VuLelwiNftiEbis24DLMZvYTU Q=; X-EBay-Corp: Yes X-IronPort-AV: E=Sophos;i="4.87,629,1363158000"; d="scan'208,217";a="16400232" Received: from den-vtenf-002.corp.ebay.com (HELO DEN-EXMHT-001.corp.ebay.com) ([10.101.112.213]) by den-mipot-002.corp.ebay.com with ESMTP; 07 May 2013 14:10:46 -0700 Received: from PHX-EXMHT-002.corp.ebay.com (10.58.12.72) by DEN-EXMHT-001.corp.ebay.com (10.241.17.148) with Microsoft SMTP Server (TLS) id 14.2.318.4; Tue, 7 May 2013 15:10:45 -0600 Received: from PHX-EXRDA-S11.corp.ebay.com ([169.254.16.15]) by PHX-EXMHT-002.corp.ebay.com ([10.58.12.72]) with mapi id 14.02.0318.004; Tue, 7 May 2013 14:10:45 -0700 From: "Ramani, Arun" To: "user@giraph.apache.org" Subject: Re: Giraph and Fair Scheduler Thread-Topic: Giraph and Fair Scheduler Thread-Index: AQHOSsN3Rh9A6vTSzkeDXnd71nc1/5j6G04AgAAd1QA= Date: Tue, 7 May 2013 21:10:44 +0000 Message-ID: <464149AF3CBBD84E95E15DBFCA05285104737202@PHX-EXRDA-S11.corp.ebay.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.2.130206 x-originating-ip: [10.58.61.10] Content-Type: multipart/alternative; boundary="_000_464149AF3CBBD84E95E15DBFCA05285104737202PHXEXRDAS11corp_" MIME-Version: 1.0 X-CFilter: Scanned den2 X-Virus-Checked: Checked by ClamAV on apache.org --_000_464149AF3CBBD84E95E15DBFCA05285104737202PHXEXRDAS11corp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi Claudio, Which timeout are you referring? Thanks Arun Ramani From: Claudio Martella > Reply-To: "user@giraph.apache.org" > Date: Tuesday, May 7, 2013 5:23 AM To: "user@giraph.apache.org" > Subject: Re: Giraph and Fair Scheduler it's probably an obvious question, but have you tried increasing the timeou= t? On Tue, May 7, 2013 at 3:37 AM, Ramani, Arun > wrote: Hi, I am running Fair scheduler with many applications in hadoop stack in my cl= uster (like pig, hive, hbase etc). I have dedicated a pool for Giraph and w= ant to run giraph along with those other applications. I have configured pr= e-emption and and set the "minsharepreemptiontimeout=3D5" (sec =96 for the = jobs submitted to this pool to wait to get the min share). I am trying to run giraph in this mode. I see that jobs from other pools ar= e getting pre-empted to give the giraph job's pool its configured min share= but my job fails with "Unable to create native thread" error. This same jo= b passes if the slots are available immediately without having to wait for = the tasks from other queues to be pre-empted. I also tried to tweak the "gi= raph.minPercentResponded=3D50.0f". My Giraph job still fails. Please help i= n this scenario. Basically, I wanted to know how to configure giraph to wait for a threshold= for the slots to be available for it through pre-emption. Thanks Arun Ramani -- Claudio Martella claudio.martella@gmail.com --_000_464149AF3CBBD84E95E15DBFCA05285104737202PHXEXRDAS11corp_ Content-Type: text/html; charset="Windows-1252" Content-ID: Content-Transfer-Encoding: quoted-printable
Hi Claudio,

Which timeout are you referring?

Thanks
Arun Ramani

From: Claudio Martella <claudio.martella@gmail.com> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
Date: Tuesday, May 7, 2013 5:23 AM<= br> To: "user@giraph.apache.org" <user@giraph.apache.org>
Subject: Re: Giraph and Fair Schedu= ler

it's probably an obvious question, but have you tried incr= easing the timeout?


On Tue, May 7, 2013 at 3:37 AM, Ramani, Arun <aramani@paypal.= com> wrote:
Hi,

I am running Fair scheduler with many applications in hadoop stack in = my cluster (like pig, hive, hbase etc). I have dedicated a pool for Giraph = and want to run giraph along with those other applications. I have configur= ed pre-emption and and set the "minsharepreemptiontimeout=3D5" (sec =96 for the jobs submitted to this pool to wait to get the min share)= .

I am trying to run giraph in this mode. I see that jobs from other poo= ls are getting pre-empted to give the giraph job's pool its configured min = share but my job fails with "Unable to create native thread" erro= r. This same job passes if the slots are available immediately without having to wait for the tasks from other queues to be p= re-empted. I also tried to tweak the "giraph.minPercentResponded=3D50.= 0f". My Giraph job still fails. Please help in this scenario.

Basically, I wanted to know how to configure giraph to wait for a thre= shold for the slots to be available for it through pre-emption.

Thanks
Arun Ramani



--
   Claudio Martella
   claudio.martella@gmail.com   
--_000_464149AF3CBBD84E95E15DBFCA05285104737202PHXEXRDAS11corp_--