Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 394F217783 for ; Thu, 23 Oct 2014 16:05:04 +0000 (UTC) Received: (qmail 70970 invoked by uid 500); 23 Oct 2014 16:05:03 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 70919 invoked by uid 500); 23 Oct 2014 16:05:03 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 70909 invoked by uid 99); 23 Oct 2014 16:05:03 -0000 Received: from ec2-54-76-25-247.eu-west-1.compute.amazonaws.com (HELO mx1-eu-west.apache.org) (54.76.25.247) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Oct 2014 16:05:03 +0000 Received: from mx1-eu-west.apache.org (localhost [127.0.0.1]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTP id E2D5C20185 for ; Thu, 23 Oct 2014 16:05:02 +0000 (UTC) Received: by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org, from userid 113) id D78AE26E5C; Thu, 23 Oct 2014 16:05:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mx1-eu-west.apache.org X-Spam-Level: * X-Spam-Status: No, score=1.5 required=10.0 tests=HTML_MESSAGE, RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_PASS,T_DKIM_INVALID, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from mail-qa0-f52.google.com (mail-qa0-f52.google.com [209.85.216.52]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id 96D3A20185 for ; Thu, 23 Oct 2014 16:05:01 +0000 (UTC) Received: by mail-qa0-f52.google.com with SMTP id v10so840017qac.25 for ; Thu, 23 Oct 2014 09:04:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=hVwXafJMVuhMeOMQfF01ALVhvFmyrEm7tbrH//KpfT4=; b=oj+xUx436/iDn8DrjJbvMI41DeMLkXU2MP+jDdKIqlQ8JO4IP6+f9zX1ICedrnXwv8 eeiI0vw8iNsUBpLfpy+GreTqHc0O4ipQkqzJj2JfxXLIPESj4Zf2dCXyqS0Z5r2VhHBw Cxd6b8MkReOdvQ7oMCA3u3Al4z4PymQiK9qHXdKAchM6IrNO52GTduq7zkahbjjPnJE2 /BsbyVxH+cmj6c8j0yjFAZgqTKdjvUn4c2aMquQgQ3hlHw+AV0KadmgyCfSncYAY6cK0 K0EGuJG7DHYLqvuGs0e4g7ehBrhuc8F9Qvy1kbqYo6qzsOTjczdvjd1UZ9FPnkOK97HK T6iA== X-Received: by 10.224.131.201 with SMTP id y9mr9482451qas.56.1414080299694; Thu, 23 Oct 2014 09:04:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.91.137 with HTTP; Thu, 23 Oct 2014 09:04:39 -0700 (PDT) In-Reply-To: References: From: Claudio Martella Date: Thu, 23 Oct 2014 18:04:39 +0200 Message-ID: Subject: Re: Resource Allocation Model Of Apache Giraph To: "user@giraph.apache.org" Content-Type: multipart/alternative; boundary=001a11c2c2c4f2b196050619392c X-Virus-Scanned: ClamAV using ClamSMTP --001a11c2c2c4f2b196050619392c Content-Type: text/plain; charset=UTF-8 the way mappers (or containers) and hence workers are assigned to machines is not under the control of giraph, but of the underlying hadoop environment (with different responsibilities that depend on the hadoop version, e.g. YARN). You'll have to tweak your hadoop configuration to control the maximum number of workers assigned to one machine (optimally one with multiple threads). On Thu, Oct 23, 2014 at 5:53 PM, Charith Wickramarachchi < charith.dhanushka@gmail.com> wrote: > Hi Folks, > > I'm wondering what is the resource allocation model for Apache Giraph. > > As I understand each worker is one to one Mapped with a Mapper and a > worker can process multiple partitions with a user defined number of > threads. > > Is it possible to make sure that one worker, only process a single > partition? Also is it possible to control the worker assignment in the > cluster nodes? (Ex: Make sure only N workers runs on a single machine, > assuming we have enough resources) > > Thanks, > Charith > > > > > > > > > -- > Charith Dhanushka Wickramaarachchi > > Tel +1 213 447 4253 > Web http://apache.org/~charith > > Blog http://charith.wickramaarachchi.org/ > > Twitter @charithwiki > > This communication may contain privileged or other confidential information > and is intended exclusively for the addressee/s. If you are not the > intended recipient/s, or believe that you may have > received this communication in error, please reply to the sender indicating > that fact and delete the copy you received and in addition, you should > not print, copy, retransmit, disseminate, or otherwise use the > information contained in this communication. Internet communications > cannot be guaranteed to be timely, secure, error or virus-free. The > sender does not accept liability for any errors or omissions > -- Claudio Martella --001a11c2c2c4f2b196050619392c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
the way mappers (or containers) and hence workers are assi= gned to machines is not under the control of giraph, but of the underlying = hadoop environment (with different responsibilities that depend on the hado= op version, e.g. YARN). You'll have to tweak your hadoop configuration = to control the maximum number of workers assigned to one machine (optimally= one with multiple threads).

On Thu, Oct 23, 2014 at 5:53 PM, Charith Wickramarachchi <= span dir=3D"ltr"><charith.dhanushka@gmail.com> wrote:
Hi Folks,=C2=A0

I'm wondering what is the resource allocation model for Apache Giraph.=C2=A0

As I understand each worker = is one to one Mapped with a=C2=A0Mapper and a worker can process multiple p= artitions with a user defined number of threads.=C2=A0

=
Is it possible to make sure that one worker, only process a single par= tition? Also is it possible to control the worker assignment in the cluster= nodes? (Ex: Make sure only N =C2=A0workers runs=C2=A0on a sin= gle machine, assuming we have enough resources)

Th= anks,
Charith =C2=A0=C2=A0








-- =
Charith Dhanushka Wickramaar= achchi

This communication may contain privil= eged or other confidential=C2=A0= information and is intended exclusively for the addressee/s. If you=C2=A0are not the intended recipient/s, = or believe that you may have
received this commu= nication in error, please reply to the sender=C2=A0indicating that fact and delete the copy you received an= d in addition,=C2=A0you should n= ot print, copy, retransmit, disseminate, or otherwise use=C2=A0the information contained in this communicat= ion. Internet=C2=A0communication= s cannot be guaranteed to be timely, secure, error or=C2=A0virus-free. The sender does not accept liability= for any errors or=C2=A0omission= s



--
=C2=A0 =C2=A0Claudio Martella
=C2=A0 =C2=A0
--001a11c2c2c4f2b196050619392c--