Return-Path: 
 <user-return-741-apmail-hama-user-archive=hama.apache.org@hama.apache.org>
X-Original-To: apmail-hama-user-archive@www.apache.org
Delivered-To: apmail-hama-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 69032D100
	for <apmail-hama-user-archive@www.apache.org>;
 Wed,  5 Dec 2012 10:58:59 +0000 (UTC)
Received: (qmail 22870 invoked by uid 500); 5 Dec 2012 10:58:59 -0000
Delivered-To: apmail-hama-user-archive@hama.apache.org
Received: (qmail 22843 invoked by uid 500); 5 Dec 2012 10:58:58 -0000
Mailing-List: contact user-help@hama.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hama.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hama.apache.org>
List-Post: <mailto:user@hama.apache.org>
List-Id: <user.hama.apache.org>
Reply-To: user@hama.apache.org
Delivered-To: mailing list user@hama.apache.org
Received: (qmail 22805 invoked by uid 99); 5 Dec 2012 10:58:58 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Dec 2012 10:58:58 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of thomas.jungblut@gmail.com
 designates 209.85.216.175 as permitted sender)
Received: from [209.85.216.175] (HELO mail-qc0-f175.google.com)
 (209.85.216.175)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Dec 2012 10:58:51 +0000
Received: by mail-qc0-f175.google.com with SMTP id j3so2793142qcs.34
        for <user@hama.apache.org>; Wed, 05 Dec 2012 02:58:30 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=p97702Hc+cLQQdNiYCAUvBwRrcuHTOX56TC1T1q76mE=;
        b=HjIk9vcYI/3syxNb77AHfdboaeyfEsP7ustcZonbD72L+sOyfKWXfPs/jyFimTMv9i
         mAxRYKAyUGXhJMOyT5JMM7pkYyVvlpGzMSh5DTajeUGZuWRNgbdvYPazN2MxiddnOWe4
         Ei/aEXBabHu2dALuCWXoLATmWc1gtxelcoDKQK7o7yJuMhdLWrZ7rsEAX9sl2Y6qfait
         1oeDSdTqKjaMOaXIXagfXDQIcvJGTxTsyfN0C7kh3L+S0JUknJm2tjp4zn01twr4R0mQ
         0MQG/JQtUK7cWbxJuR/115KQs8zJDpPVcXKlIYZS9VcBC3fIhNLGrELZTxQuFCCGIKxg
         gFAQ==
MIME-Version: 1.0
Received: by 10.229.77.8 with SMTP id e8mr6242883qck.156.1354705110652; Wed,
 05 Dec 2012 02:58:30 -0800 (PST)
Received: by 10.49.1.2 with HTTP; Wed, 5 Dec 2012 02:58:30 -0800 (PST)
In-Reply-To: <DD2B79A5-6303-4FE7-BA49-D6D15CAAB7C3@disi.unitn.it>
References: <62685C8E-DFCE-43D2-AFEB-0288CCCB2CDB@disi.unitn.it>
	<CAJ-=ysmPzmqiEzUuG62PF8E4M=taxvR4_20rKDi2cu03bAPFXQ@mail.gmail.com>
	<DD2B79A5-6303-4FE7-BA49-D6D15CAAB7C3@disi.unitn.it>
Date: Wed, 5 Dec 2012 11:58:30 +0100
Message-ID: 
 <CAJ-=ysk0CPYEOwCpHN61B+fzrrae_1=tOcyLKY1LjayGaUYMpw@mail.gmail.com>
Subject: Re: Partitioning
From: Thomas Jungblut <thomas.jungblut@gmail.com>
To: user@hama.apache.org
Content-Type: multipart/alternative; boundary=002354471968e5588404d018dc8d
X-Virus-Checked: Checked by ClamAV on apache.org

--002354471968e5588404d018dc8d
Content-Type: text/plain; charset=ISO-8859-1

So it will spawn 12 tasks. If this doesn't satisfy the load on your
machines, try to use smaller blocksizes.

2012/12/5 Benedikt Elser <elser@disi.unitn.it>

> Hi,
>
> thanks for your reply!
>
>  Total size:    49078776 B
>  Total dirs:    1
>  Total files:   12
>  Total blocks (validated):      12 (avg. block size 4089898 B)
>
> Benedikt
>
> On Dec 5, 2012, at 11:47 AM, Thomas Jungblut wrote:
>
> > So how many blocks has your data in HDFS?
> >
> > 2012/12/5 Benedikt Elser <elser@disi.unitn.it>
> >
> >> Hi List,
> >>
> >> I am using the hama-0.6.0 release to run graph jobs on various input
> >> graphs in a ec2 based cluster of size 12. However as I see in the logs
> not
> >> every node on the cluster contributes to that job (they have no
> >> tasklog/job<ID> dir and are idle). Theoretically a distribution of 1
> >> Million nodes across 12 buckets should hit every node at least once.
> >> Therefore I think its a configuration problem. So far I messed around
> with
> >> these settings:
> >>
> >>   <name>bsp.max.tasks.per.job</name>
> >>   <name>bsp.local.tasks.maximum</name>
> >>   <name>bsp.tasks.maximum</name>
> >>   <name>bsp.child.java.opts</name>
> >>
> >> Setting bsp.local.tasks.maximum to 1 and bsp.tasks.maximum.per.job to 12
> >> hat not the desired effect. I also split the input into 12 files
> (because
> >> of something in 0.5, that was fixed in 0.6).
> >>
> >> Could you recommend me some settings or guide me through the system's
> >> partition decision? I thought it would be:
> >>
> >> Input -> Input Split based on input, max* conf values -> number of tasks
> >> HashPartition.class distributes Ids across that number of tasks.
> >>
> >> Thanks,
> >>
> >> Benedikt
>
>

--002354471968e5588404d018dc8d--