Return-Path: X-Original-To: apmail-incubator-hama-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-hama-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9F66196E1 for ; Thu, 5 Apr 2012 14:36:32 +0000 (UTC) Received: (qmail 97519 invoked by uid 500); 5 Apr 2012 14:36:32 -0000 Delivered-To: apmail-incubator-hama-user-archive@incubator.apache.org Received: (qmail 97494 invoked by uid 500); 5 Apr 2012 14:36:32 -0000 Mailing-List: contact hama-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hama-user@incubator.apache.org Delivered-To: mailing list hama-user@incubator.apache.org Received: (qmail 97486 invoked by uid 99); 5 Apr 2012 14:36:32 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Apr 2012 14:36:32 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of praveensripati@gmail.com designates 209.85.217.175 as permitted sender) Received: from [209.85.217.175] (HELO mail-lb0-f175.google.com) (209.85.217.175) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Apr 2012 14:36:26 +0000 Received: by lbky2 with SMTP id y2so79760lbk.6 for ; Thu, 05 Apr 2012 07:36:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=lRhL4dxiMihxDm0oO5tpyISiGPBSrfefl9TIPn5mS9w=; b=wgaUCpmDC3b0zLtU+mHnd8D0StldVhxLZmSQS6i5ul+IXjLLx4uBwgjODcCVdl5694 0VUTsCRtNvxo0CifjqXKc2x0Xt8i6l6e0saPhsoU0npzbJOtPOiB48Vplc9QFr4FpdQf Hno5Rc2gE1FzbPTn/fA4RTtjqLGdF+BJt8yrTHgxZ69FEq7OFbrWJFUEkHhWPpTzMLJl H1N/NbFdYVSvYApbRs4BlQi3q7EZOYPAIY4wO4f2MANzznB39Ha8GEhSGftdeNleWkGA SQP9HvhSjbVrdT8gpeRFaUmBT9TndriaLnK9rM09q0WE263mtb1WlGFw7B1uzu2KL+um +U7w== MIME-Version: 1.0 Received: by 10.152.105.19 with SMTP id gi19mr3378161lab.11.1333636565559; Thu, 05 Apr 2012 07:36:05 -0700 (PDT) Received: by 10.112.7.231 with HTTP; Thu, 5 Apr 2012 07:36:05 -0700 (PDT) In-Reply-To: References: Date: Thu, 5 Apr 2012 20:06:05 +0530 Message-ID: Subject: Re: # of BSP task slots and # of InputSplits From: Praveen Sripati To: hama-user@incubator.apache.org Content-Type: multipart/alternative; boundary=f46d040716e1bfeaef04bcef75cb --f46d040716e1bfeaef04bcef75cb Content-Type: text/plain; charset=ISO-8859-1 So, if more slots are available after the initial 10 slots then Hama can't use them because the assignment has already been done. Looks like not an efficient use of the cluster. Hadoop is able to use the additional slots effectively. It would be nice to evaluate the Hadoop way also for Hama. Praveen On Thu, Apr 5, 2012 at 4:42 PM, Thomas Jungblut < thomas.jungblut@googlemail.com> wrote: > No this kind of execution is so Hadoop-y. > It will use the maximum slots (10 in your case) and tries to fit the input > to the 10 slots e.G by assigning multiple files/blocks to a single task. > > Am 5. April 2012 12:37 schrieb Praveen Sripati : > > > Hi, > > > > If the number of InputSplits is more than the number of bsp task slots > > available in the cluster (which is the case is most of the scenarios) how > > is this handled in the Hama? Will the tasks run in multiple iterations > > storing the intermediate messages in HDFS? For example, lets say there > are > > 100 InputSplits and 10 bsp slots. So, it will require 10 iterations of 10 > > bsp tasks to complete the job. > > > > Praveen > > > > > > -- > Thomas Jungblut > Berlin > --f46d040716e1bfeaef04bcef75cb--