Return-Path: X-Original-To: apmail-hama-dev-archive@www.apache.org Delivered-To: apmail-hama-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9A420DCDE for ; Thu, 28 Jun 2012 11:45:26 +0000 (UTC) Received: (qmail 2845 invoked by uid 500); 28 Jun 2012 11:45:26 -0000 Delivered-To: apmail-hama-dev-archive@hama.apache.org Received: (qmail 2654 invoked by uid 500); 28 Jun 2012 11:45:22 -0000 Mailing-List: contact dev-help@hama.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hama.apache.org Delivered-To: mailing list dev@hama.apache.org Received: (qmail 2241 invoked by uid 99); 28 Jun 2012 11:45:20 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Jun 2012 11:45:20 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of praveensripati@gmail.com designates 209.85.217.175 as permitted sender) Received: from [209.85.217.175] (HELO mail-lb0-f175.google.com) (209.85.217.175) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Jun 2012 11:45:13 +0000 Received: by lbol5 with SMTP id l5so3274852lbo.34 for ; Thu, 28 Jun 2012 04:44:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=9Xod7kXhqRd0zm7+O7z0h6JcHlayWALd4HbZbr+fiXU=; b=wFWChvchVy0oR5ckmPMBs9T9eIZ9EsSLtGQeSe0eMcvlg/m/i09l83fu0VFWzexaai CBAe7jnUZ+oKHmpdNeV39fhyKiG2QW0q0i+sI1rUuT7alJgRm/FS80hf/Z4U+UXDMsAY RYnhTVYM7qYp5Meen6rO35LFA/okfEgFVHyQW7jmQ0IDlVNwGDiDaoUgp+jFh3xiep39 DhVBiLh7JNloXx6jNlbMVYbIBK/c+BEtyU0SAzExOFtqiBeoqwmFmRLbSHhgn4tKGp7u wVQcAm/KODuu0TLPVKpJomIPl2tSZ5kCt0QDXtV8jA22/qWB820EVJvp3Dn76lcupYTg KHkA== MIME-Version: 1.0 Received: by 10.112.82.165 with SMTP id j5mr973231lby.50.1340883892405; Thu, 28 Jun 2012 04:44:52 -0700 (PDT) Received: by 10.112.114.136 with HTTP; Thu, 28 Jun 2012 04:44:52 -0700 (PDT) Date: Thu, 28 Jun 2012 17:14:52 +0530 Message-ID: Subject: Scaling Hama From: Praveen Sripati To: dev@hama.apache.org Content-Type: multipart/alternative; boundary=f46d0401f92717881204c386dc1a --f46d0401f92717881204c386dc1a Content-Type: text/plain; charset=ISO-8859-1 During the 24th minute of the recent Hadoop Summit Video [1] Avery Ching talks about how Giraph is made scalable. Since, Hama and Giraph are based on the BSP model, it should be possible to scale Hama in a similar fashion. Basically, at the end of each super step the BSP tasks sends some metrics to the master and the master further partitions the data in the most loaded BSP tasks and uses the the free available slot to process them. Thanks, Praveen [1] - http://www.youtube.com/watch?v=b5Qmz4zPj-M --f46d0401f92717881204c386dc1a--