Mailing-List: contact hama-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hama-dev@incubator.apache.org
Date: Sat, 19 May 2012 09:12:08 +0000 (UTC)
From: "praveen sripati (JIRA)" <jira@apache.org>
To: hama-dev@incubator.apache.org
Message-ID: 
 <2038462394.18421.1337418728762.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1772156078.29848.1336182468495.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HAMA-569) Make Hama scalable as more processing
 is done
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HAMA-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279500#comment-13279500 ] 

praveen sripati commented on HAMA-569:
--------------------------------------

In case, where the processing of a certain event is independent of the earlier event then it should be theoretically possible (keeping aside the Hama limitations if any) to scale up/down the # of bsp nodes for processing. Whenever there is any dependency, there is a need to keep track of # of bsp nodes at any instant of time to know which bsp node is processing a certain event.

Couldn't find much literature on using BSP for real time or on scaling BSP up/down. The closest I could get is Adaptive Parallelism in the Bulk-Synchronous Parallel Model (quickly glanced through it)

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.53.9686

Also, looks like cloudscale (http://www.cloudscale.com/index.php/technology/cloudscale-bsp) uses BSP which is scalable.
                
> Make Hama scalable as more processing is done
> ---------------------------------------------
>
>                 Key: HAMA-569
>                 URL: https://issues.apache.org/jira/browse/HAMA-569
>             Project: Hama
>          Issue Type: Improvement
>          Components: bsp core
>    Affects Versions: 0.4.0, 0.5.0
>            Reporter: praveen sripati
>             Fix For: 0.6.0
>
>
> Currently Hama doesn't scale. Once the job has been submitted, the # of the bsp tasks is fixed. So, there are fixed costs associated with the job. The JIRA is to evaluate if Hama can be made scalable automatically once the job has been submitted and provide a solution for the same. This applies to both batch and real time processing.
> For ex., in the case of real time processing the # of bsp tasks once the job has been submitted remain the same for 1 or a million inputs per second.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira