Mailing-List: contact dev-help@hama.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hama.apache.org
Date: Mon, 10 Mar 2014 02:08:45 +0000 (UTC)
From: "Edward J. Yoon (JIRA)" <jira@apache.org>
To: dev@hama.apache.org
Message-ID: <JIRA.12603751.1345136176775.32946.1394417325217@arcas>
In-Reply-To: <JIRA.12603751.1345136176775@arcas>
References: <JIRA.12603751.1345136176775@arcas>
Subject: [jira] [Commented] (HAMA-629) Improve RPC Scalability Part 2
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HAMA-629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13925398#comment-13925398 ] 

Edward J. Yoon commented on HAMA-629:
-------------------------------------

I think this hierarchical synchronization style is fit for only master aggregation scenario e.g., graph aggregators. The performance can be slower than normal style in other cases.

> Improve RPC Scalability Part 2
> ------------------------------
>
>                 Key: HAMA-629
>                 URL: https://issues.apache.org/jira/browse/HAMA-629
>             Project: Hama
>          Issue Type: Sub-task
>          Components: graph
>    Affects Versions: 0.5.0
>            Reporter: Thomas Jungblut
>             Fix For: 0.7.0
>
>
> There is a problem when all 1k peers would attempt to send to a single peer (let's say a master task in a graph algorithm that aggregates). In this case the peer will start 1k-threads which is using enourmous amount of memory. 
> I think we can coordinate the message sending either with Zookeeper or by using the task id and do a smarter sending chain.
> By the last, I mean, that each task can start at a different offset in the peer array to start sending messages to the other peers. But this won't solve the problem DDoS'ing a single master task.


--
This message was sent by Atlassian JIRA
(v6.2#6252)