hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MaoYuan Xian (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HAMA-629) Improve RPC Scalability Part 2
Date Thu, 30 Oct 2014 11:37:34 GMT

     [ https://issues.apache.org/jira/browse/HAMA-629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

MaoYuan Xian updated HAMA-629:
------------------------------
    Issue Type: Technical task  (was: Sub-task)

> Improve RPC Scalability Part 2
> ------------------------------
>
>                 Key: HAMA-629
>                 URL: https://issues.apache.org/jira/browse/HAMA-629
>             Project: Hama
>          Issue Type: Technical task
>          Components: graph
>    Affects Versions: 0.5.0
>            Reporter: Thomas Jungblut
>             Fix For: 0.7.0
>
>
> There is a problem when all 1k peers would attempt to send to a single peer (let's say
a master task in a graph algorithm that aggregates). In this case the peer will start 1k-threads
which is using enourmous amount of memory. 
> I think we can coordinate the message sending either with Zookeeper or by using the task
id and do a smarter sending chain.
> By the last, I mean, that each task can start at a different offset in the peer array
to start sending messages to the other peers. But this won't solve the problem DDoS'ing a
single master task.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message