giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maja Kabiljo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-273) Aggregators shouldn't use Zookeeper
Date Mon, 30 Jul 2012 14:50:34 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424895#comment-13424895
] 

Maja Kabiljo commented on GIRAPH-273:
-------------------------------------

For 2, I thought it would be better since we would have two phase distribution, master wouldn't
send the message to every worker. 

Dremel idea sounds interesting, my suggestion is to implement it in a simple way first, and
if the need arises in the future we can look into it then.
                
> Aggregators shouldn't use Zookeeper
> -----------------------------------
>
>                 Key: GIRAPH-273
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-273
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Maja Kabiljo
>            Assignee: Maja Kabiljo
>
> We use Zookeeper znodes to transfer aggregated values from workers to master and back.
Zookeeper is supposed to be used for coordination, and it also has a memory limit which prevents
users from having aggregators with large value objects. These are the reasons why we should
implement aggregators gathering and distribution in a different way.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message