incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: About HAMA-410
Date Fri, 08 Jul 2011 04:39:31 GMT
> What about the barrier sync of zookeeper? Does he can deal with these
> multiple tasks? Would each task be a znode?

Hmm, good question.

I think, ZK have to manage all tasks. Otherwise, it'll increase the
complexity of the program.

On Fri, Jul 8, 2011 at 11:46 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> Just FYI,
>
> To better understand, refer the diagram, described in 0.2 user guide:
>
> http://incubator.apache.org/hama/docs/r0.2.0/ApacheHama-0.2_UserGuide.pdf
>
> On Fri, Jul 8, 2011 at 6:33 AM, Thomas Jungblut
> <thomas.jungblut@googlemail.com> wrote:
>> Yes, that is already implemented in the latest patch.
>> That is quite okay, I would be +1 to let each task be a BSPPeer. Or
>> actually, has a BSPPeer, maybe we are going to add some kind of JVM reuse,
>> then we just have to set a new BSPPeer instead of swapping the whole task.
>>
>> Overall I thought of this cascading design:
>>
>>>BSPMaster
>> ->Groom1
>> -->Task1
>> -->Task2
>> ->Groom2
>> -->Task3
>>
>> So each task can directly communicate with other tasks using RPC. (Altough
>> I'm not a great friend of this RPC stuff [1])
>> Grooms are only there to communicate with each task, for pinging tasks to be
>> alive. And the BSPMaster is responsible to keep track of the availability of
>> the grooms.
>>
>> We should take care of syncs and use them as sparse as possible, since they
>> tend to be a large bottleneck.
>> What about the barrier sync of zookeeper? Does he can deal with these
>> multiple tasks? Would each task be a znode?
>>
>> [1]:
>> https://issues.apache.org/jira/browse/HAMA-358?focusedCommentId=13059229&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13059229
>>
>> 2011/7/7 Edward J. Yoon <edwardyoon@apache.org>:
>>> Invoked (child) process will become a BSPPeer.
>>>
>>> On Thu, Jul 7, 2011 at 9:14 PM, Thomas Jungblut
>>> <thomas.jungblut@googlemail.com> wrote:
>>>> Just for clarification:
>>>> What is your plan now?
>>>> To setup a BSPPeer for several tasks on a server (groom) or is the
>>>> groom now the one and only BSPPeer?
>>>>
>>>> 2011/7/7 Edward J. Yoon <edwardyoon@apache.org>:
>>>>> Hi,
>>>>>
>>>>> To support multi-tasks, I'm thinking about merging BSPPeer and Task.
>>>>> Then, communication will be occurred among Tasks directly. I think,
>>>>> there's no need to manage BSPPeers inside GroomServer.
>>>>>
>>>>> Can we think about the latent side-effects from this decision, together?
>>>>>
>>>>> Thanks.
>>>>>
>>>>> --
>>>>> Best Regards, Edward J. Yoon
>>>>> @eddieyoon
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Thomas Jungblut
>>>> Berlin
>>>>
>>>> mobile: 0170-3081070
>>>>
>>>> business: thomas.jungblut@testberichte.de
>>>> private: thomas.jungblut@gmail.com
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>> @eddieyoon
>>>
>>
>>
>>
>> --
>> Thomas Jungblut
>> Berlin
>>
>> mobile: 0170-3081070
>>
>> business: thomas.jungblut@testberichte.de
>> private: thomas.jungblut@gmail.com
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message