hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@googlemail.com>
Subject Re: About HAMA-410
Date Fri, 08 Jul 2011 05:13:51 GMT
Yeah :) So the design is good ;D

Hmm, good question.
>
> I think, ZK have to manage all tasks. Otherwise, it'll increase the
> complexity of the program.
>

You're right.

2011/7/8 Edward J. Yoon <edwardyoon@apache.org>

> Just FYI,
>
> To better understand, refer the diagram, described in 0.2 user guide:
>
> http://incubator.apache.org/hama/docs/r0.2.0/ApacheHama-0.2_UserGuide.pdf
>
> On Fri, Jul 8, 2011 at 6:33 AM, Thomas Jungblut
> <thomas.jungblut@googlemail.com> wrote:
> > Yes, that is already implemented in the latest patch.
> > That is quite okay, I would be +1 to let each task be a BSPPeer. Or
> > actually, has a BSPPeer, maybe we are going to add some kind of JVM
> reuse,
> > then we just have to set a new BSPPeer instead of swapping the whole
> task.
> >
> > Overall I thought of this cascading design:
> >
> >>BSPMaster
> > ->Groom1
> > -->Task1
> > -->Task2
> > ->Groom2
> > -->Task3
> >
> > So each task can directly communicate with other tasks using RPC.
> (Altough
> > I'm not a great friend of this RPC stuff [1])
> > Grooms are only there to communicate with each task, for pinging tasks to
> be
> > alive. And the BSPMaster is responsible to keep track of the availability
> of
> > the grooms.
> >
> > We should take care of syncs and use them as sparse as possible, since
> they
> > tend to be a large bottleneck.
> > What about the barrier sync of zookeeper? Does he can deal with these
> > multiple tasks? Would each task be a znode?
> >
> > [1]:
> >
> https://issues.apache.org/jira/browse/HAMA-358?focusedCommentId=13059229&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13059229
> >
> > 2011/7/7 Edward J. Yoon <edwardyoon@apache.org>:
> >> Invoked (child) process will become a BSPPeer.
> >>
> >> On Thu, Jul 7, 2011 at 9:14 PM, Thomas Jungblut
> >> <thomas.jungblut@googlemail.com> wrote:
> >>> Just for clarification:
> >>> What is your plan now?
> >>> To setup a BSPPeer for several tasks on a server (groom) or is the
> >>> groom now the one and only BSPPeer?
> >>>
> >>> 2011/7/7 Edward J. Yoon <edwardyoon@apache.org>:
> >>>> Hi,
> >>>>
> >>>> To support multi-tasks, I'm thinking about merging BSPPeer and Task.
> >>>> Then, communication will be occurred among Tasks directly. I think,
> >>>> there's no need to manage BSPPeers inside GroomServer.
> >>>>
> >>>> Can we think about the latent side-effects from this decision,
> together?
> >>>>
> >>>> Thanks.
> >>>>
> >>>> --
> >>>> Best Regards, Edward J. Yoon
> >>>> @eddieyoon
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Thomas Jungblut
> >>> Berlin
> >>>
> >>> mobile: 0170-3081070
> >>>
> >>> business: thomas.jungblut@testberichte.de
> >>> private: thomas.jungblut@gmail.com
> >>>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
> >
> >
> >
> > --
> > Thomas Jungblut
> > Berlin
> >
> > mobile: 0170-3081070
> >
> > business: thomas.jungblut@testberichte.de
> > private: thomas.jungblut@gmail.com
> >
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>



-- 
Thomas Jungblut
Berlin

mobile: 0170-3081070

business: thomas.jungblut@testberichte.de
private: thomas.jungblut@gmail.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message