incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Hama Yarn module, process flow diagram
Date Wed, 07 Dec 2011 06:51:11 GMT
Simply good.

Here's more explanation about my original diagram:

1. ApplicationManager of ResourceManager requests a NodeManager to
launch ApplicationMaster (BSPMaster).
2. ApplicationMaster gets the list of NodeManager to launch
BSPTaskLauncher (GroomServer).
3. ....

IMO, BSP Master, BSP Workers, and BSPTask will be understandable.

On Wed, Dec 7, 2011 at 3:21 PM, Thomas Jungblut
<thomas.jungblut@googlemail.com> wrote:
> http://people.apache.org/~tjungblut/yarn-hama-diagram.pdf
>
> This is the link to the PDF. I'm a bit dizzy in the last time.
> What do you think Edward?
>
> 2011/12/7 ChiaHung Lin <chl501@nuk.edu.tw>
>
>>
>> Thanks for the explain and diagram uploaded. It is quite clear.
>>
>> -----Original message-----
>> From:Thomas Jungblut <thomas.jungblut@googlemail.com>
>> To:hama-dev@incubator.apache.org
>> Date:Tue, 6 Dec 2011 19:11:12 +0100
>> Subject:Re: Hama Yarn module, process flow diagram
>>
>> Soo here is the apache.org upload:
>> http://people.apache.org/~tjungblut/yarn-hama-diagram.pdf<
>> http://people.apache.org/~tjungblut/Efficient%20Barrier%20Synchronization%20Mechanism.pdf
>> >
>> http://people.apache.org/~tjungblut/yarn-hama-diagram.pptx
>>
>> Let me explain it though.
>> The client submits the job to the YARN infrastructure's resource manager.
>> This will take care to schedule the BSPApplicationMaster container
>> (actually this is going through the NodeManager as well).
>> Once the BSPApplicationmaster runs, it will examine the conf, start the RPC
>> servers needed (zookeeper as well) and then reports back to the
>> Resourcemanager (actually I missed to put the arrow back to the
>> resourcemanager, it is currently pointing to the scheduler). However, the
>> Resourcemanager will then start the required containers through the
>> NodeManagers once again.
>> If the BSPRunner runs the real task, it will piggyback heartbeats to the
>> BSPApplicationMaster. The container which is running is always piggybacking
>> to the nodemanager as well.
>>
>> Sorry for the filehoster upload.
>>
>>
>> 2011/12/6 Thomas Jungblut <thomas.jungblut@googlemail.com>
>>
>> > I updated this, sorry I can't upload on apache because I have no key at
>> > this pc.
>> >
>> > http://uploading.com/files/2af323d7/yarn-hama-diagram.pdf/
>> > http://uploading.com/files/34m35aee/yarn-hama-diagram.pptx/
>> >
>> >
>> > 2011/12/6 ChiaHung Lin <chl501@nuk.edu.tw>
>> >
>> >>
>> >> Do it have pdf or png format? I can't read it because I don't have MS
>> >> office suite.
>> >>
>> >> -----Original message-----
>> >> From:Edward J. Yoon <edwardyoon@apache.org>
>> >> To:hama-dev@incubator.apache.org
>> >> Date:Tue, 6 Dec 2011 17:02:31 +0900
>> >> Subject:Hama Yarn module, process flow diagram
>> >>
>> >> Hi all,
>> >>
>> >> Roughly I just draw a process flow diagram of YARN module[1]. Please
>> >> feel free to use this for any purpose and update it.
>> >>
>> >> And this is minor issue but I'd like to renaming several classes.
>> >> e.g., BSPRunner to BSPTask if I draw correctly.
>> >>
>> >> 1.
>> http://people.apache.org/~edwardyoon/documents/yarn-hama-diagram.pptx
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> @eddieyoon
>> >>
>> >>
>> >> --
>> >> ChiaHung Lin
>> >> Department of Information Management
>> >> National University of Kaohsiung
>> >> Taiwan
>> >>
>> >
>> >
>> >
>> > --
>> > Thomas Jungblut
>> > Berlin <thomas.jungblut@gmail.com>
>> >
>>
>>
>>
>> --
>> Thomas Jungblut
>> Berlin <thomas.jungblut@gmail.com>
>>
>>
>> --
>> ChiaHung Lin
>> Department of Information Management
>> National University of Kaohsiung
>> Taiwan
>>
>
>
>
> --
> Thomas Jungblut
> Berlin <thomas.jungblut@gmail.com>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message