incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyunsik Choi <hyunsik.c...@gmail.com>
Subject Re: [Hama Wiki] Trivial Update of "Architecture" by Edward J. Yoon
Date Tue, 11 May 2010 03:58:08 GMT
I correct typos as follows:

I'm not focusing on which one is better between BSP and M/R. I misunderstood
your intention due to the above your mention. It is because we will need not
only the BSP framework but also other programs to help *users* to develop
matrix algebra/graph processing *applications*.

--
Hyunsik Choi
Database & Information Systems Group, Korea Univ.
http://diveintodata.org


On Tue, May 11, 2010 at 12:55 PM, Hyunsik Choi <hyunsik.choi@gmail.com>wrote:

> > As mentioned above, I'd like to show only some example of how you
> implement your own matrix/graph applications using BSP.
>
> I'm not focusing on which one is better between BSP and M/R. I
> misunderstood your intention due to the above your mention. It is because we
> will need not only the BSP framework but also other programs to help your to
> develop matrix algebra/graph processing.
>
> Best regards,
> --
> Hyunsik Choi
> Database & Information Systems Group, Korea Univ.
> http://diveintodata.org
>
>
> On Tue, May 11, 2010 at 11:19 AM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>
>> In this mailing
>> (
>> http://markmail.org/search/?q=hama-dev+discuss#query:hama-dev%20discuss+page:1+mid:amlvccbptom3yro3+state:results
>> )
>> and others, we discussed about:
>>
>> 1) We can't implement all of matrix algebra. 2) Even if we do that,
>> the code complexity based on M/R model is too high. As we learned, the
>> M/R model isn't fit well for the complex numerical/relational data
>> processing required for many communications between each others. 3)
>> Admittedly, BSP is flexible than M/R.
>>
>> Therefore, We've aligned our goal: to provide easy *matrix/graph
>> computing* programming environment on the distributed system (hadoop),
>> such as M/R framework. (NOT library) So, we won't develop only the BSP
>> framework if there is other solutions.
>>
>> On Tue, May 11, 2010 at 10:29 AM, Hyunsik Choi <hyunsik.choi@gmail.com>
>> wrote:
>> > According to you,  the main objective of hama appears to be developing
>> only
>> > the BSP framework. Is it right?
>> >
>> > --
>> > Hyunsik Choi
>> > Database & Information Systems Group, Korea Univ.
>> > http://diveintodata.org
>> >
>> >
>> > On Mon, May 10, 2010 at 8:34 PM, Edward J. Yoon <edwardyoon@apache.org
>> >wrote:
>> >
>> >> > In the distant future, the graph package would use both MR and BSP.
>> But,
>> >>
>> >> Where will M/R use for?
>> >>
>> >> I'd like to cut M/R out of HAMA project and leave only the pure BSP &
>> >> Graph computing framework as simple as possible. Then, the rest are
>> >> user's matrix/graph applications.
>> >>
>> >> And, I'd like to consider about building an contributor/developer
>> >> ECOsystem.
>> >>
>> >> > now i'll try to develop the graph package with only BSP. How about
>> the
>> >> > matrix package?
>> >>
>> >> As mentioned above, I'd like to show only some example of how you
>> >> implement your own matrix/graph applications using BSP.
>> >>
>> >> I think the block of hadoop families should be added on diagram only
>> >> to help understand relations between hadoop and hama,
>> >>
>> >> On Mon, May 10, 2010 at 7:40 PM, Hyunsik Choi <hyunsik.choi@gmail.com>
>> >> wrote:
>> >> > In the distant future, the graph package would use both MR and BSP.
>> But,
>> >> > now i'll try to develop the graph package with only BSP. How about
>> the
>> >> > matrix package?
>> >> >
>> >> > --
>> >> > Hyunsik Choi
>> >> > Database & Information Systems Group, Korea Univ.
>> >> > http://diveintodata.org
>> >> >
>> >> >
>> >> > On Mon, May 10, 2010 at 7:17 PM, Edward J. Yoon <
>> edwardyoon@apache.org
>> >> >wrote:
>> >> >
>> >> >> > zookeeper should be enclosed in BSP since zookeeper is not
>> disclosed
>> >> to
>> >> >> > external. It is only used inside the BSP.
>> >> >>
>> >> >> Right you are. And also, the top of M/R and BSP should be clarified,
>> >> >> but currently something is odd overall.
>> >> >>
>> >> >> On Mon, May 10, 2010 at 6:24 PM, Hyunsik Choi <
>> hyunsik.choi@gmail.com>
>> >> >> wrote:
>> >> >> > Good job! But, the matrix package is omitted. Besides, I think
>> that
>> >> >> > zookeeper should be enclosed in BSP since zookeeper is not
>> disclosed
>> >> to
>> >> >> > external. It is only used inside the BSP.
>> >> >> >
>> >> >> > Best regards,
>> >> >> > --
>> >> >> > Hyunsik Choi
>> >> >> > Database & Information Systems Group, Korea Univ.
>> >> >> > http://diveintodata.org
>> >> >> >
>> >> >> >
>> >> >> > On Mon, May 10, 2010 at 3:01 PM, Apache Wiki <
>> wikidiffs@apache.org>
>> >> >> wrote:
>> >> >> >
>> >> >> >> Dear Wiki user,
>> >> >> >>
>> >> >> >> You have subscribed to a wiki page or wiki category on
"Hama
>> Wiki"
>> >> for
>> >> >> >> change notification.
>> >> >> >>
>> >> >> >> The "Architecture" page has been changed by Edward J.
Yoon.
>> >> >> >>
>> http://wiki.apache.org/hama/Architecture?action=diff&rev1=93&rev2=94
>> >> >> >>
>> >> >> >> --------------------------------------------------
>> >> >> >>
>> >> >> >>
>> >> >> >>  This effort is still a "work in progress". Please feel
free to
>> add
>> >> >> >> comments.
>> >> >> >>  ----
>> >> >> >> +
>> >> >> >> + == The overall architecture of HAMA ==
>> >> >> >> + {{{
>> >> >> >> +      +-----------------------------------------------+
>> >> >> >> +      |                  HAMA APIs                  
 |
>> >> >> >> +      +-----------------------------------------------+
>> >> >> >> +   --------------------------------------------------------
>> >> >> >> +                          +---------------------------+
>> >> >> >> +                          | Graph Computing Framework
|
>> >> >> >> +                          +---------------------------+
>> Computing
>> >> >> Engine
>> >> >> >> +      +-----------+ +---------------------------------+
>> >> >> >> +      | MapReduce | | BSP (Bulk Synchronous Parallel)
|
>> >> >> >> +      +-----------+ +---------------------------------+
>> >> >> >> +   --------------------------------------------------------
>> >> >> >> +      +-----------------------------------------------+
>> >> >> >> +      |                  Zookeeper                  
 |
>> >> Distributed
>> >> >> >> Locking Service
>> >> >> >> +      +-----------------------------------------------+
>> >> >> >> +   --------------------------------------------------------
>> >> >> >> +      +-----------------------------------------------+
>> >> >> >> +      |                                             
 |
>> >> >> >> +      | +-----------------------------+             
 |
>> >> >> >> +      | | HBase : Distributed Storage |             
 |
>> >> >> >> +      | +-----------------------------+             
 |
>> >> Distributed
>> >> >> >> Storage Systems
>> >> >> >> +      | +---------------------------------+         
 |
>> >> >> >> +      | | HDFS : Distributed File System  |         
 |
>> >> >> >> +      | +---------------------------------+         
 |
>> >> >> >> +      +-----------------------------------------------+
>> >> >> >> + }}}
>> >> >> >> +
>> >> >> >>  == BSP ==
>> >> >> >>
>> >> >> >>  The BSP package is a implementation of BSP over Hadoop
>> RPC(sockets).
>> >> >> >>
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Best Regards, Edward J. Yoon
>> >> >> edwardyoon@apache.org
>> >> >> http://blog.udanax.org
>> >> >>
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> edwardyoon@apache.org
>> >> http://blog.udanax.org
>> >>
>> >
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message