hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hamburg" by HyunsikChoi
Date Mon, 14 Sep 2009 00:54:34 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by HyunsikChoi:
http://wiki.apache.org/hadoop/Hamburg

The comment on the change is:
Metioned the locality-preserving storage method for graphs

------------------------------------------------------------------------------
   * Fault-Tolerant Implementation
  
  == Hamburg ==
- Hambrug is an alternative to M/R programming model. It is based on bulk synchronization
parallel (BSP) model. Like MR, Hamburg will take advantages from shared-nothing architecture
(SN), so I expect that it will also show scalability without almost degradation of performance
as the number of participant nodes increases. In addition, we will provide a set of easy APIs
familiar with graph features and similar to MR.
+ Hambrug is an alternative to MR programming model. It consists of two parts, each of which
is related to locality-preserving storage method for graph in terms of connectivity and computations
with traverse interface on graphs respectively.
  
+ The main purpose of Locality-preserving storage methods for graph is to store vertices close
to one another into the same HDFS block. The computation part with this may reduce considerable
communication cost and the number of bulk sync step. It will be a kind of preprocessed step
and be implemented in MR.
+ 
+ The computation part is based on bulk synchronization parallel (BSP) model. Like MR, Hamburg
will take advantages from shared-nothing architecture (SN), so I expect that it will also
show scalability without almost degradation of performance as the number of participant nodes
increases. In addition, we will provide a set of easy APIs familiar with graph features and
similar to MR.
+ 
- A Hamburg based on BSP computation step consists of three sub steps:
+ The computation part based on BSP computation step consists of three sub steps:
   * Computation on data that reside in local storage; it is similar to map operation in M/R.
   * Each node communicates its necessary data into one another.
   * All processors synchronize which waits for all of the communications actions to complete.

Mime
View raw message