incubator-hama-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hama Wiki] Trivial Update of "Architecture" by Edward J. Yoon
Date Tue, 25 May 2010 07:28:26 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.

The "Architecture" page has been changed by Edward J. Yoon.
http://wiki.apache.org/hama/Architecture?action=diff&rev1=107&rev2=108

--------------------------------------------------

   * TODO: explain what is the BSP? and Hama BSP?
  
  === BSP Architecture/Design ===
+ One BSP cluster consists of one BSPMaster, multiple GroomServer and one or more zookeeper
servers in a network environment.
  
- ==== Job Manager System ====
+  * work in progress [[https://issues.apache.org/jira/browse/HAMA-251|HAMA-251]],[[https://issues.apache.org/jira/browse/HAMA-258|HAMA-258]]
+ ==== BSPMaster ====
+ BSPMaster is responsible to do the following: 
+  1. Maintaining groom server status. 
+  2. Controlling super steps in a cluster. 
+  3. Maintaining job progress information. 
+  4. Scheduling Jobs and Assigning tasks to groom servers 
+  5. Disseminating execution class across groom servers.
+  6. Controlling fault. 
+  7. Providing users with the cluster control interface.
  
-  * work in progress
+ A BSP Master and multiple grooms are started by the script. Then, the bsp master starts
up with a RPC server for groom servers. Groom servers starts up with a BSPPeer instance -
later, BSPPeer needs to be integrated with GroomServer - and a RPC proxy to contact the bsp
master. After started, each groom periodically sends a heartbeat message that encloses its
groom server status, including maximum task capacity, unused memory, and so on. 
  
+ Each time the bsp master receives a heartbeat message, it brings up-to-date groom server
status - the bsp master makes use of groom servers' status in order to effectively assign
tasks to idle groom servers - and returns a heartbeat response that contains assigned tasks
and others actions that a groom server has to do. For now, we have a FIFO job scheduler and
very simple task assignment algorithms.
+ ==== GroomServer ====
+ A Groom Server (shortly referred to as groom) is a process that performs bsp tasks assigned
by BSPMaster. Each groom contacts the BSPMaster, and it takes assigned tasks and reports its
status by means of periodical piggybacks with BSPMaster. Each groom is designed to run with
HDFS or other distributed storages. Basically, a groom server and a data node should be run
on one physical node.
+ 
- === Hama Examples ===
+ === BSP Example ===
-  * TODO: explain how to programming BSP program with Hama.
+  * TODO: explain how to programming BSP program.
   * work in progress [[https://issues.apache.org/jira/browse/HAMA-244|HAMA-244]]
  
  {{{
@@ -69, +83 @@

      BSPJobClient.runJob(bsp);
  }}}
  
+ 
+ ---- /!\ '''Edit conflict - other version:''' ----
+  * Hama Examples
    * Matrix Inversion
    * Pi Estimator
    * PageRank
+ 
+ ---- /!\ '''Edit conflict - your version:''' ----
+ 
+ ---- /!\ '''End of edit conflict''' ----
+ 
+ ---- /!\ '''Edit conflict - other version:''' ----
+ 
+ ---- /!\ '''Edit conflict - your version:''' ----
+     bsp.setJobName("BSP test job"); 
+ 
+     // Set in/output path and formatter 
+     bsp.setInputPath(conf, new Path("input path")); 
+     bsp.setOutputPath(conf, new Path("output path")); 
+     bsp.setInputFormat(MyInputFormat.class); 
+     bsp.setOutputFormat(MyOutputFormat.class); 
+      
+     // Set the BSP code 
+     bsp.setBSPCode(BSPProgram.class); 
+     BSPJobClient.runJob(bsp);
+ }}}
+ 
+ 
+ 
+ ---- /!\ '''End of edit conflict''' ----
  
  == Shell/DSL ==
  

Mime
View raw message