hama-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hama Wiki] Update of "DynamicGraphs" by AnastasisAndronidis
Date Mon, 23 Sep 2013 12:19:05 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hama Wiki" for change notification.

The "DynamicGraphs" page has been changed by AnastasisAndronidis:
https://wiki.apache.org/hama/DynamicGraphs?action=diff&rev1=4&rev2=5

   4. New and old vertexes, will have the same super step counter. ''Various algorithms change
their behavior though time. If such a case, you need to develop your own state counter''
  
  === User Example ===
- ''The following code is part of the Graph examples. You can find it in org.apache.hama.examples.DynamicGraph''
+ ''The following code is part of the Graph examples. You can find it in org.apache.hama.examples.!DynamicGraph''
  
  This is an example of how to manipulate Graphs dynamically. The input of this example is
a number in each row. We assume that the is a vertex with ID:1 which is responsible to create
a sum vertex that will aggregate the values of the other vertices. During the aggregation,
sum vertex will delete all other vertices. 
  
@@ -144, +144 @@

  }
  }}}
  
- Starting from creating a class that will serve as a container of our example. In this case,
the class is '''DynamicGraph'''.
+ Starting from creating a class that will serve as a container of our example. In this case,
the class is '''!DynamicGraph'''.
  
- '''DynamicGraph''' contains three important parts.
+ '''!DynamicGraph''' contains three important parts.
  
  The first component is the declaration of the input reader. This class overrides the '''parseVertex'''
method and creates vertex instances from an input file. In our case the input is a text file
in which, each line has a number. This number is assigned as both the ID and value of the
vertex.
  
@@ -263, +263 @@

    }
  }}}
  
- As we can see, the compute method is calling 3 other methods. In the '''createSumVertex'''
method we can see the creation of a new vertex with ID the text ''sum'' and value 0. Later
on, in '''sendAllValuesToSumAndRemove''' we can see that each vertex that runs this method
is deleting itself by running ''this.remove();''. In the end, '''calculateSum''' is called
that summarizes the values of all vertices in the ''sum'' vertex. The interesting part of
the last method is that also adding the number of input vertices ''this.getPeer().getCounter(GraphJobCounter.INPUT_VERTICES).getCounter()''
and the current number of vertices that exist on the running superstep ''this.getNumVertices()''.
+ As we can see, the compute method is calling 3 other methods. 
+ 
+ In the '''createSumVertex''' method we can see the creation of a new vertex with ID the
text ''sum'' and value 0. 
+ 
+ Later on, in '''sendAllValuesToSumAndRemove''' we can see that each vertex that runs this
method is deleting itself by running `this.remove()`. 
+ 
+ In the end, '''calculateSum''' is called that summarizes the values of all vertices in the
''sum'' vertex. The interesting part of the last method is that also adding the number of
input vertices `this.getPeer().getCounter(GraphJobCounter.INPUT_VERTICES).getCounter()` and
the current number of vertices that exist on the running superstep `this.getNumVertices()`.
  
  === Implementation Details ===
+ 
+ addVertex --> create new vertex instance --> find a proper BSP peer to host the new
vertex through the partitioner --> serialize the new vertex and send it as a map message
to the peer --> the destination peer, on the new superstep, during executing parseMessages
method, will receive the serialized object ---> GraphJobRunner::addVertex will add the
new vertex in the data structure that keeps all vertices (VerticesInfo)
+ 
+ meanwhile
+ 
+ in step 4 the peer, hosting the vertex where the addVertex method is invoked, is increasing
a local variable by the number of new vertices added. This local variable is collected from
all peers from aggregators and results in changing the total number of vertices in all peers.
+ 
  The `GraphJobRunner` class, contains the major steps for the graph computation. It starts
with an initial setup, where the vertexes are loaded to memory, then there is the main computation
loop, where the supersteps occur, and in the end there is the cleanup, where of the graph
and the results are written to HDFS.
  
  Our main target (concerning the operations +/-) is to enable the functionality of `GraphJobRunner::loadVertices()`
function during a superstep so the user will be able to call a function during a superstep
(e.g. `addVertex(value, List<> edges)`) to add a vertex. The user will also be able
to call a delete method (e.g. `deleteVertex(vertexID)`) to remove vertexes.

Mime
View raw message