giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wang Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-652) Not support only with edge input data in some algorithms.
Date Mon, 06 May 2013 02:56:16 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13649489#comment-13649489
] 

Wang Xu commented on GIRAPH-652:
--------------------------------

The vertices are created automatically ?

In ShortestPath example, my input data is :(just edge input data)

1\t\3\11
1\t\5\12

Before modify i run:

hadoop jar giraph-examples-1.0.0-for-hadoop-2.0.0-cdh4.1.2-jar-with-dependencies.jar org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsVertex -eif org.apache.giraph.io.formats.LongFloatTextEdgeInputFormat
-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat  -eip /shortestPathsInputGraph
-op /output-tmp/sp-37 -w 1 -ca SimpleShortestPathsVertex.sourceId=1 -ca giraph.zkList=dlxa107:2181

Result is:( hadoop fs -cat /output-tmp/sp-37/*)
5	0.0
1	0.0
3	0.0

After modify i run:

hadoop jar giraph-examples-1.0.0-for-hadoop-2.0.0-cdh4.1.2-jar-with-dependencies.jar org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsVertex -eif org.apache.giraph.io.formats.LongFloatTextEdgeInputFormat
-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat  -eip /shortestPathsInputGraph
-op /output-tmp/sp-38 -w 1 -ca SimpleShortestPathsVertex.sourceId=1 -ca giraph.zkList=dlxa107:2181

Result is:( hadoop fs -cat /output-tmp/sp-38/*)
5	12.0
1	0.0
3	11.0

Why is it ?
                
> Not support only with edge input data in some algorithms.
> ---------------------------------------------------------
>
>                 Key: GIRAPH-652
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-652
>             Project: Giraph
>          Issue Type: Wish
>          Components: graph
>    Affects Versions: 1.0.0
>            Reporter: Wang Xu
>            Priority: Minor
>             Fix For: 1.0.0
>
>
> There are two types of input vertex-centric and edge-centric. Some algorithms need vertex
centric data, such as pairs of vertex ids and initial page ranks. Some algorithms only look
at edges. For example, connected components can be run without any vertex values.
> In some conditions , we have only edge-centric data , have not vertex data .  Convert
edge-centric data to vertex-centric data is not bad, but is not better.
> In my job , I modify EdgeInputSplitsCallable.java to support load vertex auto when loading
edge。
> My mainly modify is:
> {code:java} 
>       graphState.getWorkerClientRequestProcessor().sendEdgeRequest(
>           sourceId, readerEdge);
>       context.progress(); // do this before potential data transfer
>       ++inputSplitEdgesLoaded;
>       
>       //modify begin
>       Vertex<I, V, E, M> vertex = this.configuration.createVertex();      
>       if (vertex.getValue() == null) {
>     	  vertex.setValue(configuration.createVertexValue());
>         }
>       vertex.setConf(configuration);
>       vertex.setGraphState(graphState);      
>       vertex.initialize(edgeReader.getCurrentEdge().getTargetVertexId(), vertex.getValue());
>       
>       PartitionOwner partitionOwner = 
>           bspServiceWorker.getVertexPartitionOwner(vertex.getId());
>       graphState.getWorkerClientRequestProcessor().sendVertexRequest(
>           partitionOwner, vertex);      
>       // modify end
>       // Update status every VERTICES_UPDATE_PERIOD edges
>       if (inputSplitEdgesLoaded % VERTICES_UPDATE_PERIOD == 0) {
>         totalEdgesMeter.mark(VERTICES_UPDATE_PERIOD); 
> {code} 
> After do it , my giraph job can only load edge data in any examples。
> I wish a similar features can be add into in future version。

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message