giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ameya Vilankar <ameya.vilan...@gmail.com>
Subject Re: About vertex input format..
Date Fri, 15 Nov 2013 19:41:16 GMT
No, I said you should not be bothered about the Message Type in your Vertex
Input Format Class. It only needs your Vertex id type, Vertex Value type
and the Edge Value type. It does not need the type of messages you are
sending.

You can put the VertexInputFormat File anywhere you want. Preferably keep
it in the same location as you Computation Class. I know that in giraph,
all the input formats are in the io.formats package in giraph-core. You can
keep it there if you want and it will work in your case, since you vertex
value is just LongWritable. But if you are writing a more complex giraph
program where you vertex value stores all kinds of information and you hava
a custom Writable Class. Then you would have to keep your vertex value
class somewhere in the giraph-core package just so that your InputFormat
class that is in the io package has access to it. So its best if you keep
your vertex input format class in the same directory as your Computation
Class and your Vertex value class.
Have look at the BrachaToeugComputation Example in giraph-examples. It
defines its own io formats in giraph-examples.


On Wed, Nov 13, 2013 at 9:56 PM, Jyoti Yadav <rao.jyoti26yadav@gmail.com>wrote:

> Thanks Ameya..
> I am just newbie to giraph... Would you please let me  know  the path
> where should i include this file ,so that i can compile it.
>
> Also you mentioned that i need not to bothered about
> VertexInputFormat,then while giving the jar command to execute the program
> ,what about -vif option?? Should i ignore it??
>
> Thanks
> Jyoti
>
>
> On Thu, Nov 14, 2013 at 12:20 AM, Ameya Vilankar <ameya.vilankar@gmail.com
> > wrote:
>
>> Here is class that you need. I haven't tested it. You don't need to worry
>> about the message type in InputFormatClass. This code compiled with the
>> most updated trunk branch of giraph.
>>
>>
>> import com.google.common.collect.Lists;
>> import org.apache.giraph.edge.Edge;
>> import org.apache.giraph.edge.EdgeFactory;
>> import org.apache.giraph.graph.Vertex;
>> import org.apache.giraph.io.formats.TextVertexInputFormat;
>> import org.apache.hadoop.io.LongWritable;
>> import org.apache.hadoop.io.Text;
>> import org.apache.hadoop.mapreduce.InputSplit;
>> import org.apache.hadoop.mapreduce.TaskAttemptContext;
>> import org.json.JSONArray;
>> import org.json.JSONException;
>>
>> import java.io.IOException;
>> import java.util.List;
>>
>> public class ShortestPathVertexInputFormat extends
>>         TextVertexInputFormat<LongWritable, LongWritable, LongWritable> {
>>
>>     @Override
>>     public TextVertexReader createVertexReader(InputSplit split,
>>                                                TaskAttemptContext
>> context) {
>>         return new ShortestPathVertexReader();
>>     }
>>
>>     class ShortestPathVertexReader
>>              extends
>> TextVertexReaderFromEachLineProcessedHandlingExceptions<JSONArray,
>>             JSONException> {
>>
>>         @Override
>>         protected JSONArray preprocessLine(Text line) throws
>> JSONException {
>>             return new JSONArray(line.toString());
>>         }
>>
>>         @Override
>>         protected LongWritable getId(JSONArray jsonVertex) throws
>> JSONException,
>>                 IOException {
>>
>>             // System.out.println("Got Vertex Id: " +
>> jsonVertex.getLong(0));
>>             return new LongWritable(jsonVertex.getLong(0));
>>         }
>>
>>         @Override
>>         protected LongWritable getValue(JSONArray jsonVertex)
>>                 throws JSONException, IOException {
>>
>>             return new LongWritable(0L);
>>         }
>>
>>         @Override
>>         protected Iterable<Edge<LongWritable, LongWritable>>
>>         getEdges(JSONArray jsonVertex) throws JSONException, IOException {
>>
>>             // Get the Edge array
>>             JSONArray jsonEdgeArray = jsonVertex.getJSONArray(2);
>>
>>             List<Edge<LongWritable, LongWritable>> edges =
>>
>> Lists.newArrayListWithCapacity(jsonEdgeArray.length());
>>
>>             // Get the indiviudal edges from the edge array
>>             for (int i = 0; i < jsonEdgeArray.length(); ++i) {
>>                 JSONArray jsonEdge = jsonEdgeArray.getJSONArray(i);
>>                 edges.add(EdgeFactory.create(new
>> LongWritable(jsonEdge.getLong(0)), new LongWritable(jsonEdge.getLong(1))));
>>             }
>>
>>             //System.out.println("Got The Edges for the Vertex: " +
>> jsonVertex.getLong(0));
>>             return edges;
>>         }
>>
>>         @Override
>>         protected Vertex<LongWritable, LongWritable,
>>                 LongWritable> handleException(Text line, JSONArray
>> jsonVertex,
>>                                               JSONException e) {
>>
>>             throw new IllegalArgumentException(
>>                     "Couldn't get vertex from line " + line, e);
>>         }
>>     }
>> }
>>
>>
>> On Wed, Nov 13, 2013 at 7:20 AM, Jyoti Yadav <rao.jyoti26yadav@gmail.com>wrote:
>>
>>> Hi..
>>> I am trying to execute SingleSourceShortestPath example with some
>>> changes in this.. While sending message,i am sending two values-(sender
>>> vertex value+edge weight),sender_id..
>>> For this message to send,i created my own message file
>>> MyMessageWritable.java.
>>> The graph input is given in following form..
>>>
>>> [0,0,[[1,1],[3,3]]][1,0,[[0,1],[2,2],[3,1]]][2,0,[[1,2],[4,4]]][3,0,[[0,3],[1,1],[4,4]]][4,0,[[3,4],[2,4]]]
>>>
>>>
>>> here vertex id is -long
>>>
>>> vertex value-long
>>>
>>> edge value-long
>>>
>>> Message is (long,long)
>>>
>>> what should be the vertex input format for this??
>>>
>>> Any help is really appreciated...
>>>
>>>
>>>
>>> Regards
>>>
>>> Jyoti
>>>
>>>
>>>
>>
>

Mime
View raw message