giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jyoti Yadav <rao.jyoti26ya...@gmail.com>
Subject Re: About vertex input format..
Date Sat, 16 Nov 2013 16:02:24 GMT
Thanks a lot Ameya....Your ideas really helped me a lot..I got my program
executed...
Nice job..:)


On Sat, Nov 16, 2013 at 1:11 AM, Ameya Vilankar <ameya.vilankar@gmail.com>wrote:

> No, I said you should not be bothered about the Message Type in your
> Vertex Input Format Class. It only needs your Vertex id type, Vertex Value
> type and the Edge Value type. It does not need the type of messages you are
> sending.
>
> You can put the VertexInputFormat File anywhere you want. Preferably keep
> it in the same location as you Computation Class. I know that in giraph,
> all the input formats are in the io.formats package in giraph-core. You can
> keep it there if you want and it will work in your case, since you vertex
> value is just LongWritable. But if you are writing a more complex giraph
> program where you vertex value stores all kinds of information and you hava
> a custom Writable Class. Then you would have to keep your vertex value
> class somewhere in the giraph-core package just so that your InputFormat
> class that is in the io package has access to it. So its best if you keep
> your vertex input format class in the same directory as your Computation
> Class and your Vertex value class.
> Have look at the BrachaToeugComputation Example in giraph-examples. It
> defines its own io formats in giraph-examples.
>
>
> On Wed, Nov 13, 2013 at 9:56 PM, Jyoti Yadav <rao.jyoti26yadav@gmail.com>wrote:
>
>> Thanks Ameya..
>> I am just newbie to giraph... Would you please let me  know  the path
>> where should i include this file ,so that i can compile it.
>>
>> Also you mentioned that i need not to bothered about
>> VertexInputFormat,then while giving the jar command to execute the program
>> ,what about -vif option?? Should i ignore it??
>>
>> Thanks
>> Jyoti
>>
>>
>> On Thu, Nov 14, 2013 at 12:20 AM, Ameya Vilankar <
>> ameya.vilankar@gmail.com> wrote:
>>
>>> Here is class that you need. I haven't tested it. You don't need to
>>> worry about the message type in InputFormatClass. This code compiled with
>>> the most updated trunk branch of giraph.
>>>
>>>
>>> import com.google.common.collect.Lists;
>>> import org.apache.giraph.edge.Edge;
>>> import org.apache.giraph.edge.EdgeFactory;
>>> import org.apache.giraph.graph.Vertex;
>>> import org.apache.giraph.io.formats.TextVertexInputFormat;
>>> import org.apache.hadoop.io.LongWritable;
>>> import org.apache.hadoop.io.Text;
>>> import org.apache.hadoop.mapreduce.InputSplit;
>>> import org.apache.hadoop.mapreduce.TaskAttemptContext;
>>> import org.json.JSONArray;
>>> import org.json.JSONException;
>>>
>>> import java.io.IOException;
>>> import java.util.List;
>>>
>>> public class ShortestPathVertexInputFormat extends
>>>         TextVertexInputFormat<LongWritable, LongWritable, LongWritable>
{
>>>
>>>     @Override
>>>     public TextVertexReader createVertexReader(InputSplit split,
>>>                                                TaskAttemptContext
>>> context) {
>>>         return new ShortestPathVertexReader();
>>>     }
>>>
>>>     class ShortestPathVertexReader
>>>              extends
>>> TextVertexReaderFromEachLineProcessedHandlingExceptions<JSONArray,
>>>             JSONException> {
>>>
>>>         @Override
>>>         protected JSONArray preprocessLine(Text line) throws
>>> JSONException {
>>>             return new JSONArray(line.toString());
>>>         }
>>>
>>>         @Override
>>>         protected LongWritable getId(JSONArray jsonVertex) throws
>>> JSONException,
>>>                 IOException {
>>>
>>>             // System.out.println("Got Vertex Id: " +
>>> jsonVertex.getLong(0));
>>>             return new LongWritable(jsonVertex.getLong(0));
>>>         }
>>>
>>>         @Override
>>>         protected LongWritable getValue(JSONArray jsonVertex)
>>>                 throws JSONException, IOException {
>>>
>>>             return new LongWritable(0L);
>>>         }
>>>
>>>         @Override
>>>         protected Iterable<Edge<LongWritable, LongWritable>>
>>>         getEdges(JSONArray jsonVertex) throws JSONException, IOException
>>> {
>>>
>>>             // Get the Edge array
>>>             JSONArray jsonEdgeArray = jsonVertex.getJSONArray(2);
>>>
>>>             List<Edge<LongWritable, LongWritable>> edges =
>>>
>>> Lists.newArrayListWithCapacity(jsonEdgeArray.length());
>>>
>>>             // Get the indiviudal edges from the edge array
>>>             for (int i = 0; i < jsonEdgeArray.length(); ++i) {
>>>                 JSONArray jsonEdge = jsonEdgeArray.getJSONArray(i);
>>>                 edges.add(EdgeFactory.create(new
>>> LongWritable(jsonEdge.getLong(0)), new LongWritable(jsonEdge.getLong(1))));
>>>             }
>>>
>>>             //System.out.println("Got The Edges for the Vertex: " +
>>> jsonVertex.getLong(0));
>>>             return edges;
>>>         }
>>>
>>>         @Override
>>>         protected Vertex<LongWritable, LongWritable,
>>>                 LongWritable> handleException(Text line, JSONArray
>>> jsonVertex,
>>>                                               JSONException e) {
>>>
>>>             throw new IllegalArgumentException(
>>>                     "Couldn't get vertex from line " + line, e);
>>>         }
>>>     }
>>> }
>>>
>>>
>>> On Wed, Nov 13, 2013 at 7:20 AM, Jyoti Yadav <rao.jyoti26yadav@gmail.com
>>> > wrote:
>>>
>>>> Hi..
>>>> I am trying to execute SingleSourceShortestPath example with some
>>>> changes in this.. While sending message,i am sending two values-(sender
>>>> vertex value+edge weight),sender_id..
>>>> For this message to send,i created my own message file
>>>> MyMessageWritable.java.
>>>> The graph input is given in following form..
>>>>
>>>> [0,0,[[1,1],[3,3]]][1,0,[[0,1],[2,2],[3,1]]][2,0,[[1,2],[4,4]]][3,0,[[0,3],[1,1],[4,4]]][4,0,[[3,4],[2,4]]]
>>>>
>>>>
>>>> here vertex id is -long
>>>>
>>>> vertex value-long
>>>>
>>>> edge value-long
>>>>
>>>> Message is (long,long)
>>>>
>>>> what should be the vertex input format for this??
>>>>
>>>> Any help is really appreciated...
>>>>
>>>>
>>>>
>>>>
>>>> Regards
>>>>
>>>> Jyoti
>>>>
>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message