Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EA3F710C94 for ; Thu, 14 Nov 2013 05:56:48 +0000 (UTC) Received: (qmail 24666 invoked by uid 500); 14 Nov 2013 05:56:48 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 24597 invoked by uid 500); 14 Nov 2013 05:56:40 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 24589 invoked by uid 99); 14 Nov 2013 05:56:39 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 14 Nov 2013 05:56:39 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of rao.jyoti26yadav@gmail.com designates 209.85.212.177 as permitted sender) Received: from [209.85.212.177] (HELO mail-wi0-f177.google.com) (209.85.212.177) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 14 Nov 2013 05:56:34 +0000 Received: by mail-wi0-f177.google.com with SMTP id hq4so1857496wib.10 for ; Wed, 13 Nov 2013 21:56:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=5TPnmP8FpJUJ79GKb8n/R/tyRJoAYO7BUk5tbbAPtRA=; b=S9CepIf4d9t/+pYSFa5i5YkbR5LP+8xP+9kWtwiN6ivrsH0blp7JO4icDN0aV6WLhJ 02sMpc9bpSAoOThwHdzHZCFz6BxY+wi3GTwP/SacJOi2mXmAFcnjEDpVd/PidJvSWFII dhb3Xc/vi+zG3xRKoBRgfskusX8BjKCBU0aW77I6Crr4+qgtnxDkLlxyLQ9pJfFMBXHb HNr5hStlA0tM8V+6Y+DTv8rpUmYp6BxU6WYVcPxyzb47vUNCNfXzQ3fOdNmch0ZNLBDO Lv4AUi00FknD4fnjHF4yzKJi+o+V0mF4Xs4BqYBWJTBKWQVd6eJNNUaMZxdPuDOziDoO 6tMg== MIME-Version: 1.0 X-Received: by 10.180.184.14 with SMTP id eq14mr573276wic.56.1384408571834; Wed, 13 Nov 2013 21:56:11 -0800 (PST) Received: by 10.194.234.33 with HTTP; Wed, 13 Nov 2013 21:56:11 -0800 (PST) In-Reply-To: References: Date: Thu, 14 Nov 2013 11:26:11 +0530 Message-ID: Subject: Re: About vertex input format.. From: Jyoti Yadav To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=001a11c2436e25f80f04eb1cbd50 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c2436e25f80f04eb1cbd50 Content-Type: text/plain; charset=ISO-8859-1 Thanks Ameya.. I am just newbie to giraph... Would you please let me know the path where should i include this file ,so that i can compile it. Also you mentioned that i need not to bothered about VertexInputFormat,then while giving the jar command to execute the program ,what about -vif option?? Should i ignore it?? Thanks Jyoti On Thu, Nov 14, 2013 at 12:20 AM, Ameya Vilankar wrote: > Here is class that you need. I haven't tested it. You don't need to worry > about the message type in InputFormatClass. This code compiled with the > most updated trunk branch of giraph. > > > import com.google.common.collect.Lists; > import org.apache.giraph.edge.Edge; > import org.apache.giraph.edge.EdgeFactory; > import org.apache.giraph.graph.Vertex; > import org.apache.giraph.io.formats.TextVertexInputFormat; > import org.apache.hadoop.io.LongWritable; > import org.apache.hadoop.io.Text; > import org.apache.hadoop.mapreduce.InputSplit; > import org.apache.hadoop.mapreduce.TaskAttemptContext; > import org.json.JSONArray; > import org.json.JSONException; > > import java.io.IOException; > import java.util.List; > > public class ShortestPathVertexInputFormat extends > TextVertexInputFormat { > > @Override > public TextVertexReader createVertexReader(InputSplit split, > TaskAttemptContext context) > { > return new ShortestPathVertexReader(); > } > > class ShortestPathVertexReader > extends > TextVertexReaderFromEachLineProcessedHandlingExceptions JSONException> { > > @Override > protected JSONArray preprocessLine(Text line) throws JSONException > { > return new JSONArray(line.toString()); > } > > @Override > protected LongWritable getId(JSONArray jsonVertex) throws > JSONException, > IOException { > > // System.out.println("Got Vertex Id: " + > jsonVertex.getLong(0)); > return new LongWritable(jsonVertex.getLong(0)); > } > > @Override > protected LongWritable getValue(JSONArray jsonVertex) > throws JSONException, IOException { > > return new LongWritable(0L); > } > > @Override > protected Iterable> > getEdges(JSONArray jsonVertex) throws JSONException, IOException { > > // Get the Edge array > JSONArray jsonEdgeArray = jsonVertex.getJSONArray(2); > > List> edges = > Lists.newArrayListWithCapacity(jsonEdgeArray.length()); > > // Get the indiviudal edges from the edge array > for (int i = 0; i < jsonEdgeArray.length(); ++i) { > JSONArray jsonEdge = jsonEdgeArray.getJSONArray(i); > edges.add(EdgeFactory.create(new > LongWritable(jsonEdge.getLong(0)), new LongWritable(jsonEdge.getLong(1)))); > } > > //System.out.println("Got The Edges for the Vertex: " + > jsonVertex.getLong(0)); > return edges; > } > > @Override > protected Vertex LongWritable> handleException(Text line, JSONArray > jsonVertex, > JSONException e) { > > throw new IllegalArgumentException( > "Couldn't get vertex from line " + line, e); > } > } > } > > > On Wed, Nov 13, 2013 at 7:20 AM, Jyoti Yadav wrote: > >> Hi.. >> I am trying to execute SingleSourceShortestPath example with some changes >> in this.. While sending message,i am sending two values-(sender vertex >> value+edge weight),sender_id.. >> For this message to send,i created my own message file >> MyMessageWritable.java. >> The graph input is given in following form.. >> >> [0,0,[[1,1],[3,3]]][1,0,[[0,1],[2,2],[3,1]]][2,0,[[1,2],[4,4]]][3,0,[[0,3],[1,1],[4,4]]][4,0,[[3,4],[2,4]]] >> >> >> here vertex id is -long >> >> vertex value-long >> >> edge value-long >> >> Message is (long,long) >> >> what should be the vertex input format for this?? >> >> Any help is really appreciated... >> >> >> Regards >> >> Jyoti >> >> >> > --001a11c2436e25f80f04eb1cbd50 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Thanks Ameya..
I am just newb= ie to giraph... Would you please let me=A0 know=A0 the path where should i = include this file ,so that i can compile it.

Also you mention= ed that i need not to bothered about VertexInputFormat,then while giving th= e jar command to execute the program ,what about -vif option?? Should i ign= ore it??

Thanks
Jyoti

<= br>
On Thu, Nov 14, 2013 at 12:20 AM, Ameya Vilan= kar <ameya.vilankar@gmail.com> wrote:
Here is class that you= need. I haven't tested it. You don't need to worry about the messa= ge type in InputFormatClass. This code compiled with the most updated trunk= branch of giraph.


import com.google.common.collect.Lists;
=
import org.apache.giraph.edge.Edge;
import org.apache.giraph= .edge.EdgeFactory;
import org.apache.giraph.graph.Vertex;
import org.apache.giraph.io.formats.TextVertexInputFormat;
i= mport org.apache.hadoop.io.LongWritable;
import org.apache.hadoop= .io.Text;
import org.apache.hadoop.mapreduce.InputSplit;
import org.apache.hadoop.mapreduce.TaskAttemptContext;
impor= t org.json.JSONArray;
import org.json.JSONException;
import java.io.IOException;
import java.util.List;

public class ShortestPathVertexInputFormat extends
=A0 =A0 =A0 =A0 TextVertexInputFormat<LongWritable, LongWritable, = LongWritable> {

=A0 =A0 @Override
=A0= =A0 public TextVertexReader createVertexReader(InputSplit split,
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0TaskAttemptContext context) {
=A0 =A0 = =A0 =A0 return new ShortestPathVertexReader();
=A0 =A0 }

=A0 =A0 class ShortestPathVertexReader
=A0 =A0 =A0 =A0 =A0 =A0 extends TextVertexReaderFromEachLineProcessedHandli= ngExceptions<JSONArray,
=A0 =A0 =A0 =A0 =A0 =A0 JSONException&= gt; {

=A0 =A0 =A0 =A0 @Override
=A0 =A0 = =A0 =A0 protected JSONArray preprocessLine(Text line) throws JSONException = {
=A0 =A0 =A0 =A0 =A0 =A0 return new JSONArray(line.toString());
=A0 =A0 =A0 =A0 }

=A0 =A0 =A0 =A0 @Override
=A0 =A0 =A0 =A0 protected LongWritable getId(JSONArray jsonVertex) t= hrows JSONException,
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 IOException {

=A0 =A0 =A0 =A0 =A0 =A0 // System.out.println("Got= Vertex Id: " + jsonVertex.getLong(0));
=A0 =A0 =A0 =A0 =A0 = =A0 return new LongWritable(jsonVertex.getLong(0));
=A0 =A0 =A0 = =A0 }

=A0 =A0 =A0 =A0 @Override
=A0 =A0 =A0 =A0 protected LongWritable = getValue(JSONArray jsonVertex)
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 th= rows JSONException, IOException {

=A0 =A0 =A0 =A0 = =A0 =A0 return new LongWritable(0L);
=A0 =A0 =A0 =A0 }

=A0 =A0 =A0 =A0 @Override
=A0 =A0 =A0 =A0 protected Iterable<Edge<LongWritable, LongWr= itable>>
=A0 =A0 =A0 =A0 getEdges(JSONArray jsonVertex) thr= ows JSONException, IOException {

=A0 =A0 =A0 =A0 =A0 =A0 // Get the Edge array
=A0 =A0 =A0 =A0 =A0 =A0 JSONArray jsonEdgeArray =3D jsonVertex.getJSONArra= y(2);

=A0 =A0 =A0 =A0 =A0 =A0 List<Edge<Long= Writable, LongWritable>> edges =3D
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Lists.newArrayListWithCapacity= (jsonEdgeArray.length());

=A0 =A0 =A0 =A0 =A0 =A0 = // Get the indiviudal edges from the edge array
=A0 =A0 =A0 =A0 = =A0 =A0 for (int i =3D 0; i < jsonEdgeArray.length(); ++i) {
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 JSONArray jsonEdge =3D jsonEdgeArray.g= etJSONArray(i);
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 edges.add(EdgeFac= tory.create(new LongWritable(jsonEdge.getLong(0)), new LongWritable(jsonEdg= e.getLong(1))));
=A0 =A0 =A0 =A0 =A0 =A0 }

=A0 =A0 =A0 =A0 =A0 =A0 //System.out.println("Got = The Edges for the Vertex: " + jsonVertex.getLong(0));
=A0 = =A0 =A0 =A0 =A0 =A0 return edges;
=A0 =A0 =A0 =A0 }
=A0 =A0 =A0 =A0 @Override
=A0 =A0 =A0 =A0 protected Vertex<LongWritable, LongWritable,
<= div>=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 LongWritable> handleException(Text l= ine, JSONArray jsonVertex,
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 JSONException e) {<= /div>

=A0 =A0 =A0 =A0 =A0 =A0 throw new IllegalArgumentExcept= ion(
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 "Couldn't g= et vertex from line " + line, e);
=A0 =A0 =A0 =A0 }
=A0 =A0 }
}


On Wed, Nov 13, 2013 at 7:20 AM, Jyoti Y= adav <rao.jyoti26yadav@gmail.com> wrote:
Hi..
I a= m trying to execute SingleSourceShortestPath example with some changes in t= his.. While sending message,i am sending two values-(sender vertex value+ed= ge weight),sender_id..
For this messag= e to send,i created my own message file MyMessageWritable.java.
<= /div>The graph input is give= n in following form..

[0,0,[[1,1],[3,3]]]
[1,0,[[0,1],[2,2],[3,1]]]
[2,0,[[1,2],[4,4]]]
[3,0,[[0,3],[1,1],[4,4]]]
[4,0,[[3,4],[2,4]]]



here vertex id is -long
vertex value-long


edge value-long
=
Message is (long,long)


what should be the vertex input format for this??

Any help is really appreciated...




Regards
Jyoti



--001a11c2436e25f80f04eb1cbd50--