giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: Trying to implement program to find betweenness centrality in giraph
Date Thu, 07 Feb 2013 16:37:11 GMT
Just to clarify. It is not due to pregel. The number of iterations of BFS
is bound by the longest shortest path (the diameter). If you know the
diameter and the number of random sources, you can compute the total number
of supersteps and hence counters.

Hope this helps.

Claudio

On Thursday, February 7, 2013, Jan van der Lugt wrote:

> I never actually ran bc_random, so I don't know how many supersteps it
> needs for different input sizes. If you could try this out, it would be
> worthwhile to know. Especially the relation between input size / number of
> supersteps. It's a limitation of the Pregel paradigm that you need quite a
> few superstep to do BFS, not a lot we can do about this...
>
> On Thu, Feb 7, 2013 at 3:47 PM, Claudio Martella <
> claudio.martella@gmail.com> wrote:
>
> Hi,
>
> you have to increase the counters limit in your cluster. You can increase
> the limit by setting the property mapreduce.job.counters.limit in your
> mapred-site.xml accordingly.
>
>
> On Thu, Feb 7, 2013 at 4:34 PM, pradeep kumar <pradeep0802@gmail.com>wrote:
>
> Hi Jan,
>
> well actually i just corrected my mistake.. i was running it wrong way,
> thanks..
> just 1 more question on map-reduce counters
>
> giraph by default limits the superstep counters to 120
> but in bc_random master superstep itself exceeds this, so i had to
> increase the limit..this was just for 20 nodes sample i/p i was trying
>
> my real data will be close to million nodes,  so how can i manage for such
> large input..
>
>
>
>
> On Thu, Feb 7, 2013 at 4:02 PM, pradeep kumar <pradeep0802@gmail.com>wrote:
>
> Hi lugt,
>
> Thanks for all your help.. program compiled successfully into jar..
> but actually i am not able to run programs..
> i tried following cmd to run
>
> 1) using GiraphRunner
> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner org.apache.giraph.examples.bc_randomVertex
>  -vif org.apache.giraph.examples.bc_randomVertexInputFormat -vip inputbet
>  -of org.apache.giraph.examples.bc_randomVertexOutputFormat -op outputbet
> -w 1
>
> it does maps every thing successfully but then gives following error in log
>
> java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor: ExecutionException
occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@40fd6cd8
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
>
> followed by some other errors
>
> 2) bc_random
>
> i actually couldnt understand all arg types here
>
> used :
>
> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar org.apache.giraph.examples.bc_random
org.apache.giraph.examples.bc_randomVertex --GMInputFormat ADJ  org.apache.giraph.examples.bc_randomVertexInputFormat
-i inputbet  --GMOutputFormat  ADJ org.apache.giraph.examples.bc_randomVertexOutputFormat
-o outputbet -w 1
>
>
>
>
> but gives error for k procedure arg (not sure what to give as value here, even about
about ip format i mentioned above  )
>
>
> I have uploaded sample input in hdfs in following format
>
> 10 4500 9 800 11 1000 12 1000
> 11 5500 12 1100 11 800
>
> nid val <eid val>
>
> did i made a mistake anywhere..?
>
> any suggestion will be a great help..!!
>
>
>
>
>
> On Wed, Feb 6, 2013 at 5:03 PM, Jan van der Lugt <janlugt@gmail.com>wrote:
>
> Hi Pradeep,
>
> You can check out the Green-Marl compiler from my Github (
> https://github.com/janlugt/Green-Marl), that one should be compatible
> with t
>
>

-- 
   Claudio Martella
   claudio.martella@gmail.com

Mime
View raw message