giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: Trying to implement program to find betweenness centrality in giraph
Date Thu, 07 Feb 2013 15:47:23 GMT
Hi,

you have to increase the counters limit in your cluster. You can increase
the limit by setting the property mapreduce.job.counters.limit in your
mapred-site.xml accordingly.


On Thu, Feb 7, 2013 at 4:34 PM, pradeep kumar <pradeep0802@gmail.com> wrote:

> Hi Jan,
>
> well actually i just corrected my mistake.. i was running it wrong way,
> thanks..
> just 1 more question on map-reduce counters
>
> giraph by default limits the superstep counters to 120
> but in bc_random master superstep itself exceeds this, so i had to
> increase the limit..this was just for 20 nodes sample i/p i was trying
>
> my real data will be close to million nodes,  so how can i manage for such
> large input..
>
>
>
>
> On Thu, Feb 7, 2013 at 4:02 PM, pradeep kumar <pradeep0802@gmail.com>wrote:
>
>> Hi lugt,
>>
>> Thanks for all your help.. program compiled successfully into jar..
>> but actually i am not able to run programs..
>> i tried following cmd to run
>>
>> 1) using GiraphRunner
>> hadoop jar
>> giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner org.apache.giraph.examples.bc_randomVertex
>>  -vif org.apache.giraph.examples.bc_randomVertexInputFormat -vip inputbet
>>  -of org.apache.giraph.examples.bc_randomVertexOutputFormat -op outputbet
>> -w 1
>>
>> it does maps every thing successfully but then gives following error in
>> log
>>
>> java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor:
ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@40fd6cd8
>> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
>> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
>> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
>> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
>> 	at java.security.AccessController.doPrivileged(Native Method)
>> 	at javax.security.auth.Subject.doAs(Subject.java:415)
>>
>> followed by some other errors
>>
>> 2) bc_random
>>
>> i actually couldnt understand all arg types here
>>
>> used :
>>
>> hadoop jar giraph-0.2-SNAPSHOT-for-hadoop-0.20.2-jar-with-dependencies.jar org.apache.giraph.examples.bc_random
org.apache.giraph.examples.bc_randomVertex --GMInputFormat ADJ  org.apache.giraph.examples.bc_randomVertexInputFormat
-i inputbet  --GMOutputFormat  ADJ org.apache.giraph.examples.bc_randomVertexOutputFormat
-o outputbet -w 1
>>
>>
>> but gives error for k procedure arg (not sure what to give as value here, even about
about ip format i mentioned above  )
>>
>>
>> I have uploaded sample input in hdfs in following format
>>
>> 10 4500 9 800 11 1000 12 1000
>> 11 5500 12 1100 11 800
>>
>> nid val <eid val>
>>
>> did i made a mistake anywhere..?
>>
>> any suggestion will be a great help..!!
>>
>>
>>
>>
>>
>> On Wed, Feb 6, 2013 at 5:03 PM, Jan van der Lugt <janlugt@gmail.com>wrote:
>>
>>> Hi Pradeep,
>>>
>>> You can check out the Green-Marl compiler from my Github (
>>> https://github.com/janlugt/Green-Marl), that one should be compatible
>>> with the latest version of Giraph. Please let me know if anything doesn't
>>> work for you. The changes will probably also be accepted upstream later
>>> this day.
>>>
>>> - Jan
>>>
>>>
>>> On Wed, Feb 6, 2013 at 9:48 AM, Jan van der Lugt <janlugt@gmail.com>wrote:
>>>
>>>> It should work with version 0.2, but let me check. Since the API is
>>>> changing all the time at the moment, it's very different to track all those
>>>> changes. Probably they are numerous instances of the same error.
>>>>
>>>>
>>>> On Wed, Feb 6, 2013 at 9:26 AM, pradeep kumar <pradeep0802@gmail.com>wrote:
>>>>
>>>>> Hi Lugt,
>>>>>
>>>>> Thanks for suggestion, i just tried with Green-Marl's bc_random,
>>>>> But because of so many changes in Giraph codebase, imports and many
>>>>> other functions in bc_random
>>>>> generate lots of errors almost 100+ errors.. I tried with Giraph 1.0
>>>>> and current version of 2.0,
>>>>> most of errors i have resolved but still there are many errors..
>>>>> Any suggestion on this..?
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Pradeep Kumar
>>
>
>
>
> --
> Pradeep Kumar
>



-- 
   Claudio Martella
   claudio.martella@gmail.com

Mime
View raw message