giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maria Stylianou <mars...@gmail.com>
Subject Re: InputFormat for the example SimpleMasterComputeVertex
Date Wed, 13 Feb 2013 17:52:04 GMT
Sorry, I didn't set the parameters -mc for the masterCompute and the -wc
for the WorkerContext in the execution command.

Now, I'm running
hadoop jar target/giraph-0.2-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleMasterComputeVertex
-vif org.apache.giraph.io.formats.*
JsonLongDoubleFloatDoubleVertexInputFormat* -vip
/user/hadoop/masterComputeInput
-of org.apache.giraph.io.formats.*
JsonLongDoubleFloatDoubleVertexOutputFormat* -op
/user/hadoop/masterComputeOutput
-w 3 *-mc org.apache.giraph.examples.SimpleMasterComputeVertex\$SimpleMasterCompute
-wc **
org.apache.giraph.examples.SimpleMasterComputeVertex\$SimpleMasterComputeWorkerContext
*
*
*
It works like a charm! :)

Now, I'm looking into the values that the aggregator gets. If I understand
correctly, in each superstep the aggregated value is: #_of_superstep/2+1.
So each vertex adds this aggregated value to its old value (initialized in
the file). And this is repeated in each superstep. Am I missing something?


On Wed, Feb 13, 2013 at 6:39 PM, Maja Kabiljo <majakabiljo@fb.com> wrote:

>  Hi Maria,
>
>  You also need to set the MasterCompute class, because that's where
> aggregators get registered (just like it's done in
> TstBspBasic.testBspMasterCompute where this vertex is used).
>
>  Maja
>
>   From: Claudio Martella <claudio.martella@gmail.com>
> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
> Date: Wednesday, February 13, 2013 6:56 AM
> To: "user@giraph.apache.org" <user@giraph.apache.org>
> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>
>   in the latest trunk, this is line 42:
>
>  double newValue= this.<DoubleWritable>getAggregatedValue(SMC_AGG).get();
>
>  What kind of input graph are you using?
>
>
> On Wed, Feb 13, 2013 at 1:25 PM, Maria Stylianou <marsty5@gmail.com>wrote:
>
>>  I didn't change anything in the code. I took the example as it is.
>> Actually, the whole error message is this one:
>>
>> java.lang.IllegalStateException: run: Caught an unrecoverable exception waitFor:
ExecutionException occurred while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@1ea326f9
>> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
>> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
>> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
>> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
>> 	at java.security.AccessController.doPrivileged(Native Method)
>> 	at javax.security.auth.Subject.doAs(Subject.java:415)
>> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>> 	at org.apache.hadoop.mapred.Child.main(Child.java:253)
>> Caused by: java.lang.IllegalStateException: waitFor: ExecutionException occurred
while waiting for org.apache.giraph.utils.ProgressableUtils$FutureWaitable@1ea326f9
>> 	at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:145)
>> 	at org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:105)
>> 	at org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:67)
>> 	at org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:723)
>> 	at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:270)
>> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
>> 	... 7 more
>> Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
>> 	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
>> 	at java.util.concurrent.FutureTask.get(FutureTask.java:119)
>> 	at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:232)
>> 	at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:137)
>> 	... 12 more
>> Caused by: java.lang.NullPointerException
>> 	at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(SimpleMasterComputeVertex.java:42)
>> 	at org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:204)
>> 	at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:149)
>> 	at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:63)
>> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> 	at java.lang.Thread.run(Thread.java:722)
>>
>>
>>
>>
>> On Wed, Feb 13, 2013 at 1:03 PM, Claudio Martella <
>> claudio.martella@gmail.com> wrote:
>>
>>> Are you sure that is the right line? I do not see how that would cause
>>> an exception, most probably a for loop on messages right after?
>>>
>>>
>>> On Wed, Feb 13, 2013 at 12:04 PM, Maria Stylianou <marsty5@gmail.com>wrote:
>>>
>>>> Hey,
>>>>
>>>>  I am trying to run the example SimpleMasterComputeVertex, but no
>>>> matter which Input Format and graph I give, it doesn't work. Each worker
>>>> gives the error:
>>>>
>>>> Caused by: java.lang.*NullPointerException*
>>>> 	at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>>>>
>>>> This line 42 is the first line of the compute()
>>>> public void compute(Iterable<DoubleWritable> messages){
>>>>
>>>>  So I guess, the initialization is not done correctly, because the
>>>> input file does not have the correct format.
>>>>
>>>>  Any help would be appreciated,
>>>> Thanks!
>>>>  Maria
>>>> --
>>>> Maria Stylianou
>>>> Intern at Telefonica, Barcelona, Spain
>>>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>>>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>>>  KTH Royal Institute of Technology, Stockholm, Sweden
>>>>
>>>>
>>>
>>>
>>>   --
>>>    Claudio Martella
>>>    claudio.martella@gmail.com
>>>
>>
>>
>>
>>  --
>> Maria Stylianou
>> Intern at Telefonica, Barcelona, Spain
>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>> Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>  KTH Royal Institute of Technology, Stockholm, Sweden
>>
>
>
>
>  --
>    Claudio Martella
>    claudio.martella@gmail.com
>



-- 
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed
Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
Universitat Politècnica de Catalunya - BarcelonaTech, Barcelona, Spain
KTH Royal Institute of Technology, Stockholm, Sweden
Phone: +34 622 211 885

Mime
View raw message