giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sertu─č Kaya <sertug.k...@agmlab.com>
Subject Re: Null Pointer Exception when running a job
Date Sat, 09 Nov 2013 20:58:07 GMT
Hello Artie;
I also had such problems when working on large datasets. It probably 
needs a better hadoop/mapreduce configuration. For example did you try 
increasing the memory of task tracker child processes in mapred-site.xml 
such as:
> <property>
>     <name>mapred.child.java.opts</name>
>     <value>-Xmx3000m</value>
>   </property>
Regards;
Sertug

On 08-11-2013 01:26, Artie Pesh-Imam wrote:
> Hey
>
> It does work against a smaller data set and Im running 1.0.0 for hadoop 2.
>
>
> -- 
> Artie Pesh-Imam
> Software Engineer @ Tapad
> 201-253-9647
> ------------------------------------------------------------------------
> From: Mohammad Islam Mohammad Islam <mailto:mislam77@yahoo.com>
> Reply: user@giraph.apache.org user@giraph.apache.org 
> <mailto:user@giraph.apache.org>, Mohammad Islam mislam77@yahoo.com 
> <mailto:mislam77@yahoo.com>
> Date: November 7, 2013 at 6:20:53 PM
> To: user@giraph.apache.org user@giraph.apache.org 
> <mailto:user@giraph.apache.org>
> Subject: Re: Null Pointer Exception when running a job
>> Hi Artie,
>> Hopefully someone would give you a concrete answer.
>>
>> My rambling comments:
>> Did you try with a smaller number of vertices? I want to know if the 
>> problem is due to increased load.
>> What version of Giraph are you using?
>>
>> Regards,
>> Mohammad
>>
>>
>> On Thursday, November 7, 2013 2:10 PM, Artie Pesh-Imam 
>> <artie.pesh-imam@tapad.com> wrote:
>> Please let me know if any additional context would be useful. With no 
>> detail in the error message, I have no idea what the root issue could be.
>>
>> -- 
>> Artie Pesh-Imam
>> Software Engineer @ Tapad
>> 201-253-9647
>> ------------------------------------------------------------------------
>> From: Artie Pesh-Imam Artie Pesh-Imam <mailto:artie.pesh-imam@tapad.com>
>> Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com 
>> <mailto:artie.pesh-imam@tapad.com>
>> Date: November 7, 2013 at 4:25:30 PM
>> To: user@giraph.apache.org user@giraph.apache.org 
>> <mailto:user@giraph.apache.org>
>> Subject: Null Pointer Exception when running a job
>> Hi all,
>>
>> Im running a simple connected components algorithm against a graph of 
>> almost 200 million vertices.
>>
>> I have pretty much no guess on what this issue could be:
>> java.lang.IllegalStateException: run: Caught an unrecoverable exception null
>>          at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
>>          at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
>>          at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
>>          at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>>          at java.security.AccessController.doPrivileged(Native Method)
>>          at javax.security.auth.Subject.doAs(Subject.java:415)
>>          at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>>          at org.apache.hadoop.mapred.Child.main(Child.java:262)
>> Caused by: java.lang.NullPointerException
>>          at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
>>          at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
>>          at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
>>          at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
>>          at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
>>          at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
>>          at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
>>          at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
>>
>>
>> -- 
>> Artie Pesh-Imam
>> Software Engineer @ Tapad
>> 201-253-9647
>>
>>


Mime
View raw message