giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Artie Pesh-Imam <artie.pesh-i...@tapad.com>
Subject Re: Null Pointer Exception when running a job
Date Tue, 19 Nov 2013 17:51:07 GMT
Is there no other suggestions on what to do here? I really don’t have any idea about what’s
going on? I actually ran this against a single extract from our graph. 

Here are some more logs. 

Again this is running giraph 1.0 against hadoop 2.
Thanks,

2013-11-19 17:06:04,097 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.IllegalStateException: run: Caught an unrecoverable exception null
	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
	at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
	at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
	at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
	at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
	at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
	at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
	at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)

-- 
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Artie Pesh-Imam Artie Pesh-Imam
Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com
Date: November 11, 2013 at 10:55:54 AM
To: Sertuğ Kaya sertug.kaya@agmlab.com, Mohammad Islam mislam77@yahoo.com, user@giraph.apache.org
user@giraph.apache.org
Subject:  Re: Null Pointer Exception when running a job  
I can certainly try this. Are there any other configurations which are worth exploring?


-- 
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Sertuğ Kaya Sertuğ Kaya
Reply: user@giraph.apache.org user@giraph.apache.org
Date: November 9, 2013 at 3:58:39 PM
To: user@giraph.apache.org user@giraph.apache.org, Mohammad Islam mislam77@yahoo.com
Subject:  Re: Null Pointer Exception when running a job
Hello Artie;
I also had such problems when working on large datasets. It probably needs a better hadoop/mapreduce
configuration. For example did you try increasing the memory of task tracker child processes
in mapred-site.xml such as:
<property>
    <name>mapred.child.java.opts</name>
    <value>-Xmx3000m</value>
  </property>
Regards;
Sertug

On 08-11-2013 01:26, Artie Pesh-Imam wrote:
Hey 

It does work against a smaller data set and Im running 1.0.0 for hadoop 2.


-- 
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Mohammad Islam Mohammad Islam
Reply: user@giraph.apache.org user@giraph.apache.org, Mohammad Islam mislam77@yahoo.com
Date: November 7, 2013 at 6:20:53 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject:  Re: Null Pointer Exception when running a job
Hi Artie,
Hopefully someone would give you a concrete answer. 

My rambling comments:
Did you try with a smaller number of vertices? I want to know if the problem is due to increased
load.
What version of Giraph are you using?

Regards,
Mohammad


On Thursday, November 7, 2013 2:10 PM, Artie Pesh-Imam <artie.pesh-imam@tapad.com> wrote:
Please let me know if any additional context would be useful. With no detail in the error
message, I have no idea what the root issue could be.

-- 
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647
From: Artie Pesh-Imam Artie Pesh-Imam
Reply: Artie Pesh-Imam artie.pesh-imam@tapad.com
Date: November 7, 2013 at 4:25:30 PM
To: user@giraph.apache.org user@giraph.apache.org
Subject:  Null Pointer Exception when running a job
Hi all,

Im running a simple connected components algorithm against a graph of almost 200 million vertices. 

I have pretty much no guess on what this issue could be:
java.lang.IllegalStateException: run: Caught an unrecoverable exception null
        at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
        at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.NullPointerException
        at org.apache.giraph.comm.SendCache.removeWorkerData(SendCache.java:157)
        at org.apache.giraph.comm.SendCache.removeAllData(SendCache.java:171)
        at org.apache.giraph.comm.SendMessageCache.removeAllMessages(SendMessageCache.java:92)
        at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:410)
        at org.apache.giraph.worker.BspServiceWorker.sendWorkerPartitions(BspServiceWorker.java:1316)
        at org.apache.giraph.worker.BspServiceWorker.exchangeVertexPartitions(BspServiceWorker.java:1361)
        at org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:255)
        at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)

-- 
Artie Pesh-Imam
Software Engineer @ Tapad
201-253-9647




Mime
View raw message