hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: About concurrency loading problem in GraphJobRunner.java in HamaV0.7.1
Date Fri, 01 Apr 2016 09:30:52 GMT
Hi,

Please subscribe mailing list first. Otherwise, your mail won't
delivered to subscribers.

On Thu, Mar 24, 2016 at 11:11 PM,
<dev-reject-1458828684.20517.ddaablpdbjjphejckmpp@hama.apache.org>
wrote:
>
> To approve:
>    dev-accept-1458828684.20517.ddaablpdbjjphejckmpp@hama.apache.org
> To reject:
>    dev-reject-1458828684.20517.ddaablpdbjjphejckmpp@hama.apache.org
> To give a reason to reject:
> %%% Start comment
> %%% End comment
>
>
>
> ---------- Forwarded message ----------
> From: "刘强" <edward2006@qq.com>
> To: dev <dev@hama.apache.org>
> Cc:
> Date: Thu, 24 Mar 2016 22:10:55 +0800
> Subject: About concurrency loading problem in GraphJobRunner.java in HamaV0.7.1
>
> Dear developers,
>
> I've got problem:
> 16/03/24 20:30:53 INFO graph.GraphJobRunner: Total time spent for loading vertices: 3447
ms
> 16/03/24 20:30:53 INFO graph.GraphJobRunner: Total time spent for broadcasting global
vertex count: 789 ms
> 16/03/24 20:30:54 INFO graph.GraphJobRunner: Total time spent for initial superstep:
787 ms
> 16/03/24 20:30:55 INFO graph.GraphJobRunner: Total time spent for broadcasting aggregation
values: 108 ms
> 16/03/24 20:30:55 ERROR bsp.BSPTask: Error running bsp setup and bsp function.
> java.lang.NullPointerException
> at org.apache.hama.util.UnsafeByteArrayInputStream.<init>(UnsafeByteArrayInputStream.java:63)
> at org.apache.hama.util.WritableUtils.unsafeDeserialize(WritableUtils.java:63)
> at org.apache.hama.graph.MapVerticesInfo.get(MapVerticesInfo.java:101)
> at org.apache.hama.graph.GraphJobRunner$ComputeRunnable.<init>(GraphJobRunner.java:367)
> at org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:277)
> at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:187)
> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:171)
> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:144)
> at org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255)
>
> I add some vertices print information both in loading from local vertices and transferring
vertices in partition phase (just print the vertex information. vertex.toString()... ). Then
its OK.
>
> I found that the above exception was because that while processing messages in superstep,
some vertices missed can cannot get from ConcurrentHashMap you used in the MapVerticesInfo.java.
>
> This may be because that the loadVertices() funtion has some concurrency problem. Too
many threads are started and sometimes that the loaded funtion may miss some vertices and
I doubt its the problem of ConcurrencyHashMap.
>
> Can you tell me the reason why its caused? For I'm not quite sure.
>
> Thanks very much!
> Yours , QiangLiu !
>
>
>
>
>
>



-- 
Best Regards, Edward J. Yoon

Mime
View raw message