giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michal Haris <michal.ha...@visualdna.com>
Subject problem with removeEdges mutation
Date Wed, 11 Feb 2015 14:43:15 GMT
Hi all, I have managed to build a production giraph application off the
hadoop2 profile (but only mapreduce as yarn has few unresolved bugs). I
think I have now come across a bug which prevents me from using mutations
to do  de-duplicate edges in the first superstep of the computation.

Basically whether I use requestRemoveEdges(..) or vertex.removeEdges() the
program falls over with this exception:

java.lang.IllegalStateException: next: Failed on pos 16 edge
(targetVertexId = , value = dataxu)

at org.apache.giraph.edge.ByteArrayEdges$ByteArrayEdgeIterator.next(
ByteArrayEdges.java:170)at org.apache.giraph.edge.ByteArrayEdges.remove(
ByteArrayEdges.java:111)at
org.apache.giraph.graph.DefaultVertex.removeEdges(DefaultVertex.java:232)at
net.imagini.giraph.EdgeDeduplicationJob.compute(EdgeDeduplicationJob.java:34)at
org.apache.giraph.graph.ComputeCallable.computePartition(
ComputeCallable.java:247)at org.apache.giraph.graph.ComputeCallable.call(
ComputeCallable.java:168)at org.apache.giraph.graph.ComputeCallable.call(
ComputeCallable.java:1)at
org.apache.giraph.utils.LogStacktraceCallable.call(
LogStacktraceCallable.java:51)at java.util.concurrent.FutureTask.run(
FutureTask.java:262)at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1145)at
java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:615)at java.lang.Thread.run(Thread.java:744)

I have been using my own class for VertexID but even after reverting to
simple Text to see if problem is with my writable implementation I still
get what seems a corrupt state of the serialised byte array  edges after
requesting removal.

There is no mention of this problem in jira and googling yields nothing.
Anybody tried using removal of edges as part of superstep mutation ?
Anybody seen something similar?

I am seeing the problem with trunk or release-1.1, haven't tried older.



-- 
Michal Haris
Technical Architect
direct line: +44 (0) 207 749 0229
www.visualdna.com | t: +44 (0) 207 734 7033,

Mime
View raw message