giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <ach...@apache.org>
Subject Re: Vertex exists error when processing input splits for Sequence file
Date Mon, 30 Jan 2012 07:28:12 GMT
Hi David,

So from the errors, it appears that your input has multiple vertices 
with the same vertex id.  Currently we throw an exception to prevent 
this from happening as it is typically not what you want.  You probably 
want to watch the vertices being processed from the vertex input format 
and see why you are getting duplicates.  It's likely to be either an 
error with the data actually have vertices with the same vertex id or an 
error with your custom vertex input format.

To help debug, you might want to add some logging to your record reader 
and print the vertex ids or you can add some logging to where that code 
is called in BspServiceWorker#readVerticesFromInputSplit().

Hope that helps,

Avery

On 1/29/12 8:13 PM, David Garcia wrote:
>
>
> Hello, I get this error when I try run my job:
> 2012-01-29 21:50:18,494 INFO org.apache.giraph.graph.BspServiceWorker: reserveInputSplit:
reservedPath = null, 1 of 1 InputSplits are finished.
> 2012-01-29 21:50:18,494 INFO org.apache.giraph.graph.BspServiceWorker: setup: Finally
loaded a total of (v=0, e=0)
> 2012-01-29 21:50:18,764 INFO org.apache.giraph.graph.BspService: process: inputSplitsAllDoneChanged
(all vertices sent from input splits)
> 2012-01-29 21:50:18,766 ERROR org.apache.giraph.graph.GraphMapper: setup: Caught exception
just before end of setup
> java.lang.IllegalStateException: moveVerticesToWorker: Vertex Vertex(id=zzYNBgKt2LF6ClLA2eMBzuN7SkA.,value=org.apache.hadoop.io.MapWritable@5ce8787a,#edges=0)
already exists!
> 	at org.apache.giraph.graph.BspServiceWorker.movePartitionsToWorker(BspServiceWorker.java:1389)
> 	at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:624)
> 	at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:458)
> .
> .
> .
> I'm not sure where the start debugging. . .BspServiceWorker is hella big.  All input
is welcome.  As I mentioned, I'm processing a sequenceFile that has Text keys and MapWritable
Values.  I would like the vertices to have Text indices and MapWritable values.  (I'm not
inserting any edges for the time being. . .I just want to see the file get split properly).
 I have implemented custom input formats and record readers.  Thx
> -Dave


Mime
View raw message