incubator-giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Garcia <>
Subject question about vertex instantiation location. . .
Date Fri, 10 Feb 2012 21:25:39 GMT
Hey guys. . .I have a questions about "dynamic" vertex instantiation vis
the sendMsg(. . .) method.  I have a job that starts processing on a
sequenceFile with only two vertices in it.  Each vertex has information in
it's value that tells it what vertices are adjacent to it.  The primary
reason I'm doing this is to avoid loading the entire graph into the job.
There are many vertices that won't do any processing (no need to load
them).  I would like to take my two vertices and "dynamically" build the
graph by sending messages.  So far, my experimentation shows that this is
promising. . .but I have a question WRT load balancing for new vertex
instantiation.  When I call sendMsg(newVertexID), where will the vertex be
instantiated?  If I specify 20 mappers (but with only two vertices in my
sequence file), obviously there is going to be at least one mapper without
a vertex.  Is it possible that sendMsg(newVertexID) will be instantiated
on an empty mapper?  I would like this. . .for load balancing purposes.


View raw message