giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Saltz <sal...@gmail.com>
Subject Re: Best way to know the assignment of vertices to workers
Date Fri, 28 Nov 2014 15:23:06 GMT
Kiran,

To answer your question directly, in an AbstractComputation class (or
whatever descendant you're using), you may call
getWorkerContext().getMyWorkerIndex() (here
<https://giraph.apache.org/apidocs/org/apache/giraph/worker/WorkerContext.html>).
However, if each vertex has metadata associated to it, I think the best way
to go would be to define a custom VertexReader
<https://giraph.apache.org/apidocs/org/apache/giraph/io/class-use/VertexReader.html>
and custom Vertex type to take that into account when reading the vertex.

Best,
Matthew

On Fri, Nov 28, 2014 at 1:02 PM, Garimella Kiran <kiran.garimella@aalto.fi>
wrote:

>  Hi all,
>
>  Is there a clean way to find out which worker a particular vertex is
> assigned to?
>
>  From what I tried out, I found that given n workers, each node is
> assigned to the worker with id (vertex_id % n  ). Is that a safe way to do
> this?
>
>  I’ve had a look at previous discussions, but most of them have no answer.
>
>  —————————
>
>  Why I need it:
>
>  In my application, each vertex needs to know some additional meta data,
> which is loaded from file. This metadata file is huge (>50 G) and so, on
> each worker, I only want to load the metadata corresponding to the vertices
> present on that worker.
>
>  —————————
>
>
>  Previous discussions:
> 1.
> http://mail-archives.apache.org/mod_mbox/giraph-user/201310.mbox/%3C7EC16F82718A6D4A920A99FE46CE7F4E2861F779%40MERCMBX19R.na.SAS.com%3E
> 2.
> http://mail-archives.apache.org/mod_mbox/giraph-user/201403.mbox/%3CCAMf08QYE%2BRgUv9otXT6oPJorTNjQ-Ay8p4NUiuhds8%2BzgDzs1w%40mail.gmail.com%3E
>
>
>
>  Regards,
> Kiran
>

Mime
View raw message