Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@giraph.apache.org
Date: Mon, 25 Feb 2013 02:28:12 +0000 (UTC)
From: "Alessandro Presta (JIRA)" <jira@apache.org>
To: giraph-dev@incubator.apache.org
Message-ID: <JIRA.12633259.1361388465576.330882.1361759292522@arcas>
In-Reply-To: <JIRA.12633259.1361388465576@arcas>
References: <JIRA.12633259.1361388465576@arcas>
Subject: [jira] [Commented] (GIRAPH-528) Decouple vertex implementation from
 edge storage
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/GIRAPH-528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585574#comment-13585574 ] 

Alessandro Presta commented on GIRAPH-528:
------------------------------------------

I gave this some more thought, and I've refined the idea.
Instead of having the user optionally implement EdgeStore, we can instead have him implement an Edges interface which defines how edges are stored, serialized, and iterated over. That way it's the infrastructure that deals with synchronization during input. Ownership of the edges is transferred to the vertex simply by passing a reference to the Edges object, with no copying involved.
No changes in the current Partition/PartitionStore logic.
This is what the core classes would look like: https://gist.github.com/apresta/5026916
Let me know what you think.
                
> Decouple vertex implementation from edge storage
> ------------------------------------------------
>
>                 Key: GIRAPH-528
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-528
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Alessandro Presta
>            Assignee: Alessandro Presta
>
> This is meant to address the following issues:
> 1) The Vertex hierarchy is too complex and sometimes hard to work with (Vertex, SimpleVertex, MutableVertex, SimpleMutableVertex...).
> 2) Changing the underlying edge storage implementation for an existing algorithm requires editing your vertex to extend a different one.
> 3) In the general case (e.g. when not using ByteArrayVertex with the current EdgeStore), moving edges from the EdgeStore to the vertices is an additional step that can be avoided.
> My proposal is the following:
> - Make EdgeStore an interface. An implementation should deal with (concurrent) insertion of edges during input superstep; iteration over a vertex's edges during computation; insertion/deletion of edges during mutations (optional?); checkpointing.
> - The default EdgeStore will be the current byte-array implementation, which is generic (works with any choice of <I, V, E, M>) and reasonably optimized.
> - Only one Vertex class, which the user extends for the sole purpose of defining compute(). I don't necessarily agree that it should be an interface, because we still want to provide methods like getEdges(), sendMessage(), and those should delegate to the EdgeStore/MessageStore of choice.
> - Switching edge storage implementation is done by passing the EdgeStore class as an option. One can also define his own ad-hoc EdgeStore (e.g., backed by primitive arrays).
> I think we should also extend this idea to MessageStore, making it possible to override that functionality too.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira