Return-Path: X-Original-To: apmail-giraph-dev-archive@www.apache.org Delivered-To: apmail-giraph-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4A39810FCE for ; Fri, 21 Mar 2014 16:14:07 +0000 (UTC) Received: (qmail 89451 invoked by uid 500); 21 Mar 2014 16:14:06 -0000 Delivered-To: apmail-giraph-dev-archive@giraph.apache.org Received: (qmail 89419 invoked by uid 500); 21 Mar 2014 16:14:05 -0000 Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@giraph.apache.org Delivered-To: mailing list dev@giraph.apache.org Received: (qmail 89407 invoked by uid 99); 21 Mar 2014 16:14:05 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 21 Mar 2014 16:14:05 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 5A9EB1D55CC; Fri, 21 Mar 2014 16:14:02 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============3435634402792227395==" MIME-Version: 1.0 Subject: Re: Review Request 19401: GIRAPH-873 - Specialized edge stores From: "Craig Muchinsky" To: "Craig Muchinsky" , "giraph" Date: Fri, 21 Mar 2014 16:14:02 -0000 Message-ID: <20140321161402.30591.65390@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "Craig Muchinsky" X-ReviewGroup: giraph X-ReviewRequest-URL: https://reviews.apache.org/r/19401/ X-Sender: "Craig Muchinsky" References: <20140319152542.6590.84184@reviews.apache.org> In-Reply-To: <20140319152542.6590.84184@reviews.apache.org> Reply-To: "Craig Muchinsky" X-ReviewRequest-Repository: giraph-git --===============3435634402792227395== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19401/ ----------------------------------------------------------- (Updated March 21, 2014, 4:14 p.m.) Review request for giraph. Changes ------- Updated with concurrency fixes Repository: giraph-git Description ------- This patch converts the EdgeStore class to an interface and adds 3 impls: SimpleEdgeStore: same behavior as original EdgeStore class IntEdgeStore: edge store optimized for integer ids, uses fastutil primitive collections and allows reuse of vertexId objects (which is the big performance win) LongEdgeStore: edge store optimized for long ids, uses fastutil primitive collections and allows reuse of vertexId objects (which is the big performance win) The impl to use is chosen programmatically based on interrogation of the id class type. In the future perhaps the EdgeStore impl could be specified via configuration, to allow extensibility. Diffs (updated) ----- giraph-core/src/main/java/org/apache/giraph/comm/ServerData.java 39bf504 giraph-core/src/main/java/org/apache/giraph/edge/EdgeStore.java dd8f2a3 giraph-core/src/main/java/org/apache/giraph/edge/SimpleEdgeStore.java PRE-CREATION giraph-core/src/main/java/org/apache/giraph/edge/primitives/IntEdgeStore.java PRE-CREATION giraph-core/src/main/java/org/apache/giraph/edge/primitives/LongEdgeStore.java PRE-CREATION giraph-core/src/main/java/org/apache/giraph/edge/primitives/package-info.java PRE-CREATION Diff: https://reviews.apache.org/r/19401/diff/ Testing ------- Successful "mvn clean verify" with hadoop_2 profile, and 4B vertex 5B edge graph tested on 18 node 432 core cluster. Thanks, Craig Muchinsky --===============3435634402792227395==--