Return-Path: X-Original-To: apmail-giraph-dev-archive@www.apache.org Delivered-To: apmail-giraph-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 426D11117B for ; Tue, 15 Jul 2014 23:33:26 +0000 (UTC) Received: (qmail 63468 invoked by uid 500); 15 Jul 2014 23:33:26 -0000 Delivered-To: apmail-giraph-dev-archive@giraph.apache.org Received: (qmail 63422 invoked by uid 500); 15 Jul 2014 23:33:26 -0000 Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@giraph.apache.org Delivered-To: mailing list dev@giraph.apache.org Received: (qmail 63405 invoked by uid 99); 15 Jul 2014 23:33:25 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Jul 2014 23:33:25 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 7EE1E1DB7EE; Tue, 15 Jul 2014 23:33:13 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============1501623986018047810==" MIME-Version: 1.0 Subject: Re: Review Request 23140: Fix checkpointing From: "Sergey Edunov" To: "Maja Kabiljo" , "Sergey Edunov" , "giraph" , "Vijayakumar Ramdoss" Date: Tue, 15 Jul 2014 23:33:13 -0000 Message-ID: <20140715233313.24004.6818@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "Sergey Edunov" X-ReviewGroup: giraph X-ReviewRequest-URL: https://reviews.apache.org/r/23140/ X-Sender: "Sergey Edunov" References: <20140715210859.14170.99403@reviews.apache.org> In-Reply-To: <20140715210859.14170.99403@reviews.apache.org> Reply-To: "Sergey Edunov" X-ReviewRequest-Repository: giraph-git --===============1501623986018047810== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23140/ ----------------------------------------------------------- (Updated July 15, 2014, 11:33 p.m.) Review request for giraph. Repository: giraph-git Description ------- This fix merely makes checkpointing work again. Diffs (updated) ----- giraph-core/src/main/java/org/apache/giraph/aggregators/AggregatorWrapper.java 9613805 giraph-core/src/main/java/org/apache/giraph/bsp/BspService.java 2e35373 giraph-core/src/main/java/org/apache/giraph/comm/ServerData.java 85bfe04 giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java ab0570f giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java 0275395 giraph-core/src/main/java/org/apache/giraph/partition/BasicPartitionOwner.java 545d1af giraph-core/src/main/java/org/apache/giraph/partition/HashMasterPartitioner.java 240687e giraph-core/src/main/java/org/apache/giraph/partition/HashWorkerPartitioner.java d833895 giraph-core/src/main/java/org/apache/giraph/partition/MasterGraphPartitioner.java 50c750a giraph-core/src/main/java/org/apache/giraph/partition/PartitionBalancer.java 3454d62 giraph-core/src/main/java/org/apache/giraph/partition/PartitionOwner.java 0ac74da giraph-core/src/main/java/org/apache/giraph/partition/SimpleMasterPartitioner.java f128f34 giraph-core/src/main/java/org/apache/giraph/partition/SimpleWorkerPartitioner.java 3c0de44 giraph-core/src/main/java/org/apache/giraph/partition/WorkerGraphPartitioner.java 004ea81 giraph-core/src/main/java/org/apache/giraph/utils/InternalVertexRunner.java 2c4606f giraph-core/src/main/java/org/apache/giraph/utils/io/ExtendedDataInputOutput.java af45426 giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java de7af28 giraph-core/src/main/java/org/apache/giraph/worker/WorkerContext.java 29835c5 giraph-core/src/test/java/org/apache/giraph/partition/SimpleRangePartitionFactoryTest.java 96bd5d7 giraph-examples/src/test/java/org/apache/giraph/TestCheckpointing.java PRE-CREATION Diff: https://reviews.apache.org/r/23140/diff/ Testing ------- I tested it running multiple different jobs. I run page rank on 2*10^9 vertices on 200 workers and it seems to work just fine. It only takes 2 minutes to save checkpoint. Thanks, Sergey Edunov --===============1501623986018047810==--