Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7CE1310E63 for ; Thu, 6 Mar 2014 23:30:41 +0000 (UTC) Received: (qmail 31224 invoked by uid 500); 6 Mar 2014 23:30:40 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 31181 invoked by uid 500); 6 Mar 2014 23:30:40 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 31173 invoked by uid 99); 6 Mar 2014 23:30:40 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Mar 2014 23:30:40 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of claudio.martella@gmail.com designates 209.85.216.175 as permitted sender) Received: from [209.85.216.175] (HELO mail-qc0-f175.google.com) (209.85.216.175) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Mar 2014 23:30:33 +0000 Received: by mail-qc0-f175.google.com with SMTP id e16so3874506qcx.34 for ; Thu, 06 Mar 2014 15:30:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=rfC/r1BgZqnXVgSmZRmYkZwIcQFELBY78KCz3Ist1LI=; b=ZBI5FNTNXEGp1+n9FOlgjmzCZ6CarvIr95vXEx9FQ7H4jir9toztf1g/fS65gR4Z4Z gzThWwIbI9UK5xUksnjTyG2PV0Q5yfcYeqD7wpjt+z18DkbWxY9M2ZznZE/aConUBM0n IlUtea6alf4DEaxPem7HxcElWP9PyjEmuKg+laao2JMgFprS0Iluy08ucw87X2o2McU/ voW9UbBNwb/w+wMQw0Ca/pV5VwIZUl4G9Q/vWczJ4Hmbvr00nIKSuRbAM8nFQfWu+z3H ZKPxziOQXIfOeAcG1O5U7VGzaqRSdq9dyRVdU1R0ve4efhR28jUCzKmeHYdQ00MR2l8F fIbw== X-Received: by 10.224.160.83 with SMTP id m19mr17581890qax.21.1394148612294; Thu, 06 Mar 2014 15:30:12 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.19.66 with HTTP; Thu, 6 Mar 2014 15:29:52 -0800 (PST) In-Reply-To: References: From: Claudio Martella Date: Fri, 7 Mar 2014 00:29:52 +0100 Message-ID: Subject: Re: Giraph program stucks. To: "user@giraph.apache.org" Cc: Sebastian Schelter Content-Type: multipart/alternative; boundary=047d7bacb2aaccce7804f3f884aa X-Virus-Checked: Checked by ClamAV on apache.org --047d7bacb2aaccce7804f3f884aa Content-Type: text/plain; charset=ISO-8859-1 did you actually increase the heap? On Thu, Mar 6, 2014 at 11:43 PM, Suijian Zhou wrote: > Hi, > I tried to process only 2 of the input files, i.e, 2GB + 2GB input, the > program finished successfully in 6 minutes. But as I have 39 nodes, they > should be enough to load and process the 8*2GB=16GB size graph? Can > somebody help to give some hints( Will all the nodes participate in graph > loading from HDFS or only master node load the graph?)? Thanks! > > Best Regards, > Suijian > > > > 2014-03-06 16:24 GMT-06:00 Suijian Zhou : > > Hi, Experts, >> I'm trying to process a graph by pagerank in giraph, but the program >> always stucks there. >> There are 8 input files, each one is with size ~2GB and all copied onto >> HDFS. I use 39 nodes and each node has 16GB Mem and 8 cores. It keeps >> printing the same info(as the following) on the screen after 2 hours, looks >> no progress at all. What are the possible reasons? Testing small example >> files run without problems. Thanks! >> >> 14/03/06 16:17:42 INFO job.JobProgressTracker: Data from 39 workers - >> Compute superstep 0: 5854829 out of 49200000 vertices computed; 181 out of >> 1521 partitions computed >> 14/03/06 16:17:47 INFO job.JobProgressTracker: Data from 39 workers - >> Compute superstep 0: 5854829 out of 49200000 vertices computed; 181 out of >> 1521 partitions computed >> >> Best Regards, >> Suijian >> >> > -- Claudio Martella --047d7bacb2aaccce7804f3f884aa Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
did you actually increase the heap?


On Thu, Mar 6, 2014 at 11:43 P= M, Suijian Zhou <suijian.zhou@gmail.com> wrote:
Hi,
=A0 I tried to process only 2 of the input files, i.e, 2GB + 2GB input, th= e program finished successfully in 6 minutes. But as I have 39 nodes, they = should be enough to load=A0 and process the 8*2GB=3D16GB size graph? Can so= mebody help to give some hints( Will all the nodes participate in graph loa= ding from HDFS or only master node load the graph?)? Thanks!

=A0 Best Regards,
=A0 Suijian
=A0


2014-03-06 16:24 GMT-06= :00 Suijian Zhou <suijian.zhou@gmail.com>:

Hi= , Experts,
=A0 I'm trying to process a graph by pagerank in gi= raph, but the program always stucks there.
There are 8 input files, each one is with size ~2GB and all copied on= to HDFS. I use 39 nodes and each node has 16GB Mem and 8 cores. It keeps pr= inting the same info(as the following) on the screen after 2 hours, looks n= o progress at all. What are the possible reasons? Testing small example fil= es run without problems. Thanks!

14/03/06 16:17:42 INFO job.JobProgressTracker: Data from 39 worke= rs - Compute superstep 0: 5854829 out of 49200000 vertices computed; 181 ou= t of 1521 partitions computed
14/03/06 16:17:47 INFO job.JobProgressTrac= ker: Data from 39 workers - Compute superstep 0: 5854829 out of 49200000 ve= rtices computed; 181 out of 1521 partitions computed

=A0 Best Regards,
=A0 Suijian





--
=A0 =A0Claudio Martella
=A0 =A0
--047d7bacb2aaccce7804f3f884aa--