Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7907811CEB for ; Wed, 16 Apr 2014 07:37:50 +0000 (UTC) Received: (qmail 90867 invoked by uid 500); 16 Apr 2014 07:37:49 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 90665 invoked by uid 500); 16 Apr 2014 07:37:48 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 90655 invoked by uid 99); 16 Apr 2014 07:37:48 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Apr 2014 07:37:47 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of agrta.rawat@gmail.com designates 209.85.220.182 as permitted sender) Received: from [209.85.220.182] (HELO mail-vc0-f182.google.com) (209.85.220.182) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 16 Apr 2014 07:37:43 +0000 Received: by mail-vc0-f182.google.com with SMTP id ib6so10552418vcb.13 for ; Wed, 16 Apr 2014 00:37:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=MhQtLw14E25LxO/krmRBwjXTqHvuO3yufINN7XjIjZ4=; b=w8/Fxs4eI8Bn/ocvqf9mdjX1E1QgYNzCWyxp/6Ij8bwlaCsRaxFU6EcHqrRLpDWCGV TppJIabbIdADBhxU2ZEhS7IhIXKt6BfpvruiO6xbPrg147kiQLvfNwetS8h2skFL0KTs XB8QociX6IzvNjFMyHlFm0IJcQsx59cR2lJuhzDL3D3R5YyQyEX9j+ozFAg/O4a7CV+t PIIvPWwPUOdSHs0jXFIHIPsZLJzTPukyxayX8yPMJyizpUIj5/TcGEbowfjxKd5oZU1o mWk5kgj3vy/3vP2C2DT/ymivRq7Y6EoXWdlx0q9FOufOr865jyH2LmGypYh2C0xO8l6E RjsA== MIME-Version: 1.0 X-Received: by 10.221.62.131 with SMTP id xa3mr1638724vcb.13.1397633841008; Wed, 16 Apr 2014 00:37:21 -0700 (PDT) Received: by 10.58.98.226 with HTTP; Wed, 16 Apr 2014 00:37:20 -0700 (PDT) In-Reply-To: References: Date: Wed, 16 Apr 2014 13:07:20 +0530 Message-ID: Subject: Re: Giraph Buffer Size From: Agrta Rawat To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=001a11362b189e942b04f723fceb X-Virus-Checked: Checked by ClamAV on apache.org --001a11362b189e942b04f723fceb Content-Type: text/plain; charset=UTF-8 Hi Pavan, For all the intermediate processing there would be a buffer (intermediate memory space) that stores data, messages etc.and then the complete process further. Pls correct me if I am wrong. I have set Xms and Xmx values properly. The problem is that the task runs for small datasets but as the input data size is increased, it fails. The error that I am getting in sysout logs is- # A fatal error has been detected by the Java Runtime Environment: # # SIGBUS (0x7) at pc=0x00002aaaab404144, pid=10397, tid=1144650048 # # JRE version: 6.0_25-b06 # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.0-b11 mixed mode linux-amd64 compressed oops) # Problematic frame: # J sun.nio.ch.SelectorImpl.processDeregisterQueue()V # # An error report file with more information is saved as: # /hadoopTaskTrackerLogsLocation/process_id/s_err_pid10397.log # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp Please suggest what should be done? Am I missing anything? Regards, Agrta Rawat On Wed, Apr 16, 2014 at 12:44 PM, Pavan Kumar A wrote: > > What do u mean by buffer size? Just as a note, please ensure that Xmx & > Xms values are properly set for the mapper using mapred.child.java.opts > or mapred.map.child.java.opts > Also what does the error message show: please use pastebin & post the link > here. > ------------------------------ > Date: Wed, 16 Apr 2014 12:13:29 +0530 > Subject: Giraph Buffer Size > From: agrta.rawat@gmail.com > To: user@giraph.apache.org > > > Hi All, > > I am trying to run a job in Giraph-1.0.0 on Hadoop-1.0.0 cluster with 3 > nodes. > Each node has 32gb RAM. > > In superstep-8 of my algorithm approximately 2M messages are being sent > where size of each message is more than 20 kb. But the process sticks here > and task gets failed. > > In the sysout logs, it shows Fatal Error. > > Is this error because Buffer is getting full? > How can I increase the buffer size for giraph application? > > Please suggest. > > Regards, > Agrta Rawat > > --001a11362b189e942b04f723fceb Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Pavan,

For all th= e intermediate processing there would be a buffer (intermediate memory spac= e) that stores data, messages etc.and then the complete process further.
Pls correct me if I am wrong.

I have set= Xms and Xmx values properly.

The problem is that the tas= k runs for small datasets but as the input data size is increased, it fails= .

The error that I am getting in sysout logs is-

= # A fatal error has been detected by the Java Runtime Environment:
#
= #=C2=A0 SIGBUS (0x7) at pc=3D0x00002aaaab404144, pid=3D10397, tid=3D1144650= 048
#
# JRE version: 6.0_25-b06
# Java VM: Java HotSpot(TM) 64-Bit Server= VM (20.0-b11 mixed mode linux-amd64 compressed oops)
# Problematic fram= e:
# J=C2=A0 sun.nio.ch.SelectorImpl.processDeregisterQueue()V
#
#= An error report file with more information is saved as:
# /hadoopTaskTrackerLogsLocation/process_id/s_err_pid10397.log
#
# If= you would like to submit a bug report, please visit:
#=C2=A0=C2=A0 http://java.sun.com= /webapps/bugreport/crash.jsp


Please suggest what should be done? Am I missing anything?
Regards,
Agrta Rawat



On Wed, Apr 16, 20= 14 at 12:44 PM, Pavan Kumar A <pavanka@outlook.com> wrote:=

What do u mean by buffer size? Just as a note, pl= ease ensure that Xmx & Xms values are properly set for the mapper using= =C2=A0mapred.child.java.opts or=C2=A0mapred.map.child.java.opts<= /span>
Also what does the error message show: please use pastebin & post the l= ink here.
Date: Wed, 16 Apr 2014 12:13:29 +0530
Subject: Gir= aph Buffer Size
From: agrta.rawat@gmail.com
To: user@giraph= .apache.org


Hi All,

I am trying to run a job in Gir= aph-1.0.0 on Hadoop-1.0.0 cluster with 3 nodes.
Each node has 32gb RAM.

In superstep-8 of my algori= thm approximately 2M messages are being sent where size of each message is = more than 20 kb. But the process sticks here and task gets failed.

In the sysout logs, it shows Fatal Error.

Is this error because Buffer is getting full?
How can I increase = the buffer size for giraph application?

Please suggest.

Regards,
Agrta Rawat

<= /div>

--001a11362b189e942b04f723fceb--