Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B65F3DE4A for ; Fri, 1 Feb 2013 01:01:36 +0000 (UTC) Received: (qmail 36563 invoked by uid 500); 1 Feb 2013 01:01:31 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 36379 invoked by uid 500); 1 Feb 2013 01:01:31 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 36369 invoked by uid 99); 1 Feb 2013 01:01:31 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2013 01:01:31 +0000 X-ASF-Spam-Status: No, hits=2.8 required=5.0 tests=GAPPY_SUBJECT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of rajesh.balamohan@gmail.com designates 209.85.215.53 as permitted sender) Received: from [209.85.215.53] (HELO mail-la0-f53.google.com) (209.85.215.53) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Feb 2013 01:01:23 +0000 Received: by mail-la0-f53.google.com with SMTP id fr10so2364176lab.26 for ; Thu, 31 Jan 2013 17:01:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=4pXwi58R59eHk8icEMkfpGnx7+yOmqvBnXnv3UALdD8=; b=uB1NCuN7IzSn1pFe2T5ekicc49E08k5PRI1kI5PF7k1ReRfjc2VQqm+/QrmA59laTV JlZazuDaBEyC8FKqJL/CRA6wADG4iSsc3tW+R12i6ZCCyEeZAn4WpyhNafF8wmpMmylC NLe5MiID9beQSmmzw0n5lyTJabPA3PsrRcSBHTWAbQMMR3cBPn6ahTKUcxyguqPp2yrV fTSeCS2eO+imtUUINiYS5YyIw5BOteEjUgTFNTmdcYqpeIxKz8KjCwJmoUdacG2Zz67z 2aXL3K/7boahOFcaL1e8ckXDbjZXP1ZASUbhdBb5sScL0s9mHqQRpGTN2wH4ZPi7t48L ABgw== MIME-Version: 1.0 X-Received: by 10.112.40.104 with SMTP id w8mr4195514lbk.114.1359680463424; Thu, 31 Jan 2013 17:01:03 -0800 (PST) Received: by 10.112.96.15 with HTTP; Thu, 31 Jan 2013 17:01:03 -0800 (PST) Date: Fri, 1 Feb 2013 06:31:03 +0530 Message-ID: Subject: o.a.h.m.MapTask: data buffer - Takes long time to come out From: Rajesh Balamohan To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=e0cb4efe2e8007cce004d49f47db X-Virus-Checked: Checked by ClamAV on apache.org --e0cb4efe2e8007cce004d49f47db Content-Type: text/plain; charset=UTF-8 Hi Experts, At times, we have seen tasks taking a lot long time map tasks and looking at the task logs, its spending a lot of time in "o.a.h.m.MapTask: data buffer = xxx / yyy". After sometime it prints the next line " MapTask: record buffer = xxx / yyy". I initially thought it was allocating data buffer in JVM and causing memory pressure. But it doesn't seem to be the case and no swapping is happening in the system as well. Has anyone else faced similar issue with MR-1.0? Any pointers would be appreciated. -- ~Rajesh.B --e0cb4efe2e8007cce004d49f47db Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Experts,

At times, we have seen task= s taking a lot long time map tasks and looking at the task logs, its spendi= ng a lot of time in "o.a.h.m.MapTask: data buffer =3D xxx / yyy".=

After sometime it prints the next line " MapTask: = record buffer =3D xxx / yyy". I initially thought it was allocating da= ta buffer in JVM and causing memory pressure. But it doesn't seem to be= the case and no swapping is happening in the system as well.

Has anyone else faced similar issue with MR-1.0? Any po= inters would be appreciated.

--
~Rajesh= .B
--e0cb4efe2e8007cce004d49f47db--