Return-Path: 
 <user-return-11867-apmail-hadoop-common-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-common-user-archive@www.apache.org
Delivered-To: apmail-hadoop-common-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id E6CEA1098D
	for <apmail-hadoop-common-user-archive@www.apache.org>;
 Fri, 15 Nov 2013 21:58:09 +0000 (UTC)
Received: (qmail 12887 invoked by uid 500); 15 Nov 2013 21:58:04 -0000
Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org
Received: (qmail 12774 invoked by uid 500); 15 Nov 2013 21:58:04 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Received: (qmail 12767 invoked by uid 99); 15 Nov 2013 21:58:04 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 Nov 2013 21:58:04 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates
 209.85.217.170 as permitted sender)
Received: from [209.85.217.170] (HELO mail-lb0-f170.google.com)
 (209.85.217.170)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 Nov 2013 21:57:58 +0000
Received: by mail-lb0-f170.google.com with SMTP id z5so3220025lbh.1
        for <user@hadoop.apache.org>; Fri, 15 Nov 2013 13:57:38 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=cqjMwiVHMxijTNwJTL5USPejp/c4scSWfEtTYreVe08=;
        b=Kb9+4C5Vksf1yp2qL/ChNdUdEuxdD+WL4mqiUFHBpvIIRQ1f4wWuw6rAObii6BPgx3
         YG9eOqgaFg55POboK6AWFW5Et6hGw+5Ztc9Ln/K29o41ZMg15HvuirOz20lje5kW2M86
         I9TZ7W6XeMYtkZxlB9pUxSego53l5FvqaOvyUAxNe9NZkia+6i/n7ojQEW+NKaXYGgCw
         sBeH9uMJl0IQ5P7J/8Cnnp5UdVTtuW659Q/rW3u1sUBWHujS7Rna0qQ92gCLCkooxCed
         u3S7nc05/BSsGdf02uflKvs7vVm52Tddkj8lJo4WoBirhGEWJV4yAIYqh8meQXdOGO3T
         emcg==
MIME-Version: 1.0
X-Received: by 10.152.87.144 with SMTP id ay16mr888413lab.61.1384552658411;
 Fri, 15 Nov 2013 13:57:38 -0800 (PST)
Received: by 10.112.129.40 with HTTP; Fri, 15 Nov 2013 13:57:38 -0800 (PST)
In-Reply-To: 
 <CADK15wJ74WDopYuO-sAG5_g97QZbFpqEMS98aU_mDDe27WoWeg@mail.gmail.com>
References: 
 <CADK15wJ74WDopYuO-sAG5_g97QZbFpqEMS98aU_mDDe27WoWeg@mail.gmail.com>
Date: Fri, 15 Nov 2013 13:57:38 -0800
Message-ID: 
 <CALte62z33vYN=UOtZk+Ex8imaXHhWE4rYyDgqSBYyrJA_6FJhA@mail.gmail.com>
Subject: Re: Hadoop jobtracker OOME fix applied and no OOME but JT hanged
From: Ted Yu <yuzhihong@gmail.com>
To: "common-user@hadoop.apache.org" <user@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=001a11c235e660a39804eb3e4919
X-Virus-Checked: Checked by ClamAV on apache.org

--001a11c235e660a39804eb3e4919
Content-Type: text/plain; charset=ISO-8859-1

>From the command line, can you run 'jmap -heap' ?

http://download.oracle.com/javase/1.5.0/docs/tooldocs/share/jmap.html


On Fri, Nov 15, 2013 at 10:50 AM, Viswanathan J
<jayamviswanathan@gmail.com>wrote:

> Hi guys,
>
> I had JT OOME in hadoop version 1.2.1 and applied the patch based on the
> fix given by Apache contributors for jira issue mapreduce-5508.
>
> After applying that fix the heap size gradually increases but after one
> week jobtracker completely hangs and slowdown but didn't get JT OOME. No
> error in JT logs.
>
> Without this fix weekly once JT heap size increases and got OOME.
>
> Please help its our production environment.
>
> Thanks,
>

--001a11c235e660a39804eb3e4919
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><p style=3D"margin:0px 0px 1em;padding:0px;border:0px;font=
-size:14px;vertical-align:baseline;clear:both;color:rgb(0,0,0);font-family:=
Arial,&#39;Liberation Sans&#39;,&#39;DejaVu Sans&#39;,sans-serif;line-heigh=
t:18px">
>From the command line, can you run &#39;jmap -heap&#39; ?</p><p style=3D"ma=
rgin:0px 0px 1em;padding:0px;border:0px;font-size:14px;vertical-align:basel=
ine;clear:both;color:rgb(0,0,0);font-family:Arial,&#39;Liberation Sans&#39;=
,&#39;DejaVu Sans&#39;,sans-serif;line-height:18px">
<a href=3D"http://download.oracle.com/javase/1.5.0/docs/tooldocs/share/jmap=
.html" rel=3D"nofollow" style=3D"margin:0px;padding:0px;border:0px;vertical=
-align:baseline;background-color:transparent;color:rgb(74,107,130);text-dec=
oration:none">http://download.oracle.com/javase/1.5.0/docs/tooldocs/share/j=
map.html</a></p>
</div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Fri,=
 Nov 15, 2013 at 10:50 AM, Viswanathan J <span dir=3D"ltr">&lt;<a href=3D"m=
ailto:jayamviswanathan@gmail.com" target=3D"_blank">jayamviswanathan@gmail.=
com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><p>Hi guys,</p>
<p>I had JT OOME in hadoop version 1.2.1 and applied the patch based on the=
 fix given by Apache contributors for jira issue mapreduce-5508. </p>
<p>After applying that fix the heap size gradually increases but after one =
week jobtracker completely hangs and slowdown but didn&#39;t get JT OOME. N=
o error in JT logs.</p>
<p>Without this fix weekly once JT heap size increases and got OOME.</p>
<p>Please help its our production environment.</p>
<p>Thanks,<br>
</p>
</blockquote></div><br></div>

--001a11c235e660a39804eb3e4919--