Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 56CB79529 for ; Tue, 12 Jun 2012 21:23:28 +0000 (UTC) Received: (qmail 52998 invoked by uid 500); 12 Jun 2012 21:23:27 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 52951 invoked by uid 500); 12 Jun 2012 21:23:27 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 52942 invoked by uid 99); 12 Jun 2012 21:23:27 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Jun 2012 21:23:26 +0000 X-ASF-Spam-Status: No, hits=0.0 required=5.0 tests=FSL_RCVD_USER,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [208.97.132.145] (HELO homiemail-a6.g.dreamhost.com) (208.97.132.145) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Jun 2012 21:23:20 +0000 Received: from homiemail-a6.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a6.g.dreamhost.com (Postfix) with ESMTP id 36E8A59807D for ; Tue, 12 Jun 2012 14:23:00 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=darose.net; h=message-id:date :from:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; q=dns; s=darose.net; b= vg+Qh5ONnxX++OmIpKDU1bD4rp18EvdTVcQ5rp9yZhT4a28cZog6JIWiTQHveQZj fPjcvj+AKAde5+Xl+/JlRGR8qCwwNT00P7WPeY+W4Dn1ASdt5eoOg7LgN9fPqO0V Dk7DrQppOHcDvG+s5epsi3n2blG9BqHG7Kn5ZxAgIdA= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=darose.net; h=message-id :date:from:mime-version:to:subject:references:in-reply-to :content-type:content-transfer-encoding; s=darose.net; bh=FoBuwp 9Wc/GLOyaCkSwk7aILeDc=; b=LRoFB3vxZw/O8863P3RQ0Yrz2fIyuiFcHRss67 MtnSnrR4fCPAPgm3K/tdQBA8bM4k4K0jmDU+17PhumK32g907BLCz23w/muuMi4f tbmvJ+JeU3W76ms3dDfWXa+AXNSK4hKelkv2ptVsgPynDqXYpc6AnvwE2XJJu59c nb2ak= Received: from darsys12 (cpe-68-175-23-108.nyc.res.rr.com [68.175.23.108]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: darose@darose.net) by homiemail-a6.g.dreamhost.com (Postfix) with ESMTPSA id 1F42F59807A for ; Tue, 12 Jun 2012 14:23:00 -0700 (PDT) Received: from [10.1.0.2] by darsys12 with esmtp (Exim 4.77) (envelope-from ) id 1SeYYE-0000lE-ON for mapreduce-user@hadoop.apache.org; Tue, 12 Jun 2012 17:22:58 -0400 Message-ID: <4FD7B332.6010802@darose.net> Date: Tue, 12 Jun 2012 17:22:58 -0400 From: David Rosenstrauch User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20120605 Thunderbird/13.0 MIME-Version: 1.0 To: mapreduce-user@hadoop.apache.org Subject: Re: Out of memory (heap space) errors on job tracker References: <4FD219B2.2020309@darose.net> <409DB01E-8AB7-4087-A07E-2E9029EE3510@hortonworks.com> <4FD24D8D.5080209@darose.net> <9A30EA48-62CC-4019-84A6-2DB11F93548A@hortonworks.com> In-Reply-To: <9A30EA48-62CC-4019-84A6-2DB11F93548A@hortonworks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org On 06/10/2012 08:39 PM, Arun C Murthy wrote: > Harsh - I'd be inclined to think it's worse than just setting mapreduce.jobtracker.completeuserjobs.maximum - the only case this would solve is if a single user submitted 25 *large* jobs (in terms of tasks) over a single 24-hr window. That's actually the situation we have - one user, submitting multiple jobs, each with several thousand map steps. I'll give that setting a shot and see if it clears it up. Thanks, DR