Return-Path: Delivered-To: apmail-lucene-hadoop-dev-archive@locus.apache.org Received: (qmail 28436 invoked from network); 5 Jan 2007 20:42:01 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 5 Jan 2007 20:42:01 -0000 Received: (qmail 15327 invoked by uid 500); 5 Jan 2007 20:42:05 -0000 Delivered-To: apmail-lucene-hadoop-dev-archive@lucene.apache.org Received: (qmail 15262 invoked by uid 500); 5 Jan 2007 20:42:05 -0000 Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-dev@lucene.apache.org Received: (qmail 15235 invoked by uid 99); 5 Jan 2007 20:42:04 -0000 Received: from herse.apache.org (HELO herse.apache.org) (140.211.11.133) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Jan 2007 12:42:03 -0800 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests= X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO brutus.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Jan 2007 12:41:22 -0800 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 8F8E5714311 for ; Fri, 5 Jan 2007 12:06:27 -0800 (PST) Message-ID: <13185273.1168027587569.JavaMail.jira@brutus> Date: Fri, 5 Jan 2007 12:06:27 -0800 (PST) From: "Arun C Murthy (JIRA)" To: hadoop-dev@lucene.apache.org Subject: [jira] Updated: (HADOOP-815) Investigate and fix the extremely large memory-footprint of JobTracker In-Reply-To: <28416645.1165904481055.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-815: --------------------------------- Attachment: HADOOP-815_20070106_6.patch > Investigate and fix the extremely large memory-footprint of JobTracker > ---------------------------------------------------------------------- > > Key: HADOOP-815 > URL: https://issues.apache.org/jira/browse/HADOOP-815 > Project: Hadoop > Issue Type: Bug > Components: mapred > Affects Versions: 0.9.1 > Reporter: Arun C Murthy > Assigned To: Arun C Murthy > Fix For: 0.10.0 > > Attachments: 150k_1199_774.nps, 75k_jobs.nps, HADOOP-815_20061220_1.patch, HADOOP-815_20061221_2.patch, HADOOP-815_20061222_3.patch, HADOOP-815_20061230_4.patch, HADOOP-815_20070105_5.patch, HADOOP-815_20070106_6.patch, jt_memory_profiles.tgz > > > The JobTracker's memory footprint seems excessively large, especially when many jobs are submitted. > Here is the 'top' output of a JobTracker which has scheduled ~1k jobs thus far: > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 31877 arunc 19 0 2362m 261m 13m S 14.0 12.9 24:48.08 java > Clearly VIRTual memory of 2364Mb v/s 261Mb of RESident memory is symptomatic of this issue... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira