Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <30582303.1166612482495.JavaMail.jira@brutus>
Date: Wed, 20 Dec 2006 03:01:22 -0800 (PST)
From: "Arun C Murthy (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Updated: (HADOOP-815) Investigate and fix the extremely
 large memory-footprint of JobTracker
In-Reply-To: <28416645.1165904481055.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

     [ http://issues.apache.org/jira/browse/HADOOP-815?page=all ]

Arun C Murthy updated HADOOP-815:
---------------------------------

    Attachment: HADOOP-815_20061220_1.patch

Here is a very early patch while I keep testing; appreciate any feedback... 

> Investigate and fix the extremely large memory-footprint of JobTracker
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-815
>                 URL: http://issues.apache.org/jira/browse/HADOOP-815
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Arun C Murthy
>         Assigned To: Arun C Murthy
>             Fix For: 0.10.0
>
>         Attachments: 150k_1199_774.nps, 75k_jobs.nps, HADOOP-815_20061220_1.patch
>
>
> The JobTracker's memory footprint seems excessively large, especially when many jobs are submitted.
> Here is the 'top' output of a JobTracker which has scheduled ~1k jobs thus far:
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                     
> 31877 arunc     19   0 2362m 261m  13m S 14.0 12.9  24:48.08 java  
> Clearly VIRTual memory of 2364Mb v/s 261Mb of RESident memory is symptomatic of this issue...

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira