Return-Path: X-Original-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E7B6410478 for ; Tue, 2 Jul 2013 18:01:26 +0000 (UTC) Received: (qmail 57889 invoked by uid 500); 2 Jul 2013 18:01:26 -0000 Delivered-To: apmail-hadoop-mapreduce-issues-archive@hadoop.apache.org Received: (qmail 57839 invoked by uid 500); 2 Jul 2013 18:01:26 -0000 Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-issues@hadoop.apache.org Delivered-To: mailing list mapreduce-issues@hadoop.apache.org Received: (qmail 57825 invoked by uid 99); 2 Jul 2013 18:01:26 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Jul 2013 18:01:26 +0000 Date: Tue, 2 Jul 2013 18:01:26 +0000 (UTC) From: "Sandy Ryza (JIRA)" To: mapreduce-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/MAPREDUCE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698043#comment-13698043 ] Sandy Ryza commented on MAPREDUCE-5351: --------------------------------------- [~arpitgupta], how often is this occurring for you? > JobTracker memory leak caused by CleanupQueue reopening FileSystem > ------------------------------------------------------------------ > > Key: MAPREDUCE-5351 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker > Affects Versions: 1.1.2 > Reporter: Sandy Ryza > Assignee: Sandy Ryza > Priority: Critical > Fix For: 1.2.1 > > Attachments: MAPREDUCE-5351-1.patch, MAPREDUCE-5351-2.patch, MAPREDUCE-5351.patch > > > When a job is completed, closeAllForUGI is called to close all the cached FileSystems in the FileSystem cache. However, the CleanupQueue may run after this occurs and call FileSystem.get() to delete the staging directory, adding a FileSystem to the cache that will never be closed. > People on the user-list have reported this causing their JobTrackers to OOME every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira