hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
Date Tue, 25 Jun 2013 00:12:20 GMT
Sandy Ryza created MAPREDUCE-5351:

             Summary: JobTracker memory leak caused by CleanupQueue reopening FileSystem
                 Key: MAPREDUCE-5351
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: jobtracker
    Affects Versions: 1.1.2
            Reporter: Sandy Ryza
            Assignee: Sandy Ryza
            Priority: Critical

When a job is completed, closeAllForUGI is called to close all the cached FileSystems in the
FileSystem cache.  However, the CleanupQueue may run after this occurs and call FileSystem.get()
to delete the staging directory, adding a FileSystem to the cache that will never be closed.

People on the user-list have reported this causing their JobTrackers to OOME every two weeks.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message