hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4182) SecondaryNameNode leaks NameCache entries
Date Tue, 13 Nov 2012 21:42:12 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Robert Joseph Evans updated HDFS-4182:
--------------------------------------

    Attachment: HDFS-4182.txt

The patch I am attaching does not include any tests yet.  I wanted to see if the direction
I was going in seemed OK.  I changed FSDirectory.reset to also reset the NameCache and mark
the directory as not ready.

Then in the SecondaryNameNode after loading the new image it informs the FSDirectory that
the image was loaded.

I am going to run some manual tests and then see if I can write some unit tests for it. 
                
> SecondaryNameNode leaks NameCache entries
> -----------------------------------------
>
>                 Key: HDFS-4182
>                 URL: https://issues.apache.org/jira/browse/HDFS-4182
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.23.4, 3.0.0, 2.0.2-alpha
>            Reporter: Todd Lipcon
>            Priority: Critical
>         Attachments: HDFS-4182.txt
>
>
> We recently saw an issue where a 2NN ran out of memory, even though it had a relatively
small fsimage. When we looked at the heap dump, we saw that all of the memory had gone to
entries in the NameCache.
> It appears that the NameCache is staying in "initializing" mode forever, and therefore
a long running 2NN leaks entries.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message