hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Plamen Jeliazkov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-4475) OutOfMemory by BPService.offerService() takes down DataNode
Date Wed, 06 Feb 2013 21:47:12 GMT
Plamen Jeliazkov created HDFS-4475:

             Summary: OutOfMemory by BPService.offerService() takes down DataNode
                 Key: HDFS-4475
                 URL: https://issues.apache.org/jira/browse/HDFS-4475
             Project: Hadoop HDFS
          Issue Type: Bug
    Affects Versions: 3.0.0, 2.0.3-alpha
            Reporter: Plamen Jeliazkov
            Assignee: Plamen Jeliazkov
             Fix For: 3.0.0, 2.0.3-alpha

In DataNode, there are catchs around BPService.offerService() call but no catch for OutOfMemory
as there is for the DataXeiver as introduced in 0.22.0.

The issue can be replicated like this:
1) Create a cluster of X DataNodes and 1 NameNode and low memory settings (-Xmx128M or something
2) Flood HDFS with of file creation.
3) DataNodes will hit OoM, stop blockpool service, and shutdown.

The resolution is to catch the OoMException and handle it properly when calling BlockPool.offerService()
in DataNode.java; like as done in 0.22.0 of Hadoop. DataNodes should not shutdown or crash
but remain in a sort of frozen state until memory issues are resolved by GC.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message