hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2452) OutOfMemoryError in DataXceiverServer takes down the DataNode
Date Mon, 17 Oct 2011 20:07:10 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13129130#comment-13129130
] 

Konstantin Shvachko commented on HDFS-2452:
-------------------------------------------

- Uma, yes let's split it into 2 patches: one for HDFS-2286. I'll commit it first, then this
one. Otherwise things will get messy.
- The fix looks good. A suggestion for better logging and commenting. In the comment you can
say something like "// DataNode can run out of memory if there is too many transfers. Log
the event, sleep for 30 seconds, other transfers may complete be then."
In log message you can say something like LOG.warn("DataNode is out of memory. Will retry
in 30 seconds.", e);
- For the test. You actually need to mock Daemon.start() method if possible. In your patch
OutOfMemoryError comes from DataXceiver() constructor. In the exception I posted it is thrown
in Thread.start().
                
> OutOfMemoryError in DataXceiverServer takes down the DataNode
> -------------------------------------------------------------
>
>                 Key: HDFS-2452
>                 URL: https://issues.apache.org/jira/browse/HDFS-2452
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>    Affects Versions: 0.22.0
>            Reporter: Konstantin Shvachko
>            Assignee: Uma Maheswara Rao G
>             Fix For: 0.22.0
>
>         Attachments: HDFS-2452-22branch.patch
>
>
> OutOfMemoryError brings down DataNode, when DataXceiverServer tries to spawn a new data
transfer thread.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message