hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7225) Failed DataNode lookup can crash NameNode with NullPointerException
Date Mon, 20 Oct 2014 18:57:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177307#comment-14177307
] 

Andrew Wang commented on HDFS-7225:
-----------------------------------

Nice examination here Zhe. One high-level question though, could we simplify the above by
cleaning InvalidateBlocks immediately upon seeing the new datanodeUuid? If the old volume
is brought back, the old blocks will be in the block report and the NN will re-populate InvalidateBlocks
as needed when it processes the report.

> Failed DataNode lookup can crash NameNode with NullPointerException
> -------------------------------------------------------------------
>
>                 Key: HDFS-7225
>                 URL: https://issues.apache.org/jira/browse/HDFS-7225
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.6.0
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-7225-v1.patch
>
>
> {{BlockManager#invalidateWorkForOneNode}} looks up a DataNode by the {{datanodeUuid}}
and passes the resultant {{DatanodeDescriptor}} to {{InvalidateBlocks#invalidateWork}}. However,
if a wrong or outdated {{datanodeUuid}} is used, a null pointer will be passed to {{invalidateWork}}
which will use it to lookup in a {{TreeMap}}. Since the key type is {{DatanodeDescriptor}},
key comparison is based on the IP address. A null key will crash the NameNode with an NPE.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message