hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7960) The full block report should prune zombie storages even if they're not empty
Date Fri, 20 Mar 2015 21:52:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14372191#comment-14372191
] 

Colin Patrick McCabe commented on HDFS-7960:
--------------------------------------------

bq. EMPTY_STORAGE_INFO_LIST is a little dangerous since it can still be modified, maybe use
ImmutableLists instead? Alternatively we can just allocate the empty LinkedList, it'll be
young gen and cheap.

Yeah, let's use ImmutableList

bq. double space after equals here

fixed

bq. unnecessary whitespace change?

fixed

Other changes: I thought about this a little more and I think instead of just a boolean to
tell when the last rpc is happening, we should have 3 things: the current unique (across same
DN) 64-bit block report ID, the total number of RPCs in the block report, and the current
rpc in the block report.

Can get a unique ID on the datanode from the monotonic time in nanoseconds.  (If for some
bizarre reason the clock doesn't advance between BRs, we can just add one to the previous
time).  This should also ensure that if we restart the DN, we don't re-use IDs.  The ID should
help match up DN and NN in the logs.

Having a total RPC count and a current RPC index is more flexible if we want to chunk block
report RPCs differently in the future.

> The full block report should prune zombie storages even if they're not empty
> ----------------------------------------------------------------------------
>
>                 Key: HDFS-7960
>                 URL: https://issues.apache.org/jira/browse/HDFS-7960
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>            Reporter: Lei (Eddy) Xu
>            Assignee: Colin Patrick McCabe
>            Priority: Critical
>         Attachments: HDFS-7960.002.patch, HDFS-7960.003.patch
>
>
> The full block report should prune zombie storages even if they're not empty.  We have
seen cases in production where zombie storages have not been pruned subsequent to HDFS-7575.
 This could arise any time the NameNode thinks there is a block in some old storage which
is actually not there.  In this case, the block will not show up in the "new" storage (once
old is renamed to new) and the old storage will linger forever as a zombie, even with the
HDFS-7596 fix applied.  This also happens with datanode hotplug, when a drive is removed.
 In this case, an entire storage (volume) goes away but the blocks do not show up in another
storage on the same datanode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message