hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8782) Upgrade to block ID-based DN storage layout delays DN registration
Date Thu, 31 Mar 2016 09:39:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219664#comment-15219664

Duo Zhang commented on HDFS-8782:


> Upgrade to block ID-based DN storage layout delays DN registration
> ------------------------------------------------------------------
>                 Key: HDFS-8782
>                 URL: https://issues.apache.org/jira/browse/HDFS-8782
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Haohui Mai
>            Priority: Critical
> We have seen multiple incidents at production sites that there are long delays for DNs
to register to the NN when upgrading to post 2.6 release.
> Further investigation shows that the DN is blocked when upgrading the storage layout
introduced in HDFS-6482. The new storage layout requires making up to 64k directories in the
underlying file system. Unfortunately the current implementation calls {{mkdirs()}} sequentially
and upgrades each volume in sequential order.
> As a result, upgrading a DN with a lot of disks or with blocks that have random block
ID takes a long time (usually in hours), and the DN won't register to the NN unless it finishes
upgrading all the storage directory. The excessive delays confuse operations and break the
assumption of rolling upgrades.

This message was sent by Atlassian JIRA

View raw message