hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6482) Use block ID-based block layout on datanodes
Date Thu, 31 Jul 2014 22:39:40 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14081618#comment-14081618
] 

Colin Patrick McCabe commented on HDFS-6482:
--------------------------------------------

Why don't we merge this to trunk and then open another JIRA to iron out any issues with rolling
upgrades between different DN layout versions.  At minimum, we should decide whether we support
rolling DN upgrades between different layout versions, and if we don't support it, give a
clear failure message to admins.  But this patch is big enough that I don't think cramming
all that into here is a good idea.  There also seem to be some issues with rolling DN downgrade
now (for example, HDFS-6005 removed {{datanode \-rollingupgrade \-rollback}}, but not the
usage text for it displayed in {{\-help}}.)

> Use block ID-based block layout on datanodes
> --------------------------------------------
>
>                 Key: HDFS-6482
>                 URL: https://issues.apache.org/jira/browse/HDFS-6482
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 2.5.0
>            Reporter: James Thomas
>            Assignee: James Thomas
>         Attachments: 6482-design.doc, HDFS-6482.1.patch, HDFS-6482.2.patch, HDFS-6482.3.patch,
HDFS-6482.4.patch, HDFS-6482.5.patch, HDFS-6482.6.patch, HDFS-6482.7.patch, HDFS-6482.8.patch,
HDFS-6482.9.patch, HDFS-6482.patch, hadoop-24-datanode-dir.tgz
>
>
> Right now blocks are placed into directories that are split into many subdirectories
when capacity is reached. Instead we can use a block's ID to determine the path it should
go in. This eliminates the need for the LDir data structure that facilitates the splitting
of directories when they reach capacity as well as fields in ReplicaInfo that keep track of
a replica's location.
> An extension of the work in HDFS-3290.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message