hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Danny Becker (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-14069) Better debuggability for datanode decommissioning
Date Mon, 03 Dec 2018 21:16:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Danny Becker updated HDFS-14069:
--------------------------------
    Attachment:     (was: HDFS-14069.002.patch)

> Better debuggability for datanode decommissioning
> -------------------------------------------------
>
>                 Key: HDFS-14069
>                 URL: https://issues.apache.org/jira/browse/HDFS-14069
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, hdfs, namenode
>            Reporter: Danny Becker
>            Assignee: Danny Becker
>            Priority: Major
>         Attachments: HDFS-14069.000.patch, HDFS-14069.001.patch, HDFS-14069.002.patch
>
>
> Currently, we don't provide any debugging info for decommissioning DN, it is difficult
to determine which blocks are on their last replica. We have two design options:
>  # Add block info for blocks with low replication (configurable)
>  ** Advantages:
>  *** Initial debugging information would be more thorough
>  *** Easier initial implementation
>  ** Disadvantages:
>  *** Add load to normal NN operation by checking every time a DN is decommissioned
>  *** More difficult to add debugging information later on
>  # Create a new api for querying more detailed info about one DN
>  ** Advantages:
>  *** We wouldnt be adding more load to the NN in normal operation
>  *** Much easier to extend in the future with more info
>  ** Disadvantages:
>  *** Getting the info on demand for this case will be much more expensive actually, cause
we will have to find all the blocks on that DN, and then go through all the blocks again and
count how many replicas we have etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message