hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yiqun Lin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11450) HDFS specific network topology classes with storage type info included
Date Fri, 24 Feb 2017 11:48:44 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15882506#comment-15882506
] 

Yiqun Lin edited comment on HDFS-11450 at 2/24/17 11:48 AM:
------------------------------------------------------------

Thanks for working on this, [~vagarychen]!
I wonder what do you want to do for the case when a Datanodes restarts or a disk is hot-swapped.
Currently namenode doesn't know if datanode has added or removed the disk. So it will let
us hard to sync these infos in {{NetworkTopology}}. One simple way I can imaged is that we
can add a field in {{StorageReport}} to indicate the current storage is new added. And then
we can remove and add Node again into {{NetworkTopology}} during handling heartbeat. Also
we can do this in the heartbeat protocol.


was (Author: linyiqun):
Thanks for working on this, [~vagarychen]!
I wonder what do you want to do for the case when a Datanodes restarts or a disk is hot-swapped.
Currently namenode doesn't know if datanode has added or removed the disk. So it will let
us hard to sync these infos in {{NetworkTopology}}. One simple way I can imaged is that we
can add a field in {{StorageReport}} to indicate the current storage is new added. And then
we can remove and add Node again into {{NetworkTopology}}. Also we can do this in the heartbeat
protocol.
 

> HDFS specific network topology classes with storage type info included
> ----------------------------------------------------------------------
>
>                 Key: HDFS-11450
>                 URL: https://issues.apache.org/jira/browse/HDFS-11450
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Chen Liang
>            Assignee: Chen Liang
>         Attachments: HDFS-11450.001.patch
>
>
> This JIRA adds storage type info into network topology.
> More specifically, this JIRA adds a storage type map by extending {{InnerNodeImpl}} to
describe the available storages under the current node's subtree. This map is updated when
a node is added/removed from the subtree.
> With this info, when choosing a random node with storage type requirement, the search
could then decide to/not to go deeper into a subtree by examining the available storage types
first.
> One to-do item still, is that, we might still need to separately handle the cases where
a Datanodes restarts, or a disk is hot-swapped, will file another JIRA in that case.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message