hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doris Gu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-10604) What about this?Group DNs and add DN groups--named region to HDFS model , use this region to instead of single DN when saving files.
Date Mon, 11 Jul 2016 07:56:11 GMT
Doris Gu created HDFS-10604:
-------------------------------

             Summary: What about this?Group DNs and add DN groups--named region to HDFS model
, use this region to instead of single DN when saving files.
                 Key: HDFS-10604
                 URL: https://issues.apache.org/jira/browse/HDFS-10604
             Project: Hadoop HDFS
          Issue Type: Wish
            Reporter: Doris Gu


The biggest difference this feature will bring is *strong* making blocks belong to the same
file to save in the same region(DN group).*strong*
So the process will be:
1.Config DN groups, for example
bq.Region1:dn1,dn2,dn3
bq.Region2:dn4,dn5,dn6
bq.Region3:dn7,dn8,dn9,dn10

2.Client uploads a file, first analyze whether this file has any existed blocks:
bq.i)Yes:assign new blocks to the DN group where the existed blocks belong to.
bq.ii)No:assign new blocks to a DN group which is chosen by some certain policy to avoid imbalance.

3.Other related processes,including append,balancer etc. also need to modify as well.   

The benefit we wish is when some DNs are down at the same time, the number of affected files(miss
all replicas) is small.
But we are wondering if this is worth doing or not, or if there are problems we haven't noticed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message