hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu Li (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-6009) Tools based on favored node feature for isolation
Date Tue, 25 Feb 2014 02:34:19 GMT
Yu Li created HDFS-6009:

             Summary: Tools based on favored node feature for isolation
                 Key: HDFS-6009
                 URL: https://issues.apache.org/jira/browse/HDFS-6009
             Project: Hadoop HDFS
          Issue Type: New Feature
    Affects Versions: 2.3.0
            Reporter: Yu Li
            Assignee: Yu Li
            Priority: Minor

There're scenarios like mentioned in HBASE-6721 and HBASE-4210 that in multi-tenant deployments
of HBase we prefer to specify several groups of regionservers to serve different applications,
to achieve some kind of isolation or resource allocation. However, although the regionservers
are grouped, the datanodes which store the data are not, which leads to the case that one
datanode failure affects multiple applications, as we already observed in our product environment.

To relieve the above issue, we could take usage of the favored node feature (HDFS-2576) to
make regionserver able to locate data within its group, or say make datanodes also grouped
(passively), to form some level of isolation.

In this case, or any other case that needs datanodes to group, we would need a bunch of tools
to maintain the "group", including:
1. Making balancer able to balance data among specified servers, rather than the whole set
2. Set balance bandwidth for specified servers, rather than the whole set
3. Some tool to check whether the block is "cross-group" placed, and move it back if so

This JIRA is an umbrella for the above tools.

This message was sent by Atlassian JIRA

View raw message