hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thanh Do (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6009) Tools based on favored node feature for isolation
Date Wed, 12 Mar 2014 03:13:45 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931338#comment-13931338
] 

Thanh Do commented on HDFS-6009:
--------------------------------

Hi Yu, 

You mentioned "although the regionservers are grouped, the datanodes which store the data
are not, which leads to the case that one datanode failure affects multiple applications,
as we already observed in our product environment".

Can you elaborate that scenarios? I thought a datanode failure will be ok, as the data are
replicated. 

Best,

> Tools based on favored node feature for isolation
> -------------------------------------------------
>
>                 Key: HDFS-6009
>                 URL: https://issues.apache.org/jira/browse/HDFS-6009
>             Project: Hadoop HDFS
>          Issue Type: Task
>    Affects Versions: 2.3.0
>            Reporter: Yu Li
>            Assignee: Yu Li
>            Priority: Minor
>
> There're scenarios like mentioned in HBASE-6721 and HBASE-4210 that in multi-tenant deployments
of HBase we prefer to specify several groups of regionservers to serve different applications,
to achieve some kind of isolation or resource allocation. However, although the regionservers
are grouped, the datanodes which store the data are not, which leads to the case that one
datanode failure affects multiple applications, as we already observed in our product environment.
> To relieve the above issue, we could take usage of the favored node feature (HDFS-2576)
to make regionserver able to locate data within its group, or say make datanodes also grouped
(passively), to form some level of isolation.
> In this case, or any other case that needs datanodes to group, we would need a bunch
of tools to maintain the "group", including:
> 1. Making balancer able to balance data among specified servers, rather than the whole
set
> 2. Set balance bandwidth for specified servers, rather than the whole set
> 3. Some tool to check whether the block is "cross-group" placed, and move it back if
so
> This JIRA is an umbrella for the above tools.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message