hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4664) HDFS for heterogeneous environment
Date Wed, 03 Apr 2013 19:59:15 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13621269#comment-13621269
] 

Steve Loughran commented on HDFS-4664:
--------------------------------------

Hadoop 1.3+ has the notion of "failure domains", as you can specify a topology which declares
that some nodes do not have independent failure, and that data should not be replicated more
than once on it.

Please look at HADOOP-8468 and see if that works -and if not, that should be the starting
point for changing things. 

Assuming this JIRA is only about that, long-lived vs short lived DNs, how about changing the
title to something like "HDFS to support short-lived datanodes", as that applies in more cases,
such as in-cloud deployments with spot-priced nodes
                
> HDFS for heterogeneous environment
> ----------------------------------
>
>                 Key: HDFS-4664
>                 URL: https://issues.apache.org/jira/browse/HDFS-4664
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: balancer
>    Affects Versions: 0.20.2
>         Environment: Ubuntu Linux, Institutional lab 
>            Reporter: Mohammad Mustaqeem
>
> I want to use HDFS for storing the files in the institutional labs.
> Here the point is to be noted that all the nodes in the labs are not of same type that
is some nodes stay on for longer duration while some for small duration.
> In addition to this all the labs are not same means that some lab has UPS facility and
some has more nodes.
> If I consider the lab as rack, then we should not choose the racks and node randomly
in replica placement.
> We should give more priority to those nodes that stay on for longer duration and to those
lab which has lab facility and has more system.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message