hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1094) Intelligent block placement policy to decrease probability of block loss
Date Fri, 09 Jul 2010 03:02:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886595#action_12886595

Hairong Kuang commented on HDFS-1094:

> I don't think I understand your use case. It only seem to be advantageous to do what
you say if there are multiple readers for the same file. We designed it this way because it
would be relatively easy to understand, implement, generalize, and plan for (as users).

You still do not get my point. I think the goal of this policy is to reduce the data loss
by limiting the # of nodes to place a file's data. Is this additional limitation of racks
neccessary? Or are you saying this is just easy for you to implement. I do not see how this
helps users understand or plan. In general, having less configuration parameters is easier
for users to understand or plan.

> Intelligent block placement policy to decrease probability of block loss
> ------------------------------------------------------------------------
>                 Key: HDFS-1094
>                 URL: https://issues.apache.org/jira/browse/HDFS-1094
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>            Reporter: dhruba borthakur
>            Assignee: Rodrigo Schmidt
>         Attachments: prob.pdf, prob.pdf
> The current HDFS implementation specifies that the first replica is local and the other
two replicas are on any two random nodes on a random remote rack. This means that if any three
datanodes die together, then there is a non-trivial probability of losing at least one block
in the cluster. This JIRA is to discuss if there is a better algorithm that can lower probability
of losing a block.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message