Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <27084347.51281271180479324.JavaMail.jira@thor>
Date: Tue, 13 Apr 2010 13:41:19 -0400 (EDT)
From: "Karthik Ranganathan (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-1094) Intelligent block placement policy to
 decrease probability of block loss
In-Reply-To: <24967451.18401271056063147.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12856510#action_12856510 ] 

Karthik Ranganathan commented on HDFS-1094:
-------------------------------------------

Hey Brian,

Yes, that would help with the data loss case but has some unfortunate side effects:

1. Data availability would be reduced - cannot access the data at all on inter-rack switch failure.
2. It would cause the data to be very unevenly distributed across the various racks.

Another workaround for this is to have many smaller clusters instead of one large cluster, but that becomes a maintenance nightmare (operationally), and the application above should balance the data across these clusters.


> Intelligent block placement policy to decrease probability of block loss
> ------------------------------------------------------------------------
>
>                 Key: HDFS-1094
>                 URL: https://issues.apache.org/jira/browse/HDFS-1094
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>
> The current HDFS implementation specifies that the first replica is local and the other two replicas are on any two random nodes on a random remote rack. This means that if any three datanodes die together, then there is a non-trivial probability of losing at least one block in the cluster. This JIRA is to discuss if there is a better algorithm that can lower probability of losing a block.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira