Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 23 Jul 2014 06:50:38 +0000 (UTC)
From: "Tsz Wo Nicholas Sze (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12727416.1405448388549.27209.1406098238936@arcas>
In-Reply-To: <JIRA.12727416.1405448388549@arcas>
References: <JIRA.12727416.1405448388549@arcas>
Subject: [jira] [Commented] (HDFS-6686) Archival Storage: Use fallback
 storage types
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14071430#comment-14071430 ] 

Tsz Wo Nicholas Sze commented on HDFS-6686:
-------------------------------------------

Vinay,
# Good catch.  We should pass excludedNodes.
# For getAdditionalDatanode(), since the write is already started, there are already some data in the block.  So using replication fallback is correct.  Creation fallback usually is a subset of replication fallback since if there are not enough storages available, we may fails creation.  However, if we fail replication, it may result in data loss.

Arpit,
# Good catch.  We need to pass newBlock as a parameter.
# numOfResults indeed is the number of existing replicas (numOfResults is used to determine whether local host/local rack/remote rack should be chosen from).  numOfReplicas (call it n) is the number of replicas to be chosen.  In our case, we try to select n storage types but we may only able to get m < n, where m = storageTypes.size().  So we should update numOfReplicas.

Thanks both of you for the careful reviews!

> Archival Storage: Use fallback storage types
> --------------------------------------------
>
>                 Key: HDFS-6686
>                 URL: https://issues.apache.org/jira/browse/HDFS-6686
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>         Attachments: h6686_20140721.patch, h6686_20140721c.patch
>
>
> HDFS-6671 changes replication monitor to use block storage policy for replication.  It should also use the fallback storage types when a particular type of storage is full.


--
This message was sent by Atlassian JIRA
(v6.2#6252)