Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Mon, 21 Mar 2016 20:42:25 +0000 (UTC)
From: "Tsz Wo Nicholas Sze (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12599835.1343040854000.9445.1458592945821@Atlassian.JIRA>
In-Reply-To: <JIRA.12599835.1343040854000@Atlassian.JIRA>
References: <JIRA.12599835.1343040854000@Atlassian.JIRA>
 <JIRA.12599835.1343040854884@arcas>
Subject: [jira] [Commented] (HDFS-3702) Add an option for NOT writing the
 blocks locally if there is a datanode on the same box as the client
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205079#comment-15205079 ] 

Tsz Wo Nicholas Sze commented on HDFS-3702:
-------------------------------------------

> So, the suggestion is changing public signatures to add a new parameter (Or adding a new override where there are already 6)?

For compatibility reason, we probably have to add a new override.  For better usability, we may add a Builder.

> For a client to make effective use of disfavoredNodes, they would have to figure the exact name the NN is using and volunteer it in this disfavoredNodes list? Or could they just write 'localhost' and let NN figure it out?

We should support 'localhost' in the API.  DFSClient or NN may replace 'localhost' with the corresponding name.

> Do you foresee any other use for this disfavoredNodes parameter other than for the exclusion of 'localnode'?

Yes, disfavoredNodes seems useful.  For example, some application may want to distribute its files uniformly in a cluster.  Then, it could specify the previously allocated DNs as the disfavoredNodes.

> Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client
> -----------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3702
>                 URL: https://issues.apache.org/jira/browse/HDFS-3702
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>    Affects Versions: 2.5.1
>            Reporter: Nicolas Liochon
>            Assignee: Lei (Eddy) Xu
>            Priority: Minor
>              Labels: BB2015-05-TBR
>         Attachments: HDFS-3702.000.patch, HDFS-3702.001.patch, HDFS-3702.002.patch, HDFS-3702.003.patch, HDFS-3702.004.patch, HDFS-3702.005.patch, HDFS-3702.006.patch, HDFS-3702.007.patch, HDFS-3702.008.patch, HDFS-3702_Design.pdf
>
>
> This is useful for Write-Ahead-Logs: these files are writen for recovery only, and are not read when there are no failures.
> Taking HBase as an example, these files will be read only if the process that wrote them (the 'HBase regionserver') dies. This will likely come from a hardware failure, hence the corresponding datanode will be dead as well. So we're writing 3 replicas, but in reality only 2 of them are really useful.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)