hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewan Higgs (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9807) Add an optional StorageID to writes
Date Thu, 04 May 2017 22:05:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997509#comment-15997509

Ewan Higgs commented on HDFS-9807:

{quote}Posted a new patch (v010) that reverts the unrelated changes to Host2NodesMap and BlockPlacementPolicyDefault
as Chris Douglas pointed out. Ewan Higgs, are you ok with this? The changes to those classes
in the earlier patches don't seem needed. 
Yes, that's fine. The reason Host2NodesMap was public was because I had originally made the
test you added with {{BlockPlacementPolicy}}. That requires {{Host2NodesMap}} in the {{initialize}}
function. I like your solution of using {{BlockPlacementPolicyDefault}} better.

[~linyiqun], [~virajith]'s explanation is spot on. Most notably: 
The goal of this JIRA was to only provide the plumbing needed to propagate the storageID to
the VolumeChoosingPolicy and not to implement a new VolumeChoosingPolicy. The actual policies
to use can be determined separately.

> Add an optional StorageID to writes
> -----------------------------------
>                 Key: HDFS-9807
>                 URL: https://issues.apache.org/jira/browse/HDFS-9807
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Chris Douglas
>            Assignee: Ewan Higgs
>         Attachments: HDFS-9807.001.patch, HDFS-9807.002.patch, HDFS-9807.003.patch, HDFS-9807.004.patch,
HDFS-9807.005.patch, HDFS-9807.006.patch, HDFS-9807.007.patch, HDFS-9807.008.patch, HDFS-9807.009.patch,
> The {{BlockPlacementPolicy}} considers specific storages, but when the replica is written
the DN {{VolumeChoosingPolicy}} is unaware of any preference or constraints from other policies
affecting placement. This limits heterogeneity to the declared storage types, which are treated
as fungible within the target DN. It should be possible to influence or constrain the DN policy
to select a particular storage.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message