hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yiqun Lin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9807) Add an optional StorageID to writes
Date Thu, 04 May 2017 16:38:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997038#comment-15997038
] 

Yiqun Lin commented on HDFS-9807:
---------------------------------

Thanks [~ehiggs] for the working on this. It's a nice work! I take a quick look on this and
just have some questions.
For the interface of VolumeChoosingPolicy#chooseVolume, how to decide if it should respect
the storageID chosen from namenode or just ignore this? A new setting to control this?
In addition, if VolumeChoosingPolicy can respect the chosen from namenode after this patch,
is that mean we should Improve the selection in choosing storage for blocks that did in {{DatanodeDescriptor#chooseStorage4Block}}.
Under the currently logic, it will always return the first valid storage for block writing,
I have did this improvement in HDFS-11464. I think that JIRA should be reopened and will make
sense now, right?
Thanks for the any comments.

> Add an optional StorageID to writes
> -----------------------------------
>
>                 Key: HDFS-9807
>                 URL: https://issues.apache.org/jira/browse/HDFS-9807
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Chris Douglas
>            Assignee: Ewan Higgs
>         Attachments: HDFS-9807.001.patch, HDFS-9807.002.patch, HDFS-9807.003.patch, HDFS-9807.004.patch,
HDFS-9807.005.patch, HDFS-9807.006.patch, HDFS-9807.007.patch, HDFS-9807.008.patch, HDFS-9807.009.patch
>
>
> The {{BlockPlacementPolicy}} considers specific storages, but when the replica is written
the DN {{VolumeChoosingPolicy}} is unaware of any preference or constraints from other policies
affecting placement. This limits heterogeneity to the declared storage types, which are treated
as fungible within the target DN. It should be possible to influence or constrain the DN policy
to select a particular storage.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message