hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe Pallas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9807) Add an optional StorageID to writes
Date Wed, 22 Feb 2017 18:06:44 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878891#comment-15878891

Joe Pallas commented on HDFS-9807:

How do you envision this interacting with {{VolumeChoosingPolicy}}?  Would it go away completely,
since the policy then effectively moves to the NN?  Should a DN be able to override the client's
choice of volume?

How would this fit in with federation?  With multiple block pools, a given NN has incomplete
information about the storage being managed by a datanode.  Would the name node be able to
make good decisions with the information that it has available?

> Add an optional StorageID to writes
> -----------------------------------
>                 Key: HDFS-9807
>                 URL: https://issues.apache.org/jira/browse/HDFS-9807
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Chris Douglas
>            Assignee: Ewan Higgs
> The {{BlockPlacementPolicy}} considers specific storages, but when the replica is written
the DN {{VolumeChoosingPolicy}} is unaware of any preference or constraints from other policies
affecting placement. This limits heterogeneity to the declared storage types, which are treated
as fungible within the target DN. It should be possible to influence or constrain the DN policy
to select a particular storage.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message