hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewan Higgs (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9807) Add an optional StorageID to writes
Date Thu, 04 May 2017 08:02:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996319#comment-15996319

Ewan Higgs commented on HDFS-9807:

{quote}The changes to Host2NodesMap and BlockPlacementPolicyDefault are unrelated?{quote}

The change to {{Host2NodesMap}} were from when I tried to add a custom {{BlockPlacementPolicy}}
based on the top level abstract class. This requires overloading {{initialize}} which uses
{{Host2NodesMap}} as an argument: 
  protected abstract void initialize(Configuration conf,  FSClusterStats stats,
                                     NetworkTopology clusterMap, 
                                     Host2NodesMap host2datanodeMap);

[~virajith] chose the superior solution of merely extending {{BlockPlacementPolicyDefault}}
which already implements {{initialize}}. FWIW, if {{BlockPlacementPolicy}} is public then
all types used in the interface should probably also be public.

Re: {{BlockPlacementPolicyDefault}} changes, yes those are unrelated whitespace changes.

> Add an optional StorageID to writes
> -----------------------------------
>                 Key: HDFS-9807
>                 URL: https://issues.apache.org/jira/browse/HDFS-9807
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Chris Douglas
>            Assignee: Ewan Higgs
>         Attachments: HDFS-9807.001.patch, HDFS-9807.002.patch, HDFS-9807.003.patch, HDFS-9807.004.patch,
HDFS-9807.005.patch, HDFS-9807.006.patch, HDFS-9807.007.patch, HDFS-9807.008.patch, HDFS-9807.009.patch
> The {{BlockPlacementPolicy}} considers specific storages, but when the replica is written
the DN {{VolumeChoosingPolicy}} is unaware of any preference or constraints from other policies
affecting placement. This limits heterogeneity to the declared storage types, which are treated
as fungible within the target DN. It should be possible to influence or constrain the DN policy
to select a particular storage.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message