hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewan Higgs (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9807) Add an optional StorageID to writes
Date Tue, 02 May 2017 07:44:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15992494#comment-15992494
] 

Ewan Higgs commented on HDFS-9807:
----------------------------------

Thanks [~virajith]! This is good stuff. 

In your solution using {{BlockPlacementPolicy}} and a static variable to record the block,
I had considered such a solution but was concerned that this will break when running tests
in parallel if we have multiple test cases. As it stands, we only have the one test so it
should be fine; but if someone adds another test using the same {{BlockPlacementPolicy}} there
is a risk that we introduce some flakiness into the testing infrastructure.

As per your previous question about why I had removed {{final}} from arguments in a function.
This is because checkstyle complained that I had added redundant {{final}} decls to a function
argument, and I figured that if someone deigned to turn that warning on in checkstyle then
I should fix it where I'm touching the code.

For other people watching along: findbugs is bugged; the issues reported are not touched/introduced
by this patch afaict. Same with the unit tests.

> Add an optional StorageID to writes
> -----------------------------------
>
>                 Key: HDFS-9807
>                 URL: https://issues.apache.org/jira/browse/HDFS-9807
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Chris Douglas
>            Assignee: Ewan Higgs
>         Attachments: HDFS-9807.001.patch, HDFS-9807.002.patch, HDFS-9807.003.patch, HDFS-9807.004.patch,
HDFS-9807.005.patch, HDFS-9807.006.patch, HDFS-9807.007.patch, HDFS-9807.008.patch, HDFS-9807.009.patch
>
>
> The {{BlockPlacementPolicy}} considers specific storages, but when the replica is written
the DN {{VolumeChoosingPolicy}} is unaware of any preference or constraints from other policies
affecting placement. This limits heterogeneity to the declared storage types, which are treated
as fungible within the target DN. It should be possible to influence or constrain the DN policy
to select a particular storage.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message