hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1120) Make DataNode's block-to-device placement policy pluggable
Date Sat, 26 Mar 2011 21:54:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011690#comment-13011690

Todd Lipcon commented on HDFS-1120:

Mostly looks good - just a few comments:
- when constructing the policy class, you can use conf.getClass(...). That will also have
the behavior that if a non-existent class is specified, it will throw an exception rather
than continuing on, which I think is a good thing.
- For the unit test, can you make mock FSVolumes with mockito rather than starting a minicluster?
Should be pretty simple and the more we can avoid unnecessary miniclusters the faster our
tests will run.
- RoundRobinVolumesPolicy isn't threadsafe since you didn't make chooseVolume synchronized.
This isn't really a problem since the caller is itself synchronized, but I think it would
make more sense from an API perspective to say that policies that keep state should be threadsafe,
and make chooseVolume synchronized.
- I don't think BlockVolumeChoosingPolicy should extend Configurable - this forces you to
add setConf and getConf even though you don't use them. Since we reflectively check for the
Configurable interface in ReflectionUtils.setInstance, it will still work OK for other implementers
who may want configuration.

> Make DataNode's block-to-device placement policy pluggable
> ----------------------------------------------------------
>                 Key: HDFS-1120
>                 URL: https://issues.apache.org/jira/browse/HDFS-1120
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node
>            Reporter: Jeff Hammerbacher
>            Assignee: Harsh J Chouraria
>             Fix For: 0.23.0
>         Attachments: HDFS-1120.r1.diff, HDFS-1120.r2.diff
> As discussed on the mailing list, as the number of disk drives per server increases,
it would be useful to allow the DataNode's policy for new block placement to grow in sophistication
from the current round-robin strategy.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message