hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J Chouraria (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1120) Make DataNode's block-to-device placement policy pluggable
Date Fri, 25 Mar 2011 15:01:07 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13011229#comment-13011229

Harsh J Chouraria commented on HDFS-1120:

I've done some initial work for making the volume-block choosing policy pluggable (so that
methods other than round-robin may be provided).

The following is my initial interface design, and am looking for comments/critique/etc. w.r.t.
this ticket's scope, before I start pushing out some tests + patches:

 * BlockVolumeChoosingPolicy allows a DataNode to
 * specify what policy is to be used while choosing
 * a volume for a block request.
public interface BlockVolumeChoosingPolicy extends Configurable {

  * Returns a specific FSVolume after applying a suitable choice algorithm
  * to place a given block, given a list of FSVolumes and the block
  * size sought for storage.
  * @param volumes - the array of FSVolumes that are available.
  * @param blockSize - the size of the block for which a volume is sought.
  * @return the chosen volume to store the block.
  * @throws IOException when disks are unavailable or are full.
 public FSVolume chooseVolume(FSVolume[] volumes, long blockSize)
   throws IOException;


This can be neatly used within FSVolumeSet.getNextVolume() [Maybe this too needs to be renamed,
since it may not make sense as 'next' once it becomes pluggable]

Looking forward to a discussion.

P.s. I also posted this to hdfs-dev, but I guess most of Hadoop's devel discussions occur
on the JIRA itself?

> Make DataNode's block-to-device placement policy pluggable
> ----------------------------------------------------------
>                 Key: HDFS-1120
>                 URL: https://issues.apache.org/jira/browse/HDFS-1120
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node
>            Reporter: Jeff Hammerbacher
>            Assignee: Harsh J Chouraria
> As discussed on the mailing list, as the number of disk drives per server increases,
it would be useful to allow the DataNode's policy for new block placement to grow in sophistication
from the current round-robin strategy.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message