Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <33132538.584721280141573683.JavaMail.jira@thor>
Date: Mon, 26 Jul 2010 06:52:53 -0400 (EDT)
From: "Steve Loughran (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-1120) Make DataNode's block-to-device
 placement policy pluggable
In-Reply-To: <31923739.2271272534954514.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-1120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892257#action_12892257 ] 

Steve Loughran commented on HDFS-1120:
--------------------------------------

@Ambikeshwar -you make some good points. I think we should 
# start with some monitoring of what's going on, especially on 6-12 HDD servers
# do some plugin point where people can play with block placement
# Let people with time on their hands do the good plugins
For the complex decisions, you probably do want some history -filename, created, last read, owner. 

I think your point about rebalancing is a good one, it may be better to schedule some rebalancing work on a DN that can spread stuff across the disks than try and do some smarter block placement decisions. If you can rebalance the disks then you can just use roundrobin or roulette placement. This handles the new-HDD problem better, and can allow for the block placer to be simple and fast, rebalancing can be done on selective nodes at a time of choosing.


> Make DataNode's block-to-device placement policy pluggable
> ----------------------------------------------------------
>
>                 Key: HDFS-1120
>                 URL: https://issues.apache.org/jira/browse/HDFS-1120
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node
>            Reporter: Jeff Hammerbacher
>
> As discussed on the mailing list, as the number of disk drives per server increases, it would be useful to allow the DataNode's policy for new block placement to grow in sophistication from the current round-robin strategy.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.