hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjun Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6133) Make Balancer support exclude specified path
Date Sat, 07 Feb 2015 04:05:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310533#comment-14310533

Yongjun Zhang commented on HDFS-6133:

Thanks Yunjiong for the continuous effort on this work, and Nicholas for further reviewing!

BTW, just heard from Chris Nauroth that Windows does not support sticky bit. Is there some
other ways to implement block pinning? Use file name?

[~daryn] pointed out some options earlier:

Other approaches might be using the sticky bit or the existence of a second file to denote
a pinned block.

I guess using existence of a second file is easier to be compatible across different platforms
(though sticky bit has the advantage of storing the info at the same file). 

Another option is to use a mixed mechanism, say, if it's on linux, use sticky bit, otherwise,
use existence of a second file. This method means lack of consistency between different platforms.
Just wanted to throw a thought here.


> Make Balancer support exclude specified path
> --------------------------------------------
>                 Key: HDFS-6133
>                 URL: https://issues.apache.org/jira/browse/HDFS-6133
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: balancer & mover, namenode
>            Reporter: zhaoyunjiong
>            Assignee: zhaoyunjiong
>         Attachments: HDFS-6133-1.patch, HDFS-6133-2.patch, HDFS-6133-3.patch, HDFS-6133-4.patch,
HDFS-6133-5.patch, HDFS-6133-6.patch, HDFS-6133-7.patch, HDFS-6133-8.patch, HDFS-6133.patch
> Currently, run Balancer will destroying Regionserver's data locality.
> If getBlocks could exclude blocks belongs to files which have specific path prefix, like
"/hbase", then we can run Balancer without destroying Regionserver's data locality.

This message was sent by Atlassian JIRA

View raw message