hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amos Bird (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-11256) Rebalance specific directory
Date Fri, 16 Dec 2016 08:44:58 GMT
Amos Bird created HDFS-11256:
--------------------------------

             Summary: Rebalance specific directory
                 Key: HDFS-11256
                 URL: https://issues.apache.org/jira/browse/HDFS-11256
             Project: Hadoop HDFS
          Issue Type: New Feature
          Components: balancer & mover
    Affects Versions: 3.0.0-alpha1
            Reporter: Amos Bird
             Fix For: 3.0.0-alpha1


Currently hdfs only supports rebalancing over entire cluster. This might not be utilized by
data processing systems like hive, spark, impala etl. 

In hive, we may need to maximize some fact tables IO performance by carefully sharding their
blocks evenly over all disks. Normally a INSERT SELECT is done to achieve such redistributing
progress. 

Given a table is backed by one directory on hdfs, rebalancing specific dir may be very useful.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message