hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Surendra Singh Lilhore (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-9355) Support colocation in HDFS.
Date Sat, 31 Oct 2015 12:13:27 GMT
Surendra Singh Lilhore created HDFS-9355:
--------------------------------------------

             Summary: Support colocation in HDFS.
                 Key: HDFS-9355
                 URL: https://issues.apache.org/jira/browse/HDFS-9355
             Project: Hadoop HDFS
          Issue Type: New Feature
          Components: hdfs-client
            Reporter: Surendra Singh Lilhore
            Assignee: Surendra Singh Lilhore


Through this feature client can give suggestion to HDFS to write his all the blocks on same
set of datanodes. Currently this we can achieve through HDFS-2576. HDFS-2576 give option to
hint namenode about favored nodes, but in heterogeneous cluster this will not work out. Support
client wants to write his data in directory which have COLD policy, but he don't know which
DN have ARCHIVE storage, So he will not able to give favoredNodes list. 


*Implementation*

Colocation can enable by setting "dfs.colocation.enable" true in client configuration. If
colocation is enable and  favoredNodes list is empty then {{DataStreamer}} will set first
set of datanodes as favoredNodes which is chosen for first block and subsequent block will
use the same datanodes for write. Before closing file client can get the favoredNodes list
and same he can use for writing new file.
 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message