hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2832) Enable support for heterogeneous storages in HDFS
Date Tue, 27 Aug 2013 11:38:52 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751195#comment-13751195

Junping Du commented on HDFS-2832:

Thanks for quickly reply. [~arpitagarwal].
bq. We have not considered this use case. Are you running multiple DataNodes over the same
NAS for redundancy?
I have two use cases in my mind: 
1. As one kind of DR solution, user can choose to put 1 or 2 replica on a remote reliable
storage (i.e. NAS backed with SAN). Multiple DNs connect to NAS will have more bandwidth.
2. In virtualization case, some virtual machines are backed with cluster FS (i.e. VMFS) on
shared storage rather than local disks (it may not be the most cost-effective way but not
corner case in enterprise environment). Some DNs on VMs could be backed with same shared storage.
bq. Please feel free to file a Jira with your feature idea and motivation and we can discuss
how to proceed after we have initial level of support for Heterogeneous Storages. Sound good?
Yes. That make sense. Will file JIRA later.
> Enable support for heterogeneous storages in HDFS
> -------------------------------------------------
>                 Key: HDFS-2832
>                 URL: https://issues.apache.org/jira/browse/HDFS-2832
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>    Affects Versions: 0.24.0
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: 20130813-HeterogeneousStorage.pdf
> HDFS currently supports configuration where storages are a list of directories. Typically
each of these directories correspond to a volume with its own file system. All these directories
are homogeneous and therefore identified as a single storage at the namenode. I propose, change
to the current model where Datanode * is a * storage, to Datanode * is a collection * of strorages.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message