hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brahma Reddy Battula (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9902) dfs.datanode.du.reserved should be difference between StorageType DISK and RAM_DISK
Date Mon, 07 Mar 2016 13:12:40 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182982#comment-15182982

Brahma Reddy Battula commented on HDFS-9902:

[~panyuxuan] thanks for reporting this issue..

I feel, we can introduce a general config, which can be used for all storagetypes individually.

* Configuration can be like {{dfs.datanode.du.reserved.<storage-type>}} ,storage-type
can be lowercase
   ** Conf for {color:blue}RAM_DISK{color} is {{dfs.datanode.du.reserved.ram_disk}} and 
     ** for {color:blue}SSD{color} is {{dfs.datanode.du.reserved.ssd}}

and consider the default value as value of {{dfs.datanode.du.reserved}}.

> dfs.datanode.du.reserved should be difference between StorageType DISK and RAM_DISK
> -----------------------------------------------------------------------------------
>                 Key: HDFS-9902
>                 URL: https://issues.apache.org/jira/browse/HDFS-9902
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 2.7.2
>            Reporter: Pan Yuxuan
>            Assignee: Brahma Reddy Battula
>         Attachments: HDFS-9902.patch
> Now Hadoop support different storage type for DISK, SSD, ARCHIVE and RAM_DISK, but they
share one configuration dfs.datanode.du.reserved.
> The DISK size may be several TB and the RAM_DISK size may be only several tens of GB.
> The problem is that when I configure DISK and RAM_DISK (tmpfs) in the same DN, and I
set  dfs.datanode.du.reserved values 10GB, this will waste a lot of RAM_DISK size. 
> Since the usage of RAM_DISK can be 100%, so I don't want dfs.datanode.du.reserved configured
for DISK impacts the usage of tmpfs.
> So can we make a new configuration for RAM_DISK or just skip this configuration for RAM_DISK?

This message was sent by Atlassian JIRA

View raw message