hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kumar Sudhanshu <kraan...@gmail.com>
Subject Re: [jira] [Commented] (HDFS-8989) Allow rate-limiting the DirectoryScanner via time-slicing
Date Tue, 01 Sep 2015 23:57:16 GMT
Remove me from this list.

Thanks,
~Kumar

Sent from my iPhone

> On Sep 1, 2015, at 4:45 PM, Andrew Wang (JIRA) <jira@apache.org> wrote:
> 
> 
>    [ https://issues.apache.org/jira/browse/HDFS-8989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14726434#comment-14726434
] 
> 
> Andrew Wang commented on HDFS-8989:
> -----------------------------------
> 
> This looks like a dupe of HDFS-8873 which has more watchers; want to take it there?
> 
>> Allow rate-limiting the DirectoryScanner via time-slicing
>> ---------------------------------------------------------
>> 
>>                Key: HDFS-8989
>>                URL: https://issues.apache.org/jira/browse/HDFS-8989
>>            Project: Hadoop HDFS
>>         Issue Type: Improvement
>>         Components: datanode
>>   Affects Versions: 2.0.6-alpha
>>           Reporter: Colin Patrick McCabe
>>           Assignee: Daniel Templeton
>> 
>> When the inodes and directory entries for HDFS block and meta files are not in the
cache, the DirectoryScanner can create enormous disk load via generating many seeks.  This
can effectively make the disks unusable for other purposes.  We should allow rate-limiting
the DirectoryScanner via time-slicing, so that it only runs every 100 ms out of 1 s (or whatever
other percentage we configure.)
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)

Mime
View raw message