hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9176) TestDirectoryScanner#testThrottling often fails.
Date Tue, 06 Oct 2015 18:00:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945468#comment-14945468
] 

Daniel Templeton commented on HDFS-9176:
----------------------------------------

Once the block scanner has run, there's no point in waiting any more.  The threads that shutdown()
would stop are already stopped.

I have the throttle set to 1ms, which means that only one directory will be scanned per second.
 The idea is to slow the process down as much as possible.  I'm guessing that there are sometimes
fewer directories than 2x the number of threads, resulting in the scan completing early. 
There's no way to slow it down any more than that.

I suppose one additional way to increase the likelihood of having the shutdown() thread run
would be to reduce the scheduling delay to 1s or 500ms.  Still no guarantee, but it would
increase the odds.

> TestDirectoryScanner#testThrottling often fails.
> ------------------------------------------------
>
>                 Key: HDFS-9176
>                 URL: https://issues.apache.org/jira/browse/HDFS-9176
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 2.8.0
>            Reporter: Yi Liu
>            Assignee: Daniel Templeton
>            Priority: Minor
>         Attachments: HDFS-9176.001.patch
>
>
> https://builds.apache.org/job/PreCommit-HDFS-Build/12736/testReport/
> https://builds.apache.org/job/PreCommit-HADOOP-Build/7732/testReport/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message