hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11551) Handle SlowDiskReport from DataNode at the NameNode
Date Thu, 30 Mar 2017 04:23:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15948407#comment-15948407
] 

Arpit Agarwal commented on HDFS-11551:
--------------------------------------

Thanks for the updated patch [~hanishakoneru]. The v08 patch is looking good. The remaining
comments are minor.
# isUpgradeInProgress - did you mean to call this isUpdateInProgress?
# {{// Remove is not atomic}} - I checked the source code for ConcurrentHashMap and it turns
out the version of remove you are calling is atomic. I was wrong about the locking behavior
earlier.
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/util/concurrent/ConcurrentHashMap.java#ConcurrentHashMap.Segment.remove%28java.lang.Object%2Cint%2Cjava.lang.Object%29

The test coverage looks good too. Two comments:
# The test-only method addSlowDiskForTesting looks unnecessary. Can we just call addSlowDiskReportFromDataNode
from the test case?
# Also the method getSlowDisksReportForTesting can be moved to the test class - and we can
retain a getter for {{slowDisksReport}} in the SlowDiskTracker class.
# The following code will run a tight loop which will spin the CPU. We can add a sleep() or
use GenericTestUtils#waitFor here. Also the call to updateSlowDiskReportAsync can be outside
the loop, it just needs to be called once.
{code}
    while (slowDisksReport.isEmpty()) {
      slowDiskTracker.updateSlowDiskReportAsync(timer.monotonicNow());
      slowDisksReport = slowDiskTracker.getSlowDisksReportForTesting();
    }
{code}

> Handle SlowDiskReport from DataNode at the NameNode
> ---------------------------------------------------
>
>                 Key: HDFS-11551
>                 URL: https://issues.apache.org/jira/browse/HDFS-11551
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs
>            Reporter: Hanisha Koneru
>            Assignee: Hanisha Koneru
>         Attachments: HDFS-11551.001.patch, HDFS-11551.002.patch, HDFS-11551.003.patch,
HDFS-11551.004.patch, HDFS-11551.005.patch, HDFS-11551.006.patch, HDFS-11551.007.patch, HDFS-11551.008.patch
>
>
> DataNodes send slow disk reports via heartbeats. Handle these reports at the NameNode
to find the topN slow disks.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message