hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliott Clark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9087) Add some jitter to DataNode.checkDiskErrorThread
Date Wed, 16 Sep 2015 17:05:45 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14790740#comment-14790740
] 

Elliott Clark commented on HDFS-9087:
-------------------------------------

Yeah HDFS-8845 makes things better however it still is bad to have anything in a distributed
system that can get multiple machines in sync. Lots of things happen when a disk is checked
and then listed as bad. It's good to have a large cluster spread out so that nothing lines
up.

> Add some jitter to DataNode.checkDiskErrorThread
> ------------------------------------------------
>
>                 Key: HDFS-9087
>                 URL: https://issues.apache.org/jira/browse/HDFS-9087
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>    Affects Versions: 2.6.0
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>         Attachments: HDFS-9087-v0.patch, HDFS-9087-v1.patch
>
>
> If all datanodes are started across a cluster at the same time (or errors in the network
cause ioexceptions) there can be storms where lots of datanodes check their disks at the exact
same time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message