hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaoyu Yao (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11274) Datanode should only check the failed volume upon IO errors
Date Tue, 27 Dec 2016 23:49:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781508#comment-15781508
] 

Xiaoyu Yao edited comment on HDFS-11274 at 12/27/16 11:49 PM:
--------------------------------------------------------------

Delta from patch 02:

1. Add a ScheduleResult interface to get the ListenableFuture and whether a new target check
is added to ThrottledAsyncChecker. This way, the async callback will be called once per scheduled
check.

2. Avoid cyclic volume check during volume check result handler, more specifically the BlockPoolSlice#saveDfsUsed()
during volume shutdown. 
 
3. Fix the unit test as the volume failure counter can be larger due to the async volume check
and the fact that min-check-gap is set as 0 during certain test.




was (Author: xyao):
Delta from patch 02:

1. Add a ScheduleResult interface to get the ListenableFuture and whether a new target check
is added to ThrottledAsyncChecker.

2. Avoid cyclic volume check during volume check result handler. 
 
3. Fix the unit test.



> Datanode should only check the failed volume upon IO errors 
> ------------------------------------------------------------
>
>                 Key: HDFS-11274
>                 URL: https://issues.apache.org/jira/browse/HDFS-11274
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Xiaoyu Yao
>            Assignee: Xiaoyu Yao
>         Attachments: HDFS-11274.01.patch, HDFS-11274.02.patch, HDFS-11274.03.patch
>
>
> This is a performance improvement that is possible after HDFS-11182. The goal is to trigger
async volume check with throttling only on suspected volume upon datanode file IO errors.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message