Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 27 Dec 2016 23:49:58 +0000 (UTC)
From: "Xiaoyu Yao (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13030261.1482450473000.602857.1482882598475@Atlassian.JIRA>
In-Reply-To: <JIRA.13030261.1482450473000@Atlassian.JIRA>
References: <JIRA.13030261.1482450473000@Atlassian.JIRA> <JIRA.13030261.1482450473250@arcas>
Subject: [jira] [Comment Edited] (HDFS-11274) Datanode should only check the
 failed volume upon IO errors
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 27 Dec 2016 23:50:00 -0000


    [ https://issues.apache.org/jira/browse/HDFS-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781508#comment-15781508 ] 

Xiaoyu Yao edited comment on HDFS-11274 at 12/27/16 11:49 PM:
--------------------------------------------------------------

Delta from patch 02:

1. Add a ScheduleResult interface to get the ListenableFuture and whether a new target check is added to ThrottledAsyncChecker. This way, the async callback will be called once per scheduled check.

2. Avoid cyclic volume check during volume check result handler, more specifically the BlockPoolSlice#saveDfsUsed() during volume shutdown. 
 
3. Fix the unit test as the volume failure counter can be larger due to the async volume check and the fact that min-check-gap is set as 0 during certain test.


was (Author: xyao):
Delta from patch 02:

1. Add a ScheduleResult interface to get the ListenableFuture and whether a new target check is added to ThrottledAsyncChecker.

2. Avoid cyclic volume check during volume check result handler. 
 
3. Fix the unit test.


> Datanode should only check the failed volume upon IO errors 
> ------------------------------------------------------------
>
>                 Key: HDFS-11274
>                 URL: https://issues.apache.org/jira/browse/HDFS-11274
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>            Reporter: Xiaoyu Yao
>            Assignee: Xiaoyu Yao
>         Attachments: HDFS-11274.01.patch, HDFS-11274.02.patch, HDFS-11274.03.patch
>
>
> This is a performance improvement that is possible after HDFS-11182. The goal is to trigger async volume check with throttling only on suspected volume upon datanode file IO errors. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org