Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 30 May 2014 17:31:02 +0000 (UTC)
From: "stack (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12701849.1395049471762.48696.1401471062893@arcas>
In-Reply-To: <JIRA.12701849.1395049471762@arcas>
References: <JIRA.12701849.1395049471762@arcas>
Subject: [jira] [Commented] (HDFS-6109) let sync_file_range() system call
 run in background
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14013975#comment-14013975 ] 

stack commented on HDFS-6109:
-----------------------------

Patch lgtm (I liked the [~cmccabe] suggestion that we keep FsDatasetImpl encapsulated).

> let sync_file_range() system call run in background
> ---------------------------------------------------
>
>                 Key: HDFS-6109
>                 URL: https://issues.apache.org/jira/browse/HDFS-6109
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 3.0.0, 2.3.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
>         Attachments: HDFS-6109-v2.txt, HDFS-6109-v3.txt, HDFS-6109-v4.txt, HDFS-6109.txt
>
>
> Through we passed SYNC_FILE_RANGE_WRITE to sync_file_range, to make it as asynchronous as possible, it still could be blocked, e.g. the os io request queue is full.
> Since we use sync_file_range just as a page cache advisor role:) it doesn't decide or guarantee the real durability, it would be nice if we could run it in  backgroud. At least my test log showed, a few sync_file_range calls still cost tens of ms or more, due to the happened location is in the critical write path(BlockReceiver class), from a upper view, like HBase application, will "hung" tens of ms as well during Hlog syncing.
> Generally speaking, the patch could not improve too much, but, better than before, right ? :)


--
This message was sent by Atlassian JIRA
(v6.2#6252)