hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "haosdent (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5954) Allow proper fsync support for HBase
Date Fri, 30 Aug 2013 12:35:56 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13754646#comment-13754646
] 

haosdent commented on HBASE-5954:
---------------------------------

[~xieliang007]If mount disk with "data=writeback", the dirty data may be in disk cache after
fsync system call return. Until the data more than a ratio in disk cache or timer is trigger,
them will flush to physics storage. We could improve the performance of hsync by disable journal
and write cache. But after disable write cache, the whole write performance is worse than
before. Fsync is a very heavy system call, I think it is unfeasible to call fsync after every
write operation. Just post my test result about fsync roughly below:

1.ext4,noatime,barrier=1,data=ordered, enable disk write cache, enable journal, append 4k
to a file
fdatasync 25ms
fsync 25ms
2.ext4,noatime,barrier=0,data=writeback, disable disk write cache, enable journal, append
4k to a file
fdatasync 33ms
fsync 33ms
3.ext4,noatime,barrier=0,data=writeback, disable disk write cache, disable journal, append
4k to a file
fdatasync 8ms
fsync 8ms
                
> Allow proper fsync support for HBase
> ------------------------------------
>
>                 Key: HBASE-5954
>                 URL: https://issues.apache.org/jira/browse/HBASE-5954
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>            Priority: Critical
>             Fix For: 0.98.0
>
>         Attachments: 5954-trunk-hdfs-trunk.txt, 5954-trunk-hdfs-trunk-v2.txt, 5954-trunk-hdfs-trunk-v3.txt,
5954-trunk-hdfs-trunk-v4.txt, 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt,
hbase-hdfs-744.txt
>
>
> At least get recommendation into 0.96 doc and some numbers running w/ this hdfs feature
enabled.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message