hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5954) Allow proper fsync support for HBase
Date Sun, 23 Nov 2014 05:44:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222321#comment-14222321

Lars Hofhansl commented on HBASE-5954:

Well... We do have the APIs already. We do not have a "sync always means hsync" flag, that
would have been easier, but less flexible.
We could remove the API again (as FSYNC_WAL is identical to SYNC_WAL right now anyway, see
the Durability class)... I.e. deprecating FSYNC_WAL.

The idea was that the application knows best. It might do ASYNC_WAL most of the time, then
issue a SYNC_WAL, and occasionally an FSYNC_WAL when it knows that this is necessary.

Thinking about it, it would nice to get rid of this complexity (and I would no longer hate
this patch either :) ). This needs to be a careful decision, though. If we want to avoid the
mixed handling we could not allow this on a per column family or even table level, it would
need be on or off per cluster. In earlier discussions we wanted to avoid this due to the high
cost of an fsync (until we have storage classes in HDFS or SSDs everywhere). I'd be in favor
of having this discussion again.

BTW. My patch already has hsync for META always :)

> Allow proper fsync support for HBase
> ------------------------------------
>                 Key: HBASE-5954
>                 URL: https://issues.apache.org/jira/browse/HBASE-5954
>             Project: HBase
>          Issue Type: Improvement
>          Components: HFile, wal
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>            Priority: Critical
>             Fix For: 2.0.0
>         Attachments: 5954-WIP-trunk.txt, 5954-trunk-hdfs-trunk-v2.txt, 5954-trunk-hdfs-trunk-v3.txt,
5954-trunk-hdfs-trunk-v4.txt, 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt,
5954-trunk-hdfs-trunk.txt, hbase-hdfs-744.txt
> At least get recommendation into 0.96 doc and some numbers running w/ this hdfs feature

This message was sent by Atlassian JIRA

View raw message