hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-3120) Enable hsync and hflush by default
Date Wed, 02 May 2012 04:50:51 GMT

     [ https://issues.apache.org/jira/browse/HDFS-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eli Collins updated HDFS-3120:
------------------------------

    Description: 
The work on branch-20-append was to support *sync*, for durable HBase WALs, not *append*.
The branch-20-append implementation is known to be buggy. There's been confusion about this,
we often answer queries on the list [like this|http://search-hadoop.com/m/wfed01VOIJ5]. Unfortunately,
the way to enable correct sync on branch-1 for HBase is to set dfs.support.append to true
in your config, which has the side effect of enabling append (which we don't want to do).

For v1.x let's:
# Always enable the sync path (currently only enabled if dfs.support.append is set)
# Remove the dfs.support.append configuration option. Let's keep the code paths though in
case we ever fix append on branch-1, in which case we can add the config option back

For 2.x let's
# Always enable the hsync/hflush path
# The dfs.support.appends only enables the append specific paths (since the hsync/hflush paths
are now always on). Append will still default to being enabled so there is no net effect by
default.

  was:
The work on branch-20-append was to support *sync*, for durable HBase WALs, not *append*.
The branch-20-append implementation is known to be buggy. There's been confusion about this,
we often answer queries on the list [like this|http://search-hadoop.com/m/wfed01VOIJ5]. Unfortunately,
the way to enable correct sync on branch-1 for HBase is to set dfs.support.append to true
in your config, which has the side effect of enabling append (which we don't want to do).

Let's add a new *dfs.support.sync* option that enables working sync (which is basically the
current dfs.support.append flag modulo one place where it's not referring to sync). For compatibility,
if dfs.support.append is set, dfs.support.sync will be set as well. This way someone can enable
sync for HBase and still keep the current behavior that if dfs.support.append is not set then
an append operation will result in an IOE indicating append is not supported. We should do
this on trunk as well, as there's no reason to conflate hsync and append with a single config
even if append works.

    
> Enable hsync and hflush by default
> ----------------------------------
>
>                 Key: HDFS-3120
>                 URL: https://issues.apache.org/jira/browse/HDFS-3120
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>             Fix For: 2.0.0
>
>         Attachments: hdfs-3120.txt, hdfs-3120.txt
>
>
> The work on branch-20-append was to support *sync*, for durable HBase WALs, not *append*.
The branch-20-append implementation is known to be buggy. There's been confusion about this,
we often answer queries on the list [like this|http://search-hadoop.com/m/wfed01VOIJ5]. Unfortunately,
the way to enable correct sync on branch-1 for HBase is to set dfs.support.append to true
in your config, which has the side effect of enabling append (which we don't want to do).
> For v1.x let's:
> # Always enable the sync path (currently only enabled if dfs.support.append is set)
> # Remove the dfs.support.append configuration option. Let's keep the code paths though
in case we ever fix append on branch-1, in which case we can add the config option back
> For 2.x let's
> # Always enable the hsync/hflush path
> # The dfs.support.appends only enables the append specific paths (since the hsync/hflush
paths are now always on). Append will still default to being enabled so there is no net effect
by default.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message