hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8230) Enable sync by default and disable append
Date Tue, 08 May 2012 00:51:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270111#comment-13270111
] 

Eli Collins commented on HADOOP-8230:
-------------------------------------

This change was proposed and discussed over a month ago:
https://issues.apache.org/jira/browse/HDFS-3120?focusedCommentId=13241903&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13241903

Since you're worried that the sync path being buggy I think the best remedy is testing. The
sync path in 1.0.x has been tested extensively by HBase users (including MR jobs), but we
need to spend time looking at any bugs/changed behavior that might crop up as part of testing
the Hadoop 1.x release anyway.

This change prevents *known* data loss on out of the box Hadoop 1.x installs, that seems more
important than issues that could potentially come up during testing. I think our HBase users
feel similarly. The reason I filed a separate jira for adding the flag is that I don't think
that we should let users enable a code path that we know can result in data loss, and means
we have to test two code paths. I also see your POV which is why I'm not -1 on this flag even
though I don't like it.
                
> Enable sync by default and disable append
> -----------------------------------------
>
>                 Key: HADOOP-8230
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8230
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 1.0.0
>            Reporter: Eli Collins
>            Assignee: Eli Collins
>             Fix For: 1.1.0
>
>         Attachments: hadoop-8230.txt
>
>
> Per HDFS-3120 for 1.x let's:
> - Always enable the sync path, which is currently only enabled if dfs.support.append
is set
> - Remove the dfs.support.append configuration option. We'll keep the code paths though
in case we ever fix append on branch-1, in which case we can add the config option back

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message